- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Warning message in Hive output after upgrading to hive version 1.1.0 from 0.13.0.
- Labels:
-
Apache Hive
Created ‎05-19-2016 12:51 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Our cluster was recently upgraded to CDH 5.5 and all the outputs generated by hive query has now a warning message shown below
WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked. WARN: Please see http://www.slf4j.org/codes.html#release for an explanation.
But when I use Beeline I don't see them. Is there a way I can get rid of the warning without switching to Beeline at least for now?
Hive version is 1.1.0
Created ‎05-20-2016 09:15 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
To Conclude, it is not possible to remove the warning message from the Hive query output until migrating to Beeline. I found a very simple and well known workaround to this problem which almost all of us use it in shell scripting. I used the grep tool to remove the warning message from my output. Below is how I modified my script to fix the problem.
hive -S -d ns=$hiveDB -d tab=$t -d dunsCol=$c1 -d phase="$ph1" -d error=$c2 -d ts=$eColumnArray -d reporting_window=$rDate -f $dir'select_count.hsql' | grep -v "^WARN" > $gOutPut 2> /dev/null
Created ‎05-19-2016 01:30 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This is another bug fixed in hive 1.3 and later.
Created ‎05-19-2016 02:22 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am adding the below jar but don't get any warnings. So the above Jira you mentioned is only for the Spark jars.
add jar hdfs:///data/95-utilities/hive-contrib-1.1.0-cdh5.5.2.jar;
Created ‎05-19-2016 02:27 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
yes, that was a general Jira related to spark therefore same applicable to other jar also which includes in hive classpath. Also I noticed you are using cdh? if your distro is CDH then sorry I have no idea about their distro jars.
Created ‎05-20-2016 09:15 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
To Conclude, it is not possible to remove the warning message from the Hive query output until migrating to Beeline. I found a very simple and well known workaround to this problem which almost all of us use it in shell scripting. I used the grep tool to remove the warning message from my output. Below is how I modified my script to fix the problem.
hive -S -d ns=$hiveDB -d tab=$t -d dunsCol=$c1 -d phase="$ph1" -d error=$c2 -d ts=$eColumnArray -d reporting_window=$rDate -f $dir'select_count.hsql' | grep -v "^WARN" > $gOutPut 2> /dev/null
Created ‎01-08-2020 04:27 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You can also append HIVE_SKIP_SPARK_ASSEMBLY to the command which should remove the warnings.
export HIVE_SKIP_SPARK_ASSEMBLY=true; hive -S --database dbname -e 'show tables;'
