Created 05-19-2016 12:51 PM
Our cluster was recently upgraded to CDH 5.5 and all the outputs generated by hive query has now a warning message shown below
WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked. WARN: Please see http://www.slf4j.org/codes.html#release for an explanation.
But when I use Beeline I don't see them. Is there a way I can get rid of the warning without switching to Beeline at least for now?
Hive version is 1.1.0
Created 05-20-2016 09:15 AM
To Conclude, it is not possible to remove the warning message from the Hive query output until migrating to Beeline. I found a very simple and well known workaround to this problem which almost all of us use it in shell scripting. I used the grep tool to remove the warning message from my output. Below is how I modified my script to fix the problem.
hive -S -d ns=$hiveDB -d tab=$t -d dunsCol=$c1 -d phase="$ph1" -d error=$c2 -d ts=$eColumnArray -d reporting_window=$rDate -f $dir'select_count.hsql' | grep -v "^WARN" > $gOutPut 2> /dev/null
Created 05-19-2016 01:30 PM
This is another bug fixed in hive 1.3 and later.
Created 05-19-2016 02:22 PM
I am adding the below jar but don't get any warnings. So the above Jira you mentioned is only for the Spark jars.
add jar hdfs:///data/95-utilities/hive-contrib-1.1.0-cdh5.5.2.jar;
Created 05-19-2016 02:27 PM
yes, that was a general Jira related to spark therefore same applicable to other jar also which includes in hive classpath. Also I noticed you are using cdh? if your distro is CDH then sorry I have no idea about their distro jars.
Created 05-20-2016 09:15 AM
To Conclude, it is not possible to remove the warning message from the Hive query output until migrating to Beeline. I found a very simple and well known workaround to this problem which almost all of us use it in shell scripting. I used the grep tool to remove the warning message from my output. Below is how I modified my script to fix the problem.
hive -S -d ns=$hiveDB -d tab=$t -d dunsCol=$c1 -d phase="$ph1" -d error=$c2 -d ts=$eColumnArray -d reporting_window=$rDate -f $dir'select_count.hsql' | grep -v "^WARN" > $gOutPut 2> /dev/null
Created 01-08-2020 04:27 AM
You can also append HIVE_SKIP_SPARK_ASSEMBLY to the command which should remove the warnings.
export HIVE_SKIP_SPARK_ASSEMBLY=true; hive -S --database dbname -e 'show tables;'