Hiverserver2 logs set with INFO mode but still it's generating logs in DEBUG mode

[#1]

I have configured Hiverserver2 in HA mode and started in INFO mode and it is generating logs in INFO mode on server1 but on server2 it's generating in DEBUG mode. I also checked in configuration there is no different configuration for both hiveservers.

Below are running hiveserver2 processes on server1 & server2.

Server1:~> ps -ef | grep hiveserver2

hive 6717 1 2 Jul02 ? 02:02:18 /usr/lib/jvm/jdk1.8.0_71/bin/java -Xmx1024m -Dhdp.version=2.5.3.0-37 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.5.3.0-37 -Dhadoop.log.dir=/hadoop/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.5.3.0-37/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.5.3.0-37/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -Xmx12065m -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=/usr/hdp/2.5.3.0-37/hive/bin/../conf/parquet-logging.properties -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.5.3.0-37/hive/lib/hive-service-1.2.1000.2.5.3.0-37.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.aux.jars.path=file:///usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar -hiveconf hive.metastore.uris= -hiveconf hive.log.file=hiveserver2.log -hiveconf hive.log.dir=/hadoop/var/log/hive

=======================================

Server2:~> ps -ef | grep hiveserver2

hive 4459 1 3 Jul02 ? 03:30:22 /usr/lib/jvm/jdk1.8.0_71/bin/java -Xmx1024m -Dhdp.version=2.5.3.0-37 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.5.3.0-37 -Dhadoop.log.dir=/hadoop/var/log/hadoop/hive -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/hdp/2.5.3.0-37/hadoop -Dhadoop.id.str=hive -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/hdp/current/hadoop-client/lib/native/Linux-amd64-64:/usr/hdp/2.5.3.0-37/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx1024m -Xmx12065m -Dlog4j.configurationFile=hive-log4j2.properties -Djava.util.logging.config.file=/usr/hdp/2.5.3.0-37/hive/bin/../conf/parquet-logging.properties -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /usr/hdp/2.5.3.0-37/hive/lib/hive-service-1.2.1000.2.5.3.0-37.jar org.apache.hive.service.server.HiveServer2 --hiveconf hive.aux.jars.path=file:///usr/hdp/current/hive-webhcat/share/hcatalog/hive-hcatalog-core.jar -hiveconf hive.metastore.uris= -hiveconf hive.log.file=hiveserver2.log -hiveconf hive.log.dir=/hadoop/var/log/hive

Please respond.

[#2]

@arjun more is the hive-log4j2.properties in both servers the same?

Also by any chance is Atlas/Hbase service also installed on server2 - or what other services are collocated in server2 with hive?

[#3]

I am having only hiveserver2 service on Server2.

[#4]

@arjun more check if configuration under /etc/hive2/conf is exact same. Also perhaps an environment variable on server2 is set at hive user level? Have you tried starting from command line to check if you still have same debug level on?

[#5]

I'm not sure if it matters; but it seems there is a 1.5 hour difference between the 2 ps -ef -s ; it might be possible that somehow one of the hs2 instances have not shut down cleanly earlier;

I would recommend to try shutting both hs2 down; and checking that they are both stopped by hand ; check that the configs are updated - and start both of them...