question Re: Hive servers in same cluster use different hive-log4j.properties files. in Support Questions

Hive servers in same cluster use different hive-log4j.properties files.

mungeol_heo — Fri, 28 Jul 2017 13:10:36 GMT

Hello,

Here are logs from two hiveserver2.log file.

------- first which is strange -------

2017-07-28 14:46:10,051 DEBUG [main]: server.HiveServer2 (HiveServer2.java:main(586)) -

Logging initialized using configuration in jar:file:/usr/hdp/2.6.0.3-8/hive/lib/hive-common-1.2.1000.2.6.0.3-8.jar!/hive-log4j.properties

------- second -------

2017-07-28 14:43:01,534 DEBUG [main]: server.HiveServer2 (HiveServer2.java:main(586)) - Logging initialized using configuration in file:/etc/hive/2.6.0.3-8/0/conf.server/hive-log4j.properties

What is causing this problem?

Any help will be very thankful.

Thank you.

Re: Hive servers in same cluster use different hive-log4j.properties files.

jsensharma — Fri, 28 Jul 2017 13:42:33 GMT

@mungeol heo

By any change have you configured any "config group" and assigned these different hosts to it?

Also the seconds logging config looks good.

On the first hiveserver2 instance do you have the following path existing with proper permission?

# ls -l /etc/hive/2.6.0.3-8/0/conf.server/hive-log4j.properties
-rw------- 1 hive hadoop 3091 May 15 13:33 /etc/hive/2.6.0.3-8/0/conf.server/hive-log4j.properties

Also do you have any Environment Global Variable set related to hive conf dir on these hosts? As in the first host the setting of log4j seems to be loading from insdie the jar , so suspecting that some explicitly set classpath setting might be done in some of the script/global environment level so that the JAR may be coming first in the CLASSPATH before the "conf" directory.

Re: Hive servers in same cluster use different hive-log4j.properties files.

mungeol_heo — Fri, 28 Jul 2017 14:01:29 GMT

Hello, @Jay SenSharma

There is no such file: /etc/hive/2.6.0.3-8/0/conf.server/hive-log4j.properties

It will be OK after I copy the hive-log4j.properties file to the /etc/hive/2.6.0.3-8/0/conf.server/ folder.

Now, I am trying to find out the reason why the hive-log4j.properties file is not exists while another node has it.

BTW, I am using default config group.

Thank you.

Re: Hive servers in same cluster use different hive-log4j.properties files.

jsensharma — Fri, 28 Jul 2017 14:05:33 GMT

@mungeol heo

Yes, you can copy and paste the missing hive-log4j.properties file to the first host and then as a preference it will pick that file from "conf" directory. If that file is not found in the conf directory then the log4j properties file will be picked up from any JAR present in the classpath which has the log4j property.

Re: Hive servers in same cluster use different hive-log4j.properties files.

mungeol_heo — Fri, 28 Jul 2017 14:15:58 GMT

@Jay SenSharma

The problem is any config change from ambari will not affect the hive-log4j.properties file in the first host.

Although, it does make sense, since ambari didn't create the file at the first place.

Re: Hive servers in same cluster use different hive-log4j.properties files.

cstanca — Tue, 01 Aug 2017 08:31:56 GMT

@mungeol heo

Let's clarify a little bit. Do you run two HiveServer2 instances to manage different workloads? If so, it is not a surprise to have different settings for log4j.

Re: Hive servers in same cluster use different hive-log4j.properties files.

mungeol_heo — Tue, 01 Aug 2017 13:23:47 GMT

Nope. thank you.

Re: Hive servers in same cluster use different hive-log4j.properties files.

mungeol_heo — Tue, 01 Aug 2017 13:28:42 GMT

I solved the problem by adding a new hive metastore service to the same node which has no hive-log4j.properties file in /etc/hive/2.6.0.3-8/0/conf.server folder.