Reply
Highlighted
New Contributor
Posts: 9
Registered: ‎09-02-2016
Accepted Solution

HiveMetaStore and HiveServer2 didn't log the pool thread

Hi

 

I have 2 clusters both are in CDH 5.4.7

 

First cluster when I see the log file

In HiveMetastore log, I see [pool-3-thread-xxx] every line in the log and

In HiveServer2 log, I also see [HiveServer2-Handler-Pool: Thread-xxxxxx] every line in the log

 

But in the second cluster

In HiveMetastore log, It didn't print [pool-3-thread-xxx] and

In HiveServer2 log, It didn't print [HiveServer2-Handler-Pool: Thread-xxxxxx]

 

So I want to see the pool thread in the log, I don't know why the first cluster show pool thread and second cluster didn't show 

 

please suggest, Any kind of help is appreciated.

 

Thanks,

Teng

 

 

 

Cloudera Employee
Posts: 815
Registered: ‎03-23-2015

Re: HiveMetaStore and HiveServer2 didn't log the pool thread

Hi Teng,

 

This sounds like that you are using different thrift service class, can you please copy and paste some sample HS2 and HMS logs that do not have thread information here? I would like to confirm which class you are using.

 

Thanks

New Contributor
Posts: 9
Registered: ‎09-02-2016

Re: HiveMetaStore and HiveServer2 didn't log the pool thread

I figure out that it uses different log4j configuration

The first one there is [%t] in conversionPattern

Cloudera Employee
Posts: 815
Registered: ‎03-23-2015

Re: HiveMetaStore and HiveServer2 didn't log the pool thread

Hi Teng,

 

I am glad that you have figured out the cause.

Cloudera Employee
Posts: 34
Registered: ‎08-16-2016

Re: HiveMetaStore and HiveServer2 didn't log the pool thread

Yes, HMS and HS2 have separate log4j.properties files.

However, I find it odd that one cluster has the threadname in the log records while the other does not (both CDH5.4.7). The defaults should be exactly the same on both the clusters.

 

Perhaps, you could confirm that you have not overridden the configuration in a safety valve.

 

To add a threadname to the log records for the HMS,

Select Hive Service --> Instances --> Hive Metastore Server --> Configuration
then type in log4j and look for
"Hive Metastore Server Logging Advanced Configuration Snippet (Safety Valve)" This is where I would set it to something like this ..
log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p [%t]: %c{2} (%F:%M(%L)) - %m%n
or just
%d{ISO8601} %p [%t]: %c: %m%n

 

the %t is what adds the thread name to the log records.

 

Hope this helps. Thanks