CM does not show Impala all queries. Service monitor possibly refuses queries.

[#1]

Hi,

The Cloudera Manager Impala service "Queries" tab does not show all queries. Furthermore I see a lot WARN logs in service monitor logs. Like this one:

8:13:07.890 AM

WARN

ImpalaQueryManager

Impala query: 314234f1b5c26d41:c0205b9200000000 with endTime 2018-04-03T09:13:06.973Z is outside acceptance window.

I could not find anything about what could cause this warning message. Time is configured correctly with the same timezone on all servers. 

I am using CDH 5.14 managed with CM and Kerberos authentication.

Thx for any help.

[#2]

One interesting thing I found out is that the timestamp of service monitor logs is different from the service monitor OS time. After running this command 

tail -n 1 mgmt-cmf-mgmt-SERVICEMONITOR-ba-servicemonitor-v.jumbo.hq.eset.com.log.out; date

I got:

2018-04-04 07:10:51,650 DEBUG org.mortbay.log: RESPONSE /  200 <--- Service monitor log

Wed Apr  4 08:10:51 CEST 2018   <-- date command output

Can someone give me a hint where servicemonitor looks for timezone settings?

Thx.

[#3 · Accepted solution]

After further investigation, I found out that the timezone "seen" by Java was different than in OS. I added this startup parameter to the service monitor configuration which configures the timezone

-Duser.timezone=Europe/Bratislava

After that, the logs showed the correct event timestamp and the service monitor started to accept query information from agents. Now I am able to see the queries in CM impala/queries tab.