Empty Impala queries in Cloudera Manager

[#1]

Hi,

 I have a brand new installation of CDH 5.15, where all services (MNG,Impala) are in green. I executed ~10 queries on Impala, and was checking the queries in Cloudera Manager -> Impala Queries. I noticed two issues:

- Some of the queries were in the list when there were in "running" state

- After the statements were finished, no query were reported in Impala Queries.

The obvious reason could be time filter: I checked 30m, 1h, 2h, 1d, still NO results. 

Also another obvious reason can be a search filter. The filter is empty, NO results.

I checked the Service Monitor of CM, it is in green, so I suppose it collects data.

I checked the Impala storage (firehose_impala_storage_bytes) it is 1GB.

The only "warning" is about the memory of Service Monitor what is much less, but this is a new cluster, no workload running, and the CM reports that the heap usage is under 1G

The recommended non-Java memory size is 12.0 GiB, 10.5 GiB more than is configure

What could be the problem of the empty list? Why Cloudera Manager is not collecting the Impala queries? Or maybe it is, but then why it is not reporting, showing me them?

Thanks 

[#2]

Hi @TomasTF,

One question:

   - How do you executhe that queries? With wich client?

Have you try to ctrl+R in your cloudera manager (tab impala --> queries) in your browser.

Regards,

Manu.

[#3]

Queries via jdbc client.
Yes tried refresh, but after ~ 30 min all the queries appeared.

[#4]

Hi, @TomasTF

Have your system the same timezone that your CDH?

Regards,

Manu.

[#5 · Accepted solution]

@TomasTF,

(1)

Service Monitor will pull query information from the Impala Daemons so first make sure that Service Monitor is not having trouble getting information after the queries complete. View:

/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-`hostname -f`.log.out

(2)

When you query Impala queries via Cloudera Manager, Cloudera Manager will request query information from Service Monitor.  So, check Cloudera Manager and Service Monitor logs for any log messages that may be relevant when you are trying to view queries:

/var/log/cloudera-scm-firehose/mgmt-cmf-mgmt-SERVICEMONITOR-`hostname -f`.log.out

/var/log/cloudera-scm-server/cloudera-scm-server.log

(3)

Make sure you have enough disk space on the volume where your Service Monitor Storage Directory is located.

(4)

Lastly, if there are no messages that indicate problems with the above, try increasing Impala Storage from 1GiB to 2 or 3 GiB.  Perhaps for some reason you were running out of space, but I don't think that is too likely.  Worth a try if nothing else mentioned above gives you any clues.