Getting impala daemon serves via cloudera rest API

[#1]

Hello,

I'm trying to get some key impalad daemon metrics (mem consumption, CPU cycles etc.) from CM REST api but couldn't find it. I'm following this list but only see Impala query level metrics:

https://cloudera.github.io/cm_api/apidocs/v15/

In the impalad daemon UI, I see lots of such metrics in the /memz and /metrics tab. Are those (or some of them) available from CM REST api as well?

Thanks,

S.

[#2 · Accepted solution]

If the metrics are available in CM (e.g. if they're in the Impala Charts Library in CM, or you can query them in CM) then you should be able to get them through the CM timeseries API https://cloudera.github.io/cm_api/apidocs/v15/path__timeseries.html

CM is capable of collecting all of Impala's metrics from the /metrics page, but it currently only collects a subset of them (the ones we think are most important for cluster monitoring).

I'd be interested in knowing if there are useful metrics that are not currently collected by CM, and particularly in understanding the kind of high-level questions that you're trying to answer with metrics, so I can understand if there are metrics that could be added to Impala.

[#3]

Thanks Tim! That's exactly what I want 😉

I believe the metrics listed on this page is what's available from the rest api (I checked a few of them and they are retrievable from the api): https://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_metrics_impala_daemon.html, it seems to cover all I need as of now.

Thanks,

S.

[#4]

Great to hear! CM's time series support is really pretty powerful.

[#5]

Hi @Tim Armstrong Hope you are doing well,

It will be nice if we have a metirc for the memory part of the daemons_memory_limit used by impala daemon in a given time.

So when i get a query failing on memory, i can investigate the memory usage thatwill help me to understand when to increase the limit, secondly, i can learn trend and usage over time and i can plan my increase.

Currently i see only the resident memory per node but this memory isn't used by the queries, so it's a diffcult task for me to investigate the impala behaviour once a query failed on memory.

Yes i have a metric of the total memory used by node, but i have different roles in the node, so it hard to track this issue.