Support Questions

Find answers, ask questions, and share your expertise

How to monitor resources of impala queue

avatar
Explorer

Hi guys:

        After configured admission control of impala, I met some difficulties.

        I tried to monitor the resource usage of the impala queue, but I can't find how to monitor it. Is there any way to monitor the queue of impala like yarn?

        After all , impala use fair-scheduler.xml to configure admission control .

1 ACCEPTED SOLUTION

avatar
Cloudera Employee
You can see the number of queued/admitted queries and mem used per pool by inspecting the logs, eg. impalad.INFO. Look for lines that come from admission-controller.cc

By default each time a query is admitted we log these stats for the query's pool.

If the default log level doesn't provide enough info, there's additional info that gets logged at higher levels, though of course keep in mind that more logging may slow queries down.

View solution in original post

4 REPLIES 4

avatar
Cloudera Employee
Impala has a webui, by default run on port 25000 of each impalad node, that you may find useful.

In particular, the /queries page will display a list of running or recent queries along with the resource pool they were assigned to

avatar
Explorer

Thanks,

    I find that page , but these queries do not show the queue they are on and how much resources are used. I also want to monitor the use of queues, such as how much resources have been used and how much resources are left. Do you have any good way to do this?

avatar
Cloudera Employee
You can see the number of queued/admitted queries and mem used per pool by inspecting the logs, eg. impalad.INFO. Look for lines that come from admission-controller.cc

By default each time a query is admitted we log these stats for the query's pool.

If the default log level doesn't provide enough info, there's additional info that gets logged at higher levels, though of course keep in mind that more logging may slow queries down.

avatar
Explorer
Thanks ! And I find the /metrics page can also show these information which in logs.