Support Questions

Find answers, ask questions, and share your expertise

check memory and cpu usage

avatar
New Contributor

there are like 1k finished jobs and i want to check allocated CPU and memory of those jobs. how to do that? i tried in command line but it shows when i enter job id and in my yarn UI CPU and memory metrics are not there.Screenshot from 2022-05-05 14-19-25.png

1 REPLY 1

avatar
Master Collaborator
To get the number of vcores and memory used for a particular queue in the cluster 

You can get this information in the cluster utilization report, please go through the below doc for more information:
https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_cluster_util_report.html

Also, if you want to see the resources used for each application, please go through the below KB article:
https://my.cloudera.com/knowledge/How-to-calculate-memory-and-v-core-utilization-for-each-of-the?id=...
another option is
Using yarn application -status command, you can get the Aggregate Resource Allocation for an application.
For e.g.  yarn application -status application_ID (for one of completed applications), one of the rows returned is:
Aggregate Resource Allocation : 46641 MB-seconds, 37 vcore-seconds
This gives an aggregate memory and CPU allocations in seconds.