Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

check memory and cpu usage

avatar
New Contributor

there are like 1k finished jobs and i want to check allocated CPU and memory of those jobs. how to do that? i tried in command line but it shows when i enter job id and in my yarn UI CPU and memory metrics are not there.Screenshot from 2022-05-05 14-19-25.png

1 REPLY 1

avatar
Master Collaborator
To get the number of vcores and memory used for a particular queue in the cluster 

You can get this information in the cluster utilization report, please go through the below doc for more information:
https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_cluster_util_report.html

Also, if you want to see the resources used for each application, please go through the below KB article:
https://my.cloudera.com/knowledge/How-to-calculate-memory-and-v-core-utilization-for-each-of-the?id=...
another option is
Using yarn application -status command, you can get the Aggregate Resource Allocation for an application.
For e.g.  yarn application -status application_ID (for one of completed applications), one of the rows returned is:
Aggregate Resource Allocation : 46641 MB-seconds, 37 vcore-seconds
This gives an aggregate memory and CPU allocations in seconds.