Skip to main content
SaF007
SaF007
Bronze ⭐️⭐️⭐️
May 6, 2022
Question

check memory and cpu usage

there are like 1k finished jobs and i want to check allocated CPU and memory of those jobs. how to do that? i tried in command line but it shows when i enter job id and in my yarn UI CPU and memory metrics are not there.Screenshot from 2022-05-05 14-19-25.png

1 reply

shehbazk
Gold ⭐️⭐️
shehbazk
Gold ⭐️⭐️
May 11, 2022

To get the number of vcores and memory used for a particular queue in the cluster

You can get this information in the cluster utilization report, please go through the below doc for more information:
https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_cluster_util_report.html

Also, if you want to see the resources used for each application, please go through the below KB article:
https://my.cloudera.com/knowledge/How-to-calculate-memory-and-v-core-utilization-for-each-of-the?id=...

another option is
Using yarn application -status command, you can get the Aggregate Resource Allocation for an application.
For e.g.  yarn application -status application_ID (for one of completed applications), one of the rows returned is:

Aggregate Resource Allocation : 46641 MB-seconds, 37 vcore-seconds

This gives an aggregate memory and CPU allocations in seconds.
https://my.cloudera.com/knowledge/How-to-calculate-memory-and-v-core-utilization-for-each-of-the?id=...
Translate
Terms & ConditionsAccessibility statement