Options
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
check memory and cpu usage
Labels:
- Labels:
-
Apache Hadoop
New Contributor
Created ‎05-06-2022 04:47 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
there are like 1k finished jobs and i want to check allocated CPU and memory of those jobs. how to do that? i tried in command line but it shows when i enter job id and in my yarn UI CPU and memory metrics are not there.
1 REPLY 1
Master Collaborator
Created ‎05-11-2022 01:21 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
To get the number of vcores and memory used for a particular queue in the cluster You can get this information in the cluster utilization report, please go through the below doc for more information: https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/admin_cluster_util_report.html Also, if you want to see the resources used for each application, please go through the below KB article: https://my.cloudera.com/knowledge/How-to-calculate-memory-and-v-core-utilization-for-each-of-the?id=...
another option is
Using yarn application -status command, you can get the Aggregate Resource Allocation for an application.
For e.g. yarn application -status application_ID (for one of completed applications), one of the rows returned is:
Using yarn application -status command, you can get the Aggregate Resource Allocation for an application.
For e.g. yarn application -status application_ID (for one of completed applications), one of the rows returned is:
Aggregate Resource Allocation : 46641 MB-seconds, 37 vcore-seconds
This gives an aggregate memory and CPU allocations in seconds.
