Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

What is the recommended way to find the average time per day for which YARN Memory utilization is greater than X%

Highlighted

What is the recommended way to find the average time per day for which YARN Memory utilization is greater than X%

Contributor

Hi All,

This is a more capacity planning question. Over the last few months i have observed that YARN RAM utilization of our cluster has gone up considerably , due to new process being on boarded onto the cluster. At this point i am trying to find out the length of time (cumulative) through the day for which RAM usage is peaking.

Any information on how to go about this will be helpfull.

Currently:

  1. Hadoop2.7.1.2.4.0.0-169
  2. Ambari - 2.2.1
  3. HDP 2.4
  4. Master - 2 (63 GB RAM) ,DN -6 (63 GB RAM)
  5. RAM 320 GB Total allocation.
1 REPLY 1

Re: What is the recommended way to find the average time per day for which YARN Memory utilization is greater than X%

New Contributor

@Suhel , You can try to get report from Ambari Metrics. or another way is to schedule a job to collect memory data on every machine after every 60 seconds. You can use "memfree" or "sar" command to collect memory usage with timestamp and keep appending output to a file and generate report top on that in excel or any other reporting tool to find out memory usage per node.

Regards,

Fahim

Don't have an account?
Coming from Hortonworks? Activate your account here