What is the meaning of "Cluster CPU" visualization displayed in YARN Summary ?

[#1]

The "Cluster CPU" visualization displayed in YARN Summary shows an average utilization of 50% always even when the data nodes are idle. The "CPU Usage" of individual data nodes are close to zero when no jobs are running as seen on "Summary" tab for the host in Ambari. Then why does the "Cluster CPU" graph always show 50% as the avg ? What exactly it is trying to depict ?

The definition says "Percentage of CPU utilized across all NodeManager hosts.". So, when it says 50% avg, does that mean (for example) 4 CPUs out of 8 total CPUs of all NodeManager hosts are being used ?

[#2]

Hi @Ekantheshwara Basappa

I believe Cluster CPU gives the Percentage of vCores allocated across all NodeManager hosts.

[#3]

This has me a little confused as well. There are three counters for this:

- (1) Ambari Dashboard has "CPU Usage" which always seems to look low. (makes sense for our env)

- (2) YARN has "CPU Utilization" for "% of total cores assigned to containers" which never goes under 40% (makes sense for our env)

- (3) YARN has "Cluster CPU" for "% of CPU utilization across node manager hosts" which is always between 50%-95% (This makes no sense in our env)

So, when our jobs run, 1 doesn't climb much, 2 doesn't move much, 3 quickly hits 95%. We want to run our jobs faster but between 1 & 3 I can't tell if we are high utilization of CPU or not.

Host page CPU seems to trend with 1 (quiet)

But am I hitting a ceiling? 3 can't go any higher