question from where we get the Yarn memory value from the dashboard in Archives of Support Questions (Read Only)

from where we get the Yarn memory value from the dashboard

mike_bronson7 — Sun, 18 Aug 2019 01:04:22 GMT

we have Ambari cluster with 5 workers machines ( each worker have 32G memory )

in the dashboard we see the yarn memory like this:

77% used 575G of 746G

seems that the running application take 575G

but how 575G if total workers have 32X5=160G

and how the calculte is 746G ? on total ?

Re: from where we get the Yarn memory value from the dashboard

asirna — Sun, 18 Aug 2019 01:04:14 GMT

@uri ben-ari,

746.5Gb is the total memory of all node managers included.

It is calculated as (yarn.nodemanager.resource.memory-mb) * (no of node managers) . Please check the value for yarn.nodemanager.resource.memory-mb in your YARN configs. Check the screenshot

575 GB is the memory used by all the applications. You can check the Resource manager UI to see which apps are using memory.

In the screenshot, you can see 3 apps running with one app taking 9Gb, and two apps taking 1Gb each. Therefore the total is 11Gb. You can check the similar in your environment.

Hope this helps.

Thanks,

Aditya

Re: from where we get the Yarn memory value from the dashboard

mike_bronson7 — Fri, 27 Oct 2017 00:52:41 GMT

I am little confuse in the value from yarn.nodemanager.resource.memory-mb is more then 120G and each worker machine only 32G , so how it can be that yarn.nodemanager.resource.memory-mb value much more the memory size of the worker machine ?

Re: from where we get the Yarn memory value from the dashboard

mike_bronson7 — Fri, 27 Oct 2017 00:53:24 GMT

Re: from where we get the Yarn memory value from the dashboard

asirna — Fri, 27 Oct 2017 01:11:42 GMT

@uri ben-ari,

This value is calculated using the stack advisor

'yarn.nodemanager.resource.memory-mb' = int(round(min(clusterData['containers']* clusterData['ramPerContainer'], nodemanagerMinRam))))