Resource manager heap usage

[#1]

Hello HCC,

We have 16 node cluster running on HDP 2.5.3.0, Ambari 2.5.1.0, on this cluster we allocated resource manager heap to 4G and when there is no single job running in the cluster for almost 5-6hrs, but still resource manager heap usage is going to 80-85% always.

We have another big production cluster with 800 data nodes in that cluster RM usage will be under 25-30% always.On what basis heap usage is utilized on the cluster ? and how can we control heap usage when cluster is idle.

i also checked this property "yarn.resourcemanager.max-completed-applications" which is hard coded to "${yarn.resourcemanager.max-completed-applications}" but this hard coded property not defined anywhere in Ambari.

Thanks in advance HCC.

[#2]

@Jalender were you able to find out what was eating your resource manager heap?

We are also experiencing the same thing but not daily. and our cluster is very much smaller than yours (only 29 data nodes) and when it is idle, RM heap should more or less be under 30% only. but on some days, we get very high RM heap when cluster is totally idle.

also checked "yarn.resourcemanager.state-store.max-completed-applications" and it just means value is less than or equal to "yarn.resourcemanager.max-completed-applications" which is default at 10,000.