Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Resource manager heap usage

Highlighted

Resource manager heap usage

New Contributor

Hello HCC,

We have 16 node cluster running on HDP 2.5.3.0, Ambari 2.5.1.0, on this cluster we allocated resource manager heap to 4G and when there is no single job running in the cluster for almost 5-6hrs, but still resource manager heap usage is going to 80-85% always.

We have another big production cluster with 800 data nodes in that cluster RM usage will be under 25-30% always.On what basis heap usage is utilized on the cluster ? and how can we control heap usage when cluster is idle.

i also checked this property "yarn.resourcemanager.max-completed-applications" which is hard coded to "${yarn.resourcemanager.max-completed-applications}" but this hard coded property not defined anywhere in Ambari.

Thanks in advance HCC.

1 REPLY 1

Re: Resource manager heap usage

New Contributor

@Jalender were you able to find out what was eating your resource manager heap?

We are also experiencing the same thing but not daily. and our cluster is very much smaller than yours (only 29 data nodes) and when it is idle, RM heap should more or less be under 30% only. but on some days, we get very high RM heap when cluster is totally idle.

also checked "yarn.resourcemanager.state-store.max-completed-applications" and it just means value is less than or equal to "yarn.resourcemanager.max-completed-applications" which is default at 10,000.