Support Questions

Find answers, ask questions, and share your expertise

Name node heap memory and yarn memory?

Contributor

I see in ambari cluster: If yarn memory is increases for some processes at the same time namenode heap memory are also increases. What is the relation between them?

1 ACCEPTED SOLUTION

Super Guru

@ANSARI FAHEEM AHMED I am not sure if I follow your question. You should be able to increase yarn memory independent of node node heap size. Ambari may make recommendations based on the memory available to cluster what the name node heap size should be. So if yarn memory is increase it now believes more memory is available to the cluster and may make a increase/decrease recommendation Run the yarn util scipt available here to play with different configurations.

View solution in original post

3 REPLIES 3

Super Guru

@ANSARI FAHEEM AHMED I am not sure if I follow your question. You should be able to increase yarn memory independent of node node heap size. Ambari may make recommendations based on the memory available to cluster what the name node heap size should be. So if yarn memory is increase it now believes more memory is available to the cluster and may make a increase/decrease recommendation Run the yarn util scipt available here to play with different configurations.

Contributor

Thanks, but some time namenode heap memory is automatic increasing and decreasing so what was the issue?

Rising Star

@ANSARI FAHEEM AHMED

Are you referring to the running jobs on the cluster, as they acquire resources on YARN, the namenode heap is increasing ? If yes, your processes might be under the hood making lot of namenode requests which might be the reason for increase in namenode heap size as well. The namenode heap size will eventually get down after it hits Garbage Collection.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.