question Name node heap memory and yarn memory? in Archives of Support Questions (Read Only)

Name node heap memory and yarn memory?

ansarifah — Thu, 14 Jul 2016 10:12:32 GMT

I see in ambari cluster: If yarn memory is increases for some processes at the same time namenode heap memory are also increases. What is the relation between them?

Re: Name node heap memory and yarn memory?

sunile_manjee — Thu, 14 Jul 2016 10:16:14 GMT

@ANSARI FAHEEM AHMED I am not sure if I follow your question. You should be able to increase yarn memory independent of node node heap size. Ambari may make recommendations based on the memory available to cluster what the name node heap size should be. So if yarn memory is increase it now believes more memory is available to the cluster and may make a increase/decrease recommendation Run the yarn util scipt available here to play with different configurations.

Re: Name node heap memory and yarn memory?

ansarifah — Thu, 14 Jul 2016 10:26:16 GMT

Thanks, but some time namenode heap memory is automatic increasing and decreasing so what was the issue?

Re: Name node heap memory and yarn memory?

hrongali — Thu, 14 Jul 2016 22:43:18 GMT

@ANSARI FAHEEM AHMED

Are you referring to the running jobs on the cluster, as they acquire resources on YARN, the namenode heap is increasing ? If yes, your processes might be under the hood making lot of namenode requests which might be the reason for increase in namenode heap size as well. The namenode heap size will eventually get down after it hits Garbage Collection.