Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Regarding NameNode Heap Size .

Highlighted

Regarding NameNode Heap Size .

New Contributor

As per hortonwork documentation Young Generation Size should be 1/8 of heap memory size .But I can see in below link data is not matching i.e. (5-10 million file heap size is 5376 and Young Generation Size is 768 which is not matching as 1/8 of 5376 is 672) .

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_installing_manually_book/content/ref-809...

Can anybody clear me if I am missing something ?

1 REPLY 1

Re: Regarding NameNode Heap Size .

@Abhishek Kumar

It's not mandatory to have 1/8. It's current recommendation is to set this to 1/8 to 1/6 of the total heap size.

-XX:NewSize=128M -XX:MaxNewSize=128M

These control the minimum and maximum size of the new generation in the generational garbage collector architecture. We’ve found that for large heaps, the JVM defaults are too small for this. Our current recommendation is to set this to 1/8 to 1/6 of the total heap size. We recommend setting each of these to the same value for stability reasons, similar to the above discussion of total heap size.

-XX:PermSize=128M -XX:MaxPermSize=256M

This controls the minimum and maximum size of the permanent generation, which is a portion of the heap dedicated to application metadata, such as Java class definitions and interned string variables. If this is too small, then you’ll see errors like:

java.lang.OutOfMemoryError: PermGen space

These settings are our current recommendation for what works well for our usage patterns.

For more details of calculation NN parameters you can refer to this link: https://community.hortonworks.com/articles/14170/namenode-garbage-collection-configuration-best-pra....

Hope this helps you.