Support Questions
Find answers, ask questions, and share your expertise

heap size of namenode/ blocksize

heap size of namenode/ blocksize

Explorer

Hello.

I have cluster 9 nodes  (400gb memory on each, 2 masters with NN).

Capacity will be increase every time(i add ~10TB every week).

For 31millon files used ~9gb of 15.

What maximum java heap size for NN i can set and what recommended blocksize for big cluster(now its 128mb, should i set 256/512 ?)heap.jpg

2 REPLIES 2

Re: heap size of namenode/ blocksize

Hi,

 

To know what should be the heap memory of the name node, please provide the storage capacity of each node and the replication factor.

 

Based on this we can calculate the heap memory.

 

the default block size is recommended as a default block size for the large and small clusters.

Re: heap size of namenode/ blocksize

@Yuriy_but Please follow below doc for NN heap calculations. 

https://docs.cloudera.com/documentation/enterprise/latest/topics/admin_nn_memory_config.html

 

Also there is another thread nicely explained the same thing here: https://community.cloudera.com/t5/Support-Questions/How-a-NameNode-Heap-size-is-calculated/td-p/2270...


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.