Created on 10-16-2020 07:51 AM - edited 10-16-2020 07:54 AM
Hello.
I have cluster 9 nodes (400gb memory on each, 2 masters with NN).
Capacity will be increase every time(i add ~10TB every week).
For 31millon files used ~9gb of 15.
What maximum java heap size for NN i can set and what recommended blocksize for big cluster(now its 128mb, should i set 256/512 ?)
Created 10-16-2020 11:26 AM
Hi,
To know what should be the heap memory of the name node, please provide the storage capacity of each node and the replication factor.
Based on this we can calculate the heap memory.
the default block size is recommended as a default block size for the large and small clusters.
Created 10-16-2020 11:53 AM
@Yuriy_but Please follow below doc for NN heap calculations.
https://docs.cloudera.com/documentation/enterprise/latest/topics/admin_nn_memory_config.html
Also there is another thread nicely explained the same thing here: https://community.cloudera.com/t5/Support-Questions/How-a-NameNode-Heap-size-is-calculated/td-p/2270...