I have cluster 9 nodes (400gb memory on each, 2 masters with NN).
Capacity will be increase every time(i add ~10TB every week).
For 31millon files used ~9gb of 15.
What maximum java heap size for NN i can set and what recommended blocksize for big cluster(now its 128mb, should i set 256/512 ?)
To know what should be the heap memory of the name node, please provide the storage capacity of each node and the replication factor.
Based on this we can calculate the heap memory.
the default block size is recommended as a default block size for the large and small clusters.
@Yuriy_but Please follow below doc for NN heap calculations.
Also there is another thread nicely explained the same thing here: https://community.cloudera.com/t5/Support-Questions/How-a-NameNode-Heap-size-is-calculated/td-p/2270...