Datanode Heap Size computation

[#1]

Hi All, 

Can some one help with the document/steps/formula to calculate the datanode heap size when configuring fresh cluster 

[#2]

@sarm 

Minimum heap size should be set to : 4 GB

Increase the memory for higher replica counts or a higher number of blocks per DataNode. When increasing the memory, Cloudera recommends an additional 1 GB of memory for every 1 million replicas above 4 million on the DataNodes. For example, 5 million replicas require 5 GB of memory.

Set this value using the Java Heap Size of DataNode in Bytes HDFS configuration property.

Reference:

https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_hardware_requirements.h...

Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.