Support Questions
Find answers, ask questions, and share your expertise

Datanode Heap Size computation

Datanode Heap Size computation


Hi All, 


Can some one help with the document/steps/formula to calculate the datanode heap size when configuring fresh cluster 




Re: Datanode Heap Size computation

Expert Contributor



Minimum heap size should be set to : 4 GB

Increase the memory for higher replica counts or a higher number of blocks per DataNode. When increasing the memory, Cloudera recommends an additional 1 GB of memory for every 1 million replicas above 4 million on the DataNodes. For example, 5 million replicas require 5 GB of memory.

Set this value using the Java Heap Size of DataNode in Bytes HDFS configuration property.



Hope this helps,
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.