https://community.cloudera.com/t5/Support-Questions/HOW-TO-CALCULATE-THE-HEAPSIZE-FOR-DATANODE/m-p/230517#M192367 <P>Great question and unfortunately, I don't think there is a well agreed upon formula/calculator out there as "it depends" is so often the rule. Some considerations are that the datanode doesn't really know about the directory structure; it just stores (and copies, deletes, etc) blocks as directed by the datanode (often indirectly since clients write actual blocks). Additionally, the checksums at the block level are actually stored on disk alongside the files for the data contained in a given block.</P><P>It looks like there's some good info in the following HCC Q's that might be of help to you.</P><P><A href="https://community.hortonworks.com/questions/64677/datanode-heapsize-computation.html" target="_blank">https://community.hortonworks.com/questions/64677/datanode-heapsize-computation.html</A></P><P><A href="https://community.hortonworks.com/questions/45381/do-i-need-to-tune-java-heap-size.html" target="_blank">https://community.hortonworks.com/questions/45381/do-i-need-to-tune-java-heap-size.html</A></P><P><A href="https://community.hortonworks.com/questions/78981/data-node-heap-size-warning.html" target="_blank">https://community.hortonworks.com/questions/78981/data-node-heap-size-warning.html</A></P><P>Good luck and happy Hadooping!</P> Thu, 06 Jul 2017 20:18:05 GMT LesterMartin 2017-07-06T20:18:05Z