Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to tune the "DataNode maximum Java heap size"

Highlighted

how to tune the "DataNode maximum Java heap size"

I searched in google to find info about how to tune the value for - DataNode maximum Java heap size

but not found formula to calculate the value for DataNode maximum Java heap size

the default value for DataNode maximum Java heap size , is 1G

and we increase this value to 5G , because in some case we saw from datanode logs error about heap size

but this isn't the right way to tune the value

so any suggestion or good article how to set the right value for - datanode logs error about heap size ?

lets say we have the following hadoop cluster size:

10 datanode machines , with 5 disks , while each disk has 1T

each data node have 32 CPU

each data node have 256G memory


based on this info can we find the formula that show the right value for - "datanode logs error about heap size" ?

reference - https://community.hortonworks.com/articles/74076/datanode-high-heap-size-alert.html

capture.png


capture.png
Michael-Bronson