I searched in google to find info about how to tune the value for - DataNode maximum Java heap size
but not found formula to calculate the value for DataNode maximum Java heap size
the default value for DataNode maximum Java heap size , is 1G
and we increase this value to 5G , because in some case we saw from datanode logs error about heap size
but this isn't the right way to tune the value
so any suggestion or good article how to set the right value for - datanode logs error about heap size ?
lets say we have the following hadoop cluster size:
10 datanode machines , with 5 disks , while each disk has 1T
each data node have 32 CPU
each data node have 256G memory
based on this info can we find the formula that show the right value for - "datanode logs error about heap size" ?
reference - https://community.hortonworks.com/articles/74076/datanode-high-heap-size-alert.html