How to estimate how much to allocate on my YARN container memory?

[#1]

Hi, our datanode server has 128gb Ram allocated when we created that server. but based on my yarn.nodemanager.resource.memory-mb, the allocated memory there is 26gb. How will I know how much I will allocate on the yarn.nodemanager.resource.memory-mb? Do I allocate all of 128gb on this?

[#2]

@Mondi 

The datanode server memory can be distributed across multiple service daemons.

yarn.nodemanager.resource.memory-mb represents the container memory set for the node managers on your cluster.

The current host utilization can be checked on Cloudera Manager -> Hosts -> Click on the required host -> Resources. This would depict the memory currently configured for various daemons running on the host.

You can increase the node manager memory further if you have enough available memory on the cluster host. Refer below document for more details on Yarn tuning 

https://docs.cloudera.com/documentation/enterprise/latest/topics/cdh_ig_yarn_tuning.html#concept_vbk...

Hope this helps,

Paras

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.