Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

datanode with different RAM configurations...how to utilize the max memory

Highlighted

datanode with different RAM configurations...how to utilize the max memory

Hi All,

I wanted to know what parameters needs to be changed we have cluster with nodes of different characteristics in RAM,

Eg: We have 143 datanodes with 256 GB RAM planning to upgrade few datanodes to 512 GB RAM, so how can i make sure YARN utilizes this change. Currently yarn is allocated 168 GB.

I researched and found this https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-operations/content/using_host_con... which says to add config groups

Is there any detailed document for this, as what settings needs changes.

1 REPLY 1
Highlighted

Re: datanode with different RAM configurations...how to utilize the max memory

Expert Contributor

Hi @sudi ts

Create a "Config Group" under YARN Service.

Note, just increasing RAM capacity might not be enough. You may need additional CPUs to handle more jobs / containers.

yarn-site.xml
yarn.nodemanager.resource.memory-mb
yarn.nodemanager.resource.cpu-vcores

Add Node Manager Host to this config group

Restart Respective Nodemanagers associated to this config group.

Don't have an account?
Coming from Hortonworks? Activate your account here