Support Questions

Find answers, ask questions, and share your expertise

Yarn: How to make Yarn utilize more memory and vcores

Explorer

we have a 5 node cluster managed by Yarn and running hadoop

1 Master name node 8 vcores and 24GB memory

4 data nodes each 8 vcores and 24GB memory

When i look at Yarn configuration on the ui as highlighted in the picture below it's only using 16GB and 6 vcores

12047-image.png

Our application is using all of the 16gb so want to increase the memory since its available (24 - 2gb for os so avialable is 22gb)

where do i need to configure this 22gb instead of 16gb ?

From research found yarn-site.xml might be the place so went ahead and updated it and restarted yarn but still its showing 16gb

Would grealty appreciate if any of the experts in the community will help out as we are new to Yarn

3 REPLIES 3

Expert Contributor

@Aditya, Are you using Hortonworks Hadoop Distribution with Ambari.

If you are using HDP then, go to ambari and go to Yarn configuration change to config tabs and change the Memory, Container and CPU settings to your liking. Restart Yarn, Mapreduce components if they are impacted.

12060-screen-shot-2017-02-02-at-51400-pm.png

Explorer

@Satish Bomma we are using apache hadoop 2.7.3 so will not be able to make the change you suggested

Expert Contributor

@Aditya update yarn-site.xml. Find out the parameters based on the documentation and update them to increase your resource and restart yarn and its affected components.