Posts: 23
Registered: ‎02-07-2015

Mapping yarn queue names to racks or server groups ?

I have a setup where some machines are more CPU powerful than others, and would like to direct some of the jobs to those machines. Is there a way to map a queue to a subset of the machines using any of the schedulers ?



Cloudera Employee
Posts: 285
Registered: ‎01-16-2014

Re: Mapping yarn queue names to racks or server groups ?

There is no need to do that when you use the FairScheduler that is how things work.


Resources for a container in YARN are vcores and memory. In the YARN configuration you use that everywhere also when you define the size of the node. The node that has more memory and CPU will thus get more vcores and more memory available for containers. In CM you create groups of NM's and each group of NM's get their own configuration based on their hardware size (cores and mem). The scheduler will then take care of placing more containers on the large nodes for you.