05-11-2015 04:07 AM
I have a setup where some machines are more CPU powerful than others, and would like to direct some of the jobs to those machines. Is there a way to map a queue to a subset of the machines using any of the schedulers ?
05-13-2015 06:29 PM
There is no need to do that when you use the FairScheduler that is how things work.
Resources for a container in YARN are vcores and memory. In the YARN configuration you use that everywhere also when you define the size of the node. The node that has more memory and CPU will thus get more vcores and more memory available for containers. In CM you create groups of NM's and each group of NM's get their own configuration based on their hardware size (cores and mem). The scheduler will then take care of placing more containers on the large nodes for you.