Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Mapping yarn queue names to racks or server groups ?

Mapping yarn queue names to racks or server groups ?

Explorer

I have a setup where some machines are more CPU powerful than others, and would like to direct some of the jobs to those machines. Is there a way to map a queue to a subset of the machines using any of the schedulers ?

 

Thank

1 REPLY 1

Re: Mapping yarn queue names to racks or server groups ?

Super Collaborator

There is no need to do that when you use the FairScheduler that is how things work.

 

Resources for a container in YARN are vcores and memory. In the YARN configuration you use that everywhere also when you define the size of the node. The node that has more memory and CPU will thus get more vcores and more memory available for containers. In CM you create groups of NM's and each group of NM's get their own configuration based on their hardware size (cores and mem). The scheduler will then take care of placing more containers on the large nodes for you.

 

Wilfred