Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Yarn Scheduling Management, Capacity Scheduler

Solved Go to solution

Yarn Scheduling Management, Capacity Scheduler

New Contributor

In Capacity Scheduler, How Resource Manage decide which Job runs on which Queue?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Yarn Scheduling Management, Capacity Scheduler

Expert Contributor

@Ishvari Dhimmar

The direct answer to your question is that it's not the Resource Manager that decides in which queue a job should run. This is specified explicitly by the user. If you don't specify the queue explicitly, then by default the job will be submitted to the "default" queue (if a queue named "default" doesn't exist in your config, an error will be thrown).

How you can specify the queue depends on the application, e.g. for MapReduce jobs it's defined by "-Dmapred.job.queue.name".

3 REPLIES 3

Re: Yarn Scheduling Management, Capacity Scheduler

Super Mentor

@Ishvari Dhimmar

The following tutorial will help in understanding the same with a real time example with 3 queues ("Support, Marketing, Engineering" Queues): https://hortonworks.com/hadoop-tutorial/configuring-yarn-capacity-scheduler-ambari/#configure-schedu...

.

Additionally please refer to: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_yarn_resource_mgt/content/setting_up_que...

.

Re: Yarn Scheduling Management, Capacity Scheduler

Expert Contributor

@Ishvari Dhimmar

The direct answer to your question is that it's not the Resource Manager that decides in which queue a job should run. This is specified explicitly by the user. If you don't specify the queue explicitly, then by default the job will be submitted to the "default" queue (if a queue named "default" doesn't exist in your config, an error will be thrown).

How you can specify the queue depends on the application, e.g. for MapReduce jobs it's defined by "-Dmapred.job.queue.name".

Re: Yarn Scheduling Management, Capacity Scheduler

New Contributor

Thanks for Reply. I was thinking like this, but confused. Now its clear. Thanks again.