Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Yarn Scheduling Management, Capacity Scheduler

avatar
Explorer

In Capacity Scheduler, How Resource Manage decide which Job runs on which Queue?

1 ACCEPTED SOLUTION

avatar
Expert Contributor

@Ishvari Dhimmar

The direct answer to your question is that it's not the Resource Manager that decides in which queue a job should run. This is specified explicitly by the user. If you don't specify the queue explicitly, then by default the job will be submitted to the "default" queue (if a queue named "default" doesn't exist in your config, an error will be thrown).

How you can specify the queue depends on the application, e.g. for MapReduce jobs it's defined by "-Dmapred.job.queue.name".

View solution in original post

3 REPLIES 3

avatar
Master Mentor

@Ishvari Dhimmar

The following tutorial will help in understanding the same with a real time example with 3 queues ("Support, Marketing, Engineering" Queues): https://hortonworks.com/hadoop-tutorial/configuring-yarn-capacity-scheduler-ambari/#configure-schedu...

.

Additionally please refer to: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_yarn_resource_mgt/content/setting_up_que...

.

avatar
Expert Contributor

@Ishvari Dhimmar

The direct answer to your question is that it's not the Resource Manager that decides in which queue a job should run. This is specified explicitly by the user. If you don't specify the queue explicitly, then by default the job will be submitted to the "default" queue (if a queue named "default" doesn't exist in your config, an error will be thrown).

How you can specify the queue depends on the application, e.g. for MapReduce jobs it's defined by "-Dmapred.job.queue.name".

avatar
Explorer

Thanks for Reply. I was thinking like this, but confused. Now its clear. Thanks again.