Support Questions

Find answers, ask questions, and share your expertise

Yarn Scheduling Management, Capacity Scheduler

Explorer

In Capacity Scheduler, How Resource Manage decide which Job runs on which Queue?

1 ACCEPTED SOLUTION

Expert Contributor

@Ishvari Dhimmar

The direct answer to your question is that it's not the Resource Manager that decides in which queue a job should run. This is specified explicitly by the user. If you don't specify the queue explicitly, then by default the job will be submitted to the "default" queue (if a queue named "default" doesn't exist in your config, an error will be thrown).

How you can specify the queue depends on the application, e.g. for MapReduce jobs it's defined by "-Dmapred.job.queue.name".

View solution in original post

3 REPLIES 3

Super Mentor

@Ishvari Dhimmar

The following tutorial will help in understanding the same with a real time example with 3 queues ("Support, Marketing, Engineering" Queues): https://hortonworks.com/hadoop-tutorial/configuring-yarn-capacity-scheduler-ambari/#configure-schedu...

.

Additionally please refer to: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_yarn_resource_mgt/content/setting_up_que...

.

Expert Contributor

@Ishvari Dhimmar

The direct answer to your question is that it's not the Resource Manager that decides in which queue a job should run. This is specified explicitly by the user. If you don't specify the queue explicitly, then by default the job will be submitted to the "default" queue (if a queue named "default" doesn't exist in your config, an error will be thrown).

How you can specify the queue depends on the application, e.g. for MapReduce jobs it's defined by "-Dmapred.job.queue.name".

Explorer

Thanks for Reply. I was thinking like this, but confused. Now its clear. Thanks again.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.