Reply
Highlighted
Explorer
Posts: 18
Registered: ‎04-26-2017
Accepted Solution

yarn.scheduler.fair.user-as-default-queue - Not working

Hi,

I've got many applications sat running under the default queue, which does not make sense to me as I've set:

yarn.scheduler.fair.user-as-default-queue - True

From my understanding this should mean that is a job runs under user XYZ, then as long as I have a dynamic resource pool named XYZ the job should utlize that pool?  Am i misunderstaning something?

Currently using fair scheudler.

ANy help would be great!

Thanks,
Chris 

Posts: 642
Topics: 3
Kudos: 105
Solutions: 67
Registered: ‎08-16-2016

Re: yarn.scheduler.fair.user-as-default-queue - Not working

What do you have for your placement rules? I forget the CDH defaults but I know a fresh cluster will have that setting as true and it will create user queues automatically.

The Hadoop docs do have this note regarding the settings. That is why I asked about the placement rules you have.

"If a queue placement policy is given in the allocations file, this property is ignored."

https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/FairScheduler.html
Explorer
Posts: 18
Registered: ‎04-26-2017

Re: yarn.scheduler.fair.user-as-default-queue - Not working

perfect thank you, that was the issue.  I needed to add the placement rule "Use the pool root.[Username], only if the pool exists".

Announcements