Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

yarn.scheduler.fair.user-as-default-queue - Not working

avatar
Expert Contributor

Hi,

I've got many applications sat running under the default queue, which does not make sense to me as I've set:

yarn.scheduler.fair.user-as-default-queue - True

From my understanding this should mean that is a job runs under user XYZ, then as long as I have a dynamic resource pool named XYZ the job should utlize that pool?  Am i misunderstaning something?

Currently using fair scheudler.

ANy help would be great!

Thanks,
Chris 

1 ACCEPTED SOLUTION

avatar
Expert Contributor

perfect thank you, that was the issue.  I needed to add the placement rule "Use the pool root.[Username], only if the pool exists".

View solution in original post

2 REPLIES 2

avatar
Champion
What do you have for your placement rules? I forget the CDH defaults but I know a fresh cluster will have that setting as true and it will create user queues automatically.

The Hadoop docs do have this note regarding the settings. That is why I asked about the placement rules you have.

"If a queue placement policy is given in the allocations file, this property is ignored."

https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/FairScheduler.html

avatar
Expert Contributor

perfect thank you, that was the issue.  I needed to add the placement rule "Use the pool root.[Username], only if the pool exists".