Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

yarn.scheduler.fair.user-as-default-queue - Not working

avatar
Expert Contributor

Hi,

I've got many applications sat running under the default queue, which does not make sense to me as I've set:

yarn.scheduler.fair.user-as-default-queue - True

From my understanding this should mean that is a job runs under user XYZ, then as long as I have a dynamic resource pool named XYZ the job should utlize that pool?  Am i misunderstaning something?

Currently using fair scheudler.

ANy help would be great!

Thanks,
Chris 

1 ACCEPTED SOLUTION

avatar
Expert Contributor

perfect thank you, that was the issue.  I needed to add the placement rule "Use the pool root.[Username], only if the pool exists".

View solution in original post

2 REPLIES 2

avatar
Champion
What do you have for your placement rules? I forget the CDH defaults but I know a fresh cluster will have that setting as true and it will create user queues automatically.

The Hadoop docs do have this note regarding the settings. That is why I asked about the placement rules you have.

"If a queue placement policy is given in the allocations file, this property is ignored."

https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/FairScheduler.html

avatar
Expert Contributor

perfect thank you, that was the issue.  I needed to add the placement rule "Use the pool root.[Username], only if the pool exists".