question yarn.scheduler.fair.user-as-default-queue - Not working in Archives of Support Questions (Read Only)

yarn.scheduler.fair.user-as-default-queue - Not working

chriswalton007 — Fri, 16 Sep 2022 11:42:41 GMT

Hi,

I've got many applications sat running under the default queue, which does not make sense to me as I've set:

yarn.scheduler.fair.user-as-default-queue - True

From my understanding this should mean that is a job runs under user XYZ, then as long as I have a dynamic resource pool named XYZ the job should utlize that pool? Am i misunderstaning something?

Currently using fair scheudler.

ANy help would be great!

Thanks,
Chris

Re: yarn.scheduler.fair.user-as-default-queue - Not working

mbigelow — Tue, 06 Jun 2017 17:33:19 GMT

What do you have for your placement rules? I forget the CDH defaults but I know a fresh cluster will have that setting as true and it will create user queues automatically.

The Hadoop docs do have this note regarding the settings. That is why I asked about the placement rules you have.

"If a queue placement policy is given in the allocations file, this property is ignored."

https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/FairScheduler.html

Re: yarn.scheduler.fair.user-as-default-queue - Not working

chriswalton007 — Wed, 07 Jun 2017 08:02:10 GMT

perfect thank you, that was the issue. I needed to add the placement rule "Use the pool root.[Username], only if the pool exists".