YARN jobs always running in default pool

rjsteixeira — Fri, 16 Sep 2022 10:25:12 GMT

Hello.

We are having a problem where all the jobs we run on YARN are ending on a pool called default that we didn't create it. This happens for hive, spark, etc.

Our Dynamic Resource Pool Configuration is:

root
1. root.myuser

And our placement rules only has one entry:

root.<username> pool; create the pool if it doesn't exist

We don't understand how can the jobs end in the default pool.

Relevant configs:

yarn.scheduler.fair.user-as-default-queue [x] ResourceManager Default Group

yarn.scheduler.fair.allow-undeclared-pools [x] YARN (MR2 Included) (Service-Wide)

Re: YARN jobs always running in default pool

rjsteixeira — Wed, 15 Jun 2016 11:01:11 GMT

Fixed.

Corrected by changing the YARN configuration:

yarn.resourcemanager.scheduler.class from FifoScheduler to FairScheduler

question YARN jobs always running in default pool in Archives of Support Questions (Read Only)

YARN jobs always running in default pool

Re: YARN jobs always running in default pool