Fair Share Preemption for undeclared (user) queues?

benhadoop — Fri, 16 Sep 2022 13:48:57 GMT

Hi community,

I am trying to isolate my users from each other in YARN (CDH 5.15.1).

For that, I am using a queue root.users.<username>, in which each user is directed. The queues for users are not created, but will be created on submission - they are undeclared.

To guarantee resources, I would like to active FairShare Preemption on all user queues, although they are undeclared. Enabling Preemption on the root.users Queue did achieve any preemption.

I have set:

- Preemption is activated globally

- Fair Share Preemption Threshold of root.users is set to 0.5

- Fair Share Preemption Timeout of root.users is set to 5

- Preemptable of root.users is set to true

In my test, the cluster resources are fully allocated to root.users.alice. Then bob submits a job to root.users.bob but does not receive any resources.

Best,

Benjamin

Re: Fair Share Preemption for undeclared (user) queues?

benhadoop — Thu, 18 Oct 2018 12:32:15 GMT

Well, I have solved it:

Undeclared / auto-created queues seem not to inherit their preemption threshold / timeout settings from the parent queue but from the global default settings.

These are definable in Cloudera Manager by selecting "Default Settings" in the "Dynamic Resource Pool Configuration" in Cloudera Manager.

Applications in a queue, however, seem to inherit the preemption settings from the queue they are in.

Leaving it here for other users searching such info.

question Fair Share Preemption for undeclared (user) queues? in Archives of Support Questions (Read Only)

Fair Share Preemption for undeclared (user) queues?

Re: Fair Share Preemption for undeclared (user) queues?