Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Fair Share Preemption for undeclared (user) queues?

Solved Go to solution
Highlighted

Fair Share Preemption for undeclared (user) queues?

Expert Contributor

Hi community,

 

I am trying to isolate my users from each other in YARN (CDH 5.15.1).

 

For that, I am using a queue root.users.<username>, in which each user is directed. The queues for users are not created, but will be created on submission - they are undeclared.

 

To guarantee resources, I would like to active FairShare Preemption on all user queues, although they are undeclared. Enabling Preemption on the root.users Queue did achieve any preemption.

 

I have set:

- Preemption is activated globally

- Fair Share Preemption Threshold of root.users is set to 0.5

- Fair Share Preemption Timeout of root.users is set to 5

- Preemptable of root.users is set to true

 

In my test, the cluster resources are fully allocated to root.users.alice. Then bob submits a job to root.users.bob but does not receive any resources.

 

Best,

Benjamin

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Fair Share Preemption for undeclared (user) queues?

Expert Contributor

Well, I have solved it: 

 

Undeclared / auto-created queues seem not to inherit their preemption threshold / timeout settings from the parent queue but from the global default settings.

 

These are definable in Cloudera Manager by selecting "Default Settings" in the "Dynamic Resource Pool Configuration" in Cloudera Manager.

 

Applications in a queue, however, seem to inherit the preemption settings from the queue they are in.

 

Leaving it here for other users searching such info.

1 REPLY 1

Re: Fair Share Preemption for undeclared (user) queues?

Expert Contributor

Well, I have solved it: 

 

Undeclared / auto-created queues seem not to inherit their preemption threshold / timeout settings from the parent queue but from the global default settings.

 

These are definable in Cloudera Manager by selecting "Default Settings" in the "Dynamic Resource Pool Configuration" in Cloudera Manager.

 

Applications in a queue, however, seem to inherit the preemption settings from the queue they are in.

 

Leaving it here for other users searching such info.