Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

Fair Share Preemption for undeclared (user) queues?

SOLVED Go to solution

Fair Share Preemption for undeclared (user) queues?

Contributor

Hi community,

 

I am trying to isolate my users from each other in YARN (CDH 5.15.1).

 

For that, I am using a queue root.users.<username>, in which each user is directed. The queues for users are not created, but will be created on submission - they are undeclared.

 

To guarantee resources, I would like to active FairShare Preemption on all user queues, although they are undeclared. Enabling Preemption on the root.users Queue did achieve any preemption.

 

I have set:

- Preemption is activated globally

- Fair Share Preemption Threshold of root.users is set to 0.5

- Fair Share Preemption Timeout of root.users is set to 5

- Preemptable of root.users is set to true

 

In my test, the cluster resources are fully allocated to root.users.alice. Then bob submits a job to root.users.bob but does not receive any resources.

 

Best,

Benjamin

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Fair Share Preemption for undeclared (user) queues?

Contributor

Well, I have solved it: 

 

Undeclared / auto-created queues seem not to inherit their preemption threshold / timeout settings from the parent queue but from the global default settings.

 

These are definable in Cloudera Manager by selecting "Default Settings" in the "Dynamic Resource Pool Configuration" in Cloudera Manager.

 

Applications in a queue, however, seem to inherit the preemption settings from the queue they are in.

 

Leaving it here for other users searching such info.

1 REPLY 1

Re: Fair Share Preemption for undeclared (user) queues?

Contributor

Well, I have solved it: 

 

Undeclared / auto-created queues seem not to inherit their preemption threshold / timeout settings from the parent queue but from the global default settings.

 

These are definable in Cloudera Manager by selecting "Default Settings" in the "Dynamic Resource Pool Configuration" in Cloudera Manager.

 

Applications in a queue, however, seem to inherit the preemption settings from the queue they are in.

 

Leaving it here for other users searching such info.