Support Questions
Find answers, ask questions, and share your expertise

Dynamically created resource pools in YARN / CDP 7

Highlighted

Dynamically created resource pools in YARN / CDP 7

Explorer

Hello.

 

In CDH, the process of creating a parent pool that allowed dynamic child pools to be created at runtime was trivial -- simply mark the pool as Parent.  In CDP 7 I'm running into the following issue...

 

For a leaf pool  (lets say root.testpool), I edit pool attributes in the Yarn Queue Manager UI.  

Under the "Dynamic Auto-Creation of Queue" section, I see....

 

"Is Auto Created Leaf Queue", which is unchecked and uneditable.  This makes sense since "root.testpool" was not auto created.

 

"Enable Dynamic Queue Creation' is unchecked but it also uneditable.  I feel this should be editable in order to define the "root.testpool" as supporting dynamic queues below it.

 

How do I enable dynamic queues for a given pool?

 

Thanks,

M

2 REPLIES 2

Re: Dynamically created resource pools in YARN / CDP 7

Cloudera Employee

@MikeZ Haven't checked the feature in CDP yet. but as per document with pre-existing child queue this wont work. you could try creating new parent queue and edit it's child queue options.

 

"The managed parent queue does not support other pre-configured queues to co-exist along with dynamically-created queues"

 

REF: Link

Highlighted

Re: Dynamically created resource pools in YARN / CDP 7

Explorer

There are no preconfigured queues under the parent queue.

 

That checkbox seems to be tied to placement rules.  However, for CDP, the only placement rules that allow autocreation of leafs is tied to "root.users".  In CDH, you could have a leaf queue created anywhere if "create queue if non-existent" was checked.  In CDP, the freedom to do that seems limited to things under root.users...

 

Hopefully I'm missing something.  The inability to create leaf queues under arbitrary parent queues would be a big step backwards in terms of queue flexibilty.