Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

YARN capacity confiurations Illegal capacity of 0.25 for children of queue root

Solved Go to solution
Highlighted

YARN capacity confiurations Illegal capacity of 0.25 for children of queue root

I am trying to adjust the yarn capacity settings..for a newly setup hive/druid stack.

The system was 'working' but then this week it started complaining about

Failed to submit application_1550602511762_0005 to YARN : org.apache.hadoop.security.AccessControlException: Queue root.default already has 0 applications, cannot accept submission of application: application_1550602511762_0005

it was suggested I needed to reduce the capacity of the llap queue.

my 'scheduling' config I am trying..

yarn.scheduler.capacity.default.minimum-user-limit-percent=100
yarn.scheduler.capacity.maximum-am-resource-percent=0.2
yarn.scheduler.capacity.maximum-applications=10000
yarn.scheduler.capacity.node-locality-delay=40
yarn.scheduler.capacity.resource-calculator=org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator
yarn.scheduler.capacity.root.accessible-node-labels=*
yarn.scheduler.capacity.root.acl_administer_queue=*
yarn.scheduler.capacity.root.acl_submit_applications=*
yarn.scheduler.capacity.root.capacity=100
yarn.scheduler.capacity.root.default.acl_administer_jobs=*
yarn.scheduler.capacity.root.default.acl_submit_applications=*
yarn.scheduler.capacity.root.default.capacity=0.0
yarn.scheduler.capacity.root.default.maximum-capacity=0.0
yarn.scheduler.capacity.root.default.state=RUNNING
yarn.scheduler.capacity.root.default.user-limit-factor=1
yarn.scheduler.capacity.root.llap.acl_administer_queue=hive
yarn.scheduler.capacity.root.llap.acl_submit_applications=hive
yarn.scheduler.capacity.root.llap.capacity=25.0
yarn.scheduler.capacity.root.llap.maximum-am-resource-percent=1
yarn.scheduler.capacity.root.llap.maximum-capacity=100.0
yarn.scheduler.capacity.root.llap.minimum-user-limit-percent=100
yarn.scheduler.capacity.root.llap.ordering-policy=fifo
yarn.scheduler.capacity.root.llap.state=RUNNING
yarn.scheduler.capacity.root.llap.user-limit-factor=1
yarn.scheduler.capacity.root.queues=default,llap
yarn.scheduler.capacity.schedule-asynchronously.enable=true
yarn.scheduler.capacity.schedule-asynchronously.maximum-threads=1
yarn.scheduler.capacity.schedule-asynchronously.scheduling-interval-ms=10



caused by: java.io.IOException: Failed to re-init queues : Illegal capacity of 0.25 for children of queue root    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:478)    
at org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:423)    
at org.apache.hadoop.yarn.server.resourcemanager.AdminService.refreshQueues(AdminService.java:394)    
.. 10 moreCaused by: java.lang.IllegalArgumentException: Illegal capacity of 0.25 for children of queue roots

I have tried various configs..that add up to 100 or are less than 100, but it keeps complaining..



1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: YARN capacity confiurations Illegal capacity of 0.25 for children of queue root

Super Mentor

@Dan Hops

Can you try changing the

"yarn.scheduler.capacity.root.default.capacity=0.0" to "yarn.scheduler.capacity.root.default.capacity=75"

"yarn.scheduler.capacity.root.default.maximum-capacity=0.0" to "yarn.scheduler.capacity.root.default.maximum-capacity=100"

Similarly change the "yarn.scheduler.capacity.root.llap.capacity=25.0" to "yarn.scheduler.capacity.root.llap.capacity=25"

Example:

yarn.scheduler.capacity.maximum-am-resource-percent=0.2
yarn.scheduler.capacity.maximum-applications=10000
yarn.scheduler.capacity.node-locality-delay=40
yarn.scheduler.capacity.root.accessible-node-labels=*
yarn.scheduler.capacity.root.acl_administer_queue=*
yarn.scheduler.capacity.root.capacity=100
yarn.scheduler.capacity.root.default.acl_submit_applications=*
yarn.scheduler.capacity.root.default.capacity=75
yarn.scheduler.capacity.root.default.maximum-capacity=100
yarn.scheduler.capacity.root.default.state=RUNNING
yarn.scheduler.capacity.root.default.user-limit-factor=1
yarn.scheduler.capacity.root.queues=default,llap
yarn.scheduler.capacity.queue-mappings-override.enable=false
yarn.scheduler.capacity.root.acl_submit_applications=*
yarn.scheduler.capacity.root.default.priority=0
yarn.scheduler.capacity.root.llap.acl_administer_queue=hive
yarn.scheduler.capacity.root.llap.acl_submit_applications=hive
yarn.scheduler.capacity.root.llap.capacity=25
yarn.scheduler.capacity.root.llap.maximum-am-resource-percent=1
yarn.scheduler.capacity.root.llap.maximum-capacity=100
yarn.scheduler.capacity.root.llap.minimum-user-limit-percent=100
yarn.scheduler.capacity.root.llap.ordering-policy=fifo
yarn.scheduler.capacity.root.llap.priority=0
yarn.scheduler.capacity.root.llap.state=RUNNING
yarn.scheduler.capacity.root.llap.user-limit-factor=1
yarn.scheduler.capacity.root.priority=0


View solution in original post

3 REPLIES 3
Highlighted

Re: YARN capacity confiurations Illegal capacity of 0.25 for children of queue root

Super Mentor

@Dan Hops

Can you try changing the

"yarn.scheduler.capacity.root.default.capacity=0.0" to "yarn.scheduler.capacity.root.default.capacity=75"

"yarn.scheduler.capacity.root.default.maximum-capacity=0.0" to "yarn.scheduler.capacity.root.default.maximum-capacity=100"

Similarly change the "yarn.scheduler.capacity.root.llap.capacity=25.0" to "yarn.scheduler.capacity.root.llap.capacity=25"

Example:

yarn.scheduler.capacity.maximum-am-resource-percent=0.2
yarn.scheduler.capacity.maximum-applications=10000
yarn.scheduler.capacity.node-locality-delay=40
yarn.scheduler.capacity.root.accessible-node-labels=*
yarn.scheduler.capacity.root.acl_administer_queue=*
yarn.scheduler.capacity.root.capacity=100
yarn.scheduler.capacity.root.default.acl_submit_applications=*
yarn.scheduler.capacity.root.default.capacity=75
yarn.scheduler.capacity.root.default.maximum-capacity=100
yarn.scheduler.capacity.root.default.state=RUNNING
yarn.scheduler.capacity.root.default.user-limit-factor=1
yarn.scheduler.capacity.root.queues=default,llap
yarn.scheduler.capacity.queue-mappings-override.enable=false
yarn.scheduler.capacity.root.acl_submit_applications=*
yarn.scheduler.capacity.root.default.priority=0
yarn.scheduler.capacity.root.llap.acl_administer_queue=hive
yarn.scheduler.capacity.root.llap.acl_submit_applications=hive
yarn.scheduler.capacity.root.llap.capacity=25
yarn.scheduler.capacity.root.llap.maximum-am-resource-percent=1
yarn.scheduler.capacity.root.llap.maximum-capacity=100
yarn.scheduler.capacity.root.llap.minimum-user-limit-percent=100
yarn.scheduler.capacity.root.llap.ordering-policy=fifo
yarn.scheduler.capacity.root.llap.priority=0
yarn.scheduler.capacity.root.llap.state=RUNNING
yarn.scheduler.capacity.root.llap.user-limit-factor=1
yarn.scheduler.capacity.root.priority=0


View solution in original post

Highlighted

Re: YARN capacity confiurations Illegal capacity of 0.25 for children of queue root

that seems to work no error in the log, the issue was that I had it specified in 'floating point'?

Also now it my 'job' is accepted on the applications page, but it never starts..

There is one other job running on that queue (default), however when I try to kill it..it just errors out..with Error: Kill application failed!



Highlighted

Re: YARN capacity confiurations Illegal capacity of 0.25 for children of queue root

ON the yarn queues tab, I see: root (100%), default(0%), and llap(100%)..does this may changes had no effect??


Don't have an account?
Coming from Hortonworks? Activate your account here