Support Questions
Find answers, ask questions, and share your expertise

Why does YARN Capacity Scheduling queue not reach max capacity when other queues are idle?

New Contributor

I have set up YARN Capacity Scheduling with fair policy. I would like to know that I set up max capacity to 100% so that everyone should get all resources when no one in other queue is running. However, the problem is that users only reach to their queue capacity, no more than that. As far as I know, it can be configured to utilize resource of other queues when others are not using it. What should I do to meet the requirement?


Please find attached example of my queue setting.

Thank you

40433-screenshot-from-2017-09-20-18-54-12.png

2 REPLIES 2

Cloudera Employee

Your User Limit factor is 1, which allows only a single user to use the queues full capacity and Minimum user Limit is 100%, which means that only one user gets 100% Capacity.

This blocks other users from launching jobs in parallel, which causes the later jobs to starve till first one's complete as you Ordering Policy is "fifo".

Also check the Maximum Application configuration of your base queue "root" as the configuration is Inherited.

New Contributor

Your problem is the User Limit Factor. The 1 value indicates that 1 user can take the 100% of the capacity of the queue, but in this 100% is not considered the over capacity (the capacity that other queues are not using).

In this case, you have to set the User Limit Factor to 2.

; ;