08-13-2017 07:16 AM
I'm using the capacity scheduler in YARN and I saw that there's the possibility for users to get a minimum percentage of the queue by using the property 'yarn minimum-user-limit-percent'. I set this property to 20, and what I would expect is that resources would get equally distributed up to 5 users, according to this:
But that's not the case when users start runnig applications. For instance, if I run an application when the queue is idle, YARN allocates all the requested resources for that application. When another user runs the same application afterwards, YARN allocates as much resources as there are left in the queue, and the queue gets filled up. At this point, I thought that with the property the second user would get 50% of the queue, and the first one would have less resources.
If a third user comes in, I would expect him/her to get 33% of the queue, but YARN doesn't even schedule the application because there are no available resources left.
Am I missing something? I thought this parameter made requests independent of the available resources until it hit the minimum percentage per user.