@Priyan S I tried to reproduce your issue, but couldn't (based on the given capacity configuration). I set all the properties based on your description and left everything else on default. With this setup the scheduler works as you expected.
After submitting the first two jobs, they get the maximum available resource per queue (40% each):
Then, after submitting the job for Q41 it starts getting resources (it starts with just the AM container, and then - slowly - gets more and more containers):
Finally - when preemption is enabled - the apps in Q1 and Q2 start losing resources and Q41 gets even more, and the final distribution will be like 25% - 25% - 50%, as you expected:
This means, that most likely the problem is not caused by the parameters you've provided in your question. I'd suggest you to check other parameters (you can get some of them easily from the ResourceManager Scheduler UI) - the most suspicious thing is that the apps in Q41 get so few resources, so I'd start there. Here are a few things that I can think of:
- Max Application Master Resources
- If you submit many small applications to Q41 (and big applications to Q1 and Q2), this might cause them to get hanging
- also: Max Application Master Resources Per User
- User Limit Factor
- The other way around: if you submit just one giant application to Q41
- Preemption parameters (mentioned in the previous comments)
- Maybe preemption is not set up properly - although I also tried the same setup with preemption disabled, and it worked exactly the same way, except the balancing phase took longer
- total_preemption_per_round
- max_wait_before_kill
- etc.
- Node labels
- You didn't mention in your question if you have node labels configured in your cluster. They can change the scheduling quite drastically.
If no luck, you could also check the logs of the capacity scheduler for further information.
