Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

Limit vcores with Capacity Scheduler


I am running a cluster with Capacity Scheduler. Can I limit vcores for a queue with concrete number of vcores, not percentages? I found option yarn.scheduler.capacity.<queue_path>.maximum-allocation-vcores. But it seems like it doesn't work. I added this option into "Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve)" in Cloudera Manager, but applications still use more than this restriction. Can someone explain why?



After careful exploration of logs I have found some strange behavior and cant explain it. After restart there are maximum 2 vcores, that can be allocated for this queue. However, when I launch jobs, I see more vcores allocated than max for this queue. Screenshots below are made one after another. Why does it happen?Screen Shot 2020-10-15 at 1.17.40 PM.pngScreen Shot 2020-10-15 at 1.16.48 PM.png