- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Limit vcores with Capacity Scheduler
- Labels:
-
Apache YARN
Created 10-15-2020 12:11 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am running a cluster with Capacity Scheduler. Can I limit vcores for a queue with concrete number of vcores, not percentages? I found option yarn.scheduler.capacity.<queue_path>.maximum-allocation-vcores. But it seems like it doesn't work. I added this option into "Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve)" in Cloudera Manager, but applications still use more than this restriction. Can someone explain why?
Created 10-15-2020 03:24 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
After careful exploration of logs I have found some strange behavior and cant explain it. After restart there are maximum 2 vcores, that can be allocated for this queue. However, when I launch jobs, I see more vcores allocated than max for this queue. Screenshots below are made one after another. Why does it happen?