08-23-2015 10:11 AM
These two default parametes doesnt make sense for me.
the total available cores for all containers.
"Number of CPU cores that can be allocated for containers."
the maximum number of cores that a container can request.
"The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this won't take effect, and will get capped to this value."
So if I have only 8 cores available for all containers how can the AM request a 32 core container?
These two default parameters must bem at least the same, or the yarn.scheduler.maximum-allocation-vcores less than yarn.nodemanager.resource.cpu-vcores.
Am i rigth?
08-24-2015 03:53 PM
yarn.scheduler.maximum-allocation-vcores controls the maximum vcores that any submitted job can request. yarn.nodemanager.resource.cpu-vcores, on the other hand, controls how many vcores can be scheduled on a particular NodeManager instance.
So yarn.nodemanager.resource.cpu-vcores can vary from host to host (NodeManager to NodeManager), while yarn.scheduler.maximum-allocation-vcores is a global property of the scheduler.
12-23-2015 12:00 AM - last edited on 12-23-2015 05:14 AM by cjervis
Yes... it doesn't make sense... I bumped into the same question, and the default of "yarn.scheduler.maximum-allocation-vcores" is actually 4 in the code.
I fired a JIRA for this.