Capacity Scheduler : Preemption of containers in the same queue

pierre_gunet — Mon, 06 Mar 2017 18:12:33 GMT

Hi,

I recently started to use the Capacity Scheduler. Basically I have two main queues : dev and prod. Each of them have a capacity of 50% and a max capacity of 100%. My dev queue is in Fair policy while my prod queue is in FIFO. I also set the user-limit-factor to 2 (so each user could use 100% of the cluster of available). This configuration is now in production and I observed something surprising : even if the dev queue is unused (0%) and the prod queue is below 100 %, there is still preemption of containers in the prod queue.

- Can preemption happen even if the cluster is not 100% used ?

- Can jobs preempt containers from the same queue ?

Now I disabled the preemption of the prod queue because there was too much preemption, but I'm loosing some elasticity between the dev and prod 😞

Thanks for you help

Pierre

Re: Capacity Scheduler : Preemption of containers in the same queue

ssathish — Tue, 14 Mar 2017 01:57:13 GMT

Hi @Pierre Gunet-Caplain,

Does the prod Queue has child queues? If so, since the prod Queue is configured as FIFO, preemption can happen between the child queues of prod Queue based on the child queue 's capacity.

Preemption can happen even if he cluster is not 100% used for the case where new resource required is more than the available resource.

question Re: Capacity Scheduler : Preemption of containers in the same queue in Archives of Support Questions (Read Only)

Capacity Scheduler : Preemption of containers in the same queue

Re: Capacity Scheduler : Preemption of containers in the same queue