Hi,
Our cluster uses Hadoop-2.7.3 with preemption enabled fair scheduler. There are multiple groups sharing the cluster and each group has been assigned different amounts of resources.
The issue is that, when the cluster is busy, a new job is submitted and following happen
- Jobs from the same queue are preempted;
- Too many jobs are preempted for this new job;
- Resource preempted from same job may goes back to the same job, similar as described in https://issues.apache.org/jira/browse/YARN-4931
Any suggestions on this? Or, any fix we should pick from current ymaster?
Thank you
