Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

Issue:

After enabling the preemption parameters in yarn-site.xml as per link, it was found the applications are still waiting to run. This is because other queues are utilising all of the available resources. With Preemption enabled, under-served queues can begin to claim their allocated cluster resources almost immediately, without having to wait for other queues' applications to finish running.

Resolution:

Along with the below list of parameters to enable preemption, configure this parameter "yarn.resourcemanager.monitor.capacity.preemption.max_ignored_over_capacity" in yarn-site.xml, otherwise you won't see preemption though enabled. The doc is missing the above parameter.

List of parameters to enable preemption are:

1) yarn.resourcemanager.scheduler.monitor.enable

2) yarn.resourcemanager.scheduler.monitor.policies

3) yarn.resourcemanager.monitor.capacity.preemption.monitoring_interval

4) yarn.resourcemanager.monitor.capacity.preemption.max_wait_before_kill

5) yarn.resourcemanager.monitor.capacity.preemption.total_preemption_per_round

6) yarn.resourcemanager.monitor.capacity.preemption.natural_termination_factor

Reference:

https://hortonworks.com/blog/better-slas-via-resource-preemption-in-yarns-capacityscheduler/

Addendum:

Logged internal bug to include this parameter in Hortonworks official documentation

1,053 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎03-31-2017 04:43 PM
Updated by:
 
Contributors
Top Kudoed Authors