New Contributor
Posts: 8
Registered: ‎08-26-2016

How to set yarn.resourcemanager preemption.max_wait_before_kill ?

[ Edited ]


Is it possible to set this parameter from Cloudera Express 5.14.1 UI?



I highly suspect Yarn to kill some spark executor before they can properly end.




Posts: 1,772
Kudos: 391
Solutions: 286
Registered: ‎07-31-2013

Re: How to set yarn.resourcemanager preemption.max_wait_before_kill ?

The property you refer to is for the Capacity Scheduler's preemption settings. Are you using the Capacity Scheduler in your cluster?

For parameters not available in a service's UI, you can typically add it to the safety-valve of the relevant config file. Your quoted configuration is for the ResourceManager, so add it to the following field:

YARN - Configuration - 'ResourceManager Advanced Configuration Snippet (Safety Valve) for yarn-site.xml' as property name and value in the UI.

Note: CDH and CM primarily supports the Fair Scheduler which has fair-share based preemption controls that can be fine-tuned extensively: