Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

CapacityScheduler - Job Priority preemption

Solved Go to solution

CapacityScheduler - Job Priority preemption

Super Guru

@Tim Hall

Is there any feature or functionality with capacity scheduler where it would provide resources to higher priority job which kicked off during lower priority large job which is comsuming most of the resources. Assuming same org and que.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: CapacityScheduler - Job Priority preemption

Mentor

Here's some information for you regarding job preemption, there are some parameters you can tune now and others are on the roadmap. http://hortonworks.com/blog/better-slas-via-resour...

This is how you would turn job preemption on HDP 2.3 http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-...

4 REPLIES 4

Re: CapacityScheduler - Job Priority preemption

@Sunile Manjee

https://hadoop.apache.org/docs/r1.2.1/capacity_sch...

  • Job Priorities - Queues optionally support job priorities (disabled by default). Within a queue, jobs with higher priority will have access to the queue's resources before jobs with lower priority. However, once a job is running, it will not be preempted for a higher priority job, premption is on the roadmap is currently not supported.

Re: CapacityScheduler - Job Priority preemption

Re: CapacityScheduler - Job Priority preemption

Mentor

Here's some information for you regarding job preemption, there are some parameters you can tune now and others are on the roadmap. http://hortonworks.com/blog/better-slas-via-resour...

This is how you would turn job preemption on HDP 2.3 http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-...

Re: CapacityScheduler - Job Priority preemption

Mentor

@Sunile Manjee are you still having issues with this? Can you accept the best answer or provide your own solution?