Reply
Contributor
Posts: 65
Registered: ‎05-19-2016

assign VERY_HIGH priority to an oozie workflow

I am using the default scheduler, which is Fair scheduler with Cloudera deployment of Hadoop. One of my workflows in oozie is very important and I would always like it to be allocated the reesources it needs before anything else.

 

This workflow has a sqoop job and hive queries. What do I need to do to assign VERY_HIGH priority to this workflow . A direction would be appreciated. I have tried looking it up but still pretty clueless about "how" to do it

Highlighted
Posts: 1,567
Kudos: 289
Solutions: 240
Registered: ‎07-31-2013

Re: assign VERY_HIGH priority to an oozie workflow

YARN FairScheduler uses the concepts of queues/pool weights to implicitly assign priorities. Features such as preemption and minimum allocations would help your use-case. We have a detailed blog post with several similar scenarios that you can go over: https://blog.cloudera.com/blog/2017/02/untangling-apache-hadoop-yarn-part-5-using-fairscheduler-queu...

Specifically, look for the section example titled: "Example: Implementing Priority Using Preemption"
Backline Customer Operations Engineer
Announcements