Posts: 65
Registered: ‎05-19-2016

assign VERY_HIGH priority to an oozie workflow

I am using the default scheduler, which is Fair scheduler with Cloudera deployment of Hadoop. One of my workflows in oozie is very important and I would always like it to be allocated the reesources it needs before anything else.


This workflow has a sqoop job and hive queries. What do I need to do to assign VERY_HIGH priority to this workflow . A direction would be appreciated. I have tried looking it up but still pretty clueless about "how" to do it

Posts: 1,524
Kudos: 265
Solutions: 232
Registered: ‎07-31-2013

Re: assign VERY_HIGH priority to an oozie workflow

YARN FairScheduler uses the concepts of queues/pool weights to implicitly assign priorities. Features such as preemption and minimum allocations would help your use-case. We have a detailed blog post with several similar scenarios that you can go over:

Specifically, look for the section example titled: "Example: Implementing Priority Using Preemption"
Backline Customer Operations Engineer