Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

assign VERY_HIGH priority to an oozie workflow

assign VERY_HIGH priority to an oozie workflow

Expert Contributor

I am using the default scheduler, which is Fair scheduler with Cloudera deployment of Hadoop. One of my workflows in oozie is very important and I would always like it to be allocated the reesources it needs before anything else.

 

This workflow has a sqoop job and hive queries. What do I need to do to assign VERY_HIGH priority to this workflow . A direction would be appreciated. I have tried looking it up but still pretty clueless about "how" to do it

1 REPLY 1
Highlighted

Re: assign VERY_HIGH priority to an oozie workflow

Master Guru
YARN FairScheduler uses the concepts of queues/pool weights to implicitly assign priorities. Features such as preemption and minimum allocations would help your use-case. We have a detailed blog post with several similar scenarios that you can go over: https://blog.cloudera.com/blog/2017/02/untangling-apache-hadoop-yarn-part-5-using-fairscheduler-queu...

Specifically, look for the section example titled: "Example: Implementing Priority Using Preemption"