Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

assign VERY_HIGH priority to an oozie workflow

assign VERY_HIGH priority to an oozie workflow

Expert Contributor

I am using the default scheduler, which is Fair scheduler with Cloudera deployment of Hadoop. One of my workflows in oozie is very important and I would always like it to be allocated the reesources it needs before anything else.

 

This workflow has a sqoop job and hive queries. What do I need to do to assign VERY_HIGH priority to this workflow . A direction would be appreciated. I have tried looking it up but still pretty clueless about "how" to do it

1 REPLY 1
Highlighted

Re: assign VERY_HIGH priority to an oozie workflow

Master Guru
YARN FairScheduler uses the concepts of queues/pool weights to implicitly assign priorities. Features such as preemption and minimum allocations would help your use-case. We have a detailed blog post with several similar scenarios that you can go over: https://blog.cloudera.com/blog/2017/02/untangling-apache-hadoop-yarn-part-5-using-fairscheduler-queu...

Specifically, look for the section example titled: "Example: Implementing Priority Using Preemption"