Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

how to set the yarn application priority in hive ?

avatar
Explorer

Yarn's application priority can be found in yarn 8088 resource manager's website,

How do the priority work in yarn? 

Now the version of CDH I use is 6.3.2.

Hadoop 3.0.0 

Hive 2.1.1 

I use hive on spark.

Can I use the config to manage the application priority? 

For some reason some hive sqls should have high application priority when those sqls are appending and run ahead of the other.  Instead of running the same time and equal share the compute resources. 

 

 

When I set the following config in Hive, it seems do not work well in Yarn.

MapReduce"-Dmapreduce.job.priority=xx"
Flink"-yD yarn.applicaiton.priority=xx"
Spark"spark.yarn.priority=xx"

 

In hive sql."set spark.yarn.priority=10;"

It does not work ...

 

 

1 ACCEPTED SOLUTION

avatar
Expert Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
2 REPLIES 2

avatar
Expert Contributor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Explorer

Thanks a lot.

 

This "yarn application -updatePriority 10 -appId application_xxxx_xx"

seems a config of yarn.

It does not work for spark 2.x in CDH 6.3.2 either.

bulbcat_1-1669705217757.png

bulbcat_2-1669705564244.png

Does it the same reason which means the 'Application Priority' must match the yarn version with spark version?