Created 08-30-2016 10:19 AM
I am browsing on a number of Oozie examples and the three actions I'm most interested in are Sqoop, Hive and Pig actions. On some examples, the mapreduce.job.queuename
property (mapred.queue.name
for older version) is configured, normally to default
, like this:
<action> <sqoop> ... <configuration> <property> <name>mapreduce.job.queuename</name> <value>default</value> </property> </configuration> </sqoop> <action>
On the other hand, other examples (especially on Pig, Hive and Hive2 actions) didn't specify any queue (no global configuration is specified either). Does configuring the queue necessary? What would happen if I didn't specify any?
Created 08-30-2016 11:48 AM
Setting of this property does not depend on the type of action. If you do not specify it then it will use the default value of the property. The default value for 'mapreduce.job.queuename' is 'default' only.
Created 08-30-2016 11:48 AM
Setting of this property does not depend on the type of action. If you do not specify it then it will use the default value of the property. The default value for 'mapreduce.job.queuename' is 'default' only.
Created 08-30-2016 12:06 PM
Yes, it always good to configure and specify queue for better utilization of the compute resources rather running in default queue.
Created 09-01-2016 02:37 AM
The answer is it all depends on how YARN is setup for queues. All tools(sqoop, pig, hive) have a way of specifying queue via command line (example) If you are using HUE it can even be setup to impersonate your user. So you really do need to understand how yarn is setup for queuing. You don't need to configure the queue if yarn isn't configured for queues. If it is then you have to read the configuration to know what will happen.