- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Is configuring queue optional for Oozie actions
- Labels:
-
Apache Oozie
Created ‎08-30-2016 10:19 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am browsing on a number of Oozie examples and the three actions I'm most interested in are Sqoop, Hive and Pig actions. On some examples, the mapreduce.job.queuename
property (mapred.queue.name
for older version) is configured, normally to default
, like this:
<action> <sqoop> ... <configuration> <property> <name>mapreduce.job.queuename</name> <value>default</value> </property> </configuration> </sqoop> <action>
On the other hand, other examples (especially on Pig, Hive and Hive2 actions) didn't specify any queue (no global configuration is specified either). Does configuring the queue necessary? What would happen if I didn't specify any?
Created ‎08-30-2016 11:48 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Setting of this property does not depend on the type of action. If you do not specify it then it will use the default value of the property. The default value for 'mapreduce.job.queuename' is 'default' only.
Created ‎08-30-2016 11:48 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Setting of this property does not depend on the type of action. If you do not specify it then it will use the default value of the property. The default value for 'mapreduce.job.queuename' is 'default' only.
Created ‎08-30-2016 12:06 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, it always good to configure and specify queue for better utilization of the compute resources rather running in default queue.
Created ‎09-01-2016 02:37 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The answer is it all depends on how YARN is setup for queues. All tools(sqoop, pig, hive) have a way of specifying queue via command line (example) If you are using HUE it can even be setup to impersonate your user. So you really do need to understand how yarn is setup for queuing. You don't need to configure the queue if yarn isn't configured for queues. If it is then you have to read the configuration to know what will happen.
