question Re: Is configuring queue optional for Oozie actions in Archives of Support Questions (Read Only)

Is configuring queue optional for Oozie actions

menorah84 — Tue, 30 Aug 2016 17:19:13 GMT

I am browsing on a number of Oozie examples and the three actions I'm most interested in are Sqoop, Hive and Pig actions. On some examples, the mapreduce.job.queuename property (mapred.queue.name for older version) is configured, normally to default, like this:

<action>
  <sqoop>
  ...
    <configuration>
      <property>
        <name>mapreduce.job.queuename</name>
        <value>default</value>
      </property>
    </configuration>
  </sqoop>
<action>

On the other hand, other examples (especially on Pig, Hive and Hive2 actions) didn't specify any queue (no global configuration is specified either). Does configuring the queue necessary? What would happen if I didn't specify any?

Re: Is configuring queue optional for Oozie actions

abafna — Tue, 30 Aug 2016 18:48:46 GMT

Setting of this property does not depend on the type of action. If you do not specify it then it will use the default value of the property. The default value for 'mapreduce.job.queuename' is 'default' only.

https://hadoop.apache.org/docs/r2.7.1/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml

Re: Is configuring queue optional for Oozie actions

pbishnoi — Tue, 30 Aug 2016 19:06:00 GMT

@J. D. Bacolod

Yes, it always good to configure and specify queue for better utilization of the compute resources rather running in default queue.

Re: Is configuring queue optional for Oozie actions

matt_andruff — Thu, 01 Sep 2016 09:37:23 GMT

The answer is it all depends on how YARN is setup for queues. All tools(sqoop, pig, hive) have a way of specifying queue via command line (example) If you are using HUE it can even be setup to impersonate your user. So you really do need to understand how yarn is setup for queuing. You don't need to configure the queue if yarn isn't configured for queues. If it is then you have to read the configuration to know what will happen.