Member since
01-27-2016
14
Posts
10
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2670 | 07-06-2017 10:02 AM | |
7641 | 01-17-2017 05:34 AM | |
2724 | 08-30-2016 11:48 AM |
07-06-2017
10:02 AM
Oozie executes the Shell action script file from a launcher job. The launcher can be allocated to any node in the hadoop cluster, So the error "no such file or directory' means that, launcher job node is different than node2 where you have stored the file. For the 'permission denied' error, please check the proper write permission set for the file, as user can be different while modifying it from oozie shell action. Though it is not advised to modify a local file from a Oozie action, as launched action can be on any node in the cluster. Once you move the file to HDFS, use the proper HDFS commands. You can not modify the file in HDFS, you need to get the file and after modification, put (store) the file again.
... View more
06-07-2017
08:24 AM
Order of service restart: Yarn, Oozie. Can check this: https://community.hortonworks.com/questions/52210/hdp-24-oozie-work-flow-ja009-cannot-initialize-clu.html
... View more
01-17-2017
05:58 AM
4 Kudos
When using the Oozie Proxy job submission API for submitting the Oozie Hive, Sqoop and Pig actions. To pass any configuration to the action is required to be in below format. For Hive action: oozie.hive.options.size
: The number of options you'll be passing to Pig oozie.hive.options.n
: An argument to pass to Hive, the 'n' should be an integer starting with 0 to indicate the option number <property>
<name>oozie.hive.options.1</name>
<value>-Doozie.launcher.mapreduce.job.queuename=hive</value>
</property>
<property>
<name>oozie.hive.options.0</name>
<value>-Dmapreduce.job.queuename=hive</value>
</property>
<property>
<name>oozie.hive.options.size</name>
<value>2</value>
</property> For Pig Action:
oozie.pig.options.size
: The number of options you'll be passing to Pig oozie.pig.options.n
: An argument to pass to Pig, the 'n' should be an integer starting with 0 to indicate the option number <property>
<name>oozie.pig.options.1</name>
<value>-Doozie.launcher.mapreduce.job.queuename=pig</value>
</property>
<property>
<name>oozie.pig.options.0</name>
<value>-Dmapreduce.job.queuename=pig</value>
</property>
<property>
<name>oozie.pig.options.size</name>
<value>2</value>
</property> For Sqoop Action: oozie.sqoop.options.size
: The number of options you'll be passing to Sqoop Hadoop job oozie.sqoop.options.n
: An argument to pass to Sqoop hadoop job conf, the 'n' should be an integer starting with zero(0) to indicate the option number <property>
<name>oozie.sqoop.options.1</name>
<value>-Doozie.launcher.mapreduce.job.queuename=sqoop</value>
</property>
<property>
<name>oozie.sqoop.options.0</name>
<value>-Dmapreduce.job.queuename=sqoop</value>
</property>
<property>
<name>oozie.sqoop.options.size</name>
<value>2</value>
</property>
... View more
Labels:
01-17-2017
05:34 AM
4 Kudos
@fogartyamanda The configuration properties will be passed to the sqoop job in following way: <property>
<name>oozie.sqoop.options.1</name>
<value>-Doozie.launcher.mapreduce.job.queuename=sqoop</value>
</property>
<property>
<name>oozie.sqoop.options.0</name>
<value>-Dmapreduce.job.queuename=sqoop</value>
</property>
<property>
<name>oozie.sqoop.options.size</name>
<value>2</value>
</property>
Thanks.
... View more
09-13-2016
06:59 PM
You are executing an oozie admin command, so you need to be using an oozie admin user. Usually 'oozie' user is configured as admin user in oozie You can take look at the 'adminusers.txt' file under oozie configuration. Once you switch to an oozie admin user, you should be able to update the oozie sharelib. 'sudo su - oozie' The another error which you have mentioned regarding the 'java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found', I guess you did not specify the 'oozie.use.system.libpath=true' into your "job.properties" file. This should resolve both of your issues. Note: If 'oozie.use.system.libpath=true' property was missing then just add it and submit/run the workflow, you need not to update the oozie share lib.
... View more
08-30-2016
11:48 AM
1 Kudo
Setting of this property does not depend on the type of action. If you do not specify it then it will use the default value of the property. The default value for 'mapreduce.job.queuename' is 'default' only. https://hadoop.apache.org/docs/r2.7.1/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
... View more