Created 02-12-2019 02:20 PM
Hi,
I want to run the flume job using shell action in oozie. I run the flume job manually and it was executed successfully.
I have created below shell script which initiate the flume job:
script.sh:
/usr/hdp/2.x.x.x-xxx/flume/bin/flume-ng agent -n agent1 --conf /etc/flume/conf -f flume.conf
-Dflume.root.logger=DEBUG,console
Flume job executed using shell script(.sh file) But while executing flume job from oozie , shell action get stuck in PREP mode.
job.xml :
<configuration>
<property> <name>nameNode</name> <value>hdfs://hostname:8020</value> </property>
<property> <name>resourceManager</name> <value>hdfs://hostname:8050</value> </property>
<property> <name>oozie.use.system.libpath</name> <value>true</value> </property>
<property> <name>oozie.wf.application.path</name> <value>hdfs://hostname:8020/oozie</value> </property>
<property> <name>queueName</name> <value>default</value> </property>
</configuration>
workflow.xml :
<workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
<start to="shell-node"/> <action name="shell-node">
<shell xmlns="uri:oozie:shell-action:0.2">
<job-tracker>${resourceManager}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property>
</configuration>
<exec>script.sh</exec>
<file>hdfs://hostname:8020/oozie/script.sh#script.sh</file>
<capture-output/>
</shell>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Shell action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<kill name="fail-output">
<message>Incorrect output, expected [Hello Oozie] but was [${wf:actionData('shell-node')['my_output']}]</message>
</kill>
<end name="end"/>
</workflow-app>
Thanks,