Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Execute Shell script through oozie job in all node

Execute Shell script through oozie job in all node

Contributor

I tried to execute shell script through an oozie job , It seems it just executed in jobTracker host not in other nodes. I am expecting the script to be executed in all the nodes. did I required any other specific configuration . Or I missed anything here.

 

workflow.xml

------------------------------

 

<workflow-app name="script_oozie_job" xmlns="uri:oozie:workflow:0.3">
<start to='Test' />
<action name="Test">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>CopyFiles.sh</exec>
<argument>${argument1}</argument>
<file>hdfs://nameNode-host:8020/user/oozie/script/script_oozie_job/CopyFiles.sh#CopyFiles.sh</file>
</shell>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Script failed</message>
</kill>
<end name='end' />

 

 

job.properties

---------------------------------

 

nameNode=hdfs://nameNode-host:8020
jobTracker=jobTacker-host:8032
queueName=default
argument1=""
oozie.wf.application.path=hdfs://nameNode-host:8020/user/oozie/script/script_oozie_job

 

 

 

Regards

-Khirod

2 REPLIES 2

Re: Execute Shell script through oozie job in all node

Rising Star

I recommend checking out this blog post: How To: Use Oozie Shell and Java Actions

The Oozie shell action is run as a Hadoop job with one map task and zero reduce tasks - the job runs on one arbitrary node in the cluster.

 

If you are trying to run a shell-based MapReduce job across the cluster, use Hadoop Streaming. See these blog posts:

 

Re: Execute Shell script through oozie job in all node

Contributor

Thanks Sue, for help.

 

I will try Hadoop Streaming and update here how it goes.

 

-Khirod