Community Articles

KuldeepK · ‎12-21-2015

This article is an update of one on http://hadooped.blogspot.com/2013/10/apache-oozie-part-13-oozie-ssh-action_30.html, authored by @Anagha Khanolkar.

Below are the steps to setup Oozie workflow using ssh-action:

Step 1. Create job.properties.

Example:

#************************************************* 
#  job.properties
#*************************************************
nameNode=hdfs://<namenode-machine-fqdn>:8020
jobTracker=<resource-manager-fqdn>:8050
queueName=default
oozie.libpath=${nameNode}/user/oozie/share/lib
oozie.use.system.libpath=true
oozie.wf.rerun.failnodes=true
oozieProjectRoot=${nameNode}/user/${user.name}
appPath=${oozieProjectRoot}
oozie.wf.application.path=${appPath}
inputDir=${oozieProjectRoot}
focusNodeLogin=<username>@<remote-host-where-you-have-your-shell-script(s)>
shellScriptPath=~/uploadFile.sh
emailToAddress=<email-id>

Step 2. Write workflow.xml

Example:

<!--******************************************-->
<!--workflow.xml                              -->
<!--******************************************-->
<workflow-app name="WorkFlowForSshAction" xmlns="uri:oozie:workflow:0.1">
    <start to="sshAction"/>
    <action name="sshAction">
        <ssh xmlns="uri:oozie:ssh-action:0.1">
            <host>${focusNodeLogin}</host>
       <command>${shellScriptPath}</command>
            <capture-output/>
        </ssh>
   <ok to="sendEmail"/>
        <error to="killAction"/>
    </action>
    <action name="sendEmail">
        <email xmlns="uri:oozie:email-action:0.1">
              <to>${emailToAddress}</to>
              <subject>Output of workflow ${wf:id()}</subject>
              <body>Status of the file move: ${wf:actionData('sshAction')['STATUS']}</body>
        </email>
        <ok to="end"/>
        <error to="end"/>
    </action>
    <kill name="killAction">
        <message>"Killed job due to error"</message>
    </kill>
    <end name="end"/>
</workflow-app>

Step 3. Write sample uploadFile.sh script

Example:

#!/bin/bash
hadoop fs -put ~/test /user/<username>/uploadedbyoozie

Step 4. Upload workflow.xml to ${appPath} defined in job.properties.

Step 5. Login to Oozie host by "oozie" user.

Step 6. Generate a key pair (if you don't have already ) using 'ssh-keygen' command

Step 7. On Oozie host copy ~/.ssh/id_rsa.pub and paste it on <remote-host>'s ~/.ssh/authorized_keys file (focus node)

Step 8. Test password-less ssh from oozie@oozie-host to <username>@<remote-host>

Step 9. if step 7 succeeds then go ahead and run oozie job. it should complete without error

Note - In order to get password-less ssh working please make sure that:

1. You have 700 permissions on ~/.ssh directory

2. 600 permissions on ~/.ssh/authorized_keys file on remote-host

3. 600 to ~/.ssh/id_rsa

4. 644 to ~/.ssh/id_rsa.pub

sshimpi · ‎02-15-2016

Nice one!

rodolfo_maripan · ‎09-06-2016

Also

Make sure that curl is installed in target machine
make sure that target machine can see oozie server and able to run the curl command against it

In my case I've got the timeout issue and installed curl but didn't work so I added the FQDN in /etc/hosts and it worked perfectly well.

Cloudera Community

Community Articles

Oozie ssh action

Apache Oozie

Re: Oozie ssh action

Re: Oozie ssh action