Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

[#1]

Hi,
We have an existing solution that uses a 3rd party workflow manager and which we are now looking to replace with Oozie.
Our solution comprises lots of Python scripts that ultimately use impyla (I.e. Cloudera's Python client for Impala) to issue SQL statements to Impala. We have followed (what we believe to be) the recommended Cloudera architecture of using an edge (aka gateway) node and all of our code (Python scripts etc...) are deployed to that edge node.

Here's the problem that I think we have. According to http://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/ Oozie can only execute scripts that are stored on HDFS however as I've just explained our scripts are deployed to the file system of the edge node.

Can anyone recommend a course of action that would enable us to use Oozie to execute our scripts?

TIA
JT

[#2 · Accepted solution]

Hi,

You can use the SSH Action to execute shell scripts on specific nodes.

See http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SshActionExtension.html

And here's a very detailed example of using the SSH Action:

http://hadooped.blogspot.com/2013/10/apache-oozie-part-13-oozie-ssh-action_30.html

[#3]

cool, thanks Robert