Reply
Explorer
Posts: 18
Registered: ‎02-22-2016
Accepted Solution

Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Hi,
We have an existing solution that uses a 3rd party workflow manager and which we are now looking to replace with Oozie.
Our solution comprises lots of Python scripts that ultimately use impyla (I.e. Cloudera's Python client for Impala) to issue SQL statements to Impala. We have followed (what we believe to be) the recommended Cloudera architecture of using an edge (aka gateway) node and all of our code (Python scripts etc...) are deployed to that edge node.

Here's the problem that I think we have. According to http://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/ Oozie can only execute scripts that are stored on HDFS however as I've just explained our scripts are deployed to the file system of the edge node.

Can anyone recommend a course of action that would enable us to use Oozie to execute our scripts?

TIA
JT
Highlighted
Cloudera Employee
Posts: 35
Registered: ‎07-08-2013

Re: Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Hi,

 

You can use the SSH Action to execute shell scripts on specific nodes.

See http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SshActionExtension.html

 

And here's a very detailed example of using the SSH Action:

http://hadooped.blogspot.com/2013/10/apache-oozie-part-13-oozie-ssh-action_30.html

Software Engineer | Cloudera, Inc. | http://cloudera.com
Explorer
Posts: 18
Registered: ‎02-22-2016

Re: Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

cool, thanks Robert
Announcements