Support Questions

Find answers, ask questions, and share your expertise

Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Explorer
Hi,
We have an existing solution that uses a 3rd party workflow manager and which we are now looking to replace with Oozie.
Our solution comprises lots of Python scripts that ultimately use impyla (I.e. Cloudera's Python client for Impala) to issue SQL statements to Impala. We have followed (what we believe to be) the recommended Cloudera architecture of using an edge (aka gateway) node and all of our code (Python scripts etc...) are deployed to that edge node.

Here's the problem that I think we have. According to http://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/ Oozie can only execute scripts that are stored on HDFS however as I've just explained our scripts are deployed to the file system of the edge node.

Can anyone recommend a course of action that would enable us to use Oozie to execute our scripts?

TIA
JT
1 ACCEPTED SOLUTION

Contributor

Hi,

 

You can use the SSH Action to execute shell scripts on specific nodes.

See http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SshActionExtension.html

 

And here's a very detailed example of using the SSH Action:

http://hadooped.blogspot.com/2013/10/apache-oozie-part-13-oozie-ssh-action_30.html

Software Engineer | Cloudera, Inc. | http://cloudera.com

View solution in original post

2 REPLIES 2

Contributor

Hi,

 

You can use the SSH Action to execute shell scripts on specific nodes.

See http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SshActionExtension.html

 

And here's a very detailed example of using the SSH Action:

http://hadooped.blogspot.com/2013/10/apache-oozie-part-13-oozie-ssh-action_30.html

Software Engineer | Cloudera, Inc. | http://cloudera.com

Explorer
cool, thanks Robert
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.