Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Solved Go to solution
Highlighted

Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Explorer
Hi,
We have an existing solution that uses a 3rd party workflow manager and which we are now looking to replace with Oozie.
Our solution comprises lots of Python scripts that ultimately use impyla (I.e. Cloudera's Python client for Impala) to issue SQL statements to Impala. We have followed (what we believe to be) the recommended Cloudera architecture of using an edge (aka gateway) node and all of our code (Python scripts etc...) are deployed to that edge node.

Here's the problem that I think we have. According to http://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/ Oozie can only execute scripts that are stored on HDFS however as I've just explained our scripts are deployed to the file system of the edge node.

Can anyone recommend a course of action that would enable us to use Oozie to execute our scripts?

TIA
JT
1 ACCEPTED SOLUTION

Accepted Solutions

Re: Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Contributor

Hi,

 

You can use the SSH Action to execute shell scripts on specific nodes.

See http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SshActionExtension.html

 

And here's a very detailed example of using the SSH Action:

http://hadooped.blogspot.com/2013/10/apache-oozie-part-13-oozie-ssh-action_30.html

Software Engineer | Cloudera, Inc. | http://cloudera.com
2 REPLIES 2

Re: Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Contributor

Hi,

 

You can use the SSH Action to execute shell scripts on specific nodes.

See http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SshActionExtension.html

 

And here's a very detailed example of using the SSH Action:

http://hadooped.blogspot.com/2013/10/apache-oozie-part-13-oozie-ssh-action_30.html

Software Engineer | Cloudera, Inc. | http://cloudera.com

Re: Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Explorer
cool, thanks Robert
Don't have an account?
Coming from Hortonworks? Activate your account here