question Can I use Oozie to execute scripts stored on a edge (aka gateway) node? in Archives of Support Questions (Read Only)

Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

jamiet — Fri, 16 Sep 2022 10:08:45 GMT

Hi,
We have an existing solution that uses a 3rd party workflow manager and which we are now looking to replace with Oozie.
Our solution comprises lots of Python scripts that ultimately use impyla (I.e. Cloudera's Python client for Impala) to issue SQL statements to Impala. We have followed (what we believe to be) the recommended Cloudera architecture of using an edge (aka gateway) node and all of our code (Python scripts etc...) are deployed to that edge node.

Here's the problem that I think we have. According to http://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/ Oozie can only execute scripts that are stored on HDFS however as I've just explained our scripts are deployed to the file system of the edge node.

Can anyone recommend a course of action that would enable us to use Oozie to execute our scripts?

TIA
JT

Re: Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

Robert K — Tue, 15 Mar 2016 04:34:31 GMT

Hi,

You can use the SSH Action to execute shell scripts on specific nodes.

See http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SshActionExtension.html

And here's a very detailed example of using the SSH Action:

http://hadooped.blogspot.com/2013/10/apache-oozie-part-13-oozie-ssh-action_30.html

Re: Can I use Oozie to execute scripts stored on a edge (aka gateway) node?

jamiet — Tue, 15 Mar 2016 08:22:06 GMT

cool, thanks Robert