oozie shell action

[#1]

I created a similar job as in : https://community.hortonworks.com/content/supportkb/48985/how-to-setup-oozie-shell-action.html

On execution the yarn log shows as success.

But I can't see the /tmp/output file being created anywhere. I checked on the local linux host as well as hdfs.

One question I have is in job.properties -

oozie.coord.application.path=${nameNode}/user/root/apps/shell

Should the configuration files (job.properties, coordinator.xml etc) reside in the above directory? I have them there.

Not sure what is happening.

Appreciate the insights.

[#2]

In the yarn log for the job I can see :

=================================================================

>>> Invoking Shell command line now >>

Exit code of the Shell command 0
<<< Invocation of Shell command completed <<<

<<< Invocation of Main class completed <<<

Oozie Launcher, capturing output data:
=======================

[#3]

Hi @n c

Can you please check on the node manager where this launcher was run, ideally this file should be created locally on the node manager where your shell script was run.

Please do let me know if you need any further help.

[#4]

@Kuldeep Kulkarni , any insights pls? Appreciate the help.

[#5]

>this file should be created locally on the node manager where your shell script was run

I have the file locally on a node. It is on this node that I execute the oozie command.

Are you saying that the shell script should be in hdfs? And that yarn will execute this shell script on the node manager on which this job was run? Rather than on the host from which I run the oozie command?

Appreciate the feedback.

[#6]

yes, I checked the nodes and found the output in one of them. I reran to make sure.

So the script is not really needed on the Linux box? all of them - job.properties, the shellscript, workflow.xml, coordinator.xml? They need to be only in hdfs?

Also next, how to execute a python code from oozie? Also I want it to run daily.

Appreciate the insights.

[#7]

@Kuldeep Kulkarni, I created python action based on https://community.hortonworks.com/content/supportkb/151119/how-to-run-a-python-script-using-oozie-sh...

But how do I integrate coordinator.xml with that?

I tried creating the file but it is not executing as per that.

Is there somewhere in job.properties or workflow.xml that you mention coordinator.xml?

Appreciate the feedback.

[#8 · Accepted solution]

@n c

job.properties -> Needs to be on local filesystem on Oozie client from where you wish to submit the oozie job -run command. No need to have this file on HDFS.

workflow.xml -> This needs to be on HDFS.

script -> This needs to be on HDFS.

Regarding your question on python script - Please refer below article for how to create coordinator ( Please ignore input event part )

https://community.hortonworks.com/articles/27497/oozie-coordinator-and-based-on-input-data-events.ht...

in coordinator.xml you mention "<app-path>${workflowAppUri}</app-path>" that's what is location of workflow.xml

Please use Ambari workflow manager to design coordinators and workflows with easy WebUI.

Hope this helps.

Please let me know if you have any questions. Please accept my answer if it was helpful. 🙂

[#9]

Hi Kuldeep, thanks so much for the clarification.

I will try to do as per your instructions and let you know how it went.

Thanks again.

[#10]

BTW now I use cloudera. So I guess it has to be manual. Thanks.