Support Questions

Find answers, ask questions, and share your expertise

oozie shell action

avatar
Rising Star

I created a similar job as in : https://community.hortonworks.com/content/supportkb/48985/how-to-setup-oozie-shell-action.html

On execution the yarn log shows as success.

But I can't see the /tmp/output file being created anywhere. I checked on the local linux host as well as hdfs.

One question I have is in job.properties -

oozie.coord.application.path=${nameNode}/user/root/apps/shell


Should the configuration files (job.properties, coordinator.xml etc) reside in the above directory? I have them there.

Not sure what is happening.

Appreciate the insights.

1 ACCEPTED SOLUTION

avatar
Master Guru

@n c

job.properties -> Needs to be on local filesystem on Oozie client from where you wish to submit the oozie job -run command. No need to have this file on HDFS.

workflow.xml -> This needs to be on HDFS.

script -> This needs to be on HDFS.


Regarding your question on python script - Please refer below article for how to create coordinator ( Please ignore input event part )

https://community.hortonworks.com/articles/27497/oozie-coordinator-and-based-on-input-data-events.ht...


in coordinator.xml you mention "<app-path>${workflowAppUri}</app-path>" that's what is location of workflow.xml


Please use Ambari workflow manager to design coordinators and workflows with easy WebUI.


Hope this helps.


Please let me know if you have any questions. Please accept my answer if it was helpful. 🙂

View solution in original post

12 REPLIES 12

avatar
Rising Star

In the yarn log for the job I can see :


=================================================================

>>> Invoking Shell command line now >>

Exit code of the Shell command 0
<<< Invocation of Shell command completed <<<


<<< Invocation of Main class completed <<<


Oozie Launcher, capturing output data:
=======================

avatar
Master Guru

Hi @n c

Can you please check on the node manager where this launcher was run, ideally this file should be created locally on the node manager where your shell script was run.

Please do let me know if you need any further help.

avatar
Rising Star

@Kuldeep Kulkarni , any insights pls? Appreciate the help.

avatar
Rising Star

>this file should be created locally on the node manager where your shell script was run

I have the file locally on a node. It is on this node that I execute the oozie command.

Are you saying that the shell script should be in hdfs? And that yarn will execute this shell script on the node manager on which this job was run? Rather than on the host from which I run the oozie command?

Appreciate the feedback.

avatar
Rising Star

yes, I checked the nodes and found the output in one of them. I reran to make sure.

So the script is not really needed on the Linux box? all of them - job.properties, the shellscript, workflow.xml, coordinator.xml? They need to be only in hdfs?

Also next, how to execute a python code from oozie? Also I want it to run daily.

Appreciate the insights.


avatar
Rising Star

@Kuldeep Kulkarni, I created python action based on https://community.hortonworks.com/content/supportkb/151119/how-to-run-a-python-script-using-oozie-sh...

But how do I integrate coordinator.xml with that?

I tried creating the file but it is not executing as per that.

Is there somewhere in job.properties or workflow.xml that you mention coordinator.xml?

Appreciate the feedback.

avatar
Master Guru

@n c

job.properties -> Needs to be on local filesystem on Oozie client from where you wish to submit the oozie job -run command. No need to have this file on HDFS.

workflow.xml -> This needs to be on HDFS.

script -> This needs to be on HDFS.


Regarding your question on python script - Please refer below article for how to create coordinator ( Please ignore input event part )

https://community.hortonworks.com/articles/27497/oozie-coordinator-and-based-on-input-data-events.ht...


in coordinator.xml you mention "<app-path>${workflowAppUri}</app-path>" that's what is location of workflow.xml


Please use Ambari workflow manager to design coordinators and workflows with easy WebUI.


Hope this helps.


Please let me know if you have any questions. Please accept my answer if it was helpful. 🙂

avatar
Rising Star

Hi Kuldeep, thanks so much for the clarification.

I will try to do as per your instructions and let you know how it went.

Thanks again.

avatar
Rising Star

BTW now I use cloudera. So I guess it has to be manual. Thanks.