Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oozie shell action

Solved Go to solution

oozie shell action

Explorer

I created a similar job as in : https://community.hortonworks.com/content/supportkb/48985/how-to-setup-oozie-shell-action.html

On execution the yarn log shows as success.

But I can't see the /tmp/output file being created anywhere. I checked on the local linux host as well as hdfs.

One question I have is in job.properties -

oozie.coord.application.path=${nameNode}/user/root/apps/shell


Should the configuration files (job.properties, coordinator.xml etc) reside in the above directory? I have them there.

Not sure what is happening.

Appreciate the insights.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: oozie shell action

Super Guru

@n c

job.properties -> Needs to be on local filesystem on Oozie client from where you wish to submit the oozie job -run command. No need to have this file on HDFS.

workflow.xml -> This needs to be on HDFS.

script -> This needs to be on HDFS.


Regarding your question on python script - Please refer below article for how to create coordinator ( Please ignore input event part )

https://community.hortonworks.com/articles/27497/oozie-coordinator-and-based-on-input-data-events.ht...


in coordinator.xml you mention "<app-path>${workflowAppUri}</app-path>" that's what is location of workflow.xml


Please use Ambari workflow manager to design coordinators and workflows with easy WebUI.


Hope this helps.


Please let me know if you have any questions. Please accept my answer if it was helpful. :)

12 REPLIES 12

Re: oozie shell action

Explorer

In the yarn log for the job I can see :


=================================================================

>>> Invoking Shell command line now >>

Exit code of the Shell command 0
<<< Invocation of Shell command completed <<<


<<< Invocation of Main class completed <<<


Oozie Launcher, capturing output data:
=======================

Re: oozie shell action

Super Guru

Hi @n c

Can you please check on the node manager where this launcher was run, ideally this file should be created locally on the node manager where your shell script was run.

Please do let me know if you need any further help.

Re: oozie shell action

Explorer

@Kuldeep Kulkarni , any insights pls? Appreciate the help.

Re: oozie shell action

Explorer

>this file should be created locally on the node manager where your shell script was run

I have the file locally on a node. It is on this node that I execute the oozie command.

Are you saying that the shell script should be in hdfs? And that yarn will execute this shell script on the node manager on which this job was run? Rather than on the host from which I run the oozie command?

Appreciate the feedback.

Re: oozie shell action

Explorer

yes, I checked the nodes and found the output in one of them. I reran to make sure.

So the script is not really needed on the Linux box? all of them - job.properties, the shellscript, workflow.xml, coordinator.xml? They need to be only in hdfs?

Also next, how to execute a python code from oozie? Also I want it to run daily.

Appreciate the insights.


Re: oozie shell action

Explorer

@Kuldeep Kulkarni, I created python action based on https://community.hortonworks.com/content/supportkb/151119/how-to-run-a-python-script-using-oozie-sh...

But how do I integrate coordinator.xml with that?

I tried creating the file but it is not executing as per that.

Is there somewhere in job.properties or workflow.xml that you mention coordinator.xml?

Appreciate the feedback.

Re: oozie shell action

Super Guru

@n c

job.properties -> Needs to be on local filesystem on Oozie client from where you wish to submit the oozie job -run command. No need to have this file on HDFS.

workflow.xml -> This needs to be on HDFS.

script -> This needs to be on HDFS.


Regarding your question on python script - Please refer below article for how to create coordinator ( Please ignore input event part )

https://community.hortonworks.com/articles/27497/oozie-coordinator-and-based-on-input-data-events.ht...


in coordinator.xml you mention "<app-path>${workflowAppUri}</app-path>" that's what is location of workflow.xml


Please use Ambari workflow manager to design coordinators and workflows with easy WebUI.


Hope this helps.


Please let me know if you have any questions. Please accept my answer if it was helpful. :)

Re: oozie shell action

Explorer

Hi Kuldeep, thanks so much for the clarification.

I will try to do as per your instructions and let you know how it went.

Thanks again.

Re: oozie shell action

Explorer

BTW now I use cloudera. So I guess it has to be manual. Thanks.