Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Execute python3 script every minute in workflow manager

Highlighted

Execute python3 script every minute in workflow manager

Explorer

I am trying to execute a python script every minute.

I navigate to workflow manager -> create -> coordinator

The workflow path is the python file, the frequency is minutes 1.
I then choose to validate, and I have to put in the coordinator path.

What is the coordinator path in this case ? And are these the correct steps to execute a python file every minute ?

76569-1.png

76570-2.png

5 REPLIES 5
Highlighted

Re: Execute python3 script every minute in workflow manager

@Victor

Workflow path should point to the actual workflow.xml that contains the list of actions you are performing. In your case this workflow.xml should contain a bash action to call the python script.

Coordinator path is the path to the coordinator.xml that contains the information related to the frequency of execution and reference to the workflow.xml

I recommend you take time to go over this HC article that guides you step by step on how to use the workflow view:

https://community.hortonworks.com/articles/82964/getting-started-with-apache-ambari-workflow-design....

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

Highlighted

Re: Execute python3 script every minute in workflow manager

@Victor Did the above helped you with the issue. Keep me posted! Thanks

Highlighted

Re: Execute python3 script every minute in workflow manager

Explorer

Hi @Felix Albani . It does not work, here is a visit of the steps.

I created a new folder on HDFS and added the python file in there. The location is

user -> admin -> v5_script -> scripting.py

1) Create Workflow -> Shell - > scripting.py (exec) and the location, below attached screen.

77618-o1.png

77619-o2.png

Then I validated my workflow as follows

77620-oz3.png

Then I submit the workflow

77621-oz4.png

My python file head is

77622-oz7.png

The errors are as following "error=2, No such file or directory"

2018-06-08 09:19:07,998  WARN ShellActionExecutor:523 - SERVER[sandbox-hdp.hortonworks.com] USER[admin] GROUP[-] TOKEN[] APP[v5_workflow] JOB[0000869-180604152743470-oozie-oozi-W] ACTION[0000869-180604152743470-oozie-oozi-W@shell_1] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.ShellMain], main() threw exception, Cannot run program "scripting.py" (in directory "/hadoop/yarn/local/usercache/admin/appcache/application_1528126107293_0867/container_1528126107293_0867_01_000002"): error=2, No such file or directory
2018-06-08 09:19:08,006  WARN ShellActionExecutor:523 - SERVER[sandbox-hdp.hortonworks.com] USER[admin] GROUP[-] TOKEN[] APP[v5_workflow] JOB[0000869-180604152743470-oozie-oozi-W] ACTION[0000869-180604152743470-oozie-oozi-W@shell_1] Launcher exception: Cannot run program "scripting.py" (in directory "/hadoop/yarn/local/usercache/admin/appcache/application_1528126107293_0867/container_1528126107293_0867_01_000002"): error=2, No such file or directory
java.io.IOException: Cannot run program "scripting.py" (in directory "/hadoop/yarn/local/usercache/admin/appcache/application_1528126107293_0867/container_1528126107293_0867_01_000002"): error=2, No such file or directory

Re: Execute python3 script every minute in workflow manager

Explorer
@Felix Albani

My XML file is as follows

77624-xmlfile.png

Highlighted

Re: Execute python3 script every minute in workflow manager

Explorer

The permission of the files are as follows @Felix Albani

77626-permis.png

Don't have an account?
Coming from Hortonworks? Activate your account here