Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (2)
avatar
Master Mentor

This is the third in the series of articles on WFM.

Part 1: https://community.hortonworks.com/articles/82964/getting-started-with-apache-ambari-workflow-design....

Part 2: https://community.hortonworks.com/articles/82967/apache-ambari-workflow-designer-view-for-apache-oo....

Part 4: https://community.hortonworks.com/articles/83051/apache-ambari-workflow-designer-view-for-apache-oo-...

Part 5: https://community.hortonworks.com/articles/83361/apache-ambari-workflow-manager-view-for-apache-ooz....

Part 6: https://community.hortonworks.com/articles/83787/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 7: https://community.hortonworks.com/articles/84071/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 8: https://community.hortonworks.com/articles/84394/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 9: https://community.hortonworks.com/articles/85091/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 10: https://community.hortonworks.com/articles/85354/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 11: https://community.hortonworks.com/articles/85361/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 12: https://community.hortonworks.com/articles/131389/apache-ambari-workflow-manager-view-for-apache-ooz...

In this tutorial, I will import an existing Python3 workflow and modify it to work with WFM. You need to make sure that Python3 exists on every node where YARN nodemanager is installed and Oozie is allowed to execute on that node (in essence, you want to make sure there are no YARN queue limitations on nodes with Python3 installed). Installing Python3 is beyond the scope of this tutorial, I am using Centos 6 and I followed this tutorial http://ask.xmodulo.com/install-python3-centos.html

Once Python3 is deployed across all of your nodemanagers, you can import the workflow in the same way I've shown before and configure the python3-node to your liking

12345-20.png

Nothing is out of the ordinary yet.

12346-21.png

We learned the mistake in our previous tutorial and let WFM assign Resource Manager property rather than using inherited $jobTracker. I am ready to submit, notice I left the inherited queue property in the wf and WFM prompts me to input it.

12347-23.png

On submission, we can navigate to the dashboard to track the status.

12348-24.png

My job succeeded, I want to look at the result.

12349-25.png

I'm going to click on the arrow on the right and navigate to the YARN job. Deep in the logs, I can find my desired output

12350-26.png

My python3 code by the way is below

#! /usr/bin/env /usr/local/bin/python3
import os, pwd, sys
print("who am I? " + pwd.getpwuid(os.getuid())[0])
print("this is a Python script")
print("Python Interpreter Version: " + sys.version)

As usual, my repo has more samples you can play around with https://github.com/dbist/oozie

4,843 Views
Comments

@Artem Ervits How to add the coordinator to the workflow so the script executes every minute for infinite time ?

Version history
Last update:
‎08-17-2019 04:52 AM
Updated by:
Contributors