Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (2)
avatar
Master Mentor

Part 1: https://community.hortonworks.com/articles/82964/getting-started-with-apache-ambari-workflow-design....

Part 2: https://community.hortonworks.com/articles/82967/apache-ambari-workflow-designer-view-for-apache-oo....

Part 3: https://community.hortonworks.com/articles/82988/apache-ambari-workflow-designer-view-for-apache-oo-...

Part 4: https://community.hortonworks.com/articles/83051/apache-ambari-workflow-designer-view-for-apache-oo-...

Part 6: https://community.hortonworks.com/articles/83787/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 7: https://community.hortonworks.com/articles/84071/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 8: https://community.hortonworks.com/articles/84394/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 9: https://community.hortonworks.com/articles/85091/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 10: https://community.hortonworks.com/articles/85354/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 11: https://community.hortonworks.com/articles/85361/apache-ambari-workflow-manager-view-for-apache-ooz-...

Part 12: https://community.hortonworks.com/articles/131389/apache-ambari-workflow-manager-view-for-apache-ooz...

In this tutorial, I'm going to cover a Java action along with Decision node. I'm also going to show you how to leverage EL functions in Oozie within WFM. I'm also going to show you how to manage kill nodes for workflows.

All source code for this tutorial is available at https://github.com/dbist/oozie/tree/master/apps/java. Inspiration for this tutorial is from https://cwiki.apache.org/confluence/display/OOZIE/Java+Cookbook example 2. Problem statement for this tutorial is "This example is to illustrate how action output data captured using capture output can be used in decision nodes."

First thing we need to do is pick java action node and drop it onto canvas.

12427-01-java-node.png

In the link, java-node is called java3, I prefer my name. This is important as decision node EL function relies on the name of the node so if you decide to change it to your liking, make sure you change it in EL function as well. Same goes with the Java code, by default there is no package name, I decided to place the code into org.apache.oozie.examples package, that will affect my java node configuration which I'm going to show next.

12428-02-configure-java-node.png

Notice the difference between the Oozie wiki link and my main class, it contains full Java package name. Also, I'm going to check the capture output as I want to see what the output of the process is going to be at the end. Next we're going to configure argument for my Java, it will affect the path for decision node. Looking at the code, if yes, we expect "value1" and "value2" printed, else "novalue" for key1 and key2 respectively.

12429-03-configure-java-node-args.png

Next, looking at the Oozie XML in the wiki, I'm going to add parameter to pass YARN queue.

12430-04-configure-queue-parameter.png

Let's preview XML for good measure.

12431-06-preview-xml.png

we're pretty much done with Java node, let's switch to Decision node, no pun intended. We add a new node by clicking on the arrow following your java node and then hitting the plus sign.

12432-07-add-decision.png

This is what it looks like after adding decision node.

12434-08-preview-decision-graph.png

To configure Decision node, you click on it and then hitting the gear icon.

12436-09-configure-decision.png

For condition node, we're going to paste the EL function

${(wf:actionData('java-node')['key1'] == "value1") and (wf:actionData('java-node')['key2'] == "value2")}

notice I changed java3 to java-node to reflect the name of my java node. Next, I'd like to modify my kill node as I expect default to go to "fail" rather than kill and WFM does not know what that is. Click on the button called "kill nodes"

12437-kill-node.png

Now we're going to click on create kill node and configure it. We're going to paste our custom kill node message and name in the text boxes below.

12438-11-configure-kill-node-continued.png

Notice I can edit the existing kill nodes as well, let's hit the trash icon to delete the existing kill node.

12441-13-delete-kill-node.png

We now have only one kill node called fail. Now, I need to be honest, I had an issue here to configure decision node via WFM to configure default condition to go to fail rather than end. I ended up manually editing the XML rather than handling it within WFM, this may be a bug and I will provide feedback to engineering. Remember this is still an unreleased version of product! So what I ended up doing is editing XML, uploading and then importing the wf as I've shown in the earlier tutorial. Now everything looks good.

12445-15-after-manual-edit-to-default.png

Notice, default points to fail instead of end like below. Now is a good time to preview XML to match the wiki.

12446-16-preview-xml-1.png

12447-17-preview-xml-2.png

Notice bad syntax in EL, that's another issue we're going to address as part of bug bash :).

Since I've modified the XML manually and a big promise with WFM is to reduce syntax errors, it is a good time to validate the wf, let's click on the validate button.

12448-18-validate-wf.png

It prompts to save the wf and pass the YARN queue name, remember we added that property in Java action node. All looks good, let's submit, again we need to pass that YARN queue parameter. Also we need to remember to upload the Java jar to lib directory, once validation step is executed, java-wf directory in HDFS is created, then you just need to create a lib folder and upload compiled Java code as jar into it.

hdfs dfs -mkdir oozie/java-wf/lib 
hdfs dfs -put OozieJavaExample-1.0.0.jar oozie/java-wf/lib/

12449-19-submit-wf-default-queue.png

The workflow finished successfully. Notice four rows of wf as we have two nodes, start node and end node. All OK.

12450-20-success-action-tab.png

Let's check out the flow graph before we see job result. Notice it is all green as it was all successful.

12451-22-success-flow-graph.png

Finally, let's see the result.

12452-23-success-job-output.png

As expected, we're passing a yes and if so, decision will print key1=value1 and key2=value2.

Let's switch things up a bit and pass a no as argument. We need to modify that in the Java node again.

12453-25-configure-java-node-arg-to-no.png

Once changed, I submitted the job and it succeeded, however, we see something a bit different.

12454-26-preview-wf-result-with-no.png

Again the four rows look good and all passed. However if we look at the flow graph, color scheme is different.

12455-27-flow-graph-arg-no.png

Decision resulted in triggering default condition which was expecting to go to fail. This is what it looks like currently in that case. Finally, let's see the job results.

12456-29-job-output-with-arg-no.png

And as expected, we pass a "no" and as a result, we expect key1=novalue and key2=novalue. All done here. Until next time!

4,422 Views
Comments

I can't find OozieJavaExample-1.0.0.jar on https://github.com/dbist/oozie-examples github repo.

Sorry. I realized java code. I succeeded it.

I wanted to check in the HDFS if the file is present, based on the output i wanted to make a decision . How can i do that ?

Version history
Last update:
‎08-17-2019 02:30 PM
Updated by:
Contributors