In this tutorial, I'm going to cover a Java action along with Decision node. I'm also going to show you how to leverage EL functions in Oozie within WFM. I'm also going to show you how to manage kill nodes for workflows.
All source code for this tutorial is available at https://github.com/dbist/oozie/tree/master/apps/java. Inspiration for this tutorial is from https://cwiki.apache.org/confluence/display/OOZIE/Java+Cookbook example 2. Problem statement for this tutorial is "This example is to illustrate how action output data captured using capture output can be used in decision nodes."
First thing we need to do is pick java action node and drop it onto canvas.
In the link, java-node is called java3, I prefer my name. This is important as decision node EL function relies on the name of the node so if you decide to change it to your liking, make sure you change it in EL function as well. Same goes with the Java code, by default there is no package name, I decided to place the code into org.apache.oozie.examples package, that will affect my java node configuration which I'm going to show next.
Notice the difference between the Oozie wiki link and my main class, it contains full Java package name. Also, I'm going to check the capture output as I want to see what the output of the process is going to be at the end. Next we're going to configure argument for my Java, it will affect the path for decision node. Looking at the code, if yes, we expect "value1" and "value2" printed, else "novalue" for key1 and key2 respectively.
Next, looking at the Oozie XML in the wiki, I'm going to add parameter to pass YARN queue.
Let's preview XML for good measure.
we're pretty much done with Java node, let's switch to Decision node, no pun intended. We add a new node by clicking on the arrow following your java node and then hitting the plus sign.
This is what it looks like after adding decision node.
To configure Decision node, you click on it and then hitting the gear icon.
For condition node, we're going to paste the EL function
${(wf:actionData('java-node')['key1'] == "value1") and (wf:actionData('java-node')['key2'] == "value2")}notice I changed java3 to java-node to reflect the name of my java node. Next, I'd like to modify my kill node as I expect default to go to "fail" rather than kill and WFM does not know what that is. Click on the button called "kill nodes"
Now we're going to click on create kill node and configure it. We're going to paste our custom kill node message and name in the text boxes below.
Notice I can edit the existing kill nodes as well, let's hit the trash icon to delete the existing kill node.
We now have only one kill node called fail. Now, I need to be honest, I had an issue here to configure decision node via WFM to configure default condition to go to fail rather than end. I ended up manually editing the XML rather than handling it within WFM, this may be a bug and I will provide feedback to engineering. Remember this is still an unreleased version of product! So what I ended up doing is editing XML, uploading and then importing the wf as I've shown in the earlier tutorial. Now everything looks good.
Notice, default points to fail instead of end like below. Now is a good time to preview XML to match the wiki.
Notice bad syntax in EL, that's another issue we're going to address as part of bug bash :).
Since I've modified the XML manually and a big promise with WFM is to reduce syntax errors, it is a good time to validate the wf, let's click on the validate button.
It prompts to save the wf and pass the YARN queue name, remember we added that property in Java action node. All looks good, let's submit, again we need to pass that YARN queue parameter. Also we need to remember to upload the Java jar to lib directory, once validation step is executed, java-wf directory in HDFS is created, then you just need to create a lib folder and upload compiled Java code as jar into it.
hdfs dfs -mkdir oozie/java-wf/lib hdfs dfs -put OozieJavaExample-1.0.0.jar oozie/java-wf/lib/
The workflow finished successfully. Notice four rows of wf as we have two nodes, start node and end node. All OK.
Let's check out the flow graph before we see job result. Notice it is all green as it was all successful.
Finally, let's see the result.
As expected, we're passing a yes and if so, decision will print key1=value1 and key2=value2.
Let's switch things up a bit and pass a no as argument. We need to modify that in the Java node again.
Once changed, I submitted the job and it succeeded, however, we see something a bit different.
Again the four rows look good and all passed. However if we look at the flow graph, color scheme is different.
Decision resulted in triggering default condition which was expecting to go to fail. This is what it looks like currently in that case. Finally, let's see the job results.
And as expected, we pass a "no" and as a result, we expect key1=novalue and key2=novalue. All done here. Until next time!
