Created on 02-22-2017 09:35 PM - edited 08-17-2019 02:11 PM
In this tutorial, I will demonstrate an awesome feature of WFM to publish and import assets to and from HDFS for easy reuse. What it means is any user on the WFM View node can author an action node and publish it for later user by him/her or another colleague. This allows for collaboration and action repurpose. Consider you author an action node that contains long complicated URL strings or whatnot, save it to HDFS or local database and off you go. Yes, you can publish assets to HDFS as well as local Ambari database. I'll demonstrate HDFS but steps are identical for database. Publishing to HDFS has a benefit of sharing across entire organization rather than single Ambari Views instance.
Let's start with creating an email action.
Fill out your email address, subject and body. At this point you can submit and see if it works, in my case it worked and I received an email. At this point I'd like to publish this action node to HDFS.
It is the last button on the right, you can see that there are two buttons to publish and to import, those are for database and HDFS respectively. Since I've chosen HDFS, I need to supply path.
WFM saves assets with .wfasset extension, feel free to share with a global directory.
[centos@aervits-hdp0 ~]$ hdfs dfs -ls assets/ Found 1 items -rw-r--r-- 3 centos hdfs 212 2017-02-22 20:56 assets/asset.wfasset
Let's now create a new wf where I'm going to execute a Shell action and then reuse this email action to send myself an email when job successfully finishes.
Once finished with shell node, let's click on the arrow between shell and end node and click plus to add another action node. In this case, we're going to hit on import asset from HDFS instead of any of the available actions.
Then go ahead and enter the HDFS path where asset is located. Once done, change action node names to something that makes sense.
Finally, you can execute the wf.
Another reason to rename action nodes to meaningful names is ability to identify them quickly in Oozie/WFM dashboard. Notice the Transition column.
Finally, let's look at job result
and what do you know? I received an email.
This is by no means the best way to send email on job result, I think handling job result via email is best handled via decision node or via kill node. This example is meant to demonstrate flexibility of WFM and also show something that is not available in Oozie itself. This is one of many features that will separate Oozie from WFM going forward. Until next time!
Created on 02-23-2017 01:13 PM
Thx for some nice articles. At the moment I am using Oozie View from HDP 2.5. There is many things which don't work which is very frustrating.
Your screenshots is indicating that you are using Hue or something else. Could you tell me which tool you are using?
https://oozie.apache.org/docs/4.2.0/WorkflowFunctionalSpec.html
Created on 02-23-2017 01:59 PM
@Anders Boje when you say you're using Oozie view from 2.5, are you referring to Workflow Manager View in Ambari 2.4.2? http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-views/content/ch_workflow_designer...
If so, I am using the same tool only from Ambari 2.5 (unreleased version). This is certainly not Hue. You just have to wait for it to be released soon as you pointed out we are still working out through bugs and usability issues.
Created on 02-23-2017 02:09 PM - edited 08-17-2019 02:10 PM
Thx for your reply.
I am using HDP 2.5 out of the box. Which I guess is Ambari 2.4.2.
I am having a hard time to found out what kind of upgrades is included in Ambari 2.5. Do you have a link and a idea when Ambari 2.5 is going to be released?
Created on 02-23-2017 02:29 PM
@Anders Boje yes that is a Workflow Manager view but an old version which is not mean to be production ready. Workflow Manager View AKA Oozie View will be available in Ambari 2.5. Ambari 2.5 is not released yet, it will be released in a month or two. As you noted, there are bugs and we are still working through fixing some issues so it is stable. My tutorials do cover a lot of WFM but the ideas can be applied to working with Oozie XML. You should just wait for Ambari 2.5 release if this is more what you want to do.