question Re: Does anyone have an example of custom oozie action executor? in Support Questions

Does anyone have an example of custom oozie action executor?

aervits — Fri, 13 May 2016 03:04:03 GMT

I'd like to follow an example to write my own custom oozie action.

Re: Does anyone have an example of custom oozie action executor?

jyadav — Fri, 13 May 2016 03:07:30 GMT

@Artem Ervits

http://www.infoq.com/articles/ExtendingOozie

Re: Does anyone have an example of custom oozie action executor?

aervits — Fri, 08 Jul 2016 21:02:30 GMT

To be more specific, we need an example of Oozie custom asynchronous action. @Saumil Mayani @Saurabh Jain @cnormile

Re: Does anyone have an example of custom oozie action executor?

rene_sluiter — Thu, 21 Jul 2016 19:21:30 GMT

@Artem Ervits, we use a shell script to invoke a oozie workflow. Our script polls certain folders and if there are files they will be passed to the new invoked workflow. The shell script looks something like this:

#!/bin/bash -e

for file in $(hdfs dfs -ls -R $pollfolder | grep "^-" | grep -Po "($pollfolder/[a-zA-Z]{2}_.*/[a-zA-Z]{2}_.*-[0-9]{1,}-.*.csv.gz)" | grep -vE '('$automatedfolder'|'$quarantinefolder')')
do
	oozie job -oozie $ooziebaseurl -config $jobproperties -run \
	-D file=$file \

This shell script can then be a shell action in a separate workflow that is triggered by a coordinator or can be just scheduled with cron.

*I removed the creation of the variables that also happens in this script to save some space.

Re: Does anyone have an example of custom oozie action executor?

aervits — Thu, 21 Jul 2016 19:29:09 GMT

Thank you, the amount of engineering is done to make Oozie useful is astounding