Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Does anyone have an example of custom oozie action executor?

avatar
Master Mentor

I'd like to follow an example to write my own custom oozie action.

1 ACCEPTED SOLUTION

avatar
Rising Star

@Artem Ervits, we use a shell script to invoke a oozie workflow. Our script polls certain folders and if there are files they will be passed to the new invoked workflow. The shell script looks something like this:

#!/bin/bash -e

for file in $(hdfs dfs -ls -R $pollfolder | grep "^-" | grep -Po "($pollfolder/[a-zA-Z]{2}_.*/[a-zA-Z]{2}_.*-[0-9]{1,}-.*.csv.gz)" | grep -vE '('$automatedfolder'|'$quarantinefolder')')
do
	oozie job -oozie $ooziebaseurl -config $jobproperties -run \
	-D file=$file \

This shell script can then be a shell action in a separate workflow that is triggered by a coordinator or can be just scheduled with cron.

*I removed the creation of the variables that also happens in this script to save some space.

View solution in original post

4 REPLIES 4

avatar
Super Guru

avatar
Master Mentor

To be more specific, we need an example of Oozie custom asynchronous action. @Saumil Mayani @Saurabh Jain @cnormile

avatar
Rising Star

@Artem Ervits, we use a shell script to invoke a oozie workflow. Our script polls certain folders and if there are files they will be passed to the new invoked workflow. The shell script looks something like this:

#!/bin/bash -e

for file in $(hdfs dfs -ls -R $pollfolder | grep "^-" | grep -Po "($pollfolder/[a-zA-Z]{2}_.*/[a-zA-Z]{2}_.*-[0-9]{1,}-.*.csv.gz)" | grep -vE '('$automatedfolder'|'$quarantinefolder')')
do
	oozie job -oozie $ooziebaseurl -config $jobproperties -run \
	-D file=$file \

This shell script can then be a shell action in a separate workflow that is triggered by a coordinator or can be just scheduled with cron.

*I removed the creation of the variables that also happens in this script to save some space.

avatar
Master Mentor

Thank you, the amount of engineering is done to make Oozie useful is astounding