Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Does anyone have an example of custom oozie action executor?

Mentor

I'd like to follow an example to write my own custom oozie action.

1 ACCEPTED SOLUTION

Contributor

@Artem Ervits, we use a shell script to invoke a oozie workflow. Our script polls certain folders and if there are files they will be passed to the new invoked workflow. The shell script looks something like this:

#!/bin/bash -e

for file in $(hdfs dfs -ls -R $pollfolder | grep "^-" | grep -Po "($pollfolder/[a-zA-Z]{2}_.*/[a-zA-Z]{2}_.*-[0-9]{1,}-.*.csv.gz)" | grep -vE '('$automatedfolder'|'$quarantinefolder')')
do
	oozie job -oozie $ooziebaseurl -config $jobproperties -run \
	-D file=$file \

This shell script can then be a shell action in a separate workflow that is triggered by a coordinator or can be just scheduled with cron.

*I removed the creation of the variables that also happens in this script to save some space.

View solution in original post

4 REPLIES 4

Mentor

To be more specific, we need an example of Oozie custom asynchronous action. @Saumil Mayani @Saurabh Jain @cnormile

Contributor

@Artem Ervits, we use a shell script to invoke a oozie workflow. Our script polls certain folders and if there are files they will be passed to the new invoked workflow. The shell script looks something like this:

#!/bin/bash -e

for file in $(hdfs dfs -ls -R $pollfolder | grep "^-" | grep -Po "($pollfolder/[a-zA-Z]{2}_.*/[a-zA-Z]{2}_.*-[0-9]{1,}-.*.csv.gz)" | grep -vE '('$automatedfolder'|'$quarantinefolder')')
do
	oozie job -oozie $ooziebaseurl -config $jobproperties -run \
	-D file=$file \

This shell script can then be a shell action in a separate workflow that is triggered by a coordinator or can be just scheduled with cron.

*I removed the creation of the variables that also happens in this script to save some space.

Mentor

Thank you, the amount of engineering is done to make Oozie useful is astounding

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.