Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Does anyone have an example of custom oozie action executor?

Solved Go to solution

Does anyone have an example of custom oozie action executor?

Mentor

I'd like to follow an example to write my own custom oozie action.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Does anyone have an example of custom oozie action executor?

Contributor

@Artem Ervits, we use a shell script to invoke a oozie workflow. Our script polls certain folders and if there are files they will be passed to the new invoked workflow. The shell script looks something like this:

#!/bin/bash -e

for file in $(hdfs dfs -ls -R $pollfolder | grep "^-" | grep -Po "($pollfolder/[a-zA-Z]{2}_.*/[a-zA-Z]{2}_.*-[0-9]{1,}-.*.csv.gz)" | grep -vE '('$automatedfolder'|'$quarantinefolder')')
do
	oozie job -oozie $ooziebaseurl -config $jobproperties -run \
	-D file=$file \

This shell script can then be a shell action in a separate workflow that is triggered by a coordinator or can be just scheduled with cron.

*I removed the creation of the variables that also happens in this script to save some space.

View solution in original post

4 REPLIES 4
Highlighted

Re: Does anyone have an example of custom oozie action executor?

Highlighted

Re: Does anyone have an example of custom oozie action executor?

Mentor

To be more specific, we need an example of Oozie custom asynchronous action. @Saumil Mayani @Saurabh Jain @cnormile

Highlighted

Re: Does anyone have an example of custom oozie action executor?

Contributor

@Artem Ervits, we use a shell script to invoke a oozie workflow. Our script polls certain folders and if there are files they will be passed to the new invoked workflow. The shell script looks something like this:

#!/bin/bash -e

for file in $(hdfs dfs -ls -R $pollfolder | grep "^-" | grep -Po "($pollfolder/[a-zA-Z]{2}_.*/[a-zA-Z]{2}_.*-[0-9]{1,}-.*.csv.gz)" | grep -vE '('$automatedfolder'|'$quarantinefolder')')
do
	oozie job -oozie $ooziebaseurl -config $jobproperties -run \
	-D file=$file \

This shell script can then be a shell action in a separate workflow that is triggered by a coordinator or can be just scheduled with cron.

*I removed the creation of the variables that also happens in this script to save some space.

View solution in original post

Highlighted

Re: Does anyone have an example of custom oozie action executor?

Mentor

Thank you, the amount of engineering is done to make Oozie useful is astounding

Don't have an account?
Coming from Hortonworks? Activate your account here