Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oozie: Throttling the number of forks.

Oozie: Throttling the number of forks.

New Contributor

I have a workflow.xml which is generated by one of our tool. The workflow contains a fork/join, to execute sqoop actions in parallel. What I want to archive is to limit the number of concurrent sqoop actions that can run in parallel from the fork. My questions are

1. Is there a configurations that can be used to limit/throttle maximum number of forks in parallel?

2. Is there a configuration to limit maximum number of concurrent execution of an action type (i.e. allow only 3 sqoop actions to be running at anytime in a workflow)

2 REPLIES 2

Re: Oozie: Throttling the number of forks.

Mentor

this is not Oozie specific but you can push the job to a Yarn queue with lower SLAs thus limiting resources for the user running the workflow. If you find another way, please post here.

Highlighted

Re: Oozie: Throttling the number of forks.

Mentor

@cdraper has this been resolved? Can you post your solution or accept best answer?

Don't have an account?
Coming from Hortonworks? Activate your account here