Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Time taken between 2 jobs in Oozie

Time taken between 2 jobs in Oozie

New Contributor

I have a Oozie workflow in which I have multiple process running one after the other one. But their is a time between the execution of the first process and the second one. Why is this delay. In a job that runs for X minutes I have a Idle state for 20% of the time which detoriated my performance.

 

Vinod

 

2 REPLIES 2
Highlighted

Re: Time taken between 2 jobs in Oozie

Explorer

Can you provide further details about the workflow. Diffcicult to say anything without understanding what they are...

cheers

Re: Time taken between 2 jobs in Oozie

Contributor

Usually if there is a delay between actions, it means that Oozie didn't receive the callback from Hadoop.  To not spam the JT/RM with job status requests, Oozie normally only queries it once every ten minutes (by default).  The reason you normally don't have to wait 10 minutes is that Oozie configures Hadoop to send back a message to Oozie over http once the job has finished.  If that callback doesn't go through for some reason, then you'll typically see a 10 minute delay between actions.

 

Can you check the launcher job logs for any error messages?  If it couldn't send the callback, it would probably say something there.

Software Engineer | Cloudera, Inc. | http://cloudera.com