Reply
New Contributor
Posts: 2
Registered: ‎02-10-2015

Time taken between 2 jobs in Oozie

I have a Oozie workflow in which I have multiple process running one after the other one. But their is a time between the execution of the first process and the second one. Why is this delay. In a job that runs for X minutes I have a Idle state for 20% of the time which detoriated my performance.

 

Vinod

 

Highlighted
Explorer
Posts: 23
Registered: ‎02-07-2015

Re: Time taken between 2 jobs in Oozie

Can you provide further details about the workflow. Diffcicult to say anything without understanding what they are...

cheers

Cloudera Employee
Posts: 35
Registered: ‎07-08-2013

Re: Time taken between 2 jobs in Oozie

Usually if there is a delay between actions, it means that Oozie didn't receive the callback from Hadoop.  To not spam the JT/RM with job status requests, Oozie normally only queries it once every ten minutes (by default).  The reason you normally don't have to wait 10 minutes is that Oozie configures Hadoop to send back a message to Oozie over http once the job has finished.  If that callback doesn't go through for some reason, then you'll typically see a 10 minute delay between actions.

 

Can you check the launcher job logs for any error messages?  If it couldn't send the callback, it would probably say something there.

Software Engineer | Cloudera, Inc. | http://cloudera.com
Announcements