Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Time out configuration for zeppelin Job

avatar
Explorer

I am looking for some configuration where I would like to set a timeout for jobs that are been running for a long time. I have looked up documentation and could find different properties such as 

 

zeppelin.interpreter.connect.timeout

zeppelin.interpreter.lifecyclemanager.timeout.threshold

zeppelin.interpreter.lifecyclemanager.timeout.checkinterval

 

Can someone please assist me in selecting the correct property to set time out for long running jobs and please differentiate between the above configuration parameters.

 

Best Regards,

Chandu

4 REPLIES 4

avatar
Master Collaborator

Hi Chandu,

 

The 2nd and 3rd parameters you listed have to do with a newer feature of Zeppelin called Interpreter Lifecycle Management, new in 0.8.0 release. The lifecycle manager is responsible to periodically (at checkinterval) check if the session is idle, and once a certain time passes (threshold), the manager will terminate the interpreter session. Default value for the threshold is set at 1 hour. 

 

The other parameter, zeppelin.interpreter.connect.timeout, is responsible for truncating output for a given cell. If output is being continuously produced by Zeppelin interpreter and it doesn't stop after the default value of 30000 milliseconds (or 30 seconds), then Zeppelin will truncate the output right there. At least that's my understanding.

 

Hope it helps,

Alex

avatar
Explorer

Thank you Alex for information. Form  your explanation I can understand the exact usage of the properties. But is there a property which we can use to stop long running jobs automatically with out going and killing them from UI

 

 

Best Regards,

Chandu

avatar
Master Collaborator

Hi Chandu,

 

I think you need to define what you mean by "long running jobs" and also look at some ways to kill jobs outside of Zeppelin (e.g. this thread). Keep in mind that it's one thing to close a Zeppelin session, but it's another to stop, say, Spark streaming application that was launched from Zeppelin and is running on YARN cluster indefinitely. 

 

If you are running a local job in Zeppelin, then using the 2 parameters listed should be able to do the trick.

 

 

Regards,

Alex

avatar
Community Manager

@Pamarthich Has your issue been resolved? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

 

Screen Shot 2019-08-06 at 1.54.47 PM.png


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.