Skip to main content
Pamarthich
Pamarthich
Silver ⭐️
May 25, 2021
Question

Time out configuration for zeppelin Job

I am looking for some configuration where I would like to set a timeout for jobs that are been running for a long time. I have looked up documentation and could find different properties such as 

 

zeppelin.interpreter.connect.timeout

zeppelin.interpreter.lifecyclemanager.timeout.threshold

zeppelin.interpreter.lifecyclemanager.timeout.checkinterval

 

Can someone please assist me in selecting the correct property to set time out for long running jobs and please differentiate between the above configuration parameters.

 

Best Regards,

Chandu

4 replies

aakulov
Platinum ⭐️
aakulov
Platinum ⭐️
May 25, 2021

Hi Chandu,

 

The 2nd and 3rd parameters you listed have to do with a newer feature of Zeppelin called Interpreter Lifecycle Management, new in 0.8.0 release. The lifecycle manager is responsible to periodically (at checkinterval) check if the session is idle, and once a certain time passes (threshold), the manager will terminate the interpreter session. Default value for the threshold is set at 1 hour. 

 

The other parameter, zeppelin.interpreter.connect.timeout, is responsible for truncating output for a given cell. If output is being continuously produced by Zeppelin interpreter and it doesn't stop after the default value of 30000 milliseconds (or 30 seconds), then Zeppelin will truncate the output right there. At least that's my understanding.

 

Hope it helps,

Alex

Translate
Pamarthich
PamarthichAuthor
Silver ⭐️
May 27, 2021

Thank you Alex for information. Form  your explanation I can understand the exact usage of the properties. But is there a property which we can use to stop long running jobs automatically with out going and killing them from UI

 

 

Best Regards,

Chandu

Translate
aakulov
Platinum ⭐️
aakulov
Platinum ⭐️
May 27, 2021

Hi Chandu,

 

I think you need to define what you mean by "long running jobs" and also look at some ways to kill jobs outside of Zeppelin (e.g. this thread). Keep in mind that it's one thing to close a Zeppelin session, but it's another to stop, say, Spark streaming application that was launched from Zeppelin and is running on YARN cluster indefinitely. 

 

If you are running a local job in Zeppelin, then using the 2 parameters listed should be able to do the trick.

 

 

Regards,

Alex

Translate
Terms & ConditionsAccessibility statement