question Re: Time out configuration for zeppelin Job in Support Questions

Time out configuration for zeppelin Job

Pamarthich — Tue, 25 May 2021 18:28:57 GMT

I am looking for some configuration where I would like to set a timeout for jobs that are been running for a long time. I have looked up documentation and could find different properties such as

zeppelin.interpreter.connect.timeout

zeppelin.interpreter.lifecyclemanager.timeout.threshold

zeppelin.interpreter.lifecyclemanager.timeout.checkinterval

Can someone please assist me in selecting the correct property to set time out for long running jobs and please differentiate between the above configuration parameters.

Best Regards,

Chandu

Re: Time out configuration for zeppelin Job

aakulov — Tue, 25 May 2021 19:47:33 GMT

Hi Chandu,

The 2nd and 3rd parameters you listed have to do with a newer feature of Zeppelin called Interpreter Lifecycle Management, new in 0.8.0 release. The lifecycle manager is responsible to periodically (at checkinterval) check if the session is idle, and once a certain time passes (threshold), the manager will terminate the interpreter session. Default value for the threshold is set at 1 hour.

The other parameter, zeppelin.interpreter.connect.timeout, is responsible for truncating output for a given cell. If output is being continuously produced by Zeppelin interpreter and it doesn't stop after the default value of 30000 milliseconds (or 30 seconds), then Zeppelin will truncate the output right there. At least that's my understanding.

Hope it helps,

Alex

Re: Time out configuration for zeppelin Job

Pamarthich — Thu, 27 May 2021 18:43:57 GMT

Thank you Alex for information. Form your explanation I can understand the exact usage of the properties. But is there a property which we can use to stop long running jobs automatically with out going and killing them from UI

Best Regards,

Chandu

Re: Time out configuration for zeppelin Job

aakulov — Thu, 27 May 2021 18:59:38 GMT

Hi Chandu,

I think you need to define what you mean by "long running jobs" and also look at some ways to kill jobs outside of Zeppelin (e.g. this thread). Keep in mind that it's one thing to close a Zeppelin session, but it's another to stop, say, Spark streaming application that was launched from Zeppelin and is running on YARN cluster indefinitely.

If you are running a local job in Zeppelin, then using the 2 parameters listed should be able to do the trick.

Regards,

Alex

Re: Time out configuration for zeppelin Job

cjervis — Thu, 03 Jun 2021 12:30:02 GMT

@Pamarthich Has your issue been resolved? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.