question Re: Spark2 running forever in zeppelin in Support Questions

Spark2 running forever in zeppelin

144675 — Sun, 18 Aug 2019 04:40:24 GMT

This spark job is running forever, and I cannot seem to stop it.

1) I have restarted spark2 interpreter within Zeppelin

2) I have restarted/Start/stopped Zeppelin from HDP

3) I have tried stopping it from jobs

In all cases it is not responsive. and still keeps on running.

How can I kill thois job ?

Re: Spark2 running forever in zeppelin

falbani — Thu, 31 May 2018 20:31:54 GMT

@Victor Usually a restart of the interpreter also kills the yarn application. Check on RM UI, there should be a yarn application running for the interpreter. If you kill the yarn application then you will probably stop this. I see code is pretty simple. To debug you should check the spark2 interpreter log and yarn application log to find out what is happening.

Re: Spark2 running forever in zeppelin

144675 — Sun, 18 Aug 2019 04:40:10 GMT

@Felix Albani These are my RM UI and Spark UI. How can I stop this from running ?

Re: Spark2 running forever in zeppelin

144675 — Thu, 31 May 2018 20:41:14 GMT

@Felix Albani to add this code always worked, when I added help(), it started running forever.

Re: Spark2 running forever in zeppelin

falbani — Thu, 31 May 2018 20:49:20 GMT

@Victor To kill it you can issue command:

yarn application -kill <Application ID>