Spark2 running forever in zeppelin

[#1]

This spark job is running forever, and I cannot seem to stop it.

1) I have restarted spark2 interpreter within Zeppelin

2) I have restarted/Start/stopped Zeppelin from HDP

3) I have tried stopping it from jobs

In all cases it is not responsive. and still keeps on running.

How can I kill thois job ?

[#2 · Accepted solution]

@Victor Usually a restart of the interpreter also kills the yarn application. Check on RM UI, there should be a yarn application running for the interpreter. If you kill the yarn application then you will probably stop this. I see code is pretty simple. To debug you should check the spark2 interpreter log and yarn application log to find out what is happening.

[#3]

@Felix Albani These are my RM UI and Spark UI. How can I stop this from running ?

[#4]

@Victor To kill it you can issue command:

yarn application -kill <Application ID>

[#5]

@Felix Albani to add this code always worked, when I added help(), it started running forever.