Support Questions
Find answers, ask questions, and share your expertise

Spark2 running forever in zeppelin

Explorer

This spark job is running forever, and I cannot seem to stop it.

1) I have restarted spark2 interpreter within Zeppelin

2) I have restarted/Start/stopped Zeppelin from HDP

3) I have tried stopping it from jobs

In all cases it is not responsive. and still keeps on running.

How can I kill thois job ?

76526-2.png

76525-1.png

1 ACCEPTED SOLUTION

Accepted Solutions

@Victor Usually a restart of the interpreter also kills the yarn application. Check on RM UI, there should be a yarn application running for the interpreter. If you kill the yarn application then you will probably stop this. I see code is pretty simple. To debug you should check the spark2 interpreter log and yarn application log to find out what is happening.

View solution in original post

4 REPLIES 4

@Victor Usually a restart of the interpreter also kills the yarn application. Check on RM UI, there should be a yarn application running for the interpreter. If you kill the yarn application then you will probably stop this. I see code is pretty simple. To debug you should check the spark2 interpreter log and yarn application log to find out what is happening.

View solution in original post

Explorer

@Felix Albani These are my RM UI and Spark UI. How can I stop this from running ?

76528-1.png

76529-2.png

@Victor To kill it you can issue command:

yarn application -kill <Application ID>

Explorer

@Felix Albani to add this code always worked, when I added help(), it started running forever.