Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark2 running forever in zeppelin

Solved Go to solution
Highlighted

Spark2 running forever in zeppelin

Explorer

This spark job is running forever, and I cannot seem to stop it.

1) I have restarted spark2 interpreter within Zeppelin

2) I have restarted/Start/stopped Zeppelin from HDP

3) I have tried stopping it from jobs

In all cases it is not responsive. and still keeps on running.

How can I kill thois job ?

76526-2.png

76525-1.png

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Spark2 running forever in zeppelin

@Victor Usually a restart of the interpreter also kills the yarn application. Check on RM UI, there should be a yarn application running for the interpreter. If you kill the yarn application then you will probably stop this. I see code is pretty simple. To debug you should check the spark2 interpreter log and yarn application log to find out what is happening.

View solution in original post

4 REPLIES 4
Highlighted

Re: Spark2 running forever in zeppelin

@Victor Usually a restart of the interpreter also kills the yarn application. Check on RM UI, there should be a yarn application running for the interpreter. If you kill the yarn application then you will probably stop this. I see code is pretty simple. To debug you should check the spark2 interpreter log and yarn application log to find out what is happening.

View solution in original post

Re: Spark2 running forever in zeppelin

Explorer

@Felix Albani These are my RM UI and Spark UI. How can I stop this from running ?

76528-1.png

76529-2.png

Highlighted

Re: Spark2 running forever in zeppelin

@Victor To kill it you can issue command:

yarn application -kill <Application ID>
Highlighted

Re: Spark2 running forever in zeppelin

Explorer

@Felix Albani to add this code always worked, when I added help(), it started running forever.

Don't have an account?
Coming from Hortonworks? Activate your account here