SparkContext was shut down

[#1]

how to know why sparkcontext was shut down ??ERROR yarn.Client: Application diagnostics message: User class threw exception: org.apache.spark.SparkException: Job 1 cancelled because SparkContext was shut down

[#2 · Accepted solution]

Hi @poorva 

Please check the application logs for failed applications from Resource Manager UI, there exception message is printed. Fix the exception and resubmit the job.

[#3]

Hello @poorva Did you get a chance to review the Failed Application logs from Resource Manager UI?

You can also share the exception from the same time frame so we can help you further with a plan of action. 

Let us know your thoughts.

[#4]

@poorva, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.