Skip to main content
poorva
Bronze III
Sep 16, 2022 2:07 AM UTC
Solved

SparkContext was shut down

how to know why sparkcontext was shut down ??ERROR yarn.Client: Application diagnostics message: User class threw exception: org.apache.spark.SparkException: Job 1 cancelled because SparkContext was shut down

RangaReddy
Gold ⭐️
Sep 16, 2022 12:19 PM UTC

Hi @poorva 

 

Please check the application logs for failed applications from Resource Manager UI, there exception message is printed. Fix the exception and resubmit the job.

 

3 replies

RangaReddy
Gold ⭐️
Gold ⭐️
Sep 16, 2022 12:19 PM UTC

Hi @poorva 

 

Please check the application logs for failed applications from Resource Manager UI, there exception message is printed. Fix the exception and resubmit the job.

 

Translate
vaishaakb
Gold ⭐️⭐️⭐️
Gold ⭐️⭐️⭐️
Sep 19, 2022 4:29 PM UTC

Hello @poorva Did you get a chance to review the Failed Application logs from Resource Manager UI?

 

You can also share the exception from the same time frame so we can help you further with a plan of action. 

 

Let us know your thoughts.

Translate
VidyaSargur
Community Manager
Community Manager
Sep 26, 2022 11:29 AM UTC

@poorva, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. 

Translate
Regards, Vidya Sargur Madabhushi| Senior Community Manager | Was your question answered? Make sure to mark the answer as the Best Answer. If you find a reply useful, say thanks by clicking on the thumbs up button.