how to know why sparkcontext was shut down ??ERROR yarn.Client: Application diagnostics message: User class threw exception: org.apache.spark.SparkException: Job 1 cancelled because SparkContext was shut down
Please check the application logs for failed applications from Resource Manager UI, there exception message is printed. Fix the exception and resubmit the job.
View solution in original post
Hello @poorva Did you get a chance to review the Failed Application logs from Resource Manager UI?
You can also share the exception from the same time frame so we can help you further with a plan of action.
Let us know your thoughts.
@poorva, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.