Hi @bkosaraju, I am using sc.stop() in the main where I am starting the sparkContext ,also ensuring that sc.stop is being used before sys.exit(). On the other hand, the resources are not overcommitted as I can see still YARN is 30% free. The first error I see is: 18/03/01 08:13:14 ERROR LiveListenerBus: Dropping SparkListenerEvent because no remaining room in event queue. This likely means one of the SparkListeners is too slow and cannot keep up with the rate at which tasks are being started by the scheduler. Followed by Warnings almost every minute 18/03/01 08:13:14 WARN LiveListenerBus: Dropped 1 SparkListenerEvents since Thu Jan 01 00:00:00 UTC 1970 18/03/01 08:14:14 WARN LiveListenerBus: Dropped 4643 SparkListenerEvents since Thu Mar 01 08:13:14 UTC 2018 18/03/01 08:15:14 WARN LiveListenerBus: Dropped 4759 SparkListenerEvents since Thu Mar 01 08:14:14 UTC 2018 18/03/01 08:16:15 WARN LiveListenerBus: Dropped 4869 SparkListenerEvents since Thu Mar 01 08:15:14 UTC 2018 18/03/01 08:17:15 WARN LiveListenerBus: Dropped 4885 SparkListenerEvents since Thu Mar 01 08:16:15 UTC 2018 18/03/01 08:18:15 WARN LiveListenerBus: Dropped 4653 SparkListenerEvents since Thu Mar 01 08:17:15 UTC 2018 18/03/01 08:19:17 WARN LiveListenerBus: Dropped 4549 SparkListenerEvents since Thu Mar 01 08:18:15 UTC 2018 18/03/01 08:20:17 WARN LiveListenerBus: Dropped 4344 SparkListenerEvents since Thu Mar 01 08:19:17 UTC 2018 18/03/01 08:21:17 WARN LiveListenerBus: Dropped 4559 SparkListenerEvents since Thu Mar 01 08:20:17 UTC 2018 18/03/01 08:22:18 WARN LiveListenerBus: Dropped 4367 SparkListenerEvents since Thu Mar 01 08:21:17 UTC 2018 18/03/01 08:23:18 WARN LiveListenerBus: Dropped 4525 SparkListenerEvents since Thu Mar 01 08:22:18 UTC 2018 18/03/01 08:24:18 WARN LiveListenerBus: Dropped 4442 SparkListenerEvents since Thu Mar 01 08:23:18 UTC 2018 18/03/01 08:25:18 WARN LiveListenerBus: Dropped 4404 SparkListenerEvents since Thu Mar 01 08:24:18 UTC 2018 Not sure, if I am missing anything... ... View more