@Shu, Thanks for responding. I tried to update ticket earlier but, couldn't happen. I am not sure if some configs which I had set were causing this issue but, after commenting those out it started working fine. Below are the configs: .config("spark.memory.fraction", "0.3") .config("spark.storage.memoryFraction", "0.8") .config("spark.executor.heartbeatInterval", "300") .config("spark.network.timeout", "900") .config("spark.sql.autoBroadcastJoinThreshold", "-1") ... View more