Streaming job does not get triggered - Promise already completed

[#1]

While running a streaming application, we come across this below error and streaming job does not get triggered.

This occurs so can you please guide us in resolving this issue.

User class threw exception: java.lang.IllegalStateException: Promise already completed.

at scala.concurrent.Promise$class.complete(Promise.scala:55)

at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:157)

at scala.concurrent.Promise$class.success(Promise.scala:86)

at scala.concurrent.impl.Promise$DefaultPromise.success(Promise.scala:157)

at org.apache.spark.deploy.yarn.ApplicationMaster.org$apache$spark$deploy$yarn$ApplicationMaster$$sparkContextInitialized(ApplicationMaster.scala:392)

at org.apache.spark.deploy.yarn.ApplicationMaster$.sparkContextInitialized(ApplicationMaster.scala:808)

at org.apache.spark.scheduler.cluster.YarnClusterScheduler.postStartHook(YarnClusterScheduler.scala:32)

at org.apache.spark.SparkContext.<init>(SparkContext.scala:560)

at org.apache.spark.streaming.StreamingContext$.createNewSparkContext(StreamingContext.scala:858)

at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:85)

at com.ods.realtime.ingestion.ODSRealtimeDataIngestionJob$.delayedEndpoint$com$ods$realtime$ingestion$ODSRealtimeDataIngestionJob$1(ODSRealtimeDataIngestionJob.scala:45)

at com.ods.realtime.ingestion.ODSRealtimeDataIngestionJob$delayedInit$body.apply(ODSRealtimeDataIngestionJob.scala:18)

at scala.Function0$class.apply$mcV$sp(Function0.scala:34)

at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)

at scala.App$$anonfun$main$1.apply(App.scala:76)

at scala.App$$anonfun$main$1.apply(App.scala:76)

at scala.collection.immutable.List.foreach(List.scala:392)

at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)

at scala.App$class.main(App.scala:76)

at com.ods.realtime.ingestion.ODSRealtimeDataIngestionJob$.main(ODSRealtimeDataIngestionJob.scala:18)

at com.ods.realtime.ingestion.ODSRealtimeDataIngestionJob.main(ODSRealtimeDataIngestionJob.scala)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:685)

[#2]

Can you share your code?

[#3]

Hi @Rekasri 

Due to code related issue, the above exception is occurred. Please check your code while creating/closing the SparkSession object.

Note: If already found the answer please share the issue code and fixed code, it will helpful for others.

[#4]

@Rekasri have you resolved your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.