facing issue like "Caused by: org.apache.spark.SparkException: Failed to get broadcast_4_piece0 of broadcast_4" while submitting spark job .Anyone can help me out?

[#1]

p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; min-height: 13.0px} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo} span.s1 {font-variant-ligatures: no-common-ligatures} span.Apple-tab-span {white-space:pre}

[2017-01-17 04:22:05,426] org.apache.spark.scheduler.TaskSetManager WARN: Lost task 0.0 in stage 1.0 (TID 2, hdp-iaas-dev16-worker03.dev.na1.phsdp.com): java.io.IOException: org.apache.spark.SparkException: Failed to get broadcast_4_piece0 of broadcast_4

at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1178)

at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:165)

at org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:64)

at org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:64)

at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:88)

at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)

at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:65)

at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)

at org.apache.spark.scheduler.Task.run(Task.scala:88)

at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.spark.SparkException: Failed to get broadcast_4_piece0 of broadcast_4

[#2]

It is hard to tell from a single exception.

How long your job has been running?

Do you have spark.cleaner.ttl set?

See a relevant issue in Apache here: https://issues.apache.org/jira/browse/SPARK-5594