I am using spark 1.6 to read messages from Tibco.i am taking each Rdd in dstream ,performing some tranformations and converting it to a data frame and then finally writing to a hdfs path.My job runs for few batches then it starts erroring out saying _temporary file not found. And finally the job is aborted.Below is the detailed link to stack overflow question.