Reply
New Contributor
Posts: 1
Registered: ‎09-07-2017

Spark job failure

Spark job is failing some times with the below issue. we this this error once in while. Please ket me know if any one have encountered this kind of issue . we scheduled the job to run every 10 mn. most of the time it runs succesfully but failed some times

 

17/09/07 15:50:49 ERROR yarn.ApplicationMaster: User class threw exception: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://nameservice1/data/raas/enrichment/unenriched/2017-09-07_03-50-49/accidental_loss_trans already exists
org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://nameservice1/data/raas/enrichment/unenriched/2017-09-07_03-50-49/acc_trans already exists
	at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:146)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1088)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1074)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1074)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:150)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:111)
	at org.apache.spark.rdd.RDD.withScope(RDD.scala:316)
	at org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1074)
	at com.kainos.aviva.raas.xmltransform.RaasQuoteJob$.save(RaasQuoteJob.scala:401)
	at com.kainos.aviva.raas.xmltransform.RaasQuoteJob$.saveAvro(RaasQuoteJob.scala:331)
	at com.kainos.aviva.raas.xmltransform.RaasQuoteJob$.main(RaasQuoteJob.scala:88)
	at com.kainos.aviva.raas.xmltransform.RaasQuoteJob.main(RaasQuoteJob.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:543)
17/09/07 15:50:49 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://nameservice1/data/raas/enrichment/unenriched/2017-09-07_03-50-49/accidental_loss_trans already exists)
17/09/07 15:50:49 INFO spark.SparkContext: Invoking stop() from shutdown hook
Highlighted
Explorer
Posts: 8
Registered: ‎04-26-2017

Re: Spark job failure

Hi,

 

Can you post the code on how you are trying to write to the output? 

 

Thanks,

Ravi

Announcements