Support Questions

Find answers, ask questions, and share your expertise

oozie and spark2 ERROR

avatar
Contributor

version:

oozie:4.3

spark2.1

2018-08-20 19:33:23,998 [uber-SubtaskRunner] ERROR org.apache.spark.SparkContext  - Error initializing SparkContext.
java.lang.NullPointerException
        at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190)
        at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186)
        at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:531)
        at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:527)
        at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
        at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527)
        at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)
1 ACCEPTED SOLUTION

avatar
Contributor

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"

View solution in original post

1 REPLY 1

avatar
Contributor

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"