Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

oozie and spark2 ERROR

Contributor

version:

oozie:4.3

spark2.1

2018-08-20 19:33:23,998 [uber-SubtaskRunner] ERROR org.apache.spark.SparkContext  - Error initializing SparkContext.
java.lang.NullPointerException
        at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190)
        at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186)
        at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:531)
        at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:527)
        at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
        at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527)
        at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)
1 ACCEPTED SOLUTION

Contributor

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"

View solution in original post

1 REPLY 1

Contributor

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"