Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

oozie and spark2 ERROR

Contributor

version:

oozie:4.3

spark2.1

2018-08-20 19:33:23,998 [uber-SubtaskRunner] ERROR org.apache.spark.SparkContext  - Error initializing SparkContext.
java.lang.NullPointerException
        at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190)
        at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186)
        at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:531)
        at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:527)
        at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
        at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527)
        at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)
1 ACCEPTED SOLUTION

Contributor

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"

View solution in original post

1 REPLY 1

Contributor

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.