Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oozie and spark2 ERROR

Solved Go to solution

oozie and spark2 ERROR

Contributor

version:

oozie:4.3

spark2.1

2018-08-20 19:33:23,998 [uber-SubtaskRunner] ERROR org.apache.spark.SparkContext  - Error initializing SparkContext.
java.lang.NullPointerException
        at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190)
        at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186)
        at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:531)
        at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:527)
        at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74)
        at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527)
        at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868)
        at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: oozie and spark2 ERROR

Contributor

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"

1 REPLY 1
Highlighted

Re: oozie and spark2 ERROR

Contributor

This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"

Don't have an account?
Coming from Hortonworks? Activate your account here