Created 08-20-2018 11:40 AM
version:
oozie:4.3
spark2.1
2018-08-20 19:33:23,998 [uber-SubtaskRunner] ERROR org.apache.spark.SparkContext - Error initializing SparkContext. java.lang.NullPointerException at scala.collection.mutable.ArrayOps$ofRef$.newBuilder$extension(ArrayOps.scala:190) at scala.collection.mutable.ArrayOps$ofRef.newBuilder(ArrayOps.scala:186) at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:246) at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259) at scala.collection.mutable.ArrayOps$ofRef.filter(ArrayOps.scala:186) at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:531) at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$6.apply(Client.scala:527) at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:74) at org.apache.spark.deploy.yarn.Client.prepareLocalResources(Client.scala:527) at org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:868) at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:170)
Created 08-24-2018 03:37 AM
This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"
Created 08-24-2018 03:37 AM
This problem have to add spark.yarn.jars in sparkopts.and It have to start with"hdfs://"