I am submitting a spark2 application with yarn as the master. This causes the exception:  org.apache.spark.SparkException: A master URL must be set in your configuration From what the searching I've done, I should not be getting this exception    --- Submit Script: export JAVA_HOME=/usr/java/jdk1.8.0_131/ spark2-submit --class com.example.myapp.ClusterEntry \ --name "Hello World" \ --master yarn \ --deploy-mode cluster \ --driver-memory 1g \ --executor-memory 1g \ --executor-cores 3 \ --packages org.apache.kudu:kudu-spark2_2.11:1.4.0 \ myapp.jar myconf.file Exception: 18/03/14 15:31:47 WARN scheduler.TaskSetManager: Lost task 1.0 in stage 1.0 (TID 3, vm6.adcluster, executor 1): org.apache.spark.SparkException: A master URL must be set in your configuration at org.apache.spark.SparkContext.<init>(SparkContext.scala:376) at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2509) at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:909) at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:901) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:901) at com.example.myapp.dao.KuduSink.open(KuduSink.scala:18) at org.apache.spark.sql.execution.streaming.ForeachSink$$anonfun$addBatch$1.apply(ForeachSink.scala:50) at org.apache.spark.sql.execution.streaming.ForeachSink$$anonfun$addBatch$1.apply(ForeachSink.scala:49) at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:926) at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:926) at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2062) at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2062) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:108) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)   This app when I run it locally, just not on the cluster.   One thing that may be relevant is that the application uses a Kudu Client that appears in the exception message in the class "KuduSink".    ... View more