Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

[#1]

Hello!

I am learning tutorial for HDP sandbox related to zeppelin :

https://hortonworks.com/tutorial/hands-on-tour-of-apache-spark-in-5-minutes/

I downloaded and started notebook, saved all interpreters, they all were blue. I did not change any Zeppelin or Spark configs. However, first code paragraph fails with Null Pointer Exception.

java.lang.NullPointerException
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:348)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:337)
	at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:142)
	at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:790)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:493)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
	at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

%spark2.spark
spark.version

What could be wrong?

[#2 · Accepted solution]

@Stanislav Nikitin Usually you will see these errors if the spark interpreter is not able to start/lunch a spark application in yarn. This could be due several reasons. Could you try to restart your sandbox and test again? Also you can check the spark interpreter log under /var/log/zeppelin directory.

HTH

[#3]

Sorry, it was totally my mistake. I did modify YARN scheduler configuration in a previous tutorial and this affected on spark. I created sub-queues in scheduler, but did not add spark to any of them.