Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

snikitin — Wed, 25 Jul 2018 19:22:16 GMT

Hello!

I am learning tutorial for HDP sandbox related to zeppelin :

https://hortonworks.com/tutorial/hands-on-tour-of-apache-spark-in-5-minutes/

I downloaded and started notebook, saved all interpreters, they all were blue. I did not change any Zeppelin or Spark configs. However, first code paragraph fails with Null Pointer Exception.

java.lang.NullPointerException
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:348)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:337)
	at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:142)
	at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:790)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:493)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
	at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

%spark2.spark
spark.version

What could be wrong?

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

falbani — Wed, 25 Jul 2018 20:42:31 GMT

@Stanislav Nikitin Usually you will see these errors if the spark interpreter is not able to start/lunch a spark application in yarn. This could be due several reasons. Could you try to restart your sandbox and test again? Also you can check the spark interpreter log under /var/log/zeppelin directory.

HTH

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

snikitin — Wed, 25 Jul 2018 21:23:52 GMT

Sorry, it was totally my mistake. I did modify YARN scheduler configuration in a previous tutorial and this affected on spark. I created sub-queues in scheduler, but did not add spark to any of them.

question Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox in Archives of Support Questions (Read Only)

Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox