Support Questions
Find answers, ask questions, and share your expertise

Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

Solved Go to solution

Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

Explorer

Hello!

I am learning tutorial for HDP sandbox related to zeppelin :

https://hortonworks.com/tutorial/hands-on-tour-of-apache-spark-in-5-minutes/

I downloaded and started notebook, saved all interpreters, they all were blue. I did not change any Zeppelin or Spark configs. However, first code paragraph fails with Null Pointer Exception.

java.lang.NullPointerException
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:348)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:337)
	at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:142)
	at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:790)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:493)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
	at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
%spark2.spark
spark.version

What could be wrong?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

@Stanislav Nikitin Usually you will see these errors if the spark interpreter is not able to start/lunch a spark application in yarn. This could be due several reasons. Could you try to restart your sandbox and test again? Also you can check the spark interpreter log under /var/log/zeppelin directory.

HTH

View solution in original post

2 REPLIES 2
Highlighted

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

@Stanislav Nikitin Usually you will see these errors if the spark interpreter is not able to start/lunch a spark application in yarn. This could be due several reasons. Could you try to restart your sandbox and test again? Also you can check the spark interpreter log under /var/log/zeppelin directory.

HTH

View solution in original post

Highlighted

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

Explorer

Sorry, it was totally my mistake. I did modify YARN scheduler configuration in a previous tutorial and this affected on spark. I created sub-queues in scheduler, but did not add spark to any of them.