Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

Solved Go to solution
Highlighted

Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

New Contributor

Hello!

I am learning tutorial for HDP sandbox related to zeppelin :

https://hortonworks.com/tutorial/hands-on-tour-of-apache-spark-in-5-minutes/

I downloaded and started notebook, saved all interpreters, they all were blue. I did not change any Zeppelin or Spark configs. However, first code paragraph fails with Null Pointer Exception.

java.lang.NullPointerException
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:38)
	at org.apache.zeppelin.spark.Utils.invokeMethod(Utils.java:33)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_2(SparkInterpreter.java:348)
	at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:337)
	at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:142)
	at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:790)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:493)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
	at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
%spark2.spark
spark.version

What could be wrong?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

@Stanislav Nikitin Usually you will see these errors if the spark interpreter is not able to start/lunch a spark application in yarn. This could be due several reasons. Could you try to restart your sandbox and test again? Also you can check the spark interpreter log under /var/log/zeppelin directory.

HTH

2 REPLIES 2

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

@Stanislav Nikitin Usually you will see these errors if the spark interpreter is not able to start/lunch a spark application in yarn. This could be due several reasons. Could you try to restart your sandbox and test again? Also you can check the spark interpreter log under /var/log/zeppelin directory.

HTH

Re: Spark Interpreter fails with Null Pointer Exception,Spark Interpreter Null Pointer Exception in Sandbox

New Contributor

Sorry, it was totally my mistake. I did modify YARN scheduler configuration in a previous tutorial and this affected on spark. I created sub-queues in scheduler, but did not add spark to any of them.