Greetings Experts, I am trying to run LivyServer in zeppelin in HDP 2.5TP sandbox, however my interpreter keeps failing. I tried %livy.sparkr and %livy.pyspark all getting same exceptions. See error messages below from the livy log file: LivyPySparkInterpreter while interpret
java.lang.Exception: Cannot start pyspark
INFO [2016-07-18 22:20:23,102] ({Thread-0} RemoteInterpreterServer.java[run]:81) - Starting remote interpreter server on port 41012
INFO [2016-07-18 22:20:23,420] ({pool-1-thread-4} RemoteInterpreterServer.java[createInterpreter]:169) - Instantiate interpreter org.apache.zeppelin.livy.LivySparkR
Interpreter
INFO [2016-07-18 22:20:23,457] ({pool-1-thread-4} RemoteInterpreterServer.java[createInterpreter]:169) - Instantiate interpreter org.apache.zeppelin.livy.LivySparkI
nterpreter
INFO [2016-07-18 22:20:23,461] ({pool-1-thread-4} RemoteInterpreterServer.java[createInterpreter]:169) - Instantiate interpreter org.apache.zeppelin.livy.LivyPySpar
kInterpreter
INFO [2016-07-18 22:20:23,462] ({pool-1-thread-4} RemoteInterpreterServer.java[createInterpreter]:169) - Instantiate interpreter org.apache.zeppelin.livy.LivySparkS
QLInterpreter
INFO [2016-07-18 22:20:23,505] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1468880423503 started by scheduler org.apache.zepp
elin.livy.LivySparkRInterpreter1573372211
ERROR [2016-07-18 22:20:24,722] ({pool-2-thread-2} LivyHelper.java[createSession]:89) - sessionId:0.0 state is starting
ERROR [2016-07-18 22:20:25,754] ({pool-2-thread-2} LivyHelper.java[createSession]:89) - sessionId:0.0 state is starting
ERROR [2016-07-18 22:20:26,769] ({pool-2-thread-2} LivyHelper.java[createSession]:89) - sessionId:0.0 state is starting
ERROR [2016-07-18 22:20:27,784] ({pool-2-thread-2} LivyHelper.java[createSession]:89) - sessionId:0.0 state is dead
ERROR [2016-07-18 22:20:28,799] ({pool-2-thread-2} LivyHelper.java[createSession]:89) - sessionId:0.0 state is dead
ERROR [2016-07-18 22:20:29,814] ({pool-2-thread-2} LivyHelper.java[createSession]:89) - sessionId:0.0 state is dead
....
....
ERROR [2016-07-18 22:41:10,602] ({pool-2-thread-6} LivyHelper.java[createSession]:89) - sessionId:2.0 state is starting
ERROR [2016-07-18 22:41:11,609] ({pool-2-thread-6} LivyHelper.java[createSession]:114) - Error getting session for user within 60Sec.
ERROR [2016-07-18 22:41:11,609] ({pool-2-thread-6} LivyHelper.java[createSession]:120) - Error getting session for user
java.lang.Exception: Cannot start pyspark.
at org.apache.zeppelin.livy.LivyHelper.createSession(LivyHelper.java:115)
at org.apache.zeppelin.livy.LivyPySparkInterpreter.interpret(LivyPySparkInterpreter.java:63)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
ERROR [2016-07-18 22:41:11,610] ({pool-2-thread-6} LivyPySparkInterpreter.java[interpret]:70) - Exception in LivyPySparkInterpreter while interpret
java.lang.Exception: Cannot start pyspark.
at org.apache.zeppelin.livy.LivyHelper.createSession(LivyHelper.java:115)
at org.apache.zeppelin.livy.LivyPySparkInterpreter.interpret(LivyPySparkInterpreter.java:63)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
INFO [2016-07-18 22:41:11,611] ({pool-2-thread-6} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1468881610078 finished by scheduler org.apache.ze
ppelin.livy.LivyPySparkInterpreter1321637285
... View more
