Getting error "User session not found 403" when using zeppelin with livy

[#1]

So we have a kerberized cluster and have integrated ldaps with zeppelin. But when i try to use livy i get the following error

ERROR [2016-12-01 10:00:02,442] ({pool-2-thread-4} LivyHelper.java[createSession]:128) - Error getting session for user java.lang.Exception: Cannot start spark. at org.apache.zeppelin.livy.LivyHelper.createSession(LivyHelper.java:117) at org.apache.zeppelin.livy.LivySparkSQLInterpreter.interpret(LivySparkSQLInterpreter.java:62) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341) at org.apache.zeppelin.scheduler.Job.run(Job.java:176) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ERROR [2016-12-01 10:00:02,442] ({pool-2-thread-4} LivySparkSQLInterpreter.java[interpret]:70) - Exception in LivySparkSQLInterpreter while interpret java.lang.Exception: Cannot start spark.

in zeppelin interpreter logs.

Also when i dont enable ldap for zeppelin (that is i log in as anonymous), I can see that job is been submitted to yarn but it fails as user is `zeppelin-clusterName`. But when i use ldap with zeppelin it is not even getting submit to yarn.

[#2]

Hey @kbadani, thanks for getting back. I did this but no luck, I also checked after restarting the services but the 403 error persists.

[#3]

@Bhavin Tandel Can you please post what error you are getting in livy server logs ?

[#4]

@kbadani Sorry for the delay, I have attached the live.out log

livy-out.txt

[#5]

yarn app is started, could you check the yarn app log ? appid : application_1481021618182_0004

[#6 · Accepted solution]

So guys @kbadani and @jzhang the issue finally got solved and thank you for your support. It was very trivial but very important property, livy.superuser was set to `zeppelin-<cluster name in uppercase>` but the principal was in lower case 😛 . And changing that solved the issue, @kbadani You pointed out that property but I didnt know that it is case sensitive.

[#7]

@Bhavin Tandel good to know 🙂 Thanks