Hi @Kshitij Badani got the same error as Ramon, so maybe my screens can help I`ve got hdp 2.6.3, kerberized, using microsoft AD and want to impersonate users so thay can run spark 1/2 jobs. so far I`m trying to run livy with spark 1.6.3 but after logging in with AD user and running a note I`m getting INFO [2018-05-10 16:49:41,905] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1525958424236_42692352 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session1635594872 INFO [2018-05-10 16:49:41,906] ({pool-2-thread-2} Paragraph.java[jobRun]:366) - run paragraph 20180510-152024_1120525270 using livy org.apache.zeppelin.interpreter.LazyOpenInterpreter@5b439305 INFO [2018-05-10 16:49:41,918] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:132) - Run interpreter process [/usr/hdp/current/zeppelin-server/bin/interpreter.sh, -d, /usr/hdp/current/zeppelin-server/interpreter/livy, -p, 35361, -u, mvince, -l, /usr/hdp/current/zeppelin-server/local-repo/2CKX6DGQZ, -g, livy] INFO [2018-05-10 16:49:42,473] ({pool-2-thread-2} RemoteInterpreter.java[init]:246) - Create remote interpreter org.apache.zeppelin.livy.LivySparkInterpreter INFO [2018-05-10 16:49:42,963] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:578) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CKX6DGQZ:mvince: INFO [2018-05-10 16:49:42,981] ({pool-2-thread-2} RemoteInterpreter.java[init]:246) - Create remote interpreter org.apache.zeppelin.livy.LivySparkSQLInterpreter INFO [2018-05-10 16:49:42,986] ({pool-2-thread-2} RemoteInterpreter.java[init]:246) - Create remote interpreter org.apache.zeppelin.livy.LivyPySparkInterpreter INFO [2018-05-10 16:49:42,992] ({pool-2-thread-2} RemoteInterpreter.java[init]:246) - Create remote interpreter org.apache.zeppelin.livy.LivyPySpark3Interpreter INFO [2018-05-10 16:49:42,997] ({pool-2-thread-2} RemoteInterpreter.java[init]:246) - Create remote interpreter org.apache.zeppelin.livy.LivySparkRInterpreter INFO [2018-05-10 16:49:43,005] ({pool-2-thread-2} RemoteInterpreter.java[init]:246) - Create remote interpreter org.apache.zeppelin.livy.LivySharedInterpreter WARN [2018-05-10 16:49:43,107] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2067) - Job 20180510-152024_1120525270 is finished, status: ERROR, exception: null, result: %text javax.security.auth.login.LoginException: Unable to obtain password from user at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:897) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680) at javax.security.auth.login.LoginContext.login(LoginContext.java:587) at org.springframework.security.kerberos.client.KerberosRestTemplate.doExecute(KerberosRestTemplate.java:185) at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:580) at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:498) at org.apache.zeppelin.livy.BaseLivyInterpreter.callRestAPI(BaseLivyInterpreter.java:619) at org.apache.zeppelin.livy.BaseLivyInterpreter.callRestAPI(BaseLivyInterpreter.java:599) at org.apache.zeppelin.livy.BaseLivyInterpreter.getLivyVersion(BaseLivyInterpreter.java:395) at org.apache.zeppelin.livy.LivySharedInterpreter.open(LivySharedInterpreter.java:47) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.livy.BaseLivyInterpreter.getLivySharedInterpreter(BaseLivyInterpreter.java:165) at org.apache.zeppelin.livy.BaseLivyInterpreter.open(BaseLivyInterpreter.java:139) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:493) at org.apache.zeppelin.scheduler.Job.run(Job.java:175) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) any idea what I`m doing wrong? ... View more

The issue was that the reverse dns was not correctly configured and adding all the hosts to /etc/hosts made it work. I still wonder why this was working before the stop, wait one week, start of the cluster VMs. A cache that was hiding the problem ? ... View more