Spark (no SQL/LLAP) with Livy works fine in Zeppelin
together with Kerberos. I can submit jobs in Zeppelin, and they gets executed
in Yarn correctly, with the right user and everything. So I know that the Kerberos
configuration is correct and working. But as soon as I try to run a Spark SQL
code, I get a Kerberos error saying that I don’t have a valid Kerberos ticket. (Caused by: GSSException: No valid credentials
provided (Mechanism level: Failed to find any Kerberos tgt))
Normal Spark without Livy works fine with LLAP. No problem
there at all.
Is there any other configuration, except the once that are
already described in the articles above, that I need to make to be able to get
a valid Kerberos ticket from the Spark session that Livy creates for me? Or do
you have any more information that can help me to get this functionality to
I have found the solution to my problem. Add the following to the Livy interpreter in Zeppelin. Once that is done, I can use Spark SQL with LLAP and Livy in Zeppelin. The jobs are executed as the logged in Zeppelin users (Shiro authentication to Microsoft AD) and that is verified in Hive2's history log.