Using Spark on a Kerberos Cluster

[#1]

Hi,

I came to know that Spark does not work in Stand alone mode in a Kerberos enabled cluster but works only in YARN mode.

But I was not able to figure out why?

Anyone has an explanation or documentation related to this?

Thanks,

Arun

[#2 · Accepted solution]

Why? in a kerberized environment, to access resources you need to integrate with kerberos. The Spark project hasn't implemented anything like that. YARN works with kerberos, and so it can work with kerberos by leveraging YARN. Maybe part of the answer is, why is it necessary if it works through YARN?