Support Questions

Find answers, ask questions, and share your expertise

Using Spark on a Kerberos Cluster

avatar
Explorer

Hi,

 

I came to know that Spark does not work in Stand alone mode in a Kerberos enabled cluster but works only in YARN mode.

 

But I was not able to figure out why?

 

Anyone has an explanation or documentation related to this?

 

Thanks,

Arun

 

 

1 ACCEPTED SOLUTION

avatar
Master Collaborator

Why? in a kerberized environment, to access resources you need to integrate with kerberos. The Spark project hasn't implemented anything like that. YARN works with kerberos, and so it can work with kerberos by leveraging YARN. Maybe part of the answer is, why is it necessary if it works through YARN?

View solution in original post

1 REPLY 1

avatar
Master Collaborator

Why? in a kerberized environment, to access resources you need to integrate with kerberos. The Spark project hasn't implemented anything like that. YARN works with kerberos, and so it can work with kerberos by leveraging YARN. Maybe part of the answer is, why is it necessary if it works through YARN?