Reply
Highlighted
Explorer
Posts: 62
Registered: ‎01-22-2014
Accepted Solution

Using Spark on a Kerberos Cluster

Hi,

 

I came to know that Spark does not work in Stand alone mode in a Kerberos enabled cluster but works only in YARN mode.

 

But I was not able to figure out why?

 

Anyone has an explanation or documentation related to this?

 

Thanks,

Arun

 

 

Cloudera Employee
Posts: 366
Registered: ‎07-29-2013

Re: Using Spark on a Kerberos Cluster

Why? in a kerberized environment, to access resources you need to integrate with kerberos. The Spark project hasn't implemented anything like that. YARN works with kerberos, and so it can work with kerberos by leveraging YARN. Maybe part of the answer is, why is it necessary if it works through YARN?