Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Using Spark on a Kerberos Cluster

SOLVED Go to solution
Highlighted

Using Spark on a Kerberos Cluster

Explorer

Hi,

 

I came to know that Spark does not work in Stand alone mode in a Kerberos enabled cluster but works only in YARN mode.

 

But I was not able to figure out why?

 

Anyone has an explanation or documentation related to this?

 

Thanks,

Arun

 

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Using Spark on a Kerberos Cluster

Master Collaborator

Why? in a kerberized environment, to access resources you need to integrate with kerberos. The Spark project hasn't implemented anything like that. YARN works with kerberos, and so it can work with kerberos by leveraging YARN. Maybe part of the answer is, why is it necessary if it works through YARN?

1 REPLY 1

Re: Using Spark on a Kerberos Cluster

Master Collaborator

Why? in a kerberized environment, to access resources you need to integrate with kerberos. The Spark project hasn't implemented anything like that. YARN works with kerberos, and so it can work with kerberos by leveraging YARN. Maybe part of the answer is, why is it necessary if it works through YARN?