Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

spark-sql on hdp2.3.4 ( hive on spark) authentication with LDAP

spark-sql on hdp2.3.4 ( hive on spark) authentication with LDAP

New Contributor

Hi ,

We want spark to go through authentication.

1. While launching spark-sql , is there an option that authentication can be done with LDAP

2. While using spark via spark thrift server jdbc connection, is there a option to enable authentication just like beeline.

1 REPLY 1

Re: spark-sql on hdp2.3.4 ( hive on spark) authentication with LDAP

Guru

@Bharat Rathi

SparkSQL from Zepplein or Spark-Shell will act as the user who owns the Zeppelin or shell process.

Using Thrift Server, Authentication can be enforced via Kerberose which should authenticate against an LDAP store. However, you will need to set hive.server2.enable.doas=true parameter for the Spark Thrift server to ensure that SparkSQL acts as the user that authenticated. By default, Thrift server will act as the hive user on the back end for all queries. There was a Jira for this https://issues.apache.org/jira/browse/SPARK-5159 that is now showing as resolved as of 1.5.2. However, I have not had a chance to confirm that this works.