We want spark to go through authentication.
1. While launching spark-sql , is there an option that authentication can be done with LDAP
2. While using spark via spark thrift server jdbc connection, is there a option to enable authentication just like beeline.
SparkSQL from Zepplein or Spark-Shell will act as the user who owns the Zeppelin or shell process.
Using Thrift Server, Authentication can be enforced via Kerberose which should authenticate against an LDAP store. However, you will need to set hive.server2.enable.doas=true parameter for the Spark Thrift server to ensure that SparkSQL acts as the user that authenticated. By default, Thrift server will act as the hive user on the back end for all queries. There was a Jira for this https://issues.apache.org/jira/browse/SPARK-5159 that is now showing as resolved as of 1.5.2. However, I have not had a chance to confirm that this works.