Complete hadoop security with AD, Knox, Kerberos, Ranger

[#1]

This is what I would like to achieve.

I have Kerberos setup on the cluster with Ranger policies for Hive tables. Eg: Table1 with group1 permissions and Table2 with group2 permissions. With Microsoft AD, I configure users and groups as follows: Eg: user1 and user2 belongs group1 with read/write permissions and user1 and user3 belongs to group2 with only read permission.

G1 (R/W): U1, U2

G2 (R): U1, U3

Now, I will configure KNOX to authenticate users on AD only.

The idea is for U1 to query data from Hive. What is the expected workflow through the layers of KNOX(with AD), Kerberos, Ranger? Is the following correct?

• With KNOX, a single endpoint is available. U1 authenticates itself with KNOX Api.
• Now Kerberos would validate KNOX service as a Knox user and its keytab as per this link.
• Once within the cluster, the user tries to open hive shell (which kerberos ticket is used here?)
• Once hive shell is open, the query is performed on table1 since permissions are granted for group1 (to which user belongs). Is Hadoop environment aware of this user-group mapping? Or should this be done explicitly?

I am looking for an end-to-end workflow with AD, KNOX, Kerberos, Ranger. This also does not have/explain all components.

This explained to a great extent, but I am still confused about where all kerberos plays a role.

Thanks

Nisha

[#2]

Nisha, from the above it does not seem clear what is the target goal, Are you trying to access Hive via Knox which is authenticated via Kerberos where Knox is coupled with AD authentication ?
For "Now Kerberos would validate KNOX service with its keytab as per this link." link seems to be missing.

Additionally, Kindly specify the version of Ambari/HDP you are planning to use.

[#3]

@vsuvagia

I have updated the link. Well my goal is to perform any Hadoop job/query passing all levels of security for all AD users authenticated via KNOX and Kerberos and authorized via Ranger. I am looking for all configuration details for the same.

[#4]

Nisha, Thanks for the details, (Please mention the Ambari / HDP version you will be using.)
So as I understand, your goal seems to be to access Web-HDFS, Hive etc via Knox which is to be configured for both AD and kerberos authentication. Additionally you also want the user to be authenticated by Ranger as well ?

[#5]

Yes absolutely @vsuvagia. With Ranger group policies set for certain tables, only users belonging to that group can access the respective tables. For this (as I understand from many sources of documentation) Ranger should also know the user-group mapping in AD. and so is the case of Hadoop (with user-group mapping configured in hdfs-site.xml).

What I am ideally looking for is a one time authentication to LDAP, and this internally reflects in all components in HDP. But I am not sure if this is possible.

From all the reading, what I have understood is that every component (KNOX, Ranger, HDFS) should be separately configured with LDAP details. 😞 I dont even know if Kerberos should also be connected with LDAP as well.

Sorry, forgot to mention the versions. I am using HDP 2.6 stack with Ambari 2.5.0.3.

[#6]

@Nisha, If you want you use AD users system-wide for Hadoop, Knox and Ranger as well, you can think to configure Kerberos via AD itself, you can refer AD integration with Kerberos.
Additionally you can configure Linux System service to use AD users and groups which can be then used configured in Hadoop as well. Please refer below articles:
HDP-2.6 Security Labs
Hadoop Group Mapping using LDAP/AD

Setup SSSD for AD

Hope this helps.