Created on 05-08-2017 02:39 PM - edited 09-16-2022 04:34 AM
Hi,
In my environment, Hadoop nodes are integrated with AD for authentication. AD doesn't support Kerberos. I understand that it's possible to have users + user principals serviced by AD and have only service principals serviced by a local KDC. Question is, is it possible to set up a local KDC server for both service and user principals but actual users will reside in AD ? So, I will need to host kerberos principals and manage tickets of AD users in local KDC. AD user realm and KDC realm will also be different.
Any help would be appreciated 🙂
Created 05-08-2017 04:16 PM
When a separate MIT-KDC is used, the usual design is to use it to store the host and service principals associated with the Hadoop cluster. The user principals are stored in AD, and a one-way trust is established between the AD domain and the MIT-KDC realm so that users in AD can access cluster services (but not the other way around).
This HCC article discusses one-way trusts between MIT-KDC and AD.
Created 05-08-2017 04:16 PM
When a separate MIT-KDC is used, the usual design is to use it to store the host and service principals associated with the Hadoop cluster. The user principals are stored in AD, and a one-way trust is established between the AD domain and the MIT-KDC realm so that users in AD can access cluster services (but not the other way around).
This HCC article discusses one-way trusts between MIT-KDC and AD.
Created 05-08-2017 06:43 PM
@slachterman thanks for your reply. Doesn't support means, the authentication agent used in linux systems is not able to connect to AD Kerberos ports and there are policy wise restrictions in using Kerberos on linux boxes and I know it's bit weird.
So, I was thinking about some solution that doesn't depend on KDC services by AD but still having Kerberos enabled on cluster. For example, suppose we have an AD and a local MIT-KDC. Local KDC hosts service principals. AD manages cluster users. If there exists a user - A in AD, and I create a user principal A@MIT-KDC-Realm, and if there's one way trust established between AD and KDC, will user A be able to successfully launch jobs and use services on cluster ?
Created 05-08-2017 06:45 PM
Yes, that's what the one-way trust accomplishes. You also need to consider SSSD or similar so that user identities exist locally on the cluster boxes (using, say, LDAP). Please accept/upvote the above if this answer was helpful to you.
Created 08-31-2018 02:33 AM
This does not respond the question but some topic related to the question. As far as I understand @tuxnet is asking if he would be able to use the AD only as an LDAP provider (he states clearly he doen't have the chance to use Kerberos service from AD) and use ONLY an independent MIT KDC as Kerberos authentication provider.
So in this case there is not any one-way or cross domain trust between MIT KDC and AD involved.
He will have to use some SSSD configuration for LDAP + Kerberos using AD only as an identity/authorization provider and KDC as the authentication/chpass provider, and will have to do the cluster kerberization using only the MIT KDC server.
Also the user credentials (passwords) for the cluster will be maintained only in the KDC (where the principals reside) and so he will have to keep password sync between the two sources from some external mean.
Created 05-10-2017 06:42 AM
Thanks for the reply. But, as per my understanding, one way trust means, local KDC will trust the tickets generated by central AD. Correct? If the linux servers where Hadoop runs cannot contact AD for Kerberos, or if we have just an LDAP server in place of AD, how the Kerberos auth happens? Will I have to create user principals manually in local KDC?
Created 05-10-2017 05:19 PM
@tuxnet please see new answer.
Created 08-31-2018 02:49 AM
Yes, you will have to use basically the same configuration done when using a combination of OpenLDAP and MIT KDC for authentication. The only difference is you will be using AD as your LDAP server instead of OpenLDAP, and of course you will have to consider the different schemas for users/groups (samAccountName vs uid, etc).
Created 05-10-2017 05:19 PM
The mechanics of the one-way trust are as follows:
Therefore, we can conclude that the AD user needs to be able to contact the MIT-KDC server (tcp/88).
The cluster servers need to contact the AD LDAP in order to support synchronization for Ranger, SSSD, etc.
You don't need to create user principals in the cluster MIT-KDC if these principals exist within the AD domain and you can set up the one-way trust.
Created 05-10-2017 07:32 PM
Thank you very much for the answer 🙂