Member since
06-20-2016
251
Posts
196
Kudos Received
36
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
9629 | 11-08-2017 02:53 PM | |
2048 | 08-24-2017 03:09 PM | |
7778 | 05-11-2017 02:55 PM | |
6380 | 05-08-2017 04:16 PM | |
1925 | 04-27-2017 08:05 PM |
05-11-2017
02:37 PM
2 Kudos
Kerberos is a widely-used authentication system and is used throughout the Hadoop ecosystem, in particular, for strong authentication. The KDC, or key distribution center, is the name for the Kerberos server application that exposes the Authentication Service and Ticket Granting Service, as well as hosting the Kerberos principal database. Active Directory is used in many enterprises for Identity and Access Management. A common architecture in enterprise deployment scenarios is to make use of a local cluster KDC (often using the MIT-KDC packaging) to host the service and host principals associated with the cluster, and to configure a one-way trust between the AD domain and the cluster realm. This has the advantage of offloading Kerberos traffic from the domain controller(s) and not all enterprises do not necessarily want to host cluster principals within their AD domain. So how does a one-way trust work? The first thing to note is that the trust is instantiated by the existence of a special cross-realm principal. For example, if realm B.EXAMPLE.COM trusts realm A.EXAMPLE.COM, clients in the realm A.EXAMPLE.COM can authenticate to services in B.EXAMPLE.COM. In order for a client of A.EXAMPLE.COM to access a service in the B.EXAMPLE.COM realm, both realms must share a key for a principal named krbtgt/B.EXAMPLE.COM@A.EXAMPLE.COM (and both keys must have the same key version number associated with them). To access a cross-realm service, the user first contacts their home KDC's AS (AD domain controller which exposes the KDC service, in the scenario at-hand) asking for a TGT that will be used with the TGS of the foreign realm.
If there is a direct trust relationship between the home realm and the foreign realm (practically materialized in shared inter-realm keys, per the above), the home KDC delivers the requested TGT. The user then contacts the cluster MIT-KDC, in the foreign realm, presenting the cross-realm TGT and requesting a service ticket for the service in question. Finally, the user contacts the cluster service in question, presenting the service ticket. Therefore, we can conclude that the AD user needs to be able to contact the MIT-KDC server (usually tcp/88). Please note this means that if the cluster is within a secured network zone which includes the MIT-KDC host, then there needs to be a firewall rule allowing AD clients to contact this host (again, usually over tcp/88). Please see diagram below. References: RFC 5868 Kerberos and its Application in Cross-realm Operations RedHat: Setting up Cross-realm Authentication
... View more
Labels:
05-10-2017
05:19 PM
1 Kudo
The mechanics of the one-way trust are as follows:
AD user requests cross-realm TGT from AD KDC (i.e., the domain controller exposing the KDC service) AD user contacts cluster MIT-KDC, presenting the TGT and requesting a service ticket for the service in question AD user contacts cluster service, presenting the service ticket Therefore, we can conclude that the AD user needs to be able to contact the MIT-KDC server (tcp/88). The cluster servers need to contact the AD LDAP in order to support synchronization for Ranger, SSSD, etc. You don't need to create user principals in the cluster MIT-KDC if these principals exist within the AD domain and you can set up the one-way trust.
... View more
05-10-2017
05:19 PM
@tuxnet please see new answer.
... View more
05-08-2017
06:45 PM
Yes, that's what the one-way trust accomplishes. You also need to consider SSSD or similar so that user identities exist locally on the cluster boxes (using, say, LDAP). Please accept/upvote the above if this answer was helpful to you.
... View more
05-08-2017
04:16 PM
1 Kudo
@tuxnet, please clarify what you mean by "AD doesn't support Kerberos". AD does in fact use Kerberos for authentication under the covers. When a separate MIT-KDC is used, the usual design is to use it to store the host and service principals associated with the Hadoop cluster. The user principals are stored in AD, and a one-way trust is established between the AD domain and the MIT-KDC realm so that users in AD can access cluster services (but not the other way around). This HCC article discusses one-way trusts between MIT-KDC and AD.
... View more
05-04-2017
02:08 PM
Hi @Manmeet Kaur, please post this on HCC as a separate question.
... View more
05-02-2017
06:34 PM
@Joshua Petree can you explain further what you mean by "Ambari is not allowing the DataNodes to access the /home directories?"
... View more
05-02-2017
06:18 PM
@Joshua Petree yes, you want dfs.datanode.data.dir to be backed by a mount point that has the majority of the data available for HDFS blocks. Please upvote or accept the above answer if this is helpful in resolving your issue.
... View more
05-02-2017
03:53 PM
1 Kudo
Hi @Joshua Petree, that is odd. It seems like the 2 TB disks on your data nodes are not properly associated with your HDFS storage. What do you see in dfs.datanode.data.dir? Are you sure the listed mount points are backed by the 2 TB disks you intended?
... View more
05-01-2017
07:40 PM
Do you mean DataNode capacity? Which report? Is this a virtual or bare-metal environment? What are the hardware specs?
... View more