HDP 2.3/Ambari integration with AD managed by Centrify

[#1]

We need to install a Non-Kerberized HDP 2.3 cluster and below are the requirements:

1. There is an existing Active directory maintained by Centrify and all authentication should be done thru that.
2. The service users with custom names are already created in AD under custome group name.
3. We DON'T want ambari to create any local service accounts during HDP installation and want ambari to refer to AD accounts. The main concern is If ambari creates the service accounts locally, then that might mess up group permissions for the files when tried to login with the AD accounts.
4. The requirement is NOT to work with any local accounts and all authentication needs to be done from AD managed by Centrify.
5. From install perspective, what needs to be done to achieve this ?

[#2]

@hrongali@hortonworks.com

This document will save you lot of headache. Link

Page 12 is the most important. Please feel free to reach out to me anytime.

Adding important information regarding service account

link

Use the Skip Group Modifications option to not modify the Linux groups in the cluster. Choosing this option is typically required if your environment manages groups using LDAP and not on the local Linux machines.

[#3 · Accepted solution]

@hrongali@hortonworks.com

I recently did the same. Below are quick notes and pointers to do that.

To use existing the Centrify Active Directory to configure security on an HDP cluster, please refer the the following document reference below covering the Centrify configuration to work with a Hortonwork environment.

Centrify for Hortonworks (Ambari 2.x)

Centrify for Hortonworks (Ambari 1.x)

Note: To enable Centrify to work correctly with Hortonworks, please make the following changes. (A few notes captured from earlier engagements including some gotchas)

1. Add ksh link (other wise AD profiles with ksh will fail login)

ln -s /bin/ksh93 /usr/bin/ksh

2. Edit /etc/krb5.conf file with these settings:

Forwarding=true

Renew=7d

3. Remove HTTP principle from SPN attribute of compute object in Active Directory:

• 1. On each node:
• a. Edit /etc/centrifydc/centrifydc.conf and on line 1092, delete the "# " from the front of line and remove "http" from the list so that it looks like this:
• adclient.krb5.service.principals: ftp cifs nfs
• Changed by (remove nfs also) @Ancil McBarnett

• adclient.krb5.service.principals: ftp cifs 

• b. Save the file and restart the Centrify Agent.

• adreload
  service centrifydc restart

• 2. In Active Directory Users and Computers,do a ADSI Edit (Active Directory® Service Interfaces Editor (ADSI Edit) ) then navigate to the container where the computer objects for the cluster's nodes have been created.
• 3. In each computer object, remove the HTTP principal from SPN attribute.

4. Make sure you have AD account existing for Ambari server if you want to use non-root user existing in AD for Ambari Server.

To configure Ambari for a non-root based user, please refer to the following link.

http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.0/bk_Ambari_Security_Guide/content/_configurin...

Note: Add ulimit command to the list of sudo commands besides those mentioned in Hortonworks documents as Ambari tries to modify ulimits during the HDP services installation using an Ambari user, otherwise it just keeps complaining about the same.

5. For customizing or using services users from AD, make sure all accounts are already created in AD. Make sure all machines are added to AD. And as @Neeraj mentioned in above answer, Use the Skip Group Modifications option to not modify the Linux users/groups in the cluster. Choosing this option is typically required if your environment manages groups using LDAP and not on the local Linux machines or already existing ones.

6. Make sure to change all the user's (Users for Ambari and HDP services) password policy in Active Directory from "User must change password at next logon" to "Password never expires"

7. Then refer to the following site to setup Kerberos in an HDP Cluster.

http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.0/bk_Ambari_Security_Guide/content/ch_configur...

Note: Ensure Unlimited JCE policy is installed if using Oracle JDK else Test KD connection will fail on Enable Kerberos Wizard.

Reach out to me for any further detail.

[#4]

@Pardeep thanks for sharing!

[#5]

Pardeep, thanks for detailed notes, helpful.

[#6]

Thanks Pardeep !

[#7]

@hrongali@hortonworks.com Please do update the thread in case you find anything new