Integrating Ranger with Centrify LDAP proxy

[#1]

Many deployments in hadoop use Centrify LDAP proxy to integrate Active Directory users with linux accounts. Centrify ldapproxy uses posix properties like uid, gidNumber, uidNumber, memberUid, etc... and are made available for ldap search attributes to the clients like Apache Ranger. Ranger interacts with Centrify ldapproxy for configuring hadoop authorization policies as well as for authentication to ranger UI. In order to sync users and groups from Centrify, ranger performs ldap search against Centrify using all the configured LDAP attributes. Some of the common issues noticed during this integration can be:

Issue #1. Users are unable to login to Ranger UI but were successfully sync'd to ranger:

For AD/LDAP authentication, Ranger uses the complete DN of the login user for performing an LDAP search. Ranger also uses group search base and group search filter for group membership for obtaining user role information. Few configuration parameters to verify on ranger are:

ranger.ldap.user.dnpattern - This value should match the exact DN pattern of the login user

Group Search Base - This value should be the search base of the groups which contains the login users as members

Group Search Filter - This value can be a simple filter like "cn=*"

Let's look at one small example:

=========================================

Ldapsearch with Centrify for one of the user contains following information:

dn: cn=alice bob,ou=Users,ou=us,ou=Contractor,dc=apache,dc=org

description: c001038@apache.org

email: abob@apache.org

gecos: Alice Bob

gidNumber: 422222198

homeDirectory: /home/c001038

loginShell: /bin/bash

uid: c001038

uidNumber: 42222045

userPassword:: WWIEphZ2FkZWVzYW4=

accountExpires: 131470704000000000

badPasswordTime: 131140254848671173

badPwdCount: 0

cn: Alice Bob

codePage: 0

countryCode: 0

displayName: Alice Bob

distinguishedName: CN=Alice Bob,OU=Users,OU=us,OU=C ontractor,DC=apache,DC=org

givenName: Alice

instanceType: 4

lastLogoff: 0

lastLogon: 131140358889710142

logonCount: 185

mail: Abob@hortonworks.com

manager: CN=Joe Dave,OU=Corporate,OU=Users,OU=us,DC=apache,DC=org

memberOf: CN=hdp_admin,OU=Hadoop,OU=Applications,OU=Groups,DC=apache,DC=org

memberOf: cn=hdp_group1,ou=Hadoop,ou=Groups,dc=apache,dc=org

name: Alice Bob

objectClass: top

objectClass: posixaccount

objectGUID:: W6rekfmZXECSiIgFeOTnPw==

objectSid:: AQUAAAAAAAUVAAAAYKiAHXLIYdQjI9RcxRQCAA==

primaryGroupID: 500

pwdLastSet: 131123031899439530

sAMAccountName: c001038

sAMAccountType: 805306300

sn: Bob

uSNChanged: 153583075

uSNCreated: 151863255

userAccountControl: 510

userPrincipalName: c001038@apache.org

whenChanged: 20160716210250.0Z

whenCreated: 20160705200144.0Z

As you can observe the complete DN of the user is configured as “dn: cn=alice bob,ou=Users,ou=us,ou=Contractor,dc=apache,dc=org”. But say the user is using uid (c001038) for logging in to ranger admin and the “ranger.ldap.user.dnpattern” is configured as “uid={0},ou=Users,ou=us,ou=Contractor,dc=apache,dc=org”. In this case the ranger login fails because of the mismatch DN pattern. In order to have a successful login with uid, then Centrify config has to be changed to match the DN to be “uid=c001038,ou=Users,ou=us,ou=Contractor,dc=apache,dc=org”.

Otherwise, the user can use cn (alice bob) as the login user and “ranger.ldap.user.dnpattern” can be configured as “cn={0},ou=Users,ou=us,ou=Contractor,dc=apache,dc=org”

Issue #2. Groups not sync'd to Ranger:

In order to provide flexibility and support different deployment scenarios, ranger provides multiple ways to configure group sync.

1. Sync groups from User’s Group name attribute (like “memberof” or “ismemberof”) - One of the use case here is to “Get all the users from the configured user search base and user search filter and get all the groups that the users belong to”

In this case “Group User Map Sync” is set to “true” and “Enable Group Sync” is set to false. The only way ranger gets the group information is by using the User’s Group name attribute (which is generally “memberof” or “ismemberof” attribute of the user). Since Centrify ldapproxy uses posix attributes, most of the times “memberof” or “ismemberof” attribute is not available for the user. In this case groups will not be sync’d to Ranger. There are two ways to solve this issue -

a. Without any changes to Centrify, Ranger can be configured with “Enable Group Sync” option (#2 below).

b. Modify Centrify to populate “memberof” attribute for the user as in example above

(memberOf:cn=hdp_admin,ou=Hadoop,ou=Applications, ou=Groups,dc=apache,dc=org

memberOf: cn=hdp_group1,ou=Hadoop,ou=Groups,dc=apache,dc=org)

2. Sync groups by computing group membership of the users - One of the use case here is to “Get all the users from the configured user search base and user search filter and get only particular groups that these users belong to”. This also targets the case where LDAP server doesn’t support “memberof” attribute for the user.

In this case “Group User Map Sync” should be set to “true” and “Enable Group Sync” is also should be set to true. In this case ranger first syncs all the users based on the “User Info” configuration. Then for each user ranger computes the group memberships based on the “Group Member Attribute” (like member, memberUID, etc…). Some of the common scenarios where the groups fail to sync to ranger are:

a. When “Group User Map Sync” or “Enable Group Sync” not set to true

b. When Group search base or group search filter are misconfigured and no matching groups are found.

c. When “username” that is configured for User search is not matching the value for configured Group Member Attribute. For this let’s take an example as below:

====================================

Let’s say “User name Attribute” is configured as “uid” and “Group Member Attribute” is configured as “member” in Ranger.

Following is the ldapsearch output for a group from Centrify:

dn: hdp_group1,ou=Hadoop,ou=Groups,dc=apache,dc=org

cn: hdp_group1

gidNumber: 478287198

memberUid: bdave

memberUid: c001038

memberUid: c001140

memberUid: djoe

groupType: -2122223646

member: <GUID=0b01714af9d97047aa0d55d813a20385>;<SID=0105000000000005150000006

0a8801d72c861d42323d45ce7170200>;CN=Dave Joe,OU=Users,OU=us,OU=Contractor,DC=apache,DC=org

member: <GUID=3fef7ee4aa1fd84092fce8d25cfed798>;<SID=0105000000000005150000006

0a8801d72c861d42323d45ca5170200>;CN=bob dave,OU=Users,OU=us,DC=apache,DC=org

member: <GUID=5baade91f9995c409288880578e4e73f>;<SID=0105000000000005150000006

0a8801d72c861d42323d45cc5140200>;CN=Alice Bob,OU=Users,OU=us,OU=C ontractor,DC=apache,DC=org

objectClass: top

objectClass: posixgroup

sAMAccountName: hdp_grp1

uSNChanged: 152222220

Since “User name Attribute” in ranger is configured to uid, the users are sync’d to ranger using this attribute (like “c001038”). And for computing group memberships for this user, ranger uses “Group Member Attribute” which is configured as “member”. Ranger performs ldap search to get all the group configured in the group search base that contains the “member” as “uid=c001038”. But as you can see the “member” attribute returned from Centrify doesn’t contain info with uid of the user. Hence the no groups are sync’d to ranger. This issue can be fixed by configuring “Group Member Attribute” in ranger as “memberuid” so that it matches with the value of uid.

3. Sync groups from the configured search base and group search filter - One of the use case here is to “Get all the groups from the configured group search base and group search filter”. This also targets the use case to sync groups with no members.

In this case“Group User Map Sync”, “Enable Group Sync” and “Enable Group Search First” should be set to true. The only reason that the groups are not sync’d to ranger because of the misconfiguration of Group search base or group search filter.

Issue #3: Issues with Syncing ranger users and groups with centrify AD:

Ranger retrieves only the values of the attributes configured in ranger. Hence for syncing users and/or groups ranger performs ldap search against the server (like Centrify) with all these attributes. When users and/or groups not sync’d to ranger, perform ldap search using ldapsearch utility with any particular user and/or group and with the configured attributes. Sometimes a particular attribute (like “memberof”, “member”, etc…) are not configured in Centrify and hence the user sync fails. In this case following error can be found in ranger usersync logs (/var/log/ranger/usersync/usersync.log):

javax.naming.directory.NoSuchAttributeException: [LDAP: error code 16 - cdcLdapSearch :No such attribute (cdcRC=8), errSystem=Ldap, errCode=16, errString=No such attribute];

In general since Centrify uses posix attributes, “member” attribute might not be configured for a group. Instead the users are mapped to groups using “memberuid” attribute. In order to solve this issue, verify the “Group Member Attribute” is configured accordingly (memberuid in this case) in Ranger.