Ranger sync fetched LDAP users but not showing in UI

[#1]

Ranger user sync seems to have fetched the users and groups from ApacheAD LDAP as shown in the log. But the users not showing in Ranger UI. Please help. I have tried both user filter (cn=*) and empty with no difference.

22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder initialization completed with --  ldapUrl: ldap://54.197.148.18:10389,  ldapBindDn: cn=ranger,ou=sds,dc=air,dc=org,  ldapBindPassword: ***** ,  ldapAuthenticationMechanism: simple,  searchBase: dc=air,dc=org,  userSearchBase: [ou=sds,dc=air,dc=org],  userSearchScope: 2,  userObjectClass: person,  userSearchFilter: (cn=*),  extendedUserSearchFilter: null,  userNameAttribute: cn,  userSearchAttributes: [uSNChanged, cn, modifytimestamp],  userGroupNameAttributeSet: null,  pagedResultsEnabled: true,  pagedResultsSize: 500,  groupSearchEnabled: true,  groupSearchBase: [ou=sds,dc=air,dc=org],  groupSearchScope: 2,  groupObjectClass: groupOfUniqueNames,  groupSearchFilter: ,  extendedGroupSearchFilter: (&null(|(uniqueMember={0})(uniqueMember={1}))),  extendedAllGroupsSearchFilter: null,  groupMemberAttributeName: uniqueMember,  groupNameAttribute: cn, groupSearchAttributes: [uSNChanged, cn, uniqueMember, modifytimestamp],  groupUserMapSyncEnabled: true, groupSearchFirstEnabled: false, userSearchEnabled: false,  ldapReferral: ignore
22 May 2018 14:57:04  INFO UserGroupSync [UnixUserSyncThread] - Begin: initial load of user/group from source==>sink
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder updateSink started
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - Performing user search first
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - extendedUserSearchFilter = (&(objectclass=person)(|(uSNChanged>=0)(modifyTimestamp>=19700101120000Z))(cn=*))
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder.getUsers() completed with user count: 0
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - extendedAllGroupsSearchFilter = (&(objectclass=groupOfUniqueNames)(|(uSNChanged>=0)(modifyTimestamp>=19700101120000Z)))
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - timeStampVal = 20180518183001.391Zand currentDeltaSyncTime = 1526668201000
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - longUserName: cn=ranger,ou=sds,dc=air,dc=org, userName: ranger
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - longUserName: cn=andrew,ou=sds,dc=air,dc=org, userName: andrew
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - No. of members in the group students = 2
22 May 2018 14:57:04  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder.getGroups() completed with group count: 1
22 May 2018 14:57:05  INFO UserGroupSync [UnixUserSyncThread] - End: initial load of user/group from source==>sink
22 May 2018 14:57:05  INFO UserGroupSync [UnixUserSyncThread] - Done initializing user/group source and sink
22 May 2018 14:57:09  INFO UnixAuthenticationService [main] - Enabling Unix Auth Service!
22 May 2018 14:57:09  INFO UnixAuthenticationService [main] - Enabling Protocol: [SSLv2Hello]
22 May 2018 14:57:09  INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1]
22 May 2018 14:57:09  INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1.1]
22 May 2018 14:57:09  INFO UnixAuthenticationService [main] - Enabling Protocol: [TLSv1.2]

[#2]

Further debugging information shows it is adding users and group to the database. But why it just did not show in the console?

22 May 2018 16:32:46  INFO LdapDeltaUserGroupBuilder [UnixUserSyncThread] - LdapDeltaUserGroupBuilder.getGroups() completed with group count: 1
22 May 2018 16:32:46 DEBUG LdapDeltaUserGroupBuilder [UnixUserSyncThread] - addOrUpdateGroup(): group = students users = [cn=andrew,ou=sds,dc=air,dc=org, cn=ranger,ou=sds,dc=air,dc=org]
22 May 2018 16:32:46 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - addOrUpdateGroup for students with users: [cn=andrew,ou=sds,dc=air,dc=org, cn=ranger,ou=sds,dc=air,dc=org]
22 May 2018 16:32:46 DEBUG AbstractJavaKeyStoreProvider [UnixUserSyncThread] - backing jks path initialized to file:/usr/hdp/current/ranger-usersync/conf/ugsync.jceks
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - RESPONSE for /service/xusers/groupusers/groupName/students: [{"createDate":null,"updateDate":null}]
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - addUsers = [cn=andrew,ou=sds,dc=air,dc=org, cn=ranger,ou=sds,dc=air,dc=org]
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - ==> LdapPolicyMgrUserGroupBuilder.delXGroupUserInfo students and []
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - ==> LdapPolicyMgrUserGroupBuilder.addGroupUserInfo students and [cn=andrew,ou=sds,dc=air,dc=org, cn=ranger,ou=sds,dc=air,dc=org]
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - INFO: addPMXAGroup(students)
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - INFO: addPMXAGroupToUser(students,cn=andrew,ou=sds,dc=air,dc=org)
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - INFO: addPMXAGroupToUser(students,cn=ranger,ou=sds,dc=air,dc=org)
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - GROUP USER MAPPING{"xgroupInfo":{"name":"students","description":"students - add from Unix box","groupType":"1","groupSource":"1"},"xuserInfo":[{"name":"cn\u003dandrew,ou\u003dsds,dc\u003dair,dc\u003dorg","description":"cn\u003dandrew,ou\u003dsds,dc\u003dair,dc\u003dorg - add from Unix box","groupNameList":[],"userRoleList":[]},{"name":"cn\u003dranger,ou\u003dsds,dc\u003dair,dc\u003dorg","description":"cn\u003dranger,ou\u003dsds,dc\u003dair,dc\u003dorg - add from Unix box","groupNameList":[],"userRoleList":[]}]}
22 May 2018 16:32:47 DEBUG LdapPolicyMgrUserGroupBuilder [UnixUserSyncThread] - RESPONSE: [{"createDate":null,"updateDate":null,"xuserInfo":[]}]
22 May 2018 16:32:47  INFO UserGroupSync [UnixUserSyncThread] - End: initial load of user/group from source==>sink

[#3]

@andrew chen Just in case please check what value you have for ranger.usersync.policymanager.mockrun (this should be set to false)

[#4]

Yes. It was set to 'false'

[#5]

In case you ran user sync multiple times and move/removed users from Ranger, disable incremental sync in Ranger and run again.

I find the Ranger LDAP connection tool to be very useful in these scenarios:

https://cwiki.apache.org/confluence/display/RANGER/LDAP+Connection+Check+Tool

[#6]

Yes. That tool works fine and returns the same result as the log file.

[#7]

Yes. The mock run is set to 'false'. The ldap tool returned perfect result. All users and group are retrieved using the tool. Applying the same properties and I retrieved all shown in the log but not in the database as I logged into mySQL and queried the x_user table and x_group table.

[#8]

Finally figured out: Ranger is looking for attribute uid. My users all have cn rather than uid and therefore it did retrieve the users and groups from LDAP but not inserted in the database. As far as group goes when there is no user with uid attributes in the group the group is fetched but not saved to ranger.