hadoop group mapping with openldap

[#1]

I am trying to set Hadoop group mappings to make ranger policies work for groups.I have passed below properties to core-site but still groups are not fetched(Both on Sandbox and on prod cluster as well):

hadoop.security.group.mapping.ldap.url=ldap://xxxx:389
hadoop.security.group.mapping.ldap.search.filter.user=(&(objectClass=posixAccount)(uid={0}))
hadoop.security.group.mapping.ldap.search.filter.group=(objectClass=group) [also tried with (objectClass=groupOfNames)]
hadoop.security.group.mapping.ldap.search.attr.member=member
hadoop.security.group.mapping.ldap.search.attr.group.name=cn
hadoop.security.group.mapping.ldap.bind.user=cn=Manager,dc=xyz-bigdata,dc=com
hadoop.security.group.mapping.ldap.bind.password=password
hadoop.security.group.mapping.ldap.base=dc=xyz-bigdata,dc=com
hadoop.security.group.mapping=org.apache.hadoop.security.LdapGroupsMapping

Below is the ldif file for my open ldap:

dn: dc=xyz-bigdata,dc=com
objectClass: organization
objectClass: dcObject
o: Hadoop
dc: xyz-bigdata


dn: ou=users,dc=xyz-bigdata,dc=com
objectClass: organizationalUnit
ou: users


dn: ou=groups,dc=xyz-bigdata,dc=com
objectClass: top
objectClass: organizationalUnit
ou: groups
description: xyz groups


# itpeople, groups, xyz-bigdata.com
dn: cn=itpeople,ou=groups,dc=xyz-bigdata,dc=com
objectClass: groupOfNames
cn: itpeople
description: IT security group
member: uid=testt,ou=users,dc=xyz-bigdata,dc=com
member: uid=asiddiqui,ou=users,dc=xyz-bigdata,dc=com
member: uid=root,ou=users,dc=xyz-bigdata,dc=com


# testt, users, xyz-bigdata.com
dn: uid=testt,ou=users,dc=xyz-bigdata,dc=com
objectClass: posixAccount
objectClass: top
objectClass: inetOrgPerson
gidNumber: 0
givenName: testt
initials: mr
sn: testt
displayName: testt
uid: testt
homeDirectory: /home/testt
loginShell: ksh
cn: testt
uidNumber: 4987


# hive, users, xyz-bigdata.com
dn: uid=hive,ou=users,dc=xyz-bigdata,dc=com
objectClass: posixAccount
objectClass: top
objectClass: inetOrgPerson
gidNumber: 0
givenName: System
initials: Mr.
sn: Hadoop
displayName: hive
uid: hive
homeDirectory: /home/hive
loginShell: ksh
cn: hive
uidNumber: 37807

[#2]

Hello @Anwaar Siddiqui,

I'm wondering if you are able to list you LDAP objects using the parameters that you defined in HDFS configuration. Please run these commands and share the output with us:

## To search for all users:
# ldapsearch -H ldap://xxxx:389 -x -D "cn=Manager,dc=xyz-bigdata,dc=com" -w "password" -b "dc=xyz-bigdata,dc=com" "(objectClass=posixAccount)"

## To search for all groups:
# ldapsearch -H ldap://xxxx:389 -x -D "cn=Manager,dc=xyz-bigdata,dc=com" -w "password" -b "dc=xyz-bigdata,dc=com" "(objectClass=groupOfNames)"

## To search for a user's group:
ldapsearch -H ldap://xxxx:389 -x -D "cn=Manager,dc=xyz-bigdata,dc=com" -w "password" -b "dc=xyz-bigdata,dc=com" "(&(objectClass=groupOfNames)(member=testt))"

Hope this helps !

[#3]

Also I'm assuming that you have restarted HDFS after making the configuration changes and you have executed 'hdfs dfsadmin -refreshUserToGroupsMappings' command once HDFS is fully restarted.

These two steps are critical to get this working.

[#4]

Hi Vipin Rathor

The first one is listing all users..second one list all groups but last one is empty..

dn: uid=student1,ou=users,dc=xyz-bigdata,dc=com
uid: student1
cn: student1
sn: 1
objectClass: top
objectClass: posixAccount
objectClass: inetOrgPerson
loginShell: /bin/bash
homeDirectory: /home/student1
uidNumber: 15000
gidNumber: 10000
userPassword:: e1NTSEF9Q1FHNUtIYzZiMWlpK3FvcGFWQ3NOYTE0djkrcjE0cjU=
mail: student1@xyz-bigdata.com
gecos: Student1 User
....list all 25 users

dn: cn=up_testgroup,ou=groups,dc=xyz-bigdata,dc=com
member: uid=test
member: uid=asaad,ou=users,dc=xyz-bigdata,dc=com
member: uid=ameen,ou=users,dc=xyz-bigdata,dc=com
member: uid=adevil,ou=users,dc=xyz-bigdata,dc=com
member: uid=hellobena,ou=users,dc=xyz-bigdata,dc=com
member: uid=hellobean2,ou=users,dc=xyz-bigdata,dc=com
cn: up_testgroup
objectClass: groupOfNames
...list all 4 groups

Last one is empty..

[#5]

Also where can i see the log as in the ranger log i can see what filters is getting generated...similar to that for this activity will help.