Hello We are using Ranger & Usersync from LDAP in order to manage permissions for our hdp data. Everything works fine while managing users - LDAP users appear in ranger, adding them to a policy, giving the appropriate permissions works flawless. However, when adding a group to a policy instead of users - the users of that group fail to query data even though their LDAP group has been given the right permissions. Just to clarify: Usersync works without any problems and users & groups appear in Ranger. It's just that using groups instead of users - is not working. So if i have a team of 10 users that need access to a data in my HDP - i have to specify those 10 users with select permissions. If i use an LDAP group that holds these users - they get permission denied error. Anyone has an idea why ? Maybe it's a "universal" vs "security" group in LDAP ? Adi J.
By default Hadoop will get the user/group mappings from the Linux OS. So if you haven't configured your LDAP users and groups to identify on each OS node, Hadoop won't be able to enforce your group ranger policies. Refer to this document for details:
A quick way to ensure your setup is correct is to run the 'hdfs groups' command as one of your users. This will output the groups the user belongs to - if you don't see your LDAP groups reflected here then Hadoop/HDFS doesn't know about the group mappings.
I'm facing literally the same problem. I'm wondering if it's because the OS recognises the groups by ID instead of name. But not sure how to prove this.
What happens when you do:
hdfs groups <username>
Does it show the same groups as what is in Ranger?
Because for me it shows an empty set of groups. Where as in the Ranger UI, the groups are listed.
daleb@mh0edge01:~$ hdfs groups daleb daleb :
A couple of things to check: 1. Do you get the correct group mappings when you run 'id <user>' on the OS? 2. Do your users/groups exist on all nodes? (HDFS will take the group mappings from the Namenode)
$ id daleb uid=11160(daleb) gid=11000(domain users) groups=11000(domain users),11122(hadoop users)
This is the correct mappings.
2. Yes users/groups exist on all nodes and are synced using Winbind/Samba.
# hdfs dfsadmin -refreshUserToGroupsMappings Refresh user to groups mapping successful for nn1:8020 Refresh user to groups mapping successful for nn2:8020 # hdfs groups daleb daleb :
I think the problem may be caused by the space in your group names. Please refer to this JIRA: https://issues.apache.org/jira/browse/HADOOP-12505
Could you try adding a group without a space and confirm if that resolves in HDFS?
I have changed my AD groups so that they do not contain spaces. This has successfully synchronized the groups on hdfs. The new non-spaces groups now appear in the Ranger UI. I have changed the Hive policy to reflect the groups permissions however I am still unable to query Hive as a group member. However, I can query Hive as a user.
@Sagar Shimpi @Laurence Da Luz Thank you both for responding ! Highly appreciated ! When i run 'hdfs groups' command on a user and received blank results. I went through the documentation you provided and added all the information to core-site (using Ambari) and restarted - but still same results. Maybe i should have noted that we use Ranger for Hive permissions - and not HDFS. Just Hive.
What i can't figure out is - in ranger the Active Directory groups appear, which means user-sync is configured correctly. The user-sync did sync users & groups from active directory, the problem is that using groups for hive permissions is useless - because for some reason the ranger (or hive) is unable to get the users that are inside a group.
The behaviour you're describing would be expected because your user/group mappings aren't set up correctly. To clarify my previous response, in order to have Ranger working with LDAP groups (for either Hive or HDFS) the Hadoop Group Mapping needs to be configured first. The user-sync configuration in Ranger is purely used to bring your AD user/group list into the Ranger UI. To actually have your Ranger policies enforced, Hadoop also needs to know which group each user belongs to (which at the moment you haven't configured).
You mentioned you have attempted to configure core-site with the LDAP mappings. In general we recommend configuring the mappings at the OS level (via SSSD or similar - as described in the linked doc), but the core-site.xml configuration will also work. I'd suggest checking that your LDAP search filter is correct in core-site - as you are using Active Directory your user search filter would look something like this (using sAMAccountName instead of cn):