In order to utilize the directory server's nested groups information across hadoop, nested groups information must be retrieved for hadoop group mapping as well as for Ranger policy authorization. Retrieving nested group information for Ranger is explained here. This article covers the configuration required for retrieving nested group information for hadoop LdapGroupMapping. Nested group membership in LdapGroupMapping is introduced as part of HADOOP-12291 and is available in HDP 2.6. Let's take the same directory server structure example that is used for Ranger. Sample Active directory structure with Nested groups Usecase: Admin wants "Marketing Group" and "AMER Marketing Group" to be represented as hdfs groups for users "Adam Will", "John Doe", and "Mary Sam" In the above example, user "John Doe" is a member of "US Marketing Group". But the directory server structure also contains multiple nested group levels like - “US Marketing Group” is a member of “AMER Marketing Group” which again is a member of “Marketing Group”. With out nested group membership support on hadoop, "hdfs groups" for user "John Doe" returns only the immediate group "US Marketing Group". In order for hdfs groups to retrieve parent groups like "AMER Marketing Group" and "Marketing Group", then the LdapGroupMapping must be configured with nested group membership information. Hadoop LdapGroupMapping configuration ... View more

Introduction Adding a group as a member to another group is called nesting. If a group (groupA) is a member of another group (groupB), then the users belonging to the member group (groupA) are part of the parent group (groupB) as well. Nesting can be very useful in delegating access through inheritance. Several large enterprises have their groups in LDAP/AD nested within other groups. Security admins want the users in those nested groups to be associated in Ranger so that they are available for policy authoring in Ranger Admin. In HDP 2.6 (RANGER-1735), ranger usersync introduced support of nested group membership representation for policy authoring. Note:- In order to utilize the nested group mapping across hadoop, this feature must be configured for hadoop LdapGroupMapping as well. Configuring hadoop LdapGroupMapping for nested groups is explained here. Sample Active directory structure with Nested groups Usecase: Admin wants to give access to some resources for all the users under “AMER Marketing Group” In the above sample directory structure, all the marketing users are under one OU “Marketing Users”. All these users are members of different groups based on the location like US, Canada, London, etc… For example, user “Adam Will” from “Marketing Users” OU is a member of “Canada Marketing Group”. Also, the above sample directory structure contains multiple nested group levels like - “US Marketing Group” is a member of “AMER Marketing Group” which again is a member of “Marketing Group”. Ranger Usersync configuration Ranger Usersync, by default, computes only the immediate groups for the users. For example, user “Adam Will” is part of “Canada Marketing Group” and only this information is available in ranger without nested group sync configuration. With this information, if an admin wants to provide access to all the users under “AMER Marketing Group”, then all the sub groups - “US Marketing Group” and “Canada Marketing Group” must be added in the ranger policy. In order to simplify the policy configuration at parent level groups, Ranger supports evaluating nested group memberships by configuring “ranger.usersync.ldap.grouphierarchylevels”. If ranger.usersync.ldap.grouphierarchylevels is set to “3”, Ranger Usersync computes the group memberships for user “Adam Will” as “Canada Marketing Group”, “AMER Marketing Group”, “Marketing Group”. This way, admin can configure ranger policy at the parent group level (“AMER Marketing Group”) which will be applied for all the users (Mary Sam, John Doe, and Adam Will) under each sub group (US Marketing Group and Canada Marketing Group). ... View more

This article mainly focuses mainly on what all supported options available in User sync and how they can be used to target specific use cases. For some common use cases examples and how ranger can be configured in those cases, please refer to Configuring Ranger UserSync with AD/LDAP User based search without enable group sync: This configuration can be used in cases where customers want to sync users based on a OU or a user search filter and get all the groups the users belong to. This case is supported only when the user object has the group membership attributes like “memberof” or “ismemberof”. This option is available only in Full sync case (i.e., when Incremental sync is disabled in Ranger User sync configuration) User based search with enable group sync: This configuration can be used in cases where - customers want to sync users based on a OU or a user search filter and want to limit the groups that these user belong to based on the OU or a group search filter. Group membership attributes like “memberof” is not available for the user objects. This is the default option when “Incremental Sync” is enabled in Ranger Usersync configuration This option is available for both “Incremental Sync” or “Full Sync” Group based search without enable user sync (RANGER-869): This option is available for both “Incremental Sync” or “Full Sync” This option cannot be used in cases where customers want to sync the users with “sAMAccountName” instead if “CN” This is the default option when “Group Search First Enable” is set to “true” This configuration can be used in cases where - customers want to sync groups based on a OU or a group search filter and get all the users that belong to those groups. customers want to sync the groups that don’t have any users Group based search with enable user sync (RANGER-869): This configuration can be used in cases where customers want to sync groups based on a OU or a group search filter and limit the users that belong to those groups based on a OU or a user search filter. customers want to sync groups based on a OU or a group search filter and use the “sAMAccountName” for the user instead of “CN” customers want to sync the groups that don’t have any users This option is available for both “Incremental Sync” or “Full Sync” Multiple OUs for User search base and/or Group search base (RANGER-803): This configuration can be used in cases where customers want to sync users and/or groups from multiple OUs Multiple OUs can be specified in the User search base and/or Group search base using “;” separated like ou=OU1,dc=hortonworks,dc=com;ou=OU2,dc=hortonworks,dc=com;ou=OU3,dc=hortonworks,dc=com Multiple OUs are supported for both User based search and Group based search Incremental Sync or Full sync (RANGER-1211): Incremental Sync - Ranger Usersync can be configured to perform full sync only during startup and incremental sync for the subsequent sync cycles. This is the default option for fresh installs of HDP version 2.6 onwards With this option, “Enable Group Sync” is mandatory This option is highly recommended in order to improve usersync performance and decrease the startup time of Ranger. Full sync - Ranger Usersync can be configured to perform full sync from AD/LDAP for every sync cycle. This is the default option for clusters upgraded to 2.6 or higher Username and/or Groupname case conversion: Ranger usersync support three options for case conversion of username and/or groupname - “lower”, “upper”, and “none” “None” is the default option for case conversion This option must be configured to be same as the hdfs users/groups as ranger authorization is case sensitive for username and/or groupname. Ranger usersync properties for case conversion are: ranger.usersync.ldap.username.caseconversion ranger.usersync.ldap.groupname.caseconversion Username and/or Groupname transformation (RANGER-684): Ranger Usersync can be configured to perform username and/or groupname transformations in order to make them POSIX compliant. The value for these properties should be in the following format (similar to Sed format):“s/regex/replacement/g”s – stands for substitution and is mandatory/ - delimiter and is mandatoryRegex – regular expression to matchReplacement – value to replace and is optional. If not specified, the found pattern is removed from the resulting string.g – for replacing all the occurrences. It is optional and if not specified, default to “g” as well. In order to enable this feature, following properties should be added as custom usersync site properties in Ambari: ranger.usersync.mapping.username.handler - value as “org.apache.ranger.usergroupsync.RegEx” ranger.usersync.mapping.groupname.handler - value as “org.apache.ranger.usergroupsync.RegEx” ranger.usersync.mapping.username.regex - value must be in the format defined in section #b above ranger.usersync.mapping.groupname.regex - value must be in the format defined in #b above To configure multiple transformations/mappings: new regex properties can be added as: ranger.usersync.mapping.<attribute name>.<tranformation>.1, ranger.usersync.mapping.<attribute name>.<tranformation>.2, etc… All these transformations will be applied in the order they were configured. LDAP or LDAPS or STARTTLS (RANGER-722): Ranger Usersync support both secure and non-secure communication to AD/LDAP server for performing the sync operation. Ranger usersync can be configure with LDAPS or STARTTLS for secure communication For LDAP/LDAPS, ldap url must be configured accordingly with the corresponding port. For example - For non-secure/LDAP communication, sample ldap url value is “ldap://10.10.10.10:389” For secure with LDAPS communication, sample ldap url value is “ldaps://ad.hortonworks.COM:636” In order to configure Ranger Usersync to use STARTTLS instead of LDAPS - Add “ranger.usersync.ldap.starttls” with value “true” in custom usersync site properties in Ambari Set LdapURL property to be in the format “ldap://10.10.10.10:389” ... View more