Configure LdapGroupsMapping in ClouderaManager

[#1]

Currently I have a problem with configuring the LdapGroupsMapping, I cannot get the groups of the users and it generates the following error message with the command:

$hadoop org.apache.hadoop.security.UserGroupInformation

20/03/19 16:44:20 WARN security.LdapGroupsMapping: Failed to get groups for user c12345a

(retry=0) by javax.naming.NamingException: [LDAP: error code 1 - 000004DC: LdapErr: DSID-0C090A4C, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v3839]; remaining name 'DC=sub,DC=us,DC=domain,DC=local'

$hdfs dfsadmin -refreshUserToGroupsMappings
Refresh user to groups mapping successful

$hdfs groups
c12345a@SUB.US.DOMAIN.LOCAL :

the core-site.xml configuration is as follows

-<property>
<name>hadoop.security.group.mapping</name>
<value>org.apache.hadoop.security.LdapGroupsMapping</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.url</name>
<value>ldap://sub.us.domain.local:389</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.use.ssl</name>
<value>true</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.ssl.keystore</name>
<value>{{CMF_CONF_DIR}}/cm-auto-global_truststore.jks</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.bind.user</name>
<value>ClouderaManager@SUB.US.DOMAIN.LOCAL</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.base</name>
<value>DC=sub,DC=us,DC=domain,DC=local</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.search.filter.user</name>
<value>(&(objectClass=user)(sAMAccountName={0}))</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.search.filter.group</name>
<value>(objectClass=group)</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.search.attr.member</name>
<value>member</value>
</property>
-<property>
<name>hadoop.security.group.mapping.ldap.search.attr.group.name</name>
<value>cn</value>

Thank you for your help 

[#2]

@WilsonLozano,

I believe the error you are getting indicates that the bind user defined in hadoop.security.group.mapping.ldap.bind.user does not existing in the LDAP server, but I didn't search online for confirmation.

You could try using ldapsearch to test something like this:

ldapsearch -x -H ldap://sub.us.domain.local:389 -D "ClouderaManager@SUB.US.DOMAIN.LOCAL" -W -b "DC=sub,DC=us,DC=domain,DC=local" "(&(objectClass=user)(sAMAccountName=c12345a))"

If the above returns an error, you can try using debugging in ldapsearch to get a clearer picture what failed by using the "-d1" option in the command above (after -W for instance).

[#3]

Thank you @bgooley for answering, I have been with this problem for a month.

Perform the test with the ldapsearch command and the result was as follows:

[c12345a@servernode ~]$ ldapsearch -x -H ldap://sub.us.domain.local:389 -D "ClouderaManager@SUB.US.DOMAIN.LOCAL" -W -b "DC=sub,DC=us,DC=domain,DC=local" "(&(objectClass=user)(sAMAccountName=c12345a))"
Enter LDAP Password:
# extended LDIF
#
# LDAPv3
# base <DC=sub,DC=us,DC=domain,DC=local> with scope subtree
# filter: (&(objectClass=user)(sAMAccountName=c12345a))
# requesting: ALL
#

# Charlie Brown, Computec, General Accounts, Accounts, sub.us.domain.local
dn: CN=Charlie Brown,OU=Computec,OU=General Accounts,OU=Accounts,DC=sub,DC=us,
DC=domain,DC=local
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: Charlie Brown
sn: Brown
c: C
l: city D.C.
st: city
title: Admin
postalCode: 571
physicalDeliveryOfficeName: CO city principal office
givenName: Charlie
distinguishedName: CN=Charlie Brown,OU=Computec,OU=General Accounts,OU=Accounts,DC=sub,DC=us,DC=domain,DC=local
instanceType: 4
whenCreated: 20191128190334.0Z
whenChanged: 20200318065357.0Z
displayName: Brown, Charlie
uSNCreated: 36937614
memberOf: CN=Unix_KafkaServer04_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer04_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer03_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer03_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer02_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer02_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer01_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer01_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer04_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer03_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer02_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Unix_KafkaServer01_users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=kfk_adm_Group,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=kfk_group,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=APP-MyPeopleExperience-Users,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=APP-IDCentral_SkypeOnline_DesktopSharing,OU=Identity Central,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=ADMIN DomainLATAM Centrify KAFKA,OU=UserRoleGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=APP-IDCentral_Remote_mailbox,OU=Identity Central,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=GlobalDLP,OU=Global DLP Groups,OU=Groups,DC=us,DC=domain,DC=local
memberOf: CN=APP-Sailpoint-IIQ-Users,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Allow External Email Access,OU=AzureSync,OU=Secured Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=O365.LicenseSkypefB,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=APP-Webex_Users,OU=AzureSync,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=O365.LicenseSharepointOnline,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=O365.LicenseExchangeOnlineE3,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=O365.LicenseBasicServicesE3andEMS,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=O365.LicenseTeams,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=city - Internal Users Street ,OU=O365 DL Migration Project - Exclusions,OU=Distribution Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=GN_CO_Internet_Basic,OU=Computec,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=Archer.SUB_Users,OU=Archer Groups,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
memberOf: CN=UNIFY SUB Authorized Users,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
uSNChanged: 74565754
co: CO
department: Sp Latam:Global
company: Domain S.A
proxyAddresses: x500:/o=ExchangeLabs/ou=Exchange Administrative Group (FBOH3SPDLT)/cn=Recipients/cn=9812d54ca41c79c8585597c61dc85-Charlie Brown
proxyAddresses: smtp:DMN-Charlie.Brown@domain.mail.onmicrosoft.com
proxyAddresses: SMTP:Charlie.Brown@domain.com
targetAddress: SMTP:DMN-Charlie.Brown@domain.mail.onmicrosoft.com
extensionAttribute1: 7282705
extensionAttribute2: 601
extensionAttribute6: CO city Oficina principal
extensionAttribute7: Contingent
extensionAttribute8: Contingent Worker
extensionAttribute9: Worker
mailNickname: C12345A
extensionAttribute12: NAC_
extensionAttribute13: C12345A@sub.us.domain.local
extensionAttribute15: NAC
name: Charlie Brown
objectGUID:: D8xfdlcMGkmuEghNTYiA==
userAccountControl: 512
badPwdCount: 0
codePage: 0
countryCode: 0
badPasswordTime: 132283366751489998
lastLogoff: 0
lastLogon: 132294797156347726
pwdLastSet: 132282329603092576
primaryGroupID: 513
userParameters:: bXR4Q2ZnUHJlc2VudCAgICAgICA
objectSid:: AQUAAAAAAAUVAAhkqT5EAb4UEAA==
accountExpires: 132507324000000000
logonCount: 1814
sAMAccountName: C12345A
sAMAccountType: 8030638
showInAddressBook: CN=Default Global Address List,CN=All Global Address Lists,CN=Address Lists Container,CN=Domain,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer04_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer04_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer03_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer03_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer02_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer02_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer01_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer01_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer04_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer03_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer02_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
managedObjects: CN=Unix_KafkaServer01_users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local
legacyExchangeDN: /o=Domain/ou=Exchange Administrative Group (FYDI23SPDL
T)/cn=Recipients/cn=Charlie Brown
userPrincipalName: Charlie.Brown@domain.com
lockoutTime: 0
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=domain,DC=local
msRADIUSFramedIPAddress: 17419
msRASSavedFramedIPAddress: 17419
dSCorePropagationData: 16010101000000.0Z
mS-DS-ConsistencyGuid:: D8xfdlcMGkmuEghNTYiyvA==
lastLogonTimestamp: 132289880375204341
msDS-ExternalDirectoryObjectId: User_3821ddab-be44-4cd5-956d-bdcb648ddbb7
mail: Charlie.Brown@domain.com
manager: CN=\,chief,OU=Infrastructure,OU=Technology,OU=Country,OU=Co
mputec,OU=General Accounts,OU=Accounts,DC=sub,DC=us,DC=domain,DC=local
middleName: Middle
msExchPoliciesExcluded: {261cfc-9e50-4857-861b-0cb8d7}
msExchArchiveStatus: 1
msExchRemoteRecipientType: 3
msExchVersion: 442
msExchUsageLocation: CO
msExchUMDtmfMap: emailAddress:945766
msExchUMDtmfMap: lastNameFirstName:56926694
msExchUMDtmfMap: firstNameLastName:945766566
msExchArchiveGUID:: ajkdQhlawkK6SQ3heE/lA==
msExchRecipientDisplayType: -21474842
msExchArchiveName: Personal Archive - Charlie Brown
msExchRecipientTypeDetails: 21474848

# search reference
ref: ldap://DomainDnsZones.sub.us.domain.local/DC=DomainDnsZones,DC=sub,DC=u
s,DC=domain,DC=local

# search result
search: 2
result: 0 Success

# numResponses: 3
# numEntries: 1
# numReferences: 1

The clouderamanager user has the possibility to consult the AD, what other problem could it be?

Thanks for the help

[#4]

Hi @WilsonLozano ,

Based on the fact that the ldapsearch command returned the object without issue, we can conclude that the bind user and password are correct.

Thus, I believe we can assume that the issue may involve referrals and how they are being followed.  I find this odd since I believe that ldapgroupsmapping should have referral following off by default.

Nonetheless, we see in your ldapsearch result:

ref: ldap://DomainDnsZones.sub.us.domain.local/DC=DomainDnsZones,DC=sub,DC=u
s,DC=domain,DC=local

So, what I would suggest trying is either:

• Change your search base to something more specific like "OU=Accounts,DC=sub,DC=us,
  DC=domain,DC=local" so that no referral is returned from Active Directory
• Try using the Global Catalog (port 3268 (non-TLS))

I am pretty confident that referrals are involved, but I don't know why hadoop commons would be following them.

Another thing you could do is use "tcpdump" to capture packets on port 389 and then use WireShark to decode them.  That would show us exactly what the client is trying to do and the response (in terms of LDAP protocol).

[#5 · Accepted solution]

Ok we finally got it.

The downside is that when setting the hadoop.security.group.mapping.ldap.bind.password.file property, it did not update the core-site.xml file.

Perform the following procedure:

https://community.cloudera.com/t5/Community-Articles/Secure-LDAP-bind-password-in-Hadoop-Configurati...

And add the property in Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml of hdfs in cloudera manager

Thank you very much @bgooley for your help