About WilsonLozano

WilsonLozano · ‎04-28-2020

Hi community I am executing a job spark, but after 187 hours of execution it generates the following error: Application application_1584698544596_3421 failed 2 times due to AM Container for appattempt_1584698544596_3421_000002 exited with exitCode: 1 For more detailed output, check application tracking page:http://server1.corp:8088/proxy/application_1584698544596_3421/Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_e103_1584698544596_3421_02_000001 Exit code: 1 Stack trace: ExitCodeException exitCode=1: at org.apache.hadoop.util.Shell.runCommand(Shell.java:604) at org.apache.hadoop.util.Shell.run(Shell.java:507) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:789) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.__launchContainer__(LinuxContainerExecutor.java:399) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Shell output: main : command provided 1 main : run as user is development main : requested yarn user is development Writing to tmp file /hdfs5/yarn/nm/nmPrivate/application_1584698544596_3421/container_e103_1584698544596_3421_02_000001/container_e103_1584698544596_3421_02_000001.pid.tmp Writing to cgroup task files... Container exited with a non-zero exit code 1 Failing this attempt. Failing the application. And it is not possible to see the logs either because it generates this error: Logs not available for container_e103_1584698544596_3421_01_000001. Aggregation may not be complete, Check back later or try the nodemanager at server40.corp:8041 Or see application log at http://server40.corp:8041/node/application/application_1584698544596_3421 I appreciate your help, because I don't know what can happen

WilsonLozano · ‎03-31-2020

Ok we finally got it. The downside is that when setting the hadoop.security.group.mapping.ldap.bind.password.file property, it did not update the core-site.xml file. Perform the following procedure: https://community.cloudera.com/t5/Community-Articles/Secure-LDAP-bind-password-in-Hadoop-Configuration/ta-p/247789 And add the property in Cluster-wide Advanced Configuration Snippet (Safety Valve) for core-site.xml of hdfs in cloudera manager Thank you very much @bgooley for your help

WilsonLozano · ‎03-23-2020

Thank you @bgooley for answering, I have been with this problem for a month. Perform the test with the ldapsearch command and the result was as follows: [c12345a@servernode ~]$ ldapsearch -x -H ldap://sub.us.domain.local:389 -D "ClouderaManager@SUB.US.DOMAIN.LOCAL" -W -b "DC=sub,DC=us,DC=domain,DC=local" "(&(objectClass=user)(sAMAccountName=c12345a))" Enter LDAP Password: # extended LDIF # # LDAPv3 # base <DC=sub,DC=us,DC=domain,DC=local> with scope subtree # filter: (&(objectClass=user)(sAMAccountName=c12345a)) # requesting: ALL # # Charlie Brown, Computec, General Accounts, Accounts, sub.us.domain.local dn: CN=Charlie Brown,OU=Computec,OU=General Accounts,OU=Accounts,DC=sub,DC=us, DC=domain,DC=local objectClass: top objectClass: person objectClass: organizationalPerson objectClass: user cn: Charlie Brown sn: Brown c: C l: city D.C. st: city title: Admin postalCode: 571 physicalDeliveryOfficeName: CO city principal office givenName: Charlie distinguishedName: CN=Charlie Brown,OU=Computec,OU=General Accounts,OU=Accounts,DC=sub,DC=us,DC=domain,DC=local instanceType: 4 whenCreated: 20191128190334.0Z whenChanged: 20200318065357.0Z displayName: Brown, Charlie uSNCreated: 36937614 memberOf: CN=Unix_KafkaServer04_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer04_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer03_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer03_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer02_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer02_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer01_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer01_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer04_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer03_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer02_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Unix_KafkaServer01_users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=kfk_adm_Group,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=kfk_group,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=APP-MyPeopleExperience-Users,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=APP-IDCentral_SkypeOnline_DesktopSharing,OU=Identity Central,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=ADMIN DomainLATAM Centrify KAFKA,OU=UserRoleGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=APP-IDCentral_Remote_mailbox,OU=Identity Central,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=GlobalDLP,OU=Global DLP Groups,OU=Groups,DC=us,DC=domain,DC=local memberOf: CN=APP-Sailpoint-IIQ-Users,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Allow External Email Access,OU=AzureSync,OU=Secured Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=O365.LicenseSkypefB,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=APP-Webex_Users,OU=AzureSync,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=O365.LicenseSharepointOnline,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=O365.LicenseExchangeOnlineE3,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=O365.LicenseBasicServicesE3andEMS,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=O365.LicenseTeams,OU=Office365 Licensing,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=city - Internal Users Street ,OU=O365 DL Migration Project - Exclusions,OU=Distribution Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=GN_CO_Internet_Basic,OU=Computec,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=Archer.SUB_Users,OU=Archer Groups,OU=Application Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local memberOf: CN=UNIFY SUB Authorized Users,OU=General Groups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local uSNChanged: 74565754 co: CO department: Sp Latam:Global company: Domain S.A proxyAddresses: x500:/o=ExchangeLabs/ou=Exchange Administrative Group (FBOH3SPDLT)/cn=Recipients/cn=9812d54ca41c79c8585597c61dc85-Charlie Brown proxyAddresses: smtp:DMN-Charlie.Brown@domain.mail.onmicrosoft.com proxyAddresses: SMTP:Charlie.Brown@domain.com targetAddress: SMTP:DMN-Charlie.Brown@domain.mail.onmicrosoft.com extensionAttribute1: 7282705 extensionAttribute2: 601 extensionAttribute6: CO city Oficina principal extensionAttribute7: Contingent extensionAttribute8: Contingent Worker extensionAttribute9: Worker mailNickname: C12345A extensionAttribute12: NAC_ extensionAttribute13: C12345A@sub.us.domain.local extensionAttribute15: NAC name: Charlie Brown objectGUID:: D8xfdlcMGkmuEghNTYiA== userAccountControl: 512 badPwdCount: 0 codePage: 0 countryCode: 0 badPasswordTime: 132283366751489998 lastLogoff: 0 lastLogon: 132294797156347726 pwdLastSet: 132282329603092576 primaryGroupID: 513 userParameters:: bXR4Q2ZnUHJlc2VudCAgICAgICA objectSid:: AQUAAAAAAAUVAAhkqT5EAb4UEAA== accountExpires: 132507324000000000 logonCount: 1814 sAMAccountName: C12345A sAMAccountType: 8030638 showInAddressBook: CN=Default Global Address List,CN=All Global Address Lists,CN=Address Lists Container,CN=Domain,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer04_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer04_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer03_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer03_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer02_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer02_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer01_kfk_group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer01_kfk_adm_Group,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer04_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer03_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer02_Users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local managedObjects: CN=Unix_KafkaServer01_users,OU=UnixSecondaryGroups,OU=DomainLATAM,OU=UnixGroups,OU=Groups,DC=sub,DC=us,DC=domain,DC=local legacyExchangeDN: /o=Domain/ou=Exchange Administrative Group (FYDI23SPDL T)/cn=Recipients/cn=Charlie Brown userPrincipalName: Charlie.Brown@domain.com lockoutTime: 0 objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=domain,DC=local msRADIUSFramedIPAddress: 17419 msRASSavedFramedIPAddress: 17419 dSCorePropagationData: 16010101000000.0Z mS-DS-ConsistencyGuid:: D8xfdlcMGkmuEghNTYiyvA== lastLogonTimestamp: 132289880375204341 msDS-ExternalDirectoryObjectId: User_3821ddab-be44-4cd5-956d-bdcb648ddbb7 mail: Charlie.Brown@domain.com manager: CN=\,chief,OU=Infrastructure,OU=Technology,OU=Country,OU=Co mputec,OU=General Accounts,OU=Accounts,DC=sub,DC=us,DC=domain,DC=local middleName: Middle msExchPoliciesExcluded: {261cfc-9e50-4857-861b-0cb8d7} msExchArchiveStatus: 1 msExchRemoteRecipientType: 3 msExchVersion: 442 msExchUsageLocation: CO msExchUMDtmfMap: emailAddress:945766 msExchUMDtmfMap: lastNameFirstName:56926694 msExchUMDtmfMap: firstNameLastName:945766566 msExchArchiveGUID:: ajkdQhlawkK6SQ3heE/lA== msExchRecipientDisplayType: -21474842 msExchArchiveName: Personal Archive - Charlie Brown msExchRecipientTypeDetails: 21474848 # search reference ref: ldap://DomainDnsZones.sub.us.domain.local/DC=DomainDnsZones,DC=sub,DC=u s,DC=domain,DC=local # search result search: 2 result: 0 Success # numResponses: 3 # numEntries: 1 # numReferences: 1 The clouderamanager user has the possibility to consult the AD, what other problem could it be? Thanks for the help

WilsonLozano · ‎03-19-2020

Currently I have a problem with configuring the LdapGroupsMapping, I cannot get the groups of the users and it generates the following error message with the command: $hadoop org.apache.hadoop.security.UserGroupInformation 20/03/19 16:44:20 WARN security.LdapGroupsMapping: Failed to get groups for user c12345a (retry=0) by javax.naming.NamingException: [LDAP: error code 1 - 000004DC: LdapErr: DSID-0C090A4C, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v3839]; remaining name 'DC=sub,DC=us,DC=domain,DC=local' $hdfs dfsadmin -refreshUserToGroupsMappings Refresh user to groups mapping successful $hdfs groups c12345a@SUB.US.DOMAIN.LOCAL : the core-site.xml configuration is as follows -<property> <name>hadoop.security.group.mapping</name> <value>org.apache.hadoop.security.LdapGroupsMapping</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.url</name> <value>ldap://sub.us.domain.local:389</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.use.ssl</name> <value>true</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.ssl.keystore</name> <value>{{CMF_CONF_DIR}}/cm-auto-global_truststore.jks</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.bind.user</name> <value>ClouderaManager@SUB.US.DOMAIN.LOCAL</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.base</name> <value>DC=sub,DC=us,DC=domain,DC=local</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.search.filter.user</name> <value>(&(objectClass=user)(sAMAccountName={0}))</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.search.filter.group</name> <value>(objectClass=group)</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.search.attr.member</name> <value>member</value> </property> -<property> <name>hadoop.security.group.mapping.ldap.search.attr.group.name</name> <value>cn</value> Thank you for your help

WilsonLozano · ‎02-19-2020

@Shelton I have the same problem but with cloudera, do you know what procedure I should follow to configure the zookeeper ACL but with kafka and sentry? Thank you

Online	Offline
Last Visited	‎11-04-2020 06:39 AM

Member Since	‎01-20-2020 07:13 AM
Last Visited	‎11-04-2020 06:39 AM
Posts	8
Kudos received	1

Cloudera Community

Cloudera Community

Re: Configure LdapGroupsMapping in ClouderaManager

Job Spark in yarn execution failed

Re: Configure LdapGroupsMapping in ClouderaManager

Re: Configure LdapGroupsMapping in ClouderaManager

Configure LdapGroupsMapping in ClouderaManager

Re: Topic creation and deletion are not protected ...