Member since
06-16-2022
3
Posts
0
Kudos Received
0
Solutions
06-17-2022
06:55 AM
We dont have much information in logs. we refer stderr logs...PFA logs for the failed instance.
... View more
06-17-2022
02:56 AM
Hi @mszurap , we are using CDP version 7.1.7, this is an intermittent issue, encountering twice a week. This is a single job(ingestion) where we are processing files(parquet) using shell scripts. Insertion of these files will go through map reduce.
... View more
06-16-2022
10:51 PM
Application application_1655291383854_84179 failed 2 times due to AM Container for appattempt_1655291383854_84179_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2022-06-17 00:49:32.617]Exception from container-launch. Container id: container_e1071_1655291383854_84179_02_000001 Exit code: 1 Exception message: Launch container failed Shell output: main : command provided 1 main : run as user is sapproep main : requested yarn user is sapproep Getting exit code file... Creating script paths... Writing pid file... Writing to tmp file /data/data02/yarn/nm/nmPrivate/application_1655291383854_84179/container_e1071_1655291383854_84179_02_000001/container_e1071_1655291383854_84179_02_000001.pid.tmp Writing to cgroup task files... Creating local dirs... Launching container... [2022-06-17 00:49:32.619]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error un-registering AM client java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) [2022-06-17 00:49:32.623]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error un-registering AM client java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) For more detailed output, check the application tracking page: https://itsusralsp07967.genericcompany.com:8090/cluster/app/application_1655291383854_84179 Then click on links to logs of each attempt. . Failing the application.
... View more
Labels: