Oozie job failed due to below error.

PCP2 — Fri, 17 Jun 2022 09:42:37 GMT

Application application_1655291383854_84179 failed 2 times due to AM Container for appattempt_1655291383854_84179_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2022-06-17 00:49:32.617]Exception from container-launch. Container id: container_e1071_1655291383854_84179_02_000001 Exit code: 1 Exception message: Launch container failed Shell output: main : command provided 1 main : run as user is sapproep main : requested yarn user is sapproep Getting exit code file... Creating script paths... Writing pid file... Writing to tmp file /data/data02/yarn/nm/nmPrivate/application_1655291383854_84179/container_e1071_1655291383854_84179_02_000001/container_e1071_1655291383854_84179_02_000001.pid.tmp Writing to cgroup task files... Creating local dirs... Launching container... [2022-06-17 00:49:32.619]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error un-registering AM client java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) [2022-06-17 00:49:32.623]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error un-registering AM client java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.unregisterApplicationMaster(AMRMClientImpl.java:613) at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.unregisterApplicationMaster(AMRMClientAsyncImpl.java:220) at org.apache.oozie.action.hadoop.LauncherAM.unregisterWithRM(LauncherAM.java:359) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:285) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144) For more detailed output, check the application tracking page: https://itsusralsp07967.genericcompany.com:8090/cluster/app/application_1655291383854_84179 Then click on links to logs of each attempt. . Failing the application.

Re: Oozie job failed due to below error.

mszurap — Fri, 17 Jun 2022 08:34:20 GMT

Hi @PCP2 , can you clarify which HDP/CDH/CDP version are you using?

Is this a one-off or an intermittent issue or does it always happen?
Is this affecting only a single job? What kind of an action is Oozie trying to launch?

Thanks, Miklos

Re: Oozie job failed due to below error.

PCP2 — Fri, 17 Jun 2022 09:56:34 GMT

Hi @mszurap , we are using CDP version 7.1.7, this is an intermittent issue, encountering twice a week.

This is a single job(ingestion) where we are processing files(parquet) using shell scripts. Insertion of these files will go through map reduce.

Re: Oozie job failed due to below error.

mszurap — Fri, 17 Jun 2022 10:52:07 GMT

Can you review the whole logfile? The above NPE may be just a side effect of another failure before.

Re: Oozie job failed due to below error.

PCP2 — Fri, 17 Jun 2022 13:55:02 GMT

We dont have much information in logs. we refer stderr logs...PFA logs for the failed instance.

Re: Oozie job failed due to below error.

Krisssh — Mon, 15 Aug 2022 20:36:48 GMT

@PCP2 Were you able to get any leads on this issue ?

Re: Oozie job failed due to below error.

DianaTorres — Mon, 22 Jan 2024 18:50:51 GMT

@PCP2 @Krisssh Hello, we cannot work out such issues through a community post, we recommend to open a case to look in detail for each case. Thanks!

Re: Oozie job failed due to below error.

RaghunathJosula — Mon, 22 Jan 2024 15:35:47 GMT

Dear All,

I am also facing the same issue with CDP 7.1.7 the oozie job is failing and being killed with the above error.

If any case has been opened please post reply to the same one, this is long pending issue what i am facing with the same log.

Thanks,

Raghunath Josula.

Re: Oozie job failed due to below error.

Raghunath_CDH — Tue, 27 Feb 2024 16:06:02 GMT

Dear Support what kind of case we should open I don't have any information on this issue please respond.

Re: Oozie job failed due to below error.

duhizjame — Wed, 19 Jun 2024 08:37:03 GMT

We are experiencing the same issue on CDP 7.1.7 calling a spark job from Oozie.

question Re: Oozie job failed due to below error. in Support Questions

Oozie job failed due to below error.

Re: Oozie job failed due to below error.

Re: Oozie job failed due to below error.

Re: Oozie job failed due to below error.

Re: Oozie job failed due to below error.

Re: Oozie job failed due to below error.

Re: Oozie job failed due to below error.

Re: Oozie job failed due to below error.

Re: Oozie job failed due to below error.

Re: Oozie job failed due to below error.