Reply
Highlighted
Explorer
Posts: 14
Registered: ‎01-29-2018

Nodemanager.containermanager.localizer.ResourceLocalizationService [exitCode=137]

Hi all,

 

I’m having some problems with only one server in my Hadoop Infrastructure (worker node), all the jobs that Yarn schedules to this server to be elaborated always fail...

 

Any idea about the root cause of this issue? At the moment I've stopped the Yarn service of this node. Below the Exception that I got:

 

----------------------------------------------------------------------------------------------------------------------------------------------------

org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService              

Localizer failed

java.io.IOException: Application application_1516880199882_19310 initialization failed (exitCode=137) with output:

               at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:269)

               at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1133)

Caused by: ExitCodeException exitCode=137:

               at org.apache.hadoop.util.Shell.runCommand(Shell.java:578)

               at org.apache.hadoop.util.Shell.run(Shell.java:481)

               at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:763)

               at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:260)

               ... 1 more

-----------------------------------------------------------------------------------------------------------------------------------------

WARN   org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor  

Exit code from container container_e111_1516880199882_19310_02_002043 startLocalizer is : 137

ExitCodeException exitCode=137:

               at org.apache.hadoop.util.Shell.runCommand(Shell.java:578)

               at org.apache.hadoop.util.Shell.run(Shell.java:481)

               at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:763)

               at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:260)

               at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1133)

 ----------------------------------------------------------------------------------------------------------------------------------------------------

 

If I restart the service works for a few days, but suddenly this "Exception -> ExitCodeException exitCode=137" appear again…

 

 

### Permission of container executor ###

---Sr-s--- 1 root yarn 49520 Jul 13  2016 /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop-yarn/bin/container-executor

 

Many thanks in advance for the kind cooperation!

 

Regards,

 

Alex

Announcements