Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Exception from container-launch

Exception from container-launch

New Contributor

Hi All,

 

I am using CDH 5 in RHEL6.2 64 bit environment.

I am getting the below error whenevever I am trying run a MapReduce job developed using Syncsort DMExpress 7.14.

 

14/09/04 16:35:11 INFO mapreduce.Job: Job job_1409827863691_0001 failed with state FAILED due to: Application application_1409827863691_0001 failed 2 times due to AM Container for appattempt_1409827863691_0001_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:511)
at org.apache.hadoop.util.Shell.run(Shell.java:424)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:656)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
14/09/04 16:35:11 INFO mapreduce.Job: Counters: 0
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:838)
at com.syncsort.dmexpress.hadoop.DMXRunner.main(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Job has aborted.

 

Any kind of help is appreciated. Please let me know for any more details.

 

Thanks,

Rudra

 

4 REPLIES 4

Re: Exception from container-launch

Master Guru
Can you check your Job's AM logs (stdout/stderr/syslogs) under /tmp/logs/<username of submitter>/application_1409827863691_0001/ or a similar newer directory? It should list the reason of failure more precisely.

Re: Exception from container-launch

New Contributor

Hi,

 

This is the error after checking /tmp/logs/username/--

 

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
��h��׶9�A@���P VERSIONAPPLICATION_ACLVIEW_APP
MODIFY_APP APPLICATION_OWNEcentos(&container_1471951144182_1802_02_000001��stderr1447Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 13 more
stdout0(&container_1471951144182_1802_01_000001��stderr1447Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.service.CompositeService
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 13 more
stdout0
VERSION*(&container_1471951144182_1802_01_000001none�1�1data:BCFile.indexnone�}

data:TFile.indexnone�G66data:TFile.metanone�A
���h��׶9�A@���P[centos@ip-**********~]$

 

Please help to resolve this issue.

Thanks in Advance,

Deepak

Re: Exception from container-launch

New Contributor

Hi All,

 

I an using CDH5 with hadoop version.I tried to run mapreduce on single cluster local mode and it works fine while same jar i am running ito CDH5 it  Application launcher fail.

 

Command is like-

hadoop jar /home/deepak/jar/count.jar com.scry.hadoop.WordCount /usr/input/wordcount.txt output1

 

when i run this command in to local it gives the result

16/08/25 14:46:19 INFO mapreduce.Job: map 0% reduce 0%
16/08/25 14:46:24 INFO mapreduce.Job: map 100% reduce 0%
16/08/25 14:46:29 INFO mapreduce.Job: map 100% reduce 100%
16/08/25 14:46:40 INFO mapreduce.Job: map 0% reduce 0%
16/08/25 14:46:40 INFO mapreduce.Job: Job job_1472112573155_0002 failed with state FAILED due to: Application application_1472112573155_0002 failed 2 times due to AM Container for appattempt_1472112573155_0002_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://deepak-HP-ProBook-450-G2:8088/cluster/app/application_1472112573155_0002Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1472112573155_0002_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
16/08/25 14:46:40 INFO mapreduce.Job: Counters: 0
job.isSuccessful false

 

 

while in cluster mode it gives error and Application launcher fails

 

16/08/25 09:41:10 INFO mapreduce.Job: map 0% reduce 0%
16/08/25 09:41:10 INFO mapreduce.Job: Job job_1471951144182_0399 failed with state FAILED due to: Application application_1471951144182_0399 failed 2 times due to AM Container for appattempt_1471951144182_0399_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://ip-172-31-4-45.us-west-2.compute.internal:8088/proxy/application_1471951144182_0399/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1471951144182_0399_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:543)
at org.apache.hadoop.util.Shell.run(Shell.java:460)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:720)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)


Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
16/08/25 09:41:10 INFO mapreduce.Job: Counters: 0
job.isSuccessful false

 

 

Please help anyone to resolve this issue.

 

Thanks in advance,

Deepak

Highlighted

Re: Exception from container-launch

New Contributor
In my case, I had to rerun the hadoop command with `sudo -u cloudera`

A few of my other problems required running the command under the user cloudera. I wonder why this happens