Reply
Roy
Explorer
Posts: 18
Registered: ‎10-09-2014
Accepted Solution

Error occurred during initialization of VM

 

I am trying to run sample hadoop example jobs on the new cluster with CDH 5.6.0 

 
I am running following command :
 
/usr/bin/yarn jar /usr/lib/hadoop-0.20-mapreduce/hadoop-examples.jar pi 16 100

Number of Maps  = 16
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Starting Job
16/03/11 15:24:19 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm73
16/03/11 15:24:19 INFO input.FileInputFormat: Total input paths to process : 16
16/03/11 15:24:19 INFO mapreduce.JobSubmitter: number of splits:16
16/03/11 15:24:19 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1457718893017_0004
16/03/11 15:24:20 INFO impl.YarnClientImpl: Submitted application application_1457718893017_0004
16/03/11 15:24:20 INFO mapreduce.Job: The url to track the job: http://hmn002.dev.abc.com:8088/proxy/application_1457718893017_0004/
16/03/11 15:24:20 INFO mapreduce.Job: Running job: job_1457718893017_0004
16/03/11 15:24:25 INFO mapreduce.Job: Job job_1457718893017_0004 running in uber mode : false
16/03/11 15:24:25 INFO mapreduce.Job:  map 0% reduce 0%
16/03/11 15:24:27 INFO mapreduce.Job: Task Id : attempt_1457718893017_0004_m_000004_0, Status : FAILED
Exception from container-launch.
Container id: container_1457718893017_0004_01_000005
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
 
 
Container exited with a non-zero exit code 1
 
16/03/11 15:24:27 INFO mapreduce.Job: Task Id : attempt_1457718893017_0004_m_000007_0, Status : FAILED
Exception from container-launch.
Container id: container_1457718893017_0004_01_000009
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
 
 
Container exited with a non-zero exit code 1
Its failing with above error. When I checked many people are pointing to hadoop classpath but I have checked classpath also correct in the Cloudera Manager.
 
This is the another log I found from one of the task.
 

Log Type: stderr

Log Upload Time: Fri Mar 11 20:24:44 +0000 2016

Log Length: 108

Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
 

Log Type: stdout

Log Upload Time: Fri Mar 11 20:24:44 +0000 2016

Log Length: 90

Error occurred during initialization of VM
Could not reserve enough space for object heap
Anyone know whats wrong here ?
Contributor
Posts: 47
Registered: ‎09-12-2014

Re: Error occurred during initialization of VM

Hi Roy,

 

It seems like your JVM Heap sizes configured incorrectly. Better check the JVM opts and heap spaces. (Since it is not able to launch vm)

 

Thanks,
Sathish (Satz)
Highlighted
New Contributor
Posts: 1
Registered: ‎07-13-2017

Re: Error occurred during initialization of VM

Starting a JVM like below will start it with 256MB of memory, and will allow the process to use up to 2048MB of memory:

  java -Xmx2048m -Xms256m

More about...memory management

Announcements