Error occurred during initialization of VM

[#1]

I am trying to run sample hadoop example jobs on the new cluster with CDH 5.6.0 

I am running following command :

/usr/bin/yarn jar /usr/lib/hadoop-0.20-mapreduce/hadoop-examples.jar pi 16 100

Number of Maps  = 16

Samples per Map = 100

Wrote input for Map #0

Wrote input for Map #1

Wrote input for Map #2

Wrote input for Map #3

Wrote input for Map #4

Wrote input for Map #5

Wrote input for Map #6

Wrote input for Map #7

Wrote input for Map #8

Wrote input for Map #9

Wrote input for Map #10

Wrote input for Map #11

Wrote input for Map #12

Wrote input for Map #13

Wrote input for Map #14

Wrote input for Map #15

Starting Job

16/03/11 15:24:19 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm73

16/03/11 15:24:19 INFO input.FileInputFormat: Total input paths to process : 16

16/03/11 15:24:19 INFO mapreduce.JobSubmitter: number of splits:16

16/03/11 15:24:19 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1457718893017_0004

16/03/11 15:24:20 INFO impl.YarnClientImpl: Submitted application application_1457718893017_0004

16/03/11 15:24:20 INFO mapreduce.Job: The url to track the job: http://hmn002.dev.abc.com:8088/proxy/application_1457718893017_0004/

16/03/11 15:24:20 INFO mapreduce.Job: Running job: job_1457718893017_0004

16/03/11 15:24:25 INFO mapreduce.Job: Job job_1457718893017_0004 running in uber mode : false

16/03/11 15:24:25 INFO mapreduce.Job:  map 0% reduce 0%

16/03/11 15:24:27 INFO mapreduce.Job: Task Id : attempt_1457718893017_0004_m_000004_0, Status : FAILED

Exception from container-launch.

Container id: container_1457718893017_0004_01_000005

Exit code: 1

Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)

at org.apache.hadoop.util.Shell.run(Shell.java:478)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)

at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1

16/03/11 15:24:27 INFO mapreduce.Job: Task Id : attempt_1457718893017_0004_m_000007_0, Status : FAILED

Exception from container-launch.

Container id: container_1457718893017_0004_01_000009

Exit code: 1

Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)

at org.apache.hadoop.util.Shell.run(Shell.java:478)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)

at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1

Its failing with above error. When I checked many people are pointing to hadoop classpath but I have checked classpath also correct in the Cloudera Manager.

This is the another log I found from one of the task.

Log Type: stderr

Log Upload Time: Fri Mar 11 20:24:44 +0000 2016

Log Length: 108

Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Log Type: stdout

Log Upload Time: Fri Mar 11 20:24:44 +0000 2016

Log Length: 90

Error occurred during initialization of VM
Could not reserve enough space for object heap

Anyone know whats wrong here ?

[#2 · Accepted solution]

Hi Roy,

It seems like your JVM Heap sizes configured incorrectly. Better check the JVM opts and heap spaces. (Since it is not able to launch vm)

[#3]

Starting a JVM like below will start it with 256MB of memory, and will allow the process to use up to 2048MB of memory:

  java -Xmx2048m -Xms256m

More about...memory management