Error occurred during initialization of VM

Roy — Fri, 16 Sep 2022 10:08:30 GMT

I am trying to run sample hadoop example jobs on the new cluster with CDH 5.6.0

I am running following command :

/usr/bin/yarn jar /usr/lib/hadoop-0.20-mapreduce/hadoop-examples.jar pi 16 100

Number of Maps = 16
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Starting Job
16/03/11 15:24:19 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm73
16/03/11 15:24:19 INFO input.FileInputFormat: Total input paths to process : 16
16/03/11 15:24:19 INFO mapreduce.JobSubmitter: number of splits:16
16/03/11 15:24:19 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1457718893017_0004
16/03/11 15:24:20 INFO impl.YarnClientImpl: Submitted application application_1457718893017_0004
16/03/11 15:24:20 INFO mapreduce.Job: The url to track the job: http://hmn002.dev.abc.com:8088/proxy/application_1457718893017_0004/
16/03/11 15:24:20 INFO mapreduce.Job: Running job: job_1457718893017_0004
16/03/11 15:24:25 INFO mapreduce.Job: Job job_1457718893017_0004 running in uber mode : false
16/03/11 15:24:25 INFO mapreduce.Job: map 0% reduce 0%
16/03/11 15:24:27 INFO mapreduce.Job: Task Id : attempt_1457718893017_0004_m_000004_0, Status : FAILED
Exception from container-launch.
Container id: container_1457718893017_0004_01_000005
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1

16/03/11 15:24:27 INFO mapreduce.Job: Task Id : attempt_1457718893017_0004_m_000007_0, Status : FAILED
Exception from container-launch.
Container id: container_1457718893017_0004_01_000009
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:561)
at org.apache.hadoop.util.Shell.run(Shell.java:478)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:738)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:210)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1

Its failing with above error. When I checked many people are pointing to hadoop classpath but I have checked classpath also correct in the Cloudera Manager.

This is the another log I found from one of the task.

Log Type: stderr
Log Upload Time: Fri Mar 11 20:24:44 +0000 2016
Log Length: 108
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
 
Log Type: stdout
Log Upload Time: Fri Mar 11 20:24:44 +0000 2016
Log Length: 90
Error occurred during initialization of VM
Could not reserve enough space for object heap

Anyone know whats wrong here ?

Re: Error occurred during initialization of VM

sathishkumar — Wed, 16 Mar 2016 16:52:36 GMT

Hi Roy,

It seems like your JVM Heap sizes configured incorrectly. Better check the JVM opts and heap spaces. (Since it is not able to launch vm)

Re: Error occurred during initialization of VM

dellmerca — Thu, 13 Jul 2017 10:28:37 GMT

Starting a JVM like below will start it with 256MB of memory, and will allow the process to use up to 2048MB of memory:

java -Xmx2048m -Xms256m

More about...memory management

question Re: Error occurred during initialization of VM in Archives of Support Questions (Read Only)

Error occurred during initialization of VM

Re: Error occurred during initialization of VM

Re: Error occurred during initialization of VM