Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Mapreduce in single node CDH5.4 default VM configuration

Highlighted

Mapreduce in single node CDH5.4 default VM configuration

New Contributor

Hello,

I have a map reduce java program which executes successfullyusing java main inside the CDH5.4 Quick start VM.

However, I would like to run this map reduce using hadoop. May be a trivial question, can we run map reduce on the default quickstart vm ?

When i say default, it just has single node vm and no manager installed.

I am able to create input folders and can put the files in hadoop using haddop fs commands.

 

However, when i run the executable jar using below command,

hadoop jar target/EMOTIONS-0.0.1-SNAPSHOT.jar -libjars ${LIBJAR} /test/android.txt /test/new_output

 

I get below error:

15/06/24 09:21:01 INFO main.EmotionsAggregatorMain: Startimg the Emotions MapR job
15/06/24 09:21:01 INFO main.EmotionsAggregatorMain: Args: [Ljava.lang.String;@6e29991
15/06/24 09:21:02 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/06/24 09:21:04 INFO input.FileInputFormat: Total input paths to process : 1
15/06/24 09:21:04 INFO mapreduce.JobSubmitter: number of splits:1
15/06/24 09:21:04 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1435162558622_0001
15/06/24 09:21:05 INFO impl.YarnClientImpl: Submitted application application_1435162558622_0001
15/06/24 09:21:05 INFO mapreduce.Job: The url to track the job: http://quickstart.cloudera:8088/proxy/application_1435162558622_0001/
15/06/24 09:21:05 INFO mapreduce.Job: Running job: job_1435162558622_0001
15/06/24 09:21:28 INFO mapreduce.Job: Job job_1435162558622_0001 running in uber mode : false
15/06/24 09:21:28 INFO mapreduce.Job:  map 0% reduce 0%
15/06/24 09:21:28 INFO mapreduce.Job: Job job_1435162558622_0001 failed with state FAILED due to: Application application_1435162558622_0001 failed 2 times due to AM Container for appattempt_1435162558622_0001_000002 exited with  exitCode: 1
For more detailed output, check application tracking page:http://quickstart.cloudera:8088/proxy/application_1435162558622_0001/Then, click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1435162558622_0001_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

 

===================================================================================

I looked at the log and it says something like - Log server may not have been defined and Container is either not started or finished execution.

 

Please help as I have been stuck at this step and trying to figure a way out.

 

 

 

1 REPLY 1
Highlighted

Re: Mapreduce in single node CDH5.4 default VM configuration

Master Collaborator

@Ajith wrote:

However, I would like to run this map reduce using hadoop. May be a trivial question, can we run map reduce on the default quickstart vm ?

When i say default, it just has single node vm and no manager installed.

 

You can run MapReduce jobs in the VM without modification (provided the job fits in the VM's resources), and YARN is running: it would appear from the logs the job is connecting to the ResourceManager successfully.

 

I would suggest clicking on the second link in your logs to see more information about the failures and share anything you find.

 

 

Don't have an account?
Coming from Hortonworks? Activate your account here