Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

map reduce jobs failed in CDH5

Highlighted

map reduce jobs failed in CDH5

Explorer

Hi ,

I ran a benchmarking job in CDH5  but got some problmes . map task went back to 37% after completing 77% ? didnt understand it ?

 

also this job got failed without any reason?

 

hdfs@node01:/opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-mapreduce$ hadoop jar hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.0-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 1000
15/03/01 12:12:48 INFO fs.TestDFSIO: TestDFSIO.1.7
15/03/01 12:12:48 INFO fs.TestDFSIO: nrFiles = 10
15/03/01 12:12:48 INFO fs.TestDFSIO: nrBytes (MB) = 1000.0
15/03/01 12:12:48 INFO fs.TestDFSIO: bufferSize = 1000000
15/03/01 12:12:48 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO
15/03/01 12:12:51 INFO fs.TestDFSIO: creating control file: 1048576000 bytes, 10 files
15/03/01 12:12:53 INFO fs.TestDFSIO: created control files for: 10 files
15/03/01 12:12:53 INFO client.RMProxy: Connecting to ResourceManager at node01/192.168.171.142:8032
15/03/01 12:12:53 INFO client.RMProxy: Connecting to ResourceManager at node01/192.168.171.142:8032
15/03/01 12:12:54 INFO mapred.FileInputFormat: Total input paths to process : 10
15/03/01 12:12:54 INFO mapreduce.JobSubmitter: number of splits:10
15/03/01 12:12:55 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1425229670240_0001
15/03/01 12:12:55 INFO impl.YarnClientImpl: Submitted application application_1425229670240_0001
15/03/01 12:12:56 INFO mapreduce.Job: The url to track the job: http://node01:8088/proxy/application_1425229670240_0001/
15/03/01 12:12:56 INFO mapreduce.Job: Running job: job_1425229670240_0001
15/03/01 12:13:12 INFO mapreduce.Job: Job job_1425229670240_0001 running in uber mode : false
15/03/01 12:13:12 INFO mapreduce.Job:  map 0% reduce 0%
15/03/01 12:13:32 INFO mapreduce.Job:  map 7% reduce 0%
15/03/01 12:13:34 INFO mapreduce.Job:  map 13% reduce 0%
15/03/01 12:15:44 INFO mapreduce.Job:  map 17% reduce 0%
15/03/01 12:15:51 INFO mapreduce.Job:  map 20% reduce 0%
15/03/01 12:16:04 INFO mapreduce.Job:  map 27% reduce 0%
15/03/01 12:16:31 INFO mapreduce.Job:  map 33% reduce 0%
15/03/01 12:17:18 INFO mapreduce.Job:  map 37% reduce 0%
15/03/01 12:17:30 INFO mapreduce.Job:  map 40% reduce 0%
15/03/01 12:17:37 INFO mapreduce.Job:  map 47% reduce 0%
15/03/01 12:18:04 INFO mapreduce.Job:  map 53% reduce 0%
15/03/01 12:18:47 INFO mapreduce.Job:  map 57% reduce 0%
15/03/01 12:19:02 INFO mapreduce.Job:  map 60% reduce 0%
15/03/01 12:19:08 INFO mapreduce.Job:  map 67% reduce 0%
15/03/01 12:19:59 INFO mapreduce.Job:  map 73% reduce 0%
15/03/01 12:20:00 INFO mapreduce.Job:  map 77% reduce 0%
15/03/01 12:20:01 INFO mapreduce.Job:  map 37% reduce 0% ==>  why map task went back to 37% (decreased)
15/03/01 12:20:48 INFO mapreduce.Job:  map 40% reduce 0%
15/03/01 12:26:03 INFO mapreduce.Job: Job job_1425229670240_0001 failed with state FAILED due to:
15/03/01 12:26:04 INFO mapreduce.Job: Counters: 32
 File System Counters
  FILE: Number of bytes read=0
  FILE: Number of bytes written=435883
  FILE: Number of read operations=0
  FILE: Number of large read operations=0
  FILE: Number of write operations=0
  HDFS: Number of bytes read=928
  HDFS: Number of bytes written=4194304000
  HDFS: Number of read operations=16
  HDFS: Number of large read operations=0
  HDFS: Number of write operations=4
 Job Counters
  Killed map tasks=4
  Launched map tasks=8
  Data-local map tasks=8
  Total time spent by all maps in occupied slots (ms)=850090
  Total time spent by all map tasks (ms)=850090
  Total vcore-seconds taken by all map tasks=850090
  Total megabyte-seconds taken by all map tasks=870492160
 Map-Reduce Framework
  Map input records=4
  Map output records=20
  Map output bytes=305
  Map output materialized bytes=403
  Input split bytes=480
  Combine input records=0
  Spilled Records=20
  Failed Shuffles=0
  Merged Map outputs=0
  GC time elapsed (ms)=2302
  CPU time spent (ms)=28480
  Physical memory (bytes) snapshot=2465312768
  Virtual memory (bytes) snapshot=6511689728
  Total committed heap usage (bytes)=2844717056
 File Input Format Counters
  Bytes Read=448
java.io.IOException: Job failed!
 at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:838)
 at org.apache.hadoop.fs.TestDFSIO.runIOTest(TestDFSIO.java:443)
 at org.apache.hadoop.fs.TestDFSIO.writeTest(TestDFSIO.java:425)
 at org.apache.hadoop.fs.TestDFSIO.run(TestDFSIO.java:755)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
 at org.apache.hadoop.fs.TestDFSIO.main(TestDFSIO.java:650)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:622)
 at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
 at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
 at org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:118)
 at org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:126)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:622)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Don't have an account?
Coming from Hortonworks? Activate your account here