Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Reducer failed on java heap size

Reducer failed on java heap size

Explorer

Hi all,

I have MapRed job that compacted small processed files in the HDFS.

In the Reducer log i see the following error and below is the workflow.xml parameters.

I tried to fine tune the paarameters of the memory but it's not helping.

Can you please help to understand what i'm missing?

=========================================================================

2016-08-15 17:43:48,981 INFO org.apache.hadoop.mapred.Merger: Merging 707 sorted segments
2016-08-15 17:43:49,522 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.lzo_deflate]
2016-08-15 17:43:49,531 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.lzo_deflate]
2016-08-15 17:43:49,554 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.lzo_deflate]
2016-08-15 17:43:49,606 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.lzo_deflate]
2016-08-15 17:43:49,628 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.lzo_deflate
2016-08-15 17:43:53,337 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor [.lzo_deflate]
2016-08-15 17:43:53,352 INFO org.apache.hadoop.mapred.Merger: Merging 234 intermediate segments out of a total of 486
2016-08-15 17:59:43,887 INFO org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs' truncater with mapRetainSize=-1 and reduceRetainSize=-1
2016-08-15 17:59:43,889 FATAL org.apache.hadoop.mapred.Child: Error running child : java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.IFile$Reader.readNextBlock(IFile.java:357)
at org.apache.hadoop.mapred.IFile$Reader.next(IFile.java:419)
at org.apache.hadoop.mapred.Merger$Segment.next(Merger.java:238)
at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:348)
at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:368)
at org.apache.hadoop.mapred.Merger.writeFile(Merger.java:156)
at org.apache.hadoop.mapred.Merger$MergeQueue.merge(Merger.java:517)
at org.apache.hadoop.mapred.Merger.merge(Merger.java:142)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier.createKVIterator(ReduceTask.java:2537)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:425)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)

I have the following parameters in the following in the workflow:

arg>-Dmapred.reduce.child.java.opts=-Xmx6144M</arg>
<arg>-Dmapred.map.child.java.opts=-Xmx4096M</arg>
<arg>-Dmapred.child.java.opts=-Xmx6144M</arg>
<arg>-Dmapred.job.shuffle.input.buffer.percent=0.30</arg>
<arg>-Dio.sort.factor=12</arg>
<arg>-Dio.sort.mb=256</arg>

1 REPLY 1
Highlighted

Re: Reducer failed on java heap size

Explorer

Who can help here please