Reply
New Contributor
Posts: 3
Registered: ‎09-21-2016

Error:java heap space. How do we increase the heap size ?

Error:

org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1473855979597_0004_m_000000_3: Error: Java heap space

 

How to increase the Heap size??

 

 

Please help me with correct steps .

Thank you.

Expert Contributor
Posts: 131
Registered: ‎08-08-2013

Re: Error:java heap space. How do we increase the heap size ?

Hi,

this looks like your heap size error is thrown while executing a M/R job, right ?

Assuming you are using YARN, you should check the memory settings for map-reduce, like 

mapreduce.map.memory.mb

mapreduce.reduce.memory.mb

mapreduce.map.java.opts.max.heap

mapreduce.reduce.java.opts.max.heap

 

as well as the general YARN container related ones, described by "Step 6" in the following brilliant guide:

http://www.cloudera.com/documentation/enterprise/5-3-x/topics/cdh_ig_yarn_tuning.html

 

At the end you have to adjust them based on your cluster resources.

 

HTH....

New Contributor
Posts: 3
Registered: ‎09-21-2016

Re: Error:java heap space. How do we increase the heap size ?

Hi 

We made the changes as per your instructions but still we are facing the same issue.

 

 

New Contributor
Posts: 3
Registered: ‎09-21-2016

Re: Error:java heap space. How do we increase the heap size ?

Hi 

We made the changes as per your instructions but still we are facing the same issue.

Also do we require to restart any service after making changes ??

 

 

Champion
Posts: 760
Registered: ‎05-16-2016

Re: Error:java heap space. How do we increase the heap size ?

[ Edited ]

After chaning the heap size ,Yes you have to restart the service . 

 

if you still have the problem Can you provide us the full stack trace.

 

Also try setting the Java heap size for Mapperr and Reducers to 75% 
of mapreduce.map.memory.mb and mapreduce.reduce.
memory.mb

 

 

Highlighted
Posts: 519
Topics: 14
Kudos: 91
Solutions: 45
Registered: ‎09-02-2016

Re: Error:java heap space. How do we increase the heap size ?

I've given some more clarification in the below link. Hope this helps

 

https://community.cloudera.com/t5/Hadoop-101-Training-Quickstart/Map-and-Reduce-Error-Java-heap-spac...

 

 

Thanks

Kumar

Announcements
New solutions