<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>question Re: MapReduce application failed with OutOfMemoryError in Support Questions</title>
    <link>https://community.cloudera.com/t5/Support-Questions/MapReduce-application-failed-with-OutOfMemoryError/m-p/56031#M36634</link>
    <description>&lt;P&gt;Thank You&amp;nbsp;Fawze&amp;nbsp;and sorry for my delayed answer. I was sick.&lt;/P&gt;&lt;P&gt;Is in both, the map and the reduce phase.&lt;/P&gt;&lt;P&gt;The MR job takes aprox. 3TB from HBase and adds about 30GB of data per day.&lt;/P&gt;&lt;P&gt;Cluster architecture:&lt;/P&gt;&lt;P&gt;Number of NodeManagers = 37&lt;/P&gt;&lt;P&gt;Resource Managers = 2 with YARN in HA&lt;/P&gt;&lt;P&gt;5 nodes with 22 cores and 66GB of RAM&lt;/P&gt;&lt;P&gt;32 nodes with 30 cores and 120GB of RAM&lt;/P&gt;&lt;P&gt;Total vcores = 1070&lt;/P&gt;&lt;P&gt;Total memory = 4.07 TB&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;P&gt;I realized that decreasing the memory instead of increasing got better results.&lt;/P&gt;&lt;P&gt;Now with&amp;nbsp;this params worked really well and could get all the cores.&lt;/P&gt;&lt;P&gt;&amp;nbsp;&lt;/P&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;"mapreduce.map.memory.mb" 1536&lt;BR /&gt;"mapreduce.reduce.memory.mb" 3072&lt;/DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;&lt;DIV&gt;"mapreduce.map.java.opts" -Xmx1024m&lt;BR /&gt;"mapreduce.reduce.java.opts" -Xmx2560m&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;DIV&gt;Thanks!&lt;/DIV&gt;&lt;DIV&gt;&amp;nbsp;&lt;/DIV&gt;&lt;DIV&gt;Guido.&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;&lt;/DIV&gt;</description>
    <pubDate>Fri, 16 Jun 2017 18:29:33 GMT</pubDate>
    <dc:creator>gsalerno</dc:creator>
    <dc:date>2017-06-16T18:29:33Z</dc:date>
  </channel>
</rss>

