org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.OutOfMemoryError: Java heap space

[#1]

ISSUE: Java Heap Space issue in Hive MR engine

While working on a sample data set in hive. Query such as "select count(*)" was seen to fail with below error.

Starting Job = job_1491603076412_0001, Tracking URL = http://krishna3.openstacklocal:8088/proxy/application_1491603076412_0001/
Kill Command = /usr/hdp/2.4.2.0-258/hadoop/bin/hadoop job  -kill job_1491603076412_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2017-04-07 22:18:09,736 Stage-1 map = 0%,  reduce = 0%
2017-04-07 22:18:46,065 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_1491603076412_0001 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1491603076412_0001_m_000000 (and more) from job job_1491603076412_0001




Task with the most failures(4):
-----
Task ID:
  task_1491603076412_0001_m_000000




URL:
  http://krishna3.openstacklocal:8088/taskdetails.jsp?jobid=job_1491603076412_0001&tipid=task_14916030...
-----
Diagnostic Messages for this Task:
Error: Java heap space




FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

Checking the corresponding application logs, we observe that

2017-04-07 22:25:40,828 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.OutOfMemoryError: Java heap space
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:986)
	at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:402)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:442)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

ROOT CAUSE: Insufficient HeapSpace in MR Engine for mapreduce.map.memory.mb

RESOLUTION:

Increasing mapreduce.map.memory.mb from 1.2G to 1.75G and hence increasing the

mapreduce.task.io.sort.mb to 1003

mapreduce.map.java.opts to -Xmx1433m

and restarting the necessary services did resolve the problem.

(NOTE: mapreduce.task.io.sort.mb and mapreduce.map.java.opts value recommendations were made by Ambari )