Member since
06-16-2016
1
Post
0
Kudos Received
0
Solutions
06-16-2016
09:55 AM
Hi, I am running a hive query and its failing with below error. Included are also the config details of yarn. The error says that the physical memory is 1 GB but we have set mapreduce.map.memory.mb = 4 GB. I am not sure where it picked the 1 GB value from. Can anyone please help? Diagnostics: Container [pid=62593,containerID=container_1465308864800_1323_02_000001] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 1.9 GB of 2.1 GB virtual memory used. Killing container. Container Memory Maximum yarn.scheduler.minimun-allocation-mb = 1 GB yarn.scheduler.maximum-allocation-mb = 8 GB Map Task Memory mapreduce.map.memory.mb = 4 GB Reduce Task Memory mapreduce.reduce.memory.mb = 8 GB Map Task Maximum Heap Size mapreduce.map.java.opts.max.heap = 3 GB Reduce Task Maximum Heap Size mapreduce.reduce.java.opts.max.heap = 6 GB ApplicationMaster Memory yarn.app.mapreduce.am.resource.mb = 1 GB Diagnostics: Container [pid=62593,containerID=container_1465308864800_1323_02_000001] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 1.9 GB of 2.1 GB virtual memory used. Killing container. Container Memory Maximum yarn.scheduler.minimun-allocation-mb = 1 GB yarn.scheduler.maximum-allocation-mb = 8 GB Map Task Memory mapreduce.map.memory.mb = 4 GB Reduce Task Memory mapreduce.reduce.memory.mb = 8 GB Map Task Maximum Heap Size mapreduce.map.java.opts.max.heap = 3 GB Reduce Task Maximum Heap Size mapreduce.reduce.java.opts.max.heap = 6 GB ApplicationMaster Memory yarn.app.mapreduce.am.resource.mb = 1 GB
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache YARN
-
MapReduce