Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Container -> memory.TaskMemoryManager: leak

Highlighted

Container -> memory.TaskMemoryManager: leak

Contributor

Hi.

 

I have a container throwing an error like :

 

WARN memory.TaskMemoryManager: leak 704.9 MB memory from org.apache.spark.util.collection.ExternalSorter
17/04/03 16:06:38 ERROR executor.Executor: Managed memory leak detected; size = 739093336 bytes, TID = 5883
17/04/03 16:06:38 ERROR executor.Executor: Exception in task 22.0 in stage 136.0 (TID 5883)
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Error while processing row (tag=0)
at org.apache.hadoop.hive.ql.exec.spark.SparkReduceRecordHandler.processRow(SparkReduceRecordHandler.java:303)
	at org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:49)
	at org.apache.hadoop.hive.ql.exec.spark.HiveReduceFunctionResultList.processNextRecord(HiveReduceFunctionResultList.java:28)
	at org.apache.hadoop.hive.ql.exec.spark.HiveBaseFunctionResultList$ResultIterator.hasNext(HiveBaseFunctionResultList.java:95)
	at scala.collection.convert.Wrappers$JIteratorWrapper.hasNext(Wrappers.scala:41)
	at scala.collection.Iterator$class.foreach(Iterator.scala:727)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)

 

What does it exactly mean ?

Thx