Community Articles
Find and share helpful community-sourced technical articles.
Announcements
Alert: Please see the Cloudera blog for information on the Cloudera Response to CVE-2021-4428
Labels (1)

Caused by: java.lang.OutOfMemoryError: Java heap space

This particular case is related to "Reducer tasks of hive job fails with Out Of Memory error during shuffle fetcher stage"

Fix:

  • Increase hive.tez.container.size if it is set too low.
  • tez.runtime.shuffle.memory.limit.percent from default value 0.7

Changed to 0.4

  • Decrease tez.runtime.shuffle.fetch.buffer.percent from default 0.25 to .15 if needed. (Different values were tested between the range of 0.25 to 0.10)
4,781 Views
Comments
Super Guru

Also, If we are using MR then we might need to revisit the mapper/reducer container's heap size accordingly.

Don't have an account?
Version history
Last update:
‎11-26-2015 11:49 AM
Updated by:
Contributors
Top Kudoed Authors