Community Articles

Find and share helpful community-sourced technical articles.
Labels (1)

Caused by: java.lang.OutOfMemoryError: Java heap space

This particular case is related to "Reducer tasks of hive job fails with Out Of Memory error during shuffle fetcher stage"

Fix:

  • Increase hive.tez.container.size if it is set too low.
  • tez.runtime.shuffle.memory.limit.percent from default value 0.7

Changed to 0.4

  • Decrease tez.runtime.shuffle.fetch.buffer.percent from default 0.25 to .15 if needed. (Different values were tested between the range of 0.25 to 0.10)
5,315 Views
Comments

Also, If we are using MR then we might need to revisit the mapper/reducer container's heap size accordingly.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.
Version history
Last update:
‎11-26-2015 11:49 AM
Updated by:
Contributors
Top Kudoed Authors