Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
avatar
Master Mentor

Caused by: java.lang.OutOfMemoryError: Java heap space

This particular case is related to "Reducer tasks of hive job fails with Out Of Memory error during shuffle fetcher stage"

Fix:

  • Increase hive.tez.container.size if it is set too low.
  • tez.runtime.shuffle.memory.limit.percent from default value 0.7

Changed to 0.4

  • Decrease tez.runtime.shuffle.fetch.buffer.percent from default 0.25 to .15 if needed. (Different values were tested between the range of 0.25 to 0.10)
5,823 Views
Comments
avatar
Master Guru

Also, If we are using MR then we might need to revisit the mapper/reducer container's heap size accordingly.