Community Articles

Find and share helpful community-sourced technical articles.
Labels (1)
avatar
Super Guru

SYMPTOM:

hivemetastore crashing with outofmemoryerror during ACID compactions.

ERROR [Thread-13]: compactor.Cleaner (Cleaner.java:run(140)) - Caught an exception in the main loop of compactor cleaner, java.lang.OutOfMemoryError: Java heap space 
ERROR [Thread-13]: compactor.Cleaner (Cleaner.java:run(140)) - Caught an exception in the main loop of compactor cleaner, java.lang.OutOfMemoryError: Java heap space 

ROOT CAUSE:

Enabled heap dump on outofmemory, after Analysis the heap dump we found that there are lots of entries for FileSystem$Cache$Key,FileSystem objects which was causing a memory leak

WORKAROUND:

set fs.hdfs.impl.disable.cache=true

set fs.file.impl.disable.cache=true

RESOLUTION:

this has been fixed in https://issues.apache.org/jira/browse/HIVE-13151, so apply a patch to avoid this.

1,912 Views
0 Kudos