Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

SYMPTOM:

hivemetastore crashing with outofmemoryerror during ACID compactions.

ERROR [Thread-13]: compactor.Cleaner (Cleaner.java:run(140)) - Caught an exception in the main loop of compactor cleaner, java.lang.OutOfMemoryError: Java heap space 
ERROR [Thread-13]: compactor.Cleaner (Cleaner.java:run(140)) - Caught an exception in the main loop of compactor cleaner, java.lang.OutOfMemoryError: Java heap space 

ROOT CAUSE:

Enabled heap dump on outofmemory, after Analysis the heap dump we found that there are lots of entries for FileSystem$Cache$Key,FileSystem objects which was causing a memory leak

WORKAROUND:

set fs.hdfs.impl.disable.cache=true

set fs.file.impl.disable.cache=true

RESOLUTION:

this has been fixed in https://issues.apache.org/jira/browse/HIVE-13151, so apply a patch to avoid this.

724 Views
0 Kudos
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎12-23-2016 06:37 PM
Updated by:
 
Contributors
Top Kudoed Authors