Options
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
Super Guru
Created on 12-23-2016 06:37 PM
SYMPTOM:
hivemetastore crashing with outofmemoryerror during ACID compactions.
ERROR [Thread-13]: compactor.Cleaner (Cleaner.java:run(140)) - Caught an exception in the main loop of compactor cleaner, java.lang.OutOfMemoryError: Java heap space ERROR [Thread-13]: compactor.Cleaner (Cleaner.java:run(140)) - Caught an exception in the main loop of compactor cleaner, java.lang.OutOfMemoryError: Java heap space
ROOT CAUSE:
Enabled heap dump on outofmemory, after Analysis the heap dump we found that there are lots of entries for FileSystem$Cache$Key,FileSystem objects which was causing a memory leak
WORKAROUND:
set fs.hdfs.impl.disable.cache=true
set fs.file.impl.disable.cache=true
RESOLUTION:
this has been fixed in https://issues.apache.org/jira/browse/HIVE-13151, so apply a patch to avoid this.