Created on 12-23-201606:26 PM - edited 09-16-202201:37 AM
SYMPTOM: Upon starting the App Timeline Service after an Ambari & HDP upgrade, the following errors were thrown and the service was unable to start:
ERROR:
2015-08-02 22:56:24,311 INFO service.AbstractService (AbstractService.java:noteFailure(272)) - Service org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore failed in state INITED; cause:
org.fusesource.leveldbjni.internal.NativeDB$DBException: Corruption: 116 missing files; e.g.: /tmp/hadoop/yarn/timeline/leveldb-timeline-store.ldb/001052.sst
org.fusesource.leveldbjni.internal.NativeDB$DBException: Corruption: 116 missing files; e.g.: /tmp/hadoop/yarn/timeline/leveldb-timeline-store.ldb/001052.sst
ROOT CAUSE: The issue is with corrupted SST's in the app timeline db path.
RESOLUTION: Navigate to /hadoop/yarn/timeline/leveldb-timeline-store.ldb
From there you will see a text file named "CURRENT"
Please back this file up in /tmp and then remove the file as such:
cp /hadoop/yarn/timeline/leveldb-timeline-store.ldb/CURRENT /tmp
rm /hadoop/yarn/timeline/leveldb-timeline-store.ldb/CURRENT
Restart the service via Ambari