Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st. We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here. Need help or have questions? Drop us a line at [email protected]
Created on 12-23-201606:26 PM - edited 09-16-202201:37 AM
SYMPTOM: Upon starting the App Timeline Service after an Ambari & HDP upgrade, the following errors were thrown and the service was unable to start:
ERROR:
2015-08-02 22:56:24,311 INFO service.AbstractService (AbstractService.java:noteFailure(272)) - Service org.apache.hadoop.yarn.server.timeline.LeveldbTimelineStore failed in state INITED; cause:
org.fusesource.leveldbjni.internal.NativeDB$DBException: Corruption: 116 missing files; e.g.: /tmp/hadoop/yarn/timeline/leveldb-timeline-store.ldb/001052.sst
org.fusesource.leveldbjni.internal.NativeDB$DBException: Corruption: 116 missing files; e.g.: /tmp/hadoop/yarn/timeline/leveldb-timeline-store.ldb/001052.sst
ROOT CAUSE: The issue is with corrupted SST's in the app timeline db path.
RESOLUTION: Navigate to /hadoop/yarn/timeline/leveldb-timeline-store.ldb
From there you will see a text file named "CURRENT"
Please back this file up in /tmp and then remove the file as such:
cp /hadoop/yarn/timeline/leveldb-timeline-store.ldb/CURRENT /tmp
rm /hadoop/yarn/timeline/leveldb-timeline-store.ldb/CURRENT
Restart the service via Ambari