@Artur Dushelyubov
Attempted means that the initiator attempted to schedule a compaction but it failed. As such, there will be no compaction associated with those requests. Other than visually unpleasant, there is no reason to worry about. They are part of the metastore log and it will show upto the threshold set. See threshold for attempted below. They will not be displayed after that number.
| Compaction History |
| hive.compactor.history.retention.succeeded | Default: 3 | Metastore | Number of successful compaction entries to retain in history (per partition). |
| hive.compactor.history.retention.failed | Default: 3 | Metastore | Number of failed compaction entries to retain in history (per partition). |
| hive.compactor.history.retention.attempted | Default: 2 | Metastore | Number of attempted compaction entries to retain in history (per partition). |
| hive.compactor.initiator.failed.compacts.threshold | Default: 2 | Metastore | Number of of consecutive failed compactions for a given partition after which the Initiator will stop attempting to schedule compactions automatically. It is still possible to use ALTER TABLE to initiate compaction. Once a manually initiated compaction succeeds auto initiated compactions will resume. Note that this must be less than hive.compactor.history.retention.failed. |
| hive.compactor.history.reaper.interval | Default: 2m | Metastore | Controls how often the process to purge historical record of compactions runs.
|
If this was helpful, please vote/accept best answer.
++++++
A little bit of theory below for others who may have a similar question.
SHOW COMPACTIONS returns a list of all tables and partitions currently being compacted or scheduled for compaction when Hive transactions are being used, including this information:
- database name
- table name
- partition name (if the table is partitioned)
- whether it is a major or minor compaction
- the state the compaction is in, which can be:
- "initiated" – waiting in the queue to be compacted
- "working" – being compacted
- "ready for cleaning" – the compaction has been done and the old files are scheduled to be cleaned
- "failed" – the job failed. The metastore log will have more detail.
- "succeeded" – A-ok
- "attempted" – initiator attempted to schedule a compaction but failed. The metastore log will have more information.
- thread ID of the worker thread doing the compaction (only if in working state)
- the time at which the compaction started (only if in working or ready for cleaning state)
Compactions are initiated automatically, but can also be initiated manually with an ALTER TABLE COMPACT statement.
