Support Questions
Find answers, ask questions, and share your expertise

Major compaction failure

Major compaction failure

New Contributor

Hi All, we maintian our table table compactions manual by enabling 'NO_AUTO_COMPACTION' = 'TRUE' and we run ALTER TABLE table_name COMPACT 'major'; after each new delta load we do this manual compaction though our compaction is failing due an unkown reason but in HDFS directory we see two base files as of my knowledge I am sure having two base files is not an issue, but one of the file name is differently syntaxed as this base_-9223372036854775808 is this the reason for compaction failure.

drwxrwxrwx - 502590450 502590450 0 2019-04-02 14:34 s3a://av-dl-hwx-prod-anhffp02-purpose/hive/x1_env/geair_aerosol_data_pivot_5x/base_-9223372036854775808

drwxrwxrwx - 502590450 502590450 0 2019-04-02 14:34 s3a://av-dl-hwx-prod-anhffp02-purpose/hive/x1_env/geair_aerosol_data_pivot_5x/base_0012120

drwxrwxrwx - 502590450 502590450 0 2019-04-02 14:34 s3a://av-dl-hwx-prod-anhffp02-purpose/hive/x1_env/geair_aerosol_data_pivot_5x/delta_0012120_0012120_0000

Don't have an account?