@Jack Here are the properties: hive.compactor.delta.pct.threshold Default: 0.1
Metastore
Percentage (fractional) size of the delta files relative to the base that will trigger a major compaction. 1 = 100%, so the default 0.1 = 10%. hive.compactor.abortedtxn.threshold Default: 1000
Metastore
Number of aborted transactions involving a given table or partition that will trigger a major compaction. Setting Compaction properties TBLProperties: CREATE TABLE table_name ( id int, name string ) CLUSTERED BY (id) INTO 2 BUCKETS STORED AS ORC TBLPROPERTIES ("transactional"="true", "compactor.mapreduce.map.memory.mb"="2048", -- specify compaction map job properties "compactorthreshold.hive.compactor.delta.num.threshold"="4", -- trigger minor compactionifthere are more than4delta directories "compactorthreshold.hive.compactor.delta.pct.threshold"="0.5"-- trigger major compactionifthe ratio of size of delta files to -- size of base files is greater than50% ); ALTER TABLE table_name COMPACT 'minor'
WITH OVERWRITE TBLPROPERTIES ("compactor.mapreduce.map.memory.mb"="3072"); -- specify compaction map job properties
ALTER TABLE table_name COMPACT 'major'
WITH OVERWRITE TBLPROPERTIES ("tblprops.orc.compress.size"="8192"); -- change any other Hive table properties We can trigger major compactions by using below command: alter table <table-name> partition(<partition-name>,<nested-partition-name>,..) compact 'major'; More details on this page: https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions
... View more
