Support Questions
Find answers, ask questions, and share your expertise

kudu major REDO delta compaction did not run

kudu major REDO delta compaction did not run

Explorer

kudu 1.9 in cdh 6.2

under these flag:

 

unlock_experimental_flags=true
tablet_history_max_age_sec=0
tablet_delta_store_major_compact_min_ratio=0.01
tablet_delta_store_minor_compact_max=2

 

there is a table, 4m records, with update operation on it (affect all records) over and over again, and it get slower and slower when query

so I run this command to check:

 

kudu fs list -fs_data_dirs=/data/1/kudu/data -fs_wal_dir=/data/1/kudu/wal -tablet_id=c7554f3d8614436d9b1057ee3b4dda19 -columns='tablet-id,rowset-id,block-id,block-kind,column,cfile-size,cfile-delta-stats'

 

and I got this result (only pick one rowset for short):

 

c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905188 | column      | tenant                  | 2.0K       | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905189 | column      | id                      | 8.72M      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905190 | column      | org_id                  | 1013.7K    | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905191 | column      | project_id              | 1013.7K    | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905192 | column      | date_id                 | 8.72M      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905193 | column      | record_count            | 46.5K      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905194 | column      | day_actual_amount       | 359.3K     | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905195 | column      | yesterday_actual_amount | 369.7K     | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905196 | column      | week_actual_amount      | 457.2K     | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905197 | column      | month_actual_amount     | 799.7K     | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905198 | column      | quarter_actual_amount   | 936.9K     | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905199 | column      | year_actual_amount      | 1.00M      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905200 | column      | total_actual_amount     | 1.39M      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905201 | column      | stat_year               | 19.4K      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905202 | column      | stat_quarter            | 15.8K      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905203 | column      | stat_month              | 17.6K      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905204 | column      | week_start_date         | 62.9K      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905205 | column      | stat_date               | 112.9K     | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905206 | column      | org_type                | 11.2K      | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 28163440 | redo        |                         | 46.68M     | ts range=[6588606323633659904, 6588672363915362304], delete_count=[0], reinsert_count=[0], update_counts_by_col_id=[14:2396096,13:2396096,12:2396096,11:2396096,10:2396096,9:2396096,5:1320560,2:152496,4:152496,15:2396096,3:152496,18:152496,7:2394840,17:2396096,6:1319624,16:2396096,8:2396096]
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905207 | bloom       |                         | 667.3K     | 
 c7554f3d8614436d9b1057ee3b4dda19 | 98        | 27905208 | adhoc-index |                         | 2.82M      | 

 

the result shows minor compact has run (only one redo block), but major compact did not. the REDO bolock size(46M) is even greater than the base data.

Why?