Merging tablet base and delta files in Kudu

rpaidar — Fri, 16 Sep 2022 11:28:41 GMT

Hello,

I am trying to understand how Kudu merges the base file with delta files. I found this in a blog:

For each DiskRowSet, the scanner will materialise a column at a time, and apply any delta records and predicates before moving on to the next column.

Could some one point me to the code that does this?

Thanks in advance.

Re: Merging tablet base and delta files in Kudu

dralves — Wed, 19 Apr 2017 03:30:43 GMT

Hi @rpaidar

Most, of the code that does this can be found in the "tablet" module.

Iterators for all rowsets are assembled in Tablet::CaptureConsistentIterators() (https://github.com/cloudera/kudu/blob/master/src/kudu/tablet/tablet.cc#L1572)

Then, for diskrowsets, the base data iterator is wrapped in a delta iterator. This process starts at DiskRowSet::NewRowIterator() (https://github.com/cloudera/kudu/blob/master/src/kudu/tablet/diskrowset.cc#L592)

To get more insight on how the base is materialized and deltas are applied you can follow the different iterators that are created in DiskRowSet::NewRowIterator().

HTH

David

question Re: Merging tablet base and delta files in Kudu in Archives of Support Questions (Read Only)

Merging tablet base and delta files in Kudu

Re: Merging tablet base and delta files in Kudu