Confluent Current Presentation - Apache Iceberg Puffin Files

Slide deck for my presentation at Confluent’s Current 2026.

Thursday, November 5, 2026

2:50 PM - 3:05 PM

Topic: Understanding Puffin File Format for Advanced Query Optimization in Apache Iceberg

Apache Iceberg manifests store essential statistics such as min/max values and null counts, but as tables scale, query engines increasingly require richer metadata to make efficient optimization decisions. Puffin was introduced in Iceberg to address this gap by providing a file format for storing advanced statistics, sketches, and secondary indexes that do not naturally fit within manifests.

In this session, we will explore the Puffin file format under the hood, including its structure, blob-based design, and snapshot-aware metadata model. We'll examine the two Puffin blob types supported today - Theta Sketches for approximate distinct counts and Deletion Vectors for row-level deletes introduced in Iceberg v3, and discuss how they enable smarter query planning, more efficient delete handling, and metadata-driven optimizations without rewriting data files.

We will also look at how Puffin is already being adopted in production, with systems such as AWS Glue and Impala producing and consuming Puffin metadata to power smarter optimizers and table operations. Finally, we will discuss why Puffin is becoming an increasingly important building block in Iceberg's evolution and how it lays the foundation for future metadata-driven innovations