Apache Iceberg V4 Redesigns the Table Format for Very Large Tables
V4 cuts the metadata writes needed per commit, but older readers will not be able to read V4 metadata.
Apache Iceberg V4, presented by co-creator Daniel Weeks at QCon San Francisco, is described as a major redesign of the open table format rather than an incremental update, aimed at problems that surface when tables grow very large and at new workloads. The proposal includes a new metadata tree, single-file commits, typed column statistics, relative file paths, and file and vector data types. Weeks suggested that the faster commit path could allow commits every second or at subsecond intervals, versus the current recommendation of at least two minutes. V4 also adds expression-based defaults and check, unique, and primary key constraints. Weeks confirmed in the Q&A that V4 metadata will not be readable by older readers, although existing V3 metadata can stay in place during an upgrade.
Why it matters
The redesign targets the problems that appear when tables grow very large, and it could let writers commit far more often than the current two-minute guidance allows. Teams running Iceberg at scale would see faster, more frequent updates to their tables. The lack of backward compatibility means older readers will not be able to read V4 metadata, so the shift is a real break for anyone whose tools have not moved forward. Existing V3 metadata can remain in place during an upgrade, which softens the transition but does not remove the need to plan for it.
Matters or hype?
Does this story matter, or is it hype? Vote before you see what everyone else said.