By ByteBot
Apache Iceberg v4 does not exist yet. That is the point. The specification is being written right now, in open mailing list threads and GitHub pull requests, with engineers from Apple, Google, Netflix, Snowflake, Databricks, Microsoft, and LinkedIn all reviewing the same code. What the dev list says in July 2026 matters more than any release note: this is where the next two years of data infrastructure gets decided. Here is what is actually being built, what is ratified, and what is still a live argument.
Why v4? Because v3 Exposed the Next Problem
Apache Iceberg v3 was a genuine step forward — deletion vectors cut DML overhead by up to 10x compared to v2’s delete file accumulation, and native Variant and Geometry types eliminated two common workarounds from modern pipelines. By July 2026, Snowflake and Databricks Runtime 18.0+ have v3 in production. Trino is still catching up. The ecosystem is in mid-migration.
But v3 did not touch the metadata layer. And that is exactly where the pain is for anyone running streaming workloads. Every Iceberg commit produces at minimum three metadata files: a new snapshot, a manifest list, and one or more manifest files. For batch jobs that commit hourly, this is fine. For a Flink job committing every five seconds, you accumulate hundreds of metadata files per hour. The operational tax — compaction jobs, small file problems, latency spikes — is real and well-documented on the dev list. V4’s primary job is fixing this.
Adaptive Metadata Trees: The Feature That Actually Matters
The most consequential Apache Iceberg v4 proposal restructures the metadata hierarchy entirely. The current model has a manifest list sitting between snapshot files and individual manifest files. V4 replaces this with a Root Manifest — a single entry point for a snapshot that flattens the hierarchy and gains a critical new ability: small changes can be inlined directly into it.
What this enables in practice is a single-file commit. A streaming job writing a tiny micro-batch writes one file — the new root — rather than the current cascade. Operational overhead for streaming pipelines drops dramatically. Flink writers, Kafka Connect Iceberg sinks, and any high-frequency writer benefit directly.
The important caveat: this proposal is not yet ratified. It is in active design sync with concrete design documents and open PRs. Engineers from Netflix and LinkedIn — who have the most production streaming pain — are driving it hard. The mechanism is solid. The ratification is still ahead. Do not architect around it yet, but do follow the VOTE threads on the Apache Iceberg spec page.
Relative Paths: Already Ratified, Start Planning Now
This is the only Apache Iceberg v4 feature you can confidently plan around today. Relative paths completed their VOTE thread in May 2026 and are locked into the v4 spec.
Currently, Iceberg metadata stores fully qualified URIs — every data file reference looks like s3://mybucket/warehouse/mydb/mytable/data/00001.parquet. Move the table to a new bucket, a different region, or a different cloud provider, and you must rewrite every metadata file. Disaster recovery operations, cloud migrations, and dev-to-staging promotions all carry this overhead.
Under v4, paths become relative to the table’s base location. Move the table with aws s3 cp, update the base location property, and you are done. No metadata surgery. This is a quiet but significant quality-of-life improvement for anyone managing Iceberg tables across environments.
Column Families: Real Problem, Contested Solution
ML feature stores produce tables with thousands of columns. Under current Iceberg, updating 10 feature columns forces a file rewrite that touches all thousands. Schema evolution for wide tables is painful enough that many teams split their feature store into multiple physical Iceberg tables as a workaround.
V4’s column families proposal allows column groups to be stored and evolved independently. New features can be backfilled without touching other columns. In theory, this is the right architectural answer to the ML wide-table problem.
In practice, per the dev list, this is still a live argument. Where exactly column-level updates should live in the spec, and what happens to the partition tuple, are both actively contested. This proposal is further from ratification than anything else in v4. Treat it as a direction to plan toward, not a foundation to build on.
Who Is Building This, and Why It Matters
Engineers from Apple, Google, Snowflake, Databricks, Microsoft, Netflix, and LinkedIn are in the same design discussions reviewing the same PRs. That breadth of cross-company collaboration is unusual in open source — most format specs end up dominated by one vendor’s priorities. Iceberg v4 is genuinely multi-stakeholder, which is both its strength and the reason progress moves at mailing-list speed rather than product-sprint speed.
The most interesting development: Databricks has proposed that Delta Lake 5.0 adopt the same adaptive metadata tree as Iceberg v4. The pitch is one unified metadata structure that both Delta and Iceberg can read and write natively, eliminating translation layers like UniForm entirely. Databricks presented this at Data + AI Summit 2026 as “format co-evolution.” Treat it as a proposal under community review, not a roadmap commitment. But if it lands, the data format war ends at the metadata layer — and that is worth watching closely.
What Developers Should Do Right Now
V4 is not something you can ship to production today. But it is something you should be actively shaping your planning around. Concretely:
- Adopt Iceberg v3 now on Snowflake or Databricks Runtime 18.0+. Deletion vectors alone make the upgrade worthwhile for mutation-heavy tables. Trino users should wait — v3 support is still incomplete as of July 2026.
- Evaluate PyIceberg for Python-native workflows. With 500,000 daily downloads on PyPI, it is now mature enough for production use without Spark or a JVM. The PyIceberg GitHub repo has solid quickstart documentation.
- Follow the dev list at
dev@iceberg.apache.org. The adaptive metadata tree VOTE thread is the one to watch — when it appears, the streaming use case changes meaningfully. - Do not build wide ML tables assuming column families will be ratified soon. Design for what v3 supports now, and architect for v4 optionality without depending on it.
The July 2026 dev list analysis from Alex Merced and the Iceberg Summit 2026 recap from Snowflake are both worth reading in full if you are planning any major Iceberg investment in the next 12 months.
V4 is not ready. The right response is not to wait — it is to ship on v3 and stay close to the mailing list. The data engineers who track the spec now will spend less time in emergency compaction jobs later.
