Iceberg deletion vector support (attempt #2) - #2183

Merged

mkmkme merged 66 commits into

antalya-26.6from

feature/antalya-26.6/iceberg-puffin-deletion-vectors-read-2

Aug 21, 2026

Commits on Aug 7, 2026

Merge branch 'master' into iceberg-puffin-deletion-vectors-read-2
ianton-ru
committed
13f4abe
Fix build
ianton-ru
committed
9b31333
Fix Iceberg trivial count to sum required record_count.

ianton-ru
and
cursoragent
committed
088cf75
Remove isEtagUsableAsCacheKey, this method useless in backport
ianton-ru
committed
69ab136
Fix settings history
ianton-ru
committed
0924819
Merge branch 'antalya-26.6' into feature/antalya-26.6/iceberg-puffin-deletion-vectors-read-2
ianton-ru
authored
88ea25b
Reuse the `Puffin` format reader instead of a separate `PuffinFile` parser.

ianton-ru
and
cursoragent
committed
13a7278

Commits on Aug 10, 2026

Restore `PuffinFilesRead` accounting on seekable footer parse.

ianton-ru
and
cursoragent
committed
1a9caaa

Commits on Aug 12, 2026

Reject Iceberg DELETE/UPDATE when the snapshot has deletion vectors.

ianton-ru
and
cursoragent
committed
85cac8e
Cap non-seekable Puffin buffering with the absolute size limit.

ianton-ru
and
cursoragent
committed
9d396ae
Validate SQL Puffin roaring bitmaps after readSafe.

ianton-ru
and
cursoragent
committed
3d7645f
Fail closed trivial Iceberg count on cross-manifest row overflow.

ianton-ru
and
cursoragent
committed
955b6a5
Treat zero-sized Puffin cache like use_puffin_files_cache=0.

ianton-ru
and
cursoragent
committed
45b383e
Fix puffin_files_cache_max_entries docs: zero means unlimited.

ianton-ru
and
cursoragent
committed
da13047
Fix version in settings history
ianton-ru
committed
ee53ef0
Fix SYSTEM DROP PUFFIN FILES CACHE privilege alias.

ianton-ru
and
cursoragent
committed
1036173
Count PuffinFilesRead on SQL deletion-vector blob reads.

ianton-ru
and
cursoragent
committed
83b40a9
Tolerate Parquet FileMetaData.num_rows mismatch in global offsets.

ianton-ru
and
cursoragent
committed
0da72d1
Clarify PositionDeleteKindPresence is not used by totalRows.

ianton-ru
and
cursoragent
committed
9929d10
Remove dead empty-etag fallback after Puffin cache key creation.

ianton-ru
and
cursoragent
committed
64fd335
Make Puffin cache waiter metrics gtest deterministic.

ianton-ru
and
cursoragent
committed
4289367
Move ClusterFunctionReadTaskResponse protocol tests to their own file.

ianton-ru
and
cursoragent
committed
dbc6e40
Fail closed when schema_transform cannot be sent on old protocol.

ianton-ru
and
cursoragent
committed
20bb262
Allow mixed blob types in Puffin footers for deletion-vector reads.

ianton-ru
and
cursoragent
committed
43d86a3
Include object-storage description in Puffin cache identity.

ianton-ru
and
cursoragent
committed
5aceb8c
Fail closed when iceberg_info cannot be sent on old protocol.

ianton-ru
and
cursoragent
committed
1095d34
Reject deletion-vector-v1 blobs unless snapshot and sequence are -1.

ianton-ru
and
cursoragent
committed
7c74abc
Do not cache count-from-files results under row filters.

ianton-ru
and
cursoragent
committed
a5867ff
Reject oversized Puffin DV cardinality before blob allocate.

ianton-ru
and
cursoragent
committed
d275f3e
Fix use_puffin_files_cache settings history previous_value.

ianton-ru
and
cursoragent
committed
c425bd2
Merge branch 'antalya-26.6' into feature/antalya-26.6/iceberg-puffin-deletion-vectors-read-2
ianton-ru
authored
3be5b0c
fix(Parsers): extend magic_enum range for ASTSystemQuery::Type

CheSema
authored and
ianton-ru
committed
c04e775

Commits on Aug 13, 2026

Reject Iceberg DELETE/UPDATE mutations on format version 3+.

ianton-ru
and
cursoragent
committed
ecf1a5c
Allow need_only_count for Iceberg deletion vectors and position deletes.

ianton-ru
and
cursoragent
committed
f73c24a
Cache parsed Puffin footers by storage identity, path, and etag.

ianton-ru
and
cursoragent
committed
a930aba
Charge Puffin cache keys and cell overhead in entry weight.

ianton-ru
and
cursoragent
committed
e4a00de
Simplify Puffin footer sharing to a small in-cache memo.

ianton-ru
and
cursoragent
committed
f01a2d2
Honor puffin_files_cache_size for the footer memo.

ianton-ru
and
cursoragent
committed
5a80deb

Commits on Aug 14, 2026

Force error log level in 04671 Iceberg v3 mutations test.

ianton-ru
and
cursoragent
committed
c6e46a6
Avoid iceberg count-only shortcut in 04337 reserved field-id test.

ianton-ru
and
cursoragent
committed
062c8e0
Merge antalya-26.6
ianton-ru
committed
5a7c982
Reuse Puffin format DV decode helpers from Iceberg reader.

ianton-ru
and
cursoragent
committed
bbd3099
Reject deletion-vector-v1 blobs with non-empty fields.

ianton-ru
and
cursoragent
committed
5e7487f
Preserve applied_filter on deletion-vector count-only path.

ianton-ru
and
cursoragent
committed
cfec740
Restore Iceberg snapshot-summary trivial COUNT shortcut.

ianton-ru
and
cursoragent
committed
47e8312
Keep unbucketed ParquetV3 need_only_count on FileMetaData.num_rows.

ianton-ru
and
cursoragent
committed
2926b78
Drop unreachable deletion-vector scan from Iceberg mutations.

ianton-ru
and
cursoragent
committed
4abbc8c
Drop dead _headers check from object-storage count cache.

ianton-ru
and
cursoragent
committed
c0f187d
Evict Puffin footer memo entries one-by-one under budget pressure.

ianton-ru
and
cursoragent
committed
7234aaf
Document that icebergCluster loads deletion vectors on the initiator.

ianton-ru
and
cursoragent
committed
92f7952
Allow Iceberg `_pos` in deletion-vector-v1 fields.

ianton-ru
and
cursoragent
committed
fb4f65e
Document that Puffin accepts Iceberg `_pos` in deletion-vector-v1 fields.

ianton-ru
and
cursoragent
committed
0736e28
Share Puffin DV footer and read-limit checks with the Iceberg loader.

ianton-ru
and
cursoragent
committed
ff965f0
Remove unused PositionDeleteKindPresence helper.

ianton-ru
and
cursoragent
committed
61c3ec7

Commits on Aug 17, 2026

Disable trivial COUNT in 04672 decode contrast case.

ianton-ru
and
cursoragent
committed
a69cecb
Keep deletion-vector need_only_count on cardinality-only shrink.

ianton-ru
and
cursoragent
committed
79d773f
Merge branch 'antalya-26.6' into feature/antalya-26.6/iceberg-puffin-deletion-vectors-read-2

ianton-ru
and
cursoragent
committed
bb12dec

Commits on Aug 18, 2026

Require zero position deletes for Iceberg snapshot COUNT shortcut.

ianton-ru
and
cursoragent
committed
76e3c07
Fail closed on Iceberg totalBytes overflow.

ianton-ru
and
cursoragent
committed
1a61f85
Share one Puffin cache budget between DVs and footer memo.

ianton-ru
and
cursoragent
committed
57ffb4e
Skip Puffin files cache when one DV entry cannot fit.

ianton-ru
and
cursoragent
committed
0305eb7
Cover stale position-delete totals after Spark rewrite_data_files.

ianton-ru
and
cursoragent
committed
4b149ab
Fix equality-delete path registration for s3/azure Iceberg tests.

ianton-ru
and
cursoragent
committed
7f14d3b
Cover fail-closed trivial COUNT with live Iceberg deletion vectors.

ianton-ru
and
cursoragent
committed
a6e8d62
Prefer Iceberg manifest record counts over snapshot-summary totals.

ianton-ru
and
cursoragent
committed
6554e70

Commits on Aug 20, 2026

Merge branch 'antalya-26.6' into feature/antalya-26.6/iceberg-puffin-deletion-vectors-read-2
ianton-ru
authored
c113b0c