Skip to content

fix(stream): bound range reads by metadata entries - #3595

Draft
wengsht wants to merge 1 commit into
apache:unstablefrom
wengsht:fix-stream-range-metadata-frontier
Draft

fix(stream): bound range reads by metadata entries#3595
wengsht wants to merge 1 commit into
apache:unstablefrom
wengsht:fix-stream-range-metadata-frontier

Conversation

@wengsht

@wengsht wengsht commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Problem

Stream metadata defines the logical first and last live entry IDs, but the shared range path scans the raw RocksDB entry-key interval supplied by the caller.

If raw keys outside the metadata bounds remain after an interrupted or older slot migration, they become visible again through XRANGE, XREVRANGE, and new-message XREADGROUP reads. This makes XLEN / XINFO STREAM disagree with XRANGE and can pin retention: a sweeper repeatedly starts from a stale pre-frontier entry while XTRIM correctly treats the newer metadata frontier as authoritative.

The completed #3592 prevents new trim DeleteRange records from being lost during incremental slot migration, but existing divergent streams still need a read-side invariant.

Change

Clamp the shared stream range interval to metadata's [first_entry_id, last_entry_id] bounds:

  • forward and reverse reads use the same logical bounds;
  • exclusion flags are cleared only when their boundary was replaced by a metadata bound;
  • metadata-empty streams return no entries even if stale raw keys exist.

This is read-only hardening. It does not add cleanup writes to read commands or change stream metadata.

Test

The regression trims a stream, reinserts raw entry keys below and above its metadata bounds, and verifies:

  • forward range returns only metadata-live entries;
  • reverse range returns only metadata-live entries;
  • an exact lookup of a stale entry returns empty.

Validation:

  • ./x.py format
  • ./x.py check format
  • fresh ./x.py build build-make --unittest -j 8
  • all 99 RedisStreamTest.* cases pass

AI assistance was used for diagnosis and drafting; I reviewed the code and tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant