design: two-runtime compute read isolation

[antiguru]

Motivation

A compute replica maintains indexes and serves reads on the same timely workers, and timely does not preempt a running task. So a read waits behind maintenance work, not because the machine is out of CPU but because the one run loop is busy and cannot be interrupted. The sharpest form is introspection: mz_introspection describes a replica's own state and cannot be served from any other replica, so it is unavailable exactly when an operator needs it.

This is design only. No production code. It is the design of record for the implementation stack, #38386 through #38393, with the benchmarks in #38676 and #38678, and it describes what those pull requests ship.

Description

One document, design.md. It targets the interactive runtime: a second in-process timely runtime that renders temporary dataflows and serves reads off the maintenance runtime's arrangements, zero-copy through a per-process sharing registry.

It opens with the decomposition that shaped everything else: twelve reported symptoms, the mechanisms behind them, and which of sixteen candidate solutions reaches each. Every cell carries an evidence label distinguishing measured from argued from read-off-the-code, and the table deliberately includes mechanisms nothing here addresses. Two of the widest-blast-radius ones are among those, which is the point: a decomposition listing only the causes we have answers for is not a decomposition.

Several claims the work started with are retired in it:

• The peek walk substrate, not the second runtime, is what removes head-of-line blocking between peeks.
• Moving the walk off the worker is reproducibly worse than doing nothing when a peek queues behind a long operator activation.
• The second runtime's case is temporary dataflows and introspection during hydration. Judging the sharing registry and its protocol against a peek-latency claim weighs them against the wrong benefit.

Peek placement is therefore treated as an orthogonal axis and discussed once. It needs no second runtime and this design needs none of it. It shipped separately, as budgeted peek slices on the worker with offload past the budget, and #38449 is its design. The interactive runtime inherits that path unchanged.

The protocol section describes what shipped: broadcast compaction and a standing hold per shared collection, including the counterexample that rules out broadcasting without the hold and the liveness coupling the hold introduces. Capping the frontier at the multiplexer, a synthesized AcquireHolds command, and an in-process sequence barrier are recorded as rejected alternatives with the reason each failed.

What the current revision changed

The document now matches the stacked implementation rather than the branch it was split from.

• Re-exports are aliases. An index that re-exports another's arrangement registers its id as a second name for the target's publication point, so the re-export dataflow has no operators and costs no resident memory. The document records the import-based fallback for a reader that bound first, how the target's frontiers govern the shared point, and the meet over aliases once the target drops.
• Two findings from the adversarial review were fixed and are documented as mechanism rather than kept as defects: the reader's handle mirrors TraceAgent (join on set, physical hold seeded at the chain coverage), and the import keeps no hold pinned at as_of, so a long-lived importer's compaction feeds back to the publisher.
• Fast-path peeks on the interactive runtime go through the unified IndexPeek path, with the trace provenance as the only difference.
• The testing strategy lists the driver specs, the sqllogictest, the feature and parallel benchmark scenarios and the swap scenarios, with the numbers one nightly run produced. The introspection read under hydration churn is the headline: p50 94 ms against 595 s with one runtime.

What moved out

The experimental evaluation is a project document under Interactive read isolation, which also tracks the work as issues. Evaluation artifacts do not belong in a design.

Both model checks are removed with their CI jobs. The TLA+ spec modelled the shipped mechanism; the Lean model's scope was the controller's read hold and the multiplexer's cap, and both of its counterexamples target behaviours no longer present in any form. The reasoning each contributed survives as prose in the rejected alternatives.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VDm7opomJLxbNUEP3r9BLk