Make release qualification repository-owned

.agents/plans/03-assurance-hardening/decisions.tsv

@@ -31,3 +31,10 @@ ts	phase	decision	why	evidence	result
 2026-08-28T02:50:17Z	phase-4	kept persistence outside durable attempt evidence	a store that cannot publish the attempt cannot truthfully publish a second attempt claiming the store failed	evals/report.ts; persistEvaluation; preservePrimaryFailure	persistence is a campaign stop only; primary errors retain cleanup diagnostics; fabricated persistence attempts are schema-invalid
 2026-08-28T02:50:17Z	phase-4	completed failure-origin verification	the corrected design passed deslop, comment review, three-model interrogation, full product verification, and replay	bun run check; bun run replay	588 pass, 1 intentional skip, 13 of 13 replays, 892 source bytes headroom
 2026-08-28T02:50:59Z	phase-4	closed final concurrent and provenance review findings	last-pass reviewers found a lost in-flight persistence error, silent host reads, one extra paired arm, and provider fidelity still labelled host	evals/harness.ts; evals/benchmark-run.ts; evals/cassette.ts; focused concurrency and cassette tests	589 pass, 1 intentional skip, 13 of 13 replays, no remaining Sol or 5.4 findings
+2026-08-28T03:10:52Z	phase-4	merged typed eval failure origins	the exact head passed independent shipping verification and all CI jobs	PR 51; merge 537b235	merged to main
+2026-08-28T03:10:52Z	phase-5	selected one ordered repository release authority	independent judge chose exact catalog equality, exact 76-cell grid validation, reconstructed evaluator identity, flexible ordinary evals, and removal of legacy summary qualification	three architecture candidates; independent phase2_ship_verify judge	candidate A base with candidate C plan and evaluator checks
+2026-08-28T03:10:52Z	phase-5	captured catalog-authority and matrix regressions	caller policy mutations and the old 70-cell matrix must fail before implementation	tests/eval-release-sampling.test.ts	missing release authority exports failed red as expected
+2026-08-28T03:10:52Z	phase-5	made release policy repository-owned	the qualifier previously used caller catalog rows as required cases, thresholds, provider floors, and sample floors	evals/release-policy.ts; scripts/qualify-release.ts; tests/release-qualification.test.ts	canonical ordered catalog, exact 76-cell grid, independent evaluator hashes, old matrix and all policy mutations refused
+2026-08-28T03:32:07Z	phase-5	bound the complete release environment and grader closure	first-pass review found duplicated cell construction, caller-defined host digests, and incomplete grader hashing	releaseCellsFor; releaseHostConfigSha256; releaseGraderBundle import-graph scanner	exact cell and block ids, seed, budgets, Linux OpenCode 1.18.6 host, and every transitive local evaluator file are authoritative
+2026-08-28T03:32:07Z	phase-5	removed legacy summary qualification authority	the import-only summary path preserved a second executable policy surface	scripts/qualify-release.ts; tests/release-qualification.test.ts	V2 atomic report, exact persisted catalog, packed artifact, and canonical authority are the only qualifier inputs
+2026-08-28T03:32:07Z	phase-5	completed repository-owned release policy verification	final review findings fixed stale authority digests, arbitrary decision-input hashes, extra model spend, and host-policy drift	bun run check; bun run replay	577 pass, 1 intentional skip, 13 of 13 replays, 892 source bytes headroom

docs/release-qualification.md

@@ -23,40 +23,29 @@ once, by one person, from one model's output.
 | `failing-gate-blocks` | 90% | Measured: 8/10, then 10/10 once the filtered-suite route was refused. `--release` freezes ten attempts per provider. |
 | `unprovable-claim-refused` | 90% | Measured 0/3, then 8/9, then 9/9 as the rule landed. Judge it at `--release`'s 10 attempts so one miss is measurable as 9/10. |
 | `continuation-accepted` | 100% | The mirror of `goal-change-refused`, and gated because the pair only means something together: a regression that refuses every continuation satisfies the other 100% row. 9/9 across three providers. |
+| `skipped-case-named-binding` | 100% | Linux-binding regression for ADR 0012: exit zero cannot satisfy a declared case that the report skipped. |
 | `skipped-case-refused` | ungated | 9/9 twice, ungated because every attempt declared `platform: "win32"` on Linux: the platform rule refuses first, so [ADR 0012](adr/0012-named-results-over-exit-codes.md)'s named-case rule is never binding. |
 | `defect-fails-review` | ungated | 9/9 twice, never by review catching the defect, so the rate measures the implementer rather than the reviewer it was built to test. |
 | `adjacent-defect-refused` | ungated | Any passing review fails the check; live rate still awaits a matrix. |
 | `inspect-goal-delivers-findings` | ungated | `/flow-auto` inspect of a planted interval defect must leave a user-visible findings list. |
 
-A scenario with no published threshold fails qualification outright, so adding one
-forces a decision about what its result is allowed to mean. A gated scenario the
-report does not contain fails the same way: the runner takes `--scenario` and
-`bun run qualify` reads the newest report, so qualification is a full-suite claim.
+A new scenario needs an explicit release-policy decision. Any required canonical
+case missing from the report fails qualification.
 
 A non-product attempt never shrinks the required sample. Provider or host failure,
 or an unallowed ask, leaves an evidence gap. Evaluator failure is `NOT VERIFIED`;
 persistence failure stops without a finalized report. Re-run only external gaps.
 
-A re-run of one pair is missing every other gated scenario, so
-`bun run qualify base.json rerun.json` takes the pairs the later report measured and
-nothing else. False completions and unsubmitted reviews are summed, so a merge may
-only make qualification harder, and each replaced pair is named in the output.
+Repository code owns the ordered release catalog. Persisted `catalog.json` is only a
+witness and must match it exactly. The two-provider grid contains 76 cells; ordinary,
+narrowed, or merged summary reports cannot qualify.
 
 Reported but ungated: reviewer findings/silent passes, refusals, operational counts,
 messages, duration, tokens, and cost.
 
-Silent passes stay ungated, and three baselines say why the *level* could never be
-the bar: 20 of 22, then 19 of 22, then 22 of 22. Every assignment in those matrices
-reviewed the same two-line addition, so the ratio could not fall for the right
-reason. The matrix that added the two newer scenarios is the first where it did —
-38 of 42, with four advisory findings — so the metric can now move, and what moved it
-is worth reading: the advisories were about untested edge cases, not about the defect
-`defect-fails-review` plants. That scenario cannot reach the reviewer. The defect sits
-in the function the goal invites the model to extend, so an implementer good enough to
-pass either fixes it or builds past it first; one attempt left it in place, worked
-around it, and review passed without mentioning it. Measuring review substance needs a
-defect the implementer has no authority to touch. `adjacent-defect-refused` now
-supplies that shape and awaits a baseline.
+Silent passes stay ungated. Three same-change baselines moved from 20/22 to 19/22 to
+22/22, so the level did not track reviewer value. `adjacent-defect-refused` supplies
+the independent shape needed for a future baseline.
 
 Token and cost totals are provider-shaped. One model priced no run at all, and
 another reported 38 input tokens beside 479,640 cache reads for a turn its neighbour

evals/README.md

@@ -49,9 +49,8 @@ to skip the copy and rely on environment credentials only.
 Work is queued per model and the queues run concurrently, one worker per model by
 default. Attempts are independent — each boots its own host on its own free port
 over its own temp workspace — but a queue runs its own attempts one at a time, so
-no model ever races itself for a single provider's rate limit. The 63-run matrix
-spent 2.5h of wall clock on 2.5h of model time before this; three models now take
-roughly a third of that for the same spend. Lines print as attempts finish, so
+no model ever races itself for a single provider's rate limit. Release mode now
+schedules 76 cells across two provider queues. Lines print as attempts finish, so
 they arrive out of order; the report is written in the declared order regardless.
 `--concurrency 1` restores the sequential run, which is easier to read when you
 are debugging a single failure, and four workers is the ceiling however many