fix(pipeline): give the delivery tier a log-halt latch at the claim gate (ADR 0189)

docs/CONNECTIONS.md

@@ -2285,6 +2285,13 @@ passes, and the engine starts **healthy** rather than DEGRADED. `stopped` means
 connection — deploying it is a **config change** (flip the flag, supply the values, reload), not a runtime
 action.
 
+One more outbound state sits outside that ladder: **`log_halted`** ([ADR 0189](adr/0189-a-delivery-tier-log-halt-latch-read-at-the-claim-gate-rather-than-a-gate-at-every-door.md)).
+The engine cannot write its application log and has fail-closed (#122, [ADR 0162](adr/0162-fail-closed-application-log-write-guard-detect-roll-and-stop.md)),
+so no lane in the process delivers and every outbound reports it at once. It is deliberately not
+`stopped`: nothing on that row is the fix, and start is refused until the disk is. Queued rows are
+retained PENDING throughout. `failed`, `filtered` and `not_deployed` still win over it on the display,
+because each of those is a fact about that one connection. See [SERVICE.md](SERVICE.md) for recovery.
+
 ```python
 from messagefoundry import MLLP, env, inbound, outbound
 

docs/SERVICE.md

@@ -422,6 +422,16 @@ it — the engine refuses to start otherwise, naming the collision.
    `connection_stopped` alert per halted connection naming the log as the cause, and `GET /status`'s
    `log_sinks` block, which is read from memory and still answers when the disk does not.
 
+**Every outbound reads `log_halted` on `/connections` while this is in force** (ADR 0189), not
+`stopped`. The difference is the one that decides what you do next: `stopped` means the lane is
+waiting for you to press start, and `log_halted` means it is waiting for a writable disk — pressing
+start is refused until the disk is fixed. The state is process-wide because the broken thing is, so
+every lane this engine owns shows it at once, and `outbound_running` reports false for all of them
+(so `/stats`' running/stopped split counts them as not running). A lane that failed to build, was
+parked by the DR run-profile, or is `deployed = false` keeps showing `failed` / `filtered` /
+`not_deployed` instead: those are facts about that one connection, and the halt already has its own
+alert.
+
 Recover by fixing the disk or permissions and then **restarting the affected connections** — inbounds
 *and* outbounds — from the web console, or by restarting the service; the retained queue then drains.
 **Fix the disk first — the restart is refused while the log is still unwritable.** The engine

docs/adr/README.md

@@ -209,3 +209,4 @@ what is withheld and what you can request.
 | [0186](0186-retire-the-synthetic-data-declaration-every-instance-carries-patient-data.md) | **Retire the synthetic-data declaration -- every instance carries patient data** (BACKLOG #1279) -- `[security].handles_real_patient_data = false` translated to `[ai].data_class = "synthetic"` and turned off **nineteen** start-up gates on one line (keyless at-rest, open-egress + the deny-by-default flip, the `--allow-insecure-bind` clamp, both proxy-attestation gates, MFA-at-exposure, dual-control, both TLS-terminator gates + the 12.1.1 floor probe, PHI retention, the security-notification channel + its deliverability check, the alert SMTP hop, memory-encryption-at-exposure, the API PHI serve hop, the outbound revocation hop). **Removed outright**, with `DataClass`: every instance carries patient data and the PHI gates apply unconditionally. Three findings drove it. (a) It was **not** the loud, audited opt-out the docs claimed -- `security_loosenings()` never named it, so the serve-time loosening warning never fired for the widest relaxation shipped, and the completeness test's exemption for it was both **false in its stated reason and structurally unreachable** (the loop skips non-`bool` defaults; this one defaults `None`). (b) Half the removal had already happened twice -- [ADR 0153](0153-collapse-the-posture-gradient-no-data-label-may-allow-a-cleartext-hop.md) stripped `is_phi` from the widest consumer on an argument that was never cleartext-specific, and [ADR 0148](0148-phi-default-posture-and-an-explicit-security-enforcement-level.md) GIVEN 1 left the label with opt-out as its only job. (c) Every one of the nineteen already had its **own** named, audited, separately-reported switch. Both spellings are now **REFUSED at load** (not ignored -- a config asserting the gates are off while the engine runs them all is a silent contradiction), with a message naming the per-gate replacements. `HopPosture` loses `is_phi`; `derived_posture()` / `require_posture()` return the production **tier** alone; the wire contract drops `data_class` + `synthetic_relaxation`. Retains ADR 0148 GIVEN 2, the production tier, and every per-gate + per-connection switch. **Resolves two ADR 0153 carve-outs by subtraction and makes its recorded follow-up load-bearing:** the API serve hop and the log forwarder restated the `not is_phi` ALLOW arm locally and now have **no** per-cell way to accept a risk under `enforce` (filed, unallocated -- an honest residual). CI's SQL Server load leg and the failover harness take per-gate relaxations instead, both **narrower** than the nineteen the declaration silenced. #26-clean | Accepted (2026-09-09) -- owner ruling; BUILT same day |
 | [0187](0187-bound-the-subprocess-sandbox-worker-count-a-shared-pool-or-router-phase-only-isolation.md) | **Bound the subprocess sandbox worker count: a shared pool or router-phase-only isolation** (BACKLOG #1458) -- an options memo over the growth [ADR 0087](0087-sandbox-subprocess-isolation.md)'s opt-in `mode="subprocess"` creates: one persistent, unpooled, never-evicted worker tree per traffic-carrying inbound, which is the per-connection-worker resource class [ADR 0052](0052-enterprise-scale-target.md) AC-2 names. Prices four shapes -- a bounded shared pool over both phases; router-phase-only isolation; the two combined (**recommended**); and a cap plus idle eviction -- and records per-connection `[sandbox]` settings as an orthogonal fifth. **Corrects the framing BACKLOG #1458 inherited:** the row's two candidate shapes are not alternatives, because router-phase-only isolation changes what runs in the child and not how many children exist. The load-bearing fact is that every child already runs `load_config` over the WHOLE config dir, so N children hold N identical registries and are interchangeable but for a log label. **Strongest objection carried, not buried:** the kill is the only cancellation primitive, so a pool escalates ADR 0087's accepted *"can deny its own feed"* residual into a cross-lane denial of service. **Carries a measurement** settling BACKLOG #1278's unresolved 22-versus-11 process count -- `Win32_Process` returns distinct pids in parent-child pairs, so the venv launcher's second process is real and ADR 0087's two-per-tree figure is the one to use; the live-engine test #1278 names is still unrun. **Decides nothing:** AC-2 cannot be closed by measurement while ADR 0052 `:108` records its connection-scale harness as non-existent, and the shape is the owner's ruling | **Proposed (2026-09-10)** -- awaiting the owner's shape ruling; no code, no engine behaviour change, no shape chosen. Severity conditional per CLAUDE.md section 0 -- zero deployments, and `mode="off"` ships, so this is what a deploying site that opted in would meet |
 | [0188](0188-per-connection-tls-ciphers-on-the-mllp-and-dicom-connectors.md) | **Per-connection `tls_ciphers` on the MLLP and DICOM connectors** -- the strict cipher allow-list `_APPROVED_TLS_SUITES` governed exactly ONE operator setting, `[api].tls_ciphers`, which configures the hop a browser uses to reach the web console. Every PARTNER-facing hop -- MLLP listener and destination, DICOM SCP listener and SCU destination -- had no cipher lever at all: each builds an `SSLContext`, inherits the interpreter's suite list, and `harden_cipher_suites` ASSERTS four properties on it (forward secrecy, encryption, peer authentication, a 128-bit floor) while deliberately not applying the list. So the knob existed for the engine's own console and not for the hops that carry PHI between organisations. Decision: an **opt-in** `tls_ciphers` on `MLLP()` and `DICOM()`, both directions on each, validated by `validate_tls_ciphers` itself -- the same function the `[api].tls_ciphers` field validator calls, allow-list included -- so an operator cannot put a NULL, anonymous, non-forward-secret or under-strength suite on a PHI hop. **Unset is the default and changes nothing**: no `set_ciphers`, the inherited suite list, the six CBC-SHA2 suites still negotiable. One helper, `apply_connection_tls_ciphers`, narrows; the existing `harden_cipher_suites` call stays at each of the four seams on the next line and is NOT folded into it. Folding them was the first draft and the ASVS 12.1.2 call-site guard went red on all four, correctly: that guard reads every context builder for the assertion BY NAME, and it is the only instrument that can see a seam asserting nothing. The seams narrow first and assert last, on the post-`set_ciphers` context; the draft's claim that a drifted order would be UNSAFE is withdrawn in the ADR, since `validate_tls_ciphers` runs on the string independently of the context and is strictly stronger than the assertion. What the trailing assertion adds is a check on the real context shape -- the validator probes a server context, a client context could resolve the same string differently -- and the order is now checked per seam rather than structurally guaranteed. Config surface copies [ADR 0094](0094-granular-expiry-only-tls-relaxation.md): a recognised key of the free-form `settings` mapping, not a typed field, so `connections.toml` reaches it through the same factory with no second schema. **What this deliberately does NOT do:** extend the allow-list to inherited defaults, or retire the six CBC-SHA2 suites. That retention is the ruling `harden_cipher_suites` records -- the list governs what an operator may CONFIGURE, never what a default may contain -- and retiring the six stays gated on a peer census that does not exist. Also rejected: a narrower shipped default (measured, it also ADDS two DSS suites the default did not enable) and a typed model field | **Accepted (2026-09-14)** -- built with the change. Six ACs, all six now in `tests/test_connection_tls_ciphers.py`, which the first draft cited without shipping. The load-bearing three are the unset-path set: a suite-list equality against the untouched reference construction for each context shape, a spy proving `set_ciphers` is called **not at all**, and the six CBC-SHA2 suites still negotiable. All three run per seam and are MUTATION-PROVEN -- making the unset path narrow reds all twelve cases -- and each is measured against the LOCAL OpenSSL rather than a fixed expectation, because the sub-128-bit and CBC-SHA2 populations are build properties and the two CI builds disagree. Scope: MLLP and DICOM only; the library-context hops (REST/SOAP/FHIR/LDAPS/FTPS/DICOMweb) are out of scope and `ldap3` in particular SWALLOWS an invalid `ciphers=` string. `DICOM()` stays code-first only -- it has no `connections.toml` transport name -- which is a pre-existing property of that connector, unchanged here |
+| [0189](0189-a-delivery-tier-log-halt-latch-read-at-the-claim-gate-rather-than-a-gate-at-every-door.md) | **A delivery-tier log-halt latch, read at the claim gate rather than a gate at every door** (BACKLOG #122) -- ADR 0162 makes the engine fail closed when it cannot write its application log, and CLAUDE.md section 1 says every message a connection puts out is counted and logged, so a halted engine must not deliver. The INBOUND tier enforces that with a LATCH -- `_log_halted`, teardown-surviving, read at each internal worker's loop top before the claim. **The DELIVERY tier had no state of its own:** the halt took a lane down by pausing it, which writes into `_outbound_paused`, the OPERATOR-pause set -- and `_teardown_body` CLEARS that set on purpose, and no consumer can tell a halt-pause from an operator pause because they are the same bit. So the rule could only be re-asserted at each DOOR into resuming delivery, and **four were found one at a time, each only after the previous fix shipped**: `start_outbound`, `restart_outbound`, a `stop()`+`start()` teardown, and a reload's `_unpark_outbound_lane`. Each fix was correct; the SHAPE was not -- four gates are a completeness claim nobody can verify, the SDS-3.6 liability. **Decision:** `_delivery_halted`, read at the CLAIM GATE in both claim modes -- `_delivery_worker`'s loop top (per_lane, above the pause gate and before the claim, so no row is left INFLIGHT) and `_dispatch_delivery` (pooled, the first runner-owned code a claimed row reaches, which `reschedule_claimed`s the head and parks the lane). Doors are unbounded; claim modes are two and closed. **DERIVED from `_log_write_stopped` rather than a second flag (SDS-3.5)** -- that fact is already stated once, both halt sites set it, only `_log_recovery_ok` clears it, and `_teardown_body` does not touch it, which is the teardown survival doors 3 and 4 needed. **The four door gates are KEPT as defence in depth** -- they fail fast and PAGE with the reason, which the claim gate cannot. **A `set[str]` of lane names was rejected on measurement, not taste:** `_stop_all_for_log_failure` pauses only the lanes NOT already paused, so a set built from it would omit exactly the engine-parked lane door 4 is about, and no set covers a lane BUILT AFTER the halt. Also rejected: `mark_failed` in the pooled half (spends a retry, eventually writes terminal DEAD on a row never sent -- ADR 0070's machinery-fault distinction), `release_claimed`+STOP (leaves the head past-due so the ~0.25 s sweep re-fires the gate ~4x/s, and `resume_lane` cannot re-arm a STOPPED lane), and filtering the pooled `lane_provider` alone (a partial gate that LOOKS total -- `notify_work` unions it with the known lanes and a producer wake arms the rest). **Operator-visible:** `outbound_status` gains `log_halted` ahead of running/stopping/stopped and `outbound_running` is False while it holds, so a halted lane stops reading as `stopped` -- which told an operator to press start when what the lane wanted was a writable disk -- and a lane an unguarded path brought up stops reading as `running`. `not_deployed`/`failed`/`filtered` still outrank it (per-connection facts fixed on that row), and `outbound_quiesced` still answers off the pause set so purge is unaffected | **Accepted (2026-09-15)** -- built with the change. The acceptance test uses NONE of the four doors: it drives `_start_outbound_unsafe` directly as a stand-in for the fifth door and asserts on BYTES (`list(outdir.iterdir())` over a real File connector draining a real store), with a repaired-disk control arm on the identical rig. **Measured RED on `main` in BOTH claim modes** -- `M1.hl7` reached the output directory while `guard.can_log()` read False throughout. Parametrized over `CLAIM_MODES = ["pooled", "per_lane"]`, both non-vacuous (the two modes reach the claim by different mechanisms and the gate lives in a different function in each) |