Kubernetes deployment, the mutation-testing gaps, and develop merged in - #174
Merged
Conversation
The colours encoded how an action is DRIVEN; the reader expected pass / fail / implemented. Rather than argue for one answer, the encoding is now a setting: driver, kind, priority, act, cast, flags, expect, run. There is exactly ONE painter. paintTick() writes the geometry class, the hue and a data-bucket, and it paints the timeline ticks, the legend swatches AND the settings previews. The legend's counts are then tallied OFF THE PAINTED TICKS rather than recomputed from the predicates — a second counting pass is a second thing that can disagree, which is how the previous legend came to mislabel all three of its colours. An action matching no bucket lands in a visible `unclassified` row instead of a missing tick, so the primaries always sum to 463. Pass/fail is not in recipe.jsonl, so it is not faked. The settings modal says so and then offers a Playwright JSON report to load (FileReader, no upload), joining on the [<id>] in each test title; with none loaded every tick reads "not in the report". A report with no matching ids is refused with a reason rather than half-believed, and it is deliberately NOT persisted — a status that outlives the run which produced it is a stale green. `expect` mode is the honest neighbour: it colours by what each action ASSERTS, where red means a refusal is correct. `gate` and `todo` are explained where their counts appear, in prose rather than a tooltip, because a reader had to ask what "24 gate / 65 todo" meant: gate self-skips until its RPCs exist and wakes on its own; todo is a free-text note, not a gap and not a failure. Every number in that copy is counted at runtime — a literal 24 goes stale on the next recipe edit with nothing to catch it. Exports hand out the RAW SOURCE LINES, not a re-serialisation: the embedded block is un-escaped and kept per action, so the full export is byte-identical to recipe.jsonl. JSON.stringify would have lost the file's own spacing and diffed against every line. Each button carries its row count, so nobody exports 463 believing they exported 12, and subset filenames record the filter. Modal is min(1880px, 95vw) x min(1200px, 92dvh) with the extra width given to the JSON panel (430px flat -> clamp to 519px measured). Participants are two columns, computed from cast size so three names are never split, with labels under the discs — a left-hanging label landed on the other column's discs. act mode keeps a single geometry: nine ordinal buckets do not map onto a distinguishable extent ladder, and acts are already separated by position on the bar. The mode's own copy says so and the harness exempts it with that reason recorded, rather than silently. Verified by rendering: 697 checks, 0 failures. Per mode, in both themes — legend count == the ticks it describes == counts derived independently from recipe.jsonl, labels as expected, primaries partition 463, swatch colour and extent identical to the ticks they stand for. Full export byte-identical, four subsets exporting exactly what their buttons claimed, keyboard-only mode change and export, the report loader and its refusal path, the pass/fail statement asserted on screen unscrolled, focus traps, XSS still rendered as text with exactly 2 script elements, zero non-file:// requests.
… report ## The legend note `max-width: 132ch` was wrong in both directions at once: as a block it left half a 2000px screen empty, and as a measure it still ran ~180 characters to the line. One number cannot fix both, because a character cap can only make a block smaller. The block now fills the footer (capped in px) and the text flows into columns, so the used measure lands between 62ch and ~124ch. Measured: 1440 → 1404px / 88 chars, 2000 → 1560 / 98, 2560 → 1560 / 98 out of 2524 available. ## Dragging, and why the drag wins Pointer events (mouse + touch, screen→user via the SVG's own CTM), keyboard-equal: every item is focusable with arrows to nudge, Shift for coarse, Backspace to reset one, and a DRAWN focus ring rather than `outline`, which engines disagree about painting on SVG groups. Positions clamp to the frame and persist; `Reset layout` is the way back. Dragging beats the replay by construction, not arbitration: nothing in the replay writes a position, and `animateBeam()` reads the live `pos[]` table which a drag updates ALONGSIDE the transform. Move only the transform and the beams keep arriving where the item used to be. Proven end to end — drag the Visitor, step to an action whose actor IS the Visitor, assert the beam's path starts at the moved coordinates, then nine more actions do not move it back. ## Seven dim regions, derived not invented Labels come off the data: cast groups from PEOPLE's own group column, the platform ring from every entity key starting with `sec`, the bottom band from the team/submission/asset keys, the draft from ENTITY_NAME.draft. The check that keeps it honest is a PARTITION — all 33 stage items fall in exactly one region, so an invented region leaves something in two or in none. Rects are computed from the items' own boxes plus a pad, both layers are pointer-events:none so they cannot eat the drags, and the washes share ONE opacity group because two regions genuinely interpenetrate and per-rect alpha would paint that overlap twice as a visible stripe. ## The embedded report The documented command would have failed twice: `--reporter=json` hit run.sh's catch-all and exited 2, and `> report.json` captures the Nix/devenv banner ahead of the JSON. playwright.config.ts already writes .artifacts/results.json on every run, so that is the source; a `--reporter=*` pass-through is added anyway so the flag works as written. journey 467 passed / 0 failed / 0 skipped. Reduced to id + outcome + duration: 498.3 KiB → 16.0 KiB, 463 of 463 actions matched. Shown as a DATED SNAPSHOT wherever it colours, because a status outliving its run is a stale green. A loaded file overrides it, `clear` steps back one layer at a time, and a junk file is refused while leaving the previous colouring intact. Note the file now holds THREE literal close-script tags (recipe, run-report, program), not two. The guard is updated with its reason in the harness, SKILL.md and CLAUDE.md; `</` is escaped in both data blocks; the report block sits AFTER the recipe block so the splice's terminator search is unaffected — verified idempotent by byte-identical output across two runs. ## The help modal Asked for after "gate" had to be explained twice, which is the evidence that a distant explanation does not work. Every count and example is derived from the embedded recipe at open time, so nothing goes stale on the next edit, and short glosses now sit BESIDE the jargon: "has gate — waits for its RPCs", "has todo — a design note", "expects denial — the refusal is the pass". Verified: 911 render checks (was 697), 0 failures, both themes at three widths, zero non-file:// requests, zero console errors.
…ng about it
Three faults that have been poisoning results for days. None was what it looked
like, and the first was not what I described when handing this over.
## 1. A crash loop nobody could see
The readiness probes do NOT enter the Nix shell — toolchain.nix calls grpcurl and
curl by store path. What enters it is each SERVICE's own exec (`just develop just
run`), so the whole Nix wait lands INSIDE the readiness budget, whose clock starts
when the command launches, not when a port is bound. A cold backend restart takes
486 s against a quiet lock; `just nix::develop default true` alone is 44 s, 80 s
with one competitor.
The engine was `frontend` at 54 restarts in 50 minutes: the harness's own built
server holds [::1]:8081, so vite can never bind (`Port 8081 is already in use`
x63), and uncapped `restart: on_failure` retried forever — one full `nix develop`
every ~55 s, which is what starved everything else.
It read `Running Ready` throughout, because the readiness probe is a GET of :8081
and the OTHER server answered it. **A port probe cannot say which process holds
the port.**
And the kill is invisible by design: SIGTERM after the Go handler is up exits 0,
so process-compose marks it `Completed` and `on_failure` never restarts it —
down forever, reported as a clean stop. Reproduced deterministically both on a
synthetic service and on the real one under a 12-way lock hammer.
Fixed: backend `restart: always` with `max_restarts: 3` and the threshold raised;
frontend capped too; `prod-frontend.sh ensure` now stops vite unconditionally
instead of returning early on its fast path; `wait-ready.sh` reads the restart
counters back and warns. Verified under identical pressure — restarts settle at 3
and stay, `nix develop` invocations back to zero.
## 2. The build had no lock and no atomic swap
Two callers built AND served the same `build/service`. Honest result: two
concurrent bare builds did not corrupt anything in testing — it took three people
to hit it. The INTERRUPTED build reproduces every time, leaving no entry point,
which is the missing `server/index.js` people saw.
New lib/frontend-build.sh: exclusive flock with staleness re-checked inside the
lock, build into a temp dir, pre-swap check of index.js/handler.js/server/index.js,
atomic rename. A build killed 40 s in now leaves the old tree byte-identical and
still bootable. Directory rename on the 9p mount intermittently returns EPERM —
it retries and rolls back rather than leaving no tree at all.
## 3. The browse page said "no hackathons yet" when it meant "I cannot reach the
backend"
Two causes: one module-scope channel with grpc-js's default backoff (120 s cap),
and `.catch(() => ({ hackathons: [] }))` in both public list loads — with a
comment calling an empty list "calm and truthful". It is neither. After a
7-minute outage gRPC answered again in 35 s and the page still showed 0 for a
further 51 s while the database held 8. That is why the seed data kept looking
destroyed when it was not.
Backoff capped at 2 s; both loads carry `listUnavailable` so the page says which
it means. check-reconnect.sh is the regression test: page count == gRPC count,
outage reported as an outage, recovery now 0 s.
Runs: journey 467 passed / 0 failed / 0 skipped, TWICE back to back. Smoke
137/1/2 — the failure is pre-existing and proven so with a control revert to
HEAD: 22-hackathon-pages `dragRowTo` computes endY from the destination's box
BEFORE the drag while the list reorders live, so downward drags land a row short.
Diagnosed in CLAUDE.md, not silently patched.
Not fixed, flagged: git-lfs is absent from the container, so three LFS-pointer
files read as permanently modified and the worktree is dirty with zero edits —
Nix can therefore never reach its clean-revision fast path, which is the 44 s
floor under all of the above. The fix is a Dockerfile change, i.e. a container
recreate (trap 2), so it is deliberately left to a moment when that is safe.
… keep the tint Three bugs the recipe found and had been pinning as-is. ## 1. Two definitions of "current phase", one page The hub's plan-vs-reality warning comes from `currentAndNextPhase`, which falls back to the DATES when nothing is declared. The "Enable it" button behind it posted `applyPhaseCapabilities`, which had its own narrower lookup — `phases.find(p => p.id === currentPhaseId)` — and answered 400 when that was empty. So the button was offered exactly where it could not work. Fixed by making the action resolve "current" the way the page does, not by hiding the control: the warning is TRUE in the by-dates state — the phase the calendar says is running really does name a capability participants cannot use — and declaring a phase is an explicit act nobody is obliged to perform, so gating a true, actionable warning behind a marker would report the gap in fewer situations than it exists in. There was only ever one bug: a second definition of a shared concept in one code path. The phase pointer does not move and SetCapabilities stays additive. The 400 survives for the genuinely empty case, which the panel cannot reach because `unmet` is then empty. ## 2. SetCapabilities refused the whole batch over one missing row The organiser's form posts all six; one ungoverned row answered NotFound and made the panel unusable. It now verifies the hackathon exists (a bogus id still gets NotFound — about the hackathon, which is the true statement) and upserts inside the existing transaction. Skipping the row was the dangerous alternative: UNGOVERNED is ALLOWED (`capability.State.Allowed` returns true), so dropping a row the caller set to false would report a successful save while participants kept the permission — a silent no-op on a gate. The schema already treats a full set as the invariant, so a missing row is a data gap rather than a decision. Incidental in the same handler: `capabilityStatuses` documented itself as reporting "every capability the way Get does" while building from stored rows alone, so its reply was short where Get's was six long. It uses `capabilityStatusesFromEnt` now. ## 3. .chip:hover erased .chip-active (0,2,0) beat (0,1,0), so the selected tab lost its tint under the cursor. Added `.chip-active:hover` at (0,2,0) — winning on source order, the same mechanism `.chip-active` already uses against `.chip` — rather than (0,3,0), because `chip-active` is also worn by a `btn btn-quiet` in MarkdownEditor where `.btn-quiet:hover` was erasing it identically. ## What re-specifying cost, which is the interesting part `act5.pilot.cap.unmet.bydates` pinned the 400; fixing the bug turned it red on purpose. It now asserts the switches actually move, with `nowBadge: "By dates"` as a positive control — without that it passes against a DECLARED phase, which a neighbouring action already covers. The fix also made two neighbours vacuous: `act5.pilot.phase.declare.applied` claims ADVANCING is what turns team preferences on, but the by-dates click now turns them on first, so that claim would be green whatever AdvancePhase did. Two actions were added to put the switch back and assert it. `act5.cap.ungoverned` needed re-specifying too — same request, same NotFound, different reason — so the runner gained `expect.errorMatches`: a status code says HOW a server refused, never WHAT ABOUT, and that is exactly how a re-specified action keeps passing with a lying `outcome`. The upsert path is unreachable from the API, so it is pinned in Go with a spec that deletes a row to get there. Recipe 463 → 465 actions, player re-spliced, run report regenerated from the final green run. journey 469 passed / 0 failed / 0 skipped, twice. smoke 139/1/2 (baseline 137/1/2 plus 2 new chip specs; the failure is the documented pre-existing drag bug in 22-hackathon-pages). Backend service 311/312 with 1 pending, capability 37/37, middleware 43/43. Frontend units 462. svelte-check 0 errors. Each fix was broken and watched fail before being restored: the by-id lookup (2 unit specs plus the e2e action), the old refusal (5 service specs), and the deleted hover rule (both states painting the identical pixel).
465 recipe actions ran green while only ~13 had ever been observed FAILING. This
turns "we broke things by hand once" into a check that runs, and the first run
found twelve properties nothing was guarding.
## Arenas, because the e2e suite is the wrong place for most of this
The journey is serial with chained vars, so `--grep` cannot reach an action and
`--until-act N` is the only lever; a backend mutation additionally needs the
server rebuilt against it. Minutes each, and useless while the stack is down. So
each mutation is routed to its cheapest witness: `go` (~9 s), `vitest` (4–16 s
narrowed), and `journey`/`smoke` only where nothing cheaper can see it.
The fast tier never enters `nix develop` — that shell is a repo-wide mutex
costing 44 s on a permanently-dirty worktree — and needs no running stack, which
is why it could be built straight through a container recreate. 38 mutations in
~65 min, dominated by Go recompiles and 3.5 s `git status` calls on the 9p
mount, not by tests.
## What it asserts
Exactly the expected tests go red. NO REDS fails the run loudly — that is the
signal, not an inconvenience. EXTRA REDS fails too and names what else fell.
Restoration is verified with `git status` on a trap path, and the runner refuses
to start on a dirty tree, because a mutation left applied and then committed is
the worst thing this tool could do.
## First run: 26 caught, 12 gaps
Caught exactly: all four capability states (a two-state flattening cannot hide),
both AdvancePhase boundaries, requireCapability and its organizer bypass, the
anonymous-says-Unauthenticated rule (6 reds), both global-admin gates, all three
capacity rules, all three presign conditions, all 8 client-side mutations.
The twelve gaps are all backend, and eleven cluster into three surfaces the Go
suite does not touch at all:
- requireWindowOpen — ALL of it. Deadlines never closing, the now-anchored
override ignored, registration opening early: zero reds each.
- RemoveOwner — ALL of it. The last-organizer guard, self-demotion, and
demotion-leaves-Member. Ownership is a casbin fact with no column to assert
against, which is likely why the specs were never written.
- Join's guards — private-needs-invite, already-finished, and the granted
role. Making Join hand every joiner OWNER instead of Member goes unnoticed
by every Go spec.
Plus RequireUser admitting the anonymous subject (the exact change eight
TeamService handlers were made for), and checkContentType's allowlist — the rule
keeping image/svg+xml out of an origin we serve, which has no journey witness
either.
These are gaps in the FAST tier; each is crossRef'd to the journey actions that
also pin the property. But those reds are DEDUCED from each action's declared
expect.error, never observed — so window enforcement and the last-organizer
invariant currently rest on a suite costing minutes and a whole stack.
Coupling found: flattening the client's ungoverned case reddens joinOffer as
well as capability.test.ts — the two client gates agreeing, in tests, that
UNGOVERNED permits.
## Three bugs the tool found in itself
Two edits to one file journal two backups and the second holds already-mutated
text, so replaying forwards left a file broken with every backup intact and the
journal reading as unwound — restore is newest-first now, and only a post-restore
`git status` could have caught it. `exec` in the wrapper killed its own trap,
making the belt-and-braces restore decoration. And the known-flaky filter
swallowed a genuine red: a listed test that truly witnesses a mutation belongs in
its expectReds, where the excuse cannot reach it.
Not validated: the journey and smoke arenas have never executed a mutation. The
report parser is checked against the real 469-spec results.json, and the runner
REFUSES to run a backend-mutating e2e entry against an unmutated binary — a suite
driven against stale code reports NO REDS and looks exactly like an untested
property.
…eate would lose
Three files read as permanently modified inside `dev` with zero edits:
components/frontend/static/{favicon.png,og-default.jpg} and the Keycloak theme's
favicon.ico. `.gitattributes` marks those extensions `filter=lfs`; the HEAD blob
for favicon.png is a 129-byte pointer while the worktree holds 1174 bytes of real
PNG, smudged by the Windows host, which HAS git-lfs. The container had neither
the binary nor any `filter.lfs` config, so git compared pointer against content
and reported ` M` forever. A filter that was never installed, not an edit.
`git lfs install --system`, not `--global`: /home/vscode is a named volume, so
anything the image writes to the user's home is masked at runtime.
## This does NOT fix a performance floor, and the earlier claim that it would was wrong
Measured before touching the Dockerfile, `just nix::develop default true`:
dirty tree (3 phantom LFS files) cold 14,951 ms steady ~5,600 ms
git-lfs installed, tree CLEAN cold 10,436 ms steady ~5,700 ms
A/B'd directly: with git-lfs the tree goes genuinely clean — Nix stops printing
`warning: Git tree … is dirty` — and the time does not move. Contention behaves
identically clean or dirty (3 concurrent invocations serialize to ~11 s either
way).
The real cost is self-inflicted elsewhere: tools/just/devenv.sh rewrites
.devenv/state/pwd on EVERY invocation, so the `devenv-root` flake input gets a
new lastModified and Nix's eval cache misses every single run. Against a fixed
root file the second run drops 5,466 ms → 3,677 ms. So ~1.8 s is the eval-cache
miss we cause ourselves and ~3.7 s is baseline flake eval.
The 44 s that motivated this was almost certainly measured while `frontend` was
crash-looping at one full `nix develop` per round — the 54-restarts finding
already fixed by capping max_restarts. And LFS could never have been the whole
story anyway: any UNTRACKED file makes the tree dirty for Nix too, which a peer
agent's new mutations/ directory demonstrated while this was being measured.
Keeping the change on its own merits: `git status` inside the container is now
truthful, which matters for every tool that reads it — the mutation runner
refuses to start on a dirty tree, and a repo-wide cleanliness check was already
downgraded to a scoped one because it could never pass here.
## The recreate would have silently lost 21 packages
Diffing `apt-mark showmanual` in the running container against the image found 21
Playwright dependencies living only in the writable layer — xvfb, libavcodec60,
six font packages, and the usual X/cairo/pango set. They would have gone the way
Firefox's libraries once did. All are in the image now, verified as a strict
superset of the running container's package set BEFORE recreating, with a comment
recording how to re-derive the list.
Recreated with `up -d --no-deps dev` so compose could not touch caddy or the
tunnel: same public URL, and login re-verified with the real browser round-trip
rather than by serving a page. Journey 469 passed / 0 failed / 0 skipped. Data
restored and counted over gRPC: 8 hackathons, 8 public, six editions with 4
prizes each carrying an image.
`quality-report.html` — self-contained, opens from file:// — plus the generator
beside splice-player.mjs so it can be refreshed rather than rot. It links to
recipe-player.html rather than duplicating it: the player is the animated replay
of what the suite does, this is the thing you hand someone who asks how well the
product is tested and where it is not.
Every figure is read from a file on disk and the source named. 205 figures are
registered at build time, then the FINISHED HTML is read back and each one
re-derived by a second, textual code path — regex line scans, not the parsed
objects that produced it — and any disagreement exits 1. A report nobody will
re-derive by hand is exactly where a stale number survives, and this session
produced several: a legend that mislabelled all three of its colours, hand-typed
counts that drifted on the next edit.
The strongest of those checks: the 465 `[id]` prefixes in the Playwright report
are set-equal AND order-identical to the 465 recipe ids, so "465 actions" and
"465 specs passed" are demonstrably the same 465 things.
The mutation section is blunt about what it means: 26 properties proven
catchable, 12 gaps, and a gap means nothing would notice if that behaviour
broke — `requireWindowOpen` entire, `RemoveOwner` entire, `Join`'s guards. It
carries the caveat too, that eleven of those are pinned only by journey actions
whose reds are DEDUCED from a declared expect.error rather than observed, and
that `upload.content-type-allowlist` has no witness anywhere.
Three things the derivation found while building it:
- the `--ginkgo.v` note in CLAUDE.md is STALE. internal/audit and
internal/storage both carry Ginkgo bootstraps from 1333124, an ancestor of
HEAD, and both pass with the flag. Reported as stale with the command.
- "99 of 107 RPCs have a caller" is superseded by docs/testing.md's seven; the
audit re-run at HEAD reproduces exactly those seven of 108 declarations.
Both are shown side by side rather than one silently replacing the other.
- verify.json and manifest.jsonl disagree in a way the timestamps do not
explain (the manifest's mtime is older than the run it should postdate).
Stated as-is rather than resolved.
Honest about its own limits: "RPCs the recipe calls" is labelled a FLOOR (66 of
108 is grpcurl only; the 134 browser actions reach more), the API-to-UI grep
matches by method name so a called Get masks an uncalled Get, and every number
without a machine-readable artefact behind it is marked "reported, not measured
here" with the command to re-measure.
Verified: 205/205 figures render the value they claim, both themes at 1440x900
and 390x844, zero non-file:// requests, zero console errors, no horizontal
overflow, and two consecutive builds produce an identical sha256.
…n tunnel A `plausible-stack` skill modelled on `openreplay-stack`: vendored upstream compose pinned to CE v3.2.1, non-interactive secrets into a gitignored .secrets.env that refuses to be written unless `git check-ignore` passes, its own cloudflared tunnel, and wire/restore scripts. Frontend gets a `plausible` config block, a component that injects nothing when unwired, and route-id derivation with unit tests. ## Consent: deliberately OUTSIDE the replay gate The banner asks one question in its own words — may we record your session — and consent is scoped to what was asked. Reusing that answer for a second collection is helping ourselves to permission nobody gave, the same argument that kept replay out of the registration consents. A second banner would ask about something with no artefact to permit or withdraw. What makes that honest is a property rather than a vendor claim: nothing is stored in or read from the browser, which is the thing ePrivacy requires consent for. DNT and GPC still suppress it, checked BEFORE the script is fetched, so the request is not made either. ## What it does with IPs, stated rather than glossed Plausible sees the IP and user agent and hashes them with a daily-rotated salt into a user_id; only the hash is stored. Verified against system.columns — events_v2 and sessions_v2 carry no IP or user-agent column, and verify.sh re-asks every run rather than trusting it once. No geolocation DB ships, so country and city stay empty. That is processing in transit, and the doc says so instead of calling it anonymous. ## URLs: the tracker never sends one `u` is SvelteKit's ROUTE ID — /hackathon/[id], /invite/[token] — so it cannot leak an id because it never touched one. Chosen over a scrubbing regex, which would have to enumerate what a secret looks like. Query strings dropped, internal referrers dropped, external ones reduced to origin. The cost is stated: no per-hackathon breakdowns, ever. Nothing correlates with the audit journal or with replay sessions. ## Proof A real Firefox visits /, CLICKS through to /hackathon/<uuid>, opens /invite/<token>; the captured POST bodies carry the three route ids and no uuid, token or internal referrer — and then Plausible's OWN Stats API returns exactly those three pages. Positive control first, because "no id was stored" is satisfied by an empty database. Unwired, the same script records zero requests. `--restore` proven to remove only its own key, with oidc and replay surviving intact — this is the overlay's THIRD writer and a whole-file rm there is invisible in both directions. A down --volumes / up / re-wire round trip found a real bug: the stored API key survived a wipe that deleted its row, so signup.sh probes the key and mints a new one on anything but a 200. Signup goes through `bin/plausible rpc`, not HTTP: GET /register serves a form but POST /register is 404, because registration is a LiveView handler on the websocket. Written down with its cost — it names internal Elixir functions, so it is version-pinned and fails loudly. ~750 MB RSS idle, one loopback port, coexists with the openreplay rig.
`--with-plausible` brings the Plausible stack up on its OWN tunnel and wires the frontend at it, then names the dashboard URL in the Ready block alongside the app. Its own tunnel, not this one: the tracking script and the dashboard each need a public origin, and sharing this hostname would put the app and a third-party dashboard behind a single link. Off by default, for a reason worth stating rather than discovering — it costs ~750 MB idle, and an analytics dashboard full of Playwright traffic is worse than an empty one. The Ready block says how to unwire it before a suite run. Also replaces the positional `[ "$1" = --seed ]` test with a loop that REFUSES an unrecognised argument. The old form silently ignored anything it did not expect, so `serve-public.sh --with-plausible` would have quietly started without it and looked like the feature was broken. A failure to start the rig warns and continues: the app link is the point of this script, and analytics not coming up is not a reason to withhold a working URL.
Files taken from `origin/feat/add-helm-chart` with `git archive`, not merged — that branch is based on `main`, which diverges from ours by hundreds of commits and deleted the Capability entity this branch is built on. ## The gap that mattered The chart said nothing about storage, and our backend gates the store on `cfg.Storage.Endpoint != ""` whose DEFAULT is `http://rustfs:9000` with the committed dev keys. So a chart that stays silent does not deploy "no storage" — it deploys a backend pointed at a dev host that does not exist in the cluster. The chart now always writes a `storage:` block, and `enabled: false` writes `endpoint: ""` EXPLICITLY: an honest off that answers Unavailable, rather than an omission that falls back to the dev default. Credentials never appear in values. `storage.existingSecret` is `required`, and the keys are `HACKAGON_STORAGE_ACCESSKEY`/`SECRETKEY` with no separator, because the loader lowercases and turns `_` into `.` — `ACCESS_KEY` would arrive as `storage.access.key` and be silently ignored. **No store is deployed and there is no in-cluster option.** A bucket is the one piece of state that must outlive the release, and a StatefulSet in an application chart invites `helm uninstall` to take the uploads with it. ## The Host rewrite, which is the part that would have failed in production SigV4 signs the Host header and the backend signs the STORE's hostname, so a proxy that passes the incoming host through makes every presigned PUT answer 403 SignatureDoesNotMatch while public reads keep working — the asymmetry that hid this bug for days here. Expressed for ingress-nginx (the chart's default class is the AKS app-routing addon, which is ingress-nginx) as a SEPARATE Ingress, because `rewrite-target` and `upstream-vhost` are per-Ingress and would mangle every application URL if attached to the frontend's rules. `hackagon.storageSignHost` mirrors `signHost` in internal/storage/client.go exactly — port included, bucket-qualified under virtual-hosted style. `proxy-body-size: 64m` because the largest signed rule is a 50 MiB attachment and nginx defaults to 1m. Traefik cannot express this in a core Ingress; values.yaml says so and says what to do instead. ## Other drift against our branch An `audit:` block (optional, off); `replay` and `plausible` as optional frontend blocks that render nothing when off; the backend hostname defaulting to the release's own service instead of a hard-coded `hackagon-backend`; and the realm rewrite emitting SCHEME-FULL redirect URIs — it produced `app.example.org/*`, which Keycloak reads as a relative path and rejects at login. ## Verified helm lint (also --strict) clean. Renders for storage external / off / optionals on / virtual-hosted / objects-ingress off / with the real realm JSON, all parsing as YAML including the embedded config.yaml and realm JSON. The rendered frontend config was validated by THE APP'S OWN zod schema, which is what proves the optional blocks disappear rather than rendering empty keys. Every `required` failure was triggered and its message read. No trycloudflare host, dev password or rustfs default survives in any render. ## Pre-existing problems found, deliberately not fixed here `backend.config.database.postgresPassword` renders into a ConfigMap in plaintext, and the DB/Keycloak passwords come from values.yaml — the same class as the storage credentials, decided the other way. `keycloak.realmImport. existingConfigMap` is static while the template names it `<fullname>-realm`, so the chart only works when the release is named `hackagon`. And `--set-file realmJson=…/realm-hackagon.json` imports alice/bob/charles with their dev password hashes. Not verified without a cluster: regex-path precedence over the frontend's `/` rule, ingress-nginx accepting an ExternalName upstream, and a real presigned PUT surviving the hop — argued from the rendered manifest, not observed.
Three persistent hostnames instead of a new one every restart. One tunnel per
rig — the three rigs are three compose projects on three networks, and a single
cloudflared would need all three plus a restart whenever any of them moved.
Caddy needed no change: Caddyfile.tunnel binds :80 for any Host, so the path mux
and the /objects Host rewrite apply identically. The named cloudflared is a plain
`docker run` on caddy's network, so .devcontainer/ is untouched and container
trap 2 cannot fire.
Mode is chosen automatically (named when configured, quick otherwise), forced
with --named/--quick, and PRINTED — and bringing one up stops the other, because
the OIDC issuer names exactly one hostname and the second would serve every page
while failing every login.
## The re-wiring dance is gone in named mode, but not by assuming so
A stable hostname makes the overlay byte-identical, so config-overlay.sh reports
`unchanged`. An unchanged file is not a correct process, though — that is the
whole lesson of the --restore fix — so auth-wire.sh mints a token from the wired
issuer and asks the RUNNING backend whether it accepts it before deciding there
is nothing to do.
## Credentials
The token goes to curl via `--config -` on stdin, never argv, so it cannot
appear in `ps`. The .env is refused for read AND write unless `git check-ignore`
passes.
Cloudflare API tokens scope to a ZONE, not a hostname — there is no per-subdomain
grant, and the narrowest token for this job can edit every DNS record in the
zone. Documented rather than glossed, and the tooling supplies the guard
Cloudflare cannot: cf_dns_point refuses to replace any record that is not already
a *.cfargotunnel.com CNAME.
The token is a SETUP credential and is used that way: cloudflared runs from a
per-tunnel credentials file that cannot touch DNS, enumerate the zone, or create
anything. A machine that only runs a tunnel gets that file and no .env.
## Three bugs found on the way
- the readiness probe reported success having waited for nothing:
`curl -w '%{http_code}' || echo 000` yields `000000` on failure, which is not
equal to `000`, so the loop broke on its first attempt.
- auth-wire.sh's DNS pin was gated on `getent hosts`, which says YES about a
name that resolves to an address nothing here can reach. It tests
reachability now.
- the tunnel specs hard-coded `trycloudflare\.com`. Correct while quick tunnels
were the only public path, and a lie the day a named hostname worked — every
wait would have timed out against a URL serving perfectly, reading as "login
is broken". The host derives from TUNNEL_BASE_URL now, which is what let both
modes be proven with the same specs.
## Verified against the real domain
Real browser login round-trip, and a presigned upload read back — not an HTTP
200, and not a read, since reads succeed while uploads 403 in exactly the failure
this stack has produced. Both dashboards answer on their own hostnames. Quick
mode was then re-run end to end with the same two proofs and switched back, which
reused the tunnel and found DNS already correct.
Environmental caveat, documented and not papered over: this LAN's resolver
answers AAAA-only for these names on a network with no IPv6 route, so a plain
curl from the Windows host returns 000 while --resolve to the IPv4 edge returns
200. The probes retry against a DoH-resolved IPv4 address and say "the tunnel is
fine, this machine's resolver is not" rather than reporting the tunnel down.
No token, real hostname or credential in any tracked file; both tracked
config.yaml still say localhost.
Treated as a correctness task, not a writing one. Several statements had drifted, and a confidently wrong doc is worse than a thin one. ## The 44-second Nix floor does not exist, and I had propagated it into five files `.claude/CLAUDE.md`, `toolchain.nix`, `prod-frontend.sh`, `mutate.sh` and `mutations/run.mjs` all asserted that `nix develop` costs 44 s unopposed because the worktree is permanently dirty. Re-measured across ~20 runs: **4.6–5.0 s steady state, and clean-vs-dirty is not the variable** — one modified TRACKED file measures identically. Against a fixed `devenv-root` the figure drops to 3.2–4.5 s, so ~1.7 s per entry is `tools/just/devenv.sh` rewriting `.devenv/state/pwd` and busting Nix's eval cache. Self-inflicted, and the only real recurring cost. The 44 s was measured during the frontend crash loop fixed in e06f1f3. All five sites corrected; a 36 s outlier that did not reproduce across seven later runs is recorded as an outlier rather than quietly dropped. ## Three more things that were simply wrong - the public object-store prefixes listed `hackathons/*` and `users/*` but not `site/*`, which rustfs-init.sh has granted since SITE_MEDIA landed. Confirmed with `--selftest`: three 200s and `teams/*` 403. - the Keycloak client that restricts redirect URIs is `hackagon-frontend`, not `hackagon-dev` — that is the bucket and network name. Wrong in three places. - `start.sh --tunnel` no longer means "quick tunnel"; it calls `up.sh --with-auth`, which auto-selects named mode now. The script's own banner still prints the stale wording, so the doc says the mode `up.sh` prints is authoritative. Also: the ports table was missing :8082, and the Playwright note implied the suite installs system libraries with `--with-deps` when they come from the image. ## Added git-lfs and what it did and did NOT buy; the measured shell-entry table; named tunnels with the zone-not-hostname warning, the setup-vs-runtime credential split and the AAAA-only resolver caveat with a pasteable `curl --resolve` check; "When the stack starves itself" (the crash loop, that a port probe cannot say WHICH process holds the port, and that SIGTERM after the Go handler is up exits 0 so a kill reads as a clean stop); "One writer for the frontend build" including the 9p EPERM rename quirk; the pre-recreate procedure pointing at the Dockerfile's `apt-mark showmanual` re-derivation and the 21 packages it recovered; and the mutation runner and quality report as artefacts worth knowing exist. ## Removed as obsolete Trap 4's "git-lfs is not installed / Not fixed" block — it is in the image and `git status --porcelain` in the container is empty. The mutation runner's scoped cleanliness check was justified on two grounds and one of them is now gone; the remaining reason (a repo-wide check is a check on other people's work) is stated as the reason rather than left as one of two. Every documented command was run, or is marked as not run with why. Figures quoted from the fix commits are attributed rather than re-presented as fresh measurements.
…doing it
k3d as sibling containers on the host Docker daemon. Three containers, ~2.7 GB
RAM, ~3.2 min from nothing, host ports 8090/8443/6551 chosen clear of everything
the dev stack uses. `~/.kube/config` is never touched.
One premise of mine was wrong and is corrected in SKILL.md: the devcontainer
cannot drive this — it mounts no Docker socket and has no docker CLI. The rig
runs from the host, with helm and k3d as pinned binaries in a gitignored bin/.
Two k3d defaults would have made the exercise meaningless: Traefik is disabled
and ingress-nginx installed (the /objects annotations are nginx-specific), and
the controller listens on 8090 IN-CLUSTER too, so the OIDC issuer string is true
from the browser and from the frontend pod alike. `*.localhost` rather than
nip.io — both nip.io and sslip.io were measured FAILING here, because the local
resolver applies DNS-rebinding protection.
## The three claims are now observations, not arguments
- a presigned PUT through /objects returns 200 and the bytes read back intact.
Negative control: the same signature through an Ingress identical but for the
missing `upstream-vhost` answers 403 SignatureDoesNotMatch, then works again
through the chart's route.
- the regex path takes precedence: `/` serves the document, `/objects/<k>` the
object, and `/objectsnotaprefix` does not reach the store.
- ingress-nginx accepts the ExternalName upstream, and the kill switch 503s it
on and restores it off — so the check is not vacuous.
## Five chart bugs, all found by installing it
1. keycloak-ingress.yaml hard-coded the AKS ingress class, issuer and TLS. On
any other cluster nobody claims that Ingress: the product serves and its
login does not. `keycloak.ingress.enabled` was decoration — rendered
regardless, defaulted false.
2. `backend.config.server.adminkeycloakid` ships empty and is required at
boot, so `helm install` of the chart's own defaults succeeds and the backend
crash-loops with the reason visible only in a pod log. Now `required`.
3. the OIDC callback 502'd through the chart's own Ingress — the session JWT
carries Keycloak's access and refresh tokens, so Set-Cookie is chunked and
multi-kilobyte and nginx's 4k proxy_buffer_size refuses it, AFTER Keycloak
authenticated and the app logged a successful sign-in. This hits the AKS
addon too.
4. the frontend advertised an origin that does not exist: adapter-node guesses
https, so /auth/providers pointed at https://…:8090, the sign-in form posted
nowhere and __Secure- cookies were dropped. Added protocolHeader.
5. values.yaml named the ExternalName kill switch as a ConfigMap key; it is a
flag. Setting the documented thing changes nothing, silently.
Recorded not fixed: the backend hard-exits when Keycloak's JWKS is not up yet
(self-heals in ~90 s but is indistinguishable from bug 2); the frontend's
liveness probe calls the backend, so a backend outage restarts every frontend
pod; and two keycloak.database values are ignored by the subchart and work by
coincidence.
## 37 acceptance tests, 0 failed
Including a real OIDC login round-trip and `replay`/`plausible` verified absent
from the RUNNING pod's config, read out of the live container — with the
positive control first, which earned its place: the read initially returned
"Permission denied" and both absence assertions passed against the error message.
The app is on http and Keycloak on https deliberately. An https-everywhere rig
would have agreed with adapter-node's wrong guess by accident and proved nothing.
values.k3d.yaml is tracked and carries no credential; per-cluster material is
minted into a gitignored .state/. Two warnings ride in the file, the script
output and SKILL.md: it imports the DEVELOPMENT realm because the login test
must sign someone in, and it sets NODE_TLS_REJECT_UNAUTHORIZED=0 for the rig's
self-signed cert — through the existing extraEnv hatch rather than inventing
chart surface to make a test pass.
Unverified: real TLS/cert-manager, virtual-hosted storage, storage disabled, the
frontend-proxy fallback, multi-replica, PVC-backed Postgres, Traefik. And the
app's own upload path — the published images predate this branch's storage work,
so the presign is a faithful mirror of internal/storage/sigv4.go, not the backend.
The rig served `*.localhost` over plain http with Keycloak on https — an
asymmetry chosen so an https-everywhere setup could not agree with adapter-node's
scheme guess by accident. It did its job. This is the other half: one named
Cloudflare tunnel, two ingress rules, one origin, real TLS terminated at the edge
with `X-Forwarded-Proto` arriving from cloudflared. Production's actual shape.
## One label deep, and that is forced
Cloudflare's free Universal SSL covers the apex plus ONE label. Measured against
the edge before creating anything: a two-label name gets TLS alert 40,
handshake_failure; a one-label sibling gets the zone's certificate. The token is
DNS+Tunnel scoped and cannot read the SSL settings, so this was probed rather
than read.
## The chart bug that fell out of it
`hackagon.keycloakHost` was hard-coded to `auth.{baseDomain}` while the
frontend's ingress host has always been free-form — so putting the app at a
one-label name forces Keycloak one label DEEPER, where the wildcard has no
certificate. Same shape as the hard-coded ingressClass this rig already found:
the product on a certificate that does not cover its login. Added
`keycloak.ingress.host`, defaulting to the old derivation; both branches rendered,
default unchanged.
Found and NOT fixed, recorded in SKILL.md: `hackagon.frontendHost` — which the
realm ConfigMap rewrites redirect URIs to — ignores `frontend.ingress.hosts`
entirely, giving `Invalid parameter: redirect_uri`. This rig cannot observe it
because up.sh pre-rewrites the realm.
## What real TLS proved that localhost could not
`__Secure-authjs.session-token` is issued, stored and replayed, surviving a full
page load — curl cannot test that, because `__Secure-` is a user-agent rule.
Login verified in a real browser twice across both hostnames, 13/13. Presigned
upload read back byte-identical, with the no-`upstream-vhost` control still
answering SignatureDoesNotMatch, so the Host rewrite survives the extra hop.
verify.sh 37/37 over https, twice.
## The honest result on protocolHeader
Broken and restored in both directions:
baseline https origin, sign-in 302
use-forwarded-headers: false http origin, sign-in 403 cross-site forbidden
protocolHeader: "" https origin, sign-in 302
The third row says removing `protocolHeader` breaks NOTHING over real HTTPS —
adapter-node's guess is the literal `https`, right by accident. That validates
keeping the plain-http mode rather than replacing it: only the asymmetric rig can
catch a scheme bug. Both modes stay, and `*.localhost` was re-verified 37/37
after teardown.
Also removes `NODE_TLS_REJECT_UNAUTHORIZED=0` — the frontend pod now reaches
Keycloak over the same public certificate the browser does, so the rig no longer
needs the line values.k3d.yaml calls its worst.
⚠ The tunnel publishes the DEVELOPMENT realm to the internet — alice, bob,
charles and a global-admin account on a known password. Called out in SKILL.md;
`tunnel.sh down` is one command, `destroy` also removes the DNS records. Tracked
files name no real hostname; the names live in the gitignored .env and the
generated overlay in .state/.
`docs/` had zero mentions of Helm, Kubernetes or k3d. This adds
`docs/deployment.md` beside `infrastructure.md` (which already frames "what runs
today vs what production needs"), linked from the Guides table AND from
infrastructure.md's see-also — a page reachable only by typing its URL is a
failure mode this repo has shipped more than once.
Two audiences, routed at the top: deploying it for real, and testing the chart
locally. Every bug is written with its SYMPTOM, because the symptom is how the
next person recognises it — the product serving while its login does not, an
install that succeeds while the backend crash-loops with the reason only in a pod
log, a 502 that arrives after the app has already logged a successful sign-in.
Six findings from writing it, each of which corrects or extends what I handed
over:
- the liveness-probe consequence I stated was STALE. "A backend outage restarts
every frontend pod" was true when the landing page awaited its list bare; it
catches both calls now and carries listUnavailable, so `/` returns 200 during
an outage. What remains true is the cost: the probe renders the landing page
and issues up to five gRPC calls per pod every 15 s, and liveness shares
readiness' path. The corrected version is what is written.
- the backend's hard-exit is broader than JWKS — it also exits when Postgres
refuses connections, which is what the live cluster's RESTARTS 3 actually
was.
- NEW: a config-only `helm upgrade` is a silent no-op. Configs and frontend
secrets are mounted with subPath, which never receives updates, and no
template carries a checksum/config annotation. Rotating the OIDC client
secret through `helm upgrade` changes nothing that is running.
- NOTES.txt prints the wrong frontend URL on every install (baseDomain, never
app.baseDomain) and ignores keycloak.ingress.host — the first thing an
operator reads.
- postgresPassword is in TWO ConfigMaps, not one; the keycloak-init one carries
both DB passwords in its initdb SQL.
- the unset-postgresPassword hazard is render-time, not upgrade-time: the
subchart looks up the live secret before generating, so `helm upgrade`
reuses it and only `helm template | kubectl apply` and cluster-less GitOps
mint a new one each render.
Also documented because protocolHeader depends on it: behind a TLS-terminating
edge the controller needs `use-forwarded-headers: true` paired with
`proxy-real-ip-cidr` scoped to the real proxies.
Commands were executed rather than transcribed — the required-values list was
built by rendering one failure at a time, and the /objects negative control was
re-run end to end (chart route 200 → control Ingress SignatureDoesNotMatch →
chart route 200 again). Figures quoted from b98fbdd and e0d2f6d are attributed,
and a "How this page was checked" section says which claims are which. Two
commands are marked explicitly as not run.
The tunnel mode's exposure of the development realm is a blockquoted warning
under its own commands, not a footnote. No real hostname, token or generated
credential appears anywhere in the page.
`helm upgrade` reported success, `kubectl get configmap` showed the new value,
and every running pod kept serving the old one. Rotating the OIDC client secret
that way changed nothing that was running, with no signal at all.
Two causes: config and secrets are mounted with `subPath`, which never receives
updates, and no template carried a checksum annotation — so a ConfigMap change
left the pod template byte-identical and Kubernetes correctly did nothing.
Measured with the fix reverted: the upgrade returned in 0.9 s, status `deployed`,
ConfigMap `forceDevLog: false`, live pod `forceDevLog: true`, SAME pod name, same
generation, no pod-template annotations at all. The pod was never even restarted
— so the missing annotation is what made it silent; `subPath` is only what makes
a restart necessary.
`checksum/config` on both deployments plus `checksum/secret` on the frontend.
Both `annotations:` blocks were `{{- with }}`-guarded, so the key existed only
when someone set `backend.annotations`; the checksum renders unconditionally now
and any user map merges after it. subPath mounts untouched — switching to
whole-directory mounts changes what else lands there and is a far bigger blast
radius than an annotation.
## The thing this could have traded itself for
A checksum over a template that renders non-deterministically turns a silent
no-op into a rollout on EVERY upgrade. Measured in the state most likely to break
it — rendered twice with `postgresql.auth.postgresPassword` empty, which makes
the Bitnami subchart mint a fresh password per render: its `postgres-password`
and Keycloak's `admin-password` both differed between renders while all three of
our checksums were byte-identical. Structurally, these ConfigMaps take the DB
password from a `required` value that can only come from values, so the
subchart's generated one never reaches them.
Found while checking: the chart ALREADY ships a non-deterministic checksum — the
Keycloak subchart's own `checksum/secrets` changes on every `helm template`.
Harmless under `helm upgrade` (the lookup finds the existing Secret; its
generation held at 5 across seven upgrades) but live for the
`helm template | kubectl apply` pipeline the docs already warn about. Not fixed
here, recorded so nobody blames rollout churn on this change.
## Proven against a running cluster, not a render
Bug established first with the fix reverted, then: the same change rolls the
Deployment and the live pod reads the new value with nobody running `rollout
restart`; a backend-only change moves backend 2→3 with frontend frozen at 10, and
a frontend-only change moves 10→11 with backend frozen; and three consecutive
identical upgrades leave every generation and both pod names unchanged. That last
one is the assertion that separates a fix from a rollout-on-every-upgrade bug
wearing a fix's clothes.
verify.sh 37 → 55 checks, and the checks were mutated to prove they bite: with
the templates reverted, 45 passed / 10 failed, naming
`expected forceDevLog: false, got forceDevLog: true`. Two "renders agree" checks
passed VACUOUSLY there (empty == empty) — which is what the count assertions
above them exist for, and those failed.
## On the secret hash, kept deliberately
`checksum/secret` puts a sha256 of secret material where `get deployment` can
read it while the Secret itself needs `get secret`. It is not the secret, but it
is an oracle for a guess. Every alternative is worse: anything that changes when
the secret changes is the same oracle, and anything that does not is the original
bug. Its safety rests on the secret's entropy, which is now written down as a
requirement rather than left as an assumption. Residual also recorded: a
ConfigMap edited OUTSIDE Helm still never reaches a running pod.
Behaviour that could be broken with the whole Go suite staying green. 28 new specs across four surfaces; the manifest now reads 37 EXACT / 1 GAP / 0 NO REDS. Every expectation was RECORDED, read, and frozen — never predicted. Each entry was re-run afterwards and reports EXACT, because a test that covers the function without failing under its mutation has added coverage that proves nothing, which is the disease this tooling exists to detect. Closed: all three `requireWindowOpen` rules (never closing, opening early, the now-anchored override ignored); three of the four `RemoveOwner` guards (last organizer, self-demotion, demotion restoring Member); all three `Join` guards including the ROLE it grants — making Join hand every joiner Owner used to go unnoticed; `RequireUser` admitting the anonymous subject, which is the exact change eight TeamService handlers were made for; and `checkContentType`'s allowlist, the rule keeping image/svg+xml out of an origin we serve. ## The one left open, deliberately `owner.mutex-dropped`. Now that RemoveOwner has specs, all three are SERIAL, and dropping `ownerMu` changes nothing about calls that never overlap. A Go spec that reddened under it would have to hammer two concurrent RemoveOwners and hope — passing either way most of the time. Its witness stays `act5.race.owner.remove`, the recipe action that reproduced the zero-owners race for real, and the gapReason now says why it is still a gap rather than implying nobody tried. ## Design notes Ownership is casbin state with no column to assert against, so the specs read through the enforcer the test server actually uses; RemoveOwner returns an empty message, so "returned OK" says nothing. Demotion-restores-Member is only visible on someone who did NOT already hold Member. The last-organizer guard runs before the self-demotion one, so those two specs have to be arranged around each other. Windows are written relative to time.Now() — no clock control and no sleeps, because a test that sleeps is a test that flakes. ## Found while writing, not fixed `ConfigService.OverrideWindow` cannot express an already-expired override — protovalidate holds extend_minutes to 1..1440 — so that state is written to the row directly, with the reason in a comment. And `act6.window.override` only asserts the RPC returns OK, never that the override REOPENS anything: the journey witness for `window.override-ignored` was weaker than the manifest implied. The Go spec covers it properly now. Cost: 9.3 s wall for all six packages, unchanged (service 2.46 → 2.52 s, 312 → 337 specs). No existing test weakened or deleted. Also recorded in CLAUDE.md: `capacity.oversell-by-one` returned MISMATCH once when its KNOWN_FLAKY spec stayed GREEN under its own mutation — three isolated re-runs were EXACT. An expected red that fails to arrive is a MISMATCH and the flake list cannot excuse it, which is the correct behaviour; noting it so the next person sees a known shape rather than a new bug.
develop is 40 commits ahead of the d51a0d7 base, this branch 17. Product code (components/**, helm-chart/**, docs/**, api/proto/**) merged clean; all 16 conflicts were in .claude/. Every one of those conflicts is treefmt against hand-written tooling. develop's 5a7b253 ran the formatter across the whole repo, which rewrote 103 files under .claude/ — shfmt to 4-space, prettier's trailing commas and *em* -> _em_ — and then 3ba79be EXEMPTED .claude/** from treefmt on the grounds that it "carries its own conventions (2-space shell, hand-wrapped markdown)". The reformat was never reverted, so develop's tree contradicts develop's own stated rule for that directory. Checked before choosing, not assumed: normalising both sides of every one of those 103 files (whitespace, commas and semicolons stripped) leaves 76 byte-identical, and the remaining 27 differ only in quote style, a dropped line-continuation, added parentheses around an awaited import, a union's leading `|`, and CSS reflow. There is no semantic change anywhere in develop's .claude diff. So .claude/ is taken from this branch WHOLE — not just the 16 conflicted files — because a directory in two shell styles would have been the lasting cost of splitting the difference. One behavioural disagreement survives the textual merge and is NOT resolved here: develop's 942b60a makes the hackathon manage sidebar flat and its unit test now asserts "draws no fold control at all", while this branch's recipe action act5.nav.fold and its sidebarManageFold helper assert the fold starts closed, toggles, persists and self-opens. Both sides observed the same force-open-inside-/manage/* behaviour and drew opposite conclusions from it. develop owns the product decision; the action is re-specified to it in the commit that follows, per the rule that a recipe action a fix turns red gets re-specified rather than deleted.
develop's 942b60a removed the fold from the hackathon manage sidebar — HackathonSidebar no longer passes parentItem/open/onToggle, and its unit test now asserts "draws no fold control at all". act5.nav.fold and sidebarManageFold pinned the opposite: starts folded behind "Show Manage Hackathon pages", toggles, remembers the choice per browser, and self-opens on a page inside the section. Git merged the two without a conflict, because they live in different files. Only running it finds this, which is the point worth recording. Both sides watched the SAME behaviour — the fold force-opened on entering /manage/* — and disagreed about it. Ours held that deriving the fold from the route made the chevron a no-op on every manage page, i.e. a control that lies; develop's holds that a disclosure which is always open wherever an organiser uses it is dead weight on a rail whose height then depends on a state nothing announces. develop owns the product decision. So the action is re-specified rather than deleted, per the rule that a fix turning a recipe action red is the mechanism working; the id stays act5.nav.fold, which is what the embedded run report joins on. Inverting the old assertions would not have been enough. "The entries are there" passes against a fold too, once it is open, and this fold opened itself on exactly the pages such a check looks at — so a per-page presence assertion could never have told the two designs apart. The claim now is that the rail is IDENTICAL outside Manage and inside it, which is what flat means and what a fold cannot satisfy by construction. The hub and entry being visible on a participant page is the positive control that stops the "no disclosure" zero from agreeing with an empty nav. Recipe banner and .claude/CLAUDE.md follow the same rename; the player and the quality report are re-generated from the edited JSONL (477 lines, 465 actions, 3 literal </script> tags).
22 smoke reds after the merge, in two files, all label changes and none of them a regression. Collected in one run before anything was edited — fixing reds as they appear hides the shape, and the shape here is that both files pinned chrome develop deliberately rebuilt. ## 02-login (8 reds) — the header nav develop's 143a961 makes the primary nav TWO entries, renames "Hackathons" to "All Hackathons" (the wordmark beside it already reads Hackathons, so the bare noun appeared twice in one row) and drops About from the bar — the page stays, the entry does not. The property both tests were written for is untouched: one meaning per entry, and the same set on both sides of the login. Only the membership moved, so the two are driven off one NAV_ENTRIES table now. About's removal gets a positive control rather than a bare absence. "No About in the header" is satisfied just as well by an About that was deleted from the product, so the test asserts the header has none AND that the footer's Platform nav still points at /about. An absence with no positive control agrees with everything — that is four entries in CLAUDE.md's own list. ## 21-footer (14 reds) — the rebuilt footer develop's 0265838 rebuilt the footer after datascience.ch: "Terms" is "Terms of use" (so the link text and the page's own <h1> finally agree), and there is no GitHub link at all — the off-site row is SDSC's channels. Three changes, and two of them make the checks stronger than a rename would have: - FOOTER_LINKS carries the nav LANDMARK with each label. Not tidiness: the new footer also links datascience.ch/about, so footer-wide `name: "About"` now matches two links to two different places. A `.first()` would have silently followed whichever the DOM ordered first. Reported as a product finding too — two links with one accessible name in one region is what a screen-reader link list cannot disambiguate. - the GitHub test is RETIRED rather than repaired, because its subject left the product, and replaced with a claim about the row that took its place: every off-site anchor is absolute https AND has an accessible name. The second half is new and is the point — an icon-only link that loses its aria-label is invisible to a screen reader and to every name-based locator, and looks completely fine on screen. - "the same footer on both sides of the login" compared the public footer to the FOOTER_LINKS constant, which made it a claim about the footer's SIZE; the rebuild grew it from 4 links to 14, so the constant was wrong when a copy edit landed rather than when anything broke. Same disease as 03-dashboard's `connectedCount: 3`. The property was always the EQUALITY of the two sides, and that is asserted directly now, with a non-empty check so two empty lists cannot satisfy it. The new off-site check earned itself on its first run by failing: it called the ETH and EPFL logo links nameless, because it read aria-label and textContent and those are `<a><img alt="ETH Zurich"></a>`. The alt text names the link perfectly well, so the check computes the name the way a screen reader does. A check that reports a correct page as broken is a check that gets deleted. ## Two recipe actions, same audit - act5.flow.reach.manage dropped its "Show Manage Hackathon pages" click: the rail is flat, so the chain is one click shorter. The claim — you can GET to a deadline form from the dashboard by clicking — is unchanged. - act8.form.ui.edit follows a control that MOVED rather than one that went away. develop's c596683 removed the overview's "Your registration answers -> View or edit" block and its 7603784 put the entry point on the participants roster, where View opens your own editable form and ?userId= someone else's for organisers. The product rule is the one this action exists for — SubmitRegistrationForm is an upsert precisely so a first typo is not permanent, which needs a way in from the UI — so the locator moved and the claim did not. It is matched on the href (the View with no ?userId= is your own row) rather than on a display name. smoke: 141 passed / 1 failed -> 142 passed after the accessible-name fix. The 22-hackathon-pages drag test, red in the recorded baseline, passed in both runs.
The first post-merge journey run stopped at act2.flow.bob with 337 not
run — `mode: "serial"` is load-bearing for state, so one red ends the
act. The cause was develop's +error.svelte, which replaced a single
always-Home button with a way out that knows where it is: a 403 inside
an event reads the hackathon id off the URL and offers "Back to this
hackathon", landing on the PUBLIC event page.
For a waitlisted person that is the page offering Join — i.e. the one
place a refusal should send them — so it is a better answer to the
question these two actions ask, which was never "does Home work" but
"is the refusal a dead end". act2.flow.bob and act8.flow.charles are
re-specified to it.
Both were STRENGTHENED while being rewritten rather than merely
repointed. "A link was clicked and the URL changed" passes against a
link back to anywhere, so each now asserts which event it landed on.
That matters more here than usual: the destination is derived from a
regex over the pathname, so a wrong id is exactly the failure this
design can have and a URL-shape check would have agreed with it.
Guessing which labels moved does not scale, so the rest of the audit
was mechanical: every static clickLink/clickButton/expectHeading/
expectText literal in recipe.jsonl, greppable against
components/frontend/src. After these fixes the only literals with no
hit are the ones that are DATA — a hackathon name, a persona, an
aria-label template, organiser-authored form fields. Nothing else in
the recipe asserts a control develop moved.
Baselines, both re-established post-merge and both re-derived:
journey 469 passed / 0 failed / 0 skipped (4 setup + 465 actions)
smoke 142 passed / 0 failed
backend 6 packages, ~13 s; service 336/337, capability 37,
middleware 46
frontend 29 files, 488 passed
mutate.sh check: all 38 mutations still anchor
The player and the quality report are regenerated, the latter now
carrying the 2026-08-14 run: 194 figures re-derived by a second code
path and matched. It refused the first rebuild — ".claude/CLAUDE.md
claims journey 469 but results.json recorded 120" — because the file
still held the old table while the artifacts held a smoke run. That is
the tool doing its job, and the reason the table is edited in the same
commit as the numbers it reports.
Two findings recorded rather than fixed, both in develop's own tree:
(app)/account/+page.svelte still directs people to "Your registration
answers -> View or edit", copy for a control c596683 removed; and the
rebuilt footer links datascience.ch/about beside our /about, so one
region carries two links with the accessible name "About".
And one recorded rather than claimed as fixed: the 22-hackathon-pages
drag test, deterministically red on 2026-08-13, passed on both runs
today. Nothing here touches dragRowTo, so it is left OPEN with its
diagnosis — a timing-shaped defect that stops reproducing has not been
explained.
The footer carried TWO links whose accessible name was exactly "About" — our platform page and datascience.ch's — in one region. A screen reader's link list discards surrounding context, so it read "About, About" with no way to tell which one leaves the site. Ours is now "About Hackagon" in its VISIBLE text. An aria-label was the wrong tool: it replaces the label a sighted person reads, so "click About" would stop working for voice control and the visible text would no longer be contained in the accessible name. Ours renamed rather than theirs, on a rule rather than a coin toss: "About Hackagon" is that SitePage's own <h1>, in cmd/seed/main.go and in the journey's act0.about.create, and develop's comment records that the SDSC labels were read off datascience.ch's markup rather than invented. So both links now name their destination the way that destination names itself, which a copy edit on one side cannot re-converge. Also: the account page still advertised "Your registration answers → View or edit" for a control develop removed. It now names where they actually live — the event's Participants list, View on your own row. Deliberately not a link: /account's load returns your profile and nothing else, so an href would have to guess which event you meant. The destination is proven by the spec asking the server for a 200. ## The checks assert the property, not a list `expectFooterLinkNamesUnique` asserts that inside the footer landmark no accessible name identifies more than one link, with two positive controls (there are links; none is nameless). The previous version compared against a four-entry constant, which made it an assertion about the footer's SIZE — it broke the moment develop grew the footer to fourteen links. `footerLinkNames()` only ENUMERATES candidates; each name is handed back to Playwright's own role/name engine to be counted, so a drift between our computation and its matcher surfaces as a count of zero rather than passing quietly. Two CONTROL tests are checked in and run every suite: one puts the name collision back in the live DOM and asserts rejection, the other feeds the account walker the old wording and asserts rejection. Reverting both product fixes and running the three affected spec files gives 19 failed / 25 passed, naming `"About" does not name exactly one link` with both candidate hrefs. smoke 146 passed / 0 failed (baseline 142 + 4). Frontend units 488, pnpm check 0 errors. Journey not re-run and did not need to be: no action clicks the footer's About, and act8.form.ui.edit's steps are unchanged — only its todo note. Recorded, not fixed: the five datascience.ch links carry target=_blank with no icon, suffix or word, so nothing tells a non-sighted user they leave the site. The smallest fix is an sr-only suffix inside each anchor, which APPENDS to the accessible name and so keeps voice control working — the opposite trade from an aria-label.
Develop ran treefmt across the repo (5a7b253, 103 files under .claude), then exempted `.claude/**` from it (3ba79be) WITHOUT reverting the reformat — so its tree carried formatted files while its config said the directory was exempt. The merge resolved that by keeping our hand-formatted version; this adopts develop's actual tree instead, and removes the exemption so the directory cannot drift apart again on the next edit. The exemption's stated reason was broader than "conventions": it also cleared the last shellcheck findings, all of which live in .claude. So shellcheck stays excluded for that directory — it rewrites nothing, it only fails the run, and .claude carries 54 findings whose repair means editing logic. Formatting is on so the tree stops drifting; the lint backlog is separable and would have turned CI red on day one. The two GENERATED HTMLs are excluded outright, and prettier's lockfile exclude is generalised now that two more live under .claude. ## One thing was not pure layout, and it stopped rather than guessed Prettier mangles CLAUDE.md at a `**`-containing code span inside bold: in place it DELETED a space, rendering `exempted .claude/**from treefmt`, and left a 700-char line in an 80-column hand-wrapped file; in isolation it escaped the closing `**` and killed the bold. Fixed at source to a form that round-trips — prettier is idempotent over the file now. ## And the reformat broke one anchor `build-quality-report.mjs` located its input with `indexOf("| Suite | Result | When |")`, and prettier pads table cells. Same anchor-drift class as the mutation manifest, just not the manifest — every other anchor into that file already collapsed whitespace first, and this one now does too. ## Evidence, since `git diff -w` does not settle it `-w` ignores whitespace within lines but cannot fold away the line splits and joins that are most of what shfmt and prettier did, so it reports 100 files and proves nothing. Instead: `bash -n` clean on 65/65 shell scripts; 47 files byte-identical after stripping all whitespace and, after also stripping quotes/commas/semicolons, only 28 differ — each read, all standard normalisations (case-branch expansion, leading-operator continuations to trailing, double to single quotes, YAML flow to block, `*em*` to `_em_`). All 11 markdown files render identically through a CommonMark parser, prose word for word. Every regex literal across 52 .ts/.mjs files is byte-identical, and template OUTPUT text is unchanged everywhere but the two files edited above. smoke 148/0, frontend units 488, pnpm check and lint 0 errors, all 38 mutation anchors valid, and splice-player regenerates the player byte-identically. Found and left alone: treefmt also reformats 9 files OUTSIDE .claude (backend tests, frontend, docs, a helm template) — pre-existing drift that makes CI red independently of this change. Reverted here rather than smuggled in.
The five datascience.ch links carried target=_blank with no icon, suffix or word, so nothing told a non-sighted user they leave the site. The nav landmark is named "Swiss Data Science Center", but that is exactly the context a screen reader's link list discards — the same reason the duplicate "About" existed. An sr-only suffix inside each anchor, because it APPENDS to the accessible name: the visible text stays contained in it and voice control keeps working. An aria-label would replace what a sighted person reads — the opposite trade, and why fbc81ad changed visible text instead. Applied by what each link already says, not uniformly. The five text links name the site and the tab. ETH Zurich and EPFL get the bare "(opens in a new tab)" — they already name their destination through `img alt`, so naming it twice would be noise. The three socials are icon-only, where an sr-only child is ignored because the aria-label IS the whole name, so theirs is extended instead. No visible indicator: the only existing idiom is a button-style CTA, not a footer text-link pattern, so this is the screen-reader half alone and says so. ## Three things worth knowing `footerLinkNames()` was itself wrong and had to be fixed first — it read textContent and then appended `alt`, which would have named the logo links "(opens in a new tab) ETH Zurich", back to front. It walks child nodes in document order now. Without that fix the EXISTING uniqueness check would have broken, since it hands each name back to Playwright's engine and a back-to-front name matches nothing. The collision CONTROL test went red, correctly: the suffix means "About" no longer collides with "About (datascience.ch…)", so the control had stopped reproducing the defect it exists to reproduce. It now strips the suffix AND renames ours, because two independent fixes hold that pair apart. The check asserts the PROPERTY — every footer link that opens in a new tab says so in its accessible name — with a >=4 positive control, not a list of five. A list would be an assertion about how many external links the footer has, which is the mistake that spec already made once. Proof it can fail: removing EPFL's suffix at source turned it red naming exactly `"EPFL → https://epfl.ch"`; restored, and a DOM-level control repeats it every run. 23 footer tests pass; smoke 148/0.
…rift
CI has been red since the chart landed. `helm-chart/templates/*.yaml` are Go
templates that happen to end in .yaml — a directive can stand where a key
belongs (`name: {{ include … }}`, whole blocks under `{{- if }}`) — so no YAML
parser accepts them. prettier exited 2 on backend-configmap.yaml line 4 and took
the entire formatter run down with it, which is why the failure looked unrelated
to the chart. Excluded, with the reason in the file; `helm lint` and
`helm template` are what actually check those.
With the run no longer aborting, treefmt then reformatted 9 files that had
drifted independently of any of this work — two backend tests, three frontend
sources, three docs and the quality-report generator. Committed rather than left,
because `check::format` fails on any change and there is no version of "CI is
green" that skips them.
Verified idempotent: a second `just check::format` reports 0 changed.
…te it The generator wrote the file and then checked it. A validator that runs after the write certifies nothing — the bad artefact exists either way, and on a machine where nobody reads the exit code it ships. A previous run left an `undefined`-filled report on disk that had to be reverted by hand. The document is assembled in memory and every read-back check now runs against that string. On any disagreement it prints both values, says NOTHING WAS WRITTEN and that the file still holds the previous report byte for byte, and exits before touching the disk. On a clean pass it writes a same-directory temp, fsyncs, renames (retrying EPERM — 9p, container trap 5), then re-reads the destination and throws unless it equals the verified string. Proved against the same broken input rather than assumed: the old code left the report's sha changed with two `undefined` stamps and two visible; the new code leaves the sha identical. A second proof drives the failure through a different path — reordering one table cell so the build path and the re-derivation read different numbers — giving four named disagreements, an unchanged sha and no temp files. A successful build changes the file's INODE, because it is replaced by rename rather than overwritten. Regenerated from a real run: journey 469 passed / 0 failed / 0 skipped in 3.2 m, whose spec ids are the recipe's ids in the same order. Archive restored and counted over gRPC — 8 hackathons, every edition with 4 prizes carrying images. Two consecutive builds produce an identical sha256, and 68 headline figures were re-checked by a third, independent path. ## Regenerating alone would have shipped false prose With the manifest down to one gap, the mutation section rendered "0 surfaces account for 0 of the 1 gaps" over an empty list — beneath hand-written prose still claiming windows, RemoveOwner and Join have no Go specs, which is the opposite of its own table. The smoke tile likewise rendered a red "0 failed" beside "one deterministic failure". Both derive from their sources now, as does the note about the doc being stale — which would itself have gone stale the moment the doc was fixed. ## And the recorded reason for a red CI check was wrong `just check::test -c backend` is still red, but NOT because `internal/audit` and `internal/storage` trip over `--ginkgo.v` — both report ok under the quitsh runner, with bootstraps added in 1333124. The single failure is the declared KNOWN_FLAKY capacity spec, and three consecutive re-runs of that package went green. CLAUDE.md's stale spec counts are replaced with measured ones, along with smoke 142 → 148, the todo count, and the API-to-UI figures. Left alone deliberately: `embed-run-report.mjs` has the identical write-then-validate bug, a one-block reorder, kept out to keep this diff focused. And `verify.json` is a 2026-08-13 recording that disagrees with the current manifest; re-running the mutation suite applies deliberate breakages to components/ while another session is committing here, so the report now states the discrepancy instead of hiding it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
developis merged in (40 commits) and this branch adds 17 of its own plus themerge and its follow-ups. Product code merged clean; every conflict was in
.claude/and every one of them was the formatter. Both e2e baselines arere-established after the merge, against develop's UI.
go test -tags "test unittest" ./internal/...pnpm checkmutate.sh checkhelm lint helm-chartWhat this brings
Deploying it on Kubernetes, and five chart bugs found by installing it
The chart from PR #161 is in, taught about object storage, and then actually
installed on a real cluster — which is where the bugs were. All five were
invisible to
helm template:keycloak-ingress.yamlhard-coded the AKS ingress class, issuer and TLS, soon any other cluster nobody claims that Ingress: the product serves and its
login does not.
keycloak.ingress.enabledwas decoration — renderedregardless, defaulted false.
backend.config.server.adminkeycloakidshipped empty and is required atboot, so
helm installof the chart's own defaults succeeds and the backendcrash-loops with the reason visible only in a pod log.
carries Keycloak's tokens, so
Set-Cookieis chunked and multi-kilobyte andnginx's 4k
proxy_buffer_sizerefuses it, after Keycloak authenticatedand the app logged a successful sign-in. Hits the AKS addon too.
https, so/auth/providerspointed athttps://…:8090, the sign-in formposted nowhere and
__Secure-cookies were dropped.values.yamlnamed the ExternalName kill switch as a ConfigMap key; it is aflag. Setting the documented thing changed nothing, silently.
And the one worth reading twice: a config-only
helm upgradewas a silentno-op.
helm upgradereported success,kubectl get configmapshowed the newvalue, and every running pod kept serving the old one — so rotating the OIDC
client secret that way changed nothing that was running, with no signal at all.
Two causes:
subPathmounts never receive updates, and no template carried achecksum annotation, so a ConfigMap change left the pod template byte-identical
and Kubernetes correctly did nothing. Measured with the fix reverted: upgrade
returned in 0.9 s, status
deployed, ConfigMapfalse, live podtrue, samepod name, same generation — never even restarted.
Fixed with
checksum/configon both deployments andchecksum/secreton thefrontend, and proven against a running cluster rather than a render: the value
reaches the live pod with nobody running
rollout restart; a backend-only changemoves backend 2→3 with the frontend frozen; and three consecutive identical
upgrades leave every generation and both pod names unchanged — that last one
is what separates a fix from a rollout-on-every-upgrade bug wearing a fix's
clothes.
verify.shwent 37 → 55 checks and the checks were mutated to provethey bite (reverted templates: 45 passed / 10 failed, naming the exact value).
docs/deployment.mdis new (docs/had zero mentions of Helm, Kubernetes ork3d), linked from the Guides table and from
infrastructure.md. Every bug iswritten with its symptom, because that is how the next person recognises it.
Three residual hazards are recorded rather than fixed — including that a
ConfigMap edited outside Helm still never reaches a running pod, and that the
Keycloak subchart already ships a non-deterministic checksum.
A real HTTPS hostname for the k3d rig
One named Cloudflare tunnel, two ingress rules, TLS terminated at the edge with
X-Forwarded-Protoarriving from cloudflared — production's actual shape. Itproved what localhost cannot:
__Secure-authjs.session-tokenis issued, storedand replayed across a full page load (curl cannot test that;
__Secure-is auser-agent rule). It also found a sixth chart bug on the way —
hackagon.keycloakHostwas hard-coded toauth.{baseDomain}while thefrontend's ingress host has always been free-form, so a one-label app name
forces Keycloak one label deeper, where Cloudflare's Universal SSL wildcard has
no certificate.
Closing the gaps mutation testing found
28 new Go specs across four surfaces; the manifest went 12 NO REDS → 37 EXACT
/ 1 GAP / 0 NO REDS, with the whole fast tier still at ~9.3 s. Closed: all
three
requireWindowOpenrules, three of fourRemoveOwnerguards, all threeJoinguards including the role a join grants (makingJoinhand everyjoiner Owner used to go completely unnoticed),
RequireUseradmitting theanonymous subject, and
checkContentType's allowlist.Every expectation was recorded, read, then frozen — never predicted — and each
entry re-run afterwards to confirm EXACT, because a test that covers a function
without failing under its mutation has added coverage that proves nothing. One
gap stays open deliberately (
owner.mutex-dropped: the three RemoveOwner specsare serial, so dropping the mutex changes nothing they can see; its witness is
the journey's race action).
Three product fixes
warning falls back to the DATES when no phase is declared, while the button
behind it posted a handler that looked the phase up by
current_phase_idalone and answered
400 "no current phase"whenever that was empty — i.e. inthe state most events are in, since declaring a phase is an act nobody has to
perform. One definition of "current" across the product now.
SetCapabilitiesrefused a whole batch over one ungoverned row, and thepanel posts all six switches on every save, so one absent row made the
capability screen unusable with a 404 as its only explanation and no RPC
anywhere that could create the missing row. It creates the row now. Skipping
was the dangerous alternative —
UNGOVERNEDis allowed, so dropping a rowthe caller set to
falsewould report a successful save while participantskept the permission.
.chip:hover(0,2,0) beat.chip-active(0,1,0), so pointing at the tabyou were already on erased its accent tint. The test asserts the computed
pixel, never the class:
chip-activewas on the element the whole time thebug shipped.
Stopping the stack from killing itself
nix developis a repo-wide mutex and every service in the stack goes throughit, so a crash-looping process starves everything else's startup budget. Found
live: process-compose's
frontendat 54 restarts in 50 minutes whileprocess listsaidRunning Readythroughout — its readiness probe is acurlon :8081, which the other server was answering. A probe on a port cannot
tell you which process holds it. Fixed with restart caps, a bigger backend
budget, and
wait-ready.shreading the restart counters back.Same commit: an empty list is not an answer. The browse page rendered 0
events while grpcurl returned 8 from the same database — a module-scope gRPC
channel on a 120 s reconnect backoff, plus a load doing
.catch(() => ({ hackathons: [] }))with a comment calling an empty list "calm and truthful".Calm yes; truthful no — "the database is empty" and "I cannot reach the backend"
had become the same page.
Tooling
Named Cloudflare tunnels on a domain you own (quick tunnels stay the zero-setup
path), a self-hosted Plausible rig behind its own tunnel, git-lfs baked into the
devcontainer image, and a quality report that re-derives every number it
prints from its sources by a second code path — 194 figures, and it refuses to
build when they disagree. It did exactly that during this merge, which is how the
stale suite table got caught.
The merge itself
Merge base
d51a0d78is a day old, so develop already holds this branch'swork up to it. 40 in, 17 out, and all 16 conflicts were in
.claude/—components/,helm-chart/,docs/andapi/proto/merged clean.The
.claude/conflicts were all the formatter5a7b253bran treefmt across the whole repo, rewriting 103 files under.claude/(shfmt to 4-space, prettier's trailing commas,*em*→_em_), and3ba79beathen exempted.claude/**from treefmt because it "carries itsown conventions (2-space shell, hand-wrapped markdown)" — without reverting the
reformat, so develop's tree contradicts develop's own rule for that directory.
Checked rather than assumed before picking a side: normalising both versions of
all 103 files (whitespace, commas and semicolons stripped) leaves 76
byte-identical, and the other 27 differ only in quote style, a dropped
line-continuation, parentheses around an awaited import, a union's leading
|,and CSS reflow. No semantic change anywhere. So
.claude/is taken from thisbranch whole — all 103 files, not just the 16 that conflicted — because a
directory in two shell styles is the cost that would have outlived the merge.
The conflicts that were NOT textual — six recipe actions and two smoke specs
Git merged these without a murmur, because the two sides touch different files.
Only running the suite finds them. Where develop and this branch disagreed
about how the interface behaves, develop wins and the recipe was adapted to it —
re-specified, never deleted, because an action that a fix turns red is the
mechanism working.
The reds were collected in full before anything was edited (22 in smoke, in
exactly two files; one in the journey, which is serial and therefore stops the
act). None of them was a regression.
942b60a7— the manage sidebar is flat, no foldact5.nav.fold+sidebarManageFoldasserted the fold starts closed, toggles, persists and self-opens143a9612— nav is two entries; "Hackathons" → "All Hackathons"; About dropped from the bar02-loginNAV_ENTRIEStable drives both tests; About's absence gets a positive control02658384— footer rebuilt; "Terms" → "Terms of use"; no GitHub link21-footer+error.svelte— context-aware way out of a 403act2.flow.bob,act8.flow.charlesclicked "Go back to Homepage"c596683c+76037844— registration answers moved from the overview to the participants rosteract8.form.ui.editact5.flow.reach.manageclicked the unfoldThree of these are worth spelling out, because inverting the old assertion would
have been the wrong repair:
open — and this fold force-opened on exactly the pages such a check would look
at, so a presence assertion could never have told the two designs apart. What
it asserts now is the equality of the rail outside and inside Manage, which
is what flat means and what a fold cannot satisfy by construction.
refusal a dead end". develop's answer is better — a waitlisted person lands on
the public event page, which is the one that offers Join — and since the
destination is derived from a regex over the pathname, a wrong id is exactly
the failure this design can have, so the actions now name the event they reach.
"the same footer on both sides of the login"compared the public footer to a 4-entry constant, which made it a claim about
the footer's size; the rebuild grew it to 14 links, so the constant was
wrong the moment a copy edit landed rather than when anything broke. The
property was always the equality of the two sides, and that is what it asserts.
The new off-site footer check earned itself on its first run by failing — it
called the ETH and EPFL logo links nameless, because it read
aria-labelandtextContentand those are<a><img alt="ETH Zurich"></a>. The alt text namesthe link perfectly well, so the check computes the name the way a screen reader
does. That was a bug in the check, not the page.
Guessing which labels moved does not scale, so the rest of the audit was
mechanical: every static
clickLink/clickButton/expectHeading/expectTextliteral in
recipe.jsonl, greppable againstcomponents/frontend/src. After thefixes the only literals with no hit are the ones that are data — a hackathon
name, a persona, an
aria-labeltemplate, organiser-authored form fields.For the reviewer
Three things found in develop's tree and deliberately not fixed here — they
are UX/copy calls that belong to whoever owns that work:
memberNavreturns 10entries and
manageNavanother 10, so an organiser's rail is 20 rows in asticky h-[calc(100vh-3.5rem)]column. develop is right that a disclosurewhich force-opens wherever it matters is not the answer; it does not make the
column shorter. Not reintroducing the fold to close this.
(app)/account/+page.sveltestill directs people to "Your registrationanswers → View or edit" — copy for a control
c596683cremoved. Thedestination moved to the participants roster; the instructions did not.
datascience.ch/aboutbeside our own/about,so one region carries two links with the accessible name "About" — exactly
what a screen-reader link list cannot disambiguate. It is why the footer
checks scope to a nav landmark now instead of searching footer-wide.
Nothing is known-failing. The
22-hackathon-pagesdrag test, which wasdeterministically red on 2026-08-13, passed on both post-merge runs. Nothing
here touches
dragRowTo, so it is recorded as OPEN with its diagnosis ratherthan claimed fixed — a timing-shaped defect that stops reproducing has not been
explained.
recipe-player.htmlis generated: re-spliced from the JSONL after every recipeedit (477 lines, 465 actions, 3 literal close tags) rather than hand-merged.