diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 3e8b1aa8..354818e9 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -504,6 +504,43 @@ fixes the only ones with no hit are the ones that are DATA (a hackathon name, a persona, an `aria-label` template, organiser-authored form fields). Run that before a journey rather than paying a full run per red. +## Ways a HANDOVER was confidently wrong + +Distinct from the section below, and it cost more. Those are tests that agreed +with a broken product; these are briefs that told an agent something false with +enough confidence that it could have been taken on trust. Five in one week, all +mine, all caught only because the brief said "verify this before acting on it" +and the agent did: + +- **A 44-second Nix floor that did not exist.** Handed over as fact and already + propagated into five files as the justification for design decisions. + Measured: 4.6-5.0 s steady state, and clean-vs-dirty is not the variable. The + 44 s had been measured during a frontend crash loop that was fixed hours + earlier. A wrong number stated once became load-bearing in four other files + within a day. +- **git-lfs framed as the fix for that floor.** It makes `git status` truthful, + which is worth having; it does not change the timing at all. The agent A/B'd + it before touching the Dockerfile and said so. +- **A vacuity guard removed on a wrong theory.** "A no-op preview lists nothing" + — except the table renders every planned row including unchanged ones. + Deleting the wait made the two assertions after it pass instantly against a + page that never rendered. I introduced two silent-green assertions while + explaining why I was right. +- **"The devcontainer can drive k3d."** It mounts no Docker socket and has no + docker CLI. The agent found out in its first minute and rewrote that half of + the brief. +- **A stale consequence stated as current.** "A backend outage restarts every + frontend pod" was true before the landing page learned to catch its own gRPC + failures. The cost is real (the probe issues up to five calls per pod every 15 + s) but the consequence had changed. + +What made the difference every time was a brief that said **verify this rather +than transcribe it**, and named what would count as disproof. The failure mode +is not a lie — it is a true-once observation restated after its context moved, +which is exactly the shape nobody re-checks. Two habits follow: attribute a +number to the run that produced it, and when handing over a diagnosis, hand over +the measurement that would falsify it. + ## Ways a test reported green while proving nothing The most expensive category of bug here, because nothing turns red. All of these diff --git a/.claude/skills/hackathon-e2e/.gitignore b/.claude/skills/hackathon-e2e/.gitignore index e4be43da..4da3b954 100644 --- a/.claude/skills/hackathon-e2e/.gitignore +++ b/.claude/skills/hackathon-e2e/.gitignore @@ -3,7 +3,9 @@ node_modules/ .artifacts/ test-results/ playwright-report/ -# build-quality-report.mjs stages the report beside itself and renames it into -# place, so this only exists while a build is in flight — or after one was -# killed mid-write, which is exactly when it must not read as a tracked change. +# build-quality-report.mjs and embed-run-report.mjs stage their output beside +# itself and rename it into place, so these only exist while a build is in +# flight — or after one was killed mid-write, which is exactly when they must +# not read as a tracked change. .quality-report.html.tmp-* +.recipe-player.html.tmp-* diff --git a/.claude/skills/hackathon-e2e/mutations/run.mjs b/.claude/skills/hackathon-e2e/mutations/run.mjs index 61368302..567b11b4 100644 --- a/.claude/skills/hackathon-e2e/mutations/run.mjs +++ b/.claude/skills/hackathon-e2e/mutations/run.mjs @@ -201,6 +201,47 @@ function apply(mut) { }) } +/** + * Put one backup back, retrying the 9p refusal — and never by COPYING. + * + * `fs.copyFileSync` on this repo's bind mount intermittently answers + * `EPERM: operation not permitted, copyfile` with nothing holding either file: + * the same refusal container trap 5 documents for a rename, and the same one + * `writeChecked` in build-quality-report.mjs already retries around. Here it was + * far worse than a failed restore, because **Node removes the destination when a + * copy fails after opening it** — so the file being restored was DELETED, the + * journal still claimed the mutation was applied, and every later entry aborted + * on `assertCleanTree`. Observed three times in one afternoon (2026-08-15), at + * entries 1, 7 and 9 of three separate runs; each run lost every entry after it. + * + * Two changes, for two different halves of that: + * - read-then-write instead of copyfile. A write can leave the destination + * wrong, but it cannot unlink it, so the worst case stops being "the source + * file is gone" — which is the only outcome nothing downstream can undo. + * - retry the EPERM, because it is transient (the very next restore of the + * same file succeeded, every time). + * The bytes are read back and compared, because a restore this tool did not + * verify is exactly what "restoration is verified, not assumed" is about. + */ +function restoreFile(backup, abs) { + const bytes = fs.readFileSync(backup) + const sleep = (ms) => + Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms) + for (let attempt = 1; ; attempt++) { + try { + fs.writeFileSync(abs, bytes) + break + } catch (e) { + if (attempt >= 5 || e.code !== "EPERM") throw e + sleep(250) + } + } + if (!fs.readFileSync(abs).equals(bytes)) + throw new Error( + `restore: ${abs} does not match its backup after being written back`, + ) +} + /** * Undo everything the journal records, NEWEST FIRST. * @@ -225,7 +266,7 @@ function restore({ quiet = false } = {}) { spawnSync("git", ["checkout", "--", e.file], { cwd: ROOT, env: ENV }) continue } - fs.copyFileSync(e.backup, path.join(ROOT, e.file)) + restoreFile(e.backup, path.join(ROOT, e.file)) if (!quiet) console.log(C.dim(` restored ${e.file} (${e.id})`)) } writeJournal([]) diff --git a/.claude/skills/hackathon-e2e/quality-report.html b/.claude/skills/hackathon-e2e/quality-report.html index c2f34981..d08f0380 100644 --- a/.claude/skills/hackathon-e2e/quality-report.html +++ b/.claude/skills/hackathon-e2e/quality-report.html @@ -186,7 +186,7 @@
Hackagon · e2e quality report
-
sketch/06-08-26 @ ea66ff97 · 2026-08-15
+
chore/loose-ends @ abb7f921 · 2026-08-15