Skip to content

docs: realign branch-protection sources of truth with live protection - #163

Merged
fullstackjam merged 3 commits into
mainfrom
claude/agitated-cori-7b9bed
Aug 2, 2026
Merged

docs: realign branch-protection sources of truth with live protection#163
fullstackjam merged 3 commits into
mainfrom
claude/agitated-cori-7b9bed

Conversation

@fullstackjam

@fullstackjam fullstackjam commented Aug 2, 2026

Copy link
Copy Markdown
Member

What does this PR do?

Brings .github/required-checks.txt, docs/MERGE_POLICY.md, and docs/HARNESS.md back in line with the six status checks that branch protection on main actually requires today — and corrects three factual errors about CI that those docs already contained.

Why?

#161 made contract schema (L2), curl|bash smoke, and old-cli compat run on pull_request and added them to branch protection, but the in-repo files still described the previous three-check world:

  • .github/required-checks.txt listed 3 contexts while calling itself the source of truth for required_status_checks.contexts; live protection returns 6.
  • docs/MERGE_POLICY.md still had a "Why these three" section and filed the other three under post-merge, does not block merge — the opposite of what protection now enforces.
  • docs/HARNESS.md still showed the curl|bash smoke row triggering on "push to main / dispatch".

The maintainer confirmed the six-check state is the intended one, so the in-repo files catch up to protection rather than the reverse. Live branch protection is unchanged by this PR — no gh api -X PUT was run, and none is needed to land it.

The pre-merge/post-merge split in MERGE_POLICY.md is gone rather than re-labelled: every job in test.yml fires on push, pull_request, repository_dispatch, and workflow_dispatch with no job-level if: gating, so there is no post-merge-only tier left to document.

Second commit: three CI claims that were wrong

Fact-checking the first commit against the actual workflows turned up three errors, two of them pre-existing and one I had just introduced. All are verified by reading the files, not inferred:

  1. vm-e2e does not run on push to main. vm-e2e-spike.yml:4-5 scopes its push trigger to the test/vm-e2e-speed spike branch. My new sentence "all six run on every PR and again on push to main" was false. It is PR-only, and the doc now says so — worth a maintainer's eye, since CONTRIBUTING.md:46 and auto-release.yml:179 both instruct you to confirm "L4 CI is green on the latest commit on main", which cannot happen.
  2. curl|bash smoke never executes scripts/install.sh. The job curls localhost:18888/testuser/test-config/install, which scripts/mock-server.py:46-60 serves as a synthetic stub that execs the freshly built binary in dry-run mode. grep -rn "install.sh" .github/workflows/ returns nothing. The real installer is covered in L1 by test/integration/install_script_test.go:88, which pipes it through /bin/bash -s precisely to exercise the curl | bash mode. The old table row ("Confirms scripts/install.sh still bootstraps the CLI") and my new bullet ("which no Go test exercises end to end") were both wrong.
  3. The network-dependency section said "three" but named two, and described the wrong failure mode. old-cli compat ends its release lookup with || true and gates every subsequent step on steps.prev.outputs.version != '' (test.yml:187-190), so an API blip or a missing arch asset yields a green check that ran no compat test at all. It cannot block a PR. The section now names the real asymmetry: contract schema (L2) fails closed, old-cli compat fails open.

Also corrected: the doc said the drift sensor "fails on PRs", but it is continue-on-error: true (harness.yml:110) and only annotates.

Testing

Docs and one config file; no Go code changed.

  • go vet ./... passes
  • Relevant tests added or updated — n/a, no Go code changed; no Go test reads either file (grep -rn "required-checks\|MERGE_POLICY" --include="*.go" . is empty)
  • Tested locally — replicated the required-checks alignment (drift) sensor script verbatim against a PyYAML venv: ✓ All 6 required checks have matching workflow jobs. Diffed the file's non-comment lines against gh api .../branches/main/protection --jq '.required_status_checks.contexts[]' — empty diff, exact set match. make test-unit exits 0 with zero failures; go test ./internal/archtest/... passes. All 14 CI checks green on both commits.

Cross-repo checklist

  • Does this need a docs/content update in openboot.dev? — No; CI policy documentation internal to the CLI repo.
  • Does this change the CLI ↔ server API contract? — No.

Notes for reviewer

Scope note. Corrections 2 and 3 above fix pre-existing inaccuracies on lines this PR was already editing. That is slightly wider than "realign the required-checks list", but shipping a doc whose stated purpose is accuracy while knowingly leaving false claims in it seemed worse. Happy to split them out if you'd rather.

Three findings surfaced, not fixed — each is out of scope and one touches .github/workflows/:

  1. old-cli compat fails open. Documented here, not fixed. Making it fail loudly when no previous release resolves would turn a decorative green tick into a real gate.
  2. CONTRIBUTING.md:46 and auto-release.yml:179,184 tell the release operator to verify L4 is green on main, which vm-e2e-spike.yml's trigger makes impossible. Either the workflow should gain a main push trigger or those instructions should point at the PR run.
  3. The claude-review bot has posted nothing on this PR, twice (10 permission denials on the latest run, 7 on the first). Root cause is in claude-code-review.yml: the claude_args allowlist is whitespace-split, so Bash(gh pr diff:*) is parsed as three separate entries — the run log shows "Bash(gh", "pr", "diff:*)", "view:*)", "comment:*)". Quoting the --allowed-tools value should fix it. The review gate for this PR was closed locally instead, which is what produced the second commit.

Branch protection was not touched. Worth confirming it still reads as 6 contexts when you look, but nothing here modifies it.

PR #161 made `contract schema (L2)`, `curl|bash smoke`, and `old-cli
compat` run on pull_request and added them to branch protection on main,
but left the in-repo files describing the previous three-check world.
Live protection requires six contexts; the repo claimed three.

The maintainer confirmed the six-check state is intended, so the in-repo
files catch up to protection rather than the reverse. Live protection is
unchanged by this commit.

- .github/required-checks.txt: add the three missing contexts, so the
  file matches `.required_status_checks.contexts` exactly.
- docs/MERGE_POLICY.md: drop the pre-merge/post-merge split, which no
  longer describes anything real — every job in test.yml fires on push,
  pull_request, and both dispatch events with no job-level `if:`, and
  vm-e2e on push and pull_request. "Why these three" becomes "Why these
  six". Adds the external-state trade-off these required checks carry,
  and notes the drift sensor's blind spot: it compares the file against
  workflow job names only, never against live protection, which is how
  this drift went unnoticed.
- docs/HARNESS.md: the curl|bash smoke row said "push to main / dispatch"
  and the L2 contract row said "CI"; both now say "every PR".
@github-actions github-actions Bot added the docs label Aug 2, 2026
fullstackjam and others added 2 commits August 2, 2026 11:16
A fact-check of the previous commit against the actual workflows found
three wrong claims in the new text. All verified by reading the files.

- `vm-e2e` does not run on push to `main`. vm-e2e-spike.yml scopes its
  push trigger to the `test/vm-e2e-speed` spike branch, so the claim
  "all six run on every PR and again on push to main" was false. It is
  PR-only, now stated as such.
- `curl|bash smoke` never executes `scripts/install.sh`. The job curls
  `localhost:18888/testuser/test-config/install`, which mock-server.py
  serves as a synthetic stub that execs the freshly built binary in
  dry-run mode; no workflow references scripts/install.sh at all. The
  real installer is covered in L1 by install_script_test.go, which pipes
  it through `/bin/bash -s`. Both the table row (pre-existing error) and
  the new "no Go test exercises it end to end" bullet were wrong.
- The network-dependency section said "three" but named two, and claimed
  a GitHub API blip or yanked asset blocks all PRs. `old-cli compat`
  ends its lookup with `|| true` and gates every later step on a
  non-empty version, so it passes green having tested nothing. Rewritten
  to name the real asymmetry: L2 blocks, cli-compat fails open.

Also corrects "fails on PRs" for the drift sensor, which is
continue-on-error, and notes its second blind spot: it only checks that
listed checks have jobs, never that required contexts are listed.
@fullstackjam
fullstackjam merged commit d9f7431 into main Aug 2, 2026
14 checks passed
@fullstackjam
fullstackjam deleted the claude/agitated-cori-7b9bed branch August 2, 2026 05:20
fullstackjam added a commit that referenced this pull request Aug 2, 2026
The claude-review job has never been able to post. On PR #163 it ran
three times, each ending in permission denials and "No buffered inline
comments" — 7, then 10, then silence — so no PR in this repo has had a
bot review despite the workflow being wired up.

claude_args is parsed with a shell-style splitter (the action ships
base-action/test/parse-shell-args.test.ts, which covers both quote
styles). Unquoted, `Bash(gh pr diff:*)` splits on its spaces. PR #163's
run log shows the resulting allowlist:

  "mcp__github_inline_comment__create_inline_comment",
  "Bash(gh", "pr", "diff:*)", "view:*)", "comment:*)"

None of those match a real permission, so every `gh pr diff/view/comment`
call hit the gate and the buffered review was dropped.

Double-quoting the value keeps it a single token. Verified with
shlex.split, which is the same splitting model:

  '--allowed-tools'
  'mcp__github_inline_comment__create_inline_comment,Bash(gh pr diff:*),...'

This PR is its own test: if the fix works, the bot posts a review here.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant