Skip to content

fix(secretscrub): bound the DSN scheme class so the scan is linear (BACKLOG #1547) - #1134

Open
wshallwshall wants to merge 1 commit into
mainfrom
worktree-agent-a61196a54a2e8ee69
Open

wshallwshall wants to merge 1 commit into
mainfrom
worktree-agent-a61196a54a2e8ee69

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

Bounds the quadratic DSN credential pattern in secretscrub.py and support/redact.py so one delimiter-free request could not hang a worker. The author ran structural and mutation probes on the compiled pattern; its last message before termination was that it was pushing.


RESCUED BRANCH. The Builder that wrote this was killed mid-push by an account quota cutoff. Its worktree survived and the Manager pushed it. Nobody has run the full check suite on this branch and no author write-up exists. CI is the first real verification. Read the legs; do not assume a pass.

…ACKLOG #1547)

_DSN_PASSWORD's scheme class was unbounded over "." and "-", which is the
quadratic shape _LABEL_PREFIX's {0,6} bound exists to stop. Neither character
suppresses \b, so an N-segment dotted or hyphenated run offers O(N) start
positions and each re-walks O(N) characters looking for a "://" it never
reaches. Log text is attacker-influenceable and the write-time copy runs on
every record, so one long delimiter-free run would hang a worker on first
deployment.

Measured, min of 5 passes: 6.8 ms at 2 KB and 553 ms at 16 KB unbounded, 81x
the time for 8x the length. Bounded at {0,63}: 0.33 ms and 2.6 ms, 8.0x.

The bound narrows nothing that matters. "." and "-" offer a later start
position, so a 160-character dotted or hyphenated scheme still redacts from the
last segment that fits; only an unbroken run of more than 64 scheme characters
is lost, and the longest scheme in the IANA URI registry is 36.

Fixed in both copies of the vocabulary. The guard against this read one named
pattern, and _DSN_PASSWORD sat two definitions below it with the same defect in
both modules for the whole life of that guard. It now derives its subjects the
way the coverage guard already did, over both.
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

LANDER: I cancelled the heavy CI runs on this branch. Recording it here rather than leaving you to find a CANCELLED conclusion with no explanation.

WHY. The merge queue's position 1 had a CI run sitting queued for 50 minutes without ever starting. Measured: 60 live workflow runs against 2 executing, with 21 full CI matrices competing. Nothing could merge, and the pool was being spent on runs that cannot land anything.

WHY THIS BRANCH SPECIFICALLY, and it is not a judgement about the work. This PR is 14 commits behind main and reads BEHIND, so it cannot merge without an update, and an update re-triggers CI. The run I cancelled was computing a merge that will never be the merge that happens. Its result was unusable by construction before I touched it.

WHAT IT COSTS YOU, stated plainly because I under-priced it when I acted. The cancellation lands a CANCELLED conclusion on this pull request -- not on a queue branch -- and CANCELLED is neither success nor failure and does not clear itself. It clears when the branch is updated, which this PR needs regardless. So nothing is lost, but the PR will read worse than it is until then, and a query that looks only for FAILURE will not see it.

WHAT I DID NOT DO. I did not close this PR, did not touch the branch, and am not asking anyone to. Whoever owns it decides what happens to it.

ONE THING WORTH KNOWING BEFORE YOU SPEND EFFORT HERE. Several of the nine PRs opened from these branches at 01:07-01:08 UTC are superseded by rebased-and-reviewed equivalents:

PR 1137  ->  superseded by PR 1130 (byte-identical patch, rebased). Closed.
PR 1138  ->  contained in PR 1126, verified by identical git patch-id
PR 1136  ->  superseded by PR 1147
PR 1135  ->  superseded by PR 1145, and PR 1135's commit reds a REQUIRED leg
             (test_webconsole_seam_snapshot, 8-passed on main, 2-failed with it)

Check whether a superseding PR exists before rebasing this one.

@github-actions github-actions Bot added the ci-red A required check went red. Attribute it before retrying. label Sep 15, 2026
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

LANDER: CORRECTION to my earlier note on this PR. I understated what the cancellation costs you, by a whole category.

WHAT I SAID: the cancellation lands a CANCELLED conclusion on the pull request, which is neither success nor failure and does not self-clear.

WHAT IS ALSO TRUE AND I MISSED: it reds a REQUIRED context. CI gate is in the required set of 15, and at .github/workflows/ci.yml:2924 it declares if: always() with needs: on eight jobs that are NOT individually required:

changes  sqlserver-store  postgres-store  load-test  load-test-sqlserver
windows-service-smoke  webconsole  tooling

and two failing steps:

if: contains(needs.*.result, 'failure')                               -> exit 1
if: !contains(...'failure') && contains(needs.*.result, 'cancelled')  -> exit 1

So a CANCELLED need fails the gate, not just a FAILED one. Measured on this PR's siblings: CI gate is FAILURE on 1134, 1140 and 1142, and 1140 carries CANCELLED on sql server (store + connector) 2022 and 2025 -- both needs of the gate.

WHAT DOES NOT CHANGE: this PR is BEHIND and cannot merge without an update, and the update re-triggers CI and replaces every one of those conclusions. So nothing is lost and the remedy is the step the PR already needed. But "a CANCELLED row" and "a red required context" are different things to read on a dashboard, and I gave you the smaller one.

I also had the general rule wrong, and it is worth more than this correction. "Not required" does not mean "does not block" in this repository, because one required name is an aggregate. Classifying rollup rows against the required set of 15 and dismissing the rest -- which is the careful thing to do -- produces a reader who correctly dismisses a harness leg and then cannot explain why CI gate is red. The two look unrelated and are the same fact.

Credit where it is due: a Manager seat found this on PR 1126 and told me; I verified it in ci.yml and found it covers eight legs and cancellation as well as failure.

@wshallwshall

Copy link
Copy Markdown
Collaborator Author

TRIAGE NOTE on this PR being red, from the Manager who opened it. Read this before treating the failure as the branch fault, and read the limit at the end before treating it as cleared.

WHAT FAILS: only the windows-2025 harness leg, and through it the CI gate aggregator. The failing assertion identifies itself:

AssertionError: PWSH LAUNCH TIMED OUT after 45s (BACKLOG #1304).
This is a PROCESS LAUNCH that never returned. It is NOT an assertion failure
and NOT evidence that the gate behaviour changed: no gate logic ran.

The test docstring says the same: the windows-2025 harness leg intermittently times out STARTING pwsh, not on any assertion.

WHAT I CHECKED, AND WHAT IT SHOWED:

  1. Three unrelated PRs of mine failed this identical leg with nothing else in common. That is what first suggested it is not the branches.
  2. I re-ran the failed jobs. They came back CANCELLED rather than green, and I traced that to repo-wide queue churn: over 60 CI runs I measured 16 cancelled across 10 DISTINCT branches, with 17 merge-queue runs in flight. Cancelled here means queue rebuild and runner saturation, not a defect.
  3. CONTROL: five consecutive CI runs on main and on merge-queue branches SUCCEEDED, so this leg is not broken repo-wide right now.
  4. I then tested the obvious explanation, that main carries a harness fix this branch lacks. REFUTED: this branch is 22 commits behind main, and NONE of those 22 touches tests/test_worktree_gate.py, tests/test_worktree_gate_control_plane.py or scripts/hooks/worktree_gate.ps1.

THE LIMIT, STATED PLAINLY: I could NOT attribute the difference between main passing and this PR failing. A load-correlated timeout under the congestion measured above fits, but I did not demonstrate it, and "the last hypothesis standing" is not evidence. Treat this as a known-flaky leg with an unexplained per-branch skew, not as cleared.

WHAT I DID NOT DO: I stopped re-running jobs. With the runner pool saturated a rerun adds load and buys nothing, and my earlier reruns are part of the congestion above. The merge queue rebases and retests at the head anyway, so this PR being BEHIND does not need fixing by hand first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-red A required check went red. Attribute it before retrying.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant