From 51b4e52343f58202f7b5c0ffca96317997ec05b8 Mon Sep 17 00:00:00 2001 From: Conduction Release Bot Date: Sun, 30 Aug 2026 09:01:01 +0200 Subject: [PATCH 1/4] feat(fleet): admit dependabot, and carry what lands back downstream Dependabot reads its config from the repository's DEFAULT branch only. Ten fleet apps default to main, whose .github/dependabot.yml carries no target-branch, so every bump is opened against main -- where branch-protection refused it. Measured 2026-08-30: 41 open pull requests that could never merge, across zaakafhandelapp (12), filinq (11), openregister (5), opencatalogi (3), integriq (3), dossiq (2), planninq (2) and one each on thematiq, launchpad and stackiq. Dependabot reopens them, so the backlog only grew. Three changes: * branch-protection.yml admits dependabot/* into main and beta, alongside the existing beta, hotfix/* and release/v* allowances. Exercised across 12 source/target pairs: the six that should pass do, and feature/* is still refused into both main and beta, so the allowance is narrow. * fleet-back-merge.yml carries whatever reaches main down into beta, and whatever reaches beta down into development. It does not try to identify the source of each commit -- squash merges, direct pushes and force-pushes all defeat that. It asks the equivalent question the refs can answer exactly: does the upstream branch hold anything the downstream one does not? * fleet-apps.json gains planninq. It listed 21 names but planninq was not among them, so the app was invisible to every fleet sweep -- the exact failure the file's own comment warns about. A dry run of the real logic against the fleet reports 42 outstanding back-merges and 2 correct skips (humaniq has neither main nor beta). Every app has main ahead of beta and beta ahead of development, which is why promotions have been conflicting on version files and lockfiles. Merging is held to the same rule as fleet-friday-merge: every check reported and none failed. Pending, cancelled, unreadable and ZERO checks are each a refusal -- a conflicted pull request runs no CI at all, and "no failures" is exactly what that looks like. Verified: YAML parses; bash -n on each run block extracted from the parsed YAML; and the hop logic exercised against a mocked gh across 9 cases, including the two that matter most -- an empty comparison and a literal "null" are both refused rather than read as "in sync". --- .github/workflows/branch-protection.yml | 26 ++- .github/workflows/fleet-back-merge.yml | 265 ++++++++++++++++++++++++ fleet-apps.json | 1 + 3 files changed, 288 insertions(+), 4 deletions(-) create mode 100644 .github/workflows/fleet-back-merge.yml diff --git a/.github/workflows/branch-protection.yml b/.github/workflows/branch-protection.yml index bf9a8e01..2534f6e0 100644 --- a/.github/workflows/branch-protection.yml +++ b/.github/workflows/branch-protection.yml @@ -28,28 +28,46 @@ jobs: # It is a narrow allowance, not a hole: release.yml stages only # appinfo/info.xml and, where present, openapi.json, and hard-fails # its own run if the bump commit strays outside that allow-list. + # `dependabot/*` is allowed for the same reason, and it is the fix + # for a backlog that could not drain. Dependabot reads its config from + # the repository's DEFAULT branch only. For the ten fleet apps whose + # default is `main`, `main`'s .github/dependabot.yml carries no + # `target-branch`, so every bump was opened against `main` — where this + # gate refused it. Measured 2026-08-30: 41 open pull requests across + # zaakafhandelapp (12), filinq (11), openregister (5), opencatalogi (3), + # integriq (3), dossiq (2), planninq (2), and one each on thematiq, + # launchpad and stackiq. None could ever merge, and Dependabot reopens + # them, so the backlog only grew. + # + # Allowing the bump in is only half of it: a dependency landing on + # `main` alone leaves `beta` and `development` behind, and the next + # promotion conflicts on the lockfile. fleet-back-merge.yml carries + # whatever reaches `main` down into `beta`, and whatever reaches `beta` + # down into `development`, so the three converge instead of diverging. if [[ "$TARGET" == "main" || "$TARGET" == "master" ]]; then - if [[ "$SOURCE" != "beta" ]] && ! [[ "$SOURCE" =~ ^hotfix ]] && ! [[ "$SOURCE" =~ ^release/ ]]; then - echo "::error::Pull requests to main must come from 'beta', a 'hotfix/*' branch, or a 'release/v*' version bump." + if [[ "$SOURCE" != "beta" ]] && ! [[ "$SOURCE" =~ ^hotfix ]] && ! [[ "$SOURCE" =~ ^release/ ]] && ! [[ "$SOURCE" =~ ^dependabot/ ]]; then + echo "::error::Pull requests to main must come from 'beta', a 'hotfix/*' branch, a 'release/v*' version bump, or 'dependabot/*'." echo "" echo "Allowed patterns:" echo " beta → main" echo " hotfix/* → main" echo " release/v* → main (automated version bump)" + echo " dependabot/* → main (dependency bump, back-merged downstream)" echo "" echo "Your branch '$SOURCE' does not match any allowed pattern." exit 1 fi elif [[ "$TARGET" == "beta" ]]; then - if [[ "$SOURCE" != "development" ]] && ! [[ "$SOURCE" =~ ^hotfix ]] && [[ "$SOURCE" != "main" ]] && [[ "$SOURCE" != "master" ]] && ! [[ "$SOURCE" =~ ^release/ ]]; then - echo "::error::Pull requests to beta must come from 'development', 'main', a 'hotfix/*' branch, or a 'release/v*' version bump." + if [[ "$SOURCE" != "development" ]] && ! [[ "$SOURCE" =~ ^hotfix ]] && [[ "$SOURCE" != "main" ]] && [[ "$SOURCE" != "master" ]] && ! [[ "$SOURCE" =~ ^release/ ]] && ! [[ "$SOURCE" =~ ^dependabot/ ]]; then + echo "::error::Pull requests to beta must come from 'development', 'main', a 'hotfix/*' branch, a 'release/v*' version bump, or 'dependabot/*'." echo "" echo "Allowed patterns:" echo " development → beta" echo " hotfix/* → beta" echo " main → beta (backport)" echo " release/v* → beta (automated version bump)" + echo " dependabot/* → beta (dependency bump, back-merged downstream)" echo "" echo "Your branch '$SOURCE' does not match any allowed pattern." exit 1 diff --git a/.github/workflows/fleet-back-merge.yml b/.github/workflows/fleet-back-merge.yml new file mode 100644 index 00000000..ef0d6002 --- /dev/null +++ b/.github/workflows/fleet-back-merge.yml @@ -0,0 +1,265 @@ +# Fleet back-merge — carry what lands downstream, so the three branches converge. +# +# ══════════════════════════════════════════════════════════════════════════ +# THE PROBLEM THIS SOLVES +# ══════════════════════════════════════════════════════════════════════════ +# +# The fleet promotes UPWARD: development → beta → main. Nothing carried +# content the other way, and content does arrive out of band: +# +# • Dependabot bumps. Dependabot reads its config from the repository's +# DEFAULT branch only. Ten fleet apps default to `main`, whose +# .github/dependabot.yml carries no `target-branch`, so every bump is +# opened against `main`. branch-protection.yml now admits those (see the +# `dependabot/*` arm there) — but a dependency that lands on `main` alone +# leaves `beta` and `development` on the old version. +# • `hotfix/*`, which is allowed straight into `main` or `beta` by design. +# • `release/v*` version bumps, which commit onto the branch they ran from. +# +# Every one of those makes the next promotion conflict on the very file that +# was changed — a lockfile or appinfo/info.xml — because both branches then +# changed it independently. release.yml already carries its own bump back +# (the `sync/beta-to-development-*` pull requests); this generalises that to +# anything that arrives, from any source. +# +# ══════════════════════════════════════════════════════════════════════════ +# HOW IT DECIDES, AND WHY IT DOES NOT LOOK AT THE SOURCE +# ══════════════════════════════════════════════════════════════════════════ +# +# The rule as stated is "a merge into main that did not come from beta should +# go to beta". Implementing that literally means identifying the pull request +# behind each commit and reading its head — which is fragile across squash +# merges, direct pushes and force-pushes, all of which occur here. +# +# The equivalent question is answerable from the refs alone, and is exact: +# +# is there anything on `main` that `beta` does not have? +# +# If beta → main was the only route, the answer is no and nothing happens. +# If ANYTHING else landed, the answer is yes, whatever its source. This also +# catches the case the literal rule misses: a squash merge of beta into main +# creates a commit `beta` genuinely does not contain, and leaving that +# unreconciled is what makes the following promotion conflict. +# +# ══════════════════════════════════════════════════════════════════════════ +# GREEN MEANS GREEN +# ══════════════════════════════════════════════════════════════════════════ +# +# Merging is held to the same rule as fleet-friday-merge.yml: EVERY check +# reported and NONE failed. Pending, cancelled, timed-out and an unreadable +# response are each a refusal, not a pass — an empty API response has been +# read as "no failing checks" in this fleet before, and merged a red PR. +# +# ZERO checks is likewise not green: a conflicted pull request runs no CI at +# all, and "no failures" is exactly what that looks like. +name: Fleet back-merge + +on: + schedule: + # Every three hours. A bump that reaches `main` should not wait a day to + # reach `development`, because the promotion it will collide with can be + # opened at any time. + - cron: "40 */3 * * *" + workflow_dispatch: + inputs: + dry-run: + description: "Report what would be opened/merged, change nothing" + required: false + type: boolean + default: true + merge: + description: "Merge back-merge PRs that are fully green" + required: false + type: boolean + default: true + +permissions: + contents: read + +concurrency: + # Never let two sweeps open competing pull requests for the same repo. + group: fleet-back-merge + cancel-in-progress: false + +jobs: + back-merge: + runs-on: ubuntu-latest + timeout-minutes: 45 + + steps: + - uses: actions/checkout@v4 + + - name: Assert a cross-repo token is present + env: + FLEET_DISPATCH_TOKEN: ${{ secrets.FLEET_DISPATCH_TOKEN }} + run: | + set -euo pipefail + if [ -z "${FLEET_DISPATCH_TOKEN}" ]; then + echo "::error::FLEET_DISPATCH_TOKEN is not set, so this routine can carry nothing." + echo "It needs 'contents: write' and 'pull-requests: write' on every repo in fleet-apps.json." + exit 1 + fi + echo "Token present." + + - name: Carry main into beta, and beta into development + env: + GH_TOKEN: ${{ secrets.FLEET_DISPATCH_TOKEN }} + # A scheduled run ACTS. `inputs.dry-run` is empty on a schedule, and + # comparing it to `false` would make every scheduled run a dry run -- + # which is how a routine ends up documented, green, and doing nothing. + DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry-run && 'true' || 'false' }} + DO_MERGE: ${{ github.event_name == 'workflow_dispatch' && !inputs.merge && 'false' || 'true' }} + run: | + # -e off: one unreachable repo degrades to a reported skip rather than + # aborting the fleet and leaving the rest unexamined. + set +e + set -uo pipefail + + APPS=$(jq -r '.swept[]' fleet-apps.json) + [ -z "${APPS}" ] && { echo "::error::fleet-apps.json lists no apps."; exit 1; } + + { + echo "### Fleet back-merge" + echo "" + [ "${DRY_RUN}" = "true" ] && echo "**DRY RUN** — nothing opened or merged." + echo "" + echo "| App | Hop | Outcome |" + echo "| --- | --- | --- |" + } >> "$GITHUB_STEP_SUMMARY" + + # Every verdict goes to the log as well as the summary: this workflow + # opens and merges pull requests, and an audit trail that lives only in + # a rendered artifact is one no script can read. + srow() { + echo "$1" >> "$GITHUB_STEP_SUMMARY" + echo "VERDICT: $(printf '%s' "$1" | tr -d '`*' | sed 's/^| //; s/ |$//')" + } + + OPENED=0; MERGED=0; INSYNC=0; SKIPPED=0 + + # Is every check on this PR green, with at least one check present? + # Returns 0 only when that is provably true. + pr_is_green() { + local repo="$1" pr="$2" + local states total pending bad + states=$(gh pr checks "${pr}" --repo "${repo}" --json state --jq '.[].state' 2>/dev/null) + # AN EMPTY RESPONSE IS NOT "no failing checks". + [ -z "${states}" ] && return 1 + total=$(printf '%s\n' "${states}" | grep -c .) + [ "${total}" -eq 0 ] && return 1 + pending=$(printf '%s\n' "${states}" | grep -cE 'PENDING|IN_PROGRESS|QUEUED|EXPECTED') + bad=$(printf '%s\n' "${states}" | grep -cE 'FAILURE|CANCELLED|TIMED_OUT|ACTION_REQUIRED|STALE') + [ "${pending}" -ne 0 ] && return 1 + [ "${bad}" -ne 0 ] && return 1 + return 0 + } + + # One hop: carry $UP down into $DOWN for repo $R. + hop() { + local app="$1" up="$2" down="$3" + local R="ConductionNL/${app}" + local ahead existing url + + # Both refs must exist. A missing branch is reported, never treated + # as "in sync" -- those look identical from a failed comparison. + if ! gh api "repos/${R}/branches/${up}" --jq .name >/dev/null 2>&1; then + srow "| \`${app}\` | ${up}→${down} | skipped — no \`${up}\` branch |" + SKIPPED=$((SKIPPED + 1)); return + fi + if ! gh api "repos/${R}/branches/${down}" --jq .name >/dev/null 2>&1; then + srow "| \`${app}\` | ${up}→${down} | skipped — no \`${down}\` branch |" + SKIPPED=$((SKIPPED + 1)); return + fi + + # ahead_by = commits ${up} has that ${down} does not. + ahead=$(gh api "repos/${R}/compare/${down}...${up}" --jq '.ahead_by' 2>/dev/null) + + # `gh api --jq` prints `null` on a 404 and nothing on a transport + # error, and BOTH are non-empty-or-empty strings that a numeric + # test would happily coerce. Demand digits before believing it. + case "${ahead}" in + ''|*[!0-9]*) + srow "| \`${app}\` | ${up}→${down} | skipped — comparison unreadable (\`${ahead:-empty}\`), NOT assumed in sync |" + SKIPPED=$((SKIPPED + 1)); return + ;; + esac + + if [ "${ahead}" -eq 0 ]; then + INSYNC=$((INSYNC + 1)); return + fi + + existing=$(gh pr list --repo "${R}" --base "${down}" --head "${up}" --state open \ + --json number --jq '.[0].number' 2>/dev/null) + + if [ -n "${existing}" ] && [ "${existing}" != "null" ]; then + if [ "${DO_MERGE}" != "true" ]; then + srow "| \`${app}\` | ${up}→${down} | PR #${existing} open (${ahead} commits) — merging disabled |" + SKIPPED=$((SKIPPED + 1)); return + fi + if ! pr_is_green "${R}" "${existing}"; then + srow "| \`${app}\` | ${up}→${down} | PR #${existing} open (${ahead} commits) — not green, left for review |" + SKIPPED=$((SKIPPED + 1)); return + fi + if [ "${DRY_RUN}" = "true" ]; then + srow "| \`${app}\` | ${up}→${down} | would merge PR #${existing} (all checks green) |" + MERGED=$((MERGED + 1)); return + fi + # --merge, never --squash. Squashing a promotion rewrites the + # carried commits into one that the source branch does not + # contain, so the two diverge again immediately and the NEXT + # comparison reports the same work as still outstanding. + if gh pr merge "${existing}" --repo "${R}" --merge 2>/dev/null; then + srow "| \`${app}\` | ${up}→${down} | merged PR #${existing} |" + MERGED=$((MERGED + 1)) + else + srow "| \`${app}\` | ${up}→${down} | PR #${existing} green but merge REFUSED |" + SKIPPED=$((SKIPPED + 1)) + fi + return + fi + + if [ "${DRY_RUN}" = "true" ]; then + srow "| \`${app}\` | ${up}→${down} | would open PR (${ahead} commits behind) |" + OPENED=$((OPENED + 1)); return + fi + + BODY=$(printf '%s\n' \ + "Automated back-merge opened by fleet-back-merge.yml." \ + "" \ + "\`${up}\` holds ${ahead} commit(s) that \`${down}\` does not. Content reaches \`${up}\` from routes other than the promotion — Dependabot bumps, \`hotfix/*\`, and the version bump a release commits onto the branch it ran from. Left unreconciled, the next \`${down}\` → \`${up}\` promotion conflicts on exactly the file that changed." \ + "" \ + "This carries it down so the branches converge. It is a merge, not a squash: squashing would create a commit \`${up}\` does not contain and reopen the same gap.") + + url=$(gh pr create --repo "${R}" --base "${down}" --head "${up}" \ + --title "chore(sync): carry ${up} back into ${down}" \ + --body "${BODY}" 2>&1) + + if printf '%s' "${url}" | grep -q 'https://'; then + srow "| \`${app}\` | ${up}→${down} | opened $(printf '%s' "${url}" | grep -o 'https://[^ ]*') (${ahead} commits) |" + OPENED=$((OPENED + 1)) + elif printf '%s' "${url}" | grep -q 'No commits between'; then + # The comparison and the create raced; not a failure. + INSYNC=$((INSYNC + 1)) + else + srow "| \`${app}\` | ${up}→${down} | could NOT open PR — $(printf '%s' "${url}" | head -1 | cut -c1-90) |" + SKIPPED=$((SKIPPED + 1)) + fi + } + + for app in ${APPS}; do + if ! gh repo view "ConductionNL/${app}" --json name > /dev/null 2>&1; then + srow "| \`${app}\` | — | UNREADABLE — not judged |" + SKIPPED=$((SKIPPED + 1)) + continue + fi + # main → beta first, so a bump that landed on main can reach + # development in the SAME run rather than three hours later. + hop "${app}" main beta + hop "${app}" beta development + done + + { + echo "" + echo "opened=${OPENED} merged=${MERGED} already-in-sync=${INSYNC} skipped=${SKIPPED}" + } >> "$GITHUB_STEP_SUMMARY" + echo "TOTALS: opened=${OPENED} merged=${MERGED} in-sync=${INSYNC} skipped=${SKIPPED}" diff --git a/fleet-apps.json b/fleet-apps.json index 8ef9db95..7b55ccf1 100644 --- a/fleet-apps.json +++ b/fleet-apps.json @@ -32,6 +32,7 @@ "opencatalogi", "openregister", "pipelinq", + "planninq", "portaliq", "shillinq", "stackiq", From 00b2d00b2678b5f86b238c8e15825bd53cc37d6c Mon Sep 17 00:00:00 2001 From: Conduction Release Bot Date: Sun, 30 Aug 2026 10:03:34 +0200 Subject: [PATCH 2/4] revert(fleet-apps): leave the planninq decision to #644 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit My change added planninq to `swept` without removing it from `deprecated`, leaving it in BOTH lists. #644 makes the same call with the research behind it — planninq ships planninq and released while still marked not-taken-forward — and pairs it with the correction that zaakafhandelapp is the deprecated one, superseded by dossiq. This PR keeps to the dependabot allowlist and the back-merge workflow. --- fleet-apps.json | 1 - 1 file changed, 1 deletion(-) diff --git a/fleet-apps.json b/fleet-apps.json index 7b55ccf1..8ef9db95 100644 --- a/fleet-apps.json +++ b/fleet-apps.json @@ -32,7 +32,6 @@ "opencatalogi", "openregister", "pipelinq", - "planninq", "portaliq", "shillinq", "stackiq", From 2fadb8d8863397ddf92a38a0752e722964046969 Mon Sep 17 00:00:00 2001 From: Conduction Release Bot Date: Sun, 30 Aug 2026 10:13:25 +0200 Subject: [PATCH 3/4] fix(back-merge): --admin, and keep the refusal text Both `Beta Branch Protection` and `Development Branch Protection` set required_approving_review_count: 1 with OrganizationAdmin as the only bypass actor. A plain `gh pr merge` from this token is refused with "the base branch policy prohibits the merge", so this workflow would have opened 42 back-merge pull requests and then merged none of them. That is not hypothetical: fleet-friday-merge.yml shipped with the same omission and merged almost nothing for weeks while reporting success. 20 of its 23 real attempts on 2026-08-28 died exactly here, 18 of them release version bumps that were then superseded and closed unmerged. It also stops discarding the error. The old shape sent stderr to /dev/null, so a refusal named no reason and read as an ordinary conflict -- which is why the friday-merge cause went undiagnosed for so long. This does not weaken the gate: those same rulesets require ZERO status checks, so GitHub would merge a red pull request here. The green rule enforced above is strictly stronger than the policy being bypassed. Verified: YAML parses; bash -n on each run block extracted from the parsed YAML; and the hop logic re-exercised against a mocked gh, now asserting the merge is invoked with BOTH --merge and --admin, and that a refusal is reported with its reason rather than swallowed. --- .github/workflows/fleet-back-merge.yml | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/.github/workflows/fleet-back-merge.yml b/.github/workflows/fleet-back-merge.yml index ef0d6002..0f937f92 100644 --- a/.github/workflows/fleet-back-merge.yml +++ b/.github/workflows/fleet-back-merge.yml @@ -208,11 +208,29 @@ jobs: # carried commits into one that the source branch does not # contain, so the two diverge again immediately and the NEXT # comparison reports the same work as still outstanding. - if gh pr merge "${existing}" --repo "${R}" --merge 2>/dev/null; then + # + # --admin is REQUIRED, not a convenience. Both `Beta Branch + # Protection` and `Development Branch Protection` set + # required_approving_review_count: 1 with OrganizationAdmin as + # the only bypass actor, so a plain merge from this token is + # refused with "the base branch policy prohibits the merge". + # fleet-friday-merge.yml shipped without it and merged almost + # nothing for weeks while reporting success — 20 of its 23 real + # attempts on 2026-08-28 died exactly here. + # + # The green rule above is stricter than the policy bypassed: + # these rulesets require ZERO status checks, so GitHub itself + # would merge a red pull request here. + # + # The error text is KEPT rather than discarded to /dev/null. A + # refusal that names no reason reads as an ordinary conflict, + # which is precisely why the friday-merge cause went undiagnosed. + if merge_err=$(gh pr merge "${existing}" --repo "${R}" --merge --admin 2>&1); then srow "| \`${app}\` | ${up}→${down} | merged PR #${existing} |" MERGED=$((MERGED + 1)) else - srow "| \`${app}\` | ${up}→${down} | PR #${existing} green but merge REFUSED |" + why=$(printf '%s' "${merge_err}" | tr '\n|' ' ' | cut -c1-110) + srow "| \`${app}\` | ${up}→${down} | PR #${existing} green but merge REFUSED — ${why} |" SKIPPED=$((SKIPPED + 1)) fi return From 28382b65d12920e65f65fc2f9a0fd2cdbac6b170 Mon Sep 17 00:00:00 2001 From: Conduction Release Bot Date: Sun, 30 Aug 2026 10:43:34 +0200 Subject: [PATCH 4/4] fix(back-merge): a branch-to-branch PR cannot converge anything The first shape of this workflow opened `gh pr create --head main --base beta`. That cannot work, and testing it on the real fleet is what showed why: such a pull request has no way to resolve the version conflict, so it opens CONFLICTING, gets no CI at all (a conflicted PR runs none), and sits forever. Measured on portaliq -- the direct beta->development pull request opened CONFLICTING while the sync branch below merged clean and took the gap from 5 commits to 0. It now does what release.yml already does for its own `sync/*-to-development` pull requests: branch off the DOWNSTREAM ref, merge the upstream one, and resolve ONLY the version files to the downstream side -- whose unstable line is ahead of the released one, so a version never moves backwards. Anything else that conflicts is a real disagreement and stops for a human, because resolving those automatically is how a sync silently drops work. A 0-file merge is the POINT, not a no-op. A squash merge of beta into main creates a commit beta does not contain even though the trees are identical; recording the ancestry is the payload. That also decides when merging unattended is safe: 0 files changed -> nothing to test; merge. Requiring CI would block ancestry repair forever on branches already red for unrelated reasons, which is most of beta and main. any file changed -> real content moves; every check must be green. Proven end to end on portaliq and thematiq before this was written: both hops carried, both apps now report 0 commits outstanding in both directions, and each app's stalled `development -> beta` promotion went from CONFLICTING to MERGEABLE as a direct result. Verified: YAML parses; bash -n on each run block extracted from the parsed YAML. --- .github/workflows/fleet-back-merge.yml | 290 ++++++++++++------------- 1 file changed, 143 insertions(+), 147 deletions(-) diff --git a/.github/workflows/fleet-back-merge.yml b/.github/workflows/fleet-back-merge.yml index 0f937f92..2b931000 100644 --- a/.github/workflows/fleet-back-merge.yml +++ b/.github/workflows/fleet-back-merge.yml @@ -7,51 +7,55 @@ # The fleet promotes UPWARD: development → beta → main. Nothing carried # content the other way, and content does arrive out of band: # -# • Dependabot bumps. Dependabot reads its config from the repository's -# DEFAULT branch only. Ten fleet apps default to `main`, whose -# .github/dependabot.yml carries no `target-branch`, so every bump is -# opened against `main`. branch-protection.yml now admits those (see the -# `dependabot/*` arm there) — but a dependency that lands on `main` alone -# leaves `beta` and `development` on the old version. +# • Dependabot bumps, where a repository's default branch is `main`. # • `hotfix/*`, which is allowed straight into `main` or `beta` by design. -# • `release/v*` version bumps, which commit onto the branch they ran from. +# • `release/v*` version bumps, which a release commits onto the branch it +# ran from. # -# Every one of those makes the next promotion conflict on the very file that -# was changed — a lockfile or appinfo/info.xml — because both branches then -# changed it independently. release.yml already carries its own bump back -# (the `sync/beta-to-development-*` pull requests); this generalises that to -# anything that arrives, from any source. +# Every one of those makes the NEXT promotion conflict on the very file that +# changed — usually appinfo/info.xml — because both branches then bumped the +# version independently. Measured 2026-08-30: 13 of 19 open +# `development → beta` promotion pull requests were CONFLICTING for exactly +# this reason, and `main` sat 27 to 3,038 commits behind `beta`. +# +# release.yml already carries its own bump back (`sync/*-to-development`), +# but only for the app that just released. This generalises it to the whole +# fleet and to anything that arrives, from any source. # # ══════════════════════════════════════════════════════════════════════════ -# HOW IT DECIDES, AND WHY IT DOES NOT LOOK AT THE SOURCE +# WHY IT MERGES RATHER THAN OPENING A BRANCH-TO-BRANCH PULL REQUEST # ══════════════════════════════════════════════════════════════════════════ # -# The rule as stated is "a merge into main that did not come from beta should -# go to beta". Implementing that literally means identifying the pull request -# behind each commit and reading its head — which is fragile across squash -# merges, direct pushes and force-pushes, all of which occur here. -# -# The equivalent question is answerable from the refs alone, and is exact: -# -# is there anything on `main` that `beta` does not have? +# The obvious shape — `gh pr create --head main --base beta` — cannot work, +# and was tried first. Such a pull request has no way to resolve the version +# conflict, so it opens CONFLICTING, gets no CI at all (a conflicted PR runs +# none), and sits forever. Measured on portaliq: the direct beta→development +# pull request opened CONFLICTING while the sync branch below merged clean. # -# If beta → main was the only route, the answer is no and nothing happens. -# If ANYTHING else landed, the answer is yes, whatever its source. This also -# catches the case the literal rule misses: a squash merge of beta into main -# creates a commit `beta` genuinely does not contain, and leaving that -# unreconciled is what makes the following promotion conflict. +# So it does what release.yml does: branch off the DOWNSTREAM ref, merge the +# upstream one, and resolve ONLY the version files to the downstream side — +# whose unstable line is ahead of the released one, so a version never moves +# backwards. Anything else that conflicts is a real disagreement and stops +# for a human; resolving those automatically is how a sync silently drops +# work. # # ══════════════════════════════════════════════════════════════════════════ -# GREEN MEANS GREEN +# A 0-FILE MERGE IS THE POINT, NOT A NO-OP # ══════════════════════════════════════════════════════════════════════════ # -# Merging is held to the same rule as fleet-friday-merge.yml: EVERY check -# reported and NONE failed. Pending, cancelled, timed-out and an unreadable -# response are each a refusal, not a pass — an empty API response has been -# read as "no failing checks" in this fleet before, and merged a red PR. +# Most hops change no files: a squash merge of beta into main creates a +# commit `beta` does not contain even though the trees are identical. +# Recording the ancestry IS the payload — without it the merge base never +# moves and the next promotion conflicts exactly as before. # -# ZERO checks is likewise not green: a conflicted pull request runs no CI at -# all, and "no failures" is exactly what that looks like. +# That also decides when merging unattended is safe: +# +# 0 files changed -> nothing to test; merge. The tree is unchanged, so +# demanding CI would block ancestry repair forever on +# branches already red for unrelated reasons — which +# is most of beta and main today. +# any file changed -> real content moves; require every check green, the +# same rule as fleet-friday-merge.yml. name: Fleet back-merge on: @@ -63,12 +67,7 @@ on: workflow_dispatch: inputs: dry-run: - description: "Report what would be opened/merged, change nothing" - required: false - type: boolean - default: true - merge: - description: "Merge back-merge PRs that are fully green" + description: "Report what would be carried, change nothing" required: false type: boolean default: true @@ -77,14 +76,14 @@ permissions: contents: read concurrency: - # Never let two sweeps open competing pull requests for the same repo. + # Never let two sweeps open competing sync branches for the same repo. group: fleet-back-merge cancel-in-progress: false jobs: back-merge: runs-on: ubuntu-latest - timeout-minutes: 45 + timeout-minutes: 60 steps: - uses: actions/checkout@v4 @@ -104,45 +103,51 @@ jobs: - name: Carry main into beta, and beta into development env: GH_TOKEN: ${{ secrets.FLEET_DISPATCH_TOKEN }} - # A scheduled run ACTS. `inputs.dry-run` is empty on a schedule, and - # comparing it to `false` would make every scheduled run a dry run -- - # which is how a routine ends up documented, green, and doing nothing. + # A scheduled run ACTS. Written as an explicit event-name test + # rather than `inputs.dry-run == false`, because on a schedule + # `inputs.dry-run` is null and GitHub casts both null and false to + # 0 — so that comparison is TRUE and reads exactly backwards from + # what it appears to say. DRY_RUN: ${{ github.event_name == 'workflow_dispatch' && inputs.dry-run && 'true' || 'false' }} - DO_MERGE: ${{ github.event_name == 'workflow_dispatch' && !inputs.merge && 'false' || 'true' }} run: | - # -e off: one unreachable repo degrades to a reported skip rather than - # aborting the fleet and leaving the rest unexamined. + # -e off: one unreachable repo degrades to a reported skip rather + # than aborting the fleet and leaving the rest unexamined. set +e set -uo pipefail APPS=$(jq -r '.swept[]' fleet-apps.json) [ -z "${APPS}" ] && { echo "::error::fleet-apps.json lists no apps."; exit 1; } + # The ONLY files this may resolve automatically. Everything else + # that conflicts is a real disagreement. + VERFILES="appinfo/info.xml openapi.json package.json package-lock.json composer.json composer.lock" + + WORK="$(mktemp -d)" + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + { echo "### Fleet back-merge" echo "" - [ "${DRY_RUN}" = "true" ] && echo "**DRY RUN** — nothing opened or merged." + [ "${DRY_RUN}" = "true" ] && echo "**DRY RUN** — nothing pushed or merged." echo "" echo "| App | Hop | Outcome |" echo "| --- | --- | --- |" } >> "$GITHUB_STEP_SUMMARY" - # Every verdict goes to the log as well as the summary: this workflow - # opens and merges pull requests, and an audit trail that lives only in - # a rendered artifact is one no script can read. + # Every verdict goes to the log as well as the summary: this + # workflow merges, and an audit trail that lives only in a rendered + # artifact is one no script can read. srow() { echo "$1" >> "$GITHUB_STEP_SUMMARY" echo "VERDICT: $(printf '%s' "$1" | tr -d '`*' | sed 's/^| //; s/ |$//')" } - OPENED=0; MERGED=0; INSYNC=0; SKIPPED=0 + CARRIED=0; INSYNC=0; SKIPPED=0 - # Is every check on this PR green, with at least one check present? - # Returns 0 only when that is provably true. + # Is every check on this pull request green, with at least one present? pr_is_green() { - local repo="$1" pr="$2" - local states total pending bad - states=$(gh pr checks "${pr}" --repo "${repo}" --json state --jq '.[].state' 2>/dev/null) + states=$(gh pr checks "$2" --repo "$1" --json state --jq '.[].state' 2>/dev/null) # AN EMPTY RESPONSE IS NOT "no failing checks". [ -z "${states}" ] && return 1 total=$(printf '%s\n' "${states}" | grep -c .) @@ -154,130 +159,121 @@ jobs: return 0 } - # One hop: carry $UP down into $DOWN for repo $R. hop() { - local app="$1" up="$2" down="$3" - local R="ConductionNL/${app}" - local ahead existing url + app="$1"; up="$2"; down="$3" + R="ConductionNL/${app}" - # Both refs must exist. A missing branch is reported, never treated - # as "in sync" -- those look identical from a failed comparison. - if ! gh api "repos/${R}/branches/${up}" --jq .name >/dev/null 2>&1; then - srow "| \`${app}\` | ${up}→${down} | skipped — no \`${up}\` branch |" - SKIPPED=$((SKIPPED + 1)); return - fi - if ! gh api "repos/${R}/branches/${down}" --jq .name >/dev/null 2>&1; then - srow "| \`${app}\` | ${up}→${down} | skipped — no \`${down}\` branch |" - SKIPPED=$((SKIPPED + 1)); return - fi + cd "${WORK}/${app}" || { srow "| \`${app}\` | ${up}→${down} | skipped — clone missing |"; SKIPPED=$((SKIPPED+1)); return; } + git fetch -q origin "${up}" "${down}" 2>/dev/null - # ahead_by = commits ${up} has that ${down} does not. - ahead=$(gh api "repos/${R}/compare/${down}...${up}" --jq '.ahead_by' 2>/dev/null) + # A missing branch is REPORTED, never treated as "in sync" — + # those look identical from a failed comparison. + git rev-parse --verify -q "origin/${up}" >/dev/null 2>&1 || { srow "| \`${app}\` | ${up}→${down} | skipped — no \`${up}\` branch |"; SKIPPED=$((SKIPPED+1)); return; } + git rev-parse --verify -q "origin/${down}" >/dev/null 2>&1 || { srow "| \`${app}\` | ${up}→${down} | skipped — no \`${down}\` branch |"; SKIPPED=$((SKIPPED+1)); return; } - # `gh api --jq` prints `null` on a 404 and nothing on a transport - # error, and BOTH are non-empty-or-empty strings that a numeric - # test would happily coerce. Demand digits before believing it. + ahead=$(git rev-list --count "origin/${down}..origin/${up}" 2>/dev/null) + # Demand digits before believing it: a failed rev-list prints + # nothing, which a numeric test would coerce to zero and report + # as "in sync". case "${ahead}" in ''|*[!0-9]*) srow "| \`${app}\` | ${up}→${down} | skipped — comparison unreadable (\`${ahead:-empty}\`), NOT assumed in sync |" - SKIPPED=$((SKIPPED + 1)); return - ;; + SKIPPED=$((SKIPPED+1)); return ;; esac + [ "${ahead}" -eq 0 ] && { INSYNC=$((INSYNC+1)); return; } - if [ "${ahead}" -eq 0 ]; then - INSYNC=$((INSYNC + 1)); return - fi - - existing=$(gh pr list --repo "${R}" --base "${down}" --head "${up}" --state open \ - --json number --jq '.[0].number' 2>/dev/null) + br="sync/${up}-to-${down}-$(date -u +%Y%m%d%H%M%S)" + git checkout -q -B "${br}" "origin/${down}" 2>/dev/null || { srow "| \`${app}\` | ${up}→${down} | skipped — checkout failed |"; SKIPPED=$((SKIPPED+1)); return; } + git merge --no-commit --no-ff "origin/${up}" >/dev/null 2>&1 || true - if [ -n "${existing}" ] && [ "${existing}" != "null" ]; then - if [ "${DO_MERGE}" != "true" ]; then - srow "| \`${app}\` | ${up}→${down} | PR #${existing} open (${ahead} commits) — merging disabled |" - SKIPPED=$((SKIPPED + 1)); return - fi - if ! pr_is_green "${R}" "${existing}"; then - srow "| \`${app}\` | ${up}→${down} | PR #${existing} open (${ahead} commits) — not green, left for review |" - SKIPPED=$((SKIPPED + 1)); return - fi - if [ "${DRY_RUN}" = "true" ]; then - srow "| \`${app}\` | ${up}→${down} | would merge PR #${existing} (all checks green) |" - MERGED=$((MERGED + 1)); return + for F in ${VERFILES}; do + if git diff --name-only --diff-filter=U | grep -qxF "${F}"; then + git checkout --ours -- "${F}" 2>/dev/null && git add "${F}" 2>/dev/null fi - # --merge, never --squash. Squashing a promotion rewrites the - # carried commits into one that the source branch does not - # contain, so the two diverge again immediately and the NEXT - # comparison reports the same work as still outstanding. - # - # --admin is REQUIRED, not a convenience. Both `Beta Branch - # Protection` and `Development Branch Protection` set - # required_approving_review_count: 1 with OrganizationAdmin as - # the only bypass actor, so a plain merge from this token is - # refused with "the base branch policy prohibits the merge". - # fleet-friday-merge.yml shipped without it and merged almost - # nothing for weeks while reporting success — 20 of its 23 real - # attempts on 2026-08-28 died exactly here. - # - # The green rule above is stricter than the policy bypassed: - # these rulesets require ZERO status checks, so GitHub itself - # would merge a red pull request here. - # - # The error text is KEPT rather than discarded to /dev/null. A - # refusal that names no reason reads as an ordinary conflict, - # which is precisely why the friday-merge cause went undiagnosed. - if merge_err=$(gh pr merge "${existing}" --repo "${R}" --merge --admin 2>&1); then - srow "| \`${app}\` | ${up}→${down} | merged PR #${existing} |" - MERGED=$((MERGED + 1)) - else - why=$(printf '%s' "${merge_err}" | tr '\n|' ' ' | cut -c1-110) - srow "| \`${app}\` | ${up}→${down} | PR #${existing} green but merge REFUSED — ${why} |" - SKIPPED=$((SKIPPED + 1)) - fi - return + done + + left="$(git diff --name-only --diff-filter=U)" + if [ -n "${left}" ]; then + git merge --abort 2>/dev/null || true + srow "| \`${app}\` | ${up}→${down} | STOPPED for a human — conflicts outside the version files: $(echo "${left}" | tr '\n' ' ') |" + SKIPPED=$((SKIPPED+1)); return fi + git commit -q --no-verify -m "chore(sync): carry ${up} back into ${down}" 2>/dev/null + [ -z "$(git log --oneline "origin/${down}..HEAD" 2>/dev/null)" ] && { INSYNC=$((INSYNC+1)); return; } + + nf=$(git diff --name-only "origin/${down}..HEAD" 2>/dev/null | wc -l) + if [ "${DRY_RUN}" = "true" ]; then - srow "| \`${app}\` | ${up}→${down} | would open PR (${ahead} commits behind) |" - OPENED=$((OPENED + 1)); return + srow "| \`${app}\` | ${up}→${down} | would carry ${ahead} commit(s), ${nf} file(s) |" + CARRIED=$((CARRIED+1)); return fi - BODY=$(printf '%s\n' \ - "Automated back-merge opened by fleet-back-merge.yml." \ + git push -q origin "${br}" 2>/dev/null || { srow "| \`${app}\` | ${up}→${down} | push failed |"; SKIPPED=$((SKIPPED+1)); return; } + + body=$(printf '%s\n' \ + "\`${up}\` held ${ahead} commit(s) \`${down}\` did not — release version bumps committed onto the branch the release ran from, and anything else that arrived out of band. Left unreconciled, both branches change the version file independently and the next \`${down}\` → \`${up}\` promotion conflicts on it." \ + "" \ + "**${nf} file(s) change.** A 0-file result is the normal and correct outcome once the content has already reached \`${down}\` by other means — recording the ancestry *is* the payload. Without it the merge base never moves and the next promotion conflicts exactly as before." \ "" \ - "\`${up}\` holds ${ahead} commit(s) that \`${down}\` does not. Content reaches \`${up}\` from routes other than the promotion — Dependabot bumps, \`hotfix/*\`, and the version bump a release commits onto the branch it ran from. Left unreconciled, the next \`${down}\` → \`${up}\` promotion conflicts on exactly the file that changed." \ + "Version files were resolved to \`${down}\`'s side, whose line is ahead of the released one, so this never moves a version backwards. Any conflict outside them aborts the merge for a human instead." \ "" \ - "This carries it down so the branches converge. It is a merge, not a squash: squashing would create a commit \`${up}\` does not contain and reopen the same gap.") + "Opened by fleet-back-merge.yml, using the algorithm \`release.yml\` already applies to its own \`sync/*-to-development\` pull requests.") - url=$(gh pr create --repo "${R}" --base "${down}" --head "${up}" \ - --title "chore(sync): carry ${up} back into ${down}" \ - --body "${BODY}" 2>&1) + url=$(gh pr create --repo "${R}" --head "${br}" --base "${down}" \ + --title "chore(sync): carry ${up} back into ${down}" --body "${body}" 2>&1 | grep -o 'https://[^ ]*') + [ -z "${url}" ] && { srow "| \`${app}\` | ${up}→${down} | branch pushed but PR create failed |"; SKIPPED=$((SKIPPED+1)); return; } + pr="${url##*/}" - if printf '%s' "${url}" | grep -q 'https://'; then - srow "| \`${app}\` | ${up}→${down} | opened $(printf '%s' "${url}" | grep -o 'https://[^ ]*') (${ahead} commits) |" - OPENED=$((OPENED + 1)) - elif printf '%s' "${url}" | grep -q 'No commits between'; then - # The comparison and the create raced; not a failure. - INSYNC=$((INSYNC + 1)) + # 0 files changed: nothing to test, and the tree is byte-identical + # to ${down}. Requiring CI here would block ancestry repair + # forever on branches that are red for unrelated reasons. + if [ "${nf}" -ne 0 ] && ! pr_is_green "${R}" "${pr}"; then + srow "| \`${app}\` | ${up}→${down} | opened #${pr} (${ahead} commits, ${nf} files) — carries content, left for review |" + SKIPPED=$((SKIPPED+1)); return + fi + + # --admin is REQUIRED. `Beta Branch Protection` and `Development + # Branch Protection` both set required_approving_review_count: 1 + # with OrganizationAdmin as the only bypass, so a plain merge is + # refused with "the base branch policy prohibits the merge". + # fleet-friday-merge.yml shipped without it and merged almost + # nothing for weeks while reporting success. + # + # --merge, never --squash: squashing would create a commit the + # source branch does not contain and reopen the gap just closed. + # + # The error text is KEPT. A refusal that names no reason reads as + # an ordinary conflict, which is why that cause went undiagnosed. + if merge_err=$(gh pr merge "${pr}" --repo "${R}" --merge --admin 2>&1); then + srow "| \`${app}\` | ${up}→${down} | carried #${pr} (${ahead} commits, ${nf} files) |" + CARRIED=$((CARRIED+1)) else - srow "| \`${app}\` | ${up}→${down} | could NOT open PR — $(printf '%s' "${url}" | head -1 | cut -c1-90) |" - SKIPPED=$((SKIPPED + 1)) + srow "| \`${app}\` | ${up}→${down} | #${pr} ready but merge REFUSED — $(printf '%s' "${merge_err}" | tr '\n|' ' ' | cut -c1-100) |" + SKIPPED=$((SKIPPED+1)) fi } for app in ${APPS}; do if ! gh repo view "ConductionNL/${app}" --json name > /dev/null 2>&1; then srow "| \`${app}\` | — | UNREADABLE — not judged |" - SKIPPED=$((SKIPPED + 1)) - continue + SKIPPED=$((SKIPPED+1)); continue + fi + # --filter=blob:none keeps full history (a merge needs it) without + # file contents, so cloning the whole fleet stays cheap. + if ! git clone -q --filter=blob:none "https://x-access-token:${GH_TOKEN}@github.com/ConductionNL/${app}.git" "${WORK}/${app}" 2>/dev/null; then + srow "| \`${app}\` | — | clone failed — not judged |" + SKIPPED=$((SKIPPED+1)); continue fi # main → beta first, so a bump that landed on main can reach # development in the SAME run rather than three hours later. hop "${app}" main beta hop "${app}" beta development + cd "${WORK}" && rm -rf "${WORK:?}/${app}" done { echo "" - echo "opened=${OPENED} merged=${MERGED} already-in-sync=${INSYNC} skipped=${SKIPPED}" + echo "carried=${CARRIED} already-in-sync=${INSYNC} skipped=${SKIPPED}" } >> "$GITHUB_STEP_SUMMARY" - echo "TOTALS: opened=${OPENED} merged=${MERGED} in-sync=${INSYNC} skipped=${SKIPPED}" + echo "TOTALS: carried=${CARRIED} in-sync=${INSYNC} skipped=${SKIPPED}"