diff --git a/.github/actions/start-tsp/action.yaml b/.github/actions/start-tsp/action.yaml index f9d1a6e..fa6e8ed 100644 --- a/.github/actions/start-tsp/action.yaml +++ b/.github/actions/start-tsp/action.yaml @@ -7,7 +7,7 @@ # # Example usage (be sure to decrypt the env file first): # - name: Start TSP -# uses: IronCoreLabs/workflows/.github/actions/start-tsp@start-tsp-v1 +# uses: IronCoreLabs/workflows/.github/actions/start-tsp@start-tsp-v1.0.0 # with: # gcloud-auth: ${{ secrets.GCLOUD_AUTH }} # env-file-path: .github/.env.integration diff --git a/.github/move-tags.list.sh b/.github/move-tags.list.sh index ee2f45f..a2f29e4 100755 --- a/.github/move-tags.list.sh +++ b/.github/move-tags.list.sh @@ -5,10 +5,10 @@ # content. See RELEASING.md and move-tags.yaml for details. # # Each reusable workflow and composite action declares its major version in a -# "# tag-version: vN" comment in its yaml file. For each declaration, the tag "NAME-vN" -# is printed on stdout if it doesn't exist, or if the workflow's files (the yaml file -# plus any ".github/NAME.*.sh" helper scripts, or the action's directory) differ -# between the tag and HEAD. +# "# tag-version: vN" comment in its yaml file. For each declaration, the tag +# "NAME-vN.0.0" is printed on stdout if it doesn't exist, or if the workflow's files +# (the yaml file plus any ".github/NAME.*.sh" helper scripts, or the action's +# directory) differ between the tag and HEAD. # # A reusable workflow or action without a usable declaration is a misconfiguration: # it's reported on stderr and the script exits nonzero, after checking everything. @@ -46,12 +46,22 @@ check() { return 0 fi + # The parse above stops at the first dot, so a dotted declaration would be silently + # truncated to its major, discarding whatever the author meant by the rest. Since + # the tags are dotted, that's an easy comment to write by mistake. + if grep -qE '^# tag-version: v[0-9]+\.' "$vfile" ; then + echo "$name: $vfile declares a dotted version; the comment carries the major only, as 'v$version'" 1>&2 + ERRORS=1 + return 0 + fi + # Never touch a tag older than the newest existing one. That means the comment is # stale or was decremented, and moving the old tag would break its consumers. local newest="" local t n while IFS= read -r t ; do n="${t#"$name"-v}" + n="${n%%.*}" [[ "$n" =~ ^[0-9]+$ ]] || continue if [ -z "$newest" ] || [ "$n" -gt "$newest" ] ; then newest="$n" @@ -63,7 +73,10 @@ check() { return 0 fi - local tag="$name-v$version" + # Dotted because Dependabot only recognizes a bare "vN" ref when the "v" starts it: + # a prefixed "$name-v$version" is invisible to it, so consumers pinned to that name + # never get upgrade PRs. The tag still floats; the digits are for the parser only. + local tag="$name-v$version.0.0" if ! git rev-parse -q --verify "refs/tags/$tag" > /dev/null ; then TAGS+=("$tag") elif ! git diff --quiet "refs/tags/$tag" HEAD -- "$@" ; then diff --git a/.github/spec/move_tags_spec.sh b/.github/spec/move_tags_spec.sh index 0bc3b31..bc3c5cb 100644 --- a/.github/spec/move_tags_spec.sh +++ b/.github/spec/move_tags_spec.sh @@ -38,7 +38,7 @@ Describe 'move-tags.list.sh' } It 'creates the tag for a new workflow' When call new_workflow - The output should equal 'foo-v1' + The output should equal 'foo-v1.0.0' The status should be success End @@ -46,7 +46,7 @@ Describe 'move-tags.list.sh' setup_repo reusable_workflow v1 > .github/workflows/foo.yaml commit_all - git tag foo-v1 + git tag foo-v1.0.0 "$SCRIPT" } It 'is quiet when the tag matches the content' @@ -55,18 +55,31 @@ Describe 'move-tags.list.sh' The status should be success End - changed_workflow() { + legacy_tag_only() { setup_repo reusable_workflow v1 > .github/workflows/foo.yaml commit_all git tag foo-v1 + "$SCRIPT" + } + It 'creates the dotted tag for a version that only has the legacy name' + When call legacy_tag_only + The output should equal 'foo-v1.0.0' + The status should be success + End + + changed_workflow() { + setup_repo + reusable_workflow v1 > .github/workflows/foo.yaml + commit_all + git tag foo-v1.0.0 printf '# a change\n' >> .github/workflows/foo.yaml commit_all "$SCRIPT" } It 'moves the tag when the workflow file changed' When call changed_workflow - The output should equal 'foo-v1' + The output should equal 'foo-v1.0.0' The status should be success End @@ -74,7 +87,7 @@ Describe 'move-tags.list.sh' setup_repo reusable_workflow v1 > .github/workflows/foo.yaml commit_all - git tag foo-v1 + git tag foo-v1.0.0 printf 'docs\n' > README.md commit_all "$SCRIPT" @@ -89,14 +102,14 @@ Describe 'move-tags.list.sh' setup_repo reusable_workflow v1 > .github/workflows/foo.yaml commit_all - git tag foo-v1 + git tag foo-v1.0.0 printf 'echo hi\n' > .github/foo.build.sh commit_all "$SCRIPT" } It 'moves the tag when a helper script changed' When call changed_helper_script - The output should equal 'foo-v1' + The output should equal 'foo-v1.0.0' The status should be success End @@ -104,14 +117,14 @@ Describe 'move-tags.list.sh' setup_repo reusable_workflow v1 > .github/workflows/foo.yaml commit_all - git tag foo-v1 + git tag foo-v1.0.0 reusable_workflow v2 > .github/workflows/foo.yaml commit_all "$SCRIPT" } It 'creates the new tag on a major bump, leaving the old one alone' When call major_bump - The output should equal 'foo-v2' + The output should equal 'foo-v2.0.0' The status should be success End @@ -119,7 +132,7 @@ Describe 'move-tags.list.sh' setup_repo reusable_workflow v2 > .github/workflows/foo.yaml commit_all - git tag foo-v2 + git tag foo-v2.0.0 reusable_workflow v1 > .github/workflows/foo.yaml commit_all "$SCRIPT" @@ -131,6 +144,35 @@ Describe 'move-tags.list.sh' The status should be failure End + decremented_past_legacy_tag() { + setup_repo + reusable_workflow v2 > .github/workflows/foo.yaml + commit_all + git tag foo-v2 + reusable_workflow v1 > .github/workflows/foo.yaml + commit_all + "$SCRIPT" + } + It 'counts a legacy undotted tag when checking for a decrement' + When call decremented_past_legacy_tag + The output should equal '' + The stderr should not equal '' + The status should be failure + End + + dotted_declaration() { + setup_repo + reusable_workflow v1.0.1 > .github/workflows/foo.yaml + commit_all + "$SCRIPT" + } + It 'fails for a tag-version comment carrying more than the major' + When call dotted_declaration + The output should equal '' + The stderr should not equal '' + The status should be failure + End + reusable_without_version() { setup_repo printf 'on:\n workflow_call:\n' > .github/workflows/foo.yaml @@ -153,7 +195,7 @@ Describe 'move-tags.list.sh' } It 'still lists good tags while failing on a misconfigured workflow' When call good_and_bad_workflows - The output should equal 'foo-v1' + The output should equal 'foo-v1.0.0' The stderr should include 'bad' The status should be failure End @@ -176,14 +218,14 @@ Describe 'move-tags.list.sh' mkdir -p .github/actions/tsp printf '# tag-version: v1\nname: tsp\n' > .github/actions/tsp/action.yaml commit_all - git tag tsp-v1 + git tag tsp-v1.0.0 printf 'echo hi\n' > .github/actions/tsp/helper.sh commit_all "$SCRIPT" } It 'moves the tag when any file in a composite action changed' When call changed_action - The output should equal 'tsp-v1' + The output should equal 'tsp-v1.0.0' The status should be success End @@ -196,8 +238,8 @@ Describe 'move-tags.list.sh' } It 'prints multiple tags sorted' When call multiple_workflows - The line 1 of output should equal 'bar-v2' - The line 2 of output should equal 'foo-v1' + The line 1 of output should equal 'bar-v2.0.0' + The line 2 of output should equal 'foo-v1.0.0' The status should be success End End diff --git a/.github/workflows/rebuild.yaml b/.github/workflows/rebuild.yaml index 88fe1e1..694eabd 100644 --- a/.github/workflows/rebuild.yaml +++ b/.github/workflows/rebuild.yaml @@ -5,10 +5,10 @@ # This should _not_ be used for hotfix branches, as we don't bump versions in those cases. # # Usage: -# uses: IronCoreLabs/workflows/.github/workflows/rebuild.yaml@rebuild-v1 +# uses: IronCoreLabs/workflows/.github/workflows/rebuild.yaml@rebuild-v0.0.0 # # With multiple refs: -# uses: IronCoreLabs/workflows/.github/workflows/rebuild.yaml@rebuild-v1 +# uses: IronCoreLabs/workflows/.github/workflows/rebuild.yaml@rebuild-v0.0.0 # with: # refs: '["main", "release/v1"]' diff --git a/.github/workflows/repin-consumers.yaml b/.github/workflows/repin-consumers.yaml new file mode 100644 index 0000000..841649a --- /dev/null +++ b/.github/workflows/repin-consumers.yaml @@ -0,0 +1,160 @@ +# One-shot sweep to move consuming repos off the frozen undotted tags (rust-ci-v3) and +# onto the dotted ones Dependabot can parse (rust-ci-v3.0.0). See RELEASING.md. +# +# With no inputs it scans every non-archived, non-fork repo in the org and repins the +# ones that reference this repo. Run it with dry_run first: it prints the diff it would +# open for each repo, without pushing anything. + +name: Repin Consumers + +on: + workflow_dispatch: + inputs: + repos: + description: Space-separated owner/repo list. Empty scans the whole org. + required: false + type: string + dry_run: + description: Print the diffs instead of opening PRs + type: boolean + default: true + +permissions: + contents: read + +jobs: + repin: + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 + - name: Repin consumers + env: + # Editing another repo's .github/workflows needs a token with the workflow + # scope; the job's own token can't do it. + GH_TOKEN: ${{ secrets.WORKFLOW_PAT }} + REPOS: ${{ inputs.repos }} + DRY_RUN: ${{ inputs.dry_run }} + ORG: ${{ github.repository_owner }} + SOURCE_REPO: ${{ github.repository }} + run: | + set -euo pipefail + + # Only tags that exist become rewrite targets, so no consumer can be left + # pointing at a tag that was never published. + mapfile -t NEW_TAGS < <(git tag --list '*-v*.0.0' | sort) + if [ "${#NEW_TAGS[@]}" -eq 0 ] ; then + echo "::error::No dotted tags exist yet; let Move Workflow Tags run on main first." + exit 1 + fi + + gh auth setup-git + git config --global user.email ops@ironcorelabs.com + git config --global user.name "Leeroy Travis" + + BRANCH=repin-workflow-tags + WORK=$(mktemp -d) + TITLE="Pin IronCore workflow tags to their dotted names" + cat > "$WORK/body.md" <<'EOF' + `IronCoreLabs/workflows` now publishes its floating tags under dotted names: + `rust-ci-v3.0.0` instead of `rust-ci-v3`. Same commit, same auto-updating + behavior — but Dependabot can parse the dotted form, so this repo will get a PR + when a workflow's next major ships. + + The undotted tags are frozen and no longer move, so a repo left on them stops + receiving workflow updates entirely. + + See https://github.com/IronCoreLabs/workflows/blob/main/RELEASING.md + EOF + + if [ -n "$REPOS" ] ; then + # Unquoted on purpose: the input is a space-separated list. + # shellcheck disable=SC2086 + set -- $REPOS + else + # Enumerating beats code search: search results are capped and index-lagged, + # and a repo missed here is one that silently stops receiving updates. + mapfile -t FOUND < <(gh repo list "$ORG" --source --no-archived --limit 1000 \ + --json nameWithOwner --jq '.[].nameWithOwner') + if [ "${#FOUND[@]}" -eq 0 ] ; then + echo "::error::No repos found in $ORG." + exit 1 + fi + set -- "${FOUND[@]}" + echo "Scanning $# repos in $ORG." + fi + + SCANNED=$# + IRRELEVANT=0 + for REPO in "$@" ; do + DIR="$WORK/${REPO//\//-}" + # blob:none keeps an 80-repo scan cheap. A shallow clone would be cheaper + # still, but pushing from one is not reliably accepted. + if ! git clone --quiet --filter=blob:none --no-tags "https://github.com/$REPO" "$DIR" ; then + echo "::warning::$REPO: clone failed, skipping" + continue + fi + pushd "$DIR" > /dev/null + + mapfile -t FILES < <(git ls-files -- .github | grep -E '\.ya?ml$' || true) + if [ "${#FILES[@]}" -eq 0 ] || ! grep -q "$SOURCE_REPO/" "${FILES[@]}" ; then + IRRELEVANT=$((IRRELEVANT + 1)) + popd > /dev/null + continue + fi + + for NEW in "${NEW_TAGS[@]}" ; do + OLD="${NEW%.0.0}" + # Two expressions because the ref is usually the whole end of the line, + # and the quoted or commented form needs a trailing character to match on. + # Neither can match an already-dotted ref, so re-running is safe. + sed -E -i \ + -e "s|@${OLD}[[:space:]]*\$|@${NEW}|" \ + -e "s|@${OLD}([[:space:]\"'])|@${NEW}\1|g" \ + "${FILES[@]}" + done + + # Anything still on an undotted tag has no published dotted equivalent — + # a legacy major, say — and needs a person to decide where it should land. + LEFTOVER=$(grep -nE "$SOURCE_REPO/[^@[:space:]]+@[A-Za-z0-9-]+-v[0-9]+([[:space:]\"']|\$)" "${FILES[@]}" || true) + if [ -n "$LEFTOVER" ] ; then + echo "::warning::$REPO still references undotted tags:" + echo "$LEFTOVER" + fi + + if git diff --quiet ; then + echo "- $REPO: already repinned" >> "$GITHUB_STEP_SUMMARY" + elif [ "$DRY_RUN" = "true" ] ; then + { + echo "
$REPO" + echo + echo '```diff' + git --no-pager diff + echo '```' + echo '
' + } >> "$GITHUB_STEP_SUMMARY" + else + git checkout -q -b "$BRANCH" + git commit -qam "$TITLE" + git push -q -u origin "$BRANCH" + # Keep going if one repo refuses the PR; a half-finished sweep is worse + # than a reported failure. + if URL=$(gh pr create --title "$TITLE" --body-file "$WORK/body.md" 2>&1) ; then + echo "- $REPO: $URL" >> "$GITHUB_STEP_SUMMARY" + else + echo "::warning::$REPO: could not open a PR: $URL" + echo "- $REPO: PR failed, branch pushed" >> "$GITHUB_STEP_SUMMARY" + fi + fi + + popd > /dev/null + done + + { + echo + echo "Scanned $SCANNED repos; $IRRELEVANT don't reference $SOURCE_REPO." + if [ "$DRY_RUN" = "true" ] ; then + echo "Dry run — no branches pushed and no PRs opened." + fi + } >> "$GITHUB_STEP_SUMMARY" diff --git a/README.md b/README.md index 8017312..e12273c 100644 --- a/README.md +++ b/README.md @@ -98,7 +98,7 @@ on: jobs: rust-release: - uses: IronCoreLabs/workflows/.github/workflows/rust-release.yaml@rust-release-v2 + uses: IronCoreLabs/workflows/.github/workflows/rust-release.yaml@rust-release-v2.0.0 with: # inputs is empty on release events, and '' fails dry_run's boolean type check. dry_run: ${{ inputs.dry_run || false }} diff --git a/RELEASING.md b/RELEASING.md index 57eee4c..3fde671 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -1,6 +1,16 @@ # Releasing -Each reusable workflow and composite action has a floating major version tag (e.g. `rust-ci-v1`, `docker-v2`). Consuming repos pin to these tags, so moving a tag releases the change to every consumer of that major version. +Each reusable workflow and composite action has a floating major version tag (e.g. `rust-ci-v1.0.0`, `docker-v2.0.0`). Consuming repos pin to these tags, so moving a tag releases the change to every consumer of that major version. + +## Why the tags are dotted + +The patch digits are there for Dependabot, which only recognizes a bare `vN` ref when the `v` starts it — `actions/checkout@v7` qualifies, `rust-ci-v1` can't. A consumer pinned to the undotted name never gets an upgrade PR; pinned to `rust-ci-v1.0.0`, it gets one as soon as `rust-ci-v2.0.0` exists. + +The tag is not immutable, whatever `1.0.0` suggests. It is force-moved on every content change, and no patch or minor versions are ever cut — the digits satisfy a parser, they don't promise a frozen release. + +The undotted tags (`rust-ci-v1`, `docker-v2`, …) are frozen at the last commit before this change and are no longer moved. A repo still pinned to one keeps working, but stops receiving updates. + +Superseded majors have dotted twins too — `rust-ci-v2.0.0` mirrors the frozen `rust-ci-v2`. They exist so a consumer that hasn't upgraded yet is still legible to Dependabot, which then offers it the current major. Like the tags they mirror, they never move. ## Declaring versions @@ -12,10 +22,10 @@ Every reusable workflow declares its current major version in a comment at the t ## What happens on merge -The `move-tags.yaml` workflow runs on every push to `main` and reconciles tags with content: for each declared version, if the tag `NAME-vN` doesn't exist, or if the workflow's files (its yaml file, its `.github/NAME.*.sh` helper scripts, or the action's directory) differ between the tag and `main`, the tag is created or force-moved to the head of `main`. +The `move-tags.yaml` workflow runs on every push to `main` and reconciles tags with content: for each declared version, if the tag `NAME-vN.0.0` doesn't exist, or if the workflow's files (its yaml file, its `.github/NAME.*.sh` helper scripts, or the action's directory) differ between the tag and `main`, the tag is created or force-moved to the head of `main`. - **Non-breaking change:** leave the `tag-version` comment alone. The current tag moves forward when your PR merges. -- **Breaking change:** increment the `tag-version` comment in the same PR. Merging creates the new tag; the old tag stays where it was, and consumers upgrade by editing their workflow references. +- **Breaking change:** increment the `tag-version` comment in the same PR. Merging creates the new tag; the old tag stays where it was, and consumers upgrade by merging the Dependabot PR it produces. On pull requests the same workflow runs in report-only mode: its job summary lists the tags that merging will move, so reviewers can check that a breaking change got a new major version.