Skip to content

ci: optional apt repository per build (aptly repo.sh, matching apt.new) - #35

Open
igorpecovnik wants to merge 2 commits into
mainfrom
ci-reprepro-repo-option
Open

ci: optional apt repository per build (aptly repo.sh, matching apt.new)#35
igorpecovnik wants to merge 2 commits into
mainfrom
ci-reprepro-repo-option

Conversation

@igorpecovnik

@igorpecovnik igorpecovnik commented Jul 27, 2026

Copy link
Copy Markdown
Member

Adds an opt-in path that builds a complete, signed apt repository with the same aptly tooling and keys production usestools/repository/repo.sh, the version armbian.github.io runs to build https://repo.armbian.com — so the layout matches apt.new (dists by release codename, pool by component). Groundwork for per-pull-request repositories.

Supersedes the earlier reprepro approach in this PR — switched to aptly so per-PR repos are byte-structurally identical to the production repo.

What it does

Folded into publish-debs-to-repo, reusing its single debs-to-repo-download. After the debs are downloaded (and before the rsync that moves them away):

  1. checks out the build repo's tools at repo_tool_ref (the production repo.sh);
  2. runs tools/repository/repo.sh -c update -k -i <output/debs[-beta]> -o output/repository → publishes dists/<release> + pool/<component> under output/repository/public, signing the top-level Release files with the keys imported above (GPG_KEY3 = Armbian repo, GPG_KEY4 = Igor);
  3. rsyncs the contents of public/ to storage/repositories/<path> → served at https://repo.armbian.com/repositories/<path>.

Both steps are continue-on-error, so the optional repo can never block the raw-deb publish.

Depends on armbian/build#10304

That PR brings the production repo.sh (+ 6 review fixes) onto main. Until it lands, repo_tool_ref temporarily points at its branch singlemode-to-main so this is testable now. Once #10304 merges, set repo_tool_ref to main (then it just uses the normal build checkout).

Toggle

  • build-all.yml: manual generateRepository input (default no).
  • Reusable inputs: generaterepository_default (default no), repository_target_path (→ target_path), repo_tool_ref (→ singlemode-to-main).

Notes

  • Only complete-artifact-matrix.yml + build-all.yml change; actionlint-clean on the touched steps.
  • armbian/build#10301 / #10303 remain valid fixes for the framework's debs-to-repo-reprepro, but are no longer on this path.

@igorpecovnik

Copy link
Copy Markdown
Member Author

First live test (run 30383424520, generateReprepro=yes on this branch) surfaced several issues — now addressed:

  1. reprepro built nothing — the generated reprepro.sh aborted on the first missing deb (Error 2 opening ...debset -e before export). Partial builds always miss some debs, so this is fatal in practice. → Fixed properly in repo-reprepro: skip debs missing on disk instead of aborting the whole repo build#10301 (skip debs not present on disk). This PR depends on it.
  2. Signing failedGPG_KEY3/GPG_KEY4 were not in armbian/ci secrets (only in armbian.github.io). @igorpecovnik has since added them to armbian/ci. Also hardened here: the two keyids are trimmed so a missing import yields "" (unsigned) instead of a stray space.
  3. Upload would failstorage/repositories/<path> did not exist and rsync will not create missing parents. Now the step pre-creates the tree over ssh and skips (with a warning) when the build produced no dists/.
  4. Many unauthorized ORAS fetches — that was a fresh-trunk partial state (artifacts not yet published), not a bug; ci: create + read releases in this repo (self-contained test CI) #1 makes the repo tolerant of it.

Re-test once armbian/build#10301 is merged (the reprepro job checks out build main). Still open: confirm the served storage/repositories/ path.

@igorpecovnik

Copy link
Copy Markdown
Member Author

Second test run (30400943657) — the important parts now work: GPG import succeeds (both keys imported, SignWith: 1EB30FAF236099FE 93D6889F9F0E78D5), and #10301 prevents the abort (skips missing debs instead of dying).

But it produced a ~214 MB log / 7-min step. Root cause: the reprepro build ran under bash -x, which reprints the (264 KB) for one_deb in <2165 debs> header on every iteration — ~half a GB of trace for one target. Compounded by #10301 echoing one line per missing deb.

Fixes:

Note: the log bloat obscured the tail, so I could not confirm export/signing/upload completed in that run — a clean re-test after both land will show it. Still open: the served storage/repositories/ path.

…oduction)

Adds an opt-in path that builds a complete, signed apt repository with the same
aptly tooling and keys production uses (tools/repository/repo.sh, the version
armbian.github.io runs to build repo.armbian.com), so the layout matches
apt.new (dists by release codename, pool by component) -- groundwork for
per-pull-request repositories.

Folded into the existing publish-debs-to-repo job, reusing its single
debs-to-repo-download: after the debs are downloaded (and before the rsync that
moves them away with --remove-source-files), an opt-in step checks out the build
repo's tools at repo_tool_ref and runs:

    tools/repository/repo.sh -c update -k -i <output/debs[-beta]> -o output/repository

which publishes dists/<release> + pool/<component> under output/repository/public
and signs the top-level Release files with the keys imported above (GPG_KEY3 =
Armbian repo key, GPG_KEY4 = Igor). A companion step rsyncs the contents of
public/ to storage/repositories/<path> (served at
https://repo.armbian.com/repositories/<path>). Both steps are continue-on-error
so the optional repo can never block the raw-deb publish.

repo_tool_ref TEMPORARILY points at the armbian/build PR branch
(singlemode-to-main) that carries the production repo.sh; set it to "main" once
armbian/build#10304 lands.

Wiring:
- complete-artifact-matrix.yml: inputs renamed to generaterepository_default /
  repository_target_path, new repo_tool_ref input; reprepro steps replaced with
  the repo-tool checkout + repo.sh build + public/ upload.
- build-all.yml: manual input renamed generateReprepro -> generateRepository.

Replaces the earlier reprepro approach (debs-to-repo-reprepro); armbian/build
#10301 / #10303 remain valid fixes for that framework command but are no longer
on this path.

Signed-off-by: Igor Pecovnik <igor@armbian.com>
@igorpecovnik
igorpecovnik force-pushed the ci-reprepro-repo-option branch from 67b0e52 to 840ff87 Compare July 29, 2026 11:37
@igorpecovnik igorpecovnik changed the title ci: optional reprepro apt repository (build & upload) ci: optional apt repository per build (aptly repo.sh, matching apt.new) Jul 29, 2026
The "Build signed apt repository (aptly repo.sh)" step produced an empty
repository: every aptly publish reported "publishing from empty source"
and zero packages, so pool/ shipped empty.

Root cause is a BETA mismatch between steps. The version, build and
nightlybuild-passthrough steps resolve nightly/stable as
`github.event.inputs.nightlybuild || inputs.nightlybuild_default`, so a
run driven by the parent workflow's nightlybuild_default=yes (dispatch
input unset) builds nightly -- trunk REVISION (26.8.0-trunk.5), debs
pushed to the beta namespace. But the debs-to-repo download and repo.sh
steps keyed off `github.event.inputs.nightlybuild == 'true'` alone, which
ignores nightlybuild_default and evaluates to BETA=no. The download then
looked in the stable namespace, logged "Missing deb" for every package,
left output/debs empty, and repo.sh had nothing to import.

Resolve BETA in both steps the same way the build does -- nightly unless
explicitly 'no' (mirrors the Determine version step's `[ "$NIGHTLY" = "no" ]`)
-- normalized to yes/no. Download and repo.sh now target the same
namespace/dir the build wrote to, so the signed repo is populated.

Signed-off-by: Igor Pecovnik <igor@armbian.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Work in progress Unfinished / work in progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant