Add pushes API for exporting images to remote registries by chruffins · Pull Request #353 · kernel/hypeman

chruffins · 2026-08-05T19:11:09Z

Layer 3 of remote registry push support (stacked on #348, #350).

What

The HTTP surface for outbound pushes, docker-aligned credentials included.

API (openapi.yaml → generated)

POST /pushes → 202 + Push job. Body: image (hypeman image, must be ready), target (full remote ref), optional insecure, optional credentials
GET /pushes (newest first), GET /pushes/{id}
Push object: queued → pushing → pushed/failed with queue position, error, layers/bytes, timestamps
errors: 400 invalid_name/invalid_target, 404 image not found, 409 image_not_ready, 500 otherwise

Credentials — borrow, don't store

request credentials (username/password/registry_token, mirroring docker config.json fields) map to an authn.AuthConfig and are borrowed for that single push only — never persisted (asserted in layer 2) or logged
omitted → falls back to the server's default resolution (Docker keychain)
scopes reuse image:write (POST) / image:read (GET) — no new scope plumbing, existing tokens work

Wiring

ProvidePushManager (resolver = image manager, concurrency from limits.max_concurrent_pushes, default 2)
ApiService.PushManager + wire regen
GC roots composition: compositeOCICacheRoots in main.go feeds the OCI cache GC both the registry's BuildKit cache tags and the push manager's in-flight digests (imagepush.Manager gains LiveCacheManifestDigests to satisfy ocicachegc.RootsProvider)

Tests

Handler-level with a fake push manager (hermetic, follows the images_test pattern): request + credentials mapping, nil-credentials fallback, full error-status mapping table, get-not-found, list empty/all with layers/bytes mapping.

Notes

lib/oapi/oapi.go regenerated via make oapi-generate (pinned oapi-codegen v2.5.1); the embedded-spec blob diff includes the pre-existing compression drift any regen under the current Go toolchain produces on main too
wire_gen.go regenerated with wire v0.6.0
SSE/events for pushes not included — clients poll GET /pushes/{id} (builds-style events can be a follow-up)
the 409 for not-ready images is deliberate: the image exists but can't be pushed yet

Verification

go build -tags containers_image_openpgp ./... — full tree builds
new handler tests + lib/scopes, lib/providers, cmd/api/config, and all push/image packages pass

Note

Medium Risk
Touches registry auth (request-borrowed credentials), async export to external registries, and OCI cache GC roots; behavior is heavily tested but misconfiguration or GC interaction could affect blob retention during pushes.

Overview
Adds outbound image push as a first-class API: clients can start async jobs that export ready hypeman images from the local OCI cache to a remote registry reference, then poll status via GET /pushes and GET /pushes/{id}.

HTTP surface (POST /pushes → 202): accepts image, target, optional insecure, and optional borrowed registry credentials (username/password/registry token). Handlers map domain errors to 400/404/409; empty credential objects are treated like omitted so the server keychain is not masked. Layer/byte counts are only exposed when status is pushed.

lib/imagepush: bounded queue, on-disk job metadata under data/pushes, in-flight dedup by digest+target+insecure with credential fingerprint conflicts, restart recovery (credentialed jobs fail; anonymous jobs re-queue), and LiveCacheManifestDigests so GC keeps blobs alive mid-push.

Infrastructure: limits.max_concurrent_pushes (default 2); image build concurrency moves from images.BuildQueue to shared lib/queue; OCI cache GC roots are composed from the embedded registry and the push manager. Wire/DI injects PushManager; scopes reuse existing image read/write.

^{Reviewed by Cursor Bugbot for commit 13a1a28. Bugbot is set up for automated code reviews on this repo. Configure here.}

chruffins · 2026-08-06T14:24:23Z

reviewed — solid, well-tested layer 3: clean handler wiring, thorough error mapping, good happy-path tests. the bugs, nits, and the interface/timing structural items are fixed in 9302c19e and fa579f02 (nil-body guard, status-gated counters, Manager interface trim with the GC adapter moved into main.go, GC roots re-queried after the mark, and the missing 500-path handler tests); the items below remain open by decision.

Structural / Maintainability (open)

cmd/api/api/pushes.go + lib/imagepush/storage.go — ListPushes returns every push ever created with no retention; deferred (matches repo-wide precedent, and pagination/retention is a cross-endpoint decision).

Questions (open)

lib/registrypush/provider.go — nil keychain falls back to authn.DefaultKeychain (the server's stored registry credentials); this is documented intent and kept as-is for single-operator self-host. revisit if hypeman is ever deployed multi-tenant.

Notes

queue + storage — third parallel queue+disk-recovery implementation in the repo; consolidation ticketed, not done in this stack.

status: all review findings addressed except the listed deferrals

github-actions · 2026-08-06T17:50:04Z

-->

✱ stlc build

✅ go code · compare

Your SDK build was successful.

generate ✅ → bootstrap ✅ → format ✅

114 files generated at d37b9eb (pushed)
go get github.com/kernel/hypeman-go-staging@d37b9eb6e242b13919aceb1dfd5ba7e8052dae8d

✅ typescript code · compare

Your SDK build was successful.

generate ✅ → bootstrap ✅ → format ✅

136 files generated at c1833c8 (pushed)

Diagnostics: 💡 0 new / 5 total note

Level	Code	Message	Targets

Build metadata

Build	`bd_766nJgJY-noble-pail`
Timestamp	`2026-08-10T14:53:28.201Z`
stlc	`8413509`
Spec hash	`88d816cbcc30`
Config hash	`ee536b54c599`

This comment is auto-generated by stlc and is kept up to date as you push.
If you push new commits, re-run this workflow to update this comment.
Last updated: 2026-08-10 14:53:46 UTC

Orchestrates pushing cached images to remote registries: resolves a hypeman image to its cached digest, runs push jobs on a bounded queue with digest+target deduplication, persists job status to disk with FIFO recovery across restarts, and exposes in-flight digests so the OCI cache GC can keep required blobs alive mid-push.

Docker-aligned credential flow: the caller's registry login rides along with the push request instead of living on the server. Borrowed credentials are used only for that job, never persisted or logged, and a credentialed push interrupted by a restart fails with an explanation instead of retrying under different credentials. The manager's default provider remains the fallback when no credentials are supplied.

- Release the inflight registration only after the queue slot is freed, closing a window where a new push for the same key persisted metadata but was never started because the queue still held the slot. - CreatePush adopts a pending record already on disk for the same digest+target instead of duplicating it. - Recovery dedupes same-key records (oldest wins, rest marked failed) and removes records whose status cannot be persisted, so nothing is left permanently queued. - Terminal status writes retry once and drop the record on failure so disk state cannot diverge from the WaitForPush notification.

…rsist-or-fail terminals - Defer queue completion so a panicking job still releases its slot and runs the completion hook. - An orphaned pending record that used borrowed credentials is closed with the recovery policy and replaced by a fresh job instead of being re-executed under the default provider. - When a terminal status cannot be persisted even after retry, drop the record and report the job failed with the persistence problem so WaitForPush and GetPush agree, logging the actual push outcome.

failRecovered persisted the failed status without notifying, so a WaitForPush racing the close could subscribe before the write and then wait for a notification that never comes.

- Include the insecure flag in the dedup/recovery key: the same target pushed with different transport modes is distinct work. - Log loudly when startup recovery cannot list pending pushes instead of failing silently. - Contain panics in the push goroutine: record a failed terminal and notify waiters instead of leaving the job stuck as pushing. - A request that lends credentials supersedes a credential-less orphan instead of adopting it, so the push never runs under the wrong auth.

CreatePush now refuses to merge a request into an in-flight job whose credential presence differs (one borrowed, one not), returning ErrCredentialConflict instead of silently running under the wrong auth. The orphan-adoption path in CreatePush was unreachable: startup recovery adopts every pending record before any CreatePush can run, and the create lock covers write+registration, so the per-request PushesDir scan was dead O(N) disk I/O under the lock. Remove it along with findPendingPush. listAllPushes now warns on unreadable metadata instead of swallowing it.

Extract lib/queue, a minimal in-memory bounded queue with key dedup, a concurrency cap, and an optional completion hook, and use it from imagepush and images; delete their local queue implementations (pushQueue and images' BuildQueue). builds keeps its superset with serial keys. imagepush/manager_test.go: add a testManager fixture to collapse the repeated paths+cache+resolver+NewManager setup, and merge the not-ready/unknown/ invalid-target rejection tests into one table-driven test. Net: -23% PR size (2030 -> ~1930 insertions) with behavior preserved.

Add mustPushed and writePushes helpers to collapse the repeated wait-for-push + get + assert-pushed blocks (6 sites) and the recovery fixture metadata writes, without changing coverage.

- lib/queue: replace the variadic done hook with a plain nil-able param, document that a dedup'd enqueue does not run it, and note the under-lock launch is safe - treat an empty non-nil credential config as anonymous (credsPresent) - persist push metadata durably: fsync the temp file before rename - use a distinct (non-wrapped) error for empty-digest-on-ready, a corrupted-record state rather than not-ready - sort InProgressDigests for determinism Tests: assert pending QueuePosition via the manager read surface, verify WaitForPush on a superseded recovered job fails, and cover recovery when blobs were reclaimed by GC between crash and restart.

POST /pushes creates a push job exporting a ready hypeman image from the OCI cache to a remote registry; GET /pushes and GET /pushes/{id} expose job state. Requests may lend registry credentials, which the push manager borrows for that job only and never persists; without them the server's own credentials resolve via the Docker keychain. Routes use the existing image:read/image:write scopes, the push queue concurrency is configurable, and in-flight push digests are composed into the OCI cache GC roots.

- Pin x-enum-varnames on PushStatus so generated constants carry the PushStatus prefix like the other enums, avoiding bare Failed/Queued names in the oapi package. - Treat an empty credentials object the same as absent credentials so the server's default credential resolution stays in effect instead of pushing with an empty auth config.

Without a pushes resource, generated SDK clients omit the push API even though the server exposes it. Mirrors the builds resource mapping.

…lify dedup

cursor

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 13a1a28. Configure here.}

cursor · 2026-08-10T20:00:13Z

+			// clear retryable error rather than orchestrating this caller into
+			// a successor job that does not yet exist. A retry once the entry
+			// drops creates a fresh job.
+			return nil, fmt.Errorf("%w: push job %s is being finalized after a record write failure; retry", ErrNotFound, id)


Finalization error becomes 500

Medium Severity

After removing the inflight-release wait, CreatePush can return wrapped imagepush.ErrNotFound during the persist-failure teardown window and labels it retryable, but the handler only maps images.ErrNotFound to 404 and treats this as internal_error 500.

Additional Locations (1)

cmd/api/api/pushes.go#L38-L54

^{Reviewed by Cursor Bugbot for commit 13a1a28. Configure here.}

chruffins mentioned this pull request Aug 5, 2026

Document pushing images to remote registries #354

Draft

chruffins force-pushed the hypeship/registry-push-api branch from 3529276 to 143d09b Compare August 5, 2026 19:18

cursor Bot reviewed Aug 5, 2026

View reviewed changes

Comment thread openapi.yaml

Comment thread cmd/api/api/pushes.go

chruffins force-pushed the hypeship/registry-push-api branch 2 times, most recently from 912ae22 to f2ee13c Compare August 5, 2026 19:43

cursor Bot reviewed Aug 5, 2026

View reviewed changes

Comment thread openapi.yaml

chruffins force-pushed the hypeship/registry-push-api branch 2 times, most recently from 8eefe9c to d90000d Compare August 5, 2026 20:03

chruffins force-pushed the hypeship/registry-push-api branch from 90d7da1 to 9302c19 Compare August 6, 2026 14:17

chruffins force-pushed the hypeship/registry-push-api branch from 9302c19 to fa579f0 Compare August 6, 2026 15:19

chruffins force-pushed the hypeship/registry-push-api branch from fa579f0 to b244f44 Compare August 6, 2026 16:13

chruffins force-pushed the hypeship/registry-push-api branch from b244f44 to dbd4c99 Compare August 6, 2026 17:09

cursor Bot reviewed Aug 6, 2026

View reviewed changes

Comment thread lib/ocicachegc/gc.go Outdated

cursor Bot reviewed Aug 6, 2026

View reviewed changes

Comment thread lib/ocicachegc/gc.go Outdated

chruffins force-pushed the hypeship/registry-push-api branch 2 times, most recently from 846fef3 to 7448f03 Compare August 6, 2026 21:10

chruffins force-pushed the hypeship/registry-push-api branch from 7448f03 to ad44f2a Compare August 7, 2026 13:52

cursor Bot reviewed Aug 7, 2026

View reviewed changes

Comment thread cmd/api/api/pushes.go

chruffins added 10 commits August 10, 2026 14:38

Notify subscribers when a recovered push is closed

3a74966

failRecovered persisted the failed status without notifying, so a WaitForPush racing the close could subscribe before the write and then wait for a notification that never comes.

imagepush: dedupe test wait/assert scaffolding

888c9f3

Add mustPushed and writePushes helpers to collapse the repeated wait-for-push + get + assert-pushed blocks (6 sites) and the recovery fixture metadata writes, without changing coverage.

chruffins force-pushed the hypeship/registry-push-api branch from bd67d51 to 85330bd Compare August 10, 2026 14:50

cursor Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread cmd/api/api/pushes.go

chruffins added 2 commits August 10, 2026 15:09

imagepush: deflake queue completion-hook ordering test

da865ae

images: fix build-phase metrics test after build-queue consolidation

2c7f833

chruffins force-pushed the hypeship/registry-push-api branch from 85330bd to 1a95a28 Compare August 10, 2026 15:36

imagepush: address review findings on dedup, recovery, and tests

68b750e

chruffins force-pushed the hypeship/registry-push-api branch from 1a95a28 to 2a4434d Compare August 10, 2026 16:14

chruffins added 5 commits August 10, 2026 16:42

imagepush: merge dedup waiters into successor instead of re-pushing

f461185

Fix gofmt alignment for push manager wiring

2076b55

Add pushes resource to the Stainless SDK config

a9a4500

Without a pushes resource, generated SDK clients omit the push API even though the server exposes it. Mirrors the builds resource mapping.

chruffins force-pushed the hypeship/registry-push-api branch from 2a4434d to a9a4500 Compare August 10, 2026 16:42

Base automatically changed from hypeship/registry-push-manager to main August 10, 2026 18:42

Address pushes API review: push timeout, drop dead wait surface, simp…

13a1a28

…lify dedup

chruffins mentioned this pull request Aug 10, 2026

Address pushes API review findings #385

Closed

cursor Bot reviewed Aug 10, 2026

View reviewed changes

Navigation Menu

Uh oh!

Add pushes API for exporting images to remote registries - #353

Add pushes API for exporting images to remote registries#353
chruffins wants to merge 19 commits into
mainfrom
hypeship/registry-push-api

chruffins commented Aug 5, 2026 •

edited by cursor Bot

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chruffins commented Aug 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Aug 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

chruffins commented Aug 5, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

API (openapi.yaml → generated)

Credentials — borrow, don't store

Wiring

Tests

Notes

Verification

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chruffins commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Structural / Maintainability (open)

Questions (open)

Notes

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✱ stlc build

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Aug 10, 2026

Choose a reason for hiding this comment

Finalization error becomes 500

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

chruffins commented Aug 5, 2026 •

edited by cursor Bot

Loading

chruffins commented Aug 6, 2026 •

edited

Loading

github-actions Bot commented Aug 6, 2026 •

edited

Loading