From ced76b5264d606582252c226292fec30c983af87 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Sat, 29 Aug 2026 17:58:25 +0000 Subject: [PATCH 1/3] docs(layout): reverse the layout model onto two optional booleans, and shrink the source-scope surface MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The layout question had grown to eight documents across `spec/`, `design/` and `future/`, so following the argument meant knowing which folder each step lived in. `docs/layout/` now holds it: two current-behavior contracts, one design page, and the worked examples. Everything else stays where it was. **The model reverses its own earlier thesis and leads with the reversal.** #319 shipped the ancestor walk, which made registration an invariant and retired three of the five arguments against path templates. So the template stays, and what it cannot express becomes two optional booleans on the existing `spec.placement`: - `useKustomize` — create and maintain the folder's `kustomization.yaml`. Registering a new file with a root that already governs it is an invariant, not a setting; the flag's one job is what to do when there is no root. Its `true` half is the only genuinely new machinery here, and it is what makes an empty repository bootstrappable. - `serializeNamespace` — whether the document carries `metadata.namespace`. A `*bool`, because no plain default preserves today's behavior: `false` breaks a flat folder, `true` writes a redundant line into every kustomize folder that already supplies one. Unset means infer, which is not a guess — the existing inference omits the namespace only when the governing kustomization sets it to this resource's own namespace, so an explicit `false` is an override of a correctness rule and gets a post-scan guard. The two pair up: with `useKustomize: true` the operator owns the root it writes `namespace:` into, which is what makes both the omission provable and the created kustomization meaningful rather than an empty file. `spec.layout`, `kind`, `scope`, `kustomize.create`, the LayoutProfile question, the migration and four maintainer-review findings are deleted with it. Nothing in the placement work is breaking, so none of it waits for a coordinated consumer bump. **Source scope.** Declines Flux-style impersonation and deletes `GitTarget.spec.allowedSourceNamespaces` with its selector machinery (4,569 lines, and the only cross-cluster read in the authorization path), on an API reading rather than a security one: the chain from a Git folder back to the object that fills it never leaves one namespace. Keeps `allowedNamespaces`, renamed `accessFrom`, reversing an earlier draft. Redefines `sourceNamespace: "*"` as one cluster-wide list and watch. Also raises F9's envtest to Tier 1 — the only queue item whose answer is unknown rather than whose work is unscheduled — and prices staying on `v1alpha3` as a one-consumer countdown rather than a constant. The Go changes are comment-only: doc-path citations following the move. `task lint` and `task test` pass; e2e is unaffected because no executable line changed. Co-Authored-By: Claude Opus 5 --- api/v1alpha3/gittarget_types.go | 2 +- docs/INDEX.md | 40 +- docs/TODO.md | 13 +- docs/architecture.md | 2 +- docs/design/gittarget-api-wave.md | 417 +++++----- docs/design/gittarget-layout-model.md | 525 ------------- docs/design/open-asks-priority.md | 59 +- ...cement-visibility-and-declared-defaults.md | 31 +- docs/design/source-scope-simplification.md | 247 ++++++ .../images-and-replicas-edit-through.md | 2 +- .../helm-light-support-boundary.md | 4 +- .../support-boundary/render-root-scoping.md | 23 +- .../repo-discovery-and-onboarding-scan.md | 4 +- ...ernetes-impersonation-and-flux-identity.md | 106 +++ ...fig-surface-for-a-structured-repository.md | 6 +- .../direction-and-configuration-surface.md | 66 +- ...intainer-review-status-and-config-model.md | 731 ------------------ docs/layout/README.md | 32 + .../contextual-namespace.md} | 8 +- docs/layout/examples/README.md | 108 +++ .../examples/brownfield-kustomize/README.md | 81 ++ .../config/gittarget.yaml | 14 + .../config/watchrule.yaml | 15 + .../brownfield-kustomize/expected-cache.patch | 19 + .../brownfield-kustomize/input/cache.yaml | 7 + .../repository/deployment-web.yaml | 19 + .../repository/kustomization.yaml | 6 + .../repository/service-web.yaml | 10 + .../examples/empty-repo-bootstrap/README.md | 61 ++ .../config/gittarget.yaml | 19 + .../config/watchrule.yaml | 12 + .../expected-first-write.patch | 26 + .../input/storefront.yaml | 15 + .../repository/kustomization.yaml | 5 + .../repository/storefront.yaml | 11 + docs/layout/examples/homelab-argocd/README.md | 96 +++ .../homelab-argocd/config/gittarget.yaml | 17 + .../homelab-argocd/config/watchrule.yaml | 12 + .../homelab-argocd/expected-paperless.patch | 30 + .../homelab-argocd/input/paperless.yaml | 47 ++ .../repository/application-jellyfin.yaml | 17 + .../repository/application-nextcloud.yaml | 17 + .../repository/kustomization.yaml | 6 + docs/layout/examples/homelab-flux/README.md | 130 ++++ .../homelab-flux/config/gittarget-media.yaml | 17 + .../homelab-flux/config/gittarget.yaml | 15 + .../homelab-flux/config/watchrule-media.yaml | 12 + .../homelab-flux/config/watchrule.yaml | 12 + .../homelab-flux/expected-bitnami.patch | 21 + .../examples/homelab-flux/input/bitnami.yaml | 40 + .../apps/home/media/helmrelease-jellyfin.yaml | 25 + .../apps/home/media/kustomization.yaml | 5 + .../home/sources/gitrepository-homelab.yaml | 7 + .../home/sources/helmrepository-jellyfin.yaml | 7 + .../home/sources/kustomization.yaml | 6 + .../examples/overlay-scoped-target/README.md | 75 ++ .../config/gittarget.yaml | 13 + .../config/watchrule.yaml | 12 + .../expected-image-update.patch | 9 + .../input/deployment-podinfo.yaml | 20 + .../apps/podinfo/base/deployment.yaml | 19 + .../apps/podinfo/base/kustomization.yaml | 4 + .../podinfo/overlays/prod/kustomization.yaml | 8 + docs/layout/examples/prerequisites/README.md | 38 + .../prerequisites/config/gitprovider.yaml | 16 + .../examples/tree-multi-namespace/README.md | 93 +++ .../config/clusterprovider.yaml | 9 + .../config/clusterwatchrule.yaml | 12 + .../config/gittarget.yaml | 18 + .../config/watchrule.yaml | 21 + .../expected-grafana-dashboards.patch | 12 + .../input/grafana-dashboards.yaml | 7 + .../clusterroles/homelab-viewer.yaml | 8 + .../media/apps/deployments/jellyfin.yaml | 17 + .../repository/media/configmaps/jellyfin.yaml | 8 + .../monitoring/configmaps/grafana.ini.yaml | 8 + docs/layout/model.md | 324 ++++++++ .../new-file-placement-rules.md} | 30 +- docs/spec/README.md | 10 +- docs/spec/manifest-system.md | 4 +- docs/spec/sops-single-file-no-multidoc.md | 6 +- .../gittarget_placement_validation.go | 2 +- internal/git/manifestedit/kustomization.go | 2 +- internal/git/plan_flush.go | 4 +- .../contextual_namespace_corpus_test.go | 2 +- internal/manifestanalyzer/placement.go | 8 +- internal/manifestanalyzer/store.go | 4 +- .../testdata/contextual-namespace/README.md | 2 +- internal/types/identifier.go | 2 +- test/e2e/new_file_placement_e2e_test.go | 2 +- 90 files changed, 2531 insertions(+), 1583 deletions(-) delete mode 100644 docs/design/gittarget-layout-model.md create mode 100644 docs/design/source-scope-simplification.md create mode 100644 docs/facts/kubernetes-impersonation-and-flux-identity.md delete mode 100644 docs/future/flux-maintainer-review-status-and-config-model.md create mode 100644 docs/layout/README.md rename docs/{spec/contextual-namespace-and-kustomize-folder-editing.md => layout/contextual-namespace.md} (98%) create mode 100644 docs/layout/examples/README.md create mode 100644 docs/layout/examples/brownfield-kustomize/README.md create mode 100644 docs/layout/examples/brownfield-kustomize/config/gittarget.yaml create mode 100644 docs/layout/examples/brownfield-kustomize/config/watchrule.yaml create mode 100644 docs/layout/examples/brownfield-kustomize/expected-cache.patch create mode 100644 docs/layout/examples/brownfield-kustomize/input/cache.yaml create mode 100644 docs/layout/examples/brownfield-kustomize/repository/deployment-web.yaml create mode 100644 docs/layout/examples/brownfield-kustomize/repository/kustomization.yaml create mode 100644 docs/layout/examples/brownfield-kustomize/repository/service-web.yaml create mode 100644 docs/layout/examples/empty-repo-bootstrap/README.md create mode 100644 docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml create mode 100644 docs/layout/examples/empty-repo-bootstrap/config/watchrule.yaml create mode 100644 docs/layout/examples/empty-repo-bootstrap/expected-first-write.patch create mode 100644 docs/layout/examples/empty-repo-bootstrap/input/storefront.yaml create mode 100644 docs/layout/examples/empty-repo-bootstrap/repository/kustomization.yaml create mode 100644 docs/layout/examples/empty-repo-bootstrap/repository/storefront.yaml create mode 100644 docs/layout/examples/homelab-argocd/README.md create mode 100644 docs/layout/examples/homelab-argocd/config/gittarget.yaml create mode 100644 docs/layout/examples/homelab-argocd/config/watchrule.yaml create mode 100644 docs/layout/examples/homelab-argocd/expected-paperless.patch create mode 100644 docs/layout/examples/homelab-argocd/input/paperless.yaml create mode 100644 docs/layout/examples/homelab-argocd/repository/application-jellyfin.yaml create mode 100644 docs/layout/examples/homelab-argocd/repository/application-nextcloud.yaml create mode 100644 docs/layout/examples/homelab-argocd/repository/kustomization.yaml create mode 100644 docs/layout/examples/homelab-flux/README.md create mode 100644 docs/layout/examples/homelab-flux/config/gittarget-media.yaml create mode 100644 docs/layout/examples/homelab-flux/config/gittarget.yaml create mode 100644 docs/layout/examples/homelab-flux/config/watchrule-media.yaml create mode 100644 docs/layout/examples/homelab-flux/config/watchrule.yaml create mode 100644 docs/layout/examples/homelab-flux/expected-bitnami.patch create mode 100644 docs/layout/examples/homelab-flux/input/bitnami.yaml create mode 100644 docs/layout/examples/homelab-flux/repository/apps/home/media/helmrelease-jellyfin.yaml create mode 100644 docs/layout/examples/homelab-flux/repository/apps/home/media/kustomization.yaml create mode 100644 docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/gitrepository-homelab.yaml create mode 100644 docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/helmrepository-jellyfin.yaml create mode 100644 docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/kustomization.yaml create mode 100644 docs/layout/examples/overlay-scoped-target/README.md create mode 100644 docs/layout/examples/overlay-scoped-target/config/gittarget.yaml create mode 100644 docs/layout/examples/overlay-scoped-target/config/watchrule.yaml create mode 100644 docs/layout/examples/overlay-scoped-target/expected-image-update.patch create mode 100644 docs/layout/examples/overlay-scoped-target/input/deployment-podinfo.yaml create mode 100644 docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/deployment.yaml create mode 100644 docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/kustomization.yaml create mode 100644 docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/overlays/prod/kustomization.yaml create mode 100644 docs/layout/examples/prerequisites/README.md create mode 100644 docs/layout/examples/prerequisites/config/gitprovider.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/README.md create mode 100644 docs/layout/examples/tree-multi-namespace/config/clusterprovider.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/config/clusterwatchrule.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/config/gittarget.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/config/watchrule.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/expected-grafana-dashboards.patch create mode 100644 docs/layout/examples/tree-multi-namespace/input/grafana-dashboards.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/repository/_cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/repository/media/apps/deployments/jellyfin.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/repository/media/configmaps/jellyfin.yaml create mode 100644 docs/layout/examples/tree-multi-namespace/repository/monitoring/configmaps/grafana.ini.yaml create mode 100644 docs/layout/model.md rename docs/{spec/gittarget-new-file-placement-rules.md => layout/new-file-placement-rules.md} (97%) diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 65ba8d68..dfc6125a 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -149,7 +149,7 @@ type GitTargetSpec struct { // GitTargetPlacementSpec declares where NEW resources are written when no document // for their identity exists yet in Git — one exact-type map plus a fallback // default template (Option B2 of -// docs/spec/gittarget-new-file-placement-rules.md). There is +// docs/layout/new-file-placement-rules.md). There is // deliberately no separate "sensitive" placement block: sensitivity is a // write-safety classification the controller owns (encrypt the content, keep the // path identity-complete, never append or co-mingle), not a second placement diff --git a/docs/INDEX.md b/docs/INDEX.md index 3bef7830..02bec0df 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -4,7 +4,7 @@ This page names the documents that bind the current implementation. If a documen listed here, it is either a user guide (see [`README.md`](README.md)) or historical context you can safely skip. -## The four folders, and what each one means +## The folders, and what each one means | Folder | Means | Binds? | |---|---|---| @@ -12,6 +12,7 @@ you can safely skip. | [`design/`](design/) | **We are still deciding.** Open questions, proposals, unbuilt work. | yes — as intent, not as shipped behaviour | | [`facts/`](facts/) | Durable reference: how Kubernetes behaves, and what we discovered about it. | yes, as reference | | [`finished/`](finished/) | **This happened.** Shipped plans, closed investigations. Kept for context. | **no** | +| [`layout/`](layout/README.md) | **One topic, all of it.** Where a document goes in Git: two current-behaviour contracts, the proposal, its plan, its review, and the worked examples. | per document, and each is labelled | The rule that was missing before: `design/` used to hold shipped work and `finished/` used to hold live contracts. If you are adding a document, pick the @@ -25,6 +26,13 @@ declares itself non-binding. It is labelled **built.** in the table below instea the lifecycle is still readable. `spec/` is for a contract stated as a contract; this is a plan whose reasoning the code kept. +A second exception, and this one is about topic rather than lifecycle. [`layout/`](layout/README.md) +collects the whole layout question, which had grown to eight documents across three folders, so +following the argument meant knowing which folder each step lived in. It mixes lifecycle classes +on purpose and labels every entry with the class it would have had, which its +[README](layout/README.md) does. It is the only topic folder, and adding a second one should take +the same amount of argument this one did. + ## If you are new: read these five 1. [`../README.md`](../README.md) — what the operator does. @@ -58,8 +66,6 @@ misled. Full list in [`spec/README.md`](spec/README.md); the ones that carry a | [`current-manifest-support-review.md`](spec/current-manifest-support-review.md) | all-or-nothing folder claim; never half-write a multi-doc file; **refuse rather than prune** | | [`manifestedit-field-ownership-spike.md`](spec/manifestedit-field-ownership-spike.md) | the API wins — full-object ownership, never field-subset | | [`reconcile-via-watchlist-mark-and-sweep.md`](spec/reconcile-via-watchlist-mark-and-sweep.md) | **no bookmark, no sweep** | -| [`contextual-namespace-and-kustomize-folder-editing.md`](spec/contextual-namespace-and-kustomize-folder-editing.md) | kustomize namespace inference; the supported subset | -| [`gittarget-new-file-placement-rules.md`](spec/gittarget-new-file-placement-rules.md) | where a new resource's file goes: declared, the folder's one kustomize root, canonical. Sibling inference is removed, and kept as history | | [`sops-single-file-no-multidoc.md`](spec/sops-single-file-no-multidoc.md) | one encrypted file is one document | | [`scale-subresource-audit-rehydration.md`](spec/scale-subresource-audit-rehydration.md) | `/scale` only; every other subresource ignored | | [`commit-window-refactor.md`](spec/commit-window-refactor.md) | one grouped commit = one (author, GitTarget) | @@ -96,10 +102,9 @@ Eighteen other open items: | Doc | Open question | |---|---| -| [`open-asks-priority.md`](design/open-asks-priority.md) | **the work queue.** Swept 2026-07-30 against the 0.41.0 release, which carries the attribution model and the placement break and deliberately **not** the GitTarget wave: every Tier 2 entry that changes a `GitTarget` field is part of postponed [#294](https://github.com/ConfigButler/gitops-reverser/issues/294) and is not independently schedulable, while the Tier 1 entries are not and must not wait for it. One entry moved up in the sweep, to Tier 1: a declared path into a kustomize subdirectory produces a file that is in Git and rendered by nothing, which is the product being silently wrong. Reconciled 2026-07-29 to what the attribution branch shipped: the fact stream, consumer ask #23, the name tier and metrics Phase 1 are struck from it and recorded in "already shipped", the residue they leave (the removal-wait decision, the head-of-line block on the shard, the aggregated create) is ranked, and #5 loses one of its two arguments because #23's fix retired it. Three backlogs are open at once — the gitops-api consumer asks, the maintainer review's unbuilt block (F6, F9, F10), and the config-surface proposal (B1–B6) — and they overlap. Merges them into one ordered queue under four stated tests, and makes one design call against what was asked: **delete Option C sibling inference** rather than ship an off-switch for it, because it lets a human's edit to the repository change the operator's behaviour with nothing in status recording the move, its central guard has already failed once by cascading, and the explainability its own spec made mandatory was never built. That answers the namespace-leak ask by removal, and means `spec.placement.mode` is never built. **The deletion has shipped**, together with the placement metrics the argument had said to lead *away* from — an objection to their labels, which naming the GitTarget and the type retires — and "what the deletion taught" records the two things building it found: namespace inheritance was a second implementation of a rule that belonged to the governing kustomization, and it was missing the check that the transformer names the resource's own namespace. Open: whether a fall-back to canonical also raises an Event on the GitTarget, and `status.layout` | -| [`placement-visibility-and-declared-defaults.md`](design/placement-visibility-and-declared-defaults.md) | the three questions the inference deletion left, **decided and then not built** — PR #291 shipped the deletion, the counters and the namespace-transformer fix, and none of the eight items this page had queued behind them, which the page now says. The residue is filed as [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) (correctness: a declared path into a kustomize subdirectory is never rendered, and the identity gate rejects the versionless canonical path) and [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) (visibility: `status.layout`, the ambiguous render root, the `declared` metric split, `{kindLower}`). Its Question 2 is superseded outright by the layout model. What still stands: **Keep `canonical`** as the name for the built-in path and split `declared` into `byType`/`default`, because reusing "default" for both a declaration and the absence of one makes the metric unreadable. **No CRD default for `placement.default`**, and the reason is structural rather than the two that look obvious: a defaulted default is never empty, so it shadows the kustomize-root step and every new file in an overlay would take the canonical path, in Git and rendered by nothing. The validation failure is real but the rule doing the rejecting is itself wrong, and the persistence objection is a trade we could take; defaulting the Secret route to work around the first is a floor that vanishes when a user writes any `byType` entry, because map defaults never merge. **`status.layout` instead**, with five worked examples (greenfield, kustomize overlay, brownfield missing one rule, two ambiguous roots, a refusal from an operator-configured sensitive type) over the `MarkTargetRetention` seam, which already enqueues on change and so retires the "the data plane cannot notify the GitTarget" objection. **`{kindLower}`, not a `toLower` function.** Carries three findings that changed a decision: `IdentityCompletePlacementTemplate` demanding `{version}` contradicts the versionless-path decision; two supported kustomizations still produce a file nothing renders and nothing counts; and **a declared path into a subdirectory of a kustomize folder is registered only when render-root scoping happens to be in force**, so one `byType` line reproduces the unrendered-file bug today. Fixing that last one (walk up to the nearest kustomization) also weakens the case against the CRD default from a correctness wall to a legibility trade, which the page says rather than leaving the stronger argument standing | -| [`gittarget-layout-model.md`](design/gittarget-layout-model.md) | **postponed to a later deployment, filed as [#293](https://github.com/ConfigButler/gitops-reverser/issues/293)** — the proposal the placement questions were circling around: a path template is the wrong primitive, so declare what the folder IS. `spec.layout.kind` with the values `Auto`, `Kustomize`, `Tree`, `Flat` and `Template`, plus `byType` overrides valid under every kind, with two rules that carry the value: whatever chose the path, the file is registered with the kustomization that governs it (so F10 becomes unstatable rather than fixed), and a structural kind excludes a blanket `default` (so a declared template can no longer silently disable the render root). `kind: Auto` is a safe CRD default because it NAMES the structural rule instead of standing in front of it, which is why defaulting a mode works where defaulting a path did not, and it is declared inference rather than the undeclared kind that was deleted. `kind: Kustomize` with `create: true` bootstraps an empty repository into a folder `kubectl apply -k` can build. Seven worked examples, a status shape with `declaredKind` beside the resolved `kind`, a mechanical migration for every current configuration, and an argument that the layout should NOT be its own CRD: a shared object changing where N folders write, with nothing on the GitTarget recording it, is the same defect as sibling inference with a different actor, the shared thing is four lines, and generators already solve reuse. Also carries the namespace half: `scope: SingleNamespace` is a STRUCTURAL claim that must agree with the authorization bound `allowedSourceNamespaces`, and it cannot be derived because that matcher may be absent and because the namespaces that arrive come from WatchRule objects that do not own the folder; `writeNamespace` with the values `FromContext`, `Always` and `Never` replaces the inference that decides whether `metadata.namespace` is written, which is the one inference an empty folder cannot perform, and `create: true` lets the operator ESTABLISH the convention by writing `namespace:` into the kustomization it creates. The layout is **immutable** except a widening transition, because GitTarget has no finalizer so recreating one re-adopts every document by identity, and `Auto` resolves once and pins the result so a deleted `kustomization.yaml` cannot silently re-lay-out the folder. Open: whether `scope` should be derived and materialized at creation instead of declared | -| [`gittarget-api-wave.md`](design/gittarget-api-wave.md) | **postponed, filed as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). Not in 0.41.0**, which already carries the attribution model and the placement break. One breaking wave on GitTarget, sequencing the layout model with the maintainer review's still-open API block (F6, F10, F12's reference nit, §3's pushbacks) and the queue's Tier 2 items (B4, B1, #5, #6). The batching argument is the weaker half; the stronger one is that four of them are the same decision seen from different angles: **the folder is described on the GitTarget and the connection describes only the connection**, which is why `commitWindow` and `commit.message` move off `GitProvider`. Two findings change the layout design rather than accompanying it: `spec.mode: Observe` becomes how a layout is adopted safely (a dry run over `status.layout` instead of declare-and-hope), and `spec.interval` plus an observation pass is what keeps the scan-derived half of that status fresh for a target that writes nothing. `spec.suspend` is a precondition rather than a rider, because a layout that creates a `kustomization.yaml` needs a stop button. Records that F7 already shipped the EventRecorder the placement Event was said to be too expensive for, that layout is mutable like `prune`, that F9 stays OUTSIDE the wave because its answer constrains the enum work, and that the version stays `v1alpha3` with loud rejections rather than paying for a conversion path | +| [`open-asks-priority.md`](design/open-asks-priority.md) | **the work queue.** Merges three overlapping backlogs — the gitops-api consumer asks, the API-surface block left unbuilt by the status and configuration-model review, and the config-surface proposal (B1–B6) — into one ordered queue under four stated tests, and says where we deliberately do **not** do what was asked. The standing caveat narrowed once the layout model reversed: a Tier 2 entry belongs to postponed [#294](https://github.com/ConfigButler/gitops-reverser/issues/294) only if it breaks a `GitTarget` field, and everything else is independently schedulable. Makes one design call against what was asked: **delete Option C sibling inference** rather than ship an off-switch for it, because it let a human's edit to the repository change operator behavior with nothing in status recording the move. That deletion has shipped, and "what the deletion taught" records what building it found. **F9 is Tier 1**: the only item whose answer is unknown rather than whose work is unscheduled, and it gates planning the enum work | +| [`placement-visibility-and-declared-defaults.md`](design/placement-visibility-and-declared-defaults.md) | **design.** The three questions the inference deletion left, **decided and then not built**: PR #291 shipped the deletion and none of the eight items queued behind it. The residue was filed as [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) — **which shipped in 0.42.1 via [#319](https://github.com/ConfigButler/gitops-reverser/pull/319) and is what reversed the layout model** — and [#296](https://github.com/ConfigButler/gitops-reverser/issues/296). Its Question 2 is superseded outright by [`layout/model.md`](layout/model.md). What still stands: keep `canonical` as the name for the built-in path and split `declared` into `byType`/`default`; **no CRD default for `placement.default`**, on the structural argument that a defaulted default is never empty and so shadows the kustomize-root rung; `status.layout` instead, over the `MarkTargetRetention` seam that already enqueues on change; and `{kindLower}`, not a `toLower` function | +| [`gittarget-api-wave.md`](design/gittarget-api-wave.md) | **design**, filed as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). What is left of one breaking wave on `GitTarget` after the layout model reversed and left it: B4's `commitWindow`/`commit.message` move off the connection, the source-scope deletion (the only member that makes the API smaller), and the riders. Organizing principle: **the folder is described on the GitTarget, the connection describes only the connection** — and this is where that becomes a struct boundary rather than a sentence, since grouping a field is free only in a release that is already breaking. `spec.mode` and `GitTarget.spec.interval` are both **dropped**, with re-open triggers. Records that F9's envtest stays OUTSIDE the wave and gates it, and that staying `v1alpha3` on loud rejections is a **one-consumer countdown**, not a constant | | [`target-watch-plan.md`](design/target-watch-plan.md) | **built.** The companion to [`watch-manager-ownership.md`](design/watch-manager-ownership.md): the ownership page says WHO applies a plan, this one says WHAT a plan is and what changing it may touch. A cell — group, resource, namespace, deliberately no served version — is the one identity the watch stream, the render-fidelity scope and the mark-and-sweep boundary all agree on, because a key that does not round-trip to the scope it sweeps under is the class of error that deletes user data. The plan is diffed into `keep`/`start`/`restart`/`stop` and applied per cell, so adding one WatchRule stops replaying every unrelated cell into a queue shared with other tenants; a `restart` is a served-version change, which is why the version is spec DATA rather than identity. Readiness and the fidelity revision are per scope, so a KEPT cell holds the result its own replay produced rather than being asked to prove itself again over an unrelated edit. `stop` never touches files — removal is a Git-side sweep under the target's existing `spec.prune.mode`, not a watch-layer delete. "Cut at the producer" is the accepted consequence: nothing fences the queue, so a deselected cell may leave a short tail of writes, bounded by the queue and converged afterwards. Still open: the `stop` classification wants a settled `TypeRemoved` from `typeset` (see TODO), and removal on INTENT is undecided. | | [`watch-manager-ownership.md`](design/watch-manager-ownership.md) | **built.** A rule edit used to be applied inline by the controller worker that observed it, and it re-planned EVERY GitTarget rather than the one the rule names: 1256 plan reconciles across 28 targets in one e2e run, peaking at 78 in a second, behind two network calls, on a shared worker pool. The watch manager had no owner, so eleven mutexes stood in for one. Now controllers post a trigger naming a GitTarget and return, one loop owns the plan and paces itself, and repeated triggers for one target collapse into a single pass. The debounce is framed around how the config is actually edited: a GitTarget and its rules are one piece of configuration applied together, so a per-target ROLLING SILENCE window of 2s (max wait ~10s) turns a five-object `kubectl apply` into one pass, the same mechanism `DefaultCommitWindow` already uses one layer down on the write path. That reverses an earlier revision's "never debounce the first declaration": declaring a GitTarget the instant it lands means declaring it with no rules yet, which manufactures a transient EMPTY plan on every cold start, and an empty plan is what vacuously cleared a write divergence in the fidelity gate. States the contract as "one settled configuration adjustment, not one function invocation": the window is a heuristic and never a correctness boundary (Kubernetes has no apply-complete event), a per-target DIRTY SEQUENCE means a change arriving mid-pass is never lost, and the pass reads a coherent rule-store snapshot rather than the rule that triggered it. Carries the deletion inventory, because the point is that the system got smaller: four trigger mechanisms collapsed to one (`signalCatalogRefresh` and `catalogRefreshCh` are gone), `refreshRunningTargetWatches` and its running-set filter are gone (that filter is why a target whose first declare never completed was never picked up again), and six mutexes went for stated reasons, with `RenderFidelityGate.mu` and the two event-channel locks kept and justified. The four steps shipped: the local-cluster discovery call is bounded (a real defect — the legacy non-context `ServerGroupsAndResources()` ran with no deadline at all), the owner loop carries the debounce, dirty sequence, per-target deadline and 2s/5s/10s/30s/1m backoff, the plan now carries NO lock while the projection is a published snapshot, and catalog invalidation is scoped by diffing each target's rendered plan across the re-projection. "What shipped" records where the implementation departed from the page, including the one bug only e2e caught: streams were parented to the PASS context, which a deadline cancels the moment the pass returns, so every stream died the instant its plan was applied — and it reads like health, because the plan logs `start:1`, every later pass reports `keep:1` and never restarts it, and nothing logs an error while readiness sits at `Replaying` and every WatchRule sits `Ready=False`. A stream's parent is the manager's lifetime; the pass deadline bounds the pass. A second e2e catch is a BEHAVIORAL consequence worth knowing: toggling a rule off and on inside the settle window is no longer a replay — it used to tear the stream down and re-establish it because each apply replanned synchronously, and it is now one pass over a plan that never changed. Correct (a net-zero change is no change; widening `prune.mode` remains the supported force) but a real difference in what an operator gesture does. Both specs that broke were also gating on the wrong thing: asking a GitTarget "are all your streams running" about a change to ONE rule, which a target that is already mirroring answers True to before that rule has been planned, and which a different controller publishes than the one that compiled the rule — so the rule's OWN StreamsRunning is the gate, and `waitForWatchRuleStreamsRunning` existed unused for exactly this. Departures: reports became a published snapshot rather than a second channel; ISOLATION came from taking the I/O off the loop rather than from the deadline (a pass never dials, the shared refresh runs on its own goroutine, and the deadline is the backstop it should have been — a first cut that kept two network calls on the loop had one unreachable cluster holding every healthy target, which is the same availability failure relocated); DELETION names an incarnation resolved when it is queued, because both production callers react to a NotFound and carry no UID, so a UID-less delete matched everything and could tear down the successor of a same-name recreate; and persistent failure surfaces as `WatchPlanFailing`, where pending means "no pass has ever landed", not "dirty right now". Step 3's type-to-target index did not earn its staleness. Still open: whether the settle window ever needs to be configurable | | [`docs-linting.md`](design/docs-linting.md) | how to mechanize [`style-guide.md`](style-guide.md) with markdownlint-cli2 and Vale. Both are wired into `task lint`, gated on the files [`.docs-lint-scope`](../.docs-lint-scope) lists rather than the whole tree: 102 of 174 files fail markdownlint and 148 of 174 fail Vale, so the two backlogs need different gates. Open: how the scope list grows to cover the tree, the `MD013` limit, and whether `AGENTS.md` and the chart READMEs are in scope | @@ -114,6 +119,25 @@ Eighteen other open items: | [`sensitive-resource-diagnostics-follow-up.md`](design/sensitive-resource-diagnostics-follow-up.md) | deferred diagnostics | | [`e2e-git-server-choice.md`](design/e2e-git-server-choice.md) | stay on Gitea or move to Forgejo — the `_csrf` pin is fixable in place on both, so the migration is now a preference call, not a fix; also why we adopt no SDK either way | | [`azure-devops-multi-ack.md`](design/azure-devops-multi-ack.md) | **decided and built: go-git v6** — why Azure DevOps rejects our fetches, and what to do instead of PR [#292](https://github.com/ConfigButler/gitops-reverser/pull/292)'s bundled `git` binary. The capability filter fails in two independent halves: advertising `multi_ack` is a four-line change, but v5 then cannot parse the multi-ACK **response**, which only a fetch with `have` lines provokes. That is why **Flux ships ADO support on v5 with no git binary — it never fetches**, only `CloneContext`, so it never enters the path v5 cannot serve; our persistent-clone-plus-incremental-fetch design is the opposite, which makes the trim alone insufficient for us. **go-git v6 already implements `multi_ack`** (PR #1204, in every v6 tag; upstream then deleted their ADO example saying it "works out of the box"), and its churn in the packages we import runs 96 → 39 → **1** → **9** removals per alpha, so it is one settled breaking wave rather than a moving target; the migration is four known API removals over two rewritten files, `transport.AuthMethod` being the invasive one. Prices PR #292 as measured rather than argued: the image goes **217 MB → 940 MB**, of which 723 MB is a `cp -rL` that dereferences 165 hardlinks to one binary (a one-character fix), arm64 is unaffected and native, but **Trivy reports zero findings on both images** while the new one carries git 2.54.0, OpenSSH 10.3p1 and OpenSSL 3.5.7 as loose files no package database describes — so the CRITICAL gate is blind to a third of the runtime. Also catches an unflagged non-ADO regression (`Depth: 1` dropped, so every provider full-fetches) and 10% patch coverage on an untestable path. The unlock is that **canonical `git upload-pack` advertises `multi_ack`** (verified), so the Gitea already in the e2e lab plus a 400-injecting proxy is a faithful ADO simulator — no tenant needed, and the only way any option becomes CI-testable. Four options priced, and Option A (v6) is the one shipped. Carries a measured **capability matrix** over our three network calls with two diagrams, which narrows the blast radius to **one call, `repo.Fetch`**: `receive-pack` never advertises `multi_ack` (measured), so **the atomic push is out of scope for every option** — its safety rests on the same-session advertisement plus the server-side `Old`/`New` compare-and-swap in `packp.Command`, neither of which touches `upload-pack`, and we already push from a shallow store today. v6 keeps that pattern 1:1 (`Handshake` → `GetRemoteRefs`/`Push`, same `[]*packp.Command`), which is an argument *for* migrating. Records what the migration actually cost, including the four v6 behaviour changes it surfaced — two of them settings v6 reads from the environment and fails closed on, invisible to unit tests | +| [`source-scope-simplification.md`](design/source-scope-simplification.md) | **proposal, unbuilt.** Declines Flux-style impersonation, deletes `GitTarget.spec.allowedSourceNamespaces` and its selector machinery (**4,569 lines**, and the only cross-cluster read in the authorization path), renames two `ClusterProvider` fields, and redefines `sourceNamespace: "*"` as one cluster-wide list and watch. The argument is an API reading, not a security one: the chain from a Git folder back to the object that fills it never leaves one namespace, so ordinary RBAC on `watchrules` already answers it. **Keeps `allowedNamespaces`** (renamed `accessFrom`), reversing an earlier draft — source RBAC bounds what a credential may READ, never which tenant may WIELD it. Prices what is lost: source-side label selectors. Archaeology in [`facts/kubernetes-impersonation-and-flux-identity.md`](facts/kubernetes-impersonation-and-flux-identity.md) | + +## The layout topic — [`layout/`](layout/README.md) + +Where a live object's document goes in Git, and what else has to change so that file is reachable. +Collected by topic rather than by lifecycle, so the folder mixes binding contracts with an unbuilt +proposal; its [README](layout/README.md) labels each one. Two of these are `spec/`-class and cited +by path from Go source. + +| Document | Class | What it holds | +|---|---|---| +| [`contextual-namespace.md`](layout/contextual-namespace.md) | **spec** | kustomize namespace inference; the supported subset | +| [`new-file-placement-rules.md`](layout/new-file-placement-rules.md) | **spec** | where a new resource's file goes: declared, the folder's one kustomize root, canonical. Sibling inference is removed, and kept as history | +| [`model.md`](layout/model.md) | **design** | **reversed, and much smaller than it was.** The earlier thesis wanted `spec.placement` replaced by a `spec.layout` discriminated union; three of its five arguments were retired by [#319](https://github.com/ConfigButler/gitops-reverser/pull/319), which made registration an invariant. So the template **stays** and gains two optional booleans on `spec.placement`: **`useKustomize`** (create and maintain the folder's root; registering into a root that already exists is an invariant, not a setting) and **`serializeNamespace`** (a `*bool`, because unset must keep meaning "infer" — no plain default preserves today's behavior). Carries four kustomize facts **measured** against v5.8.1, three of which contradict the earlier model; the `status.placement` stanza and the post-scan pass; and the build order. The headline is what it deletes — `spec.layout`, `kind`, `scope`, `kustomize.create`, the `LayoutProfile` question, and the migration — so the largest breaking change in the queue stops being breaking at all | + +[`examples/`](layout/examples/README.md) makes the proposed `GitTarget` layout model tangible: six +scenarios, each a repository folder with the matching proposed `GitTarget` and rule configuration, +one live input, and the exact patch proposed. They are design material, not install manifests, and +[`model.md`](layout/model.md) turns them into an executable corpus in its first PR. ## Deferred, but still wanted — [`future/`](future/) @@ -133,7 +157,7 @@ is the strategy review on top of it: the config-as-data direction as the headlin brownfield mirroring as the on-ramp, a decided Helm standpoint (declaration editing plus the values projection; helm-light inversion parked behind entry criteria), and worked examples of where the configuration surface should go. -Five more ideas sit beside them. +Four more ideas sit beside them. ## History — [`finished/`](finished/) diff --git a/docs/TODO.md b/docs/TODO.md index 62cd455e..5d79241a 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -64,11 +64,14 @@ This file is meant to track the smaller current backlog, not historical notes. Secrets and CozyStack `tenantsecrets`; resources with sensitive fields under shapes such as `spec.credentials` need an explicit field policy or full-file encryption decision. -- [ ] Revisit output layout. **Now designed and postponed, not open-ended.** The answer is to declare - what the folder *is* rather than a path template: `GitTarget.spec.layout`, designed in - [gittarget-layout-model.md](design/gittarget-layout-model.md) and filed as - [#293](https://github.com/ConfigButler/gitops-reverser/issues/293), sequenced with the rest of the - breaking `GitTarget` work as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). +- [ ] Revisit output layout. **Now designed and postponed, not open-ended, and the answer has + reversed since [#293](https://github.com/ConfigButler/gitops-reverser/issues/293) was filed.** The + path template **stays**; what it could not express becomes two optional booleans on + `spec.placement`, `useKustomize` and `serializeNamespace`. [layout/model.md](layout/model.md) + carries the reversal, the fields and the order. The placement work + is no longer breaking, so it no longer needs + [#294](https://github.com/ConfigButler/gitops-reverser/issues/294); the issues still describe the + discriminated union and want updating. Deliberately **not** in 0.41.0, which already carries the new attribution model and the sibling-inference removal. Multiple resources per file is bundle support, which exists for match-first today and is a separate question from where a *new* file goes. diff --git a/docs/architecture.md b/docs/architecture.md index 77f225be..695d665d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1139,7 +1139,7 @@ Placement runs **only for a resource with no existing document** in the target. manifest identity rather than by path, instead of recomputing placement. So a change to how new files are placed never moves a file already in Git. A new resource is placed by the first of these that applies ([internal/manifestanalyzer/placement.go](../internal/manifestanalyzer/placement.go), -[design](spec/gittarget-new-file-placement-rules.md)): +[design](layout/new-file-placement-rules.md)): 1. **Declared policy (`spec.placement`).** A `GitTarget` can declare a `byType` map (exact `[group/]version/resource` → path template) plus a `default` template, rendered from a small diff --git a/docs/design/gittarget-api-wave.md b/docs/design/gittarget-api-wave.md index b8a331df..e06c9d4f 100644 --- a/docs/design/gittarget-api-wave.md +++ b/docs/design/gittarget-api-wave.md @@ -1,132 +1,168 @@ -# One breaking wave: the folder is described on the GitTarget +# The wave after placement left it > **design**: a sequencing proposal, not a plan of record. Nothing here binds until scheduled. > Index: [`../INDEX.md`](../INDEX.md) -> Date: 2026-07-30. +> Date: 2026-08-28 (originally 2026-07-30). > -> Combines three pieces of work that are all `feat(api)!` on `GitTarget` and are cheaper together -> than apart: -> -> - the layout model, [`gittarget-layout-model.md`](gittarget-layout-model.md); -> - the still-open API-surface block of the maintainer review, -> [`flux-maintainer-review-status-and-config-model.md`](../future/flux-maintainer-review-status-and-config-model.md) -> §4 "Then": **F6**, **F9**, **F10**, plus F12's reference-type nit and §3's pushbacks; -> - the Tier 2 breaking items in [`open-asks-priority.md`](open-asks-priority.md): **B4**, **B1**, -> **#5**, **#6**. -> -> It does not touch Tier 1 (the removal-wait decision, #15's condition). Those are not breaking and -> should not wait for this. +> This document was written to sequence one breaking wave whose centrepiece was `spec.layout`. +> [`model.md`](../layout/model.md) has since reversed: the path template stays and gains two additive fields, +> so the placement work is not breaking at all. The wave lost its largest member. What is left is a +> batch, and this page is honest about being one. + +## What is in it -## Why one wave +| Member | Object | Why breaking | +|---|---|---| +| **B4**: `commitWindow` and `commit.message` move off the connection | `GitProvider` → `GitTarget` | fields change object | +| The source-scope deletion ([`source-scope-simplification.md`](source-scope-simplification.md)) | `GitTarget`, `ClusterProvider`, `WatchRule` | one removal, two renames, one redefinition | +| The riders: **#5** asserted `CommitRequest.spec.author`, the `CommitRequest` lifecycle hole, `meta.LocalObjectReference` for our six reference shapes, the `TooManyStreams` cap, the `default` `ClusterProvider` message | various | shape changes | + +Additive, and therefore **not** wave members even though they are discussed here: `spec.suspend`, +`status.placement` and the post-scan pass, the reconcile-request annotation, and the two placement +fields. They ship whenever they are ready. Tier 1 in +[`open-asks-priority.md`](open-asks-priority.md) — the removal-wait decision, #15's +condition — is untouched and should not wait for this. + +## Why still one wave The consumer pins us three ways (image, Go module, `require` line), so each breaking release costs a -coordinated bump. That argues for batching. It is the weaker half of the argument. +coordinated bump. That argues for batching, and with placement gone it is most of what is left. -The stronger half: **four of these items are the same design decision seen from different angles**, -and building them separately means deciding it four times, inconsistently. +The stronger half survives in reduced form: several of these items are the same design decision seen +from different angles, and building them separately means deciding it several times, inconsistently. > **The folder is described on the GitTarget. The connection describes only the connection.** -- `spec.layout` says what the folder **is** (layout model). -- `spec.mode: Observe|Write` says whether we write to it at all (B1). -- `spec.suspend` says whether we write to it *now* (F6). -- `commitWindow` and `commit.message` say how writes to it are batched and phrased, and today they - live on `GitProvider`, which is the connection (B4, and §3's "GitProvider is doing three jobs"). - -Ship `spec.layout` alone and the principle is asserted by one field while `commitWindow` still -contradicts it. Ship them together and the object reads as one idea: a GitTarget is a folder plus a -policy for writing it, and a GitProvider is how you reach the repository. +`spec.suspend` says whether we write to the folder; `commitWindow` and `commit.message` say how those +writes are batched and phrased, and today they live on `GitProvider`, which is the connection (B4, +and the config-model review's "GitProvider is doing three jobs"). Shipping B4 alone would still +assert half the principle, which is the reason to keep the rest together. + +## Where the fields live + +The principle above has been a sentence in a document for two drafts. This wave is the moment it +becomes a struct boundary, because grouping a field costs nothing extra in a release that is +breaking anyway, and costs a bump in every release that is not. + +`GitTargetSpec` today is flat: `providerRef`, `branch`, `path`, `encryption`, `placement`, +`clusterProviderRef`, `allowedSourceNamespaces`, `prune`. Queued on top of it were `suspend`, +`useKustomize`, `serializeNamespace`, `commitWindow` and `commit.message` — five more members on a +spec that already flattens six orthogonal axes. Left alone, this object accumulates faster than it +sheds, and moving `commit.message` off `GitProvider` for exactly that reason while doing it just +relocates the problem one hop. + +Two groupings, and one deliberate exception: + +- **`useKustomize` and `serializeNamespace` nest under `spec.placement`**, where an earlier draft of + [`model.md`](../layout/model.md) had them at the top level beside it. They are placement concerns by + their own argument — one decides what governs the produced document, the other what is inside it — + and `spec.placement` is an existing optional struct, so nesting them is **still purely additive**. + It costs no bump, it is free only before they exist, and it means the placement axis is one member + of the spec rather than three. +- **`commitWindow` and `commit.message` land as `spec.commit`**, not as two top-level fields: + `spec.commit.window` and `spec.commit.message.template`. The move is breaking either way, so the + grouping is free, and `GitProvider.spec.commit` is the shape they already have. +- **`spec.suspend` stays top-level.** It is the object-level switch, it is where every Flux user + reaches for it, and a `spec.write` wrapper invented to hold it plus `prune` plus `commit` would be + a category we made up rather than one the API already has. `prune` is its own struct already. + +After the wave the spec reads as named axes rather than a list: the immutable destination +(`providerRef`/`branch`/`path`), `encryption`, `clusterProviderRef`, `placement`, `commit`, `prune`, +and the one switch. That is the test for the next field too — a new member either joins an axis or +names a new one, and if it can do neither it is probably not a `GitTarget` field. ## The interactions that change the design These are the reasons to combine, as opposed to merely batch. Each one changes what gets built. -### 1. `spec.mode: Observe` is how a layout is adopted safely +### 1. Adoption is a dry run, and `spec.suspend` is enough to give it -The layout model's weakest point is adoption: a user pointing `kind: Kustomize` at a real repository -has to trust it before any file moves, and placement only ever affects *new* documents, so there is -nothing to preview by inspection. +Placement only ever affects *new* documents, so there is nothing to preview by inspection: you find +out where files go by letting one be written. -`Observe` mode plus `status.layout` is that preview. In `Observe` the operator scans, resolves the -layout, publishes `renderRoot`, `kind`, and what it *would* do, and writes nothing. Flip to `Write` -when the status says what you expected. That turns "declare a layout and hope" into a dry run, and -it costs nothing extra because both halves are already in the wave. +A suspended target plus `status.placement` is that preview. The operator scans, resolves the render +root, publishes what it *would* do, and writes nothing; clear `suspend` when the status says what you +expected. That needs no second field, which is why **`spec.mode: Observe|Write` (B1) was dropped**: +it bought only the difference between a temporary pause and a declared permanent read-only posture, +which is a distinction in intent, not in behavior. -This also gives `Observe` a purpose beyond "a safety switch nobody uses". It is the mode you adopt a -repository in. +The cost is stated rather than hidden: **`suspend` must keep observing.** Flux's `suspend` stops +reconciliation altogether; ours stops *writes* and keeps scanning, so the status a user is waiting on +stays fresh while they wait. That deviation belongs in the field's documentation, in one sentence, +because it is the only place we differ from a convention a Flux user brings with them. -### 2. `spec.interval` is what keeps the layout observation fresh +**Re-open trigger for `mode`**: someone who needs a target that can never write, as a property of the +object rather than a switch a colleague can flip. -The layout status has two halves ([the other document](placement-visibility-and-declared-defaults.md) -records why): a **current** half derived from the last repository scan, and a **historical** half -accumulated since. The current half is the useful one, and it has a hole: a repository scan happens -on a write or a resync, so a stable target that writes nothing may not scan for a long time, and the -field a user consults would be stamped with a revision from last week. +### 2. `GitTarget.spec.interval` is dropped, because we watch -`spec.interval` (F6) plus `Observe`'s scan-without-writing is the mechanism that closes it: a -periodic observation pass refreshes `renderRoot` and `observedRevision` whether or not anything was -written. Neither piece was proposed for this reason, and together they answer a question neither -answers alone. +Flux polls because **a Git remote cannot be watched**, which is why `GitProvider.spec.interval` stays +and is the honest one of the pair. A `GitTarget`'s inputs are API objects and we are already +streaming them, so every input that can change what a scan would conclude arrives as an event. -### 3. `spec.suspend` is a precondition for a layout that creates files +The name would also have misled: in Flux, `interval` is the drift-correction cadence, and it would +have driven neither of the two passes a Flux user pictures — the observation pass (reads this +target's folder, never writes, never decides deletions) or the resync mark-and-sweep (reads the API +via the streaming list's initial-events snapshot, and is the only thing qualified to infer a +deletion; [`event_router.go`](../../internal/watch/event_router.go), +[`reconcile-via-watchlist-mark-and-sweep.md`](../spec/reconcile-via-watchlist-mark-and-sweep.md)). -`kind: Kustomize` with `create: true` writes a `kustomization.yaml` the user did not author. That is -the right behavior and it raises the stakes on the review's central complaint (F6): *this controller -writes to a Git repository and there is no way to make it stop that is not deleting the object.* +What is left uncovered is one case: a repository whose folder was changed **by someone else** while +our target wrote nothing. The reconcile-request annotation refreshes that on demand. A stale +`observedRevision` on an idle target is a legible cost; a periodic scan on every target is not. -So `suspend` is not a rider here, it is a precondition. A layout that creates structure must ship -with the button that stops it. And `suspend` must stop bootstrap creation specifically, not only -resource writes, which is a detail worth stating before either is built. +**Re-open trigger**: a user who needs an idle target's `status.placement` to track a repository other +people edit, for whom the annotation is not enough. Then it is a scan cadence, named for scanning. -### 4. The Events question is already answered, and the layout is what to say +### 3. `spec.suspend` is a precondition for `useKustomize: true` -[`open-asks-priority.md`](open-asks-priority.md) left one thing open about the inference deletion: -whether a fall-back to canonical should raise an Event on the GitTarget, and it reasoned that this -was expensive because placement runs on the branch worker with no recorder. +Creating a root writes a `kustomization.yaml` the user did not author. That raises the stakes on the +review's central complaint: *this controller writes to a Git repository and there is no way to make +it stop that is not deleting the object.* So `suspend` is not a rider here, it is a precondition — +and it must stop bootstrap creation specifically, not only resource writes. -That is no longer true. **F7 shipped an `EventRecorder` on every reconciler** -(review §6), and the roll-up seam projects data-plane facts into status with an enqueue on change. So -the Event is now: emit when `status.layout` changes in a way a human should know about, which is -`renderRootReason` becoming `Ambiguous`, or a type falling back for the first time. One Event per -persisted change, the pattern F7 already established for `Ready`. +### 4. The Events question is answered by the recorder that shipped -### 5. Layout is immutable, which puts it with `path` rather than with `prune` +[`open-asks-priority.md`](open-asks-priority.md) left open whether a fall-back to canonical +should raise an Event, and reasoned it was expensive because placement runs on the branch worker with +no recorder. An `EventRecorder` now ships on every reconciler, and the roll-up seam projects +data-plane facts into status with an enqueue on change. So the Event is: emit when `status.placement` +changes in a way a human should know about — `LayoutResolved` becoming `Ambiguous`, or a type falling +back for the first time. One Event per persisted change, the pattern already established for `Ready`. -`providerRef`, `branch`, `path` and `clusterProviderRef` are immutable because a folder's meaning is -constituted by them (review §3 defends this well). `spec.prune` is deliberately mutable because -freezing it would destroy the one thing that cannot be rebuilt. +### 5. The source-scope deletion is the only member that shrinks the API -An earlier draft of this document put `spec.layout` with `prune`. That was wrong, and the reason is a -fact worth checking before designing around it: **`GitTarget` has no finalizer**, so deleting one -leaves the folder in Git untouched, and re-creating it at the same path re-adopts every document by -identity. Changing a layout by recreating the object costs status and a moment of mirroring, not data, -which is a different bargain from `prune` entirely. Meanwhile a mutable layout leaves a folder -permanently half one structure and half another, because existing files never move and nothing records -which file came from which. +It belongs here for the ordinary reason first: it removes a field from `GitTarget`, and B4 already +breaks `GitTarget` in the same release, so shipping them apart costs the consumer two bumps for one +object. The better reason is the review surface. Every other member adds a field; this one deletes +4,569 lines, a three-valued verdict, and the only cross-cluster read in the authorization path. A +wave that is otherwise all addition is easier to justify when the object comes out simpler than it +went in. -So: immutable, with a CEL exception for a **widening** transition (`Flat` to `Tree`) that cannot lose -the identity-completeness the folder already had. And `Auto` resolves once and is pinned in status, -because immutability of a field that says "look at the folder" pins nothing. The reasoning is in -[`gittarget-layout-model.md`](gittarget-layout-model.md). +Two interactions worth naming, because they change what gets built rather than merely when: -This settles a question **#6** would otherwise have to reopen: if `path` ever becomes movable -(`status.observedDestination`), the layout moves with it, because a new folder may have a different -structure, and an immutable layout means that transition is one deliberate act rather than two -independent edits. +- **The `SelfSubjectAccessReview` pass is the same shape as the dry run.** Both answer "tell me what + you would do before you do it". They are separate conditions on separate objects and neither + depends on the other, so the SAR work stays additive and can ship after the wave — but whoever + writes the second should read the first. +- **`TooManyStreams` is sized by the `*` decision.** `sourceNamespace: "*"` compiling to one + cluster-wide list and watch removes the fan-out the cap was queued for. What is left to bound is + explicit enumeration, so the cap is still worth a `Stalled` reason and a bound — sized against + enumerated rules, and not planned before the `*` change lands. -### 6. Namespace scope makes `allowedSourceNamespaces` and the layout answer to each other +### 6. Two facts kept from arguments that dissolved -The layout's `scope: SingleNamespace` is a structural claim, and `spec.allowedSourceNamespaces` is an -authorization bound. They are different questions about the same folder, and after this wave they are -checked against each other at admission: a matcher admitting more than one namespace beside a -single-namespace layout is refused. +`spec.layout` immutability and the namespace-agreement rule were both settled by the reversal: +neither field exists. Two facts from underneath them survive and will be reached for again. -This is the connection the wave makes available, and it is not one either item asks for alone. -Authorization already exists and already has a two-party delegation the review praises; the layout is -what turns "who may write here" into "and therefore what this folder looks like", including whether -`metadata.namespace` is written into the files at all. The namespace-in-file question is inference -today, and it is the one piece of inference an empty folder cannot perform, which is why bootstrapping -needs it declared. +- **`spec.placement` is mutable and stays mutable.** Existing files never move, so a template change + affects only what is written afterwards and a folder can hold documents placed under two templates. + Match-first identity keeps finding and editing those in place, so this is worth saying plainly + rather than fixing. +- **`GitTarget` has no finalizer**, so deleting one leaves the folder untouched and re-creating it at + the same path re-adopts every document by identity. Recreating a target costs status and a moment + of mirroring, not data — which is what made immutability affordable for `path` and unaffordable for + `prune`, and is the right test for any future field on this object. ## The GitTarget after the wave @@ -136,139 +172,126 @@ kind: GitTarget metadata: name: prod annotations: - reconcile.configbutler.ai/requestedAt: "2026-07-30T09:14:22Z" # F6 + reconcile.configbutler.ai/requestedAt: "2026-07-30T09:14:22Z" spec: # --- the connection: unchanged, and now only the connection --- providerRef: name: platform branch: main path: clusters/prod + # allowedSourceNamespaces is gone: the provider's credential bounds what can be read, + # and ClusterProvider.accessFrom bounds who may wield it. - # --- what the folder is --- - layout: # immutable, except a widening transition - kind: Kustomize # Auto | Kustomize | Tree | Flat | Template - scope: SingleNamespace # must agree with allowedSourceNamespaces - writeNamespace: Never # the created kustomization carries namespace: - kustomize: - create: true + # --- what the documents look like: ADDITIVE, not part of the wave --- + placement: byType: v1/secrets: "secrets/{name}{sensitiveSuffix}" + useKustomize: true # the created kustomization carries namespace: + serializeNamespace: false - # --- whether and when we write it --- - mode: Write # B1: Observe | Write - suspend: false # F6 - interval: 5m # F6, and what keeps status.layout fresh + # --- whether we write, and how those writes are batched and phrased --- + suspend: false # the only stop-writes switch; a suspended target still scans + commit: # B4, was GitProvider.spec.push.commitWindow and .commit.message + window: 5s + message: + template: "chore(mirror): {{ .Summary }}" prune: mode: OnEvent - - # --- how writes are batched and phrased: moved off the connection --- - commitWindow: 5s # B4, was GitProvider.spec.push.commitWindow - commit: - message: - template: "chore(mirror): {{ .Summary }}" # B4, was GitProvider.spec.commit.message status: - layout: - declaredKind: Kustomize - kind: Kustomize + observedGeneration: 4 + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + observedGeneration: 4 + placement: renderRoot: . - renderRootReason: SingleKustomization + serializeNamespace: false observedRevision: 9f3c1ab - observedTime: "2026-07-30T09:14:22Z" - placedResources: 14 - refusedResources: 0 - lastHandledReconcileAt: "2026-07-30T09:14:22Z" # F6 - observedDestination: # 6 - branch: main - path: clusters/prod + lastHandledReconcileAt: "2026-07-30T09:14:22Z" ``` -Read top to bottom it is one story: reach this repository, this is what the folder is, this is whether -and when we write it, this is how the writes look. +The placement fields are shown in place so the object reads as a whole, but they carry defaults equal +to today's behavior and are additive: only `spec.commit` and the riders make this release breaking. +The status stanza is specified where it is built, in +[`model.md`](../layout/model.md). -## What rides along without a claim of synergy +## The envtest that has to run before any of this is planned -Honesty matters more than a tidy narrative. These are in the wave because they are breaking and the -consumer should pay once, not because they interact with the layout: +One question is not in the wave, and its answer constrains the enum work, so settle it first. -- **#5, `CommitRequest.spec.author`, SAR-guarded.** Independent, and it stands on the argument that - attribution needs an audit webhook a hosted control plane will not give you. -- **F10, CommitRequest lifecycle** (`ttlSecondsAfterFinished` or an `ownerReference`, plus the - `delete` verb). Unrelated to placement; it is the other object in the API with a lifecycle hole. -- **F12's reference types**: embedding `meta.LocalObjectReference` for the name half of our six - near-identical reference shapes. If GitTarget is breaking anyway, this is the moment. -- **§3's `TooManyStreams` cap** for `sourceNamespace: "*"` fan-out. A `Stalled` reason plus a bound, - rather than discovering the cliff as apiserver watch pressure. -- **§3's `ClusterProvider` "default" message.** One error string, and the most likely first-run - support ticket. +`ClusterWatchRule`'s `rules[].scope` enum is narrowed to `Cluster` only, deliberately keeping the +field so that re-applying a stored `Namespaced` value **fails** +([`clusterwatchrule_types.go`](../../api/v1alpha3/clusterwatchrule_types.go)). For CRDs the apiserver +validates the **whole object** against the OpenAPI schema on **status-subresource** updates too. If +that holds here, the controller cannot write `Stalled=True` onto an object whose stored +`spec.rules[].scope` is `Namespaced` — the status update is rejected 422, and the one object that +most needs to explain itself is the one that cannot. CRD Validation Ratcheting skips re-validation of +*unchanged* fields and is beta and default-on from 1.30, GA in 1.33, so the exposure is older +clusters or the gate turned off — which is why the test has to name a version. -**F9** is not in the wave at all: it is one envtest against the minimum supported Kubernetes version, -and its outcome (widen the enum, or keep it) should be known *before* anyone plans an API change -around it. +**The test.** Create a `ClusterWatchRule` with `scope: Namespaced` through a client that bypasses the +enum (or against an older CRD), then attempt a status update, on the minimum supported Kubernetes +version. -## Version strategy: stay `v1alpha3` - -A wave this size invites `v1alpha4`, and I would not take it. +**The fallback if it fails.** Widen the enum back and rely on the compile-path refusal plus a loud +`Stalled` condition. Refusing at admission is nice, but not at the cost of being unable to report the +refusal. -- A new version means a **conversion path**, and the honest options are a conversion webhook (a - serving dependency for the CRD, plus a cert lifecycle) or `None` conversion with a stored-version - migration. Both cost more than the problem. -- We are `v1alpha3` and pre-1.0 with **one consumer**. The convention the repo already uses is a - **loud rejection**: keep the removed field in the schema, refuse it with a message naming the - replacement, for one release. `ClusterWatchRule.spec.rules[].scope` set that precedent, and the - reasoning holds better here: refusing a stored field the user can see beats translating it behind - their back. -- `spec.placement` therefore becomes a rejection that says "use `spec.layout`; `byType` moves - verbatim, `default` becomes `layout.kind: Template`". The mapping is mechanical, which is what makes - the rejection kind rather than merely strict. +## Version strategy: stay `v1alpha3` -If a second consumer appears before this ships, revisit: the calculus that makes loud rejection cheap -is one coordinated bump. +A wave this size invites `v1alpha4`, and I would not take it. A new version means a conversion path, +and the honest options — a conversion webhook (a serving dependency plus a cert lifecycle) or `None` +conversion with a stored-version migration — both cost more than the problem. The convention the repo +already uses is a **loud rejection**: keep the removed field in the schema, refuse it with a message +naming the replacement, for one release. `ClusterWatchRule.spec.rules[].scope` set that precedent, +and a rename is the case it is kindest on: the user's next `apply` tells them the new spelling. + +**The assumption it rests on is one consumer, and that is a countdown rather than a constant.** Loud +rejection is cheap because a single coordinated bump can absorb it; a second consumer makes the same +wave cost two, on two schedules. Two things follow, and both belong on this repo's roadmap rather +than only on the consumer's: + +- **Each wave leaves residue.** A refused field stays in the schema to say "no, not that anymore" — + `allowedSourceNamespaces` now, more later. That graveyard is a real cost to a newcomer reading the + CRD, and it is paid per wave, so the number of remaining waves on `v1alpha3` is finite. Sweeping + the refusals is what `v1alpha4` should be for, and it should be one version bump carrying the + removals rather than a version bump per change. +- **The coupling is a dependency, not a courtesy.** Staying on `v1alpha3` requires the consumer to + keep pace with this repo's release cadence. If it cannot, the choice is not "revisit" in the + abstract — it is a real API version with a real conversion path, and the time to notice is before + the wave, not during it. ## Order inside the wave Dependencies first, then the things that only need the object to be breaking. -1. **F9's envtest.** Not an API change; its answer constrains the enum work. Do it before planning. -2. **`spec.suspend`** (F6). Precondition for anything that creates files, and independently the - review's highest-value gap. -3. **`spec.layout`** with `byType`, plus rule 1 (every written file is registered with the - kustomization that governs it) and rule 2 (a structural kind excludes a blanket `default`). - `spec.placement` becomes a loud rejection. -4. **`status.layout`**, current half from the scan, historical half from the roll-up. Needed before - `Observe` is useful, because `Observe` with nothing to read is a mode that does nothing. -5. **`spec.mode: Observe|Write`** (B1). Now an adoption path rather than a switch. -6. **`spec.interval` + `requestedAt` + `lastHandledReconcileAt`** (F6, rest). Closes the freshness - hole in step 4 and gives the object the reflexes a Flux user already has. -7. **Events on layout change**, over F7's existing recorder. -8. **B4**: `commitWindow` and `commit.message` move from `GitProvider` to `GitTarget`. Last of the - principle items, and the one that makes the object coherent. -9. **The riders**: #5, F10, F12 references, `TooManyStreams`, the `default` ClusterProvider message. - -Steps 2 to 8 are one release. Step 1 gates the planning. Step 9 can be trimmed if the wave gets too -big to review, since nothing else depends on it. +1. **The `scope: Namespaced` envtest**, above. Not an API change; its answer constrains the enum + work. Do it before planning. +2. **`spec.suspend`.** Precondition for anything that creates files, and independently the review's + highest-value gap. +3. **`status.placement`** plus the post-scan validation pass. A dry run with nothing to read previews + nothing. +4. **`requestedAt` + `lastHandledReconcileAt`.** On-demand refresh of step 3. +5. **Events on a changed resolution**, over the existing recorder. +6. **B4**, as `spec.commit`. Last of the principle items, and the one that makes the object coherent. +7. **The source-scope deletion.** Independent of every step above, so it can be written in parallel; + placed here because a deletion reviews better once the additions it is not entangled with are + settled. +8. **The riders.** `TooManyStreams` must come after step 7, which removes the fan-out it was written + to bound. + +Steps 2 to 5 are additive and need no bump. Steps 6 to 8 are one release; step 1 gates the planning; +step 8 can be trimmed if the wave gets too big to review, since nothing else depends on it. The +placement fields are absent from this list on purpose: their order is +[`model.md`](../layout/model.md)'s. ## What this costs, stated plainly -- **One coordinated consumer bump**, with a mechanical migration for every field: `placement.byType` - moves verbatim, `placement.default` becomes `kind: Template`, `commitWindow` and `commit.message` - move object, everything else is additive. -- **One `docs/UPGRADING.md` entry** covering the field moves, the layout mapping, and the one real - behavior change: a declared template stops silently disabling the render root and starts - registering its files. -- **A larger review surface than any change this project has taken.** That is the argument for - trimming step 9 first and for keeping step 1 outside the wave. - -## Open questions - -- **Does `mode: Observe` write status only, or also refuse admission of new WatchRules?** Observe - should be silent about everything except what it observed, but a user in Observe mode with rules - piling up may expect to be told nothing will happen. -- **Should `suspend` and `mode: Observe` be one field?** They are close: both stop writes. They differ - in intent (temporary versus declared) and in what they do to status, and Flux keeps `suspend` - separate from everything else. Two fields, but the field docs must each say what the other is for. -- **Where does `interval` live?** `GitProvider` needs it for `ls-remote` cadence (review F6); - `GitTarget` needs it for the observation pass. Two fields with one name on two objects is a smell, - and one field on the provider cannot express a per-folder observation cadence. -- **Is `Auto` still the right layout default once `Observe` exists?** With a dry-run mode available, - requiring an explicit `kind` costs the user much less than it would have, and it would make every - target's layout self-evident. +- **One coordinated consumer bump**, for `spec.commit`, the source-scope changes and the riders. +- **One `docs/UPGRADING.md` entry.** The placement work needs no migration entry at all. The `*` + paragraph is the one to write carefully: it is the only change here that keeps its spelling and + changes its meaning. +- **A capability genuinely lost**: source-side label selectors, priced in the source-scope document. + It is the only thing in this wave a user can do today and cannot do afterwards. diff --git a/docs/design/gittarget-layout-model.md b/docs/design/gittarget-layout-model.md deleted file mode 100644 index 7b475eed..00000000 --- a/docs/design/gittarget-layout-model.md +++ /dev/null @@ -1,525 +0,0 @@ -# A layout is the declared thing, not a path - -> **design**: a proposal, not a plan of record. Nothing here binds until scheduled. -> Index: [`../INDEX.md`](../INDEX.md) -> Date: 2026-07-30. Supersedes Question 2 of -> [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md), -> which argued about whether to default a path template. The answer is that a path template is the -> wrong primitive to be defaulting. -> -> This is a `feat(api)!` change to `GitTarget`. It does not fit in PR #291 and is not proposed for it. -> How it sequences with the other breaking work on the same object (`spec.suspend`, `spec.mode`, the -> `commitWindow` move, CommitRequest lifecycle) is in -> [`gittarget-api-wave.md`](gittarget-api-wave.md), which also records the two places where those -> items change this design rather than merely accompanying it. - -## Why the current shape keeps producing dead ends - -Placement today is a ladder of four rungs, three of which are path templates and one of which is not: - -```text -byType -> default -> the folder's one kustomize root -> canonical -``` - -Every unresolved question in the review of #291 traces to that one mismatch: - -- a CRD default for `placement.default` cannot be added, because a non-empty template consumes the - slot in front of the rung that is **not** a template (F9 in the other document); -- `byType: {v1/configmaps: "configmaps/{name}.yaml"}` in a kustomize folder produces a file no - kustomization lists, so it is committed and never rendered (F10). One line of user config; -- `placement.default` set on a kustomize folder does the same thing to every type at once, which is - the same bug with a bigger blast radius; -- "where do my files go" is answered by simulating a four-rung ladder against a folder, which is why - it needed a metric and a status field to be legible at all; -- nothing in the model can **create** structure, so a repository that needs a `kustomization.yaml` - before it can render cannot be bootstrapped by the thing that writes into it. - -The primitive is wrong. A path template cannot express "beside this folder's one kustomization", -cannot be read at a glance, and cannot bring a folder into existence. What a user wants to declare is -what the folder **is**. - -## The model - -One field, a discriminated union on `kind`, plus per-type overrides that are valid under every kind. - -```yaml -apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget -spec: - providerRef: - name: platform - branch: main - path: clusters/prod - layout: - kind: Kustomize # Auto | Kustomize | Tree | Flat | Template - scope: SingleNamespace # SingleNamespace | MultiNamespace - writeNamespace: Never # FromContext | Always | Never - kustomize: - root: . # relative to spec.path; where the kustomization.yaml lives - create: true # write it if absent, so an empty repo becomes a buildable folder - fileName: "{kindLower}-{name}" # optional; the default - byType: # optional overrides, valid under every kind - v1/secrets: "secrets/{name}{sensitiveSuffix}" -``` - -| `kind` | Where a new document goes | Exists because | -|---|---|---| -| `Auto` (default) | one supported kustomization in the subtree: `Kustomize`; else exactly one namespace in scope: `Flat`; else `Tree`. Resolved once and pinned (see below) | it is what most folders want, and it is honest about reading the repository | -| `Kustomize` | beside the declared root, registered in its `resources:` list | a file that root cannot reach is never applied | -| `Tree` | the built-in `{namespaceOrCluster}/{groupPath}/{resource}/{name}` path | identity-complete by construction; right for a fleet folder | -| `Flat` | `{kindLower}-{name}.yaml` at the root of `spec.path` | the legible single-namespace folder people hand-author | -| `Template` | the declared `default` template | the escape hatch. Everything expressible today stays expressible | - -Two rules complete it, and they are where the value is: - -1. **Whatever chose the path, the file must be reachable.** If any kustomization governs the - destination, the write registers the file in it, in the same commit. This is an invariant of the - layout rather than a special case of one rung, which is what turns F10 from a bug into something - the model cannot express. -2. **A structural kind and a blanket template are mutually exclusive.** `default` is valid only under - `kind: Template`. You cannot ask for a kustomize folder and a nested canonical tree at the same - time. - -## What the model makes unstatable - -This is the headline, and it is worth more than the readability. - -| Today | Under the model | -|---|---| -| `placement.default` on a kustomize folder silently disables the render root and produces unrendered files | unstatable: `default` requires `kind: Template`, and `Template` asserts there is no structural rule | -| `byType` into a subdirectory produces a file no kustomization lists | unstatable: registration is an invariant, not a rung | -| Defaulting the fallback would shadow the structural rung | dissolved: the default is `kind: Auto`, which **names** the structural rule instead of standing in front of it | -| "Which of four rungs answered?" needs a metric to be legible | one word in the spec, and status says what `Auto` resolved to | -| An empty repository cannot be given the structure it needs to render | `kind: Kustomize` with `create: true` | - -The defaulting problem is the interesting one. It was never that defaults are bad; it was that we -tried to default a **path**, and a path is the one thing that cannot say "look at the folder". -`kind: Auto` is a safe CRD default precisely because it is a mode: it declares that the folder will be -read, which is the difference between this and the sibling inference we deleted. That inference was -undeclared. This one has a name, appears in the spec, and reports what it resolved to. - -## Namespace scope belongs to the layout - -A folder that omits the namespace from its paths is a folder for **one** namespace. That is what makes -`Flat` legible, and it is an assumption the layout has to carry, because the thing that would otherwise -carry it lives somewhere else. - -```yaml -spec: - allowedSourceNamespaces: # AUTHORIZATION: who may be mirrored here - names: [team-a] - layout: - kind: Flat - scope: SingleNamespace # STRUCTURE: what shape this folder has -``` - -**These are two different questions and they must agree.** `allowedSourceNamespaces` is a permission -bound owned by the destination -([`gittarget_types.go`](../../api/v1alpha3/gittarget_types.go)); `layout.scope` is a structural claim. -Validation checks the direction it can, at admission: a matcher that admits more than one namespace -alongside `scope: SingleNamespace` is refused. - -**Why the layout cannot derive it.** Two reasons, and the second is the one that matters: - -- `allowedSourceNamespaces` is an upper bound and may be **absent**, which - [`NamespaceMatcher`](../../api/v1alpha3/namespace_matcher.go) defines as "no policy declared" rather - than as "one namespace". There is often nothing to derive from. -- the namespaces that do arrive come from **N WatchRule objects that do not own the folder**. A - derived single-namespace assumption could be invalidated later by a rule created in another object, - which would turn a layout guarantee into a path collision. Declaring the scope makes that - invalidation a **refusal** instead: a document from a second namespace is declined with a message - naming both namespaces and counted as a placement refusal, rather than landing on a path another - object already occupies. - -That is the same distinction the whole redesign rests on. Reading the world is fine when the reading -is declared and its failure is loud; it is not fine when it silently re-decides. - -## Whether the namespace is written into the file - -Today this is **inferred**, and it cannot be inferred in the case bootstrapping cares about most. -`namespaceIsInheritedFromContext` omits `metadata.namespace` exactly when the governing kustomization's -`namespace:` equals the resource's own. An empty folder has no kustomization to read, so a folder we -are about to create cannot inherit a convention that does not exist yet. - -```yaml -layout: - writeNamespace: FromContext # FromContext (default, today's behavior) | Always | Never -``` - -| Value | Meaning | When it is valid | -|---|---|---| -| `FromContext` | omit when the governing kustomization sets this resource's namespace | always; today's behavior | -| `Always` | always write `metadata.namespace` | always; the only safe choice when nothing downstream supplies it | -| `Never` | never write it | only when something guarantees the namespace: a kustomization we control with `namespace:` set, or a declared downstream supplier such as a Flux `Kustomization.spec.targetNamespace` | - -`Never` needs that guard because omitting the namespace hands the object to whatever namespace the -applier happens to be pointed at, which is a different object with the same name. - -**And this is where bootstrapping closes its own loop.** `kind: Kustomize` with `create: true` and -`scope: SingleNamespace` lets the operator write `namespace: team-a` into the `kustomization.yaml` it -creates, and then legitimately omit `metadata.namespace` from every file it places. The convention is -**established** rather than guessed, which is the thing inference structurally cannot do on an empty -folder. - -## The layout is immutable, with one widening exception - -An earlier draft of the wave document put `layout` with `prune` as a mutable field. That was wrong. - -**Existing files never move**, so a mutable `kind` leaves a folder that is permanently half one layout -and half another, with nothing in the folder recording which file came from which. The structure of a -folder should be a property of the folder, not of the last edit to an object. - -**The cost of immutability is lower than it looks, and this is the fact that decides it:** `GitTarget` -has **no finalizer**, so deleting one leaves the folder in Git untouched, and re-creating it at the same -path re-adopts every document by identity (match-first). Changing a layout by recreating the object -costs the object's status and a moment of mirroring. It does not cost data. That is a materially -different bargain from `spec.prune`, where the review's argument for mutability was that a -delete-and-recreate would destroy the one thing that cannot be rebuilt. - -**The exception is widening.** `Flat` to `Tree` cannot break the folder: old flat files stay and remain -match-first, new files get identity-complete paths, and nothing collides. Narrowing (`Tree` to `Flat`) -is what can put two namespaces' objects on one path. So the CEL rule is "immutable except a transition -that cannot lose the identity-completeness the folder already had", which in practice means you may -widen and may not narrow. - -### `Auto` resolves once and is then pinned - -Immutability of the *field* does not pin the *resolution*, because `Auto` says "look at the folder". If -someone deletes the `kustomization.yaml`, `Auto` would silently become `Tree` and the folder would grow -a second layout without any object changing. That is the defect this project spent a release deleting, -re-entering through the default value. - -So `Auto` resolves on first observation, `status.layout.kind` records what it became, and a later -folder state that would resolve differently raises a condition rather than re-laying-out the folder. -Declared inference is fine. Silent re-decision is not. - -That also settles the earlier open question about whether `Auto` should be the default at all: it can -be, because pinning removes the harm, and it keeps the quickstart to four fields. - -## Examples - -### 1. `Auto` on a brownfield kustomize repo - -No layout declared, so the CRD default applies. The subtree has one supported `kustomization.yaml`. - -```yaml -spec: - path: clusters/prod - # layout: {kind: Auto} <- defaulted, so the field is visible in the object -``` - -A new ConfigMap `cache` in `team-a` lands at `clusters/prod/configmap-cache.yaml` and is added to -`resources:`. `metadata.namespace` is omitted if and only if the kustomization's `namespace:` is -`team-a`. Status reports what `Auto` became: - -```yaml -status: - layout: - declaredKind: Auto - kind: Kustomize - renderRoot: . - renderRootReason: SingleKustomization - observedRevision: 9f3c1ab -``` - -### 2. `Kustomize` with `create: true` on an empty repository - -The bootstrapping case. The folder does not exist yet, so nothing can be inferred from it. - -```yaml -spec: - path: clusters/prod - layout: - kind: Kustomize - kustomize: - create: true -``` - -The first write commits a folder that builds, rather than a file that happens to be YAML: - -```text -clusters/prod/ - kustomization.yaml # created, listing what was written - configmap-cache.yaml -``` - -```yaml -# clusters/prod/kustomization.yaml -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: - - configmap-cache.yaml -``` - -`kubectl apply -k clusters/prod` works on the first commit. This is the part the current model cannot -do at all, and it is the difference between mirroring into a directory and producing a GitOps folder. - -### 3. `Tree`, declared - -A fleet folder holding many namespaces, where a flat layout would collide names. - -```yaml -spec: - layout: - kind: Tree -``` - -`clusters/prod/team-a/apps/deployments/api.yaml`. Identity-complete by construction, so two -namespaces holding `api` never share a file. Naming it in the spec is the point: it is a choice now -rather than what is left when nothing matched. - -### 4. `Flat`, the layout you asked for by name - -```yaml -spec: - allowedSourceNamespaces: - names: [team-a] - layout: - kind: Flat - scope: SingleNamespace - writeNamespace: Always # nothing here supplies the namespace, so the file must carry it -``` - -`clusters/prod/deployment-simon.yaml`. Legible, and the reason it cannot be the built-in is that it is -**not identity-complete**: two namespaces with a Deployment named `simon` render one path. - -**`Flat` is also the kind that may not omit the namespace**, and this is the table above applied rather -than an exception to it. `Never` is legal only when something guarantees the namespace, and under -`kind: Flat` there is no kustomization for us to write `namespace:` into: a flat directory has no build -step at all. So the only guarantor left would be a downstream applier pointed at the right namespace, -which is a promise made outside this object and invisible to it. `Always` is the honest setting here. -An earlier draft of this example wrote `Never` with the comment "the build supplies it, or the applier -does", which is exactly the hand-wave the `Never` guard exists to refuse. - -As a *kind* rather than a template, that is checkable instead of a caveat in prose. `Flat` requires -`scope: SingleNamespace`, admission refuses it beside an `allowedSourceNamespaces` matcher that admits -more than one namespace, and a document arriving from a second namespace is refused at the write -boundary with a message naming both. A template can only document the hazard; a kind can decline it. - -### 5. `Template`, the escape hatch, with the invariant still in force - -```yaml -spec: - layout: - kind: Template - default: "{namespace}/{resource}.yaml" # a per-namespace bundle per type - byType: - v1/secrets: "secrets/{namespace}/{name}{sensitiveSuffix}" -``` - -Everything today's `placement` can express, this can express. The difference is rule 1: if a -kustomization governs `team-a/configmaps.yaml`, the file is registered in it. Today the same -declaration produces an unrendered file (F10) whenever the governing kustomization is an ancestor -rather than a sibling. - -### 6. `byType` under a structural kind - -Overrides are not exclusive to `Template`, because "kustomize folder, but Secrets in their own -directory" is an ordinary thing to want. - -```yaml -spec: - layout: - kind: Kustomize - byType: - v1/secrets: "secrets/{name}{sensitiveSuffix}" -``` - -A Secret goes to `clusters/prod/secrets/db.sops.yaml`, and rule 1 registers it in the root's -`resources:`. What is refused is a blanket `default` here, because that would be an assertion that the -folder has no structural rule alongside an assertion that it has one. - -### 7. Two roots, and the answer ambiguity has been missing - -```yaml -spec: - path: clusters # holds overlays/staging AND overlays/production - layout: - kind: Kustomize -``` - -The user asserted a single-root folder. The folder disagrees, so this is a **misconfiguration of the -GitTarget**, not a placement puzzle: `Validated=False`, naming both roots, with the fix being one -GitTarget per overlay. Under `kind: Auto` the same folder resolves to `Tree`, and status says -`renderRootReason: Ambiguous` so nobody has to guess why files stopped landing in the overlays. - -This is why the refuse-or-write question was so hard to settle in the current model: without a -declaration there was nothing to contradict. With one, refusing is not a policy preference, it is -honoring what the user said. - -## Bootstrapping, and where it stops - -`create: true` is deliberately narrow: **the layout may create only what its own invariant requires.** -For `Kustomize` that is exactly one file, the `kustomization.yaml` the layout claims exists, plus the -`resources:` entries for what we write. - -Everything else people mean by "bootstrap a GitOps repo" is a different concern: per-environment -directories, an app-of-apps root, a Flux `Kustomization`, a README, a `.sops.yaml`. The last two we -already write ([`bootstrapped_repo_template.go`](../../internal/git/bootstrapped_repo_template.go)), -which is a useful precedent and also a warning: that mechanism is per-path bootstrap staging, and it -is where a "repository template" belongs if we ever grow one. Folding a folder skeleton into -`spec.layout` would make the layout responsible for the shape of a repository it does not own. - -So the boundary is: the layout creates what it needs to be true. A repository template, if it ever -exists, is a separate object with a separate lifecycle. - -## Should the layout be its own CRD? - -The reuse instinct is real: a platform team with thirty GitTargets should not paste the same four -lines thirty times. Three shapes, and I would not build the second one yet. - -### The argument for a `LayoutProfile` - -- **Reuse.** One house layout, referenced by every target. -- **Guardrails.** A cluster-scoped profile owned by the platform team, referenced by namespaced - GitTargets, is an RBAC boundary: tenants pick a layout, they do not invent one. -- **One place to change it.** A fleet-wide layout change becomes one edit. - -### The argument against, which I find decisive today - -**A shared object that changes where N folders write, with nothing on the GitTarget recording it, is -structurally the same defect this project spent a release deleting.** Sibling inference was -removed because a human's edit to a repository changed the operator's behavior with no Kubernetes -object changing and nothing in status recording the move. A profile edited in another namespace is -that same shape with a different actor: the GitTarget that owns the folder is unchanged, unreviewed, -and yet its next new file lands somewhere else. Being an API object rather than a folder makes it -auditable, which is better, but it does not make it *local*, and locality is what made the deletion -worth doing. - -The rest is ordinary cost, and it is not small: - -- **A third place to look.** The redesign exists to make "where do my files go" answerable from the - object. A `layoutRef` re-splits the answer across two objects, plus status to reconcile them. -- **Another readiness chain.** `GitProvider` already teaches this: a missing or invalid reference is a - new `Ready=False` mode, and a profile deleted while targets reference it needs an answer (freeze the - last resolved layout, or stop writing). -- **Cross-namespace authorization, again.** A namespaced profile referenced across namespaces needs - what `ClusterProvider` needed: an `allowedNamespaces` selector and a fail-closed SAR. That was - expensive to build and is expensive to keep correct. -- **The thing being shared is four lines.** `kind: Kustomize` plus two options does not carry enough - weight to justify an object. The reuse pressure is concentrated entirely in one place: a large - `byType` map. That is worth remembering, because it means if we ever do share something, we should - share **the type map**, not the kind. -- **Reuse is already solved one layer up.** Whatever creates thirty GitTargets (Helm, kustomize, a - Flux `ResourceSet`) repeats four lines for free, and it does so in a place the user already reviews. - A CRD that exists to avoid repetition in generated YAML is solving a problem the generator does not - have. - -### The middle shape, if evidence demands it - -Keep the layout inline and authoritative, add an optional `layoutRef` whose **resolved content is -projected into `GitTarget.status`**, stamped with the profile's `generation`. Reuse without -invisibility: the target still shows what it is doing, and a fleet-wide change is observable per -target rather than only at the profile. - -**Recommendation: inline now.** Revisit when someone has a fleet-wide layout they want to -change centrally, and revisit for the `byType` map first, since that is the only part that grows. The -trigger is written down so this is a decision rather than an omission. - -## Status under the model - -The two-halves rule from the other document still applies: a **current** half derived from the last -repository scan and stamped with the revision it came from, and a **historical** half accumulated -since. Placement is sparse, so the current half must never depend on a placement having happened. - -```yaml -status: - layout: - # current, from the last scan - declaredKind: Auto # what the spec says - kind: Kustomize # what it resolved to - renderRoot: . - renderRootReason: SingleKustomization # SingleKustomization | Ambiguous | None - byTypeEntries: 1 - observedRevision: 9f3c1ab - observedTime: "2026-07-30T09:14:22Z" - # historical, since observedRevision - placedResources: 14 # appends included - overriddenTypes: 1 # types a byType entry routed - refusedResources: 0 # resources NOT mirrored - examples: - - type: v1/secrets - path: clusters/prod/secrets/db.sops.yaml - source: ByType -``` - -`declaredKind` beside `kind` is the pair that makes `Auto` honest: it says both what was asked for and -what the folder produced, so declared inference never looks like a decision the user made. - -## Metrics under the model - -`placements_total` keeps `source` (which mechanism produced the path) and gains `layout` (the resolved -kind), because they answer different questions and both are one label: - -```promql -# is any target's declared layout not the one its folder produces? -sum by (gittarget_name, layout) (increase(gitopsreverser_placements_total[24h])) -``` - -`source` values follow the model: `byType`, `layout` (the kind's own rule), and nothing else. The -current `kustomize_root` and `canonical` values become `layout` with the kind in the other label, and -`declared`/`default` collapse into `byType` plus `Template`. That is a metric-label break, and it is -free while nothing consumes them. - -## Migration - -Mechanical, and every current configuration has an exact image: - -| Today | Under the model | -|---|---| -| no `spec.placement` | `layout: {kind: Auto}` (defaulted). Same behavior | -| `placement.byType` only | `layout: {kind: Auto, byType: {...}}`. Same behavior, plus rule 1 fixing F10 | -| `placement.default` set | `layout: {kind: Template, default: "..."}`. The declared default no longer shadows the render root, because the kind now says there is no structural rule | -| `placement.byType` + `default` | `kind: Template` with both | - -The one behavior change is the good one: a declared template stops silently disabling the render root, -and starts registering its files instead. - -`spec.placement` becomes a loud rejection for one release rather than a silent alias, following the -pattern `ClusterWatchRule.spec.rules[].scope` set: refusing a stored field the user can see beats -translating it behind their back. It rides the Tier 2 breaking wave in -[`open-asks-priority.md`](open-asks-priority.md), so the consumer pays one coordinated bump. - -## What this changes about the work already queued - -Nothing already planned is wasted, and one item should wait: - -- **F10's ancestor walk** is rule 1's implementation. Build it now; the model makes it an invariant - rather than a fix. -- **`{kindLower}`** is `Flat`'s file name and a `Template` variable. Build it now. -- **The `{version}` identity fix** is needed by `Flat`'s validation and by any versionless template. - Build it now. -- **Canonical as a template constant** is `Tree`'s implementation. Build it now. -- **`status.layout`** is the same field, one release early, and `declaredKind`/`kind` slot into it. - Build it now. -- **The ambiguity policy** should wait. In this model it follows from whether the user asserted a - root, and deciding it before the model exists would bake in an answer to a question the model asks - differently. - -## Open questions - -- ~~Is `Auto` the right default?~~ **Yes**, given that it resolves once and pins the result. Pinning - is what removes the harm, and the quickstart stays four fields. -- ~~Should `Flat` be refused for a multi-namespace target?~~ **Refuse the writes, not the target.** A - scope-widening edit in another object must not break a folder; the second namespace's documents are - declined with a counted refusal naming both namespaces, and the fix is a widening layout change. -- Should `scope` be **derived and materialized** at creation (write `SingleNamespace` into the spec - when exactly one namespace is admitted) rather than declared? It would make the common case - zero-config, at the price of a mutating webhook writing spec, which this project has deliberately - avoided outside identity capture. -- Does `writeNamespace: Never` need to name its supplier (`Kustomize`, `FluxTargetNamespace`, - `Asserted`) so validation can check the guarantee rather than trust it? -- **Where does the namespace VALUE come from?** `scope: SingleNamespace` constrains cardinality; it - does not say *which* namespace, and the bootstrap case is exactly the one that cannot read it off - existing resources. Writing `namespace: team-a` into a `kustomization.yaml` we create presupposes - knowing `team-a`. The obvious source is `allowedSourceNamespaces.names` when it admits exactly one, - which every example here implicitly assumes, and that is worth making explicit rather than implied: - it would mean `create: true` plus `SingleNamespace` is refused when the admitted set is a selector - or is empty, because there is no name to write. The alternative — take it from the first document - that arrives — makes the folder's convention depend on arrival order, which is the failure mode this - whole model exists to remove. -- Does `kind: Kustomize` imply `create: true`? Asserting a folder is a kustomize folder arguably - asserts the file exists, and requiring both feels like ceremony. The argument for keeping them - separate is that creating a file in someone's repository should always be something they asked for. -- Should `Tree` remain identity-complete-by-definition, or become configurable (with or without the - version segment)? The versionless decision is deliberate and this document does not reopen it. diff --git a/docs/design/open-asks-priority.md b/docs/design/open-asks-priority.md index ae319604..04ca19ed 100644 --- a/docs/design/open-asks-priority.md +++ b/docs/design/open-asks-priority.md @@ -7,12 +7,13 @@ > **Where this stands as of the sweep.** `0.41.0` is the attribution release: the fact stream, the > sticky removal pointer, the metric relabel, the name tier, and the analyzer/encoder corrections, > plus PR #291's sibling-inference deletion and placement counters. That is a large breaking release -> on its own, and **the GitTarget work is deliberately not in it**. The layout model and the API wave -> ([`gittarget-layout-model.md`](gittarget-layout-model.md), -> [`gittarget-api-wave.md`](gittarget-api-wave.md)) are postponed to a later deployment and tracked as -> GitHub issues, so the queue below is read with one standing caveat: **every Tier 2 entry that -> changes a `GitTarget` field is now part of that postponed wave, not independently schedulable.** -> The Tier 1 entries are not, and should not wait for it. +> on its own, and **the GitTarget work is deliberately not in it**. The standing caveat has since +> narrowed, because [`../layout/model.md`](../layout/model.md) reversed and the placement work is no +> longer breaking: **a Tier 2 entry belongs to the postponed wave only if it changes a `GitTarget` +> field in a breaking way**, and [`../layout/api-wave.md`](gittarget-api-wave.md) is the one place +> that lists which those are. Everything else here — the placement fields, `suspend`, +> `status.placement`, and every Tier 1 entry — is additive and independently schedulable, and should +> not wait for a bump. > > **The queue was built bottom-up rather than top-down.** Tier 0 and Tier 1 are still unbuilt, and > the Tier 2 item nobody scheduled — the attribution fact stream — shipped anyway, together with a @@ -22,9 +23,9 @@ > a fact about the last week, not a revision of the rule. > > Three backlogs are open at once and they overlap: the gitops-api consumer asks (revision 11, -> 2026-07-28, which is the revision that filed #23), the maintainer review's unbuilt block in -> [`flux-maintainer-review-status-and-config-model.md`](../future/flux-maintainer-review-status-and-config-model.md) -> (F6, F9, F10), and the config-surface proposal in +> 2026-07-28, which is the revision that filed #23), the API-surface block left unbuilt by the +> status and configuration-model review — now sequenced in +> [`../layout/api-wave.md`](gittarget-api-wave.md) — and the config-surface proposal in > [`config-surface-for-a-structured-repository.md`](../future/config-surface-for-a-structured-repository.md) > (B1–B6). This page merges them into one queue and says where we deliberately do **not** do > what was asked. @@ -68,7 +69,7 @@ do not own. > **Built.** `resolveInferred` through `allSameDir` are gone, the kustomize-root fallback stayed, and no > enum was added. What building it added to the argument below is recorded in > [what the deletion taught](#what-the-deletion-taught). The spec's Option C sections are retained as -> history in [`gittarget-new-file-placement-rules.md`](../spec/gittarget-new-file-placement-rules.md), +> history in [`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md), > and the behaviour change has a [`docs/UPGRADING.md`](../UPGRADING.md) entry. The config-surface proposal's **B3** offers `spec.placement.mode: Infer|Declared|Strict`: an @@ -77,7 +78,7 @@ cohort ladder entirely**, keep the kustomize-root fallback, and ship no enum at ### What inference is, precisely -[`gittarget-new-file-placement-rules.md`](../spec/gittarget-new-file-placement-rules.md) Option C. +[`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md) Option C. It fires **only** for a resource that has no document in Git yet; everything already written is match-first and never moves. For that narrow case it finds the largest cohort of similar existing documents (step 1: same type + namespace; step 2: same type, any namespace) and puts the new @@ -236,25 +237,28 @@ and is not independently schedulable. |---|---|---|---|---| | 15 | A declared `auditRoute` with zero facts must say so, and a route losing them with it | gitops-api | **1** | — | | n/a | Stop paying a full grace for a delete fact that will never arrive (F, then C) | [`attribution-removal-wait-options.md`](attribution-removal-wait-options.md) | **1** | — | -| n/a | A declared path in a kustomize subdirectory is never rendered; the identity gate rejects the versionless canonical path | [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md) | **1** | [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) | -| n/a | `spec.layout`: declare what the folder is | [`gittarget-layout-model.md`](gittarget-layout-model.md) | **2** | [#293](https://github.com/ConfigButler/gitops-reverser/issues/293), wave | -| F6 | `spec.suspend`, `spec.interval`, `requestedAt` | maintainer review | **2** | wave | +| F9 | The `scope: Namespaced` status-write envtest | maintainer review | **1** | outside the wave, and **gates its planning**: the answer decides whether the narrowed enum can be kept ([`../layout/api-wave.md`](gittarget-api-wave.md)) | +| ~~n/a~~ | ~~A declared path in a kustomize subdirectory is never rendered; the identity gate rejects the versionless canonical path~~ **SHIPPED** in 0.42.1 | [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md) | — | [#295](https://github.com/ConfigButler/gitops-reverser/issues/295), [#319](https://github.com/ConfigButler/gitops-reverser/pull/319) | +| n/a | `useKustomize` and `serializeNamespace`: the two things a path template cannot say (`spec.layout` was reversed) | [`../layout/model.md`](../layout/model.md) | **2** | [#322](https://github.com/ConfigButler/gitops-reverser/issues/322), **not** breaking, so not the wave | +| F6 | `spec.suspend`, `GitProvider.spec.interval`, `requestedAt` (no `interval` on `GitTarget`, see [`../layout/api-wave.md`](gittarget-api-wave.md)) | maintainer review | **2** | wave | | 5 | `CommitRequest.spec.author`, SAR-guarded | gitops-api (#220) | **2** | wave | | B4 | `commitWindow` / `commit.message` move to GitTarget | config surface | **2** | wave | -| B1 | `GitTarget.spec.mode: Observe\|Write` | config surface | **2** | wave | +| ~~B1~~ | ~~`GitTarget.spec.mode: Observe\|Write`~~ **dropped**: `suspend` on a still-scanning target is the same dry run with one field | config surface | — | [`../layout/api-wave.md`](gittarget-api-wave.md) | | 6 | Movable destination via `status.observedDestination` | gitops-api (#220) | **2** | wave | | F10 | CommitRequest TTL / ownerRef + the `delete` verb | maintainer review | **2** | wave | | n/a | The blocking resolve is head-of-line on the shard goroutine | [`../spec/attribution.md`](../spec/attribution.md#the-wait) | **2** | — | -| B2 | `GitTarget.status.layout` | config surface | **3** | [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) | +| B2 | `GitTarget.status.placement` (was `status.layout`) | config surface | **3** | [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) | | n/a | The ambiguous render root, the `declared` metric split, `{kindLower}`, canonical-as-template | [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md) | **3** | [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) | -| F9 | The `scope: Namespaced` status-write envtest | maintainer review | **3** | outside the wave, deliberately | | B6 | The `default` ClusterProvider not-found message | config surface | **3** | — | | n/a | An aggregated create carries no name and no body: accept it, or stop waiting for it | [`../spec/attribution.md`](../spec/attribution.md#what-the-shape-driven-rules-reach-and-what-they-do-not) | **3** | — | | n/a | Entry-size ceiling and per-type stream count under a few hundred watched types | [`attribution-fact-stream.md`](../finished/attribution-fact-stream.md) | **3** | — | | 10 | Namespace-aware sibling inference *as asked* | gitops-api | **declined — answered by the deletion, SHIPPED** | — | | B3 | `spec.placement.mode` enum | config surface | **declined** | — | -**One entry moved up in this sweep.** The declared-path-in-a-subdirectory bug is Tier 1, not Tier 3, +**Two entries moved up in this sweep.** F9 is Tier 1 because it is not merely one envtest: until it +is run, nobody can plan the enum work, and the answer can force a design change in an object that is +already shipped. An unmeasured fact that gates other people's planning outranks a legibility item. +The declared-path-in-a-subdirectory bug is Tier 1, not Tier 3, under this page's own first test: one line of ordinary user configuration silently produces a file that is in Git and rendered by nothing, and nothing in status or in the counters says so. That is the product being silently wrong, which is what Tier 1 is for. It was written down as a finding rather than @@ -428,6 +432,14 @@ resolves nothing. What replaced it is a declaration plus `placements_total{source="canonical"}` per (GitTarget, type), so the same class of defect now has a query. `#10` and `B3` are answered by it and stay declined. +**F9: the `scope: Namespaced` status-write envtest.** In this tier because it is the only item on +the page whose *answer is unknown* rather than whose work is unscheduled, and because everything +downstream assumes an answer. If the apiserver validates the whole object on a status-subresource +write, the narrowed `ClusterWatchRule` enum leaves the one object that most needs to explain itself +unable to write its own `Stalled` condition. The test, the version it has to name and the fallback +are in [`../layout/api-wave.md`](gittarget-api-wave.md). Run it before planning anything that +depends on it. + ### Tier 2: the breaking wave, all at once, while `v1alpha3` These all add or change a spec field. Doing them as one `feat(api)!` sequence costs the consumer @@ -495,7 +507,10 @@ carry, and an aggregated-API create is logged with no name and no response body `#220` shape — honored only against an admission record carrying an authorized verdict, fail-closed independent of the webhook's `failurePolicy` — remains the right one, on the first argument alone. -**B4, B1, #6, F10** as written in their source documents. #6 is explicitly a lower priority than +**B4, #6, F10** as written in their source documents. **B1 has left the wave**: a suspended +`GitTarget` that keeps scanning is the same dry run with one field instead of two, so `mode` buys +only the difference between a pause and a declared posture. The re-open trigger is in the wave +document. #6 is explicitly a lower priority than when it was filed: the consumer downgraded it themselves, because branch and folder are now chosen once per repository on an object that exists because the user picked that repository. It rides the wave because it is in the wave, not because it is urgent. @@ -540,8 +555,8 @@ not alternatives, and B2 should carry the per-target record of which types resol which fell back — which is why this is worth doing in the same change as the deletion rather than after it. -**F9, B6** as filed. F9 is one envtest; B6 is one error message that will otherwise be the most -likely first-run support ticket. +**B6** as filed: one error message that will otherwise be the most likely first-run support ticket. +F9 has moved to Tier 1. **The aggregated create: decide, and the decision is small either way.** The name tier reaches an aggregated update, patch and single delete. It cannot reach a create: the `objectRef` carries no name @@ -581,7 +596,7 @@ defects. fall-back-to-canonical Event.~~ **Done for the removal and the entry**; the Event is still undecided, and the metric now carries the actionable part in the meantime. 3. ~~Rewriting Option C's sections in - [`gittarget-new-file-placement-rules.md`](../spec/gittarget-new-file-placement-rules.md).~~ + [`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md).~~ **Done**: the ladder is documented as three steps, the kustomize-root fallback keeps its section and gained the namespace-match rule, and P1–P10 are annotated one by one with which are retired by the deletion and which (P7, P9, P10) are facts about the code that remains. diff --git a/docs/design/placement-visibility-and-declared-defaults.md b/docs/design/placement-visibility-and-declared-defaults.md index 7ef490e3..cb41794b 100644 --- a/docs/design/placement-visibility-and-declared-defaults.md +++ b/docs/design/placement-visibility-and-declared-defaults.md @@ -13,7 +13,7 @@ > Reading it in order matters, because the two halves have different standing. The **findings** are > checkable facts about the tree and they are why the calls are what they are. The **calls** are still > the calls. Question 2 (a CRD default for `placement.default`) has since been superseded outright by -> [`gittarget-layout-model.md`](gittarget-layout-model.md), which argues that a path template is the +> [`model.md`](../layout/model.md), which argues that a path template is the > wrong primitive to be defaulting at all. Three questions came out of reviewing #291, and one of them (a CRD default for @@ -103,7 +103,7 @@ fixing F4. `MarkTargetRetention` records a fact from the write path into an epoch-scoped per-target roll-up and calls `enqueueGitPathChange` **on a change only**, and the controller projects it in `gitTargetRetentionStatus`. This retires the objection recorded in -[`open-asks-priority.md`](open-asks-priority.md) that placement facts cannot reach the GitTarget +[`open-asks-priority.md`](../design/open-asks-priority.md) that placement facts cannot reach the GitTarget promptly because "a refusal recorded on the data plane does not enqueue the GitTarget". One does already. `status.retention` is proof. @@ -251,7 +251,11 @@ against `ToGitPath()` byte-for-byte, and solve F9 first, because nothing else on until a defaulted value can coexist with the render-root step. Status is worth having either way, so nothing built now is wasted. -## `status.layout` +## `status.layout`, renamed `status.placement` + +> **The field is `status.placement` now.** [`model.md`](../layout/model.md) renamed it when +> `spec.layout` stopped existing, and specifies it under that name. The shape below is unchanged +> and this page is still where it is argued; read every `status.layout` here as `status.placement`. An **observation, not a condition**, in the sense [`GitTargetStatus.Retention`](../../api/v1alpha3/gittarget_types.go) already establishes: nothing @@ -496,14 +500,14 @@ because they are worth different urgency: | Item | State | Why it is where it is | |---|---|---| -| F10: register a declared path with the kustomization that governs it | **filed, correctness** | One `byType` line into a subdirectory silently produces a file nothing renders. Reachable today | -| Drop the `{version}` requirement from `IdentityCompletePlacementTemplate` | **filed, correctness** | It contradicts the versionless-path decision, and it is what makes any future spec default fail our own gate | +| F10: register a declared path with the kustomization that governs it | **SHIPPED** in 0.42.1 (#319) | One `byType` line into a subdirectory silently produced a file nothing renders. The fix made registration an invariant, which is what reversed [`model.md`](../layout/model.md) | +| Drop the `{version}` requirement from `IdentityCompletePlacementTemplate` | **SHIPPED** in 0.42.1 (#319) | It contradicted the versionless-path decision, and it was what made any future spec default fail our own gate | | `status.layout` | **filed** | The durable half of "what did the operator understand about this folder". Wants the layout model's vocabulary, so it follows it | | Split `declared` into `byType` and `default`; unify the prose on "canonical" | **filed, legibility** | A catch-all quietly swallowing a type you meant to name looks identical to a rule working | | `{kindLower}` | **filed, legibility** | Small, self-contained | | Canonical path as a template constant | **filed, cleanup** | Removes the hand-written duplication; what a future default would reuse | | `renderRootReason: Ambiguous` | **filed** | Belongs with `status.layout`, and the layout model decides the refuse-or-write policy | -| A CRD default for `placement.default` | **superseded** | [`gittarget-layout-model.md`](gittarget-layout-model.md): the primitive is wrong. `layout.kind` is the defaultable thing, because it names the structural rule instead of standing in front of it | +| A CRD default for `placement.default` | **re-opened** | The reversal took away the answer this row used to give. There is no `layout.kind`; the template stays, and now that the ancestor walk registers a defaulted path, the objection is legibility rather than correctness. It is an open question in [`model.md`](../layout/model.md) | The original build list follows, because each entry says *how* to build the thing and that is the part an issue should not have to restate. @@ -537,12 +541,15 @@ Ordered by risk, smallest first. 8. **Docs**: the spec's resolution-ladder section, `configuration.md`, `interpreting-metrics.md` for the new label values, and the `UPGRADING.md` entry extended with the metric-value split. -**Not now, with the trigger written down:** a CRD default for `placement.default`. This was already -"not now" when the page was written, and it has since become "not this shape at all", because -[`gittarget-layout-model.md`](gittarget-layout-model.md) replaces the template with a declared -`layout.kind` and `Auto` is a safe default where a path never was. The freezing question (F5) is a -trade we could take; the shadowing question is not, and the layout model dissolves it rather than -answering it. `spec.expect.layout` stays out too, on the config-surface doc's own rule: publish the +**Not now, with the trigger written down:** a CRD default for `placement.default`. This was "not +now" when the page was written and it stays "not now", but for a weaker reason than the page +originally gave. An earlier revision said the layout model dissolved the question by replacing the +template with a defaultable `layout.kind`; the model has since reversed and the template stays, so +the question is live again. What changed in its favour is the shipped ancestor walk: a defaulted +path into a subdirectory is now registered with the kustomization that governs it, so F9's +shadowing objection is no longer a correctness wall. The freezing question (F5) is a trade we could +take. What is left is legibility, and that is materially weaker than the case this page refused. +`spec.expect.layout` stays out too, on the config-surface doc's own rule: publish the observation before inventing the assertion. ## Open questions diff --git a/docs/design/source-scope-simplification.md b/docs/design/source-scope-simplification.md new file mode 100644 index 00000000..558a00d2 --- /dev/null +++ b/docs/design/source-scope-simplification.md @@ -0,0 +1,247 @@ +# Source scope: what to delete, and what to keep + +> Status: design. Nothing here is built, and nothing binds until it is scheduled. +> Date: 2026-08-28. Index: [`../INDEX.md`](../INDEX.md). +> +> Answers a design review's proposal to adopt Flux-style `serviceAccountName` impersonation for +> source reads, by declining it. Evidence for the impersonation half is in +> [`../facts/kubernetes-impersonation-and-flux-identity.md`](../facts/kubernetes-impersonation-and-flux-identity.md), +> including a correction to a claim in that review. + +## Decision + +1. **Do not build impersonation.** It buys several identities behind one credential against one + source cluster. Nobody has asked for that, and the cost is four pieces of machinery and an + `impersonate` grant. +2. **Delete `GitTarget.spec.allowedSourceNamespaces`** and its selector machinery. +3. **Rename `ClusterProvider.spec.allowSourceNamespaceOverride`** to + `allowAnySourceNamespace`, still a boolean, still defaulting to false. +4. **Decide `sourceNamespace: "*"` explicitly** (see below). It is defined in terms of the field + being deleted, so it cannot be left alone. +5. **Keep `ClusterProvider.spec.allowedNamespaces`**, renamed `accessFrom`, until a replacement is + shipped and tested. It is the one field here doing work that nothing else does. + +The name keeps `Source` deliberately. `ClusterProvider` carries two namespace planes, and +`allowAnyNamespace` sitting directly beneath `accessFrom` would read as a modifier on it. +`allowCrossNamespace` was the other candidate, borrowing Flux's `--no-cross-namespace-refs` +vocabulary, and it was not taken: in Flux the phrase means object references across namespaces in +one cluster, while here the far side is a namespace in a different cluster. The field's own design +rationale already records why that matters, and it is the sharpest statement of it in the repo: for +a remote provider "the config-plane namespace and the source namespace are on different clusters, +so their sharing a name never was a boundary". Crossing is literally true only for the in-cluster +provider; `any` is literally true for both. It stays a +boolean because there are two states and no third one is in view: impersonation and source-side +selectors are both out, so an enum would only be leaving room for something nobody can name. The +review's objection to a boolean here was that it sat among "two policy objects answering +neighbouring halves"; deleting `allowedSourceNamespaces` removes the other half, and the objection +with it. + +## Why `allowedNamespaces` stays + +An earlier draft proposed deleting it, on the evidence that the chart renders +`allowedNamespaces: {selector: {}}` and so admits every namespace. That evidence is real but +narrower than it looked: it is the value for the **chart-owned `default` provider only** +([`values.yaml`](../../charts/gitops-reverser/values.yaml)). Every user-authored `ClusterProvider`, +which is every remote cluster, is deny-by-default and carries an explicit consumer list its author +wrote on purpose. + +The boundary it draws is not available anywhere else. Source-cluster RBAC bounds **what a +credential may read**. It cannot express **which control-plane tenant may wield that credential**, +because the tenant is not a subject in the source cluster at all. Deleting the field would make a +shared source credential usable from any namespace that can create a `GitTarget`. + +A `ValidatingAdmissionPolicy` is not a drop-in replacement either, and this repository has already +written down why: [`../spec/where-validation-lives.md`](../spec/where-validation-lives.md) states +that reconcile-time is the **stronger** gate, because admission is one-shot and cannot see a policy +tightened after the object was created, so "an admission-only check is strictly less safe". A VAP +can stop a new or updated `GitTarget` from selecting a provider. It cannot stop a stored one whose +namespace lost its label. Recommending a VAP here contradicted our own doctrine. + +What can be simplified without touching the boundary: the field reads **control-cluster** namespace +labels, locally, with no cross-cluster call and no degradation path, so its selector is cheap in a +way the source-side one is not. Keeping both halves is fine. Renaming to `accessFrom` is still +worth doing, and matters more once the two `allowed*Namespaces` fields no longer sit side by side +to disambiguate each other. + +## Why the rest goes + +`GitTarget.spec.allowedSourceNamespaces` presents itself as a destination policy: "it belongs to +the DESTINATION, not to any requesting rule". It cannot be one. `WatchRule.spec.targetRef` is a +`LocalTargetReference` ("Must be in the same namespace"), `GitTarget.spec.providerRef` is local +too, and `spec.path` is immutable, so the chain from a Git folder back to the object that fills it +never leaves one namespace: + +```text +Git folder <- GitProvider <- GitTarget <- WatchRule + (same namespace, all the way) +``` + +Whoever can create a `WatchRule` there can already write into that folder. What the field really +bounds is which source namespaces the folder's own tenant may **read**, and for a credential-scoped +provider that restates what the credential already carries, in the one place that cannot revoke. +`ClusterWatchRule` bypasses it entirely today and nobody has minded. + +**The complexity is in the matcher, not the fence.** `NamespaceMatcher`'s selector half is +evaluated against `Namespace` labels **in another cluster**. That one choice produces the +three-valued verdict (so a source-cluster outage is not a denial), the `SourceScopeUnavailable` +degradation path, the five condition reasons, and the operator's need for source-cluster +`Namespace` get/list/watch. Delete the source-side selector and all of it goes. + +```mermaid +flowchart LR + subgraph before["Today"] + direction TB + B1["ClusterProvider.allowedNamespaces
control-cluster labels, local read"] + B2["ClusterProvider.allowSourceNamespaceOverride"] + B3["GitTarget.allowedSourceNamespaces
SOURCE-cluster labels, cross-cluster read"] + B4["3-valued verdict, 5 reasons,
SourceScopeUnavailable path"] + B3 --> B4 + end + subgraph after["Proposed"] + direction TB + A1["ClusterProvider.accessFrom
unchanged boundary, renamed"] + A2["ClusterProvider.allowAnySourceNamespace
boolean, default false"] + A3["a string comparison"] + end + B1 ==>|"kept"| A1 + B2 ==>|"renamed"| A2 + B3 -.->|"deleted"| A3 + B4 -.->|"deleted with it"| A3 +``` + +## `sourceNamespace: "*"` needs its own decision + +Today `*` means "every source namespace this `GitTarget` admits", resolved live through +`allowedSourceNamespaces` into a concrete set, which is then planned as one stream per namespace. +[`watchrule_types.go`](../../api/v1alpha3/watchrule_types.go) says so in the constant's own doc: +"never `every namespace that exists`". So `*` is **defined in terms of the field being deleted** +and cannot survive unchanged. + +RBAC cannot supply the missing definition. It answers "may I watch X in namespace Y", never "which +namespaces may I watch". Any set-valued reading of `*` requires listing `Namespace` objects in the +source cluster, which is the cost this whole exercise is trying to remove. Three options: + +| Option | Meaning | Cost | +|---|---|---| +| **Delete `*`** | items name namespaces | safest, and a breaking change for anyone using it | +| **Redefine as cluster-wide** | one watch and one list at `metav1.NamespaceAll`, all or nothing | implementable with no `Namespace` access, and a real widening: it reaches everything the credential can see | +| **Keep enumeration** | as today, against some other policy | keeps the cross-cluster `Namespace` read, which was the point of the deletion | + +**Decided: redefine as cluster-wide**, rejected outright while `allowAnySourceNamespace` is false. +It applies to **both** halves of a cell's traffic (the initial list that warms the cell, and the +watch that follows it), because they are the same collection read two ways, and splitting them +would mean enumerating namespaces for the replay after all, which is the read this exercise +deletes. + +The plumbing already exists, which makes this the cheapest of the three as well as the clearest. +`CellKey.Namespace` is already documented as "empty is a genuinely cluster-wide (all-namespaces) +cell" ([`cell.go`](../../internal/types/cell.go)), and both +[`openTargetWatch`](../../internal/watch/target_watch.go) and `openTargetList` already branch on +it: a non-empty namespace calls `resource.Namespace(ns)`, an empty one calls `resource.Watch` +directly, which for a namespaced GVR is the all-namespaces collection. Readiness, retention +rollup, and event routing all key on `CellKey` already, and records carry the object's own +`u.GetNamespace()` rather than the cell's, so placement is unaffected. What changes is only the +planner: `*` compiles to one cell instead of calling `EnumerateSourceNamespaces` for N. That is a +deletion in `watchrule_compile.go`, not new machinery. + +One trap, and the code already records it. A cluster-wide cell is a **peer** of a named-namespace +cell on the same type, never a replacement, because each rule carries its own `operations` filter. +`CellKey`'s doc comment names the bug from a previous attempt: collapsing the two "widened the named +rule's stream to every namespace its credential could read and discarded its operation filter". So +a target carrying both `*` and a named rule for one type runs two streams over overlapping objects, +and that is correct rather than something to optimize away. +It is what a Kubernetes reader expects `*` to mean, and its failure is a clean 403 rather than a +silent empty set. + +**It is also the largest single efficiency win in this proposal, and that is a reason in its own +right.** A `*` rule over a type in a hundred-namespace cluster is a hundred watch connections and a +hundred list calls at warm-up today, one per namespace, each with its own cursor, its own retry +schedule and its own share of apiserver watch cache. Cluster-wide makes it one of each, and the +saving grows with the cluster, which is exactly the direction the enumeration got worse in. The +`TooManyStreams` cap was queued for the fan-out this deletes; see +[`../layout/api-wave.md`](gittarget-api-wave.md), where that rider is now smaller than it was. + +## Consequences, including two breaking semantic changes + +**A declared policy can currently deny a rule's own namespace.** `allowedSourceNamespaces` is +exhaustive once declared, "with no exception for a rule's own namespace" +([`gittarget_types.go`](../../api/v1alpha3/gittarget_types.go)). So `allowAnySourceNamespace: false` +is **not** the current posture exactly. It matches the no-policy path, which is what a default install runs, and it is a +deliberate simplification for anyone who declared a policy that excluded their own namespace. Say +so in the migration note rather than claiming continuity. + +**`*` changes meaning**, per the decision above: it widens from "every namespace this GitTarget +admits" to "every namespace this credential can read", and it stops being available at all unless +`allowAnySourceNamespace` is true. For a user who had no `allowedSourceNamespaces` policy the old +`*` already resolved to whatever the credential could see, so the widening is narrower in practice +than it reads; for a user who had one, it is real. + +**Label selectors over source namespaces are lost.** Admitting every namespace carrying a label, +following namespaces as they appear, has no RBAC equivalent short of a binding per namespace. This +is the real capability cost. An N-way restriction costs N objects wherever it is expressed, and +today's version is cheap only because it enforces nothing. + +**What is deleted**, 4,569 lines in files that exist for nothing else, with coupling into shared +files running 1 to 26 mentions, so this is a deletion rather than a refactor: + +| File | Lines | +|---|---| +| `internal/authz/source_namespace.go` and its test | 1351 | +| `internal/watch/source_namespace_scope.go` and its tests | 1312 | +| `internal/controller/watchrule_source_namespace.go` and its test | 770 | +| `api/v1alpha3/namespace_matcher.go` selector half | part of 371 | +| `test/e2e/source_namespace_e2e_test.go` | 349 | + +`internal/authz/clusterprovider_admission.go` stays, since `accessFrom` stays. +`ClusterProviderNotFound` stays with it, though it is reference resolution rather than +authorization and would read better elsewhere. + +## The one thing to build + +A `SelfSubjectAccessReview` pass under the provider's own credential. It needs no new grant, since +every identity may issue one, and it reports two things a user cannot otherwise get: which of the +requested cells are reachable, and whether write verbs are permitted on them. + +Phrase the condition as **"no write permission observed for the requested resources at review +time"**. It cannot prove the mirror is unable to write: a review covers the verbs and resources +asked about, at that instant, and says nothing about other resources, subresources, or a later +change. It is a diagnostic, and a good one, not a proof. + +This is also what replaces the deleted policy in the place that matters: the user stops declaring +which namespaces are permitted, and starts being told which are reachable. + +## Migration + +One breaking change, in the wave already queued, riding the loud-rejection pattern the project uses +for superseded fields: + +- `GitTarget.spec.allowedSourceNamespaces` removed. +- `allowSourceNamespaceOverride` becomes `allowAnySourceNamespace`: same type, same default, same + semantics, so the shim is a pure rename. +- `allowedNamespaces` becomes `accessFrom`, same shape, same semantics. +- `sourceNamespace: "*"` becomes one cluster-wide list and watch, per the decision above, and is + rejected while `allowAnySourceNamespace` is false. This is a semantic change to a value that + keeps its spelling, so it needs its own `docs/UPGRADING.md` paragraph rather than a shim. + +Then the deletion, then the `SelfSubjectAccessReview` work, which is additive and can ship later. + +## Re-open triggers + +- **Impersonation**, if one source cluster must serve control-plane tenants with different + authority and they cannot each hold a credential. The cost is priced in the facts note. +- **Deleting `accessFrom`**, once an external policy recipe is shipped, tested, and its lack of + live revocation is an accepted product choice rather than an oversight. +- **Source-side selectors**, if a user is found who needs "every namespace labeled X" and cannot + generate bindings. + +## Open questions + +- ~~Which `*` option is taken.~~ **Decided: cluster-wide**, list and watch both, above. +- `CellKey.String()` renders an empty namespace as a bare type name ("configmaps"), which read as + cluster-scoped when only `ClusterWatchRule` produced those cells. Once `*` produces them for + namespaced types it wants a distinct rendering, "configmaps in all namespaces" or similar, in + logs and status messages. +- Does the chart keep an `allowAnySourceNamespace` value, and does the quickstart set it true? A + homelab wants it; a platform should refuse it. +- Does anything rely on `allowedSourceNamespaces.selector` today beyond the e2e suite, which is us + testing our own feature? diff --git a/docs/design/support-boundary/finished/images-and-replicas-edit-through.md b/docs/design/support-boundary/finished/images-and-replicas-edit-through.md index 31c2a66c..edd1a7e4 100644 --- a/docs/design/support-boundary/finished/images-and-replicas-edit-through.md +++ b/docs/design/support-boundary/finished/images-and-replicas-edit-through.md @@ -10,7 +10,7 @@ > Captured: 2026-07-06 > Related: > [../README.md](../README.md), -> [contextual-namespace-and-kustomize-folder-editing.md](../../../spec/contextual-namespace-and-kustomize-folder-editing.md), +> [../../../layout/contextual-namespace.md](../../../layout/contextual-namespace.md), > [manifestedit/DECISION.md](../../../../internal/git/manifestedit/DECISION.md) ## Problem diff --git a/docs/design/support-boundary/helm-light-support-boundary.md b/docs/design/support-boundary/helm-light-support-boundary.md index 7bd85e5e..0e909944 100644 --- a/docs/design/support-boundary/helm-light-support-boundary.md +++ b/docs/design/support-boundary/helm-light-support-boundary.md @@ -130,7 +130,7 @@ and refused **by name**, the way `patchesJson6902` refuses under its own name to `patternProperties` and an unresolved reference mean for closure before any of this is built. The schema is the chart author *declaring* the input surface, a closed leaf set, which is the same declared-over-inferred principle the layout model rests on - ([gittarget-layout-model.md](../gittarget-layout-model.md)). A values key the schema + ([../../layout/model.md](../../layout/model.md)). A values key the schema does not declare refuses the folder. 3. No `dependencies:` in `Chart.yaml`, no `charts/` directory, no `Chart.lock`. The chart is one program, not a graph of them. (Vendored dependencies are a conceivable later @@ -302,7 +302,7 @@ locally, no plugins, no network, in-memory filesystem as the jail. How helm-light surfaces on the API (`layout.kind`, or a renderer property of the structural kind) belongs to the GitTarget API wave -([gittarget-api-wave.md](../gittarget-api-wave.md)), not to this boundary. +([../../layout/api-wave.md](../gittarget-api-wave.md)), not to this boundary. ## Where the arguments live diff --git a/docs/design/support-boundary/render-root-scoping.md b/docs/design/support-boundary/render-root-scoping.md index db53cc79..417cbabd 100644 --- a/docs/design/support-boundary/render-root-scoping.md +++ b/docs/design/support-boundary/render-root-scoping.md @@ -115,7 +115,28 @@ overlay rather than writing the read-only base, and the re-render adjudicates th verification. What remains is a strategic-merge patch for a base-owned field that is *not* an image, replica, or whole-object delete. -## 5. Remaining work +## 5. The repository-level peer: paths another controller writes + +Render-root scoping keeps one GitTarget out of another target's partition. The same argument +applies one level up, to folders no GitTarget should claim at all: + +> A `GitTarget` must not point at a path another controller writes. Flux's bootstrap directory +> (`clusters//flux-system`) is the common case, and the `Kustomization` that reconciles a +> folder is not a licence to co-write it. + +`flux bootstrap` and flux-operator own `clusters//flux-system`, including the +`kustomization.yaml` that lists `gotk-components.yaml` and `gotk-sync.yaml`. An operator adding +`resources:` entries there is a second writer in a folder Flux's own sync loop reconciles, which is +the two-writers-one-folder failure this boundary exists to prevent — and the fact that Flux +*renders* the folder is what makes it look eligible, not what makes it safe. Being reconciled by a +controller is the tell, not the licence. + +This is a rule for the person choosing `spec.path`, not something the operator can enforce: nothing +in the repository marks a folder as bootstrap-owned. It is stated here so the answer exists in the +support boundary rather than only in an example, and +[`homelab-flux`](../../layout/examples/homelab-flux/README.md) is the worked case. + +## 6. Remaining work 1. Add a dedicated cluster end-to-end overlay case (discovery classification is already flipped — an external-base overlay reports accepted). diff --git a/docs/design/support-boundary/repo-discovery-and-onboarding-scan.md b/docs/design/support-boundary/repo-discovery-and-onboarding-scan.md index caf5a96b..077f72bd 100644 --- a/docs/design/support-boundary/repo-discovery-and-onboarding-scan.md +++ b/docs/design/support-boundary/repo-discovery-and-onboarding-scan.md @@ -14,7 +14,7 @@ > [kustomize-support-boundary.md](kustomize-support-boundary.md), > [finished/images-and-replicas-edit-through.md](finished/images-and-replicas-edit-through.md), > [finished/higher-level-krm-documents.md](finished/higher-level-krm-documents.md), -> [../manifest/file-agnostic-placement.md](../../spec/gittarget-new-file-placement-rules.md), +> [../manifest/file-agnostic-placement.md](../../layout/new-file-placement-rules.md), > [../../finished/current-manifest-support-review.md](../../spec/current-manifest-support-review.md) ## Why this exists @@ -49,7 +49,7 @@ Repo-wide target discovery is therefore a **new axis**, not a new operator feature. Putting it in the operator would fight both the simplicity goal and the one-owner invariant. It belongs in the CLI/library, and anything built on top of the operator consumes its report — exactly the division already recorded in -[file-agnostic-placement.md](../../spec/gittarget-new-file-placement-rules.md) and the +[file-agnostic-placement.md](../../layout/new-file-placement-rules.md) and the [README](README.md) responsibilities table: | Layer | Owns | Gains repo-wide discovery? | diff --git a/docs/facts/kubernetes-impersonation-and-flux-identity.md b/docs/facts/kubernetes-impersonation-and-flux-identity.md new file mode 100644 index 00000000..7901a3eb --- /dev/null +++ b/docs/facts/kubernetes-impersonation-and-flux-identity.md @@ -0,0 +1,106 @@ +# Impersonation and service-account identity, as Kubernetes and Flux implement them + +> **facts** — durable reference. Index: [`../INDEX.md`](../INDEX.md) +> +> Read for [`../design/source-scope-simplification.md`](../design/source-scope-simplification.md), +> which decided against adopting impersonation. This page holds the evidence so the case does not +> have to be re-derived if the decision is re-opened. +> +> Verified against `k8s.io/apiserver@v0.34.1` in the module cache and the gitignored +> `external-sources/flux/` checkout. Both are named by path rather than linked, because neither is +> tracked by this repository. + +## What the API server does + +All five are from `pkg/endpoints/filters/impersonation.go`. + +**The subject does not have to exist.** An `Impersonate-User` header of +`system:serviceaccount:homelab-config:mirror` is split by `serviceaccount.SplitUsername` into a +`ServiceAccount` object reference, and nothing ever reads that object. The identity is a string, +and the namespace half need not name a namespace that exists. + +**The authorization check is namespaced and name-scoped.** The filter builds an attributes record +with verb `impersonate`, resource `serviceaccounts`, and both the namespace and the name taken from +the header. So an RBAC rule can bound impersonation with `resourceNames`. A `ClusterRole` doing so +bounds it by name only, across every namespace; bounding by namespace needs a `Role`. + +**The service-account groups are added without a check.** When no groups are requested the filter +sets `groups = serviceaccount.MakeGroupNames(namespace)`, which is `system:serviceaccounts` and +`system:serviceaccounts:`. Those are appended after the per-request authorization loop, +so no `impersonate` check runs against them. Any privilege the cluster binds to every service +account in that namespace comes along with the identity. + +**The impersonator's own permissions are not an upper bound.** The filter constructs a fresh +`user.DefaultInfo` and replaces the request user with it; the original survives only in the audit +log. There is no intersection between what the impersonator may do and what the impersonated +subject may do. + +> This last one is worth stating loudly, because the natural assumption is the opposite, and an +> earlier design review recorded that assumption as fact — that the operator's own `ClusterRole` +> remains the outer bound, so the effective permission is the intersection of what the operator may +> read and what the impersonated subject may read. It is not. The effective permission is +> exactly what the impersonated subject may do, which is why `resourceNames` scoping would be +> load-bearing rather than tidy: without it, `impersonate` on `serviceaccounts` is +> admin-equivalent in the target cluster. + +**Impersonation is per request, so it applies to watches.** The header travels on a watch request +like any other, and when the binding behind it is removed the API server ends the stream. Whoever +reads under an identity the target cluster issued gets revocation without implementing it. + +## What Flux does + +From `pkg/runtime/client/impersonator.go` and +`flux-operator/internal/controller/resourceset_controller.go`: + +- `setImpersonationConfig` builds `system:serviceaccount:%s:%s` from `i.serviceAccountNamespace`, + and every caller passes the reconciled object's own namespace + (`WithServiceAccount(r.DefaultServiceAccount, obj.Spec.ServiceAccountName, obj.GetNamespace())`). + No field anywhere lets a user write the namespace half. That is the whole security argument: a + tenant can claim only an identity bounded by the RBAC their own namespace already grants. +- `clientForKubeConfig` calls `setImpersonationConfig` too, so impersonation **composes** with a + remote kubeconfig rather than replacing it. +- `CanImpersonate` does a `Get` for the `ServiceAccount` through the local client. On the + kubeconfig path that checks the wrong cluster, and it is an existence check where an + authorization check is wanted. RFC 0010 flags it as a known bug. + +RFC 0001 states the opposite of the second point ("All accesses that would use impersonation use +the remote client instead"). The RFC predates the code. Cite the code. + +## Impersonating into another cluster + +RFC 0010 states the rule directly: with `spec.serviceAccountName`, the authenticated identity "must +have the necessary permissions to impersonate this `ServiceAccount` in the remote cluster". + +Combined with the first fact above, this means the impersonated `ServiceAccount` is resolved in the +remote cluster and nowhere else, and the namespace in the identity string comes from the local +object. The pair is a **name convention two clusters agree on**, not an object reference. The +remote cluster's admin honors it with an ordinary `RoleBinding` naming a subject whose namespace +may not exist locally to them. + +The consequence for any design that adopts this: whoever can create a namespace in the local +cluster can mint an identity claim against every remote cluster that ever granted that name. + +## Where Flux puts multi-tenancy + +Not in its API. `--no-cross-namespace-refs` and `--default-service-account` are controller flags, +and the workload-identity profile splits the latter into three by concern +(`--default-decryption-service-account`, `--default-kubeconfig-service-account`). The multi-tenant +lockdown is a documented Kyverno profile plus `flux-operator`'s +`internal/builder/profiles.go`. The one API-level exception, `acl.AccessFrom` with its +`namespaceSelectors`, is on sources and answers "which namespaces may reference this object". + +## What adopting impersonation here would require + +Recorded so the cost is not re-estimated: + +- The tenant must pick the name half and never the namespace half, or they can name + `kube-system:default`. +- `ClusterWatchRule` is cluster-scoped and has no namespace to derive from, so it needs an explicit + `serviceAccountRef`. Acceptable only because creating a cluster-scoped object is already an admin + act. +- The precheck is a `SubjectAccessReview` against the target cluster, not a `Get`, and it must + carry `system:serviceaccounts` and `system:serviceaccounts:` as groups or it answers + narrower than reality. That needs `create subjectaccessreviews` on a credential users have + already issued. +- Identity joins cluster, GVR, and namespace in the informer key, so it multiplies the resource the + `TooManyStreams` cap exists to bound. diff --git a/docs/future/config-surface-for-a-structured-repository.md b/docs/future/config-surface-for-a-structured-repository.md index 3a8f8dec..39d05181 100644 --- a/docs/future/config-surface-for-a-structured-repository.md +++ b/docs/future/config-surface-for-a-structured-repository.md @@ -2,8 +2,10 @@ > Status: proposal — review findings and an ordered plan. Nothing here binds until scheduled. > Date: 2026-07-24 -> Companion to [flux-maintainer-review-status-and-config-model.md](flux-maintainer-review-status-and-config-model.md), -> whose §4 "Then (API surface)" block (F6, F9, F10) this plan absorbs and extends. +> Absorbs and extends the API-surface block left unbuilt by the status and configuration-model +> review — `spec.suspend`, `spec.interval` and the reconcile-request annotation, the +> `CommitRequest` lifecycle, and the `ClusterWatchRule` scope question — which is now sequenced in +> [`../layout/api-wave.md`](../design/gittarget-api-wave.md). ## The one-sentence finding diff --git a/docs/future/direction-and-configuration-surface.md b/docs/future/direction-and-configuration-surface.md index 327f97f8..cfab36d6 100644 --- a/docs/future/direction-and-configuration-surface.md +++ b/docs/future/direction-and-configuration-surface.md @@ -5,7 +5,7 @@ > Date: 2026-08-27. > Companions: [`config-surface-for-a-structured-repository.md`](config-surface-for-a-structured-repository.md) > (the field-level review this extends), -> [`../design/gittarget-layout-model.md`](../design/gittarget-layout-model.md) and +> [`../layout/model.md`](../layout/model.md) and > [`../design/gittarget-api-wave.md`](../design/gittarget-api-wave.md) (the API work this > sequences), and > [`../design/support-boundary/helm-light-support-boundary.md`](../design/support-boundary/helm-light-support-boundary.md) @@ -136,28 +136,24 @@ namespaced kind is the tenant-ownable self-service surface, the cluster kind is platform-owned, and #146's `objectSelector` is the inclusion predicate for curated content. No new source-side API is needed for either direction. -The rules and the layout model meet at `layout.scope`, and the meeting produces two -validations worth declaring: - -- **`scope: SingleNamespace` requires `allowedSourceNamespaces` to be an exact one-name - list — and that name *is* the single namespace.** The question "which namespace is the - single one?" must not be answered by the rules: N rules do not own the folder, and the - first-writer-wins alternative is exactly the silent re-deciding the layout model - exists to forbid. So the identity lives on the GitTarget, as the authorization bound - collapsing into a structural fact: one exact name, no selector (a label selector cannot - guarantee singularity), and CEL can check it at admission. Rules then merely subscribe - within it — an omitted `sourceNamespace`, an explicit match, or `"*"` all resolve to - that one namespace, and a rule naming any other namespace refuses loudly under the - existing bilateral check. -- **A `ClusterWatchRule` referencing a `scope: SingleNamespace` target is refused, at the - rule, by name.** The payoff of `SingleNamespace` plus `writeNamespace: Never` is a - *portable* folder — deployable into any namespace at apply time. Cluster-scoped content - breaks that portability, so admitting it would quietly void the structural claim the - layout declared. The recorded objection is the app folder that carries a `ClusterRole` - or CRD as a passenger; the honest escapes are `scope: MultiNamespace`, a second - GitTarget for the cluster-scoped half, or — if the pattern proves common — a future - third scope value, argued on its own. Silently mixing is the one option the surface - should not offer. +The rules and the layout model meet at the folder's own shape, and the meeting produces two +things worth declaring. Both were originally written against a `layout.scope` enum; +[`../layout/model.md`](../layout/model.md) has since reversed that, so they are restated here +against what actually exists. + +- **Singularity is structural, not declared.** A folder is single-namespace because no + `{namespace}` appears in any of its paths — there is nowhere for a second namespace to go that + would not collide. That removes the question "which namespace is the single one?" from the API + rather than answering it with an enum plus an admission rule keeping the enum in agreement with + the authorization bound. N rules still do not own the folder: a document from an unadmitted + namespace is refused at the write boundary, which was always the thing enforcing singularity. +- **Cluster-scoped content in a portable folder is a caution, not an admission rule.** The payoff + of a single-namespace folder plus `serializeNamespace: false` is a *portable* folder, deployable + into any namespace at apply time, and a `ClusterRole` or CRD riding along breaks that. The honest + escapes are a second GitTarget for the cluster-scoped half, or accepting that the folder is not + portable and saying so. Refusing a `ClusterWatchRule` by name was proposed when the portability + claim was a structural enum the operator had to defend; with no such claim in the API, the + operator has nothing to defend, and the trade belongs to whoever writes the rule. ## Where the configuration surface should go — by example @@ -183,15 +179,10 @@ spec: name: artifacts-repo branch: main path: apps/shop - mode: Write commitWindow: 30s # moved from GitProvider: batching describes this folder - allowedSourceNamespaces: - names: [shop] # under SingleNamespace: exactly one name — this IS the namespace - layout: - kind: Kustomize - create: true # an empty repo becomes a buildable folder - scope: SingleNamespace - writeNamespace: Never # the artifact is environment-agnostic; namespace at deploy + placement: + useKustomize: true # an empty repo becomes a buildable folder + serializeNamespace: false # the artifact is environment-agnostic; namespace at deploy ``` and the folder is inert until the tenant subscribes content to it: @@ -210,13 +201,12 @@ spec: resources: ["*"] # the team's configuration CRDs are the artifact's content ``` -Everything direction B needs is here and nowhere else: the folder is described on the -GitTarget, the single namespace is *named* on the GitTarget (the one-name -`allowedSourceNamespaces` list is what `scope: SingleNamespace` requires), the WatchRule -is the artifact's manifest, structure can be brought into existence, and the artifact's -shape is a declared fact a reviewer can read. A `ClusterWatchRule` pointed at this target -is refused by name: cluster-scoped content would break the portability that -`writeNamespace: Never` promises. +Everything direction B needs is here and nowhere else: the folder is described on the GitTarget, +the single namespace is structural (no `{namespace}` in any path) and authorized by the rule that +subscribes content, the WatchRule is the artifact's manifest, structure can be brought into +existence by `useKustomize`, and the artifact's shape is a fact a reviewer can read. Cluster-scoped +content pointed at this target would break the portability `serializeNamespace: false` promises, +which is a trade to make deliberately rather than a refusal the operator enforces. ### Example 2 — brownfield adoption with a dry run (direction A's front door) diff --git a/docs/future/flux-maintainer-review-status-and-config-model.md b/docs/future/flux-maintainer-review-status-and-config-model.md deleted file mode 100644 index 066b2ecb..00000000 --- a/docs/future/flux-maintainer-review-status-and-config-model.md +++ /dev/null @@ -1,731 +0,0 @@ -# Review: the configuration model and status implementation, read as a Flux maintainer - -> Status: external review — findings open, nothing here binds until scheduled. -> Date: 2026-07-21 -> Reviewed at: branch `feat/gittarget-prune-mode-pr5`, commit `f37a7ba`. -> -> **Done so far.** F12's *enum casing* (`PruneMode` is now `Never`/`OnEvent`/`Always`, taken before -> the release because it was the last moment it was free), and then the whole of §4's -> **"Before the next release"** and **"Next"** blocks: **F1, F2, F3, F5, F7, F8, F11** and the rest -> of **F12**, shipped together on `feat/flux-status-contract`. See §6 for exactly what landed and -> what each fix looks like now. -> -> **Still open:** **F4** (abnormal-true polarity — resolved the other way, deliberately; see §6), -> **F6** (`spec.suspend`, `spec.interval`, reconcile-request annotation), **F9** (the stored -> `scope: Namespaced` status-write question), **F10** (CommitRequest lifecycle). Those are §4's -> "Then (API surface)" block and are a separate change. -> -> **That change now has a shape.** F6, F10, F12's reference-type nit and §3's pushbacks are sequenced -> together with the new-file-placement redesign in -> [`gittarget-api-wave.md`](../design/gittarget-api-wave.md), on the principle that the folder is -> described on the GitTarget and the connection describes only the connection. Two of this review's -> findings change that design rather than accompanying it: `spec.mode: Observe` (config-surface B1) -> becomes the way a layout is adopted safely, and `spec.interval` is what keeps the layout's -> scan-derived status fresh. F9 is deliberately kept **outside** the wave, because its answer -> constrains the enum work and should be known before anything is planned around it. -> Stance: reviewed as if this API were proposed for the GitOps Toolkit, with Flux's own -> source (`external-sources/flux/`) and kstatus (`sigs.k8s.io/cli-utils/pkg/kstatus`) as ground -> truth rather than recollection. - -## What was read - -- The API surface: `api/v1alpha3/*.go` (all six kinds plus `NamespaceMatcher`, `PrunePolicy`). -- The status implementations: - `internal/controller/{gittarget,watchrule,clusterwatchrule,gitprovider,clusterprovider,commitrequest}_controller.go`, - `condition_helper.go`, `stream_status.go`, `gittarget_dependency_status.go`, - `gittarget_source_cluster.go`, `internal/watch/stream_readiness.go`. -- `docs/configuration.md`, `docs/spec/status-conditions-guide.md`, `docs/spec/where-validation-lives.md`, `docs/design/reconcile-triggering.md`. -- Flux as ground truth: `external-sources/flux/pkg/apis/meta`, - `external-sources/flux/pkg/runtime/{conditions,patch}`, `external-sources/flux/pkg/apis/acl`, - `external-sources/flux/flux2/rfcs/`, `external-sources/flux/flux-operator/api/v1`. -- kstatus itself: `sigs.k8s.io/cli-utils/pkg/kstatus/status` from the module cache. - -Read-only review: no builds, no tests, no edits to the tree. - ---- - -## 1. The standard - -It is **kstatus** — `sigs.k8s.io/cli-utils/pkg/kstatus`, from Kubernetes SIG-CLI. It is what -`kubectl apply --wait`, kpt, Config Sync, Argo's health model (in spirit), and the Flux CLI all -lean on. Its upstream is the **Kubernetes API conventions**, section *"typical status properties"*, -which defines the abnormal-true polarity rule. Flux codifies both in -`fluxcd/pkg/apis/meta` — `ReadyCondition` / `StalledCondition` / `ReconcilingCondition` -(`external-sources/flux/pkg/apis/meta/conditions.go:43-59`) — and the flux-operator boils -readiness down to one CEL expression -(`external-sources/flux/flux-operator/api/v1/common_types.go:23`): - -```text -status.conditions.filter(c, c.type == 'Ready').all(c, c.status == 'True' && c.observedGeneration == metadata.generation) -``` - -**The precise rules kstatus enforces** (verified by reading -`cli-utils/pkg/kstatus/status/generic.go`, not from memory): - -1. `metadata.deletionTimestamp` set → `Terminating`. -2. `status.observedGeneration != metadata.generation` → `InProgress`. -3. any condition `Reconciling` **with status True** → `InProgress`. -4. any condition `Stalled` **with status True** → `Failed`. -5. otherwise → `Current`. - -Two consequences that matter enormously here, and that are easy to get wrong: - -- kstatus **never reads `Ready`** for a custom resource. `Ready=False` with `Reconciling` and - `Stalled` both False reads as **`Current`** — "done, healthy". The `Ready` condition is for - `kubectl wait` and humans; the trio is for machines. They must never disagree. -- kstatus only reacts to those conditions **when True**. That is why the convention says they - MUST NOT be present when False. - -### Verdict on compatibility - -**This project is on the standard, deliberately and knowledgeably — closer than most projects at -this stage.** `docs/spec/status-conditions-guide.md` states the contract correctly, and -`internal/controller/gittarget_kstatus_test.go` asserts it against the *real* kstatus library -rather than a hand-rolled reimplementation. That is mergeable work. - -What is not yet mergeable is the gap between that stated contract and what the reconcilers -actually emit. Three defects (F1–F3) make the trio lie in states production will reach, and the -tests do not catch them because they assert hand-built condition sets rather than the output of a -reconcile. - -| Area | Grade | Note | -|---|---|---| -| Condition types (`Ready`/`Reconciling`/`Stalled`) | **A** | Correct vocabulary, correct meanings, documented. | -| `observedGeneration` on object **and** per-condition | **A** | Passes the flux-operator CEL health expr as written. | -| `+listType=map` / `+listMapKey=type` on conditions | **A** | Correct on all six kinds. SSA-safe. | -| kstatus **conformance tests** | **A–** | Real library, real `Compute()`. Fixtures are synthetic (see F1/F2). | -| kstatus **behaviour under real reconciles** | **D** | F1 masks `Failed`; F2 never reaches `Current`. | -| Abnormal-true polarity (MUST NOT be present when False) | **C** | Always written as `False`. Tolerated by kstatus, violates the contract. | -| Status write discipline (no-op suppression) | **D** | Unconditional writes + always-moving timestamps (F3). | -| Reason vocabulary | **C** | Ad-hoc; `Reason == Type` in several places. | -| Flux object contract (`suspend`, `interval`, `requestedAt`, Events) — see layer 3 below | **F** | None of it present. Known — `docs/design/reconcile-triggering.md`. | -| `no phase string` discipline | **A** | Conditions only, everywhere. Correct. | - -### Three different things get conflated below — only one of them is a standard - -This review keeps saying "the GitOps Toolkit object contract". That is shorthand, not an official -name, and it is worth separating the three layers because they carry very different weight. - -**1. The Kubernetes API conventions — normative, upstream.** -`kubernetes/community`, `contributors/devel/sig-architecture/api-conventions.md`, section *"typical -status properties"*. This is where abnormal-true polarity comes from, where "conditions are a map -keyed by type" comes from, and where the UpperCamelCase enum rule (F12) comes from. Anything that -calls itself a Kubernetes API is measured against this. Flux's own condition docs link to it by -URL (`external-sources/flux/pkg/apis/meta/conditions.go:49-50, 57-58`). - -**2. kstatus — a real, named, vendor-neutral convention.** -`sigs.k8s.io/cli-utils/pkg/kstatus`, owned by Kubernetes SIG-CLI. It is a *library plus a -convention*: implement the trio the way it expects and any kstatus consumer can compute your -object's status without knowing what your CRD is. Consumers include cli-utils' own applier, -`kpt live apply`, and the Config Sync / Nomos family built on cli-utils. Flux designs to it -explicitly and says so in the comment above its condition constants. - -Worth knowing what it is *not*: **Argo CD does not use kstatus.** Its health model -(`gitops-engine`, built-in checks plus Lua) is a separate, older implementation of the same idea. -So conforming here buys interoperability with the Flux/kpt/cli-utils side of the ecosystem, not -with Argo. That is still the larger side for anything condition-driven, and it is the side this -project's users already live on. - -**3. The Flux object contract — not a published cross-vendor standard, but real, importable code.** -There is no RFC and no spec document for it. What exists instead is -**`github.com/fluxcd/pkg/apis/meta`** — its own Go module, Apache-2.0, versioned independently of -the controllers, which this repo *already pins at `v1.31.0`* (`go.mod:9`, for -`KubeConfigReference`). It is written in RFC-2119 language and is deliberately the shared artifact -rather than a doc: - -| Codified in `pkg/apis/meta` | Where | -|---|---| -| `ReadyCondition` / `StalledCondition` / `ReconcilingCondition` / `HealthyCondition` | `conditions.go:43-64` | -| Generic reasons: `Succeeded`, `Failed`, `Progressing`, `ProgressingWithRetry`, `Suspended`, `DependencyNotReady`, `InvalidPath`, `InvalidURL` | `conditions.go:80-115` | -| `ReconcileRequestAnnotation = "reconcile.fluxcd.io/requestedAt"` — *"any change in value SHOULD trigger a reconciliation"* | `annotations.go:23` | -| `ForceRequestAnnotation = "reconcile.fluxcd.io/forceAt"` — explicitly *"used to standardize the mechanism across controllers"* | `annotations.go:32` | -| `ReconcileRequestStatus` — an **embeddable struct** carrying `lastHandledReconcileAt`, plus `StatusWithHandledReconcileRequest` / `ObjectWithAnnotationRequests` interfaces | `annotations.go:49-133` | -| `LocalObjectReference`, `NamespacedObjectReference`, `NamespacedObjectKindReference`, `SecretKeyReference`, `KubeConfigReference` | `reference_types.go` | -| `AccessDeniedCondition` / `AccessDeniedReason`, `AccessFrom` | sibling module `pkg/apis/acl` | - -**What is deliberately *not* in that module: `spec.suspend` and `spec.interval`.** `meta` ships -`SuspendedReason` but no `Suspend` field — the fields themselves are hand-repeated in every fluxcd -controller's own API types. So "adopt the contract" means *import the type* for conditions, -reasons, annotations and references, but *copy the field shape* for suspend/interval. F6 is -therefore a convention-matching argument, not a dependency argument. - -**Who adheres.** Verifiable from this checkout: every fluxcd controller (source-, kustomize-, -helm-, notification-, image-\*) via `flux2`; and ControlPlane's **flux-operator** — a different -vendor under a different licence (AGPL) — which adheres and then extends it with its own -`FluxObject` interface and a CEL readiness expression -(`external-sources/flux/flux-operator/api/v1/common_types.go:23, 44-64`). That second one is the -interesting data point: an independent project chose to implement the same contract rather than -invent one, because it is what makes `flux`-shaped tooling work against non-Flux kinds. - -**Why this matters for F6/F7 specifically.** The argument is interop, not compliance. Nobody will -fail an audit for lacking `spec.suspend`. But a platform team that already runs Flux has muscle -memory — `suspend` to pause, annotate `requestedAt` to force, `kubectl describe` to see what -happened, notification-controller to route failures — and every one of those reflexes currently -returns nothing here. Meanwhile the marginal cost is close to zero: the module is already a -dependency, so adopting the condition types, reason constants and `ReconcileRequestStatus` is an -import and a struct embed, not a new supply-chain decision. - ---- - -## 2. Findings - -### F1 — `downgradeReady` erases a terminal `Stalled`, downgrading kstatus `Failed` → `InProgress` (High) - -`internal/controller/gittarget_controller.go:240` runs `applyDataPlaneConditions`, which for a -refused Git path sets the correct terminal trio (`Ready=False`, `Reconciling=False`, -`Stalled=True`, reason `UnsupportedContent`) — `gittarget_controller.go:526-543`. - -Then `internal/controller/gittarget_controller.go:258` runs `projectSourceAndProvider`, which on -any source/provider imperfection calls `downgradeReady` -(`internal/controller/gittarget_source_cluster.go:224-233`): - -```go -r.setCondition(target, GitTargetConditionReady, readyStatus, reason, message) -r.setCondition(target, GitTargetConditionReconciling, metav1.ConditionTrue, reason, message) -r.setCondition(target, GitTargetConditionStalled, metav1.ConditionFalse, ReasonProgressing, ...) -``` - -It unconditionally stamps `Stalled=False, Reconciling=True`, wiping the stall set 18 lines -earlier. The doc comment says it "only ever DOWNGRADES Ready" — true of `Ready`, but for -**kstatus** it *upgrades* the object from `Failed` to `InProgress`. - -**Trigger, most likely path:** a remote-source GitTarget before first discovery has -`SourceClusterReachable=Unknown` (`gittarget_controller.go:249-255`), which hits the -`reachStatus == metav1.ConditionUnknown` branch (`gittarget_source_cluster.go:212`). Any -GitProvider blip does it too. So: a GitTarget with unsupported kustomize content in its folder, or -`RenderMatchesLive=False`, reports "still working on it, please wait" forever to every kstatus -consumer, instead of failing fast. Nothing in the folder is being written, and nothing will be. -`GitPathAccepted=False` is still there for a human who goes looking; `kubectl wait` and any -kstatus-driven CI gate hangs to timeout. - -This is exactly why Flux computes the summary **once, at the end, from a declared precedence -order** rather than by successive mutation — the `summarize` pattern over `conditions.Set` with -`patch.WithOwnedConditions` (`external-sources/flux/pkg/runtime/patch/options.go:67-72`). - -**Fix.** Collect `(status, reason, message)` candidates from every gate into a list, then derive -the trio once at the end with stated precedence: `Stalled=True` wins over `Reconciling=True` wins -over `Ready=True`. `docs/spec/status-conditions-guide.md:69-73` already states the canonical -reads — make one function the single writer of the trio, and make every gate a *contributor*, not -a *setter*. Add a table-driven test that runs the actual reconcile and feeds the resulting object -to `kstatus.Compute`, including the "path refused **and** provider unready" cell. - ---- - -### F2 — A GitTarget with no WatchRules is never `Current`, and requeues every 10 s forever (High) - -`internal/watch/stream_readiness.go:75-77`: - -```go -func (s StreamSummary) StreamsRunning() bool { return s.Total > 0 && s.Ready == s.Total } -``` - -Zero tracked types → `StreamsRunning()==false` → the `!streams.StreamsRunning()` branch at -`gittarget_controller.go:562` → `Ready=False`, `Reconciling=True`, reason `NoResolvedTypes`. -Permanently, because nothing will ever resolve. And because `streamsSettling` is true -(`gittarget_controller.go:232`), the reconcile returns `RequeueAfter: RequeueStreamSettleInterval` -= **10 s** (`constants.go:117`), forever. - -That state is not exotic — it is **step 3 of the documented setup flow** -(`docs/configuration.md:29-34`: create GitTarget, *then* create WatchRules), and it is the steady -state of any target whose rules were deleted. A user following the docs has an object that -`kubectl wait --for=condition=Ready` never returns on, and that burns a reconcile plus a status -write every 10 seconds indefinitely. - -Empty is not in-progress. Nothing is pending. "I have nothing to mirror" is a **converged** -state — Flux's Kustomization with an empty path is `Ready=True` with a "no objects" message, not -InProgress. - -**Fix.** Split "nothing resolved" from "resolving". `Total == 0` should be `Ready=True`, -`Reconciling=False`, `Stalled=False` with reason `NoWatchRules` (or `NoResolvedTypes`) and a -message saying so — kstatus `Current`, honest, and it drops back to the 5 min cadence. Keep -`status.streams.summary: "0/0"` so the zero stays visible. If that feels like hiding a -misconfiguration, that is what an Event or a `Ready` *reason* is for, not a permanent -`Reconciling=True`. - ---- - -### F3 — Always-moving timestamps and no self-predicate create a self-triggering reconcile edge (High) - -Three things compose badly. - -1. **Every reconcile stamps a fresh timestamp.** - `gittarget_controller.go:129` — `target.Status.LastReconcileTime = metav1.Now()`, unconditional. - `internal/watch/stream_readiness.go:244` — `StreamSummary{..., ObservedTime: metav1.Now()}`, - surfaced into `status.streams.observedTime` for GitTarget *and* both rule kinds - (`gittarget_controller.go:1065-1078`, `stream_status.go:26-40`). - -2. **The status write is unconditional and full-object.** `updateStatusWithRetry` - (`gittarget_controller.go:1081-1113`, and the near-identical copies in - `watchrule_controller.go:405`, `clusterprovider_controller.go:283`, - `gitprovider_controller.go:403`) does `latest.Status = target.Status; r.Status().Update(...)`. - There is no "did anything change?" check — and the timestamps guarantee something always did. - -3. **`For()` carries no predicate on GitTarget, WatchRule, or ClusterWatchRule** - (`gittarget_controller.go:1117-1118`, `watchrule_controller.go:463-464`, - `clusterwatchrule_controller.go:547`). Status-subresource writes bump `resourceVersion` and - fire an Update watch event, which `handler.EnqueueRequestForObject` puts straight back on the - queue — un-rate-limited. - -So each reconcile enqueues itself. It is not an unbounded spin — `metav1.Time` serialises at -RFC3339 second precision, so a follow-up reconcile that lands inside the same wall-clock second -produces a byte-identical status and the apiserver no-ops it. The practical shape is therefore: -**every reconcile costs roughly two reconciles and at least one etcd write, degenerating into a -sustained self-sustaining loop whenever a reconcile takes ≥1 s** — plausible here, since -`checkForConflicts` does a cluster-wide `List` of every GitTarget on every pass -(`gittarget_controller.go:793`) on top of several `Get`s and `DeclareForGitTarget`. - -Combine with F2 and an idle GitTarget writes to etcd at ~0.1 Hz forever, per object, and wakes -every controller watching GitTargets each time. - -`GitProvider` and `ClusterProvider` **do** have self-predicates -(`gitprovider_controller.go:462-465`, `clusterprovider_controller.go:332-335`), so this is an -inconsistency, not a house style. `docs/design/reconcile-triggering.md:44-50` inventories -per-controller predicates but records only the *dependency* edges — the missing `For()` predicate -on GitTarget/WatchRule is not in that table. - -**Fix, in order of value:** - -- Adopt `fluxcd/pkg/runtime/patch.Helper`. It computes a merge patch of *what actually changed* - and sets `observedGeneration` "only if there is a change" - (`external-sources/flux/pkg/runtime/patch/options.go:33-35`). No change → no request → no watch - event → no loop. A drop-in: `fluxcd/pkg/apis/meta` is already a dependency. -- Failing that: `if !equality.Semantic.DeepEqual(latest.Status, target.Status) { update }`, and - drop `LastReconcileTime` / `observedTime` from the comparison — or drop the fields. -- Ask whether `status.lastReconcileTime` and `status.streams.observedTime` earn their keep at all. - Flux deliberately does not carry a "last reconcile attempt" timestamp; a condition's - `lastTransitionTime` plus `controller_runtime_reconcile_total` answer the same question without - making every object mutable-on-read. `LastPushTime` is genuinely useful (it records a real - event) — keep that one. -- Add a `For()` predicate on the three controllers that lack one, e.g. - `predicate.Or(GenerationChangedPredicate{}, )`. - ---- - -### F4 — Abnormal-true polarity: `Reconciling`/`Stalled` are always present, including when False (Medium) - -`external-sources/flux/pkg/apis/meta/conditions.go:47-59` states it twice: - -> The Condition adheres to an "abnormal-true" polarity pattern, and **MUST only be present on the -> resource if the Condition is True**. - -Every write path here emits both unconditionally: `setStalledConditions` -(`gittarget_controller.go:450-457`), `downgradeReady`, `setRuleProgressing` -(`stream_status.go:114-123`), `setReadyConditions` / `setProgressingConditions` on both providers. - -kstatus tolerates it — it only tests for `== True` — so this is not a correctness bug. But it is a -contract violation with real costs: - -- `kubectl get gittarget -o yaml` carries six condition entries where three would do. -- Any tool that treats *presence* as signal (a fair reading of the convention) misreads it. -- `Reconciling=False, reason=UnsupportedContent, message="Reconciliation is stalled"` - (`gittarget_controller.go:538-539`) is a condition that says nothing true about reconciling. It - exists only to be overwritten. - -`docs/spec/status-conditions-guide.md:31-32` names the polarity rule correctly and the code does -the opposite. Pick one. Preferred: `DeleteCondition` on the way to False — -`conditions.Delete(obj, meta.StalledCondition)` in Flux terms — and let `Ready` carry the positive -summary. - ---- - -### F5 — `upsertCondition` reorders the condition list on every touch (Medium) - -`internal/controller/condition_helper.go:26-44` rebuilds the slice with the target type *removed* -and then appends it at the end. Touch `Ready` and it migrates to the tail; touch it again next -pass and everything else has shuffled. - -Effects: gratuitous diffs in `kubectl get -o yaml` and in any GitOps repo that mirrors these -objects (which, given what this product does, is not hypothetical — a GitTarget mirroring -GitTargets would commit condition-reordering noise); a byte-level change even when nothing -semantically changed, which feeds F3; and unstable ordering for humans. - -The `LastTransitionTime` handling is *correct* — preserved when `Status` is unchanged -(`condition_helper.go:39-41`), matching `apimeta.SetStatusCondition` and the API conventions. -Flux is actually stricter than the convention here, resetting on Reason/Message change too -(`external-sources/flux/pkg/runtime/conditions/setter.go:44-46`); this repo's is the more -conventional reading and worth keeping. - -**Fix.** Use `k8s.io/apimachinery/pkg/api/meta.SetStatusCondition`, which updates in place. Or -adopt Flux's approach and sort deterministically, with `Stalled`, `Reconciling`, `Ready` weighted -to the front for `kubectl` legibility -(`external-sources/flux/pkg/runtime/conditions/setter.go:89-92, 196-218`) — a nice touch worth -stealing regardless. - ---- - -### F6 — No `spec.suspend`, no `spec.interval`, no reconcile-request annotation (Medium) - -Nothing in `api/v1alpha3` has `suspend` or `interval` — zero hits. The cadence is a compile-time -constant: `RequeueSteadyInterval = 5 * time.Minute` (`constants.go:113`). - -Every Flux object implements all three -(`external-sources/flux/flux-operator/api/v1/common_types.go:44-64`): `GetInterval()`, -`IsDisabled()`, `SetLastHandledReconcileAt()`. They are not decoration: - -- **`spec.suspend`** is the only way to say "stop touching this while I fix the repo by hand" - without deleting the object. For a controller that *writes to a Git repository*, the absence of - a pause button is the single most surprising gap in this API. Today the only way to stop a - GitTarget writing is to delete it or its rules — and per the `GitTargetSpec` doc comment, that - is irreversible for path/branch/provider. -- **`spec.interval`** — `GitProvider` does a real network `ls-remote` against the git host on - every pass (`gitprovider_controller.go:220`, `checkRemoteConnectivity`), hardcoded at 5 min, - with no jitter. N providers against github.com from one operator, all re-synchronised into a - thundering herd after each restart. Flux ships `fluxcd/pkg/runtime/jitter` precisely for this. - At minimum: jitter the requeue. Better: `spec.interval` per object, since a GitProvider pointing - at a rate-limited enterprise host and one pointing at a local Gitea do not deserve the same - cadence. -- **`reconcile./requestedAt` + `status.lastHandledReconcileAt`** is the universal - "reconcile now" idiom (`flux reconcile`, `kubectl annotate`, webhook receivers). This is the one - piece of F6 that is *shared code* rather than a copied field shape: embed - `meta.ReconcileRequestStatus` and call `meta.ReconcileAnnotationValue` - (`external-sources/flux/pkg/apis/meta/annotations.go:23-64`) and the semantics — including the - "any change in value SHOULD trigger" token comparison — come with it. Already identified as F1 - in `docs/design/reconcile-triggering.md`; still unbuilt. - ---- - -### F7 — Zero Kubernetes Events (Medium) - -There is no `EventRecorder` anywhere in `internal/controller` or `cmd/main.go` — no `Recorder`, -`Eventf`, or `Event(` call sites. - -Consequences: `kubectl describe gittarget` shows no history; a transient push failure that -resolves before anyone looks is invisible; and there is **no integration path with -notification-controller** or any Event-driven alerting, because there is no Event to route. -Metrics say a counter moved; they cannot say *which* GitTarget failed to push and why. - -`docs/design/reconcile-triggering.md:222-227` already prescribes the fix ("F4. Conditions **and** -Events, every loop") and cites `fluxcd/pkg/runtime/events`. This ranks above the webhook-receiver -work in the same doc: Events are cheap, and a controller that writes to Git without emitting an -Event on write failure is hard to operate. - ---- - -### F8 — Reason vocabulary is ad-hoc, and in several places `Reason == Type` (Medium) - -`gitprovider_controller.go:325`, `clusterprovider_controller.go:226`, `stream_status.go:45`: - -```go -r.setCondition(gitProvider, ConditionTypeReady, metav1.ConditionTrue, ConditionTypeReady, message) -// ^^ type ^^ reason == "Ready" -const ruleReadyReason = "Ready" -``` - -`Ready=True, reason=Ready` conveys nothing. A reason answers *why*. Flux's generic set -(`external-sources/flux/pkg/apis/meta/conditions.go:80-115`) is `Succeeded`, `Failed`, -`Progressing`, `ProgressingWithRetry`, `Suspended`, `DependencyNotReady`, `InvalidPath`, -`InvalidURL`, `AccessDenied` (the last from `pkg/apis/acl`) — and it is a *shared vocabulary*, so -one alerting rule works across every kind. - -Here it is `OK`, `Ready`, `Checking`, `Resolved`, `Progressing`, `Stalled`, `Validated` scattered -across `constants.go:95-145` and four controllers. `Progressing` and `Stalled` already match Flux. -Suggested: - -- Alias the generic ones to `meta.SucceededReason` / `meta.ProgressingReason` / - `meta.FailedReason` / `meta.DependencyNotReadyReason` — `github.com/fluxcd/pkg/apis/meta` is - already imported (`clusterprovider_types.go:6`), so it is free. -- Replace `OK` and reason-equals-type with `Succeeded`. -- Keep the excellent domain-specific reasons (`UnsupportedContent`, `IgnoreShadowsManagedPath`, - `WriteBoundaryRefused`, `NoAdmittedSourceNamespaces`) — those are exactly what the Flux docs - mean by "declaration of domain common Condition reasons in the API specification is - RECOMMENDED". Consider promoting them from `internal/controller` constants to exported constants - in `api/v1alpha3`, so consumers can compile against them. - -`GitTargetReasonProviderNotFound` should probably be `meta.DependencyNotReadyReason` or at least -`DependencyNotFound`, for the same cross-kind-alerting reason. - ---- - -### F9 — A stored `ClusterWatchRule` with `scope: Namespaced` may be unable to report its own refusal (Medium — needs verification) - -`api/v1alpha3/clusterwatchrule_types.go:130-133` narrows the enum to `Cluster` only, deliberately -keeping the field so a re-apply *fails*. The reasoning in the comment above it is sound. -`DeclaresNamespacedScope()` (`:144`) then refuses a stored value at compile time. - -The concern: for CRDs, the apiserver validates the **whole object** against the OpenAPI schema on -**status-subresource** updates too, not just spec updates. If that holds here, the controller -cannot write `Stalled=True` onto an object whose stored `spec.rules[].scope` is `Namespaced` — the -status update is rejected 422, and the one object that most needs to explain itself is the one -that cannot. - -**Mitigating factor:** CRD Validation Ratcheting (beta and default-on in 1.30, GA in 1.33) skips -re-validation of *unchanged* fields, which would make this a non-issue on modern clusters. So the -exposure is older clusters, or a cluster with the feature gate off. - -Not confirmed by execution. **Worth one envtest:** create a ClusterWatchRule with -`scope: Namespaced` via a client that bypasses the enum (or against an older CRD), then attempt a -status update, on the minimum supported Kubernetes version. If it fails, the fallback is to widen -the enum back and rely solely on the compile-path refusal plus a loud `Stalled` condition — -refusing at admission is nice, but not at the cost of being unable to report the refusal. - ---- - -### F10 — CommitRequest objects accumulate forever, and the controller cannot delete them (Medium) - -`CommitRequest` is a one-shot imperative object with an immutable spec -(`commitrequest_types.go:14`). Every "save now" leaves an object in etcd. There is no TTL, no -`ownerReference`, no GC path — and the reconciler's RBAC is -`commitrequests, verbs=get;list;watch` (`commitrequest_controller.go:108`), so it could not delete -them even if it wanted to. - -A team using this as an interactive save button generates hundreds per namespace per week. -Nothing reaps them. - -The broader Flux-maintainer objection is that **this should be an annotation, not a kind**. -`reconcile./requestedAt` is the established idiom for "act now", it is free to issue, it -self-GCs by being overwritten, and it needs no CRD. The counter-argument here is accepted: a -CommitRequest carries a verbatim commit message, a collect-delay, and (via the admission webhook) -the submitter's identity, and it reports back a SHA. That is genuinely more than a trigger, and -identity capture at admission is the textbook justification made well in -`docs/spec/where-validation-lives.md:50-56`. - -But if it stays a kind, it needs a lifecycle: - -- `spec.ttlSecondsAfterFinished` (the Job precedent) or a controller-side "delete terminal - requests older than N", plus the `delete` verb. -- Or an `ownerReference` to the GitTarget so it is at least cascade-deleted. -- At minimum, document the retention expectation and ship a - `kubectl delete commitrequests --field-selector` recipe. - ---- - -### F11 — `observedGeneration` can record a generation that was never observed (Low) - -`updateStatusWithRetry` re-`Get`s the object and then does `latest.Status = target.Status` -(`gittarget_controller.go:1102`). `target.Status.ObservedGeneration` was set from the generation -read at the *top* of the reconcile (`:128`). If the spec changed in between, the new object is -stamped with an `observedGeneration` equal to a generation never actually processed — kstatus then -reports `Current` for a spec nobody looked at, until the next pass corrects it. - -Narrow window, self-correcting, low severity. Flux avoids it structurally by patching with -optimistic concurrency and setting `observedGeneration` from the object being patched -(`patch.WithStatusObservedGeneration`). Another thing that comes free with the patch helper (F3). - ---- - -### F12 — Small API-conventions nits (Low) - -- **Enum casing — DONE.** `PruneMode` values were `never` / `onEvent` / `always` - (`prune_policy.go:17-23`). Kubernetes API conventions call for UpperCamelCase enum values — - compare `imagePullPolicy: Always|Never|IfNotPresent`, `persistentVolumeReclaimPolicy: - Retain|Delete`. Flux is itself inconsistent here (`driftDetection.mode: enabled|warn|disabled`), - so this was in company, but `Never`/`OnEvent`/`Always` is the conventional spelling. This was the - moment to decide — the field was brand new and unreleased on this branch, and the branch already - carried two `feat(api)!` commits, so the rename rode along in a release that was breaking anyway; - after that release it would have been a second breaking change for a cosmetic gain. - **Resolved on `feat/gittarget-prune-mode-pr5`: the values are now `Never` / `OnEvent` / - `Always`.** The typed constants (`PruneNever`, `PruneOnEvent`, `PruneAlways`) are unchanged, so - no Go call site moved; the wire values, the CRD enum and default, and the docs did. - (Conversely, `OperationType`'s `CREATE`/`UPDATE`/`DELETE` **is** right, because it matches - `admissionregistration.k8s.io` `rules[].operations` verbatim.) -- **Duplicated representation.** `status.streams.summary` is `fmt.Sprintf("%d/%d", Ready, Total)` - (`stream_readiness.go:70-72`) over fields that are right there in the same struct. - `status-conditions-guide.md:37-38` says don't do this. It exists only to feed a printer column — - a legitimate reason, but worth naming as such in the field doc so the next reader doesn't - "clean it up". -- **Printer-column sprawl.** GitTarget declares 13 columns, 7 of them default-priority - (`gittarget_types.go:238-252`). `kubectl get gittargets` will wrap on any normal terminal. Flux - ships 3–4 (Age, Ready, Status). Push `Provider`, `Branch`, `Path` to `priority=1` and keep - `Ready`, `Reason`, `Streams`, `Age`. -- **Reference types.** Six near-identical shapes — `GitProviderReference`, - `ClusterProviderReference`, `LocalTargetReference`, `NamespacedTargetReference`, - `LocalSecretReference`, `KnownHostsReference` — while already depending on - `fluxcd/pkg/apis/meta`, which offers `LocalObjectReference`, `NamespacedObjectReference`, - `NamespacedObjectKindReference`, `SecretKeyReference`. Reusing `meta.KubeConfigReference` - (`clusterprovider_types.go:69`) clearly paid off. The `Group`+`Kind` enum-with-default pattern is - defensible (it documents what is accepted and leaves room to widen), but consider embedding - `meta.LocalObjectReference` for the name half so downstream Go consumers get interoperable types. -- **`metav1.ObjectMeta` json tags are inconsistent** — `omitempty,omitzero` on GitTarget / - GitProvider / ClusterProvider / CommitRequest, plain `omitempty` on WatchRule - (`watchrule_types.go:277`) and ClusterWatchRule (`clusterwatchrule_types.go:203`). -- **`GitProviderStatus.Conditions` lacks `+patchStrategy=merge` / `+patchMergeKey=type`** - (`gitprovider_types.go:143-146`) while the other five kinds have them. Harmless with - `listType=map`, but inconsistent. - ---- - -## 3. The configuration model itself - -Setting status aside — this is the part worth defending as-is, and that deserves saying before the -criticism. - -**What is genuinely strong:** - -- **`docs/spec/where-validation-lives.md` is better than what Flux has written down.** The - schema → CEL → reconciler ladder, and specifically the argument that *reconcile-time is the - stronger gate because admission cannot see a policy tightened after creation*, is correct and is - the thing most projects get backwards. Flux arrived at the same place empirically; here it is a - stated rule. The one webhook shipped is justified on exactly the right grounds (identity exists - only in the `AdmissionRequest`). -- **`NamespaceMatcher`'s absent-vs-declared-vs-empty trichotomy** (`namespace_matcher.go:14-34, - 106-123`). Flux's `acl.AccessFrom` (`external-sources/flux/pkg/apis/acl/`) is a flat - `namespaceSelectors` list with no way to express "declared and empty", and Flux has been bitten - by exactly that ambiguity. The `selector: {}` = everything / `{}` = nothing / absent = legacy - distinction, anchored to `LabelSelectorAsSelector`'s own `Nothing()`/`Everything()` asymmetry, is - more precise than the incumbent. Rejecting `names: ["*"]` because Kubernetes treats it as a - literal name is the kind of detail that only comes from having been burned. -- **The two-key delegation for cross-namespace source watching** — - `ClusterProvider.spec.allowSourceNamespaceOverride` (platform admin) AND - `GitTarget.spec.allowedSourceNamespaces` (destination owner) — is a materially better answer than - Flux's. RFC-0001 (`external-sources/flux/flux2/rfcs/0001-authorization/README.md:97-101`) - documents that Flux controllers simply **do not respect namespace isolation** when dereferencing - cross-namespace refs, a long-standing multi-tenancy sore spot. Deny-by-default and two-party is - the right shape. -- **Immutability where identity is at stake.** `providerRef`/`branch`/`path`/`clusterProviderRef` - immutable via CEL (`gittarget_types.go:45-53`), with the reasoning in the type doc: a folder's - meaning is constituted by those four. `spec.prune` deliberately mutable - (`gittarget_types.go:134-139`) because forcing a delete-and-recreate to re-enable convergence - would destroy the one thing that cannot be rebuilt. Well drawn. -- **The `prune.mode` design.** Separating "the source told me it was deleted" from "a snapshot - didn't mention it" is the correct decomposition, and `onEvent` as the effective default — - resolved in code via `EffectiveMode()` rather than relying on CRD defaulting, so a *stored* - pre-field object is also safe (`prune_policy.go:52-63`) — is careful in the right way. Flux's - `spec.prune` is a single bool and cannot express the middle mode. -- **`no phase string`, anywhere.** Six kinds, zero `.status.phase`. Rarer than it should be. - -**Where to push back on the model:** - -- **The missing pause button** (F6) is the biggest hole. This writes to Git. -- **`GitProvider` is doing three jobs**: remote+credentials (Flux: `GitRepository` + `Secret`), - commit identity/templates/signing (Flux: `ImageUpdateAutomation.spec.git.commit`), and push - batching policy. Defensible cohesion — they are all "how this repo gets written" — but note the - consequence: `spec.push.commitWindow` and `spec.commit.message.*` are properties of a *workload*, - yet they live on the *connection*, so two GitTargets sharing a repo cannot have different - batching or message templates. If that ever needs to differ per target, the field has to move, - and moving it is breaking. Worth writing down as a known constraint now. -- **The `GitProvider` (namespaced) / `ClusterProvider` (cluster) asymmetry** is justified well in - `docs/configuration.md:39-63` and the reasoning holds: a Git destination is a team's write - boundary, a source cluster is a shared physical identity. It *will* still surprise people, and - the doc already anticipates the follow-up ("if a platform later needs a shared, platform-owned - Git destination, that should be a separate cluster-scoped concept"). Keep that paragraph. -- **`clusterProviderRef` defaults to `{name: "default"}` for an object the operator never - creates.** A GitTarget applied to a fresh cluster is unready with `ClusterProviderNotFound` until - someone creates it. Deliberate and well defended (`docs/configuration.md:419-436`; the chart - renders it by default), and the substance is right — silently defaulting to in-cluster - credentials would bypass the authorization model. One ask: make the `ProviderNotFound` message - for the literal name `default` say *"the ClusterProvider named 'default' does not exist; the - operator never creates one — see `clusterProvider.createDefault` in the chart, or commit the - object"*. A generic "provider not found" for the **defaulted** value is the single most likely - first-run support ticket. -- **`ClusterWatchRule` is unbounded by `allowedSourceNamespaces` by construction** — correct - (cluster-scoped objects have no namespace) and clearly documented - (`clusterwatchrule_types.go:172-177`, `docs/configuration.md:935-938`). The mitigation "give each - tenant its own ClusterProvider and credential" is the right answer and matches how Flux tells - people to do hard multi-tenancy. Fine as-is; make sure the security model doc says it in one - place. -- **`sourceNamespace: "*"` fan-out.** One watch stream per (type × admitted namespace) - (`watchrule_types.go:132-134`), a cost flagged honestly in the type doc. On a broad policy across - a big cluster this is the scalability cliff — worth a hard cap with a `Stalled` reason - (`TooManyStreams`) rather than discovering it as apiserver watch pressure. - ---- - -## 4. Suggested order - -**Before the next release** (observable behaviour, cheap): - -1. **F1** — one function owns the trio; every gate contributes a candidate, precedence stated once. - Add a reconcile-output-driven `kstatus.Compute` test covering "refused path + unready provider". -2. **F2** — `Total == 0` is `Ready=True` / `Current`, not perpetual `Reconciling`. -3. **F3** — adopt `fluxcd/pkg/runtime/patch.Helper` (or a `DeepEqual` guard), drop or exclude the - always-moving timestamps, add `For()` predicates to the three controllers missing one. - -**Next** (contract alignment, low risk, high interop value): - -1. **F5** — `apimeta.SetStatusCondition`, or Flux's sorted `Set`. -2. **F8** — alias generic reasons to `fluxcd/pkg/apis/meta`; kill `reason == type`; export the - domain reasons from `api/v1alpha3`. -3. **F4** — delete `Reconciling`/`Stalled` rather than writing them False. -4. **F7** — wire an `EventRecorder`; emit on every terminal outcome and every push failure. - -**Then** (API surface — do the breaking ones while still `v1alpha3`). These are the wave, and they -are now sequenced with the layout model in -[`gittarget-api-wave.md`](../design/gittarget-api-wave.md) rather than scheduled from here: - -1. **F6** — `spec.suspend` on GitTarget/WatchRule/ClusterWatchRule/GitProvider; `spec.interval` on - GitProvider at minimum; jitter the requeue; `reconcile.configbutler.ai/requestedAt` + - `status.lastHandledReconcileAt`. -2. **F12** — ~~decide `PruneMode` casing **now**~~ (done, pre-release); trim printer columns; unify - ObjectMeta tags. -3. **F10** — CommitRequest lifecycle (TTL or ownerRef) and the `delete` verb. -4. ~~**F9**~~ — **not in this block.** F9 is deliberately kept OUTSIDE the wave, because its answer - constrains the enum work and should be known before anything is planned around it. Verify the - `scope: Namespaced` status-write path on the minimum supported Kubernetes version as its own - change, ahead of the wave. - ---- - -## 5. Bottom line - -The question asked was whether the status implementation is *compatible enough* with the open -standard. - -**The model is compatible. The implementation is compatible in three of the four states it can be -in.** The right conditions, the right per-condition `observedGeneration`, the right list semantics, -no phase string, a written contract, and conformance tests against the real kstatus library — more -than most projects have at v1alpha3, and the reason the two real defects are worth fixing rather -than redesigning around. F1 makes a `Failed` object look `InProgress`; F2 makes an idle object -never reach `Current`. Both are localized. Fix those and this passes a kstatus conformance review. - -The larger gap is not kstatus at all — it is the **Flux object contract** (§1, layer 3): `suspend`, -`interval`, reconcile-on-annotation, and Events. That one is not a formal standard and nobody fails -an audit for missing it; it is codified only as an importable Go module, -`github.com/fluxcd/pkg/apis/meta`, which this repo already depends on. But it is what makes an -object *operable* by the reflexes a platform team already has from Flux, and right now none of -those reflexes return anything here. Most of it is already designed in -`docs/design/reconcile-triggering.md`; it needs building. `spec.suspend` first, because this -controller writes to Git and there is currently no way to make it stop. - ---- - -## 6. What was built (2026-07-22, branch `feat/flux-status-contract`) - -The review's own ordering was followed: §4's "Before the next release" and "Next" blocks, plus the -cheap F12 nits. The API-surface block (F6, F9, F10) is deliberately left for a separate change, -because each of those adds or changes a spec field and deserves its own review. - -Two pieces of shared machinery carry most of it, and both replace code that had been copied five or -six times: - -**`internal/controller/readiness.go` — the accumulator that owns the trio.** Gates no longer set -`Ready`/`Reconciling`/`Stalled`; they *contribute* a verdict at one of three levels (converged, -progressing, stalled) and the trio is derived once at the end. Within a level the first contributor -wins, so the precedence is the order of a handful of adjacent calls in `gitTargetReadinessGates` / -`ruleReadiness` — readable in one place rather than emergent from which gate ran last. This is the -`summarize`-once shape the review pointed at, without taking `fluxcd/pkg/runtime` as a dependency. - -**`internal/controller/status.go` — the per-reconcile status session.** `beginStatus` captures the -object as read, `set`/`applyReadiness` collect what the reconcile wants to say, and `commit` writes -the difference exactly once. It replaced five near-identical `updateStatusWithRetry` copies and, with -them, three shared defects, plus six `setCondition`/`setTypedCondition` wrappers and eight -trio-setting helpers (`setStalledConditions` ×3, `setProgressingConditions`, `setReadyConditions`, -`setRuleStalled` ×2, `setRuleProgressing`, `downgradeReady`). - -| Finding | What landed | -|---|---| -| **F1** — `downgradeReady` erased a terminal `Stalled` | The accumulator. `downgradeReady` is gone; the source/provider projection now only *publishes* its three conditions and contributes progressing verdicts. The regression cell ("refused path **and** unready provider") is a table row in `gittarget_status_test.go` that runs the real `kstatus.Compute` over reconciler output. | -| **F2** — a GitTarget with no WatchRules never reached `Current` | `streamsAxis` reports `True` when `Ready == Total`, which is vacuously true at zero. `status.streams.summary` still reads `0/0` and the reason is still `NoResolvedTypes`, so the zero stays visible. "Data plane not wired" is now a *different* state (reason `Progressing`) so the two can never be confused. | -| **F3** — unconditional writes, moving timestamps, no self-predicate | All three legs. `commit` sends nothing when the status is unchanged; `status.lastReconcileTime` and `status.streams.observedTime` were removed rather than excluded from the comparison (they existed only to be overwritten); `GenerationChangedPredicate` added to the `For()` of GitTarget, WatchRule and ClusterWatchRule. | -| **F5** — `upsertCondition` reordered the list on every touch | Delegates to `apimeta.SetStatusCondition`, which updates in place. The test now pins position stability rather than de-duplication of an input `listType=map` makes impossible. | -| **F7** — zero Kubernetes Events | Every reconciler takes an `EventRecorder`; the shared writer emits one Event per **persisted** `Ready` transition (`Normal` on True, `Warning` otherwise). After the patch, not beside each `set`, so intermediate values a reconcile never stored are never announced. | -| **F8** — ad-hoc reasons, `Reason == Type` | Generic reasons alias `fluxcd/pkg/apis/meta`. `OK` and `Ready` as *reasons* became `Succeeded` across all six kinds and the e2e suite. Domain reasons kept. | -| **F11** — `observedGeneration` for a generation never observed | Comes free with the patch: the write uses optimistic concurrency, so a spec that moved under the reconcile loses the write instead of being mislabelled. The conflict is dropped rather than retried — the write that beat us already enqueued a fresh pass. | -| **F12** — API-conventions nits | GitTarget's default printer columns cut from seven to four (`Provider`/`Branch`/`Path` moved to `priority=1`); `patchStrategy`/`patchMergeKey` added to `GitProviderStatus.Conditions`; `ObjectMeta` json tags unified on `omitempty,omitzero`; `status.streams.summary`'s field doc now says *why* it duplicates the counts beside it. | - -### F4 was resolved the other way, on purpose - -The review is right that the code and `docs/spec/status-conditions-guide.md` disagreed, and right -that "pick one" was the answer. The one picked was **keep writing the pair when False**, and the -guide now says so and says why. - -kstatus tolerates it — it tests for `== True` and ignores everything else — so nothing downstream -misreads it. Against that: `kubectl wait --for=condition=Stalled=false` reads the explicit `False`, -this repo's e2e suite asserts it in several specs, and a condition that vanishes is harder for a -human to reason about than one that reads `False`. The cost of the deviation is three extra condition -entries in `-o yaml`; the cost of removing it is a real capability plus a suite-wide rewrite. What -*was* wrong and is now fixed is the incoherent message the review quoted -(`Reconciling=False, message="Reconciliation is stalled"` is at least true of the object, and it is -now written from one place rather than as leftover state from a gate that lost). - -The deviation is recorded in the guide as the single knowing departure from the API conventions, so -the two documents cannot silently drift again. diff --git a/docs/layout/README.md b/docs/layout/README.md new file mode 100644 index 00000000..1fb2610c --- /dev/null +++ b/docs/layout/README.md @@ -0,0 +1,32 @@ +# `docs/layout/` — where a document goes in Git, collected in one place + +This folder is organized by **topic**, which is a deliberate exception to the rule in +[`../INDEX.md`](../INDEX.md) that the other folders are picked by lifecycle. The layout question +(given a live object, which file in which folder receives it, and what else has to change so that +file is reachable) was spread across `spec/` and `design/`, and following the argument meant knowing +which of the two a given piece lived in. + +Lifecycle still decides whether a page binds, so each entry is labelled with the class it would have +had in the old layout. Read the label before you read the page. + +| Document | Class | What it holds | +|---|---|---| +| [`new-file-placement-rules.md`](new-file-placement-rules.md) | **spec** | where a brand-new resource's file goes: declared, the folder's one kustomize root, canonical. Go source cites it by path, and `task lint-docs` checks those citations | +| [`contextual-namespace.md`](contextual-namespace.md) | **spec** | kustomize graph-aware namespace inference, and the supported subset. This is the inference `serializeNamespace` overrides | +| [`model.md`](model.md) | **design** | the proposal, reversed and much smaller: the path template **stays**, and gains two optional booleans on `spec.placement` — `useKustomize` and `serializeNamespace`. Carries the status stanza, the post-scan pass, and the order the work is built in | + +[`examples/`](examples/README.md) makes the proposal tangible: six scenarios, each with a repository +folder, the matching `GitTarget` and rule configuration, one live input, and the exact patch the +operator proposes. They are design material rather than install manifests, and `model.md` turns them +into an executable corpus in its first PR. + +## What is deliberately not here + +- [`../design/gittarget-api-wave.md`](../design/gittarget-api-wave.md) sequences the breaking work on + `GitTarget`. The placement work is additive and is not part of it. +- [`../design/placement-visibility-and-declared-defaults.md`](../design/placement-visibility-and-declared-defaults.md) + holds the three questions the sibling-inference deletion left. Decided, mostly unbuilt, and its + Question 2 is superseded by [`model.md`](model.md). +- [`../design/support-boundary/`](../design/support-boundary/README.md) owns what the operator may + edit and what it refuses. Layout decides **where** a document goes; the support boundary decides + **whether** it may be written at all. diff --git a/docs/spec/contextual-namespace-and-kustomize-folder-editing.md b/docs/layout/contextual-namespace.md similarity index 98% rename from docs/spec/contextual-namespace-and-kustomize-folder-editing.md rename to docs/layout/contextual-namespace.md index c9ecd5fa..c423522c 100644 --- a/docs/spec/contextual-namespace-and-kustomize-folder-editing.md +++ b/docs/layout/contextual-namespace.md @@ -8,11 +8,9 @@ > Captured: 2026-06-08 > Updated: 2026-06-08 > Related: -> [file-agnostic-placement.md](gittarget-new-file-placement-rules.md), -> [manifest-inventory-file-agnostic-placement.md](manifest-system.md), -> [current-manifest-support-review.md](current-manifest-support-review.md), -> [version2/gittarget-repository-validity-and-placement.md](gittarget-new-file-placement-rules.md), -> [version2/gittarget-new-file-placement-rules.md](gittarget-new-file-placement-rules.md) +> [new-file-placement-rules.md](new-file-placement-rules.md), +> [manifest-system.md](../spec/manifest-system.md), +> [current-manifest-support-review.md](../spec/current-manifest-support-review.md) ## Summary diff --git a/docs/layout/examples/README.md b/docs/layout/examples/README.md new file mode 100644 index 00000000..a6f95cd6 --- /dev/null +++ b/docs/layout/examples/README.md @@ -0,0 +1,108 @@ +# GitTarget layout examples + +> **design**: concrete repository and configuration examples for the proposed `GitTarget` layout +> model. The API fields shown here are not available in the current release. +> +> Read this beside [`../model.md`](../model.md) and +> [`../api-wave.md`](../../design/gittarget-api-wave.md). The current supported Kustomize +> boundary remains in +> [`../support-boundary/kustomize-support-boundary.md`](../../design/support-boundary/kustomize-support-boundary.md). + +These folders make a review question concrete: what does a `GitTarget` say about a Git folder, and +what files does that statement cause the operator to create or update? + +Every scenario contains: + +- `repository/`: the relevant repository subtree. Each scenario states whether it is the starting + state or the state after the illustrated change. +- `config/`: the `GitTarget` and watcher objects that describe the target. The `GitTarget` files + deliberately use the proposed `useKustomize` and `serializeNamespace` fields, which do not exist + yet, and `spec.suspend`, which does not either. +- `input/`: one representative live object or source event, **as the operator receives it from the + API server** — not as it is written to Git. +- `expected-*.patch`: the exact change proposed for Git. It makes the write boundary reviewable + without asking the reader to infer a file placement from prose. +- `README.md`: the decision the layout makes, the expected Git change, and the boundary that keeps + the example safe. + +Each target needs a `GitProvider` in the target's namespace. See the +[shared prerequisites](prerequisites/README.md) for the minimal provider specimen and the source +cluster rule. Connection credentials are outside the layout decision. A source `ClusterProvider` +is named only where that choice explains the scenario. + +| Scenario | Question it exercises | +|---|---| +| [Brownfield kustomize adoption](brownfield-kustomize/README.md) | How does an existing folder become a target without a first write? | +| [Empty-repo bootstrap](empty-repo-bootstrap/README.md) | How does an empty repository become a deployable, single-namespace folder? | +| [Tree, multi-namespace](tree-multi-namespace/README.md) | How does a small cluster mirror several namespaces and cluster resources safely? | +| [Homelab Argo CD](homelab-argocd/README.md) | How does an app-of-apps folder remain a narrow, editable target? | +| [Homelab Flux](homelab-flux/README.md) | How do Flux declarations stay editable without treating chart output as source? | +| [Overlay-scoped target](overlay-scoped-target/README.md) | How can an environment overlay change a supported field without claiming ownership of its base? **Write path, not placement** | + +## Two fixture conventions + +**The inputs are live objects, and the difference is the assertion.** A captured object carries +`uid`, `resourceVersion`, `generation`, `creationTimestamp`, `managedFields`, a populated `status`, +and often a finalizer or a controller's own annotation. None of that reaches Git. So the diff +between an `input/` fixture and its `expected-*.patch` is not decoration: it *is* the sanitization +assertion, and a corpus harness that reads both gets it without a separate test. + +[`homelab-argocd`](homelab-argocd/README.md) and [`homelab-flux`](homelab-flux/README.md) keep the +full captured shape, because those are the two a Flux or Argo reviewer will look at hardest — the +first carries `argocd.argoproj.io/tracking-id`, whose leaking into Git hard-fails another +Application's sync, and the second carries `finalizers.fluxcd.io`. The other four scenarios keep +abridged inputs and say so at the top of the file. + +**The patches carry no `index` lines.** A hash in a patch header is a promise about blob contents, +and a fabricated one gives `git diff` output's authority without its guarantee — a reader who tries +to `git apply` it gets a confusing failure. `diff --git`, the `---`/`+++` pair, the mode line, and +the hunks are the reviewable content. Every patch here applies cleanly to its scenario's stated +starting state, which is the property the index line was pretending to have. + +## How to read the proposed fields + +Two optional booleans are proposed as new members of the `spec.placement` the current release +already has. Everything else in these scenarios ships today, and most scenarios set neither flag. + +`useKustomize` decides what happens when **no** kustomization governs the path: `true` creates one +at `spec.path` and registers the new file in it. When a kustomization *does* govern the path, the +new file joins its `resources:` either way — that is an invariant, not a setting, which is why +[brownfield-kustomize](brownfield-kustomize/README.md) adopts a kustomize folder with no +configuration at all. + +`serializeNamespace` decides whether `metadata.namespace` appears inside the committed document. A +path cannot express it: kustomize takes the namespace from either the document or a governing root, +and where the file sits decides neither. Left unset it is inferred, which is today's behavior and +what most folders want. `true` is the flat-folder answer — nothing downstream supplies a namespace, +so every document carries its own. `false` is honest only when something guarantees the namespace, +which the post-scan pass re-checks on every scan, and which +[empty-repo-bootstrap](empty-repo-bootstrap/README.md) makes provable by having the operator write +the root that supplies it. + +Where a new file goes is the template's job, and most scenarios here declare no template at all: the +built-in ladder places a new document beside the folder's one kustomize root, or at the canonical +identity path when there is no root. The file is named `{name}.yaml`. The operator does not copy a +naming convention from the neighbouring files — that inference was deliberately deleted — so a folder +of `deployment-web.yaml` and `service-web.yaml` receives `cache.yaml`, and +`placement.default: "{kindLower}-{name}.yaml"` is how to ask for the other convention on purpose. + +`suspend: true` is the adoption path: a suspended target still scans and publishes what it +resolved, and writes nothing. Clear it only after the resolved root and prospective paths match the +repository owner's intent. + +Every brownfield scenario here opens suspended and shows the observed status before it shows a +patch, because that is the path the paragraph above describes and a set of examples that opened +writing would be advertising something else. +[`empty-repo-bootstrap`](empty-repo-bootstrap/README.md) is the one honest exception: its repository +is empty, so there is nothing to observe. + +## One product, three names + +A reader meets all three inside a single scenario, so it is worth defusing once: `configbutler.ai` +is the API group, **GitOps Reverser** is the product, and `gitops-reverser` is the repository and +binary name. Flux is uniformly `*.toolkit.fluxcd.io` and pays nothing for it; we have a split, and +these examples are where it is most visible. + +The examples do not make Kustomize a general source inverse. They stay inside the support +boundary: one writable expression, a local render root, and a re-rendered proposal. See the +[support contract](../../design/support-boundary/support-contract.md) for the authoritative limits. diff --git a/docs/layout/examples/brownfield-kustomize/README.md b/docs/layout/examples/brownfield-kustomize/README.md new file mode 100644 index 00000000..df675abe --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/README.md @@ -0,0 +1,81 @@ +# Brownfield Kustomize adoption + +This scenario adopts an existing application folder before it writes. The folder already has one +supported Kustomize root and one namespace convention, so `Auto` can resolve to `Kustomize` during +an observation pass. + +## Starting repository + +[`repository/`](repository/) is an existing `demo` application folder. Its +[`kustomization.yaml`](repository/kustomization.yaml) supplies `namespace: demo` and lists the +Deployment and Service that it owns. + +```text +apps/demo/ + kustomization.yaml + deployment-web.yaml + service-web.yaml +``` + +## Proposed configuration + +[`config/gittarget.yaml`](config/gittarget.yaml) opens suspended and declares no `placement` at +all, and that is the scenario: the built-in ladder applies, the folder's one kustomize root places +new files beside it, and each new file joins that root's `resources:`. Adopting a kustomize folder +needs no configuration — neither proposed flag appears here. +[`config/watchrule.yaml`](config/watchrule.yaml) supplies the namespaced resource subscription. + +The first observation records a result equivalent to: + +```yaml +status: + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + placement: + renderRoot: . + serializeNamespace: false # inferred: the root supplies namespace: demo +``` + +No commit is made while the target is suspended. After a reviewer confirms that result, clearing +`suspend` lets a new ConfigMap named `cache` produce this commit: + +```text +apps/demo/ + kustomization.yaml # adds cache.yaml to resources + cache.yaml # omits metadata.namespace + deployment-web.yaml + service-web.yaml +``` + +The Kustomize root supplies the omitted namespace, so the new document still represents +`demo/cache`. The first write has a home that the existing root reaches. + +## Scenario contract + +- Starting repository: [`repository/`](repository/), already recognized while suspended. +- Live input: [`input/cache.yaml`](input/cache.yaml). +- Expected Git change: [`expected-cache.patch`](expected-cache.patch), after a + reviewer clears `suspend`. +- Expected status: `Ready=True` with `LayoutResolved=True` and `renderRoot: .` during observation. +- Boundary: a second Kustomize root or an unsupported expression makes the target refuse the write. + +## What this example rules out + +This target does not infer a custom file convention from its siblings, and the new file's name is +where that shows. The folder holds `deployment-web.yaml` and `service-web.yaml`, so a reader might +expect `configmap-cache.yaml`. The operator writes **`cache.yaml`**: the rung names a new sibling +`{name}.yaml` and does not learn a naming convention by looking at the neighbours. That inference +was deliberately deleted, and declaring `placement.default: "{kindLower}-{name}.yaml"` is how you +ask for the other convention on purpose. + +A second kustomize root in the target path is a misconfiguration of the GitTarget rather than a +placement puzzle; it does not cause the operator to pick an arbitrary root. A folder with no +kustomize root falls through to the canonical identity path. + +The example stays within the +[Kustomize support boundary](../../../design/support-boundary/kustomize-support-boundary.md): it edits local +declarations and verifies the resulting render. It does not invert generators, plugins, remote +bases, or chart output. diff --git a/docs/layout/examples/brownfield-kustomize/config/gittarget.yaml b/docs/layout/examples/brownfield-kustomize/config/gittarget.yaml new file mode 100644 index 00000000..d0871f54 --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/config/gittarget.yaml @@ -0,0 +1,14 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: demo-workloads + namespace: demo +spec: + providerRef: + name: app-repository + branch: main + path: apps/demo + suspend: true # adoption dry run: scans and publishes, writes nothing + # Nothing else is declared, and that is the point: the folder already has a + # kustomization.yaml, so new files are placed beside it and registered in its + # resources: automatically. Adoption needs no configuration. diff --git a/docs/layout/examples/brownfield-kustomize/config/watchrule.yaml b/docs/layout/examples/brownfield-kustomize/config/watchrule.yaml new file mode 100644 index 00000000..9e147528 --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/config/watchrule.yaml @@ -0,0 +1,15 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: demo-content + namespace: demo +spec: + targetRef: + name: demo-workloads + rules: + - apiGroups: [""] + apiVersions: ["v1"] + resources: ["configmaps", "services"] + - apiGroups: ["apps"] + apiVersions: ["v1"] + resources: ["deployments"] diff --git a/docs/layout/examples/brownfield-kustomize/expected-cache.patch b/docs/layout/examples/brownfield-kustomize/expected-cache.patch new file mode 100644 index 00000000..2d86a066 --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/expected-cache.patch @@ -0,0 +1,19 @@ +diff --git a/apps/demo/kustomization.yaml b/apps/demo/kustomization.yaml +--- a/apps/demo/kustomization.yaml ++++ b/apps/demo/kustomization.yaml +@@ -4,3 +4,4 @@ kind: Kustomization + resources: + - deployment-web.yaml + - service-web.yaml ++ - cache.yaml +diff --git a/apps/demo/cache.yaml b/apps/demo/cache.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/demo/cache.yaml +@@ -0,0 +1,6 @@ ++apiVersion: v1 ++kind: ConfigMap ++metadata: ++ name: cache ++data: ++ address: redis.demo.svc.cluster.local:6379 diff --git a/docs/layout/examples/brownfield-kustomize/input/cache.yaml b/docs/layout/examples/brownfield-kustomize/input/cache.yaml new file mode 100644 index 00000000..beaecda5 --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/input/cache.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: cache + namespace: demo +data: + address: redis.demo.svc.cluster.local:6379 diff --git a/docs/layout/examples/brownfield-kustomize/repository/deployment-web.yaml b/docs/layout/examples/brownfield-kustomize/repository/deployment-web.yaml new file mode 100644 index 00000000..bc80901d --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/repository/deployment-web.yaml @@ -0,0 +1,19 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: web +spec: + replicas: 2 + selector: + matchLabels: + app: web + template: + metadata: + labels: + app: web + spec: + containers: + - name: web + image: nginx:1.27.4 + ports: + - containerPort: 80 diff --git a/docs/layout/examples/brownfield-kustomize/repository/kustomization.yaml b/docs/layout/examples/brownfield-kustomize/repository/kustomization.yaml new file mode 100644 index 00000000..ef97cd1c --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/repository/kustomization.yaml @@ -0,0 +1,6 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: demo +resources: + - deployment-web.yaml + - service-web.yaml diff --git a/docs/layout/examples/brownfield-kustomize/repository/service-web.yaml b/docs/layout/examples/brownfield-kustomize/repository/service-web.yaml new file mode 100644 index 00000000..56c6121f --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/repository/service-web.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Service +metadata: + name: web +spec: + selector: + app: web + ports: + - port: 80 + targetPort: 80 diff --git a/docs/layout/examples/empty-repo-bootstrap/README.md b/docs/layout/examples/empty-repo-bootstrap/README.md new file mode 100644 index 00000000..b9511b20 --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/README.md @@ -0,0 +1,61 @@ +# Application configuration as KRM + +This scenario starts with an empty Git repository. A team edits its own configuration CRDs in an +intent cluster, and GitOps Reverser creates the folder that a deployer consumes. + +## Resulting repository folder + +[`repository/`](repository/) is the result after the first `ShopConfiguration` write. The target +creates the root and registers the first document, so the first commit is already a Kustomize +folder: + +```text +apps/shop/ + kustomization.yaml + storefront.yaml +``` + +The root supplies `namespace: shop`. The Git document omits `metadata.namespace`, while the live +input carries it. The layout establishes that convention at the root rather than waiting for a first +event to do so. + +## Proposed configuration + +[`config/gittarget.yaml`](config/gittarget.yaml) sets `useKustomize: true` and +`serializeNamespace: false`. It also constrains the source to one exact namespace. +[`config/watchrule.yaml`](config/watchrule.yaml) is the artifact manifest: it states which KRM +types the team wants included in the folder. + +The type in this example, `config.shop.example/v1alpha1`, stands for an application configuration +CRD installed in the intent cluster. It is a specimen, not a requirement of GitOps Reverser. + +## Scenario contract + +- Starting repository: empty at `apps/shop`. +- Live input: [`input/storefront.yaml`](input/storefront.yaml). +- Expected Git change: [`expected-first-write.patch`](expected-first-write.patch). +- Expected status: `Ready=True` after the created root renders the captured KRM object. +- Boundary: a source outside `shop`, an unauthorized namespace, or an unsupported KRM expression is + refused before a commit. + +## Why the two flags belong together here + +This is the one pairing in the model that closes its own loop, and it is why creating a root is +worth building at all. `serializeNamespace: false` is only honest when something guarantees the +namespace, and on an empty folder there is nothing to inspect: no kustomization exists to inherit a +convention from, so inference structurally cannot answer. `useKustomize: true` supplies the missing +half — the operator writes the `kustomization.yaml`, puts `namespace: shop` in it, and then +legitimately omits `metadata.namespace` from every document it places. That is also what makes the +created root **meaningful** rather than an empty file: the convention is established rather than +guessed. + +The folder is single-namespace by construction rather than by declaration: no `{namespace}` appears +in any path, because no `placement` is declared and the created root places files beside itself. A +second source namespace has nowhere to go that would not collide, and nothing on the target says so. +The WatchRule feeding it is namespaced and names its own source namespace, which is the whole fence +after +[`source-scope-simplification.md`](../../../design/source-scope-simplification.md). + +This example is the configuration-as-data path from +[the direction review](../../../future/direction-and-configuration-surface.md). The folder is a +reviewable artifact; the KRM objects in the intent cluster are the editing surface. diff --git a/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml b/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml new file mode 100644 index 00000000..46be08ae --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml @@ -0,0 +1,19 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: shop-configuration + namespace: shop +spec: + providerRef: + name: artifacts-repository + clusterProviderRef: + name: app-intent + branch: main + path: apps/shop + placement: + # The folder does not exist yet, so the root that supplies the namespace has to + # be created before the namespace can be left out. useKustomize is what makes + # the omission provable rather than trusted: the operator owns the + # kustomization.yaml it writes namespace: shop into. + useKustomize: true + serializeNamespace: false diff --git a/docs/layout/examples/empty-repo-bootstrap/config/watchrule.yaml b/docs/layout/examples/empty-repo-bootstrap/config/watchrule.yaml new file mode 100644 index 00000000..c098509f --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/config/watchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: shop-configuration-content + namespace: shop +spec: + targetRef: + name: shop-configuration + rules: + - apiGroups: ["config.shop.example"] + apiVersions: ["v1alpha1"] + resources: ["shopconfigurations"] diff --git a/docs/layout/examples/empty-repo-bootstrap/expected-first-write.patch b/docs/layout/examples/empty-repo-bootstrap/expected-first-write.patch new file mode 100644 index 00000000..18d523ae --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/expected-first-write.patch @@ -0,0 +1,26 @@ +diff --git a/apps/shop/kustomization.yaml b/apps/shop/kustomization.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/shop/kustomization.yaml +@@ -0,0 +1,5 @@ ++apiVersion: kustomize.config.k8s.io/v1beta1 ++kind: Kustomization ++namespace: shop ++resources: ++ - storefront.yaml +diff --git a/apps/shop/storefront.yaml b/apps/shop/storefront.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/shop/storefront.yaml +@@ -0,0 +1,11 @@ ++apiVersion: config.shop.example/v1alpha1 ++kind: ShopConfiguration ++metadata: ++ name: storefront ++spec: ++ catalog: ++ defaultCurrency: EUR ++ checkout: ++ sessionTimeout: 15m ++ theme: ++ primaryColor: teal diff --git a/docs/layout/examples/empty-repo-bootstrap/input/storefront.yaml b/docs/layout/examples/empty-repo-bootstrap/input/storefront.yaml new file mode 100644 index 00000000..3fea2632 --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/input/storefront.yaml @@ -0,0 +1,15 @@ +# Abridged: a captured object also carries uid, resourceVersion, generation, +# creationTimestamp, managedFields and status. See ../homelab-argocd/input/ for a +# fixture that keeps them, and ../README.md for why only two scenarios do. +apiVersion: config.shop.example/v1alpha1 +kind: ShopConfiguration +metadata: + name: storefront + namespace: shop +spec: + catalog: + defaultCurrency: EUR + checkout: + sessionTimeout: 15m + theme: + primaryColor: teal diff --git a/docs/layout/examples/empty-repo-bootstrap/repository/kustomization.yaml b/docs/layout/examples/empty-repo-bootstrap/repository/kustomization.yaml new file mode 100644 index 00000000..334fb27d --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/repository/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: shop +resources: + - storefront.yaml diff --git a/docs/layout/examples/empty-repo-bootstrap/repository/storefront.yaml b/docs/layout/examples/empty-repo-bootstrap/repository/storefront.yaml new file mode 100644 index 00000000..426ab08f --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/repository/storefront.yaml @@ -0,0 +1,11 @@ +apiVersion: config.shop.example/v1alpha1 +kind: ShopConfiguration +metadata: + name: storefront +spec: + catalog: + defaultCurrency: EUR + checkout: + sessionTimeout: 15m + theme: + primaryColor: teal diff --git a/docs/layout/examples/homelab-argocd/README.md b/docs/layout/examples/homelab-argocd/README.md new file mode 100644 index 00000000..374d6ab1 --- /dev/null +++ b/docs/layout/examples/homelab-argocd/README.md @@ -0,0 +1,96 @@ +# Homelab Argo CD + +This scenario captures a homelab's Argo CD `Application` declarations. The Git target owns one +app-of-apps folder, and the watched objects all live in Argo CD's `argocd` namespace. + +## Repository folder + +[`repository/`](repository/) contains one Kustomize root and two `Application` documents. A new +Application created in the Argo CD UI receives a sibling file such as `paperless.yaml` +and an entry in the root's `resources:` list. + +The tree below shows the target's path in the real repository; +[`repository/`](repository/) in this folder **is** `bootstrap/argocd-applications/`. + +```text +bootstrap/argocd-applications/ + kustomization.yaml + application-jellyfin.yaml + application-nextcloud.yaml +``` + +The folder is deliberately narrow. It contains the Argo CD declarations that tell Argo what to +deploy; it does not contain the Deployments, Services, or chart-rendered objects that Argo creates. + +## Proposed configuration + +[`config/gittarget.yaml`](config/gittarget.yaml) sets `serializeNamespace: false` and constrains +the source to one exact namespace. +[`config/watchrule.yaml`](config/watchrule.yaml) subscribes only to `argoproj.io` `Application` +objects. The folder's kustomize root supplies the namespace, so the Application files omit +`metadata.namespace`. + +**This is the scenario the post-scan guard exists for.** The root that makes the omission safe is +[`repository/kustomization.yaml`](repository/kustomization.yaml) — a file the *repository owner* +controls, not the operator. Delete it, or delete its `namespace: argocd` line, and every subsequent +document would land in whatever namespace the applier happens to be pointed at, which is a different +object with the same name. Because `serializeNamespace: false` is an explicit override of a +correctness rule, the pass that runs on every scan re-checks the supplier and reports +`Validated=False` naming the field and what the folder actually contains, rather than letting the +relocation happen silently. + +The scenario makes the write boundary easy to inspect: every new captured application has one +writable home in `bootstrap/argocd-applications`. It cannot land in the repository's application +source directories or in an Argo-generated resource. + +## What the input fixture is for + +[`input/paperless.yaml`](input/paperless.yaml) is the object as the +operator receives it from the API server, not the document it wants in Git. It carries `uid`, +`resourceVersion`, `generation`, `creationTimestamp`, `managedFields`, Argo's +`resources-finalizer.argocd.argoproj.io` finalizer, a populated `status`, and the +`argocd.argoproj.io/tracking-id` annotation. + +None of that appears in [`expected-paperless.patch`](expected-paperless.patch), +and the difference between the two files *is* the sanitization assertion. The tracking-id is the one +worth naming: it identifies the Argo CD Application that owns the live object, so a committed copy +makes the document claim ownership on behalf of another Application and hard-fails that +Application's sync. It is denied by exact key rather than by an `argocd.argoproj.io/` prefix strip, +because the rest of that prefix is user data. See +[the tracking-id landmine](../../../spec/e2e-bi-directional-corner.md#the-tracking-id-landmine). + +## Scenario contract + +- Starting repository: [`repository/`](repository/) with the two application declarations shown. +- Live input: [`input/paperless.yaml`](input/paperless.yaml). +- First observation, before any commit: + + ```yaml + status: + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + placement: + renderRoot: . + serializeNamespace: false + ``` + +- Expected Git change: + [`expected-paperless.patch`](expected-paperless.patch), after a reviewer + clears `suspend`. +- Expected status: `Ready=True` after the root renders the added Application. +- Boundary: only `Application` declarations in `argocd` are eligible. An Argo-created workload has + no writable home in this target. + +## Argo CD behavior + +For a field that both the Argo CD UI and Git can change, the Application's automated sync has +`selfHeal: false`. The Git host also sends a push webhook to Argo CD so a commit is reconciled back +to the cluster. These are the two settings that let one declaration have a live editing path and a +Git reconciliation path; see +[Argo CD and bi-directional GitOps](../../../design/support-boundary/argocd-bi-directional.md). + +This is a declaration-editing scenario. It does not reverse Argo-generated application resources, +nor does it reverse a Helm chart rendered by an Application. diff --git a/docs/layout/examples/homelab-argocd/config/gittarget.yaml b/docs/layout/examples/homelab-argocd/config/gittarget.yaml new file mode 100644 index 00000000..78d75255 --- /dev/null +++ b/docs/layout/examples/homelab-argocd/config/gittarget.yaml @@ -0,0 +1,17 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: argocd-applications + namespace: argocd +spec: + providerRef: + name: homelab-repository + branch: main + path: bootstrap/argocd-applications + suspend: true # adoption dry run: scans and publishes, writes nothing + placement: + # Safe here only because the folder's kustomization.yaml supplies + # namespace: argocd — and that file belongs to the repository owner, not to us. + # The post-scan pass checks that supplier on every scan, so deleting it turns + # the target Validated=False rather than silently relocating documents. + serializeNamespace: false diff --git a/docs/layout/examples/homelab-argocd/config/watchrule.yaml b/docs/layout/examples/homelab-argocd/config/watchrule.yaml new file mode 100644 index 00000000..e2cdda4c --- /dev/null +++ b/docs/layout/examples/homelab-argocd/config/watchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: argocd-applications + namespace: argocd +spec: + targetRef: + name: argocd-applications + rules: + - apiGroups: ["argoproj.io"] + apiVersions: ["v1alpha1"] + resources: ["applications"] diff --git a/docs/layout/examples/homelab-argocd/expected-paperless.patch b/docs/layout/examples/homelab-argocd/expected-paperless.patch new file mode 100644 index 00000000..572e7f2e --- /dev/null +++ b/docs/layout/examples/homelab-argocd/expected-paperless.patch @@ -0,0 +1,30 @@ +diff --git a/bootstrap/argocd-applications/kustomization.yaml b/bootstrap/argocd-applications/kustomization.yaml +--- a/bootstrap/argocd-applications/kustomization.yaml ++++ b/bootstrap/argocd-applications/kustomization.yaml +@@ -4,3 +4,4 @@ kind: Kustomization + resources: + - application-jellyfin.yaml + - application-nextcloud.yaml ++ - paperless.yaml +diff --git a/bootstrap/argocd-applications/paperless.yaml b/bootstrap/argocd-applications/paperless.yaml +new file mode 100644 +--- /dev/null ++++ b/bootstrap/argocd-applications/paperless.yaml +@@ -0,0 +1,17 @@ ++apiVersion: argoproj.io/v1alpha1 ++kind: Application ++metadata: ++ name: paperless ++spec: ++ project: default ++ source: ++ repoURL: https://github.com/example/homelab.git ++ path: apps/paperless ++ targetRevision: main ++ destination: ++ server: https://kubernetes.default.svc ++ namespace: paperless ++ syncPolicy: ++ automated: ++ prune: true ++ selfHeal: false diff --git a/docs/layout/examples/homelab-argocd/input/paperless.yaml b/docs/layout/examples/homelab-argocd/input/paperless.yaml new file mode 100644 index 00000000..585b9b6c --- /dev/null +++ b/docs/layout/examples/homelab-argocd/input/paperless.yaml @@ -0,0 +1,47 @@ +# The object as the operator receives it from the API server, not as it is written to +# Git. Everything above `spec` except `name` is removed by sanitization, which is why +# expected-application-paperless.patch is shorter than this file: the difference +# between the two is the sanitization assertion. +# +# The tracking-id annotation is the one that matters most here. It identifies the Argo +# CD Application that owns the live object, so committing it would make this document +# claim ownership on behalf of another Application and hard-fail that Application's +# sync. Sanitization denies it by exact key rather than by an `argocd.argoproj.io/` +# prefix strip, because the rest of that prefix is user data. +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: paperless + namespace: argocd + uid: 4d9f1e2c-6a1b-4c9e-9f2a-1b3c5d7e9f01 + resourceVersion: "184203" + generation: 2 + creationTimestamp: "2026-08-12T09:41:07Z" + finalizers: + - resources-finalizer.argocd.argoproj.io + annotations: + argocd.argoproj.io/tracking-id: paperless:argoproj.io/Application:argocd/paperless + managedFields: + - manager: argocd-server + operation: Update + apiVersion: argoproj.io/v1alpha1 + time: "2026-08-12T09:41:07Z" +spec: + project: default + source: + repoURL: https://github.com/example/homelab.git + path: apps/paperless + targetRevision: main + destination: + server: https://kubernetes.default.svc + namespace: paperless + syncPolicy: + automated: + prune: true + selfHeal: false +status: + sync: + status: Synced + revision: 7b1c4a9 + health: + status: Healthy diff --git a/docs/layout/examples/homelab-argocd/repository/application-jellyfin.yaml b/docs/layout/examples/homelab-argocd/repository/application-jellyfin.yaml new file mode 100644 index 00000000..3479d52f --- /dev/null +++ b/docs/layout/examples/homelab-argocd/repository/application-jellyfin.yaml @@ -0,0 +1,17 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: jellyfin +spec: + project: default + source: + repoURL: https://github.com/example/homelab-apps.git + targetRevision: main + path: media/jellyfin + destination: + server: https://kubernetes.default.svc + namespace: media + syncPolicy: + automated: + prune: true + selfHeal: false diff --git a/docs/layout/examples/homelab-argocd/repository/application-nextcloud.yaml b/docs/layout/examples/homelab-argocd/repository/application-nextcloud.yaml new file mode 100644 index 00000000..93d35afa --- /dev/null +++ b/docs/layout/examples/homelab-argocd/repository/application-nextcloud.yaml @@ -0,0 +1,17 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: nextcloud +spec: + project: default + source: + repoURL: https://github.com/example/homelab-apps.git + targetRevision: main + path: storage/nextcloud + destination: + server: https://kubernetes.default.svc + namespace: storage + syncPolicy: + automated: + prune: true + selfHeal: false diff --git a/docs/layout/examples/homelab-argocd/repository/kustomization.yaml b/docs/layout/examples/homelab-argocd/repository/kustomization.yaml new file mode 100644 index 00000000..502b770f --- /dev/null +++ b/docs/layout/examples/homelab-argocd/repository/kustomization.yaml @@ -0,0 +1,6 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: argocd +resources: + - application-jellyfin.yaml + - application-nextcloud.yaml diff --git a/docs/layout/examples/homelab-flux/README.md b/docs/layout/examples/homelab-flux/README.md new file mode 100644 index 00000000..c73523ab --- /dev/null +++ b/docs/layout/examples/homelab-flux/README.md @@ -0,0 +1,130 @@ +# Homelab Flux + +This scenario captures Flux declarations in the `flux-system` namespace. A homelab owner can edit a +`GitRepository`, `HelmRepository`, or `HelmRelease` through the Kubernetes API and review the +resulting Git change without asking GitOps Reverser to reverse a chart. + +## The rule this scenario exists to demonstrate + +> A `GitTarget` must not point at a path another controller writes. Flux's bootstrap directory +> (`clusters//flux-system`) is the common case, and the `Kustomization` that reconciles a +> folder is not a licence to co-write it. + +`clusters/home/flux-system` is owned by `flux bootstrap` and by flux-operator: it holds +`gotk-components.yaml`, `gotk-sync.yaml`, and a `kustomization.yaml` listing both. An operator that +adds `resources:` entries there is a second writer in a folder Flux's own sync loop reconciles, +which is the two-writers-one-folder failure the +[support contract](../../../design/support-boundary/support-contract.md) exists to prevent. The +targets below point somewhere else, and the bootstrap directory appears in the tree only to be left +alone. + +## Repository folder + +[`repository/`](repository/) is rooted at the repository root rather than at one target's path, +because this scenario has two targets on two layers: + +```text +clusters/home/flux-system/ # flux bootstrap owns this. GitOps Reverser never writes here. + gotk-components.yaml + gotk-sync.yaml + kustomization.yaml + +infrastructure/home/sources/ # spec.path of the flux-sources GitTarget + kustomization.yaml + gitrepository-homelab.yaml + helmrepository-jellyfin.yaml + +apps/home/media/ # spec.path of the flux-media GitTarget + kustomization.yaml + helmrelease-jellyfin.yaml +``` + +The bootstrap directory is shown but not committed to this fixture, because nothing in the scenario +reads or writes it. The split into `infrastructure/` and `apps/` is the shape Flux's documented +repository structures produce, and keeping the cluster layer separate from the application layer is +what those structures exist to achieve. + +For a newly created Flux declaration, the folder's kustomize root places a sibling file and +registers it in `resources:`. It does not append the declaration to an existing bundle, because a +bundle is only produced by a template that deliberately collides two identities on one path, and +none is declared here. + +**The update case is the more interesting half.** When the `jellyfin` `HelmRepository` that already +lives in the folder changes, it is not moved and not re-placed: match-first identity finds the +document where it is and edits it in place, including when it shares a file with other documents. +So "we do not author bundles" and "we do not move existing files" coexist — placement decides where +a *new* document goes and has no opinion about one that already exists. A user who consolidates two +declarations into one file keeps that choice; the operator will edit inside it forever. + +## Proposed configuration + +Two targets, one per layer: + +| Target | Path | Watches | +|---|---|---| +| [`config/gittarget.yaml`](config/gittarget.yaml) | `infrastructure/home/sources` | `GitRepository`, `HelmRepository` ([`config/watchrule.yaml`](config/watchrule.yaml)) | +| [`config/gittarget-media.yaml`](config/gittarget-media.yaml) | `apps/home/media` | `HelmRelease` ([`config/watchrule-media.yaml`](config/watchrule-media.yaml)) | + +Both declare a one-root Kustomize folder with one source namespace, and both open suspended. Each +folder's `namespace: flux-system` transformer supplies the namespace for every +document in it, so the committed documents omit `metadata.namespace`. + +The `HelmRelease` object living in `flux-system` does not put jellyfin there: +[`helmrelease-jellyfin.yaml`](repository/apps/home/media/helmrelease-jellyfin.yaml) carries +`targetNamespace: media` and `storageNamespace: flux-system`. The declaration and the release it +installs are on different sides of that field, which is exactly why the declaration is the editable +surface and the release is not. + +## What the input fixture is for + +[`input/bitnami.yaml`](input/bitnami.yaml) is the object as the +operator receives it from the API server, not the document it wants in Git. It carries `uid`, +`resourceVersion`, `generation`, `creationTimestamp`, `managedFields`, a populated `status`, and +Flux's `finalizers.fluxcd.io` finalizer. + +None of that reaches +[`expected-bitnami.patch`](expected-bitnami.patch), and the difference +between the two files *is* the sanitization assertion. The finalizer is the one worth naming: +`finalizers.fluxcd.io` is how source-controller keeps the object alive long enough to garbage-collect +its artifact storage. A committed copy would make Git assert a finalizer against an object Flux has +not adopted yet, so deleting a recreated object would block on a controller that never put it there. + +## Scenario contract + +- Starting repository: [`repository/`](repository/) with its existing sources and media folders. +- Live input: [`input/bitnami.yaml`](input/bitnami.yaml). +- First observation, before any commit: + + ```yaml + status: + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + placement: + renderRoot: . + serializeNamespace: false + ``` + +- Expected Git change: + [`expected-bitnami.patch`](expected-bitnami.patch), after a reviewer clears `suspend` on + `flux-sources`. +- Expected status: `Ready=True` after the root renders the new declaration. +- Boundary: a rendered object has no writable home; only selected Flux declarations can produce a + Git change. + +The configuration captures the layer a person would edit in Git: + +- a `GitRepository` or `HelmRepository` changes where Flux obtains source material; +- a `HelmRelease` changes a chart version, source reference, or inline values; +- the chart's rendered Deployments, Services, and Secrets remain expansion output. + +## Boundary + +This is Flux declaration editing, not Helm inversion. A chart folder is skipped as a unit, and the +operator never turns a rendered Deployment edit into a speculative values change. The current +[support contract](../../../design/support-boundary/support-contract.md) owns that boundary. + +A free-standing values file is a separate planned projection, so it is absent from this first +scenario. Inline `HelmRelease.spec.values` are KRM and stay inside the declaration surface. diff --git a/docs/layout/examples/homelab-flux/config/gittarget-media.yaml b/docs/layout/examples/homelab-flux/config/gittarget-media.yaml new file mode 100644 index 00000000..161272e8 --- /dev/null +++ b/docs/layout/examples/homelab-flux/config/gittarget-media.yaml @@ -0,0 +1,17 @@ +# The second target. HelmReleases are a different layer from the sources they pull +# from, so they get their own folder and their own GitTarget rather than sharing one +# with the sources — which is the shape Flux's documented repository structures exist +# to produce. +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: flux-media + namespace: flux-system +spec: + providerRef: + name: homelab-repository + branch: main + path: apps/home/media + suspend: true # adoption dry run: scans and publishes, writes nothing + placement: + serializeNamespace: false # the folder's own kustomization.yaml supplies it diff --git a/docs/layout/examples/homelab-flux/config/gittarget.yaml b/docs/layout/examples/homelab-flux/config/gittarget.yaml new file mode 100644 index 00000000..22bd21f2 --- /dev/null +++ b/docs/layout/examples/homelab-flux/config/gittarget.yaml @@ -0,0 +1,15 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: flux-sources + namespace: flux-system +spec: + providerRef: + name: homelab-repository + branch: main + # Not clusters/home/flux-system. That directory belongs to `flux bootstrap`, and a + # GitTarget must not point at a path another controller writes. + path: infrastructure/home/sources + suspend: true # adoption dry run: scans and publishes, writes nothing + placement: + serializeNamespace: false # the folder's own kustomization.yaml supplies it diff --git a/docs/layout/examples/homelab-flux/config/watchrule-media.yaml b/docs/layout/examples/homelab-flux/config/watchrule-media.yaml new file mode 100644 index 00000000..e236aa0e --- /dev/null +++ b/docs/layout/examples/homelab-flux/config/watchrule-media.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: flux-media + namespace: flux-system +spec: + targetRef: + name: flux-media + rules: + - apiGroups: ["helm.toolkit.fluxcd.io"] + apiVersions: ["v2"] + resources: ["helmreleases"] diff --git a/docs/layout/examples/homelab-flux/config/watchrule.yaml b/docs/layout/examples/homelab-flux/config/watchrule.yaml new file mode 100644 index 00000000..5f04fa39 --- /dev/null +++ b/docs/layout/examples/homelab-flux/config/watchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: flux-sources + namespace: flux-system +spec: + targetRef: + name: flux-sources + rules: + - apiGroups: ["source.toolkit.fluxcd.io"] + apiVersions: ["v1"] + resources: ["gitrepositories", "helmrepositories"] diff --git a/docs/layout/examples/homelab-flux/expected-bitnami.patch b/docs/layout/examples/homelab-flux/expected-bitnami.patch new file mode 100644 index 00000000..0c05e0a4 --- /dev/null +++ b/docs/layout/examples/homelab-flux/expected-bitnami.patch @@ -0,0 +1,21 @@ +diff --git a/infrastructure/home/sources/kustomization.yaml b/infrastructure/home/sources/kustomization.yaml +--- a/infrastructure/home/sources/kustomization.yaml ++++ b/infrastructure/home/sources/kustomization.yaml +@@ -3,4 +3,5 @@ kind: Kustomization + namespace: flux-system + resources: + - gitrepository-homelab.yaml + - helmrepository-jellyfin.yaml ++ - bitnami.yaml +diff --git a/infrastructure/home/sources/bitnami.yaml b/infrastructure/home/sources/bitnami.yaml +new file mode 100644 +--- /dev/null ++++ b/infrastructure/home/sources/bitnami.yaml +@@ -0,0 +1,7 @@ ++apiVersion: source.toolkit.fluxcd.io/v1 ++kind: HelmRepository ++metadata: ++ name: bitnami ++spec: ++ interval: 1h ++ url: https://charts.bitnami.com/bitnami diff --git a/docs/layout/examples/homelab-flux/input/bitnami.yaml b/docs/layout/examples/homelab-flux/input/bitnami.yaml new file mode 100644 index 00000000..2a7e7e2b --- /dev/null +++ b/docs/layout/examples/homelab-flux/input/bitnami.yaml @@ -0,0 +1,40 @@ +# The object as the operator receives it from the API server. Everything above `spec` +# except `name` is removed by sanitization, so the committed document in +# expected-helmrepository-bitnami.patch is much shorter than this file: the difference +# between the two is the sanitization assertion. +# +# The Flux finalizer is the one worth naming. `finalizers.fluxcd.io` is how +# source-controller keeps the object alive long enough to garbage-collect its artifact +# storage. A committed copy would make Git assert a finalizer against an object Flux +# has not adopted yet, so a `flux delete` of a recreated object would block on a +# controller that never put it there. +apiVersion: source.toolkit.fluxcd.io/v1 +kind: HelmRepository +metadata: + name: bitnami + namespace: flux-system + uid: 8c2a71f4-3d5e-4b18-9c07-2e6f1a4b8d33 + resourceVersion: "902441" + generation: 1 + creationTimestamp: "2026-08-19T14:02:55Z" + finalizers: + - finalizers.fluxcd.io + managedFields: + - manager: flux + operation: Apply + apiVersion: source.toolkit.fluxcd.io/v1 + time: "2026-08-19T14:02:55Z" +spec: + interval: 1h + url: https://charts.bitnami.com/bitnami +status: + observedGeneration: 1 + artifact: + revision: sha256:3f9c1e0d + lastUpdateTime: "2026-08-19T14:03:02Z" + conditions: + - type: Ready + status: "True" + reason: Succeeded + message: stored artifact for revision 'sha256:3f9c1e0d' + lastTransitionTime: "2026-08-19T14:03:02Z" diff --git a/docs/layout/examples/homelab-flux/repository/apps/home/media/helmrelease-jellyfin.yaml b/docs/layout/examples/homelab-flux/repository/apps/home/media/helmrelease-jellyfin.yaml new file mode 100644 index 00000000..b3f7f8ae --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/apps/home/media/helmrelease-jellyfin.yaml @@ -0,0 +1,25 @@ +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: jellyfin +spec: + interval: 30m + # The HelmRelease object lives in flux-system with the rest of the Flux + # declarations; the release it installs does not. targetNamespace is what puts + # jellyfin in `media`, and storageNamespace keeps its Helm storage beside the + # controller that manages it. + targetNamespace: media + storageNamespace: flux-system + chart: + spec: + chart: jellyfin + version: 2.1.0 + sourceRef: + kind: HelmRepository + name: jellyfin + values: + persistence: + config: + enabled: true + service: + type: ClusterIP diff --git a/docs/layout/examples/homelab-flux/repository/apps/home/media/kustomization.yaml b/docs/layout/examples/homelab-flux/repository/apps/home/media/kustomization.yaml new file mode 100644 index 00000000..8bcda0c3 --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/apps/home/media/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: flux-system +resources: + - helmrelease-jellyfin.yaml diff --git a/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/gitrepository-homelab.yaml b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/gitrepository-homelab.yaml new file mode 100644 index 00000000..bdb644d8 --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/gitrepository-homelab.yaml @@ -0,0 +1,7 @@ +apiVersion: source.toolkit.fluxcd.io/v1 +kind: GitRepository +metadata: + name: homelab +spec: + interval: 1m + url: https://github.com/example/homelab.git diff --git a/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/helmrepository-jellyfin.yaml b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/helmrepository-jellyfin.yaml new file mode 100644 index 00000000..5d6cc27d --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/helmrepository-jellyfin.yaml @@ -0,0 +1,7 @@ +apiVersion: source.toolkit.fluxcd.io/v1 +kind: HelmRepository +metadata: + name: jellyfin +spec: + interval: 1h + url: https://jellyfin.github.io/jellyfin-helm diff --git a/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/kustomization.yaml b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/kustomization.yaml new file mode 100644 index 00000000..3abed039 --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/kustomization.yaml @@ -0,0 +1,6 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: flux-system +resources: + - gitrepository-homelab.yaml + - helmrepository-jellyfin.yaml diff --git a/docs/layout/examples/overlay-scoped-target/README.md b/docs/layout/examples/overlay-scoped-target/README.md new file mode 100644 index 00000000..1056fdd3 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/README.md @@ -0,0 +1,75 @@ +# Overlay-scoped target + +> **This scenario exercises the write path, not placement.** Its expected patch edits an `images:` +> transformer in a file that already exists: no file is placed, no `resources:` entry is added, and +> the placement ladder never runs. It sits here because the question it answers — what a target may +> write when its render root reads a folder it does not own — is the one people ask immediately after +> the placement question, and because it is the scenario that proves `spec.path` is the write +> boundary. A corpus harness needs a separate entry point for it: seeding a worktree and comparing a +> patch is the same, but the event is a field change rather than a new document. +> +> The folder was called `external-base-overlay`. The base is in the same repository one directory +> up, so nothing about it is external — and "external base" names precisely what the Kustomize +> support boundary refuses, a remote base. The name now says what the scenario is *for*. + +This scenario gives one environment overlay a writable kustomize root while treating its shared +base as read-only input. It captures a Deployment change that the overlay can express as an image +tag update. + +## Starting repository + +[`repository/`](repository/) contains a reusable `podinfo` base and a production overlay: + +```text +apps/podinfo/ + base/ + deployment.yaml + kustomization.yaml + overlays/prod/ + kustomization.yaml +``` + +The target path is `apps/podinfo/overlays/prod`. Its Kustomize root includes `../../base`, but the +base is an input to rendering. It is outside the target's write scope. + +## Proposed configuration + +[`config/gittarget.yaml`](config/gittarget.yaml) declares one `Kustomize` root for the +`podinfo-prod` namespace. [`config/watchrule.yaml`](config/watchrule.yaml) captures Deployments. +The scenario assumes the Deployment image is a supported writable expression in the overlay's +`images:` transformer. + +## Scenario contract + +- Starting repository: + [`repository/apps/podinfo/overlays/prod/`](repository/apps/podinfo/overlays/prod/). +- Live input: [`input/deployment-podinfo.yaml`](input/deployment-podinfo.yaml). +- First observation, before any commit: + + ```yaml + status: + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + placement: + renderRoot: . + serializeNamespace: false + ``` + +- Expected Git change: [`expected-image-update.patch`](expected-image-update.patch), after a + reviewer clears `suspend`. +- Expected status: `Ready=True` after the changed overlay renders successfully. +- Boundary: the target may read `../../base` to render, but it never writes outside the overlay. + +## Boundary + +The target owns the overlay and does not own a shared base. A request that needs to change a base +field is refused as outside `spec.path`; it does not search for another location with the same +Deployment identity. If the overlay cannot express a live field change through a supported +transformer, the operator reports a refusal rather than editing the rendered base. + +The exact support boundary remains the +[Kustomize support contract](../../../design/support-boundary/support-contract.md). This example narrows its +write scope further than that contract: rendering may cross into the base, but writing does not. diff --git a/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml b/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml new file mode 100644 index 00000000..5f01c37a --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml @@ -0,0 +1,13 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: podinfo-prod-overlay + namespace: podinfo-prod +spec: + providerRef: + name: homelab-repository + branch: main + path: apps/podinfo/overlays/prod + suspend: true # adoption dry run: scans and publishes, writes nothing + placement: + serializeNamespace: false # the overlay's kustomization.yaml supplies it diff --git a/docs/layout/examples/overlay-scoped-target/config/watchrule.yaml b/docs/layout/examples/overlay-scoped-target/config/watchrule.yaml new file mode 100644 index 00000000..212617ed --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/config/watchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: podinfo-prod-overlay-content + namespace: podinfo-prod +spec: + targetRef: + name: podinfo-prod-overlay + rules: + - apiGroups: ["apps"] + apiVersions: ["v1"] + resources: ["deployments"] diff --git a/docs/layout/examples/overlay-scoped-target/expected-image-update.patch b/docs/layout/examples/overlay-scoped-target/expected-image-update.patch new file mode 100644 index 00000000..ed9de3a3 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/expected-image-update.patch @@ -0,0 +1,9 @@ +diff --git a/apps/podinfo/overlays/prod/kustomization.yaml b/apps/podinfo/overlays/prod/kustomization.yaml +--- a/apps/podinfo/overlays/prod/kustomization.yaml ++++ b/apps/podinfo/overlays/prod/kustomization.yaml +@@ -5,4 +5,4 @@ resources: + - ../../base + images: + - name: ghcr.io/stefanprodan/podinfo +- newTag: 6.6.0 ++ newTag: 6.6.1 diff --git a/docs/layout/examples/overlay-scoped-target/input/deployment-podinfo.yaml b/docs/layout/examples/overlay-scoped-target/input/deployment-podinfo.yaml new file mode 100644 index 00000000..107b5933 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/input/deployment-podinfo.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: podinfo + namespace: podinfo-prod +spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: podinfo + template: + metadata: + labels: + app.kubernetes.io/name: podinfo + spec: + containers: + - name: podinfo + image: ghcr.io/stefanprodan/podinfo:6.6.1 + ports: + - containerPort: 9898 diff --git a/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/deployment.yaml b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/deployment.yaml new file mode 100644 index 00000000..f2fce614 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/deployment.yaml @@ -0,0 +1,19 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: podinfo +spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: podinfo + template: + metadata: + labels: + app.kubernetes.io/name: podinfo + spec: + containers: + - name: podinfo + image: ghcr.io/stefanprodan/podinfo:6.5.3 + ports: + - containerPort: 9898 diff --git a/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/kustomization.yaml b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/kustomization.yaml new file mode 100644 index 00000000..9c2d28b0 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/kustomization.yaml @@ -0,0 +1,4 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - deployment.yaml diff --git a/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/overlays/prod/kustomization.yaml b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/overlays/prod/kustomization.yaml new file mode 100644 index 00000000..01f1b240 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/overlays/prod/kustomization.yaml @@ -0,0 +1,8 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: podinfo-prod +resources: + - ../../base +images: + - name: ghcr.io/stefanprodan/podinfo + newTag: 6.6.0 diff --git a/docs/layout/examples/prerequisites/README.md b/docs/layout/examples/prerequisites/README.md new file mode 100644 index 00000000..f3f66800 --- /dev/null +++ b/docs/layout/examples/prerequisites/README.md @@ -0,0 +1,38 @@ +# Shared prerequisites + +Every `GitTarget` refers to a `GitProvider` in the same namespace. The provider owns the repository +connection and writable branch policy; the target owns the folder and its layout. This separation +lets one repository serve several independently scoped targets. + +[`config/gitprovider.yaml`](config/gitprovider.yaml) is a minimal SSH provider specimen for the +`demo` namespace. Copy it into each namespace that owns a target, change its name and repository +URL, and provide the referenced credentials and `known_hosts` objects. A public HTTPS repository +can omit `secretRef` and `knownHostsRef`. + +```text +namespace demo + GitProvider/app-repository + GitTarget/demo -> GitProvider/app-repository +``` + +The provider is namespace-local because it carries credentials. A `GitTarget` cannot refer to a +provider in another namespace. + +## Source cluster authority + +The ordinary examples use the operator's in-cluster source. A target that captures objects from a +different source namespace needs its `ClusterProvider` to delegate that override. The cluster-tree +example includes a concrete +[`ClusterProvider`](../tree-multi-namespace/config/clusterprovider.yaml) with +`allowSourceNamespaceOverride: true`. + +The source-cluster rule and source-namespace authorization are separate checks. The +`ClusterProvider` decides which configuration namespaces can choose source namespaces. The +`GitTarget` decides which of those source namespaces may write into its folder. + +## Credentials stay out of this design + +This example names a credentials Secret and `known_hosts` ConfigMap but does not include either +object. Repository credentials and SSH host keys are operational inputs, not a layout convention. +The current setup contract is in +[GitProvider configuration](../../../configuration.md#gitprovider). diff --git a/docs/layout/examples/prerequisites/config/gitprovider.yaml b/docs/layout/examples/prerequisites/config/gitprovider.yaml new file mode 100644 index 00000000..d189a555 --- /dev/null +++ b/docs/layout/examples/prerequisites/config/gitprovider.yaml @@ -0,0 +1,16 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitProvider +metadata: + name: app-repository + namespace: demo +spec: + url: ssh://git@example.com/organization/application-config.git + secretRef: + name: app-repository-credentials + knownHostsRef: + name: example-com-known-hosts + allowedBranches: + - main + commit: + author: + name: GitOps Reverser diff --git a/docs/layout/examples/tree-multi-namespace/README.md b/docs/layout/examples/tree-multi-namespace/README.md new file mode 100644 index 00000000..42dde7f6 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/README.md @@ -0,0 +1,93 @@ +# Homelab cluster tree + +This scenario mirrors a small home cluster into one repository subtree. The operator captures a +few namespaced resources from two namespaces and one cluster-scoped resource. It does not need a +Kustomize root to decide where a new object goes. + +## Resulting repository folder + +[`repository/`](repository/) shows the structure after three writes. The tree below uses the +target's path in the real repository; `repository/` in this folder **is** `clusters/home/`. + +```text +clusters/home/ + _cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml + media/apps/deployments/jellyfin.yaml + media/configmaps/jellyfin.yaml + monitoring/configmaps/grafana.ini.yaml +``` + +`Tree` derives each path from the object's namespace or cluster scope, API group, resource, and +name. A Deployment named `jellyfin` in another namespace receives another path. This property is +why `Tree` is the safe default for multi-namespace capture. + +Two conventions in those paths are load-bearing, and both come from the canonical grammar in +[`new-file-placement-rules.md`](../../new-file-placement-rules.md#template-variables): + +- **The core group collapses to nothing.** `configmaps` sits directly under the namespace segment, + where `rbac.authorization.k8s.io` sits in the ClusterRole path, because the core group has no + name to write. +- **`_cluster` stands in for the namespace segment** of a cluster-scoped resource. An underscore is + invalid in a namespace name, so the sentinel cannot collide with a real namespace — which is why + it is a sentinel and not a reserved word. + +## Proposed configuration + +[`config/clusterprovider.yaml`](config/clusterprovider.yaml) gives the homelab configuration +namespace permission to choose its source namespaces. The target's +[`config/gittarget.yaml`](config/gittarget.yaml) names those namespaces and declares `Tree` plus +`MultiNamespace`. [`config/watchrule.yaml`](config/watchrule.yaml) selects namespaced content, while +[`config/clusterwatchrule.yaml`](config/clusterwatchrule.yaml) separately selects the ClusterRole. + +The separation is intentional. A `WatchRule` owns namespaced subscriptions. A +`ClusterWatchRule` owns cluster-scoped subscriptions. The folder has room for both because its +paths carry scope, but the source API keeps their authority separate. + +## Scenario contract + +- Starting repository: [`repository/`](repository/) after the three documents shown above. +- Live input: [`input/grafana-dashboards.yaml`](input/grafana-dashboards.yaml). +- First observation, before any commit. A `Tree` folder has no render root, so the reason says so + rather than leaving the field unexplained: + + ```yaml + status: + conditions: + - type: LayoutResolved + status: "True" + reason: None + message: "no kustomization governs this subtree; new files take the canonical path" + placement: + renderRoot: "" + serializeNamespace: true + ``` + +- Expected Git change: + [`expected-grafana-dashboards.patch`](expected-grafana-dashboards.patch), after + a reviewer clears `suspend`. +- Expected status: `Ready=True` with a `Tree` layout and a path derived from the input identity. +- Boundary: content outside `clusters/home`, a namespace not on the target, or an unsupported + type is refused without probing for a sibling convention. + +## Homelab use + +This is a useful first mirror for a single-owner cluster: changes made with `kubectl`, a dashboard, +or an operator appear as reviewable Git documents without choosing a bundle convention first. A +GitOps deployer can consume the tree through a recursively scanned directory or through a root +Kustomization that the repository owner maintains. + +`serializeNamespace: true` keeps each namespaced document portable outside this tree. Unlike a +single-namespace Kustomize folder, no common namespace transformer supplies an omitted value. + +**The field governs namespaced resources only.** This folder also captures a `ClusterRole` through +its `ClusterWatchRule`, and a `ClusterRole` has no namespace to serialize, so +`serializeNamespace` is ignored for it rather than being an error. A tree is the shape most likely +to carry both kinds, which is why it is worth saying here. + +## Boundary + +The target has one write scope, `clusters/home`. It does not use one target for a shared base and +several environments. The per-overlay model remains the right choice when a repository has bases +and environment-specific overlays; see [GitTarget granularity]. + +[GitTarget granularity]: ../../../design/support-boundary/gittarget-granularity-and-cross-environment-edits.md diff --git a/docs/layout/examples/tree-multi-namespace/config/clusterprovider.yaml b/docs/layout/examples/tree-multi-namespace/config/clusterprovider.yaml new file mode 100644 index 00000000..576d7d64 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/config/clusterprovider.yaml @@ -0,0 +1,9 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: ClusterProvider +metadata: + name: home +spec: + allowedNamespaces: + names: [homelab-config] + # The WatchRule selects source namespaces other than its own namespace. + allowSourceNamespaceOverride: true diff --git a/docs/layout/examples/tree-multi-namespace/config/clusterwatchrule.yaml b/docs/layout/examples/tree-multi-namespace/config/clusterwatchrule.yaml new file mode 100644 index 00000000..dcaba72d --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/config/clusterwatchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: ClusterWatchRule +metadata: + name: home-cluster-rbac +spec: + targetRef: + name: home-cluster-state + namespace: homelab-config + rules: + - apiGroups: ["rbac.authorization.k8s.io"] + apiVersions: ["v1"] + resources: ["clusterroles"] diff --git a/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml b/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml new file mode 100644 index 00000000..d1d4547a --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml @@ -0,0 +1,18 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: home-cluster-state + namespace: homelab-config +spec: + providerRef: + name: homelab-repository + clusterProviderRef: + name: home + branch: main + path: clusters/home + suspend: true # adoption dry run: scans and publishes, writes nothing + placement: + # No template declared and no kustomization anywhere in the subtree, so the + # ladder falls through to the canonical identity path. Nothing supplies a + # namespace, so every namespaced document has to carry its own. + serializeNamespace: true diff --git a/docs/layout/examples/tree-multi-namespace/config/watchrule.yaml b/docs/layout/examples/tree-multi-namespace/config/watchrule.yaml new file mode 100644 index 00000000..e3d39468 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/config/watchrule.yaml @@ -0,0 +1,21 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: home-workloads + namespace: homelab-config +spec: + targetRef: + name: home-cluster-state + rules: + - apiGroups: [""] + apiVersions: ["v1"] + resources: ["configmaps"] + sourceNamespace: media + - apiGroups: ["apps"] + apiVersions: ["v1"] + resources: ["deployments"] + sourceNamespace: media + - apiGroups: [""] + apiVersions: ["v1"] + resources: ["configmaps"] + sourceNamespace: monitoring diff --git a/docs/layout/examples/tree-multi-namespace/expected-grafana-dashboards.patch b/docs/layout/examples/tree-multi-namespace/expected-grafana-dashboards.patch new file mode 100644 index 00000000..63a0d1fb --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/expected-grafana-dashboards.patch @@ -0,0 +1,12 @@ +diff --git a/clusters/home/monitoring/configmaps/grafana-dashboards.yaml b/clusters/home/monitoring/configmaps/grafana-dashboards.yaml +new file mode 100644 +--- /dev/null ++++ b/clusters/home/monitoring/configmaps/grafana-dashboards.yaml +@@ -0,0 +1,7 @@ ++apiVersion: v1 ++kind: ConfigMap ++metadata: ++ name: grafana-dashboards ++ namespace: monitoring ++data: ++ dashboard.json: '{"title":"Home"}' diff --git a/docs/layout/examples/tree-multi-namespace/input/grafana-dashboards.yaml b/docs/layout/examples/tree-multi-namespace/input/grafana-dashboards.yaml new file mode 100644 index 00000000..9880ac84 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/input/grafana-dashboards.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: grafana-dashboards + namespace: monitoring +data: + dashboard.json: '{"title":"Home"}' diff --git a/docs/layout/examples/tree-multi-namespace/repository/_cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml b/docs/layout/examples/tree-multi-namespace/repository/_cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml new file mode 100644 index 00000000..e7131314 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/repository/_cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml @@ -0,0 +1,8 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: homelab-viewer +rules: + - apiGroups: [""] + resources: ["pods", "services"] + verbs: ["get", "list", "watch"] diff --git a/docs/layout/examples/tree-multi-namespace/repository/media/apps/deployments/jellyfin.yaml b/docs/layout/examples/tree-multi-namespace/repository/media/apps/deployments/jellyfin.yaml new file mode 100644 index 00000000..c3dd580f --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/repository/media/apps/deployments/jellyfin.yaml @@ -0,0 +1,17 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: jellyfin + namespace: media +spec: + selector: + matchLabels: + app: jellyfin + template: + metadata: + labels: + app: jellyfin + spec: + containers: + - name: jellyfin + image: jellyfin/jellyfin:10.10.7 diff --git a/docs/layout/examples/tree-multi-namespace/repository/media/configmaps/jellyfin.yaml b/docs/layout/examples/tree-multi-namespace/repository/media/configmaps/jellyfin.yaml new file mode 100644 index 00000000..cb7b463d --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/repository/media/configmaps/jellyfin.yaml @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: jellyfin + namespace: media +data: + PUID: "1000" + PGID: "1000" diff --git a/docs/layout/examples/tree-multi-namespace/repository/monitoring/configmaps/grafana.ini.yaml b/docs/layout/examples/tree-multi-namespace/repository/monitoring/configmaps/grafana.ini.yaml new file mode 100644 index 00000000..c3fc0ed5 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/repository/monitoring/configmaps/grafana.ini.yaml @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: grafana.ini + namespace: monitoring +data: + server.root_url: https://grafana.home.arpa + users.allow_sign_up: "false" diff --git a/docs/layout/model.md b/docs/layout/model.md new file mode 100644 index 00000000..a8afa2e9 --- /dev/null +++ b/docs/layout/model.md @@ -0,0 +1,324 @@ +# The template was the right primitive. Two things were missing + +> **design**: a proposal, not a plan of record. Nothing here binds until scheduled. +> Index: [`../INDEX.md`](../INDEX.md) +> Date: 2026-08-29. Supersedes this document's own earlier thesis, which argued that a path template +> is the wrong primitive and should be replaced by a `spec.layout` discriminated union. That argument +> no longer holds; why it stopped holding is the first section, because a reversal is worth more than +> a quiet edit. +> +> Concrete repository folders and matching configurations live in +> [`examples/README.md`](examples/README.md). + +Placement today is a ladder of four rungs, three of which are path templates and one of which is not: + +```text +byType -> default -> the folder's one kustomize root -> canonical +``` + +The proposal is to **keep that**, and add the two things a path genuinely cannot express: whether +this folder maintains a `kustomization.yaml`, and whether `metadata.namespace` is written into the +document. + +```yaml +spec: + path: apps/demo + placement: + byType: # unchanged, as shipped + v1/secrets: "secrets/{name}{sensitiveSuffix}" + default: "{namespace}/{resource}/{name}.yaml" # unchanged, as shipped + useKustomize: true # bool, default false + serializeNamespace: false # optional bool, unset = infer +``` + +Two booleans, inside the struct that already holds the placement axis. No discriminator, no +`spec.layout`, no `kind`, no `scope`, no new CRD, and no enum. `spec.placement` is an existing +optional struct, so both are additive: an object that says nothing behaves exactly as it does today. + +## What changed + +The earlier thesis rested on five arguments. Three were retired by +[#319](https://github.com/ConfigButler/gitops-reverser/pull/319), which shipped the ancestor walk: a +new file is registered with the nearest kustomization that governs it, whatever chose its path. + +| The argument against templates | Still true? | +|---|---| +| `byType` into a subdirectory produces a file no kustomization lists | **No.** That was [#295](https://github.com/ConfigButler/gitops-reverser/issues/295), and it is fixed | +| `placement.default` does the same to every type at once | **No.** Same bug, same fix | +| A CRD default cannot be added: a non-empty template shadows the kustomize-root rung | **Weakened**, and [`placement-visibility-and-declared-defaults.md`](../design/placement-visibility-and-declared-defaults.md) already concedes the fix turns this from a correctness wall into a legibility trade | +| "Where do my files go" needs a metric and a status field to be legible | **True** — but that is an argument for status, not against templates | +| Nothing can **create** structure, so an empty repository cannot be bootstrapped | **True**, and untouched by either design. Creating a `kustomization.yaml` is not a path question | + +The sentence the whole redesign rested on was *"a path template cannot express 'beside this folder's +one kustomization'"*. That is still true and no longer matters, because the template is no longer +asked to express it. Registration became an **invariant** rather than a rung — which was always the +best idea in the layout model, and it is the part that already shipped. + +What is left is one real gap (bootstrap), one status gap, and one flag that was always missing. + +## What kustomize actually requires + +Four facts, measured against kustomize v5.8.1 rather than recalled, because three of them contradict +assumptions the earlier model was built on. + +1. **A root does not require a flat folder.** A `kustomization.yaml` listing `configmaps/cache.yaml` + and `apps/deployments/web.yaml` builds, and the root's `namespace:` transformer applies to both. + So "`Kustomize` means flat files beside one root" was our rule, never kustomize's. +2. **Nested roots work, one per subfolder.** A parent listing `media` and `monitoring`, each holding + its own `kustomization.yaml` with its own `namespace:`, renders each document into its own + namespace. A multi-namespace folder can therefore omit `metadata.namespace` safely — if something + owns those child roots. +3. **There is no ambient pickup.** `resources: [cms/*.yaml]` fails (`evalsymlink failure`), and a bare + directory fails unless it contains its own kustomization. Every file is named explicitly or lives + under a nested root. +4. **An unlisted file in a listed subdirectory renders nothing.** The #295 class, unchanged. + +Fact 3 is why registration must be an invariant: in a kustomize folder there is no other way for a +new file to be applied. Fact 1 is why the path may be anything the user wants. Together they say the +two questions are independent, which is exactly what a single `kind` discriminator could not express. + +## `useKustomize` + +The flag that says this folder is a kustomize folder and the operator maintains its root. + +| | A kustomization governs the path | Nothing governs the path | +|---|---|---| +| unset / `false` (default) | register the new file in its `resources:` | write the file, touch nothing | +| `true` | register | **create `kustomization.yaml` at `spec.path`**, then register | + +**Registering into a root that is already there is not what this flag controls.** It happens in both +columns, because a file a kustomization does not list is a file nothing renders — that was #295, and +[#319](https://github.com/ConfigButler/gitops-reverser/pull/319) made it an invariant. The flag has +exactly one job: what to do when there is no root. + +That is also the honest reading of the name, which says less than the field does: `useKustomize: +false` does not mean "leave kustomize alone". If you do not want a folder's root touched at all, do +not point a `GitTarget` at that folder — the ancestor walk is bounded by the write jail, so a +kustomization **above** `spec.path` is never edited, and rooting the target lower is the existing, +better-tested way to say it. + +Creating the root is the only genuinely new machinery in this proposal, and it is what makes an empty +repository bootstrappable — the last surviving argument from the earlier thesis, now one boolean +rather than a reason to redesign the primitive. A created root carries `resources:`, and +`namespace:` when the folder is single-namespace, which is what makes it a **meaningful** +kustomization rather than an empty file. That pairing is the whole point of the second flag. + +## `serializeNamespace` + +Whether the committed document carries its own `metadata.namespace`. A path decides where the file +sits; it cannot decide what is inside it, and kustomize takes the namespace from exactly one of two +places — the document, or a governing kustomization's `namespace:`. + +| Value | Meaning | +|---|---| +| unset (default) | infer, which is today's behavior | +| `true` | always write it | +| `false` | never write it | + +**It is an optional boolean, and unset is not the same as `false`.** No plain default preserves +today's behavior: defaulting to `false` breaks a flat folder, whose documents must carry their own +namespace or they are ambiguous, and defaulting to `true` writes a redundant line into every +kustomize folder that already supplies one. So the field is a `*bool` and nil means infer — the +ordinary Kubernetes shape for a three-state switch that has to keep an existing default. + +**What "infer" already does, and why it is worth keeping as the default.** The inference is not a +guess. [`placement.go`](../../internal/manifestanalyzer/placement.go) omits `metadata.namespace` +only when the governing kustomization sets a `namespace:` **and sets it to this resource's own +namespace**; in every other case it writes the namespace explicitly, because omitting it there would +hand the document to a different namespace and the mirror would claim to hold an object it does not. +An explicit setting is therefore an override of a correctness rule, which is why `false` needs a +guard and unset does not. + +The two settings exist for the two shapes a user actually declares: + +- **`true` for a flat folder.** Nothing downstream supplies a namespace, so every namespaced document + has to carry one. It also keeps a document portable: it means the same thing pasted anywhere. +- **`false` beside a root that supplies it.** With `useKustomize: true` the operator owns that root + and writes `namespace:` into it, so the omission is **provable** rather than trusted. That is the + difference between establishing a convention and guessing one, and it is what inference + structurally cannot do on an empty folder — there is nothing there to infer from. + +The name deliberately avoids `writeNamespace`. "Write" is the most loaded word in this API — the +write boundary, the write jail, `WriteBoundaryRefused` — so `writeNamespace: false` invites the +reading *"never write to this namespace"*, a permission, which is precisely what the neighbouring +`sourceNamespace` fields are. `serializeNamespace` names the moment the decision is made, and cannot +be read as policy. + +**It governs namespaced resources only.** A `ClusterRole` has no namespace, so the field is ignored +for cluster-scoped documents rather than being an error — worth stating in the field documentation, +because a tree folder is the type most likely to carry both. + +### The guard on `false`, and why it is a post-scan check + +`false` where nothing supplies the namespace hands the object to whatever namespace the applier +happens to be pointed at, which is a different object with the same name. It is honest only when +something guarantees the namespace, and where the guarantee is a `kustomization.yaml` **the user +owns**, they can delete one line from their own file and every subsequent document silently +relocates. + +That precondition is a property of the observed folder, not of the spec, so no CEL rule can check it. +It is one post-scan rule, on the scan that already runs, setting `Validated=False` with a message +naming the field and what the folder actually contains. With `useKustomize: true` the rule is +satisfied by construction, because the operator wrote the supplier. + +## Collisions are already decided + +What happens when two resources resolve to the same path is specified and shipped in +[`new-file-placement-rules.md`](new-file-placement-rules.md): a unique path is a new file, a colliding +path **appends** into a plaintext multi-document file, a sensitive resource whose path already holds a +document is **refused** rather than appended, encrypted files are never appended into in either +direction, and existing documents stay match-first, so an object living inside a bundle is updated +where it is rather than moved out of it. + +So "one file per object" and "a bundle per type or per namespace" are both expressible today, by +writing a template that distinguishes identities or one that deliberately does not. + +## The four questions a user actually asks + +| Question | Answered by | +|---|---| +| Is this a kustomize folder? | `useKustomize` | +| Do my documents carry `metadata.namespace`? | `serializeNamespace` | +| Which folder do new files go in? | the directory part of the template. Fact 1: this is not constrained to flat | +| Is this folder one namespace or many? | whether `{namespace}` appears in the template. A template without it is single-namespace by construction | + +No field in that table exists to answer a question a user did not ask, which is the test the earlier +`kind`/`scope` pair failed: `scope: SingleNamespace` restated in an enum what the template already +said, and then required an admission rule to keep the two in agreement. + +## What this deletes + +`spec.layout` and its discriminator, `kind`, `type`, the `Auto`/`Kustomize`/`Tree`/`Flat`/`Template` +values, `layout.scope` and the admission rule keeping it in agreement with `allowedSourceNamespaces`, +`kustomize.create`, and with them four findings of the maintainer review (L3, L4, L5, L8) — not +renamed, gone. The `LayoutProfile` question goes too: without a `layout` block the only thing left to +share is the `byType` map, and whatever generates thirty GitTargets repeats two booleans for free. + +**And the migration.** `spec.placement` keeps its meaning and gains two optional members, so there is +no loud rejection, no `feat(api)!` on this axis, and no coordinated consumer bump for the layout work. +The layout model was the largest breaking change in the queue; on this shape it is not a breaking +change at all. + +## What it leaves standing + +- **`spec.placement` is mutable and stays mutable.** Existing files never move, so a template change + affects only files written afterwards, and a folder can hold documents placed under two templates. + Match-first identity keeps finding and updating them in place. The immutability-plus-CEL-widening + machinery an earlier draft proposed was invented to protect a discriminator that no longer exists. +- **`placements_total` keeps its `source` label** — today `declared`, `kustomize_root` and + `canonical`, with the `declared` split into `byType`/`default` still queued. It names the rung + that answered rather than a resolved layout kind, so nothing here breaks a label. +- **`{kindLower}` and the versionless identity fix** are template features and stay queued. + +## `status.placement`, and the post-scan pass + +The legibility gap is the one surviving argument from the earlier thesis that no field answers, and +it is worth building **before** either flag: placement only ever affects *new* documents, so there is +nothing to preview by inspection, and a suspended target plus this stanza is what turns adoption from +declare-and-hope into a dry run. + +```yaml +status: + observedGeneration: 4 + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization # SingleKustomization | Ambiguous | None + message: "render root '.' governs new files" + observedGeneration: 4 + placement: + renderRoot: . + serializeNamespace: false # what it resolved to for this folder + byTypeEntries: 1 + observedRevision: 9f3c1ab + observedTime: "2026-07-30T09:14:22Z" + examples: [] # capped at three, illustrative, not a tally +``` + +Three decisions are taken here rather than deferred, because each is cheaper to take before the field +exists than after: + +- **The resolution reason is a condition reason, not a field.** `renderRootReason` would have been a + reason enum in a bespoke field, and every consumer in this ecosystem already reads reasons from + `conditions`. +- **No accumulating counters.** `placedResources`, `overriddenTypes` and `refusedResources` are + metrics; `placements_total` carries them with better labels. A monotonic counter in status is a + status write per event, which re-creates the self-triggering reconcile edge the status work already + fixed once. `examples` stays, capped and fixed-size, because "show me where a Secret would land" is + not a metric. +- **`conditions` and `observedGeneration` are in the stanza**, because every scenario README already + asserts `Ready=True`. + +The current half must never depend on a placement having happened: `renderRoot` is a fact about the +folder from the last scan, available before anything is ever written. + +**The post-scan validation pass ships with it**, because it is the same scan. Two rules today, whose +precondition is a property of the observed folder rather than of the spec, so no CEL rule can reach +them: `serializeNamespace: false` requires a namespace supplier, and a folder covering two roots is +`Ambiguous` rather than silently picking one. One pass, one condition shape, `Validated=False` naming +the offending field and what the folder actually contains. + +## How it gets built + +`LocateNew` is not rewritten. The four-rung ladder is a single function, +[`LocateNew`](../../internal/manifestanalyzer/placement.go), with a single caller in +[`plan_flush.go`](../../internal/git/plan_flush.go), and everything downstream of the path decision — +registration, the render fidelity gate, refusal accounting, the metrics — already exists and stays +where it is. The two flags sit beside the ladder. + +| PR | Content | Breaking | +|---|---|---| +| 1 | The worked examples as an executable corpus | no | +| 2 | `spec.suspend`, and the reconcile-request annotation | no | +| 3 | `status.placement`, and the post-scan validation pass | no | +| 4 | `useKustomize` and `serializeNamespace` ([#322](https://github.com/ConfigButler/gitops-reverser/issues/322)) | no | + +None of it is breaking, so none of it waits for a coordinated consumer bump. What is breaking on +`GitTarget` is unrelated to placement and is sequenced in +[`gittarget-api-wave.md`](../design/gittarget-api-wave.md). + +**PR 1 is the corpus, and it is the reason the rest is reviewable.** +[`examples/README.md`](examples/README.md) already has the shape of a golden-file suite — +`repository/`, `config/`, `input/`, `expected-*.patch` — and is read by nobody but a human. Wiring it +up converts the PR 4 review from "does this prose hold together" into "does the diff match the +patch". The seam exists: `newWorktreeForTest` and `flushEventsToWorktree` in +[`internal/git`](../../internal/git/placement_test.go) already do this at a smaller scale. Per +scenario: seed a worktree from `repository/`, build the write event from `input/`, derive the flush +policy from `config/gittarget.yaml`, flush, and compare the normalized diff with `expected-*.patch`. +Blob hashes and index lines are noise; a `-update` flag that rewrites the patches keeps the corpus +cheap to extend. Scenarios describing behavior PR 4 introduces are written now and skipped with the +PR that unskips them named in the skip message, so **PR 4 is finished when the last skip is gone.** +`config/gittarget.yaml` uses fields that do not exist yet, so it parses into a harness-local struct +until PR 4 deletes that mapping — which is itself a check that the API the examples describe is the +API that got built. + +**PR 4 builds the `true` half of each flag.** Everything else is already there: registration into an +existing root shipped in #319, and inference is what +[`namespaceIsInheritedFromContext`](../../internal/manifestanalyzer/placement.go) already does. What +is new is writing a `kustomization.yaml` that does not exist, with `namespace:` set when the folder +is single-namespace, and registering into it in the same commit. Build that last and on its own: it +is the one thing that writes a file nobody asked for by name. + +Two gaps the corpus should fill in PR 1: **a refusal scenario** (every example is a happy path, and +the post-scan pass has the least coverage — a `serializeNamespace: false` with no supplier, and a +folder covering two roots, each asserting `expected-status.yaml` instead of a patch), and **the +missing `ClusterProvider`** that `empty-repo-bootstrap` references as `clusterProviderRef: app-intent` +without a specimen existing anywhere. + +## Open questions + +- Does `serializeNamespace: false` need to **name** its supplier (`KustomizeRoot`, + `FluxTargetNamespace`, `Asserted`) so the post-scan pass can check the guarantee rather than infer + which one was meant? +- Should a `useKustomize: true` folder create a **nested** root per directory the template writes + into, each carrying its own `namespace:`? Fact 2 proves it works, and it is what would make + `serializeNamespace: false` safe in a multi-namespace tree. Deferred: materially more machinery, + and nobody has asked for a multi-namespace folder without namespaces in its documents. +- Should the operator ever **refuse** a write when a root that used to govern the path is gone, + rather than reporting `LayoutResolved: None` and carrying on? Report first; escalate if someone + says the status was not enough. +- Should `placement.default` gain a CRD default now that a defaulted template no longer produces + unrendered files? The remaining objection is legibility, not correctness. +- Namespace-local `GitProvider`: the homelab examples put a `GitTarget` in `argocd`, `flux-system` + and `homelab-config`, each needing its own `GitProvider` — three copies of one credential in a + single-owner cluster. Recorded as a gap, not a blocker. diff --git a/docs/spec/gittarget-new-file-placement-rules.md b/docs/layout/new-file-placement-rules.md similarity index 97% rename from docs/spec/gittarget-new-file-placement-rules.md rename to docs/layout/new-file-placement-rules.md index d7186793..7b15b2aa 100644 --- a/docs/spec/gittarget-new-file-placement-rules.md +++ b/docs/layout/new-file-placement-rules.md @@ -15,12 +15,12 @@ > Captured: 2026-06-05. Option C removed: 2026-07-29. > Related: > [open-asks-priority.md](../design/open-asks-priority.md) — **the argument for deleting Option C**, -> [contextual-namespace-and-kustomize-folder-editing.md](contextual-namespace-and-kustomize-folder-editing.md), -> [gittarget-repository-validity-and-placement.md](gittarget-new-file-placement-rules.md), -> [current-manifest-support-review.md](current-manifest-support-review.md), -> [manifestedit-new-file-placement-spike.md](gittarget-new-file-placement-rules.md), -> [reconcile-via-watchlist-mark-and-sweep.md](reconcile-via-watchlist-mark-and-sweep.md), -> [gitpath-foreign-content-stringency.md](gitpath-foreign-content-stringency.md) +> [contextual-namespace.md](contextual-namespace.md), +> [gittarget-repository-validity-and-placement.md](new-file-placement-rules.md), +> [current-manifest-support-review.md](../spec/current-manifest-support-review.md), +> [manifestedit-new-file-placement-spike.md](new-file-placement-rules.md), +> [reconcile-via-watchlist-mark-and-sweep.md](../spec/reconcile-via-watchlist-mark-and-sweep.md), +> [gitpath-foreign-content-stringency.md](../spec/gitpath-foreign-content-stringency.md) ## Summary @@ -123,7 +123,7 @@ The current `GitTargetSpec` has `providerRef`, `branch`, `path`, and optional ([api/v1alpha3/gittarget_types.go](../../api/v1alpha3/gittarget_types.go)). The writer already uses the materialized-model direction described in -[current-manifest-support-review.md](current-manifest-support-review.md): +[current-manifest-support-review.md](../spec/current-manifest-support-review.md): - steady-state writes scan the GitTarget subtree into a content-derived store, then apply a commit-scoped plan @@ -649,7 +649,7 @@ in force, which is not something the user can see. A document in a directory whose kustomization sets a `namespace:` transformer does not carry `metadata.namespace` — the build context supplies it (see -[contextual-namespace-and-kustomize-folder-editing.md](contextual-namespace-and-kustomize-folder-editing.md)). +[contextual-namespace.md](contextual-namespace.md)). A new document placed there must follow that convention, or it breaks the folder's own style, and this applies to **every** resolved path: a declared template pointing into a governed directory is under the same obligation as the kustomize-root fallback. @@ -811,7 +811,7 @@ because the answer does not depend on the folder's history.* **P7 — A resolved path is still subject to the write-time ignore invariant.** Any resolved path — declared, kustomize-root, or canonical — can collide with a `.gittargetignore` pattern and trip the §4.3 `IgnoreShadowsManagedPath` precondition -([gitpath-foreign-content-stringency.md](gitpath-foreign-content-stringency.md)), aborting +([gitpath-foreign-content-stringency.md](../spec/gitpath-foreign-content-stringency.md)), aborting the flush. *Still live, and unrelated to inference: placement inherits this failure mode rather than creating it.* @@ -925,6 +925,18 @@ Recommended variables: | `{name}` | metadata name | | `{sensitiveSuffix}` | Optional convention helper: `.sops.yaml` for sensitive writes, `.yaml` otherwise | +**A template carries its own extension.** Nothing is appended: `"{namespace}/{name}"` is rejected by +[path validation](#path-validation), which requires a recognized YAML suffix, and +`"{namespace}/{name}.yaml"` is the correct spelling. `{sensitiveSuffix}` is the one variable that +supplies the extension itself, because it has to choose between `.yaml` and `.sops.yaml`. Any +example that shows a template without a visible suffix is wrong. + +`{kindLower}` — the lower-cased kind, so `ConfigMap` becomes `configmap` — is **decided and not yet +built**. It is a variable rather than a `toLower` function because a function invites an expression +language into a field that is deliberately not one. It is what a user writes to ask for the +`configmap-cache.yaml` convention that the built-in rungs do not produce: the kustomize root names a +new sibling `{name}.yaml` and never infers a naming convention from the folder's existing files. + With those variables, the built-in canonical layout is **namespace-first, no version segment** (as implemented in `ResourceIdentifier.ToGitPath`): diff --git a/docs/spec/README.md b/docs/spec/README.md index 4933e1f9..5132eb9c 100644 --- a/docs/spec/README.md +++ b/docs/spec/README.md @@ -16,6 +16,12 @@ here because they are the only written record of a rule the code still obeys. If you change one of these behaviours, change the document in the same commit. +Two `spec/`-class contracts live in [`../layout/`](../layout/README.md) instead: +[`new-file-placement-rules.md`](../layout/new-file-placement-rules.md) and +[`contextual-namespace.md`](../layout/contextual-namespace.md). They bind exactly as the pages here +do, and Go source cites them by path; they sit in a topic folder because the layout question spans +three lifecycle classes and was unreadable spread across them. The rows below still list them. + ## Start here - [`manifest-system.md`](manifest-system.md) — **how the whole live → Git pipeline @@ -27,9 +33,9 @@ If you change one of these behaviours, change the document in the same commit. |---|---| | [`attribution.md`](attribution.md) | **how a commit gets its author**: deletion-at-intent, the publish and join halves, the tier ladder, the audit-route partition, and what the metrics can and cannot answer | | [`current-manifest-support-review.md`](current-manifest-support-review.md) | the manifest store, plan/apply/flush, and the all-or-nothing folder claim | -| [`contextual-namespace-and-kustomize-folder-editing.md`](contextual-namespace-and-kustomize-folder-editing.md) | kustomize graph-aware namespace inference; the supported subset | +| [`../layout/contextual-namespace.md`](../layout/contextual-namespace.md) | kustomize graph-aware namespace inference; the supported subset | | [`reconcile-via-watchlist-mark-and-sweep.md`](reconcile-via-watchlist-mark-and-sweep.md) | initial reconcile; **no bookmark, no sweep** | -| [`gittarget-new-file-placement-rules.md`](gittarget-new-file-placement-rules.md) | where a brand-new resource's file goes | +| [`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md) | where a brand-new resource's file goes | | [`manifestedit-field-ownership-spike.md`](manifestedit-field-ownership-spike.md) | "the API wins" — full-object ownership, and the do-not-build list | | [`type-followability.md`](type-followability.md) | is a type followable, and if not, the single reason | | [`type-lifecycle-events-and-wobble-settling.md`](type-lifecycle-events-and-wobble-settling.md) | removal grace and flap coalescing | diff --git a/docs/spec/manifest-system.md b/docs/spec/manifest-system.md index 227c6a93..f5ee2318 100644 --- a/docs/spec/manifest-system.md +++ b/docs/spec/manifest-system.md @@ -47,7 +47,7 @@ identity (group, kind, namespace, name). The store indexes by that identity and writes an edit back to wherever the document already lives — *match-first placement*. Being strict about which file a resource "should" live in was explicitly rejected. Only a genuinely new resource needs a placement decision, and -that is [`gittarget-new-file-placement-rules.md`](gittarget-new-file-placement-rules.md). +that is [`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md). **A GitTarget makes an all-or-nothing claim on its folder.** It either manages everything in the subtree or it refuses the folder. There is no partial ownership, @@ -111,7 +111,7 @@ beside it. Raw identity (what the bytes say) and effective identity (what kustom would produce) are tracked separately, and an inherited namespace is kept *out* of the file bytes on write. The supported subset, and why everything outside it is refused rather than unimplemented, is -[`contextual-namespace-and-kustomize-folder-editing.md`](contextual-namespace-and-kustomize-folder-editing.md). +[`../layout/contextual-namespace.md`](../layout/contextual-namespace.md). The governing constraint is invertibility: an edit must round-trip in both directions. Generators, `patches*`, `namePrefix`/`nameSuffix`, `components`, remote diff --git a/docs/spec/sops-single-file-no-multidoc.md b/docs/spec/sops-single-file-no-multidoc.md index bd6de697..ab9299bc 100644 --- a/docs/spec/sops-single-file-no-multidoc.md +++ b/docs/spec/sops-single-file-no-multidoc.md @@ -5,8 +5,8 @@ > Status: decided > Captured: 2026-06-08 > Related: -> [file-agnostic-placement.md](gittarget-new-file-placement-rules.md), -> [contextual-namespace-and-kustomize-folder-editing.md](contextual-namespace-and-kustomize-folder-editing.md), +> [file-agnostic-placement.md](../layout/new-file-placement-rules.md), +> [../layout/contextual-namespace.md](../layout/contextual-namespace.md), > [../sops-repo-bootstrap-and-key-management-architecture.md](../finished/sops-repo-bootstrap-and-key-management-architecture.md), > [../sops-repo-bootstrap-out-of-scope.md](../finished/sops-repo-bootstrap-out-of-scope.md) @@ -17,7 +17,7 @@ SOPS file**. We do **not** write SOPS-encrypted multi-document YAML (no `\n---\n`-separated documents inside an encrypted file). Plaintext manifests may still be multi-document where that is convenient (see -[file-agnostic-placement.md](gittarget-new-file-placement-rules.md)); this decision is +[file-agnostic-placement.md](../layout/new-file-placement-rules.md)); this decision is scoped to files we encrypt with SOPS. ## Why this came up diff --git a/internal/controller/gittarget_placement_validation.go b/internal/controller/gittarget_placement_validation.go index dd73f15b..839a79ff 100644 --- a/internal/controller/gittarget_placement_validation.go +++ b/internal/controller/gittarget_placement_validation.go @@ -19,7 +19,7 @@ const coreSecretsTypeKey = "v1/secrets" // validatePlacementPolicy statically validates a GitTarget's declared placement // policy (Option B2: -// docs/spec/gittarget-new-file-placement-rules.md) against the +// docs/layout/new-file-placement-rules.md) against the // spec alone — no repository scan is needed, so this runs as part of the Validated // gate, the same spec-well-formedness check that already covers provider/branch // resolution and path-overlap conflicts. A nil spec (no declared policy) is always diff --git a/internal/git/manifestedit/kustomization.go b/internal/git/manifestedit/kustomization.go index c60826d3..86877ed1 100644 --- a/internal/git/manifestedit/kustomization.go +++ b/internal/git/manifestedit/kustomization.go @@ -128,7 +128,7 @@ func applyKustomizationEdit(root *yaml.Node, e KustomizationEdit) error { // AppendKustomizationResource adds one entry to an existing kustomization.yaml's // resources: sequence — the mechanism half of the "add to the right kustomize -// file" (docs/spec/gittarget-new-file-placement-rules.md): a +// file" (docs/layout/new-file-placement-rules.md): a // new sibling file placed inside a kustomize-governed directory must also be named // in that directory's resources: list, or kustomize never renders it. // diff --git a/internal/git/plan_flush.go b/internal/git/plan_flush.go index bb68231f..eb9e80f8 100644 --- a/internal/git/plan_flush.go +++ b/internal/git/plan_flush.go @@ -154,7 +154,7 @@ type writeBatch struct { // and therefore cannot see coming — form one deterministic, resource-identity- // sorted multi-document file instead of each writeWholeFile call silently // discarding the one before it. See - // docs/spec/gittarget-new-file-placement-rules.md, + // docs/layout/new-file-placement-rules.md, // "Collision and append behavior": "if several new plaintext resources in one // plan render to the same path, write a multi-document file in deterministic // resource-identity order." @@ -339,7 +339,7 @@ func wroteBytes(o upsertOutcome) bool { // createNew resolves the placement of a resource with no existing document — // declared policy (Option B), the folder's one kustomize root, or the canonical -// fallback — per docs/spec/gittarget-new-file-placement-rules.md, +// fallback — per docs/layout/new-file-placement-rules.md, // adds the kustomize resources: entry the placement may require, and writes the new // document: a brand-new file, or an additional document appended to an existing // accepted plaintext bundle. A placement LocateNew cannot honour safely (today, only diff --git a/internal/manifestanalyzer/contextual_namespace_corpus_test.go b/internal/manifestanalyzer/contextual_namespace_corpus_test.go index 9f761735..8f658498 100644 --- a/internal/manifestanalyzer/contextual_namespace_corpus_test.go +++ b/internal/manifestanalyzer/contextual_namespace_corpus_test.go @@ -25,7 +25,7 @@ type wantDoc struct { // testdata/contextual-namespace. Each folder is built as a GitTarget subtree and the // per-document namespace provenance is asserted, so the supported boundary is pinned by // real layouts rather than prose. See -// docs/spec/contextual-namespace-and-kustomize-folder-editing.md. +// docs/layout/contextual-namespace.md. func TestContextualNamespaceCorpus(t *testing.T) { cases := []struct { dir string diff --git a/internal/manifestanalyzer/placement.go b/internal/manifestanalyzer/placement.go index 8919f6b6..868d0119 100644 --- a/internal/manifestanalyzer/placement.go +++ b/internal/manifestanalyzer/placement.go @@ -13,7 +13,7 @@ import ( ) // PlacementPolicy is a resolved GitTarget placement declaration (Option B2 of -// docs/spec/gittarget-new-file-placement-rules.md): a single +// docs/layout/new-file-placement-rules.md): a single // exact-type map plus a fallback default template, consulted for every resource // regardless of sensitivity. It mirrors api/v1alpha3.GitTargetPlacementSpec // field-for-field but is defined locally so this analyzer package stays free of any @@ -35,8 +35,8 @@ type PlacementPolicy struct { // PlacementRequest describes a resource with no existing document in Git — the // only case placement runs for (an existing document is always updated in place at -// its current location; see docs/design/manifest/version2/ -// gittarget-new-file-placement-rules.md, "Existing manifests are still match-first"). +// its current location; see docs/layout/new-file-placement-rules.md, +// "Existing manifests are still match-first"). type PlacementRequest struct { Identifier types.ResourceIdentifier Kind string @@ -161,7 +161,7 @@ func (e *PlacementRefusedError) Error() string { return e.detail } func (e *PlacementRefusedError) Unwrap() error { return e.cause } // LocateNew resolves the placement of a resource with no existing document, per -// docs/spec/gittarget-new-file-placement-rules.md: a declared template (Option B) +// docs/layout/new-file-placement-rules.md: a declared template (Option B) // wins when present; otherwise the folder's one supported kustomize root, if it has // exactly one; otherwise the canonical path. // diff --git a/internal/manifestanalyzer/store.go b/internal/manifestanalyzer/store.go index 55bfaf7c..43189fb8 100644 --- a/internal/manifestanalyzer/store.go +++ b/internal/manifestanalyzer/store.go @@ -263,7 +263,7 @@ type DocumentModel struct { // It replaces an earlier "namespace came from kustomize" boolean so the store can also // explain the no-context and ambiguous cases to status, duplicate diagnostics, and // future placement — see -// docs/spec/contextual-namespace-and-kustomize-folder-editing.md. +// docs/layout/contextual-namespace.md. type NamespaceSourceKind string const ( @@ -796,7 +796,7 @@ func kustomizationInfos(kusts map[string]*kustomizationDoc) map[string]*Kustomiz // namespace transformer, its resources/bases graph entries, and whether it uses any // feature outside the supported contextual-namespace subset (which disqualifies it as // a namespace source). See the "Kustomize subset proposal" in -// docs/spec/contextual-namespace-and-kustomize-folder-editing.md. +// docs/layout/contextual-namespace.md. type kustomizationDoc struct { path string // kustomization file path (slash) namespace string // the namespace: transformer value diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/README.md b/internal/manifestanalyzer/testdata/contextual-namespace/README.md index 97551e8e..65683346 100644 --- a/internal/manifestanalyzer/testdata/contextual-namespace/README.md +++ b/internal/manifestanalyzer/testdata/contextual-namespace/README.md @@ -16,6 +16,6 @@ The `images-overlay`, `replicas-overlay`, and `ambiguous-images` folders pin the override-chain attribution the same way (`overrides_test.go`); see `docs/design/support-boundary/finished/images-and-replicas-edit-through.md`. -See `docs/design/manifest/contextual-namespace-and-kustomize-folder-editing.md` +See `docs/layout/contextual-namespace.md` (the "Supported and unsupported example folders" matrix). Add a new folder here whenever a new "can we support X?" question comes up. diff --git a/internal/types/identifier.go b/internal/types/identifier.go index 03293c5e..90712392 100644 --- a/internal/types/identifier.go +++ b/internal/types/identifier.go @@ -87,7 +87,7 @@ func (r ResourceIdentifier) Key() string { // once any layout exists in the target, sibling inference follows it, and an // existing document is always edited in place at its current location (match-first), // so changing this shape never moves a file that is already in Git. See -// docs/spec/gittarget-new-file-placement-rules.md. +// docs/layout/new-file-placement-rules.md. // // That omitted version is the other half of the decision recorded at // [ResourceIdentifier.Key]: this versionless identity is the durable one — the object stays diff --git a/test/e2e/new_file_placement_e2e_test.go b/test/e2e/new_file_placement_e2e_test.go index a2683fa1..5d2f2e70 100644 --- a/test/e2e/new_file_placement_e2e_test.go +++ b/test/e2e/new_file_placement_e2e_test.go @@ -14,7 +14,7 @@ import ( // Validates new-file placement end-to-end: a brand-new resource with no // existing document in Git — the "install something extra in test" launch use -// case (docs/spec/gittarget-new-file-placement-rules.md, +// case (docs/layout/new-file-placement-rules.md, // docs/design/support-boundary/README.md) — lands inside the kustomize-managed overlay // directory it belongs to, not the canonical GVR-tree path, and the overlay's // kustomization.yaml gains the resources: entry so kustomize actually renders it. From b55699b90d9a239bd12d5791646c20838d6f67a4 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Sat, 29 Aug 2026 18:07:50 +0000 Subject: [PATCH 2/3] docs(layout): lift serializeNamespace to the spec's top level, and say why a folder-wide claim is ordinary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `spec.placement` means "where a NEW document goes"; everything already written is match-first and never moves, which is what makes a template safe to change. `serializeNamespace` does not have that property, and the write path already proves it: plan_flush.go strips metadata.namespace from a new document using the placement result and from an UPDATE using the document's own observed namespace source, and a document whose namespace is inherited is located in the file bytes by a namespace-less identity. A field that rewrites existing documents as they are next touched, and decides how they are found, does not belong inside a struct documented as new-files-only. So it is `spec.serializeNamespace`. `useKustomize` stays in `spec.placement`, because it passes the same test: it decides whether a new file's directory has a root to join, and creates one when there is none. Group by blast radius, not by topic. Also answers whether a folder-wide namespace claim is a strange thing to state. It is not: "no document carries its namespace" is the portable-artifact convention behind every kustomize base, Flux's targetNamespace, Argo's destination.namespace and a chart's .Release.Namespace, while "every document carries it" is the convention for a folder applied directly. Cluster-scoped resources are exempt, and the one genuinely non-uniform shape — a tree of nested roots, each supplying its own namespace — is what the unset default already handles per document. The uniform claim is what an explicit setting is for; the non-uniform folder is what unset is for. Co-Authored-By: Claude Opus 5 --- docs/INDEX.md | 2 +- docs/TODO.md | 4 +- docs/design/gittarget-api-wave.md | 24 ++++++---- .../direction-and-configuration-surface.md | 2 +- docs/layout/README.md | 2 +- docs/layout/examples/README.md | 23 +++++---- .../config/gittarget.yaml | 10 ++-- .../homelab-argocd/config/gittarget.yaml | 11 ++--- .../homelab-flux/config/gittarget-media.yaml | 3 +- .../homelab-flux/config/gittarget.yaml | 3 +- .../config/gittarget.yaml | 3 +- .../config/gittarget.yaml | 9 ++-- docs/layout/model.md | 48 +++++++++++++++++-- 13 files changed, 93 insertions(+), 51 deletions(-) diff --git a/docs/INDEX.md b/docs/INDEX.md index 02bec0df..b8c9a51c 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -132,7 +132,7 @@ by path from Go source. |---|---|---| | [`contextual-namespace.md`](layout/contextual-namespace.md) | **spec** | kustomize namespace inference; the supported subset | | [`new-file-placement-rules.md`](layout/new-file-placement-rules.md) | **spec** | where a new resource's file goes: declared, the folder's one kustomize root, canonical. Sibling inference is removed, and kept as history | -| [`model.md`](layout/model.md) | **design** | **reversed, and much smaller than it was.** The earlier thesis wanted `spec.placement` replaced by a `spec.layout` discriminated union; three of its five arguments were retired by [#319](https://github.com/ConfigButler/gitops-reverser/pull/319), which made registration an invariant. So the template **stays** and gains two optional booleans on `spec.placement`: **`useKustomize`** (create and maintain the folder's root; registering into a root that already exists is an invariant, not a setting) and **`serializeNamespace`** (a `*bool`, because unset must keep meaning "infer" — no plain default preserves today's behavior). Carries four kustomize facts **measured** against v5.8.1, three of which contradict the earlier model; the `status.placement` stanza and the post-scan pass; and the build order. The headline is what it deletes — `spec.layout`, `kind`, `scope`, `kustomize.create`, the `LayoutProfile` question, and the migration — so the largest breaking change in the queue stops being breaking at all | +| [`model.md`](layout/model.md) | **design** | **reversed, and much smaller than it was.** The earlier thesis wanted `spec.placement` replaced by a `spec.layout` discriminated union; three of its five arguments were retired by [#319](https://github.com/ConfigButler/gitops-reverser/pull/319), which made registration an invariant. So the template **stays** and gains two optional booleans: **`spec.placement.useKustomize`** (create and maintain the folder's root; registering into a root that already exists is an invariant, not a setting) and **`spec.serializeNamespace`** (a `*bool`, because unset must keep meaning "infer" — no plain default preserves today's behavior), which sits one level up because it governs the bytes of every write and the identity a managed document is found by, not just new files. Carries four kustomize facts **measured** against v5.8.1, three of which contradict the earlier model; the `status.placement` stanza and the post-scan pass; and the build order. The headline is what it deletes — `spec.layout`, `kind`, `scope`, `kustomize.create`, the `LayoutProfile` question, and the migration — so the largest breaking change in the queue stops being breaking at all | [`examples/`](layout/examples/README.md) makes the proposed `GitTarget` layout model tangible: six scenarios, each a repository folder with the matching proposed `GitTarget` and rule configuration, diff --git a/docs/TODO.md b/docs/TODO.md index 5d79241a..0302c1d7 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -66,8 +66,8 @@ This file is meant to track the smaller current backlog, not historical notes. - [ ] Revisit output layout. **Now designed and postponed, not open-ended, and the answer has reversed since [#293](https://github.com/ConfigButler/gitops-reverser/issues/293) was filed.** The - path template **stays**; what it could not express becomes two optional booleans on - `spec.placement`, `useKustomize` and `serializeNamespace`. [layout/model.md](layout/model.md) + path template **stays**; what it could not express becomes two optional booleans, + `spec.placement.useKustomize` and `spec.serializeNamespace`. [layout/model.md](layout/model.md) carries the reversal, the fields and the order. The placement work is no longer breaking, so it no longer needs [#294](https://github.com/ConfigButler/gitops-reverser/issues/294); the issues still describe the diff --git a/docs/design/gittarget-api-wave.md b/docs/design/gittarget-api-wave.md index e06c9d4f..26e3b28a 100644 --- a/docs/design/gittarget-api-wave.md +++ b/docs/design/gittarget-api-wave.md @@ -46,19 +46,22 @@ breaking anyway, and costs a bump in every release that is not. `GitTargetSpec` today is flat: `providerRef`, `branch`, `path`, `encryption`, `placement`, `clusterProviderRef`, `allowedSourceNamespaces`, `prune`. Queued on top of it were `suspend`, -`useKustomize`, `serializeNamespace`, `commitWindow` and `commit.message` — five more members on a +`serializeNamespace`, `useKustomize`, `commitWindow` and `commit.message` — five more members on a spec that already flattens six orthogonal axes. Left alone, this object accumulates faster than it sheds, and moving `commit.message` off `GitProvider` for exactly that reason while doing it just relocates the problem one hop. Two groupings, and one deliberate exception: -- **`useKustomize` and `serializeNamespace` nest under `spec.placement`**, where an earlier draft of - [`model.md`](../layout/model.md) had them at the top level beside it. They are placement concerns by - their own argument — one decides what governs the produced document, the other what is inside it — - and `spec.placement` is an existing optional struct, so nesting them is **still purely additive**. - It costs no bump, it is free only before they exist, and it means the placement axis is one member - of the spec rather than three. +- **`useKustomize` nests under `spec.placement`**, where an earlier draft of + [`model.md`](../layout/model.md) had it at the top level beside it. `spec.placement` is an existing + optional struct, so nesting is **still purely additive**, and it is free only before the field + exists. The test it passes is retroactivity: like every other placement field, it only ever affects + a **new** document. +- **`serializeNamespace` stays at the top level**, and that is not an exception to the grouping rule + but the same rule applied. It governs the bytes of every write, updates included, and the identity + a managed document is located by; nesting it inside a struct documented as "new files only" would + have hidden a field that rewrites existing documents. Group by blast radius, not by topic. - **`commitWindow` and `commit.message` land as `spec.commit`**, not as two top-level fields: `spec.commit.window` and `spec.commit.message.template`. The move is breaking either way, so the grouping is free, and `GitProvider.spec.commit` is the shape they already have. @@ -68,8 +71,9 @@ Two groupings, and one deliberate exception: After the wave the spec reads as named axes rather than a list: the immutable destination (`providerRef`/`branch`/`path`), `encryption`, `clusterProviderRef`, `placement`, `commit`, `prune`, -and the one switch. That is the test for the next field too — a new member either joins an axis or -names a new one, and if it can do neither it is probably not a `GitTarget` field. +and two object-level switches — `suspend` and `serializeNamespace`. That is the test for the next +field too: a new member joins an axis, or names a new one, or is genuinely object-level — and if it +can do none of the three it is probably not a `GitTarget` field. ## The interactions that change the design @@ -183,11 +187,11 @@ spec: # and ClusterProvider.accessFrom bounds who may wield it. # --- what the documents look like: ADDITIVE, not part of the wave --- + serializeNamespace: false # every write, updates included placement: byType: v1/secrets: "secrets/{name}{sensitiveSuffix}" useKustomize: true # the created kustomization carries namespace: - serializeNamespace: false # --- whether we write, and how those writes are batched and phrased --- suspend: false # the only stop-writes switch; a suspended target still scans diff --git a/docs/future/direction-and-configuration-surface.md b/docs/future/direction-and-configuration-surface.md index cfab36d6..54606ccd 100644 --- a/docs/future/direction-and-configuration-surface.md +++ b/docs/future/direction-and-configuration-surface.md @@ -180,9 +180,9 @@ spec: branch: main path: apps/shop commitWindow: 30s # moved from GitProvider: batching describes this folder + serializeNamespace: false # the artifact is environment-agnostic; namespace at deploy placement: useKustomize: true # an empty repo becomes a buildable folder - serializeNamespace: false # the artifact is environment-agnostic; namespace at deploy ``` and the folder is inert until the tenant subscribes content to it: diff --git a/docs/layout/README.md b/docs/layout/README.md index 1fb2610c..1d30199b 100644 --- a/docs/layout/README.md +++ b/docs/layout/README.md @@ -13,7 +13,7 @@ had in the old layout. Read the label before you read the page. |---|---|---| | [`new-file-placement-rules.md`](new-file-placement-rules.md) | **spec** | where a brand-new resource's file goes: declared, the folder's one kustomize root, canonical. Go source cites it by path, and `task lint-docs` checks those citations | | [`contextual-namespace.md`](contextual-namespace.md) | **spec** | kustomize graph-aware namespace inference, and the supported subset. This is the inference `serializeNamespace` overrides | -| [`model.md`](model.md) | **design** | the proposal, reversed and much smaller: the path template **stays**, and gains two optional booleans on `spec.placement` — `useKustomize` and `serializeNamespace`. Carries the status stanza, the post-scan pass, and the order the work is built in | +| [`model.md`](model.md) | **design** | the proposal, reversed and much smaller: the path template **stays**, and gains two optional booleans — `spec.placement.useKustomize`, and `spec.serializeNamespace` one level up because it governs every write rather than only new files. Carries the status stanza, the post-scan pass, and the order the work is built in | [`examples/`](examples/README.md) makes the proposal tangible: six scenarios, each with a repository folder, the matching `GitTarget` and rule configuration, one live input, and the exact patch the diff --git a/docs/layout/examples/README.md b/docs/layout/examples/README.md index a6f95cd6..29db0022 100644 --- a/docs/layout/examples/README.md +++ b/docs/layout/examples/README.md @@ -61,16 +61,19 @@ starting state, which is the property the index line was pretending to have. ## How to read the proposed fields -Two optional booleans are proposed as new members of the `spec.placement` the current release -already has. Everything else in these scenarios ships today, and most scenarios set neither flag. - -`useKustomize` decides what happens when **no** kustomization governs the path: `true` creates one -at `spec.path` and registers the new file in it. When a kustomization *does* govern the path, the -new file joins its `resources:` either way — that is an invariant, not a setting, which is why -[brownfield-kustomize](brownfield-kustomize/README.md) adopts a kustomize folder with no -configuration at all. - -`serializeNamespace` decides whether `metadata.namespace` appears inside the committed document. A +Two optional booleans are proposed. Everything else in these scenarios ships today, and most +scenarios set neither flag. + +`spec.placement.useKustomize` decides what happens when **no** kustomization governs the path: +`true` creates one at `spec.path` and registers the new file in it. When a kustomization *does* +govern the path, the new file joins its `resources:` either way — that is an invariant, not a +setting, which is why [brownfield-kustomize](brownfield-kustomize/README.md) adopts a kustomize +folder with no configuration at all. It is inside `placement` because, like every other placement +field, it only ever affects a **new** document. + +`spec.serializeNamespace` is one level up, because it does not: it decides the bytes of every write, +including updates to documents already in Git, and it decides whether `metadata.namespace` appears +inside the committed document. A path cannot express it: kustomize takes the namespace from either the document or a governing root, and where the file sits decides neither. Left unset it is inferred, which is today's behavior and what most folders want. `true` is the flat-folder answer — nothing downstream supplies a namespace, diff --git a/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml b/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml index 46be08ae..7c75123b 100644 --- a/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml +++ b/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml @@ -10,10 +10,10 @@ spec: name: app-intent branch: main path: apps/shop + # The folder does not exist yet, so the root that supplies the namespace has to + # be created before the namespace can be left out. useKustomize is what makes + # the omission provable rather than trusted: the operator owns the + # kustomization.yaml it writes namespace: shop into. + serializeNamespace: false placement: - # The folder does not exist yet, so the root that supplies the namespace has to - # be created before the namespace can be left out. useKustomize is what makes - # the omission provable rather than trusted: the operator owns the - # kustomization.yaml it writes namespace: shop into. useKustomize: true - serializeNamespace: false diff --git a/docs/layout/examples/homelab-argocd/config/gittarget.yaml b/docs/layout/examples/homelab-argocd/config/gittarget.yaml index 78d75255..36026330 100644 --- a/docs/layout/examples/homelab-argocd/config/gittarget.yaml +++ b/docs/layout/examples/homelab-argocd/config/gittarget.yaml @@ -9,9 +9,8 @@ spec: branch: main path: bootstrap/argocd-applications suspend: true # adoption dry run: scans and publishes, writes nothing - placement: - # Safe here only because the folder's kustomization.yaml supplies - # namespace: argocd — and that file belongs to the repository owner, not to us. - # The post-scan pass checks that supplier on every scan, so deleting it turns - # the target Validated=False rather than silently relocating documents. - serializeNamespace: false + # Safe here only because the folder's kustomization.yaml supplies + # namespace: argocd — and that file belongs to the repository owner, not to us. + # The post-scan pass checks that supplier on every scan, so deleting it turns + # the target Validated=False rather than silently relocating documents. + serializeNamespace: false diff --git a/docs/layout/examples/homelab-flux/config/gittarget-media.yaml b/docs/layout/examples/homelab-flux/config/gittarget-media.yaml index 161272e8..c21739d2 100644 --- a/docs/layout/examples/homelab-flux/config/gittarget-media.yaml +++ b/docs/layout/examples/homelab-flux/config/gittarget-media.yaml @@ -13,5 +13,4 @@ spec: branch: main path: apps/home/media suspend: true # adoption dry run: scans and publishes, writes nothing - placement: - serializeNamespace: false # the folder's own kustomization.yaml supplies it + serializeNamespace: false # the folder's own kustomization.yaml supplies it diff --git a/docs/layout/examples/homelab-flux/config/gittarget.yaml b/docs/layout/examples/homelab-flux/config/gittarget.yaml index 22bd21f2..b4170f58 100644 --- a/docs/layout/examples/homelab-flux/config/gittarget.yaml +++ b/docs/layout/examples/homelab-flux/config/gittarget.yaml @@ -11,5 +11,4 @@ spec: # GitTarget must not point at a path another controller writes. path: infrastructure/home/sources suspend: true # adoption dry run: scans and publishes, writes nothing - placement: - serializeNamespace: false # the folder's own kustomization.yaml supplies it + serializeNamespace: false # the folder's own kustomization.yaml supplies it diff --git a/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml b/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml index 5f01c37a..ada828fb 100644 --- a/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml +++ b/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml @@ -9,5 +9,4 @@ spec: branch: main path: apps/podinfo/overlays/prod suspend: true # adoption dry run: scans and publishes, writes nothing - placement: - serializeNamespace: false # the overlay's kustomization.yaml supplies it + serializeNamespace: false # the overlay's kustomization.yaml supplies it diff --git a/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml b/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml index d1d4547a..d783f314 100644 --- a/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml +++ b/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml @@ -11,8 +11,7 @@ spec: branch: main path: clusters/home suspend: true # adoption dry run: scans and publishes, writes nothing - placement: - # No template declared and no kustomization anywhere in the subtree, so the - # ladder falls through to the canonical identity path. Nothing supplies a - # namespace, so every namespaced document has to carry its own. - serializeNamespace: true + # No template declared and no kustomization anywhere in the subtree, so the + # ladder falls through to the canonical identity path. Nothing supplies a + # namespace, so every namespaced document has to carry its own. + serializeNamespace: true diff --git a/docs/layout/model.md b/docs/layout/model.md index a8afa2e9..4c05ca05 100644 --- a/docs/layout/model.md +++ b/docs/layout/model.md @@ -23,17 +23,31 @@ document. ```yaml spec: path: apps/demo + serializeNamespace: false # optional bool, unset = infer placement: byType: # unchanged, as shipped v1/secrets: "secrets/{name}{sensitiveSuffix}" default: "{namespace}/{resource}/{name}.yaml" # unchanged, as shipped useKustomize: true # bool, default false - serializeNamespace: false # optional bool, unset = infer ``` -Two booleans, inside the struct that already holds the placement axis. No discriminator, no -`spec.layout`, no `kind`, no `scope`, no new CRD, and no enum. `spec.placement` is an existing -optional struct, so both are additive: an object that says nothing behaves exactly as it does today. +Two booleans. No discriminator, no `spec.layout`, no `kind`, no `scope`, no new CRD, and no enum. +Both are additive: an object that says nothing behaves exactly as it does today. + +**They sit at different levels, and the line between them is retroactivity.** `spec.placement` +decides where a **new** document goes; everything already written is match-first and never moves, +which is the guarantee that makes a template safe to change. `useKustomize` keeps that property — it +decides whether a new file's directory has a root to join, and creates one if not — so it belongs +inside `placement`. + +`serializeNamespace` does not. It governs the bytes of **every** write, not just the first one, and +the code already has two separate paths for it: [`plan_flush.go`](../../internal/git/plan_flush.go) +strips `metadata.namespace` from a new document using the placement result, and strips it from an +**update** using the document's own observed namespace source. It also decides how a managed +document is *found* — a document whose namespace is inherited is located in the file bytes by a +namespace-less identity. A field with those effects nested inside a struct documented as "new files +only" is a trap: changing it rewrites existing documents as they are next touched. So it is +`spec.serializeNamespace`, one level up, where its blast radius is legible. ## What changed @@ -148,6 +162,32 @@ be read as policy. for cluster-scoped documents rather than being an error — worth stating in the field documentation, because a tree folder is the type most likely to carry both. +### Is a folder-wide claim reasonable? + +Setting the flag says something about *every* namespaced document in the folder, which sounds like a +strong claim until you notice it is the claim the ecosystem's two commonest folder shapes already +make. + +- **"No document carries its namespace" is the portable-artifact convention.** It is what a kustomize + base is: written namespace-free, with an overlay's `namespace:` transformer stamping one at build + time. Flux says the same thing from outside the repository with `Kustomization.spec.targetNamespace`, + Argo CD with `Application.spec.destination.namespace`, and a Helm chart by templating + `.Release.Namespace` rather than hard-coding a namespace. A folder that follows it can be deployed + into any namespace; a folder that half-follows it cannot, and nothing warns you. +- **"Every document carries its namespace" is the convention for a folder applied directly.** A flat + mirror handed to `kubectl apply -f`, or a cluster-state tree, has nothing downstream to supply one, + so a document without a namespace is ambiguous rather than portable. + +Two things keep the claim from being coarser than reality. **Cluster-scoped resources are exempt**, +so a folder holding both a `ClusterRole` and a `Deployment` is ordinary rather than mixed. And a +folder is genuinely allowed to be non-uniform when it is a **tree of nested roots** — fact 2 above — +each subtree taking its namespace from its own `kustomization.yaml`. That case is exactly what the +default handles: inference runs per document against the root that governs *that* path, so a tree +resolves each subtree correctly without anyone declaring anything. + +So the uniform claim is what an explicit setting is *for*, and the non-uniform folder is what unset +is for. That is also why unset cannot be spelled `false`. + ### The guard on `false`, and why it is a post-scan check `false` where nothing supplies the namespace hands the object to whatever namespace the applier From 1649a255e130476d6ac986a06481039501a18181 Mon Sep 17 00:00:00 2001 From: Simon Koudijs Date: Sat, 29 Aug 2026 18:55:37 +0000 Subject: [PATCH 3/3] test(namespace): pin the two folder shapes where reading the folder is not enough MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Answers whether the layout situations under discussion can hurt us, by measuring rather than reasoning. Two shapes exist where the manifest store's view of a document's namespace and the namespace kustomize renders it into can disagree, and neither is refused when the folder is read: - a document carrying metadata.namespace: beta inside a folder whose kustomization sets namespace: alpha. An explicit namespace is authoritative as written and the transformer is never consulted, so the store indexes beta with NO diagnostic while the folder renders alpha. - a parent root and its child root that both assign. kustomize is deterministic here (the parent transformer runs last and wins); we decline to guess, so the document is namespace-less, unmatchable by identity, and placement treats the live object as new. Both are caught, and nothing reaches the worktree: the first refuses with "does not render to the live object after the write", and the second gets as far as proposing a second file before kustomize's own ID conflict fails the build — neither the file nor the resources: entry survives. So the render check at the write path is the backstop for this whole class, and it holds. Nothing pinned that until now: a change that relaxed the check would have turned a refusal into a silent write with no test failing. Adds the read-side halves to the contextual-namespace corpus, which docs/layout/contextual-namespace.md says is meant to grow one folder per "can we support X?" question, plus the multi-namespace tree that was missing from it — a parent assigning nothing and each child root carrying its own namespace, which is the shape the model leaves to inference rather than to a flag. The remaining gap is legibility, not safety: both failures surface as an opaque render error rather than as the one fixable thing that is wrong, which is what the post-scan validation pass is for. Co-Authored-By: Claude Opus 5 --- docs/layout/contextual-namespace.md | 22 +++ docs/layout/model.md | 10 ++ .../git/namespace_context_refusal_test.go | 132 ++++++++++++++++++ .../contextual_namespace_corpus_test.go | 32 +++++ .../kustomization.yaml | 5 + .../nested-roots-per-namespace/media/cm.yaml | 6 + .../media/kustomization.yaml | 5 + .../monitoring/cm.yaml | 6 + .../monitoring/kustomization.yaml | 5 + .../conflicting-explicit-namespace/cm.yaml | 7 + .../kustomization.yaml | 5 + .../nested-both-namespaces/kustomization.yaml | 5 + .../nested-both-namespaces/media/cm.yaml | 6 + .../media/kustomization.yaml | 5 + 14 files changed, 251 insertions(+) create mode 100644 internal/git/namespace_context_refusal_test.go create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/kustomization.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/media/cm.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/media/kustomization.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/monitoring/cm.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/monitoring/kustomization.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/unsupported/conflicting-explicit-namespace/cm.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/unsupported/conflicting-explicit-namespace/kustomization.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/kustomization.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/media/cm.yaml create mode 100644 internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/media/kustomization.yaml diff --git a/docs/layout/contextual-namespace.md b/docs/layout/contextual-namespace.md index c423522c..c6bca2c7 100644 --- a/docs/layout/contextual-namespace.md +++ b/docs/layout/contextual-namespace.md @@ -443,9 +443,12 @@ folder. |---|---|---| | `supported/flat-namespace` | one kustomization, `namespace:`, flat `resources` | namespace inherited (`Kustomize`) | | `supported/nested-base` | parent `namespace:` + child dir base with no namespace | namespace propagates through the graph | +| `supported/nested-roots-per-namespace` | parent assigns nothing; each child root carries its own `namespace:` | each document inherits from the root that governs **it**, so one folder holds two namespaces | | `supported/multi-doc` | a multi-document file in `resources` | every document inherits | | `supported/explicit-namespace` | `metadata.namespace` written in the file | kept as-is (`Explicit`) | | `unsupported/ambiguous-two-roots` | two roots assign different namespaces to one file | refused (`ambiguous-namespace`) | +| `unsupported/nested-both-namespaces` | a parent root and its child root both assign | refused (`ambiguous-namespace`), deliberately: kustomize resolves this (the parent transformer runs last and wins) and we decline to, because guessing hands the document to a namespace nobody wrote down | +| `unsupported/conflicting-explicit-namespace` | the document names a namespace the governing transformer overrides | **no diagnostic**: an explicit namespace is authoritative as written, so the store indexes `beta` while the folder renders `alpha`. The refusal lives entirely at the write path | | `unsupported/patches` | `patches:` present | not a namespace source (`None`) | | `unsupported/generators` | `configMapGenerator:` present | not a namespace source | | `unsupported/components` | `components:` present | not a namespace source | @@ -458,6 +461,25 @@ Today the store records the per-document outcome (`NamespaceSource` and the diagnostics); the `unsupported/*` folders that currently resolve to `None` are the inputs the pending `RepositoryValid` refusal will turn into a failed GitTarget. +**The last two rows are the ones where reading the folder is not enough**, and what +catches them is the render check at the write path rather than anything here. Both were +measured rather than assumed, and both are pinned by a write-side test +(`TestPlanFlush_RefusesWhenTransformerOverridesExplicitNamespace` and +`TestPlanFlush_RefusesWhenNestedRootsBothSetNamespace` in +[`internal/git`](../../internal/git/namespace_context_refusal_test.go)): + +- the conflicting-explicit folder refuses with *"does not render to the live object + after the write"*, and nothing is committed; +- the nested-both folder gets as far as **proposing a second file** for an object it + could not match by identity, and the build then fails with kustomize's own + `namespace transformation produces ID conflict`. Nothing reaches the worktree — + not the file, not the `resources:` entry it would have needed. + +So neither shape can produce a wrong commit. Both, however, surface as an opaque render +error rather than as the specific fixable thing that is wrong, which is a legibility gap +worth closing when the post-scan validation pass is built: this class is exactly what it +is for. + ## E2E test shape The fixture-backed e2e test is still the right acceptance test once the design is diff --git a/docs/layout/model.md b/docs/layout/model.md index 4c05ca05..c3d3825f 100644 --- a/docs/layout/model.md +++ b/docs/layout/model.md @@ -201,6 +201,16 @@ It is one post-scan rule, on the scan that already runs, setting `Validated=Fals naming the field and what the folder actually contains. With `useKustomize: true` the rule is satisfied by construction, because the operator wrote the supplier. +**Nothing here is the last line of defence, which is why the guard can be a report rather than a +refusal.** The render check at the write path already refuses a write whose document does not render +to the live object, and it holds for both shapes where the store's view and kustomize's disagree: a +document naming a namespace the governing transformer overrides, and a folder whose nested roots both +assign one. Both are measured, not assumed, and both are pinned by +[`namespace_context_refusal_test.go`](../../internal/git/namespace_context_refusal_test.go) with the +read-side halves in the contextual-namespace corpus. What the post-scan rule adds is not safety but +legibility: today those failures surface as an opaque render error rather than as the one fixable +thing that is wrong. + ## Collisions are already decided What happens when two resources resolve to the same path is specified and shipped in diff --git a/internal/git/namespace_context_refusal_test.go b/internal/git/namespace_context_refusal_test.go new file mode 100644 index 00000000..e45515ed --- /dev/null +++ b/internal/git/namespace_context_refusal_test.go @@ -0,0 +1,132 @@ +// SPDX-License-Identifier: Apache-2.0 + +package git + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + + "github.com/ConfigButler/gitops-reverser/api/v1alpha3" + "github.com/ConfigButler/gitops-reverser/internal/types" + "github.com/ConfigButler/gitops-reverser/internal/typeset" +) + +// The two folder shapes below are the ones where the manifest store's view of a +// document's namespace and the namespace kustomize actually renders it into can +// disagree. Neither is refused when the folder is read — one is indexed under the +// namespace the file names, the other is indexed under no namespace at all — so the +// only thing standing between them and a wrong commit is the render check at the +// write path. These tests pin that backstop: without them, a change that relaxed the +// check would turn a refusal into a silent write and nothing would fail. +// +// The read-side halves live in the contextual-namespace corpus as +// unsupported/conflicting-explicit-namespace and unsupported/nested-both-namespaces +// (docs/layout/contextual-namespace.md). + +// namespaceProbeMapper serves ConfigMap as a namespaced, followable type, which is what +// the namespace context resolution needs to run at all. +func namespaceProbeMapper() typeset.Lookup { + return typeset.NewSnapshotRegistry(typeset.Snapshot{Entries: []typeset.Entry{{ + GVK: schema.GroupVersionKind{Version: "v1", Kind: "ConfigMap"}, + GVR: schema.GroupVersionResource{Version: "v1", Resource: "configmaps"}, + Namespaced: true, + Allowed: true, + }}}) +} + +func namespaceProbeEvent(namespace, name, color string) Event { + return Event{ + Object: &unstructured.Unstructured{Object: map[string]interface{}{ + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": map[string]interface{}{"name": name, "namespace": namespace}, + "data": map[string]interface{}{"color": color}, + }}, + Identifier: types.ResourceIdentifier{ + Group: "", Version: "v1", Resource: "configmaps", Namespace: namespace, Name: name, + }, + Operation: "UPDATE", + } +} + +func seedFile(t *testing.T, root, rel, body string) string { + t.Helper() + full := filepath.Join(root, rel) + require.NoError(t, os.MkdirAll(filepath.Dir(full), 0o750)) + require.NoError(t, os.WriteFile(full, []byte(body), 0o600)) + return full +} + +// A document carrying metadata.namespace: beta inside a folder whose kustomization +// sets namespace: alpha is indexed under beta — an explicit namespace is authoritative +// as written, and the store does not consult the transformer. kustomize disagrees: its +// namespace transformer overrides an explicit metadata.namespace, so the folder renders +// alpha/cm. The write must be refused rather than committed against a document the +// folder does not actually deploy. +func TestPlanFlush_RefusesWhenTransformerOverridesExplicitNamespace(t *testing.T) { + writer := newContentWriter(types.SensitiveResourcePolicy{}) + worktree := newWorktreeForTest(t) + root := worktree.Filesystem().Root() + + seedFile(t, root, "kustomization.yaml", + "apiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\n"+ + "namespace: alpha\nresources:\n- cm.yaml\n") + docPath := seedFile(t, root, "cm.yaml", + "apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: cm\n namespace: beta\ndata:\n color: blue\n") + + worker := &BranchWorker{contentWriter: writer, mapper: namespaceProbeMapper()} + changed, err := worker.flushEventsToWorktree( + t.Context(), worktree, "", + []Event{namespaceProbeEvent("beta", "cm", "green")}, nil, v1alpha3.PruneOnEvent) + + require.Error(t, err, "the folder renders alpha/cm while the mirror holds beta/cm; the write must refuse") + assert.Contains(t, err.Error(), "does not render to the live object") + assert.False(t, changed) + + got, readErr := os.ReadFile(docPath) + require.NoError(t, readErr) + assert.Contains(t, string(got), "color: blue", "a refused flush leaves the document untouched") +} + +// A parent root and its child root both set namespace:, so two namespaces reach one +// document and the store refuses to infer either — leaving the document namespace-less +// and therefore unmatchable by identity. Placement then treats the live object as new +// and proposes a second file, which would render two ConfigMaps into the same namespace +// under the same name. The render check is what stops it, and it must stop it before +// anything reaches the worktree, including the resources: entry the new file would need. +func TestPlanFlush_RefusesWhenNestedRootsBothSetNamespace(t *testing.T) { + writer := newContentWriter(types.SensitiveResourcePolicy{}) + worktree := newWorktreeForTest(t) + root := worktree.Filesystem().Root() + + rootKustomization := "apiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\n" + + "namespace: outer\nresources:\n- media\n" + seedFile(t, root, "kustomization.yaml", rootKustomization) + seedFile(t, root, "media/kustomization.yaml", + "apiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\n"+ + "namespace: inner\nresources:\n- cm.yaml\n") + seedFile(t, root, "media/cm.yaml", + "apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: m\ndata:\n color: blue\n") + + worker := &BranchWorker{contentWriter: writer, mapper: namespaceProbeMapper()} + changed, err := worker.flushEventsToWorktree( + t.Context(), worktree, "", + []Event{namespaceProbeEvent("outer", "m", "green")}, nil, v1alpha3.PruneOnEvent) + + require.Error(t, err, "a second document for the same rendered object must not be committed") + assert.False(t, changed) + + _, statErr := os.Stat(filepath.Join(root, "outer", "configmaps", "m.yaml")) + assert.True(t, os.IsNotExist(statErr), "the duplicate file placement must not survive the refusal") + + gotRoot, readErr := os.ReadFile(filepath.Join(root, "kustomization.yaml")) + require.NoError(t, readErr) + assert.Equal(t, rootKustomization, string(gotRoot), + "the resources: entry for the refused file must not survive either") +} diff --git a/internal/manifestanalyzer/contextual_namespace_corpus_test.go b/internal/manifestanalyzer/contextual_namespace_corpus_test.go index 8f658498..af9d6c6b 100644 --- a/internal/manifestanalyzer/contextual_namespace_corpus_test.go +++ b/internal/manifestanalyzer/contextual_namespace_corpus_test.go @@ -53,6 +53,17 @@ func TestContextualNamespaceCorpus(t *testing.T) { {namespace: "app", name: "two", source: NamespaceKustomize}, }, }, + { + // A tree whose parent root assigns nothing and whose children each carry + // their own namespace: transformer. Each document takes the namespace of the + // root that governs IT, so one folder legitimately holds two namespaces — + // the shape docs/layout/model.md leaves to inference rather than to a flag. + dir: "supported/nested-roots-per-namespace", + docs: []wantDoc{ + {namespace: "media", name: "m", source: NamespaceKustomize}, + {namespace: "monitoring", name: "n", source: NamespaceKustomize}, + }, + }, { dir: "supported/explicit-namespace", docs: []wantDoc{{namespace: "explicit-ns", name: "cm", source: NamespaceExplicit}}, @@ -62,6 +73,27 @@ func TestContextualNamespaceCorpus(t *testing.T) { docs: []wantDoc{{name: "shared", source: NamespaceNone}}, ambiguousDiag: true, }, + { + // The nested form of the same conflict: a parent and its child root both + // assign, so two namespaces reach one document. kustomize is deterministic + // here (the parent transformer runs last and wins) and we deliberately are + // not: guessing would hand the document to a namespace nobody wrote down. + // The write-side consequence is pinned by + // TestPlanFlush_RefusesWhenNestedRootsBothSetNamespace. + dir: "unsupported/nested-both-namespaces", + docs: []wantDoc{{name: "m", source: NamespaceNone}}, + ambiguousDiag: true, + }, + { + // The document names beta; the governing kustomization's transformer rewrites + // it to alpha, so the folder renders an object the store does not index. The + // store reports Explicit with NO diagnostic, because an explicit namespace is + // authoritative as written and the transformer is not consulted. This is the + // one shape in the corpus whose refusal lives entirely at the write path: + // TestPlanFlush_RefusesWhenTransformerOverridesExplicitNamespace. + dir: "unsupported/conflicting-explicit-namespace", + docs: []wantDoc{{namespace: "beta", name: "cm", source: NamespaceExplicit}}, + }, {dir: "unsupported/patches", docs: []wantDoc{{name: "cm", source: NamespaceNone}}}, {dir: "unsupported/generators", docs: []wantDoc{{name: "cm", source: NamespaceNone}}}, {dir: "unsupported/components", docs: []wantDoc{{name: "cm", source: NamespaceNone}}}, diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/kustomization.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/kustomization.yaml new file mode 100644 index 00000000..88b3aa63 --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: +- media +- monitoring diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/media/cm.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/media/cm.yaml new file mode 100644 index 00000000..eac4a071 --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/media/cm.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: m +data: + k: v diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/media/kustomization.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/media/kustomization.yaml new file mode 100644 index 00000000..5737fb8f --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/media/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: media +resources: +- cm.yaml diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/monitoring/cm.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/monitoring/cm.yaml new file mode 100644 index 00000000..1f17a009 --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/monitoring/cm.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: n +data: + k: v diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/monitoring/kustomization.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/monitoring/kustomization.yaml new file mode 100644 index 00000000..f8dc051d --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/supported/nested-roots-per-namespace/monitoring/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: monitoring +resources: +- cm.yaml diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/conflicting-explicit-namespace/cm.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/conflicting-explicit-namespace/cm.yaml new file mode 100644 index 00000000..b544fbc3 --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/conflicting-explicit-namespace/cm.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: cm + namespace: beta +data: + k: v diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/conflicting-explicit-namespace/kustomization.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/conflicting-explicit-namespace/kustomization.yaml new file mode 100644 index 00000000..4f9af7f5 --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/conflicting-explicit-namespace/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: alpha +resources: +- cm.yaml diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/kustomization.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/kustomization.yaml new file mode 100644 index 00000000..0f1131da --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: outer +resources: +- media diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/media/cm.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/media/cm.yaml new file mode 100644 index 00000000..eac4a071 --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/media/cm.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: m +data: + k: v diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/media/kustomization.yaml b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/media/kustomization.yaml new file mode 100644 index 00000000..2a1523b1 --- /dev/null +++ b/internal/manifestanalyzer/testdata/contextual-namespace/unsupported/nested-both-namespaces/media/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: inner +resources: +- cm.yaml