diff --git a/api/v1alpha3/gittarget_types.go b/api/v1alpha3/gittarget_types.go index 65ba8d68..dfc6125a 100644 --- a/api/v1alpha3/gittarget_types.go +++ b/api/v1alpha3/gittarget_types.go @@ -149,7 +149,7 @@ type GitTargetSpec struct { // GitTargetPlacementSpec declares where NEW resources are written when no document // for their identity exists yet in Git — one exact-type map plus a fallback // default template (Option B2 of -// docs/spec/gittarget-new-file-placement-rules.md). There is +// docs/layout/new-file-placement-rules.md). There is // deliberately no separate "sensitive" placement block: sensitivity is a // write-safety classification the controller owns (encrypt the content, keep the // path identity-complete, never append or co-mingle), not a second placement diff --git a/docs/INDEX.md b/docs/INDEX.md index 3bef7830..7f5b2ba7 100644 --- a/docs/INDEX.md +++ b/docs/INDEX.md @@ -4,7 +4,7 @@ This page names the documents that bind the current implementation. If a documen listed here, it is either a user guide (see [`README.md`](README.md)) or historical context you can safely skip. -## The four folders, and what each one means +## The folders, and what each one means | Folder | Means | Binds? | |---|---|---| @@ -12,6 +12,7 @@ you can safely skip. | [`design/`](design/) | **We are still deciding.** Open questions, proposals, unbuilt work. | yes — as intent, not as shipped behaviour | | [`facts/`](facts/) | Durable reference: how Kubernetes behaves, and what we discovered about it. | yes, as reference | | [`finished/`](finished/) | **This happened.** Shipped plans, closed investigations. Kept for context. | **no** | +| [`layout/`](layout/README.md) | **One topic, all of it.** Where a document goes in Git: two current-behaviour contracts, the proposal, its plan, its review, and the worked examples. | per document, and each is labelled | The rule that was missing before: `design/` used to hold shipped work and `finished/` used to hold live contracts. If you are adding a document, pick the @@ -25,6 +26,13 @@ declares itself non-binding. It is labelled **built.** in the table below instea the lifecycle is still readable. `spec/` is for a contract stated as a contract; this is a plan whose reasoning the code kept. +A second exception, and this one is about topic rather than lifecycle. [`layout/`](layout/README.md) +collects the whole layout question, which had grown to eight documents across three folders, so +following the argument meant knowing which folder each step lived in. It mixes lifecycle classes +on purpose and labels every entry with the class it would have had, which its +[README](layout/README.md) does. It is the only topic folder, and adding a second one should take +the same amount of argument this one did. + ## If you are new: read these five 1. [`../README.md`](../README.md) — what the operator does. @@ -58,8 +66,6 @@ misled. Full list in [`spec/README.md`](spec/README.md); the ones that carry a | [`current-manifest-support-review.md`](spec/current-manifest-support-review.md) | all-or-nothing folder claim; never half-write a multi-doc file; **refuse rather than prune** | | [`manifestedit-field-ownership-spike.md`](spec/manifestedit-field-ownership-spike.md) | the API wins — full-object ownership, never field-subset | | [`reconcile-via-watchlist-mark-and-sweep.md`](spec/reconcile-via-watchlist-mark-and-sweep.md) | **no bookmark, no sweep** | -| [`contextual-namespace-and-kustomize-folder-editing.md`](spec/contextual-namespace-and-kustomize-folder-editing.md) | kustomize namespace inference; the supported subset | -| [`gittarget-new-file-placement-rules.md`](spec/gittarget-new-file-placement-rules.md) | where a new resource's file goes: declared, the folder's one kustomize root, canonical. Sibling inference is removed, and kept as history | | [`sops-single-file-no-multidoc.md`](spec/sops-single-file-no-multidoc.md) | one encrypted file is one document | | [`scale-subresource-audit-rehydration.md`](spec/scale-subresource-audit-rehydration.md) | `/scale` only; every other subresource ignored | | [`commit-window-refactor.md`](spec/commit-window-refactor.md) | one grouped commit = one (author, GitTarget) | @@ -97,9 +103,6 @@ Eighteen other open items: | Doc | Open question | |---|---| | [`open-asks-priority.md`](design/open-asks-priority.md) | **the work queue.** Swept 2026-07-30 against the 0.41.0 release, which carries the attribution model and the placement break and deliberately **not** the GitTarget wave: every Tier 2 entry that changes a `GitTarget` field is part of postponed [#294](https://github.com/ConfigButler/gitops-reverser/issues/294) and is not independently schedulable, while the Tier 1 entries are not and must not wait for it. One entry moved up in the sweep, to Tier 1: a declared path into a kustomize subdirectory produces a file that is in Git and rendered by nothing, which is the product being silently wrong. Reconciled 2026-07-29 to what the attribution branch shipped: the fact stream, consumer ask #23, the name tier and metrics Phase 1 are struck from it and recorded in "already shipped", the residue they leave (the removal-wait decision, the head-of-line block on the shard, the aggregated create) is ranked, and #5 loses one of its two arguments because #23's fix retired it. Three backlogs are open at once — the gitops-api consumer asks, the maintainer review's unbuilt block (F6, F9, F10), and the config-surface proposal (B1–B6) — and they overlap. Merges them into one ordered queue under four stated tests, and makes one design call against what was asked: **delete Option C sibling inference** rather than ship an off-switch for it, because it lets a human's edit to the repository change the operator's behaviour with nothing in status recording the move, its central guard has already failed once by cascading, and the explainability its own spec made mandatory was never built. That answers the namespace-leak ask by removal, and means `spec.placement.mode` is never built. **The deletion has shipped**, together with the placement metrics the argument had said to lead *away* from — an objection to their labels, which naming the GitTarget and the type retires — and "what the deletion taught" records the two things building it found: namespace inheritance was a second implementation of a rule that belonged to the governing kustomization, and it was missing the check that the transformer names the resource's own namespace. Open: whether a fall-back to canonical also raises an Event on the GitTarget, and `status.layout` | -| [`placement-visibility-and-declared-defaults.md`](design/placement-visibility-and-declared-defaults.md) | the three questions the inference deletion left, **decided and then not built** — PR #291 shipped the deletion, the counters and the namespace-transformer fix, and none of the eight items this page had queued behind them, which the page now says. The residue is filed as [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) (correctness: a declared path into a kustomize subdirectory is never rendered, and the identity gate rejects the versionless canonical path) and [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) (visibility: `status.layout`, the ambiguous render root, the `declared` metric split, `{kindLower}`). Its Question 2 is superseded outright by the layout model. What still stands: **Keep `canonical`** as the name for the built-in path and split `declared` into `byType`/`default`, because reusing "default" for both a declaration and the absence of one makes the metric unreadable. **No CRD default for `placement.default`**, and the reason is structural rather than the two that look obvious: a defaulted default is never empty, so it shadows the kustomize-root step and every new file in an overlay would take the canonical path, in Git and rendered by nothing. The validation failure is real but the rule doing the rejecting is itself wrong, and the persistence objection is a trade we could take; defaulting the Secret route to work around the first is a floor that vanishes when a user writes any `byType` entry, because map defaults never merge. **`status.layout` instead**, with five worked examples (greenfield, kustomize overlay, brownfield missing one rule, two ambiguous roots, a refusal from an operator-configured sensitive type) over the `MarkTargetRetention` seam, which already enqueues on change and so retires the "the data plane cannot notify the GitTarget" objection. **`{kindLower}`, not a `toLower` function.** Carries three findings that changed a decision: `IdentityCompletePlacementTemplate` demanding `{version}` contradicts the versionless-path decision; two supported kustomizations still produce a file nothing renders and nothing counts; and **a declared path into a subdirectory of a kustomize folder is registered only when render-root scoping happens to be in force**, so one `byType` line reproduces the unrendered-file bug today. Fixing that last one (walk up to the nearest kustomization) also weakens the case against the CRD default from a correctness wall to a legibility trade, which the page says rather than leaving the stronger argument standing | -| [`gittarget-layout-model.md`](design/gittarget-layout-model.md) | **postponed to a later deployment, filed as [#293](https://github.com/ConfigButler/gitops-reverser/issues/293)** — the proposal the placement questions were circling around: a path template is the wrong primitive, so declare what the folder IS. `spec.layout.kind` with the values `Auto`, `Kustomize`, `Tree`, `Flat` and `Template`, plus `byType` overrides valid under every kind, with two rules that carry the value: whatever chose the path, the file is registered with the kustomization that governs it (so F10 becomes unstatable rather than fixed), and a structural kind excludes a blanket `default` (so a declared template can no longer silently disable the render root). `kind: Auto` is a safe CRD default because it NAMES the structural rule instead of standing in front of it, which is why defaulting a mode works where defaulting a path did not, and it is declared inference rather than the undeclared kind that was deleted. `kind: Kustomize` with `create: true` bootstraps an empty repository into a folder `kubectl apply -k` can build. Seven worked examples, a status shape with `declaredKind` beside the resolved `kind`, a mechanical migration for every current configuration, and an argument that the layout should NOT be its own CRD: a shared object changing where N folders write, with nothing on the GitTarget recording it, is the same defect as sibling inference with a different actor, the shared thing is four lines, and generators already solve reuse. Also carries the namespace half: `scope: SingleNamespace` is a STRUCTURAL claim that must agree with the authorization bound `allowedSourceNamespaces`, and it cannot be derived because that matcher may be absent and because the namespaces that arrive come from WatchRule objects that do not own the folder; `writeNamespace` with the values `FromContext`, `Always` and `Never` replaces the inference that decides whether `metadata.namespace` is written, which is the one inference an empty folder cannot perform, and `create: true` lets the operator ESTABLISH the convention by writing `namespace:` into the kustomization it creates. The layout is **immutable** except a widening transition, because GitTarget has no finalizer so recreating one re-adopts every document by identity, and `Auto` resolves once and pins the result so a deleted `kustomization.yaml` cannot silently re-lay-out the folder. Open: whether `scope` should be derived and materialized at creation instead of declared | -| [`gittarget-api-wave.md`](design/gittarget-api-wave.md) | **postponed, filed as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). Not in 0.41.0**, which already carries the attribution model and the placement break. One breaking wave on GitTarget, sequencing the layout model with the maintainer review's still-open API block (F6, F10, F12's reference nit, §3's pushbacks) and the queue's Tier 2 items (B4, B1, #5, #6). The batching argument is the weaker half; the stronger one is that four of them are the same decision seen from different angles: **the folder is described on the GitTarget and the connection describes only the connection**, which is why `commitWindow` and `commit.message` move off `GitProvider`. Two findings change the layout design rather than accompanying it: `spec.mode: Observe` becomes how a layout is adopted safely (a dry run over `status.layout` instead of declare-and-hope), and `spec.interval` plus an observation pass is what keeps the scan-derived half of that status fresh for a target that writes nothing. `spec.suspend` is a precondition rather than a rider, because a layout that creates a `kustomization.yaml` needs a stop button. Records that F7 already shipped the EventRecorder the placement Event was said to be too expensive for, that layout is mutable like `prune`, that F9 stays OUTSIDE the wave because its answer constrains the enum work, and that the version stays `v1alpha3` with loud rejections rather than paying for a conversion path | | [`target-watch-plan.md`](design/target-watch-plan.md) | **built.** The companion to [`watch-manager-ownership.md`](design/watch-manager-ownership.md): the ownership page says WHO applies a plan, this one says WHAT a plan is and what changing it may touch. A cell — group, resource, namespace, deliberately no served version — is the one identity the watch stream, the render-fidelity scope and the mark-and-sweep boundary all agree on, because a key that does not round-trip to the scope it sweeps under is the class of error that deletes user data. The plan is diffed into `keep`/`start`/`restart`/`stop` and applied per cell, so adding one WatchRule stops replaying every unrelated cell into a queue shared with other tenants; a `restart` is a served-version change, which is why the version is spec DATA rather than identity. Readiness and the fidelity revision are per scope, so a KEPT cell holds the result its own replay produced rather than being asked to prove itself again over an unrelated edit. `stop` never touches files — removal is a Git-side sweep under the target's existing `spec.prune.mode`, not a watch-layer delete. "Cut at the producer" is the accepted consequence: nothing fences the queue, so a deselected cell may leave a short tail of writes, bounded by the queue and converged afterwards. Still open: the `stop` classification wants a settled `TypeRemoved` from `typeset` (see TODO), and removal on INTENT is undecided. | | [`watch-manager-ownership.md`](design/watch-manager-ownership.md) | **built.** A rule edit used to be applied inline by the controller worker that observed it, and it re-planned EVERY GitTarget rather than the one the rule names: 1256 plan reconciles across 28 targets in one e2e run, peaking at 78 in a second, behind two network calls, on a shared worker pool. The watch manager had no owner, so eleven mutexes stood in for one. Now controllers post a trigger naming a GitTarget and return, one loop owns the plan and paces itself, and repeated triggers for one target collapse into a single pass. The debounce is framed around how the config is actually edited: a GitTarget and its rules are one piece of configuration applied together, so a per-target ROLLING SILENCE window of 2s (max wait ~10s) turns a five-object `kubectl apply` into one pass, the same mechanism `DefaultCommitWindow` already uses one layer down on the write path. That reverses an earlier revision's "never debounce the first declaration": declaring a GitTarget the instant it lands means declaring it with no rules yet, which manufactures a transient EMPTY plan on every cold start, and an empty plan is what vacuously cleared a write divergence in the fidelity gate. States the contract as "one settled configuration adjustment, not one function invocation": the window is a heuristic and never a correctness boundary (Kubernetes has no apply-complete event), a per-target DIRTY SEQUENCE means a change arriving mid-pass is never lost, and the pass reads a coherent rule-store snapshot rather than the rule that triggered it. Carries the deletion inventory, because the point is that the system got smaller: four trigger mechanisms collapsed to one (`signalCatalogRefresh` and `catalogRefreshCh` are gone), `refreshRunningTargetWatches` and its running-set filter are gone (that filter is why a target whose first declare never completed was never picked up again), and six mutexes went for stated reasons, with `RenderFidelityGate.mu` and the two event-channel locks kept and justified. The four steps shipped: the local-cluster discovery call is bounded (a real defect — the legacy non-context `ServerGroupsAndResources()` ran with no deadline at all), the owner loop carries the debounce, dirty sequence, per-target deadline and 2s/5s/10s/30s/1m backoff, the plan now carries NO lock while the projection is a published snapshot, and catalog invalidation is scoped by diffing each target's rendered plan across the re-projection. "What shipped" records where the implementation departed from the page, including the one bug only e2e caught: streams were parented to the PASS context, which a deadline cancels the moment the pass returns, so every stream died the instant its plan was applied — and it reads like health, because the plan logs `start:1`, every later pass reports `keep:1` and never restarts it, and nothing logs an error while readiness sits at `Replaying` and every WatchRule sits `Ready=False`. A stream's parent is the manager's lifetime; the pass deadline bounds the pass. A second e2e catch is a BEHAVIORAL consequence worth knowing: toggling a rule off and on inside the settle window is no longer a replay — it used to tear the stream down and re-establish it because each apply replanned synchronously, and it is now one pass over a plan that never changed. Correct (a net-zero change is no change; widening `prune.mode` remains the supported force) but a real difference in what an operator gesture does. Both specs that broke were also gating on the wrong thing: asking a GitTarget "are all your streams running" about a change to ONE rule, which a target that is already mirroring answers True to before that rule has been planned, and which a different controller publishes than the one that compiled the rule — so the rule's OWN StreamsRunning is the gate, and `waitForWatchRuleStreamsRunning` existed unused for exactly this. Departures: reports became a published snapshot rather than a second channel; ISOLATION came from taking the I/O off the loop rather than from the deadline (a pass never dials, the shared refresh runs on its own goroutine, and the deadline is the backstop it should have been — a first cut that kept two network calls on the loop had one unreachable cluster holding every healthy target, which is the same availability failure relocated); DELETION names an incarnation resolved when it is queued, because both production callers react to a NotFound and carry no UID, so a UID-less delete matched everything and could tear down the successor of a same-name recreate; and persistent failure surfaces as `WatchPlanFailing`, where pending means "no pass has ever landed", not "dirty right now". Step 3's type-to-target index did not earn its staleness. Still open: whether the settle window ever needs to be configurable | | [`docs-linting.md`](design/docs-linting.md) | how to mechanize [`style-guide.md`](style-guide.md) with markdownlint-cli2 and Vale. Both are wired into `task lint`, gated on the files [`.docs-lint-scope`](../.docs-lint-scope) lists rather than the whole tree: 102 of 174 files fail markdownlint and 148 of 174 fail Vale, so the two backlogs need different gates. Open: how the scope list grows to cover the tree, the `MD013` limit, and whether `AGENTS.md` and the chart READMEs are in scope | @@ -114,6 +117,33 @@ Eighteen other open items: | [`sensitive-resource-diagnostics-follow-up.md`](design/sensitive-resource-diagnostics-follow-up.md) | deferred diagnostics | | [`e2e-git-server-choice.md`](design/e2e-git-server-choice.md) | stay on Gitea or move to Forgejo — the `_csrf` pin is fixable in place on both, so the migration is now a preference call, not a fix; also why we adopt no SDK either way | | [`azure-devops-multi-ack.md`](design/azure-devops-multi-ack.md) | **decided and built: go-git v6** — why Azure DevOps rejects our fetches, and what to do instead of PR [#292](https://github.com/ConfigButler/gitops-reverser/pull/292)'s bundled `git` binary. The capability filter fails in two independent halves: advertising `multi_ack` is a four-line change, but v5 then cannot parse the multi-ACK **response**, which only a fetch with `have` lines provokes. That is why **Flux ships ADO support on v5 with no git binary — it never fetches**, only `CloneContext`, so it never enters the path v5 cannot serve; our persistent-clone-plus-incremental-fetch design is the opposite, which makes the trim alone insufficient for us. **go-git v6 already implements `multi_ack`** (PR #1204, in every v6 tag; upstream then deleted their ADO example saying it "works out of the box"), and its churn in the packages we import runs 96 → 39 → **1** → **9** removals per alpha, so it is one settled breaking wave rather than a moving target; the migration is four known API removals over two rewritten files, `transport.AuthMethod` being the invasive one. Prices PR #292 as measured rather than argued: the image goes **217 MB → 940 MB**, of which 723 MB is a `cp -rL` that dereferences 165 hardlinks to one binary (a one-character fix), arm64 is unaffected and native, but **Trivy reports zero findings on both images** while the new one carries git 2.54.0, OpenSSH 10.3p1 and OpenSSL 3.5.7 as loose files no package database describes — so the CRITICAL gate is blind to a third of the runtime. Also catches an unflagged non-ADO regression (`Depth: 1` dropped, so every provider full-fetches) and 10% patch coverage on an untestable path. The unlock is that **canonical `git upload-pack` advertises `multi_ack`** (verified), so the Gitea already in the e2e lab plus a 400-injecting proxy is a faithful ADO simulator — no tenant needed, and the only way any option becomes CI-testable. Four options priced, and Option A (v6) is the one shipped. Carries a measured **capability matrix** over our three network calls with two diagrams, which narrows the blast radius to **one call, `repo.Fetch`**: `receive-pack` never advertises `multi_ack` (measured), so **the atomic push is out of scope for every option** — its safety rests on the same-session advertisement plus the server-side `Old`/`New` compare-and-swap in `packp.Command`, neither of which touches `upload-pack`, and we already push from a shallow store today. v6 keeps that pattern 1:1 (`Handshake` → `GetRemoteRefs`/`Push`, same `[]*packp.Command`), which is an argument *for* migrating. Records what the migration actually cost, including the four v6 behaviour changes it surfaced — two of them settings v6 reads from the environment and fails closed on, invisible to unit tests | +| [`crd-handling.md`](design/crd-handling.md) | **design, unbuilt.** Where a mirrored folder's custom-resource **definitions** come from, given that a `Widget` cannot exist in a cluster that has never heard of its CRD. Two consumers want different things: applicability wants the definition in the repository, while the per-branch **editing cluster** (a small workload-less cluster that hydrates a branch's documents as real API objects) only needs it *installable at spin-up*, which a reference serves better than a copy, because a copy freezes whatever happened to be installed in the mirrored cluster. The selection is nearly free — `typeset` already classifies each served type's origin as `builtin`/`crd`/`aggregated` with the CRD as evidence — and it is **derived**, so [#146](https://github.com/ConfigButler/gitops-reverser/issues/146)'s per-rule `objectSelector` cannot express it. Four options priced; recommends **default `None`** (the folder holds instances, the installer owns definitions) with opt-in **`Referenced`** (a manifest of names, versions, sources and digests, never schemas) and `Vendored` as an escape hatch. Two verified facts carry the argument: Helm's `meta.helm.sh/*` ownership metadata is **not** stripped by `sanitize`, so a vendored CRD arrives in Git still claiming another cluster's release and collides with Helm's ownership check when applied elsewhere; Flux's `kustomize.toolkit.fluxcd.io/*` **is** stripped, so provenance must be captured live and can never be read back out of the repository. Carries the finding that matters most for the vision, and its answer: a workload-less cluster runs no webhook, so installing the CRD is necessary and not always sufficient and the gap is invisible — but structural-schema defaults need nothing running, conversion only fires when the served version differs from the stored one, and `WebhookClientConfig` takes a **`url`** as well as a `service`, which is upstream's form for a webhook that does not run in the cluster. So what has to exist is an *endpoint*, not a controller, and the ladder is: narrow the installed versions, else point at the application's own endpoint where that code already lives, and only then have ConfigButler **host** composable transformations — which needs the transformation pinned per branch and any divergence from the app's real webhook marked non-authoritative, or we recreate the invisible gap ourselves. Worked both ways in [`layout/examples/crd-closure/`](layout/examples/crd-closure/README.md) | +| [`source-scope-simplification.md`](design/source-scope-simplification.md) | **proposal, unbuilt.** What to delete from the source-namespace policy surface, and what to keep. Answers a design review's proposal to adopt Flux-style service-account impersonation by **not** building it: it buys several identities behind ONE credential against ONE source cluster, which nobody has asked for. Deletes `GitTarget.spec.allowedSourceNamespaces` on an API reading rather than a security argument: `WatchRule.spec.targetRef` and `GitTarget.spec.providerRef` are both LOCAL and `spec.path` is immutable, so the chain from a Git folder back to the object that fills it never leaves one namespace and the destination question is **already answered by ordinary RBAC on `watchrules` there**. The structural finding is that **the complexity is in the matcher, not the fence**: `NamespaceMatcher`'s selector half reads `Namespace` labels in ANOTHER cluster, and that one choice produces the three-valued verdict, the `SourceScopeUnavailable` path, the five condition reasons, and the cross-cluster `Namespace` get/list/watch. Measured at 4,569 lines in files that exist for nothing else, with coupling into shared files at 1 to 26 mentions. **Keeps `allowedNamespaces`** (renamed `accessFrom`), reversing an earlier draft: the permissive chart value covers the chart-owned `default` provider ONLY, so every user-authored ClusterProvider is still deny-by-default, and the boundary it draws exists nowhere else — source RBAC bounds what a credential may READ, never which control-plane tenant may WIELD it. A `ValidatingAdmissionPolicy` is not a replacement, and [`spec/where-validation-lives.md`](spec/where-validation-lives.md) already says why: admission is one-shot and cannot revoke a stored object whose namespace state changed. Flags **`sourceNamespace: "*"` as needing its own decision**, because it is defined in terms of the deleted field ("every namespace this GitTarget admits", never "every namespace that exists") and RBAC cannot supply the missing definition: it answers "may I watch X in namespace Y", never "which namespaces may I watch". Three options priced; **decided: redefine as one cluster-wide list and watch** at `metav1.NamespaceAll`, rejected while `allowAnySourceNamespace` is false, which is also the proposal's largest efficiency win (one stream per type instead of one per namespace) and shrinks the `TooManyStreams` rider in [`layout/api-wave.md`](layout/api-wave.md). Names two breaking semantic changes rather than claiming continuity: a declared policy can today deny a rule's OWN namespace, so `allowAnySourceNamespace: false` is not the current posture exactly, and source-side label selectors are lost. The one thing to BUILD is a `SelfSubjectAccessReview` pass reporting reachable scope and write verbs, phrased as "no write permission observed for the requested resources at review time" because a review cannot prove absence. Archaeology split out to [`facts/kubernetes-impersonation-and-flux-identity.md`](facts/kubernetes-impersonation-and-flux-identity.md) | + +## The layout topic — [`layout/`](layout/README.md) + +Where a live object's document goes in Git, and what else has to change so that file is reachable. +Collected by topic rather than by lifecycle, so the folder mixes binding contracts with an unbuilt +proposal; its [README](layout/README.md) labels each one. Two of these are `spec/`-class and cited +by path from Go source. + +| Document | Class | What it holds | +|---|---|---| +| [`contextual-namespace.md`](layout/contextual-namespace.md) | **spec** | kustomize namespace inference; the supported subset | +| [`new-file-placement-rules.md`](layout/new-file-placement-rules.md) | **spec** | where a new resource's file goes: declared, the folder's one kustomize root, canonical. Sibling inference is removed, and kept as history | +| [`placement-visibility-and-declared-defaults.md`](layout/placement-visibility-and-declared-defaults.md) | **design** | the three questions the inference deletion left, **decided and then not built** — PR #291 shipped the deletion, the counters and the namespace-transformer fix, and none of the eight items this page had queued behind them, which the page now says. The residue was filed as [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) (correctness: a declared path into a kustomize subdirectory is never rendered, and the identity gate rejects the versionless canonical path), **which shipped in 0.42.1 via [#319](https://github.com/ConfigButler/gitops-reverser/pull/319) and is what reversed the layout model**, and [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) (visibility: `status.layout`, the ambiguous render root, the `declared` metric split, `{kindLower}`). Its Question 2 is superseded outright by the layout model. What still stands: **Keep `canonical`** as the name for the built-in path and split `declared` into `byType`/`default`, because reusing "default" for both a declaration and the absence of one makes the metric unreadable. **No CRD default for `placement.default`**, and the reason is structural rather than the two that look obvious: a defaulted default is never empty, so it shadows the kustomize-root step and every new file in an overlay would take the canonical path, in Git and rendered by nothing. The validation failure is real but the rule doing the rejecting is itself wrong, and the persistence objection is a trade we could take; defaulting the Secret route to work around the first is a floor that vanishes when a user writes any `byType` entry, because map defaults never merge. **`status.layout` instead**, with five worked examples (greenfield, kustomize overlay, brownfield missing one rule, two ambiguous roots, a refusal from an operator-configured sensitive type) over the `MarkTargetRetention` seam, which already enqueues on change and so retires the "the data plane cannot notify the GitTarget" objection. **`{kindLower}`, not a `toLower` function.** Carries three findings that changed a decision: `IdentityCompletePlacementTemplate` demanding `{version}` contradicts the versionless-path decision; two supported kustomizations still produce a file nothing renders and nothing counts; and **a declared path into a subdirectory of a kustomize folder is registered only when render-root scoping happens to be in force**, so one `byType` line reproduces the unrendered-file bug today. Fixing that last one (walk up to the nearest kustomization) also weakens the case against the CRD default from a correctness wall to a legibility trade, which the page says rather than leaving the stronger argument standing | +| [`model.md`](layout/model.md) | **design** | **reversed 2026-08-28, and much smaller than it was.** The earlier thesis said a path template is the wrong primitive and proposed replacing `spec.placement` with a `spec.layout` discriminated union (a `kind` of `Auto`, `Kustomize`, `Tree`, `Flat` or `Template`, plus `scope` and a namespace claim). That rested on five arguments and **three of them were retired by [#319](https://github.com/ConfigButler/gitops-reverser/pull/319)**, which made registration an invariant: a new file joins the nearest kustomization that governs it, whatever chose the path. Once the template is no longer asked to express *beside this folder's one kustomization*, the discriminator has nothing left to discriminate. So the template **stays**, and the proposal is two additive fields with defaults equal to today's behavior: **`serializeNamespace`** (`Auto`, `Always`, `Never`) — the one thing a path cannot express, since kustomize needs `metadata.namespace` from either the document or a governing root; named to avoid `writeNamespace`, which in an API carrying `mode: Write`, a write boundary and a write jail reads as a permission — and **`kustomizeRoot`** (`Adopt`, `Create`, `Require`), one axis: what to do when no kustomization governs the path, because when one does, all values register. Carries four **measured** kustomize facts that contradict the earlier model: a root does not require a flat folder, nested roots work one per subfolder and supply their own namespace, there is no ambient pickup (globs and bare directories both fail), and an unlisted file in a listed subdirectory renders nothing. Records two values considered and not taken: `Ignore` **rejected** because `spec.path` already expresses it and it is the only candidate that breaks the axis, and `CreatePerDirectory` **deferred** with its trigger written down, since it is what would make `Never` safe in a multi-namespace tree. Takes the review's status findings: `renderRootReason` becomes a condition reason, the accumulating counters go to metrics, `conditions` and `observedGeneration` are shown. **The headline is what it deletes**: `spec.layout`, `kind`, `scope`, `kustomize.create`, the `LayoutProfile` question, the immutability machinery, and the migration — so the largest breaking change in the queue stops being a breaking change at all | +| [`api-wave.md`](layout/api-wave.md) | **design** | **postponed, filed as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). Not in 0.41.0**, which already carries the attribution model and the placement break. One breaking wave on GitTarget, sequencing the layout model with the maintainer review's still-open API block (F6, F10, F12's reference nit, §3's pushbacks) and the queue's Tier 2 items (B4, B1, #5, #6). The batching argument is the weaker half; the stronger one is that four of them are the same decision seen from different angles: **the folder is described on the GitTarget and the connection describes only the connection**, which is why `commitWindow` and `commit.message` move off `GitProvider`. Adoption is a dry run over `status.placement` instead of declare-and-hope, and **`spec.mode: Observe\|Write` and `GitTarget.spec.interval` have both since been dropped**: a suspended target that keeps scanning is the same dry run with one field, and every input that changes what a scan would conclude already arrives on a watch, so the poll belongs on `GitProvider`, which polls a remote that cannot be watched. Also carries the breaking half of the source-scope simplification, the wave's only member that makes the API smaller. `spec.suspend` is a precondition rather than a rider, because a layout that creates a `kustomization.yaml` needs a stop button. Records that F7 already shipped the EventRecorder the placement Event was said to be too expensive for, that layout is mutable like `prune`, that F9 stays OUTSIDE the wave because its answer constrains the enum work, and that the version stays `v1alpha3` with loud rejections rather than paying for a conversion path | + +[`examples/`](layout/examples/README.md) makes the proposed `GitTarget` +layout model tangible. Each scenario contains a repository folder and the matching proposed +`GitTarget` and rule configuration; the examples are design material, not current install manifests. + +[`implementation-plan.md`](layout/implementation-plan.md) +is the order that work is built in: six PRs of which **only the last two are breaking, and neither +is about placement**, the corpus harness that turns the examples into the definition of done, and +the design changes the examples produced — three of which dissolved when the model reversed, because +the fields they amended stopped existing. ## Deferred, but still wanted — [`future/`](future/) @@ -133,7 +163,7 @@ is the strategy review on top of it: the config-as-data direction as the headlin brownfield mirroring as the on-ramp, a decided Helm standpoint (declaration editing plus the values projection; helm-light inversion parked behind entry criteria), and worked examples of where the configuration surface should go. -Five more ideas sit beside them. +Four more ideas sit beside them. ## History — [`finished/`](finished/) diff --git a/docs/TODO.md b/docs/TODO.md index 62cd455e..48d3892b 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -64,11 +64,14 @@ This file is meant to track the smaller current backlog, not historical notes. Secrets and CozyStack `tenantsecrets`; resources with sensitive fields under shapes such as `spec.credentials` need an explicit field policy or full-file encryption decision. -- [ ] Revisit output layout. **Now designed and postponed, not open-ended.** The answer is to declare - what the folder *is* rather than a path template: `GitTarget.spec.layout`, designed in - [gittarget-layout-model.md](design/gittarget-layout-model.md) and filed as - [#293](https://github.com/ConfigButler/gitops-reverser/issues/293), sequenced with the rest of the - breaking `GitTarget` work as [#294](https://github.com/ConfigButler/gitops-reverser/issues/294). +- [ ] Revisit output layout. **Now designed and postponed, not open-ended, and the answer has + reversed since [#293](https://github.com/ConfigButler/gitops-reverser/issues/293) was filed.** The + path template **stays**; what it could not express becomes two additive fields, + `serializeNamespace` and `kustomizeRoot`. [layout/model.md](layout/model.md) carries the reversal + and [layout/implementation-plan.md](layout/implementation-plan.md) the order. The placement work + is no longer breaking, so it no longer needs + [#294](https://github.com/ConfigButler/gitops-reverser/issues/294); the issues still describe the + discriminated union and want updating. Deliberately **not** in 0.41.0, which already carries the new attribution model and the sibling-inference removal. Multiple resources per file is bundle support, which exists for match-first today and is a separate question from where a *new* file goes. diff --git a/docs/architecture.md b/docs/architecture.md index 77f225be..695d665d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1139,7 +1139,7 @@ Placement runs **only for a resource with no existing document** in the target. manifest identity rather than by path, instead of recomputing placement. So a change to how new files are placed never moves a file already in Git. A new resource is placed by the first of these that applies ([internal/manifestanalyzer/placement.go](../internal/manifestanalyzer/placement.go), -[design](spec/gittarget-new-file-placement-rules.md)): +[design](layout/new-file-placement-rules.md)): 1. **Declared policy (`spec.placement`).** A `GitTarget` can declare a `byType` map (exact `[group/]version/resource` → path template) plus a `default` template, rendered from a small diff --git a/docs/design/crd-handling.md b/docs/design/crd-handling.md new file mode 100644 index 00000000..2c4d0aff --- /dev/null +++ b/docs/design/crd-handling.md @@ -0,0 +1,221 @@ +# CRD handling: what a folder needs to be applied, and where the definitions come from + +> Status: design, unbuilt. Nothing here binds until it is scheduled. +> Date: 2026-08-29. Index: [`../INDEX.md`](../INDEX.md). +> +> Worked example: [`../layout/examples/crd-closure/`](../layout/examples/crd-closure/README.md), +> which shows the same live objects under both candidate shapes so the choice can be read rather +> than argued. + +A mirrored folder of custom resources is not applicable on its own. A `Widget` cannot exist in a +cluster that has never heard of `widgets.apps.example.com`, so somewhere between the folder and the +cluster the definition has to arrive. This page decides where from, and deliberately does **not** +decide it by mirroring every CRD in the source cluster, which is the only thing a `WatchRule` can +express today. + +## Two consumers, and they want different things + +**1. Applicability.** Can this folder be applied to a fresh cluster? Answering yes by inspection +means the definitions are in the repository. + +**2. The per-branch editing cluster.** The direction this project is heading is that a branch under +edit gets its own small, workload-less Kubernetes cluster, and the branch's documents are hydrated +into it as real API objects so they can be edited through the API rather than as text. That cluster +has to be able to **hold** the objects, which means every custom type in the folder must be +installed before hydration. Here the definition does not have to be in the repository at all. It has +to be *installable at spin-up*. + +The second consumer is the demanding one, and it is better served by a reference than by a copy: a +reference can be resolved to the version the type's owner publishes, while a copy freezes whatever +happened to be installed in the cluster we were mirroring. + +## What we already know, at no cost + +`typeset` classifies every served type's origin as `builtin`, `crd` or `aggregated`, with the CRD +itself as evidence and a confidence that says whether an object was observed +([`model.go`](../../internal/typeset/model.go), [`type-followability.md`](../spec/type-followability.md)). + +So "which CRDs back the types this GitTarget writes" is a lookup over a join the operator already +performs. Whatever this page decides, the **selection** is close to free. The cost is in what we do +with the answer. + +This also settles one thing early: the selection cannot be expressed by a per-rule `objectSelector` +([#146](https://github.com/ConfigButler/gitops-reverser/issues/146)). The relevant set is *derived* +from the rules and changes when they do; it is not a label match anyone can write down. + +## Four options + +| | What lands in Git | Who owns the CRD | Where the branch cluster gets it | +|---|---|---|---| +| **A. Vendor** | the CRD objects | now two: the installer and this folder | the folder | +| **B. Reference** | a small manifest of names, versions, sources and digests | its installer, unchanged | resolving the manifest | +| **C. Nothing** | nothing | its installer | copied from the source cluster at spin-up | +| **D. Synthesize** | a generated minimal schema | nobody real | the generated schema | + +**A costs more than it looks.** A CRD installed by Helm carries `meta.helm.sh/release-name`, +`meta.helm.sh/release-namespace` and `app.kubernetes.io/managed-by: Helm`, and **none of those are +stripped on the way to Git**: [`sanitize`](../../internal/sanitize/types.go) removes Flux, kro and +applyset bookkeeping plus two exact Argo keys, and Helm's ownership metadata is not in either list. +So a vendored CRD arrives in the repository still claiming to belong to a release in another +cluster, and applying it somewhere else walks into Helm's own ownership check. Add the size (our +smallest CRD is 25 KB; Argo CD's `Application` CRD is about a megabyte) and the churn (every +operator upgrade rewrites the file), and a configuration repository becomes mostly vendored schema +diffs. + +**B is the one that matches who owns what.** For application configuration in particular, the app +team owns what its CRD looks like, and the CRD belongs in the app's own repository next to the +controller that serves it. A configuration folder that vendors it forks it. + +**C is right for the case it covers and silent about the rest.** Copying from the source cluster at +spin-up needs nothing committed and always matches production exactly. It fails precisely when the +editing cluster is most useful: a branch from a contributor without source-cluster access, an +environment that no longer exists, an edit made while the cluster is unreachable. + +**D is rejected.** A synthesized schema accepts objects the real API server would reject, and the +entire value of hydrating a branch into a real cluster is that it validates like the real one. + +## Recommendation + +- **Default `None`.** The folder holds instances. The installer owns the definitions. This is the + posture that does not create a second owner, and it is what a mirror of an existing cluster wants. +- **Opt-in `Referenced`**: commit a small manifest of references and digests, never schemas. +- **The branch cluster resolves in order**: the reference manifest first, the source cluster second + (which is option C kept as a fallback rather than a design), and otherwise **fail loudly, naming + the types it could not install**. Hydrating a branch into a cluster that silently lacks a type + produces an editing session that looks fine and drops objects. +- **`Vendored` stays available** for a type whose definition has no installable source. It is an + escape hatch with the ownership cost stated at the point of use, not a default. + +## Where a reference comes from, and the trap in reading it + +Provenance has to be read from the **live object**, in the operator, and can never be recovered from +the repository. `sanitize` strips `kustomize.toolkit.fluxcd.io/*` from labels and annotations, which +is exactly the evidence that would say a CRD came from a Flux `Kustomization`. Capture it at +observation time or lose it. + +The ladder, in the order it should be tried: + +1. **Helm**: `meta.helm.sh/release-name` and `release-namespace`, with + `app.kubernetes.io/managed-by: Helm`. Naming the chart and version means reading the release + object, which is a Secret in the source cluster. That is a new read, and an authorization + question, not a free one. +2. **Flux**: `kustomize.toolkit.fluxcd.io/name` and `namespace` on the live CRD, resolved through + the `Kustomization` to its source. Live-only, per the trap above. +3. **Argo CD**: no tracking annotation is stamped on CRDs, since its repo-server stamps every + *non-CRD* object it applies. Provenance for an Argo-installed CRD comes from the `Application` + that lists it, or not at all. +4. **Unattributed**: record the CRD name, its served versions and a digest, and say plainly that the + source is unknown. A reference that admits it cannot be resolved is more useful than a guess. + +## The finding that matters most for the branch cluster + +**A workload-less cluster runs no webhook of its own.** A CRD with +`spec.conversion.strategy: Webhook` needs something to convert between stored versions, and a type +whose real cluster mutates objects through a mutating admission webhook will hold *unmutated* +objects in the branch cluster. Installing the CRD is necessary and not always sufficient, and the +gap is invisible: the object is accepted, it is simply not what production would have stored. + +Three things narrow that, in increasing order of ambition. Take them in this order. + +**1. Most defaulting is not a webhook at all.** Structural-schema defaults are applied by the API +server itself +([`pkg/apiserver/schema/defaulting`](https://github.com/kubernetes/apiextensions-apiserver)), so a +CRD's `default:` values work in a cluster with nothing running. Only *mutating admission* is +missing, which is a much smaller set than "defaulting". + +**2. Conversion only fires when versions differ.** If the branch cluster installs the CRD with the +versions the folder actually uses, and serves what it stores, no conversion runs. Narrowing the +installed version list is a subset of the real CRD rather than a synthesized schema, so it does not +fall into option D, and it is the cheapest honest answer for the common case of a folder written at +one version. + +**3. The webhook can live outside the cluster, and this is the interesting one.** +`WebhookClientConfig` takes **either** `service` **or** `url` +([apiextensions v1 types](https://github.com/kubernetes/apiextensions-apiserver), verified at +v0.36.4), and the upstream field documentation is explicit that `url` is for a webhook that is *not* +running in the cluster: the API server cannot resolve in-cluster DNS, so `service` is the in-cluster +form and `url` is the external one. The same shape exists on admission webhook configurations. + +So a branch cluster can install the CRD pointed at a conversion endpoint that runs somewhere else, +and the type converts correctly with nothing deployed. The constraints, all from the field's own +contract: + +- **`https` only.** No query string, no fragment, no `user:password@`. A path *is* allowed and may + carry an arbitrary string, which upstream itself suggests using as a cluster identifier. +- **`caBundle` is optional**: omitted, the API server uses its system trust roots, so a publicly + trusted certificate needs no bundle at all. A private CA needs one, and needs it rotated. +- **The API server sends no bearer token.** Client credentials for webhooks come from API + server-level configuration, not from anything a per-CRD `url` can carry. An external endpoint + therefore authenticates its caller by mTLS or by the path identifier above, and must assume the + path is the only secret it has. +- **Reachability runs the other way from usual.** The branch cluster's control plane makes the + outbound call, so the endpoint has to be reachable from wherever we run those clusters, and every + object of that type is sent to it. That is an egress policy question and a data exposure question + before it is a design question. +- **It is in the edit path.** Latency and availability of that endpoint become latency and + availability of editing a branch. + +### Hosted transformations + +The natural extension: rather than every team exposing an endpoint, ConfigButler hosts one, with +common transformations composed by the user (clicked together, or scripted, or generated) and +addressed per branch through the URL path. + +It is a real product direction and the mechanism is already proven, since it is the same webhook +contract with us as the operator. Two things have to be true before it is safe, and they are worth +writing down now: + +- **A transformation is code, and it must be pinned.** A branch cluster whose behavior depends on + the current state of a hosted rule set is not reproducible: the same branch hydrates differently + next week. The reference manifest is the place to pin the transformation version, next to the + CRD digest it belongs to. +- **Divergence from the app's real webhook must be loud.** If our transformation and the + application's own conversion disagree, the branch cluster stores objects production would not, and + we have recreated the invisible gap this section opened with, except now we caused it. A type + served by a hosted transformation rather than by the app's own endpoint should be marked + non-authoritative in the session, and the marking should survive into whatever the branch produces. + +The honest ladder is therefore: **narrow the versions** so nothing converts; else **point at the +application's own endpoint**, which is where that code already lives and where the app team already +owns it; and only then **host a transformation**, for types whose owner has no endpoint to point at. + +## Placement consequences + +- **CRDs are cluster-scoped.** Canonically that is + `_cluster/apiextensions.k8s.io/customresourcedefinitions/{name}.yaml`; in a kustomize folder it is + a flat file beside the root, registered in `resources:` like any other placement. +- **Apply order.** A folder holding both a CRD and its custom resources has to apply the definition + first. Flux and Argo CD both handle this; `kubectl apply -k` over a plain folder is where it bites. + If we vendor, the folder has to say who guarantees the ordering. +- **Scope.** A namespaced `GitTarget` writing cluster-scoped documents widens what that tenant puts + in Git. That is an authorization question and it belongs with + [`source-scope-simplification.md`](source-scope-simplification.md), not with placement. + +## The API shape + +`GitTarget.spec.includeTypeDefinitions: None | Referenced | Vendored`, defaulting to `None`. +Additive, so it needs no coordinated bump and does not belong to the breaking wave. The derived set +is recomputed when rules change, over the isolation seam that already exists for exactly that +([`gittarget-isolation-on-rule-change.md`](../spec/gittarget-isolation-on-rule-change.md)). + +## Open questions + +- Does the reference manifest belong **in the folder** as a committed artifact, or only in + `status`? In the folder it is reviewable and travels with the branch; in status it never conflicts + and never goes stale in a PR. +- **A digest of what?** The CRD's `spec`, so a resource-version bump is not a change, or the whole + object, so any drift shows. +- One manifest **per target folder** or one **per repository**? Two targets in one repository will + reference overlapping types. +- Does the branch cluster install **only CRDs**, or the controllers too? With `url` webhooks the + question narrows: what has to run is an *endpoint*, not a controller, and it does not have to run + in the branch cluster at all. +- Does the reference manifest record a type's **conversion strategy** and, where it is `Webhook`, + the endpoint the branch cluster should point at? The manifest is the only place that knows both + the type and its origin, so it is the natural home, and it makes "this folder needs a reachable + endpoint to be editable" a reviewable fact rather than a runtime surprise. +- Who **owns the caBundle** for an app-hosted endpoint, and what happens to a branch when it + rotates? +- Should `Vendored` **strip Helm and Flux ownership metadata** on the way out? It would make the + vendored copy applicable elsewhere, at the cost of `sanitize` acquiring a rule that exists for one + option of one field. diff --git a/docs/design/gittarget-api-wave.md b/docs/design/gittarget-api-wave.md deleted file mode 100644 index b8a331df..00000000 --- a/docs/design/gittarget-api-wave.md +++ /dev/null @@ -1,274 +0,0 @@ -# One breaking wave: the folder is described on the GitTarget - -> **design**: a sequencing proposal, not a plan of record. Nothing here binds until scheduled. -> Index: [`../INDEX.md`](../INDEX.md) -> Date: 2026-07-30. -> -> Combines three pieces of work that are all `feat(api)!` on `GitTarget` and are cheaper together -> than apart: -> -> - the layout model, [`gittarget-layout-model.md`](gittarget-layout-model.md); -> - the still-open API-surface block of the maintainer review, -> [`flux-maintainer-review-status-and-config-model.md`](../future/flux-maintainer-review-status-and-config-model.md) -> §4 "Then": **F6**, **F9**, **F10**, plus F12's reference-type nit and §3's pushbacks; -> - the Tier 2 breaking items in [`open-asks-priority.md`](open-asks-priority.md): **B4**, **B1**, -> **#5**, **#6**. -> -> It does not touch Tier 1 (the removal-wait decision, #15's condition). Those are not breaking and -> should not wait for this. - -## Why one wave - -The consumer pins us three ways (image, Go module, `require` line), so each breaking release costs a -coordinated bump. That argues for batching. It is the weaker half of the argument. - -The stronger half: **four of these items are the same design decision seen from different angles**, -and building them separately means deciding it four times, inconsistently. - -> **The folder is described on the GitTarget. The connection describes only the connection.** - -- `spec.layout` says what the folder **is** (layout model). -- `spec.mode: Observe|Write` says whether we write to it at all (B1). -- `spec.suspend` says whether we write to it *now* (F6). -- `commitWindow` and `commit.message` say how writes to it are batched and phrased, and today they - live on `GitProvider`, which is the connection (B4, and §3's "GitProvider is doing three jobs"). - -Ship `spec.layout` alone and the principle is asserted by one field while `commitWindow` still -contradicts it. Ship them together and the object reads as one idea: a GitTarget is a folder plus a -policy for writing it, and a GitProvider is how you reach the repository. - -## The interactions that change the design - -These are the reasons to combine, as opposed to merely batch. Each one changes what gets built. - -### 1. `spec.mode: Observe` is how a layout is adopted safely - -The layout model's weakest point is adoption: a user pointing `kind: Kustomize` at a real repository -has to trust it before any file moves, and placement only ever affects *new* documents, so there is -nothing to preview by inspection. - -`Observe` mode plus `status.layout` is that preview. In `Observe` the operator scans, resolves the -layout, publishes `renderRoot`, `kind`, and what it *would* do, and writes nothing. Flip to `Write` -when the status says what you expected. That turns "declare a layout and hope" into a dry run, and -it costs nothing extra because both halves are already in the wave. - -This also gives `Observe` a purpose beyond "a safety switch nobody uses". It is the mode you adopt a -repository in. - -### 2. `spec.interval` is what keeps the layout observation fresh - -The layout status has two halves ([the other document](placement-visibility-and-declared-defaults.md) -records why): a **current** half derived from the last repository scan, and a **historical** half -accumulated since. The current half is the useful one, and it has a hole: a repository scan happens -on a write or a resync, so a stable target that writes nothing may not scan for a long time, and the -field a user consults would be stamped with a revision from last week. - -`spec.interval` (F6) plus `Observe`'s scan-without-writing is the mechanism that closes it: a -periodic observation pass refreshes `renderRoot` and `observedRevision` whether or not anything was -written. Neither piece was proposed for this reason, and together they answer a question neither -answers alone. - -### 3. `spec.suspend` is a precondition for a layout that creates files - -`kind: Kustomize` with `create: true` writes a `kustomization.yaml` the user did not author. That is -the right behavior and it raises the stakes on the review's central complaint (F6): *this controller -writes to a Git repository and there is no way to make it stop that is not deleting the object.* - -So `suspend` is not a rider here, it is a precondition. A layout that creates structure must ship -with the button that stops it. And `suspend` must stop bootstrap creation specifically, not only -resource writes, which is a detail worth stating before either is built. - -### 4. The Events question is already answered, and the layout is what to say - -[`open-asks-priority.md`](open-asks-priority.md) left one thing open about the inference deletion: -whether a fall-back to canonical should raise an Event on the GitTarget, and it reasoned that this -was expensive because placement runs on the branch worker with no recorder. - -That is no longer true. **F7 shipped an `EventRecorder` on every reconciler** -(review §6), and the roll-up seam projects data-plane facts into status with an enqueue on change. So -the Event is now: emit when `status.layout` changes in a way a human should know about, which is -`renderRootReason` becoming `Ambiguous`, or a type falling back for the first time. One Event per -persisted change, the pattern F7 already established for `Ready`. - -### 5. Layout is immutable, which puts it with `path` rather than with `prune` - -`providerRef`, `branch`, `path` and `clusterProviderRef` are immutable because a folder's meaning is -constituted by them (review §3 defends this well). `spec.prune` is deliberately mutable because -freezing it would destroy the one thing that cannot be rebuilt. - -An earlier draft of this document put `spec.layout` with `prune`. That was wrong, and the reason is a -fact worth checking before designing around it: **`GitTarget` has no finalizer**, so deleting one -leaves the folder in Git untouched, and re-creating it at the same path re-adopts every document by -identity. Changing a layout by recreating the object costs status and a moment of mirroring, not data, -which is a different bargain from `prune` entirely. Meanwhile a mutable layout leaves a folder -permanently half one structure and half another, because existing files never move and nothing records -which file came from which. - -So: immutable, with a CEL exception for a **widening** transition (`Flat` to `Tree`) that cannot lose -the identity-completeness the folder already had. And `Auto` resolves once and is pinned in status, -because immutability of a field that says "look at the folder" pins nothing. The reasoning is in -[`gittarget-layout-model.md`](gittarget-layout-model.md). - -This settles a question **#6** would otherwise have to reopen: if `path` ever becomes movable -(`status.observedDestination`), the layout moves with it, because a new folder may have a different -structure, and an immutable layout means that transition is one deliberate act rather than two -independent edits. - -### 6. Namespace scope makes `allowedSourceNamespaces` and the layout answer to each other - -The layout's `scope: SingleNamespace` is a structural claim, and `spec.allowedSourceNamespaces` is an -authorization bound. They are different questions about the same folder, and after this wave they are -checked against each other at admission: a matcher admitting more than one namespace beside a -single-namespace layout is refused. - -This is the connection the wave makes available, and it is not one either item asks for alone. -Authorization already exists and already has a two-party delegation the review praises; the layout is -what turns "who may write here" into "and therefore what this folder looks like", including whether -`metadata.namespace` is written into the files at all. The namespace-in-file question is inference -today, and it is the one piece of inference an empty folder cannot perform, which is why bootstrapping -needs it declared. - -## The GitTarget after the wave - -```yaml -apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget -metadata: - name: prod - annotations: - reconcile.configbutler.ai/requestedAt: "2026-07-30T09:14:22Z" # F6 -spec: - # --- the connection: unchanged, and now only the connection --- - providerRef: - name: platform - branch: main - path: clusters/prod - - # --- what the folder is --- - layout: # immutable, except a widening transition - kind: Kustomize # Auto | Kustomize | Tree | Flat | Template - scope: SingleNamespace # must agree with allowedSourceNamespaces - writeNamespace: Never # the created kustomization carries namespace: - kustomize: - create: true - byType: - v1/secrets: "secrets/{name}{sensitiveSuffix}" - - # --- whether and when we write it --- - mode: Write # B1: Observe | Write - suspend: false # F6 - interval: 5m # F6, and what keeps status.layout fresh - prune: - mode: OnEvent - - # --- how writes are batched and phrased: moved off the connection --- - commitWindow: 5s # B4, was GitProvider.spec.push.commitWindow - commit: - message: - template: "chore(mirror): {{ .Summary }}" # B4, was GitProvider.spec.commit.message -status: - layout: - declaredKind: Kustomize - kind: Kustomize - renderRoot: . - renderRootReason: SingleKustomization - observedRevision: 9f3c1ab - observedTime: "2026-07-30T09:14:22Z" - placedResources: 14 - refusedResources: 0 - lastHandledReconcileAt: "2026-07-30T09:14:22Z" # F6 - observedDestination: # 6 - branch: main - path: clusters/prod -``` - -Read top to bottom it is one story: reach this repository, this is what the folder is, this is whether -and when we write it, this is how the writes look. - -## What rides along without a claim of synergy - -Honesty matters more than a tidy narrative. These are in the wave because they are breaking and the -consumer should pay once, not because they interact with the layout: - -- **#5, `CommitRequest.spec.author`, SAR-guarded.** Independent, and it stands on the argument that - attribution needs an audit webhook a hosted control plane will not give you. -- **F10, CommitRequest lifecycle** (`ttlSecondsAfterFinished` or an `ownerReference`, plus the - `delete` verb). Unrelated to placement; it is the other object in the API with a lifecycle hole. -- **F12's reference types**: embedding `meta.LocalObjectReference` for the name half of our six - near-identical reference shapes. If GitTarget is breaking anyway, this is the moment. -- **§3's `TooManyStreams` cap** for `sourceNamespace: "*"` fan-out. A `Stalled` reason plus a bound, - rather than discovering the cliff as apiserver watch pressure. -- **§3's `ClusterProvider` "default" message.** One error string, and the most likely first-run - support ticket. - -**F9** is not in the wave at all: it is one envtest against the minimum supported Kubernetes version, -and its outcome (widen the enum, or keep it) should be known *before* anyone plans an API change -around it. - -## Version strategy: stay `v1alpha3` - -A wave this size invites `v1alpha4`, and I would not take it. - -- A new version means a **conversion path**, and the honest options are a conversion webhook (a - serving dependency for the CRD, plus a cert lifecycle) or `None` conversion with a stored-version - migration. Both cost more than the problem. -- We are `v1alpha3` and pre-1.0 with **one consumer**. The convention the repo already uses is a - **loud rejection**: keep the removed field in the schema, refuse it with a message naming the - replacement, for one release. `ClusterWatchRule.spec.rules[].scope` set that precedent, and the - reasoning holds better here: refusing a stored field the user can see beats translating it behind - their back. -- `spec.placement` therefore becomes a rejection that says "use `spec.layout`; `byType` moves - verbatim, `default` becomes `layout.kind: Template`". The mapping is mechanical, which is what makes - the rejection kind rather than merely strict. - -If a second consumer appears before this ships, revisit: the calculus that makes loud rejection cheap -is one coordinated bump. - -## Order inside the wave - -Dependencies first, then the things that only need the object to be breaking. - -1. **F9's envtest.** Not an API change; its answer constrains the enum work. Do it before planning. -2. **`spec.suspend`** (F6). Precondition for anything that creates files, and independently the - review's highest-value gap. -3. **`spec.layout`** with `byType`, plus rule 1 (every written file is registered with the - kustomization that governs it) and rule 2 (a structural kind excludes a blanket `default`). - `spec.placement` becomes a loud rejection. -4. **`status.layout`**, current half from the scan, historical half from the roll-up. Needed before - `Observe` is useful, because `Observe` with nothing to read is a mode that does nothing. -5. **`spec.mode: Observe|Write`** (B1). Now an adoption path rather than a switch. -6. **`spec.interval` + `requestedAt` + `lastHandledReconcileAt`** (F6, rest). Closes the freshness - hole in step 4 and gives the object the reflexes a Flux user already has. -7. **Events on layout change**, over F7's existing recorder. -8. **B4**: `commitWindow` and `commit.message` move from `GitProvider` to `GitTarget`. Last of the - principle items, and the one that makes the object coherent. -9. **The riders**: #5, F10, F12 references, `TooManyStreams`, the `default` ClusterProvider message. - -Steps 2 to 8 are one release. Step 1 gates the planning. Step 9 can be trimmed if the wave gets too -big to review, since nothing else depends on it. - -## What this costs, stated plainly - -- **One coordinated consumer bump**, with a mechanical migration for every field: `placement.byType` - moves verbatim, `placement.default` becomes `kind: Template`, `commitWindow` and `commit.message` - move object, everything else is additive. -- **One `docs/UPGRADING.md` entry** covering the field moves, the layout mapping, and the one real - behavior change: a declared template stops silently disabling the render root and starts - registering its files. -- **A larger review surface than any change this project has taken.** That is the argument for - trimming step 9 first and for keeping step 1 outside the wave. - -## Open questions - -- **Does `mode: Observe` write status only, or also refuse admission of new WatchRules?** Observe - should be silent about everything except what it observed, but a user in Observe mode with rules - piling up may expect to be told nothing will happen. -- **Should `suspend` and `mode: Observe` be one field?** They are close: both stop writes. They differ - in intent (temporary versus declared) and in what they do to status, and Flux keeps `suspend` - separate from everything else. Two fields, but the field docs must each say what the other is for. -- **Where does `interval` live?** `GitProvider` needs it for `ls-remote` cadence (review F6); - `GitTarget` needs it for the observation pass. Two fields with one name on two objects is a smell, - and one field on the provider cannot express a per-folder observation cadence. -- **Is `Auto` still the right layout default once `Observe` exists?** With a dry-run mode available, - requiring an explicit `kind` costs the user much less than it would have, and it would make every - target's layout self-evident. diff --git a/docs/design/gittarget-layout-model.md b/docs/design/gittarget-layout-model.md deleted file mode 100644 index 7b475eed..00000000 --- a/docs/design/gittarget-layout-model.md +++ /dev/null @@ -1,525 +0,0 @@ -# A layout is the declared thing, not a path - -> **design**: a proposal, not a plan of record. Nothing here binds until scheduled. -> Index: [`../INDEX.md`](../INDEX.md) -> Date: 2026-07-30. Supersedes Question 2 of -> [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md), -> which argued about whether to default a path template. The answer is that a path template is the -> wrong primitive to be defaulting. -> -> This is a `feat(api)!` change to `GitTarget`. It does not fit in PR #291 and is not proposed for it. -> How it sequences with the other breaking work on the same object (`spec.suspend`, `spec.mode`, the -> `commitWindow` move, CommitRequest lifecycle) is in -> [`gittarget-api-wave.md`](gittarget-api-wave.md), which also records the two places where those -> items change this design rather than merely accompanying it. - -## Why the current shape keeps producing dead ends - -Placement today is a ladder of four rungs, three of which are path templates and one of which is not: - -```text -byType -> default -> the folder's one kustomize root -> canonical -``` - -Every unresolved question in the review of #291 traces to that one mismatch: - -- a CRD default for `placement.default` cannot be added, because a non-empty template consumes the - slot in front of the rung that is **not** a template (F9 in the other document); -- `byType: {v1/configmaps: "configmaps/{name}.yaml"}` in a kustomize folder produces a file no - kustomization lists, so it is committed and never rendered (F10). One line of user config; -- `placement.default` set on a kustomize folder does the same thing to every type at once, which is - the same bug with a bigger blast radius; -- "where do my files go" is answered by simulating a four-rung ladder against a folder, which is why - it needed a metric and a status field to be legible at all; -- nothing in the model can **create** structure, so a repository that needs a `kustomization.yaml` - before it can render cannot be bootstrapped by the thing that writes into it. - -The primitive is wrong. A path template cannot express "beside this folder's one kustomization", -cannot be read at a glance, and cannot bring a folder into existence. What a user wants to declare is -what the folder **is**. - -## The model - -One field, a discriminated union on `kind`, plus per-type overrides that are valid under every kind. - -```yaml -apiVersion: configbutler.ai/v1alpha3 -kind: GitTarget -spec: - providerRef: - name: platform - branch: main - path: clusters/prod - layout: - kind: Kustomize # Auto | Kustomize | Tree | Flat | Template - scope: SingleNamespace # SingleNamespace | MultiNamespace - writeNamespace: Never # FromContext | Always | Never - kustomize: - root: . # relative to spec.path; where the kustomization.yaml lives - create: true # write it if absent, so an empty repo becomes a buildable folder - fileName: "{kindLower}-{name}" # optional; the default - byType: # optional overrides, valid under every kind - v1/secrets: "secrets/{name}{sensitiveSuffix}" -``` - -| `kind` | Where a new document goes | Exists because | -|---|---|---| -| `Auto` (default) | one supported kustomization in the subtree: `Kustomize`; else exactly one namespace in scope: `Flat`; else `Tree`. Resolved once and pinned (see below) | it is what most folders want, and it is honest about reading the repository | -| `Kustomize` | beside the declared root, registered in its `resources:` list | a file that root cannot reach is never applied | -| `Tree` | the built-in `{namespaceOrCluster}/{groupPath}/{resource}/{name}` path | identity-complete by construction; right for a fleet folder | -| `Flat` | `{kindLower}-{name}.yaml` at the root of `spec.path` | the legible single-namespace folder people hand-author | -| `Template` | the declared `default` template | the escape hatch. Everything expressible today stays expressible | - -Two rules complete it, and they are where the value is: - -1. **Whatever chose the path, the file must be reachable.** If any kustomization governs the - destination, the write registers the file in it, in the same commit. This is an invariant of the - layout rather than a special case of one rung, which is what turns F10 from a bug into something - the model cannot express. -2. **A structural kind and a blanket template are mutually exclusive.** `default` is valid only under - `kind: Template`. You cannot ask for a kustomize folder and a nested canonical tree at the same - time. - -## What the model makes unstatable - -This is the headline, and it is worth more than the readability. - -| Today | Under the model | -|---|---| -| `placement.default` on a kustomize folder silently disables the render root and produces unrendered files | unstatable: `default` requires `kind: Template`, and `Template` asserts there is no structural rule | -| `byType` into a subdirectory produces a file no kustomization lists | unstatable: registration is an invariant, not a rung | -| Defaulting the fallback would shadow the structural rung | dissolved: the default is `kind: Auto`, which **names** the structural rule instead of standing in front of it | -| "Which of four rungs answered?" needs a metric to be legible | one word in the spec, and status says what `Auto` resolved to | -| An empty repository cannot be given the structure it needs to render | `kind: Kustomize` with `create: true` | - -The defaulting problem is the interesting one. It was never that defaults are bad; it was that we -tried to default a **path**, and a path is the one thing that cannot say "look at the folder". -`kind: Auto` is a safe CRD default precisely because it is a mode: it declares that the folder will be -read, which is the difference between this and the sibling inference we deleted. That inference was -undeclared. This one has a name, appears in the spec, and reports what it resolved to. - -## Namespace scope belongs to the layout - -A folder that omits the namespace from its paths is a folder for **one** namespace. That is what makes -`Flat` legible, and it is an assumption the layout has to carry, because the thing that would otherwise -carry it lives somewhere else. - -```yaml -spec: - allowedSourceNamespaces: # AUTHORIZATION: who may be mirrored here - names: [team-a] - layout: - kind: Flat - scope: SingleNamespace # STRUCTURE: what shape this folder has -``` - -**These are two different questions and they must agree.** `allowedSourceNamespaces` is a permission -bound owned by the destination -([`gittarget_types.go`](../../api/v1alpha3/gittarget_types.go)); `layout.scope` is a structural claim. -Validation checks the direction it can, at admission: a matcher that admits more than one namespace -alongside `scope: SingleNamespace` is refused. - -**Why the layout cannot derive it.** Two reasons, and the second is the one that matters: - -- `allowedSourceNamespaces` is an upper bound and may be **absent**, which - [`NamespaceMatcher`](../../api/v1alpha3/namespace_matcher.go) defines as "no policy declared" rather - than as "one namespace". There is often nothing to derive from. -- the namespaces that do arrive come from **N WatchRule objects that do not own the folder**. A - derived single-namespace assumption could be invalidated later by a rule created in another object, - which would turn a layout guarantee into a path collision. Declaring the scope makes that - invalidation a **refusal** instead: a document from a second namespace is declined with a message - naming both namespaces and counted as a placement refusal, rather than landing on a path another - object already occupies. - -That is the same distinction the whole redesign rests on. Reading the world is fine when the reading -is declared and its failure is loud; it is not fine when it silently re-decides. - -## Whether the namespace is written into the file - -Today this is **inferred**, and it cannot be inferred in the case bootstrapping cares about most. -`namespaceIsInheritedFromContext` omits `metadata.namespace` exactly when the governing kustomization's -`namespace:` equals the resource's own. An empty folder has no kustomization to read, so a folder we -are about to create cannot inherit a convention that does not exist yet. - -```yaml -layout: - writeNamespace: FromContext # FromContext (default, today's behavior) | Always | Never -``` - -| Value | Meaning | When it is valid | -|---|---|---| -| `FromContext` | omit when the governing kustomization sets this resource's namespace | always; today's behavior | -| `Always` | always write `metadata.namespace` | always; the only safe choice when nothing downstream supplies it | -| `Never` | never write it | only when something guarantees the namespace: a kustomization we control with `namespace:` set, or a declared downstream supplier such as a Flux `Kustomization.spec.targetNamespace` | - -`Never` needs that guard because omitting the namespace hands the object to whatever namespace the -applier happens to be pointed at, which is a different object with the same name. - -**And this is where bootstrapping closes its own loop.** `kind: Kustomize` with `create: true` and -`scope: SingleNamespace` lets the operator write `namespace: team-a` into the `kustomization.yaml` it -creates, and then legitimately omit `metadata.namespace` from every file it places. The convention is -**established** rather than guessed, which is the thing inference structurally cannot do on an empty -folder. - -## The layout is immutable, with one widening exception - -An earlier draft of the wave document put `layout` with `prune` as a mutable field. That was wrong. - -**Existing files never move**, so a mutable `kind` leaves a folder that is permanently half one layout -and half another, with nothing in the folder recording which file came from which. The structure of a -folder should be a property of the folder, not of the last edit to an object. - -**The cost of immutability is lower than it looks, and this is the fact that decides it:** `GitTarget` -has **no finalizer**, so deleting one leaves the folder in Git untouched, and re-creating it at the same -path re-adopts every document by identity (match-first). Changing a layout by recreating the object -costs the object's status and a moment of mirroring. It does not cost data. That is a materially -different bargain from `spec.prune`, where the review's argument for mutability was that a -delete-and-recreate would destroy the one thing that cannot be rebuilt. - -**The exception is widening.** `Flat` to `Tree` cannot break the folder: old flat files stay and remain -match-first, new files get identity-complete paths, and nothing collides. Narrowing (`Tree` to `Flat`) -is what can put two namespaces' objects on one path. So the CEL rule is "immutable except a transition -that cannot lose the identity-completeness the folder already had", which in practice means you may -widen and may not narrow. - -### `Auto` resolves once and is then pinned - -Immutability of the *field* does not pin the *resolution*, because `Auto` says "look at the folder". If -someone deletes the `kustomization.yaml`, `Auto` would silently become `Tree` and the folder would grow -a second layout without any object changing. That is the defect this project spent a release deleting, -re-entering through the default value. - -So `Auto` resolves on first observation, `status.layout.kind` records what it became, and a later -folder state that would resolve differently raises a condition rather than re-laying-out the folder. -Declared inference is fine. Silent re-decision is not. - -That also settles the earlier open question about whether `Auto` should be the default at all: it can -be, because pinning removes the harm, and it keeps the quickstart to four fields. - -## Examples - -### 1. `Auto` on a brownfield kustomize repo - -No layout declared, so the CRD default applies. The subtree has one supported `kustomization.yaml`. - -```yaml -spec: - path: clusters/prod - # layout: {kind: Auto} <- defaulted, so the field is visible in the object -``` - -A new ConfigMap `cache` in `team-a` lands at `clusters/prod/configmap-cache.yaml` and is added to -`resources:`. `metadata.namespace` is omitted if and only if the kustomization's `namespace:` is -`team-a`. Status reports what `Auto` became: - -```yaml -status: - layout: - declaredKind: Auto - kind: Kustomize - renderRoot: . - renderRootReason: SingleKustomization - observedRevision: 9f3c1ab -``` - -### 2. `Kustomize` with `create: true` on an empty repository - -The bootstrapping case. The folder does not exist yet, so nothing can be inferred from it. - -```yaml -spec: - path: clusters/prod - layout: - kind: Kustomize - kustomize: - create: true -``` - -The first write commits a folder that builds, rather than a file that happens to be YAML: - -```text -clusters/prod/ - kustomization.yaml # created, listing what was written - configmap-cache.yaml -``` - -```yaml -# clusters/prod/kustomization.yaml -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -resources: - - configmap-cache.yaml -``` - -`kubectl apply -k clusters/prod` works on the first commit. This is the part the current model cannot -do at all, and it is the difference between mirroring into a directory and producing a GitOps folder. - -### 3. `Tree`, declared - -A fleet folder holding many namespaces, where a flat layout would collide names. - -```yaml -spec: - layout: - kind: Tree -``` - -`clusters/prod/team-a/apps/deployments/api.yaml`. Identity-complete by construction, so two -namespaces holding `api` never share a file. Naming it in the spec is the point: it is a choice now -rather than what is left when nothing matched. - -### 4. `Flat`, the layout you asked for by name - -```yaml -spec: - allowedSourceNamespaces: - names: [team-a] - layout: - kind: Flat - scope: SingleNamespace - writeNamespace: Always # nothing here supplies the namespace, so the file must carry it -``` - -`clusters/prod/deployment-simon.yaml`. Legible, and the reason it cannot be the built-in is that it is -**not identity-complete**: two namespaces with a Deployment named `simon` render one path. - -**`Flat` is also the kind that may not omit the namespace**, and this is the table above applied rather -than an exception to it. `Never` is legal only when something guarantees the namespace, and under -`kind: Flat` there is no kustomization for us to write `namespace:` into: a flat directory has no build -step at all. So the only guarantor left would be a downstream applier pointed at the right namespace, -which is a promise made outside this object and invisible to it. `Always` is the honest setting here. -An earlier draft of this example wrote `Never` with the comment "the build supplies it, or the applier -does", which is exactly the hand-wave the `Never` guard exists to refuse. - -As a *kind* rather than a template, that is checkable instead of a caveat in prose. `Flat` requires -`scope: SingleNamespace`, admission refuses it beside an `allowedSourceNamespaces` matcher that admits -more than one namespace, and a document arriving from a second namespace is refused at the write -boundary with a message naming both. A template can only document the hazard; a kind can decline it. - -### 5. `Template`, the escape hatch, with the invariant still in force - -```yaml -spec: - layout: - kind: Template - default: "{namespace}/{resource}.yaml" # a per-namespace bundle per type - byType: - v1/secrets: "secrets/{namespace}/{name}{sensitiveSuffix}" -``` - -Everything today's `placement` can express, this can express. The difference is rule 1: if a -kustomization governs `team-a/configmaps.yaml`, the file is registered in it. Today the same -declaration produces an unrendered file (F10) whenever the governing kustomization is an ancestor -rather than a sibling. - -### 6. `byType` under a structural kind - -Overrides are not exclusive to `Template`, because "kustomize folder, but Secrets in their own -directory" is an ordinary thing to want. - -```yaml -spec: - layout: - kind: Kustomize - byType: - v1/secrets: "secrets/{name}{sensitiveSuffix}" -``` - -A Secret goes to `clusters/prod/secrets/db.sops.yaml`, and rule 1 registers it in the root's -`resources:`. What is refused is a blanket `default` here, because that would be an assertion that the -folder has no structural rule alongside an assertion that it has one. - -### 7. Two roots, and the answer ambiguity has been missing - -```yaml -spec: - path: clusters # holds overlays/staging AND overlays/production - layout: - kind: Kustomize -``` - -The user asserted a single-root folder. The folder disagrees, so this is a **misconfiguration of the -GitTarget**, not a placement puzzle: `Validated=False`, naming both roots, with the fix being one -GitTarget per overlay. Under `kind: Auto` the same folder resolves to `Tree`, and status says -`renderRootReason: Ambiguous` so nobody has to guess why files stopped landing in the overlays. - -This is why the refuse-or-write question was so hard to settle in the current model: without a -declaration there was nothing to contradict. With one, refusing is not a policy preference, it is -honoring what the user said. - -## Bootstrapping, and where it stops - -`create: true` is deliberately narrow: **the layout may create only what its own invariant requires.** -For `Kustomize` that is exactly one file, the `kustomization.yaml` the layout claims exists, plus the -`resources:` entries for what we write. - -Everything else people mean by "bootstrap a GitOps repo" is a different concern: per-environment -directories, an app-of-apps root, a Flux `Kustomization`, a README, a `.sops.yaml`. The last two we -already write ([`bootstrapped_repo_template.go`](../../internal/git/bootstrapped_repo_template.go)), -which is a useful precedent and also a warning: that mechanism is per-path bootstrap staging, and it -is where a "repository template" belongs if we ever grow one. Folding a folder skeleton into -`spec.layout` would make the layout responsible for the shape of a repository it does not own. - -So the boundary is: the layout creates what it needs to be true. A repository template, if it ever -exists, is a separate object with a separate lifecycle. - -## Should the layout be its own CRD? - -The reuse instinct is real: a platform team with thirty GitTargets should not paste the same four -lines thirty times. Three shapes, and I would not build the second one yet. - -### The argument for a `LayoutProfile` - -- **Reuse.** One house layout, referenced by every target. -- **Guardrails.** A cluster-scoped profile owned by the platform team, referenced by namespaced - GitTargets, is an RBAC boundary: tenants pick a layout, they do not invent one. -- **One place to change it.** A fleet-wide layout change becomes one edit. - -### The argument against, which I find decisive today - -**A shared object that changes where N folders write, with nothing on the GitTarget recording it, is -structurally the same defect this project spent a release deleting.** Sibling inference was -removed because a human's edit to a repository changed the operator's behavior with no Kubernetes -object changing and nothing in status recording the move. A profile edited in another namespace is -that same shape with a different actor: the GitTarget that owns the folder is unchanged, unreviewed, -and yet its next new file lands somewhere else. Being an API object rather than a folder makes it -auditable, which is better, but it does not make it *local*, and locality is what made the deletion -worth doing. - -The rest is ordinary cost, and it is not small: - -- **A third place to look.** The redesign exists to make "where do my files go" answerable from the - object. A `layoutRef` re-splits the answer across two objects, plus status to reconcile them. -- **Another readiness chain.** `GitProvider` already teaches this: a missing or invalid reference is a - new `Ready=False` mode, and a profile deleted while targets reference it needs an answer (freeze the - last resolved layout, or stop writing). -- **Cross-namespace authorization, again.** A namespaced profile referenced across namespaces needs - what `ClusterProvider` needed: an `allowedNamespaces` selector and a fail-closed SAR. That was - expensive to build and is expensive to keep correct. -- **The thing being shared is four lines.** `kind: Kustomize` plus two options does not carry enough - weight to justify an object. The reuse pressure is concentrated entirely in one place: a large - `byType` map. That is worth remembering, because it means if we ever do share something, we should - share **the type map**, not the kind. -- **Reuse is already solved one layer up.** Whatever creates thirty GitTargets (Helm, kustomize, a - Flux `ResourceSet`) repeats four lines for free, and it does so in a place the user already reviews. - A CRD that exists to avoid repetition in generated YAML is solving a problem the generator does not - have. - -### The middle shape, if evidence demands it - -Keep the layout inline and authoritative, add an optional `layoutRef` whose **resolved content is -projected into `GitTarget.status`**, stamped with the profile's `generation`. Reuse without -invisibility: the target still shows what it is doing, and a fleet-wide change is observable per -target rather than only at the profile. - -**Recommendation: inline now.** Revisit when someone has a fleet-wide layout they want to -change centrally, and revisit for the `byType` map first, since that is the only part that grows. The -trigger is written down so this is a decision rather than an omission. - -## Status under the model - -The two-halves rule from the other document still applies: a **current** half derived from the last -repository scan and stamped with the revision it came from, and a **historical** half accumulated -since. Placement is sparse, so the current half must never depend on a placement having happened. - -```yaml -status: - layout: - # current, from the last scan - declaredKind: Auto # what the spec says - kind: Kustomize # what it resolved to - renderRoot: . - renderRootReason: SingleKustomization # SingleKustomization | Ambiguous | None - byTypeEntries: 1 - observedRevision: 9f3c1ab - observedTime: "2026-07-30T09:14:22Z" - # historical, since observedRevision - placedResources: 14 # appends included - overriddenTypes: 1 # types a byType entry routed - refusedResources: 0 # resources NOT mirrored - examples: - - type: v1/secrets - path: clusters/prod/secrets/db.sops.yaml - source: ByType -``` - -`declaredKind` beside `kind` is the pair that makes `Auto` honest: it says both what was asked for and -what the folder produced, so declared inference never looks like a decision the user made. - -## Metrics under the model - -`placements_total` keeps `source` (which mechanism produced the path) and gains `layout` (the resolved -kind), because they answer different questions and both are one label: - -```promql -# is any target's declared layout not the one its folder produces? -sum by (gittarget_name, layout) (increase(gitopsreverser_placements_total[24h])) -``` - -`source` values follow the model: `byType`, `layout` (the kind's own rule), and nothing else. The -current `kustomize_root` and `canonical` values become `layout` with the kind in the other label, and -`declared`/`default` collapse into `byType` plus `Template`. That is a metric-label break, and it is -free while nothing consumes them. - -## Migration - -Mechanical, and every current configuration has an exact image: - -| Today | Under the model | -|---|---| -| no `spec.placement` | `layout: {kind: Auto}` (defaulted). Same behavior | -| `placement.byType` only | `layout: {kind: Auto, byType: {...}}`. Same behavior, plus rule 1 fixing F10 | -| `placement.default` set | `layout: {kind: Template, default: "..."}`. The declared default no longer shadows the render root, because the kind now says there is no structural rule | -| `placement.byType` + `default` | `kind: Template` with both | - -The one behavior change is the good one: a declared template stops silently disabling the render root, -and starts registering its files instead. - -`spec.placement` becomes a loud rejection for one release rather than a silent alias, following the -pattern `ClusterWatchRule.spec.rules[].scope` set: refusing a stored field the user can see beats -translating it behind their back. It rides the Tier 2 breaking wave in -[`open-asks-priority.md`](open-asks-priority.md), so the consumer pays one coordinated bump. - -## What this changes about the work already queued - -Nothing already planned is wasted, and one item should wait: - -- **F10's ancestor walk** is rule 1's implementation. Build it now; the model makes it an invariant - rather than a fix. -- **`{kindLower}`** is `Flat`'s file name and a `Template` variable. Build it now. -- **The `{version}` identity fix** is needed by `Flat`'s validation and by any versionless template. - Build it now. -- **Canonical as a template constant** is `Tree`'s implementation. Build it now. -- **`status.layout`** is the same field, one release early, and `declaredKind`/`kind` slot into it. - Build it now. -- **The ambiguity policy** should wait. In this model it follows from whether the user asserted a - root, and deciding it before the model exists would bake in an answer to a question the model asks - differently. - -## Open questions - -- ~~Is `Auto` the right default?~~ **Yes**, given that it resolves once and pins the result. Pinning - is what removes the harm, and the quickstart stays four fields. -- ~~Should `Flat` be refused for a multi-namespace target?~~ **Refuse the writes, not the target.** A - scope-widening edit in another object must not break a folder; the second namespace's documents are - declined with a counted refusal naming both namespaces, and the fix is a widening layout change. -- Should `scope` be **derived and materialized** at creation (write `SingleNamespace` into the spec - when exactly one namespace is admitted) rather than declared? It would make the common case - zero-config, at the price of a mutating webhook writing spec, which this project has deliberately - avoided outside identity capture. -- Does `writeNamespace: Never` need to name its supplier (`Kustomize`, `FluxTargetNamespace`, - `Asserted`) so validation can check the guarantee rather than trust it? -- **Where does the namespace VALUE come from?** `scope: SingleNamespace` constrains cardinality; it - does not say *which* namespace, and the bootstrap case is exactly the one that cannot read it off - existing resources. Writing `namespace: team-a` into a `kustomization.yaml` we create presupposes - knowing `team-a`. The obvious source is `allowedSourceNamespaces.names` when it admits exactly one, - which every example here implicitly assumes, and that is worth making explicit rather than implied: - it would mean `create: true` plus `SingleNamespace` is refused when the admitted set is a selector - or is empty, because there is no name to write. The alternative — take it from the first document - that arrives — makes the folder's convention depend on arrival order, which is the failure mode this - whole model exists to remove. -- Does `kind: Kustomize` imply `create: true`? Asserting a folder is a kustomize folder arguably - asserts the file exists, and requiring both feels like ceremony. The argument for keeping them - separate is that creating a file in someone's repository should always be something they asked for. -- Should `Tree` remain identity-complete-by-definition, or become configurable (with or without the - version segment)? The versionless decision is deliberate and this document does not reopen it. diff --git a/docs/design/open-asks-priority.md b/docs/design/open-asks-priority.md index ae319604..e98acc35 100644 --- a/docs/design/open-asks-priority.md +++ b/docs/design/open-asks-priority.md @@ -8,8 +8,8 @@ > sticky removal pointer, the metric relabel, the name tier, and the analyzer/encoder corrections, > plus PR #291's sibling-inference deletion and placement counters. That is a large breaking release > on its own, and **the GitTarget work is deliberately not in it**. The layout model and the API wave -> ([`gittarget-layout-model.md`](gittarget-layout-model.md), -> [`gittarget-api-wave.md`](gittarget-api-wave.md)) are postponed to a later deployment and tracked as +> ([`../layout/model.md`](../layout/model.md), +> [`../layout/api-wave.md`](../layout/api-wave.md)) are postponed to a later deployment and tracked as > GitHub issues, so the queue below is read with one standing caveat: **every Tier 2 entry that > changes a `GitTarget` field is now part of that postponed wave, not independently schedulable.** > The Tier 1 entries are not, and should not wait for it. @@ -22,9 +22,9 @@ > a fact about the last week, not a revision of the rule. > > Three backlogs are open at once and they overlap: the gitops-api consumer asks (revision 11, -> 2026-07-28, which is the revision that filed #23), the maintainer review's unbuilt block in -> [`flux-maintainer-review-status-and-config-model.md`](../future/flux-maintainer-review-status-and-config-model.md) -> (F6, F9, F10), and the config-surface proposal in +> 2026-07-28, which is the revision that filed #23), the API-surface block left unbuilt by the +> status and configuration-model review — now sequenced in +> [`../layout/api-wave.md`](../layout/api-wave.md) — and the config-surface proposal in > [`config-surface-for-a-structured-repository.md`](../future/config-surface-for-a-structured-repository.md) > (B1–B6). This page merges them into one queue and says where we deliberately do **not** do > what was asked. @@ -68,7 +68,7 @@ do not own. > **Built.** `resolveInferred` through `allSameDir` are gone, the kustomize-root fallback stayed, and no > enum was added. What building it added to the argument below is recorded in > [what the deletion taught](#what-the-deletion-taught). The spec's Option C sections are retained as -> history in [`gittarget-new-file-placement-rules.md`](../spec/gittarget-new-file-placement-rules.md), +> history in [`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md), > and the behaviour change has a [`docs/UPGRADING.md`](../UPGRADING.md) entry. The config-surface proposal's **B3** offers `spec.placement.mode: Infer|Declared|Strict`: an @@ -77,7 +77,7 @@ cohort ladder entirely**, keep the kustomize-root fallback, and ship no enum at ### What inference is, precisely -[`gittarget-new-file-placement-rules.md`](../spec/gittarget-new-file-placement-rules.md) Option C. +[`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md) Option C. It fires **only** for a resource that has no document in Git yet; everything already written is match-first and never moves. For that narrow case it finds the largest cohort of similar existing documents (step 1: same type + namespace; step 2: same type, any namespace) and puts the new @@ -236,17 +236,17 @@ and is not independently schedulable. |---|---|---|---|---| | 15 | A declared `auditRoute` with zero facts must say so, and a route losing them with it | gitops-api | **1** | — | | n/a | Stop paying a full grace for a delete fact that will never arrive (F, then C) | [`attribution-removal-wait-options.md`](attribution-removal-wait-options.md) | **1** | — | -| n/a | A declared path in a kustomize subdirectory is never rendered; the identity gate rejects the versionless canonical path | [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md) | **1** | [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) | -| n/a | `spec.layout`: declare what the folder is | [`gittarget-layout-model.md`](gittarget-layout-model.md) | **2** | [#293](https://github.com/ConfigButler/gitops-reverser/issues/293), wave | -| F6 | `spec.suspend`, `spec.interval`, `requestedAt` | maintainer review | **2** | wave | +| ~~n/a~~ | ~~A declared path in a kustomize subdirectory is never rendered; the identity gate rejects the versionless canonical path~~ **SHIPPED** in 0.42.1 | [`placement-visibility-and-declared-defaults.md`](../layout/placement-visibility-and-declared-defaults.md) | — | [#295](https://github.com/ConfigButler/gitops-reverser/issues/295), [#319](https://github.com/ConfigButler/gitops-reverser/pull/319) | +| n/a | `serializeNamespace` and `kustomizeRoot`: the two things a path template cannot say (`spec.layout` was reversed) | [`../layout/model.md`](../layout/model.md) | **2** | [#322](https://github.com/ConfigButler/gitops-reverser/issues/322), **not** breaking, so not the wave | +| F6 | `spec.suspend`, `GitProvider.spec.interval`, `requestedAt` (no `interval` on `GitTarget`, see [`../layout/api-wave.md`](../layout/api-wave.md)) | maintainer review | **2** | wave | | 5 | `CommitRequest.spec.author`, SAR-guarded | gitops-api (#220) | **2** | wave | | B4 | `commitWindow` / `commit.message` move to GitTarget | config surface | **2** | wave | -| B1 | `GitTarget.spec.mode: Observe\|Write` | config surface | **2** | wave | +| ~~B1~~ | ~~`GitTarget.spec.mode: Observe\|Write`~~ **dropped**: `suspend` on a still-scanning target is the same dry run with one field | config surface | — | [`../layout/api-wave.md`](../layout/api-wave.md) | | 6 | Movable destination via `status.observedDestination` | gitops-api (#220) | **2** | wave | | F10 | CommitRequest TTL / ownerRef + the `delete` verb | maintainer review | **2** | wave | | n/a | The blocking resolve is head-of-line on the shard goroutine | [`../spec/attribution.md`](../spec/attribution.md#the-wait) | **2** | — | -| B2 | `GitTarget.status.layout` | config surface | **3** | [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) | -| n/a | The ambiguous render root, the `declared` metric split, `{kindLower}`, canonical-as-template | [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md) | **3** | [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) | +| B2 | `GitTarget.status.placement` (was `status.layout`) | config surface | **3** | [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) | +| n/a | The ambiguous render root, the `declared` metric split, `{kindLower}`, canonical-as-template | [`placement-visibility-and-declared-defaults.md`](../layout/placement-visibility-and-declared-defaults.md) | **3** | [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) | | F9 | The `scope: Namespaced` status-write envtest | maintainer review | **3** | outside the wave, deliberately | | B6 | The `default` ClusterProvider not-found message | config surface | **3** | — | | n/a | An aggregated create carries no name and no body: accept it, or stop waiting for it | [`../spec/attribution.md`](../spec/attribution.md#what-the-shape-driven-rules-reach-and-what-they-do-not) | **3** | — | @@ -495,7 +495,10 @@ carry, and an aggregated-API create is logged with no name and no response body `#220` shape — honored only against an admission record carrying an authorized verdict, fail-closed independent of the webhook's `failurePolicy` — remains the right one, on the first argument alone. -**B4, B1, #6, F10** as written in their source documents. #6 is explicitly a lower priority than +**B4, #6, F10** as written in their source documents. **B1 has left the wave**: a suspended +`GitTarget` that keeps scanning is the same dry run with one field instead of two, so `mode` buys +only the difference between a pause and a declared posture. The re-open trigger is in the wave +document. #6 is explicitly a lower priority than when it was filed: the consumer downgraded it themselves, because branch and folder are now chosen once per repository on an object that exists because the user picked that repository. It rides the wave because it is in the wave, not because it is urgent. @@ -581,7 +584,7 @@ defects. fall-back-to-canonical Event.~~ **Done for the removal and the entry**; the Event is still undecided, and the metric now carries the actionable part in the meantime. 3. ~~Rewriting Option C's sections in - [`gittarget-new-file-placement-rules.md`](../spec/gittarget-new-file-placement-rules.md).~~ + [`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md).~~ **Done**: the ladder is documented as three steps, the kustomize-root fallback keeps its section and gained the namespace-match rule, and P1–P10 are annotated one by one with which are retired by the deletion and which (P7, P9, P10) are facts about the code that remains. @@ -608,5 +611,5 @@ defects. 9. Not letting a decided-but-unbuilt list read as imminent. The placement work is filed as [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) (correctness) and [#296](https://github.com/ConfigButler/gitops-reverser/issues/296) (visibility); - [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md) + [`placement-visibility-and-declared-defaults.md`](../layout/placement-visibility-and-declared-defaults.md) now says which of its eight items shipped, which is **none of them**. diff --git a/docs/design/source-scope-simplification.md b/docs/design/source-scope-simplification.md new file mode 100644 index 00000000..51d40156 --- /dev/null +++ b/docs/design/source-scope-simplification.md @@ -0,0 +1,247 @@ +# Source scope: what to delete, and what to keep + +> Status: design. Nothing here is built, and nothing binds until it is scheduled. +> Date: 2026-08-28. Index: [`../INDEX.md`](../INDEX.md). +> +> Answers a design review's proposal to adopt Flux-style `serviceAccountName` impersonation for +> source reads, by declining it. Evidence for the impersonation half is in +> [`../facts/kubernetes-impersonation-and-flux-identity.md`](../facts/kubernetes-impersonation-and-flux-identity.md), +> including a correction to a claim in that review. + +## Decision + +1. **Do not build impersonation.** It buys several identities behind one credential against one + source cluster. Nobody has asked for that, and the cost is four pieces of machinery and an + `impersonate` grant. +2. **Delete `GitTarget.spec.allowedSourceNamespaces`** and its selector machinery. +3. **Rename `ClusterProvider.spec.allowSourceNamespaceOverride`** to + `allowAnySourceNamespace`, still a boolean, still defaulting to false. +4. **Decide `sourceNamespace: "*"` explicitly** (see below). It is defined in terms of the field + being deleted, so it cannot be left alone. +5. **Keep `ClusterProvider.spec.allowedNamespaces`**, renamed `accessFrom`, until a replacement is + shipped and tested. It is the one field here doing work that nothing else does. + +The name keeps `Source` deliberately. `ClusterProvider` carries two namespace planes, and +`allowAnyNamespace` sitting directly beneath `accessFrom` would read as a modifier on it. +`allowCrossNamespace` was the other candidate, borrowing Flux's `--no-cross-namespace-refs` +vocabulary, and it was not taken: in Flux the phrase means object references across namespaces in +one cluster, while here the far side is a namespace in a different cluster. The field's own design +rationale already records why that matters, and it is the sharpest statement of it in the repo: for +a remote provider "the config-plane namespace and the source namespace are on different clusters, +so their sharing a name never was a boundary". Crossing is literally true only for the in-cluster +provider; `any` is literally true for both. It stays a +boolean because there are two states and no third one is in view: impersonation and source-side +selectors are both out, so an enum would only be leaving room for something nobody can name. The +review's objection to a boolean here was that it sat among "two policy objects answering +neighbouring halves"; deleting `allowedSourceNamespaces` removes the other half, and the objection +with it. + +## Why `allowedNamespaces` stays + +An earlier draft proposed deleting it, on the evidence that the chart renders +`allowedNamespaces: {selector: {}}` and so admits every namespace. That evidence is real but +narrower than it looked: it is the value for the **chart-owned `default` provider only** +([`values.yaml`](../../charts/gitops-reverser/values.yaml)). Every user-authored `ClusterProvider`, +which is every remote cluster, is deny-by-default and carries an explicit consumer list its author +wrote on purpose. + +The boundary it draws is not available anywhere else. Source-cluster RBAC bounds **what a +credential may read**. It cannot express **which control-plane tenant may wield that credential**, +because the tenant is not a subject in the source cluster at all. Deleting the field would make a +shared source credential usable from any namespace that can create a `GitTarget`. + +A `ValidatingAdmissionPolicy` is not a drop-in replacement either, and this repository has already +written down why: [`../spec/where-validation-lives.md`](../spec/where-validation-lives.md) states +that reconcile-time is the **stronger** gate, because admission is one-shot and cannot see a policy +tightened after the object was created, so "an admission-only check is strictly less safe". A VAP +can stop a new or updated `GitTarget` from selecting a provider. It cannot stop a stored one whose +namespace lost its label. Recommending a VAP here contradicted our own doctrine. + +What can be simplified without touching the boundary: the field reads **control-cluster** namespace +labels, locally, with no cross-cluster call and no degradation path, so its selector is cheap in a +way the source-side one is not. Keeping both halves is fine. Renaming to `accessFrom` is still +worth doing, and matters more once the two `allowed*Namespaces` fields no longer sit side by side +to disambiguate each other. + +## Why the rest goes + +`GitTarget.spec.allowedSourceNamespaces` presents itself as a destination policy: "it belongs to +the DESTINATION, not to any requesting rule". It cannot be one. `WatchRule.spec.targetRef` is a +`LocalTargetReference` ("Must be in the same namespace"), `GitTarget.spec.providerRef` is local +too, and `spec.path` is immutable, so the chain from a Git folder back to the object that fills it +never leaves one namespace: + +```text +Git folder <- GitProvider <- GitTarget <- WatchRule + (same namespace, all the way) +``` + +Whoever can create a `WatchRule` there can already write into that folder. What the field really +bounds is which source namespaces the folder's own tenant may **read**, and for a credential-scoped +provider that restates what the credential already carries, in the one place that cannot revoke. +`ClusterWatchRule` bypasses it entirely today and nobody has minded. + +**The complexity is in the matcher, not the fence.** `NamespaceMatcher`'s selector half is +evaluated against `Namespace` labels **in another cluster**. That one choice produces the +three-valued verdict (so a source-cluster outage is not a denial), the `SourceScopeUnavailable` +degradation path, the five condition reasons, and the operator's need for source-cluster +`Namespace` get/list/watch. Delete the source-side selector and all of it goes. + +```mermaid +flowchart LR + subgraph before["Today"] + direction TB + B1["ClusterProvider.allowedNamespaces
control-cluster labels, local read"] + B2["ClusterProvider.allowSourceNamespaceOverride"] + B3["GitTarget.allowedSourceNamespaces
SOURCE-cluster labels, cross-cluster read"] + B4["3-valued verdict, 5 reasons,
SourceScopeUnavailable path"] + B3 --> B4 + end + subgraph after["Proposed"] + direction TB + A1["ClusterProvider.accessFrom
unchanged boundary, renamed"] + A2["ClusterProvider.allowAnySourceNamespace
boolean, default false"] + A3["a string comparison"] + end + B1 ==>|"kept"| A1 + B2 ==>|"renamed"| A2 + B3 -.->|"deleted"| A3 + B4 -.->|"deleted with it"| A3 +``` + +## `sourceNamespace: "*"` needs its own decision + +Today `*` means "every source namespace this `GitTarget` admits", resolved live through +`allowedSourceNamespaces` into a concrete set, which is then planned as one stream per namespace. +[`watchrule_types.go`](../../api/v1alpha3/watchrule_types.go) says so in the constant's own doc: +"never `every namespace that exists`". So `*` is **defined in terms of the field being deleted** +and cannot survive unchanged. + +RBAC cannot supply the missing definition. It answers "may I watch X in namespace Y", never "which +namespaces may I watch". Any set-valued reading of `*` requires listing `Namespace` objects in the +source cluster, which is the cost this whole exercise is trying to remove. Three options: + +| Option | Meaning | Cost | +|---|---|---| +| **Delete `*`** | items name namespaces | safest, and a breaking change for anyone using it | +| **Redefine as cluster-wide** | one watch and one list at `metav1.NamespaceAll`, all or nothing | implementable with no `Namespace` access, and a real widening: it reaches everything the credential can see | +| **Keep enumeration** | as today, against some other policy | keeps the cross-cluster `Namespace` read, which was the point of the deletion | + +**Decided: redefine as cluster-wide**, rejected outright while `allowAnySourceNamespace` is false. +It applies to **both** halves of a cell's traffic (the initial list that warms the cell, and the +watch that follows it), because they are the same collection read two ways, and splitting them +would mean enumerating namespaces for the replay after all, which is the read this exercise +deletes. + +The plumbing already exists, which makes this the cheapest of the three as well as the clearest. +`CellKey.Namespace` is already documented as "empty is a genuinely cluster-wide (all-namespaces) +cell" ([`cell.go`](../../internal/types/cell.go)), and both +[`openTargetWatch`](../../internal/watch/target_watch.go) and `openTargetList` already branch on +it: a non-empty namespace calls `resource.Namespace(ns)`, an empty one calls `resource.Watch` +directly, which for a namespaced GVR is the all-namespaces collection. Readiness, retention +rollup, and event routing all key on `CellKey` already, and records carry the object's own +`u.GetNamespace()` rather than the cell's, so placement is unaffected. What changes is only the +planner: `*` compiles to one cell instead of calling `EnumerateSourceNamespaces` for N. That is a +deletion in `watchrule_compile.go`, not new machinery. + +One trap, and the code already records it. A cluster-wide cell is a **peer** of a named-namespace +cell on the same type, never a replacement, because each rule carries its own `operations` filter. +`CellKey`'s doc comment names the bug from a previous attempt: collapsing the two "widened the named +rule's stream to every namespace its credential could read and discarded its operation filter". So +a target carrying both `*` and a named rule for one type runs two streams over overlapping objects, +and that is correct rather than something to optimize away. +It is what a Kubernetes reader expects `*` to mean, and its failure is a clean 403 rather than a +silent empty set. + +**It is also the largest single efficiency win in this proposal, and that is a reason in its own +right.** A `*` rule over a type in a hundred-namespace cluster is a hundred watch connections and a +hundred list calls at warm-up today, one per namespace, each with its own cursor, its own retry +schedule and its own share of apiserver watch cache. Cluster-wide makes it one of each, and the +saving grows with the cluster, which is exactly the direction the enumeration got worse in. The +`TooManyStreams` cap was queued for the fan-out this deletes; see +[`../layout/api-wave.md`](../layout/api-wave.md), where that rider is now smaller than it was. + +## Consequences, including two breaking semantic changes + +**A declared policy can currently deny a rule's own namespace.** `allowedSourceNamespaces` is +exhaustive once declared, "with no exception for a rule's own namespace" +([`gittarget_types.go`](../../api/v1alpha3/gittarget_types.go)). So `allowAnySourceNamespace: false` +is **not** the current posture exactly. It matches the no-policy path, which is what a default install runs, and it is a +deliberate simplification for anyone who declared a policy that excluded their own namespace. Say +so in the migration note rather than claiming continuity. + +**`*` changes meaning**, per the decision above: it widens from "every namespace this GitTarget +admits" to "every namespace this credential can read", and it stops being available at all unless +`allowAnySourceNamespace` is true. For a user who had no `allowedSourceNamespaces` policy the old +`*` already resolved to whatever the credential could see, so the widening is narrower in practice +than it reads; for a user who had one, it is real. + +**Label selectors over source namespaces are lost.** Admitting every namespace carrying a label, +following namespaces as they appear, has no RBAC equivalent short of a binding per namespace. This +is the real capability cost. An N-way restriction costs N objects wherever it is expressed, and +today's version is cheap only because it enforces nothing. + +**What is deleted**, 4,569 lines in files that exist for nothing else, with coupling into shared +files running 1 to 26 mentions, so this is a deletion rather than a refactor: + +| File | Lines | +|---|---| +| `internal/authz/source_namespace.go` and its test | 1351 | +| `internal/watch/source_namespace_scope.go` and its tests | 1312 | +| `internal/controller/watchrule_source_namespace.go` and its test | 770 | +| `api/v1alpha3/namespace_matcher.go` selector half | part of 371 | +| `test/e2e/source_namespace_e2e_test.go` | 349 | + +`internal/authz/clusterprovider_admission.go` stays, since `accessFrom` stays. +`ClusterProviderNotFound` stays with it, though it is reference resolution rather than +authorization and would read better elsewhere. + +## The one thing to build + +A `SelfSubjectAccessReview` pass under the provider's own credential. It needs no new grant, since +every identity may issue one, and it reports two things a user cannot otherwise get: which of the +requested cells are reachable, and whether write verbs are permitted on them. + +Phrase the condition as **"no write permission observed for the requested resources at review +time"**. It cannot prove the mirror is unable to write: a review covers the verbs and resources +asked about, at that instant, and says nothing about other resources, subresources, or a later +change. It is a diagnostic, and a good one, not a proof. + +This is also what replaces the deleted policy in the place that matters: the user stops declaring +which namespaces are permitted, and starts being told which are reachable. + +## Migration + +One breaking change, in the wave already queued, riding the loud-rejection pattern the project uses +for superseded fields: + +- `GitTarget.spec.allowedSourceNamespaces` removed. +- `allowSourceNamespaceOverride` becomes `allowAnySourceNamespace`: same type, same default, same + semantics, so the shim is a pure rename. +- `allowedNamespaces` becomes `accessFrom`, same shape, same semantics. +- `sourceNamespace: "*"` becomes one cluster-wide list and watch, per the decision above, and is + rejected while `allowAnySourceNamespace` is false. This is a semantic change to a value that + keeps its spelling, so it needs its own `docs/UPGRADING.md` paragraph rather than a shim. + +Then the deletion, then the `SelfSubjectAccessReview` work, which is additive and can ship later. + +## Re-open triggers + +- **Impersonation**, if one source cluster must serve control-plane tenants with different + authority and they cannot each hold a credential. The cost is priced in the facts note. +- **Deleting `accessFrom`**, once an external policy recipe is shipped, tested, and its lack of + live revocation is an accepted product choice rather than an oversight. +- **Source-side selectors**, if a user is found who needs "every namespace labeled X" and cannot + generate bindings. + +## Open questions + +- ~~Which `*` option is taken.~~ **Decided: cluster-wide**, list and watch both, above. +- `CellKey.String()` renders an empty namespace as a bare type name ("configmaps"), which read as + cluster-scoped when only `ClusterWatchRule` produced those cells. Once `*` produces them for + namespaced types it wants a distinct rendering, "configmaps in all namespaces" or similar, in + logs and status messages. +- Does the chart keep an `allowAnySourceNamespace` value, and does the quickstart set it true? A + homelab wants it; a platform should refuse it. +- Does anything rely on `allowedSourceNamespaces.selector` today beyond the e2e suite, which is us + testing our own feature? diff --git a/docs/design/support-boundary/finished/images-and-replicas-edit-through.md b/docs/design/support-boundary/finished/images-and-replicas-edit-through.md index 31c2a66c..edd1a7e4 100644 --- a/docs/design/support-boundary/finished/images-and-replicas-edit-through.md +++ b/docs/design/support-boundary/finished/images-and-replicas-edit-through.md @@ -10,7 +10,7 @@ > Captured: 2026-07-06 > Related: > [../README.md](../README.md), -> [contextual-namespace-and-kustomize-folder-editing.md](../../../spec/contextual-namespace-and-kustomize-folder-editing.md), +> [../../../layout/contextual-namespace.md](../../../layout/contextual-namespace.md), > [manifestedit/DECISION.md](../../../../internal/git/manifestedit/DECISION.md) ## Problem diff --git a/docs/design/support-boundary/helm-light-support-boundary.md b/docs/design/support-boundary/helm-light-support-boundary.md index 7bd85e5e..dfde502a 100644 --- a/docs/design/support-boundary/helm-light-support-boundary.md +++ b/docs/design/support-boundary/helm-light-support-boundary.md @@ -130,7 +130,7 @@ and refused **by name**, the way `patchesJson6902` refuses under its own name to `patternProperties` and an unresolved reference mean for closure before any of this is built. The schema is the chart author *declaring* the input surface, a closed leaf set, which is the same declared-over-inferred principle the layout model rests on - ([gittarget-layout-model.md](../gittarget-layout-model.md)). A values key the schema + ([../../layout/model.md](../../layout/model.md)). A values key the schema does not declare refuses the folder. 3. No `dependencies:` in `Chart.yaml`, no `charts/` directory, no `Chart.lock`. The chart is one program, not a graph of them. (Vendored dependencies are a conceivable later @@ -302,7 +302,7 @@ locally, no plugins, no network, in-memory filesystem as the jail. How helm-light surfaces on the API (`layout.kind`, or a renderer property of the structural kind) belongs to the GitTarget API wave -([gittarget-api-wave.md](../gittarget-api-wave.md)), not to this boundary. +([../../layout/api-wave.md](../../layout/api-wave.md)), not to this boundary. ## Where the arguments live diff --git a/docs/design/support-boundary/render-root-scoping.md b/docs/design/support-boundary/render-root-scoping.md index db53cc79..417cbabd 100644 --- a/docs/design/support-boundary/render-root-scoping.md +++ b/docs/design/support-boundary/render-root-scoping.md @@ -115,7 +115,28 @@ overlay rather than writing the read-only base, and the re-render adjudicates th verification. What remains is a strategic-merge patch for a base-owned field that is *not* an image, replica, or whole-object delete. -## 5. Remaining work +## 5. The repository-level peer: paths another controller writes + +Render-root scoping keeps one GitTarget out of another target's partition. The same argument +applies one level up, to folders no GitTarget should claim at all: + +> A `GitTarget` must not point at a path another controller writes. Flux's bootstrap directory +> (`clusters//flux-system`) is the common case, and the `Kustomization` that reconciles a +> folder is not a licence to co-write it. + +`flux bootstrap` and flux-operator own `clusters//flux-system`, including the +`kustomization.yaml` that lists `gotk-components.yaml` and `gotk-sync.yaml`. An operator adding +`resources:` entries there is a second writer in a folder Flux's own sync loop reconciles, which is +the two-writers-one-folder failure this boundary exists to prevent — and the fact that Flux +*renders* the folder is what makes it look eligible, not what makes it safe. Being reconciled by a +controller is the tell, not the licence. + +This is a rule for the person choosing `spec.path`, not something the operator can enforce: nothing +in the repository marks a folder as bootstrap-owned. It is stated here so the answer exists in the +support boundary rather than only in an example, and +[`homelab-flux`](../../layout/examples/homelab-flux/README.md) is the worked case. + +## 6. Remaining work 1. Add a dedicated cluster end-to-end overlay case (discovery classification is already flipped — an external-base overlay reports accepted). diff --git a/docs/design/support-boundary/repo-discovery-and-onboarding-scan.md b/docs/design/support-boundary/repo-discovery-and-onboarding-scan.md index caf5a96b..077f72bd 100644 --- a/docs/design/support-boundary/repo-discovery-and-onboarding-scan.md +++ b/docs/design/support-boundary/repo-discovery-and-onboarding-scan.md @@ -14,7 +14,7 @@ > [kustomize-support-boundary.md](kustomize-support-boundary.md), > [finished/images-and-replicas-edit-through.md](finished/images-and-replicas-edit-through.md), > [finished/higher-level-krm-documents.md](finished/higher-level-krm-documents.md), -> [../manifest/file-agnostic-placement.md](../../spec/gittarget-new-file-placement-rules.md), +> [../manifest/file-agnostic-placement.md](../../layout/new-file-placement-rules.md), > [../../finished/current-manifest-support-review.md](../../spec/current-manifest-support-review.md) ## Why this exists @@ -49,7 +49,7 @@ Repo-wide target discovery is therefore a **new axis**, not a new operator feature. Putting it in the operator would fight both the simplicity goal and the one-owner invariant. It belongs in the CLI/library, and anything built on top of the operator consumes its report — exactly the division already recorded in -[file-agnostic-placement.md](../../spec/gittarget-new-file-placement-rules.md) and the +[file-agnostic-placement.md](../../layout/new-file-placement-rules.md) and the [README](README.md) responsibilities table: | Layer | Owns | Gains repo-wide discovery? | diff --git a/docs/facts/kubernetes-impersonation-and-flux-identity.md b/docs/facts/kubernetes-impersonation-and-flux-identity.md new file mode 100644 index 00000000..7901a3eb --- /dev/null +++ b/docs/facts/kubernetes-impersonation-and-flux-identity.md @@ -0,0 +1,106 @@ +# Impersonation and service-account identity, as Kubernetes and Flux implement them + +> **facts** — durable reference. Index: [`../INDEX.md`](../INDEX.md) +> +> Read for [`../design/source-scope-simplification.md`](../design/source-scope-simplification.md), +> which decided against adopting impersonation. This page holds the evidence so the case does not +> have to be re-derived if the decision is re-opened. +> +> Verified against `k8s.io/apiserver@v0.34.1` in the module cache and the gitignored +> `external-sources/flux/` checkout. Both are named by path rather than linked, because neither is +> tracked by this repository. + +## What the API server does + +All five are from `pkg/endpoints/filters/impersonation.go`. + +**The subject does not have to exist.** An `Impersonate-User` header of +`system:serviceaccount:homelab-config:mirror` is split by `serviceaccount.SplitUsername` into a +`ServiceAccount` object reference, and nothing ever reads that object. The identity is a string, +and the namespace half need not name a namespace that exists. + +**The authorization check is namespaced and name-scoped.** The filter builds an attributes record +with verb `impersonate`, resource `serviceaccounts`, and both the namespace and the name taken from +the header. So an RBAC rule can bound impersonation with `resourceNames`. A `ClusterRole` doing so +bounds it by name only, across every namespace; bounding by namespace needs a `Role`. + +**The service-account groups are added without a check.** When no groups are requested the filter +sets `groups = serviceaccount.MakeGroupNames(namespace)`, which is `system:serviceaccounts` and +`system:serviceaccounts:`. Those are appended after the per-request authorization loop, +so no `impersonate` check runs against them. Any privilege the cluster binds to every service +account in that namespace comes along with the identity. + +**The impersonator's own permissions are not an upper bound.** The filter constructs a fresh +`user.DefaultInfo` and replaces the request user with it; the original survives only in the audit +log. There is no intersection between what the impersonator may do and what the impersonated +subject may do. + +> This last one is worth stating loudly, because the natural assumption is the opposite, and an +> earlier design review recorded that assumption as fact — that the operator's own `ClusterRole` +> remains the outer bound, so the effective permission is the intersection of what the operator may +> read and what the impersonated subject may read. It is not. The effective permission is +> exactly what the impersonated subject may do, which is why `resourceNames` scoping would be +> load-bearing rather than tidy: without it, `impersonate` on `serviceaccounts` is +> admin-equivalent in the target cluster. + +**Impersonation is per request, so it applies to watches.** The header travels on a watch request +like any other, and when the binding behind it is removed the API server ends the stream. Whoever +reads under an identity the target cluster issued gets revocation without implementing it. + +## What Flux does + +From `pkg/runtime/client/impersonator.go` and +`flux-operator/internal/controller/resourceset_controller.go`: + +- `setImpersonationConfig` builds `system:serviceaccount:%s:%s` from `i.serviceAccountNamespace`, + and every caller passes the reconciled object's own namespace + (`WithServiceAccount(r.DefaultServiceAccount, obj.Spec.ServiceAccountName, obj.GetNamespace())`). + No field anywhere lets a user write the namespace half. That is the whole security argument: a + tenant can claim only an identity bounded by the RBAC their own namespace already grants. +- `clientForKubeConfig` calls `setImpersonationConfig` too, so impersonation **composes** with a + remote kubeconfig rather than replacing it. +- `CanImpersonate` does a `Get` for the `ServiceAccount` through the local client. On the + kubeconfig path that checks the wrong cluster, and it is an existence check where an + authorization check is wanted. RFC 0010 flags it as a known bug. + +RFC 0001 states the opposite of the second point ("All accesses that would use impersonation use +the remote client instead"). The RFC predates the code. Cite the code. + +## Impersonating into another cluster + +RFC 0010 states the rule directly: with `spec.serviceAccountName`, the authenticated identity "must +have the necessary permissions to impersonate this `ServiceAccount` in the remote cluster". + +Combined with the first fact above, this means the impersonated `ServiceAccount` is resolved in the +remote cluster and nowhere else, and the namespace in the identity string comes from the local +object. The pair is a **name convention two clusters agree on**, not an object reference. The +remote cluster's admin honors it with an ordinary `RoleBinding` naming a subject whose namespace +may not exist locally to them. + +The consequence for any design that adopts this: whoever can create a namespace in the local +cluster can mint an identity claim against every remote cluster that ever granted that name. + +## Where Flux puts multi-tenancy + +Not in its API. `--no-cross-namespace-refs` and `--default-service-account` are controller flags, +and the workload-identity profile splits the latter into three by concern +(`--default-decryption-service-account`, `--default-kubeconfig-service-account`). The multi-tenant +lockdown is a documented Kyverno profile plus `flux-operator`'s +`internal/builder/profiles.go`. The one API-level exception, `acl.AccessFrom` with its +`namespaceSelectors`, is on sources and answers "which namespaces may reference this object". + +## What adopting impersonation here would require + +Recorded so the cost is not re-estimated: + +- The tenant must pick the name half and never the namespace half, or they can name + `kube-system:default`. +- `ClusterWatchRule` is cluster-scoped and has no namespace to derive from, so it needs an explicit + `serviceAccountRef`. Acceptable only because creating a cluster-scoped object is already an admin + act. +- The precheck is a `SubjectAccessReview` against the target cluster, not a `Get`, and it must + carry `system:serviceaccounts` and `system:serviceaccounts:` as groups or it answers + narrower than reality. That needs `create subjectaccessreviews` on a credential users have + already issued. +- Identity joins cluster, GVR, and namespace in the informer key, so it multiplies the resource the + `TooManyStreams` cap exists to bound. diff --git a/docs/future/config-surface-for-a-structured-repository.md b/docs/future/config-surface-for-a-structured-repository.md index 3a8f8dec..0985e496 100644 --- a/docs/future/config-surface-for-a-structured-repository.md +++ b/docs/future/config-surface-for-a-structured-repository.md @@ -2,8 +2,10 @@ > Status: proposal — review findings and an ordered plan. Nothing here binds until scheduled. > Date: 2026-07-24 -> Companion to [flux-maintainer-review-status-and-config-model.md](flux-maintainer-review-status-and-config-model.md), -> whose §4 "Then (API surface)" block (F6, F9, F10) this plan absorbs and extends. +> Absorbs and extends the API-surface block left unbuilt by the status and configuration-model +> review — `spec.suspend`, `spec.interval` and the reconcile-request annotation, the +> `CommitRequest` lifecycle, and the `ClusterWatchRule` scope question — which is now sequenced in +> [`../layout/api-wave.md`](../layout/api-wave.md). ## The one-sentence finding diff --git a/docs/future/direction-and-configuration-surface.md b/docs/future/direction-and-configuration-surface.md index 327f97f8..512824a6 100644 --- a/docs/future/direction-and-configuration-surface.md +++ b/docs/future/direction-and-configuration-surface.md @@ -5,8 +5,8 @@ > Date: 2026-08-27. > Companions: [`config-surface-for-a-structured-repository.md`](config-surface-for-a-structured-repository.md) > (the field-level review this extends), -> [`../design/gittarget-layout-model.md`](../design/gittarget-layout-model.md) and -> [`../design/gittarget-api-wave.md`](../design/gittarget-api-wave.md) (the API work this +> [`../layout/model.md`](../layout/model.md) and +> [`../layout/api-wave.md`](../layout/api-wave.md) (the API work this > sequences), and > [`../design/support-boundary/helm-light-support-boundary.md`](../design/support-boundary/helm-light-support-boundary.md) > (the Helm option this deliberately parks). @@ -139,16 +139,15 @@ content. No new source-side API is needed for either direction. The rules and the layout model meet at `layout.scope`, and the meeting produces two validations worth declaring: -- **`scope: SingleNamespace` requires `allowedSourceNamespaces` to be an exact one-name - list — and that name *is* the single namespace.** The question "which namespace is the - single one?" must not be answered by the rules: N rules do not own the folder, and the - first-writer-wins alternative is exactly the silent re-deciding the layout model - exists to forbid. So the identity lives on the GitTarget, as the authorization bound - collapsing into a structural fact: one exact name, no selector (a label selector cannot - guarantee singularity), and CEL can check it at admission. Rules then merely subscribe - within it — an omitted `sourceNamespace`, an explicit match, or `"*"` all resolve to - that one namespace, and a rule naming any other namespace refuses loudly under the - existing bilateral check. +- **`scope: SingleNamespace` requires `layout.namespace`, plus an exact matching one-name + `allowedSourceNamespaces.names` list.** The question "which namespace is the single one?" + must not be answered by the rules: N rules do not own the folder, and the first-writer-wins + alternative is exactly the silent re-deciding the layout model exists to forbid. The identity + therefore lives on the GitTarget as a structural layout field, while the authorization bound + stays an authorization bound. A selector cannot guarantee singularity, and CEL can check the + equality at admission. Rules then merely subscribe within it: an omitted `sourceNamespace`, an + explicit match, or `"*"` all resolve to that one namespace, and a rule naming any other namespace + refuses loudly under the existing bilateral check. - **A `ClusterWatchRule` referencing a `scope: SingleNamespace` target is refused, at the rule, by name.** The payoff of `SingleNamespace` plus `writeNamespace: Never` is a *portable* folder — deployable into any namespace at apply time. Cluster-scoped content @@ -186,12 +185,14 @@ spec: mode: Write commitWindow: 30s # moved from GitProvider: batching describes this folder allowedSourceNamespaces: - names: [shop] # under SingleNamespace: exactly one name — this IS the namespace + names: [shop] # authorizes capture from shop layout: kind: Kustomize - create: true # an empty repo becomes a buildable folder scope: SingleNamespace + namespace: shop writeNamespace: Never # the artifact is environment-agnostic; namespace at deploy + kustomize: + create: true # an empty repo becomes a buildable folder ``` and the folder is inert until the tenant subscribes content to it: @@ -211,8 +212,8 @@ spec: ``` Everything direction B needs is here and nowhere else: the folder is described on the -GitTarget, the single namespace is *named* on the GitTarget (the one-name -`allowedSourceNamespaces` list is what `scope: SingleNamespace` requires), the WatchRule +GitTarget, the single namespace is *named* on the GitTarget in `layout.namespace` and authorized by +the matching one-name `allowedSourceNamespaces` list, the WatchRule is the artifact's manifest, structure can be brought into existence, and the artifact's shape is a declared fact a reviewer can read. A `ClusterWatchRule` pointed at this target is refused by name: cluster-scoped content would break the portability that diff --git a/docs/future/flux-maintainer-review-status-and-config-model.md b/docs/future/flux-maintainer-review-status-and-config-model.md deleted file mode 100644 index 066b2ecb..00000000 --- a/docs/future/flux-maintainer-review-status-and-config-model.md +++ /dev/null @@ -1,731 +0,0 @@ -# Review: the configuration model and status implementation, read as a Flux maintainer - -> Status: external review — findings open, nothing here binds until scheduled. -> Date: 2026-07-21 -> Reviewed at: branch `feat/gittarget-prune-mode-pr5`, commit `f37a7ba`. -> -> **Done so far.** F12's *enum casing* (`PruneMode` is now `Never`/`OnEvent`/`Always`, taken before -> the release because it was the last moment it was free), and then the whole of §4's -> **"Before the next release"** and **"Next"** blocks: **F1, F2, F3, F5, F7, F8, F11** and the rest -> of **F12**, shipped together on `feat/flux-status-contract`. See §6 for exactly what landed and -> what each fix looks like now. -> -> **Still open:** **F4** (abnormal-true polarity — resolved the other way, deliberately; see §6), -> **F6** (`spec.suspend`, `spec.interval`, reconcile-request annotation), **F9** (the stored -> `scope: Namespaced` status-write question), **F10** (CommitRequest lifecycle). Those are §4's -> "Then (API surface)" block and are a separate change. -> -> **That change now has a shape.** F6, F10, F12's reference-type nit and §3's pushbacks are sequenced -> together with the new-file-placement redesign in -> [`gittarget-api-wave.md`](../design/gittarget-api-wave.md), on the principle that the folder is -> described on the GitTarget and the connection describes only the connection. Two of this review's -> findings change that design rather than accompanying it: `spec.mode: Observe` (config-surface B1) -> becomes the way a layout is adopted safely, and `spec.interval` is what keeps the layout's -> scan-derived status fresh. F9 is deliberately kept **outside** the wave, because its answer -> constrains the enum work and should be known before anything is planned around it. -> Stance: reviewed as if this API were proposed for the GitOps Toolkit, with Flux's own -> source (`external-sources/flux/`) and kstatus (`sigs.k8s.io/cli-utils/pkg/kstatus`) as ground -> truth rather than recollection. - -## What was read - -- The API surface: `api/v1alpha3/*.go` (all six kinds plus `NamespaceMatcher`, `PrunePolicy`). -- The status implementations: - `internal/controller/{gittarget,watchrule,clusterwatchrule,gitprovider,clusterprovider,commitrequest}_controller.go`, - `condition_helper.go`, `stream_status.go`, `gittarget_dependency_status.go`, - `gittarget_source_cluster.go`, `internal/watch/stream_readiness.go`. -- `docs/configuration.md`, `docs/spec/status-conditions-guide.md`, `docs/spec/where-validation-lives.md`, `docs/design/reconcile-triggering.md`. -- Flux as ground truth: `external-sources/flux/pkg/apis/meta`, - `external-sources/flux/pkg/runtime/{conditions,patch}`, `external-sources/flux/pkg/apis/acl`, - `external-sources/flux/flux2/rfcs/`, `external-sources/flux/flux-operator/api/v1`. -- kstatus itself: `sigs.k8s.io/cli-utils/pkg/kstatus/status` from the module cache. - -Read-only review: no builds, no tests, no edits to the tree. - ---- - -## 1. The standard - -It is **kstatus** — `sigs.k8s.io/cli-utils/pkg/kstatus`, from Kubernetes SIG-CLI. It is what -`kubectl apply --wait`, kpt, Config Sync, Argo's health model (in spirit), and the Flux CLI all -lean on. Its upstream is the **Kubernetes API conventions**, section *"typical status properties"*, -which defines the abnormal-true polarity rule. Flux codifies both in -`fluxcd/pkg/apis/meta` — `ReadyCondition` / `StalledCondition` / `ReconcilingCondition` -(`external-sources/flux/pkg/apis/meta/conditions.go:43-59`) — and the flux-operator boils -readiness down to one CEL expression -(`external-sources/flux/flux-operator/api/v1/common_types.go:23`): - -```text -status.conditions.filter(c, c.type == 'Ready').all(c, c.status == 'True' && c.observedGeneration == metadata.generation) -``` - -**The precise rules kstatus enforces** (verified by reading -`cli-utils/pkg/kstatus/status/generic.go`, not from memory): - -1. `metadata.deletionTimestamp` set → `Terminating`. -2. `status.observedGeneration != metadata.generation` → `InProgress`. -3. any condition `Reconciling` **with status True** → `InProgress`. -4. any condition `Stalled` **with status True** → `Failed`. -5. otherwise → `Current`. - -Two consequences that matter enormously here, and that are easy to get wrong: - -- kstatus **never reads `Ready`** for a custom resource. `Ready=False` with `Reconciling` and - `Stalled` both False reads as **`Current`** — "done, healthy". The `Ready` condition is for - `kubectl wait` and humans; the trio is for machines. They must never disagree. -- kstatus only reacts to those conditions **when True**. That is why the convention says they - MUST NOT be present when False. - -### Verdict on compatibility - -**This project is on the standard, deliberately and knowledgeably — closer than most projects at -this stage.** `docs/spec/status-conditions-guide.md` states the contract correctly, and -`internal/controller/gittarget_kstatus_test.go` asserts it against the *real* kstatus library -rather than a hand-rolled reimplementation. That is mergeable work. - -What is not yet mergeable is the gap between that stated contract and what the reconcilers -actually emit. Three defects (F1–F3) make the trio lie in states production will reach, and the -tests do not catch them because they assert hand-built condition sets rather than the output of a -reconcile. - -| Area | Grade | Note | -|---|---|---| -| Condition types (`Ready`/`Reconciling`/`Stalled`) | **A** | Correct vocabulary, correct meanings, documented. | -| `observedGeneration` on object **and** per-condition | **A** | Passes the flux-operator CEL health expr as written. | -| `+listType=map` / `+listMapKey=type` on conditions | **A** | Correct on all six kinds. SSA-safe. | -| kstatus **conformance tests** | **A–** | Real library, real `Compute()`. Fixtures are synthetic (see F1/F2). | -| kstatus **behaviour under real reconciles** | **D** | F1 masks `Failed`; F2 never reaches `Current`. | -| Abnormal-true polarity (MUST NOT be present when False) | **C** | Always written as `False`. Tolerated by kstatus, violates the contract. | -| Status write discipline (no-op suppression) | **D** | Unconditional writes + always-moving timestamps (F3). | -| Reason vocabulary | **C** | Ad-hoc; `Reason == Type` in several places. | -| Flux object contract (`suspend`, `interval`, `requestedAt`, Events) — see layer 3 below | **F** | None of it present. Known — `docs/design/reconcile-triggering.md`. | -| `no phase string` discipline | **A** | Conditions only, everywhere. Correct. | - -### Three different things get conflated below — only one of them is a standard - -This review keeps saying "the GitOps Toolkit object contract". That is shorthand, not an official -name, and it is worth separating the three layers because they carry very different weight. - -**1. The Kubernetes API conventions — normative, upstream.** -`kubernetes/community`, `contributors/devel/sig-architecture/api-conventions.md`, section *"typical -status properties"*. This is where abnormal-true polarity comes from, where "conditions are a map -keyed by type" comes from, and where the UpperCamelCase enum rule (F12) comes from. Anything that -calls itself a Kubernetes API is measured against this. Flux's own condition docs link to it by -URL (`external-sources/flux/pkg/apis/meta/conditions.go:49-50, 57-58`). - -**2. kstatus — a real, named, vendor-neutral convention.** -`sigs.k8s.io/cli-utils/pkg/kstatus`, owned by Kubernetes SIG-CLI. It is a *library plus a -convention*: implement the trio the way it expects and any kstatus consumer can compute your -object's status without knowing what your CRD is. Consumers include cli-utils' own applier, -`kpt live apply`, and the Config Sync / Nomos family built on cli-utils. Flux designs to it -explicitly and says so in the comment above its condition constants. - -Worth knowing what it is *not*: **Argo CD does not use kstatus.** Its health model -(`gitops-engine`, built-in checks plus Lua) is a separate, older implementation of the same idea. -So conforming here buys interoperability with the Flux/kpt/cli-utils side of the ecosystem, not -with Argo. That is still the larger side for anything condition-driven, and it is the side this -project's users already live on. - -**3. The Flux object contract — not a published cross-vendor standard, but real, importable code.** -There is no RFC and no spec document for it. What exists instead is -**`github.com/fluxcd/pkg/apis/meta`** — its own Go module, Apache-2.0, versioned independently of -the controllers, which this repo *already pins at `v1.31.0`* (`go.mod:9`, for -`KubeConfigReference`). It is written in RFC-2119 language and is deliberately the shared artifact -rather than a doc: - -| Codified in `pkg/apis/meta` | Where | -|---|---| -| `ReadyCondition` / `StalledCondition` / `ReconcilingCondition` / `HealthyCondition` | `conditions.go:43-64` | -| Generic reasons: `Succeeded`, `Failed`, `Progressing`, `ProgressingWithRetry`, `Suspended`, `DependencyNotReady`, `InvalidPath`, `InvalidURL` | `conditions.go:80-115` | -| `ReconcileRequestAnnotation = "reconcile.fluxcd.io/requestedAt"` — *"any change in value SHOULD trigger a reconciliation"* | `annotations.go:23` | -| `ForceRequestAnnotation = "reconcile.fluxcd.io/forceAt"` — explicitly *"used to standardize the mechanism across controllers"* | `annotations.go:32` | -| `ReconcileRequestStatus` — an **embeddable struct** carrying `lastHandledReconcileAt`, plus `StatusWithHandledReconcileRequest` / `ObjectWithAnnotationRequests` interfaces | `annotations.go:49-133` | -| `LocalObjectReference`, `NamespacedObjectReference`, `NamespacedObjectKindReference`, `SecretKeyReference`, `KubeConfigReference` | `reference_types.go` | -| `AccessDeniedCondition` / `AccessDeniedReason`, `AccessFrom` | sibling module `pkg/apis/acl` | - -**What is deliberately *not* in that module: `spec.suspend` and `spec.interval`.** `meta` ships -`SuspendedReason` but no `Suspend` field — the fields themselves are hand-repeated in every fluxcd -controller's own API types. So "adopt the contract" means *import the type* for conditions, -reasons, annotations and references, but *copy the field shape* for suspend/interval. F6 is -therefore a convention-matching argument, not a dependency argument. - -**Who adheres.** Verifiable from this checkout: every fluxcd controller (source-, kustomize-, -helm-, notification-, image-\*) via `flux2`; and ControlPlane's **flux-operator** — a different -vendor under a different licence (AGPL) — which adheres and then extends it with its own -`FluxObject` interface and a CEL readiness expression -(`external-sources/flux/flux-operator/api/v1/common_types.go:23, 44-64`). That second one is the -interesting data point: an independent project chose to implement the same contract rather than -invent one, because it is what makes `flux`-shaped tooling work against non-Flux kinds. - -**Why this matters for F6/F7 specifically.** The argument is interop, not compliance. Nobody will -fail an audit for lacking `spec.suspend`. But a platform team that already runs Flux has muscle -memory — `suspend` to pause, annotate `requestedAt` to force, `kubectl describe` to see what -happened, notification-controller to route failures — and every one of those reflexes currently -returns nothing here. Meanwhile the marginal cost is close to zero: the module is already a -dependency, so adopting the condition types, reason constants and `ReconcileRequestStatus` is an -import and a struct embed, not a new supply-chain decision. - ---- - -## 2. Findings - -### F1 — `downgradeReady` erases a terminal `Stalled`, downgrading kstatus `Failed` → `InProgress` (High) - -`internal/controller/gittarget_controller.go:240` runs `applyDataPlaneConditions`, which for a -refused Git path sets the correct terminal trio (`Ready=False`, `Reconciling=False`, -`Stalled=True`, reason `UnsupportedContent`) — `gittarget_controller.go:526-543`. - -Then `internal/controller/gittarget_controller.go:258` runs `projectSourceAndProvider`, which on -any source/provider imperfection calls `downgradeReady` -(`internal/controller/gittarget_source_cluster.go:224-233`): - -```go -r.setCondition(target, GitTargetConditionReady, readyStatus, reason, message) -r.setCondition(target, GitTargetConditionReconciling, metav1.ConditionTrue, reason, message) -r.setCondition(target, GitTargetConditionStalled, metav1.ConditionFalse, ReasonProgressing, ...) -``` - -It unconditionally stamps `Stalled=False, Reconciling=True`, wiping the stall set 18 lines -earlier. The doc comment says it "only ever DOWNGRADES Ready" — true of `Ready`, but for -**kstatus** it *upgrades* the object from `Failed` to `InProgress`. - -**Trigger, most likely path:** a remote-source GitTarget before first discovery has -`SourceClusterReachable=Unknown` (`gittarget_controller.go:249-255`), which hits the -`reachStatus == metav1.ConditionUnknown` branch (`gittarget_source_cluster.go:212`). Any -GitProvider blip does it too. So: a GitTarget with unsupported kustomize content in its folder, or -`RenderMatchesLive=False`, reports "still working on it, please wait" forever to every kstatus -consumer, instead of failing fast. Nothing in the folder is being written, and nothing will be. -`GitPathAccepted=False` is still there for a human who goes looking; `kubectl wait` and any -kstatus-driven CI gate hangs to timeout. - -This is exactly why Flux computes the summary **once, at the end, from a declared precedence -order** rather than by successive mutation — the `summarize` pattern over `conditions.Set` with -`patch.WithOwnedConditions` (`external-sources/flux/pkg/runtime/patch/options.go:67-72`). - -**Fix.** Collect `(status, reason, message)` candidates from every gate into a list, then derive -the trio once at the end with stated precedence: `Stalled=True` wins over `Reconciling=True` wins -over `Ready=True`. `docs/spec/status-conditions-guide.md:69-73` already states the canonical -reads — make one function the single writer of the trio, and make every gate a *contributor*, not -a *setter*. Add a table-driven test that runs the actual reconcile and feeds the resulting object -to `kstatus.Compute`, including the "path refused **and** provider unready" cell. - ---- - -### F2 — A GitTarget with no WatchRules is never `Current`, and requeues every 10 s forever (High) - -`internal/watch/stream_readiness.go:75-77`: - -```go -func (s StreamSummary) StreamsRunning() bool { return s.Total > 0 && s.Ready == s.Total } -``` - -Zero tracked types → `StreamsRunning()==false` → the `!streams.StreamsRunning()` branch at -`gittarget_controller.go:562` → `Ready=False`, `Reconciling=True`, reason `NoResolvedTypes`. -Permanently, because nothing will ever resolve. And because `streamsSettling` is true -(`gittarget_controller.go:232`), the reconcile returns `RequeueAfter: RequeueStreamSettleInterval` -= **10 s** (`constants.go:117`), forever. - -That state is not exotic — it is **step 3 of the documented setup flow** -(`docs/configuration.md:29-34`: create GitTarget, *then* create WatchRules), and it is the steady -state of any target whose rules were deleted. A user following the docs has an object that -`kubectl wait --for=condition=Ready` never returns on, and that burns a reconcile plus a status -write every 10 seconds indefinitely. - -Empty is not in-progress. Nothing is pending. "I have nothing to mirror" is a **converged** -state — Flux's Kustomization with an empty path is `Ready=True` with a "no objects" message, not -InProgress. - -**Fix.** Split "nothing resolved" from "resolving". `Total == 0` should be `Ready=True`, -`Reconciling=False`, `Stalled=False` with reason `NoWatchRules` (or `NoResolvedTypes`) and a -message saying so — kstatus `Current`, honest, and it drops back to the 5 min cadence. Keep -`status.streams.summary: "0/0"` so the zero stays visible. If that feels like hiding a -misconfiguration, that is what an Event or a `Ready` *reason* is for, not a permanent -`Reconciling=True`. - ---- - -### F3 — Always-moving timestamps and no self-predicate create a self-triggering reconcile edge (High) - -Three things compose badly. - -1. **Every reconcile stamps a fresh timestamp.** - `gittarget_controller.go:129` — `target.Status.LastReconcileTime = metav1.Now()`, unconditional. - `internal/watch/stream_readiness.go:244` — `StreamSummary{..., ObservedTime: metav1.Now()}`, - surfaced into `status.streams.observedTime` for GitTarget *and* both rule kinds - (`gittarget_controller.go:1065-1078`, `stream_status.go:26-40`). - -2. **The status write is unconditional and full-object.** `updateStatusWithRetry` - (`gittarget_controller.go:1081-1113`, and the near-identical copies in - `watchrule_controller.go:405`, `clusterprovider_controller.go:283`, - `gitprovider_controller.go:403`) does `latest.Status = target.Status; r.Status().Update(...)`. - There is no "did anything change?" check — and the timestamps guarantee something always did. - -3. **`For()` carries no predicate on GitTarget, WatchRule, or ClusterWatchRule** - (`gittarget_controller.go:1117-1118`, `watchrule_controller.go:463-464`, - `clusterwatchrule_controller.go:547`). Status-subresource writes bump `resourceVersion` and - fire an Update watch event, which `handler.EnqueueRequestForObject` puts straight back on the - queue — un-rate-limited. - -So each reconcile enqueues itself. It is not an unbounded spin — `metav1.Time` serialises at -RFC3339 second precision, so a follow-up reconcile that lands inside the same wall-clock second -produces a byte-identical status and the apiserver no-ops it. The practical shape is therefore: -**every reconcile costs roughly two reconciles and at least one etcd write, degenerating into a -sustained self-sustaining loop whenever a reconcile takes ≥1 s** — plausible here, since -`checkForConflicts` does a cluster-wide `List` of every GitTarget on every pass -(`gittarget_controller.go:793`) on top of several `Get`s and `DeclareForGitTarget`. - -Combine with F2 and an idle GitTarget writes to etcd at ~0.1 Hz forever, per object, and wakes -every controller watching GitTargets each time. - -`GitProvider` and `ClusterProvider` **do** have self-predicates -(`gitprovider_controller.go:462-465`, `clusterprovider_controller.go:332-335`), so this is an -inconsistency, not a house style. `docs/design/reconcile-triggering.md:44-50` inventories -per-controller predicates but records only the *dependency* edges — the missing `For()` predicate -on GitTarget/WatchRule is not in that table. - -**Fix, in order of value:** - -- Adopt `fluxcd/pkg/runtime/patch.Helper`. It computes a merge patch of *what actually changed* - and sets `observedGeneration` "only if there is a change" - (`external-sources/flux/pkg/runtime/patch/options.go:33-35`). No change → no request → no watch - event → no loop. A drop-in: `fluxcd/pkg/apis/meta` is already a dependency. -- Failing that: `if !equality.Semantic.DeepEqual(latest.Status, target.Status) { update }`, and - drop `LastReconcileTime` / `observedTime` from the comparison — or drop the fields. -- Ask whether `status.lastReconcileTime` and `status.streams.observedTime` earn their keep at all. - Flux deliberately does not carry a "last reconcile attempt" timestamp; a condition's - `lastTransitionTime` plus `controller_runtime_reconcile_total` answer the same question without - making every object mutable-on-read. `LastPushTime` is genuinely useful (it records a real - event) — keep that one. -- Add a `For()` predicate on the three controllers that lack one, e.g. - `predicate.Or(GenerationChangedPredicate{}, )`. - ---- - -### F4 — Abnormal-true polarity: `Reconciling`/`Stalled` are always present, including when False (Medium) - -`external-sources/flux/pkg/apis/meta/conditions.go:47-59` states it twice: - -> The Condition adheres to an "abnormal-true" polarity pattern, and **MUST only be present on the -> resource if the Condition is True**. - -Every write path here emits both unconditionally: `setStalledConditions` -(`gittarget_controller.go:450-457`), `downgradeReady`, `setRuleProgressing` -(`stream_status.go:114-123`), `setReadyConditions` / `setProgressingConditions` on both providers. - -kstatus tolerates it — it only tests for `== True` — so this is not a correctness bug. But it is a -contract violation with real costs: - -- `kubectl get gittarget -o yaml` carries six condition entries where three would do. -- Any tool that treats *presence* as signal (a fair reading of the convention) misreads it. -- `Reconciling=False, reason=UnsupportedContent, message="Reconciliation is stalled"` - (`gittarget_controller.go:538-539`) is a condition that says nothing true about reconciling. It - exists only to be overwritten. - -`docs/spec/status-conditions-guide.md:31-32` names the polarity rule correctly and the code does -the opposite. Pick one. Preferred: `DeleteCondition` on the way to False — -`conditions.Delete(obj, meta.StalledCondition)` in Flux terms — and let `Ready` carry the positive -summary. - ---- - -### F5 — `upsertCondition` reorders the condition list on every touch (Medium) - -`internal/controller/condition_helper.go:26-44` rebuilds the slice with the target type *removed* -and then appends it at the end. Touch `Ready` and it migrates to the tail; touch it again next -pass and everything else has shuffled. - -Effects: gratuitous diffs in `kubectl get -o yaml` and in any GitOps repo that mirrors these -objects (which, given what this product does, is not hypothetical — a GitTarget mirroring -GitTargets would commit condition-reordering noise); a byte-level change even when nothing -semantically changed, which feeds F3; and unstable ordering for humans. - -The `LastTransitionTime` handling is *correct* — preserved when `Status` is unchanged -(`condition_helper.go:39-41`), matching `apimeta.SetStatusCondition` and the API conventions. -Flux is actually stricter than the convention here, resetting on Reason/Message change too -(`external-sources/flux/pkg/runtime/conditions/setter.go:44-46`); this repo's is the more -conventional reading and worth keeping. - -**Fix.** Use `k8s.io/apimachinery/pkg/api/meta.SetStatusCondition`, which updates in place. Or -adopt Flux's approach and sort deterministically, with `Stalled`, `Reconciling`, `Ready` weighted -to the front for `kubectl` legibility -(`external-sources/flux/pkg/runtime/conditions/setter.go:89-92, 196-218`) — a nice touch worth -stealing regardless. - ---- - -### F6 — No `spec.suspend`, no `spec.interval`, no reconcile-request annotation (Medium) - -Nothing in `api/v1alpha3` has `suspend` or `interval` — zero hits. The cadence is a compile-time -constant: `RequeueSteadyInterval = 5 * time.Minute` (`constants.go:113`). - -Every Flux object implements all three -(`external-sources/flux/flux-operator/api/v1/common_types.go:44-64`): `GetInterval()`, -`IsDisabled()`, `SetLastHandledReconcileAt()`. They are not decoration: - -- **`spec.suspend`** is the only way to say "stop touching this while I fix the repo by hand" - without deleting the object. For a controller that *writes to a Git repository*, the absence of - a pause button is the single most surprising gap in this API. Today the only way to stop a - GitTarget writing is to delete it or its rules — and per the `GitTargetSpec` doc comment, that - is irreversible for path/branch/provider. -- **`spec.interval`** — `GitProvider` does a real network `ls-remote` against the git host on - every pass (`gitprovider_controller.go:220`, `checkRemoteConnectivity`), hardcoded at 5 min, - with no jitter. N providers against github.com from one operator, all re-synchronised into a - thundering herd after each restart. Flux ships `fluxcd/pkg/runtime/jitter` precisely for this. - At minimum: jitter the requeue. Better: `spec.interval` per object, since a GitProvider pointing - at a rate-limited enterprise host and one pointing at a local Gitea do not deserve the same - cadence. -- **`reconcile./requestedAt` + `status.lastHandledReconcileAt`** is the universal - "reconcile now" idiom (`flux reconcile`, `kubectl annotate`, webhook receivers). This is the one - piece of F6 that is *shared code* rather than a copied field shape: embed - `meta.ReconcileRequestStatus` and call `meta.ReconcileAnnotationValue` - (`external-sources/flux/pkg/apis/meta/annotations.go:23-64`) and the semantics — including the - "any change in value SHOULD trigger" token comparison — come with it. Already identified as F1 - in `docs/design/reconcile-triggering.md`; still unbuilt. - ---- - -### F7 — Zero Kubernetes Events (Medium) - -There is no `EventRecorder` anywhere in `internal/controller` or `cmd/main.go` — no `Recorder`, -`Eventf`, or `Event(` call sites. - -Consequences: `kubectl describe gittarget` shows no history; a transient push failure that -resolves before anyone looks is invisible; and there is **no integration path with -notification-controller** or any Event-driven alerting, because there is no Event to route. -Metrics say a counter moved; they cannot say *which* GitTarget failed to push and why. - -`docs/design/reconcile-triggering.md:222-227` already prescribes the fix ("F4. Conditions **and** -Events, every loop") and cites `fluxcd/pkg/runtime/events`. This ranks above the webhook-receiver -work in the same doc: Events are cheap, and a controller that writes to Git without emitting an -Event on write failure is hard to operate. - ---- - -### F8 — Reason vocabulary is ad-hoc, and in several places `Reason == Type` (Medium) - -`gitprovider_controller.go:325`, `clusterprovider_controller.go:226`, `stream_status.go:45`: - -```go -r.setCondition(gitProvider, ConditionTypeReady, metav1.ConditionTrue, ConditionTypeReady, message) -// ^^ type ^^ reason == "Ready" -const ruleReadyReason = "Ready" -``` - -`Ready=True, reason=Ready` conveys nothing. A reason answers *why*. Flux's generic set -(`external-sources/flux/pkg/apis/meta/conditions.go:80-115`) is `Succeeded`, `Failed`, -`Progressing`, `ProgressingWithRetry`, `Suspended`, `DependencyNotReady`, `InvalidPath`, -`InvalidURL`, `AccessDenied` (the last from `pkg/apis/acl`) — and it is a *shared vocabulary*, so -one alerting rule works across every kind. - -Here it is `OK`, `Ready`, `Checking`, `Resolved`, `Progressing`, `Stalled`, `Validated` scattered -across `constants.go:95-145` and four controllers. `Progressing` and `Stalled` already match Flux. -Suggested: - -- Alias the generic ones to `meta.SucceededReason` / `meta.ProgressingReason` / - `meta.FailedReason` / `meta.DependencyNotReadyReason` — `github.com/fluxcd/pkg/apis/meta` is - already imported (`clusterprovider_types.go:6`), so it is free. -- Replace `OK` and reason-equals-type with `Succeeded`. -- Keep the excellent domain-specific reasons (`UnsupportedContent`, `IgnoreShadowsManagedPath`, - `WriteBoundaryRefused`, `NoAdmittedSourceNamespaces`) — those are exactly what the Flux docs - mean by "declaration of domain common Condition reasons in the API specification is - RECOMMENDED". Consider promoting them from `internal/controller` constants to exported constants - in `api/v1alpha3`, so consumers can compile against them. - -`GitTargetReasonProviderNotFound` should probably be `meta.DependencyNotReadyReason` or at least -`DependencyNotFound`, for the same cross-kind-alerting reason. - ---- - -### F9 — A stored `ClusterWatchRule` with `scope: Namespaced` may be unable to report its own refusal (Medium — needs verification) - -`api/v1alpha3/clusterwatchrule_types.go:130-133` narrows the enum to `Cluster` only, deliberately -keeping the field so a re-apply *fails*. The reasoning in the comment above it is sound. -`DeclaresNamespacedScope()` (`:144`) then refuses a stored value at compile time. - -The concern: for CRDs, the apiserver validates the **whole object** against the OpenAPI schema on -**status-subresource** updates too, not just spec updates. If that holds here, the controller -cannot write `Stalled=True` onto an object whose stored `spec.rules[].scope` is `Namespaced` — the -status update is rejected 422, and the one object that most needs to explain itself is the one -that cannot. - -**Mitigating factor:** CRD Validation Ratcheting (beta and default-on in 1.30, GA in 1.33) skips -re-validation of *unchanged* fields, which would make this a non-issue on modern clusters. So the -exposure is older clusters, or a cluster with the feature gate off. - -Not confirmed by execution. **Worth one envtest:** create a ClusterWatchRule with -`scope: Namespaced` via a client that bypasses the enum (or against an older CRD), then attempt a -status update, on the minimum supported Kubernetes version. If it fails, the fallback is to widen -the enum back and rely solely on the compile-path refusal plus a loud `Stalled` condition — -refusing at admission is nice, but not at the cost of being unable to report the refusal. - ---- - -### F10 — CommitRequest objects accumulate forever, and the controller cannot delete them (Medium) - -`CommitRequest` is a one-shot imperative object with an immutable spec -(`commitrequest_types.go:14`). Every "save now" leaves an object in etcd. There is no TTL, no -`ownerReference`, no GC path — and the reconciler's RBAC is -`commitrequests, verbs=get;list;watch` (`commitrequest_controller.go:108`), so it could not delete -them even if it wanted to. - -A team using this as an interactive save button generates hundreds per namespace per week. -Nothing reaps them. - -The broader Flux-maintainer objection is that **this should be an annotation, not a kind**. -`reconcile./requestedAt` is the established idiom for "act now", it is free to issue, it -self-GCs by being overwritten, and it needs no CRD. The counter-argument here is accepted: a -CommitRequest carries a verbatim commit message, a collect-delay, and (via the admission webhook) -the submitter's identity, and it reports back a SHA. That is genuinely more than a trigger, and -identity capture at admission is the textbook justification made well in -`docs/spec/where-validation-lives.md:50-56`. - -But if it stays a kind, it needs a lifecycle: - -- `spec.ttlSecondsAfterFinished` (the Job precedent) or a controller-side "delete terminal - requests older than N", plus the `delete` verb. -- Or an `ownerReference` to the GitTarget so it is at least cascade-deleted. -- At minimum, document the retention expectation and ship a - `kubectl delete commitrequests --field-selector` recipe. - ---- - -### F11 — `observedGeneration` can record a generation that was never observed (Low) - -`updateStatusWithRetry` re-`Get`s the object and then does `latest.Status = target.Status` -(`gittarget_controller.go:1102`). `target.Status.ObservedGeneration` was set from the generation -read at the *top* of the reconcile (`:128`). If the spec changed in between, the new object is -stamped with an `observedGeneration` equal to a generation never actually processed — kstatus then -reports `Current` for a spec nobody looked at, until the next pass corrects it. - -Narrow window, self-correcting, low severity. Flux avoids it structurally by patching with -optimistic concurrency and setting `observedGeneration` from the object being patched -(`patch.WithStatusObservedGeneration`). Another thing that comes free with the patch helper (F3). - ---- - -### F12 — Small API-conventions nits (Low) - -- **Enum casing — DONE.** `PruneMode` values were `never` / `onEvent` / `always` - (`prune_policy.go:17-23`). Kubernetes API conventions call for UpperCamelCase enum values — - compare `imagePullPolicy: Always|Never|IfNotPresent`, `persistentVolumeReclaimPolicy: - Retain|Delete`. Flux is itself inconsistent here (`driftDetection.mode: enabled|warn|disabled`), - so this was in company, but `Never`/`OnEvent`/`Always` is the conventional spelling. This was the - moment to decide — the field was brand new and unreleased on this branch, and the branch already - carried two `feat(api)!` commits, so the rename rode along in a release that was breaking anyway; - after that release it would have been a second breaking change for a cosmetic gain. - **Resolved on `feat/gittarget-prune-mode-pr5`: the values are now `Never` / `OnEvent` / - `Always`.** The typed constants (`PruneNever`, `PruneOnEvent`, `PruneAlways`) are unchanged, so - no Go call site moved; the wire values, the CRD enum and default, and the docs did. - (Conversely, `OperationType`'s `CREATE`/`UPDATE`/`DELETE` **is** right, because it matches - `admissionregistration.k8s.io` `rules[].operations` verbatim.) -- **Duplicated representation.** `status.streams.summary` is `fmt.Sprintf("%d/%d", Ready, Total)` - (`stream_readiness.go:70-72`) over fields that are right there in the same struct. - `status-conditions-guide.md:37-38` says don't do this. It exists only to feed a printer column — - a legitimate reason, but worth naming as such in the field doc so the next reader doesn't - "clean it up". -- **Printer-column sprawl.** GitTarget declares 13 columns, 7 of them default-priority - (`gittarget_types.go:238-252`). `kubectl get gittargets` will wrap on any normal terminal. Flux - ships 3–4 (Age, Ready, Status). Push `Provider`, `Branch`, `Path` to `priority=1` and keep - `Ready`, `Reason`, `Streams`, `Age`. -- **Reference types.** Six near-identical shapes — `GitProviderReference`, - `ClusterProviderReference`, `LocalTargetReference`, `NamespacedTargetReference`, - `LocalSecretReference`, `KnownHostsReference` — while already depending on - `fluxcd/pkg/apis/meta`, which offers `LocalObjectReference`, `NamespacedObjectReference`, - `NamespacedObjectKindReference`, `SecretKeyReference`. Reusing `meta.KubeConfigReference` - (`clusterprovider_types.go:69`) clearly paid off. The `Group`+`Kind` enum-with-default pattern is - defensible (it documents what is accepted and leaves room to widen), but consider embedding - `meta.LocalObjectReference` for the name half so downstream Go consumers get interoperable types. -- **`metav1.ObjectMeta` json tags are inconsistent** — `omitempty,omitzero` on GitTarget / - GitProvider / ClusterProvider / CommitRequest, plain `omitempty` on WatchRule - (`watchrule_types.go:277`) and ClusterWatchRule (`clusterwatchrule_types.go:203`). -- **`GitProviderStatus.Conditions` lacks `+patchStrategy=merge` / `+patchMergeKey=type`** - (`gitprovider_types.go:143-146`) while the other five kinds have them. Harmless with - `listType=map`, but inconsistent. - ---- - -## 3. The configuration model itself - -Setting status aside — this is the part worth defending as-is, and that deserves saying before the -criticism. - -**What is genuinely strong:** - -- **`docs/spec/where-validation-lives.md` is better than what Flux has written down.** The - schema → CEL → reconciler ladder, and specifically the argument that *reconcile-time is the - stronger gate because admission cannot see a policy tightened after creation*, is correct and is - the thing most projects get backwards. Flux arrived at the same place empirically; here it is a - stated rule. The one webhook shipped is justified on exactly the right grounds (identity exists - only in the `AdmissionRequest`). -- **`NamespaceMatcher`'s absent-vs-declared-vs-empty trichotomy** (`namespace_matcher.go:14-34, - 106-123`). Flux's `acl.AccessFrom` (`external-sources/flux/pkg/apis/acl/`) is a flat - `namespaceSelectors` list with no way to express "declared and empty", and Flux has been bitten - by exactly that ambiguity. The `selector: {}` = everything / `{}` = nothing / absent = legacy - distinction, anchored to `LabelSelectorAsSelector`'s own `Nothing()`/`Everything()` asymmetry, is - more precise than the incumbent. Rejecting `names: ["*"]` because Kubernetes treats it as a - literal name is the kind of detail that only comes from having been burned. -- **The two-key delegation for cross-namespace source watching** — - `ClusterProvider.spec.allowSourceNamespaceOverride` (platform admin) AND - `GitTarget.spec.allowedSourceNamespaces` (destination owner) — is a materially better answer than - Flux's. RFC-0001 (`external-sources/flux/flux2/rfcs/0001-authorization/README.md:97-101`) - documents that Flux controllers simply **do not respect namespace isolation** when dereferencing - cross-namespace refs, a long-standing multi-tenancy sore spot. Deny-by-default and two-party is - the right shape. -- **Immutability where identity is at stake.** `providerRef`/`branch`/`path`/`clusterProviderRef` - immutable via CEL (`gittarget_types.go:45-53`), with the reasoning in the type doc: a folder's - meaning is constituted by those four. `spec.prune` deliberately mutable - (`gittarget_types.go:134-139`) because forcing a delete-and-recreate to re-enable convergence - would destroy the one thing that cannot be rebuilt. Well drawn. -- **The `prune.mode` design.** Separating "the source told me it was deleted" from "a snapshot - didn't mention it" is the correct decomposition, and `onEvent` as the effective default — - resolved in code via `EffectiveMode()` rather than relying on CRD defaulting, so a *stored* - pre-field object is also safe (`prune_policy.go:52-63`) — is careful in the right way. Flux's - `spec.prune` is a single bool and cannot express the middle mode. -- **`no phase string`, anywhere.** Six kinds, zero `.status.phase`. Rarer than it should be. - -**Where to push back on the model:** - -- **The missing pause button** (F6) is the biggest hole. This writes to Git. -- **`GitProvider` is doing three jobs**: remote+credentials (Flux: `GitRepository` + `Secret`), - commit identity/templates/signing (Flux: `ImageUpdateAutomation.spec.git.commit`), and push - batching policy. Defensible cohesion — they are all "how this repo gets written" — but note the - consequence: `spec.push.commitWindow` and `spec.commit.message.*` are properties of a *workload*, - yet they live on the *connection*, so two GitTargets sharing a repo cannot have different - batching or message templates. If that ever needs to differ per target, the field has to move, - and moving it is breaking. Worth writing down as a known constraint now. -- **The `GitProvider` (namespaced) / `ClusterProvider` (cluster) asymmetry** is justified well in - `docs/configuration.md:39-63` and the reasoning holds: a Git destination is a team's write - boundary, a source cluster is a shared physical identity. It *will* still surprise people, and - the doc already anticipates the follow-up ("if a platform later needs a shared, platform-owned - Git destination, that should be a separate cluster-scoped concept"). Keep that paragraph. -- **`clusterProviderRef` defaults to `{name: "default"}` for an object the operator never - creates.** A GitTarget applied to a fresh cluster is unready with `ClusterProviderNotFound` until - someone creates it. Deliberate and well defended (`docs/configuration.md:419-436`; the chart - renders it by default), and the substance is right — silently defaulting to in-cluster - credentials would bypass the authorization model. One ask: make the `ProviderNotFound` message - for the literal name `default` say *"the ClusterProvider named 'default' does not exist; the - operator never creates one — see `clusterProvider.createDefault` in the chart, or commit the - object"*. A generic "provider not found" for the **defaulted** value is the single most likely - first-run support ticket. -- **`ClusterWatchRule` is unbounded by `allowedSourceNamespaces` by construction** — correct - (cluster-scoped objects have no namespace) and clearly documented - (`clusterwatchrule_types.go:172-177`, `docs/configuration.md:935-938`). The mitigation "give each - tenant its own ClusterProvider and credential" is the right answer and matches how Flux tells - people to do hard multi-tenancy. Fine as-is; make sure the security model doc says it in one - place. -- **`sourceNamespace: "*"` fan-out.** One watch stream per (type × admitted namespace) - (`watchrule_types.go:132-134`), a cost flagged honestly in the type doc. On a broad policy across - a big cluster this is the scalability cliff — worth a hard cap with a `Stalled` reason - (`TooManyStreams`) rather than discovering it as apiserver watch pressure. - ---- - -## 4. Suggested order - -**Before the next release** (observable behaviour, cheap): - -1. **F1** — one function owns the trio; every gate contributes a candidate, precedence stated once. - Add a reconcile-output-driven `kstatus.Compute` test covering "refused path + unready provider". -2. **F2** — `Total == 0` is `Ready=True` / `Current`, not perpetual `Reconciling`. -3. **F3** — adopt `fluxcd/pkg/runtime/patch.Helper` (or a `DeepEqual` guard), drop or exclude the - always-moving timestamps, add `For()` predicates to the three controllers missing one. - -**Next** (contract alignment, low risk, high interop value): - -1. **F5** — `apimeta.SetStatusCondition`, or Flux's sorted `Set`. -2. **F8** — alias generic reasons to `fluxcd/pkg/apis/meta`; kill `reason == type`; export the - domain reasons from `api/v1alpha3`. -3. **F4** — delete `Reconciling`/`Stalled` rather than writing them False. -4. **F7** — wire an `EventRecorder`; emit on every terminal outcome and every push failure. - -**Then** (API surface — do the breaking ones while still `v1alpha3`). These are the wave, and they -are now sequenced with the layout model in -[`gittarget-api-wave.md`](../design/gittarget-api-wave.md) rather than scheduled from here: - -1. **F6** — `spec.suspend` on GitTarget/WatchRule/ClusterWatchRule/GitProvider; `spec.interval` on - GitProvider at minimum; jitter the requeue; `reconcile.configbutler.ai/requestedAt` + - `status.lastHandledReconcileAt`. -2. **F12** — ~~decide `PruneMode` casing **now**~~ (done, pre-release); trim printer columns; unify - ObjectMeta tags. -3. **F10** — CommitRequest lifecycle (TTL or ownerRef) and the `delete` verb. -4. ~~**F9**~~ — **not in this block.** F9 is deliberately kept OUTSIDE the wave, because its answer - constrains the enum work and should be known before anything is planned around it. Verify the - `scope: Namespaced` status-write path on the minimum supported Kubernetes version as its own - change, ahead of the wave. - ---- - -## 5. Bottom line - -The question asked was whether the status implementation is *compatible enough* with the open -standard. - -**The model is compatible. The implementation is compatible in three of the four states it can be -in.** The right conditions, the right per-condition `observedGeneration`, the right list semantics, -no phase string, a written contract, and conformance tests against the real kstatus library — more -than most projects have at v1alpha3, and the reason the two real defects are worth fixing rather -than redesigning around. F1 makes a `Failed` object look `InProgress`; F2 makes an idle object -never reach `Current`. Both are localized. Fix those and this passes a kstatus conformance review. - -The larger gap is not kstatus at all — it is the **Flux object contract** (§1, layer 3): `suspend`, -`interval`, reconcile-on-annotation, and Events. That one is not a formal standard and nobody fails -an audit for missing it; it is codified only as an importable Go module, -`github.com/fluxcd/pkg/apis/meta`, which this repo already depends on. But it is what makes an -object *operable* by the reflexes a platform team already has from Flux, and right now none of -those reflexes return anything here. Most of it is already designed in -`docs/design/reconcile-triggering.md`; it needs building. `spec.suspend` first, because this -controller writes to Git and there is currently no way to make it stop. - ---- - -## 6. What was built (2026-07-22, branch `feat/flux-status-contract`) - -The review's own ordering was followed: §4's "Before the next release" and "Next" blocks, plus the -cheap F12 nits. The API-surface block (F6, F9, F10) is deliberately left for a separate change, -because each of those adds or changes a spec field and deserves its own review. - -Two pieces of shared machinery carry most of it, and both replace code that had been copied five or -six times: - -**`internal/controller/readiness.go` — the accumulator that owns the trio.** Gates no longer set -`Ready`/`Reconciling`/`Stalled`; they *contribute* a verdict at one of three levels (converged, -progressing, stalled) and the trio is derived once at the end. Within a level the first contributor -wins, so the precedence is the order of a handful of adjacent calls in `gitTargetReadinessGates` / -`ruleReadiness` — readable in one place rather than emergent from which gate ran last. This is the -`summarize`-once shape the review pointed at, without taking `fluxcd/pkg/runtime` as a dependency. - -**`internal/controller/status.go` — the per-reconcile status session.** `beginStatus` captures the -object as read, `set`/`applyReadiness` collect what the reconcile wants to say, and `commit` writes -the difference exactly once. It replaced five near-identical `updateStatusWithRetry` copies and, with -them, three shared defects, plus six `setCondition`/`setTypedCondition` wrappers and eight -trio-setting helpers (`setStalledConditions` ×3, `setProgressingConditions`, `setReadyConditions`, -`setRuleStalled` ×2, `setRuleProgressing`, `downgradeReady`). - -| Finding | What landed | -|---|---| -| **F1** — `downgradeReady` erased a terminal `Stalled` | The accumulator. `downgradeReady` is gone; the source/provider projection now only *publishes* its three conditions and contributes progressing verdicts. The regression cell ("refused path **and** unready provider") is a table row in `gittarget_status_test.go` that runs the real `kstatus.Compute` over reconciler output. | -| **F2** — a GitTarget with no WatchRules never reached `Current` | `streamsAxis` reports `True` when `Ready == Total`, which is vacuously true at zero. `status.streams.summary` still reads `0/0` and the reason is still `NoResolvedTypes`, so the zero stays visible. "Data plane not wired" is now a *different* state (reason `Progressing`) so the two can never be confused. | -| **F3** — unconditional writes, moving timestamps, no self-predicate | All three legs. `commit` sends nothing when the status is unchanged; `status.lastReconcileTime` and `status.streams.observedTime` were removed rather than excluded from the comparison (they existed only to be overwritten); `GenerationChangedPredicate` added to the `For()` of GitTarget, WatchRule and ClusterWatchRule. | -| **F5** — `upsertCondition` reordered the list on every touch | Delegates to `apimeta.SetStatusCondition`, which updates in place. The test now pins position stability rather than de-duplication of an input `listType=map` makes impossible. | -| **F7** — zero Kubernetes Events | Every reconciler takes an `EventRecorder`; the shared writer emits one Event per **persisted** `Ready` transition (`Normal` on True, `Warning` otherwise). After the patch, not beside each `set`, so intermediate values a reconcile never stored are never announced. | -| **F8** — ad-hoc reasons, `Reason == Type` | Generic reasons alias `fluxcd/pkg/apis/meta`. `OK` and `Ready` as *reasons* became `Succeeded` across all six kinds and the e2e suite. Domain reasons kept. | -| **F11** — `observedGeneration` for a generation never observed | Comes free with the patch: the write uses optimistic concurrency, so a spec that moved under the reconcile loses the write instead of being mislabelled. The conflict is dropped rather than retried — the write that beat us already enqueued a fresh pass. | -| **F12** — API-conventions nits | GitTarget's default printer columns cut from seven to four (`Provider`/`Branch`/`Path` moved to `priority=1`); `patchStrategy`/`patchMergeKey` added to `GitProviderStatus.Conditions`; `ObjectMeta` json tags unified on `omitempty,omitzero`; `status.streams.summary`'s field doc now says *why* it duplicates the counts beside it. | - -### F4 was resolved the other way, on purpose - -The review is right that the code and `docs/spec/status-conditions-guide.md` disagreed, and right -that "pick one" was the answer. The one picked was **keep writing the pair when False**, and the -guide now says so and says why. - -kstatus tolerates it — it tests for `== True` and ignores everything else — so nothing downstream -misreads it. Against that: `kubectl wait --for=condition=Stalled=false` reads the explicit `False`, -this repo's e2e suite asserts it in several specs, and a condition that vanishes is harder for a -human to reason about than one that reads `False`. The cost of the deviation is three extra condition -entries in `-o yaml`; the cost of removing it is a real capability plus a suite-wide rewrite. What -*was* wrong and is now fixed is the incoherent message the review quoted -(`Reconciling=False, message="Reconciliation is stalled"` is at least true of the object, and it is -now written from one place rather than as leftover state from a gate that lost). - -The deviation is recorded in the guide as the single knowing departure from the API conventions, so -the two documents cannot silently drift again. diff --git a/docs/layout/README.md b/docs/layout/README.md new file mode 100644 index 00000000..a7abf6c5 --- /dev/null +++ b/docs/layout/README.md @@ -0,0 +1,50 @@ +# `docs/layout/` — where a document goes in Git, collected in one place + +This folder is organized by **topic**, which is a deliberate exception to the rule in +[`../INDEX.md`](../INDEX.md) that the other folders are picked by lifecycle. The layout question +(given a live object, which file in which folder receives it, and what else has to change so that +file is reachable) had grown to eight documents spread across `spec/`, `design/` and `future/`, and +following the argument meant knowing which of the three a given piece lived in. + +Lifecycle is still the thing that decides whether a page binds, so each entry below is labelled +with the class it would have had in the old layout. Read the label before you read the page. + +## Current behavior, and the code depends on it + +These are `spec/`-class. Go source cites them by path, and `task lint-docs` checks those citations. +If you change one of these behaviors, change the document in the same commit. + +| Document | What it pins | +|---|---| +| [`new-file-placement-rules.md`](new-file-placement-rules.md) | where a brand-new resource's file goes: declared, the folder's one kustomize root, canonical. Sibling inference is removed, and kept as history | +| [`contextual-namespace.md`](contextual-namespace.md) | kustomize graph-aware namespace inference, and the supported subset. This is the inference `writeNamespace` is proposed to replace | + +## Being decided + +These are `design/`-class: intent, not shipped behavior. + +| Document | What it proposes | +|---|---| +| [`model.md`](model.md) | the proposal, reversed and much smaller: the path template **stays**, because #319 made registration an invariant and retired three of the five arguments against it. Two additive fields instead of a discriminated union — `serializeNamespace` and `kustomizeRoot` | +| [`implementation-plan.md`](implementation-plan.md) | the order the work is built in: six PRs of which exactly one release is breaking, the corpus harness that turns the examples into the definition of done, and six design changes the examples produced | +| [`api-wave.md`](api-wave.md) | how the layout break sequences with the other `feat(api)!` work on `GitTarget`, so the consumer pays one bump | +| [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md) | the three questions the sibling-inference deletion left. Decided, mostly unbuilt, and its Question 2 is superseded by [`model.md`](model.md) | + +## Worked examples + +[`examples/`](examples/README.md) makes the proposal tangible: six scenarios, each with a +repository folder, the matching `GitTarget` and rule configuration, one live input, and the exact +patch the operator proposes. They are design material rather than install manifests, and +[`implementation-plan.md`](implementation-plan.md) turns them into an executable corpus in its +first PR. + +## What is deliberately not here + +- [`../design/support-boundary/`](../design/support-boundary/README.md) owns what the operator may + edit and what it refuses. Layout decides **where** a document goes; the support boundary decides + **whether** it may be written at all. +- [`../design/open-asks-priority.md`](../design/open-asks-priority.md) is the cross-cutting queue. + It references the layout work rather than containing it. +- [`../future/config-surface-for-a-structured-repository.md`](../future/config-surface-for-a-structured-repository.md) + is the broader configuration-surface review that the layout thesis came out of. It covers + `spec.path`, modes and `commitWindow` as well, so it stayed with its companion review. diff --git a/docs/layout/api-wave.md b/docs/layout/api-wave.md new file mode 100644 index 00000000..69be1471 --- /dev/null +++ b/docs/layout/api-wave.md @@ -0,0 +1,391 @@ +# The wave after placement left it + +> **design**: a sequencing proposal, not a plan of record. Nothing here binds until scheduled. +> Index: [`../INDEX.md`](../INDEX.md) +> Date: 2026-08-28 (originally 2026-07-30). +> +> **What changed.** This document was written to sequence one breaking wave whose centrepiece was +> `spec.layout`. [`model.md`](model.md) has since reversed: the path template stays and gains two +> additive fields, so the placement work is not breaking at all. The wave loses its largest member +> and most of its urgency, and what is left is smaller and more honest about being a batch. +> +> What remains: +> +> - the API-surface work left over from the status and configuration-model review: `spec.suspend`, +> `GitProvider.spec.interval` and the reconcile-request annotation; the `CommitRequest` lifecycle +> hole; the +> `meta.LocalObjectReference` reference-type nit; and the `TooManyStreams` cap and `default` +> `ClusterProvider` message. That review has been retired into the documents that own its +> findings, and this is where its unbuilt block landed; +> - the Tier 2 breaking items in [`open-asks-priority.md`](../design/open-asks-priority.md): **B4**, +> **#5**, **#6**. **B1 (`spec.mode: Observe|Write`) has been dropped**, see below; +> - the breaking half of +> [`source-scope-simplification.md`](../design/source-scope-simplification.md), which arrived +> after this document was written and is the only member that makes the API **smaller**. +> +> It does not touch Tier 1 (the removal-wait decision, #15's condition). Those are not breaking and +> should not wait for this. + +## Why still one wave + +The consumer pins us three ways (image, Go module, `require` line), so each breaking release costs a +coordinated bump. That argues for batching. It was always the weaker half of the argument, and with +placement gone it is now most of what is left. + +The stronger half survives in reduced form: **several of these items are the same design decision +seen from different angles**, and building them separately means deciding it several times, +inconsistently. + +> **The folder is described on the GitTarget. The connection describes only the connection.** + +- `spec.suspend` says whether we write to the folder. +- `commitWindow` and `commit.message` say how writes to it are batched and phrased, and today they + live on `GitProvider`, which is the connection (B4, and the config-model review's "GitProvider is + doing three jobs"). +- `serializeNamespace` and `kustomizeRoot` say what the documents in it look like — **and these are + additive**, so they can ship before, during or after the wave without anyone paying a bump. + +The principle is unchanged; only its most expensive expression left. Shipping `commitWindow`'s move +alone would still assert half of it, which is the reason to keep the rest together. + +## The interactions that change the design + +These are the reasons to combine, as opposed to merely batch. Each one changes what gets built. + +### 1. Adoption is a dry run, and `spec.suspend` is enough to give it + +Adoption is the weakest point of any placement scheme: placement only ever affects *new* documents, +so there is nothing to preview by inspection. You find out where files go by letting one be written. + +A suspended target plus `status.placement` is that preview. The operator scans, resolves the render +root, publishes it and what it *would* do, and writes nothing; clear `suspend` when the status says +what you expected. That turns "declare and hope" into a dry run, and it needs no second field. + +**`spec.mode: Observe|Write` (B1) was here and has been dropped.** It bought one thing over a +suspended target: the difference between a temporary pause and a declared, permanent read-only +posture. That is a distinction in intent, not in behavior, and paying an enum plus its own status +semantics for it made the wave harder to hold in your head than the capability was worth. Two of +this document's open questions were about nothing else, and both close with the deletion. + +The cost is stated rather than hidden: **`suspend` must keep observing.** Flux's `suspend` stops +reconciliation altogether; ours stops *writes* and keeps scanning, so the status a user is waiting +on stays fresh while they wait. That deviation belongs in the field's documentation, in one +sentence, because it is the only place we differ from a convention a Flux user brings with them. + +**Re-open trigger**: someone who needs a target that can never write, as a property of the object +rather than a switch a colleague can flip. `mode` is the answer if that arrives; a permission is +not, and neither is a comment. + +### 2. Dissolved: `GitTarget.spec.interval` + +**Dropped. We watch, and a watch is already the freshness mechanism.** + +The hole it was proposed for is real: the current half of `status.placement` is derived from the +last repository scan, a scan happens on a write or a resync, and a target that writes nothing could +carry a `renderRoot` stamped with last week's revision. A timer closes that. So does noticing that +nothing in this system is waiting on a timer in the first place. + +Every input that can change what a scan would conclude arrives as an event, on a watch, in +milliseconds: a live object appears, a rule changes, a spec changes. Flux polls because **a Git +remote cannot be watched**, which is exactly why `GitProvider.spec.interval` stays and is the +honest one of the pair. `GitTarget` sits on the other side of that asymmetry: its inputs are API +objects, and we are already streaming them. + +What remains uncovered is narrow enough to name in one sentence: a repository whose folder was +changed **by someone else** while our target wrote nothing. That is a poll of the output, we would +be paying it on every target forever to catch it, and the reconcile-request annotation refreshes it +on demand for the one person who wants it now. A stale `observedRevision` on an idle target is a +legible cost; a periodic scan on every target is not. + +The inverted reading, a periodic **re-list of the API** so the mirror re-derives desired state from +the cluster rather than from the event stream, is the one that would actually correspond to Flux's +interval, because desired state lives in the cluster here. It is a real design with a real cost, and +nobody has asked for it. Not now. + +**It is not a re-apply, and the name would have made people think it was.** In Flux, `interval` is +the drift-correction cadence: re-fetch the source, re-apply the desired state stored in Git. Had we +put the field on `GitTarget`, a Flux user would have read it as that, and it would have driven +neither of the two passes they were picturing: + +| | reads | writes | decides deletions | +|---|---|---|---| +| the observation pass | this target's folder, to resolve the render root and stamp `observedRevision` | never | never | +| the resync mark-and-sweep | the API, via the streaming list's initial-events snapshot | yes | yes, where `spec.prune.mode` allows | + +The sweep is enqueued from the watch plane with a cluster-gathered `desired` snapshot +([`event_router.go`](../../internal/watch/event_router.go), +[`reconcile-via-watchlist-mark-and-sweep.md`](../spec/reconcile-via-watchlist-mark-and-sweep.md)), +scoped per cell. Nothing about a Git-side scan is qualified to infer a deletion, because a document +whose object is gone and a document nobody has written yet look identical from that side. Dropping +the field removes the only place on `GitTarget` where that confusion had somewhere to attach. + +**Re-open trigger**: a user who needs an idle target's `status.placement` to track a repository +other people edit, and for whom the reconcile-request annotation is not enough. Then it is a scan +cadence, named for scanning. + +### 3. `spec.suspend` is a precondition for `kustomizeRoot: Create` + +`kustomizeRoot: Create` writes a `kustomization.yaml` the user did not author. That is +the right behavior and it raises the stakes on the review's central complaint: *this controller +writes to a Git repository and there is no way to make it stop that is not deleting the object.* + +So `suspend` is not a rider here, it is a precondition. A value that creates structure must ship +with the button that stops it. And `suspend` must stop bootstrap creation specifically, not only +resource writes, which is a detail worth stating before either is built. + +### 4. The Events question is already answered, and the resolution is what to say + +[`open-asks-priority.md`](../design/open-asks-priority.md) left one thing open about the inference deletion: +whether a fall-back to canonical should raise an Event on the GitTarget, and it reasoned that this +was expensive because placement runs on the branch worker with no recorder. + +That is no longer true. **An `EventRecorder` shipped on every reconciler** +(review §6), and the roll-up seam projects data-plane facts into status with an enqueue on change. So +the Event is now: emit when `status.placement` changes in a way a human should know about, which +is the `LayoutResolved` reason becoming `Ambiguous`, or a type falling back for the first time. One Event per +persisted change, the pattern already established for `Ready`. + +### 4b. The source-scope deletion is the only member that shrinks the API + +[`source-scope-simplification.md`](../design/source-scope-simplification.md) decided four things, +and three of them are breaking: + +| Change | Object | Kind | +|---|---|---| +| `spec.allowedSourceNamespaces` deleted, selector machinery with it | `GitTarget` | removal | +| `allowSourceNamespaceOverride` renamed `allowAnySourceNamespace` | `ClusterProvider` | pure rename | +| `allowedNamespaces` renamed `accessFrom` | `ClusterProvider` | pure rename | +| `sourceNamespace: "*"` becomes one cluster-wide list and watch | `WatchRule` semantics | redefinition | + +It belongs in this wave for the ordinary reason first: **it removes a field from `GitTarget`**, and +B4 already breaks `GitTarget` in the same release. Shipping them apart costs the consumer two bumps +for one object. + +The better reason is what it does to the review surface. Every other member of this wave adds a +field. This one deletes 4,569 lines and a whole three-valued verdict, and it deletes the only +cross-cluster read in the authorization path. A wave that is otherwise all addition is easier to +justify when the object it lands on comes out simpler than it went in, and easier to describe in +one `UPGRADING.md` entry. + +Two interactions worth naming, because they change what gets built rather than merely when: + +- **The `SelfSubjectAccessReview` pass is the same shape as the dry run.** Both answer "tell me what + you would do before you do it": a suspended target plus `status.placement` for the Git side, the + SAR pass for the source side. They are separate conditions on separate objects and neither depends on the + other, so the SAR work stays additive and can ship after the wave. Whoever writes the second + should read the first, because a user adopting a repository and a user asking which source cells + are reachable are the same user on the same afternoon. +- **`TooManyStreams` is sized by the `*` decision**, below. + +### 5. Dissolved: layout immutability + +An earlier draft of this document put `spec.layout` with `prune` as a mutable field, then reversed +to immutable-with-a-widening-exception, on the argument that a mutable structural kind leaves a +folder permanently half one shape and half another. + +There is no `spec.layout`, and `spec.placement` is mutable today. The condition the argument +described is real and stays: existing files never move, so a template change affects only what is +written afterwards, and a folder can hold documents placed under two different templates. That is +worth saying plainly rather than fixing, because match-first identity keeps finding those documents +and editing them in place. The CEL machinery — immutability plus a widening exception — was invented +to protect a discriminator that no longer exists. + +The one fact worth keeping from that argument, because it will be reached for again: **`GitTarget` +has no finalizer**, so deleting one leaves the folder in Git untouched and re-creating it at the same +path re-adopts every document by identity. Recreating a target costs status and a moment of +mirroring, not data. That is what made immutability affordable for `path` and unaffordable for +`prune`, and it is the right test for any future field on this object. + +### 6. Dissolved: the namespace agreement rule + +The layout's `scope: SingleNamespace` and `namespace` were structural claims that had to be kept in +agreement with `spec.allowedSourceNamespaces` by an admission rule. Neither field exists now: a +folder is single-namespace because no `{namespace}` appears in its paths. + +What survives is the observation underneath, which is why this section is not simply deleted. The +namespace-in-file question is inference today, and it is the one piece of inference an empty folder +cannot perform — there is no kustomization to inherit a convention from. That is exactly what +`serializeNamespace: Never` with `kustomizeRoot: Create` answers, and it needs no authorization +field to do it. + +## The GitTarget after the wave + +```yaml +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: prod + annotations: + reconcile.configbutler.ai/requestedAt: "2026-07-30T09:14:22Z" +spec: + # --- the connection: unchanged, and now only the connection --- + providerRef: + name: platform + branch: main + path: clusters/prod + # allowedSourceNamespaces is gone: the provider's credential bounds what can be read, + # and ClusterProvider.accessFrom bounds who may wield it. + + # --- what the documents in it look like: ADDITIVE, not part of the wave --- + placement: # unchanged from today + byType: + v1/secrets: "secrets/{name}{sensitiveSuffix}" + serializeNamespace: Never # the created kustomization carries namespace: + kustomizeRoot: Create + + # --- whether and when we write it --- + suspend: false # the only stop-writes switch; a suspended target still scans + prune: + mode: OnEvent + + # --- how writes are batched and phrased: moved off the connection --- + commitWindow: 5s # B4, was GitProvider.spec.push.commitWindow + commit: + message: + template: "chore(mirror): {{ .Summary }}" # B4, was GitProvider.spec.commit.message +status: + observedGeneration: 4 + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + observedGeneration: 4 + placement: + renderRoot: . + serializeNamespace: Never + observedRevision: 9f3c1ab + lastHandledReconcileAt: "2026-07-30T09:14:22Z" +``` + +The two placement fields are shown in place so the object reads as a whole, but they are marked +additive for a reason: they carry defaults equal to today's behavior, so they can ship in any +release without a bump. Only the `commitWindow` and `commit.message` move, and the riders below, +make this release breaking. + +## What rides along without a claim of synergy + +Honesty matters more than a tidy narrative. These are in the wave because they are breaking and the +consumer should pay once, not because they interact with anything else here: + +- **#5, `CommitRequest.spec.author`, SAR-guarded.** Independent, and it stands on the argument that + attribution needs an audit webhook a hosted control plane will not give you. +- **The `CommitRequest` lifecycle hole** (`ttlSecondsAfterFinished` or an `ownerReference`, plus the + `delete` verb). Unrelated to placement; it is the other object in the API with a lifecycle hole. +- **The reference types**: embedding `meta.LocalObjectReference` for the name half of our six + near-identical reference shapes. If GitTarget is breaking anyway, this is the moment. +- **The `TooManyStreams` cap**, now smaller than it was. + [`source-scope-simplification.md`](../design/source-scope-simplification.md) has decided that + `sourceNamespace: "*"` compiles to **one cluster-wide list and watch** rather than one stream per + admitted namespace, which removes the fan-out this cap was queued for. What is left to bound is + explicit enumeration (a rule naming many namespaces, or many rules on one target), so the cap is + still worth a `Stalled` reason plus a bound rather than discovering the cliff as apiserver watch + pressure, but it is no longer guarding the case that produced it. Size it against enumerated + rules, and do not let it be planned before the `*` change lands, or it will be sized against a + fan-out that no longer exists. +- **The `ClusterProvider` "default" message.** One error string, and the most likely first-run + support ticket. + +### The envtest that has to run before any of this is planned + +One question is not in the wave at all, and its answer constrains the enum work, so it should be +settled before anyone plans an API change around it. + +`ClusterWatchRule`'s `rules[].scope` enum is narrowed to `Cluster` only, deliberately keeping the +field so that re-applying a stored `Namespaced` value **fails** +([`clusterwatchrule_types.go`](../../api/v1alpha3/clusterwatchrule_types.go)). The concern is what +that does to the object's own ability to explain itself. For CRDs the apiserver validates the +**whole object** against the OpenAPI schema on **status-subresource** updates too, not only on spec +updates. If that holds here, the controller cannot write `Stalled=True` onto an object whose stored +`spec.rules[].scope` is `Namespaced` — the status update is rejected 422, and the one object that +most needs to explain itself is the one that cannot. + +The mitigating factor is CRD Validation Ratcheting, which skips re-validation of *unchanged* fields +and is beta and default-on from 1.30, GA in 1.33. So the exposure is older clusters, or a cluster +with the feature gate off — which is exactly why the test has to name a version. + +**The test.** Create a `ClusterWatchRule` with `scope: Namespaced` through a client that bypasses +the enum (or against an older CRD), then attempt a status update, on the minimum supported +Kubernetes version. + +**The fallback if it fails.** Widen the enum back and rely on the compile-path refusal plus a loud +`Stalled` condition. Refusing at admission is nice, but not at the cost of being unable to report +the refusal. + +## Version strategy: stay `v1alpha3` + +A wave this size invites `v1alpha4`, and I would not take it. + +- A new version means a **conversion path**, and the honest options are a conversion webhook (a + serving dependency for the CRD, plus a cert lifecycle) or `None` conversion with a stored-version + migration. Both cost more than the problem. +- We are `v1alpha3` and pre-1.0 with **one consumer**. The convention the repo already uses is a + **loud rejection**: keep the removed field in the schema, refuse it with a message naming the + replacement, for one release. `ClusterWatchRule.spec.rules[].scope` set that precedent, and the + reasoning holds better here: refusing a stored field the user can see beats translating it behind + their back. +- The pattern's members are now the source-scope changes rather than `spec.placement`, which no + longer moves at all. `allowedSourceNamespaces` becomes a rejection naming what replaced it (the + provider's own credential, plus `allowAnySourceNamespace` for the cluster-wide case), and the two + `ClusterProvider` renames are mechanical enough that the message can name the new field and + nothing else. A rename is exactly the case loud rejection is kindest on: the user's next `apply` + tells them the new spelling. + +If a second consumer appears before this ships, revisit: the calculus that makes loud rejection cheap +is one coordinated bump. + +## Order inside the wave + +Dependencies first, then the things that only need the object to be breaking. + +1. **The `scope: Namespaced` envtest**, above. Not an API change; its answer constrains the enum + work. Do it before planning. +2. **`spec.suspend`**. Precondition for anything that creates files, and independently the + review's highest-value gap. +3. **`status.placement`** plus the post-scan validation pass. Needed before a suspended target is + useful to look at, because a dry run with nothing to read previews nothing. +4. **`requestedAt` + `lastHandledReconcileAt`**. On-demand refresh of the status in step 3, and the + one reflex a Flux user brings that we do keep. `GitTarget.spec.interval` was step 4 and is + dropped, per interaction 2. +5. **Events on a changed resolution**, over the existing recorder. +6. **B4**: `commitWindow` and `commit.message` move from `GitProvider` to `GitTarget`. Last of the + principle items, and the one that makes the object coherent. +7. **The source-scope deletion**: `allowedSourceNamespaces` removed, the two `ClusterProvider` + renames, and `*` redefined as cluster-wide. Independent of every step above, so it can be written + in parallel; it is placed here because it is the one member whose review is a deletion, and a + deletion reviews better once the additions it is not entangled with are settled. +8. **The riders**: #5, the `CommitRequest` lifecycle, the reference types, `TooManyStreams`, the + `default` ClusterProvider message. `TooManyStreams` must come after step 7, which removes the + fan-out it was written to bound. + +**`serializeNamespace` and `kustomizeRoot` are deliberately absent from this list.** They are +additive, so they do not belong in a breaking wave's ordering at all; their order is +[`implementation-plan.md`](implementation-plan.md)'s, and it puts them after `status.placement` +because the post-scan pass is what makes them honest. + +Steps 3 to 7 are one release. Step 1 gates the planning. Step 8 can be trimmed if the wave gets too +big to review, since nothing else depends on it. + +## What this costs, stated plainly + +- **One coordinated consumer bump**, for a much smaller set than this document originally proposed: + `commitWindow` and `commit.message` change object, the riders change shape, and everything else + here is additive. +- **One `docs/UPGRADING.md` entry** covering the two field moves, the source-scope removal and + renames, the `*` redefinition, and the riders. The placement work needs no migration entry at all, + which is the largest single change to this document. The `*` paragraph is the one that has to be + written carefully: it is the only change here that keeps its spelling and changes its meaning. +- **A capability genuinely lost**: source-side label selectors, priced in the source-scope document. + It is the only thing in this wave a user can do today and cannot do afterwards. +- **A review surface that is now ordinary.** The argument for trimming the riders first still holds, + but the wave is no longer the largest change this project has taken. + +## Open questions + +- ~~**Does `mode: Observe` write status only, or also refuse admission of new WatchRules?**~~ + ~~**Should `suspend` and `mode: Observe` be one field?**~~ **Both closed by dropping `mode`**: one + field, and a suspended target keeps observing. See interaction 1. +- ~~**Where does `interval` live?**~~ **Decided: only on `GitProvider`.** Two objects with a + reconcile cadence would have been the Flux convention rather than a collision, and the name was + never the problem. The problem was that `GitTarget` did not need the field: a Git remote cannot be + watched and an API object can, so the poll belongs on the side that polls. Interaction 2. diff --git a/docs/spec/contextual-namespace-and-kustomize-folder-editing.md b/docs/layout/contextual-namespace.md similarity index 98% rename from docs/spec/contextual-namespace-and-kustomize-folder-editing.md rename to docs/layout/contextual-namespace.md index c9ecd5fa..c423522c 100644 --- a/docs/spec/contextual-namespace-and-kustomize-folder-editing.md +++ b/docs/layout/contextual-namespace.md @@ -8,11 +8,9 @@ > Captured: 2026-06-08 > Updated: 2026-06-08 > Related: -> [file-agnostic-placement.md](gittarget-new-file-placement-rules.md), -> [manifest-inventory-file-agnostic-placement.md](manifest-system.md), -> [current-manifest-support-review.md](current-manifest-support-review.md), -> [version2/gittarget-repository-validity-and-placement.md](gittarget-new-file-placement-rules.md), -> [version2/gittarget-new-file-placement-rules.md](gittarget-new-file-placement-rules.md) +> [new-file-placement-rules.md](new-file-placement-rules.md), +> [manifest-system.md](../spec/manifest-system.md), +> [current-manifest-support-review.md](../spec/current-manifest-support-review.md) ## Summary diff --git a/docs/layout/examples/README.md b/docs/layout/examples/README.md new file mode 100644 index 00000000..052bf4c9 --- /dev/null +++ b/docs/layout/examples/README.md @@ -0,0 +1,104 @@ +# GitTarget layout examples + +> **design**: concrete repository and configuration examples for the proposed `GitTarget` layout +> model. The API fields shown here are not available in the current release. +> +> Read this beside [`../model.md`](../model.md) and +> [`../api-wave.md`](../api-wave.md). The current supported Kustomize +> boundary remains in +> [`../support-boundary/kustomize-support-boundary.md`](../../design/support-boundary/kustomize-support-boundary.md). + +These folders make a review question concrete: what does a `GitTarget` say about a Git folder, and +what files does that statement cause the operator to create or update? + +Every scenario contains: + +- `repository/`: the relevant repository subtree. Each scenario states whether it is the starting + state or the state after the illustrated change. +- `config/`: the `GitTarget` and watcher objects that describe the target. The `GitTarget` files + deliberately use the proposed `serializeNamespace` and `kustomizeRoot` fields, which do not exist + yet, and `spec.suspend`, which does not either. +- `input/`: one representative live object or source event, **as the operator receives it from the + API server** — not as it is written to Git. +- `expected-*.patch`: the exact change proposed for Git. It makes the write boundary reviewable + without asking the reader to infer a file placement from prose. +- `README.md`: the decision the layout makes, the expected Git change, and the boundary that keeps + the example safe. + +Each target needs a `GitProvider` in the target's namespace. See the +[shared prerequisites](prerequisites/README.md) for the minimal provider specimen and the source +cluster rule. Connection credentials are outside the layout decision. A source `ClusterProvider` +is named only where that choice explains the scenario. + +| Scenario | Question it exercises | +|---|---| +| [Brownfield kustomize adoption](brownfield-kustomize/README.md) | How does an existing folder become a target without a first write? | +| [Empty-repo bootstrap](empty-repo-bootstrap/README.md) | How does an empty repository become a deployable, single-namespace folder? | +| [Tree, multi-namespace](tree-multi-namespace/README.md) | How does a small cluster mirror several namespaces and cluster resources safely? | +| [Homelab Argo CD](homelab-argocd/README.md) | How does an app-of-apps folder remain a narrow, editable target? | +| [Homelab Flux](homelab-flux/README.md) | How do Flux declarations stay editable without treating chart output as source? | +| [CRD closure](crd-closure/README.md) | What else does a folder of custom resources need to be applied elsewhere? **A question, not a decision** | +| [Overlay-scoped target](overlay-scoped-target/README.md) | How can an environment overlay change a supported field without claiming ownership of its base? **Write path, not placement** | + +## Two fixture conventions + +**The inputs are live objects, and the difference is the assertion.** A captured object carries +`uid`, `resourceVersion`, `generation`, `creationTimestamp`, `managedFields`, a populated `status`, +and often a finalizer or a controller's own annotation. None of that reaches Git. So the diff +between an `input/` fixture and its `expected-*.patch` is not decoration: it *is* the sanitization +assertion, and a corpus harness that reads both gets it without a separate test. + +[`homelab-argocd`](homelab-argocd/README.md) and [`homelab-flux`](homelab-flux/README.md) keep the +full captured shape, because those are the two a Flux or Argo reviewer will look at hardest — the +first carries `argocd.argoproj.io/tracking-id`, whose leaking into Git hard-fails another +Application's sync, and the second carries `finalizers.fluxcd.io`. The other four scenarios keep +abridged inputs and say so at the top of the file. + +**The patches carry no `index` lines.** A hash in a patch header is a promise about blob contents, +and a fabricated one gives `git diff` output's authority without its guarantee — a reader who tries +to `git apply` it gets a confusing failure. `diff --git`, the `---`/`+++` pair, the mode line, and +the hunks are the reviewable content. Every patch here applies cleanly to its scenario's stated +starting state, which is the property the index line was pretending to have. + +## How to read the proposed fields + +Two fields are proposed on top of the `spec.placement` the current release already has. Everything +else in these scenarios ships today. + +`serializeNamespace` (`Auto`, `Always`, `Never`) decides whether `metadata.namespace` appears inside +the committed document. A path cannot express it: kustomize takes the namespace from either the +document or a governing root, and where the file sits decides neither. `Never` is honest only when +something guarantees the namespace, which is why two scenarios pair it with `Require`. + +`kustomizeRoot` (`Adopt`, `Create`, `Require`) decides what happens when **no** kustomization governs +the path. When one does, every value registers the new file in its `resources:` — that is an +invariant, not a setting. `Adopt` writes the file anyway, `Create` writes the missing +`kustomization.yaml` first, and `Require` refuses the write. + +Where a new file goes is the template's job, and most scenarios here declare no template at all: the +built-in ladder places a new document beside the folder's one kustomize root, or at the canonical +identity path when there is no root. The file is named `{name}.yaml`. The operator does not copy a +naming convention from the neighbouring files — that inference was deliberately deleted — so a folder +of `deployment-web.yaml` and `service-web.yaml` receives `cache.yaml`, and +`placement.default: "{kindLower}-{name}.yaml"` is how to ask for the other convention on purpose. + +`suspend: true` is the adoption path: a suspended target still scans and publishes what it +resolved, and writes nothing. Clear it only after the resolved root and prospective paths match the +repository owner's intent. + +Every brownfield scenario here opens suspended and shows the observed status before it shows a +patch, because that is the path the paragraph above describes and a set of examples that opened +writing would be advertising something else. +[`empty-repo-bootstrap`](empty-repo-bootstrap/README.md) is the one honest exception: its repository +is empty, so there is nothing to observe. + +## One product, three names + +A reader meets all three inside a single scenario, so it is worth defusing once: `configbutler.ai` +is the API group, **GitOps Reverser** is the product, and `gitops-reverser` is the repository and +binary name. Flux is uniformly `*.toolkit.fluxcd.io` and pays nothing for it; we have a split, and +these examples are where it is most visible. + +The examples do not make Kustomize a general source inverse. They stay inside the support +boundary: one writable expression, a local render root, and a re-rendered proposal. See the +[support contract](../../design/support-boundary/support-contract.md) for the authoritative limits. diff --git a/docs/layout/examples/brownfield-kustomize/README.md b/docs/layout/examples/brownfield-kustomize/README.md new file mode 100644 index 00000000..05a44eda --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/README.md @@ -0,0 +1,81 @@ +# Brownfield Kustomize adoption + +This scenario adopts an existing application folder before it writes. The folder already has one +supported Kustomize root and one namespace convention, so `Auto` can resolve to `Kustomize` during +an observation pass. + +## Starting repository + +[`repository/`](repository/) is an existing `demo` application folder. Its +[`kustomization.yaml`](repository/kustomization.yaml) supplies `namespace: demo` and lists the +Deployment and Service that it owns. + +```text +apps/demo/ + kustomization.yaml + deployment-web.yaml + service-web.yaml +``` + +## Proposed configuration + +[`config/gittarget.yaml`](config/gittarget.yaml) opens suspended and declares no +`placement` at all, so the built-in ladder applies: the folder's one kustomize root places new +files beside it. `serializeNamespace: Auto` and `kustomizeRoot: Adopt` are both defaults, spelled +out here because this scenario is about what adoption resolves to. +[`config/watchrule.yaml`](config/watchrule.yaml) supplies the namespaced resource subscription. + +The first observation records a result equivalent to: + +```yaml +status: + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + placement: + renderRoot: . + serializeNamespace: Auto +``` + +No commit is made while the target is suspended. After a reviewer confirms that result, clearing +`suspend` lets a new ConfigMap named `cache` produce this commit: + +```text +apps/demo/ + kustomization.yaml # adds cache.yaml to resources + cache.yaml # omits metadata.namespace + deployment-web.yaml + service-web.yaml +``` + +The Kustomize root supplies the omitted namespace, so the new document still represents +`demo/cache`. The first write has a home that the existing root reaches. + +## Scenario contract + +- Starting repository: [`repository/`](repository/), already recognized while suspended. +- Live input: [`input/cache.yaml`](input/cache.yaml). +- Expected Git change: [`expected-cache.patch`](expected-cache.patch), after a + reviewer clears `suspend`. +- Expected status: `Ready=True` with `LayoutResolved=True` and `renderRoot: .` during observation. +- Boundary: a second Kustomize root or an unsupported expression makes the target refuse the write. + +## What this example rules out + +This target does not infer a custom file convention from its siblings, and the new file's name is +where that shows. The folder holds `deployment-web.yaml` and `service-web.yaml`, so a reader might +expect `configmap-cache.yaml`. The operator writes **`cache.yaml`**: the rung names a new sibling +`{name}.yaml` and does not learn a naming convention by looking at the neighbours. That inference +was deliberately deleted, and declaring `placement.default: "{kindLower}-{name}.yaml"` is how you +ask for the other convention on purpose. + +A second kustomize root in the target path is a misconfiguration of the GitTarget rather than a +placement puzzle; it does not cause the operator to pick an arbitrary root. A folder with no +kustomize root falls through to the canonical identity path. + +The example stays within the +[Kustomize support boundary](../../../design/support-boundary/kustomize-support-boundary.md): it edits local +declarations and verifies the resulting render. It does not invert generators, plugins, remote +bases, or chart output. diff --git a/docs/layout/examples/brownfield-kustomize/config/gittarget.yaml b/docs/layout/examples/brownfield-kustomize/config/gittarget.yaml new file mode 100644 index 00000000..f11cac51 --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/config/gittarget.yaml @@ -0,0 +1,17 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: demo-workloads + namespace: demo +spec: + providerRef: + name: app-repository + branch: main + path: apps/demo + suspend: true # adoption dry run: scans and publishes, writes nothing + # No placement declared, so the built-in ladder applies: the folder's one + # kustomize root places new files beside it. serializeNamespace: Auto is the + # default too, and both are spelled out here because the scenario is about + # what adoption resolves to. + serializeNamespace: Auto + kustomizeRoot: Adopt diff --git a/docs/layout/examples/brownfield-kustomize/config/watchrule.yaml b/docs/layout/examples/brownfield-kustomize/config/watchrule.yaml new file mode 100644 index 00000000..9e147528 --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/config/watchrule.yaml @@ -0,0 +1,15 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: demo-content + namespace: demo +spec: + targetRef: + name: demo-workloads + rules: + - apiGroups: [""] + apiVersions: ["v1"] + resources: ["configmaps", "services"] + - apiGroups: ["apps"] + apiVersions: ["v1"] + resources: ["deployments"] diff --git a/docs/layout/examples/brownfield-kustomize/expected-cache.patch b/docs/layout/examples/brownfield-kustomize/expected-cache.patch new file mode 100644 index 00000000..2d86a066 --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/expected-cache.patch @@ -0,0 +1,19 @@ +diff --git a/apps/demo/kustomization.yaml b/apps/demo/kustomization.yaml +--- a/apps/demo/kustomization.yaml ++++ b/apps/demo/kustomization.yaml +@@ -4,3 +4,4 @@ kind: Kustomization + resources: + - deployment-web.yaml + - service-web.yaml ++ - cache.yaml +diff --git a/apps/demo/cache.yaml b/apps/demo/cache.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/demo/cache.yaml +@@ -0,0 +1,6 @@ ++apiVersion: v1 ++kind: ConfigMap ++metadata: ++ name: cache ++data: ++ address: redis.demo.svc.cluster.local:6379 diff --git a/docs/layout/examples/brownfield-kustomize/input/cache.yaml b/docs/layout/examples/brownfield-kustomize/input/cache.yaml new file mode 100644 index 00000000..beaecda5 --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/input/cache.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: cache + namespace: demo +data: + address: redis.demo.svc.cluster.local:6379 diff --git a/docs/layout/examples/brownfield-kustomize/repository/deployment-web.yaml b/docs/layout/examples/brownfield-kustomize/repository/deployment-web.yaml new file mode 100644 index 00000000..bc80901d --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/repository/deployment-web.yaml @@ -0,0 +1,19 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: web +spec: + replicas: 2 + selector: + matchLabels: + app: web + template: + metadata: + labels: + app: web + spec: + containers: + - name: web + image: nginx:1.27.4 + ports: + - containerPort: 80 diff --git a/docs/layout/examples/brownfield-kustomize/repository/kustomization.yaml b/docs/layout/examples/brownfield-kustomize/repository/kustomization.yaml new file mode 100644 index 00000000..ef97cd1c --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/repository/kustomization.yaml @@ -0,0 +1,6 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: demo +resources: + - deployment-web.yaml + - service-web.yaml diff --git a/docs/layout/examples/brownfield-kustomize/repository/service-web.yaml b/docs/layout/examples/brownfield-kustomize/repository/service-web.yaml new file mode 100644 index 00000000..56c6121f --- /dev/null +++ b/docs/layout/examples/brownfield-kustomize/repository/service-web.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Service +metadata: + name: web +spec: + selector: + app: web + ports: + - port: 80 + targetPort: 80 diff --git a/docs/layout/examples/crd-closure/README.md b/docs/layout/examples/crd-closure/README.md new file mode 100644 index 00000000..3d57053b --- /dev/null +++ b/docs/layout/examples/crd-closure/README.md @@ -0,0 +1,103 @@ +# CRD closure: what else the folder needs + +> **This scenario has no decided answer, and that is deliberate.** It exists to make the choice in +> [`crd-handling.md`](../../../design/crd-handling.md) legible by showing the same event under both +> candidate shapes. Every other scenario in this folder illustrates a decision; this one illustrates +> a question, and the design page is where it gets settled. + +An application team's configuration folder holds `Widget` objects. The folder is complete as a +mirror and incomplete as an artifact: applied to a cluster that has never heard of +`widgets.apps.example.com`, every `Widget` in it is rejected. Something has to carry the definition, +and the interesting part is that it does not have to be this folder. + +## Starting repository + +[`repository/`](repository/) is `apps/shop` in the real repository: one kustomize root, one +ConfigMap, one existing `Widget`. + +```text +apps/shop/ + kustomization.yaml + configmap-storefront.yaml + widget-checkout.yaml +``` + +## Proposed configuration + +[`config/watchrule.yaml`](config/watchrule.yaml) subscribes to `widgets` and `configmaps`, and +**names no CRD at all**. That is the point: today the only way to get definitions into Git is a rule +for `customresourcedefinitions`, which mirrors every CRD in the cluster, Flux's and cert-manager's +included. The proposal derives the set instead, from the types the rules match, over a join +the operator already performs. + +[`config/gittarget.yaml`](config/gittarget.yaml) sets the proposed +`includeTypeDefinitions: Referenced`. The default is `None`. + +## The live input, and what it carries + +[`input/widget-search.yaml`](input/widget-search.yaml) is an ordinary new object. Its placement is +not the question: the folder's one root takes it as `widget-search.yaml` with a `resources:` entry, +in every option below. + +[`input/crd-widgets.yaml`](input/crd-widgets.yaml) is the definition behind it, and it is worth +reading for its metadata rather than its schema: + +- `meta.helm.sh/release-name` and `app.kubernetes.io/managed-by: Helm` say a Helm release owns this + object, and **`sanitize` does not strip them**, so they would arrive in Git intact. +- `kustomize.toolkit.fluxcd.io/name` says a Flux `Kustomization` applied it, and **`sanitize` does + strip that**, so it can only be read from the live object. + +Those two facts are most of the argument. + +## Option B: `Referenced` + +[`expected-referenced.patch`](expected-referenced.patch). The `Widget` lands as usual, and one extra +generated file appears: a `type-dependencies.yaml` naming the type, its served versions, the CRD, a +digest of its spec, and the Helm release the CRD came from. No schema, about fifteen lines, and it +changes only when the type set or the schema does. + +Whoever hydrates the folder reads it: the per-branch editing cluster installs the referenced +definitions before loading objects, and a reviewer can see at a glance which operators a branch +depends on. + +## Option A: `Vendored` + +[`expected-vendored.patch`](expected-vendored.patch). The definition itself is committed at the +canonical cluster-scoped path and registered in the root. It is self-contained, and it costs three +things the referenced shape does not: + +- **A second owner.** The committed copy still says `meta.helm.sh/release-name: widget-operator`, + so it claims to belong to a release in another cluster, and applying it elsewhere collides with + Helm's ownership check. +- **Size and churn.** Real schemas are hundreds of lines; Argo CD's `Application` CRD is about a + megabyte, and every operator upgrade rewrites it. +- **Ordering.** A folder holding a CRD and its custom resources has to apply the definition first. + Flux and Argo CD arrange that; `kubectl apply -k` over a plain folder does not. + +For application configuration it also puts the wrong team in charge: the app developer owns what the +`Widget` CRD looks like, and it belongs beside the controller that serves it. + +## Scenario contract + +- Starting repository: [`repository/`](repository/), which **is** `apps/shop/`. +- Live inputs: [`input/widget-search.yaml`](input/widget-search.yaml) and, as context rather than as + a watched object, [`input/crd-widgets.yaml`](input/crd-widgets.yaml). +- Expected Git change: [`expected-referenced.patch`](expected-referenced.patch) under the + recommendation, [`expected-vendored.patch`](expected-vendored.patch) under the alternative. +- Expected status: `Ready=True`; the type closure is an observation, never a condition. +- Boundary: the operator never installs, upgrades or deletes a CRD. It records or copies one. + +## What this scenario cannot show + +The definition is necessary and not always sufficient. `Widget` converts with nothing running, which +is why its manifest entry says `conversion: {strategy: None}` and why this scenario stays simple. A +type whose CRD declares `spec.conversion.strategy: Webhook`, or whose real cluster mutates objects +through a mutating admission webhook, needs something to answer, and a workload-less branch cluster +has nothing to answer with. The objects are accepted and are quietly not what production would have +stored. + +It does not need a *controller*, though, only an endpoint: `WebhookClientConfig` takes a `url` as +well as a `service`, and `url` is precisely the form for a webhook that does not run in the cluster. +The ladder, the constraints on that URL, and what it would take for ConfigButler to host +transformations rather than every team exposing an endpoint, are in +[`crd-handling.md`](../../../design/crd-handling.md). diff --git a/docs/layout/examples/crd-closure/config/gittarget.yaml b/docs/layout/examples/crd-closure/config/gittarget.yaml new file mode 100644 index 00000000..8476420a --- /dev/null +++ b/docs/layout/examples/crd-closure/config/gittarget.yaml @@ -0,0 +1,17 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: shop-configuration + namespace: shop +spec: + providerRef: + name: app-repository + branch: main + path: apps/shop + suspend: true # adoption dry run: scans and publishes, writes nothing + # The proposed field. None is the default and the posture this folder would have + # without the line: instances here, definitions owned by whoever installs the + # operator. Referenced commits a manifest of references, never schemas. + includeTypeDefinitions: Referenced + serializeNamespace: Never + kustomizeRoot: Require diff --git a/docs/layout/examples/crd-closure/config/watchrule.yaml b/docs/layout/examples/crd-closure/config/watchrule.yaml new file mode 100644 index 00000000..3038194d --- /dev/null +++ b/docs/layout/examples/crd-closure/config/watchrule.yaml @@ -0,0 +1,16 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: shop-configuration + namespace: shop +spec: + targetRef: + name: shop-configuration + rules: + # No rule names customresourcedefinitions. The closure is DERIVED from the + # types these rules actually match, which is the whole point: a rule for + # CRDs would mirror every CRD in the cluster. + - apiGroups: ["apps.example.com"] + resources: ["widgets"] + - apiGroups: [""] + resources: ["configmaps"] diff --git a/docs/layout/examples/crd-closure/expected-referenced.patch b/docs/layout/examples/crd-closure/expected-referenced.patch new file mode 100644 index 00000000..cc48c7b1 --- /dev/null +++ b/docs/layout/examples/crd-closure/expected-referenced.patch @@ -0,0 +1,46 @@ +diff --git a/apps/shop/kustomization.yaml b/apps/shop/kustomization.yaml +--- a/apps/shop/kustomization.yaml ++++ b/apps/shop/kustomization.yaml +@@ -3,4 +3,5 @@ kind: Kustomization + namespace: shop + resources: + - configmap-storefront.yaml + - widget-checkout.yaml ++ - widget-search.yaml +diff --git a/apps/shop/widget-search.yaml b/apps/shop/widget-search.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/shop/widget-search.yaml +@@ -0,0 +1,8 @@ ++apiVersion: apps.example.com/v1alpha1 ++kind: Widget ++metadata: ++ name: search ++spec: ++ replicas: 3 ++ variant: faceted +diff --git a/apps/shop/type-dependencies.yaml b/apps/shop/type-dependencies.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/shop/type-dependencies.yaml +@@ -0,0 +1,19 @@ ++# Generated. The types this folder contains that the API server does not serve ++# on its own, and where their definitions come from. Not applied: this file is ++# read by whatever hydrates the folder into a cluster. ++apiVersion: configbutler.ai/v1alpha3 ++kind: TypeDependencies ++observedAt: "2026-08-29T09:12:51Z" ++types: ++ - group: apps.example.com ++ resource: widgets ++ kind: Widget ++ servedVersions: [v1alpha1] ++ definition: widgets.apps.example.com ++ specDigest: sha256:8f2a1c9d ++ # Whether a cluster can serve this type with nothing running. Webhook here ++ # would name the endpoint a branch cluster has to point its CRD at. ++ conversion: {strategy: None} ++ source: ++ kind: HelmRelease ++ name: widget-operator ++ namespace: widget-system diff --git a/docs/layout/examples/crd-closure/expected-vendored.patch b/docs/layout/examples/crd-closure/expected-vendored.patch new file mode 100644 index 00000000..0aa3cb45 --- /dev/null +++ b/docs/layout/examples/crd-closure/expected-vendored.patch @@ -0,0 +1,48 @@ +# The alternative shape, for comparison. Same event, includeTypeDefinitions: Vendored. +# The Widget document and the kustomization entry are identical to the referenced +# patch and are elided here; what differs is everything below. +diff --git a/apps/shop/kustomization.yaml b/apps/shop/kustomization.yaml +--- a/apps/shop/kustomization.yaml ++++ b/apps/shop/kustomization.yaml +@@ -3,4 +3,6 @@ kind: Kustomization + namespace: shop + resources: + - configmap-storefront.yaml + - widget-checkout.yaml ++ - widget-search.yaml ++ - _cluster/apiextensions.k8s.io/customresourcedefinitions/widgets.apps.example.com.yaml +diff --git a/apps/shop/_cluster/apiextensions.k8s.io/customresourcedefinitions/widgets.apps.example.com.yaml b/apps/shop/_cluster/apiextensions.k8s.io/customresourcedefinitions/widgets.apps.example.com.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/shop/_cluster/apiextensions.k8s.io/customresourcedefinitions/widgets.apps.example.com.yaml +@@ -0,0 +1,31 @@ ++apiVersion: apiextensions.k8s.io/v1 ++kind: CustomResourceDefinition ++metadata: ++ name: widgets.apps.example.com ++ labels: ++ app.kubernetes.io/managed-by: Helm ++ annotations: ++ meta.helm.sh/release-name: widget-operator ++ meta.helm.sh/release-namespace: widget-system ++spec: ++ group: apps.example.com ++ scope: Namespaced ++ names: ++ plural: widgets ++ singular: widget ++ kind: Widget ++ versions: ++ - name: v1alpha1 ++ served: true ++ storage: true ++ schema: ++ openAPIV3Schema: ++ type: object ++ properties: ++ spec: ++ type: object ++ properties: ++ replicas: {type: integer} ++ variant: {type: string} ++# ... and the rest of a real schema, which is where the megabyte goes. diff --git a/docs/layout/examples/crd-closure/input/crd-widgets.yaml b/docs/layout/examples/crd-closure/input/crd-widgets.yaml new file mode 100644 index 00000000..80737781 --- /dev/null +++ b/docs/layout/examples/crd-closure/input/crd-widgets.yaml @@ -0,0 +1,40 @@ +# The live CustomResourceDefinition backing apps.example.com/v1alpha1 Widget, +# as the operator reads it from the source cluster. Schema trimmed; a real one +# runs to hundreds of lines, and Argo CD's Application CRD to about a megabyte. +# +# Note what it carries. The Helm ownership metadata is NOT stripped by sanitize +# (internal/sanitize/types.go strips Flux, kro, applyset and two exact Argo keys), +# so a vendored copy arrives in Git still claiming to belong to a release in +# another cluster. The Flux label IS stripped, which is why provenance has to be +# captured here, live, and can never be read back out of the repository. +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.apps.example.com + labels: + app.kubernetes.io/managed-by: Helm + kustomize.toolkit.fluxcd.io/name: widget-operator + kustomize.toolkit.fluxcd.io/namespace: flux-system + annotations: + meta.helm.sh/release-name: widget-operator + meta.helm.sh/release-namespace: widget-system +spec: + group: apps.example.com + scope: Namespaced + names: + plural: widgets + singular: widget + kind: Widget + versions: + - name: v1alpha1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + properties: + replicas: {type: integer} + variant: {type: string} diff --git a/docs/layout/examples/crd-closure/input/widget-search.yaml b/docs/layout/examples/crd-closure/input/widget-search.yaml new file mode 100644 index 00000000..0c5f3011 --- /dev/null +++ b/docs/layout/examples/crd-closure/input/widget-search.yaml @@ -0,0 +1,18 @@ +# A second Widget, created in the intent cluster. It is an ordinary placement: +# the folder's one kustomize root takes it, and the CRD question below is about +# what ELSE the folder needs to be applied anywhere but the cluster it came from. +apiVersion: apps.example.com/v1alpha1 +kind: Widget +metadata: + name: search + namespace: shop + uid: 6d1b0e4c-6f0e-4a2f-9a0e-2c1b7f0a5d31 + resourceVersion: "418302" + generation: 1 + creationTimestamp: "2026-08-29T09:12:44Z" +spec: + replicas: 3 + variant: faceted +status: + observedGeneration: 1 + readyReplicas: 3 diff --git a/docs/layout/examples/crd-closure/repository/configmap-storefront.yaml b/docs/layout/examples/crd-closure/repository/configmap-storefront.yaml new file mode 100644 index 00000000..40ad3cd9 --- /dev/null +++ b/docs/layout/examples/crd-closure/repository/configmap-storefront.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: storefront +data: + currency: EUR diff --git a/docs/layout/examples/crd-closure/repository/kustomization.yaml b/docs/layout/examples/crd-closure/repository/kustomization.yaml new file mode 100644 index 00000000..9d9be182 --- /dev/null +++ b/docs/layout/examples/crd-closure/repository/kustomization.yaml @@ -0,0 +1,6 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: shop +resources: + - configmap-storefront.yaml + - widget-checkout.yaml diff --git a/docs/layout/examples/crd-closure/repository/widget-checkout.yaml b/docs/layout/examples/crd-closure/repository/widget-checkout.yaml new file mode 100644 index 00000000..480048ed --- /dev/null +++ b/docs/layout/examples/crd-closure/repository/widget-checkout.yaml @@ -0,0 +1,7 @@ +apiVersion: apps.example.com/v1alpha1 +kind: Widget +metadata: + name: checkout +spec: + replicas: 2 + variant: express diff --git a/docs/layout/examples/empty-repo-bootstrap/README.md b/docs/layout/examples/empty-repo-bootstrap/README.md new file mode 100644 index 00000000..8636fd98 --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/README.md @@ -0,0 +1,59 @@ +# Application configuration as KRM + +This scenario starts with an empty Git repository. A team edits its own configuration CRDs in an +intent cluster, and GitOps Reverser creates the folder that a deployer consumes. + +## Resulting repository folder + +[`repository/`](repository/) is the result after the first `ShopConfiguration` write. The target +creates the root and registers the first document, so the first commit is already a Kustomize +folder: + +```text +apps/shop/ + kustomization.yaml + storefront.yaml +``` + +The root supplies `namespace: shop`. The Git document omits `metadata.namespace`, while the live +input carries it. The layout establishes that convention at the root rather than waiting for a first +event to do so. + +## Proposed configuration + +[`config/gittarget.yaml`](config/gittarget.yaml) sets `kustomizeRoot: Create` and +`serializeNamespace: Never`. It also constrains the source to one exact namespace. +[`config/watchrule.yaml`](config/watchrule.yaml) is the artifact manifest: it states which KRM +types the team wants included in the folder. + +The type in this example, `config.shop.example/v1alpha1`, stands for an application configuration +CRD installed in the intent cluster. It is a specimen, not a requirement of GitOps Reverser. + +## Scenario contract + +- Starting repository: empty at `apps/shop`. +- Live input: [`input/storefront.yaml`](input/storefront.yaml). +- Expected Git change: [`expected-first-write.patch`](expected-first-write.patch). +- Expected status: `Ready=True` after the created root renders the captured KRM object. +- Boundary: a source outside `shop`, an unauthorized namespace, or an unsupported KRM expression is + refused before a commit. + +## Why `Create` and `Never` belong together + +This is the one pairing in the model that closes its own loop. `serializeNamespace: Never` is only +honest when something guarantees the namespace, and on an empty folder there is nothing to inspect: +no kustomization exists to inherit a convention from, so inference structurally cannot answer. +`kustomizeRoot: Create` supplies the missing half — the operator writes the `kustomization.yaml`, +puts `namespace: shop` in it, and then legitimately omits `metadata.namespace` from every document +it places. The convention is **established** rather than guessed. + +The folder is single-namespace by construction rather than by declaration: no `{namespace}` appears +in any path, because no `placement` is declared and the created root places files beside itself. A +second source namespace has nowhere to go that would not collide, and nothing on the target says so. +The WatchRule feeding it is namespaced and names its own source namespace, which is the whole fence +after +[`source-scope-simplification.md`](../../../design/source-scope-simplification.md). + +This example is the configuration-as-data path from +[the direction review](../../../future/direction-and-configuration-surface.md). The folder is a +reviewable artifact; the KRM objects in the intent cluster are the editing surface. diff --git a/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml b/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml new file mode 100644 index 00000000..412d31e2 --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/config/gittarget.yaml @@ -0,0 +1,18 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: shop-configuration + namespace: shop +spec: + providerRef: + name: artifacts-repository + clusterProviderRef: + name: app-intent + branch: main + path: apps/shop + # The folder does not exist yet, so the root that supplies the namespace has to + # be created before Never can be true. Create is what makes the omission + # provable rather than trusted: the operator owns the kustomization.yaml it + # writes namespace: shop into. + serializeNamespace: Never + kustomizeRoot: Create diff --git a/docs/layout/examples/empty-repo-bootstrap/config/watchrule.yaml b/docs/layout/examples/empty-repo-bootstrap/config/watchrule.yaml new file mode 100644 index 00000000..c098509f --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/config/watchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: shop-configuration-content + namespace: shop +spec: + targetRef: + name: shop-configuration + rules: + - apiGroups: ["config.shop.example"] + apiVersions: ["v1alpha1"] + resources: ["shopconfigurations"] diff --git a/docs/layout/examples/empty-repo-bootstrap/expected-first-write.patch b/docs/layout/examples/empty-repo-bootstrap/expected-first-write.patch new file mode 100644 index 00000000..18d523ae --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/expected-first-write.patch @@ -0,0 +1,26 @@ +diff --git a/apps/shop/kustomization.yaml b/apps/shop/kustomization.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/shop/kustomization.yaml +@@ -0,0 +1,5 @@ ++apiVersion: kustomize.config.k8s.io/v1beta1 ++kind: Kustomization ++namespace: shop ++resources: ++ - storefront.yaml +diff --git a/apps/shop/storefront.yaml b/apps/shop/storefront.yaml +new file mode 100644 +--- /dev/null ++++ b/apps/shop/storefront.yaml +@@ -0,0 +1,11 @@ ++apiVersion: config.shop.example/v1alpha1 ++kind: ShopConfiguration ++metadata: ++ name: storefront ++spec: ++ catalog: ++ defaultCurrency: EUR ++ checkout: ++ sessionTimeout: 15m ++ theme: ++ primaryColor: teal diff --git a/docs/layout/examples/empty-repo-bootstrap/input/storefront.yaml b/docs/layout/examples/empty-repo-bootstrap/input/storefront.yaml new file mode 100644 index 00000000..3fea2632 --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/input/storefront.yaml @@ -0,0 +1,15 @@ +# Abridged: a captured object also carries uid, resourceVersion, generation, +# creationTimestamp, managedFields and status. See ../homelab-argocd/input/ for a +# fixture that keeps them, and ../README.md for why only two scenarios do. +apiVersion: config.shop.example/v1alpha1 +kind: ShopConfiguration +metadata: + name: storefront + namespace: shop +spec: + catalog: + defaultCurrency: EUR + checkout: + sessionTimeout: 15m + theme: + primaryColor: teal diff --git a/docs/layout/examples/empty-repo-bootstrap/repository/kustomization.yaml b/docs/layout/examples/empty-repo-bootstrap/repository/kustomization.yaml new file mode 100644 index 00000000..334fb27d --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/repository/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: shop +resources: + - storefront.yaml diff --git a/docs/layout/examples/empty-repo-bootstrap/repository/storefront.yaml b/docs/layout/examples/empty-repo-bootstrap/repository/storefront.yaml new file mode 100644 index 00000000..426ab08f --- /dev/null +++ b/docs/layout/examples/empty-repo-bootstrap/repository/storefront.yaml @@ -0,0 +1,11 @@ +apiVersion: config.shop.example/v1alpha1 +kind: ShopConfiguration +metadata: + name: storefront +spec: + catalog: + defaultCurrency: EUR + checkout: + sessionTimeout: 15m + theme: + primaryColor: teal diff --git a/docs/layout/examples/homelab-argocd/README.md b/docs/layout/examples/homelab-argocd/README.md new file mode 100644 index 00000000..d700b856 --- /dev/null +++ b/docs/layout/examples/homelab-argocd/README.md @@ -0,0 +1,94 @@ +# Homelab Argo CD + +This scenario captures a homelab's Argo CD `Application` declarations. The Git target owns one +app-of-apps folder, and the watched objects all live in Argo CD's `argocd` namespace. + +## Repository folder + +[`repository/`](repository/) contains one Kustomize root and two `Application` documents. A new +Application created in the Argo CD UI receives a sibling file such as `paperless.yaml` +and an entry in the root's `resources:` list. + +The tree below shows the target's path in the real repository; +[`repository/`](repository/) in this folder **is** `bootstrap/argocd-applications/`. + +```text +bootstrap/argocd-applications/ + kustomization.yaml + application-jellyfin.yaml + application-nextcloud.yaml +``` + +The folder is deliberately narrow. It contains the Argo CD declarations that tell Argo what to +deploy; it does not contain the Deployments, Services, or chart-rendered objects that Argo creates. + +## Proposed configuration + +[`config/gittarget.yaml`](config/gittarget.yaml) pairs `serializeNamespace: Never` with +`kustomizeRoot: Require`, and constrains the source to one exact namespace. +[`config/watchrule.yaml`](config/watchrule.yaml) subscribes only to `argoproj.io` `Application` +objects. The folder's kustomize root supplies the namespace, so the Application files omit +`metadata.namespace`. + +**`Require` is doing real work here.** The root that makes the omission safe is +[`repository/kustomization.yaml`](repository/kustomization.yaml) — a file the *repository owner* +controls, not the operator. Delete it, or delete its `namespace: argocd` line, and every subsequent +document lands in whatever namespace the applier happens to be pointed at, which is a different +object with the same name. `Require` turns that into a refusal with a condition instead of a silent +relocation. `Adopt` would keep writing. + +The scenario makes the write boundary easy to inspect: every new captured application has one +writable home in `bootstrap/argocd-applications`. It cannot land in the repository's application +source directories or in an Argo-generated resource. + +## What the input fixture is for + +[`input/paperless.yaml`](input/paperless.yaml) is the object as the +operator receives it from the API server, not the document it wants in Git. It carries `uid`, +`resourceVersion`, `generation`, `creationTimestamp`, `managedFields`, Argo's +`resources-finalizer.argocd.argoproj.io` finalizer, a populated `status`, and the +`argocd.argoproj.io/tracking-id` annotation. + +None of that appears in [`expected-paperless.patch`](expected-paperless.patch), +and the difference between the two files *is* the sanitization assertion. The tracking-id is the one +worth naming: it identifies the Argo CD Application that owns the live object, so a committed copy +makes the document claim ownership on behalf of another Application and hard-fails that +Application's sync. It is denied by exact key rather than by an `argocd.argoproj.io/` prefix strip, +because the rest of that prefix is user data. See +[the tracking-id landmine](../../../spec/e2e-bi-directional-corner.md#the-tracking-id-landmine). + +## Scenario contract + +- Starting repository: [`repository/`](repository/) with the two application declarations shown. +- Live input: [`input/paperless.yaml`](input/paperless.yaml). +- First observation, before any commit: + + ```yaml + status: + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + placement: + renderRoot: . + serializeNamespace: Never + ``` + +- Expected Git change: + [`expected-paperless.patch`](expected-paperless.patch), after a reviewer + clears `suspend`. +- Expected status: `Ready=True` after the root renders the added Application. +- Boundary: only `Application` declarations in `argocd` are eligible. An Argo-created workload has + no writable home in this target. + +## Argo CD behavior + +For a field that both the Argo CD UI and Git can change, the Application's automated sync has +`selfHeal: false`. The Git host also sends a push webhook to Argo CD so a commit is reconciled back +to the cluster. These are the two settings that let one declaration have a live editing path and a +Git reconciliation path; see +[Argo CD and bi-directional GitOps](../../../design/support-boundary/argocd-bi-directional.md). + +This is a declaration-editing scenario. It does not reverse Argo-generated application resources, +nor does it reverse a Helm chart rendered by an Application. diff --git a/docs/layout/examples/homelab-argocd/config/gittarget.yaml b/docs/layout/examples/homelab-argocd/config/gittarget.yaml new file mode 100644 index 00000000..3c4836f2 --- /dev/null +++ b/docs/layout/examples/homelab-argocd/config/gittarget.yaml @@ -0,0 +1,17 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: argocd-applications + namespace: argocd +spec: + providerRef: + name: homelab-repository + branch: main + path: bootstrap/argocd-applications + suspend: true # adoption dry run: scans and publishes, writes nothing + # Never is safe here only because the folder's kustomization.yaml supplies + # namespace: argocd — and that file belongs to the repository owner, not to us. + # Require is the guard: if the root is deleted, stop writing rather than commit + # documents that land in whatever namespace the applier defaults to. + serializeNamespace: Never + kustomizeRoot: Require diff --git a/docs/layout/examples/homelab-argocd/config/watchrule.yaml b/docs/layout/examples/homelab-argocd/config/watchrule.yaml new file mode 100644 index 00000000..e2cdda4c --- /dev/null +++ b/docs/layout/examples/homelab-argocd/config/watchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: argocd-applications + namespace: argocd +spec: + targetRef: + name: argocd-applications + rules: + - apiGroups: ["argoproj.io"] + apiVersions: ["v1alpha1"] + resources: ["applications"] diff --git a/docs/layout/examples/homelab-argocd/expected-paperless.patch b/docs/layout/examples/homelab-argocd/expected-paperless.patch new file mode 100644 index 00000000..572e7f2e --- /dev/null +++ b/docs/layout/examples/homelab-argocd/expected-paperless.patch @@ -0,0 +1,30 @@ +diff --git a/bootstrap/argocd-applications/kustomization.yaml b/bootstrap/argocd-applications/kustomization.yaml +--- a/bootstrap/argocd-applications/kustomization.yaml ++++ b/bootstrap/argocd-applications/kustomization.yaml +@@ -4,3 +4,4 @@ kind: Kustomization + resources: + - application-jellyfin.yaml + - application-nextcloud.yaml ++ - paperless.yaml +diff --git a/bootstrap/argocd-applications/paperless.yaml b/bootstrap/argocd-applications/paperless.yaml +new file mode 100644 +--- /dev/null ++++ b/bootstrap/argocd-applications/paperless.yaml +@@ -0,0 +1,17 @@ ++apiVersion: argoproj.io/v1alpha1 ++kind: Application ++metadata: ++ name: paperless ++spec: ++ project: default ++ source: ++ repoURL: https://github.com/example/homelab.git ++ path: apps/paperless ++ targetRevision: main ++ destination: ++ server: https://kubernetes.default.svc ++ namespace: paperless ++ syncPolicy: ++ automated: ++ prune: true ++ selfHeal: false diff --git a/docs/layout/examples/homelab-argocd/input/paperless.yaml b/docs/layout/examples/homelab-argocd/input/paperless.yaml new file mode 100644 index 00000000..585b9b6c --- /dev/null +++ b/docs/layout/examples/homelab-argocd/input/paperless.yaml @@ -0,0 +1,47 @@ +# The object as the operator receives it from the API server, not as it is written to +# Git. Everything above `spec` except `name` is removed by sanitization, which is why +# expected-application-paperless.patch is shorter than this file: the difference +# between the two is the sanitization assertion. +# +# The tracking-id annotation is the one that matters most here. It identifies the Argo +# CD Application that owns the live object, so committing it would make this document +# claim ownership on behalf of another Application and hard-fail that Application's +# sync. Sanitization denies it by exact key rather than by an `argocd.argoproj.io/` +# prefix strip, because the rest of that prefix is user data. +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: paperless + namespace: argocd + uid: 4d9f1e2c-6a1b-4c9e-9f2a-1b3c5d7e9f01 + resourceVersion: "184203" + generation: 2 + creationTimestamp: "2026-08-12T09:41:07Z" + finalizers: + - resources-finalizer.argocd.argoproj.io + annotations: + argocd.argoproj.io/tracking-id: paperless:argoproj.io/Application:argocd/paperless + managedFields: + - manager: argocd-server + operation: Update + apiVersion: argoproj.io/v1alpha1 + time: "2026-08-12T09:41:07Z" +spec: + project: default + source: + repoURL: https://github.com/example/homelab.git + path: apps/paperless + targetRevision: main + destination: + server: https://kubernetes.default.svc + namespace: paperless + syncPolicy: + automated: + prune: true + selfHeal: false +status: + sync: + status: Synced + revision: 7b1c4a9 + health: + status: Healthy diff --git a/docs/layout/examples/homelab-argocd/repository/application-jellyfin.yaml b/docs/layout/examples/homelab-argocd/repository/application-jellyfin.yaml new file mode 100644 index 00000000..3479d52f --- /dev/null +++ b/docs/layout/examples/homelab-argocd/repository/application-jellyfin.yaml @@ -0,0 +1,17 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: jellyfin +spec: + project: default + source: + repoURL: https://github.com/example/homelab-apps.git + targetRevision: main + path: media/jellyfin + destination: + server: https://kubernetes.default.svc + namespace: media + syncPolicy: + automated: + prune: true + selfHeal: false diff --git a/docs/layout/examples/homelab-argocd/repository/application-nextcloud.yaml b/docs/layout/examples/homelab-argocd/repository/application-nextcloud.yaml new file mode 100644 index 00000000..93d35afa --- /dev/null +++ b/docs/layout/examples/homelab-argocd/repository/application-nextcloud.yaml @@ -0,0 +1,17 @@ +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: nextcloud +spec: + project: default + source: + repoURL: https://github.com/example/homelab-apps.git + targetRevision: main + path: storage/nextcloud + destination: + server: https://kubernetes.default.svc + namespace: storage + syncPolicy: + automated: + prune: true + selfHeal: false diff --git a/docs/layout/examples/homelab-argocd/repository/kustomization.yaml b/docs/layout/examples/homelab-argocd/repository/kustomization.yaml new file mode 100644 index 00000000..502b770f --- /dev/null +++ b/docs/layout/examples/homelab-argocd/repository/kustomization.yaml @@ -0,0 +1,6 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: argocd +resources: + - application-jellyfin.yaml + - application-nextcloud.yaml diff --git a/docs/layout/examples/homelab-flux/README.md b/docs/layout/examples/homelab-flux/README.md new file mode 100644 index 00000000..fc0f42e9 --- /dev/null +++ b/docs/layout/examples/homelab-flux/README.md @@ -0,0 +1,130 @@ +# Homelab Flux + +This scenario captures Flux declarations in the `flux-system` namespace. A homelab owner can edit a +`GitRepository`, `HelmRepository`, or `HelmRelease` through the Kubernetes API and review the +resulting Git change without asking GitOps Reverser to reverse a chart. + +## The rule this scenario exists to demonstrate + +> A `GitTarget` must not point at a path another controller writes. Flux's bootstrap directory +> (`clusters//flux-system`) is the common case, and the `Kustomization` that reconciles a +> folder is not a licence to co-write it. + +`clusters/home/flux-system` is owned by `flux bootstrap` and by flux-operator: it holds +`gotk-components.yaml`, `gotk-sync.yaml`, and a `kustomization.yaml` listing both. An operator that +adds `resources:` entries there is a second writer in a folder Flux's own sync loop reconciles, +which is the two-writers-one-folder failure the +[support contract](../../../design/support-boundary/support-contract.md) exists to prevent. The +targets below point somewhere else, and the bootstrap directory appears in the tree only to be left +alone. + +## Repository folder + +[`repository/`](repository/) is rooted at the repository root rather than at one target's path, +because this scenario has two targets on two layers: + +```text +clusters/home/flux-system/ # flux bootstrap owns this. GitOps Reverser never writes here. + gotk-components.yaml + gotk-sync.yaml + kustomization.yaml + +infrastructure/home/sources/ # spec.path of the flux-sources GitTarget + kustomization.yaml + gitrepository-homelab.yaml + helmrepository-jellyfin.yaml + +apps/home/media/ # spec.path of the flux-media GitTarget + kustomization.yaml + helmrelease-jellyfin.yaml +``` + +The bootstrap directory is shown but not committed to this fixture, because nothing in the scenario +reads or writes it. The split into `infrastructure/` and `apps/` is the shape Flux's documented +repository structures produce, and keeping the cluster layer separate from the application layer is +what those structures exist to achieve. + +For a newly created Flux declaration, the folder's kustomize root places a sibling file and +registers it in `resources:`. It does not append the declaration to an existing bundle, because a +bundle is only produced by a template that deliberately collides two identities on one path, and +none is declared here. + +**The update case is the more interesting half.** When the `jellyfin` `HelmRepository` that already +lives in the folder changes, it is not moved and not re-placed: match-first identity finds the +document where it is and edits it in place, including when it shares a file with other documents. +So "we do not author bundles" and "we do not move existing files" coexist — placement decides where +a *new* document goes and has no opinion about one that already exists. A user who consolidates two +declarations into one file keeps that choice; the operator will edit inside it forever. + +## Proposed configuration + +Two targets, one per layer: + +| Target | Path | Watches | +|---|---|---| +| [`config/gittarget.yaml`](config/gittarget.yaml) | `infrastructure/home/sources` | `GitRepository`, `HelmRepository` ([`config/watchrule.yaml`](config/watchrule.yaml)) | +| [`config/gittarget-media.yaml`](config/gittarget-media.yaml) | `apps/home/media` | `HelmRelease` ([`config/watchrule-media.yaml`](config/watchrule-media.yaml)) | + +Both declare a one-root Kustomize folder with one source namespace, and both open suspended. Each +folder's `namespace: flux-system` transformer supplies the namespace for every +document in it, so the committed documents omit `metadata.namespace`. + +The `HelmRelease` object living in `flux-system` does not put jellyfin there: +[`helmrelease-jellyfin.yaml`](repository/apps/home/media/helmrelease-jellyfin.yaml) carries +`targetNamespace: media` and `storageNamespace: flux-system`. The declaration and the release it +installs are on different sides of that field, which is exactly why the declaration is the editable +surface and the release is not. + +## What the input fixture is for + +[`input/bitnami.yaml`](input/bitnami.yaml) is the object as the +operator receives it from the API server, not the document it wants in Git. It carries `uid`, +`resourceVersion`, `generation`, `creationTimestamp`, `managedFields`, a populated `status`, and +Flux's `finalizers.fluxcd.io` finalizer. + +None of that reaches +[`expected-bitnami.patch`](expected-bitnami.patch), and the difference +between the two files *is* the sanitization assertion. The finalizer is the one worth naming: +`finalizers.fluxcd.io` is how source-controller keeps the object alive long enough to garbage-collect +its artifact storage. A committed copy would make Git assert a finalizer against an object Flux has +not adopted yet, so deleting a recreated object would block on a controller that never put it there. + +## Scenario contract + +- Starting repository: [`repository/`](repository/) with its existing sources and media folders. +- Live input: [`input/bitnami.yaml`](input/bitnami.yaml). +- First observation, before any commit: + + ```yaml + status: + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + placement: + renderRoot: . + serializeNamespace: Never + ``` + +- Expected Git change: + [`expected-bitnami.patch`](expected-bitnami.patch), after a reviewer clears `suspend` on + `flux-sources`. +- Expected status: `Ready=True` after the root renders the new declaration. +- Boundary: a rendered object has no writable home; only selected Flux declarations can produce a + Git change. + +The configuration captures the layer a person would edit in Git: + +- a `GitRepository` or `HelmRepository` changes where Flux obtains source material; +- a `HelmRelease` changes a chart version, source reference, or inline values; +- the chart's rendered Deployments, Services, and Secrets remain expansion output. + +## Boundary + +This is Flux declaration editing, not Helm inversion. A chart folder is skipped as a unit, and the +operator never turns a rendered Deployment edit into a speculative values change. The current +[support contract](../../../design/support-boundary/support-contract.md) owns that boundary. + +A free-standing values file is a separate planned projection, so it is absent from this first +scenario. Inline `HelmRelease.spec.values` are KRM and stay inside the declaration surface. diff --git a/docs/layout/examples/homelab-flux/config/gittarget-media.yaml b/docs/layout/examples/homelab-flux/config/gittarget-media.yaml new file mode 100644 index 00000000..82f61a10 --- /dev/null +++ b/docs/layout/examples/homelab-flux/config/gittarget-media.yaml @@ -0,0 +1,17 @@ +# The second target. HelmReleases are a different layer from the sources they pull +# from, so they get their own folder and their own GitTarget rather than sharing one +# with the sources — which is the shape Flux's documented repository structures exist +# to produce. +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: flux-media + namespace: flux-system +spec: + providerRef: + name: homelab-repository + branch: main + path: apps/home/media + suspend: true # adoption dry run: scans and publishes, writes nothing + serializeNamespace: Never + kustomizeRoot: Require diff --git a/docs/layout/examples/homelab-flux/config/gittarget.yaml b/docs/layout/examples/homelab-flux/config/gittarget.yaml new file mode 100644 index 00000000..bbfb0aab --- /dev/null +++ b/docs/layout/examples/homelab-flux/config/gittarget.yaml @@ -0,0 +1,15 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: flux-sources + namespace: flux-system +spec: + providerRef: + name: homelab-repository + branch: main + # Not clusters/home/flux-system. That directory belongs to `flux bootstrap`, and a + # GitTarget must not point at a path another controller writes. + path: infrastructure/home/sources + suspend: true # adoption dry run: scans and publishes, writes nothing + serializeNamespace: Never + kustomizeRoot: Require diff --git a/docs/layout/examples/homelab-flux/config/watchrule-media.yaml b/docs/layout/examples/homelab-flux/config/watchrule-media.yaml new file mode 100644 index 00000000..e236aa0e --- /dev/null +++ b/docs/layout/examples/homelab-flux/config/watchrule-media.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: flux-media + namespace: flux-system +spec: + targetRef: + name: flux-media + rules: + - apiGroups: ["helm.toolkit.fluxcd.io"] + apiVersions: ["v2"] + resources: ["helmreleases"] diff --git a/docs/layout/examples/homelab-flux/config/watchrule.yaml b/docs/layout/examples/homelab-flux/config/watchrule.yaml new file mode 100644 index 00000000..5f04fa39 --- /dev/null +++ b/docs/layout/examples/homelab-flux/config/watchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: flux-sources + namespace: flux-system +spec: + targetRef: + name: flux-sources + rules: + - apiGroups: ["source.toolkit.fluxcd.io"] + apiVersions: ["v1"] + resources: ["gitrepositories", "helmrepositories"] diff --git a/docs/layout/examples/homelab-flux/expected-bitnami.patch b/docs/layout/examples/homelab-flux/expected-bitnami.patch new file mode 100644 index 00000000..0c05e0a4 --- /dev/null +++ b/docs/layout/examples/homelab-flux/expected-bitnami.patch @@ -0,0 +1,21 @@ +diff --git a/infrastructure/home/sources/kustomization.yaml b/infrastructure/home/sources/kustomization.yaml +--- a/infrastructure/home/sources/kustomization.yaml ++++ b/infrastructure/home/sources/kustomization.yaml +@@ -3,4 +3,5 @@ kind: Kustomization + namespace: flux-system + resources: + - gitrepository-homelab.yaml + - helmrepository-jellyfin.yaml ++ - bitnami.yaml +diff --git a/infrastructure/home/sources/bitnami.yaml b/infrastructure/home/sources/bitnami.yaml +new file mode 100644 +--- /dev/null ++++ b/infrastructure/home/sources/bitnami.yaml +@@ -0,0 +1,7 @@ ++apiVersion: source.toolkit.fluxcd.io/v1 ++kind: HelmRepository ++metadata: ++ name: bitnami ++spec: ++ interval: 1h ++ url: https://charts.bitnami.com/bitnami diff --git a/docs/layout/examples/homelab-flux/input/bitnami.yaml b/docs/layout/examples/homelab-flux/input/bitnami.yaml new file mode 100644 index 00000000..2a7e7e2b --- /dev/null +++ b/docs/layout/examples/homelab-flux/input/bitnami.yaml @@ -0,0 +1,40 @@ +# The object as the operator receives it from the API server. Everything above `spec` +# except `name` is removed by sanitization, so the committed document in +# expected-helmrepository-bitnami.patch is much shorter than this file: the difference +# between the two is the sanitization assertion. +# +# The Flux finalizer is the one worth naming. `finalizers.fluxcd.io` is how +# source-controller keeps the object alive long enough to garbage-collect its artifact +# storage. A committed copy would make Git assert a finalizer against an object Flux +# has not adopted yet, so a `flux delete` of a recreated object would block on a +# controller that never put it there. +apiVersion: source.toolkit.fluxcd.io/v1 +kind: HelmRepository +metadata: + name: bitnami + namespace: flux-system + uid: 8c2a71f4-3d5e-4b18-9c07-2e6f1a4b8d33 + resourceVersion: "902441" + generation: 1 + creationTimestamp: "2026-08-19T14:02:55Z" + finalizers: + - finalizers.fluxcd.io + managedFields: + - manager: flux + operation: Apply + apiVersion: source.toolkit.fluxcd.io/v1 + time: "2026-08-19T14:02:55Z" +spec: + interval: 1h + url: https://charts.bitnami.com/bitnami +status: + observedGeneration: 1 + artifact: + revision: sha256:3f9c1e0d + lastUpdateTime: "2026-08-19T14:03:02Z" + conditions: + - type: Ready + status: "True" + reason: Succeeded + message: stored artifact for revision 'sha256:3f9c1e0d' + lastTransitionTime: "2026-08-19T14:03:02Z" diff --git a/docs/layout/examples/homelab-flux/repository/apps/home/media/helmrelease-jellyfin.yaml b/docs/layout/examples/homelab-flux/repository/apps/home/media/helmrelease-jellyfin.yaml new file mode 100644 index 00000000..b3f7f8ae --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/apps/home/media/helmrelease-jellyfin.yaml @@ -0,0 +1,25 @@ +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: jellyfin +spec: + interval: 30m + # The HelmRelease object lives in flux-system with the rest of the Flux + # declarations; the release it installs does not. targetNamespace is what puts + # jellyfin in `media`, and storageNamespace keeps its Helm storage beside the + # controller that manages it. + targetNamespace: media + storageNamespace: flux-system + chart: + spec: + chart: jellyfin + version: 2.1.0 + sourceRef: + kind: HelmRepository + name: jellyfin + values: + persistence: + config: + enabled: true + service: + type: ClusterIP diff --git a/docs/layout/examples/homelab-flux/repository/apps/home/media/kustomization.yaml b/docs/layout/examples/homelab-flux/repository/apps/home/media/kustomization.yaml new file mode 100644 index 00000000..8bcda0c3 --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/apps/home/media/kustomization.yaml @@ -0,0 +1,5 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: flux-system +resources: + - helmrelease-jellyfin.yaml diff --git a/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/gitrepository-homelab.yaml b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/gitrepository-homelab.yaml new file mode 100644 index 00000000..bdb644d8 --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/gitrepository-homelab.yaml @@ -0,0 +1,7 @@ +apiVersion: source.toolkit.fluxcd.io/v1 +kind: GitRepository +metadata: + name: homelab +spec: + interval: 1m + url: https://github.com/example/homelab.git diff --git a/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/helmrepository-jellyfin.yaml b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/helmrepository-jellyfin.yaml new file mode 100644 index 00000000..5d6cc27d --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/helmrepository-jellyfin.yaml @@ -0,0 +1,7 @@ +apiVersion: source.toolkit.fluxcd.io/v1 +kind: HelmRepository +metadata: + name: jellyfin +spec: + interval: 1h + url: https://jellyfin.github.io/jellyfin-helm diff --git a/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/kustomization.yaml b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/kustomization.yaml new file mode 100644 index 00000000..3abed039 --- /dev/null +++ b/docs/layout/examples/homelab-flux/repository/infrastructure/home/sources/kustomization.yaml @@ -0,0 +1,6 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: flux-system +resources: + - gitrepository-homelab.yaml + - helmrepository-jellyfin.yaml diff --git a/docs/layout/examples/overlay-scoped-target/README.md b/docs/layout/examples/overlay-scoped-target/README.md new file mode 100644 index 00000000..411ffd17 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/README.md @@ -0,0 +1,75 @@ +# Overlay-scoped target + +> **This scenario exercises the write path, not placement.** Its expected patch edits an `images:` +> transformer in a file that already exists: no file is placed, no `resources:` entry is added, and +> the placement ladder never runs. It sits here because the question it answers — what a target may +> write when its render root reads a folder it does not own — is the one people ask immediately after +> the placement question, and because it is the scenario that proves `spec.path` is the write +> boundary. A corpus harness needs a separate entry point for it: seeding a worktree and comparing a +> patch is the same, but the event is a field change rather than a new document. +> +> The folder was called `external-base-overlay`. The base is in the same repository one directory +> up, so nothing about it is external — and "external base" names precisely what the Kustomize +> support boundary refuses, a remote base. The name now says what the scenario is *for*. + +This scenario gives one environment overlay a writable kustomize root while treating its shared +base as read-only input. It captures a Deployment change that the overlay can express as an image +tag update. + +## Starting repository + +[`repository/`](repository/) contains a reusable `podinfo` base and a production overlay: + +```text +apps/podinfo/ + base/ + deployment.yaml + kustomization.yaml + overlays/prod/ + kustomization.yaml +``` + +The target path is `apps/podinfo/overlays/prod`. Its Kustomize root includes `../../base`, but the +base is an input to rendering. It is outside the target's write scope. + +## Proposed configuration + +[`config/gittarget.yaml`](config/gittarget.yaml) declares one `Kustomize` root for the +`podinfo-prod` namespace. [`config/watchrule.yaml`](config/watchrule.yaml) captures Deployments. +The scenario assumes the Deployment image is a supported writable expression in the overlay's +`images:` transformer. + +## Scenario contract + +- Starting repository: + [`repository/apps/podinfo/overlays/prod/`](repository/apps/podinfo/overlays/prod/). +- Live input: [`input/deployment-podinfo.yaml`](input/deployment-podinfo.yaml). +- First observation, before any commit: + + ```yaml + status: + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization + message: "render root '.' governs new files" + placement: + renderRoot: . + serializeNamespace: Never + ``` + +- Expected Git change: [`expected-image-update.patch`](expected-image-update.patch), after a + reviewer clears `suspend`. +- Expected status: `Ready=True` after the changed overlay renders successfully. +- Boundary: the target may read `../../base` to render, but it never writes outside the overlay. + +## Boundary + +The target owns the overlay and does not own a shared base. A request that needs to change a base +field is refused as outside `spec.path`; it does not search for another location with the same +Deployment identity. If the overlay cannot express a live field change through a supported +transformer, the operator reports a refusal rather than editing the rendered base. + +The exact support boundary remains the +[Kustomize support contract](../../../design/support-boundary/support-contract.md). This example narrows its +write scope further than that contract: rendering may cross into the base, but writing does not. diff --git a/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml b/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml new file mode 100644 index 00000000..a8ece29d --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/config/gittarget.yaml @@ -0,0 +1,13 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: podinfo-prod-overlay + namespace: podinfo-prod +spec: + providerRef: + name: homelab-repository + branch: main + path: apps/podinfo/overlays/prod + suspend: true # adoption dry run: scans and publishes, writes nothing + serializeNamespace: Never + kustomizeRoot: Require diff --git a/docs/layout/examples/overlay-scoped-target/config/watchrule.yaml b/docs/layout/examples/overlay-scoped-target/config/watchrule.yaml new file mode 100644 index 00000000..212617ed --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/config/watchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: podinfo-prod-overlay-content + namespace: podinfo-prod +spec: + targetRef: + name: podinfo-prod-overlay + rules: + - apiGroups: ["apps"] + apiVersions: ["v1"] + resources: ["deployments"] diff --git a/docs/layout/examples/overlay-scoped-target/expected-image-update.patch b/docs/layout/examples/overlay-scoped-target/expected-image-update.patch new file mode 100644 index 00000000..ed9de3a3 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/expected-image-update.patch @@ -0,0 +1,9 @@ +diff --git a/apps/podinfo/overlays/prod/kustomization.yaml b/apps/podinfo/overlays/prod/kustomization.yaml +--- a/apps/podinfo/overlays/prod/kustomization.yaml ++++ b/apps/podinfo/overlays/prod/kustomization.yaml +@@ -5,4 +5,4 @@ resources: + - ../../base + images: + - name: ghcr.io/stefanprodan/podinfo +- newTag: 6.6.0 ++ newTag: 6.6.1 diff --git a/docs/layout/examples/overlay-scoped-target/input/deployment-podinfo.yaml b/docs/layout/examples/overlay-scoped-target/input/deployment-podinfo.yaml new file mode 100644 index 00000000..107b5933 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/input/deployment-podinfo.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: podinfo + namespace: podinfo-prod +spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: podinfo + template: + metadata: + labels: + app.kubernetes.io/name: podinfo + spec: + containers: + - name: podinfo + image: ghcr.io/stefanprodan/podinfo:6.6.1 + ports: + - containerPort: 9898 diff --git a/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/deployment.yaml b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/deployment.yaml new file mode 100644 index 00000000..f2fce614 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/deployment.yaml @@ -0,0 +1,19 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: podinfo +spec: + replicas: 2 + selector: + matchLabels: + app.kubernetes.io/name: podinfo + template: + metadata: + labels: + app.kubernetes.io/name: podinfo + spec: + containers: + - name: podinfo + image: ghcr.io/stefanprodan/podinfo:6.5.3 + ports: + - containerPort: 9898 diff --git a/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/kustomization.yaml b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/kustomization.yaml new file mode 100644 index 00000000..9c2d28b0 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/base/kustomization.yaml @@ -0,0 +1,4 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - deployment.yaml diff --git a/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/overlays/prod/kustomization.yaml b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/overlays/prod/kustomization.yaml new file mode 100644 index 00000000..01f1b240 --- /dev/null +++ b/docs/layout/examples/overlay-scoped-target/repository/apps/podinfo/overlays/prod/kustomization.yaml @@ -0,0 +1,8 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: podinfo-prod +resources: + - ../../base +images: + - name: ghcr.io/stefanprodan/podinfo + newTag: 6.6.0 diff --git a/docs/layout/examples/prerequisites/README.md b/docs/layout/examples/prerequisites/README.md new file mode 100644 index 00000000..f3f66800 --- /dev/null +++ b/docs/layout/examples/prerequisites/README.md @@ -0,0 +1,38 @@ +# Shared prerequisites + +Every `GitTarget` refers to a `GitProvider` in the same namespace. The provider owns the repository +connection and writable branch policy; the target owns the folder and its layout. This separation +lets one repository serve several independently scoped targets. + +[`config/gitprovider.yaml`](config/gitprovider.yaml) is a minimal SSH provider specimen for the +`demo` namespace. Copy it into each namespace that owns a target, change its name and repository +URL, and provide the referenced credentials and `known_hosts` objects. A public HTTPS repository +can omit `secretRef` and `knownHostsRef`. + +```text +namespace demo + GitProvider/app-repository + GitTarget/demo -> GitProvider/app-repository +``` + +The provider is namespace-local because it carries credentials. A `GitTarget` cannot refer to a +provider in another namespace. + +## Source cluster authority + +The ordinary examples use the operator's in-cluster source. A target that captures objects from a +different source namespace needs its `ClusterProvider` to delegate that override. The cluster-tree +example includes a concrete +[`ClusterProvider`](../tree-multi-namespace/config/clusterprovider.yaml) with +`allowSourceNamespaceOverride: true`. + +The source-cluster rule and source-namespace authorization are separate checks. The +`ClusterProvider` decides which configuration namespaces can choose source namespaces. The +`GitTarget` decides which of those source namespaces may write into its folder. + +## Credentials stay out of this design + +This example names a credentials Secret and `known_hosts` ConfigMap but does not include either +object. Repository credentials and SSH host keys are operational inputs, not a layout convention. +The current setup contract is in +[GitProvider configuration](../../../configuration.md#gitprovider). diff --git a/docs/layout/examples/prerequisites/config/gitprovider.yaml b/docs/layout/examples/prerequisites/config/gitprovider.yaml new file mode 100644 index 00000000..d189a555 --- /dev/null +++ b/docs/layout/examples/prerequisites/config/gitprovider.yaml @@ -0,0 +1,16 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitProvider +metadata: + name: app-repository + namespace: demo +spec: + url: ssh://git@example.com/organization/application-config.git + secretRef: + name: app-repository-credentials + knownHostsRef: + name: example-com-known-hosts + allowedBranches: + - main + commit: + author: + name: GitOps Reverser diff --git a/docs/layout/examples/tree-multi-namespace/README.md b/docs/layout/examples/tree-multi-namespace/README.md new file mode 100644 index 00000000..15e53df4 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/README.md @@ -0,0 +1,93 @@ +# Homelab cluster tree + +This scenario mirrors a small home cluster into one repository subtree. The operator captures a +few namespaced resources from two namespaces and one cluster-scoped resource. It does not need a +Kustomize root to decide where a new object goes. + +## Resulting repository folder + +[`repository/`](repository/) shows the structure after three writes. The tree below uses the +target's path in the real repository; `repository/` in this folder **is** `clusters/home/`. + +```text +clusters/home/ + _cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml + media/apps/deployments/jellyfin.yaml + media/configmaps/jellyfin.yaml + monitoring/configmaps/grafana.ini.yaml +``` + +`Tree` derives each path from the object's namespace or cluster scope, API group, resource, and +name. A Deployment named `jellyfin` in another namespace receives another path. This property is +why `Tree` is the safe default for multi-namespace capture. + +Two conventions in those paths are load-bearing, and both come from the canonical grammar in +[`new-file-placement-rules.md`](../../new-file-placement-rules.md#template-variables): + +- **The core group collapses to nothing.** `configmaps` sits directly under the namespace segment, + where `rbac.authorization.k8s.io` sits in the ClusterRole path, because the core group has no + name to write. +- **`_cluster` stands in for the namespace segment** of a cluster-scoped resource. An underscore is + invalid in a namespace name, so the sentinel cannot collide with a real namespace — which is why + it is a sentinel and not a reserved word. + +## Proposed configuration + +[`config/clusterprovider.yaml`](config/clusterprovider.yaml) gives the homelab configuration +namespace permission to choose its source namespaces. The target's +[`config/gittarget.yaml`](config/gittarget.yaml) names those namespaces and declares `Tree` plus +`MultiNamespace`. [`config/watchrule.yaml`](config/watchrule.yaml) selects namespaced content, while +[`config/clusterwatchrule.yaml`](config/clusterwatchrule.yaml) separately selects the ClusterRole. + +The separation is intentional. A `WatchRule` owns namespaced subscriptions. A +`ClusterWatchRule` owns cluster-scoped subscriptions. The folder has room for both because its +paths carry scope, but the source API keeps their authority separate. + +## Scenario contract + +- Starting repository: [`repository/`](repository/) after the three documents shown above. +- Live input: [`input/grafana-dashboards.yaml`](input/grafana-dashboards.yaml). +- First observation, before any commit. A `Tree` folder has no render root, so the reason says so + rather than leaving the field unexplained: + + ```yaml + status: + conditions: + - type: LayoutResolved + status: "True" + reason: None + message: "no kustomization governs this subtree; new files take the canonical path" + placement: + renderRoot: "" + serializeNamespace: Always + ``` + +- Expected Git change: + [`expected-grafana-dashboards.patch`](expected-grafana-dashboards.patch), after + a reviewer clears `suspend`. +- Expected status: `Ready=True` with a `Tree` layout and a path derived from the input identity. +- Boundary: content outside `clusters/home`, a namespace not on the target, or an unsupported + type is refused without probing for a sibling convention. + +## Homelab use + +This is a useful first mirror for a single-owner cluster: changes made with `kubectl`, a dashboard, +or an operator appear as reviewable Git documents without choosing a bundle convention first. A +GitOps deployer can consume the tree through a recursively scanned directory or through a root +Kustomization that the repository owner maintains. + +`serializeNamespace: Always` keeps each namespaced document portable outside this tree. Unlike a +single-namespace Kustomize folder, no common namespace transformer supplies an omitted value. + +**The field governs namespaced resources only.** This folder also captures a `ClusterRole` through +its `ClusterWatchRule`, and a `ClusterRole` has no namespace to serialize, so +`serializeNamespace` is ignored for it rather than being an error. A tree is the shape most likely +to carry both kinds, which is why it is worth saying here. + +## Boundary + +The target has one write scope, `clusters/home`. It does not use one target for a shared base and +several environments. The per-overlay model remains the right choice when a repository has bases +and environment-specific overlays; see [GitTarget granularity]. + +[GitTarget granularity]: ../../../design/support-boundary/gittarget-granularity-and-cross-environment-edits.md diff --git a/docs/layout/examples/tree-multi-namespace/config/clusterprovider.yaml b/docs/layout/examples/tree-multi-namespace/config/clusterprovider.yaml new file mode 100644 index 00000000..576d7d64 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/config/clusterprovider.yaml @@ -0,0 +1,9 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: ClusterProvider +metadata: + name: home +spec: + allowedNamespaces: + names: [homelab-config] + # The WatchRule selects source namespaces other than its own namespace. + allowSourceNamespaceOverride: true diff --git a/docs/layout/examples/tree-multi-namespace/config/clusterwatchrule.yaml b/docs/layout/examples/tree-multi-namespace/config/clusterwatchrule.yaml new file mode 100644 index 00000000..dcaba72d --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/config/clusterwatchrule.yaml @@ -0,0 +1,12 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: ClusterWatchRule +metadata: + name: home-cluster-rbac +spec: + targetRef: + name: home-cluster-state + namespace: homelab-config + rules: + - apiGroups: ["rbac.authorization.k8s.io"] + apiVersions: ["v1"] + resources: ["clusterroles"] diff --git a/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml b/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml new file mode 100644 index 00000000..b0e50b76 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/config/gittarget.yaml @@ -0,0 +1,18 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: GitTarget +metadata: + name: home-cluster-state + namespace: homelab-config +spec: + providerRef: + name: homelab-repository + clusterProviderRef: + name: home + branch: main + path: clusters/home + suspend: true # adoption dry run: scans and publishes, writes nothing + # No placement declared and no kustomization anywhere in the subtree, so the + # ladder falls through to the canonical identity path. Nothing to register, so + # Adopt does nothing; the documents must carry their own namespace. + serializeNamespace: Always + kustomizeRoot: Adopt diff --git a/docs/layout/examples/tree-multi-namespace/config/watchrule.yaml b/docs/layout/examples/tree-multi-namespace/config/watchrule.yaml new file mode 100644 index 00000000..e3d39468 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/config/watchrule.yaml @@ -0,0 +1,21 @@ +apiVersion: configbutler.ai/v1alpha3 +kind: WatchRule +metadata: + name: home-workloads + namespace: homelab-config +spec: + targetRef: + name: home-cluster-state + rules: + - apiGroups: [""] + apiVersions: ["v1"] + resources: ["configmaps"] + sourceNamespace: media + - apiGroups: ["apps"] + apiVersions: ["v1"] + resources: ["deployments"] + sourceNamespace: media + - apiGroups: [""] + apiVersions: ["v1"] + resources: ["configmaps"] + sourceNamespace: monitoring diff --git a/docs/layout/examples/tree-multi-namespace/expected-grafana-dashboards.patch b/docs/layout/examples/tree-multi-namespace/expected-grafana-dashboards.patch new file mode 100644 index 00000000..63a0d1fb --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/expected-grafana-dashboards.patch @@ -0,0 +1,12 @@ +diff --git a/clusters/home/monitoring/configmaps/grafana-dashboards.yaml b/clusters/home/monitoring/configmaps/grafana-dashboards.yaml +new file mode 100644 +--- /dev/null ++++ b/clusters/home/monitoring/configmaps/grafana-dashboards.yaml +@@ -0,0 +1,7 @@ ++apiVersion: v1 ++kind: ConfigMap ++metadata: ++ name: grafana-dashboards ++ namespace: monitoring ++data: ++ dashboard.json: '{"title":"Home"}' diff --git a/docs/layout/examples/tree-multi-namespace/input/grafana-dashboards.yaml b/docs/layout/examples/tree-multi-namespace/input/grafana-dashboards.yaml new file mode 100644 index 00000000..9880ac84 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/input/grafana-dashboards.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: grafana-dashboards + namespace: monitoring +data: + dashboard.json: '{"title":"Home"}' diff --git a/docs/layout/examples/tree-multi-namespace/repository/_cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml b/docs/layout/examples/tree-multi-namespace/repository/_cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml new file mode 100644 index 00000000..e7131314 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/repository/_cluster/rbac.authorization.k8s.io/clusterroles/homelab-viewer.yaml @@ -0,0 +1,8 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: homelab-viewer +rules: + - apiGroups: [""] + resources: ["pods", "services"] + verbs: ["get", "list", "watch"] diff --git a/docs/layout/examples/tree-multi-namespace/repository/media/apps/deployments/jellyfin.yaml b/docs/layout/examples/tree-multi-namespace/repository/media/apps/deployments/jellyfin.yaml new file mode 100644 index 00000000..c3dd580f --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/repository/media/apps/deployments/jellyfin.yaml @@ -0,0 +1,17 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: jellyfin + namespace: media +spec: + selector: + matchLabels: + app: jellyfin + template: + metadata: + labels: + app: jellyfin + spec: + containers: + - name: jellyfin + image: jellyfin/jellyfin:10.10.7 diff --git a/docs/layout/examples/tree-multi-namespace/repository/media/configmaps/jellyfin.yaml b/docs/layout/examples/tree-multi-namespace/repository/media/configmaps/jellyfin.yaml new file mode 100644 index 00000000..cb7b463d --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/repository/media/configmaps/jellyfin.yaml @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: jellyfin + namespace: media +data: + PUID: "1000" + PGID: "1000" diff --git a/docs/layout/examples/tree-multi-namespace/repository/monitoring/configmaps/grafana.ini.yaml b/docs/layout/examples/tree-multi-namespace/repository/monitoring/configmaps/grafana.ini.yaml new file mode 100644 index 00000000..c3fc0ed5 --- /dev/null +++ b/docs/layout/examples/tree-multi-namespace/repository/monitoring/configmaps/grafana.ini.yaml @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: grafana.ini + namespace: monitoring +data: + server.root_url: https://grafana.home.arpa + users.allow_sign_up: "false" diff --git a/docs/layout/implementation-plan.md b/docs/layout/implementation-plan.md new file mode 100644 index 00000000..19246e41 --- /dev/null +++ b/docs/layout/implementation-plan.md @@ -0,0 +1,343 @@ +# Building the placement work: the order + +> **plan**: the implementation order for [`model.md`](model.md) +> and [`api-wave.md`](api-wave.md), under the direction recorded in +> [`../future/direction-and-configuration-surface.md`](../future/direction-and-configuration-surface.md). +> Date: 2026-08-28. Index: [`../INDEX.md`](../INDEX.md). +> +> The designs it sequences remain proposals; this page is the sequencing decision and the +> definition of done for each step. Rewritten after the model reversed: the placement work is no +> longer breaking, so the order it once protected — everything queued behind one coordinated bump — +> mostly dissolved. + +## Why the order carries less risk than it did + +The placement work reads like a rewrite and never was one. The four-rung ladder is a single +function, [`LocateNew`](../../internal/manifestanalyzer/placement.go), with a single caller in +[`plan_flush.go`](../../internal/git/plan_flush.go). Everything downstream of the path decision — +registering a new file with the kustomization that governs it, the render fidelity gate, refusal +accounting, the metrics — already exists and stays where it is. + +Under the model as it now stands, `LocateNew` is not touched at all. The ladder stays and two fields +sit beside it, so what was the largest breaking change in the queue is a pair of additive fields +whose defaults equal today's behavior. + +That leaves the risk concentrated in one place rather than two: + +- **Review surface**, which is why the corpus in PR 1 matters more than any prose acceptance + criteria. +- The **one coordinated consumer bump** still applies to PRs 5 and 6, which are breaking for reasons + unrelated to placement. The consumer pins the image, the Go module and the `require` line, so each + breaking release is a scheduling event — but placement no longer has to wait for one. + +## The order + +| PR | Content | Breaking | Depends on | +|---|---|---|---| +| 1 | The worked examples as an executable corpus. The [#295](https://github.com/ConfigButler/gitops-reverser/issues/295) correctness fixes it was to carry **shipped** in 0.42.1 | no | — | +| 2 | `spec.suspend`, and the reconcile-request annotation | no | 1 | +| 3 | `status.placement`, and the post-scan validation pass | no | 2 | +| 4 | `serializeNamespace` and `kustomizeRoot` ([#322](https://github.com/ConfigButler/gitops-reverser/issues/322)) | no | 1, 3 | +| 5 | `commitWindow` and `commit.message` move off `GitProvider` | **yes** | — | +| 6 | The riders: the asserted CommitRequest author, its lifecycle, `meta.LocalObjectReference`, `TooManyStreams`, the ClusterProvider default message | **yes** | the source-scope change, for `TooManyStreams` only | + +**PR 4 stopped being breaking**, which is the largest change to this plan since it was written. +The model no longer replaces `spec.placement` with a discriminated union; it keeps the template and +adds two fields whose defaults equal today's behavior. So there is no loud rejection, no `LocateNew` +rewrite, and no migration — and PRs 1 through 4, the whole placement story, ship without a +coordinated consumer bump. + +What remains breaking is PRs 5 and 6, neither of which is about placement. They land in one release, +reviewed as two changes and paid for as one bump, together with the source-scope deletion, which is +not this plan's work but breaks the same object in the same release (design change 7 below). PR 6 is +the trim handle: nothing depends on it. + +`F9`'s envtest against the minimum supported Kubernetes version stays outside this order, as +the wave document says. Its answer constrains the enum work, so run it before PR 4 is planned. + +## PR 1 — an executable corpus + +**The correctness half has shipped.** Both #295 fixes are on `main` and released in 0.42.1 +([#319](https://github.com/ConfigButler/gitops-reverser/pull/319)): a declared path into a +subdirectory is registered with its nearest ancestor kustomization inside the write jail, and +`IdentityCompletePlacementTemplate` no longer demands `{version}`, so the versionless canonical +shape passes our own gate. That is why this plan reversed: the ancestor walk made registration an +invariant, which is what the path template needed to stop being the wrong primitive. + +So PR 1 is the corpus alone, and the shipped fixes become the first thing it asserts rather than +the thing it accompanies. + +### The corpus harness + +[`examples/README.md`](examples/README.md) already has the shape of a golden-file suite — +`repository/`, `config/`, `input/`, `expected-*.patch` — and is read by nobody but a human. +Wiring it up is what stops the design and the implementation drifting, and it converts the +PR 4 review from "does this prose hold together" into "does the diff match the patch". + +The harness has a seam waiting for it: `newWorktreeForTest` and `flushEventsToWorktree` in +[`internal/git`](../../internal/git/placement_test.go) already do exactly this at a smaller +scale. + +Per scenario folder: + +1. Seed a worktree from `repository/` and commit it. +2. Build the write event from `input/`. +3. Derive the flush policy from `config/gittarget.yaml`. +4. Flush, diff the worktree against the seed commit, and compare with `expected-*.patch`. +5. For a scenario carrying `expected-status.yaml` instead of a patch, compare the observation the + post-scan pass produced with that file. Two scenario shapes need this and neither can assert a + patch: a `Never` declaration with no supplier, and an ambiguous folder covering two roots. + +Two details decide whether this stays maintainable: + +- **Normalize the diff.** Blob hashes and index lines are noise; compare paths, modes and + hunks. A helper that renders a canonical patch from the worktree diff keeps the committed + `.patch` files reviewable as documents. +- **Normalize the status the same way.** `expected-status.yaml` holds the placement observation and + the conditions it implies, with the fields that cannot be stable in a fixture dropped before + comparison: `observedRevision` (a commit hash the harness creates), `observedGeneration`, and any + timestamp. Compare the remainder as parsed YAML, not as text, so field order in the fixture is + free. Until PR 3 exists there is nothing to capture, so these scenarios are skipped for it by + name, exactly like the PR 4 cases. +- **Regenerate rather than hand-edit.** A `-update` flag that rewrites the `.patch` files + makes the corpus cheap to extend, and the review of a regenerated patch is the review of + the behaviour change. + +### The skip protocol + +Most scenarios describe behaviour PR 4 introduces. Each such case is written now and skipped +with the PR that will unskip it named in the skip message. **PR 4 is finished when the last +skip is gone.** That is the definition of done this plan is built to produce, and it is worth +more than any prose acceptance criteria. + +The scenarios that assert real behaviour immediately are the ones the shipped fixes cover: +`brownfield-kustomize` (the ancestor walk) and any `byType` case pointing into a subdirectory. They +are regression cover for 0.42.1 as much as scaffolding for what follows. + +### Config parsing without the API + +`config/gittarget.yaml` uses fields that do not exist yet. Parse the scenario config into a +small harness-local struct rather than into `v1alpha3.GitTarget`, and map the fields that do +exist onto today's `PlacementPolicy`. When PR 4 lands, that mapping is deleted and the +fixtures unmarshal into the real type — which is itself a useful check that the API shape the +examples describe is the API shape that got built. + +### Done when + +- `task lint` and `task test` pass; e2e is unaffected. +- Every scenario folder is loaded by the harness; none is silently unreferenced. +- Non-skipped assertions cover both shipped #295 fixes, so the corpus is regression cover from + its first commit. +- Every skipped case names the PR that unskips it. +- The corpus gaps below are filled. + +## PR 2 — `spec.suspend`, and the reconcile-request annotation + +Additive, and the wave's order is right that `suspend` comes before anything that creates files. + +- `suspend` stops resource writes **and** bootstrap creation. State that before either exists. +- `suspend` does **not** stop scanning: a suspended target still resolves its render root and + publishes `status.placement`, which is what makes it a dry run rather than an off switch. This + deviates from Flux, where `suspend` stops reconciliation altogether, so the field's documentation + says so in one sentence. +- `reconcile.configbutler.ai/requestedAt` plus `status.lastHandledReconcileAt` refresh that + observation on demand. + +**`spec.mode: Observe|Write` and `spec.interval` were both fields here and are both dropped**, with +the reasoning in [`api-wave.md`](api-wave.md). `mode` because a suspended target that keeps +observing is the same dry run with one field instead of two; `interval` because every input that +changes what a scan would conclude already arrives on a watch, and the only gap a timer closed is a +repository someone else edits, which the annotation covers on demand. PR 2 is one field and one +annotation, which is the smallest this part of the wave has been. + +## PR 3 — `status.placement`, and the post-scan pass + +The wave put status after the layout field. Inverting that is still the highest-value change in +this plan, and now it is also where the model's remaining machinery lives: `renderRoot` and the +resolution are facts about the folder that today's placement already computes and discards. + +```yaml +status: + observedGeneration: 4 + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization # SingleKustomization | Ambiguous | None + message: "render root '.' governs new files" + observedGeneration: 4 + placement: + renderRoot: . + serializeNamespace: Auto + byTypeEntries: 1 + observedRevision: 9f3c1ab + observedTime: "2026-07-30T09:14:22Z" + examples: [] # capped at three, illustrative +``` + +Three decisions are taken here rather than deferred, because each one is cheaper to take before the +field exists than after: + +- **The resolution reason is a condition reason, not a field.** `renderRootReason` would have been a + reason enum in a bespoke field, and shipping it here — one release before the model that defines + it — would have meant breaking a field sold as stable. +- **No accumulating counters.** `placedResources`, `overriddenTypes` and `refusedResources` are + metrics; `placements_total` carries them with better labels. A monotonic counter in status is a + status write per event, which re-creates the self-triggering reconcile edge the status work + already fixed once. +- **`conditions` and `observedGeneration` are in the stanza**, because every scenario README already + asserts `Ready=True`. + +**The post-scan validation pass ships here too**, because it is the same scan. Three rules whose +precondition is a property of the observed folder rather than of the spec, so no CEL rule can reach +them: + +| Rule | Precondition | +|---|---| +| `serializeNamespace: Never` requires a namespace supplier | a kustomization with `namespace:` governs the path | +| `kustomizeRoot: Require` needs a root | one governs the path | +| a declared single-root assertion | the folder has exactly one root | + +One pass, one condition shape, `Validated=False` naming the offending field and what the folder +actually contains. A corpus scenario per row, with an `expected-status.yaml` instead of a patch. + +The pass is written in PR 3 against the two rules that exist today; PR 4 adds the third row when it +adds the field. That ordering is deliberate: the machinery is proven before a new field depends on +it. + +## PR 4 — `serializeNamespace` and `kustomizeRoot` + +Additive, both defaulting to current behavior, so an object that says nothing behaves exactly as it +does today. + +- `serializeNamespace: Auto | Always | Never`. `Auto` is what `namespaceIsInheritedFromContext` + already does, so the work is the two explicit values plus the `Never` row of the post-scan pass. +- `kustomizeRoot: Adopt | Create | Require`. `Adopt` is the ancestor walk that shipped in + [#319](https://github.com/ConfigButler/gitops-reverser/pull/319). `Require` is a refusal on the + existing scan. **`Create` is the only genuinely new machinery in the whole placement story**: + writing a `kustomization.yaml` that does not exist, with `namespace:` set when the folder is + single-namespace, and registering into it in the same commit. + +Build `Create` last and on its own. It is the one value that writes a file nobody asked for by +name, it is what makes `Never` provable on an empty folder, and it is the only part of this PR that +cannot be verified by pointing the corpus at an existing fixture. + +`LocateNew` is not rewritten. The ladder stays; the two fields sit beside it. + +## PR 5 — `commitWindow` and `commit.message` move to `GitTarget` + +The last of the principle items, and the one that makes the object coherent: batching and +phrasing describe a folder, and `GitProvider` describes a connection. Mechanical, and +independent of everything above, which is why it can be written in parallel and merged into +the same release. + +## PR 6 — the riders + +`CommitRequest.spec.author` with its SAR guard, the CommitRequest lifecycle hole, F12's +reference types, the `TooManyStreams` cap, and the ClusterProvider default message. In the +release because they are breaking and the consumer should pay once. Trim from here first. + +**`TooManyStreams` is sequenced behind the `*` decision**, which is the one dependency this PR has +on work outside the plan. `sourceNamespace: "*"` is being redefined as a single cluster-wide list +and watch (design change 7 below), and that is where most of the fan-out the cap was written for +goes. Either land the `*` change first and size the cap against enumerated rules, or drop the cap +from this PR: it is the trim handle inside the trim handle, and sizing it against today's +per-namespace fan-out would bake in a number that is wrong the moment `*` lands. + +## Design changes this plan folds in + +Each came from reviewing the worked examples, except the last, which came from the source-scope +decision and lands on the same PRs. Three of the original six are gone, because the fields they +amended no longer exist. + +### 1. Dissolved: the namespace agreement rule + +This plan used to amend `model.md`'s requirement that `scope: SingleNamespace` demand an exact +one-name `allowedSourceNamespaces` list. Neither field exists now. A folder is single-namespace +because no `{namespace}` appears in its paths, and a document from a second namespace is refused at +the write boundary — which was always the thing actually enforcing singularity. + +### 2. The post-scan validation class, generalized + +Still the right shape, and now it carries three rules instead of one. Moved into PR 3, above, where +the scan that feeds it lives. + +The brownfield example is not evidence of the violation, and an earlier draft of this plan said it +was. It declares a namespace inheritance that is valid for its folder; the violation appears only if +the `kustomization.yaml` is later deleted. That is a better illustration anyway, because it is the +argument for observing per scan: the declaration never changed, only the folder did. + +### 3. Dissolved: `kind: Template` leaves the wave + +There are no kinds to leave. `placement.default` was never in danger, because it is not being +replaced — the fear was that a blanket default silently disables the render root, and #319 made a +templated file register with the root that governs it. + +### 4. Dissolved: `Auto` stays the CRD default + +There is no `kind` to default. `serializeNamespace: Auto` is a default in the ordinary sense, and it +names today's inference rather than standing in front of a structural rule. + +### 5. Dissolved: `interval` on two objects + +Reversed twice, and it ends smaller than either draft. This plan first argued that one name on two +objects is a smell; then that it is the Flux convention and each field's docs should say what it +drives. Both were arguing about the **name**, and the name was never the problem. + +`GitTarget` does not need the field. Flux polls because a Git remote cannot be watched, which is +what `GitProvider.spec.interval` is for and why it stays. A `GitTarget`'s inputs are API objects we +are already streaming, so its status is refreshed by the events that change it, plus the +reconcile-request annotation for the one case a watch cannot see: someone else editing the +repository. One `interval`, on the object that actually polls. + +### 6. Namespace-local `GitProvider` is a recorded gap, not a blocker + +The homelab examples put a `GitTarget` in `argocd`, `flux-system` and `homelab-config`, each +needing its own `GitProvider` — three copies of one credential in a single-owner cluster. +`ClusterProvider` already has the `allowedNamespaces` and fail-closed SAR machinery a +cluster-scoped Git provider would need. Out of scope for this order; file it, and say so in the +examples' prerequisites so a reader does not read the duplication as intended design. + +### 7. The source-scope simplification shares this release + +Not placement work, and folded in here because it lands on the same objects, in the same breaking +release, and moves a number this plan quotes. +[`source-scope-simplification.md`](../design/source-scope-simplification.md) deletes +`GitTarget.spec.allowedSourceNamespaces`, renames two `ClusterProvider` fields, and redefines +`sourceNamespace: "*"` as a single cluster-wide list and watch at `metav1.NamespaceAll`, rejected +while `allowAnySourceNamespace` is false. + +Three consequences for this plan: + +- **It rides PR 5's release, not PR 5.** It breaks `GitTarget`, as PR 5 does, so it must be in the + same bump; it is otherwise independent and reviews as its own change. Sequenced as step 8 in + [`api-wave.md`](api-wave.md). +- **PR 6's `TooManyStreams` cap shrinks**, as recorded above. It was queued for `*` fan-out; after + this it bounds explicit enumeration only, so plan it after the `*` change or drop it. +- **The corpus is unaffected, and that is worth checking rather than assuming.** Records carry the + object's own `metadata.namespace` rather than the cell's, so a document arriving from a + cluster-wide cell is placed exactly as it is today, and no example's expected patch moves. The + corpus should nonetheless grow one `sourceNamespace: "*"` scenario once the change lands, because + "the cell is cluster-wide but the path still carries `{namespace}`" is precisely the pair a + reader gets wrong. + +## Corpus gaps to fill in PR 1 + +- **A refusal scenario.** Every example is a happy path, and the post-scan pass is the part with + the least coverage. One folder per row of the table in PR 3: a `serializeNamespace: Never` whose + governing kustomization has no `namespace:`, a `kustomizeRoot: Require` with no root, and a + folder covering two roots. `expected-status.yaml` in place of a patch. These are the scenarios + that prove the guards, and they are the reason the guards are worth building. +- **The missing `ClusterProvider`.** `empty-repo-bootstrap` references + `clusterProviderRef: app-intent` and no such object appears anywhere. It is the intent + cluster, which is the whole direction-B thesis, so it deserves a concrete specimen beside the + cluster-tree one. +- **The prerequisites note** from design change 6. + +## What this plan does not decide + +- The helm standpoint, which is decided in the direction review and parked with entry criteria. +- Whether the `byType` map is ever shared across targets. The trigger is written down in the model: + that map is the only part with real reuse pressure. +- Whether `kustomizeRoot: CreatePerDirectory` is built. Deferred in the model with its trigger. +- Whether `serializeNamespace: Never` names its supplier. +- The `F9` enum question, which is measured before PR 4 is planned. diff --git a/docs/layout/model.md b/docs/layout/model.md new file mode 100644 index 00000000..ad904bb1 --- /dev/null +++ b/docs/layout/model.md @@ -0,0 +1,304 @@ +# The template was the right primitive. Two things were missing + +> **design**: a proposal, not a plan of record. Nothing here binds until scheduled. +> Index: [`../INDEX.md`](../INDEX.md) +> Date: 2026-08-28. Supersedes this document's own earlier thesis, which argued that a path +> template is the wrong primitive and should be replaced by a `spec.layout` discriminated union. +> That argument no longer holds; [why it stopped holding](#what-changed) is the first section, +> because a reversal is worth more than a quiet edit. +> +> Concrete repository folders and matching configurations live in +> [`examples/README.md`](examples/README.md). + +Placement today is a ladder of four rungs, three of which are path templates and one of which is not: + +```text +byType -> default -> the folder's one kustomize root -> canonical +``` + +The proposal here is to **keep that**, and add the two things a path genuinely cannot express: whether +`metadata.namespace` is written into the document, and what to do about the `kustomization.yaml` that +decides whether anyone ever renders it. + +```yaml +spec: + path: apps/demo + placement: # unchanged, as shipped + byType: + v1/secrets: "secrets/{name}{sensitiveSuffix}" + default: "{namespace}/{resource}/{name}.yaml" + serializeNamespace: Auto # Auto | Always | Never + kustomizeRoot: Adopt # Adopt | Create | Require +``` + +Two new fields. No discriminator, no `spec.layout`, no `kind`, no `scope`, no new CRD. + +## What changed + +The earlier thesis rested on five arguments. Three of them were retired by +[#319](https://github.com/ConfigButler/gitops-reverser/pull/319), which shipped the ancestor walk: +a new file is registered with the nearest kustomization that governs it, whatever chose its path. + +| The argument against templates | Still true? | +|---|---| +| `byType` into a subdirectory produces a file no kustomization lists | **No.** That was [#295](https://github.com/ConfigButler/gitops-reverser/issues/295), and it is fixed | +| `placement.default` does the same to every type at once | **No.** Same bug, same fix | +| A CRD default cannot be added: a non-empty template shadows the kustomize-root rung | **Weakened**, and [`placement-visibility-and-declared-defaults.md`](placement-visibility-and-declared-defaults.md) already concedes the fix turns this from a correctness wall into a legibility trade | +| "Where do my files go" needs a metric and a status field to be legible | **True** — but that is an argument for status, not against templates | +| Nothing can **create** structure, so an empty repository cannot be bootstrapped | **True**, and untouched by either design. Creating a `kustomization.yaml` is not a path question | + +The sentence the whole redesign rested on was *"a path template cannot express 'beside this folder's +one kustomization'"*. That is still true and no longer matters, because the template is no longer +asked to express it. Registration became an **invariant** rather than a rung — which was always the +best idea in the layout model, and it is the part that already shipped. + +What is left is one real gap (bootstrap), one status gap, and one field that was always missing. + +## What kustomize actually requires + +Four facts, measured against kustomize v5.8.1 rather than recalled, because three of them contradict +assumptions the earlier model was built on. + +1. **A root does not require a flat folder.** A `kustomization.yaml` listing `configmaps/cache.yaml` + and `apps/deployments/web.yaml` builds, and the root's `namespace:` transformer applies to both. + So "`Kustomize` means flat files beside one root" was our rule, never kustomize's. +2. **Nested roots work, one per subfolder.** A parent listing `media` and `monitoring`, each holding + its own `kustomization.yaml` with its own `namespace:`, renders each document into its own + namespace. A multi-namespace folder can therefore omit `metadata.namespace` safely — if something + owns those child roots. +3. **There is no ambient pickup.** `resources: [cms/*.yaml]` fails (`evalsymlink failure`), and a bare + directory fails unless it contains its own kustomization (`must resolve to a file: unable to find + one of 'kustomization.yaml'`). Every file is named explicitly or lives under a nested root. +4. **An unlisted file in a listed subdirectory renders nothing.** The #295 class, unchanged. + +Fact 3 is why registration must be an invariant: in a kustomize folder there is no other way for a +new file to be applied. Fact 1 is why the path may be anything the user wants. Together they say the +two questions are independent, which is exactly what a single `kind` discriminator could not express. + +## `serializeNamespace` + +The one thing a path cannot say. A path decides where the file sits; it cannot decide whether the +document inside carries its own `metadata.namespace`, and kustomize needs that from one of exactly +two places — the document, or a governing kustomization's `namespace:`. + +| Value | Meaning | +|---|---| +| `Auto` (default) | omit `metadata.namespace` when the governing kustomization already sets this resource's namespace. Today's inferred behavior, now named | +| `Always` | always write it. The only safe choice when nothing downstream supplies it | +| `Never` | never write it, and prove something else does | + +The name deliberately avoids `writeNamespace`. "Write" is the most loaded word in this API — +the write boundary, the write jail, `WriteBoundaryRefused` — so `writeNamespace: +Never` invites the reading *"never write to this namespace"*, a permission, which is precisely what +the neighbouring `sourceNamespace` fields are. `serializeNamespace` names the moment the decision is +made (when the document is produced) and cannot be read as policy. + +### The guard on `Never`, and why it is a post-scan check + +`Never` hands the object to whatever namespace the applier happens to be pointed at, which is a +different object with the same name. So it is valid only when something guarantees the namespace. +Today three of the six worked examples set it against a `kustomization.yaml` **the user owns**: they +delete one line from their own file and every subsequent document silently relocates. + +That precondition is a property of the observed folder, not of the spec, so no CEL rule can check it. +It joins the same post-scan validation class as the ambiguous-root case: one pass that sets +`Validated=False` naming the offending field and what the folder actually contains. + +| Rule | Precondition | Checkable at admission | +|---|---|---| +| `Never` requires a namespace supplier | a kustomization with `namespace:` governs the path | never | +| `kustomizeRoot: Require` needs a root | the folder has one | never | +| a declared single-root assertion | the folder has exactly one root | never | + +Three rules, one pass, one condition shape. Each gets a corpus scenario with an +`expected-status.yaml` instead of a patch. + +**`serializeNamespace` governs namespaced resources only.** A `ClusterRole` has no namespace, so the +field is ignored for cluster-scoped documents rather than being an error — worth stating in the field +documentation, because a tree folder is the type most likely to carry both. + +## `kustomizeRoot` + +The field that answers "do we want kustomize". It is one question with one axis, and the axis is +**what to do when no kustomization governs the path** — because when one does govern, all values +register, which is the invariant. + +| Value | A kustomization governs the path | No kustomization governs the path | +|---|---|---| +| `Adopt` (default) | register the file in its `resources:` | write the file, touch nothing | +| `Create` | register | create `kustomization.yaml` at `spec.path`, then register | +| `Require` | register | **refuse the write** | + +`Adopt` is today's behavior after #319, so a user who says nothing gets what they already have. + +`Create` is the empty-repository bootstrap — the last surviving argument from the earlier thesis, now +one value of one field rather than a reason to redesign the primitive. It closes its own loop with +`serializeNamespace`: a root the operator creates can carry `namespace:`, so `Never` becomes +provable rather than trusted. That is the difference between establishing a convention and guessing +one, and it is the thing inference structurally cannot do on an empty folder. + +`Require` is the safety value, and it is the one that earns its place from the `Never` guard above. +It says: this folder is a kustomize folder; if the root disappears, stop writing rather than commit +files nothing renders. Without it, the only responses to a deleted root are to carry on silently or +to hard-code a refusal nobody asked for. + +### Values considered and not taken + +**`Ignore` — never touch a kustomization, even when one governs the path.** Rejected. The scenario +for it is a tree with two consumers, where the root belongs to somebody else's build and our +documents feed a different one. But `spec.path` already expresses that: the ancestor walk is bounded +by the write jail, so a kustomization **above** `spec.path` is never touched, and the fix for "do not +edit that root" is to root the target at the folder you own. A value whose entire job is duplicated +by an existing field is a value that will be set by mistake more often than on purpose. It is also +the only candidate that does not fit the axis above — it changes what happens when a root **is** +present, which is the half that should be invariant. Revisit if someone produces a folder they must +target, containing a kustomization they must not edit, that cannot be split. + +**`CreatePerDirectory` — a nested root in every directory the template writes into, each wired into +its parent, each carrying its own `namespace:`.** Not rejected; deferred, and recorded here because +fact 2 above proves it works. It is what would make `serializeNamespace: Never` safe in a +multi-namespace tree, since the operator would own every root the omission depends on. It is also +materially more machinery than the other three values, and it should wait until someone wants a +multi-namespace folder without namespaces in its documents. The trigger is written down so this is a +decision rather than an omission. + +### The name + +`kustomization: Adopt` was the first spelling and it collides with a real Kubernetes kind — Flux's +`Kustomization` — in a field whose values look like a mode. That is the same complaint the maintainer +review makes about `layout.kind`, applied to this proposal, so it should not survive it. +`kustomizeRoot` uses the vocabulary this project already has (`renderRoot`, "the folder's one +kustomize root") and names a folder concept rather than an object. + +## Collisions are already decided + +The remaining question a template raises — what happens when two resources resolve to the same path — +is not open. It is specified and shipped in +[`new-file-placement-rules.md`](new-file-placement-rules.md): a unique path is a new file, and a +colliding path **appends** into a plaintext multi-document file. The hard cases come with it, and +they are the reason this is worth citing rather than reinventing: + +- a sensitive resource whose path already holds a document is **refused**, never appended; +- no append into an encrypted file, in either direction; +- no cold-bundle mixing when several new resources land on one path in a single flush; +- existing documents stay match-first, so an object already living inside a bundle is updated where + it is rather than moved out of it. + +So "one file per object" and "a bundle per type or per namespace" are both expressible today, by +writing a template that distinguishes identities or one that deliberately does not. + +## The four questions a user actually asks + +| Question | Answered by | +|---|---| +| Do my documents carry `metadata.namespace`? | `serializeNamespace` | +| Is this folder one namespace or many? | whether `{namespace}` appears in the template. A template without it is single-namespace by construction | +| Which folder do new files go in? | the directory part of the template. Fact 1: this is not constrained to flat | +| A root, with children? | `spec.path` is the root, the template's directories are the children, and the ancestor walk keeps every child reachable | + +No field in that table exists to answer a question a user did not ask, which is the test the earlier +`kind`/`scope` pair failed: `scope: SingleNamespace` restated in an enum what the template already +said, and then required an admission rule to keep the two in agreement. + +## What this deletes + +Against the earlier proposal in this document, and against the maintainer review's response to it: + +- **`spec.layout`** and its discriminator. With it go `kind`, `type`, `Auto`/`Kustomize`/`Tree`/ + `Flat`/`Template`, and the review's L3 and L4 naming findings — not renamed, gone. +- **`layout.scope`** and the admission rule keeping it in agreement with `allowedSourceNamespaces`, + and with them the review's L5. The template says it. +- **`kustomize.create`** and the review's L8, folded into `kustomizeRoot: Create`. +- **The `LayoutProfile` question.** It was already answered no; without a `layout` block the only + thing left to share is the `byType` map, which was the earlier document's own conclusion about + where reuse pressure actually lives. Whatever generates thirty GitTargets repeats two fields for + free. +- **The migration.** `spec.placement` keeps its meaning, so there is no loud rejection, no + `feat(api)!` on this axis, and no coordinated consumer bump for the layout work. The two new fields + are additive with defaults equal to today's behavior. + +That last one is the point. The layout model was the largest breaking change in the queue. On this +shape it is not a breaking change at all. + +## Status + +Three things the maintainer review asked for, taken: + +```yaml +status: + observedGeneration: 4 + conditions: + - type: LayoutResolved + status: "True" + reason: SingleKustomization # SingleKustomization | Ambiguous | None + message: "render root '.' governs new files" + observedGeneration: 4 + - type: Ready + status: "True" + reason: Succeeded + placement: + renderRoot: . + serializeNamespace: Auto # what it resolved to for this folder + byTypeEntries: 1 + observedRevision: 9f3c1ab + observedTime: "2026-07-30T09:14:22Z" + examples: # capped at three, illustrative, not a tally + - type: v1/secrets + path: apps/demo/secrets/db.sops.yaml + source: ByType +``` + +- **`renderRootReason` is a condition reason, not a field.** It was a reason enum wearing a field's + clothes, and every consumer in this ecosystem already reads reasons from `conditions`. Deciding it + now avoids shipping a status field one release before the model that defines it and then breaking + it. +- **No accumulating counters.** `placedResources`, `overriddenTypes` and `refusedResources` are + metrics; `placements_total` already carries them with better labels. A monotonic counter in status + means a status write per event, which is a hundred etcd writes a minute on a busy target for + something nobody polls at that resolution — and it re-creates the self-triggering reconcile edge + the status work already fixed once. `examples` stays, capped and fixed-size, because "show me where + a Secret would land" is not a metric. +- **`conditions` and `observedGeneration` are shown**, because every scenario README already asserts + `Ready=True` and the two documents should not disagree about what status looks like. + +The current half must never depend on a placement having happened: `renderRoot` is a fact about the +folder from the last scan, available before anything is ever written. + +## Metrics + +`placements_total` keeps `source` — `byType`, `default`, `kustomizeRoot`, `canonical` — which now +names the rung that answered rather than a resolved layout kind. No label break, because the ladder +survives. + +## Mutability + +`spec.placement` is mutable today and stays mutable. Existing files never move, so changing a +template affects only files written after the change, and a folder can hold documents placed under +two different templates. That is worth saying plainly rather than fixing: match-first identity means +those documents are still found and updated in place, and the alternative — an immutable field with a +CEL widening exception — was machinery invented to protect a discriminator that no longer exists. + +## What this changes about the work already queued + +- **The ancestor walk** shipped, and it is the invariant everything here rests on. +- **`{kindLower}` and the versionless identity fix** are template features and stay queued. +- **`status.placement`** replaces `status.layout`, in the shape above. Build it next; it is the + legibility gap, and it is the only surviving argument from the earlier thesis that has not been + answered by a field. +- **The post-scan validation pass** is now the home for three rules rather than one, and it is what + makes `serializeNamespace: Never` and `kustomizeRoot: Require` honest. +- **The breaking wave** loses its largest member. What remains breaking on `GitTarget` is unrelated + to placement, and is sequenced in [`api-wave.md`](api-wave.md). + +## Open questions + +- Does `serializeNamespace: Never` need to **name** its supplier (`KustomizeRoot`, + `FluxTargetNamespace`, `Asserted`) so the post-scan pass can check the guarantee rather than infer + which one was meant? +- Is `Require` the right default for a folder that already contains a kustomization when the target + is created? Adopting a kustomize folder and then silently continuing after its root is deleted is + the failure `Require` exists to prevent, and the folder itself is evidence of intent. +- `CreatePerDirectory`, above: what is the smallest scenario that actually needs it? +- Should `placement.default` gain a CRD default now that a defaulted template no longer produces + unrendered files? The remaining objection is legibility, not correctness, which is a materially + weaker case than the one that was refused. diff --git a/docs/spec/gittarget-new-file-placement-rules.md b/docs/layout/new-file-placement-rules.md similarity index 97% rename from docs/spec/gittarget-new-file-placement-rules.md rename to docs/layout/new-file-placement-rules.md index d7186793..7b15b2aa 100644 --- a/docs/spec/gittarget-new-file-placement-rules.md +++ b/docs/layout/new-file-placement-rules.md @@ -15,12 +15,12 @@ > Captured: 2026-06-05. Option C removed: 2026-07-29. > Related: > [open-asks-priority.md](../design/open-asks-priority.md) — **the argument for deleting Option C**, -> [contextual-namespace-and-kustomize-folder-editing.md](contextual-namespace-and-kustomize-folder-editing.md), -> [gittarget-repository-validity-and-placement.md](gittarget-new-file-placement-rules.md), -> [current-manifest-support-review.md](current-manifest-support-review.md), -> [manifestedit-new-file-placement-spike.md](gittarget-new-file-placement-rules.md), -> [reconcile-via-watchlist-mark-and-sweep.md](reconcile-via-watchlist-mark-and-sweep.md), -> [gitpath-foreign-content-stringency.md](gitpath-foreign-content-stringency.md) +> [contextual-namespace.md](contextual-namespace.md), +> [gittarget-repository-validity-and-placement.md](new-file-placement-rules.md), +> [current-manifest-support-review.md](../spec/current-manifest-support-review.md), +> [manifestedit-new-file-placement-spike.md](new-file-placement-rules.md), +> [reconcile-via-watchlist-mark-and-sweep.md](../spec/reconcile-via-watchlist-mark-and-sweep.md), +> [gitpath-foreign-content-stringency.md](../spec/gitpath-foreign-content-stringency.md) ## Summary @@ -123,7 +123,7 @@ The current `GitTargetSpec` has `providerRef`, `branch`, `path`, and optional ([api/v1alpha3/gittarget_types.go](../../api/v1alpha3/gittarget_types.go)). The writer already uses the materialized-model direction described in -[current-manifest-support-review.md](current-manifest-support-review.md): +[current-manifest-support-review.md](../spec/current-manifest-support-review.md): - steady-state writes scan the GitTarget subtree into a content-derived store, then apply a commit-scoped plan @@ -649,7 +649,7 @@ in force, which is not something the user can see. A document in a directory whose kustomization sets a `namespace:` transformer does not carry `metadata.namespace` — the build context supplies it (see -[contextual-namespace-and-kustomize-folder-editing.md](contextual-namespace-and-kustomize-folder-editing.md)). +[contextual-namespace.md](contextual-namespace.md)). A new document placed there must follow that convention, or it breaks the folder's own style, and this applies to **every** resolved path: a declared template pointing into a governed directory is under the same obligation as the kustomize-root fallback. @@ -811,7 +811,7 @@ because the answer does not depend on the folder's history.* **P7 — A resolved path is still subject to the write-time ignore invariant.** Any resolved path — declared, kustomize-root, or canonical — can collide with a `.gittargetignore` pattern and trip the §4.3 `IgnoreShadowsManagedPath` precondition -([gitpath-foreign-content-stringency.md](gitpath-foreign-content-stringency.md)), aborting +([gitpath-foreign-content-stringency.md](../spec/gitpath-foreign-content-stringency.md)), aborting the flush. *Still live, and unrelated to inference: placement inherits this failure mode rather than creating it.* @@ -925,6 +925,18 @@ Recommended variables: | `{name}` | metadata name | | `{sensitiveSuffix}` | Optional convention helper: `.sops.yaml` for sensitive writes, `.yaml` otherwise | +**A template carries its own extension.** Nothing is appended: `"{namespace}/{name}"` is rejected by +[path validation](#path-validation), which requires a recognized YAML suffix, and +`"{namespace}/{name}.yaml"` is the correct spelling. `{sensitiveSuffix}` is the one variable that +supplies the extension itself, because it has to choose between `.yaml` and `.sops.yaml`. Any +example that shows a template without a visible suffix is wrong. + +`{kindLower}` — the lower-cased kind, so `ConfigMap` becomes `configmap` — is **decided and not yet +built**. It is a variable rather than a `toLower` function because a function invites an expression +language into a field that is deliberately not one. It is what a user writes to ask for the +`configmap-cache.yaml` convention that the built-in rungs do not produce: the kustomize root names a +new sibling `{name}.yaml` and never infers a naming convention from the folder's existing files. + With those variables, the built-in canonical layout is **namespace-first, no version segment** (as implemented in `ResourceIdentifier.ToGitPath`): diff --git a/docs/design/placement-visibility-and-declared-defaults.md b/docs/layout/placement-visibility-and-declared-defaults.md similarity index 94% rename from docs/design/placement-visibility-and-declared-defaults.md rename to docs/layout/placement-visibility-and-declared-defaults.md index 7ef490e3..521b393a 100644 --- a/docs/design/placement-visibility-and-declared-defaults.md +++ b/docs/layout/placement-visibility-and-declared-defaults.md @@ -13,7 +13,7 @@ > Reading it in order matters, because the two halves have different standing. The **findings** are > checkable facts about the tree and they are why the calls are what they are. The **calls** are still > the calls. Question 2 (a CRD default for `placement.default`) has since been superseded outright by -> [`gittarget-layout-model.md`](gittarget-layout-model.md), which argues that a path template is the +> [`model.md`](model.md), which argues that a path template is the > wrong primitive to be defaulting at all. Three questions came out of reviewing #291, and one of them (a CRD default for @@ -103,7 +103,7 @@ fixing F4. `MarkTargetRetention` records a fact from the write path into an epoch-scoped per-target roll-up and calls `enqueueGitPathChange` **on a change only**, and the controller projects it in `gitTargetRetentionStatus`. This retires the objection recorded in -[`open-asks-priority.md`](open-asks-priority.md) that placement facts cannot reach the GitTarget +[`open-asks-priority.md`](../design/open-asks-priority.md) that placement facts cannot reach the GitTarget promptly because "a refusal recorded on the data plane does not enqueue the GitTarget". One does already. `status.retention` is proof. @@ -251,7 +251,12 @@ against `ToGitPath()` byte-for-byte, and solve F9 first, because nothing else on until a defaulted value can coexist with the render-root step. Status is worth having either way, so nothing built now is wasted. -## `status.layout` +## `status.layout`, renamed `status.placement` + +> **The field is `status.placement` now.** [`model.md`](model.md) renamed it when `spec.layout` +> stopped existing, and [`implementation-plan.md`](implementation-plan.md) builds it under that +> name in PR 3. The shape below is unchanged and this page is still where it is argued; read every +> `status.layout` here as `status.placement`. An **observation, not a condition**, in the sense [`GitTargetStatus.Retention`](../../api/v1alpha3/gittarget_types.go) already establishes: nothing @@ -496,14 +501,14 @@ because they are worth different urgency: | Item | State | Why it is where it is | |---|---|---| -| F10: register a declared path with the kustomization that governs it | **filed, correctness** | One `byType` line into a subdirectory silently produces a file nothing renders. Reachable today | -| Drop the `{version}` requirement from `IdentityCompletePlacementTemplate` | **filed, correctness** | It contradicts the versionless-path decision, and it is what makes any future spec default fail our own gate | +| F10: register a declared path with the kustomization that governs it | **SHIPPED** in 0.42.1 (#319) | One `byType` line into a subdirectory silently produced a file nothing renders. The fix made registration an invariant, which is what reversed [`model.md`](model.md) | +| Drop the `{version}` requirement from `IdentityCompletePlacementTemplate` | **SHIPPED** in 0.42.1 (#319) | It contradicted the versionless-path decision, and it was what made any future spec default fail our own gate | | `status.layout` | **filed** | The durable half of "what did the operator understand about this folder". Wants the layout model's vocabulary, so it follows it | | Split `declared` into `byType` and `default`; unify the prose on "canonical" | **filed, legibility** | A catch-all quietly swallowing a type you meant to name looks identical to a rule working | | `{kindLower}` | **filed, legibility** | Small, self-contained | | Canonical path as a template constant | **filed, cleanup** | Removes the hand-written duplication; what a future default would reuse | | `renderRootReason: Ambiguous` | **filed** | Belongs with `status.layout`, and the layout model decides the refuse-or-write policy | -| A CRD default for `placement.default` | **superseded** | [`gittarget-layout-model.md`](gittarget-layout-model.md): the primitive is wrong. `layout.kind` is the defaultable thing, because it names the structural rule instead of standing in front of it | +| A CRD default for `placement.default` | **re-opened** | The reversal took away the answer this row used to give. There is no `layout.kind`; the template stays, and now that the ancestor walk registers a defaulted path, the objection is legibility rather than correctness. It is an open question in [`model.md`](model.md) | The original build list follows, because each entry says *how* to build the thing and that is the part an issue should not have to restate. @@ -537,12 +542,15 @@ Ordered by risk, smallest first. 8. **Docs**: the spec's resolution-ladder section, `configuration.md`, `interpreting-metrics.md` for the new label values, and the `UPGRADING.md` entry extended with the metric-value split. -**Not now, with the trigger written down:** a CRD default for `placement.default`. This was already -"not now" when the page was written, and it has since become "not this shape at all", because -[`gittarget-layout-model.md`](gittarget-layout-model.md) replaces the template with a declared -`layout.kind` and `Auto` is a safe default where a path never was. The freezing question (F5) is a -trade we could take; the shadowing question is not, and the layout model dissolves it rather than -answering it. `spec.expect.layout` stays out too, on the config-surface doc's own rule: publish the +**Not now, with the trigger written down:** a CRD default for `placement.default`. This was "not +now" when the page was written and it stays "not now", but for a weaker reason than the page +originally gave. An earlier revision said the layout model dissolved the question by replacing the +template with a defaultable `layout.kind`; the model has since reversed and the template stays, so +the question is live again. What changed in its favour is the shipped ancestor walk: a defaulted +path into a subdirectory is now registered with the kustomization that governs it, so F9's +shadowing objection is no longer a correctness wall. The freezing question (F5) is a trade we could +take. What is left is legibility, and that is materially weaker than the case this page refused. +`spec.expect.layout` stays out too, on the config-surface doc's own rule: publish the observation before inventing the assertion. ## Open questions diff --git a/docs/spec/README.md b/docs/spec/README.md index 4933e1f9..5132eb9c 100644 --- a/docs/spec/README.md +++ b/docs/spec/README.md @@ -16,6 +16,12 @@ here because they are the only written record of a rule the code still obeys. If you change one of these behaviours, change the document in the same commit. +Two `spec/`-class contracts live in [`../layout/`](../layout/README.md) instead: +[`new-file-placement-rules.md`](../layout/new-file-placement-rules.md) and +[`contextual-namespace.md`](../layout/contextual-namespace.md). They bind exactly as the pages here +do, and Go source cites them by path; they sit in a topic folder because the layout question spans +three lifecycle classes and was unreadable spread across them. The rows below still list them. + ## Start here - [`manifest-system.md`](manifest-system.md) — **how the whole live → Git pipeline @@ -27,9 +33,9 @@ If you change one of these behaviours, change the document in the same commit. |---|---| | [`attribution.md`](attribution.md) | **how a commit gets its author**: deletion-at-intent, the publish and join halves, the tier ladder, the audit-route partition, and what the metrics can and cannot answer | | [`current-manifest-support-review.md`](current-manifest-support-review.md) | the manifest store, plan/apply/flush, and the all-or-nothing folder claim | -| [`contextual-namespace-and-kustomize-folder-editing.md`](contextual-namespace-and-kustomize-folder-editing.md) | kustomize graph-aware namespace inference; the supported subset | +| [`../layout/contextual-namespace.md`](../layout/contextual-namespace.md) | kustomize graph-aware namespace inference; the supported subset | | [`reconcile-via-watchlist-mark-and-sweep.md`](reconcile-via-watchlist-mark-and-sweep.md) | initial reconcile; **no bookmark, no sweep** | -| [`gittarget-new-file-placement-rules.md`](gittarget-new-file-placement-rules.md) | where a brand-new resource's file goes | +| [`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md) | where a brand-new resource's file goes | | [`manifestedit-field-ownership-spike.md`](manifestedit-field-ownership-spike.md) | "the API wins" — full-object ownership, and the do-not-build list | | [`type-followability.md`](type-followability.md) | is a type followable, and if not, the single reason | | [`type-lifecycle-events-and-wobble-settling.md`](type-lifecycle-events-and-wobble-settling.md) | removal grace and flap coalescing | diff --git a/docs/spec/manifest-system.md b/docs/spec/manifest-system.md index 227c6a93..f5ee2318 100644 --- a/docs/spec/manifest-system.md +++ b/docs/spec/manifest-system.md @@ -47,7 +47,7 @@ identity (group, kind, namespace, name). The store indexes by that identity and writes an edit back to wherever the document already lives — *match-first placement*. Being strict about which file a resource "should" live in was explicitly rejected. Only a genuinely new resource needs a placement decision, and -that is [`gittarget-new-file-placement-rules.md`](gittarget-new-file-placement-rules.md). +that is [`../layout/new-file-placement-rules.md`](../layout/new-file-placement-rules.md). **A GitTarget makes an all-or-nothing claim on its folder.** It either manages everything in the subtree or it refuses the folder. There is no partial ownership, @@ -111,7 +111,7 @@ beside it. Raw identity (what the bytes say) and effective identity (what kustom would produce) are tracked separately, and an inherited namespace is kept *out* of the file bytes on write. The supported subset, and why everything outside it is refused rather than unimplemented, is -[`contextual-namespace-and-kustomize-folder-editing.md`](contextual-namespace-and-kustomize-folder-editing.md). +[`../layout/contextual-namespace.md`](../layout/contextual-namespace.md). The governing constraint is invertibility: an edit must round-trip in both directions. Generators, `patches*`, `namePrefix`/`nameSuffix`, `components`, remote diff --git a/docs/spec/sops-single-file-no-multidoc.md b/docs/spec/sops-single-file-no-multidoc.md index bd6de697..ab9299bc 100644 --- a/docs/spec/sops-single-file-no-multidoc.md +++ b/docs/spec/sops-single-file-no-multidoc.md @@ -5,8 +5,8 @@ > Status: decided > Captured: 2026-06-08 > Related: -> [file-agnostic-placement.md](gittarget-new-file-placement-rules.md), -> [contextual-namespace-and-kustomize-folder-editing.md](contextual-namespace-and-kustomize-folder-editing.md), +> [file-agnostic-placement.md](../layout/new-file-placement-rules.md), +> [../layout/contextual-namespace.md](../layout/contextual-namespace.md), > [../sops-repo-bootstrap-and-key-management-architecture.md](../finished/sops-repo-bootstrap-and-key-management-architecture.md), > [../sops-repo-bootstrap-out-of-scope.md](../finished/sops-repo-bootstrap-out-of-scope.md) @@ -17,7 +17,7 @@ SOPS file**. We do **not** write SOPS-encrypted multi-document YAML (no `\n---\n`-separated documents inside an encrypted file). Plaintext manifests may still be multi-document where that is convenient (see -[file-agnostic-placement.md](gittarget-new-file-placement-rules.md)); this decision is +[file-agnostic-placement.md](../layout/new-file-placement-rules.md)); this decision is scoped to files we encrypt with SOPS. ## Why this came up diff --git a/internal/controller/gittarget_placement_validation.go b/internal/controller/gittarget_placement_validation.go index dd73f15b..839a79ff 100644 --- a/internal/controller/gittarget_placement_validation.go +++ b/internal/controller/gittarget_placement_validation.go @@ -19,7 +19,7 @@ const coreSecretsTypeKey = "v1/secrets" // validatePlacementPolicy statically validates a GitTarget's declared placement // policy (Option B2: -// docs/spec/gittarget-new-file-placement-rules.md) against the +// docs/layout/new-file-placement-rules.md) against the // spec alone — no repository scan is needed, so this runs as part of the Validated // gate, the same spec-well-formedness check that already covers provider/branch // resolution and path-overlap conflicts. A nil spec (no declared policy) is always diff --git a/internal/git/manifestedit/kustomization.go b/internal/git/manifestedit/kustomization.go index c60826d3..86877ed1 100644 --- a/internal/git/manifestedit/kustomization.go +++ b/internal/git/manifestedit/kustomization.go @@ -128,7 +128,7 @@ func applyKustomizationEdit(root *yaml.Node, e KustomizationEdit) error { // AppendKustomizationResource adds one entry to an existing kustomization.yaml's // resources: sequence — the mechanism half of the "add to the right kustomize -// file" (docs/spec/gittarget-new-file-placement-rules.md): a +// file" (docs/layout/new-file-placement-rules.md): a // new sibling file placed inside a kustomize-governed directory must also be named // in that directory's resources: list, or kustomize never renders it. // diff --git a/internal/git/plan_flush.go b/internal/git/plan_flush.go index bb68231f..eb9e80f8 100644 --- a/internal/git/plan_flush.go +++ b/internal/git/plan_flush.go @@ -154,7 +154,7 @@ type writeBatch struct { // and therefore cannot see coming — form one deterministic, resource-identity- // sorted multi-document file instead of each writeWholeFile call silently // discarding the one before it. See - // docs/spec/gittarget-new-file-placement-rules.md, + // docs/layout/new-file-placement-rules.md, // "Collision and append behavior": "if several new plaintext resources in one // plan render to the same path, write a multi-document file in deterministic // resource-identity order." @@ -339,7 +339,7 @@ func wroteBytes(o upsertOutcome) bool { // createNew resolves the placement of a resource with no existing document — // declared policy (Option B), the folder's one kustomize root, or the canonical -// fallback — per docs/spec/gittarget-new-file-placement-rules.md, +// fallback — per docs/layout/new-file-placement-rules.md, // adds the kustomize resources: entry the placement may require, and writes the new // document: a brand-new file, or an additional document appended to an existing // accepted plaintext bundle. A placement LocateNew cannot honour safely (today, only diff --git a/internal/manifestanalyzer/contextual_namespace_corpus_test.go b/internal/manifestanalyzer/contextual_namespace_corpus_test.go index 9f761735..8f658498 100644 --- a/internal/manifestanalyzer/contextual_namespace_corpus_test.go +++ b/internal/manifestanalyzer/contextual_namespace_corpus_test.go @@ -25,7 +25,7 @@ type wantDoc struct { // testdata/contextual-namespace. Each folder is built as a GitTarget subtree and the // per-document namespace provenance is asserted, so the supported boundary is pinned by // real layouts rather than prose. See -// docs/spec/contextual-namespace-and-kustomize-folder-editing.md. +// docs/layout/contextual-namespace.md. func TestContextualNamespaceCorpus(t *testing.T) { cases := []struct { dir string diff --git a/internal/manifestanalyzer/placement.go b/internal/manifestanalyzer/placement.go index 8919f6b6..868d0119 100644 --- a/internal/manifestanalyzer/placement.go +++ b/internal/manifestanalyzer/placement.go @@ -13,7 +13,7 @@ import ( ) // PlacementPolicy is a resolved GitTarget placement declaration (Option B2 of -// docs/spec/gittarget-new-file-placement-rules.md): a single +// docs/layout/new-file-placement-rules.md): a single // exact-type map plus a fallback default template, consulted for every resource // regardless of sensitivity. It mirrors api/v1alpha3.GitTargetPlacementSpec // field-for-field but is defined locally so this analyzer package stays free of any @@ -35,8 +35,8 @@ type PlacementPolicy struct { // PlacementRequest describes a resource with no existing document in Git — the // only case placement runs for (an existing document is always updated in place at -// its current location; see docs/design/manifest/version2/ -// gittarget-new-file-placement-rules.md, "Existing manifests are still match-first"). +// its current location; see docs/layout/new-file-placement-rules.md, +// "Existing manifests are still match-first"). type PlacementRequest struct { Identifier types.ResourceIdentifier Kind string @@ -161,7 +161,7 @@ func (e *PlacementRefusedError) Error() string { return e.detail } func (e *PlacementRefusedError) Unwrap() error { return e.cause } // LocateNew resolves the placement of a resource with no existing document, per -// docs/spec/gittarget-new-file-placement-rules.md: a declared template (Option B) +// docs/layout/new-file-placement-rules.md: a declared template (Option B) // wins when present; otherwise the folder's one supported kustomize root, if it has // exactly one; otherwise the canonical path. // diff --git a/internal/manifestanalyzer/store.go b/internal/manifestanalyzer/store.go index 55bfaf7c..43189fb8 100644 --- a/internal/manifestanalyzer/store.go +++ b/internal/manifestanalyzer/store.go @@ -263,7 +263,7 @@ type DocumentModel struct { // It replaces an earlier "namespace came from kustomize" boolean so the store can also // explain the no-context and ambiguous cases to status, duplicate diagnostics, and // future placement — see -// docs/spec/contextual-namespace-and-kustomize-folder-editing.md. +// docs/layout/contextual-namespace.md. type NamespaceSourceKind string const ( @@ -796,7 +796,7 @@ func kustomizationInfos(kusts map[string]*kustomizationDoc) map[string]*Kustomiz // namespace transformer, its resources/bases graph entries, and whether it uses any // feature outside the supported contextual-namespace subset (which disqualifies it as // a namespace source). See the "Kustomize subset proposal" in -// docs/spec/contextual-namespace-and-kustomize-folder-editing.md. +// docs/layout/contextual-namespace.md. type kustomizationDoc struct { path string // kustomization file path (slash) namespace string // the namespace: transformer value diff --git a/internal/manifestanalyzer/testdata/contextual-namespace/README.md b/internal/manifestanalyzer/testdata/contextual-namespace/README.md index 97551e8e..65683346 100644 --- a/internal/manifestanalyzer/testdata/contextual-namespace/README.md +++ b/internal/manifestanalyzer/testdata/contextual-namespace/README.md @@ -16,6 +16,6 @@ The `images-overlay`, `replicas-overlay`, and `ambiguous-images` folders pin the override-chain attribution the same way (`overrides_test.go`); see `docs/design/support-boundary/finished/images-and-replicas-edit-through.md`. -See `docs/design/manifest/contextual-namespace-and-kustomize-folder-editing.md` +See `docs/layout/contextual-namespace.md` (the "Supported and unsupported example folders" matrix). Add a new folder here whenever a new "can we support X?" question comes up. diff --git a/internal/types/identifier.go b/internal/types/identifier.go index 03293c5e..90712392 100644 --- a/internal/types/identifier.go +++ b/internal/types/identifier.go @@ -87,7 +87,7 @@ func (r ResourceIdentifier) Key() string { // once any layout exists in the target, sibling inference follows it, and an // existing document is always edited in place at its current location (match-first), // so changing this shape never moves a file that is already in Git. See -// docs/spec/gittarget-new-file-placement-rules.md. +// docs/layout/new-file-placement-rules.md. // // That omitted version is the other half of the decision recorded at // [ResourceIdentifier.Key]: this versionless identity is the durable one — the object stays diff --git a/test/e2e/new_file_placement_e2e_test.go b/test/e2e/new_file_placement_e2e_test.go index a2683fa1..5d2f2e70 100644 --- a/test/e2e/new_file_placement_e2e_test.go +++ b/test/e2e/new_file_placement_e2e_test.go @@ -14,7 +14,7 @@ import ( // Validates new-file placement end-to-end: a brand-new resource with no // existing document in Git — the "install something extra in test" launch use -// case (docs/spec/gittarget-new-file-placement-rules.md, +// case (docs/layout/new-file-placement-rules.md, // docs/design/support-boundary/README.md) — lands inside the kustomize-managed overlay // directory it belongs to, not the canonical GVR-tree path, and the overlay's // kustomization.yaml gains the resources: entry so kustomize actually renders it.