diff --git a/README.md b/README.md index 29c6156..8b3d98d 100644 --- a/README.md +++ b/README.md @@ -81,21 +81,71 @@ hops validate --help hops xr --help ``` -## Local workbench (happy path) +## Local workbench definition -Multi-workspace local GitOps on the laptop control plane: +Keep the Kubernetes-shaped local workbench definitions together under +`.gitops/local/`. The Cluster owns the local control plane and shared +`.gitops/local/cluster/` manifests. The reusable Environment names the deploys +that make up the current checkout's environment. + +```yaml +apiVersion: hops.local/v1alpha1 +kind: Cluster +metadata: + name: project-dev +spec: + clusterProvider: kind + dockerProvider: dory + mountRoot: ../.. + manifests: + path: .gitops/local/cluster +``` + +```yaml +apiVersion: hops.local/v1alpha1 +kind: Environment +metadata: + name: local +spec: + clusterRef: + name: project-dev + root: . + values: + local: true + preview: false + deploys: + - path: apps/gateway +``` + +From that project root: ```bash -# shared control-plane tree (terminal 1) -hops local gitops cluster ./gitops/cluster \ - --cluster-provider kind --docker-provider dory --cluster-name hops +hops local up +hops local gitops cluster ./.gitops/local/cluster +hops local gitops environment ./.gitops/local/environment.yaml --name main +``` + +From another checkout of the same project: -# per-workspace tree (terminal 2) -hops local gitops worktree ./gitops/envs/local --name alice \ - --cluster-provider kind --docker-provider dory --cluster-name hops +```bash +hops local gitops environment ./.gitops/local/environment.yaml --name feature-auth ``` -Use `--name` for concurrent worktrees (`` namespaces). Full guide: [skills/claude/references/local-workbench.md](skills/claude/references/local-workbench.md). +`up` validates the Cluster before starting or reusing it. `gitops cluster` +watches shared `.gitops/local/cluster` manifests. `environment` validates the +Environment against that Cluster, renders each deploy's `.gitops/promote` +chart, applies the resulting local Applications to the runtime namespace, and +watches `.gitops/local/environment.yaml` plus the referenced +`.gitops/promote` and `.gitops/local` charts. Each application's +`.gitops/local` chart owns its editable local workload; `.gitops/deploy` is a +separate cloud workload chart selected by promotion outside local mode. +The runtime name, namespace, checkout path, and Cluster binding are local state; +they are not committed to the Cluster definition. + +An existing kind Cluster with a different exact `mountRoot` fails with an +explicit reset/recreate instruction and is never silently deleted. A legacy +directory of pre-rendered Application YAMLs is still accepted by `environment` +during migration. ## Command Areas @@ -291,7 +341,7 @@ export DOCKER_HOST=unix://$HOME/.dory/dory.sock ```bash hops local start --cluster-provider dory --docker-provider dory hops local start --cluster-provider dory --docker-provider dory --dory-name mine -hops local gitops worktree ./gitops/envs/local --name alice +hops local gitops environment ./.gitops/local/environment.yaml --name alice kubectl get nodes # context hops-dory docker info # context hops-dory diff --git a/skills/claude/SKILL.md b/skills/claude/SKILL.md index 2ea7cfc..4e0f2f7 100644 --- a/skills/claude/SKILL.md +++ b/skills/claude/SKILL.md @@ -17,7 +17,7 @@ control planes, configuration packages, providers, secrets, and live infrastruct | Command area | Purpose | |-------------|---------| -| `hops local` | Local CP (dory/colima/kind), gitops cluster/worktree, workbench | +| `hops local` | Local CP (dory/colima/kind), gitops Cluster/Environment, workbench | | `hops config` | Install Configuration packages (published **or** source + gitops) | | `hops provider` | Install/patch Providers (published **or** source + SemVer-safe tags + gitops) | | `hops secrets` | SOPS encrypt/decrypt, sync to AWS Secrets Manager or GitHub | @@ -31,7 +31,7 @@ For detailed reference on each area, see the bundled references: - **[Local source packages & providers](references/local-source-packages.md)** — **read this** when developing configs/providers on a laptop CP - [Config install modes and gitops](references/config-install.md) -- [Local workbench (gitops cluster / worktree)](references/local-workbench.md) +- [Local workbench (gitops Cluster / Environment)](references/local-workbench.md) - [Local control plane setup](references/local-setup.md) - [XR observe → adopt → manage workflow](references/xr-workflow.md) - [Secrets management](references/secrets.md) @@ -55,7 +55,7 @@ hops config install --repo hops-ops/auth-stack --version v1.6.0 \ --gitops ./gitops/cluster --local hops local gitops cluster ./gitops/cluster # watches by default; --once for CI -hops local gitops worktree ./gitops/envs/local --name dogfood +hops local gitops environment ./.gitops/local/environment.yaml --name dogfood ``` - **`--gitops`** materializes pins under the cluster tree (not one-shot-only kubectl) diff --git a/skills/claude/references/local-source-packages.md b/skills/claude/references/local-source-packages.md index bbb3282..39e1f34 100644 --- a/skills/claude/references/local-source-packages.md +++ b/skills/claude/references/local-source-packages.md @@ -140,7 +140,7 @@ Day-to-day: ```bash hops local gitops cluster ./gitops/cluster # watches by default -hops local gitops worktree ./gitops/envs/local --name dogfood +hops local gitops environment ./.gitops/local/environment.yaml --name dogfood ``` ## Do / don’t @@ -161,5 +161,5 @@ hops local gitops worktree ./gitops/envs/local --name dogfood ## See also - [config-install.md](./config-install.md) — flags and mode details -- [local-workbench.md](./local-workbench.md) — cluster vs worktree gitops +- [local-workbench.md](./local-workbench.md) — Cluster vs Environment gitops - [local-setup.md](./local-setup.md) — `hops local start` bootstrap diff --git a/skills/claude/references/local-workbench.md b/skills/claude/references/local-workbench.md index 05312a3..8992d09 100644 --- a/skills/claude/references/local-workbench.md +++ b/skills/claude/references/local-workbench.md @@ -15,8 +15,8 @@ tree copy). You do not need to learn volume types. ```bash # Dory app running (engine healthy). Product Dory Kubernetes is optional. -hops local start --cluster-provider kind --docker-provider dory \ - --cluster-name hops --gitops ./gitops/cluster +hops local up +hops local gitops cluster ./.gitops/local/cluster ``` Context is typically `kind-hops`. Confirm mounts: @@ -36,18 +36,17 @@ Stock Dory k8s (`--cluster-provider dory --docker-provider dory`) is fine for pl **cannot** hostPath-mount Mac paths into the node; delivery falls back to sync. ```bash -hops local start --cluster-provider dory --docker-provider dory \ - --gitops ./gitops/cluster +hops local up --cluster-provider dory --docker-provider dory ``` ## Daily loop ```bash # Shared CP watch (if start did not use --gitops, or after Ctrl+C) -hops local gitops cluster ./gitops/cluster +hops local gitops cluster ./.gitops/local/cluster -# Per-worktree apps (Application YAMLs → namespace = --name) — watches by default -hops local gitops worktree ./gitops/envs/local --name dogfood +# One Environment per checkout (namespace = --name) — watches by default +hops local gitops environment ./.gitops/local/environment.yaml --name dogfood # Stop either watcher with Ctrl+C. ``` @@ -60,10 +59,10 @@ Use a distinct name per worktree so namespaces and URLs stay isolated: ```bash # Terminal A -hops local gitops worktree ./gitops/envs/local --name alice +hops local gitops environment ./.gitops/local/environment.yaml --name alice # Terminal B -hops local gitops worktree ./gitops/envs/local --name bob +hops local gitops environment ./.gitops/local/environment.yaml --name bob ``` @@ -74,24 +73,25 @@ Each name maps to namespace ``. ```bash cd distributed/tests/e2e-ui # Prefer kind-on-Dory for hostPath HMR (see One-time prerequisite) -hops local start --cluster-provider kind --docker-provider dory \ - --cluster-name hops --gitops ./gitops/cluster -hops local gitops cluster ./gitops/cluster -hops local gitops worktree ./gitops/envs/local --name dogfood +hops local up +hops local gitops cluster ./.gitops/local/cluster +hops local gitops environment ./.gitops/local/environment.yaml --name dogfood ``` -Charts live under `api/.gitops/deploy` and `ui/.gitops/deploy`. You can also render them without hops: +Editable charts live under `api/.gitops/local` and `ui/.gitops/local`; +`.gitops/deploy` is reserved for independent cloud charts. You can render the +local charts without Hops: ```bash -helm template api ./api/.gitops/deploy --set local=true --set appRuntime=cluster-dev -helm template ui ./ui/.gitops/deploy --set local=true --set appRuntime=cluster-dev +helm template api ./api/.gitops/local +helm template ui ./ui/.gitops/local ``` ### Agent rules (do not skip) -Dogfood apps run as **`appRuntime: cluster-dev`** in namespace `= --name` with -source delivery into the pods. The site you must fix is **that** stack — not a -host `make run` you invent. +Dogfood apps run from their **`.gitops/local` charts** in namespace `= --name` +with source delivery into the pods. The site you must fix is **that** stack — +not a host `make run` you invent. **When the dogfood site is broken:** @@ -134,11 +134,11 @@ host `make run` you invent. ```text gitops/ cluster/ # shared CP (one per machine) — hops local gitops cluster - envs/local/ # app Applications — hops local gitops worktree + .gitops/local/environment.yaml # reusable Environment definition ``` - **cluster** — not per-worktree; packages + platform XRs on the local CP -- **worktree** — env Application YAMLs into namespace `= --name` +- **environment** — promoted local applications into namespace `= --name` ## Developing configs & providers on this CP diff --git a/src/commands/local/aws.rs b/src/commands/local/aws.rs index 26778a0..5c39401 100644 --- a/src/commands/local/aws.rs +++ b/src/commands/local/aws.rs @@ -56,7 +56,7 @@ pub struct AwsArgs { pub refresh: bool, /// Write non-secret Provider / DeploymentRuntimeConfig / ProviderConfig YAML - /// under this directory (e.g. `./gitops/cluster`). Credential Secrets are + /// under this directory (e.g. `./.gitops/local/cluster`). Credential Secrets are /// **not** written — still applied live only. #[arg(long)] pub gitops: Option, diff --git a/src/commands/local/backend/kind.rs b/src/commands/local/backend/kind.rs index df2f891..dc516c8 100644 --- a/src/commands/local/backend/kind.rs +++ b/src/commands/local/backend/kind.rs @@ -21,8 +21,10 @@ use super::SizeArgs; use crate::commands::local::package_install::{REGISTRY_PULL, REGISTRY_PUSH}; use crate::commands::local::{command_exists, run_cmd, run_cmd_output}; +use std::collections::BTreeSet; use std::error::Error; use std::io::Write; +use std::net::{Ipv4Addr, TcpListener}; use std::path::{Path, PathBuf}; use std::process::{Command, Stdio}; use std::thread; @@ -31,6 +33,28 @@ use std::time::Duration; /// Default kind cluster name (and historical hard-coded value). pub const DEFAULT_CLUSTER_NAME: &str = "hops"; const KIND_CLUSTER_NAME_ENV: &str = "HOPS_KIND_CLUSTER_NAME"; +const KIND_REGISTRY_HOST_PORT_ENV: &str = "HOPS_KIND_REGISTRY_HOST_PORT"; +const REGISTRY_HOST_PORT_START: u16 = 30500; +const REGISTRY_HOST_PORT_END: u16 = 30599; +const INOTIFY_SYSCTL_PATH: &str = "/etc/sysctl.d/99-hops-local-inotify.conf"; +const INOTIFY_MAX_USER_INSTANCES: u32 = 8192; +const INOTIFY_MAX_USER_WATCHES: u32 = 1_048_576; +const INSTALL_INOTIFY_SYSCTL_SCRIPT: &str = r#"set -eu +target="$1" +expected_instances="$2" +expected_watches="$3" +tmp="${target}.tmp" +trap 'rm -f "${tmp}"' EXIT +cat > "${tmp}" +chmod 0644 "${tmp}" +mv "${tmp}" "${target}" +trap - EXIT +sysctl -p "${target}" >/dev/null +instances="$(sysctl -n fs.inotify.max_user_instances)" +watches="$(sysctl -n fs.inotify.max_user_watches)" +test "${instances}" -ge "${expected_instances}" +test "${watches}" -ge "${expected_watches}" +"#; /// Active hops kind cluster name (`kind create --name`). pub fn active_cluster_name() -> String { @@ -51,6 +75,13 @@ pub fn set_active_cluster_name(name: &str) { } } +/// Bind the configured Cluster.mountRoot into the kind node at the same +/// absolute path. The definition loader validates and canonicalizes the path +/// before calling this process-scoped adapter. +pub fn set_extra_mount_root(path: &Path) { + std::env::set_var("HOPS_KIND_EXTRA_MOUNT", path); +} + /// kubeconfig context kind creates for the active name (`kind-`). pub fn kube_context_name() -> String { format!("kind-{}", active_cluster_name()) @@ -71,25 +102,141 @@ const MIN_KIND_VERSION: (u32, u32) = (0, 27); /// Pure registry hostPort selection (testable without docker). /// -/// When kind shares a docker engine with product Dory k8s, host 30500 is often -/// already bound — use 30501 so create succeeds (LWB-REQ-254). -pub fn pick_registry_host_port(env_override: Option, dory_k8s_present: bool) -> u16 { - if let Some(p) = env_override { - return p; - } - if dory_k8s_present { - return 30501; - } - 30500 +/// An existing cluster's published port is authoritative. New clusters honor +/// an explicit override, then take the first port not reserved by another +/// container or host process. +pub fn pick_registry_host_port( + existing_port: Option, + env_override: Option, + unavailable: &BTreeSet, +) -> Option { + existing_port.or(env_override).or_else(|| { + (REGISTRY_HOST_PORT_START..=REGISTRY_HOST_PORT_END).find(|port| !unavailable.contains(port)) + }) } /// Host port published for the in-cluster registry NodePort (container 30500). +/// +/// Once a named cluster exists, read its Docker binding rather than deriving a +/// process-global answer. This keeps package pushes and doctor checks targeted +/// at the selected cluster even when several kind clusters coexist. pub fn registry_host_port() -> u16 { - let env_override = std::env::var("HOPS_KIND_REGISTRY_HOST_PORT") + resolve_registry_host_port().unwrap_or_else(|error| { + log::warn!( + "unable to resolve kind registry host port ({error}); falling back to {REGISTRY_HOST_PORT_START}" + ); + REGISTRY_HOST_PORT_START + }) +} + +fn resolve_registry_host_port() -> Result> { + if let Some(port) = published_host_port(&node_container_name(), "30500/tcp") { + return Ok(port); + } + + let env_override = match std::env::var(KIND_REGISTRY_HOST_PORT_ENV) { + Ok(raw) if raw.trim().is_empty() => None, + Ok(raw) => Some(raw.trim().parse::().map_err(|_| { + format!( + "{KIND_REGISTRY_HOST_PORT_ENV} must be a valid TCP port, got {:?}", + raw.trim() + ) + })?), + Err(std::env::VarError::NotPresent) => None, + Err(error) => return Err(error.into()), + }; + + let unavailable = unavailable_registry_host_ports()?; + if let Some(port) = env_override { + if unavailable.contains(&port) { + return Err(format!( + "{KIND_REGISTRY_HOST_PORT_ENV}={port} is already reserved; choose another port" + ) + .into()); + } + } + + pick_registry_host_port(None, env_override, &unavailable).ok_or_else(|| { + format!( + "no free kind registry host port in {REGISTRY_HOST_PORT_START}-{REGISTRY_HOST_PORT_END}; \ + free a port or set {KIND_REGISTRY_HOST_PORT_ENV}" + ) + .into() + }) +} + +/// Read a container's configured host binding. HostConfig works for both +/// running and stopped containers, so stopped named clusters still reserve +/// their ports and can be restarted later. +fn published_host_port(container: &str, container_port: &str) -> Option { + let template = format!( + "{{{{(index (index .HostConfig.PortBindings {:?}) 0).HostPort}}}}", + container_port + ); + docker_output(&["inspect", "-f", &template, container]) .ok() - .and_then(|raw| raw.trim().parse::().ok()); - let dory_k8s = docker_output(&["inspect", "-f", "{{.Id}}", "dory-k8s"]).is_ok(); - pick_registry_host_port(env_override, dory_k8s) + .and_then(|raw| raw.trim().parse::().ok()) +} + +fn unavailable_registry_host_ports() -> Result, Box> { + let mut unavailable = docker_reserved_host_ports()?; + + // Docker reservations do not include native host processes. Probe the + // bounded allocation range as well; the listener is dropped immediately. + for port in REGISTRY_HOST_PORT_START..=REGISTRY_HOST_PORT_END { + if TcpListener::bind((Ipv4Addr::LOCALHOST, port)).is_err() { + unavailable.insert(port); + } + } + + Ok(unavailable) +} + +fn docker_reserved_host_ports() -> Result, Box> { + let mut reserved = BTreeSet::new(); + let container_ids = docker_output(&["ps", "-aq"])?; + + for container_id in container_ids + .lines() + .map(str::trim) + .filter(|id| !id.is_empty()) + { + let Ok(raw) = docker_output(&[ + "inspect", + "-f", + "{{json .HostConfig.PortBindings}}", + container_id, + ]) else { + // Containers can disappear between `ps` and `inspect`. + continue; + }; + let Ok(bindings) = serde_json::from_str::(raw.trim()) else { + log::debug!("ignoring malformed Docker port bindings for {container_id}"); + continue; + }; + let Some(bindings) = bindings.as_object() else { + continue; + }; + + for host_binding in bindings + .values() + .filter_map(serde_json::Value::as_array) + .flatten() + { + let Some(port) = host_binding + .get("HostPort") + .and_then(serde_json::Value::as_str) + .and_then(|port| port.parse::().ok()) + else { + continue; + }; + if port != 0 { + reserved.insert(port); + } + } + } + + Ok(reserved) } /// Result of checking whether the kind node can see the projects-root mount. @@ -155,6 +302,43 @@ pub fn node_sees_path(path: &str) -> bool { docker_output(&["exec", &node, "test", "-d", path]).is_ok() } +/// Fail closed when an existing named kind Cluster was created with a +/// different mountRoot. Recreating it is destructive and remains an explicit +/// user action; this check performs only `docker inspect`. +pub fn ensure_configured_mount_root(expected: &Path) -> Result<(), Box> { + let node = node_container_name(); + let raw = docker_output(&["inspect", "-f", "{{json .Mounts}}", &node])?; + if mount_inventory_has_same_path(&raw, expected)? { + return Ok(()); + } + + let name = active_cluster_name(); + Err(format!( + "kind cluster '{name}' exists with a different or missing mountRoot; expected the exact same-path mount {}. No resources were deleted. Recreate explicitly with `hops local reset --cluster-provider kind --docker-provider dory --cluster-name {name}` after confirming cluster recreation is safe", + expected.display() + ) + .into()) +} + +fn mount_inventory_has_same_path(raw: &str, expected: &Path) -> Result> { + #[derive(serde::Deserialize)] + struct DockerMount { + #[serde(rename = "Source")] + source: String, + #[serde(rename = "Destination")] + destination: String, + #[serde(rename = "RW", default)] + read_write: bool, + } + + let expected = expected.to_string_lossy(); + let mounts: Vec = serde_json::from_str(raw.trim()) + .map_err(|error| format!("unable to inspect kind node mount inventory: {error}"))?; + Ok(mounts + .iter() + .any(|mount| mount.read_write && mount.source == expected && mount.destination == expected)) +} + /// Build the kind cluster config YAML. /// /// When `extra_mount_host` is a directory, mount it at the same absolute path @@ -313,6 +497,7 @@ pub fn start(size: &SizeArgs) -> Result<(), Box> { let node = node_container_name(); if node_running() { log::info!("kind cluster '{name}' is already running"); + ensure_node_inotify_limits()?; log_mount_hint(); return Ok(()); } @@ -321,6 +506,8 @@ pub fn start(size: &SizeArgs) -> Result<(), Box> { // single-node cluster is reliable in practice but not guaranteed by kind. log::info!("Starting stopped kind node '{node}'..."); docker_run(&["start", &node])?; + ensure_node_inotify_limits()?; + normalize_dory_kubeconfig_endpoint()?; wait_for_api_after_restart() } @@ -436,12 +623,12 @@ fn preflight() -> Result<(), Box> { fn create_cluster() -> Result<(), Box> { let mount = default_extra_mount_root(); - let reg_port = registry_host_port(); + let reg_port = resolve_registry_host_port()?; let config = build_kind_config(mount.as_deref(), reg_port); if reg_port != 30500 { log::info!( - "kind registry hostPort={reg_port} (30500 busy or HOPS_KIND_REGISTRY_HOST_PORT set; \ - package push may need the same port)" + "kind registry hostPort={reg_port} ({REGISTRY_HOST_PORT_START} is already reserved or \ + {KIND_REGISTRY_HOST_PORT_ENV} was set)" ); } let name = active_cluster_name(); @@ -470,22 +657,127 @@ fn create_cluster() -> Result<(), Box> { return Err(format!("kind create cluster exited with {}", status).into()); } + normalize_dory_kubeconfig_endpoint()?; if let Some(ref m) = mount { verify_node_mount(m)?; } // Raise inotify limits: mounting large host trees (even $HOME/dev) can make // kube-proxy fail with "too many open files" under default instance caps. - raise_node_inotify_limits(); + ensure_node_inotify_limits()?; Ok(()) } -fn raise_node_inotify_limits() { +/// kind writes the Docker engine's published address into kubeconfig. Dory's +/// engine reports `0.0.0.0`, which is reachable through its local proxy but is +/// not present in the API server certificate. Rewrite only that Dory-specific +/// wildcard endpoint to the certificate's loopback SAN. +fn normalize_dory_kubeconfig_endpoint() -> Result<(), Box> { + let Some(docker_host) = resolve_docker_host() else { + return Ok(()); + }; + if !docker_host.ends_with("/.dory/dory.sock") { + return Ok(()); + } + + let config = run_cmd_output("kubectl", &["config", "view", "--raw", "-o", "json"])?; + let config: serde_json::Value = serde_json::from_str(&config)?; + let cluster_name = kube_context_name(); + let Some(current_server) = config + .get("clusters") + .and_then(serde_json::Value::as_array) + .and_then(|clusters| { + clusters.iter().find_map(|cluster| { + (cluster.get("name").and_then(serde_json::Value::as_str) + == Some(cluster_name.as_str())) + .then(|| { + cluster + .get("cluster") + .and_then(|cluster| cluster.get("server")) + .and_then(serde_json::Value::as_str) + }) + .flatten() + }) + }) + else { + return Ok(()); + }; + let Some(api_port) = published_host_port(&node_container_name(), "6443/tcp") else { + return Ok(()); + }; + let Some(server) = normalized_dory_server(current_server, api_port) else { + return Ok(()); + }; + + log::info!("Rewriting Dory kind API endpoint {current_server} -> {server}"); + run_cmd( + "kubectl", + &[ + "config", + "set-cluster", + &cluster_name, + &format!("--server={server}"), + ], + ) +} + +fn normalized_dory_server(current_server: &str, api_port: u16) -> Option { + let wildcard = current_server + .strip_prefix("https://0.0.0.0:") + .or_else(|| current_server.strip_prefix("https://[::]:"))?; + wildcard + .parse::() + .ok() + .filter(|current_port| *current_port == api_port) + .map(|_| format!("https://127.0.0.1:{api_port}")) +} + +fn inotify_sysctl_config() -> String { + format!( + "# Managed by hops local; reapplied by systemd-sysctl on kind node boot.\n\ +fs.inotify.max_user_instances = {INOTIFY_MAX_USER_INSTANCES}\n\ +fs.inotify.max_user_watches = {INOTIFY_MAX_USER_WATCHES}\n" + ) +} + +fn ensure_node_inotify_limits() -> Result<(), Box> { let node = node_container_name(); - let script = "sysctl -w fs.inotify.max_user_instances=8192 fs.inotify.max_user_watches=1048576 >/dev/null 2>&1 || true"; - match docker_output(&["exec", &node, "sh", "-c", script]) { - Ok(_) => log::info!("raised kind node inotify limits for host mounts"), - Err(e) => log::debug!("inotify sysctl skipped: {e}"), + let expected_instances = INOTIFY_MAX_USER_INSTANCES.to_string(); + let expected_watches = INOTIFY_MAX_USER_WATCHES.to_string(); + let mut child = docker_cmd(&[ + "exec", + "-i", + &node, + "sh", + "-c", + INSTALL_INOTIFY_SYSCTL_SCRIPT, + "hops-inotify-sysctl", + INOTIFY_SYSCTL_PATH, + &expected_instances, + &expected_watches, + ]) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn()?; + let mut stdin = child + .stdin + .take() + .ok_or("failed to open stdin for kind node inotify configuration")?; + stdin.write_all(inotify_sysctl_config().as_bytes())?; + drop(stdin); + let output = child.wait_with_output()?; + if !output.status.success() { + return Err(format!( + "failed to persist required kind node inotify limits in {INOTIFY_SYSCTL_PATH}: {}", + String::from_utf8_lossy(&output.stderr).trim() + ) + .into()); } + + log::info!( + "kind node inotify limits active and persistent: max_user_instances={INOTIFY_MAX_USER_INSTANCES}, max_user_watches={INOTIFY_MAX_USER_WATCHES} ({INOTIFY_SYSCTL_PATH})" + ); + Ok(()) } fn verify_node_mount(host_path: &Path) -> Result<(), Box> { @@ -617,10 +909,28 @@ mod tests { } #[test] - fn pick_registry_host_port_shifts_when_dory_k8s_present() { - assert_eq!(pick_registry_host_port(None, false), 30500); - assert_eq!(pick_registry_host_port(None, true), 30501); - assert_eq!(pick_registry_host_port(Some(30555), true), 30555); + fn pick_registry_host_port_uses_existing_binding_then_override_then_free_port() { + let unavailable = BTreeSet::from([30500, 30501]); + + assert_eq!( + pick_registry_host_port(Some(30542), Some(30555), &unavailable), + Some(30542) + ); + assert_eq!( + pick_registry_host_port(None, Some(30555), &unavailable), + Some(30555) + ); + assert_eq!( + pick_registry_host_port(None, None, &unavailable), + Some(30502) + ); + } + + #[test] + fn pick_registry_host_port_reports_exhausted_range() { + let unavailable = (REGISTRY_HOST_PORT_START..=REGISTRY_HOST_PORT_END).collect(); + + assert_eq!(pick_registry_host_port(None, None, &unavailable), None); } #[test] @@ -659,6 +969,22 @@ mod tests { assert!(cfg.contains("readOnly: false")); } + #[test] + fn kind_inotify_limits_are_persisted_for_node_restarts() { + assert!(INOTIFY_SYSCTL_PATH.starts_with("/etc/sysctl.d/")); + assert_eq!( + inotify_sysctl_config(), + "# Managed by hops local; reapplied by systemd-sysctl on kind node boot.\n\ +fs.inotify.max_user_instances = 8192\n\ +fs.inotify.max_user_watches = 1048576\n" + ); + assert!(INSTALL_INOTIFY_SYSCTL_SCRIPT.contains("sysctl -p \"${target}\"")); + assert!(INSTALL_INOTIFY_SYSCTL_SCRIPT + .contains("test \"${instances}\" -ge \"${expected_instances}\"")); + assert!(INSTALL_INOTIFY_SYSCTL_SCRIPT + .contains("test \"${watches}\" -ge \"${expected_watches}\"")); + } + #[test] fn hosts_toml_aliases_to_cluster_ip_over_https() { // Local package registry is HTTPS (self-signed); containerd must use @@ -719,4 +1045,33 @@ mod tests { let cfg = build_kind_config(Some(Path::new("/home/ci")), 30500); assert!(cfg.contains("/home/ci")); } + + #[test] + fn existing_kind_mount_requires_exact_same_path_read_write_binding() { + let exact = r#"[{"Source":"/workspace","Destination":"/workspace","RW":true}]"#; + let broader = r#"[{"Source":"/","Destination":"/","RW":true}]"#; + let read_only = r#"[{"Source":"/workspace","Destination":"/workspace","RW":false}]"#; + + assert!(mount_inventory_has_same_path(exact, Path::new("/workspace")).unwrap()); + assert!(!mount_inventory_has_same_path(broader, Path::new("/workspace")).unwrap()); + assert!(!mount_inventory_has_same_path(read_only, Path::new("/workspace")).unwrap()); + assert!(mount_inventory_has_same_path("not-json", Path::new("/workspace")).is_err()); + } + + #[test] + fn dory_wildcard_kubeconfig_endpoint_is_rewritten_to_certificate_san() { + assert_eq!( + normalized_dory_server("https://0.0.0.0:63903", 63903), + Some("https://127.0.0.1:63903".into()) + ); + assert_eq!( + normalized_dory_server("https://[::]:63903", 63903), + Some("https://127.0.0.1:63903".into()) + ); + assert_eq!( + normalized_dory_server("https://127.0.0.1:63903", 63903), + None + ); + assert_eq!(normalized_dory_server("https://0.0.0.0:63903", 6443), None); + } } diff --git a/src/commands/local/backend/providers.rs b/src/commands/local/backend/providers.rs index cec6ef6..804c857 100644 --- a/src/commands/local/backend/providers.rs +++ b/src/commands/local/backend/providers.rs @@ -166,6 +166,30 @@ pub fn resolve_provider_pair( Ok(Some(pair)) } +/// Translate the deprecated one-dimensional `--backend` flag into the +/// provider pair used by the current CLI. +/// +/// `kind` retains the platform default that was already used when only one +/// provider dimension was supplied: Dory on macOS and the default Docker +/// engine elsewhere. Product Dory and Colima remain self-paired. +pub fn provider_pair_for_legacy_backend(backend: Backend) -> ProviderPair { + match backend { + Backend::Kind if cfg!(target_os = "macos") => ProviderPair::kind_on_dory(), + Backend::Kind => ProviderPair { + cluster: ClusterProvider::Kind, + docker: DockerProvider::Docker, + }, + Backend::Dory => ProviderPair { + cluster: ClusterProvider::Dory, + docker: DockerProvider::Dory, + }, + Backend::Colima => ProviderPair { + cluster: ClusterProvider::Colima, + docker: DockerProvider::Colima, + }, + } +} + /// Apply docker-provider to process env for kind (and docker CLI). /// /// - `dory`: set DOCKER_HOST to `unix://$HOME/.dory/dory.sock` when unset @@ -262,4 +286,137 @@ mod tests { fn resolve_neither_returns_none() { assert!(resolve_provider_pair(None, None).unwrap().is_none()); } + + #[test] + fn provider_pair_matrix_preserves_baseline_acceptance() { + let accepted = [ + (ClusterProvider::Kind, DockerProvider::Dory), + (ClusterProvider::Kind, DockerProvider::Colima), + (ClusterProvider::Kind, DockerProvider::Docker), + (ClusterProvider::Dory, DockerProvider::Dory), + (ClusterProvider::Colima, DockerProvider::Colima), + ]; + let rejected = [ + (ClusterProvider::Dory, DockerProvider::Colima), + (ClusterProvider::Dory, DockerProvider::Docker), + (ClusterProvider::Colima, DockerProvider::Dory), + (ClusterProvider::Colima, DockerProvider::Docker), + ]; + + for (cluster, docker) in accepted { + let pair = ProviderPair { cluster, docker }; + assert_eq!( + resolve_provider_pair(Some(cluster), Some(docker)) + .unwrap() + .unwrap(), + pair, + "expected {cluster}+{docker} accepted" + ); + } + for (cluster, docker) in rejected { + let error = resolve_provider_pair(Some(cluster), Some(docker)).unwrap_err(); + assert_eq!( + error.to_string(), + format!( + "cluster-provider {cluster} requires docker-provider {cluster} (got {docker})" + ), + "expected {cluster}+{docker} rejected with the baseline diagnostic" + ); + } + } + + #[test] + fn provider_partial_input_matrix_preserves_platform_defaults() { + let default_docker = if cfg!(target_os = "macos") { + DockerProvider::Dory + } else { + DockerProvider::Docker + }; + + assert_eq!( + resolve_provider_pair(Some(ClusterProvider::Kind), None) + .unwrap() + .unwrap(), + ProviderPair { + cluster: ClusterProvider::Kind, + docker: default_docker, + } + ); + assert_eq!( + resolve_provider_pair(None, Some(DockerProvider::Dory)) + .unwrap() + .unwrap(), + ProviderPair::kind_on_dory() + ); + assert_eq!( + resolve_provider_pair(None, Some(DockerProvider::Colima)) + .unwrap() + .unwrap(), + ProviderPair { + cluster: ClusterProvider::Kind, + docker: DockerProvider::Colima, + } + ); + assert_eq!( + resolve_provider_pair(None, Some(DockerProvider::Docker)) + .unwrap() + .unwrap(), + ProviderPair { + cluster: ClusterProvider::Kind, + docker: DockerProvider::Docker, + } + ); + + let colima_error = resolve_provider_pair(Some(ClusterProvider::Colima), None).unwrap_err(); + assert_eq!( + colima_error.to_string(), + format!( + "cluster-provider colima requires docker-provider colima (got {default_docker})" + ) + ); + + let dory = resolve_provider_pair(Some(ClusterProvider::Dory), None); + if cfg!(target_os = "macos") { + assert_eq!( + dory.unwrap().unwrap(), + ProviderPair { + cluster: ClusterProvider::Dory, + docker: DockerProvider::Dory, + } + ); + } else { + assert_eq!( + dory.unwrap_err().to_string(), + "cluster-provider dory requires docker-provider dory (got docker)" + ); + } + } + + #[test] + fn deprecated_backend_maps_to_provider_defaults() { + let kind = provider_pair_for_legacy_backend(Backend::Kind); + assert_eq!(kind.cluster, ClusterProvider::Kind); + assert_eq!( + kind.docker, + if cfg!(target_os = "macos") { + DockerProvider::Dory + } else { + DockerProvider::Docker + } + ); + assert_eq!( + provider_pair_for_legacy_backend(Backend::Dory), + ProviderPair { + cluster: ClusterProvider::Dory, + docker: DockerProvider::Dory, + } + ); + assert_eq!( + provider_pair_for_legacy_backend(Backend::Colima), + ProviderPair { + cluster: ClusterProvider::Colima, + docker: DockerProvider::Colima, + } + ); + } } diff --git a/src/commands/local/github.rs b/src/commands/local/github.rs index aa24aa7..4a30b90 100644 --- a/src/commands/local/github.rs +++ b/src/commands/local/github.rs @@ -47,7 +47,7 @@ pub struct GithubArgs { pub refresh: bool, /// Write non-secret Provider / ProviderConfig YAML under this directory - /// (e.g. `./gitops/cluster`). Credential Secrets are not written. + /// (e.g. `./.gitops/local/cluster`). Credential Secrets are not written. #[arg(long)] pub gitops: Option, } diff --git a/src/commands/local/gitops.rs b/src/commands/local/gitops.rs index 0973053..c2c9494 100644 --- a/src/commands/local/gitops.rs +++ b/src/commands/local/gitops.rs @@ -1,33 +1,42 @@ -//! `hops local gitops` — control-plane and worktree Application reconcile. +//! `hops local gitops` — control-plane and Environment reconcile. //! //! ```text -//! hops local gitops cluster [PATH] # shared CP (meta gitops/cluster) -//! hops local gitops worktree # per-worktree apps (envs → namespace = --name) +//! hops local gitops cluster [PATH] # shared CP (.gitops/local/cluster) +//! hops local gitops environment # Environment apps → namespace = --name //! ``` //! //! Both **watch by default**; pass `--once` for a single reconcile (CI/scripts). use super::local_state_dir; -use super::workbench::application::{load_applications, resolve_delivery_host_path}; +use super::workbench::application::{ + load_applications, resolve_delivery_host_path, Application, APPLICATION_API_VERSION, + APPLICATION_KIND, +}; use super::workbench::cluster_gitops::{ reconcile_cluster_dir, resolve_cluster_path, should_reconcile_cluster_change, }; +use super::workbench::definition::{load_definition, load_environment_definition}; use super::workbench::delivery::{ attach_sync_delivery, discover_sync_targets, save_delivery_runtime, stop_delivery_runtime, DeliveryStrategy, NodePathProber, SystemNodeProber, }; use super::workbench::reconcile::{ - reconcile_applications, ReconcileOptions, SystemHelm, SystemKubectl, + reconcile_applications, HelmRunner, ReconcileOptions, SystemHelm, SystemKubectl, +}; +use super::workbench::registry::{ + activate_workspace_cluster, load_workspace, save_workspace, WorkspaceRecord, }; -use super::workbench::registry::{activate_workspace_cluster, load_workspace}; use super::workbench::watch::{ is_chart_or_env_path, should_ignore_watch_path, watch_roots_for_applications, WatchPathClass, }; use super::workbench::{namespace_for_name, slugify_name}; use clap::{Args, Subcommand}; use notify::{RecursiveMode, Watcher}; -use std::collections::BTreeMap; +use serde::Deserialize; +use serde_yaml::{Mapping, Value}; +use std::collections::{BTreeMap, BTreeSet}; use std::error::Error; +use std::fs; use std::path::{Path, PathBuf}; use std::sync::mpsc; use std::time::{Duration, Instant}; @@ -42,14 +51,15 @@ pub struct GitopsArgs { pub enum GitopsCommands { /// Shared control-plane gitops (packages, PSQLStack, AuthStack → local CP) Cluster(ClusterArgs), - /// Per-worktree app Applications (charts → namespace = --name) - Worktree(WorktreeArgs), + /// Reconcile an Environment's Applications (charts → namespace = --name) + #[command(alias = "worktree")] + Environment(EnvironmentArgs), } #[derive(Args, Debug)] pub struct ClusterArgs { /// Path to cluster gitops directory (PSQLStack, AuthStack, packages, …). - /// Default: `$HOPS_LOCAL_CLUSTER`, else walk up from cwd for `gitops/cluster`. + /// Default: `$HOPS_LOCAL_CLUSTER`, else walk up from cwd for `.gitops/local/cluster`. #[arg(value_name = "PATH")] pub path: Option, @@ -71,8 +81,8 @@ pub struct ClusterArgs { } #[derive(Args, Debug)] -pub struct WorktreeArgs { - /// Path to env directory of Application YAMLs (e.g. ./gitops/envs/local). +pub struct EnvironmentArgs { + /// Reusable Environment YAML, or a legacy directory of Application YAMLs. #[arg(value_name = "PATH")] pub path: PathBuf, @@ -80,7 +90,7 @@ pub struct WorktreeArgs { #[arg(long, short = 'n')] pub namespace: Option, - /// Workspace name for labels (defaults from namespace / path). + /// Runtime Environment name (defaults from Environment metadata / namespace / path). #[arg(long)] pub name: Option, @@ -104,7 +114,7 @@ pub struct WorktreeArgs { pub fn run(args: &GitopsArgs) -> Result<(), Box> { match &args.command { GitopsCommands::Cluster(a) => run_cluster(a), - GitopsCommands::Worktree(a) => run_worktree(a), + GitopsCommands::Environment(a) => run_environment(a), } } @@ -123,8 +133,8 @@ pub fn run_cluster(args: &ClusterArgs) -> Result<(), Box> { let cluster = resolve_cluster_path(None, args.path.as_deref()).ok_or_else(|| { "no cluster gitops directory found.\n\ - Pass a path: hops local gitops cluster ./gitops/cluster\n\ - Or set HOPS_LOCAL_CLUSTER, or create gitops/cluster at the meta repo root." + Pass a path: hops local gitops cluster ./.gitops/local/cluster\n\ + Or set HOPS_LOCAL_CLUSTER, or create .gitops/local/cluster at the project root." .to_string() })?; let cluster = cluster @@ -159,9 +169,352 @@ pub fn run_cluster(args: &ClusterArgs) -> Result<(), Box> { run_cluster_watch(&cluster, args.debounce, do_once) } -// ── worktree ───────────────────────────────────────────────────────────────── +// ── environment ────────────────────────────────────────────────────────────── + +fn run_environment(args: &EnvironmentArgs) -> Result<(), Box> { + if args.path.is_file() && yaml_kind(&args.path)?.as_deref() == Some("Environment") { + return run_environment_definition(args); + } + run_application_worktree(args) +} + +fn run_environment_definition(args: &EnvironmentArgs) -> Result<(), Box> { + let source = args + .path + .canonicalize() + .map_err(|error| format!("Environment path {}: {error}", args.path.display()))?; + let cluster_path = discover_cluster_definition(&source).ok_or_else(|| { + format!( + "no sibling or ancestor Cluster definition found for {}", + source.display() + ) + })?; + let cluster = load_definition(&cluster_path)?; + let loaded = load_environment_definition( + &source, + &cluster, + args.name.as_deref(), + args.namespace.as_deref(), + )?; + let workspace_name = loaded.environment.name.clone(); + let namespace = loaded.environment.namespace.clone(); + let worktree_root = loaded.environment.root.clone(); + let mut chart_watch_roots = BTreeSet::new(); + for deploy in &loaded.environment.deploys { + chart_watch_roots.insert(deploy.promote_chart.clone()); + chart_watch_roots.insert(deploy.application_root.join(".gitops/local")); + } + let chart_watch_roots: Vec<_> = chart_watch_roots.into_iter().collect(); + super::backend::kind::set_active_cluster_name(&cluster.cluster.name); + + let generated = if args.dry_run { + std::env::temp_dir().join(format!( + "hops-local-environment-{}-{workspace_name}", + std::process::id() + )) + } else { + local_state_dir()?.join("generated").join(&workspace_name) + }; + + let reconcile = || -> Result<(), Box> { + render_environment_applications( + &source, + &cluster_path, + &generated, + &workspace_name, + &namespace, + )?; + let legacy = EnvironmentArgs { + path: generated.clone(), + namespace: Some(namespace.clone()), + name: Some(workspace_name.clone()), + once: true, + watch: false, + debounce: args.debounce, + dry_run: args.dry_run, + }; + run_application_worktree(&legacy)?; + if !args.dry_run { + persist_environment_registration( + &workspace_name, + &source, + &worktree_root, + &cluster.cluster.name, + )?; + } + Ok(()) + }; + + reconcile()?; + if args.once || args.dry_run { + if args.dry_run { + let _ = fs::remove_dir_all(&generated); + } + return Ok(()); + } + + run_environment_watch( + &source, + &worktree_root, + &chart_watch_roots, + args.debounce, + reconcile, + ) +} + +fn yaml_kind(path: &Path) -> Result, Box> { + let text = fs::read_to_string(path)?; + let Some(document) = serde_yaml::Deserializer::from_str(&text).next() else { + return Ok(None); + }; + let value = Value::deserialize(document)?; + Ok(value + .get("kind") + .and_then(Value::as_str) + .map(str::to_string)) +} + +fn discover_cluster_definition(environment_file: &Path) -> Option { + environment_file + .parent()? + .ancestors() + .map(|directory| directory.join("cluster.yaml")) + .find(|candidate| candidate.is_file()) +} + +fn render_environment_applications( + environment_file: &Path, + cluster_file: &Path, + generated: &Path, + workspace_name: &str, + namespace: &str, +) -> Result<(), Box> { + render_environment_applications_with( + environment_file, + cluster_file, + generated, + workspace_name, + namespace, + &SystemHelm, + ) +} + +fn render_environment_applications_with( + environment_file: &Path, + cluster_file: &Path, + generated: &Path, + workspace_name: &str, + namespace: &str, + helm: &H, +) -> Result<(), Box> { + let cluster = load_definition(cluster_file)?; + let loaded = load_environment_definition( + environment_file, + &cluster, + Some(workspace_name), + Some(namespace), + )?; + fs::create_dir_all(generated)?; + for entry in fs::read_dir(generated)? { + let path = entry?.path(); + if path.is_file() + && matches!( + path.extension().and_then(|value| value.to_str()), + Some("yaml" | "yml") + ) + { + fs::remove_file(path)?; + } + } + + let mut rendered_apps = BTreeMap::::new(); + for (index, deploy) in loaded.environment.deploys.iter().enumerate() { + let mut values = loaded.environment.values.clone(); + merge_mapping(&mut values, &deploy.values); + values.insert(Value::String("local".into()), Value::Bool(true)); + values.insert( + Value::String("environment".into()), + string_mapping(&[ + ("name", &loaded.environment.name), + ("namespace", &loaded.environment.namespace), + ]), + ); + values.insert( + Value::String("source".into()), + string_mapping(&[("localPath", &deploy.application_root.to_string_lossy())]), + ); + let values_yaml = serde_yaml::to_string(&Value::Mapping(values))?; + let output = helm.template( + &format!("{}-promote-{index}", sanitize_name(workspace_name)), + &deploy.promote_chart, + &loaded.environment.namespace, + &values_yaml, + )?; + for document in serde_yaml::Deserializer::from_str(&output) { + let value = Value::deserialize(document)?; + if value.is_null() { + continue; + } + let kind = value.get("kind").and_then(Value::as_str).unwrap_or(""); + if kind != APPLICATION_KIND { + return Err(format!( + "promotion chart {} emitted unsupported local kind {kind:?}; direct KRM reconciliation belongs to the Cluster controller task", + deploy.promote_chart.display() + ) + .into()); + } + let mut application: Application = serde_yaml::from_value(value)?; + if application.api_version != APPLICATION_API_VERSION { + return Err(format!( + "promotion chart {} emitted Application apiVersion {:?}; expected {APPLICATION_API_VERSION}", + deploy.promote_chart.display(), + application.api_version + ) + .into()); + } + application.spec.source.delivery_path = + Some(loaded.environment.root.to_string_lossy().into_owned()); + application.spec.destination.namespace = Some(loaded.environment.namespace.clone()); + let name = application.metadata.name.clone(); + if rendered_apps.insert(name.clone(), application).is_some() { + return Err(format!("duplicate promoted Application name {name:?}").into()); + } + } + } + if rendered_apps.is_empty() { + return Err(format!( + "Environment {} produced no local Applications", + loaded.environment.name + ) + .into()); + } + for (name, application) in rendered_apps { + let path = generated.join(format!("{}.yaml", sanitize_name(&name))); + fs::write(path, serde_yaml::to_string(&application)?)?; + } + Ok(()) +} + +fn merge_mapping(base: &mut Mapping, overlay: &Mapping) { + for (key, value) in overlay { + match (base.get_mut(key), value) { + (Some(Value::Mapping(base_map)), Value::Mapping(overlay_map)) => { + merge_mapping(base_map, overlay_map) + } + _ => { + base.insert(key.clone(), value.clone()); + } + } + } +} + +fn string_mapping(values: &[(&str, &str)]) -> Value { + let mut mapping = Mapping::new(); + for (key, value) in values { + mapping.insert( + Value::String((*key).to_string()), + Value::String((*value).to_string()), + ); + } + Value::Mapping(mapping) +} + +fn sanitize_name(value: &str) -> String { + value + .chars() + .map(|character| { + if character.is_ascii_alphanumeric() || character == '-' { + character + } else { + '-' + } + }) + .collect::() + .trim_matches('-') + .to_string() +} + +fn persist_environment_registration( + workspace_name: &str, + source: &Path, + worktree_root: &Path, + cluster_name: &str, +) -> Result<(), Box> { + let state_dir = local_state_dir()?; + let Some(mut record) = load_workspace(&state_dir, workspace_name)? else { + return Err(format!("workspace {workspace_name:?} was not registered").into()); + }; + record.env_path = source.to_string_lossy().into_owned(); + record.project_root = Some(worktree_root.to_string_lossy().into_owned()); + record.cluster_name = Some(cluster_name.to_string()); + save_workspace(&state_dir, &record)?; + Ok(()) +} + +fn run_environment_watch( + environment_file: &Path, + worktree_root: &Path, + chart_roots: &[PathBuf], + debounce_secs: u64, + mut rebuild: F, +) -> Result<(), Box> +where + F: FnMut() -> Result<(), Box>, +{ + let debounce = Duration::from_secs(debounce_secs); + let (tx, rx) = mpsc::channel(); + let source = environment_file.to_path_buf(); + let watched_charts = chart_roots.to_vec(); + let mut watcher = + notify::recommended_watcher(move |result: notify::Result| match result { + Ok(event) => { + for path in event.paths { + if should_ignore_watch_path(&path) { + continue; + } + if is_environment_watch_path(&path, &source, &watched_charts) { + let _ = tx.send(()); + break; + } + } + } + Err(error) => log::debug!("Environment watch error: {error:?}"), + })?; + let environment_parent = environment_file.parent().ok_or_else(|| { + format!( + "Environment definition has no parent: {}", + environment_file.display() + ) + })?; + watcher.watch(environment_parent, RecursiveMode::NonRecursive)?; + for root in chart_roots { + if root.is_dir() { + watcher.watch(root, RecursiveMode::Recursive)?; + } + } + log::info!( + "Watching Environment {} and {} referenced promotion/deploy chart roots under {} (debounce {}s). Ctrl+C to stop.", + environment_file.display(), + chart_roots.len(), + worktree_root.display(), + debounce_secs + ); + loop { + rx.recv() + .map_err(|_| "Environment watcher channel closed")?; + wait_for_quiet(&rx, debounce)?; + match rebuild() { + Ok(()) => log::info!("Environment reconcile succeeded."), + Err(error) => log::error!("Environment reconcile failed: {error}"), + } + } +} + +fn is_environment_watch_path(path: &Path, source: &Path, chart_roots: &[PathBuf]) -> bool { + path == source || chart_roots.iter().any(|root| path.starts_with(root)) +} -fn run_worktree(args: &WorktreeArgs) -> Result<(), Box> { +fn run_application_worktree(args: &EnvironmentArgs) -> Result<(), Box> { let env_path = args .path .canonicalize() @@ -185,11 +538,12 @@ fn run_worktree(args: &WorktreeArgs) -> Result<(), Box> { .unwrap_or_else(|| namespace_for_name(&workspace_name)); // Sticky workspace→cluster: use bound kube context when registered. - if let Ok(state_dir) = local_state_dir() { - if let Ok(Some(rec)) = load_workspace(&state_dir, &workspace_name) { - if let Some((cluster, ctx)) = activate_workspace_cluster(&rec) { - log::info!("worktree gitops: bound cluster `{cluster}` (context {ctx})"); - } + let existing_workspace = local_state_dir() + .ok() + .and_then(|state_dir| load_workspace(&state_dir, &workspace_name).ok().flatten()); + if let Some(rec) = existing_workspace.as_ref() { + if let Some((cluster, ctx)) = activate_workspace_cluster(rec) { + log::info!("environment gitops: bound cluster `{cluster}` (context {ctx})"); } } @@ -201,7 +555,7 @@ fn run_worktree(args: &WorktreeArgs) -> Result<(), Box> { let (delivery_strategy, delivery_detail) = resolve_worktree_delivery(&app_delivery_host_paths, &SystemNodeProber)?; log::info!( - "worktree gitops: source delivery {} ({})", + "environment gitops: source delivery {} ({})", delivery_strategy.as_str(), delivery_detail ); @@ -217,7 +571,7 @@ fn run_worktree(args: &WorktreeArgs) -> Result<(), Box> { let do_once = || -> Result<(), Box> { log::info!( - "worktree gitops: Applications from {} → namespace {}", + "environment gitops: Applications from {} → namespace {}", env_path.display(), opts.namespace ); @@ -266,6 +620,15 @@ fn run_worktree(args: &WorktreeArgs) -> Result<(), Box> { }; do_once()?; + if !args.dry_run { + register_worktree( + &env_path, + &workspace_name, + &namespace, + delivery_strategy, + existing_workspace.as_ref(), + )?; + } if args.once || args.dry_run { return Ok(()); } @@ -273,12 +636,57 @@ fn run_worktree(args: &WorktreeArgs) -> Result<(), Box> { run_worktree_watch(&env_path, args.debounce, do_once) } +fn register_worktree( + env_path: &Path, + workspace_name: &str, + namespace: &str, + delivery_strategy: DeliveryStrategy, + existing: Option<&WorkspaceRecord>, +) -> Result<(), Box> { + let cluster_name = existing + .and_then(|record| record.cluster_name.clone()) + .filter(|name| !name.is_empty()) + .unwrap_or_else(super::backend::kind::active_cluster_name); + let kube_context = existing + .and_then(|record| record.kube_context.clone()) + .filter(|context| !context.is_empty()) + .or_else(super::kube_context_from_env) + .or_else(|| Some(format!("kind-{cluster_name}"))); + let project_root = discover_project_root(env_path) + .map(|path| path.to_string_lossy().into_owned()) + .or_else(|| existing.and_then(|record| record.project_root.clone())); + + let record = WorkspaceRecord { + name: workspace_name.to_string(), + namespace: namespace.to_string(), + env_path: env_path.to_string_lossy().into_owned(), + project_root, + delivery_mode: Some(delivery_strategy.as_str().to_string()), + updated_at: None, + cluster_name: Some(cluster_name), + kube_context, + }; + let path = save_workspace(&local_state_dir()?, &record)?; + log::info!( + "environment gitops: registered Environment `{workspace_name}` at {}", + path.display() + ); + Ok(()) +} + +fn discover_project_root(env_path: &Path) -> Option { + env_path + .ancestors() + .find(|candidate| candidate.join(".git").exists()) + .map(Path::to_path_buf) +} + fn resolve_worktree_delivery( app_paths: &BTreeMap, prober: &dyn NodePathProber, ) -> Result<(DeliveryStrategy, String), Box> { if app_paths.is_empty() { - return Err("worktree gitops found no Application source paths".into()); + return Err("environment gitops found no Application source paths".into()); } let mut all_visible = true; @@ -361,7 +769,7 @@ where } } log::info!( - "Worktree gitops watch active (debounce {}s). Env YAML + charts only. Ctrl+C to stop.", + "Environment gitops watch active (debounce {}s). Environment YAML + charts only. Ctrl+C to stop.", debounce_secs ); @@ -369,7 +777,7 @@ where rx.recv().map_err(|_| "watcher channel closed")?; wait_for_quiet(&rx, debounce)?; log::info!("──────────────────────────────────────────────"); - log::info!("Worktree gitops change, reconciling..."); + log::info!("Environment gitops change, reconciling..."); match rebuild() { Ok(()) => log::info!("Reconcile succeeded."), Err(e) => log::error!("Reconcile failed: {e}"), @@ -442,3 +850,223 @@ fn wait_for_quiet(rx: &mpsc::Receiver<()>, debounce: Duration) -> Result<(), Box } } } + +#[cfg(test)] +mod tests { + use super::*; + use std::fs; + use std::sync::Mutex; + + struct PromotionHelm { + values: Mutex>, + } + + impl PromotionHelm { + fn new() -> Self { + Self { + values: Mutex::new(Vec::new()), + } + } + } + + impl HelmRunner for PromotionHelm { + fn template( + &self, + _release: &str, + chart_path: &Path, + namespace: &str, + values_yaml: &str, + ) -> Result> { + self.values + .lock() + .unwrap() + .push(serde_yaml::from_str(values_yaml)?); + let application_root = chart_path + .parent() + .and_then(Path::parent) + .ok_or("promotion chart has no application root")?; + Ok(format!( + r#"apiVersion: hops.local/v1alpha1 +kind: Application +metadata: + name: gateway +spec: + source: + path: {}/.gitops/local + destination: + namespace: ignored +"#, + application_root.display() + ) + .replace("namespace: ignored", &format!("namespace: {namespace}"))) + } + } + + #[test] + fn discovers_project_root_from_git_ancestor() { + let root = std::env::temp_dir().join(format!( + "hops-gitops-root-{}-{}", + std::process::id(), + uuid::Uuid::new_v4() + )); + let env_path = root.join("gitops/envs/local"); + fs::create_dir_all(&env_path).unwrap(); + fs::write(root.join(".git"), "gitdir: /tmp/example\n").unwrap(); + + assert_eq!(discover_project_root(&env_path), Some(root.clone())); + + fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn project_root_is_unknown_without_git_ancestor() { + let root = std::env::temp_dir().join(format!( + "hops-gitops-no-root-{}-{}", + std::process::id(), + uuid::Uuid::new_v4() + )); + let env_path = root.join("gitops/envs/local"); + fs::create_dir_all(&env_path).unwrap(); + + assert_eq!(discover_project_root(&env_path), None); + + fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn renders_reusable_environment_for_runtime_identity() { + let root = std::env::temp_dir().join(format!( + "hops-gitops-environment-{}-{}", + std::process::id(), + uuid::Uuid::new_v4() + )); + fs::create_dir_all(&root).unwrap(); + let root = root.canonicalize().unwrap(); + let promote = root.join("apps/gateway/.gitops/promote"); + fs::create_dir_all(root.join(".gitops/local/cluster")).unwrap(); + fs::create_dir_all(root.join("apps/gateway/.gitops/local")).unwrap(); + fs::create_dir_all(&promote).unwrap(); + fs::write( + promote.join("Chart.yaml"), + "apiVersion: v2\nname: gateway-promote\nversion: 0.1.0\n", + ) + .unwrap(); + fs::write( + root.join(".gitops/local/cluster.yaml"), + r#"apiVersion: hops.local/v1alpha1 +kind: Cluster +metadata: + name: project-dev +spec: + clusterProvider: kind + dockerProvider: dory + mountRoot: ../.. + manifests: + path: .gitops/local/cluster +"#, + ) + .unwrap(); + fs::write( + root.join(".gitops/local/environment.yaml"), + r#"apiVersion: hops.local/v1alpha1 +kind: Environment +metadata: + name: local +spec: + clusterRef: + name: project-dev + root: . + values: + local: false + preview: false + feature: + enabled: false + deploys: + - path: apps/gateway + values: + feature: + enabled: true + revision: worktree +"#, + ) + .unwrap(); + + let generated = root.join("generated"); + let helm = PromotionHelm::new(); + render_environment_applications_with( + &root.join(".gitops/local/environment.yaml"), + &root.join(".gitops/local/cluster.yaml"), + &generated, + "feature-auth", + "feature-auth-ns", + &helm, + ) + .unwrap(); + + let values = helm.values.lock().unwrap(); + let values = values[0].as_mapping().unwrap(); + assert_eq!(values["local"], Value::Bool(true)); + assert_eq!(values["preview"], Value::Bool(false)); + assert_eq!(values["feature"]["enabled"], Value::Bool(true)); + assert_eq!(values["revision"], Value::String("worktree".into())); + assert_eq!( + values["environment"]["name"], + Value::String("feature-auth".into()) + ); + assert_eq!( + values["environment"]["namespace"], + Value::String("feature-auth-ns".into()) + ); + + let applications = load_applications(&generated).unwrap(); + assert_eq!(applications.len(), 1); + let application = &applications[0].1; + assert_eq!(application.metadata.name, "gateway"); + assert_eq!( + application.spec.destination.namespace.as_deref(), + Some("feature-auth-ns") + ); + let expected_delivery_path = root.to_string_lossy().into_owned(); + assert_eq!( + application.spec.source.delivery_path.as_deref(), + Some(expected_delivery_path.as_str()) + ); + + fs::remove_dir_all(root).unwrap(); + } + + #[test] + fn environment_watch_filters_to_definition_and_referenced_charts() { + let source = Path::new("/project/.gitops/local/environment.yaml"); + let chart_roots = vec![ + PathBuf::from("/project/apps/api/.gitops/promote"), + PathBuf::from("/project/apps/api/.gitops/local"), + ]; + assert!(is_environment_watch_path(source, source, &chart_roots)); + assert!(is_environment_watch_path( + Path::new("/project/apps/api/.gitops/promote/templates/application.yaml"), + source, + &chart_roots, + )); + assert!(is_environment_watch_path( + Path::new("/project/apps/api/.gitops/local/values.yaml"), + source, + &chart_roots, + )); + assert!(!is_environment_watch_path( + Path::new("/project/apps/api/src/main.rs"), + source, + &chart_roots, + )); + assert!(!is_environment_watch_path( + Path::new("/project/apps/api/.gitops/deploy/values.yaml"), + source, + &chart_roots, + )); + assert!(!is_environment_watch_path( + Path::new("/project/apps/other/.gitops/deploy/values.yaml"), + source, + &chart_roots, + )); + } +} diff --git a/src/commands/local/gitops_write.rs b/src/commands/local/gitops_write.rs index 725eba7..2708c2b 100644 --- a/src/commands/local/gitops_write.rs +++ b/src/commands/local/gitops_write.rs @@ -1,4 +1,4 @@ -//! Write non-secret provider manifests under a gitops/cluster directory. +//! Write non-secret provider manifests under a .gitops/local/cluster directory. //! //! Credential Secrets stay live-only until a local external-secrets story exists. //! `--gitops` writers intentionally omit Secret data. diff --git a/src/commands/local/mod.rs b/src/commands/local/mod.rs index a10aced..0149870 100644 --- a/src/commands/local/mod.rs +++ b/src/commands/local/mod.rs @@ -126,9 +126,14 @@ pub struct LocalArgs { /// Only used with cluster-provider dory. /// /// Named `--dory-name` (not `--name`) so it never collides with workspace - /// `--name` on `hops local down|status|open|gitops worktree`. + /// `--name` on `hops local down|status|open|gitops environment`. #[arg(long = "dory-name", global = true, value_name = "NAME")] pub dory_name: Option, + + /// Deprecated one-dimensional provider selection. Use + /// --cluster-provider and --docker-provider instead. + #[arg(long, global = true, value_enum)] + pub backend: Option, } #[derive(Subcommand, Debug)] @@ -139,6 +144,8 @@ pub enum LocalCommands { Reset, /// Start local k8s and ensure Crossplane control plane (skips helm when already healthy) Start(start::StartArgs), + /// Start or reuse the Cluster declared by .gitops/local/cluster.yaml + Up(workbench::definition::UpArgs), /// Resize the local cluster VM without destroying cluster state (colima cluster provider only) Resize(resize::ResizeArgs), /// Check what `hops local start` set up and report drift @@ -149,7 +156,7 @@ pub enum LocalCommands { Status(status::StatusArgs), /// Open the workspace UI URL in a browser Open(open::OpenArgs), - /// Local gitops: `cluster` (shared CP) or `worktree` (app namespaces) + /// Local gitops: `cluster` (shared CP) or `environment` (app namespaces) Gitops(gitops::GitopsArgs), /// Configure crossplane-contrib provider-family-aws and AWS ProviderConfig Aws(aws::AwsArgs), @@ -170,6 +177,20 @@ pub enum LocalCommands { } pub fn run(args: &LocalArgs) -> Result<(), Box> { + if let LocalCommands::Up(up_args) = &args.command { + return workbench::definition::run_up( + up_args, + workbench::definition::UpOverrides { + cluster_provider: args.cluster_provider, + docker_provider: args.docker_provider, + legacy_backend: args.backend, + cluster_name: args.cluster_name.as_deref(), + context: args.context.as_deref(), + dory_name: args.dory_name.as_deref(), + }, + ); + } + if let Some(name) = args .dory_name .as_deref() @@ -179,10 +200,29 @@ pub fn run(args: &LocalArgs) -> Result<(), Box> { backend::persist_dory_context_name(name)?; } + let (cluster_provider, docker_provider) = match args.backend { + Some(_) if args.cluster_provider.is_some() || args.docker_provider.is_some() => { + return Err( + "deprecated --backend cannot be combined with --cluster-provider or --docker-provider" + .into(), + ); + } + Some(legacy) => { + let pair = backend::providers::provider_pair_for_legacy_backend(legacy); + log::warn!( + "--backend is deprecated; use --cluster-provider {} --docker-provider {}", + pair.cluster, + pair.docker + ); + (Some(pair.cluster), Some(pair.docker)) + } + None => (args.cluster_provider, args.docker_provider), + }; + let explicit_context = args.context.as_deref().filter(|ctx| !ctx.is_empty()); let backend = backend::activate_with_providers( - args.cluster_provider, - args.docker_provider, + cluster_provider, + docker_provider, args.cluster_name.as_deref(), explicit_context, )?; @@ -191,6 +231,7 @@ pub fn run(args: &LocalArgs) -> Result<(), Box> { LocalCommands::Install => install::run(backend), LocalCommands::Reset => reset::run(backend), LocalCommands::Start(start_args) => start::run(backend, start_args), + LocalCommands::Up(_) => unreachable!("up dispatch returns before generic activation"), LocalCommands::Resize(resize_args) => resize::run(backend, resize_args), LocalCommands::Doctor => doctor::run(), LocalCommands::Down(down_args) => down::run(down_args), @@ -494,13 +535,13 @@ mod tests { let parsed = Cli::try_parse_from([ "hops-local-test", "gitops", - "worktree", + "environment", "./gitops/envs/local", "--name", "alice", "--once", ]) - .expect("parse gitops worktree --name alice"); + .expect("parse gitops environment --name alice"); assert!( parsed.local.dory_name.is_none(), "workspace --name must not set dory_name; got {:?}", @@ -508,11 +549,11 @@ mod tests { ); match parsed.local.command { LocalCommands::Gitops(gitops) => match gitops.command { - gitops::GitopsCommands::Worktree(worktree) => { - assert_eq!(worktree.name.as_deref(), Some("alice")); - assert!(worktree.once); + gitops::GitopsCommands::Environment(environment) => { + assert_eq!(environment.name.as_deref(), Some("alice")); + assert!(environment.once); } - other => panic!("expected gitops worktree, got {other:?}"), + other => panic!("expected gitops environment, got {other:?}"), }, other => panic!("expected Gitops, got {other:?}"), } @@ -534,19 +575,19 @@ mod tests { "--dory-name", "mine", "gitops", - "worktree", + "environment", "./env", "--name", "bob", ]) - .expect("parse --dory-name mine gitops worktree --name bob"); + .expect("parse --dory-name mine gitops environment --name bob"); assert_eq!(parsed.local.dory_name.as_deref(), Some("mine")); match parsed.local.command { LocalCommands::Gitops(gitops) => match gitops.command { - gitops::GitopsCommands::Worktree(worktree) => { - assert_eq!(worktree.name.as_deref(), Some("bob")); + gitops::GitopsCommands::Environment(environment) => { + assert_eq!(environment.name.as_deref(), Some("bob")); } - other => panic!("expected gitops worktree, got {other:?}"), + other => panic!("expected gitops environment, got {other:?}"), }, other => panic!("expected Gitops, got {other:?}"), } diff --git a/src/commands/local/status.rs b/src/commands/local/status.rs index 093d890..c590147 100644 --- a/src/commands/local/status.rs +++ b/src/commands/local/status.rs @@ -41,7 +41,7 @@ pub fn run(args: &StatusArgs) -> Result<(), Box> { if workspaces.is_empty() { println!("No local workspaces registered."); - println!("Apply one with: hops local gitops worktree --name "); + println!("Apply one with: hops local gitops environment --name "); return Ok(()); } diff --git a/src/commands/local/workbench/application.rs b/src/commands/local/workbench/application.rs index 504ffde..69fd30a 100644 --- a/src/commands/local/workbench/application.rs +++ b/src/commands/local/workbench/application.rs @@ -274,7 +274,7 @@ spec: helm: values: local: true - appRuntime: cluster-dev + preview: false syncPolicy: prune: false "#; @@ -289,7 +289,7 @@ spec: assert!(!app.spec.sync_policy.prune); let values = app.spec.source.helm.values.unwrap(); assert_eq!(values["local"], Value::Bool(true)); - assert_eq!(values["appRuntime"], Value::String("cluster-dev".into())); + assert_eq!(values["preview"], Value::Bool(false)); } #[test] diff --git a/src/commands/local/workbench/cluster_dns.rs b/src/commands/local/workbench/cluster_dns.rs index 24be2ff..6e658eb 100644 --- a/src/commands/local/workbench/cluster_dns.rs +++ b/src/commands/local/workbench/cluster_dns.rs @@ -94,6 +94,10 @@ pub fn allocate_service_ips( let mut out = BTreeMap::new(); let mut used: Vec = existing.values().cloned().collect(); for svc in services { + // A multi-port Service still owns one DNS name and one loopback IP. + if out.contains_key(&svc.name) { + continue; + } let key = alloc_key(namespace, &svc.name); if let Some(ip) = existing.get(&key) { out.insert(svc.name.clone(), ip.clone()); @@ -259,7 +263,9 @@ pub fn save_ip_alloc(state_dir: &Path, alloc: &DnsIpAlloc) -> Result<(), Box>(); + assert_eq!(alloc.bindings.len(), 16); + assert_eq!(unique_ips.len(), 16); + fs::remove_dir_all(state_dir).unwrap(); + } } diff --git a/src/commands/local/workbench/cluster_gitops.rs b/src/commands/local/workbench/cluster_gitops.rs index d51fdaf..209fffe 100644 --- a/src/commands/local/workbench/cluster_gitops.rs +++ b/src/commands/local/workbench/cluster_gitops.rs @@ -1,15 +1,15 @@ //! Reconcile a cluster gitops tree onto the **shared** local control plane. //! -//! One local CP (dory/colima/kind) serves many worktrees/projects. Cluster -//! config is **not** per-worktree — it lives at the meta repo root (or any -//! path passed via `--cluster`): +//! One local CP (dory/colima/kind) serves many worktrees. Every checkout has +//! the same committed definitions, but only one watcher reconciles the +//! Cluster-owned tree selected for that control plane: //! //! ```text -//! / # meta root -//! gitops/cluster/ # CP: PSQLStack, AuthStack, packages… -//! clients/foo/.gitops/deploy/ # per-project charts -//! platform/api/.gitops/deploy/ -//! gitops/envs/local/ # Application YAMLs → namespace = --name +//! / # checkout root +//! .gitops/local/cluster/ # CP: PSQLStack, AuthStack, packages… +//! .gitops/local/environment.yaml # reusable checkout Environment +//! clients/foo/.gitops/local/ # editable local application charts +//! platform/api/.gitops/local/ //! ``` //! //! Env Applications only isolate **app** namespaces. Cluster YAML is applied @@ -34,8 +34,12 @@ pub struct ClusterReconcileResult { /// Order: /// 1. Explicit `override_path` (`--cluster`) /// 2. Env var `HOPS_LOCAL_CLUSTER` -/// 3. Walk up from `env_path` looking for `gitops/cluster` or `cluster` -/// 4. Walk up from cwd looking for `gitops/cluster` +/// 3. Walk up from `env_path` looking for `.gitops/local/cluster` +/// 4. Walk up from cwd looking for `.gitops/local/cluster` +/// +/// The former `.gitops/cluster`, `gitops/cluster`, and `cluster` layouts +/// remain migration fallbacks after the committed `.gitops/local/cluster` +/// convention. /// /// Returns the first existing directory. Explicit override that does not exist /// is left to the caller to error on canonicalize. @@ -66,9 +70,9 @@ pub fn resolve_cluster_path( /// Discover a cluster tree near an env path (or walk to meta root). /// /// ```text -/// gitops/envs/local → sibling gitops/cluster -/// some/deep/project → walk up → /gitops/cluster -/// /gitops → /gitops/cluster +/// .gitops/local/environment.yaml → sibling .gitops/local/cluster +/// some/deep/project → walk up → /.gitops/local/cluster +/// /.gitops/local → /.gitops/local/cluster /// ``` pub fn discover_cluster_path(env_path: &Path) -> Option { let env = env_path @@ -100,15 +104,21 @@ pub fn discover_cluster_path(env_path: &Path) -> Option { } } - // Meta-root walk: any ancestor with gitops/cluster or cluster/ + // Meta-root walk: prefer the committed .gitops/local/cluster convention, + // then retain the old paths as migration fallbacks. walk_up_for_cluster(&env) } -/// Walk from `start` toward filesystem root for `gitops/cluster` or `cluster`. +/// Walk from `start` toward filesystem root for `.gitops/local/cluster` and legacy layouts. fn walk_up_for_cluster(start: &Path) -> Option { let mut cur = start.canonicalize().unwrap_or_else(|_| start.to_path_buf()); loop { - for candidate in [cur.join("gitops").join("cluster"), cur.join("cluster")] { + for candidate in [ + cur.join(".gitops").join("local").join("cluster"), + cur.join(".gitops").join("cluster"), + cur.join("gitops").join("cluster"), + cur.join("cluster"), + ] { if candidate.is_dir() { return Some(candidate); } @@ -417,6 +427,32 @@ mod tests { let _ = fs::remove_dir_all(&dir); } + #[test] + fn discover_prefers_dot_gitops_cluster() { + let dir = std::env::temp_dir().join(format!( + "hops-cg-dot-meta-{}-{}", + std::process::id(), + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + let preferred = dir.join(".gitops/local/cluster"); + let legacy = dir.join(".gitops/cluster"); + let environment = dir.join(".gitops/local/environment.yaml"); + fs::create_dir_all(&preferred).unwrap(); + fs::create_dir_all(&legacy).unwrap(); + fs::create_dir_all(environment.parent().unwrap()).unwrap(); + fs::write(&environment, "kind: Environment\n").unwrap(); + + let found = discover_cluster_path(&environment).unwrap(); + assert_eq!( + found.canonicalize().unwrap(), + preferred.canonicalize().unwrap() + ); + let _ = fs::remove_dir_all(&dir); + } + #[test] fn skips_examples_and_docs() { let dir = std::env::temp_dir().join(format!("hops-cg-skip-{}", std::process::id())); diff --git a/src/commands/local/workbench/definition.rs b/src/commands/local/workbench/definition.rs new file mode 100644 index 0000000..db384bc --- /dev/null +++ b/src/commands/local/workbench/definition.rs @@ -0,0 +1,1140 @@ +//! Kubernetes-shaped Cluster and independently reusable Environment loading. +//! +//! This module intentionally stops at a validated, immutable handoff. The +//! long-running controller consumes [`LoadedDefinition`] in the next rollout +//! task; `hops local up` currently owns definition validation and named local +//! cluster create/reuse only. + +use crate::commands::local::backend::{self, Backend, ClusterProvider, DockerProvider, SizeArgs}; +use clap::Args; +use serde::de::DeserializeOwned; +use serde::Deserialize; +use serde_yaml::{Mapping, Value}; +use std::collections::BTreeSet; +use std::error::Error; +use std::ffi::OsString; +use std::fs; +use std::io; +use std::path::{Component, Path, PathBuf}; + +pub const API_VERSION: &str = "hops.local/v1alpha1"; +pub const DEFAULT_DEFINITION_FILE: &str = ".gitops/local/cluster.yaml"; +pub const LEGACY_DEFINITION_FILE: &str = "cluster.yaml"; +pub const DEFAULT_ENVIRONMENT_FILE: &str = ".gitops/local/environment.yaml"; +pub const CLUSTER_MANIFESTS_PATH: &str = ".gitops/local/cluster"; +pub const LEGACY_CLUSTER_MANIFESTS_PATH: &str = ".gitops/cluster"; +pub const PROMOTE_CHART_PATH: &str = ".gitops/promote"; + +#[derive(Args, Debug, Clone)] +pub struct UpArgs { + /// Cluster definition. Defaults to ./.gitops/local/cluster.yaml. + #[arg(short = 'f', long = "file", value_name = "PATH")] + pub file: Option, +} + +#[derive(Debug, Clone, Copy, Default)] +pub struct UpOverrides<'a> { + pub cluster_provider: Option, + pub docker_provider: Option, + pub legacy_backend: Option, + pub cluster_name: Option<&'a str>, + pub context: Option<&'a str>, + pub dory_name: Option<&'a str>, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct LoadedDefinition { + pub source: PathBuf, + pub cluster: ClusterDefinition, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct LoadedEnvironment { + pub source: PathBuf, + pub environment: EnvironmentDefinition, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct ClusterDefinition { + pub name: String, + pub cluster_provider: ClusterProvider, + pub docker_provider: DockerProvider, + pub mount_root: PathBuf, + pub manifests_path: PathBuf, + pub secret_sync: Option, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct SecretSyncDefinition { + /// Resolved input path only. Value representation and Vault ownership are + /// intentionally left to the separately approved secret-sync contract. + pub path: PathBuf, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct EnvironmentDefinition { + pub name: String, + pub namespace: String, + pub cluster_ref: String, + pub root: PathBuf, + pub values: Mapping, + pub deploys: Vec, +} + +#[derive(Debug, Clone, PartialEq)] +pub struct DeployDefinition { + pub application_root: PathBuf, + pub promote_chart: PathBuf, + pub values: Mapping, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +struct DocumentProbe { + api_version: String, + kind: String, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ClusterDocument { + api_version: String, + kind: String, + metadata: ObjectMetadata, + spec: ClusterSpec, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct EnvironmentDocument { + api_version: String, + kind: String, + metadata: ObjectMetadata, + spec: EnvironmentSpec, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ObjectMetadata { + name: String, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ClusterSpec { + cluster_provider: ClusterProvider, + docker_provider: DockerProvider, + mount_root: PathBuf, + manifests: ManifestsSpec, + #[serde(default)] + secret_sync: Option, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ManifestsSpec { + path: PathBuf, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct SecretSyncSpec { + path: PathBuf, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct EnvironmentSpec { + cluster_ref: ClusterReference, + root: PathBuf, + #[serde(default)] + namespace: Option, + #[serde(default)] + values: Mapping, + deploys: Vec, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct ClusterReference { + name: String, +} + +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase", deny_unknown_fields)] +struct DeploySpec { + path: PathBuf, + #[serde(default)] + values: Mapping, +} + +pub fn run_up(args: &UpArgs, overrides: UpOverrides<'_>) -> Result<(), Box> { + let cwd = std::env::current_dir()?; + let source = definition_path(args.file.as_deref(), &cwd); + + // All parsing, identity, provider, and filesystem validation happens + // before process state, local state, or the cluster can be mutated. + let definition = load_definition(&source)?; + validate_overrides(&definition, overrides)?; + + if let Some(name) = overrides + .dory_name + .map(str::trim) + .filter(|name| !name.is_empty()) + { + backend::persist_dory_context_name(name)?; + } + + backend::kind::set_active_cluster_name(&definition.cluster.name); + backend::apply_docker_provider_env(definition.cluster.docker_provider)?; + + let existing_kind = definition.cluster.cluster_provider == ClusterProvider::Kind + && backend::kind::cluster_exists(); + if definition.cluster.cluster_provider == ClusterProvider::Kind { + backend::kind::set_extra_mount_root(&definition.cluster.mount_root); + if existing_kind { + backend::kind::ensure_configured_mount_root(&definition.cluster.mount_root)?; + } + } + + let explicit_context = overrides + .context + .map(str::trim) + .filter(|value| !value.is_empty()); + let active_backend = backend::activate_with_providers( + Some(definition.cluster.cluster_provider), + Some(definition.cluster.docker_provider), + Some(&definition.cluster.name), + explicit_context, + )?; + active_backend.start(&SizeArgs::default(), false)?; + backend::persist_providers( + backend::providers::ProviderPair { + cluster: definition.cluster.cluster_provider, + docker: definition.cluster.docker_provider, + }, + Some(&definition.cluster.name), + )?; + + log::info!( + "Cluster '{}' ready: context={} clusterProvider={} dockerProvider={} mountRoot={} definition={}", + definition.cluster.name, + expected_context(&definition, overrides), + definition.cluster.cluster_provider, + definition.cluster.docker_provider, + definition.cluster.mount_root.display(), + definition.source.display() + ); + log::info!( + "Validated shared manifests: {}", + definition.cluster.manifests_path.display() + ); + log::info!( + "Cluster definition contains no Environment inventory; register each checkout with `hops local gitops environment .gitops/local/environment.yaml --name `" + ); + + Ok(()) +} + +pub fn definition_path(file: Option<&Path>, cwd: &Path) -> PathBuf { + match file { + Some(path) if path.is_absolute() => path.to_path_buf(), + Some(path) => cwd.join(path), + None => { + let preferred = cwd.join(DEFAULT_DEFINITION_FILE); + let legacy = cwd.join(LEGACY_DEFINITION_FILE); + if preferred.is_file() || !legacy.is_file() { + preferred + } else { + log::warn!( + "using legacy Cluster definition {}; move it to {}", + legacy.display(), + preferred.display() + ); + legacy + } + } + } +} + +pub fn load_definition(path: &Path) -> Result> { + let source = path.canonicalize().map_err(|error| { + format!( + "unable to resolve Cluster definition {}: {error}", + path.display() + ) + })?; + if !source.is_file() { + return Err(format!("Cluster definition is not a file: {}", source.display()).into()); + } + let definition_root = source + .parent() + .ok_or_else(|| format!("Cluster definition has no parent: {}", source.display()))? + .canonicalize()?; + let yaml = fs::read_to_string(&source).map_err(|error| { + format!( + "unable to read Cluster definition {}: {error}", + source.display() + ) + })?; + + let mut clusters = Vec::::new(); + for (index, document) in serde_yaml::Deserializer::from_str(&yaml).enumerate() { + let number = index + 1; + let value = Value::deserialize(document).map_err(|error| { + format!( + "{} document {number}: invalid YAML: {error}", + source.display() + ) + })?; + if value.is_null() { + continue; + } + let probe: DocumentProbe = parse_document(value.clone(), &source, number)?; + if probe.api_version != API_VERSION { + return Err(format!( + "{} document {number}: unsupported apiVersion {:?}; expected {API_VERSION}", + source.display(), + probe.api_version + ) + .into()); + } + match probe.kind.as_str() { + "Cluster" => clusters.push(parse_document(value, &source, number)?), + "Environment" => { + return Err(format!( + "{} document {number}: Environment instances must not be committed in the Cluster definition; pass .gitops/local/environment.yaml to `hops local gitops environment`", + source.display() + ) + .into()) + } + other => { + return Err(format!( + "{} document {number}: unsupported kind {other:?}; expected Cluster", + source.display() + ) + .into()) + } + } + } + + if clusters.len() != 1 { + return Err(format!( + "{}: expected exactly one {API_VERSION} Cluster document, found {}", + source.display(), + clusters.len() + ) + .into()); + } + let raw_cluster: ClusterDocument = clusters.remove(0); + debug_assert_eq!(raw_cluster.api_version, API_VERSION); + debug_assert_eq!(raw_cluster.kind, "Cluster"); + validate_dns_label("Cluster.metadata.name", &raw_cluster.metadata.name)?; + let provider_pair = backend::providers::ProviderPair { + cluster: raw_cluster.spec.cluster_provider, + docker: raw_cluster.spec.docker_provider, + }; + provider_pair + .validate() + .map_err(|error| format!("Cluster.spec provider pair is invalid: {error}"))?; + + let mount_root = resolve_mount_root( + &definition_root, + &raw_cluster.spec.mount_root, + "Cluster.spec.mountRoot", + )?; + ensure_within(&mount_root, &definition_root, "Cluster definition")?; + let manifests_relative = &raw_cluster.spec.manifests.path; + if manifests_relative != Path::new(CLUSTER_MANIFESTS_PATH) + && manifests_relative != Path::new(LEGACY_CLUSTER_MANIFESTS_PATH) + { + return Err(format!( + "Cluster.spec.manifests.path must be {CLUSTER_MANIFESTS_PATH:?} (or legacy {LEGACY_CLUSTER_MANIFESTS_PATH:?}); got {:?}", + raw_cluster.spec.manifests.path.display().to_string() + ) + .into()); + } + if manifests_relative == Path::new(LEGACY_CLUSTER_MANIFESTS_PATH) { + log::warn!( + "using legacy Cluster manifest path {LEGACY_CLUSTER_MANIFESTS_PATH}; move it to {CLUSTER_MANIFESTS_PATH}" + ); + } + let manifests_base = if source.ends_with(DEFAULT_DEFINITION_FILE) { + &mount_root + } else { + &definition_root + }; + let manifests_path = resolve_bounded_path( + &mount_root, + manifests_base, + manifests_relative, + "Cluster.spec.manifests.path", + true, + )?; + let secret_sync = raw_cluster + .spec + .secret_sync + .map(|secret| { + resolve_bounded_path( + &mount_root, + &mount_root, + &secret.path, + "Cluster.spec.secretSync.path", + false, + ) + .map(|path| SecretSyncDefinition { path }) + }) + .transpose()?; + + Ok(LoadedDefinition { + source, + cluster: ClusterDefinition { + name: raw_cluster.metadata.name, + cluster_provider: raw_cluster.spec.cluster_provider, + docker_provider: raw_cluster.spec.docker_provider, + mount_root, + manifests_path, + secret_sync, + }, + }) +} + +pub fn load_environment_definition( + path: &Path, + cluster: &LoadedDefinition, + name_override: Option<&str>, + namespace_override: Option<&str>, +) -> Result> { + let source = path.canonicalize().map_err(|error| { + format!( + "unable to resolve Environment definition {}: {error}", + path.display() + ) + })?; + if !source.is_file() { + return Err(format!("Environment definition is not a file: {}", source.display()).into()); + } + ensure_within( + &cluster.cluster.mount_root, + &source, + "Environment definition", + )?; + let definition_root = source + .parent() + .ok_or_else(|| format!("Environment definition has no parent: {}", source.display()))? + .canonicalize()?; + let yaml = fs::read_to_string(&source).map_err(|error| { + format!( + "unable to read Environment definition {}: {error}", + source.display() + ) + })?; + + let mut environments = Vec::::new(); + for (index, document) in serde_yaml::Deserializer::from_str(&yaml).enumerate() { + let number = index + 1; + let value = Value::deserialize(document).map_err(|error| { + format!( + "{} document {number}: invalid YAML: {error}", + source.display() + ) + })?; + if value.is_null() { + continue; + } + let probe: DocumentProbe = parse_document(value.clone(), &source, number)?; + if probe.api_version != API_VERSION { + return Err(format!( + "{} document {number}: unsupported apiVersion {:?}; expected {API_VERSION}", + source.display(), + probe.api_version + ) + .into()); + } + if probe.kind != "Environment" { + return Err(format!( + "{} document {number}: unsupported kind {:?}; expected Environment", + source.display(), + probe.kind + ) + .into()); + } + environments.push(parse_document(value, &source, number)?); + } + if environments.len() != 1 { + return Err(format!( + "{}: expected exactly one {API_VERSION} Environment document, found {}", + source.display(), + environments.len() + ) + .into()); + } + + let raw = environments.remove(0); + debug_assert_eq!(raw.api_version, API_VERSION); + debug_assert_eq!(raw.kind, "Environment"); + let name = name_override + .map(str::trim) + .filter(|value| !value.is_empty()) + .unwrap_or(&raw.metadata.name) + .to_string(); + validate_dns_label("Environment runtime name", &name)?; + if raw.spec.cluster_ref.name != cluster.cluster.name { + return Err(format!( + "Environment {name:?} references Cluster {:?}, but the selected definition contains {:?}", + raw.spec.cluster_ref.name, cluster.cluster.name + ) + .into()); + } + let namespace = namespace_override + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(str::to_string) + .or(raw.spec.namespace) + .unwrap_or_else(|| name.clone()); + validate_dns_label("Environment namespace", &namespace)?; + let root_base = if source.ends_with(DEFAULT_ENVIRONMENT_FILE) { + &cluster.cluster.mount_root + } else { + &definition_root + }; + let root = resolve_bounded_path( + &cluster.cluster.mount_root, + root_base, + &raw.spec.root, + &format!("Environment {name:?} spec.root"), + true, + )?; + + let mut seen_deploys = BTreeSet::new(); + let mut deploys = Vec::with_capacity(raw.spec.deploys.len()); + for deploy in raw.spec.deploys { + let application_root = resolve_bounded_path( + &cluster.cluster.mount_root, + &root, + &deploy.path, + &format!("Environment {name:?} deploys[].path"), + true, + )?; + if !seen_deploys.insert(application_root.clone()) { + return Err(format!( + "Environment {name:?} contains duplicate deploy application root {}", + application_root.display() + ) + .into()); + } + let promote_chart = resolve_bounded_path( + &cluster.cluster.mount_root, + &application_root, + Path::new(PROMOTE_CHART_PATH), + &format!("Environment {name:?} deploy promote chart"), + false, + )?; + deploys.push(DeployDefinition { + application_root, + promote_chart, + values: deploy.values, + }); + } + + Ok(LoadedEnvironment { + source, + environment: EnvironmentDefinition { + name, + namespace, + cluster_ref: raw.spec.cluster_ref.name, + root, + values: raw.spec.values, + deploys, + }, + }) +} + +fn parse_document( + value: Value, + source: &Path, + number: usize, +) -> Result> { + serde_yaml::from_value(value).map_err(|error| { + format!( + "{} document {number}: definition schema error: {error}", + source.display() + ) + .into() + }) +} + +fn validate_overrides( + definition: &LoadedDefinition, + overrides: UpOverrides<'_>, +) -> Result<(), Box> { + if overrides.legacy_backend.is_some() + && (overrides.cluster_provider.is_some() || overrides.docker_provider.is_some()) + { + return Err( + "deprecated --backend cannot be combined with --cluster-provider or --docker-provider" + .into(), + ); + } + + if let Some(legacy) = overrides.legacy_backend { + let mapped = backend::providers::provider_pair_for_legacy_backend(legacy); + log::warn!( + "--backend is deprecated; use --cluster-provider {} --docker-provider {}", + mapped.cluster, + mapped.docker + ); + if mapped.cluster != definition.cluster.cluster_provider + || mapped.docker != definition.cluster.docker_provider + { + return Err(format!( + "deprecated --backend {legacy} maps to {}/{} but Cluster {:?} declares {}/{}; update the definition or remove the conflicting flag", + mapped.cluster, + mapped.docker, + definition.cluster.name, + definition.cluster.cluster_provider, + definition.cluster.docker_provider + ) + .into()); + } + } + if let Some(cluster_provider) = overrides.cluster_provider { + if cluster_provider != definition.cluster.cluster_provider { + return Err(format!( + "--cluster-provider {cluster_provider} conflicts with Cluster.spec.clusterProvider {}", + definition.cluster.cluster_provider + ) + .into()); + } + } + if let Some(docker_provider) = overrides.docker_provider { + if docker_provider != definition.cluster.docker_provider { + return Err(format!( + "--docker-provider {docker_provider} conflicts with Cluster.spec.dockerProvider {}", + definition.cluster.docker_provider + ) + .into()); + } + } + if let Some(cluster_name) = overrides + .cluster_name + .map(str::trim) + .filter(|name| !name.is_empty()) + { + if cluster_name != definition.cluster.name { + return Err(format!( + "--cluster-name {cluster_name:?} conflicts with Cluster.metadata.name {:?}", + definition.cluster.name + ) + .into()); + } + } + if let Some(context) = overrides + .context + .map(str::trim) + .filter(|value| !value.is_empty()) + { + let expected = expected_context(definition, overrides); + if context != expected { + return Err(format!( + "--context {context:?} conflicts with Cluster {:?}; expected {expected:?}", + definition.cluster.name + ) + .into()); + } + } + if overrides + .dory_name + .is_some_and(|name| name.trim().is_empty()) + { + return Err("--dory-name must not be empty".into()); + } + Ok(()) +} + +fn expected_context(definition: &LoadedDefinition, overrides: UpOverrides<'_>) -> String { + match definition.cluster.cluster_provider { + ClusterProvider::Kind => format!("kind-{}", definition.cluster.name), + ClusterProvider::Colima => "colima".to_string(), + ClusterProvider::Dory => overrides + .dory_name + .map(str::trim) + .filter(|name| !name.is_empty()) + .map(ToOwned::to_owned) + .unwrap_or_else(|| Backend::Dory.kube_context()), + } +} + +fn validate_dns_label(field: &str, value: &str) -> Result<(), Box> { + let valid = !value.is_empty() + && value.len() <= 63 + && value + .bytes() + .all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'-') + && value + .as_bytes() + .first() + .is_some_and(u8::is_ascii_alphanumeric) + && value + .as_bytes() + .last() + .is_some_and(u8::is_ascii_alphanumeric); + if valid { + Ok(()) + } else { + Err( + format!("{field} must be a lowercase DNS-1123 label (1-63 characters), got {value:?}") + .into(), + ) + } +} + +fn resolve_bounded_path( + boundary: &Path, + base: &Path, + relative: &Path, + field: &str, + require_directory: bool, +) -> Result> { + if relative.is_absolute() { + return Err(format!("{field} must be relative, got {}", relative.display()).into()); + } + + let mut components = Vec::::new(); + for component in relative.components() { + match component { + Component::CurDir => {} + Component::Normal(value) => components.push(value.to_os_string()), + Component::ParentDir | Component::RootDir | Component::Prefix(_) => { + return Err(format!( + "{field} contains forbidden traversal or root component: {}", + relative.display() + ) + .into()) + } + } + } + + let boundary = boundary.canonicalize().map_err(|error| { + format!( + "unable to canonicalize {field} boundary {}: {error}", + boundary.display() + ) + })?; + let mut current = base.canonicalize().map_err(|error| { + format!( + "unable to canonicalize {field} base {}: {error}", + base.display() + ) + })?; + ensure_within(&boundary, ¤t, field)?; + + let mut index = 0; + while index < components.len() { + let next = current.join(&components[index]); + match fs::symlink_metadata(&next) { + Ok(_) => { + current = next.canonicalize().map_err(|error| { + format!("unable to canonicalize {field} {}: {error}", next.display()) + })?; + ensure_within(&boundary, ¤t, field)?; + index += 1; + } + Err(error) if error.kind() == io::ErrorKind::NotFound => { + current = next; + index += 1; + while index < components.len() { + current.push(&components[index]); + index += 1; + } + } + Err(error) => { + return Err(format!("unable to inspect {field} {}: {error}", next.display()).into()) + } + } + } + ensure_within(&boundary, ¤t, field)?; + + if require_directory && !current.is_dir() { + return Err(format!("{field} directory does not exist: {}", current.display()).into()); + } + Ok(current) +} + +fn resolve_mount_root( + definition_root: &Path, + relative: &Path, + field: &str, +) -> Result> { + if relative.is_absolute() { + return Err(format!("{field} must be relative, got {}", relative.display()).into()); + } + + let candidate = definition_root.join(relative); + let resolved = candidate.canonicalize().map_err(|error| { + format!( + "unable to canonicalize {field} {}: {error}", + candidate.display() + ) + })?; + if !resolved.is_dir() { + return Err(format!("{field} directory does not exist: {}", resolved.display()).into()); + } + Ok(resolved) +} + +fn ensure_within(boundary: &Path, candidate: &Path, field: &str) -> Result<(), Box> { + if candidate == boundary || candidate.starts_with(boundary) { + Ok(()) + } else { + Err(format!( + "{field} escapes Cluster.spec.mountRoot: {} is outside {}", + candidate.display(), + boundary.display() + ) + .into()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + struct Fixture { + root: PathBuf, + } + + impl Fixture { + fn new() -> Self { + let root = std::env::temp_dir().join(format!( + "hops-cluster-definition-{}-{}", + std::process::id(), + uuid::Uuid::new_v4() + )); + fs::create_dir_all(root.join(CLUSTER_MANIFESTS_PATH)).unwrap(); + fs::create_dir_all(root.join("apps/gateway")).unwrap(); + fs::create_dir_all(root.join("services/api")).unwrap(); + let root = root.canonicalize().unwrap(); + Self { root } + } + + fn write(&self, yaml: &str) -> PathBuf { + let path = self.root.join(DEFAULT_DEFINITION_FILE); + fs::write(&path, yaml).unwrap(); + path + } + + fn write_environment(&self, yaml: &str) -> PathBuf { + let path = self.root.join(DEFAULT_ENVIRONMENT_FILE); + fs::write(&path, yaml).unwrap(); + path + } + } + + impl Drop for Fixture { + fn drop(&mut self) { + let _ = fs::remove_dir_all(&self.root); + } + } + + fn valid_yaml() -> &'static str { + r#"apiVersion: hops.local/v1alpha1 +kind: Cluster +metadata: + name: project-dev +spec: + clusterProvider: kind + dockerProvider: dory + mountRoot: ../.. + manifests: + path: .gitops/local/cluster +"# + } + + fn valid_environment_yaml() -> &'static str { + r#"apiVersion: hops.local/v1alpha1 +kind: Environment +metadata: + name: local +spec: + clusterRef: + name: project-dev + root: . + values: + local: true + deploys: + - path: apps/gateway + values: + preview: false + - path: services/api +"# + } + + #[test] + fn parses_cluster_only_and_reusable_environment() { + let fixture = Fixture::new(); + let loaded = load_definition(&fixture.write(valid_yaml())).unwrap(); + assert_eq!(loaded.cluster.name, "project-dev"); + assert_eq!(loaded.cluster.cluster_provider, ClusterProvider::Kind); + assert_eq!(loaded.cluster.docker_provider, DockerProvider::Dory); + + let environment = load_environment_definition( + &fixture.write_environment(valid_environment_yaml()), + &loaded, + Some("feature-auth"), + None, + ) + .unwrap(); + assert_eq!(environment.environment.name, "feature-auth"); + assert_eq!(environment.environment.namespace, "feature-auth"); + assert_eq!(environment.environment.root, fixture.root); + assert_eq!( + environment.environment.deploys[0].promote_chart, + fixture.root.join("apps/gateway/.gitops/promote") + ); + } + + #[test] + fn default_definition_prefers_local_layout_then_legacy_root() { + let fixture = Fixture::new(); + let preferred = fixture.write(valid_yaml()); + let legacy = fixture.root.join(LEGACY_DEFINITION_FILE); + fs::write(&legacy, valid_yaml()).unwrap(); + + assert_eq!(definition_path(None, &fixture.root), preferred); + + fs::remove_file(&preferred).unwrap(); + assert_eq!(definition_path(None, &fixture.root), legacy); + } + + #[test] + fn rejects_embedded_environment() { + let fixture = Fixture::new(); + let yaml = format!("{}\n---\n{}", valid_yaml(), valid_environment_yaml()); + let error = load_definition(&fixture.write(&yaml)).unwrap_err(); + assert!(error.to_string().contains("must not be committed")); + } + + #[test] + fn copied_worktree_definition_mounts_its_checkout_root() { + let fixture = Fixture::new(); + let worktree = fixture.root.join(".worktrees/feature-auth"); + fs::create_dir_all(worktree.join(CLUSTER_MANIFESTS_PATH)).unwrap(); + let source = worktree.join(DEFAULT_DEFINITION_FILE); + fs::write(&source, valid_yaml()).unwrap(); + + let loaded = load_definition(&source).unwrap(); + + assert_eq!(loaded.cluster.mount_root, worktree); + assert_eq!( + loaded.cluster.manifests_path, + worktree.join(CLUSTER_MANIFESTS_PATH) + ); + } + + #[test] + fn rejects_zero_or_multiple_cluster_documents() { + let fixture = Fixture::new(); + let error = load_definition(&fixture.write("\n")).unwrap_err(); + assert!(error.to_string().contains("exactly one")); + + let duplicate = format!("{}\n---\n{}\n", valid_yaml(), valid_yaml()); + let error = load_definition(&fixture.write(&duplicate)).unwrap_err(); + assert!(error.to_string().contains("found 2")); + } + + #[test] + fn rejects_unknown_fields_versions_kinds_and_cluster_refs() { + let fixture = Fixture::new(); + let unknown = valid_yaml().replacen( + " mountRoot: ../..", + " mountRoot: ../..\n unexpectedField: true", + 1, + ); + assert!(load_definition(&fixture.write(&unknown)) + .unwrap_err() + .to_string() + .contains("unknown field")); + + let version = valid_yaml().replacen(API_VERSION, "hops.local/v9", 1); + assert!(load_definition(&fixture.write(&version)) + .unwrap_err() + .to_string() + .contains("unsupported apiVersion")); + + let kind = valid_yaml().replacen("kind: Cluster", "kind: Namespace", 1); + assert!(load_definition(&fixture.write(&kind)) + .unwrap_err() + .to_string() + .contains("unsupported kind")); + + let loaded = load_definition(&fixture.write(valid_yaml())).unwrap(); + let reference = valid_environment_yaml().replacen( + "name: project-dev\n root", + "name: other\n root", + 1, + ); + assert!(load_environment_definition( + &fixture.write_environment(&reference), + &loaded, + None, + None, + ) + .unwrap_err() + .to_string() + .contains("references Cluster")); + } + + #[test] + fn rejects_duplicate_deploy_identity() { + let fixture = Fixture::new(); + let loaded = load_definition(&fixture.write(valid_yaml())).unwrap(); + let duplicate = valid_environment_yaml() + .replace(" - path: services/api", " - path: apps/gateway"); + assert!(load_environment_definition( + &fixture.write_environment(&duplicate), + &loaded, + None, + None, + ) + .unwrap_err() + .to_string() + .contains("duplicate deploy")); + } + + #[test] + fn rejects_non_mapping_values_and_invalid_names() { + let fixture = Fixture::new(); + let loaded = load_definition(&fixture.write(valid_yaml())).unwrap(); + let scalar_values = + valid_environment_yaml().replacen(" values:\n local: true", " values: true", 1); + assert!(load_environment_definition( + &fixture.write_environment(&scalar_values), + &loaded, + None, + None, + ) + .unwrap_err() + .to_string() + .contains("schema error")); + + let invalid_name = valid_yaml().replacen("name: project-dev", "name: Project_Dev", 1); + assert!(load_definition(&fixture.write(&invalid_name)) + .unwrap_err() + .to_string() + .contains("DNS-1123")); + } + + #[test] + fn requires_explicit_hidden_cluster_manifest_path() { + let fixture = Fixture::new(); + for path in [ + "gitops/cluster", + ".gitops/deploy", + "./.gitops/local/cluster", + ] { + let yaml = valid_yaml().replacen(CLUSTER_MANIFESTS_PATH, path, 1); + let error = load_definition(&fixture.write(&yaml)).unwrap_err(); + assert!(error.to_string().contains("must be"), "{error}"); + } + } + + #[test] + fn accepts_legacy_root_definition_and_manifest_layout() { + let fixture = Fixture::new(); + fs::create_dir_all(fixture.root.join(LEGACY_CLUSTER_MANIFESTS_PATH)).unwrap(); + let legacy = valid_yaml() + .replacen("mountRoot: ../..", "mountRoot: .", 1) + .replacen(CLUSTER_MANIFESTS_PATH, LEGACY_CLUSTER_MANIFESTS_PATH, 1); + let source = fixture.root.join(LEGACY_DEFINITION_FILE); + fs::write(&source, legacy).unwrap(); + + let loaded = load_definition(&source).unwrap(); + + assert_eq!(loaded.cluster.mount_root, fixture.root); + assert_eq!( + loaded.cluster.manifests_path, + fixture.root.join(LEGACY_CLUSTER_MANIFESTS_PATH) + ); + } + + #[test] + fn rejects_absolute_traversal_and_symlink_escape() { + let fixture = Fixture::new(); + let absolute = valid_yaml().replacen("mountRoot: ../..", "mountRoot: /tmp", 1); + assert!(load_definition(&fixture.write(&absolute)) + .unwrap_err() + .to_string() + .contains("must be relative")); + + let loaded = load_definition(&fixture.write(valid_yaml())).unwrap(); + let traversal = valid_environment_yaml().replacen("root: .", "root: ../outside", 1); + assert!(load_environment_definition( + &fixture.write_environment(&traversal), + &loaded, + None, + None, + ) + .unwrap_err() + .to_string() + .contains("forbidden traversal")); + + #[cfg(unix)] + { + use std::os::unix::fs::symlink; + let outside = fixture + .root + .parent() + .unwrap() + .join(format!("outside-definition-{}", uuid::Uuid::new_v4())); + fs::create_dir_all(&outside).unwrap(); + symlink(&outside, fixture.root.join("escape")).unwrap(); + let escaped = valid_environment_yaml().replacen("root: .", "root: escape", 1); + let error = load_environment_definition( + &fixture.write_environment(&escaped), + &loaded, + None, + None, + ) + .unwrap_err(); + assert!(error.to_string().contains("escapes Cluster.spec.mountRoot")); + + symlink(&outside, fixture.root.join("secret-link")).unwrap(); + let escaped_secret = valid_yaml().replacen( + " manifests:\n path: .gitops/local/cluster", + " manifests:\n path: .gitops/local/cluster\n secretSync:\n path: secret-link", + 1, + ); + let error = load_definition(&fixture.write(&escaped_secret)).unwrap_err(); + assert!(error.to_string().contains("escapes Cluster.spec.mountRoot")); + fs::remove_dir_all(outside).unwrap(); + } + } + + #[test] + fn validates_cli_identity_without_mutation() { + let fixture = Fixture::new(); + let loaded = load_definition(&fixture.write(valid_yaml())).unwrap(); + validate_overrides( + &loaded, + UpOverrides { + cluster_provider: Some(ClusterProvider::Kind), + docker_provider: Some(DockerProvider::Dory), + cluster_name: Some("project-dev"), + context: Some("kind-project-dev"), + ..UpOverrides::default() + }, + ) + .unwrap(); + + let error = validate_overrides( + &loaded, + UpOverrides { + docker_provider: Some(DockerProvider::Colima), + ..UpOverrides::default() + }, + ) + .unwrap_err(); + assert!(error.to_string().contains("conflicts")); + } +} diff --git a/src/commands/local/workbench/delivery.rs b/src/commands/local/workbench/delivery.rs index cbdaea6..6813db7 100644 --- a/src/commands/local/workbench/delivery.rs +++ b/src/commands/local/workbench/delivery.rs @@ -756,7 +756,7 @@ pub fn attach_sync_delivery( result .messages .push( - "no Running pods to sync into yet; re-run `hops local gitops worktree` after pods are Ready" + "no Running pods to sync into yet; re-run `hops local gitops environment` after pods are Ready" .into(), ); return Ok(result); @@ -981,7 +981,7 @@ fn spawn_tar_sync_watcher(targets: Vec) -> Result String { format!("{}/{}", self.namespace, self.name) } + + pub fn endpoint_key(&self) -> String { + format!("{}:{}", self.key(), self.port) + } } #[derive(Debug, Clone, PartialEq, Eq, Default)] @@ -50,7 +63,7 @@ pub struct HostAccessPlan { pub urls: BTreeMap, /// service key `ns/name` → loopback IP pub ip_map: BTreeMap, - /// service key → port + /// endpoint key `ns/name:port` → port pub service_ports: BTreeMap, } @@ -61,7 +74,8 @@ pub struct HostAccessRuntime { pub pids: Vec, #[serde(default)] pub log_path: Option, - /// service key `ns/name` → port (preferred). Also accepts bare name for older files. + /// endpoint key `ns/name:port` → port. Older single-port runtime files + /// using `ns/name` or a bare service name remain accepted. #[serde(default)] pub service_ports: BTreeMap, /// service key `ns/name` → loopback IP @@ -85,7 +99,7 @@ pub fn plan_host_access_with_ips( let mut urls = BTreeMap::new(); let mut service_ports = BTreeMap::new(); for svc in services { - let key = svc.key(); + let key = svc.endpoint_key(); urls.insert( key.clone(), format_dns_url(&svc.name, &svc.namespace, svc.port), @@ -132,19 +146,26 @@ pub fn format_status_card_with_listen( pub fn host_access_status_line(rt: &HostAccessRuntime) -> String { let alive = rt.pids.iter().filter(|p| pid_is_alive(**p)).count(); let listen = rt - .ip_map + .service_ports .iter() - .filter(|(key, ip)| { - let port = rt.service_ports.get(*key).copied().unwrap_or(80); - ip_port_listening(ip, port) + .filter(|(endpoint_key, port)| { + let service_key = service_key_from_endpoint_key(endpoint_key); + rt.ip_map + .get(service_key) + .is_some_and(|ip| ip_port_listening(ip, **port)) }) .count(); + let endpoint_count = if rt.service_ports.is_empty() { + rt.ip_map.len() + } else { + rt.service_ports.len() + }; format!( "access: dns supervisor {}/{} alive; {}/{} endpoints listening", alive, rt.pids.len().max(1), listen, - rt.ip_map.len() + endpoint_count ) } @@ -182,11 +203,41 @@ pub fn clear_host_access_runtime(state_dir: &Path, workspace: &str) { let _ = fs::remove_file(runtime_path(state_dir, workspace)); } -/// Services in `namespace` (first TCP port each). +/// Services in `namespace` (all TCP ports). pub fn discover_services(namespace: &str) -> Result, Box> { discover_services_in_namespace(namespace) } +fn service_endpoints_from_value(namespace: &str, item: &serde_json::Value) -> Vec { + let name = item["metadata"]["name"].as_str().unwrap_or(""); + if name.is_empty() || name == "kubernetes" { + return Vec::new(); + } + item["spec"]["ports"] + .as_array() + .cloned() + .unwrap_or_default() + .into_iter() + .filter_map(|p| { + let port = p["port"].as_u64().unwrap_or(0) as u16; + let protocol = p["protocol"].as_str().unwrap_or("TCP"); + if port == 0 + || (protocol != "TCP" && protocol != "tcp") + // Skip postgres-ish ports for host browser access. + || port == 5432 + { + return None; + } + Some(ServiceEndpoint { + namespace: namespace.to_string(), + name: name.to_string(), + port, + protocol: "TCP".into(), + }) + }) + .collect() +} + fn discover_services_in_namespace(namespace: &str) -> Result, Box> { let output = kubectl_command(&["get", "svc", "-n", namespace, "-o", "json"]) .output() @@ -201,34 +252,9 @@ fn discover_services_in_namespace(namespace: &str) -> Result Result, Box> { let mut by_key: BTreeMap = BTreeMap::new(); for svc in discover_services_in_namespace(namespace)? { - by_key.insert(svc.key(), svc); + by_key.insert(svc.endpoint_key(), svc); } for (ref_ns, ref_name, port_hint) in scan_pod_cluster_dns_refs(namespace)? { - if ref_ns == namespace && by_key.contains_key(&format!("{ref_ns}/{ref_name}")) { - continue; - } - let key = format!("{ref_ns}/{ref_name}"); - if by_key.contains_key(&key) { + let service_key = format!("{ref_ns}/{ref_name}"); + if ref_ns == namespace && by_key.values().any(|svc| svc.key() == service_key) { continue; } - // Resolve live service port when possible. - let port = match service_port(&ref_ns, &ref_name) { - Ok(p) => p, - Err(_) => port_hint.unwrap_or(80), - }; - if port == 5432 { - continue; + // A referenced Service is one DNS endpoint with potentially many + // useful ports. Expose every live TCP port on the same loopback IP. + let discovered = discover_named_service(&ref_ns, &ref_name).unwrap_or_default(); + if !discovered.is_empty() { + for svc in discovered { + by_key.insert(svc.endpoint_key(), svc); + } + } else { + let port = port_hint.unwrap_or(80); + if port != 5432 { + let svc = ServiceEndpoint { + namespace: ref_ns, + name: ref_name, + port, + protocol: "TCP".into(), + }; + by_key.insert(svc.endpoint_key(), svc); + } } - by_key.insert( - key, - ServiceEndpoint { - namespace: ref_ns, - name: ref_name, - port, - protocol: "TCP".into(), - }, - ); } Ok(by_key.into_values().collect()) } -fn service_port(namespace: &str, name: &str) -> Result> { - let output = kubectl_command(&[ - "get", - "svc", - name, - "-n", - namespace, - "-o", - "jsonpath={.spec.ports[0].port}", - ]) - .output() - .map_err(|e| format!("kubectl get svc {name}: {e}"))?; +fn discover_named_service( + namespace: &str, + name: &str, +) -> Result, Box> { + let output = kubectl_command(&["get", "svc", name, "-n", namespace, "-o", "json"]) + .output() + .map_err(|e| format!("kubectl get svc {name}: {e}"))?; if !output.status.success() { return Err("service not found".into()); } - let s = String::from_utf8_lossy(&output.stdout); - s.trim() - .parse() - .map_err(|_| format!("bad port for {namespace}/{name}").into()) + let item: serde_json::Value = serde_json::from_slice(&output.stdout)?; + Ok(service_endpoints_from_value(namespace, &item)) } /// Parse pod env for `http(s)://svc.ns.svc.cluster.local:port` references. @@ -613,11 +631,14 @@ if __name__ == '__main__': .append(true) .open(&log_path)?; let log_err = log_out.try_clone()?; - let child = Command::new("python3") + let mut command = Command::new("python3"); + command .arg(&script) .stdin(Stdio::null()) .stdout(Stdio::from(log_out)) - .stderr(Stdio::from(log_err)) + .stderr(Stdio::from(log_err)); + detach_process_group(&mut command); + let child = command .spawn() .map_err(|e| format!("failed to spawn macOS stub DNS: {e}"))?; let pid = child.id(); @@ -677,19 +698,26 @@ fn start_dns_supervisor( let config_path = log_dir.join(format!("{workspace}.dns-forwards.tsv")); let script_path = log_dir.join(format!("{workspace}.dns-sup.sh")); let piddir = log_dir.join(format!("{workspace}.dns-pf-pids")); + fs::create_dir_all(&piddir)?; + for entry in fs::read_dir(&piddir)?.flatten() { + if entry.path().extension().and_then(|ext| ext.to_str()) == Some("pid") { + fs::remove_file(entry.path())?; + } + } // TSV: NS \t SVC \t IP \t PORT \t KEY let mut tsv = String::new(); for svc in services { - let key = svc.key(); + let service_key = svc.key(); + let endpoint_key = svc.endpoint_key(); let ip = plan .ip_map - .get(&key) + .get(&service_key) .cloned() .unwrap_or_else(|| "127.0.0.1".into()); tsv.push_str(&format!( "{}\t{}\t{}\t{}\t{}\n", - svc.namespace, svc.name, ip, svc.port, key + svc.namespace, svc.name, ip, svc.port, endpoint_key )); } fs::write(&config_path, &tsv)?; @@ -700,7 +728,9 @@ fn start_dns_supervisor( set -u CONFIG='{config}' LOG='{log}' -PIDDIR='{piddir}' +PIDROOT='{piddir}' +OWNER="$PIDROOT/supervisor.pid" +PIDDIR="$PIDROOT/$$" export KUBECONFIG="${{KUBECONFIG:-}}" KCTX="${{HOPS_KUBE_CONTEXT:-}}" k() {{ @@ -710,6 +740,9 @@ k() {{ command kubectl "$@" fi }} +port_listening() {{ + (exec 3<>"/dev/tcp/$1/$2") >/dev/null 2>&1 +}} mkdir -p "$PIDDIR" echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) supervisor start" >>"$LOG" cleanup() {{ @@ -717,10 +750,34 @@ cleanup() {{ [ -f "$f" ] || continue kill "$(cat "$f")" 2>/dev/null || true done + rm -rf "$PIDDIR" + if [ "$(cat "$OWNER" 2>/dev/null || true)" = "$$" ]; then + rm -f "$OWNER" + fi exit 0 }} trap cleanup TERM INT HUP -while true; do +owned=false +for _ in $(seq 1 40); do + if [ "$(cat "$OWNER" 2>/dev/null || true)" = "$$" ]; then + owned=true + break + fi + sleep 0.05 +done +if [ "$owned" != true ]; then + cleanup +fi +for stale_dir in "$PIDROOT"/*; do + [ -d "$stale_dir" ] || continue + [ "$stale_dir" = "$PIDDIR" ] && continue + for f in "$stale_dir"/*.pid; do + [ -f "$f" ] || continue + kill "$(cat "$f")" 2>/dev/null || true + done + rm -rf "$stale_dir" +done +while [ "$(cat "$OWNER" 2>/dev/null || true)" = "$$" ]; do while IFS=$'\t' read -r NS SVC IP PORT KEY; do [ -z "${{NS:-}}" ] && continue safe=$(echo "$KEY" | tr '/:' '__') @@ -737,6 +794,12 @@ while true; do if [ -n "$pid" ]; then kill "$pid" 2>/dev/null || true wait "$pid" 2>/dev/null || true + fi + if port_listening "$IP" "$PORT"; then + rm -f "$pf" + continue + fi + if [ -n "$pid" ]; then echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) restart $KEY" >>"$LOG" else echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) start $KEY $IP:$PORT" >>"$LOG" @@ -746,6 +809,7 @@ while true; do done < "$CONFIG" sleep 2 done +cleanup "#, config = q(&config_path.to_string_lossy()), log = q(&log_path.to_string_lossy()), @@ -770,13 +834,45 @@ done if let Ok(ctx) = std::env::var(HOPS_KUBE_CONTEXT_ENV) { cmd.env(HOPS_KUBE_CONTEXT_ENV, ctx); } + detach_process_group(&mut cmd); let child = cmd .spawn() .map_err(|e| format!("failed to spawn dns supervisor: {e}"))?; - let pid = child.id(); + let spawned_pid = child.id(); + let owner_path = piddir.join("supervisor.pid"); + if let Err(error) = fs::write(&owner_path, spawned_pid.to_string()) { + let _ = Command::new("kill") + .args(["-TERM", &spawned_pid.to_string()]) + .status(); + return Err(format!("failed to claim dns supervisor ownership: {error}").into()); + } std::mem::forget(child); + std::thread::sleep(Duration::from_millis(150)); + let pid = fs::read_to_string(&owner_path) + .ok() + .and_then(|value| value.trim().parse::().ok()) + .ok_or("dns supervisor ownership disappeared during startup")?; + if pid != spawned_pid { + let _ = Command::new("kill") + .args(["-TERM", &spawned_pid.to_string()]) + .status(); + for _ in 0..20 { + if !pid_is_alive(spawned_pid) { + break; + } + std::thread::sleep(Duration::from_millis(50)); + } + if pid_is_alive(spawned_pid) { + let _ = Command::new("kill") + .args(["-KILL", &spawned_pid.to_string()]) + .status(); + } + } - let service_ports: BTreeMap = services.iter().map(|s| (s.key(), s.port)).collect(); + let service_ports: BTreeMap = services + .iter() + .map(|s| (s.endpoint_key(), s.port)) + .collect(); let runtime = HostAccessRuntime { namespace: plan.namespace.clone(), pids: vec![pid], @@ -835,15 +931,34 @@ pub fn host_access_needs_heal(rt: &HostAccessRuntime) -> bool { if !rt.pids.iter().any(|p| pid_is_alive(*p)) { return true; } - for (key, ip) in &rt.ip_map { - let port = rt.service_ports.get(key).copied().unwrap_or(80); - if !ip_port_listening(ip, port) { + for (endpoint_key, port) in &rt.service_ports { + let service_key = service_key_from_endpoint_key(endpoint_key); + let Some(ip) = rt.ip_map.get(service_key) else { + return true; + }; + if !ip_port_listening(ip, *port) { return true; } } false } +fn host_access_runtime_matches_services( + rt: &HostAccessRuntime, + services: &[ServiceEndpoint], +) -> bool { + let expected: BTreeMap = services + .iter() + .map(|service| (service.endpoint_key(), service.port)) + .collect(); + let expected_services: BTreeSet = services.iter().map(ServiceEndpoint::key).collect(); + rt.service_ports == expected + && rt.ip_map.len() == expected_services.len() + && expected_services + .iter() + .all(|key| rt.ip_map.contains_key(key)) +} + pub fn ensure_host_access( namespace: &str, services: &[ServiceEndpoint], @@ -852,10 +967,10 @@ pub fn ensure_host_access( ) -> Result<(HostAccessPlan, HostAccessRuntime, bool), Box> { let prior = load_host_access_runtime(state_dir, workspace)?; if let Some(rt) = &prior { - if !host_access_needs_heal(rt) { + if !host_access_needs_heal(rt) && host_access_runtime_matches_services(rt, services) { return Ok((plan_from_runtime(rt), rt.clone(), false)); } - log::info!("host access unhealthy; restarting dns supervisor"); + log::info!("host access unhealthy or endpoints changed; restarting dns supervisor"); } let services = if services.is_empty() { prior @@ -866,15 +981,21 @@ pub fn ensure_host_access( services.to_vec() }; let (plan, rt) = start_host_access(namespace, &services, state_dir, workspace)?; - std::thread::sleep(Duration::from_millis(400)); + for _ in 0..30 { + if !host_access_needs_heal(&rt) { + break; + } + std::thread::sleep(Duration::from_millis(100)); + } Ok((plan, rt, true)) } fn plan_from_runtime(rt: &HostAccessRuntime) -> HostAccessPlan { let mut urls = BTreeMap::new(); - for (key, port) in &rt.service_ports { - let (ns, name) = split_service_key(key, &rt.namespace); - urls.insert(key.clone(), format_dns_url(&name, &ns, *port)); + for (endpoint_key, port) in &rt.service_ports { + let service_key = service_key_from_endpoint_key(endpoint_key); + let (ns, name) = split_service_key(service_key, &rt.namespace); + urls.insert(endpoint_key.clone(), format_dns_url(&name, &ns, *port)); } if urls.is_empty() { for key in rt.ip_map.keys() { @@ -898,19 +1019,38 @@ fn split_service_key(key: &str, default_ns: &str) -> (String, String) { } } +fn service_key_from_endpoint_key(key: &str) -> &str { + match key.rsplit_once(':') { + Some((service_key, port)) if port.parse::().is_ok() => service_key, + _ => key, + } +} + fn services_from_runtime(rt: &HostAccessRuntime) -> Vec { - let keys: Vec = if !rt.service_ports.is_empty() { - rt.service_ports.keys().cloned().collect() - } else { - rt.ip_map.keys().cloned().collect() - }; - keys.into_iter() + if !rt.service_ports.is_empty() { + return rt + .service_ports + .iter() + .map(|(endpoint_key, port)| { + let service_key = service_key_from_endpoint_key(endpoint_key); + let (ns, name) = split_service_key(service_key, &rt.namespace); + ServiceEndpoint { + namespace: ns, + name, + port: *port, + protocol: "TCP".into(), + } + }) + .collect(); + } + rt.ip_map + .keys() .map(|key| { - let (ns, name) = split_service_key(&key, &rt.namespace); + let (ns, name) = split_service_key(key, &rt.namespace); ServiceEndpoint { namespace: ns, name, - port: rt.service_ports.get(&key).copied().unwrap_or(80), + port: 80, protocol: "TCP".into(), } }) @@ -919,9 +1059,13 @@ fn services_from_runtime(rt: &HostAccessRuntime) -> Vec { pub fn url_listen_status(plan: &HostAccessPlan) -> BTreeMap { let mut out = BTreeMap::new(); - for (key, ip) in &plan.ip_map { - let port = plan.service_ports.get(key).copied().unwrap_or(80); - out.insert(key.clone(), ip_port_listening(ip, port)); + for (endpoint_key, port) in &plan.service_ports { + let service_key = service_key_from_endpoint_key(endpoint_key); + let listening = plan + .ip_map + .get(service_key) + .is_some_and(|ip| ip_port_listening(ip, *port)); + out.insert(endpoint_key.clone(), listening); } out } @@ -940,9 +1084,14 @@ fn stop_host_access_processes_only( .status(); } } - std::thread::sleep(Duration::from_millis(200)); + for _ in 0..30 { + if rt.pids.iter().all(|pid| !pid_is_alive(*pid)) { + break; + } + std::thread::sleep(Duration::from_millis(100)); + } for pid in &rt.pids { - if pid_command_contains_all(*pid, &[&supervisor_marker]) { + if pid_is_alive(*pid) && pid_command_contains_all(*pid, &[&supervisor_marker]) { let _ = Command::new("kill") .args(["-KILL", &pid.to_string()]) .status(); @@ -1105,11 +1254,110 @@ mod tests { }]; let plan = plan_host_access("dogfood", &svcs); assert_eq!( - plan.urls.get("dogfood/e2e-ui-ui").map(String::as_str), + plan.urls.get("dogfood/e2e-ui-ui:5180").map(String::as_str), Some("http://e2e-ui-ui.dogfood.svc.cluster.local:5180") ); } + #[test] + fn service_discovery_keeps_every_tcp_port() { + let service = serde_json::json!({ + "metadata": { "name": "mailpit" }, + "spec": { + "ports": [ + { "port": 1025, "protocol": "TCP" }, + { "port": 8025, "protocol": "TCP" }, + { "port": 8125, "protocol": "UDP" }, + { "port": 5432, "protocol": "TCP" } + ] + } + }); + let endpoints = service_endpoints_from_value("harmony-system", &service); + assert_eq!( + endpoints + .iter() + .map(|endpoint| endpoint.endpoint_key()) + .collect::>(), + vec!["harmony-system/mailpit:1025", "harmony-system/mailpit:8025"] + ); + } + + #[test] + fn host_access_runtime_must_match_discovered_service_keys_and_ports() { + let services = vec![ + ServiceEndpoint { + namespace: "harmony".into(), + name: "api".into(), + port: 8080, + protocol: "TCP".into(), + }, + ServiceEndpoint { + namespace: "harmony-auth".into(), + name: "zitadel".into(), + port: 8080, + protocol: "TCP".into(), + }, + ]; + let runtime = HostAccessRuntime { + namespace: "harmony".into(), + service_ports: BTreeMap::from([ + ("harmony/api:8080".into(), 8080), + ("auth/zitadel:8080".into(), 8080), + ]), + ip_map: BTreeMap::from([ + ("harmony/api".into(), "127.53.0.2".into()), + ("auth/zitadel".into(), "127.53.0.3".into()), + ]), + ..Default::default() + }; + + assert!(!host_access_runtime_matches_services(&runtime, &services)); + + let current = HostAccessRuntime { + service_ports: BTreeMap::from([ + ("harmony/api:8080".into(), 8080), + ("harmony-auth/zitadel:8080".into(), 8080), + ]), + ip_map: BTreeMap::from([ + ("harmony/api".into(), "127.53.0.2".into()), + ("harmony-auth/zitadel".into(), "127.53.0.3".into()), + ]), + ..runtime + }; + assert!(host_access_runtime_matches_services(¤t, &services)); + } + + #[test] + fn runtime_tracks_multiple_ports_on_one_service_ip() { + let services = vec![ + ServiceEndpoint { + namespace: "harmony-system".into(), + name: "mailpit".into(), + port: 1025, + protocol: "TCP".into(), + }, + ServiceEndpoint { + namespace: "harmony-system".into(), + name: "mailpit".into(), + port: 8025, + protocol: "TCP".into(), + }, + ]; + let runtime = HostAccessRuntime { + namespace: "harmony".into(), + service_ports: BTreeMap::from([ + ("harmony-system/mailpit:1025".into(), 1025), + ("harmony-system/mailpit:8025".into(), 8025), + ]), + ip_map: BTreeMap::from([("harmony-system/mailpit".into(), "127.53.0.20".into())]), + ..Default::default() + }; + + assert!(host_access_runtime_matches_services(&runtime, &services)); + let restored = services_from_runtime(&runtime); + assert_eq!(restored, services); + } + #[test] fn parse_cluster_dns_from_env_value() { let refs = regex_lite_cluster_dns( diff --git a/src/commands/local/workbench/reconcile.rs b/src/commands/local/workbench/reconcile.rs index 9c552a3..8ee506e 100644 --- a/src/commands/local/workbench/reconcile.rs +++ b/src/commands/local/workbench/reconcile.rs @@ -785,7 +785,7 @@ mod tests { let app = serde_yaml::from_str::( r#" local: true -appRuntime: cluster-dev +preview: false image: tag: app "#, @@ -793,10 +793,10 @@ image: .unwrap(); let mut runtime = BTreeMap::new(); runtime.insert("namespace".into(), Value::String("alice".into())); - runtime.insert("appRuntime".into(), Value::String("host".into())); + runtime.insert("preview".into(), Value::Bool(true)); let merged = merge_helm_values(Some(&app), &runtime); assert_eq!(merged["local"], Value::Bool(true)); - assert_eq!(merged["appRuntime"], Value::String("host".into())); + assert_eq!(merged["preview"], Value::Bool(true)); assert_eq!(merged["namespace"], Value::String("alice".into())); assert_eq!(merged["image"]["tag"], Value::String("app".into())); } diff --git a/src/commands/local/zitadel.rs b/src/commands/local/zitadel.rs index 966579f..83eda3f 100644 --- a/src/commands/local/zitadel.rs +++ b/src/commands/local/zitadel.rs @@ -75,7 +75,7 @@ pub struct ZitadelArgs { pub refresh: bool, /// Write non-secret Provider / ProviderConfig YAML under this directory - /// (e.g. `./gitops/cluster`). Credential Secrets are not written. + /// (e.g. `./.gitops/local/cluster`). Credential Secrets are not written. #[arg(long)] pub gitops: Option, } diff --git a/tests/local_cluster_definition.rs b/tests/local_cluster_definition.rs new file mode 100644 index 0000000..192f52d --- /dev/null +++ b/tests/local_cluster_definition.rs @@ -0,0 +1,340 @@ +#![cfg(unix)] + +use std::fs; +use std::os::unix::fs::PermissionsExt; +use std::path::{Path, PathBuf}; +use std::process::{Command, Output}; + +const VALID_DEFINITION: &str = r#"apiVersion: hops.local/v1alpha1 +kind: Cluster +metadata: + name: project-dev +spec: + clusterProvider: kind + dockerProvider: dory + mountRoot: ../.. + manifests: + path: .gitops/local/cluster +"#; + +const DEFINITION_PATH: &str = ".gitops/local/cluster.yaml"; + +const ENVIRONMENT_DEFINITION: &str = r#"apiVersion: hops.local/v1alpha1 +kind: Environment +metadata: + name: local +spec: + clusterRef: + name: project-dev + root: . + deploys: + - path: apps/gateway +"#; + +const FAKE_TOOL: &str = r#"#!/bin/sh +tool=${0##*/} +printf '%s %s\n' "$tool" "$*" >> "$HOPS_TEST_COMMAND_LOG" + +case "$tool" in + kind) + if test "$1" = "--version" || test "$1" = "version"; then + echo "kind v0.32.0 go1.24 darwin/arm64" + exit 0 + fi + if test "$1" = "get" && test "$2" = "clusters"; then + if test -f "$HOPS_TEST_CLUSTER_EXISTS"; then echo project-dev; fi + exit 0 + fi + if test "$1" = "create" && test "$2" = "cluster"; then + while IFS= read -r line; do + printf 'kind-config %s\n' "$line" >> "$HOPS_TEST_COMMAND_LOG" + done + touch "$HOPS_TEST_CLUSTER_EXISTS" + exit 0 + fi + exit 0 + ;; + docker) + if test "$1" = "info"; then echo "27.0.0"; exit 0; fi + if test "$1" = "ps"; then exit 0; fi + if test "$1" = "inspect"; then + case "$*" in + *'{{json .Mounts}}'*) + if test -n "$HOPS_TEST_MOUNTS"; then + printf '%s\n' "$HOPS_TEST_MOUNTS" + else + printf '[{"Source":"%s","Destination":"%s","RW":true}]\n' \ + "$HOPS_TEST_EXPECTED_MOUNT" "$HOPS_TEST_EXPECTED_MOUNT" + fi + exit 0 + ;; + *'.State.Running'*) + if test -f "$HOPS_TEST_CLUSTER_EXISTS"; then echo true; exit 0; fi + exit 1 + ;; + *) + if test -f "$HOPS_TEST_CLUSTER_EXISTS"; then echo fake-id; exit 0; fi + exit 1 + ;; + esac + fi + if test "$1" = "exec" || test "$1" = "start"; then exit 0; fi + exit 0 + ;; + kubectl) + if test "$1" = "config" && test "$2" = "get-contexts"; then + echo kind-project-dev + fi + exit 0 + ;; +esac +"#; + +struct Fixture { + root: PathBuf, + bin: PathBuf, + command_log: PathBuf, + cluster_exists: PathBuf, +} + +impl Fixture { + fn new() -> Self { + let root = std::env::temp_dir().join(format!( + "hops-local-cluster-contract-{}-{}", + std::process::id(), + uuid::Uuid::new_v4() + )); + fs::create_dir_all(root.join(".gitops/local/cluster")).unwrap(); + fs::create_dir_all(root.join("apps/gateway")).unwrap(); + fs::create_dir_all(root.join("home")).unwrap(); + let bin = root.join("fake-bin"); + fs::create_dir_all(&bin).unwrap(); + for tool in ["kind", "docker", "kubectl"] { + write_executable(&bin.join(tool), FAKE_TOOL); + } + fs::write(root.join(DEFINITION_PATH), VALID_DEFINITION).unwrap(); + let root = root.canonicalize().unwrap(); + Self { + bin: root.join("fake-bin"), + command_log: root.join("commands.log"), + cluster_exists: root.join("cluster-exists"), + root, + } + } + + fn write_definition(&self, yaml: &str) { + fs::write(self.root.join(DEFINITION_PATH), yaml).unwrap(); + } + + fn command(&self) -> Command { + let path = format!( + "{}:{}", + self.bin.display(), + std::env::var("PATH").unwrap_or_default() + ); + let mut command = Command::new(env!("CARGO_BIN_EXE_hops-cli")); + command + .current_dir(&self.root) + .args(["local", "up"]) + .env("PATH", path) + .env("HOME", self.root.join("home")) + .env("DOCKER_HOST", "unix:///contract-test.sock") + .env("HOPS_KIND_REGISTRY_HOST_PORT", "39001") + .env("HOPS_TEST_COMMAND_LOG", &self.command_log) + .env("HOPS_TEST_CLUSTER_EXISTS", &self.cluster_exists) + .env("HOPS_TEST_EXPECTED_MOUNT", &self.root) + .env_remove("HOPS_KIND_EXTRA_MOUNT"); + command + } + + fn run(&self) -> Output { + self.command().output().unwrap() + } + + fn log(&self) -> String { + fs::read_to_string(&self.command_log).unwrap_or_default() + } + + fn clear_log(&self) { + match fs::remove_file(&self.command_log) { + Ok(()) => {} + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(error) => panic!("clear command log: {error}"), + } + } + + fn assert_no_mutation(&self) { + assert!( + self.log().is_empty(), + "external command inventory was not empty" + ); + assert!(!self.cluster_exists.exists(), "cluster marker was created"); + assert!( + !self.root.join("home/.hops/local").exists(), + "local provider state was written" + ); + } +} + +impl Drop for Fixture { + fn drop(&mut self) { + let _ = fs::remove_dir_all(&self.root); + } +} + +fn write_executable(path: &Path, body: &str) { + fs::write(path, body).unwrap(); + let mut permissions = fs::metadata(path).unwrap().permissions(); + permissions.set_mode(0o755); + fs::set_permissions(path, permissions).unwrap(); +} + +fn output_text(output: &Output) -> String { + format!( + "{}{}", + String::from_utf8_lossy(&output.stdout), + String::from_utf8_lossy(&output.stderr) + ) +} + +#[test] +fn parses_cluster_only() { + let fixture = Fixture::new(); + let first = fixture.run(); + assert!(first.status.success(), "{}", output_text(&first)); + let first_log = fixture.log(); + assert!(first_log.contains("kind create cluster --name project-dev --config -")); + assert!(first_log.contains(&format!("hostPath: \"{}\"", fixture.root.display()))); + let text = output_text(&first); + assert!(text.contains("contains no Environment inventory"), "{text}"); + let provider_state = fs::read_to_string(fixture.root.join("home/.hops/local/providers.json")) + .expect("successful up persists provider identity"); + assert!(provider_state.contains(r#""clusterProvider": "kind""#)); + assert!(provider_state.contains(r#""dockerProvider": "dory""#)); + assert!(provider_state.contains(r#""clusterName": "project-dev""#)); + + fixture.clear_log(); + let second = fixture.run(); + assert!(second.status.success(), "{}", output_text(&second)); + let second_log = fixture.log(); + assert!(!second_log.contains("kind create cluster")); + assert!(!second_log.contains("kind delete cluster")); + assert!(!second_log.contains("docker start")); +} + +#[test] +fn rejects_zero_or_multiple_clusters() { + let fixture = Fixture::new(); + fixture.write_definition("\n"); + let zero = fixture.run(); + assert!(!zero.status.success()); + assert!(output_text(&zero).contains("exactly one")); + fixture.assert_no_mutation(); + + fixture.write_definition(&format!( + "{}\n---\n{}\n", + VALID_DEFINITION, VALID_DEFINITION + )); + let multiple = fixture.run(); + assert!(!multiple.status.success()); + assert!(output_text(&multiple).contains("found 2")); + fixture.assert_no_mutation(); +} + +#[test] +fn rejects_embedded_environment_before_mutation() { + let fixture = Fixture::new(); + fixture.write_definition(&format!( + "{}\n---\n{}", + VALID_DEFINITION, ENVIRONMENT_DEFINITION + )); + let output = fixture.run(); + assert!(!output.status.success()); + assert!(output_text(&output).contains("must not be committed")); + fixture.assert_no_mutation(); +} + +#[test] +fn rejects_unknown_fields_and_escaping_paths_before_mutation() { + let fixture = Fixture::new(); + let unknown = VALID_DEFINITION.replacen( + " mountRoot: ../..", + " mountRoot: ../..\n unexpectedField: true", + 1, + ); + fixture.write_definition(&unknown); + let output = fixture.run(); + assert!(!output.status.success()); + assert!(output_text(&output).contains("unknown field")); + fixture.assert_no_mutation(); + + let escape = VALID_DEFINITION.replacen("mountRoot: ../..", "mountRoot: /tmp/outside", 1); + fixture.write_definition(&escape); + let output = fixture.run(); + assert!(!output.status.success()); + assert!(output_text(&output).contains("must be relative")); + fixture.assert_no_mutation(); +} + +#[test] +fn provider_mount_matrix() { + let fixture = Fixture::new(); + let matching = fixture + .command() + .args([ + "--cluster-provider", + "kind", + "--docker-provider", + "dory", + "--cluster-name", + "project-dev", + "--context", + "kind-project-dev", + ]) + .output() + .unwrap(); + assert!(matching.status.success(), "{}", output_text(&matching)); + + fixture.clear_log(); + let conflicting = fixture + .command() + .args(["--docker-provider", "colima"]) + .output() + .unwrap(); + assert!(!conflicting.status.success()); + assert!(output_text(&conflicting).contains("conflicts")); + assert!(fixture.log().is_empty()); + + #[cfg(target_os = "macos")] + { + let legacy = fixture + .command() + .args(["--backend", "kind"]) + .output() + .unwrap(); + assert!(legacy.status.success(), "{}", output_text(&legacy)); + assert!(output_text(&legacy).contains("--backend is deprecated")); + } +} + +#[test] +fn mount_drift_is_non_destructive() { + let fixture = Fixture::new(); + fs::write(&fixture.cluster_exists, "existing").unwrap(); + let output = fixture + .command() + .env( + "HOPS_TEST_MOUNTS", + r#"[{"Source":"/different","Destination":"/different","RW":true}]"#, + ) + .output() + .unwrap(); + let text = output_text(&output); + assert!(!output.status.success()); + assert!(text.contains("different or missing mountRoot"), "{text}"); + assert!(text.contains("No resources were deleted"), "{text}"); + let log = fixture.log(); + assert!(!log.contains("kind create cluster")); + assert!(!log.contains("kind delete cluster")); + assert!(!log.contains("docker start")); +}