From 1ed9673dce4e035a1ef761d2c63ccfcaddc19650 Mon Sep 17 00:00:00 2001 From: Chris Butler Date: Sun, 30 Aug 2026 14:10:32 +0000 Subject: [PATCH 1/2] fix: resolve recurring superlinter failures across markdown, textlint, and python Super linter has failed on every push/PR for weeks (100% failure rate on the last 50 runs). None of the failures came from vendored code (charts/vendor/kyverno/* is already excluded via FILTER_REGEX_EXCLUDE and was verified to contribute zero lint errors). - markdownlint: fix real formatting issues (missing blank lines around headings/lists/fences, bad list-item numbering, a missing table cell) across issue templates, README.md, airgap/DEPLOY-RUNBOOK.md, docs/disconnected-deployment.md, and the kubevirtvm chart README. Two of these were genuine content bugs: a skipped step number in README.md and a missing empty cell in a DEPLOY-RUNBOOK.md status table. - textlint (NATURAL_LANGUAGE): fix the 14 flagged terminology instances (git/repo/unix/indexes/READMEs) across 5 files to match the rule's suggested terms. - isort: fix import ordering in scripts/git-http-server.py. - pyink: disable VALIDATE_PYTHON_PYINK in superlinter.yml. It actively conflicts with Black (which already passes and matches the project's .flake8 comment 'match black default'); super-linter itself warns against enabling both. No other validatedpatterns/* repo enables Pyink; two explicitly disable it alongside Black. Verified locally against the exact tool versions super-linter pins (markdownlint 0.44.0-equivalent ruleset, textlint default terminology config, isort 6.0.1, black 25.1.0, flake8 7.2.0, mypy 1.15.0) with zero remaining issues. --- .github/ISSUE_TEMPLATE/bug-report.md | 9 ++++++-- .github/ISSUE_TEMPLATE/documentation.md | 6 +++++- .github/ISSUE_TEMPLATE/feature-request.md | 5 +++++ .github/workflows/superlinter.yml | 4 ++++ README.md | 11 ++++++---- airgap/DEPLOY-RUNBOOK.md | 14 +++++++----- charts/all/kubevirtconfidential/README.md | 2 +- charts/all/kubevirtvm/README.md | 2 +- docs/disconnected-deployment.md | 26 ++++++++++++++++------- scripts/git-http-server.py | 4 ++-- 10 files changed, 59 insertions(+), 24 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug-report.md b/.github/ISSUE_TEMPLATE/bug-report.md index 52ae0039..ca00205b 100644 --- a/.github/ISSUE_TEMPLATE/bug-report.md +++ b/.github/ISSUE_TEMPLATE/bug-report.md @@ -7,25 +7,30 @@ assignees: '' --- ### Description + A clear and concise description of the bug. ### Steps to Reproduce -1. Install the pattern + +1. Install the pattern 2. Go to X 3. See Y - ### Expected Behavior + Explain what you expected to happen. ### Screenshots + Add screenshots if applicable. ### Environment + - Hardware / cloud provide (e.g. Azure; Intel Barel metal; AMD Bare metal) - CPU Specs and/or cloud instance types. - Cluster topology (one cluster; SNO; multi-cluster) - Connected or disconnected ### Additional Context + Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/documentation.md b/.github/ISSUE_TEMPLATE/documentation.md index 2b32dcf8..e997382e 100644 --- a/.github/ISSUE_TEMPLATE/documentation.md +++ b/.github/ISSUE_TEMPLATE/documentation.md @@ -7,13 +7,17 @@ assignees: '' --- ### Documentation Update -What part of the documentation needs to be updated or added? Is it for the validated patterns docs site or the repo docs. + +What part of the documentation needs to be updated or added? Is it for the validated patterns docs site or the repository docs. ### Why Is This Needed? + Explain the importance of this update. ### Suggested Changes + Provide a detailed description of the changes. ### Additional Context + Include any related resources. diff --git a/.github/ISSUE_TEMPLATE/feature-request.md b/.github/ISSUE_TEMPLATE/feature-request.md index 25cec83d..0e1cefd5 100644 --- a/.github/ISSUE_TEMPLATE/feature-request.md +++ b/.github/ISSUE_TEMPLATE/feature-request.md @@ -7,16 +7,21 @@ assignees: '' --- ### Feature Description + What feature would you like to see? ### Why Is This Needed? + Explain the problem or need for this feature. ### Suggested Solutions + Describe how this feature could be implemented. ### Proposed Acceptance criteria + Discrete acceptance criteria, suitable for evaluation by a LLM ### Additional Context + Add any relevant screenshots, links, or resources. diff --git a/.github/workflows/superlinter.yml b/.github/workflows/superlinter.yml index b5e16a92..458355c8 100644 --- a/.github/workflows/superlinter.yml +++ b/.github/workflows/superlinter.yml @@ -35,6 +35,10 @@ jobs: VALIDATE_MARKDOWN_PRETTIER: false VALIDATE_KUBERNETES_KUBECONFORM: false VALIDATE_PYTHON_PYLINT: false + # Pyink and Black conflict (super-linter warns against enabling both). + # This repo formats Python with Black (.flake8 targets Black's line + # length), and no other validatedpatterns/* repo enables Pyink. + VALIDATE_PYTHON_PYINK: false VALIDATE_SHELL_SHFMT: false VALIDATE_YAML: false VALIDATE_YAML_PRETTIER: false diff --git a/README.md b/README.md index f1baf96c..57cca15a 100644 --- a/README.md +++ b/README.md @@ -33,12 +33,13 @@ Breaking change from v5. Upgrades to OSC 1.13 / Trustee 1.2, adds full airgap (d - **8.0** — OSC 1.13 / Trustee 1.2 upgrade. KBS TOML rewrite, RVPS format alignment, operator CSV pins, trustee-chart v0.10.0. Airgap deployment support: single mirror-registry architecture, `oc-mirror` v2, `airgap-post-install.sh` bootstrap, `DEPLOY-RUNBOOK.md` operational guide. Requires OCP 4.22+. **Key changes from v5:** + - **Operator upgrade:** OSC 1.12 → 1.13, Trustee 1.1 → 1.2. Breaking API changes in KBS configuration and RVPS reference value format. -- **Airgap support:** Full disconnected deployment on bare metal via a single mirror-registry. Includes `oc-mirror` imageset configs, `airgap-post-install.sh` for bootstrap (CatalogSources, IDMS/ITMS, NeverContactSource policy normalisation, ArgoCD CA injection, git HTTP server), and a step-by-step operational runbook at `airgap/DEPLOY-RUNBOOK.md`. +- **Airgap support:** Full disconnected deployment on bare metal via a single mirror-registry. Includes `oc-mirror` imageset configs, `airgap-post-install.sh` for bootstrap (CatalogSources, IDMS/ITMS, NeverContactSource policy normalisation, ArgoCD CA injection, Git HTTP server), and a step-by-step operational runbook at `airgap/DEPLOY-RUNBOOK.md`. - **Chart architecture:** trustee-chart moved to OCI Helm artifact (`quay.io/validatedpatterns/trustee:0.10.0`). Kyverno chart vendored under `charts/vendor/`. External charts (sandboxed-containers, sandboxed-policies) remain OCI. - **TDX MachineConfig:** QGS socket port `socket_port=0` now deployed via MachineConfig drop-in (replaces manual `sed` workaround). - **DCAP collateral:** `collect-dcap-collateral.sh` uses `pcsclient.py fetch -p E5` with `jq` fixup for QeIdentity (Red Hat OSC 1.13 disconnected TDX procedure). -- **Experimental:** KubeVirt TDX confidential VMs (`charts/all/kubevirtconfidential/`, `charts/all/kubevirtvm/`). Disabled by default. Requires Intel TDX hardware and KubeVirt post-v1.8.4 for full QGS attestation. See chart READMEs for details. +- **Experimental:** KubeVirt TDX confidential VMs (`charts/all/kubevirtconfidential/`, `charts/all/kubevirtvm/`). Disabled by default. Requires Intel TDX hardware and KubeVirt post-v1.8.4 for full QGS attestation. See chart readmes for details. ### Previous versions @@ -53,7 +54,8 @@ Breaking change from v5. Upgrades to OSC 1.13 / Trustee 1.2, adds full airgap (d ### Airgap (disconnected) deployment For air-gapped bare metal environments, see [`airgap/DEPLOY-RUNBOOK.md`](airgap/DEPLOY-RUNBOOK.md) for the full operational procedure. The runbook covers: -- Phase 0: One-time jump host setup (mirror registry, git HTTP server, CA certificates) + +- Phase 0: One-time jump host setup (mirror registry, Git HTTP server, CA certificates) - Phase A–B: Mirror wipe and `oc-mirror` v2 re-mirror - Phase C: Agent-based OCP install - Phase D: Pattern bootstrap (`airgap-post-install.sh`) @@ -61,6 +63,7 @@ For air-gapped bare metal environments, see [`airgap/DEPLOY-RUNBOOK.md`](airgap/ - Phase F: Verification and pass/fail checklist **Prerequisites for airgap:** + - A jump host with internet access (for `oc-mirror`) and network access to the target cluster - `docker.io/library/registry:2` container running as the mirror registry (setup documented in Phase 0) - `scripts/git-http-server.py` serving pattern repos over smart HTTP (required by the patterns-operator's go-git client) @@ -127,7 +130,7 @@ These scripts generate the cryptographic material and attestation reference valu - Options: `intel-tdx`, `amd-snp`, `intel-tdx-gpu`, `amd-snp-gpu` 3. Run `bash scripts/gen-secrets.sh` to generate KBS keys and PCCS secrets 4. `./pattern.sh make install` -6. Wait for the cluster to reboot nodes (MachineConfig updates for TDX/SEV-SNP kernel parameters and vsock) +5. Wait for the cluster to reboot nodes (MachineConfig updates for TDX/SEV-SNP kernel parameters and vsock) > **Note:** Bare metal support is currently tested on SNO (Single Node OpenShift) configurations. Multi-node bare metal clusters are expected to work but have not been validated yet. diff --git a/airgap/DEPLOY-RUNBOOK.md b/airgap/DEPLOY-RUNBOOK.md index 44b30010..d15579f4 100644 --- a/airgap/DEPLOY-RUNBOOK.md +++ b/airgap/DEPLOY-RUNBOOK.md @@ -210,7 +210,7 @@ EOF ### 0-3: Local Git Server Initialisation (ONE-TIME per jump host) -The local git HTTP server makes pattern repos available to the cluster's patterns-operator +The local Git HTTP server makes pattern repos available to the cluster's patterns-operator during the deploy. `scripts/airgap-post-install.sh --sync-repos-only` creates bare repos and starts the service; subsequent runs auto-sync via the D-1 step. @@ -387,7 +387,6 @@ python3 -c "import json; [print(k) for k in json.load(open('pull-secret.json'))[ echo "A-1: State verification complete at $(date)" 2>&1 | tee -a "$LOG" ``` - ### A-3: oc-mirror Workspace Wipe > **DESTRUCTIVE — confirm A-1 shows correct git branch and mirror-registry state before proceeding.** @@ -743,7 +742,6 @@ ls -lh ~/public_html/node-02-42208.iso 2>&1 | tee -a "$LOG" && \ echo "C-1: Config generation and ISO build complete at $(date)" 2>&1 | tee -a "$LOG" ``` - ### C-3: Mount ISO via iDRAC ```bash @@ -835,6 +833,7 @@ echo "C-6: KUBECONFIG configured at $(date)" 2>&1 | tee -a "$LOG" ### D-1: Run airgap-post-install.sh > `airgap-post-install.sh` handles all of the following in one pass: +> > - Disables OperatorHub default catalogs > - Applies CatalogSources from oc-mirror cluster-resources (deleting stale ones) > - Applies ALL oc-mirror cluster-resources: IDMS, ITMS, ClusterCatalog (OLM v1), signature ConfigMap @@ -956,6 +955,7 @@ echo "D-4: pattern.sh complete at $(date)" 2>&1 | tee -a "$LOG" > If you delay, ArgoCD may come up and fail to pull VP OCI Helm charts before credentials are loaded. > > Steps D-5 performs once vp-gitops appears: +> > 1. Inject mirror-registry private CA into ArgoCD's TLS trust store > 2. `make load-bootstrap` — pre-seeds ArgoCD with mirror-registry OCI Helm credentials > (from the `bootstrap_secrets` block in values-secret.yaml; no vault required) @@ -1391,7 +1391,7 @@ echo "=== Run Complete: $(date) ===" 2>&1 | tee -a "$LOG" | Criterion | Expected | Actual | |-----------|----------|--------| | patterns-operator CSV Succeeded | Via OLM from mirror-registry community catalog | | -| Pattern CR reconciles | targetRepo = HTTP git URL (not GitHub) | | +| Pattern CR reconciles | targetRepo = HTTP Git URL (not GitHub) | | | Vault init without manual SA creation | Sync-wave fix still working | | | 15/15 (or current count) ArgoCD apps Synced+Healthy | All green | | | KBS attestation affirming | POST /attest 200 | | @@ -1402,7 +1402,7 @@ echo "=== Run Complete: $(date) ===" 2>&1 | tee -a "$LOG" | secret.txt contains KBS resource | Fetched via CDH from KBS | | | All images from mirror-registry | No upstream refs | | | MCH Running | ACM + MCE healthy | | -| QGS socket_port=0 WITHOUT manual sed | D-06 — MCO fix working (new row) | +| QGS socket_port=0 WITHOUT manual sed | D-06 — MCO fix working (new row) | | | No mirrorSourcePolicy conflicts after D-1.5 | PASS: all IDMS/ITMS have NeverContactSource (new row) | | | scripts/deprecated/ untouched — 3 files, none invoked | D-07 — no deprecated scripts called (new row) | | | 2-line global catalogSource overlay active in values-baremetal-airgap.yaml | D-07 — Phase 24 C-05 migration active (new row) | | @@ -1492,10 +1492,12 @@ SELinux note: htpasswd file requires `container_file_t` label. If recreating, ru `chcon -t container_file_t ~/mirror-registry-config/htpasswd` CA trust for container tools (podman, oc-mirror, skopeo): + ```bash mkdir -p ~/.config/containers/certs.d/${MIRROR_REGISTRY} cp ~/mirror-registry-certs/ca.crt ~/.config/containers/certs.d/${MIRROR_REGISTRY}/ca.crt ``` + This is idempotent and is run automatically in A-0. It enables `--dest-tls-verify=true` in oc-mirror and removes the need for `--tls-verify=false` in podman login. @@ -1553,12 +1555,14 @@ export MREG_PASS=$(cat ~/.coco-pattern/mirror-registry-password) ``` Auto-set during run: + ```bash export KUBECONFIG= export MIRROR_REGISTRY=MIRROR_REGISTRY_HOST:8443 ``` Persistent in `~/.envrc`: + ```bash export LABCTL_IDRAC_USER=chbutler export LABCTL_IDRAC_PASSWORD="..." diff --git a/charts/all/kubevirtconfidential/README.md b/charts/all/kubevirtconfidential/README.md index bef677be..08d8a944 100644 --- a/charts/all/kubevirtconfidential/README.md +++ b/charts/all/kubevirtconfidential/README.md @@ -44,6 +44,6 @@ When enabled, the chart creates: 1. **HyperConverged CR patch** -- enables the `WorkloadEncryptionSEV` feature gate (which also covers TDX on Intel platforms) 2. **SELinux MachineConfig** -- installs a custom SELinux policy module - (`kubevirt-qgs`) allowing QEMU processes to connect to the QGS unix socket + (`kubevirt-qgs`) allowing QEMU processes to connect to the QGS Unix socket 3. **RHEL 9 ImageStream import job** -- imports the RHEL 9 guest image into the internal registry for VM boot via DataImportCron diff --git a/charts/all/kubevirtvm/README.md b/charts/all/kubevirtvm/README.md index 682bf469..4505c9df 100644 --- a/charts/all/kubevirtvm/README.md +++ b/charts/all/kubevirtvm/README.md @@ -44,5 +44,5 @@ When enabled with `global.kubevirt.vm.enabled: "true"`: When additionally enabled with `global.kubevirt.vm.tdx.enabled: "true"`: -2. **TDX Confidential VM** -- a RHEL 9 VM with TDX launch security enabled, +1. **TDX Confidential VM** -- a RHEL 9 VM with TDX launch security enabled, validating end-to-end confidential compute inside KubeVirt diff --git a/docs/disconnected-deployment.md b/docs/disconnected-deployment.md index 4945fe08..ae9a984f 100644 --- a/docs/disconnected-deployment.md +++ b/docs/disconnected-deployment.md @@ -12,7 +12,7 @@ Deploy the CoCo validated pattern on a disconnected OpenShift cluster using mirr ## Architecture -``` +```text Internet ──► Jump Host ──► Disconnected Network │ │ quay.io (source) quay-mirror (dest) @@ -21,10 +21,11 @@ Internet ──► Jump Host ──► Disconnected Network ``` **Content flows:** + 1. `oc-mirror` mirrors OCP release + operator catalogs + container images to Quay 2. `airgap-post-install.sh` mirrors OCI Helm charts (oc-mirror can't handle these) -3. `git-http-server.py` serves pattern git repos over smart HTTP (go-git requires this) -4. Pattern operator clones from git server, ArgoCD pulls Helm charts from Quay +3. `git-http-server.py` serves pattern Git repos over smart HTTP (go-git requires this) +4. Pattern operator clones from Git server, ArgoCD pulls Helm charts from Quay ## Step-by-Step Deployment @@ -41,6 +42,7 @@ make airgap-mirror \ ``` **What gets mirrored:** + - OCP 4.21.24 release images - Red Hat operators: gitops, sandboxed-containers, trustee, cert-manager, ACM, LVM, CNV, NFD, intel-device-plugins - Certified operators: gpu-operator @@ -49,7 +51,7 @@ make airgap-mirror \ **Known limitation:** `oc-mirror --v2` cannot mirror OCI Helm chart artifacts via `additionalImages`. The post-install script handles these separately. -**Known limitation:** `oc-mirror --v2` fails on images published as OCI image indexes (`application/vnd.oci.image.index.v1+json`) with "Manifest list must be converted" error. Currently only affects `patterns-operator`. See [patterns-operator#774](https://github.com/validatedpatterns/patterns-operator/issues/774). Workaround: deploy operator manually (automated in bootstrap script). +**Known limitation:** `oc-mirror --v2` fails on images published as OCI image indices (`application/vnd.oci.image.index.v1+json`) with "Manifest list must be converted" error. Currently only affects `patterns-operator`. See [patterns-operator#774](https://github.com/validatedpatterns/patterns-operator/issues/774). Workaround: deploy operator manually (automated in bootstrap script). ### 2. Install OpenShift @@ -100,7 +102,7 @@ make airgap-post-install | Step | Action | |------|--------| -| 1 | Validate prerequisites (oc, git, KUBECONFIG, MIRROR_REGISTRY) | +| 1 | Validate prerequisites (oc, Git, KUBECONFIG, MIRROR_REGISTRY) | | 2 | Disable default CatalogSources (prevent OLM reaching internet) | | 3 | Create mirrored CatalogSources (redhat, certified, community) | | 4 | Create ITMS for tag-based image pulls (ubi-minimal, VP images) | @@ -108,7 +110,7 @@ make airgap-post-install | 6 | Fix oc-mirror manifest list failures (skopeo fallback) | | 7 | Add mirror CA cert to ArgoCD TLS config | | 8 | Enable OVN routingViaHost (opt-in, lab networks only) | -| 9 | Set up smart HTTP git server for pattern repos | +| 9 | Set up smart HTTP Git server for pattern repos | | 10 | Create patterns-operator-config ConfigMap (GitOps channel override) | ### 5. Install patterns-operator @@ -195,28 +197,35 @@ The following Quay repos **must be public** for ArgoCD to pull OCI Helm charts ( | `make airgap-mirror` | Mirror content to disconnected registry | | `make airgap-post-install` | Run full post-install bootstrap | | `make airgap-deploy-pattern` | Deploy Pattern CR directly | -| `make airgap-sync-repos` | Sync git working copies to bare HTTP repos | +| `make airgap-sync-repos` | Sync Git working copies to bare HTTP repos | ## Troubleshooting ### "manifest unknown" during OCP install + The `openshift-install` binary version must match the mirrored release. Rebuild the ISO if certificates expired (>24 hours old). ### ArgoCD apps stuck at "Unknown" sync + Check if Helm chart repos are public on Quay. Verify with: + ```bash curl -sk https://quay.example.com:443/v2/mirror/validatedpatterns/clustergroup/tags/list # Should return 200 with JSON, not 401 ``` -### "unexpected EOF" from patterns-operator git clone +### "unexpected EOF" from patterns-operator Git clone + The patterns-operator uses go-git which doesn't support Apache dumb HTTP. Use `git-http-server.py` (smart HTTP via `git-http-backend` CGI) on port 8080. ### GitOps operator subscription wrong channel + The patterns-operator hardcodes the GitOps channel. The bootstrap script creates a `patterns-operator-config` ConfigMap to override it to `latest` (the only channel available in the mirrored catalog). ### Vault ImagePullBackOff + The VP Helm chart references `registry.connect.redhat.com/hashicorp/vault:VERSION-ubi`. Add an ITMS entry for `registry.connect.redhat.com/hashicorp` and mirror the image: + ```bash oc image mirror --insecure=true \ registry.connect.redhat.com/hashicorp/vault:1.21.4-ubi \ @@ -224,6 +233,7 @@ oc image mirror --insecure=true \ ``` ### OVN routingViaHost + Only needed when the OVN default gateway can't route to the jump host subnet. This is lab-specific — in a properly routed network, pods can reach the git/registry servers without it. Set `ENABLE_ROUTINGVIAHOST=true` before running the bootstrap. ## Known Issues diff --git a/scripts/git-http-server.py b/scripts/git-http-server.py index 00d8db4e..0f952449 100755 --- a/scripts/git-http-server.py +++ b/scripts/git-http-server.py @@ -15,10 +15,10 @@ systemctl --user enable --now git-http.service """ import os +import socketserver import subprocess import sys -import socketserver -from http.server import HTTPServer, BaseHTTPRequestHandler +from http.server import BaseHTTPRequestHandler, HTTPServer from urllib.parse import urlparse GIT_PROJECT_ROOT = os.path.expanduser( From 17bc63fca4b1885912ab0d45c1aaaa939d77a70e Mon Sep 17 00:00:00 2001 From: Chris Butler Date: Sun, 30 Aug 2026 14:10:43 +0000 Subject: [PATCH 2/2] feat: require --recreate before overwriting existing RHDP cluster state Fixes #110 rhdp-cluster-define.py unconditionally wiped the install directory (and ~/.azure) on every run, before openshift-install ever got a chance to run its own built-in refusal-to-overwrite check. Re-running any of the rhdp/wrapper*.sh scripts against a directory that already had a cluster installed silently discarded that cluster's local install state and provisioned a new one on top, with no destroy step -- orphaning any still-live Azure resources from the previous run. Add a --recreate flag, threaded through wrapper.sh, wrapper-cluster-only.sh, wrapper-multicluster.sh, and rhdp-cluster-define.py's run()/cleanup(). By default, if an install directory already contains cluster state (metadata.json), the tool now exits with an error before touching anything, and prints instructions to either destroy the existing cluster first or pass --recreate. --recreate only wipes the local install directory -- it deliberately does NOT call 'openshift-install destroy cluster' (scope kept minimal per review), and prints an explicit warning to that effect each time it's used, so cloud resources from a previous install aren't destroyed or silently orphaned without the operator's awareness. Verified with unit-level scenario tests (fresh dir, existing state without --recreate, existing state with --recreate) and an end-to-end CLI run of rhdp-cluster-define.py exercising all three paths. --- rhdp/README.md | 31 ++++++++++++++ rhdp/rhdp-cluster-define.py | 83 ++++++++++++++++++++++++++++++++++-- rhdp/wrapper-cluster-only.sh | 42 +++++++++++++++--- rhdp/wrapper-multicluster.sh | 42 +++++++++++++++--- rhdp/wrapper.sh | 20 ++++++--- 5 files changed, 199 insertions(+), 19 deletions(-) diff --git a/rhdp/README.md b/rhdp/README.md index 0e0e980e..50873544 100644 --- a/rhdp/README.md +++ b/rhdp/README.md @@ -53,3 +53,34 @@ The wrapper handles: cluster provisioning, secret generation, PCR reference valu 1. `bash ./rhdp/wrapper-cluster-only.sh eastasia` 2. Provisions the cluster without installing secrets or the pattern + +## Re-running against an existing install directory + +All three wrapper scripts (and `rhdp/rhdp-cluster-define.py` directly) refuse +to touch an install directory (e.g. `openshift-install`, +`openshift-install-hub`) that already has cluster state (`metadata.json`) +from a previous run. This prevents accidentally wiping the local record of a +still-live cluster and silently replacing it with a new one. + +If you see an error about existing cluster install state, you have two +options: + +1. **Destroy the existing cluster's cloud resources yourself first**, then + re-run the wrapper normally: + + ```shell + openshift-install destroy cluster --dir=./openshift-install + ``` + +2. **Pass `--recreate`** if you've already confirmed the cloud resources are + gone (or were never fully created): + + ```shell + bash ./rhdp/wrapper.sh --recreate eastasia + ``` + + **`--recreate` does NOT call `openshift-install destroy cluster` for + you.** It only wipes the local install directory so a fresh install can + proceed. If the previous cluster's cloud resources are still live, they + will be orphaned (left running in Azure, unmanaged) — verify and clean + those up manually via the Azure portal/CLI if needed. diff --git a/rhdp/rhdp-cluster-define.py b/rhdp/rhdp-cluster-define.py index ddc9ed00..a36b0a21 100644 --- a/rhdp/rhdp-cluster-define.py +++ b/rhdp/rhdp-cluster-define.py @@ -60,11 +60,71 @@ def get_multicluster_configs() -> List[Dict]: ] -def cleanup(pattern_dir: pathlib.Path, cluster_configs: List[Dict]) -> None: - """Cleanup directories for all clusters""" +# Files openshift-install writes early in "create cluster" that indicate an +# install directory already holds state for a (possibly still-live) cluster. +STATE_MARKER_FILES = ("metadata.json",) + +def _existing_state_dirs( + pattern_dir: pathlib.Path, cluster_configs: List[Dict] +) -> List[Dict]: + """Return the cluster configs whose install directory already holds + cluster state (i.e. a previous `create cluster` was run there).""" + existing = [] + for config in cluster_configs: + install_dir = pattern_dir / config["directory"] + if install_dir.exists() and any( + (install_dir / marker).exists() for marker in STATE_MARKER_FILES + ): + existing.append(config) + return existing + + +def cleanup( + pattern_dir: pathlib.Path, + cluster_configs: List[Dict], + recreate: bool = False, +) -> None: + """Cleanup directories for all clusters. + + Refuses to touch an install directory that already holds cluster state + unless `recreate` is explicitly set. Wiping that directory destroys the + only local record `openshift-install` has of any cloud resources it + previously provisioned there, which is what silently turns a re-run into + an unintentional "recreate" of the cluster (and can orphan the old cloud + resources). This function does NOT run `openshift-install destroy + cluster` on your behalf — see the warning printed below. + """ azure_dir = pathlib.Path.home() / ".azure" + existing = _existing_state_dirs(pattern_dir, cluster_configs) + + if existing and not recreate: + rprint("[red]ERROR: Existing cluster install state detected:[/red]") + for config in existing: + rprint(f" - {config['name']}: {pattern_dir / config['directory']}") + rprint( + "\n[yellow]Refusing to overwrite without --recreate.[/yellow]\n" + "This tool does NOT run 'openshift-install destroy cluster' for you.\n" + "Before re-running with --recreate, either:\n" + " 1. Destroy the existing cluster's cloud resources yourself:\n" + " openshift-install destroy cluster --dir=\n" + " 2. Or confirm the cloud resources are already gone / were never created.\n" + "Re-running with --recreate will DELETE the local install state above\n" + "WITHOUT destroying any associated cloud resources, which can orphan them." + ) + raise typer.Exit(code=1) + + if existing: + rprint("[yellow]--recreate specified: wiping local install state for:[/yellow]") + for config in existing: + rprint(f" - {config['name']}: {pattern_dir / config['directory']}") + rprint( + "[yellow]NOTE: this does NOT call 'openshift-install destroy cluster'. " + "If cloud resources still exist from the previous install, they will " + "be orphaned. Destroy them manually first if needed.[/yellow]" + ) + for config in cluster_configs: install_dir = pattern_dir / config["directory"] if install_dir.exists() and install_dir.is_dir(): @@ -152,6 +212,18 @@ def run( prefix: Annotated[ str, typer.Option("--prefix", help="Prefix for cluster name and directory") ] = "", + recreate: Annotated[ + bool, + typer.Option( + "--recreate", + help=( + "Required if the install directory already has cluster state. " + "Wipes the local install state so a new cluster can be created. " + "Does NOT destroy cloud resources from a previous install -- " + "destroy those yourself first if they still exist." + ), + ), + ] = False, ): """ Region flag requires an azure region key which can be (authoritatively) @@ -163,6 +235,11 @@ def run( Use --prefix to add a prefix to cluster name and install directory, enabling multiple cluster deployments (e.g., --prefix cluster1 creates coco-cluster1 in openshift-install-cluster1). + + Use --recreate to allow wiping an install directory that already has + cluster state. Without it, the command refuses to touch a directory that + looks like it belongs to a previous (possibly still-live) cluster. This + does NOT run "openshift-install destroy cluster" for you. """ validate_dir() @@ -179,7 +256,7 @@ def run( else: rprint("Setting up single cluster deployment") - cleanup(pathlib.Path.cwd(), cluster_configs) + cleanup(pathlib.Path.cwd(), cluster_configs, recreate=recreate) setup_install( pathlib.Path.cwd(), region, diff --git a/rhdp/wrapper-cluster-only.sh b/rhdp/wrapper-cluster-only.sh index f5454a72..877e640e 100755 --- a/rhdp/wrapper-cluster-only.sh +++ b/rhdp/wrapper-cluster-only.sh @@ -14,13 +14,41 @@ get_python_cmd() { fi } -if [ "$#" -ne 1 ]; then - echo "Error: Exactly one argument is required." - echo "Usage: $0 {azure-region-code}" +# Parse arguments +AZUREREGION="" +RECREATE=false + +while [[ $# -gt 0 ]]; do + case $1 in + --recreate) + RECREATE=true + shift + ;; + -*) + echo "Error: Unknown option $1" + echo "Usage: $0 [--recreate] {azure-region-code}" + echo "Example: $0 eastasia" + exit 1 + ;; + *) + if [ -z "$AZUREREGION" ]; then + AZUREREGION="$1" + else + echo "Error: Too many positional arguments." + echo "Usage: $0 [--recreate] {azure-region-code}" + exit 1 + fi + shift + ;; + esac +done + +if [ -z "$AZUREREGION" ]; then + echo "Error: Azure region is required." + echo "Usage: $0 [--recreate] {azure-region-code}" echo "Example: $0 eastasia" exit 1 fi -AZUREREGION=$1 echo "---------------------" echo "Validating configuration" @@ -97,7 +125,11 @@ echo "---------------------" echo "defining cluster" echo "---------------------" PYTHON_CMD=$(get_python_cmd) -$PYTHON_CMD rhdp/rhdp-cluster-define.py ${AZUREREGION} +DEFINE_ARGS=() +if [ "$RECREATE" = true ]; then + DEFINE_ARGS+=(--recreate) +fi +$PYTHON_CMD rhdp/rhdp-cluster-define.py "${DEFINE_ARGS[@]}" ${AZUREREGION} echo "---------------------" echo "cluster defined" echo "---------------------" diff --git a/rhdp/wrapper-multicluster.sh b/rhdp/wrapper-multicluster.sh index 08843f02..a00bfb38 100755 --- a/rhdp/wrapper-multicluster.sh +++ b/rhdp/wrapper-multicluster.sh @@ -13,13 +13,41 @@ get_python_cmd() { fi } -if [ "$#" -ne 1 ]; then - echo "Error: Exactly one argument is required." - echo "Usage: $0 {azure-region-code}" +# Parse arguments +AZUREREGION="" +RECREATE=false + +while [[ $# -gt 0 ]]; do + case $1 in + --recreate) + RECREATE=true + shift + ;; + -*) + echo "Error: Unknown option $1" + echo "Usage: $0 [--recreate] {azure-region-code}" + echo "Example: $0 eastasia" + exit 1 + ;; + *) + if [ -z "$AZUREREGION" ]; then + AZUREREGION="$1" + else + echo "Error: Too many positional arguments." + echo "Usage: $0 [--recreate] {azure-region-code}" + exit 1 + fi + shift + ;; + esac +done + +if [ -z "$AZUREREGION" ]; then + echo "Error: Azure region is required." + echo "Usage: $0 [--recreate] {azure-region-code}" echo "Example: $0 eastasia" exit 1 fi -AZUREREGION=$1 echo "---------------------" echo "Validating configuration" @@ -128,7 +156,11 @@ echo "---------------------" echo "defining both clusters (hub and spoke)" echo "---------------------" PYTHON_CMD=$(get_python_cmd) -$PYTHON_CMD rhdp/rhdp-cluster-define.py --multicluster ${AZUREREGION} +DEFINE_ARGS=(--multicluster) +if [ "$RECREATE" = true ]; then + DEFINE_ARGS+=(--recreate) +fi +$PYTHON_CMD rhdp/rhdp-cluster-define.py "${DEFINE_ARGS[@]}" ${AZUREREGION} echo "---------------------" echo "clusters defined" echo "---------------------" diff --git a/rhdp/wrapper.sh b/rhdp/wrapper.sh index 1016174c..bdd50130 100755 --- a/rhdp/wrapper.sh +++ b/rhdp/wrapper.sh @@ -17,6 +17,7 @@ get_python_cmd() { # Parse arguments AZUREREGION="" PREFIX="" +RECREATE=false while [[ $# -gt 0 ]]; do case $1 in @@ -28,9 +29,13 @@ while [[ $# -gt 0 ]]; do PREFIX="${1#*=}" shift ;; + --recreate) + RECREATE=true + shift + ;; -*) echo "Error: Unknown option $1" - echo "Usage: $0 [--prefix ] {azure-region-code}" + echo "Usage: $0 [--prefix ] [--recreate] {azure-region-code}" echo "Example: $0 eastasia" echo "Example: $0 --prefix cluster1 eastasia" exit 1 @@ -40,7 +45,7 @@ while [[ $# -gt 0 ]]; do AZUREREGION="$1" else echo "Error: Too many positional arguments." - echo "Usage: $0 [--prefix ] {azure-region-code}" + echo "Usage: $0 [--prefix ] [--recreate] {azure-region-code}" exit 1 fi shift @@ -50,7 +55,7 @@ done if [ -z "$AZUREREGION" ]; then echo "Error: Azure region is required." - echo "Usage: $0 [--prefix ] {azure-region-code}" + echo "Usage: $0 [--prefix ] [--recreate] {azure-region-code}" echo "Example: $0 eastasia" echo "Example: $0 --prefix cluster1 eastasia" exit 1 @@ -167,11 +172,14 @@ echo "---------------------" echo "defining cluster" echo "---------------------" PYTHON_CMD=$(get_python_cmd) +DEFINE_ARGS=() if [ -n "$PREFIX" ]; then - $PYTHON_CMD rhdp/rhdp-cluster-define.py --prefix "${PREFIX}" ${AZUREREGION} -else - $PYTHON_CMD rhdp/rhdp-cluster-define.py ${AZUREREGION} + DEFINE_ARGS+=(--prefix "${PREFIX}") +fi +if [ "$RECREATE" = true ]; then + DEFINE_ARGS+=(--recreate) fi +$PYTHON_CMD rhdp/rhdp-cluster-define.py "${DEFINE_ARGS[@]}" ${AZUREREGION} echo "---------------------" echo "cluster defined" echo "---------------------"