diff --git a/README.md b/README.md index e49de98..8123093 100644 --- a/README.md +++ b/README.md @@ -149,6 +149,28 @@ VM with the pairing enabled, let the reconciler apply, then assert the runtime state by querying the live service API — not the Terraform state. Eval-only or build-only success is not evidence the reconciliation works. +## Troubleshooting + +**`declarative-.service` fails with `locked provider … does not match +configured version constraint`.** The reconciler's `.terraform.lock.hcl` is host +state under the service's state directory and survives redeployments, while the +generated `required_providers` constraint is pinned to whatever provider version +nixpkgs packages — so bumping the `nixpkgs` input moves the constraint out from +under the lock. The reconciler runs `tofu init -upgrade` to re-lock by itself +(still offline: the plugin dir baked in by `opentofu.withPlugins` is the only +available source). On a host still stuck from an older revision, drop the lock +and re-run: + +```sh +rm /var/lib//declarative-terraform/.terraform.lock.hcl +systemctl restart declarative-.service +``` + +The `terraform.tfstate` next to it is untouched, so the next apply reconciles +against the existing state rather than recreating resources. For a +`DynamicUser=` service such as Keycloak the real directory lives under +`/var/lib/private/`, reachable as root through the `/var/lib/` symlink. + ## Design decisions - **Executor:** OpenTofu (MPL 2.0); `terraform` (BSL 1.1, unfree) is not used. diff --git a/modules/lib/default.nix b/modules/lib/default.nix index c9af551..5462a00 100644 --- a/modules/lib/default.nix +++ b/modules/lib/default.nix @@ -563,7 +563,13 @@ rec { for id in ${lib.escapeShellArgs (lib.attrNames allCredentials)}; do export "TF_VAR_$id=$(cat "$CREDENTIALS_DIRECTORY/$id")" done - tofu init -no-color + # -upgrade: the provider version is pinned to whatever nixpkgs + # packages, so a nixpkgs bump moves the required_providers + # constraint while .terraform.lock.hcl in the state dir still pins + # the old version -- a plain init would fail. re-selecting stays + # offline: the plugin dir baked in by withPlugins is the only + # available source. + tofu init -upgrade -no-color tofu apply -auto-approve -input=false -no-color ''; }; diff --git a/services/forgejo/checks.nix b/services/forgejo/checks.nix index 3bdbe45..5077ce0 100644 --- a/services/forgejo/checks.nix +++ b/services/forgejo/checks.nix @@ -10,7 +10,9 @@ # a repository by its numeric id — so a successful apply proves reference # resolution and apply ordering. A user with a `passwordFile` also proves # per-secret credential indirection — the value is loaded from a host file -# and kept out of the generated `.tf.json`. Requires KVM (a NixOS VM test). +# and kept out of the generated `.tf.json`. A stale +# .terraform.lock.hcl is then planted to prove the reconciler re-locks +# offline after a provider version bump. Requires KVM (a NixOS VM test). { pkgs, self }: { forgejo = pkgs.testers.runNixOSTest { @@ -145,6 +147,21 @@ assert "0 added, 0 changed, 0 destroyed" in apply_lines[-1], \ f"reapply was not a no-op: {apply_lines[-1]}" + # A provider version bump moves the generated required_providers + # constraint while .terraform.lock.hcl -- host state under the service's + # state dir -- still pins the old version. Simulate that skew by + # rewriting the recorded version: a plain `tofu init` aborts with + # "locked provider ... does not match configured version constraint", + # so this is what the reconciler's `-upgrade` buys. The VM has no + # network, so re-locking here also proves the re-selection stays offline + # (the plugin dir baked in by withPlugins is the only source). + lock = "/var/lib/forgejo/declarative-terraform/.terraform.lock.hcl" + machine.succeed(f"sed -i 's/version *=.*/version = \"0.0.1\"/' {lock}") + machine.succeed(f"grep -q 0.0.1 {lock}") + machine.succeed("systemctl restart declarative-forgejo.service") + relocked = machine.succeed(f"cat {lock}") + assert "0.0.1" not in relocked, f"stale lock was not re-locked: {relocked}" + # Adding an admin-scoped resource (a user needs write:admin + read:user) # mus work because the scopen is the maximal "all" token machine.succeed("/run/current-system/specialisation/widenScope/bin/switch-to-configuration test")