fix k3s shutdown cleanup - #816
Draft
bussyjd wants to merge 1 commit into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
k3s-killall.shbeside the configured K3s binary, beside a resolved binary symlink, or in the standard installer fallback locationsK3S_DATA_DIR, including when the PID file is missing or stalepkill containerd-shimfallback and surface cleanup failures while retaining PID evidenceDestroyfrom deleting global K3s directories or invoking the system-wide uninstaller; persistent data remains governed bystack purge --forceRoot cause
K3s intentionally leaves containers running when its service stops. Its official killall helper performs the corresponding containerd, CNI, mount, and iptables cleanup while preserving cluster data:
Obol previously looked only for
/usr/local/bin/k3s-killall.sh. The supported backend setup usesINSTALL_K3S_BIN_DIR, which installs the helper beside the K3s binary. Missing that helper caused shutdown to fall back to killing shim processes without removing K3s networking state.The previous destroy path also invoked the system-wide K3s uninstaller and removed global directories. That exceeds Obol's ownership boundary and bypasses the CLI's existing
--forcecontrol for persistent-data removal.Impact
obol stack downnow delegates runtime cleanup to the version-matched K3s helper and preserves cluster data for restart. Stale PID state no longer prevents orphan cleanup.obol stack purgepreserves K3s data unless the operator explicitly passes--force, and it no longer removes system K3s assets.Validation
No live K3s instance was exercised.
go test ./cmd/obol ./internal/tunnel ./internal/stack -count=1go test -race ./internal/stack -run 'TestK3s' -count=1golangci-lint run --new-from-rev=main ./internal/stack/...go build ./cmd/obolK3S_DATA_DIRpropagation, missing-PID cleanup, failure retention, and persistent-data preservation