Skip to content

fix: stop staging unused ECS Exec dependencies on runners - #38

Merged
hllvc merged 2 commits into
mainfrom
fix/ecs-exec-command
Aug 19, 2026
Merged

fix: stop staging unused ECS Exec dependencies on runners#38
hllvc merged 2 commits into
mainfrom
fix/ecs-exec-command

Conversation

@hllvc

@hllvc hllvc commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Stops the runner bootstrap from staging AWS's ECS Exec (execute-command) SSM binaries on disk, and removes the directory if a previous install left it there. ECS Exec is not used by any StackGuardian workflow, so these binaries were never executed — they only added unused code that customer vulnerability scanners flag.

Motivation & Context

A customer's Wiz scan flagged CVE-2026-71556 (go-git v5.19.1, High) in /var/lib/ecs/deps/execute-command/bin/3.3.4624.0/ssm-session-worker on their private runners, and asked whether the platform requires ECS Exec.

It does not. enableExecuteCommand is not set on any RunTask call — EXTERNAL, SHARED-EXTERNAL, EC2 or FARGATE — so ECS Exec is off for every workflow task we launch. The binaries arrive anyway: main.sh runs AWS's stock ecs-anywhere-install.sh, which calls exec-setup unconditionally in its main flow, hardcoding BINARY_VERSION="3.3.4624.0". AWS exposes no flag to skip it, so the only way to stop it is to intervene ourselves.

The ECS agent treats these dependencies as optional. With the directory absent, appendExecCapabilities returns cleanly and the agent simply stops advertising ecs.capability.execute-command — which is a small security gain, since a runner that does not advertise the capability cannot have an exec session opened against it.

Changes Made

  • main.sh: add disable_ecs_exec_setup, which rewrites the installer's bare top-level exec-setup call to a no-op before the script runs. This is the primary mechanism — the binaries are never downloaded, which also removes a large tarball fetch from every bootstrap.
  • main.sh: add remove_ecs_exec_deps, which deletes ECS_EXEC_DEPS_DIR afterwards. Backstop for the case where AWS restructures the installer and the rewrite no longer matches.
  • main.sh: wire both into register_instance unconditionally — the rewrite after the existing shebang sanity check, the sweep once registration settles. No flag gates this; there is no supported setup in which a runner needs those binaries.
  • main.sh: add ECS_EXEC_DEPS_DIR to the overridable filesystem-locations block, following the existing convention so tests can redirect it.
  • main.sh: add is_ecs_exec_deps_path, a guard on the removal. Because ECS_EXEC_DEPS_DIR is overridable, this was the only place in the script where an env var supplied a whole rm -rf path rather than a fixed literal, while running as root — a stray / or /etc in the environment would have been destructive. Only an absolute path naming a deps directory is accepted; anything else is a logged no-op. Raised by Copilot in review.
  • Both functions return 0 explicitly: debug() is is_debug && printf, so it returns non-zero whenever --debug is off and must not become a function's exit status.
  • test/unit/ecs_exec_deps.bats: 15 tests covering the rewrite, the sweep, idempotency, the no-exec-setup and indented-mention cases, the production default path, and that both calls are present in register_instance.
  • test/fixtures/ecs/ecs-anywhere-install.sh.sample: trimmed stand-in for AWS's installer preserving the exec-setup definition, its helpers, and the unconditional top-level call.
  • test/helpers/load.bash: redirect ECS_EXEC_DEPS_DIR into the per-test tmpdir. This is a safety requirement, not a convenience — without it a test run on a Linux host would rm -rf the real /var/lib/ecs/deps/execute-command.
  • README.md / test/README.md: document why ECS Exec is unused and why this is not configurable; add the new path override to the seam contract.

Testing

  • 23 new unit tests; full suite 138 passing, 0 failures
  • make lint clean (shellcheck)
  • Removal guard: is_ecs_exec_deps_path rejects /, bare system directories, relative paths, empty/unset, and near-miss names; verified live that ECS_EXEC_DEPS_DIR=/ is refused and returns cleanly
  • Rewrite applied to the live ecs-anywhere-install-latest.sh: produces a one-line diff at line 774, leaves the exec-setup() definition intact, result still passes bash -n
  • Two fresh registrations on the new script: ecs.capability.execute-command absent, deps directory never created, workflow runs completed normally
  • Control comparison: a runner registered two days earlier on the same agent version (1.106.1) with the old script still advertises the capability
  • Post-hoc removal on an already-registered runner: agent stayed healthy and connected through the delete, capability withdrawn after systemctl restart ecs as an attribute update (same container instance ARN, no duplicate registration), workflow runs passed on both sides
  • Reviewer: confirm nothing in your environment depends on ECS Exec before merge

Risks & Edge Cases

  • This does not fully remediate CVE-2026-71556. The same binaries also ship in the amazon-ssm-agent package at /usr/bin/ssm-session-worker and /usr/bin/ssm-agent-worker, carrying the same go-git v5.19.1. That package is required for ECS Anywhere registration and cannot be removed. Worth stating plainly in any customer-facing summary, so this is not read as "fully fixed".
  • The /usr/bin copy is installed from the latest S3 path rather than a pinned version, so it self-heals once AWS ships a build with patched go-git. The copy removed here is the one that would not self-heal, since AWS hardcodes its version in the installer.
  • The installer rewrite is best-effort by design. If AWS moves or renames the exec-setup call the rewrite silently no-ops and the sweep handles it; both are logged under --debug.
  • ECS Exec becomes unavailable on runners registered with this change. That is the intent, but it would need reverting if a future feature ever required exec-into-container.
  • Existing runners are unaffected until re-registered. Remediating them needs rm -rf /var/lib/ecs/deps/execute-command, which is safe to run against a live agent.

Deployment Notes

  • No new flags, env vars or configuration. Behaviour changes on the next runner registration once this ships.
  • Already-registered runners keep the staged binaries until they re-register or are swept manually.

Workflow tasks are never launched with enableExecuteCommand, but AWS's
ecs-anywhere-install.sh calls exec-setup unconditionally and offers no flag
to skip it, staging the SSM session binaries into
/var/lib/ecs/deps/execute-command on every registration. Those binaries are
never executed and keep surfacing in customer vulnerability scans, currently
CVE-2026-71556 in the go-git version vendored by amazon-ssm-agent 3.3.4624.0.

Neutralise the installer's exec-setup call before running it, and sweep the
directory afterwards in case the call site moves. The ECS agent treats these
dependencies as optional: with the directory absent it starts normally and
only stops advertising ecs.capability.execute-command, which we never use.
@hllvc hllvc self-assigned this Aug 19, 2026
@hllvc
hllvc requested a review from arunim2405 August 19, 2026 14:44
@hllvc
hllvc marked this pull request as ready for review August 19, 2026 14:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes AWS ECS Exec “execute-command” dependency staging from runner bootstrap by patching AWS’s ecs-anywhere-install-latest.sh at runtime and sweeping any previously staged binaries from disk, reducing false-positive vulnerability findings and removing an unused capability advertisement.

Changes:

  • Add a bootstrap-time rewrite to neutralize the installer’s unconditional exec-setup call and prevent ECS Exec SSM binaries from being downloaded.
  • Add a post-registration cleanup step to remove the ECS Exec dependency directory if it exists.
  • Add unit tests + fixtures and document the rationale and the new path override seam.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
main.sh Adds disable_ecs_exec_setup and remove_ecs_exec_deps, wires them into registration, and introduces ECS_EXEC_DEPS_DIR as an overridable path.
test/unit/ecs_exec_deps.bats Adds unit coverage for installer rewriting, cleanup behavior, idempotency, and wiring in register_instance.
test/fixtures/ecs/ecs-anywhere-install.sh.sample Adds a minimal installer fixture capturing the exec-setup definition and unconditional call.
test/helpers/load.bash Redirects ECS_EXEC_DEPS_DIR into the per-test temp dir for safe rm -rf usage.
test/README.md Documents the new ECS_EXEC_DEPS_DIR override and why it matters for test safety.
README.md Documents why ECS Exec is unused and why the behavior is intentionally not configurable.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread main.sh
ECS_EXEC_DEPS_DIR is overridable for testing, which made remove_ecs_exec_deps
the only place in the script where an env var supplies a whole rm -rf path
rather than a fixed literal, and it runs as root. A stray "/" or "/etc" in the
environment would have been destructive.

Add is_ecs_exec_deps_path, accepting only an absolute path that names a deps
directory, and refuse anything else with a logged no-op. Kept as a pure
predicate so the catastrophic inputs are unit-testable without any test
pointing rm at them; the integration test uses a sentinel inside the test
tmpdir so a regression destroys a canary rather than anything real.
@sonarqubecloud

Copy link
Copy Markdown

@hllvc
hllvc merged commit 34c77eb into main Aug 19, 2026
3 checks passed
@hllvc
hllvc deleted the fix/ecs-exec-command branch August 19, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants