[Chore] Require changed-code mutation tests before review

[zoomote]

​Created by Roomote. Follow up by mentioning @roomote, in the web UI, or in Discord.

Related GitHub Issue

Closes: N/A - requested from the linked Discord discussion.

Description

Adds a fork-safe mutation-diff pull-request check that derives executable TypeScript/JavaScript line ranges from the PR merge base and runs StrykerJS only against those ranges. The gate covers focused Vitest suites in core, cloud, telemetry, vscode-shim, webview-ui, and the extension; unsupported or non-executable changes return an explicit not-applicable success.

Any surviving or uncovered changed-code mutant fails with bounded line annotations. Each package fails closed above 500 changed executable lines or 400 generated/valid mutants, and broad or unreasoned Stryker exclusions are rejected. Mutation results are never cached, and every selected run uses --force.

Webview and extension changes use Vitest's dependency graph to discover affected tests, then prefer directly named unit specs when available before passing exact test files to Stryker. This keeps the new surfaces practical without including VS Code e2e tests. The lockfile also pins patched qs@6.15.2, and the Stryker config remains safely importable by Knip.

Test Procedure

• Run pnpm test:mutation-ci to exercise merge-base selection, executable ranges, caps, report evaluation, and exclusion policy.
• Run pnpm test for the complete workspace suite.
• Run actionlint .github/workflows/mutation-testing.yml.
• Verify real webview and extension range runs discover focused unit tests, produce JSON/HTML reports, and emit blocking annotations for known surviving or uncovered mutants.

Pre-Submission Checklist

• Issue Linked: The originating Discord request is linked above.
• Scope: Changes are limited to the StrykerJS CI gate and its tests/dependencies.
• Self-Review: The final diff received focused completion reviews.
• Testing: Diff/range, cap, exclusion, timeout, and report-policy tests were added.
• Visual Snapshot: Not applicable; this is non-visual CI tooling.
• Documentation Impact: No user-facing documentation update is required.
• Contribution Guidelines: Repository conventions and hooks were preserved.

Visual Snapshots

Not applicable.

Videos (interaction / animation only)

Not applicable.

Documentation Updates

• No documentation updates are required.
• Yes, documentation updates are required.

Additional Notes

After this workflow lands on main, a repository administrator must add the stable mutation-diff context to the required-status-check ruleset. Requiring it before the workflow exists on the default branch could leave unrelated pull requests permanently pending.

CLI, types, and VS Code e2e surfaces remain explicit not-applicable paths. Webview and extension mutation runs are bounded by the existing 12-minute per-process and 30-minute workflow timeouts; multi-package PRs can therefore fail on runtime before reaching the numeric mutant cap and may need to be split.

Merge-group runs continue to record prior PR enforcement without rerunning PR diff mutation; later applicable PRs run the real changed-line gate.

Get in Touch

Continue in the originating Discord thread or mention @roomote on the pull request.

[github-actions]

Review process

This PR was opened by an automated account. A human maintainer must verify the change intent, provenance, and validation before merging.

1. Required CI checks pass.
2. The workflow starts CodeRabbit automatically.
3. For eligible human-authored PRs, CodeRabbit reviews and approves the latest commit.
4. A human maintainer reviews and approves after CodeRabbit.

Current step: Ready for human maintainer review and approval.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[edelauna]

@CodeRabbit review

[coderabbitai]

Action performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 1c059268-8980-47e9-8192-dda9fe3dbfdb

📥 Commits

Reviewing files that changed from the base of the PR and between a1ca0c8 and f83233a.

⛔ Files ignored due to path filters (1)

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

📒 Files selected for processing (7)

• .github/workflows/mutation-testing.yml
• .gitignore
• package.json
• scripts/stryker-diff.mjs
• scripts/stryker-diff.test.mjs
• stryker.config.mjs
• webview-ui/vitest.stryker.config.ts

---

📝 Walkthrough

Summary by CodeRabbit

• Tests

  • Added automated mutation testing for changed code in pull requests and merge queues.
  • Added safeguards for uncovered, surviving, excessive, invalid, or inconclusive mutations.
  • Added detailed reporting, annotations, and mutation reports for improved test-quality visibility.
  • Added comprehensive coverage for mutation analysis, changed-line detection, test selection, and reporting behavior.

• Chores

  • Added configuration for mutation testing across supported application areas.
  • Added ignore rules for generated mutation-testing reports and temporary files.

Walkthrough

The pull request adds changed-code mutation testing with Stryker. It selects executable changed lines, discovers related Vitest tests, evaluates mutation reports, emits GitHub annotations and summaries, and runs the gate in pull-request workflows.

Changes

Changed-code mutation testing

Layer / File(s)	Summary
Mutation testing configuration and package contracts stryker.config.mjs, webview-ui/vitest.stryker.config.ts, package.json, .gitignore	Adds Stryker and Vitest configuration, mutation-testing dependencies and scripts, a qs override, and report ignore rules.
Changed executable-code selection scripts/stryker-diff.mjs, scripts/stryker-diff.test.mjs	Parses Git changes, identifies executable changed lines, routes files by package, builds mutation manifests, validates directives, and tests these behaviors.
Related tests and mutation report evaluation scripts/stryker-diff.mjs, scripts/stryker-diff.test.mjs	Discovers related Vitest tests, runs Stryker, evaluates mutant outcomes and limits, and produces annotations and summaries.
Pull-request mutation gate .github/workflows/mutation-testing.yml	Runs the mutation gate for pull requests, records merge-queue enforcement, and uploads mutation reports.

Estimated code review effort: 4 (Complex) | ~60 minutes

Suggested reviewers: edelauna

Sequence Diagram(s)

sequenceDiagram
  participant PullRequest
  participant GitHubActions
  participant stryker-diff
  participant Vitest
  participant Stryker
  participant MutationReport

  PullRequest->>GitHubActions: trigger mutation workflow
  GitHubActions->>stryker-diff: pass base and head SHAs
  stryker-diff->>Vitest: discover related test files
  stryker-diff->>Stryker: run mutations for changed executable lines
  Stryker-->>MutationReport: write mutation results
  stryker-diff->>MutationReport: evaluate mutant outcomes
  stryker-diff-->>GitHubActions: publish status, annotations, and summary

Loading

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/stryker-pr-gate-3uinx2ul0jc05

---

_{Comment @coderabbitai help to get the list of available commands.}