Make PR mutation testing git-diff-aware

[jonbaldie]

Problem Statement

Pull requests currently run both mutation engines across the entire production source set, even when a PR changes only documentation or project tooling. The mutation jobs are deliberately thorough, but the full Stryker run takes roughly twelve minutes and dominates feedback time. Contributors need mutation coverage to remain a meaningful release gate without paying the full-suite cost for files a PR cannot affect.

The two mutation engines also select targets differently today: one has an explicit production-file list while the other uses a broad source glob. Adding diff awareness separately would risk inconsistent coverage and silent gaps.

Solution

Make mutation target selection aware of the pull request's Git diff. A shared selector will compare the PR head with the merge base of its actual target branch and provide one authoritative production-file set to both mutation engines.

For a PR that changes production source, mutate only added, copied, modified, or renamed production files. Exclude deleted paths. Changes to tests, mutation configuration, mutation runners, threshold enforcement, or CI mutation orchestration must trigger a full mutation run because they can affect results for every source file. A PR with no mutation-relevant change should log why mutation was skipped and complete both mutation steps successfully.

Builds that are not associated with a pull request, including builds on main and release tags, must continue to run the complete mutation suite. If the selector cannot identify or fetch the correct PR base safely, it must fail closed by selecting the full mutation suite rather than silently skipping coverage.

User Stories

1. As a pull request author, I want mutation testing limited to the production files my PR changes, so that I receive useful feedback sooner.
2. As a documentation contributor, I want a docs-only PR to skip mutation execution cleanly, so that unrelated mutation work does not delay the PR.
3. As a maintainer, I want both mutation engines to consume the same selected targets, so that their coverage cannot drift silently.
4. As a maintainer, I want the selector to compare against the PR's actual target branch, so that stacked or non-main pull requests are evaluated correctly.
5. As a maintainer, I want added production files included, so that new behavior receives mutation coverage immediately.
6. As a maintainer, I want modified production files included, so that changed behavior remains protected by mutation thresholds.
7. As a maintainer, I want renamed or copied production files included under their head paths, so that Git path changes do not evade mutation testing.
8. As a maintainer, I want deleted production files excluded, so that mutation tools are never asked to process paths that no longer exist.
9. As a test author, I want any test-suite change to trigger full mutation testing, so that changes in mutation-killing power are measured across the complete production source set.
10. As a CI maintainer, I want changes to mutation configuration, runners, threshold checks, or mutation job orchestration to trigger a full run, so that the mutation system validates its own changes comprehensively.
11. As a release owner, I want main and release builds to retain full mutation coverage, so that PR optimization does not weaken the release gate.
12. As a maintainer, I want an unavailable or ambiguous PR base to fall back to a full run, so that metadata failures cannot create a false green result.
13. As a contributor, I want CI logs to state the comparison base, selection reason, and selected paths, so that I can understand why mutation ran or skipped.
14. As a maintainer, I want existing per-file score thresholds to apply unchanged to every selected file, so that faster execution does not lower quality standards.
15. As a maintainer, I want ordinary compilation and tests to remain the prerequisite for mutation jobs, so that mutation resources are not spent on an already-broken change.
16. As an agent implementing this work, I want one documented selector interface with deterministic output, so that both engines can integrate without duplicating Git logic.
17. As a security-conscious maintainer, I want changed filenames handled as data rather than interpolated shell fragments, so that unusual valid paths cannot alter CI commands.

Implementation Decisions

• Introduce one shared mutation-target selector and make both mutation runners consume its output. Target discovery must not be implemented independently per engine.
• The selector's observable result has three modes: selected production paths, full suite, or clean skip. It must also report the reason for the chosen mode.
• On PR builds, obtain the actual target branch from CI or pull-request metadata, fetch it explicitly, and diff from the merge base to the checked-out head commit.
• Treat added, copied, modified, and renamed production TypeScript files as selected targets. Ignore deletions and paths outside production source.
• Treat changes to tests or any mutation/CI orchestration component as globally relevant and select the full suite.
• Treat changes that cannot affect runtime or mutation behavior, such as agent documentation and editor/project metadata, as eligible for clean skip.
• Preserve the current job ordering: compilation and ordinary tests first, then Mutasaurus, then Stryker. Optimizing job parallelism is a separate concern.
• Preserve existing engine versions, mutation operators, test command, concurrency settings, and per-file threshold enforcement.
• Adapt the Mutasaurus runner to accept the selector's production paths instead of relying only on its static list.
• Adapt Stryker invocation to override its broad mutation pattern with the same selected paths for PR-scoped runs.
• Pass selected paths through a structured or delimiter-safe interface. Do not construct executable shell from raw filenames.
• If PR metadata, target fetching, merge-base calculation, or selection parsing fails, log the problem and run the full suite.
• A clean skip must be an explicit successful outcome with a clear log message, not an absent status or an accidental empty engine invocation.

Testing Decisions

• Test at one high seam: invoke the selector against real temporary Git repositories containing actual base and head commits. Do not mock Git or CI command output.
• Cover production-file addition, modification, rename, copy, and deletion.
• Cover docs-only and project-metadata-only diffs producing a clean skip.
• Cover test changes and mutation-infrastructure changes forcing a full run.
• Cover a PR targeting a branch other than main.
• Cover missing remote metadata, an unfetchable base, and a failed merge-base calculation falling back to the full suite.
• Cover filenames containing spaces and shell metacharacters to verify paths remain data.
• Add integration coverage proving both mutation runners receive the identical selected production paths.
• Retain the existing full mutation checks as the acceptance test for non-PR builds, including the current per-file threshold behavior.
• Exercise the CI-facing command as a real process and assert its exit status and human-readable selection summary rather than testing internal helper functions.
• Run a tracer-bullet PR for each mode before completion: one source change, one test or mutation-infrastructure change, and one docs/tooling-only change.

Out of Scope

• Changing Queue, Queue Manager, HTTP, authentication, persistence, or rate-limiting behavior.
• Replacing either mutation engine.
• Lowering or redesigning mutation score thresholds.
• Parallelizing the two mutation jobs.
• Adding mutation-result caching or Stryker incremental mode.
• Selecting individual changed lines or mutants; this feature scopes at the production-file level.
• Optimizing ordinary compilation or test execution.

Further Notes

The current workflow serializes ordinary tests, Mutasaurus, and Stryker. The Stryker stage is the dominant cost and currently mutates the full production-source glob, while Mutasaurus maintains an explicit source list. The shared selector should remove that mismatch and make the optimization visible and auditable.

This issue is ready for implementation when the selector's three modes and the full-run triggers above are treated as acceptance criteria.