Skip to content

Run PIT mutation testing on self-hosted performance runner #41

@astubbs

Description

@astubbs

Context

PIT mutation testing on GitHub's ubuntu-latest runners is extremely slow:

  • Coverage phase: ~7 minutes (fine)
  • Mutation phase: hit GitHub's 6-hour per-job hard limit without completing (76 mutation units targeting io.confluent.parallelconsumer.*)

Narrowed to internal.* classes only and added incremental history caching in PR #34, which should help. But the long-term fix is running PIT on the self-hosted performance runner where CPU and I/O are much faster.

Proposal

Move the mutation-testing job to runs-on: [self-hosted, performance] (same runner used by the performance tests workflow in .github/workflows/performance.yml). This would let us:

  • Target the full io.confluent.parallelconsumer.* class set
  • Complete in minutes instead of hours
  • Potentially make PIT a blocking check again

Current workaround

PIT runs on ubuntu-latest with continue-on-error: true (non-blocking), timeout-minutes: 300, targeting only internal.* classes, with incremental history cache.

See docs/SELF_HOSTED_RUNNER.md for runner setup.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions