test: add PIT mutation testing with per-module score gates - #84
Merged
Conversation
Wire the info.solidsoft.pitest plugin into the four logic-bearing modules (core, runtime, web, testing) following the existing JaCoCo convention, with a hard per-module mutationThreshold gate. Runs locally via ./gradlew pitest and as an opt-in nightly/manual CI workflow. Suppress Kotlin synthetic-mutant noise (null intrinsics, serializers, inlined comparators) via avoidCallsTo and excludedClasses, and strengthen the runtime and testing suites to 100% / 98% mutation coverage.
Add kotlin-logging (io.github.oshai) to avoidCallsTo — the bulk of web's survivors were logging-call noise — add real LibrarySourceProvider tests plus response-DTO assertions, and raise the web mutationThreshold to 90. Fix a bug the new tests surfaced: LibrarySourceProvider read the wrong properties key (projectVersion instead of version), so the playground's runtime-dependency snippet always rendered "unknown".
emaarco
force-pushed
the
emaarco/add-pitest-mutation-testing
branch
from
August 14, 2026 05:50
8066590 to
86c50c7
Compare
Contributor
|
Test Coverage
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds PIT mutation testing to complement the JaCoCo line-coverage gate. Line coverage proves a line ran; mutation testing proves a test would notice if its behaviour changed.
info.solidsoft.pitestplugin into the four logic-bearing modules (core,runtime,web,testing), following the existing JaCoCo wiring convention, with a hard per-modulemutationThresholdgate../gradlew pitest --no-configuration-cache) and as a separate opt-in CI workflow (nightly + manual dispatch), not on the PR path — PIT is too slow to gate every PR.$DefaultImpls,$Companion,$WhenMappings,kotlinx.serialization$$serializer,sortedBy$$inlined) viaexcludedClasses, and null/resource intrinsics +kotlin-loggingcalls viaavoidCallsTo. This noise alone understated the raw scores by 6–15 points per module.runtime,testingandwebsuites to close the real gaps PIT surfaced.Mutation coverage / thresholds
Thresholds sit a few points below current coverage as a regression ratchet.
webstarted at 28% — most of that waskotlin-loggingcall noise; the rest was a genuinely untestedLibrarySourceProvider.Bug found & fixed
Testing
LibrarySourceProvidersurfaced a real bug:loadProjectVersion()read the wrong properties key (projectVersioninstead ofversion), so the playground's runtime-dependency snippet always renderedunknown. Fixed here.Notes
pitest-junit5-plugindocs only list JUnit 5).pitesttask is not configuration-cache compatible, so it runs with--no-configuration-cache; applying the plugin does not affect the normal cached build.coreis still climbing (survivors concentrated in the BPMN parsing/dialect adapters) — a follow-up can push it higher.mainafter build: enforce Kotlin quality with ktlint + detekt guardrails #83 (ktlint guardrails);lintKotlin,detektandbuildare green.See ADR 019 for the full rationale.