fix: silence false-positive --explain hints and preview selected FQNs at lifecycle level#29
Merged
vedanthvdev merged 1 commit intomasterfrom Apr 22, 2026
Conversation
…in hints and preview selected FQNs at lifecycle level Two fixes from the v1.9.17 sanity-test pass on the security-service pilot, plus an upstream refactor that falls out of tightening the second one. The --explain "Hint: outOfScopeTestDirs is configured but no files matched" line fired on 5 of 9 representative MR shapes — including markdown-only diffs and empty diffs where no source-tree matcher could have bitten. The false-positive rate trained reviewers to ignore the line, which defeated its diagnostic purpose. The hint now requires the resolved Situation to be DISCOVERY_SUCCESS or DISCOVERY_EMPTY (the only two branches where a correctly-configured out-of-scope pattern could have changed the outcome). The four suppressed situations — EMPTY_DIFF, ALL_FILES_IGNORED, ALL_FILES_OUT_OF_SCOPE, UNMAPPED_FILE — each have an explicit negative test so the gate can't silently drift. SELECTED dispatches previously printed only the module summary at lifecycle level and demoted every FQN to info, so a reviewer scrolling the default CI log could see the dispatch size but not which tests ran. The new renderLifecycleDispatchPreview helper surfaces the first five FQNs per module with a "… and N more (use --info for full list)" tail on larger dispatches, keeping the default log diagnosable without breaching the 4 MiB GitHub Actions step cap that forced the info-level demotion in the first place. The helper is a pure function with its own unit-test file so the exact format stays pinned without a Gradle runtime. Hoisting FQN validation out of the per-module log loop to feed the lifecycle preview also fixed a latent header inconsistency — the "Running N affected test classes" line used the pre-validation count while the per-module counts underneath it were post-validation — and closed a pre-existing silent-fallback: a module whose entire discovered FQN set failed isValidFqn used to dispatch its bare taskPath with no --tests filter, silently escalating to a full module suite. Modules with zero valid FQNs are now dropped from dispatch, and a dispatch where zero FQNs survive validation across ALL modules throws GradleException instead of invoking Gradle with an empty task list (environment-dependent behaviour). Both changes are documented in the CHANGELOG so adopters relying on the accidental fallback can migrate to explicit runAllIfNoMatches / Action.FULL_SUITE config.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Batch-3 of the post-v1.9.17 sanity-test fixes. Two user-visible
changes, one latent-bug cleanup that falls out of the refactor.
1.
--explainhint gatingRunning v1.9.17 across nine representative MR shapes on the
security-service pilot surfaced the
Hint: outOfScopeTestDirs is configured but no file in the diff matched any configured patternline firing on five of them — including a markdown-only MR where
nothing a source-tree matcher could have bitten was even in the diff.
That 5-of-9 false-positive rate trains reviewers to ignore the hint,
which is exactly the anti-pattern it was added in v1.9.14 to prevent.
The hint is now gated on
Situation.DISCOVERY_SUCCESSorSituation.DISCOVERY_EMPTY— the only two branches where acorrectly-configured out-of-scope pattern could have changed the
outcome. The other four situations suppress:
EMPTY_DIFFALL_FILES_IGNOREDALL_FILES_OUT_OF_SCOPEUNMAPPED_FILEEach negative case has an explicit test so the gate can't drift.
2. Lifecycle FQN preview for
SELECTEDdispatchesPre-v1.9.18 the task printed only the module summary at lifecycle
level (
application:test (47 test classes)) and demoted every FQN toinfo. A reviewer scrolling the default CI log could see the dispatch
size but not which tests ran without either re-running with
--info(slow) or opening the JUnit report after the fact.renderLifecycleDispatchPreviewis a new pure helper that emits thefirst 5 FQNs per module at lifecycle level with a
… and N more (use --info for full list)tail on larger dispatches.The 5-class cap keeps total lifecycle output well under the 4 MiB
GitHub Actions step cap that forced the info-level demotion in the
first place. Info-level per-FQN logging is retained for
--infousers.
Example lifecycle output (17-class dispatch):
3. Header arithmetic fix + silent-fallback closure
Hoisting FQN validation out of the per-module log loop to feed the
lifecycle preview also fixed two latent issues:
Running N affected test classesheader used the pre-validation count while per-module counts were
post-validation. Skipped FQNs now surface in a
(K malformed FQN skipped — see WARN above)suffix.FQN set failed
isValidFqnused to dispatch its baretaskPathwith no
--testsfilter, silently escalating to a full modulesuite — the exact safety regression
isValidFqnwas added toprevent. Modules with zero valid FQNs are now dropped from dispatch,
and a cross-all-modules zero-survivors dispatch throws
GradleExceptioninstead of invoking Gradle with an empty tasklist. CHANGELOG calls both behavior deltas out for adopters.
Test plan
./gradlew check— greenAffectedTestTaskExplainFormatTest(one per suppressed situation, plus an explicitALL_FILES_OUT_OF_SCOPEpin)DISCOVERY_EMPTYfull-suite escalation path