Add JD025-JD028, pin the behaviours the JD rules claim, harden the harness - #377
Merged
Merged
Conversation
A snippet that does not compile binds no operations, so every rule stands down and an "expects nothing" assertion passes for the wrong reason. The harness returned the analyzer diagnostics without ever looking at the compiler's, which made that failure mode invisible. It is not hypothetical. Turning the check on immediately caught a JD006 test whose snippet declared a static class as its own return type and instantiated it — two CS errors, green since the day it was written. The test's intent is sound (a lookalike API from another library must not be reported); only its snippet was broken, so the class becomes sealed rather than static and the shape it was testing is unchanged. The same check caught a live false positive in the wave being written against it, which is what prompted this: a composer lambda cannot infer its result type from a throw-only body, so the snippet did not compile, no lambda was bound, and the rule stood down for a reason that had nothing to do with the guard under test. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKqyhG9Y4AfdxEYekP2Evq
Four rules for mistakes the run time either reports too late or cannot report at all, and a new JustDummies.Composition category for the two that are about assembling generators rather than constraining one. JD025 DuplicatePoolValue (Warning) — the same constant listed twice in a pool. Pools are deduplicated at construction, so the duplicate does nothing; the reading it exists to refuse is weighting, which the library declines on purpose. Nothing fails here, and that is the problem: the consequence lands elsewhere, when a distinct collection over the pool gates against the real distinct count and names a number the author cannot find in their source. JD026 EmptyRelativeUri (Warning) — a relative URI with zero path segments and no query, fragment or root. This is the one member of the constraint family whose failure lands at act time rather than at the arrange line: emptiness is only settled once the components are drawn, so the library throws inside Generate() with a stack pointing at the code under test. Moving it to build time is the whole value. JD027 UnusedCombineOperand (Warning) — an operand the composer never reads. Combine draws every part before composing, so the operand is generated, constraints and all, then dropped. Name the parameter '_' to say the draw is deliberate. JD028 InertDistinctness (Warning) — distinctness over an element type with no value equality. The default comparer falls back to reference equality and every built element is a new instance, so the requirement is met by construction and constrains nothing. Measured: six "distinct" elements over a two-value domain came back as [1, 1, 1, 2, 1, 2], green. Dogfooding corrected the model twice, and neither error was visible from the source alone. JD028 first claimed any sealed class without equality. That premise — every draw is a new instance — is false for a pool: Any.OneOf hands back the very references it was given, so Any.SetOf(Any.OneOf(first, second)) is legal and meaningful, and the library's own suite asserts exactly that. The rule now fires only where the chain provably builds the value at the call site. JD027 stood down on a composer whose body is a throw, and the first version of that guard did not work: an expression-bodied '=> throw ...' is a Return CARRYING the throw, through a conversion to the composer's result type. Both facts had to be measured. Sweep across JustDummies.UnitTests, JustDummies.PropertyTests, JustDummies.Xunit.UnitTests, FirstClassErrors.Testing.UnitTests, FirstClassErrors.UnitTests, FirstClassErrors.Usage and the binder and CLI suites: two hits, both JD025, both in the library's own tests for duplicate collapsing — code that writes the flagged shape on purpose because it is the test for it. A planted positive control fires all four. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKqyhG9Y4AfdxEYekP2Evq
Every JD rule is a claim about how the library behaves. Auditing the catalogue against the suite found the claims mostly pinned already — the empty relative URI, the collapsing pool, the inert constraint and the whole conflict family all have their example — and four that were not. All four are silent behaviours, which is why nothing missed them. Each new test writes the shape its rule reports, on purpose, and asserts the consequence. A SuppressMessage names the rule and says why the shape is deliberate, so a reader meeting the test after the analyzers are wired in finds the reason rather than a puzzle. JD006 — building a generator draws NOTHING. A counting projection built and left unmaterialized runs zero times. This is why a chain left without Generate() is silent: the arrange line reads like it did something. JD005 — a generator interpolated into text renders its type name. The assertion is against typeof(AnyInt32).ToString(), so it survives a rename and goes red the day someone adds a ToString override returning a drawn value — an override that would mask the mistake is exactly what this pins the absence of. JD027 — Combine draws every operand before composing, including one the composer never reads. Two counting operands, a composer reading one: both counters reach 1. JD028 — distinctness over an element type without value equality is inert. Percentage has no Equals and '.As' builds a new instance per draw, so six "distinct" elements over a two-value domain succeed while holding repeats. Not statistical: six draws from a domain of two cannot show more than two values. JD013 — a held collection passed to OneOf is ONE pool member. Inference binds T to the collection, so the call is legal, the draw succeeds, and what comes back is the whole list; the test asserts reference identity with the argument, and contrasts it with ElementOf on the same list. Each assertion was inverted once and confirmed red before being restored, per the suite's own rule that a test which cannot fail proves nothing. The net472 floor leg builds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKqyhG9Y4AfdxEYekP2Evq
Reefact
force-pushed
the
claude/justdummies-analyzers-analysis-z9q1xd
branch
from
July 29, 2026 17:27
96efa1a to
9114bd4
Compare
Reefact
changed the title
The JustDummies analyzers are loaded by no project in this repository. Their only standing verification is their own unit suite, where the same author writes both the rule and the snippet that tests it — so a wrong model of the library passes both sides. The evidence that this matters is already in the log. A hand-run sweep over the repository's suites has contradicted that model at least once in each of the last four waves: JD015's casing-as-a-character-pool, JD016 counting declared enum members under AllowingCombinations, JD023 on the representable extremes, JD028 assuming every draw is a fresh instance, and JD027's throw guard that did not fire. None was found by the unit suite. All were found on code written for other reasons, which is exactly why it did not share the misconception. The record proposes loading the analyzers into every project that consumes JustDummies, and consigning each deliberate violation to a suppression naming its rule. Measured before writing, not assumed: the build succeeds, exactly two diagnostics are reported, the five suppressions added in this branch already silence their rules, and two more attributes bring the surface to zero. A cold non-incremental build of that project moved from roughly six and a half seconds to nine. Three alternatives are argued and declined — keeping the sweep manual, a JustDummies sample mirroring FirstClassErrors.Usage, and an advisory CI job on the ADR-0046 model. The risks are stated rather than smoothed over: this verifies the loud rules and leaves the three Info ones and the two opt-in ones unexercised, which the follow-up actions carry. Proposed only. The maintainer accepts or declines; nothing is wired. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GKqyhG9Y4AfdxEYekP2Evq
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Four new rules for mistakes the run time either reports too late or cannot report at all, a new
JustDummies.Compositioncategory, and — following @Reefact's observation on the sweep results — an audit of the whole JD catalogue against the library's own suite, filling the four behavioural claims that turned out unpinned. A third commit hardens the analyzer test harness so a snippet that does not compile fails instead of silently passing. A fourth proposes ADR-0061, which asks whether the JD analyzers should run on this repository's own code.Rebased onto
mainafter #376.Type of change
Changes
1. The four rules
DuplicatePoolValueAny.OneOf(1, 2, 1)EmptyRelativeUriRelative().WithPathSegments(0)with no query, fragment or rootGenerate()— at act timeUnusedCombineOperandInertDistinctnessDistinct()over a type with no value equalityNew
JustDummies.Compositioncategory for JD027/JD028: they do not constrain a generator, they assemble several.EqualityFactsdecides value equality the way the default equality comparer decides it. 48 new analyzer tests (198 → 246), 8 documentation pages (EN + FR), both index tables, release-tracking rows.2. Pinning the behaviours the rules claim
The sweep turned up two JD025 hits, both in the library's own tests for duplicate collapsing. @Reefact's point: those tests should exist, and two is a suspiciously small number — every rule is a claim about the library, and each claim should have a test that writes the weird case and asserts the consequence.
The audit says the catalogue is in better shape than that suggests, and worse in four specific places. Already pinned: the empty relative URI (
AnyUriTests.EmptyRelativeFailsAtGeneration, plus a property), the collapsing pool, the inert constraint (AnyOneOfTests.ExcludingAnAbsentValueRemovesNothingis literally JD024's shape), and the whole conflict family. Four claims were not pinned — all four silent behaviours, which is why nothing missed them:typeof(AnyInt32).ToString(), so it survives a rename and goes red if aToStringoverride ever masks the mistakeCombinedraws every operand, including one the composer ignores — both counters reach 1Percentageelements over a two-value domain succeed holding repeats (not statistical: six draws from a domain of two cannot show more than two values)Any.OneOf(held)is a pool of one: the draw is the list itself, asserted by reference identity, contrasted withElementOfon the same argumentEach carries a
SuppressMessagenaming its rule and saying why the shape is deliberate.Per the suite's own rule (
WritingJustDummiesTests.en.md: "a property that returns true for the wrong reason passes silently and proves nothing"), each of the five assertions was inverted once and confirmed red before being restored. Thenet472floor leg builds.3. The harness
A snippet that does not compile binds no operations, so every rule stands down and an "expects nothing" assertion passes for the wrong reason. Turning the check on immediately caught a pre-existing JD006 test whose snippet declared a static class as its own return type and instantiated it — two CS errors, green since the day it was written. Its intent is sound; only the snippet was broken, so the class becomes
sealedrather thanstatic.It also caught a live false positive in this very wave: a throw-only composer lambda gives inference nothing to infer
TResultfrom, so my first JD027 test did not compile, no lambda was bound, and the rule stood down for a reason that had nothing to do with the guard under test. The test passed while the real site still fired.4. ADR-0061 — proposed, nothing wired
File:
doc/handwritten/for-maintainers/adr/0061-run-the-justdummies-analyzers-on-the-repository-s-own-code.md(and its French translation), in the Files changed tab.The
SuppressMessageattributes above are documentary today, because the JD analyzers are loaded by no project in this repository. ADR-0061 asks whether that should change.The argument, in short: a rule is a claim about the library, and its unit suite proves only that it fires on a snippet its own author wrote for it — so a wrong model of the library passes both sides of that test. Every model error listed below did pass its unit suite. What caught them was realistic code, and the hand-run sweep that reads it depends on whoever is working remembering to run it.
Measured before the record was written, not assumed. Loading the analyzers into
JustDummies.UnitTests: the build succeeds, exactly two diagnostics are reported (the two duplicate-collapsing tests), the five suppressions added in this branch do silence their rules, and two more attributes bring the surface to zero. The other JustDummies-consuming projects report nothing. A cold non-incremental build of that project moved from ~6.6 s to ~9.3 s.Three alternatives are argued and declined — keeping the sweep manual, a
JustDummies.Usagesample mirroringFirstClassErrors.Usage, and an advisory CI job on the ADR-0046 model. The risks are stated rather than smoothed over: this verifies the loud rules and leaves the threeInfoones and the two opt-in ones unexercised, which the follow-up actions carry.Proposed only — nothing is wired in this PR, and I have not touched a status.
Dogfooding corrected the model twice in this wave
Neither error was visible from reading the source.
JD028 first claimed any sealed class without equality. That premise — every draw is a new instance — is false for a pool:
Any.OneOfhands back the very references it was given, soAny.SetOf(Any.OneOf(first, second))is legal and meaningful, andAnyOneOfTests.ReferenceIdentityKeepsDistinctInstancesDistinctasserts exactly that. The rule now fires only where the chain provably builds the value at the call site.JD027 had to stand down on a composer whose body is a
throw— and the first version of that guard did not work. An expression-bodied arrow-throw is aReturncarrying the throw, through a conversion to the composer's result type. Both facts had to be measured against the operation tree.Sweep
Across
JustDummies.UnitTests,JustDummies.PropertyTests,JustDummies.Xunit.UnitTests,FirstClassErrors.Testing.UnitTests,FirstClassErrors.UnitTests,FirstClassErrors.Usage,FirstClassErrors.RequestBinder.UnitTestsandFirstClassErrors.Cli.UnitTests: two hits, both JD025, both the duplicate-collapsing tests discussed above. A planted positive control fires all four.Testing
dotnet build FirstClassErrors.sln— 0 warnings, 0 errorsdotnet test FirstClassErrors.sln— 2 168 tests, 0 failures across all 13 test projectsFirstClassErrors.Analyzers.UnitTests, 132) — the rules added here live inJustDummies.Analyzers.UnitTests(246);JustDummies.UnitTestsgoes 632 → 637dotnet build JustDummies.UnitTests -c Release -f net472 -p:EnableNet472Floor=true— the floor leg the new tests have to survivedotnet format style --diagnostics IDE0028 --verify-no-changes— clean on all three touched projectsMutation testing was not run locally; the per-pull-request mutation checks are green on CI.
Documentation
doc/updated —JD025–JD028pages, the rule index, the new Composition section, and ADR-0061.fr.mdrule pages, the French index, and the ADR's.fr.mdArchitecture decisions
Proposed: ADR-0061ADR-0061, Run the JustDummies analyzers on the repository's own code, is drafted as
Proposedand indexed. No status flipped, no wiring done.The rules themselves embark no architectural decision: ADR-0059 covers the recipe-versus-value boundary, and adding a category is a routine consequence of adding rules. The harness change is an implementation improvement.
Remaining
This completes the six rules I could build alone. Two are left, and both need your decision: the unsupported-regex-literal rule (the
RegexParsergrammar would have to be shared with the analyzer through a linked source file) and theAny-outside-a-test-assembly rule (needs anIsTestProjectoracle, withFirstClassErrors.Testingas a live counterexample — a shipping package that legitimately references JustDummies).🤖 Generated with Claude Code
https://claude.ai/code/session_01GKqyhG9Y4AfdxEYekP2Evq