Skip to content

Testing: close the reproducibility adoption gap (xUnit adapter, Reproducibly in the consuming suites) #249

Description

@Reefact

Problem

Any.Reproducibly is the sole way to replay a failure caused by an arbitrary value, yet it appears nowhere in the repository's two large consuming suites (FirstClassErrors.UnitTests — 15 files drawing arbitrary values — and RequestBinder.UnitTests); it is exercised only by Dummies' and Testing's own contract tests. ADR-0006 flagged exactly this risk ("an ordinary unwrapped test that fails on an arbitrary value is still not replayable from its output alone") and proposed a test-framework adapter as a follow-up that was never built and is not carried forward by ADR-0026.

Impact

The flagship determinism property — "one reported seed replays the whole run" — holds by construction in the library but not in practice where the repository dogfoods hardest; a coincidental-value failure in the main suite is unreplayable today (except the self-reporting AnyGenerationException path).

Direction

  • Build the small xUnit adapter ADR-0006 anticipated (e.g. [ReproducibleFact]/[ReproducibleTheory] in Testing or a Dummies.Xunit companion, wiring ITestOutputHelper as the seed-report sink), and adopt it in the arbitrary-value-heavy suites.
  • Or, at minimum, wrap those test classes in Reproducibly.
  • Extend ErrorAssertion while in the area: the fluent checks the guide itself recommends (OfType<TError>(), transience/inner-error assertions) so the sample tests stop falling back to the Subject escape hatch.

Acceptance criteria

  • A failing arbitrary-value test in the main suites reports a seed that replays it; the mechanism is documented in DeterministicTesting/ArbitraryTestValues (EN+FR).

Context

Surfaced by the 2026-07-20 architecture, design & ecosystem audit (doc/handwritten/for-maintainers/audit/2026-07-20-firstclasserrors-architecture-and-design-audit.md, §12.5/§14-HV17; branch claude/firstclasserrors-audit-83gap7). Related: #216 (determinism contract).

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions