Problem
(1) AnalyzerTestHarness returns only analyzer diagnostics and never asserts the snippet itself compiles: a core API rename would break the snippets, silence the analyzer, and every Does_not_report test would pass vacuously — masking regressions across all ~59 tests. (2) Assertions are mostly count+ID: no test pins a diagnostic's reported span or formatted message arguments (the part users see in the IDE), and FCE001's additionalLocations are untested. (3) The currently-perfect consistency between Descriptors.cs, the analyzer README table, and the 36 per-rule doc pages is maintained by hand — no test iterates DiagnosticIds asserting an EN and FR page exists per rule with matching severity/default, and nothing guards the hard-coded HelpLinks URLs.
Impact
The trust layer of the analyzer suite: today a vacuous pass, a location regression, or a doc-table drift would all ship silently.
Direction
- In the harness, fail any test whose compilation has Error-severity compiler diagnostics (one guard closes the hole for all tests).
- For each rule's primary positive test, assert the mapped span and
GetMessage() content.
- Add one reflection fixture: every
DiagnosticIds constant has exactly one analyzer, an analyzers/{id}.en.md + .fr.md page at the HelpLinks-derived path, and a README table row matching the descriptor's severity/enabled-by-default.
Acceptance criteria
- A non-compiling snippet fails its test; a moved diagnostic location fails its test; deleting a doc page or drifting a README severity fails the consistency fixture.
Context
Surfaced by the 2026-07-20 architecture, design & ecosystem audit (doc/handwritten/for-maintainers/audit/2026-07-20-firstclasserrors-architecture-and-design-audit.md, §8.2/§14-HV6; branch claude/firstclasserrors-audit-83gap7).
Problem
(1)
AnalyzerTestHarnessreturns only analyzer diagnostics and never asserts the snippet itself compiles: a core API rename would break the snippets, silence the analyzer, and everyDoes_not_reporttest would pass vacuously — masking regressions across all ~59 tests. (2) Assertions are mostly count+ID: no test pins a diagnostic's reported span or formatted message arguments (the part users see in the IDE), and FCE001'sadditionalLocationsare untested. (3) The currently-perfect consistency betweenDescriptors.cs, the analyzer README table, and the 36 per-rule doc pages is maintained by hand — no test iteratesDiagnosticIdsasserting an EN and FR page exists per rule with matching severity/default, and nothing guards the hard-codedHelpLinksURLs.Impact
The trust layer of the analyzer suite: today a vacuous pass, a location regression, or a doc-table drift would all ship silently.
Direction
GetMessage()content.DiagnosticIdsconstant has exactly one analyzer, ananalyzers/{id}.en.md+.fr.mdpage at the HelpLinks-derived path, and a README table row matching the descriptor's severity/enabled-by-default.Acceptance criteria
Context
Surfaced by the 2026-07-20 architecture, design & ecosystem audit (
doc/handwritten/for-maintainers/audit/2026-07-20-firstclasserrors-architecture-and-design-audit.md, §8.2/§14-HV6; branchclaude/firstclasserrors-audit-83gap7).