Skip to content

fix(report): fail closed on collapsed review evidence - #347

Merged
BenSheridanEdwards merged 4 commits into
mainfrom
fix/report-collapse-coherence
Aug 6, 2026
Merged

fix(report): fail closed on collapsed review evidence#347
BenSheridanEdwards merged 4 commits into
mainfrom
fix/report-collapse-coherence

Conversation

@BenSheridanEdwards

@BenSheridanEdwards BenSheridanEdwards commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Why does this feature exist?

StyleProof 4.7.5 can pair path-churned elements for report presentation. A late independent QA pass found a fail-open edge case: if correspondence pairs every removed/added node in a sibling reorder or nesting bijection, presentation becomes empty even though the raw certification differ still has reviewable evidence. The generated Markdown then says “All surfaces identical” and the standalone report CLI exits 0.

This must fail closed. Presentation is allowed to simplify certification evidence; it is not allowed to erase all of it and claim a clean comparison.

What changed?

  • Added a typed report-consistency assessment shared by Markdown, report.json, ReportResult, the report CLI, and the composite Action.
  • Emits presentation_collapsed_while_raw_reviewable when raw reviewable evidence remains but correspondence removes every presentation section.
  • Replaces the false identical headline with an explicit consistency-failure explanation.
  • Makes styleproof-report exit 1 for this contradiction.
  • Generates the Action report before trust classification, merges its coherence verdict into the machine payload, and maps the contradiction to unapprovable CERTIFICATION_FAILED.
  • Preserves raw findings, raw/reviewable counts, certification diff behavior, correspondence matching, and normal visual-approval behavior.
  • Added sibling-reorder, nesting-bijection, and CLI exit-code regressions.
  • Extracted consistency-failure summary rendering into a focused helper so the changed report path remains below Fallow's cognitive-complexity gate without changing output.
  • Pinned local Fallow to 3.14.0 and made pre-commit run the same dedicated production complexity gate as CI against the staged diff, in addition to the existing audit.
  • Added a hook contract test covering the exact Fallow version, staged-diff scope, baseline, production mode, and fail-on-issues behavior.
  • Updated README and changelog.

Behavioural Proof (with video and screenshots)

Not applicable — this repairs report/coherence control flow and CLI exit semantics; it does not change StyleProof's product UI or crop rendering.

Deterministic proof:

  • Before implementation, both new report regressions failed because reportConsistency was { ok: true, reason: "aligned" }.
  • After implementation, sibling reorder and nesting bijection both produce { ok: false, reason: "presentation_collapsed_while_raw_reviewable" }, the Markdown contains Report consistency failure, and it does not contain ✓ All surfaces identical.
  • The CLI regression proves styleproof-report exits 1 and never prints ✓ no changes.
  • Existing wrapper correspondence and truly identical-map tests remain green.

Verification Summary

  • npm test — 658/658 passed.
  • npm run typecheck — passed.
  • npm run lint — passed.
  • npm run format:check — passed.
  • npm run privacy:check — passed.
  • npm run demo:check — passed.
  • npm audit --audit-level=high — 0 vulnerabilities.
  • git diff --check — passed.
  • Fresh-context exact-head QA at 9d98ff5a9d63e4b7e3890f8c02226bfb860489b5 — PASS; 182/182 focused Action/report/CLI tests passed and live collapse reproduced as unapprovable CERTIFICATION_FAILED.
  • Fallow 3.14 exact PR-scope gate at 0c95f0c88bedde6512a4d1b761d45a31860c3c6e — 0 findings; summaryLines cognitive complexity reduced from 19 to 9.
  • Fresh-context QA of the Fallow remediation at 0c95f0c88bedde6512a4d1b761d45a31860c3c6e — PASS; report prose was byte-identical across all typed consistency cases and test/report.test.mjs passed 93/93.
  • Pre-commit negative probe — PASS; a staged synthetic high-complexity function was blocked with exit 1 before commit.
  • Pre-commit positive path — PASS; the real staged change passed audit plus the production health gate with 0 findings.
  • Final independent exact-head QA at 4369af018af9f7bc62a65bce4e8d049a728af284 — PASS; local and remote heads matched, report/Action fail-closed behavior passed, Fallow positive/negative hook probes passed, and all hosted checks were green on the same SHA.

PR Proof Law

  • I followed .agents/definition_of_done.md and the proof workflow.
  • Visual proof is not applicable because no rendered UI or crop pixels change.
  • Behaviour is proved by exact regression fixtures and CLI execution.
  • The PR body contains no local paths, private identifiers, or placeholders.
  • Raw certification semantics remain fail-closed and unchanged.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

🗺️ StyleProof report

📊 View the side-by-side visual report →


Coverage, determinism, or report/diff consistency evidence is incomplete — repair the capture or reflow source; visual approval cannot clear this failure.

@BenSheridanEdwards
BenSheridanEdwards merged commit 9447d35 into main Aug 6, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant