Skip to content

Document visual spot-check on regenerated binary artifacts #466

@braboj

Description

@braboj

As a contributor committing regenerated binary artifacts (PNGs, SVGs, screenshots, diagrams), I want documented guidance to spot-check the visual output before committing so that a buggy regenerator producing garbage that still tests-pass doesn't silently land on main.

What

When a regenerator produces binary artifacts (overlay PNGs, diagram SVGs, screenshot snapshots, plot images), `git diff` reports only "Binary files differ" — opaque to reviewers and to the author. The git diff itself proves nothing about correctness.

Pattern to document: open each regenerated binary in an image viewer (or the agent's image-read tool, for AI-assisted workflows) before staging. Confirm:

  • Geometry tracks the source data cleanly (overlays trace their underlying curves; diagrams match their schema; screenshots show the expected UI state)
  • No obvious corruption (truncated images, empty plots, garbage gradients)
  • Output is consistent with sibling artifacts (e.g. all 3 refreshed overlays in the same campaign should look similar in quality)

Cheap check: 1-2 minutes per artifact. Catches the failure mode where a regenerator bug produces invalid-but-syntactically-valid output that tests don't detect.

Why

Observed in me-fuji session 146: two PNG refreshes from the regenerator (TTartisan 50/1.2 max + 7artisans 50/1.2 II + new Fuji XF 23 15lp pair). All tests passed. Without the visual check, a regenerator bug producing geometrically-wrong polylines would have committed cleanly and only been caught later via manual inspection of the live chart. The visual check confirmed all 3 traced their source curves correctly before the commit.

Acceptance criteria

  • Section added to `base/workflow/ai-workflow.md` Lessons Learned (or similar)
  • Acknowledges that binary diffs are opaque to git diff and tests can be incomplete coverage
  • Recommends agent-assisted workflows use image-read tools (or open in viewer for human contributors)
  • Distinguishes from the "regenerate output before formatting" rule already in `base/core/docs.md` §"Generated files"

Where it could live

  • `base/workflow/ai-workflow.md` §"Lessons Learned" — "Verify agent calculations against the system" already covers similar territory for text output; the binary-artifact case is parallel
  • `base/core/quality.md` near the debug-statements rule
  • `base/git.md` regenerate-and-commit subsection (sibling to issue Document CRLF/whitespace filter before staging regenerated artifacts #465 if accepted there)

Flagged from me-fuji session 146 wrap. Sibling issue #465 covers the CRLF/whitespace filter step from the same workflow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — nice to havetaskAtomic implementable work

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions