Skip to content

Add 'Probe-first when issue body hypothesizes a mechanism' to ai-workflow.md #468

@braboj

Description

@braboj

Pattern from wuseria S147 (#1122) worth pinning to upstream templates.

Pattern

When an issue body names a specific code surface and a hypothesized mechanism for a bug, the temptation is to code the fix matching those hypotheses. A targeted probe — a throwaway script that dumps the relevant runtime state around the symptom — answers two questions in one run:

  1. Is the hypothesis correct?
  2. Where is the real signal?

Frequent outcome: the probe inverts the framing and the real bug is upstream of the hypothesized code surface.

Concrete example

wuseria/me-fuji#1122 hypothesized "per-column DP mis-tracks the ridge despite black pixels being available; instrument ridge.ridge_tracks_for_hue_freq_split." A ~20-minute probe disproved the framing: at the symptom column, there ARE no separate ridge candidates (the curves are physically coincident), the DP correctly coasts, and the dive originates from a spurious chrome point upstream of the DP that corrupts a downstream _densify_track interpolation. The fix landed in a different file (_column_runs) than the issue text predicted.

Without the probe, a fix would have landed in ridge_tracks_for_hue_freq_split for a non-existent problem.

Why this belongs upstream

templates/base/workflow/ai-workflow.md already has a "Lessons Learned" section with the related "Verify agent calculations against the system" pattern (don't trust the agent's mental math, check the build output). The probe-first rule is the same shape one level deeper: don't trust the issue body's hypothesis, check the runtime data.

Distinct from `templates/base/core/quality.md` §"Probe scripts," which defines what a probe script IS and how to manage it. This rule says WHEN to write one.

Proposed wording

Add to templates/base/workflow/ai-workflow.md Lessons Learned section:

Probe before trusting the issue body's mechanism

When an issue body names a specific code surface and a hypothesized cause, write a throwaway probe targeting that surface — and the symptom — BEFORE writing the fix. The probe answers two questions in one run: is the hypothesis correct, and where is the real signal? Frequent outcome: the probe inverts the framing and the real bug is in a different region of the code. The fifteen minutes spent probing is the cheapest insurance against fixing a non-existent problem.

Related candidates from same session

  • "Size code thresholds from in-codebase distributions, not first principles" — separate issue.
  • "Spot-check + global metric for bulk regeneration after a shared-path fix" — separate issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — nice to havespikeResearch or exploration

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions