Skip to content

TDD bug-fix check in the review checklist (red-test-before-fix) #283

Description

@rucka

Story Statement

As a reviewer
I want a TDD bug-fix check in the review checklist — verifying a failing test was written before the fix on bug-fix PRs
So that bug fixes prove they actually address the bug, not just appear to (R7.4)

Where: the code-review checklist — extends the code-review template (#228) and the review process; no obligation on feature PRs

Epic Context

Parent Epic: Code Quality & Testing foundations #209
Status: Refined
Priority: P1 (Should-Have)

Status Workflow

  • Refined: Story is detailed, estimated, and ready for development
  • In Progress: Story is actively being developed
  • Done: Story delivered and accepted

Acceptance Criteria

Functional Requirements

Given-When-Then Format:

  1. Given a PR classified as a bug fix
    When it is reviewed
    Then the review checklist includes a "red test before fix" item — the reviewer verifies a test reproducing the bug exists and was added as (or before) the fix, per CLAUDE.md's test-first bug-fix rule (R7.4)

  2. Given a bug-fix PR with no reproducing test
    When the checklist item is evaluated
    Then it is flagged as a review finding (the fix hasn't proven it addresses the bug) — surfaced in the review verdict, not silently passed

  3. Given a feature PR (not a bug fix)
    When it is reviewed
    Then the TDD bug-fix check does not apply — no test-first obligation on features (R7.4 explicitly: "no obligation for features")

  4. Given the review checklist item
    When the checklist is rendered
    Then it follows the verdict-first / reading-budget structure the extended review template (Extended code-review template (security + cost, reading budget) #228) already establishes — 1-line verdict, detail collapsed — consistent with the security/cost/architecture lines

Business Rules

  • The check lives in the review checklist (R7.4: "verified in review checklist") — it extends the code-review template (Extended code-review template (security + cost, reading budget) #228) and the review process, not a new skill or automated gate
  • Bug-fix-only: the check applies when the PR is a bug fix (by label/classification/story type); features are exempt (R7.4)
  • Verification is the reviewer's judgment aided by the checklist prompt (does a reproducing test exist, and does the diff show it failing-before / passing-after?) — this story does not build an automated commit-order analyzer (that would be a separate, heavier gate; explicitly out of scope here)
  • Test stack is team choice (R7.4) — the check asks "is there a reproducing test", not "is it written in framework X"

Edge Cases and Error Handling

  • PR is a mix of bug fix + feature: the check applies to the bug-fix portion — the reviewer confirms the bug-fix part has a reproducing test, the feature part is exempt
  • Bug genuinely untestable via automated test (e.g. a doc typo fix, a pure-formatting fix classified as a "fix"): the checklist item allows an explicit reviewer-recorded rationale for why no automated test applies, rather than a hard block — the point is proving the fix works, and for some "fixes" a test isn't the proof
  • PR classification ambiguous (is it a bug fix or a feature?): the checklist prompts the reviewer to decide, defaulting to applying the check if the PR references a bug/defect

Definition of Done Checklist

Development Completion

  • All acceptance criteria implemented and verified
  • Code-review template (Extended code-review template (security + cost, reading budget) #228) extended with the TDD bug-fix checklist item (verdict-first, bug-fix-conditional)
  • Review process / pair-process-review reflects the check where it drives the checklist
  • Code review completed and approved
  • Documentation updated — docs site (review process page)

Quality Assurance

  • Dry-run: a bug-fix PR with a reproducing test passes the item; one without is flagged; a feature PR correctly skips the item

Story Sizing and Sprint Readiness

Refined Story Points

Final Story Points: 2 (S)
Confidence Level: High
Sizing Justification: a checklist-item extension to the review template (#228) + the review process — no new skill, no automated analyzer; the work is defining the item precisely and its bug-fix-conditional/feature-exempt logic

Sprint Capacity Validation

Sprint Fit Assessment: yes, single sprint
Total Effort Assessment: fits — Yes

Dependencies and Coordination

Story Dependencies

Prerequisite Stories: #228 (extended code-review template — this adds one more checklist item to it); coordinates with pair-process-review (the process that runs the checklist)
Dependent Stories: none

Validation and Testing Strategy

Acceptance Testing Approach

Testing Methods: hand-traced review dry-run against three PR fixtures — a bug fix with a reproducing test (passes), a bug fix without one (flagged), a feature PR (item skipped) — matching this session's fixture-dry-run pattern for review/checklist content
Test Data Requirements: three fixture PRs (bug-fix-with-test, bug-fix-without-test, feature)

Notes and Additional Context

Refinement Session Insights: 2026-07-11 — found during the epic-completeness audit as a planned R7.4 story (epic AC4) that was never created. Confirmed it lives in the review checklist (extending #228), not as a new skill or an automated commit-order gate — R7.4 itself says "verified in review checklist". CLAUDE.md already states test-first bug fixing as policy; this story makes it a verified review step rather than just policy.
Documentation Links: Requirements R7.4 · Spec G8 · epic #209 AC4 · CLAUDE.md Bug Resolution Workflow · pair-requirements-triage.md

Technical Analysis

Implementation Approach

Technical Strategy: add a bug-fix-conditional "red test before fix" item to the code-review template (#228), following its verdict-first/reading-budget structure; update the review process (pair-process-review) to evaluate it on bug-fix PRs and skip it on features. Reviewer-judgment based (aided by the checklist prompt), not an automated commit-order analyzer.
Key Components: code-review template (#228) checklist item; pair-process-review checklist evaluation (bug-fix conditional)
Integration Points: #228 (the template this extends); pair-process-review (runs the checklist); PR classification (bug fix vs. feature — from label/story type)

Technical Requirements

Technical Risks and Mitigation

Risk Impact Probability Mitigation Strategy
Reviewers rubber-stamp the item without actually checking for a reproducing test Medium Medium The checklist prompt asks for the specific evidence (which test reproduces the bug, failing-before/passing-after) rather than a yes/no, making a rubber-stamp visibly empty
PR bug-fix-vs-feature classification is unreliable Low Medium Default to applying the check when the PR references a bug/defect; the checklist lets the reviewer record the classification decision

Task Breakdown

Checklist

  • T1 — Define the checklist item (bug-fix-conditional, evidence prompt, feature-exempt, untestable-fix rationale escape hatch)
  • T2 — Add it to the code-review template (Extended code-review template (security + cost, reading budget) #228, verdict-first structure) and wire its evaluation into pair-process-review
  • T3 — Review dry-run against the 3 PR fixtures (bug-fix-with-test / without / feature) + docs site update

Dependency Graph

T1 → T2 → T3

AC Coverage

Task AC1 AC2 AC3 AC4
T1
T2
T3

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions