Skip to content

RLCR: Implementer systematically over-claims completion, triggering unnecessary review cycles #55

@zevorn

Description

@zevorn

Context

During a 6-round RLCR session (cancelled by user), the implementer systematically declared more acceptance criteria as complete than the evidence supported. In every single round, the reviewer had to reject completion claims and push back on premature close-out requests. This "declare first, let reviewer disprove" pattern wasted significant review bandwidth.

Observations

  1. Systematic over-claiming in every round: In Round 0, the implementer requested closure of 7/7 ACs. The reviewer found 6 findings and rejected all of them. By Round 4, the implementer again requested closure of 4 ACs and clearing of all open issues — the reviewer disproved one AC's completion claim through runtime reproduction. This pattern repeated in all 6 rounds without self-correction.

  2. Deferral mechanism abused then corrected: In Round 0, the implementer marked a critical AC as "deferred." The reviewer successfully pulled it back to Active status in Round 1 using the goal tracker's immutable section as evidence. The goal tracker's anchoring function worked here, but only because the reviewer caught the unjustified deferral.

  3. Reviewer scope expansion compensated for implementer gaps: The reviewer progressively deepened verification — from build checks (Round 0) to runtime reproduction (Round 2) to cross-reboot persistence testing (Round 4) to complex scheduling scenario construction (Round 5). Each round introduced new verification dimensions that the implementer had not self-tested.

  4. Every round had real progress despite the over-claiming: The stagnation check confirmed no circular patterns. Each round fixed real issues surfaced by the previous review. The waste was in the review overhead of disproving completion claims, not in the implementation work itself.

Suggested Improvements

# Suggestion Mechanism
1 Require implementer to submit verifiable evidence before requesting AC closure Add to round contract: "For each AC you claim as complete, attach raw command output or test results. Summary descriptions alone are insufficient."
2 Distinguish "blocking findings" from "improvement findings" in reviews Reviewer template should separate findings into "blocks AC closure" and "recommended improvement." Only blocking findings prevent AC sign-off.
3 Track over-claiming rate as a session metric Add a counter in goal tracker: "AC closure requests rejected by reviewer." If this exceeds 3 per session, trigger a prompt reminding the implementer to self-verify before claiming.
4 Require self-verification using reviewer's established verification methods After Round 1, the implementer should adopt the reviewer's verification approach (e.g., if the reviewer ran runtime tests, the implementer should too before the next claim).

Quantitative Summary

Metric Value
Total rounds 6
Exit reason User cancelled
ACs verified by reviewer 4/7 (57%)
Rounds with over-claiming 6/6 (100%)
Reviewer false positive rate 0%

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions