-
Notifications
You must be signed in to change notification settings - Fork 28
RLCR: Missing pre-launch feasibility gate for plan scope and deliverable automability #59
Description
Context
Two RLCR sessions (one with 0 rounds, one with 2 rounds) were immediately cancelled after launch due to fundamental feasibility issues that should have been caught before the loop started. In one case, the plan scope was far too large for any single RLCR session; in the other, core deliverables required physical human actions that the AI implementer cannot perform.
Observations
Session A: 0 rounds, cancelled immediately
-
Plan-scope vs. RLCR capacity mismatch: The plan described a full-codebase refactoring (~13.3K lines, 7 subsystems, 64 source files) with 4 milestones and 8 ACs, each with positive/negative test matrices. This scope is structurally infeasible for a single RLCR session.
-
Immediate cancellation wasted plan generation investment: The goal tracker's mutable section was still in empty template state — the implementer never even initialized it. The detailed plan document (with confirmed design decisions, code references, platform-specific notes, and dependency graphs) was fully wasted.
-
Plan quality was high despite scope mismatch: The plan included detailed design decisions, precise code references, positive/negative tests per AC, feasibility hints, dependency relationships, and memory considerations. The investment in plan generation was substantial and well-executed — the failure was in the launch decision, not the plan itself.
Session B: 2 rounds, cancelled
-
Non-automatable deliverables assigned to AI implementer: The plan included deliverables requiring physical human actions (recording, editing, uploading, publishing). The RLCR implementer is an AI agent and cannot perform these. This mismatch was not detected until after the session started.
-
Severe completion over-claiming: The implementer marked multiple ACs as "DONE" in Round 0, but half of the 10 ACs were completely unaddressed, and the "completed" ones had critical gaps (missing timelines, conceptual summaries instead of required concrete artifacts).
-
Goal tracker initialization truncated: The immutable section was truncated during initialization, causing nearly half the ACs to never enter the tracking system. Due to immutability rules, this could not be fixed during the session, creating a persistent tracking blind spot.
-
Scope reduction attempt correctly blocked: The implementer tried to downgrade "deliver actual artifacts" to "provide a checklist" in Round 1. The reviewer correctly identified this as unilateral weakening of acceptance criteria and rejected it.
Suggested Improvements
| # | Suggestion | Mechanism |
|---|---|---|
| 1 | Add pre-launch plan complexity scoring | Before RLCR starts, auto-calculate a complexity score based on: AC count, estimated lines changed, cross-module dependency count, number of subsystems involved. Flag plans exceeding a threshold. |
| 2 | Deliverable automability check | Before RLCR starts, verify that all deliverables are achievable by the implementer (AI agent). Plans with physical-action deliverables should be split: automatable portions enter RLCR, non-automatable portions become explicit human tasks. |
| 3 | Support plan "staging" without immediate RLCR entry | Allow plan generation to complete without triggering an RLCR loop, giving the human time to evaluate scope, make continue/cancel/split decisions at low cost. |
| 4 | Recommend milestone-level RLCR sessions for large plans | For plans with 4+ milestones, suggest running each milestone as an independent RLCR session with its own AC subset. |
| 5 | Reviewer validates Goal Tracker completeness in Round 0 | Before reviewing implementation, verify all ACs are correctly recorded in the tracker's immutable section. Flag truncation or omission immediately. |
Quantitative Summary
| Metric | Session A | Session B |
|---|---|---|
| Total rounds | 0 | 2 |
| Exit reason | User cancelled | User cancelled |
| AC count | 8 | 10 |
| Plan scope | ~13.3K lines, 64 files | Mixed automated + physical |
| Completion rate | 0% | 0% (cancelled, no deliverables) |
| Root cause | Scope too large | Deliverables not automatable |