fix(lean): output-discipline for Opus lean re-ground + duo cold-open craft - #685
Conversation
Opus routine beats on lean-ON intermittently degraded craft (VM 2026-06-06, run vm2-opushi-narr):
turn 2 leaked planning ('I'll play that as a held beat...'), turn 4 produced a stage-direction
summary ('The scene is resolved and persisted...') instead of narration. The SKILL.md has all the
anti-scaffolding/anti-summary rules, but the lean re-ground directive focused entirely on STATE
re-grounding and never restated the OUTPUT discipline, so the fresh lean session shortcut into
bookkeeping. Add a tight output-discipline clause naming the exact observed failure modes
(planning note, 3rd-person summary, summarized-NPC, reworded-reply double-show).
The duo cold-open prompt lists setup steps + mentions 'the dashboard', so the DM narrated its
process ('State is grounded and the cold open is on the dashboard. Closing my turn...') before the
prose — a scaffolding leak that drags the Tolkien lens score (vm2 duo scored 4.0 with the leak).
Production (play_party.sh) uses a cleaner cold-open prompt and was clean. Add the same OUTPUT
DISCIPLINE clause so the duo measures production-representative craft.
|
Need the big picture first? Review this PR in Change Stack to see what changed before going file by file. Warning Review limit reached
More reviews will be available in 5 minutes and 51 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR tightens DM output discipline across the test suite by appending "OUTPUT DISCIPLINE" instruction blocks to two DM prompts. The lean DM system prompt now explicitly constrains responses to 2nd-person in-fiction prose and quoted dialogue, while the opening scene prompt adds PC seating and companion meeting rules alongside similar output formatting constraints. ChangesDM Output Discipline Constraints
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
… budgets + lean-ON) For the scheduled Tue-4am full sweep on the credit refresh: per-persona timeout 1500->2400s + run budget $12->$18 (Opus cold-open ~$2.4 + slower beats); duo timeout 2700->3600s + budget $2->$5. Flip lean ON (was a stale 'intentionally OFF' from the 2026-06-05 finding; #683 fixed the cross-campaign contamination + #685 the output-discipline) — lean-ON matches production + gives fast Opus beats so the sweep completes without latency give-ups/timeouts (the wasted-run vector).
…ration finding (#687) * chore(score): make the scorer model env-overridable (default sonnet, gate-constant) Adds CLAWDND_SCORER_MODEL (default sonnet) so the gate scorer stays constant by default, but a deliberate scorer-calibration probe can re-score with a stronger judge. Motivated by the analytical-Sonnet (story 4.0) vs lived-persona (~4.5 'best AI fiction') gap on Opus craft. * fix(score_openclaw): default agent=main, no forced model override, fresh session, state cap The gateway scorer was broken on the VM: default agent clawdnd-qa isn't configured (main is, model gpt-5.5), --model openai/gpt-5.4 is REJECTED for main, and the single --message argv tripped E2BIG (state.json ~137KB > MAX_ARG_STRLEN ~128KB). Fixes: default agent=main, pass --model only when explicitly set+allowed, fresh --session-id per run (no main-session pollution), and cap the state (CLAWDND_SCORER_STATE_CAP=75000; transcript carries the prose). Enables the GPT-5.5 3rd judge + the credit-saving gateway scorer the owner asked for. * chore(sweep): Opus de-risk for the Tuesday refresh sweep (durations + budgets + lean-ON) For the scheduled Tue-4am full sweep on the credit refresh: per-persona timeout 1500->2400s + run budget $12->$18 (Opus cold-open ~$2.4 + slower beats); duo timeout 2700->3600s + budget $2->$5. Flip lean ON (was a stale 'intentionally OFF' from the 2026-06-05 finding; #683 fixed the cross-campaign contamination + #685 the output-discipline) — lean-ON matches production + gives fast Opus beats so the sweep completes without latency give-ups/timeouts (the wasted-run vector). --------- Co-authored-by: Eva <arncalso@gmail.com>
After #684 made the Opus cold-open work, a GUI narrative validation (run vm2-opushi-narr) showed Opus quality is exceptional ("among the best AI collaborative fiction I've encountered") and real routine-beat latency is fine (1-2 min — the "5-15 min" the persona felt is the QA player-agent's own deliberation). The remaining issue was intermittent craft degradation on lean routine beats: a planning-leak preamble and a stage-direction summary instead of prose.
Root: the SKILL.md has all the anti-scaffolding/anti-summary rules, but the lean re-ground directive focused entirely on STATE re-grounding and never restated the OUTPUT discipline — so the fresh lean session re-grounded correctly but narrated loosely.
Fixes
qa/lib_beat_driver.sh(shared across all harnesses): a tight OUTPUT DISCIPLINE clause in the lean re-ground directive naming the exact observed failures — no planning note, no 3rd-person summary, the addressed NPC speaks a quoted line, narrate once (same prose streamed + replied).qa/run_duo.sh: same discipline on the duo cold-open prompt (it listed setup steps + said "dashboard", so the DM narrated its process — a duo-scoring artifact; production play_party.sh was clean).Validation
Opus+lean duo (leandirfix): lean beats 2-5 all clean rich prose+dialogue (the fix works at the beat level). Tolkien lens 4.0 was dragged by the duo cold-open leak (now fixed) + single-run noise; a clean-cold-open multi-run mean is the next measure. No production-path behavior change for non-lean turns.
Summary by CodeRabbit
Release Notes