Skip to content

fix(lean): output-discipline for Opus lean re-ground + duo cold-open craft - #685

Merged
100yenadmin merged 3 commits into
mainfrom
fix/lean-reground-output-discipline
Jun 6, 2026
Merged

fix(lean): output-discipline for Opus lean re-ground + duo cold-open craft#685
100yenadmin merged 3 commits into
mainfrom
fix/lean-reground-output-discipline

Conversation

@100yenadmin

@100yenadmin 100yenadmin commented Jun 6, 2026

Copy link
Copy Markdown
Member

After #684 made the Opus cold-open work, a GUI narrative validation (run vm2-opushi-narr) showed Opus quality is exceptional ("among the best AI collaborative fiction I've encountered") and real routine-beat latency is fine (1-2 min — the "5-15 min" the persona felt is the QA player-agent's own deliberation). The remaining issue was intermittent craft degradation on lean routine beats: a planning-leak preamble and a stage-direction summary instead of prose.

Root: the SKILL.md has all the anti-scaffolding/anti-summary rules, but the lean re-ground directive focused entirely on STATE re-grounding and never restated the OUTPUT discipline — so the fresh lean session re-grounded correctly but narrated loosely.

Fixes

  • qa/lib_beat_driver.sh (shared across all harnesses): a tight OUTPUT DISCIPLINE clause in the lean re-ground directive naming the exact observed failures — no planning note, no 3rd-person summary, the addressed NPC speaks a quoted line, narrate once (same prose streamed + replied).
  • qa/run_duo.sh: same discipline on the duo cold-open prompt (it listed setup steps + said "dashboard", so the DM narrated its process — a duo-scoring artifact; production play_party.sh was clean).

Validation

Opus+lean duo (leandirfix): lean beats 2-5 all clean rich prose+dialogue (the fix works at the beat level). Tolkien lens 4.0 was dragged by the duo cold-open leak (now fixed) + single-run noise; a clean-cold-open multi-run mean is the next measure. No production-path behavior change for non-lean turns.

Summary by CodeRabbit

Release Notes

  • Improvements
    • Strengthened dungeon master response formatting for improved narrative consistency.
    • Enhanced character introduction and scene setup handling.
    • Refined dialogue and prose formatting standards in game narration.

Eva added 2 commits June 6, 2026 11:10
Opus routine beats on lean-ON intermittently degraded craft (VM 2026-06-06, run vm2-opushi-narr):
turn 2 leaked planning ('I'll play that as a held beat...'), turn 4 produced a stage-direction
summary ('The scene is resolved and persisted...') instead of narration. The SKILL.md has all the
anti-scaffolding/anti-summary rules, but the lean re-ground directive focused entirely on STATE
re-grounding and never restated the OUTPUT discipline, so the fresh lean session shortcut into
bookkeeping. Add a tight output-discipline clause naming the exact observed failure modes
(planning note, 3rd-person summary, summarized-NPC, reworded-reply double-show).
The duo cold-open prompt lists setup steps + mentions 'the dashboard', so the DM narrated its
process ('State is grounded and the cold open is on the dashboard. Closing my turn...') before the
prose — a scaffolding leak that drags the Tolkien lens score (vm2 duo scored 4.0 with the leak).
Production (play_party.sh) uses a cleaner cold-open prompt and was clean. Add the same OUTPUT
DISCIPLINE clause so the duo measures production-representative craft.
@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Need the big picture first? Review this PR in Change Stack to see what changed before going file by file.

Review Change Stack

Warning

Review limit reached

@100yenadmin, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 5 minutes and 51 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6e4e64e6-3412-4c9e-aa07-3038fda01648

📥 Commits

Reviewing files that changed from the base of the PR and between 3b4dc5b and f13850a.

📒 Files selected for processing (1)
  • servers/engine/tests/test_lean_output_discipline.py
📝 Walkthrough

Walkthrough

This PR tightens DM output discipline across the test suite by appending "OUTPUT DISCIPLINE" instruction blocks to two DM prompts. The lean DM system prompt now explicitly constrains responses to 2nd-person in-fiction prose and quoted dialogue, while the opening scene prompt adds PC seating and companion meeting rules alongside similar output formatting constraints.

Changes

DM Output Discipline Constraints

Layer / File(s) Summary
System prompt and opening scene output discipline
qa/lib_beat_driver.sh, qa/run_duo.sh
The lean DM system prompt extends clawdnd_dm_lean_args with an "OUTPUT DISCIPLINE" section banning meta-narration, stat leaks, and 3rd-person director voice. The opening scene prompt (D1) in run_duo.sh adds similar output discipline rules plus explicit constraints: PC must be a living canon character via load_canon_character(..., kind="player", add_to_party=true), companion introduction only through on-screen meeting during the opening, and the opening reply must begin inside the fiction without setup narration.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

  • electricsheephq/WorldOS#412: Both PRs modify the cold-open Dungeon Master instructions to enforce stricter "solo starts" behavior around companion introduction/recruitment (main PR adds companion/output-discipline constraints; retrieved PR removes auto-recruiting so the player begins alone).
  • electricsheephq/WorldOS#647: Both PRs tighten the DM prompt's output discipline to prevent the model from echoing internal planning/runbook scaffolding and instead render lived 2nd-person prose with quoted dialogue only.

Poem

🐰 A prompt so tight, no leaks of plan,
Just fiction-speak from our DM friend,
No stats, no meta, just the van-guard stand,
Where dialogue flows 'til the opening's end.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description covers the core sections well: problem context, root cause analysis, specific fixes to two files, and validation results. However, the CLA checklist from the template is completely missing. Add the required CLA checklist (three items about contributor license agreement, confidential information, and third-party material) to comply with the repository template.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main changes: adding output-discipline constraints to both the Opus lean re-ground system and the duo cold-open prompt to fix craft degradation.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@100yenadmin
100yenadmin merged commit fb93e7e into main Jun 6, 2026
14 checks passed
@100yenadmin
100yenadmin deleted the fix/lean-reground-output-discipline branch June 6, 2026 05:09
100yenadmin pushed a commit that referenced this pull request Jun 6, 2026
… budgets + lean-ON)

For the scheduled Tue-4am full sweep on the credit refresh: per-persona timeout 1500->2400s + run
budget $12->$18 (Opus cold-open ~$2.4 + slower beats); duo timeout 2700->3600s + budget $2->$5.
Flip lean ON (was a stale 'intentionally OFF' from the 2026-06-05 finding; #683 fixed the
cross-campaign contamination + #685 the output-discipline) — lean-ON matches production + gives fast
Opus beats so the sweep completes without latency give-ups/timeouts (the wasted-run vector).
100yenadmin added a commit that referenced this pull request Jun 6, 2026
…ration finding (#687)

* chore(score): make the scorer model env-overridable (default sonnet, gate-constant)

Adds CLAWDND_SCORER_MODEL (default sonnet) so the gate scorer stays constant by default, but a
deliberate scorer-calibration probe can re-score with a stronger judge. Motivated by the
analytical-Sonnet (story 4.0) vs lived-persona (~4.5 'best AI fiction') gap on Opus craft.

* fix(score_openclaw): default agent=main, no forced model override, fresh session, state cap

The gateway scorer was broken on the VM: default agent clawdnd-qa isn't configured (main is, model
gpt-5.5), --model openai/gpt-5.4 is REJECTED for main, and the single --message argv tripped E2BIG
(state.json ~137KB > MAX_ARG_STRLEN ~128KB). Fixes: default agent=main, pass --model only when
explicitly set+allowed, fresh --session-id per run (no main-session pollution), and cap the state
(CLAWDND_SCORER_STATE_CAP=75000; transcript carries the prose). Enables the GPT-5.5 3rd judge + the
credit-saving gateway scorer the owner asked for.

* chore(sweep): Opus de-risk for the Tuesday refresh sweep (durations + budgets + lean-ON)

For the scheduled Tue-4am full sweep on the credit refresh: per-persona timeout 1500->2400s + run
budget $12->$18 (Opus cold-open ~$2.4 + slower beats); duo timeout 2700->3600s + budget $2->$5.
Flip lean ON (was a stale 'intentionally OFF' from the 2026-06-05 finding; #683 fixed the
cross-campaign contamination + #685 the output-discipline) — lean-ON matches production + gives fast
Opus beats so the sweep completes without latency give-ups/timeouts (the wasted-run vector).

---------

Co-authored-by: Eva <arncalso@gmail.com>
100yenadmin added a commit that referenced this pull request Jun 10, 2026
…ed; lean is now production+QA default) (#767)

Co-authored-by: Eva <arncalso@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant