fix: make the Opus DM default work end-to-end (per-turn budget + cold-open latency) - #684
Merged
Merged
Conversation
…max_budget_usd) #682 flipped the default DM model to Opus but left the per-turn budget cap at the Sonnet-tuned $1.50. The Opus max-effort cold-open world-build costs ~5x a Sonnet turn, so the very first (cold-open) DM turn tripped error_max_budget_usd and the faithful backend never seated a player character (VM 2026-06-06: failure_bucket=no_actor, spend $0). - scripts/play.sh + scripts/play_party.sh: model-aware per-turn + session budget DEFAULTS (opus -> $12 per-turn / $30 session; sonnet unchanged $1.50 / $15). Caps, not spends. - qa/ui_playtest_app.sh: the claude backend lane pinned CLAWDND_PLAY_BUDGET=$1.50 unconditionally, overriding the script default -> scale it to DM_MODEL too. - servers/engine/tests/test_dm_budget_scaling.py: Tier-0 guard (3 assertions). This is a release-blocker for the Opus default: every Opus cold-open (the production .app included) would otherwise fail to seat a PC.
|
Too much diff to scan? Review this PR in Change Stack to start with the highest-impact changes. Warning Review limit reached
More reviews will be available in 1 hour, 49 minutes, and 8 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThis PR makes DM budget, timeout, and polling defaults model-aware across shell scripts. Opus models now receive higher per-turn and session budgets, increased cold-open effort and timeout, and longer polling grace windows. Production play scripts and QA harness apply the new defaults; tests validate the branching logic and value constraints. ChangesModel-aware Opus defaults across shell scripts and tests
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
added 2 commits
June 6, 2026 08:10
…old-open is slow) The Opus max-effort cold-open seats the PC (~204s) long before the opening narration finishes (>400s), so the hardcoded ~2.5min grace proceeded into an empty scene. Make the player-ready poll count + the no-narration grace threshold env-configurable (WOS_APP_PLAYER_READY_POLLS / WOS_APP_NARRATION_GRACE_POLLS; defaults unchanged) so a slow Opus cold-open can be given time to write its narration before the harness grace-proceeds.
… narration-wait) The Opus --effort max cold-open world-build is generation-bound and overruns the cold-open timeout (VM 2026-06-06: max never finishes <400s, the turn is killed before it writes narration, the backend grace-proceeds into an empty scene). Measured: Opus --effort HIGH finishes ~300s WITH a full BG-caliber opening (Sorcerous Sundries / Ramazith's tower), cost $2.36, 1 clean campaign. - qa/lib_beat_driver.sh: cold-open effort model-aware (opus -> high, sonnet -> max); cold-open timeout model-aware (opus -> 500s margin, sonnet -> 400s). Routine tier unchanged (medium/200s). - qa/ui_playtest_app.sh: the part-B player-ready wait defaults longer for an Opus DM so the persona sees the real ~300s opening instead of grace-proceeding into an empty scene. - servers/engine/tests/test_opus_coldopen_tuning.py: Tier-0 guard (3 assertions). Together with the budget fix, this makes the Opus default cold-open actually work end-to-end.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
scripts/play_party.sh (1)
44-46:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUpdate header comments to reflect model-aware defaults.
Same issue as
scripts/play.sh: the comments still reference the old Sonnet-tuned defaults, but the implementation now scales based on the DM model.📝 Suggested documentation update
# Safety caps (a runaway loop self-stops; companions count toward the SAME session ceiling): -# CLAWDND_PLAY_BUDGET per-turn USD budget for one agent turn (default 1.50) -# CLAWDND_PLAY_SESSION_BUDGET aggregate USD ceiling for the session (default 15.00) +# CLAWDND_PLAY_BUDGET per-turn USD budget for one agent turn (default: model-aware; Opus $12, others $1.50) +# CLAWDND_PLAY_SESSION_BUDGET aggregate USD ceiling for the session (default: model-aware; Opus $30, others $15) # CLAWDND_PLAY_MAX_TURNS hard cap on agent turns (DM + companions)(default 40)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/play_party.sh` around lines 44 - 46, The header comments for the play_party.sh script still list the old Sonnet-tuned static defaults; update them to state that per-turn (CLAWDND_PLAY_BUDGET), session (CLAWDND_PLAY_SESSION_BUDGET) and max turns (CLAWDND_PLAY_MAX_TURNS) are model-aware and may scale based on the DM model rather than fixed Sonnet defaults—mention the dynamic scaling behavior and reference that the script computes budgets from the chosen DM model so readers know values are not constant.scripts/play.sh (1)
22-24:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUpdate header comments to reflect model-aware defaults.
The comments still reference the old Sonnet-tuned defaults (
(default 1.50)and(default 15.00)), but the implementation now scales these defaults based on the DM model. Consider updating these to something like(default: model-aware; see lines 47-59)or listing both tiers.📝 Suggested documentation update
# Safety caps (a runaway DM loop self-stops): -# CLAWDND_PLAY_BUDGET per-turn USD budget for one DM turn (default 1.50) -# CLAWDND_PLAY_SESSION_BUDGET aggregate USD ceiling for the session (default 15.00) +# CLAWDND_PLAY_BUDGET per-turn USD budget for one DM turn (default: model-aware; Opus $12, others $1.50) +# CLAWDND_PLAY_SESSION_BUDGET aggregate USD ceiling for the session (default: model-aware; Opus $30, others $15) # CLAWDND_PLAY_MAX_TURNS hard cap on DM turns (default 40)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@scripts/play.sh` around lines 22 - 24, Update the header comments for the three env vars CLAWDND_PLAY_BUDGET, CLAWDND_PLAY_SESSION_BUDGET, and CLAWDND_PLAY_MAX_TURNS to indicate that the defaults are model-aware rather than fixed values; replace the old “(default 1.50)” and “(default 15.00)” text with something like “(default: model-aware; see DM model scaling logic)” or list both tiers, and add a short pointer to the section of the script where the model-based default logic lives (the DM model scaling block) so readers know where to find the implementation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@scripts/play_party.sh`:
- Around line 44-46: The header comments for the play_party.sh script still list
the old Sonnet-tuned static defaults; update them to state that per-turn
(CLAWDND_PLAY_BUDGET), session (CLAWDND_PLAY_SESSION_BUDGET) and max turns
(CLAWDND_PLAY_MAX_TURNS) are model-aware and may scale based on the DM model
rather than fixed Sonnet defaults—mention the dynamic scaling behavior and
reference that the script computes budgets from the chosen DM model so readers
know values are not constant.
In `@scripts/play.sh`:
- Around line 22-24: Update the header comments for the three env vars
CLAWDND_PLAY_BUDGET, CLAWDND_PLAY_SESSION_BUDGET, and CLAWDND_PLAY_MAX_TURNS to
indicate that the defaults are model-aware rather than fixed values; replace the
old “(default 1.50)” and “(default 15.00)” text with something like “(default:
model-aware; see DM model scaling logic)” or list both tiers, and add a short
pointer to the section of the script where the model-based default logic lives
(the DM model scaling block) so readers know where to find the implementation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 35a00460-2888-4e3d-b390-1fabf3576175
📒 Files selected for processing (6)
qa/lib_beat_driver.shqa/ui_playtest_app.shscripts/play.shscripts/play_party.shservers/engine/tests/test_dm_budget_scaling.pyservers/engine/tests/test_opus_coldopen_tuning.py
100yenadmin
changed the title
…r the duo lens) run_duo's callers pass low per-turn budgets (sweep $2.00, fast_probe $0.80); the Opus high-effort cold-open costs ~$2.4, so the duo cold-open would trip error_max_budget_usd just like the .app backend did. Floor the per-turn cap to $4 for an Opus DM (cap, not spend).
This was referenced Jun 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#682 flipped the default DM model to Opus but the surrounding tuning was Sonnet-shaped, so the Opus default was broken at the cold-open. This PR makes it work end-to-end. All findings are measured on the VM (2026-06-06), not guessed.
Bug 1 — per-turn budget (backend never seated a PC)
The per-DM-turn cap defaulted to the Sonnet-tuned $1.50; the Opus cold-open world-build costs ~5× → the first turn tripped
error_max_budget_usd, the backend never seated a PC (failure_bucket=no_actor, spend $0).scripts/play.sh+scripts/play_party.sh: model-aware per-turn + session budget defaults (opus → $12 / $30; sonnet unchanged $1.50 / $15).qa/ui_playtest_app.sh: the claude lane pinnedCLAWDND_PLAY_BUDGET=\$1.50unconditionally → scale it toDM_MODEL.Bug 2 — cold-open latency (timed out before writing narration)
With the budget fixed, the Opus
--effort maxcold-open is generation-bound and never finished <400s — the turn was killed before writing narration, the harness grace-proceeded into an empty scene, and the timeout-retry minted a second campaign (the #640 condition).Measured: Opus
--effort highfinishes ~300s with a full, BG-caliber opening ("the lamps of Sorcerous Sundries burn low and blue… Ramazith's old tower"), cost $2.36, 1 clean campaign.qa/lib_beat_driver.sh: cold-open effort model-aware (opus → high, sonnet → max); cold-open timeout model-aware (opus → 500s, sonnet → 400s). Routine tier unchanged (medium / 200s).qa/ui_playtest_app.sh: the part-B player-ready narration-wait is now env-configurable and defaults longer for an Opus DM, so QA waits for the ~300s opening instead of grace-proceeding.Guards (Tier-0, CI-covered)
servers/engine/tests/test_dm_budget_scaling.py(3) +servers/engine/tests/test_opus_coldopen_tuning.py(3) — 6 assertions, pass locally.Validation
Cold-open proven (above). Full narrative-persona session at Opus + high-cold-open + medium-routine + lean in flight (sat / give-up / arc / story).