Skip to content

perf+qa: alwaysLoad engine tools + finish-tool for honest gave_up (validated on .app) - #574

Merged
100yenadmin merged 2 commits into
mainfrom
gate/gaveup-budget
Jun 2, 2026
Merged

perf+qa: alwaysLoad engine tools + finish-tool for honest gave_up (validated on .app)#574
100yenadmin merged 2 commits into
mainfrom
gate/gaveup-budget

Conversation

@100yenadmin

@100yenadmin 100yenadmin commented Jun 2, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Member

Validated on the built .app (finishval-vet): gave_up=FALSE + satisfaction 7 SELF-REPORTED + pass, 0 critical, DM writing praised.

  • alwaysLoad on clawdnd-engine (generated dm.mcp.json in play.sh/play_party.sh + .mcp.json + run_duo parity): un-defers engine tools → eliminates the per-beat ToolSearch round-trips (measured: cold-open ToolSearch 4→0, cold-open 248→176s). Cache-stable. Mechanism verified in claude 2.1.160 binary.
  • finish(satisfaction,verdict) tool (palette_server.js) + scorer prefers structured status.satisfaction + prompt finish-when-satisfied/give_up-when-blocked + Satisfaction:N/10: makes gave_up=false honestly reachable for a satisfied end + guarantees a self-reported score (no derived penalty). Reject keyword-reinterpretation (gaming).
  • Research workflow (4 clusters) + cross-verification. Harness: keep hand-rolled + borrow pi progressive-disclosure (alwaysLoad embodies it).

Latency verdict: DM beat (~100-126s medium) ≈ original Opus-high; NOT the GUI/harness (pure claude -p reasoning). Effort kept at medium (quality).

Summary by CodeRabbit

  • New Features

    • Added a finish tool for playtests, allowing players to formally conclude sessions with satisfaction ratings (1–10) and verdict feedback.

  • Improvements

    • Enhanced playtest satisfaction scoring to prioritize structured player feedback when available.
    • Updated playtest prompts with clearer instructions on tool usage and satisfaction reporting.
    • Improved MCP server configuration for consistent engine initialization across playtest environments.

Eva added 2 commits June 2, 2026 20:11
Eva
perf(dm)+fix(qa): alwaysLoad engine tools (kill ToolSearch deferral t…
34e06f8 
…ax) + finish-tool for honest gave_up

LATENCY (secondary lever, ~14-19s/beat, zero-risk): alwaysLoad:true on clawdnd-engine in the
generated dm.mcp.json (play.sh + play_party.sh) + repo .mcp.json — un-defers the ~15 engine tools
so the DM stops burning ~2 ToolSearch round-trips/beat re-discovering them. Cache-stable static
manifest (pi-style progressive-disclosure, owner's 'always-equipped tools' idea). alwaysLoad
verified real in claude 2.1.160 (binary-safe grep: alwaysLoad/ENABLE_TOOL_SEARCH/shouldDefer).

GAVE_UP (honest, not gamed): add finish(satisfaction,verdict) terminal tool to palette_server.js
(reason=finish => gave_up=false for a SATISFIED end; give_up stays for genuine blocks); scorer
prefers structured status.satisfaction over the brittle verdict regex; prompt: finish-when-satisfied
/ give_up-only-when-blocked + keep latency-patience + mandatory 'Satisfaction: N/10'; finish added to
codex enabled_tools. Reverted the audit-mode prompt-v1 (canonical was already clean).

Research: workflow wf_6190c02a-dfe (4 clusters, cross-verified). Harness: KEEP hand-rolled + borrow
pi progressive-disclosure; reject adopt-pi (no MCP) / nanoclaw (wrong category).
NOT VALIDATED YET — effort lever (medium vs default) pending engine-duo sweep. Do not merge until validated.
Eva
qa(run_duo): alwaysLoad engine parity (env CLAWDND_ENGINE_ALWAYSLOAD,…
ccbeea6 
… default on) — match play.sh, no ToolSearch tax in the duo harness
@100yenadmin
100yenadmin merged commit 48bd188 into main Jun 2, 2026
3 checks passed
@coderabbitai

coderabbitai Bot commented Jun 2, 2026 •
edited
Loading

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3ce10050-2a7d-4068-950c-2df821588836

📥 Commits

Reviewing files that changed from the base of the PR and between f1c796c and ccbeea6.

⛔ Files ignored due to path filters (2)
  • gaveupfix-vet.log is excluded by !**/*.log
  • gaveupfix-vet2.log is excluded by !**/*.log
📒 Files selected for processing (8)
  • .mcp.json
  • qa/playwright/node_modules
  • qa/playwright/palette_server.js
  • qa/run_duo.sh
  • qa/ui_playtest_app.sh
  • qa/ui_playtest_score.py
  • scripts/play.sh
  • scripts/play_party.sh
📝 Walkthrough

Walkthrough

The PR introduces a finish tool for playtests to report satisfaction, updates satisfaction scoring to extract this structured feedback, enables the tool in the playtest UI harness with updated player instructions, and configures the clawdnd-engine MCP server to always load across play configurations.

Changes

Playtest finish flow

Layer / File(s) Summary
finish MCP tool implementation
qa/playwright/palette_server.js		 Introduces the finish tool that accepts satisfaction (1–10) and verdict (string), captures a screenshot when possible, logs the action to actions.ndjson, writes an ended state to status.json, and returns an ended confirmation.
Engine server always-load configuration
.mcp.json, scripts/play.sh, scripts/play_party.sh, qa/run_duo.sh		 Adds alwaysLoad: true to the clawdnd-engine MCP server configuration across static config files and play script generators to ensure persistent availability.
Playtest satisfaction extraction logic
qa/ui_playtest_score.py		 Updates satisfaction computation to prefer the structured satisfaction value from status.json, with fallback to parsing from the verdict text and friction-based estimation when unavailable.
Playtest UI harness updates
qa/ui_playtest_app.sh		 Adds the finish tool to enabled MCP tools for Codex player, updates persona player prompt with explicit instructions for when to call finish vs give_up and required satisfaction line format.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

  • electricsheephq/WorldOS#325: Extends the Playwright Player MCP harness by adding the finish tool to qa/playwright/palette_server.js and updating qa/ui_playtest_score.py to extract satisfaction from the new tool's structured output.
  • electricsheephq/WorldOS#374: Both PRs modify qa/ui_playtest_app.sh; the retrieved PR introduces the playtest harness, and this PR updates persona prompts and enables the new finish tool.

Poem

🐰 A finish line for playtests grand,
With satisfaction we now stand—
The finish tool lets players rest,
And rate how well they faced the test.
Always-load keeps the engine near,
So playful feedback comes out clear. ✨

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands and usage tips.

100yenadmin added a commit that referenced this pull request Jun 2, 2026
@100yenadmin Eva
docs(skills): capture the last-few-days workflows — latency-forensics…
3e531cc 
… skill + ledger/model/flag/finish fixes (#578)

Skillify backlog (5 HIGH items, from the wf_ed9a8ad2 discovery workflow, gap-checked):
1. NEW skill .claude/skills/worldos-latency-forensics — SDK duration_api_ms measurement method (true tool-exec ~1-4%, beats are generation-bound), quality-cost lever taxonomy (neutral: alwaysLoad/streaming/scene_context/prose-trim; trading: effort), REFUTED hypotheses (Haiku-helper, --tools, --fast).
2. worldos-dev SKILL.md: re-add the ⚠ Support-VM-lane callout (heavy sweeps, not local); ledger drift fix (canonical is scores_db.py -> scores_ledger.md, SCORECARD legacy); palette now 9-tool with finish() (finish=satisfied/gave_up=false vs give_up=blocked, #574); + DISCIPLINE: 'QA must exercise the flag' (run_duo ignored CLAWDND_LEAN_BEATS -> confounded A/Bs) and 'DM latency is reasoning not the GUI/harness'.
3. docs/MODEL-TIERING-STRATEGY.md: supersede the stale proposal with the MEASURED state (generation-bound; effort is the lever; ONE model/campaign, never switch mid-campaign=cache; Opus story 4.4-4.5 vs Sonnet 4.2). Model choice left OPEN (owner's call; models are options, not bolted-on); Haiku-helper REFUTED.
4. Ledger-drift banner on qa/SCORECARD.md (LEGACY -> scores_db.py) + qa/SCORING.md + qa/QA_TOOLS.md refs.

Doc-only. The 5th HIGH item (verify-subagent-factual-claims) lands in the user-global verification-before-completion skill, separately.

Co-authored-by: Eva <arncalso@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@100yenadmin