feat(qa): make GLM QA work end-to-end (scorer isolation + all-harness profile wiring)

[100yenadmin]

Two fixes that together make GLM actually usable for QA across the board (validated overnight: 4 GLM duos completed + scored, story3.6 / mech3.6 / angry~3.1 vs Claude 4.1/3.8/3.4).

1. Scorer isolation (the blocker)

On a GLM run, qa/glm_profile.sh exports ANTHROPIC_BASE_URL=z.ai + the GLM key + a fresh CLAUDE_CONFIG_DIR globally, so the pinned-sonnet scorer's claude call hit z.ai with a Claude model name (Unknown Model) → every GLM run scored empty. qa/score.sh now neutralizes those vars for the scorer call only (env -u …) → clean Claude (~/.claude OAuth). No-op on normal Claude runs.

2. GLM profile wired into ALL harnesses

Previously only run_duo.sh sourced the GLM profile. Added it (unconditional source + no-op-for-Claude call) to run_party.sh, run_combat_sprint.sh, and ui_playtest.sh so a GLM model selection applies everywhere, not just the duo. Claude path byte-identical.

GLM verdict: reliable QA runner, near-Claude mechanical quality, slightly-below story, ~2-3× slower — a viable cheap batch-sweep engine, not a replacement for Claude on the final quality gate.

Summary by CodeRabbit

• Bug Fixes
  • Fixed quality assurance scoring consistency by running the scorer without GLM-related environment settings that could alter scoring behavior.
• New Features
  • Updated QA run scripts to initialize a GLM-only settings profile and apply it early, ensuring model-specific timeout/budget/retry settings are used for combat, party, and playtest flows.
  • Added a new behavioral gate check (dm_advanced_time) to flag sessions where the DM does not advance time tools (warning only).

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5497584c-bfe6-4dc2-97b5-efd367054b5e

📥 Commits

Reviewing files that changed from the base of the PR and between b81dd8a and c2f6ec2.

📒 Files selected for processing (5)

• qa/BEHAVIORAL_GATE_TAXONOMY.json
• qa/run_combat_sprint.sh
• qa/run_party.sh
• qa/score.sh
• qa/ui_playtest.sh

---

📝 Walkthrough

Walkthrough

The PR isolates the QA scoring subprocess from GLM-related environment variables and applies GLM settings profiles across multiple QA harness scripts. In qa/score.sh, the claude -p invocation is wrapped with env -u to unset five variables that may be inherited from GLM runs. In qa/run_combat_sprint.sh, qa/run_party.sh, and qa/ui_playtest.sh, the scripts source qa/glm_profile.sh and apply settings via worldos_apply_glm_profile early in initialization. A new behavioral gate check named dm_advanced_time is added to the taxonomy to validate DM time-advance tool invocation.

Changes

QA Environment Configuration

Layer / File(s)	Summary
Unset GLM env vars for scorer invocation qa/score.sh	The | claude -p pipeline is replaced with | env -u ANTHROPIC_BASE_URL -u ANTHROPIC_API_KEY -u ANTHROPIC_AUTH_TOKEN -u API_TIMEOUT_MS -u CLAUDE_CONFIG_DIR claude -p, stripping five GLM-related variables before the scorer subprocess executes.
Apply GLM settings profile to QA harnesses qa/run_combat_sprint.sh, qa/run_party.sh, qa/ui_playtest.sh	Three QA harness scripts now source qa/glm_profile.sh and call worldos_apply_glm_profile after model environment variables are resolved but before timeout/budget/retry knobs are consumed. The ui_playtest.sh variant also exports DM and actor model environment variables at startup.
Add dm_advanced_time behavioral gate check qa/BEHAVIORAL_GATE_TAXONOMY.json	A new dm_advanced_time gate check is added under the DM_ADHERENCE category with retest command bash qa/run_duo.sh duo-retest and a hint indicating the DM failed to invoke time-advance tools (WARN-level).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

• electricsheephq/WorldOS#1008: Introduces the same qa/glm_profile.sh and worldos_apply_glm_profile mechanism to apply GLM-only timeout/retry settings after model selection in the QA harness.
• electricsheephq/WorldOS#1024: Implements and tests the dm_advanced_time WARN gate in qa/assert_behavioral.py, corresponding to the gate definition added in this PR's taxonomy update.
• electricsheephq/WorldOS#537: Modifies the same claude -p invocation in qa/score.sh, switching prompt delivery to STDIN and adding stricter JSON/error handling.

Poem

🐇 Hop, hop, the QA scripts align,
With profiles applied, the settings shine!
The scorer runs clean, no borrowed keys,
While harnesses tune their timeouts with ease.
A bunny's refactor: isolation and care! 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description covers the changes and rationale but lacks the required Licensing/CLA section and Validation checklist from the template.	Add the required Licensing/CLA checkboxes and a Validation section listing the checks performed (e.g., overnight testing results already mentioned in prose).

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the two main changes: scorer isolation and GLM profile wiring across harnesses, which are the primary objectives of this PR.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Note

We couldn't fetch the incremental changes for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.