fix(qa+engine): make the beat heartbeat real — flip progress at ingest, decontaminate recap/FTS/lean-tail/fallback (#749) - #763
Merged
Conversation
…t, decontaminate recap/FTS/lean-tail/fallback (#749) The #743 wrapper heartbeat was (1) a NO-OP for the player — its texts are byte-identical to screen-table.jsx's _WRAPPER_PROGRESS_LINES and app.jsx sanitized the /events row to "" BEFORE notePendingProgress, so the spinner never flipped — and (2) real kind=narration rows that NO engine consumer filtered: recap recited the filler, the FTS ledger indexed it, the lean re-ground tail fed it back to the DM as canon, and dm_narration_fallback recovered it (masking fully-dead beats with stale prose + filler). Keep-and-repair, additive; engine stays the sole writer; old snapshots round-trip: (a) CLIENT: app.jsx /events ingest special-cases exact wrapper lines BEFORE the sanitize-drop — notePendingProgress() (flip streaming) + return null (never render; never sets eventsStreamedThisTurnRef so a dead beat's recovered /chat text still renders). Predicate + constant shared from screen-table.jsx as window-guarded globals. (b) ONE python source of truth: servers/engine/wrapper_progress.py (qa/wrapper_progress_lines.py re-exports it for stdlib-only qa scripts) + tests/test_wrapper_progress_sync.py regex-pins the jsx Set and BOTH sh emit rotations (lib_beat_driver.sh + play_codex_dm.sh) byte-identical. Exact-match filters in recap.format_recap, ledger.backfill (FTS), server._scene_recent_narration, and qa/dm_narration_fallback.py (where a heartbeat BREAKS the trailing block, so a heartbeat-only dead beat honestly recovers nothing). (c) FALLBACK HONESTY: clawdnd_resolve_dm_reply (direct-call front door) flags a #357 recovery; record_dm_reply / clawdnd_chatlog_dm stamp {"fallback_recovered":true} on the dm chat row (consume-once). All 9 runner call sites converted. assert_behavioral semantics unchanged. (d) DEDUP: #727's last-8 substring guard provably swallowed cadence-aligned heartbeats (a run of dead beats logs ONLY heartbeats — the 4-line rotation repeats inside the window). Wrapper lines are now EXEMPT from the dedup (always append); the repeats are inert (never rendered, fully filtered), keeping the exact-match design on every side. Tests (red-first): sync test, recap/FTS/lean-tail exclusion, fallback wrapper-skip + fallback_recovered stamps, dedup bypass + intact prose dedup, viewer node-harness heartbeat-flips-without-rendering + no-suppression guard. Engine 1783 passed, viewer 488 passed, fast_gate PASS, bash -n clean (3.2). Closes #749. Refs #743 #727 #623 #357 #720.
|
Warning Review limit reached
More reviews will be available in 14 minutes and 17 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (19)
Comment |
This was referenced Jun 10, 2026
100yenadmin
added a commit
that referenced
this pull request
Jun 11, 2026
… timeout(1) shim, cold-open guards, party heartbeat + soft-tick (audit F12-1, F12-3, F12-4, F12-5, F12-8) (#828) The five wrapper-reliability findings from docs/audits/ENGINE-AUDIT-2026-06-11.md unit 12 (skeptic-verified), fixed together because they share files: * F12-1 (enriches #753): routine beat deadline 200s -> 360s in clawdnd_dm_timeout (measured: routine p90=224s max=360s — the flat 200s killed ~18% of HEALTHY beats), and BOTH dm_turn paths (play.sh + play_party.sh) now recompute the retry deadline via the new clawdnd_dm_retry_timeout (attempt 2 escalates to the model-aware cold-open tier, never de-escalates, never reuses verbatim). * F12-8 (closes #787): worldos_timeout shim in qa/lib_beat_driver.sh — timeout(1) when present, else a python3 subprocess fallback preserving rc=124/127/126 semantics — swapped into both play-lane _dm_invoke call sites; launch_common.sh preflight now warns (non-fatal, the shim covers absence) with the brew-install-coreutils hint. * F12-3 (closes #777): play.sh cold open now ABORTS NON-ZERO on an empty opening DMSG and on a no-campaign mint (was: unflagged empty chat row + an indefinitely-"running" unplayable session), and runs the seating guard with one reseat retry then a loud abort. pc_seated() factored into the lib as clawdnd_pc_seated (snapshot-read-only, viewer _action_actor contract); play_party.sh now REUSES the shared helper instead of its local copy. * F12-4 (closes #790, completes #623): play_party.sh emits the model-independent progress heartbeat — before the cold-open turn (campaign id is pre-seeded in this lane) and after the human move BEFORE companion_moves — via the same shared helper play.sh calls (post-#763 contamination-safe). * F12-5 (closes #791): play_party.sh beat loop captures PREV_DAY/PREV_TOD pre-beat and runs clawdnd_soft_tick after record_dm_reply (mirror of play.sh:475-478/504) — it was the only beat loop without the backstop. Tests: servers/engine/tests/test_wrapper_reliability.py (22 new, red-first) — worldos_timeout behavioral matrix (rc pass-through, rc=124 deadline, PATH- stripped fallback, native-binary preference), retry-deadline escalation, clawdnd_pc_seated behavioral matrix, static anti-drift asserts on all call sites, bash -n on every touched script. test_adversarial_release.py seating assert updated to track the factored helper (intent unchanged). Full engine suite 1805 passed; qa/fast_gate.sh PASS. Engine stays the sole writer (guards are snapshot-read-only; heartbeat/tick route through log_engine_narration/advance_time). Wire contracts frozen (CLAWDND_* names kept; no new env names). Bash-3.2-clean. Co-authored-by: Eva <arncalso@gmail.com>
This was referenced Jun 11, 2026
[KEYSTONE] Live Composition — stream the DM's scene as it is written (tool-arg delta streaming)
#835
Open
100yenadmin
added a commit
that referenced
this pull request
Jun 14, 2026
…F07-1, F14-3) (#847) F07-1 (#772): the cold-open "previously on" recap and the FTS recall ledger were contaminated by combat/system BOOKKEEPING — every run recited "Tough 1 takes 5 force damage / Turn advances to Tough 2" and a recall('Rolan') probe returned 4 of 6 top hits as bookkeeping. Distinct from #749/#763, which exact-matched only the wrapper-heartbeat line. Engine combat-event rows carry payload schema clawdnd.combat_event.v1 (via _log_combat_event); the two session markers are "Session N began" / "Session ended.". recap.format_recap now drops schema-stamped combat rows (narrative combat with no/other payload stays); ledger.backfill skips schema-stamped combat rows AND the two session markers by exact prefix — while keeping every OTHER kind=system row indexed, preserving the documented DM-authored-system-note->recall path (SKILL.md:47). Derived-index-only; no schema change. F14-3 (#795): persist_beat (the every-beat write) burned ~2 beats in a real gate run. Four defects fixed by validate-then-apply: (1) decision chosen=null (and summary/rationale) crashed pydantic string_type -> None-coerce every Decision str field; (2) mem["character_id"] bare-KeyError'd -> resolve via the F14-8 _char resolver (#786) with id/npc_id aliases and an actionable "memories index N: ..." error carrying a did-you-mean; (3) events were appended to the session jsonl BEFORE memories/decision validation (a crash left a partial chronicle, a retry duplicated it) -> the whole batch is now validated BEFORE the first append_log (events-only non-atomic window closed); (4) the quadratic remembered echo (whole growing memory list per item) is now the slim {"id","fact","memory_count"}. advance stays its own sequential locked call; engine-sole-writer + one-lock-one-save preserved. Tests: +3 recap (drop schema-stamped combat; keep narrative/unrelated-payload combat), +3 ledger (skip combat-events + session markers; KEEP DM system note), +8 persist_beat (chosen:null; null str fields; actionable bad-id not bare KeyError; events-not-applied-on-later-failure atomicity; text alias honored; empty rejected; id alias resolves; remembered O(items) not quadratic). Full engine suite 2096 passed; fast_gate Tier-0 PASS. Closes #772 Closes #795 Source: docs/audits/ENGINE-AUDIT-2026-06-11.md Co-authored-by: Eva <arncalso@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #749. Refs #743 #727 #623 #357 #720.
The two audit-verified roots
_WRAPPER_PROGRESS_LINESinviewer/openworlds/screen-table.jsx, and the/eventsingest inviewer/openworlds/app.jsxranconst clean = sanitize(...); if (!clean) return null;BEFOREnotePendingProgress— the row was silently swallowed and the spinner never flipped. Zero player benefit.kind=narrationsession-log rows and no engine consumer filtered them:recap.pyrecited the filler in "Previously on…",ledger.pyindexed it for FTS recall,server.py::_scene_recent_narrationfed it to the lean re-ground (the DM was told filler is its own canon), andqa/dm_narration_fallback.pyrecovered it — masking fully-dead beats with stale prose + filler.The repair (keep-and-repair; additive; engine = sole writer)
(a) Client — flip progress at ingest.
app.jsx's/eventsingest special-cases the exact wrapper lines BEFORE the sanitize-drop:notePendingProgress()(flipsstreaming, re-arms the adaptive recovery window) +return null(never renders). It deliberately does NOT seteventsStreamedThisTurnRef/dmBeatCountRef, so a dead-DM beat whose only stream was the heartbeat still renders its recovered /chat text. The predicate + constant are shared fromscreen-table.jsx(window.isWrapperProgressLine/window.WRAPPER_PROGRESS_LINES), with a graceful fall-through to today's sanitize-drop if absent.(b) ONE python source of truth + sync test + 4 filters. Canonical constants live in
servers/engine/wrapper_progress.py(pure, dependency-free — the engine's 3 consumers import it directly and keep filtering even withoutqa/);qa/wrapper_progress_lines.py(the spec-named module) re-exports it by file-path load for the stdlib-only qa scripts.tests/test_wrapper_progress_sync.pyregex-parses the jsxSetAND BOTH sh emit rotations (qa/lib_beat_driver.shandscripts/play_codex_dm.sh— the audit cited one; both exist and both are now pinned) and fails on the first divergent byte. Exact-trimmed-match filters added in:servers/engine/recap.py(story-kind selection)servers/engine/ledger.py::backfill(FTS ingest)servers/engine/server.py::_scene_recent_narration(lean re-ground tail)qa/dm_narration_fallback.py— a wrapper line BREAKS the trailing block (like [playtest][CRITICAL] DM turn returns empty player-facing narration on engine-heavy beats #357 system-notation), so a heartbeat-only dead beat honestly recovers NOTHING instead of stitching the prior beat's stale prose.(c) Fallback honesty. New
clawdnd_resolve_dm_reply(a DIRECT-call front door overclawdnd_dm_narration_or_fallback— a command-substitution subshell could never export the flag) setsCLAWDND_FALLBACK_RECOVERED=1iff the #357 fallback recovered the prose.record_dm_reply(both branches) and the newclawdnd_chatlog_dmstamp{"fallback_recovered":true}on the dm chat row, consume-once. All 9 runner call sites converted (scripts/play.sh×2,scripts/play_party.sh×4,qa/run_duo.sh×2 + chatlog,qa/run_party.sh×2 + chatlog,qa/ui_playtest.sh— whosedm_turnnow returns the raw result so the caller can resolve+flag). Flags are jsonl-side payload only;assert_behavioralsemantics unchanged in this PR.(d) Dedup verdict: REAL collision, fixed by exemption. #727's last-8 substring guard in
lib_beat_driver.sh::log_engine_narrationprovably swallows cadence-aligned heartbeats: a run of dead beats logs ONLY heartbeats, so with a 4-line rotation, beat N+4's text always sits inside the last-8 tail → dropped → no /events row → no flip, exactly when the heartbeat is the only life sign. Chosen fix: wrapper lines are EXEMPT from the dedup (always append) rather than uniquifying the text with a beat counter — this keeps EXACT-match semantics on every filter surface (no prefix-matching contract to keep in sync), and the repeated rows are inert (never rendered; filtered from recap/FTS/lean tail/fallback).scripts/play_codex_dm.sh's ownlog_engine_narrationhas no dedup guard, so no change needed there. The intact direction is pinned: the DM's own echoed prose still dedups to one row.Tests (red-first; red proven before implementation)
servers/engine/tests/test_wrapper_progress_sync.py(7) — python==jsx==both-sh, shim re-export, window export, exact-trim semantics.servers/engine/tests/test_heartbeat_repair.py(10) — recap (store-backed + unit + all-filler→new-adventure), FTS recall exclusion,scene_contextlean-tail exclusion, dedup heartbeat-repeat-survives + real-prose-still-dedups (real bash + uv engine),fallback_recoveredstamps (recovery/no-recovery/failure-path/success-path, consume-once).servers/engine/tests/test_dm_narration_fallback.py(+2) — heartbeat-only dead beat recovers nothing; prose after a heartbeat survives.viewer/tests/test_live_narration_stream.py(+2, node harness over the real transpiled jsx) — a wrapper /events row flipspending.streamingWITHOUT rendering; a heartbeat-only stream does not suppress a later /chat-only beat.servers/engine/tests/test_dm_session_remint.py— the ui_playtest static guard updated to track the new front door (same intent, stronger: both turns + honesty stamp).Verification
uv run --directory servers/engine python -m pytest tests -q -p no:xdist→ 1783 passedpython3 -m pytest viewer/tests -q -p no:xdist→ 488 passed, 6 skippedbash qa/fast_gate.sh→ PASS (188 passed)/bin/bash -nclean on all 7 touched shell scripts (macOS system bash 3.2)Untouched (per constraints)
#746/#761 stuck-backstop code, #740 chronicle sanitizer's handling of real prose, wire contracts,
_StrictModelschemas. bash 3.2-clean throughout.