feat(viewer): stream DM narration incrementally to /chat

[100yenadmin]

What & why

The 5-persona playtest on the BUILT app (build_sha e6384e8) surfaced per-beat latency with no streaming as the #1 satisfaction-killer: a DM turn takes ~60–90s and the player sees a waiting indicator but no content appearing, so impatient personas conclude "this feels broken" and quit (adversarial gave up at 1m23s on the cold-open; narrative ~90s on turn 2). Fixes the play-loop perception blocker tracked in #393.

Approach: streaming-lite (bounded), NOT a rewrite

I assessed whether incremental narration streaming was bounded — it is, because the live mid-turn stream already exists on disk and the viewer already reads it:

• The DM logs each beat via log_event(kind="narration"/"dialogue") during its turn, and the engine appends it to campaigns/<id>/sessions/<sid>.jsonl immediately (servers/engine/store.py::append_log — append-mode open/write/close per entry, under campaign_lock). A 60–90s turn makes several of these in real time.
• The viewer's /events endpoint (viewer/server.py::_read_events) already tails that session log with a line cursor and handles session rotation.
• Only /chat was surfaced live in the chronicle, and the runners append the DM line to /chat only at turn-END — so the poll saw nothing for the whole turn, then the complete beat (the gap is documented verbatim in app.jsx lines 82–92).

So this is resolver/viewer work, not a streaming rewrite: no change to the resolver's blocking claude -p turn, no SSE, and no change to engine write-semantics (the engine stays the sole writer; the viewer purely reads state the engine already wrote).

The companion mitigation (a dynamic/reassuring waiting state — rotating cold-open flavor, live elapsed clock, a11y proof-of-life) already shipped in #385, so the remaining high-impact bounded win was the streaming piece.

What changed

viewer/openworlds/app.jsx — useLiveSession:

• Added a live /events poll (3s, visibility-aware, best-effort) alongside the existing /chat poll. New narration/dialogue rows arriving mid-turn become live, time-stamped chronicle beats — the scene visibly builds during the wait instead of a blank indicator.
• Turn-gating preserved (Option B): a streamed beat shows prose but keeps the "narrating…" indicator up (the action bar stays gated — one move at a time). The turn only resolves (indicator clears, bar re-opens) when its final line lands on /chat. Streamed progress resets the stall/stuck recovery clock (notePendingProgress) so a long-but-healthy streaming turn is never falsely declared stuck. (I chose this over clearing the bar mid-turn: the give-up cause was seeing nothing, not being unable to act — so fix the perception without relaxing the one-move-at-a-time semantics, which is race-adjacent.)
• Dedup across both sources (claimNarration, a shared text-keyed seen-set, whitespace/case-normalized): the same paragraph shows exactly once, from whichever source reached the player first (live, in practice). The turn-END /chat line still resolves the turn even when its prose was fully deduped (a turn whose entire beat streamed live still re-opens the bar — tracked via a dmLineArrived flag).
• Pending writes go through one setPendingState writer that mirrors state into a pendingRef, so the /events poll (whose deps deliberately exclude pending) reads the current turn without a stale closure; timer side-effects live outside the state updater (StrictMode-safe).

viewer/openworlds/screen-table.jsx: dedup recentEvents against the live tail — recentEvents (from /session-surface) is a trailing window of the same session log, so a streamed paragraph would otherwise appear in both the history band and the live band. Mechanics rows and genuine pre-session prose are untouched.

viewer/tests/test_live_narration_stream.py (new, 10 tests): transpiles the real .jsx with the bundled Babel-standalone and drives useLiveSession under a deterministic React + scripted-fetch harness (mirrors test_recovery_timing.py) — mid-turn /events narration + dialogue surface live; the turn stays gated while streaming; the /chat line resolves a streamed turn; whitespace/case-insensitive dedup; unstreamed /chat prose still renders; player echoes are never deduped; a fresh run resets the dedup set. 49/49 viewer tests green (10 new + 39 existing recovery-timing / cold-open / sanitize / static).

Note: the OpenWorlds UI is transpiled in-browser (<script type="text/babel">), so these JSX changes ship as-is — the Node+Babel test harness is the compile gate, and both files transpile cleanly.

Honest impact estimate

This should substantially reduce the latency give-ups: the give-up cause was seeing nothing happen, and the player now watches the scene arrive within the first poll cycle (~3s) after the DM's first log_event. It does not make the turn faster, and it relies on the DM logging narration incrementally (it does today; a turn that logged only at the very end would still feel slower — a candidate follow-up is nudging the DM prompt to log an opening line early). The action bar stays gated until turn-end by design.

Known minor tradeoff: a verbatim-repeated paragraph much later in a session would be deduped (suppressed) — verbatim repeats are rare and usually undesirable, and this errs on "never double-render."

Verification

Do NOT close on merge — verify on the next full-arc playtest against the BUILT app (no persona gives up on beat latency; scene text is visibly appearing within ~15–30s of a turn starting, not a blank wait to turn-end). Per WorldOS-OPERATING-GOAL.md §4.7, an issue is closed by a non-reproducing next build, not by a merged PR.

Summary by CodeRabbit

Release Notes

• New Features

  • Live narration now streams in real-time during game turns, appearing instantly as it's delivered.

• Bug Fixes

  • Fixed duplicate narration paragraphs appearing in the chat log when text arrives via multiple streams.
  • Improved handling of pending turn state during live streaming to prevent false timeout detection.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

useLiveSession now tracks /events updates alongside /chat to stream live narration during turns, deduplicates narration across both sources using app-level state, clears pending only on actual DM turn-end lines while showing streamed prose live, and ScreenTable prevents display duplication by filtering recent events against the merged live tail.

Changes

Live narration streaming and dedup

Layer / File(s)	Summary
App-level state infrastructure viewer/openworlds/app.jsx	Adds /events cursor, seenNarration dedup Set, pendingRef+setPendingState pattern, timer refs, and claimNarration helper to enable live streaming without stale closures.
Pending turn recovery and progress tracking viewer/openworlds/app.jsx	Updates armPending to use setPendingState, adds notePendingProgress to reset recovery timers on streamed /events beats and clear stuck flag when prose arrives, and implements run-change reset logic for cursors and dedup state.
Chat polling with separated turn-end detection viewer/openworlds/app.jsx	Refactors /chat ingestion to track DM turn-end line arrival separately from deduped narration paragraphs, clearing pending based on DM-line arrival even when narration is fully deduplicated.
Mid-turn events polling effect viewer/openworlds/app.jsx	Introduces visibility-aware /events polling effect with dedicated cursor, sanitizes and dedupes narration/dialogue via claimNarration, appends streamed beats to chat log, advances beat counter, and calls notePendingProgress for live progress tracking.
Screen-table live-tail deduplication viewer/openworlds/screen-table.jsx	Implements normalized narration deduplication: builds a Set of narration keys from merged live tail using sanitizeNarration, filters recentEvents to remove duplicate narration/dialogue rows already in the live band, and updates visibleLog accordingly.
Test harness infrastructure viewer/tests/test_live_narration_stream.py	Provides a Node-based harness that transpiles real JSX via babel-standalone, implements minimal React hook/effect stubs, scripted /events+/chat fetch queues, interval tick with async drain, and test runner for hook behavior validation.
Behavior test cases viewer/tests/test_live_narration_stream.py	Validates live narration streaming, pending state management during turns, turn resolution and final pending clear, deduplication correctness across /events and /chat, whitespace/case normalization resilience, unstreamed narration rendering (no over-dedup), player-line non-dedup, and dedup reset on campaign/run switch.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related issues

• [playtest][P0] Per-beat DM latency (~60-90s) with no streaming → impatient players give up mid-session #393: Directly implements the /events-based mid-turn streaming and narration deduplication described in the linked issue, adding the /events poll, pendingRef/setPendingState refactor, and dedup across /events and /chat.

Possibly related PRs

• electricsheephq/WorldOS#343: Also overhauls useLiveSession/pending handling by lifting in-flight DM narration state to App and coordinating chat/narration polling via shared app-level session logic.
• electricsheephq/WorldOS#328: Prior PR that introduces the pending indicator and clears it from /chat DM narration; this PR expands pending recovery using streamed /events beats.
• electricsheephq/WorldOS#351: Modifies the same pending/"stuck" timer recovery logic in useLiveSession; this PR adds per-/events progress resets while that PR introduces adaptive stuck windows.

Poem

🐰 Streams flow now where silence dwelt,
Mid-turn narration, live and felt,
Dedup keeps prose from doubling twice,
The pending bar adapts so nice—
A rabbit cheers the turn's new dance! 🎭

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The PR's focus on streaming narration and enhancing viewer responsiveness is orthogonal to issue #39, which concerns constrained player-actor tooling and role separation. The PR does not implement, address, or advance the coding requirements from #39.	Either link this PR to the correct issue (#393 is mentioned in the description and tests, and should be formally linked), or clarify how player-tool constraints are being implemented if the #39 link is intentional.
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: adding live streaming of DM narration to the /chat endpoint in the viewer, which is the primary objective across all file modifications.
Description check	✅ Passed	The description comprehensively covers the problem, approach, implementation details, test coverage, and verification steps. It follows the template structure with sections for what/why, licensing, and validation.
Out of Scope Changes check	✅ Passed	All code changes (app.jsx narration streaming, screen-table.jsx deduping, new test suite) are tightly scoped to the issue #393 objective of streaming mid-turn narration. No unrelated refactoring, formatting, or feature creep is present.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}