Skip to content

feat(replay): API-driven frame scrubbing for recorded sessions#241

Merged
delebedev merged 12 commits intomainfrom
feat/replay-scrub
Mar 26, 2026
Merged

feat(replay): API-driven frame scrubbing for recorded sessions#241
delebedev merged 12 commits intomainfrom
feat/replay-scrub

Conversation

@delebedev
Copy link
Owner

Summary

Replay recorded game sessions through Arena's renderer with frame-by-frame control via debug API. Feed captured protobuf frames back to the client — full card art, animations, board rendering — controlled by curl or any HTTP client.

Use case: "I played a real server game and forgot what cards I saw." Start replay mode, connect Arena, step through frames, visually recall the game.

Closes #238 (partial — v0 forward-only scrubbing, no backward/goto yet)

How it works

just serve-replay loads a recording directory, detects the format (proxy seat-1 payloads, raw proxy frames, or engine dumps), and starts the normal FD+MD server with a ReplayHandler instead of Forge. Arena connects normally — auth, room state, match entry all work. GRE frames are paused until the debug API advances them.

API

# Start replay
LEYLINE_PAYLOADS=recordings/2026-03-25_22-37-18 just serve-replay

# Connect Arena (Sparky match flow), then:
curl -s http://localhost:8090/api/replay/status | python3 -m json.tool
curl -s -X POST http://localhost:8090/api/replay/next | python3 -m json.tool
curl -s http://localhost:8090/api/replay/index | python3 -m json.tool
  • GET /api/replay/status — current frame, total frames, GRE type, active state
  • POST /api/replay/next — advance one frame, returns updated status
  • GET /api/replay/index — ordered metadata for all GRE frames

Smoke tested

  • Proxy recording (March 7, 228 frames) — full game replayed in Arena, cards rendered, mulligan visible, board state progressing through turns
  • Today's recording (March 25, 164 frames) — seat-1 payloads format detected and loaded

Known limitations (v1)

  • Forward-only. No backward scrub or goto — needs state accumulation (v2)
  • Seat-2 frame consumption. Both seat handlers share the frame list; Familiar connection consumes frames independently of the controller's position counter
  • No pacing control. Each POST /next fires immediately — no auto-play timer yet
  • Single-seat payloads only have seat-1 auth. Familiar gets a fallback patched auth (works but logged as warning)

Changes

  • ReplayController interface — FrameInfo, ReplayStatus, next(), status(), frameIndex
  • ReplayHandler implements ReplayController — pauses on GRE frames, advances via next()
  • LeylineServer exposes controller, DebugServer gets replay endpoints
  • Format detection: capture/seat-1/md-payloads/ > capture/frames/ (with header strip) > engine/ > legacy
  • Auth fallback for single-seat recordings
  • Uses real player.db in replay mode (fixes in-memory SQLite crash)

Test plan

  • just build compiles clean
  • Start replay with proxy recording, connect Arena, step frames via curl
  • Verify Arena renders game state progression
  • /api/replay/status shows correct frame count and position
  • /api/replay/index returns frame metadata array

@github-actions
Copy link

github-actions bot commented Mar 26, 2026

Test Results

113 files   -   139  113 suites   - 139   2m 11s ⏱️ -41s
802 tests  -   228  174 ✅  - 830  628 💤 +602  0 ❌ ±0 
802 runs   - 1 039  174 ✅  - 845  628 💤  - 194  0 ❌ ±0 

Results for commit 7ac3eaf. ± Comparison against base commit df04456.

This pull request removes 228 tests.
leyline.LeylinePathsTest ‑ createLatestSymlink creates relative symlink
leyline.LeylinePathsTest ‑ createLatestSymlink replaces old symlink with new target
leyline.LeylinePathsTest ‑ createLatestSymlink replaces stale directory with symlink
leyline.account.AccountRoutesTest ‑ age gate stub returns false
leyline.account.AccountRoutesTest ‑ doorbell returns FdURI
leyline.account.AccountRoutesTest ‑ login with unknown email returns 401
leyline.account.AccountRoutesTest ‑ login with valid credentials returns 200 + tokens
leyline.account.AccountRoutesTest ‑ login with wrong password returns 401
leyline.account.AccountRoutesTest ‑ moderate stub returns 200
leyline.account.AccountRoutesTest ‑ profile rejects JWT-shaped bearer that only exposes sub without passing token validation
…
This pull request skips 618 tests.
leyline.architecture.PackageLayeringTest ‑ bridge does not depend on game, match, or protocol
leyline.architecture.PackageLayeringTest ‑ config does not depend on any matchdoor package
leyline.architecture.PackageLayeringTest ‑ conformance does not depend on any matchdoor package
leyline.architecture.PackageLayeringTest ‑ game does not depend on match, protocol, or infra
leyline.architecture.PackageLayeringTest ‑ infra does not depend on match or game
leyline.architecture.PackageLayeringTest ‑ protocol does not depend on match
leyline.bridge.DeckConverterTest ‑ converts CardEntry list to deck text
leyline.bridge.DeckConverterTest ‑ empty sideboard omits header
leyline.bridge.DeckConverterTest ‑ includes sideboard section
leyline.bridge.DeckConverterTest ‑ skips unknown grpIds
…

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Mar 26, 2026

CI Report — Gate

Tests: 173/802 passed, 1 failed (628 skipped)

Failed tests
  • SurveilFlowTest.surveil to graveyard produces ZoneTransfer with Surveil category (0.4s)

    java.lang.AssertionError: Expected value to not be null, but was null.

Slow tests (>3s): 5
  • AiTurnNoAarTest.AI turn sends no ActionsAvailableReq to client (34.4s)
  • ActivatedAbilityPuzzleTest.Goblin Fireslinger tap-to-ping kills opponent (10.9s)
  • PvpBridgeEndToEndTest.two-player game: land, creature, turn rotation, per-seat visibility (7.2s)
  • DeclareBlockersDedupeTest.no duplicate blockers req (6.1s)
  • AiFirstTurnShapeTest.at most one post-handshake Full GSM with zones (3.2s)

@delebedev
Copy link
Owner Author

Self-review

What's good

  • Clean interface separation — ReplayController keeps DebugServer free of Netty/protobuf concerns
  • Lazy () -> ReplayController? lambda handles "controller doesn't exist until client connects" lifecycle
  • Format detection cascade (seat-1 payloads > raw frames > engine > legacy) handles all recording shapes
  • Auth fallback for single-seat recordings is pragmatic

Issues to fix in this PR

  1. grePosition drifts from actual frames sent. popGreForSeat() skips frames (seat filtering), so grePosition counts "next calls" not frame index position. status().currentFrameInfo will point to wrong frame.

  2. Thread safety on next(). Called from debug-http thread but reads pendingCtx (set by Netty I/O) and mutates greEvents via popGreForSeat(). @Volatile on pendingCtx isn't enough for the compound operation. Adding a lock.

Noted for v2 (not blocking)

  • Seat-2 handler consumes frames independently (needs per-seat frame lists)
  • uncategorized field populated but never read (pre-existing)
  • Plan doc (docs/superpowers/plans/) — consider gitignoring these
  • Auto-play timer (POST /api/replay/play?interval=500)
  • POST /api/replay/goto?frame=N once accumulator exists

@delebedev delebedev marked this pull request as ready for review March 26, 2026 09:37
delebedev and others added 12 commits March 26, 2026 09:37
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…PI-driven scrubbing

handleGre now stores pendingCtx instead of auto-popping the next frame.
handleConnect stores pendingCtx after sending roomState instead of eagerly
sending the first GRE bundle. next() pops and sends on demand.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add `replayController` field on LeylineServer (set on first MD connection in replay mode)
- Pass it to DebugServer as a lazy lambda in LeylineMain.buildDebugServer()
- Add `replayController: () -> ReplayController?` param to DebugServer constructor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /api/replay/status — current frame, total, greType, fileName, atEnd, active
- POST /api/replay/next — advance one frame, return updated status
- GET /api/replay/index — full ordered frame metadata list
- ReplayStatusDto / ReplayFrameDto serializable DTOs
- Returns inactive stub when no replay is active (safe for non-replay mode)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Raw proxy captures use NNN_MD_S-C_MATCH_DATA.bin naming in
capture/frames/, not S-C_MATCH* in capture/payloads/. Updated
format detection and justfile default path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
In-memory SQLite was failing due to Exposed connection pooling.
Use the on-disk player.db (has decks/player data needed for FD
handshake), fall back to in-memory only if no DB exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Proxy captures in capture/frames/ include the TCP frame header
(6 bytes: type + length). Must strip before parsing as protobuf.
Engine dumps don't have this header.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recent proxy sessions write decoded payloads (no frame header) to
capture/seat-1/md-payloads/. Older sessions use capture/frames/
with 6-byte headers. Detection order: seat-1 payloads > raw frames
> engine > legacy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Seat-1 payloads only contain seat-1's auth response. Familiar
(seat-2) needs auth too. Fall back to any available auth and patch
the clientId to match the requesting seat.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add replayLock synchronizing next(), handleGre, handleConnect
  (debug-http thread vs Netty I/O thread access to pendingCtx
  and greEvents)
- Track grePosition by actual frame index from popped payload,
  not call count — fixes drift when popGreForSeat skips frames

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@delebedev delebedev merged commit 5f7a1bd into main Mar 26, 2026
0 of 2 checks passed
@delebedev delebedev deleted the feat/replay-scrub branch March 26, 2026 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor: migrate Python tooling to Bun + TypeScript

1 participant