Skip to content

recording: pace voice helpers in content-time (lockstep follow-up)#191

Merged
borisbat merged 1 commit into
masterfrom
bbatkin/recording-content-time-helpers
Jun 7, 2026
Merged

recording: pace voice helpers in content-time (lockstep follow-up)#191
borisbat merged 1 commit into
masterfrom
bbatkin/recording-content-time-helpers

Conversation

@borisbat

@borisbat borisbat commented Jun 7, 2026

Copy link
Copy Markdown
Owner

Follow-up to the lockstep-recording sweep (#189). #189 converted say() to content-time and flagged the manual-hold drivers as a mechanical follow-up; this is that sweep, done centrally in the playwright helpers so every driver (here, and in the dependent implot / node-editor repos that call these same helpers) benefits at once.

Why

Under lockstep the recorder captures every rendered frame at a fixed 1/fps delay. A wall-clock sleep(ms) therefore yields render_rate/fps * ms of video — a 2 s caption shows for ~4 s on a 60 fps-rendering machine — and drifts the caption off its (real-time) voiceover. Content-time holds wait for ms/1000*fps captured frames, so the beat is exactly ms of video and caption / voice / visual stay aligned in the mux.

Changes (widgets/imgui_playwright.das)

  • 18 pacing sleep(ms)hold_content(app, ms) across hold_through_voice, drag_through_voice, type_into_voice, combo_pick_voice, toggle_tree_voice, close_button_voice, click_render_voice, menu_pick_voice. hold_content == sleep outside a recording session, so non-recording runs are unchanged. The synth mouse/key timelines ride the same fixed clock under lockstep, so the drag/click/type waits (e.g. hold_content(app, play)) stay in sync with the timeline they wait on.
  • 6 helpers measured the consumed dwell in wall-clock (get_time_usec) then padded a content-time remainder — a mode mismatch that over-padded the caption tail 2–4× under recording. They now anchor in captured frames (record_frame_count) and pad via hold_remainder_content(app, dwell, started_frames).
  • Comments updated (sleephold); the PACING-vs-POLL distinction is unchanged — pacing beats are content-time holds, effect-waits stay polled (wait_* / record_check_*).

Poll-machinery sleeps (READY/FRAME/RECORD poll intervals) are intentionally left as wall-clock.

Validation

No test impact — no test_*.das uses the voice helpers (recording-driver-only). Re-recorded edit_external_tour (drag + combo + type + hold, voiced, self-verifying):

exit 0   (all record_check_* pass -> content-time waits match the synth timelines)
dropped 0
2726 frames / 30 fps = 90.867 s exactly

🤖 Generated with Claude Code

Follow-up to the lockstep-recording sweep (#189): convert the playwright
voice/recording helpers from wall-clock sleep() pacing to content-time holds,
so beat lengths match the intended milliseconds regardless of the app's render
rate. #189 did this for say(); this finishes the manual-hold drivers it flagged.

Why it matters: under lockstep the recorder captures EVERY rendered frame at a
fixed 1/fps delay, so a wall-clock sleep(ms) yields render_rate/fps * ms of video
(a 2s caption shows for ~4s on a 60fps-rendering machine) and drifts the caption
off its voiceover. Content-time holds wait for ms/1000*fps captured frames, so the
beat is exactly ms of video and caption/voice/visual stay aligned in the mux.

- 18 pacing sleep(ms) -> hold_content(app, ms) across hold_through_voice,
  drag_through_voice, type_into_voice, combo/toggle/close/click_render/menu_pick.
  hold_content == sleep outside a recording session, so non-recording runs are
  unchanged; the synth mouse/key timelines ride the same fixed clock under
  lockstep, so the drag/click/type waits (e.g. hold_content(app, play)) stay in
  sync with the timeline they wait on.
- 6 helpers measured the consumed dwell in WALL-clock (get_time_usec) then padded
  a content-time remainder -- a mode mismatch that over-padded the tail 2-4x under
  recording. They now anchor in captured frames (record_frame_count) and pad via
  hold_remainder_content(app, dwell, started_frames).
- Comments updated (sleep -> hold; the PACING-vs-POLL distinction is unchanged:
  pacing beats are content-time holds, effect-waits stay polled).

No test impact: no test_*.das uses the voice helpers (recording-driver-only).
Validated by re-recording edit_external_tour (drag + combo + type + hold, voiced,
self-verifying): exit 0 (all record_check_* assertions pass -> content-time waits
match the synth timelines), dropped 0, 2726 frames / 30fps = 90.867s exactly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Playwright recording “voice helper” pacing so that, under lockstep recording, beats are held in content-time (captured-frame count) rather than wall-clock time, keeping captions/actions aligned with the recorded video timeline.

Changes:

  • Replaced multiple pacing sleep(ms) calls in voice helpers with hold_content(app, ms) to hold by captured frames during recording.
  • Switched several helpers from wall-clock “consumed dwell” tracking (get_time_usec) to frame-based anchoring (record_frame_count) and padding via hold_remainder_content.
  • Updated helper comments to reflect the new hold-based pacing semantics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 812 to 816
// Shared body for both say_begin forms — posts imgui_narrate with the targets list (first is
// the anchor, all hard-avoided) and returns the dwell WITHOUT sleeping. In voiceover mode the
// the anchor, all hard-avoided) and returns the dwell WITHOUT holding. In voiceover mode the
// dwell is the wav length (+gap), held by wall-clock duration_s (render fps diverges from the
// capture grid, so a tick budget would drain early and the box would vanish mid-voiceover);
// otherwise read_ms.
@borisbat borisbat merged commit e487988 into master Jun 7, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants