recording: pace voice helpers in content-time (lockstep follow-up)#191
Merged
Conversation
Follow-up to the lockstep-recording sweep (#189): convert the playwright voice/recording helpers from wall-clock sleep() pacing to content-time holds, so beat lengths match the intended milliseconds regardless of the app's render rate. #189 did this for say(); this finishes the manual-hold drivers it flagged. Why it matters: under lockstep the recorder captures EVERY rendered frame at a fixed 1/fps delay, so a wall-clock sleep(ms) yields render_rate/fps * ms of video (a 2s caption shows for ~4s on a 60fps-rendering machine) and drifts the caption off its voiceover. Content-time holds wait for ms/1000*fps captured frames, so the beat is exactly ms of video and caption/voice/visual stay aligned in the mux. - 18 pacing sleep(ms) -> hold_content(app, ms) across hold_through_voice, drag_through_voice, type_into_voice, combo/toggle/close/click_render/menu_pick. hold_content == sleep outside a recording session, so non-recording runs are unchanged; the synth mouse/key timelines ride the same fixed clock under lockstep, so the drag/click/type waits (e.g. hold_content(app, play)) stay in sync with the timeline they wait on. - 6 helpers measured the consumed dwell in WALL-clock (get_time_usec) then padded a content-time remainder -- a mode mismatch that over-padded the tail 2-4x under recording. They now anchor in captured frames (record_frame_count) and pad via hold_remainder_content(app, dwell, started_frames). - Comments updated (sleep -> hold; the PACING-vs-POLL distinction is unchanged: pacing beats are content-time holds, effect-waits stay polled). No test impact: no test_*.das uses the voice helpers (recording-driver-only). Validated by re-recording edit_external_tour (drag + combo + type + hold, voiced, self-verifying): exit 0 (all record_check_* assertions pass -> content-time waits match the synth timelines), dropped 0, 2726 frames / 30fps = 90.867s exactly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR updates the Playwright recording “voice helper” pacing so that, under lockstep recording, beats are held in content-time (captured-frame count) rather than wall-clock time, keeping captions/actions aligned with the recorded video timeline.
Changes:
- Replaced multiple pacing
sleep(ms)calls in voice helpers withhold_content(app, ms)to hold by captured frames during recording. - Switched several helpers from wall-clock “consumed dwell” tracking (
get_time_usec) to frame-based anchoring (record_frame_count) and padding viahold_remainder_content. - Updated helper comments to reflect the new hold-based pacing semantics.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
812
to
816
| // Shared body for both say_begin forms — posts imgui_narrate with the targets list (first is | ||
| // the anchor, all hard-avoided) and returns the dwell WITHOUT sleeping. In voiceover mode the | ||
| // the anchor, all hard-avoided) and returns the dwell WITHOUT holding. In voiceover mode the | ||
| // dwell is the wav length (+gap), held by wall-clock duration_s (render fps diverges from the | ||
| // capture grid, so a tick budget would drain early and the box would vanish mid-voiceover); | ||
| // otherwise read_ms. |
This was referenced Jun 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the lockstep-recording sweep (#189). #189 converted
say()to content-time and flagged the manual-hold drivers as a mechanical follow-up; this is that sweep, done centrally in the playwright helpers so every driver (here, and in the dependent implot / node-editor repos that call these same helpers) benefits at once.Why
Under lockstep the recorder captures every rendered frame at a fixed
1/fpsdelay. A wall-clocksleep(ms)therefore yieldsrender_rate/fps * msof video — a 2 s caption shows for ~4 s on a 60 fps-rendering machine — and drifts the caption off its (real-time) voiceover. Content-time holds wait forms/1000*fpscaptured frames, so the beat is exactlymsof video and caption / voice / visual stay aligned in the mux.Changes (
widgets/imgui_playwright.das)sleep(ms)→hold_content(app, ms)acrosshold_through_voice,drag_through_voice,type_into_voice,combo_pick_voice,toggle_tree_voice,close_button_voice,click_render_voice,menu_pick_voice.hold_content == sleepoutside a recording session, so non-recording runs are unchanged. The synth mouse/key timelines ride the same fixed clock under lockstep, so the drag/click/type waits (e.g.hold_content(app, play)) stay in sync with the timeline they wait on.get_time_usec) then padded a content-time remainder — a mode mismatch that over-padded the caption tail 2–4× under recording. They now anchor in captured frames (record_frame_count) and pad viahold_remainder_content(app, dwell, started_frames).sleep→hold); the PACING-vs-POLL distinction is unchanged — pacing beats are content-time holds, effect-waits stay polled (wait_*/record_check_*).Poll-machinery sleeps (READY/FRAME/RECORD poll intervals) are intentionally left as wall-clock.
Validation
No test impact — no
test_*.dasuses the voice helpers (recording-driver-only). Re-recordededit_external_tour(drag + combo + type + hold, voiced, self-verifying):🤖 Generated with Claude Code