Skip to content

feat(sampler): play bitmap video at the source fps on the Ultimate Audio sampler#48

Merged
kfox merged 1 commit into
mainfrom
feat/sampler-video-fps
Jun 28, 2026
Merged

feat(sampler): play bitmap video at the source fps on the Ultimate Audio sampler#48
kfox merged 1 commit into
mainfrom
feat/sampler-video-fps

Conversation

@kfox

@kfox kfox commented Jun 28, 2026

Copy link
Copy Markdown
Owner

What

Follow-up to the Ultimate Audio sampler integration (#46): re-tune the default frame rate for bitmap (hires/mhires) video whose audio runs through the FPGA PCM sampler, plus a small audio write-ahead-lead hardening.

1. Source-rate playback for sampler video

The sampler plays off the C64 bus and its presence forces the bus-clean REU-staged (bank-swap) video path. So sampler-audio bitmap video needs neither:

  • the 4-bit $D418 DAC's 20 fps cap (which exists only because the DAC's NMI + ring writes compete with frame uploads for the bus), nor
  • the muted half-rate (30/25) cap (host-DMA tear avoidance).

_frame_push_default_fps gains an off_bus_audio path that uncaps to the system rate (60 NTSC / 50 PAL) — but as a poll ceiling. Because VideoScene dedups (it re-pushes only when the source yields a new frame), the effective push rate equals the source video's own fps:

source effective push
24 fps clip 24/s
30 fps clip 30/s
60 fps clip 60/s

i.e. faithful source-rate playback capped at the VIC refresh — no artificial cap, no dropped frames, no wasted re-pushes. DAC video stays at 20; muted bitmap stays at 30/25; the non-sampler paths are untouched.

2. Lead-headroom decouple (sampler audio)

Decouple the sampler's startup prebuffer (stays 0.5 s → prompt start) from its runtime write-ahead lead target (0.5 s → 1.0 s). The lead is buffer depth, not A/V latency (video tracks the audio read head), so a deeper target rides out heavier PyAV decode stalls without shifting sync.

3. Diag harness

scripts/diags/run_and_capture.py gains an --app-arg passthrough (e.g. --app-arg=-v to surface the sampler write-ahead-lead telemetry) and tees the app log; HDMI captures now downscale to ~960 px before write.

Verification

  • make check green (ruff + mypy --strict + pyright + 1664 unit tests, incl. new off_bus_audio helper + build_scene wiring cases and prebuffer/lead-decouple cases).
  • Verified on real U64 hardware via HDMI capture: real ≤30 fps content pushes at its source rate with no added shimmer, audio stays clean at a genuine 60/s push, and the lead floor on a heavy 4K clip roughly doubled with the decouple.

Notes / follow-up

  • Continuous-motion shimmer scales with push rate and appears only on true >30 fps sources — that's the separate, pre-existing unsynced-bank-swap-timing issue, not this fps default.
  • A distinct A/V-sync-on-heavy-clips investigation is queued as a follow-up: the lag is supply-side (full-resolution 4K decode can't produce frames in real time), to be measured via software instrumentation (audio-clock vs displayed-frame PTS) — orthogonal to this PR.

…dio sampler

The Ultimate Audio FPGA sampler plays off the C64 bus and forces the bus-clean
REU-staged (bank-swap) video path, so sampler-audio bitmap video needs neither
the 4-bit DAC's 20fps cap nor the muted half-rate (30/25) cap. Give
_frame_push_default_fps an off_bus_audio path that uncaps to the system rate
(60/50) as a poll *ceiling*; because VideoScene dedups (re-pushes only on a new
source frame), the effective push rate then equals the source video's own fps
(a 24fps clip pushes 24/s, a 60fps clip 60/s) -- faithful source-rate playback
capped at the VIC refresh, no artificial cap. DAC video stays at 20; muted
bitmap stays at 30/25.

Verified on real U64 hardware via HDMI capture: real <=30fps content pushes at
its source rate with no added shimmer, and audio stayed clean at a genuine 60/s
push. (Continuous-motion shimmer scales with push rate and shows only on true
>30fps sources -- the separate unsynced-bank-swap-timing issue, not the fps
default.)

Also decouple the sampler's startup prebuffer from its runtime write-ahead lead:
the prebuffer stays 0.5s (prompt start) while the lead target rises to 1.0s. The
lead is buffer depth, not A/V latency (video tracks the read head), so a deeper
target rides out heavier PyAV decode stalls without shifting sync. HW-measured:
a heavy 4K clip's lead floor roughly doubled (~9KB -> ~21KB) going 0.5s -> 1.0s.

diags: run_and_capture.py gains an --app-arg passthrough (e.g. -v to surface the
sampler write-ahead-lead telemetry) and tees the app log to out/; HDMI captures
now downscale to ~960px before write.

Docs (architecture/caveats/usage), the example TOMLs, and unit tests updated.
@kfox kfox merged commit 34912f9 into main Jun 28, 2026
5 checks passed
@kfox kfox deleted the feat/sampler-video-fps branch June 28, 2026 02:14
@codecov

codecov Bot commented Jun 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.52%. Comparing base (ac0c58e) to head (0d567b7).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #48      +/-   ##
==========================================
+ Coverage   79.49%   79.52%   +0.03%     
==========================================
  Files          69       69              
  Lines       13491    13496       +5     
  Branches     2013     2014       +1     
==========================================
+ Hits        10724    10733       +9     
+ Misses       2303     2300       -3     
+ Partials      464      463       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant