Skip to content

render: parallelize frame capture across a Chrome tab pool (frames are pure functions of t) #3

Description

@darioalessandro

Summary

Frame capture is strictly sequential today, but the engine's own contract makes every frame independent: templates are pure functions of (data, t) (enforced by design — no CSS animations, no timers, __sceneSeek(t) is idempotent). That makes rendering embarrassingly parallel, and on an M-class laptop we're leaving most of the cores idle during the slowest part of the pipeline.

Where the time goes

crates/videoeditor/src/render.rs (render_web_scene):

for i in 0..total_frames {
    let t_ms = i as f64 / ep.meta.fps as f64 * 1000.0;
    chrome.seek(t_ms)?;                                   // Runtime.evaluate over one WS
    chrome.screenshot(&frames_dir.join(...))?;            // Page.captureScreenshot over the same WS
}

One Chrome (one process, one page target, one synchronous tungstenite WebSocketcrates/videoeditor-chrome/src/lib.rs) does seek → screenshot → base64-decode → fs::write for every frame, one at a time. A 1080×1920 scene at 30fps means hundreds of round-trips with zero overlap: while Chrome rasterizes + PNG-encodes, the Rust side is blocked on the socket; while Rust decodes base64 and writes the PNG, Chrome is idle. Scenes are also rendered one after another (run() loop).

Proposal: a small tab pool, frames fanned out across it

  1. Keep one Chrome process (launch is already guarded against profile races), but open N page targets via /json/new — each with its own WebSocket. Chrome gives each target its own renderer process, so rasterization genuinely parallelizes.
  2. Each worker tab does the same navigate(template) + init_scene(data_json) once, then pulls frame indices from a shared queue (work-stealing not needed — a chunked AtomicUsize counter is enough since frames cost roughly the same).
  3. rayon fits naturally on the Rust side (par_iter over frame indices with a per-thread tab via a small pool), or a plain std::thread::scope with min(available_parallelism, jobs) workers — either way the base64-decode + fs::write also moves off the single thread. Chrome is !Sync (stateful WS), so it's one tab handle per worker, never shared.
  4. Scene-level parallelism composes on top for videoeditor render <dir> with many scenes (each scene claims a tab), but frame-level is the win that also helps the common --scene X iteration loop.

Determinism is untouched: same (data, t) → same pixels, and output filenames f_{i:05}.png are index-derived, so capture order is irrelevant. encode_frames still runs once after the scene completes.

Details to keep in mind

  • sceneWarnings() should run once (on one tab), not N times, or warnings print N-fold.
  • Progress line (\r frames: i/total) needs an atomic counter instead of loop order.
  • Pool size: cap it (--jobs, default something like min(cores - 2, 8)) — each renderer target costs real memory at 1080×1920, and past ~8 tabs Page.captureScreenshot throughput tends to flatten before memory does. Worth a quick benchmark table in the PR.
  • Multiple Chrome processes (instead of tabs) would need the profile-dir nonce fixed first: the temp profile is keyed on std::process::id() of videoeditor (videoeditor-chrome/src/lib.rs), so two Chrome::launches from one run would collide. Tabs avoid this entirely; if processes ever become preferable, add a per-launch nonce.
  • video-clip scenes already bypass Chrome (ffmpeg) and are unaffected.

Expected impact

Anecdatum from a real episode (M4 Max, 1080×1920@30): a 5-scene render is a coffee break today, with 14+ performance cores never breaking a sweat. Even a conservative 4–6× on frame capture changes the director loop from "render, wait, review" to near-interactive.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions