feat(core): capture.data runtime inputs + host-supplied audio producer for capture templates - #45
Merged
Merged
Conversation
…ture templates
- capture.data: JSON-injected and passed to initVos as deps.data in both
capture-video and capture-thumbnail modes. Data-dependent compositions
(constant program, inputs in ctx.data) previously rendered with baked
config data only — server captures of such compositions were wrong.
- capture.audioProducerCode: host-supplied script defining
window.__vosAudioProducer__({ data, duration, sampleRate }) ->
AudioBuffer | null. The template runs it before the frame loop and muxes
the buffer as the output's audio track — AAC for mp4 with an automatic
Opus fallback (AAC encode is unavailable on some platforms), Opus for
webm. The engine stays schema-agnostic: how the producer interprets data
is the host's concern. No producer = zero audio code emitted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two additive capabilities for the capture templates:
capture.data— JSON-injected into the page and passed toinitVosasdeps.data, in both capture-video and capture-thumbnail modes. Compositions built on the interpreter pattern (constant program, inputs inctx.data) previously captured with only the config's baked data — server captures of data-dependent compositions were silently wrong. Runtime docs already statedeps.dataoverrides baked data; capture modes now honor that contract.capture.audioProducerCode(capture-video) — a host-supplied script that defineswindow.__vosAudioProducer__ = async ({ data, duration, sampleRate }) => AudioBuffer | null. The template runs it before the frame loop (a producer failure aborts early instead of wasting the render) and muxes the returned buffer as the output's audio track: AAC for mp4 with an automatic Opus fallback viacanEncodeAudio(AAC encode is unavailable on some platforms, e.g. desktop Linux), Opus for webm. Returningnullcaptures video-only.The engine stays schema-agnostic: it imposes no audio model — how a producer interprets
data(mixing, envelopes, sources) is entirely the host's concern, consistent with "add capabilities, not conventions."Compatibility
Fully backward compatible: without
capture.datathe page emitsdeps.datauntouched behavior (nullsentinel, no assignment); without a producer, zero audio code is emitted — byte-wise the only change to existing templates is the null data sentinel line.Tests
5 new template assertions (data injection in both modes, null default, producer embedding + mux plumbing, aac/opus preference matrix, no-producer emits no audio code) — 128 pass across the core suite; typecheck, lint, build clean. Changeset: minor.
🤖 Generated with Claude Code