Skip to content

Phase 4 Step 9: Video/audio media elements #50

@thomasnemer

Description

@thomasnemer

Parent: #41

Goal

Implement <video> and <audio> media elements with playback support, prioritizing patent-free AV1+Opus in WebM.

Prerequisites

  • Step 7 (Shadow DOM) — default media controls are rendered via shadow DOM

Implementation

  • HTMLMediaElement API in JS: play(), pause(), currentTime, duration, volume, muted, src, media events (play, pause, ended, timeupdate, etc.)
  • Primary format: AV1+Opus in WebM container (patent-free, privacy-respecting)
  • Audio pipeline: symphonia for decoding, cpal for audio output
  • Video pipeline: dav1d for AV1 decoding, frames uploaded to wgpu texture, double-buffered
  • AV sync: synchronize audio and video streams using presentation timestamps
  • Default media controls rendered via shadow DOM (play/pause button, seek bar, volume, fullscreen)
  • New crate: ie-media
  • New dependencies: symphonia, dav1d, cpal

Tests

  • Media element API tests (play, pause, seek, volume)
  • Audio decoding and playback tests
  • Video frame decoding and texture upload tests
  • AV sync tests
  • Media event firing tests
  • Default controls interaction tests

Acceptance Criteria

  • <video> and <audio> elements play AV1+Opus WebM content
  • HTMLMediaElement JS API works correctly
  • Audio outputs through system audio via cpal
  • Video frames render to wgpu texture with double buffering
  • Default media controls are functional via shadow DOM

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions