AetherEngine 2.5.0 — Live TV / DVR

Live TV / DVR (timeshift)

AetherEngine 2.5.0 adds unbounded live playback and in-session timeshift (DVR), native-first across formats. No breaking API change; existing load(url:) / load(source:) callers are unaffected.

Highlights

• Opt-in live + DVR. LoadOptions.isLive enables unbounded live mode. Pass dvrWindowSeconds (e.g. 1800) to enable in-session timeshift; omit it (nil) for live-only playback where seek() is a no-op. Hosts drive a single scrubber against a session-relative timeline (seconds since first frame), identical across both playback paths.
• Native path (H.264 / HEVC / AV1-with-HW). A forward-only producer cuts segments on the fly and serves a sliding HLS playlist (advancing #EXT-X-MEDIA-SEQUENCE, no #EXT-X-ENDLIST). Timeshift uses AVPlayer's native seekable range.
• Software path (AV1-without-HW / VP9 / MPEG-2 / VC-1). Unbounded live with a disk-spooled, keyframe-indexed PacketRingBuffer for rewind without a network round-trip.
• New API. dvrWindowSeconds, the published liveEdgeTime / seekableLiveRange / isAtLiveEdge / behindLiveSeconds, and seekToLiveEdge(). seek(to:) is reused for DVR (clamped to seekableLiveRange in a DVR session, a no-op in live-only).
• Robustness. AVIOReader endless-feed mode (no synthesized EOF, terminal error only after reconnects are exhausted), #EXT-X-DISCONTINUITY and PTS-offset repair keep the session timeline monotonic across source discontinuities, and a stable generous live #EXT-X-TARGETDURATION avoids the high-bitrate CoreMediaErrorDomain -12888 startup race.

Verification

Live sliding-window memory behavior and behindLiveSeconds accuracy were verified off-device via the aetherctl harness (resident-footprint plateau under the sliding playlist, stable behind-live at real-time pacing). On-device confirmation on Apple TV with a real broadcast feed is recommended before production use.

2.4.0 - Custom IOReader input sources

Custom input sources

A new public IOReader protocol lets hosts play media from any byte source (memory buffers, encrypted-at-rest archives, proprietary containers) instead of only URLs. No breaking API change, existing load(url:) callers are unaffected. Resolves #26.

public protocol IOReader: AnyObject, Sendable {
    func read(_ buffer: UnsafeMutablePointer<UInt8>?, size: Int32) -> Int32
    func seek(offset: Int64, whence: Int32) -> Int64
    func close()
    func cancel()                              // default no-op: unblock a blocked read at teardown
    func makeIndependentReader() -> IOReader?  // default nil: a second independent cursor
}

public enum MediaSource: Sendable {
    case url(URL)
    case custom(IOReader, formatHint: String? = nil)
}

try await engine.load(source: .custom(MyArchiveReader(), formatHint: "mp4"))

load(url:) is retained and forwards to load(source:). Internally the engine attaches the reader to the demuxer's AVFormatContext.pb, the same seam the built-in AVIOReader uses, so no FFmpeg types leak into the public surface.

What works for custom sources

• Both playback paths, video and audio. Seekable readers play on the native (AVPlayer / HLS-remux) and software decode paths. Audio-only custom sources route through the software (FFmpeg) audio path, since AVPlayer is URL-only.
• Forward-only readers (seek returns negative for SEEK_SET/CUR/END) play too, auto-routed to the software decode path.
• Full mid-playback feature set on capable readers. Audio-track switching and background-return reload work for seekable readers (the pipeline rebuilds on the retained reader). Embedded-subtitle selection and scrub-preview thumbnails work for readers that implement the optional makeIndependentReader(); readers that cannot provide a second cursor simply skip those two features.

Two contracts to implement correctly

• cancel() must only unblock the in-flight read, never invalidate the reader (the engine reuses the reader across an internal reload). It is now a protocol requirement with a default no-op, so a host override dispatches correctly through the any IOReader existential.
• makeIndependentReader() returns a fresh reader with its own cursor over the same source (a second file handle, a fresh decrypt context, etc.), or nil if the source cannot (one-shot streams). The engine owns and closes it.

Security note

On the native path the demuxed bytes are re-muxed to cleartext fMP4 and served over a loopback HLS connection to AVPlayer. Fine for encrypted-at-rest archives, worth knowing if the source is encrypted for content protection.

Acknowledgements

Thanks to @strangeliu for the proposal and to @DrHurt for the discussion on #26.

Full Changelog: 2.3.0...2.4.0

2.3.0

New public API for media metadata, plus episode-autoplay playback-reliability fixes. No breaking API change, existing 2.x callers are unaffected.

Changes

• MediaMetadata extracted on every load. The demuxer parses normalized container tags (title, artist, album, albumArtist, with whitespace cleanup) and pulls embedded cover art. The engine publishes it at load time and exposes it through SourceProbe, and aetherctl prints the parsed container metadata in its probe output. Driven by the AetherPlayer media-player work.
• Episode autoplay no longer starts audio before video. The native AVPlayer reused across native-to-native reloads (since 2.2.1) carried its previous rate=1.0 into the next item, so the new episode auto-resumed before the display-criteria handshake and played audio while the panel was still mid Match-Frame-Rate switch. The host now pauses the player across the item swap, so the post-handshake play() gates the start.
• No more mid-playback stall plus A/V desync a minute or two into a stream. SegmentCache evicted already-produced forward segments when AVPlayer did a transient backward refetch (an audio handover or decode flush moved the prune target back), which forced a cache-miss producer restart that re-muxed from a fresh init segment. The forward prune bound is now anchored on the highest stored index so produced-but-unconsumed segments survive the dip, and the restart decision no longer treats a resident segment the producer merely raced past as a pruned gap.

Full changelog: 2.2.2...2.3.0

2.2.2

Playback-clock correctness. The engine now presents a single source-PTS timeline. No breaking API change, existing 2.2.x callers are unaffected.

Changes

• Unified the playback clock onto source PTS. On the native HLS path currentTime previously mirrored AVPlayer's loopback clock (source_pts - playlistShiftSeconds) while sourceTime carried source PTS, forcing every source-timeline consumer (subtitle scheduling, media-segment intro/outro detection, resume reporting) to pick the right one of two clocks. The shift is now folded into the published currentTime, so currentTime == sourceTime on every path (the software and audio paths already ran on source time). Resume and reloadAtCurrentPosition get slightly more accurate as a result, and on a rare imprecise restart seek the reported position now reflects the true landed frame.
• seek(to:) is now source-PTS based and converts to the loopback clock internally (a no-op on the software and audio paths, where the shift is 0). A seek(toSourceTime:) alias exists but is deprecated, since seek(to:) now covers it. sourceTime stays public as a stable alias for callers that want to express source-timeline intent explicitly.

Full changelog: 2.2.1...2.2.2

AetherEngine 2.2.1

Patch release. Playback, audio, and Now-Playing fixes. No public API change, existing 2.2.x callers are unaffected.

Fixes

• Persistent forward-streaming AVIO reader for CDN direct-URL playback (#25). The fragile chunked range reader is replaced with a VLC-style single forward-streaming connection that reconnects with backoff on drops. Waiting on data is now edge-triggered, and the reconnect cap is progress-aware so a stream that keeps advancing is not killed by a transient stall.
• Multichannel audio no longer downmixes to stereo with continuous-audio off (#24). Audio-route capability is sampled after playback settles rather than at readyToPlay, when the HDMI route has not finished negotiating yet. The native path lets AVKit own audio-session activation, and the manual reassert is scoped to the renderer paths that actually need it. (Earlier session-reassert and route-renegotiation attempts in this cycle were disproven on device and reverted.)
• System Now-Playing survives native-to-native reloads (#15). Episode autoplay and audio-track switches reuse the existing native AVPlayer via replaceCurrentItem instead of building a fresh one, which previously blanked the Control Center Now-Playing card on every swap.

See CHANGELOG.md for the release index.

AetherEngine 2.2.0

AetherEngine 2.2.0

New public API: an audio-only playback path. Minor bump, purely additive. Existing 2.1.x callers compile and run unchanged.

Audio-only path

LoadOptions.audioOnly routes a source into a lean audio pipeline that never builds the HLS loopback server, the display layer, or the video producer. Decode is native-first:

• Codecs on the avPlayerCanDecodeAudio whitelist hand the URL straight to a bare AVPlayer (AudioAVPlayerHost).
• Everything else falls back to an FFmpeg decode into AVSampleBufferAudioRenderer (AudioPlaybackHost).

The engine branches load() into the audio path, routes transport (play / pause / seek) to the active host, and tears the host down in stopInternal for a clean handoff back to the video path.

System Now-Playing (tvOS / iOS)

The AVPlayer host owns a persistent per-player MPNowPlayingSession (exposed via audioNowPlayingSession) that stays the active Now-Playing app across a background pause, auto-publishes now-playing info from the player, and carries externalMetadata. The host survives across tracks (no per-track teardown) and does not pause when the app backgrounds, so audio keeps playing with the system overlay live.

All Now-Playing code is gated #if os(tvOS) || os(iOS). The path builds clean on macOS (no system session there), iOS, and tvOS.

Tooling

New aetherctl audio subcommand for audio-path smoke testing: prints the active decoder and final duration, driven under CFRunLoop so end-of-track fires at playback end rather than demux EOF.

Compatibility

Purely additive public API, no breaking changes. Pin from: "2.0.0" continues to pick this up.

2.1.3

Playback fix: rapid play/pause no longer swallowed

Transport state sync. No public API change, existing 2.1.x callers are unaffected.

Fixed

• Rapid play/pause presses no longer get swallowed on the native (AVPlayer) path. The engine never derived its state from the player, so when something other than engine.play() / pause() drove the AVPlayer (a host that keeps AVKit's transport bar active for Control Center skip routing, Control Center itself, or the hardware play/pause button AVKit handles internally), the engine's state went stale and the next togglePlayPause() resolved to the action already in effect, a visible no-op.

How it works now

• NativeAVPlayerHost publishes timeControlStatus; the engine reconciles state (playing / paused) from it, guarded to the steady transport states so loading, seeking, error and idle are never clobbered. waitingToPlayAtSpecifiedRate maps to playing, so the play/pause icon does not flicker on a rebuffer.
• togglePlayPause() decides from the live player rather than the published state, closing the async gap during fast presses.

Full diff: 2.1.2...2.1.3

AetherEngine 2.1.2

Playback fix. Head-of-stream A/V sync. No public API change. Existing 2.1.x callers are unaffected.

Fixed

Audio no longer leads video at the start of a file. On a fresh play (baseIndex 0) the producer snapped the first audio packet onto the video's tfdt (desired 0), which subtracted the audio track's intrinsic start offset from every audio packet. On sources whose first full audio frame lands well past video frame 0 (Cars: EAC3 first frame at +256 ms) this pulled the whole audio track that far ahead of the picture for the entire session, reported as a 256 ms A/V offset in the stats overlay.

Head-of-stream now derives the audio shift from the video's origin shift, so both streams undergo one shared transform and their true source-time relationship is preserved by construction. The audio fragment's tfdt then starts a little after the video's, which fmp4 / AVPlayer represent natively (audio is simply silent for the leading gap). Resume and scrub sessions were unaffected and keep the existing gate-on-video snap (sub-frame residual, part of the HEVC-resume alignment stack).

Verification

Verified on device with Cars: the diagnostic A/V gap now reports 0 ms at head-of-stream (was 256 ms). Build green.

Full changelog: 2.1.1...2.1.2

AetherEngine 2.1.1

FrameExtractor quality pass. Internal only, no public API change. Existing 2.1.0 callers are unaffected.

Fixed / Improved

HDR thumbnails now tone-map correctly. PQ (ST 2084) and HLG stills used to come out too dark / desaturated because the extractor scaled straight to sRGB with no transfer conversion. HDR frames now route through a zscale + tonemap libavfilter graph (BT.2020 PQ/HLG to SDR BT.709 RGBA, hable tone curve); SDR keeps the fast direct sws path. This relies on the avfilter + zimg additions to FFmpegBuild (already pinned by this release).

Faster, lighter remote extraction. A new .stillExtraction demuxer profile gives the extractor's AVIO a random-access shape: no read-ahead prefetch (a scrub discards it on the next seek, and it competed with playback bandwidth), a 1 MB seek chunk, and a small probe budget. Decode fast-flags (skip loop filter, fast decode) cut per-frame CPU on big HEVC keyframes.

Thumbnails fixed on sparse-keyframe HEVC. The thumbnail decode no longer sets skip_frame = NONKEY, which starved the decoder when a seek landed mid-GOP past a lone keyframe and produced nil thumbnails on some HEVC sources.

Verification

Build + full test suite green; leaks --atExit reports 0 leaks across repeated HDR extractions (the per-call filter graph is freed each time); SDR and HDR thumbnail / snapshot all produce images via aetherctl extract.

Known limitation

DV Profile 5 (IPT-PQ, no HDR10 base) thumbnails still render with wrong colours on the software decode path the extractor uses, the same class as the AV1 Profile 10.0 limitation. Full Profile 5 playback is unaffected (it routes through the native AVPlayer path).

Full changelog: 2.1.0...2.1.1

AetherEngine 2.1.0

New public API: off-playback still-image extraction.

Added

FrameExtractor — still CGImages from a media URL, fully isolated from playback.

FrameExtractor decodes through its own FFmpeg context with no contact with the playback pipeline, the HLS loopback server, or shared engine state, so a scrub-preview decode can't perturb the frame on screen. Two modes share one decode core:

• thumbnail(at:maxWidth:) snaps to the nearest keyframe, no forward decode, downscaled to maxWidth (default 320). Cheap and fast, built for scrub previews and Recents lists.
• snapshot(at:maxSize:) decodes forward to the exact PTS at full or maxSize-clamped resolution, built for user-triggered stills.

It is an actor: blocking FFmpeg work runs on a dedicated serial queue off the cooperative pool, the decode context opens lazily on first use, a superseded request cancels the in-flight decode so the latest scrub position wins, results land in a bounded LRU cache (mode-isolated stores, second-bucketed thumbnails), and the context idle-closes after 10 s (the next request reopens lazily). shutdown() is the explicit, permanent teardown that awaits release of the FFmpeg demuxer / codec / sws resources.

// Currently-playing item:
let frames = engine.makeFrameExtractor()           // nil if nothing is loaded
// Arbitrary item (e.g. a Recents row):
let frames = FrameExtractor(url: url, httpHeaders: headers)

await frames.prewarm()                             // optional: hide cold-start
let preview = await frames.thumbnail(at: 612.0)    // CGImage?, nearest keyframe
let still   = await frames.snapshot(at: 612.0)     // CGImage?, frame-accurate
await frames.shutdown()                            // prompt teardown

AetherEngine.makeFrameExtractor() vends an extractor for the currently loaded URL (carrying its HTTP headers). The engine does not retain it; the caller owns its lifecycle.

aetherctl extract subcommand for still extraction plus leak testing (--at, --snapshot, --width, --loops), backed by the same public API. --loops N pairs with leaks --atExit to validate clean decode-context teardown.

Compatibility

Purely additive public API, no breaking changes. Existing 2.0.x callers compile and run unchanged. Pinning from: "2.0.0" already picks this up.

Full changelog: 2.0.2...2.1.0