Data tracks

[pblazej]

Adds data tracks to the Swift SDK, implemented on top of the Rust data-track engine via UniFFI (livekit-uniffi-xcframework 0.1.8).

API

// Publish + push
let track = try await room.localParticipant.publishDataTrack(name: "telemetry")
try track.tryPush(frame: DataTrackFrame(payload: data))          // non-blocking
try await track.send(contentsOf: frames)                          // AsyncSequence, drop-on-full by default

// Optionally declare what the frames are, and make the schema resolvable
let schema = DataTrackSchemaId(name: "telemetry.v1", encoding: .protobuf)
try await room.localParticipant.defineSchema(schema, definition: protoSource)
let typed = try await room.localParticipant.publishDataTrack(
    name: "typed",
    options: DataTrackPublishOptions(schema: schema, frameEncoding: .protobuf)
)

// Subscribe + receive
let stream = try await remoteTrack.subscribe(bufferSize: 16)
for await frame in stream { ... }                                 // DataTrackStream is an AsyncSequence

// A subscriber can resolve the schema its publisher declared
if let declared = remoteTrack.info.schema {
    let definition = try await room.localParticipant.getSchema(declared, publishedBy: remoteTrack.publisherIdentity)
}

Remote publish/unpublish surface on RoomDelegate and ParticipantDelegate; subscribed tracks are on RemoteParticipant.dataTracks, keyed by track name. Everything is Objective-C compatible (DataTrackObjCTests covers the surface).

Design decisions

Rust owns the protocol, Swift owns the session. The UniFFI managers (LocalDataTrackManager/RemoteDataTrackManager) implement the publish/subscribe state machine, packetization, and E2EE. The SDK contributes what only it has: the signal connection, the WebRTC data channels, participant identity, and delegate fan-out. Concretely, a session-scoped DataTracks coordinator (one per Room, Room+DataTrack.swift) feeds SFU signal messages into the managers, forwards their outbound requests through SignalClient (serialized, so publish/unpublish ordering is preserved), and pumps packets between the managers and the DTP data channel.

Thin public wrappers over the bindings. The generated bindings are internal imported and every public type (LocalDataTrack, RemoteDataTrack, DataTrackFrame, DataTrackStream, DataTrackInfo, schema/encoding types, error enums) is a small SDK-owned wrapper (~900 lines total). Tradeoff considered: exposing the generated types directly would save the layer, but pins the public API to regenerated code we don't control, and UniFFI emits Swift-only structs — NSObject wrappers are what makes the API reach Objective-C. The wrappers also carry the SDK idioms (lowerCamel error enums, Track.Sid-style identifiers, delegate naming, value semantics).

Publication lifetime follows the handle (RAII). The returned LocalDataTrack is the publication: releasing the last reference unpublishes it, as does calling unpublish(). This matches rust-sdks, where the handle drops the publication. Consequently there are no local publish/unpublish delegate events — Rust dispatches room events only for remote data tracks, and the handle (isPublished, waitForUnpublish()) is the local observer. withDataTrack(name:body:) scopes a publication to a block for the common case.

Retain-cycle handling at the FFI boundary. UniFFI callback interfaces hold their Swift delegate strongly (no weak references over FFI). A dedicated ManagerDelegate object references the Room and coordinator weakly, making it the designated weak link so manager → delegate → room → manager cycles can't leak (Room+DataTrack.swift documents the shape).

Reconnect semantics. Quick reconnect preserves publications via SyncState.publishDataTracks (the managers replay their publish responses); full reconnect republishes and re-asserts subscriptions. The DataTracks subsystem survives reconnects (channels are swapped in) and is torn down only on real disconnect, which also unpublishes remote tracks and notifies delegates. The publisher-channel readiness gate is re-armed — not failed — on teardown, so a publishDataTrack issued mid-reconnect waits for the new channel instead of racing a dead transport. A local full reconnect recreates participants but not their data tracks: those are detached silently and re-attached, so no spurious unpublish is reported.

When a remote publisher full-reconnects, the Rust manager treats the republication as a SID reassignment: the subscriber's existing RemoteDataTrack survives with its SID rewritten in place and active subscriptions transparently re-requested — no unpublish/republish events fire. This is why RemoteParticipant.dataTracks is keyed by name, not SID (Rust documents SIDs as unstable across reconnects; client-sdk-js keys by name for the same reason).

Backpressure. tryPush is non-blocking and throws queueFull, handing the rejected frame back so it can be retried (parity with Rust's PushFrameError::into_frame). send(contentsOf:) defaults to dropping frames when the queue is full (DTP is lossy by design — unbounded buffering would trade a dropped frame for unbounded latency). On the channel, DataTrackFrameSender meters packets out on buffered-amount events against an 8 KiB low-water mark, keeping send latency bounded while letting a frame of any size stream out: at most one frame waits while another drains, a newer frame evicts the waiting one (drop-oldest), and frames are handled whole so a partial frame is never left on the wire. This mirrors DataChannelSender in rust-sdks; client-sdk-js instead blocks the producer, which isn't available here (the producer is a fire-and-forget FFI callback with no backpressure channel). DataTrackFrameSenderTests pins these semantics, including the cross-SDK divergence. Subscribe-side buffer is caller-configurable (subscribe(bufferSize:)).

E2EE. When the room has E2EE configured, the same key provider drives data-track frame encryption via UniFFI EncryptionProvider/DecryptionProvider shims; DataTrackInfo.usesE2ee reflects it. Unlike data-channel payloads (a per-message property), data-track encryption is a track-level protocol property subscribers key their decryption on, so it can't be consulted per frame: the publishing manager is built on the first publish and captures the toggle then, which is late enough for a setE2EEEnabled(true) issued after connecting to apply. Toggling between publishes does not, and Room.setE2EEEnabled documents that. The cryptor resolves Room.e2eeManager on each call rather than capturing it at connect, so a manager assigned late is picked up for both encryption and decryption, and a missing one throws instead of silently sending plaintext.

Protocol update

The protocol submodule moves v1.45.8 → v1.50.4 (regenerated with make proto). Two reasons:

• DataTrackInfo gained frame_encoding and schema after v1.45.8. Because handleParticipantUpdate re-serializes the parsed ParticipantInfo to hand the managers raw bytes, the older protocol silently stripped both fields before Rust saw them — declared metadata never reached subscribers. (The SFU was echoing it correctly; this was ours.)
• It brings DataBlob/StoreDataBlobRequest/GetDataBlobRequest, which defineSchema/getSchema are built on.

The general hazard remains and is worth a follow-up: any ParticipantInfo field newer than the pinned protocol is lost on the way to the managers. Threading the received wire bytes instead of re-serializing removes both the loss and a redundant encode.

Size impact

App Size job measured 16.28 MB for the SDK's uncompressed .app delta before the protocol update; the budget moves 16 → 16.5 MB to leave room for it. That figure includes the nanopb protocol migration from main, which cut 1.8 MB; the data-track share is the 0.0.6 → 0.1.8 UniFFI bump plus this integration (+0.4 MB: compiled bindings at the dead-strip floor plus the Rust dylib growth).

Isolating data tracks behind a Swift package trait was evaluated and rejected for now: the core SDK already depends on LiveKitUniFFI (tokens, log forwarding), both UniFFI components live in one dylib, and the bindings are one module — a trait could only gate the wrapper layer that dead-strip already removes. Revisit if the Rust side ships a feature-split artifact.

Known upstream issue

LocalDataTrack.tryPush cannot surface the FFI's own error: PushFrameErrorReason is declared in the livekit_datatrack UniFFI component while try_push lives in livekit_uniffi, and the cross-component error lift fails, throwing UniffiInternalError.bufferOverflow instead. Left unhandled this leaked an FFI-internal type through the public API and broke send(contentsOf:)'s drop-on-full policy, which inspects the reason. The wrapper recovers the applicable reason from isPublished; the typed catch remains first and takes over once the bindings are fixed. Worth filing against livekit-uniffi.

Not in this PR

• tryPush hands the rejected frame back, but the FFI still drops Rust's own PushFrameError payload.
• Remote pipeline options (max_partial_frames) aren't exported by UniFFI yet.
• Follow-ups filed from review: sharing the buffering state machine with DataChannelPair, and threading wire bytes through the signal path instead of re-serializing parsed join/participant updates.

Testing

Two unit suites run without a server: DataTrackFrameSenderTests covers the outbound drain, and DataTrackSendTests covers send(contentsOf:)'s queue-full policy against a sink that rejects on demand — deliberately not by saturating a live pipeline, which made the outcome depend on how fast the SFU drained.

End-to-end tests against livekit-server --dev across six suites: publish/subscribe/roundtrip (small + multi-packet frames), publish options and schema/encoding metadata round-tripping through the SFU, schema definition storage and resolution, error cases (duplicate name, unauthorized, disconnected, push after unpublish, saturated queue), E2EE on, off, and toggled after connect, delegate events, unpublish paths, RAII handle release, withDataTrack scoping including cancellation, subscribe buffer clamping and drop-oldest eviction, concurrent multi-track push and 64-track publish, plus lifecycle: pre-join publications, publish during a full reconnect, publisher full-reconnect SID reassignment, quick-reconnect sync state, remote tracks surviving the local client's full reconnect, and room moves. DataTrackObjCTests covers the Objective-C surface including NSError bridging and value semantics.

DataTrackStream only ends when its track is unpublished, so every read goes through bounded next(within:)/collect(_:) helpers — a lost frame on an unreliable channel should fail a test, not hang the job. The repeated two-room publish/subscribe preamble lives in a withPublishedDataTrack fixture; tests that control when the publish happens keep driving withRooms directly, since that timing is what they exercise.

Schema definitions are stored as participant data blobs, which the server keeps behind enable_participant_data_blob — enabled for CI, and documented in AGENTS.md for local runs.

[github-actions]

⚠️ This PR does not contain any files in the .changes directory.

[ladvoc]

LocalDataTrack.tryPush cannot surface the FFI's own error

Let's discuss this more, would like to better understand what is going on.

[ladvoc]

LGTM, amazing work @pblazej! Left a few comments/suggestions, but nothing blocking.

[ladvoc]

Here's my understanding, @pblazej correct me if I'm wrong. In GC languages, you don't want to rely on destructors for RAII since there is a non-deterministic timing gap between when the object is no longer referenced and when its destructor is run. Swift uses ARC (automatic reference counting) rather than GC, which means objects are destroyed as soon as the reference count reaches zero. The difference between this and Rust is there is not a 1:1 correlation with scope, so it couldn't be used for something like a mutex guard or accessing raw memory. For the latter, Switch uses this closure pattern to ensure you can only access a pointer within the scope of a closure. However, for the data track use case, do we really need publication to be linked exactly to scope? Maybe relying on deinit is ok in this case?

[pblazej]

Thanks @ladvoc for having an in-depth look! 2 highlights:

• RAII/memory management - I don't think this pattern is wrong per se, it's more about consistency with other APIs, surprising behavior/side effects; we can treat it as "low-level" difference between SDKs/langs, simialarly to cancellation semantics, etc.
• (Re)encoding - yes I think it can be improved like this (da72781) which was my initial idea, probably got lost after multiple iterations
  • it's just a separate notification path, nothing is enforced here (to handle raw messages for all paths etc.)