feat(audio): add AudioManager session and routing APIs#1108
Merged
Conversation
Process-wide audio session control on AudioManager: session options and management modes (automatic/manual), Android audio session configuration (mode/focus/routing) backed by an AudioSwitch-based manager in the native plugin, Apple speakerphone routing, and speaker output preferences. Platform audio sessions are global to the app process, so this lives on AudioManager rather than Room.
Picks up the aligned audioswitch revision (flutter-webrtc#2084), which resolves the Android build conflict on the shared classpath.
hiroshihorie
requested review from
cloudwebrtc and
xianshijing-lk
as code owners
|
Caution Breaking change detected without major changeset
If this is intentional, please add a changeset with |
There was a problem hiding this comment.
Pull request overview
Introduces first-class, process-wide audio session/routing management via AudioManager, consolidating previously scattered session behavior (track-counting, Hardware, and implicit native defaults) into a single, typed API with native backends on Apple and Android.
Changes:
- Adds typed audio-session configuration (
AudioSessionOptions, per-platform overrides, andAudioSessionManagementMode) plus supporting copy/serialization helpers. - Updates native plugins to let LiveKit own platform audio sessions: iOS/macOS engine-lifecycle observer drives session activation; Android adds an AudioSwitch-based manager for focus/mode/routing.
- Deprecates/rewires legacy entry points (
Hardware, track audio management mixins) to forward throughAudioManager, and updates examples + adds unit tests.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/audio/audio_session_test.dart | Adds unit tests covering session options, copyWith semantics, and serialization. |
| shared_swift/LiveKitPlugin.swift | Disables flutter_webrtc audio management; adds engine-lifecycle observer + new method-channel handlers for Apple session management and speaker routing. |
| pubspec.yaml | Bumps flutter_webrtc dependency to 1.5.1. |
| pubspec.lock | Locks flutter_webrtc to 1.5.1 with updated hash. |
| lib/src/track/remote/audio.dart | Removes legacy remote audio management mixin usage. |
| lib/src/track/local/audio.dart | Removes legacy local audio management mixin usage. |
| lib/src/track/audio_management.dart | Replaces track-counting session management with AudioManager connect/apply and Android stop handling. |
| lib/src/support/value_or_absent.dart | Adds ValueOrAbsent utility to support nullable-field copyWith APIs. |
| lib/src/support/native.dart | Extends Apple configureNativeAudio payload + adds Android/Apple session/routing method-channel APIs and engine-state callback handling. |
| lib/src/support/native_audio.dart | Updates NativeAudioConfiguration.copyWith to use ValueOrAbsent; removes old static presets. |
| lib/src/livekit.dart | Initializes AudioManager defaults and passes Android init configuration when needed. |
| lib/src/hardware/hardware.dart | Deprecates audio-related members and forwards to AudioManager. |
| lib/src/core/room.dart | Ensures audio session start/stop is cleaned up on connect failure; routes speaker toggles via AudioManager. |
| lib/src/audio/audio_session.dart | New public API types for session intent + per-platform configuration. |
| lib/src/audio/audio_manager.dart | Implements the new process-wide audio session manager, including engine-state stream and platform apply logic. |
| lib/src/audio/android_audio_session_adapter.dart | Serializes Android session configuration to method-channel wire format and applies it via native. |
| lib/livekit_client.dart | Exports the new audio session API. |
| example/lib/widgets/controls.dart | Updates example UI to read speaker state from AudioManager. |
| example/lib/pages/room.dart | Updates example to toggle speaker via AudioManager. |
| android/src/main/kotlin/io/livekit/plugin/LKAudioSwitchManager.kt | Adds AudioSwitch-based Android session/focus/mode/routing manager. |
| android/src/main/kotlin/io/livekit/plugin/LiveKitPlugin.kt | Disables flutter_webrtc audio management; wires method-channel handlers to LKAudioSwitchManager. |
| android/build.gradle | Adds JitPack repository and AudioSwitch dependency. |
| .changes/audio-manager-api | Adds a changeset entry documenting the new API. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
17
to
+21
| allprojects { | ||
| repositories { | ||
| google() | ||
| mavenCentral() | ||
| maven { url 'https://jitpack.io' } |
Member
Author
There was a problem hiding this comment.
Thanks for flagging this. A quick bit of context here. flutter_webrtc, which is a direct dependency of this plugin, already declares the JitPack repository at rootProject.allprojects scope, so every consuming app already gets it applied app wide. This block uses plain allprojects with no rootProject. prefix, which is actually a narrower scope, so it does not add anything the host app was not already getting transitively through flutter_webrtc.
We also pin the same audioswitch commit that flutter_webrtc and the LiveKit Android SDK use, so dependency resolution stays consistent across the whole stack. Keeping our own declaration here just makes the plugin self sufficient rather than relying on the transitive one.
Comment on lines
+65
to
+67
| // Audio device/focus/mode routing. Pinned to the same revision used by | ||
| // the LiveKit Android SDK (AudioSwitchHandler). | ||
| implementation 'com.github.davidliu:audioswitch:039a35aefab7747c557242fa216c9ea11743b604' |
Member
Author
There was a problem hiding this comment.
Thanks. This one is a deliberate tradeoff, and it matches the rest of the LiveKit stack. The same audioswitch commit is pinned by both flutter_webrtc and the LiveKit Android SDK, so dependency resolution stays consistent end to end. A JitPack artifact is immutable per commit, so the pin is reproducible in practice, and the main residual risk is JitPack availability rather than the artifact changing under us.
The fork is needed for its CommDeviceAudioSwitch, which provides the API 31 setCommunicationDevice routing path, and that class has no tagged release or MavenCentral coordinate to point at yet. If upstream publishes a release that carries it, we would happily switch over.
Comment on lines
+359
to
+365
| "configureAndroidAudioSession" -> { | ||
| @Suppress("UNCHECKED_CAST") | ||
| val configuration = call.arguments as? Map<String, Any?> ?: emptyMap() | ||
| audioSwitchManager?.configure(configuration) | ||
| audioSwitchManager?.start() | ||
| result.success(null) | ||
| } |
Member
Author
There was a problem hiding this comment.
Good observation. The coupling here is intentional on Android. Unlike iOS, the Android side has no audio engine lifecycle observer to defer activation to, so the session is activated when LiveKit applies its options (at connect, or on an explicit apply) and released on stop or disconnect. That keeps focus and mode ownership tied to a clear start and stop, which is the standard call session pattern.
Separating configure from activate, or adding an activate flag, is a reasonable future enhancement for finer grained manual mode control, and we can revisit it if a concrete need shows up. For this change the current behavior is the intended one, so we will keep it as is for now.
Comment on lines
+798
to
+801
| // Leave sessionActive untrue so cached state still reflects the | ||
| // live session. Flipping it to false here would make a later | ||
| // configureNativeAudio(automatic:) cache-only while the session | ||
| // is in fact still active. |
Relocate AppleAudioCategory / AppleAudioCategoryOption / AppleAudioMode from the internal wire file (support/native_audio.dart) into audio_session.dart, where the public AppleAudioSessionConfiguration already lives. native_audio.dart keeps only the internal NativeAudioConfiguration wire type and its toStringValue serialization, importing the enums from the audio layer. Removes the import-plus-export of native_audio from audio_session.dart. Public API and behavior are unchanged.
The automatic-mode session expressed the speaker preference twice: through the audio mode (videoChat for speaker, voiceChat for receiver) and again through overrideOutputAudioPort. The override hard-routes to the speaker even over a connected headset, so a plain setSpeakerphoneOn(true) ignored the documented headset priority and behaved like a forced speaker. Gate the speaker override on the forced case only (carried as the defaultToSpeaker category option) and otherwise clear it, letting the audio mode and connected devices decide. This matches the Swift SDK, which selects playAndRecordSpeaker/playAndRecordReceiver by mode and never overrides the output port. Removes the now-unused preferSpeakerOutput plumbing from the native observer and apply path. Manual mode keeps its direct route override.
Rename the new (unreleased) AudioManager speaker surface for clarity and to
match the Swift SDK vocabulary:
setSpeakerphoneOn(enable, {forceSpeakerOutput}) -> setSpeakerOutputPreferred(preferred, {force})
get preferSpeakerOutput -> get isSpeakerOutputPreferred
get forceSpeakerOutput -> get isSpeakerOutputForced
Drop the duplicate speakerphoneOn getter (was identical to preferSpeakerOutput).
Update the deprecated Hardware forwards, Room.setSpeakerOn, the example app,
the audio guide, and the tests. AudioSessionOptions.preferSpeakerOutput and the
per-platform config fields keep their names. No behavior change.
The headset-priority fix made the native side ignore the preferSpeakerOutput wire field (the speaker preference now lives entirely in the audio mode). Remove the now-dead field from NativeAudioConfiguration (field, toMap, copyWith) and stop sending it from the resolved Apple policy. The wire-format test now pins its absence.
Comment on lines
+291
to
+292
| await setAndroidAudioSessionConfiguration(config); | ||
| await Native.setAndroidSpeakerphoneOn(policy.preferSpeakerOutput); |
Comment on lines
+125
to
+132
| val device = if (enable) { | ||
| switch.availableAudioDevices.firstOrNull { it is AudioDevice.Speakerphone } | ||
| } else { | ||
| switch.availableAudioDevices.firstOrNull { | ||
| it is AudioDevice.BluetoothHeadset || it is AudioDevice.WiredHeadset || it is AudioDevice.Earpiece | ||
| } | ||
| } | ||
| switch.selectDevice(device) |
setSpeakerphoneOn(true) explicitly selected the speaker device even when a wired/Bluetooth headset was connected, forcing the speaker and ignoring the documented headset priority. Select a connected headset first and fall back to the speaker only when none is present (earpiece when the speaker is not preferred). This mirrors the iOS fix and removes the forced-speaker routing the media preset would otherwise apply on every options change.
|
|
||
| ## Per platform overrides | ||
|
|
||
| When the presets are not enough you can pin exact platform values. Anything you do not set falls back to the preset behavior. |
…erence Remove the redundant Room-level speaker persistence. The deprecated Room.setSpeakerOn forwards straight to AudioManager.instance.setSpeakerOutputPreferred without writing to roomOptions, the Room.speakerOn getter reports the process-wide AudioManager state, and the publish/unpublish replay is removed because the iOS engine observer and the Android audioswitch handler already re-assert the cached prefer and force values, neither of which is reset by track publish. A construction-time RoomOptions speakerOn is bridged into AudioManager once at connect so that default still applies.
The README Android audio-modes section still told users to configure audio through flutter_webrtc, which LiveKit now disables. Replace it with the AudioManager media-mode equivalent. Reword the per-platform override note in docs/audio.md so it no longer implies unset override fields merge with the preset. Drop two semicolons from prose comments.
AudioSwitch applies the audio mode, focus, and attributes at activate() time, so changing them on an already active switch (for example switching from communication to media mid session) did not take effect until the next restart. configure() now detects activate-time config changes and, when the session is active, cycles deactivate() and activate() to apply them, then reasserts speaker routing.
The construction-time RoomOptions speakerOn was re-applied to AudioManager on every Room.connect(), so reusing a Room across a manual disconnect and connect after a runtime speaker change would revert it. Bridge it only on the first connect so the runtime preference owned by AudioManager wins.
…undary Add an Audio processing section explaining how AudioManager spans both the audio session (this PR) and the runtime audio processing options (per-track DSP plus engine-wide state read-back), and how the communication and media intents pair with AudioProcessingOptions. Note in the iOS platform notes that receiver routing and forced speaker both require the playAndRecord category.
Comment on lines
+164
to
+170
| Future<void> setAudioSessionOptions(AudioSessionOptions options) async { | ||
| _hasExplicitRuntimeOptions = true; | ||
| _syncSpeakerPreferenceFromOptions(options); | ||
| _forceSpeakerOutput = false; | ||
| _options = options; | ||
| await applyCurrentAudioSessionOptions(); | ||
| } |
Comment on lines
+340
to
+341
| bool _speakerPreferenceForOptions(AudioSessionOptions options) => | ||
| options.apple?.preferSpeakerOutput ?? options.preferSpeakerOutput; |
Comment on lines
+182
to
+203
| private fun applyConfiguration(switch: AbstractAudioSwitch) { | ||
| switch.manageAudioFocus = manageAudioFocus | ||
| switch.audioMode = audioMode | ||
| switch.focusMode = focusMode | ||
| switch.audioStreamType = audioStreamType | ||
| switch.audioAttributeUsageType = audioAttributeUsageType | ||
| switch.audioAttributeContentType = audioAttributeContentType | ||
| switch.forceHandleAudioRouting = forceHandleAudioRouting | ||
| } | ||
|
|
||
| private fun applySpeakerRouting(switch: AbstractAudioSwitch) { | ||
| switch.setPreferredDeviceList(preferredDeviceList) | ||
| val forcedSpeaker = if (speakerOutputForced) { | ||
| switch.availableAudioDevices.firstOrNull { it is AudioDevice.Speakerphone } | ||
| } else { | ||
| null | ||
| } | ||
| // AudioSwitch selections are sticky. Use them only for forced speaker output. | ||
| // Clearing the selection lets the preferred-device list handle normal routing | ||
| // and headset hot-plug priority. | ||
| switch.selectDevice(forcedSpeaker) | ||
| } |
hiroshihorie
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds first-class, process-wide audio session and routing control through
AudioManageron iOS and Android. LiveKit owns the platform audio session by default, while apps that need exact platform behavior can switch to manual mode and apply typed session configs.API and behavior
AVAudioSessionfrom engine lifecycle events. Listen-only playout usesplayback; recording usesplayAndRecord.LKAudioSwitchManager.setAudioSessionOptions(...)anddeactivateAudioSession()switchAudioManagerto manual mode.setAudioSessionManagementMode(AudioSessionManagementMode.automatic)hands lifecycle control back to LiveKit.AudioSessionOptions.communication()andAudioSessionOptions.media()pre-fill Apple and Android configs, with per-platform overrides applied verbatim in manual mode.AudioManager.instance.setSpeakerOutputPreferred(...)owns speaker preference and forced speaker routing. Wired and Bluetooth devices still win unlessforce: true.Compatibility
Hardwareaudio members andRoom.setSpeakerOn(...)are deprecated forwarders toAudioManager.flutter_webrtcnative audio-session management is disabled so LiveKit has one owner for the session.bypassVoiceProcessingnow only controls WebRTC voice processing; it no longer changes the session intent.Docs and tests
docs/audio.mdand updates the README audio sections.dart analyze,flutter test test/audio/audio_session_test.dart, andflutter test --reporter compact.