-
Notifications
You must be signed in to change notification settings - Fork 235
feat(audio): add AudioManager session and routing APIs #1108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
10b03b6
feat(audio): AudioManager audio session management
hiroshihorie 4d7ebdc
chore: add changeset for AudioManager audio session management
hiroshihorie 003f693
chore(deps): bump flutter_webrtc to 1.5.1
hiroshihorie c31ebb9
feat(audio): add session configuration API
hiroshihorie 3e0d3bf
feat(apple): manage audio session from engine lifecycle
hiroshihorie 23f4871
fix(android): serialize audio switch lifecycle
hiroshihorie b5a83a3
refactor(audio): remove stale track counting
hiroshihorie 3e237d2
test(audio): cover session configuration behavior
hiroshihorie 115c43b
docs(audio): note session management migration
hiroshihorie 65e8a0b
adjustments
hiroshihorie e35b036
docs(apple): clarify sessionActive comment on deactivate failure
hiroshihorie f7de7f7
refactor(audio): move Apple audio enums to public audio types
hiroshihorie 2a5830d
docs(audio): add audio session guide and link from README
hiroshihorie 93b8284
fix(apple): honor headset priority for non-forced speaker preference
hiroshihorie 964cbbf
refactor(audio): rename speaker API to setSpeakerOutputPreferred
hiroshihorie c0b14df
refactor(audio): drop unused preferSpeakerOutput from native wire config
hiroshihorie 0b5656c
fix(android): honor headset priority for non-forced speaker preference
hiroshihorie 41d6eb6
fix(audio): honor forced speaker routing on mobile
hiroshihorie 24427e9
style(audio): sort audio session test imports
hiroshihorie a9ebb17
fix(audio): address audio manager review comments
hiroshihorie feacfb4
fix(audio): add explicit session deactivation API
hiroshihorie c755294
refactor(audio): make AudioManager the sole owner of the speaker pref…
hiroshihorie e7588fc
docs(audio): align README with AudioManager and tidy comments
hiroshihorie cc352d9
fix(android): re-apply audio mode on live session reconfiguration
hiroshihorie f363c9e
fix(audio): bridge the legacy room speaker preference only once
hiroshihorie 26f99f6
docs(audio): document audio processing interplay and the iOS force bo…
hiroshihorie 333119c
Merge branch 'main' into hiroshi/audio-manager-api
hiroshihorie a389cd2
refactor(audio): split managed and manual session policies
hiroshihorie 5903edc
docs(audio): document managed session model
hiroshihorie 2475cc3
refactor(audio): trim session public surface
hiroshihorie bf1fc9b
fix(audio): reapply iOS session on engine transitions
hiroshihorie 484f384
refactor(audio): simplify iOS engine session handling
hiroshihorie 11bec41
fix(android): snapshot audio switch configuration
hiroshihorie fc0a6de
refactor(apple): remove unused playout observer state
hiroshihorie ad46e12
refactor(audio): tighten copyWith helper API
hiroshihorie 2f7d38b
fix(api): preserve audio track mixin supertypes
hiroshihorie de23ad0
feat(audio): add noProcessing audio processing preset
hiroshihorie File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| minor type="added" "AudioManager audio session options with engine-driven native lifecycle and platform routing controls" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
322 changes: 322 additions & 0 deletions
322
android/src/main/kotlin/io/livekit/plugin/LKAudioSwitchManager.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,322 @@ | ||
| /* | ||
| * Copyright 2026 LiveKit, Inc. | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package io.livekit.plugin | ||
|
|
||
| import android.content.Context | ||
| import android.media.AudioAttributes | ||
| import android.media.AudioManager | ||
| import android.os.Build | ||
| import android.os.Handler | ||
| import android.os.HandlerThread | ||
| import com.twilio.audioswitch.AbstractAudioSwitch | ||
| import com.twilio.audioswitch.AudioDevice | ||
| import com.twilio.audioswitch.AudioSwitch | ||
| import com.twilio.audioswitch.CommDeviceAudioSwitch | ||
| import com.twilio.audioswitch.LegacyAudioSwitch | ||
|
|
||
| /** | ||
| * Manages the Android platform audio session (audio mode, audio focus, and | ||
| * output routing) for the LiveKit Flutter SDK, built on top of [AudioSwitch]. | ||
| * | ||
| * This is LiveKit's own port of the audio-handling best practices from the | ||
| * LiveKit Android SDK (`AudioSwitchHandler`) and flutter_webrtc | ||
| * (`AudioSwitchManager`), so the Flutter SDK can own the platform audio session | ||
| * directly instead of delegating to flutter_webrtc's native audio management. | ||
| * | ||
| * [AudioSwitch] is not thread-safe, so every interaction with it runs on a | ||
| * single dedicated [HandlerThread]. | ||
| */ | ||
| internal class LKAudioSwitchManager(private val context: Context) { | ||
| // AudioSwitch is not threadsafe, so confine all access to a single long-lived | ||
| // thread. The AudioSwitch instance is recreated per active session, while | ||
| // queued lifecycle work stays serialized on this thread. | ||
| private val thread = HandlerThread("LKAudioSwitchThread").also { it.start() } | ||
| private val handler = Handler(thread.looper) | ||
|
|
||
| private var audioSwitch: AbstractAudioSwitch? = null | ||
| private var isActive = false | ||
|
|
||
| // Configuration. Defaults mirror a communication/VoIP session and match the | ||
| // AudioSwitchHandler defaults in the LiveKit Android SDK. | ||
| private var manageAudioFocus = true | ||
| private var audioMode = AudioManager.MODE_IN_COMMUNICATION | ||
| private var focusMode = AudioManager.AUDIOFOCUS_GAIN | ||
| private var audioStreamType = AudioManager.STREAM_VOICE_CALL | ||
| private var audioAttributeUsageType = AudioAttributes.USAGE_VOICE_COMMUNICATION | ||
| private var audioAttributeContentType = AudioAttributes.CONTENT_TYPE_SPEECH | ||
| private var forceHandleAudioRouting = false | ||
|
|
||
| private var speakerOutputPreferred = true | ||
| private var speakerOutputForced = false | ||
|
|
||
| /** | ||
| * Apply an audio session configuration. Unspecified keys keep their current | ||
| * value. When the session is already active, changes that only take effect at | ||
| * activate() time trigger a deactivate and activate cycle so they apply live. | ||
| */ | ||
| @Synchronized | ||
| fun configure(configuration: Map<String, Any?>) { | ||
| val previous = sessionConfigSnapshot() | ||
| (configuration["manageAudioFocus"] as? Boolean)?.let { manageAudioFocus = it } | ||
| audioModeForName(configuration["androidAudioMode"] as? String)?.let { audioMode = it } | ||
| focusModeForName(configuration["androidAudioFocusMode"] as? String)?.let { focusMode = it } | ||
| streamTypeForName(configuration["androidAudioStreamType"] as? String)?.let { audioStreamType = it } | ||
| usageTypeForName(configuration["androidAudioAttributesUsageType"] as? String)?.let { audioAttributeUsageType = it } | ||
| contentTypeForName(configuration["androidAudioAttributesContentType"] as? String)?.let { audioAttributeContentType = it } | ||
| (configuration["forceHandleAudioRouting"] as? Boolean)?.let { forceHandleAudioRouting = it } | ||
| val sessionConfig = sessionConfigSnapshot() | ||
| val sessionConfigChanged = sessionConfig != previous | ||
| val speakerRouting = speakerRoutingSnapshot() | ||
|
|
||
| handler.post { | ||
| val switch = audioSwitch ?: return@post | ||
| applyConfiguration(switch, sessionConfig) | ||
| // AudioSwitch applies the audio mode, focus, and attributes at activate() | ||
| // time, so a live reconfiguration (e.g. communication to media) needs a | ||
| // deactivate and activate cycle to take effect on an already active | ||
| // session. Reassert speaker routing afterward. | ||
| if (isActive && sessionConfigChanged) { | ||
| switch.deactivate() | ||
| switch.activate() | ||
| applySpeakerRouting(switch, speakerRouting) | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Snapshot of the AudioSwitch properties applied only at activate() time, used | ||
| // to detect when a live session needs a deactivate and activate cycle to pick | ||
| // up a configuration change. | ||
| private fun sessionConfigSnapshot() = SessionConfig( | ||
| manageAudioFocus = manageAudioFocus, | ||
| audioMode = audioMode, | ||
| focusMode = focusMode, | ||
| audioStreamType = audioStreamType, | ||
| audioAttributeUsageType = audioAttributeUsageType, | ||
| audioAttributeContentType = audioAttributeContentType, | ||
| forceHandleAudioRouting = forceHandleAudioRouting, | ||
| ) | ||
|
|
||
| /** Create (if needed) and activate the audio session: acquire focus, set mode and routing. */ | ||
| @Synchronized | ||
| fun start() { | ||
| val sessionConfig = sessionConfigSnapshot() | ||
| val speakerRouting = speakerRoutingSnapshot() | ||
| handler.post { | ||
| val switch = audioSwitch ?: createSwitch(sessionConfig, speakerRouting).also { audioSwitch = it } | ||
| if (!isActive) { | ||
| switch.activate() | ||
| applySpeakerRouting(switch, speakerRouting) | ||
| isActive = true | ||
| } | ||
| } | ||
| } | ||
|
|
||
| /** Deactivate and tear down the audio session: release focus and restore the previous mode. */ | ||
| @Synchronized | ||
| fun stop() { | ||
| handler.post { | ||
| audioSwitch?.stop() | ||
| audioSwitch = null | ||
| isActive = false | ||
| } | ||
| } | ||
|
|
||
| /** Final cleanup when the plugin detaches. The manager must not be used after this. */ | ||
| @Synchronized | ||
| fun dispose() { | ||
| handler.post { | ||
| audioSwitch?.stop() | ||
| audioSwitch = null | ||
| isActive = false | ||
| thread.quitSafely() | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Prefer routing to/from the speaker, letting a connected headset keep priority | ||
| * unless [force] is true. | ||
| */ | ||
| @Synchronized | ||
| fun setSpeakerphoneOn(enable: Boolean, force: Boolean) { | ||
| speakerOutputPreferred = enable | ||
| speakerOutputForced = enable && force | ||
| val speakerRouting = speakerRoutingSnapshot() | ||
| handler.post { | ||
| val switch = audioSwitch ?: return@post | ||
| applySpeakerRouting(switch, speakerRouting) | ||
| } | ||
| } | ||
|
|
||
| private fun createSwitch( | ||
| sessionConfig: SessionConfig, | ||
| speakerRouting: SpeakerRouting, | ||
| ): AbstractAudioSwitch { | ||
| val focusListener = AudioManager.OnAudioFocusChangeListener { } | ||
| // API-aware switch selection, matching the LiveKit Android SDK's | ||
| // AudioSwitchHandler: CommDeviceAudioSwitch uses the modern | ||
| // AudioManager.setCommunicationDevice routing on API 31+. | ||
| val switch = when { | ||
| Build.VERSION.SDK_INT >= Build.VERSION_CODES.S -> | ||
| CommDeviceAudioSwitch(context, false, focusListener, speakerRouting.preferredDeviceList) | ||
|
|
||
| Build.VERSION.SDK_INT >= Build.VERSION_CODES.M -> | ||
| AudioSwitch(context, false, focusListener, speakerRouting.preferredDeviceList) | ||
|
|
||
| else -> | ||
| LegacyAudioSwitch(context, false, focusListener, speakerRouting.preferredDeviceList) | ||
| } | ||
| applyConfiguration(switch, sessionConfig) | ||
| switch.start { _, _ -> } | ||
| return switch | ||
| } | ||
|
|
||
| private fun applyConfiguration(switch: AbstractAudioSwitch, sessionConfig: SessionConfig) { | ||
| switch.manageAudioFocus = sessionConfig.manageAudioFocus | ||
| switch.audioMode = sessionConfig.audioMode | ||
| switch.focusMode = sessionConfig.focusMode | ||
| switch.audioStreamType = sessionConfig.audioStreamType | ||
| switch.audioAttributeUsageType = sessionConfig.audioAttributeUsageType | ||
| switch.audioAttributeContentType = sessionConfig.audioAttributeContentType | ||
| switch.forceHandleAudioRouting = sessionConfig.forceHandleAudioRouting | ||
| } | ||
|
|
||
| private fun applySpeakerRouting(switch: AbstractAudioSwitch, speakerRouting: SpeakerRouting) { | ||
| switch.setPreferredDeviceList(speakerRouting.preferredDeviceList) | ||
| val forcedSpeaker = if (speakerRouting.speakerOutputForced) { | ||
| switch.availableAudioDevices.firstOrNull { it is AudioDevice.Speakerphone } | ||
| } else { | ||
| null | ||
| } | ||
| // AudioSwitch selections are sticky. Use them only for forced speaker output. | ||
| // Clearing the selection lets the preferred-device list handle normal routing | ||
| // and headset hot-plug priority. | ||
| switch.selectDevice(forcedSpeaker) | ||
| } | ||
|
|
||
| private fun speakerRoutingSnapshot() = SpeakerRouting( | ||
| speakerOutputForced = speakerOutputForced, | ||
| preferredDeviceList = preferredDeviceList( | ||
| speakerOutputPreferred = speakerOutputPreferred, | ||
| speakerOutputForced = speakerOutputForced, | ||
| ), | ||
| ) | ||
|
|
||
| private fun preferredDeviceList( | ||
| speakerOutputPreferred: Boolean, | ||
| speakerOutputForced: Boolean, | ||
| ): List<Class<out AudioDevice>> = | ||
| when { | ||
| speakerOutputForced -> listOf( | ||
| AudioDevice.Speakerphone::class.java, | ||
| AudioDevice.BluetoothHeadset::class.java, | ||
| AudioDevice.WiredHeadset::class.java, | ||
| AudioDevice.Earpiece::class.java, | ||
| ) | ||
|
|
||
| speakerOutputPreferred -> listOf( | ||
| AudioDevice.BluetoothHeadset::class.java, | ||
| AudioDevice.WiredHeadset::class.java, | ||
| AudioDevice.Speakerphone::class.java, | ||
| AudioDevice.Earpiece::class.java, | ||
| ) | ||
|
|
||
| else -> listOf( | ||
| AudioDevice.BluetoothHeadset::class.java, | ||
| AudioDevice.WiredHeadset::class.java, | ||
| AudioDevice.Earpiece::class.java, | ||
| AudioDevice.Speakerphone::class.java, | ||
| ) | ||
| } | ||
|
|
||
| private data class SessionConfig( | ||
| val manageAudioFocus: Boolean, | ||
| val audioMode: Int, | ||
| val focusMode: Int, | ||
| val audioStreamType: Int, | ||
| val audioAttributeUsageType: Int, | ||
| val audioAttributeContentType: Int, | ||
| val forceHandleAudioRouting: Boolean, | ||
| ) | ||
|
|
||
| private data class SpeakerRouting( | ||
| val speakerOutputForced: Boolean, | ||
| val preferredDeviceList: List<Class<out AudioDevice>>, | ||
| ) | ||
| } | ||
|
|
||
| // Map the Flutter-side enum names (see android_audio_session_adapter.dart) to | ||
| // Android framework constants. Ported from flutter_webrtc's AudioUtils. | ||
|
|
||
| private fun audioModeForName(name: String?): Int? = when (name) { | ||
| null -> null | ||
| "normal" -> AudioManager.MODE_NORMAL | ||
| "callScreening" -> AudioManager.MODE_CALL_SCREENING | ||
| "inCall" -> AudioManager.MODE_IN_CALL | ||
| "inCommunication" -> AudioManager.MODE_IN_COMMUNICATION | ||
| "ringtone" -> AudioManager.MODE_RINGTONE | ||
| else -> null | ||
| } | ||
|
|
||
| private fun focusModeForName(name: String?): Int? = when (name) { | ||
| null -> null | ||
| "gain" -> AudioManager.AUDIOFOCUS_GAIN | ||
| "gainTransient" -> AudioManager.AUDIOFOCUS_GAIN_TRANSIENT | ||
| "gainTransientExclusive" -> AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE | ||
| "gainTransientMayDuck" -> AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK | ||
| else -> null | ||
| } | ||
|
|
||
| private fun streamTypeForName(name: String?): Int? = when (name) { | ||
| null -> null | ||
| "accessibility" -> AudioManager.STREAM_ACCESSIBILITY | ||
| "alarm" -> AudioManager.STREAM_ALARM | ||
| "dtmf" -> AudioManager.STREAM_DTMF | ||
| "music" -> AudioManager.STREAM_MUSIC | ||
| "notification" -> AudioManager.STREAM_NOTIFICATION | ||
| "ring" -> AudioManager.STREAM_RING | ||
| "system" -> AudioManager.STREAM_SYSTEM | ||
| "voiceCall" -> AudioManager.STREAM_VOICE_CALL | ||
| else -> null | ||
| } | ||
|
|
||
| private fun usageTypeForName(name: String?): Int? = when (name) { | ||
| null -> null | ||
| "alarm" -> AudioAttributes.USAGE_ALARM | ||
| "assistanceAccessibility" -> AudioAttributes.USAGE_ASSISTANCE_ACCESSIBILITY | ||
| "assistanceNavigationGuidance" -> AudioAttributes.USAGE_ASSISTANCE_NAVIGATION_GUIDANCE | ||
| "assistanceSonification" -> AudioAttributes.USAGE_ASSISTANCE_SONIFICATION | ||
| "assistant" -> AudioAttributes.USAGE_ASSISTANT | ||
| "game" -> AudioAttributes.USAGE_GAME | ||
| "media" -> AudioAttributes.USAGE_MEDIA | ||
| "notification" -> AudioAttributes.USAGE_NOTIFICATION | ||
| "notificationEvent" -> AudioAttributes.USAGE_NOTIFICATION_EVENT | ||
| "notificationRingtone" -> AudioAttributes.USAGE_NOTIFICATION_RINGTONE | ||
| "unknown" -> AudioAttributes.USAGE_UNKNOWN | ||
| "voiceCommunication" -> AudioAttributes.USAGE_VOICE_COMMUNICATION | ||
| "voiceCommunicationSignalling" -> AudioAttributes.USAGE_VOICE_COMMUNICATION_SIGNALLING | ||
| else -> null | ||
| } | ||
|
|
||
| private fun contentTypeForName(name: String?): Int? = when (name) { | ||
| null -> null | ||
| "movie" -> AudioAttributes.CONTENT_TYPE_MOVIE | ||
| "music" -> AudioAttributes.CONTENT_TYPE_MUSIC | ||
| "sonification" -> AudioAttributes.CONTENT_TYPE_SONIFICATION | ||
| "speech" -> AudioAttributes.CONTENT_TYPE_SPEECH | ||
| "unknown" -> AudioAttributes.CONTENT_TYPE_UNKNOWN | ||
| else -> null | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. This one is a deliberate tradeoff, and it matches the rest of the LiveKit stack. The same
audioswitchcommit is pinned by bothflutter_webrtcand the LiveKit Android SDK, so dependency resolution stays consistent end to end. A JitPack artifact is immutable per commit, so the pin is reproducible in practice, and the main residual risk is JitPack availability rather than the artifact changing under us.The fork is needed for its
CommDeviceAudioSwitch, which provides the API 31setCommunicationDevicerouting path, and that class has no tagged release or MavenCentral coordinate to point at yet. If upstream publishes a release that carries it, we would happily switch over.