Skip to content

Commit bd12891

Browse files
committed
google_rtc_audio_processing: Major rework
Lots of work on AEC, with an eye to getting it working on main, where it had bitrotten, and pulling in various features (IPC3 pipeline state management and DP scheduling) that had merged in other branches. Dynamically configure stream formats (sample format/rate and channel count) from the connected streams at prepare() time instead of relying on build-time tuning. Port the code to use the source/sink API, in as sophisticated a manner as I can find. Copies unroll cleanly into just a few instructions per sample, including integer/float conversions and de-/interleaving. Support both 16 and 32 bit sample formats, with a fairly clever inlining scheme to share as much code as possible between them. The component will select "copy" function pointers at prepare() time. This works by using the _float32 variant of the AEC API (which is actually the core internal implementation) instead of the _int16 one which involves a conversion. The large buffers required by this component (input and output staging and an internally-managed pool/heap block) are now static symbols instead of dynamic memory from the heap. These are very large, taking up about half of what is available to the linker on MTL. Relying on heap allocation is just dangerous in this context. This fully decouples AEC from the playback stream. It will run without an active reference happily, feeding zeros to the processing, and pick up in stride when the pipeline starts. This requires adding a trigger handler for pipeline control in IPC3, which will propagate certain triggers across pipeline boundaries when shutting down playback streams, breaking active capture. Note that this feature is unexercisable on IPC4, where the kernel automatically starts up connected pipelines. Fixes a few bugs and misfeatures also: + Chunk the copies by full buffer strides between AEC processing calls instead of testing at each copied frame. + Copy the reference and mic streams in tandem, preventing them from becoming out of sync if the devices weren't themselves synchronized. + Copy the AEC results to the output stream after the call to ProcessCapture() instead of before. This was a hidden latency bug in the original code, I think. + Cleans up the Kconfig to remove stale variables and guard all the component-specific tunables under the top-level component variable. Also uses a default instead of a select to couple to CONFIG_STUBS, allowing AEC to be manually tuned. Signed-off-by: Andy Ross <andyross@google.com>
1 parent 1cc7a4c commit bd12891

4 files changed

Lines changed: 386 additions & 309 deletions

File tree

src/audio/google/Kconfig

Lines changed: 12 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ config COMP_GOOGLE_HOTWORD_DETECT
1515
config COMP_GOOGLE_RTC_AUDIO_PROCESSING
1616
bool "Google Real Time Communication Audio processing"
1717
select COMP_BLOB
18-
select GOOGLE_RTC_AUDIO_PROCESSING_MOCK if COMP_STUBS
1918
default n
2019
help
2120
Select for Google real-time communication audio processing. It
@@ -24,6 +23,8 @@ config COMP_GOOGLE_RTC_AUDIO_PROCESSING
2423
This component takes raw microphones input and playback reference
2524
and outputs an echo-free microphone signal.
2625

26+
if COMP_GOOGLE_RTC_AUDIO_PROCESSING
27+
2728
config COMP_GOOGLE_RTC_AUDIO_PROCESSING_SAMPLE_RATE_HZ
2829
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
2930
int "Sample rate for Google Real Time Communication Audio processing"
@@ -32,21 +33,15 @@ config COMP_GOOGLE_RTC_AUDIO_PROCESSING_SAMPLE_RATE_HZ
3233
Sets the sample rate for the memory buffer for the Google real-time
3334
communication audio processing.
3435

35-
config COMP_GOOGLE_RTC_AUDIO_PROCESSING_NUM_CHANNELS
36-
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
37-
int "Number of channels to process for Google Real Time Communication Audio processing"
38-
default 1
39-
help
40-
Sets the number of channels to process in the Google real-time
41-
communication audio processing.
42-
43-
config COMP_GOOGLE_RTC_AUDIO_PROCESSING_NUM_AEC_REFERENCE_CHANNELS
44-
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
45-
int "Number of AEC reference channels for Google Real Time Communication Audio processing"
36+
config COMP_GOOGLE_RTC_AUDIO_PROCESSING_CHANNEL_MAX
37+
int "Max number of AEC channels"
4638
default 2
4739
help
48-
Sets the number AEC reference channels in the Google real-time
49-
communication audio processing.
40+
Sets the maximum number source/sink channels Google Real
41+
Time Communication Audio Processing will use for. This is a
42+
computation and memory budget tunable. Channel counts are
43+
retrieved at runtime, but channels higher than this number
44+
are ignored (on input) or cleared (output).
5045

5146
config COMP_GOOGLE_RTC_AUDIO_PROCESSING_MEMORY_BUFFER_SIZE_BYTES
5247
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
@@ -74,10 +69,12 @@ config COMP_GOOGLE_RTC_AUDIO_PROCESSING_MIC_HEADROOM_LINEAR
7469

7570
config GOOGLE_RTC_AUDIO_PROCESSING_MOCK
7671
bool "Google Real Time Communication Audio processing mock"
77-
default n
72+
default y if COMP_STUBS
7873
depends on COMP_GOOGLE_RTC_AUDIO_PROCESSING
7974
help
8075
Mock Google real-time communication audio processing.
8176
It allows for compilation check and basic audio flow checking.
8277

78+
endif # COMP_GOOGLE_RTC_AUDIO_PROCESSING
79+
8380
endmenu

0 commit comments

Comments
 (0)