Skip to content

feat: start Qwen3-ASR support - #654

Draft
sxwxs wants to merge 2 commits into
ROCm:mainfrom
sxwxs:feat/qwen3-asr
Draft

feat: start Qwen3-ASR support#654
sxwxs wants to merge 2 commits into
ROCm:mainfrom
sxwxs:feat/qwen3-asr

Conversation

@sxwxs

@sxwxs sxwxs commented Aug 6, 2026

Copy link
Copy Markdown

Summary

Starts Qwen3-ASR support with the parts that do not depend on a new private NPU kernel ABI:

  • add a C++ implementation of the Qwen3-ASR waveform frontend;
  • add exact audio-token length calculation;
  • add prompt construction and output parsing helpers;
  • add standalone C++ tests and a Transformers numerical parity check.

Preprocessing parity

The test covers:

  • 16 kHz mono float input;
  • minimum 8000-sample padding;
  • periodic Hann / centered reflect-padded STFT;
  • 128-bin Slaney mel filter bank;
  • Qwen log-mel normalization;
  • right-padding to 2 * n_window;
  • feature-major [mel_bins, frames] output layout.

Observed against the Transformers reference on the deterministic fixture:

shape: (128, 100)
max abs error: 3.4332275e-05
mean abs error: 2.960395e-07

Test

cd src/test/qwen3_asr_preprocessor
cmake -S . -B build -G Ninja
cmake --build build
ctest --test-dir build --output-on-failure
./build/test_qwen3_asr_preprocessor --dump /tmp/qwen3_asr_features.bin
python3 reference.py /tmp/qwen3_asr_features.bin

Remaining blocker

The existing public qwen3_npu interface accepts token IDs but does not document a payload for replacing audio placeholder embeddings. The next stage needs one of:

  • the existing qwen3_npu::prefill(..., void* payload) payload contract;
  • a mixed token/input-embedding prefill API;
  • or a Qwen3-ASR-specific engine.

Related: #616.

@sxwxs

sxwxs commented Aug 6, 2026

Copy link
Copy Markdown
Author

Local validation update:

  • Full cmake --preset linux-default && cmake --build build -j4 succeeds.
  • The resulting FLM 0.9.46 binary runs an existing Qwen3.5-2B model on the XDNA2 NPU successfully.
  • Standalone C++ tests and Transformers preprocessing parity still pass.

The remaining dependency for the next implementation stage is the mixed token/input-embedding prefill contract described in the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant