Commit d6a3401
[Feature][Model] Add MiniMax M3 model core support (vllm-project#13009)
### What this PR does / why we need it?
This PR adds the MiniMax M3 model core support on Ascend (split from
vllm-project#12734, based on the
implementation from
vllm-project#12448):
- Registers `MiniMaxM3SparseForCausalLM` in the Ascend model registry.
- Adds the MiniMax M3 sparse text model implementation.
- Adds MiniMax M3 MSA/indexer cache support.
- Adds a guard to reject `enable_fused_mc2=1` for MiniMax M3, which is
currently unsupported.
- Updates `model_runner_v1.py` to discover custom `AttentionLayerBase`
KV cache specs and attention backends through the existing
`get_kv_cache_spec()` and `get_attn_backend()` interfaces.
The runner change does not introduce MiniMax-specific imports into
`model_runner_v1.py`. It keeps model-specific behavior inside the
model/backend implementation and lets the runner depend on the common
`AttentionLayerBase` abstraction.
These changes provide the model-side components required to run MiniMax
M3 sparse attention on Ascend, together with the sparse-attention
operators added in
vllm-project#12833.
### Does this PR introduce _any_ user-facing change?
Yes. This PR introduces MiniMax M3 sparse text model support on Ascend.
It does not change an existing public CLI or API.
### How was this patch tested?
Verified MiniMax M3 service startup on 8 Ascend cards with W8A8
quantization.
Verified single curl requests:
- Text request passed.
- Image request passed.
- Video request passed.
The following checks were run:
- `bash format.sh`
- `bash format.sh ci`
- `pytest tests/ut/worker/a2/test_model_runner_v1.py`
- `pytest tests/ut/attention/a2/test_sfa_v1.py`
- `pytest tests/ut/attention/a2/test_sfa_cp_precision.py`
- `pytest tests/ut/patch/worker/test_patch_deepseek_v2.py`
- `pytest tests/ut/kv_offload/test_mooncake_connector.py -k "sfa or
indexer"`
Test results:
- `tests/ut/worker/a2/test_model_runner_v1.py`: 18 passed
- `tests/ut/attention/a2/test_sfa_v1.py`: 31 passed
- `tests/ut/attention/a2/test_sfa_cp_precision.py`: 2 passed
- `tests/ut/patch/worker/test_patch_deepseek_v2.py`: 3 passed
- `tests/ut/kv_offload/test_mooncake_connector.py -k "sfa or indexer"`:
5 passed, 99 deselected
- vLLM version: v0.25.1
- vLLM main:
vllm-project/vllm@fe784ff
Signed-off-by: muziyuhui666 <lijianfu9@huawei.com>
Co-authored-by: CXY-Katrina <katrina.cxy@gmail.com>
Co-authored-by: cywang250805 <wangchaoyu7@huawei.com>
Co-authored-by: Bill845514379 <huangjianbao2@huawei.com>
Co-authored-by: yejj710 <yejj710@gmail.com>
Co-authored-by: AuroraEmiya <Sakura.iostream@gmail.com>
Co-authored-by: HaoxinZong <116423146+HaoxinZong@users.noreply.github.com>1 parent 9e07624 commit d6a3401
6 files changed
Lines changed: 2307 additions & 3 deletions
File tree
- vllm_ascend
- models
- minimax_m3
- worker
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
156 | 156 | | |
157 | 157 | | |
158 | 158 | | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
159 | 167 | | |
160 | 168 | | |
161 | 169 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
6 | 10 | | |
7 | 11 | | |
8 | 12 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
0 commit comments