Skip to content

Commit d6a3401

Browse files
muziyuhui666CXY-Katrinacywang250805Bill845514379yejj710
authored
[Feature][Model] Add MiniMax M3 model core support (vllm-project#13009)
### What this PR does / why we need it? This PR adds the MiniMax M3 model core support on Ascend (split from vllm-project#12734, based on the implementation from vllm-project#12448): - Registers `MiniMaxM3SparseForCausalLM` in the Ascend model registry. - Adds the MiniMax M3 sparse text model implementation. - Adds MiniMax M3 MSA/indexer cache support. - Adds a guard to reject `enable_fused_mc2=1` for MiniMax M3, which is currently unsupported. - Updates `model_runner_v1.py` to discover custom `AttentionLayerBase` KV cache specs and attention backends through the existing `get_kv_cache_spec()` and `get_attn_backend()` interfaces. The runner change does not introduce MiniMax-specific imports into `model_runner_v1.py`. It keeps model-specific behavior inside the model/backend implementation and lets the runner depend on the common `AttentionLayerBase` abstraction. These changes provide the model-side components required to run MiniMax M3 sparse attention on Ascend, together with the sparse-attention operators added in vllm-project#12833. ### Does this PR introduce _any_ user-facing change? Yes. This PR introduces MiniMax M3 sparse text model support on Ascend. It does not change an existing public CLI or API. ### How was this patch tested? Verified MiniMax M3 service startup on 8 Ascend cards with W8A8 quantization. Verified single curl requests: - Text request passed. - Image request passed. - Video request passed. The following checks were run: - `bash format.sh` - `bash format.sh ci` - `pytest tests/ut/worker/a2/test_model_runner_v1.py` - `pytest tests/ut/attention/a2/test_sfa_v1.py` - `pytest tests/ut/attention/a2/test_sfa_cp_precision.py` - `pytest tests/ut/patch/worker/test_patch_deepseek_v2.py` - `pytest tests/ut/kv_offload/test_mooncake_connector.py -k "sfa or indexer"` Test results: - `tests/ut/worker/a2/test_model_runner_v1.py`: 18 passed - `tests/ut/attention/a2/test_sfa_v1.py`: 31 passed - `tests/ut/attention/a2/test_sfa_cp_precision.py`: 2 passed - `tests/ut/patch/worker/test_patch_deepseek_v2.py`: 3 passed - `tests/ut/kv_offload/test_mooncake_connector.py -k "sfa or indexer"`: 5 passed, 99 deselected - vLLM version: v0.25.1 - vLLM main: vllm-project/vllm@fe784ff Signed-off-by: muziyuhui666 <lijianfu9@huawei.com> Co-authored-by: CXY-Katrina <katrina.cxy@gmail.com> Co-authored-by: cywang250805 <wangchaoyu7@huawei.com> Co-authored-by: Bill845514379 <huangjianbao2@huawei.com> Co-authored-by: yejj710 <yejj710@gmail.com> Co-authored-by: AuroraEmiya <Sakura.iostream@gmail.com> Co-authored-by: HaoxinZong <116423146+HaoxinZong@users.noreply.github.com>
1 parent 9e07624 commit d6a3401

6 files changed

Lines changed: 2307 additions & 3 deletions

File tree

vllm_ascend/ascend_config.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,14 @@ def __init__(self, vllm_config: "VllmConfig"):
156156
ascend_envs.VLLM_ASCEND_ENABLE_FUSED_MC2,
157157
)
158158
assert self.enable_fused_mc2 in (0, 1), f"enable_fused_mc2 must be 0 or 1, got {self.enable_fused_mc2}"
159+
model_architectures = getattr(vllm_config.model_config, "architectures", None) or []
160+
assert not (
161+
self.enable_fused_mc2 == 1
162+
and any(architecture.startswith("MiniMaxM3") for architecture in model_architectures)
163+
), (
164+
"MiniMax M3 does not support enable_fused_mc2=1. Please set "
165+
"additional_config.enable_fused_mc2 to 0 or unset VLLM_ASCEND_ENABLE_FUSED_MC2."
166+
)
159167
if self.enable_fused_mc2 == 1 and self.multistream_overlap_shared_expert:
160168
self.multistream_overlap_shared_expert = False
161169
logger.warning_once(

vllm_ascend/models/__init__.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33

44
def register_model():
55
ModelRegistry.register_model("DeepseekV4ForCausalLM", "vllm_ascend.models.deepseek_v4:AscendDeepseekV4ForCausalLM")
6+
ModelRegistry.register_model(
7+
"MiniMaxM3SparseForCausalLM",
8+
"vllm_ascend.models.minimax_m3:MiniMaxM3SparseForCausalLM",
9+
)
610
ModelRegistry.register_model("DeepSeekV4MTPModel", "vllm_ascend.models.deepseek_v4_mtp:DeepSeekV4MTP")
711
ModelRegistry.register_model(
812
"DSparkDraftModel",
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# SPDX-License-Identifier: Apache-2.0
2+
3+
from vllm_ascend.models.minimax_m3.minimax_m3 import (
4+
MiniMaxM3Attention,
5+
MiniMaxM3MoE,
6+
MiniMaxM3SparseAttention,
7+
MiniMaxM3SparseForCausalLM,
8+
_get_rope_parameters,
9+
_sparse_attention_layer_ids,
10+
)
11+
12+
__all__ = [
13+
"MiniMaxM3Attention",
14+
"MiniMaxM3MoE",
15+
"MiniMaxM3SparseAttention",
16+
"MiniMaxM3SparseForCausalLM",
17+
"_get_rope_parameters",
18+
"_sparse_attention_layer_ids",
19+
]

0 commit comments

Comments
 (0)