Skip to content

Commit 2a607f0

Browse files
authored
[BugFix][MRV2] Fix DP for eager mode (vllm-project#12195)
### What this PR does / why we need it? - MoE Communication Method Selection: Moved the selection of the MoE communication method to occur after the calculation of max_tokens_across_dp to ensure consistency across all Data Parallel (DP) ranks. - Bug Fix: Resolved an issue in eager mode where using the local num_tokens instead of the global max_tokens_across_dp caused inconsistencies in communication method selection. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? export PYTHONPATH=/mnt/share/z00909740/dev/vllm:$PYTHONPATH export VLLM_VERSION=0.25.0 export HCCL_OP_EXPANSION_MODE="AIV" export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True" export VLLM_ASCEND_BALANCE_SCHEDULING=1 export OMP_PROC_BIND=false export OMP_NUM_THREADS=10 export VLLM_TORCH_PROFILER_WITH_STACK=0 export HCCL_BUFFSIZE=4096 export TASK_QUEUE_ENABLE=1 export VLLM_ENGINE_READY_TIMEOUT_S=6000 export VLLM_USE_V2_MODEL_RUNNER=1 vllm serve /mnt/weight/DeepSeek-V3.1-w4a8-perchannle \ --host xx.xx.xx.xx \ --port xxxx \ --data-parallel-size 2 \ --tensor-parallel-size 8 \ --quantization ascend \ --seed 1024 \ --served-model-name deepseek_v3 \ --enable-expert-parallel \ --max-num-seqs 32 \ --max-model-len 5500 \ --max-num-batched-tokens 5000 \ --trust-remote-code \ --no-enable-prefix-caching \ --gpu-memory-utilization 0.92 \ --no-async-scheduling \ --compilation-config '{"cudagraph_mode": "PIECEWISE"}' \ - vLLM version: v0.25.1 - vLLM main: vllm-project/vllm@fe784ff Signed-off-by: zouzy <zouzongyu@huawei.com>
1 parent f0daaf8 commit 2a607f0

1 file changed

Lines changed: 9 additions & 5 deletions

File tree

vllm_ascend/platform.py

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -958,11 +958,6 @@ def set_additional_forward_context(
958958
is_draft_model_prefill = False
959959
sinks = False
960960
in_profile_run = get_mrv2_in_profile_run()
961-
moe_comm_type = select_moe_comm_method(
962-
num_tokens,
963-
vllm_config,
964-
)
965-
moe_comm_method = get_moe_comm_method(moe_comm_type)
966961

967962
tp_world_size = get_tensor_model_parallel_world_size()
968963

@@ -999,6 +994,15 @@ def set_additional_forward_context(
999994
pad_size = padded_length - num_tokens
1000995
else:
1001996
max_tokens_across_dp = num_tokens
997+
998+
# NOTE: Must use max_tokens_across_dp instead of num_tokens for MoE comm method selection
999+
# to ensure consistent communication method across all DP ranks
1000+
moe_comm_type = select_moe_comm_method(
1001+
max_tokens_across_dp,
1002+
vllm_config,
1003+
)
1004+
moe_comm_method = get_moe_comm_method(moe_comm_type)
1005+
10021006
mc2_mask = None
10031007
padded_num_tokens = None
10041008
if num_tokens is not None:

0 commit comments

Comments
 (0)