Commit 2a607f0
authored
[BugFix][MRV2] Fix DP for eager mode (vllm-project#12195)
### What this PR does / why we need it?
- MoE Communication Method Selection: Moved the selection of the MoE
communication method to occur after the calculation of
max_tokens_across_dp to ensure consistency across all Data Parallel (DP)
ranks.
- Bug Fix: Resolved an issue in eager mode where using the local
num_tokens instead of the global max_tokens_across_dp caused
inconsistencies in communication method selection.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
export PYTHONPATH=/mnt/share/z00909740/dev/vllm:$PYTHONPATH
export VLLM_VERSION=0.25.0
export HCCL_OP_EXPANSION_MODE="AIV"
export PYTORCH_NPU_ALLOC_CONF="expandable_segments:True"
export VLLM_ASCEND_BALANCE_SCHEDULING=1
export OMP_PROC_BIND=false
export OMP_NUM_THREADS=10
export VLLM_TORCH_PROFILER_WITH_STACK=0
export HCCL_BUFFSIZE=4096
export TASK_QUEUE_ENABLE=1
export VLLM_ENGINE_READY_TIMEOUT_S=6000
export VLLM_USE_V2_MODEL_RUNNER=1
vllm serve /mnt/weight/DeepSeek-V3.1-w4a8-perchannle \
--host xx.xx.xx.xx \
--port xxxx \
--data-parallel-size 2 \
--tensor-parallel-size 8 \
--quantization ascend \
--seed 1024 \
--served-model-name deepseek_v3 \
--enable-expert-parallel \
--max-num-seqs 32 \
--max-model-len 5500 \
--max-num-batched-tokens 5000 \
--trust-remote-code \
--no-enable-prefix-caching \
--gpu-memory-utilization 0.92 \
--no-async-scheduling \
--compilation-config '{"cudagraph_mode": "PIECEWISE"}' \
- vLLM version: v0.25.1
- vLLM main:
vllm-project/vllm@fe784ff
Signed-off-by: zouzy <zouzongyu@huawei.com>1 parent f0daaf8 commit 2a607f0
1 file changed
Lines changed: 9 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
958 | 958 | | |
959 | 959 | | |
960 | 960 | | |
961 | | - | |
962 | | - | |
963 | | - | |
964 | | - | |
965 | | - | |
966 | 961 | | |
967 | 962 | | |
968 | 963 | | |
| |||
999 | 994 | | |
1000 | 995 | | |
1001 | 996 | | |
| 997 | + | |
| 998 | + | |
| 999 | + | |
| 1000 | + | |
| 1001 | + | |
| 1002 | + | |
| 1003 | + | |
| 1004 | + | |
| 1005 | + | |
1002 | 1006 | | |
1003 | 1007 | | |
1004 | 1008 | | |
| |||
0 commit comments