Skip to content

Commit acc0136

Browse files
U1stRsoulanddongyubin
andauthored
[CI]Adjust the dp_size and tp_size in the decode stage to 4 respectively (vllm-project#10773)
### What this PR does / why we need it? This PR adjusts the dp_size and tp_size in the decode stage to 4 respectively and revise the performance baseline in the Qwen3-235B-A22B-Mooncake-Layerwise configuration file. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Tested via nightly CI pipelines with the updated configuration files. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@967c5c3 --------- Signed-off-by: dongyubin <19800362117@163.com> Co-authored-by: dongyubin <19800362117@163.com>
1 parent ed8e8b7 commit acc0136

1 file changed

Lines changed: 11 additions & 11 deletions

File tree

tests/e2e/nightly/multi_node/internal_dp/config/Qwen3-235B-A22B-Mooncake-Layerwise.yaml

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,6 @@ deployment:
3939
--trust-remote-code
4040
--no-enable-prefix-caching
4141
--gpu-memory-utilization 0.9
42-
--additional-config '{"recompute_scheduler_enable": true,"enable_shared_expert_dp": true}'
4342
--kv-transfer-config
4443
'{"kv_connector": "MooncakeLayerwiseConnector",
4544
"kv_role": "kv_producer",
@@ -50,8 +49,8 @@ deployment:
5049
"tp_size": 8
5150
},
5251
"decode": {
53-
"dp_size": 2,
54-
"tp_size": 8
52+
"dp_size": 4,
53+
"tp_size": 4
5554
}
5655
}
5756
}'
@@ -62,20 +61,21 @@ deployment:
6261
vllm serve "Qwen/Qwen3-235B-A22B"
6362
--host 0.0.0.0
6463
--port $SERVER_PORT
65-
--data-parallel-size 2
66-
--data-parallel-size-local 2
64+
--data-parallel-size 4
65+
--data-parallel-size-local 4
6766
--data-parallel-address $MASTER_IP
6867
--data-parallel-rpc-port 13389
69-
--tensor-parallel-size 8
68+
--tensor-parallel-size 4
7069
--seed 1024
71-
--max-num-seqs 16
70+
--max-num-seqs 32
7271
--max-model-len 8192
7372
--max-num-batched-tokens 8192
7473
--enable-expert-parallel
7574
--trust-remote-code
7675
--no-enable-prefix-caching
7776
--gpu-memory-utilization 0.9
78-
--additional-config '{"torchair_graph_config":{"enabled":true}}'
77+
--async-scheduling
78+
--compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
7979
--kv-transfer-config
8080
'{"kv_connector": "MooncakeLayerwiseConnector",
8181
"kv_role": "kv_consumer",
@@ -86,8 +86,8 @@ deployment:
8686
"tp_size": 8
8787
},
8888
"decode": {
89-
"dp_size": 2,
90-
"tp_size": 8
89+
"dp_size": 4,
90+
"tp_size": 4
9191
}
9292
}
9393
}'
@@ -102,7 +102,7 @@ benchmarks:
102102
max_out_len: 1500
103103
batch_size: 700
104104
request_rate: 11.2
105-
baseline: 183
105+
baseline: 2046
106106
threshold: 0.97
107107
acc:
108108
case_type: accuracy

0 commit comments

Comments
 (0)