Skip to content

Commit f302cf7

Browse files
authored
[CI] Modify accuracy threshold for aime2025 dataset to 10 (vllm-project#10881)
### What this PR does / why we need it? This PR modifies the accuracy threshold of the `aime2025` dataset from 5 to 10 across multiple model configurations (GLM5_1, Kimi-K2.5, MiniMax-M2.5, and Qwen3.5) to accommodate variance in accuracy benchmarks. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested via nightly CI pipelines with the updated configuration files. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@967c5c3 --------- Signed-off-by: chen-commits <1636718796@qq.com> Signed-off-by: chen <1636718796@qq.com>
1 parent a08abea commit f302cf7

4 files changed

Lines changed: 5 additions & 3 deletions

File tree

tests/e2e/nightly/multi_node/internal_dp/config/DeepSeek-V3_2-W8A8-EP.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ deployment:
4141
--data-parallel-start-rank 0
4242
--data-parallel-size-local 1
4343
--data-parallel-address $LOCAL_IP
44+
--data-parallel-rpc-port 13389
4445
--tensor-parallel-size 16
4546
--enable-expert-parallel
4647
--speculative-config '{"num_speculative_tokens": 2, "method":"deepseek_mtp"}'
@@ -86,6 +87,7 @@ deployment:
8687
--data-parallel-start-rank 1
8788
--data-parallel-size-local 1
8889
--data-parallel-address $MASTER_IP
90+
--data-parallel-rpc-port 13389
8991
--tensor-parallel-size 16
9092
--enable-expert-parallel
9193
--speculative-config '{"num_speculative_tokens": 2, "method":"deepseek_mtp"}'

tests/e2e/nightly/single_node/models/configs/Kimi-K2.5.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ _benchmarks: &benchmarks
6363
repetition_penalty: 1.0
6464
batch_size: 32
6565
baseline: 95
66-
threshold: 5
66+
threshold: 10
6767
perf:
6868
case_type: performance
6969
dataset_path: vllm-ascend/GSM8K_prefix90_in131072_bs1000_kimi

tests/e2e/nightly/single_node/models/configs/MiniMax-M2.5-w8a8-QuaRot-A3.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ test_cases:
5151
max_out_len: 131072
5252
batch_size: 32
5353
baseline: 90
54-
threshold: 5
54+
threshold: 10
5555
bos_token_id: 200019
5656
do_sample: true
5757
eos_token_id: 200020

tests/e2e/nightly/single_node/models/configs/Qwen3.5-122B-A10B-W8A8-A3.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ test_cases:
5757
max_out_len: 65536
5858
batch_size: 32
5959
baseline: 90
60-
threshold: 5
60+
threshold: 10
6161
thinking: true
6262
temperature: 1.0
6363
top_p: 0.95

0 commit comments

Comments
 (0)