Skip to content

Commit b659959

Browse files
[BugFix] mega moe max token default value fix (vllm-project#11584)
### What this PR does / why we need it? Change the default value of mega_moe_max_tokens from 65536 to 131072. ### Does this PR introduce _any_ user-facing change? NA ### How was this patch tested? Configurable parameter, no extra testing needed. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: justice-dance <justice1717@163.com>
1 parent f926eea commit b659959

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

vllm_ascend/ascend_config.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -294,8 +294,8 @@ def __init__(self, vllm_config: "VllmConfig"):
294294
# are dropped and skipped from computation, degrading accuracy.
295295
# Do not set this too large: workspace memory scales linearly with this value, which matters
296296
# especially under long-context scenarios where the operator should not hold too much memory.
297-
# Default 65536.
298-
self.mega_moe_max_tokens = additional_config.get("mega_moe_max_tokens", 65536)
297+
# Default 131072.
298+
self.mega_moe_max_tokens = additional_config.get("mega_moe_max_tokens", 131072)
299299
if not isinstance(self.mega_moe_max_tokens, int):
300300
raise ValueError(
301301
f"mega_moe_max_tokens must be an integer, got {type(self.mega_moe_max_tokens).__name__}: "

0 commit comments

Comments
 (0)