Skip to content

Commit 0532c49

Browse files
authored
[BugFix][Core] Fix schedule new_tokens when num_computed_tokens approching max_model_len (vllm-project#11544)
### What this PR does / why we need it? This PR fixes an issue in the recompute scheduler where requests are scheduled for extra steps even when the number of computed tokens has already reached or exceeded the maximum model length (`max_model_len`). By adding a check `request.num_computed_tokens >= self.max_model_len`, we avoid scheduling unnecessary steps for requests that have already reached the model's limit. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Tested with existing CI and unit tests. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: nwpu-zxr <zhouxuerong2@huawei.com>
1 parent 6251100 commit 0532c49

1 file changed

Lines changed: 1 addition & 0 deletions

File tree

vllm_ascend/core/recompute_scheduler.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -219,6 +219,7 @@ def schedule(self) -> RecomputeSchedulerOutput:
219219
# they are all rejected.
220220
and request.num_computed_tokens + 2 - request.num_output_placeholders
221221
>= request.num_prompt_tokens + request.max_tokens
222+
or request.num_computed_tokens >= self.max_model_len
222223
):
223224
# Async scheduling: Avoid scheduling an extra step when we are sure that
224225
# the previous step has reached request.max_tokens. We don't schedule

0 commit comments

Comments
 (0)