Commit 0532c49
authored
[BugFix][Core] Fix schedule new_tokens when num_computed_tokens approching max_model_len (vllm-project#11544)
### What this PR does / why we need it?
This PR fixes an issue in the recompute scheduler where requests are
scheduled for extra steps even when the number of computed tokens has
already reached or exceeded the maximum model length (`max_model_len`).
By adding a check `request.num_computed_tokens >= self.max_model_len`,
we avoid scheduling unnecessary steps for requests that have already
reached the model's limit.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tested with existing CI and unit tests.
- vLLM version: v0.23.0
- vLLM main:
vllm-project/vllm@1f486d9
Signed-off-by: nwpu-zxr <zhouxuerong2@huawei.com>1 parent 6251100 commit 0532c49
1 file changed
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
219 | 219 | | |
220 | 220 | | |
221 | 221 | | |
| 222 | + | |
222 | 223 | | |
223 | 224 | | |
224 | 225 | | |
| |||
0 commit comments