Commit c093794
[BugFix] add logic when KVCacheSpec group has compress ratio > 1 (vllm-project#10865)
### What this PR does / why we need it?
DeepSeek-V4-Flash or self_attn and indexer_k_cache group (when
compress_ratio=4) max_num_block_per_req changing incorrect because
```py
if (kv_cache_group is not None
and hasattr(kv_cache_group, "kv_cache_spec")
and hasattr(kv_cache_group.kv_cache_spec, "compress_ratio")
):
compress_ratio = kv_cache_group.kv_cache_spec.compress_ratio
```
is unreachable. I changed the logic.
- vLLM version: v0.22.1
- vLLM main:
vllm-project/vllm@967c5c3
---------
Signed-off-by: KlyzhenkoVadim <vadimklyzhenko@gmail.com>
Signed-off-by: Vadim <87440656+KlyzhenkoVadim@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>1 parent 6673268 commit c093794
1 file changed
Lines changed: 5 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
35 | | - | |
| 35 | + | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
38 | 40 | | |
39 | 41 | | |
40 | 42 | | |
| |||
0 commit comments