Skip to content

Commit c093794

Browse files
[BugFix] add logic when KVCacheSpec group has compress ratio > 1 (vllm-project#10865)
### What this PR does / why we need it? DeepSeek-V4-Flash or self_attn and indexer_k_cache group (when compress_ratio=4) max_num_block_per_req changing incorrect because ```py if (kv_cache_group is not None and hasattr(kv_cache_group, "kv_cache_spec") and hasattr(kv_cache_group.kv_cache_spec, "compress_ratio") ): compress_ratio = kv_cache_group.kv_cache_spec.compress_ratio ``` is unreachable. I changed the logic. - vLLM version: v0.22.1 - vLLM main: vllm-project/vllm@967c5c3 --------- Signed-off-by: KlyzhenkoVadim <vadimklyzhenko@gmail.com> Signed-off-by: Vadim <87440656+KlyzhenkoVadim@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
1 parent 6673268 commit c093794

1 file changed

Lines changed: 5 additions & 3 deletions

File tree

vllm_ascend/worker/block_table.py

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from vllm.distributed import get_dcp_group, get_pcp_group
44
from vllm.utils.math_utils import cdiv
55
from vllm.v1.attention.backends.utils import PAD_SLOT_ID
6-
from vllm.v1.kv_cache_interface import KVCacheGroupSpec, MambaSpec
6+
from vllm.v1.kv_cache_interface import KVCacheGroupSpec, MambaSpec, UniformTypeKVCacheSpecs
77
from vllm.v1.utils import CpuGpuBuffer
88
from vllm.v1.worker.block_table import _compute_slot_mapping_kernel
99
from vllm.v1.worker.cp_utils import get_total_cp_world_size
@@ -32,9 +32,11 @@ def __init__(
3232
if (
3333
kv_cache_group is not None
3434
and hasattr(kv_cache_group, "kv_cache_spec")
35-
and hasattr(kv_cache_group.kv_cache_spec, "compress_ratio")
35+
and isinstance(kv_cache_group.kv_cache_spec, UniformTypeKVCacheSpecs)
3636
):
37-
compress_ratio = kv_cache_group.kv_cache_spec.compress_ratio
37+
kv_cache_spec = next(iter(kv_cache_group.kv_cache_spec.kv_cache_specs.values()), None)
38+
if kv_cache_spec is not None and hasattr(kv_cache_spec, "compress_ratio"):
39+
compress_ratio = kv_cache_spec.compress_ratio
3840
if (
3941
kv_cache_group is not None
4042
and hasattr(kv_cache_group, "kv_cache_spec")

0 commit comments

Comments
 (0)