Skip to content

Commit ec1f3f7

Browse files
authored
[BugFix][KV_Offload] RecomputeCpuOffloadConnector add main2main for releases/v0.23.0 (vllm-project#11061)
### What this PR does / why we need it? In vllm v0.23.0, a new method `resolve_kv_cache_block_sizes` is introduced to get `scheduler_block_size` and `hash_block_size`, which will be used for `get_kv_cache_coordinator`, this PR keep main2mian with v0.23.0 for `RecomputeCPUOffloadConnector`. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? By CI. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@967c5c3 Signed-off-by: nwpu-zxr <zhouxuerong2@huawei.com>
1 parent 55b8cd0 commit ec1f3f7

1 file changed

Lines changed: 4 additions & 1 deletion

File tree

  • vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload

vllm_ascend/distributed/kv_transfer/kv_pool/recompute_cpu_offload/manager.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
KVCacheCoordinator,
1717
get_kv_cache_coordinator,
1818
)
19+
from vllm.v1.core.kv_cache_utils import resolve_kv_cache_block_sizes
1920
from vllm.v1.core.sched.output import SchedulerOutput
2021
from vllm.v1.kv_cache_interface import SlidingWindowSpec, UniformTypeKVCacheSpecs
2122
from vllm.v1.outputs import KVConnectorOutput
@@ -86,6 +87,7 @@ def __init__(
8687
dcp_world_size = vllm_config.parallel_config.decode_context_parallel_size
8788
pcp_world_size = vllm_config.parallel_config.prefill_context_parallel_size
8889
assert dcp_world_size == 1 and pcp_world_size == 1
90+
scheduler_block_size, hash_block_size = resolve_kv_cache_block_sizes(kv_cache_config, vllm_config)
8991
self.cpu_coordinator: KVCacheCoordinator = get_kv_cache_coordinator(
9092
kv_cache_config=self.cpu_kv_cache_config,
9193
max_model_len=vllm_config.model_config.max_model_len,
@@ -95,7 +97,8 @@ def __init__(
9597
enable_kv_cache_events=self.enable_kv_cache_events,
9698
dcp_world_size=dcp_world_size,
9799
pcp_world_size=pcp_world_size,
98-
hash_block_size=vllm_config.cache_config.block_size,
100+
scheduler_block_size=scheduler_block_size,
101+
hash_block_size=hash_block_size,
99102
)
100103
self.cpu_block_pool: BlockPool = self.cpu_coordinator.block_pool
101104
self._gpu_block_pool: BlockPool | None = None

0 commit comments

Comments
 (0)