Skip to content

Commit b6fe31d

Browse files
authored
[BugFix][main] remove unnecessary lazy_init for memcache backend (vllm-project#13029)
Cherry-pick of PR vllm-project#13028 to main branch. `lazy_init` is only needed for mooncake backend with deepseekv4 compress model on A3 hardware. The memcache backend handles lazy initialization internally based on device type, so passing `lazy_init=True` from `pool_worker` is redundant. - vLLM version: v0.25.1 - vLLM main: vllm-project/vllm@d02df74 --------- Signed-off-by: tyy0829 <1455207791@qq.com>
1 parent 2138260 commit b6fe31d

1 file changed

Lines changed: 9 additions & 13 deletions

File tree

  • vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store

vllm_ascend/distributed/kv_transfer/kv_pool/ascend_store/pool_worker.py

Lines changed: 9 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -300,19 +300,15 @@ def _init_backend(self, parallel_config, extra_config) -> None:
300300
backend_module = importlib.import_module(backend_path)
301301
real_backend = getattr(backend_module, backend_name)
302302

303-
if self.backend.lower() == "memcache":
304-
self.m_store = real_backend( # type: ignore[misc]
305-
parallel_config,
306-
lazy_init=True,
307-
)
308-
else:
309-
backend_kwargs = {}
310-
if self.backend.lower() == "mooncake":
311-
backend_kwargs["lazy_init"] = self.use_compress
312-
self.m_store = real_backend( # type: ignore[misc]
313-
parallel_config,
314-
**backend_kwargs,
315-
)
303+
backend_kwargs = {}
304+
# lazy_init is enabled only for DSV4 compress models. The backend further
305+
# gates this based on hardware: Mooncake requires ASCEND_ENABLE_FABRIC_MEM=1
306+
# (A3 fabric memory), and Memcache requires device_sdma protocol.
307+
backend_kwargs["lazy_init"] = self.use_compress
308+
self.m_store = real_backend( # type: ignore[misc]
309+
parallel_config,
310+
**backend_kwargs,
311+
)
316312

317313
def _init_kv_events(self, vllm_config) -> None:
318314
kv_event_config = vllm_config.kv_events_config

0 commit comments

Comments
 (0)