Commit c9e95ba
authored
[MRV2][Feature] Support sleep-mode (vllm-project#13005)
### What this PR does / why we need it?
This PR adapts 2 modifications into sleep mode in MRV2.
#### Problem 1: The missing hook
The upstream `gpu_worker.py` (shared by V1 and V2) always refreshes
model-runner state after a KV-cache wake-up:
```python
# vllm/v1/worker/gpu_worker.py L222-242
def wake_up(self, tags=None):
self._get_sleep_mode_backend().resume(tags)
# ... buffer restore ...
if tags is None or "kv_cache" in tags:
self.model_runner.post_kv_cache_wake_up()
```
The Ascend worker (`vllm_ascend/worker/worker.py`) does **not** issue
this call.
Its `wake_up` performs:
1. `CaMemAllocator.wake_up(tags=tags)` — remaps memory; KV-cache chunks
get fresh `data_ptr()` values.
2. MoE expert-weight layout restoration (`w13_weight` / `w2_weight`
transpose).
3. Level-2 buffer restore.
4. `SleepWakeupManager.wakeup()` — recaptures ACL graphs.
Between steps 3 and 4, `post_kv_cache_wake_up()` is missing.
#### Problem 2: AttributeError on extra-cleanup path
`SleepWakeupManager.sleep()` accesses `model_runner.use_aclgraph`
(`sleep_mem_optimized.py` L50). V1 `NPUModelRunner.__init__` sets this
attribute explicitly; V2 `NPUModelRunner.__init__` does **not**. When
`enable_sleep_mode_extra_cleanup=True` (not used in the one-card test,
but a supported config), V2 workers would crash with `AttributeError`.
### Does this PR introduce _any_ user-facing change?
None.
### How was this patch tested?
It is tested by tests
- vllm\vllm-ascend\tests\e2e\pull_request\one_card\test_camem.py
- vllm\vllm-ascend\tests\ut\device_allocator\test_camem.py
- vLLM version: v0.25.1
- vLLM main:
vllm-project/vllm@fe784ff
Signed-off-by: Raining__ <wangruining5@huawei.com>1 parent c8710f2 commit c9e95ba
2 files changed
Lines changed: 11 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
| 25 | + | |
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| |||
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
73 | 79 | | |
74 | 80 | | |
75 | 81 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
272 | 272 | | |
273 | 273 | | |
274 | 274 | | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
275 | 279 | | |
276 | 280 | | |
277 | 281 | | |
| |||
0 commit comments