Commit 4e9b8a0
authored
[Doc][Misc]Add sleep mode documentation for unquantized MoE models (vllm-project#11110)
### What this PR does / why we need it?
Adds a "MoE vs. dense (non-MoE) model differences" section to the sleep
mode guide, documenting the expert weight layout restoration:
process_weights_after_loading() transposes w13_weight/w2_weight into the
torch_npu.npu_grouped_matmul layout at load time (assuming standard
checkpoint layout, no validation), and wake_up() re-applies the same
transpose after the allocator restores memory; dense/quantized models
skip it.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- vLLM version: v0.23.0
- vLLM main:
vllm-project/vllm@967c5c3
---------
Co-authored-by: yjyang62 <yjyang62@users.noreply.github.com>1 parent cadefd2 commit 4e9b8a0
1 file changed
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
70 | 83 | | |
71 | 84 | | |
72 | 85 | | |
| |||
0 commit comments