Commit f731be8
authored
[Feature][MRV2] GQA C8 modelrunnerv2 adaptation (vllm-project#12981)
### What this PR does / why we need it?
This PR fixes a bug in `modelrunnerv1` and `modelrunnerv2` caused by
inconsistent `attn_state` values during graph mode compilation.
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
- vLLM version: v0.25.1
- vLLM main:
vllm-project/vllm@fe784ff
---------
Signed-off-by: pichangping <1337510399@qq.com>1 parent 6fc3e33 commit f731be8
1 file changed
Lines changed: 7 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1714 | 1714 | | |
1715 | 1715 | | |
1716 | 1716 | | |
| 1717 | + | |
| 1718 | + | |
| 1719 | + | |
| 1720 | + | |
| 1721 | + | |
| 1722 | + | |
| 1723 | + | |
1717 | 1724 | | |
1718 | | - | |
1719 | | - | |
1720 | | - | |
1721 | | - | |
1722 | 1725 | | |
1723 | 1726 | | |
1724 | 1727 | | |
| |||
0 commit comments