Commit 6251100
authored
[Bugfix][SpecDecode] Fix rejection sampling spec length 1 in Triton (vllm-project#11458)
### What this PR does / why we need it?
This PR optimizes the `rejection_greedy_sample_spec_len_1_triton` kernel
by removing the loop and the helper function `bonus_renew_1`, replacing
them with a vectorized comparison and masked store. It also squeezes
`bonus_token_ids` in `rejection_greedy_sample_with_triton`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- vLLM version: v0.23.0
- vLLM main:
vllm-project/vllm@1f486d9
---------
Signed-off-by: liyishi <1252651434@qq.com>1 parent b417b95 commit 6251100
1 file changed
Lines changed: 4 additions & 23 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | 34 | | |
45 | 35 | | |
46 | 36 | | |
| |||
56 | 46 | | |
57 | 47 | | |
58 | 48 | | |
| 49 | + | |
| 50 | + | |
59 | 51 | | |
60 | 52 | | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
68 | | - | |
69 | | - | |
70 | | - | |
71 | | - | |
72 | | - | |
73 | | - | |
| 53 | + | |
| 54 | + | |
74 | 55 | | |
75 | 56 | | |
76 | 57 | | |
| |||
0 commit comments