Skip to content

Fix sparse decode paged KV handling - #10

Open
jasonlizhengjian wants to merge 1 commit into
vllm-project:devfrom
jasonlizhengjian:fix/sparse-decode-paged-kv-strides
Open

Fix sparse decode paged KV handling#10
jasonlizhengjian wants to merge 1 commit into
vllm-project:devfrom
jasonlizhengjian:fix/sparse-decode-paged-kv-strides

Conversation

@jasonlizhengjian

@jasonlizhengjian jasonlizhengjian commented Jul 27, 2026

Copy link
Copy Markdown

Fix paged sparse-decode KV addressing needed by vLLM MiniMax M3
speculative verification.

  • Propagate the actual K and V token strides to CUTLASS instead of assuming
    head_dim.
  • Bound sparse-decode trip counts by the valid page count so short contexts do
    not load padded -1 page indices.
  • Add regression coverage for interleaved FP8 paged KV and short contexts.

Validated on B300 through
vLLM #50032: 17 focused
TP1/TP4 parity, dispatch-gating, ragged-length, plan-reuse, and CUDA graph
replay cases pass. A TP4 end-to-end run also dispatched the kernel with 32
active speculative requests at a measured acceptance length of 2.81.

Signed-off-by: Jason Li <jasonlizhengjian@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant