Tracking: Kimi MLA upstream split simulation

[voipmonitor]

Kimi 2.5/2.6 MLA + DCP + SpecDecode Tracking

This document is a virtual tracking issue for splitting the current working Docker state into upstream-sized PRs, then recomposing them to verify parity.

Base

• Docker code base commit: 47fcb8ca68c1027ba7eb7a9056bb4596ee284221
• Fork: voipmonitor/vllm
• Target upstream repo: vllm-project/vllm

Upstream-candidate PR buckets

1. MLA + DCP + FP8 KV support
2. Async speculative proposer race fix
3. Draft-specific attention backend + draft KV dtype
4. Qwen3 / Qwen3-DFlash local argmax support
5. Llama Eagle3 local argmax support

Existing upstream context

• #34795 MLA + DCP + FP8 KV support
• #39419 local argmax reduction for large-vocab spec decode
• #39930 DFlash backend selection
• #39995 DFlash + FlashInfer experiments

Local-only / not for upstream in current form

• VLLM_SPECULATIVE_DISABLE_ABOVE_SEQ_LEN env workaround
• XML / NCCL_GRAPH_FILE scoping workarounds
• experimental Triton MLA split/reduction tuning
• local NCCL 2.29.7 patched build and no-XML validation
• PCIe custom allreduce opt-in env hack

Verification plan

1. Create topic branches for each PR bucket.
2. Reapply only the relevant hunks from the working Docker tree.
3. Create an integration branch by merging/cherry-picking those branches.
4. Diff the integration branch against the working Docker tree.
5. If parity is reached, open upstream issue/PRs plus a final Kimi recipe issue.

[voipmonitor]

Status update:

• Added simulated topic branches for the current Kimi MLA upstream split:
  • sim/pr-34795-mla-dcp-fp8
  • sim/pr-async-spec-proposer-race
  • sim/pr-draft-backend-kv-dtype
  • sim/pr-qwen3-local-argmax
  • sim/pr-llama-eagle3-local-argmax
• Added local-only runtime branches:
  • sim/local-spec-disable-threshold
  • sim/local-pcie-custom-allreduce
  • sim/local-triton-mla-tuning
• Added integration branch:
  • sim/integration-kimi-mla-20260422

Integration parity vs the live docker tree is exact for the main Kimi/EAGLE runtime files (mla_attention.py, gpu_model_runner.py, cp_utils.py, async_scheduler.py, custom_all_reduce.py, triton_mla.py, triton_decode_attention.py, qwen3*.py, llama_eagle3.py, and the DCP FP8 test).

No-XML benchmark update with patched NCCL 2.29.7:

• end-to-end works without NCCL_GRAPH_FILE
• but performance is not yet equivalent to the XML-based setup
• long-context decode is roughly comparable/slightly better
• short-context decode is worse
• prefill is dramatically worse

So the current conclusion is:

• patched NCCL is promising and functional
• but we cannot remove the XML-based runtime recipe yet if we want current best performance
• the XML-scoping hack remains local-only and should not be upstreamed in its current form

The tracker doc on this branch now contains the detailed branch map, parity status, and benchmark summary.

[voipmonitor]

Draft PR simulation is now complete.

Simulated upstream-candidate PRs:

• [Core] Enable FP8 KV cache with DCP for MLA #3 [Core] Enable FP8 KV cache with DCP for MLA
• [SpecDecode] Fix async proposer synchronization #4 [SpecDecode] Fix async proposer synchronization
• [SpecDecode] Allow draft-specific attention backend and KV dtype #5 [SpecDecode] Allow draft-specific attention backend and KV dtype
• [SpecDecode] Add local argmax helpers for Qwen3 drafts #6 [SpecDecode] Add local argmax helpers for Qwen3 drafts
• [SpecDecode] Add local argmax helper for Llama Eagle3 #7 [SpecDecode] Add local argmax helper for Llama Eagle3

Simulated local-only runtime PRs:

• [Local] Add seq-length gate for speculative decode #8 [Local] Add seq-length gate for speculative decode
• [Local] Add PCIe custom allreduce opt-in #9 [Local] Add PCIe custom allreduce opt-in
• [Local] Carry current Triton MLA tuning #10 [Local] Carry current Triton MLA tuning

Integration branch:

• sim/integration-kimi-mla-20260422

Current integration status:

• the integration branch reconstructs the main Kimi runtime files from the live docker tree exactly for the core working set
• remaining differences are only the local DFlash window-size additions in speculative.py, a small warning/fallback tweak in eagle.py, and comment/whitespace drift in flash_attn.py

The tracker document on this branch has been updated with the branch map, PR list, parity summary, and the patched-NCCL no-XML benchmark result.