[Feature] Add deterministic forced routing for CUDA MoE benchmarks

[sinpeyw]

Purpose

Add an opt-in CUDA benchmark control that gives native and AFD MoE runs deterministic routing with balanced token volume across EP ranks.

This extends the existing NPU benchmark semantics to CUDA without making the control model- or connector-specific.

Issue

• Related issue(s): [RFC]: Add deterministic forced routing for CUDA MoE benchmarks #242
• Closing keyword, only if fully resolved: Closes [RFC]: Add deterministic forced routing for CUDA MoE benchmarks #242

Scope

• In scope:
  • vLLM 0.26.0 CUDA routing;
  • deterministic source-rank phases across EP ranks;
  • an optional per-rank active-expert limit;
  • native and AFD benchmark processes.
• Out of scope:
  • correctness or production serving;
  • natural-routing performance claims;
  • uneven expert placement, EPLB, NPU changes, model adapters, connectors, and kernels.

Implementation Notes

• VLLM_MOE_ROUTING_SIMULATION_STRATEGY=afd_balanced selects the plugin-owned strategy through vLLM's routing simulator.
• AFD_BENCHMARK_FORCE_LB_TOPN_PER_RANK=0 uses all local experts; a positive value limits the deterministic pool on every EP rank.
• The strategy returns normalized weights and deterministic expert IDs; routing buffers are cached by topology, device, and dtype.
• vLLM continues to own strategy selection, routing capture, EPLB mapping, and index conversion. EPLB remains outside this benchmark's validated scope.
• The CUDA path uses vLLM's strategy selector rather than NPU's method-owned additional_config path because vLLM already exposes the required extension point.

No vLLM source file or router implementation is modified. Without the strategy selector, normal vLLM routing is unchanged.

Test Plan

uv run ruff check .
uv run ruff format --check .
uv run pytest -q tests/unit/model_executor/test_routing_simulator.py
git diff --check

GPU validation:

• native CUDA EP smoke with the complete local expert pool;
• asymmetric AFD smoke with a restricted expert pool;
• controlled Qwen3 native and 6A2F throughput matrix.

Test Result

• Focused routing-simulator tests: 4 passed.
• Complete unit suite with vLLM 0.26.0: 551 passed, 46 skipped.
• Native DP2/EP2 and asymmetric AFD 2A1F graph smokes: passed on A800.
• Ruff check, Ruff format, and git diff --check: passed.

Docs Impact

• Files updated: docs/design/module/compatibility_and_patches.md.
• Documents activation, compatibility scope, validation, and removal criteria.

---

Essential PR Checklist

• Purpose is clear and linked to public context.
• Scope is bounded.
• Compatibility with vLLM v0.26.0 is considered.
• No changes are made to the vLLM source checkout.
• The implementation uses vLLM's routing-simulator registration interface and does not replace router internals.
• Imports remain CPU-safe; vLLM routing is imported only during plugin
  registration when vLLM is installed.
• Final CPU and CUDA validation evidence is included.
• Documentation impact is stated.

[jiangkuaixue123]

Could we avoid this version-specific monkey patch and use vLLM's built-in routing simulator instead? For example, AFD could register its own strategy via RoutingSimulator.register_strategy(...) and select it with VLLM_MOE_ROUTING_SIMULATION_STRATEGY=<afd_strategy>.

I understand that the simulator strategy returns both routing weights and expert IDs, whereas this patch preserves the native weights and only overrides the IDs. Is preserving the exact native weights essential for the benchmark results? If the AFD strategy can reproduce the required weight semantics, this approach would encapsulate the deterministic/load-balanced routing in plugin-owned code while avoiding patches to FusedMoERouter and BaseRouter internals, which should be easier to maintain across vLLM versions.

[sinpeyw]

Could we avoid this version-specific monkey patch and use vLLM's built-in routing simulator instead? For example, AFD could register its own strategy via RoutingSimulator.register_strategy(...) and select it with VLLM_MOE_ROUTING_SIMULATION_STRATEGY=<afd_strategy>.

I understand that the simulator strategy returns both routing weights and expert IDs, whereas this patch preserves the native weights and only overrides the IDs. Is preserving the exact native weights essential for the benchmark results? If the AFD strategy can reproduce the required weight semantics, this approach would encapsulate the deterministic/load-balanced routing in plugin-owned code while avoiding patches to FusedMoERouter and BaseRouter internals, which should be easier to maintain across vLLM versions.

I have replaced the version-specific router monkey patch with a plugin-owned afd_balanced strategy registered through vLLM's RoutingSimulator. Exact native weights are not required for this benchmark, so the strategy returns normalized uniform weights while preserving deterministic expert placement and balanced token volume.