feat(dflash): add direct DeepSeek4 pure-UMA runtime path

[weicj]

Summary

This PR brings the current DeepSeek4 backend line from #353 to main, and adds a direct DeepSeek4Backend runtime path for non-layer-split DeepSeek4 requests in the C++ server.

Before this change, arch=deepseek4 always went through DeepSeek4LayerSplitAdapter + LayerSplitBackend, even for ordinary single-backend placements such as pure HIP / UMA. This PR keeps the existing layer-split route when the configured device is actually layer-split, and routes non-layer-split placements to the native DeepSeek4 backend instead.

It also adds decode-side full-step graph reuse and fuller full-step timing telemetry so the direct runtime can be validated and profiled on large UMA devices.

Changes

• Route non-layer-split deepseek4 requests to DeepSeek4Backend.
• Keep the existing DeepSeek4LayerSplitAdapter + LayerSplitBackend path for device.is_layer_split().
• Reuse the direct full-step decode graph metadata / allocator state on GPU decode steps.
• Expose full-step build / alloc / set / compute / read timings in DeepSeek4 runtime telemetry.
• Leave layer-split placement behavior unchanged.

Validation

• Runtime validation in a pure UMA gfx1151 environment with max_ctx=1048576
• Large-model startup succeeded with:
  • model buffer: 81687.7 MB
  • KV cache at 1M ctx: 6897.8 MB
• Native 1M ctx runtime checks with the default chunk=512:

Prompt / Completion	Prefill walltime	Prefill tok/s	Decode walltime	Decode tok/s
256 / 32	1228.3 ms	208.42	2001.8 ms	16.00
4096 / 128	18429.1 ms	222.26	8288.5 ms	15.40
8192 / 256	42513.7 ms	192.69	17407.2 ms	14.70
16384 / 512	108775.9 ms	150.62	38418.0 ms	13.30

above listed data is obselette due to lack of HC-correctness guard

[cubic-dev-ai]

2 issues found across 4 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[weicj]

This update stabilizes the PR489 DeepSeek4 direct pure-UMA runtime path.

The previous implementation introduced the direct DeepSeek4 pure-UMA path, but the HC backend graph path was still incomplete and unsafe as a default runtime path. This revision keeps the incomplete batch HC backend graph opt-in only, and uses the verified quality-stable CPU-side/eager HC path by default.

Changes

• Use GGUF tokenizer metadata for EOS detection instead of hardcoded token ids.
• Make the incomplete batch HC backend graph path opt-in via DFLASH_DS4_ENABLE_BACKEND_BATCH_HC_GRAPH=1.
• Keep the quality-stable HC CPU-side/eager runtime path as the default behavior.
• Preserve the performance-oriented HC prefill CPU path, including f16 x f32 matvec, token parallelism, and AVX2/F16C/FMA support.

Validation

Strict exactseq validation, default runtime, ctx=1M, chunk=512:

• bad_files=[]
• Stable hello256 prefill: 142.5~143.9 tok/s
• Stable hello256 decode: 12.45~12.6 tok/s

Replay performance, default runtime, ctx=1M, chunk=512:

Case	Prompt Tokens	Completion Tokens	Prefill tok/s	Decode tok/s
256/32	257	32	130.04	11.7
4k/128	4097	128	146.27	10.8
8k/256	8193	256	90.36	10.1

[davide221]

@weicj can you verify if this can work with the new rocmfpx support? #502

[weicj]

Verified with #502 (ROCmFPx). Combined build passes on lucebox5. Perf:

256/32 (ctx1m): prefill=144.36 tok/s, decode=12.1 tok/s
4k/128 (ctx1m): prefill=143.94 tok/s, decode=11.0 tok/s

also cleaned up the accidental expert_split_target_config dependency to keep this PR purely on the UMA/HC path (that's from the unfinished dsv4f expert_split route experiement)

[cubic-dev-ai]

All reported issues were addressed across 10 files

Not reviewed (too large): server/src/deepseek4/deepseek4_graph.cpp (~3,169 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[weicj]

2026-07-10 Update

• Shared decode scalar inputs across cached per-layer decode attention graphs. Per-step values such as RoPE positions, raw KV row, compressor rows, and compressor positions are now uploaded once through compact i32/i64 bundles instead of being set repeatedly for every layer.
• The same shared-input path is wired into both the hybrid decode path and the HC-complete deepseek4_step_layer_range path used by the PR feat(ggml): vendor ROCmFPX quant support for DeepSeek V4 Flash #502 compatible runtime flow.
• Request reset now clears compressed-cache counters and HC state so each new request starts from a clean cache topology.
• Hash-routing table handling now preserves the original GGUF table width while consuming only the effective n_expert_used prefix, keeping routing logic compatible with default and experimental widths.
• Added UTF-8 boundary filtering in the HTTP output path so byte-fallback tokens split across decode steps are emitted only after they form valid UTF-8. This fixes replacement-character artifacts without changing model compute, routing, EOS handling, or the default HC path.
• Added DFLASH_DS4_TOPK as an experimental routing override. This is not used by the default path; the default still follows GGUF expert_used_count=6.

Validation

Quality checks passed on 256/32, 4k/128, and 8k/256 validation runs. All performance numbers below use the default path, chunk=512, ctx1m, and no TOPK override:

Case	Prefill	Decode
256/32	178.43 tok/s	13.6 tok/s
4k/128	144.13 tok/s	11.6 tok/s
8k/256	123.83 tok/s	10.9 tok/s

DFLASH_DS4_TOPK=4 was also tested as an experimental mode. It improved throughput to roughly 188.18 / 14.2 on 256/32, 152.22 / 12.2 on 4k/128, and 129.41 / 11.4 on 8k/256, but output quality was not clean enough for the default path due to obvious degradation with repetitive/self-descriptive responses.

[cubic-dev-ai]

3 issues found across 11 files

Not reviewed (too large): server/src/deepseek4/deepseek4_graph.cpp (~4,552 lines) - if these are generated or fixture files, add them to ignored paths to exclude them from future reviews.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/server/http_server.cpp">

<violation number="1" location="server/src/server/http_server.cpp:182">
P2: Responses can lose a legitimate `�` character because `Utf8OutputFilter` drops valid U+FFFD code points after validating the UTF-8 sequence. Since invalid byte sequences are already skipped by the validator, preserving valid U+FFFD avoids corrupting model output while still filtering malformed bytes.</violation>
</file>

<file name="server/src/deepseek4/deepseek4_internal.h">

<violation number="1" location="server/src/deepseek4/deepseek4_internal.h:355">
P2: `want_logits=false` does not currently skip the output head/readback on this path, so prefill chunks that only need KV/routing state still perform the expensive logits projection. Threading `want_logits` into the graph implementation would make the new API contract true.</violation>
</file>

<file name="server/src/deepseek4/deepseek4_hc_cuda.cu">

<violation number="1" location="server/src/deepseek4/deepseek4_hc_cuda.cu:618">
P2: Batch HC pre can fail for valid `n_hc > 4` configs: `hc_pre_batch_device_locked` compares `mix_dim` to `kMixDim` (24) even though allocation and public validation allow `kMaxMixDim` (80). Matching this guard to `kMaxMixDim` keeps the direct batch path usable for all accepted HC counts.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: want_logits=false does not currently skip the output head/readback on this path, so prefill chunks that only need KV/routing state still perform the expensive logits projection. Threading want_logits into the graph implementation would make the new API contract true.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/deepseek4/deepseek4_internal.h, line 355:

<comment>`want_logits=false` does not currently skip the output head/readback on this path, so prefill chunks that only need KV/routing state still perform the expensive logits projection. Threading `want_logits` into the graph implementation would make the new API contract true.</comment>

<file context>
@@ -314,9 +350,34 @@ bool deepseek4_step(
-    MoeHybridStreamEngine *     stream_engine = nullptr,
+    ExpertIpcClient *  expert_worker = nullptr,
+    bool                        worker_owns_hot_ids = false,
+    bool                        want_logits = true,
+    bool                        disable_cached_decode = false,
     DeepSeek4StepTelemetry *    telemetry = nullptr,
</file context>