port(ggml): in-place GDN state write + qwen dense decode recovery (lucebox-ggml #24, #27)

[dusterbloom]

Ports two open ggml feature PRs from Luce-Org/lucebox-ggml into the in-tree vendored ggml (server/deps/llama.cpp/ggml/), on top of the vendorization from #486. Both were intentionally excluded from the initial snapshot (see VENDOR.md) and are brought in here.

Included

• lucebox-ggml docs(dflash): bump long-context example from 128K to 256K #24 — perf(cuda): add in-place GDN state write (2576c152)
  Adds public API ggml_gated_delta_net_inplace, which writes the final recurrent state directly into src[5] via op_params[1]. Threads a state_out param through the CUDA GDN kernels. Default ggml_gated_delta_net behavior is unchanged.
• lucebox-ggml Followed instructions for Megakernel Python setup, no dice #27 — perf(cuda): recover qwen dense decode path (c3d44ed4)
  • Thread-local CUDA-graph props-check skip (ggml_cuda_set_skip_props_check) to bypass per-step graph re-validation on the stable AR decode path.
  • Compile-time WRITE_INTER template specialization of the GDN kernels — the pure-AR path compiles out the intermediate-state store entirely.
  • Ampere grouped-cols default flip (env DFLASH_GDN_FORCE_GROUPED_COLS / DFLASH_GDN_NO_GROUPED_COLS); grouped-cols is off by default on Ampere (sm_86 / 3090).

Notes

• The two upstream PRs overlap in gated_delta_net.cu; applied as the union (each kernel/launch signature carries both state_out and the WRITE_INTER template, with the runtime skip_intermediate param removed).
• Scope is ggml only — no consumer changes under server/src/. The new symbols are additive and unused by current server code.
• Commits keep original authorship.

Verification

• All target symbols present in the vendored tree.
• ggml.c, gated_delta_net.cu, and ggml-cuda.cu compile clean under CUDA 12.6 / sm_86 using the existing build flags.
• Diff limited to 4 files under server/deps/llama.cpp/ggml/; zero server/src/ drift.

[cubic-dev-ai]

1 issue found across 4 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/deps/llama.cpp/ggml/src/ggml-cuda/gated_delta_net.cu">

<violation number="1" location="server/deps/llama.cpp/ggml/src/ggml-cuda/gated_delta_net.cu:593">
P2: `ggml_gated_delta_net_inplace` now has backend-dependent semantics: CUDA writes the final recurrent state into `src_state`, while the CPU GDN implementation still writes only into `dst`. A graph that falls back off CUDA will keep using stale state, so the in-place flag should either be implemented/rejected consistently in non-CUDA backends or kept out of the generic API path.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: ggml_gated_delta_net_inplace now has backend-dependent semantics: CUDA writes the final recurrent state into src_state, while the CPU GDN implementation still writes only into dst. A graph that falls back off CUDA will keep using stale state, so the in-place flag should either be implemented/rejected consistently in non-CUDA backends or kept out of the generic API path.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/deps/llama.cpp/ggml/src/ggml-cuda/gated_delta_net.cu, line 593:

<comment>`ggml_gated_delta_net_inplace` now has backend-dependent semantics: CUDA writes the final recurrent state into `src_state`, while the CPU GDN implementation still writes only into `dst`. A graph that falls back off CUDA will keep using stale state, so the in-place flag should either be implemented/rejected consistently in non-CUDA backends or kept out of the generic API path.</comment>

<file context>
@@ -586,6 +589,9 @@ void ggml_cuda_op_gated_delta_net(ggml_backend_cuda_context & ctx, ggml_tensor *
     const float * s_d   = (const float *) src_state->data;
     float *       dst_d = (float *) dst->data;
+    const int64_t attn_score_elems = S_v * H * n_tokens * n_seqs;
+    const bool    inplace_state = ggml_get_op_params_i32(dst, 1) != 0;
+    float *       state_out_d = inplace_state ? (float *) src_state->data : dst_d + attn_score_elems;
     const int *   parent_ids_d = src_parent
</file context>