Enable Q8 KV cache for DeepSeek V4 fast prefill

[Whamp]

Overview

This lets DeepSeek V4 use a q8_0 K/V cache without losing the fork's CUDA-graph prefill path. The existing F16 path stays unchanged.

The Q8 path failed in three places:

1. CUDA rejected packed Q8 concat, then the CPU fallback copied logical elements as full quantization blocks and corrupted cache rows.
2. Cache padding used F16-to-Q8 copies, which CUDA does not support in this tree. Each compressed-attention layer fell back to CPU, increasing graph splits from 5 to 87.
3. Two long-context gather paths cast compressed Q8 rows to F16 before concatenating them with raw Q8 rows. Prompts spanning multiple micro-batches hit a tensor-type assertion.

This patch adds block-aware quantized concat on CUDA and CPU, fills Q8 padding through the supported F32 conversion path, and keeps gathered rows in the cache's type. It also adds Q8 concat regression cases.

DeepSeek V4 now accepts either matching F16 K/V or matching q8_0 K/V. The Q8 path requires LLAMA_ATTN_ROT_DISABLE=1 and fails at startup when the flag is missing. Other cache-type combinations also fail at startup.

Additional information

Validation

Tested on 4× RTX 3090 at 230 W per card with Antirez DeepSeek V4 Flash IQ2_XXS, q8_0 K/V, and all DSV4 optimization flags enabled.

• test-backend-ops test -o CONCAT -b CUDA0: 45/45 passed, including Q8 dimension-2 concat and strided layer views
• verify-full.sh: 9/9 passed
• verify-stress.sh: 8/8 passed, with exact needle recall through 183,978 prompt tokens
• 430K reserved-context test: exact recall at 395,282 prompt tokens
• quality-test.sh --medium: 67/75, matching the stock b10200 Q8 baseline
• continuous soak: 25/25 responses, no errors or empty responses

At 200K reserved context, Q8 measured 1,538 prefill tokens/s at 98K prompt depth and 1,341 prefill tokens/s at 195K. This was 3–9% faster than F16 on the same fork. Decode measured 33–38 decode tokens/s.

The concat repair adapts the mainline approach from ggml-org/llama.cpp#25678 to this fork's older CUDA code. The DeepSeek cache handling also follows the constraints established in #25202 and #25871.

The downstream profile and full test record are in noonghunna/club-3090#940.

Requirements (i'm assuming these are leftover from the original upstream fork but . . .)

• I have read and agree with the contributing guidelines
• AI usage disclosure: Yes. AI was used for diagnosis, implementation, testing, and drafting this PR.