model : add Meta Muse Glimmer 30B support (port of llama.cpp#26841)

[bchap1n]

Overview

Ports upstream llama.cpp Muse Glimmer support (ggml-org#26841, commit 62bf73d25) into BeeLlama so KVarN, DFlash, and the multimodal pipeline work with Muse Glimmer models out of the box. The port is a faithful cherry-pick of the merged upstream change — every muse-glimmer source file is byte-identical to upstream master — adapted only where the fork's tree has diverged (chat/speculative integration points). BeeLlama docs are updated per CONTRIBUTING.md.

Model support

• muse-glimmer architecture: 52-layer 30B (and smaller) variants, 131072-token context, interleaved [L,L,L,G] sliding-window attention (window 2048), QK-norm with qk_scale_factor folded at conversion time, attention output gate (sigmoid) around SDPA, SwiGLU dense FFN with post-norms, final-logit tanh softcap, logit-scale output multiplier. Per-layer residuals are exposed for DFlash drafts (dflash.target_layers).
• Multimodal: 50-block ViT vision encoder (2D RoPE, sparse block-diagonal window attention — every 4th and last layer global, pixel-shuffle downsample, 3-layer adapter MLP into the LLM residual dim) with a dynamic preprocessor that picks the patch grid closest to the input aspect ratio under the per-image token cap.
• Chat template: assistant turns rendered as "<|start|>assistant to=<recipient><|message|>" messages (chain-of-thought to=self terminated by <|eom|>, final answer to=user terminated by <|eot|>), with a PEG parser and ATEM tool-call grammar.
• Conversion: convert_hf_to_gguf.py for the target, vision (mmproj), and DFlash drafter (MuseGlimmerAssistantModel, converted with --target-model-dir so it inherits the target tokenizer and emits dflash.block_size / dflash.target_layers).
• Speculative fix: the DFlash draft's KV cache is now seeded from target-layer features on embedding-bearing (multimodal) prefills, which previously skipped the batch and left the draft cache with a hole at the next injection.

Additional information

Verification — local RTX 3090 rig

Hardware: RTX 3090 24 GiB (sm_86), Windows 11, CUDA 13.2, Release, -DGGML_CUDA=ON -DGGML_CUDA_FA=ON -DGGML_CUDA_FA_ALL_QUANTS=ON -DGGML_CUDA_KVARN=ON, commit 8ec951db0 (binary b11161-8ec951db0).

Model files (K-quant GGUFs converted from the HF checkpoints, local):

• muse-glimmer-30B-kquant-17gb.gguf — 30B, 52 layers, ctx 131072, [L,L,L,G] SWA pattern, Q4_K Medium, 15.59 GiB, 27.85 B params
• dflash-kquant.gguf — DFlash drafter, 5 layers, block_size 16, target_layers [2,14,26,38,50], 2.6 B
• mmproj-kquant.gguf — vision encoder, 50-block ViT, patch 14, merge 2, projection 6656, 1.9 B

Full-stack server command (verified end to end):

llama-server -m muse-glimmer-30B-kquant-17gb.gguf --mmproj mmproj-kquant.gguf \
  --spec-type draft-dflash --spec-draft-model dflash-kquant.gguf --spec-draft-n-max 15 \
  --flash-attn on --cache-type-k kvarn5 --cache-type-v kvarn4 \
  --ctx-size 131072 -ngl 99

Check	Result
Text generation	finish=stop; correct chain-of-thought plus final answer
Multimodal	correctly described an Apollo 11 NYT front-page image through the vision encoder
DFlash decode	65–73 tok/s with drafter (text), 64–65 tok/s on image prompts vs 28–33 tok/s without (≈2.2×); draft acceptance 0.24–0.38, mean len 4.5–6.0; adaptive profit controller active (0→15, score 23.24)
KVarN	kvarn5/kvarn4 engaged (type_k/type_v in llama-bench; log enabling structured KVarN cache type kvarn_k5v4_g128); SWA override pair kvarn3/kvarn3 also verified
Context	131072 allocated; full stack fits in 23.3/24 GiB
Long prompt	128 KB text prompt prefilled at 1032 tok/s, generated a correct continuation

llama-bench (30B Q4_K, -p 512 -n 64 -t 1 -fa on):

Cache	pp512 (tok/s)	tg64 (tok/s)
f16 (default)	1173.9 ± 36.7	30.8 ± 0.2
kvarn5/kvarn4	1177.8 ± 20.7	31.4 ± 0.1
q5_0/q4_1	1218.4 ± 30.9	30.9 ± 0.2

KVarN runs at parity with f16 on this model; DFlash roughly doubles decode throughput.

Tests

• test-llama-archs covers LLM_ARCH_MUSE_GLIMMER (SWA-pattern GGUF path) and runs on the Linux CI; the test is excluded from Windows builds (NOT WIN32 OR NOT BUILD_SHARED_LIBS).
• test-chat-template and test-chat-peg-parser pass locally.

Commits

1. model : add Muse Glimmer architecture
2. gguf-py : add Muse Glimmer conversion and tensor mappings
3. mtmd : add Muse Glimmer vision encoder
4. common : add Muse Glimmer chat template
5. speculative : seed DFlash draft cache on embedding prefill
6. docs : document Muse Glimmer support

Requirements

• I have read and agree with the contributing guidelines (and the upstream llama.cpp guidelines it inherits)
• AI usage disclosure: YES — AI used for code review, commit-series crafting, and PR preparation; all code changes are a verbatim port of upstream llama.cpp#26841 (62bf73d25), byte-identical in the muse-glimmer sources and reviewed by the author