GLM-5.2 quantization crash and sparse DSA indexer loading

[csabakecskemeti]

Summary

Two bugs found while quantizing zai-org/GLM-5.2 to GGUF on current master.

---

Bug 1 — Quantization crash: Bad layer N for tensor blk.N.ffn_down_exps.weight. Must be in [0, N)

Root cause

PR #24060 (merged ~June 5 2026) refactored hparams.n_layer into a method that subtracts nextn/MTP prediction layers:

// after #24060:
n_layer() = n_layer_all - n_layer_nextn

In src/llama-quant.cpp, init_quantize_state_counters() uses this value to size the FFN layer counter arrays:

// broken after #24060:
qs.n_ffn_down = qs.n_ffn_gate = qs.n_ffn_up = (int)qs.model.hparams.n_layer();

GLM-5.2 has n_layer_all = 80 and n_layer_nextn = 1, so n_layer() = 79. But the GGUF file contains FFN tensors for all 80 blocks (including the nextn block), so when the quantizer processes block 79 it finds the array is only sized to 79 — hence Bad layer 79.

This bug affects any model with nextn_predict_layers > 0 (GLM-5.2, BailingMoE2, any future MTP model). The same issue was independently reported upstream as ggml-org#24379.

Fix

// src/llama-quant.cpp
- qs.n_ffn_down = qs.n_ffn_gate = qs.n_ffn_up = (int)qs.model.hparams.n_layer();
+ qs.n_ffn_down = qs.n_ffn_gate = qs.n_ffn_up = (int)qs.model.hparams.n_layer_all;

---

Bug 2 — Model loading crash: missing tensor 'blk.3.indexer.k_norm.weight'

Root cause

GLM-5.2 uses a Dense Sparse Attention (DSA) lightning indexer, but only in a sparse subset of layers. The selection is controlled by two model config parameters:

• index_skip_topk_offset: 3 — the first 3 blocks always get an indexer (dense lead blocks)
• index_topk_freq: 4 — after that, every 4th MoE block gets an indexer

This gives 22 out of 79 blocks with indexer tensors: [0, 1, 2, 6, 10, 14, 18, 22, 26, 30, 34, 38, 42, 46, 50, 54, 58, 62, 66, 70, 74, 78]. The source model safetensors contain exactly 110 indexer tensors (22 blocks × 5 tensor types), confirming this is by design — not an incomplete download.

The load_arch_tensors in glm-dsa.cpp iterates all 79 blocks and calls create_tensor for the 5 indexer tensors on every block without TENSOR_NOT_REQUIRED. For blocks that don't have indexers (e.g. block 3), this triggers a hard "missing tensor" abort.

Additionally, the graph builder in deepseek32.cpp enters the // lightning indexer block unconditionally, which would dereference null layer pointers on any non-indexer block.

Why Unsloth's GGUF doesn't have this problem

unsloth/GLM-5.2-GGUF avoids the crash by duplicating indexer tensors into all 79 blocks, padding the missing 57 blocks with copies of nearby tensors. This was noted in issue ggml-org#24730 by @artraze: "Unsloth likely duplicated index tensors for cached layers rather than proper index caching." That approach inflates GGUF size and masks the real design intent of the model.

Fix

src/models/glm-dsa.cpp — mark all 5 indexer tensor creates as TENSOR_NOT_REQUIRED:

layer.indexer_k_norm   = create_tensor(..., flags | TENSOR_NOT_REQUIRED);
layer.indexer_k_norm_b = create_tensor(..., flags | TENSOR_NOT_REQUIRED);
layer.indexer_proj     = create_tensor(..., flags | TENSOR_NOT_REQUIRED);
layer.indexer_attn_k   = create_tensor(..., flags | TENSOR_NOT_REQUIRED);
layer.indexer_attn_q_b = create_tensor(..., flags | TENSOR_NOT_REQUIRED);

src/models/deepseek32.cpp — guard the lightning indexer block with a null check:

• // lightning indexer
• {

• // lightning indexer - skipped when tensors are absent (e.g. GLM-5.2 only has them on certain blocks)
• if (model.layers[il].indexer_attn_q_b) {

top_k is initialized to nullptr before this block. When the indexer is absent, top_k = nullptr causes build_attn to fall back to full attention — semantically correct since those blocks were never meant to use sparse selection.

---

Note on full DSA support

A proper fused GGML_OP_LIGHTNING_INDEXER is tracked in upstream PR ggml-org#24231 (open, by @fairydreaming), which reduces compute buffer from ~168 GB to ~5.8 GB. The fixes here are about correctness (model loads and runs), not full DSA performance.

---

Test

Quantized zai-org/GLM-5.2 → Q4_K_M using this branch:

• Before bug 1: crash at blk.79.ffn_down_exps.weight — Bad layer 79 ... Must be in [0, 79)
• After bug 1 fix: quantization completes cleanly across all 80 blocks
• Before bug 2: llama-server aborts with missing tensor 'blk.3.indexer.k_norm.weight'
• After bug 2 fix: model loads and runs on patched master

Used: transformers-5.12.1(probably transformers>=5.0.0 is enough)