Eval bug: gemma4-assistant MTP draft model fails to load — "invalid vector subscript" (regression: works on b9553, broken on b9702/b9717)

[Metodv1]

Name and Version

Affected (BROKEN):
version: 9717 (8141e73) built with Clang 20.1.8 for Windows x86_64
version: 9702 (968c438)

Last known good (WORKS):
version: 9553 (9e3b928) built with Clang 19.1.5 for Windows x86_64

Official Windows CUDA 13.3 release builds.

Operating systems

Windows

GGML backends

CUDA

Hardware

Ryzen 7 5800X + 4070Ti Super 16gb

Models

Speculative decoding (MTP draft) setup:

• Target: gemma4-v2-Q8_0.gguf (arch: gemma4)
• Draft (MTP): gemma-4-12B-it-MTP-Q8_0.gguf (arch: gemma4-assistant, nextn_predict_layers=4, block_count=4)

Both from: https://huggingface.co/yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF

The draft is the unsloth-converted GGUF of Google's official gemma-4-12B-it-assistant draft head.
Gemma 4 MTP support landed in mainline via PR #23398 (merged 2026-06-07). The model author confirmed
the files are correct and not built against any private fork. Also reproduced with the -MTP-BF16 variant.

Problem description & steps to reproduce

Loading a gemma4-assistant MTP draft model for speculative decoding (--spec-type draft-mtp) crashes
during model load with "invalid vector subscript". The main/target model loads fine — only the draft
load crashes, so generation never starts.

This is a REGRESSION. The exact same models + command load cleanly and work on b9553 (~2.5x speedup,
lossless), but crash on b9702 and b9717. Something in the gemma4-assistant loader / context-init path
regressed between b9553 and b9702.

Command used (crashes on b9702/b9717, works on b9553 with the older flag names):

b9702/b9717 (new flag names):
llama-server -m gemma4-v2-Q8_0.gguf
--spec-type draft-mtp --spec-draft-model gemma-4-12B-it-MTP-Q8_0.gguf
--spec-draft-ngl 99 --ctx-size 65536 -ngl 99 --flash-attn on --jinja

b9553 (old flag names, WORKS):
llama-server -m gemma4-v2-Q8_0.gguf
--model-draft gemma-4-12B-it-MTP-Q8_0.gguf --spec-type draft-mtp
--spec-draft-n-max 4 -ngl 99 -ngld 99 -fa on --jinja

Key detail: the "invalid vector subscript" fires the instant load_tensors: loading model tensors...
begins for the draft — before any tensor data is read. So the out-of-bounds is in the gemma4-assistant
tensor-layout / model-build code, not in reading tensor data. Likely an out-of-range index over a
per-layer array (block_count=4), e.g. attention.head_count_kv=[8,8,8,1] or
sliding_window_pattern=[true,true,true,false], or nextn_predict_layers / embedding_length_out=3840 handling.

The crash occurs AFTER the memory-fitting step (which harmlessly aborts with "n_gpu_layers already set
by user to 99, abort"), at the actual draft load — so -fit off does not change it. The
"Gemma4Assistant requires ctx_other to be set (this warning is normal during memory fitting)" line also
appears on the working b9553 build and is harmless; the fatal regression is the "invalid vector subscript".

Tried — all crash identically on b9702/b9717, both quants:
b9702 + Q8_0 -> crash
b9717 + Q8_0 -> crash
b9717 + BF16 -> crash
b9553 + Q8_0 -> WORKS

So it is neither a file/quant issue nor a user-config issue. It is in the model loader and reproduces
with any binary that loads this draft.

First Bad Commit

Not bisected. Good: b9553 (9e3b928). Bad: b9702 (968c438) and b9717 (8141e73).
The regression was introduced somewhere in the b9553 -> b9702 range.

Relevant log output

Logs

 srv    load_model: loading draft model '...gemma-4-12B-it-MTP-Q8_0.gguf'
  load_tensors: loading model tensors, this can take a while... (mmap = true, direct_io = false)
  llama_model_load: error loading model: invalid vector subscript
  llama_model_load_from_file_impl: failed to load model
  srv    load_model: failed to load draft model, '...gemma-4-12B-it-MTP-Q8_0.gguf'
  srv  llama_server: exiting due to model loading error
  A pod blokiem console (poza nim) wklej link do nowego gista:
  Full verbose (-lv 5) log from b9717: <https://gist.github.com/Metodv1/e6e06c9058fdd0b32881699e4ae00e24>

[HelloItMeMort]

Confirming regression as well

[poog26]

I'm experiencing this as well.

[Kononnable]

0.09.272.097 I llama_prepare_model_devices: using device CUDA0 (NVIDIA GeForce RTX 4070 Ti SUPER) (0000:08:00.0) - 0 MiB free
Looks like a lack of resources, not a regression. You don't seem to have enough resources to run this quant with specified settings.

Why did it work before? Probably other programs were using less VRAM or some changes in llama.cpp increased the memory consumption which put your configuration over the line (increasing memory usage does not necessarily mean regression). You may want to try after closing all unnecessary applications(to free vram), or bisect if you still think this is a llama.cpp issue.

[Snegovik]

I have too. LLAMA_CPP_VERSION=b9672

Error: 500 Internal Server Error: llama-server process has terminated: llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this warning is normal during memory fitting)

llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this warning is normal during memory fitting): llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this warning is normal during memory fitting)

llama_init_from_model: failed to initialize the context: Gemma4Assistant requires ctx_other to be set (this warning is normal during memory fitting)

[samos667]

llama-server -m gemma4-v2-Q8_0.gguf
--spec-type draft-mtp --spec-draft-model gemma-4-12B-it-MTP-Q8_0.gguf
--spec-draft-ngl 99 --ctx-size 65536 -ngl 99 --flash-attn on --jinja

Using b10091 the model load but use 18Go of VRAM