Name and Version
b9935 - latest
Operating systems
Windows
Which llama.cpp modules do you know to be affected?
llama-server
Command line
llama-server.exe --alias "Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL" --model "models/Qwen3.6-35B-A3B-MTP-UD-Q4_K_XL.gguf" --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.5 --host 0.0.0.0 --port 11434 --jinja --chat-template-kwargs "{"enable_thinking": true}" --chat-template-kwargs "{"preserve_thinking": true}" --ctx-size 131072 --batch-size 2048 --ubatch-size 2048 --threads 16 --threads-batch 32 --n-cpu-moe 30 --n-gpu-layers 42 --no-mmap --mlock --flash-attn on --cache-type-k q4_0 --cache-type-v q4_0 --temperature 0.6 --top-p 0.95 --top-k 20 --min-p 0.05 --repeat-penalty 1.0 --presence-penalty 0.0Problem description & steps to reproduce
I previously applied b9180 which is the first release version supports MTP, btw I'm using qwen3.6-35b-a3b-mtp-ud-q4-k-xl. Recently I upgraded to new version b9934, then I find the version is much slower than b9180. Eventually I find the throughput drops by approx. 15-20% from this version b9235 (the version is merged into the main), wonder if someone got the same experience and can tell what happened in the "clean-up" activity.
My test environment:
CPU: AMD 9950X
GPU: 5070Ti
RAM: 96 GB
Windows 11
(although there's only 40 layers in this model, but after test I found 42 works faster, btw 99 won't help on --n-gpu-layers 42)
Comparison on b9222 vs b9235:
b9222: vram 13 GB used (start from scratch, delta vram before and after running llama-server), reply "hello" generation speed 68.4 t/s
b9235: vram 13.1 GB used, reply "hello" generation speed 63 t/s
the gap is more obvious in a long context task (a 20K tokens prompt test, code review task), generation speed is 61.3 t/s (output 4671 tokens) vs. 49.7 t/s (output 3730 tokens)
First Bad Commit
No response
Relevant log output
Logs
Name and Version
b9935 - latest
Operating systems
Windows
Which llama.cpp modules do you know to be affected?
llama-server
Command line
Problem description & steps to reproduce
I previously applied b9180 which is the first release version supports MTP, btw I'm using qwen3.6-35b-a3b-mtp-ud-q4-k-xl. Recently I upgraded to new version b9934, then I find the version is much slower than b9180. Eventually I find the throughput drops by approx. 15-20% from this version b9235 (the version is merged into the main), wonder if someone got the same experience and can tell what happened in the "clean-up" activity.
My test environment:
CPU: AMD 9950X
GPU: 5070Ti
RAM: 96 GB
Windows 11
(although there's only 40 layers in this model, but after test I found 42 works faster, btw 99 won't help on --n-gpu-layers 42)
Comparison on b9222 vs b9235:
b9222: vram 13 GB used (start from scratch, delta vram before and after running llama-server), reply "hello" generation speed 68.4 t/s
b9235: vram 13.1 GB used, reply "hello" generation speed 63 t/s
the gap is more obvious in a long context task (a 20K tokens prompt test, code review task), generation speed is 61.3 t/s (output 4671 tokens) vs. 49.7 t/s (output 3730 tokens)
First Bad Commit
No response
Relevant log output
Logs