[Vulkan][Intel][GPU] MoE Q4_K_M TG speed degradation from b8323

[savvadesogle]

Hello
Part I (#20893)
TG speed degradation starting from b8322

66 -> 49

2x Intel Arc A770 (16gb)
WIndows 11
Driver: 8626

llama-bench -m T:\models\unsloth\Qwen3-30B-A3B-Instruct-2507-GGUF\Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf  -ngl 100

b8322

model	size	params	backend	ngl	test	t/s
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	pp512	215.03 + 1.52
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	tg128	61.87 + 0.04

build: 557fe2d (8322)

b8323

model	size	params	backend	ngl	test	t/s
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	pp512	218.18 + 0.97
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	tg128	49.13 + 0.25

build: 57819b8 (8323)

Best (7400) for an example

model	size	params	backend	ngl	mmap	test	t/s
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	0	pp512	302.57 + 2.31
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	0	tg128	66.29 + 0.24

build: 0759b09 (7400)

[github-actions]

This issue might be similar or related to the following issue(s):

• [Vulkan][Intel][GPU] MoE Q4_K_M PP speed degradation from b7549 #20893: Same user/reporting same MoE Q4_K_M performance degradation on Vulkan/Intel Arc (PP speed)
• Eval bug: [Vulkan] Regression starting from b8459 #20890: Vulkan regression on Intel Arc for MoE models (Qwen3.5 35B) starting from b8459
• Misc. bug: [VULKAN[ [Intel] Qwen3-Coder-30B-A3B Low PP performance of Q4 and Q6 quants compared to Q8 on Intel Arc A770  #19887: Low PP performance of Q4/Q6 quants vs Q8 on Intel Arc A770 with MoE models
• Eval bug: Vulkan branch produces gibberish using multi-IntelGPU setup (3 x A770) past build 8183. #20097: Vulkan issues with multi-IntelGPU setup (3x A770) and Qwen3.5 MoE models
• [SYCL][Intel] Low performance on MoE models - SYCL is slower than VULKAN (A770) #19918: Low performance on MoE models with SYCL vs Vulkan on Intel Arc A770

This comment was auto-generated locally using llama.cpp version: 8456 (b31b30f) on NVIDIA DGX Spark

[TheBlueMatt]

The change between these releases was #20463. This should have been fixed by #20927 please retest.

[savvadesogle]

Hello, @TheBlueMatt

7400 vs 8914
65 vs 26 t/s 🤷‍♂

Windows 11
Driver 8724
2x A770

llama-bench -m T:\models\unsloth\Qwen3-30B-A3B-Instruct-2507-GGUF\Qwen3-30B-A3B-Instruct-2507-Q4_K_M.gguf -ngl 100

model	size	params	backend	ngl	test	t/s
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	pp512	295.57 + 2.30
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	tg128	65.77 + 0.25

build: 0759b09 (7400)

model	size	params	backend	ngl	test	t/s
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	pp512	213.24 + 1.29
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	Vulkan	100	tg128	26.92 + 5.52

build: 8bc492e (8914)

[TheBlueMatt]

Then the issue is a regression because of the correctness fix. Sadly the previous version was simply incorrect :/

[savvadesogle]

Then the issue is a regression because of the correctness fix. Sadly the previous version was simply incorrect :/

There is also regression in the PP speed

#20893 (comment)

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.