Skip to content

sycl: add Q2_K to DMMV reorder path#25064

Draft
malsbat wants to merge 1 commit into
ggml-org:masterfrom
aicss-genai:q2-k-dmmv-reorder
Draft

sycl: add Q2_K to DMMV reorder path#25064
malsbat wants to merge 1 commit into
ggml-org:masterfrom
aicss-genai:q2-k-dmmv-reorder

Conversation

@malsbat

@malsbat malsbat commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Overview

Add Q2_K to DMMV reorder path.

Additional information

The measured results are on top of #25063.

export GGML_SYCL_PRIORITIZE_DMMV=1
./build/bin/llama-bench -p 64 -n 16 -r 1 -ngl 999 -dev SYCL0 -m /models/Qwen3.5-27B-Q2.gguf
model test baseline t/s KQPI=1 t/s KQPI=1 reordered t/s
qwen35 27B Q2_K - Medium pp64 155.74 ± 0.00 157.79 ± 0.00 160.27 ± 0.00 1.028
qwen35 27B Q2_K - Medium tg16 14.60 ± 0.00 11.34 ± 0.00 18.31 ± 0.00 1.254

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES, used to author the new functions based on the existing reorder paths.

Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
@github-actions github-actions Bot added ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language labels Jun 26, 2026

@arthw arthw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's good job!
It could increase the perf on LLM Qwen3.5-27B-UD-Q2_K_XL.gguf.

Will you change it from draft to open status?
Or have you any plan to update this PR later?

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants