Skip to content

deps: bump llama.cpp - RDNA3/RDNA4 MMQ tile override (+6-8% decode)#5

Closed
geometric[bot] wants to merge 1 commit into
mainfrom
feat/rdna-mmq-tile-submodule-bump
Closed

deps: bump llama.cpp - RDNA3/RDNA4 MMQ tile override (+6-8% decode)#5
geometric[bot] wants to merge 1 commit into
mainfrom
feat/rdna-mmq-tile-submodule-bump

Conversation

@geometric

@geometric geometric Bot commented Jun 26, 2026

Copy link
Copy Markdown

Summary

Bumps the server/deps/llama.cpp submodule (luce-dflash) to pick up the
RDNA3/RDNA4 MMQ tile override for DFlash's small spec-decode verify batches.

The change uses a 48x64/4-warp tile in place of the stock 128x128/8
tile and keeps output bit-identical in the measured decode workloads.

Impact

Scenario before after gain
gfx1201 (R9700), Qwen3.6-27B Q4_K_M, --ddtree-budget=22, n_gen=256 54.65 tok/s 59.37 tok/s +8.3%
gfx1100 (RX 7900 XTX), Qwen3.6-27B Q4_K_M, --ddtree-budget=22, n_gen=256 56.78 tok/s 60.18 tok/s +6.0%

Implementation

  • Bumps server/deps/llama.cpp to a luce-dflash commit containing the smaller RDNA MMQ tile.
  • Documents the HIP performance context in server/docs/HIP_PERF_PLAN.md.
  • Leaves gfx1151/RDNA3.5 on the default path because it was not benchmarked.

Runtime Flags / Configuration

No new runtime flags or configuration switches.

Validation

Check Result
10-prompt HE mean, Qwen3.6-27B Q4_K_M, --ddtree-budget=22, n_gen=256 Output bit-identical; gfx1201 improves 54.65 -> 59.37 tok/s, gfx1100 improves 56.78 -> 60.18 tok/s.

Notes

Depends on Luce-Org/lucebox-ggml#18 for the underlying kernel
change. This branch points the submodule at that PR's head commit; before
merge, repoint the SHA to the luce-dflash merge commit of Luce-Org#18.

This is marked draft for testing the GitHub App manual PR flow.

Bumps server/deps/llama.cpp (luce-dflash) to pick up the smaller 48x64/4-warp
MMQ tile for DFlash spec-decode verify batches on consumer RDNA. Output is
bit-identical; decode at --ddtree-budget=22, Qwen3.6-27B Q4_K_M:
  gfx1201 (R9700):       54.65 -> 59.37 tok/s (+8.3%)
  gfx1100 (RX 7900 XTX): 56.78 -> 60.18 tok/s (+6.0%)

Depends on Luce-Org/lucebox-ggml#18; the submodule SHA should be
repointed to the luce-dflash merge commit before this lands.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@robbiemccorkell robbiemccorkell deleted the feat/rdna-mmq-tile-submodule-bump branch June 26, 2026 15:12
@robbiemccorkell robbiemccorkell restored the feat/rdna-mmq-tile-submodule-bump branch July 1, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants