ggml, llama : add KV cache size limiting and block tracking infrastructure#18747

Open

pestopoppa wants to merge 17 commits into

ggml-org:masterfrom

pestopoppa:feature/paged-attention

Commits on Jan 10, 2026

feat: add --moe-n-expert flag for MoE expert count override (Hard Mask)

pestopoppa
and
claude
committed
feat: add layer skip / early exit support for speculative decoding

pestopoppa
and
claude
committed
feat: add layer skip support for qwen3vl-moe and qwen3next

pestopoppa
and
claude
committed
lookahead: fix n_seq_max and kv_unified configuration
pestopoppa
committed
lookup, lookahead: fix crash when n_ctx not specified
pestopoppa
committed
ggml-cpu: parallelize tensor repacking with OpenMP
pestopoppa
committed
docs: add branch management rules to prevent build issues

pestopoppa
and
claude
committed
kv-cache : optimize SWA slot reuse with forward-looking masking
pestopoppa
committed
kv-cache: fix SWA cell reuse to ensure mathematical correctness
pestopoppa
committed
feat: implement CPU paged attention for flash attention

pestopoppa
and
claude
committed
feat: implement dynamic block allocation for paged attention

pestopoppa
and
claude
committed
feat: add block pool statistics for debugging paged attention

pestopoppa
and
claude
committed
feat: add KV cache memory reduction for paged attention

pestopoppa
and
claude
committed
test: add unit tests for block pool and table

pestopoppa
and
claude
committed
feat: add CLI flags for paged attention

pestopoppa
and
claude
committed

Commits on Jan 11, 2026

refactor: trim verbose comments in llama-kv-block.h

pestopoppa
and
claude
committed

Commits on Jan 12, 2026

refactor: remove unrelated changes from KV cache PR

pestopoppa
and
claude
committed