[llama.cpp report] M1 Pro

[limoncc]

💾 CPU model

M1-Pro-10core

🎮 GPU model

M1-Pro GPU (16-core)

📊 Performance data

./llama-bench -m ../models/rwkv7-g1-2.9b/rwkv7-2.9B-g1-Q8_0.gguf

model	size	params	backend	threads	test	t/s
rwkv7 2.9B Q8_0	3.03 GiB	2.95 B	Metal,BLAS,RPC	8	pp512	482.84 ± 2.19
rwkv7 2.9B Q8_0	3.03 GiB	2.95 B	Metal,BLAS,RPC	8	tg128	10.14 ± 0.02

build: c8ade300 (5960)

./llama-bench -m ../models/rwkv7-g1-2.9b/rwkv7-2.9B-g1-F16.gguf

model	size	params	backend	threads	test	t/s
rwkv7 2.9B F16	5.52 GiB	2.95 B	Metal,BLAS,RPC	8	pp512	502.42 ± 4.90
rwkv7 2.9B F16	5.52 GiB	2.95 B	Metal,BLAS,RPC	8	tg128	8.75 ± 0.17

build: c8ade300 (5960)

[shoumenchougou]

LGTM

[shoumenchougou]

We will soon increase the RWKV metal backend inference speed of llama.cpp.

[shoumenchougou]

can you test again with llama.cpp build: 9285325c (6817) ?

In issue #22, we found Apple M2 8-core can reaches rwkv7 2.9B F16 | Metal,BLAS | 4 threads | 11.26 t/s 🌹