See how vLLM's KV cache and scheduler actually behave — no GPU, no model weights.
Live demo · vLLM simulator #47922
Evaluating KV-cache behavior normally means standing up the whole model. But for recorded traces the outputs are already known, so the weights don't matter — only the scheduling and cache path do. kvlens replays traces through vLLM's native simulator, which loads model metadata only yet runs the real KV-cache and scheduling code, and shows what happens turn by turn: prefix reuse, context growth, KV-cache groups and attention types, block store/evict. The numbers are the engine's — so you can study large models on a laptop.
Static, no build, ships with a sample — just open it:
git clone https://github.com/sagearc/kvlens && cd kvlens
python -m http.server 8000 --directory web # → http://localhost:8000Two tabs: Sessions (per-turn cached vs new) and Radix tree (blocks shared across sessions / evicted).
The web app is a thin painter: all radix-tree derivation (run-merging, prune/dim,
divergence, leaf counts) runs in Python (src/kvlens/tree.py) and ships as a
per-frame view-model the browser just draws. Same view-model, whether it comes
from the static file or the live stream below.
Instead of replaying a static file, serve the same view-model over
server-sent events (needs the serve extra: pip install -e '.[serve]'):
kvlens serve --replay # stream the committed sample over SSE (a demo)
kvlens serve --live --traces <trace.json> --indices 0,1,2 # live vLLM (needs [capture])Then open http://localhost:8000/tree.html?live=1. Plain kvlens serve stays a
zero-dependency stdlib static server.
Needs vLLM with the KV-cache simulator (#47922, until it lands upstream), built from source — see the vLLM CPU build-from-source guide. Then:
kvlens capture --traces <trace.json> --indices 0,1,2 # ShareGPT format
kvlens serveThe shipped demo is synthetic (examples/gen_demo_trace.py). Real traces can
carry licensed content — never commit captures made from them (a pre-commit hook
guards web/*.json).
make setup, then make serve (no vLLM needed). Design notes live in
AGENTS.md. PRs welcome.
Apache-2.0 · built on vLLM · avatars via DiceBear · AI assistance was used.