residual-stream

Mechanistic experiments on how LLMs represent and compute arithmetic internally, with strict no-parser controls, reproducible audits, and an interactive article.

Hyperprobe is the Python implementation of the framework proposed in the paper "Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures".

Model forensics through geometry: residual-stream projections, activation traces, and attention routing.

Configurable character-level transformer training suite with built-in mechanistic interpretability toolkit — scale to 150M+ parameters and beyond, no ceilings, only hardware limits. Inspect attention weights, hidden states, and head specialisation across all layers. Documented circuit findings included.

Measurement harness and raw data: a cheap robust statistic, sigma_med*sqrt(d), predicts the tolerance boundary of additive interventions in language models. Kappa stays in [0.2, 1.2] across 11 checkpoints and 6 architectures, while raw boundaries span ~4600x.

An empirical study probing the residual stream vectors of google/gemma-2-2b-it during contrastive corporate finance compliance constraints. Features a native PyTorch layer-tracking notebook and an interactive 4-column Streamlit web dashboard analyzing geometric activation divergence deltas.

Unsupervised latent-work and residual-stream interference profiling for language models

PDSF: prediction-anchored decomposition of transformer residual streams — pipeline, prompt sets, and non-regenerable data for Guda (2026)

Do LLMs encode "I'm being shut down" differently from "another model is shut down"? A 10-model residual-stream study — and a cautionary tale: the naive difference-of-means result looks strong (10/10), but honest placebo controls show it's largely trivial. Negative result, useful method.

Experimental Recurrent Residual Stream for GPT-2

Empirical evidence for predictive coding tendencies in the GPT-2 family: residual stream convergence, activation patching, MLP transform analysis, zero-ablation, and logit lens across 7 languages.

Lightweight residual-stream adapters for frozen LLMs: match LoRA at equal parameters, add inference-time steering, and run many behaviors per token on one small model for edge and local AI.