Archive readout-aligned geometry finding (CLOSED, not shipped)

docs/research/behavioural-scaleup-probe.md

@@ -5,6 +5,27 @@
 > [`tech-debt-backlog` §4.4](../../openspec/changes/tech-debt-backlog/tasks.md).
 > Reproducible via `python examples/behavioural_gram_scaleup.py`.
 
+## Readout-aligned head-to-head (2026-06-13, `add-readout-aligned-geometry-profile`)
+
+Does building the geometry in GPT-2's **readout subspace** (top-`r` SVD of `gain⊙U`) instead of on raw decoder
+vectors raise the Spearman? **No — it strictly hurts.** Head-to-head on `blocks.10`
+(`--profile {clustered,readout-aligned} --readout-rank R`):
+
+| profile / rank | Spearman(Polygram, Jaccard) |
+|---|---|
+| `clustered` (raw decoder) | **0.640** |
+| `readout-aligned` r=64 | 0.267 |
+| `readout-aligned` r=256 | 0.067 |
+| `readout-aligned` r=768 (full) | **0.640** (= baseline) |
+
+Full-rank readout-projection is an orthonormal rotation (k-means is rotation-invariant) so it reproduces
+`clustered` exactly; every truncation degrades. **Mechanism:** co-firing Jaccard is an **encoder-side**
+phenomenon (which features fire together, in the full-residual geometry); the readout subspace is **decode-side**
+(what the argmax reads through). Readout-alignment is the right basis for the *decode* tax (R2: +52/+31/+40pp
+open-class R@32) but the **wrong** basis for *co-firing*. So this **vindicates** the raw-decoder geometry for
+this claim — 0.640 is not a basis artifact. The `readout-aligned` profile ships as a tool for decode-relevant
+geometry; a decode-side behavioural metric (e.g. logit-attribution overlap) is where it would be expected to help.
+
 ## Context
 
 PR #20 ([§4.2 / `behavioural-gram-probe.md`](behavioural-gram-probe.md))

openspec/changes/archive/2026-06-13-add-readout-aligned-geometry-profile/proposal.md

@@ -0,0 +1,133 @@
+# Readout-aligned dictionary geometry — build the Gram on the basis the model reads through
+
+> **STATUS: CLOSED — NOT SHIPPED (archived as a research finding, 2026-06-13).** The behavioural head-to-head
+> **falsified** the premise: readout-alignment *hurts* the co-firing Spearman (0.640 → 0.27; full-rank recovers
+> the baseline because it's a rotation) — co-firing is **encoder-side** while the readout subspace is
+> **decode-side**. This *vindicates* Polygram's raw-decoder geometry for the co-firing claim. The
+> implementation (PR #113, closed) was correct + tested but is **not** on `main`; the spec delta below was
+> **not** applied to the canonical specs. See the "Gate RESULT" section and `behavioural-scaleup-probe.md`.
+
+Add a `readout-aligned` geometric profile that builds Polygram's dictionary geometry (clustering + knob
+assignment) on feature directions **projected onto the model's readout subspace**, instead of on the **raw
+decoder vectors** it uses today. This is the highest-leverage test of Polygram's load-bearing claim — and it is
+now backed by hard cross-model evidence, not a hunch.
+
+## Why
+
+Polygram's load-bearing claim is **behavioural**: the predicted dictionary geometry must track real co-firing /
+ablation-KL — measured as `Spearman(Polygram-overlap, co-firing Jaccard) = 0.637`
+(`docs/research/behavioural-scaleup-probe.md`). But that geometry is built on **raw decoder vectors**:
+`polygram/geometry/clustered.py` feeds the raw per-feature projections straight into k-means
+(`clustered.py:165`), the cluster centroids (`:176`), the residual-variance fidelity (`:177`), and the
+per-cluster-PCA γ assignment (`:182`). The raw decoder/activation spectrum is exactly the **deceptively
+compressible** basis the research program warns against — the *behaviourally* relevant object is the
+**readout-aligned decision geometry** (the directions the model's argmax actually reads through), not raw
+decoder norm.
+
+That used to be a hypothesis; it is now a **measured, model-general** fact. A powered cross-model run
+(fieldrun `tau_star_powered.py`, wikitext 20k tokens, 3 seeds) shows a trained rank-`r` projection onto the
+**readout-aligned** subspace beats the frozen SVD lens on open-class next-token recovery at every rung:
+**GPT-2 +52pp, Pythia-70m +31pp, Pythia-160m +40pp** (open-class R@32, all 6 model×rank cells win). The
+readout-aligned directions, not the energy-weighted decoder spectrum, govern behaviour — across architectures.
+
+**So if Polygram fits its geometry in the raw-decoder basis, its 0.637 Spearman is partly measuring the
+basis, not the model.** Fixing the basis is the single highest-leverage change to the geometry-tracks-behaviour
+claim — and it is a fair, falsifiable test of **Reckoning #3** (is the quantum Gram load-bearing, or a
+removable scaffold?): if the *right* basis raises the Spearman, the Gram earns its keep; if it doesn't move,
+the predictive power isn't basis-limited and the prune becomes more likely.
+
+## What
+
+### 1. A `readout-aligned` geometric profile — `polygram/geometry/readout_aligned.py`
+
+Mirror `clustered.py`'s `ClusteredKnobAssignment` / `clustered()` factory with a `ReadoutAlignedKnobAssignment`
+/ `readout_aligned()` that, **before** clustering + knob assignment, projects the decoder vectors onto the
+**readout subspace**:
+
+```
+readout subspace  R = top-r right singular directions of  (gain ⊙ U)     # U: (vocab, d_model) unembed
+proj_readout      = projections @ Rᵀ                                      # (n_features, r)
+# then k-means / centroids / residual-variance / γ-PCA run on proj_readout, not the raw projections
+```
+
+This is the same readout-aligned construction R2 validated (the unembed the model reads through, optionally
+weighted by the final-norm `gain`). The strategy's `.assign(projections, ...)` does the projection internally,
+so the `from_sae_lens` call site (`sae_import.py:813`) is otherwise unchanged. Register `readout_aligned()` in
+`polygram/geometry/__init__.py` alongside `clustered()` / `uniform_sphere()`.
+
+### 2. Thread the readout geometry in — `from_sae_lens(u_matrix=, gain=)`
+
+`from_sae_lens` already takes `profile=`. Add net-new optional `u_matrix: np.ndarray | None` (the host unembed,
+`(vocab, d_model)`) and `gain: np.ndarray | float | None` (final-norm gain, default ones). When
+`profile="readout-aligned"`, `u_matrix` is **required** — raise a clear `ValueError` if missing. Polygram does
+**not** load a host model itself (keeps the core numpy-only / low-dep); the caller supplies `U`/`gain` (the
+behavioural extra's `runtime._load_host_model` already exposes a host for the probe).
+
+### 3. Before/after behavioural comparison
+
+Extend `examples/behavioural_gram_scaleup.py` to run **both** profiles (`clustered` vs
+`readout-aligned`, same features/SAE/host) and emit a `polygram_overlap_readout_aligned` column into
+`docs/research/data/scaleup_pairs.csv`; record the head-to-head Spearman table in
+`docs/research/behavioural-scaleup-probe.md` (no new page).
+
+## Scope / what this is NOT
+
+- **Geometry / knob-assignment only.** The profile changes how β/γ are assigned; the encoding's structural form
+  and the **Q-Orca emission are transparent** to it (`_qorca_emit.py` / `Dictionary.gram()` consume the final
+  knob values, not the projection vectors). No change to the quantum machine structure.
+- **Caller supplies `U`/`gain`.** No host-model load inside the Polygram core (no new hard dep on transformers).
+- **v1 host: GPT-2-small** (the existing scaleup-probe substrate). Cross-arch (Pythia, via the now-merged
+  forge adapter) is a follow-up once the GPT-2 result lands.
+- **`clustered` stays the default.** `readout-aligned` is opt-in; existing dictionaries/tests are byte-identical
+  unless the profile is selected.
+
+## Falsifiable acceptance gate (descriptive, both outcomes first-class)
+
+Re-run the behavioural scaleup probe on GPT-2-small head-to-head (raw `clustered` vs `readout-aligned`), same
+panels:
+
+- **WIN:** readout-aligned raises `Spearman(Polygram-overlap, co-firing Jaccard)` to **≥ 0.70** (from 0.637) →
+  readout-alignment is the right basis; the Gram's predictive power was basis-limited (and the quantum lens
+  earns its keep on the right basis).
+- **NO IMPROVEMENT:** Spearman does not move → the Gram's predictive power is **not** basis-limited; an
+  equally-valuable result that makes **Reckoning #3**'s prune (quantum lens = removable scaffold) more likely.
+
+Either way the verdict is the head-to-head table, descriptive; no necessity claims.
+
+### Gate RESULT (2026-06-13) — NO IMPROVEMENT; the readout basis is *wrong for co-firing*
+
+Implemented + run head-to-head on GPT-2-small `blocks.10` (`examples/behavioural_gram_scaleup.py
+--profile {clustered,readout-aligned} --readout-rank R`):
+
+| profile / rank | `Spearman(Polygram-overlap, co-firing Jaccard)` |
+|---|---|
+| `clustered` (raw decoder, baseline) | **0.640** |
+| `readout-aligned`, rank 64 (default) | 0.267 |
+| `readout-aligned`, rank 256 | 0.067 |
+| `readout-aligned`, rank 768 (full `d_model`) | **0.640** (exactly the baseline) |
+
+**Readout-alignment does NOT raise the Spearman — it strictly *hurts*, recovering the baseline only at full
+rank.** The full-rank result is the key tell: a full-rank readout projection is just an orthonormal rotation
+(plus gain weighting), and k-means is rotation-invariant — so it reproduces `clustered` exactly (0.640). Every
+*truncated* rank discards residual dimensions and degrades the co-firing signal.
+
+**The mechanism — and the real lesson — is a basis/metric mismatch:** Polygram's behavioural metric is
+**co-firing** (which features *activate together* — an **encoder-side** phenomenon living in the full-residual
+feature geometry), whereas R2's readout subspace is **decode-side** (the directions the model's *argmax* reads
+through). Readout-alignment is the right basis for the *decode* tax (R2's +52/+31/+40pp) but the **wrong** basis
+for *co-firing*. So this is the **NO-IMPROVEMENT** branch — and it **vindicates** Polygram's raw-decoder
+geometry for the co-firing claim: the 0.640 is *not* a basis artifact, and Reckoning #3's basis-limited concern
+does not apply here.
+
+**What ships:** the `readout-aligned` profile is implemented, tested, and correct — it is a legitimate tool for
+*decode-relevant* geometry questions; it is simply **not** the right basis for the co-firing Gram, which this
+gate establishes cleanly. (A future probe with a *decode-side* behavioural metric — e.g. logit-attribution
+overlap — is where readout-alignment would be expected to help.)
+
+## Related
+
+- `docs/research/behavioural-scaleup-probe.md` — the 0.637 baseline this targets.
+- Manifesto Reckoning #3 (quantum lens load-bearing?) — this is a fair test of it.
+- fieldrun R2 / `tau_star_powered.py` — the powered, model-general evidence that readout-aligned directions
+  govern behaviour; sae-forge `add-capability-ceiling-diagnostic` — the consumer (readout-aligned atom
+  *selection* is the action when its `selection_gap` is large).

openspec/changes/archive/2026-06-13-add-readout-aligned-geometry-profile/specs/readout-aligned-profile/spec.md

@@ -0,0 +1,51 @@
+# readout-aligned-profile Specification (delta)
+
+## ADDED Requirements
+
+### Requirement: Readout-aligned geometric profile
+
+Polygram SHALL provide a `readout-aligned` geometric profile (registered alongside `clustered` /
+`uniform_sphere`) that builds the dictionary geometry — clustering, cluster centroids, residual-variance
+fidelity, and γ assignment — on feature directions **projected onto the model's readout subspace**, not on the
+raw decoder vectors.
+
+The readout subspace SHALL be the top-`r` right singular directions of `gain ⊙ U` (`U`: the host unembed,
+`(vocab, d_model)`; `gain`: the final-norm gain, `(d_model,)`, default ones), and the geometry SHALL be built
+on `projections @ Rᵀ`. The projection SHALL occur **inside** the profile's knob-assignment strategy, leaving
+the `from_sae_lens` call site and the `GeometricProfile`/registry plumbing structurally unchanged.
+
+`get_profile("readout-aligned")` SHALL resolve to this profile. The `clustered` profile SHALL remain the
+default; a run that does not select `readout-aligned` SHALL be byte-identical to current behaviour.
+
+#### Scenario: readout-aligned geometry differs from raw-decoder geometry
+
+- **GIVEN** a feature set and a host unembed `U` whose readout subspace re-orders the raw decoder directions
+- **WHEN** a dictionary is built with `profile="readout-aligned"` and again with `profile="clustered"`
+- **THEN** the two SHALL produce different cluster assignments / β/γ knobs (the geometry is built on a
+  different basis), while the `clustered` result is unchanged from the pre-change behaviour
+
+#### Scenario: deterministic given the readout inputs
+
+- **GIVEN** identical `(projections, U, gain, seed)`
+- **WHEN** the readout-aligned profile assigns knobs
+- **THEN** the assignment SHALL be identical across runs
+
+### Requirement: `from_sae_lens` accepts the readout geometry
+
+`from_sae_lens` SHALL accept optional `u_matrix: np.ndarray | None` (`(vocab, d_model)`) and
+`gain: np.ndarray | float | None` (default ones). When the resolved profile is `readout-aligned`, `u_matrix`
+SHALL be **required** and 2-D with its second axis equal to the feature dimension; absent or malformed,
+construction SHALL raise a clear `ValueError`. Polygram SHALL NOT load a host model itself to obtain `U`/`gain`
+— the caller supplies them (keeping the core dependency-light).
+
+#### Scenario: readout-aligned profile without a host unembed is rejected
+
+- **GIVEN** `profile="readout-aligned"` and `u_matrix=None`
+- **WHEN** `from_sae_lens(...)` is called
+- **THEN** it SHALL raise `ValueError` naming the missing `u_matrix`
+
+#### Scenario: default profile needs no readout inputs
+
+- **GIVEN** no `profile` (or `profile="clustered"`) and no `u_matrix`
+- **WHEN** `from_sae_lens(...)` is called
+- **THEN** it SHALL succeed and produce a result byte-identical to the pre-change behaviour

openspec/changes/archive/2026-06-13-add-readout-aligned-geometry-profile/tasks.md

@@ -0,0 +1,47 @@
+# Implementation tasks
+
+## 0. Design pre-locks (blocking)
+
+- [ ] 0.1 Lock the readout subspace construction: `R = top-r right singular directions of (gain ⊙ U)`
+  (`U`: `(vocab, d_model)` unembed; `gain`: `(d_model,)` final-norm, default ones). `r` defaults to a
+  capped rank (e.g. `min(64, d_model)`); the geometry is built on `projections @ Rᵀ`. Confirm this matches
+  the R2 / sae-forge `_readout_aligned_order` construction.
+- [ ] 0.2 Lock that the projection happens **inside** `ReadoutAlignedKnobAssignment.assign(...)` so the
+  `from_sae_lens` call site (`sae_import.py:813`) and the `GeometricProfile`/registry plumbing are unchanged.
+- [ ] 0.3 Lock scope: knob assignment (β/γ) only; Q-Orca emission + `Dictionary.gram()` consume final knobs and
+  are transparent. No host-model load in the core.
+
+## 1. `polygram/geometry/readout_aligned.py` — the profile
+
+- [ ] 1.1 `ReadoutAlignedKnobAssignment` mirroring `ClusteredKnobAssignment` (`clustered.py:143`), carrying
+  `u_matrix` + `gain` (+ `readout_rank`); `.assign(projections, ...)` projects onto the readout subspace, then
+  runs the existing k-means / centroid / residual-variance / γ-PCA path on the projected vectors.
+- [ ] 1.2 `ReadoutAlignedFidelity` (implements the `GeometricFidelity` protocol) + `readout_aligned()` factory
+  (mirroring `clustered()` at `clustered.py:258`).
+- [ ] 1.3 Register `readout_aligned()` in `polygram/geometry/__init__.py` (+ `__all__`); ensure
+  `get_profile("readout-aligned")` resolves.
+
+## 2. `polygram/sae_import.py` — thread `u_matrix` / `gain`
+
+- [ ] 2.1 Add `u_matrix: np.ndarray | None = None`, `gain: np.ndarray | float | None = None` to `from_sae_lens`
+  (`:615`). When the resolved profile is `readout-aligned`, **require** `u_matrix` (2-D, second axis == d_model)
+  — raise a clear `ValueError` otherwise. Inject `u_matrix`/`gain` into the strategy before `.assign` (`:813`).
+- [ ] 2.2 Default profile unchanged (`clustered`); a run without `profile="readout-aligned"` is byte-identical.
+
+## 3. Tests
+
+- [ ] 3.1 Synthetic: a feature set whose raw-decoder clustering differs from its readout-aligned clustering
+  (construct `U` so the readout subspace re-orders the directions) → the two profiles assign different β/γ /
+  cluster labels; `readout-aligned` without `u_matrix` raises `ValueError`.
+- [ ] 3.2 `clustered` default path byte-identical (existing geometry tests green); `get_profile("readout-aligned")`
+  registered.
+- [ ] 3.3 Determinism: same `(projections, U, gain, seed)` → identical assignment.
+
+## 4. Behavioural gate (blocking — the decomposition IS the result)
+
+- [ ] 4.1 Extend `examples/behavioural_gram_scaleup.py` to run `clustered` vs `readout-aligned` head-to-head on
+  GPT-2-small (supply `U`/`gain` from the host the probe already loads); emit
+  `polygram_overlap_readout_aligned` into `docs/research/data/scaleup_pairs.csv`.
+- [ ] 4.2 Record the head-to-head Spearman table in `docs/research/behavioural-scaleup-probe.md` (in place, no
+  new page). **Descriptive verdict, pre-committed both ways:** WIN (Spearman ≥ 0.70) or NO-IMPROVEMENT
+  (basis isn't the limit → Reckoning #3 prune more likely). No necessity claims.