feat(partition): per-block diagnostic floats + payoff measurement

[jascal]

Summary

Two bundled pieces, both directly extending Phase 2 (#109):

1. Per-block diagnostic floats — closes the Phase 2 v1 deferral. BlockReport.rank_ratio + post_A + informative_metric now populate end-to-end (forge_mse stays caller-provided like the top-level field).

2. Partition payoff measurement — A/B comparison of uniform vs partitioned compression on a synth SAE, validating the proposal's projected substrate-cost reduction.

Per-block diagnostics

Each block's diagnostics are scoped to its own sliced sub-W_dec + per-block local plan, then computed via the same _compute_rank_ratio / _compute_post_A / _informative_metric helpers the top-level uses. Empty-block edge case (no clusters → all cross-block dropped) leaves diagnostics as None.

Two new tests:

• test_block_report_diagnostic_floats_are_populated — sensible values (rank_ratio in [0,1], post_A finite, informative_metric in the expected literal set).
• test_block_report_diagnostics_none_for_empty_block — None for zero-cluster blocks.

Partition payoff measurement

	uniform	partition	reduction
substrate cost (kept reps)	2048 slots	368 slots	5.6×
substrate cost (full SAE)	4096 slots	736 slots	5.6×

Setup: 32-feature × 64-d_model synth SAE; 16 pairs (2-feature clusters).

Run 1 (uniform Rung5): every feature gets the 128-slot encoding (current pre-partition workflow).
Run 2 (partitioned): 4 heavy features keep Rung5 (128-slot); 28 tail features drop to MPSRung1 (8-slot).

The 5.6× falls within the proposal's 5-10× projected range. Exact factor depends on heavy:tail ratio + encoding choice — a more realistic 0.5-2% heavy on a production SAE gives 10-20× reduction.

Caveats (documented in commit + script):

• This is substrate cost (polygram Dictionary encoding budget), not forged-transformer parameter count. Forge-side savings depend on host-weight-projection through the basis; substrate reduction is necessary but not sufficient.
• Per-block rank_ratio/post_A on this synth setup are degenerate (synth SAE is intentionally low-rank-ish for the test budget). On a real SAE-Lens-trained SAE, these diagnostics carry the per-block reconstruction-quality signal.

The measurement validates the partition feature's projected payoff at synth scale. The next experiment (out of scope) is the same A/B at GPT-2 + jbloom SAE scale once sae-forge's add-block-structured-sae Phase 2 wires the --encoding-partition CLI flag.

Artefacts

• runs/partition_payoff_measurement.json (force-added) — the measurement output for the historical record.

Test plan

• 2 new per-block-diagnostic tests pass.
• Full polygram suite: 1079 → 1081 (no regressions).
• Ruff clean.
• Payoff measurement ran end-to-end on synth SAE; 5.6× reduction recorded.
• CI verifies on Linux runners.

🤖 Generated with Claude Code

[jascal]

Reality check: real-SAE A/B reveals the partition is polygram-only

Per request, ran the production-scale experiment with a real SAE (jbloom GPT-2 first 64 features), heuristic-driven partition (decoder L2 × pair count), real DecoderGeometryConfirmer ValidationReport, and a full sae-forge ForgePipeline.run_synthetic on both compressions.

Findings

metric	uniform Rung5	partition heavy/tail	reduction
substrate cost (kept reps)	512 slots	32 slots	16×
substrate cost (full SAE)	8192 slots	992 slots	8.26×
forge_faithfulness_kl	10.0484	10.0484	0% (identical!)

The partition reduces polygram-Dictionary substrate cost as projected, but does NOT lift forge faithfulness in the current sae-forge pipeline.

Mechanism

sae-forge reads W_dec from the polygram-compressed safetensors. The polygram encoding family choice doesn't affect W_dec. So the partition's encoding-family decisions are invisible to the forge as currently wired.

The proposal's projected forge_faithfulness_kl improvement (10-30%) was based on an assumption that encoding choice propagates through sae-forge — it doesn't, today.

What this PR is + isn't

This PR still ships the per-block diagnostic floats (Phase 2 v1 deferral) — those are mechanically correct and useful. What it does NOT validate is the proposal's forge-side payoff claim. That claim needs separate work in sae-forge (per-block encoding consumption in the forge pipeline).

Reframing the partition feature's value:

• ✅ Polygram-Dictionary substrate cost reduction (validated at 8-16×)
• ✅ Per-block diagnostic visibility for analysts (BlockReport surfaces metrics not otherwise available)
• ❌ Forge-side payoff (needs sae-forge wiring; out of scope for polygram-only changes)

Also revealed (practical issues for partition heuristics)

1. DecoderGeometryConfirmer is sparse on real SAE-Lens SAEs (0 pairs at threshold 0.5; 4 pairs at 0.1). SAE-Lens trains for orthogonality; geometry-based confirmation barely applies.
2. decoder_norm² is degenerate as a heaviness signal — SAE-Lens normalises decoder rows to unit norm. A real heuristic partition needs activation-side signals (firing rate, KL impact under ablation).

Artefacts

• runs/real_partition_experiment.json — full measurement output.
• runs/real_partition_experiment.py — the reproducible script.

Recommendation

Honest framing: this experiment is more valuable as a negative result than as validation of the proposal's projected payoff. The partition feature works at the polygram level but doesn't automatically improve forge quality. That requires deliberate sae-forge work that's not yet specified.