Reproduce FEGA

NOTICE.txt

@@ -9,10 +9,9 @@ file for the full license text.
 -------------------------------------------------------------------------------
 Third party legal information
 
-Murano does not bundle or redistribute third-party source code. It depends on
-the following open-source projects, which are installed separately (via pip) and
-remain under their own licenses. Refer to each project's own distribution for
-the full license text:
+Murano also depends on the following open-source projects, which are installed
+separately (via pip) and remain under their own licenses. Refer to each
+project's own distribution for the full license text:
 
   - nnsight                     (https://github.com/ndif-team/nnsight)
   - nnterp                      (https://github.com/Butanium/nnterp)
@@ -26,4 +25,3 @@ the full license text:
   - seaborn                     (https://github.com/mwaskom/seaborn)
   - plotly                      (https://github.com/plotly/plotly.py)
   - sae-lens                    (https://github.com/jbloomAus/SAELens)
-

docs/src/content/docs/docs/reproductions/submit.mdx

@@ -37,14 +37,18 @@ We do not expect bit-exact reproductions. We do expect intellectual honesty: if
 
    Use a short slug that identifies the paper, e.g. `repro/todd-2024-function-vectors` or `repro/tmlr2026-shape-happens`.
 
-2. **Add your notebook**
+2. **Add your notebook or bundle**
 
-   Drop your `.ipynb` into `notebooks/reproductions/` with a descriptive filename:
+   Add a single `.ipynb` to `notebooks/reproductions/`, or use a nested directory
+   when the reproduction needs local method code, artifacts, or license files:
 
    <FileTree>
    - notebooks/
      - reproductions/
        - **your_paper_slug.ipynb**
+       - your_paper_slug/
+         - **README.md**
+         - **your_paper_slug.ipynb**
    </FileTree>
 
    The notebook should:
@@ -53,6 +57,9 @@ We do not expect bit-exact reproductions. We do expect intellectual honesty: if
    - Run the full experiment end-to-end without manual intervention
    - Print or plot the key results at the end
 
+   A nested bundle should include a `README.md` that states its contents,
+   dependencies, scope, provenance, and any local license information.
+
 3. **Write the gallery entry**
 
    Create a new Astro page at `docs/src/pages/reproductions/<your-slug>.astro`. The easiest way is to copy an existing one:

docs/src/pages/reproductions/feature_effect_geometry.mdx

@@ -0,0 +1,201 @@
+---
+layout: ../../layouts/ReproLayout.astro
+title: "Sparse Autoencoders Encode Both Concepts and Functions"
+description: "Reproduce Feature-Effect Geometry Analysis with Murano: remove one SAE feature across prompts, recover all eight geometry families from original FEGA examples, and compare the numerical and visual results."
+---
+
+<div class="mc-header">
+  <a href="/murano/reproductions" class="mc-back">← Back to gallery</a>
+
+  <div class="mc-badges">
+    <span class="badge badge--venue">arXiv 2026</span>
+    <span class="badge badge--tag">feature effects</span>
+    <span class="badge badge--tag">SAE</span>
+    <span class="badge badge--tag">geometry</span>
+  </div>
+
+  <h1 class="mc-title">
+    Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects
+  </h1>
+
+  <p class="mc-authors">
+    Phu Gia Hoang, Anwoy Chatterjee, Tanmoy Chakraborty, Iryna Gurevych, Subhabrata Dutta
+  </p>
+
+  <div class="mc-actions">
+    <a href="https://github.com/UKPLab/murano/blob/main/notebooks/reproductions/fega/hoang2026_feature_effect_geometry.ipynb" class="btn btn--primary" target="_blank" rel="noopener">
+      <svg xmlns="http://www.w3.org/2000/svg" width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round"><polygon points="5 3 19 12 5 21 5 3"/></svg>
+      View executed notebook
+    </a>
+    <a href="https://arxiv.org/abs/2607.24645" class="btn btn--outline" target="_blank" rel="noopener">arXiv</a>
+    <a href="https://ukplab.github.io/FEGA/" class="btn btn--outline" target="_blank" rel="noopener">Project page</a>
+    <a href="https://github.com/UKPLab/FEGA" class="btn btn--outline" target="_blank" rel="noopener">Original code</a>
+  </div>
+</div>
+
+<hr class="divider" />
+
+<article class="mc-body">
+
+## TL;DR
+
+An SAE feature can activate on a recognizable concept without having one consistent effect on the model. **Feature-Effect Geometry Analysis (FEGA)** tests this by removing the same feature from many prompts and comparing how the model's vocabulary scores change. If those changes point in one direction, the feature behaves consistently; if they split into several directions or spread broadly, its effect depends on context. This reproduction uses Murano to collect effects for one Gemma 2 feature, then recovers all eight FEGA geometry families from examples produced by the original experiments. The numerical results match the original outputs, and the page compares newly generated 3D sphere and 2D projection plots with the original figures.
+
+## Abstract
+
+> The wide-scale use of sparse autoencoders (SAEs) as interpretability tools is limited by inconsistent links between SAE features and model behavior. Features with clear activation descriptions may have weak or unexpected causal effects; steering can vary across prompts or oppose the intended direction; and activation-based feature selection can miss features that produce the desired output change. Prior work has studied feature geometry inside the model, where features are computed. We instead study the geometry of changes in model logits caused by feature interventions. We introduce Feature-Effect Geometry Analysis (FEGA), an unsupervised framework that removes the same active SAE feature across contexts and analyzes the resulting cloud of logit changes. Across SAE variants, consistent one-dimensional effects are rare: few features behave like reusable directions. To interpret this variation, we distinguish value-like features, tied to static information such as factual attributes, from pointer-like features, associated with context-dependent operations. Value-like features more often exhibit structured, low-dimensional effects, although these effects typically span several directions. Pointer-like features, by contrast, predominantly exhibit diffuse effects. Our results show that a feature can be interpretable and causally relevant without providing a stable direction for steering.
+
+<hr class="divider" />
+
+## Reproducing with Murano
+
+For one prompt, FEGA runs the model twice at a selected token. The baseline uses the complete SAE reconstruction. The intervention uses the same reconstruction after setting one feature activation to zero. Their difference describes which vocabulary logits rise or fall when that feature is removed. Repeating the intervention across prompts produces a matrix whose rows are feature effects; FEGA studies the shape of those rows.
+
+This reproduction uses **Gemma 2 2B**, a width-65k SAE trained on the residual stream after layer 12, and feature `33760`. The live example contains 64 prompts in which that feature is active.
+
+### Step 1: Load the model and its SAE
+
+The SAE must be attached to the layer on which it was trained. Murano loads Gemma and exposes that layer and the model's unembedding matrix, which maps final activations to vocabulary scores. The SAE wrapper supplies the encoder and decoder needed to construct both reconstructions.
+
+```python
+model = MuranoModel(
+    "google/gemma-2-2b",
+    dtype=torch.bfloat16,
+    attn_implementation="eager",
+)
+
+sae = DictionarySAEModel(
+    SAE_REPO,
+    SAE_ID,
+    MODEL_ID,
+    device=str(model.unembed_weight.device),
+    dtype=torch.bfloat16,
+)
+```
+
+Run the notebook from a cloned Murano repository. The live example requires a CUDA GPU and access to Hugging Face on the first run to download the Gemma and SAE checkpoints.
+
+### Step 2: Remove one feature across 64 prompts
+
+Each prompt includes the token position at which feature `33760` should be removed. `FEGADataPrep` encodes that activation and selects the feature; `FEGAComputeEffect` runs the complete and feature-removed reconstructions and records their difference. Keeping the other SAE activations fixed isolates the effect of this one feature.
+
+```python
+results = Results()
+results[murano_keys.PROMPTS] = prompt_batch
+
+FEGADataPrep(
+    model,
+    SAE_REPO,
+    SAE_ID,
+    [33760],
+    position=logical_targets,
+    batch_size=8,
+    sae_model=sae,
+    preselected_context_indices={33760: tuple(range(64))},
+)(results)
+
+FEGAComputeEffect(model, batch_size=8, sae_model=sae)(results)
+effects = results[fega_keys.EFFECTS]
+```
+
+All 64 prompts produce a positive activation and a finite effect. On this prompt set, most effects point in one direction, so feature `33760` receives the `directed_ray` label. Its stability check is unsuccessful, however, so this example demonstrates the collection procedure rather than claiming that the feature has the same geometry on every prompt population.
+
+### Step 3: Recover all eight geometry families
+
+The paper searched many models, SAEs, and features. Repeating that entire search would make a repository example unnecessarily expensive, so the reproduction includes eight small effect matrices exported from the original FEGA runs—one example for each geometry family. Each row is the effect of removing one feature from one prompt.
+
+The same analysis is applied to every matrix. It measures directional alignment and dimensionality, fits a von Mises-Fisher mixture when several recurring directions may be present, checks the stability of the result, assigns a geometry family, and draws a 3D sphere and 2D projection.
+
+```python
+effect_store = build_effect_store_from_raw_cloud(
+    delta_rows,
+    feature_inputs,
+    effects.gram,
+)
+
+results = Results()
+results[fega_keys.EFFECTS] = effect_store
+config = FEGAConfig()
+
+for step in (
+    FEGAGeometryMetrics(config),
+    FEGAVMF(model, config, n_jobs=1),
+    FEGAStability(config),
+    FEGAGeometryReporting(),
+    FEGAVisualize(
+        output_dir,
+        top_k_per_family=1,
+        figures=("sphere_surface", "projection_2d"),
+    ),
+):
+    step(results)
+```
+
+### Step 4: Compare the results with the original implementation
+
+After the eight examples have been analyzed, the reproduction loads the corresponding outputs from the original FEGA implementation. It compares the final label, representative geometry measurements, directional-mixture result, and stability decision.
+
+The visual comparison uses the distances between plotted points rather than raw pixels. A 2D or 3D projection can rotate or reflect while preserving exactly the same structure; pairwise distances and directional-group membership capture what the figure means.
+
+```python
+assert reporting.primary_label == expected["expected_family"]
+assert reporting.selected_k == expected["reporting"]["selected_k"]
+
+np.testing.assert_allclose(
+    pairwise_squared_distances(new_coordinates),
+    pairwise_squared_distances(original_coordinates),
+    rtol=1e-6,
+    atol=1e-6,
+)
+```
+
+<p class="aside">
+  Feature 33760 appears twice for different reasons. Its live run shows how Murano collects effects from prompts. Its saved effect matrix, produced by the original FEGA run, tests whether the geometry analysis recovers the original result.
+</p>
+
+<hr class="divider" />
+
+## Key results
+
+| Feature | Original FEGA family | This reproduction | Selected k |
+| --- | --- | --- | --- |
+| `33760` | `directed_ray` | `directed_ray` | — |
+| `14513` | `axis_or_antipodal` | `axis_or_antipodal` | — |
+| `54361` | `global_2D_directional_subspace` | `global_2D_directional_subspace` | `2` |
+| `32542` | `global_kD_directional_subspace` | `global_kD_directional_subspace` | `8` |
+| `59154` | `oneD_diffuse` | `oneD_diffuse` | — |
+| `19224` | `residual_lowD_k` | `residual_lowD_k` | `4` |
+| `34636` | `unresolved_high_dimensional_or_diffuse` | `unresolved_high_dimensional_or_diffuse` | — |
+| `1425` (Matryoshka) | `multi_mode_directional_geometry` | `multi_mode_directional_geometry` | — |
+
+All eight examples recover the original label, representative geometry measurements, directional-mixture result, and stability decision. The 16 newly generated figures—one sphere and one 2D projection per feature—also preserve the point and group structure of the original FEGA figures.
+
+The live run of feature `33760` retains all 64 prompts and reports `directed_ray`, with no selected `k` and an unstable stability result.
+
+<p class="aside">
+  The live example uses batches of eight. Changing the batch grouping can slightly change reduced-precision model activations and may change the resulting geometry, so batch size eight should be kept when repeating the displayed live result.
+</p>
+
+<p class="aside">
+  This reproduction reruns one model intervention and the complete geometry analysis for eight original examples. It does not repeat the paper's dataset construction, model-wide feature search, full model and SAE sweeps, aggregate prevalence results, or complete feature atlas.
+</p>
+
+The [reproduction README](https://github.com/UKPLab/murano/blob/main/notebooks/reproductions/fega/README.md) lists the runtime requirements, included files, source information, and licenses.
+
+<hr class="divider" />
+
+## Citation
+
+```bibtex
+@misc{hoang2026sparseautoencodersencodeconcepts,
+  title         = {Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects},
+  author        = {Phu Gia Hoang and Anwoy Chatterjee and Tanmoy Chakraborty and Iryna Gurevych and Subhabrata Dutta},
+  year          = {2026},
+  eprint        = {2607.24645},
+  archivePrefix = {arXiv},
+  primaryClass  = {cs.LG},
+  url           = {https://arxiv.org/abs/2607.24645},
+}
+```
+
+</article>

docs/src/pages/reproductions/index.astro

@@ -45,16 +45,27 @@ const reproductions = [
     paper: 'https://arxiv.org/abs/2510.01025',
     code: 'https://github.com/UKPLab/supervised-multidimensional-scaling',
   },
+  {
+    slug: 'feature_effect_geometry',
+    title: 'Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects',
+    venue: 'arXiv 2026',
+    authors: 'Hoang, Chatterjee, Chakraborty, Gurevych, Dutta',
+    description:
+      'Reruns FEGA downstream analysis for eight historical raw effect clouds, compares 16 fresh/source views, and reports one fixed-batch live illustration.',
+    tags: ['feature effects', 'geometry', 'SAE'],
+    paper: 'https://arxiv.org/abs/2607.24645',
+    code: 'https://github.com/UKPLab/FEGA',
+  },
 ];
 ---
 
-<ReproLayout title="Reproduction Gallery" description="End-to-end reproductions of published mechanistic interpretability results using Murano.">
+<ReproLayout title="Reproduction Gallery" description="Executable Murano studies of published mechanistic interpretability work.">
   <div class="gallery">
     <div class="gallery-header">
       <h1 class="gallery-title">Reproduction Gallery</h1>
       <p class="gallery-subtitle">
-        End-to-end reproductions of published interpretability results, implemented with Murano pipelines.
-        Each entry includes a runnable notebook, paper reference, and walkthrough.
+        Executable Murano studies of published interpretability work.
+        Each entry states its scope and includes a notebook, paper reference, and walkthrough.
       </p>
     </div>
 

notebooks/README.md

@@ -101,3 +101,5 @@ application notebooks.
   — the linear structure of truth in Llama-2-13B.
 - [`reproductions/todd2024_function_vectors.ipynb`](reproductions/todd2024_function_vectors.ipynb)
   — function vectors in GPT-J-6B.
+- [`reproductions/hoang2026_feature_effect_geometry.ipynb`](reproductions/fega/hoang2026_feature_effect_geometry.ipynb) 
+  — Feature Effect Geometry Analysis of Gemma-2-2B SAE features.