Add shapiq -> shap.Explanation bridge helper#292
Conversation
Introduces `shapiq_to_shap_explanation(explainer, X, *, budget, feature_names=None)` that runs a shapiq explainer over a batch of rows and wraps the first-order Shapley values in a `shap.Explanation` ready for the `shap.plots.*` family. Lives in `src/tabpfn_extensions/interpretability/shap.py` — that filename was freed up when #283 removed the legacy SHAP-based explainer, and the new role ("shap library bridge for plotting") maps naturally to it. Re-exported from `tabpfn_extensions.interpretability` so users can import it directly. `shap_example.py` collapses the previous 15-line "loop over rows / stack / average baseline / construct Explanation" boilerplate to a single helper call. `interpretability/README.md` updated: the paragraph that mentioned converting to `shap.Explanation` now names the helper explicitly and shows a one-liner, while keeping the "use shapiq's native plots" option visible as the alternative. `shap` is intentionally NOT added to the `interpretability` extra (same choice #283 made). The import lives inside the function body so the rest of the interpretability surface stays importable without shap installed. Verified end-to-end with a stub shapiq explainer in a clean venv that has shap + shapiq + tabpfn-extensions installed: values/base_values/data shapes correct, returned object is a real `shap.Explanation`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces a bridge helper function, shapiq_to_shap_explanation, to facilitate the use of the SHAP library's plotting ecosystem with Shapley values computed via shapiq. The changes include the implementation of the helper in a new shap.py module, updates to the interpretability documentation, and a refactor of the SHAP example script to utilize this new utility. Feedback was provided to improve the robustness of the helper by handling empty input cases and using per-row baseline values instead of a global average to better support diverse explainer configurations.
shap.Explanation accepts a 1-d (n,) array for base_values natively. Passing per-row baselines straight through is more correct than averaging: - For the imputation path (our default, get_tabpfn_imputation_explainer) baseline_value comes from the fixed background dataset and is identical across rows, so this matches the old behavior exactly. - For the Rundel remove-and-recontextualize path baselines genuinely vary per row; averaging would silently lose that signal. Also avoids the np.mean-on-empty RuntimeWarning Gemini flagged. Skipping Gemini's other suggestion (explicit raise on n==0) — np.stack's native ValueError is fine and adding the check is one more thing to keep. Verified end-to-end locally by running the actual shap_example.py against the v3 California-housing setup (n_explain=30, budget=2^8=256). All five plots (summary, scatter, bar, beeswarm, waterfall) rendered correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous commit accidentally staged the test-plot artifacts I'd generated under .local/shap_example_plots/ to verify the helper end-to-end. Those shouldn't be on the PR. Removing them from the index and re-adding .local/ to .gitignore (we'd un-ignored it during the v3 cleanup work and no longer have a use for that path being tracked). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
The previous commit on this branch re-added .local/ to .gitignore. That's unrelated to the shap-bridge-helper change and just here because I'd accidentally let pre-commit stage test-plot artifacts under .local/ in a prior commit on this branch. The untrack-from-index part of that commit is correct and stays — it cleans up the accidental staging. The .gitignore line itself is reverted; .local/ stays as it was on main. The local plot artifacts at .local/shap_example_plots/ have been deleted from my working tree (they were never on this PR after 1fab4d0). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
Adds a small bridge helper so users who want to compute Shapley values with
shapiq(faster and TabPFN-friendly) but plot with theshaplibrary (mature plotting ecosystem) don't have to write the 15-line "loop / stack / average baseline / construct Explanation" boilerplate themselves.What changed
New module —
src/tabpfn_extensions/interpretability/shap.py:Mirrors the pattern in
examples/interpretability/shap_example.py: one.explain(...)per row, stack the first-order arrays into(n, d), average baseline values, return ashap.Explanation. Theimport shapis inside the function body so the rest of the interpretability surface stays usable withoutshapinstalled.That filename was freed up when #283 removed the legacy SHAP-based explainer; reusing it for the bridge helper maps naturally (
shap.py= shap library integration,shapiq.py= shapiq library integration).Re-export —
interpretability/__init__.pynow exportsshapiq_to_shap_explanationso the import is short:Example shrunk —
examples/interpretability/shap_example.py: the previous 15-line boilerplate collapses to a singleshapiq_to_shap_explanation(...)call. The fiveshap.plots.*demonstrations are unchanged.README —
src/tabpfn_extensions/interpretability/README.md: the paragraph that mentioned "or convert the values toshap.Explanationand useshap.plots.*" now names the helper explicitly and shows the one-liner, with the "use shapiq's native plots" path still presented as the alternative.What this is not
shapto theinterpretabilityextra. Same decision [RES-1467] Drop SHAP, refresh shapiq examples around v3 KV cache #283 made —shapiqis the runtime dep,shapis opt-in for plotting. Documented in the helper's docstring.shap.Explanationdoesn't represent them; for those, users should keep using shapiq's native plots. Documented in the docstring.Verification
Smoke-tested end-to-end in a clean venv (Python 3.12,
shap+shapiq+tabpfn-extensionsinstalled from local source) with a stub explainer that returnsshapiq.InteractionValues. Checks:shapiq_to_shap_explanation(...)import pathshap.Explanationvalues.shape == (n, d)base_values.shape == (n,)feature_namespassed through🤖 Generated with Claude Code