feat: 論文本番経路（W-Dist tune → 30ep×5seed → paper eval）と再現性基盤を公開正本として整備

.github/workflows/ruff.yml

@@ -27,14 +27,34 @@ jobs:
       - name: Ensure pytorch-topological (uv path dependency)
         run: ./scripts/ensure_pytorch_topological.sh
 
+      # pyproject path dependency; pin matches third_party/ellphi.ref.
+      - name: Ensure ellphi_repo (uv path dependency)
+        run: ./scripts/ensure_ellphi_repo.sh
+
       - name: Install uv and Python 3.12
         uses: astral-sh/setup-uv@v5
         with:
           python-version: "3.12"
           enable-cache: true
 
       - name: Install dependencies (all groups, incl. dev / ruff)
-        run: uv sync --all-groups --frozen
+        run: uv sync --all-groups --extra experiments --frozen
+
+      - name: Ellphi pin and grad API smoke
+        run: |
+          uv run python - <<'PY'
+          from pathlib import Path
+          from tda_ml.reproducibility import (
+              assert_ellphi_differentiable_available,
+              assert_ellphi_repo_matches_pin,
+              read_pinned_ellphi_revision,
+          )
+          pin = read_pinned_ellphi_revision(Path.cwd())
+          assert pin, "missing third_party/ellphi.ref"
+          assert_ellphi_repo_matches_pin(project_root=Path.cwd())
+          impl = assert_ellphi_differentiable_available(ellphi_differentiable=True)
+          print(f"ellphi ok pin={pin[:12]} impl={impl}")
+          PY
 
       - name: Ruff check (repo root; F-1 canonical command)
         run: uv run ruff check .

.gitignore

@@ -198,6 +198,7 @@ cython_debug/
 #  Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
 #  exclude from AI features like autocomplete and code analysis. Recommended for sensitive data
 #  refer to https://docs.cursor.com/context/ignore-files
+.cursor/
 .cursorignore
 .cursorindexingignore
 
@@ -216,6 +217,7 @@ docs/experiments/
 data/
 trash/
 ellphi_repo/
+pytorch-topological
 docs/JSAIM2026UT.zip
 
 # OS artifacts

README.md

@@ -1,119 +1,112 @@
-# TDA-ML Reproducibility Repository
+# TDA-ML
 
-This repository provides a reproducibility-focused implementation for anisotropic topological denoising experiments.
+Clean-room, reproducibility-focused implementation for anisotropic topological
+denoising (Letters / applied-math reference code).
 
-For **data layout, checkpoints, figure-related scripts, and CI tests**, see [`REPRODUCIBILITY.md`](REPRODUCIBILITY.md) (reader-facing).
+Reader-facing protocol, manifests, and failure semantics:
+[`REPRODUCIBILITY.md`](REPRODUCIBILITY.md).
 
-## Setup
-
-Use `uv` as the canonical dependency manager.
+Computational discipline follows
+[computational-reproducibility](https://github.com/t-uda/skills/blob/main/skills/computational-reproducibility/SKILL.md)
+(no silent fallback; declared numerical constants in `tda_ml/numerical_eps.py`).
 
-`torch_topological` is a **local path dependency** (`pyproject.toml` → `pytorch-topological/`). That directory is not always present after a plain `git clone`; sync it to the pinned commit in `third_party/pytorch_topological.ref` (same as CI):
+## Claim (main table)
 
-```bash
-./scripts/ensure_pytorch_topological.sh
-```
+Proposed method (W-Dist-tuned weights, 30 epochs × 5 data seeds) shows
+**comparable** outlier-removal performance to Euclidean DBSCAN and ADBSCAN on
+**MCC / G-Mean** (descriptive mean ± sample std over seeds; no equivalence test).
+Main table does **not** include a Topo W. column.
 
-Optional: if this repository records `pytorch-topological` as a git submodule gitlink, `git submodule update --init --recursive` may populate the tree first; the ensure script still verifies the pinned commit. See `third_party/README.md` when bumping the pin.
+ADBSCAN uses fixed local-PCA ellipses (no training). The proposed method trains
+for 30 epochs on the same data; the comparison is not compute-matched.
 
-Then install Python dependencies:
+## Setup
 
 ```bash
+./scripts/ensure_pytorch_topological.sh
+./scripts/ensure_ellphi_repo.sh
 uv sync
-```
-
-For development dependencies:
-
-```bash
+# development (tests, ruff):
 uv sync --all-groups
+# Optuna tune drivers:
+uv sync --extra experiments
 ```
 
-Optional package extras (e.g. `robustness_sweep`, HomCloud animation, explicit Pillow):
-
-```bash
-uv sync --extra experiments --extra repro-pd-animation --extra images
-```
+`torch_topological` and `ellphi` are **local path dependencies** (pinned under
+`third_party/*.ref`). A plain `git clone` is not enough; run the ensure scripts
+before `uv sync`. See `third_party/README.md` when bumping pins.
 
-If you use `pip` instead of `uv`, run `./scripts/ensure_pytorch_topological.sh` from the repository root, then install from `pyproject.toml` with `pip install .` or `pip install -e .` for an editable install; add optional extras when needed (for example `pip install -e ".[experiments,repro-pd-animation,images]"`). There is no `requirements.txt`; dependency pins live in `uv.lock` for `uv` users.
+## Paper production path (primary)
 
-## Minimal Reproduction
+Main table: **W-Dist-tuned weights**, 30 epochs × 5 data seeds, `w_class=0`,
+H1-only, local-PCA teacher, ellphi distance, checkpoint `best_model.pth`
+(`selection=val_topo`), then val DBSCAN grid → test MCC / G-Mean.
 
-Use the following command as the official reproduction entrypoint:
+Config: `paper_n100_o20_nocls_h1_ellphi_lpca_power`.
 
 ```bash
-uv run python experiments/run_backend_multiseed.py \
-  --base-config reproduce \
-  --epochs 50 \
-  --seeds 42 123 456 789 1024 \
-  --backends mahalanobis ellphi \
-  --out-base outputs/backend_compare
-```
-
-Run this command from the repository root.
-
-Expected artifacts:
-
-- `outputs/backend_compare/progress_summary.csv`
-- `outputs/backend_compare/backend_stats.csv`
-- `outputs/backend_compare/*/logs/metrics.csv`
-
-Direct execution of `tda_ml/main.py` is non-official and not part of the canonical reproduction path.
+# 1) Tune once (Optuna sampler seeds differ per worker; data seed in YAML is 42).
+#    Default MODE=both also runs the secondary MCC study; main table needs wdist.
+MODE=wdist bash experiments/tune_objectives.sh
 
-### Resume Behavior
+# 2) Fixed W-Dist weights → 30ep × 5 seeds → paper eval (+ baselines separately)
+bash experiments/run_paper_30ep_multiseed.sh wdist
 
-- `run_backend_multiseed.py` skips completed runs using the key `(backend, seed, epochs)`.
-- Changing `--epochs` creates a different run key and will not be skipped.
-- Use `--rerun-completed` to force rerun even when the same key already exists.
-- `progress_summary.csv` header is validated on startup. If header mismatch occurs, use a fresh `--out-base` or fix the CSV manually.
-
-### Operational Notes
+uv run python experiments/eval_baselines.py \
+  --base-config paper_n100_o20_nocls_h1_ellphi_lpca_power \
+  --out-dir outputs/paper_baselines
+```
 
-- `run_backend_multiseed.py` creates a lock file under `--out-base` and aborts if another process is active there.
-- `progress_summary.csv` stores `run_dir` as provided by `--out-base`; avoid sharing logs with personal absolute paths if privacy is a concern.
-- Keep `metrics.csv` schema compatible with the current trainer output (`val_mcc`, `val_recall`, `val_loss` are required).
+Details and contract keys: `REPRODUCIBILITY.md` / `configs/README.md`.
+Generated artifacts stay under `outputs/` (not committed).
 
-## Quick Smoke Run
+## Backend pipeline comparison (secondary / CI)
 
-For a fast wiring check before the full run:
+`experiments/run_backend_multiseed.py` compares **full training pipelines** under
+`configs/reproduce.yaml`. It is **not** the paper main-table entrypoint and
+**not** a pure distance-backend ablation.
 
 ```bash
+# CI-style smoke
 uv run python experiments/run_backend_multiseed.py \
   --base-config reproduce \
-  --epochs 1 \
-  --seeds 42 \
-  --backends mahalanobis \
+  --epochs 1 --seeds 42 --backends mahalanobis \
   --out-base outputs/smoke
-```
 
-## Continuous integration
-
-On pull requests and pushes to `main` / `feature/**`, CI runs:
+# Full secondary comparison (local / own runner; not CI)
+uv run python experiments/run_backend_multiseed.py \
+  --base-config reproduce \
+  --epochs 50 --seeds 42 123 456 789 1024 \
+  --backends mahalanobis ellphi \
+  --out-base outputs/backend_compare
+```
 
-- `ruff check`
-- `pytest`
-- A **repro smoke** only: `run_backend_multiseed.py` with `--epochs 1`, one seed, and `mahalanobis` (see `.github/workflows/ruff.yml`).
+Expected under `--out-base`: `progress_summary.csv`, `backend_stats.csv`, and
+per-run `*/logs/metrics.csv`. Resume skips completed `(backend, seed, epochs)`
+keys; use `--rerun-completed` to force. A lock file under `--out-base` aborts if
+another process is active.
 
-The full official command (50 epochs × five seeds × two backends) is **not** executed in CI. Run that locally or in your own runner when validating paper numbers.
+`tda_ml.main` is an internal trainer entry used by the drivers above; do not
+treat direct invocation as the public protocol.
 
-## Known Constraints
+## Continuous integration
 
-- The official comparison uses a fixed five-seed set: `42 123 456 789 1024`.
-- Runtime and numeric behavior can vary by `device`, `thread`, and `dtype`; effective values should be logged per run.
-- External implementations are reference-only and are not redistributed in this repository.
-- Scripts under `tda_ml/experiments/` are **non-official** and require your own checkpoints (see `configs/README.md`).
-- `run_backend_multiseed.py` multiseed backend comparison is **not** a pure distance-backend-only ablation; it compares full training pipelines, not an isolated backend switch.
-- For topological loss, **`mahalanobis`** can incorporate predicted **outlier-probability weighting** in the distance matrix; **`ellphi`** does not (geometry-only distances; `probs` ignored).
-- Read outcomes as **two full pipelines** under the same schedule and config surface, not as isolating distance-backend effects alone.
+PRs and pushes to `main` / `feature/**` run `ruff`, tests, and the **1-epoch
+mahalanobis smoke** above. Paper 30ep × 5-seed production is not run in CI.
 
-### Backend comparison: outlier-probability weighting
+## Known constraints
 
-For **topological loss**, the batched distance matrix is built per `model.topology_loss.distance_backend`. With **`mahalanobis`**, the implementation can incorporate **predicted outlier probabilities** when forming pairwise distances (see `tda_ml.topology.compute_anisotropic_distance_matrix`). With **`ellphi`**, tangency distances are computed from ellipse geometry only; **probability-based weighting is not implemented** and `probs` are ignored (a warning is emitted once per process; see `tda_ml.distance_backend.compute_distance_matrix_batch`). Hyperparameters in `configs/reproduce.yaml` are shared across backends, but **the induced metric for topological loss is not identical across backends** by design: the intended read is a reproducible pipeline comparison (same data, schedule, and config surface), not a claim that both backends optimize the exact same weighted distance objective. Elliptic contact distances are left as defined by `ellphi`; no synthetic “prob-equivalent” weighting is applied on the ellphi path.
+- Fixed paper seed set: `42 123 456 789 1024`.
+- Runtime depends on device / threads / dtype; each run records a manifest.
+- For topological loss, **mahalanobis** may use outlier-probability weighting;
+  **ellphi** is geometry-only and requires `prob_weighting=false` (hard-fail
+  otherwise). Do not read backend comparison as an isolated metric swap.
 
 ## License / Attribution
 
-This project is licensed under the MIT License. See `LICENSE`.
-
-External implementations (for example `ellphi_repo` and `pytorch-topological`) are reference-only and are not redistributed here.
+MIT — see `LICENSE`.
 
-- `https://github.com/t-uda/ellphi`
-- `https://github.com/aidos-lab/pytorch-topological`
+- **ellphi (differentiable tangency):** pinned fork via
+  `./scripts/ensure_ellphi_repo.sh` (`third_party/ellphi.ref`). PyPI
+  `ellphi==0.1.2` alone lacks the training `ellphi.grad` API.
+- **pytorch-topological:** via `./scripts/ensure_pytorch_topological.sh`.