bench: publish 2026-07 refresh results - regenerated benchmarks page + committed artifacts

Full gated run (20 cells x 3 arms, zero flags, single environment
fingerprint) on the released diff-diff 3.7.0 wheel vs R 4.5.2
(did 2.5.1 / fixest 0.14.2 / synthdid 0.0.9), Apple M4 Max:

- CallawaySantAnna: 4.6x (small) -> 15.5x (10k) / 14.8x (20k) faster
  than R, advantage now GROWS with scale (previously published numbers
  showed it shrinking, 11x -> 4x); exact SE parity (0.0%) and all
  per-(g,t)/event-study/group surfaces at ~5e-11.
- SyntheticDiD: Rust backend 18-55x faster than R at matched
  200-replication placebo variance (the invalidated published table
  showed Rust SLOWER than pure Python); id-aligned unit/time weights
  reproduce R at <= 5e-13; SE gaps 3.1-11.5% = documented placebo Monte
  Carlo dispersion.
- Honest regressions published as-is: warmed-up fixest wins the sub-35ms
  BasicDiD/TWFE interaction-OLS cells at 10k-20k (0.6-0.9x); pure-Python
  SDID at equal bootstrap counts is 0.5-4.4x vs R (old 2.4-16.5x claim
  reflected a 50-vs-200 replication asymmetry).
- MPDTA real data: exact ATT match (-0.039951), ~5x faster.

docs/benchmarks.rst fully regenerated from the committed artifact
(gen_benchmark_tables.py --check clean) with prose reconciled: new
median/warm-up protocol wording, split BasicDiD vs absorbed-FE TWFE
tables + estimator-mapping row, rewritten Rust-backend and SDID notes,
honest Key Observations, corrected stale gelsy/14x claims, refresh
harness documented as the source of the published tables (legacy runner
labeled). llms.txt Rust-backend line updated to the measured range.

Internal optimization story committed (version_story.{json,md}),
CS-focused per direction: CallawaySantAnna ~4.9-5.0x faster on the 3.7.0
wheel than 3.5.3 at 20k units and 2M rows with identical estimates;
BasicDiD/MultiPeriodDiD context rows 1.15-1.31x.

CHANGELOG updated with the public before/after; delivered phase-2 TODO
row removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GPX5Rv8ozQXPdUV23QTfjr

Author: igerber

CHANGELOG.md

@@ -30,8 +30,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   mislabeled cell can never publish; the legacy `--type twfe` no-op on the
   BasicDiD scripts now fails loudly (absorbed-FE TWFE has its own dedicated
   script pair, and the refresh benchmarks it separately from BasicDiD).
-  Regenerated page numbers land together with the committed results
-  artifact once the timed refresh run completes.
+  The full gated run is committed (`results/refresh_results.json`) and
+  `docs/benchmarks.rst` is regenerated from it: CallawaySantAnna is
+  4.6-15.5x faster than R `did` with the advantage GROWING with scale
+  (15.5x at 10k units / 150k observations, 14.8x at 20k units / 360k
+  observations; the previously published numbers showed the advantage
+  shrinking); SyntheticDiD's Rust backend is
+  18-55x faster than R `synthdid` at matched 200-replication placebo
+  variance (the previously published table - captured before the current
+  Frank-Wolfe kernels and at unequal bootstrap counts - showed Rust
+  SLOWER than pure Python); MPDTA real-data validation is ~5x faster with
+  exact ATT match. Honest regressions from the fairness fixes are
+  published as-is: warmed-up fixest now beats the sub-35ms BasicDiD/TWFE
+  interaction-OLS cells at 10k-20k (0.6-0.9x), and pure-Python
+  SyntheticDiD at equal bootstrap counts is 0.5-4.4x vs R (the old
+  2.4-16.5x claim reflected a 50-vs-200 replication asymmetry). All
+  20 cells pass every publication gate with zero flags.
 
 ### Fixed
 - **`docs/benchmarks.rst` SDID weight-parity claim now machine-verified (and

TODO.md

@@ -45,13 +45,6 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 
 ### Testing / docs
 
-| Issue | Location | Origin | Effort | Priority |
-|-------|----------|--------|--------|----------|
-| Benchmark refresh phase 2 (same PR #672): run the gated timed refresh on an idle machine, commit `benchmarks/refresh_2026_07/results/refresh_results.json`, regenerate the marker-bounded regions of `docs/benchmarks.rst` via `gen_benchmark_tables.py`, and reconcile the remaining pre-refresh prose (protocol bullets "3 replications / mean ± std", combined BasicDiD/TWFE wording, SDID note under the perf table, Key Observations, "Reproducing Benchmarks" section, `llms.txt` speedup cross-references). Row removed by the phase-2 push itself. | `benchmarks/refresh_2026_07/`, `docs/benchmarks.rst` | #672 | Mid | Medium |
-
-| Issue | Location | Origin | Effort | Priority |
-|-------|----------|--------|--------|----------|
-
 ---
 
 ## Deferred / Documented