test: anchor ImputationDiD leave-one-out SE against Stata did_imputation (2nd Stata arm, 1st SSC-dependent)

The Borusyak-Jaravel-Spiess (2024) Supplementary Appendix A.9 leave-one-out
finite-sample variance (opt-in `leave_one_out=True`) is a shipped, user-facing
option that no independent implementation had ever confirmed: R `didimputation`
omits LOO, so REGISTRY validated it only by an internal psi-identity + hand-calc
+ MC coverage. The authors' own Stata `did_imputation` ships the same option
(`leaveout`); this turns that into a measured anchor.

benchmarks/stata/generate_imputation_loo_golden.do runs
`did_imputation y unit time Ei, [horizons(0/5)] leaveout avgeffectsby(Ei t)
cluster(unit)` on the committed R-arm panel didimputation_test_panel.csv (no
clean-sample reconstruction; the only mapping is Ei = first_treat, missing for
never-treated). Agreement is cross-implementation (did_imputation goes through
reghdfe, the library through its own sparse IF solver): the LOO SE matches to
~1e-9 and the point to ~2e-8 across the overall ATT and all 6 event-study
horizons; the parity test gates at abs=1e-7. The non-LOO cluster SE is confirmed
three-way (library = R = Stata on the same panel).

First SSC-dependent arm (LPDiD used native teffects). The chain
did_imputation -> reghdfe -> require + ftools is not pinned by `version 19` (SSC
has no version history), so: the generator does NOT auto-install (that would
fetch latest every run, breaking byte-identical regeneration) - install once via
benchmarks/stata/requirements.do - and it records each package's version/date in
meta.ssc_versions (asserted to be a real version line, not a description) for
drift detection. avgeffectsby(Ei t) is pinned explicitly (== library
aux_partition="cohort_horizon", also did_imputation's current default) to keep
the estimand self-describing and robust to a future default change.

tests/test_imputation_loo_stata_parity.py (23 cases) gates per horizon + overall
at abs=1e-7, rel=0 (the repo's imputation SE-parity tolerance): point, LOO SE
anchor, non-LOO SE, non-LOO three-way vs R, and a no-A.9-singleton-warning gate
proving the LOO rescale is genuinely exercised.

No diff_diff/ source changed. Docs: REGISTRY LOO note (inferred -> measured),
borusyak review, benchmarks.rst (+ SSC prerequisites), benchmarks/README,
benchmarks/stata/README, doc-deps (register the Stata README under imputation.py),
CHANGELOG, TODO.md (coarser aux_partition follow-up). Stata is node-locked so -
exactly like the R arm - the golden is committed and the test skips when absent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9m4hRxwqRnEJsv5ZhHY3i

Author: igerber

.gitignore

@@ -122,6 +122,6 @@ benchmarks/refresh_2026_07/results/raw/
 benchmarks/refresh_2026_07/results/refresh_results_smoke.json
 
 # Stata batch mode writes <basename>.log to the cwd (repo root) - never committed;
-# the golden JSON it produces IS committed (benchmarks/data/lpdid_ra_stata_golden.json).
-# Root-anchored to the generator's own log so it doesn't hide logs elsewhere.
-/generate_lpdid_ra_golden.log
+# the golden JSON each generator produces IS committed (benchmarks/data/*.json).
+# Root-anchored to the generators' own logs so it doesn't hide logs elsewhere.
+/generate_*_golden.log

CHANGELOG.md

@@ -8,6 +8,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **ImputationDiD leave-one-out SE now anchored against Stata `did_imputation, leaveout`
+  (no library behavior change).** The Borusyak-Jaravel-Spiess (2024) App. A.9 LOO variance
+  (`leave_one_out=True`) has no runnable R reference (R `didimputation` omits LOO), so it
+  was validated only by an internal psi-identity + hand-calc + MC coverage. It is now
+  cross-validated against the authors' own Stata `did_imputation ..., leaveout` via a
+  committed golden (`benchmarks/stata/generate_imputation_loo_golden.do` →
+  `didimputation_loo_stata_golden.json`; `tests/test_imputation_loo_stata_parity.py`):
+  the library LOO SE matches to ~1e-9 (cross-implementation; the point to ~2e-8, gated at
+  `abs=1e-7`) at the overall ATT and all 6 event-study horizons, with the non-LOO SE
+  confirmed three-way (library = R = Stata).
+  Second Stata parity arm and the first SSC-dependent one — `benchmarks/stata/requirements.do`
+  documents the one-time install; the golden records `ssc_versions` for drift detection.
 - **LP-DiD regression-adjustment SE now anchored against Stata `teffects` (no library
   behavior change).** The RA influence-function cluster SE has no runnable R reference
   (`alexCardazzi/lpdid` does direct covariate inclusion, not RA), so it was previously

TODO.md

@@ -48,3 +48,4 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 | MMM interop PR-B: calibration tutorial notebook (fit DiD/CS -> scope -> `to_pymc_marketing_lift_test` / `to_meridian_roi_prior`) + a `llms-practitioner.txt` Step 8 pointer to the exporters as the MMM hand-off. | `docs/tutorials/`, `diff_diff/guides/llms-practitioner.txt` | mmm-interop | Mid | Low |
 | Tracking-file contract guard test: reject NEW active deferred-work pointers at `TODO.md` (deferred rows live in `DEFERRED.md`; allowlist for historical/past-tense prose and actionable-row pointers) and assert rows cross-linking a `docs/v4-deprecations.yaml` `M-xxx` id don't restate ledger status. Origin: tracking-split local review R2. | `tests/`, `TODO.md`, `DEFERRED.md` | tracking-split | Quick | Low |
 | Real-data CI canary for dataset-backed replication tests: `test_methodology_lwdid.py`'s Prop 99 / Walmart goldens skip (visibly) when loaders fall back to synthetic; add a lane or canary asserting `df.attrs["source"] == "lwdid_ssc_ancillary"` in CI so network regressions cannot silently de-gate the replication tests. Pairs with the loader-fallback repair row above. | `tests/test_methodology_lwdid.py`, `.github/workflows/` | LWDiD validation suite | Quick | Low |
+| Extend the ImputationDiD-LOO Stata anchor to the coarser `aux_partition ∈ {"cohort", "horizon"}` variants (Stata `did_imputation ..., leaveout avgeffectsby(Ei)` / `avgeffectsby(K)`, `K = t - Ei`). These differ from the default only at the *overall* aggregate (a no-op per-horizon) and are the least-validated surface — no R analogue, currently only hand-calc. Extends `benchmarks/stata/generate_imputation_loo_golden.do` + golden. | `benchmarks/stata/`, `tests/test_imputation_loo_stata_parity.py` | stata-arm | Mid | Low |

benchmarks/README.md

@@ -51,12 +51,18 @@ and CI never needs Stata. Generators live in `stata/` and are run headless:
 
 ```bash
 # macOS, StataSE 19 (binary is not on PATH by default)
-/Applications/Stata/StataSE.app/Contents/MacOS/stata-se -b do \
-    benchmarks/stata/generate_lpdid_ra_golden.do
+STATA=/Applications/Stata/StataSE.app/Contents/MacOS/stata-se
+$STATA -b do benchmarks/stata/requirements.do            # one-time SSC install
+$STATA -b do benchmarks/stata/generate_lpdid_ra_golden.do
+$STATA -b do benchmarks/stata/generate_imputation_loo_golden.do
 ```
 
-Current arm uses only **native** Stata commands (`teffects`) — no SSC packages —
-so `version 19` pins behavior. See `stata/README.md`.
+The `LPDiD` arm uses only **native** Stata commands (`teffects`), pinned by
+`version 19`. The `ImputationDiD` arm depends on SSC packages
+(`did_imputation`/`reghdfe`/`ftools`/`require`), which `version 19` does NOT pin
+(SSC has no version history) — install them once via `requirements.do` (the
+generators do not auto-install) and each golden records `ssc_versions` for drift
+detection. See `stata/README.md`.
 
 ## Directory Structure
 
@@ -72,8 +78,10 @@ benchmarks/
 │   ├── benchmark_honest.R    # HonestDiD
 │   └── benchmark_fixest.R    # Basic DiD / TWFE
 ├── stata/
-│   ├── README.md                       # Stata arm docs
-│   └── generate_lpdid_ra_golden.do     # LPDiD RA SE vs teffects ra
+│   ├── README.md                          # Stata arm docs
+│   ├── requirements.do                    # one-time SSC install (did_imputation etc.)
+│   ├── generate_lpdid_ra_golden.do        # LPDiD RA SE vs teffects ra
+│   └── generate_imputation_loo_golden.do  # ImputationDiD LOO SE vs did_imputation leaveout
 ├── python/
 │   ├── utils.py              # Common utilities
 │   ├── benchmark_callaway.py # CallawaySantAnna
@@ -96,6 +104,7 @@ benchmarks/
 | `SyntheticDiD` | `synthdid::synthdid_estimate` | Arkhangelsky et al. (2021) | ✓ Integrated |
 | `DifferenceInDifferences` | `fixest::feols` | Standard DiD | ✓ Integrated |
 | `LPDiD` (RA SE) | Stata `teffects ra ... atet` | Dube, Girardi, Jorda & Taylor (2025) | ✓ Integrated |
+| `ImputationDiD` (LOO SE) | Stata `did_imputation, leaveout` | Borusyak, Jaravel & Spiess (2024) App. A.9 | ✓ Integrated |
 | `HonestDiD` | `HonestDiD::createSensitivityResults` | Rambachan & Roth (2023) | Planned |
 
 Note: HonestDiD benchmark scripts exist but are not yet integrated into the main runner.

benchmarks/data/didimputation_loo_stata_golden.json

@@ -0,0 +1,28 @@
+{
+  "meta": {
+    "estimator": "ImputationDiD leave-one-out (BJS 2024 App. A.9) SE - Stata did_imputation leaveout",
+    "generator": "benchmarks/stata/generate_imputation_loo_golden.do",
+    "source_panel": "benchmarks/data/didimputation_test_panel.csv",
+    "point_anchor": "benchmarks/data/didimputation_golden.json (overall.att, event_study.att)",
+    "cmd": "did_imputation y unit time Ei, [horizons(0/5)] leaveout avgeffectsby(Ei t) cluster(unit)",
+    "avgeffectsby": "Ei t (== library aux_partition=cohort_horizon; pinned explicitly)",
+    "se_convention": "A.9 leave-one-out finite-sample variance; unit-clustered; se_nonloo is the non-leaveout cluster SE (== R didimputation).",
+    "ssc_versions": {
+      "did_imputation": "Version: November 22, 2023",
+      "reghdfe": "version 6.13.1 10Jan2026",
+      "ftools": "version 2.50.0 09jan2026",
+      "require": "version 1.3.1 19sep2023"
+    },
+    "stata_edition": "SE",
+    "stata_version": 19.0
+  },
+  "overall": {"att": 2.0456678967434003, "se": 0.021041161682015575, "se_nonloo": 0.020879383566722923, "N": 1440},
+  "event_study": {
+    "0": {"att": 0.99293658500530635, "se": 0.029085929067998187, "se_nonloo": 0.028791608348336495},
+    "1": {"att": 1.5129648623314707, "se": 0.02548745171356798, "se_nonloo": 0.025253856015380709},
+    "2": {"att": 2.006499293218484, "se": 0.030371068022456742, "se_nonloo": 0.030155388323006894},
+    "3": {"att": 2.4998545630646829, "se": 0.029299586939814812, "se_nonloo": 0.029048460851672311},
+    "4": {"att": 2.9648169454740363, "se": 0.043460391002896281, "se_nonloo": 0.043064691747540094},
+    "5": {"att": 3.4673514576204787, "se": 0.043504637171120686, "se_nonloo": 0.043073537575459654}
+  }
+}

benchmarks/stata/README.md

@@ -115,19 +115,89 @@ sample matches the library's realized size and cluster count — which, together
 the point (1e-10) and SE (~1e-16) agreement, strongly corroborates the same clean
 sample. No timestamp — the golden regenerates byte-identically.
 
+---
+
+# `did_imputation, leaveout` parity for ImputationDiD LOO SE
+
+`benchmarks/stata/generate_imputation_loo_golden.do` produces
+`benchmarks/data/didimputation_loo_stata_golden.json`, consumed by
+`tests/test_imputation_loo_stata_parity.py` to verify that
+`ImputationDiD(leave_one_out=True)` — the Borusyak-Jaravel-Spiess (2024) Supplementary
+Appendix A.9 finite-sample variance refinement — matches Stata `did_imputation ...,
+leaveout` at the overall ATT and all 6 event-study horizons.
+
+## Why Stata `did_imputation`
+
+The A.9 leave-one-out (LOO) variance has **no runnable R reference** — R
+`didimputation` omits LOO entirely — so the library LOO SE was validated only by an
+internal psi-identity + hand-calc + MC coverage. The authors' own Stata `did_imputation`
+(Borusyak) ships the same option (`leaveout`); this arm turns it into a measured anchor.
+
+## This is the first SSC-dependent arm
+
+Unlike the native-`teffects` LPDiD arm, `did_imputation` is an SSC package with a
+dependency chain `did_imputation → reghdfe → require + ftools`, none pinned by
+`version 19`. The generator does **not** install them — run
+`benchmarks/stata/requirements.do` once first — and it records each package's version
+in `meta.ssc_versions` (the `*!` ado header line) so drift is detectable. Byte-identical
+regeneration is therefore scoped to a fixed Stata + fixed installed SSC versions.
+
+## What the generator does
+
+No clean-sample reconstruction: `did_imputation` consumes the raw committed R-arm panel
+`didimputation_test_panel.csv` (180 units, cohorts {3,5} + never-treated); the only
+mapping is `Ei = first_treat` (missing for never-treated). The partition is pinned
+`avgeffectsby(Ei t)` **explicitly** (== library `aux_partition="cohort_horizon"`). This
+is also `did_imputation`'s current default, but pinning it keeps the validation estimand
+self-describing and robust to a future default change. The per-horizon command is:
+
+```stata
+did_imputation y unit time Ei, horizons(0/5) leaveout avgeffectsby(Ei t) cluster(unit)
+```
+
+Agreement is cross-implementation, not bit-identical (`did_imputation` goes through
+`reghdfe`, the library through its own sparse IF solver): the **SE** agrees to ~1e-9 and
+the **point** to ~2e-8; the parity test gates at `abs=1e-7`. Both the LOO and the non-LOO
+cluster SE are emitted; the non-LOO SE additionally three-way-confirms against the R
+golden (`didimputation_golden.json`).
+
+**Panel-coupling caveat:** `didimputation_test_panel.csv` is owned by the R generator
+(`benchmarks/R/generate_didimputation_golden.R`); if it is ever regenerated, the R golden
+AND this Stata golden must both be regenerated.
+
+## Regenerating
+
+```bash
+STATA=/Applications/Stata/StataSE.app/Contents/MacOS/stata-se
+$STATA -b do benchmarks/stata/requirements.do            # one-time SSC install
+$STATA -b do benchmarks/stata/generate_imputation_loo_golden.do
+grep -E '^r\([0-9]+\);' generate_imputation_loo_golden.log   # must print nothing
+```
+
+## JSON schema
+
+```json
+{
+  "meta": { "...": "...", "avgeffectsby": "Ei t (== cohort_horizon)",
+            "ssc_versions": {"did_imputation": "...", "reghdfe": "...", "...": "..."} },
+  "overall": {"att": <f>, "se": <LOO>, "se_nonloo": <f>, "N": <int>},
+  "event_study": { "0": {"att": <f>, "se": <LOO>, "se_nonloo": <f>}, "...": "..." }
+}
+```
+
 ## Known constraints
 
 - **Batch mode always exits 0**, even on a hard error (`r(NNN);`). Never trust the
-  shell exit code — parse the `.log` for `^r\([0-9]+\);` (the generator also runs
-  an in-`.do` point smoke gate at 1e-8 that surfaces as `r(9);` on a gross port
-  bug; the Python test's 1e-10 gate is authoritative).
+  shell exit code — parse the `.log` for `^r\([0-9]+\);`. Each generator also runs an
+  informational in-`.do` point smoke gate (LPDiD 1e-8, ImputationDiD 1e-6) that
+  surfaces as `r(9);` on a gross bug; the Python parity test is authoritative (LPDiD
+  point 1e-10; ImputationDiD `abs=1e-7`).
 - **`c(flavor)` misreports the edition** as `IC` on StataSE, and `c(edition)` is
   unreliable. The generator derives the edition from the `c(MP)` / `c(SE)` 0/1 flags,
   with BE by elimination (`c(BE)` is undefined and `cond()` evaluates all branches
   eagerly, so it must not be referenced). `"SE"` is simply the committed golden's
   current value.
 - **SSC has no version history.** `ssc install` always fetches latest and there is
-  no lockfile / archive to pin against. This arm is exempt because `teffects` is
-  native (pinned by `version 19`), but any *future* SSC-dependent Stata generator
-  must record `which <pkg>` output verbatim into its golden's `meta` so drift is
-  at least detectable.
+  no lockfile / archive to pin against. The LPDiD arm is exempt (`teffects` is native,
+  pinned by `version 19`); the ImputationDiD arm records its SSC package versions in
+  `meta.ssc_versions` so drift is at least detectable — new SSC arms should do the same.