feat(v4): complete event-study df provenance - SunAbraham/dCDH event_study_df, LPDiD pooled_df (M-092 follow-up)

Author: igerber

CHANGELOG.md

@@ -28,6 +28,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   metacharacter — so no path composed into a shell command can execute. `persist`
   self-cleans the whole invocation on any failure (no post-persist cleanup
   needed) and `abort` is strict (a missing state token is an error).
+- **df provenance completed for every event-study producer that exposes one
+  (4.0 program Phase 2, ledger row M-092 follow-up).** `SunAbrahamResults` gains
+  `event_study_df` (a per-relative-time dict: the per-event Bell-McCaffrey contrast df
+  under `hc2_bm`, the survey design df - post-drop under replicate refits - on survey
+  fits, NaN where inference is normal-theory or its BM DOF was non-finite), and
+  `ChaisemartinDHaultfoeuilleResults` gains a scalar `event_study_df` (one value, since
+  its effect and placebo rows are computed from the same design df and refreshed
+  together under replicate weights). Both clear under bootstrap, whose percentile
+  p-values never used a df; SunAbraham's clear is deliberately narrower than its
+  `event_study_vcov` clear, since a replicate refit's rows genuinely used the replicate
+  df. `LPDiDResults` gains `pooled_df` (`{"pre": ..., "post": ...}`), the headline
+  pooled-window analogue of `DiDResults.inference_df`, surfaced in `to_dict()`; the
+  native pooled frame schema is unchanged. With these, `EventStudyResults.df` is
+  populated for every producer whose inference records a df.
 - **Docs: navigation and SEO polish.** A stable/latest version switcher in the navbar
   (`docs/_static/switcher.json`); a custom `robots.txt` that keeps the ~60 thin
   `_modules/` source-view pages out of crawlers (documentation pages allowed as
@@ -123,6 +137,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   **Estimator equations, weighting, variance, and numerical output are unchanged.**
 
 ### Changed
+- **dCDH Phase 1 (`L_max=None`) event-study row is now re-synced by the final-df
+  refresh.** Under replicate-weight designs the refresh recomputes `overall_*` with the
+  final effective df, but the Phase 1 event-study row is a VALUE COPY of that inference
+  taken beforehand, and the refresh's per-horizon loops cover only the Phase 2 surface -
+  so the copy could in principle retain an intermediate df's t/p/CI while
+  `survey_metadata.df_survey` reported the final one. The copy is now mirrored from the
+  refreshed values, making the stored row agree with the overall estimate it duplicates
+  by construction. No output change is reproducible on any configuration exercised in
+  the test suite (the contributing sites agree on their valid-replicate counts, so the
+  intermediate and final df coincide and the mirror is a no-op); this closes the
+  structural hazard rather than a demonstrated defect.
 - **DiagnosticReport's parallel-trends check upgrades from the Bonferroni fallback to
   the joint Wald test for StackedDiD and TwoStageDiD event-study fits.** The check's
   joint-Wald path runs whenever the source fit exposes `event_study_vcov`; now that

TODO.md

@@ -23,7 +23,6 @@ Related tracking surfaces:
 |-------|----------|--------|--------|----------|
 | `SyntheticControl` conformal (CWZ 2021) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies — the residual-permutation shortcut is only valid for time-permutation-invariant proxies (SC/Lasso/DiD); an AR proxy needs innovation permutation. | `diff_diff/conformal.py`, `diff_diff/synthetic_control_results.py` | CWZ-2021 | Heavy | Low |
 | `ContinuousDiD` CGBS-2024 remaining extensions (earlier phases — `covariates=` reg/dr, `treatment_type="discrete"`, single-cohort `control_group="lowest_dose"` with estimand `ATT(d)−ATT(d_L)` — are already supported; see REGISTRY Note #7). Remaining (all deferred `NotImplementedError`, documented): `estimation_method="ipw"` on the dose curve (scalar-adjustment / degenerate); `covariates=` × `survey_design=` (weighted OR + weighted nuisance IF); multi-cohort **heterogeneous-support** discrete aggregation (support-aware: average each dose only over the cohorts that observe it); **multi-cohort `lowest_dose`** (within-cohort `d_L` reference + support-aware cross-cohort aggregation); and **`covariates=` × `lowest_dose`** (conditional-PT-relative-to-`d_L` estimand). Single-cohort / 2-period / shared-support multi-cohort are supported. | `continuous_did.py` | CGBS-2024 | Heavy | Low |
-| `event_study_df` provenance for the remaining producers (4.0 program Phase 2 follow-up; the Stacked/TwoStage vcov + CS/MPD/Stacked/TwoStage/LPDiD df round-trip landed with the per-row `EventStudyResults.df` schema). Remaining: SunAbraham (per-event BM Satterthwaite df under `hc2_bm` at `sun_abraham.py:1206`, scalar survey df at `:1256` - mechanism identical to StackedDiD's recorded-at-use dict); dCDH (its event-study inference receives a `df_inference` argument, `chaisemartin_dhaultfoeuille.py:6311`, never stored - needs tracing to the stored rows); LPDiD pooled-window df (popped and discarded at fit assembly). All purely additive: their surfaces show df=NaN ("unexposed") until threaded. | `diff_diff/sun_abraham.py`, `diff_diff/chaisemartin_dhaultfoeuille.py`, `diff_diff/lpdid.py` | #M-092 | Quick | Low |
 
 ### Performance
 

diff_diff/chaisemartin_dhaultfoeuille.py

@@ -4039,8 +4039,11 @@ def fit(
         # and HonestDiD. Re-run safe_inference with the FINAL
         # effective df so every surface agrees.
         _final_eff_df = _effective_df_survey(resolved_survey, _replicate_n_valid_list)
+        # ONE canonical final inference df, shared by the refresh below and
+        # by the `event_study_df` provenance recorded at construction, so the
+        # two can never drift apart.
+        _final_inf_df = _inference_df(_final_eff_df, resolved_survey)
         if _replicate_n_valid_list:
-            _final_inf_df = _inference_df(_final_eff_df, resolved_survey)
             # Recompute `effective_overall_*` directly — that's what
             # ships in results at line ~2776+. `effective_overall_att/se`
             # may differ from the raw `overall_att/se` under the delta
@@ -4087,6 +4090,22 @@ def fit(
                     _info_r2["t_stat"] = _t_r2
                     _info_r2["p_value"] = _p_r2
                     _info_r2["conf_int"] = _ci_r2
+            else:
+                # Phase 1 (L_max=None): `event_study_effects[1]` is a VALUE
+                # COPY of the overall inference, built above from the
+                # PRE-refresh `overall_*`; the loop just above covers only
+                # the Phase 2 `multi_horizon_inference` surface. Without
+                # this mirror the stored l=1 row keeps the INTERMEDIATE
+                # df's t/p/CI while `overall_*` and
+                # `survey_metadata.df_survey` report the final (tightened)
+                # df - the same value-copy hazard the placebo mirror below
+                # documents. Re-sync from the refreshed `overall_*` so the
+                # single event-study row agrees with the overall estimate it
+                # is a copy of.
+                if event_study_effects and 1 in event_study_effects:
+                    event_study_effects[1]["t_stat"] = overall_t
+                    event_study_effects[1]["p_value"] = overall_p
+                    event_study_effects[1]["conf_int"] = overall_ci
             if placebo_horizon_inference is not None:
                 for _lag_r2, _info_r2 in list(placebo_horizon_inference.items()):
                     _t_r2, _p_r2, _ci_r2 = safe_inference(
@@ -4181,6 +4200,24 @@ def fit(
         if survey_metadata is not None:
             survey_metadata.df_survey = _final_eff_df
 
+        # df PROVENANCE (spec section 5, row M-092): the scalar df every
+        # stored event-study / placebo row's safe_inference received.
+        # Reuses `_final_inf_df` - the SAME value the refresh above applied
+        # to every stored row - so provenance and inference cannot drift.
+        # Recorded iff finite and > 0.
+        # Cleared under bootstrap, whose percentile p/CIs never used a df:
+        # required, not defensive, because TSL survey + bootstrap is a live
+        # mode (only REPLICATE + bootstrap is rejected) where this
+        # expression evaluates finite.
+        _es_df_provenance: Optional[float] = None
+        if (
+            bootstrap_results is None
+            and _final_inf_df is not None
+            and np.isfinite(_final_inf_df)
+            and _final_inf_df > 0
+        ):
+            _es_df_provenance = float(_final_inf_df)
+
         results = ChaisemartinDHaultfoeuilleResults(
             overall_att=effective_overall_att,
             overall_se=effective_overall_se,
@@ -4226,6 +4263,7 @@ def fit(
             event_study_effects=event_study_effects,
             L_max=L_max,
             placebo_event_study=placebo_event_study_dict,
+            event_study_df=_es_df_provenance,
             twfe_weights=twfe_weights_df,
             twfe_fraction_negative=twfe_fraction_negative,
             twfe_sigma_fe=twfe_sigma_fe,

diff_diff/chaisemartin_dhaultfoeuille_results.py

@@ -372,6 +372,15 @@ class ChaisemartinDHaultfoeuilleResults(BaseResults):
         ``1..L_max`` when ``L_max >= 1``. When ``L_max >= 1``, uses the
         per-group ``DID_{g,l}`` path; when ``L_max=None``, uses the
         per-period ``DID_M`` path.
+    event_study_df : float, optional
+        Inference degrees-of-freedom PROVENANCE for the event-study and
+        placebo surfaces: the single df every stored row's
+        ``safe_inference`` received. One scalar suffices because both
+        surfaces are computed from the same design df (and are refreshed
+        together to the final effective df under replicate weights).
+        ``None`` when the rows used normal theory (no survey design), when
+        the df was undefined, or when bootstrap overrode the stored
+        inference with percentile values that never used a df.
     normalized_effects : dict, optional
         Normalized estimator ``DID^n_l``. Populated when ``L_max >= 1``.
     cost_benefit_delta : dict, optional
@@ -567,6 +576,18 @@ class ChaisemartinDHaultfoeuilleResults(BaseResults):
     # Dynamic placebos DID^{pl}_l with negative horizon keys.
     # None in Phase 1; populated as {-1: {...}, -2: {...}} in Phase 2.
     placebo_event_study: Optional[Dict[int, Dict[str, Any]]] = field(default=None, repr=False)
+    # event_study_df (spec section 5, row M-092): the ONE inference df every
+    # stored event-study AND placebo row's ``safe_inference`` actually
+    # received - a single scalar because both surfaces are computed from the
+    # same design df (and, under replicate weights, are re-run together to
+    # the final effective df). None means the rows used normal theory (no
+    # survey design), the df was undefined (the df<=0 replicate sentinel,
+    # which yields NaN inference), or bootstrap overrode the stored p/CIs
+    # with percentile values that never used a df. Note the bootstrap clear
+    # drops the whole channel even when a partial override leaves some rows
+    # analytic - a conservative under-claim, consistent with the other
+    # producers.
+    event_study_df: Optional[float] = field(default=None, repr=False)
 
     # --- TWFE decomposition diagnostic (Theorem 1 of AER 2020) ---
     twfe_weights: Optional[pd.DataFrame] = field(default=None, repr=False)

diff_diff/lpdid.py

@@ -1460,6 +1460,7 @@ def fit(
         pooled = None
 
         event_study_df: Optional[Dict[int, float]] = None
+        pooled_df: Optional[Dict[str, float]] = None
         if not only_pooled:
             event_rows = []
             event_study_df = {}
@@ -1528,11 +1529,17 @@ def fit(
                 absorb=absorb,
                 survey_design=survey_design,
             )
-            # Discard the pooled windows' df provenance (native pooled frame
-            # schema unchanged; pooled-window df threading is a tracked
-            # follow-up in TODO.md).
-            pre_estimate.pop("df", None)
-            post_estimate.pop("df", None)
+            # Capture the pooled windows' df provenance BEFORE building the
+            # frame so the native pooled schema stays unchanged (it feeds
+            # summary()'s table). Each value is the df that window's
+            # safe_inference actually received; NaN when it used normal
+            # theory or the df was undefined.
+            _pre_df = pre_estimate.pop("df", None)
+            _post_df = post_estimate.pop("df", None)
+            pooled_df = {
+                "pre": float(_pre_df) if _pre_df is not None else float("nan"),
+                "post": float(_post_df) if _post_df is not None else float("nan"),
+            }
             pooled = pd.DataFrame(
                 [
                     {
@@ -1566,6 +1573,7 @@ def fit(
             event_study=event_study,
             pooled=pooled,
             event_study_df=event_study_df,
+            pooled_df=pooled_df,
             n_obs=len(data),
             n_treated_units=int(treatment_by_unit.gt(0).sum()),
             n_control_units=int(treatment_by_unit.eq(0).sum()),

diff_diff/lpdid_results.py

@@ -21,6 +21,13 @@ class LPDiDResults(BaseResults):
     ``control_group="clean"`` the realized control pool at each horizon also
     includes not-yet-treated cohorts, whose per-horizon counts live in the
     ``n_obs`` / ``n_clusters`` columns of the tables.
+
+    ``event_study_df`` and ``pooled_df`` carry inference degrees-of-freedom
+    PROVENANCE - the df each row's ``safe_inference`` actually received -
+    keyed by horizon and by ``"pre"``/``"post"`` window respectively. Both
+    are NaN per entry where that row used normal theory or an undefined df.
+    They are metadata only: the native tables above are unchanged, and
+    ``event_study_df`` is what the unified event-study surface reads.
     """
 
     event_study: Optional[pd.DataFrame]
@@ -63,6 +70,14 @@ class LPDiDResults(BaseResults):
     # of which the native tables record. The synthetic ``horizon == -1``
     # base row has no entry. None when no event study was fit.
     event_study_df: Optional[Dict[int, float]] = None
+    # pooled_df: the same df PROVENANCE for the POOLED windows, keyed
+    # ``"pre"`` / ``"post"`` to match the ``window`` column. The pooled-post
+    # row is the headline ATT, which lives outside the unified event-study
+    # surface, so this is standalone headline provenance (the
+    # ``DiDResults.inference_df`` analogue) rather than a surface input. NaN
+    # per window when that window's inference used normal theory or an
+    # undefined df; None when no pooled windows were fit (``only_event``).
+    pooled_df: Optional[Dict[str, float]] = None
 
     # ------------------------------------------------------------------
     # internal helpers
@@ -164,6 +179,8 @@ def to_dict(self) -> Dict[str, Any]:
             result["cluster_name"] = self.cluster_name
         if self.n_clusters is not None:
             result["n_clusters"] = self.n_clusters
+        if self.pooled_df is not None:
+            result["pooled_df"] = self.pooled_df
         if self.non_absorbing is not None:
             result["non_absorbing"] = self.non_absorbing
             result["stabilization_window"] = self.stabilization_window

diff_diff/results_base.py

@@ -214,9 +214,11 @@ class EventStudyResults(BaseResults):
         from the producer. Accepts None (no df exposed -> all-NaN column), a
         scalar (broadcast to every row - e.g. CallawaySantAnna, whose
         explicit-survey event study applies ONE conservative df, the minimum
-        per-horizon effective df, to all rows), or a length-n array (per-row
-        producers: StackedDiD ``hc2_bm`` per-event Bell-McCaffrey df, LPDiD
-        per-horizon cluster df, MultiPeriodDiD ``hc2_bm`` per-period df).
+        per-horizon effective df, to all rows; or de
+        Chaisemartin-D'Haultfoeuille, whose effect and placebo rows share one
+        design df), or a length-n array (per-row producers: StackedDiD and
+        SunAbraham ``hc2_bm`` per-event Bell-McCaffrey df, LPDiD per-horizon
+        cluster df, MultiPeriodDiD ``hc2_bm`` per-period df).
         NaN on any row means normal-theory inference, an undefined df,
         bootstrap-overridden inference, or a producer that records none;
         reference rows and rows with NaN p-values are always NaN.
@@ -949,6 +951,12 @@ def _from_dcdh(results: Any) -> EventStudyResults:
         cband_crit_value=cband_crit,
         alpha=getattr(results, "alpha", 0.05),
         source=type(results).__name__,
+        # ONE scalar df for both merged surfaces: the effect rows and the
+        # placebo rows are computed from the same design df (and, under
+        # replicate weights, refreshed together to the final effective df),
+        # so a scalar is faithful - the container broadcasts it and NaN-masks
+        # the synthesized reference row at 0 plus any NaN-p rows.
+        df=getattr(results, "event_study_df", None),
     )
 
 

diff_diff/stacked_did.py

@@ -840,9 +840,10 @@ def _refit_stacked(w_r):
                 # failed OR noise-floor NaN guard fired), emit all-NaN inference
                 # rather than fall back to normal-theory CIs/p-values. Mirrors
                 # the fix in LinearRegression.get_inference() from PR #475 R7
-                # (linalg.py:3689-3706). Without this, safe_inference(df=NaN)
-                # would pass df comparison >= 0 (NaN < 0 is False) and emit
-                # finite t_stat with NaN p/CI — silent wrong inference.
+                # (linalg.py:3689-3706). `safe_inference` itself has guarded
+                # non-finite / <= 0 df since PR #620 (utils.py) with the same
+                # all-NaN result; this explicit branch predates that guard and
+                # is kept for explicitness (it also skips the call entirely).
                 _is_hc2bm_path = self.vcov_type == "hc2_bm" and not _uses_replicate_sd
                 _bm_df = _bm_contrast_dof_per_event.get(h)
                 if _is_hc2bm_path and (_bm_df is None or not np.isfinite(_bm_df)):