diff --git a/CHANGELOG.md b/CHANGELOG.md index 85ea4a12..0caa3f3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 metacharacter — so no path composed into a shell command can execute. `persist` self-cleans the whole invocation on any failure (no post-persist cleanup needed) and `abort` is strict (a missing state token is an error). +- **df provenance completed for every event-study producer that exposes one + (4.0 program Phase 2, ledger row M-092 follow-up).** `SunAbrahamResults` gains + `event_study_df` (a per-relative-time dict: the per-event Bell-McCaffrey contrast df + under `hc2_bm`, the survey design df - post-drop under replicate refits - on survey + fits, NaN where inference is normal-theory or its BM DOF was non-finite), and + `ChaisemartinDHaultfoeuilleResults` gains a scalar `event_study_df` (one value, since + its effect and placebo rows are computed from the same design df and refreshed + together under replicate weights). Both clear under bootstrap, whose percentile + p-values never used a df; SunAbraham's clear is deliberately narrower than its + `event_study_vcov` clear, since a replicate refit's rows genuinely used the replicate + df. `LPDiDResults` gains `pooled_df` (`{"pre": ..., "post": ...}`), the headline + pooled-window analogue of `DiDResults.inference_df`, surfaced in `to_dict()`; the + native pooled frame schema is unchanged. With these, `EventStudyResults.df` is + populated for every producer whose inference records a df. - **Docs: navigation and SEO polish.** A stable/latest version switcher in the navbar (`docs/_static/switcher.json`); a custom `robots.txt` that keeps the ~60 thin `_modules/` source-view pages out of crawlers (documentation pages allowed as @@ -123,6 +137,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 **Estimator equations, weighting, variance, and numerical output are unchanged.** ### Changed +- **dCDH Phase 1 (`L_max=None`) event-study row is now re-synced by the final-df + refresh.** Under replicate-weight designs the refresh recomputes `overall_*` with the + final effective df, but the Phase 1 event-study row is a VALUE COPY of that inference + taken beforehand, and the refresh's per-horizon loops cover only the Phase 2 surface - + so the copy could in principle retain an intermediate df's t/p/CI while + `survey_metadata.df_survey` reported the final one. The copy is now mirrored from the + refreshed values, making the stored row agree with the overall estimate it duplicates + by construction. No output change is reproducible on any configuration exercised in + the test suite (the contributing sites agree on their valid-replicate counts, so the + intermediate and final df coincide and the mirror is a no-op); this closes the + structural hazard rather than a demonstrated defect. - **DiagnosticReport's parallel-trends check upgrades from the Bonferroni fallback to the joint Wald test for StackedDiD and TwoStageDiD event-study fits.** The check's joint-Wald path runs whenever the source fit exposes `event_study_vcov`; now that diff --git a/TODO.md b/TODO.md index 19941dc2..77be276a 100644 --- a/TODO.md +++ b/TODO.md @@ -23,7 +23,6 @@ Related tracking surfaces: |-------|----------|--------|--------|----------| | `SyntheticControl` conformal (CWZ 2021) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies — the residual-permutation shortcut is only valid for time-permutation-invariant proxies (SC/Lasso/DiD); an AR proxy needs innovation permutation. | `diff_diff/conformal.py`, `diff_diff/synthetic_control_results.py` | CWZ-2021 | Heavy | Low | | `ContinuousDiD` CGBS-2024 remaining extensions (earlier phases — `covariates=` reg/dr, `treatment_type="discrete"`, single-cohort `control_group="lowest_dose"` with estimand `ATT(d)−ATT(d_L)` — are already supported; see REGISTRY Note #7). Remaining (all deferred `NotImplementedError`, documented): `estimation_method="ipw"` on the dose curve (scalar-adjustment / degenerate); `covariates=` × `survey_design=` (weighted OR + weighted nuisance IF); multi-cohort **heterogeneous-support** discrete aggregation (support-aware: average each dose only over the cohorts that observe it); **multi-cohort `lowest_dose`** (within-cohort `d_L` reference + support-aware cross-cohort aggregation); and **`covariates=` × `lowest_dose`** (conditional-PT-relative-to-`d_L` estimand). Single-cohort / 2-period / shared-support multi-cohort are supported. | `continuous_did.py` | CGBS-2024 | Heavy | Low | -| `event_study_df` provenance for the remaining producers (4.0 program Phase 2 follow-up; the Stacked/TwoStage vcov + CS/MPD/Stacked/TwoStage/LPDiD df round-trip landed with the per-row `EventStudyResults.df` schema). Remaining: SunAbraham (per-event BM Satterthwaite df under `hc2_bm` at `sun_abraham.py:1206`, scalar survey df at `:1256` - mechanism identical to StackedDiD's recorded-at-use dict); dCDH (its event-study inference receives a `df_inference` argument, `chaisemartin_dhaultfoeuille.py:6311`, never stored - needs tracing to the stored rows); LPDiD pooled-window df (popped and discarded at fit assembly). All purely additive: their surfaces show df=NaN ("unexposed") until threaded. | `diff_diff/sun_abraham.py`, `diff_diff/chaisemartin_dhaultfoeuille.py`, `diff_diff/lpdid.py` | #M-092 | Quick | Low | ### Performance diff --git a/diff_diff/chaisemartin_dhaultfoeuille.py b/diff_diff/chaisemartin_dhaultfoeuille.py index be4ec2ae..2fd34f7e 100644 --- a/diff_diff/chaisemartin_dhaultfoeuille.py +++ b/diff_diff/chaisemartin_dhaultfoeuille.py @@ -4039,8 +4039,11 @@ def fit( # and HonestDiD. Re-run safe_inference with the FINAL # effective df so every surface agrees. _final_eff_df = _effective_df_survey(resolved_survey, _replicate_n_valid_list) + # ONE canonical final inference df, shared by the refresh below and + # by the `event_study_df` provenance recorded at construction, so the + # two can never drift apart. + _final_inf_df = _inference_df(_final_eff_df, resolved_survey) if _replicate_n_valid_list: - _final_inf_df = _inference_df(_final_eff_df, resolved_survey) # Recompute `effective_overall_*` directly — that's what # ships in results at line ~2776+. `effective_overall_att/se` # may differ from the raw `overall_att/se` under the delta @@ -4087,6 +4090,22 @@ def fit( _info_r2["t_stat"] = _t_r2 _info_r2["p_value"] = _p_r2 _info_r2["conf_int"] = _ci_r2 + else: + # Phase 1 (L_max=None): `event_study_effects[1]` is a VALUE + # COPY of the overall inference, built above from the + # PRE-refresh `overall_*`; the loop just above covers only + # the Phase 2 `multi_horizon_inference` surface. Without + # this mirror the stored l=1 row keeps the INTERMEDIATE + # df's t/p/CI while `overall_*` and + # `survey_metadata.df_survey` report the final (tightened) + # df - the same value-copy hazard the placebo mirror below + # documents. Re-sync from the refreshed `overall_*` so the + # single event-study row agrees with the overall estimate it + # is a copy of. + if event_study_effects and 1 in event_study_effects: + event_study_effects[1]["t_stat"] = overall_t + event_study_effects[1]["p_value"] = overall_p + event_study_effects[1]["conf_int"] = overall_ci if placebo_horizon_inference is not None: for _lag_r2, _info_r2 in list(placebo_horizon_inference.items()): _t_r2, _p_r2, _ci_r2 = safe_inference( @@ -4181,6 +4200,24 @@ def fit( if survey_metadata is not None: survey_metadata.df_survey = _final_eff_df + # df PROVENANCE (spec section 5, row M-092): the scalar df every + # stored event-study / placebo row's safe_inference received. + # Reuses `_final_inf_df` - the SAME value the refresh above applied + # to every stored row - so provenance and inference cannot drift. + # Recorded iff finite and > 0. + # Cleared under bootstrap, whose percentile p/CIs never used a df: + # required, not defensive, because TSL survey + bootstrap is a live + # mode (only REPLICATE + bootstrap is rejected) where this + # expression evaluates finite. + _es_df_provenance: Optional[float] = None + if ( + bootstrap_results is None + and _final_inf_df is not None + and np.isfinite(_final_inf_df) + and _final_inf_df > 0 + ): + _es_df_provenance = float(_final_inf_df) + results = ChaisemartinDHaultfoeuilleResults( overall_att=effective_overall_att, overall_se=effective_overall_se, @@ -4226,6 +4263,7 @@ def fit( event_study_effects=event_study_effects, L_max=L_max, placebo_event_study=placebo_event_study_dict, + event_study_df=_es_df_provenance, twfe_weights=twfe_weights_df, twfe_fraction_negative=twfe_fraction_negative, twfe_sigma_fe=twfe_sigma_fe, diff --git a/diff_diff/chaisemartin_dhaultfoeuille_results.py b/diff_diff/chaisemartin_dhaultfoeuille_results.py index e7536f37..7aa94be5 100644 --- a/diff_diff/chaisemartin_dhaultfoeuille_results.py +++ b/diff_diff/chaisemartin_dhaultfoeuille_results.py @@ -372,6 +372,15 @@ class ChaisemartinDHaultfoeuilleResults(BaseResults): ``1..L_max`` when ``L_max >= 1``. When ``L_max >= 1``, uses the per-group ``DID_{g,l}`` path; when ``L_max=None``, uses the per-period ``DID_M`` path. + event_study_df : float, optional + Inference degrees-of-freedom PROVENANCE for the event-study and + placebo surfaces: the single df every stored row's + ``safe_inference`` received. One scalar suffices because both + surfaces are computed from the same design df (and are refreshed + together to the final effective df under replicate weights). + ``None`` when the rows used normal theory (no survey design), when + the df was undefined, or when bootstrap overrode the stored + inference with percentile values that never used a df. normalized_effects : dict, optional Normalized estimator ``DID^n_l``. Populated when ``L_max >= 1``. cost_benefit_delta : dict, optional @@ -656,6 +665,21 @@ class ChaisemartinDHaultfoeuilleResults(BaseResults): bootstrap_results: Optional[DCDHBootstrapResults] = field(default=None, repr=False) _estimator_ref: Optional[Any] = field(default=None, repr=False) + # event_study_df (spec section 5, row M-092): the ONE inference df every + # stored event-study AND placebo row's ``safe_inference`` actually + # received - a single scalar because both surfaces are computed from the + # same design df (and, under replicate weights, are re-run together to + # the final effective df). None means the rows used normal theory (no + # survey design), the df was undefined (the df<=0 replicate sentinel, + # which yields NaN inference), or bootstrap overrode the stored p/CIs + # with percentile values that never used a df. Note the bootstrap clear + # drops the whole channel even when a partial override leaves some rows + # analytic - a conservative under-claim, consistent with the other + # producers. + # Declared LAST so every pre-existing field keeps its positional index + # in the generated __init__ (the constructor signature is public API). + event_study_df: Optional[float] = field(default=None, repr=False) + # ------------------------------------------------------------------ # Repr / properties # ------------------------------------------------------------------ diff --git a/diff_diff/lpdid.py b/diff_diff/lpdid.py index eef33131..2bec51ad 100644 --- a/diff_diff/lpdid.py +++ b/diff_diff/lpdid.py @@ -1460,6 +1460,7 @@ def fit( pooled = None event_study_df: Optional[Dict[int, float]] = None + pooled_df: Optional[Dict[str, float]] = None if not only_pooled: event_rows = [] event_study_df = {} @@ -1528,11 +1529,17 @@ def fit( absorb=absorb, survey_design=survey_design, ) - # Discard the pooled windows' df provenance (native pooled frame - # schema unchanged; pooled-window df threading is a tracked - # follow-up in TODO.md). - pre_estimate.pop("df", None) - post_estimate.pop("df", None) + # Capture the pooled windows' df provenance BEFORE building the + # frame so the native pooled schema stays unchanged (it feeds + # summary()'s table). Each value is the df that window's + # safe_inference actually received; NaN when it used normal + # theory or the df was undefined. + _pre_df = pre_estimate.pop("df", None) + _post_df = post_estimate.pop("df", None) + pooled_df = { + "pre": float(_pre_df) if _pre_df is not None else float("nan"), + "post": float(_post_df) if _post_df is not None else float("nan"), + } pooled = pd.DataFrame( [ { @@ -1566,6 +1573,7 @@ def fit( event_study=event_study, pooled=pooled, event_study_df=event_study_df, + pooled_df=pooled_df, n_obs=len(data), n_treated_units=int(treatment_by_unit.gt(0).sum()), n_control_units=int(treatment_by_unit.eq(0).sum()), diff --git a/diff_diff/lpdid_results.py b/diff_diff/lpdid_results.py index 7de1bd71..7164aa65 100644 --- a/diff_diff/lpdid_results.py +++ b/diff_diff/lpdid_results.py @@ -21,6 +21,13 @@ class LPDiDResults(BaseResults): ``control_group="clean"`` the realized control pool at each horizon also includes not-yet-treated cohorts, whose per-horizon counts live in the ``n_obs`` / ``n_clusters`` columns of the tables. + + ``event_study_df`` and ``pooled_df`` carry inference degrees-of-freedom + PROVENANCE - the df each row's ``safe_inference`` actually received - + keyed by horizon and by ``"pre"``/``"post"`` window respectively. Both + are NaN per entry where that row used normal theory or an undefined df. + They are metadata only: the native tables above are unchanged, and + ``event_study_df`` is what the unified event-study surface reads. """ event_study: Optional[pd.DataFrame] @@ -63,6 +70,14 @@ class LPDiDResults(BaseResults): # of which the native tables record. The synthetic ``horizon == -1`` # base row has no entry. None when no event study was fit. event_study_df: Optional[Dict[int, float]] = None + # pooled_df: the same df PROVENANCE for the POOLED windows, keyed + # ``"pre"`` / ``"post"`` to match the ``window`` column. The pooled-post + # row is the headline ATT, which lives outside the unified event-study + # surface, so this is standalone headline provenance (the + # ``DiDResults.inference_df`` analogue) rather than a surface input. NaN + # per window when that window's inference used normal theory or an + # undefined df; None when no pooled windows were fit (``only_event``). + pooled_df: Optional[Dict[str, float]] = None # ------------------------------------------------------------------ # internal helpers @@ -164,6 +179,8 @@ def to_dict(self) -> Dict[str, Any]: result["cluster_name"] = self.cluster_name if self.n_clusters is not None: result["n_clusters"] = self.n_clusters + if self.pooled_df is not None: + result["pooled_df"] = self.pooled_df if self.non_absorbing is not None: result["non_absorbing"] = self.non_absorbing result["stabilization_window"] = self.stabilization_window diff --git a/diff_diff/results_base.py b/diff_diff/results_base.py index 51e8e307..2c5a308c 100644 --- a/diff_diff/results_base.py +++ b/diff_diff/results_base.py @@ -214,9 +214,11 @@ class EventStudyResults(BaseResults): from the producer. Accepts None (no df exposed -> all-NaN column), a scalar (broadcast to every row - e.g. CallawaySantAnna, whose explicit-survey event study applies ONE conservative df, the minimum - per-horizon effective df, to all rows), or a length-n array (per-row - producers: StackedDiD ``hc2_bm`` per-event Bell-McCaffrey df, LPDiD - per-horizon cluster df, MultiPeriodDiD ``hc2_bm`` per-period df). + per-horizon effective df, to all rows; or de + Chaisemartin-D'Haultfoeuille, whose effect and placebo rows share one + design df), or a length-n array (per-row producers: StackedDiD and + SunAbraham ``hc2_bm`` per-event Bell-McCaffrey df, LPDiD per-horizon + cluster df, MultiPeriodDiD ``hc2_bm`` per-period df). NaN on any row means normal-theory inference, an undefined df, bootstrap-overridden inference, or a producer that records none; reference rows and rows with NaN p-values are always NaN. @@ -949,6 +951,12 @@ def _from_dcdh(results: Any) -> EventStudyResults: cband_crit_value=cband_crit, alpha=getattr(results, "alpha", 0.05), source=type(results).__name__, + # ONE scalar df for both merged surfaces: the effect rows and the + # placebo rows are computed from the same design df (and, under + # replicate weights, refreshed together to the final effective df), + # so a scalar is faithful - the container broadcasts it and NaN-masks + # the synthesized reference row at 0 plus any NaN-p rows. + df=getattr(results, "event_study_df", None), ) diff --git a/diff_diff/stacked_did.py b/diff_diff/stacked_did.py index 012ae674..cd07c6d9 100644 --- a/diff_diff/stacked_did.py +++ b/diff_diff/stacked_did.py @@ -840,9 +840,10 @@ def _refit_stacked(w_r): # failed OR noise-floor NaN guard fired), emit all-NaN inference # rather than fall back to normal-theory CIs/p-values. Mirrors # the fix in LinearRegression.get_inference() from PR #475 R7 - # (linalg.py:3689-3706). Without this, safe_inference(df=NaN) - # would pass df comparison >= 0 (NaN < 0 is False) and emit - # finite t_stat with NaN p/CI — silent wrong inference. + # (linalg.py:3689-3706). `safe_inference` itself has guarded + # non-finite / <= 0 df since PR #620 (utils.py) with the same + # all-NaN result; this explicit branch predates that guard and + # is kept for explicitness (it also skips the call entirely). _is_hc2bm_path = self.vcov_type == "hc2_bm" and not _uses_replicate_sd _bm_df = _bm_contrast_dof_per_event.get(h) if _is_hc2bm_path and (_bm_df is None or not np.isfinite(_bm_df)): diff --git a/diff_diff/sun_abraham.py b/diff_diff/sun_abraham.py index 79bca487..9826fa4a 100644 --- a/diff_diff/sun_abraham.py +++ b/diff_diff/sun_abraham.py @@ -43,6 +43,17 @@ class SunAbrahamResults(BaseResults): event_study_effects : dict Dictionary mapping relative time to effect dictionaries with keys: 'effect', 'se', 't_stat', 'p_value', 'conf_int', 'n_groups'. + event_study_df : dict, optional + Per-relative-time inference degrees-of-freedom PROVENANCE: the df + each stored event-study row's ``safe_inference`` actually received. + Under ``vcov_type="hc2_bm"`` this is that event time's own + Bell-McCaffrey contrast DOF; under a survey design it is the design + df (post-drop when replicates were dropped by the refit); it is NaN + where inference was normal-theory or where a non-finite BM DOF made + the row's inference undefined. ``None`` under bootstrap, whose + percentile p-values/CIs never used a df - a narrower clearing rule + than ``event_study_vcov``, which also clears under replicate refits + (whose rows DID use a genuine df). overall_att : float Overall average treatment effect (weighted average of post-treatment effects). overall_se : float @@ -140,6 +151,24 @@ class SunAbrahamResults(BaseResults): reference_period: Optional[int] = None reference_observed: bool = False + # event_study_df (spec section 5, row M-092): per-event-time df + # PROVENANCE - maps each estimated relative time to the df its stored + # p-value/CI's safe_inference actually received (the per-event + # Bell-McCaffrey contrast df under hc2_bm; the survey design df - + # post-drop under replicate refits - on survey fits; NaN on plain + # analytic fits, which use normal theory, and on rows whose BM DOF was + # non-finite, where safe_inference's non-finite-df guard yields all-NaN + # inference). None under bootstrap: the stored percentile p/CIs never + # used a df (note this clears the WHOLE channel even when a partial + # bootstrap override leaves some rows analytic - a conservative + # under-claim, consistent with the other producers). Deliberately + # narrower clearing than event_study_vcov above: replicate refits KEEP + # the df (it genuinely governed the recomputed rows) while the vcov + # clears. + # Declared LAST so every pre-existing field keeps its positional index + # in the generated __init__ (the constructor signature is public API). + event_study_df: Optional[Dict[int, float]] = field(default=None, repr=False) + # --- Inference-field aliases (balance/external-adapter compatibility) --- @property def att(self) -> float: @@ -1073,6 +1102,22 @@ def _refit_sa(w_r): survey_df=_sa_survey_df, ) + # Per-row df PROVENANCE (spec section 5, row M-092): the df each + # stored ES row's safe_inference actually received, recorded iff + # finite and > 0 else NaN. Baseline = the scalar every + # _compute_iw_effects row used (None on plain analytic fits -> NaN; + # the TSL design df on survey fits). Overwritten below at the hc2_bm + # and replicate-refit override sites, and cleared under bootstrap, + # whose stored percentile p/CIs never used a df. + es_df_used: Dict[int, float] = { + e: ( + float(_sa_survey_df) + if _sa_survey_df is not None and np.isfinite(_sa_survey_df) and _sa_survey_df > 0 + else float("nan") + ) + for e in event_study_effects + } + # Build full event-study VCV via W-matrix aggregation (PR-B 2026-05-17). # event_study_effects[e] = Σ_g w_{g,e} * cohort_effects[(g, e)] with # w_{g,e} = cohort_weights[e][g]. The full event-study VCV is @@ -1203,10 +1248,19 @@ def _refit_sa(w_r): for e, df_e in _es_contrast_dofs.items(): eff_e = event_study_effects[e]["effect"] se_e = event_study_effects[e]["se"] + # A non-finite BM DOF fails closed INSIDE safe_inference + # (its non-finite/<=0 df guard, PR #620): all-NaN t/p/CI. + # The provenance record mirrors that - NaN df for a row + # whose stored inference is the guard's all-NaN output. t_e, p_e, ci_e = safe_inference(eff_e, se_e, alpha=self.alpha, df=df_e) event_study_effects[e]["t_stat"] = t_e event_study_effects[e]["p_value"] = p_e event_study_effects[e]["conf_int"] = ci_e + es_df_used[e] = ( + float(df_e) + if df_e is not None and np.isfinite(df_e) and df_e > 0 + else float("nan") + ) overall_t, overall_p, overall_ci = safe_inference( overall_att, @@ -1258,6 +1312,17 @@ def _refit_sa(w_r): event_study_effects[e]["t_stat"] = t_e event_study_effects[e]["p_value"] = p_e event_study_effects[e]["conf_int"] = ci_e + # Rows recomputed here used the POST-DROP replicate df + # (possibly tightened above when replicates were + # dropped) - genuine provenance, kept (unlike the vcov, + # which clears under replicate refits). + es_df_used[e] = ( + float(_sa_survey_df) + if _sa_survey_df is not None + and np.isfinite(_sa_survey_df) + and _sa_survey_df > 0 + else float("nan") + ) # Cohort-level replicate SEs: second refit for raw (g,e) coefficients _keys_ordered = sorted(coef_index_map.keys(), key=lambda k: coef_index_map[k]) @@ -1350,6 +1415,14 @@ def _refit_sa_cohort(w_r): if bootstrap_results is not None or _uses_replicate_sa: es_vcov = None es_vcov_index = None + # df provenance clears on BOOTSTRAP ONLY - a deliberately narrower + # predicate than the vcov clear above: under a replicate refit the + # recomputed rows genuinely used the (post-drop) replicate df, so + # keeping it is faithful provenance; under bootstrap the stored + # percentile p/CIs never used any df. + es_df_final: Optional[Dict[int, float]] = es_df_used + if bootstrap_results is not None: + es_df_final = None # Store results self.results_ = SunAbrahamResults( @@ -1374,6 +1447,7 @@ def _refit_sa_cohort(w_r): survey_metadata=survey_metadata, event_study_vcov=es_vcov, event_study_vcov_index=es_vcov_index, + event_study_df=es_df_final, conley_lag_cutoff=(self.conley_lag_cutoff if self.vcov_type == "conley" else None), cluster_name=(self.cluster if self.vcov_type == "conley" else None), reference_period=self._reference_period, diff --git a/docs/api/_autosummary/diff_diff.ChaisemartinDHaultfoeuilleResults.rst b/docs/api/_autosummary/diff_diff.ChaisemartinDHaultfoeuilleResults.rst index 85cebec5..db0243ea 100644 --- a/docs/api/_autosummary/diff_diff.ChaisemartinDHaultfoeuilleResults.rst +++ b/docs/api/_autosummary/diff_diff.ChaisemartinDHaultfoeuilleResults.rst @@ -33,6 +33,7 @@ ~ChaisemartinDHaultfoeuilleResults.cost_benefit_delta ~ChaisemartinDHaultfoeuilleResults.covariate_residuals ~ChaisemartinDHaultfoeuilleResults.design2_effects + ~ChaisemartinDHaultfoeuilleResults.event_study_df ~ChaisemartinDHaultfoeuilleResults.event_study_effects ~ChaisemartinDHaultfoeuilleResults.heterogeneity_effects ~ChaisemartinDHaultfoeuilleResults.honest_did_results diff --git a/docs/api/_autosummary/diff_diff.SunAbrahamResults.rst b/docs/api/_autosummary/diff_diff.SunAbrahamResults.rst index d55db5db..5d03aa9f 100644 --- a/docs/api/_autosummary/diff_diff.SunAbrahamResults.rst +++ b/docs/api/_autosummary/diff_diff.SunAbrahamResults.rst @@ -34,6 +34,7 @@ ~SunAbrahamResults.conf_int ~SunAbrahamResults.conley_lag_cutoff ~SunAbrahamResults.control_group + ~SunAbrahamResults.event_study_df ~SunAbrahamResults.event_study_vcov ~SunAbrahamResults.event_study_vcov_index ~SunAbrahamResults.is_significant diff --git a/docs/api/_autosummary/diff_diff.lpdid_results.LPDiDResults.rst b/docs/api/_autosummary/diff_diff.lpdid_results.LPDiDResults.rst index ff3f4a3e..ae4bb6ab 100644 --- a/docs/api/_autosummary/diff_diff.lpdid_results.LPDiDResults.rst +++ b/docs/api/_autosummary/diff_diff.lpdid_results.LPDiDResults.rst @@ -38,6 +38,7 @@ ~LPDiDResults.n_strata ~LPDiDResults.non_absorbing ~LPDiDResults.p_value + ~LPDiDResults.pooled_df ~LPDiDResults.rank_deficient_action ~LPDiDResults.se ~LPDiDResults.stabilization_window diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 85530ec0..0482d270 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -868,6 +868,18 @@ The multiplier bootstrap uses random weights w_i with E[w]=0 and Var(w)=1: - **Note:** Multi-switch groups (those with more than one treatment-change period) are dropped before estimation when `drop_larger_lower=True` (the default, matching R `DIDmultiplegtDYN`). For binary treatment, >1 change means a reversal (e.g., 0->1->0). For non-binary, >1 change includes both reversals (0->2->1) and monotone multi-step paths (0->1->2); both are dropped because the per-group `DID_{g,l}` building block attributes the full outcome change from `F_g-1` to `F_g-1+l` to the first treatment change, and a second change would confound that attribution. A single jump of any magnitude (0->3->3->3) has 1 change period and is kept. Each drop emits a warning with the count and example group IDs. - Singleton-baseline groups — groups whose `D_{g,1}` value is unique in the post-drop dataset — are excluded from the **variance computation only** (per footnote 15 of the dynamic paper, they have no cohort peer). They are **retained** in the point-estimate sample as period-based stable controls. Each emits a warning. See the singleton-baseline Note below. - Never-switching groups (`S_g = 0`) participate in the variance computation when they serve as stable controls under the full influence function. The `n_groups_dropped_never_switching` results field is reported for backwards compatibility but the count no longer represents an actual exclusion. +- **Note:** Provenance exposure (4.0 program row M-092, no numeric change): + `ChaisemartinDHaultfoeuilleResults.event_study_df` records the ONE df every + stored event-study AND placebo row's `safe_inference` received. A single + scalar is faithful because both surfaces are computed from the same design + df: None without a survey design (normal-theory inference), the design df + under Taylor linearization, and the final effective df under replicate + weights (where the late refresh re-runs every stored row - including, from + this change, the Phase 1 `L_max=None` event-study row, which is a value + copy of the overall inference - to that single value). Cleared under + bootstrap, whose percentile p/CIs never used a df; that clear is required + rather than defensive, since Taylor-linearized survey designs DO compose + with bootstrap (only replicate weights + bootstrap is rejected). - **Balanced-baseline panel required (deviation from R `DIDmultiplegtDYN`).** Every group must have an observation at the **first global period** (the panel's earliest time value); groups missing this baseline raise `ValueError` with the offending group IDs. Groups with **interior period gaps** (missing observations between their first and last observed period) are dropped with a `UserWarning`. **Terminal missingness** (groups observed at the baseline but missing one or more *later* periods) is **retained**: the group contributes from its observed periods only, masked out of the missing transitions by the per-period `present = (N_mat[:, t] > 0) & (N_mat[:, t-1] > 0)` guard. See the ragged-panel deviation Note below. - **Period-index semantics.** The estimator operates on **sorted period indices**, not calendar dates. Per-period DIDs use `Y_{g,t} - Y_{g,t-1}` where `t-1` is the *previous observed period in the sorted panel*, not the previous calendar unit. A panel with periods `[2000, 2001, 2003]` (missing year 2002 for ALL groups) is treated as a valid 3-period panel where 2003 is the immediate successor of 2001. The estimator does NOT validate that periods are evenly spaced or that calendar gaps have been imputed. This matches the AER 2020 paper's Theorem 3, which defines transition sets by adjacent sorted periods without assuming calendar regularity, and is consistent with R `DIDmultiplegtDYN`'s behavior. If your data has calendar gaps that should be treated as missing periods rather than adjacent transitions, insert placeholder rows for the missing periods with the group's lagged treatment value and a reasonable imputed outcome (e.g., the group's last observed outcome), so the cell-aggregation step treats the gap as a stable-treatment period rather than a missing one. The validator rejects NaN in outcome and treatment columns, so placeholders must have finite values. - Per-period Assumption 11 violations (joiners exist but no stable-untreated controls in some period, or leavers exist but no stable-treated controls) trigger zero-retention behavior with a consolidated warning. See the A11 Note below. @@ -1486,6 +1498,19 @@ where weights ŵ_{g,e} = n_{g,e} / Σ_g n_{g,e} (sample share of cohort g at eve If the linalg helper fails (rank-deficient design, singular bread), the aggregated inference falls back to the shared analytical df with an explicit `UserWarning`. +- **Note:** Provenance exposure (4.0 program row M-092, no numeric change): + `SunAbrahamResults.event_study_df` records, per relative time, the df each + stored event-study row's `safe_inference` actually received - the per-event + BM contrast DOF above under `hc2_bm`, the survey design df (post-drop under + a replicate refit) on survey fits, and NaN on plain analytic fits (normal + theory) as well as on rows whose BM DOF was non-finite, where + `safe_inference`'s own non-finite-df guard already yields all-NaN + inference. Note the two failure modes differ and are both faithfully + recorded: a helper-level failure warns and falls back to the shared + analytical df (documented above), while a per-row non-finite DOF fails + closed. The channel clears under bootstrap (percentile p/CIs used no df) - + deliberately narrower than the `event_study_vcov` clear, which also fires + under replicate refits whose rows DID use a genuine df. - **Deviation from R (HC1 finite-sample correction):** SA's within-transform HC1 SE differs from `fixest::sunab(cluster=~unit)` by ~1-2% on typical panel sizes. fixest's correction counts the diff --git a/docs/v4-deprecations.yaml b/docs/v4-deprecations.yaml index 5c54d1c1..f4f52cdd 100644 --- a/docs/v4-deprecations.yaml +++ b/docs/v4-deprecations.yaml @@ -931,7 +931,7 @@ rows: phase: 2 test_ref: tests/test_event_study_surface.py code_refs: [diff_diff/results_base.py, diff_diff/__init__.py] - notes: "Phase 2 unified event-study representation (spec section 5): EventStudyResults container + builders for the 14 producers (CallawaySantAnna, SunAbraham, ImputationDiD, TwoStageDiD, StackedDiD, SpilloverDiD, ContinuousDiD, EfficientDiD, WooldridgeDiD, StaggeredTripleDifference, MultiPeriodDiD, LPDiD, ChaisemartinDHaultfoeuille, HeterogeneousAdoptionDiD). Canonical quintet columns, explicit is_reference marking (successor to the retiring sentinels [M-093]), vcov+vcov_index ordering, cband columns, event_time_convention metadata. Public exposure rides aggregate(type='event_study') in Phase 2 PR (b); merged TWFE returns it in Phase 3 [M-010]. introduced_in gates the 3.9 cut, mirroring [M-091]. Born done in this introducing diff (builder is package-internal; the class is exported). Amended pre-cut (same test_ref, introduced_in 3.9 not yet released): df became PER-ROW (one entry per event time, the df each stored p/CI actually used; joins the pinned to_dataframe schema) and StackedDiD/TwoStageDiD persist their internal full ES VCVs (event_study_vcov/_index/_df container fields; mode-gated for TwoStageDiD bootstrap/replicate)." + notes: "Phase 2 unified event-study representation (spec section 5): EventStudyResults container + builders for the 14 producers (CallawaySantAnna, SunAbraham, ImputationDiD, TwoStageDiD, StackedDiD, SpilloverDiD, ContinuousDiD, EfficientDiD, WooldridgeDiD, StaggeredTripleDifference, MultiPeriodDiD, LPDiD, ChaisemartinDHaultfoeuille, HeterogeneousAdoptionDiD). Canonical quintet columns, explicit is_reference marking (successor to the retiring sentinels [M-093]), vcov+vcov_index ordering, cband columns, event_time_convention metadata. Public exposure rides aggregate(type='event_study') in Phase 2 PR (b); merged TWFE returns it in Phase 3 [M-010]. introduced_in gates the 3.9 cut, mirroring [M-091]. Born done in this introducing diff (builder is package-internal; the class is exported). Amended pre-cut (same test_ref, introduced_in 3.9 not yet released): df became PER-ROW (one entry per event time, the df each stored p/CI actually used; joins the pinned to_dataframe schema) and StackedDiD/TwoStageDiD persist their internal full ES VCVs (event_study_vcov/_index/_df container fields; mode-gated for TwoStageDiD bootstrap/replicate). Completed pre-cut by the remaining producer channels: SunAbraham (per-event dict) and de Chaisemartin-D'Haultfoeuille (scalar) event_study_df, plus LPDiD pooled_df for the headline pre/post windows - every producer whose inference records a df now exposes it." - id: M-093 kind: behavior group: results-contract diff --git a/tests/test_event_study_surface.py b/tests/test_event_study_surface.py index cfe0c6d0..80365f07 100644 --- a/tests/test_event_study_surface.py +++ b/tests/test_event_study_surface.py @@ -1523,3 +1523,222 @@ def test_lpdid_per_horizon_df(surfaces): assert df_val == ncl[et] - 1 # The native frame schema is UNCHANGED (df lives on event_study_df only). assert "df" not in frame.columns + + +# =========================================================================== +# df provenance: remaining producers (SunAbraham, dCDH) + LPDiD pooled window +# =========================================================================== + + +def _sa_df_panel(seed=4): + rng = np.random.default_rng(seed) + rows = [] + for u in range(60): + g = [4, 6, 0][u % 3] + for t in range(1, 9): + y = 1.0 + 0.1 * t + u * 0.01 + (1.0 if g and t >= g else 0.0) + rng.normal(0, 0.3) + rows.append({"unit": u, "time": t, "outcome": y, "first_treat": g}) + return pd.DataFrame(rows) + + +def _sa_kwargs(): + return dict(outcome="outcome", unit="unit", time="time", first_treat="first_treat") + + +def test_sun_abraham_hc2_bm_per_row_df(): + data = _sa_df_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + bm = SunAbraham(vcov_type="hc2_bm").fit(data, **_sa_kwargs()) + plain = SunAbraham().fit(data, **_sa_kwargs()) + + # hc2_bm: each finite-p row carries its own BM contrast df, bit-equal to + # the producer's provenance dict (pure copies, no recomputation). + s_bm = build_event_study_surface(bm) + finite_p = np.isfinite(s_bm.p_value) + assert finite_p.any() + assert np.isfinite(s_bm.df[finite_p]).all() + for et, df_val in zip(s_bm.event_time.tolist(), s_bm.df.tolist()): + if et in bm.event_study_df and np.isfinite(bm.event_study_df[et]): + assert df_val == bm.event_study_df[et] + + # Plain analytic fit: normal-theory inference -> no df anywhere. + assert all(np.isnan(v) for v in plain.event_study_df.values()) + assert np.isnan(build_event_study_surface(plain).df).all() + + +def test_sun_abraham_survey_df_broadcast(): + from diff_diff.survey import SurveyDesign + + data = _sa_df_panel() + data["w"] = 1.0 + data["strat"] = data["unit"] % 4 + data["psu_id"] = data["unit"] + design = SurveyDesign(weights="w", weight_type="pweight", strata="strat", psu="psu_id") + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = SunAbraham().fit(data, survey_design=design, **_sa_kwargs()) + assert res.survey_metadata is not None and res.survey_metadata.df_survey is not None + surface = build_event_study_surface(res) + finite_p = np.isfinite(surface.p_value) + assert finite_p.any() + assert set(surface.df[finite_p].tolist()) == {float(max(res.survey_metadata.df_survey, 1))} + + +def test_sun_abraham_replicate_df_kept_while_vcov_cleared(): + # The clear predicates deliberately DIFFER: the analytical vcov is + # invalidated by the replicate refit, but the recomputed rows genuinely + # used the (post-drop) replicate df, so that provenance is kept. + from diff_diff.survey import SurveyDesign + + data = _sa_df_panel() + data["w"] = 1.0 + units = np.sort(data["unit"].unique()) + n_rep = 8 + unit_pos = {u: i for i, u in enumerate(units)} + rows_idx = data["unit"].map(unit_pos).to_numpy() + per = max(len(units) // n_rep, 1) + rep_cols = [] + for r in range(n_rep): + w_r = np.ones(len(units)) + w_r[r * per : min((r + 1) * per, len(units))] = 0.0 + nz = w_r > 0 + w_r[nz] = w_r[nz] * n_rep / (n_rep - 1) + data[f"rep_{r}"] = w_r[rows_idx] + rep_cols.append(f"rep_{r}") + design = SurveyDesign( + weights="w", + weight_type="pweight", + replicate_weights=rep_cols, + replicate_method="JK1", + ) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = SunAbraham().fit(data, survey_design=design, **_sa_kwargs()) + assert res.event_study_vcov is None # existing clear + assert res.event_study_df is not None # provenance KEPT + surface = build_event_study_surface(res) + finite_p = np.isfinite(surface.p_value) + # Unconditional: a regression that NaN-ed every replicate row's + # inference must fail here rather than pass vacuously. + assert finite_p.any() + assert np.isfinite(surface.df[finite_p]).all() + # The surfaced values are the producer's record, which is the (post-drop) + # design df the replicate refit's safe_inference calls received. + expected = float(max(res.survey_metadata.df_survey, 1)) + assert set(surface.df[finite_p].tolist()) == {expected} + for et, df_val in zip(surface.event_time.tolist(), surface.df.tolist()): + if et in res.event_study_df and np.isfinite(res.event_study_df[et]): + assert df_val == res.event_study_df[et] + + +def test_sun_abraham_bootstrap_clears_df(): + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = SunAbraham(n_bootstrap=20, seed=1).fit(_sa_df_panel(), **_sa_kwargs()) + assert res.bootstrap_results is not None + assert res.event_study_df is None + surface = build_event_study_surface(res) + # Bootstrap p-values are FINITE, so only the producer-side clear (not the + # container's NaN-p masking) can prevent a false df from surfacing. + assert np.isfinite(surface.p_value).any() + assert np.isnan(surface.df).all() + + +def _dcdh_df_panel(seed=8): + rng = np.random.default_rng(seed) + rows = [] + for u in range(60): + g = [4, 6, 0][u % 3] + for t in range(1, 9): + y = 1.0 + 0.1 * t + u * 0.01 + (1.0 if g and t >= g else 0.0) + rng.normal(0, 0.3) + rows.append( + { + "unit": u, + "time": t, + "outcome": y, + "treat": 1 if (g and t >= g) else 0, + "w": 1.0, + "strat": u % 4, + "psu_id": u, + } + ) + return pd.DataFrame(rows) + + +def _dcdh_kwargs(): + return dict(outcome="outcome", group="unit", time="time", treatment="treat", L_max=2) + + +def test_dcdh_analytic_df_all_nan(): + from diff_diff.chaisemartin_dhaultfoeuille import ChaisemartinDHaultfoeuille + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = ChaisemartinDHaultfoeuille().fit(_dcdh_df_panel(), **_dcdh_kwargs()) + assert res.event_study_df is None # z-inference, no df + assert np.isnan(build_event_study_surface(res).df).all() + + +def test_dcdh_survey_df_on_surface_covers_placebos(): + from diff_diff.chaisemartin_dhaultfoeuille import ChaisemartinDHaultfoeuille + from diff_diff.survey import SurveyDesign + + design = SurveyDesign(weights="w", weight_type="pweight", strata="strat", psu="psu_id") + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = ChaisemartinDHaultfoeuille().fit( + _dcdh_df_panel(), survey_design=design, **_dcdh_kwargs() + ) + assert res.event_study_df == float(res.survey_metadata.df_survey) + surface = build_event_study_surface(res) + # Placebo rows (negative keys) are merged into the same surface and share + # the ONE design df - that is why a scalar is faithful here. + assert (surface.event_time < 0).any() + finite_p = np.isfinite(surface.p_value) + assert finite_p.any() + assert set(surface.df[finite_p].tolist()) == {float(res.event_study_df)} + assert np.isnan(surface.df[surface.is_reference]).all() + + +def test_dcdh_survey_bootstrap_clears_df(): + # TSL survey + bootstrap is a LIVE mode (only replicate + bootstrap is + # rejected): the df expression evaluates finite while the stored p/CIs + # are bootstrap percentiles, so the clear is load-bearing here. + from diff_diff.chaisemartin_dhaultfoeuille import ChaisemartinDHaultfoeuille + from diff_diff.survey import SurveyDesign + + design = SurveyDesign(weights="w", weight_type="pweight", strata="strat", psu="psu_id") + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = ChaisemartinDHaultfoeuille(n_bootstrap=20, seed=1).fit( + _dcdh_df_panel(), survey_design=design, **_dcdh_kwargs() + ) + assert res.survey_metadata is not None and res.survey_metadata.df_survey is not None + assert res.event_study_df is None + assert np.isnan(build_event_study_surface(res).df).all() + + +def test_lpdid_pooled_df_threaded(): + from diff_diff import LPDiD + + rng = np.random.default_rng(11) + rows = [] + for u in range(40): + g = [5, 7, 0][u % 3] + for t in range(1, 12): + y = 1.0 + 0.05 * t + u * 0.02 + (1.0 if g and t >= g else 0.0) + rng.normal(0, 0.3) + rows.append({"unit": u, "time": t, "outcome": y, "treat": 1 if (g and t >= g) else 0}) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = LPDiD(pre_window=3, post_window=3).fit( + pd.DataFrame(rows), outcome="outcome", unit="unit", time="time", treatment="treat" + ) + assert res.pooled_df is not None and set(res.pooled_df) == {"pre", "post"} + ncl = dict(zip(res.pooled["window"].tolist(), res.pooled["n_clusters"].tolist())) + for window, df_val in res.pooled_df.items(): + if np.isfinite(df_val): + assert df_val == ncl[window] - 1 + assert res.to_dict()["pooled_df"] == res.pooled_df + # Native pooled frame schema is UNCHANGED. + assert "df" not in res.pooled.columns diff --git a/tests/test_sun_abraham.py b/tests/test_sun_abraham.py index fa274e7f..0fb85f67 100644 --- a/tests/test_sun_abraham.py +++ b/tests/test_sun_abraham.py @@ -1531,6 +1531,67 @@ def test_hc2_bm_finite_with_auto_cluster(self): res_hc1 = SunAbraham(vcov_type="hc1").fit(data, **kwargs) assert res.overall_se != res_hc1.overall_se + def test_hc2_bm_nan_contrast_dof_regression_pin(self, monkeypatch): + """NaN BM contrast DOF -> all-NaN row inference + NaN df provenance. + + Regression PIN of existing behavior (no fix in this change): + ``safe_inference`` has guarded non-finite / <= 0 df since PR #620, so + a noise-floor NaN Bell-McCaffrey DOF already yields NaN t/p/CI rather + than a finite t-stat. What is NEW here is the provenance record: the + row's ``event_study_df`` entry must be NaN, honestly documenting that + no usable df governed its (undefined) stored inference. + """ + import diff_diff.linalg as _linalg + + real_helper = _linalg._compute_cr2_bm_contrast_dof + + def _nan_dofs(*args, **kwargs): + out = real_helper(*args, **kwargs) + return np.full_like(np.asarray(out, dtype=float), np.nan) + + monkeypatch.setattr(_linalg, "_compute_cr2_bm_contrast_dof", _nan_dofs) + data = self._panel() + kwargs = dict(outcome="outcome", unit="unit", time="time", first_treat="first_treat") + res = SunAbraham(vcov_type="hc2_bm").fit(data, **kwargs) + + assert res.event_study_df is not None + nan_df_rows = [e for e, v in res.event_study_df.items() if not np.isfinite(v)] + assert nan_df_rows, "monkeypatched NaN DOFs must produce NaN df provenance" + for e in nan_df_rows: + row = res.event_study_effects[e] + # SE stays FINITE here (the NaN comes from the df, not the SE), + # so assert the inference triple directly rather than via + # assert_nan_inference (which requires a non-finite SE). + assert np.isnan(row["t_stat"]) + assert np.isnan(row["p_value"]) + assert np.isnan(row["conf_int"][0]) and np.isnan(row["conf_int"][1]) + + def test_hc2_bm_contrast_dof_helper_failure_falls_back(self): + """Helper-level BM failure warns and falls back to normal theory. + + Distinct from the per-row NaN-DOF case above: when the helper RAISES, + SA leaves the ``_compute_iw_effects`` output in place (finite + normal-theory inference) after warning, and the df provenance records + the baseline (NaN on a non-survey fit) - not a fabricated df. + """ + import diff_diff.linalg as _linalg + + def _boom(*args, **kwargs): + raise ValueError("forced BM helper failure") + + data = self._panel() + kwargs = dict(outcome="outcome", unit="unit", time="time", first_treat="first_treat") + with pytest.MonkeyPatch.context() as mp: + mp.setattr(_linalg, "_compute_cr2_bm_contrast_dof", _boom) + with pytest.warns(UserWarning): + res = SunAbraham(vcov_type="hc2_bm").fit(data, **kwargs) + + finite_p = [e for e, row in res.event_study_effects.items() if np.isfinite(row["p_value"])] + assert finite_p, "helper failure must fall back to finite normal-theory inference" + assert res.event_study_df is not None + for e in finite_p: + assert np.isnan(res.event_study_df[e]), "normal theory records NO df" + def test_hc2_bm_explicit_cluster_works(self): """Explicit cluster= overrides the auto-cluster default; CR2-BM at named cluster.""" data = self._panel() diff --git a/tests/test_survey_dcdh_replicate_psu.py b/tests/test_survey_dcdh_replicate_psu.py index a1a64729..18d19a9b 100644 --- a/tests/test_survey_dcdh_replicate_psu.py +++ b/tests/test_survey_dcdh_replicate_psu.py @@ -1374,3 +1374,75 @@ def reduce_to_one(psi, resolved_arg): for info in res.normalized_effects.values(): assert np.isnan(info["t_stat"]) assert np.isnan(info["p_value"]) + + +class TestPhase1FinalDfSync: + """Phase 1 (``L_max=None``) stores the single event-study row as a VALUE + COPY of the overall inference, taken BEFORE the final-df refresh. The + refresh's per-horizon loops cover only the Phase 2 + ``multi_horizon_inference`` surface, so the copy needs its own mirror - + the same value-copy hazard the placebo mirror in that block documents. + + These pin the INVARIANT (the stored row can never disagree with the + refreshed overall it is a copy of) and the df provenance that reports + it. They are not a reproduction of a specific drift: in every + configuration reachable here the contributing sites agree on n_valid, so + the intermediate and final df coincide and the mirror is a no-op. + """ + + def test_phase1_event_study_row_matches_refreshed_overall(self, base_panel): + from scipy import stats as _stats + + R = 12 + df = _attach_replicate_weights(base_panel, R=R, method="BRR", seed=7) + # Rank deficiency: 2 duplicated pairs -> rank = R - 2. + df["rep1"] = df["rep0"] + df["rep3"] = df["rep2"] + sd = _build_replicate_design(R, "BRR") + with warnings.catch_warnings(): + warnings.filterwarnings("ignore") + res = ChaisemartinDHaultfoeuille(seed=1).fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + survey_design=sd, + ) + assert res.L_max is None, "fixture must exercise the Phase 1 path" + final_df = res.survey_metadata.df_survey + assert final_df is not None + + row = res.event_study_effects[1] + # The stored copy agrees with the refreshed overall inference... + assert row["t_stat"] == res.overall_t_stat + assert row["p_value"] == res.overall_p_value + # ...and that inference is the FINAL df's, not an intermediate one. + t_stat = res.overall_att / res.overall_se + expected_p = 2 * _stats.t.sf(abs(t_stat), df=final_df) + assert row["p_value"] == pytest.approx(expected_p, rel=1e-6) + + def test_phase1_surface_df_matches_refreshed_survey_df(self, base_panel): + from diff_diff.results_base import build_event_study_surface + + R = 12 + df = _attach_replicate_weights(base_panel, R=R, method="BRR", seed=7) + df["rep1"] = df["rep0"] + df["rep3"] = df["rep2"] + sd = _build_replicate_design(R, "BRR") + with warnings.catch_warnings(): + warnings.filterwarnings("ignore") + res = ChaisemartinDHaultfoeuille(seed=1).fit( + df, + outcome="outcome", + group="group", + time="period", + treatment="treatment", + survey_design=sd, + ) + final_df = res.survey_metadata.df_survey + assert res.event_study_df == float(final_df) + surface = build_event_study_surface(res) + finite_p = np.isfinite(surface.p_value) + assert finite_p.any() + assert set(surface.df[finite_p].tolist()) == {float(final_df)}