Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
metacharacter — so no path composed into a shell command can execute. `persist`
self-cleans the whole invocation on any failure (no post-persist cleanup
needed) and `abort` is strict (a missing state token is an error).
- **df provenance completed for every event-study producer that exposes one
(4.0 program Phase 2, ledger row M-092 follow-up).** `SunAbrahamResults` gains
`event_study_df` (a per-relative-time dict: the per-event Bell-McCaffrey contrast df
under `hc2_bm`, the survey design df - post-drop under replicate refits - on survey
fits, NaN where inference is normal-theory or its BM DOF was non-finite), and
`ChaisemartinDHaultfoeuilleResults` gains a scalar `event_study_df` (one value, since
its effect and placebo rows are computed from the same design df and refreshed
together under replicate weights). Both clear under bootstrap, whose percentile
p-values never used a df; SunAbraham's clear is deliberately narrower than its
`event_study_vcov` clear, since a replicate refit's rows genuinely used the replicate
df. `LPDiDResults` gains `pooled_df` (`{"pre": ..., "post": ...}`), the headline
pooled-window analogue of `DiDResults.inference_df`, surfaced in `to_dict()`; the
native pooled frame schema is unchanged. With these, `EventStudyResults.df` is
populated for every producer whose inference records a df.
- **Docs: navigation and SEO polish.** A stable/latest version switcher in the navbar
(`docs/_static/switcher.json`); a custom `robots.txt` that keeps the ~60 thin
`_modules/` source-view pages out of crawlers (documentation pages allowed as
Expand Down Expand Up @@ -123,6 +137,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
**Estimator equations, weighting, variance, and numerical output are unchanged.**

### Changed
- **dCDH Phase 1 (`L_max=None`) event-study row is now re-synced by the final-df
refresh.** Under replicate-weight designs the refresh recomputes `overall_*` with the
final effective df, but the Phase 1 event-study row is a VALUE COPY of that inference
taken beforehand, and the refresh's per-horizon loops cover only the Phase 2 surface -
so the copy could in principle retain an intermediate df's t/p/CI while
`survey_metadata.df_survey` reported the final one. The copy is now mirrored from the
refreshed values, making the stored row agree with the overall estimate it duplicates
by construction. No output change is reproducible on any configuration exercised in
the test suite (the contributing sites agree on their valid-replicate counts, so the
intermediate and final df coincide and the mirror is a no-op); this closes the
structural hazard rather than a demonstrated defect.
- **DiagnosticReport's parallel-trends check upgrades from the Bonferroni fallback to
the joint Wald test for StackedDiD and TwoStageDiD event-study fits.** The check's
joint-Wald path runs whenever the source fit exposes `event_study_vcov`; now that
Expand Down
1 change: 0 additions & 1 deletion TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ Related tracking surfaces:
|-------|----------|--------|--------|----------|
| `SyntheticControl` conformal (CWZ 2021) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies — the residual-permutation shortcut is only valid for time-permutation-invariant proxies (SC/Lasso/DiD); an AR proxy needs innovation permutation. | `diff_diff/conformal.py`, `diff_diff/synthetic_control_results.py` | CWZ-2021 | Heavy | Low |
| `ContinuousDiD` CGBS-2024 remaining extensions (earlier phases — `covariates=` reg/dr, `treatment_type="discrete"`, single-cohort `control_group="lowest_dose"` with estimand `ATT(d)−ATT(d_L)` — are already supported; see REGISTRY Note #7). Remaining (all deferred `NotImplementedError`, documented): `estimation_method="ipw"` on the dose curve (scalar-adjustment / degenerate); `covariates=` × `survey_design=` (weighted OR + weighted nuisance IF); multi-cohort **heterogeneous-support** discrete aggregation (support-aware: average each dose only over the cohorts that observe it); **multi-cohort `lowest_dose`** (within-cohort `d_L` reference + support-aware cross-cohort aggregation); and **`covariates=` × `lowest_dose`** (conditional-PT-relative-to-`d_L` estimand). Single-cohort / 2-period / shared-support multi-cohort are supported. | `continuous_did.py` | CGBS-2024 | Heavy | Low |
| `event_study_df` provenance for the remaining producers (4.0 program Phase 2 follow-up; the Stacked/TwoStage vcov + CS/MPD/Stacked/TwoStage/LPDiD df round-trip landed with the per-row `EventStudyResults.df` schema). Remaining: SunAbraham (per-event BM Satterthwaite df under `hc2_bm` at `sun_abraham.py:1206`, scalar survey df at `:1256` - mechanism identical to StackedDiD's recorded-at-use dict); dCDH (its event-study inference receives a `df_inference` argument, `chaisemartin_dhaultfoeuille.py:6311`, never stored - needs tracing to the stored rows); LPDiD pooled-window df (popped and discarded at fit assembly). All purely additive: their surfaces show df=NaN ("unexposed") until threaded. | `diff_diff/sun_abraham.py`, `diff_diff/chaisemartin_dhaultfoeuille.py`, `diff_diff/lpdid.py` | #M-092 | Quick | Low |

### Performance

Expand Down
40 changes: 39 additions & 1 deletion diff_diff/chaisemartin_dhaultfoeuille.py
Original file line number Diff line number Diff line change
Expand Up @@ -4039,8 +4039,11 @@ def fit(
# and HonestDiD. Re-run safe_inference with the FINAL
# effective df so every surface agrees.
_final_eff_df = _effective_df_survey(resolved_survey, _replicate_n_valid_list)
# ONE canonical final inference df, shared by the refresh below and
# by the `event_study_df` provenance recorded at construction, so the
# two can never drift apart.
_final_inf_df = _inference_df(_final_eff_df, resolved_survey)
if _replicate_n_valid_list:
_final_inf_df = _inference_df(_final_eff_df, resolved_survey)
# Recompute `effective_overall_*` directly — that's what
# ships in results at line ~2776+. `effective_overall_att/se`
# may differ from the raw `overall_att/se` under the delta
Expand Down Expand Up @@ -4087,6 +4090,22 @@ def fit(
_info_r2["t_stat"] = _t_r2
_info_r2["p_value"] = _p_r2
_info_r2["conf_int"] = _ci_r2
else:
# Phase 1 (L_max=None): `event_study_effects[1]` is a VALUE
# COPY of the overall inference, built above from the
# PRE-refresh `overall_*`; the loop just above covers only
# the Phase 2 `multi_horizon_inference` surface. Without
# this mirror the stored l=1 row keeps the INTERMEDIATE
# df's t/p/CI while `overall_*` and
# `survey_metadata.df_survey` report the final (tightened)
# df - the same value-copy hazard the placebo mirror below
# documents. Re-sync from the refreshed `overall_*` so the
# single event-study row agrees with the overall estimate it
# is a copy of.
if event_study_effects and 1 in event_study_effects:
event_study_effects[1]["t_stat"] = overall_t
event_study_effects[1]["p_value"] = overall_p
event_study_effects[1]["conf_int"] = overall_ci
if placebo_horizon_inference is not None:
for _lag_r2, _info_r2 in list(placebo_horizon_inference.items()):
_t_r2, _p_r2, _ci_r2 = safe_inference(
Expand Down Expand Up @@ -4181,6 +4200,24 @@ def fit(
if survey_metadata is not None:
survey_metadata.df_survey = _final_eff_df

# df PROVENANCE (spec section 5, row M-092): the scalar df every
# stored event-study / placebo row's safe_inference received.
# Reuses `_final_inf_df` - the SAME value the refresh above applied
# to every stored row - so provenance and inference cannot drift.
# Recorded iff finite and > 0.
# Cleared under bootstrap, whose percentile p/CIs never used a df:
# required, not defensive, because TSL survey + bootstrap is a live
# mode (only REPLICATE + bootstrap is rejected) where this
# expression evaluates finite.
_es_df_provenance: Optional[float] = None
if (
bootstrap_results is None
and _final_inf_df is not None
and np.isfinite(_final_inf_df)
and _final_inf_df > 0
):
_es_df_provenance = float(_final_inf_df)

results = ChaisemartinDHaultfoeuilleResults(
overall_att=effective_overall_att,
overall_se=effective_overall_se,
Expand Down Expand Up @@ -4226,6 +4263,7 @@ def fit(
event_study_effects=event_study_effects,
L_max=L_max,
placebo_event_study=placebo_event_study_dict,
event_study_df=_es_df_provenance,
twfe_weights=twfe_weights_df,
twfe_fraction_negative=twfe_fraction_negative,
twfe_sigma_fe=twfe_sigma_fe,
Expand Down
24 changes: 24 additions & 0 deletions diff_diff/chaisemartin_dhaultfoeuille_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,15 @@ class ChaisemartinDHaultfoeuilleResults(BaseResults):
``1..L_max`` when ``L_max >= 1``. When ``L_max >= 1``, uses the
per-group ``DID_{g,l}`` path; when ``L_max=None``, uses the
per-period ``DID_M`` path.
event_study_df : float, optional
Inference degrees-of-freedom PROVENANCE for the event-study and
placebo surfaces: the single df every stored row's
``safe_inference`` received. One scalar suffices because both
surfaces are computed from the same design df (and are refreshed
together to the final effective df under replicate weights).
``None`` when the rows used normal theory (no survey design), when
the df was undefined, or when bootstrap overrode the stored
inference with percentile values that never used a df.
normalized_effects : dict, optional
Normalized estimator ``DID^n_l``. Populated when ``L_max >= 1``.
cost_benefit_delta : dict, optional
Expand Down Expand Up @@ -656,6 +665,21 @@ class ChaisemartinDHaultfoeuilleResults(BaseResults):
bootstrap_results: Optional[DCDHBootstrapResults] = field(default=None, repr=False)
_estimator_ref: Optional[Any] = field(default=None, repr=False)

# event_study_df (spec section 5, row M-092): the ONE inference df every
# stored event-study AND placebo row's ``safe_inference`` actually
# received - a single scalar because both surfaces are computed from the
# same design df (and, under replicate weights, are re-run together to
# the final effective df). None means the rows used normal theory (no
# survey design), the df was undefined (the df<=0 replicate sentinel,
# which yields NaN inference), or bootstrap overrode the stored p/CIs
# with percentile values that never used a df. Note the bootstrap clear
# drops the whole channel even when a partial override leaves some rows
# analytic - a conservative under-claim, consistent with the other
# producers.
# Declared LAST so every pre-existing field keeps its positional index
# in the generated __init__ (the constructor signature is public API).
event_study_df: Optional[float] = field(default=None, repr=False)

# ------------------------------------------------------------------
# Repr / properties
# ------------------------------------------------------------------
Expand Down
18 changes: 13 additions & 5 deletions diff_diff/lpdid.py
Original file line number Diff line number Diff line change
Expand Up @@ -1460,6 +1460,7 @@ def fit(
pooled = None

event_study_df: Optional[Dict[int, float]] = None
pooled_df: Optional[Dict[str, float]] = None
if not only_pooled:
event_rows = []
event_study_df = {}
Expand Down Expand Up @@ -1528,11 +1529,17 @@ def fit(
absorb=absorb,
survey_design=survey_design,
)
# Discard the pooled windows' df provenance (native pooled frame
# schema unchanged; pooled-window df threading is a tracked
# follow-up in TODO.md).
pre_estimate.pop("df", None)
post_estimate.pop("df", None)
# Capture the pooled windows' df provenance BEFORE building the
# frame so the native pooled schema stays unchanged (it feeds
# summary()'s table). Each value is the df that window's
# safe_inference actually received; NaN when it used normal
# theory or the df was undefined.
_pre_df = pre_estimate.pop("df", None)
_post_df = post_estimate.pop("df", None)
pooled_df = {
"pre": float(_pre_df) if _pre_df is not None else float("nan"),
"post": float(_post_df) if _post_df is not None else float("nan"),
}
pooled = pd.DataFrame(
[
{
Expand Down Expand Up @@ -1566,6 +1573,7 @@ def fit(
event_study=event_study,
pooled=pooled,
event_study_df=event_study_df,
pooled_df=pooled_df,
n_obs=len(data),
n_treated_units=int(treatment_by_unit.gt(0).sum()),
n_control_units=int(treatment_by_unit.eq(0).sum()),
Expand Down
17 changes: 17 additions & 0 deletions diff_diff/lpdid_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ class LPDiDResults(BaseResults):
``control_group="clean"`` the realized control pool at each horizon also
includes not-yet-treated cohorts, whose per-horizon counts live in the
``n_obs`` / ``n_clusters`` columns of the tables.

``event_study_df`` and ``pooled_df`` carry inference degrees-of-freedom
PROVENANCE - the df each row's ``safe_inference`` actually received -
keyed by horizon and by ``"pre"``/``"post"`` window respectively. Both
are NaN per entry where that row used normal theory or an undefined df.
They are metadata only: the native tables above are unchanged, and
``event_study_df`` is what the unified event-study surface reads.
"""

event_study: Optional[pd.DataFrame]
Expand Down Expand Up @@ -63,6 +70,14 @@ class LPDiDResults(BaseResults):
# of which the native tables record. The synthetic ``horizon == -1``
# base row has no entry. None when no event study was fit.
event_study_df: Optional[Dict[int, float]] = None
# pooled_df: the same df PROVENANCE for the POOLED windows, keyed
# ``"pre"`` / ``"post"`` to match the ``window`` column. The pooled-post
# row is the headline ATT, which lives outside the unified event-study
# surface, so this is standalone headline provenance (the
# ``DiDResults.inference_df`` analogue) rather than a surface input. NaN
# per window when that window's inference used normal theory or an
# undefined df; None when no pooled windows were fit (``only_event``).
pooled_df: Optional[Dict[str, float]] = None

# ------------------------------------------------------------------
# internal helpers
Expand Down Expand Up @@ -164,6 +179,8 @@ def to_dict(self) -> Dict[str, Any]:
result["cluster_name"] = self.cluster_name
if self.n_clusters is not None:
result["n_clusters"] = self.n_clusters
if self.pooled_df is not None:
result["pooled_df"] = self.pooled_df
if self.non_absorbing is not None:
result["non_absorbing"] = self.non_absorbing
result["stabilization_window"] = self.stabilization_window
Expand Down
14 changes: 11 additions & 3 deletions diff_diff/results_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -214,9 +214,11 @@ class EventStudyResults(BaseResults):
from the producer. Accepts None (no df exposed -> all-NaN column), a
scalar (broadcast to every row - e.g. CallawaySantAnna, whose
explicit-survey event study applies ONE conservative df, the minimum
per-horizon effective df, to all rows), or a length-n array (per-row
producers: StackedDiD ``hc2_bm`` per-event Bell-McCaffrey df, LPDiD
per-horizon cluster df, MultiPeriodDiD ``hc2_bm`` per-period df).
per-horizon effective df, to all rows; or de
Chaisemartin-D'Haultfoeuille, whose effect and placebo rows share one
design df), or a length-n array (per-row producers: StackedDiD and
SunAbraham ``hc2_bm`` per-event Bell-McCaffrey df, LPDiD per-horizon
cluster df, MultiPeriodDiD ``hc2_bm`` per-period df).
NaN on any row means normal-theory inference, an undefined df,
bootstrap-overridden inference, or a producer that records none;
reference rows and rows with NaN p-values are always NaN.
Expand Down Expand Up @@ -949,6 +951,12 @@ def _from_dcdh(results: Any) -> EventStudyResults:
cband_crit_value=cband_crit,
alpha=getattr(results, "alpha", 0.05),
source=type(results).__name__,
# ONE scalar df for both merged surfaces: the effect rows and the
# placebo rows are computed from the same design df (and, under
# replicate weights, refreshed together to the final effective df),
# so a scalar is faithful - the container broadcasts it and NaN-masks
# the synthesized reference row at 0 plus any NaN-p rows.
df=getattr(results, "event_study_df", None),
)


Expand Down
7 changes: 4 additions & 3 deletions diff_diff/stacked_did.py
Original file line number Diff line number Diff line change
Expand Up @@ -840,9 +840,10 @@ def _refit_stacked(w_r):
# failed OR noise-floor NaN guard fired), emit all-NaN inference
# rather than fall back to normal-theory CIs/p-values. Mirrors
# the fix in LinearRegression.get_inference() from PR #475 R7
# (linalg.py:3689-3706). Without this, safe_inference(df=NaN)
# would pass df comparison >= 0 (NaN < 0 is False) and emit
# finite t_stat with NaN p/CI — silent wrong inference.
# (linalg.py:3689-3706). `safe_inference` itself has guarded
# non-finite / <= 0 df since PR #620 (utils.py) with the same
# all-NaN result; this explicit branch predates that guard and
# is kept for explicitness (it also skips the call entirely).
_is_hc2bm_path = self.vcov_type == "hc2_bm" and not _uses_replicate_sd
_bm_df = _bm_contrast_dof_per_event.get(h)
if _is_hc2bm_path and (_bm_df is None or not np.isfinite(_bm_df)):
Expand Down
Loading
Loading