diff --git a/CHANGELOG.md b/CHANGELOG.md index 1bce6ae0..b216cd04 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,14 +65,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 in every post cell: `ATT(3,8) = -0.0096`, `ATT(5,9) = -0.2393`, and `overall_att = 1.0023`, with **no warning** under `rank_deficient_action="silent"`. - **This is a capability regression, taken deliberately.** Panels with no - never-treated group — where periods in which every unit is treated have no - eligible comparison — previously produced those wrong numbers and now fail - closed, including under `survey_design=` and `cohort_trends=True`. Wooldridge - (2025) Section 5.4 defines the correct treatment (the last cohort serves as - the control, its columns dropped deliberately rather than by QR); the library - already applies that to the cohort-trend column but not to the cells, and - implementing the cell half is tracked to restore these fits. + Panels with no never-treated group — where periods in which every unit is + treated have no eligible comparison — previously produced those wrong numbers. + They briefly failed closed, and now **estimate correctly** via the + comparison-support filter below; the gate remains as the backstop for what + that filter cannot see. +- **`WooldridgeDiD` estimates all-eventually-treated panels again, implementing + Wooldridge (2025) Section 5.4 for the cell and cohort-trend families.** With + no never-treated group the last cohort serves as the reference, so periods at + which no unit is untreated carry no identified `ATT(g, t)` and are removed + from the estimation sample before the solve. Measured on a `{3, 5, 8}` panel + over `t = 1..9` with a true ATT of 1.5: cells `(3, 3..7)` and `(5, 5..7)`, + `overall_att = 1.5502`, cohort 8 correctly receiving none. Anchored to Stata + `jwdid` on the `mpdta` panel with never-treated counties dropped — identical + cell set, identical `N` (764 of 955), ATTs agreeing to ~1e-15. + **Nothing is dropped silently.** Every fit that reduces the sample reports the + periods, the observation count and the cause, and separately names any cohort + left without cells or stripped of every row; neither warning is gated on + `rank_deficient_action`. On the `never_treated` OLS branch a filtered period + can legitimately move a cohort's reference, renormalizing that cohort's ATTs + — this warns by name rather than passing silently. (Stata `jwdid` performs the + same reduction and reports only a smaller `N`.) + `cohort_trends=True` works on these panels for the first time, surfacing + `G - 1` trend slopes with the last cohort absent as the baseline. Under + `survey_design=` the combination is refused rather than subsetted, for the + same reason as the unidentified-cohort refusal below. + On an **unbalanced** panel a unit observed only at unsupported periods is + removed with them, which shrinks the cohort-share weight `N_g` (Eqs. 7.4/7.6) + below the cohort actually supplied — measured at a 2x swing in + `aggregate(weights="cohort_share")`. The paper defines `N_g` on a balanced + panel and does not settle which reading applies, so that aggregation now + **fails closed** naming the affected cohorts and unit counts instead of + choosing one silently. `weights="cell"` (the default) never reads `N_g`, and + balanced panels drop whole periods and no units, so neither is affected. + Not included: the `D_{G_max} × X` covariate normalization, so *some* covariate + specifications on such a panel remain rank-deficient — `exovar` and `xgvar`, + and `xtvar` under `demean_covariates=False`. Default `xtvar` (with + `demean_covariates=True`) is **full rank** and fits cleanly even under + `rank_deficient_action="error"`. Where the deficiency does bite, coefficients + are unaffected but `"error"` raises. Tracked in `TODO.md`. - **`WooldridgeDiD` refuses `survey_design=` combined with an unidentified cohort** (`NotImplementedError`, all three methods). Excluding a cohort under a complex survey design is domain estimation, which this path does not diff --git a/DEFERRED.md b/DEFERRED.md index 78f6ed75..19827426 100644 --- a/DEFERRED.md +++ b/DEFERRED.md @@ -46,7 +46,7 @@ exists but parity can't be verified without a local toolchain. | `StaggeredTripleDifference` R cross-validation: CSV fixtures not committed (gitignored); tests skip without local R + `triplediff`. Commit fixtures or generate deterministically. | `tests/test_methodology_staggered_triple_diff.py` | #245 | Medium | | `StaggeredTripleDifference` R parity: benchmark only tests the no-covariate path (`xformla=~1`). Add covariate-adjusted scenarios + aggregation-SE parity assertions. | `benchmarks/R/benchmark_staggered_triplediff.R` | #245 | Medium | | `StaggeredTripleDifference` per-cohort group-effect SEs include WIF (conservative vs R's `wif=NULL`); documented in REGISTRY. Could override the mixin for an exact R match (verification needs R `triplediff`). | `staggered_triple_diff.py` | #245 | Low | -| **WooldridgeDiD follow-up cluster** (PR-B Stage D/E fail-closed surfaces; re-enable after R/Stata validation):
• QMLE sandwich uses `aweight` cluster adjustment `(G/(G-1))·(n-1)/(n-k)` vs Stata's `G/(G-1)` (conservative); add a `qmle` weight type if Stata goldens confirm a material difference (`wooldridge.py`, `linalg.py`).
• response-scale APE / log-link coefficient bridge for R `etwfe(family=poisson|logit)` cell-level parity — needs `emfx()` APE extraction or link-inversion with baseline-mean adjustment (`generate_wooldridge_golden.R`, `test_methodology_wooldridge.py`).
• `aggregate(weights="cohort_share")` on survey-weighted fits: `_n_g_per_cohort` uses raw `unit.nunique()`; implement design-weighted unit totals per cohort (paper W2025 §7) and lift the `ValueError` gate (`wooldridge.py`, `wooldridge_results.py`).
• unconditional inference for `cohort_share` accounting for ω̂_g sampling uncertainty (W2025 §7.5); currently NaN-closed (`wooldridge_results.py`).
• `cohort_trends=True × survey_design` and `× control_group="never_treated"` raise `NotImplementedError` (unvalidated TSL variance / trend columns spanned jointly by the placebo cells and the unit FE, which absorb the cohort indicator and recover the omitted reference) (`wooldridge.py`).
• Stata `jwdid` golden-value `TestReferenceValues` (`tests/test_wooldridge.py`). | `wooldridge.py`, `wooldridge_results.py`, `linalg.py`, benchmarks | #216 · PR-B | Med-Low | +| **WooldridgeDiD follow-up cluster** (PR-B Stage D/E fail-closed surfaces; re-enable after R/Stata validation):
• QMLE sandwich uses `aweight` cluster adjustment `(G/(G-1))·(n-1)/(n-k)` vs Stata's `G/(G-1)` (conservative); add a `qmle` weight type if Stata goldens confirm a material difference (`wooldridge.py`, `linalg.py`).
• response-scale APE / log-link coefficient bridge for R `etwfe(family=poisson|logit)` cell-level parity — needs `emfx()` APE extraction or link-inversion with baseline-mean adjustment (`generate_wooldridge_golden.R`, `test_methodology_wooldridge.py`).
• `aggregate(weights="cohort_share")` on survey-weighted fits: `_n_g_per_cohort` uses raw `unit.nunique()`; implement design-weighted unit totals per cohort (paper W2025 §7) and lift the `ValueError` gate (`wooldridge.py`, `wooldridge_results.py`).
• unconditional inference for `cohort_share` accounting for ω̂_g sampling uncertainty (W2025 §7.5); currently NaN-closed (`wooldridge_results.py`).
• `cohort_trends=True × survey_design` and `× control_group="never_treated"` raise `NotImplementedError` (unvalidated TSL variance / trend columns spanned jointly by the placebo cells and the unit FE, which absorb the cohort indicator and recover the omitted reference) (`wooldridge.py`).
• ~~Stata `jwdid` golden-value `TestReferenceValues`~~ RESOLVED: the golden ships four arms pinned by `tests/test_etwfe_cs_stata_parity.py` (no `TestReferenceValues` symbol was ever added). The QMLE bullet above REMAINS OPEN -- every golden arm is linear `jwdid`, so no QMLE cluster-SE reference exists, and SEs are pinned only as a ratio. | `wooldridge.py`, `wooldridge_results.py`, `linalg.py`, benchmarks | #216 · PR-B | Med-Low | | Extend `WooldridgeDiD` `method ∈ {logit, poisson}` with `vcov_type ∈ {classical, hc2, hc2_bm}`: composing HC2 leverage + Bell-McCaffrey DOF with the QMLE pseudo-residual sandwich needs derivation + R parity vs `clubSandwich::vcovCR(glm, type="CR2")`. Rejected at `__init__`. | `wooldridge.py` | follow-up | Medium | | Multi-constraint CR2 parallel-trends test (AHT/HTZ) for `hc2_bm` fits: DiagnosticReport's PT check routes `vcov_type="hc2_bm"` sources to Bonferroni over the BM-adjusted per-row p-values because the generic chi-square joint Wald would discard the CR2 small-sample correction (see REPORTING.md "hc2_bm parallel-trends policy"). The proper joint test is the AHT/HTZ Wald with a Satterthwaite-style denominator df over the pre-period contrast block; needs derivation for the stacked/pooled WLS-CR2 layout + parity vs `clubSandwich::Wald_test(..., test="HTZ")`. | `diagnostic_report.py`, `linalg.py` | vcov/df round-trip PR | Low | | `PreTrendsPower` CS/SA `anticipation=1` R-parity fixture: R `pretrends` has no anticipation parameter, so the Python `_extract_pre_period_params` anticipation filter isn't R-parity-locked. Build a synthetic CS/SA result with `anticipation=1` and assert γ_p matches R's `slope_for_power()`. (Mechanism already covered by MC + full-VCV tests.) | `tests/test_methodology_pretrends.py`, `generate_pretrends_golden.R` | PR-C | Low | diff --git a/METHODOLOGY_REVIEW.md b/METHODOLOGY_REVIEW.md index e441f6f6..e5147f8e 100644 --- a/METHODOLOGY_REVIEW.md +++ b/METHODOLOGY_REVIEW.md @@ -650,7 +650,7 @@ and covariate-adjusted specifications.) **Deviations from the paper / from R / library extensions:** See REGISTRY.md `## WooldridgeDiD (ETWFE)` → `### Deviations from the paper / from R / library extensions` block for the consolidated list (HC1 finite-sample factor, QMLE sandwich `(n-1)/(n-k)` term, nonlinear-vs-fixest direct QMLE, logit cohort+time additive dummies, anticipation + aggregation, cell-count default with opt-in cohort-share). **Outstanding Concerns:** -- **Stata `jwdid` golden values** (DEFERRED.md WooldridgeDiD follow-up cluster row): Stata-side parity infrastructure deferred until Stata install is available; R `etwfe` side covered in PR-B Stage D. +- **Stata `jwdid` golden values**: RESOLVED (#729, extended here). `benchmarks/data/etwfe_cs_stata_golden.json` ships four arms -- `csdid`, `jwdid`, `jwdid_never` (the issue #724 reference-period anchor) and `jwdid_alltreated` (the W2025 Section 5.4 anchor) -- generated with local StataSE 19 and pinned by `tests/test_etwfe_cs_stata_parity.py`. Point estimates match to ~1e-15. What remains open is the `hc1` SE gap (library SEs uniformly SMALLER than jwdid's by a cluster-count-dependent factor: 1.0280@G=20, 1.0132@G=40, 1.00264@G=191, 1.0010@G=500), tracked in TODO.md; each arm pins its own measured ratio because the constant does not transfer between cluster counts. No QMLE (logit/Poisson) golden exists yet -- every arm is linear `jwdid` -- so the QMLE cluster-SE comparison in DEFERRED.md stays open. R `etwfe` side covered in PR-B Stage D. - **Response-scale APE / log-link bridge for Poisson + logit R parity** (DEFERRED.md WooldridgeDiD follow-up cluster row, added in PR-B): direct cell-level numerical parity between diff-diff's response-scale ATT and R `etwfe` log-link coefficients requires either `emfx()`-based APE extraction on the R side or link-function inversion with baseline-mean adjustment. - **QMLE sandwich Stata-parity `qmle` weight type** (DEFERRED.md WooldridgeDiD follow-up cluster row): diff-diff's `(G/(G-1)) × ((n-1)/(n-k))` is conservative vs Stata's `G/(G-1)` only; awaiting Stata golden values to confirm material difference. - **Repeated cross-sections** (paper p. 2581 → Deb et al. 2024): not in 2025 paper's main body; future PR. diff --git a/TODO.md b/TODO.md index caf3cd2b..6621adf7 100644 --- a/TODO.md +++ b/TODO.md @@ -23,18 +23,21 @@ Related tracking surfaces: |-------|----------|--------|--------|----------| | `SyntheticControl` conformal (CWZ 2021) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies — the residual-permutation shortcut is only valid for time-permutation-invariant proxies (SC/Lasso/DiD); an AR proxy needs innovation permutation. | `diff_diff/conformal.py`, `diff_diff/synthetic_control_results.py` | CWZ-2021 | Heavy | Low | | Make the post-fit `results.aggregate("event_study")` container consumable downstream. `EventStudyResults` is rejected by all THREE consumers that read a CS event study — `compute_honest_did` (`honest_did.py`, dispatches on `CallawaySantAnnaResults` and raises `TypeError`), `compute_pretrends_power` (`pretrends.py`, same), and `plot_event_study` (`visualization`, same) — so `fit(aggregate="event_study")` is still the only route for them and their error messages say so explicitly. Needs an `EventStudyResults` branch in each extraction path (consuming `event_time` / `is_reference` / `vcov` / `vcov_index` / per-row `df`) PLUS `base_period` and `anticipation` provenance, which the unified container does not carry and HonestDiD needs for its universal-base-period warning and pre-period classification. Gate with end-to-end tests: `compute_honest_did(res.aggregate("event_study"))` at `base_period="universal"`, and `compute_pretrends_power(...)` at `anticipation=1`. | `diff_diff/honest_did.py`, `diff_diff/pretrends.py`, `diff_diff/results_base.py` | #726 | Mid | Medium | -| Derive and fix the `WooldridgeDiD` `hc1` SE gap vs Stata `jwdid`. Measured: every SE is uniformly SMALLER than `jwdid`'s, by 1.0280 at G=20 / 1.0132 at G=40 / 1.0010 at G=500 (ATT(g,t) points match exactly). The gap tracks `sqrt(G/(G-1))` but sits consistently above it, and `solve_ols` already applies the full CR1 `(G/(G-1))*((n-1)/(n-k))`, so a missing cluster factor is ruled out -- the likely source is the within-transform `k` accounting vs `hdfe`/`reghdfe`'s. Derive the exact factor FIRST; a `sqrt(G/(G-1))` patch would match only approximately and would force a loose tolerance on the parity test, defeating its purpose. Move the REGISTRY note and the pinned ratio in `tests/test_etwfe_cs_stata_parity.py` together with the fix. **Required artifact:** a committed subsample LADDER in the Stata golden (rosters = first N units per `first_treat` by sorted `countyreal`; rungs spanning G≈20..500, storing Stata `G`/`n`/`df_a`/`rank`/`df_r` and per-cell `att`/`se`), with parameterized ratio assertions. Only the G=500 ratio is pinned today, so the few-cluster behavior — where the gap is materially largest (~2.8%) — is ungated, and the ladder is also the instrument for comparing Stata's `df_a`/`rank` against the library's within-transform `k`, which is the leading hypothesis for the factor. | `diff_diff/wooldridge.py`, `diff_diff/linalg.py` | #723-followup | Mid | Medium | +| Derive and fix the `WooldridgeDiD` `hc1` SE gap vs Stata `jwdid`. Measured: every SE is uniformly SMALLER than `jwdid`'s, by 1.0280 at G=20 / 1.0132 at G=40 / 1.00264 at G=191 / 1.0010 at G=500 (ATT(g,t) points match exactly). The gap tracks `sqrt(G/(G-1))` but sits consistently above it, and `solve_ols` already applies the full CR1 `(G/(G-1))*((n-1)/(n-k))`, so a missing cluster factor is ruled out -- the likely source is the within-transform `k` accounting vs `hdfe`/`reghdfe`'s. Derive the exact factor FIRST; a `sqrt(G/(G-1))` patch would match only approximately and would force a loose tolerance on the parity test, defeating its purpose. Move the REGISTRY note and the pinned ratio in `tests/test_etwfe_cs_stata_parity.py` together with the fix. **Required artifact:** a committed subsample LADDER in the Stata golden (rosters = first N units per `first_treat` by sorted `countyreal`; rungs spanning G≈20..500, storing Stata `G`/`n`/`df_a`/`rank`/`df_r` and per-cell `att`/`se`), with parameterized ratio assertions. Only G=500 (full panel) and G=191 (all-eventually-treated arm) are pinned today, so the few-cluster behavior — where the gap is materially largest (~2.8%) — is ungated, and the ladder is also the instrument for comparing Stata's `df_a`/`rank` against the library's within-transform `k`, which is the leading hypothesis for the factor. | `diff_diff/wooldridge.py`, `diff_diff/linalg.py` | #723-followup | Mid | Medium | | `SunAbraham`: a cohort not observed at its own reference relative period (`e = -1 - anticipation`) makes that cohort's block collinear, so QR drops an unnamed column (`dropping 1 of 12 columns (column 9)`) and `overall_att` comes back **NaN**. Found by auditing the sibling estimator while fixing the ETWFE analogue (#724); PRE-EXISTING, not introduced there. Lower severity than #724 — that returned a silently WRONG finite number, this returns NaN with a rank warning — but the event-study surface still looks complete, so a user may not notice the loss. SA already omits its reference explicitly and tracks `_reference_observed`, so the fix is per-cohort support for that flag rather than the ETWFE-style redesign. | `diff_diff/sun_abraham.py` | #724-audit | Mid | Low | -| `WooldridgeDiD` + `survey_design=` does not support DOMAIN ESTIMATION, so unidentified-cohort exclusion is currently REFUSED (`NotImplementedError`, all three methods) rather than performed. Implementing it properly means zero-padding the excluded rows' weights while retaining strata/PSU/FPC, per REGISTRY *Subpopulation Analysis (Phase 6)* / Lumley (2004) 3.4, so TSL variance and `df_survey = n_PSU - n_strata` use the full design (naive deletion measured 22 -> 14 on a two-stratum panel). `SurveyDesign.subpopulation()` already implements the contract and SpilloverDiD Wave E.3 is the in-repo precedent; the blocker is that the weighted within-transform rejects zero-weight units, shared machinery behind 7 estimators. Landing it would turn the refusal back into a supported fit. Gate with a `SurveyDesign.subpopulation()` parity test on ATT, TSL SE and survey df where the excluded cohort exhausts a PSU. | `diff_diff/wooldridge.py`, `diff_diff/utils.py` | #724-codex-R4/R5 | Heavy | Medium | +| Define `N_g` (W2025 Eqs. 7.4/7.6) for UNBALANCED panels where comparison-support filtering removes every observation of some units in an estimated cohort, then replace the fail-closed guard with the defined behavior. `_n_g_per_cohort` is read off the final sample, so those units vanish from the cohort-share weights; measured on a cohort supplied with 100 units of which 90 appear only at a dropped period, `aggregate(weights="cohort_share")` moves 1.8078 -> 3.8157. The paper assumes a balanced panel and does not say whether `N_g` counts the supplied cohort or the surviving units, and the two disagree materially, so `aggregate` currently raises naming the cohorts and counts ([M-125]); `weights="cell"` is unaffected and balanced panels never trip it. Settle the estimand (likely: count the supplied cohort, since ATT(g,t) is a cohort-level quantity, but that weights units with no retained observation) and gate with a test computing Eq. 7.4 by hand on unequal cohort sizes. | `diff_diff/wooldridge_results.py`, `diff_diff/wooldridge.py` | #729-followup | Mid | Medium | +| `WooldridgeDiD` + `survey_design=` does not support DOMAIN ESTIMATION, so BOTH row-deleting paths are currently REFUSED (`NotImplementedError`, all three methods) rather than performed: unidentified-cohort exclusion ([M-123]) and comparison-support period filtering ([M-125]). One fix unblocks both. Implementing it properly means zero-padding the excluded rows' weights while retaining strata/PSU/FPC, per REGISTRY *Subpopulation Analysis (Phase 6)* / Lumley (2004) 3.4, so TSL variance and `df_survey = n_PSU - n_strata` use the full design (naive deletion measured 22 -> 14 on a two-stratum panel). `SurveyDesign.subpopulation()` already implements the contract and SpilloverDiD Wave E.3 is the in-repo precedent; the blocker is that the weighted within-transform rejects zero-weight units, shared machinery behind 7 estimators. Landing it would turn both refusals back into supported fits. Gate with a `SurveyDesign.subpopulation()` parity test on ATT, TSL SE and survey df where the excluded cohort exhausts a PSU. | `diff_diff/wooldridge.py`, `diff_diff/utils.py` | #724-codex-R4/R5 | Heavy | Medium | | `WooldridgeDiD` REFUSES a fit whose only treatment cells fall inside the anticipation window, discarding estimates it successfully computed. `_require_estimable_overall_att` ([M-124]) raises when no cell has `t >= g`, because the overall ATT averages only `t >= g` (W2025 excludes anticipation leads) while cells from `t >= g - anticipation` are ESTIMATED. For a cohort never observed at or after its own treatment date, those anticipation-window ATT(g, t) are real, identified estimates and are thrown away with the fit. **The refusal is a stopgap for the missing estimand semantics, not the intended end state.** Real fix: decide what such a fit should return — most likely the per-cell ATT(g, t) plus an overall that is explicitly undefined with a stated reason (not a bare NaN, per the no-silent-NaN convention) — then relax the guard to that. Needs a REGISTRY note defining the estimand and an `aggregate()` story for the anticipation-only case. | `diff_diff/wooldridge.py` | #724-codex-R2 | Mid | Medium | -| `WooldridgeDiD` has no same-period COMPARISON-SUPPORT accounting; the `has_untreated` guard is a proxy that counts a treated cohort's OWN pre-treatment rows as comparison observations. So a design with no usable control at any post period passes the upfront check and fails late and opaquely — as rank reduction, then the blanket `_require_estimable_overall_att` refusal ([M-124]). Measured: cohort count is NOT a valid proxy either — two surviving treated cohorts can still be fully degenerate when neither post cell has a same-period control (`TestOverallAttFailsClosed::test_two_cohorts_without_same_period_controls_fail_closed`). **The post-hoc refusal is a stopgap for a missing upfront diagnostic.** Real fix: compute per-`(g, t)` eligible-control support (never-treated, or not-yet-treated at that period), report exactly which cells are unidentified and why BEFORE solving, and only refuse when none survives. This also subsumes the reviewer's R2 item (b). | `diff_diff/wooldridge.py` | #724-codex-R2 | Mid | Medium | -| `WooldridgeDiD` DROPS the observations of a cohort with no supported pre-period before `g - anticipation` ([M-123]) rather than identifying it. Excluding the rows is correct given `g-1` normalization — leaving them in silently loads the cohort's effect onto the time FE — but dropping a cohort a user supplied is a lossy last resort, and it is the trigger for the survey refusal in the row above. **Evaluate whether such cohorts are identifiable at all before accepting exclusion as permanent.** Two candidate routes, both needing a methodology decision recorded in REGISTRY: (a) an explicit user-supplied reference period per cohort — W2025 Section 6.1 says any pre-treatment period may serve as reference and the pre-trend `t`-test is invariant to the choice, so a cohort with ANY supported pre-period is a candidate even when `g-1` is missing; (b) the paper's no-never-treated last-cohort normalization (`wooldridge-2025-review.md:461-463`), which applies ONLY when there is no never-treated group and so does not cover `control_group='never_treated'`. If neither identifies the cohort, convert this row into a REGISTRY Note recording exclusion as the deliberate final answer. | `diff_diff/wooldridge.py`, `docs/methodology/REGISTRY.md` | #724 | Heavy | Medium | +| `WooldridgeDiD` comparison-support accounting is PARTIAL. Per-period support now runs before the solve and removes periods with no eligible comparison, reporting them (REGISTRY *per-period comparison support*). What remains is the per-`(g, t)` half: the completeness gate still refuses when a cell is lost to a cause the period filter cannot see -- treated cohorts sharing no comparison period with each other, and covariate collinearity -- so those users get a refusal naming the cell rather than an upfront diagnostic naming the cause. Real fix: compute per-cell eligible-control support and report exactly which cells are unidentified and why BEFORE solving. Note the cohort-count proxy remains invalid and is still pinned (`TestOverallAttFailsClosed::test_two_cohorts_without_same_period_controls_fail_closed`, verified unaffected by the period filter). | `diff_diff/wooldridge.py` | #724-codex-R2 | Mid | Medium | +| `WooldridgeDiD` DROPS the observations of a cohort with no supported pre-period before `g - anticipation` ([M-123]) rather than identifying it. Excluding the rows is correct given `g-1` normalization -- leaving them in silently loads the cohort's effect onto the time FE -- but dropping a cohort a user supplied is a lossy last resort. **Route (b) is now SETTLED NEGATIVELY and is not the answer:** the paper's no-never-treated last-cohort normalization shipped (W2025 Sec 5.4, per-period comparison support), and it does NOT identify these cohorts -- `wooldridge-2025-review.md:477` is explicit that in the final period the last cohort's ATT is unidentified, and the implementation still excludes any cohort whose reference is `None`. **Route (a) remains open:** an explicit user-supplied reference period per cohort -- W2025 Section 6.1 says any pre-treatment period may serve and the pre-trend `t`-test is invariant to the choice, so a cohort with ANY supported pre-period is a candidate even when `g-1` is missing. If route (a) also fails to identify the cohort, convert this row into a REGISTRY Note recording exclusion as the deliberate final answer. | `diff_diff/wooldridge.py`, `docs/methodology/REGISTRY.md` | #724 | Heavy | Medium | | `WooldridgeDiD` REFUSES a panel whose units split into disconnected support groups within a cohort, rather than estimating what IS identified. The connectivity guard (REGISTRY *within-cohort support connectivity*) correctly detects that a closed component's cells are collinear with the unit FE — previously QR dropped one silently and the overall ATT averaged an incomplete set (issue #724's failure mode via unit support). **Refusing is the safe answer, not the complete one.** The connected component containing the reference is still fully identified, so the estimable resolution is either (a) estimate the connected component and report the disconnected units as excluded, with the estimand restated (a sub-population of units, so it needs a REGISTRY definition and interacts with the survey-domain row above), or (b) per-component references, if a component with its own pre-period can carry its own normalization — needs a methodology decision, since components then are not comparable on one baseline. Gate with the split-support fixture in `TestWithinCohortSupportConnectivity`. | `diff_diff/wooldridge.py` | #724-codex-R7 | Heavy | Medium | -| `WooldridgeDiD` REFUSES all-eventually-treated panels (no never-treated group) because at fully-treated periods the cohort cells are jointly collinear with the time FE; since the completeness gate ([M-124]) a lost cell fails closed. **This is a capability regression traded for correctness** — those fits previously returned finite, silently WRONG ATTs (measured on a true-1.5 DGP: `ATT(3,8) = -0.0096`, `ATT(5,9) = -0.2393`, `overall_att = 1.0023`), because the survivors identify contrasts BETWEEN treated cohorts while keeping their `ATT(g,t)` labels. **W2025 Section 5.4 defines the fix**: with no never-treated group the LAST COHORT serves as the control, so its columns are dropped deliberately rather than by QR, leaving the remaining cells identified (relative to the last cohort's effect — an estimand change that needs a REGISTRY definition). The library ALREADY implements this for the cohort-TREND column (`cohort_trend_coefs` drops the last cohort) but not for the cells, so the two halves of one paper normalization are inconsistent and the trend half is now unreachable end-to-end. Landing it turns `TestAllEventuallyTreated` and `test_cohort_trends_true_all_treated_panel_is_refused` back into fits. | `diff_diff/wooldridge.py`, `docs/methodology/REGISTRY.md` | #724-codex-R8 | Heavy | High | | `WooldridgeDiD` fully resolves the `SurveyDesign` TWICE on every supported survey fit. The pre-exclusion validation pass (added so invalid metadata cannot hide in rows that cohort exclusion deletes) calls `survey_design.resolve(sample)`, and each fitter then calls `_resolve_survey_for_wooldridge` -> `_resolve_survey_for_fit` on the same frame, repeating weight normalization, strata/PSU/FPC validation and design-array construction. Any fit that REACHES the second resolve has an unchanged sample (survey + unidentified-cohort exclusion raises first), so the first result is reusable: capture the `_resolve_survey_for_fit` 4-tuple early and thread it into the three fitters as an optional `pre_resolved`. **Caveat that makes this non-trivial:** `sample = sample.reset_index(drop=True)` runs BETWEEN the two calls, so the reused object must be verified index-independent (resolution extracts positional numpy arrays, but `_inject_cluster_as_psu` and the metadata recompute need checking), and the early call must stop suppressing warnings or the user loses the weight-normalization notice. Gate with a survey fit asserting one normalization warning and byte-identical SEs. | `diff_diff/wooldridge.py` | #724-codex-R10 | Mid | Low | | Bootstrap re-aggregation for `CallawaySantAnnaResults.aggregate()` — a bootstrapped fit currently RAISES rather than substituting analytical inference for percentile-bootstrap statistics. The value-bound `BootstrapReplaySpec` (bit-identical replay, picklable, immune to post-fit `set_params`) is already in-tree and spike-verified; wiring it needs the per-`(g,t)` / per-event-time draw retention plus `assert_allclose` parity tests against the fit-time bootstrap numbers (NOT bit-identity — the fused GEMM's column count differs post-fit, ~1 ULP reassociation). | `diff_diff/aggregation.py`, `diff_diff/staggered_results.py` | #726 | Mid | Medium | | Consolidate the inference-df precedence duplicated across `honest_did.py` (3 copies at ~L655/L836/L1004) onto the shared `resolve_inference_df()` helper added in `diff_diff/aggregation.py`. The copies are correct today; the risk is drift if the survey/replicate precedence changes in one place only. | `diff_diff/honest_did.py` | #726 | Quick | Low | | `ContinuousDiD` CGBS-2024 remaining extensions (earlier phases — `covariates=` reg/dr, `treatment_type="discrete"`, single-cohort `control_group="lowest_dose"` with estimand `ATT(d)−ATT(d_L)` — are already supported; see REGISTRY Note #7). Remaining (all deferred `NotImplementedError`, documented): `estimation_method="ipw"` on the dose curve (scalar-adjustment / degenerate); `covariates=` × `survey_design=` (weighted OR + weighted nuisance IF); multi-cohort **heterogeneous-support** discrete aggregation (support-aware: average each dose only over the cohorts that observe it); **multi-cohort `lowest_dose`** (within-cohort `d_L` reference + support-aware cross-cohort aggregation); and **`covariates=` × `lowest_dose`** (conditional-PT-relative-to-`d_L` estimand). Single-cohort / 2-period / shared-support multi-cohort are supported. | `continuous_did.py` | CGBS-2024 | Heavy | Low | +| `WooldridgeDiD` does not apply the W2025 Sec 5.4 `D_{G_max} x X` covariate normalization, and three sibling covariate rank deficiencies are pre-existing. Measured with the period range pinned and only the never-treated units toggled: (1) time-invariant `exovar` is absorbed by the unit FE, 4 of 26 columns, IDENTICALLY with and without never-treated units; (2) `xgvar`'s cell x covariate block, 19 of 41, identical on both panels; (3) `xtvar` under `demean_covariates=False` does exhibit the `sum_g D_g x = x` dependency that the default demeaning removes; (4) the newly-reachable case -- time-VARYING data passed through `exovar`, which its own docstring reserves for time-invariant covariates -- where the paper's `dT_i` rule would give a deterministic `D_{G_max} x X` drop instead of QR's arbitrary pick (coefficients unaffected, `1.35e-14`; `rank_deficient_action="error"` raises). REGISTRY's narrowed Sec 5.4 note cross-references this row. **Trap for whoever takes it:** `xtvar` under the DEFAULT `demean_covariates=True` is FULL RANK -- the raw block carries demeaned values while `D_g x X` carries raw ones -- and forcing the drop there moves `overall_att` 1.11903 -> 1.46269. Pinned as-is by `TestComparisonSupportFiltering::test_cells_derived_groups_did_not_leak_into_the_design`. | `diff_diff/wooldridge.py` | #729-followup | Heavy | Medium | +| `WooldridgeDiD.n_control_units` counts never-treated UNITS on `control_group="never_treated"` regardless of method, but on the nonlinear paths (`logit`/`poisson`) treated units' pre-treatment rows ARE the identifying comparison -- only the OLS path absorbs them into their own cells. So the reported count under-states the comparison pool exactly where the REGISTRY control-pool asymmetry note applies. Widen to `not_yet_treated or (never_treated and method != "ols")`, or document the count as never-treated-units-by-definition. Behavior is PRE-EXISTING; documented for now in the REGISTRY control-pool Note rather than changed, because widening moves a public results field and wants its own ledger row and test matrix. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low | +| `WooldridgeDiD` has no opt-out for comparison-support period filtering: a user who would rather see the refusal than a reduced sample cannot ask for it. Adding one means a constructor parameter (`get_params`/`set_params` propagation, transactional validation), a ledger row, and a test matrix across both predicate branches and all three `rank_deficient_action` modes -- deliberately out of scope for the change that introduced the filter. The always-on warning is the interim answer. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low | ### Performance @@ -70,3 +73,4 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m | plan-review skill dual-mode peer cancellation: when reviewer 1 (Opus subagent) fails, terminate the concurrently-running codex `codex_review.py` subprocess instead of waiting out its 1200s ceiling (and vice-versa). Interactive analogue of the eval-harness peer-cancellation row above; needs the codex process handle surfaced from `codex_review.py` so the skill can signal it. | `.claude/skills/plan-review/SKILL.md`, `.claude/skills/plan-review/codex_review.py` | plan-review skill local review R2 | Mid | Low | | Clean-negative precision re-run: Campaign 1's `s3_negative` plans were base_sha-contaminated (real defects at the pinned base), so hallucination rate cleared (dual 3.4%) but trivia-flooding on a *genuinely* clean plan is unmeasured. Small future run (~30 s3-style reviews on 3-5 constructed-clean plans, not the full 120-matrix). | `tools/plan-review-eval/` | campaign-1 verdict | Mid | Low | | `trop-silent-drop` criteria regression: the rewritten `criteria.md` dropped a silent-failure catch the OLD criteria had (arm A caught it, B/C missed — campaign-1 A-vs-B contrast). Patch the criteria + re-validate (re-opens the criteria identity, so gated behind a re-validation run). | `.claude/skills/plan-review/criteria.md`, `tools/plan-review-eval/` | campaign-1 verdict | Mid | Medium | +| The Stata `jwdid` parity arms cover BALANCED panels only. `jwdid_alltreated` pins the all-eventually-treated cell set, `N` and SE ratio on the `mpdta` subset (191 units, 955 rows, 764 estimated), but every arm is a balanced frame, so REGISTRY's parity claim is scoped to that case. The comparison-support predicate's UNBALANCED behavior -- where `G_max` may be unobserved at later periods so the closed form `t < G_max - anticipation` does not hold and support is set by whichever cohorts are actually observed -- is verified against the predicate directly, not against Stata. Add an unbalanced `jwdid` arm to close that gap. | `benchmarks/stata/generate_etwfe_cs_golden.do`, `tests/test_etwfe_cs_stata_parity.py` | #729-followup | Mid | Low | diff --git a/benchmarks/data/etwfe_cs_stata_golden.json b/benchmarks/data/etwfe_cs_stata_golden.json index 033efa83..9127f9d0 100644 --- a/benchmarks/data/etwfe_cs_stata_golden.json +++ b/benchmarks/data/etwfe_cs_stata_golden.json @@ -13,6 +13,7 @@ "csdid_cmd": "csdid lemp, ivar(countyreal) time(year) gvar(first_treat) notyet method(reg)", "jwdid_cmd": "jwdid lemp, ivar(countyreal) tvar(year) gvar(first_treat)", "jwdid_never_cmd": "jwdid lemp, ivar(countyreal) tvar(year) gvar(first_treat) never", + "jwdid_alltreated_cmd": "drop if first_treat == 0; jwdid lemp, ivar(countyreal) tvar(year) gvar(first_treat)", "note": "No covariates, so csdid method is immaterial. ETWFE and CS genuinely DISAGREE at (2007,2007).", "stata_version": 19 }, @@ -66,5 +67,16 @@ "2007.first_treat#2005.year#c.__tr__": {"att": 0.031087119389688365, "se": 0.017952980500827595}, "2007.first_treat#2007.year#c.__tr__": {"att": -0.026054410719196439, "se": 0.016725745592717861}, "_cons": {"att": 5.7736089304641052, "se": 0.0033784078861242291} + }, + "jwdid_alltreated": { + "n": 764, + "n_units": 191, + "cells": { + "2004b.first_treat#2004bn.year#c.__tr__": {"att": -0.035399014515583818, "se": 0.023546467824157553}, + "2004b.first_treat#2005.year#c.__tr__": {"att": -0.092587202900096421, "se": 0.032812547504897691}, + "2004b.first_treat#2006.year#c.__tr__": {"att": -0.13020984713392353, "se": 0.038510767102853845}, + "2006.first_treat#2006.year#c.__tr__": {"att": 0.018480380807537715, "se": 0.023079607844173931}, + "_cons": {"att": 6.0092361449684812, "se": 0.002610873988496222} + } } } diff --git a/benchmarks/stata/README.md b/benchmarks/stata/README.md index 1c36f51c..860c4bb3 100644 --- a/benchmarks/stata/README.md +++ b/benchmarks/stata/README.md @@ -214,9 +214,21 @@ external anchors, with the ETWFE-vs-CS gap recorded rather than asserted away. - **Point estimates** — both estimators match their reference to `atol=1e-6` (observed ~3e-8, i.e. Stata's log-output rounding) on all 7 post-treatment cells. - **CS SEs** — match `csdid` outright (`rtol=1e-5`). +- **All-eventually-treated cell set and row count** (`jwdid_alltreated`) — the + same panel with `first_treat == 0` dropped: 191 units, 955 rows. `jwdid` + succeeds and estimates on 764 of them, because with no never-treated group the + last cohort (2007) becomes the reference and the fully-treated periods carry + no identified ATT (W2025 Section 5.4). It reports only the smaller `N`. The + library computes the identical cell set + (`(2004,2004), (2004,2005), (2004,2006), (2006,2006)`) on the identical row + count, with ATTs matching to ~1e-15 — and warns about the reduction rather + than passing it over in silence. `n` and `n_units` are serialized because the + row count is the finding, not incidental. - **ETWFE SEs** — do **not** match `jwdid`. Every cell is uniformly SMALLER than Stata's, by a factor that shrinks as the cluster count grows: 1.0280 at - G=20, 1.0132 at G=40, 1.0010 at G=500. **The mechanism is not identified.** + G=20, 1.0132 at G=40, 1.00264 at G=191, 1.0010 at G=500. Each arm therefore + pins its OWN measured ratio; the constant does not transfer between arms. + **The mechanism is not identified.** The gap tracks `sqrt(G/(G-1))` but sits consistently ABOVE it, and the CR1 factor in `linalg.py` already applies `(G/(G-1)) * ((n-1)/(n-k))` — so a missing cluster term is ruled out. The test pins the observed ratio and its diff --git a/benchmarks/stata/generate_etwfe_cs_golden.do b/benchmarks/stata/generate_etwfe_cs_golden.do index 437ac984..6e1195fb 100644 --- a/benchmarks/stata/generate_etwfe_cs_golden.do +++ b/benchmarks/stata/generate_etwfe_cs_golden.do @@ -187,6 +187,32 @@ matrix jn_b = e(b) matrix jn_V = e(V) local jn_names : colnames jn_b +* ------------------------------------------------------------------------------ +* JWDID on an ALL-EVENTUALLY-TREATED panel: the same pinned data with the +* never-treated counties dropped. External anchor for W2025 Section 5.4 -- the +* last cohort (2007) becomes the reference, so jwdid estimates only the earlier +* cohorts and reports a SMALLER N without comment. The library computes the same +* cell set, and warns. +* +* Plain `jwdid`, NOT `jwdid ... never`: there is no never-treated group left for +* the latter to use. +* +* preserve/restore so the subset cannot reach the schema asserts above or the +* three arms already estimated, which share this in-memory dataset. e(N) is +* captured because the row-count claim is the point of this arm and the golden +* otherwise records only coefficients and SEs. +* ------------------------------------------------------------------------------ +preserve +drop if first_treat == 0 +quietly levelsof countyreal, local(_at_units) +jwdid lemp, ivar(countyreal) tvar(year) gvar(first_treat) +matrix ja_b = e(b) +matrix ja_V = e(V) +local ja_names : colnames ja_b +local ja_N = e(N) +local ja_nunits : word count `_at_units' +restore + * ------------------------------------------------------------------------------ * Emit the golden. * ------------------------------------------------------------------------------ @@ -210,6 +236,7 @@ file write `fh' `" "stata_edition": "`sedition'","' _n file write `fh' `" "csdid_cmd": "csdid lemp, ivar(countyreal) time(year) gvar(first_treat) notyet method(reg)","' _n file write `fh' `" "jwdid_cmd": "jwdid lemp, ivar(countyreal) tvar(year) gvar(first_treat)","' _n file write `fh' `" "jwdid_never_cmd": "jwdid lemp, ivar(countyreal) tvar(year) gvar(first_treat) never","' _n +file write `fh' `" "jwdid_alltreated_cmd": "drop if first_treat == 0; jwdid lemp, ivar(countyreal) tvar(year) gvar(first_treat)","' _n file write `fh' `" "note": "No covariates, so csdid method is immaterial. ETWFE and CS genuinely DISAGREE at (2007,2007).","' _n file write `fh' `" "stata_version": `sver'"' _n file write `fh' " }," _n @@ -271,7 +298,32 @@ forvalues i = 1/`k' { local sep "," } } -file write `fh' _n " }" _n +file write `fh' _n " }," _n + +* --- jwdid on the all-eventually-treated subset (W2025 Sec 5.4 anchor) -------- +* Carries "n" because the row-count reduction IS the finding: jwdid drops the +* fully-treated periods silently and only reports a smaller N. +file write `fh' `" "jwdid_alltreated": {"' _n +file write `fh' `" "n": `ja_N',"' _n +file write `fh' `" "n_units": `ja_nunits',"' _n +file write `fh' `" "cells": {"' +local sep "" +local k = colsof(ja_b) +forvalues i = 1/`k' { + local nm : word `i' of `ja_names' + scalar bval = ja_b[1, `i'] + scalar seval = sqrt(ja_V[`i', `i']) + if !missing(bval) { + _jnum bval + local a = r(s) + _jnum seval + local s3 = r(s) + file write `fh' "`sep'" _n `" "`nm'": {"att": `a', "se": `s3'}"' + local sep "," + } +} +file write `fh' _n " }" _n +file write `fh' " }" _n file write `fh' "}" _n file close `fh' diff --git a/diff_diff/guides/llms-autonomous.txt b/diff_diff/guides/llms-autonomous.txt index ddd05e4f..b2384e8d 100644 --- a/diff_diff/guides/llms-autonomous.txt +++ b/diff_diff/guides/llms-autonomous.txt @@ -531,6 +531,24 @@ When `has_never_treated == False`: `StackedDiD` / `WooldridgeDiD` - use the last-treated or untreated- until-late units as implicit controls; estimators do not error, but consider whether the implicit control structure is what you want. +- `WooldridgeDiD` specifically: use `control_group="not_yet_treated"` + (the default). `control_group="never_treated"` raises when no + cohort-0 units exist. On an all-eventually-treated panel the last + cohort becomes the reference per W2025 Section 5.4, so periods at + which every unit is treated carry no identified ATT(g, t) and are + REMOVED from the estimation sample before the solve. The fit emits a + `UserWarning` naming the dropped periods, the observation count and + the cause, plus a second naming any cohort left with no estimated + cells (the last cohort, normally). `results.groups` excludes those + cohorts. If your agent surfaces warnings to a user, surface these: + the estimate is computed on fewer rows than were supplied. Stata + `jwdid` performs the same reduction but reports only a smaller `N`. + SOME covariate specifications on such a panel are still rank-deficient + (`exovar`, `xgvar`, and `xtvar` with `demean_covariates=False`), because + `D_{G_max} x X` is not normalized; `rank_deficient_action="error"` + raises on those, and coefficients are unaffected either way. Default + `xtvar` (`demean_covariates=True`) is FULL RANK -- do not steer users + away from it. ### §4.5 Non-absorbing binary treatment (treatment switches back to 0) diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index da6452c8..67e20e63 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -1360,9 +1360,39 @@ WooldridgeDiD( conley_metric: str = "haversine", # "haversine" | "euclidean" conley_kernel: str = "bartlett", # "bartlett" | "uniform" conley_lag_cutoff: int | None = None, # within-unit Bartlett max lag (0 = spatial-only) + cohort_trends: bool = False, # Linear dg_i*t cohort trends (W2025 Sec 8 / Eq 8.1); OLS only. + # On all-eventually-treated panels the last cohort's trend + # column is dropped per Sec 5.4, so cohort_trend_coefs + # carries G-1 entries. Rejected with survey_design= and + # with control_group="never_treated". ) ``` +`fit()` additionally accepts `survey_design=` (a `SurveyDesign`) on all three +methods; see the Survey Support section. + +**All-eventually-treated panels (no never-treated group).** Use +`control_group="not_yet_treated"` — `"never_treated"` raises when no cohort-0 +units exist. Periods at which every unit is treated carry no identified +ATT(g, t), so they are REMOVED from the estimation sample before the solve and +the last cohort becomes the reference (W2025 Section 5.4). The fit warns naming +the dropped periods, the observation count and the cause, and separately names +any cohort left with no estimated cells; `results.groups` excludes those +cohorts. Stata `jwdid` performs the same reduction silently, reporting only a +smaller `N`. SOME covariate specifications on such a panel are still +rank-deficient -- `exovar`, `xgvar`, and `xtvar` with +`demean_covariates=False` -- because `D_{G_max} x X` is not normalized; +`rank_deficient_action="error"` raises on those and coefficients are +unaffected either way. Default `xtvar` (`demean_covariates=True`) is FULL +RANK and fits cleanly. + +**Survey designs refuse row-dropping paths.** `survey_design=` combined with +either comparison-support period filtering or unidentified-cohort exclusion +raises `NotImplementedError` rather than deleting rows, because deletion would +remove their PSUs and strata from the TSL variance. Restrict the frame yourself +and re-fit. The refusals are conditional: survey fits that drop nothing are +unaffected. + **Alias:** `ETWFE` **fit() parameters:** diff --git a/diff_diff/wooldridge.py b/diff_diff/wooldridge.py index 9e844530..d0d38325 100644 --- a/diff_diff/wooldridge.py +++ b/diff_diff/wooldridge.py @@ -15,7 +15,7 @@ import warnings from dataclasses import dataclass -from typing import Any, Dict, List, Optional, Tuple +from typing import Any, Dict, List, Optional, Set, Tuple import numpy as np import pandas as pd @@ -148,9 +148,11 @@ def _require_complete_cell_set( builder deliberately never emitted (the reference, unobserved pairs) are not in ``requested_keys`` and so are not counted as losses. - The same-period comparison-support diagnostic that would catch this BEFORE - the solve, with a better message, is tracked in TODO.md; this is the - correctness backstop that must hold regardless. + The same-period comparison-support filter now catches the most common cause + BEFORE the solve, with a better message: periods where no unit is untreated + are removed from the estimation sample and reported. This remains the + correctness backstop for what that filter cannot see -- cohorts sharing no + comparison period, and covariate collinearity. """ missing = [ k @@ -168,11 +170,13 @@ def _require_complete_cell_set( "the survivors identify whatever contrast the reduced design supports " "(e.g. a difference between two treated cohorts), so reporting them " "under their original labels, or averaging them into the overall ATT, " - "would be silently wrong. This usually means those periods have no " - "eligible comparison group: every unit is already treated, or the " - "never-treated units are not observed then. Restrict the panel to " - "periods with an available comparison group, or add never-treated " - "observations covering them." + "would be silently wrong. Periods with no eligible comparison group are " + "already removed before the solve, so this points at one of two other " + "causes: treated cohorts that share no comparison period with each " + "other, or a covariate collinear with the treatment cells. Check your " + "covariates first if any were supplied; otherwise restrict the panel to " + "cohorts observed over a common window, or add never-treated " + "observations." ) @@ -208,10 +212,12 @@ def _require_estimable_overall_att( no-silent-NaN convention, ledger row ``M-124``). NOT every branch is a final answer. Cause (2) discards anticipation-window - ATT(g, t) that ARE identified, and cause (3) is a post-hoc stand-in for the - same-period comparison-support diagnostic this estimator does not yet - compute. Both are STOPGAPS with follow-up rows in ``TODO.md``; only cause - (1) and the zero-weight case are terminal. + ATT(g, t) that ARE identified, and remains a STOPGAP with a follow-up row in + ``TODO.md``. Cause (3) was a post-hoc stand-in for the comparison-support + diagnostic, which the estimator now computes BEFORE the solve -- so it is + reached only by what that filter cannot see (cohorts sharing no comparison + period, covariate collinearity). Cause (1) and the zero-weight case are + terminal. """ if np.isfinite(overall.get("att", float("nan"))): return @@ -227,10 +233,11 @@ def _require_estimable_overall_att( raise ValueError( "No treatment effect is identified: every cohort-time cell was " "removed from the design, so the overall ATT is undefined. This " - "usually means the treatment cells are collinear with the absorbed " - f"fixed effects -- with control_group={control_group!r}, a single " - "treated cohort and no never-treated units leaves every post-period " - f"cell equal to a time indicator. {hint}" + "means the treatment cells are collinear with the absorbed " + f"fixed effects even after unsupported periods were removed -- with " + f"control_group={control_group!r}, this happens when the surviving " + "cohorts share no comparison period, or when a supplied covariate is " + f"collinear with the cells. {hint}" ) if not post: raise ValueError( @@ -551,6 +558,98 @@ class InteractionDiagnostics: disconnected: List[Tuple[Any, Any]] +def _compute_cell_support( + cohort_vals: np.ndarray, + time_vals: np.ndarray, + weights: Optional[np.ndarray] = None, +) -> Set[Tuple[Any, Any]]: + """The set of (cohort, period) cells carrying POSITIVE total weight. + + A cell "exists" only if it carries positive total weight. Survey weights + scale the design by sqrt(w) (`solve_ols`), so a cell whose rows all carry + zero pweight is absent from the effective regression even though its rows + are present in the frame - and a reference chosen from raw presence would + then omit nothing, leaving the cohort's block spanning the absorbed cohort + dummy and QR free to drop a real post-treatment cell (issue #724 again, + this time on the survey path). Weight NORMALIZATION is multiplicative, so + a raw zero stays zero and reading the pre-resolution column here is + equivalent FOR ZEROS -- which is the whole of what that argument covers. + It says nothing about NaN / negative / Inf, and `nansum` would read those + cells as unsupported, silently reshaping the sample. `fit()` therefore + runs `survey.validate_raw_weights` on the raw column before it reaches + this function, so by here the vector is finite and non-negative. + + Computed ONCE by grouped accumulation rather than per (g, t). The naive + form allocated an n-row boolean mask for every cohort-period pair during + reference selection and again while building columns -- O(n*G*T) on top of + the matrix construction, material on large staggered panels (codex R7). + `nan_to_num` reproduces the previous `np.nansum` semantics: a NaN weight + contributes 0 rather than poisoning the cell's total. (`fit()` rejects NaN + weights upstream; direct callers of this function may still pass them.) + """ + totals = ( + pd.Series( + np.ones(len(cohort_vals)) + if weights is None + else np.nan_to_num(np.asarray(weights, dtype=float), nan=0.0) + ) + .groupby([pd.Series(cohort_vals), pd.Series(time_vals)], sort=False) + .sum() + ) + return {key for key, total in totals.items() if total > 0} + + +def _compute_references( + groups: List[Any], + cohort_vals: np.ndarray, + time_vals: np.ndarray, + anticipation: int, + supported: Set[Tuple[Any, Any]], +) -> Dict[Any, Any]: + """Per-cohort ETWFE reference period, or ``None`` when unidentified. + + W2025 Eq. 6.1/6.4: the ``g-1`` cell is EXCLUDED so it serves as the + reference. Computed over each cohort's OWN observed support, never the + panel-wide period list: a cohort unobserved at the globally-latest + pre-period would otherwise have an all-zero column omitted, leaving its + cell block spanning the unit-FE-absorbed cohort dummy and letting QR drop + an arbitrary -- possibly post-treatment -- cell instead (issue #724). + ``None`` marks a cohort with no pre-treatment support at all, whose ATTs + are unidentified; the caller excludes it. + + Callable on the PRE-filter frame so `fit()` can detect a reference that + moves when unsupported periods are dropped (branch 1) or assert that none + can move (branch 2). + """ + references: Dict[Any, Any] = {} + for g in groups: + observed = np.unique(time_vals[cohort_vals == g]) + # latest SUPPORTED pre-period, not merely the latest observed one + pre = [t for t in observed if t < g - anticipation and (g, t) in supported] + references[g] = max(pre) if pre else None + return references + + +def _cells_derived_groups(gt_effects: Dict) -> List[Any]: + """Cohorts carrying at least one ESTIMATED cell, for results metadata. + + Distinct from the fit-level ``groups`` list, which is the cohorts PRESENT in + the estimation frame. The two diverge whenever a cohort is retained purely + as a control: cohort ``G_max`` on an all-eventually-treated panel receives + no cells by the W2025 Section 5.4 normalization, and a cohort can lose every + row to comparison-support filtering. Reporting either as an estimated cohort + produces the "phantom zero-member cohort" the results contract forbids + (``set(results.groups) == {g for (g, t) in group_time_effects}``). + + Use for ``results.groups`` and ``_n_g_per_cohort`` ONLY. The design-side + readers of ``groups`` -- the covariate blocks (``_prepare_covariates``, + ``D_g × X``) and the cohort-trend block -- require the PRESENT list: a + cells-derived list there drops ``D_{G_max} × X``, silently applying a + normalization this release deliberately defers, and shifts every ATT. + """ + return sorted({g for (g, _t) in gt_effects}) + + def _build_interaction_matrix( data: pd.DataFrame, cohort: str, @@ -595,54 +694,15 @@ def _build_interaction_matrix( # not_yet_treated: post-treatment only (always) include_pre = control_group == "never_treated" and method == "ols" - # ETWFE reference period, per cohort (W2025 Eq. 6.1/6.4: the `g-1` cell is - # EXCLUDED so it serves as the reference). Computed over each cohort's OWN - # observed support, never the panel-wide `times`: a cohort unobserved at the - # globally-latest pre-period would otherwise have an all-zero column omitted, - # leaving its cell block spanning the unit-FE-absorbed cohort dummy and - # letting QR drop an arbitrary — possibly post-treatment — cell instead - # (issue #724). ``None`` marks a cohort with no pre-treatment support at all, - # whose ATTs are unidentified; the caller excludes it. - # A cell "exists" only if it carries POSITIVE total weight. Survey weights - # scale the design by sqrt(w) (`solve_ols`), so a cell whose rows all carry - # zero pweight is absent from the effective regression even though its rows - # are present in the frame - and a reference chosen from raw presence would - # then omit nothing, leaving the cohort's block spanning the absorbed cohort - # dummy and QR free to drop a real post-treatment cell (issue #724 again, - # this time on the survey path). Weight NORMALIZATION is multiplicative, so - # a raw zero stays zero and reading the pre-resolution column here is - # equivalent FOR ZEROS -- which is the whole of what that argument covers. - # It says nothing about NaN / negative / Inf, and `nansum` would read those - # cells as unsupported, silently reshaping the sample. `fit()` therefore - # runs `survey.validate_raw_weights` on the raw column before it reaches - # this function, so by here the vector is finite and non-negative. - # Computed ONCE by grouped accumulation rather than per (g, t). The naive - # form allocated an n-row boolean mask for every cohort-period pair during - # reference selection and again while building columns -- O(n*G*T) on top of - # the matrix construction, material on large staggered panels (codex R7). - # `nan_to_num` reproduces the previous `np.nansum` semantics: a NaN weight - # contributes 0 rather than poisoning the cell's total. (`fit()` rejects NaN - # weights upstream; direct callers of this function may still pass them.) - _cell_totals = ( - pd.Series( - np.ones(len(data)) - if weights is None - else np.nan_to_num(np.asarray(weights, dtype=float), nan=0.0) - ) - .groupby([pd.Series(cohort_vals), pd.Series(time_vals)], sort=False) - .sum() - ) - _supported_pairs = {key for key, total in _cell_totals.items() if total > 0} + # Cell support and per-cohort references live in module-level helpers so + # `fit()` can compute both on the PRE-filter frame (comparison-support + # filtering needs the references before this matrix is built). + _supported_pairs = _compute_cell_support(cohort_vals, time_vals, weights) def _cell_supported(g: Any, t: Any) -> bool: return (g, t) in _supported_pairs - references: Dict[Any, Any] = {} - for g in groups: - observed = np.unique(time_vals[cohort_vals == g]) - # latest SUPPORTED pre-period, not merely the latest observed one - pre = [t for t in observed if t < g - anticipation and _cell_supported(g, t)] - references[g] = max(pre) if pre else None + references = _compute_references(groups, cohort_vals, time_vals, anticipation, _supported_pairs) cols = [] col_names = [] @@ -788,8 +848,21 @@ class WooldridgeDiD: switch; suppress via ``warnings.filterwarnings``. control_group : {"not_yet_treated", "never_treated"} Which units serve as the comparison group. "not_yet_treated" (jwdid - default) uses all untreated observations at each time period; - "never_treated" uses only units never treated throughout the sample. + default) uses all untreated observations at each time period. + + "never_treated" restricts the comparison pool to never-treated units + ON THE OLS PATH ONLY, where every ``(g, t)`` cell except each cohort's + reference is emitted, so treated units' pre-treatment rows sit in their + own indicators rather than the baseline. On the **nonlinear** paths + (``method="logit"`` / ``"poisson"``) only post-treatment cells are + emitted -- including all cells would make each cohort dummy collinear + with the sum of its own indicators -- so treated units' pre-treatment + rows ARE part of the identifying comparison there, exactly as under + ``"not_yet_treated"``. This asymmetry is pre-existing and structural; + see the REGISTRY note. Note that ``n_control_units`` counts + never-treated UNITS on this setting regardless of method, so on the + nonlinear paths it under-reports the rows actually doing the + comparison (tracked in ``TODO.md``). anticipation : int Number of periods before treatment onset to include as treatment cells (anticipation effects). 0 means no anticipation. @@ -849,15 +922,17 @@ class WooldridgeDiD: only: ``cohort_trends=True`` + ``method ∈ {"logit","poisson"}`` raises ``NotImplementedError`` at ``__init__``. Auto-routes to the full-dummy design regardless of ``vcov_type`` (matching the - absorb→fixed_effects auto-route). Each treated cohort must have - ≥ 2 observed pre-periods in the analysis sample for ``dg_i · t`` - to be separately identified from cohort + time FE; ``fit()`` - raises ``ValueError`` otherwise. On all-eventually-treated - panels the last cohort's trend column is dropped per paper - Section 5.4 -- but note such panels currently RAISE before that - matters, because the corresponding cell-level normalization is - not implemented and the design is rank-deficient at - fully-treated periods (see REGISTRY, tracked in TODO.md). ``cohort_trends=True`` + ``survey_design`` raises + absorb→fixed_effects auto-route). Each cohort that RECEIVES a + trend column must have ≥ 2 observed pre-periods in the final + analysis sample for ``dg_i · t`` to be separately identified + from cohort + time FE; ``fit()`` raises ``ValueError`` + otherwise. The check runs after comparison-support filtering and + unidentified-cohort exclusion, and skips the last cohort on + all-eventually-treated panels because that cohort gets no trend + column. On such panels the last cohort's trend column is dropped + per paper Section 5.4, matching the cell-level normalization + applied to the design, so ``cohort_trend_coefs`` carries ``G-1`` + entries. ``cohort_trends=True`` + ``survey_design`` raises ``NotImplementedError`` at ``fit()`` (deferred follow-up). ``cohort_trends=True`` + ``control_group="never_treated"`` also raises ``NotImplementedError`` at ``fit()``. The @@ -1298,38 +1373,9 @@ def fit( "time periods. Use 'never_treated' with a never-treated group." ) - # 1c. Identification check for cohort_trends=True (paper W2025 Section 8 / - # Eq. 8.1). Each treated cohort needs at least 2 distinct pre-treatment - # periods (``t < g - anticipation``) for the cohort-specific linear trend - # ``dg_i · t`` to be separately identified from cohort + time FE. With - # only 1 pre-period the linear trend is observationally equivalent to - # cohort FE on that single point. - # - # Counts pre-treatment periods OBSERVED FOR THIS COHORT (per-cohort - # sample subset) rather than the global panel time set — on - # unbalanced panels a cohort can have only one observed pre-period - # even when the global panel has many, and the linear trend is - # still underidentified (per codex R2 P1 fix). - if self.cohort_trends: - for g in groups: - cohort_pre_times = sample.loc[ - (sample[cohort] == g) & (sample[time] < g - self.anticipation), - time, - ].unique() - n_pre_periods = len(cohort_pre_times) - if n_pre_periods < 2: - raise ValueError( - f"cohort_trends=True requires at least 2 pre-treatment " - f"periods OBSERVED FOR EACH TREATED COHORT (paper W2025 " - f"Section 8 / Eq. 8.1 identification). Cohort g={g} has " - f"only {n_pre_periods} pre-treatment period(s) observed " - f"in the analysis sample (t < g - anticipation = " - f"{g - self.anticipation}); the cohort-specific linear " - f"trend dg_i · t is not separately identified from " - f"cohort + time fixed effects on a single point. Drop " - f"cohort_trends=True or use a panel where each treated " - f"cohort has at least 2 observed pre-periods." - ) + # 1c. The cohort_trends=True identification check MOVED -- it now runs on + # the FINAL sample and final `groups`, after period filtering and + # unidentified-cohort exclusion. See "1c (relocated)" below. # 2. Build interaction matrix. # @@ -1383,6 +1429,181 @@ def fit( if self.method == "ols": _reject_zero_weight_groups(_cell_w, sample, unit, time) + # ---- Per-period comparison support (W2025 Section 5.4) ------------- + # + # A period is retained only if some unit is UNTREATED there, i.e. only + # if ATT(g, t) at that period has an eligible comparison outcome. The + # eligible set differs by path, because the regression baseline does: + # + # include_pre -> cohort == 0 only. On `never_treated` + OLS the + # builder emits a cell for every (g, t) except each cohort's + # reference, so a later cohort's rows sit in their OWN indicator, + # not the baseline. The omitted reference rows ARE in the + # baseline, but they do not IDENTIFY the period: cohort h's + # indicator is absorbed by the unit FE + # (1{h,t} = D_h - sum_{t' != t} h_{t'}), so the period dummy stays + # reproducible from the emitted cells and the design is still + # collinear. Counting reference rows as support therefore makes a + # period look identified when it is not. + # otherwise -> cohort == 0 OR not-yet-treated rows, which carry no + # emitted cell below t = g - anticipation and so are baseline. + # + # Weight-aware: a zero-weight row is absent from the sqrt(w)-scaled + # regression, so it cannot supply support. + _include_pre = self.control_group == "never_treated" and self.method == "ols" + _cohort_arr = sample[cohort].to_numpy() + _time_arr = sample[time].to_numpy() + _w_arr = ( + np.ones(len(sample)) + if _cell_w is None + else np.nan_to_num(np.asarray(_cell_w, dtype=float), nan=0.0) + ) + + _eligible = _cohort_arr == 0 + if not _include_pre: + _eligible = _eligible | ((_cohort_arr - self.anticipation) > _time_arr) + _eligible = _eligible & (_w_arr > 0) + + _supported_periods = set(np.unique(_time_arr[_eligible]).tolist()) + _unsupported_periods = sorted(set(np.unique(_time_arr).tolist()) - _supported_periods) + + # References are needed BEFORE the filter -- not by the predicate above, + # which reads only cohort/time/anticipation and row weights, but to tell + # a legitimate reference move (branch 1) from a silent renormalization. + _pre_refs = _compute_references( + groups, + _cohort_arr, + _time_arr, + self.anticipation, + _compute_cell_support(_cohort_arr, _time_arr, _cell_w), + ) + _pre_filter_groups = list(groups) + # Cohorts surviving the period filter, captured BEFORE unidentified-cohort + # exclusion recomputes `groups` again. Warning (b) differences against + # THIS list: a cohort removed later by exclusion already has its own + # warning, and naming it here too would double-report it. + _post_filter_groups = list(groups) + # Per-cohort unit counts on the PRE-filter frame. `_n_g_per_cohort` (the + # W2025 Eq. 7.4/7.6 cohort-share weight) is read off the FINAL sample, + # and period filtering is the first thing in this estimator that can + # remove SOME units of a RETAINED cohort -- `_filter_sample` keeps every + # row of every treated unit, and unidentified-cohort exclusion removes + # whole cohorts. On an unbalanced panel a unit observed only at + # unsupported periods vanishes entirely, shrinking N_g and silently + # reweighting the aggregate (measured: a cohort supplied with 100 units + # of which 90 appear only at a dropped period is counted as 10, moving + # aggregate(weights="cohort_share") from 1.8078 to 3.8157). Which N_g + # the paper intends is genuinely ambiguous there -- W2025 Section 7 + # assumes a balanced panel -- so the count is carried to the results + # object and `aggregate` fails closed rather than picking one silently. + _pre_filter_unit_counts = { + g: int(sample.loc[sample[cohort] == g, unit].nunique()) for g in groups + } + + if _unsupported_periods: + if survey_design is not None: + # Same naive-subsetting problem the unidentified-cohort path + # refuses below: deleting rows removes their PSUs and strata + # from the Taylor-linearized meat and from + # `df_survey = n_PSU - n_strata`. Decided and raised BEFORE any + # row is removed. `SurveyDesign.subpopulation()` is NOT the + # remedy here -- it zero-pads the excluded rows, which + # `_reject_zero_weight_groups` then refuses on the OLS path. + _plabels = ", ".join(str(t) for t in _unsupported_periods) + raise NotImplementedError( + f"Period(s) {_plabels} have no eligible comparison group, so " + "they carry no identified ATT(g, t) and would be dropped from " + "the estimation sample. Deleting rows under `survey_design=` " + "is naive subsetting: it removes their PSUs and strata from " + "the TSL variance and from `df_survey = n_PSU - n_strata`, so " + "the surviving estimates would be reported with a variance " + "computed on a design you never specified. Restrict the frame " + "to the supported periods explicitly and re-fit -- but first " + "confirm every PSU and stratum survives that restriction. On " + "an unbalanced panel a PSU observed only at these periods " + "disappears with them, which changes the variance in exactly " + "the way this refusal exists to prevent." + ) + + _keep_rows = ~pd.Series(_time_arr, index=sample.index).isin(_unsupported_periods) + _n_dropped = int((~_keep_rows).sum()) + _n_total = len(sample) + _n_periods_total = len(set(np.unique(_time_arr).tolist())) + sample = sample.loc[_keep_rows.to_numpy()].copy() + if _cell_w is not None: + _cell_w = _cell_w[_keep_rows.to_numpy()] + + if _include_pre: + _cause = ( + "no never-treated units are observed at those periods, so " + "ATT(g, t) there is not identified against any untreated " + "outcome" + ) + else: + _cause = ( + "every unit is already treated (accounting for " + f"`anticipation={self.anticipation}`), so ATT(g, t) there is " + "not identified against any untreated outcome" + ) + warnings.warn( + f"Dropped {_n_dropped} of {_n_total} observations " + f"({len(_unsupported_periods)} of {_n_periods_total} periods: " + f"{', '.join(str(t) for t in _unsupported_periods)}) from the " + f"estimation sample: no eligible comparison group exists at those " + f"periods -- {_cause}. To estimate those periods, add " + "never-treated units or restrict the panel.", + UserWarning, + stacklevel=2, + ) + + # `groups` is fixed pre-filter and otherwise refreshed only inside + # the unidentified-cohort branch. A cohort can lose EVERY row here, + # which would leave `_prepare_covariates` emitting an all-zero + # D{g}_x_{cov} column for a cohort no longer present. + groups = sorted(g for g in sample[cohort].unique() if g > 0) + _post_filter_groups = list(groups) + + # Reference movement. Branch 2 cannot move a reference (eligibility + # `t < g - anticipation` IS the predicate's second disjunct, so any + # reference period is supported by construction) -- assert that. + # Branch 1 CAN, legitimately: the ATTs stay correctly labelled and + # validly identified, just normalized against a different baseline + # period, so warn instead of raising. + _post_refs = _compute_references( + groups, + sample[cohort].to_numpy(), + sample[time].to_numpy(), + self.anticipation, + _compute_cell_support(sample[cohort].to_numpy(), sample[time].to_numpy(), _cell_w), + ) + _moved = [ + (g, _pre_refs[g], _post_refs[g]) + for g in _post_refs + if g in _pre_refs + and _pre_refs[g] is not None + and _post_refs[g] is not None + and _pre_refs[g] != _post_refs[g] + ] + if _moved and not _include_pre: + _detail = "; ".join(f"cohort {g}: {old} -> {new}" for g, old, new in _moved) + raise ValueError( + "Internal invariant violated: filtering unsupported periods " + f"moved a reference period on the not-yet-treated path ({_detail}). " + "Reference eligibility is the support predicate's own second " + "disjunct, so this cannot happen; the ATTs would be silently " + "renormalized (issue #724). Please report this panel." + ) + for _g, _old, _new in _moved: + warnings.warn( + f"Cohort {_g}'s reference period moved from {_old} to {_new} " + f"because period {_old} has no never-treated observations. " + f"ATT({_g}, .) are normalized against t={_new}.", + UserWarning, + stacklevel=2, + ) + # A reference that became None is NOT a move: the cohort is now + # unidentified and `_exclude_unidentified_cohorts` warns by name. + def _build(frame: pd.DataFrame, w: Optional[np.ndarray]): return _build_interaction_matrix( frame, @@ -1445,6 +1666,112 @@ def _build(frame: pd.DataFrame, w: Optional[np.ndarray]): _cell_w = _cell_w[keep_mask.to_numpy()] groups = sorted(g for g in sample[cohort].unique() if g > 0) X_int, int_col_names, gt_keys, interaction_diagnostics = _build(sample, _cell_w) + + # 1c (relocated). Identification check for cohort_trends=True (paper + # W2025 Section 8 / Eq. 8.1). Each cohort that RECEIVES a trend column + # needs at least 2 distinct pre-treatment periods (``t < g - + # anticipation``) for ``dg_i · t`` to be separately identified from + # cohort + time FE; with 1 pre-period the trend is observationally + # equivalent to cohort FE on that single point. + # + # Counts pre-treatment periods OBSERVED FOR THIS COHORT rather than the + # global panel time set -- on unbalanced panels a cohort can have one + # observed pre-period even when the panel has many (codex R2 P1). + # + # Runs HERE, not before the filter, for two reasons measured during + # review: (a) a cohort whose only observation is a to-be-filtered period + # would be rejected instead of filtered, making the auto-filtered fit + # diverge from the same frame filtered by hand; (b) on `{3,5,8}` at + # anticipation=2 a cohort can keep its rows but have no reference, so + # the guard would fire before exclusion removes it. + # + # Iterates `trend_groups`, NOT every present cohort: with no + # never-treated group the last cohort's trend column is deliberately + # dropped (the Section 5.4 normalization, see the trend block below), so + # demanding two pre-periods of a cohort that receives no trend column is + # a spurious refusal. + if self.cohort_trends: + _has_nt_trend = bool((sample[cohort] == 0).any()) + _trend_check_groups = groups if _has_nt_trend else groups[:-1] + for g in _trend_check_groups: + cohort_pre_times = sample.loc[ + (sample[cohort] == g) & (sample[time] < g - self.anticipation), + time, + ].unique() + n_pre_periods = len(cohort_pre_times) + if n_pre_periods < 2: + raise ValueError( + f"cohort_trends=True requires at least 2 pre-treatment " + f"periods OBSERVED FOR EACH TREATED COHORT (paper W2025 " + f"Section 8 / Eq. 8.1 identification). Cohort g={g} has " + f"only {n_pre_periods} pre-treatment period(s) observed " + f"in the analysis sample (t < g - anticipation = " + f"{g - self.anticipation}); the cohort-specific linear " + f"trend dg_i · t is not separately identified from " + f"cohort + time fixed effects on a single point. Drop " + f"cohort_trends=True or use a panel where each treated " + f"cohort has at least 2 observed pre-periods." + ) + + # Warning (b): cohorts left with NO estimated cells, and cohorts that + # lost every row to period filtering. Emitted after the FINAL build -- + # the zero-cell set differs between the two builds whenever + # unidentified-cohort exclusion also fires, and reading the first build + # would double-report a cohort that already has its own exclusion + # warning. Not gated on `rank_deficient_action`: that governs how rank + # warnings surface, not whether the estimation sample changed. + _emitted_cohorts = {g for g, _t in gt_keys} + _zero_cell = [g for g in groups if g not in _emitted_cohorts] + _fully_dropped = [g for g in _pre_filter_groups if g not in _post_filter_groups] + if _zero_cell or _fully_dropped: + _parts = [] + if _zero_cell: + _parts.append( + f"Cohort(s) {', '.join(str(g) for g in _zero_cell)} have NO " + "estimated cells and are excluded from `results.groups`" + ) + if _fully_dropped: + _parts.append( + f"cohort(s) {', '.join(str(g) for g in _fully_dropped)} lost " + "every observation to comparison-support filtering" + ) + # The Section 5.4 normalization explains exactly ONE zero-cell + # cohort: the LAST one, and only when no never-treated group is + # present. Every other zero-cell cohort has a different cause -- a + # cohort not treated within the observed periods is the common one + # -- and attributing it to a deliberate normalization would tell the + # user the wrong thing about their own panel (measured: cohorts + # {0, 3, 7} over t=1..5 warns on cohort 7, which is zero-cell only + # because t never reaches 7, on a panel that HAS never-treated + # units). Explain each cause against the cohorts it actually covers. + _ref_cohort = groups[-1] if groups and not bool((sample[cohort] == 0).any()) else None + _normalized = [g for g in _zero_cell if g == _ref_cohort] + _other_zero = [g for g in _zero_cell if g != _ref_cohort] + _why = [] + if _normalized: + _why.append( + f"Cohort {_normalized[0]} is the W2025 Section 5.4 reference: " + "with no never-treated group the last cohort serves as the " + "comparison and receives no cells of its own." + ) + if _other_zero: + _why.append( + f"Cohort(s) {', '.join(str(g) for g in _other_zero)} yielded no " + "estimable cell -- typically the cohort is not treated within " + "the observed periods, or each of its cells falls at a period " + "with no eligible comparison group." + ) + if _fully_dropped: + _why.append( + "Cohorts that lost every observation were observed only at " + "periods with no eligible comparison group." + ) + warnings.warn( + "; ".join(_parts) + ". Their ATT(g, t) are not estimated. " + " ".join(_why), + UserWarning, + stacklevel=2, + ) + if interaction_diagnostics.skipped_unobserved and self.rank_deficient_action != "silent": _pairs = ", ".join( f"({g}, {t})" for g, t in interaction_diagnostics.skipped_unobserved[:8] @@ -1659,6 +1986,21 @@ def _build(frame: pd.DataFrame, w: Optional[np.ndarray]): survey_design=survey_design, ) + # Units a RETAINED, ESTIMATED cohort lost to comparison-support + # filtering. Attached here rather than threaded through three fitter + # signatures: all three paths converge on this line, and every one of + # them builds `_n_g_per_cohort` from the post-filter sample. Empty + # whenever nothing was filtered, so the balanced all-eventually-treated + # deliverable -- which drops whole PERIODS and no units at all -- is + # untouched. + results._cohort_units_dropped = { + g: _pre_filter_unit_counts[g] - results._n_g_per_cohort[g] + for g in results.groups + if g in _pre_filter_unit_counts + and g in results._n_g_per_cohort + and _pre_filter_unit_counts[g] > results._n_g_per_cohort[g] + } + self._results = results self.is_fitted_ = True return results @@ -1807,13 +2149,18 @@ def _fit_ols( # "all variables in regression (5.3) involving dT_i get # dropped" when the last cohort serves as control). Drop # the LAST cohort's trend column deterministically — that - # cohort then acts as the trend baseline. NOTE: the - # matching last-cohort handling for cohort × time CELLS - # is NOT implemented in ``_build_interaction_matrix``, so - # this branch is currently unreachable end-to-end -- such - # panels are rank-deficient at fully-treated periods and - # fail the completeness gate. Kept because implementing - # the cell half is what restores them (TODO.md). + # cohort then acts as the trend baseline. The matching + # last-cohort handling for cohort × time CELLS is the + # comparison-support filter in ``fit()``, so this branch + # is now reachable end-to-end: the fully-treated periods + # are removed before the solve rather than sinking the + # design, and ``cohort_trend_coefs`` surfaces G-1 entries. + # + # `groups` here MUST be the PRESENT-cohort list, not the + # cells-derived one used for results metadata: the trend + # baseline is exactly the zero-cell cohort G_max, so a + # cells-derived list would drop the second-to-last + # cohort's trend column instead and shift every ATT. has_never_treated = (sample[cohort] == 0).any() trend_groups = groups if has_never_treated else groups[:-1] trend_cols: List[np.ndarray] = [] @@ -2223,7 +2570,12 @@ def _fit_ols( all_times = sorted(sample[time].unique().tolist()) # Per-cohort unit counts ``N_g`` (paper Eqs. 7.4, 7.6) — needed by # ``aggregate(weights="cohort_share")``. - n_g_per_cohort = {g: int(sample[sample[cohort] == g][unit].nunique()) for g in groups} + # Results metadata is CELLS-derived (see `_cells_derived_groups`); the + # design-side `groups` above stays PRESENT-cohort. + result_groups = _cells_derived_groups(gt_effects) + n_g_per_cohort = { + g: int(sample[sample[cohort] == g][unit].nunique()) for g in result_groups + } _require_complete_cell_set(gt_keys, gt_effects) _require_estimable_overall_att(overall, gt_effects, self.anticipation, self.control_group) @@ -2237,7 +2589,7 @@ def _fit_ols( overall_conf_int=overall["conf_int"], method=self.method, control_group=self.control_group, - groups=groups, + groups=result_groups, time_periods=all_times, n_obs=len(sample), n_treated_units=n_treated, @@ -2565,7 +2917,7 @@ def _avg_ax0(a, cell_mask): overall_conf_int=overall["conf_int"], method=self.method, control_group=self.control_group, - groups=groups, + groups=_cells_derived_groups(gt_effects), time_periods=sorted(sample[time].unique().tolist()), n_obs=len(sample), n_treated_units=int(sample[sample[cohort] > 0][unit].nunique()), @@ -2585,7 +2937,10 @@ def _avg_ax0(a, cell_mask): cluster_name=(None if _has_survey else cluster_col), n_clusters=(None if _has_survey else int(np.unique(cluster_ids).size)), _gt_weights=gt_weights, - _n_g_per_cohort={g: int(sample[sample[cohort] == g][unit].nunique()) for g in groups}, + _n_g_per_cohort={ + g: int(sample[sample[cohort] == g][unit].nunique()) + for g in _cells_derived_groups(gt_effects) + }, _gt_vcov=gt_vcov, _gt_keys=gt_keys_ordered, _df_survey=df_inf, @@ -2821,7 +3176,7 @@ def _avg_ax0(a, cell_mask): overall_conf_int=overall["conf_int"], method=self.method, control_group=self.control_group, - groups=groups, + groups=_cells_derived_groups(gt_effects), time_periods=sorted(sample[time].unique().tolist()), n_obs=len(sample), n_treated_units=int(sample[sample[cohort] > 0][unit].nunique()), @@ -2839,7 +3194,10 @@ def _avg_ax0(a, cell_mask): cluster_name=(None if _has_survey else cluster_col), n_clusters=(None if _has_survey else int(np.unique(cluster_ids).size)), _gt_weights=gt_weights, - _n_g_per_cohort={g: int(sample[sample[cohort] == g][unit].nunique()) for g in groups}, + _n_g_per_cohort={ + g: int(sample[sample[cohort] == g][unit].nunique()) + for g in _cells_derived_groups(gt_effects) + }, _gt_vcov=gt_vcov, _gt_keys=gt_keys_ordered, _df_survey=df_inf, diff --git a/diff_diff/wooldridge_results.py b/diff_diff/wooldridge_results.py index cdbe3d4b..e3950778 100644 --- a/diff_diff/wooldridge_results.py +++ b/diff_diff/wooldridge_results.py @@ -102,11 +102,15 @@ class WooldridgeDiDResults(BaseResults): # never-treated cohort exists, per the all-eventually-treated drop # rule). On all-treated panels the last cohort is intentionally # absent from the dict; its slope is the baseline (zero in deviation - # form). The all-treated branch is currently UNREACHABLE: the paper - # rule is applied to the trend columns but not to the cohort × time - # cells, so those panels raise at fit() (TODO.md). See REGISTRY - # ``## WooldridgeDiD (ETWFE)`` → "Heterogeneous cohort trends" Notes - # for the exact normalization contract. + # form). The all-treated branch is reachable: the same Section 5.4 + # rule is applied to the cohort × time CELLS by comparison-support + # filtering in fit(), so such panels estimate and this dict carries + # G-1 entries. Note it is keyed on PRESENT cohorts while + # ``results.groups`` is derived from the emitted cells, so a cohort + # retained only as the trend baseline appears in neither -- but a + # cohort with a trend slope and no cells appears here and not there. + # See REGISTRY ``## WooldridgeDiD (ETWFE)`` → "Heterogeneous cohort + # trends" Notes for the exact normalization contract. cohort_trend_coefs: Dict[Any, float] = field(default_factory=dict, repr=False) # Flag set by ``_fit_ols`` when ``n_bootstrap > 0`` AND the multiplier @@ -156,6 +160,19 @@ class WooldridgeDiDResults(BaseResults): weights ``ω̂_{ge}``. Empty dict on fits that pre-date the PR-B cohort-share surface (no information loss — ``weights="cell"`` is unaffected).""" + _cohort_units_dropped: Dict[Any, int] = field(default_factory=dict, repr=False) + """Units an ESTIMATED cohort lost to comparison-support filtering, by + cohort. Non-empty only on UNBALANCED panels where some unit is observed + exclusively at periods that carry no eligible comparison group: whole + periods go, and a unit living only in them goes with them. ``N_g`` above + is then read off a strictly smaller unit set than the cohort the user + supplied, which reweights ``aggregate(weights="cohort_share")`` + materially (measured: 1.8078 vs 3.8157 with 90 of a cohort's 100 units + lost). W2025 Section 7 defines ``N_g`` on a balanced panel and does not + say which reading applies here, so ``aggregate`` refuses rather than + picking one — mirroring the survey + ``cohort_share`` refusal below. + Empty whenever nothing was filtered, which includes every balanced + all-eventually-treated fit.""" _gt_vcov: Optional[np.ndarray] = field(default=None, repr=False) """Full vcov of all β_{g,t} coefficients (ordered same as sorted group_time_effects keys).""" _gt_keys: List[Tuple[Any, Any]] = field(default_factory=list, repr=False) @@ -406,6 +423,31 @@ def _build_effect( "version, or use weights='cell' (default) on legacy fits." ) + # Comparison-support filtering + cohort_share is refused for the same + # reason: ``_n_g_per_cohort`` counts units in the FINAL sample, so when + # filtering removed every observation of some units in an estimated + # cohort, those weights describe a cohort strictly smaller than the one + # supplied. Both readings of N_g are defensible and they disagree + # materially, so fail closed rather than return a confident number from + # an estimand the paper does not define (W2025 Section 7 assumes a + # balanced panel). ``weights="cell"`` never reads N_g and is unaffected. + if weights == "cohort_share" and self._cohort_units_dropped: + _detail = ", ".join( + f"cohort {g}: {n} unit(s)" for g, n in sorted(self._cohort_units_dropped.items()) + ) + raise ValueError( + "aggregate(weights='cohort_share') is not supported on this fit: " + "comparison-support filtering removed every observation of some " + f"units belonging to estimated cohort(s) ({_detail}), so the " + "per-cohort unit counts N_g (paper W2025 Eqs. 7.4/7.6) no longer " + "describe the cohorts you supplied. Whether N_g should count the " + "supplied cohort or only the units that survived filtering is " + "undefined for unbalanced panels -- the two disagree materially " + "-- so this library refuses rather than choosing silently. Use " + "weights='cell' (default), or restrict the panel to periods with " + "an eligible comparison group so that no unit is dropped." + ) + # Survey + cohort_share composition is not yet supported. Codex R3 # P0 fix: ``_n_g_per_cohort`` is populated as raw ``unit.nunique()`` # counts, so composing design-weighted ATT estimates (survey TSL) diff --git a/docs/api/wooldridge_etwfe.rst b/docs/api/wooldridge_etwfe.rst index c315ea6b..faf72ad4 100644 --- a/docs/api/wooldridge_etwfe.rst +++ b/docs/api/wooldridge_etwfe.rst @@ -71,14 +71,16 @@ paper W2025 Section 5.4's all-eventually-treated drop rule). On all-treated panels the last cohort is intentionally absent from the dict; its slope is the baseline (zero in deviation form). -.. warning:: - - The all-eventually-treated branch is currently **unreachable**: the - paper's Section 5.4 rule is applied to the trend columns but NOT to - the cohort × time cells, so such panels are rank-deficient at - fully-treated periods and ``fit()`` raises rather than returning - relabeled contrasts. Implementing the cell half restores these fits - (tracked in ``TODO.md``). +.. note:: + + All-eventually-treated panels **estimate**. The paper's Section 5.4 + rule is applied to the cohort × time cells as well as the trend + columns: periods where no unit is untreated carry no identified + ATT(g, t), so they are removed from the estimation sample before the + solve and the last cohort becomes the reference. The reduction is + always reported — the number of observations and periods dropped, the + reason, and any cohort left without cells. Stata's ``jwdid`` performs + the same reduction silently, reporting only a smaller ``N``. See ``docs/methodology/REGISTRY.md`` → ``## WooldridgeDiD (ETWFE)`` → "Heterogeneous cohort trends" for the full normalization contract. diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 6683c0ab..036853aa 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -1961,7 +1961,14 @@ The saturated ETWFE regression includes: The interaction coefficient `δ_{g,t}` identifies `ATT(g, t)` under parallel trends. - **Note (reference-period normalization, W2025 Eq. 6.1/6.4 — `never_treated` OLS only):** this applies to the **lead-and-lag** specification, i.e. `control_group="never_treated"` on the OLS path, which is the only configuration that emits pre-treatment cells (`include_pre`). The default `not_yet_treated` (and both nonlinear paths) emit `t >= g − anticipation` cells only — the lag-only specification — so no cell is omitted from those designs and no placebo cells are produced; the reference period is still computed there, but solely to detect cohorts that have none (see the unidentified-cohort Note). For the lead-and-lag specification, for each treated cohort `g` the design OMITS one cell — the reference period — because the cohort's full cell block spans the cohort indicator `1{G_i = g}`, which the unit fixed effects absorb. The paper is explicit: Eq. 6.1 excludes `dg_i · f(g-1)_t` "so that `s = g − 1` is the reference period", and Eq. 6.4 describes the saturated regression as a collection of 2×2 DiDs using `g − 1` as reference. The library omits `ref(g) = max{t : t < g − anticipation AND cohort g is OBSERVED at t}` — `g − 1` on a balanced panel, and the cohort's own latest available pre-period otherwise. The support must be **per-cohort**: a panel-wide rule omits an identically-zero column whenever that cohort is unobserved at the panel's latest pre-period, leaving the collinearity intact. Anchored to Stata `jwdid ... never`, which omits the same cell for every cohort (`tests/test_etwfe_cs_stata_parity.py::TestNeverTreatedVsStataJwdid`). **Before v3.9 the reference was emitted and generic QR rank detection dropped an arbitrary column instead** — on `mpdta` that silently removed two genuine post-treatment effects, `(2004, 2004)` and `(2006, 2007)` (issue #724). - **Note (reference-period sensitivity, W2025 Section 6.1):** the choice of reference is a NORMALIZATION, not an identifying assumption. The paper notes any set of pre-treatment periods may serve, and the pre-trend `t`-test is numerically identical whichever is used. `g − 1` is the paper's canonical choice and the one this library implements. **The point estimates are NOT invariant to it** — only the two-sided pre-trend test is (Section 6.1). Different references give different finite-sample 2×2 contrasts, all consistent for ATT(g, t) under parallel trends: on `mpdta`, ATT(2007, 2007) computed by hand against references 2006/2005/2004/2003 gives −0.0261 / −0.0571 / −0.0599 / −0.0294. This is exactly why issue #724 mattered — QR dropped `g2007_t2005`, silently making 2005 the reference, and the estimator returned −0.0571: a correct 2×2 DiD against the wrong baseline. It also means the unbalanced-panel fallback (the cohort's latest AVAILABLE pre-period) yields a different contrast from `g − 1`, which is legitimate but should be read as such. -- **Note (unidentified cohorts — library identification limit):** a cohort with NO observed period before `g − anticipation` has no reference cell, so none of its ATTs are identified. The library warns naming the cohort and **excludes its observations** from the estimation sample. Excluding only its columns would be worse than the original bug: `_filter_sample` retains every treated row, so the cohort would join the omitted baseline beside the controls and load its treatment effect onto the time fixed effects — measured on `mpdta` at `anticipation=1`, that moved `ATT(2006, 2006)` by 0.0077 and `ATT(2006, 2007)` by 0.0055, past the 5e-3 this library treats as material. Exclusion can cascade (the removed rows may have been another cohort's not-yet-treated comparison), so the surviving sample is re-checked and `fit()` raises when no comparison observations remain rather than returning an all-NaN fit. This is a **library limit, not the paper's last-cohort result** — W2025's "the last cohort `T` plays the role of the never-treated group" is conditioned on there being no never-treated group at all, a different case. +- **Note (unidentified cohorts — library identification limit):** a cohort with NO observed period before `g − anticipation` has no reference cell, so none of its ATTs are identified. The library warns naming the cohort and **excludes its observations** from the estimation sample. *(Conditional since comparison-support filtering landed: when the filter removes that cohort's rows first, the cohort never reaches this path and is instead named by the zero-cell / fully-dropped warning above. The observations are excluded either way; only which warning names them differs.)* Excluding only its columns would be worse than the original bug: `_filter_sample` retains every treated row, so the cohort would join the omitted baseline beside the controls and load its treatment effect onto the time fixed effects — measured on `mpdta` at `anticipation=1`, that moved `ATT(2006, 2006)` by 0.0077 and `ATT(2006, 2007)` by 0.0055, past the 5e-3 this library treats as material. Exclusion can cascade (the removed rows may have been another cohort's not-yet-treated comparison), so the surviving sample is re-checked and `fit()` raises when no comparison observations remain rather than returning an all-NaN fit. This is a **library limit, not the paper's last-cohort result** — W2025's "the last cohort `T` plays the role of the never-treated group" is conditioned on there being no never-treated group at all, a different case. +- **Note (per-period comparison support, W2025 Section 5.4 — the cell half):** a period at which **no unit is untreated** carries no identified `ATT(g, t)`, because there is no untreated outcome to difference against. Such periods are removed from the estimation sample **before the solve**. The eligible set is keyed on the regression baseline, not on the method: on the lead-and-lag branch (`never_treated` + OLS) only never-treated rows qualify, because every `(g, t)` except each cohort's reference is emitted, so a later cohort's rows sit in their own indicator; elsewhere not-yet-treated rows qualify too. **The omitted reference cells do NOT count as support** even though they sit in the baseline: cohort `h`'s indicator is absorbed by the unit FE (`1{h,t} = D_h − Σ_{t'≠t} h_{t'}`), so the period dummy stays reproducible from the emitted cells and the design remains collinear — measured on a panel with never-treated units through `t=4` and cohorts 3, 6 through `t=6`, retaining `t=5` on the strength of `ref(6)=5` raises on the lost `(6,1)`, while dropping it fits at `overall_att = 1.0171`. On an all-eventually-treated panel this yields exactly Eq. 5.15's cell set: cohort `G_max` is the reference and receives nothing. Anchored to Stata `jwdid` (`tests/test_etwfe_cs_stata_parity.py::TestAllEventuallyTreatedVsStataJwdid`): identical cell set, identical `N` (764 of 955), ATTs agreeing to ~1e-15. +- **Note (the reduction is always reported — deviation from `jwdid`):** Stata `jwdid` performs the same reduction **silently**, reporting only a smaller `N`. This library warns on every fit that drops rows, naming the periods, the observation count, and the branch-correct cause, and separately naming any cohort left with no estimated cells or stripped of every row. Neither warning is gated on `rank_deficient_action` — that setting governs how rank warnings surface, not whether the estimation sample changed. Being explicit is a deliberate improvement, not a numerical deviation: the estimates are identical. +- **Note (reference movement under filtering):** on the `not_yet_treated` branch a reference period can never be filtered — reference eligibility (`t < g − anticipation`) IS the support predicate's own second disjunct evaluated at that cohort's rows — and the estimator asserts this at runtime, raising if it is ever violated (silent renormalization is the issue #724 defect class). On the `never_treated` + OLS branch it CAN move, because eligibility there does not imply never-treated presence: measured, dropping `t=5` moves `ref(6)` from 5 to 4. That is legitimate — the ATTs stay correctly labelled and validly identified, just normalized against a different baseline period — so it **warns** naming the cohort and both periods rather than raising. A reference that becomes `None` is not a move: that cohort is unidentified and the exclusion path already warns by name. +- **Note (zero-cell cohorts are warned, not raised):** a cohort retained only as a control — cohort `G_max` under the Section 5.4 normalization — or one that loses every row to filtering produces no `ATT(g, t)`. `fit()` reports it by name and continues. `results.groups` and `_n_g_per_cohort` are derived from the **emitted cell set** so such a cohort is not advertised as estimated (`set(results.groups) == {g for (g, t) in group_time_effects}`). `cohort_trend_coefs` is keyed on PRESENT cohorts instead, because the trend baseline is exactly the zero-cell cohort; the two therefore differ by design, and the invariant on that dict is `set(cohort_trend_coefs) ⊆ present cohorts`. +- **Note (`aggregate(weights="cohort_share")` is REFUSED when filtering removed units):** `_n_g_per_cohort` — `N_g` in W2025 Eqs. 7.4/7.6 — is read off the FINAL sample. Comparison-support filtering is the first thing in this estimator that can remove SOME units of a RETAINED cohort: `_filter_sample` keeps every row of every treated unit, and unidentified-cohort exclusion removes whole cohorts, so before this the count was always the full supplied cohort. On an **unbalanced** panel a unit observed only at unsupported periods vanishes with them, and `N_g` then describes a strictly smaller cohort than the user supplied — measured on a cohort supplied with 100 units of which 90 appear only at a dropped period: `aggregate(weights="cohort_share")` moves from 1.8078 to 3.8157. W2025 Section 7 defines `N_g` on a balanced panel and does not say whether it counts the supplied cohort or the surviving units; the two disagree materially, so `aggregate` **fails closed** naming the cohorts and unit counts rather than choosing silently (mirroring the survey + `cohort_share` refusal). `weights="cell"` never reads `N_g` and is unaffected, and the refusal cannot fire on a **balanced** panel — filtering there removes whole periods and no units — so the all-eventually-treated capability is untouched. Defining the unbalanced estimand is tracked in `TODO.md`. +- **Note (comparison-support filtering is REFUSED under `survey_design=`):** a second survey boundary alongside the unidentified-cohort refusal below, and for the same reason — deleting rows removes their PSUs and strata from the Taylor-linearized meat and from `df_survey = n_PSU − n_strata`. Raised **before any row is removed**, and strictly conditional on the filter actually dropping rows (survey fits that lose nothing are unaffected). The message points at **explicit frame restriction**, not `SurveyDesign.subpopulation()`: subpopulation zero-pads the excluded rows, which `_reject_zero_weight_groups` then refuses on the OLS path (measured: `Survey weights sum to zero for time period(s) [8, 9]`). **That workaround is exact only when every PSU and stratum survives the restriction**, which holds on a BALANCED panel — measured there: no PSU and no stratum is removed, `df_survey` is unchanged, and ATTs/SEs agree to ≤5e-15. It does NOT hold in general: on an unbalanced panel a PSU or stratum observed only at unsupported periods disappears with them (measured: restricting a 6-period frame to its 5 supported periods dropped one PSU and one stratum entirely), which changes the Taylor-linearized meat and can change `df_survey`. Before relying on the restriction, confirm the restricted frame retains every PSU and stratum of the design you specified and is the survey universe you intend; if it does not, there is no supported path until domain estimation lands. +- **Note (control-pool asymmetry on `never_treated`):** `control_group="never_treated"` restricts the comparison pool to never-treated units **on the OLS path only**, where `include_pre` emits every `(g, t)` except each cohort's reference and treated units' pre-treatment rows therefore sit in their own indicators. On `method="logit"` / `"poisson"` only post-treatment cells are emitted — including all cells would make each cohort dummy collinear with the sum of its own indicators — so treated units' pre-treatment rows ARE the identifying comparison there, exactly as under `not_yet_treated`. Measured at HEAD on `never_treated` + Poisson with never-treated units observed only at `t=1..3` and cohorts 4, 7 at `t=1..5`: the fit succeeds and perturbing only cohort-7 rows at `t∈{4,5}` moves `ATT(4,4)` and `ATT(4,5)` materially, while the OLS counterpart on a panel where that fit is defined is invariant to ~1e-15. This asymmetry is **pre-existing and structural**, not introduced by the support filter. Related: `n_control_units` counts never-treated UNITS on this setting regardless of method, so on the nonlinear paths it under-reports the rows actually doing the comparison — widening it is tracked in `TODO.md`. - **Note:** unobserved `(g, t)` pairs are skipped rather than emitted as identically-zero columns, and the skipped pairs are named in a `UserWarning` (gated on `rank_deficient_action`, so `"silent"` remains silent). - **Note (within-cohort support connectivity — OLS/unit-FE path only):** per-period support is **necessary but not sufficient** for identification. Identification runs through the unit fixed effects, so each emitted `(g, t)` cell must be connected to an **unemitted** period in the bipartite graph whose nodes are that cohort's units and periods (edge = a supported observation). Within any connected component in which *every* period is a treatment cell, those columns sum to the component's unit indicators — absorbed by the unit FE — so the block is rank-deficient and QR drops one, potentially a genuine post-treatment effect, leaving the overall ATT an average over an incomplete cell set. This is issue #724's failure mode reached through **unit** support rather than through the reference period, and per-period observation counts cannot detect it. Measured on a panel where cohort 4's units split into `{2, 4}` and `{1, 5}` groups: `g4_t5` was silently dropped and `overall_att` computed from the single surviving post cell. `fit()` now **fails closed** naming the unidentified cells (not gated on `rank_deficient_action` — that setting governs how rank warnings surface, not whether an unidentified estimand may be returned as a number). On `not_yet_treated` and the nonlinear paths pre-treatment periods are not emitted, so a component holding any pre-period is safe — which is why the condition is "contains an unemitted period", not "contains the reference". The condition applies **only to the OLS path**, whose within-transformation absorbs the unit fixed effects: `logit`/`poisson` use explicit cohort + time dummies, nothing absorbs a component's cell block, and such designs are full rank — so the check is gated on `method="ols"` and those paths estimate the same panel normally. Component-aware estimation (rather than refusal) is tracked in `TODO.md`. - **Note (unidentified-cohort exclusion is REFUSED under `survey_design=`):** the row exclusion above is naive subsetting, which under a complex survey design is not domain estimation — it removes the excluded rows' PSUs and strata from the Taylor-linearized meat and from `df_survey = n_PSU − n_strata`, so surviving ATTs would be reported with variance computed on a design the user never specified (measured on a two-stratum panel: `df_survey` 22 → 14, finite SE, no diagnostic). *Subpopulation Analysis (Phase 6)* below and Lumley (2004) §3.4 require the opposite — zero the excluded rows' weights and RETAIN the strata/PSU layout — and `SurveyDesign.subpopulation()` implements that contract, but composing it here additionally requires the weighted within-transformation to tolerate zero-weight units (shared machinery). Until that lands, `fit()` raises `NotImplementedError` when `survey_design=` is combined with an unidentified cohort, on all three methods, rather than returning a confident number from the wrong design (tracked in `TODO.md`). Relatedly, the **full** `SurveyDesign` is resolved against the **pre-exclusion** sample for validation only (weights, `weight_type`, strata, PSU, FPC, nesting): every check that ran after exclusion was blind to whatever the deleted rows contained, so invalid metadata confined to an excluded cohort used to vanish silently. @@ -1991,7 +1998,7 @@ where `g(·)` is the link inverse (logistic or exp), `η_i` is the individual li - **Note:** QMLE sandwich uses `weight_type="aweight"` which applies `(G/(G-1)) * ((n-1)/(n-k))` small-sample adjustment. Stata `jwdid` uses `G/(G-1)` only. The `(n-1)/(n-k)` term is conservative (inflates SEs slightly). For typical ETWFE panels where n >> k, the difference is negligible. *Variance families (`vcov_type`, OLS path only):* -- `hc1` (default) — CR1 Liang-Zeger cluster-robust on the within-transformed design. Bit-equal to prior behavior (FWL preserves the score). The natural R anchor is `fixest::feols(y ~ | unit + time, cluster=~unit)` or Stata `jwdid` (both within-transform). **Deviation from Stata `jwdid` (measured 2026-07-26, `tests/test_etwfe_cs_stata_parity.py`):** the ATT(g,t) POINT estimates match `jwdid` exactly (~3e-8 on the `mpdta` panel, i.e. Stata's log-output rounding), but every `hc1` SE is SMALLER than `jwdid`'s by a factor that is **uniform across cells** (spread < 1e-6 within a fit) and shrinks as the cluster count grows: 1.0280 at G=20, 1.0132 at G=40, 1.0086 at G=60, 1.0046 at G=110, 1.0010 at G=500. **Only the G=500 full-panel ratio is PINNED by CI** (`tests/test_etwfe_cs_stata_parity.py`); the smaller-G figures were measured ad hoc on subsampled panels during the #724 investigation and no committed artifact reproduces them, so treat them as the shape of the trend rather than as regression-gated constants. Committing that subsample ladder as a golden block is part of the derivation work tracked in `TODO.md` — the ladder is the instrument the derivation needs, not a separate chore. The library is therefore systematically **anti-conservative** relative to the reference - negligibly with many clusters (~0.1% at G=500) and materially in few-cluster designs (~2.8% at G=20). **The mechanism is NOT yet identified.** The gap tracks `sqrt(G/(G-1))` closely but lies consistently ABOVE it (by ~0.2% at G=20 down to ~0.001% at G=500), and `solve_ols` already applies the full CR1 `(G/(G-1)) * ((n-1)/(n-k))` - so a missing `G/(G-1)` is ruled out as the explanation. This is recorded as a measured deviation, not a diagnosed one; deriving the true factor from the within-transform `k` accounting against `hdfe`/`reghdfe`'s is tracked in `TODO.md`, and no correction should be applied until it reproduces `jwdid` exactly rather than approximately. This is distinct from the `lm + clubSandwich` deviation below, whose factor is `k`-based. `CallawaySantAnna` shows no such gap - its SEs match Stata `csdid` outright - which localizes this to the ETWFE path rather than a library-wide convention. **Deviation from R `lm + clubSandwich::vcovCR(type="CR1S")`:** the full-dummy `lm` SE differs by a factor of `sqrt((n - k_within) / (n - k_total))` because clubSandwich's `(n-1)/(n-p)` finite-sample correction counts ALL columns (intercept + treatment + unit dummies + time dummies = `k_total`) while WooldridgeDiD's `solve_ols` on the within-transformed design counts only the treatment-cell columns (`k_within`). On the 240-obs / 51-column R-parity fixture this is ~11%; on typical larger panels (n >> k_total) the gap shrinks to <2%. No public WooldridgeDiD code path exposes the `lm + CR1S` (CR1 cluster-robust on the full-dummy design) finite-sample correction — `vcov_type="hc2_bm"` routes to the CR2 Bell-McCaffrey sandwich on the full-dummy design (different variance estimator entirely), not CR1S. Users who need exact `lm + clubSandwich::vcovCR(type="CR1S")` parity must call `solve_ols` directly on a full-dummy design or fit via R. Same deviation pattern as SunAbraham PR #472 (`fixest::sunab` vs `lm + clubSandwich`). +- `hc1` (default) — CR1 Liang-Zeger cluster-robust on the within-transformed design. Bit-equal to prior behavior (FWL preserves the score). The natural R anchor is `fixest::feols(y ~ | unit + time, cluster=~unit)` or Stata `jwdid` (both within-transform). **Deviation from Stata `jwdid` (measured 2026-07-26, `tests/test_etwfe_cs_stata_parity.py`):** the ATT(g,t) POINT estimates match `jwdid` exactly (~3e-8 on the `mpdta` panel, i.e. Stata's log-output rounding), but every `hc1` SE is SMALLER than `jwdid`'s by a factor that is **uniform across cells** (spread < 1e-6 within a fit) and shrinks as the cluster count grows: 1.0280 at G=20, 1.0132 at G=40, 1.0086 at G=60, 1.0046 at G=110, 1.00264 at G=191, 1.0010 at G=500. **Two ratios are PINNED by CI** (`tests/test_etwfe_cs_stata_parity.py`): G=500 on the full `mpdta` panel, and G=191 on the all-eventually-treated arm — each arm measures its own, because the constant does not transfer between cluster counts. The remaining smaller-G figures were measured ad hoc on subsampled panels during the #724 investigation and no committed artifact reproduces them, so treat them as the shape of the trend rather than as regression-gated constants. Committing that subsample ladder as a golden block is part of the derivation work tracked in `TODO.md` — the ladder is the instrument the derivation needs, not a separate chore. The library is therefore systematically **anti-conservative** relative to the reference - negligibly with many clusters (~0.1% at G=500) and materially in few-cluster designs (~2.8% at G=20). **The mechanism is NOT yet identified.** The gap tracks `sqrt(G/(G-1))` closely but lies consistently ABOVE it (by ~0.2% at G=20 down to ~0.001% at G=500), and `solve_ols` already applies the full CR1 `(G/(G-1)) * ((n-1)/(n-k))` - so a missing `G/(G-1)` is ruled out as the explanation. This is recorded as a measured deviation, not a diagnosed one; deriving the true factor from the within-transform `k` accounting against `hdfe`/`reghdfe`'s is tracked in `TODO.md`, and no correction should be applied until it reproduces `jwdid` exactly rather than approximately. This is distinct from the `lm + clubSandwich` deviation below, whose factor is `k`-based. `CallawaySantAnna` shows no such gap - its SEs match Stata `csdid` outright - which localizes this to the ETWFE path rather than a library-wide convention. **Deviation from R `lm + clubSandwich::vcovCR(type="CR1S")`:** the full-dummy `lm` SE differs by a factor of `sqrt((n - k_within) / (n - k_total))` because clubSandwich's `(n-1)/(n-p)` finite-sample correction counts ALL columns (intercept + treatment + unit dummies + time dummies = `k_total`) while WooldridgeDiD's `solve_ols` on the within-transformed design counts only the treatment-cell columns (`k_within`). On the 240-obs / 51-column R-parity fixture this is ~11%; on typical larger panels (n >> k_total) the gap shrinks to <2%. No public WooldridgeDiD code path exposes the `lm + CR1S` (CR1 cluster-robust on the full-dummy design) finite-sample correction — `vcov_type="hc2_bm"` routes to the CR2 Bell-McCaffrey sandwich on the full-dummy design (different variance estimator entirely), not CR1S. Users who need exact `lm + clubSandwich::vcovCR(type="CR1S")` parity must call `solve_ols` directly on a full-dummy design or fit via R. Same deviation pattern as SunAbraham PR #472 (`fixest::sunab` vs `lm + clubSandwich`). - `hc2_bm` — CR2 Bell-McCaffrey via auto-route to full-dummy design (`[intercept, X_design, unit_dummies, time_dummies]`), then `solve_ols(..., vcov_type="hc2_bm")` through the clubSandwich port (PR #475). FWL does NOT preserve the hat matrix; HC2 leverage + BM DOF require the full-projection design. Per-coefficient SE matches `clubSandwich::vcovCR(lm(...), cluster=~unit, type="CR2")` at atol=1e-10. Per-cell `(g, t)` inference fields use `coef_test()$df_Satt` Bell-McCaffrey DOF (pinned at atol=1e-6 from CI half-width inversion). Aggregated inference (overall ATT + `.aggregate("group" | "calendar" | "event")`) uses contrast-specific BM DOFs from `_compute_cr2_bm_contrast_dof` (matches R `Wald_test(constraints=matrix(w, 1), vcov=vcov_CR2, test="HTZ")$df_denom`); the overall ATT contrast DOF is computed at fit time, the other three aggregations lazily on each `.aggregate(...)` call from BM artifacts (the REDUCED kept-column `X` / `cluster_ids` / bread matrix + the reduced-space coef-index map) stored on the Results object — using the reduced design after rank-deficient drops keeps the bread non-singular and matches the subspace `solve_ols` actually estimated in. Fail-closed across all surfaces: when BM DOF is unavailable (helper raises or returns non-finite), the affected inference fields are NaN — not normal-theory fallback (per `feedback_bm_contrast_dof_fail_closed`). - `classical`, `hc2` — supported via auto-route to full-dummy AND auto-drop of the unit auto-cluster (one-way families don't compose with `cluster_ids` per the linalg validator). Set `self.cluster=None` (default) for these; explicit `cluster="state"` + one-way family raises at the linalg validator. SE matches `summary(lm(...))$coefficients` (classical) and `sandwich::vcovHC(type="HC2")` respectively. Per-cell + aggregate p-values/CIs use the residual DOF `n - rank(X)` (matches R `lm()` / `coef_test()` t-distribution under both classical OLS SE and `sandwich::vcovHC` defaults) — not normal-theory, so inference is correct under small samples. - `conley` (spatial-HAC, Conley 1999) — supported on the **OLS path** via the within-transform design (or the full-dummy design when `cohort_trends=True`, like the other full-dummy families — see the cohort-trends row below), threading the `conley_*` params through `solve_ols` / `conley.py` (`conley_lag_cutoff=0` = within-period spatial only; `>0` adds within-unit Bartlett serial — the panel-aware path, since `conley_time`/`conley_unit` are always supplied, not pooled cross-sectional). Reuses the already-`conleyreg`-validated machinery (no new variance code). The unit auto-cluster is dropped on the conley path (an explicit `cluster=` enables the spatial+cluster product kernel); `survey_design=` / `weights` / `n_bootstrap>0` are rejected, and `method ∈ {logit, poisson}` + conley remains rejected (the `method != "ols"` guard — a QMLE-on-pseudo-residuals Conley sandwich is a separate derivation). FWL-composability (the within-transform conley SE equals the full-dummy conley SE) is pinned in `tests/test_conley_vcov.py::TestConleyWooldridge::test_fwl_composability_vs_full_dummy`. @@ -2025,7 +2032,7 @@ where `g(·)` is the link inverse (logistic or exp), `η_i` is the individual li *Control groups:* - `not_yet_treated` (default): Control pool includes units not yet treated at time t (same as Callaway-Sant'Anna) -- `never_treated`: Control pool restricted to never-treated units only +- `never_treated`: Control pool restricted to never-treated units **on the OLS path only**. See the control-pool asymmetry Note below — on the nonlinear paths this setting does NOT restrict the comparison to never-treated units. *Edge cases:* - Single cohort (no staggered adoption): Reduces to standard 2×2 DiD @@ -2077,7 +2084,7 @@ where `g(·)` is the link inverse (logistic or exp), `η_i` is the individual li - **Note:** Polynomial-trend extensions (`"quadratic"`, `"cubic"` per paper p. 2572 footnote) are NOT yet exposed — `cohort_trends` is a binary `True/False` flag for linear `dg_i · t` only. - **Note:** `cohort_trends=True` + `survey_design` is **NOT yet supported** (raises `NotImplementedError` at `fit()`). The full-dummy auto-route composed with the survey TSL variance has not been validated against R-parity goldens. Tracked in DEFERRED.md follow-up. - **Note:** `cohort_trends=True` + `control_group="never_treated"` is **NOT yet supported** (raises `NotImplementedError` at `fit()`). The OLS + never_treated branch emits the `(g, t)` placebo cells (paper W2025 Section 4.4 placebo coverage) MINUS each cohort's `ref(g)` reference cell (W2025 Eq. 6.1/6.4; see the reference-period Note below). The appended `dg_i · t` trend columns are still spanned — jointly by the emitted cells and the unit fixed effects, which absorb `1{cohort=g}` and thereby recover the omitted reference — so the Section 8 trend specification remains unidentified on this branch. Use `control_group="not_yet_treated"` (the default) for `cohort_trends=True`. Tracked in DEFERRED.md follow-up. -- **Note:** Identification + baseline normalization for `cohort_trend_coefs` on all-eventually-treated panels: when a never-treated cohort (`g = 0`) is present, **all** `G` treated cohorts get a `dg_i · t` interaction column and `cohort_trend_coefs[g]` reports each cohort's linear slope relative to the never-treated baseline (absorbed by time FE). When **no** never-treated cohort exists, the **last cohort's** trend column is dropped deterministically per paper W2025 Section 5.4 ("all variables in regression (5.3) involving `dT_i` get dropped"); that cohort serves as the trend baseline, and `cohort_trend_coefs` surfaces `G - 1` entries (the last cohort is absent — its slope is the baseline in deviation form). **Note (deviation from W2025 Section 5.4):** the paper's rule is applied to the TREND columns only. The matching normalization for the cohort x time CELLS is NOT implemented, so on an all-eventually-treated panel the cells at fully-treated periods are jointly collinear with the time FE and `fit()` RAISES (the completeness gate, ledger `M-124`) rather than returning the relabeled contrasts it used to. The trend-drop path described here is therefore currently unreachable end-to-end; it is retained because implementing the cell half is what restores these fits (tracked in `TODO.md`, High). +- **Note:** Identification + baseline normalization for `cohort_trend_coefs` on all-eventually-treated panels: when a never-treated cohort (`g = 0`) is present, **all** `G` treated cohorts get a `dg_i · t` interaction column and `cohort_trend_coefs[g]` reports each cohort's linear slope relative to the never-treated baseline (absorbed by time FE). When **no** never-treated cohort exists, the **last cohort's** trend column is dropped deterministically per paper W2025 Section 5.4 ("all variables in regression (5.3) involving `dT_i` get dropped"); that cohort serves as the trend baseline, and `cohort_trend_coefs` surfaces `G - 1` entries (the last cohort is absent — its slope is the baseline in deviation form). **Note (deviation from W2025 Section 5.4 — narrowed to the covariate block):** Section 5.4's "all variables in regression (5.3) involving `dT_i` get dropped" governs four regressor families: the cells `dT·f_t`, the cell x covariate block `dT·f_t·ẋ`, the cohort trend `dT·t`, and the cohort x covariate block `D_T × X`. The library implements the first three — comparison-support filtering (below) removes the periods at which no unit is untreated, so cohort `G_max` receives no cells and no trend column, and the trend-drop path described here is reachable end-to-end. **The `D_{G_max} × X` normalization is NOT applied:** when covariates are supplied, `D_g × X` is still built for every cohort including `G_max`, so `Σ_g (D_g · x) = x` and the design is rank-deficient. Coefficients are unaffected (QR's arbitrary pick spans the same column space, `max |Δ| = 1.35e-14`), but `rank_deficient_action="error"` raises and `"warn"` drops an arbitrarily-chosen cohort's column rather than deterministically the last. Tracked in `TODO.md`; note for whoever takes it that `xtvar` under the default `demean_covariates=True` is FULL RANK (the raw block carries demeaned values while `D_g × X` carries raw ones, so the dependency does not arise) and forcing the drop there moves `overall_att` 1.11903 → 1.46269. ### Deviations from the paper / from R / library extensions diff --git a/docs/methodology/papers/wooldridge-2025-review.md b/docs/methodology/papers/wooldridge-2025-review.md index d9044c13..ceecba00 100644 --- a/docs/methodology/papers/wooldridge-2025-review.md +++ b/docs/methodology/papers/wooldridge-2025-review.md @@ -742,7 +742,7 @@ For nonlinear extensions (logit, Poisson, fractional outcomes, ASF counterfactua 10. **Heterogeneous-trends interpretation when pre-trends bleed into post-treatment.** The paper warns (Eq. 6.7 commentary): "If the violation of PT carries into the treated periods, including the pre-treatment indicators can actually exacerbate the bias compared with not including them." This is a methodological caveat about both leads-and-lags and the heterogeneous-trends Section 8 specification, not currently surfaced in REGISTRY. -11. **All-cohort-treated mechanics.** Section 5.4 explains that when all units are eventually treated and the last cohort `T` serves as control, "all variables in regression (5.3) (or its TWFE version) involving `dT_i` get dropped." The shipped library applies this rule to the cohort-TREND columns only; the matching drop for the cohort x time CELLS is not implemented, so `control_group="not_yet_treated"` on an all-eventually-treated panel now RAISES rather than estimating (previously it returned relabeled cohort contrasts under ATT(g,t) labels). Implementing the cell half is the open work. +11. **All-cohort-treated mechanics.** Section 5.4 explains that when all units are eventually treated and the last cohort `T` serves as control, "all variables in regression (5.3) (or its TWFE version) involving `dT_i` get dropped." That sentence governs four regressor families: the cells `dT·f_t`, the cell x covariate block `dT·f_t·ẋ`, the cohort trend `dT·t`, and the cohort x covariate block `D_T × X`. The shipped library applies it to the first three: comparison-support filtering removes the periods at which no unit is untreated, so cohort `T` receives no cells and no trend column, and `control_group="not_yet_treated"` on an all-eventually-treated panel ESTIMATES (verified against Stata `jwdid` on the `mpdta` panel with never-treated counties dropped: identical cell set, identical `N` = 764 of 955, ATTs agreeing to ~1e-15). The `D_{G_max} × X` covariate normalization is NOT applied — the remaining open work, which surfaces only when covariates are supplied. 12. **Cohort centering for `dg_i · x_i` interactions.** The paper notes (p. 2560) that centering `dg_i · x_i` around `x̄_g` makes the `dg_i` coefficient an "average selection effect" estimate. This is currently not exposed as a user-facing parameter in the shipped library; the existing implementation does NOT center the `dg_i · x_i` interactions by default (the cohort-centering applies only to the `w_it · dg_i · fs_t · ẍ_ig` treatment-interaction term per Eq. 5.2). This is an interpretive nuance, not a numerical issue for ATT estimates. diff --git a/docs/survey-roadmap.md b/docs/survey-roadmap.md index 408b6c95..222ce7cc 100644 --- a/docs/survey-roadmap.md +++ b/docs/survey-roadmap.md @@ -242,6 +242,13 @@ Logit/Poisson use survey-weighted IRLS + X_tilde linearization for TSL vcov. Replicate-weight designs raise `NotImplementedError`; bootstrap + survey is rejected. +Two further combinations raise rather than subsetting the frame in place: +comparison-support filtering (periods with no untreated unit) and +unidentified-cohort exclusion. Both would delete rows, which under a complex +design removes their PSUs and strata from the variance — see the Current +Limitations table. The refusals are conditional: a survey fit that drops +nothing is unaffected. + ### 10g. Practitioner Guidance ✅ Subsumed by the practitioner decision tree @@ -268,6 +275,8 @@ the limitation and suggested alternative. | ImputationDiD | `pretrend_test()` + replicate weights | Use analytical survey design instead | | DiD, TWFE | `inference='wild_bootstrap'` + `survey_design` | Use analytical survey inference (default) | | EfficientDiD | `cluster` + `survey_design` | Use `survey_design` with PSU/strata | +| WooldridgeDiD | Unsupported-period filtering + `survey_design` | Restrict the frame to the supported periods explicitly and re-fit. Deleting rows in-place is naive subsetting: it removes their PSUs and strata from the TSL meat and from `df_survey = n_PSU - n_strata`. Exact only if every PSU and stratum survives the restriction (true on a balanced panel; NOT in general — an unbalanced frame can hold a PSU observed only at unsupported periods). Verify before relying on it. | +| WooldridgeDiD | Unidentified-cohort exclusion + `survey_design` | Same reason (ledger `M-123`). Drop the cohort from the frame yourself, or supply a panel where every cohort has a pre-treatment period. | | All bootstrap estimators | Bootstrap + replicate weights | These are alternative variance methods; pick one | **Warning/fallback (no error):** MultiPeriodDiD with `wild_bootstrap` + diff --git a/docs/tutorials/16_wooldridge_etwfe.ipynb b/docs/tutorials/16_wooldridge_etwfe.ipynb index fb279a2d..133c9ddd 100644 --- a/docs/tutorials/16_wooldridge_etwfe.ipynb +++ b/docs/tutorials/16_wooldridge_etwfe.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "id": "a1b2c3d4", "metadata": {}, - "source": "# Wooldridge Extended Two-Way Fixed Effects (ETWFE)\n\nThis tutorial demonstrates the `WooldridgeDiD` estimator (alias: `ETWFE`), which implements Wooldridge's (2021, 2023) Extended Two-Way Fixed Effects approach — the basis of the Stata `jwdid` package.\n\n**What ETWFE does:** Estimates cohort×time Average Treatment Effects (ATT(g,t)) via a single saturated regression that interacts treatment indicators with cohort×time cells. Unlike standard TWFE, it correctly handles heterogeneous treatment effects across cohorts and time periods. The key insight is to include all cohort×time interaction terms simultaneously, with unit and time fixed effects absorbed via within-transformation.\n\n**Key features:**\n- Follows the Stata `jwdid` specification (OLS and nonlinear paths; see Methodology Registry for documented SE/aggregation deviations)\n- Supports **linear (OLS)**, **Poisson**, and **logit** link functions\n- Nonlinear ATTs use the Average Structural Function (ASF): E[f(η₁)] − E[f(η₀)]\n- Delta-method standard errors for all aggregations\n- Cluster-robust sandwich variance\n\n**Topics covered:**\n1. Basic OLS estimation\n2. Cohort×time cell estimates ATT(g,t)\n3. Aggregation: event-study, group, simple\n4. Poisson QMLE for count / non-negative outcomes\n5. Logit for binary outcomes\n6. Comparison with Callaway-Sant'Anna\n7. Parameter reference and guidance\n\n*Prerequisites: [Tutorial 02](02_staggered_did.ipynb) (Staggered DiD).*\n\n*See also: [Tutorial 15](15_efficient_did.ipynb) for Efficient DiD, [Tutorial 11](11_imputation_did.ipynb) for Imputation DiD.*" + "source": "# Wooldridge Extended Two-Way Fixed Effects (ETWFE)\n\nThis tutorial demonstrates the `WooldridgeDiD` estimator (alias: `ETWFE`), which implements Wooldridge's Extended Two-Way Fixed Effects approach — the linear design from Wooldridge (2025) and the nonlinear paths from Wooldridge (2023) — the basis of the Stata `jwdid` package.\n\n**What ETWFE does:** Estimates cohort×time Average Treatment Effects (ATT(g,t)) via a single saturated regression that interacts treatment indicators with cohort×time cells. Unlike standard TWFE, it correctly handles heterogeneous treatment effects across cohorts and time periods. The key insight is to include all cohort×time interaction terms simultaneously, with unit and time fixed effects absorbed via within-transformation.\n\n**Key features:**\n- Follows the Stata `jwdid` specification (OLS and nonlinear paths; see Methodology Registry for documented SE/aggregation deviations)\n- Supports **linear (OLS)**, **Poisson**, and **logit** link functions\n- Nonlinear ATTs use the Average Structural Function (ASF): E[f(η₁)] − E[f(η₀)]\n- Delta-method standard errors for all aggregations\n- Cluster-robust sandwich variance\n\n**Topics covered:**\n1. Basic OLS estimation\n2. Cohort×time cell estimates ATT(g,t)\n3. Aggregation: event-study, group, simple\n4. Poisson QMLE for count / non-negative outcomes\n5. Logit for binary outcomes\n6. Comparison with Callaway-Sant'Anna\n7. All-eventually-treated panels (no never-treated group)\n8. Parameter reference and guidance\n\n*Prerequisites: [Tutorial 02](02_staggered_did.ipynb) (Staggered DiD).*\n\n*See also: [Tutorial 15](15_efficient_did.ipynb) for Efficient DiD, [Tutorial 11](11_imputation_did.ipynb) for Imputation DiD.*" }, { "cell_type": "code", @@ -427,11 +427,68 @@ " print(\"Install matplotlib to see the comparison plot: pip install matplotlib\")" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## All-Eventually-Treated Panels (no never-treated group)\n", + "\n", + "Many applied panels have **no never-treated units** — every unit is eventually\n", + "treated, just at different times. At the late periods where *everyone* is\n", + "treated there is no untreated outcome left to difference against, so those\n", + "`ATT(g, t)` are not identified.\n", + "\n", + "Wooldridge (2025) Section 5.4 gives the answer: the **last cohort serves as the\n", + "reference**, and \"all variables in regression (5.3) involving `dT_i` get\n", + "dropped\". `WooldridgeDiD` implements that — it removes the unsupported periods\n", + "before estimating and tells you it did.\n", + "\n", + "Use `control_group=\"not_yet_treated\"` (the default); `\"never_treated\"` raises\n", + "when there are no never-treated units to use.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import warnings\n", + "\n", + "# Three cohorts, no never-treated group: everyone is treated by t=8.\n", + "rng = np.random.default_rng(7)\n", + "rows = []\n", + "for u in range(200):\n", + " g = 3 if u < 70 else (5 if u < 140 else 8)\n", + " for t in range(1, 10):\n", + " rows.append({\"unit\": u, \"time\": t, \"cohort\": g,\n", + " \"y\": rng.standard_normal() + 1.5 * (t >= g)})\n", + "all_treated = pd.DataFrame(rows)\n", + "\n", + "with warnings.catch_warnings(record=True) as caught:\n", + " warnings.simplefilter(\"always\")\n", + " res_at = WooldridgeDiD(control_group=\"not_yet_treated\").fit(\n", + " all_treated, outcome=\"y\", unit=\"unit\", time=\"time\", cohort=\"cohort\"\n", + " )\n", + "\n", + "for w in caught:\n", + " print(f\"WARNING: {w.message}\\n\")\n", + "\n", + "print(f\"overall ATT = {res_at.overall_att:.4f} (true effect: 1.5)\")\n", + "print(f\"estimated cohorts = {sorted(int(g) for g in res_at.groups)}\")\n", + "print(f\"cells = {sorted((int(g), int(t)) for g, t in res_at.group_time_effects)}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "Two things to read from that output.\n\n**The estimation sample shrank, and you were told.** Periods 8 and 9 have no\nuntreated unit, so their rows are dropped before the solve. Stata's `jwdid`\nperforms the same reduction but reports only a smaller `N` — here the periods,\nthe row count and the reason are all stated. If you are wrapping this in a\npipeline, surface these warnings.\n\n**Cohort 8 has no estimates.** It is the reference — the role a never-treated\ngroup would otherwise play — so it receives no cells and is excluded from\n`results.groups`. That is the Section 5.4 normalization, not a failure.\n\nThe retained ATTs are the **absolute** effects (≈ 1.5 here), not effects\nrelative to the last cohort, because cohort 8 is untreated throughout every\nperiod that survives.\n\n> **Covariates on these panels.** `exovar` / `xgvar`, and `xtvar` with\n> `demean_covariates=False`, are still rank-deficient here: `D_g × X` is built\n> for the last cohort too, which the paper's rule would drop. Coefficients are\n> unaffected, but `rank_deficient_action=\"error\"` will raise. **Default `xtvar`\n> (`demean_covariates=True`) is full rank** and fits normally. Tracked as\n> follow-up work.\n" + }, { "cell_type": "markdown", "id": "f0a1b2c3", "metadata": {}, - "source": "## Summary\n\n**Key takeaways:**\n\n1. **ETWFE via a single regression**: all ATT(g,t) cells estimated jointly, not separately — computationally efficient and internally consistent\n2. **OLS path** follows the Stata `jwdid` specification: unit + time FEs (absorbed via within-transformation), treatment interaction dummies\n3. **Nonlinear paths** (Poisson, Logit) use the ASF formula: E[f(η₁)] − E[f(η₀)] — the only valid ATT definition for nonlinear models\n4. **Four aggregations** mirror Stata's `estat` commands: event, group, calendar, simple\n5. **Delta-method SEs** for all aggregations, including nonlinear paths\n6. **When to prefer ETWFE**: nonlinear outcomes, or when a single-regression framework is preferred\n7. **When to prefer CS/ImputationDiD**: covariate adjustment via IPW/DR, or multiplier bootstrap inference\n\n**Parameter reference:**\n\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `method` | `'ols'` | `'ols'`, `'poisson'`, or `'logit'` |\n| `control_group` | `'not_yet_treated'` | `'not_yet_treated'` or `'never_treated'` |\n| `anticipation` | `0` | Anticipation periods before treatment |\n| `alpha` | `0.05` | Significance level |\n| `cluster` | `None` | Column for clustering (default: unit variable) |\n\n**References:**\n- Wooldridge, J. M. (2021). Two-Way Fixed Effects, the Two-Way Mundlak Regression, and Difference-in-Differences Estimators. *SSRN 3906345*.\n- Wooldridge, J. M. (2023). Simple approaches to nonlinear difference-in-differences with panel data. *The Econometrics Journal*, 26(3), C31–C66.\n- Friosavila, F. (2021). `jwdid`: Stata module for ETWFE. SSC s459114.\n\n*See also: [Tutorial 02](02_staggered_did.ipynb) for Callaway-Sant'Anna, [Tutorial 15](15_efficient_did.ipynb) for Efficient DiD.*" + "source": "## Summary\n\n**Key takeaways:**\n\n1. **ETWFE via a single regression**: all ATT(g,t) cells estimated jointly, not separately — computationally efficient and internally consistent\n2. **OLS path** follows the Stata `jwdid` specification: unit + time FEs (absorbed via within-transformation), treatment interaction dummies\n3. **Nonlinear paths** (Poisson, Logit) use the ASF formula: E[f(η₁)] − E[f(η₀)] — the only valid ATT definition for nonlinear models\n4. **Four aggregations** mirror Stata's `estat` commands: event, group, calendar, simple\n5. **Delta-method SEs** for all aggregations, including nonlinear paths\n6. **All-eventually-treated panels** estimate via the Section 5.4 normalization: the last cohort is the reference, unsupported periods are dropped, and the reduction is always reported\n7. **When to prefer ETWFE**: nonlinear outcomes, or when a single-regression framework is preferred\n8. **When to prefer CS/ImputationDiD**: covariate adjustment via IPW/DR, or multiplier bootstrap inference\n\n**Parameter reference:**\n\n| Parameter | Default | Description |\n|-----------|---------|-------------|\n| `method` | `'ols'` | `'ols'`, `'poisson'`, or `'logit'` |\n| `control_group` | `'not_yet_treated'` | `'not_yet_treated'` or `'never_treated'` |\n| `anticipation` | `0` | Anticipation periods before treatment |\n| `alpha` | `0.05` | Significance level |\n| `cluster` | `None` | Column for clustering (default: unit variable) |\n\n**References:**\n- Wooldridge, J. M. (2025). Two-Way Fixed Effects, the Two-Way Mundlak Regression, and Difference-in-Differences Estimators. *Empirical Economics*, 69(5), 2545–2587. Published version of SSRN 3906345 / NBER Working Paper 29154; cited as Wooldridge (2025) throughout this tutorial, including Eq. 6.1/6.4 (reference period) and Section 5.4 (all-eventually-treated panels).\n- Wooldridge, J. M. (2023). Simple approaches to nonlinear difference-in-differences with panel data. *The Econometrics Journal*, 26(3), C31–C66.\n- Friosavila, F. (2021). `jwdid`: Stata module for ETWFE. SSC s459114.\n\n*See also: [Tutorial 02](02_staggered_did.ipynb) for Callaway-Sant'Anna, [Tutorial 15](15_efficient_did.ipynb) for Efficient DiD.*" } ], "metadata": { diff --git a/docs/v4-deprecations.yaml b/docs/v4-deprecations.yaml index f4353621..2d013e4d 100644 --- a/docs/v4-deprecations.yaml +++ b/docs/v4-deprecations.yaml @@ -1327,6 +1327,19 @@ rows: test_ref: tests/test_aggregate_contract.py code_refs: [diff_diff/staggered.py, diff_diff/staggered_results.py, diff_diff/aggregation.py] notes: "balance_e moves from fit() onto aggregate() with [M-020]. Previously tracked only as the prose 'balance_e moves to aggregate() in the same PR' inside M-020's notes, which no test could assert - the same un-rowed-obligation class the gating-completeness amendment closed. Applies to event-study aggregation ONLY (the shipped code threads it nowhere else), so aggregate(type='simple'|'group', balance_e=...) raises rather than silently ignoring it. The other three balance_e sites (ImputationDiD, TwoStageDiD, EfficientDiD) get their own rows in the PRs that migrate them." + - id: M-125 + kind: behavior + group: etwfe-reference-period + old: "diff_diff.wooldridge:WooldridgeDiD.fit" + new: null + introduced_in: "3.9" + deprecated_in: null + removed_in: null + status: done + phase: 2 + test_ref: tests/test_wooldridge.py + code_refs: [diff_diff/wooldridge.py, docs/methodology/REGISTRY.md, tests/test_etwfe_cs_stata_parity.py] + notes: "Per-period COMPARISON-SUPPORT filtering: WooldridgeDiD.fit() now removes periods at which no unit is untreated from the estimation sample BEFORE the solve, instead of carrying them into a rank-deficient design. This is the cell half of W2025 Section 5.4 - with no never-treated group the last cohort serves as the reference, and 'all variables in regression (5.3) involving dT_i get dropped' - so all-eventually-treated panels ESTIMATE again, resolving the capability regression [M-124] recorded. Measured on a {3,5,8} panel over t=1..9 with a true ATT of 1.5: cells (3,3..7) and (5,5..7), overall_att=1.5502, cohort 8 receiving none. Anchored externally to Stata jwdid on the mpdta panel with never-treated counties dropped (golden block jwdid_alltreated): identical cell set, identical N (764 of 955), ATTs agreeing to ~1e-15. The eligible set is keyed on the regression BASELINE, not the method: on never_treated + OLS only never-treated rows qualify, because include_pre emits every (g,t) except each cohort's reference so a later cohort's rows sit in their own indicator; elsewhere not-yet-treated rows qualify too. Omitted reference cells deliberately do NOT count as support - they sit in the baseline but do not identify the period, since the cohort indicator is absorbed by the unit FE and the period dummy stays reproducible from the emitted cells (measured: retaining a period on the strength of ref(6)=5 raises on the lost (6,1), while dropping it fits at overall_att=1.0171). SAMPLE REDUCTION IS ALWAYS REPORTED, in two parts and neither gated on rank_deficient_action: at filter time the periods, observation count and branch-correct cause (emitted there so it survives a downstream raise), and after the final build any cohort left with no cells or stripped of every row (read from the FINAL cell set so a cohort that unidentified-cohort exclusion also removes is not double-reported). Stata jwdid performs the same reduction silently, reporting only a smaller N; being explicit is a deliberate improvement, not a numerical deviation. REFERENCE MOVEMENT: on the not_yet_treated branch a reference can never be filtered (eligibility t < g - anticipation IS the predicate's own second disjunct) and the estimator asserts this at runtime; on never_treated + OLS it CAN move, which renormalizes that cohort's ATTs against a different baseline period, so that case warns naming the cohort and both periods rather than raising. RESULTS METADATA: results.groups and _n_g_per_cohort are now derived from the emitted cell set so a cohort retained only as a control is not advertised as estimated; cohort_trend_coefs stays keyed on PRESENT cohorts because the trend baseline is exactly the zero-cell cohort. The design-side readers of the cohort list (covariate blocks, trend block) deliberately keep the present-cohort list - routing them through the cells-derived one would drop D_{G_max} x X and silently apply a covariate normalization this row does NOT include (measured: 8/8 ATTs move, max 0.0648). Under survey_design= the filter REFUSES rather than subsetting, conditional on rows actually dropping, for the same reason as [M-123]. COHORT-SHARE BOUNDARY: `_n_g_per_cohort` (N_g, Eqs. 7.4/7.6) is read off the FINAL sample, and this filter is the first thing in the estimator that can remove SOME units of a RETAINED cohort - `_filter_sample` keeps every row of every treated unit and unidentified-cohort exclusion removes whole cohorts, so before this N_g was always the full supplied cohort. On an UNBALANCED panel a unit observed only at unsupported periods vanishes with them and N_g then describes a strictly smaller cohort (measured: 90 of a cohort's 100 units observed only at a dropped period moves aggregate(weights='cohort_share') from 1.8078 to 3.8157). W2025 Section 7 assumes a balanced panel and does not say which reading applies, so aggregate() FAILS CLOSED naming the cohorts and unit counts rather than choosing silently, mirroring the existing survey + cohort_share refusal. weights='cell' never reads N_g and is unaffected; a BALANCED panel drops whole periods and no units, so the all-eventually-treated capability this row delivers cannot trip it. Defining the unbalanced estimand is tracked in TODO.md. SURVEY WORKAROUND IS CONDITIONAL: the refusal message points at explicit frame restriction, which is exact only when every PSU and stratum survives the restriction - true on a balanced panel, NOT in general (measured: restricting a 6-period unbalanced frame to its 5 supported periods removed one PSU and one stratum outright). REGISTRY, the survey roadmap and the error message all state that condition rather than claiming equivalence. NOT INCLUDED, tracked in TODO.md: the D_{G_max} x X covariate normalization, so covariates on such a panel remain rank-deficient (coefficients unaffected, but rank_deficient_action='error' raises); and an opt-out parameter for the filtering. introduced_in gates the 3.9 cut; deprecated_in stays null so the early-flip guard does not fire against the PR that ships it." - id: M-124 kind: behavior group: etwfe-reference-period @@ -1339,7 +1352,7 @@ rows: phase: 2 test_ref: tests/test_wooldridge.py code_refs: [diff_diff/wooldridge.py, docs/methodology/REGISTRY.md] - notes: "WooldridgeDiD.fit() now FAILS CLOSED whenever it cannot produce a finite overall ATT POINT ESTIMATE, instead of returning a result object whose overall_att is NaN - which reads as a completed estimate and silently defeats downstream aggregation and plotting. Scope is the POINT ESTIMATE only: overall_se and the inference fields may still be NaN on a successful fit when the VARIANCE alone is unidentifiable (hc2_bm with no Bell-McCaffrey DOF; aggregate(weights='cohort_share')), both of which fail closed through safe_inference() by design - so the guard checks overall_att and nothing else (codex R6 DT-1 corrected an earlier overstatement of this row). Enforced as ONE invariant at two levels rather than as per-symptom patches. (1) BUILD time: the pre-3.9 guard tested `X_int.shape[1] == 0`, strictly weaker than the error message it carried, because on the never_treated + OLS path `include_pre` also emits placebo (t < g) columns - so a cohort observed only before its own treatment start still produced a NON-empty design; it now conditions on post-treatment cells (t >= g - anticipation). (2) POST-SOLVE, in `_require_estimable_overall_att`, called before every WooldridgeDiDResults construction (OLS + both nonlinear fitters), which catches what the build-time check cannot see: coefficients removed by rank reduction, and cells that exist but all fall inside the anticipation window. The two levels are NOT redundant - the build-time notion of a treatment cell (t >= g - anticipation) is deliberately WIDER than the aggregation's (t >= g, anticipation leads excluded per W2025), so each catches cases the other misses. Three causes are distinguished with their own messages: no cell survived the solve; cells survived but none has t >= g; post cells survived with NaN coefficients. Surfaced by codex review R2 (finding M1). BOTH post-solve cases were verified PRE-EXISTING - identical all-NaN return, same warnings, at the pre-PR base commit 7e4026a0 - so this row records a deliberate fix of long-standing behavior, not a regression introduced by the reference-period work. Notably a design with TWO surviving treated cohorts can still be fully degenerate (neither post cell has a same-period comparison), so cohort count is not a valid proxy for comparison support; pinned by tests/test_wooldridge.py::TestOverallAttFailsClosed. Separate row from [M-123] because the trigger differs: M-123 changes WHICH ROWS are estimated (unidentified-cohort exclusion), this changes whether a degenerate design RETURNS or RAISES, and it fires on configurations with no unidentified cohort at all. introduced_in gates the 3.9 cut; deprecated_in stays null so the early-flip guard does not fire against the PR that ships it. No opt-out: the alternative is an all-NaN result presented as a successful fit. NOT ALL BRANCHES ARE TERMINAL: the 'no cell has t >= g' branch (every treatment cell inside the anticipation window) discards per-cell ATTs that ARE identified, and the 'post cells all NaN' branch is a post-hoc substitute for the same-period comparison-support diagnostic the estimator lacks. Both are STOPGAPS with follow-up rows in TODO.md; the invalid-weight and no-post-treatment-cell branches are terminal (genuinely unidentified). COMPLETENESS GATE (codex R8, P0): a finite overall ATT does NOT imply the reported cell set is complete. When rank reduction removes SOME treatment columns the survivors are silently re-identified as whatever contrast the reduced design supports and KEEP their ATT(g,t) labels - measured on an all-eventually-treated panel with a true ATT of 1.5 everywhere: ATT(3,8)=-0.0096, ATT(5,9)=-0.2393, overall_att=1.0023, and with rank_deficient_action='silent' no warning at all. _require_complete_cell_set now compares the cells the builder emitted against the coefficients that survived and fails closed on any loss, not gated on rank_deficient_action (that setting governs how rank warnings surface, not whether a mislabeled contrast may be returned as an ATT). CAPABILITY REGRESSION, deliberate: all-eventually-treated panels (no never-treated group) now RAISE where they previously returned those wrong numbers, including under survey designs and cohort_trends=True. W2025 Section 5.4 defines the correct treatment (last cohort serves as control, its columns dropped deliberately rather than by QR); the library already applies it to the cohort-TREND column but not to the cells, so implementing the cell half restores the capability - tracked in TODO.md at High priority." + notes: "WooldridgeDiD.fit() now FAILS CLOSED whenever it cannot produce a finite overall ATT POINT ESTIMATE, instead of returning a result object whose overall_att is NaN - which reads as a completed estimate and silently defeats downstream aggregation and plotting. Scope is the POINT ESTIMATE only: overall_se and the inference fields may still be NaN on a successful fit when the VARIANCE alone is unidentifiable (hc2_bm with no Bell-McCaffrey DOF; aggregate(weights='cohort_share')), both of which fail closed through safe_inference() by design - so the guard checks overall_att and nothing else (codex R6 DT-1 corrected an earlier overstatement of this row). Enforced as ONE invariant at two levels rather than as per-symptom patches. (1) BUILD time: the pre-3.9 guard tested `X_int.shape[1] == 0`, strictly weaker than the error message it carried, because on the never_treated + OLS path `include_pre` also emits placebo (t < g) columns - so a cohort observed only before its own treatment start still produced a NON-empty design; it now conditions on post-treatment cells (t >= g - anticipation). (2) POST-SOLVE, in `_require_estimable_overall_att`, called before every WooldridgeDiDResults construction (OLS + both nonlinear fitters), which catches what the build-time check cannot see: coefficients removed by rank reduction, and cells that exist but all fall inside the anticipation window. The two levels are NOT redundant - the build-time notion of a treatment cell (t >= g - anticipation) is deliberately WIDER than the aggregation's (t >= g, anticipation leads excluded per W2025), so each catches cases the other misses. Three causes are distinguished with their own messages: no cell survived the solve; cells survived but none has t >= g; post cells survived with NaN coefficients. Surfaced by codex review R2 (finding M1). BOTH post-solve cases were verified PRE-EXISTING - identical all-NaN return, same warnings, at the pre-PR base commit 7e4026a0 - so this row records a deliberate fix of long-standing behavior, not a regression introduced by the reference-period work. Notably a design with TWO surviving treated cohorts can still be fully degenerate (neither post cell has a same-period comparison), so cohort count is not a valid proxy for comparison support; pinned by tests/test_wooldridge.py::TestOverallAttFailsClosed. Separate row from [M-123] because the trigger differs: M-123 changes WHICH ROWS are estimated (unidentified-cohort exclusion), this changes whether a degenerate design RETURNS or RAISES, and it fires on configurations with no unidentified cohort at all. introduced_in gates the 3.9 cut; deprecated_in stays null so the early-flip guard does not fire against the PR that ships it. No opt-out: the alternative is an all-NaN result presented as a successful fit. NOT ALL BRANCHES ARE TERMINAL: the 'no cell has t >= g' branch (every treatment cell inside the anticipation window) discards per-cell ATTs that ARE identified, and the 'post cells all NaN' branch is a post-hoc substitute for the same-period comparison-support diagnostic the estimator lacks. Both are STOPGAPS with follow-up rows in TODO.md; the invalid-weight and no-post-treatment-cell branches are terminal (genuinely unidentified). COMPLETENESS GATE (codex R8, P0): a finite overall ATT does NOT imply the reported cell set is complete. When rank reduction removes SOME treatment columns the survivors are silently re-identified as whatever contrast the reduced design supports and KEEP their ATT(g,t) labels - measured on an all-eventually-treated panel with a true ATT of 1.5 everywhere: ATT(3,8)=-0.0096, ATT(5,9)=-0.2393, overall_att=1.0023, and with rank_deficient_action='silent' no warning at all. _require_complete_cell_set now compares the cells the builder emitted against the coefficients that survived and fails closed on any loss, not gated on rank_deficient_action (that setting governs how rank warnings surface, not whether a mislabeled contrast may be returned as an ATT). The capability regression this gate originally carried has since been RESOLVED by [M-125]: all-eventually-treated panels briefly raised here, and now estimate again via per-period comparison-support filtering, which applies the W2025 Section 5.4 rule to the cohort x time CELLS as well as the trend column. This gate remains as the backstop for losses the period filter cannot see - cohorts sharing no comparison period, and covariate collinearity. The cohort-count observation above still holds and is still pinned by tests/test_wooldridge.py::TestOverallAttFailsClosed::test_two_cohorts_without_same_period_controls_fail_closed, which was verified UNAFFECTED by the filter (that panel loses only t=5 and still raises on the lost cell)." - id: M-123 kind: behavior group: etwfe-reference-period @@ -1352,7 +1365,7 @@ rows: phase: 2 test_ref: tests/test_wooldridge.py code_refs: [diff_diff/wooldridge.py, docs/methodology/REGISTRY.md, tests/test_etwfe_cs_stata_parity.py] - notes: "ETWFE reference-period normalization (issue #724). The saturated design now OMITS each cohort's reference cell explicitly - ref(g) = latest supported period before g - anticipation, per-cohort and positive-weight - per Wooldridge 2025 Eq. 6.1/6.4, anchored to Stata jwdid ... never. Previously the reference was emitted and generic QR rank detection dropped an arbitrary column, which on real mpdta silently removed ATT(2004,2004) and ATT(2006,2007) AND left every surviving coefficient measured against the wrong reference (errors of 0.010-0.041, up to 8x the 5e-3 material threshold). ESTIMATION-SAMPLE CHANGE, which is why this carries a row rather than landing as a bug-fix side effect: a cohort with NO valid reference is unidentified and its OBSERVATIONS are now excluded (with a warning naming the cohort), on the DEFAULT not_yet_treated path as well as never_treated. Retaining those rows - the pre-3.9 behavior - put a treated cohort into the omitted baseline and loaded its effect onto the time FE, moving other cohorts' ATTs by up to 0.0077. This falsifies the previously documented invariant that _filter_sample expresses the control-group choice through the design matrix and never by dropping rows, so the fitted sample and the full outcome column no longer share support by construction; the _suggest_nonlinear_method hint that cited that invariant remains correct because outcome TYPE is a property of the variable, not the sample. introduced_in gates the 3.9 cut ([M-091]/[M-092]/[M-122] pattern); deprecated_in stays null so the early-flip guard does not fire against the PR that ships it. No opt-out: the alternative is a silently contaminated estimate. BOUNDARY (codex R5): the exclusion is REFUSED, not performed, when survey_design= is supplied - NotImplementedError across ols/logit/poisson. Deleting rows under a complex design is naive subsetting rather than domain estimation: it removes their PSUs/strata from the Taylor-linearized meat and from df_survey = n_PSU - n_strata, so surviving ATTs would carry variance from a design the user never specified (measured on a two-stratum panel: df_survey 22 -> 14, finite SE, no signal). REGISTRY 'Subpopulation Analysis (Phase 6)' and Lumley (2004) 3.4 require zero-padding excluded rows while RETAINING strata/PSU; SurveyDesign.subpopulation() implements that contract but composing it here also needs the weighted within-transform to tolerate zero-weight units (shared machinery behind several estimators), so it is tracked in TODO.md rather than folded in. Relatedly the FULL SurveyDesign is now resolved against the PRE-exclusion sample for validation only - weights, weight_type, strata, PSU, FPC and nesting - because every check that ran after exclusion was blind to whatever the deleted rows contained." + notes: "ETWFE reference-period normalization (issue #724). The saturated design now OMITS each cohort's reference cell explicitly - ref(g) = latest supported period before g - anticipation, per-cohort and positive-weight - per Wooldridge 2025 Eq. 6.1/6.4, anchored to Stata jwdid ... never. Previously the reference was emitted and generic QR rank detection dropped an arbitrary column, which on real mpdta silently removed ATT(2004,2004) and ATT(2006,2007) AND left every surviving coefficient measured against the wrong reference (errors of 0.010-0.041, up to 8x the 5e-3 material threshold). ESTIMATION-SAMPLE CHANGE, which is why this carries a row rather than landing as a bug-fix side effect: a cohort with NO valid reference is unidentified and its OBSERVATIONS are now excluded (with a warning naming the cohort), on the DEFAULT not_yet_treated path as well as never_treated. Retaining those rows - the pre-3.9 behavior - put a treated cohort into the omitted baseline and loaded its effect onto the time FE, moving other cohorts' ATTs by up to 0.0077. This falsifies the previously documented invariant that _filter_sample expresses the control-group choice through the design matrix and never by dropping rows, so the fitted sample and the full outcome column no longer share support by construction; the _suggest_nonlinear_method hint that cited that invariant remains correct because outcome TYPE is a property of the variable, not the sample. introduced_in gates the 3.9 cut ([M-091]/[M-092]/[M-122] pattern); deprecated_in stays null so the early-flip guard does not fire against the PR that ships it. No opt-out: the alternative is a silently contaminated estimate. BOUNDARY (codex R5): the exclusion is REFUSED, not performed, when survey_design= is supplied - NotImplementedError across ols/logit/poisson. Deleting rows under a complex design is naive subsetting rather than domain estimation: it removes their PSUs/strata from the Taylor-linearized meat and from df_survey = n_PSU - n_strata, so surviving ATTs would carry variance from a design the user never specified (measured on a two-stratum panel: df_survey 22 -> 14, finite SE, no signal). REGISTRY 'Subpopulation Analysis (Phase 6)' and Lumley (2004) 3.4 require zero-padding excluded rows while RETAINING strata/PSU; SurveyDesign.subpopulation() implements that contract but composing it here also needs the weighted within-transform to tolerate zero-weight units (shared machinery behind several estimators), so it is tracked in TODO.md rather than folded in. Relatedly the FULL SurveyDesign is now resolved against the PRE-exclusion sample for validation only - weights, weight_type, strata, PSU, FPC and nesting - because every check that ran after exclusion was blind to whatever the deleted rows contained. CONDITIONAL SINCE [M-125]: when per-period comparison-support filtering removes an unidentified cohort's rows first, that cohort never reaches this exclusion path and is instead named by the filter's own zero-cell / fully-dropped warning. The observations are excluded either way and no estimate changes; only which warning names them differs." - id: M-122 kind: behavior group: aggregate-postfit diff --git a/docs/v4-design.md b/docs/v4-design.md index 8198d72e..327fb46c 100644 --- a/docs/v4-design.md +++ b/docs/v4-design.md @@ -743,10 +743,11 @@ forever - a removed symbol resurrecting is a test failure. class row (schema-enforced). Top-level `diff_diff:Name` class/function rows and alias rows also assert `__all__` membership consistent with their status (stale `import *` entries fail). The shipped row ids are a - committed snapshot in the enforcement test (99 as of the - gating-completeness amendment: Phase 1 + the diagnostic-family amendment + + committed snapshot in the enforcement test (104 as of the ETWFE + comparison-support row: Phase 1 + the diagnostic-family amendment + the M-092/M-093 results-contract rows + the M-094..M-096 amendment rows + - the M-097..M-115 completeness sweep; + the M-097..M-115 completeness sweep + M-117/M-122 + the ETWFE + reference-period pair M-123/M-124 + M-125; the snapshot extends by a new id range in the same diff that appends rows): ids are never deleted or reused, and the test fails if any snapshot id disappears. diff --git a/tests/test_etwfe_cs_stata_parity.py b/tests/test_etwfe_cs_stata_parity.py index 854053c6..f4b61b84 100644 --- a/tests/test_etwfe_cs_stata_parity.py +++ b/tests/test_etwfe_cs_stata_parity.py @@ -77,12 +77,17 @@ def _stata_csdid_cells(golden: dict) -> dict: def _stata_jwdid_cells(golden: dict, block: str = "jwdid") -> dict: """``2004bn.first_treat#2005.year#c.__tr__`` -> ``(2004, 2005)``. - ``block`` selects the arm: ``"jwdid"`` (not-yet-treated, the default) or - ``"jwdid_never"`` (never-treated controls, the issue #724 anchor). Both use - the same coefficient-name shape. + ``block`` selects the arm: ``"jwdid"`` (not-yet-treated, the default), + ``"jwdid_never"`` (never-treated controls, the issue #724 anchor), or + ``"jwdid_alltreated"`` (the W2025 Section 5.4 anchor). All use the same + coefficient-name shape; the all-treated block nests its cells under + ``"cells"`` because it also records ``n`` / ``n_units``. """ + raw = golden[block] + if "cells" in raw: + raw = raw["cells"] out = {} - for key, rec in golden[block].items(): + for key, rec in raw.items(): m = re.match(r"(\d+)b?n?\.first_treat#(\d+)b?n?\.year", key) if not m: continue # _cons and any non-cell terms @@ -118,6 +123,13 @@ def test_golden_records_every_reference_implementation(): assert _stata_csdid_cells(golden), "golden has no csdid cells" assert _stata_jwdid_cells(golden, "jwdid"), "golden has no jwdid cells" assert _stata_jwdid_cells(golden, "jwdid_never"), "golden has no jwdid_never cells" + assert _stata_jwdid_cells(golden, "jwdid_alltreated"), "golden has no jwdid_alltreated cells" + # The all-treated arm's row count is part of its finding, not incidental. + assert golden["jwdid_alltreated"]["n"] > 0 + assert golden["jwdid_alltreated"]["n_units"] > 0 + # Every arm records the exact command that produced it. + for key in ("csdid_cmd", "jwdid_cmd", "jwdid_never_cmd", "jwdid_alltreated_cmd"): + assert golden["meta"].get(key), f"golden meta is missing {key}" assert golden["meta"]["source_sha256"] == _PANEL_SHA256 @@ -356,3 +368,84 @@ def test_etwfe_and_cs_genuinely_disagree_on_real_data(fits): # Far beyond the 5e-3 the retired test asserted. assert gaps[worst_key] > 5e-3 np.testing.assert_allclose(gaps[worst_key], 0.017052, rtol=1e-3) + + +@pytest.fixture(scope="module") +def alltreated_fit(): + """The pinned panel with never-treated counties dropped: 191 units, 955 rows. + + Mirrors the generator's `drop if first_treat == 0` exactly, so the library + and Stata see the same frame. + """ + df = _panel() + sub = df[df["first_treat"] != 0].copy() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return WooldridgeDiD(method="ols", control_group="not_yet_treated").fit( + sub, outcome="lemp", unit="countyreal", time="year", cohort="first_treat" + ) + + +class TestAllEventuallyTreatedVsStataJwdid: + """External anchor for W2025 Section 5.4 on a real all-eventually-treated panel. + + Stata `jwdid` succeeds here and silently estimates on a reduced sample: the + 2007 cohort becomes the reference, the fully-treated periods carry no + identified ATT, and jwdid reports only a smaller N. The library computes the + same cell set on the same rows -- and says so. + + Uses `control_group="not_yet_treated"`, the only option available: with no + cohort-0 rows left, `never_treated` raises. + """ + + def test_cell_set_matches_jwdid(self, alltreated_fit): + """The Section 5.4 cell set is an EXTERNAL result, not our convention.""" + stata = _stata_jwdid_cells(_golden(), "jwdid_alltreated") + assert sorted(stata) == [(2004, 2004), (2004, 2005), (2004, 2006), (2006, 2006)] + lib = {(int(g), int(t)) for g, t in alltreated_fit.group_time_effects} + assert lib == set(stata) + # The last cohort is the reference and receives nothing. + assert 2007 not in {g for g, _t in lib} + + def test_att_matches_jwdid(self, alltreated_fit): + """Point estimates are the tight gate: agreement is to machine precision.""" + stata = _stata_jwdid_cells(_golden(), "jwdid_alltreated") + for key, rec in stata.items(): + np.testing.assert_allclose( + alltreated_fit.group_time_effects[key]["att"], rec["att"], atol=1e-12 + ) + + def test_estimation_sample_size_matches_jwdid(self, alltreated_fit): + """The row count IS the finding -- 764 of 955, i.e. the fully-treated + periods are gone from both implementations' estimation samples. + + Pinned because a silent divergence here would mean the two are + estimating on different data while agreeing on coefficients by luck. + """ + golden = _golden()["jwdid_alltreated"] + assert golden["n_units"] == 191 + assert golden["n"] == 764 + assert alltreated_fit.n_obs == golden["n"] + + def test_se_gap_is_a_freshly_measured_ratio_at_this_cluster_count(self, alltreated_fit): + """The `hc1` SE gap is cluster-count dependent, so the ratio is measured + HERE rather than inherited. + + The sibling arms pin 1.001006 at G=500. This arm has G=191 and shows + 1.00264201 -- copying the sibling constant would fail on first run. The + observed sequence (1.0280@G=20, 1.0132@G=40, 1.00264@G=191, + 1.0010@G=500) is monotone in G, which constrains any mechanism later + proposed for the gap (still open, TODO.md). + + As with the sibling tests, no closed form is asserted -- only the + observed factor, and that the library's SE sits BELOW jwdid's. + """ + stata = _stata_jwdid_cells(_golden(), "jwdid_alltreated") + ratios = [ + rec["se"] / alltreated_fit.group_time_effects[key]["se"] for key, rec in stata.items() + ] + assert len(ratios) == 4, f"expected 4 all-treated cells, got {len(ratios)}" + assert max(ratios) - min(ratios) < 1e-5, f"SE ratio is not uniform: {ratios}" + mean_ratio = float(np.mean(ratios)) + assert mean_ratio > 1.0, "library SE should be below jwdid's, not above" + np.testing.assert_allclose(mean_ratio, 1.002642, rtol=1e-4) diff --git a/tests/test_methodology_wooldridge.py b/tests/test_methodology_wooldridge.py index cb89cdc2..fdcfd968 100644 --- a/tests/test_methodology_wooldridge.py +++ b/tests/test_methodology_wooldridge.py @@ -1749,8 +1749,13 @@ def test_plot_event_study_propagates_weights_kwarg(self) -> None: "leads." ) - def test_cohort_trends_true_all_treated_panel_is_refused(self) -> None: - """``cohort_trends=True`` on an all-eventually-treated panel is REFUSED. + def test_cohort_trends_true_all_treated_panel_estimates(self) -> None: + """``cohort_trends=True`` on an all-eventually-treated panel ESTIMATES. + + It used to be refused: the Section 5.4 rule reached the trend columns + but not the cohort x time cells, so the design sank at fully-treated + periods. Comparison-support filtering supplies the cell half, so the + trend-drop path below is now reachable end to end. Paper W2025 Section 5.4: when all units are eventually treated and the last cohort serves as control, "all variables in @@ -1776,23 +1781,32 @@ def test_cohort_trends_true_all_treated_panel_is_refused(self) -> None: 3, 4, ], f"DGP precondition: treated cohorts should be [3, 4], got {treated_cohorts}" - # The trend-drop path is currently UNREACHABLE through fit(): on an - # all-eventually-treated panel the treatment CELLS at fully-treated - # periods are jointly collinear with the time FE, and since codex R8 a - # lost cell fails closed rather than returning relabeled contrasts. The - # last-cohort trend drop below is the same W2025 Section 5.4 - # normalization applied to the TREND columns only -- applying it to the - # cells is what would make these panels estimable again (TODO.md). Until - # then this test pins the refusal, so the trend logic is not silently - # deleted while it is untestable end-to-end. - with pytest.raises(ValueError, match="not identified and were removed"): - WooldridgeDiD(method="ols", cohort_trends=True).fit( + # The last-cohort trend drop is the SAME W2025 Section 5.4 normalization + # the cell filter now applies, and this is the first test to exercise it + # end-to-end. Previously unreachable: the treatment CELLS at fully-treated + # periods were jointly collinear with the time FE, so the fit failed + # closed before the trend logic mattered. Comparison-support filtering + # removes those periods first, leaving cohort 4 (= G_max) as the + # reference -- it receives neither a cell nor a trend column. + with pytest.warns(UserWarning, match="no eligible comparison group"): + res = WooldridgeDiD(method="ols", cohort_trends=True).fit( full_panel, outcome="y", unit="unit", time="time", cohort="cohort", ) + + # G - 1 = 1 trend coefficient, the last cohort deliberately absent. + assert set(res.cohort_trend_coefs) == {3}, ( + f"expected only cohort 3 to carry a trend slope, got " + f"{sorted(res.cohort_trend_coefs)}" + ) + assert 4 not in res.cohort_trend_coefs + assert np.isfinite(res.cohort_trend_coefs[3]) + # Cohort 4 is the reference: no cells, and not advertised as estimated. + assert sorted((int(g), int(t)) for g, t in res.group_time_effects) == [(3, 3)] + assert set(int(g) for g in res.groups) == {3} return def test_cohort_trends_true_with_never_treated_keeps_all_cohort_trends(self) -> None: diff --git a/tests/test_v4_matrix.py b/tests/test_v4_matrix.py index d089d42e..b5fcb5e7 100644 --- a/tests/test_v4_matrix.py +++ b/tests/test_v4_matrix.py @@ -113,11 +113,13 @@ # plus the 2 Phase 2a results-contract rows (M-092/M-093) = 77, plus the 3 # gating-completeness amendment rows (M-094..M-096) = 80, plus the 19-row # completeness sweep over public FUNCTIONS and the dCDH results mirror -# (M-097..M-115) = 99, plus Phase 2b PR 1's two rows (M-117, M-122) = 101. +# (M-097..M-115) = 99, plus Phase 2b PR 1's two rows (M-117, M-122) = 101, +# plus the ETWFE reference-period pair (M-123, M-124) = 103, plus the +# comparison-support row (M-125) = 104. # Ids are never reused and terminal rows are never # deleted, so the ledger only grows - raise the floor when rows are added; a # lower parse count means scanner/format drift or an illegal row deletion. -ROW_COUNT_FLOOR = 103 +ROW_COUNT_FLOOR = 104 # Committed snapshot of the shipped id set ("ids are never deleted or reused" # contract - a delete-one-add-one edit keeps the count above the floor but trips @@ -149,6 +151,7 @@ (97, 115), (117, 117), (122, 124), + (125, 125), ] EXPECTED_INITIAL_IDS = frozenset( f"M-{n:03d}" for lo, hi in _INITIAL_ID_RANGES for n in range(lo, hi + 1) @@ -545,12 +548,12 @@ def test_initial_ids_never_deleted(): """The shipped id set is immutable: ids are never deleted or reused (spec section 11). ROW_COUNT_FLOOR alone would let a delete-one-add-one edit pass; this snapshot cannot. - Extends as rows ship (99 as of the gating-completeness amendment: Phase 1 + + Extends as rows ship (104 as of the ETWFE comparison-support row: Phase 1 + diagnostic-family + M-092/M-093 + M-094..M-096 + the M-097..M-115 - public-function completeness sweep).""" + public-function completeness sweep + M-117/M-122 + M-123/M-124 + M-125).""" missing = sorted(EXPECTED_INITIAL_IDS - set(_ROW_IDS)) assert not missing, f"ledger rows deleted (ids are permanent): {missing}" - assert len(EXPECTED_INITIAL_IDS) == 103 + assert len(EXPECTED_INITIAL_IDS) == 104 def test_version_tuple_pads_to_three_components(): diff --git a/tests/test_wooldridge.py b/tests/test_wooldridge.py index c5c53ef5..1826c77a 100644 --- a/tests/test_wooldridge.py +++ b/tests/test_wooldridge.py @@ -679,28 +679,23 @@ def test_xgvar_fit_runs(self): class TestAllEventuallyTreated: def test_no_never_treated_not_yet_treated_control(self): - """All units eventually treated: the fit is REFUSED, not approximated. + """All units eventually treated: the fit SUCCEEDS via W2025 Section 5.4. - This test previously asserted that the earlier cohorts' cells came back - finite once the latest cohort's rank-deficient columns were dropped. - They were finite and WRONG. Measured on this DGP (true ATT = 1.5 for - every post cell) before the completeness gate:: + This panel used to be refused. Before the #729 completeness gate it was + worse than refused -- it returned finite and WRONG numbers, because once + QR dropped one of the jointly-collinear cells at a fully-treated period + the survivors identified contrasts BETWEEN treated cohorts while keeping + their ATT(g, t) labels. Measured then (true ATT = 1.5 everywhere):: ATT(3, 8) = -0.0096 ATT(3, 9) = -0.1038 ATT(5, 8) = -0.0214 ATT(5, 9) = -0.2393 overall_att = 1.0023 (true 1.5) - Once QR drops one of the jointly-collinear cells at a period where every - unit is treated, the survivors identify contrasts BETWEEN treated - cohorts while keeping their ATT(g, t) labels. Asserting finiteness is - exactly what let that through (codex R8 P0). - - W2025 Section 5.4 does define the right answer for these panels -- the - last cohort serves as the control, so its columns are dropped - DELIBERATELY rather than by QR, which leaves the rest identified. The - library already does this for the cohort-TREND column but not for the - cells, so until that lands the honest behavior is to refuse. Tracked in - TODO.md; landing it turns this test back into a fit. + Comparison-support filtering now removes those periods BEFORE the solve, + which is exactly the paper's rule: with no never-treated group the last + cohort serves as the reference and "all dT_i-containing regressors get + dropped" (W2025 Eq. 5.13-5.15). Cohort 8 therefore receives no cells, + and the retained cells are the absolute tau_gt on t <= G_max - 1. """ rng = np.random.default_rng(7) rows = [] @@ -719,8 +714,59 @@ def test_no_never_treated_not_yet_treated_control(self): rows.append({"unit": u, "time": t, "cohort": cohort, "y": y}) df = pd.DataFrame(rows) est = WooldridgeDiD(control_group="not_yet_treated") - with pytest.raises(ValueError, match="not identified and were removed"): - est.fit(df, outcome="y", unit="unit", time="time", cohort="cohort") + with pytest.warns(UserWarning, match="no eligible comparison group"): + res = est.fit(df, outcome="y", unit="unit", time="time", cohort="cohort") + + # Eq. 5.15 on a balanced panel: cells {(g, t) : g <= G_max - 1, + # g <= t <= G_max - 1}. G_max = 8, so periods 8 and 9 are dropped and + # cohort 8 emits nothing. + assert sorted((int(g), int(t)) for g, t in res.group_time_effects) == [ + (3, 3), + (3, 4), + (3, 5), + (3, 6), + (3, 7), + (5, 5), + (5, 6), + (5, 7), + ] + # The dropped cohort is not advertised as an estimated one. + assert set(int(g) for g in res.groups) == {3, 5} + assert 8 not in res.groups + # Absolute ATTs, not relative: true effect is 1.5 for every post cell. + # Scaled by each cell's own SE rather than a fixed band -- per-cell SEs + # here are ~0.17-0.21, so a fixed tolerance is either flaky or vacuous. + # The pre-fix values this guards against sat ~9 SE away (-0.0096 etc.). + assert res.overall_att == pytest.approx(1.5502, abs=1e-3) + for key, eff in res.group_time_effects.items(): + assert ( + abs(eff["att"] - 1.5) < 3 * eff["se"] + ), f"ATT{key} = {eff['att']:.4f} is more than 3 SE from the true 1.5" + + def test_all_treated_names_the_zero_cell_cohort(self): + """The user is told which cohort lost its cells, not just that rows went. + + Dropping data silently is the failure mode this whole path exists to + avoid: `jwdid` drops the same rows and only reports a smaller N. + """ + rng = np.random.default_rng(7) + rows = [] + for u in range(120): + cohort = 3 if u < 60 else 6 + for t in range(1, 7): + rows.append( + { + "unit": u, + "time": t, + "cohort": cohort, + "y": rng.standard_normal() + 1.5 * int(t >= cohort), + } + ) + df = pd.DataFrame(rows) + with pytest.warns(UserWarning, match=r"Cohort\(s\) 6 have NO estimated cells"): + WooldridgeDiD(control_group="not_yet_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) class TestEmptyCells: @@ -1608,24 +1654,28 @@ def test_poisson_zero_weight_cell(self, survey_panel): assert np.isfinite(r.overall_att) assert np.isfinite(r.overall_se) - def test_ols_survey_rank_deficient_now_fails_closed(self, survey_panel): - """An all-eventually-treated survey design is REFUSED, not approximated. - - This test previously asserted only that `overall_att` / `overall_se` - were finite on a deliberately rank-deficient design. They were finite - and WRONG: once rank reduction drops one of the jointly-collinear - final-period cells, the survivors identify a contrast between two - treated cohorts and keep their ATT(g, t) labels. Measured on an - equivalent panel with true effects 1.0 and 3.0, the surviving cell came - back as -1.94. Asserting finiteness is what let that through, so the - assertion is now the refusal itself (codex R8 P0). + def test_ols_survey_all_treated_refuses_period_filtering(self, survey_panel): + """Comparison-support filtering under `survey_design=` is REFUSED. + + This panel previously returned finite and WRONG numbers (a surviving + cell came back as -1.94 against true effects of 1.0 and 3.0), then was + made to fail closed with a rank-reduction ValueError. Now the periods + that caused it are unsupported and would be FILTERED -- but deleting + rows under a complex survey design is naive subsetting: it removes + their PSUs and strata from the TSL meat and from + `df_survey = n_PSU - n_strata`, so the surviving estimates would carry a + variance computed on a design the user never specified. + + The refusal is raised BEFORE any row is removed, and points at explicit + frame restriction -- NOT `SurveyDesign.subpopulation()`, whose + zero-weight padding `_reject_zero_weight_groups` refuses on this path. """ from diff_diff.survey import SurveyDesign - # Remove never-treated (cohort=0) to create rank-deficient design + # Remove never-treated (cohort=0) so later periods lose all comparisons. df = survey_panel[survey_panel["cohort"] > 0].copy() sd = SurveyDesign(weights="weight", strata="stratum", psu="unit") - with pytest.raises(ValueError, match="not identified and were removed"): + with pytest.raises(NotImplementedError, match="no eligible comparison group"): WooldridgeDiD(control_group="not_yet_treated").fit( df, outcome="y", @@ -1635,6 +1685,26 @@ def test_ols_survey_rank_deficient_now_fails_closed(self, survey_panel): survey_design=sd, ) + def test_survey_refusal_is_conditional_on_rows_actually_dropping(self, survey_panel): + """The refusal must NOT fire on survey fits that lose nothing. + + Instrumented over the suite, 59 `fit` calls pass `survey_design=` and + exactly one has an unsupported period. An unconditional refusal would + break the other 58, so the guard is pinned as conditional here. + """ + from diff_diff.survey import SurveyDesign + + sd = SurveyDesign(weights="weight", strata="stratum", psu="unit") + res = WooldridgeDiD(control_group="not_yet_treated").fit( + survey_panel, # retains its never-treated units at every period + outcome="y", + unit="unit", + time="time", + cohort="cohort", + survey_design=sd, + ) + assert np.isfinite(res.overall_att) + def test_ols_survey_zero_weight_unit_rejected(self, survey_panel): """Zero-weight unit raises ValueError before within_transform.""" from diff_diff.survey import SurveyDesign @@ -2904,11 +2974,17 @@ def test_all_post_columns_rank_dropped_fails_closed(self): The upstream ``has_untreated`` guard passes here because it counts this cohort's OWN pre-treatment rows as comparison observations -- which is precisely why the check has to also run after the solve. + + Still fails closed, but now with a DIFFERENT and more accurate message. + Periods 3 and 4 have no untreated unit, so support filtering removes + them; cohort 3 then has no observation at or after its own treatment + start, so there is nothing to estimate. The old rank-reduction path is + no longer reached. """ df = self._unbalanced({3: [1, 2, 3, 4]}) with warnings.catch_warnings(): warnings.simplefilter("ignore") - with pytest.raises(ValueError, match="not identified and were removed"): + with pytest.raises(ValueError, match="No estimable post-treatment cells found"): WooldridgeDiD(method="ols", control_group="not_yet_treated").fit( df, outcome="y", unit="unit", time="time", cohort="cohort" ) @@ -3602,52 +3678,69 @@ def _no_comparison_final_period(): uid += 1 return pd.DataFrame(rows) - @pytest.mark.parametrize( - "action,expected", - [ - ("warn", "not identified and were removed"), - ("silent", "not identified and were removed"), - # "error" fails EARLIER, inside solve_ols, with its own message - # naming the dependent column -- also fail-closed, just a different - # path. Pinned explicitly so a future refactor cannot quietly move - # this mode onto the softer branch. - ("error", "rank-deficient"), - ], - ) - def test_partial_cell_loss_raises_for_every_rank_deficient_action(self, action, expected): - """Including `"silent"`: that setting controls how rank WARNINGS - surface, not whether a mislabeled contrast may be returned as an ATT. - Measured before the gate under `"silent"`: ATT(3,5) came back as - -1.9393 (true +1.0, wrong SIGN) inside overall_att=+0.7703 (true ~1.8), - with no warning of any kind. + @pytest.mark.parametrize("action", ["warn", "silent", "error"]) + def test_partial_cell_loss_is_now_filtered_for_every_rank_deficient_action(self, action): + """The unsupported period is removed BEFORE the solve, in every mode. + + This panel used to raise under all three settings, and before the #729 + gate it was worse: under `"silent"` ATT(3,5) came back as -1.9393 (true + +1.0, wrong SIGN) inside overall_att=+0.7703 (true ~1.8), with no + warning of any kind. + + `t=5` has no never-treated observations, so comparison-support filtering + drops it and the remaining design is exactly the one + `test_the_same_panel_fits_once_the_bad_period_is_removed` fits by hand. + Parameterized across all three modes because the drop is NOT gated on + `rank_deficient_action` -- that setting governs how rank warnings + surface, not whether the estimation sample changes. """ - with warnings.catch_warnings(): - warnings.simplefilter("ignore") - with pytest.raises(ValueError, match=expected): - WooldridgeDiD( - method="ols", control_group="never_treated", rank_deficient_action=action - ).fit( - self._no_comparison_final_period(), - outcome="y", - unit="unit", - time="time", - cohort="cohort", - ) + with pytest.warns(UserWarning, match="no eligible comparison group"): + res = WooldridgeDiD( + method="ols", control_group="never_treated", rank_deficient_action=action + ).fit( + self._no_comparison_final_period(), + outcome="y", + unit="unit", + time="time", + cohort="cohort", + ) + assert sorted((int(g), int(t)) for g, t in res.group_time_effects) == [ + (3, 1), + (3, 3), + (3, 4), + (4, 1), + (4, 2), + (4, 4), + ] + # The values the pre-gate bug corrupted, now recovered. + assert res.group_time_effects[(3, 3)]["att"] == pytest.approx(1.0, abs=0.1) + assert res.group_time_effects[(4, 4)]["att"] == pytest.approx(3.0, abs=0.1) - def test_the_error_names_the_lost_cell(self): - """A user cannot act on 'something was dropped' -- the message has to - say which cell, since the fix is to restrict the panel or add controls - for that period.""" - with warnings.catch_warnings(): - warnings.simplefilter("ignore") - with pytest.raises(ValueError, match=r"\(4, 5\)|\(3, 5\)"): - WooldridgeDiD(method="ols", control_group="never_treated").fit( - self._no_comparison_final_period(), - outcome="y", - unit="unit", - time="time", - cohort="cohort", - ) + def test_the_drop_warning_names_the_period_and_row_count(self): + """A user cannot act on 'something was dropped' -- the message has to say + WHICH period and how much data, since the remedy is theirs to choose + (add never-treated units, or restrict the panel deliberately). + + Replaces an earlier test that pinned the lost-CELL error message; that + error is no longer reached on this panel because the period is filtered + before the solve. + """ + with pytest.warns(UserWarning) as rec: + WooldridgeDiD(method="ols", control_group="never_treated").fit( + self._no_comparison_final_period(), + outcome="y", + unit="unit", + time="time", + cohort="cohort", + ) + msgs = [str(w.message) for w in rec] + drop = [m for m in msgs if "no eligible comparison group" in m] + assert drop, f"no comparison-support warning emitted; got {msgs}" + assert "1 of 5 periods: 5" in drop[0] + assert "observations" in drop[0] + # Branch 1 (never_treated + OLS): the cause is missing never-treated + # rows, NOT "every unit is already treated". + assert "no never-treated units are observed" in drop[0] def test_the_same_panel_fits_once_the_bad_period_is_removed(self): """The gate must not be a blanket refusal: dropping the period that has @@ -3714,3 +3807,517 @@ def test_never_treated_emits_pre_cells_except_the_reference(self): emitted = {t for (gg, t) in gt if gg == g} expected = set(range(1, 7)) - {ref} assert emitted == expected, f"cohort {g}: expected all periods but {ref}" + + +class TestComparisonSupportFiltering: + """Per-period comparison support (W2025 Section 5.4). + + Every panel and every number in this class was measured before the feature + was written; the plan that specifies them was revised nine times against + executed output, twice because a figure had been paired with the wrong + frame. Assertions therefore name the panel they were taken on. + """ + + @staticmethod + def _all_treated(cohorts=(3, 5, 8), periods=9, n_per=70, seed=7, effect=1.5): + """All-eventually-treated panel: no cohort 0 anywhere.""" + rng = np.random.default_rng(seed) + rows = [] + uid = 0 + for g in cohorts: + for _ in range(n_per): + for t in range(1, periods + 1): + rows.append( + { + "unit": uid, + "time": t, + "cohort": g, + "y": rng.standard_normal() + effect * int(t >= g), + } + ) + uid += 1 + return pd.DataFrame(rows) + + @pytest.mark.parametrize("anticipation,expected_kept", [(0, 7), (1, 6), (2, 5)]) + def test_eq_5_15_cell_set_on_a_balanced_panel(self, anticipation, expected_kept): + """The retained cell set is Eq. 5.15's, intersected with observed times. + + `{(g, t) : g <= G_max - 1, g - anticipation <= t <= G_max - 1 - + anticipation}`. The window bound is `t >= g - anticipation` (the builder + emits anticipation-window cells), NOT `t >= g` -- an earlier draft of + this assertion used `g <= t` and would have failed at anticipation > 0. + """ + df = self._all_treated() + g_max = 8 + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = WooldridgeDiD(control_group="not_yet_treated", anticipation=anticipation).fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + + kept = sorted({int(t) for (_g, t) in res.group_time_effects}) + assert max(kept) == g_max - 1 - anticipation + assert len(set(range(1, g_max - anticipation))) == expected_kept + + observed = set(range(1, 10)) + expected = { + (g, t) + for g in (3, 5, 8) + for t in observed + if g <= g_max - 1 + # A cohort with no observed period before `g - anticipation` has no + # reference and is excluded as unidentified, even though it meets + # the Eq. 5.15 cohort bound. Observations start at t=1, so that is + # `g - anticipation > 1`. At anticipation=2 this removes cohort 3, + # leaving exactly {(5,3),(5,4),(5,5)} -- measured. + and g - anticipation > 1 and g - anticipation <= t <= g_max - 1 - anticipation + } + assert {(int(g), int(t)) for g, t in res.group_time_effects} == expected + # cohort G_max is the reference: no cells of its own. + assert g_max not in {int(g) for (g, _t) in res.group_time_effects} + + def test_no_op_when_every_period_has_a_comparison(self): + """Filtering is a no-op iff every period has an eligible comparison -- + not merely 'a never-treated group exists'.""" + rng = np.random.default_rng(21) + rows = [] + for u in range(120): + g = 0 if u < 40 else (3 if u < 80 else 5) + for t in range(1, 7): + rows.append( + { + "unit": u, + "time": t, + "cohort": g, + "y": rng.standard_normal() + 1.0 * int(g > 0 and t >= g), + } + ) + df = pd.DataFrame(rows) + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + res = WooldridgeDiD(control_group="not_yet_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + assert not [m for m in w if "no eligible comparison group" in str(m.message)] + assert res.n_obs == len(df) + + def test_branch_one_reference_move_warns_by_name(self): + """`never_treated` + OLS: a reference CAN move, and must say so. + + The ATTs stay correctly labelled and validly identified -- they are just + normalized against a different baseline period than an unfiltered fit + would use. Silent renormalization is the #724 defect class, so this + warns rather than raising. + """ + rng = np.random.default_rng(5) + rows = [] + uid = 0 + for _ in range(30): # never-treated, observed t=1..4 only + for t in range(1, 5): + rows.append( + {"unit": uid, "time": t, "cohort": 0, "y": 0.2 * t + rng.normal(0, 0.05)} + ) + uid += 1 + for g in (3, 6): + for _ in range(30): + for t in range(1, 7): + rows.append( + { + "unit": uid, + "time": t, + "cohort": g, + "y": 0.2 * t + (1.0 if t >= g else 0.0) + rng.normal(0, 0.05), + } + ) + uid += 1 + df = pd.DataFrame(rows) + with pytest.warns(UserWarning, match=r"reference period moved from 5 to 4"): + res = WooldridgeDiD(method="ols", control_group="never_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + assert res.overall_att == pytest.approx(1.0171, abs=5e-3) + + def test_branch_one_no_move_when_never_treated_covers_the_period(self): + """Companion to the above, on a frame truncated to t <= 5. + + Stated explicitly as truncated: on the untruncated 6-period frame t=6 + is still unsupported, so rows WOULD drop and warning (a) would fire + correctly -- an earlier draft described this case as 'nothing is + filtered', which is false. + """ + rng = np.random.default_rng(5) + rows = [] + uid = 0 + for _ in range(30): # never-treated now observed through t=5 + for t in range(1, 6): + rows.append( + {"unit": uid, "time": t, "cohort": 0, "y": 0.2 * t + rng.normal(0, 0.05)} + ) + uid += 1 + for g in (3, 6): + for _ in range(30): + for t in range(1, 7): + rows.append( + { + "unit": uid, + "time": t, + "cohort": g, + "y": 0.2 * t + (1.0 if t >= g else 0.0) + rng.normal(0, 0.05), + } + ) + uid += 1 + df = pd.DataFrame(rows) + df = df[df["time"] <= 5] + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + res = WooldridgeDiD(method="ols", control_group="never_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + assert not [m for m in w if "reference period moved" in str(m.message)] + assert res.overall_att == pytest.approx(0.9951, abs=5e-3) + + def test_branch_two_references_are_invariant(self): + """`not_yet_treated`: references provably cannot move, so no warning. + + Reference eligibility (`t < g - anticipation`) IS the predicate's second + disjunct evaluated at the cohort's own rows, so any reference period is + supported by construction. The estimator asserts this at runtime; a + violation would mean silent renormalization. + """ + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + WooldridgeDiD(control_group="not_yet_treated").fit( + self._all_treated(), outcome="y", unit="unit", time="time", cohort="cohort" + ) + assert not [m for m in w if "reference period moved" in str(m.message)] + + @staticmethod + def _all_treated_nonlinear(kind, cohorts=(3, 5, 8), periods=9, n_per=25, seed=17): + """All-eventually-treated panel with a binary / count outcome.""" + rng = np.random.default_rng(seed) + rows = [] + uid = 0 + for g in cohorts: + for _ in range(n_per): + uid += 1 + fe = rng.standard_normal() * 0.4 + for t in range(1, periods + 1): + lin = -0.2 + fe + 0.05 * t + 0.8 * int(t >= g) + y = ( + rng.binomial(1, 1.0 / (1.0 + np.exp(-lin))) + if kind == "logit" + else rng.poisson(np.exp(lin)) + ) + rows.append({"unit": uid, "time": t, "cohort": g, "y": y}) + return pd.DataFrame(rows) + + @pytest.mark.parametrize("kind", ["logit", "poisson"]) + def test_nonlinear_all_treated_paths_filter_identically(self, kind): + """The filter runs on logit/Poisson too, and they were untested. + + These paths differ from OLS in design (explicit cohort + time dummies, + no within-transform) and in control-pool semantics, so OLS coverage + does not carry over. The Section 5.4 cell set is a property of the + PERIODS, not of the link, so all three must agree on it exactly. + """ + df = self._all_treated_nonlinear(kind) + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + res = WooldridgeDiD(method=kind, control_group="not_yet_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + # Same cell set as the OLS path: cohorts 3 and 5 over t <= 7, none for 8. + assert sorted((int(g), int(t)) for g, t in res.group_time_effects) == [ + (3, 3), + (3, 4), + (3, 5), + (3, 6), + (3, 7), + (5, 5), + (5, 6), + (5, 7), + ] + assert sorted(int(g) for g in res.groups) == [3, 5] + # 2 of 9 periods removed -> 7/9 of the rows survive. + assert res.n_obs == len(df) * 7 // 9 + assert np.isfinite(res.overall_att) and np.isfinite(res.overall_se) + + # Both warnings fire here, exactly as on OLS. + assert [m for m in w if "no eligible comparison group" in str(m.message)] + assert [m for m in w if "have NO estimated cells" in str(m.message)] + # References cannot move on this branch, whatever the link. + assert not [m for m in w if "reference period moved" in str(m.message)] + + @pytest.mark.parametrize("kind", ["logit", "poisson"]) + def test_nonlinear_survey_refusal_also_fires(self, kind): + """The survey refusal is decided before any row is removed, so it must + fire on every method -- not only the OLS default.""" + from diff_diff.survey import SurveyDesign + + df = self._all_treated_nonlinear(kind) + df["w"] = 1.0 + design = SurveyDesign(weights="w") + with pytest.raises(NotImplementedError, match="no eligible comparison group"): + WooldridgeDiD(method=kind, control_group="not_yet_treated").fit( + df, + outcome="y", + unit="unit", + time="time", + cohort="cohort", + survey_design=design, + ) + + def test_warning_a_survives_a_downstream_raise(self): + """The drop warning fires at filter time, before anything can raise. + + On this panel `_exclude_unidentified_cohorts` raises after the filter, + so a build-time-only warning would never reach the user -- they would + see an error about comparison observations with no indication that + half their rows had already been removed. + """ + rng = np.random.default_rng(9) + rows = [] + uid = 0 + for g in (1, 3): + for _ in range(20): + for t in range(1, 5): + rows.append({"unit": uid, "time": t, "cohort": g, "y": rng.standard_normal()}) + uid += 1 + df = pd.DataFrame(rows) + with pytest.warns(UserWarning, match="no eligible comparison group"): + with pytest.raises(ValueError): + WooldridgeDiD(control_group="not_yet_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + + def test_zero_cell_cohort_is_reported_once_not_twice(self): + """Warning (b) reads the FINAL cell set. + + On `{3,5,8}` at anticipation=2 the filter drops rows AND cohort 3 is + then excluded as unidentified. Cohort 3 already has its own exclusion + warning, so reading the first build's cell set would double-report it. + """ + df = self._all_treated() + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + try: + WooldridgeDiD(control_group="not_yet_treated", anticipation=2).fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + except ValueError: + pass + zero_cell = [m for m in w if "have NO estimated cells" in str(m.message)] + for m in zero_cell: + assert ( + "3" not in str(m.message).split("have NO estimated cells")[0].split("Cohort(s)")[-1] + ), f"cohort 3 double-reported: {m.message}" + + def test_zero_cell_warning_does_not_misattribute_the_cause(self): + """A zero-cell cohort is explained by ITS cause, not by Section 5.4. + + Section 5.4 explains exactly one cohort -- the last, and only when no + never-treated group exists. Here a never-treated group IS present and + cohort 7 is zero-cell purely because the panel never reaches t=7, so + telling the user this is "the W2025 Section 5.4 normalization when no + never-treated group exists" states something false about their panel. + The companion all-treated case must keep the Section 5.4 wording. + """ + rng = np.random.default_rng(21) + rows = [] + uid = 0 + for cohort in (0, 3, 7): + for _ in range(40): + uid += 1 + fe = rng.standard_normal() + for t in range(1, 6): + eff = 1.5 if (cohort > 0 and t >= cohort) else 0.0 + rows.append( + { + "unit": uid, + "time": t, + "cohort": cohort, + "y": fe + 0.1 * t + eff + rng.standard_normal() * 0.3, + } + ) + df = pd.DataFrame(rows) + + with warnings.catch_warnings(record=True) as w: + warnings.simplefilter("always") + res = WooldridgeDiD(control_group="not_yet_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + assert 7 not in res.groups + zero = [m for m in w if "have NO estimated cells" in str(m.message)] + assert len(zero) == 1, f"expected one zero-cell warning, got {len(zero)}" + text = str(zero[0].message) + assert "7" in text + assert "Section 5.4" not in text, f"misattributed to Section 5.4: {text}" + assert "not treated within the observed periods" in text + + # The genuine Section 5.4 case still says so. + with warnings.catch_warnings(record=True) as w2: + warnings.simplefilter("always") + WooldridgeDiD(control_group="not_yet_treated").fit( + self._all_treated(), outcome="y", unit="unit", time="time", cohort="cohort" + ) + sec54 = [m for m in w2 if "have NO estimated cells" in str(m.message)] + assert len(sec54) == 1 + assert "Section 5.4" in str(sec54[0].message) + + def test_cohort_share_fails_closed_when_filtering_removes_units(self): + """`N_g` is read off the FINAL sample, so filtering can shrink it. + + Period filtering is the first thing in this estimator that removes SOME + units of a RETAINED cohort: `_filter_sample` keeps every row of every + treated unit, and unidentified-cohort exclusion removes whole cohorts. + On an unbalanced panel a unit observed only at unsupported periods + vanishes, and `aggregate(weights="cohort_share")` silently reweights -- + measured on this panel: 1.8078 with the supplied N_g against 3.8157 + with the post-filter one, a 2x swing on a headline number. + + W2025 Section 7 defines N_g on a balanced panel and does not say which + reading applies, so `aggregate` refuses rather than picking one. + `weights="cell"` never reads N_g and must keep working. + """ + rng = np.random.default_rng(11) + rows = [] + uid = 0 + + def add(u, c, ts, eff): + fe = rng.standard_normal() + for t in ts: + rows.append( + { + "unit": u, + "time": t, + "cohort": c, + "y": fe + 0.1 * t + (eff if t >= c else 0.0) + rng.standard_normal() * 0.25, + } + ) + + every = list(range(1, 8)) + for _ in range(10): + uid += 1 + add(uid, 2, every, 1.0) + for _ in range(90): # observed ONLY at t=7, which has no comparison + uid += 1 + add(uid, 2, [7], 1.0) + for _ in range(40): + uid += 1 + add(uid, 4, every, 5.0) + for _ in range(40): + uid += 1 + add(uid, 7, every, 9.0) + df = pd.DataFrame(rows) + + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = WooldridgeDiD(control_group="not_yet_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + # 90 of cohort 2's 100 units lived only in the dropped period. + assert res._cohort_units_dropped == {2: 90} + assert res._n_g_per_cohort[2] == 10 + + with pytest.raises(ValueError, match="comparison-support filtering removed"): + res.aggregate(type="simple", weights="cohort_share") + # The default weighting is untouched. + assert np.isfinite(res.aggregate(type="simple", weights="cell").overall_att) + + def test_every_aggregation_surface_reads_the_filtered_cell_set(self): + """`group` / `calendar` / `event` all consume the REDUCED cell set. + + `results.groups` and `time_periods` both shrink under filtering, and + each aggregation keys off a different one of them. A stale read would + surface an all-NaN row for `G_max`, or a calendar entry for a period + that was removed before the solve -- either of which would advertise an + estimate the fit never produced. + + Cells here are (3, 3..7) and (5, 5..7), so k = t - g spans 0..4 and + cohort 8 contributes nothing to any of the three. + """ + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = WooldridgeDiD(control_group="not_yet_treated").fit( + self._all_treated(), outcome="y", unit="unit", time="time", cohort="cohort" + ) + assert res.time_periods == [1, 2, 3, 4, 5, 6, 7], "dropped periods leaked into metadata" + + # group: G_max has no cells and must not appear. + grp = res.aggregate(type="group") + assert sorted(int(g) for g in grp.group_effects) == [3, 5] + + # calendar: only periods carrying an estimated cell, so never 8 or 9. + cal = res.aggregate(type="calendar") + cal_keys = sorted(int(t) for t in cal.calendar_effects) + assert cal_keys == [3, 4, 5, 6, 7] + assert not {8, 9} & set(cal_keys) + + # event: exposure times of the retained cells only. + evt = res.aggregate(type="event") + assert sorted(int(k) for k in evt.event_study_effects) == [0, 1, 2, 3, 4] + + for agg in (grp, cal, evt): + assert np.isfinite(agg.overall_att) and np.isfinite(agg.overall_se) + + def test_cohort_share_still_works_when_no_unit_is_lost(self): + """The refusal must not fire on the balanced deliverable. + + Filtering there removes whole PERIODS and no units at all, so `N_g` is + exactly the supplied cohort size and cohort-share weighting is + well-defined. A guard that fired here would break the capability this + change exists to restore. + """ + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = WooldridgeDiD(control_group="not_yet_treated").fit( + self._all_treated(), outcome="y", unit="unit", time="time", cohort="cohort" + ) + assert res._cohort_units_dropped == {} + agg = res.aggregate(type="simple", weights="cohort_share") + assert np.isfinite(agg.overall_att) + + def test_cells_derived_groups_did_not_leak_into_the_design(self): + """`results.groups` is cells-derived; the DESIGN keeps present cohorts. + + This is the regression guard for the most dangerous defect review found: + routing the covariate blocks through a cells-derived list would drop + `D_{G_max} x X` and silently apply the Section 5.4 covariate + normalization this release defers -- measured at 8/8 ATTs moving, max + 0.0648, on a dropped column with R^2 = 0.5551 against the rest of the + design (i.e. real information, not a spanned duplicate). + + Detected here by the presence of the rank deficiency itself: if the + list had leaked, `D8_x_x` would be gone and this would fit cleanly. + Baseline control matters -- with no covariates the two lists agree + exactly, so a covariate-free test proves nothing. + """ + rng = np.random.default_rng(11) + df = self._all_treated(n_per=60, seed=11) + df["x"] = rng.standard_normal(len(df)) + + with pytest.raises(ValueError, match="rank-deficient"): + WooldridgeDiD(control_group="not_yet_treated", rank_deficient_action="error").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort", exovar=["x"] + ) + + # xtvar is full rank under the default demeaning and must stay that way. + res = WooldridgeDiD(control_group="not_yet_treated").fit( + df, outcome="y", unit="unit", time="time", cohort="cohort", xtvar=["x"] + ) + assert np.isfinite(res.overall_att) + + def test_bootstrap_runs_on_a_filtered_fit(self): + """Bootstrap consumes the FILTERED sample's residuals and cluster ids. + + Row alignment between the reduced frame and the resampling machinery is + exercised here rather than assumed. + """ + df = self._all_treated(n_per=40, seed=13) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = WooldridgeDiD(control_group="not_yet_treated", n_bootstrap=49, seed=42).fit( + df, outcome="y", unit="unit", time="time", cohort="cohort" + ) + assert np.isfinite(res.overall_att) + assert np.isfinite(res.overall_se)