Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 39 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
in every post cell: `ATT(3,8) = -0.0096`, `ATT(5,9) = -0.2393`, and
`overall_att = 1.0023`, with **no warning** under
`rank_deficient_action="silent"`.
**This is a capability regression, taken deliberately.** Panels with no
never-treated group — where periods in which every unit is treated have no
eligible comparison — previously produced those wrong numbers and now fail
closed, including under `survey_design=` and `cohort_trends=True`. Wooldridge
(2025) Section 5.4 defines the correct treatment (the last cohort serves as
the control, its columns dropped deliberately rather than by QR); the library
already applies that to the cohort-trend column but not to the cells, and
implementing the cell half is tracked to restore these fits.
Panels with no never-treated group — where periods in which every unit is
treated have no eligible comparison — previously produced those wrong numbers.
They briefly failed closed, and now **estimate correctly** via the
comparison-support filter below; the gate remains as the backstop for what
that filter cannot see.
- **`WooldridgeDiD` estimates all-eventually-treated panels again, implementing
Wooldridge (2025) Section 5.4 for the cell and cohort-trend families.** With
no never-treated group the last cohort serves as the reference, so periods at
which no unit is untreated carry no identified `ATT(g, t)` and are removed
from the estimation sample before the solve. Measured on a `{3, 5, 8}` panel
over `t = 1..9` with a true ATT of 1.5: cells `(3, 3..7)` and `(5, 5..7)`,
`overall_att = 1.5502`, cohort 8 correctly receiving none. Anchored to Stata
`jwdid` on the `mpdta` panel with never-treated counties dropped — identical
cell set, identical `N` (764 of 955), ATTs agreeing to ~1e-15.
**Nothing is dropped silently.** Every fit that reduces the sample reports the
periods, the observation count and the cause, and separately names any cohort
left without cells or stripped of every row; neither warning is gated on
`rank_deficient_action`. On the `never_treated` OLS branch a filtered period
can legitimately move a cohort's reference, renormalizing that cohort's ATTs
— this warns by name rather than passing silently. (Stata `jwdid` performs the
same reduction and reports only a smaller `N`.)
`cohort_trends=True` works on these panels for the first time, surfacing
`G - 1` trend slopes with the last cohort absent as the baseline. Under
`survey_design=` the combination is refused rather than subsetted, for the
same reason as the unidentified-cohort refusal below.
On an **unbalanced** panel a unit observed only at unsupported periods is
removed with them, which shrinks the cohort-share weight `N_g` (Eqs. 7.4/7.6)
below the cohort actually supplied — measured at a 2x swing in
`aggregate(weights="cohort_share")`. The paper defines `N_g` on a balanced
panel and does not settle which reading applies, so that aggregation now
**fails closed** naming the affected cohorts and unit counts instead of
choosing one silently. `weights="cell"` (the default) never reads `N_g`, and
balanced panels drop whole periods and no units, so neither is affected.
Not included: the `D_{G_max} × X` covariate normalization, so *some* covariate
specifications on such a panel remain rank-deficient — `exovar` and `xgvar`,
and `xtvar` under `demean_covariates=False`. Default `xtvar` (with
`demean_covariates=True`) is **full rank** and fits cleanly even under
`rank_deficient_action="error"`. Where the deficiency does bite, coefficients
are unaffected but `"error"` raises. Tracked in `TODO.md`.
- **`WooldridgeDiD` refuses `survey_design=` combined with an unidentified
cohort** (`NotImplementedError`, all three methods). Excluding a cohort under
a complex survey design is domain estimation, which this path does not
Expand Down
2 changes: 1 addition & 1 deletion DEFERRED.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ exists but parity can't be verified without a local toolchain.
| `StaggeredTripleDifference` R cross-validation: CSV fixtures not committed (gitignored); tests skip without local R + `triplediff`. Commit fixtures or generate deterministically. | `tests/test_methodology_staggered_triple_diff.py` | #245 | Medium |
| `StaggeredTripleDifference` R parity: benchmark only tests the no-covariate path (`xformla=~1`). Add covariate-adjusted scenarios + aggregation-SE parity assertions. | `benchmarks/R/benchmark_staggered_triplediff.R` | #245 | Medium |
| `StaggeredTripleDifference` per-cohort group-effect SEs include WIF (conservative vs R's `wif=NULL`); documented in REGISTRY. Could override the mixin for an exact R match (verification needs R `triplediff`). | `staggered_triple_diff.py` | #245 | Low |
| **WooldridgeDiD follow-up cluster** (PR-B Stage D/E fail-closed surfaces; re-enable after R/Stata validation): <br>• QMLE sandwich uses `aweight` cluster adjustment `(G/(G-1))·(n-1)/(n-k)` vs Stata's `G/(G-1)` (conservative); add a `qmle` weight type if Stata goldens confirm a material difference (`wooldridge.py`, `linalg.py`). <br>• response-scale APE / log-link coefficient bridge for R `etwfe(family=poisson|logit)` cell-level parity — needs `emfx()` APE extraction or link-inversion with baseline-mean adjustment (`generate_wooldridge_golden.R`, `test_methodology_wooldridge.py`). <br>• `aggregate(weights="cohort_share")` on survey-weighted fits: `_n_g_per_cohort` uses raw `unit.nunique()`; implement design-weighted unit totals per cohort (paper W2025 §7) and lift the `ValueError` gate (`wooldridge.py`, `wooldridge_results.py`). <br>• unconditional inference for `cohort_share` accounting for ω̂_g sampling uncertainty (W2025 §7.5); currently NaN-closed (`wooldridge_results.py`). <br>• `cohort_trends=True × survey_design` and `× control_group="never_treated"` raise `NotImplementedError` (unvalidated TSL variance / trend columns spanned jointly by the placebo cells and the unit FE, which absorb the cohort indicator and recover the omitted reference) (`wooldridge.py`). <br>• Stata `jwdid` golden-value `TestReferenceValues` (`tests/test_wooldridge.py`). | `wooldridge.py`, `wooldridge_results.py`, `linalg.py`, benchmarks | #216 · PR-B | Med-Low |
| **WooldridgeDiD follow-up cluster** (PR-B Stage D/E fail-closed surfaces; re-enable after R/Stata validation): <br>• QMLE sandwich uses `aweight` cluster adjustment `(G/(G-1))·(n-1)/(n-k)` vs Stata's `G/(G-1)` (conservative); add a `qmle` weight type if Stata goldens confirm a material difference (`wooldridge.py`, `linalg.py`). <br>• response-scale APE / log-link coefficient bridge for R `etwfe(family=poisson|logit)` cell-level parity — needs `emfx()` APE extraction or link-inversion with baseline-mean adjustment (`generate_wooldridge_golden.R`, `test_methodology_wooldridge.py`). <br>• `aggregate(weights="cohort_share")` on survey-weighted fits: `_n_g_per_cohort` uses raw `unit.nunique()`; implement design-weighted unit totals per cohort (paper W2025 §7) and lift the `ValueError` gate (`wooldridge.py`, `wooldridge_results.py`). <br>• unconditional inference for `cohort_share` accounting for ω̂_g sampling uncertainty (W2025 §7.5); currently NaN-closed (`wooldridge_results.py`). <br>• `cohort_trends=True × survey_design` and `× control_group="never_treated"` raise `NotImplementedError` (unvalidated TSL variance / trend columns spanned jointly by the placebo cells and the unit FE, which absorb the cohort indicator and recover the omitted reference) (`wooldridge.py`). <br>• ~~Stata `jwdid` golden-value `TestReferenceValues`~~ RESOLVED: the golden ships four arms pinned by `tests/test_etwfe_cs_stata_parity.py` (no `TestReferenceValues` symbol was ever added). The QMLE bullet above REMAINS OPEN -- every golden arm is linear `jwdid`, so no QMLE cluster-SE reference exists, and SEs are pinned only as a ratio. | `wooldridge.py`, `wooldridge_results.py`, `linalg.py`, benchmarks | #216 · PR-B | Med-Low |
| Extend `WooldridgeDiD` `method ∈ {logit, poisson}` with `vcov_type ∈ {classical, hc2, hc2_bm}`: composing HC2 leverage + Bell-McCaffrey DOF with the QMLE pseudo-residual sandwich needs derivation + R parity vs `clubSandwich::vcovCR(glm, type="CR2")`. Rejected at `__init__`. | `wooldridge.py` | follow-up | Medium |
| Multi-constraint CR2 parallel-trends test (AHT/HTZ) for `hc2_bm` fits: DiagnosticReport's PT check routes `vcov_type="hc2_bm"` sources to Bonferroni over the BM-adjusted per-row p-values because the generic chi-square joint Wald would discard the CR2 small-sample correction (see REPORTING.md "hc2_bm parallel-trends policy"). The proper joint test is the AHT/HTZ Wald with a Satterthwaite-style denominator df over the pre-period contrast block; needs derivation for the stacked/pooled WLS-CR2 layout + parity vs `clubSandwich::Wald_test(..., test="HTZ")`. | `diagnostic_report.py`, `linalg.py` | vcov/df round-trip PR | Low |
| `PreTrendsPower` CS/SA `anticipation=1` R-parity fixture: R `pretrends` has no anticipation parameter, so the Python `_extract_pre_period_params` anticipation filter isn't R-parity-locked. Build a synthetic CS/SA result with `anticipation=1` and assert γ_p matches R's `slope_for_power()`. (Mechanism already covered by MC + full-VCV tests.) | `tests/test_methodology_pretrends.py`, `generate_pretrends_golden.R` | PR-C | Low |
Expand Down
2 changes: 1 addition & 1 deletion METHODOLOGY_REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -650,7 +650,7 @@ and covariate-adjusted specifications.)
**Deviations from the paper / from R / library extensions:** See REGISTRY.md `## WooldridgeDiD (ETWFE)` → `### Deviations from the paper / from R / library extensions` block for the consolidated list (HC1 finite-sample factor, QMLE sandwich `(n-1)/(n-k)` term, nonlinear-vs-fixest direct QMLE, logit cohort+time additive dummies, anticipation + aggregation, cell-count default with opt-in cohort-share).

**Outstanding Concerns:**
- **Stata `jwdid` golden values** (DEFERRED.md WooldridgeDiD follow-up cluster row): Stata-side parity infrastructure deferred until Stata install is available; R `etwfe` side covered in PR-B Stage D.
- **Stata `jwdid` golden values**: RESOLVED (#729, extended here). `benchmarks/data/etwfe_cs_stata_golden.json` ships four arms -- `csdid`, `jwdid`, `jwdid_never` (the issue #724 reference-period anchor) and `jwdid_alltreated` (the W2025 Section 5.4 anchor) -- generated with local StataSE 19 and pinned by `tests/test_etwfe_cs_stata_parity.py`. Point estimates match to ~1e-15. What remains open is the `hc1` SE gap (library SEs uniformly SMALLER than jwdid's by a cluster-count-dependent factor: 1.0280@G=20, 1.0132@G=40, 1.00264@G=191, 1.0010@G=500), tracked in TODO.md; each arm pins its own measured ratio because the constant does not transfer between cluster counts. No QMLE (logit/Poisson) golden exists yet -- every arm is linear `jwdid` -- so the QMLE cluster-SE comparison in DEFERRED.md stays open. R `etwfe` side covered in PR-B Stage D.
- **Response-scale APE / log-link bridge for Poisson + logit R parity** (DEFERRED.md WooldridgeDiD follow-up cluster row, added in PR-B): direct cell-level numerical parity between diff-diff's response-scale ATT and R `etwfe` log-link coefficients requires either `emfx()`-based APE extraction on the R side or link-function inversion with baseline-mean adjustment.
- **QMLE sandwich Stata-parity `qmle` weight type** (DEFERRED.md WooldridgeDiD follow-up cluster row): diff-diff's `(G/(G-1)) × ((n-1)/(n-k))` is conservative vs Stata's `G/(G-1)` only; awaiting Stata golden values to confirm material difference.
- **Repeated cross-sections** (paper p. 2581 → Deb et al. 2024): not in 2025 paper's main body; future PR.
Expand Down
Loading
Loading