Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,47 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
- **Post-fit aggregation: `results.aggregate(type=...)` on CallawaySantAnna
(4.0 program Phase 2b, ledger rows M-020 / M-117 / M-122).** Estimate once,
aggregate as a post-fit step — the ecosystem norm (`did::aggte`,
`etwfe::emfx`, Stata `estat aggregation`). `results.aggregate("event_study")`
returns the unified `EventStudyResults` container, finally giving [M-092]'s
export a public producer; `"group"` and `"simple"` return the new
`AggregationResult`, a tabular container with a pinned 12-column schema and
`summary()` / `to_dict()` / `to_dataframe()`. **`fit(aggregate=)` and
`fit(balance_e=)` now emit a `FutureWarning`** (removed at 4.0) via a
sentinel default, so the warning fires only when the argument is actually
passed and never on a plain `fit()`; the deprecated path still returns the
fully populated legacy surface. `balance_e` moves onto `aggregate()`, where
it applies to event-study aggregation only — passing it with `"simple"` or
`"group"` raises rather than being silently ignored, matching where the
shipped code actually threads it.
**No numerical change:** post-fit `aggregate()` reproduces the fit-time
numbers at `atol=rtol=1e-14` for every aggregation type, across a `balance_e`
sweep, a balanced panel, repeated cross-sections and `anticipation=1`.
Three schema decisions are grounded in what the estimator produces rather
than assumed: `target` is per-row so one container can carry two aligned
estimands; `n_kind` draws on a vocabulary now shared with
`EventStudyResults` and validated on construction (`"cells"` for CS's group
level, `"units"` for the overall); and `weight` is `None` for `"group"`,
because that aggregation weights `(g,t)` cells equally *within* each cohort
and forms no cross-cohort mass — reporting one would be fabricated.
Fail-closed elsewhere too: `aggregate("calendar")` raises (CS has no
calendar aggregator; the DEFERRED row stands), a non-`None` `weights`
raises, and `aggregate()` on a **bootstrapped** fit raises rather than
substituting analytical inference for percentile-bootstrap statistics —
bootstrap replay is tracked in `TODO.md`.
`AggregationResult.df` is per-row provenance: the degrees of freedom that
actually produced that row's stored p-value and interval, read from the
carrier the fit used (`survey_metadata.df_survey` on explicit survey
designs, `df_inference` only as the bare-`cluster=` fallback).
Internally, the CS event-study aggregator became **pure**: it returned one
dict and stashed four more values on `self`, and now returns all five, which
is what allows re-aggregation without mutating the result.
`StaggeredTripleDifference` inherits the same mixin and is the only reader of
the Eq. 4.14 overall, so it moved in the same change. Aggregation reads the
retained bookkeeping instead of the fit-time frame, so **no results object
retains the source panel**.
- **Internal: 4.0 gating-completeness amendment to the design spec and ledger.**
An audit of the public surface against the spec's own contract-rename rules
found two renames the initial inventory missed and three Phase 2 obligations
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ For rigorous DiD analysis, follow these 8 steps. Skipping diagnostic steps produ
4. **Choose estimator** - staggered adoption -> CS/SA/BJS (NOT plain TWFE); few treated units -> SDiD; factor confounding -> TROP; simple 2x2 -> DiD. Run `BaconDecomposition` to diagnose TWFE bias.
5. **Estimate** - `estimator.fit(data, ...)`. Always print the cluster count first and choose inference method based on the result (cluster-robust if >= 50 clusters, wild bootstrap if fewer).
6. **Sensitivity analysis** - `compute_honest_did(results)` for bounds under PT violations (MultiPeriodDiD, CS, or dCDH), `run_all_placebo_tests()` for 2x2 falsification, specification comparisons for staggered designs.
7. **Heterogeneity** - CS: `aggregate='group'`/`'event_study'`; SA: `results.event_study_effects` / `to_dataframe(level='cohort')`; subgroup re-estimation.
7. **Heterogeneity** - CS: `results.aggregate('group')`/`'event_study'` (post-fit, no refit); SA: `results.event_study_effects` / `to_dataframe(level='cohort')`; subgroup re-estimation.
8. **Robustness** - compare 2-3 estimators (CS vs SA vs BJS), report with and without covariates (shows whether conditioning drives identification), present pre-trends and sensitivity bounds.

Full guide: `diff_diff.get_llm_guide("practitioner")`.
Expand Down
3 changes: 3 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ Related tracking surfaces:
| Issue | Location | Origin | Effort | Priority |
|-------|----------|--------|--------|----------|
| `SyntheticControl` conformal (CWZ 2021) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies — the residual-permutation shortcut is only valid for time-permutation-invariant proxies (SC/Lasso/DiD); an AR proxy needs innovation permutation. | `diff_diff/conformal.py`, `diff_diff/synthetic_control_results.py` | CWZ-2021 | Heavy | Low |
| Make the post-fit `results.aggregate("event_study")` container consumable downstream. `EventStudyResults` is rejected by all THREE consumers that read a CS event study — `compute_honest_did` (`honest_did.py`, dispatches on `CallawaySantAnnaResults` and raises `TypeError`), `compute_pretrends_power` (`pretrends.py`, same), and `plot_event_study` (`visualization`, same) — so `fit(aggregate="event_study")` is still the only route for them and their error messages say so explicitly. Needs an `EventStudyResults` branch in each extraction path (consuming `event_time` / `is_reference` / `vcov` / `vcov_index` / per-row `df`) PLUS `base_period` and `anticipation` provenance, which the unified container does not carry and HonestDiD needs for its universal-base-period warning and pre-period classification. Gate with end-to-end tests: `compute_honest_did(res.aggregate("event_study"))` at `base_period="universal"`, and `compute_pretrends_power(...)` at `anticipation=1`. | `diff_diff/honest_did.py`, `diff_diff/pretrends.py`, `diff_diff/results_base.py` | #726 | Mid | Medium |
| Bootstrap re-aggregation for `CallawaySantAnnaResults.aggregate()` — a bootstrapped fit currently RAISES rather than substituting analytical inference for percentile-bootstrap statistics. The value-bound `BootstrapReplaySpec` (bit-identical replay, picklable, immune to post-fit `set_params`) is already in-tree and spike-verified; wiring it needs the per-`(g,t)` / per-event-time draw retention plus `assert_allclose` parity tests against the fit-time bootstrap numbers (NOT bit-identity — the fused GEMM's column count differs post-fit, ~1 ULP reassociation). | `diff_diff/aggregation.py`, `diff_diff/staggered_results.py` | #726 | Mid | Medium |
| Consolidate the inference-df precedence duplicated across `honest_did.py` (3 copies at ~L655/L836/L1004) onto the shared `resolve_inference_df()` helper added in `diff_diff/aggregation.py`. The copies are correct today; the risk is drift if the survey/replicate precedence changes in one place only. | `diff_diff/honest_did.py` | #726 | Quick | Low |
| `ContinuousDiD` CGBS-2024 remaining extensions (earlier phases — `covariates=` reg/dr, `treatment_type="discrete"`, single-cohort `control_group="lowest_dose"` with estimand `ATT(d)−ATT(d_L)` — are already supported; see REGISTRY Note #7). Remaining (all deferred `NotImplementedError`, documented): `estimation_method="ipw"` on the dose curve (scalar-adjustment / degenerate); `covariates=` × `survey_design=` (weighted OR + weighted nuisance IF); multi-cohort **heterogeneous-support** discrete aggregation (support-aware: average each dose only over the cohorts that observe it); **multi-cohort `lowest_dose`** (within-cohort `d_L` reference + support-aware cross-cohort aggregation); and **`covariates=` × `lowest_dose`** (conditional-PT-relative-to-`d_L` estimand). Single-cohort / 2-period / shared-support multi-cohort are supported. | `continuous_did.py` | CGBS-2024 | Heavy | Low |

### Performance
Expand Down
4 changes: 4 additions & 0 deletions diff_diff/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@
)
from diff_diff._guides_api import get_llm_guide
from diff_diff.agent_workflow import agent_workflow
from diff_diff.aggregation import (
AggregationResult,
)
from diff_diff.bacon import (
BaconDecomposition,
BaconDecompositionResults,
Expand Down Expand Up @@ -586,6 +589,7 @@
"BaseResults",
"Diagnostic",
"EventStudyResults",
"AggregationResult",
]

# Agent-facing entrypoints surface first in dir(diff_diff). LLM agents
Expand Down
Loading
Loading