docs(v4): 4.0 design spec + CI-enforced deprecation ledger (Phase 1)

Normative design for the 4.0 program (docs/v4-design.md): three estimator
merges (TWFE absorbs MultiPeriodDiD with a unit-FE event-study default and
spec="pooled" compatibility, TripleDifference facade over the DDD pair, CiC
absorbs QDiD via method=), post-fit results.aggregate() with the unified
{simple, event_study, group, calendar} vocabulary, canonical results quintet
storage flip, library-wide param/column contract rules (post vs time,
first_treat, covariates, no _col suffixes), one inference surface, the
final alias table, and the 3.9-shim-then-4.0-enforce release sequencing
with the maint/3.8 patch-line rule.

Every deprecation lifecycle is a row in docs/v4-deprecations.yaml (69 rows:
the six locked design decisions expanded per-surface plus all five
pre-existing 4.0 obligations found by repo sweep, including the
df_convention default flip that carries no in-code warning).
tests/test_v4_matrix.py asserts every row's status against reality at HEAD:
planned rows tripwire on new surface appearing unflipped, shimmed/removed
rows pin both sides plus dedicated-test existence, alias rows assert
identity and __all__ membership, default-flip rows pin signature defaults,
and the release gate is two-sided (due rows must flip; early removals
before the scheduled version fail). Wired into docs-tests.yml so docs-only
ledger/spec edits cannot bypass enforcement. REGISTRY.md MultiPeriodDiD
gains a labeled Note reconciling the unit-FE parity target with the 3.x
pooled default and the scheduled 4.0 migration.

Docs + test only; no public API or numerical behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017eVKbBqzenDKa5idwtexQa

Author: igerber

.claude/memory.md

@@ -81,22 +81,36 @@ results = estimator.fit(df, outcome='y', treatment='treated', ...)
 print(results.summary())
 ```
 
-### Results Objects
-All results have:
-- `.att` - Point estimate
-- `.se` - Standard error
-- `.pvalue` - Two-sided p-value
-- `.ci` - Tuple of (lower, upper) confidence interval
-- `.summary()` - Print formatted summary
-- `.to_dict()` - Export to dictionary
-- `.to_dataframe()` - Export to DataFrame
-
-### Column Naming
-- `unit` or `unit_id` - Unit identifier
-- `time` or `time_id` - Time period identifier
-- `treated` - Binary treatment indicator (0/1)
-- `post` - Binary post-period indicator (0/1)
-- `cohort` or `treatment_time` - First treatment period for staggered designs
+### Results Objects — CURRENT (3.x)
+The canonical inference quintet is `att` / `se` / `t_stat` / `p_value` /
+`conf_int` (never `.pvalue` / `.ci`). Every results class exposes all five,
+but native STORAGE varies in 3.x (`overall_att` on the staggered family,
+`avg_att` on MultiPeriodDiD, with canonical names as properties). `summary()`
+everywhere; `to_dict()` / `to_dataframe()` on most classes. See
+`.claude/../docs/methodology/REGISTRY.md` per estimator.
+
+### Results Objects — 4.0 TARGET (do not write against this pre-4.0)
+Canonical quintet becomes the native fields on every class; `overall_att`
+family becomes FutureWarning properties (removed 5.0); one unified
+event-study representation; aggregation via post-fit
+`results.aggregate(type=)`. Normative spec: `docs/v4-design.md`; per-surface
+lifecycle: `docs/v4-deprecations.yaml` (CI-enforced by
+`tests/test_v4_matrix.py`).
+
+### Column Naming — CURRENT (3.x)
+- `unit` unit id (`unit_col` on HAD; `group` on dCDH — both slated for 4.0)
+- `time` calendar period, EXCEPT DifferenceInDifferences / TripleDifference /
+  static TwoWayFixedEffects, where `time` is the 0/1 post dummy (TWFE warns
+  on >2 unique values; 4.0 renames these to `post`, and TWFE's `time`
+  becomes the event-study calendar column)
+- `treatment` 0/1 treated-group indicator; `first_treat` cohort column
+  (`cohort` on WooldridgeDiD — slated for 4.0)
+- `covariates` covariate list (`controls` on dCDH — slated for 4.0)
+
+### Column Naming — 4.0 TARGET
+`outcome` / `unit` / `time` (calendar) / `post` (0/1) / `treatment` (0/1) /
+`first_treat` / `covariates` / `partition` (DDD), no `_col` suffixes. Rules:
+`docs/v4-design.md` section 8.
 
 ## Session Notes
 

.github/workflows/docs-tests.yml

@@ -8,6 +8,7 @@ on:
       - 'diff_diff/**'
       - 'tests/test_doc_snippets.py'
       - 'tests/test_doc_deps_integrity.py'
+      - 'tests/test_v4_matrix.py'
       # tests/conftest.py is auto-loaded by pytest for the snippet
       # test run and mutates sys.path + MPLBACKEND (conftest.py:14, 18);
       # changes there can break snippet exec without touching the test
@@ -26,6 +27,7 @@ on:
       - 'diff_diff/**'
       - 'tests/test_doc_snippets.py'
       - 'tests/test_doc_deps_integrity.py'
+      - 'tests/test_v4_matrix.py'
       - 'tests/conftest.py'
       - 'pyproject.toml'
       # sphinx-build job mirrors RTD setup; trigger when RTD config drifts
@@ -79,6 +81,13 @@ jobs:
         # step does not install diff_diff.
         run: PYTHONPATH=. DIFF_DIFF_BACKEND=python pytest tests/test_doc_deps_integrity.py -v
 
+      - name: Run v4 deprecation-matrix enforcement
+        # Asserts every docs/v4-deprecations.yaml row's status against reality
+        # at HEAD (schema: docs/v4-design.md section 11). Runs here so
+        # ledger/spec-only edits (docs/** paths) cannot bypass enforcement -
+        # the main test matrix does not trigger on docs-only diffs.
+        run: PYTHONPATH=. DIFF_DIFF_BACKEND=python pytest tests/test_v4_matrix.py -v
+
   sphinx-build:
     name: Sphinx HTML build (-W warnings as errors)
     # Skip unrelated label churn: a non-ready-for-ci label add/remove won't run this job.

CHANGELOG.md

@@ -5,6 +5,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+- **Internal: 4.0 API design spec + machine-checked deprecation matrix.** The
+  normative design for the 4.0 program (three estimator merges, post-fit
+  `results.aggregate()`, canonical results quintet, library-wide param/column
+  unification, 3.9-shim-then-4.0-enforce sequencing) lives in
+  `docs/v4-design.md`; every deprecation's lifecycle is a row in
+  `docs/v4-deprecations.yaml`, enforced against reality at HEAD
+  by `tests/test_v4_matrix.py` (new-surface tripwires, removal pins,
+  default-flip and warning-retirement sweeps). Repo-internal documentation +
+  test only - **no public API or numerical behavior change.**
+
 ## [3.8.0] - 2026-07-18
 
 ### Added

CLAUDE.md

@@ -176,6 +176,7 @@ When adding new functionality, the source of truth is:
 | File | Contains |
 |------|----------|
 | `docs/methodology/REGISTRY.md` | Academic foundations, equations, edge cases — **consult before methodology changes** |
+| `docs/v4-design.md` + `docs/v4-deprecations.yaml` | 4.0 program design spec + CI-enforced deprecation ledger — **consult before any 4.0-program PR**; deviations must edit both in the same diff |
 | `docs/doc-deps.yaml` | Source-to-documentation dependency map — **consult when any source file changes** |
 | `CONTRIBUTING.md` | Documentation requirements, test writing guidelines |
 | `.claude/commands/dev-checklists.md` | Checklists for params, methodology, warnings, reviews, bugs (run `/dev-checklists`) |

ROADMAP.md

@@ -8,6 +8,10 @@ Forward-looking plan for diff-diff, organized as queued work, candidates under c
 
 Queued work, ordered by expected leverage. Each item is its own PR. Ordering is priority-sequenced, not time-committed.
 
+### 4.0 API unification
+
+- **The 4.0 program: estimator consolidation + one API contract.** Three merges (TwoWayFixedEffects absorbs MultiPeriodDiD, TripleDifference absorbs StaggeredTripleDifference, ChangesInChanges absorbs QDiD), post-fit `results.aggregate()`, a canonical results quintet, and library-wide column/param unification - sequenced as a 3.9 shim release (new surface + FutureWarnings), 4.0 enforcement, and a 4.1 `event_study()` comparison front door. Normative spec: `docs/v4-design.md`; every queued deprecation is tracked in `docs/v4-deprecations.yaml` and enforced by CI (`tests/test_v4_matrix.py`).
+
 ### Practitioner-ready output
 
 - **Context-aware `practitioner_next_steps()`.** Substitutes actual column names from fitted results instead of generic placeholders, so next-step guidance is executable rather than illustrative. (Standalone follow-up to the `BusinessReport` / `DiagnosticReport` layer; tracked under the AI-Agent Track too.)

TODO.md

@@ -42,7 +42,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 |-------|----------|--------|--------|----------|
 | `EfficientDiD` conditional path: the largest remaining O(n) stage is the sieve/nuisance construction outside the tiled pass (~9s at 10k). (The `_ridge_solve_weights` Python-prep shave landed 2026-07-07 — the `omega_stack[rest]` fancy-index copy and tail scatter are skipped when no row is zero-masked, byte-identical outputs; the `zero_mask` abs scan itself remains, needed for correctness.) | `efficient_did_covariates.py` | CS-scaling | Mid | Low |
 | `_rq_fit` LP assembly is dense (`A_eq = [X, I, -I]` with dense identity blocks, rebuilt per cell fit): a `scipy.sparse` construction would cut memory and likely HiGHS time for large cells / bootstrap-heavy covariate CiC/QDiD fits. CAVEAT before doing it: a different matrix representation can change HiGHS's vertex selection at degenerate/tied QR optima - end-to-end covariate goldens are tie-selection-gated (fine), but the `qr_cases` tight coefficient matches may shift to the equal-loss branch; re-run the parity suite and re-calibrate if needed. | `diff_diff/changes_in_changes.py::_rq_fit` | covariates PR | Quick | Low |
-| Evaluate flipping `DIFF_DIFF_SOLVE_OLS_FASTPATH` default-ON after an opt-in soak (the 2026-07 certified normal-equations Cholesky fast path, both backends). A flip needs: golden/parity-suite recapture at the tol-bounded posture (fitted ~1e-8 abs / SE ~1e-6 rel — the default today is byte-pinned in several benchmark conventions), certification-rate telemetry across real workloads (any decline is silent-correct but forfeits the speedup), and the staged default-flip protocol used for `df_convention` (v4-class change). | `diff_diff/linalg.py::_resolve_solve_ols_fastpath`, `rust/src/linalg.rs::solve_ols_chol` | CS-scaling | Mid | Low |
+| Evaluate flipping `DIFF_DIFF_SOLVE_OLS_FASTPATH` default-ON after an opt-in soak (the 2026-07 certified normal-equations Cholesky fast path, both backends). A flip needs: golden/parity-suite recapture at the tol-bounded posture (fitted ~1e-8 abs / SE ~1e-6 rel — the default today is byte-pinned in several benchmark conventions), certification-rate telemetry across real workloads (any decline is silent-correct but forfeits the speedup), and the staged default-flip protocol used for `df_convention` (v4-class change). Lifecycle tracked in docs/v4-deprecations.yaml (M-008). | `diff_diff/linalg.py::_resolve_solve_ols_fastpath`, `rust/src/linalg.rs::solve_ols_chol` | CS-scaling | Mid | Low |
 
 ### Testing / docs
 
@@ -119,7 +119,7 @@ Doable in principle, but no current caller and/or explicitly out of paper scope.
 | Staggered/multi-period distributional DiD (Athey-Imbens Section 6 / Ciaccio arXiv:2408.01208v2; `ecic` is the staggered event-study CiC lineage - a distinct method from Ciaccio's copula approach, do not conflate). Reviewed: `docs/methodology/papers/ciaccio-2024-review.md`; ROADMAP row is reviewed-deferred pending demand. | `diff_diff/changes_in_changes.py` | #682 | Low |
 | ChangesInChanges treatment-on-the-controls (Athey-Imbens Theorem 3.2: group-label exchange + negation; no qte equivalent to anchor parity). | `diff_diff/changes_in_changes.py` | #682 | Low |
 | Rust-backend CR2 Bell-McCaffrey port (`return_dof` in the Rust vcov dispatch + CR2 algebra) — **premise re-scoped 2026-07-09**: the scores-based DOF + low-rank factored `A_g` changes made the NumPy CR2-BM path BLAS-bound (`O(n_g k²)` per cluster; 4.1s→38ms at n=100k/k=40), so a Rust port buys ~nothing and adds a parity surface. Revisit only if profiling shows CR2-BM hot again. | `rust/src/linalg.rs` | — | Low |
-| Clustered-CR1 inference df **default flip to `"cluster"` (G−1) at v4** — the opt-in `df_convention=` knob landed 2026-07 (DiD/TWFE/MPD + LinearRegression; REGISTRY §TwoWayFixedEffects deviation note); the remaining work is the major-version default change (moves every clustered p-value/CI) + migration note + flipping `TestDfConvention`/`test_moderate_t_pins_residual_df_convention` expectations. Also evaluate extending the knob to standalone estimators with CR1-t inference at that time. | `diff_diff/linalg.py::LinearRegression`, `diff_diff/estimators.py`, `diff_diff/twfe.py` | — | Medium |
+| Clustered-CR1 inference df **default flip to `"cluster"` (G−1) at v4** — the opt-in `df_convention=` knob landed 2026-07 (DiD/TWFE/MPD + LinearRegression; REGISTRY §TwoWayFixedEffects deviation note); the remaining work is the major-version default change (moves every clustered p-value/CI) + migration note + flipping `TestDfConvention`/`test_moderate_t_pins_residual_df_convention` expectations. Also evaluate extending the knob to standalone estimators with CR1-t inference at that time. Lifecycle tracked in docs/v4-deprecations.yaml (M-004..M-006). | `diff_diff/linalg.py::LinearRegression`, `diff_diff/estimators.py`, `diff_diff/twfe.py` | — | Medium |
 | CallawaySantAnna **unbalanced-panel R parity — LANDED** via `allow_unbalanced_panel=True` (matches R `did::att_gt(allow_unbalanced_panel=TRUE)` / `DRDID::reg_did_rc`: ATT bit-exact on cells AND dynamic aggregation via fixed unit-cohort-mass `pg` + a per-unit WIF; SE up to the documented CR1 `sqrt(G/(G-1))` factor). The earlier "weighting" framing was a mis-diagnosis — on unbalanced panels the dominant divergence from R is the *estimator* (within-cell differencing vs RC-on-pooled-obs), not only the weighting; both are resolved by the flag. The DEFAULT path keeps within-cell differencing as a documented design choice and now emits a `UserWarning` on unbalanced input (no-silent-failures). **Remaining deferred:** `survey_design=` × `allow_unbalanced_panel=` (per-obs vs per-unit weight resolution — currently fail-closed `NotImplementedError`); and covariate / ipw / dr × the flag R-parity verification (the RC path supports them; the committed golden covers `reg` no-cov). | `staggered.py`, `staggered_aggregation.py` | SE-audit D3 | Low |
 | CallawaySantAnna event-study bucket/weight construction is duplicated between the analytical aggregator (`staggered_aggregation.py::_aggregate_event_study`) and the multiplier bootstrap (`staggered_bootstrap.py`): both group (g,t) by `e = t - g`, apply the finite/NaN/reference masks, and read cohort weights. Both already consume the same source-materialized universal reference cells (so they agree), but the bucket logic is copy-pasted. Extract one shared helper returning per-event-time buckets (finite cells, NaN cells, reference flags, cohort weights, combined-IF inputs) used by both. Pure refactor; gate on byte-identical analytical + bootstrap output. | `staggered_aggregation.py`, `staggered_bootstrap.py` | SE-audit D3 | Low |
 | `StackedDiD` survey re-resolution intra-file dedup (raw-weight extraction ×3, compose-normalize ×3, resolve-on-stacked ×2). The cross-estimator ContinuousDiD/EfficientDiD panel-to-unit collapse consolidation LANDED (#226 shared helpers `ResolvedSurveyDesign.subset_to_units_by_row_idx` / `build_unit_first_row_index`); StackedDiD is deliberately NOT on that path (control units are duplicated across sub-experiments, so it re-resolves at stacked granularity rather than collapsing to one row per unit). The residual is stacked-specific, low value, and touches the numerically-sensitive composed-weight renormalization. Post-filter re-resolution / metadata-recompute unification across the three estimators was assessed and is not warranted — they use genuinely different mechanisms and already delegate to shared `_resolve_survey_for_fit` / `compute_survey_metadata`. | `stacked_did.py` | #226 | Low |

docs/conf.py

@@ -56,6 +56,7 @@
     "performance-scenarios.md",
     "practitioner-guide-evaluation.md",
     "survey-roadmap.md",
+    "v4-design.md",
     "methodology/continuous-did.md",
     "methodology/survey-theory.md",
     # Internal paper-review notes (methodology validation artifacts).

docs/methodology/REGISTRY.md

@@ -208,6 +208,22 @@ where interactions are included for ALL periods (pre and post), not just post-tr
 Pre-treatment coefficients (e < -1) test the parallel trends assumption:
 under H0 of parallel trends, δ_e = 0 for all e < 0.
 
+- **Note:** The 3.x DEFAULT implementation estimates the pooled form of this
+  target - treatment-group main effect + period dummies + interactions, with
+  NO unit fixed effects unless `absorb`/`fixed_effects` is passed
+  (`estimators.py` design build). The unit-FE form above (the
+  `fixest::feols(... | unit + time)` parity target) and the pooled default
+  produce identical point estimates only under balanced panels with no
+  covariates and simultaneous adoption; with unbalanced panels or covariates
+  the two projections differ. The 4.0 program (see `docs/v4-design.md`
+  section 4.1 and `docs/v4-deprecations.yaml` [M-010]) migrates the
+  event-study default to the unit-FE spec on the merged TwoWayFixedEffects,
+  keeping the pooled model reachable via `spec="pooled"` (required for
+  repeated cross-sections and for reproducing 3.x MPD numbers exactly); the
+  Phase 3 PR gates on a balanced-equivalence test, an
+  unbalanced-or-covariate divergence test, and pooled bit-exact parity vs
+  3.x MPD.
+
 Post-treatment coefficients (e ≥ 0) estimate dynamic treatment effects.
 
 Average ATT over post-treatment periods: