Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
purpose-phrasing, restoring the case as a genuinely clean negative control.

### Added
- **Internal: 4.0 diagnostic-family amendment to the design spec.** `docs/v4-design.md`
gains section 3.5 formalizing a third object kind - diagnostics: the library's
existing "Diagnostics & Sensitivity" family (parallel-trends testing, placebo
suite, HonestDiD, pre-trends power, power analysis) made checkable, with
`BaconDecomposition` and `RDPlot` re-homed into it at 4.0 and the upcoming
density test born into it. One load-bearing bit: estimator results carry the
canonical inference quintet, diagnostic results do not (exempt BY TYPE via a
single `Diagnostic` marker base on the diagnostic RESULT containers, shipping
additively in Phase 2 with a ledger-gated roster test (M-091) - zero new
classes beyond the base, no import-path moves; raw-dict diagnostic functions
participate in the docs family only). The 4.0 docs/roster
reorganization is ledger-tracked (M-090) so the release cut cannot miss it,
and `RDPlot.fit()`'s post-freeze `outcome_col`/`running_col` params get their
standard rename rows (M-088/M-089) under the spec's missed-rename clause.
Docs + ledger + test-count updates only - **no public API or numerical behavior
change.**
- **Reviewer-eval harness: N-arm matrix, blinded grading, corpus grown 2 -> 11
(`tools/reviewer-eval/`, prep for the GPT-5.6 reviewer evaluation).**
`config/configs.json` moves from the two-arm `control`/`candidate` shape to an
Expand Down
50 changes: 50 additions & 0 deletions docs/v4-deprecations.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -869,6 +869,56 @@ rows:
code_refs: [diff_diff/wooldridge_results.py]
notes: "summary() unifies to the library-wide summary(alpha=None) signature (spec section 5); aggregation selection lives on aggregate(). The alpha param arrives additively in the same PR."

- id: M-088
kind: param
group: renames-col-suffix
old: "diff_diff:RDPlot.fit[outcome_col]"
new: "diff_diff:RDPlot.fit[outcome]"
introduced_in: "3.9"
deprecated_in: "3.9"
removed_in: "4.0"
status: planned
phase: 2
warning: FutureWarning
code_refs: [diff_diff/rdplot.py]
notes: "RDPlot shipped post-ledger-freeze with _col suffixes; the spec section 8 missed-rename clause applies (matches RegressionDiscontinuity [M-040])."
- id: M-089
kind: param
group: renames-col-suffix
old: "diff_diff:RDPlot.fit[running_col]"
new: "diff_diff:RDPlot.fit[running]"
introduced_in: "3.9"
deprecated_in: "3.9"
removed_in: "4.0"
status: planned
phase: 2
warning: FutureWarning
code_refs: [diff_diff/rdplot.py]
notes: "Companion to [M-088]; matches [M-041]."
- id: M-090
kind: behavior
group: diagnostic-family
old: "diff_diff:BaconDecomposition"
new: null
deprecated_in: "4.0"
removed_in: null
status: planned
phase: 5
code_refs: [README.md, diff_diff/guides/llms.txt, docs/api/index.rst, docs/index.rst, diff_diff/bacon.py]
notes: "4.0 diagnostic-family docs/roster reorganization (spec section 3.5): Bacon re-homed OUT of the API estimator roster (misfiled today); RDPlot (already documented as a diagnostic) consolidated under the unified family grouping; README/llms.txt/API/alias groupings (incl. docs/api/index.rst AND the docs homepage docs/index.rst, both listing Bacon as an estimator today) and estimator-count claims split into estimators + diagnostics; the flip's terminal test asserts the estimator/diagnostic grouping on the named surfaces. The Phase 2 marker introduction is gated separately by [M-091]."
- id: M-091
kind: behavior
group: diagnostic-family
old: "diff_diff:BaconDecompositionResults"
new: null
introduced_in: "3.9"
deprecated_in: null
removed_in: null
status: planned
phase: 2
code_refs: [diff_diff/bacon.py, diff_diff/rdplot.py, diff_diff/honest_did.py, diff_diff/pretrends.py, diff_diff/power.py, diff_diff/diagnostics.py, diff_diff/had_pretests.py, diff_diff/diagnostic_report.py, diff_diff/business_report.py, diff_diff/practitioner.py, diff_diff/_reporting_helpers.py, diff_diff/__init__.py]
notes: "Phase 2 Diagnostic marker base on the RESULT roster (spec section 3.5): BaconDecompositionResults, RDPlotResult, HonestDiDResults + SensitivityResults, PreTrendsPowerResults + PreTrendsPowerCurve, PowerResults + Simulation*Results, PlaceboTestResults, and the HAD pretest containers (QUGTestResults, StuteTestResults, YatchewTestResults, StuteJointResult, HADPretestReport), and DiagnosticReportResults (serialization pair moves/delegates from the builder onto the container). Flip to done ALSO requires consumer-propagation tests: BusinessReport rejects marked diagnostics as primary estimator input BY TYPE (not by name); practitioner_next_steps routes marked diagnostics through diagnostic handling (not the unknown-result estimator fallback); and DiagnosticReport routes by the marker - at least one non-Bacon marked result handled as a diagnostic, never via estimator fallback, with Bacon's existing read-out retained and tested. Flip to done requires the dedicated roster test (isinstance + summary/to_dataframe on every member; estimator results NOT marked). Raw-dict functions and TWFEWeightsResult are docs-family only (narrowed, spec section 3.5). introduced_in gates the 3.9 cut: the marker cannot be silently skipped."

# ---- Behavior policies (schema-tracked, spec-governed; no reality probe) -
- id: M-080
kind: behavior
Expand Down
101 changes: 92 additions & 9 deletions docs/v4-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ Derived from `diff_diff.__all__` at v3.8.0. 24 estimator classes -> 21.
| ChangesInChanges | Keep + absorbs QDiD via `method=` [M-015] |
| QDiD | Removed 4.0 [M-015] |
| SyntheticDiD, SyntheticControl, TROP, ContinuousDiD, HeterogeneousAdoptionDiD, RegressionDiscontinuity, SpilloverDiD, ChaisemartinDHaultfoeuille | Keep |
| BaconDecomposition | Keep (documented as a diagnostic, not an estimator, in the API docs and alias-table grouping) |
| BaconDecomposition | Keep - moves into the diagnostic family (section 3.5) |

### 3.2 Final alias table

Expand Down Expand Up @@ -148,6 +148,86 @@ signatures. Lifecycle facts live in the cited rows.
[M-045]..[M-047] - fully redundant with `vcov_type`, and its default even
differed across estimators (True/True/False).

### 3.5 The diagnostic family

diff-diff 4.0 formalizes a THIRD object kind alongside estimators and
results: **diagnostics** (locked with the maintainer 2026-07-19). Exactly
ONE bit is load-bearing: an estimator's result carries a causal-effect
inference row (the section 5 quintet); a diagnostic's result does NOT -
it assesses a design, an identifying assumption, or robustness (a
decomposition table, binned plot data, a pre-trends p-value, sensitivity
bounds, a power curve).

**Membership: a NEW CANONICAL CONSOLIDATION anchored on the
"Diagnostics & Sensitivity" docs family** (which today is close but not
identical - the consolidation is the source of truth from 4.0):
BaconDecomposition (reclassified OUT of the README/API estimator lists),
RDPlot (consolidated from the RD grouping), parallel-trends testing, the
placebo suite, HonestDiD, PreTrendsPower, PowerAnalysis,
`twowayfeweights`, the HAD pretests, DiagnosticReport's results, and the
upcoming CJM 2020 density test (born into the family). Explicit
NON-members despite current docs placement: Conley spatial-HAC (an
embedded inference method on estimators, not a diagnostic object) and
other design elements listed under the README section. (A narrower
classes-only tier with separate categories for HonestDiD/PowerAnalysis
was drafted and REJECTED 2026-07-19: the boundary needed adjudication,
and the family line above is the decided one.)

**Mechanics - the marker lives on RESULT containers.** Consumers hold
results, not entry points, so the `Diagnostic` marker base (shipping
ADDITIVELY in Phase 2 [M-091]) attaches to the class-backed diagnostic
RESULT types: `BaconDecompositionResults`, `RDPlotResult`,
`HonestDiDResults` + `SensitivityResults`, `PreTrendsPowerResults` +
`PreTrendsPowerCurve`, `PowerResults` + the `Simulation*Results` family,
`PlaceboTestResults`, the HAD pretest containers (`QUGTestResults`,
`StuteTestResults`, `YatchewTestResults`, `StuteJointResult`,
`HADPretestReport` - all already exposing the serialization pair), and
`DiagnosticReportResults` (whose `summary()`/`to_dataframe()` live on the
BUILDER today - Phase 2 moves or delegates them onto the container to meet
the contract; BusinessReport exports no result container, so nothing to
mark there) - plus every future diagnostic's result type (the density
test's first). Entry-point classes are NOT marked (one
marker, one meaning). Contract: marked results expose `summary()` /
`to_dataframe()` and are exempt from the section 5 quintet BY TYPE
(isinstance-checkable, so BusinessReport / DiagnosticReport /
practitioner routing stop special-casing by result-class name). The
family's entry points keep their existing shapes - data-in (Bacon,
RDPlot) and results-in (HonestDiD, PreTrendsPower) both belong; the
marker does not constrain them. NARROWING: function-shaped members that
return raw dicts (`check_parallel_trends` and variants,
`equivalence_test_trends`) and `TWFEWeightsResult` (no
`summary()`/`to_dataframe()` today) participate in the DOCS family only,
not the type contract; upgrading them to marked containers is optional
Phase 2 follow-up work, not part of this contract. The Phase 2 gate
[M-091] requires a dedicated roster test AND consumer propagation: every
enumerated result type is `isinstance(result, Diagnostic)` and exposes
the serialization pair, representative ESTIMATOR results are NOT, and the
CONSUMERS actually switch to the marker - BusinessReport rejects marked
diagnostics as its primary estimator input by type (today it
special-cases only Bacon by name), `practitioner_next_steps()` routes
marked diagnostics through diagnostic-specific handling instead of its
unknown-result estimator fallback, and DiagnosticReport itself routes by
the marker: at least one non-Bacon marked result is handled as a
diagnostic (never through estimator fallback) while Bacon's existing
read-out behavior is retained and tested. Import paths do NOT move - the
flat top-level namespace is kept (rejected: `diff_diff.diagnostics.*`
moves). Zero new classes beyond the single marker.

**4.0 reorganization [M-090].** At 4.0 the public surface claims split
into "estimators + diagnostics": Bacon is re-homed OUT of the API
estimator roster (where it is misfiled today); RDPlot - already
documented under Diagnostics & Sensitivity - is consolidated under the
unified family grouping; README catalog, llms.txt, API reference and
alias-table groupings, and estimator-count claims on JOSS-adjacent
surfaces all split accordingly. Tracked as a behavior row so the cut
sweep cannot miss it.

**RDPlot contract compliance [M-088] [M-089].** `RDPlot.fit()` shipped
with `outcome_col` / `running_col` after the ledger froze - the section 8
missed-rename clause applies, and the params migrate to bare `outcome` /
`running` on the standard 3.9-shim / 4.0-removal path (matching
RegressionDiscontinuity's [M-040] [M-041]).

## 4. The three merges

### 4.1 TwoWayFixedEffects absorbs MultiPeriodDiD [M-010]
Expand Down Expand Up @@ -272,9 +352,11 @@ removed [M-015] [M-061].

**Canonical quintet.** `att`, `se`, `t_stat`, `p_value`, `conf_int` - bound to
ONE coherent inference row (locked library principle: uniform names never
carry altered meaning). As of v3.8.0 every results class already exposes the
full quintet as properties (verified 2026-07-18), so 3.9 needs no additive
property work. At 4.0 the STORAGE flips [M-050]..[M-058]: the canonical names
carry altered meaning). The quintet contract applies to ESTIMATOR results;
diagnostic-family results (section 3.5) are exempt BY TYPE - they expose
`summary()` / `to_dataframe()` but no inference row. As of v3.8.0 every
ESTIMATOR result class already exposes the full quintet as properties
(verified 2026-07-18), so 3.9 needs no additive property work. At 4.0 the STORAGE flips [M-050]..[M-058]: the canonical names
become the native dataclass fields; `overall_att` (and sibling `overall_*`
inference fields, plus ContinuousDiD's deviant `overall_att_*` family
[M-058]) become FutureWarning properties, removed 5.0. MultiPeriodDiD's
Expand Down Expand Up @@ -409,10 +491,10 @@ above; anything only one PR cares about stays in that PR's plan.**
| Phase | Ships in | PRs (each: dedicated shim/removal tests + matrix flips + CHANGELOG naming flipped row ids) |
|---|---|---|
| 1 (this PR) | - | Spec + matrix + enforcement test + support edits |
| 2: contract foundations | 3.9 | (a) results base + unified event-study representation + to_dict completion; (b) `aggregate()` + fit(aggregate=) shims [M-020..M-027]; (c) param renames [M-030..M-047] + BaseEstimator mixin + ContinuousDiD covariates move; (d) alias introductions [M-062] [M-063] + wrapper deprecations [M-070..M-077] + n_bootstrap docs [M-081] |
| 2: contract foundations | 3.9 | (a) results base + unified event-study representation + to_dict completion + the Diagnostic marker base on the diagnostic result roster [M-091] (section 3.5); (b) `aggregate()` + fit(aggregate=) shims [M-020..M-027]; (c) param renames [M-030..M-047] + BaseEstimator mixin + ContinuousDiD covariates move; (d) alias introductions [M-062] [M-063] + wrapper deprecations [M-070..M-077] + n_bootstrap docs [M-081] |
| 3: merges | 3.9 | (a) TWFE event-study mode [M-010] + EventStudy warn [M-060] (gates: section 4.1's equivalence/divergence/pooled-parity test triple); (b) TripleDifference facade [M-013]; (c) CiC method= [M-015] |
| 4: release + soak | 3.9 cut | Migration guide written (skeleton: section 10); maintainer cuts 3.9; maint/3.8 rule active |
| 5: enforcement | 4.0 | Removals [M-010..M-016, M-030..M-047 old names, M-060, M-061, M-070..M-077, M-001..M-003]; storage flips [M-050..M-058]; default policies [M-004..M-006, M-080]; warning retirement [M-007]; fastpath go/no-go [M-008]; docs/llms.txt/README refresh |
| 5: enforcement | 4.0 | Removals [M-010..M-016, M-030..M-047 old names, M-060, M-061, M-070..M-077, M-001..M-003]; storage flips [M-050..M-058]; default policies [M-004..M-006, M-080]; warning retirement [M-007]; fastpath go/no-go [M-008]; diagnostic-family docs/roster reorganization [M-090]; docs/llms.txt/README refresh |
| 6: front door | 4.1 | `event_study(data, outcome, unit, time, first_treat, estimator=...)` comparison entry point over the staggered family (sketch only; specified in its own plan) |

**4.0-cut checklist (final item):** the due-row sweep is AUTOMATED -
Expand Down Expand Up @@ -505,8 +587,9 @@ forever - a removed symbol resurrecting is a test failure.
`removed_in` (symbol/alias rows), flip version (`deprecated_in` on
default-flip/warning-retirement/behavior rows), or `decision_due`
(env-default rows), the status must be terminal; a due `introduced_in`
means the row may no longer be `planned` (the new surface must have
shipped - this is what gates introduce-only aliases); symbol rows with a
means the row may no longer be `planned` OR `evaluate` (the new surface
must have shipped; evaluate cannot satisfy an introduction - this is what
gates introduce-only aliases and the Phase 2 marker); symbol rows with a
due `deprecated_in` and a declared `warning` may no longer be `planned`.
The gate is two-sided: an EARLY-removal guard fails any row that goes
terminal while its scheduled version is still in the future (the shim
Expand All @@ -527,7 +610,7 @@ forever - a removed symbol resurrecting is a test failure.
the same object as its target, so the deprecation warning rides the parent
class row (schema-enforced). Top-level `diff_diff:Name` class/function rows
and alias rows also assert `__all__` membership consistent with their
status (stale `import *` entries fail). The initial 71 row ids are a
status (stale `import *` entries fail). The initial 75 row ids are a
committed snapshot in the enforcement test: ids are never deleted or
reused, and the test fails if any snapshot id disappears.

Expand Down
Loading
Loading