Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,87 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]

### Added
- **Internal: 4.0 gating-completeness amendment to the design spec and ledger.**
An audit of the public surface against the spec's own contract-rename rules
found two renames the initial inventory missed and three Phase 2 obligations
the automated release gate could not see. (a) Two ledger rows added for
RESULTS fields that mirror already-tracked params: `M-094`
(`RegressionDiscontinuityResults.treatment_col` → `takeup`, mirroring
`M-042`) and `M-095` (`StackedDiDResults.clean_control` → `control_group`,
mirroring `M-043`). Both are public dataclass fields emitted by `to_dict()`,
so renaming the params alone would have left deprecated spellings on the
serialized surface that spec section 5 forbids from 4.0; each flips in the
same diff as its param sibling. `docs/v4-design.md` section 8 gains rule 9
making the mirror obligation normative, so the next rename sweep greps for it
rather than rediscovering it. (b) `M-081` (`n_bootstrap` semantic
unification) gains `introduced_in: 3.9` and a new `M-096` covers the uniform
`inference="wild_bootstrap"` exposure (spec section 7) - both were real Phase
2 obligations that no row gated, so a 3.9 cut could have shipped without
them. Both use `introduced_in` rather than `deprecated_in` deliberately: for
behavior rows the early-flip guard keys off `deprecated_in`, so a flip
version would have failed the very PR that implements the obligation (it
lands while `__version__` is still 3.8.x) - the `M-091`/`M-092` pattern.
`M-096` covers the `inference="wild_bootstrap"` selector contract, which
governs wild CLUSTER bootstrap offered as an alternative to analytical SEs -
a roster of `DifferenceInDifferences` + `TwoWayFixedEffects`. Two things the
audit established. (i) The selector does **not** fail closed today:
`inference=` is stored with no valid-value check anywhere, and DiD routes
`wild_bootstrap` without `cluster=` silently to analytical (a currently
*pinned* behavior), so a typo or a missing prerequisite quietly changes which
SE/p-value/CI procedure runs. Phase 2 must validate the accepted value set on
`__init__` and transactional `set_params` and handle the missing-cluster case
explicitly. (ii) The apparent `inference=` vs `n_bootstrap>0` split is not
drift - estimators whose bootstrap *is* their inference method run materially
different procedures (CallawaySantAnna an influence-function multiplier
bootstrap; SunAbraham a unit-pairs bootstrap, Rao-Wu rescaled on survey
paths; dCDH a group-level influence-function multiplier bootstrap upgrading
to Hall-Mammen PSU wild clustering under strictly-coarser survey PSUs) versus
DiD/TWFE's residual wild cluster bootstrap with test-inverted CIs, so one
value spelling for all of them would make a uniform name carry altered
meaning. They keep `n_bootstrap` as documented domain vocabulary, and section
7 now records the boundary; an `inference=` selector for any of them would
name its own method and is additive post-4.0 work.
(c) The `BaseEstimator` mixin and the R-equivalents docs table are genuine
refactor/docs work that cannot be ledger rows at all, so section 9 gains a
3.9-cut checklist alongside the existing 4.0 one. `M-008`'s notes now record
that its telemetry soak window *is* the 3.9→4.0 gap. (d) A completeness
sweep over surfaces the Phase 1 inventory never audited. That inventory was
derived from the estimator roster, so module-level public **functions** were
never checked against section 8 at all: `M-097` renames
`twowayfeweights(group=)` → `unit=` (section 3.3 **retains** that diagnostic
past 4.0, so no wrapper removal covers it), `M-098`..`M-112` rename the five
`_col` params on each of the three exported HAD pretest entry points
(`joint_pretrends_test`, `joint_homogeneity_test`,
`did_had_pretest_workflow`), `M-113` renames `trim_weights(weight_col=)` →
`weights=`, `M-114` is the dCDH results mirror (`.groups` → `.units`, whose
"groups" really are unit ids), and `M-115` adds the fourth `robust` site
(`LinearRegression`, which section 7 said dies "everywhere it exists" while
only three rows enumerated it). The same sweep found ~26 hits that are NOT
violations and are now documented as such in section 8 so they are not
re-litigated: the staggered family's `group`/`groups` name the ATT(g,t)
**cohort** in Callaway-Sant'Anna's notation (rule 8 protects it; renaming to
`unit` would be wrong), and `TripleDifference.fit[group]` is the
treated-group indicator rule 3 explicitly reserves the name for. Section 8
gains rule 10 binding the rules to public functions, not just classes, and
rule 11 putting every READER of a renamed name in the row's scope — the
`getattr(obj, "old_name", default)` pattern degrades silently to the default
after removal, so a consumer reports the wrong thing instead of crashing.
`M-043`/`M-095` accordingly carry their reporting and power/practitioner
consumers (and the `clean_control` serialized reporting key, retired at 4.0
by the section 5 canonical-names policy) in `code_refs`.
(e) `M-042`/`M-094`'s rename target moves from `treatment` to `takeup`:
fuzzy RD's observed-take-up column accepts non-binary dose values (matching
R's `fuzzy=`, with the estimand label degrading from complier-LATE to a bare
local Wald ratio), while section 8 rule 7 reserves `treatment` for a 0/1
indicator and never a treatment level - so the original target would have
made a uniform name carry altered meaning. `takeup` is the spelling the RD
docstring already uses; no shim had shipped, so the retarget is free. Rule 7
gains the corollary that a column which MAY be non-binary does not take the
`treatment` name even when it is conceptually take-up.
Ledger row count 77 → 99 (enforcement-test floor and id snapshot extended in
the same diff); rows gating a 3.9 bump 45 → 68. Docs + ledger + test-count
updates only - **no public API or numerical behavior change** (every new row
is `planned`; the shims themselves land in Phase 2c).
- **Internal tooling: plan-review engine → dual-reviewer skill (no library
change).** `/review-plan` + `/revise-plan` are retired for a
`.claude/skills/plan-review/` skill that runs the **dual** review engine
Expand Down
1 change: 1 addition & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
| Tighten the mypy suppressions that back the enforced-zero posture: burn down `prep_dgp`'s per-module `[index]` override (needs a None-vs-array restructure that preserves the seeded RNG stream), and evaluate re-enabling the globally disabled codes (`arg-type`, `return-value`, `var-annotated`, `assignment`) one at a time — `assignment` alone hid several real annotation drifts found during the 2026-07 triage. | `pyproject.toml` `[tool.mypy]`, `diff_diff/prep_dgp.py` | lint-CI | Mid | Low |
| MMM interop follow-up: Meridian `roi_calibration_period` mask builder - accept the MMM's time index + channel order and emit the boolean `(n_media_times, n_media_channels)` mask so `.to_code()` scopes the prior to the experiment window automatically (today the caller passes a mask expression / `full_model_window=True`). | `diff_diff/mmm.py` | mmm-interop | Quick | Low |
| MMM interop PR-B: calibration tutorial notebook (fit DiD/CS -> scope -> `to_pymc_marketing_lift_test` / `to_meridian_roi_prior`) + a `llms-practitioner.txt` Step 8 pointer to the exporters as the MMM hand-off. | `docs/tutorials/`, `diff_diff/guides/llms-practitioner.txt` | mmm-interop | Mid | Low |
| Section-8 naming-completeness guard test: sweep the public surface (module-level functions, class `__init__`/`fit` params, results dataclass fields) for contract-rename violations and fail unless each hit is either a `docs/v4-deprecations.yaml` row or an allowlist entry carrying a stated reason (seed the allowlist with the documented domain-vocabulary exemptions: the staggered family's ATT(g,t) `group`/`groups`, `TripleDifference.fit[group]`, wrapper params dying with `M-070..M-077`, and params inherited from `DifferenceInDifferences.__init__`). Today the rules are normative prose and the ledger holds only what an audit remembered — two successive manual sweeps each found surfaces the prior one missed (`M-094`/`M-095`, then the 19-row `M-097..M-115` function sweep). Should land BEFORE Phase 2c so the rename PR works from a mechanically-verified list. Two further checks belong in the same guard, each having been missed by hand across successive review rounds: (a) **phase-table agreement** — every row's `phase` must appear in the matching `docs/v4-design.md` section 9 table entry and vice versa (the section 9 checklist asserts this agreement but nothing verifies it, and three separate rounds caught a stale table); (b) **consumer coverage** (section 8 rule 11) — for each rename row, grep `diff_diff/` and `docs/methodology/` for the old name and require every hit to be either in the row's `code_refs` or allowlisted, since `getattr(obj, "old_name", default)` degrades silently rather than raising after removal. | `tests/`, `docs/v4-design.md` | gating-completeness amendment | Mid | Medium |
| Tracking-file contract guard test: reject NEW active deferred-work pointers at `TODO.md` (deferred rows live in `DEFERRED.md`; allowlist for historical/past-tense prose and actionable-row pointers) and assert rows cross-linking a `docs/v4-deprecations.yaml` `M-xxx` id don't restate ledger status. Origin: tracking-split local review R2. | `tests/`, `TODO.md`, `DEFERRED.md` | tracking-split | Quick | Low |
| Real-data CI canary for dataset-backed replication tests: `test_methodology_lwdid.py`'s Prop 99 / Walmart goldens skip (visibly) when loaders fall back to synthetic; add a lane or canary asserting `df.attrs["source"] == "lwdid_ssc_ancillary"` in CI so network regressions cannot silently de-gate the replication tests. Pairs with the loader-fallback repair row above. | `tests/test_methodology_lwdid.py`, `.github/workflows/` | LWDiD validation suite | Quick | Low |
| `worktree-rm` safety via a tested argv helper: the prose rewrite (ask-before-remove confirmation gate, detached-HEAD reachability/rescue, tip-identity force-delete guard) was reverted to the main version because editing the prose repeatedly reintroduced shell-injection (last: sourcing a state file built from a git-derived branch name). Restore those guards in a `worktree_rm.py` that takes the name via file ingress, invokes git through argv arrays, and has a metacharacter-branch/path injection regression test — the pattern that worked for `pr_prepare.py`/`premerge_scan.py`. | `.claude/commands/worktree-rm.md`, `.claude/scripts/` | skill-audit | Heavy | Medium |
Expand Down
Loading
Loading