Skip to content

Commit 7cd3cf9

Browse files
igerberclaude
andcommitted
feat: MMM calibration export (diff_diff.mmm) - PyMC-Marketing lift tests + Meridian ROI priors
New interop module converting DiD experiment results into Marketing Mix Model calibration inputs, with zero MMM-package dependency: - to_pymc_marketing_lift_test(): emits the lift-test DataFrame consumed by pymc_marketing MMM.add_lift_test_measurements (channel/dims/x/delta_x/ delta_y/sigma; explicit unit contract with aggregate= mean/sum scaling, n_units= geo-to-national scaling, on_wrong_sign= policy mirroring the upstream monotonicity check, x+delta_x>=0 post-test spend guard, reserved- column dims validation; prophetverse-compatible schema) - to_meridian_roi_prior(): emits Google Meridian lognormal ROI prior params (MeridianROIPrior with mu/sigma via Google's closed form - bit-identical parity vs meridian prior_distribution.lognormal_dist_from_mean_std at 1.7.0; spend-weighted multi-experiment pooling per the Meridian FAQ recommendation; se_widening transferability knob; channel-scoped .to_code() snippet - roi_m is per-channel so vector priors require the model channel order, scalar output needs explicit single_channel=True) - Duck-typed against the flat att/se surface every headline results class exposes; treated counts are never inferred from result metadata (n_treated is a row count on several classes); n_units/n_periods must be positive integers - 51 behavioral tests incl. units-not-rows pin, scipy lognormal round-trip, channel-scope regression, and cross-estimator contract (DiD/SDID/MPD/CS) - End-to-end smoke verified against real pymc-marketing 0.19.4 (frame accepted into the model graph; wrong-signed row raises NonMonotonicError) - Docs: REGISTRY.md interop section, api/mmm.rst, llms.txt + llms-full.txt, README one-liner, references.rst, doc-deps.yaml, CHANGELOG, TODO follow-ups Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014wAAYSVo1Yf3XMcxNMrqsj
1 parent 7aed8f1 commit 7cd3cf9

13 files changed

Lines changed: 1416 additions & 0 deletions

File tree

CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,25 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [Unreleased]
9+
10+
### Added
11+
- **MMM calibration export (`diff_diff.mmm`)** - interop bridge converting DiD
12+
experiment results into Marketing Mix Model calibration inputs, with no MMM
13+
package dependency. `to_pymc_marketing_lift_test()` emits the lift-test
14+
DataFrame consumed by PyMC-Marketing's `MMM.add_lift_test_measurements`
15+
(columns `channel`/dims/`x`/`delta_x`/`delta_y`/`sigma`; `aggregate=` per-period
16+
vs cumulative scaling, `n_units=` geo-to-national scaling, explicit
17+
`on_wrong_sign=` policy mirroring PyMC-Marketing's monotonicity check).
18+
`to_meridian_roi_prior()` emits Google Meridian lognormal ROI prior parameters
19+
(`MeridianROIPrior` with `mu`/`sigma` via Google's closed form, spend-weighted
20+
multi-experiment pooling, `se_widening=` transferability knob, and a
21+
channel-scoped `.to_code()` snippet helper - Meridian's `roi_m` prior is
22+
per-channel, so the snippet requires the experiment channel plus the model's
23+
channel order, or an explicit `single_channel=True`). Works with any result
24+
exposing scalar `att`/`se`;
25+
treated counts are never inferred from result metadata.
26+
827
## [3.8.0] - 2026-07-18
928

1029
### Added

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,7 @@ Full guide: `diff_diff.get_llm_guide("practitioner")`.
129129
- [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html) - Rambachan & Roth (2023) sensitivity analysis: robust CI under PT violations, breakdown values
130130
- [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html) - Roth (2022) minimum detectable violation and power curves
131131
- [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html) - analytical and simulation-based MDE, sample size, power curves for study design
132+
- [MMM Calibration Export](https://diff-diff.readthedocs.io/en/stable/api/mmm.html) - convert experiment results into MMM calibration inputs: PyMC-Marketing lift-test frames and Google Meridian lognormal ROI priors
132133
- Conley spatial HAC SE (`vcov_type="conley"`) on cross-sectional `LinearRegression` / `compute_robust_vcov` plus panel `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` (with `conley_lag_cutoff` for within-unit Bartlett temporal HAC) - Conley (1999) spatial-correlation-aware SEs with parity vs R `conleyreg` on cross-sectional + panel fixtures, optional combined spatial + cluster product kernel via explicit `cluster=`, auto-activating sparse k-d-tree fast path for `n > 5_000`
133134

134135
## Survey Support

TODO.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
4949
| Issue | Location | Origin | Effort | Priority |
5050
|-------|----------|--------|--------|----------|
5151
| ChangesInChanges/QDiD tutorial notebook (2x2 distributional walkthrough: QTE grid, interior range, uniform bands, CiC-vs-QDiD comparison) - deferred from the implementation PR as a documented decision. | `docs/tutorials/` | #682 | Mid | Low |
52+
| MMM interop follow-up: Meridian `roi_calibration_period` boolean-mask builder (align the ROI prior's definition window to the experiment dates; needs the user's Meridian time index + channel ordering as inputs) - deferred from v1 as a documented decision. | `diff_diff/mmm.py` | mmm-interop | Quick | Low |
53+
| MMM interop follow-up: `practitioner_next_steps` handler step + `llms-practitioner.txt` Step 8 pointer recommending the MMM exporters as the reporting hand-off (tutorial PR-B scope). | `diff_diff/practitioner.py`, `diff_diff/guides/llms-practitioner.txt` | mmm-interop | Quick | Low |
5254
| Tighten the mypy suppressions that back the enforced-zero posture: burn down `prep_dgp`'s per-module `[index]` override (needs a None-vs-array restructure that preserves the seeded RNG stream), and evaluate re-enabling the globally disabled codes (`arg-type`, `return-value`, `var-annotated`, `assignment`) one at a time — `assignment` alone hid several real annotation drifts found during the 2026-07 triage. | `pyproject.toml` `[tool.mypy]`, `diff_diff/prep_dgp.py` | lint-CI | Mid | Low |
5355
| Align the four legacy dataset loaders (`load_card_krueger`, `load_castle_doctrine`, `load_divorce_laws`, `load_mpdta`) with the loud-fallback pattern of `load_prop99`/`load_walmart`: `UserWarning` + `df.attrs["source"]` marker on synthetic fallback (currently silent), plus optional checksum pinning for the CSV downloads. **Upgraded to a live defect 2026-07-13: the `causaldata/causal_datasets` GitHub repo backing castle/card_krueger/divorce is dead (404), so those loaders silently serve synthetic data everywhere - needs loud fallback + replacement sources.** | `diff_diff/datasets.py` | LWDiD precursor | Quick | Medium |
5456
| Real-data CI canary for dataset-backed replication tests: `test_methodology_lwdid.py`'s Prop 99 / Walmart goldens skip (visibly) when loaders fall back to synthetic; add a lane or canary asserting `df.attrs["source"] == "lwdid_ssc_ancillary"` in CI so network regressions cannot silently de-gate the replication tests. Pairs with the loader-fallback repair row above. | `tests/test_methodology_lwdid.py`, `.github/workflows/` | LWDiD validation suite | Quick | Low |
@@ -118,6 +120,9 @@ Doable in principle, but no current caller and/or explicitly out of paper scope.
118120
| ChangesInChanges analytical SEs (Athey-Imbens Theorems 5.1-5.3 influence functions, panel 5.5-5.7, Appendix B covariances; needs the footnote-31 boundary density estimator - note the review's suspected half-range/midpoint typo). Bootstrap is the shipped inference. | `diff_diff/changes_in_changes.py` | #682 | Low |
119121
| Staggered/multi-period distributional DiD (Athey-Imbens Section 6 / Ciaccio arXiv:2408.01208v2; `ecic` is the staggered event-study CiC lineage - a distinct method from Ciaccio's copula approach, do not conflate). Reviewed: `docs/methodology/papers/ciaccio-2024-review.md`; ROADMAP row is reviewed-deferred pending demand. | `diff_diff/changes_in_changes.py` | #682 | Low |
120122
| ChangesInChanges treatment-on-the-controls (Athey-Imbens Theorem 3.2: group-label exchange + negation; no qte equivalent to anchor parity). | `diff_diff/changes_in_changes.py` | #682 | Low |
123+
| MMM interop: event-study temporal lift export (time-resolved effect path -> per-period calibration rows) - no consumer exists yet; PyMC-Marketing's temporal/adstock lift calibration is an open upstream feature request (pymc-marketing #2433) and the current `add_lift_test_measurements` API consumes only scalar lifts. Revisit when upstream ships. | `diff_diff/mmm.py` | mmm-interop | Low |
124+
| MMM interop: Robyn `calibration_input` emitter - the only functional consumer is R Robyn (the official `robynpy` Python beta ships calibration data classes but its public API hardcodes `calibration_input=None` and feeds a constant-zero calibration objective; verified 2026-07-18). Revisit if Python Robyn revives. | `diff_diff/mmm.py` | mmm-interop | Low |
125+
| MMM interop: Meridian GeoX emitter - GeoX ("converts experiment results into priors", SDiD on roadmap) is announced but unshipped (waitlist-only, no repo/spec as of 2026-07-18). Revisit at GeoX GA. | `diff_diff/mmm.py` | mmm-interop | Low |
121126
| Rust-backend CR2 Bell-McCaffrey port (`return_dof` in the Rust vcov dispatch + CR2 algebra) — **premise re-scoped 2026-07-09**: the scores-based DOF + low-rank factored `A_g` changes made the NumPy CR2-BM path BLAS-bound (`O(n_g k²)` per cluster; 4.1s→38ms at n=100k/k=40), so a Rust port buys ~nothing and adds a parity surface. Revisit only if profiling shows CR2-BM hot again. | `rust/src/linalg.rs` || Low |
122127
| Clustered-CR1 inference df **default flip to `"cluster"` (G−1) at v4** — the opt-in `df_convention=` knob landed 2026-07 (DiD/TWFE/MPD + LinearRegression; REGISTRY §TwoWayFixedEffects deviation note); the remaining work is the major-version default change (moves every clustered p-value/CI) + migration note + flipping `TestDfConvention`/`test_moderate_t_pins_residual_df_convention` expectations. Also evaluate extending the knob to standalone estimators with CR1-t inference at that time. | `diff_diff/linalg.py::LinearRegression`, `diff_diff/estimators.py`, `diff_diff/twfe.py` || Medium |
123128
| CallawaySantAnna **unbalanced-panel R parity — LANDED** via `allow_unbalanced_panel=True` (matches R `did::att_gt(allow_unbalanced_panel=TRUE)` / `DRDID::reg_did_rc`: ATT bit-exact on cells AND dynamic aggregation via fixed unit-cohort-mass `pg` + a per-unit WIF; SE up to the documented CR1 `sqrt(G/(G-1))` factor). The earlier "weighting" framing was a mis-diagnosis — on unbalanced panels the dominant divergence from R is the *estimator* (within-cell differencing vs RC-on-pooled-obs), not only the weighting; both are resolved by the flag. The DEFAULT path keeps within-cell differencing as a documented design choice and now emits a `UserWarning` on unbalanced input (no-silent-failures). **Remaining deferred:** `survey_design=` × `allow_unbalanced_panel=` (per-obs vs per-unit weight resolution — currently fail-closed `NotImplementedError`); and covariate / ipw / dr × the flag R-parity verification (the RC path supports them; the committed golden covers `reg` no-cov). | `staggered.py`, `staggered_aggregation.py` | SE-audit D3 | Low |

diff_diff/__init__.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,11 @@
154154
)
155155
from diff_diff.lpdid import LPDiD
156156
from diff_diff.lpdid_results import LPDiDResults
157+
from diff_diff.mmm import (
158+
MeridianROIPrior,
159+
to_meridian_roi_prior,
160+
to_pymc_marketing_lift_test,
161+
)
157162
from diff_diff.power import (
158163
PowerAnalysis,
159164
PowerResults,
@@ -559,6 +564,10 @@
559564
"Alert",
560565
"OutcomeShape",
561566
"TreatmentDoseShape",
567+
# MMM calibration export (interop)
568+
"to_pymc_marketing_lift_test",
569+
"to_meridian_roi_prior",
570+
"MeridianROIPrior",
562571
# LLM guide accessor
563572
"get_llm_guide",
564573
]

diff_diff/guides/llms-full.txt

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2667,3 +2667,86 @@ cross-period aggregations are enumerated in
26672667
current release; see that document for phrasing rules, the
26682668
no-traffic-light decision, unit-translation policy, and schema
26692669
stability policy.
2670+
2671+
## MMM Calibration Export
2672+
2673+
Interop exporters converting experiment results into Marketing Mix Model
2674+
(MMM) calibration inputs. Pure numpy/pandas — no MMM package is imported;
2675+
outputs are a plain DataFrame / frozen dataclass. Any result exposing
2676+
scalar `att`/`se` qualifies (all headline results classes, natively or via
2677+
the flat property aliases). Experiment context an estimation result cannot
2678+
know — channel, spend levels, treated-unit count, period count — is always
2679+
explicit user input. Treated counts are NEVER inferred from result
2680+
metadata: `n_treated` on several result classes counts treated rows
2681+
(unit-periods), not units.
2682+
2683+
```python
2684+
from diff_diff import (
2685+
SyntheticDiD,
2686+
to_pymc_marketing_lift_test,
2687+
to_meridian_roi_prior,
2688+
)
2689+
2690+
result = SyntheticDiD().fit(panel, outcome="revenue", treatment="treated",
2691+
unit="geo", time="week")
2692+
2693+
# PyMC-Marketing (also prophetverse): lift-test DataFrame for
2694+
# MMM.add_lift_test_measurements — columns [channel, *dims, x, delta_x,
2695+
# delta_y, sigma], all in original data units.
2696+
df_lift = to_pymc_marketing_lift_test(
2697+
result,
2698+
channel="tv", # must match the MMM's channel_columns
2699+
x=50_000.0, # baseline per-period spend
2700+
delta_x=20_000.0, # spend change during the test (nonzero;
2701+
# negative for go-dark tests)
2702+
dims={"geo": "US-CA"}, # optional model-dim columns (geo-level MMM)
2703+
aggregate="mean", # "mean" per-period lift | "sum" cumulative
2704+
# (requires n_periods AND cumulative x/delta_x)
2705+
on_wrong_sign="raise", # "raise" | "drop" | "keep" — PyMC-Marketing
2706+
# hard-rejects sign(delta_y) != sign(delta_x)
2707+
)
2708+
2709+
# Google Meridian: lognormal ROI prior parameters for roi_m calibration.
2710+
prior = to_meridian_roi_prior(
2711+
result, # or a sequence — pooled with spend weights
2712+
spend=200_000.0, # incremental spend during the experiment
2713+
n_units=30, # treated geos (explicit, from your design)
2714+
n_periods=8, # post-treatment periods the ATT spans
2715+
se_widening=1.5, # >1 encodes experiment-to-MMM skepticism
2716+
)
2717+
prior.roi_mean, prior.roi_sd # pooled ROI moments
2718+
prior.mu, prior.sigma # LogNormal params (Google's closed form,
2719+
# matches lognormal_dist_from_mean_std)
2720+
prior.to_dict() # JSON-ready
2721+
print(prior.to_code( # PriorDistribution snippet (pinned to
2722+
channel="tv", # Meridian 1.7.0). roi_m is PER-CHANNEL and a
2723+
media_channels=["search", "tv"], # scalar broadcasts to every channel,
2724+
)) # so channel scope is required: vector prior in
2725+
# model channel order (others keep Meridian's
2726+
# default LogNormal(0.2, 0.9)), or
2727+
# single_channel=True for 1-channel models.
2728+
```
2729+
2730+
Key semantics:
2731+
2732+
- Lift frame `delta_y = att * (n_units or 1)` under `aggregate="mean"`
2733+
(leave `n_units=None` for geo-level MMM rows; pass the treated-geo count
2734+
when calibrating a national MMM from a geo experiment), additionally
2735+
`* n_periods` under `aggregate="sum"`; `sigma` scales identically.
2736+
UNIT CONTRACT: x/delta_x/delta_y/sigma must share one aggregation unit
2737+
(one MMM row's unit) - under "sum" the x/delta_x you pass must be
2738+
cumulative spends over the experiment window, valid only when one MMM
2739+
row spans that window.
2740+
- Meridian per-experiment `roi = att * n_units * n_periods / spend` (or
2741+
an explicit `incremental_outcome=` override); multi-experiment pooling
2742+
is spend-weighted with `roi_sd = sqrt(sum((w_i * sd_i)^2))` (independence
2743+
assumed — widen via `se_widening` for correlated experiments).
2744+
- Guards: non-finite `att` / non-positive `se` raise (bootstrap NaN
2745+
contract upstream); wrong-signed lift rows raise by default with
2746+
`NonMonotonicError` guidance; `x + delta_x < 0` raises (post-test spend
2747+
cannot be negative); `n_units`/`n_periods` must be positive integers;
2748+
a non-positive pooled ROI mean raises
2749+
(lognormal positivity) pointing to pooling / wider priors / Meridian's
2750+
contribution parameterizations; `on_wrong_sign="drop"` raises rather
2751+
than emit an empty frame; array-valued `att` (event-study results) is
2752+
rejected — aggregate to a headline ATT first.

diff_diff/guides/llms.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@ Full practitioner guide: call `diff_diff.get_llm_guide("practitioner")`
8282
- [Honest DiD](https://diff-diff.readthedocs.io/en/stable/api/honest_did.html): Rambachan & Roth (2023) sensitivity analysis — robust CI under parallel trends violations, breakdown values
8383
- [Pre-Trends Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/pretrends.html): Roth (2022) Section II.A-B no-individually-significant (NIS) box-probability pretest power + minimum detectable violation; `pretest_form='nis'` (default) implements the paper's primary form, `pretest_form='wald'` retained as paper-supported alternative (Propositions 1+3+4 all apply); linear-violation MDV in Roth's γ units when relative-time labels are threaded through `fit()`; full Σ_22 routing on non-bootstrap CallawaySantAnna and SunAbraham adapters
8484
- [Power Analysis](https://diff-diff.readthedocs.io/en/stable/api/power.html): Analytical and simulation-based power analysis — MDE, sample size, power curves for study design
85+
- [MMM Calibration Export](https://diff-diff.readthedocs.io/en/stable/api/mmm.html): Convert experiment results into Marketing Mix Model calibration inputs — `to_pymc_marketing_lift_test()` emits the PyMC-Marketing/prophetverse lift-test DataFrame (`channel`/dims/`x`/`delta_x`/`delta_y`/`sigma` for `MMM.add_lift_test_measurements`), `to_meridian_roi_prior()` emits Google Meridian lognormal ROI prior parameters (spend-weighted multi-experiment pooling, `se_widening` transferability knob, channel-scoped `.to_code()` snippet - `roi_m` is per-channel, so it requires `channel=`+`media_channels=` or `single_channel=True`). Works with any result exposing scalar `att`/`se`; experiment context (channel, spend, `n_units`, `n_periods`) is always explicit — treated counts are never inferred from result metadata
8586
- Conley spatial HAC SE (`vcov_type="conley"`) on cross-sectional `LinearRegression` / `compute_robust_vcov` PLUS panel `DifferenceInDifferences` / `MultiPeriodDiD` / `TwoWayFixedEffects` (with `conley_lag_cutoff=<int>` for within-unit Bartlett temporal HAC) — Conley (1999) spatial-correlation-aware SEs with haversine/euclidean/callable distance metric and Bartlett/uniform spatial kernel; panel path uses the R `conleyreg`-form block-decomposed sandwich (within-period spatial + within-unit Bartlett serial, same-time excluded); parity vs R `conleyreg` (Düsterhöft 2021) on cross-sectional AND panel `lag_cutoff > 0` fixtures. Combining with explicit `cluster=<col>` applies the combined spatial + cluster product kernel `K_total[i,j] = K_space · 1{c_i = c_j}` (cluster must be constant within each unit across periods on the panel path; validator-enforced). DiD takes `unit=<col>` as a fit-time kwarg when `vcov_type="conley"` (not on `__init__`). Sparse k-d-tree fast path auto-activates for `n > 5_000` with bartlett kernel + haversine/euclidean metric
8687

8788
## Tutorials

0 commit comments

Comments
 (0)