From e54102242594e3218423ebb184c587499bd91f86 Mon Sep 17 00:00:00 2001 From: igerber Date: Sun, 12 Jul 2026 10:25:38 -0400 Subject: [PATCH 1/2] docs: add CiC/QDiD paper reviews (Athey-Imbens 2006, Callaway-Li-Oka 2018, Melly-Santangelo 2015, Ciaccio 2024) PR-A of the Changes-in-Changes / distributional DiD initiative: four version-pinned methodology reviews scoping the ChangesInChanges + QDiD v1 estimators (2x2, continuous outcomes, bootstrap inference, panel + repeated cross-section modes, R qte parity) and documenting the reviewed-but-deferred extensions (covariates, discrete-outcome bounds, analytical SEs, staggered adoption). - athey-imbens-2006-review.md: core CiC + QDiD source (published Econometrica typeset); exact inverse-CDF conventions, influence- function inference, discrete bounds, Section 6 multi-group machinery - callaway-li-oka-2018-review.md: two-period panel QTT under copula invariance (arXiv v1); exchangeable-bootstrap validity machinery - melly-santangelo-2015-review.md: CiC with covariates via quantile regression (Oct 2015 working paper); QR pipeline blueprint - ciaccio-2024-review.md: staggered distributional DiD (arXiv v2); un-parks the ROADMAP row to reviewed-deferred status Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011hismLBFcUbUzvDRq8ruWb --- .../papers/athey-imbens-2006-review.md | 413 ++++++++++++ .../papers/callaway-li-oka-2018-review.md | 417 ++++++++++++ .../methodology/papers/ciaccio-2024-review.md | 619 ++++++++++++++++++ .../papers/melly-santangelo-2015-review.md | 522 +++++++++++++++ 4 files changed, 1971 insertions(+) create mode 100644 docs/methodology/papers/athey-imbens-2006-review.md create mode 100644 docs/methodology/papers/callaway-li-oka-2018-review.md create mode 100644 docs/methodology/papers/ciaccio-2024-review.md create mode 100644 docs/methodology/papers/melly-santangelo-2015-review.md diff --git a/docs/methodology/papers/athey-imbens-2006-review.md b/docs/methodology/papers/athey-imbens-2006-review.md new file mode 100644 index 000000000..688c7b0d9 --- /dev/null +++ b/docs/methodology/papers/athey-imbens-2006-review.md @@ -0,0 +1,413 @@ +# Paper Review: Identification and Inference in Nonlinear Difference-in-Differences Models + +**Authors:** Susan Athey and Guido W. Imbens +**Citation:** Athey, S., & Imbens, G. W. (2006). Identification and Inference in Nonlinear Difference-in-Differences Models. *Econometrica*, 74(2), 431-497. https://doi.org/10.1111/j.1468-0262.2006.00668.x +**PDF reviewed:** **Published Econometrica version of record** (Vol. 74, No. 2, March 2006, journal pp. 431-497; 67 PDF pages, PDF page N = journal page 430+N). Per the project's PDFs-never-committed convention the local PDF is kept outside the repository (gitignored `papers/athey-imbens-2006-econometrica.pdf`); the journal/DOI version (https://doi.org/10.1111/j.1468-0262.2006.00668.x) is the authoritative source. All equation, theorem, lemma, and assumption numbers below are the published version's - the NBER Technical Working Paper t0280 (2002) draft renumbers these; do not reconcile against it. +**Review date:** 2026-07-12 + +--- + +## Methodology Registry Entry + +*Formatted to match docs/methodology/REGISTRY.md structure. Heading levels and labels align with existing entries - copy the `## ChangesInChanges (CiC)` section into the appropriate category in the registry.* + +## ChangesInChanges (CiC) + +**Primary source:** Athey, S., & Imbens, G. W. (2006). Identification and Inference in Nonlinear Difference-in-Differences Models. *Econometrica*, 74(2), 431-497. https://doi.org/10.1111/j.1468-0262.2006.00668.x +- Empirical application and several proofs (Theorems 5.3, 5.5, 5.7; Lemmas A.1, A.5, A.6, A.10, A.11) are in the Econometrica Supplementary Material (Athey and Imbens (2006), Vol. 74, http://econometricsociety.org/ecta/supmat/4035extensions.pdf), not the journal pages. +- The working-paper version "AI" (Athey and Imbens (2002)) is cited for the discrete-outcome QDiD case, the joint chi-squared quantile-equality test, and the asymptotic variance of the covariance-adjusted (residual-based) CIC estimator. + +**Model (Sections 2 and 3.1, journal pp. 435-439):** + +Data: two groups x two periods. Individual i has group `G_i ∈ {0,1}` (1 = treatment group), period `T_i ∈ {0,1}` (1 = post), treatment indicator `I_i = G_i * T_i`, and realized outcome `Y_i = Y_i^N * (1 - I_i) + I_i * Y_i^I` (Rubin potential outcomes, p. 435). Untreated outcomes are generated by a nonlinear "production function" of a scalar unobservable: + +``` +(3) Y^N = h(U, T) +``` + +Notation: `Y_gt ~d Y | G=g, T=t` with CDF `F_Y,gt`; `U_g ~d U | G=g`; supports written `Y_gt`, `U_g` (p. 438). + +- **Assumption 3.1 (Model):** `Y^N = h(U, T)` - outcomes do not depend directly on the group indicator; all relevant unobservables enter through a single index U (p. 439). +- **Assumption 3.2 (Strict Monotonicity):** `h(u, t)` is strictly increasing in u for t ∈ {0, 1} (p. 439). +- **Assumption 3.3 (Time Invariance Within Groups):** `U ⊥ T | G` - "at the heart of both the DID and CIC approaches" (p. 439). +- **Assumption 3.4 (Support):** `U_1 ⊆ U_0`; implies `Y_10 ⊆ Y_00` and `Y^N_11 ⊆ Y_01` (pp. 439-440). Invoked selectively; relaxed in Corollary 3.1. + +Assumptions 3.1-3.3 are the CIC model. With continuous outcomes the model imposes NO testable restrictions on the data (p. 439) and its assumptions are invariant to monotone transformations of the outcome (levels vs logs, p. 437). Treated outcomes: `Y^I = h^I(U, T)` with `h^I(u, t)` increasing in u (p. 438); no functional-form restriction links `h^I(u,1) - h(u,1)`, so treatment effects vary arbitrarily across individuals and across groups (through `F_U,g`). + +The standard linear DiD model `(1) Y^N = alpha + beta*T + gamma*G + eps`, `eps ⊥ (G,T)` is nested: it equals model (3) plus `(4)` additivity `U = alpha + gamma*G + eps`, `(5)` single-index `h(u,t) = phi(u + delta*t)` with phi strictly increasing, and `(6)` phi = identity (pp. 435-437). Keeping (4)-(5) but relaxing (6) gives the intermediate transformation model `(7) Y^N = phi(alpha + gamma*G + delta*T + eps)` (footnote 13: still rules out, e.g., simultaneous mean and variance shifts). Nesting caveat (footnote 10, p. 436): CIC nests DiD with FULL independence `eps ⊥ (G,T)`, not the mean-independence-only DiD model. + +The paper also defines the **reverse model CIC-r** (p. 444): roles of time and group exchanged (production function fixed over time, population composition changes over time). With continuous outcomes neither CIC nor CIC-r is testable, so data cannot distinguish them although they yield different estimates - the role of each dimension must be justified substantively. + +**Key implementation requirements:** + +*Assumption checks / warnings:* +- Exactly two groups and two periods in v1, with all four (g,t) cells non-empty (Assumption 5.1(ii): `alpha_gt = Pr(T=t, G=g) > 0`, p. 461). +- Support condition `Y_10 ⊆ Y_00` (Assumption 5.1(iv), the observable implication of Assumption 3.4): warn when treated pre-period outcomes fall outside the control pre-period support. Without it the counterfactual CDF is identified only on `Y_01` and quantile effects only on `[q_lower, q_upper]` per eq. (17) (Corollary 3.1, p. 443). Standard DiD "identifies" the ATT beyond the control support only through extrapolation (p. 443). +- Continuous-outcome regularity (Assumption 5.1(iii), p. 461): the four `Y_gt` continuous with continuously differentiable densities bounded above and bounded away from zero on compact support `[y_lower_gt, y_upper_gt]`. Heavy ties / discrete outcomes violate this: warn and point to the (deferred) discrete-outcome bounds of Section 4. Applying the continuous machinery to discrete data silently delivers one endpoint of the bounds, not the point estimate (see the discrete subsection below). +- Quantile effects only for q strictly inside `(q_lower, q_upper)` (Theorem 5.3, p. 465): NaN or error outside, never extrapolate. +- Additive random group-time effects violate `U ⊥ T | G` and BIAS CiC (unlike linear DiD, where they only complicate inference - Bertrand, Duflo, and Mullainathan (2004), Donald and Lang (2001) fixes rely on estimator linearity). "There appears to be no simple adjustment to remove this bias. Fortunately, the presence of such effects is testable using Theorem 6.4" when >2 groups/periods are available (p. 476). Document this in the estimator's docstring warning. +- Panel mode does NOT require rank invariance over time: only `U_i0 | G_i ~d U_i1 | G_i` (marginals equal within group), so fixed-effect-plus-noise structures `U_it = eps_i + nu_it` are allowed (Section 3.4, pp. 447-448; footnote 17, p. 444). + +*Inverse-CDF conventions (exact - implementations diverge from the paper precisely here):* + +1. Population inverse, used throughout the continuous theory (eq. 8, p. 440) - inf-based, WEAK inequality `>=`, over the support of Y (compact support assumed): + +``` +(8) F_Y^{-1}(q) = inf{ y ∈ Y : F_Y(y) >= q }, q ∈ [0,1] +``` + +Implies inverse CDFs are continuous from the left; `F_Y(F_Y^{-1}(q)) >= q`, with equality everywhere for continuous Y and at discontinuity points of `F_Y^{-1}(q)` for discrete Y; `F_Y^{-1}(F_Y(y)) <= y`, with equality for continuous or discrete Y but not necessarily for mixed Y (p. 440). + +2. Empirical CDF and empirical inverse (eqs. 34-35, p. 462) - same inf/`>=` convention; at q = 0 it returns the lower support endpoint, not -infinity: + +``` +(34) F_hat_Y,gt(y) = (1/N_gt) * sum_{i=1}^{N_gt} 1{Y_gt,i <= y} +(35) F_hat_Y,gt^{-1}(q) = inf{ y ∈ Y_gt : F_hat_Y,gt(y) >= q }, so F_Y,gt^{-1}(0) = y_lower_gt +``` + +3. Order-statistic form and sandwich identities (eqs. A.1-A.2, p. 478) - the asymptotic theory is proved for exactly this convention; interpolating quantile types (e.g., NumPy's default `linear`) do NOT satisfy these identities: + +``` +(A.1) F_hat_Y^{-1}(q) = Y_(ceil(N*q)) = inf{ y ∈ Y : F_hat_Y(y) >= q }, F_hat_Y^{-1}(0) = y_lower +(A.2) q <= F_hat_Y(F_hat_Y^{-1}(q)) < q + 1/N, + with equality F_hat_Y(F_hat_Y^{-1}(q)) = q iff q = j/N for integer j ∈ {0,...,N}; + y - max_{i=1..N}(Y_(i) - Y_(i-1)) < F_hat_Y^{-1}(F_hat_Y(y)) <= y, Y_(0) = y_lower, + with equality F_hat_Y^{-1}(F_hat_Y(y)) = y exactly at all sample values. +``` + +4. Lemma A.1 (p. 478) - Galois-connection identities for any nondecreasing right-continuous `g: Y -> U` with `g^{-1}(u) = inf{y ∈ Y : g(y) >= u}` (holds exactly for the EDF / inverse-EDF pair above, which is why the plug-in works): + (i) `g(g^{-1}(u)) >= u`; (ii) `g^{-1}(g(y)) <= y`; (iii) `g(g^{-1}(g(y))) = g(y)`; (iv) `g^{-1}(g(g^{-1}(u))) = g^{-1}(u)`; (v) `{(u,y) : u <= g(y)} = {(u,y) : g^{-1}(u) <= y}`. + +5. Discrete-case ALTERNATIVE inverse (eq. 24, p. 453) - sup-based, WEAK inequality `<=`, over the support augmented with -infinity, convention `F_Y(-inf) = 0`: + +``` +(24) F_Y^{(-1)}(q) = sup{ y ∈ Y ∪ {-inf} : F_Y(y) <= q } +``` + +With `Q = { q ∈ [0,1] : exists y ∈ Y with F_Y(y) = q }` (p. 453): for q ∈ Q the two inverses agree, `F_Y^{(-1)}(q) = F_Y^{-1}(q)` and `F_Y^{-1}(F_Y(y)) = F_Y^{(-1)}(F_Y(y)) = y`; for q ∉ Q, `F_Y^{(-1)}(q) < F_Y^{-1}(q)` and `F_Y(F_Y^{(-1)}(q)) < q`; for all q, `F_Y(F_Y^{(-1)}(q)) <= q <= F_Y(F_Y^{-1}(q))`. + +6. Strict-inequality CDF for the discrete bounds (eq. 41, p. 467): + +``` +(41) F_lower_Y,00(y) = Pr(Y_00 < y) [STRICT inequality, vs F_Y,00(y) = Pr(Y_00 <= y)] +``` + +*Estimator equation (Theorem 3.1, Equation 9; transformation Equation 15; ATT Equation 16; plug-in Equation 36 - as implemented):* + +Counterfactual distribution of the treated group's untreated second-period outcome (Theorem 3.1, p. 440; under Assumptions 3.1-3.4, U continuous or discrete): + +``` +(9) F_{Y^N,11}(y) = F_Y,10( F_Y,00^{-1}( F_Y,01(y) ) ) +``` + +CIC transformation (eq. 15, p. 441) - the second-period outcome of an individual whose unobservable satisfies `h(u,0) = y`; the distribution of `Y_11^N` equals that of `k^CIC(Y_10)`: + +``` +(15) k^CIC(y) = F_Y,01^{-1}( F_Y,00(y) ) +``` + +Graphical construction (Figure 1, p. 442): for value y at quantile q' = `F_Y,00(y)` in the control pre distribution, `Delta^CIC = F_Y,01^{-1}(q') - y` and `k^CIC(y) = y + Delta^CIC`. Contrast the standard DiD transformation `k^DID(y) = y + E[Y_01] - E[Y_00]` (p. 443). + +ATT estimand (eq. 16, p. 441) and sample plug-in (eq. 36, p. 462): + +``` +(16) tau^CIC = E[Y_11^I] - E[ F_Y,01^{-1}( F_Y,00(Y_10) ) ] + +(36) tau_hat^CIC = (1/N_11) * sum_{i=1}^{N_11} Y_11,i + - (1/N_10) * sum_{i=1}^{N_10} F_hat_Y,01^{-1}( F_hat_Y,00(Y_10,i) ) +``` + +where: +- `F_Y,gt`, `F_hat_Y,gt` = population / empirical CDF of the (g,t) cell, conventions above +- `N_gt` = cell sample size; `Y_10,i` = treated-group pre-period observations +- the empirical CDFs are deliberately unsmoothed - the estimator "does not require the choice of smoothing parameters" (p. 451) + +Identification proof chain (pp. 440-441, from the central relation `(10) F_{Y^N,gt}(y) = F_U,g(h^{-1}(y;t))` applied to all four cells): `(11) h(u,0) = F_Y,00^{-1}(F_U,0(u))` for u ∈ U_0; `(12) F_U,0^{-1}(F_Y,01(y)) = h^{-1}(y;1)` for y ∈ Y_01; `(13) h(h^{-1}(y;1),0) = F_Y,00^{-1}(F_Y,01(y))`; `(14) F_Y,10(h(u,0)) = F_U,1(u)`. Footnote 16 (p. 440): the support restriction matters in (11) - for u ∉ U_0 it is not necessarily true that `F_Y,00^{-1}(F_Y,00(h(u,0))) = h(u,0)`. + +*Quantile treatment effects (Equations 17-18, Corollary 3.1, p. 443; plug-in p. 464):* + +``` +(17) q_lower = min_{y ∈ Y_00} F_Y,10(y), q_upper = max_{y ∈ Y_00} F_Y,10(y) + +(18) tau_q^CIC = F_Y,11^{-1}(q) - F_Y,01^{-1}( F_Y,00( F_Y,10^{-1}(q) ) ) + + tau_hat_q^CIC = F_hat_Y,11^{-1}(q) - F_hat_Y,01^{-1}( F_hat_Y,00( F_hat_Y,10^{-1}(q) ) ) +``` + +Valid for q ∈ [q_lower, q_upper] even without Assumption 3.4 (Corollary 3.1 requires `Y_00 = [y_lower_00, y_upper_00]` with no holes); asymptotics (Theorem 5.3) only on the open interior `(q_lower, q_upper)`. For bounded g(.), Manski (1990, 1995)-style bounds on `E[g(Y_11^I) - g(Y_11^N)]` are available outside the point-identified region. + +*Treatment effect on the controls (Theorem 3.2, Equations 19-21, pp. 444-446):* + +With the added assumption `Y^I = h^I(U, T)`, `h^I(u,t)` strictly increasing in u (U continuous or discrete): + +``` +(19) Y_01^I ~d h^I( h^{-1}(Y_00; 0), 1 ) +(20) F_{Y^I,11}^{-1}( F_Y,10(y) ) = h^I( h^{-1}(y; 0), 1 ) for y ∈ Y_10 +(21) F_{Y^I,01}(y) = F_Y,00( F_Y,10^{-1}( F_{Y^I,11}(y) ) ) for all y ∈ Y^I_11 +``` + +Equation (20) is k^CIC with the roles of groups 0 and 1 reversed; implement by exchanging group labels and calculating the negative of the treatment effect for group 0 (pp. 445-446). Identified everywhere if `U_0 ⊆ U_1`; the CIC asymptotic theory applies unchanged (footnote 18, p. 445: first-period treated outcomes `Y_00^I, Y_10^I` are NOT identified without much stronger assumptions such as `h^I(u,0) = h^I(u,1)`). + +*Standard errors (Section 5):* +- Default (paper): analytical, influence-function based (Theorem 5.1, p. 462). Under Assumption 5.1 (iid sampling within cells; positive cell shares; densities as above; `Y_10 ⊆ Y_00`): + +``` +(i) tau_hat^CIC - tau^CIC = O_p(N^{-1/2}) +(ii) sqrt(N)*(tau_hat^CIC - tau^CIC) -->d N( 0, V^p/alpha_00 + V^q/alpha_01 + V^r/alpha_10 + V^s/alpha_11 ) +``` + + with influence functions (eqs. 37-40, p. 462; `V^p = E[p(Y_00)^2]` etc., `alpha_gt = Pr(T=t, G=g)`): + +``` +(37) P(y,z) = ( 1 / f_Y,01( F_Y,01^{-1}(F_Y,00(z)) ) ) * ( 1{y <= z} - F_Y,00(z) ), p(y) = E[P(y, Y_10)] +(38) Q(y,z) = -( 1 / f_Y,01( F_Y,01^{-1}(F_Y,00(z)) ) ) + * ( 1{ F_Y,01(y) <= F_Y,00(z) } - F_Y,00(z) ), q(y) = E[Q(y, Y_10)] +(39) r(y) = F_Y,01^{-1}(F_Y,00(y)) - E[ F_Y,01^{-1}(F_Y,00(Y_10)) ] +(40) s(y) = y - E[Y_11] +``` + + p captures estimation error from F_hat_Y,00, q from F_hat_Y,01^{-1}, r sampling variation of the transformed Y_10 draws, s the Y_11 mean. Asymptotically linear representation (p. 463): `tau_hat = tau + (1/N_00) sum p(Y_00,i) + (1/N_01) sum q(Y_01,i) + (1/N_10) sum r(Y_10,i) + (1/N_11) sum s(Y_11,i) + o_p(N^{-1/2})`. +- Variance estimation (Theorem 5.2, pp. 463-464): plug in empirical CDFs/inverses and ANY uniformly consistent nonparametric density estimator (uniform consistency must include the boundary, footnote 31); `p_hat(y) = (1/N_10) sum_j P_hat(y, Y_10,j)`, `q_hat(y) = (1/N_10) sum_j Q_hat(y, Y_10,j)`, then `V_hat^p = (1/N_00) sum_i p_hat(Y_00,i)^2` (and analogously q, r, s), `alpha_hat_gt = N_gt/N`; all converge in probability to their targets. Footnote 31's concrete density estimator: one-sided difference quotient of the EDF with bandwidth `N^{-1/3}`, forward difference on the lower half of the support and backward on the upper half (split at the support midpoint `Y_tilde_gt = (y_upper_gt - y_lower_gt)/2`), guaranteeing boundary-uniform consistency: + +``` +f_hat_Y,gt(y) = ( F_hat_Y,gt(y + N^{-1/3}) - F_hat_Y,gt(y) ) / N^{-1/3} if y <= Y_tilde_gt +f_hat_Y,gt(y) = ( F_hat_Y,gt(y) - F_hat_Y,gt(y - N^{-1/3}) ) / N^{-1/3} if y > Y_tilde_gt +``` + +- Quantile SEs (Theorem 5.3, pp. 464-465): same four-cell structure with quantile influence functions `p_q, q_q, r_q, s_q` (p. 464; all divide by densities evaluated at composed quantile positions - see edge cases): + +``` +p_q(y) = ( 1 / f_Y,01( F_Y,01^{-1}( F_Y,00( F_Y,10^{-1}(q) ) ) ) ) * ( 1{ y <= F_Y,10^{-1}(q) } - F_Y,00( F_Y,10^{-1}(q) ) ) +q_q(y) = -( 1 / f_Y,01( F_Y,01^{-1}( F_Y,00( F_Y,10^{-1}(q) ) ) ) ) * ( 1{ F_Y,01(y) <= F_Y,00( F_Y,10^{-1}(q) ) } - F_Y,00( F_Y,10^{-1}(q) ) ) +r_q(y) = -( f_Y,00( F_Y,10^{-1}(q) ) / ( f_Y,01( F_Y,01^{-1}( F_Y,00( F_Y,10^{-1}(q) ) ) ) * f_Y,10( F_Y,10^{-1}(q) ) ) ) * ( 1{ F_Y,10(y) <= q } - q ) +s_q(y) = -( 1 / f_Y,11( F_Y,11^{-1}(q) ) ) * ( 1{ y <= F_Y,11^{-1}(q) } - q ) +``` + + `sqrt(N)*(tau_hat_q^CIC - tau_q^CIC) -->d N(0, V_q^p/alpha_00 + V_q^q/alpha_01 + V_q^r/alpha_10 + V_q^s/alpha_11)` for q ∈ (q_lower, q_upper); variance estimated by the same plug-in recipe (p. 465). Proof in the supplement. +- Panel data (Section 5.3, Theorems 5.5-5.6, pp. 468-469): the POINT estimator is identical; only the variance changes. Under Assumption 5.3 (per-group iid draws of the PAIR (Y_i0, Y_i1); `alpha_g = Pr(G=g) > 0`; same density regularity; `N_0 = N_00 = N_01`, `N_1 = N_10 = N_11`), the four linearization terms are no longer independent - within-unit cross-period covariances enter: + +``` +sqrt(N)*(tau_hat^CIC - tau^CIC) -->d N( 0, V^p/alpha_0 + V^q/alpha_0 + C^pq/alpha_0 + V^r/alpha_1 + V^s/alpha_1 + C^rs/alpha_1 ) +C^pq = E[p(Y_00)*q(Y_01)], C^rs = E[r(Y_10)*s(Y_11)] = Cov( k(Y_10), Y_11 ) +``` + + with plug-in covariance estimators `C_hat^pq = (1/N_0) sum_i p_hat(Y_00,i)*q_hat(Y_01,i)`, `C_hat^rs = (1/N_1) sum_i r_hat(Y_10,i)*s_hat(Y_11,i)`; Theorem 5.6 (consistency of all variance/covariance estimators) additionally requires `Y_10 ⊆ Y_00`. +- Bootstrap: **not discussed anywhere in the paper** - all inference is analytical/asymptotic with normal critical values; no degrees-of-freedom or small-sample adjustments are discussed. diff-diff v1 ships bootstrap inference as the default (the qte parity target's convention), with the analytical SEs above deferred; in panel mode resample units (pairs), in repeated cross-section mode resample within cells. +- Clustering: not discussed in the paper (sampling is iid within group-period cells, Assumptions 5.1(i)/5.3(i)). +- Hypothesis tests on distributions (p. 465): estimate `tau_hat_q^CIC` at several quantiles (three quartiles or nine deciles) and jointly test equality via a chi-squared test (details in AI (2002)); Kolmogorov-Smirnov or Cramer-von Mises tests on the entire distribution "may be possible"; first-/second-order stochastic dominance tests per Abadie (2002). + +*Edge cases:* +- Support violation (`Y_10 ⊄ Y_00`): treated-pre values outside the control-pre support -> `F_hat_Y,00` maps them to 0 or 1 and eq. (35) clamps to support endpoints -> warn; report Corollary 3.1 partial identification (quantiles limited to `[q_lower, q_upper]`; Manski-style ATT bounds under bounded support). +- Densities near zero: influence functions (37)-(38) and all quantile analogs divide by `f_Y,01(.)` (and `f_Y,10`, `f_Y,11` for quantiles); sparse regions of the control-post distribution blow up analytical variances. Assumption 5.1(iii) (densities bounded away from zero) is the substantive regularity condition -> detection: near-zero estimated densities at evaluation points -> surface a warning (analytical path; deferred in v1). +- Discrete outcomes / ties: violates Assumption 5.1(iii) -> only bounds are point-consistent without extra assumptions (Section 4, deferred); additionally Assumption 5.2 ties (`F_Y,01(y) = F_Y,00(y')` for some y, y') make even the bound estimates NON-convergent (p. 466; footnote 33's binary-median analogy) -> detection: duplicated outcome values / near-equal CDF values across the two control cells -> warn. +- Boundary quantiles: q at or outside `(q_lower, q_upper)` -> NaN inference (Theorem 5.3 is interior-only). +- Mixed continuous/discrete outcomes: eq. (8)'s identity `F_Y^{-1}(F_Y(y)) = y` can fail for mixed distributions (p. 440) -> out of scope; warn. +- Additive random group-time shocks: bias, not just inference distortion (p. 476) -> not detectable in a 2x2 design; testable with >2 periods via Theorem 6.4 (deferred) -> documented warning. +- Empty or tiny cells: `alpha_hat_gt = 0` breaks everything -> validation error. + +*Algorithm (no numbered algorithm in the paper; assembled from eqs. 34-36, Section 5.1, pp. 461-462):* +1. Validate the 2x2 design; split the sample into the four (g,t) cells. +2. Compute the empirical CDFs `F_hat_Y,gt` (eq. 34) for cells (0,0), (0,1), (1,0), and the sample mean of the (1,1) cell. +3. Transform every treated pre-period observation: `y_i -> F_hat_Y,01^{-1}( F_hat_Y,00(Y_10,i) )`, using exactly the eq. (35)/(A.1) inverse (ceiling order statistic `Y_(ceil(Nq))`, `F_hat^{-1}(0) = ` sample minimum). +4. `tau_hat^CIC = mean(Y_11) - mean(transformed Y_10)` (eq. 36). +5. Quantile effects at requested q ∈ (q_lower, q_upper): `tau_hat_q^CIC = F_hat_Y,11^{-1}(q) - F_hat_Y,01^{-1}(F_hat_Y,00(F_hat_Y,10^{-1}(q)))`. +6. Inference: v1 bootstrap (units in panel mode, within-cell in repeated cross-section mode); paper's analytical variance (Theorems 5.1-5.3, 5.5-5.6) deferred. +7. Treatment on the controls: rerun with group labels exchanged and negate (p. 446). + +### QDiD variant (Section 3.3, journal pp. 446-447) + +The paper formalizes quantile DiD as the natural comparison estimator; the library ships it alongside CiC (planned class `QDiD`). Applying DiD quantile-by-quantile (coefficients `alpha_q, beta_q, gamma_q` indexed by quantile) gives the transformation (unnumbered display, p. 447): + +``` +k^QDID(y) = y + F_Y,01^{-1}( F_Y,10(y) ) - F_Y,00^{-1}( F_Y,10(y) ) +``` + +with counterfactual distribution `F_{Y^N,11}(y) = Pr( k^QDID(Y_10) <= y )`. Computation: for fixed y, find its quantile in the treated pre distribution, `q = F_Y,10(y)`; add the over-time change in the control group at that quantile, `Delta^QDID = F_Y,01^{-1}(q) - F_Y,00^{-1}(q)`. In QDiD individuals are compared across both groups and time by their quantile; in CiC across groups by outcome value and across time by quantile (p. 447). + +Justifying model for continuous outcomes (eq. 22, p. 447): + +``` +(22) Y^N = h_tilde(U, G, T) = h_tilde^G(U, G) + h_tilde^T(U, T) +``` + +with (i) `h_tilde(u,g,t)` strictly increasing in u and (ii) `U ⊥ (G, T)`. Standard DiD is a special case of QDiD. Under continuity the counterfactual MEANS agree, `E[k^DID(Y_10)] = E[k^QDID(Y_10)]`, although the counterfactual distributions differ (p. 447) - so QDiD's mean ATT matches standard DiD; its value-add is distributional. + +The authors recommend CiC over QDiD (p. 447): (i) additive separability of h_tilde is hard to justify and is NOT invariant to monotone rescaling of y; (ii) QDiD forces the unobservable distribution to be identical in ALL subpopulations (`U ⊥ (G,T)`), eliminating intrinsic group heterogeneity; (iii) QDiD places testable restrictions on the data (footnote 21: `k^QDID` is not necessarily monotone without restrictions on `F_Y,00, F_Y,01, F_Y,10` - an implementation should check/report non-monotonicity of the fitted transformation). The CIC model with continuous outcomes imposes no testable restrictions and is scale-invariant. No asymptotic theory is given for QDiD in the paper; the discrete-outcome QDiD case is delegated to AI (2002). v1 pairs QDiD with bootstrap inference (matching `qte::QDiD()`). + +### Discrete outcomes: bounds and DCIC (Sections 4.1-4.2 and 5.2; deferred, documented) + +Weak monotonicity replaces strict (Assumption 4.2: `h(u,t)` nondecreasing in u); Assumption 4.3: `U_0, U_1` continuously distributed. **Theorem 4.1 (p. 453)** - under Assumptions 3.1, 3.3, 3.4, 4.2, 4.3 the counterfactual CDF is tightly bounded, using the two inverse conventions (eqs. 8 and 24): + +``` +(25) F_{Y^N,11}^LB(y) = F_Y,10( F_Y,00^{(-1)}( F_Y,01(y) ) ) [sup-based inverse, eq. 24] + F_{Y^N,11}^UB(y) = F_Y,10( F_Y,00^{-1}( F_Y,01(y) ) ) [inf-based inverse, eq. 8] +``` + +(both bounds 0 for `y < inf Y_01`, 1 for `y > sup Y_01`). The support condition can be weakened to `{inf U_1, sup U_1} ⊆ U_0` (p. 456). Binary-outcome ATT bounds (p. 453): + +``` +tau ∈ [E[Y_11^I] - 1, E[Y_11^I] - E[Y_10]] if E[Y_01] > E[Y_00] +tau = E[Y_11^I] - E[Y_10] if E[Y_01] = E[Y_00] +tau ∈ [E[Y_11^I] - E[Y_10], E[Y_11^I]] if E[Y_01] < E[Y_00] +``` + +Bound transformations and estimators (Section 5.2, eqs. 41-44, p. 467), with `F_lower` the strict-inequality CDF (eq. 41): + +``` +(42) k_lower(y) = F_Y,01^{-1}( F_lower_Y,00(y) ), k_upper(y) = F_Y,01^{-1}( F_Y,00(y) ) +(43) F_lower_hat_Y,00(y) = (1/N_00) sum_i 1{Y_00,i < y} +(44) k_lower_hat(y) = F_hat_Y,01^{-1}( F_lower_hat_Y,00(y) ), k_upper_hat(y) = F_hat_Y,01^{-1}( F_hat_Y,00(y) ) +``` + +Note `k_upper(y) ≡ k^CIC(y)`. **Lemma A.12 (pp. 489-490):** the CDF of `k_lower(Y_10)` is `F^UB_{Y^N,11}` and the CDF of `k_upper(Y_10)` is `F^LB_{Y^N,11}` (a larger transformation shifts the distribution up, hence a pointwise LOWER CDF), and + +``` +tau ∈ [ E[Y_11^I] - E[F_Y,01^{-1}(F_Y,00(Y_10))], E[Y_11^I] - E[F_Y,01^{-1}(F_lower_Y,00(Y_10))] ] +tau^LB = E[Y_11] - E[k_upper(Y_10)], tau^UB = E[Y_11] - E[k_lower(Y_10)] +``` + +Implementation subtlety (pp. 456, 467; NOT a contradiction): applied to discrete data, the continuous CDF composition eq. (9) evaluates to `F^UB` (identical expression to eq. 25's upper bound), and computing `E[Y_11^N]` from `F^UB` gives the LOWER bound on `E[Y_11^N]`, hence the UPPER bound on the ATE; whereas the continuous draw-level ATT estimator eq. (36) transforms draws by `k_upper_hat = k_hat^CIC` and therefore estimates `tau^LB`, the LOWER bound on the ATE. The two continuous-recipe entry points land on OPPOSITE endpoints of the bound interval; a discrete-aware implementation must compute both transformations explicitly. + +**Assumption 5.2 (Absence of Ties, p. 466):** Y finite and `F_Y,01(y) != F_Y,00(y')` for all y, y'. Without it the estimated bounds do NOT converge (footnote 33). Under it, the bound transformations are superconsistent - `sqrt(N)*(k_hat - k) ->p 0` exactly (Lemma A.13, pp. 491-492: with probability -> 1 the estimated transformation EQUALS the truth, via the minimum CDF gap `nu > 0`), so first-stage estimation contributes no first-order variance (footnote 34). **Theorem 5.4 (p. 467):** under Assumptions 5.1(i),(ii),(iv) and 5.2, + +``` +sqrt(N)*(tau_hat^UB - tau^UB) -->d N(0, V^s/alpha_11 + V_lower^r/alpha_10) +sqrt(N)*(tau_hat^LB - tau^LB) -->d N(0, V^s/alpha_11 + V_upper^r/alpha_10) +``` + +with `V_lower^r = Var(k_lower(Y_10))`, `V_upper^r = Var(k_upper(Y_10))`; confidence intervals for the partially identified parameter per Imbens and Manski (2004). Panel version: Theorem 5.7 (p. 469) adds `C_lower^rs = Cov(k_lower(Y_10), Y_11)` / `C_upper^rs = Cov(k_upper(Y_10), Y_11)` terms. + +Point identification restorations: **Theorem 4.2 (DCIC, pp. 457-458)** - under Assumptions 3.1, 3.3, 3.4 and 4.1-4.4 (Assumption 4.4, Conditional Independence: `U ⊥ G | Y, T`, p. 456; strict monotonicity implies 4.2 and 4.4, footnote 28), with the range of h a discrete set `{lambda_0,...,lambda_L}`: + +``` +(29) F_{Y^N,11}^DCIC(y) = F_Y,10( F_Y,00^{(-1)}(F_Y,01(y)) ) + + ( F_Y,10( F_Y,00^{-1}(F_Y,01(y)) ) - F_Y,10( F_Y,00^{(-1)}(F_Y,01(y)) ) ) + * [ F_Y,01(y) - F_Y,00( F_Y,00^{(-1)}(F_Y,01(y)) ) ] + / [ F_Y,00( F_Y,00^{-1}(F_Y,01(y)) ) - F_Y,00( F_Y,00^{(-1)}(F_Y,01(y)) ) ] +``` + +when the denominator is > 0, else `F_Y,10(F_Y,00^{-1}(F_Y,01(y)))` - a linear interpolation between the two bound CDFs (U_1 | Y = lambda_l taken uniform, p. 459); with 4.4 imposed the discrete model is fully parametric and "inference becomes standard using likelihood methods" (p. 466). Binary special case (p. 457) - always stays in [0,1], unlike linear DiD (p. 451 example: E[Y_10]=0.5, E[Y_00]=0.8, E[Y_01]=0.2): + +``` +E[Y_11^N] = (E[Y_01]/E[Y_00]) * E[Y_10] if E[Y_01] <= E[Y_00] +E[Y_11^N] = 1 - ((1-E[Y_01])/(1-E[Y_00])) * (1 - E[Y_10]) if E[Y_01] > E[Y_00] +``` + +Alternatively, **Theorems 4.3-4.4 (pp. 459-461)** restore identification via covariates without Assumption 4.4: under Assumptions 4.5 (`Y^N = h(U,T,X)`), 4.6 (h nondecreasing in u, continuous in x), 4.7 (`U ⊥ X | G`), 3.3, 3.4, 4.3, and `X_0t = X_1t`, bound-generating pairs (eqs. 30-31, p. 460) give conditional bounds `F^LB_{Y^N,11|X}(y|x) = F_{Y|X,10}(K_lower(y;x) | L_lower(y;x))`, `F^UB = F_{Y|X,10}(K_upper(y;x) | L_upper(y;x))`; and with `(33) S_t(y) = {u : exists x ∈ X_0t with u = F_{Y,0t|X}(y|x)}`, if `S_1(y) ⊆ Union_{y ∈ Y_00} S_0(y)` for all y ∈ Y_01, the distribution of `Y_11^N | X` is POINT identified (Theorem 4.4; proof normalizes `U_0 ~ U[0,1]` and matches conditional CDF values across periods, p. 461). + +### Covariates (Section 5.1, pp. 465-466; deferred, documented) + +Identification with covariates requires all assumptions conditional on X (p. 443); this also relaxes group-composition stability if all relevant changing factors are observable (p. 444). Three estimation routes: +1. Discrete X, fully nonparametric: apply the Theorem 5.1 estimator within covariate cells; average over the covariate distribution (p. 465). +2. Smoothing: estimate each `F_Y,gt( . | X)` by kernel/series, average per-X effects (p. 465). +3. Parametric residual route (pp. 465-466): `h(u,t,x) = h(u,t) + x'beta`, `h^I(u,t,x) = h^I(u,t) + x'beta`, `U ⊥ (T,X) | G` (treatment effect does not vary with X). Two-stage: OLS `Y_i = D_i'delta + X_i'beta + eps_i` with `D = ((1-T)(1-G), T(1-G), (1-T)G, TG)'` (four cell dummies, no intercept); form `Y_hat_i = Y_i - X_i'beta_hat = D_i'delta_hat + eps_hat_i` (group-time effects left in); apply CIC to the empirical distributions of the augmented residuals. Consistency/asymptotic normality and the asymptotic variance are shown in AI (2002), not the published paper. Footnote 32: extensions to `h(u,t) + m(x)` (series/kernel m) and group/time-varying beta. + +### Multiple groups and periods (Section 6, pp. 469-476; deferred, documented) + +Setup: groups `G = {1,...,N_G}`, periods `T = {1,...,N_T}`; `I` = set of treated (t,g) pairs, `N_I = |I|`; no group treated in the initial period (`(1,g) ∉ I`); treatment absorbing (staggered adoption). Effects are intrinsically group-and-period specific: `tau^CIC_{g,t} = E[Y^I_{g,t}] - E[h(U,t) | G=g]` (p. 471). For each treated `(g_1,t_1)` and each valid control/baseline pair `(g_0,t_0)` (all of `(g_0,t_0), (g_0,t_1), (g_1,t_0) ∉ I`), under Assumptions 3.1-3.3 and 6.1 (common support of U across all groups; relaxable as in 2x2): + +``` +(45) F_{Y^N,g_1,t_1}(y) = F_Y,g_1,t_0( F_Y,g_0,t_0^{-1}( F_Y,g_0,t_1(y) ) ) [Theorem 6.1] + +(46) kappa_hat_{g_0,g_1,t_0,t_1} = (1/N_{g_1,t_1}) sum_i Y_{g_1,t_1,i} + - (1/N_{g_1,t_0}) sum_i F_hat_Y,g_0,t_1^{-1}( F_hat_Y,g_0,t_0(Y_{g_1,t_0,i}) ) +``` + +`tau^CIC_{g_1,t_1} = kappa_{g_0,g_1,t_0,t_1}` for ALL valid quadruples - overidentification. Stack the `N_J` valid quadruples: Lemma 6.1 (asymptotic linearity, per-quadruple variance `V^p/alpha_{g_0,t_0} + V^q/alpha_{g_0,t_1} + V^r/alpha_{g_1,t_0} + V^s/alpha_{g_1,t_1}`), Theorem 6.2 (`sqrt(N)(kappa_hat_J - kappa_J) -->d N(0, V_J)`), Lemma 6.2 (plug-in `V_hat_J` consistent). Cross-covariances between quadruples follow the **Appendix B shared-cell rule** (pp. 493-494, 26 cases): two 2x2 sub-designs covary only through shared (group, period) cells; each shared cell contributes one term `N*E[mu_hat * mu_hat']`, with each factor's influence function determined by the ROLE the cell plays in that design (control-pre -> p, control-post -> q, treated-pre -> r, treated-post -> s); designs sharing no cell are asymptotically independent (case 26, C = 0); identical designs give the four-term variance sum (case 1). `V_J` is NOT necessarily full rank (footnote 39 constructs a 4-group example with an exact linear dependency) - use the Moore-Penrose generalized inverse `V_hat_J^(-)`. Combination via minimum distance with the `N_J x N_I` 0-1 map `A` (`kappa_J = A tau^CIC_I`): + +``` +tau_hat^CIC_I = (A' V_hat_J^(-) A)^{-1} (A' V_hat_J^(-) kappa_hat_J) [Theorem 6.3: sqrt(N)-normal, variance (A' V_J^(-) A)^{-1}] +``` + +Aggregation `tau_Lambda = Lambda' tau^CIC_I` (columns of Lambda sum to 1): sample-size weights `Lambda_{g,t} = N_{g,t}/sum N_{g,t}` are the natural default; variance-minimizing weights are appropriate only under constant effects (p. 475). Tests: **Theorem 6.4** overidentification/specification statistic `N (kappa_hat_J - A tau_hat_I)' V_hat_J^(-) (kappa_hat_J - A tau_hat_I) -->d Chi2(rank(V_J) - N_I)` - has power against violations of `U ⊥ T | G`, including additive random group-time effects (p. 476); equality-of-effects statistic `-->d Chi2(N_I - 1)` under `tau_{g,t} = tau` for all treated pairs (GLS-weighted pooled estimate; p. 476). diff-diff defers this section (Ciaccio / `ecic` staggered event-study CiC is the reference for the extension). + +**Reference implementation(s):** +- R: `qte::CiC()` and `qte::QDiD()` (Callaway's qte package - the project's chosen parity target; bootstrap inference, panel and repeated cross-section support) +- Stata: `cic` (Kranker) - implements the analytical SEs (Theorems 5.1-5.3) and the discrete-outcome bounds +- R: `ecic` (Kluser) - staggered event-study CiC (relevant to the deferred Section 6 extension) +- The paper itself points to the Econometrica supplement for the empirical application (Meyer-Viscusi-Durbin (1995) injury-duration data; headline finding: CiC vs standard DiD results can differ in both magnitude and significance, p. 477) + +**Requirements checklist:** +- [ ] Empirical inverse CDF implements the eq. (35)/(A.1) convention exactly: `inf{y : F_hat(y) >= q}` = ceiling order statistic `Y_(ceil(Nq))`, with `F_hat^{-1}(0)` = sample minimum (no interpolating quantile types) +- [ ] Point estimator matches eq. (36); parity with `qte::CiC()` point estimates on golden fixtures (panel + repeated cross-section) +- [ ] QDiD transformation matches the p. 447 display; parity with `qte::QDiD()`; non-monotone `k^QDID` detected and reported (footnote 21) +- [ ] QDiD mean effect equals standard DiD ATT on continuous data (internal consistency check, p. 447) +- [ ] Quantile effects restricted to interior `(q_lower, q_upper)` per eqs. (17)-(18) / Theorem 5.3; NaN-consistent inference outside +- [ ] Support check `Y_10 ⊆ Y_00` with warning and Corollary 3.1 messaging on violation +- [ ] Panel and repeated cross-section modes produce identical point estimates; resampling scheme differs (units vs within-cell) +- [ ] Discrete/tied outcomes detected -> warning citing Section 4 bounds and Assumption 5.2 (bounds themselves deferred) +- [ ] Treatment-on-controls via group-label exchange and negation (p. 446) +- [ ] Docstring warns that additive random group-time effects bias CiC and are untestable in the 2x2 design (p. 476) + +--- + +## Implementation Notes + +**Planned diff-diff v1 scope (2026-07-12):** +- Ship: `ChangesInChanges` + `QDiD`, 2x2 design, continuous outcomes, bootstrap inference, panel + repeated cross-section modes. +- Deferred: covariates (Theorems 4.3-4.4 and the Section 5.1 residual route; Melly-Santangelo is the modern reference), discrete-outcome bounds (Section 4 identification / Section 5.2 inference, incl. Imbens-Manski intervals), analytical SEs (Theorems 5.1-5.3, 5.5-5.7; Appendix B covariances), multiple groups/periods (Section 6; Ciaccio / `ecic`). +- All deferred material is reviewed in this document so the deferral is documented, not silent. + +### Data Structure Requirements +- Long format: one row per observation with outcome, group indicator (0/1), and period indicator (0/1); all four (g,t) cells non-empty (Assumption 5.1(ii)). +- Repeated cross-section mode: four independent samples, one per cell, sizes `N_00, N_01, N_10, N_11` (Assumption 5.1(i)). +- Panel mode: unit identifier required; each unit observed in both periods; per-group iid draws of the pair `(Y_i0, Y_i1)` (Assumption 5.3); the point estimator ignores the pairing (identical to repeated cross-section), but inference must respect it (resample units). +- Continuous outcomes; heavy ties trigger the discrete-outcome warning. +- Covariates: not accepted in v1 (deferred routes documented above). + +### Computational Considerations +- Sorting the four cells dominates: O(N log N). The transformation `F_hat_01^{-1}(F_hat_00(y))` evaluates via two `searchsorted` passes over sorted arrays - O(N_10 log N) for all treated-pre points. Memory: O(N). +- Bootstrap multiplies the estimation cost by B; each replicate re-sorts resampled cells (or re-uses sorted-order tricks). Embarrassingly parallel across replicates. +- The estimator is deliberately smoothing-free (p. 451); no kernels or bandwidths are needed for point estimation. The analytical variance path (deferred) additionally needs the footnote 31 boundary-consistent density estimator (bandwidth `N^{-1/3}`, one-sided difference quotient switching sides at the support midpoint) and, for `V_hat^p`/`V_hat^q`, V-statistic double sums `(1/(N_00 N_10)) sum_i sum_j P_hat(Y_00,i, Y_10,j)` - naively O(N^2), vectorizable. +- Exactness matters more than speed: the eq. (A.2) sandwich identities hold exactly for the ceiling-order-statistic inverse, and Lemma A.13's superconsistency argument (discrete case) relies on exact CDF comparisons - do not substitute interpolated quantiles anywhere in the pipeline. +- Proof-side rate constants (for reference, not code): Lemmas A.2/A.3 give uniform EDF and inverse-EDF convergence at any rate `N^delta, delta < 1/2`; Lemma A.5 (oscillation, via Stute (1982), Lemma A.4) holds for `0 < eta < 3/4`, `delta > max(2*eta - 1, eta/2)`; Lemma A.6 (Bahadur-type uniform linearity of the quantile process) for `0 < eta < 5/7` - the better-than-root-N rate that makes the two-step plug-in root-N normal without trimming. The Theorem 5.1 proof uses `delta = 1/3, eta = 1/2` (Lemma A.8), Lemma A.7 (V-statistic projection: `mu_hat^p - mu_hat^P = o_p(N^{-1/2})`), and Lemma A.9 (CLT across independent cells). + +### Tuning Parameters + +| Parameter | Type | Default | Selection Method | +|-----------|------|---------|-----------------| +| `quantiles` (quantile effects grid) | array of floats in (0,1) | deciles (library choice; paper suggests quartiles or deciles for joint tests, p. 465) | must lie strictly inside `(q_lower, q_upper)` from eq. (17); estimator NaNs/warns outside | +| `n_bootstrap` | int | library convention (paper: none - inference is analytical; qte parity target uses bootstrap) | SE stability; align with qte for parity tests | +| `mode` | {"panel", "rc"} | inferred from unit id presence | Assumption 5.3 vs 5.1 sampling; affects resampling only | +| `variant` | CiC vs QDiD (separate classes) | CiC | authors recommend CiC (p. 447); QDiD kept as comparison estimator | +| density bandwidth (deferred analytical SEs) | float | `N^{-1/3}` (footnote 31) | paper's one-sided EDF difference quotient; any boundary-uniform-consistent estimator admissible | + +### Relation to Existing diff-diff Estimators +- `DifferenceInDifferences` is the nested special case (eqs. 1, 4-6): when the linear model holds with full independence, CiC and DiD probability limits coincide (p. 463) - a natural cross-check test. Note neither estimator dominates on efficiency: `Var(tau_hat^CIC)` can equal, exceed, or fall below `Var(tau_hat^DID)` (p. 463; footnote 30's scale example `Y_g1 ~d sigma*Y_g0`: CiC strictly more efficient iff `sigma^2 < 1`). +- QDiD's mean effect reproduces the standard DiD ATT under continuity (p. 447) - reuse `DifferenceInDifferences` output as an oracle in tests. +- Reuse: 2x2 cell validation patterns from `DifferenceInDifferences`; bootstrap utilities and `safe_inference()` joint-NaN contract from `diff_diff.utils`; results-dataclass conventions (flat `att`/`se`/`conf_int` family). CiC is a standalone estimator (own `get_params`/`set_params` per the estimator-inheritance map in CLAUDE.md). +- The wage-decomposition estimand `E[Y_B1] - E[F_Y,W1^{-1}(F_Y,W0(Y_B0))]` (Altonji-Blank / Juhn-Murphy-Pierce, Section 3.5, pp. 448-449) is algebraically identical to `tau^CIC`, so the same code serves decomposition use cases (interpretation differs). +- The "selection on observables" panel alternative `F_{Y^N,11}(y) = E[F_{Y_01|Y_00}(y | Y_10)]` generally differs from the CiC counterfactual; they coincide iff `U_i0 = U_i1` (perfect rank correlation over time, p. 448) - do not confuse CiC with conditioning-on-lagged-outcome estimators. +- Future: the Section 6 multi-group machinery (overidentified `kappa` quadruples + minimum-distance combination + Theorem 6.4 specification test) is structurally similar to `CallawaySantAnna`'s (g,t) cell architecture; the Appendix B shared-cell covariance rule would slot into an influence-function aggregation layer. + +--- + +## Gaps and Uncertainties + +**Suspected typos in the published version (flagged by extraction; transcribed as printed, not silently corrected):** +- Theorem 5.7, second display (p. 469): the `V_upper^r` term's denominator prints as `alpha_10` where the panel notation elsewhere in the theorem uses `alpha_1`. Likely a typo; an implementation should use `alpha_1`. +- Lemma A.4 (pp. 479-480): the printed oscillation modulus reads `F_hat_Y(y+x) - F_hat_Y(x)` where the standard Stute (1982) modulus has `F_hat_Y(y+x) - F_hat_Y(y)`, matching the population term `F_Y(y+x) - F_Y(y)`. Likely a typesetting slip. +- Appendix B, case 2 (p. 493): printed with explicit argument notation `(Y_{g0,t0})` and a division by `alpha_{g0,t0}`, unlike every other case; the two readings are numerically equivalent (`N*E[mu_hat mu_hat'] = E[infl-fn product]/alpha_gt` for same-cell averages). Treat the per-shared-cell `N*E[mu_hat mu_hat']` pattern as the intended formula. +- Theorem 5.4 proof (p. 492): the final estimation-error term's summand sign is printed inconsistently with the preceding line's decomposition (`k_lower_hat - k_lower` vs `k_lower - k_lower_hat`); immaterial since the term converges to zero (Lemma A.13). +- Equality-of-effects test (p. 476): typeset ambiguity between `A` and `A_I` in the weight matrix `Lambda = (iota' A_I' V_J^(-) A iota)^{-1} iota' A' V_J^(-) A` and in the test statistic's sandwich; the structure (GLS weighting with the generalized inverse) is clear, but consult p. 476 before implementing. + +**Extraction-file discrepancy (RESOLVED):** the empirical-application citation was rendered "Durbin" in one extraction pass and "Dubin" in two others. Verified against the paper's reference list (journal p. 496): "MEYER, B., K. VISCUSI, AND D. DURBIN (1995): 'Workers' Compensation and Injury Duration: Evidence from a Natural Experiment,' American Economic Review, 85, 322-340" - Durbin is correct. + +**Equations/statements not transcribed by any extraction pass (consult journal pages directly if needed):** +- Equation (23) (Section 4 opening, pp. 451-453) and equations (26)-(28) (Section 4.1-4.2, pp. 453-457) - intermediate displays between the transcribed eq. (22) and eqs. (24)-(25)/(29). +- Equation (32) (Section 4.3, pp. 460-461), between the transcribed eqs. (30)-(31) and (33). +- Assumption 4.1's verbatim statement (Section 4.1, ~p. 452; from Theorem 4.2's citation it is the discreteness assumption on the range of h, `{lambda_0,...,lambda_L}`). Assumptions 4.2/4.3 were captured as paraphrases (weak monotonicity; U continuously distributed), not verbatim. +- Corollary 6.1 (Section 6.2, pp. 472-475), referenced in the Theorem 6.2 proof (p. 492). +- The full verbatim 26-case Appendix B enumeration is preserved in the extraction archive; this review carries the complete generating rule (shared-cell role mapping), which determines all 26 cases. + +**Material delegated by the paper to external sources (not in the journal pages):** +- Proofs of Theorems 5.3, 5.5, 5.7 and Lemmas A.1, A.5, A.6, A.10, A.11 - Econometrica supplement. +- The disability-insurance empirical application - supplement (no simulations or applications appear in the journal pages). +- Discrete-outcome QDiD, the joint chi-squared quantile-equality test details, and the asymptotic variance of the covariance-adjusted (residual) CIC estimator - AI (2002) working paper. + +**Substantive uncertainties for the implementation:** +- Bootstrap inference has NO basis in this paper - the paper's inference is exclusively analytical (influence functions, Theorems 5.1-5.7). The v1 bootstrap default follows the qte parity target; validity arguments must be sourced elsewhere (e.g., the functional delta method literature); document this in the estimator entry when shipping. +- No QDiD asymptotic theory exists in the paper at all; QDiD inference is entirely a library/parity-target convention. +- The discrete-case bound duality (continuous eq. (9) at CDF level -> `F^UB` -> ATE upper bound; continuous eq. (36) at draw level -> `tau^LB`; pp. 456, 467, 489-490) is easy to get backwards; both statements are correct at their respective levels and were cross-checked against Lemma A.12 during synthesis. +- The paper gives no guidance on bandwidth-free small-sample behavior, degrees-of-freedom corrections, or clustered sampling - all inference is first-order asymptotic under iid-within-cell sampling. +- `q_lower`/`q_upper` (eq. 17) are population objects; the paper does not specify a finite-sample rule for declaring a requested quantile "interior" - a plug-in check with the empirical CDFs is the natural choice, but it is a library decision, not a paper prescription. diff --git a/docs/methodology/papers/callaway-li-oka-2018-review.md b/docs/methodology/papers/callaway-li-oka-2018-review.md new file mode 100644 index 000000000..01e07c52b --- /dev/null +++ b/docs/methodology/papers/callaway-li-oka-2018-review.md @@ -0,0 +1,417 @@ +# Paper Review: Quantile Treatment Effects in Difference in Differences Models under Dependence Restrictions and with only Two Time Periods + +**Authors:** Brantly Callaway, Tong Li, and Tatsushi Oka +**Citation:** Callaway, B., Li, T., & Oka, T. (2018). Quantile Treatment Effects in Difference in Differences Models under Dependence Restrictions and with only Two Time Periods. *Journal of Econometrics*, 206(2), 395-413. https://doi.org/10.1016/j.jeconom.2018.06.008 +**PDF reviewed:** **arXiv:1702.03618v1** (https://arxiv.org/abs/1702.03618v1, submitted 13 Feb 2017; title page reads 'This version: February 14, 2017'; 34 PDF pages, PDF page N = printed page N-1). Per the project's PDFs-never-committed convention the local PDF is kept outside the repository (gitignored `papers/callaway-li-oka-1702.03618v1.pdf`). The published Journal of Econometrics 206(2) version (https://doi.org/10.1016/j.jeconom.2018.06.008) is the version of record and may renumber equations/assumptions - all numbers below are pinned to arXiv v1; reconcile against the published version before implementing anything sensitive. +**Review date:** 2026-07-12 + +--- + +## Methodology Registry Entry + +*Drafted in docs/methodology/REGISTRY.md format for FUTURE use. Do not copy this section into the registry now - see the status line below.* + +## PanelQTT (Callaway-Li-Oka two-period QTT) + +**Status: NOT shipping as a diff-diff estimator in the CiC/QDiD v1 (scope decision 2026-07-12).** This paper is reviewed as (a) the methodological grounding for panel-mode distributional DiD and the dependence-restriction (Copula Invariance) approach, and (b) part of the foundations of the R `qte` package, the initiative's chosen parity target for CiC/QDiD. Any future implementation of this estimator would be a separate scoped PR. + +**Primary source:** Callaway, B., Li, T., & Oka, T. (2018). Quantile Treatment Effects in Difference in Differences Models under Dependence Restrictions and with only Two Time Periods. *Journal of Econometrics*, 206(2), 395-413. https://doi.org/10.1016/j.jeconom.2018.06.008 +- All proofs are in the Appendix (printed pp. 22-28); Tables 1-2 (Monte Carlo) are on printed pp. 29-30 and Tables 3-4 / Figure 1 (application) on printed pp. 31-33, after the References (printed pp. 18-21). +- **Numbering quirk (verified by full-text search during extraction):** the main text has Assumptions A1, A2, A3, A4, A6, and B. There is NO Assumption A5 anywhere in the paper - the label "A5" appears only as appendix equation (A5) and appendix Lemma A5. Lemma 1, Proposition 1, Theorem 2, Lemma 2, and Theorem 3 all cite "Assumption A1-A6", which in practice means A1-A4 plus A6. + +**Model (Section 2.1, printed pp. 3-4):** + +Panel data with exactly two periods. All individuals are untreated in period `t-1`; a fraction becomes treated in period `t`. Data: `{(Y_{i,t-1}, Y_it, X_i, D_it)}_{i=1}^n`. Observed outcomes (eq. (1), printed p. 4): + +``` +(1) Y_{i,t-1} := Y_{i,t-1}(0) and Y_it := (1 - D_it) Y_it(0) + D_it Y_it(1) +``` + +The two panel observables that drive everything are the outcome change `DeltaY_it := Y_it - Y_{i,t-1}` and the initial outcome level `Y_{i,t-1}`. `DeltaY_it(0) := Y_it(0) - Y_{i,t-1}(0)` is the time-difference of untreated potential outcomes (printed p. 5); for untreated units `DeltaY_it = DeltaY_it(0)`. + +Why two periods suffice (Introduction, printed pp. 1-2): the Copula Invariance assumption replaces the treated group's unknown copula (between the change and the initial level of untreated potential outcomes) with the untreated group's *observed* copula in the same two periods. This contrasts with Callaway and Li (2015), which needs at least three periods of panel data because it replaces an unknown unconditional copula in the last two periods with an observed copula from the first two periods. + +Notation (printed pp. 3-4, 8-9): + +| Symbol | Meaning | +|---|---| +| `D_it` | Treatment indicator: 1 if individual i treated in period t; nobody treated in t-1 | +| `Y_is(0)`, `Y_is(1)` | Potential outcomes in period `s ∈ {t-1, t}` | +| `X_i`, `X` (calligraphic) | Covariate vector (may include time-varying and time-invariant variables); `X` = common support of `X_i` for treated and untreated groups | +| `F_{Y_s(j)|X,D_t}`, `F_{Y_s|X,D_t}` | Conditional distributions of potential / observed outcomes | +| `F^{-1}_{Y_t(j)|X,D_t}(tau) := inf{y ∈ R : F_{Y_t(j)|X,D_t}(y) >= tau}` | Conditional quantile function, j = 0, 1 (printed p. 4; inf-based, weak inequality) | +| `C_{DeltaY_t(0),Y_{t-1}(0)|X,D_t}` | Conditional copula of `(DeltaY_it(0), Y_{i,t-1}(0))` given `X_i` and `D_it` | +| `tau ∈ T ⊂ (0,1)` | Quantile index; T a compact subset strictly inside the unit interval (printed p. 9) | +| `delta^{(d)}_{i,x} := 1{X_i = x, D_it = d}`, `n_x^{(d)} = SUM_i delta^{(d)}_{i,x}` | Cell indicator and cell size for discrete covariates (printed p. 8) | +| `r_x^{(j)} := lim_{n->inf} (n / n_x^{(j)})^{1/2} ∈ (0, inf)` | Relative cell-size limits, j = 0, 1 (Assumption A6(b)) | +| `m^(s)` | Sample size of the period-s repeated cross section (Corollary 1) | +| `W_i^{(d)}` | Exchangeable bootstrap weights for group d (Assumption B) | +| `∘` | Function composition | + +Sampling scheme: i.i.d. observations *within* treatment and control group (conditional random sampling), which "allows for the possibility that the marginal or joint distributions of potential outcomes can be different between treatment and control groups" (printed p. 4). + +Covariates are discrete for the estimation theory (Section 3, printed p. 8): "we consider the case where all covariates are discrete, which allows for nonparametric estimation that does not suffer from the curse of dimensionality" (motivated on printed p. 3, citing Chernozhukov et al. 2013b and Graham et al. 2015). + +**Target parameters (printed pp. 4, 7):** + +CQTT (Conditional Quantile Treatment effect on the Treated) - the QTE within the subpopulation with `D_it = 1` and common history `X_i = x`. For `x ∈ X`, `tau ∈ T ⊂ (0,1)`: + +``` +Delta_x^QTT(tau) := F^{-1}_{Y_t(1)|X=x,D_t=1}(tau) - F^{-1}_{Y_t(0)|X=x,D_t=1}(tau) +``` + +`F_{Y_t(1)|X=x,D_t=1}` is identified directly by the observed `F_{Y_t|X=x,D_t=1}`; the entire problem is the counterfactual `F_{Y_t(0)|X=x,D_t=1}`. The unconditional QTT is identified by averaging the conditional counterfactual distribution of Theorem 1 over covariates and inverting (discussion after Theorem 1, printed p. 7), but "for the rest of the paper we focus only on the CQTT" - the paper gives no formal aggregation theory (see Gaps). + +**Assumptions (exact statements):** + +*Assumption A1 (Random sampling) - printed p. 4:* + +> The two-periods panel data consists of observations `{(Y_{i,t-1}, Y_it, X_i, D_it)}_{i=1}^n` from the structure in (1). The potential outcomes `(Y_{i,t-1}(0), Y_{i,t-1}(1))` and `(Y_it(0), Y_it(1))` are cross-sectionally i.i.d. conditional on treatment status `D_it`. + +*Assumption A2 (Distributional Difference in Differences) - printed p. 5:* + +``` +Pr{ DeltaY_it(0) <= Deltay | X_i, D_it = 1 } = Pr{ DeltaY_it(0) <= Deltay | X_i, D_it = 0 }, +for all Deltay ∈ supp(DeltaY_it(0) | X_i). +``` + +- The distributional extension of mean parallel trends `E[DeltaY_it(0)|X_i, D_it = 1] = E[DeltaY_it(0)|X_i, D_it = 0]` (necessary for the ATT; Heckman et al. 1998, Abadie 2005). "The distributional restriction in Assumption A2 replaces this standard mean restriction." +- A2 restricts only the *change* `DeltaY_it(0)`; it does NOT require the marginal distribution of the initial level `Y_{i,t-1}(0)` to be the same across groups: "the initial distribution of outcomes can be different for the two groups" (printed p. 6). +- "If multiple pre-treatment periods in sample are available, then this assumption can be tested under a strict stationary assumption" (printed p. 5). + +*Assumption A3 (Copula Invariance) - printed p. 5 (the "dependence restriction"):* + +> For each `x ∈ X` and for all `(u, v) ∈ [0,1]^2`, + +``` +C_{DeltaY_t(0),Y_{t-1}(0)|X=x,D_t=1}(u, v) = C_{DeltaY_t(0),Y_{t-1}(0)|X=x,D_t=0}(u, v). +``` + +Interpretation (printed p. 6): +- The copula captures the "rank dependency" between `DeltaY_it(0)` and `Y_{i,t-1}(0)`; A3 requires this rank dependency to be the same for treated and control groups. Intuition: "if ... observations in the control group at the top of the distribution of initial outcomes tend to experience the largest increases in outcomes over time, the Copula Invariance assumption implies that this would also occur for the treated group." +- A3 does NOT rule out the joint distribution of `(DeltaY_it(0), Y_{i,t-1}(0))` differing between groups (only the copula is restricted), and it "allows for the marginal distribution of untreated outcomes in the period before treatment to differ for the treated and control groups" (printed p. 1). +- A3 does not imply A2 and A2 does not imply A3 - they are separate restrictions ("A3 restricts only the copula, not the marginal distribution of the change in untreated potential outcomes over time"). +- Testability: A2 and A3 "are not directly testable. However, in the spirit of placebo testing in DID models, they both can be tested using periods before the treated group becomes treated." Simpler route: implement the procedure in earlier periods and test `Delta_x^QTT(tau) = 0` for all `tau ∈ T`. + +*Assumption A4 (Continuous distributions) - printed p. 6:* + +> Each random variable of `DeltaY_it(0)` and `Y_{i,t-1}(0)` has a continuous distribution conditional on `X_i` and `D_it` and a random variable `Y_it(1)` also has a continuous distribution conditional on `X_i` and `D_it = 1`. Each distribution has a compact support with densities uniformly bounded away from 0 and infinity over the support. + +- Continuity guarantees the conditional copulas in A3 are unique; identification uses the Rosenblatt transform (Rosenblatt, 1952). +- The compact-support part is "as in Athey and Imbens (2006) in order to avoid technical difficulties in the rest of analysis, while this condition is not used for our identification analysis and can be replaced by other conditions for the rest of the results." + +*Assumption A6 - printed p. 9 (estimation; note there is no A5):* + +> (a) A pair of random variables `(DeltaY_it, Y_{i,t-1})` is continuously distributed conditional on `X_i` and `D_it = 0` over a compact support with a distribution `F_{DeltaY_t,Y_{t-1}|X,D_t=0}` and a density `f_{DeltaY_t,Y_{t-1}|X,D_t=0}`. A random variable `DeltaY_it` is continuously distributed conditional on `Y_{i,t-1}`, `X_i`, and `D_it = 0` with a uniformly continuous density `f_{DeltaY_t|Y_{t-1},X,D_t=0}` over a compact support. (b) The sample sizes `n_x^{(0)}` and `n_x^{(1)}` go to infinity as `n -> infinity`, while `r_x^{(j)} := lim_{n->inf} (n/n_x^{(j)})^{1/2} ∈ (0, inf)` for `j = 0, 1`. + +For Theorem 2 additionally: `F_{Y_t(0)|X,D_t=1}` admits a positive continuous density on an interval `[a,b]` containing an epsilon-enlargement of `{F^{-1}_{Y_t(0)|X,D_t=1}(tau) : tau ∈ T}` (standard for quantile inversion / Hadamard differentiability; printed pp. 10-11). + +*Assumption B (bootstrap weights) - printed p. 11:* + +> For each `d ∈ {0,1}`, let `(W_1^{(d)}, ..., W_n^{(d)})` be an n-dimensional vector of exchangeable, nonnegative random variables. The vectors `(W_1^{(0)}, ..., W_n^{(0)})` and `(W_1^{(1)}, ..., W_n^{(1)})` are independent of the original sample as well as each other. The vectors of random weights, depending on the size of each group, satisfy: + +``` +max_{1<=i<=n} E|W_i^{(d)}|^{2+eps} < inf, +W_bar_{n,x}^{(d)} := (1/n_x^{(d)}) SUM_i W_i^{(d)} delta^{(d)}_{i,x} ->_p 1, +(1/n_x^{(d)}) SUM_i (W_i^{(d)} - W_bar_{n,x}^{(d)})^2 delta^{(d)}_{i,x} ->_p 1, +for each d ∈ {0,1}. +``` + +**Identification (Theorem 1, printed p. 7):** + +Under A2 alone, for the treated group one identifies (i) `F_{Y_{t-1}(0)|X,D_t=1}` from observed outcomes and (ii) `F_{DeltaY_t(0)|X,D_t=1}` via A2; hence the ATT is identified. But the CQTT is NOT point-identified: "many possible distributions of untreated potential outcomes in period t are observationally equivalent" - `F_{Y_t(0)|X,D_t=1}` is highly unequal if change and initial level are strongly positively dependent, less unequal if independent or negatively dependent. The CQTT "can be partially identified along the line of Fan and Yu (2012)." By Sklar's theorem (printed p. 5) the missing object is exactly the conditional copula, which A3 supplies from the untreated group. + +> **Theorem 1.** Suppose that Assumption A1-A4 hold. Then, + +``` +F_{Y_t(0)|X=x,D_t=1}(y) + = Pr{ DeltaY_it + F^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_{Y_{t-1}|X=x,D_t=0}(Y_{i,t-1}) <= y | X_i = x, D_it = 0 }, +``` + +> for all `x ∈ X` and `y ∈ supp(Y_it(0) | X_i = x, D_it = 1)`. + +Reading: the counterfactual distribution is identified entirely from observed outcomes of *untreated* individuals (`D_it = 0`). Each untreated unit's observed change `DeltaY_it` is added to a quantile-quantile transplanted initial level: the untreated unit's period t-1 outcome is mapped to its rank in the untreated t-1 distribution (`F_{Y_{t-1}|X=x,D_t=0}`), then to the treated group's t-1 outcome at that same rank (`F^{-1}_{Y_{t-1}|X=x,D_t=1}`). "This implies that treated and untreated groups must be similar in the distributional sense of not only marginal distribution but also some dependency over periods, and thus Assumption A2 and A3 play a crucial role." + +Identification mechanics (Proof of Theorem 1, printed p. 22): define the ranks + +``` +(A1) U_i^d := F_{DeltaY_t(0)|X=x,D_t=d}(DeltaY_it(0)) and V_i^d := F_{Y_{t-1}(0)|X=x,D_t=d}(Y_{i,t-1}(0)), +``` + +for `d ∈ {0,1}`; under Assumption A4, + +``` +(A2) DeltaY_it(0) = F^{-1}_{DeltaY_t(0)|X=x,D_t=d}(U_i^d) and Y_{i,t-1}(0) = F^{-1}_{Y_{t-1}(0)|X=x,D_t=d}(V_i^d) +``` + +almost surely (Rosenblatt, 1952). The joint distribution of `(U_i^d, V_i^d)` given `(X_i, D_it) = (x, d)` is the conditional copula `C_{DeltaY_t(0),Y_{t-1}(0)|X=x,D_t=d}`, invariant with respect to `D_it` under A3; combining with A2 (`F^{-1}_{DeltaY_t(0)|X=x,D_t=1}(.) = F^{-1}_{DeltaY_t(0)|X=x,D_t=0}(.)`) delivers the counterfactual `F_{Y_t(0)|X=x,D_t=1}`. Note the copula is handled *implicitly* by pairing each untreated observation's own (change, initial level) - it is never estimated as an explicit copula object. + +**Repeated cross sections (Corollary 1, printed p. 7):** + +> **Corollary 1.** Consider the repeated cross sections `{(Y_is, X_i, D_is)}_{i=1}^{m^(s)}` in period `s ∈ {t-1, t}` with `m^(s)` being the sample size. Suppose that the data generating process for the repeated cross sections satisfy Assumption A1-A4 hold. If the conditional copula of `(Y_{i,t-1}(0), Y_{i,t}(0))` given `X_i = x` and `D_it = 1` satisfies the rank invariance: for every `(u, v) ∈ [0,1]^2`, + +``` +C_{Y_{t-1}(0),Y_t(0)|X=x,D_t=1}(u, v) = min{u, v}, +``` + +> then, for `y ∈ supp(Y_it(0) | X_i = x, D_it = 1)`, + +``` +F_{Y_t(0)|X=x,D_t=1}(y) + = Pr{ DeltaY_tilde_it + F^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_{Y_{t-1}|X=x,D_t=0}(Y_{i,t-1}) <= y | X_i = x, D_it = 0 }, + +where DeltaY_tilde_it := F^{-1}_{Y_t|X,D_t=0} ∘ F_{Y_{t-1}|X,D_t=0}(Y_{i,t-1}) - Y_{i,t-1}. +``` + +Discussion (printed pp. 7-8): conditional rank invariance means "for observations with the same observed covariates, individuals maintain their rank in the distribution of outcomes over time." It is "weaker than unconditional rank invariance as some individuals can change their rank"; it "does not imply nor is implied by the Copula Invariance assumption, nor does it imply conditional rank invariance between `DeltaY_it(0)` and `Y_{it-1}(0)`." The extension requires restricting to time-invariant covariates. Proof mechanics (printed p. 22): the individual change `DeltaY_it(0)` is not observed in repeated cross sections, so under rank invariance `F_{Y_t(0)|X=x,D_t=0}(Y_it(0)) = F_{Y_{t-1}(0)|X=x,D_t=0}(Y_{i,t-1}(0))` and the change for untreated individuals is imputed by the `DeltaY_tilde_it` display above. + +**IMPORTANT - unresolved conditioning flag:** the Corollary 1 statement conditions the rank-invariance copula on `D_it = 1` (treated group), but the identification formula recovers `DeltaY_tilde_it` from *untreated* observables and the appendix proof applies rank invariance to untreated units (`D_t = 0` in both displays of the proof, printed p. 22). See Gaps and Uncertainties - resolve against the published JoE version before any implementation. + +Contrast with Athey-Imbens CiC: CiC's repeated cross-section identification (Athey and Imbens 2006, Section 3.4) requires only equal within-group marginals of the unobservable over time - CiC does not need rank invariance over time, whereas this paper's panel-to-RC extension does. Panel data is what lets this paper avoid rank invariance for the main result. + +**Estimator (Section 3.1, printed p. 8; discrete covariates, cell-by-cell):** + +Step 1 - conditional empirical distributions (eq. (2)), for `s ∈ {t-1, t}` and `d ∈ {0, 1}`: + +``` +(2) F_hat_{Y_s|X=x,D_t=d}(y) := (1 / n_x^{(d)}) SUM_{i=1}^n 1{Y_is <= y} delta^{(d)}_{i,x} +``` + +The treated-outcome distribution estimator is `F_hat_{Y_t(1)|X=x,D_t=1}(y) = F_hat_{Y_t|X=x,D_t=1}(y)` (since `Y_it = Y_it(1)` if `D_it = 1`). + +Step 2 - plug-in counterfactual distribution (eq. (3)): obtain estimated quantiles `F_hat^{-1}_{Y_{t-1}|X=x,D_t=1}` from the empirical distribution, then set + +``` +(3) F_hat_{Y_t(0)|X=x,D_t=1}(y) + := (1 / n_x^{(0)}) SUM_{i=1}^n 1{ DeltaY_it + F_hat^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_hat_{Y_{t-1}|X=x,D_t=0}(Y_{i,t-1}) <= y } delta^{(0)}_{i,x} +``` + +for `y ∈ R` - the empirical analogue of Theorem 1, summing over untreated observations in cell x, each transformed by the empirical quantile-quantile map. + +Step 3 - CQTT by quantile inversion (unnumbered display, printed p. 8): + +``` +Delta_hat_x^QTT(tau) := F_hat^{-1}_{Y_t(1)|X=x,D_t=1}(tau) - F_hat^{-1}_{Y_t(0)|X=x,D_t=1}(tau), (tau, x) ∈ T x X. +``` + +Quantile functions use the inf-based convention `F^{-1}(tau) := inf{y : F(y) >= tau}` (printed p. 4). The estimator is a pure empirical-distribution plug-in: no bandwidths, kernels, or smoothing parameters anywhere; covariates are handled as discrete cells, not by smoothing. + +Appendix plug-in map (display (A4), printed p. 24): the counterfactual estimator "can be considered as an empirical distribution indexed by estimated distribution functions" (Proof of Proposition 1, printed p. 27), i.e. `F_hat_{Y_t(0)|X=x,D_t=1} = phi_n(F_hat_{Y_{t-1}|X=x,D_t=0}, F_hat_{Y_{t-1}|X=x,D_t=1})` for a generic sample map: for `F = (G, H)` a pair of distribution functions and `w` in a compact set `W`, + +``` +(A4) phi_n(F)(w) := n^{-1} SUM_{i=1}^n 1{ V_{1i} + G^{-1} ∘ H(V_{2i}) <= w }, +``` + +with population counterpart `phi(F)(w) := Pr{ V_1 + G^{-1} ∘ H(V_2) <= w }` (Lemma A2). In the paper's objects, `V_1` is the change and `V_2` the initial level for the untreated group. (The two extraction passes disagree on which of G, H maps to the treated vs untreated initial-level CDF in this generic notation - see Gaps and Uncertainties; the main-text eq. (3) composition above is unambiguous.) + +**Asymptotic theory (Section 3.2, printed pp. 9-11; proofs printed pp. 26-27):** + +Infeasible-estimator machinery (printed p. 9): `Y_tilde_it := DeltaY_it + F^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_{Y_{t-1}|X=x,D_t=0}(Y_{i,t-1})` (true, not estimated, transforms) with empirical process + +``` +(4) G_tilde_{t,x}^{(0)}(y) := sqrt(n) ( F_tilde_{Y_t(0)|X=x,D_t=1}(y) - F_{Y_t(0)|X=x,D_t=1}(y) ), y ∈ Y_{t|x,1}(0). +``` + +(The v1 text describes `F_tilde` as "based on observations {Y_tilde_it} with X_i = x and D_it = 1 as in (2)"; note the feasible counterpart (3) and its bootstrap analogue both weight by `delta^{(0)}_{i,x}`, i.e. untreated observations - the `D_t = 1` in `F_tilde_{Y_t(0)|X=x,D_t=1}` names the distribution being estimated, not the observations used.) + +| Result | Page | Establishes | +|---|---|---| +| **Lemma 1** | p. 9 | Functional CLT: `(G_tilde_{t,x}^{(0)}, G_hat_{t-1,x}^{(0)}, G_hat_{t,x}^{(1)}, G_hat_{t-1,x}^{(1)}) ⇝ (V_x^{(0)}, W_x^{(0)}, V_x^{(1)}, W_x^{(1)})` in `S_x := l^inf(Y_{t|x,1}(0)) x l^inf(Y_{t-1|x,0}) x l^inf(Y_{t|x,1}) x l^inf(Y_{t-1|x,1})`, a tight zero-mean Gaussian process with covariance kernel `diag{Sigma_x^{(0)}, Sigma_x^{(1)}}` (under A1-A6, i.e. A1-A4 + A6). Proof (p. 26): functional CLT for empirical distribution functions, van der Vaart and Wellner (1996), Ch. 2 | +| **Proposition 1** | p. 10 | Joint limit of the estimated potential-outcome distributions: `(Z_hat_x^{(0)}, Z_hat_x^{(1)}) ⇝ (Z_x^{(0)}, Z_x^{(1)})` where `Z_hat_x^{(j)}(y) := sqrt(n)(F_hat_{Y_t(j)|X=x,D_t=1}(y) - F_{Y_t(j)|X=x,D_t=1}(y))`; `Z_x^{(0)} := r_x^{(0)} V_x^{(0)} + kappa_x(W_x^{(0)}, W_x^{(1)})` and `Z_x^{(1)} = r_x^{(1)} V_x^{(1)}` - the extra `kappa_x` term reflects first-step estimation error, so the limit is NOT nuisance-parameter free | +| **Theorem 2** | pp. 10-11 | Functional CLT for the CQTT process: `sqrt(n)(Delta_hat_x^QTT(tau) - Delta_x^QTT(tau)) ⇝ Z_bar_x^{(1)}(tau) - Z_bar_x^{(0)}(tau)` in `(l^inf(T))^2`, with `Z_bar_x^{(j)}(tau) := Z_x^{(j)}(F^{-1}_{Y_t(j)|X=x,D_t=1}(tau)) / f_{Y_t(j)|X=x,D_t=1}(F^{-1}_{Y_t(j)|X=x,D_t=1}(tau))` - parametric sqrt(n) rate, via the functional delta method | +| **Corollary 2** | p. 11 | Under `H_0: Delta_x^QTT(tau) = 0` for all `tau ∈ T`: `KS_x ->_d sup_{tau∈T} |Z_bar_x^{(1)}(tau) - Z_bar_x^{(0)}(tau)|`; basis for uniform confidence bands. Proof (p. 27): continuous mapping theorem (Kosorok 2007, Sec. 2.1) | +| **Lemma 2** | p. 13 | Bootstrap empirical processes consistently estimate the Lemma 1 limits (conditional weak convergence in probability, `⇝_p`), under A1-A6 + B. Proof (p. 28): Theorem 3.6.13 of van der Vaart and Wellner (1996) | +| **Theorem 3** | p. 13 | First-order validity of the exchangeable bootstrap: `(Z_hat_x^{(0)*}, Z_hat_x^{(1)*}) ⇝_p (Z_x^{(0)}, Z_x^{(1)})` and `sqrt(n)(Delta_hat_x^{QTT*}(tau) - Delta_hat_x^QTT(tau)) ⇝_p Z_bar_x^{(1)}(tau) - Z_bar_x^{(0)}(tau)`, `tau ∈ T` | + +Key auxiliary objects in Proposition 1 (printed p. 10): + +``` +kappa_x(W_0, W_1) := INTEGRAL { r_x^{(0)} W_0(v) - r_x^{(1)} W_1 ∘ F^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_{Y_{t-1}|X=x,D_t=0}(v) } omega_x(y, v) dv, + +omega_x(y, v) := f_{DeltaY_t,Y_{t-1}|X=x,D_t=0}( y - F^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_{Y_{t-1}|X=x,D_t=0}(v), v ) + / f_{Y_{t-1}|X=x,D_t=1} ∘ F^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_{Y_{t-1}|X=x,D_t=0}(v), + +for (y, v) ∈ Y_{t|x,1}(0) x Y_{t-1|x,0}. +``` + +Proof structure (Appendix, printed pp. 23-27), useful for any influence-function or analytical-SE work later: +- **Lemma A1** (p. 23): the quantile-quantile map `psi(F) = G^{-1} ∘ H` for `F = (G,H)` (E = distribution functions with strictly positive, bounded density) is Hadamard differentiable with derivative `psi'_{F_0}(gamma) = (gamma_2 - gamma_1 ∘ G_0^{-1} ∘ H_0) / (g_0 ∘ G_0^{-1} ∘ H_0)`. Implementation relevance: the transform is only differentiable where the base density is strictly positive and bounded - degenerate/atomic outcome distributions violate the framework. +- **Lemma A2** (p. 23): the plug-in map `phi(F)(w) := Pr{V_1 + G^{-1} ∘ H(V_2) <= w}` is Hadamard differentiable with derivative `phi'_{F_0}(gamma)(w) = INTEGRAL (gamma_2(v_2) - gamma_1 ∘ G_0^{-1} ∘ H_0(v_2)) * f_{V1|V2}(w - G_0^{-1} ∘ H_0(v_2) | v_2) / (g_0 ∘ G_0^{-1} ∘ H_0(v_2)) dv_2`; requires `f_{V1|V2}` uniformly continuous and uniformly bounded. +- **Lemma A3** (p. 25): stochastic equicontinuity `sup_{w∈W} |nu_n(F_n) - nu_n(F_0)|(w) = o_p(1)` for `nu_n(F) := sqrt(n)(phi_n(F) - phi(F))`, along the line of Theorem 2.3 of van der Vaart and Wellner (2007) (empirical processes indexed by estimated functions). +- **Lemma A4** (p. 26): asymptotically linear expansion `sqrt(n)(phi_n(F_n) - phi(F_0)) = nu_n(F_0) + phi'_{F_0}(sqrt(n)(F_n - F_0)) + o_p(1)` uniformly in W. +- **Proof of Proposition 1** (p. 27): `sqrt(n)(F_hat_{Y_t(0)|X=x,D_t=1} - F_{Y_t(0)|X=x,D_t=1}) = r_x^{(0)} G_tilde_{t,x}^{(0)} + kappa_x(G_hat_{t-1,x}^{(0)}, G_hat_{t-1,x}^{(1)}) + o_p(1)` uniformly, where `kappa_x` is the Hadamard derivative of `phi_n` from Lemma A2; concluded via the extended continuous mapping theorem with Lemma 1. +- **Proof of Theorem 2** (p. 27): when `F_hat_{Y_t(j)|X=x,D_t=1}(y)` is weakly increasing in y, the quantile map is Hadamard differentiable; the functional delta method gives `sqrt(n)(F_hat^{-1}_{Y_t(j)|X=x,D_t=1}(tau) - F^{-1}_{Y_t(j)|X=x,D_t=1}(tau)) ⇝ (Z_x^{(j)} / f_{Y_t(j)|X=x,D_t=1}) ∘ F^{-1}_{Y_t(j)|X=x,D_t=1}(tau)`. Quantile inference requires positive density at the relevant quantiles. +- Regularity used throughout (pp. 22, 24): compact supports; densities bounded away from 0 and infinity; `V_tilde := V_1 + G_0^{-1} ∘ H_0(V_2)` has compact support with continuous density bounded away from 0 and infinity. + +Extensions noted (printed p. 11): via Proposition 1 + delta method one could obtain limit processes for other Hadamard-differentiable functionals (Lorenz curve, Gini coefficient). + +**Bootstrap inference (Section 3.3, printed pp. 11-13; proofs printed pp. 27-28):** + +Bootstrap analogues (unnumbered displays, printed p. 12), with exchangeable weights `W_i^{(d)}` satisfying Assumption B: + +``` +F*_{Y_s|X=x,D_t=d}(y) := (1 / n_x^{(d)}) SUM_i W_i^{(d)} 1{Y_is <= y} delta^{(d)}_{i,x}, + +F*_{Y_t(0)|X=x,D_t=1}(y) + := (1 / n_x^{(0)}) SUM_i W_i^{(0)} 1{ DeltaY_it + F*^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F*_{Y_{t-1}|X=x,D_t=0}(Y_{i,t-1}) <= y } delta^{(0)}_{i,x}, + +Delta_hat_x^{QTT*}(tau) := F*^{-1}_{Y_t(1)|X=x,D_t=1}(tau) - F*^{-1}_{Y_t(0)|X=x,D_t=1}(tau), tau ∈ T. +``` + +- The empirical bootstrap is the special case where `(W_1^{(d)}, ..., W_n^{(d)})` is multinomial with probabilities `delta^{(d)}_{i,x} * (1/n_x^{(d)}, ..., 1/n_x^{(d)})` (printed p. 12). The framework (Praestgaard and Wellner 1993) also covers weighted bootstrap, wild bootstrap, and subsampling. Paper's recommendations: empirical bootstrap for moderate samples, weighted bootstrap if estimation is time-consuming, subsampling if the sample is extremely large. +- Validity (Theorem 3, proof printed p. 28): linearized bootstrap process `Z_tilde_x^{(0)*} := r_x^{(0)} G_tilde_{t,x}^{(0)*} + kappa_x(G_hat*_{t-1,x}, G_hat*_{t,x})`; triangle-inequality split over the bounded-Lipschitz class `BL_1` (displays (A12)-(A13)), with (A12) -> 0 in probability via (A14)-(A16), Lemma 2, Lemma A5 (the bootstrap analogue of Lemma A4, displays (A10)-(A11)), and Markov; (A13) -> 0 via Lemma 2 + continuous mapping. Final step: "Theorem 3.9.11 of van der Vaart and Wellner (1996) shows that the functional delta method can apply for Hadamard differentiable maps under resampling. Since the map from distribution to quantile is Hadamard differentiable, the desired result follows." +- **KS test and uniform bands** (printed p. 11): `KS_x := sqrt(n) sup_{tau∈T} |Delta_hat_x^QTT(tau)|`; simultaneous (1-alpha)% confidence bands `(Delta_hat_x^QTT - c_{1-alpha} n^{-1/2}, Delta_hat_x^QTT + c_{1-alpha} n^{-1/2})` with `c_{1-alpha}` the KS critical value, obtained in practice by bootstrap. "The confidence bands are obtained by inverting the Kolmogorov-Smirnov test" (printed p. 16). +- **Iterations used everywhere: 1000** (Monte Carlo rejection frequencies, application KS critical values, pointwise SEs, and Figure 1 bands all use 1000 bootstrap iterations; printed pp. 14, 16). + +**Monte Carlo evidence (Section 4, printed pp. 14-15; Tables 1-2, printed pp. 29-30):** + +DGP: `Y_it(d) = mu(d) + theta_t + v_i + eps_it`, treatment effect constant across all quantiles `= mu(1) - mu(0)` set to 0 or 1 ("TE"); `theta_t = 1` (common time fixed effect); `v_i` time-invariant heterogeneity that may differ across treated/untreated; `eps_it` time-varying unobservables. 1000 Monte Carlo simulations, 1000 bootstrap iterations each, nominal size 5%. + +*DGP 1 (both this paper's model and Athey-Imbens CiC hold):* `v_i | D=d ~ N(d, 1)`; `eps_it` standard normal. CiC is the benchmark, with CiC standard errors from the "empirical block bootstrap" on the same sample (1000 iterations; no software attribution). Table 1 (bias and rejection probabilities at quantiles 0.1/0.5/0.9; DDID = this paper's estimator): + +| | N | DDID 0.1 | DDID 0.5 | DDID 0.9 | CIC 0.1 | CIC 0.5 | CIC 0.9 | +|---|---|---|---|---|---|---|---| +| TE=0 Bias | 100 | 0.044 | 0.045 | 0.081 | 0.012 | -0.097 | -0.295 | +| | 200 | 0.016 | 0.021 | 0.066 | -0.009 | -0.048 | -0.141 | +| | 500 | 0.016 | 0.008 | 0.023 | -0.005 | -0.042 | -0.074 | +| TE=0 Rej. Prob. | 100 | 0.042 | 0.037 | 0.023 | 0.042 | 0.041 | 0.100 | +| | 200 | 0.049 | 0.050 | 0.044 | 0.051 | 0.056 | 0.076 | +| | 500 | 0.043 | 0.047 | 0.034 | 0.035 | 0.043 | 0.069 | +| TE=1 Bias | 100 | 0.059 | 0.064 | 0.109 | 0.251 | -0.051 | -0.293 | +| | 200 | 0.031 | 0.027 | 0.049 | 0.128 | -0.052 | -0.200 | +| | 500 | 0.014 | 0.019 | 0.025 | 0.053 | -0.019 | -0.090 | +| TE=1 Rej. Prob. | 100 | 0.397 | 0.675 | 0.359 | 0.409 | 0.617 | 0.548 | +| | 200 | 0.742 | 0.949 | 0.703 | 0.713 | 0.859 | 0.614 | +| | 500 | 0.994 | 1.000 | 0.992 | 0.983 | 0.992 | 0.756 | + +Paper's reading (printed p. 14): DDID is less biased than CiC in finite samples, "especially at the 0.9th quantile"; with only 100 observations DDID inference is "somewhat undersized, but it exhibits good size properties with 200 or 500 observations"; power increases rapidly from 100 to 200 to 500 and is higher at the median than at the 0.1/0.9 quantiles. Note CiC's TE=0 rejection at the 0.9 quantile is oversized (0.100 at N=100, still 0.069 at N=500). + +*DGP 2 (Copula Invariance violated while Distributional DiD holds):* + +``` +(v_i, eps_i2, eps_i1) | D=d ~ N(0, V_d), V_d = [ 1 rho_dv2 rho_dv1 + rho_dv2 1 rho_d12 + rho_dv1 rho_d12 1 ] +``` + +`(Y_i1(0), DeltaY_i2(0) | D=d)` is bivariate normal with correlation parameter `rho_d2 - rho_d1 + rho_d12 - 1` (consistent with the DGP algebra: Cov(v_i + eps_i1, eps_i2 - eps_i1), writing rho_d1 / rho_d2 for the matrix entries rho_dv1 / rho_dv2); the copula is Gaussian. They set `rho_d1 = 0` and `rho_d12 = 1/2` for both d; then `rho_d2 = d * rho_bar` and vary `rho_bar`. `rho_bar = 0` means Copula Invariance holds; `rho_bar != 0` violates it. N = 200 throughout. Table 2 (bias and RMSE): + +| rho_bar | TE=0 0.1 | TE=0 0.5 | TE=0 0.9 | TE=1 0.1 | TE=1 0.5 | TE=1 0.9 | +|---|---|---|---|---|---|---| +| Bias 0.00 | 0.020 | 0.034 | 0.037 | 0.023 | 0.023 | 0.050 | +| Bias 0.05 | 0.073 | 0.023 | 0.012 | 0.088 | 0.029 | 0.008 | +| Bias 0.10 | 0.121 | 0.028 | -0.033 | 0.112 | 0.019 | -0.032 | +| Bias 0.50 | 0.425 | 0.013 | -0.374 | 0.435 | 0.027 | -0.353 | +| RMSE 0.00 | 0.348 | 0.261 | 0.340 | 0.342 | 0.248 | 0.359 | +| RMSE 0.05 | 0.348 | 0.256 | 0.324 | 0.358 | 0.258 | 0.342 | +| RMSE 0.10 | 0.374 | 0.260 | 0.352 | 0.374 | 0.259 | 0.346 | +| RMSE 0.50 | 0.565 | 0.272 | 0.529 | 0.566 | 0.264 | 0.508 | + +Paper's reading (printed p. 15): small violations (`rho_bar` = 0.05 or 0.10) cause small bias increases - at the 0.1 quantile bias goes 0.020 -> 0.073 -> 0.121; a large violation (`rho_bar` = 0.50) causes much larger tail bias (0.425 at the 0.1 quantile; symmetric negative bias -0.374 at the 0.9 quantile). **The median (0.5 quantile) is almost completely insensitive** to Copula Invariance violations: bias 0.013 even at `rho_bar` = 0.5, RMSE barely changes. Implementation takeaway: tail quantiles are where copula-invariance failure bites; the median is robust. + +**Empirical application (Section 5, printed pp. 15-17; Tables 3-4 / Figure 1, printed pp. 31-33) - context and potential replication target:** + +Effect of Q1 2007 state minimum-wage increases on the earnings distribution. Treated: 5 states raising their minimum wage with close geographic proximity to a federal-minimum state (Arizona, Colorado, Minnesota, Missouri, North Carolina); control: 14 federal-minimum states (Georgia, Idaho, Iowa, Kansas, Kentucky, Nebraska, New Mexico, North Dakota, South Carolina, South Dakota, Tennessee, Utah, Virginia, Wyoming). Data: CPS (Flood et al. 2015), earnings asked in the 4th and 8th interview months - exactly one year apart; IPUMS longitudinal links (Drew et al. 2014); earnings > $10/week; 8,256 linked individuals (2 observations each). Covariates: 8 discrete cells by gender x race (white/non-white) x education (college/non-college); Table 3 motivates conditioning (treated states have higher college share, p = 0.00, and 6.5 log points higher 2006 earnings, p = 0.00 - cross-sectional comparisons would be upward-biased). Per subgroup: KS test of CQTT = 0 at all quantiles, grid tau = 0.05 to 0.95 by 0.01, critical values from 1000 bootstrap iterations, 5% size. Rejections in 3 of 8 subgroups (white female college; non-white male non-college; non-white female non-college) - lower-earning groups, consistent with the minimum wage binding only at the bottom. Table 4 (printed p. 32), CQTT with pointwise bootstrap SEs: + +| Race | Gender | Education | N | Reject H0 | 0.1 | 0.5 | 0.9 | +|---|---|---|---|---|---|---|---| +| White | Male | College | 1617 | | 0.004 (0.048) | 0.007 (0.031) | 0.000 (0.037) | +| White | Male | Non-College | 2306 | | -0.021 (0.068) | -0.019 (0.028) | 0.075 (0.068) | +| White | Female | College | 1629 | Yes | -0.029 (0.056) | 0.027 (0.033) | -0.046 (0.052) | +| White | Female | Non-College | 1980 | | -0.038 (0.054) | -0.043 (0.035) | -0.086 (0.068) | +| Non-White | Male | College | 156 | | -0.492 (0.265) | -0.07 (0.161) | 0.292 (0.268) | +| Non-White | Male | Non-College | 282 | Yes | -0.087 (0.186) | -0.044 (0.095) | 0.036 (0.136) | +| Non-White | Female | College | 209 | | 0.097 (0.172) | 0.033 (0.088) | 0.007 (0.121) | +| Non-White | Female | Non-College | 340 | Yes | -0.25 (0.175) | -0.026 (0.086) | -0.043 (0.162) | + +Significant effects are concentrated in the lower part of the distribution and negative in sign - explained by earnings mixing wages and hours (hours may fall even if wages rise); footnote 1 (printed p. 17) robustness check: the 10th percentile of earnings rose 3.0 log points for treated vs 10.5 for untreated between 2006 and 2007. + +**Relation to other methods:** + +- **Athey-Imbens CiC** (printed p. 2): a distinct identification strategy for the same distributional target. CiC posits a structural monotone production function in a scalar unobservable; this paper keeps the DiD flavor (Distributional DiD on the *change* in untreated potential outcomes) and adds Copula Invariance. The compact-support condition of A4 is borrowed "as in Athey and Imbens (2006)". Melly and Santangelo (2015) extend CiC to covariate-conditional assumptions. In Monte Carlo DGP 1, where both models hold, DDID is less biased in finite samples, especially at the 0.9 quantile. +- **Mean DiD / parallel trends** (printed p. 5): A2 is the exact distributional strengthening of parallel trends. Under A2 alone the ATT is identified but the CQTT is only partially identified. +- **Fan and Yu (2012)** (printed pp. 2, 5): partial identification of the QTT via Frechet-Hoeffding copula bounds under Distributional DiD. Copula Invariance is exactly what upgrades that partial identification to point identification with two periods. +- **Callaway and Li (2015)** (printed p. 2): point-identifies the QTT with at least three periods of panel data by replacing an unknown unconditional copula in the last two periods with an observed copula from the first two periods. This paper needs only two periods, works conditionally on covariates, and recovers the missing dependence cross-sectionally (treated vs untreated group) rather than across time. +- **Broader QTE literature** (printed p. 2): conditional QTE via quantile regression (Koenker-Bassett 1978; Koenker 2005); IV-based conditional QTE (Abadie et al. 2002; Chernozhukov-Hansen 2005, 2006, 2013); unconditional QTE via propensity-score weighting (Firpo 2007); IV after conditioning (Abadie 2003; Frolich-Melly 2013); counterfactual-distribution methods (Firpo et al. 2009; Rothe 2012; Chernozhukov et al. 2013a); nonseparable panel models under time homogeneity (Chernozhukov et al. 2013b); repeated cross sections (D'Haultfoeuille et al. 2015). Note the paper does NOT use the quantile-DiD transform (quantile-by-quantile differencing of observed distributions) anywhere; its counterfactual comes from Theorem 1's copula argument. +- **Positioning** (Conclusion, printed p. 17): "the key innovation is to recover the unknown dependence between the change and initial level of untreated potential outcomes for the treated group from the observed dependence from the untreated group." The authors flag replacing an unknown copula with one observed for another group as a reusable strategy (finance, auction models, duration models). + +**Reference implementation(s):** +- R: `qte` package by Brantly Callaway (first author of this paper) - the initiative's chosen parity target. Per the qte documentation the two-period panel QTT estimator of this paper is `qte::ddid2()`; **verify this function-name mapping against the qte docs/source during implementation** - it is asserted from package-documentation recall, not from the paper. +- The paper itself mentions no software, package, or replication code anywhere (verified across printed pp. 1-33; the References list contains only academic works, and the Monte Carlo's CiC benchmark carries no software attribution). + +**Requirements checklist (for the future scoped PR, not v1):** +- [ ] Verify `qte::ddid2()` is the function implementing this paper's two-period panel QTT (qte docs/source), then build golden parity fixtures against it +- [ ] Reconcile all equation/assumption numbers against the published JoE 206(2) version of record +- [ ] Resolve the Corollary 1 conditioning flag and the (A4) mapping-direction discrepancy (see Gaps) before writing any repeated-cross-section code +- [ ] Empirical CDFs per eq. (2); plug-in counterfactual per eq. (3); inf-based quantile inversion (no interpolating quantile types) +- [ ] Exchangeable bootstrap with weights satisfying Assumption B; empirical bootstrap as default; 1000 iterations as the paper's convention +- [ ] KS statistic `KS_x` and uniform bands by inverting the KS test; tau grid restricted to a compact `T` strictly inside (0,1) +- [ ] Discrete-covariate cells: common support of X across groups; warn on tiny cells (application shows cells as small as N = 156; A6(b) requires proportional growth) +- [ ] Pre-treatment placebo hook: with extra pre-periods, run the estimator on pre-periods and test `Delta_x^QTT(tau) = 0` for all tau (the paper's suggested A2/A3 diagnostic) +- [ ] Continuity/ties check (A4): warn on heavy ties or atoms; copulas non-unique and Hadamard machinery fails on degenerate distributions +- [ ] Document tail-quantile sensitivity to Copula Invariance violations (DGP 2: 0.1-quantile bias 0.020 -> 0.425 as rho_bar goes 0 -> 0.5) and median robustness +- [ ] Panel mode operates on `(DeltaY_it, Y_{i,t-1})` pairs; repeated cross-section mode requires the extra rank-invariance assumption and time-invariant covariates, with a distinct warning + +--- + +## Implementation Notes + +**Relevance to diff-diff CiC/QDiD v1 (2026-07-12):** +- (i) The exchangeable/empirical bootstrap machinery (Assumption B, functional delta method, Theorem 3 via van der Vaart-Wellner Theorems 3.6.13 and 3.9.11) is the closest published validity argument for the bootstrap inference diff-diff v1 ships with CiC/QDiD-style plug-in distributional estimators - Athey-Imbens (2006) itself contains no bootstrap theory at all. +- (ii) The Copula Invariance assumption clarifies exactly what panel data adds beyond repeated cross-sections for QTT: the observed within-unit pairing of (change, initial level) among untreated units. CiC itself does not need it - Athey-Imbens Section 3.4 requires only equal within-group marginals of the unobservable over time for panel data. +- (iii) The Monte Carlo DDID-vs-CiC comparison (Table 1) informs our docs on when to recommend which estimator: when both models hold, DDID is less biased in the tails (especially the 0.9 quantile) and CiC's upper-tail test over-rejects; DGP 2 shows where DDID itself breaks (tail bias under copula-invariance violations, median robust). +- (iv) The paper's own comparison of its assumptions vs Athey-Imbens CiC (distributional parallel trends + copula invariance on observables' building blocks, vs a monotone structural function in a scalar unobservable with time-invariant within-group distribution) belongs in the choosing-an-estimator docs. + +### Data Structure Requirements +- Two-period panel: one pair `(Y_{i,t-1}, Y_it)` per unit plus covariates and a period-t treatment indicator; nobody treated in the first period. The estimator consumes `(DeltaY_it, Y_{i,t-1})` for untreated units and `Y_{i,t-1}`, `Y_it` marginals for treated units, all within covariate cells. +- Repeated cross sections: supported only via Corollary 1, at the cost of conditional rank invariance over time and time-invariant covariates. +- Covariates must be discrete (cells); the theory is fully nonparametric within cells with no curse of dimensionality (printed pp. 3, 8). Both groups must be represented in every cell used (X is the common support), and both cell sizes must grow proportionally with n (A6(b)). +- Continuous outcomes (A4): ties/atoms break copula uniqueness and the Hadamard-differentiability framework. + +### Computational Considerations +- Pure ECDF plug-in: sorting the per-cell arrays dominates, O(n log n); the quantile-quantile transform `F_hat^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_hat_{Y_{t-1}|X=x,D_t=0}` evaluates via two searchsorted passes over sorted arrays. No kernels, no bandwidths, no density estimation for point estimates or bootstrap inference. +- The copula is never estimated explicitly - Theorem 1's formula handles it implicitly through the observed pairing of each untreated unit's own change and initial level. Any implementation that separately estimates a copula object is over-engineering the estimator. +- Bootstrap multiplies cost by B (paper convention B = 1000); replicates are embarrassingly parallel. The exchangeable-weights formulation means the empirical bootstrap can be implemented as within-cell multinomial weights rather than physical resampling. +- Densities (`f_{Y_t(j)|X=x,D_t=1}`, the omega_x kernel) appear ONLY in the asymptotic limit expressions, not in the estimator or the bootstrap - inference never requires estimating them. This is a practical advantage over the Athey-Imbens analytical-SE path, which needs a boundary-consistent density estimator. +- The Proposition 1 limit is not nuisance-parameter free (the kappa_x first-step term), which is exactly why the paper routes inference through the bootstrap rather than plug-in analytical SEs. + +### Tuning Parameters + +| Parameter | Type | Default | Selection Method | +|-----------|------|---------|-----------------| +| `n_bootstrap` | int | 1000 (paper's convention in both simulation and application) | SE/band stability; align with qte for parity tests | +| bootstrap weight scheme | {empirical, weighted, wild, subsampling} | empirical (paper recommendation for moderate samples) | weighted if estimation is time-consuming; subsampling if the sample is extremely large (printed p. 12) | +| `quantiles` (tau grid) | floats in compact `T ⊂ (0,1)` | 0.05 to 0.95 by 0.01 (paper's application grid, printed p. 16) | T must be compact and strictly inside (0,1) (printed p. 9); no extreme quantiles | +| covariate cells | discrete values of X | all common-support cells | both groups present per cell; warn on tiny cells (A6(b)) | + +No smoothing parameters exist anywhere in the method. + +### Relation to Existing diff-diff Estimators +- `ChangesInChanges` / `QDiD` (v1): same 2x2-flavored distributional target (counterfactual `F_{Y_t(0)}` for the treated), different identification. CiC assumes a monotone structural function and within-group time-invariance of a scalar unobservable; this paper assumes Distributional DiD (A2) + Copula Invariance (A3) directly on the building blocks of the observed outcomes. Neither nests the other (A2/A3 are non-nested with the CiC model; DGP 1 is a case where both hold). The choosing-an-estimator docs should carry the Table 1 and Table 2 evidence. +- `DifferenceInDifferences`: mean DiD's parallel trends is the mean implication of A2; under A2 alone the ATT is identified (so the ATT from this method should match mean DiD asymptotically when A2 holds) while the CQTT needs A3 on top. +- `CallawaySantAnna`: same first author; the CS (2021) group-time ATT framework is the mean-DiD multi-period sibling. Callaway and Li (2015) is the three-period panel QTT sibling of this paper (unconditional copula recovered across time instead of across groups). +- Bootstrap utilities: the exchangeable-bootstrap validity argument here (functional delta method + Hadamard differentiability of the ECDF-composition and quantile maps) is the citation to use when documenting bootstrap inference for the v1 distributional estimators; reuse `safe_inference()` joint-NaN conventions from `diff_diff.utils` for any future implementation. + +--- + +## Gaps and Uncertainties + +**Contradictions between extraction passes (both versions preserved; do NOT resolve by guessing - check the arXiv PDF and the published JoE version):** + +1. **Corollary 1 rank-invariance conditioning (printed pp. 7 and 22).** The Corollary 1 *statement* (printed p. 7) conditions the rank-invariance copula on the treated group: `C_{Y_{t-1}(0),Y_t(0)|X=x,D_t=1}(u, v) = min{u, v}`. But the identification formula recovers the imputed change `DeltaY_tilde_it` from *untreated* observables, and the appendix *proof* (printed p. 22) applies rank invariance to untreated units: `F_{Y_t(0)|X=x,D_t=0}(Y_it(0)) = F_{Y_{t-1}(0)|X=x,D_t=0}(Y_{i,t-1}(0))`, i.e. it is used to "identify `DeltaY_it(0)` for individuals with `X_i = x` and `D_it = 0`". Either the statement's `D_t = 1` is a typo for `D_t = 0`, or the proof invokes an unstated symmetric condition. Resolve against the published JoE version before implementing the repeated-cross-section path. + +2. **Direction of the quantile-quantile transform in the generic appendix map (printed pp. 7-8 vs p. 24).** The main-text extraction (Theorem 1, printed p. 7; eq. (3), printed p. 8) has the transform `F^{-1}_{Y_{t-1}|X=x,D_t=1} ∘ F_{Y_{t-1}|X=x,D_t=0}` applied to *untreated* units' initial levels (transporting untreated initial levels onto the treated scale) - internally consistent with eq. (3) weighting by `delta^{(0)}_{i,x}`. The appendix extraction's prose mapping of display (A4) (printed p. 24) instead describes `G^{-1} ∘ H` as `F^{-1}_{Y_{t-1}|X=x,D_t=0} ∘ F_{Y_{t-1}|X=x,D_t=1}`, "transporting treated-group initial levels onto the untreated scale". The generic display (A4) itself is not in question; the conflict is only in which paper object G and which H denotes. This review's main body follows the main-text version throughout (it is the self-consistent one), but verify the appendix mapping against printed p. 24 before reusing the Lemma A1/A2 derivative formulas with concrete distributions substituted in. + +**Verified quirks (not contradictions, but easy to trip over):** + +3. **No Assumption A5 exists.** The main text has A1, A2, A3, A4, A6, B; "A5" appears only as appendix equation (A5) and appendix Lemma A5. Citations reading "Assumption A1-A6" (Lemma 1, Proposition 1, Theorem 2, Lemma 2, Theorem 3) mean A1-A4 plus A6. Verified by full-text search during extraction. +4. **`F_tilde` notation on printed p. 9.** The v1 text describes the infeasible `F_tilde_{Y_t(0)|X=x,D_t=1}` as "based on observations {Y_tilde_it} with X_i = x and D_it = 1 as in (2)", but the feasible eq. (3) and its bootstrap analogue weight by `delta^{(0)}_{i,x}` (untreated observations). The `D_t = 1` subscript names the distribution being *estimated*, not the observations used. +5. **DGP 2 "correlation parameter"** `rho_d2 - rho_d1 + rho_d12 - 1` was transcribed identically by both extraction passes; it is the *covariance* Cov(v_i + eps_i1, eps_i2 - eps_i1) of `(Y_i1(0), DeltaY_i2(0))` under the stated V_d, and the text's rho_d1/rho_d2 correspond to the matrix entries rho_dv1/rho_dv2. Consistent, but the naming is loose - transcribed as printed. + +**Version and coverage gaps:** + +6. **arXiv v1 vs published version of record.** All equation, theorem, and assumption numbers above are pinned to arXiv:1702.03618v1. The published JoE 206(2) version may renumber or reword (in particular the Corollary 1 conditioning in item 1); reconcile before implementing anything sensitive. +7. **Unconditional QTT aggregation is informal.** The paper identifies the unconditional QTT by averaging the conditional counterfactual over covariates and inverting (printed p. 7) but then "focus[es] only on the CQTT" - there is no formal aggregation estimator or inference theory in the paper. Any aggregated-QTT implementation needs its own delta-method argument (or the qte package's approach, whichever it uses - check during implementation). +8. **Tables sit after the References** (printed pp. 29-33, References pp. 18-21). The in-text quantitative claims (printed pp. 14-15) were cross-checked against the table transcriptions during synthesis and are consistent. +9. **No reference implementation named by the paper.** The `qte::ddid2()` mapping comes from qte package documentation recall, not from the paper; verify against qte docs/source. The Monte Carlo's CiC benchmark ("empirical block bootstrap") carries no software attribution either. +10. **Proof details relied on but not independently checked:** Lemma A3's displays (A5)-(A9) (equicontinuity envelopes), Lemma A5's (A10)-(A11), and Theorem 3's (A12)-(A16) were extracted as structure only; the epsilon-delta detail was not verified line-by-line. Lemma 2's proof is explicitly abbreviated in the paper ("follows from Theorem 3.6.13 of van der Vaart and Wellner (1996)"). diff --git a/docs/methodology/papers/ciaccio-2024-review.md b/docs/methodology/papers/ciaccio-2024-review.md new file mode 100644 index 000000000..3720cbf59 --- /dev/null +++ b/docs/methodology/papers/ciaccio-2024-review.md @@ -0,0 +1,619 @@ +# Paper Review: Distributional Difference-in-Differences Models with Multiple Time Periods + +**Author:** Andrea Ciaccio +**Citation:** Ciaccio, A. (2024). Distributional Difference-in-Differences Models with Multiple Time Periods. *arXiv preprint* arXiv:2408.01208. +**PDF reviewed:** **arXiv:2408.01208v2** (https://arxiv.org/abs/2408.01208v2, v2 submitted 21 May 2025; v1 was 2 Aug 2024; 44 PDF pages, printed page = PDF page; main text pp. 1-32, References pp. 33-36, Appendix A proofs pp. 37-40, Appendix B repeated cross sections pp. 41-42, Appendix C additional simulations pp. 43-44). Per the project's PDFs-never-committed convention the local PDF is kept outside the repository (gitignored `papers/ciaccio-2408.01208v2.pdf`). This is an unpublished preprint as of the review date (2026-07-12) - check for a published version (which may renumber results) before citing it in shipped docs. All numbers below are pinned to arXiv v2. +**Review date:** 2026-07-12 + +--- + +## Methodology Registry Entry + +*Drafted in docs/methodology/REGISTRY.md format for FUTURE use. Do not copy this section into the registry now - see the status line below.* + +## Staggered Distributional DiD (Ciaccio) + +**Status: NOT shipping in diff-diff CiC/QDiD v1 (scope decision 2026-07-12).** The staggered/multi-period distributional extension is reviewed-but-deferred; this review un-parks the ROADMAP's "Distributional DiD for staggered timing (Ciaccio)" row from "commit when a user reports need" to "reviewed, implementation deferred pending demand". Any future implementation would be a separate scoped PR. + +**Primary source:** Ciaccio, A. (2024). Distributional Difference-in-Differences Models with Multiple Time Periods. arXiv:2408.01208 (reviewed at v2, 21 May 2025). +- Preprint-status note (p. 1 footnote, quoted as printed): "This is a preliminary version of the paper where most of the comments and feedback received still need to be addressed. All the Monte Carlo simulations were run in STATA. The ado files of the command used for implementing the methodology presented in the paper, `qtt`, are available upon request at the time of writing." +- **This paper version has NO empirical application - simulations only.** Section 4 (Monte Carlo, pp. 23-30) is followed directly by Section 5 (Discussion, pp. 31-32) and Section 6 (Conclusion, p. 32); References begin on p. 33. The section numbering 2 -> 3 -> 4 -> 5 -> 6 leaves no room for an application anywhere in the paper. +- The paper is the staggered/multi-period generalization of Callaway and Li (2019) (2-period/2-group panel QTT), using the copula-invariance-across-groups device of Callaway, Li, and Oka (2018) and the group(cohort)-time parameter + aggregation architecture of Callaway and Sant'Anna (2021). + +**Model (Section 2.1, pp. 6-8):** + +- `T` available periods, `t = 1, ..., T`; classic 2x2 DiD is `T = 2` with nobody treated in `t = 1`; this paper allows `T > 2`. `D_{i,t}` is a binary indicator = 1 if unit `i` (of `i = 1, ..., N`) is treated in period `t`. `q >= 2` is the first period the policy is implemented; staggered rollout starts at `q`. +- **Cohorts** (p. 7): assuming a never-treated group exists, define `T - q + 2` mutually exclusive cohort dummies `d_r` (`r = q, ..., T`) denoting the period in which unit `i` FIRST receives treatment. `C` is a dummy = 1 for never-treated groups. `d-bar = max_i d_{i,r}` is the maximum period where a unit may become treated. Always-treated units are dropped (footnote 3, p. 7: "there is no pre-treatment period for these units"). +- **Comparison group**: main results use the NEVER-TREATED group; results "can be generalized to cases where not-yet-treated units serve as the comparison group, as they follow from symmetric arguments (Callaway and Sant'Anna, 2021)" (p. 7). Trade-off (pp. 5, 7): not-yet-treated expands the pool of valid comparisons and can improve inference, but if later-treated units adjust behavior in response to the policy's introduction, never-treated is the more reliable counterfactual (footnote 5 cites Ciaccio (2023) for an empirical application). +- **Covariates** (p. 7): a complete set of PRE-TREATMENT covariates `X` with support `chi = supp(X) subset R^K`, `dim(X) = k`. +- **Generalized propensity score** (p. 7): `P_{r,t}(X) = P(d_r = 1 | X, d_r + C = 1)` - the probability of being first treated in period `r`, conditional on covariates and on being either in cohort `r` or never-treated (pairwise cohort-vs-never-treated conditioning, CS-2021 style, NOT multinomial across cohorts). Footnote 6 (p. 7) gives the not-yet-treated version `P_{r,s}(X) = P(d_r = 1 | X, d_r + (1 - D_s)(1 - d_r) = 1)`. +- **Potential outcomes** (p. 7): dynamic potential outcomes (Robins 1986, 1987) combined with dynamic treatment adoption (Heckman et al., 2016). `Y_{i,t}(r)` = outcome at `t` had the policy been introduced by period `r`; `Y_{i,t}(0)` = outcome had treatment never been received. Observed outcome (Eq. (1), p. 7): + +``` +Y_{i,t} = Y_{i,t}(0) + sum_{r=q}^{T} d_{i,r} * (Y_{i,t}(r) - Y_{i,t}(0)) (1) +``` + +- Observation rule (pp. 7-8): never-treated units reveal `Y(0)` for all `t`; units first treated in `r` reveal `Y(0)` for `t < r` and `Y(r)` for `t >= r`. +- Quantile notation (p. 8): `q_tau = F_T^{-1}(tau) := inf{ t : F_T(t) >= tau }` (inf-based, weak inequality). Long difference (p. 16): `Delta_{[r-rho-1, t]} Y(0) = Y_t(0) - Y_{r-rho-1}(0)`. Short difference (p. 12): `Delta Y_t = Y_t - Y_{t-1}`. + +**Target parameters (Sections 2.2 and 2.4, pp. 10-11, 18-20):** + +Cohort-time distributional/quantile treatment effect on the treated (name borrowed from Callaway and Sant'Anna 2021's "group-time" parameters, p. 11). Eq. (2), p. 10: + +``` +QTT_{r,t}(tau) = F^{-1}_{Y_t(r)|d_r=1}(tau) - F^{-1}_{Y_t(0)|d_r=1}(tau), + tau in [0,1], r = q, ..., T, and t >= r (2) +``` + +`F_{Y_t(r)|d_r=1}` is directly identified from treated outcomes of cohort `r` at `t >= r`; the whole identification problem is the counterfactual `F_{Y_t(0)|d_r=1}` (p. 11). Heterogeneity is allowed across `tau`, across cohorts `r`, and over time `t`. + +Post-identification version with anticipation (Section 2.4.1, p. 18): + +``` +QTT_{r,t,rho}(tau) = F^{-1}_{Y_t(r)|d_r=1}(tau) - F^{-1}_{Y_t(inf)|d_r=1}(tau) + for all r, t in {q, ..., T}, t >= r - rho +``` + +(As printed on p. 18 the subtrahend is typeset with what appears to be `Y_t(infinity)` - the never-treated potential outcome, used interchangeably with `Y_t(0)` elsewhere; the p. 19 restatement uses `Y_t(0)`. See Gaps and Uncertainties.) Conditional version (p. 18): `QTT_{r,t,rho}(tau; x) = F^{-1}_{Y_t(r)|X,d_r=1}(tau|x) - F^{-1}_{Y_t(0)|X,d_r=1}(tau|x)` for all `r, t in {q, ..., T}` and `t >= r - rho`. Rank invariance is required to interpret the QTT as the quantile treatment effect for specific units. + +Stochastic-dominance parameters (Sections 2.2 and 2.4.2, pp. 8-11, 20): for each post-treatment `t` and cohort `r`, test whether `Y_t(r)|d_r=1` FSD or SSD `Y_t(0)|d_r=1`. These avoid the rank-invariance assumption implicit in quantile-by-quantile QTT comparisons (Maasoumi and Wang 2019 show data often reject rank invariance). With `U_1` the class of increasing von Neumann-Morgenstern utilities and `U_2` its concave subclass, FSD holds iff any of (pp. 9-10, restated p. 20): (1) `E u(Y_t(r)|d_r=1) >= E u(Y_t(0)|d_r=1)` for all `u in U_1`, strict for some; (2) `F_{Y_t(r)|d_r=1}(y) <= F_{Y_t(0)|d_r=1}(y)` for all `Y`, strict for some values; (3) quantile-by-quantile dominance at all points of the support. SSD analogously with `U_2` / integrated CDFs / integrated quantiles; FSD implies SSD (p. 10). + +ATT as by-product: footnote 12 (p. 13) - once the counterfactual distribution is estimated, `E(Y(0)|d_r=1) = int_0^1 F^{-1}_{Y(0)|d_r=1}(tau) d tau`, so the ATT is straightforwardly retrieved. Footnote 14 (p. 16): under Assumptions 1-4 alone the ATT is identified in the staggered multi-period context. + +**Assumptions (exact statements):** + +*Assumption 1 (Irreversibility of Treatment), p. 7:* + +``` +D_j = 0 for all j = 1, ..., (q-1) almost surely (a.s.). +For t = q, ..., T, D_{t-1} = 1 implies that D_t = 1 a.s. +``` + +No unit is treated before `q`; once treated, always treated ("staggered treatment adoption"). Interpreted, following Callaway and Sant'Anna (2021) and Sun and Abraham (2021), as units changing behavior "forever" once treated. Assumption 1 automatically defines each unit's cohort. + +*Assumption 2 (Random Sampling), p. 8:* + +``` +{Y_{i,1}, Y_{i,2}, ..., Y_{i,t}, X_i, D_{i,1}, D_{i,2}, ..., D_{i,tau}}_{i=1}^{n} is independent and identically distributed. +``` + +(Subscripts as printed; the last treatment subscript is typeset `tau` in the paper.) Requires panel data; Appendix B extends to repeated cross sections. Neither rules out time-series dependence nor restricts the relation between `D_{i,t}` and `(Y(0), Y(r))`. iid sampling rules out interference; together with treatment consistency this constitutes SUTVA (p. 8). + +*Assumption 3 (Limited Treatment Anticipation), p. 11:* + +``` +There is a known rho >= 0 such that +P{Y_t(r) <= y | X, d_r = 1} = P{Y_t(0) <= y | X, d_r = 1} a.s. +for all r = q, ..., T and t = 1, ..., T such that t < r - rho. +``` + +Extension from the mean to the ENTIRE DISTRIBUTION of the Limited Treatment Anticipation assumption of Callaway and Sant'Anna (2021). `rho = 0` reduces to No Anticipation. Implies the distributional treatment effect is 0 for all `t < r - rho`. + +*Assumption 4 (Conditional Distributional Parallel Trends based on a "Never-treated" Group), p. 12:* + +``` +Let rho be defined as in Assumption 3. For each r, t in {q, ..., T} such that t >= q - rho, +P(Delta Y_t(0) <= Delta y | X, d_r = 1) = P(Delta Y_t(0) <= Delta y | X, C = 1) a.s. +where Delta Y_t = Y_t - Y_{t-1}. +``` + +(The time restriction is printed as `t >= q - rho` - see Gaps and Uncertainties.) In the absence of treatment, the counterfactual distribution for cohort `r` would have evolved in parallel to the never-treated units. Extends common trends from the average to the entire distribution (cites Fan and Yu 2012; Callaway and Li 2019; Miller 2023). Weaker than selection-on-observables (Firpo 2007) - unobservable confounders may differ between treated and untreated cohorts - and weaker than Assumptions 1-2 of Bonhomme and Sauder (2011) (p. 12). Testable under strict stationarity of the changes in untreated potential outcomes if the pre-treatment period is long (Callaway et al. 2018) (p. 13). Footnote 10 (p. 12): if it fails, the bias varies across quantiles. + +*Assumption 5 (Conditional Copula Invariance based on a "Never-treated" Group), p. 13:* + +``` +For all x in chi and for all (u, v) in [0,1]^2 +C_{Delta Y_t(0), Y_{t-1}(0) | X, d_r = 1}(u, v) = C_{Delta Y_t(0), Y_{t-1}(0) | X, C = 1}(u, v) +``` + +Background (p. 13): Assumption 4 alone is NOT sufficient to point-identify `F_{Y_t(0)|d_r=1}` (Fan and Yu 2012 - partial identification only). The missing object is the dependence (copula) between `Y_{t-1}(0)` and `Delta Y_t(0)`; via Sklar's theorem, `F_{T,W} = C_{T,W}(F_T(t), F_W(w))`. Assumption 5 replaces the UNKNOWN copula for the treated cohort with the OBSERVED dependence of the never-treated group. Key contrasts (pp. 13-14): +- **vs Callaway and Li (2019) copula STABILITY**: CL19 instead impose that the copula between `Y_{t-1}(0)` and `Delta Y_t(0)` is stable ACROSS TIME, which requires panel data with at least two pre-treatment periods. Copula INVARIANCE (across GROUPS, following Callaway et al. 2018) needs neither, and permits repeated cross sections (Appendix B). +- Assumption 5 imposes NO restriction on marginal distributions - initial distributions can differ between treated and never-treated groups; only the dependence is restricted. +- Assumption 4 does not imply Assumption 5, nor the reverse (p. 14; noted by Callaway et al. 2018 and Callaway and Li 2019). Joint-normal illustration (p. 14): if `corr(Y_{t-1}(0), Delta Y_t(0)) = rho_t`, Assumption 5 requires `rho_t` be independent of treatment assignment, conditional on X and on being in cohort r or never-treated; means/variances are unrestricted. +- No formal test exists; Callaway and Li (2019) suggest rank-correlation measures (Kendall's tau) in pre-treatment periods to assess plausibility (p. 14). + +*Assumption 6 (Continuity), p. 15:* + +``` +The random variables Y_{t-1}(0) and Delta Y_t(0) have continuous distribution conditional on either +being part of the treated cohort r (i.e., d_r = 1) or being never-treated (i.e., C = 1), and +Y_t(r)|d_r = 1 is also continuously distributed on its support. Moreover, each of these distributions +has a compact support with marginal distributions, which are (uniformly) bounded away from 0 and 1 +over their respective support. +``` + +Needed because the copula representation is not unique unless the random variables are continuous (Joe 1997; Nelsen 2006); follows Callaway et al. (2018) and Callaway and Li (2019). + +*Assumption 7 (Overlap), p. 15:* + +``` +For each r, t in {q, ..., T}, there exists some epsilon > 0 s.t. +P(d_r = 1) > epsilon and P_{r,t}(X) < 1 - epsilon a.s. +``` + +Required ONLY for the covariate-conditional results (Propositions 1-2). Generalizes overlap in Firpo (2007), Callaway et al. (2018), Callaway and Li (2019) to multiple periods/groups. + +**Identification (Theorem 1, p. 15):** + +``` +Theorem 1. Suppose Assumptions 1, 2, 6, and unconditional version of Assumptions 3-5 hold. +Then F_{Y_t(0)|d_r=1}(tau) is identified: + +F_{Y_t(0)|d_r=1} = P(Y_t(0) <= y | d_r = 1) + = E[ 1( Delta_{[r-rho-1, t]} Y(0) <= y + - F^{-1}_{Y_{r-rho-1}(0)|d_r=1} ( F_{Y_{r-rho-1}(0)|C=1} ( Y_{r-rho-1}(0) ) ) ) | C = 1 ] + +where Delta_{[r-rho-1, t]} Y(0) = Y_t(0) - Y_{r-rho-1}(0) is the long difference. +``` + +Reading: the expectation runs over never-treated units (`C = 1`). For each never-treated unit, take its long difference `Y_t(0) - Y_{r-rho-1}(0)` and its base-period rank `F_{Y_{r-rho-1}(0)|C=1}(Y_{r-rho-1}(0))`; map that rank into the TREATED cohort's observed base-period marginal via `F^{-1}_{Y_{r-rho-1}(0)|d_r=1}(.)`. This quantile-quantile transform is where the copula-invariance dependence structure of the never-treated is transplanted onto the treated cohort's marginals. The base period is `r - rho - 1`, the last period unaffected by anticipation. + +Mechanism (p. 16): `P(Y_t(0) <= y | d_r = 1) = E[ 1( Delta_{[r-rho-1,t]} Y(0) + Y_{r-rho-1}(0) <= y ) | d_r = 1 ]` is an integral over the JOINT distribution of the pre-treatment level and the change in the untreated potential outcome. Assumption 5 identifies this joint by replacing the unknown treated copula with the observed never-treated one; since `Delta_{[r-rho-1,t]} Y(0)` is not observed for treated cohort `r`, Assumption 4 replaces `F^{-1}_{Delta_{[r-rho-1,t]} Y(0)|d_r=1}(.)` with `F^{-1}_{Delta_{[r-rho-1,t]} Y(0)|C=1}(.)`. The copula is used IMPLICITLY - it is never estimated; the formula involves only marginal CDFs/quantile functions and the observed joint behavior of never-treated units. Once `F_{Y_t(0)|d_r=1}` is identified, so is its inverse (p. 16). Full proof in Appendix A (below). + +*Example 1 (p. 16)* - TWFE DGP satisfying the assumptions: `Y_it(0) = alpha_t + eta_i + u_it`, `rho = 0`, panel data. For the CS-2021 ATT it suffices that `E(Delta u_it | d_r = 1) = E(Delta u_it | C = 1)` (mean version of Assumption 4). For this paper's method, two sufficient conditions: (i) `Delta u_it` independent of `D`, (ii) `C_{Delta u_t, u_{t-1} | d_r = 1} = C_{Delta u_t, u_{t-1} | C = 1}`. These allow the time-varying shock's distribution to vary over time (serial correlation allowed) and `u_it` correlated with `eta_i`, in contrast to Bonhomme and Sauder (2011). Only the UNTREATED potential outcome's generation is restricted; nothing is said about the treated potential outcome (pp. 16-17). + +**Covariate scenarios (p. 17):** it is "highly unlikely" Assumptions 3-5 hold unconditionally. Three identification scenarios (cf. p. 5): unconditional (Theorem 1); (i) Assumptions 3-4 conditional on covariates, Assumption 5 unconditional -> Proposition 1 (IPW); (ii) Assumptions 3-5 all conditional -> Proposition 2. + +**Proposition 1 (conditional PT, unconditional copula invariance; IPW), p. 17:** + +``` +Proposition 1. Under Assumptions 1-4, 6, 7, and unconditional version of Assumption 5 hold, +F_{Y_t(0)|d_r=1}(tau) is identified: + +F_{Y_t(0)|d_r=1} = E[ 1[ F^{p,-1}_{(Y_t(0) - Y_{r-rho-1}(0))|d_r=1} ( F_{(Y_t(0) - Y_{r-rho-1}(0))|C=1} ( Y_t(0) - Y_{r-rho-1}(0) ) ) + <= y - F^{-1}_{Y_{r-rho-1}(0)|d_r=1} ( F_{Y_{r-rho-1}(0)|C=1} ( Y_{r-rho-1}(0) ) ) ] | C = 1 ] + +where + +F^{p}_{Delta_{[r-rho-1,t]} Y(0)|d_r=1}(delta) + = E[ (C / p_r) * ( p_r(x) / (1 - p_r(x)) ) * 1{ Y_t - Y_{r-rho-1} <= delta } ] (3) + +which is identified. +``` + +Notation as printed: `p_r(x)` is the generalized propensity score (the `P_{r,t}(X)` object) and `p_r = p(d_r = 1)` its unconditional cohort probability normalizer; `C` is the never-treated dummy, so the expectation runs over never-treated units reweighted by the odds `p_r(x)/(1 - p_r(x))` - the implementable IPW weight on never-treated units is `C * p_r(X) / (p_r * (1 - p_r(X)))` (Abadie 2005 / CS-2021 style reweighting of the never-treated change distribution to match cohort r's covariate distribution; the Appendix A proof makes this explicit). The superscript `p` marks the propensity-reweighted distribution of the long difference; `F^{p,-1}` its inverse. The only part of Theorem 1 needing modification is the identification of `F_{Delta Y(0)|d_r=1}`; the rest stays valid because Assumption 5 holds unconditionally. Generalizes Firpo (2007)'s reweighting (as readapted by Callaway and Li 2019) to staggered adoption; "almost identical to the one proposed by Abadie (2005) and Firpo (2007) with some minor changes" (p. 17). Requires a parametric specification of `P_{r,t}(X)` (no guarantee it is correct). Footnote 15 (p. 17): a doubly-robust estimator of `F_{Y(0)|d_r=1}` (as in Miller 2023) would relax reliance on correct parametric specification but is beyond the paper's scope. + +*Example 2 (p. 18)* - conditional-validity DGP: `Y_it(0) = alpha_t + eta_i + X'_it beta + u_it`, `X` possibly distributed differently between treated and never-treated units; `u_it = rho u_{i,t-1} + epsilon_it` with `epsilon ~ WNN process`, `|rho| < 1`. Sufficient conditions: (i) `Delta u_it` independent of `D | X`, (ii) `C_{Delta u_t, u_{t-1}|d_r=1} = C_{Delta u_t, u_{t-1}|C=1}`. + +**Proposition 2 (both assumptions conditional on covariates), p. 18:** + +``` +Proposition 2. Suppose that the random variables Y_{t-1}(0) and Delta Y_t(0) are continuously +distributed conditionally on x, and that this is true for all x in chi and both group r or the +never-treated units. Then, under Assumptions 1-7, F_{Y_t(0)|d_r=1}(tau) is identified: + +P(Y_t(0) <= y | X = x, d_r = 1) + = E[ 1[ F^{-1}_{Delta_{[r-rho-1,t]} Y(0)|X,C=1} ( F_{Delta_{r-rho-1,t} Y(0)|X,C=1} ( Delta_{[r-rho-1,t]} Y(0) ) | X ) + <= y - F^{-1}_{Y_{r-rho-1}(0)|X,d_r=1} ( F_{Y_{r-rho-1}(0)|X,C=1} ( Y_{r-rho-1}(0) ) | X ) ] | X = x, C = 1 ] + +and + +P(Y_t(0) <= y | d_r = 1) = int_{chi} P(Y_t(0) <= y | X = x, d_r = 1) dF_{X|d_r=1}(x) +``` + +(Transcription note: in the first inner composed term both the outer inverse CDF and the inner CDF of the long difference are printed with conditioning `|X, C=1`; under conditional Assumption 4 the `d_r=1` and `C=1` conditional distributions of the long difference coincide, so the composition is written entirely with never-treated conditionals.) The only difference vs Theorem 1: compute CONDITIONAL distributions first, then integrate covariates out against `dF_{X|d_r=1}` to obtain the unconditional counterfactual (p. 18). Computational caveat (p. 5): estimating the QTT under Conditional Copula Invariance requires estimating FIVE conditional distributions (as noted in Callaway and Li 2019), often infeasible; the practical implementation (following Callaway et al. 2018) assumes DISCRETE covariates and computes the conditional counterfactual for each covariate value. Note also that Callaway et al. (2018)'s conditional copula invariance results pertain to the CONDITIONAL QTT; this paper additionally delivers the unconditional QTT by integrating out X (p. 5). + +**Aggregation (Section 2.4.1, pp. 19-20; following Callaway and Sant'Anna 2021):** + +Generic weighted form, Eq. (4), p. 19: + +``` +theta(tau) = sum_{t=q}^{T} sum_{r=q}^{T} w(r,t) * QTT_{r,t,rho}(tau), r in {q, ..., T} (4) +``` + +with researcher-chosen weighting functions `w(r,t)` answering, e.g.: how the QTT varies across groups; with length of exposure; cumulative distributional effect across all groups until `t`; overall impact. + +*Event-study (exposure) aggregation* (p. 19), with event time `e = t - r`: + +``` +theta^{e}_{exp}(tau) = sum_{r=q}^{T} 1{r + e <= T} P(d_r = 1 | r + e <= T) QTT_{r, r+e, rho}(tau), + r in {q, ..., T} +``` + +`theta^e_exp(tau)` = effect, on units at the tau-th quantile of the outcome distribution, of being exposed to treatment for `e` periods; computed from all units exactly `e` periods past initial treatment (dynamic event-study analog, cf. Sun and Abraham 2021; de Chaisemartin and d'Haultfoeuille 2022). + +*Overall aggregation* (p. 20): + +``` +theta^{o}_{weight}(tau) = (1/kappa) sum_{t=q}^{T} sum_{r=q}^{T} 1{t >= r} P(d_r = 1 | r <= T) QTT_{r,t,rho}(tau), + r in {q, ..., T} +where kappa = sum_{t=q}^{T} sum_{r=q}^{T} 1{t >= r} P(d_r = 1 | r <= T) +``` + +Weighted average putting more weight on QTTs from larger cohorts; drawback: systematically overweights groups participating longer in the treatment (Callaway and Sant'Anna, 2021) (p. 20). Weighting functions are estimated by sample analogs (analogy principle, Manski 1994) (p. 23). The paper notes richer aggregations along the distribution of Y are conceivable (e.g., cumulative effect on the bottom two deciles for a minimum-wage evaluation) but deliberately limits itself to generalizing the CS-2021 schemes (p. 19). + +*Stochastic dominance rankings* (Section 2.4.2, p. 20): if rank invariance is implausible (or `QTT_{r,t,rho}(tau)` changes sign in `tau`), use stochastic-dominance tests or inequality measures instead of the QTT, comparing `F_{Y_t(r)|d_r=1}` vs `F_{Y_t(0)|d_r=1}` for treated group `r`. + +**Estimation (Section 3, pp. 20-23):** + +Plug-in via empirical distribution functions (EDFs), by the analogy principle (Manski 1994), as in Callaway and Li (2019). + +*Case A - no covariates (Theorem 1) (p. 21):* + +1. Estimate the treated-outcome quantile directly by EDF + left-continuous (inf-based) inversion: + ``` + QTT-hat_{r,t,rho}(tau) = F-hat^{-1}_{Y_t(r)|d_r=1}(tau) - F-hat^{-1}_{Y_t(0)|d_r=1}(tau) + F-hat^{-1}_{Y_t(r)|d_r=1}(tau) = inf{ y : F-hat_{Y_t(r)|d_r=1}(y) >= tau } + ``` +2. Estimate the counterfactual CDF by the sample analog of Theorem 1 (p. 21): + ``` + F-hat_{Y_t(0)|d_r=1}(y) = (1/n_0) sum_{i in 0} 1[ F-hat^{-1}_{Delta_{[r-rho-1,t]}Y|C=1}( F-hat_{Delta_{[r-rho-1,t]}Y|C=1}( Delta_{[r-rho-1,t]}Y ) ) + <= y - F-hat^{-1}_{Y_{r-rho-1}|d_r=1}( F-hat_{Y_{r-rho-1}|C=1}( Y_{r-rho-1} ) ) ] + ``` + where `n_0` = number of never-treated units in periods `r-rho-1` and `t`; the sum runs over never-treated units. (As printed, the first composed term is `F-hat^{-1}_{DeltaY|C=1}(F-hat_{DeltaY|C=1}(.))` - both conditionals `C=1`, i.e. the never-treated quantile substituted for the treated one per Assumption 4, mirroring the Theorem 1 statement; under continuity this composition is the identity, matching the (A.6) -> (A.7) simplification in the proof.) +3. Invert the counterfactual CDF: `F-hat^{-1}_{Y_t(0)|d_r=1}(tau) = inf{ y : F-hat_{Y_t(0)|d_r=1}(y) >= tau }`. +4. Difference of quantiles gives `QTT-hat_{r,t,rho}(tau)`; aggregate with sample-analog weights (p. 23). + +Footnote 16 (p. 21): under Theorem 1's assumptions plus compact support of `Y(a)`, `a in {r, 0}`, Callaway and Li (2019) show the EDF-based estimator is consistent and, via a functional central limit theorem, converges uniformly to a Gaussian process. "These results can be readily extended to the current context, as the only difference lies in the construction of the benchmark, not in the estimator itself." No new asymptotic theorem is stated in this paper. + +*Case B - covariates via IPW (Proposition 1) (p. 22):* + +The only additional term to estimate (shown in Appendix A) is `F^{p,-1}_{Delta_{[r-rho-1,t]}Y(0)|d_r=1}(delta)`, computed by a generalization of Firpo (2007) and Callaway and Li (2019): + +``` +F^{p,-1}_{Delta_{[r-rho-1,t]}Y(0)|d_r=1}(delta) = + [ (1/n_{r,t}) sum_{i in r} (C/p_r) ( p-hat_r(x_i) / (1 - p-hat_r(x_i)) ) 1{ DeltaY_{t,r-rho-1} <= delta } ] + / [ (1/n_{r,t}) sum_{i in r} (C/p_r) ( p-hat_r(x_i) / (1 - p-hat_r(x_i)) ) ] +``` + +(As printed the LHS carries the `p,-1` superscript although the display is the reweighted CDF of Eq. (3), i.e. a Hajek-normalized estimator; inversion then yields the quantile.) `n_{r,t}` = number of units used to compute `QTT-hat_{r,t,rho}(tau)`; `p-hat(X)` is an estimator of the generalized propensity score; the denominator normalizes weights to sum to 1 in finite samples, ensuring `F^p(.)` is a proper distribution function. The propensity score can be estimated parametrically or non-parametrically (both Firpo 2007 and Callaway-Li 2019 show this). + +*Case C - fully conditional (Proposition 2) (p. 22):* + +Computationally demanding: FIVE conditional distributions, possibly infeasible when `dim(X)` is large and `n` small. Remedies: +- quantile regression for the relevant conditional quantities (as Melly and Santangelo 2015 do for CiC with covariates), or +- for discrete covariates, the cell-by-cell method of Callaway, Li, and Oka (2018): estimate `F-hat_{Y_t(r)|X,d_r=1}` by EDF for each value of `x`, invert; similarly the counterfactual per `x` (p. 22): + ``` + F-hat_{Y_t(0)|X=x,d_r=1}(y) = (1/n_{0,x}) sum_{i in 0} 1[ (Delta_{[r-rho-1,t]}Y) + <= y - F-hat^{-1}_{Y_{r-rho-1}|X=x,C=1}( F-hat_{Y_{r-rho-1}|X=x,C=1}( Y_{r-rho-1} ) ) | X = x ] + ``` + (as printed; each quantity estimated by EDFs), valid for each `tau in (0,1)` and each `x in chi`. Then compute the conditional QTT and, if the unconditional parameter is wanted, integrate x out against `F-hat_{X|d_r=1}`. + +**Inference (pp. 21-23):** + +The choice of procedure should be guided by the dependence structure of the application. + +1. *No serial correlation / clustering unlikely: empirical bootstrap of Callaway and Li (2019)* (p. 21), yielding uniform confidence bands covering `QTT(tau)` with fixed probability for all `tau in [eps, 1-eps] subset (0,1)` for some small `eps > 0`: + - Let `QTT-hat*(tau)` be a bootstrap estimate computed with the same plug-in procedure on a bootstrap sample; `B` total iterations, `b = 1, ..., B`. + - For each `b` compute the sup-t statistic: + ``` + I^b = sup_{tau in T} Sigma-hat(tau)^{-1/2} | sqrt(n) ( QTT-hat^b(tau) - QTT-hat(tau) ) | + ``` + with the bootstrap robust variance scale + ``` + Sigma-hat(tau)^{1/2} = (q_{0.75}(tau) - q_{0.25}(tau)) / (z_{0.75} - z_{0.25}) + ``` + - the bootstrap interquartile range of the estimator divided by the IQR of a standard normal (p. 21). + - The (1-alpha) uniform band (p. 22): + ``` + C-hat_{QTT(tau)} = QTT-hat*(tau) +/- c^B_{1-alpha} Sigma-hat(tau)^{1/2} / sqrt(n) + ``` + where `c^B_{1-alpha}` is the (1-alpha) empirical quantile of `{I^b}_{b=1}^B`. **Centering discrepancy (flag for any implementation):** the paper prints the band centered at `QTT-hat*(tau)` (the bootstrap estimate); in Callaway-Li (2019) the band is centered at the original-sample estimate `QTT-hat(tau)`. See Gaps and Uncertainties. + - Footnote 17 (p. 22): for extreme quantiles, use alternative inference procedures (Chernozhukov et al., 2016 - extremal quantile regression). +2. *Clustered dependence likely*: adapt the wild cluster bootstrap to the QTT setting "with minor modifications"; with few treated clusters use the subcluster wild bootstrap (MacKinnon and Webb, 2018). Reviews: Cameron and Miller (2015); MacKinnon, Nielsen, and Webb (2023). (p. 22; no algorithmic detail given.) +3. *Stochastic dominance tests* (p. 23): generalized Kolmogorov-Smirnov statistics of Linton, Maasoumi, and Whang (2005) for FSD and SSD: + ``` + d_{r,t,rho} = sqrt( (n_{r,t} * n_{0,t}) / (n_{r,t} + n_{0,t}) ) min sup ( F-hat_{Y_t(r)|d_r=1}(y) - F-hat_{Y_t(0)|d_r=1}(y) ) + + s_{r,t,rho} = sqrt( (n_{r,t} * n_{0,t}) / (n_{r,t} + n_{0,t}) ) min sup int_{-inf}^{y} ( F-hat_{Y_t(r)|d_r=1}(z) - F-hat_{Y_t(0)|d_r=1}(z) ) dz + ``` + where `n_{r,t}`, `n_{0,t}` are the sample sizes used to estimate the two EDFs. Following Maasoumi and Wang (2019), a PAIR BOOTSTRAP test for iid samples gives the probability of either SD statistic falling in a given interval and a p-value; e.g., if `P(d <= 0)` is large (above .90) and `d` is non-positive, conclude `Y_t(r)|d_r=1` FSD `Y_t(0)|d_r=1` with high confidence. More sophisticated tests are context-dependent (survey: Maasoumi 2003); detailed testing strategies are declared beyond the paper's scope. + +Iterations: no `B` is fixed in the algorithm text; the Monte Carlo table notes (Tables 1-4, pp. 26, 27, 29, 30, and Tables C.1-C.4) state "Each Monte Carlo simulation uses 2,000 bootstrap replications"; all simulation results are based on 2,000 Monte Carlo simulations (p. 25). + +**Monte Carlo evidence (Section 4, pp. 23-30; Tables 1-4):** + +Common setup: no anticipation (`rho = 0`), policy starts from period 2 (`q = 2`), panel data, cohorts `r in {2, ..., T}`, `T = R` (positive treatment probability every period from 2 on); performance measured as average (simulated) bias and root-MSE over 2,000 Monte Carlo simulations; quantiles `tau in {.25, .50, .75}`. The main text reports the counterfactual quantile `F^{-1}_{Y_2(0)|d_2=1,rho=0}(tau)` only; the focus is recovering `F_{Y(0)|d_r=1}`, not the QTT itself (footnote 18, p. 23: rank invariance may plausibly hold since nothing is assumed about `Y_t(r)|d_r=1`). "Unc" rows = estimator based on the unconditional distributional PT ignoring covariates (Theorem 1); "Cond" rows = covariate-based estimator. + +*Section 4.1 - varying N and T (pp. 23-27):* + +- **DGP 1** (Eq. (5), p. 24): `Y_it(0) = alpha_t + eta_i + u_it` with `alpha_t = t`, `eta|d_r=1 ~ N(r,1)` for `r in {2,...,T}` (selection on unobservables), `u_t ~ N(0,1)`; `P(d_r=1) = 1/T` (no covariates). Then `Y_it(0) ~ N(alpha_t + r, 2)`, true quantile `(alpha_t + r) + sqrt(2) Phi^{-1}(tau)`. +- **DGP 2** (Eqs. (6)-(7), pp. 24-25): TWFE with one covariate, mimicking Callaway-Sant'Anna (2021); selection on observables via the generalized propensity score: + ``` + P(d_r = 1 | X) = exp(X'gamma_r) / (1 + sum_r exp(X'gamma_r)) (6) gamma_r = 0.5 r / T + Y_it(0) = alpha_t + eta_i + X_it + u_it (7) + ``` + with `X ~ N(0,1)`, `alpha_t = t`, `eta|d_r=1 ~ N(r,1)`, `u_t ~ N(0,1)`; `Y_it(0) ~ N(alpha_t + r, 3)`, true quantile `(alpha_t + r) + sqrt(3) Phi^{-1}(tau)`. Appendix C variant replaces `alpha_t = t` with the quadratic trend `alpha_t = t + t^2` (Table C.4). +- **DGP 3** (Eq. (8), p. 25): panel quantile regression, re-adaptation of Machado and Santos Silva (2019): + ``` + Y_it(0) = alpha_t + eta_i + X_it + (1 + X_it) u_it (8) + ``` + treatment probability still Eq. (6); `u_t ~ N(0,1)`, `alpha_t = t`, `eta|d_r=1 ~ Z_r`, `gamma_r = r/(4T)`, `X_it = (1/8) chi_it`, with `Z_r = r + T`, `T ~ chi-squared(1)` and `chi_it ~ chi-squared(1)` (note: this `T` is a chi-squared random variable - a notation clash with the number of periods). No analytical quantile formula; population quantiles approximated by simulating Eq. (8) with 1 million observations. + +All three DGPs satisfy the identifying assumptions (footnote 19, p. 25: iid errors per period suffice in the linear models; in the nonlinear DGP the term `(1+X_it)u_it` means distributional PT holds conditionally, and copula invariance holds unconditionally as long as X and u are independent of treatment assignment; footnote 20: copula invariance stays unconditional because no RHS random variable is distributed conditionally on X - estimation gets more demanding when the copula holds only conditionally). + +Scenarios: (i) `T = R = 4` and (ii) `T = R = 10`; `n in {100, 1000}`. + +**Table 1 (p. 26), T = R = 4, selected values as extracted** (bias / RMSE for `F^{-1}_{Y_2(0)|d_2=1}`): + +| DGP | Est | n | tau=.25 | tau=.50 | tau=.75 | +|-----|-----|---|-------|-------|-------| +| DGP 1 | Unc | 100 | 0.112 / 0.525 | 0.097 / 0.489 | 0.122 / 0.509 | +| DGP 2 | Unc | 100 | 0.331 / 0.695 | 0.341 / 0.684 | 0.391 / 0.714 | +| DGP 2 | Cond | 100 | 0.086 / 0.531 | 0.091 / 0.515 | 0.146 / 0.541 | +| DGP 3 | Unc | 100 | 0.178 / 0.550 | 0.244 / 0.583 | 0.361 / 0.721 | +| DGP 3 | Cond | 100 | 0.114 / 0.571 | 0.203 / 0.590 | 0.335 / 0.730 | +| DGP 1 | Unc | 1000 | 0.011 / 0.152 | 0.007 / 0.150 | 0.013 / 0.157 | +| DGP 2 | Unc | 1000 | 0.258 / 0.314 | 0.250 / 0.304 | 0.257 / 0.315 | +| DGP 2 | Cond | 1000 | 0.018 / 0.154 | 0.011 / 0.147 | 0.018 / 0.158 | +| DGP 3 | Unc | 1000 | 0.061 / 0.167 | 0.085 / 0.180 | 0.132 / 0.230 | +| DGP 3 | Cond | 1000 | -0.005 / 0.163 | 0.040 / 0.169 | 0.101 / 0.218 | + +Findings (pp. 25-26): performance is poor with few units per cohort (n = 100), especially for the nonlinear DGP 3; improves substantially with n (largest bias-reduction gain for DGP 3, attributed to fewer observations per quantile under a nonlinear DGP); ignoring covariates ("Unc" for DGPs 2-3) substantially worsens bias/RMSE and leads to unreliable inference, as the theory predicts. + +**Table 2 (p. 27), T = R = 10, selected values as extracted**: DGP 1 Unc n=100 tau=.25: bias 0.047 / RMSE 0.858; n=1000: 0.013 / 0.245. DGP 2 Cond n=1000 tau=.25: 0.008 / 0.247; DGP 3 Cond n=1000 tau=.25: -0.005 / 0.261. Findings (p. 27): average biases mostly smaller in magnitude than T=4 (except tau=.75, n=100), but RMSEs almost always LARGER (much smaller cohorts, imprecise estimates); with `T = R = 10` and small n, DGP 2/DGP 3 parameters could not be estimated in many simulations due to too few treated and/or never-treated units (results not shown). Overall performance does not necessarily improve relative to T = R = 4. + +Cross-cutting results (pp. 27-28): all results align with theory and with simulations in Callaway et al. (2018), Callaway-Li (2019), Callaway-Sant'Anna (2021); RMSE shrinks by ~`1/sqrt(10)` when n goes 100 -> 1000 across DGPs and conditioning, supporting sqrt(n)-consistency of the estimator of `F^{-1}_{Y_t(0)|d_r=1,rho}(tau)`. Power is relatively larger at the .50 quantile (p. 28, in line with Callaway et al. 2018). + +Robustness discussion (pp. 26-27): the Table C.4 quadratic-trend variant gives nearly identical bias/RMSE - the key requirement of Assumption 4 is that untreated-outcome TRENDS be independent of treatment assignment, not linear. The same conclusion would hold for heteroskedastic or within-cluster-correlated errors, as long as the heteroskedasticity/clustering structure is unaffected by treatment assignment (e.g., a skedastic function `exp{X beta}` with the same functional form for treated and never-treated); Copula Invariance requires the dependence between the pre-treatment error level `u_{ir'}` and its change `Delta_{r-1,t}u` to be unaffected by treatment assignment. + +*Section 4.2 - violations of the main identifying assumptions (pp. 28-30):* + +Fixed setup: n = 1000, T = R = 4, `X ~ N(0,1)`, cohort probabilities via Eq. (6) with `gamma_r = 0.5r/T`. Reported parameter: `F^{-1}_{Y_2(0)|d_2=1,rho=0}(tau)`. + +- **DGP 4** (Eq. (9), p. 28) - violation of (Conditional) Distributional Parallel Trends (Assumption 4), copula invariance intact: + ``` + Y_it(0) = alpha_{t,d} + eta_i + X_it + u_it (9) + ``` + `eta|d_r=1 ~ N(r,1)`, `r = {2,...,4}`, `u_t ~ N(0,1)`, and `alpha_{t,d} = t(1 + eps-bar d)` with d = 0 for never-treated, d = 1 for treated cohorts; `eps-bar` = degree of violation (`eps-bar = 0` -> Assumption 4 holds; larger `eps-bar d` -> larger deviation; no cohort-specific violations, to keep the analysis simple). Then `Y_it(0) ~ N(t(1+eps-bar d) + r, 3)`, true quantile `t(1+eps-bar d) + r + sqrt(3) Phi^{-1}(tau)`. Grid: `eps-bar in {0.00, 0.05, 0.10, 0.50}`. + + **Table 3 (p. 29)** - selected Cond rows at tau=.50: bias 0.012 (eps-bar=0), -0.037 (0.05), -0.086 (0.10), -0.488 (0.50); RMSE 0.15 -> 0.158 -> 0.166 -> 0.51. Unc tau=.50 bias: 0.253, 0.203, 0.155, -0.248. Findings (pp. 28-29): SMALL violations of Assumption 4 -> only minor increases in bias/RMSE; large violation (eps-bar=0.50) -> substantial bias. Same pattern at .25/.75. Ignoring covariates significantly worsens things EXCEPT when the Assumption-4 violation is large: with a big trend deviation, the single covariate (which by construction has a time-constant effect, unrelated to the trend in Y(0)) cannot capture it, and conditioning on X merely adds estimation noise within a quantile - so "Cond" is worse than "Unc" at eps-bar=0.50. + +- **DGP 5** (pp. 29-30) - violation of unconditional Copula Invariance (Assumption 5), Assumption 4 intact; similar to DGP 2 of Callaway et al. (2018). Untreated potential outcome follows Eq. (7) with + ``` + (eta_i, u_{i,t}, u_{i,r-1}) | d_r = 1 ~ N(mu_r, V_r), mu_r = [r, 0, 0]^T + + V_r = [ [1, rho_{r,u_t}, rho_{r,u_{r-1}}], + [rho_{r,u_t}, 1, rho_{u_t,u_{r-1}}], + [rho_{r,u_{r-1}}, rho_{u_t,u_{r-1}}, 1] ] r in {2,3,4}, t in {1,...,4} + ``` + `V_r` symmetric; `(Delta_{r-1,t}y(0), Y_{r-1})` is bivariate normal with correlation parameter `rho_{u_t,u_{r-1}} + rho_{r,u_t} - rho_{r,u_{r-1}} - 2`; for a bivariate normal the copula is Gaussian with dependence = correlation coefficient (Callaway et al. 2018). Violation device (p. 30): set `rho_{u_t,u_{r-1}} = 0.5` for all treated and never-treated units and + ``` + rho_{r,u_t} = rho-bar d for all t >= r + = 0 for all t < r + ``` + with d = 0 for never-treated, d = 1 for treated (any cohort); `rho-bar` = degree of copula-invariance violation. Grid: `rho-bar in {0.00, 0.05, 0.10, 0.50}`. + + **Table 4 (p. 30)** - selected Cond rows at tau=.25: bias 0.023 (rho-bar=0), 0.042 (0.05), 0.061 (0.10), 0.203 (0.50). Findings (p. 30): small copula violations -> minimal bias/RMSE increases; substantial violation (rho-bar=0.50) -> pronounced increase (Cond .25 bias 0.061 at rho-bar=0.10 rises to 0.203 at rho-bar=0.50; similar at .75). The **.50 quantile is insensitive** to copula violations: the deviation only affects the variance of `Y_it(0)`, and `Phi^{-1}(0.50) = 0` so `F^{-1}_{Y_t(0)|d_r=1}(tau) = (alpha_t + r)` - aligning with Callaway et al. (2018). Ignoring covariates significantly increases bias/RMSE in nearly all cases; the single exception is tau=.75 at rho-bar=0.50, where the unconditional bias is lower - flagged by the author as "a puzzling result." + +**Appendix A - proof structure (pp. 37-40):** + +*Proof of Theorem 1, route 1 (copula-pdf route, pp. 37-38):* adapted from Callaway and Li (2019); uses two Sklar's-Theorem results, Lemma A.1 and Lemma A.2 in Appendix A of Callaway and Li (2019) (deliberately NOT reproduced in this paper: "To save space, I refer the reader to their paper," p. 37 - external cross-reference). Setup assumes no anticipation (`rho = 0`); abbreviations `f_{t|d_r=1}`, `f_{t|C=1}` for the joint pdfs of `(Y_t(0)-Y_{r-rho-1}(0), Y_{r-rho-1}(0))` and `c_{t|d_r=1}`, `c_{t|C=1}` for the corresponding copula pdfs; supports `DeltaY` for the long difference and `Y` for the pre-period level (support notation printed as `Y_{t-1}(0)` in one place and `Y_{r-1}(0)` elsewhere, as extracted). Chain: write `F_{Y_t(0)|d_r=1}` as a double integral over the joint pdf; **(A.1)** rewrites the joint via the copula pdf (CL19 Lemma A.1); **(A.2)** substitutes the never-treated copula `c_{t|C=1}` for the unobservable treated copula by Assumption 5; **(A.3)** rewrites the copula pdf as the never-treated joint distribution in density-ratio form (CL19 Lemma A.2). A change of variables `u = F^{-1}_{Delta|C=1}(F_{Delta|d_r=1}(delta))`, `v = F^{-1}_{Y_{r-1}(0)|C=1}(F_{Y_{r-1}(0)|d_r=1}(y'))` (with implied equalities numbered 1-4 in the paper for `delta`, `y'`, `d delta/d u`, `d y'/d v`) turns (A.3) into **(A.4)** (integral over the never-treated joint), **(A.5)** (definition of expectation), **(A.6)** (replace the unknown `F^{-1}_{Delta_{[r-1,t]}Y(0)|d_r=1}` with the never-treated change distribution by Assumption 4 holding unconditionally), and **(A.7)**: + +``` +F_{Y_t(0)|d_r=1} = E[ 1( Delta_{[r-1,t]}Y(0) <= y + - F^{-1}_{Y_{r-1}(0)|d_r=1}( F_{Y_{r-1}(0)|C=1}( Y_{r-1}(0) ) ) ) | C=1 ] (A.7) +``` + +which proves the result since each remaining distribution is identified by its sample counterpart. QED. + +*Proof of Theorem 1, route 2 (Sklar route, pp. 38-39):* "an alternative and more direct approach" exploiting Assumption 6 (continuity) and Sklar's Theorem, as in Callaway et al. (2018). Write **(A.8)** `F_{Y_t(0)|d_r=1} = P(Delta_{[r-1,t]}Y_i(0) + Y_{i,r-1}(0) <= y | d_r=1)`; under continuity, rank representations **(A.9)** for cohort-r units (`Delta = F^{-1}_{Delta|d_r=1}(u_i^r)`, `Y_{r-1} = F^{-1}_{Y_{r-1}|d_r=1}(v_i^r)` with `u_i^r`, `v_i^r` the treated ranks) and **(A.10)** for never-treated units (ranks `u_i^0`, `v_i^0`). Substituting into (A.8) gives **(A.11)** `F_{Y_t(0)|d_r=1} = P( F^{-1}_{Delta_{[r-1,t]}Y(0)|d_r=1}(u_i^r) + F^{-1}_{Y_{r-1}(0)|d_r=1}(v_i^r) <= y | d_r=1 )`. The joint distribution of `(u_i^r, v_i^r)` is the unknown treated copula; by Sklar's Theorem and the unconditional version of Assumption 5, replace it with the never-treated copula (conditioning switches to `C=1`, ranks to `u_i^0, v_i^0`); `F^{-1}_{Y_{r-1}(0)|d_r=1}` is observable but `F^{-1}_{Delta_{[r-1,t]}Y(0)|d_r=1}` is not, so Assumption 4 replaces it with the never-treated change distribution, giving the final estimable form `F_{Y_t(0)|d_r=1} = P( Delta_{[r-1,t]}Y_i(0) + F^{-1}_{Y_{r-1}(0)|d_r=1}( F_{Y_{r-1}(0)|C=1}( Y_{i,r-1}(0) ) ) <= y | C=1 )`. QED. + +*Implementation-relevant constructions revealed by the proofs:* the final identification formula requires estimating only (1) the never-treated pre-treatment CDF `F_{Y_{r-1}(0)|C=1}`, (2) the treated cohort-r pre-treatment quantile function `F^{-1}_{Y_{r-1}(0)|d_r=1}` (composed as a quantile-quantile transform of never-treated pre-period ranks), and (3) the observed changes `Y_{i,t} - Y_{i,r-1}` among never-treated units. The counterfactual CDF is the never-treated empirical mean of `1{ Delta_i + QQ(Y_{i,r-1}) <= y }` - one `F^{-1} o F` composition per unit; NO density or copula estimation at the estimation stage (densities/copula pdfs appear only in the proof). + +*Proof of Proposition 1 (pp. 39-40):* all Theorem 1 steps remain valid; the only change is step **(A.6)**, which used unconditional Assumption 4 to identify `F_{Delta Y(0)|d_r=1}`. With covariates this object is instead identified by the reweighted distribution of Eq. (3). It suffices to prove `F_{Delta Y(0)|d_r=1} = F^p_{Delta Y(0)|d_r=1}`: + +``` +F_{Delta Y(0)|d_r=1} = P(Delta Y_t(0) <= delta | d_r=1) + = P(Delta Y_t(0) <= delta, d_r=1) / p_r (A.12) + = E( P(Delta Y_t(0) <= delta, d_r=1 | X) / p_r ) + = E( (P_r(X)/p_r) * P(Delta Y_t(0) <= delta | d_r=1, X) ) + = E( (P_r(X)/p_r) * P(Delta Y_t(0) <= delta | X, C=1) ) (A.13) + = E( (P_r(X)/p_r) * E[ C * 1{Delta Y <= delta} | X, C=1 ] ) (A.14) + = E( (P_r(X) / (p_r (1 - p_r(X)))) * E[ C * 1{Delta Y_t <= delta} | X ] ) + = E( (C p_r(X) / (p_r (1 - p_r(X)))) * 1{Delta Y_t <= delta} ) (A.11) +``` + +Step logic: (A.12) definition of conditional probability; **(A.13) holds by Assumption 4** (conditional distributional PT); (A.14) definition of probability, then multiplication by `C` (valid since the inner expectation conditions on `C=1`); conditioning on `C=1` lets the potential outcome be rewritten as the observed outcome; the last equality uses the Law of Iterated Expectations. QED. **Labeling quirk (p. 40): the last display of this chain is printed with tag "(A.11)", duplicating the (A.11) of the Alternative Proof on p. 39** - presumably a typo (should be (A.15) or a back-reference to Eq. (3)'s weight); transcribed as printed. Also note the capitalization inconsistency `P_r(X)` vs `p_r(X)` within the chain (as printed). Definitions (p. 40): `p_r = p(d_r = 1)` (fraction of treated units in a given cohort); `p_r(X) = P(d_r = 1 | X, d_r + C = 1)`, the generalized propensity score as defined in Assumption 7, which "models the conditional probability of belonging to cohort r and either being part of cohort r or the never-treated group." + +*Proof of Proposition 2 (p. 40):* "follows directly from Theorem 1, where now all the steps hold after conditioning on covariates." QED (one line; the outcome-regression/conditional analogue). + +**Appendix B - repeated cross sections (pp. 41-42):** + +Setting: extends identification to repeated cross sections (RCS), following Callaway et al. (2018) and CS (2021). For each cross-sectional unit the researcher observes `(Y, d_r, ..., d_T, C, t, X)` where `r = q, ..., T` and `t = 1, ..., T` is the period in which unit i is observed; `S_t` is a dummy = 1 if the observation is observed in period t; a random sample is available in each period. + +*Assumption B.1 (p. 41, exact statement):* "Conditional on period t, `(Y, d_r, ..., d_T, C, X)` are cross sectionally independent and identically distributed for all `t in {1, ..., T}`, where `(d_r, ..., d_T, C, X)` is invariant to t." + +Stated to be identical to the corresponding assumption in CS (2021); the pooled cross section is composed of independent draws from the mixture distribution + +``` +F_M(Y, d_r, ..., d_T, c, t, x) = sum_{t=1}^{T} P(S_t = 1) * F_{Y, d_r, ..., d_T, C, t, X}(Y, d_r, ..., d_T, c, x | t) +``` + +Also similar to Abadie (2005) and Sant'Anna and Zhao (2020); **excludes the possibility of compositional changes over time**. + +Key problem: even if Assumptions 1, B.1, 6, and unconditional 3-5 hold, the difference `Y_it - Y_{i,r-1}` is NOT observed for the same unit in a pooled cross section (setup: covariates play no role; `rho = 0` WLOG). Following Callaway et al. (2018), impose **rank invariance on potential outcomes over time** within the never-treated group - units preserve their relative position in the distribution of Y over time; the paper flags it as "a strong assumption and is often rejected in empirical data (as discussed in the main text), [but] it is necessary to recover the unknown distribution at time t." + +*Corollary 1 (p. 41, exact statement):* "Suppose we have access to repeated cross sectional data, specifically `{(Y, d_ir, C_i)}_{i=1}^{n^s}` for period `s in {r-1, t}` where `r = q, ..., T`, `t = 1, ..., T`, and `n^s` denotes the sample size of the cross section. Suppose further that Assumptions 1, 6, B1, and the unconditional version of Assumptions 3-5 hold. If the copula of `(Y_{i,r-1}(0), Y_{i,t}(0) | C=1)` satisfies rank invariance, then for every `(u,v) in [0,1]^2` + +``` +C_{Y_{i,r-1}(0), Y_{i,t}(0) | C=1}(u, v) = min{u, v} +``` + +Thus, for `y in supp(Y_{i,t}(0)|d_r=1)`, we obtain + +``` +F_{Y_t(0)|d_r=1}(y) = P{ Delta~_{[r-1,t]}Y(0) + + F^{-1}_{Y_{r-1}(0)|d_r=1}( F_{Y_{r-1}(0)|C=1}( Y_{r-1}(0) ) ) <= y | C=1 } + +where Delta~_{[r-1,t]}Y(0) := F^{-1}_{Y_t(0)|C=1}( F_{Y_{r-1}(0)|C=1}( Y_{r-1}(0) ) ) - Y_{r-1} +``` + +(`Delta~` denotes the tilde-Delta imputed change; the trailing term is printed as `Y_{r-1}`, without the `(0)`.) `min{u,v}` is the comonotone (Frechet upper bound) copula. Proof (p. 42): under the stated assumptions Theorem 1's result holds, but with RCS `Delta_{[r-1,t]}Y(0)` cannot be identified from observed never-treated outcomes; rank invariance yields `F_{Y_t(0)|C=1}(Y_t(0)) = F_{Y_{r-1}(0)|C=1}(Y_{r-1}(0))`, and since both CDFs are identifiable from observed outcomes, the never-treated change is imputable as `Delta~` above. QED. + +*Caveat (p. 42, exact):* "As noted in Callaway et al. (2018), the rank invariance assumption on the copula of `(Y_{i,r-1}(0), Y_{i,t}(0)|C=1)` **neither implies nor is implied by Assumption 5**." (Rank invariance here is a within-never-treated-group over-time copula condition; Assumption 5 is copula invariance across treatment groups.) + +*Covariates in RCS (p. 42):* Proposition 2's result generalizes by making all steps conditional on X; extending Proposition 1 requires a MODIFIED definition of the generalized propensity score, as outlined in Appendix B of Callaway and Sant'Anna (2021) (external reference; not derived in this paper). + +**Appendix C - additional simulation results (pp. 43-44; Tables C.1-C.4):** + +All tables: `T = R = 4`, `tau = .5` (except C.4, which reports tau = .25/.50/.75), 2,000 bootstrap replications per Monte Carlo simulation. Six group-time parameters per table: `F^{-1}_{Y_t(0)|d_g=1}` for (g,t) in {(2,2),(2,3),(2,4),(3,3),(3,4),(4,4)}. + +*Table C.1 - DGP 1 (Eq. (5)), tau = .5 (p. 43):* no Unc/Cond split (DGP 1 is the no-covariate baseline; note the table-notes boilerplate still mentions rows labeled 'UNC' although none appear - minor inconsistency). + +| Parameter | True | n=100 Bias | n=100 RMSE | n=1,000 Bias | n=1,000 RMSE | +|---|---|---|---|---|---| +| F^{-1}_{Y_2(0)\|d_2=1} | 4 | 0.097 | 0.489 | 0.007 | 0.15 | +| F^{-1}_{Y_3(0)\|d_2=1} | 5 | 0.089 | 0.491 | 0.009 | 0.148 | +| F^{-1}_{Y_4(0)\|d_2=1} | 6 | 0.097 | 0.492 | 0.008 | 0.148 | +| F^{-1}_{Y_3(0)\|d_3=1} | 6 | 0.087 | 0.451 | 0.013 | 0.133 | +| F^{-1}_{Y_4(0)\|d_3=1} | 7 | 0.095 | 0.451 | 0.012 | 0.133 | +| F^{-1}_{Y_4(0)\|d_4=1} | 8 | 0.101 | 0.416 | 0.015 | 0.126 | + +Pattern: small-sample bias ~0.09-0.10 shrinking to ~0.01 at n=1,000; RMSE roughly one third (consistent estimator under correct specification). + +*Table C.2 - DGP 2 (Eq. (7)), tau = .5 (p. 43):* + +| Block | Parameter | True | n=100 Bias | n=100 RMSE | n=1,000 Bias | n=1,000 RMSE | +|---|---|---|---|---|---|---| +| Unc | Y_2\|d_2 | 4 | 0.341 | 0.684 | 0.25 | 0.304 | +| Unc | Y_3\|d_2 | 5 | 0.344 | 0.686 | 0.248 | 0.303 | +| Unc | Y_4\|d_2 | 6 | 0.341 | 0.689 | 0.248 | 0.302 | +| Unc | Y_3\|d_3 | 6 | 0.469 | 0.723 | 0.371 | 0.41 | +| Unc | Y_4\|d_3 | 7 | 0.47 | 0.728 | 0.372 | 0.41 | +| Unc | Y_4\|d_4 | 8 | 0.571 | 0.794 | 0.479 | 0.509 | +| Cond | Y_2\|d_2 | 4 | 0.091 | 0.515 | 0.011 | 0.147 | +| Cond | Y_3\|d_2 | 5 | 0.099 | 0.522 | 0.007 | 0.147 | +| Cond | Y_4\|d_2 | 6 | 0.091 | 0.519 | 0.005 | 0.145 | +| Cond | Y_3\|d_3 | 6 | 0.099 | 0.508 | 0.014 | 0.156 | +| Cond | Y_4\|d_3 | 7 | 0.098 | 0.515 | 0.012 | 0.151 | +| Cond | Y_4\|d_4 | 8 | 0.09 | 0.507 | 0.007 | 0.153 | + +Pattern: the unconditional estimator's bias does NOT vanish with n (0.25-0.48 at n=1,000) and GROWS with later-treated cohorts (0.25 for d_2 -> 0.371 for d_3 -> 0.479 for d_4); Cond bias vanishes (<= 0.014 at n=1,000). A clear demonstration that when trends depend on covariates the unconditional estimator is inconsistent. + +*Table C.3 - DGP 3 (Eq. (8)), tau = .5 (p. 44):* + +| Block | Parameter | True | n=100 Bias | n=100 RMSE | n=1,000 Bias | n=1,000 RMSE | +|---|---|---|---|---|---|---| +| Unc | Y_2\|d_2 | 4.839 | 0.244 | 0.583 | 0.085 | 0.18 | +| Unc | Y_3\|d_2 | 5.841 | 0.239 | 0.571 | 0.083 | 0.179 | +| Unc | Y_4\|d_2 | 6.867 | 0.2 | 0.55 | 0.061 | 0.171 | +| Unc | Y_3\|d_3 | 6.837 | 0.239 | 0.561 | 0.092 | 0.178 | +| Unc | Y_4\|d_3 | 7.869 | 0.206 | 0.544 | 0.064 | 0.163 | +| Unc | Y_4\|d_4 | 8.867 | 0.206 | 0.527 | 0.065 | 0.162 | +| Cond | Y_2\|d_2 | 4.839 | 0.203 | 0.59 | 0.04 | 0.169 | +| Cond | Y_3\|d_2 | 5.841 | 0.195 | 0.576 | 0.036 | 0.168 | +| Cond | Y_4\|d_2 | 6.867 | 0.16 | 0.56 | 0.015 | 0.167 | +| Cond | Y_3\|d_3 | 6.837 | 0.191 | 0.573 | 0.041 | 0.168 | +| Cond | Y_4\|d_3 | 7.869 | 0.16 | 0.565 | 0.012 | 0.163 | +| Cond | Y_4\|d_4 | 8.867 | 0.157 | 0.545 | 0.017 | 0.158 | + +Pattern: under the nonlinear DGP 3 (distributional PT holds conditionally per footnote 19), both estimators carry modest residual bias at n=1,000 (Unc 0.06-0.09, Cond 0.01-0.04); Cond still dominates but the gap is smaller than DGP 2. Non-integer true values here (4.839, 5.841, ...) vs integer true values in C.1/C.2, since DGP 3 has no analytical quantile formula. + +*Table C.4 - DGP 2 with quadratic trend `alpha_t = t + t^2`, parameter `F^{-1}_{Y_2(0)|d_2=1, rho=0}(tau)` (p. 44):* + +| n | Est | tau=.25 True | Bias | RMSE | tau=.50 True | Bias | RMSE | tau=.75 True | Bias | RMSE | +|---|---|---|---|---|---|---|---|---|---|---| +| 100 | Unc | 6.832 | 0.331 | 0.684 | 8 | 0.344 | 0.672 | 9.168 | 0.392 | 0.715 | +| 100 | Cond | 6.832 | 0.082 | 0.526 | 8 | 0.09 | 0.508 | 9.168 | 0.152 | 0.553 | +| 1000 | Unc | 6.832 | 0.256 | 0.315 | 8 | 0.252 | 0.305 | 9.168 | 0.25 | 0.31 | +| 1000 | Cond | 6.832 | 0.017 | 0.158 | 8 | 0.013 | 0.149 | 9.168 | 0.011 | 0.156 | + +Nonlinear-trend result: with a quadratic covariate-dependent trend, Unc bias persists at ~0.25 for all three quantiles at n=1,000 (inconsistent), while Cond bias falls to 0.011-0.017 - the method does not rely on linearity of trends, only on conditioning on the covariates that drive them. At n=100, Cond bias is mildly larger in the upper tail (0.152 at tau=.75 vs 0.082 at tau=.25). + +**Relation to other methods:** + +- **Callaway and Li (2019)** (2-period/2-group QTT under distributional PT + copula STABILITY): this paper is its generalization to multiple periods and staggered adoption, applying the CL19 machinery "to each pair of treated cohorts and never-treated units" (p. 3). Key substitution: copula stability over time (needs panel + >= 2 pre-treatment periods) is replaced by copula invariance across groups (Assumption 5), which needs neither and extends to repeated cross sections (pp. 3, 13). CL19's empirical bootstrap is retained for QTT inference (p. 4); CL19's IPW (after Firpo 2007) is generalized for the conditional-PT case (p. 17); the asymptotics of the EDF plug-in are inherited from CL19 (footnote 16, p. 21). +- **Callaway, Li, and Oka (2018)** ("Callaway et al. (2018)"): source of the copula-invariance-across-groups idea and of the repeated cross-sections / discrete-covariates conditional implementation (pp. 3, 5, 13). CLO18's conditional copula invariance results pertain to the CONDITIONAL QTT; this paper also delivers the unconditional QTT (Proposition 2, integrating out X) (p. 5). +- **Callaway and Sant'Anna (2021)**: supplies the group(cohort)-time parameter architecture, never-treated vs not-yet-treated comparison-group logic, the Limited Treatment Anticipation extension, and the aggregation schemes (Eq. (4), event-time and overall weights) - all generalized from means (ATT(g,t)) to distributions/quantiles (pp. 3-4, 7, 11, 19-20). Assumptions 1-3 closely align with CS (2021) (p. 32). +- **Athey and Imbens (2006) CiC** (pp. 2, 6, 31): relates the untreated outcome to group, time, and unobservables via a monotonic production function; allows selection on unobservables but assumes the unobservables' distribution within a group is stable over time. This paper does NOT restrict the functional form relating groups, time, and covariates to outcomes - but, unlike CiC, it is **not scale-invariant** (p. 31). Melly and Santangelo (2015) extend CiC to covariates via quantile regression (cited as the Case C remedy, p. 22). All the alternative counterfactual-distribution methods in Section 5 are declared extendable to staggered adoption using the paper's intuition (p. 31). +- **Bonhomme and Sauder (2011)** (pp. 12, 16, 31): requires the production function mapping groups, time, and covariates into outcomes to be ADDITIVE. This paper's assumptions are weaker on serial correlation (allowed) and on correlation between time-varying shocks and unobserved heterogeneity (allowed), and it does not require the time-varying unobservable to be independent of treatment assignment; but unlike Bonhomme-Sauder it does NOT allow returns to unobserved skills to change after the policy (p. 31). The generality "comes at the cost of an additional assumption regarding the missing dependence (copula) between the change in the untreated potential outcome and its pre-treatment level" (p. 31). +- **Li and Lin (2024)** (concurrent; the only other QTT identification in staggered DiD known to the author, pp. 3, 5-6): both use distributional PT + copula invariance and extend to repeated cross sections, but (a) Li-Lin extend Callaway et al. (2018) whereas this paper generalizes Callaway-Li (2019); (b) Li-Lin use a NOT-YET-TREATED comparison group vs never-treated here; (c) Li-Lin offer NO estimator for the QTT and do not specify covariate nature, leaving unconditional QTT estimation under conditional copula invariance unclear; this paper provides estimators, addresses the five-conditional-distributions challenge via three identification scenarios, and adds aggregation schemes. +- **Fan and Yu (2012)** (pp. 3, 11, 13): distributional PT alone only PARTIALLY identifies the distributional effect - motivates the copula assumption. +- **Firpo (2007)** / **Abadie (2005)** (pp. 3, 12, 17): selection-on-observables IPW quantile treatment effects; distributional PT is weaker; the reweighting lineage generalized in Proposition 1. +- **Miller (2023)** (footnote 15, p. 17): distributional PT usage; the doubly-robust route not pursued here. +- **Maasoumi and Wang (2019)** (pp. 4, 8-10, 20, 23): evidence against rank invariance; source of the FSD/SSD framework and the pair-bootstrap SD test adapted here. Linton, Maasoumi, and Whang (2005): the generalized KS statistics. +- **Sensitivity caveat vs CS-2021 (Discussion, p. 32)**: the estimator "is sensitive to strong violations of the Copula Invariance assumption (not required in Callaway and Sant'Anna, 2021) and may underperform, compared to the method proposed by Callaway and Sant'Anna (2021), when the PT assumption holds on average but not across the entire distribution" - researchers should weigh assumption plausibility against other approaches. Footnote 21: Imbens and Wooldridge (2009) for a review of common counterfactual-estimation assumptions; footnote 22: Wooldridge (2021) for a comparison of CS 2021 / Sun-Abraham 2021 / Wooldridge 2021. Use-case example (pp. 31-32): minimum wage / teen employment as in CS (2021), distributional effects "particularly on the lower deciles of the income distribution" - relevant for equity assessments. +- Broader staggered-ATT literature cited for context (p. 3): Borusyak et al. (2021), Sun and Abraham (2021), Wooldridge (2021), de Chaisemartin and d'Haultfoeuille (2022); Roth et al. (2023) review. + +**Reference implementation(s):** +- **The initiative knows of NO verified reference implementation of this paper's estimator.** The paper's own p. 1 footnote states the Monte Carlo simulations were run in STATA and that "the ado files of the command used for implementing the methodology presented in the paper, `qtt`, are available upon request at the time of writing" - i.e., a private Stata implementation exists but is not published; no software, package, or replication code is mentioned anywhere in pp. 16-32 (verified during extraction). +- **Do NOT conflate with the R `ecic` package (Kluser)**: `ecic` implements a RELATED but methodologically DIFFERENT staggered event-study CiC (the Athey-Imbens transformation applied per cohort-period), not this paper's copula-based Callaway-Li-style method. Verify any candidate parity target's methodology against Theorem 1 / Proposition 1 before a future implementation PR. + +**Requirements checklist (for the future scoped PR, not v1):** +- [ ] Identify or build a parity target first: request the author's Stata `qtt` ado, or construct simulation-based golden values from the paper's DGPs 1-5 (analytical true quantiles exist for DGPs 1, 2, 4); there is no published reference implementation to test against +- [ ] Re-verify all equation/assumption/table numbers against any published version of record (this review is pinned to arXiv v2) +- [ ] Resolve the Assumption 4 `t >= q - rho` restriction and the uniform-band centering discrepancy (see Gaps and Uncertainties) before writing code +- [ ] (g,t)-cell architecture: reuse diff-diff's CallawaySantAnna cohort-time cell and aggregation infrastructure (Eq. (4); event-time and overall weights with sample-analog estimation; base period `r - rho - 1`; anticipation parameter `rho`; never-treated default with a not-yet-treated option) +- [ ] EDF plug-in estimation per Section 3 Case A; inf-based quantile inversion; no smoothing anywhere +- [ ] IPW path (Proposition 1): parametric generalized propensity score `P_{r,t}(X)` with pairwise cohort-vs-never-treated conditioning; Hajek weight normalization; overlap/trimming warnings as `p_r(X) -> 1` (Assumption 7) +- [ ] Fully conditional path (Proposition 2): discrete-covariate cells per CLO18, warn on the five-conditional-distributions cost; integrate out against `F-hat_{X|d_r=1}` for the unconditional QTT +- [ ] Empirical bootstrap sup-t uniform bands with the IQR-based variance scale; tau grid restricted to a compact `[eps, 1-eps]`; extreme-quantile warning (footnote 17); wild/subcluster options documented for clustered data +- [ ] Stochastic dominance tests (`d_{r,t,rho}`, `s_{r,t,rho}`) with pair bootstrap as the rank-invariance-free summary +- [ ] Continuity/ties check (Assumption 6): warn on heavy ties or atoms - the `F^{-1} o F` compositions and copula uniqueness break on discrete outcomes +- [ ] Small-cohort guards: the paper's own simulations failed to compute for T = R = 10 with n = 100; warn on tiny cohort/never-treated cells +- [ ] Document tail-quantile sensitivity to copula-invariance violations (Table 4: Cond .25 bias 0.023 -> 0.203 as rho-bar goes 0 -> 0.5) and median robustness; document the DGP 4 caveat that conditioning can be worse than not conditioning under large PT violations +- [ ] RCS mode only via Corollary 1 with an explicit rank-invariance warning (strong, often rejected empirically; neither implies nor is implied by Assumption 5); RCS-with-covariates IPW needs the modified propensity score from CS (2021) Appendix B, which this paper does not derive + +--- + +## Implementation Notes + +**Relevance to diff-diff CiC/QDiD v1 (2026-07-12):** +- (i) This review documents the deferred staggered extension, so the ROADMAP's "Distributional DiD for staggered timing (Ciaccio)" row is now backed by a reviewed scope rather than a bare citation - the row moves from "commit when a user reports need" to "reviewed, implementation deferred pending demand". +- (ii) The Assumption 4 (distributional parallel trends) / Assumption 5 (copula invariance) structure is the multi-period generalization of the Callaway-Li-Oka (2018) machinery also reviewed today - together the three reviews map the whole panel-copula design space: CLO18 (2 periods, copula invariance across groups), CL19 (>= 3 periods, copula stability across time), Ciaccio (staggered multi-period, copula invariance across groups per cohort-time cell). +- (iii) The Unc-vs-Cond Monte Carlo results are quantitative evidence for why v1's covariates deferral must be documented loudly rather than silently: the unconditional estimator's bias does not vanish with n and GROWS across later-treated cohorts (Table C.2 at n=1,000: 0.25 for d_2, 0.371 for d_3, 0.479 for d_4) and persists under quadratic trends (Table C.4: ~0.25 at all three quantiles at n=1,000). Ignoring relevant covariates is an inconsistency, not a finite-sample nuisance. +- (iv) The CS-2021-style aggregation architecture (cohort-time cells, Eq. (4) weights, event-time and overall schemes, sample-analog weight estimation) means a future staggered distributional PR can reuse diff-diff's existing CallawaySantAnna (g,t) cell/aggregation infrastructure rather than building parallel plumbing. + +### Data Structure Requirements +- Balanced-in-spirit panel with `T > 2` periods, unit-level first-treatment cohort `d_r`, and a never-treated group (`C = 1`); always-treated units must be dropped (footnote 3, p. 7). Not-yet-treated comparisons are possible by symmetry but not spelled out. +- The estimator consumes, per (r,t) cell: never-treated units' long differences `Y_t - Y_{r-rho-1}` and base-period levels `Y_{r-rho-1}`, plus treated cohort r's base-period marginal (for the quantile-quantile transform) and post-treatment outcomes (for the treated quantile). +- Covariates must be PRE-TREATMENT; the fully conditional path effectively requires discrete covariates (cell-by-cell EDFs) or a quantile-regression approximation. +- Continuous outcomes (Assumption 6): ties/atoms break copula uniqueness and every `F^{-1} o F` composition. +- Repeated cross sections: supported only via Corollary 1 at the cost of over-time rank invariance within the never-treated group (comonotone copula), compositional stability (Assumption B.1), and - with covariates on the IPW path - a modified propensity score this paper does not derive. + +### Computational Considerations +- Pure ECDF plug-in per (r,t) cell: sorting dominates, O(n log n) per cell; the quantile-quantile transform evaluates via two searchsorted passes over sorted arrays. No kernels, bandwidths, or density estimation for point estimates or bootstrap inference (densities/copula pdfs appear only in the Appendix A proofs). +- The copula is never estimated explicitly - Theorem 1 handles it implicitly through each never-treated unit's own (long difference, base level) pairing. An implementation that separately estimates a copula object is over-engineering the estimator. +- The number of (r,t) cells grows as O(T^2); each bootstrap iteration re-runs all cells, and the paper's convention is B = 2,000 - budget accordingly. Replicates are embarrassingly parallel. +- The IPW path adds one propensity-score fit per cohort r (pairwise cohort-vs-never-treated, NOT one multinomial fit) plus Hajek reweighting of the never-treated change distribution. +- Small cells are the binding constraint in practice: with T = R = 10 and n = 100 the paper's own simulations frequently failed to produce estimates (p. 27). More periods shrink cohorts; guard and warn. + +### Tuning Parameters + +| Parameter | Type | Default | Selection Method | +|-----------|------|---------|-----------------| +| `rho` (anticipation) | int >= 0 | 0 (no anticipation) | Known from institutional context (Assumption 3 requires a KNOWN rho); shifts the base period to `r - rho - 1` | +| `n_bootstrap` | int | 2,000 (paper's Monte Carlo convention; no B fixed in the algorithm text) | Band stability | +| `quantiles` (tau grid) | floats in `[eps, 1-eps] subset (0,1)` | paper reports .25/.50/.75 in simulations | Compact grid strictly inside (0,1); avoid extreme quantiles (footnote 17) | +| comparison group | {never-treated, not-yet-treated} | never-treated | Not-yet-treated by symmetric arguments (CS 2021); prefer never-treated if later-treated units anticipate (pp. 5, 7) | +| covariate path | {unconditional, IPW (Prop 1), fully conditional (Prop 2)} | unconditional only if Assumptions 3-5 plausibly hold unconditionally ("highly unlikely", p. 17) | IPW when only PT is conditional; Prop 2 (discrete cells) when the copula is also conditional | +| propensity-score model | parametric spec of `P_{r,t}(X)` | logit-style (Eq. (6) used in simulations) | Misspecification risk; doubly-robust alternative (Miller 2023) noted but out of scope | +| aggregation | {none (cell-level), event-time, overall} | cell-level QTT_{r,t,rho}(tau) | Eq. (4) weights, sample-analog estimation (p. 23) | + +No smoothing parameters exist anywhere in the method. + +### Relation to Existing diff-diff Estimators +- `CallawaySantAnna`: the mean-ATT sibling and architectural template - cohort definitions, limited anticipation, never-/not-yet-treated comparison logic, and the aggregation schemes are lifted from CS (2021) and generalized to quantiles. A future implementation should reuse the CallawaySantAnna (g,t) cell enumeration and aggregation weight code paths. Note the CS-2021 sensitivity caveat (p. 32): when PT holds on average but not distributionally, CS-2021 is the safer tool - the docs comparison table should carry this. +- `ChangesInChanges` / `QDiD` (v1): 2x2 distributional siblings with a DIFFERENT identification strategy (monotone production function vs distributional PT + copula invariance). The Discussion's trade-offs belong in the choosing-an-estimator docs: this method is not scale-invariant (CiC is), does not allow post-policy changes in returns to unobserved skills (Bonhomme-Sauder does), but imposes no functional form and tolerates serial correlation and shock-heterogeneity correlation. +- The ATT is recoverable as a by-product by integrating the counterfactual quantile function (footnote 12, p. 13) - a natural cross-check against `CallawaySantAnna` output in any future test suite (under Assumptions 1-4 alone the ATT is identified, footnote 14, p. 16). +- Bootstrap utilities: the sup-t band with the IQR-based variance scale is the same Callaway-Li (2019) construction the CiC/QDiD v1 bootstrap docs cite; any implementation must reuse `safe_inference()` joint-NaN conventions from `diff_diff.utils` and resolve the band-centering discrepancy (Gaps item 1) before shipping bands. + +--- + +## Gaps and Uncertainties + +**Contradictions and as-printed flags (both versions preserved; do NOT resolve by guessing - check the PDF and any published version):** + +1. **Uniform-band centering (p. 22).** The paper prints the (1-alpha) uniform band as `C-hat_{QTT(tau)} = QTT-hat*(tau) +/- c^B_{1-alpha} Sigma-hat(tau)^{1/2} / sqrt(n)`, centered on `QTT-hat*(tau)` - the BOOTSTRAP estimate. In Callaway-Li (2019), whose empirical bootstrap this section adopts, the band is centered on the ORIGINAL-SAMPLE estimate `QTT-hat(tau)`. Either the printed `*` is a typo or the author intends a bootstrap-centered band; a future implementation must pick the CL19 centering deliberately and document the deviation, or reconcile against a published version. +2. **Assumption 4's time restriction as printed (p. 12).** The statement reads "For each r, t in {q, ..., T} such that `t >= q - rho`". The parallel objects elsewhere in the paper (the QTT definitions on pp. 18-19, Assumption 3's `t < r - rho` cutoff, and the base period `r - rho - 1`) are all expressed relative to the cohort date `r`, so `q` may be a typo for `r` - but the extraction transcribed the printed `q`, and this review does not resolve it. Verify against any published version before implementing the assumption's period-eligibility logic. +3. **`Y_t(inf)` notation in the p. 18 QTT definition.** The post-identification `QTT_{r,t,rho}(tau)` on p. 18 is printed with what appears to be `F^{-1}_{Y_t(infinity)|d_r=1}(tau)` as the subtrahend (the never-treated potential outcome), while the p. 19 restatement of the same object uses `F^{-1}_{Y_t(0)|d_r=1}(tau)`. The two notations are used interchangeably for the never-treated potential outcome; transcribed as printed, not normalized. +4. **Duplicate "(A.11)" display label (p. 40).** The final display of the Proposition 1 proof chain (the IPW weight form `E[(C p_r(X) / (p_r (1 - p_r(X)))) 1{Delta Y_t <= delta}]`) is tagged "(A.11)", duplicating the (A.11) of the Alternative Proof on p. 39 - presumably a typo for (A.15) or a back-reference to Eq. (3). Do not cite "(A.11)" without page context. The same chain also mixes `P_r(X)` and `p_r(X)` capitalization (as printed). + +**Verified quirks (not contradictions, but easy to trip over):** + +5. Assumption 2's last treatment subscript is typeset `tau` (`D_{i,tau}`) in the printed statement - transcribed as printed. +6. The Case A step-2 estimator (p. 21) prints the first composed term as `F-hat^{-1}_{DeltaY|C=1}(F-hat_{DeltaY|C=1}(.))` - both conditionals never-treated, an identity composition under continuity. This mirrors the (A.6) -> (A.7) simplification (Assumption 4 substitutes the never-treated change quantile for the treated one); an implementation can skip the redundant composition but should document why. +7. Appendix A notation slips (as printed): the support of the pre-period level is written `Y_{t-1}(0)` in one place and `Y_{r-1}(0)` elsewhere (p. 37); the (A.9)/(A.10) rank definitions carry redundant conditioning in the printed `v_i^r`/`v_i^0` definitions; Corollary 1's imputed change ends with `- Y_{r-1}` (no `(0)`). +8. Table C.1's notes boilerplate references 'UNC' rows that do not exist in the no-covariate DGP 1 table (p. 43). +9. DGP 3 notation clash (p. 25): `T` denotes both the number of periods and a chi-squared(1) random variable in `Z_r = r + T`. + +**Version and coverage gaps:** + +10. **Preprint status.** arXiv v2 is explicitly "a preliminary version" (p. 1 footnote); no published version of record exists as of 2026-07-12. Any published version may renumber assumptions/equations/tables and may resolve items 1-4 above; re-verify before citing in shipped docs. +11. **No empirical application** exists in this version (simulations only) - there is no published-number replication target; parity work must go through the author's unreleased Stata `qtt` ado or simulation-based golden values. +12. **No new asymptotic theory.** Section 3 states no theorem; consistency and the functional CLT are inherited from Callaway and Li (2019) via footnote 16 with the claim that "the only difference lies in the construction of the benchmark, not in the estimator itself." Hadamard-differentiability machinery is not restated; formal proofs of the staggered extension's asymptotics are not in the paper. +13. **External proof dependencies.** Lemma A.1 and Lemma A.2 (Sklar's-theorem results used in the Theorem 1 proof) live in Appendix A of Callaway and Li (2019) and are deliberately not reproduced (p. 37). The RCS-with-covariates IPW path defers the modified generalized propensity score to Appendix B of Callaway and Sant'Anna (2021). +14. **Wild/subcluster bootstrap is a pointer, not a procedure** (p. 22): the clustered-inference route is described as adapting MacKinnon-Webb (2018) "with minor modifications" with no algorithmic detail; a future implementation would have to derive the QTT adaptation itself. +15. **Extraction coverage.** This review synthesizes three extraction passes (pp. 1-20, pp. 16-35, pp. 31-44; printed page = PDF page) with overlaps on pp. 16-20 and 31-35 reconciled as documented; References (pp. 33-36) were skipped by design. Tables 2-4 numbers above are the selected values transcribed by the extraction, not full tables. +16. **No verified reference implementation** (see Reference implementation(s) above); in particular the R `ecic` package is NOT an implementation of this paper despite the overlapping "staggered distributional DiD" label. diff --git a/docs/methodology/papers/melly-santangelo-2015-review.md b/docs/methodology/papers/melly-santangelo-2015-review.md new file mode 100644 index 000000000..7686b43f1 --- /dev/null +++ b/docs/methodology/papers/melly-santangelo-2015-review.md @@ -0,0 +1,522 @@ +# Paper Review: The Changes-in-Changes Model with Covariates + +**Authors:** Blaise Melly and Giulia Santangelo +**Citation:** Melly, B., & Santangelo, G. (2015). The Changes-in-Changes Model with Covariates. *Working paper*, Bern University and European Commission - JRC. +**PDF reviewed:** **October 2015 working-paper version** (title page: "First version: January 2015. This version: October 2015. Preliminary!"; 32 PDF pages, printed page = PDF page; main text pp. 1-25, Figures pp. 26-30, References pp. 31-32). Distributed via Blaise Melly's website (https://sites.google.com/site/blaisemelly/home/computer-programs/cic_stata). Per the project's PDFs-never-committed convention the local PDF is kept outside the repository (gitignored `papers/melly-santangelo-cic-covariates.pdf`). This paper remains an unpublished working paper as of the review date (2026-07-12); it is nonetheless the standard reference for CiC with covariates. All equation/assumption/theorem numbers below are pinned to the October 2015 draft - later drafts, if any, may renumber. +**Review date:** 2026-07-12 + +--- + +## Methodology Registry Entry + +*Drafted in docs/methodology/REGISTRY.md format for FUTURE use. Do not copy this section into the registry now - see the status line below.* + +## CiC with Covariates (Melly-Santangelo) + +**Status: NOT shipping in diff-diff CiC/QDiD v1 (scope decision 2026-07-12).** Covariates are explicitly deferred from v1; this review documents the covariate extension so the deferral is informed and any future covariates PR starts from a reviewed scope. + +**Primary source:** Melly, B., & Santangelo, G. (2015). The Changes-in-Changes Model with Covariates. *Working paper*, Bern University and European Commission - JRC. Distributed via https://sites.google.com/site/blaisemelly/home/computer-programs/cic_stata. +- Layout: Sections 1 (Introduction, pp. 2-3), 2 (Model and identification, pp. 4-9), 3 (Estimation, pp. 10-12), 4 (Asymptotic results, pp. 12-21; 4.4 "Inference" pp. 19-21 includes the time-invariance specification test), 5 (Application, pp. 21-24), 6 (Conclusion, pp. 24-25); Figures 1-5 on pp. 26-30; References pp. 31-32. +- **Numbering note (as extracted):** results are numbered sequentially across result types - Proposition 1 (p. 6), Lemmas 2-3 (p. 14), Theorem 4 (p. 15), Corollaries 5-6 (p. 16), Theorem 7 (pp. 16-17), Corollaries 8-9 (p. 18), Theorem 10 (p. 20). There is no "Lemma 1" or "Theorem 1". Equations share one sequence (1)-(15). The draft is flagged "Preliminary!" on its title page and contains printed typos - see the suspected-typos block in Gaps and Uncertainties. + +**Motivation (Introduction, pp. 2-3; DiD-bias illustration pp. 22-23, Figures 1-2):** + +Athey-Imbens (2006) CiC does not depend on the scale of the dependent variable and recovers the whole counterfactual distribution; its main identifying restriction is time invariance of the distribution of unobservables within each group (p. 2). For covariates, AI suggest either (a) a fully nonparametric strategy, which "naturally suffers from the curse of dimensionality and is not practicable in many applications," or (b) a parametric strategy based on additive separability (OLS regression, then unconditional CiC on the residuals), which "imposes restrictions that limit the appeal of the method to analyze heterogeneous effects" - the paper argues it would be contradictory to use CiC (motivated by non-additive effects) while imposing additive separability for covariates (p. 2). Per Lechner's (2011) survey, the lack of a tractable estimator with covariates is one of the main reasons for the low number of empirical CiC applications (p. 2). + +Why covariates matter (p. 2): time-constant covariates help when trends differ by covariates; time-varying covariates capture time-varying differences between groups and allow linear trends as regressors, relaxing the time invariance assumption; even when time invariance holds unconditionally, covariates can bring efficiency gains. Conditioning on X *weakens* the unconditional AI time-invariance requirement (Assumption 3 below is imposed conditionally on X): unconditional CiC assumptions can fail when the conditional ones hold - e.g., when group covariate compositions differ and trends vary by covariates. + +The paper also quantifies how badly the common empirical practice of DiD-on-indicators fails for distributional targets (pp. 22-23; Figures 1-2, pp. 26-27): +- Setup: `Y_i ~ N(0.5*T + 0.5*G, 1)` with a uniformly zero treatment effect (no TxG interaction). +- DiD applied to `1(Y <= y)` is inconsistent except in three knife-edge cases (p. 22): "(i) There is no time effect. (ii) There is no group effect. (iii) The dependent variable is uniformly distributed." Root cause: "the time effect and the group effect cannot be additive if Y is not uniformly distributed." +- Analytic probability limit of the DiD estimator for `1(Y <= -0.5)` (p. 23): `Phi(-1.5) - Phi(-1) - Phi(-1) + Phi(-0.5) ~= 0.058`, "large relative to the observe[d] probability of 1(Y <= -0.5) which is 0.067." +- Figure 1 (p. 26): asymptotic DiD bias for the distribution treatment effect across all cutoffs; positive at the lower tail, negative at the upper tail (roughly +/-0.057 over cutoffs -3 to 4). +- Figure 2 (p. 27): asymptotic bias of the QTE implied by DiD on the distribution function; strongly negative at low quantiles (approaching about -1.5 as tau -> 0), rising through zero mid-distribution, positive (~+0.3) near tau = 1 - "very misleading given that the true effects are zero uniformly over (0,1). Therefore, the results reported by Almond, Hoynes, and Schanzenbach (2011) for the effects on the distribution should be taken with caution" (p. 23). +- Footnote 10 (p. 22): additional issues from linear regression of a binary dependent variable on a non-saturated covariate set, especially for conditional probabilities. +- No Monte Carlo simulations appear anywhere in the paper - this analytic illustration is the only numerical-bias evidence. + +**Model (Section 2.1, pp. 4-6):** + +Two periods and two groups for Sections 2-4 (the paper states it uses "as much as possible the same notation as AI"; the multi-group/multi-period setup appears only in the application, Section 5). Observed data: `(Y, G, T, X)`. + +| Symbol | Meaning | +|---|---| +| `G in {0,1}` | Group indicator; group 1 is the treatment group | +| `T in {0,1}` | Time period; only group 1 in period 1 is treated | +| `X` | Covariate vector with support `X` (script) | +| `Y^N`, `Y^I` | Potential outcomes without / with treatment | +| `I = G*T` | Treatment indicator; realized outcome `Y = Y^N*(1-I) + Y^I*I` (p. 4, unnumbered) | +| `U` | Unobservable component of Y (Assumption 1) | +| `Y^N_gtx ~d Y^N | G=g, T=t, X=x` (etc.) | Distributional shorthands (p. 5); conditional CDFs `F_{Y^N|gtx}`, `F_{Y^I|gtx}`, `F_{Y|gtx}`, `F_{U|gx}` with supports `Y^N_gtx`, `Y^I_gtx`, `Y_gtx`, `U_gx` | +| `F_{X|11}` | Covariate distribution in group 1, period 1 (integration measure for the unconditional estimands) | +| `alpha_gt = Pr(T_i=t, G_i=g) > 0` | Cell probabilities (Assumption 5(ii)) | +| `n_11` | Number of group-1 period-1 observations | +| `u_1 < ... < u_S`, mesh width `delta` | Quantile grid for the QR process, `delta*sqrt(n) -> 0` (p. 11) | + +The distribution of `Y^I_11x` is identified directly by the observed `Y_11x`; the identification problem is `Y^N_11x` (p. 5). Footnote 2 (p. 5): the paper focuses on group 1 / period 1 because the treated outcome is only observed there; effects for group 0 or period 0 would require much stronger assumptions. + +Discrete outcomes (p. 6): the model in principle allows them, but strict monotonicity is very restrictive there (the discreteness must come from the unobserved term; ordered logit or rounded outcomes are excluded), so the authors "do not advise using this model for discrete outcomes." Footnote 4 (p. 6): AI's weakly-increasing-`h` case gives only partial identification; deferred to future work. The Conclusion (p. 25) adds that QR-based estimators "are not well suited for discrete outcomes" and flags distribution regression as the natural route for a discrete/mixed-outcome extension. + +**Assumptions (exact statements; conditional restatement of Athey-Imbens Assumptions 3.1-3.4 plus estimation conditions):** + +*Assumption 1 (potential outcome) - p. 4:* + +> The outcome of an individual in the absence of intervention satisfies the relationship `Y^N = h(X, T, U)`. + +Without the other assumptions this does not restrict the DGP - any `Y^N` can be represented as `h(X, T, U)` for unrestricted `h` and `U` (p. 4). Note the control outcome is a *nonseparable* function of covariates, time, and the unobservable. + +*Assumption 2 (strict monotonicity) - p. 5:* + +> The production function `h(t, x, u)` is strictly increasing in `u` for `t in {0, 1}` and for all `x in X`. + +*Assumption 3 (time invariance) - p. 5:* + +> We have `U _||_ T | G, X`. + +The key restriction: it allows extrapolation of ranks from one period to the other and is "the CIC counterpart of the common trend assumption in the DID model" (p. 5). Only equality in distribution over time is required (rank similarity; footnote 3 cites Chernozhukov and Hansen 2005), not the stronger rank invariance/preservation where `U` is constant per individual. This is AI time invariance imposed *conditionally on X* (as well as G). + +*Assumption 4 (support) - p. 5:* + +> `U_1x subset U_0x` for all `x in X`. + +Guarantees group-0 observations with similar `U` as group-1 observations exist. Testable implication: `Y_10x subset Y_00x` for all `x in X`; if violated, only the part of the counterfactual distribution on the overlap of `U_0x` with `U_1x` is identified (p. 6). Assumptions 1-3 are not testable with only 2 periods and 2 groups; testing time invariance with more periods is in Section 4.4 (p. 6). + +Consequence of Assumptions 1-2 (p. 5, unnumbered): `F_{Y^N}^{-1}(tau | X=x, T=t) = h(x, t, F_U^{-1}(tau | X=x, T=t))`. `h` is not identified without a normalization (e.g., `U | X=x, T=t ~ U(0,1)`, under which `h` is the quantile function of `Y^N`); the normalization is not needed because `h` is not the object of interest (p. 5; cites Matzkin 2003). + +*Assumption 5 (Data generating process) - p. 12:* + +> (i) Conditional on `T_i = t` and `G_i = g`, `(Y_i, X_i)` is a random draw from the subpopulation with `G_i = g` during period `t` that has probability law `P_gt`. (ii) For all `t, g in {0,1}`, `alpha_gt := Pr(T_i = t, G_i = g) > 0`. + +Repeated cross-sectional sampling (matches the application). For panel data the results must be modified: as in Section 5.3 of AI, additional terms would account for correlation of estimated conditional distributions over time within groups (p. 12). + +*Assumption 6 (Quantile regression regularity conditions) - p. 13:* + +> (i) The conditional quantile function takes the form `F_{Y|G,T,X}^{-1}(u | g, t, x) = x' beta_gt(u)` for all `u in (0,1)` and `x in X`. (ii) The conditional density function `f_{Y|G,T,X}(y|x)` exists, is uniformly bounded, and is uniformly continuous in `(y, x)` in the support `Y_gt x X`, which is a compact subset of `R^{K+1}`. (iii) The minimal eigenvalue of `J_gt(u) := E[f_{Y|gtX}(X'beta(u)) X X']` is bounded away from zero uniformly over `u`. (iv) `E||X||^{2+eps} < infinity` for some `eps > 0`. + +Described as standard QR regularity conditions. Part (i) is the semiparametric restriction: linear-in-parameters conditional quantile functions in each of the four `(g,t)` cells. The paper notes QR can approximate the true conditional quantile function arbitrarily well with rich enough transformations of the regressors when Y has a smooth conditional density, and that distribution regression "could be used as well" as the first stage (p. 10). + +**Target parameters (pp. 8-10):** + +Conditional QTE process for group 1 in period 1 at covariate value `x` (p. 8, unnumbered; uses `F_{Y|11x}^{-1}(tau) = F_{Y^I|11x}^{-1}(tau)` since group 1 is treated in period 1): + +``` +Delta^{QE}(.|x) = F_{Y^I|11x}^{-1}(.) - F_{Y^N|11x}^{-1}(.) + = F_{Y|11x}^{-1}(.) - F_{Y|01x}^{-1}( F_{Y|00x}( F_{Y|10x}^{-1}(.) ) ) +``` + +Conditional average and distribution effects (p. 9, unnumbered): + +``` +Delta^{AE}(x) = E[Y^I_11x] - E[Y^N_11x] +Delta^{DE}(.|x) = F_{Y^I|11x}(.) - F_{Y^N|11x}(.) +``` + +These are identified for all `x` in the support (heterogeneity analysis in observables), but high-dimensional functions are hard to communicate, so the primary interest is unconditional effects (p. 9). Unconditional distributions (p. 9, unnumbered), both integrating over `F_{X|11}` - the covariate distribution of the *treated group in period 1* - so the estimands are effects on the treated: + +``` +F_{Y^I|11}(y) = F_{Y|11}(y) = INT_X F_{Y|11x}(y) dF_{X|11}(x) + +F_{Y^N|11}(y) = INT_X F_{Y^N|11x}(y) dF_{X|11}(x) + = INT_X F_{Y|10x}( F_{Y|00x}^{-1}( F_{Y|01x}(y) ) ) dF_{X|11}(x) +``` + +"All the elements are observable in the last expression." Headline estimand, the unconditional quantile effect process (p. 9, unnumbered): + +``` +Delta^{QE}(.) = F_{Y^I|11}^{-1}(.) - F_{Y^N|11}^{-1}(.) +``` + +All functionals of these marginal distributions are identified: distribution functions, quantile functions, quantile effects, distribution effects, average effects, Lorenz curves, Gini coefficients (p. 3). + +Distribution of individual treatment effects (p. 9, eq. (6)): with panel data and strengthening time invariance (rank similarity) to rank invariance: + +``` +(6) F_{Delta|11}(delta) := F_{Y^I - Y^N|11}(delta) + = INT_X F_{Y^I - Y^N|11x}(delta) dF_{X|11}(x) + = INT_X INT_0^1 1( Delta^{QE}(.|x) <= delta ) dF_{X|11}(x) +``` + +(inner integral over the quantile index; identifies e.g. the proportion who benefit). Not pursued further - the application has only repeated cross-sections; the limiting distribution of the plug-in estimator based on (6) can be derived by combining the paper's results with Appendix C of Chernozhukov, Fernandez-Val, and Melly (2009) (p. 10). + +**Identification (Proposition 1, p. 6; proof pp. 7-8, eqs. (1)-(5)):** + +> **Proposition 1 (identification of the conditional distribution).** Suppose that Assumptions 1-4 hold and let `0 < tau < 1`. Then the `tau` quantile of `Y^N_11x` is identified for all `x in X` with + +``` +F_{Y^N|11x}^{-1}(tau) = F_{Y|01x}^{-1}( F_{Y|00x}( F_{Y|10x}^{-1}(tau) ) ). +``` + +This "trivially extends Theorem 3.1 in AI to the case with covariates" (p. 6) - the Athey-Imbens CiC quantile map applied *within each covariate cell x*. Footnote 5 credits Fortin and Lemieux (1998) and Altonji and Blank (1999) with similar expressions in a different context. + +Proof chain (pp. 7-8). Central equation, valid for all four `(g,t)` combinations, using invertibility of `h(t,x,u)` in `u` with inverse `h^{-1}(t,x,y)`: + +``` +(1) F_{Y^N|gtx}(y) = Pr(h(t,x,U) <= y | G=g, T=t, X=x) + = Pr(U <= h^{-1}(t,x,y) | G=g, T=t, X=x) + = Pr(U <= h^{-1}(t,x,y) | G=g, X=x) [time invariance] + = F_{U|gx}(h^{-1}(t,x,y)) +``` + +Applying (1) with `(g,t) = (0,0)`, substituting `y = h(0,x,u)`, and applying `F_{Y|00x}^{-1}`, for all `u in U_0x`: + +``` +(2) h(0, x, u) = F_{Y|00x}^{-1}( F_{U|0x}(u) ) +``` + +Applying (1) with `(g,t) = (0,1)`, using `h^{-1}(1,x,y) in U_0x` for all `y in Y_01x`, and applying `F_{U|0x}^{-1}` to both sides of `F_{Y|01x}(y) = F_{U|0x}(h^{-1}(1,x,y))`: + +``` +(3) F_{U|0x}^{-1}( F_{Y|01x}(y) ) = h^{-1}(1, x, y) +``` + +Combining (2) and (3), for all `y in Y_01x`: + +``` +(4) h(0, x, h^{-1}(1, x, y)) = F_{Y|00x}^{-1}( F_{Y|01x}(y) ) +``` + +Interpretation (p. 8): `h(0, x, h^{-1}(1, x, y))` is the period-0 outcome for an individual with characteristics `x` and the `u`-realization corresponding to outcome `y` in group 0, period 1; (4) shows it is determined by observable distributions. Applying (1) with `(g,t) = (1,0)` and `y = h(0,x,u)`: + +``` +(5) F_{Y|10x}(y) = F_{U|1x}( h^{-1}(0, x, h(0, x, u)) ) = F_{U|1x}(u) +``` + +Combining (4) and (5) and substituting into (1) with `(g,t) = (1,1)`, for all `y in Y_01` (p. 8, unnumbered): + +``` +F_{Y^N|11x}(y) = F_{U|1x}( h^{-1}(1,x,y) ) = F_{Y|10x}( h(0,x,h^{-1}(1,x,y)) ) + = F_{Y|10x}( F_{Y|00x}^{-1}( F_{Y|01x}(y) ) ) +``` + +By Assumption 4, `Y^N_11 subset Y_01`, so the directly observable `F_{Y|10x}`, `F_{Y|00x}`, `F_{Y|01x}` identify `F_{Y^N|11x}`; inverting gives Proposition 1. Intuition (p. 8): an individual at a given quantile of one group's period-0 conditional outcome distribution maps to a relative rank in the other group's same-period distribution; by time invariance this relative rank is unchanged over time. (The printed intuition sentence swaps group labels relative to the formula - see suspected typos; transcribed here consistently with the displays.) + +Identification of the *unconditional* counterfactual follows constructively by integrating `F_{Y^N|11x}` over `F_{X|11}` (p. 9 displays); there is no separately numbered theorem for the unconditional identification step - it is the display sequence on p. 9. + +**Estimator (Section 3, pp. 10-12) - the four-step QR pipeline:** + +Plug-in principle: identification is constructive; replace all observables by consistent estimators. At least three conditional distribution/quantile functions must be estimated. + +*Step 1 - four QR coefficient processes (p. 10).* Linear quantile regression (Koenker and Bassett 1978) separately in the four `(g,t)` samples: + +``` +beta_hat_gt(u) = argmin_{b in R^{K+1}} SUM_{i: G_i=g, T_i=t} (u - 1(Y_i <= X_i'b)) * (Y_i - X_i'b) +``` + +giving `F_hat_{Y|gtx}^{-1}(u) = x' beta_hat_gt(u)`. Practical discretization: the QR process is estimated on a fine mesh `u_1 < ... < u_S` with mesh width `delta` such that `delta*sqrt(n) -> 0`. Footnote 7 (p. 11): estimating *all* distinct quantile regressions is possible - the estimates change at only finitely many points, `O(n log n)` distinct solutions (Portnoy 1991). Computation is reduced ~10x with the algorithms of Melly (2014). Tail trimming "seems unavoidable in practice" - the paper abstracts from it for notational simplicity; formulas can be adapted as in Chernozhukov, Fernandez-Val, and Melly (2013). (The draft's tail-trimming sentence is printed incomplete - see suspected typos.) + +*Step 2 - monotonization via the integrated-indicator CDF representation (pp. 10-11).* Estimated conditional quantile functions may be non-monotonic in `u` and cannot be directly inverted. Instead use the sample analog of `F_{Y|gtx}(y) = INT_0^1 1( F_{Y|gtx}^{-1}(u) <= y ) du`: + +``` +F_hat_{Y|gtx}(y) = INT_0^1 1( F_hat_{Y|gtx}^{-1}(u) <= y ) du + = delta * SUM_{s=1}^S 1( x' beta_hat_gt(u_s) <= y ) [p. 11 discretization] +``` + +Statistical properties of this monotonized (rearrangement-type) CDF estimator are studied in Chernozhukov, Fernandez-Val, and Galichon (2010). This is the key extra numerical ingredient relative to covariate-free CiC. + +*Step 3 - conditional CiC transformation at each x (pp. 11-12).* The inner probability-probability step `F_hat_{Y|00x}(x'beta_hat_10(tau))` is computed by the monotonized-CDF integral, and its value is the quantile index at which the group-0 period-1 QR process is evaluated: + +``` +F_hat_{Y^N|11x}^{-1}(tau) = x' beta_hat_01( INT_0^1 1( x' beta_hat_00(u) <= x' beta_hat_10(tau) ) du ) +F_hat_{Y^I|11x}^{-1}(tau) = x' beta_hat_11(tau) +``` + +Conditional QTE estimator (p. 12, eq. (7)): + +``` +(7) Delta_hat^{QE}(.|x) = x' ( beta_hat_11(tau) - beta_hat_01( INT_0^1 1( x'beta_hat_00(u) <= x'beta_hat_10(tau) ) du ) ) +``` + +*Step 4 - integration over the treated-group covariate distribution (p. 12).* Integrate the conditional counterfactual CDF over the *empirical* distribution of X among group-1 period-1 observations: + +``` +F_hat_{Y^N|11}(y) = INT_X F_hat_{Y^N|11x}(y) dF_{X|11}(x) + = (1/n_11) SUM_{i: G_i=1, T_i=1} SUM_{j=1}^S 1( x_i' beta_hat_01( INT_0^1 1( x_i'beta_hat_00(u_s) <= x_i'beta_hat_10(tau) ) du ) <= y ) +``` + +(transcribed as printed - the draft's indexing is inconsistent here and the mesh-width factor is missing; see suspected typos. The intended estimator, consistent with the p. 11 discretization, evaluates the outer indicator on the mesh `{u_j}` - i.e., a counterfactual tau-grid - carries the `delta` weight, and averages, mirroring the treated-side formula.) `F_{Y^I|11}(y)` can be estimated by the empirical distribution of `Y_i` in cell (1,1), or - the option used in the application, keeping treated and counterfactual sides methodologically symmetric - by integrating the QR-estimated conditional distribution: + +``` +F_hat_{Y^I|11}(y) = (1/n_11) SUM_{i: G_i=1, T_i=1} SUM_{j=1}^S 1( x_i' beta_hat_11(u_s) <= y ) +``` + +(same printed index inconsistency and implicit mesh weight). Unconditional QTE estimator (p. 12, eq. (8)), taking (generalized) inverses of the integrated CDF estimators: + +``` +(8) Delta_hat^{QE}(.) = F_hat_{Y^I|11}^{-1}(y) - F_hat_{Y^N|11}^{-1}(y) +``` + +(printed with argument `(y)` though the process is indexed by the quantile level - see suspected typos). + +**Asymptotic theory (Section 4, pp. 12-18):** + +All limit results are at the parametric `sqrt(n)` rate (a consequence of the linear QR restriction, Assumption 6(i)), as processes, enabling uniform (functional) inference. Section 4.2's "four main ingredients" (pp. 13-15) combine via the functional delta method: + +*Ingredient 1 - QR process FCLTs (p. 13).* Under Assumptions 5 and 6, Corollary 5.2 of Chernozhukov, Fernandez-Val, and Melly (2013) implies, for all `t, g in {0,1}`, as `n -> infinity` in `l^inf(U_g X)`: + +``` +(9) sqrt(n) ( F_hat_{Y|gtx}^{-1}(u) - F_{Y|gtx}^{-1}(u) ) ⇝ Z^Q_gt(u, x) +``` + +as stochastic processes indexed by `(u,x)`, with independent tight zero-mean Gaussian limits and covariance function + +``` +V^Q_gt(u, x, u~, x~) = alpha_gt^{-1} * x' J_gt(u)^{-1} (min(u,u~) - u*u~) E[XX' | G=g, T=t] J_gt(u~)^{-1} x~ +``` + +(the evaluation point of the second Jacobian is hard to resolve at print resolution; transcribed as `J_gt(u~)^{-1}`, the standard QR-process covariance form). The same corollary gives the conditional distribution process, as `n -> infinity` in `l^inf(Y_gt X)`: + +``` +(10) sqrt(n*alpha_gt) ( F_hat_{Y|gtx}(y) - F_{Y|gtx}(y) ) ⇝ Z^F_gt(u, x) := -f_{Y|gtx}(y) Z_gt( F_{Y|gtx}(y), x ) +``` + +as processes indexed by `(y,x)` (the index is printed `(u,x)` and the scaling differs from (9) by `sqrt(alpha_gt)`, as printed - see suspected typos), with covariance + +``` +V^F_gt(y, x, y~, x~) = f_{Y|gtx}(y) f_{Y|gtx~}(y~) V^Q_gt( F_{Y|gtx}(y), x, F_{Y|gtx~}(y~), x~ ) +``` + +*Ingredient 2 - Lemma 2 (quantile-quantile transformation, p. 14).* For CDFs F, G with compact support, continuously differentiable with strictly positive densities f, g, the map `phi^{QQ}(F, G) = F o G^{-1}` is Hadamard-differentiable at (F, G) tangentially to functions `h_1, h_2`, with derivative map + +``` +phi'_{F,G}(h_1, h_2) = h_1 o G^{-1} - ( f o G^{-1} / g o G^{-1} ) * h_2 o G^{-1} +``` + +(proof via Hadamard differentiability of the inverse map, Lemma 3.9.23(ii), and the chain rule, Lemma 3.9.27, in Van der Vaart and Wellner 1996; footnote 8 also cites problem 4, p. 398 of VdV-W). + +*Ingredient 3 - Lemma 3 (probability-probability transformation, p. 14).* For F with compact support, continuously differentiable, strictly positive density f, the map `phi^{PP}(F, G) = F^{-1} o G` is Hadamard-differentiable with derivative map + +``` +phi'_{F,G}(h_1, h_2) = -( h_1 / f ) o F^{-1} o G - ( h_2 / (f o F^{-1}) ) o G +``` + +*Ingredient 4 - counterfactual operator (p. 15).* Lemma D.1 of Chernozhukov, Fernandez-Val, and Melly (2013): `phi^C(F, G) = INT F(y, x) dG(x)` is Hadamard-differentiable with derivative map + +``` +phi^C_{F_{Y|X}, F_X}(gamma, pi) = INT gamma(y, x) dF_X(x) + pi( F_{Y|X}(y|x) ) +``` + +This is the operator integrating the conditional distribution over the covariate distribution. + +> **Theorem 4 (limiting distribution of the conditional CiC estimator) - p. 15.** Suppose that Assumptions 1 to 6 hold. Then, (i) `sqrt(n)(F_hat_{Y^I|11x}^{-1}(tau) - F_{Y^I|11x}^{-1}(tau)) ⇝ Z^Q_11(tau, x)` and `sqrt(n)(F_hat_{Y^N|11x}^{-1}(tau) - F_{Y^N|11x}^{-1}(tau)) ⇝ Z^Q_N(tau, x)` as stochastic processes indexed by `(tau, x) in (0,1) x X` and where `Z^Q_11(tau, x)` and `Z^Q_N(tau, x)` are independent tight zero-mean Gaussian process defined in (9) and (11). + +Proof intermediates (pp. 15-16): the treated side is (9) directly. For the counterfactual side, by Lemma 2 (QQ step) and the functional delta method: + +``` +sqrt(n)( F_hat_{Y|00x}(F_hat_{Y|10x}^{-1}(tau)) - F_{Y|00x}(F_{Y|10x}^{-1}(tau)) ) ⇝ Z^R(tau, x) + := Z^F_00( F_{Y|10x}^{-1}(tau), x ) + [ f_{Y|00x}(F_{Y|10x}^{-1}(tau)) / f_{Y|10x}(F_{Y|10x}^{-1}(tau)) ] Z^Q_10(tau, x) +``` + +then by Lemma 3 (PP step) and the delta method (p. 16): + +``` +(11) Z^Q_N(tau, x) := Z^Q_01( F_{Y|00x}(F_{Y|10x}^{-1}(tau)), x ) + + Z^R(tau, x) / f_{Y|01x}( F_{Y|01x}^{-1}( F_{Y|00x}(F_{Y|10x}^{-1}(tau)) ) ) +``` + +(the density in the denominator is the density of `Y|0,1,x` evaluated at the counterfactual quantile). + +> **Corollary 5 (conditional QTE process) - p. 16.** Under Assumptions 1-6, `sqrt(n)(Delta_hat^{QE}(tau|x) - Delta^{QE}(tau|x)) ⇝ Z^{QTE}(tau, x) = Z^Q_11(tau,x) - Z^Q_N(tau,x)`, a tight zero-mean Gaussian process indexed by `(tau, x) in (0,1) x X`. Proof: "Trivial by the functional delta method." + +> **Corollary 6 (Hadamard-differentiable functionals, conditional) - p. 16.** For `phi(F_{Y^I|11x}^{-1}, F_{Y^N|11x}^{-1}, w)` Hadamard differentiable with derivatives `phi'_I` and `phi'_N`: `sqrt(n)(phi(F_hat^{-1}_{Y^I|11x}, F_hat^{-1}_{Y^N|11x}, w) - phi(F^{-1}_{Y^I|11x}, F^{-1}_{Y^N|11x}, w)) ⇝ phi'_I(Z^Q_11(., x), w) + phi'_N(Z^Q_N(., x), w)` as a stochastic process indexed by `w`. + +> **Theorem 7 (limiting distribution of the unconditional quantile processes) - pp. 16-17.** Under Assumptions 1-6, `sqrt(n)(F_hat_{Y^I|11}^{-1}(tau) - F_{Y^I|11}^{-1}(tau)) ⇝ Z^Q_11(tau)` and `sqrt(n)(F_hat_{Y^N|11}^{-1}(tau) - F_{Y^N|11}^{-1}(tau)) ⇝ Z^Q_N(tau)` **jointly** as stochastic processes indexed by `tau in (0,1)`, with tight zero-mean Gaussian limits defined in (13) and (12). + +Proof intermediates (p. 17): applying Corollary 6 with `phi(.) = F_{Y^N|11x}(y)` gives the conditional distribution process `sqrt(n)(F_hat_{Y^N|11x}(y) - F_{Y^N|11x}(y)) ⇝ Z^F_N(y,x) = f_{Y^N|11x}(y) Z^Q_N(F_{Y^N|11x}(y), x)`. By the Donsker theorem, the empirical covariate distribution in cell (1,1) satisfies + +``` +(1/sqrt(n)) SUM_{i: G_i=T_i=1} ( f(Y_i, X_i) - INT f(Y_i, X_i) dP_11 ) ⇝ Z^X_11( f(y,x) ) +``` + +indexed by `f in F`, a universal Donsker class; the limits are tight `P_11`-Brownian bridges with covariance `V^X_11(y,x,y~,x~) = alpha_11^{-1} * ( INT f(y,x) f(y~,x~) dP_11 - INT f(y,x) dP_11 INT f(y,x) dP_11 )`. By the functional delta method and the counterfactual-operator differentiability: + +``` +sqrt(n)( F_hat_{Y^N|11}(y) - F_{Y^N|11}(y) ) ⇝ INT_X Z^F_N(y,x) dF_{X|11}(x) + Z^X_11( F_{Y^N|11x}(y) ) := Z^F_N(y) +``` + +and finally (inverse map / delta method): + +``` +(12) sqrt(n)( F_hat_{Y^N|11}^{-1}(tau) - F_{Y^N|11}^{-1}(tau) ) ⇝ -Z^F_N( F_{Y^N|11}^{-1}(tau) ) / f_{Y^N|11}( F_{Y^N|11}^{-1}(tau) ) := Z^Q_N(tau) +``` + +For the treated outcome, directly from Theorem 4.1(ii) of Chernozhukov, Fernandez-Val, and Melly (2013): + +``` +sqrt(n)( F_hat_{Y^I|11}(y) - F_{Y^I|11}(y) ) ⇝ INT_X f_{Y|11x}(y) Z^Q_11( F_{Y|11x}(y), x ) dF_{X|11}(x) + Z^X_11( F_{Y|11x}(y) ) := Z^F_11(y) + +(13) sqrt(n)( F_hat_{Y^I|11}^{-1}(tau) - F_{Y^I|11}^{-1}(tau) ) ⇝ -Z^F_11( F_{Y|11}^{-1}(tau) ) / f_{Y|11}( F_{Y|11}^{-1}(tau) ) := Z^Q_11(tau) +``` + +> **Corollary 8 (unconditional QTE process) - p. 18.** `sqrt(n)(Delta_hat^{QE}(tau) - Delta^{QE}(tau)) ⇝ Z^{QTE}(tau) = Z^Q_11(tau) - Z^Q_N(tau)`, tight zero-mean Gaussian, indexed by `tau in (0,1)`. + +> **Corollary 9 (Hadamard-differentiable functionals, unconditional) - p. 18.** Same structure as Corollary 6 for functionals of the unconditional processes. + +Remarks on analytical SEs (p. 18): the functional results imply pointwise estimators (e.g., QTE at a single quantile) are asymptotically normal, and "these formulas can be used to develop analytical estimators of the standard errors. However, all asymptotic variances contain terms that are difficult to estimate such as the conditional density of the dependent variable given the covariates. Therefore, we suggest using resampling methods to estimate the standard errors of the estimates. This method also allows performing inference on the whole processes." The corollaries cover all Hadamard-differentiable functionals: ATE as a simple special case; Lorenz curve and Gini coefficient as a more involved one. Without covariates (X = constant, pointwise), the results simplify to AI; even there the paper contributes the limiting distribution of the whole quantile and distribution processes, with a simpler empirical-process-theory proof. + +**Bootstrap inference (Section 4.4 "Inference", pp. 19-21):** + +The paper proves validity of the **exchangeable bootstrap**, which "incorporates many popular forms of resampling as special cases, namely the empirical bootstrap, weighted bootstrap, m out of n bootstrap, and subsampling" (p. 19). Motivation: "in small samples with categorical covariates, we might want to use the weighted bootstrap to gain accuracy and robustness to 'small cells,' whereas in large samples, where computational tractability can be an important consideration, we might prefer subsampling" (p. 19). + +*Condition BW (bootstrap weights) - p. 19.* For each `t, g in {0,1}`, let `(w_gt1, ..., w_gtn_gt)` be an exchangeable (footnote 9: permutation-invariant), nonnegative random vector, independent of the data, such that for some `eps > 0`: + +``` +sup_{n_k} E[ w_gt1^{2+eps} ] < infinity +n_gt^{-1} SUM_{i=1}^{n_gt} ( w_gti - w_bar_gt )^2 ->_p 1 +w_bar_gt = n_gt^{-1} SUM_{i=1}^{n_gt} w_gti ->_p 1 +``` + +with the weight vectors independent across `(g,t)`. Multinomial weights with probabilities `(1/n_k, ..., 1/n_k)` give the empirical bootstrap. + +*Bootstrap versions of the estimators (pp. 19-20).* Weighted QR: + +``` +beta_hat*_gt(u) = argmin_{b in R^{K+1}} SUM_{i: G_i=g, T_i=t} w_gti * ( u - 1(Y_i <= X_i'b) ) * ( Y_i - X_i'b ) +``` + +bootstrap conditional quantile function `F_hat*^{-1}_{Y|gtx}(u) = x' beta_hat*_gt(u)`, and bootstrap unconditional counterfactual distribution (p. 20): + +``` +F_hat*_{Y^N|11}(y) = (1/n_11) SUM_{i: G_i=1, T_i=1} SUM_{j=1}^S w_11i * 1( x_i' beta_hat*_01( INT_0^1 1( x_i' beta_hat*_00(u_s) <= x_i' beta_hat*_10(tau) ) ) <= y ) +``` + +(the p. 20 display is typographically dense; the structure is the weighted empirical analog of the QQ/PP composition over the S-point u-grid, transcribed as printed modulo layout - see suspected typos). + +*Uniform confidence bands (p. 20).* An asymptotic simultaneous (1-alpha) band for the unconditional QTE process is defined by end-point functions + +``` +(14) Delta_hat^{QE}(tau)+/- = Delta_hat^{QE}(tau) +/- t_hat_{1-alpha} * Sigma_hat^{QE}(tau)^{1/2} / sqrt(n) + +(15) lim_{n->inf} Pr{ Delta^{QE}(tau) in [Delta_hat^{QE}(tau)^-, Delta_hat^{QE}(tau)^+] for all tau in (0,1) } = 1 - alpha +``` + +`Sigma_hat^{QE}(tau)` is a uniformly consistent estimator of the asymptotic variance function of `sqrt(n)(Delta_hat^{QE}(tau) - Delta^{QE}(tau))`. The critical value `t_hat_{1-alpha}` consistently estimates the (1-alpha)-quantile of the Kolmogorov-Smirnov maximal t-statistic `t = sup_{tau in (0,1)} sqrt(n) Sigma_hat^{QE}(tau)^{-1/2} |Delta_hat^{QE}(tau) - Delta^{QE}(tau)|`. Implementation: draw B bootstrap samples; set `t_hat_{1-alpha}` to the (1-alpha) sample quantile of `{t_hat_b : 1 <= b <= B}` where `t_hat_b = sup_{tau in (0,1)} Sigma_hat^{QE}(tau)^{-1/2} |sqrt(n)(Delta_hat^{QE}(tau)* - Delta_hat^{QE}(tau))|`. + +> **Theorem 10 (bootstrap validity, "third main result of this paper") - p. 20.** Assumptions 1 to 6 hold and the bootstrap weights follow the condition BW. Then, the exchangeable bootstrap consistently estimates the law of the limit stochastic processes in the Theorems and Corollaries 4 to 9. The confidence bands have a correct coverage probability. + +Proof: Corollary 5.2(ii) of Chernozhukov, Fernandez-Val, and Melly (2013) gives bootstrap validity for the QR-based conditional distribution/quantile estimators; the result follows by the functional delta method for the bootstrap (chapter 3.9, Van der Vaart and Wellner 1996) plus Hadamard differentiability of all functionals involved. + +*Functional hypothesis tests and the time-invariance specification test (p. 21).* The bands can test no-effect, positive-effect, or stochastic-dominance hypotheses (check whether the entire null falls within the band). They also give a **specification test of time invariance** when a second pre-treatment period exists: if `(Y, X)` is observed in period -1 with no one treated, Assumptions 1-4 imply + +``` +F_{Y,0-1x}( F^{-1}_{Y,1-1x}(tau) ) = F_{Y,00x}( F^{-1}_{Y,10x}(tau) ) for all tau in (0,1) and x in X +``` + +All quantiles and covariate values must be considered to detect deviations; Kolmogorov-Smirnov or Cramer-von-Mises type tests are "the natural way", with Theorem 10 justifying exchangeable-bootstrap critical values. + +**Empirical application (Section 5, pp. 21-24; Figures 3-5, pp. 28-30) - food stamps and the birthweight distribution:** + +- *Data and design:* same data as Almond, Hoynes, and Schanzenbach (2011) (AHS), complementing their DiD analysis with CiC. The Food Stamp Program rolled out county-by-county during the 1960s and early 1970s across roughly 3,100 U.S. counties; AHS analyzed 1968 (when ~40% of counties were already treated) to 1977. Melly-Santangelo must (i) exclude counties already treated at the start (their counterfactual is not identified) and (ii) stop in June 1974 (effects not identified once everyone is treated) (p. 21). +- *Sources (pp. 21-22):* administrative USDA annual reports on county caseloads (month/year of each county's FSP implementation = policy variable); Vital Statistics Natality Data from 1968 onward, ~2 million observations/year (100% or 50% sample of births by state-year) with birthweight, gender, race; covariates from the 1960 Census of Population, Census of Agriculture, and BEA REIS data. +- *Specification (p. 22):* results only for white mothers, using AHS's first specification: 1960 county variables (log population, % land in farming, % population black, urban, age below 5, age above 65, income < $3,000), each interacted with a linear time trend, plus per capita county transfer income (public assistance, medical care, retirement and disability benefits) and county real per capita income. +- *Practical complications (p. 23):* (1) 26 quarters -> 26 treatment-timing groups; any pair of periods identifies the effect for counties untreated in the first and treated in the second; presenting all pairs would be "almost impossible... and highly imprecise", so they present average results over all possible pairs, weighted to be representative of the treated counties in the data. (2) Time-trend-interacted covariates are not identified in period-by-period QR, so they estimate a single pooled quantile regression process for all non-treated observations, including indicator variables for the time periods and the groups. (3) Scale: CiC needs the whole micro-level distribution (no collapsing to county-quarter cell means), keeping above 5,000,000 observations; "This computational task becomes insurmountable if we need to bootstrap the results. Our solution was to use the newly developed quantile regression algorithms developed in Melly (2014) and subsampling instead of the bootstrap" (pp. 23-24). Footnote 11: each draw samples 500 of the 3,000+ counties, allowing arbitrary within-county correlation (cluster-level subsampling; the number of draws is not stated). +- *Findings (p. 24):* + - Figure 3 (p. 28): QTE of food stamps on birthweight with subsampling-based uniform band. Positive effect, as in AHS; "The absolute value of our estimated effect is larger but the standard errors are also larger." Effects stronger at the lower tail ("certainly a good outcome for the policy because this is where it matters the most"); contrary to the original article, also a larger effect at the upper end (visually: ~100-150 g near tau ~ 0, a plateau of roughly ~5-15 g through the middle, rising again to ~100-150 g near tau ~ 1; bands exclude zero) - a U-shaped QTE. + - Figure 4 (p. 29): quantile functions of control vs treated potential outcomes (roughly 1,500-4,600 g): "While the effects we find are strongly significant, this figure makes clear that they are not large in absolute value. The two quantile functions are almost impossible to distinguish." + - Figure 5 (p. 30): QTE with vs without covariates: "in this application the results depend on the presence of covariates. Without covariates the effects are very close to zero for most of the distribution. On the other hand, the U-shaped pattern is the same with and without covariates." +- *Unexplained tail behavior:* both QTE figures (3 and 5) show large spikes at both extremes (tau near 0 and 1). The paper offers no boundary-artifact discussion of these spikes; the only related statements are substantive (stronger lower-tail and upper-tail effects). No trimming of the quantile grid or tail-truncation rule is reported for the application. Densities appear in denominators of every limit process ((11), (12), (13), Lemmas 2-3 derivative maps), so vanishing tail densities would inflate the asymptotic variance - consistent with, but not explicitly linked by the paper to, the wide bands at the extremes of Figure 3. + +**Edge cases and boundary conditions:** + +- Quantile crossing: population conditional quantile functions are strictly increasing under Assumption 6; in estimation, non-monotonic QR processes are handled by the integrated-indicator CDF construction (pp. 10-11, CFG 2010) rather than direct inversion. No additional rearrangement step is mentioned. +- Support/compactness: all Hadamard-differentiability lemmas require compact support and strictly positive continuously differentiable densities; Assumption 6(ii) requires `Y_gt x X` compact in `R^{K+1}`. +- Uniform (0,1) indexing: FCLTs are stated over `tau in (0,1)` and the KS statistic takes `sup_{tau in (0,1)}`; no explicit trimming to a compact sub-interval is stated in the theorems (the compact-support and bounded-density assumptions do the work). Separately, tail trimming of the QR grid is called practically unavoidable (p. 11) but left unspecified in this draft. +- Small cells with categorical covariates: acknowledged (p. 19) as a finite-sample fragility; the weighted bootstrap is recommended for "accuracy and robustness to 'small cells'". +- Staggered adoption identification boundaries: already-treated-at-start units must be dropped and the sample must end before universal treatment (p. 21). +- Support violation (Assumption 4): testable via `Y_10x subset Y_00x`; if violated, only the overlap part of the counterfactual distribution is identified (p. 6). +- Discrete outcomes: advised against (p. 6); "the effects are only partially identified with discrete outcomes" (citing AI); distribution regression flagged for a future discrete/mixed-outcome extension (p. 25). + +**Relation to other methods:** + +- *Athey-Imbens (2006) unconditional CiC:* the model is AI's nonseparable-outcome model `Y^N = h(X, T, U)` with all four AI assumptions imposed conditionally on X; Proposition 1 trivially extends AI's Theorem 3.1 (p. 6). The contribution is not identification but a practicable estimator: AI's own covariate strategies are either fully nonparametric (curse of dimensionality) or additively separable (contradicting CiC's non-additive motivation) (p. 2). Conditional time invariance also weakens the unconditional AI requirement. For panel data, AI Section 5.3-style correlation adjustments would be required (p. 12). Without covariates the paper still adds process-level limit theory that AI does not have (p. 18). +- *DiD / common trends:* Assumption 3 is "the CIC counterpart of the common trend assumption in the DID model" (p. 5). The pp. 22-23 analysis shows DiD applied to outcome indicators is inconsistent for distributional/quantile effects except in knife-edge cases (Figures 1-2). +- *Chernozhukov-Fernandez-Val-Melly (2013) counterfactual framework:* the estimation and inference machinery is built directly on CFM 2013 - the QR-process FCLT and conditional distribution process (Corollary 5.2, eqs. (9)-(10)); the counterfactual operator (Lemma D.1, ingredient 4); the treated-side unconditional FCLT (Theorem 4.1(ii), Theorem 7 proof); tail-trimming adaptations; and exchangeable-bootstrap validity for the QR coefficient process (Corollary 5.2(ii), Theorem 10 proof). The paper's addition on top of CFM is the pair of CiC-specific Hadamard-differentiable maps - the QQ transformation (Lemma 2) and PP transformation (Lemma 3) - composed with the counterfactual operator via the functional delta method. +- *Other connections:* monotonized QR-based CDFs - Chernozhukov, Fernandez-Val, and Galichon (2010); individual-treatment-effect-distribution asymptotics - Appendix C of Chernozhukov, Fernandez-Val, and Melly (2009) (p. 10); nonseparable-model identification - Matzkin (2003); rank similarity vs rank invariance - Chernozhukov and Hansen (2005); similar expressions in decomposition contexts - Fortin and Lemieux (1998), Altonji and Blank (1999); computation - Melly (2014), Portnoy (1991); alternative first stage - distribution regression (p. 10). Conclusion (pp. 24-25) situates the paper next to the instrumental CiC of de Chaisemartin and D'Haultfoeuille (2014) under partial compliance; the "translated quantile treatment effects" of Strittmatter (2014) and Bitler, Domina, and Hoynes (2014) (corresponds to the counterfactual non-treated distribution in the CiC model); Wuthrich (2014)'s closed-form IVQR estimator (Chernozhukov and Hansen 2006) with a binary instrument ("very similar to the expressions for which we have developed estimators"); and Melly and Santangelo (2014)'s panel sample-selection correction imputing conditional ranks across years via time invariance. + +**Reference implementation(s):** + +- Stata: Blaise Melly's website (https://sites.google.com/site/blaisemelly/home/computer-programs/cic_stata) distributes a Stata implementation accompanying this paper. The paper's Conclusion says only "we provide codes that implement the estimation and inference procedures developed in this paper" (p. 24) and names no command on the reviewed pages - the command-name attribution comes from the website, not the paper. +- **IMPORTANT disambiguation:** this is DISTINCT from Kranker's SSC `cic` Stata module, which implements Athey-Imbens 2006 (unconditional CiC with analytical SEs and discrete bounds). Verify the provenance of any Stata artifact before using it as a parity reference in a future covariates PR. +- No R implementation is known to the initiative for covariate CiC. A future PR would need simulation-based validation - this motivated deferring covariates from v1. + +**Requirements checklist (for the future covariates PR, not v1):** +- [ ] Four per-cell QR coefficient processes on a fine mesh `u_1..u_S` with `delta*sqrt(n) -> 0`; Koenker-Bassett check-function objective per (g,t) cell +- [ ] Monotonized conditional CDFs via the integrated-indicator representation (pp. 10-11, CFG 2010); never invert raw QR processes directly +- [ ] Conditional CiC composition per eq. (7); covariate integration over the empirical `F_{X|11}` per the p. 12 displays (resolving the printed index/mesh-weight typos); treated side either empirical CDF or symmetric QR-based integration (application uses the latter) +- [ ] Unconditional QTE per eq. (8) via generalized inverses of the integrated CDFs +- [ ] Exchangeable bootstrap satisfying Condition BW (weighted QR per draw); weighted bootstrap for small cells, subsampling for very large n; cluster-level subsampling option (footnote 11 pattern) +- [ ] KS maximal-t uniform bands per (14)-(15) with a uniformly consistent variance function estimate from the bootstrap draws +- [ ] Pre-period time-invariance specification test (p. 21) when 2+ pre-treatment periods exist (KS/CvM over all tau and x, bootstrap critical values via Theorem 10) +- [ ] Tail-trimming policy: the paper calls it unavoidable but the draft sentence is incomplete; adapt Chernozhukov, Fernandez-Val, and Melly (2013) +- [ ] Support diagnostic: test `Y_10x subset Y_00x` (Assumption 4's observable implication); warn and restrict to the overlap if violated +- [ ] Staggered adoption: pairwise-period estimates averaged with weights representative of treated units; pooled QR with period and group dummies when covariates are interacted with trends (Section 5 strategy - application-level, no general theorem) +- [ ] Error on discrete outcomes (authors advise against; only partial identification) +- [ ] Validation strategy: simulation-based (no R reference exists); verify any Stata parity artifact is Melly's covariate-CiC code, NOT Kranker's SSC `cic` (Athey-Imbens unconditional) + +--- + +## Implementation Notes + +**Relevance to diff-diff CiC/QDiD v1 (2026-07-12):** +- (i) v1 ships covariate-free CiC, and this paper's Figures 1-2 bias analysis of DiD (and the analogous concern for unconditional CiC when group compositions differ) is the documented motivation for the covariates deferral being explicit rather than silent - unconditional CiC assumptions can fail when conditional ones hold. +- (ii) The four-step QR pipeline (estimate conditional CDFs by quantile regression for all four group-period cells, apply CiC transformations conditionally, integrate over the treated-group covariate distribution, invert) is the blueprint for a future covariates PR. +- (iii) The exchangeable bootstrap + KS-band machinery is shared with Callaway-Li-Oka 2018 and reinforces the bootstrap-first inference choice for v1. +- (iv) The monotonization/rearrangement step for QR-estimated conditional CDFs is the key extra numerical ingredient v1 does not need. + +### Data Structure Requirements +- Repeated cross sections with four (g,t) cells, each with positive probability `alpha_gt` (Assumption 5). Panel data requires modified theory (AI Section 5.3-style within-group over-time correlation terms) that the paper does not develop. +- Continuous outcomes only; the authors advise against discrete outcomes (p. 6); distribution regression is the flagged route for a discrete/mixed extension (p. 25). +- Covariates enter through linear-in-parameters QR per cell (Assumption 6(i)); flexible transformations of regressors can approximate the truth arbitrarily well when the conditional density is smooth (p. 10). +- Compact support `Y_gt x X` with uniformly bounded, uniformly continuous conditional densities (Assumption 6(ii)); densities appear in the denominators of all limit processes. +- Micro-level data required end to end - CiC needs the whole conditional distribution; collapsing to cell aggregates is not possible (p. 23). +- Multi-group/multi-period designs are handled by the application's strategy (pairwise 2x2 comparisons averaged with treated-representative weights; pooled QR with period/group dummies), not by general theorems. + +### Computational Considerations +- The dominant cost is estimating four full QR coefficient processes. The number of distinct QR solutions is `O(n log n)` (Portnoy 1991, footnote 7); the algorithms of Melly (2014) cut computation ~10x; the application runs a pooled QR process on 5,000,000+ observations. +- Bootstrapping the full QR process at that scale is "insurmountable" (pp. 23-24); the paper substitutes cluster-level subsampling (500 of 3,000+ counties per draw). The exchangeable-bootstrap framework (Condition BW) covers empirical, weighted, m-of-n, and subsampling variants under one validity theorem (Theorem 10). +- Densities appear only in the asymptotic-variance expressions, never in the estimator or the bootstrap - the paper explicitly recommends resampling over analytical SEs because the variances contain conditional densities of Y given X that are hard to estimate (p. 18). This matches the bootstrap-first inference design of the v1 CiC/QDiD estimators. +- The monotonized-CDF step is an S-term indicator sum per evaluation point (searchsorted-style over the per-cell QR coefficient mesh after projecting onto x); the conditional CiC composition is two such evaluations plus one QR-process lookup per (tau, x). +- KS maximal-t bands need a uniformly consistent variance function `Sigma_hat^{QE}(tau)`, obtainable from the same bootstrap/subsampling draws. + +### Tuning Parameters + +| Parameter | Type | Paper guidance | Notes | +|-----------|------|----------------|-------| +| QR grid size S / mesh width `delta` | int / float | `delta*sqrt(n) -> 0` (p. 11); estimating all distinct QRs is feasible (footnote 7) | mesh weight `delta` (or 1/S) must multiply the integrated-indicator sums; the p. 12/p. 20 displays omit it (printed typo) | +| Resampling scheme | {empirical, weighted, m-of-n, subsampling} | weighted bootstrap for small samples with categorical covariates; subsampling for very large samples (p. 19) | all covered by Condition BW / Theorem 10 | +| Number of draws B | int | not reported for the application | library convention required | +| Subsample size (subsampling) | int | application: 500 of 3,000+ counties per draw, cluster-level (footnote 11) | allows arbitrary within-cluster correlation | +| Tail trimming | float pair | "seems unavoidable in practice" but unspecified (incomplete sentence, p. 11); adapt CFM (2013) | a future PR must pick and document a rule | +| tau grid for bands | floats in (0,1) | theorems stated over `tau in (0,1)`; no explicit compact trimming in the statements | compact-support + bounded-density assumptions do the work; tails are where the application's bands blow up | + +### Relation to Existing diff-diff Estimators +- `ChangesInChanges` (v1): the exact special case X = constant. The paper's remark 3 on p. 18 notes that even without covariates it contributes the limiting distribution of the whole quantile and distribution processes with a simpler proof than AI - a useful citation when documenting process-level (uniform-band) inference for the v1 estimator. +- `QDiD` (v1): the pp. 22-23 DiD-on-indicators analysis is the paper's cautionary tale about additive-in-distribution shortcuts for distributional targets; relevant background for the choosing-an-estimator docs alongside the AI (2006) QDiD discussion, though the object analyzed (DiD applied to `1(Y <= y)`) is not the QDiD transform itself. +- Callaway-Li-Oka (2018) review (sibling, same initiative): shared inference stack - exchangeable bootstrap weights, functional delta method over Hadamard-differentiable ECDF/quantile compositions, KS-based uniform bands. Both ultimately lean on Van der Vaart-Wellner chapter 3.9; this paper routes through CFM (2013) Corollary 5.2 for the QR first stage. +- A future covariates implementation should reuse the `safe_inference()` joint-NaN contract from `diff_diff.utils`, the results-dataclass conventions, and the bootstrap utilities, and would be a standalone estimator (own `get_params`/`set_params` per the estimator-inheritance map in CLAUDE.md). + +--- + +## Gaps and Uncertainties + +**Suspected typos in the October 2015 preliminary draft (flagged by extraction; transcribed as printed, not silently corrected):** +- p. 11: the tail-trimming sentence is printed incomplete - "Tail trimming seems unavoidable in practice because ." - so the draft never states the reason or a rule; the surrounding text points to Chernozhukov, Fernandez-Val, and Melly (2013) for adapted formulas. +- p. 11: the inline restatement of Proposition 1's counterfactual quantile writes the left-hand argument as `(y)` where the p. 6 display has `(tau)`. +- p. 12 integration displays (both `F_hat_{Y^N|11}` and `F_hat_{Y^I|11}`): the outer sum runs over `j = 1..S` while the summand prints `u_s` (and, on the counterfactual side, `tau`), and the mesh-width factor `delta` (equivalently 1/S) from the p. 11 discretization is not printed. The intended estimator evaluates on the mesh `{u_j}` with the `delta` weight, mirroring the p. 11 formula. +- eq. (8) (p. 12): both right-hand quantile functions are printed with argument `(y)` although the process is indexed by the quantile level; both sides are (generalized) inverses of the integrated CDF estimators. +- p. 8: the intuition sentence below the Proposition 1 proof swaps group labels relative to the displayed identification formula; this review transcribes consistently with the displays. +- eq. (10) (p. 13): the limit process is printed with index `(u, x)` where the display is a process in `(y, x)`, and the scaling prints `sqrt(n*alpha_gt)` where (9) uses `sqrt(n)` - transcribed as printed. +- `V^Q_gt` covariance (p. 13): the evaluation point of the second Jacobian is hard to resolve at print resolution; transcribed as `J_gt(u~)^{-1}`, the standard QR-process covariance form. +- p. 20 bootstrap display: typographically dense, with the same missing mesh-weight issue as the p. 12 displays; the structure is the Condition-BW-weighted empirical analog of the QQ/PP composition over the S-point u-grid. +- Sign-convention asymmetry (as printed, identically in both extraction passes): eq. (10) defines `Z^F_gt` with a leading minus (`-f * Z(F(y), x)`) while the Theorem 7 proof defines `Z^F_N` with a plus (`+f * Z^Q_N(F(y), x)`). Immaterial for the limit laws (zero-mean Gaussian processes are sign-symmetric), but confusing if the formulas are transplanted into covariance code - re-derive signs if analytical variances are ever implemented. + +**Extraction cross-check (overlap pp. 13-17):** the two extraction passes double-covered pp. 13-17 (Assumption 6, eqs. (9)-(11), Lemmas 2-3, Theorem 4, Corollaries 5-6, Theorem 7 and its proof intermediates). All displays were transcribed identically by both passes; no contradictions between extraction files were found. Section labeling is consistent: the p. 6 forward reference to "Section 4.4" for time-invariance testing resolves to Section 4.4 "Inference" (pp. 19-21), which contains the specification test. + +**Version and coverage gaps:** +- **Preliminary working paper.** The reviewed October 2015 draft is flagged "Preliminary!" and contains the printed typos above. All equation/assumption/theorem numbers are pinned to this draft; later drafts, if any, may renumber or repair the typos. The paper remains unpublished as of 2026-07-12. +- **No Stata command name in the paper.** The Conclusion (p. 24) says "we provide codes" without naming a command anywhere on the reviewed pages; the command-name attribution rests on Melly's website. Verify provenance before parity use, and do not confuse it with Kranker's SSC `cic` module (Athey-Imbens 2006 unconditional CiC). +- **No R implementation and no Monte Carlo.** The paper reports no simulations (only the analytic DiD-bias illustration, pp. 22-23), and no R implementation of covariate CiC is known to the initiative - a future PR needs simulation-based validation and possibly a purpose-built oracle. +- **Number of resampling draws unreported.** No B is stated for the application's subsampling (500 of 3,000+ counties per draw, footnote 11; draw count absent). +- **Tail spikes unexplained.** Figures 3 and 5 show large QTE spikes at both extremes; the paper offers no boundary-artifact discussion, no quantile-grid trimming, and no tail-truncation rule for the application. The density-in-denominator structure of (11)-(13) is a plausible mechanical explanation but is not drawn by the paper. +- **Staggered-adoption aggregation is informal.** The pairwise-averaging weights ("representative of the treated counties") and the pooled-QR-with-dummies device are described in prose (p. 23) without formulas or dedicated asymptotic theory; the formal results cover only the 2x2 case. +- **Panel data not covered.** Assumption 5 is repeated cross sections; the paper notes AI Section 5.3-style modifications would be needed for panels (p. 12) but does not derive them. The individual-effect-distribution estimand (eq. (6)) needs panel data plus rank invariance and is left to a combination with Chernozhukov, Fernandez-Val, and Melly (2009, Appendix C). +- **Weak monotonicity / discrete outcomes.** Only strict monotonicity is handled; AI's weakly-increasing-`h` partial-identification case is deferred to future work (footnote 4, p. 6), and distribution regression for discrete/mixed outcomes is only sketched in the Conclusion (p. 25). +- **Efficiency claims unquantified.** The introduction asserts covariates can bring efficiency gains even when unconditional time invariance holds (p. 2); no formal efficiency comparison appears in the paper. From ce5972ca057f70e4b4cd098b37ab187ef8f71d49 Mon Sep 17 00:00:00 2001 From: igerber Date: Sun, 12 Jul 2026 10:36:30 -0400 Subject: [PATCH 2/2] docs: address AI review round 1 on CiC/QDiD paper reviews - Separate the ecic (staggered event-study CiC, Section 6 lineage) and Ciaccio (copula-based staggered distributional DiD) threads in the Athey-Imbens review's Section 6 deferral notes (Finding 1) - Flag Ciaccio p. 22 IPW display's sum_{i in r} + C-indicator inconsistency; give the implementable Hajek-normalized never-treated form inline and as Gaps item 17 (Finding 2) - Verify AI 2006 footnote 31 against journal p. 463: the paper itself prints half-range (y_upper - y_lower)/2 while calling it the support midpoint; flag as suspected typo, instruct true-midpoint split (Finding 3) - Verify MS 2015 Lemma 3 against p. 14: the paper prints the negative h_2 sign (and cites G^{-1} inversion in a proof that inverts F); flag as suspected copy-paste typo from Lemma 2 with re-derive instruction (Finding 4) - Update ROADMAP Ciaccio row to reviewed-deferred status so it matches the review doc's claim (Finding 5) - Fix inline-code backticks on the F_hat^{-1}(0) convention (Finding 6) Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_011hismLBFcUbUzvDRq8ruWb --- ROADMAP.md | 2 +- docs/methodology/papers/athey-imbens-2006-review.md | 11 ++++++----- docs/methodology/papers/ciaccio-2024-review.md | 3 ++- .../papers/melly-santangelo-2015-review.md | 3 +++ 4 files changed, 12 insertions(+), 7 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 67e70683f..bae652f15 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -32,7 +32,7 @@ Research-informed candidates. Each has a rationale, a tractability note, and a c ### Methodology extensions - **Nonparametric / flexible outcome regression for `EfficientDiD` DR covariate path** (Chen, Sant'Anna & Xie, arXiv:2506.17729, 2025, Section 4). The shipped staggered `EfficientDiD` uses a linear OLS outcome regression in its doubly-robust covariate path; that preserves DR consistency but does not generically attain the semiparametric efficiency bound unless the conditional mean is linear in the covariates. Replacing the OLS outcome regression with sieve / kernel / ML nuisance estimation (as the paper's Section 4 allows) would close the efficiency gap on the covariate path. Tractability: medium; the hook points are in `diff_diff/efficient_did_covariates.py`. **Commit when**: a paper-review synthesis is written, with an implementation plan for the nonparametric OR that preserves the existing DR consistency guarantees and survey-weighted variance surface. -- **Distributional DiD for staggered timing** (Ciaccio, arXiv:2408.01208, 2024). New estimator extending Callaway-Li QTT to staggered adoption. `CallawaySantAnna` currently gives mean ATT only; this unlocks quantile effects. Tractability: medium. **Commit when**: a health-econ or public-health user reports need for quantile effects in a repeated-cross-section design. +- **Distributional DiD for staggered timing** (Ciaccio, arXiv:2408.01208, 2024). New estimator extending Callaway-Li QTT to staggered adoption. `CallawaySantAnna` currently gives mean ATT only; this unlocks quantile effects. Tractability: medium. **Reviewed 2026-07** (`docs/methodology/papers/ciaccio-2024-review.md`); implementation deferred pending demand. **Commit when**: a health-econ or public-health user reports need for quantile effects in a repeated-cross-section design. - **Local Projections DiD** (Dube, Girardi, Jordà & Taylor, JAE 2025). New estimator with flexible impulse-response and robustness to dynamic misspecification; natural for anticipation-prone settings. Tractability: well-scoped. **Commit when**: a methodology review confirms the dynamic variant's variance derivation fits our SE helpers. - **Few-treated-units inference option** (Alvarez, Ferman & Wüthrich, arXiv:2504.19841, 2025). `inference=` option covering t(G-1) corrections, randomization inference, and Ferman-Pinto-style permutation tests. Current SE paths assume large-G asymptotics. Tractability: medium. **Commit when**: a user reports sparse-treatment pain. - **Riesz-representation sensitivity** (Bach et al., arXiv:2510.09064, 2025). Confounder-based sensitivity bound complementing HonestDiD's trend-based bound. Tractability: medium. **Commit when**: HonestDiD users ask for confounder bounds. diff --git a/docs/methodology/papers/athey-imbens-2006-review.md b/docs/methodology/papers/athey-imbens-2006-review.md index 688c7b0d9..dae82bc7a 100644 --- a/docs/methodology/papers/athey-imbens-2006-review.md +++ b/docs/methodology/papers/athey-imbens-2006-review.md @@ -167,7 +167,7 @@ Equation (20) is k^CIC with the roles of groups 0 and 1 reversed; implement by e ``` p captures estimation error from F_hat_Y,00, q from F_hat_Y,01^{-1}, r sampling variation of the transformed Y_10 draws, s the Y_11 mean. Asymptotically linear representation (p. 463): `tau_hat = tau + (1/N_00) sum p(Y_00,i) + (1/N_01) sum q(Y_01,i) + (1/N_10) sum r(Y_10,i) + (1/N_11) sum s(Y_11,i) + o_p(N^{-1/2})`. -- Variance estimation (Theorem 5.2, pp. 463-464): plug in empirical CDFs/inverses and ANY uniformly consistent nonparametric density estimator (uniform consistency must include the boundary, footnote 31); `p_hat(y) = (1/N_10) sum_j P_hat(y, Y_10,j)`, `q_hat(y) = (1/N_10) sum_j Q_hat(y, Y_10,j)`, then `V_hat^p = (1/N_00) sum_i p_hat(Y_00,i)^2` (and analogously q, r, s), `alpha_hat_gt = N_gt/N`; all converge in probability to their targets. Footnote 31's concrete density estimator: one-sided difference quotient of the EDF with bandwidth `N^{-1/3}`, forward difference on the lower half of the support and backward on the upper half (split at the support midpoint `Y_tilde_gt = (y_upper_gt - y_lower_gt)/2`), guaranteeing boundary-uniform consistency: +- Variance estimation (Theorem 5.2, pp. 463-464): plug in empirical CDFs/inverses and ANY uniformly consistent nonparametric density estimator (uniform consistency must include the boundary, footnote 31); `p_hat(y) = (1/N_10) sum_j P_hat(y, Y_10,j)`, `q_hat(y) = (1/N_10) sum_j Q_hat(y, Y_10,j)`, then `V_hat^p = (1/N_00) sum_i p_hat(Y_00,i)^2` (and analogously q, r, s), `alpha_hat_gt = N_gt/N`; all converge in probability to their targets. Footnote 31's (p. 463) concrete density estimator: one-sided difference quotient of the EDF with bandwidth `N^{-1/3}`, forward difference below the split point `Y_tilde_gt` and backward above it, guaranteeing boundary-uniform consistency. As printed the footnote calls `Y_tilde_gt` "the midpoint of the support" but defines `Y_tilde_gt = (y_upper_gt - y_lower_gt)/2`, which is the half-range, not the midpoint `(y_upper_gt + y_lower_gt)/2` - see the suspected-typos block in Gaps and Uncertainties; an implementation should split at the true midpoint: ``` f_hat_Y,gt(y) = ( F_hat_Y,gt(y + N^{-1/3}) - F_hat_Y,gt(y) ) / N^{-1/3} if y <= Y_tilde_gt @@ -208,7 +208,7 @@ C^pq = E[p(Y_00)*q(Y_01)], C^rs = E[r(Y_10)*s(Y_11)] = Cov( k(Y_10), Y_11 ) *Algorithm (no numbered algorithm in the paper; assembled from eqs. 34-36, Section 5.1, pp. 461-462):* 1. Validate the 2x2 design; split the sample into the four (g,t) cells. 2. Compute the empirical CDFs `F_hat_Y,gt` (eq. 34) for cells (0,0), (0,1), (1,0), and the sample mean of the (1,1) cell. -3. Transform every treated pre-period observation: `y_i -> F_hat_Y,01^{-1}( F_hat_Y,00(Y_10,i) )`, using exactly the eq. (35)/(A.1) inverse (ceiling order statistic `Y_(ceil(Nq))`, `F_hat^{-1}(0) = ` sample minimum). +3. Transform every treated pre-period observation: `y_i -> F_hat_Y,01^{-1}( F_hat_Y,00(Y_10,i) )`, using exactly the eq. (35)/(A.1) inverse (ceiling order statistic `Y_(ceil(Nq))`, with `F_hat^{-1}(0)` = sample minimum). 4. `tau_hat^CIC = mean(Y_11) - mean(transformed Y_10)` (eq. 36). 5. Quantile effects at requested q ∈ (q_lower, q_upper): `tau_hat_q^CIC = F_hat_Y,11^{-1}(q) - F_hat_Y,01^{-1}(F_hat_Y,00(F_hat_Y,10^{-1}(q)))`. 6. Inference: v1 bootstrap (units in panel mode, within-cell in repeated cross-section mode); paper's analytical variance (Theorems 5.1-5.3, 5.5-5.6) deferred. @@ -319,12 +319,12 @@ Setup: groups `G = {1,...,N_G}`, periods `T = {1,...,N_T}`; `I` = set of treated tau_hat^CIC_I = (A' V_hat_J^(-) A)^{-1} (A' V_hat_J^(-) kappa_hat_J) [Theorem 6.3: sqrt(N)-normal, variance (A' V_J^(-) A)^{-1}] ``` -Aggregation `tau_Lambda = Lambda' tau^CIC_I` (columns of Lambda sum to 1): sample-size weights `Lambda_{g,t} = N_{g,t}/sum N_{g,t}` are the natural default; variance-minimizing weights are appropriate only under constant effects (p. 475). Tests: **Theorem 6.4** overidentification/specification statistic `N (kappa_hat_J - A tau_hat_I)' V_hat_J^(-) (kappa_hat_J - A tau_hat_I) -->d Chi2(rank(V_J) - N_I)` - has power against violations of `U ⊥ T | G`, including additive random group-time effects (p. 476); equality-of-effects statistic `-->d Chi2(N_I - 1)` under `tau_{g,t} = tau` for all treated pairs (GLS-weighted pooled estimate; p. 476). diff-diff defers this section (Ciaccio / `ecic` staggered event-study CiC is the reference for the extension). +Aggregation `tau_Lambda = Lambda' tau^CIC_I` (columns of Lambda sum to 1): sample-size weights `Lambda_{g,t} = N_{g,t}/sum N_{g,t}` are the natural default; variance-minimizing weights are appropriate only under constant effects (p. 475). Tests: **Theorem 6.4** overidentification/specification statistic `N (kappa_hat_J - A tau_hat_I)' V_hat_J^(-) (kappa_hat_J - A tau_hat_I) -->d Chi2(rank(V_J) - N_I)` - has power against violations of `U ⊥ T | G`, including additive random group-time effects (p. 476); equality-of-effects statistic `-->d Chi2(N_I - 1)` under `tau_{g,t} = tau` for all treated pairs (GLS-weighted pooled estimate; p. 476). diff-diff defers this section. Two distinct follow-on threads exist and must not be conflated: the R `ecic` package (Kluser) implements a staggered event-study extension of the CiC transformation itself (the natural descendant of this Section 6 machinery), while Ciaccio's staggered distributional DiD (reviewed separately in `ciaccio-2024-review.md`) is a copula-based Callaway-Li-style QTT method with different identifying assumptions - not an implementation of Section 6. **Reference implementation(s):** - R: `qte::CiC()` and `qte::QDiD()` (Callaway's qte package - the project's chosen parity target; bootstrap inference, panel and repeated cross-section support) - Stata: `cic` (Kranker) - implements the analytical SEs (Theorems 5.1-5.3) and the discrete-outcome bounds -- R: `ecic` (Kluser) - staggered event-study CiC (relevant to the deferred Section 6 extension) +- R: `ecic` (Kluser) - staggered event-study CiC, i.e. the CiC transformation applied per cohort-period (the implementation lineage closest to the deferred Section 6 extension). Not to be conflated with Ciaccio's copula-based staggered distributional DiD, which is a distinct method reviewed separately - The paper itself points to the Econometrica supplement for the empirical application (Meyer-Viscusi-Durbin (1995) injury-duration data; headline finding: CiC vs standard DiD results can differ in both magnitude and significance, p. 477) **Requirements checklist:** @@ -345,7 +345,7 @@ Aggregation `tau_Lambda = Lambda' tau^CIC_I` (columns of Lambda sum to 1): sampl **Planned diff-diff v1 scope (2026-07-12):** - Ship: `ChangesInChanges` + `QDiD`, 2x2 design, continuous outcomes, bootstrap inference, panel + repeated cross-section modes. -- Deferred: covariates (Theorems 4.3-4.4 and the Section 5.1 residual route; Melly-Santangelo is the modern reference), discrete-outcome bounds (Section 4 identification / Section 5.2 inference, incl. Imbens-Manski intervals), analytical SEs (Theorems 5.1-5.3, 5.5-5.7; Appendix B covariances), multiple groups/periods (Section 6; Ciaccio / `ecic`). +- Deferred: covariates (Theorems 4.3-4.4 and the Section 5.1 residual route; Melly-Santangelo is the modern reference), discrete-outcome bounds (Section 4 identification / Section 5.2 inference, incl. Imbens-Manski intervals), analytical SEs (Theorems 5.1-5.3, 5.5-5.7; Appendix B covariances), multiple groups/periods (Section 6; `ecic` for the staggered event-study CiC lineage - Ciaccio's copula-based staggered distributional DiD is a distinct method, reviewed separately). - All deferred material is reviewed in this document so the deferral is documented, not silent. ### Data Structure Requirements @@ -388,6 +388,7 @@ Aggregation `tau_Lambda = Lambda' tau^CIC_I` (columns of Lambda sum to 1): sampl - Theorem 5.7, second display (p. 469): the `V_upper^r` term's denominator prints as `alpha_10` where the panel notation elsewhere in the theorem uses `alpha_1`. Likely a typo; an implementation should use `alpha_1`. - Lemma A.4 (pp. 479-480): the printed oscillation modulus reads `F_hat_Y(y+x) - F_hat_Y(x)` where the standard Stute (1982) modulus has `F_hat_Y(y+x) - F_hat_Y(y)`, matching the population term `F_Y(y+x) - F_Y(y)`. Likely a typesetting slip. - Appendix B, case 2 (p. 493): printed with explicit argument notation `(Y_{g0,t0})` and a division by `alpha_{g0,t0}`, unlike every other case; the two readings are numerically equivalent (`N*E[mu_hat mu_hat'] = E[infl-fn product]/alpha_gt` for same-cell averages). Treat the per-shared-cell `N*E[mu_hat mu_hat']` pattern as the intended formula. +- Footnote 31 (p. 463): the split point of the one-sided density estimator is described as "the midpoint of the support" but printed as `Y_tilde_gt = (y_upper_gt - y_lower_gt)/2` - the half-range, not the midpoint `(y_upper_gt + y_lower_gt)/2` (the two coincide only when `y_lower_gt = 0`). An implementation of the deferred analytical-SE path should split at the true midpoint; the printed half-range can fall outside the support (e.g. negative-valued outcomes), which would apply the wrong one-sided difference near a boundary. - Theorem 5.4 proof (p. 492): the final estimation-error term's summand sign is printed inconsistently with the preceding line's decomposition (`k_lower_hat - k_lower` vs `k_lower - k_lower_hat`); immaterial since the term converges to zero (Lemma A.13). - Equality-of-effects test (p. 476): typeset ambiguity between `A` and `A_I` in the weight matrix `Lambda = (iota' A_I' V_J^(-) A iota)^{-1} iota' A' V_J^(-) A` and in the test statistic's sandwich; the structure (GLS weighting with the generalized inverse) is clear, but consult p. 476 before implementing. diff --git a/docs/methodology/papers/ciaccio-2024-review.md b/docs/methodology/papers/ciaccio-2024-review.md index 3720cbf59..0419efb03 100644 --- a/docs/methodology/papers/ciaccio-2024-review.md +++ b/docs/methodology/papers/ciaccio-2024-review.md @@ -256,7 +256,7 @@ F^{p,-1}_{Delta_{[r-rho-1,t]}Y(0)|d_r=1}(delta) = / [ (1/n_{r,t}) sum_{i in r} (C/p_r) ( p-hat_r(x_i) / (1 - p-hat_r(x_i)) ) ] ``` -(As printed the LHS carries the `p,-1` superscript although the display is the reweighted CDF of Eq. (3), i.e. a Hajek-normalized estimator; inversion then yields the quantile.) `n_{r,t}` = number of units used to compute `QTT-hat_{r,t,rho}(tau)`; `p-hat(X)` is an estimator of the generalized propensity score; the denominator normalizes weights to sum to 1 in finite samples, ensuring `F^p(.)` is a proper distribution function. The propensity score can be estimated parametrically or non-parametrically (both Firpo 2007 and Callaway-Li 2019 show this). +(Two as-printed notation issues here - see Gaps item 17. First, the LHS carries the `p,-1` superscript although the display is the reweighted CDF of Eq. (3), i.e. a Hajek-normalized estimator; inversion then yields the quantile. Second, both sums are written `sum_{i in r}` while numerator and denominator carry the never-treated dummy `C` - over literal cohort-r units every term would be zero. Read `i in r` as the (r,t) estimation subsample - cohort r plus never-treated - with `C_i` selecting the never-treated contributions. The implementable form is a Hajek-normalized CDF over never-treated units with weight `w_i = C_i * p-hat_r(x_i) / (p-hat_r * (1 - p-hat_r(x_i)))`: compute `F-hat^p(delta) = sum_i w_i * 1{DeltaY_i <= delta} / sum_i w_i`, then invert.) `n_{r,t}` = number of units used to compute `QTT-hat_{r,t,rho}(tau)`; `p-hat(X)` is an estimator of the generalized propensity score; the denominator normalizes weights to sum to 1 in finite samples, ensuring `F^p(.)` is a proper distribution function. The propensity score can be estimated parametrically or non-parametrically (both Firpo 2007 and Callaway-Li 2019 show this). *Case C - fully conditional (Proposition 2) (p. 22):* @@ -617,3 +617,4 @@ No smoothing parameters exist anywhere in the method. 14. **Wild/subcluster bootstrap is a pointer, not a procedure** (p. 22): the clustered-inference route is described as adapting MacKinnon-Webb (2018) "with minor modifications" with no algorithmic detail; a future implementation would have to derive the QTT adaptation itself. 15. **Extraction coverage.** This review synthesizes three extraction passes (pp. 1-20, pp. 16-35, pp. 31-44; printed page = PDF page) with overlaps on pp. 16-20 and 31-35 reconciled as documented; References (pp. 33-36) were skipped by design. Tables 2-4 numbers above are the selected values transcribed by the extraction, not full tables. 16. **No verified reference implementation** (see Reference implementation(s) above); in particular the R `ecic` package is NOT an implementation of this paper despite the overlapping "staggered distributional DiD" label. +17. **IPW display's sum index vs `C` indicator (p. 22).** The Proposition 1 estimator display prints `sum_{i in r}` in both numerator and denominator while each summand carries the never-treated dummy `C` - a literal cohort-r sum would be identically zero. The intended reading (consistent with the (A.11)-(A.14) proof chain and the `n_{r,t}` definition) is a sum over the (r,t) estimation subsample - cohort r plus never-treated - with `C_i` selecting never-treated units. Implement as a Hajek-normalized CDF over never-treated observations with weight `C_i * p_r(X_i) / (p_r * (1 - p_r(X_i)))`, then invert; do not copy the printed display literally. diff --git a/docs/methodology/papers/melly-santangelo-2015-review.md b/docs/methodology/papers/melly-santangelo-2015-review.md index 7686b43f1..61b025beb 100644 --- a/docs/methodology/papers/melly-santangelo-2015-review.md +++ b/docs/methodology/papers/melly-santangelo-2015-review.md @@ -288,6 +288,8 @@ phi'_{F,G}(h_1, h_2) = h_1 o G^{-1} - ( f o G^{-1} / g o G^{-1} ) * h_2 o G^{-1} phi'_{F,G}(h_1, h_2) = -( h_1 / f ) o F^{-1} o G - ( h_2 / (f o F^{-1}) ) o G ``` +(as printed - the sign of the `h_2` term is suspect; see the suspected-typos block in Gaps and Uncertainties) + *Ingredient 4 - counterfactual operator (p. 15).* Lemma D.1 of Chernozhukov, Fernandez-Val, and Melly (2013): `phi^C(F, G) = INT F(y, x) dG(x)` is Hadamard-differentiable with derivative map ``` @@ -504,6 +506,7 @@ All quantiles and covariate values must be considered to detect deviations; Kolm - eq. (8) (p. 12): both right-hand quantile functions are printed with argument `(y)` although the process is indexed by the quantile level; both sides are (generalized) inverses of the integrated CDF estimators. - p. 8: the intuition sentence below the Proposition 1 proof swaps group labels relative to the displayed identification formula; this review transcribes consistently with the displays. - eq. (10) (p. 13): the limit process is printed with index `(u, x)` where the display is a process in `(y, x)`, and the scaling prints `sqrt(n*alpha_gt)` where (9) uses `sqrt(n)` - transcribed as printed. +- Lemma 3 (p. 14): the derivative map for `phi^{PP}(F, G) = F^{-1} o G` prints a NEGATIVE sign on the `h_2` term, `- (h_2/(f o F^{-1})) o G`. Standard inverse-map calculus (Van der Vaart-Wellner Lemma 3.9.23(ii) plus the chain rule) gives a POSITIVE `h_2` term: the G-perturbation enters through `(F^{-1})' = 1/(f o F^{-1})` with no sign flip; only the `h_1` term (through inverting F) is negative. The printed proof also cites Hadamard differentiability of "the inverse map G^{-1}" although the PP map inverts F - both oddities are consistent with a copy-paste slip from Lemma 2 (the QQ map, whose proof genuinely inverts G). Re-derive this derivative before any analytical covariance implementation that composes these maps; the bootstrap-based inference path does not depend on it. - `V^Q_gt` covariance (p. 13): the evaluation point of the second Jacobian is hard to resolve at print resolution; transcribed as `J_gt(u~)^{-1}`, the standard QR-process covariance form. - p. 20 bootstrap display: typographically dense, with the same missing mesh-weight issue as the p. 12 displays; the structure is the Condition-BW-weighted empirical analog of the QQ/PP composition over the S-point u-grid. - Sign-convention asymmetry (as printed, identically in both extraction passes): eq. (10) defines `Z^F_gt` with a leading minus (`-f * Z(F(y), x)`) while the Theorem 7 proof defines `Z^F_N` with a plus (`+f * Z^Q_N(F(y), x)`). Immaterial for the limit laws (zero-mean Gaussian processes are sign-symmetric), but confusing if the formulas are transplanted into covariance code - re-derive signs if analytical variances are ever implemented.