diff --git a/CHANGELOG.md b/CHANGELOG.md index 75e05f9cb..111298001 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,8 +8,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added -- **`RegressionDiscontinuity` - sharp AND fuzzy regression discontinuity estimation - with robust bias-corrected inference (alias `RDD`).** Local-polynomial RD per +- **`RegressionDiscontinuity` - sharp, fuzzy, AND covariate-adjusted regression + discontinuity estimation with robust bias-corrected inference (alias `RDD`).** Local-polynomial RD per Calonico, Cattaneo & Titiunik (2014), parity-targeting R `rdrobust` 4.0.0 end-to-end: all 10 data-driven bandwidth selectors (`mserd` default, `msetwo`/`msesum`/comb and the CER-optimal variants), @@ -29,6 +29,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 robust CI contains zero (documented deviation - R is silent; CCT 2014 Theorem 3 "guard and warn", Feir-Lemieux-Marmer weak-IV inference a documented seam); R's exact no-variation-no-jump identification error is raised on both entry points. + **Covariate adjustment** via `fit(..., covariates=[...])` (R's `covs=`; CCFT 2019): + additive common-coefficient adjustment with a pooled-across-sides gamma that + leaves the estimand UNCHANGED (precision only - explicitly unlike the DiD + estimators' conditional-parallel-trends `covariates` role; the operative, + testable requirement is covariate balance at the cutoff, and the placebo recipe + - fit each covariate as the outcome - is documented); bandwidths are + covariate-aware (Z stacked into every pilot with a per-pilot gamma); collinear + columns are dropped with a warning NAMING them under `covs_drop=True` (R's + default and its exact dqrdc2 rank/pivot semantics incl. the name-length column + sort, ported directly; `covs_drop=False` = deterministic strict error); fitted + gammas are exposed name-keyed (`covariate_coefficients`, fuzzy + `first_stage_covariate_coefficients`) and `covariates`/`covariates_dropped`/ + `covs_drop` echo on the results. Degenerate adjustments are GUARDED, not + reproduced (documented deviation): R's `ginv(tol=1e-20)` inverts a float-noise + singular value on constant covariates / full dummy sets, silently returning + platform-dependent estimates; diff-diff excludes degenerate columns (a constant + covariate reproduces the fit without it to numerical precision), applies a + scale-invariant stabilized cut for rank-deficient sets (a full dummy set + reproduces the drop-one-category fit to numerical precision), and warns + naming the columns. **Canonical binding:** `att`/`se`/`t_stat`/`p_value`/`conf_int` are ONE coherent row - the robust bias-corrected row (`att = tau_bc`, CI centered on it, `t_stat == att/se`), preserving the library-wide field identities; the new @@ -37,20 +57,26 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 row, and `summary()` prints the familiar three-row table. Estimation-path port (`rdrobust_fit`: Q_q bias-correction score matrix, conventional/robust NN sandwiches, fuzzy ratio/first-stage variances) validated against a new estimates - golden (`benchmarks/data/rdrobust_estimates_golden.json`, 23 configurations incl. - the Senate anchors and 7 fuzzy configs - default/sharpbw/manual-h/epa/msetwo/ - one-sided-compliance/ties) at rtol=1e-9 in `tests/test_rdd_parity.py` (+ port-level - linearized-bias pins in `tests/test_rdrobust_port.py`); R-free methodology - anchors (CCT 2014 Remark 7 bias-corrected == local-quadratic equivalence at rel - 1e-10 across all kernels, perfect-compliance == sharp reproduction, bandwidth - auto-switch locks, invariances, joint-NaN degenerate contracts) in + golden (`benchmarks/data/rdrobust_estimates_golden.json`, 32 configurations incl. + the Senate anchors, 7 fuzzy configs - default/sharpbw/manual-h/epa/msetwo/ + one-sided-compliance/ties - and 9 covariate configs - default/manual-h/msetwo/ + cercomb2/epa/collinear-drop/ties/fuzzy/fuzzy-sharpbw with `coef_covs` gamma + pins) at rtol=1e-9 in `tests/test_rdd_parity.py` (+ port-level linearized-bias, + gamma-matrix, and dqrdc2 rank/pivot pins in `tests/test_rdrobust_port.py`); + R-free methodology anchors (CCT 2014 Remark 7 bias-corrected == local-quadratic + equivalence at rel 1e-10 across all kernels, perfect-compliance == sharp + reproduction, bandwidth auto-switch locks, invariances, joint-NaN degenerate + contracts, and the CCFT 2019 partial-out identity - exact at common manual + bandwidths - plus covariate span-/order-invariance and CI-shrinkage anchors) in `tests/test_rdd_methodology.py`; API/validation suite in `tests/test_rdd.py`. Deviations from R (each labeled in the REGISTRY section): warn-instead-of-silent NaN drops, warn-and-ignore `b`-without-`h`, the weak-first-stage warning, - warn-and-ignore `sharpbw` on sharp fits, fail-closed targeted errors on - degenerate designs, and the canonical-binding note above. Covariates (CCFT 2019 - - review on file), cluster-robust variance, weights, kink estimands, weak-IV-robust - fuzzy inference, and rdplot/density diagnostics are documented follow-ups. + warn-and-ignore `sharpbw` on sharp fits (and `covs_drop=False` without + covariates), fail-closed targeted errors on degenerate designs, the guarded + degenerate covariate adjustment, and the canonical-binding note above. + Cluster-robust variance, weights, kink estimands, weak-IV-robust fuzzy + inference, a packaged covariate-balance helper, and rdplot/density diagnostics + are documented follow-ups. - **Internal: mypy enforced at zero errors.** Triaged the 184 pre-existing `mypy diff_diff` errors to an enforceable zero and added a blocking Mypy job to the Lint CI workflow (pinned `mypy==2.1.0` + pinned numpy/pandas/scipy for stub @@ -134,9 +160,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - `diff_diff/guides/llms-autonomous.txt` no longer lists regression discontinuity as - out of scope: sharp AND fuzzy RD route to `RegressionDiscontinuity`; only kink - designs and the covariate-adjusted / cluster-robust RD variants are referred to - external tooling. + out of scope: sharp, fuzzy, AND covariate-adjusted RD route to + `RegressionDiscontinuity`; only kink designs and the cluster-robust RD variant + are referred to external tooling. - **Internal: repo-wide lint normalization + pinned tooling.** black/ruff/mypy are now pinned exactly in the `dev` extra (`black==26.3.1`, `ruff==0.15.13`, `mypy==2.1.0`; the tools require Python >= 3.10 — the library floor stays 3.9); full `black` + diff --git a/README.md b/README.md index 81f54c863..d4ca17737 100644 --- a/README.md +++ b/README.md @@ -112,7 +112,7 @@ Full guide: `diff_diff.get_llm_guide("practitioner")`. - [TripleDifference](https://diff-diff.readthedocs.io/en/stable/api/triple_diff.html) - triple difference (DDD) estimator for designs requiring two criteria for treatment eligibility - [ContinuousDiD](https://diff-diff.readthedocs.io/en/stable/api/continuous_did.html) - Callaway, Goodman-Bacon & Sant'Anna (2024) continuous treatment DiD with dose-response curves - [HeterogeneousAdoptionDiD](https://diff-diff.readthedocs.io/en/stable/api/had.html) - de Chaisemartin, Ciccia, D'Haultfœuille & Knau (2026) for designs where **no unit remains untreated**; local-linear estimator at the dose support boundary returning Weighted Average Slope (WAS) on Design 1' (`d̲ = 0` / QUG) or `WAS_{d̲}` on Design 1 (`d̲ > 0`, continuous-near-d̲ or mass-point), with a multi-period event-study extension (last-treatment cohort, pointwise CIs). **Panel-only** in this release - repeated cross-sections rejected by the validator. Alias `HAD`. -- [RegressionDiscontinuity](https://diff-diff.readthedocs.io/en/stable/api/regression_discontinuity.html) - Calonico, Cattaneo & Titiunik (2014) sharp AND fuzzy regression discontinuity with robust bias-corrected inference and rdrobust-parity bandwidth selection (all 10 selectors, mass-point handling; fuzzy via `treatment_col=` with a first-stage block and weak-identification warning). Canonical `att` is the bias-corrected estimate with a coherent robust CI (rdrobust's printed headline is `att_conventional`). Alias `RDD`. +- [RegressionDiscontinuity](https://diff-diff.readthedocs.io/en/stable/api/regression_discontinuity.html) - Calonico, Cattaneo & Titiunik (2014) sharp, fuzzy, AND covariate-adjusted regression discontinuity with robust bias-corrected inference and rdrobust-parity bandwidth selection (all 10 selectors, mass-point handling; fuzzy via `treatment_col=` with a first-stage block and weak-identification warning; covariates via `covariates=` - CCFT 2019, same estimand, covariate-aware bandwidths). Canonical `att` is the bias-corrected estimate with a coherent robust CI (rdrobust's printed headline is `att_conventional`). Alias `RDD`. - [StackedDiD](https://diff-diff.readthedocs.io/en/stable/api/stacked_did.html) - Wing, Freedman & Hollingsworth (2024) stacked DiD with Q-weights and sub-experiments; optional covariate balancing (Ustyuzhanin 2026) - [EfficientDiD](https://diff-diff.readthedocs.io/en/stable/api/efficient_did.html) - Chen, Sant'Anna & Xie (2025) efficient DiD with optimal weighting for tighter SEs - [TROP](https://diff-diff.readthedocs.io/en/stable/api/trop.html) - Triply Robust Panel estimator (Athey et al. 2025) with nuclear norm factor adjustment diff --git a/benchmarks/R/generate_rdrobust_estimates_golden.R b/benchmarks/R/generate_rdrobust_estimates_golden.R index f9f4b0d0c..bcf8a2fc7 100644 --- a/benchmarks/R/generate_rdrobust_estimates_golden.R +++ b/benchmarks/R/generate_rdrobust_estimates_golden.R @@ -1,8 +1,11 @@ -# Golden-value generator for the diff-diff RD ESTIMATION port - sharp AND -# fuzzy (diff_diff/_rdrobust_port.py::rdrobust_fit and the public -# RegressionDiscontinuity estimator; 23 configs across four synthetic DGPs -# + the Senate data, incl. 7 fuzzy configs with full first-stage -# tau_T/se_T/z_T/pv_T/ci_T blocks and per-side take-up coefficients). +# Golden-value generator for the diff-diff RD ESTIMATION port - sharp, +# fuzzy, AND covariate-adjusted (diff_diff/_rdrobust_port.py::rdrobust_fit +# and the public RegressionDiscontinuity estimator; 32 configs across five +# synthetic DGPs + the Senate data, incl. 7 fuzzy configs with full +# first-stage tau_T/se_T/z_T/pv_T/ci_T blocks and 9 covariate configs with +# coef_covs (gamma) pins; covariate names deliberately differ in length +# and are passed UNSORTED so every covariate config also pins rdrobust's +# order(nchar(colnames)) column sort, rdrobust.R:131). # # Deliberately a SEPARATE file/JSON from generate_rdrobust_golden.R so the # bandwidth fixtures reviewed in the machinery PR are never regenerated. @@ -27,19 +30,24 @@ TARBALL_SHA256 <- "78f0d6b4bdec4091cc8f42f6f1598704747f95926446d3aaee381ea1d613a run_estimate <- function(y, x, c = 0, masspoints = "adjust", kernel = "tri", p = 1, q = 2, h = NULL, b = NULL, rho = NULL, level = 95, bwselect = "mserd", - fuzzy = NULL, sharpbw = FALSE) { + fuzzy = NULL, sharpbw = FALSE, + covs = NULL, covs_drop = TRUE) { args <- list(y = y, x = x, c = c, masspoints = masspoints, kernel = kernel, p = p, q = q, level = level, bwselect = bwselect, - sharpbw = sharpbw) + sharpbw = sharpbw, covs_drop = covs_drop) if (!is.null(h)) args$h <- h if (!is.null(b)) args$b <- b if (!is.null(rho)) args$rho <- rho if (!is.null(fuzzy)) args$fuzzy <- fuzzy + if (!is.null(covs)) args$covs <- covs r <- suppressWarnings(do.call(rdrobust, args)) out <- list( c = c, masspoints = masspoints, kernel = kernel, p = p, q = q, bwselect = bwselect, fuzzy_in = !is.null(fuzzy), sharpbw = sharpbw, + covs_in = !is.null(covs), + covs_names = if (is.null(covs)) NA else colnames(covs), + covs_drop = covs_drop, h_in = if (is.null(h)) NA else h, b_in = if (is.null(b)) NA else b, rho_in = if (is.null(rho)) NA else rho, @@ -66,6 +74,12 @@ run_estimate <- function(y, x, c = 0, masspoints = "adjust", kernel = "tri", out$beta_t_p_l <- unname(as.vector(r$beta_T_p_l)) out$beta_t_p_r <- unname(as.vector(r$beta_T_p_r)) } + if (!is.null(covs)) { + # Common projection coefficients gamma (dZ_kept x 1 sharp, x 2 fuzzy) + # over the covariates KEPT after covs_drop, in R's nchar-sorted column + # order; the row count pins WHICH columns survived the drop. + out$coef_covs <- unname(as.matrix(r$coef_covs)) + } out } @@ -75,14 +89,15 @@ golden$metadata <- list( rdrobust_version = as.character(packageVersion("rdrobust")), rdrobust_tarball_sha256 = TARBALL_SHA256, seeds = list(dgp_lee_smooth = 42L, dgp_ties_moderate = 123L, - dgp_asymmetric_scaled = 777L, dgp_fuzzy = 314L), + dgp_asymmetric_scaled = 777L, dgp_fuzzy = 314L, + dgp_covs = 2718L), generator = "benchmarks/R/generate_rdrobust_estimates_golden.R", algorithm = paste( - "rdrobust() sharp AND fuzzy estimation blocks (three-row coef/se/z/pv/ci,", - "counts, per-side beta_p; fuzzy configs add the first-stage", - "tau_T/se_T/z_T/pv_T/ci_T rows and per-side beta_T_p) for the vce='nn'", - "no-covariate path, complementing the bandwidth fixtures in", - "rdrobust_golden.json." + "rdrobust() sharp, fuzzy, AND covariate-adjusted estimation blocks", + "(three-row coef/se/z/pv/ci, counts, per-side beta_p; fuzzy configs add", + "the first-stage tau_T/se_T/z_T/pv_T/ci_T rows and per-side beta_T_p;", + "covariate configs add the coef_covs gamma matrix) for the vce='nn'", + "path, complementing the bandwidth fixtures in rdrobust_golden.json." ), r_version = R.version.string ) @@ -162,6 +177,42 @@ golden$dgp_fuzzy <- list( ) ) +# Covariate DGP: two informative covariates with NAME LENGTHS that differ +# and are passed UNSORTED (c("zlong", "zb")) so R's order(nchar) column +# sort (rdrobust.R:131) is exercised by every config; zdup is an EXACT +# linear combination for the covs_drop config. covs_ties reuses the +# 2dp-rounded running variable (masspoints machinery x covariates). +set.seed(2718) +n5 <- 1200 +x5 <- 2 * rbeta(n5, 2, 4) - 1 +zlong <- 0.5 * x5 + rnorm(n5, sd = 0.8) +zb <- rbinom(n5, 1, 0.4) +y5 <- 0.4 * x5 + 0.9 * (x5 >= 0) + 0.7 * zlong + 0.3 * zb + rnorm(n5, sd = 0.3) +t5 <- rbinom(n5, 1, ifelse(x5 >= 0, 0.75, 0.2)) +zdup <- 1.5 * zlong - 0.5 * zb +x5_ties <- round(x5, 2) +covs2 <- cbind(zlong = zlong, zb = zb) +covs3 <- cbind(zlong = zlong, zb = zb, zdup = zdup) + +golden$dgp_covs <- list( + x = x5, y = y5, t = t5, zlong = zlong, zb = zb, zdup = zdup, + x_ties = x5_ties, + configs = list( + covs_default = run_estimate(y5, x5, covs = covs2), + covs_manual_h = run_estimate(y5, x5, covs = covs2, h = 0.2), + covs_msetwo = run_estimate(y5, x5, covs = covs2, + bwselect = "msetwo"), + covs_cercomb2 = run_estimate(y5, x5, covs = covs2, + bwselect = "cercomb2"), + covs_epa = run_estimate(y5, x5, covs = covs2, kernel = "epa"), + covs_drop_collinear = run_estimate(y5, x5, covs = covs3), + covs_ties = run_estimate(y5, x5_ties, covs = covs2), + fuzzy_covs = run_estimate(y5, x5, covs = covs2, fuzzy = t5), + fuzzy_covs_sharpbw = run_estimate(y5, x5, covs = covs2, fuzzy = t5, + sharpbw = TRUE) + ) +) + senate_path <- "benchmarks/data/rdrobust_senate.csv" stopifnot(file.exists(senate_path)) senate <- read.csv(senate_path) diff --git a/benchmarks/data/rdrobust_estimates_golden.json b/benchmarks/data/rdrobust_estimates_golden.json index 6102953d9..85a8b1580 100644 --- a/benchmarks/data/rdrobust_estimates_golden.json +++ b/benchmarks/data/rdrobust_estimates_golden.json @@ -6,10 +6,11 @@ "dgp_lee_smooth": 42, "dgp_ties_moderate": 123, "dgp_asymmetric_scaled": 777, - "dgp_fuzzy": 314 + "dgp_fuzzy": 314, + "dgp_covs": 2718 }, "generator": "benchmarks/R/generate_rdrobust_estimates_golden.R", - "algorithm": "rdrobust() sharp AND fuzzy estimation blocks (three-row coef/se/z/pv/ci, counts, per-side beta_p; fuzzy configs add the first-stage tau_T/se_T/z_T/pv_T/ci_T rows and per-side beta_T_p) for the vce='nn' no-covariate path, complementing the bandwidth fixtures in rdrobust_golden.json.", + "algorithm": "rdrobust() sharp, fuzzy, AND covariate-adjusted estimation blocks (three-row coef/se/z/pv/ci, counts, per-side beta_p; fuzzy configs add the first-stage tau_T/se_T/z_T/pv_T/ci_T rows and per-side beta_T_p; covariate configs add the coef_covs gamma matrix) for the vce='nn' path, complementing the bandwidth fixtures in rdrobust_golden.json.", "r_version": "R version 4.5.2 (2025-10-31)" }, "dgp_lee_smooth": { @@ -25,6 +26,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -57,6 +61,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": 0.14999999999999999, "b_in": null, "rho_in": null, @@ -89,6 +96,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": 0.14999999999999999, "b_in": null, "rho_in": 2, @@ -121,6 +131,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": 2, @@ -153,6 +166,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -185,6 +201,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -217,6 +236,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -249,6 +271,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -281,6 +306,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -313,6 +341,9 @@ "bwselect": "msetwo", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -345,6 +376,9 @@ "bwselect": "cercomb2", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -383,6 +417,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -415,6 +452,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -453,6 +493,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -494,6 +537,9 @@ "bwselect": "mserd", "fuzzy_in": true, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -534,6 +580,9 @@ "bwselect": "mserd", "fuzzy_in": true, "sharpbw": true, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -574,6 +623,9 @@ "bwselect": "mserd", "fuzzy_in": true, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": 0.20000000000000001, "b_in": null, "rho_in": null, @@ -614,6 +666,9 @@ "bwselect": "mserd", "fuzzy_in": true, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -654,6 +709,9 @@ "bwselect": "msetwo", "fuzzy_in": true, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -694,6 +752,9 @@ "bwselect": "mserd", "fuzzy_in": true, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -734,6 +795,9 @@ "bwselect": "mserd", "fuzzy_in": true, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -767,6 +831,384 @@ } } }, + "dgp_covs": { + "x": [-0.098792384331534144, -0.75096334573173196, 0.0086369055122323957, -0.58231053535155564, -0.61844154088880932, -0.57919913305373716, 0.19194280904418792, 0.016184004707901156, -0.33935372148729737, 0.66450238601453071, -0.54417327725669473, -0.34747015878078447, -0.57184304435264566, -0.57038050371626769, -0.91337847894721957, -0.13181046572266242, -0.12370909469295754, -0.43037612887623256, -0.38640456556971436, -0.18882268504053834, -0.2281502145133355, -0.71187032080824952, -0.098277118400281194, -0.38275422422180339, 0.045833811577561612, -0.53929749347102163, -0.28175345547031039, -0.62544573275242077, -0.31113924106069801, 0.16737579540245706, -0.58594251263420283, -0.4611334662135621, -0.52838549244685873, -0.021938264552633591, -0.39748559087889435, 0.26278270504814838, -0.029810348240146389, -0.31048095796982156, -0.84218693586804383, 0.017961865246677089, -0.43519490068556121, -0.8319573276309189, -0.50051698587460591, -0.036640964991057734, -0.51514243580815766, -0.81253511637511266, -0.10536790764621429, 0.11689506867324861, -0.383647730020102, -0.85780387976681394, -0.80180212052832867, -0.68717069196020608, 0.47714026015635613, 0.22941345085709641, -0.47995614236511164, -0.99017306427728302, -0.6857974116441734, -0.53000635838905841, 0.027743963939840244, -0.35548316762851495, -0.12401143015649529, -0.2557778635135306, -0.42299086787388263, 0.49554344071328638, -0.53100428951467771, -0.38451249181233915, -0.10694155786566251, -0.96659271796507695, -0.83566326367175192, -0.80704016215559515, -0.46010586918297613, -0.28908108235435093, -0.60910521199716883, -0.97997134221363547, -0.41560648202646822, -0.44150395123763031, 0.31193873372588632, -0.4094951160764525, -0.58784777702270996, -0.13139983275711264, 0.24440373413385807, -0.30692039398268867, -0.78924474430918223, -0.29216415562644249, -0.87858027792106597, -0.82231849124069345, -0.48576149002973845, -0.77937538212258983, 0.11607279337138388, 0.12074108301186959, -0.64154003662097314, -0.51711899136776129, -0.73079519600964304, -0.6347578793732811, -0.54987177773346996, -0.18942914226710461, -0.85577968009027883, -0.51938749504120463, -0.52113342450800615, -0.69795219725553448, -0.80749700200303365, 0.083533000565306548, -0.54233217956160062, -0.45919565359340186, -0.21406815766539444, -0.84132721503120489, -0.22251902999179396, -0.69728287861837934, -0.47224325391515065, -0.56245018709293904, -0.37402160817941121, 0.215566561862369, 0.21966919433287391, -0.21703375380315337, -0.32576167794380384, 0.12873302340584281, -0.37168364742213666, -0.66991165684692178, 0.22171753540903594, -0.61097187930575902, -0.21676087583200598, -0.43886708083993631, -0.80974780260122015, -0.63307623300860283, -0.049042418789582154, -0.86837090651072046, 0.017603745694690032, -0.64001631526542924, -0.55133140867140407, -0.85472338256250024, -0.42075408665435077, -0.2648935319935054, 0.38272586809372977, -0.75355862818841679, -0.29619390739185203, -0.63860690780952079, -0.48940832106827759, -0.38545056410395262, -0.0020045226218831091, -0.54001027094322263, -0.43132513350722124, -0.16704453447213186, 0.3861999646927825, -0.83070404681633248, -0.22257436163342248, -0.55423439150696041, 0.0092813350749805235, -0.53622735055090831, -0.7699555749716841, -0.74175109042975773, -0.3619446906548065, -0.19641776488338025, -0.25234036768291657, 0.23460739578662126, -0.56281402327311036, 0.036904220040889779, -0.065341981136180283, -0.53120290093306655, -0.60472880137332596, -0.73645773101552803, -0.22730409193739276, -0.28355333710198327, -0.72015744231032208, -0.39303398352341534, 0.11485816928232628, -0.76537554090264437, -0.3672211913054666, -0.72227774067940864, -0.049535414673555045, -0.29473560756128125, 0.2923063695081678, -0.52455910097402481, -0.10770767895560385, 0.2436831325289992, 0.27506668533950318, -0.52964992921655629, -0.59547065592423221, -0.6404925081631665, 0.067501749963643665, 0.65629942477735903, 0.057496282005974786, -0.60882533967742436, -0.46578077881822955, 0.042474156947414965, -0.35788422539985365, 0.19845863752327064, 0.019484995960031393, -0.40405242456220969, 0.17973143166364292, -0.077079101659043192, -0.10468424044032909, -0.20919527976146213, -0.59304278780163344, -0.88033089538371745, 0.22206530040664552, -0.42141959552802211, -0.28543739410271918, -0.60827503763912794, -0.97311623086068744, 0.15770617153148869, -0.65816103570797457, -0.44446336722018287, -0.31116416987362783, 0.34979496366354268, -0.40721089962318813, -0.23731761628448267, 0.051220966341900098, -0.70546549639310574, -0.57340358470173203, -0.37683723843803696, -0.67179983947111177, -0.40501160982151507, 0.13422979409061497, -0.35398084063760427, -0.75265800393552018, -0.3155115055600306, 0.004010175242989078, -0.71959358320061506, -0.66625716845961636, -0.45647428123552269, -0.17164408047360991, 0.18871762912908463, -0.10953171733315703, -0.21924520298553429, -0.37410115937724497, -0.9198071246984012, -0.88833496704217607, -0.047378659923094091, -0.043053226128340616, -0.65338269197194554, -0.33339615647704279, -0.46786374699927979, -0.14434578754734961, -0.27159689167371337, -0.019577383737302001, -0.5219343045943301, -0.13614399702561375, -0.10968817612562765, -0.62438545444357718, -0.4518164199055047, -0.40854887552533847, -0.45758067378885736, -0.63115893643230248, -0.41190877949228877, -0.73451969448252519, 0.1231083870624825, -0.13092147334837834, -0.39767418514912223, -0.83455051998131702, -0.41347843048631883, -0.56090164798165176, -0.30547133794981451, -0.93912329210610213, 0.048797054736913426, -0.90875658726232666, -0.62996355059015752, -0.19252050754943306, 0.32105408467023011, -0.2312609623545876, -0.24161101437755106, -0.51721341173852498, -0.98306947417036983, 0.34509010974247456, -0.52951406863286543, -0.50927134787868744, 0.24792037169940939, -0.59908942402644727, -0.22124290897398635, -0.82045316346632025, -0.68454625499200272, -0.38635106917481521, 0.14026933382349083, -0.16214963465846244, -0.72058382092680451, -0.093562526419845971, -0.09822321558613567, -0.4446259232978258, -0.31882849784414657, -0.32681448847567673, -0.82494631113367378, -0.87528642304788318, 0.2351526004972464, -0.63177489578150059, 0.018217287779625613, -0.45175097940774089, -0.77550830099247881, -0.46676301538832987, -0.42388722133538803, -0.56392874298086637, 0.4737402389248222, 0.20164614300994343, -0.12072486423829099, 0.46461552354493585, -0.20288303593965074, 0.13720153923861345, -0.68787900263550905, -0.53303213552201623, -0.86463165541861808, -0.086978404137494136, -0.69466860825454324, -0.68252021466921797, 0.3174657062610633, -0.30632229436113489, -0.49784968970162247, -0.79831678338700185, -0.4925138327419416, -0.058743633649716909, -0.041993007049933695, -0.50535498434685533, -0.11143230285606753, 0.0094280212194395308, -0.44472620224539372, -0.65308879649855978, -0.37027413355197303, -0.76290502691898798, -0.25083165651817751, -0.35108317737422634, -0.45971957492291071, -0.2103676543809585, -0.62562402896025726, -0.41499490310957754, -0.46685914652652816, 0.22371793306291465, -0.38318341684588886, -0.042986802903247079, -0.48654919952713904, -0.8972908033020166, -0.8507375849701948, -0.52593998813716836, 0.036516827288895604, -0.037962139127617811, -0.51300058044779218, 0.39613223010505338, -0.2741835647376053, -0.32975680817013442, -0.28640193617400689, -0.67579097906198449, -0.3283178862703765, -0.58261849297747559, -0.68547889724492661, -0.28787821376929246, 0.29063028306400618, -0.48895919501676854, -0.71612388337955446, -0.56403825817951203, -0.19685753145564011, -0.22202206014561388, 0.70435831132166804, -0.64374939107272056, -0.30993096235935536, -0.36683850844973831, -0.31137688561716403, -0.7101486906965222, -0.48596093155488518, 0.39852470781558669, -0.027203650945398561, -0.3629284209420055, -0.22335402560933426, -0.62020381403813651, -0.79081658998629678, -0.36939263277658652, -0.3351714705428499, -0.71965319680456885, 0.52386562742512632, -0.31002604923685007, 0.3292580549301023, -0.58352128309210638, -0.86172617613014801, -0.094805995010356914, -0.42528419918582439, 0.29935757307320321, -0.13245721653817377, -0.29678821815962342, 0.051019017790813859, -0.37676712119911315, -0.55708872753349459, -0.35010298144267038, 0.51934589364747974, 0.09604247598621396, -0.55813737705357758, -0.94300374231125683, -0.014867231560730887, -0.56502944408218325, -0.53070358706697585, -0.39786671692489151, -0.74738137378413716, -0.17342153417310513, -0.35303013142295192, -0.029979702156410881, -0.011089008729807781, -0.45422560972016857, -0.35003749406606721, -0.6698666315179207, 0.22844012281167569, -0.72672443122960539, -0.44699190922912446, -0.40584524710823722, -0.080178633509720543, -0.5204579378154951, -0.18762826892695506, -0.056835611918414131, -0.15355315644739753, -0.32347105998832337, -0.45902627679591301, -0.85190534386165206, -0.11920519971547261, -0.42272638418917319, -0.34513494677239454, 0.40430346805407535, -0.96915170750901991, -0.59925756027579724, -0.18351770880746088, -0.25568698704941473, -0.098832640533163874, -0.23835281540150055, -0.34725085885903695, -0.55198804650181654, -0.70297839147043417, 0.79577370194508301, -0.12981361632816746, -0.49577801133178911, -0.31539093298071341, -0.84412535352635953, -0.62117750014456763, 0.39436560003865684, -0.5333333545264678, -0.3526896108821671, -0.3838964309196774, 0.067955783595918362, 0.024988387936158096, -0.13735090396238847, -0.38159017795073069, 0.032014345868207394, -0.27773819937727973, -0.8245794469588078, 0.36005458864173212, -0.65013383001936764, -0.096312251223793721, -0.97931350622481483, 0.45881674832308517, -0.040298075913581588, 0.092667534406884799, -0.32686351371434363, -0.59367561541907632, -0.96649229276082371, -0.35459250164097533, -0.59398370977852721, -0.58075317256457437, -0.72012348686219019, -0.26323421628181387, -0.16281342977218494, -0.23003130877674449, -0.53681878826490315, -0.57143384555109011, -0.41867310768981336, 0.1064671798414718, 0.034280114099256354, -0.11819029373102763, -0.43309084730975922, -0.82926692194204532, 0.0026268944663658811, -0.23006623552027872, -0.26772850483612676, -0.4238830336716829, -0.63304419433423686, -0.37976254229426976, -0.4684646803371928, 0.094140352952430284, -0.41266885063165604, 0.30271798104739744, 0.068383050239177789, -0.25817017310372647, -0.34510403812443202, -0.6695284201370042, 0.13593877678595057, -0.074932683666365296, -0.55937153953618535, -0.11444294661129539, -0.62376680399854523, -0.66658426083902289, -0.68779202264597838, 0.082638474161639719, -0.36385156118307327, -0.15004747793545425, -0.46392349279692235, -0.3349240857529272, -0.39735875989458092, -0.3035030114754953, -0.45474559527540304, -0.24039973895254085, -0.19832533314394907, -0.29121547298366979, -0.32979523199923555, -0.18671111501269055, -0.32152511966620323, -0.72465425505661751, -0.66439873559417939, -0.53864465516788962, -0.015089580872771791, -0.84306712653535654, 0.30698060088099077, 0.019629185591858445, -0.33213605764106147, -0.40905321840236786, -0.33452015008595337, -0.11843791236187473, -0.33584201936440405, -0.92838199679987665, -0.064730371426660716, 0.16706181921638708, -0.26332339630849821, -0.31209748650544933, -0.16146379678519884, -0.70657838437681419, 0.068618467509983905, -0.20029445825164793, -0.61444800150359291, 0.38200930248965204, -0.26147320516474515, 0.059500154586515919, -0.15392643962225649, 0.042506577712479565, 0.37429047669950455, 0.51100617208472943, 0.15586688885512312, -0.32055254078458673, -0.56861445965156565, -0.79761483432220204, -0.80130882519636915, -0.9597677584314549, -0.77680282062025818, 0.20988003699550362, 0.46995718150945121, -0.85488612193696967, -0.71998985931442483, -0.67034757829061475, 0.6970096823343177, -0.79081076966897157, -0.20022824437270748, -0.10781507341074714, -0.6974941461438442, -0.55322280375813437, -0.48938751937687797, -0.50609291783072397, -0.75461451102281507, -0.45974304463089144, -0.28386243113357013, 0.40824242610484096, -0.099915156666916083, 0.4144762055507778, -0.10828631302525737, -0.80058114555614568, -0.37989162502594065, -0.3884795031386028, -0.65687370030060599, -0.64056772022030661, 0.071960746762922634, -0.88824741383236039, -0.40855020657660235, -0.72883678728525203, 0.0027247412179991493, 0.4836693616894725, -0.16796306364006308, -0.21204623631423436, -0.30404786335425349, -0.34631161443425873, -0.65582277420887336, -0.058923304156132894, -0.46434401974553297, -0.64567118942408719, -0.14187140150103561, -0.44638138690737061, -0.30550787111194744, -0.9011220563371074, -0.94307305141118514, 0.014467304227663735, 0.072088069982451985, -0.55387984047009597, -0.88010150663158448, -0.62515856138555281, -0.085691793796358828, -0.56413043028303289, -0.61367979466014955, -0.92780573094936858, -0.19704558005770267, -0.78830024902549567, -0.43858472315813146, 0.32633716243738209, -0.67401935322283424, -0.46680172516166274, -0.059506688238280869, -0.4837390305258229, -0.91513863605829049, -0.004444554222998498, 0.12622987481469683, 0.24074542744234151, -0.24524274854123884, 0.42584152593391367, -0.75309839328317874, -0.54838166670002109, -0.24072384681666115, -0.28991047317341134, -0.59939055661259721, 0.52030912559288223, -0.58409624711007069, -0.0048261880936668611, -0.35456791247497865, -0.48673986066440389, -0.20427349928138938, -0.42295244556227685, -0.34290735730776223, -0.7568612071987284, 0.07098622211120631, -0.11735025203237082, -0.810576268149937, 0.428236455803368, 0.17945565440203803, -0.32089545530447827, -0.54786517648999833, -0.17576713650875153, -0.015185414764083416, -0.49433883050162752, -0.78709671990766228, 0.10091623782282921, -0.097360842818867144, -0.40622316927679714, -0.70512288503370968, -0.11387396080064804, -0.41891100587507601, -0.22331444330597439, -0.066569783092853996, -0.46894872340693927, -0.50872958885970976, 0.0569422655527696, -0.78282891691401835, -0.40937660964177469, -0.016278588311873343, -0.89547550785023711, 0.083111139889086472, -0.85093700766943914, -0.59977749728963303, -0.80194982523506508, -0.52170859443382822, -0.4198990569624933, -0.49854275413130278, -0.20835414062135427, -0.32789155339167297, -0.74352702333468756, 0.017302635017663492, -0.17028308105106083, 0.49434530581245872, -0.052809519568527086, -0.48284179769002544, -0.26566555841950512, -0.3588053373447595, -0.55607265197777478, -0.78113113288775726, -0.33963086507654816, -0.70468931387590839, -0.90670852812212044, -0.69376180914038055, -0.77930126595709126, -0.17479623252185394, -0.36009987627257389, -0.5241704087306811, -0.39601139190068424, -0.50333499541259563, -0.32372588655270795, -0.10772013443750217, -0.49272440322917233, -0.51849602351547397, -0.087016340110643631, -0.51131291908690124, -0.13631934584195105, -0.60767388111654663, -0.41982535446196034, -0.14604011185673338, -0.71251889323084927, -0.74591974114223181, -0.74971180837066387, -0.3557903706686939, -0.67019972525215155, -0.42679258840395407, -0.17935996995363568, -0.35827333685295359, -0.77419829408608043, -0.44272537628688025, -0.60857974812283366, 0.090634916031693269, -0.23880263864025808, 0.11715482037863989, -0.48752032614751573, -0.50284770886410668, 0.044709943617880166, -0.51598702063299118, -0.82395626291452184, -0.49691142158676915, -0.89511667278154761, -0.10022192794791718, -0.31274894815473742, 0.029175408932053504, 0.75948958446230841, -0.12851050051368662, -0.0081919466076016967, -0.86395568971008363, -0.07257319904162518, -0.64457551243994815, 0.0539758299395936, -0.38033176296603555, -0.39897581629407863, -0.42057857965055867, -0.39325763544765069, 0.047379307275839722, -0.32789221845504346, -0.95877614155857493, -0.42522418456284095, -0.83036612622203609, 0.53639343268665152, -0.44156038846281676, -0.069073278782669445, -0.11545379758803986, -0.057252444357611454, 0.22414660851340318, -0.81127667463120134, 0.024794532737111297, -0.29305643634952572, -0.58875293490642533, -0.13426358139652428, -0.42886060449902541, -0.042419975287671274, -0.7191922069019252, -0.48393123520400394, -0.33364053503804636, -0.68406691086445193, -0.22488432439457939, 0.47785222772993019, -0.13221910963176631, -0.28846175929371287, 0.028183832252330676, -0.66698385956271133, 0.40049614608924333, -0.5174446937771412, -0.20568578025399331, -0.39299643540563833, 0.35324264024830487, 0.022743842474690634, -0.31168203662061422, -0.55027008751256634, -0.78386075853639769, -0.82768140039828453, -0.098795033587973924, -0.28690017880402963, -0.55245922879595732, 0.21892590063794559, -0.28248604128460308, -0.55127104187513964, -0.40991348812615969, -0.84014816404836157, -0.68859572927990254, -0.14699023660537058, -0.43147035326302363, -0.52224307160686245, -0.62426543845096338, -0.23201982686594413, 0.019463663362214723, -0.28945529966959926, -0.27065339652508802, -0.21810313504971757, -0.6755090320831143, -0.44639597321426117, -0.6663759608302714, -0.47825808542012671, -0.43398922308223375, 0.59523582365599137, -0.59874996571493244, -0.20487169537485206, -0.14934615573918053, -0.8885579486891525, -0.51189421430211479, -0.89306725454884051, -0.57394032004067563, -0.27364466939657561, -0.37711658584302521, -0.19802560831751259, -0.36479307007143014, -0.74324709304464454, -0.71929869367974764, -0.45856937355825034, 0.11544320617553616, 0.3161883420200573, 0.094873309460513866, -0.79032938971127131, -0.19541411493595551, -0.50399331917261581, -0.47118916794396482, -0.14545868984709098, -0.73769634910048487, -0.63411834214787199, -0.70305844527831662, -0.338866287228064, 0.090985515719038679, -0.81747436532814466, -0.82821107533240224, -0.5065926174775981, -0.50057090634110801, -0.94704761952580296, 0.36375401999612955, -0.14524513556671992, -0.30981018816532069, -0.58587112830957289, -0.62919949195673941, -0.88193681275548452, -0.081092355346911726, 0.013635753207260848, 0.056244508715797004, -0.2882140514401984, -0.87996533673141752, -0.048355884264355775, -0.096308737574737791, -0.4408897547531444, 0.27740633690384175, -0.070302880501387222, -0.68639053172596842, -0.094088233351785067, -0.6636050041311885, 0.352414793230867, -0.22224716569588632, -0.68854386024889469, -0.72973820794337496, -0.6071796248036101, -0.0020733120419724749, -0.68603679661699735, -0.2260477837678796, -0.19503601700367523, -0.29905841222651064, -0.22311406389419119, 0.27387555119619278, -0.75737327020699496, 0.05885824670319062, -0.39177179553571762, -0.44767373405001309, -0.13450418849484058, -0.60609321420979245, -0.72007492693508313, -0.79155367571973312, -0.59428839761009633, -0.57715247579435225, -0.79483641923797466, -0.46329583442211952, -0.0021120886103243031, 0.32361300204659416, -0.45027264984231274, -0.15099048950319993, -0.58693857028801366, -0.28311452183067143, -0.43323196486428883, -0.75671354668630686, -0.052841031479609168, -0.23070777925996278, -0.61745890708265083, -0.71713765038642763, -0.18185458402787724, 0.039265222310605585, 0.20158692868382988, 0.37513456448300686, -0.71120374496351324, -0.4935493690980548, 0.28015801740544322, -0.62001767123689622, 0.05970363539455481, -0.90904503346080812, 0.25514232366626555, 0.11060480379884385, 0.10956599685251112, -0.50107152073630534, -0.61127430118415704, -0.10103928859943789, -0.079623949075648182, -0.68622743270689646, -0.41522140458968337, -0.83681361437006518, -0.71192864309756287, -0.39460625950653583, -0.28322557754663003, -0.084624078330580943, -0.1008624569892469, -0.041223907085178646, -0.37327718335923854, 0.1215765687095296, -0.20539066023665087, -0.35777934074625961, 0.23308109447666192, -0.4297331474657301, -0.49415618813193563, -0.87348981247991875, -0.6907779268251899, -0.43558932690911356, -0.5328393231585935, -0.26187147184108861, 0.024563446719762849, -0.091697491290315347, -0.33865548856675787, -0.39524482287114215, -0.58450744429243195, -0.081174506380354727, -0.67887036861302752, -0.001277904168865196, -0.88618520286023716, -0.54611596261772966, -0.083563731607905867, -0.38613022172728073, -0.43795256589881892, 0.25220911722902617, 0.52966539991751271, -0.45196304245528252, -0.44979231932973895, 0.50793736672559087, -0.17514367189702928, -0.66967582073645127, 0.076626020030316822, -0.98604946113201053, -0.76824074932268371, -0.23235351353488021, -0.44654544229614357, -0.10876378530364361, -0.044840259119883763, -0.55912223431866725, 0.21543406275132337, 0.032599920411757966, -0.70456771549005814, -0.37849872671733387, 0.1872288026718012, 0.15645040117219922, -0.85708754814641375, -0.32403225188669083, -0.66492741285892687, 0.30823059508619211, -0.43551212966699471, -0.23639325097901898, -0.68857053832642401, 0.085034641424363011, -0.081918079277684575, -0.72890781628333612, -0.018880225730401712, -0.66184365551072766, -0.064127961583089199, -0.75149939905867658, -0.55668300708692708, 0.45959420482674673, -0.55469628089070866, -0.70713137062671394, -0.46881613643816167, -0.34338191018306963, -0.1154425525297007, 0.45539087709225035, -0.36673188452417826, -0.88999880274154941, -0.57811872834042699, -0.22763480802575731, -0.85830243342974955, -0.072536319628997425, -0.61108782850909948, -0.26693115951688884, -0.008997196422357745, -0.97064811985435073, -0.48417749948082145, -0.40653139599296528, -0.025449069098520183, -0.1936787089150791, -0.58472307168534776, -0.65083831253159496, 0.042905389466186161, 0.15233123216777189, 0.14703542019471061, -0.31412085425275438, -0.62360345997083211, 0.2782022351423159, -0.76458472597107585, -0.67902509557850554, -0.74416722058253859, -0.10061513516407472, -0.60167776310961651, -0.43333782014073408, 0.28936162177445834, -0.55853308512643429, -0.30000242046735215, 0.19518212690631254, -0.54114956995318808, -0.28207935436217135, 0.16205264822689691, -0.13783712773121115, -0.551636445171656, 0.48884692853206535, -0.59093046842597463, -0.14003179296022517, -0.53096936857345756, -0.1698919200806347, -0.66986252872960894, -0.21713171351400684, -0.90887223795476801, -0.19395466574047082, -0.3379292151203882, -0.42884264669791372, -0.50645805297604241, -0.27095363924671501, -0.22032116672711288, -0.69314695506228041, -0.62801739241882104, -0.72569334605767721, -0.057540318815002545, -0.44110538977696867, -0.22455421976698509, -0.62684266194175997, -0.010228232375623758, -0.69235109209535906, 0.3129454124592379, -0.40448000659032191, -0.49338091591983047, 0.020122953959172074, 0.034968624621620981, 0.020855880098475943, -0.54101571747279265, 0.43686137609403075, 0.0013864084873174143, -0.33467015598956951, 0.0088080571361563997, -0.38350923798600334, -0.87492285841603401, 0.41236609626656162, -0.050468664160111643, -0.46066415646036563, -0.41900163513465583, -0.67369051306229522, -0.30189662118745741, -0.66034833982688212, -0.60331870869869819, 0.34305752186554739, -0.019700148900862358, -0.27255410397780566, 0.017733297311805396, -0.53172501686925078, -0.76095483408946485, 0.25663269961670565, -0.29009466573683407, -0.4428723368467633, -0.7493794772515332, 0.17867264220419887, -0.054126792362298892, -0.45302201455298485, -0.19581833328031217, 0.20059487890778249, 0.059309725117445922, -0.11415757994743225, -0.30071111199056844, -0.50934203865738792, -0.72577962501347881, -0.69651778202143277, -0.6127108279577318, -0.54469673586728751, -0.21906460777108816, -0.31360347775570951, 0.0038145700520435533, -0.3464776792159171, -0.57431879340096992, 0.31033675569243724, -0.33118995188218581, -0.83538537608737018, -0.3641415027952698, -0.66683430848528835, -0.2032783716950427, -0.16397844227803793, -0.38114309233721877, -0.41795252249953807, -0.49985729040528448, -0.67226991607594488, -0.48874759966485737, -0.23581701208683226, 0.28622212637436406, -0.54483380964141559, -0.66445683704913827, -0.24364217642301234, -0.16553267552460849, -0.7911133804941638, -0.093184177949807934, 0.068803746322119386, -0.21160398756686494, -0.25086237382442433, -0.26438225991362274, -0.0029671944823971286, -0.56188283733320343, -0.49164675927237877, -0.36489178848536408, 0.14686245983929669, 0.34521594169973135, -0.58716170069826934, -0.57377552123164688, -0.16202084638858727, -0.24795403135168714, -0.42720694950978777, -0.45784337717557622, 0.0082970905371213544, 0.49782989066266659, -0.58707696421989963, -0.36679813577192777, -0.31592317735910413, -0.39151623531675228, -0.69250161757823836, 0.043532151851067624, -0.47069720791316139, -0.49277111585665423, -0.49375030120419205, 0.052548884679745056, -0.82119589941644622, -0.065657409592212668, -0.43287781473498965, -0.43419952103915072, 0.039027783458181142, -0.48295004471798009, -0.61184535385438954, 0.15246348330118109, -0.60199706595508162, -0.77274719967588401, -0.73598759590861496, -0.58074272575104291, -0.31208496000902597, -0.80474580369366311, -0.86731703017942019, -0.61431483972335155, -0.11542728701549565, -0.73766244560493899, -0.88010621777612985, 0.47113401126445176, -0.89369780357091766, -0.70117998288275463, -0.46339084465687308, -0.33095128589896294, -0.56401130231682339, 0.32008390522469354, -0.23657624123240983, -0.51531733059551277, -0.0097643808593248504, -0.19941077618140213, -0.43340248352072852, -0.39385372206095259, -0.56704580994299403, 0.37678217439235961, -0.021172063523960394, -0.74887697341482795, -0.051581805847525808, -0.017098902405743077, -0.91304354006600141, -0.38203194564730558, -0.16759442241428324, 0.36338952465454377, -0.24287100170247933, -0.47783427279931501, 0.48005446427752707, 0.020198191945755672, 0.3639902055115356, -0.46676872036382422, -0.12521171186675994, 0.064505426689067002, -0.6338345033914341, -0.069641430757653722, -0.071923660688000268, -0.3822092404683497, -0.37310786962191611, -0.75443131396695118, -0.3706376097533115, -0.81184975274457183, 0.34639660111513892, -0.35299489000596229, -0.4172261089022572, -0.8248504321515483, -0.60573132672215313, -0.37699010786977682, 0.20593978323242768, -0.27743298233566283, -0.65785980972595204, -0.70645288362529923, -0.18224299597931615, -0.57699466267586641, -0.16592407969902934, -0.74038950836419781, -0.56113744645930974, -0.64875582798163633, -0.19863027951876455, 0.091174894523250449, -0.2420794901554244, -0.90910801106243677, -0.40740571306896589, -0.068092302264587179, -0.032357328422611187, -0.81853953075696373, -0.34005876518322176, -0.66617164570751364, -0.81943266635674916, 0.16218390771646929, -0.24501799946860991, -0.49457411141945207], + "y": [-0.14946637752396147, -0.96892668919340419, 1.5672107716158625, -0.60727988922773257, -0.8194740518209267, -0.92922947641272668, 2.1813302866216189, 0.91710840611294142, 0.725710307718613, 1.6158433797780463, 0.050260314025605046, -0.066943026637642844, -0.48414399289152515, -0.82630499120637679, 0.72677510025559511, 0.35731428346451499, -0.082984053572651706, 0.29279623454829062, -0.42514398197600084, 0.43593330879624675, 0.00090668355334746348, -0.93506617105988776, -0.45923329464404883, -0.36097472051172586, 1.8633285550562069, -0.84204388795322593, -0.27473942403351137, -1.1195829187965045, 0.89060914010838022, 1.722757358435864, -0.91557026628483673, 0.096280093788579518, 0.41973771866343657, -1.0277010422608885, 0.68579288129479532, 1.1601136982359208, -0.28817912301803372, -0.50046988855937247, -0.98868784146066735, 0.93940679341861255, 0.0018505519949417182, -1.4382299480225307, 0.029938165133194194, 0.2281052672768549, 0.029433709213160256, -0.95424681210915929, 0.70953626687715965, 1.7764111715985409, -0.62628122527174446, 0.15424366053601971, 0.09167164883005352, -0.22190218325179073, 0.63644954597210712, 2.0539024553408169, -2.0851859802415413, -0.21854124623810911, 0.70854338832540076, -0.91857315573540788, -0.53114692902690719, 0.90434390658165775, -0.44026575197968854, -1.3030729225190572, -0.63992313245174071, 1.184613094955369, 0.048895574664282196, -0.95333591075839541, -0.12752445916575406, 0.28404767758332161, -1.3749731800965557, -0.7657719191813015, 0.96678267772575843, 0.72253573125358039, -0.94620367628352642, 0.32241131934146727, -0.23128272523644403, -0.75102640509112006, 1.206086837684182, -1.0990734786783358, 0.073695823081999803, -0.14028704787641624, 1.8588989169217254, -0.88235203798743145, -1.2805866075648837, -1.0499366310337099, -0.47287208921408458, -1.0552923067423206, -0.57238722654857366, -0.14420926578641791, 0.25910683526503459, 1.0716964391954695, -0.95232969952722712, 0.36709165268622923, 0.030733669524023066, 0.61046979929745115, -0.71585214618903503, -0.087875546503710045, -0.95508626403597563, -0.77834278601315543, 0.16323872926019056, -1.48797780594544, -0.55607760738350276, 0.73058484713753602, -1.2717444692062003, -0.14294806332646942, -0.7358229898031049, -1.5537615617645897, 0.45828235906801795, -1.3753022166969755, -0.21679628580473356, 0.63612558711831846, 0.47406290439567911, 1.3697223132305552, 0.3277488942172242, -0.62321677673630638, -1.169995100208804, 0.88750023498058728, -0.61114633789157013, -0.17316379754626784, 1.1866139105919353, -0.3993121534355536, -0.35044213405485969, 0.32199792296773233, -0.59837465291278691, -0.47656965960198217, -0.75002223005772561, 0.036567175599648269, 1.4346362779590558, -0.29712732707140743, -1.0085964803365375, -1.5128110962481878, -1.5698056622654382, 0.42015191610540481, 0.73258262551504683, 0.4862553520399826, -1.2816366551817024, -0.20359927921296889, 0.012389566497077675, 0.55832933981861765, -0.066498735115461227, -0.40508053873399619, -0.93151241630906112, -0.2298999748306601, 1.7850150314456674, -1.7210857988464234, -0.23427065150825493, -1.0141814637110189, 0.57723575024314389, -0.1730243023970218, -0.55282043522512747, 0.90304108145483108, -0.75625963973189991, 0.006055961538978194, -0.0012264438632859642, 0.51802011002679649, 0.49016003822679605, 1.5170518588532436, -0.69079621043019213, 0.016595161763278243, -0.89852310704018201, -0.27260545360284932, 0.31113995256685217, -1.0999084334674749, -1.5150353738756606, -0.41415775878991656, 2.1224952768402048, -0.71086863142543455, -0.14433996685475337, -0.062758991338140113, -0.18855144641962809, 0.032694423785979809, 1.701556110556907, -0.49680319352453695, 0.90444773207924334, 2.0838248883040662, 2.2550903201013184, -0.18565209812026695, 0.12822998904962513, -0.66949144673290428, 1.5305904068613136, 1.3546549422728553, 1.6677854503191547, -0.01672538330847393, -0.50487241360776103, 1.3326123620564416, 0.27963335890768798, 0.82193399982285342, 1.5849223193743593, -0.45248929208249145, -0.1575811907175152, -0.065225441593091127, -0.26131300875098662, -0.54462662074299395, -0.80530722301219126, 0.40566319435317127, 0.76583638817675981, 0.94815489406910181, -0.31952668544095225, -1.6900237626928445, 0.51199619305914301, 0.38673012940593454, -1.0547010841674807, 0.0040849712288062168, -0.063632348338276523, 1.7089902310485925, 0.98940677433535895, 0.6276352357307442, 0.42861815542375464, -0.13955035181179701, 0.33860873583934592, -0.69191521561249203, 0.34895393512025891, 0.095455186797481328, 1.0523856338649824, 1.1401242330364769, -1.9828083489306063, -0.12551702945430968, 0.46924933653793877, -0.17275274589984096, -1.001032573480213, 0.32552447724392702, -0.39545064570344368, 0.83082507261855687, -0.6263579013036078, 0.021396631437232472, 0.71466526816781328, -0.62671536240488901, 1.0097892257325798, -0.33094584719558701, -0.8300650012174654, -0.54524254661944804, -0.51671214544771193, -0.30374347145269542, 0.15217973685968, 0.32861914037303219, 0.030353430058857912, -0.80999648011858949, -0.23749478838823884, 0.51328711434313279, 0.96253831648589361, 0.10840970582415452, -1.0402841336378663, -0.47826291293502543, -0.55130388432547073, 0.3242268022803062, 0.31275567158118112, 1.9444912444396187, 0.011985280515446922, -1.5045663457689002, -0.12643799360632552, 0.13904836139716203, -0.85285785248717605, -0.30542351147316915, 0.52806544010242007, 1.567190377848672, -0.80681984862250533, -0.71439224995865924, 0.072583565884623391, 2.1781339192948965, -0.53718366955038077, 0.15630492339835247, -1.1465236432451384, -2.2296759809996267, 1.7905942725777111, -0.88906564459465187, -0.82094104856084793, 0.78247498301537122, -0.30814848977305226, 1.0147592234508784, -0.04754819463497531, 0.48662891026624805, 0.92474009652077671, 0.6358782965758798, 0.18464767922237535, -0.35699093551239452, 0.69921708380707992, -1.4901189597003657, -0.4623817977196179, 0.43065493043826347, -0.44219776096692709, -0.63763345372685576, -0.8513965157390343, 1.4311882624558212, -1.5767503947960542, -0.22278387529148613, -1.9566322274272325, -0.20193954852442894, 0.20610897153524185, -1.4896979748699353, -1.188721272521196, 1.2675357536551393, 1.3830706805424871, -0.51594137024661768, 1.053350274632316, -0.11207740905624908, 1.4920644147791371, 0.28787772799459899, -0.32675413618042182, -0.71371917834908272, 1.2769319862922721, -0.37667613765954017, -1.0629217861668161, 2.0648393439707151, -0.30361990648693987, -0.98226244464262746, -0.799983975861839, -0.11647224134432177, 0.27993260540803611, 0.075156732947265881, -0.89790450502320251, 0.98737203949595687, 2.3562492893808407, -0.54582612325020197, -0.3930742592458194, -0.13191542015322089, -0.25261266417876954, 0.007330257548523178, 0.73152757776741817, -0.41346440687536778, -0.14033733705007262, -0.69290945440171225, -0.94477051987820748, -0.75545470451390651, 0.83304177637839261, -1.0474043632975722, 1.280583363113909, -1.1820412252076751, -0.45735937541301508, -0.5331441321441579, -1.0940076041868005, 1.1470946920420115, -0.085451170321817344, 0.40515273223372072, 1.467584149025003, 0.048016278834154713, -0.32305753387289016, 0.2115603912173194, 0.6800251290830599, 0.85013743200286251, 0.10197626233002871, -1.2856935829317002, -0.2541596035623318, 1.7233065316379994, 0.0020798641432028808, -1.0939714530156845, -0.47007369913096836, -0.23146244176272357, 0.92665481317617793, 1.8389102353686462, -0.088989497813119448, 0.028461681158511054, 0.23175388091217441, 0.54955710198947727, -0.54268482851455491, -0.69848074733025112, 0.15595194255788697, 0.16961097242805107, -0.57853992423028588, -0.00025799600468573436, -0.20983397741753101, -1.0006446114786898, -2.319907221156118, -0.18452354616538622, 0.45131328014094996, 1.7783112532128476, 0.70118286659119244, 2.0163466992986985, -0.19112407766414263, -0.61508407269194143, -0.59443791538021395, -0.28545858657743417, 1.2291779029901306, -0.21391803831992226, -0.34426399805710339, -0.18845015483943794, -0.29785274251440075, 0.43077367606043693, -1.2074225762697186, 0.85915384686839424, 1.3128517003029117, 1.0603772185010565, -0.51252992264801556, 0.9357702510847562, 0.28506316953600841, -0.98476833445231837, 1.3702947806629755, 0.40966322072331135, -0.35734338381646757, 0.16561891428742387, -0.31250322322901525, 0.9011351696883837, -0.42302099896099071, -0.93835120058146715, -0.43998696667105791, 1.1817475886513185, -0.29269218389782259, -0.98601196265903301, -0.15839018136489214, 0.45655712864673209, 1.3567240139301009, -0.43102246574363257, 0.34922875142505161, -0.58007351343975777, 0.37139049305521793, 0.05037498563274484, 0.71940819151669844, -0.6846186922154216, -0.46575875590907057, -0.91031641074338088, 1.192590628910196, -0.1754108954463906, -0.8511052329665344, 1.1756735590423588, 0.27316107421834257, -0.38445608481131177, -0.24619313633410106, -0.58126838260060298, -0.17206328778318281, -1.2196010690341734, 2.3272777324258866, -0.41015946271562853, -1.6487809076734701, -0.024846669816053601, -0.64660093702924781, -0.78732050706229573, 0.8641186163075868, -1.9244271944051667, -1.9092164890390333, -0.60418263938730699, 1.0477819619454625, 1.5102177563161681, 0.31393791999134368, 0.62464305496213968, 0.96410801791062073, -1.2012193366681294, 0.010765141400362555, -0.085025070041730141, 0.29572876687824579, -0.45884339628231374, -0.38572893474369552, 0.41655103857294751, 0.3281304566405282, 2.1271765807075425, 0.19790428594822645, -0.2949506662725575, -1.166686908192214, 0.77779518684057947, -0.15905210220558846, -0.84058281801072421, -0.32220365707192444, -0.40123355072941191, -0.31011106713751019, -0.19015330952639314, 0.08223529076507452, -0.41053491596564584, -0.73415321819112878, 1.5759770129703699, 1.3692557348759089, -0.54082990584412438, -0.28207228375923671, 0.25090909651455651, 1.0074508499653809, 0.59572270236946212, 0.07387868225218247, 0.68474852708917777, 0.46712139067661873, -0.6046344397045339, -0.5573179259778418, 0.37341293503890338, 0.39837370886731216, 2.1500817836594579, 1.4245591925796901, 0.32577953370209739, -0.022458867631085233, -0.94890842949968435, 1.0972830518577263, -1.2556477293032486, -0.72991346090328968, 1.582997304951602, -0.50119898750857972, -1.348857687297027, -0.84474302541690205, 1.2461324013027566, -0.036869607009013122, -0.070240527958892851, 0.44365005952447817, 0.69622617673109755, 1.7928372079284283, -0.57906087686079488, 0.19849967939518443, -0.033075952269122105, 1.1581956134622651, -0.18602048129982521, -1.6509764246160707, -1.3745896655989647, -1.1579325725401062, -0.7656587593755404, -0.9271288307990746, -0.92374464164112324, -0.79018376736992768, -0.61037220282794591, 0.6202875435620302, 1.7831327131264834, -0.074217522889963292, -0.60531376326085184, 0.10083893245834659, -1.7417522975109838, -0.90327457072951667, -1.0189616237478414, 0.28997364743237669, 2.3420988332756867, -1.0138268852290835, 0.21520459201546277, -0.44199517164712809, -0.54050043142240245, 1.5400775649425185, -1.1194983865722141, -0.15561128124518186, 1.8057673464860924, 0.40552761298867906, 0.038280654651396115, 0.14618051506876334, 1.0582789600710667, 1.2762169655489259, 0.55539929614077155, 0.10071508551225183, 0.79082706020965188, 0.82731287079575666, -0.78755306034431893, -0.38190218619587579, -0.71762482747402745, -1.326379580222236, 0.64572159186047384, 1.6000635863812356, 0.72075836030278129, -0.88410280322800805, -0.87717865275544482, 1.5064228532008594, -1.0755929503883928, 0.45989905399483655, -0.24177143790441027, -1.6866282146975475, 0.35879165823651032, -0.66479920976548945, -0.17569903249384056, -0.53668243549911487, -0.39076713499345328, -1.1547049052409428, 1.4279572730065149, 0.29081102617764942, 1.4103261075210112, -0.22590495960119716, -1.0607611611325909, -1.3968550528148431, -0.11887059624073359, 0.46448671101867745, -0.99193192616846504, 1.8809147893839768, -0.72095453840886825, -0.45011612155511244, -0.74231765910158387, 1.6042129685830657, 2.0639293568747386, 0.020237197712582589, -0.17864511872931421, 0.40927140272508872, 0.4729106344623708, -0.40063323974321075, 0.053008940137441563, 0.53223234172249745, -1.1749726285480162, 0.46960047350144413, -0.72414943871234683, -0.47154628185415504, -0.92946789058670654, -1.741921208259515, 2.4587270494293487, 1.6135965788659157, 0.45855430849221146, -0.94053492030818742, -0.95343501969917011, -1.3307699173331984, -0.020643896144612595, 0.079069668324727796, -0.42349478291009768, 0.14013558038536467, -0.7481795882255613, -0.69247870517128796, 1.0712854432061401, -0.98305869989371519, 0.018891406286724644, -0.8694633131628563, -0.89604356169831134, -1.7041845548455181, 1.1448439586247072, 0.78077748986485584, 1.4098842969994438, -0.17678539791937131, 1.6845939364177669, -0.87976216687538056, 0.70680956333851386, 0.91652453313317661, -0.48222543195730022, 0.36184704094733372, 1.0432215964154705, 0.08204719981215991, -0.012710653978337316, -1.1668366307542404, -1.1396603715009661, -0.40532937788861179, 0.40299558286230885, 0.64341082262368532, -0.08967481393858312, 1.1698847702819841, -0.83400771128201867, -1.1606263238299523, 1.3725033394642105, 0.75577338680870176, 0.15238503026458899, -1.0316886745076184, -0.32460855449896059, -0.88610658158946676, 0.3053218800059343, 0.13639711635026514, 1.0701895535924615, 0.0022932911062459732, -1.1305614425744765, -0.70893869333568249, -0.42026408279425226, 0.57274099021880698, 0.36202998531093455, -0.28680591807974015, -0.88329116637671046, -0.06919412431528657, 1.7491093724064568, -0.29096698317532371, 1.2400857449785159, 0.066853023526703373, -0.32839898712109594, 0.99624986107858815, -0.55255214339607128, 0.51946308374281347, -0.77102359393582842, 0.21225913593798712, 0.78949435156435144, 0.066014472910101249, 0.74193754924266853, 0.68237237757313429, -0.044428898824108776, 0.4659087576796333, -0.59803745468281944, 1.5588258373532295, -0.19352739313054051, 0.23538640760177754, 0.23776577565568863, 0.056592431571153588, -1.2409993217496182, 0.61652910669854921, -0.10476547203022685, -0.28902994096905166, -0.71637090850466545, -0.65366644988744271, -0.52715012933881511, -1.1236506930730052, -0.32404807257527257, -0.7176578036156549, -0.23173740047142277, 0.10005189760749049, 0.29633003553819826, 0.7403203001107459, 0.25551267367914748, -0.20485405441325932, 1.3231836141385613, 0.011086615549484202, 0.089935598642298276, 0.18236505384402807, -1.125346050969007, -0.42537894385527714, 0.43911524674899272, -0.47561037248024746, -0.47058386741158242, 0.54488228159531871, -0.47867592744255705, -1.0661015629162263, -0.36245095203470606, 0.25874967912461766, -0.011363818846049645, -2.2596982788263964, 0.16196189520868504, 0.66524131991125646, 0.1869606458718584, 1.0741118838805985, -0.0096127058623892703, -0.066446418815824485, 1.2584595986645222, -0.43115540607044611, -0.68303343276418937, -0.85865102533662752, 0.64488456183540965, 0.60057473385314697, -1.1607058790600495, 1.338148676153265, 0.41512404509206324, -1.0536822828214498, 0.21465026661528525, -1.0632968348609628, 0.88996232434809763, -0.87735585014318052, 0.47163700888050997, -0.48493453432512329, -0.46897153904857675, -0.7642992128666195, 0.45493680140136084, 1.3366098856099788, -0.23516412772345549, -0.60122548259092801, -1.3520690169102387, 0.1605849221836321, 2.3128337327178574, -0.7578529488972171, -0.43564663802608727, -0.1281396759417778, 0.42393235538310847, 0.81095988538038932, 0.076161325520402515, 1.2284032159007752, 1.3392400133739786, 0.17162605049548518, -0.16541455929993881, 0.69025685627039057, 1.2055238279378335, -0.2857785620194444, -0.020840011872149705, 0.061940193830332946, -0.11253446224358019, 0.32281831684209994, 1.0217745582255142, 0.24755122789868444, 0.061583184521382395, 0.25369654058121327, -0.49009480752241008, 1.3598541886369877, -0.48916297026739319, -0.63706648470811378, 0.64467867400326717, 2.1677234600977546, 1.2276768211168569, -0.42930018311492624, -0.33372987155949341, 0.34126757225825632, 0.0095115146518554988, -0.054590087923108821, -0.74202566539989934, -1.0102856387725954, 0.83954603644618608, 0.11583993213752283, -0.10446618687466125, 0.20026370035417557, -0.37420527397589048, 0.29933865124461911, -0.95288355023160787, 0.42241509878893768, 0.43582810981029246, -1.6176103359206055, 0.22197101886020426, 1.302554218061259, -0.32098116970465451, 0.0020705341228337004, 0.55178026774770972, 0.54196184640527045, 0.094068071819459753, -0.58387541828655687, -0.091877401835875089, 0.48281538134146784, 2.3365019307323518, -0.15160308984325649, -1.0841517629574153, 0.25852332344883677, -0.85568133039127803, 0.059464185201964087, -1.1120036721087581, 0.5967058575020735, -0.14981179999097249, -0.38807335025915179, -0.06760013079354478, 0.94078583227349444, -0.87945475193141609, -0.17499487680092113, -1.5733764042049523, 1.1700227145064424, 2.4827151397450637, 1.4746487860255584, -0.79504211903197819, 0.47847297737279681, 0.72329612888730366, 0.52562044305396072, -0.25202069078273115, -0.92027572311314731, -0.6405916789601418, 0.32427765810603026, -0.97948023501345294, 0.71763300307063493, -0.43125146536001302, -1.6983900422616107, -0.38262250041618362, 0.23253527231379739, -0.79124815235442547, 0.97270462333062879, 0.79446113721131217, 0.8256366597068836, -0.80791308130255901, -1.0412480433707811, -1.3925365483419609, -0.32335792876964775, 0.74209650796581617, 2.2263420462112689, -0.25636528695489841, -0.31304037722904632, -0.24721698163051298, -0.31148101639548648, 0.81216450661729955, 1.7767800547530546, -0.92898791331939878, -1.0716611093380681, -0.36616589920982234, -0.67147820902807764, 1.2407956668570814, -0.05261361925963165, -1.0588478101687178, -1.1392363556119605, -0.69979820843900331, 2.1055975557487603, -0.1296371412880073, -0.43880159338425656, -0.32092404971876748, -1.1000448327082717, 0.13420888101235309, 0.52731135144336461, -0.64604671738694308, 1.9514932513226539, -0.6457615691545846, -1.5062682935678462, 0.26512257794399335, -0.042212298032995421, -1.4488878646351346, -1.2815450327642486, 0.1599627553528005, -1.234403347100917, -1.2894404354785236, 0.28730337112673338, -0.19582051272712062, 0.75451974108194297, 1.1457893144717799, 0.6810566831496061, -0.22230688563871417, -0.24876117617170251, 1.1090968854429688, -0.54237209523966379, 0.29964185835466972, -0.40888106770122645, -0.27423356404845034, 0.37872670425001237, 0.15037585986569973, 1.7866872863871162, 0.28006367815487432, 0.94856801479481079, 1.130943232856894, 0.81221563348169079, 1.3161863957395865, 0.71810943537008121, 1.9680039224002885, 0.28910437702521258, 2.113623908565184, 0.68544433266263927, 0.70473208095382356, -0.82916177423512816, 0.65258541542935133, 0.37405143576599681, -0.073887309786674127, 0.45682962605713762, -0.10213809472579802, -0.46072442981036865, -0.79762779480976609, -0.42513580210758639, -1.1793436369754935, -0.81091116655958428, 0.84219943787312324, -0.21946587816493479, 0.51369240570041463, 2.2933918782082503, 0.97135127175976788, 1.412517568684827, 1.8896856930726997, -0.054743155597349762, -0.8966742672710083, -0.40305682365697026, -1.0774354555812682, 0.053958245524442339, -1.0714779669873147, 0.38634279197252031, 0.98107943757737215, -0.17843869187728842, -0.144607328631365, 0.27884229049658604, -0.89866035268115507, 0.019824718435122651, -0.1937336400614427, -0.32939545155801153, -1.4020783222366453, -0.84627163450911413, -0.28444138739568481, 0.10787255347892985, -0.78576776848407248, 0.92246308189723525, 1.3071291188507237, -0.47099274834958371, -0.61733239848332733, 1.4391854997638334, -0.10238918583235457, -0.36418857032343444, 2.0699743238014015, -0.79510464251117119, -0.91367021865916653, -0.25833472389617324, 0.25565427701937149, -0.084482273773695868, 0.93569123020598988, -0.99793841899604463, 1.3303621868729676, 0.059859548457970924, 0.38121662554828484, -0.44358497946020592, 0.39708783820780863, 0.929856282664283, -0.090829343684968783, 2.2879941920666047, 0.5049993810596588, 1.2087598283025793, 0.062821355148853897, -0.77295807217384926, 0.12896698272173618, 1.7015496431721377, 0.18638860380549149, 0.25197530732591011, -0.68217575626956473, -0.19742606358040699, 1.001841752990033, -0.75716181151899664, -1.7020527482495285, 1.6727242088058449, 0.19646475251784942, -1.0156637484172204, 0.52962062491963491, 0.60546246651484203, 1.058649397764698, 2.0284869992432037, 0.078410669168321248, 0.17272596918155536, 0.1071038889059846, 0.015761549646340734, -0.050909889536773612, 0.45352265972091638, -0.57300472411185954, -0.51238561439380292, -0.57672651557419963, -1.4291094818056489, 0.28288658487342522, 2.3324875519913686, -0.40787227477714511, 0.32464222409462301, -0.043189620207725321, -1.1981330274077757, 0.70842249433123117, 0.97068942096962862, 0.89482694153872566, -1.0837841709521674, 0.24533136863897342, 0.94252766250196729, -0.1342986111952264, -0.40345707146375548, -0.38318482034050838, -0.17339364197756341, 0.38740233706726906, -0.17335635297947405, 1.0626114802081279, -1.2763583189474785, 0.44088573102477491, 1.4862428676095272, -0.32483260120306479, 0.61275338729606954, 0.83795598723333542, -0.53683416858823041, -1.3582071172952916, 0.49465241659032189, -0.22409167041168068, -1.0745364050278221, -1.0138337936032589, 0.77047363525970125, 0.30619212648043825, 0.32014447033420046, 0.60600536338300848, -0.053567314262050997, 0.61987026940634915, -0.96984988440530173, -0.8203672193667515, -0.079513805352688038, -1.2546466151243734, 0.50964122410062196, -1.0720139052654019, 0.15122706595599872, -1.2583181464121171, -0.0041586128055232374, 0.57136617148418944, -1.080329722473071, -0.36233861840293602, 0.36446972087254553, 1.4311226165897597, -0.51560812293300606, 0.061441331739723382, 1.0794267884164459, 1.0198214038397244, 3.3998840704854354, -0.89420914924051687, 0.84258210397509881, 0.75503123973789343, 0.081696979045706219, 1.0834813691843332, 0.69735265291597748, -0.55129341989168157, 0.964049985995592, 0.70386971341219218, -0.446861575377968, 0.68462950333479433, 0.28390113006877182, -0.91651295240309505, 0.0097697444265862909, 0.90824182687104282, 1.0909519111941834, -0.25424790539001879, -0.95455287425848878, 0.0044229782872118317, -0.20370075984564384, -0.51695023555516417, 1.3182455925598169, -1.2434124836954157, -0.49436380215160131, -1.2382330787218863, 1.2862450674485437, 0.44489437132075826, -1.4972066693927686, 0.9186261625856873, 0.93724324313536045, 0.8998100475168187, -0.7525674395587707, 0.5222629317362949, -0.789277978896188, 0.704396780806678, -0.0039820629264996563, -0.086879343346521198, -0.10875466701483411, -0.2660882016361405, -1.8690315414227621, 0.28107200676527916, -0.058979699761949889, -0.80435276881107376, 0.72946874023517139, -0.45355698829114238, 0.23713777395164776, 0.42853542354754925, -0.5191564034643652, 0.0058870640011425279, -0.12341343876112176, -0.053013071922591706, -0.64018965099359892, -0.31382185952576602, -1.4977606985940326, -0.10323482291491506, -0.27172201988255262, 1.596818586088286, -0.13146039231116002, 1.1103357463580414, -0.58439143783542513, -0.31526864868022264, -0.15804512770235402, -0.71429955629809816, 1.5390560939625917, -0.25698497112261925, 0.18646500360045593, 0.69218491028095763, -0.96572804046153626, -0.13544391846396697, -0.76153584745170644, 0.014560083267485591, 0.65820814981100173, 0.23678991417617656, 0.16639444633636416, 0.052536724147071702, 0.087343589706573799, 0.17242400568635669, 0.21005113956743804, 1.6342995739847608, 1.4237670812554637, 0.65222488043926252, -1.1621213332973941, 0.82226353451146039, 0.28388758859841301, -0.97373278862422574, 0.84258764148602716, 0.73289448337293706, -1.5074149107026868, 0.51340624020562309, 0.47845642101793229, 2.6802218529310471, -0.13584827371272212, -0.65628878219057407, -1.3151897818019627, -0.373794361562101, 0.72433108994441076, 0.46868048940861007, -1.119017018338972, 1.0793574910597801, -0.012552788191290298, -1.0110095836748869, -1.2569283718304765, -1.3405132347752056, -0.16693265038033711, -0.81897978993925391, -0.49274425046862919, -0.43689184731613895, -0.55930097361087194, -1.6852635886556655, 0.71614258297674493, 1.5151459625774082, -1.1482125345519227, -0.24930577376968951, 0.54411111087249464, 0.57413104078672272, -0.39028193043811343, 1.2874393038271306, 0.22269021655597215, 0.12538122390689671, 0.13491592054619089, 0.41152353370838429, 0.57905345159183286, 1.0218406813737653, -0.67072979460166682, 0.68040410772237037, -0.50656691995271808, -0.86770834154605747, -0.63082240507499121, -0.28200822519355379, -0.54029216370943844, -1.2617792388610711, 0.29081363460449761, 1.8876275364374684, -0.67137093261413683, 0.048030490263855585, 1.5730978247225478, 0.50185475933930046, 0.72690875241093522, -0.67810953078446001, -0.93134653348894914, 0.82951243916838391, -0.12908316861550714, -0.41726478524918914, -0.679742606008919, 0.043687188307701519, 0.2204981626812057, 0.060234727306202274, -0.97959056040270454, -1.3385531565064068, 0.63515527946244754, -0.47844761062988994, -0.35966241154392287, -1.20907801282122, -0.48869828530393211, -1.0677354194206836, 0.74226684355547856, 0.90861595251999772, -1.0069732528464033, -1.201783133843406, 0.30518271478831349, -0.58541214095961358, -0.25341617792125254, -1.2045382863685292, 0.029106564245061237, -1.5683064116523144, -1.5700485793739998, 1.1423933889335465, -0.75458742645926058, -0.46913410076059581, -0.20424386020688176, 1.9043095089665105, 0.72140845057712366, -0.70606728340394831, 0.33960378634811966, -0.40162360134915231, -0.62011713618110353, 1.5662282626963762, 0.27068712506610892, 0.026932137739002319], + "t": [1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1], + "zlong": [-0.48637761509529798, -1.3440161000291933, 1.1169080946239636, -0.9656366772468481, -0.45716517690541592, -0.45134310812019018, 1.5345719820739672, -0.025249937980976453, 0.44684879703103547, 0.4267432812901984, -0.45317256215947305, -0.40458888966224793, 0.58834965122504945, -0.84534587180906529, 0.70535551612867464, 0.25682794198823494, -0.069949688741047539, 0.35888779559191714, 0.21356415284114699, 0.54295013729292652, -0.23122445063346769, -0.71229348221535005, -1.2389021026020044, -0.41374088454125379, 0.10180440926254748, -0.70227874197774331, -0.75466545680051311, -0.29454304725051378, 1.8354224938100421, 0.60040937498728431, -0.89114150623887078, -0.062832724463063788, 1.0458551468224875, -1.0774607610921743, 0.87361721997869113, -0.36852182637149672, -0.41073164727035105, -0.41279127506997015, -1.2448991037083754, 0.26205083710202037, -0.54422584707397559, -1.1916172446763538, -0.32481822459028953, 0.53983089728487443, 0.078681136228205828, -1.4063458156224671, 1.0407175974883851, 1.1509541023852181, -0.276029280997655, 0.11892805465643497, 0.78306318719031021, -0.41428735870691102, -0.075002690831497021, 0.46602529030023587, -2.2306797712161974, -0.90446652886852019, 0.95297829923860422, -1.5442497463121576, -2.0101636979704871, 0.6954850890542188, -1.2709564132224733, -1.6246652711347103, -0.61957067159179446, 0.4468104237916154, -0.037388725336379353, -0.73509605816557788, -0.27087209430412107, 0.46603608979143568, -1.3016752050404654, -0.89789024570564979, 1.7133066570895836, 1.4418395160031034, -1.0471409367078055, 0.011551867421860151, 0.21360213928930671, -0.62907901852005876, 0.1593926431039239, -1.2016633192306496, 0.17313620962880216, -0.1240198151132887, 1.382047269270287, -1.0083728718041143, -0.55286277356045654, -1.3714535964661021, 0.47910865487575122, -2.1478721498136073, -0.56460703669446499, -0.64941736039439568, -1.3935273376769377, -0.34393497435914272, -1.3803777377359769, 0.44345686793603661, 0.19517998952378435, 0.59738073498606326, -1.1833080500144018, -0.8209213414192279, -1.2632262453256553, -1.3726003493244232, 0.58486681359149184, -1.57565947465794, 0.073078934906072368, -0.67864139268757928, -1.0556395565618455, -0.64151017397435217, -0.46231878673775384, -1.1563448434019239, 0.37589086253152143, -1.0356144906821143, -0.42195269285690157, 0.74559308182538597, 0.2279701334486634, 0.27713590377744579, -0.12951831609791423, -0.48734937385535565, -1.0151129375868528, -0.63836082095493385, -0.79147008287913767, -0.031617082588171697, 0.25273538135585039, -0.90134575754905122, -0.23644026727080389, 0.16016003314562949, -0.12722863028404691, -0.092578193194890274, -0.90855964354449648, 0.37607818443448099, 0.072351420125371643, -0.81411378880972274, -0.9706167806495104, -1.2114149753393519, -1.9561644759106609, 0.5272275076576467, -0.58224910435915112, 1.035177350702555, -1.0295998418906862, -0.022953135249420831, -0.59819802386335752, 1.4673934657156671, 0.62821071923448901, -0.97644544613948425, -0.68076997296859565, -0.34662501441577992, 0.69026853389736864, -2.0805445829938165, -1.0203608119333263, -1.0330994076016558, -1.2722164653082291, -0.2898826284382372, -0.36981070938678873, 0.7420673126965085, -0.81214278233721227, 0.55693769804301951, -0.77953966978636968, -0.61371738947610621, 1.4453893800597954, -0.019150205833996826, -0.74944031686315138, 0.3431638965313647, -0.44731813515061503, -0.71851023161653882, -0.19462896459290152, -0.78009500758669548, -1.6822784045260439, -0.043268287523045568, 1.1304761873207867, -0.94592988540446998, -0.78061917412360715, -0.10481808353932326, -0.31078858132851467, 0.23973610450460048, 0.82494744594743175, -0.67310820914889047, 1.2440880121955997, 1.9770425884644958, 1.4172778221246547, -0.18164729613226854, 0.3981537890812209, -0.7908043189366325, 0.28273324098704244, 0.56685920007851487, 1.0578975371119661, -0.033976679360608664, -0.21428144558669057, 0.39292974265695918, 0.51401557445079438, -0.66506138466645015, 0.95110414365261664, -0.43076037076263657, -1.0195078709993652, -0.73698827976762815, -1.3398858469449351, -1.3386660960204866, 0.21797126434341968, 0.11454479263098372, -0.63256735676424403, 0.74388987135956541, -0.3463508679715811, -2.058328405053568, 0.91177409492323491, -0.87654074952776961, -1.9614431056502795, -0.36134352316530527, -0.89647711916006589, 0.63642144261089217, 1.3900953929247475, 0.13862825250699451, -0.38949350627802654, 0.24518128935924621, 0.35627972974004729, -0.67804679101035337, 0.29245007315822358, -0.22701822180184034, -0.75396922664682331, 1.1811204156632369, -1.9760716540980712, -0.26442781689660588, -0.38705325946620195, -0.48989701737556263, -1.4718246537437429, 0.64665144020316356, -0.31295462353055131, -0.17506497061565512, 0.18526007115385817, -0.56161394410884469, 0.9897993336202876, -0.37586397856217146, 1.2929060932781591, 0.15791219064065096, -1.2032203442747449, -0.70725819908592524, -0.59029566422132573, -0.14334891080856399, 0.78306433479767312, -0.17288212003694939, 0.10104998014226702, -1.1996565843748153, -0.66526578773912415, 0.040056788798426807, 1.4656292508601094, -0.27449272627332988, -1.565425386986794, -1.1354841010484147, -0.1564000201526646, 0.88638359761321817, 0.20664616050840667, 1.1855702722992185, -0.2198269580713702, -1.1607374131940951, 0.36277167373694164, 0.13334849272308891, -0.45524264032273654, -0.017804781914709522, 1.3932754152161331, 1.0724455661030463, -0.43629988755162691, -1.0705395170846321, 0.16950288593322749, 1.2267513195653033, -0.6808278341693762, -0.35384107183333491, -1.9964744189309882, -2.7347921383025735, 0.9186723780164292, -0.72231578374503269, -0.27157555559290053, -0.54957690939121007, -0.67824181110534787, 0.69541174494328151, 0.61051416945137138, 0.98716361701800981, 0.84113780911476299, -0.044517204256313891, -0.1712953934420095, -0.43827589781574605, 0.30356063441635633, -1.1430790621904274, -0.80979566552547622, -0.40753223715482678, -0.14747772442542323, 0.082034960410943214, -0.70947816042127188, -0.39774135871567606, -1.1153894591965265, -1.9135851667983679, -2.4525137093353031, 0.18272684507344972, 0.031004222636946022, -2.3374396764022101, -1.1479714258607274, 0.049502682589132763, -0.12588058811859695, -0.41630021135316059, -0.85372297287373866, -0.65261611070132575, 0.36760961273311793, 0.64904162992521652, -0.40955886863713886, -1.7416816417368099, 1.1122519134088424, -0.68778466618677236, -0.92529576583870521, 0.43575108089279224, 0.40651610813168176, -1.0370131634721984, 0.40491527198390959, -0.23580378438394614, 1.3002431144082052, 0.57276831048172117, -0.7612995208174671, 0.99416962991305735, 1.2068739466274776, -0.27635816030200067, -0.727588318113358, 0.90889854389670865, -0.52807884996090004, -0.26071470443920397, 0.73199669361510877, -0.35584765329407986, 0.14707849870836553, -0.41857850366476046, -1.5765068675901683, -0.93553293353831934, -0.51837136766520286, -1.7535322194216465, 1.2031535235869901, -1.3954098225195259, -0.7208624053349233, -0.19118963484276552, -1.5933966747426658, -0.61743224549197384, -0.32906669866822136, 0.77537748014811669, 0.58209901185907786, -0.15598046002940932, 0.38910666399848109, 0.040503083161101322, 0.55899488674524478, 0.33626659585809299, 0.41975445825042146, -0.73764811582220302, 0.27437597076870568, 0.51107939591144458, -0.18234994163788626, -1.0943308960653102, -0.16201369703198484, -0.095643903258366261, 1.2793021404656484, 1.2028954323506238, -0.31575119362072956, -0.96016703250568081, 0.15871671571011942, 0.26316567054004092, -0.76750460249329477, -0.38198060420480173, -1.4054213324142897, -0.44622560573682085, -1.6060734396393914, -0.21532578610507591, -0.55473347206988255, -0.74060159685643967, -2.1927034578096127, 0.29871887438167771, 0.72844832509291524, 0.71327883771084233, 0.061972935386670425, 0.57761948690633402, 0.10950884660530402, -0.7469501806667701, -1.213126280712997, -0.2577196742488459, 0.46550869410576046, -0.03630738772011502, -0.26339356838386241, -1.1647934693307951, -0.92361437004959224, 0.18468113711830697, -1.5933181501028872, -0.59435505289423374, 0.017437135435074417, 0.84289232808952064, -0.27003945768913434, 0.62075140377953031, 0.42146677882821837, -0.60001903248762911, 1.308341126176058, 0.6652528171430212, -0.34041644602325832, 0.40606976858005922, -0.48588793787013851, 0.5309468466800662, -0.56925337063758596, -1.3546834580986307, -0.14577320805999905, 0.40341392617032273, -0.093310077007432046, -0.88318854494198074, -0.10714282113962241, 0.56417359589790084, 2.2349133470511129, -0.68281739076253267, 0.17483723135163751, -0.33966304169762146, 0.30029921267474496, -0.36475558441338479, 0.38890298587616373, -1.2302666652790972, -0.36282129052557388, -0.13211359963782379, -0.34102133712834193, -0.091544605459405959, -0.86701446050202158, 1.7700548082955938, 0.39677276726525612, -0.48412351548351323, 0.13047542072336085, -0.56922701297975253, -0.99653157208151455, -0.86391551345972328, 1.010841963886989, -0.75473242417730502, -1.8350024223561268, 0.45416528025794789, -0.59243422109424693, -1.2366665646138926, -0.62249533324597717, -1.9747703814243689, -2.3502323416845838, -0.45647817760292853, -0.3756404891065856, 1.523780486875963, -0.47045033840024558, 0.76354468534259989, -0.15087154325751065, -0.90870517966173137, -0.36459215436689213, -1.9158259875266708, 0.022099615043111365, 0.093702019038185064, -0.52878281683078265, -1.0839649991090208, -0.23146669394938654, 0.93375731197038514, 0.56784998595247682, -0.063081938213561256, -1.2208582564307049, 0.50521722057653129, 0.31713882135020588, -0.78522407411045592, 0.23763976593584157, -1.1109087765777046, 0.16699033289927026, 0.49392774729821232, -0.45678083857360507, -0.50025219758805017, -0.64355384219540812, 0.71351191922287382, 1.0546289933531914, -0.49823067660352188, -0.008581745065142643, 0.26966220186630313, -0.70069009578452557, 0.80888835559221128, -0.97098784767472957, 0.79279066343338545, 0.055315155126939963, -0.48783997917320165, -0.47913942380094154, -0.53919414397626919, 0.66118913837135407, 1.0985122860664713, 0.50841755312406589, 0.63415926916175236, -0.239085915615591, -0.64863929037702706, -0.32642737035930769, -1.1339290177172938, -0.5270971611148223, 1.6869103040049545, 0.19713479507618642, -1.5898721388355965, -1.4083906065799907, -0.017124562745663457, 0.46047988091146208, -0.91500996703467607, -0.47542068127416232, 1.3111498127185084, 1.9946818321420166, -0.53482075564905052, -0.50209556197320215, -0.17262752568061754, 1.5613049923244533, -0.079963705773140714, -1.9537864505344031, -1.7091387473021959, -0.45774051452857367, 0.39851990345036425, -1.2036942803011863, -1.0676524200579376, -1.3494288222587203, -0.33249489263361026, 0.0036184061341157925, 1.025523484497179, -0.31748062025501156, -0.025841568567048778, 0.14007462509041246, -2.4838581667693673, -0.6772038162962154, -0.97657581394265991, 0.4923468994872523, 0.98340370955316003, -0.096480294513022341, 0.15628653982609442, -1.0849878950120373, -0.13090344894058548, 0.18278408761949813, -2.1329846022312089, -0.65481454301584352, 0.95644617653187158, 1.2121067032655735, -1.1415836468462786, 0.25998544054334199, 0.67277731181492628, 0.75232887507523893, 0.20165027488738382, -0.69953923652153738, 0.63056835185143578, 1.3892838631493869, -1.0016244334505788, -0.41862839687128872, -0.3837192845852429, -1.2065878039511828, -0.1939390512839172, -0.089685815757368448, 0.85611578279955347, -1.4041967629191618, -1.4676376413976477, 0.5421144961751504, -0.89157936777004054, 0.3641442184767415, -0.24991475835265695, -1.5680398000882909, 0.23735748916200738, -1.2558205166648828, -0.7768069545153804, -0.48019265503181768, -0.5400137896433812, -1.5025478569164346, 0.14387566891529444, 0.16323613041054152, 0.10644243880165864, -0.50207386044803171, -1.1334144888180926, -1.6356385493879002, 0.63794290267921505, 0.16857732440579798, -1.1540017960232325, 0.89984785740521711, -1.0253587017065506, -0.76906126812252662, -0.47327558081744836, 1.8167976866813393, 1.001640501089496, -0.29146495592826294, 0.18734968251593859, 0.037053857588682404, 0.6146151608122743, 0.23207606721742235, 0.29328214516808931, 0.058096062923265468, -1.4388446260872978, 0.61768115590655381, -0.76171718911533048, -1.1351776541915175, -0.63160304303996773, -2.2699484552533562, 1.6091157778298066, 0.42481883027204709, 0.43633830024660658, -0.66558921168407981, -0.59372142790758953, -1.4957248513855528, 0.1040363236444406, 0.360708994378512, -0.40211045571044857, 0.0036965229083212175, -0.14886874519234333, -0.99999637795273455, 0.028698794617585011, -1.9210372135671947, -0.02430136191368526, -1.1126424355215669, -1.8518546569549859, -1.2549812663921522, 0.30562315265062212, 0.25757209542745463, 0.33756202912882599, 0.10060772649558472, 0.37035328270367662, -1.4284276040537227, 0.54903858202195654, 1.6043966067949671, -0.49831594920879474, 0.45666230553103049, -0.28291170443488112, -0.28902878427942963, -0.10283096973508082, -1.6753135404039468, -0.94915943063836861, -0.62031052155365285, 0.23861140562497646, 0.098665613335697344, -0.0078056631105946073, -1.1134094821943663, -1.2869925976977359, -1.1277226537042115, 0.21583770029664356, 0.083330443361795209, 0.36530562045141546, -1.508897154962785, -0.37717388291955672, -1.6338365690193477, 0.1223132669568181, 0.24739137144849449, -0.46833249157318979, 0.23544157848038738, -1.2885804477449538, -0.09944300538692602, -0.64462406407920714, 1.6552313677578541, 0.36998432595983044, -0.84959157751892977, -0.58306851863877363, 0.032424859743326928, 0.58205676385538729, -0.33558349247581637, 1.0247267338230563, -0.055512361319765779, 0.016730951489388912, -0.08720558058319991, 0.06608141550766411, 1.0296729617802853, -1.0849766828845648, 0.3016401836249375, 1.1587398269794695, 0.44687045706889239, 1.1417139413029291, 0.38610433318643572, -0.39642422172176478, -0.061981660653677204, -1.0174283281684011, 0.072270007444530965, -0.2055713839315407, 0.66893439185948922, 0.025876840113132488, 0.26358869160365778, -1.1901475916547852, 0.45369762525850221, -0.078036651536088242, 0.3806058288297966, -0.73767287454553498, -1.30284464945899, 0.23240013336124843, -0.47799028647496883, -0.082908933229019086, -0.47350212108561873, -0.32165054524035602, -0.052390902828259978, 0.26502150761697763, 1.2692928387748248, 0.82402203097049265, -0.33057443513966472, 0.37092359420313142, -0.33364790474032957, -0.16107676041690017, -0.50309496354880578, -1.181719483301733, -0.92721235597910767, 0.71452686621498229, -0.56909478221117149, -0.81639976158724803, 0.95322833607541124, -0.72781989174839545, -0.79116988089083051, -0.39838637565255453, 0.28363610944376061, -0.52638851685960486, -2.8873239788074545, 0.45543593288079864, -0.40799734685206346, 0.37218015195194226, -0.18766062180870058, 0.35892211770829635, 0.096938406375938035, -1.0093450749219288, -0.42819624415352875, -0.27884733723901883, -1.0344270679424461, 0.66808726385978523, 1.0535072472435891, -1.3824126136907153, 0.42859873705727991, -0.88904409985276123, -1.0877140925679112, -0.481633920434548, -1.2372052469471804, 1.088224109690064, -1.4184942357541104, -0.28381807852909241, -0.61193912311259058, -0.57509416165366778, -1.3866200808548712, 0.52540896899759904, -0.2425564401629961, -0.13646344492080592, -1.2231695898447463, -0.93577379870990685, 0.97458364581213708, 0.64799767580113898, -1.1098217113013886, -0.1501311867978985, -0.38987170088083622, -0.20550395797769119, -0.6646399531404672, 0.56664382382151124, -0.4778267403965113, 0.97008041285223823, -0.32599992039966486, -0.25742436495271515, 0.63306742881129541, 0.92632905112412378, -0.92278459017493597, 0.35128634183038843, 0.027123956887957379, -0.54240945425567588, 0.41440922353391424, -0.64467225185934052, 0.024569711719841805, -0.37937642498444935, -0.47055691540863237, -0.40587793539789663, 0.23949390095688183, -0.52699758926588314, -1.3358268853477115, 0.026562406353848961, 1.7085125838923791, -0.082713542457450884, -0.060188100236589728, -0.056532114035488856, 0.63584993190382233, -0.018413742978602543, 0.54265153546479028, -0.89704376599718016, -0.78312118795809837, -1.0114855010270358, 0.37593861926735439, -0.080752926571063782, -0.26784675896329724, -0.014209249004031421, 0.50776074762690471, -2.5307739803058569, 0.68190571765849439, 0.550501462359686, -2.1511725498927854, 0.58609631530444939, 0.82312859245623693, 0.58643755387350138, -0.58461083660642299, 1.4521027626139649, 0.91887419862634068, -0.20988475459937456, -0.89277346684414072, -0.086997807600015409, 0.79497839622242117, 1.5235647976907973, 0.28179891998302387, -1.3639191909467951, 0.084863040128695055, -0.56508718505804278, 0.63417265763386599, -0.76640596797234206, 0.50498978706438324, 0.39930742151960913, -0.50206882057387592, -0.078365052024526108, 0.51219777983729375, -0.96058136624852786, -0.19764549768851289, -2.241159024178029, 0.74671051737355942, 0.93823572675466271, 0.9627827979887883, -0.77689207139509264, -0.23322579259501056, 0.10512564783892736, 0.46187895489257014, -0.37850874675186758, -0.24324610949431746, -0.15420172068444549, 0.68412060956792831, -1.5715679020306228, 0.16593176101815557, 0.2653559998852546, -1.7592095582317795, -0.34966509822573694, 0.033584680608582884, -1.0874476274959639, -0.49105504380389131, 0.55137430007467614, 1.1968544819316052, -0.64991143372389204, -1.6503904869614998, -1.3821841801326988, -0.66981390690056042, -0.16257927715546691, 1.4904507545792796, -0.67961644535225862, 0.37472168927955352, -0.82189743269758186, -0.88303716524872711, 0.67992228384205289, 0.71781302309055817, -0.16971080319420731, -1.4965211045823918, 0.12057246751787404, -0.089468187301840724, -0.31376186829730685, 0.4714504462651426, -1.1273514496701373, -0.88737108274013365, -0.44659521937691427, 2.3121688827893019, 0.10346150960616618, -0.66980671792646207, -0.76792659705515431, -0.56091632667071478, 0.28621622582690426, -0.79755995797047063, -0.39402139869781949, 1.1767562993756466, -0.41016218430769108, -1.2095464482713476, 0.14179629818235984, 0.44948108788760283, -1.693663233987817, -1.1832695725435929, 0.64113593917896161, -1.1466207473609464, -1.3302105920530609, 0.51471431032614978, -0.21751255314461662, -0.21708803971694868, 1.9719018800042247, 1.1476081752725358, -0.13913552654495542, -0.92270409916279705, 0.64669983703136069, -1.0515113856057572, 0.013610034975177633, -0.88199726837082748, -0.49563869553871343, -0.3093159723299867, 0.67588854040573931, 0.96785770626501821, -0.4765599381428457, -0.13705989205176861, 1.0800083003836591, 0.40794548520186447, 0.97261408906575719, 1.3657506625120659, 0.88496154340358801, 0.53468034029600298, 0.97703611240784882, -0.66324168035158904, -0.020116152715180066, -0.93840502785336466, 0.75304558099492969, 0.14644665562819875, -0.77053156803885181, 0.74532359210025778, -0.6385469198826379, -0.28572679438697801, -0.97294204354300384, -0.19047535924152428, -1.4874404448746481, -2.2876786166313301, 1.5492852837669804, -0.45772757386709534, 0.6097179013005577, 1.0120064991932243, 1.0250914460798515, 1.749972930699127, 0.92313298854577064, -0.27554848531584669, -0.93182511404373947, 0.16281994729085131, -1.042976655600077, -0.46569593296211909, -0.91474604409185611, 0.82503483404182787, -0.7742366900345552, 0.10056301088686737, -0.46355951768413678, 0.011479525889298986, -1.126736965734491, -0.18876323716793786, -0.56624998597142295, -0.23587249272498487, -1.1527479118822348, -0.56831548443413138, -0.028725632073457796, 0.5751378853549336, -0.82854830723841233, -0.26486638186337208, 0.042969675747515368, -1.0502522053718759, -0.61443606333194878, 0.21827196479875857, -0.11378441448267138, -1.0925506503670239, 0.96418048787858612, -0.71392962653398151, -0.87163152405778388, 0.36086765495607714, 0.44146505699201316, -0.068981994506296274, 0.75133640505686139, -0.61848610431167417, 0.68957211802083751, -0.54929249894850896, 0.055243242476059939, -0.033604541700982676, -0.37840494673317715, -0.62075103427495193, 0.53281925883748782, 2.9754388524500808, 0.15657996206723912, 0.59100955673298949, -0.091487671596632431, -0.66851036852495715, 0.4809547510896397, 0.92279827766740097, 0.10008165979815581, 0.94376207460471084, -0.72934611174175257, 0.30011095960286061, 0.81733354462687824, -0.75334811207835339, -1.8155491635597611, 1.2806799867387726, 0.64203175273306257, -1.3526183589289962, 0.2714372591763044, 0.62228032436532943, 0.61898933686038049, 0.44205908233631996, 0.033165188854475891, 0.21455542020733548, 0.25800908209881912, -0.28433373278240726, 0.56212494557206261, 0.25785147749151877, -0.22135869080533105, -0.13827176555909651, -0.40461606750057, -1.4108760637733768, 0.9461415127601478, 2.0551782624451533, -0.45124636760990938, 0.1954322902864703, -0.059410374288066586, -1.4403155151723783, -0.36409703599629828, 1.1645494477917433, 0.40901035230630101, -0.66787501921690828, -0.29483757066830973, -0.24595743705867706, -0.11504067315699179, -0.22973071507462894, -1.0435010405500129, -0.67001746912020821, -0.11850918758915827, -0.032189922418672495, -0.18479820706846917, -1.4639801291493382, 0.43506740130216326, 0.32015631271007239, -0.21855057333455435, 0.86273580478725054, 0.1597360069703328, -0.54378576185930161, -1.5517785044275674, -0.59619890626843142, 0.24961931104915436, -1.254185499971161, -1.1049653462542475, 0.058321920222424134, 0.6874405558617086, -0.52919401496073493, 0.81675215863624628, 0.32481253921460379, 0.52544432337061675, -1.3932348231274481, -1.412738211659641, 0.12905486377552089, -1.2284516384001887, 0.80535928867707463, -1.4119960974262948, 0.29978732724772633, -0.80784092007061525, 0.34268253552371314, 0.33919017041049326, -1.0360259000461594, -0.39599958986846134, 0.52252664235652502, 1.016551901260105, -0.020288353131479864, 0.59795464896993722, 0.043254146729118995, 0.42382650879343542, 2.3166413921803226, -0.77304763983627911, -0.33037295070608563, -0.50355750883433947, 0.48606594046021157, -0.90829612040874741, 0.10440815385918678, -0.30232201134383851, -1.1525483792798012, 0.71058695468945299, -1.0349901402924162, 0.47278918520167607, 0.56425651744738581, -0.26966614778561981, -0.78593943265402522, 1.2528688313531546, -0.59517298775419014, -1.0470523738882731, -1.790005364966762, -0.7646747053259173, 0.19625221634781143, -1.1863586124009116, 0.066842711152691997, -1.4838821826769271, -0.80580758675965058, -0.993667334838923, 0.17088043156059768, 0.84717122837276149, -1.2706066596297749, 1.3428020003108647, -0.13822964858806674, 0.65951331914755373, -1.1166354659302129, -0.019538999740630253, -0.62421789172758035, 1.1833786376674571, 0.43229352589190906, -0.41367145766940677, -0.30387093717936342, -0.20038784553915109, -2.6665996007758821, -1.0071237814138558, -0.35828187497028818, -0.36796973574934044, -0.97381630206773651, -0.2745781392199575, 0.36435513834437561, 0.211425544664883, -1.0950947303370344, 0.61029035594320225, 0.47631188399461699, -0.26602947794876136, -1.1085769517957427, -0.054445291738422219, -2.1025719292479241, -0.47222767019281497, -0.42306585915595168, 0.42810098991076262, -0.29628772739840747, 0.86770268993890576, -0.22314296013481238, -0.62267206376130824, 0.28645249272632772, -0.69439125579488026, 0.72242779409014257, -0.79615809541839222, 0.49629302433438444, 0.25053136381559721, -1.4409444119282606, -0.20917562224583358, -0.31741649662132188, 0.52675668458564506, -0.4512685117973817, -0.83372469630036161, 0.67793894412327038, 0.48377869607529256, 0.070558042032781182, 0.77945265872117941, 0.19991199406579901, 2.4303114444698934, 0.34510653036960864, -0.95116605112329689, -0.56830128845333139, 1.1445036820775201, 0.034211390737170694, -0.98888119540988728, 0.46658711350956561, -0.47754889435744574, -1.5161978387951578, 0.66431525609253894, 0.9515780750533942, 1.5561003250662222, -0.63021444213890243, -0.91639312770533632, -1.4621741285854974, -0.46821192635176473, -0.52638810888730503, 0.53355500616604856, -1.0838282703907072, 0.75312652395186563, -0.34494194972685155, -1.696112029485749, -1.2669976164580983, -1.7443014457959833, -0.093993055993619901, -0.77608327252603948, -0.19373210900330279, -0.096651600728096282, -0.25680401788649171, -1.9454892762552332, 1.4134821702874776, 0.45898639489050552, -1.0177858818383938, -0.58958948986941029, 0.57401355038850299, 0.72891732281529287, -1.1850470417694456, -0.040118295798734493, 0.94296342791606902, 0.55648141544242702, 0.38659073140987948, 1.0273534261649009, 1.040231145502289, 1.2013886676968029, -0.42079875742490536, -0.31009157125733655, -0.54765737776560797, -1.2657625406999911, -0.74588244269029191, -0.21869728033836389, -0.58835881903048981, -1.175826655158104, -0.49317667692038553, 0.39023330498748954, -0.33151117469138697, 0.38375037511827614, 0.64042738333722316, 0.1714093733587353, -0.13051614584457683, 0.14769550233701523, -1.2768262323199586, -0.53091216245998274, -0.23977565424123848, -0.27024743240373511, -0.68701650594692476, 0.8247372871854628, -0.57974813791493207, 0.091453516519726885, -0.75028774830855305, -1.0450169988000668, 0.04837658160632656, 0.21278057276244045, 0.38455395690680655, -1.2380909627705481, -0.17584416970277109, -1.5311641996167051, -0.72908796364218809, 1.7188440754236272, -0.5974603600571311, -1.3608557846768339, 0.31617448676535936, -0.36872647673392717, -0.292661348334602, -1.0654969807798056, 0.41421216719914233, -1.5689368551457985, -1.7823013527099913, 0.52762294850537617, -1.3221452185005962, -0.34638485743793018, -0.1775725206202155, 2.429529758295037, -0.0066764902329111038, -0.13300614054449617, -0.45958600911729969, -0.076456950205252006, -1.6352717170869522, 0.13645741947653708, -0.1599307248528942, -0.12039032775557937], + "zb": [0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1], + "zdup": [-0.72956642264294702, -2.0160241500437897, 1.6753621419359455, -1.9484550158702723, -0.68574776535812387, -0.67701466218028528, 2.301857973110951, -0.53787490697146467, 0.67027319554655318, 0.64011492193529762, -0.67975884323920954, -0.60688333449337195, 0.88252447683757418, -1.7680188077135979, 0.55803327419301185, -0.11475808701764756, -0.10492453311157131, 0.53833169338787568, -0.17965377073827948, 0.31442520593938972, -0.84683667595020151, -1.568440223323025, -2.3583531539030065, -1.1206113268118807, -0.34729338610617877, -1.0534181129666149, -1.6319981852007697, -0.4418145708757707, 2.7531337407150631, 0.90061406248092646, -1.3367122593583063, -0.59424908669459564, 1.5687827202337312, -1.6161911416382613, 1.3104258299680367, -0.55278273955724511, -0.61609747090552658, -1.1191869126049552, -1.8673486555625631, 0.39307625565303056, -1.3163387706109635, -1.7874258670145307, -0.98722733688543429, 0.80974634592731165, -0.38197829565769126, -2.6095187234337005, 1.0610763962325778, 1.7264311535778272, -0.4140439214964825, -0.32160791801534755, 1.1745947807854653, -1.1214310380603665, -0.11250403624724553, 0.19903793545035375, -3.3460196568242964, -1.8566997933027802, 0.92946744885790622, -2.8163746194682364, -3.0152455469557307, 0.5432276335813282, -1.9064346198337101, -2.4369979067020653, -0.92935600738769164, 0.67021563568742315, -0.55608308800456907, -1.1026440872483669, -0.4063081414561816, 0.19905413468715349, -2.4525128075606979, -1.3468353685584746, 2.0699599856343753, 1.6627592740046553, -2.0707114050617084, -0.4826721988672098, 0.32040320893396007, -0.94361852778008815, -0.26091103534411414, -1.8024949788459743, 0.25970431444320324, -0.18602972266993306, 2.0730709039054305, -2.0125593077061712, -0.82929416034068482, -2.5571803946991531, 0.7186629823136268, -3.221808224720411, -0.84691055504169754, -1.4741260405915935, -2.5902910065154066, -1.015902461538714, -2.5705666066039656, 0.16518530190405489, 0.29276998428567652, 0.39607110247909483, -2.2749620750216026, -1.7313820121288419, -2.394839367988483, -2.0589005239866349, 0.87730022038723776, -2.8634892119869102, 0.10961840235910855, -1.517962089031369, -1.5834593348427681, -0.96226526096152831, -0.69347818010663076, -1.7345172651028857, 0.56383629379728217, -1.5534217360231715, -1.1329290392853524, 1.118389622738079, 0.34195520017299508, 0.41570385566616869, -0.19427747414687135, -0.7310240607830335, -1.5226694063802793, -0.95754123143240077, -1.1872051243187065, -0.54742562388225757, 0.37910307203377558, -1.8520186363235769, -0.35466040090620582, 0.24024004971844423, -0.19084294542607036, -0.13886728979233542, -1.3628394653167448, 0.56411727665172151, 0.10852713018805746, -1.2211706832145841, -1.4559251709742655, -1.8171224630090279, -2.9342467138659911, 0.79084126148647005, -1.3733736565387267, 1.0527660260538325, -1.5443997628360293, -0.034429702874131246, -1.3972970357950363, 2.2010901985735005, 0.94231607885173352, -1.9646681692092263, -1.0211549594528935, -0.51993752162366991, 0.53540280084605296, -3.1208168744907248, -2.0305412178999895, -1.5496491114024837, -1.9083246979623436, -0.4348239426573558, -0.55471606408018315, 0.61310096904476286, -1.2182141735058183, 0.83540654706452933, -1.1693095046795545, -0.92057608421415926, 2.1680840700896931, -0.52872530875099522, -1.124160475294727, 0.014745844797046992, -0.67097720272592254, -1.5777653474248083, -0.79194344688935225, -1.1701425113800432, -2.5234176067890659, -0.064902431284568352, 1.19571428098118, -1.9188948281067049, -1.6709287611854107, -0.15722712530898489, -0.96618287199277197, -0.14039584324309928, 1.2374211689211476, -1.0096623137233358, 1.3661320182933996, 2.9655638826967436, 2.1259167331869819, -0.27247094419840279, 0.59723068362183129, -1.1862064784049489, 0.42409986148056367, 0.8502888001177723, 1.0868463056679492, -0.550965019040913, -0.32142216838003584, 0.5893946139854388, 0.77102336167619157, -1.4975920769996751, 1.4266562154789248, -1.1461405561439548, -1.529261806499048, -1.6054824196514423, -2.0098287704174025, -2.5079991440307299, 0.32695689651512949, 0.17181718894647557, -0.94885103514636604, 1.1158348070393482, -1.0195263019573717, -3.5874926075803519, 0.86766114238485237, -1.8148111242916545, -3.4421646584754191, -1.0420152847479578, -1.8447156787400989, 0.45463216391633821, 1.5851430893871212, -0.29205762123950824, -0.58424025941703983, 0.36777193403886932, 0.53441959461007094, -1.0170701865155301, 0.43867510973733537, -0.34052733270276048, -1.630953839970235, 1.2716806234948552, -2.9641074811471069, -0.39664172534490882, -0.58057988919930292, -0.73484552606334397, -2.2077369806156142, 0.96997716030474534, -0.46943193529582694, -0.26259745592348271, 0.27789010673078729, -1.342420916163267, 0.98469900043043146, -0.56379596784325714, 1.9393591399172387, 0.23686828596097642, -1.8048305164121174, -1.060887298628888, -0.8854434963319886, -0.71502336621284601, 1.1745965021965097, -0.7593231800554241, 0.15157497021340052, -1.7994848765622229, -1.4978986816086861, -0.43991481680235978, 2.1984438762901641, -0.91173908940999482, -2.848138080480191, -2.2032261515726219, -0.73460003022899689, 1.3295753964198274, -0.19003075923738999, 1.7783554084488278, -0.32974043710705531, -1.7411061197911426, 0.5441575106054124, -0.29997726091536664, -0.6828639604841048, -0.026707172872064283, 2.0899131228241998, 1.6086683491545695, -0.65444983132744039, -2.1058092756269482, -0.24574567110015877, 1.3401269793479549, -1.0212417512540644, -1.0307616077500024, -3.4947116283964821, -4.1021882074538603, 0.8780085670246438, -1.0834736756175491, -0.40736333338935082, -0.82436536408681516, -1.0173627166580217, 0.54311761741492237, 0.91577125417705707, 1.4807454255270147, 0.76170671367214449, -0.066775806384470837, -0.25694309016301425, -0.65741384672361902, -0.044659048375465527, -1.7146185932856413, -1.7146934982882143, -1.1112983557322402, -0.22121658663813484, 0.12305244061641482, -1.0642172406319079, -0.59661203807351404, -1.6730841887947898, -3.3703777501975516, -3.6787705640029547, 0.27409026761017458, -0.45349366604458097, -4.0061595146033149, -1.7219571387910912, -0.42574597611630083, -0.68882088217789539, -0.62445031702974085, -1.2805844593106079, -1.4789241660519887, 0.051414419099676922, 0.97356244488782484, -0.61433830295570835, -3.1125224626052148, 1.6683778701132637, -1.0316769992801587, -1.3879436487580579, 0.15362662133918836, 0.60977416219752267, -1.5555197452082976, 0.60737290797586441, -0.35370567657591923, 1.9503646716123078, 0.35915246572258175, -1.1419492812262007, 0.99125444486958614, 1.3103109199412164, -0.414537240453001, -1.0913824771700371, 1.3633478158450629, -1.29211827494135, -0.39107205665880596, 0.59799504042266305, -0.53377147994111973, 0.2206177480625483, -0.62786775549714069, -2.3647603013852523, -1.903299400307479, -0.7775570514978043, -2.6302983291324695, 1.8047302853804852, -2.0931147337792888, -1.5812936080023849, -0.78678445226414828, -2.3900950121139988, -1.4261483682379608, -0.49360004800233204, 1.1630662202221751, 0.8731485177886168, -0.73397069004411397, 0.58365999599772167, 0.060754624741651983, 0.33849233011786717, 0.50439989378713945, 0.62963168737563224, -1.1064721737333045, 0.41156395615305852, 0.76661909386716687, -0.27352491245682942, -2.1414963440979653, -0.24302054554797725, -0.1434658548875494, 1.9189532106984726, 1.8043431485259358, -0.97362679043109435, -1.9402505487585211, -0.2619249264348209, -0.10525149418993862, -1.1512569037399421, -0.5729709063072026, -2.6081319986214346, -1.1693384086052312, -2.9091101594590869, -0.32298867915761387, -0.83210020810482388, -1.6109023952846595, -3.2890551867144193, 0.44807831157251654, 1.0926724876393727, 1.0699182565662635, -0.40704059691999439, 0.36642923035950103, 0.16426326990795603, -1.6204252710001552, -2.3196894210694956, -0.88657951137326885, 0.69826304115864068, -0.05446108158017253, -0.39509035257579361, -1.7471902039961926, -1.3854215550743882, -0.22297829432253957, -2.3899772251543308, -0.89153257934135066, 0.026155703152611627, 0.76433849213428084, -0.90505918653370154, 0.43112710566929546, 0.6322001682423275, -0.90002854873144367, 1.462511689264087, 0.4978792257145318, -0.51062466903488746, 0.60910465287008886, -0.72883190680520782, 0.2964202700200993, -1.3538800559563788, -2.532025187147946, -0.21865981208999857, 0.10512088925548413, -0.6399651155111481, -1.324782817412971, -0.16071423170943361, 0.84626039384685126, 3.3523700205766693, -1.0242260861437991, 0.26225584702745625, -0.50949456254643222, 0.45044881901211742, -0.54713337662007722, 0.083354478814245625, -1.845399997918646, -1.0442319357883609, -0.1981703994567357, -0.51153200569251289, -0.13731690818910894, -1.3005216907530324, 2.6550822124433906, 0.59515915089788418, -1.2261852732252698, -0.30428686891495871, -0.8538405194696288, -1.9947973581222718, -1.295873270189585, 1.0162629458304835, -1.1320986362659575, -2.7525036335341904, 0.68124792038692183, -1.3886513316413704, -2.3549998469208386, -0.9337429998689657, -2.9621555721365533, -3.5253485125268758, -0.68471726640439279, -0.56346073365987837, 1.7856707303139445, -1.2056755076003682, 1.1453170280138998, -0.22630731488626599, -1.3630577694925972, -1.0468882315503381, -3.373738981290006, 0.033149422564667047, 0.14055302855727758, -1.293174225246174, -2.1259474986635309, -0.84720004092407986, 1.4006359679555778, 0.85177497892871523, -0.094622907320341884, -1.8312873846460573, 0.75782583086479693, -0.024291767974691181, -1.1778361111656839, 0.35645964890376236, -1.6663631648665569, 0.2504854993489054, 0.74089162094731842, -1.1851712578604077, -1.2503782963820753, -0.96533076329311218, 0.57026787883431074, 1.5819434900297871, -1.2473460149052829, -0.012872617597713965, -0.095506697200545276, -1.0510351436767884, 1.2133325333883169, -1.9564817715120943, 1.1891859951500781, -0.41702726730959006, -0.73175996875980243, -1.2187091357014124, -1.3087912159644037, 0.49178370755703105, 1.147768429099707, 0.76262632968609889, 0.95123890374262854, -0.35862887342338651, -0.97295893556554058, -0.48964105553896153, -1.7008935265759408, -0.79064574167223345, 2.0303654560074316, 0.29570219261427966, -2.3848082082533946, -2.6125859098699862, -0.025686844118495186, 0.69071982136719312, -1.8725149505520142, -1.2131310219112434, 1.9667247190777626, 2.4920227482130248, -1.3022311334735757, -1.2531433429598033, -0.25894128852092629, 2.3419574884866798, -0.11994555865971107, -2.9306796758016045, -2.5637081209532937, -0.68661077179286045, 0.59777985517554644, -2.3055414204517795, -2.1014786300869064, -2.5241432333880804, -0.49874233895041542, -0.49457239079882631, 1.5382852267457685, -0.97622093038251734, -0.038762352850573167, 0.2101119376356187, -4.2257872501540508, -1.015805724444323, -1.4648637209139899, 0.2385203492308785, 1.4751055643297399, -0.14472044176953353, 0.23442980973914163, -2.1274818425180557, -0.19635517341087821, -0.22582386857075282, -3.1994769033468131, -1.4822218145237653, 0.93466926479780743, 1.8181600548983603, -1.7123754702694178, -0.11002183918498698, 1.0091659677223894, 1.1284933126128585, 0.30247541233107572, -1.0493088547823062, 0.44585252777715367, 2.0839257947240801, -1.5024366501758681, -1.1279425953069331, -0.57557892687786438, -1.8098817059267742, -0.29090857692587579, -0.63452872363605262, 1.2841736741993302, -2.606295144378743, -2.7014564620964716, 0.81317174426272554, -1.3373690516550609, 0.54621632771511219, -0.37487213752898541, -2.3520597001324361, -0.14396376625698892, -2.383730774997324, -1.6652104317730707, -0.72028898254772655, -0.81002068446507181, -2.7538217853746518, 0.21581350337294164, -0.25514580438418771, -0.34033634179751204, -1.2531107906720476, -1.7001217332271388, -2.9534578240818501, 0.45691435401882252, -0.24713401339130303, -1.7310026940348489, 0.84977178610782556, -2.0380380525598261, -1.6535919021837899, -1.2099133712261725, 2.7251965300220089, 1.0024607516342439, -0.4371974338923944, 0.28102452377390785, -0.44441921361697639, 0.92192274121841145, 0.34811410082613353, 0.43992321775213394, -0.41285590561510177, -2.1582669391309466, 0.92652173385983072, -1.1425757836729957, -2.2027664812872763, -0.94740456455995159, -3.9049226828800343, 1.9136736667447098, 0.63722824540807066, 0.15450745036990987, -0.99838381752611971, -0.89058214186138429, -2.2435872770783289, 0.1560544854666609, 0.54106349156776801, -1.1031656835656729, -0.49445521563751815, -0.72330311778851497, -1.9999945669291019, 0.043048191926377516, -3.3815558203507923, -0.03645204287052789, -1.6689636532823504, -3.2777819854324788, -1.8824718995882284, -0.041565271024066819, 0.38635814314118194, 0.50634304369323901, 0.15091158974337709, 0.055529924055514934, -2.6426414060805841, 0.32355787303293482, 2.4065949101924504, -0.74747392381319211, 0.18499345829654579, -0.92436755665232173, -0.43354317641914442, -0.65424645460262121, -2.5129703106059202, -1.423739145957553, -0.93046578233047927, 0.35791710843746471, -0.35200157999645398, -0.51170849466589186, -2.1701142232915496, -2.4304888965466036, -2.1915839805563175, 0.32375655044496532, 0.12499566504269281, 0.54795843067712324, -2.2633457324441775, -0.56576082437933506, -2.9507548535290216, -0.31653009956477285, 0.37108705717274171, -0.70249873735978463, 0.35316236772058107, -2.4328706716174309, -0.14916450808038903, -0.96693609611881071, 2.4828470516367811, 0.55497648893974572, -1.7743873662783947, -0.87460277795816044, -0.45136271038500964, 0.37308514578308094, -0.50337523871372458, 1.0370901007345845, -0.083268541979648669, 0.025096427234083368, -0.6308083708747998, 0.099122123261496164, 1.544509442670428, -1.6274650243268471, -0.047539724562593755, 1.7381097404692043, 0.67030568560333859, 1.2125709119543937, 0.079156499779653555, -1.0946363325826471, -0.092972490980515798, -2.0261424922526015, -0.39159498883320354, -0.80835707589731109, 0.50340158778923394, 0.038815260169698731, 0.39538303740548664, -1.7852213874821778, 0.18054643788775326, -0.61705497730413239, 0.57090874324469487, -1.6065093118183025, -2.4542669741884851, 0.34860020004187264, -0.71698542971245327, -0.12436339984352862, -0.7102531816284281, -0.48247581786053406, -0.57858635424238991, 0.39753226142546644, 1.4039392581622372, 1.236033046455739, -0.99586165270949711, 0.056385391304697108, -1.0004718571104942, -0.24161514062535025, -1.2546424453232086, -1.7725792249525996, -1.8908185339686616, 0.57179029932247349, -0.85364217331675718, -1.724599642380872, 1.4298425041131169, -1.0917298376225932, -1.1867548213362458, -0.59757956347883179, 0.42545416416564091, -1.2895827752894073, -4.3309859682111815, 0.68315389932119797, -1.1119960202780952, 0.55827022792791337, -0.78149093271305081, 0.5383831765624445, 0.14540760956390705, -2.0140176123828932, -0.64229436623029312, -0.41827100585852828, -1.551640601913669, 0.50213089578967773, 1.5802608708653838, -2.5736189205360729, 0.64289810558591987, -1.3335661497791418, -1.6315711388518668, -1.2224508806518219, -1.8558078704207706, 1.632336164535096, -2.6277413536311656, -0.42572711779363859, -1.4179086846688858, -0.86264124248050167, -2.0799301212823069, 0.2881134534963985, -0.86383466024449418, -0.70469516738120885, -2.3347543847671197, -1.4036606980648603, 1.4618754687182056, 0.47199651370170848, -2.1647325669520829, -0.72519678019684775, -1.0848075513212543, -0.80825593696653675, -0.9969599297107008, 0.84996573573226686, -0.71674011059476694, 0.95512061927835745, -0.98899988059949728, -0.38613654742907272, 0.94960114321694311, 0.88949357668618578, -1.8841768852624039, 0.52692951274558264, 0.040685935331936068, -1.3136141813835138, 0.62161383530087133, -1.4670083777890108, -0.46314543242023731, -0.56906463747667402, -0.70583537311294853, -0.60881690309684489, 0.35924085143532275, -1.2904963838988248, -2.0037403280215673, -0.46015639046922657, 2.5627688758385689, -0.62407031368617627, -0.090282150354884599, -0.084798171053233284, 0.45377489785573344, -0.52762061446790387, 0.81397730319718542, -1.3455656489957701, -1.1746817819371476, -2.017228251540554, 0.56390792890103159, -0.62112938985659572, -0.90177013844494591, -0.52131387350604719, 0.76164112144035712, -4.2961609704587858, 1.0228585764877416, 0.325752193539529, -3.7267588248391781, 0.87914447295667408, 0.73469288868435534, 0.87965633081025207, -1.3769162549096345, 2.1781541439209473, 0.8783112979395109, -0.31482713189906186, -1.3391602002662111, -0.63049671140002306, 1.1924675943336318, 2.285347196536196, -0.077301620025464202, -2.5458787864201926, -0.37270543980695742, -0.84763077758706418, 0.95125898645079898, -1.1496089519585131, 0.25748468059657492, 0.59896113227941372, -0.75310323086081388, -0.11754757803678917, 0.26829666975594058, -1.9408720493727918, -0.29646824653276932, -3.3617385362670436, 1.1200657760603392, 0.90735359013199401, 1.4441741969831825, -1.165338107092639, -0.84983868889251579, 0.15768847175839104, 0.19281843233885521, -1.0677631201278013, -0.3648691642414762, -0.23130258102666823, 0.52618091435189251, -2.8573518530459343, 0.24889764152723337, 0.3980339998278819, -2.6388143373476693, -0.52449764733860538, -0.44962297908712567, -1.631171441243946, -1.2365825657058369, 0.32706145011201415, 1.7952817228974078, -1.4748671505858382, -2.9755857304422495, -2.0732762701990484, -1.0047208603508406, -0.24386891573320035, 1.7356761318689191, -1.519424668028388, 0.56208253391933027, -1.2328461490463729, -1.8245557478730907, 0.51988342576307933, 1.0767195346358371, -0.25456620479131098, -2.7447816568735877, 0.18085870127681106, -0.1342022809527611, -0.47064280244596024, 0.70717566939771392, -1.6910271745052059, -1.3310566241102004, -0.6698928290653714, 2.9682533241839528, 0.15519226440924927, -1.004710076889693, -1.1518898955827315, -0.84137449000607223, 0.42932433874035636, -1.696339936955706, -1.0910320980467292, 1.2651344490634699, -0.61524327646153665, -1.8143196724070214, 0.21269444727353976, 0.67422163183140427, -2.5404948509817258, -1.7749043588153892, 0.96170390876844247, -2.2199311210414194, -1.9953158880795914, 0.77207146548922467, -0.82626882971692495, -0.32563205957542302, 2.457852820006337, 1.7214122629088036, -0.70870328981743314, -1.8840561487441956, 0.47004975554704109, -1.5772670784086358, -0.47958494753723357, -1.3229959025562412, -0.74345804330807019, -0.96397395849498002, 1.013832810608609, 1.4517865593975272, -0.7148399072142686, -0.70558983807765285, 1.1200124505754887, 0.1119182278027967, 1.4589211335986358, 1.5486259937680988, 1.327442315105382, 0.30202051044400446, 1.4655541686117732, -1.4948625205273836, -0.030174229072770099, -1.9076075417800471, 0.62956837149239453, -0.28033001655770184, -1.1557973520582778, 1.1179853881503867, -1.4578203798239568, -0.42859019158046702, -1.9594130653145059, -0.28571303886228644, -2.7311606673119719, -3.9315179249469949, 2.3239279256504703, -0.68659136080064298, 0.4145768519508366, 1.0180097487898365, 1.5376371691197772, 2.1249593960486903, 0.88469948281865607, -0.91332272797377001, -1.3977376710656091, 0.24422992093627696, -1.5644649834001156, -1.1985438994431785, -1.3721190661377842, 1.2375522510627417, -1.6613550350518329, 0.15084451633030105, -0.6953392765262052, -0.48278071116605153, -1.6901054486017366, -0.28314485575190679, -1.3493749789571345, -0.85380873908747734, -1.7291218678233522, -1.3524732266511972, -0.54308844811018675, 0.8627068280324004, -1.2428224608576186, -0.39729957279505812, -0.43554548637872692, -2.0753783080578136, -0.92165409499792317, 0.32740794719813787, -0.17067662172400705, -2.1388259755505361, 1.4462707318178791, -1.0708944398009723, -1.3074472860866759, 0.54130148243411569, 0.66219758548801977, -0.10347299175944441, 0.6270046075852922, -0.9277291564675112, 1.0343581770312562, -0.82393874842276338, 0.082864863714089909, -0.050406812551474015, -0.56760742009976572, -1.4311265514124278, 0.79922888825623173, 4.4631582786751212, -0.26513005689914132, 0.88651433509948419, -0.13723150739494866, -1.0027655527874357, 0.72143212663445955, 0.88419741650110151, -0.34987751030276626, 0.91564311190706626, -1.594019167612629, 0.45016643940429091, 0.7260003169403173, -1.13002216811753, -2.7233237453396417, 1.9210199801081589, 0.96304762909959385, -2.0289275383934942, -0.09284411123554337, 0.93342048654799414, 0.42848400529057074, 0.16308862350447995, -0.45025221671828619, -0.17816686968899675, 0.38701362314822868, -0.92650059917361083, 0.34318741835809385, -0.11322278376272188, -0.33203803620799655, -0.20740764833864478, -0.60692410125085505, -2.1163140956600652, 1.4192122691402216, 2.58276739366773, -0.67686955141486405, -0.20685156457029452, -0.08911556143209988, -2.6604732727585674, -0.54614555399444742, 1.7468241716876149, 0.61351552845945156, -1.0018125288253623, -0.4422563560024646, -0.36893615558801562, -0.67256100973548771, -0.34459607261194342, -1.5652515608250193, -1.0050262036803124, -0.17776378138373738, -0.54828488362800876, -0.77719731060270369, -2.1959701937240075, 0.65260110195324494, -0.019765530934891418, -0.32782586000183156, 1.2941037071808759, 0.2396040104554992, -1.3156786427889524, -2.3276677566413513, -0.89429835940264713, 0.37442896657373154, -1.8812782499567415, -1.6574480193813712, -0.41251711966636379, 0.53116083379256285, -1.2937910224411024, 1.2251282379543693, 0.48721880882190571, 0.78816648505592513, -2.0898522346911719, -2.6191073174894615, -0.30641770433671867, -2.3426774576002831, 0.70803893301561205, -2.1179941461394423, -0.050319009128410475, -1.2117613801059228, 0.51402380328556974, 0.0087852556157399508, -2.0540388500692393, -1.0939993848026921, 0.78378996353478758, 1.5248278518901575, -0.030432529697219796, 0.89693197345490583, 0.064881220093678499, 0.63573976319015313, 2.9749620882704839, -1.1595714597544187, -0.49555942605912845, -1.2553362632515093, 0.2290989106903174, -1.8624441806131211, -0.34338776921121983, -0.45348301701575777, -2.2288225689197017, 0.56588043203417948, -1.5524852104386242, 0.70918377780251407, 0.84638477617107877, -0.40449922167842972, -1.6789091489810377, 1.8793032470297319, -0.89275948163128516, -2.0705785608324097, -3.1850080474501432, -1.1470120579888761, -0.20562167547828286, -2.2795379186013673, -0.39973593327096202, -2.2258232740153905, -1.708711380139476, -1.4905010022583844, 0.25632064734089655, 1.2707568425591422, -1.9059099894446625, 2.014203000466297, -0.70734447288210012, 0.9892699787213306, -1.6749531988953192, -0.029308499610945379, -0.93632683759137048, 1.7750679565011858, 0.64844028883786353, -0.62050718650411019, -0.95580640576904519, -0.30058176830872663, -3.9998994011638231, -2.0106856721207835, -0.53742281245543233, -0.55195460362401061, -1.4607244531016048, -0.41186720882993622, 0.046532707516563354, -0.18286168300267547, -2.1426420955055514, 0.91543553391480337, 0.71446782599192549, -0.89904421692314207, -2.1628654276936139, -0.081667937607633329, -3.6538578938718862, -1.2083415052892224, -0.6345987887339275, 0.14215148486614393, -0.94443159109761121, 0.80155403490835875, -0.33471444020221858, -1.4340080956419623, 0.42967873908949161, -1.0415868836923203, 1.0836416911352138, -1.6942371431275882, 0.24443953650157668, -0.12420295427660422, -2.661416617892391, -0.81376343336875034, -0.47612474493198281, 0.79013502687846759, -0.6769027676960726, -1.7505870444505425, 1.0169084161849056, 0.72566804411293884, -0.39416293695082821, 1.1691789880817691, -0.20013200890130145, 3.6454671667048402, 0.017659795554412994, -1.4267490766849453, -0.85245193267999708, 1.7167555231162801, 0.051317086105756041, -1.4833217931148308, 0.19988067026434841, -1.2163233415361687, -2.2742967581927367, 0.49647288413880841, 1.4273671125800913, 2.3341504875993335, -0.94532166320835365, -1.3745896915580045, -2.1932611928782464, -1.202317889527647, -0.78958216333095754, 0.30033250924907284, -2.1257424055860605, 1.1296897859277983, -1.0174129245902774, -3.0441680442286234, -1.9004964246871474, -2.616452168693975, -0.14098958399042985, -1.6641249087890593, -0.29059816350495415, -0.14497740109214441, -0.38520602682973759, -2.9182339143828497, 1.6202232554312164, 0.68847959233575828, -1.5266788227575907, -1.3843842348041155, 0.36102032558275443, 0.59337598422293936, -1.7775705626541685, -0.06017744369810174, 1.4144451418741035, 0.83472212316364058, 0.5798860971148192, 1.5410301392473513, 1.5603467182534336, 1.3020830015452043, -0.63119813613735798, -0.46513735688600483, -0.82148606664841195, -2.3986438110499866, -1.1188236640354379, -0.32804592050754583, -1.3825382285457346, -2.2637399827371558, -0.73976501538057826, 0.085349957481234306, -0.99726676203708042, 0.075625562677414271, 0.96064107500583473, 0.25711406003810294, -0.19577421876686524, 0.22154325350552284, -1.9152393484799379, -0.79636824368997416, -0.85966348136185777, -0.40537114860560264, -1.0305247589203872, 1.2371059307781942, -0.86962220687239811, 0.13718027477959033, -1.1254316224628296, -1.5675254982001001, 0.072564872409489833, -0.18082914085633933, 0.5768309353602098, -1.857136444155822, -0.26376625455415664, -2.2967462994250578, -1.0936319454632821, 2.5782661131354407, -0.89619054008569665, -2.0412836770152509, -0.025738269851960927, -0.55308971510089078, -0.93899202250190306, -1.5982454711697085, 0.6213182507987135, -2.3534052827186978, -3.173452029064987, 0.7914344227580643, -2.4832178277508943, -1.0195772861568952, -0.76635878093032328, 3.1442946374425556, -0.5100147353493667, -0.19950921081674425, -1.1893790136759494, -0.11468542530787801, -2.9529075756304284, -0.29531387078519439, -0.73989608727934131, -0.68058549163336912], + "x_ties": [-0.10000000000000001, -0.75, 0.01, -0.57999999999999996, -0.62, -0.57999999999999996, 0.19, 0.02, -0.34000000000000002, 0.66000000000000003, -0.54000000000000004, -0.34999999999999998, -0.56999999999999995, -0.56999999999999995, -0.91000000000000003, -0.13, -0.12, -0.42999999999999999, -0.39000000000000001, -0.19, -0.23000000000000001, -0.70999999999999996, -0.10000000000000001, -0.38, 0.050000000000000003, -0.54000000000000004, -0.28000000000000003, -0.63, -0.31, 0.17000000000000001, -0.58999999999999997, -0.46000000000000002, -0.53000000000000003, -0.02, -0.40000000000000002, 0.26000000000000001, -0.029999999999999999, -0.31, -0.83999999999999997, 0.02, -0.44, -0.82999999999999996, -0.5, -0.040000000000000001, -0.52000000000000002, -0.81000000000000005, -0.11, 0.12, -0.38, -0.85999999999999999, -0.80000000000000004, -0.68999999999999995, 0.47999999999999998, 0.23000000000000001, -0.47999999999999998, -0.98999999999999999, -0.68999999999999995, -0.53000000000000003, 0.029999999999999999, -0.35999999999999999, -0.12, -0.26000000000000001, -0.41999999999999998, 0.5, -0.53000000000000003, -0.38, -0.11, -0.96999999999999997, -0.83999999999999997, -0.81000000000000005, -0.46000000000000002, -0.28999999999999998, -0.60999999999999999, -0.97999999999999998, -0.41999999999999998, -0.44, 0.31, -0.40999999999999998, -0.58999999999999997, -0.13, 0.23999999999999999, -0.31, -0.79000000000000004, -0.28999999999999998, -0.88, -0.81999999999999995, -0.48999999999999999, -0.78000000000000003, 0.12, 0.12, -0.64000000000000001, -0.52000000000000002, -0.72999999999999998, -0.63, -0.55000000000000004, -0.19, -0.85999999999999999, -0.52000000000000002, -0.52000000000000002, -0.69999999999999996, -0.81000000000000005, 0.080000000000000002, -0.54000000000000004, -0.46000000000000002, -0.20999999999999999, -0.83999999999999997, -0.22, -0.69999999999999996, -0.46999999999999997, -0.56000000000000005, -0.37, 0.22, 0.22, -0.22, -0.33000000000000002, 0.13, -0.37, -0.67000000000000004, 0.22, -0.60999999999999999, -0.22, -0.44, -0.81000000000000005, -0.63, -0.050000000000000003, -0.87, 0.02, -0.64000000000000001, -0.55000000000000004, -0.84999999999999998, -0.41999999999999998, -0.26000000000000001, 0.38, -0.75, -0.29999999999999999, -0.64000000000000001, -0.48999999999999999, -0.39000000000000001, -0, -0.54000000000000004, -0.42999999999999999, -0.17000000000000001, 0.39000000000000001, -0.82999999999999996, -0.22, -0.55000000000000004, 0.01, -0.54000000000000004, -0.77000000000000002, -0.73999999999999999, -0.35999999999999999, -0.20000000000000001, -0.25, 0.23000000000000001, -0.56000000000000005, 0.040000000000000001, -0.070000000000000007, -0.53000000000000003, -0.59999999999999998, -0.73999999999999999, -0.23000000000000001, -0.28000000000000003, -0.71999999999999997, -0.39000000000000001, 0.11, -0.77000000000000002, -0.37, -0.71999999999999997, -0.050000000000000003, -0.28999999999999998, 0.28999999999999998, -0.52000000000000002, -0.11, 0.23999999999999999, 0.28000000000000003, -0.53000000000000003, -0.59999999999999998, -0.64000000000000001, 0.070000000000000007, 0.66000000000000003, 0.059999999999999998, -0.60999999999999999, -0.46999999999999997, 0.040000000000000001, -0.35999999999999999, 0.20000000000000001, 0.02, -0.40000000000000002, 0.17999999999999999, -0.080000000000000002, -0.10000000000000001, -0.20999999999999999, -0.58999999999999997, -0.88, 0.22, -0.41999999999999998, -0.28999999999999998, -0.60999999999999999, -0.96999999999999997, 0.16, -0.66000000000000003, -0.44, -0.31, 0.34999999999999998, -0.40999999999999998, -0.23999999999999999, 0.050000000000000003, -0.70999999999999996, -0.56999999999999995, -0.38, -0.67000000000000004, -0.40999999999999998, 0.13, -0.34999999999999998, -0.75, -0.32000000000000001, 0, -0.71999999999999997, -0.67000000000000004, -0.46000000000000002, -0.17000000000000001, 0.19, -0.11, -0.22, -0.37, -0.92000000000000004, -0.89000000000000001, -0.050000000000000003, -0.040000000000000001, -0.65000000000000002, -0.33000000000000002, -0.46999999999999997, -0.14000000000000001, -0.27000000000000002, -0.02, -0.52000000000000002, -0.14000000000000001, -0.11, -0.62, -0.45000000000000001, -0.40999999999999998, -0.46000000000000002, -0.63, -0.40999999999999998, -0.72999999999999998, 0.12, -0.13, -0.40000000000000002, -0.82999999999999996, -0.40999999999999998, -0.56000000000000005, -0.31, -0.93999999999999995, 0.050000000000000003, -0.91000000000000003, -0.63, -0.19, 0.32000000000000001, -0.23000000000000001, -0.23999999999999999, -0.52000000000000002, -0.97999999999999998, 0.34999999999999998, -0.53000000000000003, -0.51000000000000001, 0.25, -0.59999999999999998, -0.22, -0.81999999999999995, -0.68000000000000005, -0.39000000000000001, 0.14000000000000001, -0.16, -0.71999999999999997, -0.089999999999999997, -0.10000000000000001, -0.44, -0.32000000000000001, -0.33000000000000002, -0.81999999999999995, -0.88, 0.23999999999999999, -0.63, 0.02, -0.45000000000000001, -0.78000000000000003, -0.46999999999999997, -0.41999999999999998, -0.56000000000000005, 0.46999999999999997, 0.20000000000000001, -0.12, 0.46000000000000002, -0.20000000000000001, 0.14000000000000001, -0.68999999999999995, -0.53000000000000003, -0.85999999999999999, -0.089999999999999997, -0.68999999999999995, -0.68000000000000005, 0.32000000000000001, -0.31, -0.5, -0.80000000000000004, -0.48999999999999999, -0.059999999999999998, -0.040000000000000001, -0.51000000000000001, -0.11, 0.01, -0.44, -0.65000000000000002, -0.37, -0.76000000000000001, -0.25, -0.34999999999999998, -0.46000000000000002, -0.20999999999999999, -0.63, -0.40999999999999998, -0.46999999999999997, 0.22, -0.38, -0.040000000000000001, -0.48999999999999999, -0.90000000000000002, -0.84999999999999998, -0.53000000000000003, 0.040000000000000001, -0.040000000000000001, -0.51000000000000001, 0.40000000000000002, -0.27000000000000002, -0.33000000000000002, -0.28999999999999998, -0.68000000000000005, -0.33000000000000002, -0.57999999999999996, -0.68999999999999995, -0.28999999999999998, 0.28999999999999998, -0.48999999999999999, -0.71999999999999997, -0.56000000000000005, -0.20000000000000001, -0.22, 0.69999999999999996, -0.64000000000000001, -0.31, -0.37, -0.31, -0.70999999999999996, -0.48999999999999999, 0.40000000000000002, -0.029999999999999999, -0.35999999999999999, -0.22, -0.62, -0.79000000000000004, -0.37, -0.34000000000000002, -0.71999999999999997, 0.52000000000000002, -0.31, 0.33000000000000002, -0.57999999999999996, -0.85999999999999999, -0.089999999999999997, -0.42999999999999999, 0.29999999999999999, -0.13, -0.29999999999999999, 0.050000000000000003, -0.38, -0.56000000000000005, -0.34999999999999998, 0.52000000000000002, 0.10000000000000001, -0.56000000000000005, -0.93999999999999995, -0.01, -0.56999999999999995, -0.53000000000000003, -0.40000000000000002, -0.75, -0.17000000000000001, -0.34999999999999998, -0.029999999999999999, -0.01, -0.45000000000000001, -0.34999999999999998, -0.67000000000000004, 0.23000000000000001, -0.72999999999999998, -0.45000000000000001, -0.40999999999999998, -0.080000000000000002, -0.52000000000000002, -0.19, -0.059999999999999998, -0.14999999999999999, -0.32000000000000001, -0.46000000000000002, -0.84999999999999998, -0.12, -0.41999999999999998, -0.34999999999999998, 0.40000000000000002, -0.96999999999999997, -0.59999999999999998, -0.17999999999999999, -0.26000000000000001, -0.10000000000000001, -0.23999999999999999, -0.34999999999999998, -0.55000000000000004, -0.69999999999999996, 0.80000000000000004, -0.13, -0.5, -0.32000000000000001, -0.83999999999999997, -0.62, 0.39000000000000001, -0.53000000000000003, -0.34999999999999998, -0.38, 0.070000000000000007, 0.02, -0.14000000000000001, -0.38, 0.029999999999999999, -0.28000000000000003, -0.81999999999999995, 0.35999999999999999, -0.65000000000000002, -0.10000000000000001, -0.97999999999999998, 0.46000000000000002, -0.040000000000000001, 0.089999999999999997, -0.33000000000000002, -0.58999999999999997, -0.96999999999999997, -0.34999999999999998, -0.58999999999999997, -0.57999999999999996, -0.71999999999999997, -0.26000000000000001, -0.16, -0.23000000000000001, -0.54000000000000004, -0.56999999999999995, -0.41999999999999998, 0.11, 0.029999999999999999, -0.12, -0.42999999999999999, -0.82999999999999996, 0, -0.23000000000000001, -0.27000000000000002, -0.41999999999999998, -0.63, -0.38, -0.46999999999999997, 0.089999999999999997, -0.40999999999999998, 0.29999999999999999, 0.070000000000000007, -0.26000000000000001, -0.34999999999999998, -0.67000000000000004, 0.14000000000000001, -0.070000000000000007, -0.56000000000000005, -0.11, -0.62, -0.67000000000000004, -0.68999999999999995, 0.080000000000000002, -0.35999999999999999, -0.14999999999999999, -0.46000000000000002, -0.33000000000000002, -0.40000000000000002, -0.29999999999999999, -0.45000000000000001, -0.23999999999999999, -0.20000000000000001, -0.28999999999999998, -0.33000000000000002, -0.19, -0.32000000000000001, -0.71999999999999997, -0.66000000000000003, -0.54000000000000004, -0.02, -0.83999999999999997, 0.31, 0.02, -0.33000000000000002, -0.40999999999999998, -0.33000000000000002, -0.12, -0.34000000000000002, -0.93000000000000005, -0.059999999999999998, 0.17000000000000001, -0.26000000000000001, -0.31, -0.16, -0.70999999999999996, 0.070000000000000007, -0.20000000000000001, -0.60999999999999999, 0.38, -0.26000000000000001, 0.059999999999999998, -0.14999999999999999, 0.040000000000000001, 0.37, 0.51000000000000001, 0.16, -0.32000000000000001, -0.56999999999999995, -0.80000000000000004, -0.80000000000000004, -0.95999999999999996, -0.78000000000000003, 0.20999999999999999, 0.46999999999999997, -0.84999999999999998, -0.71999999999999997, -0.67000000000000004, 0.69999999999999996, -0.79000000000000004, -0.20000000000000001, -0.11, -0.69999999999999996, -0.55000000000000004, -0.48999999999999999, -0.51000000000000001, -0.75, -0.46000000000000002, -0.28000000000000003, 0.40999999999999998, -0.10000000000000001, 0.40999999999999998, -0.11, -0.80000000000000004, -0.38, -0.39000000000000001, -0.66000000000000003, -0.64000000000000001, 0.070000000000000007, -0.89000000000000001, -0.40999999999999998, -0.72999999999999998, 0, 0.47999999999999998, -0.17000000000000001, -0.20999999999999999, -0.29999999999999999, -0.34999999999999998, -0.66000000000000003, -0.059999999999999998, -0.46000000000000002, -0.65000000000000002, -0.14000000000000001, -0.45000000000000001, -0.31, -0.90000000000000002, -0.93999999999999995, 0.01, 0.070000000000000007, -0.55000000000000004, -0.88, -0.63, -0.089999999999999997, -0.56000000000000005, -0.60999999999999999, -0.93000000000000005, -0.20000000000000001, -0.79000000000000004, -0.44, 0.33000000000000002, -0.67000000000000004, -0.46999999999999997, -0.059999999999999998, -0.47999999999999998, -0.92000000000000004, -0, 0.13, 0.23999999999999999, -0.25, 0.42999999999999999, -0.75, -0.55000000000000004, -0.23999999999999999, -0.28999999999999998, -0.59999999999999998, 0.52000000000000002, -0.57999999999999996, -0, -0.34999999999999998, -0.48999999999999999, -0.20000000000000001, -0.41999999999999998, -0.34000000000000002, -0.76000000000000001, 0.070000000000000007, -0.12, -0.81000000000000005, 0.42999999999999999, 0.17999999999999999, -0.32000000000000001, -0.55000000000000004, -0.17999999999999999, -0.02, -0.48999999999999999, -0.79000000000000004, 0.10000000000000001, -0.10000000000000001, -0.40999999999999998, -0.70999999999999996, -0.11, -0.41999999999999998, -0.22, -0.070000000000000007, -0.46999999999999997, -0.51000000000000001, 0.059999999999999998, -0.78000000000000003, -0.40999999999999998, -0.02, -0.90000000000000002, 0.080000000000000002, -0.84999999999999998, -0.59999999999999998, -0.80000000000000004, -0.52000000000000002, -0.41999999999999998, -0.5, -0.20999999999999999, -0.33000000000000002, -0.73999999999999999, 0.02, -0.17000000000000001, 0.48999999999999999, -0.050000000000000003, -0.47999999999999998, -0.27000000000000002, -0.35999999999999999, -0.56000000000000005, -0.78000000000000003, -0.34000000000000002, -0.69999999999999996, -0.91000000000000003, -0.68999999999999995, -0.78000000000000003, -0.17000000000000001, -0.35999999999999999, -0.52000000000000002, -0.40000000000000002, -0.5, -0.32000000000000001, -0.11, -0.48999999999999999, -0.52000000000000002, -0.089999999999999997, -0.51000000000000001, -0.14000000000000001, -0.60999999999999999, -0.41999999999999998, -0.14999999999999999, -0.70999999999999996, -0.75, -0.75, -0.35999999999999999, -0.67000000000000004, -0.42999999999999999, -0.17999999999999999, -0.35999999999999999, -0.77000000000000002, -0.44, -0.60999999999999999, 0.089999999999999997, -0.23999999999999999, 0.12, -0.48999999999999999, -0.5, 0.040000000000000001, -0.52000000000000002, -0.81999999999999995, -0.5, -0.90000000000000002, -0.10000000000000001, -0.31, 0.029999999999999999, 0.76000000000000001, -0.13, -0.01, -0.85999999999999999, -0.070000000000000007, -0.64000000000000001, 0.050000000000000003, -0.38, -0.40000000000000002, -0.41999999999999998, -0.39000000000000001, 0.050000000000000003, -0.33000000000000002, -0.95999999999999996, -0.42999999999999999, -0.82999999999999996, 0.54000000000000004, -0.44, -0.070000000000000007, -0.12, -0.059999999999999998, 0.22, -0.81000000000000005, 0.02, -0.28999999999999998, -0.58999999999999997, -0.13, -0.42999999999999999, -0.040000000000000001, -0.71999999999999997, -0.47999999999999998, -0.33000000000000002, -0.68000000000000005, -0.22, 0.47999999999999998, -0.13, -0.28999999999999998, 0.029999999999999999, -0.67000000000000004, 0.40000000000000002, -0.52000000000000002, -0.20999999999999999, -0.39000000000000001, 0.34999999999999998, 0.02, -0.31, -0.55000000000000004, -0.78000000000000003, -0.82999999999999996, -0.10000000000000001, -0.28999999999999998, -0.55000000000000004, 0.22, -0.28000000000000003, -0.55000000000000004, -0.40999999999999998, -0.83999999999999997, -0.68999999999999995, -0.14999999999999999, -0.42999999999999999, -0.52000000000000002, -0.62, -0.23000000000000001, 0.02, -0.28999999999999998, -0.27000000000000002, -0.22, -0.68000000000000005, -0.45000000000000001, -0.67000000000000004, -0.47999999999999998, -0.42999999999999999, 0.59999999999999998, -0.59999999999999998, -0.20000000000000001, -0.14999999999999999, -0.89000000000000001, -0.51000000000000001, -0.89000000000000001, -0.56999999999999995, -0.27000000000000002, -0.38, -0.20000000000000001, -0.35999999999999999, -0.73999999999999999, -0.71999999999999997, -0.46000000000000002, 0.12, 0.32000000000000001, 0.089999999999999997, -0.79000000000000004, -0.20000000000000001, -0.5, -0.46999999999999997, -0.14999999999999999, -0.73999999999999999, -0.63, -0.69999999999999996, -0.34000000000000002, 0.089999999999999997, -0.81999999999999995, -0.82999999999999996, -0.51000000000000001, -0.5, -0.94999999999999996, 0.35999999999999999, -0.14999999999999999, -0.31, -0.58999999999999997, -0.63, -0.88, -0.080000000000000002, 0.01, 0.059999999999999998, -0.28999999999999998, -0.88, -0.050000000000000003, -0.10000000000000001, -0.44, 0.28000000000000003, -0.070000000000000007, -0.68999999999999995, -0.089999999999999997, -0.66000000000000003, 0.34999999999999998, -0.22, -0.68999999999999995, -0.72999999999999998, -0.60999999999999999, -0, -0.68999999999999995, -0.23000000000000001, -0.20000000000000001, -0.29999999999999999, -0.22, 0.27000000000000002, -0.76000000000000001, 0.059999999999999998, -0.39000000000000001, -0.45000000000000001, -0.13, -0.60999999999999999, -0.71999999999999997, -0.79000000000000004, -0.58999999999999997, -0.57999999999999996, -0.79000000000000004, -0.46000000000000002, -0, 0.32000000000000001, -0.45000000000000001, -0.14999999999999999, -0.58999999999999997, -0.28000000000000003, -0.42999999999999999, -0.76000000000000001, -0.050000000000000003, -0.23000000000000001, -0.62, -0.71999999999999997, -0.17999999999999999, 0.040000000000000001, 0.20000000000000001, 0.38, -0.70999999999999996, -0.48999999999999999, 0.28000000000000003, -0.62, 0.059999999999999998, -0.91000000000000003, 0.26000000000000001, 0.11, 0.11, -0.5, -0.60999999999999999, -0.10000000000000001, -0.080000000000000002, -0.68999999999999995, -0.41999999999999998, -0.83999999999999997, -0.70999999999999996, -0.39000000000000001, -0.28000000000000003, -0.080000000000000002, -0.10000000000000001, -0.040000000000000001, -0.37, 0.12, -0.20999999999999999, -0.35999999999999999, 0.23000000000000001, -0.42999999999999999, -0.48999999999999999, -0.87, -0.68999999999999995, -0.44, -0.53000000000000003, -0.26000000000000001, 0.02, -0.089999999999999997, -0.34000000000000002, -0.40000000000000002, -0.57999999999999996, -0.080000000000000002, -0.68000000000000005, -0, -0.89000000000000001, -0.55000000000000004, -0.080000000000000002, -0.39000000000000001, -0.44, 0.25, 0.53000000000000003, -0.45000000000000001, -0.45000000000000001, 0.51000000000000001, -0.17999999999999999, -0.67000000000000004, 0.080000000000000002, -0.98999999999999999, -0.77000000000000002, -0.23000000000000001, -0.45000000000000001, -0.11, -0.040000000000000001, -0.56000000000000005, 0.22, 0.029999999999999999, -0.69999999999999996, -0.38, 0.19, 0.16, -0.85999999999999999, -0.32000000000000001, -0.66000000000000003, 0.31, -0.44, -0.23999999999999999, -0.68999999999999995, 0.089999999999999997, -0.080000000000000002, -0.72999999999999998, -0.02, -0.66000000000000003, -0.059999999999999998, -0.75, -0.56000000000000005, 0.46000000000000002, -0.55000000000000004, -0.70999999999999996, -0.46999999999999997, -0.34000000000000002, -0.12, 0.46000000000000002, -0.37, -0.89000000000000001, -0.57999999999999996, -0.23000000000000001, -0.85999999999999999, -0.070000000000000007, -0.60999999999999999, -0.27000000000000002, -0.01, -0.96999999999999997, -0.47999999999999998, -0.40999999999999998, -0.029999999999999999, -0.19, -0.57999999999999996, -0.65000000000000002, 0.040000000000000001, 0.14999999999999999, 0.14999999999999999, -0.31, -0.62, 0.28000000000000003, -0.76000000000000001, -0.68000000000000005, -0.73999999999999999, -0.10000000000000001, -0.59999999999999998, -0.42999999999999999, 0.28999999999999998, -0.56000000000000005, -0.29999999999999999, 0.20000000000000001, -0.54000000000000004, -0.28000000000000003, 0.16, -0.14000000000000001, -0.55000000000000004, 0.48999999999999999, -0.58999999999999997, -0.14000000000000001, -0.53000000000000003, -0.17000000000000001, -0.67000000000000004, -0.22, -0.91000000000000003, -0.19, -0.34000000000000002, -0.42999999999999999, -0.51000000000000001, -0.27000000000000002, -0.22, -0.68999999999999995, -0.63, -0.72999999999999998, -0.059999999999999998, -0.44, -0.22, -0.63, -0.01, -0.68999999999999995, 0.31, -0.40000000000000002, -0.48999999999999999, 0.02, 0.029999999999999999, 0.02, -0.54000000000000004, 0.44, 0, -0.33000000000000002, 0.01, -0.38, -0.87, 0.40999999999999998, -0.050000000000000003, -0.46000000000000002, -0.41999999999999998, -0.67000000000000004, -0.29999999999999999, -0.66000000000000003, -0.59999999999999998, 0.34000000000000002, -0.02, -0.27000000000000002, 0.02, -0.53000000000000003, -0.76000000000000001, 0.26000000000000001, -0.28999999999999998, -0.44, -0.75, 0.17999999999999999, -0.050000000000000003, -0.45000000000000001, -0.20000000000000001, 0.20000000000000001, 0.059999999999999998, -0.11, -0.29999999999999999, -0.51000000000000001, -0.72999999999999998, -0.69999999999999996, -0.60999999999999999, -0.54000000000000004, -0.22, -0.31, 0, -0.34999999999999998, -0.56999999999999995, 0.31, -0.33000000000000002, -0.83999999999999997, -0.35999999999999999, -0.67000000000000004, -0.20000000000000001, -0.16, -0.38, -0.41999999999999998, -0.5, -0.67000000000000004, -0.48999999999999999, -0.23999999999999999, 0.28999999999999998, -0.54000000000000004, -0.66000000000000003, -0.23999999999999999, -0.17000000000000001, -0.79000000000000004, -0.089999999999999997, 0.070000000000000007, -0.20999999999999999, -0.25, -0.26000000000000001, -0, -0.56000000000000005, -0.48999999999999999, -0.35999999999999999, 0.14999999999999999, 0.34999999999999998, -0.58999999999999997, -0.56999999999999995, -0.16, -0.25, -0.42999999999999999, -0.46000000000000002, 0.01, 0.5, -0.58999999999999997, -0.37, -0.32000000000000001, -0.39000000000000001, -0.68999999999999995, 0.040000000000000001, -0.46999999999999997, -0.48999999999999999, -0.48999999999999999, 0.050000000000000003, -0.81999999999999995, -0.070000000000000007, -0.42999999999999999, -0.42999999999999999, 0.040000000000000001, -0.47999999999999998, -0.60999999999999999, 0.14999999999999999, -0.59999999999999998, -0.77000000000000002, -0.73999999999999999, -0.57999999999999996, -0.31, -0.80000000000000004, -0.87, -0.60999999999999999, -0.12, -0.73999999999999999, -0.88, 0.46999999999999997, -0.89000000000000001, -0.69999999999999996, -0.46000000000000002, -0.33000000000000002, -0.56000000000000005, 0.32000000000000001, -0.23999999999999999, -0.52000000000000002, -0.01, -0.20000000000000001, -0.42999999999999999, -0.39000000000000001, -0.56999999999999995, 0.38, -0.02, -0.75, -0.050000000000000003, -0.02, -0.91000000000000003, -0.38, -0.17000000000000001, 0.35999999999999999, -0.23999999999999999, -0.47999999999999998, 0.47999999999999998, 0.02, 0.35999999999999999, -0.46999999999999997, -0.13, 0.059999999999999998, -0.63, -0.070000000000000007, -0.070000000000000007, -0.38, -0.37, -0.75, -0.37, -0.81000000000000005, 0.34999999999999998, -0.34999999999999998, -0.41999999999999998, -0.81999999999999995, -0.60999999999999999, -0.38, 0.20999999999999999, -0.28000000000000003, -0.66000000000000003, -0.70999999999999996, -0.17999999999999999, -0.57999999999999996, -0.17000000000000001, -0.73999999999999999, -0.56000000000000005, -0.65000000000000002, -0.20000000000000001, 0.089999999999999997, -0.23999999999999999, -0.91000000000000003, -0.40999999999999998, -0.070000000000000007, -0.029999999999999999, -0.81999999999999995, -0.34000000000000002, -0.67000000000000004, -0.81999999999999995, 0.16, -0.25, -0.48999999999999999], + "configs": { + "covs_default": { + "c": 0, + "masspoints": "adjust", + "kernel": "tri", + "p": 1, + "q": 2, + "bwselect": "mserd", + "fuzzy_in": false, + "sharpbw": false, + "covs_in": true, + "covs_names": ["zlong", "zb"], + "covs_drop": true, + "h_in": null, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.23622165744802312, + "h_r": 0.23622165744802312, + "b_l": 0.35190952629406053, + "b_r": 0.35190952629406053, + "tau_cl": 1.0235383496178843, + "tau_bc": 1.0342626044730419, + "se_cl": 0.07585880061996754, + "se_rb": 0.091676835561743164, + "z": [13.492677728264381, 13.634048996561692, 11.28161326834279], + "pv": [1.7271190139343732e-41, 2.5124469604808002e-42, 1.6175210614565219e-29], + "ci_lower": [0.87485783249234317, 0.88558208734750077, 0.85457930855542452], + "ci_upper": [1.1722188667434252, 1.1829431215985828, 1.2139459003906592], + "N": [978, 222], + "N_h": [223, 134], + "N_b": [351, 170], + "bias": [0.0095959738414735513, -0.0011282810136840071], + "beta_p_l": [-0.096195308200317728, 0.017021444909685771], + "beta_p_r": [0.92734304141756652, 0.18005233723705047], + "coef_covs": [ + [0.34298832822539976], + [0.67932932918533373] + ] + }, + "covs_manual_h": { + "c": 0, + "masspoints": "adjust", + "kernel": "tri", + "p": 1, + "q": 2, + "bwselect": "mserd", + "fuzzy_in": false, + "sharpbw": false, + "covs_in": true, + "covs_names": ["zlong", "zb"], + "covs_drop": true, + "h_in": 0.20000000000000001, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.20000000000000001, + "h_r": 0.20000000000000001, + "b_l": 0.20000000000000001, + "b_r": 0.20000000000000001, + "tau_cl": 1.0172960104105004, + "tau_bc": 0.96042865191986704, + "se_cl": 0.08372836713746333, + "se_rb": 0.12340303413149525, + "z": [12.149956402951547, 11.470767730882141, 7.7828609213648487], + "pv": [5.7396194416247355e-34, 1.8501054613737493e-30, 7.0902566661118497e-15], + "ci_lower": [0.85319142633672529, 0.79632406784609189, 0.71856314943916944], + "ci_upper": [1.1814005944842756, 1.1245332359936422, 1.2022941544005648], + "N": [978, 222], + "N_h": [181, 116], + "N_b": [181, 116], + "bias": [0.0082542767021558661, 0.06512163519278924], + "beta_p_l": [-0.099328408540088967, -0.0064957774208383289], + "beta_p_r": [0.91796760187041149, 0.3596813231222461], + "coef_covs": [ + [0.34710023530281336], + [0.68055421419544582] + ] + }, + "covs_msetwo": { + "c": 0, + "masspoints": "adjust", + "kernel": "tri", + "p": 1, + "q": 2, + "bwselect": "msetwo", + "fuzzy_in": false, + "sharpbw": false, + "covs_in": true, + "covs_names": ["zlong", "zb"], + "covs_drop": true, + "h_in": null, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.34281623267384098, + "h_r": 0.23195146468793282, + "b_l": 0.49705047284549525, + "b_r": 0.33992140251285025, + "tau_cl": 1.0103593765507555, + "tau_bc": 1.0169812318677967, + "se_cl": 0.070806913245264683, + "se_rb": 0.086209466523893205, + "z": [14.269219349401942, 14.362739247580585, 11.796630612323037], + "pv": [3.4036736582973705e-46, 8.8652680371722167e-47, 4.0625447519621191e-32], + "ci_lower": [0.87158037673358457, 0.87820223205062575, 0.8480137823545546], + "ci_upper": [1.1491383763679264, 1.1557602316849676, 1.1859486813810387], + "N": [978, 222], + "N_h": [340, 131], + "N_b": [555, 165], + "bias": [0.0058746692471862427, -0.00074718606985502145], + "beta_p_l": [-0.080265857507996377, 0.20728256626047092], + "beta_p_r": [0.93009351904275916, 0.18210644680337121], + "coef_covs": [ + [0.33776729761563479], + [0.67291277097973023] + ] + }, + "covs_cercomb2": { + "c": 0, + "masspoints": "adjust", + "kernel": "tri", + "p": 1, + "q": 2, + "bwselect": "cercomb2", + "fuzzy_in": false, + "sharpbw": false, + "covs_in": true, + "covs_names": ["zlong", "zb"], + "covs_drop": true, + "h_in": null, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.17716809004988585, + "h_r": 0.16571455315568015, + "b_l": 0.37121726331035432, + "b_r": 0.35190952629406053, + "tau_cl": 1.0059041090958898, + "tau_bc": 1.0097940610357354, + "se_cl": 0.090273443056098454, + "se_rb": 0.098712113744085353, + "z": [11.142857467736029, 11.185948235166139, 10.229687347731833], + "pv": [7.7589228710353727e-29, 4.7777071455583826e-29, 1.4598993157020582e-24], + "ci_lower": [0.82897141194550938, 0.83286136388535503, 0.816321873259507], + "ci_upper": [1.1828368062462702, 1.1867267581861158, 1.2032662488119639], + "N": [978, 222], + "N_h": [160, 106], + "N_b": [377, 170], + "bias": [0.0033152331610886154, -0.00057471877875681088], + "beta_p_l": [-0.10198828827990909, -0.041316118992524249], + "beta_p_r": [0.90391582081598076, 0.71654371383772375], + "coef_covs": [ + [0.34888140720651761], + [0.68030630883365861] + ] + }, + "covs_epa": { + "c": 0, + "masspoints": "adjust", + "kernel": "epa", + "p": 1, + "q": 2, + "bwselect": "mserd", + "fuzzy_in": false, + "sharpbw": false, + "covs_in": true, + "covs_names": ["zlong", "zb"], + "covs_drop": true, + "h_in": null, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.22934710092082919, + "h_r": 0.22934710092082919, + "b_l": 0.35589690248802852, + "b_r": 0.35589690248802852, + "tau_cl": 1.0299211743361389, + "tau_bc": 1.0434113086545214, + "se_cl": 0.073783251855645868, + "se_rb": 0.088681239509088461, + "z": [13.958739259027789, 14.141573899398145, 11.765862931444342], + "pv": [2.7831353041478444e-44, 2.1052270359413155e-45, 5.8524855034657242e-32], + "ci_lower": [0.88530865803682501, 0.89879879235520743, 0.86959927311233753], + "ci_upper": [1.1745336906354529, 1.1880238249538353, 1.2172233441967051], + "N": [978, 222], + "N_h": [216, 130], + "N_b": [359, 172], + "bias": [0.0091814085319422006, -0.0043087257864400375], + "beta_p_l": [-0.092707273636242538, 0.037392710064515874], + "beta_p_r": [0.93721390069989652, 0.063071834839395727], + "coef_covs": [ + [0.338450864742316], + [0.67743558159807282] + ] + }, + "covs_drop_collinear": { + "c": 0, + "masspoints": "adjust", + "kernel": "tri", + "p": 1, + "q": 2, + "bwselect": "mserd", + "fuzzy_in": false, + "sharpbw": false, + "covs_in": true, + "covs_names": ["zlong", "zb", "zdup"], + "covs_drop": true, + "h_in": null, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.23622165744800572, + "h_r": 0.23622165744800572, + "b_l": 0.35190952629403932, + "b_r": 0.35190952629403932, + "tau_cl": 1.0235383496178858, + "tau_bc": 1.0342626044730454, + "se_cl": 0.075858800619971078, + "se_rb": 0.091676835561746578, + "z": [13.492677728263772, 13.634048996561102, 11.28161326834241], + "pv": [1.7271190139486489e-41, 2.5124469605011102e-42, 1.6175210614635118e-29], + "ci_lower": [0.87485783249233784, 0.88558208734749744, 0.8545793085554213], + "ci_upper": [1.1722188667434339, 1.1829431215985935, 1.2139459003906694], + "N": [978, 222], + "N_h": [223, 134], + "N_b": [351, 170], + "bias": [0.0095959738414730239, -0.0011282810136865606], + "beta_p_l": [-0.096195308200319546, 0.017021444909676411], + "beta_p_r": [0.92734304141756629, 0.18005233723704009], + "coef_covs": [ + [0.56943143795384654], + [0.45288621945688984] + ] + }, + "covs_ties": { + "c": 0, + "masspoints": "adjust", + "kernel": "tri", + "p": 1, + "q": 2, + "bwselect": "mserd", + "fuzzy_in": false, + "sharpbw": false, + "covs_in": true, + "covs_names": ["zlong", "zb"], + "covs_drop": true, + "h_in": null, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.11406189159087098, + "h_r": 0.11406189159087098, + "b_l": 0.22051392088736438, + "b_r": 0.22051392088736438, + "tau_cl": 0.61931390891889571, + "tau_bc": 0.5692056129172901, + "se_cl": 0.11947060380824369, + "se_rb": 0.13406367058267316, + "z": [5.1838183551237895, 4.7643988962414019, 4.2457856811124506], + "pv": [2.1738866292056973e-07, 1.8941746195147891e-06, 2.1782876947528428e-05], + "ci_lower": [0.38515582824348432, 0.33504753224187872, 0.30644564694000886], + "ci_upper": [0.85347198959430703, 0.80336369359270154, 0.83196557889457134], + "N": [971, 229], + "N_h": [100, 91], + "N_b": [205, 136], + "bias": [-0.0073436651168284238, 0.042764630884777111], + "beta_p_l": [-0.014649953796012145, 0.81522558758934593], + "beta_p_r": [0.60466395512288362, 7.6400537649466145], + "coef_covs": [ + [0.27802824774668222], + [0.68085048027000916] + ] + }, + "fuzzy_covs": { + "c": 0, + "masspoints": "adjust", + "kernel": "tri", + "p": 1, + "q": 2, + "bwselect": "mserd", + "fuzzy_in": true, + "sharpbw": false, + "covs_in": true, + "covs_names": ["zlong", "zb"], + "covs_drop": true, + "h_in": null, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.24416229315463486, + "h_r": 0.24416229315463486, + "b_l": 0.35228284984636288, + "b_r": 0.35228284984636288, + "tau_cl": 2.0549233978948473, + "tau_bc": 2.0912639576246477, + "se_cl": 0.43428267278546612, + "se_rb": 0.53853204061660775, + "z": [4.7317646470090029, 4.8154441534850818, 3.8832674750980365], + "pv": [2.2257638895608473e-06, 1.4687283703335387e-06, 0.00010306208812062817], + "ci_lower": [1.203745000125541, 1.2400855598553413, 1.0357605534952352], + "ci_upper": [2.9061017956641537, 2.942442355393954, 3.1467673617540601], + "N": [978, 222], + "N_h": [234, 136], + "N_b": [351, 170], + "bias": [0.044063077003082256, 0.0077225172732816585], + "beta_p_l": [-0.095342715800728534, 0.026708492078393924], + "beta_p_r": [0.927187164985811, 0.19071839750762803], + "tau_T": [0.49759999902383872, 0.49439967112760075, 0.49439967112760075], + "se_T": [0.10053071082131863, 0.10053071082131863, 0.12532867244443002], + "z_T": [4.9497312309694443, 4.9178969002451129, 3.9448249270079403], + "pv_T": [7.4316036851685268e-07, 8.7478962330267613e-07, 7.9858301797904112e-05], + "ci_T_lower": [0.30056342647384315, 0.29736309857760523, 0.2487599869063005], + "ci_T_upper": [0.69463657157383429, 0.69143624367759626, 0.74003935534890097], + "beta_t_p_l": [0.18337329801788124, 0.039885957063470867], + "beta_t_p_r": [0.68097329704171994, 0.18011186306537419], + "coef_covs": [ + [0.34230034383331909, 0.052556699154883237], + [0.67877042204383309, -0.022956338395740447] + ] + }, + "fuzzy_covs_sharpbw": { + "c": 0, + "masspoints": "adjust", + "kernel": "tri", + "p": 1, + "q": 2, + "bwselect": "mserd", + "fuzzy_in": true, + "sharpbw": true, + "covs_in": true, + "covs_names": ["zlong", "zb"], + "covs_drop": true, + "h_in": null, + "b_in": null, + "rho_in": null, + "level": 95, + "h_l": 0.23622165744802312, + "h_r": 0.23622165744802312, + "b_l": 0.35190952629406053, + "b_r": 0.35190952629406053, + "tau_cl": 2.0557777826839514, + "tau_bc": 2.0896424828298588, + "se_cl": 0.44349010617720935, + "se_rb": 0.54191516593254008, + "z": [4.6354535401123593, 4.7118130793088797, 3.8560324829329216], + "pv": [3.5615571473789308e-06, 2.4552253450316699e-06, 0.00011524218117730857], + "ci_lower": [1.1865531470767769, 1.2204178472226843, 1.0275082749260334], + "ci_upper": [2.9250024182911258, 2.9588671184370332, 3.1517766907336844], + "N": [978, 222], + "N_h": [223, 134], + "N_b": [351, 170], + "bias": [0.040940922969568055, 0.0070762228236604455], + "beta_p_l": [-0.096195308200317728, 0.017021444909685771], + "beta_p_r": [0.92734304141756652, 0.18005233723705047], + "tau_T": [0.49788374903127341, 0.49489878194728476, 0.49489878194728476], + "se_T": [0.10279604377539109, 0.10279604377539109, 0.12620037560965161], + "z_T": [4.8434135278508119, 4.8143757655560799, 3.9215317668946437], + "pv_T": [1.2762727887833485e-06, 1.4766073011006875e-06, 8.798784572324958e-05], + "ci_T_lower": [0.29640720547830413, 0.29342223839431547, 0.24755059091694062], + "ci_T_upper": [0.6993602925842427, 0.69637532550025405, 0.74224697297762887], + "beta_t_p_l": [0.18297432148556073, 0.026226232274922979], + "beta_t_p_r": [0.68085807051683422, 0.1881475722318636], + "coef_covs": [ + [0.34298832822539976, 0.051765111147150385], + [0.67932932918533373, -0.022490120008713043] + ] + } + } + }, "senate": { "csv": "benchmarks/data/rdrobust_senate.csv", "configs": { @@ -779,6 +1221,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, @@ -811,6 +1256,9 @@ "bwselect": "mserd", "fuzzy_in": false, "sharpbw": false, + "covs_in": false, + "covs_names": null, + "covs_drop": true, "h_in": null, "b_in": null, "rho_in": null, diff --git a/diff_diff/_rdrobust_port.py b/diff_diff/_rdrobust_port.py index e452277e3..34a1492ab 100644 --- a/diff_diff/_rdrobust_port.py +++ b/diff_diff/_rdrobust_port.py @@ -1,9 +1,9 @@ """In-house port of rdrobust's RD bandwidth-selection and estimation -machinery - sharp and fuzzy paths. +machinery - sharp, fuzzy, and covariate-adjusted paths. -Faithful Python translation of the sharp and fuzzy no-covariate/no-cluster -``nn`` branches of ``rdbwselect`` and ``rdrobust`` from the R package -``rdrobust`` 4.0.0, +Faithful Python translation of the sharp, fuzzy, and covariate-adjusted +no-cluster ``nn`` branches of ``rdbwselect`` and ``rdrobust`` from the R +package ``rdrobust`` 4.0.0, ported from the CRAN source tarball (sha256 below), cross-checked against ``deparse(getFromNamespace(, "rdrobust"))`` of the installed 4.0.0 package. The unreleased GitHub development tree (4.1.0-dev) differs from @@ -23,12 +23,15 @@ ``compute_dups_dupsid(x_sorted)`` rle blocks (rdbwselect.R:322-327) ``rdrobust_res_nn(...)`` ``rdrobust_res`` vce="nn" branch (functions.R:146-181) -``rdrobust_vce(RX, res)`` ``rdrobust_vce`` null-cluster d==0 - branch (functions.R:374-378) -``rdrobust_bw(...)`` ``rdrobust_bw`` sharp path +``rdrobust_vce(RX, res)`` ``rdrobust_vce`` null-cluster + branches (functions.R:374-385) +``rdrobust_bw(...)`` ``rdrobust_bw`` (functions.R:207-355) ``rdbwselect(...)`` ``rdbwselect`` main flow (rdbwselect.R; anchors inline) +``covs_drop_fun(z)`` ``covs_drop_fun`` + (functions.R:683-688) via + LINPACK dqrdc2 rank/pivot ========================================== =================================== Deviations from rdrobust (documented; see REGISTRY.md RegressionDiscontinuity @@ -52,6 +55,13 @@ ``numpy.linalg.pinv(G, rcond=sqrt(eps))`` - both are Moore-Penrose pseudo-inverses with the same default singular-value cutoff. Reachable only on degenerate (rank-deficient) kernel windows. +* Degenerate covariate adjustment is GUARDED instead of reproduced: R's + ``ginv(ZWZ, tol=1e-20)`` inverts a float-noise singular value on + exactly-degenerate partialled systems (constant covariate, full dummy + set), making its output platform-noise. See :func:`_covs_gamma` for the + guard (per-column exclusion + scale-invariant stabilized cut + warning); + well-posed systems reproduce R exactly. Rank-0 covariate matrices fail + closed with a clear error. Nothing in this module is shared with ``diff_diff._nprobust_port``: the corresponding nprobust primitives differ in kernel scaling (``/h``), @@ -81,6 +91,7 @@ "compute_dups_dupsid", "rdrobust_res_nn", "rdrobust_vce", + "covs_drop_fun", "rdrobust_bw", "rdbwselect", "quantile_type2", @@ -215,6 +226,7 @@ def rdrobust_res_nn( dups: np.ndarray, dupsid: np.ndarray, t: Optional[np.ndarray] = None, + z: Optional[np.ndarray] = None, ) -> np.ndarray: """Nearest-neighbor variance residuals (functions.R:146-181, ``vce == "nn"`` branch). @@ -225,14 +237,17 @@ def rdrobust_res_nn( deliberately absent). Equal left/right distances expand BOTH directions (functions.R:162-165). Returns the (n,) residual vector ``sqrt(J/(J+1)) * (y_i - mean(y_neighbors))`` for the sharp - outcome-only case, or the (n, 2) residual matrix with the fuzzy - take-up column ``sqrt(J/(J+1)) * (t_i - mean(t_neighbors))`` appended - when ``t`` is supplied (functions.R:171-174; T shares Y's neighbor - sets exactly - both depend only on ``x``). + outcome-only case, or the (n, 1+dT+dZ) residual matrix with the fuzzy + take-up column (functions.R:171-174) and covariate columns + (functions.R:175-180) appended when ``t`` / ``z`` are supplied - the + extra responses share Y's neighbor sets exactly (all depend only on + ``x``). Column order matches R's response stack: [Y, T, Z...]. """ n = y.shape[0] fuzzy = t is not None - res = np.empty((n, 2) if fuzzy else n, dtype=np.float64) + dZ = 0 if z is None else z.shape[1] + ncol = 1 + (1 if fuzzy else 0) + dZ + res = np.empty((n, ncol) if ncol > 1 else n, dtype=np.float64) limit = min(matches, n - 1) for pos in range(n): # R pos is 1-based; comments track R indices rpos = int(dups[pos] - dupsid[pos]) @@ -264,13 +279,20 @@ def rdrobust_res_nn( y_J = float(np.sum(y[lo : hi + 1])) - float(y[pos]) Ji = (hi - lo + 1) - 1 r_y = np.sqrt(Ji / (Ji + 1)) * (y[pos] - y_J / Ji) + if ncol == 1: + res[pos] = r_y + continue + res[pos, 0] = r_y + col = 1 if fuzzy: assert t is not None t_J = float(np.sum(t[lo : hi + 1])) - float(t[pos]) # functions.R:172 - res[pos, 0] = r_y - res[pos, 1] = np.sqrt(Ji / (Ji + 1)) * (t[pos] - t_J / Ji) - else: - res[pos] = r_y + res[pos, col] = np.sqrt(Ji / (Ji + 1)) * (t[pos] - t_J / Ji) + col += 1 + for i in range(dZ): # functions.R:175-180 + assert z is not None + z_J = float(np.sum(z[lo : hi + 1, i])) - float(z[pos, i]) + res[pos, col + i] = np.sqrt(Ji / (Ji + 1)) * (z[pos, i] - z_J / Ji) return res @@ -329,6 +351,201 @@ def _fuzzy_identification_stop(t_l: np.ndarray, t_r: np.ndarray) -> None: ) +def covs_drop_fun(z: np.ndarray, tol: float = 1e-7) -> Tuple[np.ndarray, int]: + """Redundant-covariate detection: R's ``covs_drop_fun`` + (functions.R:683-688) = ``qr(z, tol=1e-7)`` rank/pivot, keep + ``sort(pivot[1:rank])``. + + R's default ``qr()`` is LINPACK ``dqrdc2``, whose limited pivoting + cycles a column to the right edge when its REDUCED norm falls below + ``tol`` times that column's OWN original norm (zero-norm columns take + an original norm of 1.0, dqrdc2.f:8) - a per-column relative rule, so + small-but-independent covariates are never dropped. LAPACK's ``geqp3`` + pivots differently (greedy by current norm), so the dqrdc2 loop is + ported directly rather than approximated; pivot order decides WHICH of + a collinear set survives. Returns ``(keep, rank)`` with ``keep`` the + sorted 0-based indices of retained columns. + """ + x = np.array(z, dtype=np.float64, copy=True) + n, p = x.shape + jpvt = np.arange(p) + qraux = np.sqrt((x * x).sum(axis=0)) + work1 = qraux.copy() # dqrdc2 work(j,1): recompute reference norm + work2 = qraux.copy() # dqrdc2 work(j,2): original norm for the tol test + work2[work2 == 0.0] = 1.0 # dqrdc2.f:8 zero-norm fixup + k = p + 1 + rank = 0 + for ll in range(min(n, p)): + # Cycle negligible columns to the right edge (dqrdc2.f:80-120); + # the ll < k-1 guard prevents infinite cycling. + while ll < k - 1 and qraux[ll] < work2[ll] * tol: + x[:, ll:p] = np.roll(x[:, ll:p], -1, axis=1) + jpvt[ll:p] = np.roll(jpvt[ll:p], -1) + qraux[ll:p] = np.roll(qraux[ll:p], -1) + work1[ll:p] = np.roll(work1[ll:p], -1) + work2[ll:p] = np.roll(work2[ll:p], -1) + k -= 1 + rank = ll + 1 + # Householder for column ll + LINPACK norm downdate with the + # 0.05-heuristic recompute (dqrdc2.f main loop). + nrmxl = float(np.sqrt((x[ll:, ll] ** 2).sum())) + if nrmxl == 0.0: + continue + if x[ll, ll] != 0.0: + nrmxl = float(np.copysign(nrmxl, x[ll, ll])) + x[ll:, ll] /= nrmxl + x[ll, ll] += 1.0 + for j in range(ll + 1, p): + tval = -(x[ll:, ll] @ x[ll:, j]) / x[ll, ll] + x[ll:, j] += tval * x[ll:, ll] + if qraux[j] != 0.0: + tt = 1.0 - (abs(x[ll, j]) / qraux[j]) ** 2 + tt = max(tt, 0.0) + t_keep = tt + tt = 1.0 + 0.05 * tt * (qraux[j] / work1[j]) ** 2 + if tt != 1.0: + qraux[j] *= float(np.sqrt(t_keep)) + else: + qraux[j] = float(np.sqrt((x[ll + 1 :, j] ** 2).sum())) + work1[j] = qraux[j] + qraux[ll] = 0.0 + rank = min(rank, k - 1) + keep = np.sort(jpvt[:rank]) + return keep, int(rank) + + +def _covs_gamma( + ZWZ: np.ndarray, + ZWY: np.ndarray, + diag_pre: np.ndarray, + covs_drop: bool, +) -> Tuple[np.ndarray, np.ndarray, bool]: + """Covariate-projection solve ``gamma`` from the partialled normal + equations (rdrobust.R:659-671 / functions.R:246-257). + + ``covs_drop=False``: R's ``chol2inv(chol(ZWZ))`` - a strict Cholesky + solve that fails hard on a collinear system (clear ``ValueError`` + here instead of R's opaque ``chol()`` error). + + ``covs_drop=True``: R uses ``MASS::ginv(ZWZ, tol=1e-20)``. On a + well-posed system that equals ``np.linalg.pinv(rcond=1e-20)`` and is + reproduced exactly. On an EXACTLY-degenerate system (covariates + collinear with the local polynomial design after partialling: a + constant covariate, or a full dummy set - both pass the intercept-free + ``covs_drop_fun`` QR check) R inverts a FLOAT-NOISE singular value + (~1e-16 * sv_max > 1e-20 * sv_max), making its gamma platform-noise + and shifting tau silently. Documented deviation from R - guarded + instead of reproduced: + + * per-column: ``diag(ZWZ)_j / (z_j' W z_j) < 1e-14`` means column j is + numerically fully explained by the design -> excluded (gamma row 0; + a constant covariate then contributes exactly nothing and the fit + matches the one without it to floating-point roundoff - not + bit-for-bit, because the response matrix still carries the excluded + column and BLAS matmul kernels differ with matrix SHAPE on some + platforms); + * set-level: equilibrated (scale-invariant) singular values of the + remaining block with ``sv_min < 1e-12 * sv_max`` -> stabilized + equilibrated pseudo-inverse with the noise directions cut + (``rcond=1e-12``); tau then equals any identified reparametrization + of the same covariate span (e.g. dropping one dummy category); + * otherwise the raw ``pinv(rcond=1e-20)`` solve, R-identical + (tiny-SCALED independent covariates stay on this path - the + equilibration makes the check scale-invariant). + + Returns ``(gamma, excluded_mask, set_degenerate)``. + """ + dZ = ZWZ.shape[0] + n_rhs = ZWY.shape[1] + with np.errstate(divide="ignore", invalid="ignore"): + ratio = np.diag(ZWZ) / diag_pre + excluded = ratio < 1e-14 # NaN (0/0) compares False -> handled below + excluded |= ~np.isfinite(ratio) + keep = np.flatnonzero(~excluded) + set_degenerate = False + zwz_k = dvec = eq = None + if keep.size > 0: + zwz_k = ZWZ[np.ix_(keep, keep)] + dvec = np.sqrt(np.diag(zwz_k)) + eq = zwz_k / np.outer(dvec, dvec) + sv = np.linalg.svd(eq, compute_uv=False) + set_degenerate = bool(sv[-1] < 1e-12 * sv[0]) + if not covs_drop: + # Strict mode: fail DETERMINISTICALLY on any degeneracy. (R's + # covs_drop=FALSE relies on chol() erroring, which on an + # exactly-singular float matrix is roundoff-dependent - it can + # "succeed" through a tiny positive pivot and return noise; the + # explicit check makes the strict contract reliable.) + if excluded.any() or set_degenerate or keep.size == 0: + raise ValueError( + "Covariates are collinear with each other or with the " + "local polynomial design (the partialled covariate Gram " + "matrix is singular) and covs_drop=False requests a " + "strict solve. Remove the redundant covariates or use " + "covs_drop=True." + ) + try: + cf = _scipy_linalg.cho_factor(ZWZ, lower=False) + gamma = _scipy_linalg.cho_solve(cf, ZWY) + except _scipy_linalg.LinAlgError: + raise ValueError( + "Covariates are collinear (the partialled covariate Gram " + "matrix is not positive definite) and covs_drop=False " + "requests a strict solve. Remove the redundant covariates " + "or use covs_drop=True." + ) from None + return gamma, np.zeros(dZ, dtype=bool), False + gamma = np.zeros((dZ, n_rhs)) + if keep.size == 0: + return gamma, excluded, True + assert zwz_k is not None and dvec is not None and eq is not None + zwy = ZWY[keep] + if set_degenerate: + gamma_k = np.linalg.pinv(eq, rcond=1e-12) @ (zwy / dvec[:, None]) + gamma_k /= dvec[:, None] + else: + gamma_k = np.linalg.pinv(zwz_k, rcond=1e-20) @ zwy + gamma[keep] = gamma_k + return gamma, excluded, set_degenerate + + +def _covs_entry_drop( + covs: np.ndarray, covs_drop: bool, warn: bool = True +) -> Tuple[np.ndarray, np.ndarray]: + """Entry-point redundant-covariate drop, shared by :func:`rdbwselect` + and :func:`rdrobust_fit` (rdbwselect.R:164-181 == rdrobust.R:121-140, + minus the name-length column sort - the port takes an unnamed matrix, + for which R's ``order(nchar(...))`` sort is a stable no-op; the + estimator applies the name sort before building the matrix). + + Returns ``(reduced_covs, dropped_indices)``. Rank 0 fails closed with + a clear error (R would index a nonexistent column downstream). + """ + dZ = covs.shape[1] + if not covs_drop: + return covs, np.array([], dtype=np.int64) + keep, rank = covs_drop_fun(covs) + if rank == 0: + raise ValueError( + "All covariates are numerically zero (rank-0 covariate " + "matrix); remove the covariates instead." + ) + if rank < dZ: + dropped = np.setdiff1d(np.arange(dZ), keep) + if warn: + # R's message (rdrobust.R:138) with the dropped 0-based column + # indices appended; the estimator maps indices to column names + # and warns itself instead. + warnings.warn( + "Multicollinearity issue detected in covs. Redundant " + f"covariates dropped (column indices {dropped.tolist()}).", + UserWarning, + stacklevel=3, + ) + return covs[:, keep], dropped + return covs, np.array([], dtype=np.int64) + + @dataclass class _BwPilot: """Per-side pilot block returned by :func:`rdrobust_bw` @@ -356,26 +573,33 @@ def rdrobust_bw( dups: np.ndarray, dupsid: np.ndarray, t: Optional[np.ndarray] = None, + z: Optional[np.ndarray] = None, + covs_drop: bool = True, vcache: Optional[Dict[str, Tuple[float, float, Optional[np.ndarray]]]] = None, ) -> _BwPilot: - """Per-side pilot V/B(/R) block (functions.R:207-355, no-covariate - sharp and fuzzy paths). - - Sharp (``t=None``): Z = C = W = NULL so the combination vector ``s`` - is the scalar 1 (functions.R:234) and the response is the outcome - column alone. Fuzzy: T is stacked as a second response column into - BOTH the V-fit and B-fit designs (functions.R:236-240, 315-318) and - the pilot ratio + delta vector ``s = [1/tau_T, -tau_Y/tau_T^2]`` is - computed from the V-fit coefficients (functions.R:264-268), then - threaded into the V/B variance meats and the bias constant + """Per-side pilot V/B(/R) block (functions.R:207-355, sharp, fuzzy, + and covariate-adjusted paths). + + Sharp (``t=None, z=None``): C = W = NULL so the combination vector + ``s`` is the scalar 1 (functions.R:234) and the response is the + outcome column alone. Fuzzy: T is stacked as a second response column + into BOTH the V-fit and B-fit designs (functions.R:236-240, 315-318) + and the pilot ratio + delta vector ``s = [1/tau_T, -tau_Y/tau_T^2]`` + is computed from the V-fit coefficients (functions.R:264-268). + Covariates: Z stacks after T; a PER-PILOT gamma comes from the + partialled normal equations inside the V-window (functions.R:241-258; + degenerate systems take the silent stabilized solve documented at + :func:`_covs_gamma`), giving ``s = [1, -gamma[,1]]`` (sharp+covs) or + the length-(2+dZ) fuzzy+covs vector of functions.R:269-274. ``s`` + threads into the V/B variance meats and the bias constant ``t(s) %*% beta_B[o+2,]`` (functions.R:294, 346, 349). A pilot window with no take-up variation makes ``tau_T == 0``; the division follows R's Inf/NaN flow-on (numpy float under ``errstate``) and the downstream stage assembly fails closed on the non-finite bandwidth. ``vcache`` shares the fixed-``h_V`` V-fit across pilot calls keyed on ``(o, nu)`` (functions.R:216-222) and stores ``(V_V, BConst, s)`` - - the cached ``V_V`` embeds the fuzzy ``s``, so ``s`` must be reused on - cache hits exactly as R's environment cache does. + the cached ``V_V`` embeds the fuzzy/covariate ``s``, so ``s`` must be + reused on cache hits exactly as R's environment cache does. """ if vce != "nn": raise NotImplementedError( @@ -395,28 +619,69 @@ def rdrobust_bw( eW = w[ind_V] R_V = rdrobust_vander(eX - c, o) invG_V = qrXXinv(R_V * np.sqrt(eW)[:, None]) - if t is None: + eT = t[ind_V] if t is not None else None # functions.R:236-240 + eZ = z[ind_V] if z is not None else None # functions.R:241-244 + if eT is None and eZ is None: # R computes beta_V here (functions.R:263) but the sharp/nn # path never consumes it (it feeds the fuzzy ratio and hc # predictions); omitted - no numeric effect on V, B, or R. s = None res_V = rdrobust_res_nn(eX, eY, nnmatch, dups[ind_V], dupsid[ind_V]) # functions.R:293 else: - eT = t[ind_V] # functions.R:236-240 - D_V = np.column_stack([eY, eT]) + dT = 0 if eT is None else 1 + resp = [eY] + ([eT] if eT is not None else []) + D_V = np.column_stack(resp + ([eZ] if eZ is not None else [])) + s = None + gamma = None + if eZ is not None: + # Per-pilot partialled gamma (functions.R:245-257). + U = (R_V * eW[:, None]).T @ D_V + ZWD = (eZ * eW[:, None]).T @ D_V + colsZ = slice(1 + dT, D_V.shape[1]) + UiGU = U[:, colsZ].T @ (invG_V @ U) + gamma, _, _ = _covs_gamma( + ZWD[:, colsZ] - UiGU[:, colsZ], + ZWD[:, : 1 + dT] - UiGU[:, : 1 + dT], + np.diag(ZWD[:, colsZ]).copy(), + covs_drop, + ) + s = np.concatenate([[1.0], -gamma[:, 0]]) # functions.R:257 beta_V = invG_V @ (R_V * eW[:, None]).T @ D_V # functions.R:263 - # Fuzzy pilot ratio + delta vector (functions.R:264-268); R row - # nu+1 (1-based) is 0-based nu. - tau_Y = float(math.factorial(nu)) * float(beta_V[nu, 0]) - tau_T = float(math.factorial(nu)) * float(beta_V[nu, 1]) - with np.errstate(divide="ignore", invalid="ignore"): - s = np.array( - [ - float(np.float64(1.0) / np.float64(tau_T)), - float(-(np.float64(tau_Y) / np.float64(tau_T) ** 2)), - ] + if eT is not None and eZ is None: + # Fuzzy pilot ratio + delta vector (functions.R:264-268); R + # row nu+1 (1-based) is 0-based nu. + tau_Y = float(math.factorial(nu)) * float(beta_V[nu, 0]) + tau_T = float(math.factorial(nu)) * float(beta_V[nu, 1]) + with np.errstate(divide="ignore", invalid="ignore"): + s = np.array( + [ + float(np.float64(1.0) / np.float64(tau_T)), + float(-(np.float64(tau_Y) / np.float64(tau_T) ** 2)), + ] + ) + elif eT is not None and eZ is not None: + # Fuzzy + covariates (functions.R:269-274): adjusted ratio + # from the covariate-combined coefficients, then the + # extended delta vector. + assert gamma is not None and s is not None + s_T = np.concatenate([[1.0], -gamma[:, 1]]) + colsZ = slice(2, D_V.shape[1]) + tau_Y = float(math.factorial(nu)) * float( + s @ np.concatenate([[beta_V[nu, 0]], beta_V[nu, colsZ]]) ) - res_V = rdrobust_res_nn(eX, eY, nnmatch, dups[ind_V], dupsid[ind_V], t=eT) + tau_T = float(math.factorial(nu)) * float( + s_T @ np.concatenate([[beta_V[nu, 1]], beta_V[nu, colsZ]]) + ) + with np.errstate(divide="ignore", invalid="ignore"): + inv_tT = float(np.float64(1.0) / np.float64(tau_T)) + ratio2 = float(np.float64(tau_Y) / np.float64(tau_T) ** 2) + s = np.concatenate( + [ + [inv_tT, -ratio2], + -inv_tT * gamma[:, 0] + ratio2 * gamma[:, 1], + ] + ) + res_V = rdrobust_res_nn(eX, eY, nnmatch, dups[ind_V], dupsid[ind_V], t=eT, z=eZ) aux = rdrobust_vce(R_V * eW[:, None], res_V, s) # functions.R:294 V_V = float((invG_V @ aux @ invG_V)[nu, nu]) # functions.R:295 v = (R_V * eW[:, None]).T @ ((eX - c) / h_V) ** (o + 1) # :296 @@ -432,23 +697,22 @@ def rdrobust_bw( eW = w[ind] R_B = rdrobust_vander(eX - c, o_B) invG_B = qrXXinv(R_B * np.sqrt(eW)[:, None]) - if t is None: + eT_B = t[ind] if t is not None else None # functions.R:315-318 + eZ_B = z[ind] if z is not None else None # functions.R:319-322 + if eT_B is None and eZ_B is None: beta_B = invG_B @ (R_B * eW[:, None]).T @ eY # functions.R:326 # functions.R:349-353 with sharp s == 1: t(s) %*% beta_B[o+2,] is # the scalar coefficient (R row o+2 1-based = 0-based o+1). beta_B_comb = float(beta_B[o + 1]) else: - eT_B = t[ind] # functions.R:315-318 - D_B = np.column_stack([eY, eT_B]) + resp_B = [eY] + ([eT_B] if eT_B is not None else []) + D_B = np.column_stack(resp_B + ([eZ_B] if eZ_B is not None else [])) beta_B = invG_B @ (R_B * eW[:, None]).T @ D_B # functions.R:326 assert s is not None beta_B_comb = float(s @ beta_B[o + 1, :]) # functions.R:349 BWreg = 0.0 if scale > 0: # functions.R:328-348 - if t is None: - res_B = rdrobust_res_nn(eX, eY, nnmatch, dups[ind], dupsid[ind]) - else: - res_B = rdrobust_res_nn(eX, eY, nnmatch, dups[ind], dupsid[ind], t=t[ind]) + res_B = rdrobust_res_nn(eX, eY, nnmatch, dups[ind], dupsid[ind], t=eT_B, z=eZ_B) V_B = float( (invG_B @ rdrobust_vce(R_B * eW[:, None], res_B, s) @ invG_B)[o + 1, o + 1] ) # functions.R:346 - R row/col o+2 is 0-based (o+1, o+1) @@ -520,10 +784,12 @@ def rdbwselect( warn_masspoints: bool = True, fuzzy: Optional[np.ndarray] = None, sharpbw: bool = False, + covs: Optional[np.ndarray] = None, + covs_drop: bool = True, ) -> RdBwselectResult: """RD data-driven bandwidth selection, all 10 selectors - (rdbwselect.R main flow at the anchors cited inline; sharp and fuzzy - no-covariate paths). + (rdbwselect.R main flow at the anchors cited inline; sharp, fuzzy, + and covariate-adjusted paths). Always computes the full selector matrix (R's ``all=TRUE``): the ten selectors share the same six per-side pilot blocks, so the marginal @@ -540,9 +806,17 @@ def rdbwselect( ``sharpbw=True`` OR either side has zero take-up variance (``perf_comp``, one-sided perfect compliance), T is nulled for SELECTION ONLY and the sharp reduced-form objective on Y is used - (rdbwselect.R:334-346); estimation always remains fuzzy. Standardize - note: R's ``stdvars`` scales y and x only - the fuzzy column is never - standardized (rdbwselect.R:120-129). + (rdbwselect.R:334-346); estimation always remains fuzzy. + + Covariates (``covs`` = (n, dZ) matrix): bandwidths are + COVARIATE-AWARE - after the entry-point redundant-column drop + (rdbwselect.R:164-181, ``covs_drop``), Z is threaded into every pilot + of all three chains alongside T (rdbwselect.R:330-332, 386-457), so + the pilot V/B constants are those of the covariate-adjusted estimator. + ``perf_comp``/``sharpbw`` null ONLY T - Z always stays in selection + (rdbwselect.R:343-345). Standardize note: R's ``stdvars`` scales y and + x only - the fuzzy and covariate columns are never standardized + (rdbwselect.R:120-129). """ y = np.asarray(y, dtype=np.float64) x = np.asarray(x, dtype=np.float64) @@ -575,6 +849,25 @@ def rdbwselect( "estimator warns-and-drops; R's complete.cases filter " "includes the fuzzy column)." ) + if not isinstance(covs_drop, (bool, np.bool_)): + raise ValueError(f"covs_drop must be a bool; got {covs_drop!r}.") + if covs is not None: + covs = np.asarray(covs, dtype=np.float64) + if covs.ndim == 1: + covs = covs.reshape(-1, 1) + if covs.ndim != 2: + raise ValueError( + f"covs must be a 1-D vector or (n, dZ) matrix; got shape {covs.shape}." + ) + if covs.shape[0] != x.shape[0]: + raise ValueError(f"covs must have {x.shape[0]} rows to match x; got {covs.shape[0]}.") + if not np.all(np.isfinite(covs)): + raise ValueError( + "covs must be finite and complete-case; drop or impute " + "missing values before bandwidth selection (the public " + "estimator warns-and-drops; R's complete.cases filter " + "includes the covariate columns)." + ) if not (np.all(np.isfinite(y)) and np.all(np.isfinite(x))): raise ValueError( "y and x must be finite and complete-case; drop or impute " @@ -625,6 +918,14 @@ def rdbwselect( y = y[order_x] if fuzzy is not None: fuzzy = fuzzy[order_x] # rdbwselect.R:112 (fuzzy = fuzzy[order_x,]) + if covs is not None: + covs = covs[order_x] # rdbwselect.R:110 + + # --- Entry-point redundant-covariate drop (rdbwselect.R:164-181), + # after the row sort so near-threshold QR rank decisions see the same + # row order as R (and as rdrobust_fit). --- + if covs is not None: + covs, _ = _covs_entry_drop(covs, covs_drop) # --- Degeneracy guards BEFORE any standardization division: a constant # running variable must surface as the assumption failure it is, not as @@ -741,6 +1042,16 @@ def rdbwselect( if not (perf_comp or sharpbw): # rdbwselect.R:344-346 null-out T_sel_l, T_sel_r = T_l_full, T_r_full + # --- Covariate split (rdbwselect.R:330-332). Z is NEVER nulled by + # perf_comp/sharpbw - those switches drop only T (rdbwselect.R:344), + # so sharpbw-with-covariates selects on the covariate-adjusted sharp + # objective. --- + Z_sel_l: Optional[np.ndarray] = None + Z_sel_r: Optional[np.ndarray] = None + if covs is not None: + Z_sel_l = covs[ind_l] + Z_sel_r = covs[ind_r] + # --- NN tie blocks (rdbwselect.R:322-327) --- dups_l, dupsid_l = compute_dups_dupsid(X_l) dups_r, dupsid_r = compute_dups_dupsid(X_r) @@ -750,9 +1061,10 @@ def rdbwselect( def _bw(side: str, o: int, nu: int, o_B: int, h_B: float, scale: float) -> _BwPilot: # Single funnel for ALL 14 pilot calls across the mserd, msetwo, - # and msesum chains: threading T here guarantees every chain's - # pilots receive the fuzzy column (R passes T_l/T_r into each - # chain's calls individually, rdbwselect.R:386-457). + # and msesum chains: threading T and Z here guarantees every + # chain's pilots receive the fuzzy and covariate columns (R passes + # T_l/T_r, Z_l/Z_r into each chain's calls individually, + # rdbwselect.R:386-457). if side == "l": return rdrobust_bw( Y_l, @@ -770,6 +1082,8 @@ def _bw(side: str, o: int, nu: int, o_B: int, h_B: float, scale: float) -> _BwPi dups_l, dupsid_l, t=T_sel_l, + z=Z_sel_l, + covs_drop=bool(covs_drop), vcache=vcache_l, ) return rdrobust_bw( @@ -788,6 +1102,8 @@ def _bw(side: str, o: int, nu: int, o_B: int, h_B: float, scale: float) -> _BwPi dups_r, dupsid_r, t=T_sel_r, + z=Z_sel_r, + covs_drop=bool(covs_drop), vcache=vcache_r, ) @@ -977,7 +1293,7 @@ def _stage_bw(num, den, rate, clamp_max=None, floors=()): @dataclass class RdFitResult: """RD point estimates and variances (rdrobust.R estimation body, - sharp and fuzzy no-covariate paths). + sharp, fuzzy, and covariate-adjusted paths). ``tau_cl`` is the conventional RD estimate (the fuzzy ratio ``tau_Y_cl/tau_T_cl`` on fuzzy fits), ``tau_bc`` the bias-corrected @@ -986,15 +1302,25 @@ class RdFitResult: three output rows map as Conventional = (tau_cl, se_cl), Bias-Corrected = (tau_bc, se_cl), Robust = (tau_bc, se_rb) (rdrobust.R:854-863). ``beta_p_l``/``beta_p_r`` are the per-side - order-p outcome coefficient vectors (rdplot seam); - ``bias_l``/``bias_r`` the per-side estimated biases (sharp: - rdrobust.R:629-630; fuzzy: the LINEARIZED ``s_Y . B_F_side``, - rdrobust.R:649-652 - a different formula, not the per-component - difference). Fuzzy-only fields (None on sharp fits): the first-stage - ``tau_T_cl/tau_T_bc/se_T_cl/se_T_rb`` (rdrobust.R:637-638, 800-822) - and per-side take-up coefficient vectors ``beta_t_p_l/beta_t_p_r`` - (raw, like ``beta_p_*``; R applies ``scalepar*factorial(deriv)`` to - both - identical at the public deriv=0/scalepar=1 surface). + order-p outcome coefficient vectors (rdplot seam; on + covariate-adjusted fits these are the ADJUSTED vectors ``s_Y`` + applied across the response columns, matching R's ``beta_Y_p_*``, + rdrobust.R:685-686/706-709); ``bias_l``/``bias_r`` the per-side + estimated biases (sharp: rdrobust.R:629-630; fuzzy: the LINEARIZED + ``s_Y . B_F_side``, rdrobust.R:649-652 - a different formula, not the + per-component difference). Fuzzy-only fields (None on sharp fits): + the first-stage ``tau_T_cl/tau_T_bc/se_T_cl/se_T_rb`` + (rdrobust.R:637-638, 800-822) and per-side take-up coefficient + vectors ``beta_t_p_l/beta_t_p_r`` (raw, like ``beta_p_*``; R applies + ``scalepar*factorial(deriv)`` to both - identical at the public + deriv=0/scalepar=1 surface). Covariate-only fields (None otherwise): + ``gamma_p`` = R's ``coef_covs``, the (dZ, 1+dT) common projection + coefficients over the covariates KEPT by the entry-point drop + (column 0 = outcome equation, column 1 = first-stage equation on + fuzzy fits, rdrobust.R:907); ``covs_excluded`` = per-kept-column + bool mask of covariates excluded by the degeneracy guard (see + :func:`_covs_gamma`); ``covs_set_degenerate`` = True when the + set-level stabilized cut engaged. """ tau_cl: float @@ -1015,6 +1341,9 @@ class RdFitResult: se_T_rb: Optional[float] = None beta_t_p_l: Optional[np.ndarray] = None beta_t_p_r: Optional[np.ndarray] = None + gamma_p: Optional[np.ndarray] = None + covs_excluded: Optional[np.ndarray] = None + covs_set_degenerate: bool = False def rdrobust_fit( @@ -1032,9 +1361,12 @@ def rdrobust_fit( vce: str = "nn", nnmatch: int = 3, t: Optional[np.ndarray] = None, + covs: Optional[np.ndarray] = None, + covs_drop: bool = True, + warn_covs_degenerate: bool = True, ) -> RdFitResult: - """RD estimation at known bandwidths (rdrobust.R:533-822, sharp and - fuzzy no-covariate/no-cluster paths with ``scalepar = 1``). + """RD estimation at known bandwidths (rdrobust.R:533-822, sharp, + fuzzy, and covariate-adjusted no-cluster paths with ``scalepar = 1``). Inputs must be complete-case 1-D arrays (same contract as :func:`rdbwselect`); sorting, side-splitting, and NN tie blocks @@ -1055,14 +1387,30 @@ def rdrobust_fit( ``tau_bc = tau_cl - s_Y . B_F`` (rdrobust.R:636-657). The identification guard (both-sides-constant T with no jump) raises here too, covering manual-bandwidth fits that skip selection. + Covariates (``covs`` = (n, dZ) matrix): after the entry-point + redundant-column drop (rdrobust.R:121-140, ``covs_drop``), Z + stacks after T through the SAME fits (rdrobust.R:593-598); a + common POOLED-ACROSS-SIDES gamma comes from the per-side + partialled normal equations summed (rdrobust.R:659-671; degenerate + systems take the guarded solve documented at :func:`_covs_gamma` - + ``warn_covs_degenerate=False`` lets the estimator own that warning + with column names, the masspoints pattern), and the adjusted + estimates apply ``s_Y = [1, -gamma[,1]]`` across the response + columns (rdrobust.R:672-686; the R branch omits + ``factorial(deriv)`` present in the no-covariate branch - + identical at the fixed deriv=0 surface, replicated verbatim). + Fuzzy + covariates composes both: adjusted Y and T jumps, their + ratio, and the extended delta vectors of rdrobust.R:688-723. 3. Variances: conventional sandwiches ``R_p * W_h`` with same-side NN residuals; robust sandwiches ``Q_q`` with the SAME residuals (``res_b = res_h`` for vce="nn", rdrobust.R:753-754; the h==b special branches at rdrobust.R:773-786 are cluster-only and never - taken on this path). Fuzzy: the (n, 2) residual matrix is collapsed - by ``s_Y`` for the ratio variance and by ``sV_T = [0, 1]`` for the - first-stage variance (rdrobust.R:769-822); a zero first-stage jump - follows R's Inf/NaN flow-on (numpy float under ``errstate``). + taken on this path). Fuzzy/covariates: the (n, 1+dT+dZ) residual + matrix is collapsed by the delta vector for the main variance and + by ``sV_T`` (``[0, 1]``, or ``[0, 1, -gamma[,2]]`` with + covariates) for the first-stage variance (rdrobust.R:769-822); a + zero first-stage jump follows R's Inf/NaN flow-on (numpy float + under ``errstate``). """ y = np.asarray(y, dtype=np.float64) x = np.asarray(x, dtype=np.float64) @@ -1090,6 +1438,23 @@ def rdrobust_fit( "t must be finite and complete-case; drop or impute missing " "values before estimation." ) + if not isinstance(covs_drop, (bool, np.bool_)): + raise ValueError(f"covs_drop must be a bool; got {covs_drop!r}.") + if covs is not None: + covs = np.asarray(covs, dtype=np.float64) + if covs.ndim == 1: + covs = covs.reshape(-1, 1) + if covs.ndim != 2: + raise ValueError( + f"covs must be a 1-D vector or (n, dZ) matrix; got shape {covs.shape}." + ) + if covs.shape[0] != x.shape[0]: + raise ValueError(f"covs must have {x.shape[0]} rows to match x; got {covs.shape[0]}.") + if not np.all(np.isfinite(covs)): + raise ValueError( + "covs must be finite and complete-case; drop or impute " + "missing values before estimation." + ) if vce != "nn": raise NotImplementedError( "Only vce='nn' is ported in v1 (rdrobust default); hc0-hc3 and " @@ -1122,6 +1487,12 @@ def rdrobust_fit( y = y[order_x] if t is not None: t = t[order_x] # rdrobust.R:115 + if covs is not None: + covs = covs[order_x] # rdrobust.R:114 + # Entry-point redundant-covariate drop (rdrobust.R:121-140), after + # the row sort so near-threshold QR rank decisions see the same + # row order as R (and as rdbwselect). + covs, _ = _covs_entry_drop(covs, covs_drop) ind_l = x < c ind_r = x >= c X_l, X_r = x[ind_l], x[ind_r] @@ -1138,6 +1509,8 @@ def rdrobust_fit( # estimation entry point too, so manual-bandwidth fuzzy fits that # never touch bandwidth selection still fail closed. _fuzzy_identification_stop(T_l, T_r) + Z_l = covs[ind_l] if covs is not None else None # rdrobust.R:190-193 + Z_r = covs[ind_r] if covs is not None else None dups_l, dupsid_l = compute_dups_dupsid(X_l) dups_r, dupsid_r = compute_dups_dupsid(X_r) @@ -1145,6 +1518,7 @@ def _side( X: np.ndarray, Y: np.ndarray, T: Optional[np.ndarray], + Z: Optional[np.ndarray], h: float, b: float, dups: np.ndarray, @@ -1164,6 +1538,7 @@ def _side( eY = Y[ind] eX = X[ind] eT = T[ind] if T is not None else None # rdrobust.R:588-590 + eZ = Z[ind] if Z is not None else None # rdrobust.R:594-596 W_h = w_h[ind] W_b = w_b[ind] edups = dups[ind] @@ -1203,27 +1578,100 @@ def _side( M = (R_q @ invG_q) * W_b[:, None] Q_q = R_p * W_h[:, None] - h ** (p + 1) * np.outer(M[:, p + 1], L) # Point estimates (rdrobust.R:609-614). Fuzzy stacks T as the - # second response column (rdrobust.R:588-591); the sharp branch - # keeps the original vector products verbatim (bit-identity). - if eT is None: + # second response column (rdrobust.R:588-591), covariates stack + # after T (rdrobust.R:593-598); the sharp branch keeps the + # original vector products verbatim (bit-identity). + if eT is None and eZ is None: beta_p = invG_p @ (R_p * W_h[:, None]).T @ eY beta_bc = invG_p @ Q_q.T @ eY res_h = rdrobust_res_nn(eX, eY, nnmatch, edups, edupsid) + zblocks = None else: - eD = np.column_stack([eY, eT]) + resp = [eY] + ([eT] if eT is not None else []) + eD = np.column_stack(resp + ([eZ] if eZ is not None else [])) beta_p = invG_p @ (R_p * W_h[:, None]).T @ eD beta_bc = invG_p @ Q_q.T @ eD - # NN residual matrix, T sharing Y's neighbor sets - # (rdrobust.R:750-754; functions.R:171-174). - res_h = rdrobust_res_nn(eX, eY, nnmatch, edups, edupsid, t=eT) - return beta_p, beta_bc, invG_p, R_p * W_h[:, None], Q_q, res_h, N_h, N_b - - beta_p_l, beta_bc_l, invG_p_l, RX_cl_l, Q_q_l, res_l, N_h_l, N_b_l = _side( - X_l, Y_l, T_l, h_l, b_l, dups_l, dupsid_l, "left" - ) - beta_p_r, beta_bc_r, invG_p_r, RX_cl_r, Q_q_r, res_r, N_h_r, N_b_r = _side( - X_r, Y_r, T_r, h_r, b_r, dups_r, dupsid_r, "right" - ) + # NN residual matrix, T and Z sharing Y's neighbor sets + # (rdrobust.R:750-754; functions.R:171-180). + res_h = rdrobust_res_nn(eX, eY, nnmatch, edups, edupsid, t=eT, z=eZ) + zblocks = None + if eZ is not None: + # Per-side partialled normal-equation blocks + # (rdrobust.R:597, 659-667); summed across sides by the + # caller for the POOLED gamma. + dT_loc = 0 if eT is None else 1 + U_p = (R_p * W_h[:, None]).T @ eD + ZWD_p = (eZ * W_h[:, None]).T @ eD + colsZ = slice(1 + dT_loc, eD.shape[1]) + UiGU = U_p[:, colsZ].T @ (invG_p @ U_p) + zblocks = ( + ZWD_p[:, colsZ] - UiGU[:, colsZ], + ZWD_p[:, : 1 + dT_loc] - UiGU[:, : 1 + dT_loc], + np.diag(ZWD_p[:, colsZ]).copy(), + ) + return beta_p, beta_bc, invG_p, R_p * W_h[:, None], Q_q, res_h, N_h, N_b, zblocks + + ( + beta_p_l, + beta_bc_l, + invG_p_l, + RX_cl_l, + Q_q_l, + res_l, + N_h_l, + N_b_l, + zblocks_l, + ) = _side(X_l, Y_l, T_l, Z_l, h_l, b_l, dups_l, dupsid_l, "left") + ( + beta_p_r, + beta_bc_r, + invG_p_r, + RX_cl_r, + Q_q_r, + res_r, + N_h_r, + N_b_r, + zblocks_r, + ) = _side(X_r, Y_r, T_r, Z_r, h_r, b_r, dups_r, dupsid_r, "right") + + # ---- Pooled covariate projection gamma (rdrobust.R:659-671) ---- + gamma_p: Optional[np.ndarray] = None + covs_excluded: Optional[np.ndarray] = None + covs_set_degenerate = False + if covs is not None: + assert zblocks_l is not None and zblocks_r is not None + gamma_p, covs_excluded, covs_set_degenerate = _covs_gamma( + zblocks_l[0] + zblocks_r[0], + zblocks_l[1] + zblocks_r[1], + zblocks_l[2] + zblocks_r[2], + covs_drop, + ) + if warn_covs_degenerate and (covs_excluded.any() or covs_set_degenerate): + # Deviation from R (which silently inverts a noise singular + # value here, making the result platform-dependent); the + # estimator passes warn_covs_degenerate=False and re-warns + # with column names. + parts = [] + if covs_excluded.any(): + parts.append( + "covariate column(s) at index " + f"{np.flatnonzero(covs_excluded).tolist()} are " + "numerically collinear with the local polynomial " + "design (e.g. constant near the cutoff) and were " + "excluded from the adjustment" + ) + if covs_set_degenerate: + parts.append( + "the covariate set is numerically rank-deficient " + "after partialling (e.g. a full dummy set); a " + "stabilized pseudo-inverse cut was used - consider " + "dropping a reference category" + ) + warnings.warn( + "Degenerate covariate adjustment: " + "; ".join(parts) + ".", + UserWarning, + stacklevel=2, + ) # factorial(deriv) scaling per rdrobust.R:621-622 (deriv=0 -> 1). fact = float(math.factorial(deriv)) @@ -1233,7 +1681,7 @@ def _v(invG_p, RX, res, s): # fuzzy s collapses the residual matrix (functions.R:379-385). return invG_p @ rdrobust_vce(RX, res, s) @ invG_p - if t is None: + if t is None and covs is None: tau_cl = fact * float(beta_p_r[deriv] - beta_p_l[deriv]) tau_bc = fact * float(beta_bc_r[deriv] - beta_bc_l[deriv]) bias_l = fact * float(beta_p_l[deriv]) - fact * float(beta_bc_l[deriv]) @@ -1259,6 +1707,140 @@ def _v(invG_p, RX, res, s): N_b_r=N_b_r, ) + if t is None: + # ---- Covariate-adjusted sharp assembly (rdrobust.R:672-686, + # scalepar = 1). NOTE: R's covariate branch applies NO + # factorial(deriv) to the point estimates/biases (unlike the + # no-covariate branch at rdrobust.R:621-630) while the VARIANCES + # keep factorial^2 (rdrobust.R:796-797) - identical at the fixed + # deriv=0 surface; replicated verbatim, not "fixed". ---- + assert gamma_p is not None + s_Y = np.concatenate([[1.0], -gamma_p[:, 0]]) # rdrobust.R:672 + tau_cl = float(s_Y @ (beta_p_r[deriv, :] - beta_p_l[deriv, :])) + tau_bc = float(s_Y @ (beta_bc_r[deriv, :] - beta_bc_l[deriv, :])) + # Per-side adjusted taus -> biases (rdrobust.R:678-683). + bias_l = float(s_Y @ beta_p_l[deriv, :]) - float(s_Y @ beta_bc_l[deriv, :]) + bias_r = float(s_Y @ beta_p_r[deriv, :]) - float(s_Y @ beta_bc_r[deriv, :]) + # Adjusted per-side coefficient vectors (rdrobust.R:685-686). + beta_Y_p_l = s_Y @ beta_p_l.T + beta_Y_p_r = s_Y @ beta_p_r.T + V_tau_cl = fact**2 * float( + (_v(invG_p_l, RX_cl_l, res_l, s_Y) + _v(invG_p_r, RX_cl_r, res_r, s_Y))[deriv, deriv] + ) + V_tau_rb = fact**2 * float( + (_v(invG_p_l, Q_q_l, res_l, s_Y) + _v(invG_p_r, Q_q_r, res_r, s_Y))[deriv, deriv] + ) + return RdFitResult( + tau_cl=tau_cl, + tau_bc=tau_bc, + se_cl=float(np.sqrt(V_tau_cl)), + se_rb=float(np.sqrt(V_tau_rb)), + bias_l=bias_l, + bias_r=bias_r, + beta_p_l=beta_Y_p_l, + beta_p_r=beta_Y_p_r, + N_h_l=N_h_l, + N_h_r=N_h_r, + N_b_l=N_b_l, + N_b_r=N_b_r, + gamma_p=gamma_p, + covs_excluded=covs_excluded, + covs_set_degenerate=covs_set_degenerate, + ) + + if covs is not None: + # ---- Fuzzy + covariates assembly (rdrobust.R:688-723, 769-822; + # scalepar = 1): covariate-adjusted Y and T jumps via + # s_Y = [1, -gamma[,1]] / s_T = [1, -gamma[,2]], their ratio, + # the linearized bias correction, and the EXTENDED delta vector + # for the variance collapse. ---- + assert gamma_p is not None + s_Y0 = np.concatenate([[1.0], -gamma_p[:, 0]]) # rdrobust.R:672 + s_T0 = np.concatenate([[1.0], -gamma_p[:, 1]]) # rdrobust.R:689 + colsZ = slice(2, beta_p_l.shape[1]) + + def _adj(bmat: np.ndarray, srow: np.ndarray, col: int) -> float: + # rdrobust.R:691-704: s applied to [response col, covariate + # cols] of the (deriv+1) coefficient row. + return float(srow @ np.concatenate([[bmat[deriv, col]], bmat[deriv, colsZ]])) + + tau_Y_cl = fact * float(_adj(beta_p_r, s_Y0, 0) - _adj(beta_p_l, s_Y0, 0)) + tau_Y_bc = fact * float(_adj(beta_bc_r, s_Y0, 0) - _adj(beta_bc_l, s_Y0, 0)) + tau_T_cl = fact * float(_adj(beta_p_r, s_T0, 1) - _adj(beta_p_l, s_T0, 1)) + tau_T_bc = fact * float(_adj(beta_bc_r, s_T0, 1) - _adj(beta_bc_l, s_T0, 1)) + with np.errstate(divide="ignore", invalid="ignore"): + tau_cl = float(np.float64(tau_Y_cl) / np.float64(tau_T_cl)) + inv_tT = float(np.float64(1.0) / np.float64(tau_T_cl)) + ratio2 = float(np.float64(tau_Y_cl) / np.float64(tau_T_cl) ** 2) + s_ratio = np.array([inv_tT, -ratio2]) # rdrobust.R:716 + # Extended variance delta vector (rdrobust.R:722). + s_V = np.concatenate([s_ratio, -inv_tT * gamma_p[:, 0] + ratio2 * gamma_p[:, 1]]) + B_F = np.array([tau_Y_cl - tau_Y_bc, tau_T_cl - tau_T_bc]) # :712 + tau_bc = float(tau_cl - s_ratio @ B_F) # rdrobust.R:717 + sV_T = np.concatenate([[0.0, 1.0], -gamma_p[:, 1]]) # rdrobust.R:690 + # Per-side linearized biases from the ADJUSTED per-side taus + # (rdrobust.R:696-704, 713-720). + B_F_l = np.array( + [ + fact * (_adj(beta_p_l, s_Y0, 0) - _adj(beta_bc_l, s_Y0, 0)), + fact * (_adj(beta_p_l, s_T0, 1) - _adj(beta_bc_l, s_T0, 1)), + ] + ) + B_F_r = np.array( + [ + fact * (_adj(beta_p_r, s_Y0, 0) - _adj(beta_bc_r, s_Y0, 0)), + fact * (_adj(beta_p_r, s_T0, 1) - _adj(beta_bc_r, s_T0, 1)), + ] + ) + bias_l = float(s_ratio @ B_F_l) + bias_r = float(s_ratio @ B_F_r) + # Adjusted per-side coefficient vectors (rdrobust.R:706-709; the + # fuzzy-covariate branch DOES carry factorial(deriv), unlike the + # sharp-covariate one - deriv=0 either way). + + def _adj_vec(bmat: np.ndarray, srow: np.ndarray, col: int) -> np.ndarray: + return fact * (srow @ np.vstack([bmat[:, col][None, :], bmat[:, colsZ].T])) + + beta_Y_p_l = _adj_vec(beta_p_l, s_Y0, 0) + beta_Y_p_r = _adj_vec(beta_p_r, s_Y0, 0) + beta_T_p_l = _adj_vec(beta_p_l, s_T0, 1) + beta_T_p_r = _adj_vec(beta_p_r, s_T0, 1) + V_tau_cl = fact**2 * float( + (_v(invG_p_l, RX_cl_l, res_l, s_V) + _v(invG_p_r, RX_cl_r, res_r, s_V))[deriv, deriv] + ) + V_tau_rb = fact**2 * float( + (_v(invG_p_l, Q_q_l, res_l, s_V) + _v(invG_p_r, Q_q_r, res_r, s_V))[deriv, deriv] + ) + V_T_cl = fact**2 * float( + (_v(invG_p_l, RX_cl_l, res_l, sV_T) + _v(invG_p_r, RX_cl_r, res_r, sV_T))[deriv, deriv] + ) + V_T_rb = fact**2 * float( + (_v(invG_p_l, Q_q_l, res_l, sV_T) + _v(invG_p_r, Q_q_r, res_r, sV_T))[deriv, deriv] + ) + return RdFitResult( + tau_cl=tau_cl, + tau_bc=tau_bc, + se_cl=float(np.sqrt(V_tau_cl)), + se_rb=float(np.sqrt(V_tau_rb)), + bias_l=bias_l, + bias_r=bias_r, + beta_p_l=beta_Y_p_l, + beta_p_r=beta_Y_p_r, + N_h_l=N_h_l, + N_h_r=N_h_r, + N_b_l=N_b_l, + N_b_r=N_b_r, + tau_T_cl=tau_T_cl, + tau_T_bc=tau_T_bc, + se_T_cl=float(np.sqrt(V_T_cl)), + se_T_rb=float(np.sqrt(V_T_rb)), + beta_t_p_l=beta_T_p_l, + beta_t_p_r=beta_T_p_r, + gamma_p=gamma_p, + covs_excluded=covs_excluded, + covs_set_degenerate=covs_set_degenerate, + ) + # ---- Fuzzy assembly (rdrobust.R:636-657, 769-822; scalepar = 1) ---- tau_Y_cl = fact * float(beta_p_r[deriv, 0] - beta_p_l[deriv, 0]) tau_Y_bc = fact * float(beta_bc_r[deriv, 0] - beta_bc_l[deriv, 0]) diff --git a/diff_diff/guides/llms-autonomous.txt b/diff_diff/guides/llms-autonomous.txt index 5fc888854..ddd05e4f4 100644 --- a/diff_diff/guides/llms-autonomous.txt +++ b/diff_diff/guides/llms-autonomous.txt @@ -355,7 +355,7 @@ supported / out of scope; `warn` supported but with documented caveats; | `StaggeredTripleDifference` | ✓ | ✓ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ | ✓ | | `ContinuousDiD` | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✓ | | `HeterogeneousAdoptionDiD` | ✗ | partial | partial | ✗ | ✗ | ✗ | ✗ | ✓ | warn | -| `RegressionDiscontinuity` | ✗ (cross-sectional; sharp: treatment = running >= cutoff; fuzzy: observed take-up via `treatment_col=`) | ✗ | ✗ | ✗ | ✗ | ✗ (follow-up) | ✗ | ✗ | ✗ (follow-up) | +| `RegressionDiscontinuity` | ✗ (cross-sectional; sharp: treatment = running >= cutoff; fuzzy: observed take-up via `treatment_col=`) | ✗ | ✗ | ✗ | ✗ | ✓ (precision only - estimand unchanged, unlike the DiD conditional-PT role; `covariates=`) | ✗ | ✗ | ✗ (follow-up) | **Footnotes.** - `TwoWayFixedEffects` + staggered: fits but mixes positive and negative @@ -1299,13 +1299,15 @@ This guide does **not**: - Cover methods outside diff-diff's estimator suite (e.g., instrumental variables, regression KINK designs, synthetic control for a single treated unit). When those apply, point the user at dedicated - libraries. Regression discontinuity IS in scope - BOTH sharp and - fuzzy: route running-variable/threshold designs to + libraries. Regression discontinuity IS in scope - sharp, fuzzy, AND + covariate-adjusted: route running-variable/threshold designs to `RegressionDiscontinuity` (alias `RDD`); imperfect compliance at the threshold is the fuzzy design (`fit(..., treatment_col=...)`, local - Wald ratio with a first-stage block). Covariate adjustment, - cluster-robust RD variance, and weak-IV-robust fuzzy inference are - documented follow-ups, so point users needing those at R rdrobust. + Wald ratio with a first-stage block); precision covariates go in via + `fit(..., covariates=[...])` (same estimand - check covariate balance + first by fitting each covariate as the outcome). Cluster-robust RD + variance and weak-IV-robust fuzzy inference are documented follow-ups, + so point users needing those at R rdrobust. **If in doubt, consult the primary references in §8 and use `get_llm_guide("practitioner")` for the Baker et al. workflow.** diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index cba25797d..93fefd88b 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -826,7 +826,7 @@ es = est.fit(data_mp, outcome_col='y', unit_col='unit', ### RegressionDiscontinuity -Regression discontinuity estimator - sharp and fuzzy (Calonico, Cattaneo & Titiunik 2014), parity-targeting R rdrobust 4.0.0. SHARP (default): treatment is assigned by a known threshold of an observed running variable (`running >= cutoff`; units exactly at the cutoff are treated); no treatment column. FUZZY: pass the OBSERVED take-up column via `fit(..., treatment_col=...)` (R's `fuzzy=`) - the estimand becomes the local Wald ratio (complier LATE at the cutoff for BINARY take-up under monotonicity; ratio-of-jumps otherwise - the `estimand` field says which) with a linearized bias correction, and the results gain a full `first_stage*` three-row block. Point estimation via kernel-weighted local polynomials on each side; data-driven MSE/CER-optimal bandwidths (all 10 rdrobust selectors; fuzzy selects on the ratio objective by default, with a sharp-on-Y switch under one-sided perfect compliance or `sharpbw=True`); robust bias-corrected inference. Cross-sectional - no panel/time dimension. +Regression discontinuity estimator - sharp and fuzzy, with optional covariate adjustment (Calonico, Cattaneo & Titiunik 2014; covariates per Calonico, Cattaneo, Farrell & Titiunik 2019), parity-targeting R rdrobust 4.0.0. SHARP (default): treatment is assigned by a known threshold of an observed running variable (`running >= cutoff`; units exactly at the cutoff are treated); no treatment column. FUZZY: pass the OBSERVED take-up column via `fit(..., treatment_col=...)` (R's `fuzzy=`) - the estimand becomes the local Wald ratio (complier LATE at the cutoff for BINARY take-up under monotonicity; ratio-of-jumps otherwise - the `estimand` field says which) with a linearized bias correction, and the results gain a full `first_stage*` three-row block. Point estimation via kernel-weighted local polynomials on each side; data-driven MSE/CER-optimal bandwidths (all 10 rdrobust selectors; fuzzy selects on the ratio objective by default, with a sharp-on-Y switch under one-sided perfect compliance or `sharpbw=True`); robust bias-corrected inference. COVARIATE ADJUSTMENT: pass `fit(..., covariates=[...])` (R's `covs=`) for the CCFT 2019 additive common-coefficient adjustment - the estimand is UNCHANGED (precision only, unlike the DiD estimators' conditional-parallel-trends role); requires covariate BALANCE at the cutoff (testable: fit each covariate as the outcome and inspect its RD p-value); bandwidths are covariate-aware. Cross-sectional - no panel/time dimension. ```python RegressionDiscontinuity( @@ -845,6 +845,7 @@ RegressionDiscontinuity( bwrestrict: bool = True, # Clamp bandwidths to the observed running-variable range scaleregul: float = 1.0, # IK-style regularization scale (0 removes) sharpbw: bool = False, # Fuzzy only: select bandwidths on the sharp reduced form (R's sharpbw); auto under one-sided perfect compliance + covs_drop: bool = True, # Covariate fits only: drop collinear covariates with a warning naming them (R default); False = strict error alpha: float = 0.05, # rdrobust level = 100*(1-alpha) ) ``` @@ -859,6 +860,7 @@ rd.fit( outcome_col: str, running_col: str, treatment_col: str | None = None, # None = sharp; a column name = fuzzy (observed take-up; any numeric, typically binary) + covariates: list[str] | None = None, # Pre-determined covariate columns (R's covs=); additive common-coefficient adjustment, SAME estimand ) -> RegressionDiscontinuityResults ``` @@ -877,6 +879,10 @@ fuzzy = rd.fit(df, "y", "score", treatment_col="takeup") # fuzzy RD fuzzy.att # linearized bias-corrected local Wald ratio, robust row (complier LATE for binary take-up) fuzzy.first_stage # take-up jump (bias-corrected; full three-row first_stage* mirror available) fuzzy.estimand # "fuzzy (LATE for compliers at the cutoff)" (binary take-up) / "fuzzy (local Wald ratio at the cutoff; non-binary take-up)" / "sharp (ATE at the cutoff)" + +adj = rd.fit(df, "y", "score", covariates=["age", "income"]) # covariate-adjusted (estimand unchanged, shorter CIs) +adj.covariate_coefficients # {"age": ..., "income": ...} - nuisance projection gammas, NOT causal effects +balance = rd.fit(df, "age", "score") # balance placebo: covariate as outcome; small p-value = imbalance, do not adjust ``` Canonical fields are ONE coherent row (the robust row): att = bias-corrected @@ -884,11 +890,15 @@ estimate, se = robust SE. rdrobust prints the conventional estimate as its headline - that is att_conventional here, with a full inference row of its own. Fuzzy fits warn when the first-stage robust CI contains zero (weak identification; R is silent) and raise R's exact error when the take-up -variable has no variation and no jump. Covariates, cluster-robust variance, -weights, kink estimands, and weak-IV-robust fuzzy inference are documented -follow-ups; missing rows are dropped WITH a warning (R drops silently); -N < 20 falls back to full-range bandwidths exactly as rdrobust does -(overriding manual h). +variable has no variation and no jump. Covariate-adjusted fits drop collinear +covariates with a warning naming them (covs_drop=True, R's default; False = +strict error) and guard degenerate adjustments (constant covariates are +excluded, full dummy sets take a stabilized cut - both warned; R silently +returns platform-dependent noise there). Cluster-robust variance, weights, +kink estimands, weak-IV-robust fuzzy inference, and a packaged +covariate-balance helper are documented follow-ups; missing rows are dropped +WITH a warning (R drops silently); N < 20 falls back to full-range bandwidths +exactly as rdrobust does (overriding manual h). ### StackedDiD diff --git a/diff_diff/guides/llms.txt b/diff_diff/guides/llms.txt index c28910269..2bea9ce6d 100644 --- a/diff_diff/guides/llms.txt +++ b/diff_diff/guides/llms.txt @@ -64,7 +64,7 @@ Full practitioner guide: call `diff_diff.get_llm_guide("practitioner")` - [TripleDifference](https://diff-diff.readthedocs.io/en/stable/api/triple_diff.html): Triple difference (DDD) estimator for designs requiring two criteria for treatment eligibility - [ContinuousDiD](https://diff-diff.readthedocs.io/en/stable/api/continuous_did.html): Callaway, Goodman-Bacon & Sant'Anna (2024) continuous treatment DiD with dose-response curves - [HeterogeneousAdoptionDiD](https://diff-diff.readthedocs.io/en/stable/api/had.html): de Chaisemartin, Ciccia, D'Haultfœuille & Knau (2026) for designs where **no unit remains untreated**; local-linear estimator at the dose support boundary returning Weighted Average Slope (WAS) on Design 1' (`d̲=0` / QUG) or `WAS_{d̲}` on Design 1 (`d̲>0`, continuous-near-d̲ or mass-point), with multi-period event-study extension (last-treatment cohort, pointwise CIs). **Panel-only** in this release (repeated cross-sections rejected by the validator). Alias `HAD`. -- [RegressionDiscontinuity](https://diff-diff.readthedocs.io/en/stable/api/regression_discontinuity.html): Calonico, Cattaneo & Titiunik (2014) sharp AND fuzzy regression discontinuity with robust bias-corrected inference, parity-targeting R rdrobust 4.0.0 (all 10 data-driven bandwidth selectors, mass-point handling, three-row conventional/bias-corrected/robust output; canonical `att` = the bias-corrected estimate with a coherent robust CI - rdrobust's printed headline is `att_conventional`). Fuzzy via `fit(..., treatment_col=...)`: local Wald ratio (complier LATE for binary take-up under monotonicity; ratio-of-jumps otherwise - the `estimand` field says which), first-stage `first_stage*` block, weak-first-stage warning. Covariates/cluster are documented follow-ups. Alias `RDD`. +- [RegressionDiscontinuity](https://diff-diff.readthedocs.io/en/stable/api/regression_discontinuity.html): Calonico, Cattaneo & Titiunik (2014) sharp AND fuzzy regression discontinuity with robust bias-corrected inference, parity-targeting R rdrobust 4.0.0 (all 10 data-driven bandwidth selectors, mass-point handling, three-row conventional/bias-corrected/robust output; canonical `att` = the bias-corrected estimate with a coherent robust CI - rdrobust's printed headline is `att_conventional`). Fuzzy via `fit(..., treatment_col=...)`: local Wald ratio (complier LATE for binary take-up under monotonicity; ratio-of-jumps otherwise - the `estimand` field says which), first-stage `first_stage*` block, weak-first-stage warning. Covariate adjustment via `fit(..., covariates=[...])` (CCFT 2019 additive common-coefficient, R's `covs=`): SAME estimand, precision only; requires covariate balance at the cutoff (testable: fit each covariate as the outcome); covariate-aware bandwidths; collinear columns dropped with a warning (`covs_drop`). Cluster-robust variance is a documented follow-up. Alias `RDD`. - [StackedDiD](https://diff-diff.readthedocs.io/en/stable/api/stacked_did.html): Wing, Freedman & Hollingsworth (2024) stacked DiD with Q-weights and sub-experiments; optional covariate balancing (`balance="entropy"`, Ustyuzhanin 2026) - [EfficientDiD](https://diff-diff.readthedocs.io/en/stable/api/efficient_did.html): Chen, Sant'Anna & Xie (2025) efficient DiD with optimal weighting for tighter SEs - [TROP](https://diff-diff.readthedocs.io/en/stable/api/trop.html): Triply Robust Panel estimator (Athey et al. 2025) with nuclear norm factor adjustment (absorbing by default; `non_absorbing=True` for on/off treatment, method='local') diff --git a/diff_diff/rdd.py b/diff_diff/rdd.py index 374e54a81..f6dcef0bf 100644 --- a/diff_diff/rdd.py +++ b/diff_diff/rdd.py @@ -1,6 +1,7 @@ """ -Regression discontinuity design (RDD) estimation - sharp and fuzzy - with -robust bias-corrected inference, parity-targeting R ``rdrobust`` 4.0.0. +Regression discontinuity design (RDD) estimation - sharp and fuzzy, with +optional covariate adjustment - and robust bias-corrected inference, +parity-targeting R ``rdrobust`` 4.0.0. Implements the local-polynomial RD estimators of Calonico, Cattaneo & Titiunik (2014). SHARP (default): treatment is assigned by @@ -18,6 +19,30 @@ linearization of the ratio (not per-component), matching CCT 2014 Section 3.2 and rdrobust exactly. +Covariate adjustment (``fit(..., covariates=[...])``; Calonico, Cattaneo, +Farrell & Titiunik 2019, R's ``covs=``): covariates enter ADDITIVELY with +a common coefficient pooled across sides (CCFT 2019 Equation 2 - the only +specification with a clean guarantee; treatment-interacted and demeaned +variants are documented as inconsistent-or-inferior there). UNLIKE the +library's DiD estimators, where ``covariates`` switches identification to +conditional parallel trends, RD covariates DO NOT change the estimand - +the ``att`` still measures the same cutoff jump/ratio and the +``estimand`` label is unchanged; adjustment buys precision (shorter CIs) +when covariates predict the outcome near the cutoff. The operative +requirement is covariate BALANCE at the cutoff (zero RD effect on each +covariate); imbalanced covariates make the adjusted estimator +inconsistent, and adjusting "for" imbalance cannot restore +identification. Balance is testable with the estimator itself:: + + balance = RegressionDiscontinuity().fit(df, outcome_col="z1", + running_col="x") + balance.p_value # small p = imbalance; do not adjust for z1 + +Bandwidths are covariate-AWARE (covariates propagate into selection, not +just estimation, as in R). Collinear covariates are dropped with a +warning under ``covs_drop=True`` (R's default; the warning names the +dropped columns). + Canonical inference binding --------------------------- ``RegressionDiscontinuityResults`` binds the library-canonical fields to ONE @@ -52,12 +77,14 @@ ``nnmatch`` ``nnmatch`` ``treatment_col`` (fit) ``fuzzy`` (observed take-up variable) ``sharpbw`` ``sharpbw`` (same default and semantics) +``covariates`` (fit) ``covs`` (column names instead of a matrix) +``covs_drop`` ``covs_drop`` (same default and semantics) ======================= ========================================== -Not in v1 (documented seams, see REGISTRY.md): covariate adjustment, -cluster-robust variance, weights, ``deriv``/kink estimands, ``scalepar``, -``stdvars``, hc0-hc3 variance modes, weak-IV-robust fuzzy inference -(Feir-Lemieux-Marmer). +Not in v1 (documented seams, see REGISTRY.md): cluster-robust variance, +weights, ``deriv``/kink estimands, ``scalepar``, ``stdvars``, hc0-hc3 +variance modes, weak-IV-robust fuzzy inference (Feir-Lemieux-Marmer), +and a packaged covariate-balance helper (the recipe above covers it). References ---------- @@ -70,13 +97,16 @@ - Calonico, S., Cattaneo, M. D., & Farrell, M. H. (2018). On the Effect of Bias Estimation on Coverage Accuracy in Nonparametric Inference. *JASA*, 113(522), 767-779. +- Calonico, S., Cattaneo, M. D., Farrell, M. H., & Titiunik, R. (2019). + Regression Discontinuity Designs Using Covariates. *Review of Economics + and Statistics*, 101(3), 442-451. """ from __future__ import annotations import warnings from dataclasses import dataclass, field -from typing import Any, Dict, Optional, Tuple +from typing import Any, Dict, List, Optional, Tuple import numpy as np import pandas as pd @@ -85,10 +115,11 @@ BWSELECT_OPTIONS, _fuzzy_identification_stop, _normalize_kernel, + covs_drop_fun, rdbwselect, rdrobust_fit, ) -from diff_diff.utils import safe_inference +from diff_diff.utils import safe_inference, validate_covariate_names __all__ = [ "RegressionDiscontinuity", @@ -194,11 +225,14 @@ class RegressionDiscontinuityResults: # (the complier-LATE reading does not apply to dose take-up). # ``treatment_col`` is the fit-time take-up column name # (None on sharp fits; no ``_input`` suffix - that convention is - # reserved for constructor arguments); ``sharpbw`` echoes the - # constructor flag. + # reserved for constructor arguments); ``sharpbw`` and ``covs_drop`` + # echo the constructor flags. The estimand label deliberately does NOT + # change under covariate adjustment: CCFT 2019 covariates target the + # SAME estimand (precision only) - see ``covariates`` below. estimand: str sharpbw: bool treatment_col: Optional[str] + covs_drop: bool # First-stage (take-up jump) three-row mirror - fuzzy fits only, all # None on sharp fits. Same binding rule as the main estimate: the @@ -220,9 +254,26 @@ class RegressionDiscontinuityResults: first_stage_p_value_bias_corrected: Optional[float] = None first_stage_conf_int_bias_corrected: Optional[Tuple[float, float]] = None + # Covariate adjustment (CCFT 2019) - all None on unadjusted fits. + # ``covariates`` echoes the fit-time column names AS PASSED; + # ``covariates_dropped`` lists columns removed as collinear by + # covs_drop ([] when nothing was dropped); ``covariate_coefficients`` + # maps each RETAINED covariate name to its common (pooled across + # sides) outcome-equation projection coefficient gamma - these are + # nuisance coefficients for the adjustment, NOT causal effects of the + # covariates. Fuzzy fits add ``first_stage_covariate_coefficients`` + # (the take-up-equation gamma). Name-keyed dicts make R's internal + # name-length column sort invisible to users. + covariates: Optional[List[str]] = None + covariates_dropped: Optional[List[str]] = None + covariate_coefficients: Optional[Dict[str, float]] = None + first_stage_covariate_coefficients: Optional[Dict[str, float]] = None + # Per-side order-p coefficient vectors (rdplot seam); the outcome pair # is always populated by fit(), so typed non-Optional despite the - # dataclass default; the take-up pair is fuzzy-only. + # dataclass default; the take-up pair is fuzzy-only. On + # covariate-adjusted fits these are the ADJUSTED vectors (gamma + # combination applied), matching R's beta_Y_p_* / beta_T_p_*. beta_p_left: np.ndarray = field(repr=False, default=None) beta_p_right: np.ndarray = field(repr=False, default=None) beta_t_p_left: Optional[np.ndarray] = field(repr=False, default=None) @@ -235,10 +286,20 @@ def summary(self) -> str: lines = [] lines.append("=" * width) design = "Fuzzy" if self.first_stage is not None else "Sharp" + if self.covariates: + # Mirrors R's rdmodel string ("Covariate-adjusted ... RD + # estimates"); the estimand line below is deliberately + # UNCHANGED - covariates buy precision, not a new estimand. + design = f"Covariate-adjusted {design}" lines.append(f"{design} Regression Discontinuity (rdrobust parity)".center(width)) lines.append("=" * width) lines.append(f"Cutoff: {self.cutoff:g}") lines.append(f"Estimand: {self.estimand}") + if self.covariates: + cov_line = f"Covariates ({len(self.covariates)}): " + ", ".join(self.covariates) + if self.covariates_dropped: + cov_line += " [dropped: " + ", ".join(self.covariates_dropped) + "]" + lines.append(cov_line) lines.append(f"Kernel: {self.kernel:<14} Bandwidth selector: {self.bwselect}") lines.append( f"Order (p, q): ({self.p}, {self.q}) VCE: {self.vcov_type} " @@ -394,6 +455,13 @@ def to_dict(self) -> Dict[str, Any]: "estimand": self.estimand, "sharpbw": self.sharpbw, "treatment_col": self.treatment_col, + "covs_drop": self.covs_drop, + # List/dict-valued covariate echoes (None on unadjusted fits; + # the lpdid/continuous_did echo convention). + "covariates": self.covariates, + "covariates_dropped": self.covariates_dropped, + "covariate_coefficients": self.covariate_coefficients, + "first_stage_covariate_coefficients": self.first_stage_covariate_coefficients, "first_stage": self.first_stage, "first_stage_se": self.first_stage_se, "first_stage_t_stat": self.first_stage_t_stat, @@ -420,8 +488,8 @@ def to_dataframe(self) -> pd.DataFrame: class RegressionDiscontinuity: - """Regression discontinuity estimator, sharp and fuzzy (rdrobust - 4.0.0 parity). + """Regression discontinuity estimator - sharp and fuzzy, with + optional covariate adjustment (rdrobust 4.0.0 parity). SHARP (default): treatment is defined by the running variable crossing a known cutoff (``running >= cutoff`` treated, matching rdrobust: @@ -429,13 +497,19 @@ class RegressionDiscontinuity: take-up column via ``fit(..., treatment_col=...)`` - the estimand becomes the local Wald ratio (complier LATE at the cutoff for binary take-up under monotonicity; the ``estimand`` results field says which - reading applies) and the results gain a first-stage block. Point + reading applies) and the results gain a first-stage block. + COVARIATE ADJUSTMENT: pass ``fit(..., covariates=[...])`` (R's + ``covs=``) for the CCFT 2019 additive common-coefficient adjustment - + the estimand is UNCHANGED (precision only; requires covariate balance + at the cutoff, see the module docstring), bandwidths become + covariate-aware, and collinear columns are dropped with a warning + under ``covs_drop=True``. Point estimation uses kernel-weighted local polynomials of order ``p`` on each side; inference is robust bias-corrected per Calonico, Cattaneo & Titiunik (2014). Defaults reproduce ``rdrobust(y, x)`` / - ``rdrobust(y, x, fuzzy=t)``: ``p=1``, ``q=2``, triangular kernel, - ``bwselect="mserd"``, nearest-neighbor variance with 3 matches, - ``masspoints="adjust"``. + ``rdrobust(y, x, fuzzy=t)`` / ``rdrobust(y, x, covs=Z)``: ``p=1``, + ``q=2``, triangular kernel, ``bwselect="mserd"``, nearest-neighbor + variance with 3 matches, ``masspoints="adjust"``, ``covs_drop=True``. Parameters ---------- @@ -489,7 +563,17 @@ class RegressionDiscontinuity: this flag - under one-sided perfect compliance (zero take-up variance on either side), exactly as in R. On sharp fits the flag has no effect and a warning is emitted (R ignores it silently - - documented deviation). + documented deviation). Never drops covariates from selection - + with ``covariates`` it selects on the covariate-adjusted sharp + objective, as in R. + covs_drop : bool, default True + Covariate-adjusted fits only (``fit(..., covariates=[...])``): + when True (R's default), redundant (collinear) covariate columns + are dropped with a warning naming them before fitting, and the + covariate projection uses a pseudo-inverse; when False the solve + is strict and collinear covariates raise a clear error. Without + ``covariates`` the flag has no effect and setting it to False + emits a warning (same pattern as ``sharpbw`` on sharp fits). alpha : float, default 0.05 Significance level (rdrobust ``level = 100*(1-alpha)``). @@ -519,6 +603,7 @@ def __init__( bwrestrict: bool = True, scaleregul: float = 1.0, sharpbw: bool = False, + covs_drop: bool = True, alpha: float = 0.05, ): self.cutoff = cutoff @@ -536,6 +621,7 @@ def __init__( self.bwrestrict = bwrestrict self.scaleregul = scaleregul self.sharpbw = sharpbw + self.covs_drop = covs_drop self.alpha = alpha self._validate_constructor_args() @@ -595,6 +681,8 @@ def _validate_constructor_args(self) -> None: raise ValueError(f"bwrestrict must be a bool; got {self.bwrestrict!r}.") if not isinstance(self.sharpbw, (bool, np.bool_)): raise ValueError(f"sharpbw must be a bool; got {self.sharpbw!r}.") + if not isinstance(self.covs_drop, (bool, np.bool_)): + raise ValueError(f"covs_drop must be a bool; got {self.covs_drop!r}.") if not ( self._is_real_scalar(self.scaleregul) and np.isfinite(self.scaleregul) @@ -623,6 +711,7 @@ def get_params(self, deep: bool = True) -> Dict[str, Any]: "bwrestrict": self.bwrestrict, "scaleregul": self.scaleregul, "sharpbw": self.sharpbw, + "covs_drop": self.covs_drop, "alpha": self.alpha, } @@ -648,8 +737,10 @@ def fit( outcome_col: str, running_col: str, treatment_col: Optional[str] = None, + covariates: Optional[List[str]] = None, ) -> RegressionDiscontinuityResults: - """Estimate the RD effect at the cutoff (sharp or fuzzy). + """Estimate the RD effect at the cutoff (sharp or fuzzy, optionally + covariate-adjusted). Parameters ---------- @@ -672,10 +763,45 @@ def fit( not apply there. A take-up column that is deterministic in the running variable reproduces the sharp fit exactly (first stage == 1). + covariates : list of str or None, default None + Column names of pre-determined covariates for the additive + common-coefficient adjustment of CCFT (2019) (R's ``covs=``). + The estimand is UNCHANGED - unlike the DiD estimators' + conditional-parallel-trends role, RD covariates buy precision + only, and require covariate BALANCE at the cutoff (zero RD + effect on each covariate; testable by fitting each covariate + as the outcome - imbalanced covariates make the adjusted + estimator inconsistent). Continuous, discrete, or mixed + columns are accepted; covariates propagate into bandwidth + selection (covariate-aware, as in R). Collinear columns are + dropped with a warning under ``covs_drop=True``; see the + ``covariates*`` results fields for the echo and the fitted + projection coefficients. """ cols = [outcome_col, running_col] if treatment_col is not None: cols.append(treatment_col) + if covariates is not None: + if isinstance(covariates, str): + # A bare string would iterate characters; fail closed. + raise ValueError(f"covariates must be a list of column names; got {covariates!r}.") + # Materialize BEFORE validating: a generator would be consumed + # by the all() check and then silently collapse to an empty + # list (disabling adjustment without a whisper). + covariates = list(covariates) + if not all(isinstance(name, str) for name in covariates): + raise ValueError(f"covariates must be a list of column names; got {covariates!r}.") + if not covariates: + covariates = None # empty list == no adjustment + if covariates is not None: + # Duplicate names and collisions with the fit's structural + # columns corrupt the name-keyed coefficient dict. + validate_covariate_names( + covariates, + cols, + estimator="RegressionDiscontinuity", + ) + cols.extend(covariates) for col in cols: if col not in data.columns: raise ValueError(f"Column {col!r} not found in data.") @@ -688,6 +814,14 @@ def fit( UserWarning, stacklevel=2, ) + if not self.covs_drop and covariates is None: + # Same pattern as sharpbw-on-sharp: a non-default knob that + # cannot apply must not pass silently. + warnings.warn( + "covs_drop=False has no effect without covariates and is ignored.", + UserWarning, + stacklevel=2, + ) y_raw = np.asarray(pd.to_numeric(data[outcome_col], errors="coerce"), dtype=np.float64) x_raw = np.asarray(pd.to_numeric(data[running_col], errors="coerce"), dtype=np.float64) ok = np.isfinite(y_raw) & np.isfinite(x_raw) @@ -699,12 +833,26 @@ def fit( pd.to_numeric(data[treatment_col], errors="coerce"), dtype=np.float64 ) ok = ok & np.isfinite(t_raw) + z_raw: Optional[np.ndarray] = None + if covariates is not None: + # R's complete.cases filter includes the covariate columns + # (rdrobust.R:80-84) - the joint drop must too. Column order + # here is AS PASSED; the R name-length sort applies below. + z_raw = np.column_stack( + [ + np.asarray(pd.to_numeric(data[name], errors="coerce"), dtype=np.float64) + for name in covariates + ] + ) + ok = ok & np.all(np.isfinite(z_raw), axis=1) n_dropped = int(y_raw.shape[0] - np.sum(ok)) if n_dropped > 0: # Deviation from R (which drops silently via complete.cases): dropped_cols = f"{outcome_col!r}/{running_col!r}" if fuzzy_fit: dropped_cols += f"/{treatment_col!r}" + if covariates is not None: + dropped_cols += "/covariates" warnings.warn( f"Dropping {n_dropped} row(s) with missing or non-numeric " f"values in {dropped_cols}.", @@ -714,6 +862,7 @@ def fit( y = y_raw[ok] x = x_raw[ok] t = t_raw[ok] if t_raw is not None else None + z = z_raw[ok] if z_raw is not None else None N = int(y.shape[0]) if N == 0: raise ValueError("No complete-case observations to fit on.") @@ -727,6 +876,57 @@ def fit( q = int(self.q) if self.q is not None else p + 1 kernel = _normalize_kernel(self.kernel) + # --- Covariate column sort + redundant-column drop (hoisted from + # rdrobust.R:121-140, like the fuzzy identification hoist below; + # R's order: NaN drop -> covs_drop -> fuzzy stop -> mass points). + # Under covs_drop=True R first sorts columns by NAME LENGTH + # (order(nchar), stable - rdrobust.R:131); the sort decides which + # of a collinear set survives, and all user-facing surfaces are + # name-keyed so the internal order never leaks. The QR runs on + # x-SORTED rows - the row order R (and the port entry points) use + # - so near-threshold rank decisions cannot diverge from the + # downstream calls. Passing the already-reduced matrix down means + # the port's own entry-point drop finds full rank and stays + # silent (no double warning). + model_covariates: Optional[List[str]] = None + covariates_dropped: Optional[List[str]] = None + if covariates is not None: + assert z is not None + model_covariates = list(covariates) + covariates_dropped = [] + if self.covs_drop: + model_covariates = sorted(model_covariates, key=len) + z = np.column_stack([z[:, covariates.index(name)] for name in model_covariates]) + keep_idx, rank = covs_drop_fun(z[np.argsort(x, kind="stable")]) + if rank == 0: + raise ValueError( + "All covariates are numerically zero (rank-0 " + "covariate matrix); remove the covariates instead." + ) + if rank < len(model_covariates): + covariates_dropped = [ + name + for i, name in enumerate(model_covariates) + if i not in set(keep_idx.tolist()) + ] + # R's warning is a generic "Multicollinearity issue + # detected in covs." - naming the dropped columns is a + # documented enhancement. + warnings.warn( + "Multicollinearity detected in covariates: " + f"dropped redundant column(s) {covariates_dropped} " + "(covs_drop=True; set covs_drop=False for a strict " + "error instead).", + UserWarning, + stacklevel=2, + ) + model_covariates = [ + name + for i, name in enumerate(model_covariates) + if i in set(keep_idx.tolist()) + ] + z = z[:, keep_idx] + # --- Fuzzy identification check (rdrobust.R:164-185) --- # Hoisted to run immediately after the NaN drop and BEFORE # mass-point detection, matching R's rdrobust ordering exactly @@ -824,6 +1024,8 @@ def fit( warn_masspoints=False, # fit() already warned (rdrobust.R:365-380) fuzzy=t, sharpbw=bool(self.sharpbw), + covs=z, + covs_drop=bool(self.covs_drop), ) h_l, h_r, b_l, b_r = bw.bws[self.bwselect] n_unique_left = bw.M_l if self.masspoints != "off" else n_unique_left @@ -848,8 +1050,43 @@ def fit( vce=self.vcov_type, nnmatch=int(self.nnmatch), t=t, + covs=z, + covs_drop=bool(self.covs_drop), + # The estimator owns the degeneracy warning (with column + # names) - same plumbing pattern as warn_masspoints. + warn_covs_degenerate=False, ) + # --- Degenerate covariate adjustment warning (estimator-level, + # with column names; deviation from R, which silently inverts a + # noise singular value on these systems - see the port's + # _covs_gamma for the guard) --- + if model_covariates is not None and fit.covs_excluded is not None: + excluded_names = [ + name for name, flag in zip(model_covariates, fit.covs_excluded) if bool(flag) + ] + parts = [] + if excluded_names: + parts.append( + f"covariate(s) {excluded_names} are numerically " + "collinear with the local polynomial design (e.g. " + "constant near the cutoff) and were excluded from " + "the adjustment" + ) + if fit.covs_set_degenerate: + parts.append( + "the covariate set is numerically rank-deficient " + "after partialling (e.g. a full dummy set); a " + "stabilized pseudo-inverse cut was used - consider " + "dropping a reference category" + ) + if parts: + warnings.warn( + "Degenerate covariate adjustment: " + "; ".join(parts) + ".", + UserWarning, + stacklevel=2, + ) + # Estimand label: the complier-LATE reading requires BINARY # take-up (plus monotonicity); non-binary (dose) take-up - accepted, # matching R's fuzzy= - is the ratio-of-jumps estimand and must not @@ -962,6 +1199,19 @@ def fit( estimand=estimand, sharpbw=bool(self.sharpbw), treatment_col=treatment_col, + covs_drop=bool(self.covs_drop), + covariates=None if covariates is None else list(covariates), + covariates_dropped=covariates_dropped, + covariate_coefficients=( + None + if model_covariates is None or fit.gamma_p is None + else {name: float(fit.gamma_p[i, 0]) for i, name in enumerate(model_covariates)} + ), + first_stage_covariate_coefficients=( + None + if not fuzzy_fit or model_covariates is None or fit.gamma_p is None + else {name: float(fit.gamma_p[i, 1]) for i, name in enumerate(model_covariates)} + ), beta_p_left=fit.beta_p_l, beta_p_right=fit.beta_p_r, **fs, diff --git a/docs/api/regression_discontinuity.rst b/docs/api/regression_discontinuity.rst index 8b667d656..398e5eff0 100644 --- a/docs/api/regression_discontinuity.rst +++ b/docs/api/regression_discontinuity.rst @@ -1,8 +1,9 @@ Regression Discontinuity ======================== -Regression discontinuity estimation - sharp and fuzzy - with robust -bias-corrected inference, parity-targeting R ``rdrobust`` 4.0.0. +Regression discontinuity estimation - sharp and fuzzy, with optional +covariate adjustment - and robust bias-corrected inference, +parity-targeting R ``rdrobust`` 4.0.0. **Sharp** (default): treatment is assigned by a known threshold of an observed running variable (``running >= cutoff``; units exactly at the @@ -17,6 +18,14 @@ full ``first_stage*`` block. Both designs use kernel-weighted local polynomials on each side with data-driven MSE/CER-optimal bandwidths (all 10 rdrobust selectors) and robust bias-corrected inference per Calonico, Cattaneo & Titiunik (2014). +**Covariate adjustment** (``fit(..., covariates=[...])``; R's ``covs=``, +per Calonico, Cattaneo, Farrell & Titiunik 2019): additive +common-coefficient adjustment that leaves the estimand UNCHANGED - +unlike the DiD estimators' conditional-parallel-trends role, RD +covariates buy precision only, and require covariate balance at the +cutoff (testable by fitting each covariate as the outcome). Bandwidths +are covariate-aware; collinear covariates are dropped with a warning +naming them (``covs_drop=True``, R's default). .. note:: @@ -34,12 +43,12 @@ robust bias-corrected inference per Calonico, Cattaneo & Titiunik (2014). .. note:: - **Scope of this release.** Sharp and fuzzy designs with the - nearest-neighbor variance estimator (rdrobust's default). Covariate - adjustment, cluster-robust variance, weights, kink estimands, - weak-IV-robust fuzzy inference, and the rdplot/density-test - diagnostics are documented follow-ups - see the methodology registry - for the full deviations and seams list. + **Scope of this release.** Sharp, fuzzy, and covariate-adjusted + designs with the nearest-neighbor variance estimator (rdrobust's + default). Cluster-robust variance, weights, kink estimands, + weak-IV-robust fuzzy inference, a packaged covariate-balance helper, + and the rdplot/density-test diagnostics are documented follow-ups - + see the methodology registry for the full deviations and seams list. RegressionDiscontinuity ----------------------- diff --git a/docs/choosing_estimator.rst b/docs/choosing_estimator.rst index c57108d38..701a18134 100644 --- a/docs/choosing_estimator.rst +++ b/docs/choosing_estimator.rst @@ -706,7 +706,7 @@ differences helps interpret results and choose appropriate inference. - Two SE regimes per :doc:`api/had`. **Unweighted**: continuous-dose paths use the CCT-2014 robust SE from the in-house ``lprobust`` port; mass-point uses a 2SLS sandwich. **``survey_design=SurveyDesign(weights="col", ...)``** (the sole weighting entry as of the 3.7.0 ``survey=`` / ``weights=`` removal): both paths compose Binder (1983) Taylor-series linearization (``variance_formula="survey_binder_tsl"`` / ``"survey_binder_tsl_2sls"``); the mass-point survey path rejects ``vcov_type="classical"`` (requires ``hc1`` / ``robust=True``), and ``survey_design=`` + ``cluster=`` is rejected outright (route weighted clustering via ``SurveyDesign(weights=, psu=)``; a bare ``cluster=`` gives unweighted CR1). Per-horizon CIs are pointwise; sup-t bands available on the event-study path via ``cband=True`` whenever ``survey_design=`` or ``cluster=`` is supplied. * - ``RegressionDiscontinuity`` - Robust bias-corrected (CCT 2014, NN variance) - - Sharp and fuzzy RD with rdrobust-4.0.0-parity inference (fuzzy via ``fit(..., treatment_col=...)``: local Wald ratio with a linearized bias correction, first-stage block, and a weak-first-stage warning). Canonical ``att``/``se``/``conf_int`` are the ROBUST bias-corrected row (``att`` = bias-corrected estimate, CI centered on it); the conventional estimate rdrobust prints as its headline is ``att_conventional`` with its own inference row. Only ``vcov_type="nn"`` in this release; cluster-robust RD variance is a documented follow-up. + - Sharp, fuzzy, and covariate-adjusted RD with rdrobust-4.0.0-parity inference (fuzzy via ``fit(..., treatment_col=...)``: local Wald ratio with a linearized bias correction, first-stage block, and a weak-first-stage warning; covariates via ``fit(..., covariates=[...])``: same estimand, precision only, covariate-aware bandwidths). Canonical ``att``/``se``/``conf_int`` are the ROBUST bias-corrected row (``att`` = bias-corrected estimate, CI centered on it); the conventional estimate rdrobust prints as its headline is ``att_conventional`` with its own inference row. Only ``vcov_type="nn"`` in this release; cluster-robust RD variance is a documented follow-up. * - ``SunAbraham`` - Cluster-robust (unit level) - Clusters at unit level by default. Specify ``cluster`` to override. Use ``n_bootstrap`` for pairs bootstrap inference. diff --git a/docs/doc-deps.yaml b/docs/doc-deps.yaml index eca260c59..004d1af3b 100644 --- a/docs/doc-deps.yaml +++ b/docs/doc-deps.yaml @@ -419,7 +419,7 @@ sources: - path: diff_diff/guides/llms-autonomous.txt section: "Estimator-support matrix + out-of-scope list" type: user_guide - note: "Sharp AND fuzzy RD are IN scope; the out-of-scope bullet routes only kink designs / covariate-adjusted / cluster-robust RD elsewhere. Keep both in sync with the estimator's v1 seams." + note: "Sharp, fuzzy, AND covariate-adjusted RD are IN scope; the out-of-scope bullet routes only kink designs / cluster-robust RD elsewhere. Keep both in sync with the estimator's v1 seams." - path: docs/choosing_estimator.rst section: "SE methodology + Survey Design Support tables" type: user_guide diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 159568486..313b95785 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -3600,7 +3600,9 @@ Robust Nonparametric Confidence Intervals for Regression-Discontinuity Designs. *Econometrica*, 82(6), 2295-2326. https://doi.org/10.3982/ECTA11757. Software/parity reference: Calonico, Cattaneo, Farrell & Titiunik (2017), *Stata Journal* 17(2), 372-404 (rdrobust); CER-optimal bandwidth theory: -Calonico, Cattaneo & Farrell (2018), *JASA* 113(522), 767-779. +Calonico, Cattaneo & Farrell (2018), *JASA* 113(522), 767-779; covariate +adjustment: Calonico, Cattaneo, Farrell & Titiunik (2019), *REStat* +101(3), 442-451. https://doi.org/10.1162/rest_a_00760. **Estimand and estimator (sharp RD):** with running variable `X`, known cutoff `c`, and treatment `T = 1(X >= c)` (units exactly at the cutoff are @@ -3646,6 +3648,49 @@ above the treatment effects, as R does. A take-up column deterministic in `X` reproduces the sharp fit (first stage == 1, ULP-level agreement - the ratio divides by a float-solved 1). +**Covariate adjustment (`fit(..., covariates=[...])`; CCFT 2019, R's +`covs=`):** covariates enter ADDITIVELY with a COMMON coefficient pooled +across sides - CCFT 2019 Equation 2, the paper's recommended (and only +implemented) specification. Lemma 1 derives the probability limits of the +five candidate specifications: the treatment-interacted variant +(equivalent to separate per-side fits) needs the strictly stronger +condition `mu'_Z+ gamma_Y+ = mu'_Z- gamma_Y-` and the three +demeaning-based variants have slower rates and extra misspecification +bias - all four are deliberately NOT implemented. The ESTIMAND IS +UNCHANGED (`tau_SRD`/`tau_FRD` as above): adjustment buys precision, and +the `estimand` label deliberately does not change (unlike the DiD +estimators, where `covariates` switches identification to conditional +parallel trends). The operative consistency condition is covariate +BALANCE at the cutoff (`tau_Z = 0`, the testable sufficient condition of +Lemma 1: the plim shifts by `[mu_Z+ - mu_Z-]' gamma_Y` under imbalance, +and adjusting "for" imbalance cannot restore identification without +functional-form assumptions). The balance placebo is the estimator +itself - fit each covariate as `outcome_col` and inspect its RD p-value +- documented in the module docstring; a packaged `covariate_balance` +helper is a named follow-up (diagnostics wave). Implementation stacks +`Z` after `(Y, T)` as extra response columns through the SAME +local-polynomial fits (rdrobust.R:593-598); the common gamma solves the +PARTIALLED normal equations with per-side blocks summed +(`ZWZ = sum_side [Z'W D - U' invG U]`, rdrobust.R:659-671), and the +adjusted estimates are the delta-vector combinations +`s_Y = [1, -gamma[,1]]` (sharp; rdrobust.R:672-686 - the R branch omits +`factorial(deriv)` present in the no-covariate branch, identical at the +fixed `deriv=0` surface and replicated verbatim) or the fuzzy+covariates +system of rdrobust.R:688-723 (adjusted Y and T jumps via +`s_Y`/`s_T = [1, -gamma[,2]]`, their ratio, linearized bias correction, +and the EXTENDED length-`(2+dZ)` variance vector +`[1/tau_T, -tau_Y/tau_T^2, -(1/tau_T)gamma[,1] + (tau_Y/tau_T^2)gamma[,2]]`; +first-stage selector `sV_T = [0, 1, -gamma[,2]]`). Variances reuse the +same sandwiches with the `(n, 1+dT+dZ)` NN residual matrix collapsed by +the corresponding vector (functions.R:146-204, 374-385). The partial-out +identity `tau_adj = tau_unadj - gamma' tau_Z` (each covariate fit as an +outcome) holds EXACTLY at common manual `(h, b)` by Frisch-Waugh and is +locked for both the conventional and bias-corrected rows +(`test_partial_out_identity_exact`). Fitted gammas are exposed +name-keyed (`covariate_coefficients`, and +`first_stage_covariate_coefficients` on fuzzy fits) as NUISANCE +coefficients, not causal effects. + **Bandwidth selection:** all 10 rdrobust data-driven selectors (`mserd` default; `msetwo`/`msesum`/`msecomb1`/`msecomb2`; CER-optimal `cer*` variants = the matching MSE selector's `h` shrunk by @@ -3663,10 +3708,18 @@ the pilot ratio + delta vector feed the V/B constants (approach 2); (approach 1); and ONE-SIDED PERFECT COMPLIANCE (zero take-up variance on either side) auto-switches to approach 1 regardless of the flag (`perf_comp`, rdrobust.R:164-185 / rdbwselect.R:334-346) - selection -only; estimation always remains fuzzy. In the port, `T` threads through -the shared `_bw` closure so all three selector chains (mserd, msetwo, -msesum - 14 pilot call sites) receive it; the `msetwo` fuzzy golden -config pins the per-side chains. +only; estimation always remains fuzzy. Covariate-adjusted bandwidths are +COVARIATE-AWARE (CCFT 2019 Theorem 1: the bias/variance constants differ +from the no-covariate case, so selecting `h` unadjusted and then adding +covariates is not MSE-optimal): `Z` is stacked into every pilot fit with +a PER-PILOT partialled gamma and the extended `s` vector +(functions.R:241-274, 349). `perf_comp`/`sharpbw` null ONLY `T` - `Z` +always stays in selection (rdbwselect.R:343-345), so `sharpbw` with +covariates selects on the covariate-adjusted sharp objective. In the +port, `T` and `Z` thread through the shared `_bw` closure so all three +selector chains (mserd, msetwo, msesum - 14 pilot call sites) receive +them; the `msetwo`/`cercomb2` fuzzy and covariate golden configs pin the +per-side and comb chains. - **Note (canonical inference binding; deviation from R's printed output):** the result's canonical `att`/`se`/`t_stat`/`p_value`/ @@ -3767,14 +3820,61 @@ config pins the per-side chains. bandwidth switch instead. The R `var(T_side) == 0` test is implemented as exact constancy (R's two-pass `mean()` makes its variance of a constant vector exactly zero; numpy's single-pass mean does not). +- **Note (covariate redundancy pipeline, R-exact under `covs_drop=True`):** + R first sorts covariate columns by NAME LENGTH (`order(nchar)`, stable + - rdrobust.R:131; the sort decides WHICH of a collinear set survives), + then drops redundant columns via a rank-revealing pivoted QR + (`qr(z, tol=1e-7)`, LINPACK dqrdc2's per-column relative rule: a + column is negligible when its reduced norm falls below tol times its + OWN original norm, so small-but-independent covariates are never + dropped; `covs_drop_fun`, functions.R:683-688). Both are replicated + exactly - the port implements the dqrdc2 loop directly (LAPACK + pivoting differs on near-ties) and the estimator applies the name sort + before building the matrix; the sort never leaks because every + user-facing covariate surface is name-keyed (order-invariance tested). + The drop runs hoisted in `fit()` on x-sorted rows - the same rows R's + QR sees - immediately after the NaN drop and before the fuzzy + identification stop, matching rdrobust.R:121-140's ordering. + **Deviation from R:** the drop warning NAMES the dropped columns + (`covariates_dropped` echoes them) where R's message is generic, and a + rank-0 covariate matrix fails closed with a targeted `ValueError` + where R would error opaquely downstream. `covs_drop=False` (strict + mode) rejects ANY covariate degeneracy - mutual collinearity or + collinearity with the local polynomial design - with a deterministic + `ValueError`; R's `covs_drop=FALSE` relies on `chol()` erroring, which + on an exactly-singular float matrix is roundoff-dependent. +- **Deviation from R (degenerate covariate adjustment is guarded, not + reproduced):** R solves the partialled system with + `MASS::ginv(ZWZ, tol=1e-20)`; on an EXACTLY-degenerate system - + covariates collinear with the local polynomial design after + partialling, e.g. a constant covariate or a full one-hot dummy set + (both pass the intercept-free QR check above) - that tolerance INVERTS + a float-noise singular value, making R's gamma platform-noise + (observed 28% cross-implementation spread) and silently shifting tau + (~0.5% in the smoke). The port instead (a) EXCLUDES per-column + degeneracies - explained-ratio `diag(ZWZ)_j/(z_j'Wz_j) < 1e-14` - + zeroing their gamma rows, so a constant covariate reproduces the fit + without it to floating-point roundoff (tested at rel 1e-12; not + bit-for-bit - the response matrix still carries the excluded column + and BLAS matmul kernels differ with matrix shape across platforms); + (b) cuts SET-level noise directions with an + equilibrated (scale-invariant) pseudo-inverse (`sv_min < 1e-12 * + sv_max` -> `rcond=1e-12` cut), so a full dummy set reproduces the + drop-one-category fit (span invariance, tested at rel 1e-9); and (c) + warns once per fit naming the affected columns. Well-posed systems + take `np.linalg.pinv(rcond=1e-20)` - the same semantics as R's ginv - + and match R at machine precision (a 1e-9-scaled independent covariate + stays on this path untouched; scale-invariance tested). Bandwidth + pilots apply the same guard silently (per-window transients would + otherwise warn dozens of times per fit). - **Note (v1 scope seams):** only `vcov_type="nn"` (rdrobust's default) ships; `hc0`-`hc3` and cluster modes raise `NotImplementedError`. - Covariate adjustment (CCFT 2019 - review on file), weights, kink - estimands (`deriv`), `scalepar`, `stdvars`, per-side manual - bandwidths, weak-IV-robust fuzzy inference (Feir-Lemieux-Marmer), and - the rdplot/density diagnostics are documented follow-ups; - covariates/cluster/weights are not constructor parameters at all. The - port's `deriv` machinery is golden-covered for `deriv in {0, 1}` only. + Weights, kink estimands (`deriv`), `scalepar`, `stdvars`, per-side + manual bandwidths, weak-IV-robust fuzzy inference + (Feir-Lemieux-Marmer), the rdplot/density diagnostics, and a packaged + covariate-balance helper are documented follow-ups; cluster/weights + are not parameters at all. The port's `deriv` machinery is + golden-covered for `deriv in {0, 1}` only. - **Note (p/q surface, R-exact):** public `p`/`q` validation mirrors rdrobust.R:47-57 exactly - integers in 0:20 with `q > p`; `p=0` is R's local-constant fit and is accepted. R resolves a NULL `q` to `p + 1` @@ -3822,18 +3922,26 @@ config pins the per-side chains. **Validation:** bandwidth goldens `benchmarks/data/rdrobust_golden.json` (17 configs x 10 selectors, `tests/test_rdrobust_port.py`); estimation -goldens `benchmarks/data/rdrobust_estimates_golden.json` (23 configs +goldens `benchmarks/data/rdrobust_estimates_golden.json` (32 configs incl. 7 fuzzy - default/sharpbw/manual-h/epa/msetwo/one-sided-perf_comp/ -ties - with full first-stage three-row blocks; the per-side LINEARIZED -fuzzy biases are pinned at port level in -`tests/test_rdrobust_port.py::TestFuzzyPortGoldenParity`; +ties - with full first-stage three-row blocks, and 9 covariate configs - +default/manual-h/msetwo/cercomb2/epa/collinear-drop/ties/fuzzy/ +fuzzy-sharpbw - with `coef_covs` gamma pins and UNSORTED +differing-length names so every config also pins the nchar column sort; +the per-side LINEARIZED fuzzy biases and the covariate gamma matrices +are pinned at port level in +`tests/test_rdrobust_port.py::TestFuzzyPortGoldenParity` / +`::TestCovsPortGoldenParity`, with `covs_drop_fun` unit-pinned against +live-R `qr()` rank/pivot results in `::TestCovsDropFun`; `tests/test_rdd_parity.py` pins everything the public results expose); vendored Senate data (`benchmarks/data/rdrobust_senate.csv`, Cattaneo-Frandsen-Titiunik 2015) anchoring the published 2017 Stata Journal numbers under `masspoints="off"`; R-free methodology anchors in `tests/test_rdd_methodology.py` (Remark 7 equivalence, invariances, perfect-compliance == sharp, perf_comp/sharpbw bandwidth switches, -weak-first-stage warning gate, NaN/degenerate contracts). +weak-first-stage warning gate, NaN/degenerate contracts, and the +covariate partial-out identity / span-invariance / order-invariance / +CI-shrinkage anchors). **Paper reviews on file:** `docs/methodology/papers/calonico-cattaneo-titiunik-2014-review.md` (CCT 2014, @@ -3842,7 +3950,7 @@ Econometrica - robust bias-corrected RD inference), rdrobust software reference this port parity-targets), `calonico-cattaneo-farrell-2018-review.md` (JASA - CER-optimal bandwidths), `calonico-cattaneo-farrell-titiunik-2019-review.md` (REStat - covariate -adjustment, deferred). +adjustment, implemented). @@ -4447,7 +4555,7 @@ should be a deliberate user choice. | QDiD | qte | `QDiD()` | | BaconDecomposition | bacondecomp | `bacon()` | | HonestDiD | HonestDiD | `createSensitivityResults()` | -| RegressionDiscontinuity | rdrobust | `rdrobust()` + `rdbwselect()` (4.0.0; sharp + fuzzy, nn path; `treatment_col` = R's `fuzzy=`) | +| RegressionDiscontinuity | rdrobust | `rdrobust()` + `rdbwselect()` (4.0.0; sharp + fuzzy + covariate-adjusted, nn path; `treatment_col` = R's `fuzzy=`, `covariates` = R's `covs=`) | | PreTrendsPower | pretrends | `pretrends()` | | PowerAnalysis | pwr / DeclareDesign / pcpanel | `pwr::pwr.norm.test` (analytical, normal-based — D1) + `pcpanel` (Burlig 2020 panel, equicorrelated case) + simulation. The analytical multiplier is normal (z), so `pwr.t.test` is **not** the faithful parity target. | diff --git a/docs/methodology/papers/calonico-cattaneo-farrell-titiunik-2017-review.md b/docs/methodology/papers/calonico-cattaneo-farrell-titiunik-2017-review.md index 03c8b8983..5cc02027a 100644 --- a/docs/methodology/papers/calonico-cattaneo-farrell-titiunik-2017-review.md +++ b/docs/methodology/papers/calonico-cattaneo-farrell-titiunik-2017-review.md @@ -182,8 +182,8 @@ Output structure (pp. 392-399): two inference rows — `Conventional` (point est - [ ] Units at exactly the cutoff are treated (`X_i >= x̄`) - [ ] Conventional, bias-corrected, and robust bias-corrected inference all computed; Robust row reports z/p/CI only - [ ] `p = 1`, `q = 2`, triangular kernel, `mserd`, `vce(nn 3)` defaults reproduced -- [ ] Covariate adjustment uses the single joint regression with common `gamma`; recovers the unadjusted estimator exactly when `d = 0` -- [ ] Covariates, clustering, and weights propagate into bandwidth selection, not just variance +- [x] Covariate adjustment uses the single joint regression with common `gamma`; recovers the unadjusted estimator exactly when `d = 0` (shipped: pooled partialled gamma in the port; `covariates=[]`/`None` is bit-identical to the unadjusted fit, tested) +- [x] Covariates, clustering, and weights propagate into bandwidth selection, not just variance (shipped for COVARIATES: Z stacks into every pilot with a per-pilot gamma, `covs_msetwo`/`covs_cercomb2` goldens; clustering and weights remain documented v1 seams) - [ ] CER selectors shrink `h` only and reuse the corresponding MSE selector's `b` - [ ] `rho()` computes `h` only and sets `b = h/rho` - [ ] IK-style regularization on by default; `scaleregul(0)` removes it diff --git a/docs/methodology/papers/calonico-cattaneo-farrell-titiunik-2019-review.md b/docs/methodology/papers/calonico-cattaneo-farrell-titiunik-2019-review.md index 9cd36e88b..3423437e7 100644 --- a/docs/methodology/papers/calonico-cattaneo-farrell-titiunik-2019-review.md +++ b/docs/methodology/papers/calonico-cattaneo-farrell-titiunik-2019-review.md @@ -88,13 +88,13 @@ where `P^bc_+/-` are computable from the data and the only unknowns are the `n(1 - Stata: `rdrobust ..., covs(z1 z2 ...)` **Requirements checklist:** -- [ ] `covariates=` accepting continuous/discrete/mixed columns; additive-with-common-gamma specification ONLY (Equation 2) -- [ ] Covariate-aware MSE-optimal bandwidth constants (NOT the no-covariate constants with covariates bolted on) -- [ ] Joint `(Y, Z)` NN / plug-in-residual variance for the `s' ⊗ P^bc` sandwich; heteroskedastic + cluster forms -- [ ] Partial-out identity test: `tau_tilde = tau_hat - gamma_tilde' tau_hat_Z` (up to the WLS algebra) as an internal consistency check -- [ ] Covariate balance placebo diagnostic + warning on rejection -- [ ] CER rescaling `n^{-1/20}` (p = 1) applies unchanged to the covariate-adjusted bandwidth -- [ ] Head Start numbers as parity smoke test: standard `tau_hat = -2.41` (h = 6.81, b = 10.72, n- = 234, n+ = 180); covariate-adjusted with covariate-aware bandwidths `tau_tilde = -2.47`, robust 95% CI `[-5.21, -0.37]`, h = 6.98, b = 11.64, n- = 240, n+ = 184 (Table 1; triangular kernel, NN het-robust variance, 9 Census covariates) +- [x] `covariates=` accepting continuous/discrete/mixed columns; additive-with-common-gamma specification ONLY (Equation 2) +- [x] Covariate-aware MSE-optimal bandwidth constants (NOT the no-covariate constants with covariates bolted on) +- [x] Joint `(Y, Z)` NN / plug-in-residual variance for the `s' ⊗ P^bc` sandwich (heteroskedastic NN form; cluster variance remains a documented v1 seam alongside the RD estimator's other cluster paths) +- [x] Partial-out identity test: `tau_tilde = tau_hat - gamma_tilde' tau_hat_Z` (up to the WLS algebra) as an internal consistency check (`tests/test_rdd_methodology.py::TestCovariates::test_partial_out_identity_exact` - exact at common manual (h, b), both conventional and bias-corrected rows) +- [x] Covariate balance placebo diagnostic + warning on rejection - the RECIPE is documented (module docstring + REGISTRY: fit each covariate as `outcome_col`); a packaged `covariate_balance` helper with automatic warning stays a named follow-up (diagnostics wave), matching rdrobust's scope (R does not auto-test balance either) +- [x] CER rescaling `n^{-1/20}` (p = 1) applies unchanged to the covariate-adjusted bandwidth (the `cer*` selectors rescale the covariate-aware MSE `h`; `covs_cercomb2` golden config) +- [ ] Head Start numbers as parity smoke test: standard `tau_hat = -2.41` (h = 6.81, b = 10.72, n- = 234, n+ = 180); covariate-adjusted with covariate-aware bandwidths `tau_tilde = -2.47`, robust 95% CI `[-5.21, -0.37]`, h = 6.98, b = 11.64, n- = 240, n+ = 184 (Table 1; triangular kernel, NN het-robust variance, 9 Census covariates) - NOT shipped: needs the external replication dataset; the library's parity policy prefers live-R end-to-end goldens (9 covariate configs vs installed rdrobust 4.0.0) over published-number replication --- diff --git a/docs/references.rst b/docs/references.rst index e565c45b6..0cee64928 100644 --- a/docs/references.rst +++ b/docs/references.rst @@ -89,7 +89,7 @@ Nonparametric Bias-Corrected Inference - **Calonico, S., Cattaneo, M. D., Farrell, M. H., & Titiunik, R. (2019).** "Regression Discontinuity Designs Using Covariates." *The Review of Economics and Statistics*, 101(3), 442-451. https://doi.org/10.1162/rest_a_00760 - Covariate-adjusted RD (additive common-coefficient specification and its consistency conditions) - the documented fast-follow for :class:`diff_diff.RegressionDiscontinuity`; review on file. + Covariate-adjusted RD (additive common-coefficient specification and its consistency conditions) - the methodology behind :class:`diff_diff.RegressionDiscontinuity`'s ``fit(..., covariates=[...])`` adjustment (same estimand, covariate-aware bandwidths, balance as the operative testable condition). - **Feir, D., Lemieux, T., & Marmer, V. (2016).** "Weak Identification in Fuzzy Regression Discontinuity Designs." *Journal of Business & Economic Statistics*, 34(2), 185-196. https://doi.org/10.1080/07350015.2015.1024836 diff --git a/tests/test_rdd.py b/tests/test_rdd.py index facd8c5b9..0ec9d18c4 100644 --- a/tests/test_rdd.py +++ b/tests/test_rdd.py @@ -37,6 +37,7 @@ def test_constructor_defaults_match_rdrobust(self): "bwrestrict": True, "scaleregul": 1.0, "sharpbw": False, + "covs_drop": True, "alpha": 0.05, } @@ -477,3 +478,112 @@ def test_sharpbw_set_params_roundtrip(self): with pytest.raises(ValueError): rd.set_params(sharpbw="yes") assert rd.sharpbw is True # transactional: unchanged after failure + + +def _covs_df(n=600, seed=8): + rng = np.random.default_rng(seed) + x = rng.uniform(-1, 1, n) + zlong = 0.5 * x + rng.normal(size=n) + zb = rng.binomial(1, 0.4, n).astype(float) + y = 0.4 * x + 0.9 * (x >= 0) + 0.6 * zlong + 0.3 * zb + rng.standard_normal(n) * 0.2 + return pd.DataFrame({"x": x, "y": y, "zlong": zlong, "zb": zb}) + + +class TestCovariatesAPI: + def test_missing_covariate_col_raises(self): + with pytest.raises(ValueError, match="not found"): + RegressionDiscontinuity().fit(_covs_df(), "y", "x", covariates=["nope"]) + + def test_bare_string_covariates_rejected(self): + # A bare string would silently iterate characters. + with pytest.raises(ValueError, match="list of column names"): + RegressionDiscontinuity().fit(_covs_df(), "y", "x", covariates="zlong") + + def test_non_string_entry_rejected(self): + with pytest.raises(ValueError, match="list of column names"): + RegressionDiscontinuity().fit(_covs_df(), "y", "x", covariates=["zlong", 3]) + + def test_generator_covariates_materialized_not_swallowed(self): + # A generator must behave exactly like the equivalent list - the + # validation pass must not consume it into a silent no-adjustment + # fit (local-review P2). + df = _covs_df() + gen = (name for name in ["zlong", "zb"]) + r_gen = RegressionDiscontinuity().fit(df, "y", "x", covariates=gen) + r_list = RegressionDiscontinuity().fit(df, "y", "x", covariates=["zlong", "zb"]) + assert r_gen.covariates == ["zlong", "zb"] + assert r_gen.att == r_list.att and r_gen.se == r_list.se + + def test_duplicate_covariate_rejected(self): + with pytest.raises(ValueError, match="[Dd]uplicate"): + RegressionDiscontinuity().fit(_covs_df(), "y", "x", covariates=["zlong", "zlong"]) + + def test_collision_with_structural_columns_rejected(self): + df = _covs_df() + df["t"] = (df["x"] >= 0).astype(float) + for clash in ("y", "x"): + with pytest.raises(ValueError, match="collide"): + RegressionDiscontinuity().fit(df, "y", "x", covariates=[clash]) + with pytest.raises(ValueError, match="collide"): + RegressionDiscontinuity().fit(df, "y", "x", treatment_col="t", covariates=["t"]) + + def test_echo_fields_adjusted_fit(self): + df = _covs_df() + r = RegressionDiscontinuity().fit(df, "y", "x", covariates=["zlong", "zb"]) + assert r.covariates == ["zlong", "zb"] # as passed, not sorted + assert r.covariates_dropped == [] + assert r.covs_drop is True + assert set(r.covariate_coefficients) == {"zlong", "zb"} + assert all(np.isfinite(v) for v in r.covariate_coefficients.values()) + assert r.first_stage_covariate_coefficients is None # sharp fit + # estimand label NEVER changes under adjustment + assert r.estimand == "sharp (ATE at the cutoff)" + + def test_echo_fields_unadjusted_fit_none_safe(self): + r = RegressionDiscontinuity().fit(_covs_df(), "y", "x") + assert r.covariates is None + assert r.covariates_dropped is None + assert r.covariate_coefficients is None + assert r.first_stage_covariate_coefficients is None + d = r.to_dict() + assert d["covariates"] is None + assert d["covariate_coefficients"] is None + assert d["covs_drop"] is True + + def test_to_dict_adjusted(self): + r = RegressionDiscontinuity().fit(_covs_df(), "y", "x", covariates=["zlong"]) + d = r.to_dict() + assert d["covariates"] == ["zlong"] + assert d["covariates_dropped"] == [] + assert set(d["covariate_coefficients"]) == {"zlong"} + + def test_summary_covariate_lines_only_when_adjusted(self): + df = _covs_df() + plain = RegressionDiscontinuity().fit(df, "y", "x").summary() + adj = RegressionDiscontinuity().fit(df, "y", "x", covariates=["zlong", "zb"]).summary() + assert "Covariate-adjusted" not in plain + assert "Covariates" not in plain + assert "Covariate-adjusted Sharp" in adj + assert "Covariates (2): zlong, zb" in adj + + def test_canonical_identities_on_adjusted_fit(self): + r = RegressionDiscontinuity().fit(_covs_df(), "y", "x", covariates=["zlong", "zb"]) + assert r.t_stat == pytest.approx(r.att / r.se, rel=1e-14) + mid = 0.5 * (r.conf_int[0] + r.conf_int[1]) + assert mid == pytest.approx(r.att, rel=1e-12) + + def test_covs_drop_strict_bool(self): + with pytest.raises(ValueError, match="covs_drop must be a bool"): + RegressionDiscontinuity(covs_drop=1) + with pytest.raises(ValueError, match="covs_drop must be a bool"): + RegressionDiscontinuity(covs_drop="yes") + + def test_covs_drop_set_params_roundtrip(self): + est = RegressionDiscontinuity() + assert est.get_params()["covs_drop"] is True + est.set_params(covs_drop=False) + assert est.get_params()["covs_drop"] is False + with pytest.raises(ValueError, match="covs_drop must be a bool"): + est.set_params(covs_drop="no") + # failed set_params must not have mutated + assert est.get_params()["covs_drop"] is False diff --git a/tests/test_rdd_methodology.py b/tests/test_rdd_methodology.py index d81726d5b..58600eb38 100644 --- a/tests/test_rdd_methodology.py +++ b/tests/test_rdd_methodology.py @@ -403,3 +403,216 @@ def test_constant_outcome_fuzzy_manual_h(self): assert np.isfinite(r.first_stage) and np.isfinite(r.first_stage_se) assert r.first_stage_se > 0 assert np.isfinite(r.first_stage_t_stat) + + +class TestCovariates: + """CCFT 2019 covariate adjustment: internal-consistency anchors that + require no R (the R-parity pins live in test_rdd_parity.py / + test_rdrobust_port.py).""" + + @staticmethod + def _cov_df(n=900, seed=7): + rng = np.random.default_rng(seed) + x = 2 * rng.beta(2, 4, n) - 1 + zlong = 0.5 * x + rng.normal(size=n) + zb = rng.binomial(1, 0.4, n).astype(float) + y = 0.4 * x + 0.9 * (x >= 0) + 0.7 * zlong + 0.3 * zb + rng.normal(0, 0.3, n) + t = rng.binomial(1, np.where(x >= 0, 0.8, 0.15)).astype(float) + return pd.DataFrame({"x": x, "y": y, "t": t, "zlong": zlong, "zb": zb}) + + def test_partial_out_identity_exact(self): + # CCFT 2019 Section IV representation, exact at common manual + # (h, b) by Frisch-Waugh: tau_adjusted == tau_unadjusted - + # gamma' tau_Z, where tau_Z stacks the SAME RD estimator applied + # to each covariate as the outcome. Holds row-by-row for the + # conventional AND bias-corrected estimates. + df = self._cov_df() + h0 = 0.3 + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + adj = RegressionDiscontinuity(h=h0).fit(df, "y", "x", covariates=["zlong", "zb"]) + unadj = RegressionDiscontinuity(h=h0).fit(df, "y", "x") + placebo = { + name: RegressionDiscontinuity(h=h0).fit(df, name, "x") for name in ("zlong", "zb") + } + gamma = adj.covariate_coefficients + assert gamma is not None + tau_cl = unadj.att_conventional - sum( + gamma[name] * placebo[name].att_conventional for name in gamma + ) + tau_bc = unadj.att - sum(gamma[name] * placebo[name].att for name in gamma) + assert adj.att_conventional == pytest.approx(tau_cl, rel=1e-12) + assert adj.att == pytest.approx(tau_bc, rel=1e-12) + + def test_empty_and_none_covariates_identical_to_unadjusted(self): + df = self._cov_df() + plain = RegressionDiscontinuity().fit(df, "y", "x") + empty = RegressionDiscontinuity().fit(df, "y", "x", covariates=[]) + assert empty.att == plain.att and empty.se == plain.se + assert empty.covariates is None and empty.covariate_coefficients is None + + def test_irrelevant_covariate_small_change(self): + # CCFT 2019 Section V.B model 1: an irrelevant covariate "hardly + # changes empirical results" - same estimand, small perturbation. + df = self._cov_df() + rng = np.random.default_rng(99) + df["noise"] = rng.normal(size=len(df)) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + plain = RegressionDiscontinuity(h=0.3).fit(df, "y", "x") + adj = RegressionDiscontinuity(h=0.3).fit(df, "y", "x", covariates=["noise"]) + assert adj.att == pytest.approx(plain.att, rel=0.05) + assert adj.estimand == plain.estimand # label NEVER changes + + def test_informative_covariates_shrink_robust_ci(self): + # The point of the adjustment: covariates in the outcome DGP + # reduce residual variance at the cutoff -> shorter robust CI + # (CCFT 2019 Section V; ~10% in their Head Start application). + df = self._cov_df() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + plain = RegressionDiscontinuity(h=0.3).fit(df, "y", "x") + adj = RegressionDiscontinuity(h=0.3).fit(df, "y", "x", covariates=["zlong", "zb"]) + width = lambda r: r.conf_int[1] - r.conf_int[0] # noqa: E731 + assert width(adj) < width(plain) + + def test_collinear_dropped_equals_reduced_fit(self): + # zdup = exact combination; after the name-length sort the QR + # keeps the FIRST spanning subset in sorted order and the fit + # equals the reduced one (span invariance of the projection). + df = self._cov_df() + df["zdup"] = 1.5 * df["zlong"] - 0.5 * df["zb"] + with pytest.warns(UserWarning, match="Multicollinearity"): + full = RegressionDiscontinuity(h=0.3).fit( + df, "y", "x", covariates=["zlong", "zb", "zdup"] + ) + reduced = RegressionDiscontinuity(h=0.3).fit(df, "y", "x", covariates=["zlong", "zb"]) + # Sorted order (zb, zdup, zlong) keeps (zb, zdup) - the same SPAN + # as (zb, zlong), so tau/se agree to float noise. + assert full.covariates_dropped == ["zlong"] + assert sorted(full.covariate_coefficients) == ["zb", "zdup"] + assert full.att == pytest.approx(reduced.att, rel=1e-10) + assert full.se == pytest.approx(reduced.se, rel=1e-10) + + def test_covs_drop_false_collinear_raises(self): + df = self._cov_df() + df["zdup"] = 1.5 * df["zlong"] - 0.5 * df["zb"] + with pytest.raises(ValueError, match="covs_drop"): + RegressionDiscontinuity(h=0.3, covs_drop=False).fit( + df, "y", "x", covariates=["zlong", "zb", "zdup"] + ) + + def test_covs_drop_false_without_covariates_warns(self): + df = self._cov_df() + with pytest.warns(UserWarning, match="covs_drop=False has no effect"): + RegressionDiscontinuity(covs_drop=False).fit(df, "y", "x") + + def test_constant_covariate_excluded_warns(self): + # Deviation from R (which silently inverts a noise singular value + # and returns platform-dependent estimates here): the constant + # column is excluded with a warning and the fit is BIT-IDENTICAL + # to the fit without it. + df = self._cov_df() + df["zconst"] = 0.7 + with pytest.warns(UserWarning, match="Degenerate covariate adjustment"): + fit_c = RegressionDiscontinuity(h=0.3).fit(df, "y", "x", covariates=["zlong", "zconst"]) + fit_1 = RegressionDiscontinuity(h=0.3).fit(df, "y", "x", covariates=["zlong"]) + assert fit_c.covariate_coefficients["zconst"] == 0.0 + # Equality holds mathematically (the excluded column's gamma is + # exactly 0) but only to float roundoff numerically: the response + # matrix still carries the constant column, and BLAS matmul + # kernels differ with matrix SHAPE across platforms (bit-equal on + # Accelerate, last-ULP off on OpenBLAS/Windows - CI round 1). + assert fit_c.att == pytest.approx(fit_1.att, rel=1e-12) + assert fit_c.se == pytest.approx(fit_1.se, rel=1e-12) + + def test_dummy_set_stabilized_equals_drop_one(self): + # A full one-hot set passes the covariate-only QR but is + # rank-deficient after partialling on the (intercept-carrying) + # local polynomial design. The stabilized cut must reproduce any + # identified reparametrization of the same span - here, dropping + # a reference category - which R's noise-inverting ginv does not. + df = self._cov_df() + rng = np.random.default_rng(5) + cat = rng.integers(0, 3, size=len(df)) + for k in range(3): + df[f"d{k}"] = (cat == k).astype(float) + with pytest.warns(UserWarning, match="rank-deficient after partialling"): + full = RegressionDiscontinuity(h=0.3).fit(df, "y", "x", covariates=["d0", "d1", "d2"]) + reduced = RegressionDiscontinuity(h=0.3).fit(df, "y", "x", covariates=["d0", "d1"]) + assert full.att == pytest.approx(reduced.att, rel=1e-9) + assert full.se == pytest.approx(reduced.se, rel=1e-9) + + def test_all_covariates_dropped_raises(self): + df = self._cov_df() + df["z0"] = 0.0 + with pytest.raises(ValueError, match="rank-0"): + RegressionDiscontinuity(h=0.3).fit(df, "y", "x", covariates=["z0"]) + + def test_name_order_invariance(self): + # The internal name-length sort must never leak: results are + # identical whichever order the user lists the columns in. + df = self._cov_df() + a = RegressionDiscontinuity().fit(df, "y", "x", covariates=["zlong", "zb"]) + b = RegressionDiscontinuity().fit(df, "y", "x", covariates=["zb", "zlong"]) + assert a.att == b.att and a.se == b.se + assert a.covariate_coefficients == b.covariate_coefficients + assert a.h_left == b.h_left + + def test_fuzzy_covs_canonical_identities_and_first_stage(self): + df = self._cov_df() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + r = RegressionDiscontinuity().fit( + df, "y", "x", treatment_col="t", covariates=["zlong", "zb"] + ) + assert r.t_stat == pytest.approx(r.att / r.se, rel=1e-14) + mid = 0.5 * (r.conf_int[0] + r.conf_int[1]) + assert mid == pytest.approx(r.att, rel=1e-12) + assert r.first_stage_covariate_coefficients is not None + assert set(r.first_stage_covariate_coefficients) == {"zlong", "zb"} + assert np.isfinite(r.first_stage) and 0 < r.first_stage < 1.2 + assert r.estimand == "fuzzy (LATE for compliers at the cutoff)" + + def test_perfect_compliance_covs_reproduces_adjusted_sharp(self): + # T deterministic in x: the covariate-adjusted fuzzy fit must + # reproduce the covariate-adjusted SHARP fit (first stage == 1); + # bandwidths are bit-equal via the perf_comp selection switch + # (T nulled, Z kept), estimates equal to the ratio's last ULP. + df = self._cov_df() + df["t_det"] = (df["x"] >= 0).astype(float) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + fz = RegressionDiscontinuity().fit( + df, "y", "x", treatment_col="t_det", covariates=["zlong", "zb"] + ) + sh = RegressionDiscontinuity().fit(df, "y", "x", covariates=["zlong", "zb"]) + assert fz.h_left == sh.h_left and fz.b_left == sh.b_left + assert fz.att == pytest.approx(sh.att, rel=1e-12) + assert fz.se == pytest.approx(sh.se, rel=1e-12) + assert fz.first_stage_conventional == pytest.approx(1.0, rel=1e-12) + + def test_sharpbw_covs_keeps_covariates_in_selection(self): + # sharpbw nulls ONLY the take-up column in selection; with + # covariates it must select the covariate-adjusted SHARP + # bandwidths - i.e. exactly the bandwidths of the adjusted sharp + # fit, not the unadjusted ones. + df = self._cov_df() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + fz = RegressionDiscontinuity(sharpbw=True).fit( + df, "y", "x", treatment_col="t", covariates=["zlong", "zb"] + ) + sh_adj = RegressionDiscontinuity().fit(df, "y", "x", covariates=["zlong", "zb"]) + sh_plain = RegressionDiscontinuity().fit(df, "y", "x") + assert fz.h_left == sh_adj.h_left + assert fz.h_left != sh_plain.h_left + + def test_nan_in_covariate_joins_drop_count(self): + df = self._cov_df() + df.loc[3, "zlong"] = np.nan + df.loc[10, "zb"] = np.nan + with pytest.warns(UserWarning, match="Dropping 2 row"): + r = RegressionDiscontinuity().fit(df, "y", "x", covariates=["zlong", "zb"]) + assert r.n_dropped == 2 + assert r.n_obs == len(df) - 2 diff --git a/tests/test_rdd_parity.py b/tests/test_rdd_parity.py index 34643e102..7317ddb56 100644 --- a/tests/test_rdd_parity.py +++ b/tests/test_rdd_parity.py @@ -51,6 +51,20 @@ def _frame(golden, dgp_name, cfg_name=None): x = entry["x_ties"] if cfg_name == "ties_adjust" else entry["x"] t = entry["t_one"] if cfg_name == "one_sided" else entry["t"] return pd.DataFrame({"x": x, "y": entry["y"], "t": t}) + if dgp_name == "dgp_covs": + # Covariate DGP: covs_ties reuses the 2dp-rounded running + # variable; the covariate columns ride along for every config. + x = entry["x_ties"] if cfg_name == "covs_ties" else entry["x"] + return pd.DataFrame( + { + "x": x, + "y": entry["y"], + "t": entry["t"], + "zlong": entry["zlong"], + "zb": entry["zb"], + "zdup": entry["zdup"], + } + ) return pd.DataFrame({"x": entry["x"], "y": entry["y"]}) @@ -77,10 +91,17 @@ def _kwargs_from_config(cfg): def _fit(golden, dgp_name, cfg, cfg_name=None): df = _frame(golden, dgp_name, cfg_name) treatment_col = "t" if cfg.get("fuzzy_in") else None + # covs_names records the columns AS PASSED to R (unsorted, so R's + # order(nchar) column sort is part of what parity pins). + covariates = list(cfg["covs_names"]) if cfg.get("covs_in") else None with warnings.catch_warnings(): warnings.simplefilter("ignore") return RegressionDiscontinuity(**_kwargs_from_config(cfg)).fit( - df, "y", "x", treatment_col=treatment_col + df, + "y", + "x", + treatment_col=treatment_col, + covariates=covariates, ) @@ -169,9 +190,33 @@ def test_all_configs(self, golden): np.testing.assert_allclose( r.beta_t_p_right, cfg["beta_t_p_r"], rtol=RTOL, err_msg=label ) + if cfg.get("covs_in"): + # coef_covs = R's gamma over the KEPT covariates in + # nchar-sorted order; our name-keyed dicts preserve + # model order (Python dicts are insertion-ordered), so + # values() aligns row-for-row. Row count pins WHICH + # columns survived covs_drop. + gamma = np.asarray(cfg["coef_covs"], dtype=float) + gamma = gamma.reshape(gamma.shape[0], -1) + assert r.covariate_coefficients is not None, label + assert len(r.covariate_coefficients) == gamma.shape[0], label + np.testing.assert_allclose( + list(r.covariate_coefficients.values()), + gamma[:, 0], + rtol=RTOL, + err_msg=label, + ) + if cfg.get("fuzzy_in"): + assert r.first_stage_covariate_coefficients is not None, label + np.testing.assert_allclose( + list(r.first_stage_covariate_coefficients.values()), + gamma[:, 1], + rtol=RTOL, + err_msg=label, + ) n_checked += 1 - # 23 configurations; fail loudly if the golden shrinks. - assert n_checked == 23 + # 32 configurations; fail loudly if the golden shrinks. + assert n_checked == 32 class TestSenatePublished2017: diff --git a/tests/test_rdrobust_port.py b/tests/test_rdrobust_port.py index e5f948ded..6211a4d72 100644 --- a/tests/test_rdrobust_port.py +++ b/tests/test_rdrobust_port.py @@ -25,6 +25,7 @@ RDROBUST_TARBALL_SHA256, RDROBUST_VERSION, compute_dups_dupsid, + covs_drop_fun, qrXXinv, quantile_type2, rdbwselect, @@ -620,3 +621,279 @@ def test_column_vector_t_accepted(self): a = rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, t=t) b = rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, t=t.reshape(-1, 1)) assert a.tau_bc == b.tau_bc and a.se_T_rb == b.se_T_rb + + +class TestCovsPortGoldenParity: + """Port-level covariate parity incl. the per-side biases and the gamma + matrix (R's coef_covs). The covs_msetwo / covs_cercomb2 configs pin Z + threading through ALL THREE selector chains (the port always computes + mserd+msetwo+msesum; partial threading would silently ship + non-covariate-aware bandwidths for 8 of the 10 selectors), and + covs_cercomb2 additionally locks CER-rescaling of the covariate-aware + h. Golden covariate columns were passed to R with UNSORTED names of + differing lengths, so parity also pins rdrobust's order(nchar) column + sort (reproduced here by sorting the name list by length before + building the matrix).""" + + def test_covs_configs_with_bias_and_gamma(self, estimates_golden): + entry = estimates_golden["dgp_covs"] + y = np.array(entry["y"]) + t_all = np.array(entry["t"], dtype=np.float64) + cols = { + "zlong": np.array(entry["zlong"]), + "zb": np.array(entry["zb"], dtype=np.float64), + "zdup": np.array(entry["zdup"]), + } + n_checked = 0 + for name, cfg in entry["configs"].items(): + x = np.array(entry["x_ties"] if name == "covs_ties" else entry["x"]) + t = t_all if cfg["fuzzy_in"] else None + names_sorted = sorted(cfg["covs_names"], key=len) + z = np.column_stack([cols[c] for c in names_sorted]) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + if cfg["h_in"] is not None: + h_l = h_r = b_l = b_r = float(cfg["h_in"]) # h alone -> b = h + else: + bw = rdbwselect( + y, + x, + kernel=cfg["kernel"], + masspoints=cfg["masspoints"], + fuzzy=t, + sharpbw=bool(cfg["sharpbw"]), + covs=z, + ) + h_l, h_r, b_l, b_r = bw.bws[cfg["bwselect"]] + fit = rdrobust_fit(y, x, 0.0, h_l, h_r, b_l, b_r, kernel=cfg["kernel"], t=t, covs=z) + pairs = [ + ("h_l", h_l, cfg["h_l"]), + ("h_r", h_r, cfg["h_r"]), + ("b_l", b_l, cfg["b_l"]), + ("tau_cl", fit.tau_cl, cfg["tau_cl"]), + ("tau_bc", fit.tau_bc, cfg["tau_bc"]), + ("se_cl", fit.se_cl, cfg["se_cl"]), + ("se_rb", fit.se_rb, cfg["se_rb"]), + ("bias_l", fit.bias_l, cfg["bias"][0]), + ("bias_r", fit.bias_r, cfg["bias"][1]), + ] + if cfg["fuzzy_in"]: + pairs += [ + ("tau_T_cl", fit.tau_T_cl, cfg["tau_T"][0]), + ("tau_T_bc", fit.tau_T_bc, cfg["tau_T"][1]), + ("se_T_cl", fit.se_T_cl, cfg["se_T"][0]), + ("se_T_rb", fit.se_T_rb, cfg["se_T"][2]), + ] + for label, got, want in pairs: + assert got == pytest.approx( + want, rel=1e-9, abs=1e-12 + ), f"{name}:{label}: {got} vs {want}" + gamma = np.asarray(cfg["coef_covs"], dtype=float) + gamma = gamma.reshape(gamma.shape[0], -1) + assert fit.gamma_p is not None + # Row count pins WHICH columns survived the entry-point drop + # (covs_drop_collinear: 3 passed, 2 kept). + assert fit.gamma_p.shape == gamma.shape, f"{name}: gamma shape" + np.testing.assert_allclose(fit.gamma_p, gamma, rtol=1e-9, err_msg=name) + # Adjusted per-side coefficient vectors (R's beta_Y_p_*). + np.testing.assert_allclose( + fit.beta_p_l, np.ravel(cfg["beta_p_l"]), rtol=1e-9, err_msg=name + ) + np.testing.assert_allclose( + fit.beta_p_r, np.ravel(cfg["beta_p_r"]), rtol=1e-9, err_msg=name + ) + n_checked += 1 + assert n_checked == 9 + + +class TestCovsPortValidation: + def _yxz(self, n=200, seed=17): + rng = np.random.default_rng(seed) + x = rng.uniform(-1, 1, n) + z1 = 0.5 * x + rng.normal(size=n) + z2 = rng.binomial(1, 0.4, n).astype(float) + y = 0.3 * x + 0.8 * (x >= 0) + 0.6 * z1 + 0.2 * z2 + rng.normal(0, 0.2, n) + return y, x, np.column_stack([z1, z2]) + + def test_three_dim_covs_rejected(self): + y, x, z = self._yxz() + with pytest.raises(ValueError, match="1-D vector or"): + rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z.reshape(20, 10, 2)) + with pytest.raises(ValueError, match="1-D vector or"): + rdbwselect(y, x, covs=z.reshape(20, 10, 2)) + + def test_covs_length_mismatch_rejected(self): + y, x, z = self._yxz() + with pytest.raises(ValueError, match="rows to match x"): + rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z[:-3]) + with pytest.raises(ValueError, match="rows to match x"): + rdbwselect(y, x, covs=z[:-3]) + + def test_nonfinite_covs_rejected(self): + y, x, z = self._yxz() + z_bad = z.copy() + z_bad[5, 0] = np.nan + with pytest.raises(ValueError, match="finite and complete-case"): + rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z_bad) + with pytest.raises(ValueError, match="finite and complete-case"): + rdbwselect(y, x, covs=z_bad) + + def test_covs_drop_strict_bool(self): + y, x, z = self._yxz() + with pytest.raises(ValueError, match="covs_drop must be a bool"): + rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z, covs_drop=1) + with pytest.raises(ValueError, match="covs_drop must be a bool"): + rdbwselect(y, x, covs=z, covs_drop="yes") + + def test_rank_zero_covs_fails_closed(self): + # Deviation from R: an all-zero covariate matrix would make R + # index a nonexistent column downstream (opaque error); the port + # raises a targeted ValueError from both entry points. + y, x, _ = self._yxz() + zeros = np.zeros((y.shape[0], 2)) + with pytest.raises(ValueError, match="rank-0"): + rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=zeros) + with pytest.raises(ValueError, match="rank-0"): + rdbwselect(y, x, covs=zeros) + + def test_entry_drop_warns_with_r_message_and_matches_reduced(self): + y, x, z = self._yxz() + z_dup = np.column_stack([z, z[:, 0]]) # exact duplicate appended + with pytest.warns(UserWarning, match="Multicollinearity issue detected"): + fit_dup = rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z_dup) + fit_red = rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z) + # The appended duplicate is cycled out by the pivoted QR, leaving + # the ORIGINAL columns - bit-identical fit. + assert fit_dup.tau_bc == fit_red.tau_bc + assert fit_dup.se_rb == fit_red.se_rb + + def test_covs_drop_false_collinear_raises(self): + y, x, z = self._yxz() + z_dup = np.column_stack([z, z[:, 0]]) + with pytest.raises(ValueError, match="covs_drop=True"): + rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z_dup, covs_drop=False) + + def test_column_vector_covs_accepted(self): + y, x, z = self._yxz() + a = rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z[:, 0]) + b = rdrobust_fit(y, x, 0.0, 0.5, 0.5, 0.5, 0.5, covs=z[:, 0].reshape(-1, 1)) + assert a.tau_bc == b.tau_bc and a.se_rb == b.se_rb + + +class TestCovsDropFun: + """Unit pins for the LINPACK dqrdc2 rank/pivot port against live-R + ``qr(z, tol=1e-7)`` results (values captured from R 4.5.2 during the + pre-implementation smoke; deterministic given the constructions). + + The load-bearing property is dqrdc2's PER-COLUMN relative rule (a + column is negligible when its reduced norm falls below tol times its + OWN original norm) - a small-but-independent column must never be + dropped, while exact and near (1e-8) linear combinations must be.""" + + def _base(self, n=50, seed=42): + rng = np.random.default_rng(seed) + return rng.normal(size=n), rng.normal(size=n), rng.normal(size=n) + + def test_exact_duplicate_dropped(self): + a, b, _ = self._base() + keep, rank = covs_drop_fun(np.column_stack([a, b, a])) + assert rank == 2 and keep.tolist() == [0, 1] + + def test_exact_combination_dropped(self): + a, b, _ = self._base() + keep, rank = covs_drop_fun(np.column_stack([a, b, 2 * a - 3 * b])) + assert rank == 2 and keep.tolist() == [0, 1] + + def test_near_collinear_dropped_at_tol(self): + a, b, noise = self._base() + keep, rank = covs_drop_fun(np.column_stack([a, b, a + 1e-8 * noise])) + assert rank == 2 and keep.tolist() == [0, 1] + + def test_tiny_scaled_independent_column_kept(self): + # The |R[0,0]|-relative rule (LAPACK-style) would wrongly drop + # this column; dqrdc2's own-norm rule keeps it, as R does. + a, b, noise = self._base() + keep, rank = covs_drop_fun(np.column_stack([a, b, 1e-9 * noise])) + assert rank == 3 and keep.tolist() == [0, 1, 2] + + def test_constant_column_kept_by_qr(self): + # No intercept in the covariate-only QR, so a constant nonzero + # column is full-rank HERE; its collinearity with the polynomial + # design surfaces later, in the guarded gamma solve. + a, _, b = self._base() + keep, rank = covs_drop_fun(np.column_stack([a, np.full(50, 0.7), b])) + assert rank == 3 and keep.tolist() == [0, 1, 2] + + def test_zero_matrix_rank_zero(self): + # dqrdc2's zero-norm fixup (work(j,2) = 1) makes all-zero columns + # negligible: R gives rank 0, pivot 1:3 (verified live). + keep, rank = covs_drop_fun(np.zeros((5, 3))) + assert rank == 0 and keep.size == 0 + + def test_zero_column_cycled_to_end(self): + # R: qr(cbind(a, 0, b))$rank == 2, pivot == c(1, 3, 2). + rng = np.random.default_rng(3) + a, b = rng.normal(size=5), rng.normal(size=5) + keep, rank = covs_drop_fun(np.column_stack([a, np.zeros(5), b])) + assert rank == 2 and keep.tolist() == [0, 2] + + +class TestCovsDegenerateGuard: + """The guarded gamma solve (documented Deviation from R): R's + ginv(tol=1e-20) INVERTS a float-noise singular value on + exactly-degenerate partialled systems, making its output + platform-noise (observed 28% cross-implementation gamma spread and + ~0.5% tau shifts in the pre-implementation smoke). The port excludes + per-column degeneracies, cuts set-level noise directions with an + equilibrated (scale-invariant) pseudo-inverse, and warns.""" + + def _data(self, n=400, seed=11): + rng = np.random.default_rng(seed) + x = rng.uniform(-1, 1, n) + z1 = 0.4 * x + rng.normal(size=n) + y = 0.3 * x + 0.8 * (x >= 0) + 0.6 * z1 + rng.normal(0, 0.2, n) + return y, x, z1, rng + + def test_constant_covariate_excluded_equals_fit_without_it(self): + y, x, z1, _ = self._data() + z = np.column_stack([z1, np.full_like(x, 0.7)]) + with pytest.warns(UserWarning, match="collinear with the local polynomial"): + fit_c = rdrobust_fit(y, x, 0.0, 0.4, 0.4, 0.4, 0.4, covs=z) + fit_1 = rdrobust_fit(y, x, 0.0, 0.4, 0.4, 0.4, 0.4, covs=z1) + assert fit_c.covs_excluded is not None + assert fit_c.covs_excluded.tolist() == [False, True] + # gamma row zeroed -> the constant contributes exactly nothing. + assert fit_c.gamma_p is not None and fit_c.gamma_p[1, 0] == 0.0 + # Mathematically identical; numerically only to float roundoff - + # the (n, 3) vs (n, 2) response shapes route through different + # BLAS matmul kernels on some platforms (CI round 1: last-ULP + # diffs on OpenBLAS/Windows, bit-equal on Accelerate). + assert fit_c.tau_bc == pytest.approx(fit_1.tau_bc, rel=1e-12) + assert fit_c.se_rb == pytest.approx(fit_1.se_rb, rel=1e-12) + + def test_dummy_set_stabilized_equals_drop_one(self): + # A full one-hot set passes the intercept-free QR (rank 3) but the + # partialled system is rank-deficient; the stabilized cut must + # give the SAME tau as any identified reparametrization of the + # same span (drop one category) - the span-invariance property R's + # noise-inverting solve does not have. + y, x, _z1, rng = self._data() + cat = rng.integers(0, 3, size=x.shape[0]) + dummies = np.column_stack([(cat == k).astype(float) for k in range(3)]) + with pytest.warns(UserWarning, match="rank-deficient after partialling"): + fit3 = rdrobust_fit(y, x, 0.0, 0.4, 0.4, 0.4, 0.4, covs=dummies) + fit2 = rdrobust_fit(y, x, 0.0, 0.4, 0.4, 0.4, 0.4, covs=dummies[:, :2]) + assert fit3.covs_set_degenerate + assert fit3.tau_bc == pytest.approx(fit2.tau_bc, rel=1e-9) + assert fit3.se_rb == pytest.approx(fit2.se_rb, rel=1e-9) + + def test_tiny_scaled_covariate_not_flagged(self): + # Scale-invariance of the guard: a genuinely independent covariate + # scaled to 1e-9 must go through the R-exact solve untouched. + y, x, z1, rng = self._data() + z = np.column_stack([z1, 1e-9 * rng.normal(size=x.shape[0])]) + with warnings.catch_warnings(): + warnings.simplefilter("error") + fit = rdrobust_fit(y, x, 0.0, 0.4, 0.4, 0.4, 0.4, covs=z) + assert fit.covs_excluded is not None and not fit.covs_excluded.any() + assert not fit.covs_set_degenerate