Competing-risks survival analysis of every US interconnection request, 1995–2025. Which queued power projects get built is predictable on day one. Which ones die is not — and that turns out to be a structural fact, not a modelling failure.
→ Live results page · headline numbers as JSON · decisions log (ADRs)
Three things this repo shows:
- The finding. A single competing-risks hazard, scored out-of-cohort, ranks which projects get built within 5 years at AUC 0.72 (95% CI 0.69–0.74) / 3.7× top-decile lift, against an 8.3% base rate. The top 10% of the queue contains 36.6% of everything that actually gets built. It's a modest model, and the ablation ladder shows most of the signal is entry-knowable structure (ISO, resource, service type). The honest test — does it still discriminate within each market, not just across them? — it passes: within-ISO AUC 0.69, against 0.55 for age-and-size alone.
- The negative finding, published rather than buried. Withdrawal — the majority outcome — is near-random at entry (AUC 0.58), and county-level geography explains none of it (r = +0.005 between political lean and withdrawal rate, across 1,434 counties). Both were measured, not assumed: two independent candidate data sources were pulled and rejected on coverage first (ADR-012).
- The validation. The "built" label is confirmed against an independent federal source: ~70% of LBNL-built projects match a real operating generator in EIA-860M.
Why it matters if you don't work in energy. Data-center load growth has put a lot of capital behind projects sitting in these queues, and ~63% of everything ever proposed is withdrawn. This shows the money question splits cleanly in two: which projects are real is answerable the day they file, while which will collapse is structurally unknowable at entry, because the thing that kills them (network-upgrade cost surprises from cluster studies) hasn't happened yet.
Across ~38,000 projects (1997–2025), the interconnection queue is mostly a story of attrition: 12.5% built · 63.4% withdrawn · 22.3% still active (16.5% built among resolved projects). The question is whether you can tell, at entry, which projects beat those odds.
You can — for completion:
A single competing-risks hazard, scored out-of-cohort, ranks which projects get built within 5 years with AUC 0.72 and a top-decile lift of 3.7× — the top 10% of projects by score are built at ~3.7× the base rate. Every number is produced by
make benchmark, never hand-typed.
The mirror finding is just as important: withdrawal is near-random at entry. It's the majority outcome (72% of a fully-observed cohort withdraws within 5 years, AUC ~0.58), so who fails is largely a post-entry story — cost-allocation surprises, cluster restudies — not something visible on day one. We report withdrawal by AUC, not lift, because lift is meaningless for a majority-class outcome.
That "post-entry story" is not just an assertion — you can watch it happen. Stand at a 2-year landmark and look only at projects that have survived that long: those that have executed an interconnection agreement by then withdraw 14% of the time over the next five years; those still stuck in studies withdraw 41%. Reaching the agreement is the tell. But that signal can't be turned into a forecast on public data — the interconnection-agreement date is the only dated post-entry milestone LBNL publishes, and its recording thins out for exactly the recent cohorts you'd predict on (11% of 2000–2010 entrants to 3.5% of 2019–2023). So a landmark model barely beats the at-entry 0.58. The data that would forecast withdrawal — dated study-phase transitions, cost arrival — lives in non-public LBNL extracts, not the open queue file (ADR-020). The honest boundary: this repo can show the post-entry story but not forecast it, and neither can anyone else on open data.
The "built" label is validated against an independent federal source: ~70% of LBNL-built
projects fuzzy-match a real operating generator in the EIA-860M inventory (same state, technology,
comparable capacity and online year) — so the flagship is predicting something real, not an artifact
of one dataset's bookkeeping (src/gridqueue/unify/eia_match.py).
make benchmark builds the model one feature group at a time and reports the completion delta:
| Model | AUC | Top-decile lift |
|---|---|---|
| Age + size only | 0.55 | 1.3× |
| + ISO / resource / hybrid / interconnection-service type | 0.72 | 3.5× |
| + queue-congestion-at-entry (MW ahead in-ISO on entry day) | 0.72 | 3.7× |
| + finer congestion (county-level MW ahead, queue arrival velocity) | 0.71 | 3.4× |
| + external county/state features (rural-metro, political lean, RPS) | 0.72 | 3.5× |
The two-feature model is near-random; the entry-knowable categoricals carry the signal. Two
plausible additions don't help and are shown anyway: finer-grained congestion (localising MW
ahead to the county, plus how fast the queue was filling at entry) is flat-to-slightly-worse than the
ISO-level version — ISO is the granularity that matters (ADR-017). External
county-level data — rural/metro status, local political lean, state RPS — does not improve
prediction (an honest negative result; see docs/decisions.md, ADR-008).
"So the model just learned ISO and technology base rates?" Partly, and it says so — but not only. The decisive test is whether the ranking still works inside a single market, where the base rate is fixed. It does: the eight ISOs with enough resolved projects average within-ISO AUC 0.69 (n-weighted), against 0.55 for age-and-size alone. The big markets carry it (PJM 0.74, West 0.74, SPP 0.84); ERCOT and CAISO are more marginal (~0.60), which is itself reported rather than hidden. So only ~0.03 of the pooled 0.72 is cross-market base-rate separation — the rest is within-market signal. The per-ISO scorecard is figure 0a on the live page.
The most causally-direct opposition source, the NREL local siting-ordinance database, has since been pulled and evaluated: it is freely downloadable (an earlier note calling it access-gated was wrong), but only 3.4% of wind projects have an ordinance that is both dated and predates their queue entry — 68% of dated matches postdate entry, so the naive join is mostly leakage, not signal. It is not usable as a leakage-safe feature at this coverage (ADR-012).
The AUC doesn't move because the remaining signal isn't there to find at entry — not because the feature search stopped early. Five independent, leakage-safe additions were built or probed and each came back null: finer-grained congestion (ADR-017), external county geography (ADR-008), Henry Hub gas price at entry (ADR-018), and transmission topology from 75,308 HIFLD substations — distance-to- nearest and local density both help AUC by a fraction while hurting top-decile lift (ADR-019). The two features the literature and the commercial comparable (Enverus) lean on hardest — developer track record and interconnection cost — are either coverage-dead in the public data (ADR-012) or arrive only after entry, so they can't enter an at-entry model without leaking. The honest read: completion is predictable at entry to about AUC 0.72, and that is a structural ceiling. Who fails is the post-entry story; who gets built is largely legible on day one, but not much more legible than the market, resource, and service type already make it.
- Real, out-of-cohort, publishable: the LBNL loader → project-year panel → fixed-horizon competing-risks labels → multinomial hazard → lift / AUC / calibration / pre-2023 drift pipeline.
- Real external layer: USDA rural-urban codes, county presidential returns, and state RPS, joined leakage-safe as-of entry — committed as derived fixtures. Measured and reported; it just doesn't move the metric.
- Real cross-source checks: EIA-860M validates the completion label (~70% of built projects
confirmed as operating generators); live per-ISO queues (
gridstatus) corroborate LBNL's withdrawal picture for CAISO/MISO/ISO-NE and surface data caveats for the others.
Everything above is shipped and reproducible; what isn't built yet is in Roadmap.
Anti-leakage is enforced, not intended: every feature carries an as_of date and the entry model
asserts as_of ≤ entry_date in the fit path, with a negative test that a deliberately leaky feature
(or leaky external join) is caught.
Predicting interconnection-queue outcomes is not a new idea, and this repo doesn't claim to have discovered that completion is predictable:
- Enverus Intelligence Research runs a proprietary gradient-boosting model that scores the probability of success for projects in the queue, trained on historically operating vs. withdrawn projects — the same supervision signal used here. Its 2025 outlook predicts ~10% of projects come online in three years. It is commercial and publishes no AUC.
- Gorman, Rand et al., Joule (2024) — "Grid connection barriers to renewable energy deployment in the United States" — document the drivers of withdrawal (notably large network-upgrade costs) on the same LBNL Queued Up dataset. Descriptive, not a released classifier.
- Johnston, Liu & Yang, NBER w31946 (2023) structurally model withdrawal as a function of interconnection cost and queue congestion.
What's actually new here: an open, reproducible, competing-risks survival model on the public LBNL dataset, with transparent AUC / bootstrap CIs / calibration, an enforced as-of leakage contract, and honest negative results — not a novel prediction task. The contribution is the open version you can run and audit, and the methodology transparency, not the finding that queue completion is learnable.
Things a domain reader will (correctly) ask about, stated up front:
- 12.5% built vs. LBNL's 19%. LBNL reports ~19% completion by count for the 2000–2018 cohort as of 2023. This repo's headline funnel (12.5% built) is over all 38,201 projects including the 22.3% still active — recent filings that cannot have been built yet drag the rate down. Restrict to a comparably aged cohort and the numbers converge; the model's own base rate (8.3%) is the fully-observed eval cohort's, a third denominator. The point is that "completion rate" is denominator-dependent, and this repo states its denominator everywhere.
- Censoring is handled by cohort restriction, not IPCW. The completion AUC is computed only on cohorts old enough that entry + 5 years ≤ the data vintage (entry 2016–2020 for the 5-year horizon). A project that hasn't had its five years is excluded, not imputed — no inverse-probability weighting, no assumption about the censoring distribution.
- FERC Order 2023 makes this backward-looking. The July 2023 shift to first-ready cluster studies structurally changed the queue. Within the evaluable window the model is stable — per-cohort completion AUC is flat-to-improving (0.66 in 2016 to 0.72–0.78 for 2018–2020), and a stress split trained only on pre-2018 entries still scores 0.78 on 2018–2020. But the honest limit is real: 2021+ entries aren't 5-year-observable yet, and the Order-2023 cluster regime only shows up in cohorts that won't mature until ~2028 — so predictiveness into that regime is untestable here, not merely untested. Treat this as a descriptive account of who got built, not a forward forecast.
- Withdrawal reporting varies by ISO. LBNL flags earlier-year values as approximate and in-service
dates as incompletely covered. The
gridstatuscross-check corroborates the withdrawal picture for CAISO/MISO/ISO-NE and surfaces caveats for the rest, rather than treating all ISOs as equally clean.
Python 3.12 · uv for env/deps · scikit-learn + statsmodels · DuckDB/parquet · pytest · ruff · GitHub Actions CI. Everything runs offline and keyless against committed sample fixtures.
uv sync --extra dev # set up the environment
make lint # ruff check + format --check
make test # pytest (offline, keyless)
make demo # funnel + first hazard model (<2 min)
make benchmark # reproduce every published number + the ablation ladder
make viz # render the figures to build/queue.htmlTo reproduce the full-data headline numbers in one step:
uv sync --extra ingest # requests / openpyxl for the download
make data # fetch the public LBNL "Queued Up" xlsx -> data/raw/ (gitignored)
GRIDQUEUE_LBNL_PATH=data/raw/lbnl_full.parquet make benchmarkmake data pulls the public Queued Up workbook from the LBNL file host (it needs a browser-like
User-Agent — the file host 403s a bare curl/requests client; the downloader handles this). The
file is not redistributed here for licensing reasons, so this is the one-step path to it.
make benchmark and make viz default to the committed LBNL sample so CI stays keyless; set
GRIDQUEUE_LBNL_PATH to point them at the full dataset (the headline numbers above). It is the
single override point for the whole stack. CI enforces consistency, not correctness — the tests
assert the README and figures match the committed results JSON (no stale or hand-typed numbers), on
the sample fixture; that is a drift guard, not a claim of statistical validity.
Expect the sample to print weaker numbers than this README — that's correct, not a red flag. The
committed fixture is a 4,203-row stratified slice of the 38,201-project dataset, so make demo
reports a top-decile lift near 2.05× (vs 3.7× published) and the null correlation lands around
r = −0.07 (vs +0.005), simply because there are ~121 counties clearing the 5-resolved-project
floor instead of 1,434. Small samples are noisier in both directions. Every published figure comes
from the full dataset and is committed at results/benchmark_full.json;
the figures page prints which source it was built from and how many counties cleared the floor.
make viz builds a self-contained page (ADR-013).
Figures regenerate with make viz-assets; they are screenshots of the real page, so they cannot
drift from it.
Standalone storage went 11.6% → 31.3% of entries; counting hybrids, half of everything filed
since 2023 involves a battery. It is national — 1,139 counties across 48 states — not a
California-and-Texas story. That share uses the same unify.resources fold as make benchmark, so
the page and the benchmark cannot report different numbers (ADR-014).
County political lean does not predict withdrawal: r = +0.005 across 1,434 counties and 24,406 resolved projects. Over a 54-point swing in partisanship the withdrawal rate moves 5.4 points and never trends. The live page pairs this with the two choropleths it debunks.
There is deliberately no "withdrawal risk by county" map. It would be the most shareable thing here and it would be fiction — withdrawal is a post-entry story (ADR-004) and geography carries no at-entry signal (ADR-008). Even mapping the completion model, which does work, would mostly draw ISO boundaries and label them geography. The maps here are descriptive; the model stays off them.
grid-crunch/
├── pyproject.toml · Makefile · .github/workflows/ci.yml
├── fixtures/ # committed real slices: LBNL sample + external crosswalks
├── src/gridqueue/
│ ├── paths.py # canonical paths, data vintage, status vocab
│ ├── ingest/ # LBNL + external-feature pulls (network; run once)
│ ├── unify/lbnl.py # canonical typed queue loader
│ ├── survival/ # panel · labels · multinomial hazard
│ ├── features/ # as_of leakage contract · congestion · external joins
│ ├── benchmark/ # lift/AUC · baselines · drift · ablation · run.py · demo
│ └── viz/ # the two figures: drift maps + the published null result
├── tests/ # mirrors src/ (offline, keyless)
└── docs/ # decisions log (ADRs) + label/eval spec
Not built yet, and listed here rather than mixed in above so everything earlier on this page is shipped work:
- "Who pays" capacity-cost view — PJM zone clearing prices, sourced from the "For Public Use" Base Residual Auction reports. No PJM Data Miner data will be used: its terms bar non-members from republishing data derived from it, so the "publish only aggregates" mitigation does not hold (ADR-011).
- Data-center squeeze map — PNNL IM3 Atlas facility locations (ODbL: derived columns segregated and attributed).
- NSRDB resource quality — the last untested feature idea that clears both the leakage and coverage bars. It targets completion, which already works, so expect sharpening rather than a fix.
Closed, deliberately: developer/EDGAR entity resolution and NREL siting ordinances — both pulled, measured, and rejected on coverage (ADR-012).
MIT © 2026 owgreen-dev. Code is MIT; the committed data fixtures are derived slices of
public sources, each attributed in fixtures/FIXTURES.md:
- LBNL "Queued Up" © Lawrence Berkeley National Laboratory — cited, not redistributed in full.
- US EIA (Form 860M generator inventory) — public domain.
- US Census Bureau (county cartographic boundary file, used for the maps) — public domain.
- USDA ERS (rural-urban continuum codes), county election returns, and NCSL/DSIRE (state RPS).
- Live ISO queues are pulled with
gridstatus(BSD-3-Clause); the library is BSD, the data carries each ISO's own terms.
No PJM Data Miner data is included, deliberately — the who-pays view sources PJM's "For Public Use" Base Residual Auction reports instead (ADR-011).