[blob-store 5/5] Retire the database: pipeline + /v3 + /v3/site db-less behind DISABLE_DB

[chondl]

Track 2 · blob-store stack — PR 5 of 5 (behind DISABLE_DB; zero default change). Base: duckdb-api (#10); merge after #10. Stack order: #2 → #7 → #9 → #10 → #11. The end state: pipeline + /v3 + /v3/site run with no relational DB. No cross-track conflicts.

---

Stacked on #10 (duckdb-api). Base duckdb-api, head db-retirement, fork-only. Draft: the default backend is unchanged — with DISABLE_DB unset everything runs exactly as before. This is the last PR in the stack: the flip that runs the entire system — update pipeline, /v3, and /v3/site — with no relational database at all.

It closes the "cutover gap" #10 named: the last DB reads move onto the snapshot (#9) and Parquet (#10) blobs, /v3/site's two direct DB readers get the same facade /v3 already has, and one flag turns the DB off end to end.

Why each change

Add DISABLE_DB flag; construct no engine or session db-less (src/constants.py, src/db/main.py). DISABLE_DB=True (env, mirrors PROD). Under it, src/db/main.py sets engine = Session = None, so no code path constructs a SQLAlchemy engine or session — importing src/db stays harmless (models still register on Base.metadata), but nothing can open a connection. DISABLE_DB implies the DuckDB serving backend everywhere.

Serve /v3 and /v3/site from DuckDB when DISABLE_DB (src/api/backend.py, src/db_duckdb/, src/site/backend.py, src/site/match.py). #10's src/api/backend.py facade already routes the six /v3 routers to src/db_duckdb; this makes DISABLE_DB select it too. /v3/site's read functions call the /v3 get_*_cached wrappers, so they already ride that facade — except get_noteworthy_matches / get_upcoming_matches, which query the DB directly (Match join Event, top-N). Those get DuckDB implementations (parameterized SQL joins over the Parquet blobs, same signatures) behind a src/site/backend.py facade mirroring src/api/backend.py; because these are cross-table joins (Match join Event), each one captures the read layer's base dir once and resolves matches and events from a single manifest generation (#10's atomic Parquet set), so a top-N never pairs tables from different cycles. A missing year-file in the read layer now returns empty rather than raising, matching the DB's "no rows for this year" semantics.

Move last pipeline DB reads to Parquet; skip DB writes db-less (src/data/backend.py, src/data/main.py, src/data/router.py, src/google/storage.py). The pipeline's remaining DB reads:

• TBA etags already travel in the snapshot 6-tuple (objs[5], [blob-store 3/5] Pipeline state as a snapshot blob; DB becomes a downstream consumer #9) and are consumed from that list in process_year_tba — verified nothing re-reads them from the DB. No change needed.
• Cross-year EPA seeds (prior years' team_years, feeding get_init_epa) and post_process's norm_epa aggregates now read through src/data/backend.py, which dispatches to src/db_duckdb (parquet/{year}/team_years.parquet) under DISABLE_DB. When a prior year's Parquet is absent the read returns empty -> rookie seeding. Unlike an empty DB (which is a deliberate first-ever run), a partial/forgotten backfill silently regressing every team to the rookie mean is a misconfiguration — so db-less mode now logs a prominent warning and sets DB_LESS_SEED_INCOMPLETE, surfaced via /info, when it seeds a cross-season year with no prior data. Backfill prior years to Parquet before running db-less, or EPA seeds will be wrong (parity verified to 0.0 only with the backfill present).
• Historical Parquet is DB-independent. The current-year cycle folds Parquet into the site manifest ([blob-store 4/5] DuckDB read layer + Parquet publish for /v3 (demonstrative) #10); historical years write Parquet on their own manifest, now outside the DISABLE_DB branch, so a db-less backfill still produces Parquet.
• The best-effort healthy-cycle DB read from [blob-store 3/5] Pipeline state as a snapshot blob; DB becomes a downstream consumer #9 (db_orig = read_objs_db) and every DB upsert are skipped cleanly db-less; snapshot + blobs + Parquet remain the source of truth.
• The bucket-first site blobs that storage.write_objs pre-renders each cycle (events/all, team pages, noteworthy, upcoming) are generated through the same facades, so the bucket-first frontend stays fresh db-less.
• refresh_teams is a DB-reconciliation maintenance op — no-op db-less, because team metadata already flows through the snapshot every cycle. The site update-trigger reads etags/events from the snapshot when db-less.

Deliberate non-goals: no incremental-EPA checkpoint and no stale-while-revalidate refresh loop — full deterministic replay stays the pipeline's contract, so a cycle is always a clean recompute rather than an in-place mutation that could drift.

Evidence (local rig: fake-GCS + full 2026 season; CockroachDB toggled)

Default mode unchanged. DISABLE_DB unset, DB up: shared smoke 10/10 incl. --run-update.

Db-less cold start (the money shot). CockroachDB container stopped (:26257 connection-refused), DISABLE_DB=True, servers booted cold:

• Multiple full update cycles ran — load-from-snapshot -> TBA fetch -> EPA compute -> publish (blobs + Parquet + snapshot). No Write DB step, no session ever opened, no tracebacks. ~20-23s/cycle.
• /v3 (/v3/team/254 -> 254 sane) and /v3/site (team_years/2026, noteworthy_matches/2026 with all 6 score groups, upcoming_matches) all 200, served from DuckDB over Parquet.
• Shared smoke 10/10 incl. --run-update, consistency probe max EPA diff 0.0000. (Check-semantics note: the suite labels /v3/team/254 and /v3/site/team_years as "db-backed reads"; db-less those are served from DuckDB/Parquet, so the label is historical — no check inherently requires CockroachDB, all pass.)

Heal on the DB's return. Restarted CockroachDB, deleted 25 team_years rows to simulate drift (3724 -> 3699). One default-mode cycle (loading the snapshot state the db-less cycles accumulated) diff-upserted the missing rows back (3699 -> 3724); smoke 10/10. This is #9's honest-diff heal, preserved intact.

Cross-year Parquet seed parity. Fabricated a prior year (2025) from real 2026 team_years, written identically to the DB and to parquet/2025/team_years.parquet. get_init_epa() computed from the Parquet seed vs the DB seed over 60 teams x all EPA components: max diff 0.00e+00.

Diff: +252/-37, 11 files. Rig-only; staging never touched.

First-deploy / bootstrap order (db-less)

An empty bucket has no Parquet and no snapshot, so serving and the pipeline must be brought up in order:

1. Backfill history to Parquet first (reset_all_years, or a run with the DB still up that writes Parquet) so prior years exist before any db-less cross-season seed — otherwise every team rookie-seeds (now loudly flagged via /info, see above).
2. Until the first cycle writes Parquet, /v3 and /v3/site return empty (200), not 500 — team endpoints included (_teams_source glob fallback). The site rides its bucket blobs meanwhile.
3. Run one current-year cycle to populate the snapshot + current-year Parquet + manifest; subsequent cycles load from the snapshot.

Verified on the rig: a cold get_team(254) against an empty bucket returns None (no crash); after one folded cycle publishes Parquet, DuckDB serves 254 -> "The Cheesy Poofs", year 2026, team_year epa from Parquet.

Decommission checklist (maintainer's one-way step)

This PR leaves src/db/ and the CockroachDB dependency in place — with DISABLE_DB set the DB simply has no readers and no writers. When you are ready to make it final:

1. Run production under DISABLE_DB=True for a soak period (pipeline + /v3 + /v3/site all db-less), watching the Parquet/snapshot blobs as the sole source of truth.
2. Take a final snapshot/backup of the CockroachDB cluster (cold archive).
3. Delete src/db/ and its now-unused imports; collapse src/data/backend.py / src/site/backend.py / src/api/backend.py to the DuckDB arm only (the else arms and DISABLE_DB guards become dead).
4. Remove sqlalchemy-cockroachdb / psycopg2 from dependencies; delete clean_db, read_objs/write_objs, src/db/functions.
5. Turn off the CockroachDB cluster; collapse the three App Engine services to the one Cloud Run service serving DuckDB-over-Parquet.

After that the relational DB is gone and Parquet + snapshot blobs are the whole system.

[chondl]

Superseded by #23 — reopened as #23 [05] on cph-db-retirement.