From 7528dbd34e6850f956154e0c0945271b743fc7be Mon Sep 17 00:00:00 2001 From: Connor Date: Wed, 22 Apr 2026 18:49:47 -0700 Subject: [PATCH 1/3] ci(coverage): scope 70% floor to monolithic run, not per-shard The 70% coverage floor set in 7e37296 was inherited by every per-shard pytest job via pyproject.toml `addopts`. Each shard only exercises its own source, so total coverage across `[tool.coverage.run] source` never gets close to 70% (ebird shard bottoms out at ~19% because noaa/usgs/ databox internals all report 0%). Every merge has been red since. Two changes: 1. Shard jobs (`tests-core`, `tests-ebird`, `tests-noaa`, `tests-usgs`) now override `addopts` to drop `--cov-fail-under=70` while keeping `--cov --cov-report=term-missing` so the coverage artifact is still visible in the log. 2. New `tests-all` job runs `uv run pytest` across the whole workspace and inherits the unchanged `--cov-fail-under=70`. This is the one place the floor is enforced, and it's the one place where every source is actually under test. Local run: 75.86% total. No code behavior change. Preserves the 70% floor intent from 7e37296 while making it actually enforceable. Co-Authored-By: Claude Opus 4.7 --- .github/workflows/ci.yaml | 33 +++++++++++++++++++++++++++++---- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 041316e..4109fa6 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -142,7 +142,10 @@ jobs: - name: Install run: uv sync --all-extras --dev - name: pytest core (tests/ + packages/databox) - run: uv run pytest tests packages/databox --override-ini="testpaths=tests packages/databox" + # Shard runs drop --cov-fail-under; the monolithic `tests-all` job + # enforces the 70% floor against the whole workspace (where each + # source package is actually exercised). + run: uv run pytest tests packages/databox -o "testpaths=tests packages/databox" -o "addopts=-v --tb=short --cov --cov-report=term-missing" tests-ebird: name: pytest (ebird) @@ -158,7 +161,7 @@ jobs: - name: Install run: uv sync --all-extras --dev - name: pytest ebird - run: uv run pytest packages/databox-sources/tests/ebird --override-ini="testpaths=packages/databox-sources/tests/ebird" + run: uv run pytest packages/databox-sources/tests/ebird -o "testpaths=packages/databox-sources/tests/ebird" -o "addopts=-v --tb=short --cov --cov-report=term-missing" tests-noaa: name: pytest (noaa) @@ -174,7 +177,7 @@ jobs: - name: Install run: uv sync --all-extras --dev - name: pytest noaa - run: uv run pytest packages/databox-sources/tests/noaa --override-ini="testpaths=packages/databox-sources/tests/noaa" + run: uv run pytest packages/databox-sources/tests/noaa -o "testpaths=packages/databox-sources/tests/noaa" -o "addopts=-v --tb=short --cov --cov-report=term-missing" tests-usgs: name: pytest (usgs) @@ -190,7 +193,29 @@ jobs: - name: Install run: uv sync --all-extras --dev - name: pytest usgs - run: uv run pytest packages/databox-sources/tests/usgs --override-ini="testpaths=packages/databox-sources/tests/usgs" + run: uv run pytest packages/databox-sources/tests/usgs -o "testpaths=packages/databox-sources/tests/usgs" -o "addopts=-v --tb=short --cov --cov-report=term-missing" + + tests-all: + # Monolithic whole-workspace pytest. This is the one job that enforces + # `--cov-fail-under=70` (inherited from `[tool.pytest.ini_options]` in + # pyproject.toml). Shards above keep giving fast, scoped signal on + # source-specific failures; this job guarantees the coverage floor is + # measured across the full surface (where every source is actually + # exercised), not against a shard's blind spot. + name: pytest (all, coverage floor) + needs: changes + if: needs.changes.outputs.needs_full == 'true' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 + - uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0 + with: + enable-cache: true + cache-dependency-glob: uv.lock + - name: Install + run: uv sync --all-extras --dev + - name: pytest (whole workspace, --cov-fail-under=70) + run: uv run pytest sqlmesh-lint: name: SQLMesh lint From 20fd043469ac5ae265538aa9c1941c4d468ae8af Mon Sep 17 00:00:00 2001 From: Connor Date: Wed, 22 Apr 2026 18:53:03 -0700 Subject: [PATCH 2/3] ci(coverage): drop coverage.py fail_under so shard overrides actually take effect MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Follow-up to the previous commit on this branch. Dropping `--cov-fail-under` from the pytest `addopts` override isn't enough: pytest-cov invokes `coverage.Coverage().report()` which reads `fail_under` directly from `[tool.coverage.report]` in pyproject.toml regardless of whether the CLI flag is present. That's why shard jobs still exited 1 after the previous commit (4 tests passed, coverage.py then raised CoverageException). Removing `fail_under = 70` here keeps the floor intact: the `tests-all` job still passes `--cov-fail-under=70` via the inherited `addopts`, and pytest-cov enforces that flag directly. Also regenerates `docs/dictionary/analytics/platform_health.md` — it was stale on origin/main (missing `main._dlt_loads` and `main.events` as externals), breaking the `generate_docs.py --check` docs job. Co-Authored-By: Claude Opus 4.7 --- docs/dictionary/analytics/platform_health.md | 2 ++ pyproject.toml | 7 ++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/docs/dictionary/analytics/platform_health.md b/docs/dictionary/analytics/platform_health.md index d1b0d0c..f3bd5fd 100644 --- a/docs/dictionary/analytics/platform_health.md +++ b/docs/dictionary/analytics/platform_health.md @@ -43,6 +43,8 @@ Per-source load observability — most recent dlt load id, completion time, stat - `main._dlt_loads` (external) - `main.daily_values` (external) - `main.sites` (external) +- `main._dlt_loads` (external) +- `main.events` (external) ## Example query diff --git a/pyproject.toml b/pyproject.toml index 1bdc92e..9b8b114 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -102,7 +102,12 @@ omit = [ ] [tool.coverage.report] -fail_under = 70 +# The 70% floor is enforced by the `tests-all` CI job via +# `--cov-fail-under=70` (inherited from `[tool.pytest.ini_options] +# addopts`). It is intentionally NOT set here because coverage.py's +# `fail_under` is read during `coverage.report()` whether or not +# pytest-cov's CLI flag is passed — which means shard jobs would keep +# failing even after overriding `addopts` to drop `--cov-fail-under`. skip_covered = false show_missing = true exclude_lines = [ From 88740efc282ffa8e7eeb851417459f09d14491a9 Mon Sep 17 00:00:00 2001 From: Connor Date: Wed, 22 Apr 2026 18:56:01 -0700 Subject: [PATCH 3/3] ci(coverage): run tests-all shards sequentially via coverage run --append MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Running the whole workspace in one pytest session triggers pytest-recording VCR cassette mismatches because cassette matching is global in-process — a noaa test's request can match an ebird/usgs cassette loaded earlier in the session. Each shard is self-consistent but the combined run isn't. Switch `tests-all` to: coverage erase coverage run --append -m pytest coverage run --append -m pytest ... coverage report --fail-under=70 Per-shard pytest invocations isolate VCR state; `--append` accumulates coverage across them. Local sim: 119 tests pass, 76% total coverage. Co-Authored-By: Claude Opus 4.7 --- .github/workflows/ci.yaml | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 4109fa6..deba662 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -196,12 +196,13 @@ jobs: run: uv run pytest packages/databox-sources/tests/usgs -o "testpaths=packages/databox-sources/tests/usgs" -o "addopts=-v --tb=short --cov --cov-report=term-missing" tests-all: - # Monolithic whole-workspace pytest. This is the one job that enforces - # `--cov-fail-under=70` (inherited from `[tool.pytest.ini_options]` in - # pyproject.toml). Shards above keep giving fast, scoped signal on - # source-specific failures; this job guarantees the coverage floor is - # measured across the full surface (where every source is actually - # exercised), not against a shard's blind spot. + # Coverage-floor gate. Runs each shard sequentially under `coverage run + # --append` so per-source VCR cassettes do not collide (pytest-recording + # matches requests globally in-process, so running ebird+noaa+usgs in + # one pytest session produces cross-source cassette mismatches). The + # accumulated .coverage file is then reported with `--fail-under=70`. + # This is the one place the 70% floor is enforced; shard jobs above + # skip enforcement but still give fast per-source failure signal. name: pytest (all, coverage floor) needs: changes if: needs.changes.outputs.needs_full == 'true' @@ -214,8 +215,18 @@ jobs: cache-dependency-glob: uv.lock - name: Install run: uv sync --all-extras --dev - - name: pytest (whole workspace, --cov-fail-under=70) - run: uv run pytest + - name: Erase previous coverage + run: uv run coverage erase + - name: Coverage — core (tests/ + packages/databox) + run: uv run coverage run --append -m pytest tests packages/databox -o "testpaths=tests packages/databox" -o "addopts=-v --tb=short" + - name: Coverage — ebird + run: uv run coverage run --append -m pytest packages/databox-sources/tests/ebird -o "testpaths=packages/databox-sources/tests/ebird" -o "addopts=-v --tb=short" + - name: Coverage — noaa + run: uv run coverage run --append -m pytest packages/databox-sources/tests/noaa -o "testpaths=packages/databox-sources/tests/noaa" -o "addopts=-v --tb=short" + - name: Coverage — usgs + run: uv run coverage run --append -m pytest packages/databox-sources/tests/usgs -o "testpaths=packages/databox-sources/tests/usgs" -o "addopts=-v --tb=short" + - name: Coverage report (fail under 70%) + run: uv run coverage report --fail-under=70 --show-missing sqlmesh-lint: name: SQLMesh lint