Skip to content

Commit b53bcab

Browse files
committed
chore(ci): only coverage-report runs daily; keep scrape + ingest weekly
Supersedes the blanket weekly->daily switch. Coverage runs off the already- curated data/ tree, so it is safe to run daily (now with a concurrency guard). The live multi-source scrape (weekly-refresh) and the curator-review ingest PR (weekly-ingest) stay weekly: daily scraping is excessive on upstream ToS / rate limits, and a daily review PR would swamp curators. Daily dump+PR is deferred until app.dump timestamps are deterministic (otherwise every run emits a no-op ~1400-file diff).
1 parent 8ac30f3 commit b53bcab

4 files changed

Lines changed: 18 additions & 13 deletions

File tree

.github/workflows/coverage-report.yml

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,18 @@ name: coverage-report
44
# the gap list as a sticky issue (auto-updates the existing one each run).
55
on:
66
schedule:
7-
- cron: "23 6 * * *" # Daily 06:23 UTC, after weekly-refresh (06:00)
7+
- cron: "23 6 * * *" # Daily 06:23 UTC (reads curated data/, so safe to run every day)
88
workflow_dispatch:
99

1010
permissions:
1111
contents: read
1212
issues: write
1313

14+
# Guard against a slow run overlapping the next day's scheduled run.
15+
concurrency:
16+
group: coverage-report
17+
cancel-in-progress: false
18+
1419
jobs:
1520
report:
1621
runs-on: ubuntu-latest

.github/workflows/refresh-data.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,9 @@ name: refresh-data
22

33
# Dump smoke-test: when engine code (app/**) changes, rebuild the static dump
44
# from the latest TechAPI data and validate it — a fast guard that `app.dump`
5-
# still generates end-to-end. The daily full refresh (enrich + integrity gate +
5+
# still generates end-to-end. The weekly full refresh (enrich + integrity gate +
66
# dump + PR to TechAPI) lives in weekly-refresh.yml; this no longer runs on a
7-
# schedule, to avoid duplicating that scheduled run.
7+
# schedule, to avoid duplicating that weekly run.
88
on:
99
push:
1010
branches: [main]

.github/workflows/weekly-ingest.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
name: weekly-ingest
22

3-
# Daily: scrape upstream catalogs, draft missing SKUs into a TechAPI worktree,
3+
# Weekly: scrape upstream catalogs, draft missing SKUs into a TechAPI worktree,
44
# open a PR for curator review.
55
on:
66
schedule:
7-
- cron: "29 6 * * *" # Daily 06:29 UTC, after coverage-report (06:23)
7+
- cron: "29 6 * * 1" # Mondays 06:29 UTC (weekly to keep curator review load sane)
88
workflow_dispatch:
99
inputs:
1010
category:
@@ -68,7 +68,7 @@ jobs:
6868
6969
# Variant-safe benchmark backfill on existing CPU records (PassMark).
7070
# CPU-only; never overwrites, only fills nulls on exact heading matches.
71-
# Non-fatal: a scrape hiccup must not sink the daily ingest PR.
71+
# Non-fatal: a scrape hiccup must not sink the weekly ingest PR.
7272
- name: Enrich benchmarks (PassMark, cpu only)
7373
if: env.CATEGORY == 'cpu'
7474
continue-on-error: true
@@ -125,14 +125,14 @@ jobs:
125125
git config user.email "289859915+TechEngineBot@users.noreply.github.com"
126126
git checkout -b "$BRANCH"
127127
git add data/
128-
git commit -m "feat(data/${CATEGORY}): daily ingest"
128+
git commit -m "feat(data/${CATEGORY}): weekly ingest"
129129
git push -u origin "$BRANCH"
130130
DRAFT_FLAG=""
131131
if [ "$INCLUDE_DRAFTS" = "true" ]; then
132132
DRAFT_FLAG="--draft"
133133
fi
134134
gh pr create \
135-
--title "feat(data/${CATEGORY}): daily ingest" \
135+
--title "feat(data/${CATEGORY}): weekly ingest" \
136136
--body-file ../pr-body.md \
137137
--base main \
138138
--head "$BRANCH" \

.github/workflows/weekly-refresh.yml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
name: weekly-refresh
22

3-
# Daily automated data refresh:
3+
# Weekly automated data refresh:
44
# 1. live-scrape benchmark sources into a TechAPI checkout
55
# 2. gate on FULL-dataset integrity (schema + cross-source anomalies)
66
# 3. regenerate the static v1 dump + openapi.json
@@ -15,7 +15,7 @@ name: weekly-refresh
1515
# (TechAPI Contents:write + Pull requests:write) as TECHAPI_TOKEN to enable PRs.
1616
on:
1717
schedule:
18-
- cron: "0 6 * * *" # Daily 06:00 UTC
18+
- cron: "0 6 * * 1" # Mondays 06:00 UTC (live scrape: weekly to respect upstream ToS/rate limits)
1919
workflow_dispatch:
2020
inputs:
2121
sleep:
@@ -105,7 +105,7 @@ jobs:
105105
- name: Build PR body
106106
run: |
107107
{
108-
echo "# Daily data refresh — ${{ steps.meta.outputs.date }}"
108+
echo "# Weekly data refresh — ${{ steps.meta.outputs.date }}"
109109
echo
110110
echo "Automated live re-scrape + full-dataset integrity gate + static dump."
111111
echo
@@ -159,8 +159,8 @@ jobs:
159159
data
160160
site/public/v1
161161
site/public/openapi.json
162-
commit-message: "chore(data): daily refresh ${{ steps.meta.outputs.date }}"
163-
title: "chore(data): daily refresh ${{ steps.meta.outputs.date }}"
162+
commit-message: "chore(data): weekly refresh ${{ steps.meta.outputs.date }}"
163+
title: "chore(data): weekly refresh ${{ steps.meta.outputs.date }}"
164164
body-path: pr-body.md
165165
committer: TechEngineBot <289859915+TechEngineBot@users.noreply.github.com>
166166
author: TechEngineBot <289859915+TechEngineBot@users.noreply.github.com>

0 commit comments

Comments
 (0)