Skip to content

Commit 702b77d

Browse files
committed
Run difftest weekly
The tool only protects the repo if it runs. It cannot join the ordinary CI job, which must not depend on downloading and compiling SQLite, so it gets a workflow of its own: weekly, on demand, with the artifacts cached against the pin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JzBeCg7rjweVW3uGPg5G7T
1 parent 99f61c1 commit 702b77d

2 files changed

Lines changed: 43 additions & 3 deletions

File tree

.github/workflows/difftest.yml

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Differential testing against a real SQLite build.
2+
#
3+
# This is separate from CI because it downloads the pinned SQLite release
4+
# and compiles it, which the ordinary test run must not depend on. It is the
5+
# only thing that exercises error fidelity at scale: the vendored corpus is
6+
# almost all valid SQL, so mutating it is how divergent messages and offsets
7+
# get found.
8+
name: difftest
9+
10+
on:
11+
schedule:
12+
- cron: "17 4 * * 1" # Mondays, early
13+
workflow_dispatch:
14+
inputs:
15+
seed:
16+
description: "PRNG seed"
17+
default: "1"
18+
per:
19+
description: "mutations per corpus case"
20+
default: "20"
21+
22+
jobs:
23+
difftest:
24+
runs-on: ubuntu-latest
25+
steps:
26+
- uses: actions/checkout@v4
27+
- uses: actions/setup-go@v5
28+
with:
29+
go-version-file: go.mod
30+
- name: Cache the pinned SQLite artifacts
31+
uses: actions/cache@v4
32+
with:
33+
path: .sqlite
34+
key: sqlite-${{ hashFiles('internal/sqlitesrc/sqlitesrc.go', 'internal/sqlitesrc/oracle/oracle.c') }}
35+
- run: >
36+
go run ./cmd/difftest
37+
-per ${{ github.event.inputs.per || '20' }}
38+
-depth 2
39+
-seed ${{ github.event.inputs.seed || '1' }}

CLAUDE.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -77,9 +77,10 @@ go run ./cmd/difftest -files select1,expr # a few files
7777
go run ./cmd/difftest -per 60 -seed 7 # dig harder, reproducibly
7878
```
7979

80-
It needs the oracle, so it is a development tool rather than a CI test. What
81-
it finds belongs in `parser/errors_test.go`, whose expectations are taken
82-
from the oracle rather than written by hand.
80+
It needs the oracle, so it is not part of `go test ./...`; a separate
81+
workflow (`.github/workflows/difftest.yml`) runs it weekly and on demand.
82+
What it finds belongs in `parser/errors_test.go`, whose expectations are
83+
taken from the oracle rather than written by hand.
8384

8485
Two situations it deliberately skips, because the harness cannot compare
8586
them rather than because meyer might be wrong: a `;` inside parentheses,

0 commit comments

Comments
 (0)