store: convert every pre-convention table to SQLite STRICT (v27)#183
Merged
Conversation
Closes #148. STRICT rejects any value whose storage class can't be losslessly converted to the column's declared type instead of coercing it through type affinity. The schema was already STRICT-compatible — only INTEGER/TEXT/BLOB columns, ns-integer timestamps, INTEGER CHECK (x IN (0,1)) booleans, fixed-length BLOB hashes — so no row in a healthy database changes. What changes is the failure mode of a future mishap (a stray string bound into size_bytes, a concatenated query, a reflection accident): a hard error at the write instead of a wrong-storage-class row. STRICT can't be switched on with ALTER, so migrate_v27.go rebuilds each of the eighteen tables that predate the convention: create a STRICT copy, INSERT…SELECT every row, drop the original, rename the copy into place, recreate its indexes and triggers. The sweep runs in one transaction with foreign keys off — off because the drops and renames would otherwise trip the dense FK graph, and because a rename with FKs on would rewrite other tables' REFERENCES clauses to point at the scratch name — and verifies PRAGMA foreign_key_check before commit, like the v4→v5 and v24→v25 rebuilds. Per-table row counts are compared so a short copy fails the migration instead of losing observations, and a table the chain never materialised is skipped rather than diagnosed here. Tests: TestSchemaIsAllStrict makes the AGENTS.md convention executable (any non-STRICT table at SchemaVersion fails, including in future migrations); TestStrictRebuildsCoverEveryV26Table pins the conversion list against the non-STRICT tables a v26 database actually carries; TestMigrateV26ToV27PreservesEveryRow drives the real chain to v26, seeds every table, and checks every row, id, blob, index and trigger survives with the FK graph clean; TestMigrateV26ToV27KeepsSchemaGuarantees checks the contents append-only triggers and the one-live-row-per-path index still bite, and that STRICT now refuses a mistyped bind. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HompuyJMt84zb3KGZENkTF
There was a problem hiding this comment.
Pull request overview
This PR bumps the store schema to v27 and performs a one-time bulk migration to rebuild all pre-convention tables as SQLite STRICT, aligning the on-disk schema with the documented/enforced “all tables must be STRICT” convention.
Changes:
- Add a v26→v27 migration (
migrate_v27.go) that rebuilds all legacy non-STRICT tables as STRICT in a single FK-off transaction withforeign_key_checkverification. - Enforce the “all tables STRICT at SchemaVersion” rule via new tests, plus additional migration safety/behavior tests.
- Regenerate
store/schema.sql, updateAGENTS.mdguidance, and adjust CLI schema-print expectations for quoted table names / STRICT DDL.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| store/schema.sql | Regenerated schema snapshot at v27 reflecting STRICT tables and updated stored DDL formatting/quoting. |
| store/migrations.go | Bumps SchemaVersion to 27, registers the v27 migration, and makes bootstrap schema_version creation STRICT. |
| store/migrate_v27.go | New bulk rebuild migration converting all remaining legacy tables to STRICT and recreating dropped indexes/triggers. |
| store/migrate_v27_test.go | Adds tests enforcing STRICT-at-head and validating the v26→v27 rebuild preserves rows and schema guarantees. |
| cmd/squirrel/db_test.go | Updates CLI DDL expectations for quoted table names and STRICT output. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Two review points from the PR feed, both about the new test file: - assertFilesRowsIntact scanned folder_id and content_id and then compared only name/status, so a swapped content_id would have passed. The binding of a path to a *particular* content is the observation, so compare the full (folder_id, name, content_id, status) tuple. - The foreign_key_check loop counted rows without checking rows.Err() — a swallowed iteration error would have read as "clean graph". Extracted countFKViolations, which defers Close and fails on an iteration error. Also added the missing rows.Err() check to the files read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HompuyJMt84zb3KGZENkTF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #148.
Both halves of the issue in one PR: the new-table convention (already in
AGENTS.mdand honoured since v25'sdestination_alarms) is now enforced by a test, and the eighteen tables that predated it are converted in bulk by a dedicated migration.What STRICT buys
STRICT rejects any value whose storage class can't be losslessly converted to the column's declared type, instead of coercing it through type affinity. The schema was already STRICT-compatible — only
INTEGER/TEXT/BLOBcolumns, ns-integer timestamps,INTEGER … CHECK (x IN (0,1))booleans, fixed-lengthBLOBhashes — so no row in a healthy database changes. What changes is the failure mode of a future mishap (a stray string bound intosize_bytes, a concatenated query, a reflection accident): a hard error at the write instead of a wrong-storage-class row that reads back as garbage. Belt-and-suspenders over the existingCHECK/NOT NULL/ append-only-trigger discipline, in the same integrity-first spirit — not a fix for a known bug.The migration (
store/migrate_v27.go)STRICT can't be switched on with
ALTER, so each table is rebuilt with the standard recipe: create a STRICT copy,INSERT … SELECTevery row, drop the original, rename the copy into place, recreate its indexes and triggers. Table bodies are transcribed verbatim from v26 withSTRICTappended and nothing else changed — same columns in the same order (theALTER-appended ones keep their trailing positions), same CHECKs, same FKs, same defaults.Safety properties:
REFERENCESclauses to point at the scratch name.PRAGMA foreign_key_checkruns before commit, exactly as the v4→v5 and v24→v25 rebuilds do.contentskeeps both append-only triggers,fileskeepsuniq_files_live_per_path, andrunsis copied whole.Cost, called out in the code: it rewrites the entire database, so it's O(index size) in time and needs room for a second copy of the largest table. One-time at upgrade, with Open's automatic pre-migration snapshot as the rollback surface.
schema_versionis rebuilt too, and the bootstrapCREATE TABLE IF NOT EXISTSinmigrate()now carriesSTRICTfor fresh databases (a no-op on anything that predates v27, which the migration converts instead).Tests
TestSchemaIsAllStrict— makes theAGENTS.mdconvention executable: any non-STRICT table atSchemaVersionfails, so a future migration that forgets the keyword is caught.TestStrictRebuildsCoverEveryV26Table— pins the conversion list against the non-STRICT tables a v26 database actually carries, so a forgotten or stale entry fails.TestMigrateV26ToV27PreservesEveryRow— drives the real chain to v26 (not a hand-written fixture), seeds every table, then opens with the current binary and checks per-table row counts, ids, a blob byte-for-byte with its storage class, the files statuses, the full set of indexes and triggers, and a cleanforeign_key_check.TestMigrateV26ToV27KeepsSchemaGuarantees— thecontentsappend-only triggers and the one-live-row-per-path partial unique index still bite after the rebuild, STRICT now refuses a mistyped bind, and a well-typed insert still works.Also in the diff
store/schema.sqlregenerated (go test ./store -update-schema). Table names come back quoted — SQLite quotes the substituted name in the stored DDL after a rename — and the rebuilt bodies pick up uniform one-tab indentation, which makes the snapshot considerably easier to read than the deep-indented original.AGENTS.mdamended in the same PR, per its own rule: the "~13 tables are not STRICT yet" paragraph is replaced by the v27 statement and a pointer to the enforcing test.cmd/squirrel/db_test.goexpectations updated for the quoted names, plus a) STRICTassertion.go vet ./...andgo test ./...are clean.golangci-lintcould not run in this environment (the installed binary is built with go1.25, the module targets 1.26.1) — CI will cover it.Generated by Claude Code