fix(migrate): D1/SQLite verify no longer reports false schema drift

meta verify --dialect d1 reported permanent, unfixable schema drift for a
hand-migrated D1/SQLite database that genuinely matches its metadata, because the
shared sqlite/d1 diff path compared distinctions SQLite cannot physically store.
Four fixes, scoped to sqlite/d1 (Postgres unchanged):

- Type canon: on sqlite/d1, collapse json->text and drop text maxLength before
  comparing (a hand-written bare TEXT column IS the maxLength'd / jsonb one — SQLite
  stores neither). Also stops meta migrate emitting a no-op change for a
  metadata-only maxLength change on sqlite.
- Anonymous CHECK reconciliation: introspection parses unnamed inline CHECK(...)
  (masking comments + string literals so a CHECK( inside them is never mis-parsed),
  and the diff matches a modeled named check to an actual by normalized expression on
  sqlite/d1. Comma spacing in an IN(...) list is normalized outside literals only.
- Bare-NULL default: the D1/wrangler runner stringifies SQL NULL to "null", read as a
  literal default on every no-default column (permanent, destructive-recreate false
  drift). A bare null/NULL is now no default; a quoted 'null' stays a literal.

Gated by a new sqlite-hand-migrated-verify integration test + parseSqliteChecks /
normalizeCheckExpr unit gates (598 pass). An adversarial review caught two regressions
(a literal-unaware comma-collapse hiding regex/predicate drift; the relaxed CHECK
finder matching inside comments/literals) — both fixed + pinned.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014xy8powhHYJ6gfFt9Ut8dL

Author: dmealing

CHANGELOG.md

@@ -7,6 +7,35 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 ## [Unreleased]
 
+### Fixed — D1/SQLite `verify` false schema drift (npm-only; `migrate-ts` + `cli`)
+
+`meta verify --dialect d1` reported permanent, unfixable schema drift for a
+hand-migrated D1/SQLite database whose schema genuinely matches its metadata,
+because the shared sqlite/d1 diff path compared distinctions SQLite cannot
+physically represent. Four fixes, all scoped to sqlite/d1 (Postgres behavior
+unchanged) and gated by a new `sqlite-hand-migrated-verify` integration test plus
+`parseSqliteChecks` / `normalizeCheckExpr` unit gates:
+
+- **`json` and `VARCHAR(N)` length no longer read as drift.** SQLite/D1 has one text
+  storage class — a `field.object @storage:jsonb` column is stored as `TEXT`, and a
+  `VARCHAR(N)` length is cosmetic (not enforced). A hand-written bare `TEXT` column is
+  the same physical column as the maxLength'd / jsonb one, so the diff now canonicalizes
+  `json→text` and drops text `maxLength` on sqlite/d1 before comparing. This also
+  corrects `meta migrate`: a metadata-only `maxLength` change on sqlite/d1 now emits no
+  migration — nothing physical changes in SQLite.
+- **Anonymous inline `CHECK (…)` constraints now reconcile.** Hand-written migrations
+  write unnamed inline checks; introspection parses them (previously skipped, and now
+  with a mask so a `CHECK (` inside a comment or string literal is never mis-parsed) and
+  the diff matches a modeled named check against an actual check by normalized expression
+  on sqlite/d1, so an already-enforced constraint is no longer re-proposed as add-check.
+  Comma spacing in an `IN (…)` list is normalized *outside string literals only*
+  (`'a', 'b'` == `'a','b'`, but a comma inside a literal/regex stays significant).
+- **A bare `NULL` default is now no default.** The D1/wrangler runner stringifies a SQL
+  `NULL` `dflt_value` to the string `"null"`, which was read as a literal default on
+  every no-default column — permanent (and, on SQLite/D1, destructive recreate-and-copy)
+  false drift on every `verify`/`migrate`. `DEFAULT NULL` is equivalent to no default; a
+  quoted `'null'` stays a genuine string literal.
+
 ## [0.20.1] — 2026-07-25
 
 **npm `0.20.1`** (patch; NuGet 0.19.3 / PyPI 0.19.5 / Maven 7.11.3 unchanged). `meta init` quickstart polish: the post-init next-steps no longer lists the working `meta gen` / `meta docs` under "ship in later sub-projects" (only `ingest`/`serve`/`install-hooks` are unshipped), and `meta init` now scaffolds a minimal root `.gitignore` (node_modules/, *.sqlite, dist/) when the project has none, so a `git add -A` right after init does not stage node_modules or a local dev DB. No API or codegen-output change.

server/typescript/packages/migrate-ts/src/check-expr-compare.ts

@@ -36,7 +36,49 @@ export function normalizeCheckExpr(expr: string): string {
     .trim();
   // PG stores `col IN (…)` as `col = ANY (ARRAY[…])`; after the bracket strip above
   // that reads `col = any array …`. Fold it back to the `col in …` form we emit.
-  return stripped.replace(/=\s*any\s+array/g, "in").replace(/\s+/g, " ").trim();
+  const folded = stripped.replace(/=\s*any\s+array/g, "in").replace(/\s+/g, " ").trim();
+  // Normalize spacing AROUND COMMAS so a generated IN-list (`'a', 'b'`) compares equal
+  // to a hand-written one (`'a','b'`) — the separator commas are never semantically
+  // meaningful, and a hand SQLite migration omits the space. This is done OUTSIDE
+  // single-quoted literals only: a comma inside a string/regex literal (`'a, b'`, or a
+  // quantifier `'{1, 3}'`) IS meaningful, so two checks that differ only there must
+  // stay distinct (else a real regex/predicate change reads as clean drift).
+  return collapseCommaSpacingOutsideQuotes(folded);
+}
+
+/** Collapse `\s*,\s*` → `,` OUTSIDE single-quoted literals (see normalizeCheckExpr). */
+function collapseCommaSpacingOutsideQuotes(s: string): string {
+  let out = "";
+  let i = 0;
+  const n = s.length;
+  while (i < n) {
+    const ch = s[i]!;
+    if (ch === "'") {
+      // Copy the single-quoted literal verbatim ('' escapes honored) — never touched.
+      const start = i;
+      i++;
+      while (i < n) {
+        if (s[i] === "'") {
+          if (s[i + 1] === "'") { i += 2; continue; }
+          i++;
+          break;
+        }
+        i++;
+      }
+      out += s.slice(start, i);
+      continue;
+    }
+    if (ch === ",") {
+      out = out.replace(/\s+$/, "");           // drop whitespace already emitted before it
+      out += ",";
+      i++;
+      while (i < n && /\s/.test(s[i]!)) i++;    // skip whitespace after it
+      continue;
+    }
+    out += ch;
+    i++;
+  }
+  return out;
 }
 
 /** True when two CHECK expressions are equivalent after normalization. */

server/typescript/packages/migrate-ts/src/diff/index.ts

@@ -3,6 +3,7 @@ import type {
   ViewDescriptor,
   DependentRelation,
   Change, ChangeStatus, DiffResult, AllowOptions, AmbiguousCallback, Dialect,
+  CheckDescriptor,
 } from "../types.js";
 import type { SqlType } from "../sql-type.js";
 import { sqlTypeEquals } from "../sql-type.js";
@@ -212,7 +213,7 @@ export async function diff(
   for (const [id, expectedTable] of expectedTables) {
     const actualTable = actualTables.get(id);
     if (!actualTable) continue;
-    diffTableColumns(expectedTable, actualTable, changes);
+    diffTableColumns(expectedTable, actualTable, changes, args.dialect);
     diffTableIndexes(expectedTable, actualTable, changes);
     diffTableForeignKeys(expectedTable, actualTable, changes, args.dialect);
     // CHECK constraints on existing tables are evolved whenever a dialect is
@@ -223,7 +224,7 @@ export async function diff(
     // no dialect (legacy positional callers) checks stay un-diffed — those
     // callers may hold hand-built snapshots with empty check lists, and
     // diffing them would re-propose every modeled check forever.
-    if (args.dialect !== undefined) diffTableChecks(expectedTable, actualTable, changes);
+    if (args.dialect !== undefined) diffTableChecks(expectedTable, actualTable, changes, args.dialect);
   }
 
   // Pass 2b: views. Identity is (schema, name). How "changed" is decided is
@@ -272,10 +273,33 @@ function isDiffArgs(x: DiffArgs | SchemaSnapshot): x is DiffArgs {
   return "expected" in x && "actual" in x;
 }
 
+/**
+ * Collapse a SqlType to what SQLite/D1 can PHYSICALLY store, so the diff never
+ * reports drift the database could never represent (and thus could never fix).
+ *
+ * SQLite/D1 has a single text storage class: `json` is stored as TEXT (no native
+ * json type), and a `VARCHAR(N)` length is cosmetic — not enforced — so a bounded
+ * and an unbounded text column are the same physical column. Both collapse to bare
+ * `text`. The tool's own emit round-trips are unaffected (VARCHAR(8) and TEXT both
+ * canon to `text`), while a hand-written `TEXT` column now matches maxLength'd /
+ * jsonb metadata. Postgres represents both faithfully and is left exact.
+ */
+function canonTypeForDialect(t: SqlType, dialect: Dialect | undefined): SqlType {
+  if (dialect !== "sqlite" && dialect !== "d1") return t;
+  if (t.kind === "json") return { kind: "text" };
+  if (t.kind === "text" && t.maxLength !== undefined) return { kind: "text" };
+  return t;
+}
+
+function sqlTypeEqualsForDialect(a: SqlType, b: SqlType, dialect: Dialect | undefined): boolean {
+  return sqlTypeEquals(canonTypeForDialect(a, dialect), canonTypeForDialect(b, dialect));
+}
+
 function diffTableColumns(
   expected: TableDescriptor,
   actual: TableDescriptor,
   changes: Change[],
+  dialect?: Dialect,
 ): void {
   const table = expected.name;
   const sx = schemaSpread(expected.schema);
@@ -288,8 +312,11 @@ function diffTableColumns(
       changes.push({ kind: "add-column", table, ...sx, column: ec, status: ALLOWED });
       continue;
     }
-    // Compare type, nullable, default — emit per-aspect change.
-    if (!sqlTypeEquals(ec.sqlType, ac.sqlType)) {
+    // Compare type, nullable, default — emit per-aspect change. Type equality is
+    // dialect-aware: SQLite/D1 cannot physically represent a VARCHAR length or a
+    // native json type, so those distinctions must not read as drift (see
+    // canonTypeForDialect). Postgres stays exact.
+    if (!sqlTypeEqualsForDialect(ec.sqlType, ac.sqlType, dialect)) {
       changes.push({
         kind: "change-column-type", table, ...sx, column: name,
         from: ac.sqlType, to: ec.sqlType, status: ALLOWED,
@@ -441,23 +468,46 @@ function diffTableForeignKeys(
   }
 }
 
-function diffTableChecks(expected: TableDescriptor, actual: TableDescriptor, changes: Change[]): void {
+function diffTableChecks(
+  expected: TableDescriptor,
+  actual: TableDescriptor,
+  changes: Change[],
+  dialect?: Dialect,
+): void {
   const sx = schemaSpread(expected.schema);
-  const expectedChk = new Map(expected.checks.map((c) => [c.name, c]));
-  const actualChk = new Map(actual.checks.map((c) => [c.name, c]));
-  for (const [name, ec] of expectedChk) {
-    const ac = actualChk.get(name);
-    if (!ac) {
-      changes.push({ kind: "add-check", table: expected.name, ...sx, check: ec, status: ALLOWED });
-    } else if (!checkExprEquals(ec.expression, ac.expression)) {
-      changes.push({ kind: "drop-check", table: expected.name, ...sx, check: name, restore: ac, status: ALLOWED });
-      changes.push({ kind: "add-check", table: expected.name, ...sx, check: ec, status: ALLOWED });
+  // SQLite/D1 stores no constraint identity for an inline `CHECK (…)`, so hand-written
+  // DDL yields anonymous (empty-name) checks. When an expected named check has no
+  // same-name actual, fall back to matching by NORMALIZED EXPRESSION so an
+  // already-enforced constraint is not re-proposed as add-check (and its actual is not
+  // re-proposed as drop-check). The fallback scans ALL still-unconsumed actual checks,
+  // so it also reconciles a name-MISMATCHED actual (a check the DB named differently
+  // than the model's generated `<table>_<col>_chk`), not only truly-anonymous ones —
+  // beneficial and safe: the name-keyed pass runs first, so a genuine expression change
+  // on a name-matched check is still caught as drop+add. Postgres constraints are always
+  // named, so it keeps the exact name-keyed behavior (exprFallback off).
+  const exprFallback = dialect === "sqlite" || dialect === "d1";
+  const actualByName = new Map(actual.checks.filter((c) => c.name !== "").map((c) => [c.name, c]));
+  const consumed = new Set<CheckDescriptor>();
+
+  for (const ec of expected.checks) {
+    const ac = actualByName.get(ec.name);
+    if (ac) {
+      consumed.add(ac);
+      if (!checkExprEquals(ec.expression, ac.expression)) {
+        changes.push({ kind: "drop-check", table: expected.name, ...sx, check: ec.name, restore: ac, status: ALLOWED });
+        changes.push({ kind: "add-check", table: expected.name, ...sx, check: ec, status: ALLOWED });
+      }
+      continue;
     }
-  }
-  for (const [name, ac] of actualChk) {
-    if (!expectedChk.has(name)) {
-      changes.push({ kind: "drop-check", table: expected.name, ...sx, check: name, restore: ac, status: ALLOWED });
+    if (exprFallback) {
+      const match = actual.checks.find((a) => !consumed.has(a) && checkExprEquals(ec.expression, a.expression));
+      if (match) { consumed.add(match); continue; }
     }
+    changes.push({ kind: "add-check", table: expected.name, ...sx, check: ec, status: ALLOWED });
+  }
+  for (const ac of actual.checks) {
+    if (consumed.has(ac)) continue;
+    changes.push({ kind: "drop-check", table: expected.name, ...sx, check: ac.name, restore: ac, status: ALLOWED });
   }
 }
 

server/typescript/packages/migrate-ts/src/introspect/sqlite-shared.ts

@@ -18,6 +18,16 @@ export const SQLITE_EXPR_DEFAULT_PATTERNS = [
 export function parseSqliteDefault(raw: string | null): ColumnDefault | undefined {
   if (raw === null || raw === undefined || raw === "") return undefined;
 
+  // A bare (unquoted) NULL keyword is SQL for "no default": `DEFAULT NULL` is
+  // functionally identical to declaring no default at all. It reaches us as the
+  // JSON string "null" from the D1/wrangler runner (which stringifies a SQL NULL
+  // dflt_value) or as `NULL` from parsed DDL. A GENUINE string literal default of
+  // "null" is QUOTED (`'null'`) and is matched by the quoted branch below, so this
+  // bare test is unambiguous. Without it every no-default column on D1 reads back a
+  // literal "null" default that no metadata field ever declares — permanent, and on
+  // SQLite/D1 destructive (recreate-and-copy) false drift on every verify/migrate.
+  if (raw.trim().toLowerCase() === "null") return undefined;
+
   // A QUOTE-WRAPPED value is a string literal BY CONSTRUCTION — SQLite always quotes a
   // literal string default. This test MUST come before the expr patterns: one of those
   // patterns is a bare /\(.*\)/, so a perfectly ordinary literal containing parentheses
@@ -95,9 +105,13 @@ export function sqliteTypeToSqlType(declaredType: string): SqlType {
  * Without it the actual side always reported `checks: []` and every expected
  * check re-surfaced as add-check on every single run.
  *
- * Unnamed checks (`CHECK (…)` with no CONSTRAINT clause — hand-written DDL) are
- * NOT parsed: they have no identity to match on. A modeled check over such a
- * table converges after one recreate (which rewrites it in named form).
+ * Unnamed inline checks (`CHECK (…)` with no CONSTRAINT clause — the idiomatic
+ * hand-written-migration form) ARE parsed, with an empty name. They have no
+ * identity to match by name, so the diff reconciles them by NORMALIZED EXPRESSION
+ * on sqlite/d1: an anonymous DB check that already enforces a modeled constraint
+ * is matched to the expected named check rather than re-proposed as add-check.
+ * (Before, they were skipped entirely, so every modeled check over such a table
+ * re-surfaced as false drift on `verify --dialect d1` on every run.)
  *
  * The expression is scanned with balanced parens and string-literal awareness
  * (an enum member may contain `(`/`)`), and returned verbatim — the diff's
@@ -106,10 +120,18 @@ export function sqliteTypeToSqlType(declaredType: string): SqlType {
 export function parseSqliteChecks(createSql: string | null | undefined): CheckDescriptor[] {
   if (createSql === null || createSql === undefined || createSql === "") return [];
   const out: CheckDescriptor[] = [];
-  const re = /\bCONSTRAINT\s+(?:"((?:[^"]|"")+)"|([A-Za-z_][A-Za-z0-9_$]*))\s+CHECK\s*\(/gi;
+  // Find `CHECK (` on a MASKED copy (comments + single-quoted literals blanked to
+  // spaces of equal length) so a `CHECK (` appearing inside a `--`/`/* */` comment
+  // or a string literal (`DEFAULT 'see CHECK (x)'`) can NEVER be mis-parsed as a
+  // constraint. Positions are 1:1, so the balanced expression is sliced from the
+  // ORIGINAL (with its real quotes intact).
+  const masked = maskCommentsAndStrings(createSql);
+  // An optional `CONSTRAINT <name>` prefix then `CHECK (`. Named → group 1/2;
+  // bare inline `CHECK (` → name defaults to "" (anonymous, expression-matched).
+  const re = /(?:\bCONSTRAINT\s+(?:"((?:[^"]|"")+)"|([A-Za-z_][A-Za-z0-9_$]*))\s+)?\bCHECK\s*\(/gi;
   let m: RegExpExecArray | null;
-  while ((m = re.exec(createSql)) !== null) {
-    const name = m[1] !== undefined ? m[1].replace(/""/g, '"') : m[2]!;
+  while ((m = re.exec(masked)) !== null) {
+    const name = m[1] !== undefined ? m[1].replace(/""/g, '"') : (m[2] ?? "");
     const open = re.lastIndex - 1; // position of the "(" the regex just consumed
     let depth = 0;
     let inString = false;
@@ -137,6 +159,42 @@ export function parseSqliteChecks(createSql: string | null | undefined): CheckDe
   return out;
 }
 
+/**
+ * Same-length copy of a CREATE TABLE DDL with SQL comments (`--` line, `/* *\/`
+ * block) and single-quoted string literals blanked to spaces, so a token scan
+ * (e.g. the CHECK finder) cannot match inside them. Double-quoted identifiers are
+ * PRESERVED — a constraint or column name may be double-quoted, and none legitimately
+ * contains a `CHECK (` token. Positions are unchanged for downstream slicing.
+ */
+function maskCommentsAndStrings(sql: string): string {
+  const chars = sql.split("");
+  let i = 0;
+  const n = sql.length;
+  while (i < n) {
+    const ch = sql[i];
+    if (ch === "'") {
+      const end = skipSingleQuoted(sql, i);
+      for (let j = i; j < end; j++) chars[j] = " ";
+      i = end;
+    } else if (ch === '"') {
+      i = skipDoubleQuoted(sql, i); // preserve the identifier verbatim
+    } else if (ch === "-" && sql[i + 1] === "-") {
+      let j = i;
+      while (j < n && sql[j] !== "\n") { chars[j] = " "; j++; }
+      i = j;
+    } else if (ch === "/" && sql[i + 1] === "*") {
+      let j = i + 2;
+      while (j < n && !(sql[j] === "*" && sql[j + 1] === "/")) j++;
+      const end = Math.min(n, j + 2);
+      for (let k = i; k < end; k++) chars[k] = " ";
+      i = end;
+    } else {
+      i++;
+    }
+  }
+  return chars.join("");
+}
+
 /** Skip a single-quoted SQL string starting at `i` (position of the opening
  *  quote); returns the position just past the closing quote ('' escapes honored). */
 function skipSingleQuoted(s: string, i: number): number {

server/typescript/packages/migrate-ts/test/check-evolution/normalize.test.ts

@@ -20,7 +20,7 @@ describe("normalizeCheckExpr", () => {
       "status = ANY (ARRAY['OPEN'::text, 'CLOSED'::text])",
     )).toBe(true);
     expect(normalizeCheckExpr("status = ANY (ARRAY['OPEN'::text, 'CLOSED'::text])"))
-      .toBe("status in 'open', 'closed'");
+      .toBe("status in 'open','closed'");
     // different member sets remain distinct after the fold
     expect(checkExprEquals(
       "status IN ('OPEN', 'CLOSED')",
@@ -49,6 +49,16 @@ describe("normalizeCheckExpr", () => {
   test("genuinely different expressions are not equal", () => {
     expect(checkExprEquals("col >= 0", "col >= 5")).toBe(false);
   });
+
+  test("separator commas normalize, but a comma INSIDE a literal stays meaningful", () => {
+    // Separator commas (outside quotes) collapse, so a generated IN-list matches a
+    // hand-written one regardless of spacing...
+    expect(checkExprEquals("status IN ('a','b')", "status IN ('a', 'b')")).toBe(true);
+    // ...but a comma inside a string/regex literal is real: two checks differing only
+    // there must NOT compare equal, or a genuine regex/predicate change reads as clean.
+    expect(checkExprEquals("label <> 'a, b'", "label <> 'a,b'")).toBe(false);
+    expect(checkExprEquals("code ~ '^x{1, 3}$'", "code ~ '^x{1,3}$'")).toBe(false);
+  });
   test("undefined is never equal", () => {
     expect(checkExprEquals(undefined, "x")).toBe(false);
   });