diff --git a/CHANGELOG.md b/CHANGELOG.md index 6d80763..6a8f8c0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,31 @@ While pre-1.0, the public API may change between 0.x releases. _Nothing yet._ +## [0.3.3] — 2026-06-13 + +### Fixed + +- **A filtered subscription's membership no longer depends on which path + decided it (ADR-0013).** The SQL snapshot and the JS delta/catch-up evaluators + disagreed on two operators, so two clients could see different rows for the + same `where` depending on connection timing: + - **`ne` crashed the delta path and hung the client.** The SQL floor accepted + `ne` but `@tanstack/db`'s evaluator has no `ne` (not-equal is `not(eq(...))`); + its compile error escaped `handleSub` uncaught, so no `reset` was sent. `ne` + is now off the floor and rejected with `reset` like any unsupported operator, + and a defensive guard turns any predicate-compile failure into a `reset` + rather than a hang. Use `not(eq(...))` for not-equal (unchanged for real + clients). + - **`like` was case-insensitive in the snapshot but case-sensitive in deltas.** + The DO now sets `PRAGMA case_sensitive_like = ON`, making SQLite `LIKE` match + `@tanstack/db`'s case-sensitive `like` on every path. (#17) + +### Changed + +- **Operator floor for server-side filtering dropped `ne`** — it is now + `eq, gt, gte, lt, lte, like, in, and, or, not`, exactly the set the SQL and JS + evaluators agree on row-for-row. `like` is now case-sensitive. (#17) + ## [0.3.2] — 2026-06-13 ### Added diff --git a/docs/adr/0013-predicate-floor-one-evaluator.md b/docs/adr/0013-predicate-floor-one-evaluator.md new file mode 100644 index 0000000..7054417 --- /dev/null +++ b/docs/adr/0013-predicate-floor-one-evaluator.md @@ -0,0 +1,94 @@ +# ADR-0013: Filtered-subscription membership — one evaluator is the source of truth; the floor is the verified-agreeing set + +**Status**: Accepted +**Date**: 2026-06-13 +**Characterization**: `tests/predicate-parity.test.ts` (originally plan 003) + +## Context + +A filtered subscription's membership is decided by **two different evaluators** +depending on the path: + +- The **initial snapshot** filters in SQLite: the `where` IR is lowered to SQL + by `src/server/sql-compiler.ts` (`"col" = ?`, `"col" LIKE ?`, …). +- **Live deltas and reconnect catch-up** filter in JavaScript: the same IR is + compiled by `@tanstack/db`'s `compileSingleRowExpression` + `toBooleanPredicate` + in `src/server/subscriptions.ts` (`sync-do.ts` delta/catch-up paths). The client's + eager-write preflight (`do-collection.ts`) uses that same JS evaluator, and so do + the client's own live queries. + +If the two disagree on any row, a client's view of a filtered subset depends on +*when it connected* — a row excluded by the snapshot can be pushed in by a later +delta, and two clients silently diverge. Characterization found two real divergences: + +1. **`ne` — crash + hang.** `sql-compiler.ts` accepted `ne` (lowered to `!=`), but + `@tanstack/db`'s evaluator has no `ne` in its registry (not-equal is expressed as + `not(eq(...))`). `compileSingleRowExpression` threw a `QueryCompilationError` from + inside `subs.add`, which runs in `handleSub` **after** the `try/catch` that wraps + `compileSubsetQuery`. The throw escaped uncaught: no `reset` was sent and the + subscriber hung until timeout. +2. **`like` — case divergence.** SQLite `LIKE` is ASCII case-*insensitive* by default; + `@tanstack/db`'s `like` is case-*sensitive* (its case-insensitive variant is + `ilike`, which is off-floor). A row `"HELLO"` matched `like "hello%"` in the SQL + snapshot but not in the JS deltas. + +## Decisions + +### D1: `@tanstack/db`'s evaluator is the source of truth for membership + +It already decides membership on three of the four sites (delta, catch-up, client +preflight), and the client's live queries use the same library. The SQL snapshot is +an *optimization* that must reproduce exactly the rows that evaluator accepts. So when +the two disagree, **SQL conforms to `@tanstack/db`** — never the reverse. + +### D2: The operator floor is the verified-agreeing set + +Floor = `{ eq, gt, gte, lt, lte, like, in, and, or, not }`. **`ne` is removed** — it +is not in `@tanstack/db`'s evaluator, and a real client emits `not(eq(...))` (which +both paths handle identically). Anything outside the floor is rejected with +`UnsupportedPredicateError` → `reset`, as before. `tests/predicate-parity.test.ts` +pins row-for-row agreement for each floor operator across both paths; **any operator +added to `COMPARATORS` must add a parity case** (enforced by review). + +### D3: `LIKE` is made case-sensitive to match + +The DO sets `PRAGMA case_sensitive_like = ON` in the `SyncDurableObject` constructor. +The pragma is connection-scoped, and the constructor runs on every instantiation — +including a hibernation wake — so it is always in force before any query (the same +lifecycle the existing `setWebSocketAutoResponse` registration relies on). It is +contained: the IR→SQL compiler is the only producer of `LIKE` in the codebase. +(`case_sensitive_like` is a *setter-only* pragma — there is no getter — so it is +verified behaviorally in tests, not by readback.) Case-*insensitive* matching is +`ilike`, which stays off-floor (see deferred). + +The `like` **pattern must be a string literal**: `@tanstack/db`'s `evaluateLike` +returns `false` unless both operands are strings, whereas SQLite `LIKE` would +coerce a non-string pattern (`123` → `'123'`) and match rows the JS path rejects. +The compiler rejects a non-string `like` pattern with `UnsupportedPredicateError` +(→ `reset`), keeping the floor *verified*-agreeing rather than merely usually so. + +### D4: A defensive fail-loud guard at predicate compile + +`compilePredicate` (`subscriptions.ts`) now wraps the `@tanstack/db` compile and +re-throws any failure as `UnsupportedPredicateError`; `handleSub` catches it around +`subs.add` and answers with `reset`. With the floors aligned this is belt-and- +suspenders, but it guarantees fail-loud (a `reset`, never a hang) for any future +operator that lands in one floor and not the other. + +## Consequences + +- **Behavior change (observable):** a `ne` subscription is now rejected with `reset` + (was: an indefinite hang). `like` on a filtered subscription is now case-sensitive + on every path (was: case-insensitive in the snapshot only, and divergent from + deltas) — `"HELLO"` no longer matches `like "hello%"`. This aligns snapshot, delta, + catch-up, and client-side semantics. Real `@tanstack/db` clients are unaffected by + the `ne` removal — they emit `not(eq(...))`, which is fully supported. +- **No idle timers / hibernation impact:** the pragma is set synchronously in the + constructor; no polling introduced. +- **Deferred:** `ilike` (case-insensitive `LIKE`) remains off-floor. Adding it would + lower to `lower("col") LIKE lower(?)` (or equivalent) plus a parity case — a + separate decision. +- **Test coverage:** `tests/predicate-parity.test.ts` (6 cases, both paths) pins + floor-operator agreement; `tests/subscriptions.test.ts` pins the JS-floor guard at + the unit level; `tests/sql-compiler.test.ts` pins `ne` rejection and `not(eq)` + lowering. diff --git a/docs/adr/README.md b/docs/adr/README.md index e375d7c..bff4f18 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -19,3 +19,4 @@ explains the displacement. | [0009](./0009-changelog-time-retention.md) | Changelog time-based retention; reset stale reconnects | Accepted | | [0010](./0010-typed-mutations-collection-manifest.md) | Typed mutations via a collection-row manifest on `SyncRegistry` | Accepted | | [0012](./0012-wire-input-hardening.md) | Wire-input hardening: frame-shape guards, inbound limits, sanitized execute errors | Accepted | +| [0013](./0013-predicate-floor-one-evaluator.md) | Filtered-subscription membership: one evaluator is the source of truth; the floor is the verified-agreeing set | Accepted | diff --git a/package-lock.json b/package-lock.json index 4bae81f..49a9a1e 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "tanstack-do-db-collection", - "version": "0.3.2", + "version": "0.3.3", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "tanstack-do-db-collection", - "version": "0.3.2", + "version": "0.3.3", "license": "MIT", "dependencies": { "@msgpack/msgpack": "^3.0.0" diff --git a/package.json b/package.json index 631ef26..3f71306 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "tanstack-do-db-collection", - "version": "0.3.2", + "version": "0.3.3", "description": "Sync a TanStack DB collection to a Cloudflare Durable Object over WebSockets — optimistic mutations, live queries, and single-ordered-stream write confirmation.", "type": "module", "license": "MIT", diff --git a/src/server/sql-compiler.ts b/src/server/sql-compiler.ts index 995487c..42f216e 100644 --- a/src/server/sql-compiler.ts +++ b/src/server/sql-compiler.ts @@ -2,10 +2,18 @@ // `where` plus orderBy/limit/offset into a parameterised SELECT, so subset // shaping runs in SQLite rather than scanning every row in memory. // -// Supported operator floor (D6): eq, ne, gt, gte, lt, lte, like, in, and, or, -// not. Anything outside it — ilike, isNull, functional/nested-path predicates — -// throws UnsupportedPredicateError. The caller rejects such a subscription -// rather than silently falling back to a full scan (fail loud). +// Supported operator floor (D6): eq, gt, gte, lt, lte, like, in, and, or, +// not. Anything outside it — ne, ilike, isNull, functional/nested-path +// predicates — throws UnsupportedPredicateError. The caller rejects such a +// subscription rather than silently falling back to a full scan (fail loud). +// +// The floor is exactly the set of operators that the SQL snapshot path and +// @tanstack/db's JS evaluator (delta/catch-up path) agree on, row-for-row — +// see ADR-0013. `ne` is excluded because @tanstack/db has no `ne` (it only +// knows `not(eq(...))`); accepting it here desynced the two paths and crashed +// the delta path. `like` stays in the floor but is only correct because the DO +// sets `PRAGMA case_sensitive_like = ON` (SyncDurableObject constructor), which +// makes SQLite LIKE case-sensitive to match @tanstack/db's case-sensitive `like`. export class UnsupportedPredicateError extends Error { constructor(message: string) { @@ -17,11 +25,12 @@ export class UnsupportedPredicateError extends Error { const IDENT = /^[A-Za-z_][A-Za-z0-9_]*$/ const COMPARATORS: Record = { eq: "=", - ne: "!=", gt: ">", gte: ">=", lt: "<", lte: "<=", + // LIKE is case-sensitive only because the DO sets PRAGMA case_sensitive_like + // = ON; see the floor note above and ADR-0013. like: "LIKE", } @@ -60,6 +69,17 @@ function compileExpr(node: unknown, params: Array): string { if (name in COMPARATORS) { if (args.length !== 2) throw new UnsupportedPredicateError(`'${name}' expects 2 arguments`) + if (name === "like") { + // @tanstack/db's evaluateLike returns false unless BOTH operands are + // strings, whereas SQLite LIKE coerces a non-string pattern (e.g. 123 → + // '123') and could match rows the JS delta/catch-up path rejects. Restrict + // the pattern to a string literal so the two paths stay in lockstep — the + // floor must be verified-agreeing, not merely "usually agreeing" (ADR-0013). + const lit = args[1] + if (!isNode(lit) || lit.type !== "val" || typeof lit.value !== "string") { + throw new UnsupportedPredicateError("'like' pattern must be a string literal") + } + } return `${column(args[0])} ${COMPARATORS[name]} ${literal(args[1], params)}` } if (name === "and" || name === "or") { @@ -84,7 +104,7 @@ function compileExpr(node: unknown, params: Array): string { } throw new UnsupportedPredicateError( `operator '${name}' is not supported for server-side filtering ` + - `(floor: eq, ne, gt, gte, lt, lte, like, in, and, or, not)`, + `(floor: eq, gt, gte, lt, lte, like, in, and, or, not)`, ) } diff --git a/src/server/subscriptions.ts b/src/server/subscriptions.ts index 1674d8e..5eae90c 100644 --- a/src/server/subscriptions.ts +++ b/src/server/subscriptions.ts @@ -7,6 +7,7 @@ // client's operator semantics exactly (no second predicate implementation). import { compileSingleRowExpression, toBooleanPredicate } from "@tanstack/db" +import { UnsupportedPredicateError } from "./sql-compiler.ts" export interface Sub { subId: string @@ -19,9 +20,20 @@ export interface Sub { function compilePredicate(where: unknown): (row: Record) => boolean { if (where === undefined || where === null) return () => true - const evaluate = compileSingleRowExpression(where as never) as ( - row: Record, - ) => boolean | null + let evaluate: (row: Record) => boolean | null + try { + evaluate = compileSingleRowExpression(where as never) as (row: Record) => boolean | null + } catch (e) { + // @tanstack/db's evaluator rejects any operator outside its registry (e.g. a + // hand-built `ne`, which it does not know — only `not(eq(...))`). The SQL + // snapshot floor (sql-compiler.ts) and this JS delta floor MUST be the same + // set, or a sub's membership depends on which path decided it. Re-throw as + // UnsupportedPredicateError so handleSub rejects the sub with a `reset` + // (fail loud) instead of letting this escape uncaught and hang the client. (ADR-0013) + throw new UnsupportedPredicateError( + `predicate not compilable by @tanstack/db evaluator: ${e instanceof Error ? e.message : String(e)}`, + ) + } // toBooleanPredicate collapses SQL 3-valued null to false, matching SQL. return (row) => toBooleanPredicate(evaluate(row)) } diff --git a/src/server/sync-do.ts b/src/server/sync-do.ts index 0cee7ca..0caecdf 100644 --- a/src/server/sync-do.ts +++ b/src/server/sync-do.ts @@ -33,7 +33,7 @@ import { Broadcaster } from "./broadcast.ts" import { decodeResult, encodeResult, lookupTx, recordTx, type SeenTx, sweepDedup } from "./dedup.ts" import type { SyncRegistry } from "./registry.ts" import { andPredicates, compileSubsetQuery, UnsupportedPredicateError } from "./sql-compiler.ts" -import { SubscriptionRegistry } from "./subscriptions.ts" +import { SubscriptionRegistry, type Sub } from "./subscriptions.ts" export abstract class SyncDurableObject extends DurableObject { /** Set by `registerSync` — the collections/mutations/commands this DO serves. */ @@ -88,6 +88,12 @@ export abstract class SyncDurableObject extends super(ctx, env) // Auto-pong via the runtime: survives hibernation, no per-message billing. this.ctx.setWebSocketAutoResponse(new WebSocketRequestResponsePair("ping", "pong")) + // Make SQLite LIKE case-sensitive so the SQL snapshot path matches + // @tanstack/db's case-sensitive `like` evaluator on the delta path — the + // single source of truth for filtered-subscription membership (ADR-0013). + // Connection-scoped pragma; re-applied on every instantiation, including a + // hibernation wake (same lifecycle as the auto-response registration above). + this.sql.exec("PRAGMA case_sensitive_like = ON") // Restore the live-socket set after a hibernation wake. for (const ws of this.ctx.getWebSockets()) this.liveWs.add(ws) this.broadcaster = new Broadcaster((ws, frame) => this.send(ws, frame), this.tickMs) @@ -609,7 +615,23 @@ export abstract class SyncDurableObject extends // before the tick loses the write (reconnect resumes past it). this.broadcaster.flushOne(ws) - const sub = this.subs.add(ws, frame.subId, frame.collection, frame.where) + // Registering compiles the predicate in @tanstack/db's evaluator. If the + // predicate is outside the JS floor (e.g. an operator the SQL floor somehow + // let through), that throws UnsupportedPredicateError — reject with `reset` + // rather than letting it escape uncaught and hang the client (ADR-0013). + // With the floors aligned this is belt-and-suspenders, but it must hold for + // any future operator added to one floor and not the other. + let sub: Sub + try { + sub = this.subs.add(ws, frame.subId, frame.collection, frame.where) + } catch (e) { + if (e instanceof UnsupportedPredicateError) { + console.error(`sub '${frame.subId}' on '${frame.collection}' rejected: ${e.message}`) + this.send(ws, { t: "reset", sub: frame.subId }) + return + } + throw e + } const seq = String(currentSeq(this.sql)) // Reconnect catch-up: a `since` cursor asks for changes after that point diff --git a/tests/predicate-parity.test.ts b/tests/predicate-parity.test.ts new file mode 100644 index 0000000..fa8f767 --- /dev/null +++ b/tests/predicate-parity.test.ts @@ -0,0 +1,199 @@ +// WHY: a filtered subscription's membership must not depend on which evaluator +// decided it — the SQL snapshot path and the JS delta path must agree on every +// row, or two clients diverge by connection timing. Specifically: +// - The snapshot path filters rows in SQLite via sql-compiler.ts (lowered SQL). +// - The delta path filters rows in JS via @tanstack/db's compileSingleRowExpression +// + toBooleanPredicate in subscriptions.ts. +// SQL and JS disagree at the edges (NULL three-valued logic, LIKE case-folding, +// IN with NULLs), so the operator floor is defined as exactly the set on which the +// two agree row-for-row, and the DO forces SQLite LIKE case-sensitive to match +// @tanstack/db's case-sensitive `like` (ADR-0013). These cases pin that agreement +// and fail loudly if it ever regresses. +// +// History: plan 003 first landed these as `it.fails` characterizations of two real +// divergences — `ne` (which @tanstack/db cannot compile, crashing the delta path +// and hanging the client) and `like` (SQLite case-insensitive vs JS case-sensitive). +// ADR-0013 resolved both: `ne` dropped from the floor (→ reset), LIKE made +// case-sensitive. The cases below now assert the fixed behavior. + +import { env, runInDurableObject, SELF } from "cloudflare:test" +import { describe, expect, it } from "vitest" +import { createFrameCodec } from "../src/wire/frame-codec.ts" +import type { ClientFrame, ServerFrame } from "../src/wire/frames.ts" + +const codec = createFrameCodec() + +async function openWs(room: string): Promise { + const res = await SELF.fetch(`https://example.com/sync/${room}`, { headers: { Upgrade: "websocket" } }) + const ws = res.webSocket + if (!ws) throw new Error("no webSocket") + ws.accept() + return ws +} + +function collectUntil(ws: WebSocket, done: (f: ServerFrame) => boolean, timeoutMs = 3000): Promise> { + return new Promise((resolve, reject) => { + const out: Array = [] + const timer = setTimeout(() => reject(new Error(`timeout; got [${out.map((f) => f.t).join(",")}]`)), timeoutMs) + const onMsg = (e: MessageEvent): void => { + out.push(codec.decode(e.data as ArrayBuffer) as ServerFrame) + if (done(out[out.length - 1]!)) { + clearTimeout(timer) + ws.removeEventListener("message", onMsg) + resolve(out) + } + } + ws.addEventListener("message", onMsg) + }) +} + +const send = (ws: WebSocket, f: ClientFrame): void => ws.send(codec.encode(f)) + +// IR helpers — plain JSON matching the wire shape (type/name/args). +const ref = (col: string): unknown => ({ type: "ref", path: [col] }) +const val = (value: unknown): unknown => ({ type: "val", value }) +const fn = (name: string, ...args: Array): unknown => ({ type: "func", name, args }) + +// Seed rows: +// n1 — body IS NULL (tests NULL semantics in every case) +// lo — body = "hello" (lowercase) +// up — body = "HELLO" (uppercase — tests LIKE case-sensitivity) +// x — body = "x" (value used in eq/in tests) +type Seed = { id: string; body: string | null } +const SEEDS: Array = [ + { id: "n1", body: null }, + { id: "lo", body: "hello" }, + { id: "up", body: "HELLO" }, + { id: "x", body: "x" }, +] + +/** + * Snapshot membership for a `where` IR: insert seed rows server-side via + * runInDurableObject (raw SQL — so body can be NULL), then sub from a fresh + * socket and collect keys from `snap` frames until `snap-end`. Returns the + * sorted set of keys the SQL snapshot matched. + */ +async function snapshotMembers(room: string, where: unknown): Promise> { + const stub = env.SYNC_DO.get(env.SYNC_DO.idFromName(room)) + await runInDurableObject(stub, (_i, s) => { + for (const seed of SEEDS) { + if (seed.body === null) s.storage.sql.exec("INSERT INTO messages(id) VALUES (?)", seed.id) + else s.storage.sql.exec("INSERT INTO messages(id, body) VALUES (?, ?)", seed.id, seed.body) + } + }) + const ws = await openWs(room) + send(ws, { t: "sub", subId: "s1", collection: "messages", where }) + const frames = await collectUntil(ws, (f) => f.t === "snap-end") + ws.close() + return frames + .filter((f): f is Extract => f.t === "snap") + .map((f) => f.key as string) + .sort() +} + +/** + * Delta membership for a `where` IR: sub first (empty snapshot), then insert seed + * rows via `mut` frames so they drain synchronously. Each inserted key arrives as + * a `d`/insert (member) or `d`/delete (non-member, the always-emit rule). Returns + * the sorted set of keys that arrived as non-delete deltas. + * + * The NULL row is sent with `body` omitted (undefined → NULL in SQLite, the normal + * client path); `explicitNull` instead sends `body: null` to prove both spellings + * produce identical membership (no codec/binding divergence). + */ +async function deltaMembers(room: string, where: unknown, explicitNull = false): Promise> { + const ws = await openWs(room) + send(ws, { t: "sub", subId: "s1", collection: "messages", where }) + await collectUntil(ws, (f) => f.t === "snap-end") + + const members = new Set() + let txCounter = 0 + for (const seed of SEEDS) { + const txId = `tx-${++txCounter}` + const cols: Record = + seed.body === null ? (explicitNull ? { id: seed.id, body: null } : { id: seed.id }) : { id: seed.id, body: seed.body } + send(ws, { t: "mut", txId, collection: "messages", ops: [{ type: "insert", key: seed.id, cols }] }) + const frames = await collectUntil(ws, (f) => f.t === "committed" && f.txId === txId) + const delta = frames.find((f): f is Extract => f.t === "d" && (f.key as string) === seed.id) + if (delta && delta.op !== "delete") members.add(seed.id) + } + ws.close() + return Array.from(members).sort() +} + +/** Subscribe and return the type of the first terminal frame: "reset" or "snap-end". */ +async function subTerminal(room: string, where: unknown): Promise { + const ws = await openWs(room) + send(ws, { t: "sub", subId: "s1", collection: "messages", where }) + const frames = await collectUntil(ws, (f) => f.t === "reset" || f.t === "snap-end") + ws.close() + return frames[frames.length - 1]!.t +} + +describe("SQL/JS predicate parity (ADR-0013)", () => { + // ne: an off-floor operator @tanstack/db cannot compile. The SQL floor now + // rejects it too (it was removed from COMPARATORS), so BOTH paths answer with a + // `reset` — fail loud, never the old uncaught-throw hang. The supported + // not-equal is not(eq(...)), covered below and in subscriptions.test.ts. + it("ne: rejected with reset, never a hang (the old crash, fixed)", async () => { + const where = fn("ne", ref("body"), val("x")) + expect(await subTerminal("pp-ne-snap", where), "ne sub must be rejected with reset").toBe("reset") + }) + + // eq: exact match, no NULL edge. SQL `=` and JS `eq` both case-sensitive → only "lo". + it("eq: exact match agrees across paths", async () => { + const where = fn("eq", ref("body"), val("hello")) + const snap = await snapshotMembers("pp-eq-snap", where) + const delta = await deltaMembers("pp-eq-delta", where) + expect(snap, `eq mismatch — snap=${snap} delta=${delta}`).toEqual(delta) + expect(snap).toEqual(["lo"]) + }) + + // like: the headline fix. With PRAGMA case_sensitive_like=ON, SQLite LIKE matches + // @tanstack/db's case-sensitive `like`: "HELLO" does NOT match "hello%" on either + // path. Both → ["lo"]. (Before ADR-0013: snap=["lo","up"], delta=["lo"] — divergent.) + it("like: case-sensitive on both paths (HELLO excluded)", async () => { + const where = fn("like", ref("body"), val("hello%")) + const snap = await snapshotMembers("pp-like-snap", where) + const delta = await deltaMembers("pp-like-delta", where) + expect(snap, `like mismatch — snap=${snap} delta=${delta}`).toEqual(delta) + expect(snap).toEqual(["lo"]) + }) + + // gt: two edges, both agreeing across paths. (1) NULL: SQL NULL > 'a' is NULL → + // n1 excluded; JS toBooleanPredicate(null) → false → excluded. (2) Case ordering: + // both SQLite BINARY collation and JS string comparison are byte/code-unit based, + // so "HELLO" (H=0x48) < "a" (0x61) → up excluded, while "hello"/"x" > "a" → in. + // Members are ["lo","x"] on BOTH paths — another case-sensitivity agreement. + it("gt: NULL excluded (three-valued) and uppercase sorts below 'a' on both paths", async () => { + const where = fn("gt", ref("body"), val("a")) + const snap = await snapshotMembers("pp-gt-snap", where) + const delta = await deltaMembers("pp-gt-delta", where) + const deltaExplicit = await deltaMembers("pp-gt-delta-explicit", where, true) + expect(snap, `gt mismatch — snap=${snap} delta=${delta}`).toEqual(delta) + expect(delta, `gt null-spelling mismatch — omit=${delta} explicit=${deltaExplicit}`).toEqual(deltaExplicit) + expect(snap).toEqual(["lo", "x"]) + }) + + // not(eq): the supported not-equal. SQL: NOT (NULL = 'x') → NULL → n1 excluded. JS: same. + it("not(eq): NULL body excluded under three-valued NOT", async () => { + const where = fn("not", fn("eq", ref("body"), val("x"))) + const snap = await snapshotMembers("pp-not-snap", where) + const delta = await deltaMembers("pp-not-delta", where) + const deltaExplicit = await deltaMembers("pp-not-delta-explicit", where, true) + expect(snap, `not(eq) mismatch — snap=${snap} delta=${delta}`).toEqual(delta) + expect(delta, `not(eq) null-spelling mismatch — omit=${delta} explicit=${deltaExplicit}`).toEqual(deltaExplicit) + expect(snap).toEqual(["lo", "up"]) + }) + + // in: SQL NULL IN (...) → NULL → n1 excluded; JS .includes(null) → false. "HELLO" not in list. + it("in: NULL body excluded, exact match only (no case folding)", async () => { + const where = fn("in", ref("body"), val(["hello", "x"])) + const snap = await snapshotMembers("pp-in-snap", where) + const delta = await deltaMembers("pp-in-delta", where) + const deltaExplicit = await deltaMembers("pp-in-delta-explicit", where, true) + expect(snap, `in mismatch — snap=${snap} delta=${delta}`).toEqual(delta) + expect(delta, `in null-spelling mismatch — omit=${delta} explicit=${deltaExplicit}`).toEqual(deltaExplicit) + expect(snap).toEqual(["lo", "x"]) + }) +}) diff --git a/tests/sql-compiler.test.ts b/tests/sql-compiler.test.ts index a6457a0..a081487 100644 --- a/tests/sql-compiler.test.ts +++ b/tests/sql-compiler.test.ts @@ -18,6 +18,13 @@ describe("IR -> SQL compiler (M6)", () => { expect(compileWhere(fn("like", ref("body"), val("h%")))).toEqual({ sql: `"body" LIKE ?`, params: ["h%"] }) }) + it("rejects a non-string `like` pattern (SQL would coerce, @tanstack/db would not — ADR-0013)", () => { + // SQLite LIKE coerces 123 → '123' and could match; the JS evaluator returns + // false for a non-string pattern. Reject so the two paths can't diverge. + expect(() => compileWhere(fn("like", ref("body"), val(123)))).toThrow(/pattern must be a string/) + expect(() => compileWhere(fn("like", ref("body"), val(null)))).toThrow(UnsupportedPredicateError) + }) + it("compiles and/or/not with grouping", () => { const e = fn("and", fn("gt", ref("n"), val(5)), fn("not", fn("eq", ref("k"), val("z")))) expect(compileWhere(e)).toEqual({ sql: `("n" > ? AND (NOT "k" = ?))`, params: [5, "z"] }) @@ -34,12 +41,24 @@ describe("IR -> SQL compiler (M6)", () => { expect(compileWhere(fn("in", ref("id"), val([])))).toEqual({ sql: `0`, params: [] }) }) - it("rejects operators outside the floor (ilike, isNull, functions)", () => { + it("rejects operators outside the floor (ne, ilike, isNull, functions)", () => { + // `ne` was removed from the floor (ADR-0013): @tanstack/db's evaluator has no + // `ne` (only `not(eq(...))`), so accepting it in SQL desynced the snapshot and + // delta paths. It must now be rejected like any other off-floor operator. + expect(() => compileWhere(fn("ne", ref("body"), val("x")))).toThrow(UnsupportedPredicateError) expect(() => compileWhere(fn("ilike", ref("body"), val("x")))).toThrow(UnsupportedPredicateError) expect(() => compileWhere(fn("isNull", ref("body")))).toThrow(UnsupportedPredicateError) expect(() => compileWhere(fn("upper", ref("body")))).toThrow(/not supported/) }) + it("the supported 'not equal' is not(eq(...)), which lowers correctly", () => { + // The canonical not-equal a @tanstack/db client emits — both floors agree on it. + expect(compileWhere(fn("not", fn("eq", ref("body"), val("x"))))).toEqual({ + sql: `(NOT "body" = ?)`, + params: ["x"], + }) + }) + it("rejects nested/aliased column paths", () => { expect(() => compileWhere(fn("eq", { type: "ref", path: ["a", "b"] }, val(1)))).toThrow(/nested/) }) diff --git a/tests/subscriptions.test.ts b/tests/subscriptions.test.ts new file mode 100644 index 0000000..61c0e40 --- /dev/null +++ b/tests/subscriptions.test.ts @@ -0,0 +1,42 @@ +import { describe, expect, it } from "vitest" +import { UnsupportedPredicateError } from "../src/server/sql-compiler.ts" +import { SubscriptionRegistry } from "../src/server/subscriptions.ts" + +// WHY: a filtered sub's predicate is compiled with @tanstack/db's evaluator (the +// delta/catch-up path). That JS floor MUST agree with the SQL snapshot floor, and +// a predicate it cannot compile MUST fail loud as UnsupportedPredicateError — not +// escape uncaught and hang the subscriber. This is the unit-level pin of the `ne` +// crash fix (ADR-0013): before, compileSingleRowExpression("ne") threw a raw +// QueryCompilationError that escaped handleSub past its UnsupportedPredicateError +// catch, so the client got no `reset` and waited forever. These tests pin the JS +// floor directly — the wire test (predicate-parity) covers the SQL floor, which +// now also rejects `ne` one step earlier. + +const ref = (col: string) => ({ type: "ref", path: [col] }) +const val = (value: unknown) => ({ type: "val", value }) +const fn = (name: string, ...args: Array) => ({ type: "func", name, args }) + +describe("SubscriptionRegistry predicate floor (ADR-0013)", () => { + // The registry uses ws only as a WeakMap key; no methods are called on it. + const ws = {} as unknown as WebSocket + + it("rejects an operator @tanstack/db cannot compile (`ne`) as UnsupportedPredicateError", () => { + const reg = new SubscriptionRegistry() + expect(() => reg.add(ws, "s1", "messages", fn("ne", ref("body"), val("x")))).toThrow(UnsupportedPredicateError) + }) + + it("accepts the supported not-equal form not(eq(...)) and mirrors SQL 3-valued NULL", () => { + const reg = new SubscriptionRegistry() + const sub = reg.add(ws, "s2", "messages", fn("not", fn("eq", ref("body"), val("x")))) + expect(sub.predicate({ body: "y" })).toBe(true) + expect(sub.predicate({ body: "x" })).toBe(false) + // NOT (NULL = 'x') is NULL in SQL → toBooleanPredicate collapses it to false. + expect(sub.predicate({ body: null })).toBe(false) + }) + + it("an unfiltered sub matches every row", () => { + const reg = new SubscriptionRegistry() + const sub = reg.add(ws, "s3", "messages") + expect(sub.predicate({ anything: 1 })).toBe(true) + }) +})