From 838d839c667004da00a34586b6a7f8af6530dd92 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 16:42:34 -0700 Subject: [PATCH 01/33] guard against machine-local paths in tracked files (dee-88da) Sweep confirmed no /Users, /home, /private/tmp, /var/folders, or file:// home paths remain in test vectors, fixtures, docs, or configs (the one CI caught in lore-views.test.ts was already fixed in 54aa178). Add scripts/no-machine-local-paths.sh: git-greps tracked files for machine-local absolute paths and exits non-zero if any reappear. Bare /tmp stays allowed (mktemp/os.tmpdir use it portably). Wired into 'pnpm verify' so it runs locally and in CI before tests. --- package.json | 3 ++- scripts/no-machine-local-paths.sh | 26 ++++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 1 deletion(-) create mode 100755 scripts/no-machine-local-paths.sh diff --git a/package.json b/package.json index f523793..758e98a 100644 --- a/package.json +++ b/package.json @@ -11,7 +11,8 @@ "build": "pnpm -r build", "test": "vitest run", "typecheck": "pnpm build && pnpm -r typecheck", - "verify": "pnpm test && pnpm typecheck" + "guard:paths": "bash scripts/no-machine-local-paths.sh", + "verify": "pnpm guard:paths && pnpm test && pnpm typecheck" }, "devDependencies": { "@types/node": "^22.14.0", diff --git a/scripts/no-machine-local-paths.sh b/scripts/no-machine-local-paths.sh new file mode 100755 index 0000000..71f5405 --- /dev/null +++ b/scripts/no-machine-local-paths.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +# Guard against machine-local absolute paths leaking into tracked files +# (test vectors, fixtures, docs, configs). A path like +# /Users/someone/... or /home/someone/... makes a test pass on the machine +# that wrote it and fail everywhere else, including CI. See ticket dee-88da. +# +# Bare /tmp is allowed: mktemp and os.tmpdir() use it portably. We only catch +# user-home directories and machine-specific temp roots. +set -euo pipefail + +cd "$(git rev-parse --show-toplevel)" + +pattern='/Users/|/home/[a-z]|/private/tmp|/var/folders/|file:///(Users|home)|[A-Za-z]:\\Users' + +# Scan tracked files only. Exclude this guard itself, since it names the +# patterns it forbids. +if git grep -nIE "$pattern" -- . ':!scripts/no-machine-local-paths.sh'; then + echo "" + echo "ERROR: machine-local absolute path found above." + echo "Make it machine-independent: build paths from the file's own location" + echo "(e.g. fileURLToPath(import.meta.url) in TS, Path(__file__).parent in" + echo "Python) or use a mktemp / os.tmpdir() temp directory." + exit 1 +fi + +echo "guard: no machine-local paths in tracked files" From 2e2669efac01be50af1adaea6a03b8e23864ae6e Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 16:43:59 -0700 Subject: [PATCH 02/33] dee-v8rh: make README Ninety-Second Story runnable from a clone A cold reader who clones and runs pnpm install && pnpm build has no 'lync' on PATH, so every bare 'lync ' in the headline first-mile example failed with 'command not found'. Switch the block to 'pnpm exec lync' (the same invocation the Development section and fresh-clone-smoke.sh already use) and state up front that a global/published install drops the prefix. Re-ran the whole block verbatim from a fresh worktree: init, append, verify, view transcript, append to imported, merge, view tree all succeed; merged.lync is a 2-line deduplicated union. --- README.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 8900b98..fc30cff 100644 --- a/README.md +++ b/README.md @@ -14,16 +14,19 @@ replacement in `dee-9l2l`; it is not the data model. ## Ninety-Second Story The shipped `lync` CLI has five verbs: `verify`, `merge`, `view`, `init`, and -`append`. A complete first mile looks like this: +`append`. From a fresh clone, run these from the repo root after +`pnpm install && pnpm build`; `pnpm exec lync` resolves the workspace binary. A +published or globally installed package drops the `pnpm exec` prefix and you +call `lync` directly. A complete first mile looks like this: ```bash -lync init story.lync -printf '%s\n' '{"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"deepfates","via":"example@0.1"},"parents":[],"payload":{"text":"Once..."}}' | lync append story.lync -lync verify story.lync -lync view story.lync --as transcript -printf '%s\n' '{"id":"note-2","kind":"notes/text","at":"2026-07-06T04:13:00Z","author":{"actor":"deepfates","via":"example@0.1"},"parents":["root"],"payload":{"text":"Then..."}}' | lync append imported.lync -lync merge story.lync imported.lync -o merged.lync -lync view merged.lync --as tree +pnpm exec lync init story.lync +printf '%s\n' '{"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"deepfates","via":"example@0.1"},"parents":[],"payload":{"text":"Once..."}}' | pnpm exec lync append story.lync +pnpm exec lync verify story.lync +pnpm exec lync view story.lync --as transcript +printf '%s\n' '{"id":"note-2","kind":"notes/text","at":"2026-07-06T04:13:00Z","author":{"actor":"deepfates","via":"example@0.1"},"parents":["root"],"payload":{"text":"Then..."}}' | pnpm exec lync append imported.lync +pnpm exec lync merge story.lync imported.lync -o merged.lync +pnpm exec lync view merged.lync --as tree ``` Under those verbs, every line has the same envelope: From 931fcf790d34b8310cb418208da6c5d535c5a93b Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 16:44:07 -0700 Subject: [PATCH 03/33] fix(core): model idb-log storage arrays instead of unknown[] / any-cast Replace the escape-hatch typings in idb-log storage with concrete row types. getAll now annotates IDBRequest (no 'as T[]' cast), clearAndPut is generic over the row type (no 'unknown[]'), and conflict/pending rows carry an explicit ConflictRow/PendingRow type modeling the composite [id, digest] key. dumpRecords() is called once per persist instead of three times. tsc --noEmit clean, 94 tests green. --- packages/core/src/lore/idb-log.ts | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/packages/core/src/lore/idb-log.ts b/packages/core/src/lore/idb-log.ts index d3cfe6b..eb27c2c 100644 --- a/packages/core/src/lore/idb-log.ts +++ b/packages/core/src/lore/idb-log.ts @@ -7,6 +7,15 @@ export interface IndexedDbEventStoreOptions { const VERSION = 1; +/** + * A conflict/pending record as it lives in IndexedDB: the in-memory record plus + * the composite `[id, digest]` primary key the object store is keyed on. Events + * are stored as-is (keyed on their own `id`), so they need no wrapper row type. + */ +type StoreKey = [string, string]; +type ConflictRow = ConflictRecord & { key: StoreKey }; +type PendingRow = PendingRecord & { key: StoreKey }; + export class IndexedDbEventStore extends BaseEventStore { private readonly dbName: string; private readonly idb: IDBFactory; @@ -52,10 +61,13 @@ export class IndexedDbEventStore extends BaseEventStore { protected override async persist(): Promise { const db = await openDb(this.idb, this.dbName); const tx = db.transaction(["events", "conflicts", "pending"], "readwrite"); + const records = this.dumpRecords(); + const conflicts: ConflictRow[] = records.conflicts.map((record) => ({ ...record, key: [record.id, record.digest] })); + const pending: PendingRow[] = records.pending.map((record) => ({ ...record, key: [record.missingParent, record.digest] })); await Promise.all([ - clearAndPut(tx.objectStore("events"), this.dumpRecords().events), - clearAndPut(tx.objectStore("conflicts"), this.dumpRecords().conflicts.map((record) => ({ ...record, key: [record.id, record.digest] }))), - clearAndPut(tx.objectStore("pending"), this.dumpRecords().pending.map((record) => ({ ...record, key: [record.missingParent, record.digest] }))), + clearAndPut(tx.objectStore("events"), records.events), + clearAndPut(tx.objectStore("conflicts"), conflicts), + clearAndPut(tx.objectStore("pending"), pending), txDone(tx), ]); db.close(); @@ -66,8 +78,8 @@ export class IndexedDbEventStore extends BaseEventStore { const tx = db.transaction(["events", "conflicts", "pending"], "readonly"); const [events, conflicts, pending] = await Promise.all([ getAll(tx.objectStore("events")), - getAll(tx.objectStore("conflicts")), - getAll(tx.objectStore("pending")), + getAll(tx.objectStore("conflicts")), + getAll(tx.objectStore("pending")), txDone(tx), ]); await this.loadRecords({ events, conflicts, pending }); @@ -104,13 +116,13 @@ function openDb(idb: IDBFactory, dbName: string): Promise { function getAll(store: IDBObjectStore): Promise { return new Promise((resolve, reject) => { - const req = store.getAll(); - req.onsuccess = () => resolve(req.result as T[]); + const req: IDBRequest = store.getAll(); + req.onsuccess = () => resolve(req.result); req.onerror = () => reject(req.error); }); } -async function clearAndPut(store: IDBObjectStore, records: unknown[]): Promise { +async function clearAndPut(store: IDBObjectStore, records: readonly T[]): Promise { await requestDone(store.clear()); for (const record of records) await requestDone(store.put(record)); } From 336975f448d5f1a032e3ba896959fb97d75117d3 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 16:45:07 -0700 Subject: [PATCH 04/33] lync: prep @lync/core + @lync/cli for public npm publish - add publishConfig.access=public to both scoped packages (scoped packages default to restricted; publish would fail without this) - fix @lync/cli bin path (drop leading ./) so npm publish stops auto-correcting bin[lync] and warning it was removed - add repository/homepage/bugs metadata for usable npm pages - add per-package README so npm package pages are not bare Proven in a clean-room: pnpm-packed tarballs installed in a fresh npm project (no workspace links) exercise append/verify/lossless divergent merge via @lync/core and all five CLI verbs via the lync bin. --- packages/cli/README.md | 17 +++++++++++++++++ packages/cli/package.json | 12 +++++++++++- packages/core/README.md | 27 +++++++++++++++++++++++++++ packages/core/package.json | 10 ++++++++++ 4 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 packages/cli/README.md create mode 100644 packages/core/README.md diff --git a/packages/cli/README.md b/packages/cli/README.md new file mode 100644 index 0000000..031bdd8 --- /dev/null +++ b/packages/cli/README.md @@ -0,0 +1,17 @@ +# @lync/cli + +Command-line tools for lync files: append-only loom logs, one JSON event per +line, merged losslessly by set-union. + +```bash +npm install -g @lync/cli + +lync init story.lync +printf '%s\n' '{"kind":"notes/text","author":{"actor":"you"},"payload":{"text":"Once..."}}' | lync append story.lync +lync verify story.lync +lync view story.lync --as transcript +lync merge story.lync other.lync -o merged.lync +``` + +Five verbs: `init`, `append`, `verify`, `merge`, `view`. Run `lync --help` for +usage. Full docs: https://github.com/deepfates/lync#readme diff --git a/packages/cli/package.json b/packages/cli/package.json index 405e48f..757de25 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -5,8 +5,18 @@ "type": "module", "license": "MIT", "sideEffects": false, + "repository": { + "type": "git", + "url": "git+https://github.com/deepfates/lync.git", + "directory": "packages/cli" + }, + "homepage": "https://github.com/deepfates/lync#readme", + "bugs": "https://github.com/deepfates/lync/issues", + "publishConfig": { + "access": "public" + }, "bin": { - "lync": "./bin/lync.js" + "lync": "bin/lync.js" }, "main": "./dist/index.js", "types": "./dist/index.d.ts", diff --git a/packages/core/README.md b/packages/core/README.md new file mode 100644 index 0000000..a1bcfa9 --- /dev/null +++ b/packages/core/README.md @@ -0,0 +1,27 @@ +# @lync/core + +Core APIs for lync files: append-only loom logs where each line is one JSON +event, files are merged losslessly by set-union, and every physical line is +classified and kept. + +```ts +import { parseLoreFiles } from "@lync/core/lore/events"; +import { createMemoryEventStore } from "@lync/core/lore/memory-log"; + +const store = createMemoryEventStore(); +await store.append({ + v: 1, + id: "root", + kind: "lync/loom", + at: "2026-07-06T04:12:31Z", + author: { actor: "you" }, + parents: [], + payload: { meta: { title: "Story" } }, +}); + +const parsed = parseLoreFiles([{ file: "story.lync", bytes: line }]); +console.log(parsed.unionEventIds); +``` + +Full format spec, subpath exports, and examples: +https://github.com/deepfates/lync#readme diff --git a/packages/core/package.json b/packages/core/package.json index 8ed9654..e8aaf8f 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -5,6 +5,16 @@ "type": "module", "license": "MIT", "sideEffects": false, + "repository": { + "type": "git", + "url": "git+https://github.com/deepfates/lync.git", + "directory": "packages/core" + }, + "homepage": "https://github.com/deepfates/lync#readme", + "bugs": "https://github.com/deepfates/lync/issues", + "publishConfig": { + "access": "public" + }, "main": "./dist/index.js", "types": "./dist/index.d.ts", "exports": { From fd9d96320024b8f9fe77bbca8dce761250a30114 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 16:47:30 -0700 Subject: [PATCH 05/33] fix(core): keep node builtins off the browser lore path (dee-mm7n) The browser lore bundle pulled node:crypto, node:fs, and node:path. Two independent leaks, both fixed at the source (no fs shims, no fakes): - node:crypto: events.ts and store.ts hashed via createHash("sha256"). Replaced with a dependency-free, synchronous pure-TS SHA-256 (lore/sha256.ts) that runs identically in Node and the browser. Verified byte-for-byte against node:crypto across NIST vectors, every padding-block boundary (len 0..200), random fuzz, and subarray views (lore-sha256.test.ts). - node:fs/node:path: file-log.ts is a genuinely node-only file store, but looms.ts statically imported it (for createFileLoreLooms) and index.ts re-exported it, poisoning every browser-reachable entry and the main barrel. Moved createFileLoreLooms into file-log.ts (its node-only home) and dropped file-log from the index barrel. The node file store now lives only at the explicit @lync/core/lore/file-log subpath. Result: @lync/core (barrel), /lore/events, /store, /idb-log, /views, /looms, /memory-log, and @lync/client/browser all bundle for platform=browser with zero node builtins. Wire kind strings (lore/*) untouched. README updated to import createFileLoreLooms from the file-log subpath. --- README.md | 7 +- packages/core/src/index.ts | 3 +- packages/core/src/lore/events.ts | 6 +- packages/core/src/lore/file-log.ts | 12 ++++ packages/core/src/lore/looms.ts | 9 --- packages/core/src/lore/sha256.ts | 93 ++++++++++++++++++++++++++ packages/core/src/lore/store.ts | 4 +- packages/core/test/lore-sha256.test.ts | 41 ++++++++++++ 8 files changed, 155 insertions(+), 20 deletions(-) create mode 100644 packages/core/src/lore/sha256.ts create mode 100644 packages/core/test/lore-sha256.test.ts diff --git a/README.md b/README.md index 8900b98..f83b9c8 100644 --- a/README.md +++ b/README.md @@ -80,9 +80,9 @@ vocabulary. ```ts import { LoreUnion, exportCarriedLoreBytes, parseLoreFiles } from "@lync/core/lore/events"; -import { createFileEventStore } from "@lync/core/lore/file-log"; +import { createFileEventStore, createFileLoreLooms } from "@lync/core/lore/file-log"; import { createIndexedDbEventStore } from "@lync/core/lore/idb-log"; -import { createLoreLooms, createFileLoreLooms, createBrowserLoreLooms } from "@lync/core/lore/looms"; +import { createLoreLooms, createBrowserLoreLooms } from "@lync/core/lore/looms"; import { createMemoryEventStore } from "@lync/core/lore/memory-log"; import { BaseEventStore, serializeLoreEvent } from "@lync/core/lore/store"; import { @@ -99,7 +99,8 @@ The seven format-layer package exports are: incremental union. - `@lync/core/lore/memory-log`: in-memory event store for tests and embedded runtimes. -- `@lync/core/lore/file-log`: file-backed event store. +- `@lync/core/lore/file-log`: file-backed event store and `createFileLoreLooms` + (node-only; keeps `node:fs`/`node:path` off the browser path). - `@lync/core/lore/idb-log`: IndexedDB-backed event store. - `@lync/core/lore/store`: base event-store contract and serialization helpers. - `@lync/core/lore/views`: branch tree, transcript, memory, and leaderboard diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 6bef074..a3fe7e2 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -1,5 +1,6 @@ export * from "./errors.js"; -export * from "./lore/file-log.js"; +// The node:fs-backed file store lives only at the explicit "@lync/core/lore/file-log" +// subpath so the main barrel stays importable in the browser with zero node builtins. export * from "./lore/idb-log.js"; export * from "./lore/looms.js"; export * from "./lore/memory-log.js"; diff --git a/packages/core/src/lore/events.ts b/packages/core/src/lore/events.ts index 4c632c0..b282512 100644 --- a/packages/core/src/lore/events.ts +++ b/packages/core/src/lore/events.ts @@ -1,4 +1,4 @@ -import { createHash } from "node:crypto"; +import { sha256Hex } from "./sha256.js"; export type LoreLineClass = | "accepted" @@ -735,10 +735,6 @@ function parseJsonNoDuplicateKeys(text: string): JsonParsed { return { value }; } -function sha256Hex(bytes: Uint8Array): string { - return createHash("sha256").update(bytes).digest("hex"); -} - function bytesEqual(a: Uint8Array | undefined, b: Uint8Array | undefined): boolean { if (!a || !b || a.byteLength !== b.byteLength) return false; for (let i = 0; i < a.byteLength; i++) if (a[i] !== b[i]) return false; diff --git a/packages/core/src/lore/file-log.ts b/packages/core/src/lore/file-log.ts index e72566a..64d9e47 100644 --- a/packages/core/src/lore/file-log.ts +++ b/packages/core/src/lore/file-log.ts @@ -1,5 +1,7 @@ import fs from "node:fs/promises"; import path from "node:path"; +import type { Looms } from "../types.js"; +import { createLoreLooms, type LoreLoomsOptions } from "./looms.js"; import { BaseEventStore, type GarbageRecord } from "./store.js"; const STORE_FILE = "events.json"; @@ -100,6 +102,16 @@ export function createFileEventStore(dir: string): FileEventStore { return new FileEventStore({ dir }); } +// File-backed looms live here, not in looms.ts, so the browser-reachable +// modules never statically import this node:fs/node:path file. +export function createFileLoreLooms< + TPayload = unknown, + TLoomMeta = unknown, + TTurnMeta = unknown, +>(dir: string, options: Omit): Looms { + return createLoreLooms({ ...options, store: createFileEventStore(dir) }); +} + function isEventFile(file: string): boolean { return EVENT_FILE_EXTENSIONS.some((extension) => file.endsWith(extension)); } diff --git a/packages/core/src/lore/looms.ts b/packages/core/src/lore/looms.ts index c9d6b6a..797e462 100644 --- a/packages/core/src/lore/looms.ts +++ b/packages/core/src/lore/looms.ts @@ -20,7 +20,6 @@ import type { TurnId, } from "../types.js"; import type { LoreEventBody } from "./events.js"; -import { createFileEventStore } from "./file-log.js"; import { createIndexedDbEventStore, type IndexedDbEventStoreOptions } from "./idb-log.js"; import type { EventStore, StoredEvent } from "./store.js"; @@ -144,14 +143,6 @@ export function createLoreLooms< }; } -export function createFileLoreLooms< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, ->(dir: string, options: Omit): Looms { - return createLoreLooms({ ...options, store: createFileEventStore(dir) }); -} - export function createBrowserLoreLooms< TPayload = unknown, TLoomMeta = unknown, diff --git a/packages/core/src/lore/sha256.ts b/packages/core/src/lore/sha256.ts new file mode 100644 index 0000000..6ce3969 --- /dev/null +++ b/packages/core/src/lore/sha256.ts @@ -0,0 +1,93 @@ +// SHA-256 in pure TypeScript: synchronous, dependency-free, and byte-for-byte +// identical in Node and the browser. Lore hashing runs on the same code path +// everywhere, so this file never reaches for node:crypto and the browser +// bundle stays free of node builtins. WebCrypto's subtle.digest is async and +// cannot back the synchronous line parser, which is why we implement it here. + +const K = new Uint32Array([ + 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, + 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, + 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, + 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, + 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, + 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, + 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, + 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2, +]); + +export function sha256Hex(bytes: Uint8Array): string { + let h0 = 0x6a09e667; + let h1 = 0xbb67ae85; + let h2 = 0x3c6ef372; + let h3 = 0xa54ff53a; + let h4 = 0x510e527f; + let h5 = 0x9b05688c; + let h6 = 0x1f83d9ab; + let h7 = 0x5be0cd19; + + const bitLen = bytes.length * 8; + const withOne = bytes.length + 1; + const total = withOne + ((56 - (withOne % 64) + 64) % 64) + 8; + const msg = new Uint8Array(total); + msg.set(bytes); + msg[bytes.length] = 0x80; + + const view = new DataView(msg.buffer); + view.setUint32(total - 8, Math.floor(bitLen / 0x100000000)); + view.setUint32(total - 4, bitLen >>> 0); + + const w = new Uint32Array(64); + for (let off = 0; off < total; off += 64) { + for (let i = 0; i < 16; i++) w[i] = view.getUint32(off + i * 4); + for (let i = 16; i < 64; i++) { + const s0 = rotr(w[i - 15], 7) ^ rotr(w[i - 15], 18) ^ (w[i - 15] >>> 3); + const s1 = rotr(w[i - 2], 17) ^ rotr(w[i - 2], 19) ^ (w[i - 2] >>> 10); + w[i] = (w[i - 16] + s0 + w[i - 7] + s1) | 0; + } + + let a = h0; + let b = h1; + let c = h2; + let d = h3; + let e = h4; + let f = h5; + let g = h6; + let h = h7; + + for (let i = 0; i < 64; i++) { + const s1 = rotr(e, 6) ^ rotr(e, 11) ^ rotr(e, 25); + const ch = (e & f) ^ (~e & g); + const t1 = (h + s1 + ch + K[i] + w[i]) | 0; + const s0 = rotr(a, 2) ^ rotr(a, 13) ^ rotr(a, 22); + const maj = (a & b) ^ (a & c) ^ (b & c); + const t2 = (s0 + maj) | 0; + h = g; + g = f; + f = e; + e = (d + t1) | 0; + d = c; + c = b; + b = a; + a = (t1 + t2) | 0; + } + + h0 = (h0 + a) | 0; + h1 = (h1 + b) | 0; + h2 = (h2 + c) | 0; + h3 = (h3 + d) | 0; + h4 = (h4 + e) | 0; + h5 = (h5 + f) | 0; + h6 = (h6 + g) | 0; + h7 = (h7 + h) | 0; + } + + return toHex8(h0) + toHex8(h1) + toHex8(h2) + toHex8(h3) + toHex8(h4) + toHex8(h5) + toHex8(h6) + toHex8(h7); +} + +function rotr(x: number, n: number): number { + return (x >>> n) | (x << (32 - n)); +} + +function toHex8(x: number): string { + return (x >>> 0).toString(16).padStart(8, "0"); +} diff --git a/packages/core/src/lore/store.ts b/packages/core/src/lore/store.ts index da2628d..ce873e1 100644 --- a/packages/core/src/lore/store.ts +++ b/packages/core/src/lore/store.ts @@ -1,6 +1,6 @@ -import { createHash } from "node:crypto"; import type { LoreEventBody } from "./events.js"; import { parseLoreFiles } from "./events.js"; +import { sha256Hex } from "./sha256.js"; export interface StoredEvent { body: LoreEventBody; @@ -317,7 +317,7 @@ function stripSplice(line: string): string { } function bodyDigest(bytes: Uint8Array): string { - return createHash("sha256").update(bytes).digest("hex"); + return sha256Hex(bytes); } function conflictKey(id: string, digest: string): string { diff --git a/packages/core/test/lore-sha256.test.ts b/packages/core/test/lore-sha256.test.ts new file mode 100644 index 0000000..ddaddf1 --- /dev/null +++ b/packages/core/test/lore-sha256.test.ts @@ -0,0 +1,41 @@ +import { createHash, randomBytes } from "node:crypto"; +import { describe, expect, it } from "vitest"; +import { sha256Hex } from "../src/lore/sha256.js"; + +function nodeHex(bytes: Uint8Array): string { + return createHash("sha256").update(bytes).digest("hex"); +} + +describe("sha256Hex", () => { + it("matches published NIST vectors", () => { + expect(sha256Hex(new Uint8Array(0))).toBe( + "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + ); + expect(sha256Hex(new TextEncoder().encode("abc"))).toBe( + "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad", + ); + }); + + it("agrees with node:crypto across every padding-block boundary", () => { + // Lengths 0..200 cover the 55/56-byte and 119/120-byte padding edges where + // a length that pushes the 64-bit count into a fresh block is easy to break. + for (let len = 0; len <= 200; len++) { + const bytes = new Uint8Array(len); + for (let i = 0; i < len; i++) bytes[i] = (i * 31 + len * 7) & 0xff; + expect(sha256Hex(bytes)).toBe(nodeHex(bytes)); + } + }); + + it("agrees with node:crypto on random inputs", () => { + for (let t = 0; t < 500; t++) { + const bytes = new Uint8Array(randomBytes(Math.floor(Math.random() * 1024))); + expect(sha256Hex(bytes)).toBe(nodeHex(bytes)); + } + }); + + it("hashes only the viewed window of a subarray", () => { + const backing = new Uint8Array(100).map((_, i) => i & 0xff); + const view = backing.subarray(10, 40); + expect(sha256Hex(view)).toBe(nodeHex(view)); + }); +}); From d5a335aff5b8a575c2792ec0f3f0c72327e37834 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 17:01:32 -0700 Subject: [PATCH 06/33] rename packages to unscoped lync-core / lync-cli (owner ruling 2026-07-08): @lync npm org unavailable; prefix-as-namespace across workspace, docs, imports --- README.md | 48 +++++++++++++-------------- package.json | 2 +- packages/cli/README.md | 4 +-- packages/cli/package.json | 4 +-- packages/cli/src/index.ts | 4 +-- packages/client/package.json | 6 ++-- packages/client/src/browser.ts | 6 ++-- packages/client/src/create.ts | 4 +-- packages/client/src/node.ts | 4 +-- packages/client/src/testing.ts | 4 +-- packages/client/src/types.ts | 4 +-- packages/client/test/testing.test.ts | 2 +- packages/core/README.md | 6 ++-- packages/core/package.json | 2 +- packages/core/src/index.ts | 2 +- packages/index/package.json | 4 +-- packages/index/src/automerge.ts | 4 +-- packages/index/src/entries.ts | 2 +- packages/index/src/memory.ts | 4 +-- packages/index/src/types.ts | 2 +- packages/index/test/automerge.test.ts | 2 +- packages/index/test/memory.test.ts | 2 +- packages/sync-server/package.json | 2 +- pnpm-lock.yaml | 20 +++++------ 24 files changed, 72 insertions(+), 72 deletions(-) diff --git a/README.md b/README.md index 17214c0..1f1c32b 100644 --- a/README.md +++ b/README.md @@ -68,12 +68,12 @@ The short version: ## Packages -- `@lync/core`: format parsing, event stores, computed views, references, and +- `lync-core`: format parsing, event stores, computed views, references, and the compatibility loom API. -- `@lync/client`: browser, Node, and test runtime clients for the compatibility +- `lync-client`: browser, Node, and test runtime clients for the compatibility API. -- `@lync/sync-server`: the current Automerge WebSocket relay. -- `@lync/index`: legacy synced indexes of loom references. +- `lync-sync-server`: the current Automerge WebSocket relay. +- `lync-index`: legacy synced indexes of loom references. ## Format-Layer Imports @@ -82,39 +82,39 @@ compatibility. Import them by path; do not treat those path segments as public vocabulary. ```ts -import { LoreUnion, exportCarriedLoreBytes, parseLoreFiles } from "@lync/core/lore/events"; -import { createFileEventStore, createFileLoreLooms } from "@lync/core/lore/file-log"; -import { createIndexedDbEventStore } from "@lync/core/lore/idb-log"; -import { createLoreLooms, createBrowserLoreLooms } from "@lync/core/lore/looms"; -import { createMemoryEventStore } from "@lync/core/lore/memory-log"; -import { BaseEventStore, serializeLoreEvent } from "@lync/core/lore/store"; +import { LoreUnion, exportCarriedLoreBytes, parseLoreFiles } from "lync-core/lore/events"; +import { createFileEventStore, createFileLoreLooms } from "lync-core/lore/file-log"; +import { createIndexedDbEventStore } from "lync-core/lore/idb-log"; +import { createLoreLooms, createBrowserLoreLooms } from "lync-core/lore/looms"; +import { createMemoryEventStore } from "lync-core/lore/memory-log"; +import { BaseEventStore, serializeLoreEvent } from "lync-core/lore/store"; import { loreBranchTreeView, loreLeaderboardView, loreMemoryView, loreTranscriptView, -} from "@lync/core/lore/views"; +} from "lync-core/lore/views"; ``` The seven format-layer package exports are: -- `@lync/core/lore/events`: line parsing, carried-byte export, downsets, and +- `lync-core/lore/events`: line parsing, carried-byte export, downsets, and incremental union. -- `@lync/core/lore/memory-log`: in-memory event store for tests and embedded +- `lync-core/lore/memory-log`: in-memory event store for tests and embedded runtimes. -- `@lync/core/lore/file-log`: file-backed event store and `createFileLoreLooms` +- `lync-core/lore/file-log`: file-backed event store and `createFileLoreLooms` (node-only; keeps `node:fs`/`node:path` off the browser path). -- `@lync/core/lore/idb-log`: IndexedDB-backed event store. -- `@lync/core/lore/store`: base event-store contract and serialization helpers. -- `@lync/core/lore/views`: branch tree, transcript, memory, and leaderboard +- `lync-core/lore/idb-log`: IndexedDB-backed event store. +- `lync-core/lore/store`: base event-store contract and serialization helpers. +- `lync-core/lore/views`: branch tree, transcript, memory, and leaderboard view helpers. -- `@lync/core/lore/looms`: compatibility loom API backed by event stores. +- `lync-core/lore/looms`: compatibility loom API backed by event stores. ## Parse, Union, View ```ts -import { parseLoreFiles } from "@lync/core/lore/events"; -import { loreBranchTreeView, loreMemoryView } from "@lync/core/lore/views"; +import { parseLoreFiles } from "lync-core/lore/events"; +import { loreBranchTreeView, loreMemoryView } from "lync-core/lore/views"; const bytes = new TextEncoder().encode( '{"v":1,"id":"a","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"Once..."}}\n', @@ -137,7 +137,7 @@ their first missing parent arrives. ## Storage ```ts -import { createMemoryEventStore } from "@lync/core/lore/memory-log"; +import { createMemoryEventStore } from "lync-core/lore/memory-log"; const store = createMemoryEventStore(); await store.append({ @@ -173,8 +173,8 @@ The loom API remains for existing users and for the current Automerge-backed clients. It now has an event-store implementation: ```ts -import { createLoreLooms } from "@lync/core/lore/looms"; -import { createMemoryEventStore } from "@lync/core/lore/memory-log"; +import { createLoreLooms } from "lync-core/lore/looms"; +import { createMemoryEventStore } from "lync-core/lore/memory-log"; const looms = createLoreLooms<{ text: string }, { title: string }>({ store: createMemoryEventStore(), @@ -212,7 +212,7 @@ exports can be mixed with newly migrated roots during a transition. ## Sync Server -`@lync/sync-server` provides the current Automerge WebSocket relay. Its default +`lync-sync-server` provides the current Automerge WebSocket relay. Its default WebSocket path is `/lync`. The exported factory is `createLyncServer`. `authenticate` is synchronous by design in the server API. Return `false` to diff --git a/package.json b/package.json index 758e98a..cbe4de7 100644 --- a/package.json +++ b/package.json @@ -18,7 +18,7 @@ "@types/node": "^22.14.0", "typescript": "^5.8.3", "vitest": "^3.1.1", - "@lync/cli": "workspace:*" + "lync-cli": "workspace:*" }, "dependencies": { "@automerge/automerge-repo": "^2.5.5", diff --git a/packages/cli/README.md b/packages/cli/README.md index 031bdd8..25755b5 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -1,10 +1,10 @@ -# @lync/cli +# lync-cli Command-line tools for lync files: append-only loom logs, one JSON event per line, merged losslessly by set-union. ```bash -npm install -g @lync/cli +npm install -g lync-cli lync init story.lync printf '%s\n' '{"kind":"notes/text","author":{"actor":"you"},"payload":{"text":"Once..."}}' | lync append story.lync diff --git a/packages/cli/package.json b/packages/cli/package.json index 757de25..3f061d6 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,5 +1,5 @@ { - "name": "@lync/cli", + "name": "lync-cli", "version": "0.2.0", "description": "Command-line tools for lync files.", "type": "module", @@ -36,6 +36,6 @@ "typecheck": "tsc -p tsconfig.json --noEmit" }, "dependencies": { - "@lync/core": "workspace:*" + "lync-core": "workspace:*" } } diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index 0b1cc55..a603f68 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -4,8 +4,8 @@ import { parseLoreFiles, type LoreLineClass, type LoreLineDiagnostic, -} from "@lync/core/lore/events"; -import { loreBranchTreeView as coreTreeView, loreTranscriptView as coreTranscriptView } from "@lync/core/lore/views"; +} from "lync-core/lore/events"; +import { loreBranchTreeView as coreTreeView, loreTranscriptView as coreTranscriptView } from "lync-core/lore/views"; export interface LyncCliIO { stdout?: Pick; diff --git a/packages/client/package.json b/packages/client/package.json index 5ddb0a7..5e4e492 100644 --- a/packages/client/package.json +++ b/packages/client/package.json @@ -1,5 +1,5 @@ { - "name": "@lync/client", + "name": "lync-client", "version": "0.2.0", "description": "Runtime clients for Lync browser, Node, and tests.", "type": "module", @@ -44,8 +44,8 @@ "dependencies": { "@automerge/automerge-repo": "^2.5.5", "@automerge/automerge-repo-network-websocket": "^2.5.5", - "@lync/core": "workspace:*", - "@lync/index": "workspace:*", + "lync-core": "workspace:*", + "lync-index": "workspace:*", "isomorphic-ws": "^5.0.0" } } diff --git a/packages/client/src/browser.ts b/packages/client/src/browser.ts index 12acca9..a26ef28 100644 --- a/packages/client/src/browser.ts +++ b/packages/client/src/browser.ts @@ -2,15 +2,15 @@ import type { Repo } from "@automerge/automerge-repo"; import { createAutomergeLooms, type AutomergeLoomsOptions, -} from "@lync/core/automerge"; +} from "lync-core/automerge"; import { createBrowserAutomergeRepo, type BrowserAutomergeRepoOptions, -} from "@lync/core/browser"; +} from "lync-core/browser"; import { createAutomergeLoomIndexes, type AutomergeLoomIndexesOptions, -} from "@lync/index/automerge"; +} from "lync-index/automerge"; import { createLoomClient } from "./create.js"; import type { LoomClient } from "./types.js"; diff --git a/packages/client/src/create.ts b/packages/client/src/create.ts index 1c3bd14..707a535 100644 --- a/packages/client/src/create.ts +++ b/packages/client/src/create.ts @@ -12,8 +12,8 @@ import { turnRef, type LoomReference, type Looms, -} from "@lync/core"; -import type { LoomIndexes } from "@lync/index"; +} from "lync-core"; +import type { LoomIndexes } from "lync-index"; import type { LoomClient } from "./types.js"; export interface CreateLoomClientOptions< diff --git a/packages/client/src/node.ts b/packages/client/src/node.ts index 565991c..db53ad2 100644 --- a/packages/client/src/node.ts +++ b/packages/client/src/node.ts @@ -10,11 +10,11 @@ import { import { createAutomergeLooms, type AutomergeLoomsOptions, -} from "@lync/core/automerge"; +} from "lync-core/automerge"; import { createAutomergeLoomIndexes, type AutomergeLoomIndexesOptions, -} from "@lync/index/automerge"; +} from "lync-index/automerge"; import { createLoomClient } from "./create.js"; import { createWebSocketSyncAdapter, diff --git a/packages/client/src/testing.ts b/packages/client/src/testing.ts index a3474c0..f5dda57 100644 --- a/packages/client/src/testing.ts +++ b/packages/client/src/testing.ts @@ -1,5 +1,5 @@ -import { createMemoryLooms, type MemoryLoomsOptions } from "@lync/core/memory"; -import { createMemoryLoomIndexes, type MemoryLoomIndexesOptions } from "@lync/index/memory"; +import { createMemoryLooms, type MemoryLoomsOptions } from "lync-core/memory"; +import { createMemoryLoomIndexes, type MemoryLoomIndexesOptions } from "lync-index/memory"; import { createLoomClient } from "./create.js"; import type { LoomClient } from "./types.js"; diff --git a/packages/client/src/types.ts b/packages/client/src/types.ts index 67dca41..e8ecc7d 100644 --- a/packages/client/src/types.ts +++ b/packages/client/src/types.ts @@ -13,8 +13,8 @@ import type { encodeReference, decodeReference, parseReference, -} from "@lync/core"; -import type { LoomIndex, LoomIndexes } from "@lync/index"; +} from "lync-core"; +import type { LoomIndex, LoomIndexes } from "lync-index"; export type ReferenceHelpers = { loom: typeof loomRef; diff --git a/packages/client/test/testing.test.ts b/packages/client/test/testing.test.ts index af8d899..575005c 100644 --- a/packages/client/test/testing.test.ts +++ b/packages/client/test/testing.test.ts @@ -5,7 +5,7 @@ import { type TextStoryLoomMeta, type TextStoryTurnMeta, type TextStoryTurnPayload, -} from "@lync/core/profiles/text-story"; +} from "lync-core/profiles/text-story"; import { createTestLoomClient } from "../src/testing.js"; describe("test loom client", () => { diff --git a/packages/core/README.md b/packages/core/README.md index a1bcfa9..169c538 100644 --- a/packages/core/README.md +++ b/packages/core/README.md @@ -1,12 +1,12 @@ -# @lync/core +# lync-core Core APIs for lync files: append-only loom logs where each line is one JSON event, files are merged losslessly by set-union, and every physical line is classified and kept. ```ts -import { parseLoreFiles } from "@lync/core/lore/events"; -import { createMemoryEventStore } from "@lync/core/lore/memory-log"; +import { parseLoreFiles } from "lync-core/lore/events"; +import { createMemoryEventStore } from "lync-core/lore/memory-log"; const store = createMemoryEventStore(); await store.append({ diff --git a/packages/core/package.json b/packages/core/package.json index e8aaf8f..f4bd128 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -1,5 +1,5 @@ { - "name": "@lync/core", + "name": "lync-core", "version": "0.2.0", "description": "Core APIs for local-first addressable looms.", "type": "module", diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index a3fe7e2..9a4626b 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -1,5 +1,5 @@ export * from "./errors.js"; -// The node:fs-backed file store lives only at the explicit "@lync/core/lore/file-log" +// The node:fs-backed file store lives only at the explicit "lync-core/lore/file-log" // subpath so the main barrel stays importable in the browser with zero node builtins. export * from "./lore/idb-log.js"; export * from "./lore/looms.js"; diff --git a/packages/index/package.json b/packages/index/package.json index 045bc30..97f05a2 100644 --- a/packages/index/package.json +++ b/packages/index/package.json @@ -1,5 +1,5 @@ { - "name": "@lync/index", + "name": "lync-index", "version": "0.2.0", "description": "Index APIs for linked Lync looms.", "type": "module", @@ -42,6 +42,6 @@ "typecheck": "tsc -p tsconfig.json --noEmit" }, "dependencies": { - "@lync/core": "workspace:*" + "lync-core": "workspace:*" } } diff --git a/packages/index/src/automerge.ts b/packages/index/src/automerge.ts index 3a37672..c814443 100644 --- a/packages/index/src/automerge.ts +++ b/packages/index/src/automerge.ts @@ -1,6 +1,6 @@ import { DocHandle, Repo, type AutomergeUrl } from "@automerge/automerge-repo"; -import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "@lync/core"; -import type { IndexId, LoomId, LoomReference } from "@lync/core"; +import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "lync-core"; +import type { IndexId, LoomId, LoomReference } from "lync-core"; import type { LoomIndex, LoomIndexEntry, diff --git a/packages/index/src/entries.ts b/packages/index/src/entries.ts index 4733e63..f7405cb 100644 --- a/packages/index/src/entries.ts +++ b/packages/index/src/entries.ts @@ -1,4 +1,4 @@ -import type { LoomReference } from "@lync/core"; +import type { LoomReference } from "lync-core"; import type { LoomIndex, LoomIndexEntry, diff --git a/packages/index/src/memory.ts b/packages/index/src/memory.ts index 7151d88..a4c0dc5 100644 --- a/packages/index/src/memory.ts +++ b/packages/index/src/memory.ts @@ -1,5 +1,5 @@ -import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "@lync/core"; -import type { IndexId, LoomId, LoomReference } from "@lync/core"; +import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "lync-core"; +import type { IndexId, LoomId, LoomReference } from "lync-core"; import type { LoomIndex, LoomIndexEntry, diff --git a/packages/index/src/types.ts b/packages/index/src/types.ts index 439595e..a65448f 100644 --- a/packages/index/src/types.ts +++ b/packages/index/src/types.ts @@ -1,4 +1,4 @@ -import type { IndexId, LoomId, LoomReference } from "@lync/core"; +import type { IndexId, LoomId, LoomReference } from "lync-core"; export interface LoomIndexInfo { id: IndexId; diff --git a/packages/index/test/automerge.test.ts b/packages/index/test/automerge.test.ts index 512dcb7..b8f5bbb 100644 --- a/packages/index/test/automerge.test.ts +++ b/packages/index/test/automerge.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; import { Repo } from "@automerge/automerge-repo"; -import { loomRef } from "@lync/core"; +import { loomRef } from "lync-core"; import { createAutomergeLoomIndexes } from "../src/automerge.js"; function deterministicAutomergeIndexes() { diff --git a/packages/index/test/memory.test.ts b/packages/index/test/memory.test.ts index 443e24c..c09a2be 100644 --- a/packages/index/test/memory.test.ts +++ b/packages/index/test/memory.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { loomRef } from "@lync/core"; +import { loomRef } from "lync-core"; import { createMemoryLoomIndexes } from "../src/memory.js"; import { upsertLoom } from "../src/entries.js"; diff --git a/packages/sync-server/package.json b/packages/sync-server/package.json index f84e65b..eadbb72 100644 --- a/packages/sync-server/package.json +++ b/packages/sync-server/package.json @@ -1,5 +1,5 @@ { - "name": "@lync/sync-server", + "name": "lync-sync-server", "version": "0.1.0", "description": "Automerge WebSocket sync relay for Lync.", "type": "module", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index af5f2ba..d1229f9 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -30,12 +30,12 @@ importers: specifier: ^14.0.0 version: 14.0.0 devDependencies: - '@lync/cli': - specifier: workspace:* - version: link:packages/cli '@types/node': specifier: ^22.14.0 version: 22.19.17 + lync-cli: + specifier: workspace:* + version: link:packages/cli typescript: specifier: ^5.8.3 version: 5.9.3 @@ -45,7 +45,7 @@ importers: packages/cli: dependencies: - '@lync/core': + lync-core: specifier: workspace:* version: link:../core @@ -57,15 +57,15 @@ importers: '@automerge/automerge-repo-network-websocket': specifier: ^2.5.5 version: 2.5.5 - '@lync/core': + isomorphic-ws: + specifier: ^5.0.0 + version: 5.0.0(ws@8.20.0) + lync-core: specifier: workspace:* version: link:../core - '@lync/index': + lync-index: specifier: workspace:* version: link:../index - isomorphic-ws: - specifier: ^5.0.0 - version: 5.0.0(ws@8.20.0) packages/core: dependencies: @@ -75,7 +75,7 @@ importers: packages/index: dependencies: - '@lync/core': + lync-core: specifier: workspace:* version: link:../core From f0317052e7b4d3cdc6cabdc97327f8275c2dccf6 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 17:16:41 -0700 Subject: [PATCH 07/33] README: state plainly which packages are published vs in-repo pre-release --- README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/README.md b/README.md index 1f1c32b..a8c0142 100644 --- a/README.md +++ b/README.md @@ -75,6 +75,10 @@ The short version: - `lync-sync-server`: the current Automerge WebSocket relay. - `lync-index`: legacy synced indexes of loom references. +Published on npm: `lync-core` and `lync-cli` (plus the `lync` command). +The other packages live in this repo and are pre-release — install them +from source if you want to experiment. + ## Format-Layer Imports The format-layer subpaths currently keep their internal path names for From 087d41a1b2cf5d824588ca24c22e9c4f6a3294dc Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 17:30:08 -0700 Subject: [PATCH 08/33] client: never hang silently on an incomplete peer handshake MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pre-handshake, re-send the idempotent join on a bounded interval; if the handshake still hasn't completed, fail loudly and close so the reconnect path takes over. Duplicate peer replies no longer re-announce the peer. Honest status: fixes a real silent-hang path (join or peer-reply lost on an OPEN socket left the adapter waiting forever), but does NOT cure the suite flake — 3/30 full-suite failures remain, all correlated with '[Lync] repo shutdown failed: DocHandle is not ready' bleeding from the Automerge repo teardown (sync-server/src/index.ts:198-204 swallows it). --- packages/client/src/sync.ts | 44 +++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/packages/client/src/sync.ts b/packages/client/src/sync.ts index 1c19e7d..0b45d8a 100644 --- a/packages/client/src/sync.ts +++ b/packages/client/src/sync.ts @@ -93,6 +93,13 @@ function shouldUseNativeAdapter(options: WebSocketSyncOptions) { ); } +// A join (or the server's peer reply) can be lost even on an OPEN socket. An +// open socket with no completed handshake must never hang silently: re-send +// the idempotent join a bounded number of times, then fail loudly and hand +// control to the reconnect path. +const JOIN_RETRY_MS = 400; +const MAX_JOIN_ATTEMPTS = 8; + class ResilientWebSocketClientAdapter extends NetworkAdapter { private socket?: WebSocket; private ready = false; @@ -102,6 +109,8 @@ class ResilientWebSocketClientAdapter extends NetworkAdapter { }); private retryIntervalId?: IntervalId; private retryTimeoutId?: TimeoutId; + private joinRetryId?: IntervalId; + private joinAttempts = 0; private readonly retryInterval: number; private readonly mode: SyncMode; private abandonedHandshakeRetryAt = 0; @@ -164,6 +173,7 @@ class ResilientWebSocketClientAdapter extends NetworkAdapter { if (this.retryTimeoutId) clearTimeout(this.retryTimeoutId); this.retryIntervalId = undefined; this.retryTimeoutId = undefined; + this.clearJoinRetry(); if (this.socket) { this.closeSocket(this.socket); @@ -197,10 +207,32 @@ class ResilientWebSocketClientAdapter extends NetworkAdapter { private onOpen = () => { this.clearRetryInterval(); this.abandonedHandshakeRetryAt = 0; + this.joinAttempts = 0; this.join(); + this.clearJoinRetry(); + this.joinRetryId = setInterval(() => { + if (this.remotePeerId) { + this.clearJoinRetry(); + return; + } + this.joinAttempts += 1; + if (this.joinAttempts >= MAX_JOIN_ATTEMPTS) { + this.clearJoinRetry(); + this.reportError( + new Error( + `Peer handshake did not complete after ${MAX_JOIN_ATTEMPTS} join attempts; closing socket to trigger reconnect`, + ), + true, + ); + if (this.socket) this.closeSocket(this.socket); + return; + } + this.join(); + }, JOIN_RETRY_MS); }; private onClose = () => { + this.clearJoinRetry(); if (this.remotePeerId) { this.emit("peer-disconnected", { peerId: this.remotePeerId }); this.remotePeerId = undefined; @@ -257,8 +289,15 @@ class ResilientWebSocketClientAdapter extends NetworkAdapter { if (isPeerMessage(message)) { this.forceReady(); + const isNewPeer = this.remotePeerId !== message.senderId; this.remotePeerId = message.senderId; this.clearRetryInterval(); + this.clearJoinRetry(); + if (!isNewPeer) { + // A re-sent join can earn a duplicate peer reply; the handshake is + // already complete, so don't re-announce the peer downstream. + return; + } this.options.onStatus?.({ state: "connected", url: this.options.url, @@ -287,6 +326,11 @@ class ResilientWebSocketClientAdapter extends NetworkAdapter { this.retryIntervalId = undefined; } + private clearJoinRetry() { + if (this.joinRetryId) clearInterval(this.joinRetryId); + this.joinRetryId = undefined; + } + private reportError(error: Error, recoverable: boolean) { this.options.onError?.(error); this.options.onStatus?.({ From d8d74d248467a13b41249db1018b0db2a46d68f6 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 17:47:16 -0700 Subject: [PATCH 09/33] cutover: one vocabulary, one complete product - front window is lync-core + lync-cli only; the Automerge transport (client ws-sync, sync-server, index), browser repo wiring, and the migration script move to branch attic/automerge-transport - full public-API rename: parseLyncFiles, LyncUnion, serializeLyncEvent, createLyncLooms, lync* views; core subpaths flatten (lync-core/events, /store, /views, /looms, /file-log, /idb-log, /memory-log) - wire cutover: kinds are lync/* (spec, vectors regenerated with fresh digests, tests); loom ids mint lync:; file store reads .lync only; no legacy .lore reading, no migration story - lync-core has zero runtime dependencies; root automerge deps removed - FORMAT.md drops the legacy-vocabulary caveat; README describes exactly what ships and nothing else --- FORMAT.md | 26 +- README.md | 117 ++--- package.json | 9 - packages/cli/src/index.ts | 38 +- packages/cli/test/cli.test.ts | 2 +- packages/client/package.json | 51 --- packages/client/src/browser.ts | 56 --- packages/client/src/create.ts | 94 ---- packages/client/src/index.ts | 2 - packages/client/src/node.ts | 178 -------- packages/client/src/sync.ts | 424 ------------------ packages/client/src/testing.ts | 36 -- packages/client/src/types.ts | 77 ---- packages/client/test/browser.test.ts | 84 ---- packages/client/test/node.test.ts | 240 ---------- packages/client/test/testing.test.ts | 78 ---- packages/client/tsconfig.json | 8 - packages/core/README.md | 6 +- packages/core/package.json | 69 ++- packages/core/src/automerge.ts | 330 -------------- packages/core/src/browser.ts | 105 ----- packages/core/src/{lore => }/events.ts | 146 +++--- packages/core/src/{lore => }/file-log.ts | 20 +- packages/core/src/{lore => }/idb-log.ts | 2 +- packages/core/src/index.ts | 12 +- packages/core/src/{lore => }/looms.ts | 54 +-- packages/core/src/{lore => }/memory-log.ts | 0 packages/core/src/{lore => }/sha256.ts | 2 +- packages/core/src/{lore => }/store.ts | 20 +- packages/core/src/{lore => }/views.ts | 86 ++-- packages/core/test/automerge-browser.test.ts | 70 --- packages/core/test/automerge.test.ts | 127 ------ .../{lore-events.test.ts => events.test.ts} | 126 +++--- packages/core/test/references.test.ts | 2 +- .../{lore-sha256.test.ts => sha256.test.ts} | 2 +- .../{lore-storage.test.ts => storage.test.ts} | 36 +- .../04-garbage-classes/input.lore | 7 - .../06-graph-obstacles/input.lore | 7 - .../08-spelling-vs-value/input.lore | 6 - .../09-marked-at-semantics/input.lore | 4 - .../lore-vectors-draft/10-merge-union/a.lore | 3 - .../lore-vectors-draft/10-merge-union/b.lore | 3 - .../11-nonconforming-carried/input.lore | 2 - .../12-invalid-sig-splice/input.lore | 1 - .../01-valid-events/expected.json | 10 +- .../01-valid-events/input.lync} | 8 +- .../02-splice-anchoring/expected.json | 10 +- .../02-splice-anchoring/input.lync} | 8 +- .../03-damaged-digest/expected.json | 6 +- .../03-damaged-digest/input.lync} | 4 +- .../04-garbage-classes/expected.json | 16 +- .../vectors/v0/04-garbage-classes/input.lync | 7 + .../05-conflicts-and-duplicates/expected.json | 16 +- .../05-conflicts-and-duplicates/input.lync} | 14 +- .../06-graph-obstacles/expected.json | 16 +- .../vectors/v0/06-graph-obstacles/input.lync | 7 + .../07-critical-suppression/expected.json | 20 +- .../07-critical-suppression/input.lync} | 10 +- .../08-spelling-vs-value/expected.json | 14 +- .../v0/08-spelling-vs-value/input.lync | 6 + .../09-marked-at-semantics/expected.json | 10 +- .../v0/09-marked-at-semantics/input.lync | 4 + .../test/vectors/v0/10-merge-union/a.lync | 3 + .../test/vectors/v0/10-merge-union/b.lync | 3 + .../10-merge-union/expected.json | 16 +- .../11-nonconforming-carried/expected.json | 6 +- .../v0/11-nonconforming-carried/input.lync | 2 + .../12-invalid-sig-splice/expected.json | 4 +- .../v0/12-invalid-sig-splice/input.lync | 1 + .../13-sig-without-digest/expected.json | 4 +- .../13-sig-without-digest/input.lync} | 2 +- .../OPEN-QUESTIONS.md | 0 .../{lore-vectors-draft => v0}/README.md | 6 +- .../{lore-vectors-draft => v0}/generate.py | 198 ++++---- .../{lore-views.test.ts => views.test.ts} | 40 +- packages/index/package.json | 47 -- packages/index/src/automerge.ts | 280 ------------ packages/index/src/entries.ts | 17 - packages/index/src/index.ts | 2 - packages/index/src/memory.ts | 239 ---------- packages/index/src/types.ts | 78 ---- packages/index/test/automerge.test.ts | 64 --- packages/index/test/memory.test.ts | 97 ---- packages/index/tsconfig.json | 8 - packages/sync-server/package.json | 29 -- packages/sync-server/src/index.ts | 314 ------------- packages/sync-server/test/sync-server.test.ts | 166 ------- packages/sync-server/tsconfig.json | 8 - pnpm-lock.yaml | 290 +----------- scripts/migrate-automerge-to-lync.ts | 366 --------------- vitest.config.ts | 82 +--- 91 files changed, 613 insertions(+), 4713 deletions(-) delete mode 100644 packages/client/package.json delete mode 100644 packages/client/src/browser.ts delete mode 100644 packages/client/src/create.ts delete mode 100644 packages/client/src/index.ts delete mode 100644 packages/client/src/node.ts delete mode 100644 packages/client/src/sync.ts delete mode 100644 packages/client/src/testing.ts delete mode 100644 packages/client/src/types.ts delete mode 100644 packages/client/test/browser.test.ts delete mode 100644 packages/client/test/node.test.ts delete mode 100644 packages/client/test/testing.test.ts delete mode 100644 packages/client/tsconfig.json delete mode 100644 packages/core/src/automerge.ts delete mode 100644 packages/core/src/browser.ts rename packages/core/src/{lore => }/events.ts (86%) rename packages/core/src/{lore => }/file-log.ts (87%) rename packages/core/src/{lore => }/idb-log.ts (98%) rename packages/core/src/{lore => }/looms.ts (91%) rename packages/core/src/{lore => }/memory-log.ts (100%) rename packages/core/src/{lore => }/sha256.ts (98%) rename packages/core/src/{lore => }/store.ts (95%) rename packages/core/src/{lore => }/views.ts (78%) delete mode 100644 packages/core/test/automerge-browser.test.ts delete mode 100644 packages/core/test/automerge.test.ts rename packages/core/test/{lore-events.test.ts => events.test.ts} (85%) rename packages/core/test/{lore-sha256.test.ts => sha256.test.ts} (96%) rename packages/core/test/{lore-storage.test.ts => storage.test.ts} (81%) delete mode 100644 packages/core/test/vectors/lore-vectors-draft/04-garbage-classes/input.lore delete mode 100644 packages/core/test/vectors/lore-vectors-draft/06-graph-obstacles/input.lore delete mode 100644 packages/core/test/vectors/lore-vectors-draft/08-spelling-vs-value/input.lore delete mode 100644 packages/core/test/vectors/lore-vectors-draft/09-marked-at-semantics/input.lore delete mode 100644 packages/core/test/vectors/lore-vectors-draft/10-merge-union/a.lore delete mode 100644 packages/core/test/vectors/lore-vectors-draft/10-merge-union/b.lore delete mode 100644 packages/core/test/vectors/lore-vectors-draft/11-nonconforming-carried/input.lore delete mode 100644 packages/core/test/vectors/lore-vectors-draft/12-invalid-sig-splice/input.lore rename packages/core/test/vectors/{lore-vectors-draft => v0}/01-valid-events/expected.json (91%) rename packages/core/test/vectors/{lore-vectors-draft/01-valid-events/input.lore => v0/01-valid-events/input.lync} (58%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/02-splice-anchoring/expected.json (87%) rename packages/core/test/vectors/{lore-vectors-draft/02-splice-anchoring/input.lore => v0/02-splice-anchoring/input.lync} (61%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/03-damaged-digest/expected.json (83%) rename packages/core/test/vectors/{lore-vectors-draft/03-damaged-digest/input.lore => v0/03-damaged-digest/input.lync} (75%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/04-garbage-classes/expected.json (81%) create mode 100644 packages/core/test/vectors/v0/04-garbage-classes/input.lync rename packages/core/test/vectors/{lore-vectors-draft => v0}/05-conflicts-and-duplicates/expected.json (86%) rename packages/core/test/vectors/{lore-vectors-draft/05-conflicts-and-duplicates/input.lore => v0/05-conflicts-and-duplicates/input.lync} (50%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/06-graph-obstacles/expected.json (92%) create mode 100644 packages/core/test/vectors/v0/06-graph-obstacles/input.lync rename packages/core/test/vectors/{lore-vectors-draft => v0}/07-critical-suppression/expected.json (88%) rename packages/core/test/vectors/{lore-vectors-draft/07-critical-suppression/input.lore => v0/07-critical-suppression/input.lync} (71%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/08-spelling-vs-value/expected.json (89%) create mode 100644 packages/core/test/vectors/v0/08-spelling-vs-value/input.lync rename packages/core/test/vectors/{lore-vectors-draft => v0}/09-marked-at-semantics/expected.json (86%) create mode 100644 packages/core/test/vectors/v0/09-marked-at-semantics/input.lync create mode 100644 packages/core/test/vectors/v0/10-merge-union/a.lync create mode 100644 packages/core/test/vectors/v0/10-merge-union/b.lync rename packages/core/test/vectors/{lore-vectors-draft => v0}/10-merge-union/expected.json (88%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/11-nonconforming-carried/expected.json (90%) create mode 100644 packages/core/test/vectors/v0/11-nonconforming-carried/input.lync rename packages/core/test/vectors/{lore-vectors-draft => v0}/12-invalid-sig-splice/expected.json (86%) create mode 100644 packages/core/test/vectors/v0/12-invalid-sig-splice/input.lync rename packages/core/test/vectors/{lore-vectors-draft => v0}/13-sig-without-digest/expected.json (86%) rename packages/core/test/vectors/{lore-vectors-draft/13-sig-without-digest/input.lore => v0/13-sig-without-digest/input.lync} (68%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/OPEN-QUESTIONS.md (100%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/README.md (94%) rename packages/core/test/vectors/{lore-vectors-draft => v0}/generate.py (79%) rename packages/core/test/{lore-views.test.ts => views.test.ts} (86%) delete mode 100644 packages/index/package.json delete mode 100644 packages/index/src/automerge.ts delete mode 100644 packages/index/src/entries.ts delete mode 100644 packages/index/src/index.ts delete mode 100644 packages/index/src/memory.ts delete mode 100644 packages/index/src/types.ts delete mode 100644 packages/index/test/automerge.test.ts delete mode 100644 packages/index/test/memory.test.ts delete mode 100644 packages/index/tsconfig.json delete mode 100644 packages/sync-server/package.json delete mode 100644 packages/sync-server/src/index.ts delete mode 100644 packages/sync-server/test/sync-server.test.ts delete mode 100644 packages/sync-server/tsconfig.json delete mode 100644 scripts/migrate-automerge-to-lync.ts diff --git a/FORMAT.md b/FORMAT.md index 7c7eed3..715f7e5 100644 --- a/FORMAT.md +++ b/FORMAT.md @@ -1,4 +1,4 @@ -# The lync format — files of lore +# The lync format Status: v0 draft. Conventional extension: `.lync`. @@ -283,20 +283,20 @@ agree on. A pact binds its signatories; the protocol binds everyone. Where a community has already converged, borrow; mint only where you are genuinely first. Current recommendations: -- `lore/artifact`: a thing someone produced from prior things, such as prose, +- `lync/artifact`: a thing someone produced from prior things, such as prose, code, a prediction, a tool result, an imported message, or a computed excerpt. `parents` are what it was made from, in order. -- `lore/annotation`: an authored claim about one or more events. `parents` are +- `lync/annotation`: an authored claim about one or more events. `parents` are the targets. Scores, critiques, rewards, receipts, labels, and selections are relations-as-events. -- `lore/pointer`: a named reference that moves without mutating. Payload +- `lync/pointer`: a named reference that moves without mutating. Payload `{"name": "...", "target": ""}`; live value is newest per actor and name. Older pointers are history. -- `lore/tombstone`: retraction, written `critical: true` so rule 3 binds even +- `lync/tombstone`: retraction, written `critical: true` so rule 3 binds even readers that have never heard of tombstones. `parents[0]` is the target. Historical namespace spellings are frozen wire vocabulary. Shipped kind strings -such as `lore/annotation`, `lore/artifact`, `lync/turn`, and `lync/loom` are +such as `lync/annotation`, `lync/artifact`, `lync/turn`, and `lync/loom` are exact-match data in stored files; the lync brand does not rename shipped kinds. Annotation labels may include `selection`, `score`, `decision`, `no-train`, @@ -378,11 +378,11 @@ elided for readability only. Conforming writers mint UUIDv7 ids and should splice digests per "Bytes Are Canonical." ```jsonl -{"v":1,"id":"A","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"The bear stood at the lip of the falls."}} -{"v":1,"id":"B","kind":"lore/artifact","at":"2026-07-06T04:10:09Z","author":{"actor":"claude-haiku-4-5","operator":"deepfates","via":"textile@0.9"},"parents":["A"],"payload":{"text":"It did not move for an hour, and the river brought it everything.","ordinal":0}} -{"v":1,"id":"C","kind":"lore/artifact","at":"2026-07-06T04:10:09Z","author":{"actor":"claude-haiku-4-5","operator":"deepfates","via":"textile@0.9"},"parents":["A"],"payload":{"text":"Downstream, the younger bears fought over shallows.","ordinal":1}} -{"v":1,"id":"D","kind":"lore/annotation","at":"2026-07-06T04:10:11Z","author":{"actor":"witness-panel-v3"},"parents":["B"],"payload":{"label":"score","value":0.91}} -{"v":1,"id":"E","kind":"lore/annotation","at":"2026-07-06T04:10:15Z","author":{"actor":"deepfates"},"parents":["B","C"],"payload":{"label":"selection","chosen":["B"],"shown":["B","C"],"basis":"human pick"}} +{"v":1,"id":"A","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"The bear stood at the lip of the falls."}} +{"v":1,"id":"B","kind":"lync/artifact","at":"2026-07-06T04:10:09Z","author":{"actor":"claude-haiku-4-5","operator":"deepfates","via":"textile@0.9"},"parents":["A"],"payload":{"text":"It did not move for an hour, and the river brought it everything.","ordinal":0}} +{"v":1,"id":"C","kind":"lync/artifact","at":"2026-07-06T04:10:09Z","author":{"actor":"claude-haiku-4-5","operator":"deepfates","via":"textile@0.9"},"parents":["A"],"payload":{"text":"Downstream, the younger bears fought over shallows.","ordinal":1}} +{"v":1,"id":"D","kind":"lync/annotation","at":"2026-07-06T04:10:11Z","author":{"actor":"witness-panel-v3"},"parents":["B"],"payload":{"label":"score","value":0.91}} +{"v":1,"id":"E","kind":"lync/annotation","at":"2026-07-06T04:10:15Z","author":{"actor":"deepfates"},"parents":["B","C"],"payload":{"label":"selection","chosen":["B"],"shown":["B","C"],"basis":"human pick"}} ``` Event `E` exists because nothing was extended yet, so the graph alone cannot @@ -397,9 +397,5 @@ walks every parent, and lies about nothing. ## Ambiguity Notes -The public vocabulary has moved to lync, but the current TypeScript package -still exposes some legacy import paths and identifier names. Those names are -not wire-format semantics. - Reserved top-level field names such as `digest` and `sig` are wire-format semantics and are not renamed. diff --git a/README.md b/README.md index a8c0142..e14d451 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,15 @@ # lync -lync is the TypeScript reference implementation of the lync format: files of -lore stored as `.lync` append-only JSONL interaction histories. Each line is +lync is the TypeScript reference implementation of the lync format: `.lync` +append-only JSONL files of interaction history. Each line is one immutable event with an envelope, parent links, provenance, and a payload owned by the event kind. Merge is set union by event id. Branch trees, transcripts, memory views, and leaderboards are computed views over the same event set. -The current library keeps the old loom/turn API while the format layer becomes -the durable center. Automerge is the current sync transport, scheduled for -replacement in `dee-9l2l`; it is not the data model. +The format layer is the durable center. A loom/turn API ships on top of the +same event stores for programs that want turns and threads instead of raw +events. ## Ninety-Second Story @@ -69,79 +69,68 @@ The short version: ## Packages - `lync-core`: format parsing, event stores, computed views, references, and - the compatibility loom API. -- `lync-client`: browser, Node, and test runtime clients for the compatibility - API. -- `lync-sync-server`: the current Automerge WebSocket relay. -- `lync-index`: legacy synced indexes of loom references. - -Published on npm: `lync-core` and `lync-cli` (plus the `lync` command). -The other packages live in this repo and are pre-release — install them -from source if you want to experiment. + the loom API. No runtime dependencies. +- `lync-cli`: the `lync` command — `init`, `append`, `verify`, `merge`, `view`. ## Format-Layer Imports -The format-layer subpaths currently keep their internal path names for -compatibility. Import them by path; do not treat those path segments as public -vocabulary. - ```ts -import { LoreUnion, exportCarriedLoreBytes, parseLoreFiles } from "lync-core/lore/events"; -import { createFileEventStore, createFileLoreLooms } from "lync-core/lore/file-log"; -import { createIndexedDbEventStore } from "lync-core/lore/idb-log"; -import { createLoreLooms, createBrowserLoreLooms } from "lync-core/lore/looms"; -import { createMemoryEventStore } from "lync-core/lore/memory-log"; -import { BaseEventStore, serializeLoreEvent } from "lync-core/lore/store"; +import { LyncUnion, exportCarriedLyncBytes, parseLyncFiles } from "lync-core/events"; +import { createFileEventStore, createFileLyncLooms } from "lync-core/file-log"; +import { createIndexedDbEventStore } from "lync-core/idb-log"; +import { createLyncLooms, createBrowserLyncLooms } from "lync-core/looms"; +import { createMemoryEventStore } from "lync-core/memory-log"; +import { BaseEventStore, serializeLyncEvent } from "lync-core/store"; import { - loreBranchTreeView, - loreLeaderboardView, - loreMemoryView, - loreTranscriptView, -} from "lync-core/lore/views"; + lyncBranchTreeView, + lyncLeaderboardView, + lyncMemoryView, + lyncTranscriptView, +} from "lync-core/views"; ``` The seven format-layer package exports are: -- `lync-core/lore/events`: line parsing, carried-byte export, downsets, and +- `lync-core/events`: line parsing, carried-byte export, downsets, and incremental union. -- `lync-core/lore/memory-log`: in-memory event store for tests and embedded +- `lync-core/memory-log`: in-memory event store for tests and embedded runtimes. -- `lync-core/lore/file-log`: file-backed event store and `createFileLoreLooms` +- `lync-core/file-log`: file-backed event store and `createFileLyncLooms` (node-only; keeps `node:fs`/`node:path` off the browser path). -- `lync-core/lore/idb-log`: IndexedDB-backed event store. -- `lync-core/lore/store`: base event-store contract and serialization helpers. -- `lync-core/lore/views`: branch tree, transcript, memory, and leaderboard +- `lync-core/idb-log`: IndexedDB-backed event store. +- `lync-core/store`: base event-store contract and serialization helpers. +- `lync-core/views`: branch tree, transcript, memory, and leaderboard view helpers. -- `lync-core/lore/looms`: compatibility loom API backed by event stores. +- `lync-core/looms`: compatibility loom API backed by event stores. ## Parse, Union, View ```ts -import { parseLoreFiles } from "lync-core/lore/events"; -import { loreBranchTreeView, loreMemoryView } from "lync-core/lore/views"; +import { parseLyncFiles } from "lync-core/events"; +import { lyncBranchTreeView, lyncMemoryView } from "lync-core/views"; const bytes = new TextEncoder().encode( '{"v":1,"id":"a","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"Once..."}}\n', ); -const parsed = parseLoreFiles([{ file: "story.lync", bytes }]); -const tree = loreBranchTreeView(parsed); -const memory = loreMemoryView(parsed); +const parsed = parseLyncFiles([{ file: "story.lync", bytes }]); +const tree = lyncBranchTreeView(parsed); +const memory = lyncMemoryView(parsed); console.log(parsed.lines[0].class, tree.roots, memory.frontierIds); ``` -`parseLoreFiles` classifies every physical line and keeps the original bytes, +`parseLyncFiles` classifies every physical line and keeps the original bytes, including garbage, damaged lines, nonconforming-but-carried lines, and conflict -variants. `exportCarriedLoreBytes(parsed)` re-emits the carried bytes. +variants. `exportCarriedLyncBytes(parsed)` re-emits the carried bytes. -`LoreUnion` performs the same union incrementally and can buffer children until +`LyncUnion` performs the same union incrementally and can buffer children until their first missing parent arrives. ## Storage ```ts -import { createMemoryEventStore } from "lync-core/lore/memory-log"; +import { createMemoryEventStore } from "lync-core/memory-log"; const store = createMemoryEventStore(); await store.append({ @@ -171,16 +160,16 @@ The store API accepts raw lines through `union(line)` and structured event bodies through `append(event)`. It reports conflicts, pending parents, garbage, and accepted events without making file order meaningful. -## Compatibility Looms +## Looms -The loom API remains for existing users and for the current Automerge-backed -clients. It now has an event-store implementation: +The loom API gives programs turns and threads instead of raw events, backed by +any event store: ```ts -import { createLoreLooms } from "lync-core/lore/looms"; -import { createMemoryEventStore } from "lync-core/lore/memory-log"; +import { createLyncLooms } from "lync-core/looms"; +import { createMemoryEventStore } from "lync-core/memory-log"; -const looms = createLoreLooms<{ text: string }, { title: string }>({ +const looms = createLyncLooms<{ text: string }, { title: string }>({ store: createMemoryEventStore(), author: { actor: "deepfates", via: "example@0.1" }, createId: (() => { @@ -197,32 +186,6 @@ const next = await loom.appendTurn(first.id, { text: "Then..." }); console.log((await loom.threadTo(next.id)).map((turn) => turn.payload.text)); ``` -## Migration - -`scripts/migrate-automerge-to-lync.ts` migrates old Automerge loom storage into -the event-store implementation. Build first, then run the script against an -Automerge storage directory and an output directory: - -```bash -pnpm build -node scripts/migrate-automerge-to-lync.ts -``` - -The script writes a migration report as it goes, verifies migrated snapshots are -isomorphic to the source loom shape, and records per-document failures instead -of aborting the whole migration. Migrated roots are written as `.lync` files. -The file event store reads both `.lync` and legacy `.lore` files so old -exports can be mixed with newly migrated roots during a transition. - -## Sync Server - -`lync-sync-server` provides the current Automerge WebSocket relay. Its default -WebSocket path is `/lync`. The exported factory is `createLyncServer`. - -`authenticate` is synchronous by design in the server API. Return `false` to -reject an upgrade; if the predicate throws, lync rejects the upgrade instead of -accepting it. - ## Development ```bash diff --git a/package.json b/package.json index cbe4de7..2f911c0 100644 --- a/package.json +++ b/package.json @@ -19,14 +19,5 @@ "typescript": "^5.8.3", "vitest": "^3.1.1", "lync-cli": "workspace:*" - }, - "dependencies": { - "@automerge/automerge-repo": "^2.5.5", - "@automerge/automerge-repo-network-broadcastchannel": "^2.5.5", - "@automerge/automerge-repo-network-websocket": "^2.5.5", - "@automerge/automerge-repo-storage-indexeddb": "^2.5.5", - "@types/ws": "^8.18.1", - "isomorphic-ws": "^5.0.0", - "uuid": "^14.0.0" } } diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index a603f68..d63842d 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -1,11 +1,11 @@ import { randomUUID } from "node:crypto"; import { appendFile, readFile, stat, writeFile } from "node:fs/promises"; import { - parseLoreFiles, - type LoreLineClass, - type LoreLineDiagnostic, -} from "lync-core/lore/events"; -import { loreBranchTreeView as coreTreeView, loreTranscriptView as coreTranscriptView } from "lync-core/lore/views"; + parseLyncFiles, + type LyncLineClass, + type LyncLineDiagnostic, +} from "lync-core/events"; +import { lyncBranchTreeView as coreTreeView, lyncTranscriptView as coreTranscriptView } from "lync-core/views"; export interface LyncCliIO { stdout?: Pick; @@ -17,7 +17,7 @@ export interface LyncCliIO { type ExitCode = 0 | 1 | 2; -const classes: LoreLineClass[] = ["accepted", "nonconforming", "garbage", "damaged", "conflict-variant"]; +const classes: LyncLineClass[] = ["accepted", "nonconforming", "garbage", "damaged", "conflict-variant"]; const textEncoder = new TextEncoder(); export async function runLyncCli(argv: string[], io: LyncCliIO = {}): Promise { @@ -78,9 +78,9 @@ async function verify( return 2; } - const result = parseLoreFiles(await readInputs(args)); + const result = parseLyncFiles(await readInputs(args)); const totals = zeroCounts(); - const byFile = new Map>(); + const byFile = new Map>(); for (const line of result.lines) { const counts = byFile.get(line.file) ?? zeroCounts(); counts[line.class]++; @@ -122,7 +122,7 @@ async function merge( return 2; } - const result = parseLoreFiles(await readInputs(files)); + const result = parseLyncFiles(await readInputs(files)); await writeFile(output, mergeBytes(result.lines, new Set(result.unionEventIds))); return 0; } @@ -142,7 +142,7 @@ async function view( return 2; } - const result = parseLoreFiles(await readInputs(files)); + const result = parseLyncFiles(await readInputs(files)); if (as === "tree") { out.write(`${JSON.stringify(printableTreeView(coreTreeView(result)), null, 2)}\n`); return result.graphDiagnostics.length || result.conflictIds.length ? 1 : 0; @@ -192,7 +192,7 @@ async function append( } const line = `${JSON.stringify(built.event)}\n`; - const parsed = parseLoreFiles([{ file: args[0], bytes: line }]).lines[0]; + const parsed = parseLyncFiles([{ file: args[0], bytes: line }]).lines[0]; if (parsed?.class !== "accepted") { err.write(`That JSON is not a valid event: ${parsed?.reason ?? "unknown validation failure"}.\n`); return 1; @@ -238,9 +238,9 @@ function buildAppendEvent(value: unknown, io: LyncCliIO): return { ok: true, event }; } -function mergeBytes(lines: LoreLineDiagnostic[], unionIds: Set): Uint8Array { - const eventChoices = new Map(); - const carried: LoreLineDiagnostic[] = []; +function mergeBytes(lines: LyncLineDiagnostic[], unionIds: Set): Uint8Array { + const eventChoices = new Map(); + const carried: LyncLineDiagnostic[] = []; for (const line of lines) { if (line.id && unionIds.has(line.id) && (line.class === "accepted" || line.class === "nonconforming")) { const existing = eventChoices.get(line.id); @@ -257,7 +257,7 @@ function mergeBytes(lines: LoreLineDiagnostic[], unionIds: Set): Uint8Ar return joinLines(ordered); } -function joinLines(lines: LoreLineDiagnostic[]): Uint8Array { +function joinLines(lines: LyncLineDiagnostic[]): Uint8Array { const chunks = lines.flatMap((line) => line.terminator ? [line.bytes, textEncoder.encode(line.terminator)] : [line.bytes]); const total = chunks.reduce((sum, chunk) => sum + chunk.byteLength, 0); const bytes = new Uint8Array(total); @@ -273,19 +273,19 @@ async function readInputs(files: string[]) { return Promise.all(files.map(async (file) => ({ file, bytes: await readFile(file) }))); } -function zeroCounts(): Record { +function zeroCounts(): Record { return { accepted: 0, nonconforming: 0, garbage: 0, damaged: 0, "conflict-variant": 0 }; } -function formatCounts(counts: Record): string { +function formatCounts(counts: Record): string { return classes.map((kind) => `${kind}=${counts[kind]}`).join(" "); } -function hasVerifyIssues(lines: LoreLineDiagnostic[], pending: number, obstacles: number): boolean { +function hasVerifyIssues(lines: LyncLineDiagnostic[], pending: number, obstacles: number): boolean { return pending > 0 || obstacles > 0 || lines.some((line) => line.class !== "accepted"); } -function richness(line: LoreLineDiagnostic): number { +function richness(line: LyncLineDiagnostic): number { return (line.sig ? 2 : 0) + (line.digest ? 1 : 0); } diff --git a/packages/cli/test/cli.test.ts b/packages/cli/test/cli.test.ts index f26abd8..36b4b66 100644 --- a/packages/cli/test/cli.test.ts +++ b/packages/cli/test/cli.test.ts @@ -14,7 +14,7 @@ const vectorsRoot = join( "core", "test", "vectors", - "lore-vectors-draft", + "v0", ); function event(fields: { diff --git a/packages/client/package.json b/packages/client/package.json deleted file mode 100644 index 5e4e492..0000000 --- a/packages/client/package.json +++ /dev/null @@ -1,51 +0,0 @@ -{ - "name": "lync-client", - "version": "0.2.0", - "description": "Runtime clients for Lync browser, Node, and tests.", - "type": "module", - "license": "MIT", - "sideEffects": false, - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - }, - "./browser": { - "types": "./dist/browser.d.ts", - "import": "./dist/browser.js", - "default": "./dist/browser.js" - }, - "./node": { - "types": "./dist/node.d.ts", - "import": "./dist/node.js", - "default": "./dist/node.js" - }, - "./testing": { - "types": "./dist/testing.d.ts", - "import": "./dist/testing.js", - "default": "./dist/testing.js" - }, - "./types": { - "types": "./dist/types.d.ts", - "import": "./dist/types.js", - "default": "./dist/types.js" - } - }, - "files": [ - "dist" - ], - "scripts": { - "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit" - }, - "dependencies": { - "@automerge/automerge-repo": "^2.5.5", - "@automerge/automerge-repo-network-websocket": "^2.5.5", - "lync-core": "workspace:*", - "lync-index": "workspace:*", - "isomorphic-ws": "^5.0.0" - } -} diff --git a/packages/client/src/browser.ts b/packages/client/src/browser.ts deleted file mode 100644 index a26ef28..0000000 --- a/packages/client/src/browser.ts +++ /dev/null @@ -1,56 +0,0 @@ -import type { Repo } from "@automerge/automerge-repo"; -import { - createAutomergeLooms, - type AutomergeLoomsOptions, -} from "lync-core/automerge"; -import { - createBrowserAutomergeRepo, - type BrowserAutomergeRepoOptions, -} from "lync-core/browser"; -import { - createAutomergeLoomIndexes, - type AutomergeLoomIndexesOptions, -} from "lync-index/automerge"; -import { createLoomClient } from "./create.js"; -import type { LoomClient } from "./types.js"; - -export interface BrowserLoomClientOptions< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, -> { - repo?: Repo; - browser?: BrowserAutomergeRepoOptions; - looms?: Omit; - indexes?: Omit; -} - -export function createBrowserLoomClient< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, ->( - options: BrowserLoomClientOptions< - TPayload, - TLoomMeta, - TTurnMeta, - TEntryMeta, - TIndexMeta - > = {}, -): LoomClient { - const repo = options.repo ?? createBrowserAutomergeRepo(options.browser); - const looms = createAutomergeLooms({ - ...options.looms, - repo, - }); - const indexes = createAutomergeLoomIndexes({ - ...options.indexes, - repo, - }); - - return createLoomClient({ repo, looms, indexes }); -} diff --git a/packages/client/src/create.ts b/packages/client/src/create.ts deleted file mode 100644 index 707a535..0000000 --- a/packages/client/src/create.ts +++ /dev/null @@ -1,94 +0,0 @@ -import type { Repo } from "@automerge/automerge-repo"; -import { - brokenTopology, - decodeReference, - encodeReference, - indexRef, - loomRef, - parseReference, - referenceFromUrl, - referenceToUrl, - threadRef, - turnRef, - type LoomReference, - type Looms, -} from "lync-core"; -import type { LoomIndexes } from "lync-index"; -import type { LoomClient } from "./types.js"; - -export interface CreateLoomClientOptions< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, -> { - repo?: Repo; - looms: Looms; - indexes: LoomIndexes; - close?: () => Promise | void; -} - -export function createLoomClient< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, ->( - options: CreateLoomClientOptions< - TPayload, - TLoomMeta, - TTurnMeta, - TEntryMeta, - TIndexMeta - >, -): LoomClient { - const { repo, looms, indexes } = options; - - return { - ...(repo ? { repo } : {}), - looms, - indexes, - references: { - loom: loomRef, - turn: turnRef, - thread: threadRef, - index: indexRef, - encode: encodeReference, - decode: decodeReference, - parse: parseReference, - toUrl: referenceToUrl, - fromUrl: referenceFromUrl, - }, - async openReference(ref: LoomReference) { - switch (ref.kind) { - case "loom": { - const loom = await looms.open(ref.loomId); - return { kind: "loom", ref, loom }; - } - case "turn": { - const loom = await looms.open(ref.loomId); - const turn = await loom.getTurn(ref.turnId); - if (!turn) throw brokenTopology(`Reference target turn not found: ${ref.turnId}`); - return { kind: "turn", ref, loom, turn }; - } - case "thread": { - const loom = await looms.open(ref.loomId); - const thread = await loom.threadTo(ref.turnId); - const target = thread.at(-1); - if (!target) throw brokenTopology(`Reference target thread is empty: ${ref.turnId}`); - return { kind: "thread", ref, loom, thread, target }; - } - case "index": { - const index = await indexes.open(ref.indexId); - return { kind: "index", ref, index }; - } - } - }, - async close() { - await options.close?.(); - await repo?.shutdown(); - }, - }; -} diff --git a/packages/client/src/index.ts b/packages/client/src/index.ts deleted file mode 100644 index 7a12a9d..0000000 --- a/packages/client/src/index.ts +++ /dev/null @@ -1,2 +0,0 @@ -export * from "./create.js"; -export * from "./types.js"; diff --git a/packages/client/src/node.ts b/packages/client/src/node.ts deleted file mode 100644 index db53ad2..0000000 --- a/packages/client/src/node.ts +++ /dev/null @@ -1,178 +0,0 @@ -import fs from "node:fs/promises"; -import path from "node:path"; -import { - Repo, - type Chunk, - type RepoConfig, - type StorageAdapterInterface, - type StorageKey, -} from "@automerge/automerge-repo"; -import { - createAutomergeLooms, - type AutomergeLoomsOptions, -} from "lync-core/automerge"; -import { - createAutomergeLoomIndexes, - type AutomergeLoomIndexesOptions, -} from "lync-index/automerge"; -import { createLoomClient } from "./create.js"; -import { - createWebSocketSyncAdapter, - type WebSocketSyncOptions, -} from "./sync.js"; -import type { LoomClient } from "./types.js"; - -export type { - SyncAuth, - SyncMode, - SyncStatus, - WebSocketSyncOptions, -} from "./sync.js"; - -export interface NodeLoomClientOptions< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, -> { - repo?: Repo; - storageDir?: string | false; - syncUrl?: string | false; - sync?: false | WebSocketSyncOptions; - websocket?: false | WebSocketSyncOptions; - repoConfig?: Omit; - looms?: Omit; - indexes?: Omit; -} - -export function createNodeLoomClient< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, ->( - options: NodeLoomClientOptions< - TPayload, - TLoomMeta, - TTurnMeta, - TEntryMeta, - TIndexMeta - > = {}, -): LoomClient { - const repo = options.repo ?? createNodeRepo(options); - const looms = createAutomergeLooms({ - ...options.looms, - repo, - }); - const indexes = createAutomergeLoomIndexes({ - ...options.indexes, - repo, - }); - - return createLoomClient({ repo, looms, indexes }); -} - -function createNodeRepo(options: NodeLoomClientOptions): Repo { - const websocket = resolveWebSocketOptions(options); - - return new Repo({ - ...options.repoConfig, - storage: - options.storageDir === false - ? undefined - : new FileStorageAdapter(options.storageDir ?? ".lync"), - network: - websocket === false - ? [] - : [createWebSocketSyncAdapter(websocket)], - }); -} - -function resolveWebSocketOptions( - options: NodeLoomClientOptions, -): false | WebSocketSyncOptions { - return ( - options.sync ?? - options.websocket ?? - (options.syncUrl === undefined - ? false - : options.syncUrl === false - ? false - : { url: options.syncUrl }) - ); -} - -export class FileStorageAdapter implements StorageAdapterInterface { - constructor(private readonly dir: string) {} - - async load(key: StorageKey): Promise { - try { - return toUint8Array(await fs.readFile(this.filePath(key))); - } catch (error) { - if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined; - throw error; - } - } - - async save(key: StorageKey, data: Uint8Array): Promise { - await fs.mkdir(this.dir, { recursive: true }); - await fs.writeFile(this.filePath(key), data); - } - - async remove(key: StorageKey): Promise { - try { - await fs.unlink(this.filePath(key)); - } catch (error) { - if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; - } - } - - async loadRange(keyPrefix: StorageKey): Promise { - await fs.mkdir(this.dir, { recursive: true }); - const prefix = this.keyToFilename(keyPrefix); - const files = await fs.readdir(this.dir); - return Promise.all( - files - .filter((file) => this.matchesPrefix(file, prefix)) - .map(async (file) => ({ - key: this.filenameToKey(file), - data: toUint8Array(await fs.readFile(path.join(this.dir, file))), - })), - ); - } - - async removeRange(keyPrefix: StorageKey): Promise { - await fs.mkdir(this.dir, { recursive: true }); - const prefix = this.keyToFilename(keyPrefix); - const files = await fs.readdir(this.dir); - await Promise.all( - files - .filter((file) => this.matchesPrefix(file, prefix)) - .map((file) => fs.unlink(path.join(this.dir, file))), - ); - } - - private filePath(key: StorageKey) { - return path.join(this.dir, this.keyToFilename(key)); - } - - private keyToFilename(key: StorageKey) { - return key.map((part) => encodeURIComponent(part)).join("."); - } - - private filenameToKey(filename: string): StorageKey { - return filename.split(".").map((part) => decodeURIComponent(part)); - } - - private matchesPrefix(filename: string, prefix: string) { - return !prefix || filename === prefix || filename.startsWith(`${prefix}.`); - } -} - -function toUint8Array(data: Uint8Array) { - return new Uint8Array( - data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength), - ); -} diff --git a/packages/client/src/sync.ts b/packages/client/src/sync.ts deleted file mode 100644 index 0b45d8a..0000000 --- a/packages/client/src/sync.ts +++ /dev/null @@ -1,424 +0,0 @@ -import { - NetworkAdapter, - cbor, - type Message, - type PeerId, - type PeerMetadata, -} from "@automerge/automerge-repo/slim"; -import { WebSocketClientAdapter } from "@automerge/automerge-repo-network-websocket"; -import WebSocket from "isomorphic-ws"; - -const PROTOCOL_V1 = "1"; -const swallowSocketAbortError = () => {}; - -type TimeoutId = ReturnType; -type IntervalId = ReturnType; -type DestroyableSocket = { destroy: () => void; destroyed?: boolean }; -type WebSocketInternals = { - _req?: { abort?: () => void; socket?: DestroyableSocket }; - _socket?: DestroyableSocket; -}; - -export type SyncMode = "best-effort" | "required"; - -export type SyncStatus = - | { state: "connecting"; url: string } - | { state: "connected"; url: string; peerId: PeerId } - | { state: "disconnected"; url: string; retryInMs?: number } - | { state: "failed"; url: string; error: Error; recoverable: boolean }; - -export type SyncAuth = - | { type: "bearer"; token: string } - | { type: "api-key"; token: string; header?: string }; - -export interface WebSocketSyncOptions { - kind?: "websocket"; - url: string; - retryInterval?: number; - adapter?: "auto" | "native" | "resilient"; - mode?: SyncMode; - headers?: Record; - auth?: SyncAuth; - onStatus?: (status: SyncStatus) => void; - onError?: (error: Error) => void; -} - -type JoinMessage = { - type: "join"; - senderId: PeerId; - peerMetadata: PeerMetadata; - supportedProtocolVersions: string[]; -}; - -type PeerMessage = { - type: "peer"; - senderId: PeerId; - peerMetadata: PeerMetadata; - selectedProtocolVersion: string; - targetId: PeerId; -}; - -type ErrorMessage = { - type: "error"; - senderId: PeerId; - message: string; - targetId: PeerId; -}; - -type FromClientMessage = JoinMessage | Message; -type FromServerMessage = PeerMessage | ErrorMessage | Message; - -export function createWebSocketSyncAdapter(options: WebSocketSyncOptions) { - if (shouldUseNativeAdapter(options)) { - return new WebSocketClientAdapter(options.url, options.retryInterval); - } - return new ResilientWebSocketClientAdapter(options); -} - -function shouldUseNativeAdapter(options: WebSocketSyncOptions) { - if (options.adapter === "resilient") return false; - if (options.adapter === "native") { - if (options.auth || hasHeaders(options.headers)) { - throw new Error("The native Automerge websocket adapter does not support auth headers"); - } - return true; - } - - return ( - !options.auth && - !hasHeaders(options.headers) && - !options.onStatus && - !options.onError && - options.mode !== "required" - ); -} - -// A join (or the server's peer reply) can be lost even on an OPEN socket. An -// open socket with no completed handshake must never hang silently: re-send -// the idempotent join a bounded number of times, then fail loudly and hand -// control to the reconnect path. -const JOIN_RETRY_MS = 400; -const MAX_JOIN_ATTEMPTS = 8; - -class ResilientWebSocketClientAdapter extends NetworkAdapter { - private socket?: WebSocket; - private ready = false; - private readyResolver?: () => void; - private readyPromise: Promise = new Promise((resolve) => { - this.readyResolver = resolve; - }); - private retryIntervalId?: IntervalId; - private retryTimeoutId?: TimeoutId; - private joinRetryId?: IntervalId; - private joinAttempts = 0; - private readonly retryInterval: number; - private readonly mode: SyncMode; - private abandonedHandshakeRetryAt = 0; - - remotePeerId?: PeerId; - - constructor(private readonly options: WebSocketSyncOptions) { - super(); - this.retryInterval = options.retryInterval ?? 5_000; - this.mode = options.mode ?? "best-effort"; - } - - isReady() { - return this.ready; - } - - whenReady() { - return this.readyPromise; - } - - connect(peerId: PeerId, peerMetadata?: PeerMetadata) { - if (Date.now() < this.abandonedHandshakeRetryAt) return; - - if (!this.socket || !this.peerId) { - this.peerId = peerId; - this.peerMetadata = peerMetadata ?? {}; - } else if (peerId !== this.peerId) { - this.reportError(new Error("Cannot reconnect websocket with a new peer id"), false); - return; - } else { - const previousSocket = this.socket; - this.closeSocket(previousSocket, { reportConnectingFailure: true }); - if (previousSocket.readyState !== WebSocket.CLOSED) { - return; - } - } - - if (!this.retryIntervalId && this.retryInterval > 0) { - this.retryIntervalId = setInterval(() => { - this.connect(peerId, peerMetadata); - }, this.retryInterval); - } - - this.options.onStatus?.({ state: "connecting", url: this.options.url }); - this.socket = new WebSocket(this.options.url, { - headers: syncHeaders(this.options), - }); - this.socket.binaryType = "arraybuffer"; - this.socket.addEventListener("open", this.onOpen); - this.socket.addEventListener("close", this.onClose); - this.socket.addEventListener("message", this.onMessage); - this.socket.addEventListener("error", this.onSocketError); - - setTimeout(() => this.forceReady(), 1_000); - this.join(); - } - - disconnect() { - if (this.retryIntervalId) clearInterval(this.retryIntervalId); - if (this.retryTimeoutId) clearTimeout(this.retryTimeoutId); - this.retryIntervalId = undefined; - this.retryTimeoutId = undefined; - this.clearJoinRetry(); - - if (this.socket) { - this.closeSocket(this.socket); - } - if (this.remotePeerId) { - this.emit("peer-disconnected", { peerId: this.remotePeerId }); - this.remotePeerId = undefined; - } - this.socket = undefined; - } - - send(message: FromClientMessage) { - if ("data" in message && message.data?.byteLength === 0) { - this.reportError(new Error("Tried to send a zero-length sync message"), false); - return; - } - if (!this.peerId || !this.socket || this.socket.readyState !== WebSocket.OPEN) { - if (this.mode === "required") { - this.reportError(new Error("Websocket not ready"), true); - } - return; - } - - try { - this.socket.send(toArrayBuffer(cbor.encode(message))); - } catch (error) { - this.reportError(toError(error), true); - } - } - - private onOpen = () => { - this.clearRetryInterval(); - this.abandonedHandshakeRetryAt = 0; - this.joinAttempts = 0; - this.join(); - this.clearJoinRetry(); - this.joinRetryId = setInterval(() => { - if (this.remotePeerId) { - this.clearJoinRetry(); - return; - } - this.joinAttempts += 1; - if (this.joinAttempts >= MAX_JOIN_ATTEMPTS) { - this.clearJoinRetry(); - this.reportError( - new Error( - `Peer handshake did not complete after ${MAX_JOIN_ATTEMPTS} join attempts; closing socket to trigger reconnect`, - ), - true, - ); - if (this.socket) this.closeSocket(this.socket); - return; - } - this.join(); - }, JOIN_RETRY_MS); - }; - - private onClose = () => { - this.clearJoinRetry(); - if (this.remotePeerId) { - this.emit("peer-disconnected", { peerId: this.remotePeerId }); - this.remotePeerId = undefined; - } - - const retryInMs = this.retryInterval > 0 ? this.retryInterval : undefined; - this.options.onStatus?.({ - state: "disconnected", - url: this.options.url, - retryInMs, - }); - - if (retryInMs && !this.retryTimeoutId) { - this.retryTimeoutId = setTimeout(() => { - this.retryTimeoutId = undefined; - if (this.peerId) this.connect(this.peerId, this.peerMetadata); - }, retryInMs); - } - }; - - private onMessage = (event: WebSocket.MessageEvent) => { - this.receiveMessage(event.data as Uint8Array); - }; - - private onSocketError = (event: Event | WebSocket.ErrorEvent) => { - this.reportError("error" in event ? toError(event.error) : new Error("WebSocket error"), true); - }; - - private join() { - if (!this.peerId || !this.socket) return; - if (this.socket.readyState === WebSocket.OPEN) { - this.send({ - type: "join", - senderId: this.peerId, - peerMetadata: this.peerMetadata ?? {}, - supportedProtocolVersions: [PROTOCOL_V1], - }); - } - } - - private receiveMessage(messageBytes: Uint8Array) { - let message: FromServerMessage; - try { - message = cbor.decode(new Uint8Array(messageBytes)); - } catch (error) { - this.reportError(toError(error), true); - return; - } - - if (messageBytes.byteLength === 0) { - this.reportError(new Error("Received a zero-length sync message"), true); - return; - } - - if (isPeerMessage(message)) { - this.forceReady(); - const isNewPeer = this.remotePeerId !== message.senderId; - this.remotePeerId = message.senderId; - this.clearRetryInterval(); - this.clearJoinRetry(); - if (!isNewPeer) { - // A re-sent join can earn a duplicate peer reply; the handshake is - // already complete, so don't re-announce the peer downstream. - return; - } - this.options.onStatus?.({ - state: "connected", - url: this.options.url, - peerId: message.senderId, - }); - this.emit("peer-candidate", { - peerId: message.senderId, - peerMetadata: message.peerMetadata, - }); - } else if (isErrorMessage(message)) { - this.reportError(new Error(message.message), true); - } else { - this.emit("message", message); - } - } - - private forceReady() { - if (!this.ready) { - this.ready = true; - this.readyResolver?.(); - } - } - - private clearRetryInterval() { - if (this.retryIntervalId) clearInterval(this.retryIntervalId); - this.retryIntervalId = undefined; - } - - private clearJoinRetry() { - if (this.joinRetryId) clearInterval(this.joinRetryId); - this.joinRetryId = undefined; - } - - private reportError(error: Error, recoverable: boolean) { - this.options.onError?.(error); - this.options.onStatus?.({ - state: "failed", - url: this.options.url, - error, - recoverable, - }); - if (this.mode === "required" && !recoverable) { - throw error; - } - } - - private removeSocketListeners(socket: WebSocket) { - socket.removeEventListener("open", this.onOpen); - socket.removeEventListener("close", this.onClose); - socket.removeEventListener("message", this.onMessage); - socket.removeEventListener("error", this.onSocketError); - } - - private closeSocket( - socket: WebSocket, - options: { reportConnectingFailure?: boolean } = {}, - ) { - this.removeSocketListeners(socket); - socket.addEventListener("error", swallowSocketAbortError); - if (socket.readyState === WebSocket.CLOSED || socket.readyState === WebSocket.CLOSING) { - return; - } - - if (socket.readyState === WebSocket.CONNECTING) { - if (options.reportConnectingFailure) { - this.reportError(new Error("WebSocket handshake timed out"), true); - } - this.abandonedHandshakeRetryAt = Date.now() + Math.max(this.retryInterval, 5_000); - this.destroySocketTransport(socket); - socket.terminate(); - return; - } - - socket.close(); - const timeout = setTimeout(() => { - if (socket.readyState !== WebSocket.CLOSED) { - this.destroySocketTransport(socket); - socket.terminate(); - } - }, 1_000); - timeout.unref?.(); - } - - private destroySocketTransport(socket: WebSocket) { - const internals = socket as WebSocket & WebSocketInternals; - internals._req?.abort?.(); - if (internals._req?.socket && !internals._req.socket.destroyed) { - internals._req.socket.destroy(); - } - if (internals._socket && !internals._socket.destroyed) { - internals._socket.destroy(); - } - } -} - -function syncHeaders(options: WebSocketSyncOptions) { - const headers = { ...options.headers }; - if (options.auth?.type === "bearer") { - headers.authorization = `Bearer ${options.auth.token}`; - } else if (options.auth?.type === "api-key") { - headers[options.auth.header ?? "x-api-key"] = options.auth.token; - } - return headers; -} - -function hasHeaders(headers: Record | undefined) { - return Boolean(headers && Object.keys(headers).length > 0); -} - -function isPeerMessage(message: FromServerMessage): message is PeerMessage { - return message.type === "peer"; -} - -function isErrorMessage(message: FromServerMessage): message is ErrorMessage { - return message.type === "error"; -} - -function toArrayBuffer(bytes: Uint8Array) { - return bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + bytes.byteLength); -} - -function toError(error: unknown) { - return error instanceof Error ? error : new Error(String(error)); -} diff --git a/packages/client/src/testing.ts b/packages/client/src/testing.ts deleted file mode 100644 index f5dda57..0000000 --- a/packages/client/src/testing.ts +++ /dev/null @@ -1,36 +0,0 @@ -import { createMemoryLooms, type MemoryLoomsOptions } from "lync-core/memory"; -import { createMemoryLoomIndexes, type MemoryLoomIndexesOptions } from "lync-index/memory"; -import { createLoomClient } from "./create.js"; -import type { LoomClient } from "./types.js"; - -export interface TestLoomClientOptions { - createId?: () => string; - now?: () => number; - looms?: MemoryLoomsOptions; - indexes?: MemoryLoomIndexesOptions; -} - -export function createTestLoomClient< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, ->( - options: TestLoomClientOptions = {}, -): LoomClient { - const defaultOptions = { - createId: options.createId, - now: options.now, - }; - const looms = createMemoryLooms({ - ...defaultOptions, - ...options.looms, - }); - const indexes = createMemoryLoomIndexes({ - ...defaultOptions, - ...options.indexes, - }); - - return createLoomClient({ looms, indexes }); -} diff --git a/packages/client/src/types.ts b/packages/client/src/types.ts deleted file mode 100644 index e8ecc7d..0000000 --- a/packages/client/src/types.ts +++ /dev/null @@ -1,77 +0,0 @@ -import type { Repo } from "@automerge/automerge-repo"; -import type { - Loom, - LoomReference, - Looms, - Turn, - indexRef, - loomRef, - referenceFromUrl, - referenceToUrl, - threadRef, - turnRef, - encodeReference, - decodeReference, - parseReference, -} from "lync-core"; -import type { LoomIndex, LoomIndexes } from "lync-index"; - -export type ReferenceHelpers = { - loom: typeof loomRef; - turn: typeof turnRef; - thread: typeof threadRef; - index: typeof indexRef; - encode: typeof encodeReference; - decode: typeof decodeReference; - parse: typeof parseReference; - toUrl: typeof referenceToUrl; - fromUrl: typeof referenceFromUrl; -}; - -export type OpenedReference< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, -> = - | { - kind: "loom"; - ref: Extract; - loom: Loom; - } - | { - kind: "turn"; - ref: Extract; - loom: Loom; - turn: Turn; - } - | { - kind: "thread"; - ref: Extract; - loom: Loom; - thread: Turn[]; - target: Turn; - } - | { - kind: "index"; - ref: Extract; - index: LoomIndex; - }; - -export interface LoomClient< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, - TEntryMeta = unknown, - TIndexMeta = unknown, -> { - repo?: Repo; - looms: Looms; - indexes: LoomIndexes; - references: ReferenceHelpers; - openReference( - ref: LoomReference, - ): Promise>; - close(): Promise; -} diff --git a/packages/client/test/browser.test.ts b/packages/client/test/browser.test.ts deleted file mode 100644 index 2337f44..0000000 --- a/packages/client/test/browser.test.ts +++ /dev/null @@ -1,84 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { Repo } from "@automerge/automerge-repo"; -import { - createBrowserLoomClient, - type BrowserLoomClientOptions, -} from "../src/browser.js"; - -describe("browser loom client", () => { - it("creates looms and indexes on one shared browser repo", async () => { - const repo = new Repo(); - const client = createBrowserLoomClient< - { text: string }, - { title: string }, - never, - { title: string }, - { app: string } - >({ - repo, - }); - - const info = await client.looms.create({ title: "Story" }); - const loom = await client.looms.open(info.id); - const turn = await loom.appendTurn(null, { text: "Hello" }); - - const index = await client.indexes.create({ app: "textile" }); - await index.addLoom(client.references.loom(info.id), { title: "Story" }); - - expect(info.id.startsWith("automerge:")).toBe(true); - expect(turn.loomId).toBe(info.id); - await expect(index.entries()).resolves.toEqual([ - expect.objectContaining({ - ref: { v: 1, kind: "loom", loomId: info.id }, - title: "Story", - }), - ]); - expect(client.repo).toBe(repo); - - await client.close(); - }); - - it("opens loom, turn, thread, and index references", async () => { - const client = createBrowserLoomClient<{ text: string }>({ repo: new Repo() }); - const info = await client.looms.create(); - const loom = await client.looms.open(info.id); - const first = await loom.appendTurn(null, { text: "A" }); - const second = await loom.appendTurn(first.id, { text: "B" }); - const index = await client.indexes.create(); - - await expect(client.openReference(client.references.loom(info.id))).resolves.toMatchObject({ - kind: "loom", - loom: { id: info.id }, - }); - await expect( - client.openReference(client.references.turn(info.id, second.id)), - ).resolves.toMatchObject({ kind: "turn", turn: second }); - await expect( - client.openReference(client.references.thread(info.id, second.id)), - ).resolves.toMatchObject({ kind: "thread", thread: [first, second], target: second }); - await expect(client.openReference(client.references.index(index.id))).resolves.toMatchObject({ - kind: "index", - index: { id: index.id }, - }); - - await client.close(); - }); - - it("assembles a browser client from runtime options", async () => { - const options: BrowserLoomClientOptions = { - repo: new Repo(), - browser: { - indexedDb: false, - broadcastChannel: false, - websocket: false, - }, - }; - - const client = createBrowserLoomClient(options); - - expect(client.looms).toBeDefined(); - expect(client.indexes).toBeDefined(); - - await client.close(); - }); -}); diff --git a/packages/client/test/node.test.ts b/packages/client/test/node.test.ts deleted file mode 100644 index 4b389eb..0000000 --- a/packages/client/test/node.test.ts +++ /dev/null @@ -1,240 +0,0 @@ -import fs from "node:fs/promises"; -import http from "node:http"; -import type net from "node:net"; -import os from "node:os"; -import path from "node:path"; -import { cbor, type PeerId } from "@automerge/automerge-repo/slim"; -import { describe, expect, it } from "vitest"; -import { WebSocketServer } from "isomorphic-ws"; -import { createNodeLoomClient, type SyncStatus } from "../src/node.js"; -import { createWebSocketSyncAdapter } from "../src/sync.js"; - -type JoinMessage = { - type: "join"; - senderId: PeerId; -}; - -describe("node loom client", () => { - it("persists looms through the filesystem storage adapter", async () => { - const storageDir = await fs.mkdtemp(path.join(os.tmpdir(), "lync-node-")); - - const client = createNodeLoomClient<{ text: string }>({ - storageDir, - syncUrl: false, - }); - const info = await client.looms.create({ title: "Node script" }); - const loom = await client.looms.open(info.id); - const first = await loom.appendTurn(null, { text: "Hello" }); - await client.close(); - - const reopened = createNodeLoomClient<{ text: string }>({ - storageDir, - syncUrl: false, - }); - const reopenedLoom = await reopened.looms.open(info.id); - - await expect(reopenedLoom.childrenOf(null)).resolves.toEqual([first]); - - await reopened.close(); - await fs.rm(storageDir, { recursive: true, force: true }); - }); - - it("can run without persistence or network for short-lived scripts", async () => { - const client = createNodeLoomClient<{ text: string }>({ - storageDir: false, - syncUrl: false, - }); - - const info = await client.looms.create(); - const loom = await client.looms.open(info.id); - - await expect(loom.appendTurn(null, { text: "Transient" })).resolves.toMatchObject({ - loomId: info.id, - parentId: null, - payload: { text: "Transient" }, - }); - - await client.close(); - }); - - it("keeps local loom operations alive when websocket sync is unavailable", async () => { - const server = http.createServer(); - server.on("upgrade", (_request, socket) => { - socket.end("HTTP/1.1 503 Service Unavailable\r\nConnection: close\r\n\r\n"); - }); - await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); - const address = server.address(); - if (!address || typeof address === "string") throw new Error("Expected TCP server"); - - const statuses: SyncStatus[] = []; - const client = createNodeLoomClient<{ text: string }>({ - storageDir: false, - sync: { - url: `ws://127.0.0.1:${address.port}/lync`, - retryInterval: 0, - onStatus: (status) => statuses.push(status), - }, - }); - - const info = await client.looms.create({ title: "Offline tolerant" }); - const loom = await client.looms.open(info.id); - await expect(loom.appendTurn(null, { text: "Still works" })).resolves.toMatchObject({ - payload: { text: "Still works" }, - }); - await new Promise((resolve) => setTimeout(resolve, 50)); - - expect(statuses.some((status) => status.state === "failed")).toBe(true); - - await client.close(); - await new Promise((resolve) => server.close(() => resolve())); - }); - - it("closes a hanging websocket before retrying", async () => { - const upgradeSockets = new Set(); - const statuses: SyncStatus[] = []; - const server = http.createServer(); - server.on("upgrade", (_request, socket) => { - upgradeSockets.add(socket); - socket.on("close", () => upgradeSockets.delete(socket)); - }); - await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); - const address = server.address(); - if (!address || typeof address === "string") throw new Error("Expected TCP server"); - - const adapter = createWebSocketSyncAdapter({ - url: `ws://127.0.0.1:${address.port}/lync`, - retryInterval: 20, - onStatus: (status) => statuses.push(status), - }); - - adapter.connect("peer-a" as PeerId); - await new Promise((resolve) => setTimeout(resolve, 120)); - - expect(upgradeSockets.size).toBeLessThanOrEqual(1); - expect( - statuses.some( - (status) => - status.state === "failed" && - status.recoverable && - status.error.message === "WebSocket handshake timed out", - ), - ).toBe(true); - - adapter.disconnect(); - for (const socket of upgradeSockets) socket.destroy(); - await new Promise((resolve) => server.close(() => resolve())); - }); - - it("does not reconnect after the peer handshake completes", async () => { - const server = new WebSocketServer({ port: 0 }); - await new Promise((resolve) => server.once("listening", resolve)); - let connectionCount = 0; - server.on("connection", (socket) => { - connectionCount += 1; - socket.send( - cbor.encode({ - type: "peer", - senderId: "peer-server", - peerMetadata: {}, - selectedProtocolVersion: "1", - targetId: "peer-a", - }), - ); - }); - const address = server.address(); - if (!address || typeof address === "string") throw new Error("Expected TCP server"); - - const statuses: SyncStatus[] = []; - const adapter = createWebSocketSyncAdapter({ - url: `ws://127.0.0.1:${address.port}/lync`, - retryInterval: 20, - onStatus: (status) => statuses.push(status), - }); - - adapter.connect("peer-a" as PeerId); - await new Promise((resolve) => setTimeout(resolve, 100)); - - expect(connectionCount).toBe(1); - expect(statuses.some((status) => status.state === "connected")).toBe(true); - - adapter.disconnect(); - await new Promise((resolve) => server.close(() => resolve())); - }); - - it("can use the native Automerge websocket adapter for unauthenticated sync", async () => { - const server = new WebSocketServer({ port: 0 }); - await new Promise((resolve) => server.once("listening", resolve)); - let receivedJoin = false; - server.on("connection", (socket) => { - socket.on("message", (messageBytes) => { - const message = cbor.decode(new Uint8Array(messageBytes as Buffer)) as JoinMessage; - if (message.type !== "join") return; - receivedJoin = true; - socket.send( - cbor.encode({ - type: "peer", - senderId: "peer-server", - peerMetadata: {}, - selectedProtocolVersion: "1", - targetId: message.senderId, - }), - ); - }); - }); - const address = server.address(); - if (!address || typeof address === "string") throw new Error("Expected TCP server"); - - const client = createNodeLoomClient<{ text: string }>({ - storageDir: false, - sync: { - url: `ws://127.0.0.1:${address.port}/lync`, - retryInterval: 20, - adapter: "native", - }, - }); - - await new Promise((resolve) => setTimeout(resolve, 50)); - expect(receivedJoin).toBe(true); - - await client.close(); - await new Promise((resolve) => server.close(() => resolve())); - }); - - it("rejects native adapter selection when auth headers are required", () => { - expect(() => - createWebSocketSyncAdapter({ - url: "ws://127.0.0.1:3030/lync", - adapter: "native", - auth: { type: "bearer", token: "secret" }, - }), - ).toThrow("does not support auth headers"); - }); - - it("does not report a timeout when disconnecting during a handshake", async () => { - const upgradeSockets = new Set(); - const statuses: SyncStatus[] = []; - const server = http.createServer(); - server.on("upgrade", (_request, socket) => { - upgradeSockets.add(socket); - socket.on("close", () => upgradeSockets.delete(socket)); - }); - await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); - const address = server.address(); - if (!address || typeof address === "string") throw new Error("Expected TCP server"); - - const adapter = createWebSocketSyncAdapter({ - url: `ws://127.0.0.1:${address.port}/lync`, - retryInterval: 20, - onStatus: (status) => statuses.push(status), - }); - - adapter.connect("peer-a" as PeerId); - adapter.disconnect(); - await new Promise((resolve) => setTimeout(resolve, 50)); - - expect(statuses.some((status) => status.state === "failed")).toBe(false); - - for (const socket of upgradeSockets) socket.destroy(); - await new Promise((resolve) => server.close(() => resolve())); - }); -}); diff --git a/packages/client/test/testing.test.ts b/packages/client/test/testing.test.ts deleted file mode 100644 index 575005c..0000000 --- a/packages/client/test/testing.test.ts +++ /dev/null @@ -1,78 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { - assertTextStoryThread, - textStoryLoomMeta, - type TextStoryLoomMeta, - type TextStoryTurnMeta, - type TextStoryTurnPayload, -} from "lync-core/profiles/text-story"; -import { createTestLoomClient } from "../src/testing.js"; - -describe("test loom client", () => { - it("provides deterministic in-memory looms, indexes, and references", async () => { - let nextId = 1; - const client = createTestLoomClient<{ text: string }, { title: string }>({ - createId: () => `id-${nextId++}`, - now: () => 123, - }); - - const info = await client.looms.create({ title: "Test story" }); - const loom = await client.looms.open(info.id); - const first = await loom.appendTurn(null, { text: "A" }); - const second = await loom.appendTurn(first.id, { text: "B" }); - const index = await client.indexes.create(); - await index.addLoom(client.references.loom(info.id), { title: "Test story" }); - - await expect(client.openReference(client.references.thread(info.id, second.id))).resolves.toMatchObject({ - kind: "thread", - thread: [first, second], - }); - await expect(index.entries()).resolves.toEqual([ - expect.objectContaining({ - ref: client.references.loom(info.id), - title: "Test story", - }), - ]); - - await client.close(); - }); - - it("lets independent writers create text-story looms that readers open by reference", async () => { - let nextId = 1; - const writer = createTestLoomClient< - TextStoryTurnPayload, - TextStoryLoomMeta, - TextStoryTurnMeta - >({ - createId: () => `id-${nextId++}`, - now: () => 456, - }); - - const info = await writer.looms.create( - textStoryLoomMeta({ title: "External story" }), - ); - const loom = await writer.looms.open(info.id); - const opening = await loom.appendTurn( - null, - { text: "Once" }, - { role: "prose" }, - ); - const next = await loom.appendTurn( - opening.id, - { text: " later" }, - { role: "prose" }, - ); - - const opened = await writer.openReference( - writer.references.thread(info.id, next.id), - ); - expect(opened.kind).toBe("thread"); - if (opened.kind !== "thread") throw new Error("Expected thread reference"); - assertTextStoryThread(opened.thread); - expect(opened.thread.map((turn) => turn.payload.text).join("")).toBe( - "Once later", - ); - - await writer.close(); - }); -}); diff --git a/packages/client/tsconfig.json b/packages/client/tsconfig.json deleted file mode 100644 index df59da5..0000000 --- a/packages/client/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*.ts"] -} diff --git a/packages/core/README.md b/packages/core/README.md index 169c538..b74bb3f 100644 --- a/packages/core/README.md +++ b/packages/core/README.md @@ -5,8 +5,8 @@ event, files are merged losslessly by set-union, and every physical line is classified and kept. ```ts -import { parseLoreFiles } from "lync-core/lore/events"; -import { createMemoryEventStore } from "lync-core/lore/memory-log"; +import { parseLyncFiles } from "lync-core/events"; +import { createMemoryEventStore } from "lync-core/memory-log"; const store = createMemoryEventStore(); await store.append({ @@ -19,7 +19,7 @@ await store.append({ payload: { meta: { title: "Story" } }, }); -const parsed = parseLoreFiles([{ file: "story.lync", bytes: line }]); +const parsed = parseLyncFiles([{ file: "story.lync", bytes: line }]); console.log(parsed.unionEventIds); ``` diff --git a/packages/core/package.json b/packages/core/package.json index f4bd128..4d58c78 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -23,16 +23,6 @@ "import": "./dist/index.js", "default": "./dist/index.js" }, - "./automerge": { - "types": "./dist/automerge.d.ts", - "import": "./dist/automerge.js", - "default": "./dist/automerge.js" - }, - "./browser": { - "types": "./dist/browser.d.ts", - "import": "./dist/browser.js", - "default": "./dist/browser.js" - }, "./errors": { "types": "./dist/errors.d.ts", "import": "./dist/errors.js", @@ -43,40 +33,40 @@ "import": "./dist/memory.js", "default": "./dist/memory.js" }, - "./lore/events": { - "types": "./dist/lore/events.d.ts", - "import": "./dist/lore/events.js", - "default": "./dist/lore/events.js" + "./events": { + "types": "./dist/events.d.ts", + "import": "./dist/events.js", + "default": "./dist/events.js" }, - "./lore/file-log": { - "types": "./dist/lore/file-log.d.ts", - "import": "./dist/lore/file-log.js", - "default": "./dist/lore/file-log.js" + "./file-log": { + "types": "./dist/file-log.d.ts", + "import": "./dist/file-log.js", + "default": "./dist/file-log.js" }, - "./lore/idb-log": { - "types": "./dist/lore/idb-log.d.ts", - "import": "./dist/lore/idb-log.js", - "default": "./dist/lore/idb-log.js" + "./idb-log": { + "types": "./dist/idb-log.d.ts", + "import": "./dist/idb-log.js", + "default": "./dist/idb-log.js" }, - "./lore/looms": { - "types": "./dist/lore/looms.d.ts", - "import": "./dist/lore/looms.js", - "default": "./dist/lore/looms.js" + "./looms": { + "types": "./dist/looms.d.ts", + "import": "./dist/looms.js", + "default": "./dist/looms.js" }, - "./lore/memory-log": { - "types": "./dist/lore/memory-log.d.ts", - "import": "./dist/lore/memory-log.js", - "default": "./dist/lore/memory-log.js" + "./memory-log": { + "types": "./dist/memory-log.d.ts", + "import": "./dist/memory-log.js", + "default": "./dist/memory-log.js" }, - "./lore/store": { - "types": "./dist/lore/store.d.ts", - "import": "./dist/lore/store.js", - "default": "./dist/lore/store.js" + "./store": { + "types": "./dist/store.d.ts", + "import": "./dist/store.js", + "default": "./dist/store.js" }, - "./lore/views": { - "types": "./dist/lore/views.d.ts", - "import": "./dist/lore/views.js", - "default": "./dist/lore/views.js" + "./views": { + "types": "./dist/views.d.ts", + "import": "./dist/views.js", + "default": "./dist/views.js" }, "./profiles/text-story": { "types": "./dist/profiles/text-story.d.ts", @@ -100,8 +90,5 @@ "scripts": { "build": "tsc -p tsconfig.json", "typecheck": "tsc -p tsconfig.json --noEmit" - }, - "dependencies": { - "@automerge/automerge": "^3.2.6" } } diff --git a/packages/core/src/automerge.ts b/packages/core/src/automerge.ts deleted file mode 100644 index 92db889..0000000 --- a/packages/core/src/automerge.ts +++ /dev/null @@ -1,330 +0,0 @@ -import { DocHandle, Repo, type AutomergeUrl } from "@automerge/automerge-repo"; -import { - brokenTopology, - closedHandle, - cycleDetected, - duplicateTurnId, - missingParent, - unknownLoom, -} from "./errors.js"; -import { assertJsonEncodable, cloneJson } from "./json.js"; -import type { - Loom, - LoomEvent, - LoomId, - LoomInfo, - LoomListener, - Looms, - LoomSnapshot, - Turn, - TurnId, -} from "./types.js"; - -const ROOT_CHILDREN_KEY = "__root__"; - -type LoomDoc = { - version: 1; - root: LoomInfo; - nodes: Record>; - children: Record; -}; - -export interface AutomergeLoomsOptions { - repo?: Repo; - createTurnId?: () => string; - now?: () => number; -} - -export function createAutomergeLooms< - TPayload = unknown, - TLoomMeta = unknown, - TTurnMeta = unknown, ->( - options: AutomergeLoomsOptions = {}, -): Looms { - const repo = options.repo ?? new Repo(); - const createTurnId = options.createTurnId ?? (() => crypto.randomUUID()); - const now = options.now ?? (() => Date.now()); - - return { - async create(meta) { - assertJsonEncodable(meta, "loom meta"); - const handle = repo.create>({ - version: 1, - root: { - id: "" as LoomId, - ...(meta === undefined ? {} : { meta: cloneJson(meta) }), - createdAt: now(), - }, - nodes: {}, - children: { [ROOT_CHILDREN_KEY]: [] }, - }); - handle.change((doc) => { - doc.root.id = handle.url; - }); - return cloneJson(handle.doc().root); - }, - - async get(loomId) { - try { - const handle = await repo.find>( - loomId as AutomergeUrl, - ); - await handle.whenReady(); - return cloneJson(handle.doc().root); - } catch { - return null; - } - }, - - async open(loomId) { - let handle: DocHandle>; - try { - handle = await repo.find>( - loomId as AutomergeUrl, - ); - await handle.whenReady(); - } catch { - throw unknownLoom(loomId); - } - return new AutomergeLoom(loomId, handle, createTurnId, now); - }, - - async import(snapshot) { - validateSnapshot(snapshot); - const handle = repo.create>({ - version: 1, - root: { - id: "" as LoomId, - ...(snapshot.loom.meta === undefined ? {} : { meta: cloneJson(snapshot.loom.meta) }), - createdAt: snapshot.loom.createdAt, - }, - nodes: {}, - children: { [ROOT_CHILDREN_KEY]: [] }, - }); - handle.change((doc) => { - doc.root.id = handle.url; - for (const imported of snapshot.turns) { - const turn = { - ...cloneJson(imported), - loomId: handle.url, - }; - doc.nodes[turn.id] = turn; - doc.children[turn.id] ??= []; - const key = parentKeyOf(turn.parentId); - doc.children[key] ??= []; - doc.children[key].push(turn.id); - } - }); - return cloneJson(handle.doc().root); - }, - }; -} - -class AutomergeLoom - implements Loom -{ - private closed = false; - private listeners = new Set>(); - private knownTurnIds: Set; - - constructor( - readonly id: LoomId, - private readonly handle: DocHandle>, - private readonly createTurnId: () => string, - private readonly now: () => number, - ) { - this.knownTurnIds = new Set(Object.keys(this.handle.doc().nodes ?? {})); - this.handle.on("change", ({ doc }) => { - const currentIds = new Set(Object.keys(doc.nodes ?? {})); - for (const turnId of currentIds) { - if (!this.knownTurnIds.has(turnId)) { - const turn = doc.nodes[turnId]; - if (turn) this.emit({ type: "turn-added", loomId: this.id, turn: cloneJson(turn) }); - } - } - this.knownTurnIds = currentIds; - }); - } - - async info(): Promise> { - this.assertOpen(); - return cloneJson(this.doc().root); - } - - async updateMeta(meta: TLoomMeta): Promise> { - this.assertOpen(); - assertJsonEncodable(meta, "loom meta"); - this.handle.change((doc) => { - doc.root.meta = cloneJson(meta) as TLoomMeta; - }); - const loom = cloneJson(this.doc().root); - this.emit({ type: "loom-updated", loom }); - return loom; - } - - async appendTurn( - parentId: TurnId | null, - payload: TPayload, - meta?: TTurnMeta, - ): Promise> { - this.assertOpen(); - assertJsonEncodable(payload, "turn payload"); - assertJsonEncodable(meta, "turn meta"); - if (parentId !== null && !this.doc().nodes[parentId]) throw missingParent(parentId); - - const turnId = this.createTurnId(); - if (turnId === ROOT_CHILDREN_KEY || this.doc().nodes[turnId]) throw duplicateTurnId(turnId); - - const turn = omitUndefined({ - id: turnId, - loomId: this.id, - parentId, - payload: cloneJson(payload), - meta: cloneJson(meta), - createdAt: this.now(), - }) as Turn; - - this.handle.change((doc) => { - doc.nodes[turn.id] = turn; - doc.children[turn.id] ??= []; - const key = parentKeyOf(parentId); - doc.children[key] ??= []; - doc.children[key].push(turn.id); - }); - - return cloneJson(turn); - } - - async getTurn(turnId: TurnId): Promise | null> { - this.assertOpen(); - const turn = this.doc().nodes[turnId]; - return turn ? cloneJson(turn) : null; - } - - async hasTurn(turnId: TurnId): Promise { - this.assertOpen(); - return Boolean(this.doc().nodes[turnId]); - } - - async childrenOf(parentId: TurnId | null): Promise[]> { - this.assertOpen(); - const doc = this.doc(); - if (parentId !== null && !doc.nodes[parentId]) throw missingParent(parentId); - return (doc.children[parentKeyOf(parentId)] ?? []).map((turnId) => { - const turn = doc.nodes[turnId]; - if (!turn) throw brokenTopology(`Child list references missing turn: ${turnId}`); - return cloneJson(turn); - }); - } - - async threadTo(turnId: TurnId): Promise[]> { - this.assertOpen(); - const doc = this.doc(); - const thread: Turn[] = []; - const seen = new Set(); - let currentId: TurnId | null = turnId; - - while (currentId !== null) { - if (seen.has(currentId)) throw cycleDetected(currentId); - seen.add(currentId); - const turn: Turn | undefined = doc.nodes[currentId]; - if (!turn) throw brokenTopology(`Thread references missing turn: ${currentId}`); - thread.push(turn); - currentId = turn.parentId; - } - - return cloneJson(thread.reverse()); - } - - async leaves(): Promise[]> { - this.assertOpen(); - const doc = this.doc(); - const leaves: Turn[] = []; - const visit = (parentId: TurnId | null) => { - for (const turnId of doc.children[parentKeyOf(parentId)] ?? []) { - const turn = doc.nodes[turnId]; - if (!turn) throw brokenTopology(`Child list references missing turn: ${turnId}`); - if ((doc.children[turnId] ?? []).length === 0) { - leaves.push(turn); - } else { - visit(turnId); - } - } - }; - visit(null); - return cloneJson(leaves); - } - - subscribe(listener: LoomListener): () => void { - this.assertOpen(); - this.listeners.add(listener); - return () => this.listeners.delete(listener); - } - - async export(): Promise> { - this.assertOpen(); - const doc = this.doc(); - const turns: Turn[] = []; - const visit = (parentId: TurnId | null) => { - for (const turnId of doc.children[parentKeyOf(parentId)] ?? []) { - const turn = doc.nodes[turnId]; - if (!turn) throw brokenTopology(`Child list references missing turn: ${turnId}`); - turns.push(turn); - visit(turnId); - } - }; - visit(null); - return cloneJson({ loom: doc.root, turns }); - } - - close(): void { - this.closed = true; - this.listeners.clear(); - } - - private doc() { - return this.handle.doc(); - } - - private assertOpen(): void { - if (this.closed) throw closedHandle(); - } - - private emit(event: LoomEvent): void { - for (const listener of this.listeners) listener(event); - } -} - -function validateSnapshot(snapshot: LoomSnapshot): void { - const ids = new Set(); - for (const turn of snapshot.turns) { - if (turn.id === ROOT_CHILDREN_KEY) throw duplicateTurnId(turn.id); - if (ids.has(turn.id)) throw duplicateTurnId(turn.id); - ids.add(turn.id); - } - for (const turn of snapshot.turns) { - if (turn.parentId !== null && !ids.has(turn.parentId)) throw missingParent(turn.parentId); - } - for (const turn of snapshot.turns) { - const seen = new Set(); - let currentId: TurnId | null = turn.id; - while (currentId !== null) { - if (seen.has(currentId)) throw cycleDetected(currentId); - seen.add(currentId); - const current = snapshot.turns.find((candidate) => candidate.id === currentId); - if (!current) throw brokenTopology(`Missing turn while validating thread: ${currentId}`); - currentId = current.parentId; - } - } -} - -function parentKeyOf(parentId: TurnId | null): string { - return parentId ?? ROOT_CHILDREN_KEY; -} - -function omitUndefined>(value: T): T { - return Object.fromEntries( - Object.entries(value).filter(([, entryValue]) => entryValue !== undefined), - ) as T; -} diff --git a/packages/core/src/browser.ts b/packages/core/src/browser.ts deleted file mode 100644 index 9b9343b..0000000 --- a/packages/core/src/browser.ts +++ /dev/null @@ -1,105 +0,0 @@ -import "@automerge/automerge"; -import { Repo, type RepoConfig } from "@automerge/automerge-repo"; -import { IndexedDBStorageAdapter } from "@automerge/automerge-repo-storage-indexeddb"; -import { BroadcastChannelNetworkAdapter } from "@automerge/automerge-repo-network-broadcastchannel"; -import { WebSocketClientAdapter } from "@automerge/automerge-repo-network-websocket"; - -type StorageConstructor = new (database?: string, store?: string) => unknown; -type BroadcastConstructor = new (options?: { - channelName: string; - peerWaitMs?: number; -}) => unknown; -type WebSocketConstructor = new (url: string, retryInterval?: number) => unknown; - -export interface BrowserAutomergeRepoOptions { - location?: Pick; - syncPath?: string; - indexedDb?: false | { - database?: string; - store?: string; - }; - broadcastChannel?: false | { - channelName?: string; - peerWaitMs?: number; - }; - websocket?: false | true | { - url: string; - retryInterval?: number; - }; - adapters?: { - IndexedDBStorageAdapter?: StorageConstructor; - BroadcastChannelNetworkAdapter?: BroadcastConstructor; - WebSocketClientAdapter?: WebSocketConstructor; - }; -} - -export function createBrowserAutomergeRepo(options: BrowserAutomergeRepoOptions = {}): Repo { - return new Repo(createBrowserAutomergeRepoConfig(options) as RepoConfig); -} - -export function createBrowserAutomergeRepoConfig( - options: BrowserAutomergeRepoOptions = {}, -): { - storage?: unknown; - network: unknown[]; -} { - const IndexedDB = - options.adapters?.IndexedDBStorageAdapter ?? IndexedDBStorageAdapter; - const Broadcast = - options.adapters?.BroadcastChannelNetworkAdapter ?? BroadcastChannelNetworkAdapter; - const WebSocket = - options.adapters?.WebSocketClientAdapter ?? WebSocketClientAdapter; - - const indexedDbOptions = options.indexedDb ?? {}; - const broadcastOptions = options.broadcastChannel ?? {}; - const websocketOptions = options.websocket ?? true; - - const network = []; - if (broadcastOptions !== false) { - network.push( - new Broadcast({ - channelName: broadcastOptions.channelName ?? "lync", - peerWaitMs: broadcastOptions.peerWaitMs, - }), - ); - } - if (websocketOptions !== false) { - const websocketUrl = - websocketOptions === true - ? defaultWebSocketUrl({ - location: options.location, - path: options.syncPath, - }) - : websocketOptions.url; - const retryInterval = - websocketOptions === true ? undefined : websocketOptions.retryInterval; - if (websocketUrl) network.push(new WebSocket(websocketUrl, retryInterval)); - } - - return { - storage: - indexedDbOptions === false - ? undefined - : new IndexedDB(indexedDbOptions.database, indexedDbOptions.store), - network, - }; -} - -export interface DefaultWebSocketUrlOptions { - location?: Pick; - path?: string; -} - -export function defaultWebSocketUrl(options: DefaultWebSocketUrlOptions = {}) { - const location = - options.location ?? - (typeof window === "undefined" ? undefined : window.location); - if (!location) return null; - const protocol = location.protocol === "https:" ? "wss:" : "ws:"; - const path = normalizeSyncPath(options.path ?? "/lync"); - return `${protocol}//${location.host}${path}`; -} - -function normalizeSyncPath(path: string) { - return path.startsWith("/") ? path : `/${path}`; -} diff --git a/packages/core/src/lore/events.ts b/packages/core/src/events.ts similarity index 86% rename from packages/core/src/lore/events.ts rename to packages/core/src/events.ts index b282512..5600bba 100644 --- a/packages/core/src/lore/events.ts +++ b/packages/core/src/events.ts @@ -1,13 +1,13 @@ import { sha256Hex } from "./sha256.js"; -export type LoreLineClass = +export type LyncLineClass = | "accepted" | "nonconforming" | "garbage" | "damaged" | "conflict-variant"; -export interface LoreEventBody { +export interface LyncEventBody { v: number; id: string; kind: string; @@ -20,17 +20,17 @@ export interface LoreEventBody { [key: string]: unknown; } -export interface LoreLineDiagnostic { +export interface LyncLineDiagnostic { file: string; line: number; - class: LoreLineClass; + class: LyncLineClass; reason: string; id?: string; hasDigest?: boolean; hasSig?: boolean; duplicateSighting?: boolean; metadataDisagreement?: boolean; - event?: LoreEventBody; + event?: LyncEventBody; bytes: Uint8Array; terminator: "" | "\n"; bodyBytes?: Uint8Array; @@ -40,17 +40,17 @@ export interface LoreLineDiagnostic { nonconformingReasons?: string[]; } -export interface LoreConflictVariant { +export interface LyncConflictVariant { id: string; digest: string; file: string; line: number; bytes: Uint8Array; bodyBytes: Uint8Array; - event: LoreEventBody; + event: LyncEventBody; } -export interface LorePendingDiagnostic { +export interface LyncPendingDiagnostic { missingParent: string; digest: string; file: string; @@ -59,27 +59,27 @@ export interface LorePendingDiagnostic { bytes: Uint8Array; } -export interface LoreObstacle { +export interface LyncObstacle { class: "cycle" | "dangling" | "unavailable-due-to-conflict"; ids?: string[]; missing?: string; id?: string; } -export interface LoreParseResult { - lines: LoreLineDiagnostic[]; +export interface LyncParseResult { + lines: LyncLineDiagnostic[]; unionEventIds: string[]; viewEligibleIds: string[]; conflictIds: string[]; - conflictVariants: LoreConflictVariant[]; - pending: LorePendingDiagnostic[]; + conflictVariants: LyncConflictVariant[]; + pending: LyncPendingDiagnostic[]; pendingOverflowCount: number; suppression: { suppressedPayloadIds: string[]; notSuppressedIds: string[]; danglingTargetNoEffectUntilUnion: string[]; }; - graphDiagnostics: LoreObstacle[]; + graphDiagnostics: LyncObstacle[]; } interface JsonParsed { @@ -103,7 +103,7 @@ const authorFields = new Set(["actor", "operator", "via", "imported_by", "source const textDecoder = new TextDecoder("utf-8", { fatal: true }); const textEncoder = new TextEncoder(); -export type LoreUnionStatus = +export type LyncUnionStatus = | "added" | "duplicate" | "conflict" @@ -111,34 +111,34 @@ export type LoreUnionStatus = | "damaged" | "garbage"; -export interface LoreUnionIngestResult { - status: LoreUnionStatus; - line: LoreLineDiagnostic; - conflictVariants?: LoreConflictVariant[]; +export interface LyncUnionIngestResult { + status: LyncUnionStatus; + line: LyncLineDiagnostic; + conflictVariants?: LyncConflictVariant[]; missingParent?: string; - drained?: LoreUnionIngestResult[]; + drained?: LyncUnionIngestResult[]; } -export interface LoreUnionOptions { +export interface LyncUnionOptions { pendingLimit?: number; } -export class LoreUnion { +export class LyncUnion { private readonly pendingLimit: number; - private readonly lines: LoreLineDiagnostic[] = []; - private readonly acceptedById = new Map(); + private readonly lines: LyncLineDiagnostic[] = []; + private readonly acceptedById = new Map(); private readonly conflictIds = new Set(); - private readonly conflictVariants = new Map(); - private readonly pendingByParent = new Map(); + private readonly conflictVariants = new Map(); + private readonly pendingByParent = new Map(); private pendingCount = 0; private pendingOverflowCount = 0; - constructor(options: LoreUnionOptions = {}) { + constructor(options: LyncUnionOptions = {}) { this.pendingLimit = options.pendingLimit ?? 1024; } - union(input: { file: string; bytes: Uint8Array | string }): LoreUnionIngestResult[] { - const results: LoreUnionIngestResult[] = []; + union(input: { file: string; bytes: Uint8Array | string }): LyncUnionIngestResult[] { + const results: LyncUnionIngestResult[] = []; for (const raw of parsePhysicalLines(input.file, input.bytes)) { const line = parseLine(raw); this.lines.push(line); @@ -147,11 +147,11 @@ export class LoreUnion { return results; } - result(): LoreParseResult { + result(): LyncParseResult { return buildResult(this.lines, this.acceptedById, this.conflictIds, [...this.conflictVariants.values()], this.pending(), this.pendingOverflowCount); } - private ingestParsedLine(line: LoreLineDiagnostic): LoreUnionIngestResult { + private ingestParsedLine(line: LyncLineDiagnostic): LyncUnionIngestResult { if (line.class === "damaged") return { status: "damaged", line }; if (line.class === "garbage") return { status: "garbage", line }; if (!isUnionCandidate(line)) return { status: "garbage", line }; @@ -165,7 +165,7 @@ export class LoreUnion { return this.acceptLine(line); } - private acceptLine(line: LoreLineDiagnostic): LoreUnionIngestResult { + private acceptLine(line: LyncLineDiagnostic): LyncUnionIngestResult { const existing = this.acceptedById.get(line.id!); if (this.conflictIds.has(line.id!)) { return { status: "conflict", line, conflictVariants: [this.markConflictVariant(line)] }; @@ -188,7 +188,7 @@ export class LoreUnion { this.acceptedById.delete(line.id!); this.conflictIds.add(line.id!); - const variants = new Map(); + const variants = new Map(); for (const variantLine of this.lines.filter((variant) => variant.id === line.id! && isUnionCandidate(variant))) { const variant = this.markConflictVariant(variantLine); variants.set(`${variant.id}\0${variant.digest}`, variant); @@ -198,7 +198,7 @@ export class LoreUnion { return drained.length ? { status: "conflict", line, conflictVariants, drained } : { status: "conflict", line, conflictVariants }; } - private markConflictVariant(line: LoreLineDiagnostic): LoreConflictVariant { + private markConflictVariant(line: LyncLineDiagnostic): LyncConflictVariant { line.class = "conflict-variant"; line.reason = "same id with different body bytes"; const variant = conflictVariantFor(line); @@ -206,7 +206,7 @@ export class LoreUnion { return variant; } - private bufferPending(missingParent: string, line: LoreLineDiagnostic): void { + private bufferPending(missingParent: string, line: LyncLineDiagnostic): void { const bucket = this.pendingByParent.get(missingParent) ?? []; bucket.push(line); this.pendingByParent.set(missingParent, bucket); @@ -215,17 +215,17 @@ export class LoreUnion { if (this.pendingCount > this.pendingLimit) this.pendingOverflowCount++; } - private drain(parent: string): LoreUnionIngestResult[] { + private drain(parent: string): LyncUnionIngestResult[] { const bucket = this.pendingByParent.get(parent); if (!bucket) return []; this.pendingByParent.delete(parent); this.pendingCount -= bucket.length; - const results: LoreUnionIngestResult[] = []; + const results: LyncUnionIngestResult[] = []; for (const line of bucket) results.push(this.ingestParsedLine(line)); return results; } - private pending(): LorePendingDiagnostic[] { + private pending(): LyncPendingDiagnostic[] { return [...this.pendingByParent.entries()].flatMap(([missingParent, lines]) => lines.map((line) => ({ missingParent, @@ -239,15 +239,15 @@ export class LoreUnion { } } -export function parseLoreFiles( +export function parseLyncFiles( inputs: { file: string; bytes: Uint8Array | string }[], -): LoreParseResult { +): LyncParseResult { const lines = inputs.flatMap((input) => parsePhysicalLines(input.file, input.bytes).map(parseLine)); const { acceptedById, conflictIds, conflictVariants } = markConflicts(lines); return buildResult(lines, acceptedById, conflictIds, conflictVariants, [], 0); } -export function exportCarriedLoreBytes(result: LoreParseResult): Uint8Array { +export function exportCarriedLyncBytes(result: LyncParseResult): Uint8Array { const chunks: Uint8Array[] = []; let size = 0; for (const line of result.lines) { @@ -268,12 +268,12 @@ export function exportCarriedLoreBytes(result: LoreParseResult): Uint8Array { return out; } -export function loreDownset(result: LoreParseResult, id: string): { +export function lyncDownset(result: LyncParseResult, id: string): { ids: string[]; partial: boolean; - obstacles: LoreObstacle[]; + obstacles: LyncObstacle[]; } { - const events = new Map(); + const events = new Map(); const conflicts = new Set(result.conflictIds); for (const line of result.lines) { if ((line.class === "accepted" || line.class === "nonconforming") && line.id) { @@ -281,7 +281,7 @@ export function loreDownset(result: LoreParseResult, id: string): { } } const ids = new Set(); - const obstacles: LoreObstacle[] = []; + const obstacles: LyncObstacle[] = []; const stack: { id: string; path: string[] }[] = [{ id, path: [] }]; const seen = new Set(); @@ -330,7 +330,7 @@ function parsePhysicalLines(file: string, bytesOrString: Uint8Array | string) { return lines; } -function parseLine(raw: { file: string; line: number; bytes: Uint8Array; terminator: "" | "\n" }): LoreLineDiagnostic { +function parseLine(raw: { file: string; line: number; bytes: Uint8Array; terminator: "" | "\n" }): LyncLineDiagnostic { const base = { file: raw.file, line: raw.line, bytes: raw.bytes, terminator: raw.terminator } as const; const spliced = splitSplice(raw.bytes); if (spliced.digest && sha256Hex(spliced.bodyBytes) !== spliced.digest.slice("sha256:".length)) { @@ -390,12 +390,12 @@ function latin1Decode(bytes: Uint8Array): string { return text; } -function markConflicts(lines: LoreLineDiagnostic[]): { - acceptedById: Map; +function markConflicts(lines: LyncLineDiagnostic[]): { + acceptedById: Map; conflictIds: Set; - conflictVariants: LoreConflictVariant[]; + conflictVariants: LyncConflictVariant[]; } { - const byId = new Map(); + const byId = new Map(); for (const line of lines) { if (isUnionCandidate(line)) { const bucket = byId.get(line.id) ?? []; @@ -403,9 +403,9 @@ function markConflicts(lines: LoreLineDiagnostic[]): { byId.set(line.id, bucket); } } - const acceptedById = new Map(); + const acceptedById = new Map(); const conflictIds = new Set(); - const conflictVariants = new Map(); + const conflictVariants = new Map(); for (const bucket of byId.values()) { const bodies = new Set(bucket.map((line) => line.bodyDigest)); if (bodies.size > 1) { @@ -432,7 +432,7 @@ function markConflicts(lines: LoreLineDiagnostic[]): { } function validateEnvelope(value: unknown): - | { ok: true; event: LoreEventBody; nonconforming: string[] } + | { ok: true; event: LyncEventBody; nonconforming: string[] } | { ok: false; reason: string } { if (!isRecord(value)) return { ok: false, reason: "top-level JSON value is not an object" }; if ("digest" in value || "sig" in value) return { ok: false, reason: "reserved top-level digest/sig body member" }; @@ -467,17 +467,17 @@ function validateEnvelope(value: unknown): for (const key of Object.keys(value.author)) { if (!authorFields.has(key)) nonconforming.push(`unknown author field ${key}`); } - return { ok: true, event: value as LoreEventBody, nonconforming }; + return { ok: true, event: value as LyncEventBody, nonconforming }; } function buildResult( - lines: LoreLineDiagnostic[], - acceptedById: Map, + lines: LyncLineDiagnostic[], + acceptedById: Map, conflictIds: Set, - conflictVariants: LoreConflictVariant[], - pending: LorePendingDiagnostic[], + conflictVariants: LyncConflictVariant[], + pending: LyncPendingDiagnostic[], pendingOverflowCount: number, -): LoreParseResult { +): LyncParseResult { const viewEligibleIds = [...acceptedById.keys()].filter((id) => !conflictIds.has(id)).sort(); const graphDiagnostics = graphObstacles(acceptedById, conflictIds); const suppression = computeSuppression(acceptedById, conflictIds); @@ -495,26 +495,26 @@ function buildResult( }; } -function isUnionCandidate(line: LoreLineDiagnostic): line is LoreLineDiagnostic & { +function isUnionCandidate(line: LyncLineDiagnostic): line is LyncLineDiagnostic & { id: string; - event: LoreEventBody; + event: LyncEventBody; bodyBytes: Uint8Array; bodyDigest: string; } { return (line.class === "accepted" || line.class === "nonconforming") && Boolean(line.id && line.event && line.bodyBytes && line.bodyDigest); } -function sameBody(a: LoreLineDiagnostic, b: LoreLineDiagnostic): boolean { +function sameBody(a: LyncLineDiagnostic, b: LyncLineDiagnostic): boolean { return a.bodyDigest === b.bodyDigest && bytesEqual(a.bodyBytes, b.bodyBytes); } -function isRicherLine(candidate: LoreLineDiagnostic, current: LoreLineDiagnostic): boolean { +function isRicherLine(candidate: LyncLineDiagnostic, current: LyncLineDiagnostic): boolean { if (Boolean(candidate.sig) !== Boolean(current.sig)) return Boolean(candidate.sig); if (Boolean(candidate.digest) !== Boolean(current.digest)) return Boolean(candidate.digest); return false; } -function conflictVariantFor(line: LoreLineDiagnostic): LoreConflictVariant { +function conflictVariantFor(line: LyncLineDiagnostic): LyncConflictVariant { if (!line.id || !line.event || !line.bodyBytes || !line.bodyDigest) { throw new Error("conflict variant missing parsed event body"); } @@ -530,8 +530,8 @@ function conflictVariantFor(line: LoreLineDiagnostic): LoreConflictVariant { } function firstMissingParent( - line: LoreLineDiagnostic & { event: LoreEventBody }, - acceptedById: Map, + line: LyncLineDiagnostic & { event: LyncEventBody }, + acceptedById: Map, conflictIds: Set, ): string | undefined { const parent = line.event.parents[0]; @@ -541,7 +541,7 @@ function firstMissingParent( return parent; } -function computeSuppression(acceptedById: Map, conflictIds: Set) { +function computeSuppression(acceptedById: Map, conflictIds: Set) { const suppressed = new Set(); const dangling = new Set(); const eventIds = new Set([...acceptedById.keys()].filter((id) => !conflictIds.has(id))); @@ -565,8 +565,8 @@ function computeSuppression(acceptedById: Map, confl }; } -function graphObstacles(acceptedById: Map, conflictIds: Set): LoreObstacle[] { - const obstacles: LoreObstacle[] = []; +function graphObstacles(acceptedById: Map, conflictIds: Set): LyncObstacle[] { + const obstacles: LyncObstacle[] = []; for (const line of acceptedById.values()) { const event = line.event; if (!event || conflictIds.has(event.id)) continue; @@ -583,7 +583,7 @@ function graphObstacles(acceptedById: Map, conflictI return normalizeObstacles(obstacles); } -function findCycle(id: string, acceptedById: Map, conflictIds: Set): string[] { +function findCycle(id: string, acceptedById: Map, conflictIds: Set): string[] { const visit = (current: string, path: string[]): string[] => { if (path.includes(current)) return path.slice(path.indexOf(current)); if (conflictIds.has(current)) return []; @@ -598,9 +598,9 @@ function findCycle(id: string, acceptedById: Map, co return visit(id, []); } -function normalizeObstacles(obstacles: LoreObstacle[]): LoreObstacle[] { +function normalizeObstacles(obstacles: LyncObstacle[]): LyncObstacle[] { const seen = new Set(); - const out: LoreObstacle[] = []; + const out: LyncObstacle[] = []; for (const obstacle of obstacles) { const key = JSON.stringify(obstacle); if (seen.has(key)) continue; @@ -610,7 +610,7 @@ function normalizeObstacles(obstacles: LoreObstacle[]): LoreObstacle[] { return out; } -function names(event: LoreEventBody): Set { +function names(event: LyncEventBody): Set { return new Set([event.author.actor, event.author.operator, event.author.imported_by].filter((v): v is string => typeof v === "string" && v.length > 0)); } diff --git a/packages/core/src/lore/file-log.ts b/packages/core/src/file-log.ts similarity index 87% rename from packages/core/src/lore/file-log.ts rename to packages/core/src/file-log.ts index 64d9e47..6f12b1c 100644 --- a/packages/core/src/lore/file-log.ts +++ b/packages/core/src/file-log.ts @@ -1,11 +1,11 @@ import fs from "node:fs/promises"; import path from "node:path"; -import type { Looms } from "../types.js"; -import { createLoreLooms, type LoreLoomsOptions } from "./looms.js"; +import type { Looms } from "./types.js"; +import { createLyncLooms, type LyncLoomsOptions } from "./looms.js"; import { BaseEventStore, type GarbageRecord } from "./store.js"; const STORE_FILE = "events.json"; -const EVENT_FILE_EXTENSIONS = [".lync", ".lore"]; +const EVENT_FILE_EXTENSIONS = [".lync"]; export interface FileEventStoreOptions { dir: string; @@ -57,7 +57,7 @@ export class FileEventStore extends BaseEventStore { protected override async persist(): Promise { await fs.mkdir(this.options.dir, { recursive: true }); await fs.writeFile(path.join(this.options.dir, STORE_FILE), JSON.stringify(this.dumpRecords(), null, 2)); - await this.writeLoreFiles(); + await this.writeLyncFiles(); } private async load(): Promise { @@ -67,11 +67,11 @@ export class FileEventStore extends BaseEventStore { await this.loadRecords(JSON.parse(raw) as Parameters[0]); } catch (error) { if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; - await this.loadLoreFiles(); + await this.loadLyncFiles(); } } - private async loadLoreFiles(): Promise { + private async loadLyncFiles(): Promise { const files = await fs.readdir(this.options.dir); for (const file of files.filter(isEventFile).sort()) { const raw = await fs.readFile(path.join(this.options.dir, file), "utf8"); @@ -81,7 +81,7 @@ export class FileEventStore extends BaseEventStore { } } - private async writeLoreFiles(): Promise { + private async writeLyncFiles(): Promise { const records = this.dumpRecords(); const roots = new Set(records.events.map((event) => event.root)); for (const root of roots) { @@ -104,12 +104,12 @@ export function createFileEventStore(dir: string): FileEventStore { // File-backed looms live here, not in looms.ts, so the browser-reachable // modules never statically import this node:fs/node:path file. -export function createFileLoreLooms< +export function createFileLyncLooms< TPayload = unknown, TLoomMeta = unknown, TTurnMeta = unknown, ->(dir: string, options: Omit): Looms { - return createLoreLooms({ ...options, store: createFileEventStore(dir) }); +>(dir: string, options: Omit): Looms { + return createLyncLooms({ ...options, store: createFileEventStore(dir) }); } function isEventFile(file: string): boolean { diff --git a/packages/core/src/lore/idb-log.ts b/packages/core/src/idb-log.ts similarity index 98% rename from packages/core/src/lore/idb-log.ts rename to packages/core/src/idb-log.ts index eb27c2c..6879a6e 100644 --- a/packages/core/src/lore/idb-log.ts +++ b/packages/core/src/idb-log.ts @@ -23,7 +23,7 @@ export class IndexedDbEventStore extends BaseEventStore { constructor(options: IndexedDbEventStoreOptions = {}) { super(); - this.dbName = options.dbName ?? "lync-lore"; + this.dbName = options.dbName ?? "lync"; this.idb = options.indexedDB ?? indexedDB; this.ready = this.load(); } diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 9a4626b..c11208b 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -1,10 +1,10 @@ export * from "./errors.js"; -// The node:fs-backed file store lives only at the explicit "lync-core/lore/file-log" +// The node:fs-backed file store lives only at the explicit "lync-core/file-log" // subpath so the main barrel stays importable in the browser with zero node builtins. -export * from "./lore/idb-log.js"; -export * from "./lore/looms.js"; -export * from "./lore/memory-log.js"; -export * from "./lore/store.js"; -export * from "./lore/views.js"; +export * from "./idb-log.js"; +export * from "./looms.js"; +export * from "./memory-log.js"; +export * from "./store.js"; +export * from "./views.js"; export * from "./references.js"; export * from "./types.js"; diff --git a/packages/core/src/lore/looms.ts b/packages/core/src/looms.ts similarity index 91% rename from packages/core/src/lore/looms.ts rename to packages/core/src/looms.ts index 797e462..24abbb3 100644 --- a/packages/core/src/lore/looms.ts +++ b/packages/core/src/looms.ts @@ -6,8 +6,8 @@ import { invalidSnapshot, missingParent, unknownLoom, -} from "../errors.js"; -import { assertJsonEncodable, cloneJson } from "../json.js"; +} from "./errors.js"; +import { assertJsonEncodable, cloneJson } from "./json.js"; import type { Loom, LoomEvent, @@ -18,14 +18,14 @@ import type { LoomSnapshot, Turn, TurnId, -} from "../types.js"; -import type { LoreEventBody } from "./events.js"; +} from "./types.js"; +import type { LyncEventBody } from "./events.js"; import { createIndexedDbEventStore, type IndexedDbEventStoreOptions } from "./idb-log.js"; import type { EventStore, StoredEvent } from "./store.js"; -const LORE_PREFIX = "lore:"; +const LYNC_PREFIX = "lync:"; -export interface LoreAuthor { +export interface LyncAuthor { actor: string; operator?: string; via?: string; @@ -33,9 +33,9 @@ export interface LoreAuthor { source?: string; } -export interface LoreLoomsOptions { +export interface LyncLoomsOptions { store: EventStore; - author: LoreAuthor; + author: LyncAuthor; now?: () => number; createId?: () => string; } @@ -46,11 +46,11 @@ interface Fold { children: Map; } -export function createLoreLooms< +export function createLyncLooms< TPayload = unknown, TLoomMeta = unknown, TTurnMeta = unknown, ->(options: LoreLoomsOptions): Looms { +>(options: LyncLoomsOptions): Looms { validateAuthor(options.author); const now = options.now ?? (() => Date.now()); const createId = options.createId ?? createUuidLike; @@ -60,9 +60,9 @@ export function createLoreLooms< parents: string[], payload: Record, atMs = now(), - author: LoreAuthor = options.author, + author: LyncAuthor = options.author, marked?: string, - ): LoreEventBody => ({ + ): LyncEventBody => ({ v: 1, id: createId(), kind, @@ -79,7 +79,7 @@ export function createLoreLooms< const event = mint("lync/loom", [], omitUndefined({ meta: cloneJson(meta) })); const result = await options.store.append(event); if (result.status !== "added" && result.status !== "duplicate") { - throw new Error(`Unable to create lore loom: ${result.status}`); + throw new Error(`Unable to create lync loom: ${result.status}`); } return eventToLoomInfo(result.event.body); }, @@ -97,7 +97,7 @@ export function createLoreLooms< if (!root) throw unknownLoom(loomId); const event = await options.store.byId(root); if (!event || event.body.kind !== "lync/loom") throw unknownLoom(loomId); - return new LoreLoom(loomId, root, options.store, mint); + return new LyncLoom(loomId, root, options.store, mint); }, async import(snapshot) { @@ -113,7 +113,7 @@ export function createLoreLooms< importMarked, ); await options.store.append(loomEvent); - const newLoomId = `${LORE_PREFIX}${loomEvent.id}`; + const newLoomId = `${LYNC_PREFIX}${loomEvent.id}`; const idMap = new Map(); const ordered = topological(snapshot.turns); const siblingOrdinal = new Map(); @@ -143,16 +143,16 @@ export function createLoreLooms< }; } -export function createBrowserLoreLooms< +export function createBrowserLyncLooms< TPayload = unknown, TLoomMeta = unknown, TTurnMeta = unknown, ->(idbOptions: IndexedDbEventStoreOptions & Omit): Looms { +>(idbOptions: IndexedDbEventStoreOptions & Omit): Looms { const { author, now, createId, ...storeOptions } = idbOptions; - return createLoreLooms({ author, now, createId, store: createIndexedDbEventStore(storeOptions) }); + return createLyncLooms({ author, now, createId, store: createIndexedDbEventStore(storeOptions) }); } -class LoreLoom +class LyncLoom implements Loom { private closed = false; @@ -168,7 +168,7 @@ class LoreLoom parents: string[], payload: Record, atMs?: number, - ) => LoreEventBody, + ) => LyncEventBody, ) { this.unsubscribe = store.subscribe(root, async (event) => { if (this.closed) return; @@ -314,7 +314,7 @@ function foldLoom( events: StoredEvent[], loomId: LoomId, ): Fold { - const root = events.find((event) => event.body.kind === "lync/loom" && `${LORE_PREFIX}${event.body.id}` === loomId); + const root = events.find((event) => event.body.kind === "lync/loom" && `${LYNC_PREFIX}${event.body.id}` === loomId); if (!root) throw unknownLoom(loomId); let loom = eventToLoomInfo(root.body); const metaEvents = events.filter((event) => event.body.kind === "lync/loom-meta").sort(compareNewest); @@ -345,9 +345,9 @@ function foldLoom( return { loom, turns, children }; } -function eventToLoomInfo(event: LoreEventBody): LoomInfo { +function eventToLoomInfo(event: LyncEventBody): LoomInfo { return omitUndefined({ - id: `${LORE_PREFIX}${event.id}`, + id: `${LYNC_PREFIX}${event.id}`, meta: cloneJson(event.payload.meta as TMeta), createdAt: Date.parse(event.at), }); @@ -371,16 +371,16 @@ function mustTurn(fold: Fold, } function rootId(id: LoomId): string | null { - return id.startsWith(LORE_PREFIX) ? id.slice(LORE_PREFIX.length) : null; + return id.startsWith(LYNC_PREFIX) ? id.slice(LYNC_PREFIX.length) : null; } -function validateAuthor(author: LoreAuthor): void { +function validateAuthor(author: LyncAuthor): void { if (!author || typeof author.actor !== "string" || author.actor.length === 0) { - throw new Error("Lore author.actor is required"); + throw new Error("Lync author.actor is required"); } } -function compactAuthor(author: LoreAuthor): LoreEventBody["author"] { +function compactAuthor(author: LyncAuthor): LyncEventBody["author"] { return omitUndefined({ actor: author.actor, operator: author.operator || undefined, diff --git a/packages/core/src/lore/memory-log.ts b/packages/core/src/memory-log.ts similarity index 100% rename from packages/core/src/lore/memory-log.ts rename to packages/core/src/memory-log.ts diff --git a/packages/core/src/lore/sha256.ts b/packages/core/src/sha256.ts similarity index 98% rename from packages/core/src/lore/sha256.ts rename to packages/core/src/sha256.ts index 6ce3969..5bf790b 100644 --- a/packages/core/src/lore/sha256.ts +++ b/packages/core/src/sha256.ts @@ -1,5 +1,5 @@ // SHA-256 in pure TypeScript: synchronous, dependency-free, and byte-for-byte -// identical in Node and the browser. Lore hashing runs on the same code path +// identical in Node and the browser. Lync hashing runs on the same code path // everywhere, so this file never reaches for node:crypto and the browser // bundle stays free of node builtins. WebCrypto's subtle.digest is async and // cannot back the synchronous line parser, which is why we implement it here. diff --git a/packages/core/src/lore/store.ts b/packages/core/src/store.ts similarity index 95% rename from packages/core/src/lore/store.ts rename to packages/core/src/store.ts index ce873e1..81e44ba 100644 --- a/packages/core/src/lore/store.ts +++ b/packages/core/src/store.ts @@ -1,9 +1,9 @@ -import type { LoreEventBody } from "./events.js"; -import { parseLoreFiles } from "./events.js"; +import type { LyncEventBody } from "./events.js"; +import { parseLyncFiles } from "./events.js"; import { sha256Hex } from "./sha256.js"; export interface StoredEvent { - body: LoreEventBody; + body: LyncEventBody; bytes: string; root: string; } @@ -16,7 +16,7 @@ export type AppendResult = | { status: "garbage"; reason: string; bytes: string }; export interface EventStore { - append(ev: LoreEventBody): Promise; + append(ev: LyncEventBody): Promise; union(line: string): Promise; byId(id: string): Promise; byRoot(rootId: string): Promise; @@ -66,8 +66,8 @@ export abstract class BaseEventStore implements EventStore { protected readonly garbage: GarbageRecord[] = []; private readonly listeners = new Map void>>(); - async append(ev: LoreEventBody): Promise { - return this.ingest(serializeLoreEvent(ev), false); + async append(ev: LyncEventBody): Promise { + return this.ingest(serializeLyncEvent(ev), false); } async union(line: string): Promise { @@ -243,7 +243,7 @@ export abstract class BaseEventStore implements EventStore { } } -export function serializeLoreEvent(ev: LoreEventBody): string { +export function serializeLyncEvent(ev: LyncEventBody): string { const fields: [string, unknown][] = [ ["v", ev.v], ["id", ev.id], @@ -262,10 +262,10 @@ function parseStoredLine(line: string): | { ok: true; event: StoredEvent; bodyBytes: Uint8Array } | { ok: false; reason: string } { const normalized = line.endsWith("\n") ? line.slice(0, -1) : line; - const parsed = parseLoreFiles([{ file: "", bytes: `${normalized}\n` }]); + const parsed = parseLyncFiles([{ file: "", bytes: `${normalized}\n` }]); const diagnostic = parsed.lines[0]; if (!diagnostic?.event || (diagnostic.class !== "accepted" && diagnostic.class !== "nonconforming")) { - return { ok: false, reason: diagnostic?.reason ?? "unparseable lore line" }; + return { ok: false, reason: diagnostic?.reason ?? "unparseable lync line" }; } return { ok: true, @@ -274,7 +274,7 @@ function parseStoredLine(line: string): }; } -function validateKnownLyncEvent(body: LoreEventBody): string | null { +function validateKnownLyncEvent(body: LyncEventBody): string | null { if (!body.kind.startsWith("lync/")) return null; if (body.kind === "lync/loom" || body.kind === "lync/index") { return body.parents.length === 0 ? null : `${body.kind} must not have parents`; diff --git a/packages/core/src/lore/views.ts b/packages/core/src/views.ts similarity index 78% rename from packages/core/src/lore/views.ts rename to packages/core/src/views.ts index ba3291c..648cdd0 100644 --- a/packages/core/src/lore/views.ts +++ b/packages/core/src/views.ts @@ -1,87 +1,87 @@ -import { loreDownset, type LoreEventBody, type LoreLineDiagnostic, type LoreObstacle, type LoreParseResult } from "./events.js"; +import { lyncDownset, type LyncEventBody, type LyncLineDiagnostic, type LyncObstacle, type LyncParseResult } from "./events.js"; -export interface LoreViewEvent { +export interface LyncViewEvent { id: string; - event: LoreEventBody; - line: LoreLineDiagnostic; + event: LyncEventBody; + line: LyncLineDiagnostic; payloadSuppressed: boolean; } -export interface LoreBranchTreeNode extends LoreViewEvent { +export interface LyncBranchTreeNode extends LyncViewEvent { parents: string[]; children: string[]; missingParents: string[]; conflictedParents: string[]; } -export interface LoreBranchTreeView { - nodes: LoreBranchTreeNode[]; +export interface LyncBranchTreeView { + nodes: LyncBranchTreeNode[]; roots: string[]; leaves: string[]; - obstacles: LoreObstacle[]; + obstacles: LyncObstacle[]; partial: boolean; } -export interface LoreTranscriptEntry extends LoreViewEvent { +export interface LyncTranscriptEntry extends LyncViewEvent { depth: number; } -export interface LoreTranscriptView { +export interface LyncTranscriptView { head: string; - entries: LoreTranscriptEntry[]; + entries: LyncTranscriptEntry[]; downsetIds: string[]; - obstacles: LoreObstacle[]; + obstacles: LyncObstacle[]; partial: boolean; } -export interface LoreMemoryView { - events: LoreViewEvent[]; +export interface LyncMemoryView { + events: LyncViewEvent[]; frontierIds: string[]; suppressedPayloadIds: string[]; conflictIds: string[]; - obstacles: LoreObstacle[]; + obstacles: LyncObstacle[]; partial: boolean; } -export interface LoreScoreReference { +export interface LyncScoreReference { annotationId: string; value: number; - author: LoreEventBody["author"]; + author: LyncEventBody["author"]; at: string; basis?: unknown; } -export interface LoreSelectionReference { +export interface LyncSelectionReference { annotationId: string; selected: boolean; - author: LoreEventBody["author"]; + author: LyncEventBody["author"]; at: string; basis?: unknown; } -export interface LoreLeaderboardEntry { +export interface LyncLeaderboardEntry { targetId: string; - event?: LoreViewEvent; + event?: LyncViewEvent; scoreTotal: number; scoreCount: number; scoreMean: number | null; selectedCount: number; selectionCount: number; - scores: LoreScoreReference[]; - selections: LoreSelectionReference[]; + scores: LyncScoreReference[]; + selections: LyncSelectionReference[]; rank: number; } -export interface LoreLeaderboardView { - entries: LoreLeaderboardEntry[]; +export interface LyncLeaderboardView { + entries: LyncLeaderboardEntry[]; ignoredAnnotationIds: string[]; } -export interface LoreTranscriptOptions { - chooseParent?: (event: LoreEventBody, candidates: string[]) => string | undefined; +export interface LyncTranscriptOptions { + chooseParent?: (event: LyncEventBody, candidates: string[]) => string | undefined; } -export function loreBranchTreeView(result: LoreParseResult): LoreBranchTreeView { +export function lyncBranchTreeView(result: LyncParseResult): LyncBranchTreeView { const index = eligibleEventIndex(result); const children = new Map(); const parentRefs = new Set(); @@ -118,14 +118,14 @@ export function loreBranchTreeView(result: LoreParseResult): LoreBranchTreeView }; } -export function loreTranscriptView( - result: LoreParseResult, +export function lyncTranscriptView( + result: LyncParseResult, head: string, - options: LoreTranscriptOptions = {}, -): LoreTranscriptView { + options: LyncTranscriptOptions = {}, +): LyncTranscriptView { const index = eligibleEventIndex(result); - const downset = loreDownset(result, head); - const path: LoreViewEvent[] = []; + const downset = lyncDownset(result, head); + const path: LyncViewEvent[] = []; const seen = new Set(); let current: string | undefined = head; @@ -152,8 +152,8 @@ export function loreTranscriptView( }; } -export function loreMemoryView(result: LoreParseResult): LoreMemoryView { - const tree = loreBranchTreeView(result); +export function lyncMemoryView(result: LyncParseResult): LyncMemoryView { + const tree = lyncBranchTreeView(result); const index = eligibleEventIndex(result); return { @@ -166,9 +166,9 @@ export function loreMemoryView(result: LoreParseResult): LoreMemoryView { }; } -export function loreLeaderboardView(result: LoreParseResult): LoreLeaderboardView { +export function lyncLeaderboardView(result: LyncParseResult): LyncLeaderboardView { const index = eligibleEventIndex(result); - const entries = new Map>(); + const entries = new Map>(); const ignoredAnnotationIds: string[] = []; const ensure = (targetId: string) => { @@ -192,7 +192,7 @@ export function loreLeaderboardView(result: LoreParseResult): LoreLeaderboardVie for (const viewEvent of index.events.values()) { const event = viewEvent.event; - if (event.kind !== "lore/annotation" || viewEvent.payloadSuppressed) continue; + if (event.kind !== "lync/annotation" || viewEvent.payloadSuppressed) continue; const label = event.payload["label"]; if (label === "score") { const value = numericScore(event.payload); @@ -232,10 +232,10 @@ export function loreLeaderboardView(result: LoreParseResult): LoreLeaderboardVie }; } -function eligibleEventIndex(result: LoreParseResult) { +function eligibleEventIndex(result: LyncParseResult) { const eligibleIds = new Set(result.viewEligibleIds); const suppressedPayloadIds = new Set(result.suppression.suppressedPayloadIds); - const events = new Map(); + const events = new Map(); for (const line of result.lines) { if (!line.id || !line.event || !eligibleIds.has(line.id) || events.has(line.id)) continue; @@ -264,8 +264,8 @@ function stringSet(value: unknown): Set { } function compareLeaderboardEntries( - a: Omit, - b: Omit, + a: Omit, + b: Omit, ): number { return ( b.selectedCount - a.selectedCount || diff --git a/packages/core/test/automerge-browser.test.ts b/packages/core/test/automerge-browser.test.ts deleted file mode 100644 index b70a4cf..0000000 --- a/packages/core/test/automerge-browser.test.ts +++ /dev/null @@ -1,70 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { - createBrowserAutomergeRepoConfig, - defaultWebSocketUrl, -} from "../src/browser.js"; - -class FakeStorage { - constructor( - readonly database?: string, - readonly store?: string, - ) {} -} - -class FakeBroadcast { - constructor(readonly options?: { channelName: string; peerWaitMs?: number }) {} -} - -class FakeWebSocket { - constructor( - readonly url: string, - readonly retryInterval?: number, - ) {} -} - -describe("browser Automerge repo factory", () => { - it("assembles IndexedDB, BroadcastChannel, and WebSocket adapters", async () => { - const config = createBrowserAutomergeRepoConfig({ - indexedDb: { database: "loom-test", store: "docs" }, - broadcastChannel: { channelName: "loom-test-channel", peerWaitMs: 5 }, - websocket: { url: "wss://sync.example", retryInterval: 50 }, - adapters: { - IndexedDBStorageAdapter: FakeStorage, - BroadcastChannelNetworkAdapter: FakeBroadcast, - WebSocketClientAdapter: FakeWebSocket, - }, - }); - - expect(config.storage).toBeInstanceOf(FakeStorage); - expect((config.storage as FakeStorage).database).toBe("loom-test"); - expect(config.network).toHaveLength(2); - expect(config.network[0]).toBeInstanceOf(FakeBroadcast); - expect(config.network[1]).toBeInstanceOf(FakeWebSocket); - }); - - it("uses a same-origin WebSocket adapter by default when a location is supplied", () => { - const config = createBrowserAutomergeRepoConfig({ - location: { protocol: "https:", host: "loom.test" }, - adapters: { - IndexedDBStorageAdapter: FakeStorage, - BroadcastChannelNetworkAdapter: FakeBroadcast, - WebSocketClientAdapter: FakeWebSocket, - }, - }); - - expect(config.network).toHaveLength(2); - expect(config.network[1]).toBeInstanceOf(FakeWebSocket); - expect((config.network[1] as FakeWebSocket).url).toBe( - "wss://loom.test/lync", - ); - }); - - it("derives default WebSocket URLs from locations", () => { - expect( - defaultWebSocketUrl({ - location: { protocol: "http:", host: "localhost:5173" }, - path: "sync", - }), - ).toBe("ws://localhost:5173/sync"); - }); -}); diff --git a/packages/core/test/automerge.test.ts b/packages/core/test/automerge.test.ts deleted file mode 100644 index 03a483c..0000000 --- a/packages/core/test/automerge.test.ts +++ /dev/null @@ -1,127 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { Repo } from "@automerge/automerge-repo"; -import { createAutomergeLooms } from "../src/automerge.js"; -import type { LoomSnapshot } from "../src/types.js"; - -type Payload = { text: string }; -type LoomMeta = { title: string }; - -function deterministicAutomergeLooms() { - let nextId = 0; - let nextTime = 3000; - return createAutomergeLooms({ - repo: new Repo(), - createTurnId: () => `turn-${++nextId}`, - now: () => nextTime++, - }); -} - -describe("automerge looms", () => { - it("uses the Automerge document URL as the loom id", async () => { - const looms = deterministicAutomergeLooms(); - const info = await looms.create({ title: "Story 1" }); - - expect(info.id.startsWith("automerge:")).toBe(true); - await expect(looms.open(info.id)).resolves.toMatchObject({ id: info.id }); - }); - - it("appends turns and preserves canonical child-list order", async () => { - const looms = deterministicAutomergeLooms(); - const info = await looms.create({ title: "Story 1" }); - const loom = await looms.open(info.id); - - const first = await loom.appendTurn(null, { text: "Once" }); - const left = await loom.appendTurn(first.id, { text: " left" }); - const right = await loom.appendTurn(first.id, { text: " right" }); - - expect(await loom.childrenOf(first.id)).toEqual([left, right]); - expect(await loom.threadTo(right.id)).toEqual([first, right]); - expect(await loom.leaves()).toEqual([left, right]); - }); - - it("imports a snapshot with preserved turn ids and a new loom id", async () => { - const looms = deterministicAutomergeLooms(); - const snapshot: LoomSnapshot = { - loom: { id: "snapshot:story", meta: { title: "Imported" }, createdAt: 10 }, - turns: [ - { - id: "a", - loomId: "snapshot:story", - parentId: null, - payload: { text: "A" }, - createdAt: 11, - }, - { - id: "b", - loomId: "snapshot:story", - parentId: "a", - payload: { text: "B" }, - createdAt: 12, - }, - ], - }; - - const info = await looms.import(snapshot); - const loom = await looms.open(info.id); - const exported = await loom.export(); - - expect(info.id).not.toBe(snapshot.loom.id); - expect(exported.turns.map((turn) => turn.id)).toEqual(["a", "b"]); - expect(exported.turns.every((turn) => turn.loomId === info.id)).toBe(true); - }); - - it("emits turn-added when another handle observes the same document change", async () => { - const looms = deterministicAutomergeLooms(); - const info = await looms.create({ title: "Story 1" }); - const observer = await looms.open(info.id); - const writer = await looms.open(info.id); - const events: string[] = []; - observer.subscribe((event) => { - if (event.type === "turn-added") events.push(event.turn.id); - }); - - const first = await writer.appendTurn(null, { text: "Remote-ish" }); - - expect(events).toEqual([first.id]); - }); - - it("rejects invalid imported topologies", async () => { - const looms = deterministicAutomergeLooms(); - await expect( - looms.import({ - loom: { id: "snapshot:story", meta: { title: "Bad" }, createdAt: 10 }, - turns: [ - { - id: "a", - loomId: "snapshot:story", - parentId: "missing", - payload: { text: "A" }, - createdAt: 11, - }, - ], - }), - ).rejects.toMatchObject({ code: "MISSING_PARENT" }); - - await expect( - looms.import({ - loom: { id: "snapshot:story", meta: { title: "Bad" }, createdAt: 10 }, - turns: [ - { - id: "a", - loomId: "snapshot:story", - parentId: "b", - payload: { text: "A" }, - createdAt: 11, - }, - { - id: "b", - loomId: "snapshot:story", - parentId: "a", - payload: { text: "B" }, - createdAt: 12, - }, - ], - }), - ).rejects.toMatchObject({ code: "CYCLE_DETECTED" }); - }); -}); diff --git a/packages/core/test/lore-events.test.ts b/packages/core/test/events.test.ts similarity index 85% rename from packages/core/test/lore-events.test.ts rename to packages/core/test/events.test.ts index 7ec7081..1245ca1 100644 --- a/packages/core/test/lore-events.test.ts +++ b/packages/core/test/events.test.ts @@ -4,14 +4,14 @@ import { dirname, basename, join } from "node:path"; import { fileURLToPath } from "node:url"; import { describe, expect, it } from "vitest"; import { - exportCarriedLoreBytes, - LoreUnion, - loreDownset, - parseLoreFiles, - type LoreLineDiagnostic, -} from "../src/lore/events.js"; + exportCarriedLyncBytes, + LyncUnion, + lyncDownset, + parseLyncFiles, + type LyncLineDiagnostic, +} from "../src/events.js"; -const vectorsRoot = join(dirname(fileURLToPath(import.meta.url)), "vectors", "lore-vectors-draft"); +const vectorsRoot = join(dirname(fileURLToPath(import.meta.url)), "vectors", "v0"); interface ExpectedLine { file: string; @@ -49,7 +49,7 @@ interface ExpectedFixture { function loadFixture(name: string) { const dir = join(vectorsRoot, name); const expected = JSON.parse(readFileSync(join(dir, "expected.json"), "utf8")) as ExpectedFixture; - const result = parseLoreFiles( + const result = parseLyncFiles( expected.inputs.map((file) => ({ file, bytes: readFileSync(join(dir, file)), @@ -83,7 +83,7 @@ function digestFor(bytes: Uint8Array) { return `sha256:${createHash("sha256").update(bytes).digest("hex")}`; } -function simplifyLine(line: LoreLineDiagnostic): ExpectedLine { +function simplifyLine(line: LyncLineDiagnostic): ExpectedLine { return { file: line.file, line: line.line, @@ -96,7 +96,7 @@ function simplifyLine(line: LoreLineDiagnostic): ExpectedLine { }; } -function lineForExpected(line: LoreLineDiagnostic, expected: ExpectedLine): ExpectedLine { +function lineForExpected(line: LyncLineDiagnostic, expected: ExpectedLine): ExpectedLine { const simplified = simplifyLine(line); return { file: simplified.file, @@ -110,7 +110,7 @@ function lineForExpected(line: LoreLineDiagnostic, expected: ExpectedLine): Expe }; } -describe("LORE-V0 line parser vectors", () => { +describe("lync v0 line parser vectors", () => { for (const name of [ "01-valid-events", "02-splice-anchoring", @@ -156,7 +156,7 @@ describe("LORE-V0 line parser vectors", () => { } for (const [viewName, view] of Object.entries(expected.views ?? {})) { const id = viewName.slice("downset:".length); - const actual = loreDownset(result, id); + const actual = lyncDownset(result, id); expect(actual.ids).toEqual(view.ids); expect(actual.partial).toBe(view.partial); expect(actual.obstacles).toEqual(view.obstacles); @@ -166,12 +166,12 @@ describe("LORE-V0 line parser vectors", () => { it("round-trips garbage and damaged lines through carried export", () => { const { dir, expected, result } = loadFixture("03-damaged-digest"); - const exported = exportCarriedLoreBytes(result); + const exported = exportCarriedLyncBytes(result); const original = Buffer.concat(expected.inputs.map((file) => readFileSync(join(dir, file)))); expect(Buffer.from(exported).equals(original)).toBe(true); const garbage = loadFixture("04-garbage-classes"); - const garbageExported = exportCarriedLoreBytes(garbage.result); + const garbageExported = exportCarriedLyncBytes(garbage.result); const garbageOriginal = Buffer.concat( garbage.expected.inputs.map((file) => readFileSync(join(garbage.dir, file))), ); @@ -183,9 +183,9 @@ describe("LORE-V0 line parser vectors", () => { try { // @ts-expect-error exercises the browser bundle path under a Node test runner. delete globalThis.Buffer; - const body = eventBody({ id: "browser-no-buffer", kind: "lore/artifact" }); + const body = eventBody({ id: "browser-no-buffer", kind: "lync/artifact" }); const line = `${body.slice(0, -1)},"digest":"${digestFor(new TextEncoder().encode(body))}"}\n`; - const result = parseLoreFiles([{ file: "browser.lore", bytes: new TextEncoder().encode(line) }]); + const result = parseLyncFiles([{ file: "browser.lync", bytes: new TextEncoder().encode(line) }]); expect(result.lines[0]?.class).toBe("accepted"); expect(result.lines[0]?.id).toBe("browser-no-buffer"); @@ -216,26 +216,26 @@ describe("LORE-V0 line parser vectors", () => { it("keeps source filenames in diagnostics", () => { const { result } = loadFixture("10-merge-union"); expect(result.lines.map((line) => basename(line.file))).toEqual([ - "a.lore", - "a.lore", - "a.lore", - "b.lore", - "b.lore", - "b.lore", + "a.lync", + "a.lync", + "a.lync", + "b.lync", + "b.lync", + "b.lync", ]); expect(result.conflictVariants).toHaveLength(2); expect(result.conflictVariants.map((variant) => `${variant.id}:${variant.digest}`)).toEqual([ - "018f0000-0000-7000-8000-000000000094:608a33dee8afdf84ce8ac2fa7306d494c18690f122c69236a7865ab1014a227b", - "018f0000-0000-7000-8000-000000000094:f55b52407b5226b217dc43ab1aa3a513b05ba567ca4bfc7999e6bc9a27ce438d", + "018f0000-0000-7000-8000-000000000094:803e3ebd8093762e392c7a3e89dd3c2213f14346945e7a1f7ffd649785a6b44c", + "018f0000-0000-7000-8000-000000000094:af8ea1492ec51753fff3a07ef5c7ae79a9072bed3485292f9ac3e9b2c6c7ca21", ]); }); it("surfaces same-id different body variants and excludes them from normal views", () => { const first = `${eventBody({ id: "same", payload: { text: "first" } })}\n`; const second = `${eventBody({ id: "same", payload: { text: "second" } })}\n`; - const result = parseLoreFiles([ - { file: "a.lore", bytes: first }, - { file: "b.lore", bytes: second }, + const result = parseLyncFiles([ + { file: "a.lync", bytes: first }, + { file: "b.lync", bytes: second }, ]); expect(result.lines.map((line) => line.class)).toEqual(["conflict-variant", "conflict-variant"]); @@ -249,19 +249,19 @@ describe("LORE-V0 line parser vectors", () => { }); it("buffers missing first-parent arrivals and drains pending children in cascade", () => { - const union = new LoreUnion({ pendingLimit: 1 }); + const union = new LyncUnion({ pendingLimit: 1 }); const grandchild = `${eventBody({ id: "grandchild", parents: ["child"] })}\n`; const child = `${eventBody({ id: "child", parents: ["root"] })}\n`; const root = `${eventBody({ id: "root" })}\n`; - const first = union.union({ file: "grandchild.lore", bytes: grandchild }); - const second = union.union({ file: "child.lore", bytes: child }); + const first = union.union({ file: "grandchild.lync", bytes: grandchild }); + const second = union.union({ file: "child.lync", bytes: child }); expect(first[0]?.status).toBe("buffered"); expect(second[0]?.status).toBe("buffered"); expect(union.result().pending.map((line) => line.id).sort()).toEqual(["child", "grandchild"]); expect(union.result().pendingOverflowCount).toBe(1); - const third = union.union({ file: "root.lore", bytes: root }); + const third = union.union({ file: "root.lync", bytes: root }); expect(third[0]?.status).toBe("added"); expect(third[0]?.drained?.map((result) => result.status)).toEqual(["added"]); expect(third[0]?.drained?.[0]?.drained?.map((result) => result.status)).toEqual(["added"]); @@ -269,7 +269,7 @@ describe("LORE-V0 line parser vectors", () => { const result = union.result(); expect(result.pending).toEqual([]); expect(result.unionEventIds).toEqual(["child", "grandchild", "root"]); - expect(loreDownset(result, "grandchild")).toEqual({ + expect(lyncDownset(result, "grandchild")).toEqual({ ids: ["child", "grandchild", "root"], partial: false, obstacles: [], @@ -293,8 +293,8 @@ describe("LORE-V0 line parser vectors", () => { ] as const; const snapshots = orderings.slice(0, 3).map((ordering) => { - const union = new LoreUnion(); - for (const name of ordering) union.union({ file: `${name}.lore`, bytes: lines[name] }); + const union = new LyncUnion(); + for (const name of ordering) union.union({ file: `${name}.lync`, bytes: lines[name] }); const result = union.result(); return { unionEventIds: result.unionEventIds, @@ -317,14 +317,14 @@ describe("LORE-V0 line parser vectors", () => { acceptedLineIds: ["child", "grandchild", "independent"], }); - const conflictThenChildUnion = new LoreUnion(); - for (const name of orderings[3]) conflictThenChildUnion.union({ file: `${name}.lore`, bytes: lines[name] }); + const conflictThenChildUnion = new LyncUnion(); + for (const name of orderings[3]) conflictThenChildUnion.union({ file: `${name}.lync`, bytes: lines[name] }); const conflictThenChild = conflictThenChildUnion.result(); expect(conflictThenChild.conflictIds).toEqual(["conflicted-parent"]); expect(conflictThenChild.unionEventIds).toEqual(["conflict-child", "independent"]); expect(conflictThenChild.pending).toEqual([ - { id: "grandchild", missingParent: "child", digest: expect.any(String), file: "grandchild.lore", line: 1, bytes: expect.any(Uint8Array) }, - { id: "child", missingParent: "missing-root", digest: expect.any(String), file: "child.lore", line: 1, bytes: expect.any(Uint8Array) }, + { id: "grandchild", missingParent: "child", digest: expect.any(String), file: "grandchild.lync", line: 1, bytes: expect.any(Uint8Array) }, + { id: "child", missingParent: "missing-root", digest: expect.any(String), file: "child.lync", line: 1, bytes: expect.any(Uint8Array) }, ]); expect(conflictThenChild.graphDiagnostics).toEqual([ { class: "unavailable-due-to-conflict", id: "conflicted-parent" }, @@ -337,18 +337,18 @@ describe("LORE-V0 line parser vectors", () => { a2: `${eventBody({ id: "A", payload: { value: 2 } })}\n`, b: `${eventBody({ id: "B", parents: ["A"] })}\n`, }; - const batch = parseLoreFiles([ - { file: "a1.lore", bytes: lines.a1 }, - { file: "b.lore", bytes: lines.b }, - { file: "a2.lore", bytes: lines.a2 }, + const batch = parseLyncFiles([ + { file: "a1.lync", bytes: lines.a1 }, + { file: "b.lync", bytes: lines.b }, + { file: "a2.lync", bytes: lines.a2 }, ]); const snapshots = [ ["a1", "b", "a2"], ["a1", "a2", "b"], ].map((ordering) => { - const union = new LoreUnion(); - for (const name of ordering) union.union({ file: `${name}.lore`, bytes: lines[name as keyof typeof lines] }); + const union = new LyncUnion(); + for (const name of ordering) union.union({ file: `${name}.lync`, bytes: lines[name as keyof typeof lines] }); const result = union.result(); return { unionEventIds: result.unionEventIds, @@ -377,14 +377,14 @@ describe("LORE-V0 line parser vectors", () => { }); it("records every streaming same-id different-body conflict variant after the first conflict", () => { - const union = new LoreUnion(); + const union = new LyncUnion(); const first = `${eventBody({ id: "same", payload: { text: "first" } })}\n`; const second = `${eventBody({ id: "same", payload: { text: "second" } })}\n`; const third = `${eventBody({ id: "same", payload: { text: "third" } })}\n`; - expect(union.union({ file: "a.lore", bytes: first })[0]?.status).toBe("added"); - expect(union.union({ file: "b.lore", bytes: second })[0]?.status).toBe("conflict"); - expect(union.union({ file: "c.lore", bytes: third })[0]?.status).toBe("conflict"); + expect(union.union({ file: "a.lync", bytes: first })[0]?.status).toBe("added"); + expect(union.union({ file: "b.lync", bytes: second })[0]?.status).toBe("conflict"); + expect(union.union({ file: "c.lync", bytes: third })[0]?.status).toBe("conflict"); const result = union.result(); expect(result.lines.map((line) => line.class)).toEqual([ @@ -411,18 +411,18 @@ describe("LORE-V0 line parser vectors", () => { ["t1", "t3", "t1dup"], ["t3", "t1", "t1dup"], ] as const; - const batch = parseLoreFiles([ - { file: "t1.lore", bytes: lines.t1 }, - { file: "t1dup.lore", bytes: lines.t1dup }, - { file: "t3.lore", bytes: lines.t3 }, + const batch = parseLyncFiles([ + { file: "t1.lync", bytes: lines.t1 }, + { file: "t1dup.lync", bytes: lines.t1dup }, + { file: "t3.lync", bytes: lines.t3 }, ]); const batchClasses = batch.lines.map((line) => line.class); expect(batchClasses).toEqual(["conflict-variant", "conflict-variant", "conflict-variant"]); for (const order of orders) { - const union = new LoreUnion(); - for (const name of order) union.union({ file: `${name}.lore`, bytes: lines[name] }); + const union = new LyncUnion(); + for (const name of order) union.union({ file: `${name}.lync`, bytes: lines[name] }); const result = union.result(); expect(result.lines.map((line) => line.class)).toEqual(batchClasses); @@ -444,7 +444,7 @@ describe("LORE-V0 line parser vectors", () => { extra: { critical: true, future: "unknown-top-level" }, }); const input = `${target}\n${critical}\n`; - const result = parseLoreFiles([{ file: "hostile.lore", bytes: input }]); + const result = parseLyncFiles([{ file: "hostile.lync", bytes: input }]); expect(result.lines.map((line) => ({ class: line.class, id: line.id }))).toEqual([ { class: "accepted", id: "target" }, @@ -452,7 +452,7 @@ describe("LORE-V0 line parser vectors", () => { ]); expect(result.suppression.suppressedPayloadIds).toEqual([]); expect(result.suppression.notSuppressedIds).toEqual(["crit-nonconforming", "target"]); - expect(Buffer.from(exportCarriedLoreBytes(result)).toString("utf8")).toBe(input); + expect(Buffer.from(exportCarriedLyncBytes(result)).toString("utf8")).toBe(input); }); it("detects digest splice before decoding invalid UTF-8", () => { @@ -464,35 +464,35 @@ describe("LORE-V0 line parser vectors", () => { '"},"digest":"sha256:0000000000000000000000000000000000000000000000000000000000000000"}\n', ); const bytes = Buffer.concat([prefix, invalid, suffix]); - const result = parseLoreFiles([{ file: "raw.lore", bytes }]); + const result = parseLyncFiles([{ file: "raw.lync", bytes }]); expect(result.lines[0]?.class).toBe("damaged"); expect(result.lines[0]?.reason).toBe("sha256 mismatch"); expect(result.lines[0]?.digest).toBe("sha256:0000000000000000000000000000000000000000000000000000000000000000"); expect(result.lines[0]?.bodyBytes && digestFor(result.lines[0].bodyBytes)).not.toBe(result.lines[0]?.digest); - expect(Buffer.from(exportCarriedLoreBytes(result)).equals(bytes)).toBe(true); + expect(Buffer.from(exportCarriedLyncBytes(result)).equals(bytes)).toBe(true); }); it("preserves structurally valid signatures without verifying them", () => { const body = eventBody({ id: "signed-but-unverified" }); const sig = "QUJDRA=="; const input = `${body.slice(0, -1)},"digest":"${digestFor(Buffer.from(body))}","sig":"${sig}"}\n`; - const result = parseLoreFiles([{ file: "signed.lore", bytes: input }]); + const result = parseLyncFiles([{ file: "signed.lync", bytes: input }]); expect(result.lines[0]?.class).toBe("accepted"); expect(result.lines[0]?.hasDigest).toBe(true); expect(result.lines[0]?.hasSig).toBe(true); expect(result.lines[0]?.sig).toBe(sig); - expect(Buffer.from(exportCarriedLoreBytes(result)).toString("utf8")).toBe(input); + expect(Buffer.from(exportCarriedLyncBytes(result)).toString("utf8")).toBe(input); }); it("treats invalid signature syntax as body instead of repairing the splice", () => { const body = eventBody({ id: "invalid-signature-syntax" }); const digest = digestFor(Buffer.from(body)); const base = body.slice(0, -1); - const result = parseLoreFiles([ - { file: "url.lore", bytes: `${base},"digest":"${digest}","sig":"abc-_"}\n` }, - { file: "mod.lore", bytes: `${base},"digest":"${digest}","sig":"abc"}\n` }, + const result = parseLyncFiles([ + { file: "url.lync", bytes: `${base},"digest":"${digest}","sig":"abc-_"}\n` }, + { file: "mod.lync", bytes: `${base},"digest":"${digest}","sig":"abc"}\n` }, ]); expect(result.lines.map((line) => ({ class: line.class, hasDigest: line.hasDigest, hasSig: line.hasSig }))).toEqual([ diff --git a/packages/core/test/references.test.ts b/packages/core/test/references.test.ts index c1e68f5..812b7e1 100644 --- a/packages/core/test/references.test.ts +++ b/packages/core/test/references.test.ts @@ -59,7 +59,7 @@ describe("references", () => { globalThis.btoa = (value: string) => originalBuffer.from(value, "binary").toString("base64"); globalThis.atob = (value: string) => originalBuffer.from(value, "base64").toString("binary"); - const ref = threadRef("lore:loom", "turn-unicode-\u2713"); + const ref = threadRef("lync:loom", "turn-unicode-\u2713"); expect(decodeReference(encodeReference(ref))).toEqual(ref); } finally { globalThis.Buffer = originalBuffer; diff --git a/packages/core/test/lore-sha256.test.ts b/packages/core/test/sha256.test.ts similarity index 96% rename from packages/core/test/lore-sha256.test.ts rename to packages/core/test/sha256.test.ts index ddaddf1..58dc7c8 100644 --- a/packages/core/test/lore-sha256.test.ts +++ b/packages/core/test/sha256.test.ts @@ -1,6 +1,6 @@ import { createHash, randomBytes } from "node:crypto"; import { describe, expect, it } from "vitest"; -import { sha256Hex } from "../src/lore/sha256.js"; +import { sha256Hex } from "../src/sha256.js"; function nodeHex(bytes: Uint8Array): string { return createHash("sha256").update(bytes).digest("hex"); diff --git a/packages/core/test/lore-storage.test.ts b/packages/core/test/storage.test.ts similarity index 81% rename from packages/core/test/lore-storage.test.ts rename to packages/core/test/storage.test.ts index 6a8b750..0499ffe 100644 --- a/packages/core/test/lore-storage.test.ts +++ b/packages/core/test/storage.test.ts @@ -2,41 +2,41 @@ import fs from "node:fs/promises"; import os from "node:os"; import path from "node:path"; import { describe, expect, it } from "vitest"; -import { createLoreLooms } from "../src/lore/looms.js"; -import { createFileEventStore } from "../src/lore/file-log.js"; -import { createIndexedDbEventStore } from "../src/lore/idb-log.js"; -import { createMemoryEventStore } from "../src/lore/memory-log.js"; -import type { EventStore } from "../src/lore/store.js"; +import { createLyncLooms } from "../src/looms.js"; +import { createFileEventStore } from "../src/file-log.js"; +import { createIndexedDbEventStore } from "../src/idb-log.js"; +import { createMemoryEventStore } from "../src/memory-log.js"; +import type { EventStore } from "../src/store.js"; type Payload = { text: string }; type LoomMeta = { title: string }; -describe("lore storage backends", () => { +describe("lync storage backends", () => { it("round-trips byte-identical events through the memory store", async () => { await assertRoundTrip(createMemoryEventStore()); }); it("round-trips byte-identical events through the file store", async () => { - const dir = await fs.mkdtemp(path.join(os.tmpdir(), "lync-lore-file-")); + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "lync-file-")); const written = await assertRoundTrip(createFileEventStore(dir)); expect(await createFileEventStore(dir).exportRootBytes?.(written.rootId)).toEqual(written.bytes); }); - it("loads mixed .lync and legacy .lore event files", async () => { - const dir = await fs.mkdtemp(path.join(os.tmpdir(), "lync-lore-mixed-")); + it("loads only .lync event files and ignores other extensions", async () => { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "lync-mixed-")); await fs.writeFile( path.join(dir, "new.lync"), - '{"v":1,"id":"new-root","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"tester"},"parents":[],"payload":{"text":"new extension"}}\n', + '{"v":1,"id":"new-root","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"tester"},"parents":[],"payload":{"text":"new extension"}}\n', ); await fs.writeFile( - path.join(dir, "legacy.lore"), - '{"v":1,"id":"legacy-root","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"tester"},"parents":[],"payload":{"text":"legacy extension"}}\n', + path.join(dir, "other.txt"), + '{"v":1,"id":"other-root","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"tester"},"parents":[],"payload":{"text":"not an event file"}}\n', ); const store = createFileEventStore(dir); await expect(store.byId("new-root")).resolves.toMatchObject({ body: { id: "new-root" } }); - await expect(store.byId("legacy-root")).resolves.toMatchObject({ body: { id: "legacy-root" } }); - await expect(store.diagnostics()).resolves.toMatchObject({ events: 2 }); + await expect(store.byId("other-root")).resolves.toBeNull(); + await expect(store.diagnostics()).resolves.toMatchObject({ events: 1 }); }); it("round-trips byte-identical events through the IndexedDB-shaped store", async () => { @@ -49,7 +49,7 @@ describe("lore storage backends", () => { async function assertRoundTrip(store: EventStore) { let nextId = 0; let nextTime = 1000; - const looms = createLoreLooms({ + const looms = createLyncLooms({ store, author: { actor: "tester" }, createId: () => `event-${++nextId}`, @@ -59,18 +59,18 @@ async function assertRoundTrip(store: EventStore) { const loom = await looms.open(info.id); const first = await loom.appendTurn(null, { text: "A" }); await loom.appendTurn(first.id, { text: "B" }); - const before = await store.exportRootBytes?.(info.id.slice("lore:".length)); + const before = await store.exportRootBytes?.(info.id.slice("lync:".length)); const clone = createMemoryEventStore(); for (const line of (before ?? "").trimEnd().split("\n")) { if (line) await clone.union(line); } - const after = await clone.exportRootBytes?.(info.id.slice("lore:".length)); + const after = await clone.exportRootBytes?.(info.id.slice("lync:".length)); expect(after).toEqual(before); const reopened = await looms.open(info.id); expect(await reopened.export()).toEqual(await loom.export()); - return { rootId: info.id.slice("lore:".length), bytes: before }; + return { rootId: info.id.slice("lync:".length), bytes: before }; } function createFakeIndexedDB(): IDBFactory { diff --git a/packages/core/test/vectors/lore-vectors-draft/04-garbage-classes/input.lore b/packages/core/test/vectors/lore-vectors-draft/04-garbage-classes/input.lore deleted file mode 100644 index a88691c..0000000 --- a/packages/core/test/vectors/lore-vectors-draft/04-garbage-classes/input.lore +++ /dev/null @@ -1,7 +0,0 @@ -{"v":1,"id":"dup-depth","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{"x":1,"x":2}} - {"v":1,"id":"leading-space","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}} -{"v":1,"id":"bad-kind","kind":"lorename","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}} -{"v":2,"id":"unimplemented-v","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}} - -{"v":1,"id":"crlf","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}} -{"v":1,"id":"018f0000-0000-7000-8000-000000000031","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"good after bad"},"digest":"sha256:04f7cc982dee7f7a75f9d631f6248abe053005d5631610be4070e5e2e2c0f269"} diff --git a/packages/core/test/vectors/lore-vectors-draft/06-graph-obstacles/input.lore b/packages/core/test/vectors/lore-vectors-draft/06-graph-obstacles/input.lore deleted file mode 100644 index a451e23..0000000 --- a/packages/core/test/vectors/lore-vectors-draft/06-graph-obstacles/input.lore +++ /dev/null @@ -1,7 +0,0 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000051","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000052"],"payload":{"text":"cycle A"},"digest":"sha256:c8c821dc036febfa2d9ccd7d74c9cdcfce1d92f4401f0bc568f45d6f89fd4f5d"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000052","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000051"],"payload":{"text":"cycle B"},"digest":"sha256:b9afaada4fb288b4927c3b0800128fbdd9e08bcce298124d23d4f1e80228ce2a"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000053","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000051"],"payload":{"text":"descendant of cycle"},"digest":"sha256:88548c0bd4bdd4ed5f5473017fbda24e717464e452468f5e2950c7ab5a2072b8"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000054","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-00000000ffff"],"payload":{"text":"dangling parent"},"digest":"sha256:bdb32d867548c71eadb66bc659c5597af88380bffb40406036a83711bee087b4"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000055","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"conflicted parent A"},"digest":"sha256:89897153cfb4c68604cdba87f61b840207d60bdeb04a9777749286bef7936e87"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000055","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"conflicted parent B"},"digest":"sha256:d9c4b2fcc702e75d6614f7dbacdfea032ce4edc5653213836580598945c6b9ce"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000056","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000055"],"payload":{"text":"child of conflicted id"},"digest":"sha256:d0d8d0ca66cf733bbab2b045602a581782ec80492fb8fe98bfcec2d3d838442f"} diff --git a/packages/core/test/vectors/lore-vectors-draft/08-spelling-vs-value/input.lore b/packages/core/test/vectors/lore-vectors-draft/08-spelling-vs-value/input.lore deleted file mode 100644 index 3dc4cc1..0000000 --- a/packages/core/test/vectors/lore-vectors-draft/08-spelling-vs-value/input.lore +++ /dev/null @@ -1,6 +0,0 @@ -{"v":1,"id":"spell-\u0061","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"\u0061lice"},"parents":[],"payload":{"text":"escaped id and actor"},"digest":"sha256:24dda4f838a7650ec999430505a69735c315865a9b84d3a75b0c0a2fbe2c4a0f"} -{"v":1,"id":"spell-a","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":[],"payload":{"text":"same decoded id, different body"},"digest":"sha256:1af59d433ebcfe5b77f366128d09616ab19f9f705365f9967906655e315df5cd"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000071","kind":"lore\u002fartifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":["spell-a"],"payload":{},"digest":"sha256:5c7f76c2c519b0b239731bc801c1596549ef473caf03b298cd6e3152dea1963a"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000072","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":[],"payload":{"a":1,"\u0061":2}} -{"v":1,"id":"018f0000-0000-7000-8000-000000000073","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"\u0061lice"},"parents":[],"payload":{"text":"suppression target"},"digest":"sha256:f6fafc74ab83c110282a4733939cdced8a6a6cbc9b51884452cb1031262ea05e"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000074","kind":"future/embargo","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":["018f0000-0000-7000-8000-000000000073"],"payload":{},"critical":true,"digest":"sha256:e28ab7605441906daa26b08118d3c8ff9dc5943463510b61e3337aa7d69b1395"} diff --git a/packages/core/test/vectors/lore-vectors-draft/09-marked-at-semantics/input.lore b/packages/core/test/vectors/lore-vectors-draft/09-marked-at-semantics/input.lore deleted file mode 100644 index c66093e..0000000 --- a/packages/core/test/vectors/lore-vectors-draft/09-marked-at-semantics/input.lore +++ /dev/null @@ -1,4 +0,0 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000081","kind":"lore/artifact","at":"2026-07-06T04:10:00-07:00","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"imported later"},"marked":"2026-07-07T01:02:03.123456Z","digest":"sha256:10f0986db7febb06f833b2378b73112d0e9218033dc48b35b7920ad18db775a0"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000082","kind":"lore/artifact","at":"2026-12-31T23:59:60Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"RFC3339 leap second ABNF"},"digest":"sha256:58c91550b39faa0804a7162aff0d1b632e2c6699824cecce7ccf4ad7c183a7c5"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000083","kind":"lore/artifact","at":"2026-07-06 04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"space not T"},"digest":"sha256:8ba1419895a624b6e649377b833df59c3f7c84c7e9e913a337971fbb91f3286e"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000084","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"bad marked"},"marked":"not-a-time","digest":"sha256:15e288edb42fdf551cdb7c5311f108a5c013d86e286e426decbbfe8b8141f5b1"} diff --git a/packages/core/test/vectors/lore-vectors-draft/10-merge-union/a.lore b/packages/core/test/vectors/lore-vectors-draft/10-merge-union/a.lore deleted file mode 100644 index 55f256a..0000000 --- a/packages/core/test/vectors/lore-vectors-draft/10-merge-union/a.lore +++ /dev/null @@ -1,3 +0,0 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000091","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"root"},"digest":"sha256:15087c7895d8db026a73ebbfdd632964a4b5388e4f86f8eedc24d78fd96b7b51"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000092","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000091"],"payload":{"text":"from A"},"digest":"sha256:f3e60dc941e185fb04528e0b028973cfa270de9df31213bd851a42442dec3a96"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000094","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"merge conflict A"},"digest":"sha256:f55b52407b5226b217dc43ab1aa3a513b05ba567ca4bfc7999e6bc9a27ce438d"} diff --git a/packages/core/test/vectors/lore-vectors-draft/10-merge-union/b.lore b/packages/core/test/vectors/lore-vectors-draft/10-merge-union/b.lore deleted file mode 100644 index d321484..0000000 --- a/packages/core/test/vectors/lore-vectors-draft/10-merge-union/b.lore +++ /dev/null @@ -1,3 +0,0 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000092","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000091"],"payload":{"text":"from A"},"digest":"sha256:f3e60dc941e185fb04528e0b028973cfa270de9df31213bd851a42442dec3a96"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000093","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000092"],"payload":{"text":"from B completes context"},"digest":"sha256:9cb54ae0e4f0731f3a9ab61679d339d41d4df76248c470d74b582b69b3c58a5f"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000094","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"merge conflict B"},"digest":"sha256:608a33dee8afdf84ce8ac2fa7306d494c18690f122c69236a7865ab1014a227b"} diff --git a/packages/core/test/vectors/lore-vectors-draft/11-nonconforming-carried/input.lore b/packages/core/test/vectors/lore-vectors-draft/11-nonconforming-carried/input.lore deleted file mode 100644 index bcd3725..0000000 --- a/packages/core/test/vectors/lore-vectors-draft/11-nonconforming-carried/input.lore +++ /dev/null @@ -1,2 +0,0 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000101","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"unknown top"},"mood":"future","digest":"sha256:11c3bc22efb5e8ce313a2dddb18344188519a57c1332d634734a6c2518f02b3a"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000102","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates","role":"extra"},"parents":[],"payload":{"text":"unknown author"},"digest":"sha256:cc2913959f2825afe909ca4953f628291fc379bd55538447b4dd253a52efb98c"} diff --git a/packages/core/test/vectors/lore-vectors-draft/12-invalid-sig-splice/input.lore b/packages/core/test/vectors/lore-vectors-draft/12-invalid-sig-splice/input.lore deleted file mode 100644 index bbaef43..0000000 --- a/packages/core/test/vectors/lore-vectors-draft/12-invalid-sig-splice/input.lore +++ /dev/null @@ -1 +0,0 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000111","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"invalid sig grammar"},"digest":"sha256:717f0c9f24921b0e9291642c7f6577d59266a2b3740e38f64069f15391fa334a","sig":"abc-_"} diff --git a/packages/core/test/vectors/lore-vectors-draft/01-valid-events/expected.json b/packages/core/test/vectors/v0/01-valid-events/expected.json similarity index 91% rename from packages/core/test/vectors/lore-vectors-draft/01-valid-events/expected.json rename to packages/core/test/vectors/v0/01-valid-events/expected.json index da1db3f..3a2c72c 100644 --- a/packages/core/test/vectors/lore-vectors-draft/01-valid-events/expected.json +++ b/packages/core/test/vectors/v0/01-valid-events/expected.json @@ -1,12 +1,12 @@ { "fixture": "01-valid-events", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "has_digest": false, "has_sig": false, "id": "018f0000-0000-7000-8000-000000000001", @@ -14,7 +14,7 @@ }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "has_digest": true, "has_sig": false, "id": "018f0000-0000-7000-8000-000000000002", @@ -22,7 +22,7 @@ }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "has_digest": true, "has_sig": true, "id": "018f0000-0000-7000-8000-000000000003", @@ -30,7 +30,7 @@ }, { "class": "nonconforming", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000004", "line": 4, "reason": "final line missing LF; still accepted and view-eligible" diff --git a/packages/core/test/vectors/lore-vectors-draft/01-valid-events/input.lore b/packages/core/test/vectors/v0/01-valid-events/input.lync similarity index 58% rename from packages/core/test/vectors/lore-vectors-draft/01-valid-events/input.lore rename to packages/core/test/vectors/v0/01-valid-events/input.lync index d0b7750..704cab6 100644 --- a/packages/core/test/vectors/lore-vectors-draft/01-valid-events/input.lore +++ b/packages/core/test/vectors/v0/01-valid-events/input.lync @@ -1,4 +1,4 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000001","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"root"}} -{"v":1,"id":"018f0000-0000-7000-8000-000000000002","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000001"],"payload":{"text":"digested child","digest":"payload-ok","sig":"payload-ok"},"digest":"sha256:4e1597091f2eaa244f4737d2c7158aac9ff725506cead963113034a896e10c99"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000003","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000002"],"payload":{"text":"signed child"},"marked":"2026-07-06T05:00:00Z","digest":"sha256:2b2529b7d66f8ca0dfe4bbd7afad40ef109e4f86e8aefdbf65797013cc972903","sig":"QUJDRA=="} -{"v":1,"id":"018f0000-0000-7000-8000-000000000004","kind":"lore/artifact","at":"2026-07-06t04:10:03z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"lowercase timestamp accepted"}} \ No newline at end of file +{"v":1,"id":"018f0000-0000-7000-8000-000000000001","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"root"}} +{"v":1,"id":"018f0000-0000-7000-8000-000000000002","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000001"],"payload":{"text":"digested child","digest":"payload-ok","sig":"payload-ok"},"digest":"sha256:660967f4e97b4d3c20afc71a44291b48d590dc36c006bebaa97c85e805bce97b"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000003","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000002"],"payload":{"text":"signed child"},"marked":"2026-07-06T05:00:00Z","digest":"sha256:ed59f4b5a591f817b3049da0c09940c6c66e2cfcaf1b1b3719be14cd96ccc6d2","sig":"QUJDRA=="} +{"v":1,"id":"018f0000-0000-7000-8000-000000000004","kind":"lync/artifact","at":"2026-07-06t04:10:03z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"lowercase timestamp accepted"}} \ No newline at end of file diff --git a/packages/core/test/vectors/lore-vectors-draft/02-splice-anchoring/expected.json b/packages/core/test/vectors/v0/02-splice-anchoring/expected.json similarity index 87% rename from packages/core/test/vectors/lore-vectors-draft/02-splice-anchoring/expected.json rename to packages/core/test/vectors/v0/02-splice-anchoring/expected.json index 9170078..4ab70fa 100644 --- a/packages/core/test/vectors/lore-vectors-draft/02-splice-anchoring/expected.json +++ b/packages/core/test/vectors/v0/02-splice-anchoring/expected.json @@ -1,32 +1,32 @@ { "fixture": "02-splice-anchoring", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000011", "line": 1, "reason": "payload marker-like bytes are not a splice" }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000012", "line": 2, "reason": "reserved names inside payload are allowed" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 3, "reason": "near-splice is body; parsed body contains reserved top-level digest" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 4, "reason": "top-level digest in body is reserved" } diff --git a/packages/core/test/vectors/lore-vectors-draft/02-splice-anchoring/input.lore b/packages/core/test/vectors/v0/02-splice-anchoring/input.lync similarity index 61% rename from packages/core/test/vectors/lore-vectors-draft/02-splice-anchoring/input.lore rename to packages/core/test/vectors/v0/02-splice-anchoring/input.lync index e4aa90c..84a76b5 100644 --- a/packages/core/test/vectors/lore-vectors-draft/02-splice-anchoring/input.lore +++ b/packages/core/test/vectors/v0/02-splice-anchoring/input.lync @@ -1,4 +1,4 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000011","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"payload contains marker bytes ,\"digest\":\"sha256:0000000000000000000000000000000000000000000000000000000000000000\"} and keeps going"},"digest":"sha256:d4200d13a932dba373608a5cb4f3b462e3be5a0adfdc761380d94f5b110bb039"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000012","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"body has reserved words at the end","digest":"payload name","sig":"payload name"},"digest":"sha256:f70155556516d8b02767de27ffb8c1828e571fcd181ccadee621c2f943b623f3"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000013","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"near"},"digest":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000014","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"reserved top level"},"digest":"not-line-metadata"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000011","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"payload contains marker bytes ,\"digest\":\"sha256:0000000000000000000000000000000000000000000000000000000000000000\"} and keeps going"},"digest":"sha256:b4056d15f5bdc29507608d1697952620d628a8dd64411380cf65bd3a2633bb7f"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000012","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"body has reserved words at the end","digest":"payload name","sig":"payload name"},"digest":"sha256:a15f1dc1fb6a0f434084f4a40681c7ff2e86c65617f0b75d989cf4b73b002521"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000013","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"near"},"digest":"sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000014","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"reserved top level"},"digest":"not-line-metadata"} diff --git a/packages/core/test/vectors/lore-vectors-draft/03-damaged-digest/expected.json b/packages/core/test/vectors/v0/03-damaged-digest/expected.json similarity index 83% rename from packages/core/test/vectors/lore-vectors-draft/03-damaged-digest/expected.json rename to packages/core/test/vectors/v0/03-damaged-digest/expected.json index 45bc2a0..bdaff3f 100644 --- a/packages/core/test/vectors/lore-vectors-draft/03-damaged-digest/expected.json +++ b/packages/core/test/vectors/v0/03-damaged-digest/expected.json @@ -1,18 +1,18 @@ { "fixture": "03-damaged-digest", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "damaged", - "file": "input.lore", + "file": "input.lync", "line": 1, "reason": "sha256 mismatch" }, { "class": "damaged", - "file": "input.lore", + "file": "input.lync", "line": 2, "reason": "sha256 mismatch; do not parse duplicate member names after damage" } diff --git a/packages/core/test/vectors/lore-vectors-draft/03-damaged-digest/input.lore b/packages/core/test/vectors/v0/03-damaged-digest/input.lync similarity index 75% rename from packages/core/test/vectors/lore-vectors-draft/03-damaged-digest/input.lore rename to packages/core/test/vectors/v0/03-damaged-digest/input.lync index d885ef5..d71e777 100644 --- a/packages/core/test/vectors/lore-vectors-draft/03-damaged-digest/input.lore +++ b/packages/core/test/vectors/v0/03-damaged-digest/input.lync @@ -1,2 +1,2 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000021","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"tampered"},"digest":"sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000022","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"x":1,"x":2},"digest":"sha256:eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000021","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"tampered"},"digest":"sha256:ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000022","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"x":1,"x":2},"digest":"sha256:eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"} diff --git a/packages/core/test/vectors/lore-vectors-draft/04-garbage-classes/expected.json b/packages/core/test/vectors/v0/04-garbage-classes/expected.json similarity index 81% rename from packages/core/test/vectors/lore-vectors-draft/04-garbage-classes/expected.json rename to packages/core/test/vectors/v0/04-garbage-classes/expected.json index 0d017e3..8f0280b 100644 --- a/packages/core/test/vectors/lore-vectors-draft/04-garbage-classes/expected.json +++ b/packages/core/test/vectors/v0/04-garbage-classes/expected.json @@ -1,48 +1,48 @@ { "fixture": "04-garbage-classes", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 1, "reason": "duplicate member name at nested object depth" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 2, "reason": "bytes outside object: leading whitespace" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 3, "reason": "kind lacks namespace/name slash" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 4, "reason": "unimplemented v" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 5, "reason": "empty line" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 6, "reason": "CR before LF is trailing content" }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000031", "line": 7 } diff --git a/packages/core/test/vectors/v0/04-garbage-classes/input.lync b/packages/core/test/vectors/v0/04-garbage-classes/input.lync new file mode 100644 index 0000000..4845e94 --- /dev/null +++ b/packages/core/test/vectors/v0/04-garbage-classes/input.lync @@ -0,0 +1,7 @@ +{"v":1,"id":"dup-depth","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{"x":1,"x":2}} + {"v":1,"id":"leading-space","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}} +{"v":1,"id":"bad-kind","kind":"noslashname","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}} +{"v":2,"id":"unimplemented-v","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}} + +{"v":1,"id":"crlf","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}} +{"v":1,"id":"018f0000-0000-7000-8000-000000000031","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"good after bad"},"digest":"sha256:99c791638de72ffbc3113a2c829c7acf1d73f8542927fca61331a7f9f97f93aa"} diff --git a/packages/core/test/vectors/lore-vectors-draft/05-conflicts-and-duplicates/expected.json b/packages/core/test/vectors/v0/05-conflicts-and-duplicates/expected.json similarity index 86% rename from packages/core/test/vectors/lore-vectors-draft/05-conflicts-and-duplicates/expected.json rename to packages/core/test/vectors/v0/05-conflicts-and-duplicates/expected.json index 18808a6..e0dc54f 100644 --- a/packages/core/test/vectors/lore-vectors-draft/05-conflicts-and-duplicates/expected.json +++ b/packages/core/test/vectors/v0/05-conflicts-and-duplicates/expected.json @@ -4,52 +4,52 @@ ], "fixture": "05-conflicts-and-duplicates", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000041", "line": 1 }, { "class": "accepted", "duplicate_sighting": true, - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000041", "line": 2 }, { "class": "accepted", "duplicate_sighting": true, - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000041", "line": 3, "metadata_disagreement": true }, { "class": "conflict-variant", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000042", "line": 4 }, { "class": "conflict-variant", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000042", "line": 5 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000043", "line": 6 }, { "class": "accepted", "duplicate_sighting": true, - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000043", "line": 7, "metadata_disagreement": true diff --git a/packages/core/test/vectors/lore-vectors-draft/05-conflicts-and-duplicates/input.lore b/packages/core/test/vectors/v0/05-conflicts-and-duplicates/input.lync similarity index 50% rename from packages/core/test/vectors/lore-vectors-draft/05-conflicts-and-duplicates/input.lore rename to packages/core/test/vectors/v0/05-conflicts-and-duplicates/input.lync index 3162053..91e93df 100644 --- a/packages/core/test/vectors/lore-vectors-draft/05-conflicts-and-duplicates/input.lore +++ b/packages/core/test/vectors/v0/05-conflicts-and-duplicates/input.lync @@ -1,7 +1,7 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000041","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"same event"},"digest":"sha256:50aea91687438d63da7c4a5138f2a6dbd03fdf67d3ba19a83dd456391a98425c"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000041","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"same event"},"digest":"sha256:50aea91687438d63da7c4a5138f2a6dbd03fdf67d3ba19a83dd456391a98425c"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000041","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"same event"},"digest":"sha256:50aea91687438d63da7c4a5138f2a6dbd03fdf67d3ba19a83dd456391a98425c","sig":"QUJDRA=="} -{"v":1,"id":"018f0000-0000-7000-8000-000000000042","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"variant A"},"digest":"sha256:01c0f5b179b437fa7736d6d09e26350ece23de563aea7481d2a92e63fa8e42b6"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000042","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"variant B"},"digest":"sha256:4d6cb0d48e83be2a13689d2985821c2964ffaf2c24b196f65d5749c3da30584c"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000043","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"metadata disagreement"},"digest":"sha256:16c2b60f2e3ac988ffc15444e7fb221889ee4a85e828d2699db32b09df8282ba"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000043","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"metadata disagreement"}} +{"v":1,"id":"018f0000-0000-7000-8000-000000000041","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"same event"},"digest":"sha256:86187fedd06467b04fcf7e1e7823d6b05a4c054a537de0039c5bf781fdbe7ef9"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000041","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"same event"},"digest":"sha256:86187fedd06467b04fcf7e1e7823d6b05a4c054a537de0039c5bf781fdbe7ef9"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000041","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"same event"},"digest":"sha256:86187fedd06467b04fcf7e1e7823d6b05a4c054a537de0039c5bf781fdbe7ef9","sig":"QUJDRA=="} +{"v":1,"id":"018f0000-0000-7000-8000-000000000042","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"variant A"},"digest":"sha256:58e92afbb37a79fb47328a54d28e8729fab522b9df947ed4bc8f29b866710758"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000042","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"variant B"},"digest":"sha256:68d5b51f88d23e78628d243e292b67ba30bea0d0c52742716ae121f08d55a1d0"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000043","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"metadata disagreement"},"digest":"sha256:3e031dce31df3c8efdeff5d17bc586505de7b420e45589d6bcf699dcfb5d1a0c"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000043","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"metadata disagreement"}} diff --git a/packages/core/test/vectors/lore-vectors-draft/06-graph-obstacles/expected.json b/packages/core/test/vectors/v0/06-graph-obstacles/expected.json similarity index 92% rename from packages/core/test/vectors/lore-vectors-draft/06-graph-obstacles/expected.json rename to packages/core/test/vectors/v0/06-graph-obstacles/expected.json index f1ff2e3..5628303 100644 --- a/packages/core/test/vectors/lore-vectors-draft/06-graph-obstacles/expected.json +++ b/packages/core/test/vectors/v0/06-graph-obstacles/expected.json @@ -23,48 +23,48 @@ } ], "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000051", "line": 1 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000052", "line": 2 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000053", "line": 3 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000054", "line": 4 }, { "class": "conflict-variant", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000055", "line": 5 }, { "class": "conflict-variant", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000055", "line": 6 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000056", "line": 7 } diff --git a/packages/core/test/vectors/v0/06-graph-obstacles/input.lync b/packages/core/test/vectors/v0/06-graph-obstacles/input.lync new file mode 100644 index 0000000..5f49920 --- /dev/null +++ b/packages/core/test/vectors/v0/06-graph-obstacles/input.lync @@ -0,0 +1,7 @@ +{"v":1,"id":"018f0000-0000-7000-8000-000000000051","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000052"],"payload":{"text":"cycle A"},"digest":"sha256:35d8306bb462d80ecf049300d81156bf68af2c0ceb0e05f1dd9c4b158a7e66af"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000052","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000051"],"payload":{"text":"cycle B"},"digest":"sha256:4c3ccb3efe630e65d637108bcec6a2eb24c05e150c4e7d8e366f726b31567d5d"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000053","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000051"],"payload":{"text":"descendant of cycle"},"digest":"sha256:496bcecf3f448c73ef8876e9fcbc2afbc2e7a33a52bf105cfe125ec536a041aa"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000054","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-00000000ffff"],"payload":{"text":"dangling parent"},"digest":"sha256:fe256fee1df6a8441da554ed74f903594065b7a47d4f479a4f256d66db0b18dd"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000055","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"conflicted parent A"},"digest":"sha256:19143157b4bff8594dc4559d12b6b37c1c00b82a090ec40afff26ab6c7f01d15"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000055","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"conflicted parent B"},"digest":"sha256:95e8c54c66ebfcefeee60358a9562281f7c78f60d080273b91f1cfcf2a351c36"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000056","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000055"],"payload":{"text":"child of conflicted id"},"digest":"sha256:fd38ed33747cd11fe74fa5aae7b1d37eba39785ca9837829a588cb8a447343eb"} diff --git a/packages/core/test/vectors/lore-vectors-draft/07-critical-suppression/expected.json b/packages/core/test/vectors/v0/07-critical-suppression/expected.json similarity index 88% rename from packages/core/test/vectors/lore-vectors-draft/07-critical-suppression/expected.json rename to packages/core/test/vectors/v0/07-critical-suppression/expected.json index 5ead30f..ae2034f 100644 --- a/packages/core/test/vectors/lore-vectors-draft/07-critical-suppression/expected.json +++ b/packages/core/test/vectors/v0/07-critical-suppression/expected.json @@ -1,60 +1,60 @@ { "fixture": "07-critical-suppression", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000061", "line": 1 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000062", "line": 2 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000063", "line": 3 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000064", "line": 4 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000065", "line": 5 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000066", "line": 6 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000067", "line": 7 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000068", "line": 8 }, { "class": "damaged", - "file": "input.lore", + "file": "input.lync", "line": 9, "reason": "critical damaged line suppresses nothing" } diff --git a/packages/core/test/vectors/lore-vectors-draft/07-critical-suppression/input.lore b/packages/core/test/vectors/v0/07-critical-suppression/input.lync similarity index 71% rename from packages/core/test/vectors/lore-vectors-draft/07-critical-suppression/input.lore rename to packages/core/test/vectors/v0/07-critical-suppression/input.lync index fe69eef..f8a13d6 100644 --- a/packages/core/test/vectors/lore-vectors-draft/07-critical-suppression/input.lore +++ b/packages/core/test/vectors/v0/07-critical-suppression/input.lync @@ -1,8 +1,8 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000061","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":[],"payload":{"text":"alice actor target"},"digest":"sha256:74462549f54296390ee969663e1991940289534cbe630b75fa293684f1860dd6"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000062","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"model-x","operator":"alice"},"parents":[],"payload":{"text":"operator target"},"digest":"sha256:368810e7af891c720fe6739dfbc21e7dd1c5dd930c00370f8931538c0e8f198c"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000063","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"unknown","imported_by":"alice"},"parents":[],"payload":{"text":"imported target"},"digest":"sha256:297e9f074a464a74edb8591e605ed06e07247418c32f48e403440b69cef4bbd6"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000064","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":[],"payload":{"text":"bob target"},"digest":"sha256:6dde67bef19738dd74781bb3c2dd43c683eda954cc022726596180c0d8a61c9a"} -{"v":1,"id":"018f0000-0000-7000-8000-000000000065","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"charlie","operator":""},"parents":[],"payload":{"text":"empty operator target"},"digest":"sha256:51173bd29cca46c8c4dce8ddd3832855c9802993f02b979d0052b380a6270d48"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000061","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":[],"payload":{"text":"alice actor target"},"digest":"sha256:d776ae14ec92208e58042148b77301d6cdfa13815865e046c818e793d67a8dfb"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000062","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"model-x","operator":"alice"},"parents":[],"payload":{"text":"operator target"},"digest":"sha256:78e5017fbbc069739538269ddbfa75be7d9b0f4b9ea5e12b362cd8021d8881ce"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000063","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"unknown","imported_by":"alice"},"parents":[],"payload":{"text":"imported target"},"digest":"sha256:b11687996b52eaa9fd6f9db815e734c6f488622b9b54d8330fb0138d4c2a080f"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000064","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":[],"payload":{"text":"bob target"},"digest":"sha256:bc585641f4e87cf70bedbf168ae0974e3b04adcf3d3147ea644912ffafe97484"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000065","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"charlie","operator":""},"parents":[],"payload":{"text":"empty operator target"},"digest":"sha256:70146995d8b97109353b201cb4ab561f38d9a3e761f733607d5aa30b37b194bc"} {"v":1,"id":"018f0000-0000-7000-8000-000000000066","kind":"future/embargo","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":["018f0000-0000-7000-8000-000000000061","018f0000-0000-7000-8000-000000000062","018f0000-0000-7000-8000-000000000063","018f0000-0000-7000-8000-000000000064","018f0000-0000-7000-8000-00000000aaaa"],"payload":{"reason":"unknown critical kind"},"critical":true,"digest":"sha256:6269e4eed3d54218fbd52be978f5833dc558fc04a3808938716503fd706c377a"} {"v":1,"id":"018f0000-0000-7000-8000-000000000067","kind":"future/embargo","at":"2026-07-06T04:10:00Z","author":{"actor":"mallory"},"parents":["018f0000-0000-7000-8000-000000000061"],"payload":{"reason":"spoof-shaped negative"},"critical":true,"digest":"sha256:a5f71b5e06ed9357dc51201436b8a825d6b43533ccae537058b134e089a6f47b"} {"v":1,"id":"018f0000-0000-7000-8000-000000000068","kind":"future/embargo","at":"2026-07-06T04:10:00Z","author":{"actor":"mallory","operator":""},"parents":["018f0000-0000-7000-8000-000000000065"],"payload":{"reason":"empty string must be dropped"},"critical":true,"digest":"sha256:81b3a08c16b5d547d110f90e523a6952a732628c405bc9c5b4a002d95f6df5a7"} diff --git a/packages/core/test/vectors/lore-vectors-draft/08-spelling-vs-value/expected.json b/packages/core/test/vectors/v0/08-spelling-vs-value/expected.json similarity index 89% rename from packages/core/test/vectors/lore-vectors-draft/08-spelling-vs-value/expected.json rename to packages/core/test/vectors/v0/08-spelling-vs-value/expected.json index 019f993..f9253b4 100644 --- a/packages/core/test/vectors/lore-vectors-draft/08-spelling-vs-value/expected.json +++ b/packages/core/test/vectors/v0/08-spelling-vs-value/expected.json @@ -4,45 +4,45 @@ ], "fixture": "08-spelling-vs-value", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "conflict-variant", - "file": "input.lore", + "file": "input.lync", "id": "spell-a", "line": 1, "reason": "decoded id equals line 2, body differs" }, { "class": "conflict-variant", - "file": "input.lore", + "file": "input.lync", "id": "spell-a", "line": 2, "reason": "decoded id equals line 1, body differs" }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000071", "line": 3, "reason": "decoded kind contains slash" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 4, "reason": "duplicate decoded member name in payload" }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000073", "line": 5 }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000074", "line": 6 } diff --git a/packages/core/test/vectors/v0/08-spelling-vs-value/input.lync b/packages/core/test/vectors/v0/08-spelling-vs-value/input.lync new file mode 100644 index 0000000..adfda38 --- /dev/null +++ b/packages/core/test/vectors/v0/08-spelling-vs-value/input.lync @@ -0,0 +1,6 @@ +{"v":1,"id":"spell-\u0061","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"\u0061lice"},"parents":[],"payload":{"text":"escaped id and actor"},"digest":"sha256:b7fad5825038648cb63e36e4d035b872431d3a4ee2443481a59e4df937c16b36"} +{"v":1,"id":"spell-a","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":[],"payload":{"text":"same decoded id, different body"},"digest":"sha256:9d4e40d781b2c5419e4009d6494f6c1709022df9591115f62333431226f66799"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000071","kind":"lync\u002fartifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":["spell-a"],"payload":{},"digest":"sha256:512612d7bcb93e833f192a14ef9fab2fe840186ae168549f2ed4e85df9964606"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000072","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":[],"payload":{"a":1,"\u0061":2}} +{"v":1,"id":"018f0000-0000-7000-8000-000000000073","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"\u0061lice"},"parents":[],"payload":{"text":"suppression target"},"digest":"sha256:370476f05bc37c3c62dc5c01aa5c6588f3fdee3b04c2d0691f2858791b99f1b1"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000074","kind":"future/embargo","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":["018f0000-0000-7000-8000-000000000073"],"payload":{},"critical":true,"digest":"sha256:e28ab7605441906daa26b08118d3c8ff9dc5943463510b61e3337aa7d69b1395"} diff --git a/packages/core/test/vectors/lore-vectors-draft/09-marked-at-semantics/expected.json b/packages/core/test/vectors/v0/09-marked-at-semantics/expected.json similarity index 86% rename from packages/core/test/vectors/lore-vectors-draft/09-marked-at-semantics/expected.json rename to packages/core/test/vectors/v0/09-marked-at-semantics/expected.json index 8c4fab8..7870a04 100644 --- a/packages/core/test/vectors/lore-vectors-draft/09-marked-at-semantics/expected.json +++ b/packages/core/test/vectors/v0/09-marked-at-semantics/expected.json @@ -1,31 +1,31 @@ { "fixture": "09-marked-at-semantics", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000081", "line": 1, "marked_effective": "2026-07-07T01:02:03.123456Z" }, { "class": "accepted", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000082", "line": 2 }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 3, "reason": "at fails RFC3339 ABNF" }, { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 4, "reason": "marked fails RFC3339 ABNF" } diff --git a/packages/core/test/vectors/v0/09-marked-at-semantics/input.lync b/packages/core/test/vectors/v0/09-marked-at-semantics/input.lync new file mode 100644 index 0000000..fa61d7f --- /dev/null +++ b/packages/core/test/vectors/v0/09-marked-at-semantics/input.lync @@ -0,0 +1,4 @@ +{"v":1,"id":"018f0000-0000-7000-8000-000000000081","kind":"lync/artifact","at":"2026-07-06T04:10:00-07:00","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"imported later"},"marked":"2026-07-07T01:02:03.123456Z","digest":"sha256:02c6d2a7f9f1f17d55fd758edacc4ffa62e576218b6db1a6caac4e27c4437a4a"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000082","kind":"lync/artifact","at":"2026-12-31T23:59:60Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"RFC3339 leap second ABNF"},"digest":"sha256:e5be4a88ace7ff4d60142e400071e0ffc974f53f1f09d15d7f7d2840e2cc681a"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000083","kind":"lync/artifact","at":"2026-07-06 04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"space not T"},"digest":"sha256:92400fd10e7d966a64cf7407a21f67d1938a06ba8a2e205eb8499b8136206482"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000084","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"bad marked"},"marked":"not-a-time","digest":"sha256:e81dfddd672acc6dafe70f1ece6b6b70a9a0023e60faad4df286cea710da223b"} diff --git a/packages/core/test/vectors/v0/10-merge-union/a.lync b/packages/core/test/vectors/v0/10-merge-union/a.lync new file mode 100644 index 0000000..81c6ebe --- /dev/null +++ b/packages/core/test/vectors/v0/10-merge-union/a.lync @@ -0,0 +1,3 @@ +{"v":1,"id":"018f0000-0000-7000-8000-000000000091","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"root"},"digest":"sha256:267a63380efb40a71006d568f954fff61ef2a981a4fdaf3a5b71b3760e6bb5bc"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000092","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000091"],"payload":{"text":"from A"},"digest":"sha256:4a8aa95e5256aa75215e7ef63c5bf410c874290955410d648fa0ca59557ac75d"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000094","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"merge conflict A"},"digest":"sha256:803e3ebd8093762e392c7a3e89dd3c2213f14346945e7a1f7ffd649785a6b44c"} diff --git a/packages/core/test/vectors/v0/10-merge-union/b.lync b/packages/core/test/vectors/v0/10-merge-union/b.lync new file mode 100644 index 0000000..2327667 --- /dev/null +++ b/packages/core/test/vectors/v0/10-merge-union/b.lync @@ -0,0 +1,3 @@ +{"v":1,"id":"018f0000-0000-7000-8000-000000000092","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000091"],"payload":{"text":"from A"},"digest":"sha256:4a8aa95e5256aa75215e7ef63c5bf410c874290955410d648fa0ca59557ac75d"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000093","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":["018f0000-0000-7000-8000-000000000092"],"payload":{"text":"from B completes context"},"digest":"sha256:85c491fa201993654639e2b7c4e41883427836a3edcca4393c1b2a4fe21716e7"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000094","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"merge conflict B"},"digest":"sha256:af8ea1492ec51753fff3a07ef5c7ae79a9072bed3485292f9ac3e9b2c6c7ca21"} diff --git a/packages/core/test/vectors/lore-vectors-draft/10-merge-union/expected.json b/packages/core/test/vectors/v0/10-merge-union/expected.json similarity index 88% rename from packages/core/test/vectors/lore-vectors-draft/10-merge-union/expected.json rename to packages/core/test/vectors/v0/10-merge-union/expected.json index 1451352..173fcf7 100644 --- a/packages/core/test/vectors/lore-vectors-draft/10-merge-union/expected.json +++ b/packages/core/test/vectors/v0/10-merge-union/expected.json @@ -4,44 +4,44 @@ ], "fixture": "10-merge-union", "inputs": [ - "a.lore", - "b.lore" + "a.lync", + "b.lync" ], "line_classifications": [ { "class": "accepted", - "file": "a.lore", + "file": "a.lync", "id": "018f0000-0000-7000-8000-000000000091", "line": 1 }, { "class": "accepted", - "file": "a.lore", + "file": "a.lync", "id": "018f0000-0000-7000-8000-000000000092", "line": 2 }, { "class": "conflict-variant", - "file": "a.lore", + "file": "a.lync", "id": "018f0000-0000-7000-8000-000000000094", "line": 3 }, { "class": "accepted", "duplicate_sighting": true, - "file": "b.lore", + "file": "b.lync", "id": "018f0000-0000-7000-8000-000000000092", "line": 1 }, { "class": "accepted", - "file": "b.lore", + "file": "b.lync", "id": "018f0000-0000-7000-8000-000000000093", "line": 2 }, { "class": "conflict-variant", - "file": "b.lore", + "file": "b.lync", "id": "018f0000-0000-7000-8000-000000000094", "line": 3 } diff --git a/packages/core/test/vectors/lore-vectors-draft/11-nonconforming-carried/expected.json b/packages/core/test/vectors/v0/11-nonconforming-carried/expected.json similarity index 90% rename from packages/core/test/vectors/lore-vectors-draft/11-nonconforming-carried/expected.json rename to packages/core/test/vectors/v0/11-nonconforming-carried/expected.json index 8b0120a..a5d4168 100644 --- a/packages/core/test/vectors/lore-vectors-draft/11-nonconforming-carried/expected.json +++ b/packages/core/test/vectors/v0/11-nonconforming-carried/expected.json @@ -1,19 +1,19 @@ { "fixture": "11-nonconforming-carried", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "nonconforming", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000101", "line": 1, "reason": "unknown top-level field carried and surfaced" }, { "class": "nonconforming", - "file": "input.lore", + "file": "input.lync", "id": "018f0000-0000-7000-8000-000000000102", "line": 2, "reason": "unknown author field carried and surfaced" diff --git a/packages/core/test/vectors/v0/11-nonconforming-carried/input.lync b/packages/core/test/vectors/v0/11-nonconforming-carried/input.lync new file mode 100644 index 0000000..20935ec --- /dev/null +++ b/packages/core/test/vectors/v0/11-nonconforming-carried/input.lync @@ -0,0 +1,2 @@ +{"v":1,"id":"018f0000-0000-7000-8000-000000000101","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"unknown top"},"mood":"future","digest":"sha256:8976762581b63619a2e687a941af8f6dbce201ade3229957b410234960672db3"} +{"v":1,"id":"018f0000-0000-7000-8000-000000000102","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates","role":"extra"},"parents":[],"payload":{"text":"unknown author"},"digest":"sha256:1f804b0d4424509240d6f15fbb84661b129dadfcf48978054b141a1e69bb230a"} diff --git a/packages/core/test/vectors/lore-vectors-draft/12-invalid-sig-splice/expected.json b/packages/core/test/vectors/v0/12-invalid-sig-splice/expected.json similarity index 86% rename from packages/core/test/vectors/lore-vectors-draft/12-invalid-sig-splice/expected.json rename to packages/core/test/vectors/v0/12-invalid-sig-splice/expected.json index da31887..e0e4693 100644 --- a/packages/core/test/vectors/lore-vectors-draft/12-invalid-sig-splice/expected.json +++ b/packages/core/test/vectors/v0/12-invalid-sig-splice/expected.json @@ -1,12 +1,12 @@ { "fixture": "12-invalid-sig-splice", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 1, "reason": "invalid sig grammar means no splice; reserved top-level digest/sig remain in body" } diff --git a/packages/core/test/vectors/v0/12-invalid-sig-splice/input.lync b/packages/core/test/vectors/v0/12-invalid-sig-splice/input.lync new file mode 100644 index 0000000..e158ac0 --- /dev/null +++ b/packages/core/test/vectors/v0/12-invalid-sig-splice/input.lync @@ -0,0 +1 @@ +{"v":1,"id":"018f0000-0000-7000-8000-000000000111","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"invalid sig grammar"},"digest":"sha256:793ad69ce0d376a27c78432e5ebe745e6febc41ed425b4d8a0ee883faaa2e6d2","sig":"abc-_"} diff --git a/packages/core/test/vectors/lore-vectors-draft/13-sig-without-digest/expected.json b/packages/core/test/vectors/v0/13-sig-without-digest/expected.json similarity index 86% rename from packages/core/test/vectors/lore-vectors-draft/13-sig-without-digest/expected.json rename to packages/core/test/vectors/v0/13-sig-without-digest/expected.json index f637800..a365876 100644 --- a/packages/core/test/vectors/lore-vectors-draft/13-sig-without-digest/expected.json +++ b/packages/core/test/vectors/v0/13-sig-without-digest/expected.json @@ -1,12 +1,12 @@ { "fixture": "13-sig-without-digest", "inputs": [ - "input.lore" + "input.lync" ], "line_classifications": [ { "class": "garbage", - "file": "input.lore", + "file": "input.lync", "line": 1, "reason": "sig without digest is not a valid splice; reserved top-level sig remains in body" } diff --git a/packages/core/test/vectors/lore-vectors-draft/13-sig-without-digest/input.lore b/packages/core/test/vectors/v0/13-sig-without-digest/input.lync similarity index 68% rename from packages/core/test/vectors/lore-vectors-draft/13-sig-without-digest/input.lore rename to packages/core/test/vectors/v0/13-sig-without-digest/input.lync index b343e61..4c48707 100644 --- a/packages/core/test/vectors/lore-vectors-draft/13-sig-without-digest/input.lore +++ b/packages/core/test/vectors/v0/13-sig-without-digest/input.lync @@ -1 +1 @@ -{"v":1,"id":"018f0000-0000-7000-8000-000000000121","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"sig without digest"},"sig":"QUJDRA=="} +{"v":1,"id":"018f0000-0000-7000-8000-000000000121","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"sig without digest"},"sig":"QUJDRA=="} diff --git a/packages/core/test/vectors/lore-vectors-draft/OPEN-QUESTIONS.md b/packages/core/test/vectors/v0/OPEN-QUESTIONS.md similarity index 100% rename from packages/core/test/vectors/lore-vectors-draft/OPEN-QUESTIONS.md rename to packages/core/test/vectors/v0/OPEN-QUESTIONS.md diff --git a/packages/core/test/vectors/lore-vectors-draft/README.md b/packages/core/test/vectors/v0/README.md similarity index 94% rename from packages/core/test/vectors/lore-vectors-draft/README.md rename to packages/core/test/vectors/v0/README.md index 005a87a..3caa886 100644 --- a/packages/core/test/vectors/lore-vectors-draft/README.md +++ b/packages/core/test/vectors/v0/README.md @@ -1,9 +1,9 @@ # Lore v0 Draft Vector Suite -Draft conformance vectors for `LORE-V0.md`, Part I. These are wrangling fixtures, not a +Draft conformance vectors for the lync format spec (FORMAT.md), Part I. These are wrangling fixtures, not a ratified format for vector metadata. -Each fixture directory contains one or more `.lore` inputs and one `expected.json`. +Each fixture directory contains one or more `.lync` inputs and one `expected.json`. `10-merge-union/` intentionally has two input files. `generate.py` is the deterministic source for the fixture bytes and computes real sha256 splices. @@ -67,7 +67,7 @@ requirements. Run: ```sh -python3 lore-vectors-draft/generate.py +python3 v0/generate.py ``` This rewrites all fixture directories. The generator intentionally emits exact bytes for diff --git a/packages/core/test/vectors/lore-vectors-draft/generate.py b/packages/core/test/vectors/v0/generate.py similarity index 79% rename from packages/core/test/vectors/lore-vectors-draft/generate.py rename to packages/core/test/vectors/v0/generate.py index a3f4c32..14a1973 100644 --- a/packages/core/test/vectors/lore-vectors-draft/generate.py +++ b/packages/core/test/vectors/v0/generate.py @@ -22,7 +22,7 @@ def spliced(body, sig=None, override_digest=None): return body[:-1] + suffix + "}" -def event(id_, kind="lore/artifact", at="2026-07-06T04:10:00Z", author=None, parents=None, payload=None, **extra): +def event(id_, kind="lync/artifact", at="2026-07-06T04:10:00Z", author=None, parents=None, payload=None, **extra): obj = { "v": 1, "id": id_, @@ -58,14 +58,14 @@ def main(): b = line("018f0000-0000-7000-8000-000000000002", parents=["018f0000-0000-7000-8000-000000000001"], payload={"text": "digested child", "digest": "payload-ok", "sig": "payload-ok"}) c = line("018f0000-0000-7000-8000-000000000003", parents=["018f0000-0000-7000-8000-000000000002"], payload={"text": "signed child"}, marked="2026-07-06T05:00:00Z") d = line("018f0000-0000-7000-8000-000000000004", at="2026-07-06t04:10:03z", payload={"text": "lowercase timestamp accepted"}) - write_fixture("01-valid-events", [("input.lore", [a, spliced(b), spliced(c, "QUJDRA=="), d], False)], { + write_fixture("01-valid-events", [("input.lync", [a, spliced(b), spliced(c, "QUJDRA=="), d], False)], { "fixture": "01-valid-events", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000001", "has_digest": False, "has_sig": False}, - {"file": "input.lore", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000002", "has_digest": True, "has_sig": False}, - {"file": "input.lore", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000003", "has_digest": True, "has_sig": True}, - {"file": "input.lore", "line": 4, "class": "nonconforming", "id": "018f0000-0000-7000-8000-000000000004", "reason": "final line missing LF; still accepted and view-eligible"}, + {"file": "input.lync", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000001", "has_digest": False, "has_sig": False}, + {"file": "input.lync", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000002", "has_digest": True, "has_sig": False}, + {"file": "input.lync", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000003", "has_digest": True, "has_sig": True}, + {"file": "input.lync", "line": 4, "class": "nonconforming", "id": "018f0000-0000-7000-8000-000000000004", "reason": "final line missing LF; still accepted and view-eligible"}, ], "union_event_ids": [ "018f0000-0000-7000-8000-000000000001", @@ -98,14 +98,14 @@ def main(): p2 = line("018f0000-0000-7000-8000-000000000012", payload={"text": "body has reserved words at the end", "digest": "payload name", "sig": "payload name"}) near_splice = line("018f0000-0000-7000-8000-000000000013", payload={"text": "near"})[:-1] + ',"digest":"sha256:' + "a" * 63 + '"}' top_reserved = line("018f0000-0000-7000-8000-000000000014", payload={"text": "reserved top level"})[:-1] + ',"digest":"not-line-metadata"}' - write_fixture("02-splice-anchoring", [("input.lore", [spliced(p1), spliced(p2), near_splice, top_reserved], True)], { + write_fixture("02-splice-anchoring", [("input.lync", [spliced(p1), spliced(p2), near_splice, top_reserved], True)], { "fixture": "02-splice-anchoring", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000011", "reason": "payload marker-like bytes are not a splice"}, - {"file": "input.lore", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000012", "reason": "reserved names inside payload are allowed"}, - {"file": "input.lore", "line": 3, "class": "garbage", "reason": "near-splice is body; parsed body contains reserved top-level digest"}, - {"file": "input.lore", "line": 4, "class": "garbage", "reason": "top-level digest in body is reserved"}, + {"file": "input.lync", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000011", "reason": "payload marker-like bytes are not a splice"}, + {"file": "input.lync", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000012", "reason": "reserved names inside payload are allowed"}, + {"file": "input.lync", "line": 3, "class": "garbage", "reason": "near-splice is body; parsed body contains reserved top-level digest"}, + {"file": "input.lync", "line": 4, "class": "garbage", "reason": "top-level digest in body is reserved"}, ], "union_event_ids": ["018f0000-0000-7000-8000-000000000011", "018f0000-0000-7000-8000-000000000012"], "view_eligible_ids": ["018f0000-0000-7000-8000-000000000011", "018f0000-0000-7000-8000-000000000012"], @@ -113,13 +113,13 @@ def main(): # 03: digest mismatch wins over parse/envelope inspection. damaged_body = line("018f0000-0000-7000-8000-000000000021", payload={"text": "tampered"}) - dup_inside_damaged = '{"v":1,"id":"018f0000-0000-7000-8000-000000000022","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"x":1,"x":2}}' - write_fixture("03-damaged-digest", [("input.lore", [spliced(damaged_body, override_digest="f" * 64), spliced(dup_inside_damaged, override_digest="e" * 64)], True)], { + dup_inside_damaged = '{"v":1,"id":"018f0000-0000-7000-8000-000000000022","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"deepfates"},"parents":[],"payload":{"x":1,"x":2}}' + write_fixture("03-damaged-digest", [("input.lync", [spliced(damaged_body, override_digest="f" * 64), spliced(dup_inside_damaged, override_digest="e" * 64)], True)], { "fixture": "03-damaged-digest", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "damaged", "reason": "sha256 mismatch"}, - {"file": "input.lore", "line": 2, "class": "damaged", "reason": "sha256 mismatch; do not parse duplicate member names after damage"}, + {"file": "input.lync", "line": 1, "class": "damaged", "reason": "sha256 mismatch"}, + {"file": "input.lync", "line": 2, "class": "damaged", "reason": "sha256 mismatch; do not parse duplicate member names after damage"}, ], "union_event_ids": [], "view_eligible_ids": [], @@ -128,25 +128,25 @@ def main(): # 04: garbage classes. good_after_bad = line("018f0000-0000-7000-8000-000000000031", payload={"text": "good after bad"}) garbage_lines = [ - '{"v":1,"id":"dup-depth","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{"x":1,"x":2}}', - ' {"v":1,"id":"leading-space","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}}', - '{"v":1,"id":"bad-kind","kind":"lorename","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}}', - '{"v":2,"id":"unimplemented-v","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}}', + '{"v":1,"id":"dup-depth","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{"x":1,"x":2}}', + ' {"v":1,"id":"leading-space","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}}', + '{"v":1,"id":"bad-kind","kind":"noslashname","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}}', + '{"v":2,"id":"unimplemented-v","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}}', '', - '{"v":1,"id":"crlf","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}}\r', + '{"v":1,"id":"crlf","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"a"},"parents":[],"payload":{}}\r', spliced(good_after_bad), ] - write_fixture("04-garbage-classes", [("input.lore", garbage_lines, True)], { + write_fixture("04-garbage-classes", [("input.lync", garbage_lines, True)], { "fixture": "04-garbage-classes", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "garbage", "reason": "duplicate member name at nested object depth"}, - {"file": "input.lore", "line": 2, "class": "garbage", "reason": "bytes outside object: leading whitespace"}, - {"file": "input.lore", "line": 3, "class": "garbage", "reason": "kind lacks namespace/name slash"}, - {"file": "input.lore", "line": 4, "class": "garbage", "reason": "unimplemented v"}, - {"file": "input.lore", "line": 5, "class": "garbage", "reason": "empty line"}, - {"file": "input.lore", "line": 6, "class": "garbage", "reason": "CR before LF is trailing content"}, - {"file": "input.lore", "line": 7, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000031"}, + {"file": "input.lync", "line": 1, "class": "garbage", "reason": "duplicate member name at nested object depth"}, + {"file": "input.lync", "line": 2, "class": "garbage", "reason": "bytes outside object: leading whitespace"}, + {"file": "input.lync", "line": 3, "class": "garbage", "reason": "kind lacks namespace/name slash"}, + {"file": "input.lync", "line": 4, "class": "garbage", "reason": "unimplemented v"}, + {"file": "input.lync", "line": 5, "class": "garbage", "reason": "empty line"}, + {"file": "input.lync", "line": 6, "class": "garbage", "reason": "CR before LF is trailing content"}, + {"file": "input.lync", "line": 7, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000031"}, ], "union_event_ids": ["018f0000-0000-7000-8000-000000000031"], "view_eligible_ids": ["018f0000-0000-7000-8000-000000000031"], @@ -158,17 +158,17 @@ def main(): conflict_a = line("018f0000-0000-7000-8000-000000000042", payload={"text": "variant A"}) conflict_b = line("018f0000-0000-7000-8000-000000000042", payload={"text": "variant B"}) meta_disagree_body = line("018f0000-0000-7000-8000-000000000043", payload={"text": "metadata disagreement"}) - write_fixture("05-conflicts-and-duplicates", [("input.lore", [spliced(same), spliced(same), same_alt_meta, spliced(conflict_a), spliced(conflict_b), spliced(meta_disagree_body), meta_disagree_body], True)], { + write_fixture("05-conflicts-and-duplicates", [("input.lync", [spliced(same), spliced(same), same_alt_meta, spliced(conflict_a), spliced(conflict_b), spliced(meta_disagree_body), meta_disagree_body], True)], { "fixture": "05-conflicts-and-duplicates", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000041"}, - {"file": "input.lore", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000041", "duplicate_sighting": True}, - {"file": "input.lore", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000041", "duplicate_sighting": True, "metadata_disagreement": True}, - {"file": "input.lore", "line": 4, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000042"}, - {"file": "input.lore", "line": 5, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000042"}, - {"file": "input.lore", "line": 6, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000043"}, - {"file": "input.lore", "line": 7, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000043", "duplicate_sighting": True, "metadata_disagreement": True}, + {"file": "input.lync", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000041"}, + {"file": "input.lync", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000041", "duplicate_sighting": True}, + {"file": "input.lync", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000041", "duplicate_sighting": True, "metadata_disagreement": True}, + {"file": "input.lync", "line": 4, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000042"}, + {"file": "input.lync", "line": 5, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000042"}, + {"file": "input.lync", "line": 6, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000043"}, + {"file": "input.lync", "line": 7, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000043", "duplicate_sighting": True, "metadata_disagreement": True}, ], "union_event_ids": ["018f0000-0000-7000-8000-000000000041", "018f0000-0000-7000-8000-000000000043"], "conflict_ids": ["018f0000-0000-7000-8000-000000000042"], @@ -183,17 +183,17 @@ def main(): conf_a = line("018f0000-0000-7000-8000-000000000055", payload={"text": "conflicted parent A"}) conf_b = line("018f0000-0000-7000-8000-000000000055", payload={"text": "conflicted parent B"}) child_conf = line("018f0000-0000-7000-8000-000000000056", parents=["018f0000-0000-7000-8000-000000000055"], payload={"text": "child of conflicted id"}) - write_fixture("06-graph-obstacles", [("input.lore", [spliced(ca), spliced(cb), spliced(child_cycle), spliced(dangling), spliced(conf_a), spliced(conf_b), spliced(child_conf)], True)], { + write_fixture("06-graph-obstacles", [("input.lync", [spliced(ca), spliced(cb), spliced(child_cycle), spliced(dangling), spliced(conf_a), spliced(conf_b), spliced(child_conf)], True)], { "fixture": "06-graph-obstacles", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000051"}, - {"file": "input.lore", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000052"}, - {"file": "input.lore", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000053"}, - {"file": "input.lore", "line": 4, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000054"}, - {"file": "input.lore", "line": 5, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000055"}, - {"file": "input.lore", "line": 6, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000055"}, - {"file": "input.lore", "line": 7, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000056"}, + {"file": "input.lync", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000051"}, + {"file": "input.lync", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000052"}, + {"file": "input.lync", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000053"}, + {"file": "input.lync", "line": 4, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000054"}, + {"file": "input.lync", "line": 5, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000055"}, + {"file": "input.lync", "line": 6, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000055"}, + {"file": "input.lync", "line": 7, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000056"}, ], "union_event_ids": [ "018f0000-0000-7000-8000-000000000051", @@ -243,20 +243,20 @@ def main(): crit_spoof = line("018f0000-0000-7000-8000-000000000067", kind="future/embargo", author={"actor": "mallory"}, parents=["018f0000-0000-7000-8000-000000000061"], payload={"reason": "spoof-shaped negative"}, critical=True) crit_empty = line("018f0000-0000-7000-8000-000000000068", kind="future/embargo", author={"actor": "mallory", "operator": ""}, parents=["018f0000-0000-7000-8000-000000000065"], payload={"reason": "empty string must be dropped"}, critical=True) damaged_crit = spliced(line("018f0000-0000-7000-8000-000000000069", kind="future/embargo", author={"actor": "bob"}, parents=["018f0000-0000-7000-8000-000000000064"], payload={}, critical=True), override_digest="d" * 64) - write_fixture("07-critical-suppression", [("input.lore", [spliced(t_actor), spliced(t_operator), spliced(t_import), spliced(t_bob), spliced(t_empty), spliced(crit), spliced(crit_spoof), spliced(crit_empty), damaged_crit], True)], { + write_fixture("07-critical-suppression", [("input.lync", [spliced(t_actor), spliced(t_operator), spliced(t_import), spliced(t_bob), spliced(t_empty), spliced(crit), spliced(crit_spoof), spliced(crit_empty), damaged_crit], True)], { "fixture": "07-critical-suppression", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "reader_assumption": "ignorant of future/embargo", "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000061"}, - {"file": "input.lore", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000062"}, - {"file": "input.lore", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000063"}, - {"file": "input.lore", "line": 4, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000064"}, - {"file": "input.lore", "line": 5, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000065"}, - {"file": "input.lore", "line": 6, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000066"}, - {"file": "input.lore", "line": 7, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000067"}, - {"file": "input.lore", "line": 8, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000068"}, - {"file": "input.lore", "line": 9, "class": "damaged", "reason": "critical damaged line suppresses nothing"}, + {"file": "input.lync", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000061"}, + {"file": "input.lync", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000062"}, + {"file": "input.lync", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000063"}, + {"file": "input.lync", "line": 4, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000064"}, + {"file": "input.lync", "line": 5, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000065"}, + {"file": "input.lync", "line": 6, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000066"}, + {"file": "input.lync", "line": 7, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000067"}, + {"file": "input.lync", "line": 8, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000068"}, + {"file": "input.lync", "line": 9, "class": "damaged", "reason": "critical damaged line suppresses nothing"}, ], "suppression": { "suppressed_payload_ids": [ @@ -286,23 +286,23 @@ def main(): }) # 08: decoded string values win over spelling for id, kind, author matching, duplicate names. - escaped_id = '{"v":1,"id":"spell-\\u0061","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"\\u0061lice"},"parents":[],"payload":{"text":"escaped id and actor"}}' - plain_same_id = '{"v":1,"id":"spell-a","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":[],"payload":{"text":"same decoded id, different body"}}' - escaped_kind = '{"v":1,"id":"018f0000-0000-7000-8000-000000000071","kind":"lore\\u002fartifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":["spell-a"],"payload":{}}' - dup_escaped_keys = '{"v":1,"id":"018f0000-0000-7000-8000-000000000072","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":[],"payload":{"a":1,"\\u0061":2}}' - target_spelled = '{"v":1,"id":"018f0000-0000-7000-8000-000000000073","kind":"lore/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"\\u0061lice"},"parents":[],"payload":{"text":"suppression target"}}' + escaped_id = '{"v":1,"id":"spell-\\u0061","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"\\u0061lice"},"parents":[],"payload":{"text":"escaped id and actor"}}' + plain_same_id = '{"v":1,"id":"spell-a","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":[],"payload":{"text":"same decoded id, different body"}}' + escaped_kind = '{"v":1,"id":"018f0000-0000-7000-8000-000000000071","kind":"lync\\u002fartifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":["spell-a"],"payload":{}}' + dup_escaped_keys = '{"v":1,"id":"018f0000-0000-7000-8000-000000000072","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"bob"},"parents":[],"payload":{"a":1,"\\u0061":2}}' + target_spelled = '{"v":1,"id":"018f0000-0000-7000-8000-000000000073","kind":"lync/artifact","at":"2026-07-06T04:10:00Z","author":{"actor":"\\u0061lice"},"parents":[],"payload":{"text":"suppression target"}}' suppress_plain = '{"v":1,"id":"018f0000-0000-7000-8000-000000000074","kind":"future/embargo","at":"2026-07-06T04:10:00Z","author":{"actor":"alice"},"parents":["018f0000-0000-7000-8000-000000000073"],"payload":{},"critical":true}' - write_fixture("08-spelling-vs-value", [("input.lore", [spliced(escaped_id), spliced(plain_same_id), spliced(escaped_kind), dup_escaped_keys, spliced(target_spelled), spliced(suppress_plain)], True)], { + write_fixture("08-spelling-vs-value", [("input.lync", [spliced(escaped_id), spliced(plain_same_id), spliced(escaped_kind), dup_escaped_keys, spliced(target_spelled), spliced(suppress_plain)], True)], { "fixture": "08-spelling-vs-value", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "reader_assumption": "ignorant of future/embargo", "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "conflict-variant", "id": "spell-a", "reason": "decoded id equals line 2, body differs"}, - {"file": "input.lore", "line": 2, "class": "conflict-variant", "id": "spell-a", "reason": "decoded id equals line 1, body differs"}, - {"file": "input.lore", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000071", "reason": "decoded kind contains slash"}, - {"file": "input.lore", "line": 4, "class": "garbage", "reason": "duplicate decoded member name in payload"}, - {"file": "input.lore", "line": 5, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000073"}, - {"file": "input.lore", "line": 6, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000074"}, + {"file": "input.lync", "line": 1, "class": "conflict-variant", "id": "spell-a", "reason": "decoded id equals line 2, body differs"}, + {"file": "input.lync", "line": 2, "class": "conflict-variant", "id": "spell-a", "reason": "decoded id equals line 1, body differs"}, + {"file": "input.lync", "line": 3, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000071", "reason": "decoded kind contains slash"}, + {"file": "input.lync", "line": 4, "class": "garbage", "reason": "duplicate decoded member name in payload"}, + {"file": "input.lync", "line": 5, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000073"}, + {"file": "input.lync", "line": 6, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000074"}, ], "union_event_ids": ["018f0000-0000-7000-8000-000000000071", "018f0000-0000-7000-8000-000000000073", "018f0000-0000-7000-8000-000000000074"], "conflict_ids": ["spell-a"], @@ -324,14 +324,14 @@ def main(): leap = line("018f0000-0000-7000-8000-000000000082", at="2026-12-31T23:59:60Z", payload={"text": "RFC3339 leap second ABNF"}) bad_at = line("018f0000-0000-7000-8000-000000000083", at="2026-07-06 04:10:00Z", payload={"text": "space not T"}) bad_marked = line("018f0000-0000-7000-8000-000000000084", marked="not-a-time", payload={"text": "bad marked"}) - write_fixture("09-marked-at-semantics", [("input.lore", [spliced(marked_ok), spliced(leap), spliced(bad_at), spliced(bad_marked)], True)], { + write_fixture("09-marked-at-semantics", [("input.lync", [spliced(marked_ok), spliced(leap), spliced(bad_at), spliced(bad_marked)], True)], { "fixture": "09-marked-at-semantics", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000081", "marked_effective": "2026-07-07T01:02:03.123456Z"}, - {"file": "input.lore", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000082"}, - {"file": "input.lore", "line": 3, "class": "garbage", "reason": "at fails RFC3339 ABNF"}, - {"file": "input.lore", "line": 4, "class": "garbage", "reason": "marked fails RFC3339 ABNF"}, + {"file": "input.lync", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000081", "marked_effective": "2026-07-07T01:02:03.123456Z"}, + {"file": "input.lync", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000082"}, + {"file": "input.lync", "line": 3, "class": "garbage", "reason": "at fails RFC3339 ABNF"}, + {"file": "input.lync", "line": 4, "class": "garbage", "reason": "marked fails RFC3339 ABNF"}, ], "union_event_ids": ["018f0000-0000-7000-8000-000000000081", "018f0000-0000-7000-8000-000000000082"], "view_eligible_ids": ["018f0000-0000-7000-8000-000000000081", "018f0000-0000-7000-8000-000000000082"], @@ -344,18 +344,18 @@ def main(): m_conf_a = line("018f0000-0000-7000-8000-000000000094", payload={"text": "merge conflict A"}) m_conf_b = line("018f0000-0000-7000-8000-000000000094", payload={"text": "merge conflict B"}) write_fixture("10-merge-union", [ - ("a.lore", [spliced(m1), spliced(m2), spliced(m_conf_a)], True), - ("b.lore", [spliced(m2), spliced(m3), spliced(m_conf_b)], True), + ("a.lync", [spliced(m1), spliced(m2), spliced(m_conf_a)], True), + ("b.lync", [spliced(m2), spliced(m3), spliced(m_conf_b)], True), ], { "fixture": "10-merge-union", - "inputs": ["a.lore", "b.lore"], + "inputs": ["a.lync", "b.lync"], "line_classifications": [ - {"file": "a.lore", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000091"}, - {"file": "a.lore", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000092"}, - {"file": "a.lore", "line": 3, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000094"}, - {"file": "b.lore", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000092", "duplicate_sighting": True}, - {"file": "b.lore", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000093"}, - {"file": "b.lore", "line": 3, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000094"}, + {"file": "a.lync", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000091"}, + {"file": "a.lync", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000092"}, + {"file": "a.lync", "line": 3, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000094"}, + {"file": "b.lync", "line": 1, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000092", "duplicate_sighting": True}, + {"file": "b.lync", "line": 2, "class": "accepted", "id": "018f0000-0000-7000-8000-000000000093"}, + {"file": "b.lync", "line": 3, "class": "conflict-variant", "id": "018f0000-0000-7000-8000-000000000094"}, ], "union_event_ids": [ "018f0000-0000-7000-8000-000000000091", @@ -379,12 +379,12 @@ def main(): # 11: unknown top-level and author fields are nonconforming but carried. unknown_top = line("018f0000-0000-7000-8000-000000000101", payload={"text": "unknown top"}, mood="future") unknown_author = line("018f0000-0000-7000-8000-000000000102", author={"actor": "deepfates", "role": "extra"}, payload={"text": "unknown author"}) - write_fixture("11-nonconforming-carried", [("input.lore", [spliced(unknown_top), spliced(unknown_author)], True)], { + write_fixture("11-nonconforming-carried", [("input.lync", [spliced(unknown_top), spliced(unknown_author)], True)], { "fixture": "11-nonconforming-carried", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "nonconforming", "id": "018f0000-0000-7000-8000-000000000101", "reason": "unknown top-level field carried and surfaced"}, - {"file": "input.lore", "line": 2, "class": "nonconforming", "id": "018f0000-0000-7000-8000-000000000102", "reason": "unknown author field carried and surfaced"}, + {"file": "input.lync", "line": 1, "class": "nonconforming", "id": "018f0000-0000-7000-8000-000000000101", "reason": "unknown top-level field carried and surfaced"}, + {"file": "input.lync", "line": 2, "class": "nonconforming", "id": "018f0000-0000-7000-8000-000000000102", "reason": "unknown author field carried and surfaced"}, ], "union_event_ids": ["018f0000-0000-7000-8000-000000000101", "018f0000-0000-7000-8000-000000000102"], "view_eligible_ids": ["018f0000-0000-7000-8000-000000000101", "018f0000-0000-7000-8000-000000000102"], @@ -393,11 +393,11 @@ def main(): # 12: invalid sig splice grammar means no splice; body parse then finds reserved names. invalid_sig = line("018f0000-0000-7000-8000-000000000111", payload={"text": "invalid sig grammar"}) invalid_sig = invalid_sig[:-1] + ',"digest":"sha256:' + digest(invalid_sig) + '","sig":"abc-_"}' - write_fixture("12-invalid-sig-splice", [("input.lore", [invalid_sig], True)], { + write_fixture("12-invalid-sig-splice", [("input.lync", [invalid_sig], True)], { "fixture": "12-invalid-sig-splice", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "garbage", "reason": "invalid sig grammar means no splice; reserved top-level digest/sig remain in body"}, + {"file": "input.lync", "line": 1, "class": "garbage", "reason": "invalid sig grammar means no splice; reserved top-level digest/sig remain in body"}, ], "union_event_ids": [], "view_eligible_ids": [], @@ -406,11 +406,11 @@ def main(): # 13: sig metadata requires digest metadata, so sig alone is body and reserved-name garbage. sig_without_digest = line("018f0000-0000-7000-8000-000000000121", payload={"text": "sig without digest"}) sig_without_digest = sig_without_digest[:-1] + ',"sig":"QUJDRA=="}' - write_fixture("13-sig-without-digest", [("input.lore", [sig_without_digest], True)], { + write_fixture("13-sig-without-digest", [("input.lync", [sig_without_digest], True)], { "fixture": "13-sig-without-digest", - "inputs": ["input.lore"], + "inputs": ["input.lync"], "line_classifications": [ - {"file": "input.lore", "line": 1, "class": "garbage", "reason": "sig without digest is not a valid splice; reserved top-level sig remains in body"}, + {"file": "input.lync", "line": 1, "class": "garbage", "reason": "sig without digest is not a valid splice; reserved top-level sig remains in body"}, ], "union_event_ids": [], "view_eligible_ids": [], diff --git a/packages/core/test/lore-views.test.ts b/packages/core/test/views.test.ts similarity index 86% rename from packages/core/test/lore-views.test.ts rename to packages/core/test/views.test.ts index 5d38647..9af4a70 100644 --- a/packages/core/test/lore-views.test.ts +++ b/packages/core/test/views.test.ts @@ -2,15 +2,15 @@ import { readFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; import { describe, expect, it } from "vitest"; -import { parseLoreFiles } from "../src/lore/events.js"; +import { parseLyncFiles } from "../src/events.js"; import { - loreBranchTreeView, - loreLeaderboardView, - loreMemoryView, - loreTranscriptView, -} from "../src/lore/views.js"; + lyncBranchTreeView, + lyncLeaderboardView, + lyncMemoryView, + lyncTranscriptView, +} from "../src/views.js"; -const vectorsRoot = join(dirname(fileURLToPath(import.meta.url)), "vectors", "lore-vectors-draft"); +const vectorsRoot = join(dirname(fileURLToPath(import.meta.url)), "vectors", "v0"); interface ExpectedFixture { inputs: string[]; @@ -19,7 +19,7 @@ interface ExpectedFixture { function loadFixture(name: string) { const dir = join(vectorsRoot, name); const expected = JSON.parse(readFileSync(join(dir, "expected.json"), "utf8")) as ExpectedFixture; - return parseLoreFiles( + return parseLyncFiles( expected.inputs.map((file) => ({ file, bytes: readFileSync(join(dir, file)), @@ -38,7 +38,7 @@ function event(fields: { return JSON.stringify({ v: 1, id: fields.id, - kind: fields.kind ?? "lore/artifact", + kind: fields.kind ?? "lync/artifact", at: fields.at ?? "2026-07-06T04:10:00Z", author: fields.author ?? { actor: "deepfates" }, parents: fields.parents ?? [], @@ -49,7 +49,7 @@ function event(fields: { describe("LORE views", () => { it("computes a branch tree DAG from vector parent links", () => { const result = loadFixture("01-valid-events"); - const tree = loreBranchTreeView(result); + const tree = lyncBranchTreeView(result); expect(tree.roots).toEqual([ "018f0000-0000-7000-8000-000000000001", @@ -70,8 +70,8 @@ describe("LORE views", () => { it("surfaces graph obstacles in branch and memory views from vector 06", () => { const result = loadFixture("06-graph-obstacles"); - const tree = loreBranchTreeView(result); - const memory = loreMemoryView(result); + const tree = lyncBranchTreeView(result); + const memory = lyncMemoryView(result); expect(tree.partial).toBe(true); expect(tree.obstacles).toEqual(result.graphDiagnostics); @@ -87,7 +87,7 @@ describe("LORE views", () => { it("computes a linear transcript over the chosen head downset", () => { const result = loadFixture("10-merge-union"); - const transcript = loreTranscriptView(result, "018f0000-0000-7000-8000-000000000093"); + const transcript = lyncTranscriptView(result, "018f0000-0000-7000-8000-000000000093"); expect(transcript.entries.map((entry) => entry.id)).toEqual([ "018f0000-0000-7000-8000-000000000091", @@ -104,7 +104,7 @@ describe("LORE views", () => { it("computes memory as current eligible events plus leaf frontier", () => { const result = loadFixture("10-merge-union"); - const memory = loreMemoryView(result); + const memory = lyncMemoryView(result); expect(memory.events.map((entry) => entry.id)).toEqual([ "018f0000-0000-7000-8000-000000000091", @@ -115,35 +115,35 @@ describe("LORE views", () => { expect(memory.conflictIds).toEqual(["018f0000-0000-7000-8000-000000000094"]); }); - it("ranks scored and selected drafts from lore annotation events", () => { + it("ranks scored and selected drafts from lync annotation events", () => { const input = [ event({ id: "A", payload: { text: "root" } }), event({ id: "B", parents: ["A"], payload: { text: "patient bear" } }), event({ id: "C", parents: ["A"], payload: { text: "younger bears" } }), event({ id: "D", - kind: "lore/annotation", + kind: "lync/annotation", author: { actor: "witness-panel-v3" }, parents: ["B"], payload: { label: "score", value: 0.91, basis: "panel" }, }), event({ id: "E", - kind: "lore/annotation", + kind: "lync/annotation", author: { actor: "deepfates" }, parents: ["B", "C"], payload: { label: "selection", chosen: ["B"], shown: ["B", "C"], basis: "human pick" }, }), event({ id: "F", - kind: "lore/annotation", + kind: "lync/annotation", author: { actor: "witness-panel-v3" }, parents: ["C"], payload: { label: "score", value: 0.2 }, }), ].join("\n") + "\n"; - const result = parseLoreFiles([{ file: "worked.lore", bytes: input }]); - const leaderboard = loreLeaderboardView(result); + const result = parseLyncFiles([{ file: "worked.lync", bytes: input }]); + const leaderboard = lyncLeaderboardView(result); expect(leaderboard.ignoredAnnotationIds).toEqual([]); expect(leaderboard.entries.map((entry) => ({ diff --git a/packages/index/package.json b/packages/index/package.json deleted file mode 100644 index 97f05a2..0000000 --- a/packages/index/package.json +++ /dev/null @@ -1,47 +0,0 @@ -{ - "name": "lync-index", - "version": "0.2.0", - "description": "Index APIs for linked Lync looms.", - "type": "module", - "license": "MIT", - "sideEffects": false, - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - }, - "./automerge": { - "types": "./dist/automerge.d.ts", - "import": "./dist/automerge.js", - "default": "./dist/automerge.js" - }, - "./entries": { - "types": "./dist/entries.d.ts", - "import": "./dist/entries.js", - "default": "./dist/entries.js" - }, - "./memory": { - "types": "./dist/memory.d.ts", - "import": "./dist/memory.js", - "default": "./dist/memory.js" - }, - "./types": { - "types": "./dist/types.d.ts", - "import": "./dist/types.js", - "default": "./dist/types.js" - } - }, - "files": [ - "dist" - ], - "scripts": { - "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit" - }, - "dependencies": { - "lync-core": "workspace:*" - } -} diff --git a/packages/index/src/automerge.ts b/packages/index/src/automerge.ts deleted file mode 100644 index c814443..0000000 --- a/packages/index/src/automerge.ts +++ /dev/null @@ -1,280 +0,0 @@ -import { DocHandle, Repo, type AutomergeUrl } from "@automerge/automerge-repo"; -import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "lync-core"; -import type { IndexId, LoomId, LoomReference } from "lync-core"; -import type { - LoomIndex, - LoomIndexEntry, - LoomIndexEntryInput, - LoomIndexEntryPatch, - LoomIndexes, - LoomIndexEvent, - LoomIndexInfo, - LoomIndexListener, - LoomIndexSnapshot, -} from "./types.js"; - -type IndexDoc = { - version: 1; - index: LoomIndexInfo; - entries: Record>; - order: LoomId[]; -}; - -export interface AutomergeLoomIndexesOptions { - repo?: Repo; - now?: () => number; -} - -export function createAutomergeLoomIndexes< - TEntryMeta = unknown, - TIndexMeta = unknown, ->(options: AutomergeLoomIndexesOptions = {}): LoomIndexes { - const repo = options.repo ?? new Repo(); - const now = options.now ?? (() => Date.now()); - - return { - async create(meta) { - assertJsonEncodable(meta, "index meta"); - const handle = repo.create>({ - version: 1, - index: { - id: "" as IndexId, - ...(meta === undefined ? {} : { meta: cloneJson(meta) }), - createdAt: now(), - }, - entries: {}, - order: [], - }); - handle.change((doc) => { - doc.index.id = handle.url; - }); - return new AutomergeLoomIndex(handle.url, handle, now); - }, - - async open(indexId) { - let handle: DocHandle>; - try { - handle = await repo.find>( - indexId as AutomergeUrl, - ); - await handle.whenReady(); - } catch { - throw unknownIndex(indexId); - } - return new AutomergeLoomIndex(indexId, handle, now); - }, - - async import(snapshot) { - validateSnapshot(snapshot); - const handle = repo.create>({ - version: 1, - index: { - id: "" as IndexId, - ...(snapshot.index.meta === undefined - ? {} - : { meta: cloneJson(snapshot.index.meta) }), - createdAt: snapshot.index.createdAt, - }, - entries: {}, - order: [], - }); - handle.change((doc) => { - doc.index.id = handle.url; - for (const entry of snapshot.entries) { - doc.entries[entry.ref.loomId] = cloneJson(entry); - doc.order.push(entry.ref.loomId); - } - }); - return new AutomergeLoomIndex(handle.url, handle, now); - }, - }; -} - -class AutomergeLoomIndex - implements LoomIndex -{ - private closed = false; - private listeners = new Set>(); - private knownLoomIds: Set; - - constructor( - readonly id: IndexId, - private readonly handle: DocHandle>, - private readonly now: () => number, - ) { - this.knownLoomIds = new Set(Object.keys(this.handle.doc().entries ?? {})); - this.handle.on("change", ({ doc }) => { - const current = new Set(Object.keys(doc.entries ?? {})); - for (const loomId of current) { - if (!this.knownLoomIds.has(loomId)) { - const entry = doc.entries[loomId]; - if (entry) this.emit({ type: "entry-added", indexId: this.id, entry: cloneJson(entry) }); - } - } - for (const loomId of this.knownLoomIds) { - if (!current.has(loomId)) this.emit({ type: "entry-removed", indexId: this.id, loomId }); - } - this.knownLoomIds = current; - }); - } - - async info(): Promise> { - this.assertOpen(); - return cloneJson(this.doc().index); - } - - async updateMeta(meta: TIndexMeta): Promise> { - this.assertOpen(); - assertJsonEncodable(meta, "index meta"); - this.handle.change((doc) => { - doc.index.meta = cloneJson(meta) as TIndexMeta; - }); - const index = cloneJson(this.doc().index); - this.emit({ type: "index-updated", index }); - return index; - } - - async entries(): Promise[]> { - this.assertOpen(); - const doc = this.doc(); - return doc.order.map((loomId) => { - const entry = doc.entries[loomId]; - if (!entry) throw new LoomError("BROKEN_TOPOLOGY", `Index order references missing loom: ${loomId}`); - return cloneJson(entry); - }); - } - - async get(loomId: LoomId): Promise | null> { - this.assertOpen(); - const entry = this.doc().entries[loomId]; - return entry ? cloneJson(entry) : null; - } - - async has(loomId: LoomId): Promise { - this.assertOpen(); - return Boolean(this.doc().entries[loomId]); - } - - async addLoom( - ref: Extract, - input: LoomIndexEntryInput = {}, - ): Promise> { - this.assertOpen(); - assertJsonEncodable(input, "index entry"); - if (this.doc().entries[ref.loomId]) { - throw duplicateLoomId(ref.loomId); - } - const entry = omitUndefined({ - ref: loomRef(ref.loomId), - title: input.title, - kind: input.kind, - meta: cloneJson(input.meta), - addedAt: this.now(), - updatedAt: input.updatedAt, - }) as LoomIndexEntry; - this.handle.change((doc) => { - doc.entries[ref.loomId] = entry; - doc.order.push(ref.loomId); - }); - this.emit({ type: "entry-added", indexId: this.id, entry }); - return cloneJson(entry); - } - - async updateLoom( - loomId: LoomId, - patch: LoomIndexEntryPatch, - ): Promise> { - this.assertOpen(); - assertJsonEncodable(patch, "index entry patch"); - const existing = this.doc().entries[loomId]; - if (!existing) throw new LoomError("UNKNOWN_LOOM", `Index does not contain loom: ${loomId}`); - const updated = omitUndefined({ - ...existing, - ...patch, - meta: patch.meta === undefined ? existing.meta : cloneJson(patch.meta), - updatedAt: patch.updatedAt ?? this.now(), - }) as LoomIndexEntry; - this.handle.change((doc) => { - doc.entries[loomId] = updated; - }); - const output = cloneJson(updated); - this.emit({ type: "entry-updated", indexId: this.id, entry: output }); - return output; - } - - async removeLoom(loomId: LoomId): Promise { - this.assertOpen(); - if (!this.doc().entries[loomId]) return; - this.handle.change((doc) => { - delete doc.entries[loomId]; - const index = doc.order.indexOf(loomId); - if (index >= 0) doc.order.splice(index, 1); - }); - this.emit({ type: "entry-removed", indexId: this.id, loomId }); - } - - subscribe(listener: LoomIndexListener): () => void { - this.assertOpen(); - this.listeners.add(listener); - return () => this.listeners.delete(listener); - } - - async export(): Promise> { - this.assertOpen(); - return cloneJson({ - index: this.doc().index, - entries: await this.entries(), - }); - } - - close(): void { - this.closed = true; - this.listeners.clear(); - } - - private doc() { - return this.handle.doc(); - } - - private assertOpen(): void { - if (this.closed) throw new LoomError("CLOSED_HANDLE", "This loom index handle is closed"); - } - - private emit(event: LoomIndexEvent): void { - for (const listener of this.listeners) listener(event); - } -} - -function validateSnapshot(snapshot: LoomIndexSnapshot): void { - assertJsonEncodable(snapshot, "index snapshot"); - const seen = new Set(); - for (const entry of snapshot.entries) { - if (!entry?.ref || entry.ref.kind !== "loom" || typeof entry.ref.loomId !== "string") { - throw new LoomError("INVALID_SNAPSHOT", "Every index entry needs a loom reference"); - } - if (seen.has(entry.ref.loomId)) { - throw duplicateLoomId(entry.ref.loomId); - } - seen.add(entry.ref.loomId); - } -} - -function assertJsonEncodable(value: unknown, label: string): void { - if (value === undefined) return; - try { - JSON.stringify(value); - } catch { - throw new LoomError("INVALID_SNAPSHOT", `${label} must be JSON-encodable`); - } -} - -function cloneJson(value: T): T { - if (value === undefined) return value; - return JSON.parse(JSON.stringify(value)) as T; -} - -function omitUndefined>(value: T): T { - return Object.fromEntries( - Object.entries(value).filter(([, entryValue]) => entryValue !== undefined), - ) as T; -} diff --git a/packages/index/src/entries.ts b/packages/index/src/entries.ts deleted file mode 100644 index f7405cb..0000000 --- a/packages/index/src/entries.ts +++ /dev/null @@ -1,17 +0,0 @@ -import type { LoomReference } from "lync-core"; -import type { - LoomIndex, - LoomIndexEntry, - LoomIndexEntryInput, -} from "./types.js"; - -export async function upsertLoom( - index: LoomIndex, - ref: Extract, - entry: LoomIndexEntryInput = {}, -): Promise> { - if (await index.has(ref.loomId)) { - return index.updateLoom(ref.loomId, entry); - } - return index.addLoom(ref, entry); -} diff --git a/packages/index/src/index.ts b/packages/index/src/index.ts deleted file mode 100644 index c82e54d..0000000 --- a/packages/index/src/index.ts +++ /dev/null @@ -1,2 +0,0 @@ -export * from "./entries.js"; -export * from "./types.js"; diff --git a/packages/index/src/memory.ts b/packages/index/src/memory.ts deleted file mode 100644 index a4c0dc5..0000000 --- a/packages/index/src/memory.ts +++ /dev/null @@ -1,239 +0,0 @@ -import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "lync-core"; -import type { IndexId, LoomId, LoomReference } from "lync-core"; -import type { - LoomIndex, - LoomIndexEntry, - LoomIndexEntryInput, - LoomIndexEntryPatch, - LoomIndexes, - LoomIndexEvent, - LoomIndexInfo, - LoomIndexListener, - LoomIndexSnapshot, - MemoryLoomIndexesOptions, -} from "./types.js"; - -export type { MemoryLoomIndexesOptions } from "./types.js"; - -type InternalIndex = { - info: LoomIndexInfo; - entries: Map>; - order: LoomId[]; - listeners: Set>; -}; - -export function createMemoryLoomIndexes< - TEntryMeta = unknown, - TIndexMeta = unknown, ->(options: MemoryLoomIndexesOptions = {}): LoomIndexes { - const createId = options.createId ?? (() => crypto.randomUUID()); - const now = options.now ?? (() => Date.now()); - const indexes = new Map>(); - - const createInternal = (meta?: TIndexMeta): InternalIndex => { - assertJsonEncodable(meta, "index meta"); - return { - info: omitUndefined({ - id: `memory-index:${createId()}`, - meta: cloneJson(meta), - createdAt: now(), - }), - entries: new Map(), - order: [], - listeners: new Set(), - }; - }; - - return { - async create(meta) { - const index = createInternal(meta); - indexes.set(index.info.id, index); - return new MemoryLoomIndex(index.info.id, index, now); - }, - - async open(indexId) { - const index = indexes.get(indexId); - if (!index) throw unknownIndex(indexId); - return new MemoryLoomIndex(indexId, index, now); - }, - - async import(snapshot) { - validateSnapshot(snapshot); - const index = createInternal(snapshot.index.meta); - index.info.createdAt = snapshot.index.createdAt; - for (const entry of snapshot.entries) { - const cloned = cloneJson(entry); - index.entries.set(cloned.ref.loomId, cloned); - index.order.push(cloned.ref.loomId); - } - indexes.set(index.info.id, index); - return new MemoryLoomIndex(index.info.id, index, now); - }, - }; -} - -class MemoryLoomIndex - implements LoomIndex -{ - private closed = false; - - constructor( - readonly id: IndexId, - private readonly index: InternalIndex, - private readonly now: () => number, - ) {} - - async info(): Promise> { - this.assertOpen(); - return cloneJson(this.index.info); - } - - async updateMeta(meta: TIndexMeta): Promise> { - this.assertOpen(); - assertJsonEncodable(meta, "index meta"); - this.index.info = omitUndefined({ ...this.index.info, meta: cloneJson(meta) }); - this.emit({ type: "index-updated", index: cloneJson(this.index.info) }); - return cloneJson(this.index.info); - } - - async entries(): Promise[]> { - this.assertOpen(); - return this.index.order.map((loomId) => { - const entry = this.index.entries.get(loomId); - if (!entry) throw new LoomError("BROKEN_TOPOLOGY", `Index order references missing loom: ${loomId}`); - return cloneJson(entry); - }); - } - - async get(loomId: LoomId): Promise | null> { - this.assertOpen(); - const entry = this.index.entries.get(loomId); - return entry ? cloneJson(entry) : null; - } - - async has(loomId: LoomId): Promise { - this.assertOpen(); - return this.index.entries.has(loomId); - } - - async addLoom( - ref: Extract, - input: LoomIndexEntryInput = {}, - ): Promise> { - this.assertOpen(); - assertJsonEncodable(input, "index entry"); - if (this.index.entries.has(ref.loomId)) { - throw duplicateLoomId(ref.loomId); - } - const entry = omitUndefined({ - ref: loomRef(ref.loomId), - title: input.title, - kind: input.kind, - meta: cloneJson(input.meta), - addedAt: this.now(), - updatedAt: input.updatedAt, - }) as LoomIndexEntry; - this.index.entries.set(ref.loomId, entry); - this.index.order.push(ref.loomId); - const output = cloneJson(entry); - this.emit({ type: "entry-added", indexId: this.id, entry: output }); - return output; - } - - async updateLoom( - loomId: LoomId, - patch: LoomIndexEntryPatch, - ): Promise> { - this.assertOpen(); - assertJsonEncodable(patch, "index entry patch"); - const existing = this.index.entries.get(loomId); - if (!existing) throw new LoomError("UNKNOWN_LOOM", `Index does not contain loom: ${loomId}`); - const updated = omitUndefined({ - ...existing, - ...patch, - meta: patch.meta === undefined ? existing.meta : cloneJson(patch.meta), - updatedAt: patch.updatedAt ?? this.now(), - }) as LoomIndexEntry; - this.index.entries.set(loomId, updated); - const output = cloneJson(updated); - this.emit({ type: "entry-updated", indexId: this.id, entry: output }); - return output; - } - - async removeLoom(loomId: LoomId): Promise { - this.assertOpen(); - if (!this.index.entries.has(loomId)) return; - this.index.entries.delete(loomId); - this.index.order = this.index.order.filter((candidate) => candidate !== loomId); - this.emit({ type: "entry-removed", indexId: this.id, loomId }); - } - - subscribe(listener: LoomIndexListener): () => void { - this.assertOpen(); - this.index.listeners.add(listener); - return () => this.index.listeners.delete(listener); - } - - async export(): Promise> { - this.assertOpen(); - return cloneJson({ - index: this.index.info, - entries: await this.entries(), - }); - } - - close(): void { - this.closed = true; - } - - private assertOpen() { - if (this.closed) throw new LoomError("CLOSED_HANDLE", "This loom index handle is closed"); - } - - private emit(event: LoomIndexEvent): void { - for (const listener of this.index.listeners) listener(event); - } -} - -function validateSnapshot(snapshot: LoomIndexSnapshot): void { - if (!snapshot || typeof snapshot !== "object") { - throw new LoomError("INVALID_SNAPSHOT", "Index snapshot must be an object"); - } - if (!snapshot.index || typeof snapshot.index.id !== "string") { - throw new LoomError("INVALID_SNAPSHOT", "Index snapshot needs an index id"); - } - if (!Array.isArray(snapshot.entries)) { - throw new LoomError("INVALID_SNAPSHOT", "Index snapshot entries must be an array"); - } - assertJsonEncodable(snapshot, "index snapshot"); - const seen = new Set(); - for (const entry of snapshot.entries) { - if (!entry?.ref || entry.ref.kind !== "loom" || typeof entry.ref.loomId !== "string") { - throw new LoomError("INVALID_SNAPSHOT", "Every index entry needs a loom reference"); - } - if (seen.has(entry.ref.loomId)) { - throw duplicateLoomId(entry.ref.loomId); - } - seen.add(entry.ref.loomId); - } -} - -function omitUndefined>(value: T): T { - return Object.fromEntries( - Object.entries(value).filter(([, entryValue]) => entryValue !== undefined), - ) as T; -} - -function assertJsonEncodable(value: unknown, label: string): void { - if (value === undefined) return; - try { - JSON.stringify(value); - } catch { - throw new LoomError("INVALID_SNAPSHOT", `${label} must be JSON-encodable`); - } -} - -function cloneJson(value: T): T { - if (value === undefined) return value; - return JSON.parse(JSON.stringify(value)) as T; -} diff --git a/packages/index/src/types.ts b/packages/index/src/types.ts deleted file mode 100644 index a65448f..0000000 --- a/packages/index/src/types.ts +++ /dev/null @@ -1,78 +0,0 @@ -import type { IndexId, LoomId, LoomReference } from "lync-core"; - -export interface LoomIndexInfo { - id: IndexId; - meta?: TIndexMeta; - createdAt: number; -} - -export interface LoomIndexEntry { - ref: Extract; - title?: string; - kind?: string; - meta?: TEntryMeta; - addedAt: number; - updatedAt?: number; -} - -export type LoomIndexEntryInput = Partial< - Omit, "ref" | "addedAt"> ->; - -export type LoomIndexEntryPatch = Partial< - Pick, "title" | "kind" | "meta" | "updatedAt"> ->; - -export interface LoomIndexSnapshot { - index: LoomIndexInfo; - entries: LoomIndexEntry[]; -} - -export type LoomIndexEvent = - | { type: "entry-added"; indexId: IndexId; entry: LoomIndexEntry } - | { type: "entry-updated"; indexId: IndexId; entry: LoomIndexEntry } - | { type: "entry-removed"; indexId: IndexId; loomId: LoomId } - | { type: "index-updated"; index: LoomIndexInfo } - | { type: "sync-state"; indexId: IndexId; online: boolean; syncing: boolean }; - -export type LoomIndexListener = ( - event: LoomIndexEvent, -) => void; - -export interface LoomIndex { - id: IndexId; - - info(): Promise>; - updateMeta(meta: TIndexMeta): Promise>; - - entries(): Promise[]>; - get(loomId: LoomId): Promise | null>; - has(loomId: LoomId): Promise; - - addLoom( - ref: Extract, - entry?: LoomIndexEntryInput, - ): Promise>; - updateLoom( - loomId: LoomId, - patch: LoomIndexEntryPatch, - ): Promise>; - removeLoom(loomId: LoomId): Promise; - - subscribe(listener: LoomIndexListener): () => void; - export(): Promise>; - close(): void; -} - -export interface LoomIndexes { - create(meta?: TIndexMeta): Promise>; - open(indexId: IndexId): Promise>; - import( - snapshot: LoomIndexSnapshot, - ): Promise>; -} - -export interface MemoryLoomIndexesOptions { - createId?: () => string; - now?: () => number; -} diff --git a/packages/index/test/automerge.test.ts b/packages/index/test/automerge.test.ts deleted file mode 100644 index b8f5bbb..0000000 --- a/packages/index/test/automerge.test.ts +++ /dev/null @@ -1,64 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { Repo } from "@automerge/automerge-repo"; -import { loomRef } from "lync-core"; -import { createAutomergeLoomIndexes } from "../src/automerge.js"; - -function deterministicAutomergeIndexes() { - let nextTime = 4000; - return createAutomergeLoomIndexes<{ app: string }, { owner: string }>({ - repo: new Repo(), - now: () => nextTime++, - }); -} - -describe("automerge loom indexes", () => { - it("uses the Automerge document URL as the index id", async () => { - const indexes = deterministicAutomergeIndexes(); - const index = await indexes.create({ owner: "me" }); - - expect(index.id.startsWith("automerge:")).toBe(true); - await expect(index.info()).resolves.toMatchObject({ id: index.id }); - }); - - it("stores ordered loom references", async () => { - const indexes = deterministicAutomergeIndexes(); - const index = await indexes.create(); - - const first = await index.addLoom(loomRef("automerge:first"), { - title: "First", - kind: "story", - meta: { app: "textile" }, - }); - const second = await index.addLoom(loomRef("automerge:second"), { title: "Second" }); - - expect(await index.entries()).toEqual([first, second]); - expect(await index.get("automerge:first")).toEqual(first); - }); - - it("imports snapshots with a new index id", async () => { - const indexes = deterministicAutomergeIndexes(); - const index = await indexes.create({ owner: "me" }); - await index.addLoom(loomRef("automerge:first"), { title: "First" }); - const snapshot = await index.export(); - - const imported = await indexes.import(snapshot); - - expect(imported.id).not.toBe(index.id); - expect(await imported.entries()).toEqual(snapshot.entries); - }); - - it("emits entry-added for changes observed through another handle", async () => { - const indexes = deterministicAutomergeIndexes(); - const index = await indexes.create(); - const observer = await indexes.open(index.id); - const writer = await indexes.open(index.id); - const events: string[] = []; - observer.subscribe((event) => { - if (event.type === "entry-added") events.push(event.entry.ref.loomId); - }); - - await writer.addLoom(loomRef("automerge:first"), { title: "First" }); - - expect(events).toEqual(["automerge:first"]); - }); -}); diff --git a/packages/index/test/memory.test.ts b/packages/index/test/memory.test.ts deleted file mode 100644 index c09a2be..0000000 --- a/packages/index/test/memory.test.ts +++ /dev/null @@ -1,97 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { loomRef } from "lync-core"; -import { createMemoryLoomIndexes } from "../src/memory.js"; -import { upsertLoom } from "../src/entries.js"; - -function deterministicIndexes() { - let nextId = 0; - let nextTime = 2000; - return createMemoryLoomIndexes<{ app: string }, { owner: string }>({ - createId: () => `idx-${++nextId}`, - now: () => nextTime++, - }); -} - -describe("memory loom indexes", () => { - it("creates an index and stores ordered loom references", async () => { - const indexes = deterministicIndexes(); - const index = await indexes.create({ owner: "me" }); - - const first = await index.addLoom(loomRef("automerge:first"), { - title: "First", - kind: "story", - meta: { app: "textile" }, - }); - const second = await index.addLoom(loomRef("automerge:second"), { title: "Second" }); - - expect(await index.entries()).toEqual([first, second]); - expect(await index.get("automerge:first")).toEqual(first); - expect(await index.has("automerge:missing")).toBe(false); - }); - - it("updates and removes loom links without implying loom deletion", async () => { - const indexes = deterministicIndexes(); - const index = await indexes.create(); - await index.addLoom(loomRef("automerge:first"), { title: "First" }); - - const updated = await index.updateLoom("automerge:first", { - title: "Renamed", - kind: "story", - }); - expect(updated.title).toBe("Renamed"); - expect(updated.updatedAt).toBe(2002); - - await index.removeLoom("automerge:first"); - expect(await index.entries()).toEqual([]); - }); - - it("emits entry events and exports/imports deterministic snapshots with a new index id", async () => { - const indexes = deterministicIndexes(); - const index = await indexes.create({ owner: "me" }); - const events: string[] = []; - index.subscribe((event) => events.push(event.type)); - - await index.addLoom(loomRef("automerge:first"), { title: "First" }); - await index.updateLoom("automerge:first", { title: "Renamed" }); - await index.removeLoom("automerge:first"); - - expect(events).toEqual(["entry-added", "entry-updated", "entry-removed"]); - - await index.addLoom(loomRef("automerge:first"), { title: "First" }); - const snapshot = await index.export(); - const imported = await indexes.import(snapshot); - - expect(imported.id).not.toBe(index.id); - expect(await imported.entries()).toEqual(snapshot.entries); - }); - - it("rejects duplicate loom links", async () => { - const indexes = deterministicIndexes(); - const index = await indexes.create(); - await index.addLoom(loomRef("automerge:first")); - - await expect(index.addLoom(loomRef("automerge:first"))).rejects.toMatchObject({ - code: "DUPLICATE_LOOM_ID", - }); - }); - - it("upserts loom links so shared imports can refresh metadata", async () => { - const indexes = deterministicIndexes(); - const index = await indexes.create(); - - const added = await upsertLoom(index, loomRef("automerge:first"), { - title: "First", - kind: "story", - meta: { app: "old" }, - }); - const updated = await upsertLoom(index, loomRef("automerge:first"), { - title: "Renamed", - kind: "story", - meta: { app: "new" }, - }); - - expect(added.addedAt).toBe(updated.addedAt); - expect(updated.updatedAt).toBe(2002); - expect(await index.entries()).toEqual([updated]); - }); -}); diff --git a/packages/index/tsconfig.json b/packages/index/tsconfig.json deleted file mode 100644 index df59da5..0000000 --- a/packages/index/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*.ts"] -} diff --git a/packages/sync-server/package.json b/packages/sync-server/package.json deleted file mode 100644 index eadbb72..0000000 --- a/packages/sync-server/package.json +++ /dev/null @@ -1,29 +0,0 @@ -{ - "name": "lync-sync-server", - "version": "0.1.0", - "description": "Automerge WebSocket sync relay for Lync.", - "type": "module", - "license": "MIT", - "sideEffects": false, - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - } - }, - "files": [ - "dist" - ], - "scripts": { - "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit" - }, - "dependencies": { - "@automerge/automerge-repo": "^2.5.5", - "@automerge/automerge-repo-network-websocket": "^2.5.5", - "isomorphic-ws": "^5.0.0" - } -} diff --git a/packages/sync-server/src/index.ts b/packages/sync-server/src/index.ts deleted file mode 100644 index 76027cf..0000000 --- a/packages/sync-server/src/index.ts +++ /dev/null @@ -1,314 +0,0 @@ -import fs from "node:fs/promises"; -import http from "node:http"; -import path from "node:path"; -import type { Duplex } from "node:stream"; -import { Repo, type RepoConfig } from "@automerge/automerge-repo"; -import type { - Chunk, - StorageAdapterInterface, - StorageKey, -} from "@automerge/automerge-repo"; -import { WebSocketServerAdapter } from "@automerge/automerge-repo-network-websocket"; -import WebSocket from "isomorphic-ws"; - -const { WebSocketServer } = WebSocket; -type LyncWebSocketServer = InstanceType; - -export type LyncUpgradeAuthenticator = (request: http.IncomingMessage) => boolean; - -export interface LyncServerOptions { - port?: number; - host?: string; - path?: string; - storageDir?: string; - keepAliveInterval?: number; - maxConnections?: number; - authenticate?: LyncUpgradeAuthenticator; - repoConfig?: Omit; -} - -export interface LyncServer { - repo: Repo; - server: LyncWebSocketServer; - url: string; - close(): Promise; -} - -export function createLyncServer(options: LyncServerOptions = {}): LyncServer { - const port = options.port ?? 0; - const host = options.host ?? "127.0.0.1"; - const socketPath = normalizeSyncPath(options.path ?? "/lync"); - const httpServer = http.createServer(); - const relay = attachLyncServer(httpServer, options); - httpServer.listen(port, host); - - return { - repo: relay.repo, - server: relay.server, - get url() { - const address = httpServer.address(); - if (typeof address === "string" || address === null) { - return `ws://${formatWebSocketHost(host)}:${port}${socketPath}`; - } - return `ws://${formatWebSocketHost(address.address)}:${address.port}${socketPath}`; - }, - async close() { - await relay.close(); - httpServer.closeAllConnections?.(); - await withTimeout( - new Promise((resolve, reject) => { - httpServer.close((error?: Error) => { - if ((error as NodeJS.ErrnoException | undefined)?.code === "ERR_SERVER_NOT_RUNNING") { - resolve(); - } else if (error) reject(error); - else resolve(); - }); - }), - 2_000, - ); - }, - }; -} - -export interface AttachLyncServerOptions extends Omit { - repo?: Repo; -} - -export function attachLyncServer( - server: http.Server, - options: AttachLyncServerOptions = {}, -) { - const socketPath = normalizeSyncPath(options.path ?? "/lync"); - const socketServer = new WebSocketServer({ - noServer: true, - }); - const repo = options.repo ?? createRelayRepo(socketServer, options); - const upgradeSockets = new Set(); - let closePromise: Promise | null = null; - let closing = false; - const closeOnce = () => { - closing = true; - closePromise ??= closeRelay(repo, socketServer, upgradeSockets); - return closePromise; - }; - const onUpgrade = ( - request: http.IncomingMessage, - socket: Duplex, - head: Buffer, - ) => { - if (!isSocketPath(request, socketPath)) return; - console.log("[Lync] upgrade received by relay"); - if (closing) { - console.log("[Lync] rejecting upgrade: server closing"); - rejectUpgrade(socket, "503 Service Unavailable"); - return; - } - if (!isAuthorized(options.authenticate, request)) { - console.log("[Lync] rejecting upgrade: unauthorized"); - rejectUpgrade(socket, "401 Unauthorized"); - return; - } - if ( - options.maxConnections !== undefined && - socketServer.clients.size >= options.maxConnections - ) { - console.log("[Lync] rejecting upgrade: max connections"); - rejectUpgrade(socket, "503 Service Unavailable"); - return; - } - upgradeSockets.add(socket); - socket.once("close", () => upgradeSockets.delete(socket)); - socketServer.handleUpgrade(request, socket, head, (websocket) => { - console.log("[Lync] upgrade accepted by relay"); - socketServer.emit("connection", websocket, request); - }); - }; - - server.on("upgrade", onUpgrade); - - server.on("close", () => { - void closeOnce(); - }); - - return { - repo, - server: socketServer, - close: async () => { - server.off("upgrade", onUpgrade); - await closeOnce(); - }, - }; -} - -function isSocketPath(request: http.IncomingMessage, socketPath: string) { - try { - const url = new URL(request.url ?? "/", "http://localhost"); - return url.pathname === socketPath; - } catch { - return false; - } -} - -function isAuthorized( - authenticate: LyncUpgradeAuthenticator | undefined, - request: http.IncomingMessage, -) { - if (!authenticate) return true; - try { - return authenticate(request); - } catch { - return false; - } -} - -function rejectUpgrade(socket: Duplex, status: string) { - socket.write( - `HTTP/1.1 ${status}\r\nConnection: close\r\nContent-Length: 0\r\n\r\n`, - () => socket.end(), - ); -} - -async function closeRelay( - repo: Repo, - socketServer: LyncWebSocketServer, - upgradeSockets: Set, -) { - for (const client of socketServer.clients) { - client.close(1001, "server shutting down"); - setTimeout(() => { - if (client.readyState !== WebSocket.CLOSED) client.terminate(); - }, 1_000).unref?.(); - } - setTimeout(() => { - for (const socket of upgradeSockets) socket.destroy(); - }, 1_500).unref?.(); - - await shutdownRepo(repo); - await withTimeout( - new Promise((resolve, reject) => { - socketServer.close((error?: Error) => { - if (error) reject(error); - else resolve(); - }); - }), - 2_000, - ); -} - -async function shutdownRepo(repo: Repo) { - try { - await repo.shutdown(); - } catch (error) { - console.warn("[Lync] repo shutdown failed; continuing shutdown", error); - } -} - -function withTimeout(promise: Promise, ms: number): Promise { - return new Promise((resolve, reject) => { - const timeout = setTimeout(resolve, ms); - timeout.unref?.(); - promise.then( - (value) => { - clearTimeout(timeout); - resolve(value); - }, - (error) => { - clearTimeout(timeout); - reject(error); - }, - ); - }); -} - -function createRelayRepo( - server: LyncWebSocketServer, - options: Pick, -) { - const adapter = new WebSocketServerAdapter(server, options.keepAliveInterval); - return new Repo({ - ...options.repoConfig, - storage: options.storageDir - ? new FileStorageAdapter(options.storageDir) - : options.repoConfig?.storage, - network: [adapter], - }); -} - -function normalizeSyncPath(path: string) { - return path.startsWith("/") ? path : `/${path}`; -} - -function formatWebSocketHost(host: string) { - return host.includes(":") && !host.startsWith("[") ? `[${host}]` : host; -} - -export class FileStorageAdapter implements StorageAdapterInterface { - constructor(private readonly dir: string) {} - - async load(key: StorageKey): Promise { - try { - return toUint8Array(await fs.readFile(this.filePath(key))); - } catch (error) { - if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined; - throw error; - } - } - - async save(key: StorageKey, data: Uint8Array): Promise { - await fs.mkdir(this.dir, { recursive: true }); - await fs.writeFile(this.filePath(key), data); - } - - async remove(key: StorageKey): Promise { - try { - await fs.unlink(this.filePath(key)); - } catch (error) { - if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; - } - } - - async loadRange(keyPrefix: StorageKey): Promise { - await fs.mkdir(this.dir, { recursive: true }); - const prefix = this.keyToFilename(keyPrefix); - const files = await fs.readdir(this.dir); - return Promise.all( - files - .filter((file) => this.matchesPrefix(file, prefix)) - .map(async (file) => ({ - key: this.filenameToKey(file), - data: toUint8Array(await fs.readFile(path.join(this.dir, file))), - })), - ); - } - - async removeRange(keyPrefix: StorageKey): Promise { - await fs.mkdir(this.dir, { recursive: true }); - const prefix = this.keyToFilename(keyPrefix); - const files = await fs.readdir(this.dir); - await Promise.all( - files - .filter((file) => this.matchesPrefix(file, prefix)) - .map((file) => fs.unlink(path.join(this.dir, file))), - ); - } - - private filePath(key: StorageKey) { - return path.join(this.dir, this.keyToFilename(key)); - } - - private keyToFilename(key: StorageKey) { - return key.map((part) => encodeURIComponent(part)).join("."); - } - - private filenameToKey(filename: string): StorageKey { - return filename.split(".").map((part) => decodeURIComponent(part)); - } - - private matchesPrefix(filename: string, prefix: string) { - return !prefix || filename === prefix || filename.startsWith(`${prefix}.`); - } -} - -function toUint8Array(data: Uint8Array) { - return new Uint8Array(data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength)); -} diff --git a/packages/sync-server/test/sync-server.test.ts b/packages/sync-server/test/sync-server.test.ts deleted file mode 100644 index 68f1181..0000000 --- a/packages/sync-server/test/sync-server.test.ts +++ /dev/null @@ -1,166 +0,0 @@ -import { describe, expect, it } from "vitest"; -import http from "node:http"; -import net from "node:net"; -import fs from "node:fs/promises"; -import os from "node:os"; -import path from "node:path"; -import crypto from "node:crypto"; -import WebSocket from "isomorphic-ws"; -import { - attachLyncServer, - createLyncServer, - FileStorageAdapter, -} from "../src/index.js"; - -describe("lync server", () => { - it("starts a WebSocket-backed Automerge repo and closes cleanly", async () => { - const server = createLyncServer(); - - expect(server.url.startsWith("ws://")).toBe(true); - expect(new URL(server.url).pathname).toBe("/lync"); - expect(server.repo.peerId).toBeTruthy(); - - await server.close(); - }); - - it("normalizes custom sync paths in the standalone server URL", async () => { - const server = createLyncServer({ path: "sync" }); - - expect(new URL(server.url).pathname).toBe("/sync"); - - await server.close(); - }); - - it("attaches a relay to an existing HTTP server", async () => { - const httpServer = http.createServer(); - const relay = attachLyncServer(httpServer); - - expect(relay.repo.peerId).toBeTruthy(); - - await relay.close(); - httpServer.close(); - }); - - it("does not fail shutdown when the repo cannot flush", async () => { - const httpServer = http.createServer(); - const relay = attachLyncServer(httpServer, { - repo: { - peerId: "broken-shutdown-repo", - shutdown: async () => { - throw new Error("DocHandle is not ready"); - }, - } as never, - }); - - await expect(relay.close()).resolves.toBeUndefined(); - httpServer.close(); - }); - - it("can authenticate websocket upgrades", async () => { - const httpServer = http.createServer(); - const seenAuthHeaders: Array = []; - const relay = attachLyncServer(httpServer, { - authenticate: (request) => { - seenAuthHeaders.push(request.headers.authorization); - return request.headers.authorization === "Bearer ok"; - }, - }); - - await new Promise((resolve) => httpServer.listen(0, "127.0.0.1", resolve)); - const address = httpServer.address(); - if (typeof address === "string" || address === null) { - throw new Error("Expected TCP server address"); - } - const url = `ws://127.0.0.1:${address.port}/lync`; - - await expect(sendUpgrade(address.port)).resolves.toBe("closed"); - expect(seenAuthHeaders).toContain(undefined); - await expect(connect(url, { authorization: "Bearer ok" })).resolves.toBeUndefined(); - expect(seenAuthHeaders).toContain("Bearer ok"); - - await relay.close(); - httpServer.close(); - }); - - it("rejects websocket upgrades when authentication throws", async () => { - const httpServer = http.createServer(); - const relay = attachLyncServer(httpServer, { - authenticate: () => { - throw new Error("auth backend unavailable"); - }, - }); - - await new Promise((resolve) => httpServer.listen(0, "127.0.0.1", resolve)); - const address = httpServer.address(); - if (typeof address === "string" || address === null) { - throw new Error("Expected TCP server address"); - } - - const url = `ws://127.0.0.1:${address.port}/lync`; - await expect(sendUpgrade(address.port)).resolves.toBe("closed"); - - await relay.close(); - httpServer.close(); - }); - - it("persists storage chunks to the filesystem", async () => { - const dir = await fs.mkdtemp(path.join(os.tmpdir(), "lync-storage-")); - const storage = new FileStorageAdapter(dir); - await storage.save(["doc", "snapshot"], new Uint8Array([1, 2, 3])); - - await expect(storage.load(["doc", "snapshot"])).resolves.toEqual( - new Uint8Array([1, 2, 3]), - ); - await expect(storage.loadRange(["doc"])).resolves.toHaveLength(1); - - await storage.removeRange(["doc"]); - await expect(storage.load(["doc", "snapshot"])).resolves.toBeUndefined(); - }); -}); - -function sendUpgrade(port: number) { - return new Promise((resolve, reject) => { - let settled = false; - const socket = net.createConnection({ host: "127.0.0.1", port }, () => { - socket.write( - [ - "GET /lync HTTP/1.1", - "Host: 127.0.0.1", - "Connection: Upgrade", - "Upgrade: websocket", - "Sec-WebSocket-Version: 13", - `Sec-WebSocket-Key: ${crypto.randomBytes(16).toString("base64")}`, - "", - "", - ].join("\r\n"), - ); - }); - - socket.setTimeout(1000, () => { - socket.destroy(new Error("Timed out waiting for websocket rejection")); - }); - socket.on("data", (chunk) => { - if (!chunk.toString().startsWith("HTTP/1.1")) return; - settled = true; - socket.destroy(); - resolve("closed"); - }); - socket.on("close", () => { - if (!settled) resolve("closed"); - }); - socket.on("error", (error) => { - if (!settled) reject(error); - }); - }); -} - -function connect(url: string, headers: Record = {}) { - return new Promise((resolve, reject) => { - const socket = new WebSocket(url, { headers }); - socket.once("open", () => { - socket.close(); - resolve(); - }); - socket.once("error", reject); - }); -} diff --git a/packages/sync-server/tsconfig.json b/packages/sync-server/tsconfig.json deleted file mode 100644 index df59da5..0000000 --- a/packages/sync-server/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*.ts"] -} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index d1229f9..e519b6d 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -7,28 +7,6 @@ settings: importers: .: - dependencies: - '@automerge/automerge-repo': - specifier: ^2.5.5 - version: 2.5.5 - '@automerge/automerge-repo-network-broadcastchannel': - specifier: ^2.5.5 - version: 2.5.5 - '@automerge/automerge-repo-network-websocket': - specifier: ^2.5.5 - version: 2.5.5 - '@automerge/automerge-repo-storage-indexeddb': - specifier: ^2.5.5 - version: 2.5.5 - '@types/ws': - specifier: ^8.18.1 - version: 8.18.1 - isomorphic-ws: - specifier: ^5.0.0 - version: 5.0.0(ws@8.20.0) - uuid: - specifier: ^14.0.0 - version: 14.0.0 devDependencies: '@types/node': specifier: ^22.14.0 @@ -49,95 +27,10 @@ importers: specifier: workspace:* version: link:../core - packages/client: - dependencies: - '@automerge/automerge-repo': - specifier: ^2.5.5 - version: 2.5.5 - '@automerge/automerge-repo-network-websocket': - specifier: ^2.5.5 - version: 2.5.5 - isomorphic-ws: - specifier: ^5.0.0 - version: 5.0.0(ws@8.20.0) - lync-core: - specifier: workspace:* - version: link:../core - lync-index: - specifier: workspace:* - version: link:../index - - packages/core: - dependencies: - '@automerge/automerge': - specifier: ^3.2.6 - version: 3.2.6 - - packages/index: - dependencies: - lync-core: - specifier: workspace:* - version: link:../core - - packages/sync-server: - dependencies: - '@automerge/automerge-repo': - specifier: ^2.5.5 - version: 2.5.5 - '@automerge/automerge-repo-network-websocket': - specifier: ^2.5.5 - version: 2.5.5 - isomorphic-ws: - specifier: ^5.0.0 - version: 5.0.0(ws@8.20.0) + packages/core: {} packages: - '@automerge/automerge-repo-network-broadcastchannel@2.5.5': - resolution: {integrity: sha512-yYSW2lEd+aJyY6HRS2Q0PCWUmtWiGhp8oRdmL61KipRC31dZaYo+qsRVt+xw45MAC0ZBWUYHuvF6xqCPDB4Q1A==} - - '@automerge/automerge-repo-network-websocket@2.5.5': - resolution: {integrity: sha512-pwHNXTsTTfofU3X/wtFa9L3lWfAJBI7v1+3EKgFDgEodUJo9FPDH0hcy4HUsQiDrQPADO7FP9fVOQSXVm8n5VA==} - - '@automerge/automerge-repo-storage-indexeddb@2.5.5': - resolution: {integrity: sha512-pH8tw8uLqEtv1POhy2IFnpBDFpGqiR6YM3w4Rk0NkmerstUxQwrqkkeABlkvF5Al6krlu6dC47LnF8v5cHB3Fg==} - - '@automerge/automerge-repo@2.5.5': - resolution: {integrity: sha512-A7vrMvIx5axW3smczZStONaZsksFSjKK8e0Th0u+oEV3aMsylaExpDvjRE2ZIZotJT30+l3tCUlge/n/XGK25Q==} - - '@automerge/automerge@3.2.6': - resolution: {integrity: sha512-9/GXXfYYWNVGpnbRrGQzTNU4fWZ3XaEMeEg0OrpK4pvlQSpkmUBoirEb/4TMK6BwMysZGV5Yeneq3wwc7RNGfg==} - - '@cbor-extract/cbor-extract-darwin-arm64@2.2.2': - resolution: {integrity: sha512-ZKZ/F8US7JR92J4DMct6cLW/Y66o2K576+zjlEN/MevH70bFIsB10wkZEQPLzl2oNh2SMGy55xpJ9JoBRl5DOA==} - cpu: [arm64] - os: [darwin] - - '@cbor-extract/cbor-extract-darwin-x64@2.2.2': - resolution: {integrity: sha512-32b1mgc+P61Js+KW9VZv/c+xRw5EfmOcPx990JbCBSkYJFY0l25VinvyyWfl+3KjibQmAcYwmyzKF9J4DyKP/Q==} - cpu: [x64] - os: [darwin] - - '@cbor-extract/cbor-extract-linux-arm64@2.2.2': - resolution: {integrity: sha512-wfqgzqCAy/Vn8i6WVIh7qZd0DdBFaWBjPdB6ma+Wihcjv0gHqD/mw3ouVv7kbbUNrab6dKEx/w3xQZEdeXIlzg==} - cpu: [arm64] - os: [linux] - - '@cbor-extract/cbor-extract-linux-arm@2.2.2': - resolution: {integrity: sha512-tNg0za41TpQfkhWjptD+0gSD2fggMiDCSacuIeELyb2xZhr7PrhPe5h66Jc67B/5dmpIhI2QOUtv4SBsricyYQ==} - cpu: [arm] - os: [linux] - - '@cbor-extract/cbor-extract-linux-x64@2.2.2': - resolution: {integrity: sha512-rpiLnVEsqtPJ+mXTdx1rfz4RtUGYIUg2rUAZgd1KjiC1SehYUSkJN7Yh+aVfSjvCGtVP0/bfkQkXpPXKbmSUaA==} - cpu: [x64] - os: [linux] - - '@cbor-extract/cbor-extract-win32-x64@2.2.2': - resolution: {integrity: sha512-dI+9P7cfWxkTQ+oE+7Aa6onEn92PHgfWXZivjNheCRmTBDBf2fx6RyTi0cmgpYLnD1KLZK9ZYrMxaPZ4oiXhGA==} - cpu: [x64] - os: [win32] - '@esbuild/aix-ppc64@0.27.7': resolution: {integrity: sha512-EKX3Qwmhz1eMdEJokhALr0YiD0lhQNwDqkPYyPhiSwKrh7/4KRjQc04sZ8db+5DVVnZ1LmbNDI1uAMPEUBnQPg==} engines: {node: '>=18'} @@ -297,10 +190,6 @@ packages: '@jridgewell/sourcemap-codec@1.5.5': resolution: {integrity: sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==} - '@noble/hashes@1.8.0': - resolution: {integrity: sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==} - engines: {node: ^14.21.3 || >=16} - '@rollup/rollup-android-arm-eabi@4.60.2': resolution: {integrity: sha512-dnlp69efPPg6Uaw2dVqzWRfAWRnYVb1XJ8CyyhIbZeaq4CA5/mLeZ1IEt9QqQxmbdvagjLIm2ZL8BxXv5lH4Yw==} cpu: [arm] @@ -438,9 +327,6 @@ packages: '@types/node@22.19.17': resolution: {integrity: sha512-wGdMcf+vPYM6jikpS/qhg6WiqSV/OhG+jeeHT/KlVqxYfD40iYJf9/AE1uQxVWFvU7MipKRkRv8NSHiCGgPr8Q==} - '@types/ws@8.18.1': - resolution: {integrity: sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg==} - '@vitest/expect@3.2.4': resolution: {integrity: sha512-Io0yyORnB6sikFlt8QW5K7slY4OjqNX9jmJQ02QDda8lyM6B5oNgVWoSoKPac8/kgnCUzuHQKrSLtu/uOqqrig==} @@ -474,26 +360,10 @@ packages: resolution: {integrity: sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==} engines: {node: '>=12'} - base-x@4.0.1: - resolution: {integrity: sha512-uAZ8x6r6S3aUM9rbHGVOIsR15U/ZSc82b3ymnCPsT45Gk1DDvhDPdIgB5MrhirZWt+5K0EEPQH985kNqZgNPFw==} - - bs58@5.0.0: - resolution: {integrity: sha512-r+ihvQJvahgYT50JD05dyJNKlmmSlMoOGwn1lCcEzanPglg7TxYjioQUYehQ9mAR/+hOSd2jRc/Z2y5UxBymvQ==} - - bs58check@3.0.1: - resolution: {integrity: sha512-hjuuJvoWEybo7Hn/0xOrczQKKEKD63WguEjlhLExYs2wUBcebDC1jDNK17eEAD2lYfw82d5ASC1d7K3SWszjaQ==} - cac@6.7.14: resolution: {integrity: sha512-b6Ilus+c3RrdDk+JhLKUAQfzzgLEPy6wcXqS7f/xe1EETvsDP6GORG7SFuOs6cID5YkqchW/LXZbX5bc8j7ZcQ==} engines: {node: '>=8'} - cbor-extract@2.2.2: - resolution: {integrity: sha512-hlSxxI9XO2yQfe9g6msd3g4xCfDqK5T5P0fRMLuaLHhxn4ViPrm+a+MUfhrvH2W962RGxcBwEGzLQyjbDG1gng==} - hasBin: true - - cbor-x@1.6.4: - resolution: {integrity: sha512-UGKHjp6RHC6QuZ2yy5LCKm7MojM4716DwoSaqwQpaH4DvZvbBTGcoDNTiG9Y2lByXZYFEs9WRkS5tLl96IrF1Q==} - chai@5.3.3: resolution: {integrity: sha512-4zNhdJD/iOjSH0A05ea+Ke6MU5mmpQcbQsSOkgdaUMJ9zTlDTD/GYlwohmIE2u0gaxHYiVHEn1Fw9mZ/ktJWgw==} engines: {node: '>=18'} @@ -515,10 +385,6 @@ packages: resolution: {integrity: sha512-h5k/5U50IJJFpzfL6nO9jaaumfjO/f2NjK/oYB2Djzm4p9L+3T9qWpZqZ2hAbLPuuYq9wrU08WQyBTL5GbPk5Q==} engines: {node: '>=6'} - detect-libc@2.1.2: - resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==} - engines: {node: '>=8'} - es-module-lexer@1.7.0: resolution: {integrity: sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==} @@ -530,16 +396,10 @@ packages: estree-walker@3.0.3: resolution: {integrity: sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==} - eventemitter3@5.0.4: - resolution: {integrity: sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==} - expect-type@1.3.0: resolution: {integrity: sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==} engines: {node: '>=12.0.0'} - fast-sha256@1.3.0: - resolution: {integrity: sha512-n11RGP/lrWEFI/bWdygLxhI+pVeo1ZYIVwvvPkW7azl/rOy+F3HYRZ2K5zeE9mmkhQppyv9sQFx0JM9UabnpPQ==} - fdir@6.5.0: resolution: {integrity: sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==} engines: {node: '>=12.0.0'} @@ -554,11 +414,6 @@ packages: engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} os: [darwin] - isomorphic-ws@5.0.0: - resolution: {integrity: sha512-muId7Zzn9ywDsyXgTIafTry2sV3nySZeUDe6YedVd1Hvuuep5AsIlqK+XefWpYTyJG5e503F2xIuT2lcU6rCSw==} - peerDependencies: - ws: '*' - js-tokens@9.0.1: resolution: {integrity: sha512-mxa9E9ITFOt0ban3j6L5MpjwegGz6lBQmM1IJkWeBZGcMxto50+eWdjC/52xDbS2vy0k7vIMK0Fe2wfL9OQSpQ==} @@ -576,10 +431,6 @@ packages: engines: {node: ^10 || ^12 || ^13.7 || ^14 || >=15.0.1} hasBin: true - node-gyp-build-optional-packages@5.1.1: - resolution: {integrity: sha512-+P72GAjVAbTxjjwUmwjVrqrdZROD4nf8KgpBoDxqXXTiYZZt/ud60dE5yvCSr9lRO8e8yv6kgJIC0K0PfZFVQw==} - hasBin: true - pathe@2.0.3: resolution: {integrity: sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==} @@ -649,15 +500,6 @@ packages: undici-types@6.21.0: resolution: {integrity: sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==} - uuid@14.0.0: - resolution: {integrity: sha512-Qo+uWgilfSmAhXCMav1uYFynlQO7fMFiMVZsQqZRMIXp0O7rR7qjkj+cPvBHLgBqi960QCoo/PH2/6ZtVqKvrg==} - hasBin: true - - uuid@9.0.1: - resolution: {integrity: sha512-b+1eJOlsR9K8HJpow9Ok3fiWOWSIcIzXodvv0rQjVoOVNpWMpxf1wZNpt4y9h10odCNrqnYp1OBzRktckBe3sA==} - deprecated: uuid@10 and below is no longer supported. For ESM codebases, update to uuid@latest. For CommonJS codebases, use uuid@11 (but be aware this version will likely be deprecated in 2028). - hasBin: true - vite-node@3.2.4: resolution: {integrity: sha512-EbKSKh+bh1E1IFxeO0pg1n4dvoOTt0UDiXMd/qn++r98+jPO1xtJilvXldeuQ8giIB5IkpjCgMleHMNEsGH6pg==} engines: {node: ^18.0.0 || ^20.0.0 || >=22.0.0} @@ -736,81 +578,8 @@ packages: engines: {node: '>=8'} hasBin: true - ws@8.20.0: - resolution: {integrity: sha512-sAt8BhgNbzCtgGbt2OxmpuryO63ZoDk/sqaB/znQm94T4fCEsy/yV+7CdC1kJhOU9lboAEU7R3kquuycDoibVA==} - engines: {node: '>=10.0.0'} - peerDependencies: - bufferutil: ^4.0.1 - utf-8-validate: '>=5.0.2' - peerDependenciesMeta: - bufferutil: - optional: true - utf-8-validate: - optional: true - - xstate@5.30.0: - resolution: {integrity: sha512-mIzIuMjtYVkqXq9dUzYQoag7b/dF1CBS/yhliuPLfR0FwKPC18HiUivb/crcqY2gknhR8gJEhnppLg6ubQ0gGw==} - snapshots: - '@automerge/automerge-repo-network-broadcastchannel@2.5.5': - dependencies: - '@automerge/automerge-repo': 2.5.5 - transitivePeerDependencies: - - supports-color - - '@automerge/automerge-repo-network-websocket@2.5.5': - dependencies: - '@automerge/automerge-repo': 2.5.5 - cbor-x: 1.6.4 - debug: 4.4.3 - eventemitter3: 5.0.4 - isomorphic-ws: 5.0.0(ws@8.20.0) - ws: 8.20.0 - transitivePeerDependencies: - - bufferutil - - supports-color - - utf-8-validate - - '@automerge/automerge-repo-storage-indexeddb@2.5.5': - dependencies: - '@automerge/automerge-repo': 2.5.5 - transitivePeerDependencies: - - supports-color - - '@automerge/automerge-repo@2.5.5': - dependencies: - '@automerge/automerge': 3.2.6 - bs58check: 3.0.1 - cbor-x: 1.6.4 - debug: 4.4.3 - eventemitter3: 5.0.4 - fast-sha256: 1.3.0 - uuid: 9.0.1 - xstate: 5.30.0 - transitivePeerDependencies: - - supports-color - - '@automerge/automerge@3.2.6': {} - - '@cbor-extract/cbor-extract-darwin-arm64@2.2.2': - optional: true - - '@cbor-extract/cbor-extract-darwin-x64@2.2.2': - optional: true - - '@cbor-extract/cbor-extract-linux-arm64@2.2.2': - optional: true - - '@cbor-extract/cbor-extract-linux-arm@2.2.2': - optional: true - - '@cbor-extract/cbor-extract-linux-x64@2.2.2': - optional: true - - '@cbor-extract/cbor-extract-win32-x64@2.2.2': - optional: true - '@esbuild/aix-ppc64@0.27.7': optional: true @@ -891,8 +660,6 @@ snapshots: '@jridgewell/sourcemap-codec@1.5.5': {} - '@noble/hashes@1.8.0': {} - '@rollup/rollup-android-arm-eabi@4.60.2': optional: true @@ -981,10 +748,6 @@ snapshots: dependencies: undici-types: 6.21.0 - '@types/ws@8.18.1': - dependencies: - '@types/node': 22.19.17 - '@vitest/expect@3.2.4': dependencies: '@types/chai': 5.2.3 @@ -1029,35 +792,8 @@ snapshots: assertion-error@2.0.1: {} - base-x@4.0.1: {} - - bs58@5.0.0: - dependencies: - base-x: 4.0.1 - - bs58check@3.0.1: - dependencies: - '@noble/hashes': 1.8.0 - bs58: 5.0.0 - cac@6.7.14: {} - cbor-extract@2.2.2: - dependencies: - node-gyp-build-optional-packages: 5.1.1 - optionalDependencies: - '@cbor-extract/cbor-extract-darwin-arm64': 2.2.2 - '@cbor-extract/cbor-extract-darwin-x64': 2.2.2 - '@cbor-extract/cbor-extract-linux-arm': 2.2.2 - '@cbor-extract/cbor-extract-linux-arm64': 2.2.2 - '@cbor-extract/cbor-extract-linux-x64': 2.2.2 - '@cbor-extract/cbor-extract-win32-x64': 2.2.2 - optional: true - - cbor-x@1.6.4: - optionalDependencies: - cbor-extract: 2.2.2 - chai@5.3.3: dependencies: assertion-error: 2.0.1 @@ -1074,9 +810,6 @@ snapshots: deep-eql@5.0.2: {} - detect-libc@2.1.2: - optional: true - es-module-lexer@1.7.0: {} esbuild@0.27.7: @@ -1112,12 +845,8 @@ snapshots: dependencies: '@types/estree': 1.0.8 - eventemitter3@5.0.4: {} - expect-type@1.3.0: {} - fast-sha256@1.3.0: {} - fdir@6.5.0(picomatch@4.0.4): optionalDependencies: picomatch: 4.0.4 @@ -1125,10 +854,6 @@ snapshots: fsevents@2.3.3: optional: true - isomorphic-ws@5.0.0(ws@8.20.0): - dependencies: - ws: 8.20.0 - js-tokens@9.0.1: {} loupe@3.2.1: {} @@ -1141,11 +866,6 @@ snapshots: nanoid@3.3.11: {} - node-gyp-build-optional-packages@5.1.1: - dependencies: - detect-libc: 2.1.2 - optional: true - pathe@2.0.3: {} pathval@2.0.1: {} @@ -1222,10 +942,6 @@ snapshots: undici-types@6.21.0: {} - uuid@14.0.0: {} - - uuid@9.0.1: {} - vite-node@3.2.4(@types/node@22.19.17): dependencies: cac: 6.7.14 @@ -1304,7 +1020,3 @@ snapshots: dependencies: siginfo: 2.0.0 stackback: 0.0.2 - - ws@8.20.0: {} - - xstate@5.30.0: {} diff --git a/scripts/migrate-automerge-to-lync.ts b/scripts/migrate-automerge-to-lync.ts deleted file mode 100644 index da6ad05..0000000 --- a/scripts/migrate-automerge-to-lync.ts +++ /dev/null @@ -1,366 +0,0 @@ -import fs from "node:fs/promises"; -import path from "node:path"; -import { fileURLToPath } from "node:url"; -import { - Automerge, - initializeWasm, - type Chunk, - type DocumentId, - type StorageAdapterInterface, - type StorageKey, -} from "@automerge/automerge-repo/slim"; -import { createLoreLooms } from "../packages/core/dist/lore/looms.js"; -import { BaseEventStore } from "../packages/core/dist/lore/store.js"; -import type { LoomSnapshot, Turn } from "../packages/core/dist/types.js"; - -const ROOT_CHILDREN_KEY = "__root__"; - -interface LoomDoc { - version: 1; - root: { id: string; meta?: unknown; createdAt: number }; - nodes: Record>; - children: Record; -} - -interface MigrationFailure { - docId: string; - reason: string; - chunkBytes: number; - chunkFiles: number; - chunks: string[]; - beforeEvents?: number; - afterEvents?: number; -} - -interface MigrationReport { - sourceDir: string; - outDir: string; - docsSeen: number; - loomsMigrated: number; - beforeEvents: number; - afterEvents: number; - docs: MigrationDocReport[]; - failures: MigrationFailure[]; -} - -interface MigrationDocReport { - docId: string; - beforeEvents: number; - afterEvents: number; - chunkBytes: number; - chunkFiles: number; -} - -async function migrate(source: string, out: string): Promise { - await initializeAutomerge(); - await fs.mkdir(out, { recursive: true }); - const docIds = await listDocumentIds(source); - const store = new MigrationEventStore(out); - const looms = createLoreLooms({ - store, - author: { actor: "unknown", imported_by: "lync-automerge-migrator@0.1" }, - }); - const failures: MigrationFailure[] = []; - const docs: MigrationDocReport[] = []; - let beforeEvents = 0; - let afterEvents = 0; - let loomsMigrated = 0; - let report: MigrationReport = { - sourceDir: source, - outDir: out, - docsSeen: docIds.length, - loomsMigrated, - beforeEvents, - afterEvents, - docs, - failures, - }; - await writeReport(out, report); - - for (const docId of docIds) { - const stats = await chunkStats(source, docId); - let docBeforeEvents: number | undefined; - let docAfterEvents: number | undefined; - try { - const doc = await loadDoc(source, docId); - if (!isLoomDoc(doc)) { - failures.push(failure(docId, "loaded Automerge doc is not loom-shaped", stats)); - report = updateReport(report, { loomsMigrated, beforeEvents, afterEvents }); - await writeReport(out, report); - continue; - } - docBeforeEvents = countLoomDocEvents(doc); - const snapshot = snapshotFromDoc(doc); - const imported = await looms.import(snapshot); - const loom = await looms.open(imported.id); - const migrated = await loom.export(); - assertIsomorphic(snapshot, migrated); - docAfterEvents = countSnapshotEvents(migrated); - if (docBeforeEvents !== docAfterEvents) { - throw new Error(`event count mismatch: before=${docBeforeEvents} after=${docAfterEvents}`); - } - docs.push({ - docId, - beforeEvents: docBeforeEvents, - afterEvents: docAfterEvents, - chunkBytes: stats.bytes, - chunkFiles: stats.files, - }); - beforeEvents += docBeforeEvents; - afterEvents += docAfterEvents; - loomsMigrated++; - await store.flushRoot(rootId(imported.id)); - } catch (error) { - failures.push(failure(docId, error instanceof Error ? error.message : String(error), stats, docBeforeEvents, docAfterEvents)); - } - report = updateReport(report, { loomsMigrated, beforeEvents, afterEvents }); - await writeReport(out, report); - } - - return report; -} - -async function initializeAutomerge(): Promise { - const slimEntrypoint = fileURLToPath(import.meta.resolve("@automerge/automerge-repo/slim")); - const wasmPath = path.resolve(path.dirname(slimEntrypoint), "../../..", "automerge", "dist", "automerge.wasm"); - await initializeWasm(await fs.readFile(wasmPath)); -} - -async function loadDoc(source: string, docId: string): Promise { - const adapter = new FileStorageAdapter(source); - const binary = await loadDocData(adapter, docId); - if (!binary) throw new Error("no Automerge chunks found for document"); - return Automerge.loadIncremental(Automerge.init(), binary); -} - -async function loadDocData(adapter: StorageAdapterInterface, docId: string): Promise { - const chunks = [ - ...(await adapter.loadRange([docId as DocumentId, "snapshot"])), - ...(await adapter.loadRange([docId as DocumentId, "incremental"])), - ]; - const binaries = chunks.map((chunk) => chunk.data).filter((data): data is Uint8Array => data !== undefined); - if (binaries.length === 0) return null; - return mergeArrays(binaries); -} - -function isLoomDoc(value: unknown): value is LoomDoc { - if (!value || typeof value !== "object") return false; - const candidate = value as Partial; - return candidate.version === 1 && isRecord(candidate.root) && isRecord(candidate.nodes) && isRecord(candidate.children); -} - -function snapshotFromDoc(doc: LoomDoc): LoomSnapshot { - const turns: Turn[] = []; - const seen = new Set(); - const visit = (parentId: string | null) => { - const key = parentId ?? ROOT_CHILDREN_KEY; - for (const turnId of doc.children[key] ?? []) { - if (seen.has(turnId)) throw new Error(`cycle or duplicate child reference at ${turnId}`); - const turn = doc.nodes[turnId]; - if (!turn) throw new Error(`child list references missing turn ${turnId}`); - seen.add(turnId); - turns.push(turn); - visit(turnId); - } - }; - visit(null); - for (const turn of Object.values(doc.nodes)) { - if (!seen.has(turn.id)) throw new Error(`unreachable turn ${turn.id}`); - } - return { loom: doc.root, turns }; -} - -function countLoomDocEvents(doc: LoomDoc): number { - return Object.keys(doc.nodes).length + 1; -} - -function countSnapshotEvents(snapshot: LoomSnapshot): number { - return snapshot.turns.length + 1; -} - -function assertIsomorphic( - before: LoomSnapshot, - after: LoomSnapshot, -): void { - const normalize = (snapshot: LoomSnapshot) => - snapshot.turns.map((turn) => ({ - parentIndex: turn.parentId === null ? null : snapshot.turns.findIndex((candidate) => candidate.id === turn.parentId), - payload: turn.payload, - meta: turn.meta, - createdAt: turn.createdAt, - })); - if (JSON.stringify(before.loom.meta) !== JSON.stringify(after.loom.meta)) { - throw new Error("loom meta mismatch after migration"); - } - if (JSON.stringify(normalize(before)) !== JSON.stringify(normalize(after))) { - throw new Error("turn topology/payload mismatch after migration"); - } -} - -async function listDocumentIds(dir: string): Promise { - const files = await fs.readdir(dir); - return [ - ...new Set( - files - .map((file) => file.split(".")) - .filter((parts) => parts[1] === "snapshot" || parts[1] === "incremental") - .map(([docId]) => docId) - .filter(Boolean), - ), - ].sort(); -} - -async function chunkStats(dir: string, docId: string): Promise<{ bytes: number; files: number; chunks: string[] }> { - const files = (await fs.readdir(dir)).filter((file) => file.startsWith(`${docId}.`)).sort(); - let bytes = 0; - for (const file of files) bytes += (await fs.stat(path.join(dir, file))).size; - return { bytes, files: files.length, chunks: files.map((file) => path.join(dir, file)) }; -} - -function failure( - docId: string, - reason: string, - stats: { bytes: number; files: number; chunks: string[] }, - beforeEvents?: number, - afterEvents?: number, -): MigrationFailure { - return { - docId, - reason, - chunkBytes: stats.bytes, - chunkFiles: stats.files, - chunks: stats.chunks, - ...(beforeEvents === undefined ? {} : { beforeEvents }), - ...(afterEvents === undefined ? {} : { afterEvents }), - }; -} - -function updateReport( - report: MigrationReport, - counts: Pick, -): MigrationReport { - return { - ...report, - ...counts, - }; -} - -async function writeReport(out: string, report: MigrationReport): Promise { - const reportPath = path.join(out, "migration-report.json"); - const tmpPath = `${reportPath}.tmp`; - await fs.writeFile(tmpPath, JSON.stringify(report, null, 2)); - await fs.rename(tmpPath, reportPath); -} - -function mergeArrays(arrays: Uint8Array[]): Uint8Array { - const size = arrays.reduce((total, array) => total + array.length, 0); - const merged = new Uint8Array(size); - let offset = 0; - for (const array of arrays) { - merged.set(array, offset); - offset += array.length; - } - return merged; -} - -function rootId(loomId: string): string { - if (!loomId.startsWith("lore:")) throw new Error(`expected imported lync loom id, got ${loomId}`); - return loomId.slice("lore:".length); -} - -class MigrationEventStore extends BaseEventStore { - private readonly dir: string; - - constructor(dir: string) { - super(); - this.dir = dir; - } - - async flushRoot(root: string): Promise { - await fs.mkdir(this.dir, { recursive: true }); - await fs.writeFile(path.join(this.dir, `${encodeURIComponent(root)}.lync`), await this.exportRootBytes(root)); - } -} - -class FileStorageAdapter implements StorageAdapterInterface { - private readonly dir: string; - - constructor(dir: string) { - this.dir = dir; - } - - async load(key: StorageKey): Promise { - try { - return toUint8Array(await fs.readFile(this.filePath(key))); - } catch (error) { - if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined; - throw error; - } - } - - async save(key: StorageKey, data: Uint8Array): Promise { - await fs.mkdir(this.dir, { recursive: true }); - await fs.writeFile(this.filePath(key), data); - } - - async remove(key: StorageKey): Promise { - try { - await fs.unlink(this.filePath(key)); - } catch (error) { - if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; - } - } - - async loadRange(keyPrefix: StorageKey): Promise { - const prefix = this.keyToFilename(keyPrefix); - const files = await fs.readdir(this.dir); - return Promise.all( - files - .filter((file) => !prefix || file === prefix || file.startsWith(`${prefix}.`)) - .map(async (file) => ({ - key: this.filenameToKey(file), - data: toUint8Array(await fs.readFile(path.join(this.dir, file))), - })), - ); - } - - async removeRange(keyPrefix: StorageKey): Promise { - const prefix = this.keyToFilename(keyPrefix); - const files = await fs.readdir(this.dir); - await Promise.all( - files - .filter((file) => !prefix || file === prefix || file.startsWith(`${prefix}.`)) - .map((file) => fs.unlink(path.join(this.dir, file))), - ); - } - - private filePath(key: StorageKey) { - return path.join(this.dir, this.keyToFilename(key)); - } - - private keyToFilename(key: StorageKey) { - return key.map((part) => encodeURIComponent(part)).join("."); - } - - private filenameToKey(filename: string): StorageKey { - return filename.split(".").map((part) => decodeURIComponent(part)); - } -} - -function toUint8Array(data: Uint8Array) { - return new Uint8Array(data.buffer.slice(data.byteOffset, data.byteOffset + data.byteLength)); -} - -function isRecord(value: unknown): value is Record { - return typeof value === "object" && value !== null && !Array.isArray(value); -} - -const [sourceDir, outDir] = process.argv.slice(2); -if (!sourceDir || !outDir) { - console.error("usage: node --experimental-strip-types scripts/migrate-automerge-to-lync.ts "); - process.exit(2); -} - -const report = await migrate(sourceDir, outDir); -console.log(JSON.stringify(report, null, 2)); -if (report.failures.length) process.exitCode = 1; diff --git a/vitest.config.ts b/vitest.config.ts index 097ff67..6e7df0c 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -4,94 +4,20 @@ export default defineConfig({ resolve: { alias: [ { - find: /^@lync\/core\/automerge$/, - replacement: new URL( - "./packages/core/src/automerge.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/core\/browser$/, - replacement: new URL( - "./packages/core/src/browser.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/core\/memory$/, - replacement: new URL( - "./packages/core/src/memory.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/core\/profiles\/text-story$/, + find: /^lync-core\/profiles\/text-story$/, replacement: new URL( "./packages/core/src/profiles/text-story.ts", import.meta.url, ).pathname, }, { - find: /^@lync\/core\/lore\/events$/, - replacement: new URL( - "./packages/core/src/lore/events.ts", - import.meta.url, - ).pathname, + find: /^lync-core\/([a-z0-9-]+)$/, + replacement: new URL("./packages/core/src/", import.meta.url).pathname + "$1.ts", }, { - find: /^@lync\/core\/lore\/views$/, - replacement: new URL( - "./packages/core/src/lore/views.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/core$/, + find: /^lync-core$/, replacement: new URL("./packages/core/src/index.ts", import.meta.url).pathname, }, - { - find: /^@lync\/index\/automerge$/, - replacement: new URL( - "./packages/index/src/automerge.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/index\/memory$/, - replacement: new URL( - "./packages/index/src/memory.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/index$/, - replacement: new URL("./packages/index/src/index.ts", import.meta.url).pathname, - }, - { - find: /^@lync\/client\/browser$/, - replacement: new URL( - "./packages/client/src/browser.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/client\/node$/, - replacement: new URL( - "./packages/client/src/node.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/client\/testing$/, - replacement: new URL( - "./packages/client/src/testing.ts", - import.meta.url, - ).pathname, - }, - { - find: /^@lync\/client$/, - replacement: new URL("./packages/client/src/index.ts", import.meta.url).pathname, - }, ], }, test: { From 92ac3541760dbe300f6cec5aa790d47026b7f291 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 18:45:57 -0700 Subject: [PATCH 10/33] =?UTF-8?q?sync:=20native=20line-sync=20ships=20?= =?UTF-8?q?=E2=80=94=20lync=20serve=20+=20lync=20sync=20[--follow]?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The dumb event-union relay from the line-sync design: five frames (sub/ev/live/presence/err), per-root append-only .lync storage, seq as resume cursor, echo-safe under union, same-id conflicts surfaced to both sides and never resolved, truncated tails sealed as damaged and surfaced, presence relayed never stored, optional bearer-token auth. Client verb converges any file one-shot or stays live with --follow (pushes local appends, appends remote events, persists the cursor). Protocol codecs are pure and dependency-free in lync-core/sync-protocol; the ws wiring lives in the CLI. Acceptance from the design ticket holds: two clients converge, offline resume is exact, kill-9 recovery surfaces, unreachable relays fail loudly instead of hanging. 71 tests, 30 consecutive green full-suite runs including the live relay tests. --- README.md | 24 +++ packages/cli/package.json | 6 +- packages/cli/src/index.ts | 101 ++++++++++ packages/cli/src/serve.ts | 241 +++++++++++++++++++++++ packages/cli/src/sync.ts | 217 ++++++++++++++++++++ packages/cli/test/sync.test.ts | 223 +++++++++++++++++++++ packages/core/package.json | 5 + packages/core/src/sync-protocol.ts | 131 ++++++++++++ packages/core/test/sync-protocol.test.ts | 32 +++ pnpm-lock.yaml | 28 +++ 10 files changed, 1007 insertions(+), 1 deletion(-) create mode 100644 packages/cli/src/serve.ts create mode 100644 packages/cli/src/sync.ts create mode 100644 packages/cli/test/sync.test.ts create mode 100644 packages/core/src/sync-protocol.ts create mode 100644 packages/core/test/sync-protocol.test.ts diff --git a/README.md b/README.md index e14d451..bd055af 100644 --- a/README.md +++ b/README.md @@ -186,6 +186,30 @@ const next = await loom.appendTurn(first.id, { text: "Then..." }); console.log((await loom.threadTo(next.id)).map((turn) => turn.payload.text)); ``` +## Sync + +Any lync file can converge with any other copy through a relay: + +```bash +lync serve ./rooms --port 8787 # the relay: one append-only file per root +lync sync story.lync ws://host:8787 # one-shot: push what it lacks, pull what you lack +lync sync story.lync ws://host:8787 --follow # stay live: stream both ways until Ctrl-C +``` + +The relay is deliberately dumb. Events are immutable and merge is union by +id, so the protocol has no merge logic: five JSON frames (`sub`, `ev`, +`live`, `presence`, `err`) that move canonical line bytes. The server never +parses a line beyond extracting its id, stores each root as a plain `.lync` +file you can read with any lync tool, and echoes accepted events to every +subscriber — echoes are duplicate no-ops under union. `seq` is a per-root +arrival counter used as a resume cursor (`.sync.json`), so an offline +client reconnects exactly where it left off. Same-id-different-body is never +resolved: both variants are kept (the relay writes a `.conflicts` sidecar) +and both sides are told loudly. Presence frames are relayed, never stored. +A truncated final line after a crash is sealed and surfaced as damaged, +never eaten. `--token T` on the server requires `Authorization: Bearer T` +to connect. + ## Development ```bash diff --git a/packages/cli/package.json b/packages/cli/package.json index 3f061d6..a3aef25 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -36,6 +36,10 @@ "typecheck": "tsc -p tsconfig.json --noEmit" }, "dependencies": { - "lync-core": "workspace:*" + "lync-core": "workspace:*", + "ws": "^8.18.0" + }, + "devDependencies": { + "@types/ws": "^8.18.1" } } diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index d63842d..3bfcbd6 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -42,6 +42,10 @@ export async function runLyncCli(argv: string[], io: LyncCliIO = {}): Promise [--as transcript|tree]", " init [file] create an empty valid lync file", " append read JSON from stdin and append one event", + " serve [dir] [--port N] [--token T] run the line-sync relay over a directory of roots", + " sync [--root R] [--follow] converge a file with a relay; --follow stays live", "", "verify exits 0 only when every line is accepted. It exits 1 for nonconforming, garbage, damaged, conflict, pending, or graph issues, and 2 for usage or I/O errors.", "", @@ -362,3 +368,98 @@ function isRecord(value: unknown): value is Record { function messageOf(error: unknown): string { return error instanceof Error ? error.message : String(error); } + +async function serveVerb( + args: string[], + out: Pick, + err: Pick, +): Promise { + const { startLyncServe } = await import("./serve.js"); + const positional: string[] = []; + let port: number | undefined; + let token: string | undefined; + for (let index = 0; index < args.length; index += 1) { + const arg = args[index]; + if (arg === "--port") { + port = Number(args[++index]); + if (!Number.isInteger(port) || port < 0 || port > 65_535) { + err.write("lync serve: --port must be an integer between 0 and 65535\n"); + return 2; + } + } else if (arg === "--token") { + token = args[++index]; + if (!token) { + err.write("lync serve: --token requires a value\n"); + return 2; + } + } else { + positional.push(arg); + } + } + if (positional.length > 1) { + err.write("Usage: lync serve [dir] [--port N] [--token T]\n"); + return 2; + } + const server = await startLyncServe({ + dir: positional[0] ?? ".", + port, + token, + log: (message) => err.write(`${message}\n`), + }); + out.write(`lync serve: listening on ws://localhost:${server.port} over ${positional[0] ?? "."}\n`); + await new Promise((resolve) => { + process.once("SIGINT", resolve); + process.once("SIGTERM", resolve); + }); + await server.close(); + out.write("lync serve: closed\n"); + return 0; +} + +async function syncVerb( + args: string[], + out: Pick, + err: Pick, +): Promise { + const { syncOnce } = await import("./sync.js"); + const positional: string[] = []; + let root: string | undefined; + let follow = false; + for (let index = 0; index < args.length; index += 1) { + const arg = args[index]; + if (arg === "--root") { + root = args[++index]; + if (!root) { + err.write("lync sync: --root requires a value\n"); + return 2; + } + } else if (arg === "--follow") { + follow = true; + } else { + positional.push(arg); + } + } + if (positional.length !== 2) { + err.write("Usage: lync sync [--root R] [--follow]\n"); + return 2; + } + const stopper = new AbortController(); + if (follow) { + process.once("SIGINT", () => stopper.abort()); + process.once("SIGTERM", () => stopper.abort()); + } + const result = await syncOnce({ + file: positional[0], + url: positional[1], + root, + follow, + stopSignal: stopper.signal, + out, + err, + }); + out.write( + `lync sync: sent ${result.sent}, received ${result.received} new, ` + + `${result.duplicates} duplicates, ${result.surfaced} surfaced, cursor at seq ${result.seq}\n`, + ); + return result.conflicts > 0 ? 1 : 0; +} diff --git a/packages/cli/src/serve.ts b/packages/cli/src/serve.ts new file mode 100644 index 0000000..007c6f1 --- /dev/null +++ b/packages/cli/src/serve.ts @@ -0,0 +1,241 @@ +import { createServer, type IncomingMessage, type Server } from "node:http"; +import { appendFile, mkdir, readFile } from "node:fs/promises"; +import { existsSync } from "node:fs"; +import { join } from "node:path"; +import { WebSocketServer, type WebSocket } from "ws"; +import { decodeFrame, encodeFrame, extractLineId, type SyncFrame } from "lync-core/sync-protocol"; + +/** + * `lync serve` — the dumb event-union relay from the line-sync design. + * + * One append-only `.lync` file per root. `seq` is the per-root count of + * stored lines: a resume cursor, nothing more. The server never parses a line + * beyond extracting its id. Accepted events fan out to every subscriber of + * the root, sender included — echoes are duplicate no-ops under union. + * Same-id-different-body is never resolved: both sides keep their bytes, the + * variant line goes to a `.conflicts` sidecar, and an err frame goes to + * everyone. Presence frames are relayed and never stored. Nothing fails + * invisibly: malformed input earns an err frame, damaged recovery is loud. + */ + +export interface LyncServeOptions { + dir: string; + port?: number; + token?: string; + log?: (message: string) => void; +} + +export interface LyncSyncServer { + port: number; + close: () => Promise; +} + +interface Room { + root: string; + seq: number; + lines: string[]; + byId: Map; + subscribers: Set; + writeChain: Promise; + recoveryNote?: string; +} + +const ROOT_NAME = /^[A-Za-z0-9._-]+$/; + +export async function startLyncServe(options: LyncServeOptions): Promise { + const log = options.log ?? ((message: string) => process.stderr.write(`${message}\n`)); + await mkdir(options.dir, { recursive: true }); + const rooms = new Map>(); + + const httpServer: Server = createServer((_request, response) => { + response.writeHead(404).end(); + }); + const socketServer = new WebSocketServer({ noServer: true }); + const sockets = new Set(); + + httpServer.on("upgrade", (request, socket, head) => { + if (options.token && !authorized(request, options.token)) { + log("[lync serve] rejected upgrade: bad or missing token"); + socket.write("HTTP/1.1 401 Unauthorized\r\n\r\n"); + socket.destroy(); + return; + } + socketServer.handleUpgrade(request, socket, head, (websocket) => { + socketServer.emit("connection", websocket, request); + }); + }); + + socketServer.on("connection", (socket: WebSocket) => { + sockets.add(socket); + const subscribed = new Set(); + // Frames from one socket are handled strictly in arrival order, so a + // client that pushes its lines and then subscribes is guaranteed to see + // any resulting errors before its backlog and `live`. + let frameChain = Promise.resolve(); + + socket.on("message", (raw) => { + const frame = decodeFrame(raw.toString()); + frameChain = frameChain.then(() => handleFrame(socket, subscribed, frame)); + }); + socket.on("close", () => { + sockets.delete(socket); + void detach(socket, subscribed); + }); + socket.on("error", (error) => { + log(`[lync serve] socket error: ${String(error)}`); + }); + }); + + async function handleFrame(socket: WebSocket, subscribed: Set, frame: SyncFrame): Promise { + try { + switch (frame.t) { + case "err": + // A decode failure or a client-reported error: answer loudly, never store. + send(socket, frame.reason === "malformed-frame" || frame.reason === "unknown-frame-kind" ? frame : { t: "err", reason: "client-error-received", detail: frame.reason }); + return; + case "sub": { + const room = await openRoom(frame.root); + room.subscribers.add(socket); + subscribed.add(room.root); + if (room.recoveryNote) { + send(socket, { t: "err", root: room.root, reason: "recovered-damaged-tail", detail: room.recoveryNote }); + } + for (let index = frame.since; index < room.lines.length; index += 1) { + send(socket, { t: "ev", root: room.root, seq: index + 1, line: room.lines[index] }); + } + send(socket, { t: "live", root: room.root, seq: room.seq }); + return; + } + case "ev": { + const room = await openRoom(frame.root); + const id = extractLineId(frame.line); + if (id === undefined) { + send(socket, { t: "err", root: room.root, reason: "line-without-id", detail: truncate(frame.line) }); + return; + } + const existing = room.byId.get(id); + if (existing !== undefined) { + if (existing === frame.line) return; // duplicate: a no-op by union + await appendSerialized(room, join(options.dir, `${room.root}.conflicts`), frame.line); + broadcast(room, { t: "err", root: room.root, reason: "same-id-conflict", detail: id }, socket); + send(socket, { t: "err", root: room.root, reason: "same-id-conflict", detail: id }); + return; + } + room.byId.set(id, frame.line); + room.lines.push(frame.line); + room.seq += 1; + const seq = room.seq; + await appendSerialized(room, join(options.dir, `${room.root}.lync`), frame.line); + broadcast(room, { t: "ev", root: room.root, seq, line: frame.line }); + return; + } + case "presence": { + const room = await openRoom(frame.root); + broadcast(room, frame, socket); + return; + } + case "live": + send(socket, { t: "err", root: frame.root, reason: "unexpected-live-from-client" }); + return; + } + } catch (error) { + log(`[lync serve] frame handling failed: ${String(error)}`); + send(socket, { t: "err", reason: "server-error", detail: String(error) }); + } + } + + function openRoom(root: string): Promise { + if (!ROOT_NAME.test(root)) { + return Promise.reject(new Error(`invalid root name: ${truncate(root)}`)); + } + let pending = rooms.get(root); + if (!pending) { + pending = recoverRoom(root); + rooms.set(root, pending); + } + return pending; + } + + async function recoverRoom(root: string): Promise { + const room: Room = { root, seq: 0, lines: [], byId: new Map(), subscribers: new Set(), writeChain: Promise.resolve() }; + const path = join(options.dir, `${root}.lync`); + if (!existsSync(path)) return room; + const text = await readFile(path, "utf8"); + const endsClean = text.length === 0 || text.endsWith("\n"); + const lines = text.split("\n"); + if (lines.at(-1) === "") lines.pop(); + if (!endsClean && lines.length > 0) { + // Kill-9 mid-append left a truncated tail. Seal it with a newline so + // future appends start clean; readers classify it as damaged. Loud, + // never eaten. + const tail = lines.at(-1) ?? ""; + room.recoveryNote = `sealed truncated final line (${tail.length} bytes) as damaged`; + log(`[lync serve] ${root}: ${room.recoveryNote}`); + await appendFile(path, "\n"); + } + for (const line of lines) { + room.lines.push(line); + room.seq += 1; + const id = extractLineId(line); + if (id !== undefined && !room.byId.has(id)) room.byId.set(id, line); + } + return room; + } + + function appendSerialized(room: Room, path: string, line: string): Promise { + room.writeChain = room.writeChain.then(() => appendFile(path, `${line}\n`)); + return room.writeChain; + } + + function broadcast(room: Room, frame: SyncFrame, except?: WebSocket): void { + const encoded = encodeFrame(frame); + for (const subscriber of room.subscribers) { + if (subscriber === except) continue; + if (subscriber.readyState === subscriber.OPEN) subscriber.send(encoded); + } + } + + function send(socket: WebSocket, frame: SyncFrame): void { + if (socket.readyState === socket.OPEN) socket.send(encodeFrame(frame)); + } + + async function detach(socket: WebSocket, subscribed: Set): Promise { + for (const root of subscribed) { + const room = await rooms.get(root); + room?.subscribers.delete(socket); + } + } + + await new Promise((resolve, reject) => { + httpServer.once("error", reject); + httpServer.listen(options.port ?? 0, () => resolve()); + }); + const address = httpServer.address(); + if (address === null || typeof address === "string") { + throw new Error("lync serve: could not determine listening port"); + } + + return { + port: address.port, + close: async () => { + for (const socket of sockets) socket.terminate(); + await new Promise((resolve) => socketServer.close(() => resolve())); + await new Promise((resolve, reject) => { + httpServer.close((error) => (error ? reject(error) : resolve())); + }); + // Let every in-flight append land before we report closed. + for (const pending of rooms.values()) { + const room = await pending; + await room.writeChain; + } + }, + }; +} + +function authorized(request: IncomingMessage, token: string): boolean { + return request.headers.authorization === `Bearer ${token}`; +} + +function truncate(text: string): string { + return text.length > 80 ? `${text.slice(0, 77)}...` : text; +} diff --git a/packages/cli/src/sync.ts b/packages/cli/src/sync.ts new file mode 100644 index 0000000..7ca2624 --- /dev/null +++ b/packages/cli/src/sync.ts @@ -0,0 +1,217 @@ +import { appendFile, readFile, writeFile } from "node:fs/promises"; +import { existsSync, watch } from "node:fs"; +import { basename } from "node:path"; +import WebSocket from "ws"; +import { decodeFrame, encodeFrame, extractLineId } from "lync-core/sync-protocol"; + +/** + * `lync sync ` — one-shot convergence with a `lync serve` relay. + * + * The file itself is the offline queue: every local line is offered to the + * server (duplicates are no-ops under union), and every server line we lack + * is appended locally. The resume cursor lives in `.sync.json`; it + * advances only after a received line has reached a durable local state — + * appended, recognized as a duplicate, or surfaced as unusable. A sync that + * cannot reach `live` within the timeout fails loudly; nothing hangs. + */ + +export interface LyncSyncOptions { + file: string; + url: string; + root?: string; + timeoutMs?: number; + /** + * Stay connected after `live`: keep appending incoming events and push + * local appends as they land. Resolves when the socket closes or + * `stopSignal` aborts. + */ + follow?: boolean; + stopSignal?: AbortSignal; + out: Pick; + err: Pick; +} + +export interface LyncSyncResult { + sent: number; + received: number; + duplicates: number; + surfaced: number; + conflicts: number; + seq: number; +} + +interface Cursor { + url: string; + root: string; + seq: number; +} + +export async function syncOnce(options: LyncSyncOptions): Promise { + const root = options.root ?? defaultRoot(options.file); + const cursorPath = `${options.file}.sync.json`; + const cursor = await readCursor(cursorPath, options.url, root); + + let text = existsSync(options.file) ? await readFile(options.file, "utf8") : ""; + if (text.length > 0 && !text.endsWith("\n")) { + options.err.write(`lync sync: ${options.file} has a truncated final line; sealing it as damaged\n`); + await appendFile(options.file, "\n"); + text += "\n"; + } + const localLines = text.split("\n").filter((line) => line.length > 0); + const localIds = new Set(); + for (const line of localLines) { + const id = extractLineId(line); + if (id !== undefined) localIds.add(id); + } + + const socket = new WebSocket(options.url); + const result: LyncSyncResult = { sent: 0, received: 0, duplicates: 0, surfaced: 0, conflicts: 0, seq: cursor.seq }; + let appendChain = Promise.resolve(); + let following = false; + // Byte offset of everything we've already offered the server, so follow + // mode can push only newly appended complete lines. + let localOffset = Buffer.byteLength(text, "utf8"); + let watcher: import("node:fs").FSWatcher | undefined; + let pushChain = Promise.resolve(); + + const persistCursor = () => + writeFile(cursorPath, `${JSON.stringify({ url: options.url, root, seq: result.seq } satisfies Cursor, null, 2)}\n`); + + await new Promise((resolve, reject) => { + const timeout = setTimeout(() => { + reject(new Error(`lync sync: no 'live' from ${options.url} within ${options.timeoutMs ?? 15_000}ms`)); + socket.terminate(); + }, options.timeoutMs ?? 15_000); + + const stop = () => { + watcher?.close(); + resolve(); + }; + options.stopSignal?.addEventListener("abort", stop, { once: true }); + + function startFollowingLocalAppends(): void { + watcher = watch(options.file, () => { + pushChain = pushChain.then(async () => { + const current = await readFile(options.file, "utf8"); + const fresh = current.slice(localOffset); + const upto = fresh.lastIndexOf("\n"); + if (upto < 0) return; // no complete new line yet + for (const line of fresh.slice(0, upto).split("\n")) { + if (line.length === 0) continue; + const id = extractLineId(line); + if (id !== undefined && localIds.has(id)) continue; // our own echo, already appended + if (id !== undefined) localIds.add(id); + socket.send(encodeFrame({ t: "ev", root, line })); + result.sent += 1; + } + localOffset += Buffer.byteLength(fresh.slice(0, upto + 1), "utf8"); + }); + }); + } + + socket.on("close", () => { + clearTimeout(timeout); + watcher?.close(); + if (following) resolve(); + }); + + socket.on("error", (error) => { + clearTimeout(timeout); + watcher?.close(); + reject(error); + }); + + socket.on("open", () => { + // Push before subscribing: the server handles our frames in order, so + // every conflict or rejection for our own lines arrives before the + // backlog and `live`, and the backlog then covers our accepted lines + // (duplicate no-ops that settle the resume cursor in one pass). + for (const line of localLines) { + socket.send(encodeFrame({ t: "ev", root, line })); + result.sent += 1; + } + socket.send(encodeFrame({ t: "sub", root, since: cursor.seq })); + }); + + socket.on("message", (raw) => { + const frame = decodeFrame(raw.toString()); + switch (frame.t) { + case "ev": { + const id = extractLineId(frame.line); + if (id === undefined) { + options.err.write(`lync sync: server sent a line without an id; surfaced, not appended\n`); + result.surfaced += 1; + } else if (localIds.has(id)) { + result.duplicates += 1; + } else { + localIds.add(id); + result.received += 1; + const line = frame.line; + appendChain = appendChain.then(async () => { + await appendFile(options.file, `${line}\n`); + // Appends of our own making must not be re-pushed by the watcher. + localOffset += Buffer.byteLength(`${line}\n`, "utf8"); + }); + } + if (typeof frame.seq === "number") result.seq = Math.max(result.seq, frame.seq); + if (following) appendChain = appendChain.then(() => persistCursor()); + return; + } + case "live": { + clearTimeout(timeout); + result.seq = Math.max(result.seq, frame.seq); + if (!options.follow) { + resolve(); + return; + } + if (!following) { + following = true; + options.out.write(`lync sync: live at seq ${result.seq}; following (Ctrl-C to stop)\n`); + void persistCursor(); + startFollowingLocalAppends(); + } + return; + } + case "err": { + if (frame.reason === "same-id-conflict") { + result.conflicts += 1; + options.err.write(`lync sync: same-id conflict surfaced by server: ${frame.detail ?? "?"}\n`); + } else if (frame.reason === "recovered-damaged-tail") { + options.err.write(`lync sync: server note: ${frame.detail ?? frame.reason}\n`); + } else { + options.err.write(`lync sync: server error: ${frame.reason}${frame.detail ? ` (${frame.detail})` : ""}\n`); + } + return; + } + default: + return; + } + }); + }).finally(() => { + watcher?.close(); + socket.close(); + }); + + await appendChain; + await pushChain; + await persistCursor(); + return result; +} + +function defaultRoot(file: string): string { + return basename(file).replace(/\.lync$/, ""); +} + +async function readCursor(path: string, url: string, root: string): Promise { + if (!existsSync(path)) return { url, root, seq: 0 }; + try { + const stored = JSON.parse(await readFile(path, "utf8")) as Cursor; + if (stored.url === url && stored.root === root && typeof stored.seq === "number" && stored.seq >= 0) { + return stored; + } + // Different server or root: the stored cursor means nothing here. + return { url, root, seq: 0 }; + } catch { + return { url, root, seq: 0 }; + } +} diff --git a/packages/cli/test/sync.test.ts b/packages/cli/test/sync.test.ts new file mode 100644 index 0000000..82802e7 --- /dev/null +++ b/packages/cli/test/sync.test.ts @@ -0,0 +1,223 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { appendFile, mkdtemp, readFile, writeFile } from "node:fs/promises"; +import { existsSync } from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { startLyncServe, type LyncSyncServer } from "../src/serve.js"; +import { syncOnce } from "../src/sync.js"; + +const quiet = { write: () => true } as const; + +function collect() { + const chunks: string[] = []; + return { + io: { write: (chunk: string) => (chunks.push(chunk), true) }, + text: () => chunks.join(""), + }; +} + +function eventLine(id: string, parents: string[], text: string): string { + return JSON.stringify({ + v: 1, + id, + kind: "lync/artifact", + at: "2026-07-08T21:00:00Z", + author: { actor: "sync-test" }, + parents, + payload: { text }, + }); +} + +function idsOf(text: string): string[] { + return text + .split("\n") + .filter((line) => line.length > 0) + .map((line) => { + try { + return (JSON.parse(line) as { id?: string }).id ?? ""; + } catch { + return ""; + } + }) + .sort(); +} + +describe("lync serve + sync", () => { + let server: LyncSyncServer | undefined; + + afterEach(async () => { + await server?.close(); + server = undefined; + }); + + it("converges two divergent files through the relay", async () => { + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + + const fileA = path.join(clientDir, "story.lync"); + const fileB = path.join(clientDir, "b", "..", "story-b.lync"); + await writeFile(fileA, `${eventLine("root", [], "once")}\n${eventLine("a1", ["root"], "fork a")}\n`); + await writeFile(fileB, `${eventLine("root", [], "once")}\n${eventLine("b1", ["root"], "fork b")}\n`); + + await syncOnce({ file: fileA, url, root: "story", out: quiet, err: quiet }); + await syncOnce({ file: fileB, url, root: "story", out: quiet, err: quiet }); + await syncOnce({ file: fileA, url, root: "story", out: quiet, err: quiet }); + + const a = idsOf(await readFile(fileA, "utf8")); + const b = idsOf(await readFile(fileB, "utf8")); + expect(a).toEqual(["a1", "b1", "root"]); + expect(b).toEqual(a); + expect(new Set(a).size).toBe(a.length); + }); + + it("resumes exactly from the stored cursor: new events only, no re-fetch of the backlog", async () => { + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + + const fileA = path.join(clientDir, "a.lync"); + const fileB = path.join(clientDir, "b.lync"); + await writeFile(fileA, `${eventLine("root", [], "once")}\n`); + await writeFile(fileB, ""); + + await syncOnce({ file: fileA, url, root: "tale", out: quiet, err: quiet }); + await syncOnce({ file: fileA, url, root: "tale", out: quiet, err: quiet }); // settles cursor past echoes + const offlineCursor = JSON.parse(await readFile(`${fileA}.sync.json`, "utf8")) as { seq: number }; + expect(offlineCursor.seq).toBeGreaterThanOrEqual(1); + + // While A is offline, B contributes one new event. + await writeFile(fileB, `${eventLine("root", [], "once")}\n${eventLine("late", ["root"], "while away")}\n`); + await syncOnce({ file: fileB, url, root: "tale", out: quiet, err: quiet }); + + const result = await syncOnce({ file: fileA, url, root: "tale", out: quiet, err: quiet }); + expect(result.received).toBe(1); + const ids = idsOf(await readFile(fileA, "utf8")); + expect(ids).toEqual(["late", "root"]); + expect(new Set(ids).size).toBe(ids.length); + }); + + it("recovers a kill-9 truncated tail: sealed as damaged, surfaced, never eaten", async () => { + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + const full = eventLine("survivor", [], "made it"); + const partial = '{"v":1,"id":"torn","kind":"lync/artifa'; + await writeFile(path.join(serverDir, "crash.lync"), `${full}\n${partial}`); + + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + const file = path.join(clientDir, "crash.lync"); + await writeFile(file, ""); + + const errs = collect(); + const result = await syncOnce({ file, url, root: "crash", out: quiet, err: errs.io }); + + expect(result.received).toBe(1); // the survivor + expect(result.surfaced).toBe(1); // the sealed damaged tail: surfaced, not appended + expect(errs.text()).toContain("sealed truncated final line"); + expect(idsOf(await readFile(file, "utf8"))).toEqual(["survivor"]); + // The damaged bytes still live in the server file — sealed, never eaten. + const serverText = await readFile(path.join(serverDir, "crash.lync"), "utf8"); + expect(serverText).toContain(partial); + expect(serverText.endsWith("\n")).toBe(true); + }); + + it("surfaces same-id-different-body as a conflict on both sides and keeps both bytes", async () => { + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + + const fileA = path.join(clientDir, "a.lync"); + const fileB = path.join(clientDir, "b.lync"); + await writeFile(fileA, `${eventLine("same-id", [], "first telling")}\n`); + await writeFile(fileB, `${eventLine("same-id", [], "second telling")}\n`); + + await syncOnce({ file: fileA, url, root: "duel", out: quiet, err: quiet }); + const errs = collect(); + const result = await syncOnce({ file: fileB, url, root: "duel", out: quiet, err: errs.io }); + + expect(result.conflicts).toBe(1); + expect(errs.text()).toContain("same-id conflict surfaced by server"); + const sidecar = path.join(serverDir, "duel.conflicts"); + expect(existsSync(sidecar)).toBe(true); + expect(await readFile(sidecar, "utf8")).toContain("second telling"); + // B keeps its own bytes; the server keeps A's. Nothing was resolved. + expect(await readFile(fileB, "utf8")).toContain("second telling"); + expect(await readFile(path.join(serverDir, "duel.lync"), "utf8")).toContain("first telling"); + }); + + it("fails loudly instead of hanging when the relay is unreachable", async () => { + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + const file = path.join(clientDir, "alone.lync"); + await writeFile(file, ""); + await expect( + syncOnce({ file, url: "ws://localhost:9", root: "alone", timeoutMs: 3_000, out: quiet, err: quiet }), + ).rejects.toThrow(); + }); + + it("rejects unauthorized clients when a token is required", async () => { + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + server = await startLyncServe({ dir: serverDir, token: "sesame", log: () => {} }); + const file = path.join(clientDir, "locked.lync"); + await writeFile(file, ""); + await expect( + syncOnce({ file, url: `ws://localhost:${server.port}`, root: "locked", timeoutMs: 3_000, out: quiet, err: quiet }), + ).rejects.toThrow(); + }); +}); + +describe("lync sync --follow", () => { + let server: LyncSyncServer | undefined; + + afterEach(async () => { + await server?.close(); + server = undefined; + }); + + it("streams events both ways while live", async () => { + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + + const fileA = path.join(clientDir, "a.lync"); + const fileB = path.join(clientDir, "b.lync"); + await writeFile(fileA, `${eventLine("root", [], "once")}\n`); + await writeFile(fileB, ""); + + const stopper = new AbortController(); + const follower = syncOnce({ + file: fileA, url, root: "livewire", follow: true, stopSignal: stopper.signal, out: quiet, err: quiet, + }); + + // Wait until the follower's push landed on the relay. + await waitFor(async () => existsSync(path.join(serverDir, "livewire.lync"))); + + // A remote peer contributes while A is following: A receives it live. + await writeFile(fileB, `${eventLine("root", [], "once")}\n${eventLine("remote", ["root"], "from B")}\n`); + await syncOnce({ file: fileB, url, root: "livewire", out: quiet, err: quiet }); + await waitFor(async () => (await readFile(fileA, "utf8")).includes('"remote"')); + + // A local append while following: pushed to the relay without re-syncing. + await appendFile(fileA, `${eventLine("local-live", ["root"], "typed live")}\n`); + await waitFor(async () => (await readFile(path.join(serverDir, "livewire.lync"), "utf8")).includes('"local-live"')); + + stopper.abort(); + const result = await follower; + expect(result.received).toBeGreaterThanOrEqual(1); + expect(idsOf(await readFile(fileA, "utf8"))).toEqual(["local-live", "remote", "root"]); + }); +}); + +async function waitFor(check: () => Promise, timeoutMs = 5_000): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (await check()) return; + await new Promise((resolve) => setTimeout(resolve, 25)); + } + throw new Error("waitFor: condition not met within timeout"); +} diff --git a/packages/core/package.json b/packages/core/package.json index 4d58c78..2d66dab 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -82,6 +82,11 @@ "types": "./dist/types.d.ts", "import": "./dist/types.js", "default": "./dist/types.js" + }, + "./sync-protocol": { + "types": "./dist/sync-protocol.d.ts", + "import": "./dist/sync-protocol.js", + "default": "./dist/sync-protocol.js" } }, "files": [ diff --git a/packages/core/src/sync-protocol.ts b/packages/core/src/sync-protocol.ts new file mode 100644 index 0000000..78eb9cf --- /dev/null +++ b/packages/core/src/sync-protocol.ts @@ -0,0 +1,131 @@ +/** + * The lync line-sync protocol: a dumb event-union relay. + * + * Events are immutable and merge is union by id, so the protocol has no + * merge logic at all — it moves canonical line bytes between peers and lets + * union make redundancy harmless. Five frame kinds: + * + * client → server {"t":"sub", "root": string, "since": number} + * server → client {"t":"ev", "root": string, "seq": number, "line": string} + * server → client {"t":"live", "root": string, "seq": number} + * client → server {"t":"ev", "root": string, "line": string} + * either direction {"t":"presence", "root": string, "data"?: unknown} + * either direction {"t":"err", "root"?: string, "reason": string, "detail"?: string} + * + * `seq` is the server's own per-root arrival counter — a resume cursor, not + * event order. The server echoes accepted events to every subscriber of the + * root, sender included; echoes are duplicate no-ops under union and still + * advance the cursor. This module is pure: frame codecs and guards only. + */ + +export interface SubFrame { + t: "sub"; + root: string; + since: number; +} + +export interface EvFrame { + t: "ev"; + root: string; + line: string; + seq?: number; +} + +export interface LiveFrame { + t: "live"; + root: string; + seq: number; +} + +export interface PresenceFrame { + t: "presence"; + root: string; + data?: unknown; +} + +export interface ErrFrame { + t: "err"; + root?: string; + reason: string; + detail?: string; +} + +export type SyncFrame = SubFrame | EvFrame | LiveFrame | PresenceFrame | ErrFrame; + +const FRAME_KINDS = new Set(["sub", "ev", "live", "presence", "err"]); + +export function encodeFrame(frame: SyncFrame): string { + return JSON.stringify(frame); +} + +/** + * Decode one frame. Returns an ErrFrame (never throws) on anything + * malformed, so transport code stays loud without try/catch pyramids. + */ +export function decodeFrame(raw: string | Uint8Array): SyncFrame { + const text = typeof raw === "string" ? raw : new TextDecoder().decode(raw); + let value: unknown; + try { + value = JSON.parse(text); + } catch (error) { + return { t: "err", reason: "malformed-frame", detail: String(error) }; + } + if (typeof value !== "object" || value === null || Array.isArray(value)) { + return { t: "err", reason: "malformed-frame", detail: "frame is not an object" }; + } + const frame = value as Record; + if (typeof frame.t !== "string" || !FRAME_KINDS.has(frame.t)) { + return { t: "err", reason: "unknown-frame-kind", detail: String(frame.t) }; + } + switch (frame.t) { + case "sub": + if (typeof frame.root !== "string" || typeof frame.since !== "number" || frame.since < 0) { + return { t: "err", reason: "malformed-sub" }; + } + return { t: "sub", root: frame.root, since: frame.since }; + case "ev": + if (typeof frame.root !== "string" || typeof frame.line !== "string") { + return { t: "err", reason: "malformed-ev" }; + } + return { + t: "ev", + root: frame.root, + line: frame.line, + ...(typeof frame.seq === "number" ? { seq: frame.seq } : {}), + }; + case "live": + if (typeof frame.root !== "string" || typeof frame.seq !== "number") { + return { t: "err", reason: "malformed-live" }; + } + return { t: "live", root: frame.root, seq: frame.seq }; + case "presence": + if (typeof frame.root !== "string") { + return { t: "err", reason: "malformed-presence" }; + } + return { t: "presence", root: frame.root, data: frame.data }; + default: + return { + t: "err", + reason: typeof frame.reason === "string" ? frame.reason : "unspecified", + ...(typeof frame.root === "string" ? { root: frame.root } : {}), + ...(typeof frame.detail === "string" ? { detail: frame.detail } : {}), + }; + } +} + +/** + * Extract the event id from a canonical line without trusting anything else + * in it. The relay never parses beyond this. Returns undefined when no id + * can be extracted — callers surface that loudly, never drop it silently. + */ +export function extractLineId(line: string): string | undefined { + try { + const value = JSON.parse(line); + if (typeof value === "object" && value !== null && typeof (value as { id?: unknown }).id === "string") { + return (value as { id: string }).id; + } + } catch { + return undefined; + } + return undefined; +} diff --git a/packages/core/test/sync-protocol.test.ts b/packages/core/test/sync-protocol.test.ts new file mode 100644 index 0000000..8d856f2 --- /dev/null +++ b/packages/core/test/sync-protocol.test.ts @@ -0,0 +1,32 @@ +import { describe, expect, it } from "vitest"; +import { decodeFrame, encodeFrame, extractLineId, type SyncFrame } from "lync-core/sync-protocol"; + +describe("lync sync protocol frames", () => { + it("round-trips every frame kind", () => { + const frames: SyncFrame[] = [ + { t: "sub", root: "story", since: 0 }, + { t: "ev", root: "story", line: '{"id":"a"}', seq: 3 }, + { t: "ev", root: "story", line: '{"id":"b"}' }, + { t: "live", root: "story", seq: 7 }, + { t: "presence", root: "story", data: { cursor: 4 } }, + { t: "err", root: "story", reason: "same-id-conflict", detail: "a" }, + ]; + for (const frame of frames) { + expect(decodeFrame(encodeFrame(frame))).toEqual(frame); + } + }); + + it("returns err frames for malformed input instead of throwing", () => { + expect(decodeFrame("not json").t).toBe("err"); + expect(decodeFrame("[1,2]").t).toBe("err"); + expect(decodeFrame('{"t":"warp"}')).toMatchObject({ t: "err", reason: "unknown-frame-kind" }); + expect(decodeFrame('{"t":"sub","root":"r","since":-1}')).toMatchObject({ t: "err", reason: "malformed-sub" }); + expect(decodeFrame('{"t":"live","root":"r"}')).toMatchObject({ t: "err", reason: "malformed-live" }); + }); + + it("extracts ids without trusting the rest of the line", () => { + expect(extractLineId('{"id":"x","junk":{"id":"y"}}')).toBe("x"); + expect(extractLineId("{broken")).toBeUndefined(); + expect(extractLineId('{"noid":true}')).toBeUndefined(); + }); +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index e519b6d..f243f29 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -26,6 +26,13 @@ importers: lync-core: specifier: workspace:* version: link:../core + ws: + specifier: ^8.18.0 + version: 8.21.0 + devDependencies: + '@types/ws': + specifier: ^8.18.1 + version: 8.18.1 packages/core: {} @@ -327,6 +334,9 @@ packages: '@types/node@22.19.17': resolution: {integrity: sha512-wGdMcf+vPYM6jikpS/qhg6WiqSV/OhG+jeeHT/KlVqxYfD40iYJf9/AE1uQxVWFvU7MipKRkRv8NSHiCGgPr8Q==} + '@types/ws@8.18.1': + resolution: {integrity: sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg==} + '@vitest/expect@3.2.4': resolution: {integrity: sha512-Io0yyORnB6sikFlt8QW5K7slY4OjqNX9jmJQ02QDda8lyM6B5oNgVWoSoKPac8/kgnCUzuHQKrSLtu/uOqqrig==} @@ -578,6 +588,18 @@ packages: engines: {node: '>=8'} hasBin: true + ws@8.21.0: + resolution: {integrity: sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==} + engines: {node: '>=10.0.0'} + peerDependencies: + bufferutil: ^4.0.1 + utf-8-validate: '>=5.0.2' + peerDependenciesMeta: + bufferutil: + optional: true + utf-8-validate: + optional: true + snapshots: '@esbuild/aix-ppc64@0.27.7': @@ -748,6 +770,10 @@ snapshots: dependencies: undici-types: 6.21.0 + '@types/ws@8.18.1': + dependencies: + '@types/node': 22.19.17 + '@vitest/expect@3.2.4': dependencies: '@types/chai': 5.2.3 @@ -1020,3 +1046,5 @@ snapshots: dependencies: siginfo: 2.0.0 stackback: 0.0.2 + + ws@8.21.0: {} From 17949b438c9b363455e4ee63787f76d6fc4be7e0 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 22:23:24 -0700 Subject: [PATCH 11/33] embedded reactive sync: the browser story, on the native protocol MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CLI file-sync shipped earlier served terminal users; it did not serve the real consumer story — an app that embeds lync, opens looms through an index, and shows live collaborative edits in-process. textile is exactly that (its data path is already union-based; its /lync socket was only a heartbeat). This builds the missing piece. lync-core: - createSyncedStore(store, transport): wraps any EventStore. Local appends push to the relay; remote lines ingest through union. Looms and indexes recompute reactively via the store's existing subscribe — no merge logic, because immutable events + union-by-id make redundancy harmless. - createWebSocketTransport: zero-dep, browser + Node (global WebSocket), reconnecting, queues sends while offline. Transport is an interface, so the decorator is testable without sockets. - loomRootId(loomId): the store root for a loom, so apps can sync a loom's root before opening it. Revived on the native protocol (Automerge dropped entirely): - lync-index: the memory loom index (reactive subscribe), no automerge. - lync-client: createLoomClient facade (looms + indexes + reference resolution), the automerge repo coupling and transport files removed. Proof: - core unit tests (mock transport): push, reactive remote ingest with no echo loop, status, reconnect re-push. - cli integration test (real relay, two clients over global WebSocket): a turn on client A reaches client B's loom reactively, both directions. - client facade test; index memory test. - 83 tests, 30 consecutive green full-suite runs including live sockets; all four packages typecheck; fresh-clone smoke green. - clean-room from tarballs: all four packages installed in a fresh project, a relay started from the bin, two embedded clients converge reactively. Published surface is now four packages: lync-core, lync-cli, lync-index, lync-client. --- README.md | 39 ++- .../cli/test/synced-store.integration.test.ts | 88 ++++++ packages/client/README.md | 3 + packages/client/package.json | 43 +++ packages/client/src/create.ts | 90 ++++++ packages/client/src/index.ts | 2 + packages/client/src/types.ts | 75 +++++ packages/client/test/create.test.ts | 46 +++ packages/client/tsconfig.json | 8 + packages/core/package.json | 5 + packages/core/src/looms.ts | 11 + packages/core/src/synced-store.ts | 290 ++++++++++++++++++ packages/core/test/synced-store.test.ts | 119 +++++++ packages/index/README.md | 3 + packages/index/package.json | 52 ++++ packages/index/src/entries.ts | 17 + packages/index/src/index.ts | 2 + packages/index/src/memory.ts | 239 +++++++++++++++ packages/index/src/types.ts | 78 +++++ packages/index/test/memory.test.ts | 97 ++++++ packages/index/tsconfig.json | 8 + pnpm-lock.yaml | 15 + vitest.config.ts | 8 + 23 files changed, 1335 insertions(+), 3 deletions(-) create mode 100644 packages/cli/test/synced-store.integration.test.ts create mode 100644 packages/client/README.md create mode 100644 packages/client/package.json create mode 100644 packages/client/src/create.ts create mode 100644 packages/client/src/index.ts create mode 100644 packages/client/src/types.ts create mode 100644 packages/client/test/create.test.ts create mode 100644 packages/client/tsconfig.json create mode 100644 packages/core/src/synced-store.ts create mode 100644 packages/core/test/synced-store.test.ts create mode 100644 packages/index/README.md create mode 100644 packages/index/package.json create mode 100644 packages/index/src/entries.ts create mode 100644 packages/index/src/index.ts create mode 100644 packages/index/src/memory.ts create mode 100644 packages/index/src/types.ts create mode 100644 packages/index/test/memory.test.ts create mode 100644 packages/index/tsconfig.json diff --git a/README.md b/README.md index bd055af..a751cc1 100644 --- a/README.md +++ b/README.md @@ -68,9 +68,14 @@ The short version: ## Packages -- `lync-core`: format parsing, event stores, computed views, references, and - the loom API. No runtime dependencies. -- `lync-cli`: the `lync` command — `init`, `append`, `verify`, `merge`, `view`. +- `lync-core`: format parsing, event stores, computed views, references, the + loom API, and live sync (`createSyncedStore`). No runtime dependencies. +- `lync-cli`: the `lync` command — `init`, `append`, `verify`, `merge`, `view`, + `serve`, `sync`. +- `lync-index`: an index of many looms, with reactive subscription. Depends + only on `lync-core`. +- `lync-client`: the loom client — resolves references and opens looms and + indexes. Depends on `lync-core` and `lync-index`. ## Format-Layer Imports @@ -210,6 +215,34 @@ A truncated final line after a crash is sealed and surfaced as damaged, never eaten. `--token T` on the server requires `Authorization: Bearer T` to connect. +### Sync inside an app + +The same protocol runs in a browser or Node app with no CLI. Wrap any event +store in `createSyncedStore`; looms and indexes built over it update live as +collaborators append, because they already recompute through the store's +`subscribe`: + +```ts +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createLyncLooms } from "lync-core/looms"; +import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; + +const transport = createWebSocketTransport("wss://host/lync"); +const store = createSyncedStore(createMemoryEventStore(), transport, { + onStatus: (s) => console.log("sync:", s.connection, "live:", s.liveRoots), +}); +const looms = createLyncLooms({ store, author: { actor: "alice" } }); + +const loom = await looms.open(loomId); +loom.subscribe(() => render(loom)); // fires on local AND remote turns +await loom.appendTurn(parentId, { text: "typed live" }); +``` + +Local appends are pushed to the relay; remote lines are ingested through the +same `union` path and surface reactively. Offline appends queue and flush on +reconnect; the store re-subscribes automatically. The transport is an +interface — pass your own for tests or a non-WebSocket carrier. + ## Development ```bash diff --git a/packages/cli/test/synced-store.integration.test.ts b/packages/cli/test/synced-store.integration.test.ts new file mode 100644 index 0000000..4d966da --- /dev/null +++ b/packages/cli/test/synced-store.integration.test.ts @@ -0,0 +1,88 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { mkdtemp } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createLyncLooms, loomRootId } from "lync-core/looms"; +import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; +import { startLyncServe, type LyncSyncServer } from "../src/serve.js"; + +/** + * The embedded browser story, proven end to end: a real relay, two clients + * over the global WebSocket, looms built on synced stores. A turn appended on + * client A appears in client B's loom reactively — the thing textile's + * status-only socket never actually did. + */ + +async function waitFor(check: () => Promise | boolean, timeoutMs = 5_000): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (await check()) return; + await new Promise((r) => setTimeout(r, 20)); + } + throw new Error("waitFor: condition not met within timeout"); +} + +function client(url: string, actor: string) { + const inner = createMemoryEventStore(); + const transport = createWebSocketTransport(url, { reconnectMs: 0 }); + const store = createSyncedStore(inner, transport); + const looms = createLyncLooms<{ text: string }, { title: string }, { role: string }>({ + store, + author: { actor }, + }); + return { store, looms }; +} + +describe("embedded synced looms over a real relay", () => { + let server: LyncSyncServer | undefined; + const closers: Array<() => void> = []; + + afterEach(async () => { + for (const close of closers.splice(0)) close(); + await server?.close(); + server = undefined; + }); + + it("delivers a turn appended on client A to client B's loom, reactively", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "lync-embed-")); + server = await startLyncServe({ dir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + + const a = client(url, "alice"); + const b = client(url, "bob"); + closers.push(a.store.close, b.store.close); + + // A creates a loom and seeds a turn. + const info = await a.looms.create({ title: "shared story" }); + const loomA = await a.looms.open(info.id); + const first = await loomA.appendTurn(null, { text: "Once upon a time" }, { role: "prose" }); + + // B learns the loom id (in an app this comes via the index), syncs its + // root, waits for the backlog, then opens it. + const root = loomRootId(info.id); + b.store.syncRoot(root); + await waitFor(async () => (await b.store.byId(root)) !== null); + const loomB = await b.looms.open(info.id); + + let bReacted = 0; + loomB.subscribe(() => { + bReacted += 1; + }); + + // A appends a second turn (child of the first) AFTER B is watching. + const second = await loomA.appendTurn(first.id, { text: "the end." }, { role: "prose" }); + + // B converges on the second turn and its full thread reads root→first→second. + await waitFor(async () => (await b.store.byId(second.id)) !== null); + const threadB = await loomB.threadTo(second.id); + expect(threadB.map((t) => t.payload.text)).toEqual(["Once upon a time", "the end."]); + expect(bReacted).toBeGreaterThanOrEqual(1); // B's loom recomputed live + + // And the reverse direction: B appends, A sees it live. + const reply = await loomB.appendTurn(second.id, { text: "a reply from bob" }, { role: "prose" }); + await waitFor(async () => (await a.store.byId(reply.id)) !== null); + const threadA = await loomA.threadTo(reply.id); + expect(threadA.map((t) => t.payload.text)).toEqual(["Once upon a time", "the end.", "a reply from bob"]); + }); +}); diff --git a/packages/client/README.md b/packages/client/README.md new file mode 100644 index 0000000..e81f66c --- /dev/null +++ b/packages/client/README.md @@ -0,0 +1,3 @@ +# lync-client + +The lync loom client: a small facade over looms and indexes that resolves loom/turn/thread/index references. See https://github.com/deepfates/lync. diff --git a/packages/client/package.json b/packages/client/package.json new file mode 100644 index 0000000..fabd43f --- /dev/null +++ b/packages/client/package.json @@ -0,0 +1,43 @@ +{ + "name": "lync-client", + "version": "0.2.0", + "description": "The lync loom client: resolve references, open looms and indexes.", + "type": "module", + "license": "MIT", + "sideEffects": false, + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js", + "default": "./dist/index.js" + }, + "./types": { + "types": "./dist/types.d.ts", + "import": "./dist/types.js", + "default": "./dist/types.js" + } + }, + "files": [ + "dist" + ], + "scripts": { + "build": "tsc -p tsconfig.json", + "typecheck": "tsc -p tsconfig.json --noEmit" + }, + "dependencies": { + "lync-core": "workspace:*", + "lync-index": "workspace:*" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepfates/lync.git", + "directory": "packages/client" + }, + "homepage": "https://github.com/deepfates/lync#readme", + "bugs": "https://github.com/deepfates/lync/issues", + "publishConfig": { + "access": "public" + } +} diff --git a/packages/client/src/create.ts b/packages/client/src/create.ts new file mode 100644 index 0000000..2b1d700 --- /dev/null +++ b/packages/client/src/create.ts @@ -0,0 +1,90 @@ +import { + brokenTopology, + decodeReference, + encodeReference, + indexRef, + loomRef, + parseReference, + referenceFromUrl, + referenceToUrl, + threadRef, + turnRef, + type LoomReference, + type Looms, +} from "lync-core"; +import type { LoomIndexes } from "lync-index"; +import type { LoomClient } from "./types.js"; + +export interface CreateLoomClientOptions< + TPayload = unknown, + TLoomMeta = unknown, + TTurnMeta = unknown, + TEntryMeta = unknown, + TIndexMeta = unknown, +> { + looms: Looms; + indexes: LoomIndexes; + close?: () => Promise | void; +} + +export function createLoomClient< + TPayload = unknown, + TLoomMeta = unknown, + TTurnMeta = unknown, + TEntryMeta = unknown, + TIndexMeta = unknown, +>( + options: CreateLoomClientOptions< + TPayload, + TLoomMeta, + TTurnMeta, + TEntryMeta, + TIndexMeta + >, +): LoomClient { + const { looms, indexes } = options; + + return { + looms, + indexes, + references: { + loom: loomRef, + turn: turnRef, + thread: threadRef, + index: indexRef, + encode: encodeReference, + decode: decodeReference, + parse: parseReference, + toUrl: referenceToUrl, + fromUrl: referenceFromUrl, + }, + async openReference(ref: LoomReference) { + switch (ref.kind) { + case "loom": { + const loom = await looms.open(ref.loomId); + return { kind: "loom", ref, loom }; + } + case "turn": { + const loom = await looms.open(ref.loomId); + const turn = await loom.getTurn(ref.turnId); + if (!turn) throw brokenTopology(`Reference target turn not found: ${ref.turnId}`); + return { kind: "turn", ref, loom, turn }; + } + case "thread": { + const loom = await looms.open(ref.loomId); + const thread = await loom.threadTo(ref.turnId); + const target = thread.at(-1); + if (!target) throw brokenTopology(`Reference target thread is empty: ${ref.turnId}`); + return { kind: "thread", ref, loom, thread, target }; + } + case "index": { + const index = await indexes.open(ref.indexId); + return { kind: "index", ref, index }; + } + } + }, + async close() { + await options.close?.(); + }, + }; +} diff --git a/packages/client/src/index.ts b/packages/client/src/index.ts new file mode 100644 index 0000000..7a12a9d --- /dev/null +++ b/packages/client/src/index.ts @@ -0,0 +1,2 @@ +export * from "./create.js"; +export * from "./types.js"; diff --git a/packages/client/src/types.ts b/packages/client/src/types.ts new file mode 100644 index 0000000..c560206 --- /dev/null +++ b/packages/client/src/types.ts @@ -0,0 +1,75 @@ +import type { + Loom, + LoomReference, + Looms, + Turn, + indexRef, + loomRef, + referenceFromUrl, + referenceToUrl, + threadRef, + turnRef, + encodeReference, + decodeReference, + parseReference, +} from "lync-core"; +import type { LoomIndex, LoomIndexes } from "lync-index"; + +export type ReferenceHelpers = { + loom: typeof loomRef; + turn: typeof turnRef; + thread: typeof threadRef; + index: typeof indexRef; + encode: typeof encodeReference; + decode: typeof decodeReference; + parse: typeof parseReference; + toUrl: typeof referenceToUrl; + fromUrl: typeof referenceFromUrl; +}; + +export type OpenedReference< + TPayload = unknown, + TLoomMeta = unknown, + TTurnMeta = unknown, + TEntryMeta = unknown, + TIndexMeta = unknown, +> = + | { + kind: "loom"; + ref: Extract; + loom: Loom; + } + | { + kind: "turn"; + ref: Extract; + loom: Loom; + turn: Turn; + } + | { + kind: "thread"; + ref: Extract; + loom: Loom; + thread: Turn[]; + target: Turn; + } + | { + kind: "index"; + ref: Extract; + index: LoomIndex; + }; + +export interface LoomClient< + TPayload = unknown, + TLoomMeta = unknown, + TTurnMeta = unknown, + TEntryMeta = unknown, + TIndexMeta = unknown, +> { + looms: Looms; + indexes: LoomIndexes; + references: ReferenceHelpers; + openReference( + ref: LoomReference, + ): Promise>; + close(): Promise; +} diff --git a/packages/client/test/create.test.ts b/packages/client/test/create.test.ts new file mode 100644 index 0000000..f1d23ee --- /dev/null +++ b/packages/client/test/create.test.ts @@ -0,0 +1,46 @@ +import { describe, expect, it } from "vitest"; +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createLyncLooms } from "lync-core/looms"; +import { createMemoryLoomIndexes } from "lync-index/memory"; +import { upsertLoom } from "lync-index/entries"; +import { createLoomClient } from "../src/create.js"; + +function makeClient() { + const store = createMemoryEventStore(); + const looms = createLyncLooms<{ text: string }, { title: string }, { role: string }>({ + store, + author: { actor: "tester" }, + }); + const indexes = createMemoryLoomIndexes<{ note: string }, { app: string }>(); + return createLoomClient({ looms, indexes }); +} + +describe("createLoomClient", () => { + it("composes looms and indexes and resolves a loom reference", async () => { + const client = makeClient(); + const info = await client.looms.create({ title: "Story" }); + + const opened = await client.openReference(client.references.loom(info.id)); + expect(opened.kind).toBe("loom"); + if (opened.kind === "loom") { + const turn = await opened.loom.appendTurn(null, { text: "hi" }, { role: "prose" }); + expect(turn.payload.text).toBe("hi"); + } + await client.close(); + }); + + it("resolves an index reference and lists upserted looms", async () => { + const client = makeClient(); + const index = await client.indexes.create({ app: "test" }); + const loom = await client.looms.create({ title: "In Index" }); + await upsertLoom(index, client.references.loom(loom.id), { note: "first" }); + + const opened = await client.openReference(client.references.index(index.id)); + expect(opened.kind).toBe("index"); + if (opened.kind === "index") { + const entries = await opened.index.entries(); + expect(entries).toHaveLength(1); + } + await client.close(); + }); +}); diff --git a/packages/client/tsconfig.json b/packages/client/tsconfig.json new file mode 100644 index 0000000..df59da5 --- /dev/null +++ b/packages/client/tsconfig.json @@ -0,0 +1,8 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "dist" + }, + "include": ["src/**/*.ts"] +} diff --git a/packages/core/package.json b/packages/core/package.json index 2d66dab..5500dd8 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -87,6 +87,11 @@ "types": "./dist/sync-protocol.d.ts", "import": "./dist/sync-protocol.js", "default": "./dist/sync-protocol.js" + }, + "./synced-store": { + "types": "./dist/synced-store.d.ts", + "import": "./dist/synced-store.js", + "default": "./dist/synced-store.js" } }, "files": [ diff --git a/packages/core/src/looms.ts b/packages/core/src/looms.ts index 24abbb3..ac8a4e8 100644 --- a/packages/core/src/looms.ts +++ b/packages/core/src/looms.ts @@ -374,6 +374,17 @@ function rootId(id: LoomId): string | null { return id.startsWith(LYNC_PREFIX) ? id.slice(LYNC_PREFIX.length) : null; } +/** + * The event-store root id for a loom. A loom id is a `lync:`-prefixed handle; + * its events live under the bare root. Apps use this to sync a loom's root on + * a synced store before opening it. + */ +export function loomRootId(loomId: LoomId): string { + const root = rootId(loomId); + if (!root) throw new Error(`not a lync loom id: ${loomId}`); + return root; +} + function validateAuthor(author: LyncAuthor): void { if (!author || typeof author.actor !== "string" || author.actor.length === 0) { throw new Error("Lync author.actor is required"); diff --git a/packages/core/src/synced-store.ts b/packages/core/src/synced-store.ts new file mode 100644 index 0000000..d0e0db7 --- /dev/null +++ b/packages/core/src/synced-store.ts @@ -0,0 +1,290 @@ +import type { EventStore, StoredEvent, AppendResult } from "./store.js"; +import type { LyncEventBody } from "./events.js"; +import { decodeFrame, encodeFrame, type SyncFrame } from "./sync-protocol.js"; + +/** + * Live sync for an EventStore, built on the dumb line-union protocol. + * + * A synced store is an ordinary EventStore: looms and indexes built over it + * recompute reactively through its existing `subscribe`, whether an event + * arrived from a local append or a remote peer. The decorator adds exactly + * two behaviors — local appends are pushed to the relay, and remote lines are + * ingested through the same `union` path — because immutable events plus + * union-by-id make redundancy harmless and merge logic unnecessary. + * + * The transport is abstract so this is testable without a socket. A + * browser-and-Node WebSocket transport is provided below with zero + * dependencies (global `WebSocket`, present in browsers and Node ≥ 21). + */ + +export type SyncConnectionState = "connecting" | "online" | "offline"; + +export interface SyncStatus { + connection: SyncConnectionState; + /** Roots that have received their backlog and are live. */ + liveRoots: string[]; + /** Ids that arrived as same-id-different-body conflicts, surfaced never resolved. */ + conflicts: string[]; +} + +export interface SyncTransport { + send(frame: SyncFrame): void; + /** Register a frame handler; returns an unsubscribe. */ + onFrame(handler: (frame: SyncFrame) => void): () => void; + /** Fires on every (re)connection, so callers can re-subscribe. */ + onOpen(handler: () => void): () => void; + onStateChange(handler: (state: SyncConnectionState) => void): () => void; + readonly state: SyncConnectionState; + close(): void; +} + +export interface SyncedStoreOptions { + onStatus?: (status: SyncStatus) => void; + onPresence?: (root: string, data: unknown) => void; +} + +export interface SyncedStore extends EventStore { + /** Begin syncing a root: push local backlog, then subscribe from the cursor. */ + syncRoot(rootId: string): void; + /** Relay an ephemeral presence frame for a root; never stored. */ + presence(root: string, data: unknown): void; + status(): SyncStatus; + close(): void; +} + +export function createSyncedStore( + inner: EventStore, + transport: SyncTransport, + options: SyncedStoreOptions = {}, +): SyncedStore { + const syncedRoots = new Set(); + const liveRoots = new Set(); + const cursors = new Map(); + const conflicts = new Set(); + let connection: SyncConnectionState = transport.state; + + const emitStatus = () => { + options.onStatus?.({ + connection, + liveRoots: [...liveRoots], + conflicts: [...conflicts], + }); + }; + + const pushLine = (root: string, line: string) => { + transport.send({ t: "ev", root, line }); + }; + + const ensureSynced = (rootId: string) => { + if (syncedRoots.has(rootId)) return; + syncedRoots.add(rootId); + void resync(rootId); + }; + + // Push a locally-added event. If its root isn't synced yet, begin syncing — + // resync re-pushes the whole local backlog (this event included), so we must + // not also push it here, or the relay sees it twice. + const pushAdded = (event: StoredEvent) => { + if (syncedRoots.has(event.root)) { + pushLine(event.root, event.bytes); + } else { + ensureSynced(event.root); + } + }; + + async function resync(rootId: string): Promise { + if (transport.state !== "online") return; // resumes on the next onOpen + // Push any local events the relay may not have (offline appends included); + // duplicates are no-ops under union on the server. + for (const event of await inner.byRoot(rootId)) { + pushLine(rootId, event.bytes); + } + transport.send({ t: "sub", root: rootId, since: cursors.get(rootId) ?? 0 }); + } + + transport.onOpen(() => { + for (const rootId of syncedRoots) void resync(rootId); + }); + + transport.onStateChange((state) => { + connection = state; + if (state !== "online") liveRoots.clear(); + emitStatus(); + }); + + transport.onFrame((frame) => { + switch (frame.t) { + case "ev": { + // Remote line: ingest through union WITHOUT re-pushing (the relay has + // already fanned it out). Subscribers fire via the inner store. + void inner.union(frame.line); + if (typeof frame.seq === "number") { + cursors.set(frame.root, Math.max(cursors.get(frame.root) ?? 0, frame.seq)); + } + return; + } + case "live": { + cursors.set(frame.root, Math.max(cursors.get(frame.root) ?? 0, frame.seq)); + liveRoots.add(frame.root); + emitStatus(); + return; + } + case "presence": { + options.onPresence?.(frame.root, frame.data); + return; + } + case "err": { + if (frame.reason === "same-id-conflict" && frame.detail) { + conflicts.add(frame.detail); + emitStatus(); + } + return; + } + default: + return; + } + }); + + return { + async append(ev: LyncEventBody): Promise { + const result = await inner.append(ev); + if (result.status === "added") pushAdded(result.event); + return result; + }, + async union(line: string): Promise { + const result = await inner.union(line); + if (result.status === "added") pushAdded(result.event); + return result; + }, + byId: (id) => inner.byId(id), + byRoot: (rootId) => { + ensureSynced(rootId); + return inner.byRoot(rootId); + }, + subscribe: (rootId, listener) => { + ensureSynced(rootId); + return inner.subscribe(rootId, listener); + }, + roots: (kind) => inner.roots(kind), + ...(inner.exportRootBytes ? { exportRootBytes: (rootId: string) => inner.exportRootBytes!(rootId) } : {}), + ...(inner.diagnostics ? { diagnostics: () => inner.diagnostics!() } : {}), + syncRoot: ensureSynced, + presence: (root, data) => transport.send({ t: "presence", root, data }), + status: () => ({ connection, liveRoots: [...liveRoots], conflicts: [...conflicts] }), + close: () => transport.close(), + }; +} + +export interface WebSocketTransportOptions { + /** Override the WebSocket constructor (e.g. `ws` in Node < 21, or a fake in tests). */ + WebSocketImpl?: typeof WebSocket; + /** Reconnect backoff in ms. Default 1500. Set 0 to disable auto-reconnect. */ + reconnectMs?: number; +} + +/** + * A reconnecting WebSocket transport. Sends while offline are queued and + * flushed on connect; a dropped socket schedules a reconnect and the synced + * store re-subscribes via `onOpen`. Nothing is silently dropped: an unsent + * frame waits in the queue rather than vanishing. + */ +export function createWebSocketTransport(url: string, options: WebSocketTransportOptions = {}): SyncTransport { + const WS = options.WebSocketImpl ?? (globalThis as { WebSocket?: typeof WebSocket }).WebSocket; + if (!WS) { + throw new Error("createWebSocketTransport: no WebSocket implementation available; pass options.WebSocketImpl"); + } + const reconnectMs = options.reconnectMs ?? 1500; + const frameHandlers = new Set<(frame: SyncFrame) => void>(); + const openHandlers = new Set<() => void>(); + const stateHandlers = new Set<(state: SyncConnectionState) => void>(); + const queue: SyncFrame[] = []; + + let socket: WebSocket | undefined; + let state: SyncConnectionState = "connecting"; + let closed = false; + let reconnectTimer: ReturnType | undefined; + + const setState = (next: SyncConnectionState) => { + if (state === next) return; + state = next; + for (const handler of stateHandlers) handler(next); + }; + + const connect = () => { + if (closed) return; + setState("connecting"); + const ws = new WS(url); + socket = ws; + ws.addEventListener("open", () => { + if (socket !== ws) return; + setState("online"); + while (queue.length > 0) ws.send(encodeFrame(queue.shift()!)); + for (const handler of openHandlers) handler(); + }); + ws.addEventListener("message", (event: MessageEvent) => { + if (socket !== ws) return; + const raw = typeof event.data === "string" ? event.data : String(event.data); + const frame = decodeFrame(raw); + for (const handler of frameHandlers) handler(frame); + }); + ws.addEventListener("close", () => { + if (socket !== ws) return; + socket = undefined; + setState("offline"); + scheduleReconnect(); + }); + ws.addEventListener("error", () => { + if (socket !== ws) return; + // A failed connection surfaces as a close on most stacks; force it. + try { + ws.close(); + } catch { + socket = undefined; + setState("offline"); + scheduleReconnect(); + } + }); + }; + + const scheduleReconnect = () => { + if (closed || reconnectMs <= 0 || reconnectTimer) return; + reconnectTimer = setTimeout(() => { + reconnectTimer = undefined; + connect(); + }, reconnectMs); + }; + + connect(); + + return { + send(frame) { + if (socket && state === "online" && socket.readyState === socket.OPEN) { + socket.send(encodeFrame(frame)); + } else { + queue.push(frame); + } + }, + onFrame(handler) { + frameHandlers.add(handler); + return () => frameHandlers.delete(handler); + }, + onOpen(handler) { + openHandlers.add(handler); + return () => openHandlers.delete(handler); + }, + onStateChange(handler) { + stateHandlers.add(handler); + return () => stateHandlers.delete(handler); + }, + get state() { + return state; + }, + close() { + closed = true; + if (reconnectTimer) clearTimeout(reconnectTimer); + socket?.close(); + socket = undefined; + setState("offline"); + }, + }; +} diff --git a/packages/core/test/synced-store.test.ts b/packages/core/test/synced-store.test.ts new file mode 100644 index 0000000..b83f4ec --- /dev/null +++ b/packages/core/test/synced-store.test.ts @@ -0,0 +1,119 @@ +import { describe, expect, it } from "vitest"; +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createLyncLooms } from "lync-core/looms"; +import { + createSyncedStore, + type SyncConnectionState, + type SyncStatus, + type SyncTransport, +} from "lync-core/synced-store"; +import type { SyncFrame } from "lync-core/sync-protocol"; +import { serializeLyncEvent } from "lync-core/store"; + +function mockTransport(initial: SyncConnectionState = "online") { + const frameHandlers = new Set<(frame: SyncFrame) => void>(); + const openHandlers = new Set<() => void>(); + const stateHandlers = new Set<(state: SyncConnectionState) => void>(); + const sent: SyncFrame[] = []; + let state = initial; + const transport: SyncTransport = { + send: (frame) => sent.push(frame), + onFrame: (h) => (frameHandlers.add(h), () => frameHandlers.delete(h)), + onOpen: (h) => (openHandlers.add(h), () => openHandlers.delete(h)), + onStateChange: (h) => (stateHandlers.add(h), () => stateHandlers.delete(h)), + get state() { + return state; + }, + close: () => {}, + }; + return { + transport, + sent, + inject: (frame: SyncFrame) => frameHandlers.forEach((h) => h(frame)), + open: () => openHandlers.forEach((h) => h()), + setState: (next: SyncConnectionState) => { + state = next; + stateHandlers.forEach((h) => h(next)); + }, + }; +} + +const body = (id: string, parents: string[], text: string) => ({ + v: 1 as const, + id, + kind: "lync/artifact", + at: "2026-07-08T21:00:00Z", + author: { actor: "test" }, + parents, + payload: { text }, +}); + +describe("createSyncedStore", () => { + it("pushes local appends to the relay and subscribes to the touched root", async () => { + const mock = mockTransport(); + const store = createSyncedStore(createMemoryEventStore(), mock.transport); + const result = await store.append(body("root", [], "hello")); + expect(result.status).toBe("added"); + + const evFrames = mock.sent.filter((f) => f.t === "ev"); + const subFrames = mock.sent.filter((f) => f.t === "sub"); + expect(evFrames).toHaveLength(1); + expect(evFrames[0]).toMatchObject({ t: "ev" }); + expect(subFrames.length).toBeGreaterThanOrEqual(1); + }); + + it("ingests a remote event reactively — a root subscriber fires — without echoing it back", async () => { + const mock = mockTransport(); + const store = createSyncedStore(createMemoryEventStore(), mock.transport); + + // Establish the root locally, then watch it for reactive updates. + await store.union(serializeLyncEvent(body("r1", [], "seed"))); + let fired = 0; + store.subscribe("r1", () => { + fired += 1; + }); + + const sentBefore = mock.sent.length; + // A remote peer's event arrives over the transport. + mock.inject({ t: "ev", root: "r1", seq: 2, line: serializeLyncEvent(body("remote", ["r1"], "from afar")) }); + await new Promise((r) => setTimeout(r, 10)); + + expect(fired).toBeGreaterThanOrEqual(1); // the subscriber recomputed reactively + expect(await store.byId("remote")).not.toBeNull(); // ingested via union + // The remote line was NOT re-pushed to the relay (no echo loop). + const freshEv = mock.sent.slice(sentBefore).filter((f) => f.t === "ev" && f.line.includes("from afar")); + expect(freshEv).toHaveLength(0); + }); + + it("reflects live and conflict frames in status", async () => { + const statuses: SyncStatus[] = []; + const mock = mockTransport(); + const store = createSyncedStore(createMemoryEventStore(), mock.transport, { + onStatus: (s) => statuses.push(s), + }); + store.syncRoot("r1"); + mock.inject({ t: "live", root: "r1", seq: 5 }); + mock.inject({ t: "err", root: "r1", reason: "same-id-conflict", detail: "dup-id" }); + + const status = store.status(); + expect(status.liveRoots).toContain("r1"); + expect(status.conflicts).toContain("dup-id"); + expect(statuses.length).toBeGreaterThanOrEqual(2); + }); + + it("re-pushes local backlog and re-subscribes on reconnect", async () => { + const mock = mockTransport("online"); + const store = createSyncedStore(createMemoryEventStore(), mock.transport); + await store.append(body("root", [], "made offline-ish")); + const before = mock.sent.length; + + mock.setState("offline"); + mock.setState("online"); + mock.open(); // transport re-announces + await new Promise((r) => setTimeout(r, 10)); + + const after = mock.sent.slice(before); + expect(after.some((f) => f.t === "sub" && f.root === "root")).toBe(true); + expect(after.some((f) => f.t === "ev")).toBe(true); // backlog re-pushed + }); +}); diff --git a/packages/index/README.md b/packages/index/README.md new file mode 100644 index 0000000..bca19b4 --- /dev/null +++ b/packages/index/README.md @@ -0,0 +1,3 @@ +# lync-index + +The lync index: track a collection of looms and subscribe to changes. Depends only on `lync-core`. See https://github.com/deepfates/lync. diff --git a/packages/index/package.json b/packages/index/package.json new file mode 100644 index 0000000..7e9de2a --- /dev/null +++ b/packages/index/package.json @@ -0,0 +1,52 @@ +{ + "name": "lync-index", + "version": "0.2.0", + "description": "An index of many lync looms, with reactive subscription.", + "type": "module", + "license": "MIT", + "sideEffects": false, + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js", + "default": "./dist/index.js" + }, + "./entries": { + "types": "./dist/entries.d.ts", + "import": "./dist/entries.js", + "default": "./dist/entries.js" + }, + "./memory": { + "types": "./dist/memory.d.ts", + "import": "./dist/memory.js", + "default": "./dist/memory.js" + }, + "./types": { + "types": "./dist/types.d.ts", + "import": "./dist/types.js", + "default": "./dist/types.js" + } + }, + "files": [ + "dist" + ], + "scripts": { + "build": "tsc -p tsconfig.json", + "typecheck": "tsc -p tsconfig.json --noEmit" + }, + "dependencies": { + "lync-core": "workspace:*" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepfates/lync.git", + "directory": "packages/index" + }, + "homepage": "https://github.com/deepfates/lync#readme", + "bugs": "https://github.com/deepfates/lync/issues", + "publishConfig": { + "access": "public" + } +} diff --git a/packages/index/src/entries.ts b/packages/index/src/entries.ts new file mode 100644 index 0000000..f7405cb --- /dev/null +++ b/packages/index/src/entries.ts @@ -0,0 +1,17 @@ +import type { LoomReference } from "lync-core"; +import type { + LoomIndex, + LoomIndexEntry, + LoomIndexEntryInput, +} from "./types.js"; + +export async function upsertLoom( + index: LoomIndex, + ref: Extract, + entry: LoomIndexEntryInput = {}, +): Promise> { + if (await index.has(ref.loomId)) { + return index.updateLoom(ref.loomId, entry); + } + return index.addLoom(ref, entry); +} diff --git a/packages/index/src/index.ts b/packages/index/src/index.ts new file mode 100644 index 0000000..c82e54d --- /dev/null +++ b/packages/index/src/index.ts @@ -0,0 +1,2 @@ +export * from "./entries.js"; +export * from "./types.js"; diff --git a/packages/index/src/memory.ts b/packages/index/src/memory.ts new file mode 100644 index 0000000..a4c0dc5 --- /dev/null +++ b/packages/index/src/memory.ts @@ -0,0 +1,239 @@ +import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "lync-core"; +import type { IndexId, LoomId, LoomReference } from "lync-core"; +import type { + LoomIndex, + LoomIndexEntry, + LoomIndexEntryInput, + LoomIndexEntryPatch, + LoomIndexes, + LoomIndexEvent, + LoomIndexInfo, + LoomIndexListener, + LoomIndexSnapshot, + MemoryLoomIndexesOptions, +} from "./types.js"; + +export type { MemoryLoomIndexesOptions } from "./types.js"; + +type InternalIndex = { + info: LoomIndexInfo; + entries: Map>; + order: LoomId[]; + listeners: Set>; +}; + +export function createMemoryLoomIndexes< + TEntryMeta = unknown, + TIndexMeta = unknown, +>(options: MemoryLoomIndexesOptions = {}): LoomIndexes { + const createId = options.createId ?? (() => crypto.randomUUID()); + const now = options.now ?? (() => Date.now()); + const indexes = new Map>(); + + const createInternal = (meta?: TIndexMeta): InternalIndex => { + assertJsonEncodable(meta, "index meta"); + return { + info: omitUndefined({ + id: `memory-index:${createId()}`, + meta: cloneJson(meta), + createdAt: now(), + }), + entries: new Map(), + order: [], + listeners: new Set(), + }; + }; + + return { + async create(meta) { + const index = createInternal(meta); + indexes.set(index.info.id, index); + return new MemoryLoomIndex(index.info.id, index, now); + }, + + async open(indexId) { + const index = indexes.get(indexId); + if (!index) throw unknownIndex(indexId); + return new MemoryLoomIndex(indexId, index, now); + }, + + async import(snapshot) { + validateSnapshot(snapshot); + const index = createInternal(snapshot.index.meta); + index.info.createdAt = snapshot.index.createdAt; + for (const entry of snapshot.entries) { + const cloned = cloneJson(entry); + index.entries.set(cloned.ref.loomId, cloned); + index.order.push(cloned.ref.loomId); + } + indexes.set(index.info.id, index); + return new MemoryLoomIndex(index.info.id, index, now); + }, + }; +} + +class MemoryLoomIndex + implements LoomIndex +{ + private closed = false; + + constructor( + readonly id: IndexId, + private readonly index: InternalIndex, + private readonly now: () => number, + ) {} + + async info(): Promise> { + this.assertOpen(); + return cloneJson(this.index.info); + } + + async updateMeta(meta: TIndexMeta): Promise> { + this.assertOpen(); + assertJsonEncodable(meta, "index meta"); + this.index.info = omitUndefined({ ...this.index.info, meta: cloneJson(meta) }); + this.emit({ type: "index-updated", index: cloneJson(this.index.info) }); + return cloneJson(this.index.info); + } + + async entries(): Promise[]> { + this.assertOpen(); + return this.index.order.map((loomId) => { + const entry = this.index.entries.get(loomId); + if (!entry) throw new LoomError("BROKEN_TOPOLOGY", `Index order references missing loom: ${loomId}`); + return cloneJson(entry); + }); + } + + async get(loomId: LoomId): Promise | null> { + this.assertOpen(); + const entry = this.index.entries.get(loomId); + return entry ? cloneJson(entry) : null; + } + + async has(loomId: LoomId): Promise { + this.assertOpen(); + return this.index.entries.has(loomId); + } + + async addLoom( + ref: Extract, + input: LoomIndexEntryInput = {}, + ): Promise> { + this.assertOpen(); + assertJsonEncodable(input, "index entry"); + if (this.index.entries.has(ref.loomId)) { + throw duplicateLoomId(ref.loomId); + } + const entry = omitUndefined({ + ref: loomRef(ref.loomId), + title: input.title, + kind: input.kind, + meta: cloneJson(input.meta), + addedAt: this.now(), + updatedAt: input.updatedAt, + }) as LoomIndexEntry; + this.index.entries.set(ref.loomId, entry); + this.index.order.push(ref.loomId); + const output = cloneJson(entry); + this.emit({ type: "entry-added", indexId: this.id, entry: output }); + return output; + } + + async updateLoom( + loomId: LoomId, + patch: LoomIndexEntryPatch, + ): Promise> { + this.assertOpen(); + assertJsonEncodable(patch, "index entry patch"); + const existing = this.index.entries.get(loomId); + if (!existing) throw new LoomError("UNKNOWN_LOOM", `Index does not contain loom: ${loomId}`); + const updated = omitUndefined({ + ...existing, + ...patch, + meta: patch.meta === undefined ? existing.meta : cloneJson(patch.meta), + updatedAt: patch.updatedAt ?? this.now(), + }) as LoomIndexEntry; + this.index.entries.set(loomId, updated); + const output = cloneJson(updated); + this.emit({ type: "entry-updated", indexId: this.id, entry: output }); + return output; + } + + async removeLoom(loomId: LoomId): Promise { + this.assertOpen(); + if (!this.index.entries.has(loomId)) return; + this.index.entries.delete(loomId); + this.index.order = this.index.order.filter((candidate) => candidate !== loomId); + this.emit({ type: "entry-removed", indexId: this.id, loomId }); + } + + subscribe(listener: LoomIndexListener): () => void { + this.assertOpen(); + this.index.listeners.add(listener); + return () => this.index.listeners.delete(listener); + } + + async export(): Promise> { + this.assertOpen(); + return cloneJson({ + index: this.index.info, + entries: await this.entries(), + }); + } + + close(): void { + this.closed = true; + } + + private assertOpen() { + if (this.closed) throw new LoomError("CLOSED_HANDLE", "This loom index handle is closed"); + } + + private emit(event: LoomIndexEvent): void { + for (const listener of this.index.listeners) listener(event); + } +} + +function validateSnapshot(snapshot: LoomIndexSnapshot): void { + if (!snapshot || typeof snapshot !== "object") { + throw new LoomError("INVALID_SNAPSHOT", "Index snapshot must be an object"); + } + if (!snapshot.index || typeof snapshot.index.id !== "string") { + throw new LoomError("INVALID_SNAPSHOT", "Index snapshot needs an index id"); + } + if (!Array.isArray(snapshot.entries)) { + throw new LoomError("INVALID_SNAPSHOT", "Index snapshot entries must be an array"); + } + assertJsonEncodable(snapshot, "index snapshot"); + const seen = new Set(); + for (const entry of snapshot.entries) { + if (!entry?.ref || entry.ref.kind !== "loom" || typeof entry.ref.loomId !== "string") { + throw new LoomError("INVALID_SNAPSHOT", "Every index entry needs a loom reference"); + } + if (seen.has(entry.ref.loomId)) { + throw duplicateLoomId(entry.ref.loomId); + } + seen.add(entry.ref.loomId); + } +} + +function omitUndefined>(value: T): T { + return Object.fromEntries( + Object.entries(value).filter(([, entryValue]) => entryValue !== undefined), + ) as T; +} + +function assertJsonEncodable(value: unknown, label: string): void { + if (value === undefined) return; + try { + JSON.stringify(value); + } catch { + throw new LoomError("INVALID_SNAPSHOT", `${label} must be JSON-encodable`); + } +} + +function cloneJson(value: T): T { + if (value === undefined) return value; + return JSON.parse(JSON.stringify(value)) as T; +} diff --git a/packages/index/src/types.ts b/packages/index/src/types.ts new file mode 100644 index 0000000..a65448f --- /dev/null +++ b/packages/index/src/types.ts @@ -0,0 +1,78 @@ +import type { IndexId, LoomId, LoomReference } from "lync-core"; + +export interface LoomIndexInfo { + id: IndexId; + meta?: TIndexMeta; + createdAt: number; +} + +export interface LoomIndexEntry { + ref: Extract; + title?: string; + kind?: string; + meta?: TEntryMeta; + addedAt: number; + updatedAt?: number; +} + +export type LoomIndexEntryInput = Partial< + Omit, "ref" | "addedAt"> +>; + +export type LoomIndexEntryPatch = Partial< + Pick, "title" | "kind" | "meta" | "updatedAt"> +>; + +export interface LoomIndexSnapshot { + index: LoomIndexInfo; + entries: LoomIndexEntry[]; +} + +export type LoomIndexEvent = + | { type: "entry-added"; indexId: IndexId; entry: LoomIndexEntry } + | { type: "entry-updated"; indexId: IndexId; entry: LoomIndexEntry } + | { type: "entry-removed"; indexId: IndexId; loomId: LoomId } + | { type: "index-updated"; index: LoomIndexInfo } + | { type: "sync-state"; indexId: IndexId; online: boolean; syncing: boolean }; + +export type LoomIndexListener = ( + event: LoomIndexEvent, +) => void; + +export interface LoomIndex { + id: IndexId; + + info(): Promise>; + updateMeta(meta: TIndexMeta): Promise>; + + entries(): Promise[]>; + get(loomId: LoomId): Promise | null>; + has(loomId: LoomId): Promise; + + addLoom( + ref: Extract, + entry?: LoomIndexEntryInput, + ): Promise>; + updateLoom( + loomId: LoomId, + patch: LoomIndexEntryPatch, + ): Promise>; + removeLoom(loomId: LoomId): Promise; + + subscribe(listener: LoomIndexListener): () => void; + export(): Promise>; + close(): void; +} + +export interface LoomIndexes { + create(meta?: TIndexMeta): Promise>; + open(indexId: IndexId): Promise>; + import( + snapshot: LoomIndexSnapshot, + ): Promise>; +} + +export interface MemoryLoomIndexesOptions { + createId?: () => string; + now?: () => number; +} diff --git a/packages/index/test/memory.test.ts b/packages/index/test/memory.test.ts new file mode 100644 index 0000000..26315aa --- /dev/null +++ b/packages/index/test/memory.test.ts @@ -0,0 +1,97 @@ +import { describe, expect, it } from "vitest"; +import { loomRef } from "lync-core"; +import { createMemoryLoomIndexes } from "../src/memory.js"; +import { upsertLoom } from "../src/entries.js"; + +function deterministicIndexes() { + let nextId = 0; + let nextTime = 2000; + return createMemoryLoomIndexes<{ app: string }, { owner: string }>({ + createId: () => `idx-${++nextId}`, + now: () => nextTime++, + }); +} + +describe("memory loom indexes", () => { + it("creates an index and stores ordered loom references", async () => { + const indexes = deterministicIndexes(); + const index = await indexes.create({ owner: "me" }); + + const first = await index.addLoom(loomRef("lync:first"), { + title: "First", + kind: "story", + meta: { app: "textile" }, + }); + const second = await index.addLoom(loomRef("lync:second"), { title: "Second" }); + + expect(await index.entries()).toEqual([first, second]); + expect(await index.get("lync:first")).toEqual(first); + expect(await index.has("lync:missing")).toBe(false); + }); + + it("updates and removes loom links without implying loom deletion", async () => { + const indexes = deterministicIndexes(); + const index = await indexes.create(); + await index.addLoom(loomRef("lync:first"), { title: "First" }); + + const updated = await index.updateLoom("lync:first", { + title: "Renamed", + kind: "story", + }); + expect(updated.title).toBe("Renamed"); + expect(updated.updatedAt).toBe(2002); + + await index.removeLoom("lync:first"); + expect(await index.entries()).toEqual([]); + }); + + it("emits entry events and exports/imports deterministic snapshots with a new index id", async () => { + const indexes = deterministicIndexes(); + const index = await indexes.create({ owner: "me" }); + const events: string[] = []; + index.subscribe((event) => events.push(event.type)); + + await index.addLoom(loomRef("lync:first"), { title: "First" }); + await index.updateLoom("lync:first", { title: "Renamed" }); + await index.removeLoom("lync:first"); + + expect(events).toEqual(["entry-added", "entry-updated", "entry-removed"]); + + await index.addLoom(loomRef("lync:first"), { title: "First" }); + const snapshot = await index.export(); + const imported = await indexes.import(snapshot); + + expect(imported.id).not.toBe(index.id); + expect(await imported.entries()).toEqual(snapshot.entries); + }); + + it("rejects duplicate loom links", async () => { + const indexes = deterministicIndexes(); + const index = await indexes.create(); + await index.addLoom(loomRef("lync:first")); + + await expect(index.addLoom(loomRef("lync:first"))).rejects.toMatchObject({ + code: "DUPLICATE_LOOM_ID", + }); + }); + + it("upserts loom links so shared imports can refresh metadata", async () => { + const indexes = deterministicIndexes(); + const index = await indexes.create(); + + const added = await upsertLoom(index, loomRef("lync:first"), { + title: "First", + kind: "story", + meta: { app: "old" }, + }); + const updated = await upsertLoom(index, loomRef("lync:first"), { + title: "Renamed", + kind: "story", + meta: { app: "new" }, + }); + + expect(added.addedAt).toBe(updated.addedAt); + expect(updated.updatedAt).toBe(2002); + expect(await index.entries()).toEqual([updated]); + }); +}); diff --git a/packages/index/tsconfig.json b/packages/index/tsconfig.json new file mode 100644 index 0000000..df59da5 --- /dev/null +++ b/packages/index/tsconfig.json @@ -0,0 +1,8 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "dist" + }, + "include": ["src/**/*.ts"] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index f243f29..1998af4 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -34,8 +34,23 @@ importers: specifier: ^8.18.1 version: 8.18.1 + packages/client: + dependencies: + lync-core: + specifier: workspace:* + version: link:../core + lync-index: + specifier: workspace:* + version: link:../index + packages/core: {} + packages/index: + dependencies: + lync-core: + specifier: workspace:* + version: link:../core + packages: '@esbuild/aix-ppc64@0.27.7': diff --git a/vitest.config.ts b/vitest.config.ts index 6e7df0c..8c21d62 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -18,6 +18,14 @@ export default defineConfig({ find: /^lync-core$/, replacement: new URL("./packages/core/src/index.ts", import.meta.url).pathname, }, + { + find: /^lync-index\/([a-z0-9-]+)$/, + replacement: new URL("./packages/index/src/", import.meta.url).pathname + "$1.ts", + }, + { + find: /^lync-index$/, + replacement: new URL("./packages/index/src/index.ts", import.meta.url).pathname, + }, ], }, test: { From 0f856a23155a021d24977aef66343f1aaeb18024 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 22:55:42 -0700 Subject: [PATCH 12/33] lync-server: the embeddable relay package MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Extract the relay from the CLI into lync-server so apps can embed it: - createLyncRelay({ dir, token?, authenticate?, onConnection? }): handleUpgrade / handleConnection / close — mount on any Node server. - attachLyncServer(server, { storageDir, path, authenticate, keepAliveInterval, maxConnections }): adds an upgrade listener for one path, passes the rest through. This is the shape textile's server mounts. - startLyncServe(): standalone, used by the CLI. The CLI now imports startLyncServe from lync-server instead of its own copy. lync-client gains createTestLoomClient (memory looms + index, deterministic with createId/now) for app tests. Tests: relay mounted on an app server converges two clients; attach filters its path and passes others through; authenticate rejection blocks a client. Published surface is now five packages. 86 tests, all typecheck. --- packages/cli/package.json | 3 +- packages/cli/src/index.ts | 2 +- packages/cli/test/sync.test.ts | 2 +- .../cli/test/synced-store.integration.test.ts | 2 +- packages/client/package.json | 5 + packages/client/src/testing.ts | 37 +++++ packages/server/README.md | 7 + packages/server/package.json | 41 ++++++ packages/server/src/attach.ts | 80 +++++++++++ packages/server/src/index.ts | 3 + .../{cli/src/serve.ts => server/src/relay.ts} | 126 +++++++++--------- packages/server/src/serve.ts | 45 +++++++ packages/server/test/attach.test.ts | 80 +++++++++++ packages/server/test/relay.test.ts | 74 ++++++++++ packages/server/tsconfig.json | 8 ++ pnpm-lock.yaml | 16 +++ 16 files changed, 463 insertions(+), 68 deletions(-) create mode 100644 packages/client/src/testing.ts create mode 100644 packages/server/README.md create mode 100644 packages/server/package.json create mode 100644 packages/server/src/attach.ts create mode 100644 packages/server/src/index.ts rename packages/{cli/src/serve.ts => server/src/relay.ts} (67%) create mode 100644 packages/server/src/serve.ts create mode 100644 packages/server/test/attach.test.ts create mode 100644 packages/server/test/relay.test.ts create mode 100644 packages/server/tsconfig.json diff --git a/packages/cli/package.json b/packages/cli/package.json index a3aef25..e8dc88f 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -37,7 +37,8 @@ }, "dependencies": { "lync-core": "workspace:*", - "ws": "^8.18.0" + "ws": "^8.18.0", + "lync-server": "workspace:*" }, "devDependencies": { "@types/ws": "^8.18.1" diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index 3bfcbd6..4d3269e 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -374,7 +374,7 @@ async function serveVerb( out: Pick, err: Pick, ): Promise { - const { startLyncServe } = await import("./serve.js"); + const { startLyncServe } = await import("lync-server"); const positional: string[] = []; let port: number | undefined; let token: string | undefined; diff --git a/packages/cli/test/sync.test.ts b/packages/cli/test/sync.test.ts index 82802e7..b18fdf9 100644 --- a/packages/cli/test/sync.test.ts +++ b/packages/cli/test/sync.test.ts @@ -3,7 +3,7 @@ import { appendFile, mkdtemp, readFile, writeFile } from "node:fs/promises"; import { existsSync } from "node:fs"; import os from "node:os"; import path from "node:path"; -import { startLyncServe, type LyncSyncServer } from "../src/serve.js"; +import { startLyncServe, type LyncSyncServer } from "lync-server"; import { syncOnce } from "../src/sync.js"; const quiet = { write: () => true } as const; diff --git a/packages/cli/test/synced-store.integration.test.ts b/packages/cli/test/synced-store.integration.test.ts index 4d966da..5aacc13 100644 --- a/packages/cli/test/synced-store.integration.test.ts +++ b/packages/cli/test/synced-store.integration.test.ts @@ -5,7 +5,7 @@ import path from "node:path"; import { createMemoryEventStore } from "lync-core/memory-log"; import { createLyncLooms, loomRootId } from "lync-core/looms"; import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; -import { startLyncServe, type LyncSyncServer } from "../src/serve.js"; +import { startLyncServe, type LyncSyncServer } from "lync-server"; /** * The embedded browser story, proven end to end: a real relay, two clients diff --git a/packages/client/package.json b/packages/client/package.json index fabd43f..6fca14d 100644 --- a/packages/client/package.json +++ b/packages/client/package.json @@ -17,6 +17,11 @@ "types": "./dist/types.d.ts", "import": "./dist/types.js", "default": "./dist/types.js" + }, + "./testing": { + "types": "./dist/testing.d.ts", + "import": "./dist/testing.js", + "default": "./dist/testing.js" } }, "files": [ diff --git a/packages/client/src/testing.ts b/packages/client/src/testing.ts new file mode 100644 index 0000000..4e1e0f8 --- /dev/null +++ b/packages/client/src/testing.ts @@ -0,0 +1,37 @@ +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createLyncLooms, type LyncAuthor } from "lync-core/looms"; +import { createMemoryLoomIndexes } from "lync-index/memory"; +import { createLoomClient } from "./create.js"; +import type { LoomClient } from "./types.js"; + +/** + * A fully in-memory loom client for tests and embedded experiments: looms and + * an index over memory event stores, no network. Deterministic when you pass + * `createId` and `now`. + */ +export interface TestLoomClientOptions { + author?: LyncAuthor; + createId?: () => string; + now?: () => number; +} + +export function createTestLoomClient< + TPayload = unknown, + TLoomMeta = unknown, + TTurnMeta = unknown, + TEntryMeta = unknown, + TIndexMeta = unknown, +>(options: TestLoomClientOptions = {}): LoomClient { + const store = createMemoryEventStore(); + const looms = createLyncLooms({ + store, + author: options.author ?? { actor: "test" }, + createId: options.createId, + now: options.now, + }); + const indexes = createMemoryLoomIndexes({ + createId: options.createId, + now: options.now, + }); + return createLoomClient({ looms, indexes }); +} diff --git a/packages/server/README.md b/packages/server/README.md new file mode 100644 index 0000000..dc8e6b9 --- /dev/null +++ b/packages/server/README.md @@ -0,0 +1,7 @@ +# lync-server + +The lync line-sync relay. Run it standalone with `startLyncServe`, or embed it +in an existing Node HTTP server with `createLyncRelay` and call `handleUpgrade` +from your own `upgrade` listener. It stores each root as a plain append-only +`.lync` file and never parses a line beyond its id. See +https://github.com/deepfates/lync. diff --git a/packages/server/package.json b/packages/server/package.json new file mode 100644 index 0000000..1d548f2 --- /dev/null +++ b/packages/server/package.json @@ -0,0 +1,41 @@ +{ + "name": "lync-server", + "version": "0.2.0", + "description": "The lync line-sync relay: mountable on any Node server, or run standalone.", + "type": "module", + "license": "MIT", + "sideEffects": false, + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js", + "default": "./dist/index.js" + } + }, + "files": [ + "dist" + ], + "scripts": { + "build": "tsc -p tsconfig.json", + "typecheck": "tsc -p tsconfig.json --noEmit" + }, + "dependencies": { + "lync-core": "workspace:*", + "ws": "^8.18.0" + }, + "devDependencies": { + "@types/ws": "^8.18.1" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepfates/lync.git", + "directory": "packages/server" + }, + "homepage": "https://github.com/deepfates/lync#readme", + "bugs": "https://github.com/deepfates/lync/issues", + "publishConfig": { + "access": "public" + } +} diff --git a/packages/server/src/attach.ts b/packages/server/src/attach.ts new file mode 100644 index 0000000..816924e --- /dev/null +++ b/packages/server/src/attach.ts @@ -0,0 +1,80 @@ +import type { IncomingMessage, Server } from "node:http"; +import type { Duplex } from "node:stream"; +import type { WebSocket } from "ws"; +import { createLyncRelay, type LyncRelayOptions } from "./relay.js"; + +/** + * Mount the relay on an existing Node HTTP server. Adds an `upgrade` listener + * that handles only `path` (default `/lync`) and passes the rest through, so a + * web app can share one server between its routes and lync sync. + */ + +export interface AttachLyncServerOptions extends Omit { + /** Directory of per-root append-only files. */ + storageDir: string; + /** URL path to serve the relay on. Default `/lync`. */ + path?: string; + /** Send a WebSocket ping on this interval (ms) to keep proxies from idling out. */ + keepAliveInterval?: number; + /** Reject upgrades once this many sockets are connected. */ + maxConnections?: number; +} + +export interface AttachedLyncServer { + close: () => Promise; +} + +export function attachLyncServer(server: Server, options: AttachLyncServerOptions): AttachedLyncServer { + const path = options.path ?? "/lync"; + const log = options.log ?? ((message: string) => process.stderr.write(`${message}\n`)); + const live = new Set(); + + const pingTimer = + options.keepAliveInterval && options.keepAliveInterval > 0 + ? setInterval(() => { + for (const ws of live) { + if (ws.readyState === ws.OPEN) { + try { + ws.ping(); + } catch { + live.delete(ws); + } + } + } + }, options.keepAliveInterval) + : undefined; + pingTimer?.unref?.(); + + const relay = createLyncRelay({ + dir: options.storageDir, + token: options.token, + authenticate: options.authenticate, + log, + onConnection: (ws) => { + live.add(ws); + ws.on("close", () => live.delete(ws)); + }, + }); + + const onUpgrade = (request: IncomingMessage, socket: Duplex, head: Buffer) => { + if ((request.url ?? "").split("?")[0] !== path) return; + if (options.maxConnections !== undefined && live.size >= options.maxConnections) { + log(`[lync relay] rejecting upgrade: at max connections (${options.maxConnections})`); + socket.write("HTTP/1.1 503 Service Unavailable\r\n\r\n"); + socket.destroy(); + return; + } + relay.handleUpgrade(request, socket, head); + }; + + server.on("upgrade", onUpgrade); + + return { + close: async () => { + server.off("upgrade", onUpgrade); + if (pingTimer) clearInterval(pingTimer); + live.clear(); + await relay.close(); + }, + }; +} diff --git a/packages/server/src/index.ts b/packages/server/src/index.ts new file mode 100644 index 0000000..e1869cc --- /dev/null +++ b/packages/server/src/index.ts @@ -0,0 +1,3 @@ +export { createLyncRelay, type LyncRelay, type LyncRelayOptions } from "./relay.js"; +export { startLyncServe, type LyncServeOptions, type LyncSyncServer } from "./serve.js"; +export { attachLyncServer, type AttachLyncServerOptions, type AttachedLyncServer } from "./attach.js"; diff --git a/packages/cli/src/serve.ts b/packages/server/src/relay.ts similarity index 67% rename from packages/cli/src/serve.ts rename to packages/server/src/relay.ts index 007c6f1..c64b184 100644 --- a/packages/cli/src/serve.ts +++ b/packages/server/src/relay.ts @@ -1,33 +1,46 @@ -import { createServer, type IncomingMessage, type Server } from "node:http"; import { appendFile, mkdir, readFile } from "node:fs/promises"; import { existsSync } from "node:fs"; import { join } from "node:path"; +import type { IncomingMessage } from "node:http"; +import type { Duplex } from "node:stream"; import { WebSocketServer, type WebSocket } from "ws"; import { decodeFrame, encodeFrame, extractLineId, type SyncFrame } from "lync-core/sync-protocol"; /** - * `lync serve` — the dumb event-union relay from the line-sync design. + * The lync line-sync relay — a dumb event-union relay, mountable on any Node + * HTTP server or run standalone. * * One append-only `.lync` file per root. `seq` is the per-root count of - * stored lines: a resume cursor, nothing more. The server never parses a line - * beyond extracting its id. Accepted events fan out to every subscriber of - * the root, sender included — echoes are duplicate no-ops under union. - * Same-id-different-body is never resolved: both sides keep their bytes, the - * variant line goes to a `.conflicts` sidecar, and an err frame goes to - * everyone. Presence frames are relayed and never stored. Nothing fails - * invisibly: malformed input earns an err frame, damaged recovery is loud. + * stored lines: a resume cursor, nothing more. The relay never parses a line + * beyond extracting its id. Accepted events fan out to every subscriber of the + * root, sender included — echoes are duplicate no-ops under union. Same-id, + * different-body is never resolved: both sides keep their bytes, the variant + * goes to a `.conflicts` sidecar, and an err frame goes to everyone. + * Presence is relayed, never stored. Nothing fails invisibly. */ -export interface LyncServeOptions { +export interface LyncRelayOptions { + /** Directory of per-root append-only files. Created if absent. */ dir: string; - port?: number; + /** If set, upgrades require `Authorization: Bearer `. */ token?: string; + /** + * Per-upgrade authorization. Return false to reject. Runs after the token + * check (if any). Use for cookie/session auth on an embedded relay. + */ + authenticate?: (request: IncomingMessage) => boolean | Promise; + /** Called with each wired socket; for connection counting and keepalive. */ + onConnection?: (socket: WebSocket) => void; log?: (message: string) => void; } -export interface LyncSyncServer { - port: number; - close: () => Promise; +export interface LyncRelay { + /** Handle an HTTP upgrade: authorize, upgrade, and wire the socket. */ + handleUpgrade(request: IncomingMessage, socket: Duplex, head: Buffer): void; + /** Wire a socket you upgraded yourself. */ + handleConnection(socket: WebSocket): void; + /** Close all sockets and flush every pending append. */ + close(): Promise; } interface Room { @@ -42,37 +55,44 @@ interface Room { const ROOT_NAME = /^[A-Za-z0-9._-]+$/; -export async function startLyncServe(options: LyncServeOptions): Promise { +export function createLyncRelay(options: LyncRelayOptions): LyncRelay { const log = options.log ?? ((message: string) => process.stderr.write(`${message}\n`)); - await mkdir(options.dir, { recursive: true }); + const dirReady = mkdir(options.dir, { recursive: true }).then(() => undefined); const rooms = new Map>(); - - const httpServer: Server = createServer((_request, response) => { - response.writeHead(404).end(); - }); - const socketServer = new WebSocketServer({ noServer: true }); const sockets = new Set(); + const wss = new WebSocketServer({ noServer: true }); - httpServer.on("upgrade", (request, socket, head) => { - if (options.token && !authorized(request, options.token)) { - log("[lync serve] rejected upgrade: bad or missing token"); - socket.write("HTTP/1.1 401 Unauthorized\r\n\r\n"); - socket.destroy(); + function handleUpgrade(request: IncomingMessage, socket: Duplex, head: Buffer): void { + if (options.token && request.headers.authorization !== `Bearer ${options.token}`) { + reject(socket, "bad or missing token"); return; } - socketServer.handleUpgrade(request, socket, head, (websocket) => { - socketServer.emit("connection", websocket, request); - }); - }); + if (!options.authenticate) { + wss.handleUpgrade(request, socket, head, (ws) => handleConnection(ws)); + return; + } + Promise.resolve(options.authenticate(request)).then( + (ok) => { + if (ok) wss.handleUpgrade(request, socket, head, (ws) => handleConnection(ws)); + else reject(socket, "authenticate() returned false"); + }, + (error) => reject(socket, `authenticate() threw: ${String(error)}`), + ); + } + + function reject(socket: Duplex, why: string): void { + log(`[lync relay] rejected upgrade: ${why}`); + socket.write("HTTP/1.1 401 Unauthorized\r\n\r\n"); + socket.destroy(); + } - socketServer.on("connection", (socket: WebSocket) => { + function handleConnection(socket: WebSocket): void { sockets.add(socket); + options.onConnection?.(socket); const subscribed = new Set(); // Frames from one socket are handled strictly in arrival order, so a - // client that pushes its lines and then subscribes is guaranteed to see - // any resulting errors before its backlog and `live`. + // client that pushes then subscribes sees any errors before its backlog. let frameChain = Promise.resolve(); - socket.on("message", (raw) => { const frame = decodeFrame(raw.toString()); frameChain = frameChain.then(() => handleFrame(socket, subscribed, frame)); @@ -81,16 +101,13 @@ export async function startLyncServe(options: LyncServeOptions): Promise { - log(`[lync serve] socket error: ${String(error)}`); - }); - }); + socket.on("error", (error) => log(`[lync relay] socket error: ${String(error)}`)); + } async function handleFrame(socket: WebSocket, subscribed: Set, frame: SyncFrame): Promise { try { switch (frame.t) { case "err": - // A decode failure or a client-reported error: answer loudly, never store. send(socket, frame.reason === "malformed-frame" || frame.reason === "unknown-frame-kind" ? frame : { t: "err", reason: "client-error-received", detail: frame.reason }); return; case "sub": { @@ -139,7 +156,7 @@ export async function startLyncServe(options: LyncServeOptions): Promise recoverRoom(root)); rooms.set(root, pending); } return pending; @@ -165,12 +182,9 @@ export async function startLyncServe(options: LyncServeOptions): Promise 0) { - // Kill-9 mid-append left a truncated tail. Seal it with a newline so - // future appends start clean; readers classify it as damaged. Loud, - // never eaten. const tail = lines.at(-1) ?? ""; room.recoveryNote = `sealed truncated final line (${tail.length} bytes) as damaged`; - log(`[lync serve] ${root}: ${room.recoveryNote}`); + log(`[lync relay] ${root}: ${room.recoveryNote}`); await appendFile(path, "\n"); } for (const line of lines) { @@ -206,24 +220,12 @@ export async function startLyncServe(options: LyncServeOptions): Promise((resolve, reject) => { - httpServer.once("error", reject); - httpServer.listen(options.port ?? 0, () => resolve()); - }); - const address = httpServer.address(); - if (address === null || typeof address === "string") { - throw new Error("lync serve: could not determine listening port"); - } - return { - port: address.port, + handleUpgrade, + handleConnection, close: async () => { for (const socket of sockets) socket.terminate(); - await new Promise((resolve) => socketServer.close(() => resolve())); - await new Promise((resolve, reject) => { - httpServer.close((error) => (error ? reject(error) : resolve())); - }); - // Let every in-flight append land before we report closed. + await new Promise((resolve) => wss.close(() => resolve())); for (const pending of rooms.values()) { const room = await pending; await room.writeChain; @@ -232,10 +234,6 @@ export async function startLyncServe(options: LyncServeOptions): Promise 80 ? `${text.slice(0, 77)}...` : text; } diff --git a/packages/server/src/serve.ts b/packages/server/src/serve.ts new file mode 100644 index 0000000..90d3752 --- /dev/null +++ b/packages/server/src/serve.ts @@ -0,0 +1,45 @@ +import { createServer, type Server } from "node:http"; +import { createLyncRelay, type LyncRelayOptions } from "./relay.js"; + +/** + * Run the relay standalone on its own HTTP server. For embedding in an + * existing server, use `createLyncRelay` and call `handleUpgrade` from your + * own `upgrade` listener. + */ + +export interface LyncServeOptions extends LyncRelayOptions { + /** Port to listen on. 0 (default) picks a free port. */ + port?: number; +} + +export interface LyncSyncServer { + port: number; + close: () => Promise; +} + +export async function startLyncServe(options: LyncServeOptions): Promise { + const relay = createLyncRelay(options); + const httpServer: Server = createServer((_request, response) => { + response.writeHead(404).end(); + }); + httpServer.on("upgrade", (request, socket, head) => relay.handleUpgrade(request, socket, head)); + + await new Promise((resolve, reject) => { + httpServer.once("error", reject); + httpServer.listen(options.port ?? 0, () => resolve()); + }); + const address = httpServer.address(); + if (address === null || typeof address === "string") { + throw new Error("lync serve: could not determine listening port"); + } + + return { + port: address.port, + close: async () => { + await relay.close(); + await new Promise((resolve, reject) => { + httpServer.close((error) => (error ? reject(error) : resolve())); + }); + }, + }; +} diff --git a/packages/server/test/attach.test.ts b/packages/server/test/attach.test.ts new file mode 100644 index 0000000..fe71acb --- /dev/null +++ b/packages/server/test/attach.test.ts @@ -0,0 +1,80 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { createServer, type Server } from "node:http"; +import { mkdtemp } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createLyncLooms, loomRootId } from "lync-core/looms"; +import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; +import { attachLyncServer, type AttachedLyncServer } from "../src/attach.js"; + +async function listen(server: Server): Promise { + return new Promise((resolve) => server.listen(0, () => { + const addr = server.address(); + resolve(typeof addr === "object" && addr ? addr.port : 0); + })); +} + +async function waitFor(check: () => Promise | boolean, timeoutMs = 4_000): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (await check()) return true; + await new Promise((r) => setTimeout(r, 20)); + } + return false; +} + +describe("attachLyncServer", () => { + let server: Server | undefined; + let attached: AttachedLyncServer | undefined; + const closers: Array<() => void> = []; + + afterEach(async () => { + for (const c of closers.splice(0)) c(); + await attached?.close(); + await new Promise((resolve) => (server ? server.close(() => resolve()) : resolve())); + server = attached = undefined; + }); + + it("converges through the mounted path and passes other upgrades through", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "lync-attach-")); + server = createServer((_r, res) => res.writeHead(200).end()); + let otherUpgrade = 0; + server.on("upgrade", (req, socket) => { + if (req.url !== "/lync") { + otherUpgrade += 1; + socket.destroy(); + } + }); + attached = attachLyncServer(server, { storageDir: dir, path: "/lync", log: () => {} }); + const port = await listen(server); + + const store = createSyncedStore(createMemoryEventStore(), createWebSocketTransport(`ws://localhost:${port}/lync`, { reconnectMs: 0 })); + closers.push(store.close); + const looms = createLyncLooms<{ text: string }, { title: string }, unknown>({ store, author: { actor: "a" } }); + const info = await looms.create({ title: "t" }); + const loom = await looms.open(info.id); + const turn = await loom.appendTurn(null, { text: "mounted path works" }); + + // A second client reads it back through the same mount. + const store2 = createSyncedStore(createMemoryEventStore(), createWebSocketTransport(`ws://localhost:${port}/lync`, { reconnectMs: 0 })); + closers.push(store2.close); + store2.syncRoot(loomRootId(info.id)); + expect(await waitFor(async () => (await store2.byId(turn.id)) !== null)).toBe(true); + expect(otherUpgrade).toBe(0); // relay handled /lync; nothing leaked to the app listener + }); + + it("rejects upgrades when authenticate returns false", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "lync-attach-")); + server = createServer((_r, res) => res.writeHead(200).end()); + attached = attachLyncServer(server, { storageDir: dir, authenticate: () => false, log: () => {} }); + const port = await listen(server); + + const store = createSyncedStore(createMemoryEventStore(), createWebSocketTransport(`ws://localhost:${port}/lync`, { reconnectMs: 0 })); + closers.push(store.close); + store.syncRoot("nope"); + // The socket is rejected, so the root never goes live. + const wentLive = await waitFor(() => store.status().liveRoots.includes("nope"), 1_500); + expect(wentLive).toBe(false); + }); +}); diff --git a/packages/server/test/relay.test.ts b/packages/server/test/relay.test.ts new file mode 100644 index 0000000..cd06250 --- /dev/null +++ b/packages/server/test/relay.test.ts @@ -0,0 +1,74 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { createServer, type Server } from "node:http"; +import { mkdtemp } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createLyncLooms, loomRootId } from "lync-core/looms"; +import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; +import { createLyncRelay } from "../src/relay.js"; + +/** + * The relay mounted on an app's own HTTP server at a path — the embedding + * textile needs. Two synced clients converge through it. + */ + +async function waitFor(check: () => Promise | boolean, timeoutMs = 5_000): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (await check()) return; + await new Promise((r) => setTimeout(r, 20)); + } + throw new Error("waitFor: condition not met within timeout"); +} + +describe("createLyncRelay mounted on an existing server", () => { + let server: Server | undefined; + const closers: Array<() => void> = []; + + afterEach(async () => { + for (const close of closers.splice(0)) close(); + await new Promise((resolve) => (server ? server.close(() => resolve()) : resolve())); + server = undefined; + }); + + it("relays only its own path and converges two embedded clients", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "lync-relay-")); + const relay = createLyncRelay({ dir, log: () => {} }); + closers.push(() => void relay.close()); + + server = createServer((_req, res) => res.writeHead(200).end("app")); + server.on("upgrade", (req, socket, head) => { + if (req.url === "/lync") relay.handleUpgrade(req, socket, head); + else socket.destroy(); + }); + const port = await new Promise((resolve) => { + server!.listen(0, () => { + const addr = server!.address(); + resolve(typeof addr === "object" && addr ? addr.port : 0); + }); + }); + const url = `ws://localhost:${port}/lync`; + + const mk = (actor: string) => { + const store = createSyncedStore(createMemoryEventStore(), createWebSocketTransport(url, { reconnectMs: 0 })); + const looms = createLyncLooms<{ text: string }, { title: string }, unknown>({ store, author: { actor } }); + closers.push(store.close); + return { store, looms }; + }; + const a = mk("alice"); + const b = mk("bob"); + await new Promise((r) => setTimeout(r, 200)); + + const info = await a.looms.create({ title: "mounted" }); + const loomA = await a.looms.open(info.id); + const t1 = await loomA.appendTurn(null, { text: "hello from the app server" }); + + const root = loomRootId(info.id); + b.store.syncRoot(root); + await waitFor(async () => (await b.store.byId(t1.id)) !== null); + const loomB = await b.looms.open(info.id); + const thread = await loomB.threadTo(t1.id); + expect(thread.map((t) => t.payload.text)).toEqual(["hello from the app server"]); + }); +}); diff --git a/packages/server/tsconfig.json b/packages/server/tsconfig.json new file mode 100644 index 0000000..df59da5 --- /dev/null +++ b/packages/server/tsconfig.json @@ -0,0 +1,8 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "dist" + }, + "include": ["src/**/*.ts"] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 1998af4..c380408 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -26,6 +26,9 @@ importers: lync-core: specifier: workspace:* version: link:../core + lync-server: + specifier: workspace:* + version: link:../server ws: specifier: ^8.18.0 version: 8.21.0 @@ -51,6 +54,19 @@ importers: specifier: workspace:* version: link:../core + packages/server: + dependencies: + lync-core: + specifier: workspace:* + version: link:../core + ws: + specifier: ^8.18.0 + version: 8.21.0 + devDependencies: + '@types/ws': + specifier: ^8.18.1 + version: 8.18.1 + packages: '@esbuild/aix-ppc64@0.27.7': From 489a793e86cb9627a932c4aead1bd69ad3c2cbda Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 23:23:01 -0700 Subject: [PATCH 13/33] server: relay/serve shutdown never hangs Socket teardown and wss.close()/httpServer.close() callbacks can block indefinitely on some runtimes (bun). Relay close now races an orderly shutdown against a hard cap and flushes pending appends first; standalone serve drops lingering connections, unrefs the listen handle, and caps the close wait. Production embedding via attachLyncServer is unaffected (it owns no server), but this keeps graceful shutdown honest everywhere. 86 tests, 20 consecutive green. --- packages/server/src/relay.ts | 38 ++++++++++++++++++++++++++++++------ packages/server/src/serve.ts | 13 ++++++++++-- 2 files changed, 43 insertions(+), 8 deletions(-) diff --git a/packages/server/src/relay.ts b/packages/server/src/relay.ts index c64b184..6acd3f1 100644 --- a/packages/server/src/relay.ts +++ b/packages/server/src/relay.ts @@ -224,12 +224,38 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { handleUpgrade, handleConnection, close: async () => { - for (const socket of sockets) socket.terminate(); - await new Promise((resolve) => wss.close(() => resolve())); - for (const pending of rooms.values()) { - const room = await pending; - await room.writeChain; - } + // Shutting the relay down must never hang. On some ws builds (notably + // under bun) socket teardown and wss.close() can block indefinitely, so + // the whole sequence races a hard cap; pending appends are flushed + // first since those resolve promptly. + const orderly = (async () => { + for (const pending of rooms.values()) { + try { + const room = await pending; + await room.writeChain; + } catch { + // A room that never recovered can't have pending writes worth waiting on. + } + } + for (const socket of sockets) { + try { + socket.terminate(); + } catch { + // Already gone. + } + } + await new Promise((resolve) => { + const timer = setTimeout(resolve, 300); + wss.close(() => { + clearTimeout(timer); + resolve(); + }); + }); + })(); + await Promise.race([ + orderly, + new Promise((resolve) => setTimeout(resolve, 1500)), + ]); }, }; } diff --git a/packages/server/src/serve.ts b/packages/server/src/serve.ts index 90d3752..648d778 100644 --- a/packages/server/src/serve.ts +++ b/packages/server/src/serve.ts @@ -37,8 +37,17 @@ export async function startLyncServe(options: LyncServeOptions): Promise { await relay.close(); - await new Promise((resolve, reject) => { - httpServer.close((error) => (error ? reject(error) : resolve())); + // Drop any lingering connections and release the listen handle. The + // close callback is unreliable under some runtimes (bun), so cap the + // wait and unref the server so it can never keep the loop alive. + (httpServer as { closeAllConnections?: () => void }).closeAllConnections?.(); + httpServer.unref(); + await new Promise((resolve) => { + const timer = setTimeout(resolve, 1000); + httpServer.close(() => { + clearTimeout(timer); + resolve(); + }); }); }, }; From 780eabf1f64a932aa41d54cfa6463d4424e53fae Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 23:47:05 -0700 Subject: [PATCH 14/33] relay: durability failures are loud and non-wedging (adversarial review fixes) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A rival-team review found a silent-black-hole path and two minor gaps: - MAJOR: a single appendFile failure permanently wedged a room — the rejected writeChain skipped every future write, and the throw skipped the broadcast, so the room silently accepted events into memory while persisting and delivering nothing. Now appendSerialized clears prior rejections, keeps the chain resolved (recoverable), and reports ok/fail; the ev handler broadcasts regardless of persistence and emits a loud 'persist-failed' err on durability failure. Nothing fails invisibly. - recoverRoom no longer replays a damaged/sealed-truncated line as a phantom event: it stays on disk (never eaten) but is not served. - close() flushes every pending append BEFORE the capped socket teardown, honoring the interface's durability promise. Regression test: a read-only relay dir forces append failures; the room still broadcasts both events and surfaces a persist-failed err for each, proving live delivery survives and the failure is never hidden. Cosmetic: test fixtures/descriptions cleaned of lore/automerge vocabulary. 87 tests, 15 consecutive green. --- packages/cli/test/sync.test.ts | 4 +- packages/core/test/references.test.ts | 10 ++-- packages/core/test/vectors/v0/README.md | 2 +- packages/core/test/views.test.ts | 2 +- packages/server/src/relay.ts | 79 ++++++++++++++++--------- packages/server/test/relay.test.ts | 52 ++++++++++++++++ 6 files changed, 112 insertions(+), 37 deletions(-) diff --git a/packages/cli/test/sync.test.ts b/packages/cli/test/sync.test.ts index b18fdf9..d32a5a6 100644 --- a/packages/cli/test/sync.test.ts +++ b/packages/cli/test/sync.test.ts @@ -115,7 +115,9 @@ describe("lync serve + sync", () => { const result = await syncOnce({ file, url, root: "crash", out: quiet, err: errs.io }); expect(result.received).toBe(1); // the survivor - expect(result.surfaced).toBe(1); // the sealed damaged tail: surfaced, not appended + // The sealed damaged tail is surfaced via the recovery note, NOT replayed + // as a phantom event — so it is never mis-served, and never appended. + expect(result.surfaced).toBe(0); expect(errs.text()).toContain("sealed truncated final line"); expect(idsOf(await readFile(file, "utf8"))).toEqual(["survivor"]); // The damaged bytes still live in the server file — sealed, never eaten. diff --git a/packages/core/test/references.test.ts b/packages/core/test/references.test.ts index 812b7e1..0cb9490 100644 --- a/packages/core/test/references.test.ts +++ b/packages/core/test/references.test.ts @@ -14,10 +14,10 @@ import { describe("references", () => { it("roundtrips every reference kind through encoding", () => { const refs = [ - loomRef("automerge:loom"), - turnRef("automerge:loom", "turn-1"), - threadRef("automerge:loom", "turn-1"), - indexRef("automerge:index"), + loomRef("lync:loom"), + turnRef("lync:loom", "turn-1"), + threadRef("lync:loom", "turn-1"), + indexRef("lync:index"), ]; for (const ref of refs) { @@ -36,7 +36,7 @@ describe("references", () => { }); it("roundtrips through ?ref= urls without slug or title hints", () => { - const ref = threadRef("automerge:loom", "turn-1"); + const ref = threadRef("lync:loom", "turn-1"); const url = referenceToUrl( ref, new URL("https://loom.test/story?old=1#stale"), diff --git a/packages/core/test/vectors/v0/README.md b/packages/core/test/vectors/v0/README.md index 3caa886..7a3714f 100644 --- a/packages/core/test/vectors/v0/README.md +++ b/packages/core/test/vectors/v0/README.md @@ -1,4 +1,4 @@ -# Lore v0 Draft Vector Suite +# lync v0 draft vector suite Draft conformance vectors for the lync format spec (FORMAT.md), Part I. These are wrangling fixtures, not a ratified format for vector metadata. diff --git a/packages/core/test/views.test.ts b/packages/core/test/views.test.ts index 9af4a70..5b18635 100644 --- a/packages/core/test/views.test.ts +++ b/packages/core/test/views.test.ts @@ -46,7 +46,7 @@ function event(fields: { }); } -describe("LORE views", () => { +describe("lync views", () => { it("computes a branch tree DAG from vector parent links", () => { const result = loadFixture("01-valid-events"); const tree = lyncBranchTreeView(result); diff --git a/packages/server/src/relay.ts b/packages/server/src/relay.ts index 6acd3f1..7ab79fa 100644 --- a/packages/server/src/relay.ts +++ b/packages/server/src/relay.ts @@ -142,8 +142,13 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { room.lines.push(frame.line); room.seq += 1; const seq = room.seq; - await appendSerialized(room, join(options.dir, `${room.root}.lync`), frame.line); + const persisted = await appendSerialized(room, join(options.dir, `${room.root}.lync`), frame.line); + // Live delivery is the relay's primary job: fan out even if the disk + // write failed. A durability failure is surfaced loudly, never hidden. broadcast(room, { t: "ev", root: room.root, seq, line: frame.line }); + if (!persisted.ok) { + broadcast(room, { t: "err", root: room.root, reason: "persist-failed", detail: id }); + } return; } case "presence": { @@ -188,17 +193,34 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { await appendFile(path, "\n"); } for (const line of lines) { + const id = extractLineId(line); + // A damaged or sealed-truncated line stays on disk (never eaten) but is + // not replayed to subscribers — it isn't a real event. + if (id === undefined) continue; room.lines.push(line); room.seq += 1; - const id = extractLineId(line); - if (id !== undefined && !room.byId.has(id)) room.byId.set(id, line); + if (!room.byId.has(id)) room.byId.set(id, line); } return room; } - function appendSerialized(room: Room, path: string, line: string): Promise { - room.writeChain = room.writeChain.then(() => appendFile(path, `${line}\n`)); - return room.writeChain; + // Serialize appends per room. A write failure must never wedge the room or + // vanish silently: clear any prior rejection so the next write still runs, + // keep the chain resolved so the room recovers, and report ok/failure to the + // caller so a persistence error can be surfaced loudly. + function appendSerialized(room: Room, path: string, line: string): Promise<{ ok: boolean }> { + const attempt = room.writeChain + .catch(() => undefined) + .then(() => appendFile(path, `${line}\n`)) + .then( + () => ({ ok: true }), + (error) => { + log(`[lync relay] persist failed for ${path}: ${String(error)}`); + return { ok: false }; + }, + ); + room.writeChain = attempt.then(() => undefined); + return attempt; } function broadcast(room: Room, frame: SyncFrame, except?: WebSocket): void { @@ -224,36 +246,35 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { handleUpgrade, handleConnection, close: async () => { - // Shutting the relay down must never hang. On some ws builds (notably - // under bun) socket teardown and wss.close() can block indefinitely, so - // the whole sequence races a hard cap; pending appends are flushed - // first since those resolve promptly. - const orderly = (async () => { - for (const pending of rooms.values()) { - try { - const room = await pending; - await room.writeChain; - } catch { - // A room that never recovered can't have pending writes worth waiting on. - } + // Flush every pending append first — writeChains are kept resolved (never + // rejected) by appendSerialized, so this settles promptly and no accepted + // write is dropped. This honors the durability promise in the interface. + for (const pending of rooms.values()) { + try { + const room = await pending; + await room.writeChain; + } catch { + // A room that never recovered has no pending writes worth waiting on. } - for (const socket of sockets) { - try { - socket.terminate(); - } catch { - // Already gone. - } + } + // Then tear down sockets and the server under a hard cap: on some ws + // builds (notably bun) socket teardown and wss.close() can block + // indefinitely, so shutdown must never hang. + for (const socket of sockets) { + try { + socket.terminate(); + } catch { + // Already gone. } - await new Promise((resolve) => { + } + await Promise.race([ + new Promise((resolve) => { const timer = setTimeout(resolve, 300); wss.close(() => { clearTimeout(timer); resolve(); }); - }); - })(); - await Promise.race([ - orderly, + }), new Promise((resolve) => setTimeout(resolve, 1500)), ]); }, diff --git a/packages/server/test/relay.test.ts b/packages/server/test/relay.test.ts index cd06250..da6acf8 100644 --- a/packages/server/test/relay.test.ts +++ b/packages/server/test/relay.test.ts @@ -72,3 +72,55 @@ describe("createLyncRelay mounted on an existing server", () => { expect(thread.map((t) => t.payload.text)).toEqual(["hello from the app server"]); }); }); + +describe("createLyncRelay durability failures", () => { + it("surfaces a persist failure loudly, still broadcasts, and does not wedge the room", async () => { + const { chmod, mkdtemp } = await import("node:fs/promises"); + const os = await import("node:os"); + const nodePath = await import("node:path"); + const { createServer } = await import("node:http"); + const { createWebSocketTransport } = await import("lync-core/synced-store"); + + const dir = await mkdtemp(nodePath.join(os.tmpdir(), "lync-persist-")); + // Read-only dir: recovery (no existing files) succeeds, but every append fails. + await chmod(dir, 0o555); + + const relay = createLyncRelay({ dir, log: () => {} }); + const server = createServer((_r, res) => res.writeHead(200).end()); + server.on("upgrade", (req, socket, head) => relay.handleUpgrade(req, socket, head)); + const port = await new Promise((resolve) => server.listen(0, () => { + const a = server.address(); + resolve(typeof a === "object" && a ? a.port : 0); + })); + const url = `ws://localhost:${port}`; + + const evs: string[] = []; + const errs: string[] = []; + const t = createWebSocketTransport(url, { reconnectMs: 0 }); + t.onFrame((f) => { + if (f.t === "err") errs.push(f.reason); + if (f.t === "ev") evs.push(f.line); + }); + const line = (id: string) => JSON.stringify({ v: 1, id, kind: "lync/artifact", at: "2026-07-08T21:00:00Z", author: { actor: "x" }, parents: [], payload: {} }); + + try { + t.send({ t: "sub", root: "wedged", since: 0 }); + t.send({ t: "ev", root: "wedged", line: line("e1") }); + await new Promise((r) => setTimeout(r, 250)); + // Second write proves the first failure did NOT wedge the room. + t.send({ t: "ev", root: "wedged", line: line("e2") }); + await new Promise((r) => setTimeout(r, 250)); + + // Both events were broadcast (live delivery survived the disk failure)... + expect(evs.filter((l) => l.includes('"e1"')).length).toBeGreaterThanOrEqual(1); + expect(evs.filter((l) => l.includes('"e2"')).length).toBeGreaterThanOrEqual(1); + // ...and each durability failure was surfaced loudly, never hidden. + expect(errs.filter((r) => r === "persist-failed").length).toBeGreaterThanOrEqual(2); + } finally { + t.close(); + await relay.close(); + await new Promise((resolve) => server.close(() => resolve())); + await chmod(dir, 0o755); + } + }); +}); From ee7584103ca42e6b5ce7f2841fbd7ac260baeca3 Mon Sep 17 00:00:00 2001 From: deepfates Date: Wed, 8 Jul 2026 23:54:04 -0700 Subject: [PATCH 15/33] test: alias lync-server and lync-client to src for build-less CI CI runs 'pnpm verify' = guard && test && typecheck with no build step, so bare-specifier imports must resolve to src. lync-core and lync-index were aliased but lync-server (imported by the CLI serve/sync tests) and lync-client were not, so vitest failed to resolve lync-server's dist entry in CI. Added the missing src aliases; verify now passes with no dist. --- vitest.config.ts | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/vitest.config.ts b/vitest.config.ts index 8c21d62..3246350 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -26,6 +26,18 @@ export default defineConfig({ find: /^lync-index$/, replacement: new URL("./packages/index/src/index.ts", import.meta.url).pathname, }, + { + find: /^lync-server$/, + replacement: new URL("./packages/server/src/index.ts", import.meta.url).pathname, + }, + { + find: /^lync-client\/([a-z0-9-]+)$/, + replacement: new URL("./packages/client/src/", import.meta.url).pathname + "$1.ts", + }, + { + find: /^lync-client$/, + replacement: new URL("./packages/client/src/index.ts", import.meta.url).pathname, + }, ], }, test: { From 25abf5e4bf53671ed2646a50d5b968155cbce8bb Mon Sep 17 00:00:00 2001 From: deepfates Date: Fri, 10 Jul 2026 11:36:23 -0700 Subject: [PATCH 16/33] =?UTF-8?q?fix:=20codex=20gauntlet=20findings=20?= =?UTF-8?q?=E2=80=94=20cursor=20integrity=20+=20conflict-sidecar=20honesty?= =?UTF-8?q?=20(dee-inzc)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A rival-family (codex) pre-publish review found what same-family review missed, both with working repros: BLOCKER — fractional resume cursors skipped the backlog silently: decodeFrame accepted any nonnegative number for sub.since, the relay indexed lines[0.5] = undefined (malformed ev frames), the client appended nothing, then persisted the live seq — a PERMANENT missed backlog. Fixed at every entry: isCursor() (nonnegative integer) enforced in decodeFrame for sub.since / live.seq / ev.seq, and readCursor() resets an unusable cursor file to 0 (re-receiving the backlog is a harmless union no-op). MAJOR — same-id conflict variants could be silently dropped: the sidecar append's failure result was ignored, so clients were told same-id-conflict (implying retention) while the bytes vanished with only a server log. The relay now emits conflict-persist-failed to sender and subscribers when the sidecar write fails. Regressions added for both (fractional cursor file → full backlog received, cursor repaired; read-only dir → conflict-persist-failed surfaced). 90 tests, 10/10 consecutive green. --- packages/cli/src/sync.ts | 10 +++- packages/cli/test/sync.test.ts | 73 ++++++++++++++++++++++++ packages/core/src/sync-protocol.ts | 24 ++++++-- packages/core/test/sync-protocol.test.ts | 12 ++++ packages/server/src/relay.ts | 8 ++- 5 files changed, 118 insertions(+), 9 deletions(-) diff --git a/packages/cli/src/sync.ts b/packages/cli/src/sync.ts index 7ca2624..ba24ef1 100644 --- a/packages/cli/src/sync.ts +++ b/packages/cli/src/sync.ts @@ -2,7 +2,7 @@ import { appendFile, readFile, writeFile } from "node:fs/promises"; import { existsSync, watch } from "node:fs"; import { basename } from "node:path"; import WebSocket from "ws"; -import { decodeFrame, encodeFrame, extractLineId } from "lync-core/sync-protocol"; +import { decodeFrame, encodeFrame, extractLineId, isCursor } from "lync-core/sync-protocol"; /** * `lync sync ` — one-shot convergence with a `lync serve` relay. @@ -206,10 +206,14 @@ async function readCursor(path: string, url: string, root: string): Promise= 0) { + // seq must be a nonnegative INTEGER: a fractional cursor (corrupt or + // hand-edited file) would make the relay skip the whole backlog and then + // get persisted as live — a permanent silent miss. Reset to 0 instead; + // re-receiving the backlog is a harmless union no-op. + if (stored.url === url && stored.root === root && isCursor(stored.seq)) { return stored; } - // Different server or root: the stored cursor means nothing here. + // Different server/root, or an unusable cursor: start from 0. return { url, root, seq: 0 }; } catch { return { url, root, seq: 0 }; diff --git a/packages/cli/test/sync.test.ts b/packages/cli/test/sync.test.ts index d32a5a6..5c72224 100644 --- a/packages/cli/test/sync.test.ts +++ b/packages/cli/test/sync.test.ts @@ -223,3 +223,76 @@ async function waitFor(check: () => Promise, timeoutMs = 5_000): Promis } throw new Error("waitFor: condition not met within timeout"); } + +describe("cursor corruption recovery (dee-inzc blocker)", () => { + let server: LyncSyncServer | undefined; + + afterEach(async () => { + await server?.close(); + server = undefined; + }); + + it("a fractional cursor file resets to 0 and receives the FULL backlog, never skipping it", async () => { + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + // Server already holds two events. + await writeFile( + path.join(serverDir, "story.lync"), + `${eventLine("root", [], "one")}\n${eventLine("late", ["root"], "two")}\n`, + ); + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + + const file = path.join(clientDir, "story.lync"); + await writeFile(file, ""); + // A corrupt (fractional) cursor — pre-fix this silently skipped the whole + // backlog and then persisted seq 2 as live: a permanent miss. + await writeFile(`${file}.sync.json`, `${JSON.stringify({ url, root: "story", seq: 0.5 })}\n`); + + const result = await syncOnce({ file, url, root: "story", out: quiet, err: quiet }); + expect(result.received).toBe(2); // full backlog delivered + expect(idsOf(await readFile(file, "utf8"))).toEqual(["late", "root"]); + const cursor = JSON.parse(await readFile(`${file}.sync.json`, "utf8")) as { seq: number }; + expect(Number.isInteger(cursor.seq)).toBe(true); + expect(cursor.seq).toBe(2); + }); +}); + +describe("conflict sidecar durability (dee-inzc major)", () => { + let server: LyncSyncServer | undefined; + let lockedDir: string | undefined; + + afterEach(async () => { + if (lockedDir) await (await import("node:fs/promises")).chmod(lockedDir, 0o755).catch(() => {}); + await server?.close(); + server = undefined; + }); + + it("tells clients loudly when the conflict variant could NOT be retained", async () => { + const { chmod } = await import("node:fs/promises"); + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-client-")); + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + + const fileA = path.join(clientDir, "a.lync"); + const fileB = path.join(clientDir, "b.lync"); + await writeFile(fileA, `${eventLine("same", [], "first telling")}\n`); + await writeFile(fileB, `${eventLine("same", [], "second telling")}\n`); + + // A's version lands and persists normally... + await syncOnce({ file: fileA, url, root: "duel", out: quiet, err: quiet }); + // ...then the relay dir goes read-only, so the conflict sidecar CANNOT be written. + lockedDir = serverDir; + await chmod(serverDir, 0o555); + + const errs = collect(); + const result = await syncOnce({ file: fileB, url, root: "duel", out: quiet, err: errs.io }); + + expect(result.conflicts).toBe(1); // the conflict itself is still surfaced + // ...and so is the retention failure — the client must never believe the + // sidecar promise was kept when it wasn't. + expect(errs.text()).toContain("conflict-persist-failed"); + expect(existsSync(path.join(serverDir, "duel.conflicts"))).toBe(false); + }); +}); diff --git a/packages/core/src/sync-protocol.ts b/packages/core/src/sync-protocol.ts index 78eb9cf..e9c8973 100644 --- a/packages/core/src/sync-protocol.ts +++ b/packages/core/src/sync-protocol.ts @@ -54,6 +54,14 @@ export type SyncFrame = SubFrame | EvFrame | LiveFrame | PresenceFrame | ErrFram const FRAME_KINDS = new Set(["sub", "ev", "live", "presence", "err"]); +/** + * A resume cursor / sequence number: a nonnegative integer. Fractional or + * non-finite values must never pass — they index into backlogs downstream. + */ +export function isCursor(value: unknown): value is number { + return typeof value === "number" && Number.isInteger(value) && value >= 0; +} + export function encodeFrame(frame: SyncFrame): string { return JSON.stringify(frame); } @@ -79,25 +87,31 @@ export function decodeFrame(raw: string | Uint8Array): SyncFrame { } switch (frame.t) { case "sub": - if (typeof frame.root !== "string" || typeof frame.since !== "number" || frame.since < 0) { + // Cursors are array indices and resume positions: a fractional or + // non-finite `since` silently skips the backlog downstream (lines[0.5] + // is undefined), so anything but a nonnegative integer is malformed. + if (typeof frame.root !== "string" || !isCursor(frame.since)) { return { t: "err", reason: "malformed-sub" }; } - return { t: "sub", root: frame.root, since: frame.since }; + return { t: "sub", root: frame.root, since: frame.since as number }; case "ev": if (typeof frame.root !== "string" || typeof frame.line !== "string") { return { t: "err", reason: "malformed-ev" }; } + if (frame.seq !== undefined && !isCursor(frame.seq)) { + return { t: "err", reason: "malformed-ev", detail: "seq must be a nonnegative integer" }; + } return { t: "ev", root: frame.root, line: frame.line, - ...(typeof frame.seq === "number" ? { seq: frame.seq } : {}), + ...(frame.seq !== undefined ? { seq: frame.seq as number } : {}), }; case "live": - if (typeof frame.root !== "string" || typeof frame.seq !== "number") { + if (typeof frame.root !== "string" || !isCursor(frame.seq)) { return { t: "err", reason: "malformed-live" }; } - return { t: "live", root: frame.root, seq: frame.seq }; + return { t: "live", root: frame.root, seq: frame.seq as number }; case "presence": if (typeof frame.root !== "string") { return { t: "err", reason: "malformed-presence" }; diff --git a/packages/core/test/sync-protocol.test.ts b/packages/core/test/sync-protocol.test.ts index 8d856f2..d5a18ce 100644 --- a/packages/core/test/sync-protocol.test.ts +++ b/packages/core/test/sync-protocol.test.ts @@ -30,3 +30,15 @@ describe("lync sync protocol frames", () => { expect(extractLineId('{"noid":true}')).toBeUndefined(); }); }); + +describe("cursor integrity (dee-inzc blocker)", () => { + it("rejects fractional and non-finite cursors in sub/live/ev frames", () => { + expect(decodeFrame('{"t":"sub","root":"r","since":0.5}')).toMatchObject({ t: "err", reason: "malformed-sub" }); + expect(decodeFrame('{"t":"sub","root":"r","since":null}')).toMatchObject({ t: "err", reason: "malformed-sub" }); + expect(decodeFrame('{"t":"live","root":"r","seq":1.5}')).toMatchObject({ t: "err", reason: "malformed-live" }); + expect(decodeFrame('{"t":"ev","root":"r","line":"{}","seq":2.5}')).toMatchObject({ t: "err", reason: "malformed-ev" }); + // Integers still pass. + expect(decodeFrame('{"t":"sub","root":"r","since":0}').t).toBe("sub"); + expect(decodeFrame('{"t":"live","root":"r","seq":7}').t).toBe("live"); + }); +}); diff --git a/packages/server/src/relay.ts b/packages/server/src/relay.ts index 7ab79fa..c26c150 100644 --- a/packages/server/src/relay.ts +++ b/packages/server/src/relay.ts @@ -133,9 +133,15 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { const existing = room.byId.get(id); if (existing !== undefined) { if (existing === frame.line) return; // duplicate: a no-op by union - await appendSerialized(room, join(options.dir, `${room.root}.conflicts`), frame.line); + const kept = await appendSerialized(room, join(options.dir, `${room.root}.conflicts`), frame.line); broadcast(room, { t: "err", root: room.root, reason: "same-id-conflict", detail: id }, socket); send(socket, { t: "err", root: room.root, reason: "same-id-conflict", detail: id }); + if (!kept.ok) { + // The variant bytes were NOT retained — clients must not believe + // the sidecar promise was kept. Loud, to everyone, exactly once. + broadcast(room, { t: "err", root: room.root, reason: "conflict-persist-failed", detail: id }, socket); + send(socket, { t: "err", root: room.root, reason: "conflict-persist-failed", detail: id }); + } return; } room.byId.set(id, frame.line); From a48fdc7ede14d9a8ed7a5e625de39f1dddb80553 Mon Sep 17 00:00:00 2001 From: deepfates Date: Fri, 10 Jul 2026 11:56:21 -0700 Subject: [PATCH 17/33] docs+spec+writers: publish-readiness fixes from the review pass MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Docs (adopter-facing truth): - README: resolve the TODO(positioning) marker that shipped on the front page; seven verbs (serve/sync were omitted — the flagship feature); lync-server added to the Packages section (it was absent entirely). - lync-cli README: seven verbs, serve/sync described. - ROADMAP rewritten to post-sync reality: no internal ticket ids, no Automerge-transition language, no publish-sequencing that contradicts the release. Spec+writers (close the conformance gap both ways): - FORMAT.md id row: generators mint UUIDv7; importers transcribing pre-existing events may derive deterministic UUIDv8 from source identity (re-import = union no-op; upstream edit = loud same-id conflict). - Shipped writers now actually mint UUIDv7: new zero-dep lync-core/uuid, wired into looms' default createId and the CLI append fallback (was UUIDv4 randomUUID, nonconforming with our own spec). Publish process hardening: - engines: node >=22 on all five packages (synced-store needs global WebSocket; CI pins 22) and prepublishOnly build scripts so a stale or missing dist cannot ship. - rm stray packages/sync-server dir (node_modules-only leftover). 91 tests green (adds uuidv7 shape/order/uniqueness test). --- FORMAT.md | 4 ++-- README.md | 11 ++++++----- ROADMAP.md | 19 +++++++++--------- packages/cli/README.md | 4 +++- packages/cli/package.json | 6 +++++- packages/cli/src/index.ts | 4 ++-- packages/client/package.json | 6 +++++- packages/core/package.json | 11 ++++++++++- packages/core/src/looms.ts | 5 +++-- packages/core/src/uuid.ts | 25 ++++++++++++++++++++++++ packages/core/test/sync-protocol.test.ts | 11 +++++++++++ packages/index/package.json | 6 +++++- packages/server/package.json | 6 +++++- 13 files changed, 91 insertions(+), 27 deletions(-) create mode 100644 packages/core/src/uuid.ts diff --git a/FORMAT.md b/FORMAT.md index 715f7e5..9a00a9a 100644 --- a/FORMAT.md +++ b/FORMAT.md @@ -43,7 +43,7 @@ One shape. Everything is an instance of it. | Field | Required | Type | Meaning | |---|---|---|---| | `v` | yes | int | Envelope version. Writers never invent top-level fields; readers tolerate unknown ones as future-version diagnostics. | -| `id` | yes | string; UUIDv7 is a writer obligation | Identity of the event, not the content. Two identical generations are two events. The embedded timestamp is untrusted input; `at` and `marked` carry time claims. Readers compare ids as opaque decoded strings and validate nothing about their shape. | +| `id` | yes | string; UUID is a writer obligation | Identity of the event, not the content. Generators mint UUIDv7 (two identical generations are two events); importers transcribing pre-existing events may instead derive a deterministic UUIDv8 from source identity, so re-importing the same source is a union no-op and an upstream edit surfaces as a same-id conflict. The embedded timestamp is untrusted input; `at` and `marked` carry time claims. Readers compare ids as opaque decoded strings and validate nothing about their shape. | | `kind` | yes | string, `namespace/name` | What sort of event this is. Opaque to the protocol. Must contain at least one `/`; the part before the first `/` is the namespace and the rest is the name, both non-empty. The name may itself contain `/`. The namespace tells you whose pact defines it; the protocol never interprets it and there is no registry. | | `at` | yes | RFC 3339 | When the content came into being, as claimed by the author. A claim, not a proof. | | `author` | yes | object | Provenance: who made this, under whose responsibility, through what. | @@ -374,7 +374,7 @@ not data. Five events: a paragraph, two alternatives, a judge's score, and a declared choice. Non-normative shorthand: ids are shown as `A` to `E` and digests are -elided for readability only. Conforming writers mint UUIDv7 ids and should +elided for readability only. Conforming generators mint UUIDv7 ids (importers may derive deterministic UUIDv8 — see the `id` row) and should splice digests per "Bytes Are Canonical." ```jsonl diff --git a/README.md b/README.md index a751cc1..1a0c554 100644 --- a/README.md +++ b/README.md @@ -13,8 +13,9 @@ events. ## Ninety-Second Story -The shipped `lync` CLI has five verbs: `verify`, `merge`, `view`, `init`, and -`append`. From a fresh clone, run these from the repo root after +The shipped `lync` CLI has seven verbs: `init`, `append`, `verify`, `merge`, +`view`, `serve`, and `sync`. This first mile uses the file verbs; see +[Sync](#sync) for `serve` and `sync`. From a fresh clone, run these from the repo root after `pnpm install && pnpm build`; `pnpm exec lync` resolves the workspace binary. A published or globally installed package drops the `pnpm exec` prefix and you call `lync` directly. A complete first mile looks like this: @@ -40,9 +41,6 @@ byte, and read by software that has never heard of `notes/text`. Unknown kinds are carried and traversed; meaning belongs to pacts layered above the format. -TODO(positioning): pending the market-sweep verdict, tighten the public -positioning paragraph against adjacent products using the old contested term. - ## The Format See [FORMAT.md](./FORMAT.md) for the normative lync format specification. @@ -72,6 +70,9 @@ The short version: loom API, and live sync (`createSyncedStore`). No runtime dependencies. - `lync-cli`: the `lync` command — `init`, `append`, `verify`, `merge`, `view`, `serve`, `sync`. +- `lync-server`: the line-sync relay — `createLyncRelay` to mount on your own + Node server, `attachLyncServer` for one path on an existing server, + `startLyncServe` standalone. Depends on `lync-core` and `ws`. - `lync-index`: an index of many looms, with reactive subscription. Depends only on `lync-core`. - `lync-client`: the loom client — resolves references and opens looms and diff --git a/ROADMAP.md b/ROADMAP.md index 616948c..84ebb39 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -5,24 +5,23 @@ meaning should land in pacts, not in the envelope. ## Now -- Harden the shipped `lync` CLI for `verify`, `merge`, `view`, `init`, and - `append` workflows over `.lync` files. -- Finish the native sync replacement (`dee-9l2l`) so Automerge becomes only the - legacy transport path. +- First public release of the five packages: `lync-core`, `lync-cli`, + `lync-index`, `lync-client`, `lync-server`. - Keep `FORMAT.md` and the test vectors aligned as the reference other languages can port. ## Next - Flesh out pacts for ordering, authorship, selections, scoring, retraction, - and training-export obligations. -- Rename legacy internal paths once compatibility allows it. -- Publish packages after Textile consumes lync as a dependency instead of a - vendored copy. + import/transcription (deterministic ids), and training-export obligations. +- Digest splicing in the shipped writers (FORMAT recommends it; readers already + verify spliced digests). +- Conformance fixtures and test-suite ports for non-TypeScript + implementations. ## Later - Build a focused viewer for branch trees, transcripts, memory/frontier views, conflicts, damaged lines, and suppression explanations. -- Add deployment notes for sync services once native sync is the default. -- Add conformance fixtures for non-TypeScript implementations. +- Deployment notes for long-running relays (systemd, containers, auth + patterns beyond bearer tokens). diff --git a/packages/cli/README.md b/packages/cli/README.md index 25755b5..0370c42 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -13,5 +13,7 @@ lync view story.lync --as transcript lync merge story.lync other.lync -o merged.lync ``` -Five verbs: `init`, `append`, `verify`, `merge`, `view`. Run `lync --help` for +Seven verbs: `init`, `append`, `verify`, `merge`, `view`, `serve` (the +line-sync relay), and `sync` (converge a file with a relay, `--follow` to stay +live). Run `lync --help` for usage. Full docs: https://github.com/deepfates/lync#readme diff --git a/packages/cli/package.json b/packages/cli/package.json index e8dc88f..d6a4577 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -33,7 +33,8 @@ ], "scripts": { "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit" + "typecheck": "tsc -p tsconfig.json --noEmit", + "prepublishOnly": "tsc -p tsconfig.json" }, "dependencies": { "lync-core": "workspace:*", @@ -42,5 +43,8 @@ }, "devDependencies": { "@types/ws": "^8.18.1" + }, + "engines": { + "node": ">=22" } } diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index 4d3269e..6497373 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -1,4 +1,4 @@ -import { randomUUID } from "node:crypto"; +import { uuidv7 } from "lync-core/uuid"; import { appendFile, readFile, stat, writeFile } from "node:fs/promises"; import { parseLyncFiles, @@ -232,7 +232,7 @@ function buildAppendEvent(value: unknown, io: LyncCliIO): const event: Record = { v: 1, - id: typeof value.id === "string" ? value.id : (io.randomId?.() ?? randomUUID()), + id: typeof value.id === "string" ? value.id : (io.randomId?.() ?? uuidv7()), kind: value.kind, at: typeof value.at === "string" ? value.at : (io.now?.() ?? new Date()).toISOString(), author: value.author, diff --git a/packages/client/package.json b/packages/client/package.json index 6fca14d..dc7f821 100644 --- a/packages/client/package.json +++ b/packages/client/package.json @@ -29,7 +29,8 @@ ], "scripts": { "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit" + "typecheck": "tsc -p tsconfig.json --noEmit", + "prepublishOnly": "tsc -p tsconfig.json" }, "dependencies": { "lync-core": "workspace:*", @@ -44,5 +45,8 @@ "bugs": "https://github.com/deepfates/lync/issues", "publishConfig": { "access": "public" + }, + "engines": { + "node": ">=22" } } diff --git a/packages/core/package.json b/packages/core/package.json index 5500dd8..73b3ba0 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -92,6 +92,11 @@ "types": "./dist/synced-store.d.ts", "import": "./dist/synced-store.js", "default": "./dist/synced-store.js" + }, + "./uuid": { + "types": "./dist/uuid.d.ts", + "import": "./dist/uuid.js", + "default": "./dist/uuid.js" } }, "files": [ @@ -99,6 +104,10 @@ ], "scripts": { "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit" + "typecheck": "tsc -p tsconfig.json --noEmit", + "prepublishOnly": "tsc -p tsconfig.json" + }, + "engines": { + "node": ">=22" } } diff --git a/packages/core/src/looms.ts b/packages/core/src/looms.ts index ac8a4e8..2f7ab12 100644 --- a/packages/core/src/looms.ts +++ b/packages/core/src/looms.ts @@ -8,6 +8,7 @@ import { unknownLoom, } from "./errors.js"; import { assertJsonEncodable, cloneJson } from "./json.js"; +import { uuidv7 } from "./uuid.js"; import type { Loom, LoomEvent, @@ -448,6 +449,6 @@ function omitUndefined>(value: T): T { } function createUuidLike(): string { - if (typeof crypto !== "undefined" && "randomUUID" in crypto) return crypto.randomUUID(); - return `${Date.now().toString(16)}-${Math.random().toString(16).slice(2)}`; + // FORMAT.md: generators mint UUIDv7. + return uuidv7(); } diff --git a/packages/core/src/uuid.ts b/packages/core/src/uuid.ts new file mode 100644 index 0000000..76a912d --- /dev/null +++ b/packages/core/src/uuid.ts @@ -0,0 +1,25 @@ +/** + * UUIDv7 minting for lync writers. FORMAT.md makes UUID ids a writer + * obligation: generators mint UUIDv7 (time-ordered, unique per generation); + * importers transcribing pre-existing events may instead derive deterministic + * UUIDv8 from source identity. Zero dependencies; uses the Web Crypto global + * available in browsers and Node. + */ + +export function uuidv7(now: number = Date.now()): string { + const bytes = new Uint8Array(16); + globalThis.crypto.getRandomValues(bytes); + // 48-bit big-endian unix-ms timestamp. + const ms = BigInt(now); + bytes[0] = Number((ms >> 40n) & 0xffn); + bytes[1] = Number((ms >> 32n) & 0xffn); + bytes[2] = Number((ms >> 24n) & 0xffn); + bytes[3] = Number((ms >> 16n) & 0xffn); + bytes[4] = Number((ms >> 8n) & 0xffn); + bytes[5] = Number(ms & 0xffn); + // Version 7 in the high nibble of byte 6; RFC 4122 variant in byte 8. + bytes[6] = (bytes[6] & 0x0f) | 0x70; + bytes[8] = (bytes[8] & 0x3f) | 0x80; + const hex = Array.from(bytes, (b) => b.toString(16).padStart(2, "0")).join(""); + return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}`; +} diff --git a/packages/core/test/sync-protocol.test.ts b/packages/core/test/sync-protocol.test.ts index d5a18ce..a0770cc 100644 --- a/packages/core/test/sync-protocol.test.ts +++ b/packages/core/test/sync-protocol.test.ts @@ -42,3 +42,14 @@ describe("cursor integrity (dee-inzc blocker)", () => { expect(decodeFrame('{"t":"live","root":"r","seq":7}').t).toBe("live"); }); }); + +describe("uuidv7 minting", () => { + it("mints valid, time-ordered UUIDv7", async () => { + const { uuidv7 } = await import("lync-core/uuid"); + const a = uuidv7(1_700_000_000_000); + const b = uuidv7(1_700_000_000_001); + expect(a).toMatch(/^[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/); + expect(a.slice(0, 13) < b.slice(0, 13) || a.slice(0, 13) === b.slice(0, 13)).toBe(true); + expect(uuidv7()).not.toEqual(uuidv7()); // two generations are two events + }); +}); diff --git a/packages/index/package.json b/packages/index/package.json index 7e9de2a..27ae046 100644 --- a/packages/index/package.json +++ b/packages/index/package.json @@ -34,7 +34,8 @@ ], "scripts": { "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit" + "typecheck": "tsc -p tsconfig.json --noEmit", + "prepublishOnly": "tsc -p tsconfig.json" }, "dependencies": { "lync-core": "workspace:*" @@ -48,5 +49,8 @@ "bugs": "https://github.com/deepfates/lync/issues", "publishConfig": { "access": "public" + }, + "engines": { + "node": ">=22" } } diff --git a/packages/server/package.json b/packages/server/package.json index 1d548f2..d89822e 100644 --- a/packages/server/package.json +++ b/packages/server/package.json @@ -19,7 +19,8 @@ ], "scripts": { "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit" + "typecheck": "tsc -p tsconfig.json --noEmit", + "prepublishOnly": "tsc -p tsconfig.json" }, "dependencies": { "lync-core": "workspace:*", @@ -37,5 +38,8 @@ "bugs": "https://github.com/deepfates/lync/issues", "publishConfig": { "access": "public" + }, + "engines": { + "node": ">=22" } } From 737c0ee596f23d97c484997e1295d089810d131a Mon Sep 17 00:00:00 2001 From: deepfates Date: Fri, 10 Jul 2026 12:41:04 -0700 Subject: [PATCH 18/33] pacts: codify import (transcription) and export (projection) law import.md writes down the shipped splice producer convention: deterministic UUIDv8 from source identity (re-import = union no-op, upstream edit = same-id conflict, and that's a feature), the five-axis author envelope with imported_by never actor, source-namespaced kinds, complete-source payloads, RFC 3339 at with surfaced repairs, opt-in marked for byte determinism, zero silent drops with a reconciliation invariant, and verify-clean output. Grounded in splice src/outputs/lync.ts + tests; example id is reproducible from the stated recipe. export.md codifies exports as regenerable projections that say what they dropped: the two blessed families (readable transcripts via branch-tree/ transcript views; training data via the leaderboard view over score/selection annotations, matching FORMAT.md's worked example), provenance columns back to event ids, and the never-list (invent content, silently resolve conflicts, leak suppressed payloads, claim authority). Honest v0: column schemas listed as open points. --- pacts/export.md | 127 +++++++++++++++++++++++++++++++++ pacts/import.md | 183 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 310 insertions(+) create mode 100644 pacts/export.md create mode 100644 pacts/import.md diff --git a/pacts/export.md b/pacts/export.md new file mode 100644 index 0000000..42ca866 --- /dev/null +++ b/pacts/export.md @@ -0,0 +1,127 @@ +# Export Pact + +Status: v0, and younger than the import pact. The view functions it names are +shipped (`packages/core/src/views.ts`); the export file formats built on them +are still settling. The principles are law; the column schemas are early. + +An export is a projection: a view computed over the event set, written down +for a consumer that cannot or will not read `.lync`. It is never a mutation. +The `.lync` file remains the source of truth; the export is a derived, +disposable artifact. If an export and the events disagree, the events win and +the export is stale. Nothing an exporter does writes back — a judgment made +while exporting (a score, a selection, a no-train flag) is a new event +appended by whoever made it, and the next export sees it. + +Two obligations define a conforming export: + +- **Regenerable.** Same event set plus same exporter version yields the same + export. No hidden state, no clock in the output rows (an export manifest + may carry a timestamp; the rows may not depend on one). +- **Says what it dropped.** Every projection excludes things — that is what + makes it a projection. The exclusions are reported, never silent: damaged + and garbage lines, conflict variants, suppressed payloads, annotations it + could not interpret (`ignoredAnnotationIds`), events filtered by + `no-train` or tombstones, and any traversal obstacle (dangling parent, + cycle, conflict) that made the result `partial`. A partial export must say + it is partial. Silence is the only forbidden response; that rule does not + stop at the file boundary. + +All exports project over view-eligible events only (FORMAT.md's taxonomy: +accepted and nonconforming union events, minus conflict variants, with +critical suppression applied). An exporter never reaches past that line: a +suppressed payload stays out of the export, and a same-id conflict is +reported as a conflict, never resolved by picking a favorite. + +## The Two Blessed Projection Families + +### Readable transcripts + +View functions: `lyncBranchTreeView` and `lyncTranscriptView`. + +The branch tree is the whole graph made legible: every eligible event with +its parents, children, roots, and leaves, plus explicit `missingParents` and +`conflictedParents` per node and a `partial` flag when anything is +unresolved. The transcript is one thread through it: from a chosen head, +walk parents to a root (caller-supplied `chooseParent` decides at fan-in; +default is the first held parent), reverse, and number by depth. The view +carries the head's full downset and its obstacles alongside the path, so a +transcript that could not see everything says so. + +A transcript export renders those entries — actor, time, payload text — in +whatever format the consumer reads. What it must keep: each rendered entry +points back to its event id, and the head, the parent choices, and any +`partial` flag ride along. A transcript is a choice of path through +alternatives; an export that hides which path was chosen is not regenerable. + +### Training data + +View function: `lyncLeaderboardView`, over `lync/annotation` events. This +family is not an add-on; it is in the spec's worked example. Five events — +a paragraph `A`, alternatives `B` and `C`, a judge's `score` on `B`, and a +declared `selection` of `B` over `C` — already contain a preference pair: +`B` over `C`, chosen by `deepfates`, with a judge's score on record. The +format's unusual commitment is recording alternatives and choice at +generation time so training exports are reads, not reconstructions. + +The annotation payloads the view interprets, exactly as shipped: + +- `label: "score"` — numeric value in payload `value` (fallback `score`); + targets are the annotation's parents; each target accumulates + `scoreTotal`, `scoreCount`, `scoreMean`, and a per-score record of the + judge (`author`), time, and `basis`. +- `label: "selection"` — payload `chosen` (array of ids) and `shown` (array + of ids); targets are `shown` when non-empty, else the annotation's + parents; each target learns whether it was selected, by whom, and on what + `basis`. Selection is revealed before declared: an extended branch already + IS the choice, so declared selections exist only where topology cannot + reveal one. +- Anything the view cannot interpret (non-numeric score, empty `chosen`) + lands in `ignoredAnnotationIds` — dropped-but-reported. + +From these the two export shapes FORMAT.md names: + +- **Preference pairs (DPO-shaped):** chosen-over-shown-but-not-chosen, from + revealed choices (extended branches) and declared choices (selection + annotations), each pair with its full context (the shared downset), the + judge's identity, and the basis. +- **SFT rows:** root-to-leaf threads of artifacts, filtered through + tombstones and `no-train`, with a windowed mode (per-step slices with + immediate context). Scores and selections rank which leaves are worth + exporting; the leaderboard's ordering (selected count, then mean score, + then total, then id) is the shipped tie-break. + +## What an Exporter Must Preserve, and Must Never Do + +Must preserve, in every family: + +- **Provenance columns.** Every exported row points back to the event ids it + was computed from: the artifact ids in a transcript or SFT row, the + chosen/rejected ids and the annotation id behind a preference pair. An + export you cannot trace back to events is an assertion, not a projection. +- **Author axes as axes.** Judge identity is the annotation's `author` + object; keep `actor`, `operator`, `via`, `imported_by` distinct rather + than collapsing them into one display string (authorship pact). +- **The drop report and the partial flag**, as above. + +Must never: + +- **Invent content.** No paraphrase, no synthesized turns, no filled gaps. A + hole in the graph is exported as a hole (and reported), not smoothed over. +- **Silently dedupe or resolve conflicts.** Two same-id variants are a + surfaced disagreement; an exporter that quietly picks one has forged + history. Exclude both from rows, report the conflict. +- **Leak suppressed payloads.** Honoring critical suppression in exports is + the testable conformance claim of FORMAT.md's obligations section; an + exporter that ships a tombstoned payload has lost the badge. +- **Claim authority.** An export is never merged back as truth and never + cited as evidence over the events it came from. + +## Open Points + +- Exact column schemas for the SFT and DPO row formats, and the manifest + shape for the drop report. +- The windowed SFT mode's window rules. +- How blob-referenced payloads (`{"blob":"sha256:..."}`) ride in exports: + inline, sidecar, or reference. +- Whether a transcript export standard (markdown? ChatML?) is worth blessing + or should stay per-consumer. diff --git a/pacts/import.md b/pacts/import.md new file mode 100644 index 0000000..04cb124 --- /dev/null +++ b/pacts/import.md @@ -0,0 +1,183 @@ +# Import Pact + +Status: v0. This pact codifies a shipped convention, not a proposal. The +reference implementation is splice's lync producer (`src/outputs/lync.ts` and +its tests in `tests/lync/`), which imports Twitter archives and glowfic-dl +JSON exports. Everything below is law that code already obeys. + +An import is a transcription of pre-existing events, not a generation. The +content already happened, somewhere else, at some earlier time, authored by +someone who was not the importer. The importer's whole job is to carry that +fact into the envelope without adding, losing, or claiming anything. Every +rule in this pact is that sentence applied to one field. + +## Identity: Deterministic Ids + +Generators mint UUIDv7 because two identical generations are two events. +Imports are the opposite case, and FORMAT.md's `id` row blesses it: an +importer transcribing pre-existing events derives a deterministic UUIDv8 from +source identity. Same source record in, same event id out, every time, on +every machine. + +Two consequences follow, and both are the point: + +- **Re-import is a union no-op.** Run the importer twice, or on two machines, + and merge the outputs: same id, same body bytes, one event seen twice. + Rule 2 unions them silently. No dedup pass, no "already imported" state. +- **An upstream edit surfaces as a same-id conflict.** If the source record + changes and you re-import, the id is the same but the body bytes differ. + Rule 2 refuses to pick a winner and surfaces both loudly. This is a + FEATURE: the format itself flags that the upstream mutated something it + presented as history. Do not "fix" this by salting ids with import time — + that trades tamper evidence for silent duplication. + +For those consequences to hold, the derivation must be a pure function of +source identity: no clocks, no randomness, no importer hostname, stable +across importer versions. The reference recipe: + +1. Take the identity parts, most general first, e.g. + `("glowfic", "post", "", "")` or + `("twitter", "item", "")`. Include the source namespace and + record type so ids cannot collide across sources or across record types + within one source. +2. SHA-256 over the UTF-8 bytes of each part, each part followed by a single + NUL byte (`0x00`) as terminator — including the last. The NUL terminator + is what keeps `("a","bc")` and `("ab","c")` distinct. +3. Take the first 16 bytes of the digest. Set byte 6 to + `(b[6] & 0x0f) | 0x80` (UUID version 8, RFC 9562 "custom") and byte 8 to + `(b[8] & 0x3f) | 0x80` (RFC 4122 variant). +4. Format as a lowercase hex UUID: `8-4-4-4-12`. + +Readers validate nothing about id shape (FORMAT.md), so the recipe binds +writers only. Any recipe with the same properties conforms; use the reference +recipe unless you have a reason, because two importers that share a recipe +and identity parts converge on the same ids for the same source, and their +outputs union. + +Parents are derived with the same recipe. That means a parent reference +resolves correctly even when the parent was imported by a different run, or +has not been imported yet — a dangling parent is legal, always, and fills in +when union delivers it. + +## The Author Envelope + +Every axis answers a different question. The importer keeps them apart: + +| Axis | Value for imports | Question it answers | +|---|---|---| +| `actor` | The ORIGINAL source identity, the most specific the source record offers (reference order: character display name, character handle, author, screen name, username, handle, account id), fallback `"unknown"` | Who produced the content? | +| `operator` | The human on whose behalf the import runs (reference default: `"deepfates"`) | Under whose responsibility? | +| `via` | The fetching tool that produced the source material, `@`, e.g. `glowfic-dl@unknown`, `twitter-archive@unknown` | What mediated it out of the source system? | +| `imported_by` | The importer itself, `@`, e.g. `splice/glowfic-json@0.1` | What transcribed it into lync? | +| `source` | Locator: `:`, e.g. `https://glowfic.com/posts/5506:reply-1739834` | Where does the original live? | + +The rule with teeth, from FORMAT.md and the authorship pact: **`imported_by` +is never used as `actor`.** The importer did not write the content; it may +not claim it. When the source record carries no identity at all, the honest +actor is `"unknown"` — still not the importer. The reference tests assert +`author.actor !== author.imported_by` on every event. + +## Kinds Are Source-Namespaced + +The namespace is the source system: `glowfic/thread`, `glowfic/post`, +`twitter/tweet`. Do not launder imported material into generic kinds; the +kind string is where a reader learns which pact (this one, plus the source's +own shape) explains the payload. A container record (a thread, a +conversation) gets its own event, and its items parent to it. + +Parents transcribe the source's structure: explicit reply linkage when the +source has it; when the source is a strict sequence with no finer reply +metadata (glowfic posts are), each item parents to the previous item and the +first parents to the container event. Sequence-as-parents is honest there +because the sequence IS the source's structure. + +## Payload and Time + +The payload is the complete original source object. Not a summary, not the +fields you currently need — all of it, verbatim. The envelope never +interprets payloads, storage is cheap, and the field you dropped is the field +the next decade wants. Only when the source provides no raw object at all +does the importer's own normalized record stand in as payload; either way, +nothing is discarded. + +`at` is the source-claimed creation time, normalized to RFC 3339: + +- Already RFC 3339: use the source string verbatim. Bytes are canonical; + never reformat a timestamp that already conforms. +- Parseable but non-conforming (e.g. `"Jan 04, 2022 7:55 PM"`): convert, and + record the repair — original value, value used, reason — in stats. +- Missing or unparseable: substitute a deterministic fallback (the epoch, or + the opt-in import time below) and record that too. Never substitute an + unrecorded "now": it breaks byte determinism invisibly. + +`marked` (import time) is OPT-IN, and off by default. This is a determinism +rule, not a style preference: a default of "now" would make two imports of +the same source differ in body bytes under one id, which union then surfaces +as a same-id conflict — a false alarm that buries the real one. Identical +imports must be byte-identical. Callers who want import time on record pass +it explicitly and accept that their output no longer byte-matches other runs. + +A conforming import of one glowfic post, wrapped here for reading only +(stored form is one line). The id is the reference recipe applied to +`("glowfic", "post", "5506", "reply-1739834")` — reproduce it to check your +implementation: + +```json +{ + "v": 1, + "id": "bebac0c3-c485-8748-bdca-bc12dedab993", + "kind": "glowfic/post", + "at": "2018-06-04T21:39:00.000Z", + "author": { + "actor": "Carissa Sevar", + "operator": "deepfates", + "via": "glowfic-dl@unknown", + "imported_by": "splice/glowfic-json@0.1", + "source": "https://glowfic.com/posts/5506:reply-1739834" + }, + "parents": [""], + "payload": { "post_id": "reply-1739834", "author": "lintamande", "...": "the complete original post object" } +} +``` + +## Zero Silent Drops + +Every source record either becomes an event or lands in an explicit skip +entry. There is no third path. The importer returns stats: + +- `sourceRecords`: how many records the source presented. +- `emitted`: how many events were produced. +- `skipped`: one entry per record that could not become an event, with its + index, a reason, and the offending value for audit. A record with no + stable source id cannot mint a deterministic event id — that is the + canonical skip reason, and it is reported, not swallowed. +- `timestampFallbacks`: one entry per repaired or substituted timestamp. + +The reconciliation invariant is the whole point: +`emitted + skipped.length === sourceRecords`. The reference implementation +throws rather than return stats that do not reconcile. + +## Verify Before You Believe + +Writing the file is not the end of the import. The importer re-parses its own +output with a conforming reader and requires EVERY line to classify +`accepted` — zero garbage, zero damaged, zero nonconforming, zero conflict +variants — and the accepted count to equal the emitted count. A failed verify +is a loud error, not a warning. Testimony ("I wrote 31 events") is not +evidence; the verifier's counts are. + +## Conformance Checklist + +An importer conforms to this pact when: + +1. Ids are a pure deterministic function of source identity (UUIDv8), and + re-running the importer on unchanged source yields byte-identical output. +2. `actor` is the original source identity or `"unknown"`; `imported_by` + names the importer and is never the actor; `operator`, `via`, and `source` + are set as above. +3. Kinds are namespaced by source system; payloads carry the complete + original source object; parents transcribe source structure. +4. `at` is source-claimed time in RFC 3339, verbatim when already + conforming; every repair and fallback is recorded; `marked` is opt-in. +5. Stats reconcile exactly and every drop is an explicit entry. +6. The written file verifies clean: all lines accepted. From 0eec60c4624ce66e9696a6776c7414bc6d3e2a2d Mon Sep 17 00:00:00 2001 From: deepfates Date: Sun, 12 Jul 2026 15:23:06 -0700 Subject: [PATCH 19/33] polish: make the npm storefronts truthful and complete A stranger's first contact with lync is a package page on npm, not the repo README. That surface was half-built: - Per-package READMEs were stubs (152-731 bytes); core's example didn't even run. All five are now real pages: pitch, install, runnable examples, guarantees, links to the spec and pacts. - No tarball shipped a LICENSE (root LICENSE doesn't ride files:[dist]). Every package now carries its own. - Zero keywords anywhere; lync-core's description still said 'addressable looms' (pre-cutover vocabulary). Fixed both. - The repo README named pacts without linking them; import/export pacts are now discoverable. Enforcement: scripts/readme-examples-smoke.mjs extracts the fenced examples from every package README and executes them against the built packages (self-reference resolution, scratch cwd). It already caught four lies in my own drafts (upsert->addLoom, open->openReference, missing toUrl location arg, wrong lync-index import path) before they shipped. Wired into pnpm verify, so CI now fails if a doc example rots. --- README.md | 5 +- package.json | 3 +- packages/cli/LICENSE | 21 ++++ packages/cli/README.md | 36 +++++-- packages/cli/package.json | 15 ++- packages/client/LICENSE | 21 ++++ packages/client/README.md | 32 +++++- packages/client/package.json | 13 ++- packages/core/LICENSE | 21 ++++ packages/core/README.md | 88 ++++++++++++++--- packages/core/package.json | 16 ++- packages/index/LICENSE | 21 ++++ packages/index/README.md | 29 +++++- packages/index/package.json | 12 ++- packages/server/LICENSE | 21 ++++ packages/server/README.md | 56 ++++++++++- packages/server/package.json | 15 ++- scripts/readme-examples-smoke.mjs | 155 ++++++++++++++++++++++++++++++ 18 files changed, 542 insertions(+), 38 deletions(-) create mode 100644 packages/cli/LICENSE create mode 100644 packages/client/LICENSE create mode 100644 packages/core/LICENSE create mode 100644 packages/index/LICENSE create mode 100644 packages/server/LICENSE create mode 100644 scripts/readme-examples-smoke.mjs diff --git a/README.md b/README.md index 1a0c554..4cbe235 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,10 @@ Under those verbs, every line has the same envelope: That line can be copied to another file, merged back later, verified byte for byte, and read by software that has never heard of `notes/text`. Unknown kinds are carried and traversed; meaning belongs to pacts layered above the -format. +format — see [pacts/import.md](./pacts/import.md) (imports are transcription: +deterministic ids, provenance preserved, zero silent drops) and +[pacts/export.md](./pacts/export.md) (exports are projections of the event +log, including training data). ## The Format diff --git a/package.json b/package.json index 2f911c0..eaebd81 100644 --- a/package.json +++ b/package.json @@ -12,7 +12,8 @@ "test": "vitest run", "typecheck": "pnpm build && pnpm -r typecheck", "guard:paths": "bash scripts/no-machine-local-paths.sh", - "verify": "pnpm guard:paths && pnpm test && pnpm typecheck" + "verify": "pnpm guard:paths && pnpm test && pnpm typecheck && pnpm smoke:readme", + "smoke:readme": "node scripts/readme-examples-smoke.mjs" }, "devDependencies": { "@types/node": "^22.14.0", diff --git a/packages/cli/LICENSE b/packages/cli/LICENSE new file mode 100644 index 0000000..cfbc3e3 --- /dev/null +++ b/packages/cli/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 deepfates + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/packages/cli/README.md b/packages/cli/README.md index 0370c42..58c74fb 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -1,11 +1,16 @@ # lync-cli -Command-line tools for lync files: append-only loom logs, one JSON event per -line, merged losslessly by set-union. +The `lync` command: work with `.lync` files — append-only JSONL interaction +history where each line is one immutable event and merge is set union by +event id. ```bash npm install -g lync-cli +``` + +## Seven verbs +```bash lync init story.lync printf '%s\n' '{"kind":"notes/text","author":{"actor":"you"},"payload":{"text":"Once..."}}' | lync append story.lync lync verify story.lync @@ -13,7 +18,26 @@ lync view story.lync --as transcript lync merge story.lync other.lync -o merged.lync ``` -Seven verbs: `init`, `append`, `verify`, `merge`, `view`, `serve` (the -line-sync relay), and `sync` (converge a file with a relay, `--follow` to stay -live). Run `lync --help` for -usage. Full docs: https://github.com/deepfates/lync#readme +`append` fills the envelope for you: a UUIDv7 id, the current timestamp, `v`, +and `parents` default in; anything you supply is kept. `verify` reports what +every physical line is — accepted, nonconforming, damaged, garbage, or +conflict variant — and never drops bytes. `view` renders `transcript` or +`tree`. + +## Sync + +Any lync file can converge with any other copy through a relay: + +```bash +lync serve ./rooms --port 8787 # the relay: one append-only file per root +lync sync story.lync ws://host:8787 # one-shot: push what it lacks, pull what you lack +lync sync story.lync ws://host:8787 --follow # stay live until Ctrl-C +``` + +The relay stores each root as a plain `.lync` file you can read with any lync +tool. Same-id-different-bytes is never resolved: both variants are kept and +both sides are told loudly. An interrupted sync resumes from a per-root cursor +(`.sync.json`). + +Run `lync --help` for full usage. Format spec and docs: +https://github.com/deepfates/lync#readme diff --git a/packages/cli/package.json b/packages/cli/package.json index d6a4577..0ad4129 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,7 +1,7 @@ { "name": "lync-cli", "version": "0.2.0", - "description": "Command-line tools for lync files.", + "description": "The lync command: init, append, verify, merge, view, serve, sync for .lync event-log files.", "type": "module", "license": "MIT", "sideEffects": false, @@ -46,5 +46,16 @@ }, "engines": { "node": ">=22" - } + }, + "keywords": [ + "lync", + "jsonl", + "append-only", + "event-log", + "crdt-adjacent", + "local-first", + "cli", + "sync", + "merge" + ] } diff --git a/packages/client/LICENSE b/packages/client/LICENSE new file mode 100644 index 0000000..cfbc3e3 --- /dev/null +++ b/packages/client/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 deepfates + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/packages/client/README.md b/packages/client/README.md index e81f66c..82a407b 100644 --- a/packages/client/README.md +++ b/packages/client/README.md @@ -1,3 +1,33 @@ # lync-client -The lync loom client: a small facade over looms and indexes that resolves loom/turn/thread/index references. See https://github.com/deepfates/lync. +The lync loom client: one object that pairs looms +([lync-core](https://www.npmjs.com/package/lync-core)) with an index +([lync-index](https://www.npmjs.com/package/lync-index)) and resolves +loom/turn/thread/index references to and from URLs. + +```bash +npm install lync-client +``` + +```ts +import { createLyncLooms } from "lync-core/looms"; +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createMemoryLoomIndexes } from "lync-index/memory"; +import { createLoomClient } from "lync-client"; + +const client = createLoomClient({ + looms: createLyncLooms({ store: createMemoryEventStore(), author: { actor: "you" } }), + indexes: createMemoryLoomIndexes(), +}); + +const info = await client.looms.create({ title: "Story" }); +const ref = client.references.loom(info.id); + +// Round-trip a reference through a shareable URL (?ref=...). +// In the browser, pass `window.location` instead of a URL object. +const url = client.references.toUrl(ref, new URL("https://example.com/story")); +const opened = await client.openReference(client.references.fromUrl(new URL(url))); +console.log(opened.kind); // "loom" — opened.loom is ready to appendTurn +``` + +Full docs: https://github.com/deepfates/lync#readme diff --git a/packages/client/package.json b/packages/client/package.json index dc7f821..debcff7 100644 --- a/packages/client/package.json +++ b/packages/client/package.json @@ -48,5 +48,16 @@ }, "engines": { "node": ">=22" - } + }, + "keywords": [ + "lync", + "jsonl", + "append-only", + "event-log", + "crdt-adjacent", + "local-first", + "client", + "loom", + "references" + ] } diff --git a/packages/core/LICENSE b/packages/core/LICENSE new file mode 100644 index 0000000..cfbc3e3 --- /dev/null +++ b/packages/core/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 deepfates + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/packages/core/README.md b/packages/core/README.md index b74bb3f..df11c27 100644 --- a/packages/core/README.md +++ b/packages/core/README.md @@ -1,27 +1,85 @@ # lync-core -Core APIs for lync files: append-only loom logs where each line is one JSON -event, files are merged losslessly by set-union, and every physical line is -classified and kept. +The reference implementation of the lync format: `.lync` append-only JSONL +files of interaction history. Each line is one immutable event with an +envelope, parent links, provenance, and a payload owned by the event kind. +Merge is set union by event id. Branch trees, transcripts, memory views, and +leaderboards are computed views over the same event set. + +Zero runtime dependencies. Runs in Node (>=22) and the browser. + +```bash +npm install lync-core +``` + +## Parse, union, view ```ts import { parseLyncFiles } from "lync-core/events"; +import { lyncBranchTreeView, lyncTranscriptView } from "lync-core/views"; + +const bytes = new TextEncoder().encode( + '{"v":1,"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"you"},"parents":[],"payload":{"text":"Once..."}}\n', +); + +const parsed = parseLyncFiles([{ file: "story.lync", bytes }]); +console.log(parsed.lines[0].class); // "accepted" +console.log(lyncBranchTreeView(parsed).roots); +console.log(lyncTranscriptView(parsed, "root").path); +``` + +`parseLyncFiles` classifies every physical line and keeps the original bytes — +accepted events, nonconforming-but-carried lines, damaged lines, garbage, and +conflict variants (same id, different bytes) are all preserved and reported, +never silently dropped. `exportCarriedLyncBytes(parsed)` re-emits the carried +bytes. + +## Stores and looms + +Event stores share one contract over memory, file, and IndexedDB backends. The +loom API gives programs turns and threads instead of raw events, on top of any +store: + +```ts +import { createLyncLooms } from "lync-core/looms"; import { createMemoryEventStore } from "lync-core/memory-log"; -const store = createMemoryEventStore(); -await store.append({ - v: 1, - id: "root", - kind: "lync/loom", - at: "2026-07-06T04:12:31Z", - author: { actor: "you" }, - parents: [], - payload: { meta: { title: "Story" } }, +const looms = createLyncLooms({ + store: createMemoryEventStore(), + author: { actor: "you", via: "my-app@0.1" }, }); -const parsed = parseLyncFiles([{ file: "story.lync", bytes: line }]); -console.log(parsed.unionEventIds); +const info = await looms.create({ title: "Story" }); +const loom = await looms.open(info.id); +const first = await loom.appendTurn(null, { text: "Once..." }); +await loom.appendTurn(first.id, { text: "Then..." }); +``` + +## Live sync + +Wrap any store in `createSyncedStore` and it converges with a relay +([lync-server](https://www.npmjs.com/package/lync-server)) over five JSON +frames. Local appends push, remote lines surface reactively, offline appends +queue and flush on reconnect: + +```ts +import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; + +const store = createSyncedStore(localStore, createWebSocketTransport("wss://host/lync")); ``` -Full format spec, subpath exports, and examples: +## Subpath exports + +- `lync-core/events` — line parsing, carried-byte export, incremental union +- `lync-core/store` — the event-store contract and serialization +- `lync-core/memory-log`, `lync-core/file-log`, `lync-core/idb-log` — stores +- `lync-core/views` — branch tree, transcript, memory, leaderboard +- `lync-core/looms` — the loom/turn API +- `lync-core/synced-store` — live sync decorator and WebSocket transport +- `lync-core/uuid` — zero-dep UUIDv7 for event ids + +Normative format spec: +[FORMAT.md](https://github.com/deepfates/lync/blob/main/FORMAT.md). Import and +export conventions: +[pacts/](https://github.com/deepfates/lync/tree/main/pacts). Full docs: https://github.com/deepfates/lync#readme diff --git a/packages/core/package.json b/packages/core/package.json index 73b3ba0..1c26f89 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -1,7 +1,7 @@ { "name": "lync-core", "version": "0.2.0", - "description": "Core APIs for local-first addressable looms.", + "description": "The lync format: append-only JSONL event logs merged by set union. Parsing, stores, views, looms, live sync. Zero dependencies.", "type": "module", "license": "MIT", "sideEffects": false, @@ -109,5 +109,17 @@ }, "engines": { "node": ">=22" - } + }, + "keywords": [ + "lync", + "jsonl", + "append-only", + "event-log", + "crdt-adjacent", + "local-first", + "sync", + "merge", + "loom", + "transcript" + ] } diff --git a/packages/index/LICENSE b/packages/index/LICENSE new file mode 100644 index 0000000..cfbc3e3 --- /dev/null +++ b/packages/index/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 deepfates + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/packages/index/README.md b/packages/index/README.md index bca19b4..5d0da81 100644 --- a/packages/index/README.md +++ b/packages/index/README.md @@ -1,3 +1,30 @@ # lync-index -The lync index: track a collection of looms and subscribe to changes. Depends only on `lync-core`. See https://github.com/deepfates/lync. +An index of many lync looms: track a collection, upsert entries, and subscribe +to changes. Depends only on +[lync-core](https://www.npmjs.com/package/lync-core). + +```bash +npm install lync-index +``` + +```ts +import { loomRef } from "lync-core"; +import { createMemoryLoomIndexes } from "lync-index/memory"; + +const indexes = createMemoryLoomIndexes(); +const index = await indexes.create({ title: "My looms" }); + +index.subscribe((event) => console.log("index changed:", event.type)); +await index.addLoom(loomRef("loom-1"), { title: "Story" }); + +console.log((await index.entries()).map((entry) => entry.title)); +``` + +Entries carry a loom reference, optional title/kind/meta, and timestamps. +`export()`/`import()` round-trip a whole index as a snapshot. + +Typically used through +[lync-client](https://www.npmjs.com/package/lync-client), which pairs an index +with looms and reference resolution. Full docs: +https://github.com/deepfates/lync#readme diff --git a/packages/index/package.json b/packages/index/package.json index 27ae046..6283d9c 100644 --- a/packages/index/package.json +++ b/packages/index/package.json @@ -52,5 +52,15 @@ }, "engines": { "node": ">=22" - } + }, + "keywords": [ + "lync", + "jsonl", + "append-only", + "event-log", + "crdt-adjacent", + "local-first", + "index", + "loom" + ] } diff --git a/packages/server/LICENSE b/packages/server/LICENSE new file mode 100644 index 0000000..cfbc3e3 --- /dev/null +++ b/packages/server/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 deepfates + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/packages/server/README.md b/packages/server/README.md index dc8e6b9..a834bc8 100644 --- a/packages/server/README.md +++ b/packages/server/README.md @@ -1,7 +1,53 @@ # lync-server -The lync line-sync relay. Run it standalone with `startLyncServe`, or embed it -in an existing Node HTTP server with `createLyncRelay` and call `handleUpgrade` -from your own `upgrade` listener. It stores each root as a plain append-only -`.lync` file and never parses a line beyond its id. See -https://github.com/deepfates/lync. +The lync line-sync relay. It moves canonical `.lync` line bytes between +subscribers over five JSON frames (`sub`, `ev`, `live`, `presence`, `err`) and +has no merge logic — lync events are immutable and merge is set union by id, +so echoes are duplicate no-ops. Each root is stored as a plain append-only +`.lync` file you can read with any lync tool. The relay never parses a line +beyond extracting its id. + +```bash +npm install lync-server +``` + +## Standalone + +```ts +import { startLyncServe } from "lync-server"; + +const server = await startLyncServe({ dir: "./rooms", port: 8787 }); +console.log("relay on", server.port); +// later: await server.close(); +``` + +## On an existing HTTP server + +```ts +import { createServer } from "node:http"; +import { attachLyncServer } from "lync-server"; + +const httpServer = createServer(app); +const lync = attachLyncServer(httpServer, { + storageDir: "./rooms", + path: "/lync", // default + keepAliveInterval: 30_000, // optional: ping through idle proxies + maxConnections: 500, // optional + authenticate: (req) => checkSession(req), // optional, after token check +}); +httpServer.listen(3000); +``` + +For full control, `createLyncRelay` gives you `handleUpgrade` to call from +your own `upgrade` listener. + +Guarantees: same-id-different-bytes is never resolved — both variants are +kept (a `.conflicts` sidecar) and both sides are told loudly. Persist failures +are broadcast, never swallowed. A truncated final line after a crash is +sealed and surfaced as damaged, never eaten. `token` requires +`Authorization: Bearer ` on every upgrade. + +Client side: `lync sync` from +[lync-cli](https://www.npmjs.com/package/lync-cli), or `createSyncedStore` +from [lync-core](https://www.npmjs.com/package/lync-core) inside an app. Full +docs: https://github.com/deepfates/lync#readme diff --git a/packages/server/package.json b/packages/server/package.json index d89822e..b174b6f 100644 --- a/packages/server/package.json +++ b/packages/server/package.json @@ -1,7 +1,7 @@ { "name": "lync-server", "version": "0.2.0", - "description": "The lync line-sync relay: mountable on any Node server, or run standalone.", + "description": "The lync line-sync relay: mount on any Node server, or run standalone. Stores each root as a plain .lync file.", "type": "module", "license": "MIT", "sideEffects": false, @@ -41,5 +41,16 @@ }, "engines": { "node": ">=22" - } + }, + "keywords": [ + "lync", + "jsonl", + "append-only", + "event-log", + "crdt-adjacent", + "local-first", + "websocket", + "relay", + "sync-server" + ] } diff --git a/scripts/readme-examples-smoke.mjs b/scripts/readme-examples-smoke.mjs new file mode 100644 index 0000000..df2cad0 --- /dev/null +++ b/scripts/readme-examples-smoke.mjs @@ -0,0 +1,155 @@ +#!/usr/bin/env node +// Executes the ```ts examples in packages/*/README.md against the built +// packages, so the npm landing pages can never drift into lies. +// +// Each snippet runs as-written from inside its package directory: Node +// resolves the package's own name (and its workspace deps) through +// package.json self-reference, exactly like an installed consumer. +// +// Blocks marked fragment:true are illustrative partials (free variables like +// an existing `app` or `localStore`) and are skipped, listed loudly. +import { execFileSync, spawn } from "node:child_process"; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; + +const root = join(fileURLToPath(import.meta.url), "..", ".."); + +const PLAN = [ + { pkg: "core", blocks: [{ i: 0 }, { i: 1 }, { i: 2, fragment: "wraps an undefined localStore; needs a live relay" }] }, + { pkg: "cli", bash: true }, + { pkg: "server", blocks: [{ i: 0, daemon: "relay on" }, { i: 1, fragment: "embeds into an existing app server (free vars: app, checkSession)" }] }, + { pkg: "index", blocks: [{ i: 0 }] }, + { pkg: "client", blocks: [{ i: 0 }] }, +]; + +function tsBlocks(markdown) { + const blocks = []; + const re = /```ts\n([\s\S]*?)```/g; + let m; + while ((m = re.exec(markdown)) !== null) blocks.push(m[1]); + return blocks; +} + +function bashBlocks(markdown) { + const blocks = []; + const re = /```bash\n([\s\S]*?)```/g; + let m; + while ((m = re.exec(markdown)) !== null) blocks.push(m[1]); + return blocks; +} + +let failures = 0; +const scratchRoots = []; + +function runSnippet(pkg, index, code, daemonMatch) { + const pkgDir = join(root, "packages", pkg); + const scriptPath = join(pkgDir, `.readme-smoke-${index}.tmp.mjs`); + const scratch = mkdtempSync(join(tmpdir(), "lync-readme-smoke-")); + scratchRoots.push(scratch); + // Run with cwd in a scratch dir so relative paths ("./rooms") never touch + // the repo; module resolution follows the script's location, not cwd. + writeFileSync(scriptPath, code); + try { + if (!daemonMatch) { + execFileSync(process.execPath, [scriptPath], { cwd: scratch, stdio: ["ignore", "pipe", "pipe"], timeout: 30_000 }); + console.log(`ok ${pkg} README block ${index}`); + return Promise.resolve(); + } + // Long-running example: pass = expected line appears, then we kill it. + return new Promise((resolve) => { + const child = spawn(process.execPath, [scriptPath], { cwd: scratch }); + let out = ""; + const timer = setTimeout(() => { + child.kill("SIGKILL"); + console.error(`FAIL ${pkg} README block ${index}: never printed ${JSON.stringify(daemonMatch)}\n${out}`); + failures += 1; + resolve(); + }, 15_000); + child.stdout.on("data", (chunk) => { + out += String(chunk); + if (out.includes(daemonMatch)) { + clearTimeout(timer); + child.kill("SIGTERM"); + console.log(`ok ${pkg} README block ${index} (daemon: saw ${JSON.stringify(daemonMatch)})`); + resolve(); + } + }); + child.on("exit", () => { + rmSync(scriptPath, { force: true }); + }); + }).finally(() => rmSync(scriptPath, { force: true })); + } catch (error) { + console.error(`FAIL ${pkg} README block ${index}:\n${error.stderr ?? error.message}`); + failures += 1; + return Promise.resolve(); + } finally { + if (!daemonMatch) rmSync(scriptPath, { force: true }); + } +} + +function runCliBlock(pkg, index, code) { + const scratch = mkdtempSync(join(tmpdir(), "lync-readme-smoke-cli-")); + scratchRoots.push(scratch); + const bin = join(root, "packages", "cli", "bin", "lync.js"); + const lines = code + .split("\n") + .map((line) => line.trim()) + .filter((line) => line && !line.startsWith("#") && !line.startsWith("npm install")); + for (const line of lines) { + // serve/sync need a live relay pair; covered by the sync test suite. + if (line.startsWith("lync serve") || line.startsWith("lync sync")) { + console.log(`skip ${pkg} README block ${index} line (needs live relay): ${line}`); + continue; + } + if (line.startsWith("lync merge")) { + // The example merges with an `other.lync` the reader is presumed to + // have; give it one so the command runs as written. + execFileSync("node", [bin, "init", "other.lync"], { cwd: scratch }); + } + // Only rewrite `lync` as a command token (line start or after a pipe), + // never the substring inside filenames like story.lync. + const cmd = line.replace(/(^|\| )lync /g, `$1node ${bin} `); + try { + execFileSync("bash", ["-c", cmd], { cwd: scratch, stdio: ["ignore", "pipe", "pipe"], timeout: 30_000 }); + } catch (error) { + console.error(`FAIL ${pkg} README block ${index} line: ${line}\n${error.stderr}`); + failures += 1; + return; + } + } + console.log(`ok ${pkg} README block ${index} (cli story)`); +} + +for (const entry of PLAN) { + const markdown = readFileSync(join(root, "packages", entry.pkg, "README.md"), "utf8"); + if (entry.bash) { + bashBlocks(markdown).forEach((code, i) => runCliBlock(entry.pkg, i, code)); + continue; + } + const blocks = tsBlocks(markdown); + const planned = entry.blocks ?? []; + const extras = blocks.length - planned.length; + if (extras !== 0) { + console.error(`FAIL ${entry.pkg}: README has ${blocks.length} ts blocks but the plan covers ${planned.length} — update scripts/readme-examples-smoke.mjs`); + failures += 1; + } + for (const spec of planned) { + const code = blocks[spec.i]; + if (code === undefined) continue; + if (spec.fragment) { + console.log(`skip ${entry.pkg} README block ${spec.i} (fragment: ${spec.fragment})`); + continue; + } + await runSnippet(entry.pkg, spec.i, code, spec.daemon); + } +} + +for (const scratch of scratchRoots) rmSync(scratch, { recursive: true, force: true }); + +if (failures > 0) { + console.error(`readme-examples-smoke: ${failures} failure(s)`); + process.exit(1); +} +console.log("readme-examples-smoke passed"); From cef2267eb897302b2871a9cdc7e42928cac09136 Mon Sep 17 00:00:00 2001 From: deepfates Date: Sun, 12 Jul 2026 17:29:16 -0700 Subject: [PATCH 20/33] check-readme-examples: the README is the contract, no out-of-band plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces readme-examples-smoke. The old script carried a hardcoded list of which fenced blocks to run and which to skip — configuration that would drift the moment anyone adds an example. Now the READMEs carry their own contract: every fenced ts/bash block executes by default; a block that can't run alone declares itself with an comment directly above the code (invisible on npm), and long-running examples declare the output line that proves startup. The checker is a dumb executor with no special cases, and it also fails if a published package has no README or no examples at all. cli merge story made self-contained (append creates other.lync in-story) instead of the runner conjuring the file. --- README.md | 7 +- package.json | 4 +- packages/cli/README.md | 2 + packages/core/README.md | 1 + packages/server/README.md | 2 + scripts/check-readme-examples.mjs | 154 +++++++++++++++++++++++++++++ scripts/readme-examples-smoke.mjs | 155 ------------------------------ 7 files changed, 166 insertions(+), 159 deletions(-) create mode 100644 scripts/check-readme-examples.mjs delete mode 100644 scripts/readme-examples-smoke.mjs diff --git a/README.md b/README.md index 4cbe235..767ab4c 100644 --- a/README.md +++ b/README.md @@ -257,8 +257,11 @@ pnpm test pnpm verify ``` -`pnpm verify` runs tests, builds packages, and typechecks emitted package -surfaces. +`pnpm verify` runs tests, builds packages, typechecks emitted package +surfaces, and executes every fenced example in the package READMEs against +the built packages (`pnpm check:examples`). README examples are contract: +a block runs as-written unless an `` comment above +it declares why it can't run alone. For a fresh clone, `pnpm install && pnpm build` is the supported setup sequence. After that, `pnpm exec lync --help` should print the CLI help from the workspace diff --git a/package.json b/package.json index eaebd81..dd291a7 100644 --- a/package.json +++ b/package.json @@ -12,8 +12,8 @@ "test": "vitest run", "typecheck": "pnpm build && pnpm -r typecheck", "guard:paths": "bash scripts/no-machine-local-paths.sh", - "verify": "pnpm guard:paths && pnpm test && pnpm typecheck && pnpm smoke:readme", - "smoke:readme": "node scripts/readme-examples-smoke.mjs" + "verify": "pnpm guard:paths && pnpm test && pnpm typecheck && pnpm check:examples", + "check:examples": "node scripts/check-readme-examples.mjs" }, "devDependencies": { "@types/node": "^22.14.0", diff --git a/packages/cli/README.md b/packages/cli/README.md index 58c74fb..5a1d9eb 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -15,6 +15,7 @@ lync init story.lync printf '%s\n' '{"kind":"notes/text","author":{"actor":"you"},"payload":{"text":"Once..."}}' | lync append story.lync lync verify story.lync lync view story.lync --as transcript +printf '%s\n' '{"kind":"notes/text","author":{"actor":"friend"},"payload":{"text":"Then..."}}' | lync append other.lync lync merge story.lync other.lync -o merged.lync ``` @@ -28,6 +29,7 @@ conflict variant — and never drops bytes. `view` renders `transcript` or Any lync file can converge with any other copy through a relay: + ```bash lync serve ./rooms --port 8787 # the relay: one append-only file per root lync sync story.lync ws://host:8787 # one-shot: push what it lacks, pull what you lack diff --git a/packages/core/README.md b/packages/core/README.md index df11c27..e8940ba 100644 --- a/packages/core/README.md +++ b/packages/core/README.md @@ -62,6 +62,7 @@ Wrap any store in `createSyncedStore` and it converges with a relay frames. Local appends push, remote lines surface reactively, offline appends queue and flush on reconnect: + ```ts import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; diff --git a/packages/server/README.md b/packages/server/README.md index a834bc8..49cf331 100644 --- a/packages/server/README.md +++ b/packages/server/README.md @@ -13,6 +13,7 @@ npm install lync-server ## Standalone + ```ts import { startLyncServe } from "lync-server"; @@ -23,6 +24,7 @@ console.log("relay on", server.port); ## On an existing HTTP server + ```ts import { createServer } from "node:http"; import { attachLyncServer } from "lync-server"; diff --git a/scripts/check-readme-examples.mjs b/scripts/check-readme-examples.mjs new file mode 100644 index 0000000..78ec328 --- /dev/null +++ b/scripts/check-readme-examples.mjs @@ -0,0 +1,154 @@ +#!/usr/bin/env node +// Executes every fenced example in packages/*/README.md against the built +// packages. The package READMEs are the npm landing pages; their examples +// are contract, not illustration — this check fails `pnpm verify` (and CI) +// when one stops running. +// +// The contract lives in the READMEs themselves. A fenced ```ts or ```bash +// block runs as-written unless an HTML comment directly above it declares +// otherwise: +// +// +// +// +// There is no other configuration: a new example is checked by default. +// +// ts blocks execute from inside their package directory — Node's package +// self-reference resolves the package's own name and its workspace deps +// exactly like an installed consumer — with cwd in a scratch dir so relative +// paths never touch the repo. bash blocks run with the `lync` command token +// rewritten to the workspace bin; `npm install` lines are skipped (noted), +// since installing is the reader's step, not the example's. +import { execFileSync, spawn } from "node:child_process"; +import { mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; + +const root = join(fileURLToPath(import.meta.url), "..", ".."); +const packagesDir = join(root, "packages"); + +const BLOCK_RE = /(?:\s*\n)?```(ts|bash)\n([\s\S]*?)```/g; + +function parseDirective(text) { + if (!text) return { mode: "run" }; + if (text.startsWith("fragment")) return { mode: "fragment", reason: text }; + if (text.startsWith("daemon")) { + const expect = text.match(/expect "([^"]+)"/); + if (!expect) throw new Error(`daemon directive without an expect string: ${text}`); + return { mode: "daemon", expect: expect[1] }; + } + throw new Error(`unknown example directive: ${text}`); +} + +let failures = 0; +const scratchRoots = []; + +function scratchDir(label) { + const dir = mkdtempSync(join(tmpdir(), `lync-readme-${label}-`)); + scratchRoots.push(dir); + return dir; +} + +function runTs(pkg, index, code, directive) { + const scriptPath = join(packagesDir, pkg, `.readme-example-${index}.tmp.mjs`); + writeFileSync(scriptPath, code); + const cwd = scratchDir(pkg); + try { + if (directive.mode !== "daemon") { + execFileSync(process.execPath, [scriptPath], { cwd, stdio: ["ignore", "pipe", "pipe"], timeout: 30_000 }); + console.log(`ok ${pkg} ts example ${index}`); + return Promise.resolve(); + } + return new Promise((resolve) => { + const child = spawn(process.execPath, [scriptPath], { cwd }); + let out = ""; + const timer = setTimeout(() => { + child.kill("SIGKILL"); + console.error(`FAIL ${pkg} ts example ${index}: never printed ${JSON.stringify(directive.expect)}\n${out}`); + failures += 1; + resolve(); + }, 15_000); + child.stdout.on("data", (chunk) => { + out += String(chunk); + if (out.includes(directive.expect)) { + clearTimeout(timer); + child.kill("SIGTERM"); + console.log(`ok ${pkg} ts example ${index} (daemon: saw ${JSON.stringify(directive.expect)})`); + resolve(); + } + }); + }).finally(() => rmSync(scriptPath, { force: true })); + } catch (error) { + console.error(`FAIL ${pkg} ts example ${index}:\n${error.stderr ?? error.message}`); + failures += 1; + return Promise.resolve(); + } finally { + if (directive.mode !== "daemon") rmSync(scriptPath, { force: true }); + } +} + +function runBash(pkg, index, code) { + const cwd = scratchDir(`${pkg}-bash`); + const bin = join(packagesDir, "cli", "bin", "lync.js"); + const lines = code + .split("\n") + .map((line) => line.trim()) + .filter((line) => line && !line.startsWith("#")); + let ran = 0; + for (const line of lines) { + if (line.startsWith("npm install")) { + console.log(`note ${pkg} bash example ${index}: install line left to the reader: ${line}`); + continue; + } + const cmd = line.replace(/(^|\| )lync /g, `$1node ${bin} `); + try { + execFileSync("bash", ["-c", cmd], { cwd, stdio: ["ignore", "pipe", "pipe"], timeout: 30_000 }); + ran += 1; + } catch (error) { + console.error(`FAIL ${pkg} bash example ${index} line: ${line}\n${error.stderr}`); + failures += 1; + return; + } + } + console.log(`ok ${pkg} bash example ${index} (${ran} command${ran === 1 ? "" : "s"})`); +} + +const packages = readdirSync(packagesDir).sort(); +for (const pkg of packages) { + let markdown; + try { + markdown = readFileSync(join(packagesDir, pkg, "README.md"), "utf8"); + } catch { + console.error(`FAIL ${pkg}: no README.md — every published package needs a landing page`); + failures += 1; + continue; + } + let index = 0; + let match; + BLOCK_RE.lastIndex = 0; + while ((match = BLOCK_RE.exec(markdown)) !== null) { + const [, directiveText, lang, code] = match; + const directive = parseDirective(directiveText); + if (directive.mode === "fragment") { + console.log(`skip ${pkg} ${lang} example ${index} (${directive.reason})`); + index += 1; + continue; + } + if (lang === "ts") await runTs(pkg, index, code, directive); + else runBash(pkg, index, code); + index += 1; + } + if (index === 0) { + console.error(`FAIL ${pkg}: README has no fenced examples — a landing page shows the thing working`); + failures += 1; + } +} + +for (const dir of scratchRoots) rmSync(dir, { recursive: true, force: true }); + +if (failures > 0) { + console.error(`check-readme-examples: ${failures} failure(s)`); + process.exit(1); +} +console.log("check-readme-examples passed"); diff --git a/scripts/readme-examples-smoke.mjs b/scripts/readme-examples-smoke.mjs deleted file mode 100644 index df2cad0..0000000 --- a/scripts/readme-examples-smoke.mjs +++ /dev/null @@ -1,155 +0,0 @@ -#!/usr/bin/env node -// Executes the ```ts examples in packages/*/README.md against the built -// packages, so the npm landing pages can never drift into lies. -// -// Each snippet runs as-written from inside its package directory: Node -// resolves the package's own name (and its workspace deps) through -// package.json self-reference, exactly like an installed consumer. -// -// Blocks marked fragment:true are illustrative partials (free variables like -// an existing `app` or `localStore`) and are skipped, listed loudly. -import { execFileSync, spawn } from "node:child_process"; -import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; -import { tmpdir } from "node:os"; -import { join } from "node:path"; -import { fileURLToPath } from "node:url"; - -const root = join(fileURLToPath(import.meta.url), "..", ".."); - -const PLAN = [ - { pkg: "core", blocks: [{ i: 0 }, { i: 1 }, { i: 2, fragment: "wraps an undefined localStore; needs a live relay" }] }, - { pkg: "cli", bash: true }, - { pkg: "server", blocks: [{ i: 0, daemon: "relay on" }, { i: 1, fragment: "embeds into an existing app server (free vars: app, checkSession)" }] }, - { pkg: "index", blocks: [{ i: 0 }] }, - { pkg: "client", blocks: [{ i: 0 }] }, -]; - -function tsBlocks(markdown) { - const blocks = []; - const re = /```ts\n([\s\S]*?)```/g; - let m; - while ((m = re.exec(markdown)) !== null) blocks.push(m[1]); - return blocks; -} - -function bashBlocks(markdown) { - const blocks = []; - const re = /```bash\n([\s\S]*?)```/g; - let m; - while ((m = re.exec(markdown)) !== null) blocks.push(m[1]); - return blocks; -} - -let failures = 0; -const scratchRoots = []; - -function runSnippet(pkg, index, code, daemonMatch) { - const pkgDir = join(root, "packages", pkg); - const scriptPath = join(pkgDir, `.readme-smoke-${index}.tmp.mjs`); - const scratch = mkdtempSync(join(tmpdir(), "lync-readme-smoke-")); - scratchRoots.push(scratch); - // Run with cwd in a scratch dir so relative paths ("./rooms") never touch - // the repo; module resolution follows the script's location, not cwd. - writeFileSync(scriptPath, code); - try { - if (!daemonMatch) { - execFileSync(process.execPath, [scriptPath], { cwd: scratch, stdio: ["ignore", "pipe", "pipe"], timeout: 30_000 }); - console.log(`ok ${pkg} README block ${index}`); - return Promise.resolve(); - } - // Long-running example: pass = expected line appears, then we kill it. - return new Promise((resolve) => { - const child = spawn(process.execPath, [scriptPath], { cwd: scratch }); - let out = ""; - const timer = setTimeout(() => { - child.kill("SIGKILL"); - console.error(`FAIL ${pkg} README block ${index}: never printed ${JSON.stringify(daemonMatch)}\n${out}`); - failures += 1; - resolve(); - }, 15_000); - child.stdout.on("data", (chunk) => { - out += String(chunk); - if (out.includes(daemonMatch)) { - clearTimeout(timer); - child.kill("SIGTERM"); - console.log(`ok ${pkg} README block ${index} (daemon: saw ${JSON.stringify(daemonMatch)})`); - resolve(); - } - }); - child.on("exit", () => { - rmSync(scriptPath, { force: true }); - }); - }).finally(() => rmSync(scriptPath, { force: true })); - } catch (error) { - console.error(`FAIL ${pkg} README block ${index}:\n${error.stderr ?? error.message}`); - failures += 1; - return Promise.resolve(); - } finally { - if (!daemonMatch) rmSync(scriptPath, { force: true }); - } -} - -function runCliBlock(pkg, index, code) { - const scratch = mkdtempSync(join(tmpdir(), "lync-readme-smoke-cli-")); - scratchRoots.push(scratch); - const bin = join(root, "packages", "cli", "bin", "lync.js"); - const lines = code - .split("\n") - .map((line) => line.trim()) - .filter((line) => line && !line.startsWith("#") && !line.startsWith("npm install")); - for (const line of lines) { - // serve/sync need a live relay pair; covered by the sync test suite. - if (line.startsWith("lync serve") || line.startsWith("lync sync")) { - console.log(`skip ${pkg} README block ${index} line (needs live relay): ${line}`); - continue; - } - if (line.startsWith("lync merge")) { - // The example merges with an `other.lync` the reader is presumed to - // have; give it one so the command runs as written. - execFileSync("node", [bin, "init", "other.lync"], { cwd: scratch }); - } - // Only rewrite `lync` as a command token (line start or after a pipe), - // never the substring inside filenames like story.lync. - const cmd = line.replace(/(^|\| )lync /g, `$1node ${bin} `); - try { - execFileSync("bash", ["-c", cmd], { cwd: scratch, stdio: ["ignore", "pipe", "pipe"], timeout: 30_000 }); - } catch (error) { - console.error(`FAIL ${pkg} README block ${index} line: ${line}\n${error.stderr}`); - failures += 1; - return; - } - } - console.log(`ok ${pkg} README block ${index} (cli story)`); -} - -for (const entry of PLAN) { - const markdown = readFileSync(join(root, "packages", entry.pkg, "README.md"), "utf8"); - if (entry.bash) { - bashBlocks(markdown).forEach((code, i) => runCliBlock(entry.pkg, i, code)); - continue; - } - const blocks = tsBlocks(markdown); - const planned = entry.blocks ?? []; - const extras = blocks.length - planned.length; - if (extras !== 0) { - console.error(`FAIL ${entry.pkg}: README has ${blocks.length} ts blocks but the plan covers ${planned.length} — update scripts/readme-examples-smoke.mjs`); - failures += 1; - } - for (const spec of planned) { - const code = blocks[spec.i]; - if (code === undefined) continue; - if (spec.fragment) { - console.log(`skip ${entry.pkg} README block ${spec.i} (fragment: ${spec.fragment})`); - continue; - } - await runSnippet(entry.pkg, spec.i, code, spec.daemon); - } -} - -for (const scratch of scratchRoots) rmSync(scratch, { recursive: true, force: true }); - -if (failures > 0) { - console.error(`readme-examples-smoke: ${failures} failure(s)`); - process.exit(1); -} -console.log("readme-examples-smoke passed"); From 4bceb0e8025ac183b752449f1917baf0043bffef Mon Sep 17 00:00:00 2001 From: deepfates Date: Sun, 12 Jul 2026 19:55:48 -0700 Subject: [PATCH 21/33] core: linear-time, order-independent parent-cycle detection MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refutation from the dee-07pu adversarial review (splice session-importer port): parseLyncFiles ran an unmemoized recursive findCycle from EVERY accepted id — per-step path copying and linear path scans made deep parent chains (the exact shape codex session importers emit) ~10x slower per doubling. Measured: n=1000 1.8s, n=2000 15.4s, n=4000 152.6s; the largest real corpus file (107,711 chained events) extrapolated to hours-to-days. Recursion also meant stack overflow was one deep loom away. Rewrite: one shared iterative three-color DFS over the whole graph. O(events + parent edges); the 107k chain now parses in 1.6s (n=4000: 75ms). Each distinct cycle is reported once in canonical rotation (smallest id first) instead of one rotation per member in file-line order — cycle output is now order-independent, same disease family as the leaderboard float-order note. 93/93 with the gauntlet shapes as regression tests. --- packages/core/src/events.ts | 70 ++++++++++++++++++++++++------- packages/core/test/events.test.ts | 44 +++++++++++++++++++ 2 files changed, 98 insertions(+), 16 deletions(-) diff --git a/packages/core/src/events.ts b/packages/core/src/events.ts index 5600bba..5cec9ac 100644 --- a/packages/core/src/events.ts +++ b/packages/core/src/events.ts @@ -575,27 +575,65 @@ function graphObstacles(acceptedById: Map, conflictI else if (!acceptedById.has(parent)) obstacles.push({ class: "dangling", missing: parent }); } } - for (const id of acceptedById.keys()) { - if (conflictIds.has(id)) continue; - const cycle = findCycle(id, acceptedById, conflictIds); - if (cycle.length) obstacles.push({ class: "cycle", ids: cycle }); + for (const cycle of findCycles(acceptedById, conflictIds)) { + obstacles.push({ class: "cycle", ids: cycle }); } return normalizeObstacles(obstacles); } -function findCycle(id: string, acceptedById: Map, conflictIds: Set): string[] { - const visit = (current: string, path: string[]): string[] => { - if (path.includes(current)) return path.slice(path.indexOf(current)); - if (conflictIds.has(current)) return []; - const event = acceptedById.get(current)?.event; - if (!event) return []; - for (const parent of event.parents) { - const found = visit(parent, [...path, current]); - if (found.length) return found; +/** + * Every distinct parent-cycle among accepted events, each reported once in a + * canonical rotation (smallest id first). One shared three-color DFS over the + * whole graph — iterative (no recursion depth limit) and O(events + parent + * edges), so a 100k-deep chain costs one walk, not one walk per event. + */ +function findCycles(acceptedById: Map, conflictIds: Set): string[][] { + const GRAY = 1; + const BLACK = 2; + const color = new Map(); + const cycles: string[][] = []; + const seenCycles = new Set(); + + for (const startId of acceptedById.keys()) { + if (conflictIds.has(startId) || color.get(startId) === BLACK) continue; + const stack: { id: string; nextParent: number }[] = [{ id: startId, nextParent: 0 }]; + const path: string[] = [startId]; + const pathIndex = new Map([[startId, 0]]); + color.set(startId, GRAY); + + while (stack.length > 0) { + const frame = stack[stack.length - 1]; + const event = acceptedById.get(frame.id)?.event; + const parents = event ? event.parents : []; + if (frame.nextParent < parents.length) { + const parent = parents[frame.nextParent]; + frame.nextParent += 1; + if (conflictIds.has(parent) || !acceptedById.has(parent)) continue; + const parentColor = color.get(parent); + if (parentColor === GRAY) { + const cycle = path.slice(pathIndex.get(parent)); + const smallest = cycle.indexOf([...cycle].sort()[0]); + const canonical = [...cycle.slice(smallest), ...cycle.slice(0, smallest)]; + const key = canonical.join(""); + if (!seenCycles.has(key)) { + seenCycles.add(key); + cycles.push(canonical); + } + } else if (parentColor !== BLACK) { + color.set(parent, GRAY); + pathIndex.set(parent, path.length); + path.push(parent); + stack.push({ id: parent, nextParent: 0 }); + } + } else { + stack.pop(); + color.set(frame.id, BLACK); + path.pop(); + pathIndex.delete(frame.id); + } } - return []; - }; - return visit(id, []); + } + return cycles; } function normalizeObstacles(obstacles: LyncObstacle[]): LyncObstacle[] { diff --git a/packages/core/test/events.test.ts b/packages/core/test/events.test.ts index 1245ca1..e65f524 100644 --- a/packages/core/test/events.test.ts +++ b/packages/core/test/events.test.ts @@ -505,3 +505,47 @@ describe("lync v0 line parser vectors", () => { ]); }); }); + +describe("parent-cycle detection (dee-07pu gauntlet: linear-time, order-independent)", () => { + it("reports each distinct cycle once, canonical rotation, regardless of file order", () => { + const lines = [ + `${eventBody({ id: "b", parents: ["a"] })}\n`, + `${eventBody({ id: "a", parents: ["b"] })}\n`, + `${eventBody({ id: "c", parents: ["a"] })}\n`, + ]; + const orderings = [lines, [lines[2], lines[0], lines[1]], [lines[1], lines[2], lines[0]]]; + for (const ordering of orderings) { + const result = parseLyncFiles([ + { file: "cycle.lync", bytes: new TextEncoder().encode(ordering.join("")) }, + ]); + expect(result.graphDiagnostics.filter((o) => o.class === "cycle")).toEqual([ + { class: "cycle", ids: ["a", "b"] }, + ]); + } + }); + + it( + "parses a 20k-deep parent chain in linear time (codex session shape)", + () => { + // Before the shared-DFS rewrite this shape was ~10x slower per doubling + // (measured 152s at n=4000); a quadratic regression would blow far past + // this bound, a linear pass stays well under it. + const n = 20_000; + const chain: string[] = []; + for (let i = 0; i < n; i += 1) { + chain.push( + `${eventBody({ id: `e-${String(i).padStart(7, "0")}`, parents: i === 0 ? [] : [`e-${String(i - 1).padStart(7, "0")}`] })}\n`, + ); + } + const started = performance.now(); + const result = parseLyncFiles([ + { file: "chain.lync", bytes: new TextEncoder().encode(chain.join("")) }, + ]); + const elapsed = performance.now() - started; + expect(result.lines.filter((line) => line.class === "accepted")).toHaveLength(n); + expect(result.graphDiagnostics).toEqual([]); + expect(elapsed).toBeLessThan(10_000); + }, + 30_000, + ); +}); From 91834d845a72236b5536e375d0b39106078ab745 Mon Sep 17 00:00:00 2001 From: deepfates Date: Sun, 12 Jul 2026 20:54:06 -0700 Subject: [PATCH 22/33] core: leaderboard sums in sorted-annotation-id order; selection references carry chosen/shown dee-aeq1: lyncLeaderboardView accumulated scoreTotal in file-line order, so scoreMean's float addition order (and its low bits) depended on input file order. The view now walks annotations by sorted id, making means bit-stable across shuffled files; regression test parses the same events in two file orders and asserts Object.is on scoreMean. dee-oh98: LyncSelectionReference gains additive chosen/shown string-array fields populated from the annotation payload (empty arrays when absent), so consumers no longer re-read the annotation event to enumerate alternatives. --- packages/core/src/views.ts | 18 ++++++++- packages/core/test/views.test.ts | 63 ++++++++++++++++++++++++++++++++ 2 files changed, 79 insertions(+), 2 deletions(-) diff --git a/packages/core/src/views.ts b/packages/core/src/views.ts index 648cdd0..e102b1a 100644 --- a/packages/core/src/views.ts +++ b/packages/core/src/views.ts @@ -54,6 +54,8 @@ export interface LyncScoreReference { export interface LyncSelectionReference { annotationId: string; selected: boolean; + chosen: string[]; + shown: string[]; author: LyncEventBody["author"]; at: string; basis?: unknown; @@ -190,7 +192,11 @@ export function lyncLeaderboardView(result: LyncParseResult): LyncLeaderboardVie return entry; }; - for (const viewEvent of index.events.values()) { + // Walk annotations in sorted-id order, not file-line order: float addition + // is order-sensitive, so summing by sorted annotation id keeps scoreTotal + // and scoreMean bit-stable no matter how the input files are shuffled. + for (const id of index.ids) { + const viewEvent = index.events.get(id)!; const event = viewEvent.event; if (event.kind !== "lync/annotation" || viewEvent.payloadSuppressed) continue; const label = event.payload["label"]; @@ -220,7 +226,15 @@ export function lyncLeaderboardView(result: LyncParseResult): LyncLeaderboardVie const entry = ensure(targetId); entry.selectionCount += 1; if (selected) entry.selectedCount += 1; - entry.selections.push({ annotationId: event.id, selected, author: event.author, at: event.at, basis: event.payload["basis"] }); + entry.selections.push({ + annotationId: event.id, + selected, + chosen: [...chosen], + shown: [...shown], + author: event.author, + at: event.at, + basis: event.payload["basis"], + }); } } } diff --git a/packages/core/test/views.test.ts b/packages/core/test/views.test.ts index 5b18635..dbee5ae 100644 --- a/packages/core/test/views.test.ts +++ b/packages/core/test/views.test.ts @@ -156,5 +156,68 @@ describe("lync views", () => { { targetId: "B", rank: 1, scoreMean: 0.91, selectedCount: 1, selectionCount: 1 }, { targetId: "C", rank: 2, scoreMean: 0.2, selectedCount: 0, selectionCount: 1 }, ]); + expect(leaderboard.entries.find((entry) => entry.targetId === "B")?.selections).toEqual([ + { annotationId: "E", selected: true, chosen: ["B"], shown: ["B", "C"], author: { actor: "deepfates" }, at: "2026-07-06T04:10:00Z", basis: "human pick" }, + ]); + }); + + it("computes bit-identical scoreMean regardless of file order", () => { + const files = [ + { file: "a.lync", bytes: [event({ id: "A", payload: { text: "root" } }), event({ + id: "S1", + kind: "lync/annotation", + author: { actor: "witness-panel-v3" }, + parents: ["A"], + payload: { label: "score", value: 0.1 }, + })].join("\n") + "\n" }, + { file: "b.lync", bytes: event({ + id: "S2", + kind: "lync/annotation", + author: { actor: "witness-panel-v3" }, + parents: ["A"], + payload: { label: "score", value: 0.2 }, + }) + "\n" }, + { file: "c.lync", bytes: event({ + id: "S3", + kind: "lync/annotation", + author: { actor: "witness-panel-v3" }, + parents: ["A"], + payload: { label: "score", value: 0.3 }, + }) + "\n" }, + ]; + const forward = lyncLeaderboardView(parseLyncFiles(files)); + const reversed = lyncLeaderboardView(parseLyncFiles([...files].reverse())); + + const meanForward = forward.entries.find((entry) => entry.targetId === "A")?.scoreMean; + const meanReversed = reversed.entries.find((entry) => entry.targetId === "A")?.scoreMean; + expect(meanForward).not.toBeNull(); + expect(meanForward).toBeDefined(); + expect(Object.is(meanForward, meanReversed)).toBe(true); + }); + + it("carries empty chosen/shown arrays when the selection payload omits shown", () => { + const input = [ + event({ id: "A", payload: { text: "root" } }), + event({ id: "B", parents: ["A"], payload: { text: "left" } }), + event({ id: "C", parents: ["A"], payload: { text: "right" } }), + event({ + id: "G", + kind: "lync/annotation", + author: { actor: "deepfates" }, + parents: ["B", "C"], + payload: { label: "selection", chosen: ["B"] }, + }), + ].join("\n") + "\n"; + const result = parseLyncFiles([{ file: "worked.lync", bytes: input }]); + const leaderboard = lyncLeaderboardView(result); + + const selectionsFor = (targetId: string) => + leaderboard.entries.find((entry) => entry.targetId === targetId)?.selections; + expect(selectionsFor("B")).toEqual([ + { annotationId: "G", selected: true, chosen: ["B"], shown: [], author: { actor: "deepfates" }, at: "2026-07-06T04:10:00Z", basis: undefined }, + ]); + expect(selectionsFor("C")).toEqual([ + { annotationId: "G", selected: false, chosen: ["B"], shown: [], author: { actor: "deepfates" }, at: "2026-07-06T04:10:00Z", basis: undefined }, + ]); }); }); From ded690d04b36473ee51f688df7e4c1a15e678de6 Mon Sep 17 00:00:00 2001 From: deepfates Date: Sun, 12 Jul 2026 23:53:53 -0700 Subject: [PATCH 23/33] =?UTF-8?q?consolidate:=20lync=20ships=20as=20ONE=20?= =?UTF-8?q?package,=20lync-core=200.3.0,=20zero=20dependencies=20=E2=80=94?= =?UTF-8?q?=20truthfully?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Design ruling dee-6prz (2026-07-12, 4-lens adversarial panel): - Fold lync-index -> lync-core/indexes{,/entries,/memory,/types}, lync-client -> lync-core/client{,/testing,/types}, lync-server -> lync-core/relay, lync-cli -> the lync bin. Every prior core subpath kept; subpath parity is a consumer contract. - NO ws declaration at all (an optional peer ws@^8 hard-ERESOLVEs consumers carrying ws@7; no declaration = that failure class cannot exist). The relay acquires ws lazily via createRequire inside the still-synchronous factory; a missing ws throws one helpful error. Bundlers must mark ws external. - Kill the type leak: ws ships no .d.ts, so the relay's public types are structural (LyncRelaySocket) — tsc for a consumer without ws passes. - cli sync rewritten onto Node's built-in WebSocket (engines >=22): EventTarget listeners, MessageEvent.data (arraybuffer, never Blob), close()-based hard-abort instead of terminate(). Sync tests unmodified in what they assert. - One README telling one story: format first, conformance vectors as product, then library / command / relay. FORMAT.md gains the layering paragraph — sync protocols stay deliberately absent from the format. - Version 0.3.0 deliberately: same-version tarball swaps defeat npm caching. --- FORMAT.md | 9 + README.md | 413 ++++++----- bin/lync.js | 2 + package.json | 179 ++++- packages/cli/LICENSE | 21 - packages/cli/README.md | 45 -- packages/cli/bin/lync.js | 2 - packages/cli/package.json | 61 -- packages/cli/tsconfig.json | 8 - packages/client/LICENSE | 21 - packages/client/README.md | 33 - packages/client/package.json | 63 -- packages/client/tsconfig.json | 8 - packages/core/LICENSE | 21 - packages/core/README.md | 86 --- packages/core/package.json | 125 ---- packages/core/tsconfig.json | 8 - packages/index/LICENSE | 21 - packages/index/README.md | 30 - packages/index/package.json | 66 -- packages/index/tsconfig.json | 8 - packages/server/LICENSE | 21 - packages/server/README.md | 55 -- packages/server/package.json | 56 -- packages/server/tsconfig.json | 8 - pnpm-lock.yaml | 649 ++++++++---------- pnpm-workspace.yaml | 2 - scripts/check-readme-examples.mjs | 97 ++- scripts/fresh-clone-smoke.sh | 14 +- {packages/cli/src => src/cli}/bin.ts | 0 {packages/cli/src => src/cli}/index.ts | 8 +- {packages/cli/src => src/cli}/sync.ts | 32 +- {packages/client/src => src/client}/create.ts | 4 +- {packages/client/src => src/client}/index.ts | 0 .../client/src => src/client}/testing.ts | 6 +- {packages/client/src => src/client}/types.ts | 4 +- {packages/core/src => src}/errors.ts | 0 {packages/core/src => src}/events.ts | 0 {packages/core/src => src}/file-log.ts | 0 {packages/core/src => src}/idb-log.ts | 0 {packages/core/src => src}/index.ts | 0 .../index/src => src/indexes}/entries.ts | 2 +- {packages/index/src => src/indexes}/index.ts | 0 {packages/index/src => src/indexes}/memory.ts | 4 +- {packages/index/src => src/indexes}/types.ts | 2 +- {packages/core/src => src}/json.ts | 0 {packages/core/src => src}/looms.ts | 0 {packages/core/src => src}/memory-log.ts | 0 {packages/core/src => src}/memory.ts | 0 .../core/src => src}/profiles/text-story.ts | 0 {packages/core/src => src}/references.ts | 0 {packages/server/src => src/relay}/attach.ts | 5 +- {packages/server/src => src/relay}/index.ts | 2 +- {packages/server/src => src/relay}/relay.ts | 65 +- {packages/server/src => src/relay}/serve.ts | 0 {packages/core/src => src}/sha256.ts | 0 {packages/core/src => src}/store.ts | 0 {packages/core/src => src}/sync-protocol.ts | 0 {packages/core/src => src}/synced-store.ts | 0 {packages/core/src => src}/types.ts | 0 {packages/core/src => src}/uuid.ts | 0 {packages/core/src => src}/views.ts | 0 {packages/cli/test => test/cli}/cli.test.ts | 5 +- {packages/cli/test => test/cli}/sync.test.ts | 4 +- .../cli}/synced-store.integration.test.ts | 2 +- .../test => test/client}/create.test.ts | 6 +- {packages/core/test => test}/events.test.ts | 0 .../test => test/indexes}/memory.test.ts | 4 +- {packages/core/test => test}/memory.test.ts | 0 .../core/test => test}/references.test.ts | 0 .../server/test => test/relay}/attach.test.ts | 2 +- .../server/test => test/relay}/relay.test.ts | 2 +- {packages/core/test => test}/sha256.test.ts | 0 {packages/core/test => test}/storage.test.ts | 0 .../core/test => test}/sync-protocol.test.ts | 0 .../core/test => test}/synced-store.test.ts | 0 .../test => test}/text-story-profile.test.ts | 0 .../vectors/v0/01-valid-events/expected.json | 0 .../vectors/v0/01-valid-events/input.lync | 0 .../v0/02-splice-anchoring/expected.json | 0 .../vectors/v0/02-splice-anchoring/input.lync | 0 .../v0/03-damaged-digest/expected.json | 0 .../vectors/v0/03-damaged-digest/input.lync | 0 .../v0/04-garbage-classes/expected.json | 0 .../vectors/v0/04-garbage-classes/input.lync | 0 .../05-conflicts-and-duplicates/expected.json | 0 .../v0/05-conflicts-and-duplicates/input.lync | 0 .../v0/06-graph-obstacles/expected.json | 0 .../vectors/v0/06-graph-obstacles/input.lync | 0 .../v0/07-critical-suppression/expected.json | 0 .../v0/07-critical-suppression/input.lync | 0 .../v0/08-spelling-vs-value/expected.json | 0 .../v0/08-spelling-vs-value/input.lync | 0 .../v0/09-marked-at-semantics/expected.json | 0 .../v0/09-marked-at-semantics/input.lync | 0 .../vectors/v0/10-merge-union/a.lync | 0 .../vectors/v0/10-merge-union/b.lync | 0 .../vectors/v0/10-merge-union/expected.json | 0 .../v0/11-nonconforming-carried/expected.json | 0 .../v0/11-nonconforming-carried/input.lync | 0 .../v0/12-invalid-sig-splice/expected.json | 0 .../v0/12-invalid-sig-splice/input.lync | 0 .../v0/13-sig-without-digest/expected.json | 0 .../v0/13-sig-without-digest/input.lync | 0 .../vectors/v0/OPEN-QUESTIONS.md | 0 .../core/test => test}/vectors/v0/README.md | 0 .../core/test => test}/vectors/v0/generate.py | 0 {packages/core/test => test}/views.test.ts | 0 tsconfig.base.json => tsconfig.json | 7 +- vitest.config.ts | 32 +- 110 files changed, 901 insertions(+), 1429 deletions(-) create mode 100755 bin/lync.js delete mode 100644 packages/cli/LICENSE delete mode 100644 packages/cli/README.md delete mode 100755 packages/cli/bin/lync.js delete mode 100644 packages/cli/package.json delete mode 100644 packages/cli/tsconfig.json delete mode 100644 packages/client/LICENSE delete mode 100644 packages/client/README.md delete mode 100644 packages/client/package.json delete mode 100644 packages/client/tsconfig.json delete mode 100644 packages/core/LICENSE delete mode 100644 packages/core/README.md delete mode 100644 packages/core/package.json delete mode 100644 packages/core/tsconfig.json delete mode 100644 packages/index/LICENSE delete mode 100644 packages/index/README.md delete mode 100644 packages/index/package.json delete mode 100644 packages/index/tsconfig.json delete mode 100644 packages/server/LICENSE delete mode 100644 packages/server/README.md delete mode 100644 packages/server/package.json delete mode 100644 packages/server/tsconfig.json delete mode 100644 pnpm-workspace.yaml rename {packages/cli/src => src/cli}/bin.ts (100%) rename {packages/cli/src => src/cli}/index.ts (98%) rename {packages/cli/src => src/cli}/sync.ts (85%) rename {packages/client/src => src/client}/create.ts (96%) rename {packages/client/src => src/client}/index.ts (100%) rename {packages/client/src => src/client}/testing.ts (85%) rename {packages/client/src => src/client}/types.ts (95%) rename {packages/core/src => src}/errors.ts (100%) rename {packages/core/src => src}/events.ts (100%) rename {packages/core/src => src}/file-log.ts (100%) rename {packages/core/src => src}/idb-log.ts (100%) rename {packages/core/src => src}/index.ts (100%) rename {packages/index/src => src/indexes}/entries.ts (90%) rename {packages/index/src => src/indexes}/index.ts (100%) rename {packages/index/src => src/indexes}/memory.ts (98%) rename {packages/index/src => src/indexes}/types.ts (97%) rename {packages/core/src => src}/json.ts (100%) rename {packages/core/src => src}/looms.ts (100%) rename {packages/core/src => src}/memory-log.ts (100%) rename {packages/core/src => src}/memory.ts (100%) rename {packages/core/src => src}/profiles/text-story.ts (100%) rename {packages/core/src => src}/references.ts (100%) rename {packages/server/src => src/relay}/attach.ts (94%) rename {packages/server/src => src/relay}/index.ts (64%) rename {packages/server/src => src/relay}/relay.ts (83%) rename {packages/server/src => src/relay}/serve.ts (100%) rename {packages/core/src => src}/sha256.ts (100%) rename {packages/core/src => src}/store.ts (100%) rename {packages/core/src => src}/sync-protocol.ts (100%) rename {packages/core/src => src}/synced-store.ts (100%) rename {packages/core/src => src}/types.ts (100%) rename {packages/core/src => src}/uuid.ts (100%) rename {packages/core/src => src}/views.ts (100%) rename {packages/cli/test => test/cli}/cli.test.ts (98%) rename {packages/cli/test => test/cli}/sync.test.ts (99%) rename {packages/cli/test => test/cli}/synced-store.integration.test.ts (98%) rename {packages/client/test => test/client}/create.test.ts (89%) rename {packages/core/test => test}/events.test.ts (100%) rename {packages/index/test => test/indexes}/memory.test.ts (96%) rename {packages/core/test => test}/memory.test.ts (100%) rename {packages/core/test => test}/references.test.ts (100%) rename {packages/server/test => test/relay}/attach.test.ts (97%) rename {packages/server/test => test/relay}/relay.test.ts (98%) rename {packages/core/test => test}/sha256.test.ts (100%) rename {packages/core/test => test}/storage.test.ts (100%) rename {packages/core/test => test}/sync-protocol.test.ts (100%) rename {packages/core/test => test}/synced-store.test.ts (100%) rename {packages/core/test => test}/text-story-profile.test.ts (100%) rename {packages/core/test => test}/vectors/v0/01-valid-events/expected.json (100%) rename {packages/core/test => test}/vectors/v0/01-valid-events/input.lync (100%) rename {packages/core/test => test}/vectors/v0/02-splice-anchoring/expected.json (100%) rename {packages/core/test => test}/vectors/v0/02-splice-anchoring/input.lync (100%) rename {packages/core/test => test}/vectors/v0/03-damaged-digest/expected.json (100%) rename {packages/core/test => test}/vectors/v0/03-damaged-digest/input.lync (100%) rename {packages/core/test => test}/vectors/v0/04-garbage-classes/expected.json (100%) rename {packages/core/test => test}/vectors/v0/04-garbage-classes/input.lync (100%) rename {packages/core/test => test}/vectors/v0/05-conflicts-and-duplicates/expected.json (100%) rename {packages/core/test => test}/vectors/v0/05-conflicts-and-duplicates/input.lync (100%) rename {packages/core/test => test}/vectors/v0/06-graph-obstacles/expected.json (100%) rename {packages/core/test => test}/vectors/v0/06-graph-obstacles/input.lync (100%) rename {packages/core/test => test}/vectors/v0/07-critical-suppression/expected.json (100%) rename {packages/core/test => test}/vectors/v0/07-critical-suppression/input.lync (100%) rename {packages/core/test => test}/vectors/v0/08-spelling-vs-value/expected.json (100%) rename {packages/core/test => test}/vectors/v0/08-spelling-vs-value/input.lync (100%) rename {packages/core/test => test}/vectors/v0/09-marked-at-semantics/expected.json (100%) rename {packages/core/test => test}/vectors/v0/09-marked-at-semantics/input.lync (100%) rename {packages/core/test => test}/vectors/v0/10-merge-union/a.lync (100%) rename {packages/core/test => test}/vectors/v0/10-merge-union/b.lync (100%) rename {packages/core/test => test}/vectors/v0/10-merge-union/expected.json (100%) rename {packages/core/test => test}/vectors/v0/11-nonconforming-carried/expected.json (100%) rename {packages/core/test => test}/vectors/v0/11-nonconforming-carried/input.lync (100%) rename {packages/core/test => test}/vectors/v0/12-invalid-sig-splice/expected.json (100%) rename {packages/core/test => test}/vectors/v0/12-invalid-sig-splice/input.lync (100%) rename {packages/core/test => test}/vectors/v0/13-sig-without-digest/expected.json (100%) rename {packages/core/test => test}/vectors/v0/13-sig-without-digest/input.lync (100%) rename {packages/core/test => test}/vectors/v0/OPEN-QUESTIONS.md (100%) rename {packages/core/test => test}/vectors/v0/README.md (100%) rename {packages/core/test => test}/vectors/v0/generate.py (100%) rename {packages/core/test => test}/views.test.ts (100%) rename tsconfig.base.json => tsconfig.json (76%) diff --git a/FORMAT.md b/FORMAT.md index 9a00a9a..14d5c74 100644 --- a/FORMAT.md +++ b/FORMAT.md @@ -370,6 +370,15 @@ Ordering for a live multiplayer world is its own pact. If a world needs a global sequence, it carries `seq` in payload. Who may fork it is governance, not data. +Absent from the format does not mean absent from the toolbox — it means +layered above it. The reference implementation ships a sync relay and a sync +client beside this spec; they are tools that happen to move `.lync` lines, +not part of the format. Because events are immutable and merge is set union +by id, any transport that delivers canonical line bytes converges the same +files: a WebSocket relay, `rsync`, an email attachment, or a USB stick are +all conformant sync mechanisms. An implementation of this document is +complete without implementing any of them. + ## Worked Example Five events: a paragraph, two alternatives, a judge's score, and a declared diff --git a/README.md b/README.md index 767ab4c..2f1e5f2 100644 --- a/README.md +++ b/README.md @@ -1,52 +1,20 @@ # lync -lync is the TypeScript reference implementation of the lync format: `.lync` -append-only JSONL files of interaction history. Each line is -one immutable event with an envelope, parent links, provenance, and a payload -owned by the event kind. Merge is set union by event id. Branch trees, -transcripts, memory views, and leaderboards are computed views over the same -event set. - -The format layer is the durable center. A loom/turn API ships on top of the -same event stores for programs that want turns and threads instead of raw -events. - -## Ninety-Second Story - -The shipped `lync` CLI has seven verbs: `init`, `append`, `verify`, `merge`, -`view`, `serve`, and `sync`. This first mile uses the file verbs; see -[Sync](#sync) for `serve` and `sync`. From a fresh clone, run these from the repo root after -`pnpm install && pnpm build`; `pnpm exec lync` resolves the workspace binary. A -published or globally installed package drops the `pnpm exec` prefix and you -call `lync` directly. A complete first mile looks like this: +lync is a file format for append-only interaction history, and `lync-core` is +its reference implementation — one package that ships the parser, event +stores, computed views, the loom API, live sync, the `lync` command, and the +sync relay. Zero runtime dependencies. -```bash -pnpm exec lync init story.lync -printf '%s\n' '{"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"deepfates","via":"example@0.1"},"parents":[],"payload":{"text":"Once..."}}' | pnpm exec lync append story.lync -pnpm exec lync verify story.lync -pnpm exec lync view story.lync --as transcript -printf '%s\n' '{"id":"note-2","kind":"notes/text","at":"2026-07-06T04:13:00Z","author":{"actor":"deepfates","via":"example@0.1"},"parents":["root"],"payload":{"text":"Then..."}}' | pnpm exec lync append imported.lync -pnpm exec lync merge story.lync imported.lync -o merged.lync -pnpm exec lync view merged.lync --as tree -``` - -Under those verbs, every line has the same envelope: - -```json -{"v":1,"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"deepfates","via":"example@0.1"},"parents":[],"payload":{"text":"Once..."}} -``` - -That line can be copied to another file, merged back later, verified byte for -byte, and read by software that has never heard of `notes/text`. Unknown -kinds are carried and traversed; meaning belongs to pacts layered above the -format — see [pacts/import.md](./pacts/import.md) (imports are transcription: -deterministic ids, provenance preserved, zero silent drops) and -[pacts/export.md](./pacts/export.md) (exports are projections of the event -log, including training data). +A `.lync` file is UTF-8 JSONL: each line is one immutable event with an +envelope, parent links, provenance, and a payload owned by the event kind. +Merge is set union by event id. Branch trees, transcripts, memory views, and +leaderboards are computed views over the same event set — never stored as +truth themselves. ## The Format -See [FORMAT.md](./FORMAT.md) for the normative lync format specification. +The normative specification is [FORMAT.md](./FORMAT.md). It is self-contained: +no code in this repository is required to implement it. The short version: @@ -64,168 +32,155 @@ The short version: - Stored line metadata may splice `digest` and `sig` at the end of the line. `digest` and `sig` are reserved top-level body names; payloads may use those names freely. -- Views are computed. The package currently ships branch tree, transcript, - memory, and leaderboard helpers. +- Views are computed. This package ships branch tree, transcript, memory, and + leaderboard helpers. -## Packages +Every line has the same envelope: -- `lync-core`: format parsing, event stores, computed views, references, the - loom API, and live sync (`createSyncedStore`). No runtime dependencies. -- `lync-cli`: the `lync` command — `init`, `append`, `verify`, `merge`, `view`, - `serve`, `sync`. -- `lync-server`: the line-sync relay — `createLyncRelay` to mount on your own - Node server, `attachLyncServer` for one path on an existing server, - `startLyncServe` standalone. Depends on `lync-core` and `ws`. -- `lync-index`: an index of many looms, with reactive subscription. Depends - only on `lync-core`. -- `lync-client`: the loom client — resolves references and opens looms and - indexes. Depends on `lync-core` and `lync-index`. +```json +{"v":1,"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"deepfates","via":"example@0.1"},"parents":[],"payload":{"text":"Once..."}} +``` -## Format-Layer Imports +That line can be copied to another file, merged back later, verified byte for +byte, and read by software that has never heard of `notes/text`. Unknown +kinds are carried and traversed; meaning belongs to pacts layered above the +format — see [pacts/import.md](./pacts/import.md) (imports are transcription: +deterministic ids, provenance preserved, zero silent drops) and +[pacts/export.md](./pacts/export.md) (exports are projections of the event +log, including training data). -```ts -import { LyncUnion, exportCarriedLyncBytes, parseLyncFiles } from "lync-core/events"; -import { createFileEventStore, createFileLyncLooms } from "lync-core/file-log"; -import { createIndexedDbEventStore } from "lync-core/idb-log"; -import { createLyncLooms, createBrowserLyncLooms } from "lync-core/looms"; -import { createMemoryEventStore } from "lync-core/memory-log"; -import { BaseEventStore, serializeLyncEvent } from "lync-core/store"; -import { - lyncBranchTreeView, - lyncLeaderboardView, - lyncMemoryView, - lyncTranscriptView, -} from "lync-core/views"; -``` +## Conformance Vectors + +The format is meant to be implemented in other languages, and the test +vectors are the product that makes a port checkable: +[test/vectors/v0](./test/vectors/v0). Each case is a directory holding a raw +input file (`input.lync`, or `a.lync` + `b.lync` for the merge case) and an +`expected.json` with the required classification of every physical line — +accepted, nonconforming, garbage, damaged, conflict-variant — plus union ids, +pending parents, and graph diagnostics. The thirteen cases cover valid events, +splice anchoring, damaged digests, garbage classes, same-id conflicts and +duplicates, graph obstacles, critical suppression, spelling-versus-value +equality, `marked`/`at` semantics, merge union, carried nonconforming lines, +and invalid signature splices. + +A new implementation ports the vector suite first, this package's test suite +second, and its design never. `test/vectors/v0/README.md` documents the +expected-output schema; `generate.py` regenerates digests deterministically. -The seven format-layer package exports are: +## The Library + +```bash +npm install lync-core +``` -- `lync-core/events`: line parsing, carried-byte export, downsets, and - incremental union. -- `lync-core/memory-log`: in-memory event store for tests and embedded - runtimes. -- `lync-core/file-log`: file-backed event store and `createFileLyncLooms` - (node-only; keeps `node:fs`/`node:path` off the browser path). -- `lync-core/idb-log`: IndexedDB-backed event store. -- `lync-core/store`: base event-store contract and serialization helpers. -- `lync-core/views`: branch tree, transcript, memory, and leaderboard - view helpers. -- `lync-core/looms`: compatibility loom API backed by event stores. +Runs in Node (>=22) and the browser. No dependencies. -## Parse, Union, View +### Parse, union, view ```ts import { parseLyncFiles } from "lync-core/events"; -import { lyncBranchTreeView, lyncMemoryView } from "lync-core/views"; +import { lyncBranchTreeView, lyncTranscriptView } from "lync-core/views"; const bytes = new TextEncoder().encode( - '{"v":1,"id":"a","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"deepfates"},"parents":[],"payload":{"text":"Once..."}}\n', + '{"v":1,"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"you"},"parents":[],"payload":{"text":"Once..."}}\n', ); const parsed = parseLyncFiles([{ file: "story.lync", bytes }]); -const tree = lyncBranchTreeView(parsed); -const memory = lyncMemoryView(parsed); - -console.log(parsed.lines[0].class, tree.roots, memory.frontierIds); +console.log(parsed.lines[0].class); // "accepted" +console.log(lyncBranchTreeView(parsed).roots); +console.log(lyncTranscriptView(parsed, "root").entries.map((entry) => entry.id)); ``` -`parseLyncFiles` classifies every physical line and keeps the original bytes, -including garbage, damaged lines, nonconforming-but-carried lines, and conflict -variants. `exportCarriedLyncBytes(parsed)` re-emits the carried bytes. +`parseLyncFiles` classifies every physical line and keeps the original bytes — +accepted events, nonconforming-but-carried lines, damaged lines, garbage, and +conflict variants (same id, different bytes) are all preserved and reported, +never silently dropped. `exportCarriedLyncBytes(parsed)` re-emits the carried +bytes. `LyncUnion` performs the same union incrementally and can buffer +children until their first missing parent arrives. -`LyncUnion` performs the same union incrementally and can buffer children until -their first missing parent arrives. +### Stores and looms -## Storage +Event stores share one contract over memory, file, and IndexedDB backends. The +loom API gives programs turns and threads instead of raw events, on top of any +store: ```ts +import { createLyncLooms } from "lync-core/looms"; import { createMemoryEventStore } from "lync-core/memory-log"; -const store = createMemoryEventStore(); -await store.append({ - v: 1, - id: "root", - kind: "lync/loom", - at: "2026-07-06T04:12:31Z", - author: { actor: "deepfates", via: "example@0.1" }, - parents: [], - payload: { meta: { title: "Story" } }, -}); - -await store.append({ - v: 1, - id: "turn-1", - kind: "lync/turn", - at: "2026-07-06T04:12:32Z", - author: { actor: "deepfates", via: "example@0.1" }, - parents: ["root"], - payload: { payload: { text: "Once..." }, ordinal: 0 }, +const looms = createLyncLooms({ + store: createMemoryEventStore(), + author: { actor: "you", via: "my-app@0.1" }, }); -console.log((await store.byRoot("root")).map((event) => event.body.id)); +const info = await looms.create({ title: "Story" }); +const loom = await looms.open(info.id); +const first = await loom.appendTurn(null, { text: "Once..." }); +await loom.appendTurn(first.id, { text: "Then..." }); ``` The store API accepts raw lines through `union(line)` and structured event bodies through `append(event)`. It reports conflicts, pending parents, garbage, and accepted events without making file order meaningful. -## Looms +### Indexes -The loom API gives programs turns and threads instead of raw events, backed by -any event store: +An index tracks a collection of looms — upsert entries, subscribe to changes: ```ts -import { createLyncLooms } from "lync-core/looms"; -import { createMemoryEventStore } from "lync-core/memory-log"; +import { loomRef } from "lync-core"; +import { createMemoryLoomIndexes } from "lync-core/indexes/memory"; -const looms = createLyncLooms<{ text: string }, { title: string }>({ - store: createMemoryEventStore(), - author: { actor: "deepfates", via: "example@0.1" }, - createId: (() => { - let n = 0; - return () => `id-${++n}`; - })(), -}); +const indexes = createMemoryLoomIndexes(); +const index = await indexes.create({ title: "My looms" }); -const info = await looms.create({ title: "Story" }); -const loom = await looms.open(info.id); -const first = await loom.appendTurn(null, { text: "Once..." }); -const next = await loom.appendTurn(first.id, { text: "Then..." }); +index.subscribe((event) => console.log("index changed:", event.type)); +await index.addLoom(loomRef("loom-1"), { title: "Story" }); -console.log((await loom.threadTo(next.id)).map((turn) => turn.payload.text)); +console.log((await index.entries()).map((entry) => entry.title)); ``` -## Sync +Entries carry a loom reference, optional title/kind/meta, and timestamps. +`export()`/`import()` round-trip a whole index as a snapshot. -Any lync file can converge with any other copy through a relay: +### The loom client -```bash -lync serve ./rooms --port 8787 # the relay: one append-only file per root -lync sync story.lync ws://host:8787 # one-shot: push what it lacks, pull what you lack -lync sync story.lync ws://host:8787 --follow # stay live: stream both ways until Ctrl-C +One object that pairs looms with an index and resolves +loom/turn/thread/index references to and from URLs: + +```ts +import { createLyncLooms } from "lync-core/looms"; +import { createMemoryEventStore } from "lync-core/memory-log"; +import { createMemoryLoomIndexes } from "lync-core/indexes/memory"; +import { createLoomClient } from "lync-core/client"; + +const client = createLoomClient({ + looms: createLyncLooms({ store: createMemoryEventStore(), author: { actor: "you" } }), + indexes: createMemoryLoomIndexes(), +}); + +const info = await client.looms.create({ title: "Story" }); +const ref = client.references.loom(info.id); + +// Round-trip a reference through a shareable URL (?ref=...). +// In the browser, pass `window.location` instead of a URL object. +const url = client.references.toUrl(ref, new URL("https://example.com/story")); +const opened = await client.openReference(client.references.fromUrl(new URL(url))); +console.log(opened.kind); // "loom" — opened.loom is ready to appendTurn ``` -The relay is deliberately dumb. Events are immutable and merge is union by -id, so the protocol has no merge logic: five JSON frames (`sub`, `ev`, -`live`, `presence`, `err`) that move canonical line bytes. The server never -parses a line beyond extracting its id, stores each root as a plain `.lync` -file you can read with any lync tool, and echoes accepted events to every -subscriber — echoes are duplicate no-ops under union. `seq` is a per-root -arrival counter used as a resume cursor (`.sync.json`), so an offline -client reconnects exactly where it left off. Same-id-different-body is never -resolved: both variants are kept (the relay writes a `.conflicts` sidecar) -and both sides are told loudly. Presence frames are relayed, never stored. -A truncated final line after a crash is sealed and surfaced as damaged, -never eaten. `--token T` on the server requires `Authorization: Bearer T` -to connect. +`lync-core/client/testing` ships `createTestLoomClient`, a fully in-memory +client for tests and embedded experiments — deterministic when you pass +`createId` and `now`. -### Sync inside an app +### Live sync inside an app -The same protocol runs in a browser or Node app with no CLI. Wrap any event +The sync protocol runs in a browser or Node app with no CLI. Wrap any event store in `createSyncedStore`; looms and indexes built over it update live as collaborators append, because they already recompute through the store's `subscribe`: + ```ts import { createMemoryEventStore } from "lync-core/memory-log"; import { createLyncLooms } from "lync-core/looms"; @@ -245,25 +200,147 @@ await loom.appendTurn(parentId, { text: "typed live" }); Local appends are pushed to the relay; remote lines are ingested through the same `union` path and surface reactively. Offline appends queue and flush on reconnect; the store re-subscribes automatically. The transport is an -interface — pass your own for tests or a non-WebSocket carrier. +interface — pass your own for tests or a non-WebSocket carrier. The client +side uses the platform's built-in WebSocket: no dependency, in the browser or +in Node. + +### Subpath exports + +- `lync-core/events` — line parsing, carried-byte export, incremental union +- `lync-core/store` — the event-store contract and serialization +- `lync-core/memory-log`, `lync-core/file-log`, `lync-core/idb-log` — stores + (`file-log` is node-only; it keeps `node:fs` off the browser path) +- `lync-core/views` — branch tree, transcript, memory, leaderboard +- `lync-core/looms` — the loom/turn API +- `lync-core/references` — loom/turn/thread/index references and URLs +- `lync-core/synced-store` — live sync decorator and WebSocket transport +- `lync-core/sync-protocol` — the five sync frames, encode/decode +- `lync-core/uuid` — zero-dep UUIDv7 for event ids +- `lync-core/indexes`, `lync-core/indexes/entries`, + `lync-core/indexes/memory`, `lync-core/indexes/types` — loom indexes +- `lync-core/client`, `lync-core/client/testing`, `lync-core/client/types` — + the loom client +- `lync-core/relay` — the sync relay (see [The Relay](#the-relay)) + +## The Command + +The package installs a `lync` bin with seven verbs: `init`, `append`, +`verify`, `merge`, `view`, `serve`, and `sync`. + +```bash +npm install -g lync-core +``` + +```bash +lync init story.lync +printf '%s\n' '{"kind":"notes/text","author":{"actor":"you"},"payload":{"text":"Once..."}}' | lync append story.lync +lync verify story.lync +lync view story.lync --as transcript +printf '%s\n' '{"kind":"notes/text","author":{"actor":"friend"},"payload":{"text":"Then..."}}' | lync append other.lync +lync merge story.lync other.lync -o merged.lync +lync view merged.lync --as tree +``` + +`append` fills the envelope for you: a UUIDv7 id, the current timestamp, `v`, +and `parents` default in; anything you supply is kept. `verify` reports what +every physical line is — accepted, nonconforming, damaged, garbage, or +conflict variant — and never drops bytes; it exits 0 only when every line is +accepted. `view` renders `transcript` or `tree`. + +Any lync file can converge with any other copy through a relay: + + +```bash +lync serve ./rooms --port 8787 # the relay: one append-only file per root +lync sync story.lync ws://host:8787 # one-shot: push what it lacks, pull what you lack +lync sync story.lync ws://host:8787 --follow # stay live until Ctrl-C +``` + +`lync sync` uses Node's built-in WebSocket — no install beyond the package. +`lync serve` runs the relay and needs `ws` present (`npm install ws`); see +below. + +## The Relay + +The relay is deliberately dumb. Events are immutable and merge is union by +id, so the protocol has no merge logic: five JSON frames (`sub`, `ev`, +`live`, `presence`, `err`) that move canonical line bytes. The server never +parses a line beyond extracting its id, stores each root as a plain `.lync` +file you can read with any lync tool, and echoes accepted events to every +subscriber — echoes are duplicate no-ops under union. `seq` is a per-root +arrival counter used as a resume cursor (`.sync.json`), so an offline +client reconnects exactly where it left off. + +Running a relay is the one thing that needs a WebSocket server, and Node does +not ship one — so the relay acquires [`ws`](https://www.npmjs.com/package/ws) +lazily at the moment you construct it. `lync-core` declares no dependency on +`ws` at all: install it yourself next to your server +(`npm install ws`), and everything else in the package works without it. +If you bundle a server that runs the relay, mark `ws` as external — the +acquisition is a dynamic require that bundlers cannot see through. + +Standalone: + + +```ts +import { startLyncServe } from "lync-core/relay"; + +const server = await startLyncServe({ dir: "./rooms", port: 8787 }); +console.log("relay on", server.port); +// later: await server.close(); +``` + +On an existing HTTP server: + + +```ts +import { createServer } from "node:http"; +import { attachLyncServer } from "lync-core/relay"; + +const httpServer = createServer(app); +const lync = attachLyncServer(httpServer, { + storageDir: "./rooms", + path: "/lync", // default + keepAliveInterval: 30_000, // optional: ping through idle proxies + maxConnections: 500, // optional + authenticate: (req) => checkSession(req), // optional, after token check +}); +httpServer.listen(3000); +``` + +For full control, `createLyncRelay` gives you `handleUpgrade` to call from +your own `upgrade` listener. + +Guarantees: same-id-different-bytes is never resolved — both variants are +kept (a `.conflicts` sidecar) and both sides are told loudly. Persist failures +are broadcast, never swallowed. A truncated final line after a crash is +sealed and surfaced as damaged, never eaten. Presence frames are relayed, +never stored. `--token T` (or `token` in the API) requires +`Authorization: Bearer T` on every upgrade. + +The relay is a tool shipped beside the format, not part of it: FORMAT.md +deliberately excludes sync protocols, and any transport that moves canonical +line bytes and unions by id converges the same files without this relay. ## Development + ```bash pnpm install pnpm build -pnpm exec lync --help +node bin/lync.js --help pnpm test pnpm verify ``` -`pnpm verify` runs tests, builds packages, typechecks emitted package -surfaces, and executes every fenced example in the package READMEs against -the built packages (`pnpm check:examples`). README examples are contract: -a block runs as-written unless an `` comment above -it declares why it can't run alone. - -For a fresh clone, `pnpm install && pnpm build` is the supported setup sequence. -After that, `pnpm exec lync --help` should print the CLI help from the workspace -root. `scripts/fresh-clone-smoke.sh` verifies that sequence in a temporary clone -and runs the CLI story path: init, append, view, concatenate, merge, and verify. +`pnpm verify` runs the path guard, tests, build + typecheck, and executes +every fenced example in this README against the built package +(`pnpm check:examples`). README examples are contract: a block runs +as-written unless an `` comment above it declares +why it can't run alone. + +For a fresh clone, `pnpm install && pnpm build` is the supported setup +sequence. After that, `node bin/lync.js --help` prints the CLI help. +`scripts/fresh-clone-smoke.sh` verifies that sequence in a temporary clone +and runs the CLI story path: init, append, view, concatenate, merge, and +verify. diff --git a/bin/lync.js b/bin/lync.js new file mode 100755 index 0000000..39cd7f3 --- /dev/null +++ b/bin/lync.js @@ -0,0 +1,2 @@ +#!/usr/bin/env node +import "../dist/cli/bin.js"; diff --git a/package.json b/package.json index dd291a7..5568374 100644 --- a/package.json +++ b/package.json @@ -1,24 +1,185 @@ { - "name": "lync-workspace", - "private": true, + "name": "lync-core", + "version": "0.3.0", + "description": "The lync format: append-only JSONL event logs merged by set union. Parsing, stores, views, looms, live sync, loom client, indexes, the sync relay, and the lync command. Zero dependencies.", "type": "module", "license": "MIT", + "sideEffects": false, + "repository": { + "type": "git", + "url": "git+https://github.com/deepfates/lync.git" + }, + "homepage": "https://github.com/deepfates/lync#readme", + "bugs": "https://github.com/deepfates/lync/issues", + "publishConfig": { + "access": "public" + }, "packageManager": "pnpm@9.15.0", - "workspaces": [ - "packages/*" + "bin": { + "lync": "bin/lync.js" + }, + "main": "./dist/index.js", + "types": "./dist/index.d.ts", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js", + "default": "./dist/index.js" + }, + "./errors": { + "types": "./dist/errors.d.ts", + "import": "./dist/errors.js", + "default": "./dist/errors.js" + }, + "./memory": { + "types": "./dist/memory.d.ts", + "import": "./dist/memory.js", + "default": "./dist/memory.js" + }, + "./events": { + "types": "./dist/events.d.ts", + "import": "./dist/events.js", + "default": "./dist/events.js" + }, + "./file-log": { + "types": "./dist/file-log.d.ts", + "import": "./dist/file-log.js", + "default": "./dist/file-log.js" + }, + "./idb-log": { + "types": "./dist/idb-log.d.ts", + "import": "./dist/idb-log.js", + "default": "./dist/idb-log.js" + }, + "./looms": { + "types": "./dist/looms.d.ts", + "import": "./dist/looms.js", + "default": "./dist/looms.js" + }, + "./memory-log": { + "types": "./dist/memory-log.d.ts", + "import": "./dist/memory-log.js", + "default": "./dist/memory-log.js" + }, + "./store": { + "types": "./dist/store.d.ts", + "import": "./dist/store.js", + "default": "./dist/store.js" + }, + "./views": { + "types": "./dist/views.d.ts", + "import": "./dist/views.js", + "default": "./dist/views.js" + }, + "./profiles/text-story": { + "types": "./dist/profiles/text-story.d.ts", + "import": "./dist/profiles/text-story.js", + "default": "./dist/profiles/text-story.js" + }, + "./references": { + "types": "./dist/references.d.ts", + "import": "./dist/references.js", + "default": "./dist/references.js" + }, + "./types": { + "types": "./dist/types.d.ts", + "import": "./dist/types.js", + "default": "./dist/types.js" + }, + "./sync-protocol": { + "types": "./dist/sync-protocol.d.ts", + "import": "./dist/sync-protocol.js", + "default": "./dist/sync-protocol.js" + }, + "./synced-store": { + "types": "./dist/synced-store.d.ts", + "import": "./dist/synced-store.js", + "default": "./dist/synced-store.js" + }, + "./uuid": { + "types": "./dist/uuid.d.ts", + "import": "./dist/uuid.js", + "default": "./dist/uuid.js" + }, + "./indexes": { + "types": "./dist/indexes/index.d.ts", + "import": "./dist/indexes/index.js", + "default": "./dist/indexes/index.js" + }, + "./indexes/entries": { + "types": "./dist/indexes/entries.d.ts", + "import": "./dist/indexes/entries.js", + "default": "./dist/indexes/entries.js" + }, + "./indexes/memory": { + "types": "./dist/indexes/memory.d.ts", + "import": "./dist/indexes/memory.js", + "default": "./dist/indexes/memory.js" + }, + "./indexes/types": { + "types": "./dist/indexes/types.d.ts", + "import": "./dist/indexes/types.js", + "default": "./dist/indexes/types.js" + }, + "./client": { + "types": "./dist/client/index.d.ts", + "import": "./dist/client/index.js", + "default": "./dist/client/index.js" + }, + "./client/testing": { + "types": "./dist/client/testing.d.ts", + "import": "./dist/client/testing.js", + "default": "./dist/client/testing.js" + }, + "./client/types": { + "types": "./dist/client/types.d.ts", + "import": "./dist/client/types.js", + "default": "./dist/client/types.js" + }, + "./relay": { + "types": "./dist/relay/index.d.ts", + "import": "./dist/relay/index.js", + "default": "./dist/relay/index.js" + } + }, + "files": [ + "bin", + "dist" ], + "dependencies": {}, "scripts": { - "build": "pnpm -r build", + "build": "tsc -p tsconfig.json", "test": "vitest run", - "typecheck": "pnpm build && pnpm -r typecheck", + "typecheck": "pnpm build && tsc -p tsconfig.json --noEmit", "guard:paths": "bash scripts/no-machine-local-paths.sh", "verify": "pnpm guard:paths && pnpm test && pnpm typecheck && pnpm check:examples", - "check:examples": "node scripts/check-readme-examples.mjs" + "check:examples": "node scripts/check-readme-examples.mjs", + "prepublishOnly": "tsc -p tsconfig.json" }, "devDependencies": { "@types/node": "^22.14.0", "typescript": "^5.8.3", "vitest": "^3.1.1", - "lync-cli": "workspace:*" - } + "ws": "^8.18.0" + }, + "engines": { + "node": ">=22" + }, + "keywords": [ + "lync", + "jsonl", + "append-only", + "event-log", + "crdt-adjacent", + "local-first", + "sync", + "merge", + "loom", + "transcript", + "cli", + "websocket", + "relay", + "index", + "client" + ] } diff --git a/packages/cli/LICENSE b/packages/cli/LICENSE deleted file mode 100644 index cfbc3e3..0000000 --- a/packages/cli/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2026 deepfates - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/packages/cli/README.md b/packages/cli/README.md deleted file mode 100644 index 5a1d9eb..0000000 --- a/packages/cli/README.md +++ /dev/null @@ -1,45 +0,0 @@ -# lync-cli - -The `lync` command: work with `.lync` files — append-only JSONL interaction -history where each line is one immutable event and merge is set union by -event id. - -```bash -npm install -g lync-cli -``` - -## Seven verbs - -```bash -lync init story.lync -printf '%s\n' '{"kind":"notes/text","author":{"actor":"you"},"payload":{"text":"Once..."}}' | lync append story.lync -lync verify story.lync -lync view story.lync --as transcript -printf '%s\n' '{"kind":"notes/text","author":{"actor":"friend"},"payload":{"text":"Then..."}}' | lync append other.lync -lync merge story.lync other.lync -o merged.lync -``` - -`append` fills the envelope for you: a UUIDv7 id, the current timestamp, `v`, -and `parents` default in; anything you supply is kept. `verify` reports what -every physical line is — accepted, nonconforming, damaged, garbage, or -conflict variant — and never drops bytes. `view` renders `transcript` or -`tree`. - -## Sync - -Any lync file can converge with any other copy through a relay: - - -```bash -lync serve ./rooms --port 8787 # the relay: one append-only file per root -lync sync story.lync ws://host:8787 # one-shot: push what it lacks, pull what you lack -lync sync story.lync ws://host:8787 --follow # stay live until Ctrl-C -``` - -The relay stores each root as a plain `.lync` file you can read with any lync -tool. Same-id-different-bytes is never resolved: both variants are kept and -both sides are told loudly. An interrupted sync resumes from a per-root cursor -(`.sync.json`). - -Run `lync --help` for full usage. Format spec and docs: -https://github.com/deepfates/lync#readme diff --git a/packages/cli/bin/lync.js b/packages/cli/bin/lync.js deleted file mode 100755 index 3fcf46f..0000000 --- a/packages/cli/bin/lync.js +++ /dev/null @@ -1,2 +0,0 @@ -#!/usr/bin/env node -import "../dist/bin.js"; diff --git a/packages/cli/package.json b/packages/cli/package.json deleted file mode 100644 index 0ad4129..0000000 --- a/packages/cli/package.json +++ /dev/null @@ -1,61 +0,0 @@ -{ - "name": "lync-cli", - "version": "0.2.0", - "description": "The lync command: init, append, verify, merge, view, serve, sync for .lync event-log files.", - "type": "module", - "license": "MIT", - "sideEffects": false, - "repository": { - "type": "git", - "url": "git+https://github.com/deepfates/lync.git", - "directory": "packages/cli" - }, - "homepage": "https://github.com/deepfates/lync#readme", - "bugs": "https://github.com/deepfates/lync/issues", - "publishConfig": { - "access": "public" - }, - "bin": { - "lync": "bin/lync.js" - }, - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - } - }, - "files": [ - "bin", - "dist" - ], - "scripts": { - "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit", - "prepublishOnly": "tsc -p tsconfig.json" - }, - "dependencies": { - "lync-core": "workspace:*", - "ws": "^8.18.0", - "lync-server": "workspace:*" - }, - "devDependencies": { - "@types/ws": "^8.18.1" - }, - "engines": { - "node": ">=22" - }, - "keywords": [ - "lync", - "jsonl", - "append-only", - "event-log", - "crdt-adjacent", - "local-first", - "cli", - "sync", - "merge" - ] -} diff --git a/packages/cli/tsconfig.json b/packages/cli/tsconfig.json deleted file mode 100644 index df59da5..0000000 --- a/packages/cli/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*.ts"] -} diff --git a/packages/client/LICENSE b/packages/client/LICENSE deleted file mode 100644 index cfbc3e3..0000000 --- a/packages/client/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2026 deepfates - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/packages/client/README.md b/packages/client/README.md deleted file mode 100644 index 82a407b..0000000 --- a/packages/client/README.md +++ /dev/null @@ -1,33 +0,0 @@ -# lync-client - -The lync loom client: one object that pairs looms -([lync-core](https://www.npmjs.com/package/lync-core)) with an index -([lync-index](https://www.npmjs.com/package/lync-index)) and resolves -loom/turn/thread/index references to and from URLs. - -```bash -npm install lync-client -``` - -```ts -import { createLyncLooms } from "lync-core/looms"; -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createMemoryLoomIndexes } from "lync-index/memory"; -import { createLoomClient } from "lync-client"; - -const client = createLoomClient({ - looms: createLyncLooms({ store: createMemoryEventStore(), author: { actor: "you" } }), - indexes: createMemoryLoomIndexes(), -}); - -const info = await client.looms.create({ title: "Story" }); -const ref = client.references.loom(info.id); - -// Round-trip a reference through a shareable URL (?ref=...). -// In the browser, pass `window.location` instead of a URL object. -const url = client.references.toUrl(ref, new URL("https://example.com/story")); -const opened = await client.openReference(client.references.fromUrl(new URL(url))); -console.log(opened.kind); // "loom" — opened.loom is ready to appendTurn -``` - -Full docs: https://github.com/deepfates/lync#readme diff --git a/packages/client/package.json b/packages/client/package.json deleted file mode 100644 index debcff7..0000000 --- a/packages/client/package.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "name": "lync-client", - "version": "0.2.0", - "description": "The lync loom client: resolve references, open looms and indexes.", - "type": "module", - "license": "MIT", - "sideEffects": false, - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - }, - "./types": { - "types": "./dist/types.d.ts", - "import": "./dist/types.js", - "default": "./dist/types.js" - }, - "./testing": { - "types": "./dist/testing.d.ts", - "import": "./dist/testing.js", - "default": "./dist/testing.js" - } - }, - "files": [ - "dist" - ], - "scripts": { - "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit", - "prepublishOnly": "tsc -p tsconfig.json" - }, - "dependencies": { - "lync-core": "workspace:*", - "lync-index": "workspace:*" - }, - "repository": { - "type": "git", - "url": "git+https://github.com/deepfates/lync.git", - "directory": "packages/client" - }, - "homepage": "https://github.com/deepfates/lync#readme", - "bugs": "https://github.com/deepfates/lync/issues", - "publishConfig": { - "access": "public" - }, - "engines": { - "node": ">=22" - }, - "keywords": [ - "lync", - "jsonl", - "append-only", - "event-log", - "crdt-adjacent", - "local-first", - "client", - "loom", - "references" - ] -} diff --git a/packages/client/tsconfig.json b/packages/client/tsconfig.json deleted file mode 100644 index df59da5..0000000 --- a/packages/client/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*.ts"] -} diff --git a/packages/core/LICENSE b/packages/core/LICENSE deleted file mode 100644 index cfbc3e3..0000000 --- a/packages/core/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2026 deepfates - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/packages/core/README.md b/packages/core/README.md deleted file mode 100644 index e8940ba..0000000 --- a/packages/core/README.md +++ /dev/null @@ -1,86 +0,0 @@ -# lync-core - -The reference implementation of the lync format: `.lync` append-only JSONL -files of interaction history. Each line is one immutable event with an -envelope, parent links, provenance, and a payload owned by the event kind. -Merge is set union by event id. Branch trees, transcripts, memory views, and -leaderboards are computed views over the same event set. - -Zero runtime dependencies. Runs in Node (>=22) and the browser. - -```bash -npm install lync-core -``` - -## Parse, union, view - -```ts -import { parseLyncFiles } from "lync-core/events"; -import { lyncBranchTreeView, lyncTranscriptView } from "lync-core/views"; - -const bytes = new TextEncoder().encode( - '{"v":1,"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"you"},"parents":[],"payload":{"text":"Once..."}}\n', -); - -const parsed = parseLyncFiles([{ file: "story.lync", bytes }]); -console.log(parsed.lines[0].class); // "accepted" -console.log(lyncBranchTreeView(parsed).roots); -console.log(lyncTranscriptView(parsed, "root").path); -``` - -`parseLyncFiles` classifies every physical line and keeps the original bytes — -accepted events, nonconforming-but-carried lines, damaged lines, garbage, and -conflict variants (same id, different bytes) are all preserved and reported, -never silently dropped. `exportCarriedLyncBytes(parsed)` re-emits the carried -bytes. - -## Stores and looms - -Event stores share one contract over memory, file, and IndexedDB backends. The -loom API gives programs turns and threads instead of raw events, on top of any -store: - -```ts -import { createLyncLooms } from "lync-core/looms"; -import { createMemoryEventStore } from "lync-core/memory-log"; - -const looms = createLyncLooms({ - store: createMemoryEventStore(), - author: { actor: "you", via: "my-app@0.1" }, -}); - -const info = await looms.create({ title: "Story" }); -const loom = await looms.open(info.id); -const first = await loom.appendTurn(null, { text: "Once..." }); -await loom.appendTurn(first.id, { text: "Then..." }); -``` - -## Live sync - -Wrap any store in `createSyncedStore` and it converges with a relay -([lync-server](https://www.npmjs.com/package/lync-server)) over five JSON -frames. Local appends push, remote lines surface reactively, offline appends -queue and flush on reconnect: - - -```ts -import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; - -const store = createSyncedStore(localStore, createWebSocketTransport("wss://host/lync")); -``` - -## Subpath exports - -- `lync-core/events` — line parsing, carried-byte export, incremental union -- `lync-core/store` — the event-store contract and serialization -- `lync-core/memory-log`, `lync-core/file-log`, `lync-core/idb-log` — stores -- `lync-core/views` — branch tree, transcript, memory, leaderboard -- `lync-core/looms` — the loom/turn API -- `lync-core/synced-store` — live sync decorator and WebSocket transport -- `lync-core/uuid` — zero-dep UUIDv7 for event ids - -Normative format spec: -[FORMAT.md](https://github.com/deepfates/lync/blob/main/FORMAT.md). Import and -export conventions: -[pacts/](https://github.com/deepfates/lync/tree/main/pacts). Full docs: -https://github.com/deepfates/lync#readme diff --git a/packages/core/package.json b/packages/core/package.json deleted file mode 100644 index 1c26f89..0000000 --- a/packages/core/package.json +++ /dev/null @@ -1,125 +0,0 @@ -{ - "name": "lync-core", - "version": "0.2.0", - "description": "The lync format: append-only JSONL event logs merged by set union. Parsing, stores, views, looms, live sync. Zero dependencies.", - "type": "module", - "license": "MIT", - "sideEffects": false, - "repository": { - "type": "git", - "url": "git+https://github.com/deepfates/lync.git", - "directory": "packages/core" - }, - "homepage": "https://github.com/deepfates/lync#readme", - "bugs": "https://github.com/deepfates/lync/issues", - "publishConfig": { - "access": "public" - }, - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - }, - "./errors": { - "types": "./dist/errors.d.ts", - "import": "./dist/errors.js", - "default": "./dist/errors.js" - }, - "./memory": { - "types": "./dist/memory.d.ts", - "import": "./dist/memory.js", - "default": "./dist/memory.js" - }, - "./events": { - "types": "./dist/events.d.ts", - "import": "./dist/events.js", - "default": "./dist/events.js" - }, - "./file-log": { - "types": "./dist/file-log.d.ts", - "import": "./dist/file-log.js", - "default": "./dist/file-log.js" - }, - "./idb-log": { - "types": "./dist/idb-log.d.ts", - "import": "./dist/idb-log.js", - "default": "./dist/idb-log.js" - }, - "./looms": { - "types": "./dist/looms.d.ts", - "import": "./dist/looms.js", - "default": "./dist/looms.js" - }, - "./memory-log": { - "types": "./dist/memory-log.d.ts", - "import": "./dist/memory-log.js", - "default": "./dist/memory-log.js" - }, - "./store": { - "types": "./dist/store.d.ts", - "import": "./dist/store.js", - "default": "./dist/store.js" - }, - "./views": { - "types": "./dist/views.d.ts", - "import": "./dist/views.js", - "default": "./dist/views.js" - }, - "./profiles/text-story": { - "types": "./dist/profiles/text-story.d.ts", - "import": "./dist/profiles/text-story.js", - "default": "./dist/profiles/text-story.js" - }, - "./references": { - "types": "./dist/references.d.ts", - "import": "./dist/references.js", - "default": "./dist/references.js" - }, - "./types": { - "types": "./dist/types.d.ts", - "import": "./dist/types.js", - "default": "./dist/types.js" - }, - "./sync-protocol": { - "types": "./dist/sync-protocol.d.ts", - "import": "./dist/sync-protocol.js", - "default": "./dist/sync-protocol.js" - }, - "./synced-store": { - "types": "./dist/synced-store.d.ts", - "import": "./dist/synced-store.js", - "default": "./dist/synced-store.js" - }, - "./uuid": { - "types": "./dist/uuid.d.ts", - "import": "./dist/uuid.js", - "default": "./dist/uuid.js" - } - }, - "files": [ - "dist" - ], - "scripts": { - "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit", - "prepublishOnly": "tsc -p tsconfig.json" - }, - "engines": { - "node": ">=22" - }, - "keywords": [ - "lync", - "jsonl", - "append-only", - "event-log", - "crdt-adjacent", - "local-first", - "sync", - "merge", - "loom", - "transcript" - ] -} diff --git a/packages/core/tsconfig.json b/packages/core/tsconfig.json deleted file mode 100644 index df59da5..0000000 --- a/packages/core/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*.ts"] -} diff --git a/packages/index/LICENSE b/packages/index/LICENSE deleted file mode 100644 index cfbc3e3..0000000 --- a/packages/index/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2026 deepfates - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/packages/index/README.md b/packages/index/README.md deleted file mode 100644 index 5d0da81..0000000 --- a/packages/index/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# lync-index - -An index of many lync looms: track a collection, upsert entries, and subscribe -to changes. Depends only on -[lync-core](https://www.npmjs.com/package/lync-core). - -```bash -npm install lync-index -``` - -```ts -import { loomRef } from "lync-core"; -import { createMemoryLoomIndexes } from "lync-index/memory"; - -const indexes = createMemoryLoomIndexes(); -const index = await indexes.create({ title: "My looms" }); - -index.subscribe((event) => console.log("index changed:", event.type)); -await index.addLoom(loomRef("loom-1"), { title: "Story" }); - -console.log((await index.entries()).map((entry) => entry.title)); -``` - -Entries carry a loom reference, optional title/kind/meta, and timestamps. -`export()`/`import()` round-trip a whole index as a snapshot. - -Typically used through -[lync-client](https://www.npmjs.com/package/lync-client), which pairs an index -with looms and reference resolution. Full docs: -https://github.com/deepfates/lync#readme diff --git a/packages/index/package.json b/packages/index/package.json deleted file mode 100644 index 6283d9c..0000000 --- a/packages/index/package.json +++ /dev/null @@ -1,66 +0,0 @@ -{ - "name": "lync-index", - "version": "0.2.0", - "description": "An index of many lync looms, with reactive subscription.", - "type": "module", - "license": "MIT", - "sideEffects": false, - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - }, - "./entries": { - "types": "./dist/entries.d.ts", - "import": "./dist/entries.js", - "default": "./dist/entries.js" - }, - "./memory": { - "types": "./dist/memory.d.ts", - "import": "./dist/memory.js", - "default": "./dist/memory.js" - }, - "./types": { - "types": "./dist/types.d.ts", - "import": "./dist/types.js", - "default": "./dist/types.js" - } - }, - "files": [ - "dist" - ], - "scripts": { - "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit", - "prepublishOnly": "tsc -p tsconfig.json" - }, - "dependencies": { - "lync-core": "workspace:*" - }, - "repository": { - "type": "git", - "url": "git+https://github.com/deepfates/lync.git", - "directory": "packages/index" - }, - "homepage": "https://github.com/deepfates/lync#readme", - "bugs": "https://github.com/deepfates/lync/issues", - "publishConfig": { - "access": "public" - }, - "engines": { - "node": ">=22" - }, - "keywords": [ - "lync", - "jsonl", - "append-only", - "event-log", - "crdt-adjacent", - "local-first", - "index", - "loom" - ] -} diff --git a/packages/index/tsconfig.json b/packages/index/tsconfig.json deleted file mode 100644 index df59da5..0000000 --- a/packages/index/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*.ts"] -} diff --git a/packages/server/LICENSE b/packages/server/LICENSE deleted file mode 100644 index cfbc3e3..0000000 --- a/packages/server/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2026 deepfates - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/packages/server/README.md b/packages/server/README.md deleted file mode 100644 index 49cf331..0000000 --- a/packages/server/README.md +++ /dev/null @@ -1,55 +0,0 @@ -# lync-server - -The lync line-sync relay. It moves canonical `.lync` line bytes between -subscribers over five JSON frames (`sub`, `ev`, `live`, `presence`, `err`) and -has no merge logic — lync events are immutable and merge is set union by id, -so echoes are duplicate no-ops. Each root is stored as a plain append-only -`.lync` file you can read with any lync tool. The relay never parses a line -beyond extracting its id. - -```bash -npm install lync-server -``` - -## Standalone - - -```ts -import { startLyncServe } from "lync-server"; - -const server = await startLyncServe({ dir: "./rooms", port: 8787 }); -console.log("relay on", server.port); -// later: await server.close(); -``` - -## On an existing HTTP server - - -```ts -import { createServer } from "node:http"; -import { attachLyncServer } from "lync-server"; - -const httpServer = createServer(app); -const lync = attachLyncServer(httpServer, { - storageDir: "./rooms", - path: "/lync", // default - keepAliveInterval: 30_000, // optional: ping through idle proxies - maxConnections: 500, // optional - authenticate: (req) => checkSession(req), // optional, after token check -}); -httpServer.listen(3000); -``` - -For full control, `createLyncRelay` gives you `handleUpgrade` to call from -your own `upgrade` listener. - -Guarantees: same-id-different-bytes is never resolved — both variants are -kept (a `.conflicts` sidecar) and both sides are told loudly. Persist failures -are broadcast, never swallowed. A truncated final line after a crash is -sealed and surfaced as damaged, never eaten. `token` requires -`Authorization: Bearer ` on every upgrade. - -Client side: `lync sync` from -[lync-cli](https://www.npmjs.com/package/lync-cli), or `createSyncedStore` -from [lync-core](https://www.npmjs.com/package/lync-core) inside an app. Full -docs: https://github.com/deepfates/lync#readme diff --git a/packages/server/package.json b/packages/server/package.json deleted file mode 100644 index b174b6f..0000000 --- a/packages/server/package.json +++ /dev/null @@ -1,56 +0,0 @@ -{ - "name": "lync-server", - "version": "0.2.0", - "description": "The lync line-sync relay: mount on any Node server, or run standalone. Stores each root as a plain .lync file.", - "type": "module", - "license": "MIT", - "sideEffects": false, - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "exports": { - ".": { - "types": "./dist/index.d.ts", - "import": "./dist/index.js", - "default": "./dist/index.js" - } - }, - "files": [ - "dist" - ], - "scripts": { - "build": "tsc -p tsconfig.json", - "typecheck": "tsc -p tsconfig.json --noEmit", - "prepublishOnly": "tsc -p tsconfig.json" - }, - "dependencies": { - "lync-core": "workspace:*", - "ws": "^8.18.0" - }, - "devDependencies": { - "@types/ws": "^8.18.1" - }, - "repository": { - "type": "git", - "url": "git+https://github.com/deepfates/lync.git", - "directory": "packages/server" - }, - "homepage": "https://github.com/deepfates/lync#readme", - "bugs": "https://github.com/deepfates/lync/issues", - "publishConfig": { - "access": "public" - }, - "engines": { - "node": ">=22" - }, - "keywords": [ - "lync", - "jsonl", - "append-only", - "event-log", - "crdt-adjacent", - "local-first", - "websocket", - "relay", - "sync-server" - ] -} diff --git a/packages/server/tsconfig.json b/packages/server/tsconfig.json deleted file mode 100644 index df59da5..0000000 --- a/packages/server/tsconfig.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "extends": "../../tsconfig.base.json", - "compilerOptions": { - "rootDir": "src", - "outDir": "dist" - }, - "include": ["src/**/*.ts"] -} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index c380408..bd51c77 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -10,217 +10,171 @@ importers: devDependencies: '@types/node': specifier: ^22.14.0 - version: 22.19.17 - lync-cli: - specifier: workspace:* - version: link:packages/cli + version: 22.20.1 typescript: specifier: ^5.8.3 version: 5.9.3 vitest: specifier: ^3.1.1 - version: 3.2.4(@types/node@22.19.17) - - packages/cli: - dependencies: - lync-core: - specifier: workspace:* - version: link:../core - lync-server: - specifier: workspace:* - version: link:../server + version: 3.2.7(@types/node@22.20.1) ws: specifier: ^8.18.0 version: 8.21.0 - devDependencies: - '@types/ws': - specifier: ^8.18.1 - version: 8.18.1 - - packages/client: - dependencies: - lync-core: - specifier: workspace:* - version: link:../core - lync-index: - specifier: workspace:* - version: link:../index - - packages/core: {} - - packages/index: - dependencies: - lync-core: - specifier: workspace:* - version: link:../core - - packages/server: - dependencies: - lync-core: - specifier: workspace:* - version: link:../core - ws: - specifier: ^8.18.0 - version: 8.21.0 - devDependencies: - '@types/ws': - specifier: ^8.18.1 - version: 8.18.1 packages: - '@esbuild/aix-ppc64@0.27.7': - resolution: {integrity: sha512-EKX3Qwmhz1eMdEJokhALr0YiD0lhQNwDqkPYyPhiSwKrh7/4KRjQc04sZ8db+5DVVnZ1LmbNDI1uAMPEUBnQPg==} + '@esbuild/aix-ppc64@0.28.1': + resolution: {integrity: sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==} engines: {node: '>=18'} cpu: [ppc64] os: [aix] - '@esbuild/android-arm64@0.27.7': - resolution: {integrity: sha512-62dPZHpIXzvChfvfLJow3q5dDtiNMkwiRzPylSCfriLvZeq0a1bWChrGx/BbUbPwOrsWKMn8idSllklzBy+dgQ==} + '@esbuild/android-arm64@0.28.1': + resolution: {integrity: sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==} engines: {node: '>=18'} cpu: [arm64] os: [android] - '@esbuild/android-arm@0.27.7': - resolution: {integrity: sha512-jbPXvB4Yj2yBV7HUfE2KHe4GJX51QplCN1pGbYjvsyCZbQmies29EoJbkEc+vYuU5o45AfQn37vZlyXy4YJ8RQ==} + '@esbuild/android-arm@0.28.1': + resolution: {integrity: sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==} engines: {node: '>=18'} cpu: [arm] os: [android] - '@esbuild/android-x64@0.27.7': - resolution: {integrity: sha512-x5VpMODneVDb70PYV2VQOmIUUiBtY3D3mPBG8NxVk5CogneYhkR7MmM3yR/uMdITLrC1ml/NV1rj4bMJuy9MCg==} + '@esbuild/android-x64@0.28.1': + resolution: {integrity: sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==} engines: {node: '>=18'} cpu: [x64] os: [android] - '@esbuild/darwin-arm64@0.27.7': - resolution: {integrity: sha512-5lckdqeuBPlKUwvoCXIgI2D9/ABmPq3Rdp7IfL70393YgaASt7tbju3Ac+ePVi3KDH6N2RqePfHnXkaDtY9fkw==} + '@esbuild/darwin-arm64@0.28.1': + resolution: {integrity: sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==} engines: {node: '>=18'} cpu: [arm64] os: [darwin] - '@esbuild/darwin-x64@0.27.7': - resolution: {integrity: sha512-rYnXrKcXuT7Z+WL5K980jVFdvVKhCHhUwid+dDYQpH+qu+TefcomiMAJpIiC2EM3Rjtq0sO3StMV/+3w3MyyqQ==} + '@esbuild/darwin-x64@0.28.1': + resolution: {integrity: sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==} engines: {node: '>=18'} cpu: [x64] os: [darwin] - '@esbuild/freebsd-arm64@0.27.7': - resolution: {integrity: sha512-B48PqeCsEgOtzME2GbNM2roU29AMTuOIN91dsMO30t+Ydis3z/3Ngoj5hhnsOSSwNzS+6JppqWsuhTp6E82l2w==} + '@esbuild/freebsd-arm64@0.28.1': + resolution: {integrity: sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==} engines: {node: '>=18'} cpu: [arm64] os: [freebsd] - '@esbuild/freebsd-x64@0.27.7': - resolution: {integrity: sha512-jOBDK5XEjA4m5IJK3bpAQF9/Lelu/Z9ZcdhTRLf4cajlB+8VEhFFRjWgfy3M1O4rO2GQ/b2dLwCUGpiF/eATNQ==} + '@esbuild/freebsd-x64@0.28.1': + resolution: {integrity: sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==} engines: {node: '>=18'} cpu: [x64] os: [freebsd] - '@esbuild/linux-arm64@0.27.7': - resolution: {integrity: sha512-RZPHBoxXuNnPQO9rvjh5jdkRmVizktkT7TCDkDmQ0W2SwHInKCAV95GRuvdSvA7w4VMwfCjUiPwDi0ZO6Nfe9A==} + '@esbuild/linux-arm64@0.28.1': + resolution: {integrity: sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==} engines: {node: '>=18'} cpu: [arm64] os: [linux] - '@esbuild/linux-arm@0.27.7': - resolution: {integrity: sha512-RkT/YXYBTSULo3+af8Ib0ykH8u2MBh57o7q/DAs3lTJlyVQkgQvlrPTnjIzzRPQyavxtPtfg0EopvDyIt0j1rA==} + '@esbuild/linux-arm@0.28.1': + resolution: {integrity: sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==} engines: {node: '>=18'} cpu: [arm] os: [linux] - '@esbuild/linux-ia32@0.27.7': - resolution: {integrity: sha512-GA48aKNkyQDbd3KtkplYWT102C5sn/EZTY4XROkxONgruHPU72l+gW+FfF8tf2cFjeHaRbWpOYa/uRBz/Xq1Pg==} + '@esbuild/linux-ia32@0.28.1': + resolution: {integrity: sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==} engines: {node: '>=18'} cpu: [ia32] os: [linux] - '@esbuild/linux-loong64@0.27.7': - resolution: {integrity: sha512-a4POruNM2oWsD4WKvBSEKGIiWQF8fZOAsycHOt6JBpZ+JN2n2JH9WAv56SOyu9X5IqAjqSIPTaJkqN8F7XOQ5Q==} + '@esbuild/linux-loong64@0.28.1': + resolution: {integrity: sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==} engines: {node: '>=18'} cpu: [loong64] os: [linux] - '@esbuild/linux-mips64el@0.27.7': - resolution: {integrity: sha512-KabT5I6StirGfIz0FMgl1I+R1H73Gp0ofL9A3nG3i/cYFJzKHhouBV5VWK1CSgKvVaG4q1RNpCTR2LuTVB3fIw==} + '@esbuild/linux-mips64el@0.28.1': + resolution: {integrity: sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==} engines: {node: '>=18'} cpu: [mips64el] os: [linux] - '@esbuild/linux-ppc64@0.27.7': - resolution: {integrity: sha512-gRsL4x6wsGHGRqhtI+ifpN/vpOFTQtnbsupUF5R5YTAg+y/lKelYR1hXbnBdzDjGbMYjVJLJTd2OFmMewAgwlQ==} + '@esbuild/linux-ppc64@0.28.1': + resolution: {integrity: sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==} engines: {node: '>=18'} cpu: [ppc64] os: [linux] - '@esbuild/linux-riscv64@0.27.7': - resolution: {integrity: sha512-hL25LbxO1QOngGzu2U5xeXtxXcW+/GvMN3ejANqXkxZ/opySAZMrc+9LY/WyjAan41unrR3YrmtTsUpwT66InQ==} + '@esbuild/linux-riscv64@0.28.1': + resolution: {integrity: sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==} engines: {node: '>=18'} cpu: [riscv64] os: [linux] - '@esbuild/linux-s390x@0.27.7': - resolution: {integrity: sha512-2k8go8Ycu1Kb46vEelhu1vqEP+UeRVj2zY1pSuPdgvbd5ykAw82Lrro28vXUrRmzEsUV0NzCf54yARIK8r0fdw==} + '@esbuild/linux-s390x@0.28.1': + resolution: {integrity: sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==} engines: {node: '>=18'} cpu: [s390x] os: [linux] - '@esbuild/linux-x64@0.27.7': - resolution: {integrity: sha512-hzznmADPt+OmsYzw1EE33ccA+HPdIqiCRq7cQeL1Jlq2gb1+OyWBkMCrYGBJ+sxVzve2ZJEVeePbLM2iEIZSxA==} + '@esbuild/linux-x64@0.28.1': + resolution: {integrity: sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==} engines: {node: '>=18'} cpu: [x64] os: [linux] - '@esbuild/netbsd-arm64@0.27.7': - resolution: {integrity: sha512-b6pqtrQdigZBwZxAn1UpazEisvwaIDvdbMbmrly7cDTMFnw/+3lVxxCTGOrkPVnsYIosJJXAsILG9XcQS+Yu6w==} + '@esbuild/netbsd-arm64@0.28.1': + resolution: {integrity: sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==} engines: {node: '>=18'} cpu: [arm64] os: [netbsd] - '@esbuild/netbsd-x64@0.27.7': - resolution: {integrity: sha512-OfatkLojr6U+WN5EDYuoQhtM+1xco+/6FSzJJnuWiUw5eVcicbyK3dq5EeV/QHT1uy6GoDhGbFpprUiHUYggrw==} + '@esbuild/netbsd-x64@0.28.1': + resolution: {integrity: sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==} engines: {node: '>=18'} cpu: [x64] os: [netbsd] - '@esbuild/openbsd-arm64@0.27.7': - resolution: {integrity: sha512-AFuojMQTxAz75Fo8idVcqoQWEHIXFRbOc1TrVcFSgCZtQfSdc1RXgB3tjOn/krRHENUB4j00bfGjyl2mJrU37A==} + '@esbuild/openbsd-arm64@0.28.1': + resolution: {integrity: sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==} engines: {node: '>=18'} cpu: [arm64] os: [openbsd] - '@esbuild/openbsd-x64@0.27.7': - resolution: {integrity: sha512-+A1NJmfM8WNDv5CLVQYJ5PshuRm/4cI6WMZRg1by1GwPIQPCTs1GLEUHwiiQGT5zDdyLiRM/l1G0Pv54gvtKIg==} + '@esbuild/openbsd-x64@0.28.1': + resolution: {integrity: sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==} engines: {node: '>=18'} cpu: [x64] os: [openbsd] - '@esbuild/openharmony-arm64@0.27.7': - resolution: {integrity: sha512-+KrvYb/C8zA9CU/g0sR6w2RBw7IGc5J2BPnc3dYc5VJxHCSF1yNMxTV5LQ7GuKteQXZtspjFbiuW5/dOj7H4Yw==} + '@esbuild/openharmony-arm64@0.28.1': + resolution: {integrity: sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==} engines: {node: '>=18'} cpu: [arm64] os: [openharmony] - '@esbuild/sunos-x64@0.27.7': - resolution: {integrity: sha512-ikktIhFBzQNt/QDyOL580ti9+5mL/YZeUPKU2ivGtGjdTYoqz6jObj6nOMfhASpS4GU4Q/Clh1QtxWAvcYKamA==} + '@esbuild/sunos-x64@0.28.1': + resolution: {integrity: sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==} engines: {node: '>=18'} cpu: [x64] os: [sunos] - '@esbuild/win32-arm64@0.27.7': - resolution: {integrity: sha512-7yRhbHvPqSpRUV7Q20VuDwbjW5kIMwTHpptuUzV+AA46kiPze5Z7qgt6CLCK3pWFrHeNfDd1VKgyP4O+ng17CA==} + '@esbuild/win32-arm64@0.28.1': + resolution: {integrity: sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==} engines: {node: '>=18'} cpu: [arm64] os: [win32] - '@esbuild/win32-ia32@0.27.7': - resolution: {integrity: sha512-SmwKXe6VHIyZYbBLJrhOoCJRB/Z1tckzmgTLfFYOfpMAx63BJEaL9ExI8x7v0oAO3Zh6D/Oi1gVxEYr5oUCFhw==} + '@esbuild/win32-ia32@0.28.1': + resolution: {integrity: sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==} engines: {node: '>=18'} cpu: [ia32] os: [win32] - '@esbuild/win32-x64@0.27.7': - resolution: {integrity: sha512-56hiAJPhwQ1R4i+21FVF7V8kSD5zZTdHcVuRFMW0hn753vVfQN8xlx4uOPT4xoGH0Z/oVATuR82AiqSTDIpaHg==} + '@esbuild/win32-x64@0.28.1': + resolution: {integrity: sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==} engines: {node: '>=18'} cpu: [x64] os: [win32] @@ -228,128 +182,128 @@ packages: '@jridgewell/sourcemap-codec@1.5.5': resolution: {integrity: sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==} - '@rollup/rollup-android-arm-eabi@4.60.2': - resolution: {integrity: sha512-dnlp69efPPg6Uaw2dVqzWRfAWRnYVb1XJ8CyyhIbZeaq4CA5/mLeZ1IEt9QqQxmbdvagjLIm2ZL8BxXv5lH4Yw==} + '@rollup/rollup-android-arm-eabi@4.62.2': + resolution: {integrity: sha512-6o7ZLZK+BeenkZCFNDXqpbjw9bD6nuWonvS/lwQJp7NoVVxm6p3qE7qQ5jGuBjiFsgvqjD8mZAU5oWxTmbOeOg==} cpu: [arm] os: [android] - '@rollup/rollup-android-arm64@4.60.2': - resolution: {integrity: sha512-OqZTwDRDchGRHHm/hwLOL7uVPB9aUvI0am/eQuWMNyFHf5PSEQmyEeYYheA0EPPKUO/l0uigCp+iaTjoLjVoHg==} + '@rollup/rollup-android-arm64@4.62.2': + resolution: {integrity: sha512-BaH7BllCACHoH1LguOU56UItGfUWjujlO65kS9LAodViaN4bwIKd7oeW/ZHJ/4ljr/7MIiENnNy3HJ0zXv8Zkw==} cpu: [arm64] os: [android] - '@rollup/rollup-darwin-arm64@4.60.2': - resolution: {integrity: sha512-UwRE7CGpvSVEQS8gUMBe1uADWjNnVgP3Iusyda1nSRwNDCsRjnGc7w6El6WLQsXmZTbLZx9cecegumcitNfpmA==} + '@rollup/rollup-darwin-arm64@4.62.2': + resolution: {integrity: sha512-v39RCCvj4He82I9sFmk+M1VZ0PLM9sfsLVikjfx2hYBNALhrrOR2D3JjQA6AhlaSOgcR+RzrKY7e1+bT6SUO/A==} cpu: [arm64] os: [darwin] - '@rollup/rollup-darwin-x64@4.60.2': - resolution: {integrity: sha512-gjEtURKLCC5VXm1I+2i1u9OhxFsKAQJKTVB8WvDAHF+oZlq0GTVFOlTlO1q3AlCTE/DF32c16ESvfgqR7343/g==} + '@rollup/rollup-darwin-x64@4.62.2': + resolution: {integrity: sha512-yl0y2vq3S3lHeuXhEdss6TWfKW8vkujImO12tn4ZkG/4oghr09LvdYm2RElVjokTQiUvDUGXLGsYeLqUMCKpGA==} cpu: [x64] os: [darwin] - '@rollup/rollup-freebsd-arm64@4.60.2': - resolution: {integrity: sha512-Bcl6CYDeAgE70cqZaMojOi/eK63h5Me97ZqAQoh77VPjMysA/4ORQBRGo3rRy45x4MzVlU9uZxs8Uwy7ZaKnBw==} + '@rollup/rollup-freebsd-arm64@4.62.2': + resolution: {integrity: sha512-tT4pvt4qXD+vEoezupCWi+a1F0vvDiksiHc+PxRlYTOH1I6/X4id9jPxTP+Fg+545euaFT1jJVs4CEdHZAU1vw==} cpu: [arm64] os: [freebsd] - '@rollup/rollup-freebsd-x64@4.60.2': - resolution: {integrity: sha512-LU+TPda3mAE2QB0/Hp5VyeKJivpC6+tlOXd1VMoXV/YFMvk/MNk5iXeBfB4MQGRWyOYVJ01625vjkr0Az98OJQ==} + '@rollup/rollup-freebsd-x64@4.62.2': + resolution: {integrity: sha512-6nU5F2wCW+qvCBhTn1pdIU3bzsIoF7EUwsCDRxilWGprQR6yd508YnH9+OKFCwpfS8pjZqDUmnCAr7exax0XCg==} cpu: [x64] os: [freebsd] - '@rollup/rollup-linux-arm-gnueabihf@4.60.2': - resolution: {integrity: sha512-2QxQrM+KQ7DAW4o22j+XZ6RKdxjLD7BOWTP0Bv0tmjdyhXSsr2Ul1oJDQqh9Zf5qOwTuTc7Ek83mOFaKnodPjg==} + '@rollup/rollup-linux-arm-gnueabihf@4.62.2': + resolution: {integrity: sha512-n1GJHPOvpIfhi3TmrCeh6S6URt9BFCt0KQE3qvexyGCTAKpR4Lg+eWvNZEqu7epxwus/8ElT3hacYEucm49SZg==} cpu: [arm] os: [linux] - '@rollup/rollup-linux-arm-musleabihf@4.60.2': - resolution: {integrity: sha512-TbziEu2DVsTEOPif2mKWkMeDMLoYjx95oESa9fkQQK7r/Orta0gnkcDpzwufEcAO2BLBsD7mZkXGFqEdMRRwfw==} + '@rollup/rollup-linux-arm-musleabihf@4.62.2': + resolution: {integrity: sha512-JqgflS8wEB+UXV/vS1RpRbifGBeN4D5lz8D8oOFbFZw4vedvdOgCFAjfBmIMdW3yL10XpQQ0Ambepw6MXrhOnA==} cpu: [arm] os: [linux] - '@rollup/rollup-linux-arm64-gnu@4.60.2': - resolution: {integrity: sha512-bO/rVDiDUuM2YfuCUwZ1t1cP+/yqjqz+Xf2VtkdppefuOFS2OSeAfgafaHNkFn0t02hEyXngZkxtGqXcXwO8Rg==} + '@rollup/rollup-linux-arm64-gnu@4.62.2': + resolution: {integrity: sha512-wnFJkogWvN4jm/hQRF2UBaeUmk20j5+DmHvoyWii2b8HJDyvz1MF2OU/6ynXt2KR63rbZLWkFpoytpdc/yBuSA==} cpu: [arm64] os: [linux] - '@rollup/rollup-linux-arm64-musl@4.60.2': - resolution: {integrity: sha512-hr26p7e93Rl0Za+JwW7EAnwAvKkehh12BU1Llm9Ykiibg4uIr2rbpxG9WCf56GuvidlTG9KiiQT/TXT1yAWxTA==} + '@rollup/rollup-linux-arm64-musl@4.62.2': + resolution: {integrity: sha512-HVu2bp0zhvJ8xHEV9+UUs7S90VadmBSY3LcIMvozbPo4AuMGDWlz3ymHLHZPX4hR67TKTt8Qp5PJ5RBg/i+RMQ==} cpu: [arm64] os: [linux] - '@rollup/rollup-linux-loong64-gnu@4.60.2': - resolution: {integrity: sha512-pOjB/uSIyDt+ow3k/RcLvUAOGpysT2phDn7TTUB3n75SlIgZzM6NKAqlErPhoFU+npgY3/n+2HYIQVbF70P9/A==} + '@rollup/rollup-linux-loong64-gnu@4.62.2': + resolution: {integrity: sha512-mQqqAV8QaoSgr9I2fKDLY2BAVvmKjWoGiu/cSYQonsLvtqwEn1E4QYfnCOcp5zoEqNhsDYin1s6jx/VJmrxlZg==} cpu: [loong64] os: [linux] - '@rollup/rollup-linux-loong64-musl@4.60.2': - resolution: {integrity: sha512-2/w+q8jszv9Ww1c+6uJT3OwqhdmGP2/4T17cu8WuwyUuuaCDDJ2ojdyYwZzCxx0GcsZBhzi3HmH+J5pZNXnd+Q==} + '@rollup/rollup-linux-loong64-musl@4.62.2': + resolution: {integrity: sha512-IxKLoxCQ2IWi6bT2akyDUBGsOImDKB+sPp4EsTmwFQ/fMwpCKm8uLSSgP/Kx/QYUgKis6SEZ5/Nlhup0DIA0PQ==} cpu: [loong64] os: [linux] - '@rollup/rollup-linux-ppc64-gnu@4.60.2': - resolution: {integrity: sha512-11+aL5vKheYgczxtPVVRhdptAM2H7fcDR5Gw4/bTcteuZBlH4oP9f5s9zYO9aGZvoGeBpqXI/9TZZihZ609wKw==} + '@rollup/rollup-linux-ppc64-gnu@4.62.2': + resolution: {integrity: sha512-Mk5ha2RQSgyFfmYYLkBpPnUk8D8FriBxesO1u9O75X0mHgXL1UQcH5Itl2lurWL2tj0RxV9b9tJgipac0hRY9A==} cpu: [ppc64] os: [linux] - '@rollup/rollup-linux-ppc64-musl@4.60.2': - resolution: {integrity: sha512-i16fokAGK46IVZuV8LIIwMdtqhin9hfYkCh8pf8iC3QU3LpwL+1FSFGej+O7l3E/AoknL6Dclh2oTdnRMpTzFQ==} + '@rollup/rollup-linux-ppc64-musl@4.62.2': + resolution: {integrity: sha512-CjvEnqJL/0/TQ3TXX3OPIJ/kmBellrWd4heXUmHeJlTnmwjKpSJzoehLaL6Xk0ZnMHBu9dZuFADNOrtjF4v+2w==} cpu: [ppc64] os: [linux] - '@rollup/rollup-linux-riscv64-gnu@4.60.2': - resolution: {integrity: sha512-49FkKS6RGQoriDSK/6E2GkAsAuU5kETFCh7pG4yD/ylj9rKhTmO3elsnmBvRD4PgJPds5W2PkhC82aVwmUcJ7A==} + '@rollup/rollup-linux-riscv64-gnu@4.62.2': + resolution: {integrity: sha512-1SiZbzwdkaDURsew/tSOrooKiYy7EQGT6m8ufavAi9NEyQb/6VuIxFXAL1fqa4iZe3g4NbNk4P7J32z2tw5Mgg==} cpu: [riscv64] os: [linux] - '@rollup/rollup-linux-riscv64-musl@4.60.2': - resolution: {integrity: sha512-mjYNkHPfGpUR00DuM1ZZIgs64Hpf4bWcz9Z41+4Q+pgDx73UwWdAYyf6EG/lRFldmdHHzgrYyge5akFUW0D3mQ==} + '@rollup/rollup-linux-riscv64-musl@4.62.2': + resolution: {integrity: sha512-nQts12zJ3NQRoE6uYljOH89v7szzLDvG2JD/vsX+vGXU8w/At1GowTZ5/7qeFQ8m7L55rpR8Okugnuo5bgjy2Q==} cpu: [riscv64] os: [linux] - '@rollup/rollup-linux-s390x-gnu@4.60.2': - resolution: {integrity: sha512-ALyvJz965BQk8E9Al/JDKKDLH2kfKFLTGMlgkAbbYtZuJt9LU8DW3ZoDMCtQpXAltZxwBHevXz5u+gf0yA0YoA==} + '@rollup/rollup-linux-s390x-gnu@4.62.2': + resolution: {integrity: sha512-E9/ll019jhPIJgpzfZoIkBGhcz+kKNgVWYRY0zr9srBdPPFVpvOKW8VaJKUbeK+eZXyQF9ltME+Kk6affeaPgg==} cpu: [s390x] os: [linux] - '@rollup/rollup-linux-x64-gnu@4.60.2': - resolution: {integrity: sha512-UQjrkIdWrKI626Du8lCQ6MJp/6V1LAo2bOK9OTu4mSn8GGXIkPXk/Vsp4bLHCd9Z9Iz2OTEaokUE90VweJgIYQ==} + '@rollup/rollup-linux-x64-gnu@4.62.2': + resolution: {integrity: sha512-5BqxR/pshjey51iliyzTD5Xi3EN0aLmQ2lZ3lvefVV9c82BvrLo2/6OT55iifpWBufs6kdwWbuOKS841DrmK9A==} cpu: [x64] os: [linux] - '@rollup/rollup-linux-x64-musl@4.60.2': - resolution: {integrity: sha512-bTsRGj6VlSdn/XD4CGyzMnzaBs9bsRxy79eTqTCBsA8TMIEky7qg48aPkvJvFe1HyzQ5oMZdg7AnVlWQSKLTnw==} + '@rollup/rollup-linux-x64-musl@4.62.2': + resolution: {integrity: sha512-uNN83XxQrRAh/w0/pmAfibcwyb6YWt4gP+dpnQKPVJshAloQ785ii8CT8ZCIxkGg9opVsvAlGhFitSm6D1Jjpg==} cpu: [x64] os: [linux] - '@rollup/rollup-openbsd-x64@4.60.2': - resolution: {integrity: sha512-6d4Z3534xitaA1FcMWP7mQPq5zGwBmGbhphh2DwaA1aNIXUu3KTOfwrWpbwI4/Gr0uANo7NTtaykFyO2hPuFLg==} + '@rollup/rollup-openbsd-x64@4.62.2': + resolution: {integrity: sha512-srjEIxSH3LRnJN6THczDHWQplqEMFiAJrTab0msUryh9kwNpkICf3Ea6q6MN/2cZwRFUNx5w+h6Hpi4QuHS6Zg==} cpu: [x64] os: [openbsd] - '@rollup/rollup-openharmony-arm64@4.60.2': - resolution: {integrity: sha512-NetAg5iO2uN7eB8zE5qrZ3CSil+7IJt4WDFLcC75Ymywq1VZVD6qJ6EvNLjZ3rEm6gB7XW5JdT60c6MN35Z85Q==} + '@rollup/rollup-openharmony-arm64@4.62.2': + resolution: {integrity: sha512-8hOJnxgbyObnCm5AlRA3A931xX19xq80RjVTKgJOvEKWqJruP/Uf12IbAOaDjjEXYRewwHLfmF0YRIdK3OwKWA==} cpu: [arm64] os: [openharmony] - '@rollup/rollup-win32-arm64-msvc@4.60.2': - resolution: {integrity: sha512-NCYhOotpgWZ5kdxCZsv6Iudx0wX8980Q/oW4pNFNihpBKsDbEA1zpkfxJGC0yugsUuyDZ7gL37dbzwhR0VI7pQ==} + '@rollup/rollup-win32-arm64-msvc@4.62.2': + resolution: {integrity: sha512-mmF4AY1i0hG/bLWUctUq59gtmgaSIRa3cu/A3JFRp/sCNEme2bgDEiDS22P9FbnJB8NJNF4jPJiSP5RHQpUTDg==} cpu: [arm64] os: [win32] - '@rollup/rollup-win32-ia32-msvc@4.60.2': - resolution: {integrity: sha512-RXsaOqXxfoUBQoOgvmmijVxJnW2IGB0eoMO7F8FAjaj0UTywUO/luSqimWBJn04WNgUkeNhh7fs7pESXajWmkg==} + '@rollup/rollup-win32-ia32-msvc@4.62.2': + resolution: {integrity: sha512-DZgkknc6jhHrk46V25vbAM0zZkyP0nSDkJB8/dRkLTxv470dOmWDqGoEJl/9A0dFfS7yE3REOwNDxpHwSLSt0Q==} cpu: [ia32] os: [win32] - '@rollup/rollup-win32-x64-gnu@4.60.2': - resolution: {integrity: sha512-qdAzEULD+/hzObedtmV6iBpdL5TIbKVztGiK7O3/KYSf+HIzU257+MX1EXJcyIiDbMAqmbwaufcYPvyRryeZtA==} + '@rollup/rollup-win32-x64-gnu@4.62.2': + resolution: {integrity: sha512-T6xr6ucWSFto+VGajA8YH26LdpHRuP4YLHEKAtCWvJDOlnmWcDZVCI2Jmjr+IFHDlt2zRaTAKE4tfjTaWLgJBg==} cpu: [x64] os: [win32] - '@rollup/rollup-win32-x64-msvc@4.60.2': - resolution: {integrity: sha512-Nd/SgG27WoA9e+/TdK74KnHz852TLa94ovOYySo/yMPuTmpckK/jIF2jSwS3g7ELSKXK13/cVdmg1Z/DaCWKxA==} + '@rollup/rollup-win32-x64-msvc@4.62.2': + resolution: {integrity: sha512-BfzEnDJOt9T8M989/lA37EcJgat01wLRnoi5dQf3QzOH7jzpqTAzdDbVfRljVr5r+jzKqpbHeyOfAaXxAd0PAA==} cpu: [x64] os: [win32] @@ -359,20 +313,17 @@ packages: '@types/deep-eql@4.0.2': resolution: {integrity: sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==} - '@types/estree@1.0.8': - resolution: {integrity: sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==} - - '@types/node@22.19.17': - resolution: {integrity: sha512-wGdMcf+vPYM6jikpS/qhg6WiqSV/OhG+jeeHT/KlVqxYfD40iYJf9/AE1uQxVWFvU7MipKRkRv8NSHiCGgPr8Q==} + '@types/estree@1.0.9': + resolution: {integrity: sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==} - '@types/ws@8.18.1': - resolution: {integrity: sha512-ThVF6DCVhA8kUGy+aazFQ4kXQ7E1Ty7A3ypFOe0IcJV8O/M511G99AW24irKrW56Wt44yG9+ij8FaqoBGkuBXg==} + '@types/node@22.20.1': + resolution: {integrity: sha512-EANqOCF9QFyra+4pfxUcX9STKJpCLjMbObVzljIJomAWSnuSIEAvyzEU53GaajbXJEgdh0iEcPL+DGvpUd4k1Q==} - '@vitest/expect@3.2.4': - resolution: {integrity: sha512-Io0yyORnB6sikFlt8QW5K7slY4OjqNX9jmJQ02QDda8lyM6B5oNgVWoSoKPac8/kgnCUzuHQKrSLtu/uOqqrig==} + '@vitest/expect@3.2.7': + resolution: {integrity: sha512-E8eBXaKibuvH2pSZErOjdVb5vF4PbKYcrnluBTYxEk1l/VhhwZg1kZQsdtjq+CsF5CFydf2Rdkz7jDHKSisi3w==} - '@vitest/mocker@3.2.4': - resolution: {integrity: sha512-46ryTE9RZO/rfDd7pEqFl7etuyzekzEhUbTW3BvmeO/BcCMEgq59BKhek3dXDWgAj4oMK6OZi+vRr1wPW6qjEQ==} + '@vitest/mocker@3.2.7': + resolution: {integrity: sha512-Trr0hYO9CM3Wj6ksWHRhK9IZpIY6wTMO5u/MqXurMxT57sWBaOPEtP3Oq60ihZuh5JsiagKfz95OcxdEP6dBrA==} peerDependencies: msw: ^2.4.9 vite: ^5.0.0 || ^6.0.0 || ^7.0.0-0 @@ -382,20 +333,20 @@ packages: vite: optional: true - '@vitest/pretty-format@3.2.4': - resolution: {integrity: sha512-IVNZik8IVRJRTr9fxlitMKeJeXFFFN0JaB9PHPGQ8NKQbGpfjlTx9zO4RefN8gp7eqjNy8nyK3NZmBzOPeIxtA==} + '@vitest/pretty-format@3.2.7': + resolution: {integrity: sha512-KUHlwqVu0sRlhCdyPdQ/wBoTfRahjUky1MubOmYw9fWfIZy1gNoHpuaaQBPAaMaVYdQYHJLurzj8ECCj5OwTqA==} - '@vitest/runner@3.2.4': - resolution: {integrity: sha512-oukfKT9Mk41LreEW09vt45f8wx7DordoWUZMYdY/cyAk7w5TWkTRCNZYF7sX7n2wB7jyGAl74OxgwhPgKaqDMQ==} + '@vitest/runner@3.2.7': + resolution: {integrity: sha512-sB9y4ovltoQP+WaUPwmSxO9WIg9Ig694Di5PalVPsYHklAdE027mehpWF2SQSVq+k6sFgaivbTjTJwZLSHbedA==} - '@vitest/snapshot@3.2.4': - resolution: {integrity: sha512-dEYtS7qQP2CjU27QBC5oUOxLE/v5eLkGqPE0ZKEIDGMs4vKWe7IjgLOeauHsR0D5YuuycGRO5oSRXnwnmA78fQ==} + '@vitest/snapshot@3.2.7': + resolution: {integrity: sha512-7C+MwShwtBSI5Buwoyg3s/iY1eHL9PKAf+O1wVh/TdnjXUtkoL/9YQtre90i4MtNXM6edP1wJ2zOBpfCyhIS7g==} - '@vitest/spy@3.2.4': - resolution: {integrity: sha512-vAfasCOe6AIK70iP5UD11Ac4siNUNJ9i/9PZ3NKx07sG6sUxeag1LWdNrMWeKKYBLlzuK+Gn65Yd5nyL6ds+nw==} + '@vitest/spy@3.2.7': + resolution: {integrity: sha512-Q2eQGI6d2L/hBtZ0qNuKcAGid68XK6cv1xsoaIma6PaJhHPoqcEJhYpXZ/5myCMqkNgtP6UKuBhbc0nHKnrkuQ==} - '@vitest/utils@3.2.4': - resolution: {integrity: sha512-fB2V0JFrQSMsCo9HiSq3Ezpdv4iYaXRG1Sx8edX3MwxfyNn83mKiGzOcH+Fkxt4MHxr3y42fQi1oeAInqgX2QA==} + '@vitest/utils@3.2.7': + resolution: {integrity: sha512-x6BDOd7dyo3PFLY3I9/HJ25X/6OurhGXk2/B9gOZNPF7XDVjeBK4k01lQE5uvDpbuheErh91qYuE1E2OEjK3Rw==} assertion-error@2.0.1: resolution: {integrity: sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==} @@ -429,16 +380,16 @@ packages: es-module-lexer@1.7.0: resolution: {integrity: sha512-jEQoCwk8hyb2AZziIOLhDqpm5+2ww5uIE6lkO/6jcOCusfk6LhMHpXXfBLXTZ7Ydyt0j4VoUQv6uGNYbdW+kBA==} - esbuild@0.27.7: - resolution: {integrity: sha512-IxpibTjyVnmrIQo5aqNpCgoACA/dTKLTlhMHihVHhdkxKyPO1uBBthumT0rdHmcsk9uMonIWS0m4FljWzILh3w==} + esbuild@0.28.1: + resolution: {integrity: sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==} engines: {node: '>=18'} hasBin: true estree-walker@3.0.3: resolution: {integrity: sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==} - expect-type@1.3.0: - resolution: {integrity: sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==} + expect-type@1.4.0: + resolution: {integrity: sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==} engines: {node: '>=12.0.0'} fdir@6.5.0: @@ -467,8 +418,8 @@ packages: ms@2.1.3: resolution: {integrity: sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==} - nanoid@3.3.11: - resolution: {integrity: sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==} + nanoid@3.3.16: + resolution: {integrity: sha512-bzlKTyNJ7+LdGIIwy8ijFpIqEQIvafahV7eYykJ8Cvh42EdJeODoJ6gUJXpQJvej1BddH8OqTXZNE/KfbWAu8Q==} engines: {node: ^10 || ^12 || ^13.7 || ^14 || >=15.0.1} hasBin: true @@ -482,16 +433,16 @@ packages: picocolors@1.1.1: resolution: {integrity: sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==} - picomatch@4.0.4: - resolution: {integrity: sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==} + picomatch@4.0.5: + resolution: {integrity: sha512-RvwwcruNjI1ncT5xRakeyS9Lf8lcItv34KD+aif+VH9kduAyfYBipGh12274xtenIPZ119/R9BdTBa8gAwSh0A==} engines: {node: '>=12'} - postcss@8.5.10: - resolution: {integrity: sha512-pMMHxBOZKFU6HgAZ4eyGnwXF/EvPGGqUr0MnZ5+99485wwW41kW91A4LOGxSHhgugZmSChL5AlElNdwlNgcnLQ==} + postcss@8.5.18: + resolution: {integrity: sha512-xdB1oSLHbz1vRWgCDalrCqEFTWzFlhqFC5tIHLMOSUIjhm3XXQ1qrFy8S/ESr1JYRRXqM3c1QFiMZUJdUTqyMQ==} engines: {node: ^10 || ^12 || >=14} - rollup@4.60.2: - resolution: {integrity: sha512-J9qZyW++QK/09NyN/zeO0dG/1GdGfyp9lV8ajHnRVLfo/uFsbji5mHnDgn/qYdUHyCkM2N+8VyspgZclfAh0eQ==} + rollup@4.62.2: + resolution: {integrity: sha512-RFnrW4lhXA3s3eqHDZvN654g8OTjzRfqpIRJYczCGB6HzphckVAi/Qh4tbPUbRuDi7s1Llv8g/NspLkttY3gTA==} engines: {node: '>=18.0.0', npm: '>=8.0.0'} hasBin: true @@ -517,8 +468,8 @@ packages: tinyexec@0.3.2: resolution: {integrity: sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==} - tinyglobby@0.2.16: - resolution: {integrity: sha512-pn99VhoACYR8nFHhxqix+uvsbXineAasWm5ojXoN8xEwK5Kd3/TrhNn1wByuD52UxWRLy8pu+kRMniEi6Eq9Zg==} + tinyglobby@0.2.17: + resolution: {integrity: sha512-wXR/dYpcqKmfWpEdZjiKJOwCNFndD0DMnrW/cYjVGttEkBfVgcLFHoNrlj47mjOVic9yyNu65alsgF4NQyTa2g==} engines: {node: '>=12.0.0'} tinypool@1.1.1: @@ -546,8 +497,8 @@ packages: engines: {node: ^18.0.0 || ^20.0.0 || >=22.0.0} hasBin: true - vite@7.3.2: - resolution: {integrity: sha512-Bby3NOsna2jsjfLVOHKes8sGwgl4TT0E6vvpYgnAYDIF/tie7MRaFthmKuHx1NSXjiTueXH3do80FMQgvEktRg==} + vite@7.3.6: + resolution: {integrity: sha512-4XP60spRGjSZFf1qYH+dJIkK2znL3zQfl9KkOV9MkkRR/3Dls0dxaBsQPTloEc5BLXWPL9vsOxopxyKoMmDueg==} engines: {node: ^20.19.0 || >=22.12.0} hasBin: true peerDependencies: @@ -586,16 +537,16 @@ packages: yaml: optional: true - vitest@3.2.4: - resolution: {integrity: sha512-LUCP5ev3GURDysTWiP47wRRUpLKMOfPh+yKTx3kVIEiu5KOMeqzpnYNsKyOoVrULivR8tLcks4+lga33Whn90A==} + vitest@3.2.7: + resolution: {integrity: sha512-KrxIJ62Fd89gfysR4WotlgZABiz2dqFPgqGzX7s+CwsqLFomRH7777ZcrOD6+WVAh7khPQP41A+BKbpcJFrdEg==} engines: {node: ^18.0.0 || ^20.0.0 || >=22.0.0} hasBin: true peerDependencies: '@edge-runtime/vm': '*' '@types/debug': ^4.1.12 '@types/node': ^18.0.0 || ^20.0.0 || >=22.0.0 - '@vitest/browser': 3.2.4 - '@vitest/ui': 3.2.4 + '@vitest/browser': 3.2.7 + '@vitest/ui': 3.2.7 happy-dom: '*' jsdom: '*' peerDependenciesMeta: @@ -633,159 +584,159 @@ packages: snapshots: - '@esbuild/aix-ppc64@0.27.7': + '@esbuild/aix-ppc64@0.28.1': optional: true - '@esbuild/android-arm64@0.27.7': + '@esbuild/android-arm64@0.28.1': optional: true - '@esbuild/android-arm@0.27.7': + '@esbuild/android-arm@0.28.1': optional: true - '@esbuild/android-x64@0.27.7': + '@esbuild/android-x64@0.28.1': optional: true - '@esbuild/darwin-arm64@0.27.7': + '@esbuild/darwin-arm64@0.28.1': optional: true - '@esbuild/darwin-x64@0.27.7': + '@esbuild/darwin-x64@0.28.1': optional: true - '@esbuild/freebsd-arm64@0.27.7': + '@esbuild/freebsd-arm64@0.28.1': optional: true - '@esbuild/freebsd-x64@0.27.7': + '@esbuild/freebsd-x64@0.28.1': optional: true - '@esbuild/linux-arm64@0.27.7': + '@esbuild/linux-arm64@0.28.1': optional: true - '@esbuild/linux-arm@0.27.7': + '@esbuild/linux-arm@0.28.1': optional: true - '@esbuild/linux-ia32@0.27.7': + '@esbuild/linux-ia32@0.28.1': optional: true - '@esbuild/linux-loong64@0.27.7': + '@esbuild/linux-loong64@0.28.1': optional: true - '@esbuild/linux-mips64el@0.27.7': + '@esbuild/linux-mips64el@0.28.1': optional: true - '@esbuild/linux-ppc64@0.27.7': + '@esbuild/linux-ppc64@0.28.1': optional: true - '@esbuild/linux-riscv64@0.27.7': + '@esbuild/linux-riscv64@0.28.1': optional: true - '@esbuild/linux-s390x@0.27.7': + '@esbuild/linux-s390x@0.28.1': optional: true - '@esbuild/linux-x64@0.27.7': + '@esbuild/linux-x64@0.28.1': optional: true - '@esbuild/netbsd-arm64@0.27.7': + '@esbuild/netbsd-arm64@0.28.1': optional: true - '@esbuild/netbsd-x64@0.27.7': + '@esbuild/netbsd-x64@0.28.1': optional: true - '@esbuild/openbsd-arm64@0.27.7': + '@esbuild/openbsd-arm64@0.28.1': optional: true - '@esbuild/openbsd-x64@0.27.7': + '@esbuild/openbsd-x64@0.28.1': optional: true - '@esbuild/openharmony-arm64@0.27.7': + '@esbuild/openharmony-arm64@0.28.1': optional: true - '@esbuild/sunos-x64@0.27.7': + '@esbuild/sunos-x64@0.28.1': optional: true - '@esbuild/win32-arm64@0.27.7': + '@esbuild/win32-arm64@0.28.1': optional: true - '@esbuild/win32-ia32@0.27.7': + '@esbuild/win32-ia32@0.28.1': optional: true - '@esbuild/win32-x64@0.27.7': + '@esbuild/win32-x64@0.28.1': optional: true '@jridgewell/sourcemap-codec@1.5.5': {} - '@rollup/rollup-android-arm-eabi@4.60.2': + '@rollup/rollup-android-arm-eabi@4.62.2': optional: true - '@rollup/rollup-android-arm64@4.60.2': + '@rollup/rollup-android-arm64@4.62.2': optional: true - '@rollup/rollup-darwin-arm64@4.60.2': + '@rollup/rollup-darwin-arm64@4.62.2': optional: true - '@rollup/rollup-darwin-x64@4.60.2': + '@rollup/rollup-darwin-x64@4.62.2': optional: true - '@rollup/rollup-freebsd-arm64@4.60.2': + '@rollup/rollup-freebsd-arm64@4.62.2': optional: true - '@rollup/rollup-freebsd-x64@4.60.2': + '@rollup/rollup-freebsd-x64@4.62.2': optional: true - '@rollup/rollup-linux-arm-gnueabihf@4.60.2': + '@rollup/rollup-linux-arm-gnueabihf@4.62.2': optional: true - '@rollup/rollup-linux-arm-musleabihf@4.60.2': + '@rollup/rollup-linux-arm-musleabihf@4.62.2': optional: true - '@rollup/rollup-linux-arm64-gnu@4.60.2': + '@rollup/rollup-linux-arm64-gnu@4.62.2': optional: true - '@rollup/rollup-linux-arm64-musl@4.60.2': + '@rollup/rollup-linux-arm64-musl@4.62.2': optional: true - '@rollup/rollup-linux-loong64-gnu@4.60.2': + '@rollup/rollup-linux-loong64-gnu@4.62.2': optional: true - '@rollup/rollup-linux-loong64-musl@4.60.2': + '@rollup/rollup-linux-loong64-musl@4.62.2': optional: true - '@rollup/rollup-linux-ppc64-gnu@4.60.2': + '@rollup/rollup-linux-ppc64-gnu@4.62.2': optional: true - '@rollup/rollup-linux-ppc64-musl@4.60.2': + '@rollup/rollup-linux-ppc64-musl@4.62.2': optional: true - '@rollup/rollup-linux-riscv64-gnu@4.60.2': + '@rollup/rollup-linux-riscv64-gnu@4.62.2': optional: true - '@rollup/rollup-linux-riscv64-musl@4.60.2': + '@rollup/rollup-linux-riscv64-musl@4.62.2': optional: true - '@rollup/rollup-linux-s390x-gnu@4.60.2': + '@rollup/rollup-linux-s390x-gnu@4.62.2': optional: true - '@rollup/rollup-linux-x64-gnu@4.60.2': + '@rollup/rollup-linux-x64-gnu@4.62.2': optional: true - '@rollup/rollup-linux-x64-musl@4.60.2': + '@rollup/rollup-linux-x64-musl@4.62.2': optional: true - '@rollup/rollup-openbsd-x64@4.60.2': + '@rollup/rollup-openbsd-x64@4.62.2': optional: true - '@rollup/rollup-openharmony-arm64@4.60.2': + '@rollup/rollup-openharmony-arm64@4.62.2': optional: true - '@rollup/rollup-win32-arm64-msvc@4.60.2': + '@rollup/rollup-win32-arm64-msvc@4.62.2': optional: true - '@rollup/rollup-win32-ia32-msvc@4.60.2': + '@rollup/rollup-win32-ia32-msvc@4.62.2': optional: true - '@rollup/rollup-win32-x64-gnu@4.60.2': + '@rollup/rollup-win32-x64-gnu@4.62.2': optional: true - '@rollup/rollup-win32-x64-msvc@4.60.2': + '@rollup/rollup-win32-x64-msvc@4.62.2': optional: true '@types/chai@5.2.3': @@ -795,55 +746,51 @@ snapshots: '@types/deep-eql@4.0.2': {} - '@types/estree@1.0.8': {} + '@types/estree@1.0.9': {} - '@types/node@22.19.17': + '@types/node@22.20.1': dependencies: undici-types: 6.21.0 - '@types/ws@8.18.1': - dependencies: - '@types/node': 22.19.17 - - '@vitest/expect@3.2.4': + '@vitest/expect@3.2.7': dependencies: '@types/chai': 5.2.3 - '@vitest/spy': 3.2.4 - '@vitest/utils': 3.2.4 + '@vitest/spy': 3.2.7 + '@vitest/utils': 3.2.7 chai: 5.3.3 tinyrainbow: 2.0.0 - '@vitest/mocker@3.2.4(vite@7.3.2(@types/node@22.19.17))': + '@vitest/mocker@3.2.7(vite@7.3.6(@types/node@22.20.1))': dependencies: - '@vitest/spy': 3.2.4 + '@vitest/spy': 3.2.7 estree-walker: 3.0.3 magic-string: 0.30.21 optionalDependencies: - vite: 7.3.2(@types/node@22.19.17) + vite: 7.3.6(@types/node@22.20.1) - '@vitest/pretty-format@3.2.4': + '@vitest/pretty-format@3.2.7': dependencies: tinyrainbow: 2.0.0 - '@vitest/runner@3.2.4': + '@vitest/runner@3.2.7': dependencies: - '@vitest/utils': 3.2.4 + '@vitest/utils': 3.2.7 pathe: 2.0.3 strip-literal: 3.1.0 - '@vitest/snapshot@3.2.4': + '@vitest/snapshot@3.2.7': dependencies: - '@vitest/pretty-format': 3.2.4 + '@vitest/pretty-format': 3.2.7 magic-string: 0.30.21 pathe: 2.0.3 - '@vitest/spy@3.2.4': + '@vitest/spy@3.2.7': dependencies: tinyspy: 4.0.4 - '@vitest/utils@3.2.4': + '@vitest/utils@3.2.7': dependencies: - '@vitest/pretty-format': 3.2.4 + '@vitest/pretty-format': 3.2.7 loupe: 3.2.1 tinyrainbow: 2.0.0 @@ -869,44 +816,44 @@ snapshots: es-module-lexer@1.7.0: {} - esbuild@0.27.7: + esbuild@0.28.1: optionalDependencies: - '@esbuild/aix-ppc64': 0.27.7 - '@esbuild/android-arm': 0.27.7 - '@esbuild/android-arm64': 0.27.7 - '@esbuild/android-x64': 0.27.7 - '@esbuild/darwin-arm64': 0.27.7 - '@esbuild/darwin-x64': 0.27.7 - '@esbuild/freebsd-arm64': 0.27.7 - '@esbuild/freebsd-x64': 0.27.7 - '@esbuild/linux-arm': 0.27.7 - '@esbuild/linux-arm64': 0.27.7 - '@esbuild/linux-ia32': 0.27.7 - '@esbuild/linux-loong64': 0.27.7 - '@esbuild/linux-mips64el': 0.27.7 - '@esbuild/linux-ppc64': 0.27.7 - '@esbuild/linux-riscv64': 0.27.7 - '@esbuild/linux-s390x': 0.27.7 - '@esbuild/linux-x64': 0.27.7 - '@esbuild/netbsd-arm64': 0.27.7 - '@esbuild/netbsd-x64': 0.27.7 - '@esbuild/openbsd-arm64': 0.27.7 - '@esbuild/openbsd-x64': 0.27.7 - '@esbuild/openharmony-arm64': 0.27.7 - '@esbuild/sunos-x64': 0.27.7 - '@esbuild/win32-arm64': 0.27.7 - '@esbuild/win32-ia32': 0.27.7 - '@esbuild/win32-x64': 0.27.7 + '@esbuild/aix-ppc64': 0.28.1 + '@esbuild/android-arm': 0.28.1 + '@esbuild/android-arm64': 0.28.1 + '@esbuild/android-x64': 0.28.1 + '@esbuild/darwin-arm64': 0.28.1 + '@esbuild/darwin-x64': 0.28.1 + '@esbuild/freebsd-arm64': 0.28.1 + '@esbuild/freebsd-x64': 0.28.1 + '@esbuild/linux-arm': 0.28.1 + '@esbuild/linux-arm64': 0.28.1 + '@esbuild/linux-ia32': 0.28.1 + '@esbuild/linux-loong64': 0.28.1 + '@esbuild/linux-mips64el': 0.28.1 + '@esbuild/linux-ppc64': 0.28.1 + '@esbuild/linux-riscv64': 0.28.1 + '@esbuild/linux-s390x': 0.28.1 + '@esbuild/linux-x64': 0.28.1 + '@esbuild/netbsd-arm64': 0.28.1 + '@esbuild/netbsd-x64': 0.28.1 + '@esbuild/openbsd-arm64': 0.28.1 + '@esbuild/openbsd-x64': 0.28.1 + '@esbuild/openharmony-arm64': 0.28.1 + '@esbuild/sunos-x64': 0.28.1 + '@esbuild/win32-arm64': 0.28.1 + '@esbuild/win32-ia32': 0.28.1 + '@esbuild/win32-x64': 0.28.1 estree-walker@3.0.3: dependencies: - '@types/estree': 1.0.8 + '@types/estree': 1.0.9 - expect-type@1.3.0: {} + expect-type@1.4.0: {} - fdir@6.5.0(picomatch@4.0.4): + fdir@6.5.0(picomatch@4.0.5): optionalDependencies: - picomatch: 4.0.4 + picomatch: 4.0.5 fsevents@2.3.3: optional: true @@ -921,7 +868,7 @@ snapshots: ms@2.1.3: {} - nanoid@3.3.11: {} + nanoid@3.3.16: {} pathe@2.0.3: {} @@ -929,43 +876,43 @@ snapshots: picocolors@1.1.1: {} - picomatch@4.0.4: {} + picomatch@4.0.5: {} - postcss@8.5.10: + postcss@8.5.18: dependencies: - nanoid: 3.3.11 + nanoid: 3.3.16 picocolors: 1.1.1 source-map-js: 1.2.1 - rollup@4.60.2: + rollup@4.62.2: dependencies: - '@types/estree': 1.0.8 + '@types/estree': 1.0.9 optionalDependencies: - '@rollup/rollup-android-arm-eabi': 4.60.2 - '@rollup/rollup-android-arm64': 4.60.2 - '@rollup/rollup-darwin-arm64': 4.60.2 - '@rollup/rollup-darwin-x64': 4.60.2 - '@rollup/rollup-freebsd-arm64': 4.60.2 - '@rollup/rollup-freebsd-x64': 4.60.2 - '@rollup/rollup-linux-arm-gnueabihf': 4.60.2 - '@rollup/rollup-linux-arm-musleabihf': 4.60.2 - '@rollup/rollup-linux-arm64-gnu': 4.60.2 - '@rollup/rollup-linux-arm64-musl': 4.60.2 - '@rollup/rollup-linux-loong64-gnu': 4.60.2 - '@rollup/rollup-linux-loong64-musl': 4.60.2 - '@rollup/rollup-linux-ppc64-gnu': 4.60.2 - '@rollup/rollup-linux-ppc64-musl': 4.60.2 - '@rollup/rollup-linux-riscv64-gnu': 4.60.2 - '@rollup/rollup-linux-riscv64-musl': 4.60.2 - '@rollup/rollup-linux-s390x-gnu': 4.60.2 - '@rollup/rollup-linux-x64-gnu': 4.60.2 - '@rollup/rollup-linux-x64-musl': 4.60.2 - '@rollup/rollup-openbsd-x64': 4.60.2 - '@rollup/rollup-openharmony-arm64': 4.60.2 - '@rollup/rollup-win32-arm64-msvc': 4.60.2 - '@rollup/rollup-win32-ia32-msvc': 4.60.2 - '@rollup/rollup-win32-x64-gnu': 4.60.2 - '@rollup/rollup-win32-x64-msvc': 4.60.2 + '@rollup/rollup-android-arm-eabi': 4.62.2 + '@rollup/rollup-android-arm64': 4.62.2 + '@rollup/rollup-darwin-arm64': 4.62.2 + '@rollup/rollup-darwin-x64': 4.62.2 + '@rollup/rollup-freebsd-arm64': 4.62.2 + '@rollup/rollup-freebsd-x64': 4.62.2 + '@rollup/rollup-linux-arm-gnueabihf': 4.62.2 + '@rollup/rollup-linux-arm-musleabihf': 4.62.2 + '@rollup/rollup-linux-arm64-gnu': 4.62.2 + '@rollup/rollup-linux-arm64-musl': 4.62.2 + '@rollup/rollup-linux-loong64-gnu': 4.62.2 + '@rollup/rollup-linux-loong64-musl': 4.62.2 + '@rollup/rollup-linux-ppc64-gnu': 4.62.2 + '@rollup/rollup-linux-ppc64-musl': 4.62.2 + '@rollup/rollup-linux-riscv64-gnu': 4.62.2 + '@rollup/rollup-linux-riscv64-musl': 4.62.2 + '@rollup/rollup-linux-s390x-gnu': 4.62.2 + '@rollup/rollup-linux-x64-gnu': 4.62.2 + '@rollup/rollup-linux-x64-musl': 4.62.2 + '@rollup/rollup-openbsd-x64': 4.62.2 + '@rollup/rollup-openharmony-arm64': 4.62.2 + '@rollup/rollup-win32-arm64-msvc': 4.62.2 + '@rollup/rollup-win32-ia32-msvc': 4.62.2 + '@rollup/rollup-win32-x64-gnu': 4.62.2 + '@rollup/rollup-win32-x64-msvc': 4.62.2 fsevents: 2.3.3 siginfo@2.0.0: {} @@ -984,10 +931,10 @@ snapshots: tinyexec@0.3.2: {} - tinyglobby@0.2.16: + tinyglobby@0.2.17: dependencies: - fdir: 6.5.0(picomatch@4.0.4) - picomatch: 4.0.4 + fdir: 6.5.0(picomatch@4.0.5) + picomatch: 4.0.5 tinypool@1.1.1: {} @@ -999,13 +946,13 @@ snapshots: undici-types@6.21.0: {} - vite-node@3.2.4(@types/node@22.19.17): + vite-node@3.2.4(@types/node@22.20.1): dependencies: cac: 6.7.14 debug: 4.4.3 es-module-lexer: 1.7.0 pathe: 2.0.3 - vite: 7.3.2(@types/node@22.19.17) + vite: 7.3.6(@types/node@22.20.1) transitivePeerDependencies: - '@types/node' - jiti @@ -1020,45 +967,45 @@ snapshots: - tsx - yaml - vite@7.3.2(@types/node@22.19.17): + vite@7.3.6(@types/node@22.20.1): dependencies: - esbuild: 0.27.7 - fdir: 6.5.0(picomatch@4.0.4) - picomatch: 4.0.4 - postcss: 8.5.10 - rollup: 4.60.2 - tinyglobby: 0.2.16 + esbuild: 0.28.1 + fdir: 6.5.0(picomatch@4.0.5) + picomatch: 4.0.5 + postcss: 8.5.18 + rollup: 4.62.2 + tinyglobby: 0.2.17 optionalDependencies: - '@types/node': 22.19.17 + '@types/node': 22.20.1 fsevents: 2.3.3 - vitest@3.2.4(@types/node@22.19.17): + vitest@3.2.7(@types/node@22.20.1): dependencies: '@types/chai': 5.2.3 - '@vitest/expect': 3.2.4 - '@vitest/mocker': 3.2.4(vite@7.3.2(@types/node@22.19.17)) - '@vitest/pretty-format': 3.2.4 - '@vitest/runner': 3.2.4 - '@vitest/snapshot': 3.2.4 - '@vitest/spy': 3.2.4 - '@vitest/utils': 3.2.4 + '@vitest/expect': 3.2.7 + '@vitest/mocker': 3.2.7(vite@7.3.6(@types/node@22.20.1)) + '@vitest/pretty-format': 3.2.7 + '@vitest/runner': 3.2.7 + '@vitest/snapshot': 3.2.7 + '@vitest/spy': 3.2.7 + '@vitest/utils': 3.2.7 chai: 5.3.3 debug: 4.4.3 - expect-type: 1.3.0 + expect-type: 1.4.0 magic-string: 0.30.21 pathe: 2.0.3 - picomatch: 4.0.4 + picomatch: 4.0.5 std-env: 3.10.0 tinybench: 2.9.0 tinyexec: 0.3.2 - tinyglobby: 0.2.16 + tinyglobby: 0.2.17 tinypool: 1.1.1 tinyrainbow: 2.0.0 - vite: 7.3.2(@types/node@22.19.17) - vite-node: 3.2.4(@types/node@22.19.17) + vite: 7.3.6(@types/node@22.20.1) + vite-node: 3.2.4(@types/node@22.20.1) why-is-node-running: 2.3.0 optionalDependencies: - '@types/node': 22.19.17 + '@types/node': 22.20.1 transitivePeerDependencies: - jiti - less diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml deleted file mode 100644 index dee51e9..0000000 --- a/pnpm-workspace.yaml +++ /dev/null @@ -1,2 +0,0 @@ -packages: - - "packages/*" diff --git a/scripts/check-readme-examples.mjs b/scripts/check-readme-examples.mjs index 78ec328..47a8123 100644 --- a/scripts/check-readme-examples.mjs +++ b/scripts/check-readme-examples.mjs @@ -1,11 +1,11 @@ #!/usr/bin/env node -// Executes every fenced example in packages/*/README.md against the built -// packages. The package READMEs are the npm landing pages; their examples -// are contract, not illustration — this check fails `pnpm verify` (and CI) -// when one stops running. +// Executes every fenced example in README.md against the built package. The +// README is the npm landing page; its examples are contract, not +// illustration — this check fails `pnpm verify` (and CI) when one stops +// running. // -// The contract lives in the READMEs themselves. A fenced ```ts or ```bash -// block runs as-written unless an HTML comment directly above it declares +// The contract lives in the README itself. A fenced ```ts or ```bash block +// runs as-written unless an HTML comment directly above it declares // otherwise: // // @@ -13,20 +13,19 @@ // // There is no other configuration: a new example is checked by default. // -// ts blocks execute from inside their package directory — Node's package -// self-reference resolves the package's own name and its workspace deps -// exactly like an installed consumer — with cwd in a scratch dir so relative -// paths never touch the repo. bash blocks run with the `lync` command token -// rewritten to the workspace bin; `npm install` lines are skipped (noted), -// since installing is the reader's step, not the example's. +// ts blocks execute from inside the repo root — Node's package self-reference +// resolves "lync-core" and its subpaths exactly like an installed consumer — +// with cwd in a scratch dir so relative paths never touch the repo. bash +// blocks run with the `lync` command token rewritten to the workspace bin; +// `npm install` lines are skipped (noted), since installing is the reader's +// step, not the example's. import { execFileSync, spawn } from "node:child_process"; -import { mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { fileURLToPath } from "node:url"; const root = join(fileURLToPath(import.meta.url), "..", ".."); -const packagesDir = join(root, "packages"); const BLOCK_RE = /(?:\s*\n)?```(ts|bash)\n([\s\S]*?)```/g; @@ -50,14 +49,14 @@ function scratchDir(label) { return dir; } -function runTs(pkg, index, code, directive) { - const scriptPath = join(packagesDir, pkg, `.readme-example-${index}.tmp.mjs`); +function runTs(index, code, directive) { + const scriptPath = join(root, `.readme-example-${index}.tmp.mjs`); writeFileSync(scriptPath, code); - const cwd = scratchDir(pkg); + const cwd = scratchDir("ts"); try { if (directive.mode !== "daemon") { execFileSync(process.execPath, [scriptPath], { cwd, stdio: ["ignore", "pipe", "pipe"], timeout: 30_000 }); - console.log(`ok ${pkg} ts example ${index}`); + console.log(`ok ts example ${index}`); return Promise.resolve(); } return new Promise((resolve) => { @@ -65,7 +64,7 @@ function runTs(pkg, index, code, directive) { let out = ""; const timer = setTimeout(() => { child.kill("SIGKILL"); - console.error(`FAIL ${pkg} ts example ${index}: never printed ${JSON.stringify(directive.expect)}\n${out}`); + console.error(`FAIL ts example ${index}: never printed ${JSON.stringify(directive.expect)}\n${out}`); failures += 1; resolve(); }, 15_000); @@ -74,13 +73,13 @@ function runTs(pkg, index, code, directive) { if (out.includes(directive.expect)) { clearTimeout(timer); child.kill("SIGTERM"); - console.log(`ok ${pkg} ts example ${index} (daemon: saw ${JSON.stringify(directive.expect)})`); + console.log(`ok ts example ${index} (daemon: saw ${JSON.stringify(directive.expect)})`); resolve(); } }); }).finally(() => rmSync(scriptPath, { force: true })); } catch (error) { - console.error(`FAIL ${pkg} ts example ${index}:\n${error.stderr ?? error.message}`); + console.error(`FAIL ts example ${index}:\n${error.stderr ?? error.message}`); failures += 1; return Promise.resolve(); } finally { @@ -88,9 +87,9 @@ function runTs(pkg, index, code, directive) { } } -function runBash(pkg, index, code) { - const cwd = scratchDir(`${pkg}-bash`); - const bin = join(packagesDir, "cli", "bin", "lync.js"); +function runBash(index, code) { + const cwd = scratchDir("bash"); + const bin = join(root, "bin", "lync.js"); const lines = code .split("\n") .map((line) => line.trim()) @@ -98,7 +97,7 @@ function runBash(pkg, index, code) { let ran = 0; for (const line of lines) { if (line.startsWith("npm install")) { - console.log(`note ${pkg} bash example ${index}: install line left to the reader: ${line}`); + console.log(`note bash example ${index}: install line left to the reader: ${line}`); continue; } const cmd = line.replace(/(^|\| )lync /g, `$1node ${bin} `); @@ -106,43 +105,33 @@ function runBash(pkg, index, code) { execFileSync("bash", ["-c", cmd], { cwd, stdio: ["ignore", "pipe", "pipe"], timeout: 30_000 }); ran += 1; } catch (error) { - console.error(`FAIL ${pkg} bash example ${index} line: ${line}\n${error.stderr}`); + console.error(`FAIL bash example ${index} line: ${line}\n${error.stderr}`); failures += 1; return; } } - console.log(`ok ${pkg} bash example ${index} (${ran} command${ran === 1 ? "" : "s"})`); + console.log(`ok bash example ${index} (${ran} command${ran === 1 ? "" : "s"})`); } -const packages = readdirSync(packagesDir).sort(); -for (const pkg of packages) { - let markdown; - try { - markdown = readFileSync(join(packagesDir, pkg, "README.md"), "utf8"); - } catch { - console.error(`FAIL ${pkg}: no README.md — every published package needs a landing page`); - failures += 1; - continue; - } - let index = 0; - let match; - BLOCK_RE.lastIndex = 0; - while ((match = BLOCK_RE.exec(markdown)) !== null) { - const [, directiveText, lang, code] = match; - const directive = parseDirective(directiveText); - if (directive.mode === "fragment") { - console.log(`skip ${pkg} ${lang} example ${index} (${directive.reason})`); - index += 1; - continue; - } - if (lang === "ts") await runTs(pkg, index, code, directive); - else runBash(pkg, index, code); +const markdown = readFileSync(join(root, "README.md"), "utf8"); +let index = 0; +let match; +BLOCK_RE.lastIndex = 0; +while ((match = BLOCK_RE.exec(markdown)) !== null) { + const [, directiveText, lang, code] = match; + const directive = parseDirective(directiveText); + if (directive.mode === "fragment") { + console.log(`skip ${lang} example ${index} (${directive.reason})`); index += 1; + continue; } - if (index === 0) { - console.error(`FAIL ${pkg}: README has no fenced examples — a landing page shows the thing working`); - failures += 1; - } + if (lang === "ts") await runTs(index, code, directive); + else runBash(index, code); + index += 1; +} +if (index === 0) { + console.error("FAIL README.md has no fenced examples — a landing page shows the thing working"); + failures += 1; } for (const dir of scratchRoots) rmSync(dir, { recursive: true, force: true }); diff --git a/scripts/fresh-clone-smoke.sh b/scripts/fresh-clone-smoke.sh index 650ea58..adb944b 100755 --- a/scripts/fresh-clone-smoke.sh +++ b/scripts/fresh-clone-smoke.sh @@ -15,24 +15,24 @@ cd "$clone_dir" pnpm install --frozen-lockfile pnpm build -pnpm exec lync --help > help.txt +node bin/lync.js --help > help.txt -pnpm exec lync init demo.lync +node bin/lync.js init demo.lync first_id="$( printf '%s\n' '{"kind":"note/text","author":{"actor":"smoke"},"payload":{"text":"first line"}}' \ - | pnpm exec lync append demo.lync + | node bin/lync.js append demo.lync )" -pnpm exec lync view demo.lync --as transcript > transcript.json +node bin/lync.js view demo.lync --as transcript > transcript.json cp demo.lync a.lync second_id="$( printf '%s\n' '{"kind":"note/text","author":{"actor":"smoke"},"payload":{"text":"second line"}}' \ - | pnpm exec lync append demo.lync + | node bin/lync.js append demo.lync )" cp demo.lync b.lync cat a.lync b.lync > concatenated.lync -pnpm exec lync merge a.lync b.lync -o merged.lync -pnpm exec lync verify merged.lync > verify.txt +node bin/lync.js merge a.lync b.lync -o merged.lync +node bin/lync.js verify merged.lync > verify.txt test "$(grep -F -c "\"id\":\"$first_id\"" merged.lync)" = "1" test "$(grep -F -c "\"id\":\"$second_id\"" merged.lync)" = "1" diff --git a/packages/cli/src/bin.ts b/src/cli/bin.ts similarity index 100% rename from packages/cli/src/bin.ts rename to src/cli/bin.ts diff --git a/packages/cli/src/index.ts b/src/cli/index.ts similarity index 98% rename from packages/cli/src/index.ts rename to src/cli/index.ts index 6497373..36c4b5c 100644 --- a/packages/cli/src/index.ts +++ b/src/cli/index.ts @@ -1,11 +1,11 @@ -import { uuidv7 } from "lync-core/uuid"; +import { uuidv7 } from "../uuid.js"; import { appendFile, readFile, stat, writeFile } from "node:fs/promises"; import { parseLyncFiles, type LyncLineClass, type LyncLineDiagnostic, -} from "lync-core/events"; -import { lyncBranchTreeView as coreTreeView, lyncTranscriptView as coreTranscriptView } from "lync-core/views"; +} from "../events.js"; +import { lyncBranchTreeView as coreTreeView, lyncTranscriptView as coreTranscriptView } from "../views.js"; export interface LyncCliIO { stdout?: Pick; @@ -374,7 +374,7 @@ async function serveVerb( out: Pick, err: Pick, ): Promise { - const { startLyncServe } = await import("lync-server"); + const { startLyncServe } = await import("../relay/index.js"); const positional: string[] = []; let port: number | undefined; let token: string | undefined; diff --git a/packages/cli/src/sync.ts b/src/cli/sync.ts similarity index 85% rename from packages/cli/src/sync.ts rename to src/cli/sync.ts index ba24ef1..222da47 100644 --- a/packages/cli/src/sync.ts +++ b/src/cli/sync.ts @@ -1,8 +1,14 @@ import { appendFile, readFile, writeFile } from "node:fs/promises"; import { existsSync, watch } from "node:fs"; import { basename } from "node:path"; -import WebSocket from "ws"; -import { decodeFrame, encodeFrame, extractLineId, isCursor } from "lync-core/sync-protocol"; +import { decodeFrame, encodeFrame, extractLineId, isCursor } from "../sync-protocol.js"; + +// Sync rides Node's built-in WebSocket (global since Node 22, matching +// engines) — no dependency. It is an EventTarget, not an EventEmitter: +// listeners via addEventListener, payloads on MessageEvent.data, and no +// terminate(); the hard-abort timeout below settles the promise by rejection +// and then close() tears the socket down (aborting the handshake if still +// connecting). /** * `lync sync ` — one-shot convergence with a `lync serve` relay. @@ -65,6 +71,9 @@ export async function syncOnce(options: LyncSyncOptions): Promise((resolve, reject) => { const timeout = setTimeout(() => { + // The rejection is the hard abort — it settles the promise no matter + // what the socket does next. close() then aborts the handshake (if + // still connecting) or starts teardown; there is no terminate() here. reject(new Error(`lync sync: no 'live' from ${options.url} within ${options.timeoutMs ?? 15_000}ms`)); - socket.terminate(); + socket.close(); }, options.timeoutMs ?? 15_000); const stop = () => { @@ -109,19 +121,20 @@ export async function syncOnce(options: LyncSyncOptions): Promise { + socket.addEventListener("close", () => { clearTimeout(timeout); watcher?.close(); if (following) resolve(); }); - socket.on("error", (error) => { + socket.addEventListener("error", (event) => { clearTimeout(timeout); watcher?.close(); - reject(error); + const detail = (event as { message?: unknown }).message; + reject(new Error(`lync sync: socket error from ${options.url}${typeof detail === "string" ? `: ${detail}` : ""}`)); }); - socket.on("open", () => { + socket.addEventListener("open", () => { // Push before subscribing: the server handles our frames in order, so // every conflict or rejection for our own lines arrives before the // backlog and `live`, and the backlog then covers our accepted lines @@ -133,8 +146,9 @@ export async function syncOnce(options: LyncSyncOptions): Promise { - const frame = decodeFrame(raw.toString()); + socket.addEventListener("message", (event) => { + const raw = event.data; + const frame = decodeFrame(typeof raw === "string" ? raw : new TextDecoder().decode(raw as ArrayBuffer)); switch (frame.t) { case "ev": { const id = extractLineId(frame.line); diff --git a/packages/client/src/create.ts b/src/client/create.ts similarity index 96% rename from packages/client/src/create.ts rename to src/client/create.ts index 2b1d700..1afe92d 100644 --- a/packages/client/src/create.ts +++ b/src/client/create.ts @@ -11,8 +11,8 @@ import { turnRef, type LoomReference, type Looms, -} from "lync-core"; -import type { LoomIndexes } from "lync-index"; +} from "../index.js"; +import type { LoomIndexes } from "../indexes/index.js"; import type { LoomClient } from "./types.js"; export interface CreateLoomClientOptions< diff --git a/packages/client/src/index.ts b/src/client/index.ts similarity index 100% rename from packages/client/src/index.ts rename to src/client/index.ts diff --git a/packages/client/src/testing.ts b/src/client/testing.ts similarity index 85% rename from packages/client/src/testing.ts rename to src/client/testing.ts index 4e1e0f8..f84c780 100644 --- a/packages/client/src/testing.ts +++ b/src/client/testing.ts @@ -1,6 +1,6 @@ -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createLyncLooms, type LyncAuthor } from "lync-core/looms"; -import { createMemoryLoomIndexes } from "lync-index/memory"; +import { createMemoryEventStore } from "../memory-log.js"; +import { createLyncLooms, type LyncAuthor } from "../looms.js"; +import { createMemoryLoomIndexes } from "../indexes/memory.js"; import { createLoomClient } from "./create.js"; import type { LoomClient } from "./types.js"; diff --git a/packages/client/src/types.ts b/src/client/types.ts similarity index 95% rename from packages/client/src/types.ts rename to src/client/types.ts index c560206..6686a3b 100644 --- a/packages/client/src/types.ts +++ b/src/client/types.ts @@ -12,8 +12,8 @@ import type { encodeReference, decodeReference, parseReference, -} from "lync-core"; -import type { LoomIndex, LoomIndexes } from "lync-index"; +} from "../index.js"; +import type { LoomIndex, LoomIndexes } from "../indexes/index.js"; export type ReferenceHelpers = { loom: typeof loomRef; diff --git a/packages/core/src/errors.ts b/src/errors.ts similarity index 100% rename from packages/core/src/errors.ts rename to src/errors.ts diff --git a/packages/core/src/events.ts b/src/events.ts similarity index 100% rename from packages/core/src/events.ts rename to src/events.ts diff --git a/packages/core/src/file-log.ts b/src/file-log.ts similarity index 100% rename from packages/core/src/file-log.ts rename to src/file-log.ts diff --git a/packages/core/src/idb-log.ts b/src/idb-log.ts similarity index 100% rename from packages/core/src/idb-log.ts rename to src/idb-log.ts diff --git a/packages/core/src/index.ts b/src/index.ts similarity index 100% rename from packages/core/src/index.ts rename to src/index.ts diff --git a/packages/index/src/entries.ts b/src/indexes/entries.ts similarity index 90% rename from packages/index/src/entries.ts rename to src/indexes/entries.ts index f7405cb..b625c87 100644 --- a/packages/index/src/entries.ts +++ b/src/indexes/entries.ts @@ -1,4 +1,4 @@ -import type { LoomReference } from "lync-core"; +import type { LoomReference } from "../index.js"; import type { LoomIndex, LoomIndexEntry, diff --git a/packages/index/src/index.ts b/src/indexes/index.ts similarity index 100% rename from packages/index/src/index.ts rename to src/indexes/index.ts diff --git a/packages/index/src/memory.ts b/src/indexes/memory.ts similarity index 98% rename from packages/index/src/memory.ts rename to src/indexes/memory.ts index a4c0dc5..4ab48bb 100644 --- a/packages/index/src/memory.ts +++ b/src/indexes/memory.ts @@ -1,5 +1,5 @@ -import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "lync-core"; -import type { IndexId, LoomId, LoomReference } from "lync-core"; +import { LoomError, duplicateLoomId, loomRef, unknownIndex } from "../index.js"; +import type { IndexId, LoomId, LoomReference } from "../index.js"; import type { LoomIndex, LoomIndexEntry, diff --git a/packages/index/src/types.ts b/src/indexes/types.ts similarity index 97% rename from packages/index/src/types.ts rename to src/indexes/types.ts index a65448f..fcaf8a2 100644 --- a/packages/index/src/types.ts +++ b/src/indexes/types.ts @@ -1,4 +1,4 @@ -import type { IndexId, LoomId, LoomReference } from "lync-core"; +import type { IndexId, LoomId, LoomReference } from "../index.js"; export interface LoomIndexInfo { id: IndexId; diff --git a/packages/core/src/json.ts b/src/json.ts similarity index 100% rename from packages/core/src/json.ts rename to src/json.ts diff --git a/packages/core/src/looms.ts b/src/looms.ts similarity index 100% rename from packages/core/src/looms.ts rename to src/looms.ts diff --git a/packages/core/src/memory-log.ts b/src/memory-log.ts similarity index 100% rename from packages/core/src/memory-log.ts rename to src/memory-log.ts diff --git a/packages/core/src/memory.ts b/src/memory.ts similarity index 100% rename from packages/core/src/memory.ts rename to src/memory.ts diff --git a/packages/core/src/profiles/text-story.ts b/src/profiles/text-story.ts similarity index 100% rename from packages/core/src/profiles/text-story.ts rename to src/profiles/text-story.ts diff --git a/packages/core/src/references.ts b/src/references.ts similarity index 100% rename from packages/core/src/references.ts rename to src/references.ts diff --git a/packages/server/src/attach.ts b/src/relay/attach.ts similarity index 94% rename from packages/server/src/attach.ts rename to src/relay/attach.ts index 816924e..9386cf4 100644 --- a/packages/server/src/attach.ts +++ b/src/relay/attach.ts @@ -1,7 +1,6 @@ import type { IncomingMessage, Server } from "node:http"; import type { Duplex } from "node:stream"; -import type { WebSocket } from "ws"; -import { createLyncRelay, type LyncRelayOptions } from "./relay.js"; +import { createLyncRelay, type LyncRelayOptions, type LyncRelaySocket } from "./relay.js"; /** * Mount the relay on an existing Node HTTP server. Adds an `upgrade` listener @@ -27,7 +26,7 @@ export interface AttachedLyncServer { export function attachLyncServer(server: Server, options: AttachLyncServerOptions): AttachedLyncServer { const path = options.path ?? "/lync"; const log = options.log ?? ((message: string) => process.stderr.write(`${message}\n`)); - const live = new Set(); + const live = new Set(); const pingTimer = options.keepAliveInterval && options.keepAliveInterval > 0 diff --git a/packages/server/src/index.ts b/src/relay/index.ts similarity index 64% rename from packages/server/src/index.ts rename to src/relay/index.ts index e1869cc..22301ac 100644 --- a/packages/server/src/index.ts +++ b/src/relay/index.ts @@ -1,3 +1,3 @@ -export { createLyncRelay, type LyncRelay, type LyncRelayOptions } from "./relay.js"; +export { createLyncRelay, type LyncRelay, type LyncRelayOptions, type LyncRelaySocket } from "./relay.js"; export { startLyncServe, type LyncServeOptions, type LyncSyncServer } from "./serve.js"; export { attachLyncServer, type AttachLyncServerOptions, type AttachedLyncServer } from "./attach.js"; diff --git a/packages/server/src/relay.ts b/src/relay/relay.ts similarity index 83% rename from packages/server/src/relay.ts rename to src/relay/relay.ts index c26c150..a225476 100644 --- a/packages/server/src/relay.ts +++ b/src/relay/relay.ts @@ -1,10 +1,10 @@ import { appendFile, mkdir, readFile } from "node:fs/promises"; import { existsSync } from "node:fs"; +import { createRequire } from "node:module"; import { join } from "node:path"; import type { IncomingMessage } from "node:http"; import type { Duplex } from "node:stream"; -import { WebSocketServer, type WebSocket } from "ws"; -import { decodeFrame, encodeFrame, extractLineId, type SyncFrame } from "lync-core/sync-protocol"; +import { decodeFrame, encodeFrame, extractLineId, type SyncFrame } from "../sync-protocol.js"; /** * The lync line-sync relay — a dumb event-union relay, mountable on any Node @@ -19,6 +19,48 @@ import { decodeFrame, encodeFrame, extractLineId, type SyncFrame } from "lync-co * Presence is relayed, never stored. Nothing fails invisibly. */ +/** + * The members of a `ws` WebSocket the relay actually touches, as a structural + * type. `ws` ships no type declarations, so the relay's public surface must + * never name its types: a consumer without `ws` installed still typechecks. + */ +export interface LyncRelaySocket { + readyState: number; + readonly OPEN: number; + send(data: string): void; + ping(): void; + terminate(): void; + on(event: "message", listener: (data: { toString(): string }) => void): this; + on(event: "close", listener: () => void): this; + on(event: "error", listener: (error: unknown) => void): this; +} + +interface WebSocketServerLike { + handleUpgrade( + request: IncomingMessage, + socket: Duplex, + head: Buffer, + done: (ws: LyncRelaySocket) => void, + ): void; + close(done?: () => void): void; +} + +/** + * `ws` is acquired lazily, so the package truthfully has zero dependencies: + * only running a relay needs it, and the factory stays synchronous. Bundler + * note: this dynamic require is opaque to esbuild/webpack — mark "ws" as + * external when bundling a server that runs the relay. + */ +function acquireWebSocketServer(): new (options: { noServer: true }) => WebSocketServerLike { + try { + const require = createRequire(import.meta.url); + const ws = require("ws") as { WebSocketServer: new (options: { noServer: true }) => WebSocketServerLike }; + return ws.WebSocketServer; + } catch { + throw new Error("running a relay requires ws: npm install ws"); + } +} + export interface LyncRelayOptions { /** Directory of per-root append-only files. Created if absent. */ dir: string; @@ -30,7 +72,7 @@ export interface LyncRelayOptions { */ authenticate?: (request: IncomingMessage) => boolean | Promise; /** Called with each wired socket; for connection counting and keepalive. */ - onConnection?: (socket: WebSocket) => void; + onConnection?: (socket: LyncRelaySocket) => void; log?: (message: string) => void; } @@ -38,7 +80,7 @@ export interface LyncRelay { /** Handle an HTTP upgrade: authorize, upgrade, and wire the socket. */ handleUpgrade(request: IncomingMessage, socket: Duplex, head: Buffer): void; /** Wire a socket you upgraded yourself. */ - handleConnection(socket: WebSocket): void; + handleConnection(socket: LyncRelaySocket): void; /** Close all sockets and flush every pending append. */ close(): Promise; } @@ -48,7 +90,7 @@ interface Room { seq: number; lines: string[]; byId: Map; - subscribers: Set; + subscribers: Set; writeChain: Promise; recoveryNote?: string; } @@ -59,7 +101,8 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { const log = options.log ?? ((message: string) => process.stderr.write(`${message}\n`)); const dirReady = mkdir(options.dir, { recursive: true }).then(() => undefined); const rooms = new Map>(); - const sockets = new Set(); + const sockets = new Set(); + const WebSocketServer = acquireWebSocketServer(); const wss = new WebSocketServer({ noServer: true }); function handleUpgrade(request: IncomingMessage, socket: Duplex, head: Buffer): void { @@ -86,7 +129,7 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { socket.destroy(); } - function handleConnection(socket: WebSocket): void { + function handleConnection(socket: LyncRelaySocket): void { sockets.add(socket); options.onConnection?.(socket); const subscribed = new Set(); @@ -104,7 +147,7 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { socket.on("error", (error) => log(`[lync relay] socket error: ${String(error)}`)); } - async function handleFrame(socket: WebSocket, subscribed: Set, frame: SyncFrame): Promise { + async function handleFrame(socket: LyncRelaySocket, subscribed: Set, frame: SyncFrame): Promise { try { switch (frame.t) { case "err": @@ -229,7 +272,7 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { return attempt; } - function broadcast(room: Room, frame: SyncFrame, except?: WebSocket): void { + function broadcast(room: Room, frame: SyncFrame, except?: LyncRelaySocket): void { const encoded = encodeFrame(frame); for (const subscriber of room.subscribers) { if (subscriber === except) continue; @@ -237,11 +280,11 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { } } - function send(socket: WebSocket, frame: SyncFrame): void { + function send(socket: LyncRelaySocket, frame: SyncFrame): void { if (socket.readyState === socket.OPEN) socket.send(encodeFrame(frame)); } - async function detach(socket: WebSocket, subscribed: Set): Promise { + async function detach(socket: LyncRelaySocket, subscribed: Set): Promise { for (const root of subscribed) { const room = await rooms.get(root); room?.subscribers.delete(socket); diff --git a/packages/server/src/serve.ts b/src/relay/serve.ts similarity index 100% rename from packages/server/src/serve.ts rename to src/relay/serve.ts diff --git a/packages/core/src/sha256.ts b/src/sha256.ts similarity index 100% rename from packages/core/src/sha256.ts rename to src/sha256.ts diff --git a/packages/core/src/store.ts b/src/store.ts similarity index 100% rename from packages/core/src/store.ts rename to src/store.ts diff --git a/packages/core/src/sync-protocol.ts b/src/sync-protocol.ts similarity index 100% rename from packages/core/src/sync-protocol.ts rename to src/sync-protocol.ts diff --git a/packages/core/src/synced-store.ts b/src/synced-store.ts similarity index 100% rename from packages/core/src/synced-store.ts rename to src/synced-store.ts diff --git a/packages/core/src/types.ts b/src/types.ts similarity index 100% rename from packages/core/src/types.ts rename to src/types.ts diff --git a/packages/core/src/uuid.ts b/src/uuid.ts similarity index 100% rename from packages/core/src/uuid.ts rename to src/uuid.ts diff --git a/packages/core/src/views.ts b/src/views.ts similarity index 100% rename from packages/core/src/views.ts rename to src/views.ts diff --git a/packages/cli/test/cli.test.ts b/test/cli/cli.test.ts similarity index 98% rename from packages/cli/test/cli.test.ts rename to test/cli/cli.test.ts index 36b4b66..be4b7b1 100644 --- a/packages/cli/test/cli.test.ts +++ b/test/cli/cli.test.ts @@ -5,14 +5,11 @@ import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { Readable, Writable } from "node:stream"; import { describe, expect, it } from "vitest"; -import { runLyncCli } from "../src/index.js"; +import { runLyncCli } from "../../src/cli/index.js"; const vectorsRoot = join( dirname(fileURLToPath(import.meta.url)), "..", - "..", - "core", - "test", "vectors", "v0", ); diff --git a/packages/cli/test/sync.test.ts b/test/cli/sync.test.ts similarity index 99% rename from packages/cli/test/sync.test.ts rename to test/cli/sync.test.ts index 5c72224..f6897a3 100644 --- a/packages/cli/test/sync.test.ts +++ b/test/cli/sync.test.ts @@ -3,8 +3,8 @@ import { appendFile, mkdtemp, readFile, writeFile } from "node:fs/promises"; import { existsSync } from "node:fs"; import os from "node:os"; import path from "node:path"; -import { startLyncServe, type LyncSyncServer } from "lync-server"; -import { syncOnce } from "../src/sync.js"; +import { startLyncServe, type LyncSyncServer } from "lync-core/relay"; +import { syncOnce } from "../../src/cli/sync.js"; const quiet = { write: () => true } as const; diff --git a/packages/cli/test/synced-store.integration.test.ts b/test/cli/synced-store.integration.test.ts similarity index 98% rename from packages/cli/test/synced-store.integration.test.ts rename to test/cli/synced-store.integration.test.ts index 5aacc13..009de56 100644 --- a/packages/cli/test/synced-store.integration.test.ts +++ b/test/cli/synced-store.integration.test.ts @@ -5,7 +5,7 @@ import path from "node:path"; import { createMemoryEventStore } from "lync-core/memory-log"; import { createLyncLooms, loomRootId } from "lync-core/looms"; import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; -import { startLyncServe, type LyncSyncServer } from "lync-server"; +import { startLyncServe, type LyncSyncServer } from "lync-core/relay"; /** * The embedded browser story, proven end to end: a real relay, two clients diff --git a/packages/client/test/create.test.ts b/test/client/create.test.ts similarity index 89% rename from packages/client/test/create.test.ts rename to test/client/create.test.ts index f1d23ee..56f1e89 100644 --- a/packages/client/test/create.test.ts +++ b/test/client/create.test.ts @@ -1,9 +1,9 @@ import { describe, expect, it } from "vitest"; import { createMemoryEventStore } from "lync-core/memory-log"; import { createLyncLooms } from "lync-core/looms"; -import { createMemoryLoomIndexes } from "lync-index/memory"; -import { upsertLoom } from "lync-index/entries"; -import { createLoomClient } from "../src/create.js"; +import { createMemoryLoomIndexes } from "lync-core/indexes/memory"; +import { upsertLoom } from "lync-core/indexes/entries"; +import { createLoomClient } from "../../src/client/create.js"; function makeClient() { const store = createMemoryEventStore(); diff --git a/packages/core/test/events.test.ts b/test/events.test.ts similarity index 100% rename from packages/core/test/events.test.ts rename to test/events.test.ts diff --git a/packages/index/test/memory.test.ts b/test/indexes/memory.test.ts similarity index 96% rename from packages/index/test/memory.test.ts rename to test/indexes/memory.test.ts index 26315aa..4f3c541 100644 --- a/packages/index/test/memory.test.ts +++ b/test/indexes/memory.test.ts @@ -1,7 +1,7 @@ import { describe, expect, it } from "vitest"; import { loomRef } from "lync-core"; -import { createMemoryLoomIndexes } from "../src/memory.js"; -import { upsertLoom } from "../src/entries.js"; +import { createMemoryLoomIndexes } from "../../src/indexes/memory.js"; +import { upsertLoom } from "../../src/indexes/entries.js"; function deterministicIndexes() { let nextId = 0; diff --git a/packages/core/test/memory.test.ts b/test/memory.test.ts similarity index 100% rename from packages/core/test/memory.test.ts rename to test/memory.test.ts diff --git a/packages/core/test/references.test.ts b/test/references.test.ts similarity index 100% rename from packages/core/test/references.test.ts rename to test/references.test.ts diff --git a/packages/server/test/attach.test.ts b/test/relay/attach.test.ts similarity index 97% rename from packages/server/test/attach.test.ts rename to test/relay/attach.test.ts index fe71acb..3249483 100644 --- a/packages/server/test/attach.test.ts +++ b/test/relay/attach.test.ts @@ -6,7 +6,7 @@ import path from "node:path"; import { createMemoryEventStore } from "lync-core/memory-log"; import { createLyncLooms, loomRootId } from "lync-core/looms"; import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; -import { attachLyncServer, type AttachedLyncServer } from "../src/attach.js"; +import { attachLyncServer, type AttachedLyncServer } from "../../src/relay/attach.js"; async function listen(server: Server): Promise { return new Promise((resolve) => server.listen(0, () => { diff --git a/packages/server/test/relay.test.ts b/test/relay/relay.test.ts similarity index 98% rename from packages/server/test/relay.test.ts rename to test/relay/relay.test.ts index da6acf8..5ac0c41 100644 --- a/packages/server/test/relay.test.ts +++ b/test/relay/relay.test.ts @@ -6,7 +6,7 @@ import path from "node:path"; import { createMemoryEventStore } from "lync-core/memory-log"; import { createLyncLooms, loomRootId } from "lync-core/looms"; import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; -import { createLyncRelay } from "../src/relay.js"; +import { createLyncRelay } from "../../src/relay/relay.js"; /** * The relay mounted on an app's own HTTP server at a path — the embedding diff --git a/packages/core/test/sha256.test.ts b/test/sha256.test.ts similarity index 100% rename from packages/core/test/sha256.test.ts rename to test/sha256.test.ts diff --git a/packages/core/test/storage.test.ts b/test/storage.test.ts similarity index 100% rename from packages/core/test/storage.test.ts rename to test/storage.test.ts diff --git a/packages/core/test/sync-protocol.test.ts b/test/sync-protocol.test.ts similarity index 100% rename from packages/core/test/sync-protocol.test.ts rename to test/sync-protocol.test.ts diff --git a/packages/core/test/synced-store.test.ts b/test/synced-store.test.ts similarity index 100% rename from packages/core/test/synced-store.test.ts rename to test/synced-store.test.ts diff --git a/packages/core/test/text-story-profile.test.ts b/test/text-story-profile.test.ts similarity index 100% rename from packages/core/test/text-story-profile.test.ts rename to test/text-story-profile.test.ts diff --git a/packages/core/test/vectors/v0/01-valid-events/expected.json b/test/vectors/v0/01-valid-events/expected.json similarity index 100% rename from packages/core/test/vectors/v0/01-valid-events/expected.json rename to test/vectors/v0/01-valid-events/expected.json diff --git a/packages/core/test/vectors/v0/01-valid-events/input.lync b/test/vectors/v0/01-valid-events/input.lync similarity index 100% rename from packages/core/test/vectors/v0/01-valid-events/input.lync rename to test/vectors/v0/01-valid-events/input.lync diff --git a/packages/core/test/vectors/v0/02-splice-anchoring/expected.json b/test/vectors/v0/02-splice-anchoring/expected.json similarity index 100% rename from packages/core/test/vectors/v0/02-splice-anchoring/expected.json rename to test/vectors/v0/02-splice-anchoring/expected.json diff --git a/packages/core/test/vectors/v0/02-splice-anchoring/input.lync b/test/vectors/v0/02-splice-anchoring/input.lync similarity index 100% rename from packages/core/test/vectors/v0/02-splice-anchoring/input.lync rename to test/vectors/v0/02-splice-anchoring/input.lync diff --git a/packages/core/test/vectors/v0/03-damaged-digest/expected.json b/test/vectors/v0/03-damaged-digest/expected.json similarity index 100% rename from packages/core/test/vectors/v0/03-damaged-digest/expected.json rename to test/vectors/v0/03-damaged-digest/expected.json diff --git a/packages/core/test/vectors/v0/03-damaged-digest/input.lync b/test/vectors/v0/03-damaged-digest/input.lync similarity index 100% rename from packages/core/test/vectors/v0/03-damaged-digest/input.lync rename to test/vectors/v0/03-damaged-digest/input.lync diff --git a/packages/core/test/vectors/v0/04-garbage-classes/expected.json b/test/vectors/v0/04-garbage-classes/expected.json similarity index 100% rename from packages/core/test/vectors/v0/04-garbage-classes/expected.json rename to test/vectors/v0/04-garbage-classes/expected.json diff --git a/packages/core/test/vectors/v0/04-garbage-classes/input.lync b/test/vectors/v0/04-garbage-classes/input.lync similarity index 100% rename from packages/core/test/vectors/v0/04-garbage-classes/input.lync rename to test/vectors/v0/04-garbage-classes/input.lync diff --git a/packages/core/test/vectors/v0/05-conflicts-and-duplicates/expected.json b/test/vectors/v0/05-conflicts-and-duplicates/expected.json similarity index 100% rename from packages/core/test/vectors/v0/05-conflicts-and-duplicates/expected.json rename to test/vectors/v0/05-conflicts-and-duplicates/expected.json diff --git a/packages/core/test/vectors/v0/05-conflicts-and-duplicates/input.lync b/test/vectors/v0/05-conflicts-and-duplicates/input.lync similarity index 100% rename from packages/core/test/vectors/v0/05-conflicts-and-duplicates/input.lync rename to test/vectors/v0/05-conflicts-and-duplicates/input.lync diff --git a/packages/core/test/vectors/v0/06-graph-obstacles/expected.json b/test/vectors/v0/06-graph-obstacles/expected.json similarity index 100% rename from packages/core/test/vectors/v0/06-graph-obstacles/expected.json rename to test/vectors/v0/06-graph-obstacles/expected.json diff --git a/packages/core/test/vectors/v0/06-graph-obstacles/input.lync b/test/vectors/v0/06-graph-obstacles/input.lync similarity index 100% rename from packages/core/test/vectors/v0/06-graph-obstacles/input.lync rename to test/vectors/v0/06-graph-obstacles/input.lync diff --git a/packages/core/test/vectors/v0/07-critical-suppression/expected.json b/test/vectors/v0/07-critical-suppression/expected.json similarity index 100% rename from packages/core/test/vectors/v0/07-critical-suppression/expected.json rename to test/vectors/v0/07-critical-suppression/expected.json diff --git a/packages/core/test/vectors/v0/07-critical-suppression/input.lync b/test/vectors/v0/07-critical-suppression/input.lync similarity index 100% rename from packages/core/test/vectors/v0/07-critical-suppression/input.lync rename to test/vectors/v0/07-critical-suppression/input.lync diff --git a/packages/core/test/vectors/v0/08-spelling-vs-value/expected.json b/test/vectors/v0/08-spelling-vs-value/expected.json similarity index 100% rename from packages/core/test/vectors/v0/08-spelling-vs-value/expected.json rename to test/vectors/v0/08-spelling-vs-value/expected.json diff --git a/packages/core/test/vectors/v0/08-spelling-vs-value/input.lync b/test/vectors/v0/08-spelling-vs-value/input.lync similarity index 100% rename from packages/core/test/vectors/v0/08-spelling-vs-value/input.lync rename to test/vectors/v0/08-spelling-vs-value/input.lync diff --git a/packages/core/test/vectors/v0/09-marked-at-semantics/expected.json b/test/vectors/v0/09-marked-at-semantics/expected.json similarity index 100% rename from packages/core/test/vectors/v0/09-marked-at-semantics/expected.json rename to test/vectors/v0/09-marked-at-semantics/expected.json diff --git a/packages/core/test/vectors/v0/09-marked-at-semantics/input.lync b/test/vectors/v0/09-marked-at-semantics/input.lync similarity index 100% rename from packages/core/test/vectors/v0/09-marked-at-semantics/input.lync rename to test/vectors/v0/09-marked-at-semantics/input.lync diff --git a/packages/core/test/vectors/v0/10-merge-union/a.lync b/test/vectors/v0/10-merge-union/a.lync similarity index 100% rename from packages/core/test/vectors/v0/10-merge-union/a.lync rename to test/vectors/v0/10-merge-union/a.lync diff --git a/packages/core/test/vectors/v0/10-merge-union/b.lync b/test/vectors/v0/10-merge-union/b.lync similarity index 100% rename from packages/core/test/vectors/v0/10-merge-union/b.lync rename to test/vectors/v0/10-merge-union/b.lync diff --git a/packages/core/test/vectors/v0/10-merge-union/expected.json b/test/vectors/v0/10-merge-union/expected.json similarity index 100% rename from packages/core/test/vectors/v0/10-merge-union/expected.json rename to test/vectors/v0/10-merge-union/expected.json diff --git a/packages/core/test/vectors/v0/11-nonconforming-carried/expected.json b/test/vectors/v0/11-nonconforming-carried/expected.json similarity index 100% rename from packages/core/test/vectors/v0/11-nonconforming-carried/expected.json rename to test/vectors/v0/11-nonconforming-carried/expected.json diff --git a/packages/core/test/vectors/v0/11-nonconforming-carried/input.lync b/test/vectors/v0/11-nonconforming-carried/input.lync similarity index 100% rename from packages/core/test/vectors/v0/11-nonconforming-carried/input.lync rename to test/vectors/v0/11-nonconforming-carried/input.lync diff --git a/packages/core/test/vectors/v0/12-invalid-sig-splice/expected.json b/test/vectors/v0/12-invalid-sig-splice/expected.json similarity index 100% rename from packages/core/test/vectors/v0/12-invalid-sig-splice/expected.json rename to test/vectors/v0/12-invalid-sig-splice/expected.json diff --git a/packages/core/test/vectors/v0/12-invalid-sig-splice/input.lync b/test/vectors/v0/12-invalid-sig-splice/input.lync similarity index 100% rename from packages/core/test/vectors/v0/12-invalid-sig-splice/input.lync rename to test/vectors/v0/12-invalid-sig-splice/input.lync diff --git a/packages/core/test/vectors/v0/13-sig-without-digest/expected.json b/test/vectors/v0/13-sig-without-digest/expected.json similarity index 100% rename from packages/core/test/vectors/v0/13-sig-without-digest/expected.json rename to test/vectors/v0/13-sig-without-digest/expected.json diff --git a/packages/core/test/vectors/v0/13-sig-without-digest/input.lync b/test/vectors/v0/13-sig-without-digest/input.lync similarity index 100% rename from packages/core/test/vectors/v0/13-sig-without-digest/input.lync rename to test/vectors/v0/13-sig-without-digest/input.lync diff --git a/packages/core/test/vectors/v0/OPEN-QUESTIONS.md b/test/vectors/v0/OPEN-QUESTIONS.md similarity index 100% rename from packages/core/test/vectors/v0/OPEN-QUESTIONS.md rename to test/vectors/v0/OPEN-QUESTIONS.md diff --git a/packages/core/test/vectors/v0/README.md b/test/vectors/v0/README.md similarity index 100% rename from packages/core/test/vectors/v0/README.md rename to test/vectors/v0/README.md diff --git a/packages/core/test/vectors/v0/generate.py b/test/vectors/v0/generate.py similarity index 100% rename from packages/core/test/vectors/v0/generate.py rename to test/vectors/v0/generate.py diff --git a/packages/core/test/views.test.ts b/test/views.test.ts similarity index 100% rename from packages/core/test/views.test.ts rename to test/views.test.ts diff --git a/tsconfig.base.json b/tsconfig.json similarity index 76% rename from tsconfig.base.json rename to tsconfig.json index d631028..fd8f468 100644 --- a/tsconfig.base.json +++ b/tsconfig.json @@ -11,6 +11,9 @@ "esModuleInterop": true, "forceConsistentCasingInFileNames": true, "isolatedModules": true, - "skipLibCheck": true - } + "skipLibCheck": true, + "rootDir": "src", + "outDir": "dist" + }, + "include": ["src/**/*.ts"] } diff --git a/vitest.config.ts b/vitest.config.ts index 3246350..11edff1 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -6,41 +6,41 @@ export default defineConfig({ { find: /^lync-core\/profiles\/text-story$/, replacement: new URL( - "./packages/core/src/profiles/text-story.ts", + "./src/profiles/text-story.ts", import.meta.url, ).pathname, }, { - find: /^lync-core\/([a-z0-9-]+)$/, - replacement: new URL("./packages/core/src/", import.meta.url).pathname + "$1.ts", + find: /^lync-core\/indexes\/([a-z0-9-]+)$/, + replacement: new URL("./src/indexes/", import.meta.url).pathname + "$1.ts", }, { - find: /^lync-core$/, - replacement: new URL("./packages/core/src/index.ts", import.meta.url).pathname, + find: /^lync-core\/indexes$/, + replacement: new URL("./src/indexes/index.ts", import.meta.url).pathname, }, { - find: /^lync-index\/([a-z0-9-]+)$/, - replacement: new URL("./packages/index/src/", import.meta.url).pathname + "$1.ts", + find: /^lync-core\/client\/([a-z0-9-]+)$/, + replacement: new URL("./src/client/", import.meta.url).pathname + "$1.ts", }, { - find: /^lync-index$/, - replacement: new URL("./packages/index/src/index.ts", import.meta.url).pathname, + find: /^lync-core\/client$/, + replacement: new URL("./src/client/index.ts", import.meta.url).pathname, }, { - find: /^lync-server$/, - replacement: new URL("./packages/server/src/index.ts", import.meta.url).pathname, + find: /^lync-core\/relay$/, + replacement: new URL("./src/relay/index.ts", import.meta.url).pathname, }, { - find: /^lync-client\/([a-z0-9-]+)$/, - replacement: new URL("./packages/client/src/", import.meta.url).pathname + "$1.ts", + find: /^lync-core\/([a-z0-9-]+)$/, + replacement: new URL("./src/", import.meta.url).pathname + "$1.ts", }, { - find: /^lync-client$/, - replacement: new URL("./packages/client/src/index.ts", import.meta.url).pathname, + find: /^lync-core$/, + replacement: new URL("./src/index.ts", import.meta.url).pathname, }, ], }, test: { - include: ["packages/*/test/**/*.test.ts"], + include: ["test/**/*.test.ts"], }, }); From 2081462e0cd1991a65133437e4420639b72e0c20 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 00:00:30 -0700 Subject: [PATCH 24/33] gitignore: packed tarballs --- .gitignore | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.gitignore b/.gitignore index fd80fff..2e50e26 100644 --- a/.gitignore +++ b/.gitignore @@ -10,3 +10,6 @@ dist/ /VERDICT-*.md /.tickets/ /FRESHREAD*.md + +# packed tarballs +*.tgz From 4e859919a15e7b8aee11eaf2104235d5d9d3d394 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 00:01:47 -0700 Subject: [PATCH 25/33] docs: ROADMAP and pacts speak the one-package truth (no five-package ghost, no packages/ path) --- ROADMAP.md | 4 ++-- pacts/export.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/ROADMAP.md b/ROADMAP.md index 84ebb39..147814f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -5,8 +5,8 @@ meaning should land in pacts, not in the envelope. ## Now -- First public release of the five packages: `lync-core`, `lync-cli`, - `lync-index`, `lync-client`, `lync-server`. +- First public release of the one package: `lync-core` (library, indexes, + client, relay, and the `lync` command). - Keep `FORMAT.md` and the test vectors aligned as the reference other languages can port. diff --git a/pacts/export.md b/pacts/export.md index 42ca866..3efbb27 100644 --- a/pacts/export.md +++ b/pacts/export.md @@ -1,7 +1,7 @@ # Export Pact Status: v0, and younger than the import pact. The view functions it names are -shipped (`packages/core/src/views.ts`); the export file formats built on them +shipped (`src/views.ts`); the export file formats built on them are still settling. The principles are law; the column schemas are early. An export is a projection: a view computed over the event set, written down From f05535517e6b27ff23289c261a73a2636f902a95 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 01:04:12 -0700 Subject: [PATCH 26/33] rename: the package is @deepfates/lync (bin stays lync; format vocabulary unchanged) --- README.md | 66 +++++++++++------------ ROADMAP.md | 2 +- package.json | 2 +- scripts/check-readme-examples.mjs | 2 +- src/index.ts | 2 +- test/cli/sync.test.ts | 2 +- test/cli/synced-store.integration.test.ts | 8 +-- test/client/create.test.ts | 8 +-- test/indexes/memory.test.ts | 2 +- test/relay/attach.test.ts | 6 +-- test/relay/relay.test.ts | 8 +-- test/sync-protocol.test.ts | 4 +- test/synced-store.test.ts | 10 ++-- vitest.config.ts | 16 +++--- 14 files changed, 69 insertions(+), 69 deletions(-) diff --git a/README.md b/README.md index 2f1e5f2..93370e5 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # lync -lync is a file format for append-only interaction history, and `lync-core` is +lync is a file format for append-only interaction history, and `@deepfates/lync` is its reference implementation — one package that ships the parser, event stores, computed views, the loom API, live sync, the `lync` command, and the sync relay. Zero runtime dependencies. @@ -70,7 +70,7 @@ expected-output schema; `generate.py` regenerates digests deterministically. ## The Library ```bash -npm install lync-core +npm install @deepfates/lync ``` Runs in Node (>=22) and the browser. No dependencies. @@ -78,8 +78,8 @@ Runs in Node (>=22) and the browser. No dependencies. ### Parse, union, view ```ts -import { parseLyncFiles } from "lync-core/events"; -import { lyncBranchTreeView, lyncTranscriptView } from "lync-core/views"; +import { parseLyncFiles } from "@deepfates/lync/events"; +import { lyncBranchTreeView, lyncTranscriptView } from "@deepfates/lync/views"; const bytes = new TextEncoder().encode( '{"v":1,"id":"root","kind":"notes/text","at":"2026-07-06T04:12:31Z","author":{"actor":"you"},"parents":[],"payload":{"text":"Once..."}}\n', @@ -105,8 +105,8 @@ loom API gives programs turns and threads instead of raw events, on top of any store: ```ts -import { createLyncLooms } from "lync-core/looms"; -import { createMemoryEventStore } from "lync-core/memory-log"; +import { createLyncLooms } from "@deepfates/lync/looms"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; const looms = createLyncLooms({ store: createMemoryEventStore(), @@ -128,8 +128,8 @@ and accepted events without making file order meaningful. An index tracks a collection of looms — upsert entries, subscribe to changes: ```ts -import { loomRef } from "lync-core"; -import { createMemoryLoomIndexes } from "lync-core/indexes/memory"; +import { loomRef } from "@deepfates/lync"; +import { createMemoryLoomIndexes } from "@deepfates/lync/indexes/memory"; const indexes = createMemoryLoomIndexes(); const index = await indexes.create({ title: "My looms" }); @@ -149,10 +149,10 @@ One object that pairs looms with an index and resolves loom/turn/thread/index references to and from URLs: ```ts -import { createLyncLooms } from "lync-core/looms"; -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createMemoryLoomIndexes } from "lync-core/indexes/memory"; -import { createLoomClient } from "lync-core/client"; +import { createLyncLooms } from "@deepfates/lync/looms"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createMemoryLoomIndexes } from "@deepfates/lync/indexes/memory"; +import { createLoomClient } from "@deepfates/lync/client"; const client = createLoomClient({ looms: createLyncLooms({ store: createMemoryEventStore(), author: { actor: "you" } }), @@ -169,7 +169,7 @@ const opened = await client.openReference(client.references.fromUrl(new URL(url) console.log(opened.kind); // "loom" — opened.loom is ready to appendTurn ``` -`lync-core/client/testing` ships `createTestLoomClient`, a fully in-memory +`@deepfates/lync/client/testing` ships `createTestLoomClient`, a fully in-memory client for tests and embedded experiments — deterministic when you pass `createId` and `now`. @@ -182,9 +182,9 @@ collaborators append, because they already recompute through the store's ```ts -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createLyncLooms } from "lync-core/looms"; -import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createLyncLooms } from "@deepfates/lync/looms"; +import { createSyncedStore, createWebSocketTransport } from "@deepfates/lync/synced-store"; const transport = createWebSocketTransport("wss://host/lync"); const store = createSyncedStore(createMemoryEventStore(), transport, { @@ -206,21 +206,21 @@ in Node. ### Subpath exports -- `lync-core/events` — line parsing, carried-byte export, incremental union -- `lync-core/store` — the event-store contract and serialization -- `lync-core/memory-log`, `lync-core/file-log`, `lync-core/idb-log` — stores +- `@deepfates/lync/events` — line parsing, carried-byte export, incremental union +- `@deepfates/lync/store` — the event-store contract and serialization +- `@deepfates/lync/memory-log`, `@deepfates/lync/file-log`, `@deepfates/lync/idb-log` — stores (`file-log` is node-only; it keeps `node:fs` off the browser path) -- `lync-core/views` — branch tree, transcript, memory, leaderboard -- `lync-core/looms` — the loom/turn API -- `lync-core/references` — loom/turn/thread/index references and URLs -- `lync-core/synced-store` — live sync decorator and WebSocket transport -- `lync-core/sync-protocol` — the five sync frames, encode/decode -- `lync-core/uuid` — zero-dep UUIDv7 for event ids -- `lync-core/indexes`, `lync-core/indexes/entries`, - `lync-core/indexes/memory`, `lync-core/indexes/types` — loom indexes -- `lync-core/client`, `lync-core/client/testing`, `lync-core/client/types` — +- `@deepfates/lync/views` — branch tree, transcript, memory, leaderboard +- `@deepfates/lync/looms` — the loom/turn API +- `@deepfates/lync/references` — loom/turn/thread/index references and URLs +- `@deepfates/lync/synced-store` — live sync decorator and WebSocket transport +- `@deepfates/lync/sync-protocol` — the five sync frames, encode/decode +- `@deepfates/lync/uuid` — zero-dep UUIDv7 for event ids +- `@deepfates/lync/indexes`, `@deepfates/lync/indexes/entries`, + `@deepfates/lync/indexes/memory`, `@deepfates/lync/indexes/types` — loom indexes +- `@deepfates/lync/client`, `@deepfates/lync/client/testing`, `@deepfates/lync/client/types` — the loom client -- `lync-core/relay` — the sync relay (see [The Relay](#the-relay)) +- `@deepfates/lync/relay` — the sync relay (see [The Relay](#the-relay)) ## The Command @@ -228,7 +228,7 @@ The package installs a `lync` bin with seven verbs: `init`, `append`, `verify`, `merge`, `view`, `serve`, and `sync`. ```bash -npm install -g lync-core +npm install -g @deepfates/lync ``` ```bash @@ -273,7 +273,7 @@ client reconnects exactly where it left off. Running a relay is the one thing that needs a WebSocket server, and Node does not ship one — so the relay acquires [`ws`](https://www.npmjs.com/package/ws) -lazily at the moment you construct it. `lync-core` declares no dependency on +lazily at the moment you construct it. `@deepfates/lync` declares no dependency on `ws` at all: install it yourself next to your server (`npm install ws`), and everything else in the package works without it. If you bundle a server that runs the relay, mark `ws` as external — the @@ -283,7 +283,7 @@ Standalone: ```ts -import { startLyncServe } from "lync-core/relay"; +import { startLyncServe } from "@deepfates/lync/relay"; const server = await startLyncServe({ dir: "./rooms", port: 8787 }); console.log("relay on", server.port); @@ -295,7 +295,7 @@ On an existing HTTP server: ```ts import { createServer } from "node:http"; -import { attachLyncServer } from "lync-core/relay"; +import { attachLyncServer } from "@deepfates/lync/relay"; const httpServer = createServer(app); const lync = attachLyncServer(httpServer, { diff --git a/ROADMAP.md b/ROADMAP.md index 147814f..b1d1ca9 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -5,7 +5,7 @@ meaning should land in pacts, not in the envelope. ## Now -- First public release of the one package: `lync-core` (library, indexes, +- First public release of the one package: `@deepfates/lync` (library, indexes, client, relay, and the `lync` command). - Keep `FORMAT.md` and the test vectors aligned as the reference other languages can port. diff --git a/package.json b/package.json index 5568374..67b1c3d 100644 --- a/package.json +++ b/package.json @@ -1,5 +1,5 @@ { - "name": "lync-core", + "name": "@deepfates/lync", "version": "0.3.0", "description": "The lync format: append-only JSONL event logs merged by set union. Parsing, stores, views, looms, live sync, loom client, indexes, the sync relay, and the lync command. Zero dependencies.", "type": "module", diff --git a/scripts/check-readme-examples.mjs b/scripts/check-readme-examples.mjs index 47a8123..97b1099 100644 --- a/scripts/check-readme-examples.mjs +++ b/scripts/check-readme-examples.mjs @@ -14,7 +14,7 @@ // There is no other configuration: a new example is checked by default. // // ts blocks execute from inside the repo root — Node's package self-reference -// resolves "lync-core" and its subpaths exactly like an installed consumer — +// resolves "@deepfates/lync" and its subpaths exactly like an installed consumer — // with cwd in a scratch dir so relative paths never touch the repo. bash // blocks run with the `lync` command token rewritten to the workspace bin; // `npm install` lines are skipped (noted), since installing is the reader's diff --git a/src/index.ts b/src/index.ts index c11208b..5414d37 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1,5 +1,5 @@ export * from "./errors.js"; -// The node:fs-backed file store lives only at the explicit "lync-core/file-log" +// The node:fs-backed file store lives only at the explicit "@deepfates/lync/file-log" // subpath so the main barrel stays importable in the browser with zero node builtins. export * from "./idb-log.js"; export * from "./looms.js"; diff --git a/test/cli/sync.test.ts b/test/cli/sync.test.ts index f6897a3..1e5968e 100644 --- a/test/cli/sync.test.ts +++ b/test/cli/sync.test.ts @@ -3,7 +3,7 @@ import { appendFile, mkdtemp, readFile, writeFile } from "node:fs/promises"; import { existsSync } from "node:fs"; import os from "node:os"; import path from "node:path"; -import { startLyncServe, type LyncSyncServer } from "lync-core/relay"; +import { startLyncServe, type LyncSyncServer } from "@deepfates/lync/relay"; import { syncOnce } from "../../src/cli/sync.js"; const quiet = { write: () => true } as const; diff --git a/test/cli/synced-store.integration.test.ts b/test/cli/synced-store.integration.test.ts index 009de56..13a52af 100644 --- a/test/cli/synced-store.integration.test.ts +++ b/test/cli/synced-store.integration.test.ts @@ -2,10 +2,10 @@ import { afterEach, describe, expect, it } from "vitest"; import { mkdtemp } from "node:fs/promises"; import os from "node:os"; import path from "node:path"; -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createLyncLooms, loomRootId } from "lync-core/looms"; -import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; -import { startLyncServe, type LyncSyncServer } from "lync-core/relay"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createLyncLooms, loomRootId } from "@deepfates/lync/looms"; +import { createSyncedStore, createWebSocketTransport } from "@deepfates/lync/synced-store"; +import { startLyncServe, type LyncSyncServer } from "@deepfates/lync/relay"; /** * The embedded browser story, proven end to end: a real relay, two clients diff --git a/test/client/create.test.ts b/test/client/create.test.ts index 56f1e89..ee67750 100644 --- a/test/client/create.test.ts +++ b/test/client/create.test.ts @@ -1,8 +1,8 @@ import { describe, expect, it } from "vitest"; -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createLyncLooms } from "lync-core/looms"; -import { createMemoryLoomIndexes } from "lync-core/indexes/memory"; -import { upsertLoom } from "lync-core/indexes/entries"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createLyncLooms } from "@deepfates/lync/looms"; +import { createMemoryLoomIndexes } from "@deepfates/lync/indexes/memory"; +import { upsertLoom } from "@deepfates/lync/indexes/entries"; import { createLoomClient } from "../../src/client/create.js"; function makeClient() { diff --git a/test/indexes/memory.test.ts b/test/indexes/memory.test.ts index 4f3c541..3cfc26e 100644 --- a/test/indexes/memory.test.ts +++ b/test/indexes/memory.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { loomRef } from "lync-core"; +import { loomRef } from "@deepfates/lync"; import { createMemoryLoomIndexes } from "../../src/indexes/memory.js"; import { upsertLoom } from "../../src/indexes/entries.js"; diff --git a/test/relay/attach.test.ts b/test/relay/attach.test.ts index 3249483..7db23f8 100644 --- a/test/relay/attach.test.ts +++ b/test/relay/attach.test.ts @@ -3,9 +3,9 @@ import { createServer, type Server } from "node:http"; import { mkdtemp } from "node:fs/promises"; import os from "node:os"; import path from "node:path"; -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createLyncLooms, loomRootId } from "lync-core/looms"; -import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createLyncLooms, loomRootId } from "@deepfates/lync/looms"; +import { createSyncedStore, createWebSocketTransport } from "@deepfates/lync/synced-store"; import { attachLyncServer, type AttachedLyncServer } from "../../src/relay/attach.js"; async function listen(server: Server): Promise { diff --git a/test/relay/relay.test.ts b/test/relay/relay.test.ts index 5ac0c41..3c2d042 100644 --- a/test/relay/relay.test.ts +++ b/test/relay/relay.test.ts @@ -3,9 +3,9 @@ import { createServer, type Server } from "node:http"; import { mkdtemp } from "node:fs/promises"; import os from "node:os"; import path from "node:path"; -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createLyncLooms, loomRootId } from "lync-core/looms"; -import { createSyncedStore, createWebSocketTransport } from "lync-core/synced-store"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createLyncLooms, loomRootId } from "@deepfates/lync/looms"; +import { createSyncedStore, createWebSocketTransport } from "@deepfates/lync/synced-store"; import { createLyncRelay } from "../../src/relay/relay.js"; /** @@ -79,7 +79,7 @@ describe("createLyncRelay durability failures", () => { const os = await import("node:os"); const nodePath = await import("node:path"); const { createServer } = await import("node:http"); - const { createWebSocketTransport } = await import("lync-core/synced-store"); + const { createWebSocketTransport } = await import("@deepfates/lync/synced-store"); const dir = await mkdtemp(nodePath.join(os.tmpdir(), "lync-persist-")); // Read-only dir: recovery (no existing files) succeeds, but every append fails. diff --git a/test/sync-protocol.test.ts b/test/sync-protocol.test.ts index a0770cc..bee445f 100644 --- a/test/sync-protocol.test.ts +++ b/test/sync-protocol.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "vitest"; -import { decodeFrame, encodeFrame, extractLineId, type SyncFrame } from "lync-core/sync-protocol"; +import { decodeFrame, encodeFrame, extractLineId, type SyncFrame } from "@deepfates/lync/sync-protocol"; describe("lync sync protocol frames", () => { it("round-trips every frame kind", () => { @@ -45,7 +45,7 @@ describe("cursor integrity (dee-inzc blocker)", () => { describe("uuidv7 minting", () => { it("mints valid, time-ordered UUIDv7", async () => { - const { uuidv7 } = await import("lync-core/uuid"); + const { uuidv7 } = await import("@deepfates/lync/uuid"); const a = uuidv7(1_700_000_000_000); const b = uuidv7(1_700_000_000_001); expect(a).toMatch(/^[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/); diff --git a/test/synced-store.test.ts b/test/synced-store.test.ts index b83f4ec..9cbbd6a 100644 --- a/test/synced-store.test.ts +++ b/test/synced-store.test.ts @@ -1,14 +1,14 @@ import { describe, expect, it } from "vitest"; -import { createMemoryEventStore } from "lync-core/memory-log"; -import { createLyncLooms } from "lync-core/looms"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createLyncLooms } from "@deepfates/lync/looms"; import { createSyncedStore, type SyncConnectionState, type SyncStatus, type SyncTransport, -} from "lync-core/synced-store"; -import type { SyncFrame } from "lync-core/sync-protocol"; -import { serializeLyncEvent } from "lync-core/store"; +} from "@deepfates/lync/synced-store"; +import type { SyncFrame } from "@deepfates/lync/sync-protocol"; +import { serializeLyncEvent } from "@deepfates/lync/store"; function mockTransport(initial: SyncConnectionState = "online") { const frameHandlers = new Set<(frame: SyncFrame) => void>(); diff --git a/vitest.config.ts b/vitest.config.ts index 11edff1..c4a70ce 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -4,38 +4,38 @@ export default defineConfig({ resolve: { alias: [ { - find: /^lync-core\/profiles\/text-story$/, + find: /^@deepfates\/lync\/profiles\/text-story$/, replacement: new URL( "./src/profiles/text-story.ts", import.meta.url, ).pathname, }, { - find: /^lync-core\/indexes\/([a-z0-9-]+)$/, + find: /^@deepfates\/lync\/indexes\/([a-z0-9-]+)$/, replacement: new URL("./src/indexes/", import.meta.url).pathname + "$1.ts", }, { - find: /^lync-core\/indexes$/, + find: /^@deepfates\/lync\/indexes$/, replacement: new URL("./src/indexes/index.ts", import.meta.url).pathname, }, { - find: /^lync-core\/client\/([a-z0-9-]+)$/, + find: /^@deepfates\/lync\/client\/([a-z0-9-]+)$/, replacement: new URL("./src/client/", import.meta.url).pathname + "$1.ts", }, { - find: /^lync-core\/client$/, + find: /^@deepfates\/lync\/client$/, replacement: new URL("./src/client/index.ts", import.meta.url).pathname, }, { - find: /^lync-core\/relay$/, + find: /^@deepfates\/lync\/relay$/, replacement: new URL("./src/relay/index.ts", import.meta.url).pathname, }, { - find: /^lync-core\/([a-z0-9-]+)$/, + find: /^@deepfates\/lync\/([a-z0-9-]+)$/, replacement: new URL("./src/", import.meta.url).pathname + "$1.ts", }, { - find: /^lync-core$/, + find: /^@deepfates\/lync$/, replacement: new URL("./src/index.ts", import.meta.url).pathname, }, ], From 43034ced1feb21738521bc883e250232a996f3a1 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 01:16:14 -0700 Subject: [PATCH 27/33] =?UTF-8?q?relay:=20log-generation=20id=20=E2=80=94?= =?UTF-8?q?=20a=20cursor=20is=20only=20meaningful=20inside=20the=20generat?= =?UTF-8?q?ion=20that=20issued=20it=20(dee-u6tq)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit recoverRoom mints a random generation id (never persisted: every restart is a new generation), carried additively on ev/live frames. The CLI sync cursor file stores {seq, generation}; a mismatch resets to 0 and resubscribes — union makes the re-download duplicate no-ops. Stale lives from superseded subs are counted, not trusted. Regression test is the bug's exact shape: persisted events, a broadcast whose disk write failed consuming a seq, a restart, a stale cursor — the client ends with every persisted event (verified failing with the reset neutered). --- src/cli/sync.ts | 62 ++++++++++++++++++++++- src/relay/relay.ts | 17 +++++-- src/sync-protocol.ts | 36 +++++++++++-- test/cli/sync.test.ts | 101 +++++++++++++++++++++++++++++++++++++ test/sync-protocol.test.ts | 27 ++++++++++ 5 files changed, 235 insertions(+), 8 deletions(-) diff --git a/src/cli/sync.ts b/src/cli/sync.ts index 222da47..698a59f 100644 --- a/src/cli/sync.ts +++ b/src/cli/sync.ts @@ -19,6 +19,13 @@ import { decodeFrame, encodeFrame, extractLineId, isCursor } from "../sync-proto * advances only after a received line has reached a durable local state — * appended, recognized as a duplicate, or surfaced as unusable. A sync that * cannot reach `live` within the timeout fails loudly; nothing hangs. + * + * The cursor stores the server's log generation alongside seq: a seq is only + * meaningful inside the generation that issued it (a broadcast whose disk + * write failed still consumed a seq, so after a server restart the recovered + * log can sit behind our cursor). When the server's `gen` differs from the + * stored one, the cursor resets to 0 and we resubscribe from scratch — union + * makes the re-download a set of duplicate no-ops, and nothing is skipped. */ export interface LyncSyncOptions { @@ -50,6 +57,8 @@ interface Cursor { url: string; root: string; seq: number; + /** Server log generation the seq belongs to. Absent: old cursor file or old server. */ + generation?: string; } export async function syncOnce(options: LyncSyncOptions): Promise { @@ -83,8 +92,43 @@ export async function syncOnce(options: LyncSyncOptions): Promise - writeFile(cursorPath, `${JSON.stringify({ url: options.url, root, seq: result.seq } satisfies Cursor, null, 2)}\n`); + writeFile( + cursorPath, + `${JSON.stringify({ url: options.url, root, seq: result.seq, ...(generation !== undefined ? { generation } : {}) } satisfies Cursor, null, 2)}\n`, + ); + + /** + * Returns true when the server's generation differs from the one our cursor + * was saved under — in which case the cursor has been reset to 0 and a fresh + * `sub` from 0 is already on the wire. Frames without gen (old server) never + * trigger a reset. + */ + const generationChanged = (gen: string | undefined): boolean => { + if (gen === undefined || gen === generation) return false; + if (generation === undefined) { + generation = gen; // first sighting: adopt, nothing to reset + return false; + } + options.err.write( + `lync sync: server log generation changed (${generation} -> ${gen}); resyncing ${root} from 0\n`, + ); + generation = gen; + result.seq = 0; + awaitedLives += 1; + socket.send(encodeFrame({ t: "sub", root, since: 0 })); + return true; + }; await new Promise((resolve, reject) => { const timeout = setTimeout(() => { @@ -151,6 +195,10 @@ export async function syncOnce(options: LyncSyncOptions): Promise 0) return; clearTimeout(timeout); result.seq = Math.max(result.seq, frame.seq); if (!options.follow) { @@ -225,6 +280,11 @@ async function readCursor(path: string, url: string, root: string): Promise; @@ -161,9 +170,9 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { send(socket, { t: "err", root: room.root, reason: "recovered-damaged-tail", detail: room.recoveryNote }); } for (let index = frame.since; index < room.lines.length; index += 1) { - send(socket, { t: "ev", root: room.root, seq: index + 1, line: room.lines[index] }); + send(socket, { t: "ev", root: room.root, seq: index + 1, line: room.lines[index], gen: room.generation }); } - send(socket, { t: "live", root: room.root, seq: room.seq }); + send(socket, { t: "live", root: room.root, seq: room.seq, gen: room.generation }); return; } case "ev": { @@ -194,7 +203,7 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { const persisted = await appendSerialized(room, join(options.dir, `${room.root}.lync`), frame.line); // Live delivery is the relay's primary job: fan out even if the disk // write failed. A durability failure is surfaced loudly, never hidden. - broadcast(room, { t: "ev", root: room.root, seq, line: frame.line }); + broadcast(room, { t: "ev", root: room.root, seq, line: frame.line, gen: room.generation }); if (!persisted.ok) { broadcast(room, { t: "err", root: room.root, reason: "persist-failed", detail: id }); } @@ -228,7 +237,7 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { } async function recoverRoom(root: string): Promise { - const room: Room = { root, seq: 0, lines: [], byId: new Map(), subscribers: new Set(), writeChain: Promise.resolve() }; + const room: Room = { root, generation: randomUUID(), seq: 0, lines: [], byId: new Map(), subscribers: new Set(), writeChain: Promise.resolve() }; const path = join(options.dir, `${root}.lync`); if (!existsSync(path)) return room; const text = await readFile(path, "utf8"); diff --git a/src/sync-protocol.ts b/src/sync-protocol.ts index e9c8973..21d816e 100644 --- a/src/sync-protocol.ts +++ b/src/sync-protocol.ts @@ -6,8 +6,8 @@ * union make redundancy harmless. Five frame kinds: * * client → server {"t":"sub", "root": string, "since": number} - * server → client {"t":"ev", "root": string, "seq": number, "line": string} - * server → client {"t":"live", "root": string, "seq": number} + * server → client {"t":"ev", "root": string, "seq": number, "line": string, "gen"?: string} + * server → client {"t":"live", "root": string, "seq": number, "gen"?: string} * client → server {"t":"ev", "root": string, "line": string} * either direction {"t":"presence", "root": string, "data"?: unknown} * either direction {"t":"err", "root"?: string, "reason": string, "detail"?: string} @@ -16,6 +16,17 @@ * event order. The server echoes accepted events to every subscriber of the * root, sender included; echoes are duplicate no-ops under union and still * advance the cursor. This module is pure: frame codecs and guards only. + * + * `gen` is the server's log GENERATION: a random id minted every time a room + * is recovered from disk (so every server restart is a new generation). A + * cursor is only meaningful inside the generation that issued it — a broadcast + * whose disk write failed still consumes a seq, so after a restart the + * recovered log can sit BEHIND a client's saved cursor and the client would + * silently skip the next persisted event forever. A client that sees `gen` + * change from what its cursor was saved under must reset to 0 and resync from + * scratch; union makes the re-download a harmless set of duplicate no-ops. + * The field is additive: frames without it (old servers) decode fine, and old + * clients ignore it. */ export interface SubFrame { @@ -29,12 +40,16 @@ export interface EvFrame { root: string; line: string; seq?: number; + /** Server log generation (see module doc). Absent from old servers and client→server frames. */ + gen?: string; } export interface LiveFrame { t: "live"; root: string; seq: number; + /** Server log generation (see module doc). Absent from old servers. */ + gen?: string; } export interface PresenceFrame { @@ -101,17 +116,32 @@ export function decodeFrame(raw: string | Uint8Array): SyncFrame { if (frame.seq !== undefined && !isCursor(frame.seq)) { return { t: "err", reason: "malformed-ev", detail: "seq must be a nonnegative integer" }; } + // gen is additive: absence is fine (old peers). Present-but-not-a-string + // is malformed — a client resetting its cursor over garbage would be + // acting on noise. + if (frame.gen !== undefined && typeof frame.gen !== "string") { + return { t: "err", reason: "malformed-ev", detail: "gen must be a string" }; + } return { t: "ev", root: frame.root, line: frame.line, ...(frame.seq !== undefined ? { seq: frame.seq as number } : {}), + ...(frame.gen !== undefined ? { gen: frame.gen as string } : {}), }; case "live": if (typeof frame.root !== "string" || !isCursor(frame.seq)) { return { t: "err", reason: "malformed-live" }; } - return { t: "live", root: frame.root, seq: frame.seq as number }; + if (frame.gen !== undefined && typeof frame.gen !== "string") { + return { t: "err", reason: "malformed-live", detail: "gen must be a string" }; + } + return { + t: "live", + root: frame.root, + seq: frame.seq as number, + ...(frame.gen !== undefined ? { gen: frame.gen as string } : {}), + }; case "presence": if (typeof frame.root !== "string") { return { t: "err", reason: "malformed-presence" }; diff --git a/test/cli/sync.test.ts b/test/cli/sync.test.ts index 1e5968e..f9fa612 100644 --- a/test/cli/sync.test.ts +++ b/test/cli/sync.test.ts @@ -258,6 +258,107 @@ describe("cursor corruption recovery (dee-inzc blocker)", () => { }); }); +describe("log generation (dee-u6tq): a cursor is only meaningful inside the generation that issued it", () => { + let server: LyncSyncServer | undefined; + let lockedFile: string | undefined; + + afterEach(async () => { + if (lockedFile) await (await import("node:fs/promises")).chmod(lockedFile, 0o644).catch(() => {}); + lockedFile = undefined; + await server?.close(); + server = undefined; + }); + + it("the bug's exact shape: persisted events, a broadcast whose disk write failed consuming a seq, a restart, a stale cursor — the client ends with EVERY persisted event", async () => { + const { chmod } = await import("node:fs/promises"); + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-gen-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-gen-client-")); + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const port = server.port; + const url = `ws://localhost:${port}`; + const roomFile = path.join(serverDir, "story.lync"); + + // N events persisted: a producer contributes e1, e2. + const producer = path.join(clientDir, "producer.lync"); + await writeFile(producer, `${eventLine("e1", [], "one")}\n${eventLine("e2", ["e1"], "two")}\n`); + await syncOnce({ file: producer, url, root: "story", out: quiet, err: quiet }); + expect(idsOf(await readFile(roomFile, "utf8"))).toEqual(["e1", "e2"]); + + // The victim client syncs and saves its cursor — with the generation. + const victim = path.join(clientDir, "victim.lync"); + await writeFile(victim, ""); + await syncOnce({ file: victim, url, root: "story", out: quiet, err: quiet }); + const cursor1 = JSON.parse(await readFile(`${victim}.sync.json`, "utf8")) as { seq: number; generation?: string }; + expect(cursor1.seq).toBe(2); + expect(typeof cursor1.generation).toBe("string"); // additive: persisted alongside seq + + // One broadcast with a FAILED disk write consumes seq 3: the room file + // goes read-only, e3 is accepted and fanned out but never persisted. + lockedFile = roomFile; + await chmod(roomFile, 0o444); + await appendFile(producer, `${eventLine("e3", ["e2"], "three, lost to disk")}\n`); + const producerErrs = collect(); + await syncOnce({ file: producer, url, root: "story", out: quiet, err: producerErrs.io }); + expect(idsOf(await readFile(roomFile, "utf8"))).toEqual(["e1", "e2"]); // not on disk + + // The victim, connected during that generation, advances its cursor to 3. + await syncOnce({ file: victim, url, root: "story", out: quiet, err: quiet }); + const cursor2 = JSON.parse(await readFile(`${victim}.sync.json`, "utf8")) as { seq: number; generation?: string }; + expect(cursor2.seq).toBe(3); + expect(idsOf(await readFile(victim, "utf8"))).toEqual(["e1", "e2", "e3"]); + + // Server restart: the disk heals, the relay recovers e1+e2 from disk and + // mints a NEW generation. seq 3 now means something else entirely. + await chmod(roomFile, 0o644); + lockedFile = undefined; + await server.close(); + server = await startLyncServe({ dir: serverDir, port, log: () => {} }); + + // A fresh writer persists e4 in the new generation (its seq: 3). + const writer = path.join(clientDir, "writer.lync"); + await writeFile(writer, `${eventLine("e4", ["e2"], "four, post-restart")}\n`); + await syncOnce({ file: writer, url, root: "story", out: quiet, err: quiet }); + + // The victim reconnects with its stale cursor {seq:3, gen:old}. Pre-fix + // it subscribed since 3 and silently skipped e4 forever. The generation + // mismatch must force a resync from 0 — loudly. + const victimErrs = collect(); + await syncOnce({ file: victim, url, root: "story", out: quiet, err: victimErrs.io }); + expect(victimErrs.text()).toContain("generation changed"); + const victimIds = idsOf(await readFile(victim, "utf8")); + expect(victimIds).toContain("e4"); // the event the old bug skipped forever + expect(victimIds).toEqual(["e1", "e2", "e3", "e4"]); + // Every event persisted on the relay is in the victim's file... + for (const id of idsOf(await readFile(roomFile, "utf8"))) { + expect(victimIds).toContain(id); + } + // ...and the victim's push restored e3 (lost to the dead disk) to it. + expect(idsOf(await readFile(roomFile, "utf8"))).toContain("e3"); + // The cursor now belongs to the new generation. + const cursor3 = JSON.parse(await readFile(`${victim}.sync.json`, "utf8")) as { seq: number; generation?: string }; + expect(cursor3.generation).not.toBe(cursor2.generation); + expect(cursor3.seq).toBeGreaterThanOrEqual(4); + }); + + it("tolerates a pre-generation cursor file (old client state) without resetting", async () => { + const serverDir = await mkdtemp(path.join(os.tmpdir(), "lync-gen-serve-")); + const clientDir = await mkdtemp(path.join(os.tmpdir(), "lync-gen-client-")); + await writeFile(path.join(serverDir, "tale.lync"), `${eventLine("root", [], "one")}\n`); + server = await startLyncServe({ dir: serverDir, log: () => {} }); + const url = `ws://localhost:${server.port}`; + + const file = path.join(clientDir, "tale.lync"); + await writeFile(file, `${eventLine("root", [], "one")}\n`); + // An old cursor file: no generation field at all. + await writeFile(`${file}.sync.json`, `${JSON.stringify({ url, root: "tale", seq: 1 })}\n`); + + const result = await syncOnce({ file, url, root: "tale", out: quiet, err: quiet }); + expect(result.received).toBe(0); // first gen sighting adopts; no spurious resync + const cursor = JSON.parse(await readFile(`${file}.sync.json`, "utf8")) as { seq: number; generation?: string }; + expect(typeof cursor.generation).toBe("string"); // upgraded in place + }); +}); + describe("conflict sidecar durability (dee-inzc major)", () => { let server: LyncSyncServer | undefined; let lockedDir: string | undefined; diff --git a/test/sync-protocol.test.ts b/test/sync-protocol.test.ts index bee445f..bd30fbe 100644 --- a/test/sync-protocol.test.ts +++ b/test/sync-protocol.test.ts @@ -43,6 +43,33 @@ describe("cursor integrity (dee-inzc blocker)", () => { }); }); +describe("log generation field (dee-u6tq)", () => { + it("round-trips gen on ev and live frames", () => { + const frames: SyncFrame[] = [ + { t: "ev", root: "story", line: '{"id":"a"}', seq: 3, gen: "gen-1" }, + { t: "live", root: "story", seq: 7, gen: "gen-1" }, + ]; + for (const frame of frames) { + expect(decodeFrame(encodeFrame(frame))).toEqual(frame); + } + }); + + it("stays tolerant of gen's absence — old peers decode fine, both directions", () => { + // Old server -> new client: no gen on the wire. + expect(decodeFrame('{"t":"ev","root":"r","line":"{}","seq":2}')).toEqual({ t: "ev", root: "r", line: "{}", seq: 2 }); + expect(decodeFrame('{"t":"live","root":"r","seq":7}')).toEqual({ t: "live", root: "r", seq: 7 }); + // New client -> old server: encoding without gen adds nothing. + expect(encodeFrame({ t: "ev", root: "r", line: "{}" })).not.toContain("gen"); + // An unknown extra field from a NEWER peer is dropped, not fatal. + expect(decodeFrame('{"t":"live","root":"r","seq":7,"gen":"g","future":true}')).toEqual({ t: "live", root: "r", seq: 7, gen: "g" }); + }); + + it("rejects a non-string gen — a cursor reset must never act on noise", () => { + expect(decodeFrame('{"t":"ev","root":"r","line":"{}","seq":2,"gen":42}')).toMatchObject({ t: "err", reason: "malformed-ev" }); + expect(decodeFrame('{"t":"live","root":"r","seq":7,"gen":{}}')).toMatchObject({ t: "err", reason: "malformed-live" }); + }); +}); + describe("uuidv7 minting", () => { it("mints valid, time-ordered UUIDv7", async () => { const { uuidv7 } = await import("@deepfates/lync/uuid"); From e65c5bdd16208c14b8d36e1b3b19a099f4ce2624 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 01:18:31 -0700 Subject: [PATCH 28/33] synced-store: await + inspect every union BEFORE the cursor advances (dee-s6dc) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Frames apply strictly in arrival order (serialized chain). A store-write failure freezes the root's cursor — live frames cannot leapfrog it — and screams through the new additive SyncStatus.failures channel; the line is re-fetched on the next resubscribe. Conflicts and garbage are surfaced too. Also carries the generation reset (dee-u6tq) on the synced-store side, with stale-live counting so a superseded sub cannot re-poison a reset cursor. Neuter-verified: freezing disabled makes the regression tests fail. --- src/synced-store.ts | 118 ++++++++++++++++++-- test/synced-store.test.ts | 222 +++++++++++++++++++++++++++++++++++++- 2 files changed, 332 insertions(+), 8 deletions(-) diff --git a/src/synced-store.ts b/src/synced-store.ts index d0e0db7..76b49c2 100644 --- a/src/synced-store.ts +++ b/src/synced-store.ts @@ -25,6 +25,13 @@ export interface SyncStatus { liveRoots: string[]; /** Ids that arrived as same-id-different-body conflicts, surfaced never resolved. */ conflicts: string[]; + /** + * Every ingest failure, surfaced never swallowed: a remote line the local + * store could not durably accept (store write threw) or rejected as + * garbage. A store-write failure also freezes the root's resume cursor so + * the line is re-fetched on the next resubscribe instead of being skipped. + */ + failures: string[]; } export interface SyncTransport { @@ -61,6 +68,24 @@ export function createSyncedStore( const liveRoots = new Set(); const cursors = new Map(); const conflicts = new Set(); + const failures: string[] = []; + // Server log generation per root. A cursor is only meaningful inside the + // generation that issued it (a failed relay disk write still consumes a + // seq, so a restarted server's recovered log can sit BEHIND our cursor). + // On a generation change the cursor resets to 0 and we resubscribe; union + // makes the re-download duplicate no-ops. + const generations = new Map(); + // Roots whose cursor is frozen because a union failed (store write threw): + // the cursor must not advance past the hole, or the line would be skipped + // forever. Cleared on resync — the resubscribe re-fetches from the frozen + // cursor and each refetched line advances it again as its union succeeds. + const stalledRoots = new Set(); + // Outstanding `sub` frames per root: each sub is answered by exactly one + // `live`, in order. While a generation-reset sub is stacked behind an + // earlier one (count > 1), cursor advances are suppressed — frames from the + // superseded sub carry seqs the reset backlog has not re-covered yet, and + // trusting them would re-poison the freshly reset cursor. + const pendingLives = new Map(); let connection: SyncConnectionState = transport.state; const emitStatus = () => { @@ -68,6 +93,7 @@ export function createSyncedStore( connection, liveRoots: [...liveRoots], conflicts: [...conflicts], + failures: [...failures], }); }; @@ -99,6 +125,12 @@ export function createSyncedStore( for (const event of await inner.byRoot(rootId)) { pushLine(rootId, event.bytes); } + // A frozen cursor thaws here: the sub below re-fetches from it, and each + // refetched line advances it again as its union succeeds. + stalledRoots.delete(rootId); + // A fresh connection: any lives owed by subs on the dead connection will + // never arrive, so the count restarts at this sub's one. + pendingLives.set(rootId, 1); transport.send({ t: "sub", root: rootId, since: cursors.get(rootId) ?? 0 }); } @@ -112,19 +144,78 @@ export function createSyncedStore( emitStatus(); }); - transport.onFrame((frame) => { + /** + * Returns true when the server's generation differs from the one this root's + * cursor belongs to — in which case the cursor has been reset to 0 and a + * fresh `sub` from 0 is already on the wire. Frames without gen (old + * servers) never trigger a reset. + */ + const generationChanged = (root: string, gen: string | undefined): boolean => { + if (gen === undefined) return false; + const known = generations.get(root); + if (known === gen) return false; + generations.set(root, gen); + if (known === undefined) return false; // first sighting: adopt + failures.push(`generation changed for ${root} (${known} -> ${gen}); resyncing from 0`); + cursors.set(root, 0); + stalledRoots.delete(root); + pendingLives.set(root, (pendingLives.get(root) ?? 0) + 1); + emitStatus(); + transport.send({ t: "sub", root, since: 0 }); + return true; + }; + + const advanceCursor = (root: string, seq: number) => { + if (stalledRoots.has(root)) return; // frozen behind a failed union + if ((pendingLives.get(root) ?? 0) > 1) return; // a reset-sub's backlog is still owed + cursors.set(root, Math.max(cursors.get(root) ?? 0, seq)); + }; + + async function handleFrame(frame: SyncFrame): Promise { switch (frame.t) { case "ev": { // Remote line: ingest through union WITHOUT re-pushing (the relay has - // already fanned it out). Subscribers fire via the inner store. - void inner.union(frame.line); - if (typeof frame.seq === "number") { - cursors.set(frame.root, Math.max(cursors.get(frame.root) ?? 0, frame.seq)); + // already fanned it out). Subscribers fire via the inner store. The + // union is awaited and inspected BEFORE the cursor advances — a line + // the local store failed to accept must be re-fetched, never skipped. + generationChanged(frame.root, frame.gen); + let outcome: AppendResult; + try { + outcome = await inner.union(frame.line); + } catch (error) { + // Store write failed: the line is NOT durable locally. Freeze the + // cursor so the next resubscribe re-fetches it, and scream. + failures.push(`store failed to ingest a synced line for ${frame.root}: ${String(error)}`); + stalledRoots.add(frame.root); + emitStatus(); + return; + } + switch (outcome.status) { + case "conflict": + conflicts.add(outcome.event.body.id); + emitStatus(); + break; + case "garbage": + // Unusable bytes stay unusable on any re-fetch: surfaced loudly, + // and the cursor may advance past them. + failures.push(`synced line rejected as garbage for ${frame.root}: ${outcome.reason}`); + emitStatus(); + break; + default: + break; // added / duplicate / buffered: durably in the store's hands } + if (typeof frame.seq === "number") advanceCursor(frame.root, frame.seq); return; } case "live": { - cursors.set(frame.root, Math.max(cursors.get(frame.root) ?? 0, frame.seq)); + // A stale live is not live: either its generation is dead (the + // resubscribe from 0 is already on the wire) or it answers a sub a + // generation reset has since superseded. Wait for the real one. + const changed = generationChanged(frame.root, frame.gen); + const outstanding = Math.max(0, (pendingLives.get(frame.root) ?? 1) - 1); + pendingLives.set(frame.root, outstanding); + if (changed || outstanding > 0) return; + advanceCursor(frame.root, frame.seq); liveRoots.add(frame.root); emitStatus(); return; @@ -143,6 +234,19 @@ export function createSyncedStore( default: return; } + } + + // Frames apply strictly in arrival order: each union is awaited before the + // next frame is touched, so a slow union can never let a later frame (or a + // `live` cursor jump) leapfrog a failure. handleFrame never rejects — the + // catch above is the only throw path and it returns — but the chain guards + // anyway so one surprise cannot wedge sync forever. + let frameChain: Promise = Promise.resolve(); + transport.onFrame((frame) => { + frameChain = frameChain.then( + () => handleFrame(frame), + () => handleFrame(frame), + ); }); return { @@ -170,7 +274,7 @@ export function createSyncedStore( ...(inner.diagnostics ? { diagnostics: () => inner.diagnostics!() } : {}), syncRoot: ensureSynced, presence: (root, data) => transport.send({ t: "presence", root, data }), - status: () => ({ connection, liveRoots: [...liveRoots], conflicts: [...conflicts] }), + status: () => ({ connection, liveRoots: [...liveRoots], conflicts: [...conflicts], failures: [...failures] }), close: () => transport.close(), }; } diff --git a/test/synced-store.test.ts b/test/synced-store.test.ts index 9cbbd6a..833f6b3 100644 --- a/test/synced-store.test.ts +++ b/test/synced-store.test.ts @@ -8,7 +8,7 @@ import { type SyncTransport, } from "@deepfates/lync/synced-store"; import type { SyncFrame } from "@deepfates/lync/sync-protocol"; -import { serializeLyncEvent } from "@deepfates/lync/store"; +import { serializeLyncEvent, type AppendResult, type EventStore, type StoredEvent } from "@deepfates/lync/store"; function mockTransport(initial: SyncConnectionState = "online") { const frameHandlers = new Set<(frame: SyncFrame) => void>(); @@ -94,6 +94,9 @@ describe("createSyncedStore", () => { store.syncRoot("r1"); mock.inject({ t: "live", root: "r1", seq: 5 }); mock.inject({ t: "err", root: "r1", reason: "same-id-conflict", detail: "dup-id" }); + // Frames apply in a serialized chain (unions are awaited in arrival + // order), so settle before reading status. + await new Promise((r) => setTimeout(r, 10)); const status = store.status(); expect(status.liveRoots).toContain("r1"); @@ -117,3 +120,220 @@ describe("createSyncedStore", () => { expect(after.some((f) => f.t === "ev")).toBe(true); // backlog re-pushed }); }); + +/** + * An EventStore decorator whose union can be made to fail on command — the + * shape of a full disk or a dead IndexedDB. Records call/finish order so + * ordering tests can prove unions are serialized. + */ +function breakableStore(inner: EventStore) { + let broken = false; + let delayNextMs = 0; + const unionLog: string[] = []; + const store: EventStore = { + append: (ev) => inner.append(ev), + union: async (line: string): Promise => { + unionLog.push(`start:${JSON.parse(line).id}`); + const delay = delayNextMs; + delayNextMs = 0; + if (delay > 0) await new Promise((r) => setTimeout(r, delay)); + if (broken) { + unionLog.push(`fail:${JSON.parse(line).id}`); + throw new Error("injected store-write failure"); + } + const result = await inner.union(line); + unionLog.push(`done:${JSON.parse(line).id}`); + return result; + }, + byId: (id) => inner.byId(id), + byRoot: (rootId) => inner.byRoot(rootId), + subscribe: (rootId, listener) => inner.subscribe(rootId, listener), + roots: (kind) => inner.roots(kind), + }; + return { + store, + unionLog, + setBroken: (b: boolean) => (broken = b), + delayNext: (ms: number) => (delayNextMs = ms), + }; +} + +const settle = () => new Promise((r) => setTimeout(r, 25)); + +describe("awaited union (dee-s6dc): the receive cursor advances only on inspected success", () => { + const line = (id: string, parents: string[], text: string) => serializeLyncEvent(body(id, parents, text)); + + it("a failed union on frame k freezes the cursor at k-1, surfaces the failure, and the event applies after heal + resubscribe", async () => { + const statuses: SyncStatus[] = []; + const mock = mockTransport(); + const flaky = breakableStore(createMemoryEventStore()); + const store = createSyncedStore(flaky.store, mock.transport, { onStatus: (s) => statuses.push(s) }); + store.syncRoot("r1"); + await settle(); + + // Frame 1 lands; the store then breaks; frame 2 (seq k=2) fails; frame 3 + // still applies (arrival order is preserved past the failure). + mock.inject({ t: "ev", root: "r1", seq: 1, line: line("r1", [], "one") }); + await settle(); + flaky.setBroken(true); + mock.inject({ t: "ev", root: "r1", seq: 2, line: line("lost", ["r1"], "two, refused by disk") }); + await settle(); + flaky.setBroken(false); + mock.inject({ t: "ev", root: "r1", seq: 3, line: line("later", ["r1"], "three") }); + mock.inject({ t: "live", root: "r1", seq: 3 }); + await settle(); + + // The failure screamed through onStatus and status(). + expect(store.status().failures.some((f) => f.includes("injected store-write failure"))).toBe(true); + expect(statuses.some((s) => s.failures.some((f) => f.includes("injected store-write failure")))).toBe(true); + // The failed line is NOT in the store; the later one is (order held). + expect(await store.byId("lost")).toBeNull(); + expect(await store.byId("later")).not.toBeNull(); + + // The cursor stayed at k-1 = 1: the resubscribe after reconnect asks the + // relay for everything from there — the lost line gets re-fetched. + mock.setState("offline"); + mock.setState("online"); + mock.open(); + await settle(); + const resub = mock.sent.filter((f) => f.t === "sub" && f.root === "r1").at(-1)!; + expect(resub).toMatchObject({ t: "sub", since: 1 }); + + // The store has healed; the relay replays from the cursor; all applies. + mock.inject({ t: "ev", root: "r1", seq: 2, line: line("lost", ["r1"], "two, refused by disk") }); + mock.inject({ t: "ev", root: "r1", seq: 3, line: line("later", ["r1"], "three") }); + mock.inject({ t: "live", root: "r1", seq: 3 }); + await settle(); + expect(await store.byId("lost")).not.toBeNull(); + + // And the cursor thawed: the next resubscribe resumes past the hole. + mock.setState("offline"); + mock.setState("online"); + mock.open(); + await settle(); + const finalSub = mock.sent.filter((f) => f.t === "sub" && f.root === "r1").at(-1)!; + expect(finalSub).toMatchObject({ t: "sub", since: 3 }); + }); + + it("a live frame cannot leapfrog a failed union: the frozen cursor wins over live's seq", async () => { + const mock = mockTransport(); + const flaky = breakableStore(createMemoryEventStore()); + const store = createSyncedStore(flaky.store, mock.transport, {}); + store.syncRoot("r1"); + await settle(); + + flaky.setBroken(true); + mock.inject({ t: "ev", root: "r1", seq: 1, line: line("r1", [], "refused") }); + mock.inject({ t: "live", root: "r1", seq: 5 }); // relay is far ahead + await settle(); + flaky.setBroken(false); + + mock.setState("offline"); + mock.setState("online"); + mock.open(); + await settle(); + const resub = mock.sent.filter((f) => f.t === "sub" && f.root === "r1").at(-1)!; + expect(resub).toMatchObject({ t: "sub", since: 0 }); // NOT 5 + }); + + it("unions apply strictly in arrival order — a slow union never lets a later frame pass it", async () => { + const mock = mockTransport(); + const flaky = breakableStore(createMemoryEventStore()); + const store = createSyncedStore(flaky.store, mock.transport, {}); + store.syncRoot("r1"); + await settle(); + + flaky.delayNext(60); // frame 1's union is slow + mock.inject({ t: "ev", root: "r1", seq: 1, line: line("r1", [], "slow") }); + mock.inject({ t: "ev", root: "r1", seq: 2, line: line("fast", ["r1"], "fast") }); + await new Promise((r) => setTimeout(r, 150)); + + expect(await store.byId("fast")).not.toBeNull(); + // The second union START comes after the first union DONE: serialized. + const relevant = flaky.unionLog.filter((entry) => entry.endsWith(":r1") || entry.endsWith(":fast")); + expect(relevant).toEqual(["start:r1", "done:r1", "start:fast", "done:fast"]); + }); + + it("garbage from the relay is surfaced, never silently skipped — and never wedges the cursor", async () => { + const statuses: SyncStatus[] = []; + const mock = mockTransport(); + const store = createSyncedStore(createMemoryEventStore(), mock.transport, { onStatus: (s) => statuses.push(s) }); + store.syncRoot("r1"); + await settle(); + + mock.inject({ t: "ev", root: "r1", seq: 1, line: '{"id":"junk","not":"a lync event"}' }); + mock.inject({ t: "ev", root: "r1", seq: 2, line: line("r1", [], "real") }); + mock.inject({ t: "live", root: "r1", seq: 2 }); + await settle(); + + expect(store.status().failures.some((f) => f.includes("garbage"))).toBe(true); + expect(await store.byId("r1")).not.toBeNull(); + // Unusable bytes stay unusable on any re-fetch: the cursor moves past them. + mock.setState("offline"); + mock.setState("online"); + mock.open(); + await settle(); + const resub = mock.sent.filter((f) => f.t === "sub" && f.root === "r1").at(-1)!; + expect(resub).toMatchObject({ t: "sub", since: 2 }); + }); +}); + +describe("generation change in the synced store (dee-u6tq)", () => { + const line = (id: string, parents: string[], text: string) => serializeLyncEvent(body(id, parents, text)); + + it("resets the cursor to 0 and resubscribes when the server's generation changes", async () => { + const statuses: SyncStatus[] = []; + const mock = mockTransport(); + const store = createSyncedStore(createMemoryEventStore(), mock.transport, { onStatus: (s) => statuses.push(s) }); + store.syncRoot("r1"); + await settle(); + + // Generation g1: three events, cursor 3. + mock.inject({ t: "ev", root: "r1", seq: 1, line: line("r1", [], "one"), gen: "g1" }); + mock.inject({ t: "ev", root: "r1", seq: 2, line: line("two", ["r1"], "two"), gen: "g1" }); + mock.inject({ t: "ev", root: "r1", seq: 3, line: line("three", ["r1"], "three"), gen: "g1" }); + mock.inject({ t: "live", root: "r1", seq: 3, gen: "g1" }); + await settle(); + + // The server restarted (lost the unpersisted third event): new generation, + // and its live sits BEHIND our cursor. Pre-fix we would idle forever and + // silently skip the next persisted event. + const before = mock.sent.length; + mock.inject({ t: "live", root: "r1", seq: 2, gen: "g2" }); + await settle(); + + const resub = mock.sent.slice(before).filter((f) => f.t === "sub" && f.root === "r1"); + expect(resub).toEqual([{ t: "sub", root: "r1", since: 0 }]); + // The reset is surfaced, not silent. + expect(store.status().failures.some((f) => f.includes("generation changed"))).toBe(true); + + // The new generation's backlog replays; a NEW event (seq 3 in g2) lands. + mock.inject({ t: "ev", root: "r1", seq: 1, line: line("r1", [], "one"), gen: "g2" }); + mock.inject({ t: "ev", root: "r1", seq: 2, line: line("two", ["r1"], "two"), gen: "g2" }); + mock.inject({ t: "ev", root: "r1", seq: 3, line: line("fresh", ["r1"], "post-restart"), gen: "g2" }); + mock.inject({ t: "live", root: "r1", seq: 3, gen: "g2" }); + await settle(); + expect(await store.byId("fresh")).not.toBeNull(); + + // Cursor now belongs to g2: next resubscribe resumes from 3. + mock.setState("offline"); + mock.setState("online"); + mock.open(); + await settle(); + const finalSub = mock.sent.filter((f) => f.t === "sub" && f.root === "r1").at(-1)!; + expect(finalSub).toMatchObject({ t: "sub", since: 3 }); + }); + + it("frames without gen (old server) never trigger a reset", async () => { + const mock = mockTransport(); + const store = createSyncedStore(createMemoryEventStore(), mock.transport, {}); + store.syncRoot("r1"); + await settle(); + mock.inject({ t: "ev", root: "r1", seq: 1, line: line("r1", [], "one") }); + mock.inject({ t: "live", root: "r1", seq: 1 }); + await settle(); + expect(mock.sent.filter((f) => f.t === "sub" && f.since === 0)).toHaveLength(1); // only the original + expect(store.status().failures).toEqual([]); + }); +}); + From 39a30d4c7c9e63b7d698866f32aa01cd92b8605a Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 01:27:12 -0700 Subject: [PATCH 29/33] synced-store: surface every relay-side error, never silently drop it (dee-i1wc) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The synced store swallowed all relay errs except same-id-conflict — a persist-failed durability failure fanned out but never reached the client's status channel, so a store consumer could not see the relay lose a write. Route every non-conflict relay err (persist-failed, conflict-persist-failed, recovered-damaged-tail, line-without-id, server-error, ...) into the failures channel. Nothing fails invisibly. Unit test asserts persist-failed surfaces and is not miscategorised as a conflict. --- src/synced-store.ts | 10 +++++++++- test/synced-store.test.ts | 17 +++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/src/synced-store.ts b/src/synced-store.ts index 76b49c2..c5fd74c 100644 --- a/src/synced-store.ts +++ b/src/synced-store.ts @@ -227,8 +227,16 @@ export function createSyncedStore( case "err": { if (frame.reason === "same-id-conflict" && frame.detail) { conflicts.add(frame.detail); - emitStatus(); + } else { + // Every other relay-side failure — persist-failed, conflict-persist-failed, + // recovered-damaged-tail, line-without-id, unexpected-live-from-client, + // server-error — is a failure the client must see, never a silent drop. + // A durability failure on the relay reaches the client's status channel. + const where = frame.root ? ` for ${frame.root}` : ""; + const detail = frame.detail ? ` (${frame.detail})` : ""; + failures.push(`relay error${where}: ${frame.reason}${detail}`); } + emitStatus(); return; } default: diff --git a/test/synced-store.test.ts b/test/synced-store.test.ts index 833f6b3..31875a1 100644 --- a/test/synced-store.test.ts +++ b/test/synced-store.test.ts @@ -104,6 +104,23 @@ describe("createSyncedStore", () => { expect(statuses.length).toBeGreaterThanOrEqual(2); }); + it("surfaces every non-conflict relay error into the failures channel, never silently drops it", async () => { + const statuses: SyncStatus[] = []; + const mock = mockTransport(); + const store = createSyncedStore(createMemoryEventStore(), mock.transport, { + onStatus: (s) => statuses.push(s), + }); + store.syncRoot("r1"); + // A relay durability failure: the fan-out landed but the disk write did not. + mock.inject({ t: "err", root: "r1", reason: "persist-failed", detail: "evt-9" }); + await new Promise((r) => setTimeout(r, 10)); + + const status = store.status(); + expect(status.failures.some((f) => f.includes("persist-failed") && f.includes("evt-9"))).toBe(true); + expect(status.conflicts).toEqual([]); // a persist failure is not a conflict + expect(statuses.some((s) => s.failures.some((f) => f.includes("persist-failed")))).toBe(true); + }); + it("re-pushes local backlog and re-subscribes on reconnect", async () => { const mock = mockTransport("online"); const store = createSyncedStore(createMemoryEventStore(), mock.transport); From 48e4690c11a0784390cd2732a4ad515440670241 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 01:27:26 -0700 Subject: [PATCH 30/33] =?UTF-8?q?test:=20the=20loss-free=20trial=20?= =?UTF-8?q?=E2=80=94=20milestone-6=20durability=20proof=20(dee-i1wc)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A real relay plus real synced stores over the global WebSocket, run through the full gauntlet against one shared root: (a) a client's socket drops mid-stream and auto-reconnects; (b) the relay's .lync file goes read-only, so a write is fanned out but refused by disk and surfaced as persist-failed; (c) the server restarts into a new log generation. Final invariant: every event a client successfully appended — including the one the dead disk refused — ends up in every other client's store AND on the relay's on-disk .lync file, and every failure (persist-failed, generation-changed) was surfaced on every client's status channel. Neuter-verified: dropping the err surfacing fails leg (b); neutering the generation reset fails leg (c). --- test/cli/loss-free-trial.integration.test.ts | 231 +++++++++++++++++++ 1 file changed, 231 insertions(+) create mode 100644 test/cli/loss-free-trial.integration.test.ts diff --git a/test/cli/loss-free-trial.integration.test.ts b/test/cli/loss-free-trial.integration.test.ts new file mode 100644 index 0000000..3455e8e --- /dev/null +++ b/test/cli/loss-free-trial.integration.test.ts @@ -0,0 +1,231 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { chmod, mkdtemp, readFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createSyncedStore, createWebSocketTransport, type SyncStatus } from "@deepfates/lync/synced-store"; +import { startLyncServe, type LyncSyncServer } from "@deepfates/lync/relay"; + +/** + * The loss-free trial (dee-i1wc) — the world-charter milestone-6 proof. + * + * A real relay plus real synced stores over the global WebSocket, run through + * the full durability gauntlet, in three legs against ONE shared root: + * + * (a) a client disconnects mid-stream and reconnects; + * (b) the relay's storage fails mid-run (the .lync file goes read-only) and + * then recovers; + * (c) the server process restarts — a new log generation. + * + * The invariant asserted after the gauntlet, and the surfacing asserted per + * leg: every event any client SUCCESSFULLY appended (local append returned + * `added`) ends up in every other client's store AND in the relay's on-disk + * .lync file — nothing lost, nothing silently skipped — and every failure that + * occurred was surfaced through a status/err channel, not swallowed. + * + * Why leg (b)'s lost-to-disk event only reaches disk after leg (c): a relay + * whose write fails still fans the event out and holds it in memory, so a + * same-generation re-push is a byId duplicate no-op — the line never re-hits + * disk until a restart drops the in-memory copy, mints a new generation, and + * the clients resync from 0 and re-push it fresh. That is the loss-free + * property under a storage failure: the generation reset is what closes it. + */ + +const ROOT = "trial"; + +function idsOf(text: string): string[] { + return text + .split("\n") + .filter((line) => line.length > 0) + .map((line) => { + try { + return (JSON.parse(line) as { id?: string }).id ?? ""; + } catch { + return ""; + } + }); +} + +/** + * A WebSocket subclass that records every instance it constructs, so a test + * can force-close ONE client's live socket — a network drop for that client + * alone, leaving the transport's auto-reconnect to bring it back. + */ +function trackedWebSocket(): { impl: typeof WebSocket; sockets: WebSocket[] } { + const sockets: WebSocket[] = []; + const Real = (globalThis as { WebSocket: typeof WebSocket }).WebSocket; + class Tracked extends Real { + constructor(url: string | URL, protocols?: string | string[]) { + super(url, protocols); + sockets.push(this as unknown as WebSocket); + } + } + return { impl: Tracked as unknown as typeof WebSocket, sockets }; +} + +function makeClient(url: string, actor: string) { + const inner = createMemoryEventStore(); + const tracker = trackedWebSocket(); + const statuses: SyncStatus[] = []; + const transport = createWebSocketTransport(url, { reconnectMs: 30, WebSocketImpl: tracker.impl }); + const store = createSyncedStore(inner, transport, { onStatus: (s) => statuses.push(s) }); + // Every appended id whose LOCAL append returned "added" — the events the + // trial promises never to lose. + const appended = new Set(); + const append = async (id: string, parents: string[], text: string) => { + const result = await store.append({ + v: 1, + id, + kind: "lync/artifact", + at: "2026-07-08T21:00:00Z", + author: { actor }, + parents, + payload: { text }, + }); + if (result.status === "added") appended.add(id); + return result; + }; + const sawFailure = (needle: string) => + statuses.some((s) => s.failures.some((f) => f.includes(needle))) || + store.status().failures.some((f) => f.includes(needle)); + return { actor, store, statuses, appended, append, sawFailure, dropSocket: () => tracker.sockets.at(-1)?.close() }; +} +type Client = ReturnType; + +async function waitFor(check: () => Promise | boolean, timeoutMs = 8_000): Promise { + const deadline = Date.now() + timeoutMs; + let last = "condition not met"; + while (Date.now() < deadline) { + try { + if (await check()) return; + } catch (error) { + last = String(error); + } + await new Promise((r) => setTimeout(r, 25)); + } + throw new Error(`waitFor: ${last} within ${timeoutMs}ms`); +} + +describe("loss-free trial (dee-i1wc): the milestone-6 durability proof", () => { + let server: LyncSyncServer | undefined; + let clients: Client[] = []; + + afterEach(async () => { + for (const c of clients) c.store.close(); + clients = []; + await server?.close(); + server = undefined; + }); + + it("runs the full gauntlet (disconnect, storage failure, restart) and loses nothing, hiding nothing", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "lync-loss-free-")); + const roomFile = path.join(dir, `${ROOT}.lync`); + server = await startLyncServe({ dir, log: () => {} }); + const port = server.port; + const url = `ws://localhost:${port}`; + + const a = makeClient(url, "alice"); + const b = makeClient(url, "bob"); + const c = makeClient(url, "carol"); + clients = [a, b, c]; + + // All three sync the shared root and reach live. + for (const cl of clients) cl.store.syncRoot(ROOT); + // The union of everything every client appended — the loss-free promise set. + const promised = () => new Set([...a.appended, ...b.appended, ...c.appended]); + + // Every promised id is durable in every client's store AND on the relay's + // on-disk .lync file. Polls to absorb replication/persist latency. + const assertConverged = async (label: string, extraTimeout = 8_000) => { + const want = [...promised()]; + await waitFor(async () => { + for (const cl of clients) { + for (const id of want) if ((await cl.store.byId(id)) === null) return false; + } + const onDisk = new Set(idsOf(await readFile(roomFile, "utf8"))); + return want.every((id) => onDisk.has(id)); + }, extraTimeout).catch(async (error) => { + // Loud, never a silent skip: report exactly what is missing where. + const onDisk = new Set(idsOf(await readFile(roomFile, "utf8").catch(() => ""))); + const missingDisk = want.filter((id) => !onDisk.has(id)); + const missingStores: string[] = []; + for (const cl of clients) + for (const id of want) if ((await cl.store.byId(id)) === null) missingStores.push(`${cl.actor}:${id}`); + throw new Error(`${label}: not converged — off disk [${missingDisk}], missing in stores [${missingStores}] (${error})`); + }); + }; + + // Root event first (a real root so children have a parent to attach to). + await a.append(ROOT, [], "the trial begins"); + await assertConverged("seed"); + + // ---- Leg (a): a client disconnects mid-stream and reconnects ---------- + // Bob's socket drops while Alice keeps appending. Bob must catch up on + // reconnect with nothing skipped. + b.dropSocket(); + await waitFor(() => b.store.status().connection !== "online"); + await a.append("a1", [ROOT], "appended while bob is dark"); + await a.append("a2", ["a1"], "and another"); + // Bob's transport auto-reconnects (reconnectMs) and resyncs from its cursor. + await waitFor(() => b.store.status().connection === "online"); + await assertConverged("leg-a disconnect/reconnect"); + expect(await b.store.byId("a1")).not.toBeNull(); + expect(await b.store.byId("a2")).not.toBeNull(); + + // ---- Leg (b): the relay's storage fails mid-run, then recovers -------- + // The .lync file goes read-only. Carol appends x1: the relay fans it out + // to every client (so it is in every store) but the disk write fails and + // is surfaced as `persist-failed`. The event is NOT yet on disk — that is + // the point; leg (c)'s generation reset is what restores it. + await chmod(roomFile, 0o444); + const diskBeforeFail = new Set(idsOf(await readFile(roomFile, "utf8"))); + await c.append("x1", ["a2"], "carol's line, refused by the disk"); + // x1 reaches every client's store despite the failed persist... + await waitFor(async () => { + for (const cl of clients) if ((await cl.store.byId("x1")) === null) return false; + return true; + }); + // ...and the durability failure screamed through the status channel on + // every subscriber (asserting the surfacing, not just the recovery). + await waitFor(() => a.sawFailure("persist-failed") && b.sawFailure("persist-failed") && c.sawFailure("persist-failed")); + expect(a.sawFailure("persist-failed")).toBe(true); + expect(b.sawFailure("persist-failed")).toBe(true); + expect(c.sawFailure("persist-failed")).toBe(true); + expect(diskBeforeFail.has("x1")).toBe(false); // never hit disk + expect(new Set(idsOf(await readFile(roomFile, "utf8"))).has("x1")).toBe(false); + // Storage heals. + await chmod(roomFile, 0o644); + + // ---- Leg (c): the server process restarts — a new generation --------- + // The recovered log (from disk) lacks x1 and its seq sits behind the + // clients' cursors. On restart every client detects the generation change, + // resyncs from 0, and re-pushes its backlog — including x1, which the + // fresh room now persists. A post-restart event also flows end to end. + await server.close(); + server = await startLyncServe({ dir, port, log: () => {} }); + await waitFor(() => clients.every((cl) => cl.store.status().connection === "online")); + // The generation reset was surfaced on every client, not silently applied. + await waitFor(() => clients.every((cl) => cl.sawFailure("generation changed"))); + for (const cl of clients) expect(cl.sawFailure("generation changed")).toBe(true); + + // A brand-new event in the new generation, appended by Alice post-restart. + await a.append("post", ["a2"], "after the restart, still one story"); + + // ---- Overall: nothing lost, nothing hidden -------------------------- + // Every promised event — including x1, the one the dead disk refused — is + // now in every client's store AND on the relay's on-disk .lync file. + await assertConverged("overall (post-restart, disk restored)", 12_000); + const finalDisk = new Set(idsOf(await readFile(roomFile, "utf8"))); + for (const id of [ROOT, "a1", "a2", "x1", "post"]) { + expect(finalDisk.has(id)).toBe(true); + } + // The event the dead disk refused survived to disk via the generation reset. + expect(finalDisk.has("x1")).toBe(true); + // And every client converged on the full set. + for (const cl of clients) { + for (const id of [ROOT, "a1", "a2", "x1", "post"]) { + expect(await cl.store.byId(id)).not.toBeNull(); + } + } + }, 30_000); +}); From f9bb1c4965261d2a43a82b77dab4223651908599 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 01:45:16 -0700 Subject: [PATCH 31/33] relay: converge the on-disk log without a restart (dee-1pfp) A relay whose disk write failed kept the line in memory and broadcast it, but a same-generation re-push was a byId duplicate no-op that never retried the write. The durable log stayed silently incomplete until a restart rebuilt byId from disk -- undercutting the format's thesis that the saved log is the truth. Now each room tracks its unpersisted lines (ordered by id = append order). persistPending() drains them, in order, as one serialized unit before the next append to the room, and a same-line re-push retries the write instead of no-oping. A successful flush clears the line; a still-dead disk stops the drain (later lines stay pending, never reordered) and re-surfaces persist-failed. The on-disk log heals on its own, no restart required. Proof: test/cli/loss-free-trial.heal-without-restart.integration.test.ts -- disk fails for X (X reaches all clients, off disk, persist-failed surfaces), disk heals, another client appends Y to the same root with NO restart, and both X and Y land on disk in append order and in every client's store. The retry is load-bearing: neuter the pending flush and the leg fails. --- src/relay/relay.ts | 73 +++++++- ...l.heal-without-restart.integration.test.ts | 169 ++++++++++++++++++ 2 files changed, 237 insertions(+), 5 deletions(-) create mode 100644 test/cli/loss-free-trial.heal-without-restart.integration.test.ts diff --git a/src/relay/relay.ts b/src/relay/relay.ts index 8c08861..b683d65 100644 --- a/src/relay/relay.ts +++ b/src/relay/relay.ts @@ -99,6 +99,14 @@ interface Room { seq: number; lines: string[]; byId: Map; + /** + * Lines accepted into memory (byId/lines) and broadcast, but NOT yet on + * disk because an append failed. Keyed by id, insertion order = append + * order — the relay's durable log must converge on this, in order, with no + * restart. Drained before the next append to the room and on a same-line + * re-push; a line clears only once its bytes reach disk. + */ + unpersisted: Map; subscribers: Set; writeChain: Promise; recoveryNote?: string; @@ -184,7 +192,20 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { } const existing = room.byId.get(id); if (existing !== undefined) { - if (existing === frame.line) return; // duplicate: a no-op by union + if (existing === frame.line) { + // Duplicate under union — normally a pure no-op. But if this line + // is still not on disk (a prior append failed), the re-push is our + // chance to heal without a restart: retry it (and any earlier + // pending line, in order). Nothing fails invisibly — a still-dead + // disk re-surfaces persist-failed. + if (room.unpersisted.has(id)) { + const { failedId } = await persistPending(room); + if (failedId !== undefined) { + broadcast(room, { t: "err", root: room.root, reason: "persist-failed", detail: failedId }); + } + } + return; + } const kept = await appendSerialized(room, join(options.dir, `${room.root}.conflicts`), frame.line); broadcast(room, { t: "err", root: room.root, reason: "same-id-conflict", detail: id }, socket); send(socket, { t: "err", root: room.root, reason: "same-id-conflict", detail: id }); @@ -200,12 +221,17 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { room.lines.push(frame.line); room.seq += 1; const seq = room.seq; - const persisted = await appendSerialized(room, join(options.dir, `${room.root}.lync`), frame.line); + // Before writing this line, first drain any earlier lines that failed + // to persist — the on-disk log converges here, in order, with no + // restart. If an earlier line is still unwritable the disk write for + // this one is deferred too (it must not jump ahead), and this line + // joins the pending set to be retried on the next activity or re-push. + const { failedId } = await persistPending(room, { id, line: frame.line }); // Live delivery is the relay's primary job: fan out even if the disk // write failed. A durability failure is surfaced loudly, never hidden. broadcast(room, { t: "ev", root: room.root, seq, line: frame.line, gen: room.generation }); - if (!persisted.ok) { - broadcast(room, { t: "err", root: room.root, reason: "persist-failed", detail: id }); + if (failedId !== undefined) { + broadcast(room, { t: "err", root: room.root, reason: "persist-failed", detail: failedId }); } return; } @@ -237,7 +263,7 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { } async function recoverRoom(root: string): Promise { - const room: Room = { root, generation: randomUUID(), seq: 0, lines: [], byId: new Map(), subscribers: new Set(), writeChain: Promise.resolve() }; + const room: Room = { root, generation: randomUUID(), seq: 0, lines: [], byId: new Map(), unpersisted: new Map(), subscribers: new Set(), writeChain: Promise.resolve() }; const path = join(options.dir, `${root}.lync`); if (!existsSync(path)) return room; const text = await readFile(path, "utf8"); @@ -281,6 +307,43 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { return attempt; } + // Converge the room's on-disk log with no restart. As ONE serialized unit + // on the room's writeChain, write every currently-unpersisted line in append + // order, then `tail` (a freshly accepted line, if any). Each line that lands + // clears from `unpersisted`; the first failure stops the run and leaves that + // line and every LATER one pending, in order — a later line is never written + // ahead of an earlier one for the same room. Lines already on disk are never + // re-written (only the pending set and the new tail are touched), and each + // append writes one whole `line\n`, so a partial failure never corrupts the + // file. Returns the id of the first line that still could not persist, or + // undefined if everything (including tail) reached disk. + function persistPending(room: Room, tail?: { id: string; line: string }): Promise<{ failedId?: string }> { + const path = join(options.dir, `${room.root}.lync`); + const attempt = room.writeChain.catch(() => undefined).then(async (): Promise<{ failedId?: string }> => { + const queue: Array<[string, string]> = [...room.unpersisted]; + if (tail) queue.push([tail.id, tail.line]); + for (let index = 0; index < queue.length; index += 1) { + const [id, line] = queue[index]; + try { + await appendFile(path, `${line}\n`); + room.unpersisted.delete(id); + } catch (error) { + log(`[lync relay] persist failed for ${path}: ${String(error)}`); + // This line and every later one stay pending, in append order, so + // the next activity retries from here without reordering the log. + for (let rest = index; rest < queue.length; rest += 1) { + const [pendingId, pendingLine] = queue[rest]; + if (!room.unpersisted.has(pendingId)) room.unpersisted.set(pendingId, pendingLine); + } + return { failedId: id }; + } + } + return {}; + }); + room.writeChain = attempt.then(() => undefined, () => undefined); + return attempt; + } + function broadcast(room: Room, frame: SyncFrame, except?: LyncRelaySocket): void { const encoded = encodeFrame(frame); for (const subscriber of room.subscribers) { diff --git a/test/cli/loss-free-trial.heal-without-restart.integration.test.ts b/test/cli/loss-free-trial.heal-without-restart.integration.test.ts new file mode 100644 index 0000000..f762fb3 --- /dev/null +++ b/test/cli/loss-free-trial.heal-without-restart.integration.test.ts @@ -0,0 +1,169 @@ +import { afterEach, describe, expect, it } from "vitest"; +import { chmod, mkdtemp, readFile } from "node:fs/promises"; +import os from "node:os"; +import path from "node:path"; +import { createMemoryEventStore } from "@deepfates/lync/memory-log"; +import { createSyncedStore, createWebSocketTransport, type SyncStatus } from "@deepfates/lync/synced-store"; +import { startLyncServe, type LyncSyncServer } from "@deepfates/lync/relay"; + +/** + * Heal-without-restart (dee-1pfp) — the last known durability hole in the relay. + * + * The sibling of the loss-free trial. That trial proves a disk failure heals + * across a server RESTART (the generation reset re-pushes the lost line). This + * proves the relay's on-disk log converges ON ITS OWN, with NO restart: + * + * 1. The relay's `.lync` goes read-only. Alice appends X: it fans out + * to every client (in every store) but the disk write fails and + * `persist-failed` surfaces. X is NOT on disk — the relay holds it as + * pending-unpersisted, in memory, broadcast, but off the durable log. + * 2. The disk heals. NO server restart, NO reconnect, NO new generation. + * 3. Bob appends Y to the SAME root. That next activity first flushes the + * pending X (now that the disk is writable), in append order, then Y. + * + * Assert: BOTH X and Y are on the relay's on-disk .lync file, and BOTH are in + * every client's store. The previously-refused X reached disk via the + * next-activity flush alone — the retry is load-bearing (neuter it and X never + * lands without a restart, and this test fails). + */ + +const ROOT = "heal"; + +function idsOf(text: string): string[] { + return text + .split("\n") + .filter((line) => line.length > 0) + .map((line) => { + try { + return (JSON.parse(line) as { id?: string }).id ?? ""; + } catch { + return ""; + } + }); +} + +function makeClient(url: string, actor: string) { + const inner = createMemoryEventStore(); + const statuses: SyncStatus[] = []; + const transport = createWebSocketTransport(url, { reconnectMs: 30 }); + const store = createSyncedStore(inner, transport, { onStatus: (s) => statuses.push(s) }); + const appended = new Set(); + const append = async (id: string, parents: string[], text: string) => { + const result = await store.append({ + v: 1, + id, + kind: "lync/artifact", + at: "2026-07-08T21:00:00Z", + author: { actor }, + parents, + payload: { text }, + }); + if (result.status === "added") appended.add(id); + return result; + }; + const sawFailure = (needle: string) => + statuses.some((s) => s.failures.some((f) => f.includes(needle))) || + store.status().failures.some((f) => f.includes(needle)); + return { actor, store, statuses, appended, append, sawFailure }; +} +type Client = ReturnType; + +async function waitFor(check: () => Promise | boolean, timeoutMs = 8_000): Promise { + const deadline = Date.now() + timeoutMs; + let last = "condition not met"; + while (Date.now() < deadline) { + try { + if (await check()) return; + } catch (error) { + last = String(error); + } + await new Promise((r) => setTimeout(r, 25)); + } + throw new Error(`waitFor: ${last} within ${timeoutMs}ms`); +} + +describe("heal-without-restart (dee-1pfp): the relay's on-disk log converges with no restart", () => { + let server: LyncSyncServer | undefined; + let clients: Client[] = []; + + afterEach(async () => { + for (const c of clients) c.store.close(); + clients = []; + await server?.close(); + server = undefined; + }); + + it("a transiently-failed line reaches disk on the next activity, no restart", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "lync-heal-")); + const roomFile = path.join(dir, `${ROOT}.lync`); + server = await startLyncServe({ dir, log: () => {} }); + const startedPort = server.port; + const url = `ws://localhost:${startedPort}`; + + const a = makeClient(url, "alice"); + const b = makeClient(url, "bob"); + clients = [a, b]; + for (const cl of clients) cl.store.syncRoot(ROOT); + + // Seed a real root, and prove it is durable everywhere before we break the + // disk (so the failure below is the ONLY thing off disk). + await a.append(ROOT, [], "the story begins"); + await waitFor(async () => { + for (const cl of clients) if ((await cl.store.byId(ROOT)) === null) return false; + return new Set(idsOf(await readFile(roomFile, "utf8"))).has(ROOT); + }); + + // ---- Storage fails: Alice's X reaches every client but not disk -------- + await chmod(roomFile, 0o444); + await a.append("X", [ROOT], "alice's line, refused by the disk"); + // X is in every client's store despite the failed persist... + await waitFor(async () => { + for (const cl of clients) if ((await cl.store.byId("X")) === null) return false; + return true; + }); + // ...the durability failure screamed on every subscriber... + await waitFor(() => a.sawFailure("persist-failed") && b.sawFailure("persist-failed")); + expect(a.sawFailure("persist-failed")).toBe(true); + expect(b.sawFailure("persist-failed")).toBe(true); + // ...and X genuinely never hit disk. + expect(new Set(idsOf(await readFile(roomFile, "utf8"))).has("X")).toBe(false); + + // ---- The disk heals. NO restart, NO reconnect, NO new generation. ----- + await chmod(roomFile, 0o644); + + // ---- Next activity on the SAME root, SAME running server -------------- + // Bob appends Y. The relay flushes the pending X first (append order), then + // Y. Both must land on disk without any restart having happened. + await b.append("Y", ["X"], "bob's line, after the disk healed"); + + // The heart of the proof: BOTH X (the refused line) and Y are now on the + // relay's on-disk .lync file. + await waitFor(async () => { + const onDisk = new Set(idsOf(await readFile(roomFile, "utf8"))); + return onDisk.has("X") && onDisk.has("Y") && onDisk.has(ROOT); + }, 8_000).catch(async (error) => { + const onDisk = [...new Set(idsOf(await readFile(roomFile, "utf8").catch(() => "")))]; + throw new Error(`X did not heal to disk without a restart — on disk: [${onDisk}] (${error})`); + }); + const finalDisk = new Set(idsOf(await readFile(roomFile, "utf8"))); + expect(finalDisk.has("X")).toBe(true); + expect(finalDisk.has("Y")).toBe(true); + expect(finalDisk.has(ROOT)).toBe(true); + + // Strict on-disk append order: the earlier line X is never written after Y. + const order = idsOf(await readFile(roomFile, "utf8")); + expect(order.indexOf("X")).toBeLessThan(order.indexOf("Y")); + + // Both lines are in every client's store too. + for (const cl of clients) { + expect(await cl.store.byId("X")).not.toBeNull(); + expect(await cl.store.byId("Y")).not.toBeNull(); + } + + // The server was never restarted: same instance, same port throughout. + expect(server.port).toBe(startedPort); + // No generation change was ever surfaced (a restart would have minted one). + expect(a.sawFailure("generation changed")).toBe(false); + expect(b.sawFailure("generation changed")).toBe(false); + }, 30_000); +}); From e2b22e30a2860b26edd4ab8cac2827c339488da5 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 02:19:56 -0700 Subject: [PATCH 32/33] relay: add read-only status() surface (dee-aic5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Give operators a way to see inside a running relay. status() returns a per-room snapshot { root, generation, seq, subscribers, pendingUnpersisted } where pendingUnpersisted is the durability lag the dee-1pfp retry-persist work introduced (lines in memory but not yet on disk). Strictly additive and read-only: it reads existing room state via a side map of resolved rooms and mutates nothing — no touch to the write path, sync protocol, union path, or close(). Threaded through startLyncServe (LyncSyncServer.status) and attachLyncServer (AttachedLyncServer.status); LyncRoomStatus exported from the relay subpath. Tests: N-room subscriber/seq accuracy; the durability-observability case (disk write fails -> pendingUnpersisted >= 1, heal + next append -> back to 0); and that repeated status() calls leave seq/subscribers unchanged. README relay section shows status() as a runnable daemon example. --- README.md | 11 +++- src/relay/attach.ts | 5 +- src/relay/index.ts | 2 +- src/relay/relay.ts | 45 ++++++++++++++++ src/relay/serve.ts | 5 +- test/relay/relay.test.ts | 110 +++++++++++++++++++++++++++++++++++++++ 6 files changed, 174 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 93370e5..01b77b1 100644 --- a/README.md +++ b/README.md @@ -287,6 +287,13 @@ import { startLyncServe } from "@deepfates/lync/relay"; const server = await startLyncServe({ dir: "./rooms", port: 8787 }); console.log("relay on", server.port); + +// Look inside a running relay — read-only, mutates nothing. Each record is +// { root, generation, seq, subscribers, pendingUnpersisted }, where +// pendingUnpersisted is the durability lag: lines in memory but not yet on disk. +for (const room of server.status()) { + console.log(room.root, "seq", room.seq, "subs", room.subscribers, "lag", room.pendingUnpersisted); +} // later: await server.close(); ``` @@ -309,7 +316,9 @@ httpServer.listen(3000); ``` For full control, `createLyncRelay` gives you `handleUpgrade` to call from -your own `upgrade` listener. +your own `upgrade` listener. All three (`createLyncRelay`, `startLyncServe`, +`attachLyncServer`) expose `status()` — a read-only snapshot of every live +room's seq, subscriber count, and pending-unpersisted durability lag. Guarantees: same-id-different-bytes is never resolved — both variants are kept (a `.conflicts` sidecar) and both sides are told loudly. Persist failures diff --git a/src/relay/attach.ts b/src/relay/attach.ts index 9386cf4..8feeaa3 100644 --- a/src/relay/attach.ts +++ b/src/relay/attach.ts @@ -1,6 +1,6 @@ import type { IncomingMessage, Server } from "node:http"; import type { Duplex } from "node:stream"; -import { createLyncRelay, type LyncRelayOptions, type LyncRelaySocket } from "./relay.js"; +import { createLyncRelay, type LyncRelayOptions, type LyncRelaySocket, type LyncRoomStatus } from "./relay.js"; /** * Mount the relay on an existing Node HTTP server. Adds an `upgrade` listener @@ -20,6 +20,8 @@ export interface AttachLyncServerOptions extends Omit LyncRoomStatus[]; close: () => Promise; } @@ -69,6 +71,7 @@ export function attachLyncServer(server: Server, options: AttachLyncServerOption server.on("upgrade", onUpgrade); return { + status: () => relay.status(), close: async () => { server.off("upgrade", onUpgrade); if (pingTimer) clearInterval(pingTimer); diff --git a/src/relay/index.ts b/src/relay/index.ts index 22301ac..8b8673f 100644 --- a/src/relay/index.ts +++ b/src/relay/index.ts @@ -1,3 +1,3 @@ -export { createLyncRelay, type LyncRelay, type LyncRelayOptions, type LyncRelaySocket } from "./relay.js"; +export { createLyncRelay, type LyncRelay, type LyncRelayOptions, type LyncRelaySocket, type LyncRoomStatus } from "./relay.js"; export { startLyncServe, type LyncServeOptions, type LyncSyncServer } from "./serve.js"; export { attachLyncServer, type AttachLyncServerOptions, type AttachedLyncServer } from "./attach.js"; diff --git a/src/relay/relay.ts b/src/relay/relay.ts index b683d65..8003821 100644 --- a/src/relay/relay.ts +++ b/src/relay/relay.ts @@ -77,11 +77,37 @@ export interface LyncRelayOptions { log?: (message: string) => void; } +/** + * A read-only snapshot of one live room, as `status()` reports it. Every field + * is a copy of existing room state — reading it mutates nothing. + */ +export interface LyncRoomStatus { + /** The room's root name (its `.lync` file). */ + root: string; + /** Log generation: fresh per recovery, carried on ev/live frames. */ + generation: string; + /** Per-root arrival counter: the count of accepted lines, resume cursor. */ + seq: number; + /** Live socket count subscribed to this room right now. */ + subscribers: number; + /** + * Lines accepted into memory and broadcast but NOT yet on disk (a prior + * append failed) — the durability lag. 0 when the log is fully persisted. + */ + pendingUnpersisted: number; +} + export interface LyncRelay { /** Handle an HTTP upgrade: authorize, upgrade, and wire the socket. */ handleUpgrade(request: IncomingMessage, socket: Duplex, head: Buffer): void; /** Wire a socket you upgraded yourself. */ handleConnection(socket: LyncRelaySocket): void; + /** + * A read-only snapshot of every live room — its seq, live subscriber count, + * and durability lag (pending-unpersisted lines). Strictly observational: + * it reads existing room state and mutates nothing. + */ + status(): LyncRoomStatus[]; /** Close all sockets and flush every pending append. */ close(): Promise; } @@ -118,6 +144,9 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { const log = options.log ?? ((message: string) => process.stderr.write(`${message}\n`)); const dirReady = mkdir(options.dir, { recursive: true }).then(() => undefined); const rooms = new Map>(); + // Resolved rooms, for the read-only status() surface only. A room lands here + // once recovery settles; never read on the write, sync, union, or close path. + const ready = new Map(); const sockets = new Set(); const WebSocketServer = acquireWebSocketServer(); const wss = new WebSocketServer({ noServer: true }); @@ -258,6 +287,9 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { if (!pending) { pending = dirReady.then(() => recoverRoom(root)); rooms.set(root, pending); + // Record the resolved room for status() to read. Purely observational: + // a recovery failure is left to the caller's rejection, never masked. + void pending.then((room) => ready.set(root, room), () => {}); } return pending; } @@ -363,9 +395,22 @@ export function createLyncRelay(options: LyncRelayOptions): LyncRelay { } } + // A fresh array of plain records built from current room state — no handle + // into the room's mutable maps escapes, so a caller can never disturb it. + function status(): LyncRoomStatus[] { + return [...ready.values()].map((room) => ({ + root: room.root, + generation: room.generation, + seq: room.seq, + subscribers: room.subscribers.size, + pendingUnpersisted: room.unpersisted.size, + })); + } + return { handleUpgrade, handleConnection, + status, close: async () => { // Flush every pending append first — writeChains are kept resolved (never // rejected) by appendSerialized, so this settles promptly and no accepted diff --git a/src/relay/serve.ts b/src/relay/serve.ts index 648d778..8618e9f 100644 --- a/src/relay/serve.ts +++ b/src/relay/serve.ts @@ -1,5 +1,5 @@ import { createServer, type Server } from "node:http"; -import { createLyncRelay, type LyncRelayOptions } from "./relay.js"; +import { createLyncRelay, type LyncRelayOptions, type LyncRoomStatus } from "./relay.js"; /** * Run the relay standalone on its own HTTP server. For embedding in an @@ -14,6 +14,8 @@ export interface LyncServeOptions extends LyncRelayOptions { export interface LyncSyncServer { port: number; + /** Read-only snapshot of the relay's live rooms. See `LyncRelay.status`. */ + status: () => LyncRoomStatus[]; close: () => Promise; } @@ -35,6 +37,7 @@ export async function startLyncServe(options: LyncServeOptions): Promise relay.status(), close: async () => { await relay.close(); // Drop any lingering connections and release the listen handle. The diff --git a/test/relay/relay.test.ts b/test/relay/relay.test.ts index 3c2d042..7a5be9b 100644 --- a/test/relay/relay.test.ts +++ b/test/relay/relay.test.ts @@ -124,3 +124,113 @@ describe("createLyncRelay durability failures", () => { } }); }); + +describe("createLyncRelay status() — read-only observability", () => { + const cleanups: Array<() => Promise | void> = []; + + afterEach(async () => { + for (const c of cleanups.splice(0).reverse()) await c(); + }); + + // Boot a throwaway HTTP server around a relay; return its ws url. Teardown + // (relay flush + server close) is registered so each test stays isolated. + async function boot(relay: ReturnType): Promise { + const httpServer = createServer((_r, res) => res.writeHead(200).end()); + httpServer.on("upgrade", (req, socket, head) => relay.handleUpgrade(req, socket, head)); + const port = await new Promise((resolve) => + httpServer.listen(0, () => { + const a = httpServer.address(); + resolve(typeof a === "object" && a ? a.port : 0); + }), + ); + cleanups.push(async () => { + await relay.close(); + await new Promise((resolve) => httpServer.close(() => resolve())); + }); + return `ws://localhost:${port}`; + } + + const line = (id: string) => + JSON.stringify({ v: 1, id, kind: "lync/artifact", at: "2026-07-08T21:00:00Z", author: { actor: "x" }, parents: [], payload: {} }); + + it("reports each room's seq and live subscriber count accurately", async () => { + const dir = await mkdtemp(path.join(os.tmpdir(), "lync-status-")); + const relay = createLyncRelay({ dir, log: () => {} }); + const url = await boot(relay); + + // Three sockets on "alpha", one on "beta". + const subs = Array.from({ length: 4 }, () => createWebSocketTransport(url, { reconnectMs: 0 })); + cleanups.push(() => { + for (const t of subs) t.close(); + }); + subs[0].send({ t: "sub", root: "alpha", since: 0 }); + subs[1].send({ t: "sub", root: "alpha", since: 0 }); + subs[2].send({ t: "sub", root: "alpha", since: 0 }); + subs[3].send({ t: "sub", root: "beta", since: 0 }); + // Two accepted lines into alpha => seq 2; one into beta => seq 1. + subs[0].send({ t: "ev", root: "alpha", line: line("a1") }); + subs[0].send({ t: "ev", root: "alpha", line: line("a2") }); + subs[3].send({ t: "ev", root: "beta", line: line("b1") }); + + const byRoot = () => new Map(relay.status().map((r) => [r.root, r])); + await waitFor(() => { + const s = byRoot(); + return ( + s.get("alpha")?.seq === 2 && + s.get("alpha")?.subscribers === 3 && + s.get("beta")?.seq === 1 && + s.get("beta")?.subscribers === 1 + ); + }); + + const s = byRoot(); + expect(s.get("alpha")).toMatchObject({ root: "alpha", seq: 2, subscribers: 3, pendingUnpersisted: 0 }); + expect(s.get("beta")).toMatchObject({ root: "beta", seq: 1, subscribers: 1, pendingUnpersisted: 0 }); + // generation is a stable non-empty id for a live room. + expect(typeof s.get("alpha")?.generation).toBe("string"); + expect((s.get("alpha")?.generation ?? "").length).toBeGreaterThan(0); + }); + + it("shows durability lag, heals to 0 on the next flush, and mutates nothing when read", async () => { + const { chmod } = await import("node:fs/promises"); + const dir = await mkdtemp(path.join(os.tmpdir(), "lync-status-lag-")); + const relay = createLyncRelay({ dir, log: () => {} }); + const url = await boot(relay); + cleanups.push(async () => void (await chmod(dir, 0o755).catch(() => {}))); + + const t = createWebSocketTransport(url, { reconnectMs: 0 }); + cleanups.push(() => t.close()); + t.send({ t: "sub", root: "lag", since: 0 }); + await waitFor(() => relay.status().some((r) => r.root === "lag")); + + const of = (root: string) => relay.status().find((r) => r.root === root); + + // Freeze the disk so the next append cannot land — the line is accepted + // into memory (seq consumed) but stays unpersisted. + await chmod(dir, 0o555); + t.send({ t: "ev", root: "lag", line: line("d1") }); + await waitFor(() => (of("lag")?.pendingUnpersisted ?? 0) >= 1); + + const stuck = of("lag")!; + expect(stuck.pendingUnpersisted).toBe(1); + expect(stuck.seq).toBe(1); + + // status() is strictly read-only: repeated calls change nothing. + for (let i = 0; i < 5; i += 1) relay.status(); + const afterReads = of("lag")!; + expect(afterReads.seq).toBe(1); + expect(afterReads.pendingUnpersisted).toBe(1); + expect(afterReads.subscribers).toBe(1); + + // Heal the disk, then trigger a flush with the next append — persistPending + // drains the backlog in order before writing the new line. + await chmod(dir, 0o755); + t.send({ t: "ev", root: "lag", line: line("d2") }); + await waitFor(() => (of("lag")?.pendingUnpersisted ?? 1) === 0); + + const healed = of("lag")!; + expect(healed.pendingUnpersisted).toBe(0); + // The room still works: the second line was accepted, so seq advanced. + expect(healed.seq).toBe(2); + }); +}); From cca6baa538e84fe5679538cfe613ebd01810fee0 Mon Sep 17 00:00:00 2001 From: deepfates Date: Mon, 13 Jul 2026 02:33:26 -0700 Subject: [PATCH 33/33] readme: lead with what lync is and why, in plain words The npm landing page opened with the dense technical summary. It now opens with the human version: most software forgets; lync keeps every version by only ever adding, never editing; and everything good (full history, safe merging, honesty on conflicts, a file that outlives its tools, training data from your own choices) falls out of that one rule. The precise spec, the conformance vectors, and every runnable example are unchanged below it. check-readme-examples still passes. --- README.md | 39 +++++++++++++++++++++++++++++---------- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 01b77b1..38545e8 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,34 @@ # lync -lync is a file format for append-only interaction history, and `@deepfates/lync` is -its reference implementation — one package that ships the parser, event -stores, computed views, the loom API, live sync, the `lync` command, and the -sync relay. Zero runtime dependencies. - -A `.lync` file is UTF-8 JSONL: each line is one immutable event with an -envelope, parent links, provenance, and a payload owned by the event kind. -Merge is set union by event id. Branch trees, transcripts, memory views, and -leaderboards are computed views over the same event set — never stored as -truth themselves. +Most software forgets. Edit a document and yesterday's version is gone. Write +with an AI that offers three options and the two you do not pick vanish. Let a +tool merge two people's edits and you get one result with no memory of who did +what or what was dropped. The story of how a thing came to be is thrown away at +every step. + +lync keeps it. A `.lync` file is a list of events, one per line, in plain text. +Each line records one thing that happened. Someone wrote a sentence, marked a +version as good, or chose one branch over another. You never edit a line. To +change something you add a new line that points at the old one and says what +changed. The old line stays. + +That one rule, only ever add and never edit, is where everything good comes +from. You can always see the whole history, every version and every branch you +did not take. Merging two copies is safe and boring, because you keep every +event from both and there are no edits to fight over. If the same event ever +shows up two different ways, lync keeps both and says so out loud instead of +quietly picking one. And because the file is plain text with a written-down +spec, you can still read it in twenty years, on a computer that has never heard +of this software. + +A lync file is more than a transcript, because it keeps the branches and the +choices. From the same file you can make a readable document to hand someone, +or training data built from the exact versions you marked as good. + +`@deepfates/lync` is the reference implementation. One package, zero runtime +dependencies, holding the parser, the event stores, the computed views, the +loom API, live sync, the `lync` command, and the sync relay. The format is the +durable center. Everything else is a tool that reads and writes it. ## The Format