From 32fb4d33a7402bf5f8f56748a678dd010f018e05 Mon Sep 17 00:00:00 2001 From: illodev Date: Fri, 7 Aug 2026 20:21:15 +0200 Subject: [PATCH 1/5] Doctor stops trusting what a repository's healthCheck hands it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit T-0213 asked whether `doctor` should call a `healthCheck` declared by `project.config.mjs` at all. It should: `loadWorkspace` already `import()`s that module in every command, so the module body runs earlier and more quietly than any hook, and removing the call closes nothing while costing the feature. LRN-0029 lists all three channels that execute repository-declared code, with every caller and where it runs. What was actually broken is that `doctor` trusted the hook's answer: - a throw propagated out of `runDoctor` and took every caller down with it, so the report that would have named the broken integration never printed; - a hook that never settled had no bound at all; - `{ severity: "catastrophe" }` wrote NaN into the counts, landed in no bucket, left `ok` true and made the comparator sort on NaN — an integration could decide whether the repository passed, by typo. Each call is now isolated, bounded at 10s and validated, attributed to its integration by id, and all three failures are errors because `doctor` is a gate. Well-formed entries in a rejected batch still land. A project that declares no integrations sees none of it, proven by a test. The bound is partial on purpose and the code says so: a hook runs on `doctor`'s own event loop, so it catches an awaited hang and not a synchronous spin. The timer is deliberately not `unref`ed — that was the first draft, and it let Node exit before the bound fired, printing no report at all. The three generated CI templates now state both hops rather than the first one on GitHub only. "It imports a config file" reads as loading settings, and somebody pricing this from that sentence prices it wrongly. Cards: T-0213 Filed: T-0218 Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01D3LTdq3mzMAQ98rwBegGjU --- ...s-a-healthcheck-the-repository-declares.md | 30 ++- ...sue-never-says-which-module-produced-it.md | 42 ++++ ...-healthcheck-and-stops-trusting-what-it.md | 91 ++++++++ ...ory-declared-code-and-where-each-one-ru.md | 73 +++++++ packages/workfile/src/modules/ci/ci.ts | 33 ++- .../src/modules/integrations/registry.ts | 201 +++++++++++++++++- packages/workfile/strict-baseline.json | 2 +- packages/workfile/test/ci-targets.test.ts | 14 ++ packages/workfile/test/integrations.test.ts | 194 +++++++++++++++++ 9 files changed, 662 insertions(+), 18 deletions(-) create mode 100644 .project/cards/T-0218-a-doctor-issue-never-says-which-module-produced-it.md create mode 100644 .project/memory/decisions/ADR-0019-doctor-keeps-calling-a-declared-healthcheck-and-stops-trusting-what-it.md create mode 100644 .project/memory/learnings/LRN-0029-every-surface-that-runs-repository-declared-code-and-where-each-one-ru.md diff --git a/.project/cards/T-0213-doctor-calls-a-healthcheck-the-repository-declares.md b/.project/cards/T-0213-doctor-calls-a-healthcheck-the-repository-declares.md index 480453c..564478d 100644 --- a/.project/cards/T-0213-doctor-calls-a-healthcheck-the-repository-declares.md +++ b/.project/cards/T-0213-doctor-calls-a-healthcheck-the-repository-declares.md @@ -1,16 +1,22 @@ --- id: T-0213 title: doctor calls a healthCheck the repository declares, on every runner -status: backlog +status: done type: audit priority: high area: infra tags: [security] effort: S -scope: [packages/workfile/src/modules/integrations] +scope: [packages/workfile/src/modules/integrations, packages/workfile/src/modules/health, packages/workfile/src/modules/ci, packages/workfile/test/integrations.test.ts, packages/workfile/test/ci-targets.test.ts, packages/workfile/strict-baseline.json] origin: [T-0188, LRN-0028] created: 2026-08-05 -updated: 2026-08-05 +updated: 2026-08-07 +related: [ADR-0019, LRN-0029, T-0218] +verified: + at: "2026-08-07T16:48:16.725Z" + method: local + commit: 4e8da0782fecb7e52899f7916be21ad7f3d4c775 + digest: "sha256:0d50a6371d79d42b7d4bd6d0ac6b1ec47f552ff820f2f7ebc6708756b7be3d0f" --- Found while auditing T-0188, and one hop further out than LRN-0025 recorded. @@ -47,7 +53,17 @@ What to establish, in order: ## Acceptance criteria -- [ ] Every surface that calls repository-declared code is listed, with where it runs. -- [ ] A decision is recorded on whether `doctor` calls a declared `healthCheck`, with the reason. -- [ ] The generated CI documentation states what a repository's own config can execute, rather than leaving it to be discovered. -- [ ] If the behaviour changes, a project that declares no integrations behaves exactly as before, proven by a test. +- [x] Every surface that calls repository-declared code is listed, with where it runs. +- [x] A decision is recorded on whether `doctor` calls a declared `healthCheck`, with the reason. +- [x] The generated CI documentation states what a repository's own config can execute, rather than leaving it to be discovered. +- [x] If the behaviour changes, a project that declares no integrations behaves exactly as before, proven by a test. + +## Activity + +- 2026-08-07 16:32Z illodev@local#42eb42f5 · claimed +- 2026-08-07 16:48Z illodev@local#42eb42f5 · released + +## Notes + +- 2026-08-07 16:48Z illodev@local#42eb42f5 — Audited and closed. The premise the card was filed on holds — runDoctor does hand doctor a repository-supplied function and call it — but the conclusion does not: removing the call closes nothing, because loadWorkspace already import()s project.config.mjs in every command, so the module body runs earlier and more quietly than any hook. LRN-0029 lists all three channels with their callers; ADR-0019 records the decision. What the audit actually found was a robustness defect, verified against the packaged CLI on a scratch workspace: a healthCheck that threw propagated out of runDoctor and took the command down with a raw Error, so the report that would have named the broken integration never printed — the CLI, /api/v2/health, /api/health and both MCP doctor surfaces all died with it. A hook returning { severity: "catastrophe" } wrote NaN into doctor's counts, landed in no bucket, left ok true, and made the severity comparator sort on NaN — an integration could decide whether the repository passed, by typo. And a hook that never settled had no bound at all. Now each call is isolated, bounded at 10s and validated: real binary, two hooks declared, reports integration-health-check-failed naming "no credentials configured" and integration-health-check-invalid naming "1 of 1 diagnostics that could not be counted", both error, ok: false, exit 1. Hanging hook at the real default: integration-health-check-timeout with timeoutMs 10000 after 11s wall clock, report printed. No integrations declared: 0 errors, 0 warnings, zero integration findings. The bound is partial on purpose and the code says so — a hook runs on doctor's own event loop, so it catches an awaited hang and cannot catch a synchronous spin. One trap worth the record: the timer was unref'd in the first draft, which let Node exit before the bound fired and printed no report at all — strictly worse than the hang. Filed T-0218 for the provenance gap the audit surfaced: doctor flattens away the module that produced an issue, so a well-formed diagnostic from a repository's own hook is indistinguishable from one Workfile wrote. +- 2026-08-07 16:48Z illodev@local#42eb42f5 — local verification: pnpm run check green: 460+7 tests pass, strictNullChecks held at 488 with registry.ts ratcheted 10 to 9. Packaged CLI against scratch workspaces: throwing and malformed hooks reported as integration-health-check-failed and -invalid at error severity with ok false and exit 1 instead of crashing runDoctor; hanging hook bounded at the 10s default (timeoutMs 10000, 11s wall clock, report printed); a workspace declaring no integrations reports 0 errors, 0 warnings and zero integration findings. Repo doctor 0/0, memory verify 0/0. diff --git a/.project/cards/T-0218-a-doctor-issue-never-says-which-module-produced-it.md b/.project/cards/T-0218-a-doctor-issue-never-says-which-module-produced-it.md new file mode 100644 index 0000000..1b5301b --- /dev/null +++ b/.project/cards/T-0218-a-doctor-issue-never-says-which-module-produced-it.md @@ -0,0 +1,42 @@ +--- +id: T-0218 +title: A doctor issue never says which module produced it +status: backlog +type: task +priority: low +area: core +tags: [health] +effort: S +scope: [packages/workfile/src/modules/health] +origin: [T-0213, LRN-0029] +created: 2026-08-07 +updated: 2026-08-07 +--- + +Found while auditing T-0213. + +Every module that reports health returns `{ module, issues }` — `diagnoseCards`, +the docs, changelog and memory reports, `checkCiTemplates`, and +`healthReports` with `module: "integration:"`. `runDoctor` then does +`reports.flatMap((report) => report.issues)` and the module is gone. What +reaches the reader is a flat list where nothing says where a finding came from. + +Mostly this is invisible, because a core `code` implies its module to anyone who +knows the codebase. It stops being invisible for integrations, which are the one +source that is not ours: a diagnostic a repository's own `healthCheck` returned +reads exactly like one Workfile produced. ADR-0019 made the three +`integration-health-check-*` failures name their integration, but only because +those are authored here — a well-formed diagnostic from a hook still arrives +anonymous. + +The fix is small and the decision is what to do with `code`. Stamping the module +into the issue is additive and safe; namespacing the `code` would be clearer and +would break `issueIdentity`, so every baseline accepted with +`doctor --accept-baseline` would go stale at once. Probably: carry `module` as a +field, leave `code` alone, and have the CLI and the Health view group by it. + +## Acceptance criteria + +- [ ] A doctor issue carries the module that produced it, integrations included. +- [ ] An existing accepted baseline still matches, proven by a test. +- [ ] The CLI and `/api/v2/health` consumers can tell an integration's finding from a core one. diff --git a/.project/memory/decisions/ADR-0019-doctor-keeps-calling-a-declared-healthcheck-and-stops-trusting-what-it.md b/.project/memory/decisions/ADR-0019-doctor-keeps-calling-a-declared-healthcheck-and-stops-trusting-what-it.md new file mode 100644 index 0000000..85f2c9c --- /dev/null +++ b/.project/memory/decisions/ADR-0019-doctor-keeps-calling-a-declared-healthcheck-and-stops-trusting-what-it.md @@ -0,0 +1,91 @@ +--- +id: ADR-0019 +title: doctor keeps calling a declared healthCheck, and stops trusting what it returns +status: accepted +related: [T-0213, T-0188, LRN-0028, LRN-0025, LRN-0029] +tags: [security, integrations, health] +created: 2026-08-07 +updated: 2026-08-07 +--- + +## Context + +T-0213 asked whether `doctor` should call a `healthCheck` that +`project.config.mjs` declares. The exposure is real and one hop further out +than LRN-0025 recorded: `runDoctor` builds an integration registry from the +repository's own config module and calls each declared hook, inside the one +command the generated CI workflow exists to run. On a GitLab job that sees +every unprotected variable, or in the generic script that inherits whatever +environment invokes it, the templates cannot protect anything. + +The audit's finding is that the framing was wrong, and LRN-0029 lists the +surfaces that make it wrong. `loadWorkspace` `import()`s `project.config.mjs` +before anything else happens, so the module body has already run — in every +command, not just `doctor`. Anything a `healthCheck` can do, the module body +can do earlier and more quietly. There is no capability the hook adds and no +boundary that removing the call would restore. + +What the audit did find is a different defect, and it is not about trust +boundaries at all. `doctor` treated the hook's *answer* as its own: + +- A hook that threw propagated out of `runDoctor` and took every caller with + it — the CLI, `/api/v2/health`, `/api/health`, the MCP `project_doctor` tool + and the MCP resource. A misconfigured integration was indistinguishable from + a broken repository, and the report that would have said so never printed. +- A hook that never settled hung `doctor` with no bound at all, so CI died at + its own job timeout with nothing to read. +- A returned diagnostic went uninspected into the report. `runDoctor` derives + `counts` and `ok` from `issue.severity`, so `{ severity: "catastrophe" }` + wrote NaN into the counts, landed in no bucket, left `ok` true, and made the + comparator sort on NaN. An integration could decide whether the repository + passed, by typo. + +## Decision + +`doctor` keeps calling declared `healthCheck` hooks. Removing the call closes +nothing — the module body runs regardless — and costs a feature whose whole +point is that an integration can report on itself in the command people already +run. + +What changes is that the hook no longer speaks for `doctor`. Each call is +isolated, bounded and validated in +`packages/workfile/src/modules/integrations/registry.ts`: + +- A throw becomes `integration-health-check-failed`, attributed to the + integration by id. +- A hook that does not settle within ten seconds becomes + `integration-health-check-timeout`, and `doctor` answers without it. +- A returned value that is not diagnostics, or an entry whose severity is not + `error`, `warning` or `info`, becomes `integration-health-check-invalid`. + Well-formed entries in the same batch still land: rejecting one entry is not + rejecting the integration. + +All three are errors, not warnings, because `doctor` is a gate. A declared +check that could not answer is not a pass, and there is no way to tell what a +malformed entry was trying to say. + +A project that declares no integrations sees none of this, proven by a test. + +## Consequences + +The generated CI templates now state both hops — the module body runs, and +`doctor` calls what it declared — on all three targets rather than the first +hop on GitHub only. "It imports a config file" reads as loading settings, and +somebody pricing this from that sentence prices it wrongly. + +The ten-second bound is honest about being partial, and the code says so. A +hook runs on `doctor`'s own event loop, so the timer catches an awaited hang +and cannot catch a synchronous spin: `while (true) {}` starves the timer too. +Bounding that needs the hook in a worker or a subprocess, which is a different +feature and is not built here. + +The bound is reachable through `createIntegrationRegistry`'s +`healthCheckTimeoutMs` so the timeout is testable in milliseconds. `runDoctor` +does not pass it; an untested timeout is a timeout that regresses quietly, and +this one already did once — the timer was `unref`ed in the first draft, which +let Node exit before the bound fired and printed no report at all. + +Containment for a repository nobody has reviewed still belongs to the job, not +to Workfile: no secrets, no write token, no evidence written back from an +unreviewed head. That is unchanged from T-0188 and is the only answer that +survives here. diff --git a/.project/memory/learnings/LRN-0029-every-surface-that-runs-repository-declared-code-and-where-each-one-ru.md b/.project/memory/learnings/LRN-0029-every-surface-that-runs-repository-declared-code-and-where-each-one-ru.md new file mode 100644 index 0000000..656b7c6 --- /dev/null +++ b/.project/memory/learnings/LRN-0029-every-surface-that-runs-repository-declared-code-and-where-each-one-ru.md @@ -0,0 +1,73 @@ +--- +id: LRN-0029 +title: Every surface that runs repository-declared code, and where each one runs +status: active +category: infra +confidence: high +related: [T-0213, T-0188, ADR-0019, LRN-0028, LRN-0025] +tags: [security, integrations, ci] +created: 2026-08-07 +updated: 2026-08-07 +--- + +Audited for T-0213 against 0.8.1. LRN-0025 recorded the first channel and +LRN-0028 recorded what the generated CI targets hold; this is the complete list +of places Workfile executes code the repository supplied, with where each one +runs. There are three channels and no fourth: `ProjectIntegration` declares +exactly two hooks, `healthCheck` and `semanticSearchProvider.search`. + +**Channel 1 — the config module body. Everywhere, unconditionally.** +`loadWorkspace` does `await import()` on `project.config.mjs` out of the +checkout (`src/workspace/load-workspace.ts:270`). Every CLI command, the HTTP +server, both MCP transports, the UI backend and `doctor` load a workspace, so +all of them run that file's top-level code before doing anything else. This is +the channel that decides the shape of the whole question: the module body can +import anything and reach the network and the filesystem, so no bound placed on +the hooks below is a security boundary. ADR-0019 is the decision that follows +from it. + +**Channel 2 — `healthCheck`. Only through `doctor`, which is what CI runs.** +`runDoctor` builds a registry from `workspace.integrations` and calls every +declared hook (`src/modules/health/doctor.ts:100-103` → +`src/modules/integrations/registry.ts:290`). Six callers reach it: + +- `workfile doctor` — `bin/workfile.ts:2658`. This is the command all three + generated CI templates run, and the one a maintainer runs locally on a branch + they only meant to read. +- `GET /api/v2/health` — `src/server/http.ts:949`. +- `GET /api/health` — `src/server/http.ts:1620`. +- The `project_doctor` MCP tool — `src/modules/mcp/tools.ts:600`. +- The MCP doctor resource — `src/modules/mcp/resources.ts:170`. +- `scripts/bench.ts:64`, which is why a hook with a bad bound shows up as a + benchmark regression rather than as itself. + +**Channel 3 — `semanticSearchProvider.search`. Only when a provider resolves.** +Called from `src/modules/search/search.ts:310`. The registry is built at +`bin/workfile.ts:2476` (`workfile search`, skipped entirely under +`--mode lexical`), `src/server/http.ts:736` (resolved once at server start, +serving `/api/v2/search` and `/api/v2/records`) and +`src/modules/mcp/server.ts:249,292`. The hook receives record bodies, so it is +the channel that sees content rather than just paths. + +**Where a repository is not already trusted.** Two places, both from channel 2 +via `doctor`: a CI runner, and a maintainer's own machine on a branch they are +reviewing. GitHub is not one of them in the way it first looks — a fork's pull +request builds the fork's head without the base repository's secrets, and +LRN-0028 explains why arguing a fork boundary here misleads. GitLab and the +generic script are the cases the templates cannot protect. + +**A property worth knowing before reading a report.** `healthReports` returns +`module: "integration:"` per integration, and `runDoctor` then flattens +`report.issues` and discards the module — for every module equally, cards and +docs included. So a *valid* diagnostic from an integration is indistinguishable +from one Workfile produced itself. Only the three +`integration-health-check-*` codes name their integration, because those are +authored here rather than by the hook. + +**How to apply.** When asked what a repository's own config can execute, answer +from channel 1 and never from the hook list: the hooks are a hop past a door +that is already open, and describing them as the exposure invites somebody to +"fix" it by removing a feature. When adding a surface that calls a +repository-declared function, it belongs on this list, and it must not let that +function's throw, hang or return value decide the surface's own answer — see +`healthCheckDiagnostics` for the shape. diff --git a/packages/workfile/src/modules/ci/ci.ts b/packages/workfile/src/modules/ci/ci.ts index b331ad5..7e8d1de 100644 --- a/packages/workfile/src/modules/ci/ci.ts +++ b/packages/workfile/src/modules/ci/ci.ts @@ -37,15 +37,34 @@ export const CI_TARGETS = Object.freeze({ * * Every command below loads the workspace, and loading a workspace `import()`s * `project.config.mjs` out of the checkout — so on a pull request this job - * executes code the pull request wrote, before it reads a single card. That is - * the ordinary cost of building a pull request rather than a defect, and it is - * why the useful controls here are about what the job *holds* rather than about - * what it runs. The three targets differ sharply on that, and each says what it - * can enforce and what it cannot. + * executes code the pull request wrote, before it reads a single card. Two hops: + * `doctor` then calls every `healthCheck` that module declared. That is the + * ordinary cost of building a pull request rather than a defect, and it is why + * the useful controls here are about what the job *holds* rather than about what + * it runs. The three targets differ sharply on that, and each says what it can + * enforce and what it cannot. + * + * Each generated file states both hops, because a reader who only knows the + * first will price this wrongly: "it imports a config file" sounds like reading + * settings, and it is not. See ADR-0019 and LRN-0028. */ +const EXECUTES_REPOSITORY_CODE = [ + "Both commands load the workspace, which `import()`s project.config.mjs from", + "the checkout: this job runs that file's module body, and `doctor` then calls", + "every healthCheck and search provider it declares. On a pull request that is", + "code the pull request wrote. Nothing in Workfile sandboxes it — containment", + "is whatever this job holds, described below." +]; + +function executesRepositoryCode() { + return EXECUTES_REPOSITORY_CODE.map((line) => `# ${line}`).join("\n"); +} + function githubBody(workspace) { const node = String(workspace.config.ci.nodeVersion || "22"); return `# Generated by @illodev/workfile ${PACKAGE_VERSION} +# +${executesRepositoryCode()} name: Workfile on: @@ -85,6 +104,8 @@ function gitlabBody(workspace) { const node = String(workspace.config.ci.nodeVersion || "22"); return `# Generated by @illodev/workfile ${PACKAGE_VERSION} # +${executesRepositoryCode()} +# # GitLab has no per-job permission scope. This job sees every CI/CD variable # that is not marked protected, and the branch rule below fires on any branch # push, so anyone who can push a branch gets them — mark the variables @@ -111,6 +132,8 @@ function genericBody() { return `#!/usr/bin/env sh # Generated by @illodev/workfile ${PACKAGE_VERSION} # +${executesRepositoryCode()} +# # There is no permission model to configure here: this script inherits the # entire environment of whatever invokes it — a credential block on a build # agent, a developer's ~/.npmrc and ssh-agent, an instance metadata endpoint. diff --git a/packages/workfile/src/modules/integrations/registry.ts b/packages/workfile/src/modules/integrations/registry.ts index de8c0ff..1eb8644 100644 --- a/packages/workfile/src/modules/integrations/registry.ts +++ b/packages/workfile/src/modules/integrations/registry.ts @@ -1,15 +1,185 @@ import { ValidationError } from "../../core/errors.js"; import type { + ProjectDiagnostic, ProjectIndex, ProjectIntegration, ProjectWorkspace, SemanticSearchProvider } from "../../types.js"; +/** + * How long a declared `healthCheck` may take before `doctor` answers without it. + * + * Generous on purpose: a health check that reaches a model or a socket is the + * kind worth declaring, and this is not a performance budget. It exists so that + * a hook which never settles produces a named finding in ten seconds instead of + * a CI job that dies at its own timeout with nothing to read. + * + * The bound is real for an awaited hang and worthless against a synchronous + * spin: a hook runs on `doctor`'s own event loop, so `while (true) {}` starves + * the timer too. Bounding that would mean running the hook in a worker, which is + * a different feature — see ADR-0019. + */ +const HEALTH_CHECK_TIMEOUT_MS = 10_000; + +const HEALTH_CHECK_TIMED_OUT = Symbol("health-check-timed-out"); + +const DIAGNOSTIC_SEVERITIES = new Set(["error", "warning", "info"]); + function validId(value) { return /^[a-z0-9](?:[a-z0-9._-]{0,62}[a-z0-9])?$/.test(String(value || "")); } +function describe(value: unknown) { + if (value === null) return "null"; + if (Array.isArray(value)) return "array"; + return typeof value; +} + +/** + * Split what a `healthCheck` returned into diagnostics `doctor` can count and + * entries it cannot. + * + * `runDoctor` derives `counts` and `ok` from `issue.severity` and sorts on it, + * so an entry carrying anything else does not merely look wrong: it lands in no + * bucket, leaves `ok` true, and makes the comparator sort on NaN. That is the + * failure this guards — an integration cannot hand back a value that decides + * whether the repository passes. + */ +function partitionDiagnostics(raw: unknown[]) { + const issues: ProjectDiagnostic[] = []; + const rejected: string[] = []; + raw.forEach((entry, position) => { + const at = `[${position}]`; + if (!entry || typeof entry !== "object" || Array.isArray(entry)) { + rejected.push(`${at} is ${describe(entry)}, not a diagnostic object`); + return; + } + const diagnostic = entry as Record; + const problems: string[] = []; + if (!DIAGNOSTIC_SEVERITIES.has(String(diagnostic.severity))) { + problems.push( + `severity ${JSON.stringify(diagnostic.severity)} is not error, warning or info` + ); + } + if (typeof diagnostic.code !== "string" || !diagnostic.code) { + problems.push("code is not a non-empty string"); + } + if (typeof diagnostic.message !== "string" || !diagnostic.message) { + problems.push("message is not a non-empty string"); + } + if (problems.length) { + rejected.push(`${at} ${problems.join("; ")}`); + return; + } + issues.push(diagnostic as unknown as ProjectDiagnostic); + }); + return { issues, rejected }; +} + +/** + * Call one declared `healthCheck` and turn whatever it does into diagnostics. + * + * The repository declaring the hook already runs its own code on every command + * — `loadWorkspace` `import()`s `project.config.mjs` — so this is not a + * sandbox and does not pretend to be one. What it does is stop a hook from + * speaking for `doctor`: a throw, a hang or a malformed diagnostic becomes a + * finding *about the integration*, attributed to it by id, instead of taking + * down the one command the generated CI workflow exists to run. + * + * Each failure is an error rather than a warning because `doctor` is a gate. A + * declared check that could not answer is not a pass, and there is no way to + * tell what a malformed entry was trying to say. + */ +async function healthCheckDiagnostics( + integration: ProjectIntegration, + context: { workspace: ProjectWorkspace; index: ProjectIndex }, + timeoutMs: number +): Promise { + const details = { integration: integration.id }; + let timer: ReturnType | undefined; + let report: unknown; + try { + const settled = Promise.resolve(integration.healthCheck!(context)); + // A hook that rejects after the race is already decided still needs a + // handler here, or Node takes the process down for an unhandled + // rejection well after `doctor` has printed its report. + settled.catch(() => {}); + report = await Promise.race([ + settled, + // Deliberately not `unref`ed. An unreferenced timer lets Node exit + // once the hung hook is the only thing left, so `workfile doctor` + // would die silently having printed no report at all — the failure + // this bound exists to replace. `clearTimeout` in the `finally` is + // what keeps a fast hook from holding the process for ten seconds. + new Promise((resolve) => { + timer = setTimeout( + () => resolve(HEALTH_CHECK_TIMED_OUT), + timeoutMs + ); + }) + ]); + } catch (error) { + return [ + { + severity: "error", + code: "integration-health-check-failed", + message: + `Integration ${integration.id} declares a healthCheck that threw: ` + + `${(error as Error)?.message || String(error)}. Its findings are missing from this report.`, + details: { + ...details, + error: (error as Error)?.message || String(error) + } + } + ]; + } finally { + clearTimeout(timer); + } + if (report === HEALTH_CHECK_TIMED_OUT) { + return [ + { + severity: "error", + code: "integration-health-check-timeout", + message: + `Integration ${integration.id} declares a healthCheck that did not settle within ` + + `${timeoutMs}ms. Its findings are missing from this report.`, + details: { ...details, timeoutMs } + } + ]; + } + // Nothing to say is a valid answer, and stays indistinguishable from an + // integration that declares no hook at all. + if (!report) return null; + const raw = Array.isArray(report) + ? report + : (report as { issues?: unknown }).issues; + if (!Array.isArray(raw)) { + return [ + { + severity: "error", + code: "integration-health-check-invalid", + message: + `Integration ${integration.id} declares a healthCheck that returned ` + + `${describe(report)}, not an array of diagnostics or an object with an \`issues\` array.`, + details: { ...details, returned: describe(report) } + } + ]; + } + const { issues, rejected } = partitionDiagnostics(raw); + if (rejected.length) { + issues.push({ + severity: "error", + code: "integration-health-check-invalid", + message: + `Integration ${integration.id} returned ${rejected.length} of ${raw.length} ` + + `diagnostics that could not be counted, so they were dropped: ${rejected.join(", ")}.`, + details: { ...details, rejected, returned: raw.length } + }); + } + return issues; +} + export function defineProjectIntegration( definition: ProjectIntegration ): Readonly { @@ -55,15 +225,32 @@ export interface ProjectIntegrationRegistry { list(): ProjectIntegration[]; get(id: string): ProjectIntegration | null; semanticSearchProvider(preferredId?: string): SemanticSearchProvider | null; + /** + * `module`, not `integration`: the shape every other `doctor` report has, + * which is what the returned value has always actually carried. + */ healthReports( workspace: ProjectWorkspace, index: ProjectIndex - ): Promise>; + ): Promise>; +} + +export interface IntegrationRegistryOptions { + /** + * Override the bound on a declared `healthCheck`. Exists so the bound is + * testable in milliseconds rather than only at its ten-second default — + * `runDoctor` does not pass it, and an untested timeout is a timeout that + * regresses quietly. + */ + healthCheckTimeoutMs?: number; } export function createIntegrationRegistry( - integrations: ProjectIntegration[] = [] + integrations: ProjectIntegration[] = [], + options: IntegrationRegistryOptions = {} ): Readonly { + const healthCheckTimeoutMs = + options.healthCheckTimeoutMs ?? HEALTH_CHECK_TIMEOUT_MS; const ordered = []; const byId = new Map(); for (const candidate of integrations) { @@ -104,11 +291,15 @@ export function createIntegrationRegistry( const reports = []; for (const integration of ordered) { if (!integration.healthCheck) continue; - const report = await integration.healthCheck({ workspace, index }); - if (!report) continue; + const issues = await healthCheckDiagnostics( + integration, + { workspace, index }, + healthCheckTimeoutMs + ); + if (!issues) continue; reports.push({ module: `integration:${integration.id}`, - issues: Array.isArray(report) ? report : report.issues || [] + issues }); } return reports; diff --git a/packages/workfile/strict-baseline.json b/packages/workfile/strict-baseline.json index dc8f223..c519cf0 100644 --- a/packages/workfile/strict-baseline.json +++ b/packages/workfile/strict-baseline.json @@ -20,7 +20,7 @@ "src/modules/docs/validation.ts": 4, "src/modules/health/doctor.ts": 19, "src/modules/init/initializer.ts": 17, - "src/modules/integrations/registry.ts": 10, + "src/modules/integrations/registry.ts": 9, "src/modules/mcp/server.ts": 5, "src/modules/memory/memory.ts": 8, "src/modules/memory/validation.ts": 4, diff --git a/packages/workfile/test/ci-targets.test.ts b/packages/workfile/test/ci-targets.test.ts index 5970a7e..a135ee0 100644 --- a/packages/workfile/test/ci-targets.test.ts +++ b/packages/workfile/test/ci-targets.test.ts @@ -74,6 +74,20 @@ test("the generic script states the contract it cannot enforce", async () => { assert.match(generic, /inherits the\n# entire environment/); }); +test("every target states that the checkout's own config executes, both hops", async () => { + // T-0213. The first hop was already stated on GitHub only, and stating it + // alone reads as "it imports a settings file" — the second hop is that + // `doctor` then *calls functions the config handed it*, which is what makes + // this worth pricing. A reader who has to discover that from the source has + // been told the wrong thing, so it is pinned on all three. + for (const [id, body] of Object.entries(await bodies())) { + assert.match(body, /`import\(\)`s project\.config\.mjs/, id); + assert.match(body, /module body/, id); + assert.match(body, /healthCheck/, id); + assert.match(body, /Nothing in Workfile sandboxes it/, id); + } +}); + test("no generated target lowers a card-declared command into a shell", async () => { // The pin that keeps the next card honest. A `verify[].run` is an argument // vector precisely so that no shell parses it; writing one into a YAML diff --git a/packages/workfile/test/integrations.test.ts b/packages/workfile/test/integrations.test.ts index cf365fe..cadaea9 100644 --- a/packages/workfile/test/integrations.test.ts +++ b/packages/workfile/test/integrations.test.ts @@ -268,6 +268,200 @@ test("a malformed integrations export fails on load, naming the config", async ( } }); +/** + * T-0213: a declared `healthCheck` is a foreign call inside `doctor`. + * + * The hook is repository-supplied code, and the config module body that declares + * it already ran on import — so none of this is containment and ADR-0019 says so. + * What these pin is narrower and is the part that was broken: a hook cannot speak + * *for* `doctor`. It cannot take the command down, it cannot hang it forever, and + * it cannot hand back a value that decides whether the repository passes. + */ +test("a healthCheck that throws becomes a finding, not a dead doctor", async () => { + const root = await workspaceWithConfig( + `export default { schemaVersion: 2, name: "Throwing" }; +export const integrations = [ + { id: "sync-boom", healthCheck() { throw new Error("exploded on the spot"); } }, + { id: "async-boom", async healthCheck() { throw new Error("exploded later"); } } +]; +` + ); + try { + const workspace = await loadWorkspace({ root }); + // Before this, the raw error propagated out of runDoctor and took every + // caller with it: the CLI, /api/v2/health, and the MCP doctor tool. + const report = await runDoctor(workspace); + const failures = report.issues.filter( + (issue) => issue.code === "integration-health-check-failed" + ); + assert.equal(failures.length, 2); + for (const failure of failures) { + assert.equal(failure.severity, "error"); + // Attributed, or the reader cannot tell which of their integrations + // to go and fix. + assert.match(failure.message, /^Integration (sync|async)-boom /); + assert.match(failure.details.error, /exploded (on the spot|later)/); + } + // A declared check that could not answer is not a pass. + assert.equal(report.ok, false); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("a healthCheck returning an uncountable diagnostic cannot decide `ok`", async () => { + const root = await workspaceWithConfig( + `export default { schemaVersion: 2, name: "Junk" }; +export const integrations = [{ id: "junk", healthCheck() { return { issues: [ + { severity: "catastrophe", code: 42 }, + { severity: "warning", code: "well-formed", message: "this one counts" } +] }; } }]; +` + ); + try { + const workspace = await loadWorkspace({ root }); + const report = await runDoctor(workspace); + + // The failure this replaces: `counts[issue.severity] += 1` on an unknown + // severity wrote NaN into the counts, the issue landed in no bucket, and + // `ok` stayed true while the comparator sorted on NaN. + assert.deepEqual(Object.keys(report.counts).sort(), [ + "error", + "info", + "warning" + ]); + for (const count of Object.values(report.counts)) { + assert.equal(Number.isInteger(count), true); + } + assert.equal( + report.issues.some((issue) => issue.severity === "catastrophe"), + false + ); + + const invalid = report.issues.find( + (issue) => issue.code === "integration-health-check-invalid" + ); + assert.equal(invalid.severity, "error"); + assert.match(invalid.message, /1 of 2/); + assert.equal(invalid.details.integration, "junk"); + + // And the well-formed sibling in the same batch still lands: rejecting + // one entry is not rejecting the integration. + assert.equal( + report.issues.some((issue) => issue.code === "well-formed"), + true + ); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("a healthCheck returning something that is not diagnostics at all is named", async () => { + const root = await workspaceWithConfig( + `export default { schemaVersion: 2, name: "Stringy" }; +export const integrations = [{ id: "stringy", healthCheck() { return "not diagnostics"; } }]; +` + ); + try { + const workspace = await loadWorkspace({ root }); + const report = await runDoctor(workspace); + const invalid = report.issues.find( + (issue) => issue.code === "integration-health-check-invalid" + ); + assert.equal(invalid.severity, "error"); + assert.match(invalid.message, /returned string/); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + +test("a healthCheck with nothing to say is indistinguishable from no hook", async () => { + const quiet = await workspaceWithConfig( + `export default { schemaVersion: 2, name: "Quiet", cards: { areas: ["api", "web", "infra", "docs"] } }; +export const integrations = [{ id: "quiet", healthCheck() { return null; } }]; +` + ); + const none = await workspaceWithConfig( + `export default { schemaVersion: 2, name: "Quiet", cards: { areas: ["api", "web", "infra", "docs"] } }; +` + ); + try { + const withHook = await runDoctor(await loadWorkspace({ root: quiet })); + const without = await runDoctor(await loadWorkspace({ root: none })); + assert.deepEqual(withHook.counts, without.counts); + assert.deepEqual( + withHook.issues.map((issue) => issue.code), + without.issues.map((issue) => issue.code) + ); + } finally { + await rm(quiet, { recursive: true, force: true }); + await rm(none, { recursive: true, force: true }); + } +}); + +test("a healthCheck that never settles is bounded, and doctor still answers", async () => { + // Bounded through the injectable seam so this costs milliseconds; the + // default is ten seconds and `runDoctor` uses it. Verified once at the real + // default against the built binary — see the card's note. + const integrationRegistry = createIntegrationRegistry( + [ + defineProjectIntegration({ + id: "hang", + healthCheck() { + return new Promise(() => {}); + } + }) + ], + { healthCheckTimeoutMs: 25 } + ); + const workspace = await loadWorkspace({ root: fixture }); + // Through `runDoctor` rather than the registry alone, because the claim is + // about the command: it returns a report instead of waiting on the hook. + const report = await runDoctor(workspace, { integrationRegistry }); + const timedOut = report.issues.find( + (issue) => issue.code === "integration-health-check-timeout" + ); + assert.equal(timedOut.severity, "error"); + assert.equal(timedOut.details.timeoutMs, 25); + assert.equal(timedOut.details.integration, "hang"); + assert.equal(report.ok, false); +}); + +test("declaring no integrations leaves doctor exactly as it was", async () => { + // The criterion that keeps the rest of this honest: everything above adds + // machinery to the health path, and the overwhelming majority of workspaces + // — including this repository — declare no integrations at all. They must + // not pay for it, and must not see a single issue they did not see before. + const root = await workspaceWithConfig( + `export default { + schemaVersion: 2, + name: "No integrations", + cards: { areas: ["api", "web", "infra", "docs"] } +}; +` + ); + try { + const workspace = await loadWorkspace({ root }); + assert.deepEqual(workspace.integrations, []); + const report = await runDoctor(workspace); + assert.equal( + report.issues.some((issue) => + String(issue.code).startsWith("integration-health-check") + ), + false + ); + // No health hook to call means no bounding timer was ever armed, so the + // report is produced and the process is free to exit immediately. + assert.deepEqual(Object.keys(report.counts).sort(), [ + "error", + "info", + "warning" + ]); + } finally { + await rm(root, { recursive: true, force: true }); + } +}); + test("doctor flags a search.provider no declared integration satisfies", async () => { const root = await workspaceWithConfig( `export default { From f3f0824f3c805d08ef76eb7b3d9f23c492e9e64c Mon Sep 17 00:00:00 2001 From: illodev Date: Fri, 7 Aug 2026 20:21:32 +0200 Subject: [PATCH 2/5] Two claims are one process only when one session, never when one actor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The skip that decided a conflict compared `claimed_by`, which reads as a session check only because `resolveActor` writes a session discriminator into the actor's tail. So it was right often enough to look correct and wrong exactly where it matters: two plain terminals both resolve to `user@host`, and two agents handed the same `--actor` both resolve to that. Each is two processes about to overwrite each other, and each was dropped as one person. `claimSeparation` now answers "provably the same process" and names its evidence — `sessions-differ`, `actors-differ`, or `unproven`. A session is recovered from either place that carries it, the session file or the actor's tail, both normalised through `sessionDiscriminator`, now exported so there is one definition instead of two that can drift. A second defect fed the same rule: `claimState` resolved a claim's session with one `find` over `cardId === card.id || actor === claimed_by`, so two cards held by one actor string could both be attributed to whichever session came first — erasing the exact evidence the comparison needs. Attribution now prefers the session that names the card. ADR-0020 records the sessionless case: `unproven` is reported, not dropped and not prompted on. Dropping it is the bug that let two terminals collide with no trace; prompting on it interrupts somebody about a card they claimed themselves, which is the guard people switch off. The scope guard needed no behaviour change, and that is the finding rather than a shortcut — because the session lives in the actor's tail, its string comparison *is* the session comparison for every pairing it can see. Rather than argue that, `separatesFromMe` names it and a new pin drives the real hook over six pairings against `claimSeparation`. One asymmetry stays, recorded rather than half-fixed: the snapshot can read a session from a session file and the guard cannot, because `board.json` carries none and the hook imports nothing from the package on a p95-under- 30ms budget. LRN-0030 and T-0219. Cards: T-0206 Filed: T-0219 Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01D3LTdq3mzMAQ98rwBegGjU --- ...held-by-one-resolved-actor-collide-invi.md | 31 +++- ...oard-carries-no-session-so-the-guard-mi.md | 46 +++++ ...-when-one-session-and-an-unproven-colli.md | 82 +++++++++ ...a-claim-than-the-activity-snapshot-does.md | 52 ++++++ packages/workfile/src/core/actor.ts | 10 +- packages/workfile/src/modules/cards/claims.ts | 71 +++++++- packages/workfile/src/modules/cards/index.ts | 2 + .../workfile/src/runtime/claude/hooks.mjs | 29 ++- packages/workfile/test/claims.test.ts | 171 ++++++++++++++++++ packages/workfile/test/claude-surface.test.ts | 123 +++++++++++++ packages/workfile/ui/src/types.ts | 12 +- plugins/workfile/runtime/hooks.mjs | 29 ++- 12 files changed, 641 insertions(+), 17 deletions(-) create mode 100644 .project/cards/T-0219-the-claim-board-carries-no-session-so-the-guard-mi.md create mode 100644 .project/memory/decisions/ADR-0020-two-claims-are-one-process-only-when-one-session-and-an-unproven-colli.md create mode 100644 .project/memory/learnings/LRN-0030-the-scope-guard-sees-less-of-a-claim-than-the-activity-snapshot-does.md diff --git a/.project/cards/T-0206-two-claims-held-by-one-resolved-actor-collide-invi.md b/.project/cards/T-0206-two-claims-held-by-one-resolved-actor-collide-invi.md index 64d647f..243deee 100644 --- a/.project/cards/T-0206-two-claims-held-by-one-resolved-actor-collide-invi.md +++ b/.project/cards/T-0206-two-claims-held-by-one-resolved-actor-collide-invi.md @@ -1,16 +1,22 @@ --- id: T-0206 title: Two claims held by one resolved actor collide invisibly -status: backlog +status: done type: bug priority: medium area: core tags: [claims] effort: S -scope: [packages/workfile/src/modules/cards/claims.ts] +scope: [packages/workfile/src/modules/cards/claims.ts, packages/workfile/src/core/actor.ts, packages/workfile/src/runtime/claude/hooks.mjs, plugins/workfile/runtime/hooks.mjs, packages/workfile/test/claims.test.ts, packages/workfile/test/claude-surface.test.ts, packages/workfile/ui/src/types.ts] origin: [T-0196] created: 2026-08-05 -updated: 2026-08-05 +updated: 2026-08-07 +related: [ADR-0020, LRN-0030, T-0219, T-0196] +verified: + at: "2026-08-07T17:02:17.071Z" + method: local + commit: 4e8da0782fecb7e52899f7916be21ad7f3d4c775 + digest: "sha256:32694851285580f3877b44d3ff44fcf51cff487cf052e8131da85045701783d8" --- `activity.conflicts` pairs claimed cards whose scopes overlap and skips pairs @@ -35,7 +41,18 @@ Surfaced by T-0196, whose popover can only report the conflicts it is handed. ## Acceptance criteria -- [ ] Two claims from different sessions with overlapping scopes are reported as a conflict whatever their actors resolve to. -- [ ] A single session moving between its own overlapping cards is still not a conflict. -- [ ] The rule for two claims with no session is decided, recorded and tested. -- [ ] The claim board the scope guard reads applies the same rule as the activity snapshot. +- [x] Two claims from different sessions with overlapping scopes are reported as a conflict whatever their actors resolve to. +- [x] A single session moving between its own overlapping cards is still not a conflict. +- [x] The rule for two claims with no session is decided, recorded and tested. +- [x] The claim board the scope guard reads applies the same rule as the activity snapshot. + +## Activity + +- 2026-08-07 16:52Z illodev@local#42eb42f5 · claimed +- 2026-08-07 17:02Z illodev@local#42eb42f5 · released + +## Notes + +- 2026-08-07 17:02Z illodev@local#42eb42f5 — Fixed. The skip compared actors, which reads as a session check only because resolveActor writes the discriminator into the actor's tail — so it was right often enough to look correct and wrong exactly where it mattered. claimSeparation now answers "provably the same process" and names its evidence: sessions-differ, actors-differ, or unproven. A session is recovered from either place that carries it, the session file or the actor's tail, both normalised through sessionDiscriminator, now exported from core/actor.ts so there is one definition instead of two that can drift. Found a second defect feeding the same rule: claimState resolved a claim's session with one find over `cardId === card.id || actor === claimed_by`, so two cards held by one actor string could both be attributed to whichever session came first — erasing the exact evidence the comparison needs. Attribution now prefers the session that names the card. The decision on the sessionless case is ADR-0020: unproven is reported, not dropped and not prompted on. Dropping it is the bug that let two terminals collide with no trace; prompting on it interrupts somebody about a card they claimed themselves, which is the guard people switch off. So the snapshot carries it with its basis and the scope guard stays silent on it. Criterion 4 turned out not to need a behaviour change, and that is the finding rather than a shortcut: because the session lives in the actor's tail, the guard's string comparison IS the session comparison for every pairing it can see. Rather than argue that, separatesFromMe names it and a new pin drives the real hook over six pairings and requires its silence to match claimSeparation — verified by mutation, flipping the comparison to `return true` fails four of the six with the diagnostic naming both verdicts, and restoring it goes green. One asymmetry stays and is recorded rather than half-fixed: the snapshot can read a session from a session file, the guard cannot, because board.json carries no session and the hook imports nothing from the package on a p95-under-30ms budget. So two agents sharing an explicit --actor are a reported conflict the guard will not prompt about — LRN-0030 and T-0219. +- 2026-08-07 17:02Z illodev@local#42eb42f5 — local verification: pnpm run check green: 464+7 tests pass (4 new), strictNullChecks held at 488. New rule tests cover the session table, two terminals sharing an actor reported as unproven, one actor string over two sessions reported as sessions-differ, and one session over two overlapping cards reported as no conflict. Guard pin drives the real PreToolUse hook over six pairings against claimSeparation; mutation-checked by flipping separatesFromMe to , which fails 4 of 6, and restoring it returns 19/19. Plugin runtime regenerated so both copies match. doctor 0/0, memory verify 0/0. +- 2026-08-07 17:02Z illodev@local#42eb42f5 — Correction to the verification entry above: a shell substitution ate two words from it. The mutation check flipped separatesFromMe to return true — that is what failed 4 of the 6 guard pairings before restoring it returned 19/19. diff --git a/.project/cards/T-0219-the-claim-board-carries-no-session-so-the-guard-mi.md b/.project/cards/T-0219-the-claim-board-carries-no-session-so-the-guard-mi.md new file mode 100644 index 0000000..0a93ef5 --- /dev/null +++ b/.project/cards/T-0219-the-claim-board-carries-no-session-so-the-guard-mi.md @@ -0,0 +1,46 @@ +--- +id: T-0219 +title: The claim board carries no session, so the guard misses a shared actor +status: backlog +type: task +priority: low +area: core +tags: [claims, hooks] +effort: S +scope: [packages/workfile/src/modules/cards/claims.ts, packages/workfile/src/runtime/claude/hooks.mjs] +related: [T-0089] +origin: [T-0206, LRN-0030] +created: 2026-08-07 +updated: 2026-08-07 +--- + +The residual ADR-0020 left open, recorded in full in LRN-0030. + +`claimSeparation` decides whether two claims are two processes, and both the +activity snapshot and the `PreToolUse` scope guard apply it. They do not always +agree, because the board the guard reads carries `claimedBy` and no session: +`claimBoardEntry` and the hook's own `buildBoard` write the same six fields. So +the guard recovers a session only from the actor's tail, and a `claimed_by` +written from an explicit `--actor` has no tail. + +One case is therefore invisible to the guard and reported by the snapshot: two +agents claiming overlapping scopes with the same explicit `--actor`. The guard +sees one string equal to its own and stays silent. + +The fix is to resolve the session once where the board is built — both writers +hold the session files at that moment — and put it on the entry, so the hook +reads it for free. Teaching the hook to read the session directory instead is the +wrong shape: a `PreToolUse` runs before every matching tool call and its budget +is p95 under 30 ms. + +Low priority because a hand-typed `--actor` already breaks other things — it +guards you out of your own card on release — so the configuration this protects +is one the protocol steers away from. Worth doing when the board is next touched; +T-0089 is already about board staleness and would pass through the same code. + +## Acceptance criteria + +- [ ] A board entry carries the session resolved for its claim, from either source. +- [ ] Both writers of the board — `rebuildClaimBoard` and the hook's `buildBoard` — produce the same entry for the same card, proven by a test. +- [ ] Two agents sharing an explicit `--actor` make the guard prompt, proven by driving the real hook. +- [ ] The hook's latency budget still holds. diff --git a/.project/memory/decisions/ADR-0020-two-claims-are-one-process-only-when-one-session-and-an-unproven-colli.md b/.project/memory/decisions/ADR-0020-two-claims-are-one-process-only-when-one-session-and-an-unproven-colli.md new file mode 100644 index 0000000..88853a0 --- /dev/null +++ b/.project/memory/decisions/ADR-0020-two-claims-are-one-process-only-when-one-session-and-an-unproven-colli.md @@ -0,0 +1,82 @@ +--- +id: ADR-0020 +title: Two claims are one process only when one session, and an unproven collision is reported not prompted +status: accepted +related: [T-0206, T-0196, LRN-0030] +tags: [claims, protocol] +created: 2026-08-07 +updated: 2026-08-07 +--- + +## Context + +`activity.conflicts` paired claimed cards whose scopes overlapped and skipped +any pair sharing a `claimed_by`. The skip was written for a true case — one +person moving between their own cards is not a collision — and it read as +correct because `resolveActor` appends a session discriminator to the actor, so +two agents in one checkout usually differ. + +It silently covered two cases that are two processes about to overwrite each +other: + +- Two plain terminals. Both resolve to `user@host` with no discriminator, both + claims look like the same person, and the overlap was dropped. +- Two agents handed the same `--actor`, which is what the generated protocol + used to teach. Same string, different sessions. + +The attribution feeding the comparison had its own defect. `claimState` found a +claim's session with one `find` over `cardId === card.id || actor === +claimed_by`, so for two cards held by one actor string it could return whichever +session came first and attribute both cards to it — erasing exactly the evidence +the rule needs. + +## Decision + +The question is not "same actor" but "provably the same process", and the answer +carries its own evidence. `claimSeparation` in +`packages/workfile/src/modules/cards/claims.ts` returns null for one process, or +what told the pair apart: + +- `sessions-differ` — two sessions, seen. Also the verdict when one side has a + session and the other does not. +- `actors-differ` — no session either side, different actors. Two people. +- `unproven` — no session either side and the same actor. + +A session is recovered from either place that carries it: a live session file +knows its own id, and a `claimed_by` written by a process that resolved its own +actor carries the discriminator in its tail, which outlives the session file and +survives into git. Both are normalized through `sessionDiscriminator`, now +exported from `core/actor.ts` so there is one definition rather than two that can +drift. Session attribution prefers the session naming the card over any session +merely sharing its actor. + +**`unproven` is reported, not dropped, and not prompted on.** One person holding +two overlapping cards and two terminals racing each other are the same record; +nothing in the workspace distinguishes them. Dropping it is the bug — it is what +let two terminals collide with no trace anywhere. Prompting on it is the other +failure: a guard that interrupts somebody about a card they claimed themselves is +the guard they switch off, and then it protects nothing. So the activity snapshot +reports it with its basis, where a reader can weigh it and nobody is interrupted, +and the `PreToolUse` scope guard stays silent on it. + +## Consequences + +The guard needed no behavioural change, and that is worth stating rather than +leaving to look like luck: because `actorFor` writes the session into the actor's +tail, comparing `claimed_by` against this session's actor *is* comparing +sessions, for every pairing the guard can see. The equivalence is now named +(`separatesFromMe`) and driven by a test that runs the real hook over every +pairing and requires its silence to match `claimSeparation` — flipping the +guard's comparison fails four of the six cases. + +Consumers gain an optional `basis` on each conflict. The popover from T-0196 can +render an unproven overlap as possible rather than certain; nothing is required +to read the field. + +One asymmetry remains and LRN-0030 records it: the snapshot can resolve a session +from a session file, and the guard cannot, because the board it reads carries +only `claimed_by` and the hook deliberately imports nothing from the package. So +two agents sharing an explicit `--actor` are a reported conflict the guard will +not prompt about. Closing it means putting the session on the board, and the +hook's latency budget is the reason that is a separate decision rather than a +line in this one. diff --git a/.project/memory/learnings/LRN-0030-the-scope-guard-sees-less-of-a-claim-than-the-activity-snapshot-does.md b/.project/memory/learnings/LRN-0030-the-scope-guard-sees-less-of-a-claim-than-the-activity-snapshot-does.md new file mode 100644 index 0000000..a7315e0 --- /dev/null +++ b/.project/memory/learnings/LRN-0030-the-scope-guard-sees-less-of-a-claim-than-the-activity-snapshot-does.md @@ -0,0 +1,52 @@ +--- +id: LRN-0030 +title: The scope guard sees less of a claim than the activity snapshot does +status: active +category: infra +confidence: high +related: [T-0206, ADR-0020, T-0089] +tags: [claims, hooks] +created: 2026-08-07 +updated: 2026-08-07 +--- + +Recorded for T-0206 against 0.8.1. Both surfaces apply one rule — +`claimSeparation` — and they do not always reach the same verdict, because they +do not see the same evidence. Knowing which is which saves the next person the +investigation. + +**A session can be carried in two places.** A session file under +`.project/.cache/activity/sessions` knows its own id and, when written by +`recordAgentSignal`, the card it is on. A `claimed_by` carries the session in its +tail — `solo@box#e55eab30` — whenever the claiming process resolved its own +actor. The tail outlives the session file and survives into git; the file is the +only source when an explicit `--actor` was passed, because a hand-typed actor has +no tail. + +**The snapshot reads both. The guard reads only the tail.** `buildActivitySnapshot` +holds the session files, so `claimSession` prefers the file that names the card. +The `PreToolUse` guard reads `board.json`, whose entries are `id`, `title`, +`status`, `claimedBy`, `claimedAt` and `scope` — no session — and the hook +deliberately imports nothing from the package, so it cannot resolve one. Its +latency budget is p95 under 30 ms and a `PreToolUse` runs before *every* matching +call, which is the whole reason for that constraint. + +**What this costs, exactly.** One case: two agents claiming overlapping scopes +with the same explicit `--actor`. The snapshot reports `sessions-differ` from the +two session files; the guard sees one actor string equal to its own and stays +silent. Every other pairing agrees, because the tail makes actor equality and +session equality the same comparison — pinned by a test in +`test/claude-surface.test.ts` that runs the real hook over each case. + +**How to apply.** Do not read the guard's silence as "no conflict": read +`activity.conflicts`, which is the surface with the evidence. Closing the gap +means putting the resolved session on the board rather than teaching the hook to +read session files — the board is already written by both the package +(`rebuildClaimBoard`) and the hook's own `buildBoard`, so it is the one place a +session can be resolved once and read for free. Note that board staleness is a +separate known problem (T-0089): it is built at session start, so a claim taken +mid-session is invisible to the guard whatever the entries carry. + +**And the trap underneath all of it.** Never compare claims by actor and call it +a session check. It is right often enough to look correct and wrong exactly when +it matters — two plain terminals, or one `--actor` shared by two agents. diff --git a/packages/workfile/src/core/actor.ts b/packages/workfile/src/core/actor.ts index 5ad1580..126f224 100644 --- a/packages/workfile/src/core/actor.ts +++ b/packages/workfile/src/core/actor.ts @@ -147,8 +147,16 @@ export function resolveActor(options: ResolveActorOptions = {}): ResolvedActor { * A UUID's first block is already distinct enough to separate the handful of * sessions that can share one checkout, and it stays short enough that the * actor is still a name rather than a token. + * + * Exported because it is not only how an actor is *written*: it is also how a + * claim is read back. `claimSession` in `modules/cards/claims.ts` recovers the + * session from a `claimed_by` written by an earlier process, and a second copy + * of this normalization there would let the two drift apart silently — the + * comparison would start answering "different session" for one session. */ -function sessionDiscriminator(sessionId: string | undefined): string | undefined { +export function sessionDiscriminator( + sessionId: string | undefined +): string | undefined { if (!sessionId) return undefined; const cleaned = sessionId.replace(/[^A-Za-z0-9]/g, ""); if (!cleaned) return undefined; diff --git a/packages/workfile/src/modules/cards/claims.ts b/packages/workfile/src/modules/cards/claims.ts index 48be519..18d541c 100644 --- a/packages/workfile/src/modules/cards/claims.ts +++ b/packages/workfile/src/modules/cards/claims.ts @@ -1,6 +1,7 @@ import { readdir, readFile, rm } from "node:fs/promises"; import { join } from "node:path"; +import { sessionDiscriminator } from "../../core/actor.js"; import { writeFileAtomic } from "../../core/filesystem.js"; import { withFileLock } from "../../core/locks.js"; import { exists } from "../../core/fs-utils.js"; @@ -239,10 +240,13 @@ export function claimState(card, sessions, { leaseHours, now = new Date() }) { if (!card.claimed_by) return { state: "unclaimed" }; const claimedAt = Date.parse(card.claimed_at || ""); const ageMs = Number.isFinite(claimedAt) ? now.getTime() - claimedAt : null; - const session = sessions.find( - (candidate) => - candidate.cardId === card.id || candidate.actor === card.claimed_by - ); + // The card's own session wins over any session merely sharing its actor. + // As one `find` over an `||` this returned whichever session came first, so + // two cards held by one actor string could both be attributed to the same + // session — which is exactly the evidence the conflict rule below reads. + const session = + sessions.find((candidate) => candidate.cardId === card.id) || + sessions.find((candidate) => candidate.actor === card.claimed_by); const base = { by: card.claimed_by, at: card.claimed_at || null, @@ -263,6 +267,60 @@ export function claimState(card, sessions, { leaseHours, now = new Date() }) { return { ...base, state: "held" }; } +/** + * The session a claim was made from, however it can be recovered. + * + * Two places carry it and neither is always present. A live session file knows + * its own id; a `claimed_by` written by any process that resolved its own actor + * carries the discriminator in its tail, which outlives the session file and + * survives into git. Normalized through `sessionDiscriminator` so a full id from + * a session file and an eight-character tail from an actor compare equal. + */ +export function claimSession(claim: { + by?: string | null; + sessionId?: string | null; +}): string | null { + const fromSession = sessionDiscriminator(claim.sessionId || undefined); + if (fromSession) return fromSession; + const tail = /#([A-Za-z0-9]+)$/.exec(String(claim.by || "")); + return tail ? sessionDiscriminator(tail[1]) || null : null; +} + +/** + * What tells two claims apart, or nothing if they are one process. + * + * The rule this replaces compared `claimed_by`, and that reads as a session only + * by accident: `resolveActor` appends a session discriminator, so two agents + * *usually* differ. Two plain terminals resolve to the same `user@host` and were + * dropped as one person; so were two agents that were handed the same `--actor`. + * Both are two processes about to overwrite each other. + * + * So the question is not "same actor" but "provably the same process", and the + * answer names its own evidence, because the three cases are not equally strong: + * + * - `sessions-differ` — two sessions, seen. Two processes. + * - `actors-differ` — no session either side, different actors. Two people. + * - `unproven` — no session either side and the same actor. One person holding + * two overlapping cards and two terminals racing each other are the same + * record; nothing in the workspace distinguishes them. + * + * `unproven` is reported rather than dropped, and that is the decision T-0206 + * had to make. Silence is the bug — it is what let two terminals collide with + * no trace. But a consumer that interrupts somebody must be able to tell a + * verdict from a guess, which is what the label is for: the popover can show it, + * and the scope guard does not prompt on it (see `plugins/workfile/runtime/hooks.mjs`). + */ +export function claimSeparation( + a: { by?: string | null; sessionId?: string | null }, + b: { by?: string | null; sessionId?: string | null } +): "sessions-differ" | "actors-differ" | "unproven" | null { + const left = claimSession(a); + const right = claimSession(b); + if (left && right) return left === right ? null : "sessions-differ"; + if (left || right) return "sessions-differ"; + return a.by === b.by ? "unproven" : "actors-differ"; +} + /** * Everything that is happening in the workspace right now. * @@ -296,7 +354,8 @@ export async function buildActivitySnapshot( for (let right = left + 1; right < claims.length; right += 1) { const a = claims[left]; const b = claims[right]; - if (a.claim.by === b.claim.by) continue; + const basis = claimSeparation(a.claim, b.claim); + if (!basis) continue; const shared = a.scope.filter((path) => b.scope.some( (other) => @@ -306,7 +365,7 @@ export async function buildActivitySnapshot( ) ); if (shared.length) { - conflicts.push({ cards: [a.id, b.id], paths: shared }); + conflicts.push({ cards: [a.id, b.id], paths: shared, basis }); } } } diff --git a/packages/workfile/src/modules/cards/index.ts b/packages/workfile/src/modules/cards/index.ts index 66042f9..2f3d0e7 100644 --- a/packages/workfile/src/modules/cards/index.ts +++ b/packages/workfile/src/modules/cards/index.ts @@ -55,6 +55,8 @@ export { buildActivitySnapshot, claimBoardChanged, claimBoardEntry, + claimSeparation, + claimSession, claimState, readActiveLocks, readClaimBoard, diff --git a/packages/workfile/src/runtime/claude/hooks.mjs b/packages/workfile/src/runtime/claude/hooks.mjs index 6b27631..3700059 100644 --- a/packages/workfile/src/runtime/claude/hooks.mjs +++ b/packages/workfile/src/runtime/claude/hooks.mjs @@ -167,6 +167,33 @@ const actorFor = (input) => { return `${user}@${host}${suffix}`; }; +/** + * Whether a claim belongs to some process other than this one. + * + * The rule is `claimSeparation` in `modules/cards/claims.ts`: two claims are one + * process only when provably one session, and an actor is not a session. Here it + * collapses back to comparing the strings, and that is worth stating rather than + * leaving to look like a coincidence — `actorFor` writes the session + * discriminator into the tail, so for every pairing this guard can see, actor + * equality *is* session equality: + * + * - both tails present and equal, or both absent with the same actor → one + * process, or `unproven` and deliberately not prompted on. A configured + * `WORKFILE_ACTOR` is somebody declaring an identity, and interrupting them + * about their own claim is how a guard rail gets switched off. + * - tails differing, or one present and one absent → two processes. + * - no tails and different actors → two people. + * + * So this stays a string comparison, and the pinning test in + * `test/claude-surface.test.ts` drives both derivations over every case rather + * than trusting the paragraph above. What the guard cannot see is a session that + * exists only in a session file — `claimed_by` written from an explicit + * `--actor` carries no tail — and the snapshot can. That residual is LRN-0030. + */ +function separatesFromMe(claimedBy, mine) { + return claimedBy !== mine; +} + const SESSIONS = `${CACHE}/sessions`; /** @@ -368,7 +395,7 @@ async function preToolUse(input) { const conflict = board.claims.find( (claim) => claim.status === "doing" && - claim.claimedBy !== mine && + separatesFromMe(claim.claimedBy, mine) && claim.scope.length && scopeCovers(claim.scope, repoPath) ); diff --git a/packages/workfile/test/claims.test.ts b/packages/workfile/test/claims.test.ts index 97461ca..0498a7f 100644 --- a/packages/workfile/test/claims.test.ts +++ b/packages/workfile/test/claims.test.ts @@ -10,6 +10,8 @@ import { createTestWorkspace } from "./support/workspace.ts"; import { buildActivitySnapshot, claimCard, + claimSeparation, + claimSession, claimState, createCard, loadCards, @@ -308,3 +310,172 @@ test("the activity snapshot answers who is working on what", async () => { await cleanup(); } }); + +/** + * T-0206: the skip that decided a conflict compared actors, not processes. + * + * `resolveActor` writes a session discriminator into the actor's tail, so two + * agents *usually* differ and the old comparison looked right. Two plain + * terminals both resolve to `user@host`, and two agents handed the same + * `--actor` both resolve to that — in each case the pair was dropped as one + * person, and each is two processes about to overwrite each other. + */ +test("what separates two claims is the session, and it names its own evidence", () => { + // A full session id from a session file and an eight-character tail from an + // actor are the same session; if these ever normalise differently the whole + // comparison starts answering "different" for one process. + assert.equal( + claimSession({ by: "solo@box#e55eab30", sessionId: null }), + "e55eab30" + ); + assert.equal( + claimSession({ by: "solo@box", sessionId: "E55EAB30-b661-4290-bd58" }), + "e55eab30" + ); + assert.equal(claimSession({ by: "solo@box", sessionId: null }), null); + + const same = { by: "solo@box#aaaaaaaa", sessionId: null }; + const other = { by: "solo@box#bbbbbbbb", sessionId: null }; + assert.equal(claimSeparation(same, { ...same }), null, "one session"); + + // The case the old rule missed: one actor string, two sessions. + assert.equal( + claimSeparation( + { by: "shared", sessionId: "sess-a" }, + { by: "shared", sessionId: "sess-b" } + ), + "sessions-differ" + ); + assert.equal(claimSeparation(same, other), "sessions-differ"); + assert.equal( + claimSeparation(same, { by: "solo@box", sessionId: null }), + "sessions-differ", + "a session on one side is still two processes" + ); + + // No session either side. Different actors are two people; the same actor is + // the case nothing in the workspace can decide. + assert.equal( + claimSeparation( + { by: "alvaro@box", sessionId: null }, + { by: "other@box", sessionId: null } + ), + "actors-differ" + ); + assert.equal( + claimSeparation( + { by: "solo@box", sessionId: null }, + { by: "solo@box", sessionId: null } + ), + "unproven" + ); +}); + +test("two terminals sharing one actor are reported, marked as unproven", async () => { + const { workspace, cleanup } = await createTestWorkspace({ + prefix: "workfile-claims-unproven-" + }); + try { + const first = await createCard(workspace, { title: "One", area: "api" }); + const second = await createCard(workspace, { title: "Two", area: "api" }); + // Exactly what two plain terminals write: no session, so no tail. + await claimCard(workspace, first.id, { + actor: "solo@box", + scope: ["src/api"] + }); + await claimCard(workspace, second.id, { + actor: "solo@box", + scope: ["src/api/billing"] + }); + + const { cards } = await loadCards(workspace); + const snapshot = await buildActivitySnapshot(workspace, cards); + + // Before this the pair was dropped: two processes overwriting each other + // with nothing anywhere saying so. + assert.equal(snapshot.conflicts.length, 1); + assert.deepEqual(snapshot.conflicts[0].cards.sort(), [ + first.id, + second.id + ].sort()); + assert.equal( + snapshot.conflicts[0].basis, + "unproven", + "one person with two cards is the same record as two terminals racing" + ); + } finally { + await cleanup(); + } +}); + +test("one actor string over two sessions is a conflict, and one session is not", async () => { + const { workspace, cleanup } = await createTestWorkspace({ + prefix: "workfile-claims-sessions-" + }); + try { + const left = await createCard(workspace, { title: "Left", area: "api" }); + const right = await createCard(workspace, { title: "Right", area: "api" }); + // Two agents handed the same `--actor`, which is what the protocol used + // to teach. The actor cannot tell them apart; the sessions can. + await claimCard(workspace, left.id, { + actor: "shared-agent", + scope: ["src/api"] + }); + await claimCard(workspace, right.id, { + actor: "shared-agent", + scope: ["src/api/billing"] + }); + await recordAgentSignal(workspace, { + sessionId: "aaaaaaaa-1111", + actor: "shared-agent", + cardId: left.id, + files: ["src/api/billing.ts"] + }); + await recordAgentSignal(workspace, { + sessionId: "bbbbbbbb-2222", + actor: "shared-agent", + cardId: right.id, + files: ["src/api/billing.ts"] + }); + + const { cards } = await loadCards(workspace); + const snapshot = await buildActivitySnapshot(workspace, cards); + assert.equal(snapshot.conflicts.length, 1); + assert.equal(snapshot.conflicts[0].basis, "sessions-differ"); + + // And the other half of the rule: one session holding both overlapping + // cards is not colliding with itself. Attribution has to prefer the + // session that names the card — as a single `find` over an `||` this + // returned whichever session came first, so both cards could be + // attributed to one session and a real conflict disappeared. + const solo = await createTestWorkspace({ + prefix: "workfile-claims-solo-" + }); + try { + const a = await createCard(solo.workspace, { + title: "A", + area: "api" + }); + const b = await createCard(solo.workspace, { + title: "B", + area: "api" + }); + for (const card of [a, b]) { + await claimCard(solo.workspace, card.id, { + actor: "solo@box#cccccccc", + scope: ["src/api"] + }); + } + const listing = await loadCards(solo.workspace); + const own = await buildActivitySnapshot( + solo.workspace, + listing.cards + ); + assert.deepEqual(own.conflicts, []); + } finally { + await solo.cleanup(); + } + } finally { + await cleanup(); + } +}); diff --git a/packages/workfile/test/claude-surface.test.ts b/packages/workfile/test/claude-surface.test.ts index 47c5fca..df64d4b 100644 --- a/packages/workfile/test/claude-surface.test.ts +++ b/packages/workfile/test/claude-surface.test.ts @@ -18,6 +18,7 @@ import { buildActivitySnapshot, checkClaudeSurface, claimCard, + claimSeparation, claudeCommandFiles, claudeHooksFile, claudeMcpFile, @@ -1616,3 +1617,125 @@ test("every matcher covers the events its handler acts on", async () => { await rm(root, { recursive: true, force: true }); } }); + +/** + * T-0206: the guard and the activity snapshot must answer one question. + * + * The snapshot decides whether two claims are two processes; the guard decides + * whether *this* process holds the claim covering the file. Same question, and + * the rule is `claimSeparation`. The guard reaches it by comparing `claimed_by` + * against its own actor, which reads as an actor comparison and is a session + * comparison — `actorFor` writes the session into the tail. That equivalence is + * the kind of thing that is true until somebody changes one side, so it is + * driven rather than argued: the real hook runs for each pairing and its silence + * has to match what `claimSeparation` says about the same two identities. + * + * `unproven` counts as silence. Two sessionless claims under one actor cannot be + * told apart, and a guard that prompts a person about a card they claimed + * themselves is the guard people switch off — the snapshot reports it instead, + * where nobody is interrupted. + */ +const SEPARATION_CASES: ReadonlyArray<{ + label: string; + claimedBy: string; + session: string | null; + actorEnv?: string; +}> = [ + { + label: "my own session", + claimedBy: "solo@box#e55eab30", + session: "e55eab30-b661-4290-bd58-d3b3a82f3b48" + }, + { + label: "another agent's session", + claimedBy: "solo@box#aaaaaaaa", + session: "e55eab30-b661-4290-bd58-d3b3a82f3b48" + }, + { + label: "a plain terminal, seen by an agent", + claimedBy: "solo@box", + session: "e55eab30-b661-4290-bd58-d3b3a82f3b48" + }, + { + label: "another person entirely", + claimedBy: "someone@else", + session: "e55eab30-b661-4290-bd58-d3b3a82f3b48" + }, + { + label: "a configured actor claiming its own card", + claimedBy: "ci-runner", + session: "e55eab30-b661-4290-bd58-d3b3a82f3b48", + actorEnv: "ci-runner" + }, + { + label: "a configured actor over somebody else's card", + claimedBy: "solo@box#aaaaaaaa", + session: "e55eab30-b661-4290-bd58-d3b3a82f3b48", + actorEnv: "ci-runner" + } +]; + +test("the scope guard and the activity snapshot apply one separation rule", async () => { + for (const scenario of SEPARATION_CASES) { + const root = await mkdtemp(join(tmpdir(), "workfile-separation-")); + try { + await cp(fixture, root, { recursive: true }); + const workspace = await loadWorkspace({ root }); + const card = await createCard(workspace, { + title: scenario.label, + area: "api" + }); + await claimCard(workspace, card.id, { + actor: scenario.claimedBy, + scope: ["src/api"] + }); + + const env = { + ...BLANK, + USER: "solo", + HOSTNAME: "box", + ...(scenario.actorEnv ? { WORKFILE_ACTOR: scenario.actorEnv } : {}) + }; + const mine = + scenario.actorEnv || + resolveActor({ sessionId: scenario.session, env }).actor; + + // What the snapshot would say about these two identities. The guard + // compares a claim against a live process rather than two claims, so + // the second side is this session's identity. + const basis = claimSeparation( + { by: scenario.claimedBy, sessionId: null }, + { by: mine, sessionId: scenario.actorEnv ? null : scenario.session } + ); + const shouldPrompt = Boolean(basis) && basis !== "unproven"; + + // The board the guard reads is written at session start. + await runHook( + "session-start", + { session_id: scenario.session }, + root, + env + ); + const guard = await runHook( + "pre-tool-use", + { + session_id: scenario.session, + tool_name: "Edit", + tool_input: { file_path: join(root, "src/api/billing.ts") } + }, + root, + env + ); + const prompted = guard.stdout.includes("permissionDecision"); + assert.equal( + prompted, + shouldPrompt, + `${scenario.label}: claimSeparation said ${basis ?? "one process"}, the guard ${ + prompted ? "prompted" : "stayed silent" + }` + ); + } finally { + await rm(root, { recursive: true, force: true }); + } + } +}); diff --git a/packages/workfile/ui/src/types.ts b/packages/workfile/ui/src/types.ts index 7f1346f..6b12bdf 100644 --- a/packages/workfile/ui/src/types.ts +++ b/packages/workfile/ui/src/types.ts @@ -383,7 +383,17 @@ export interface ActivitySnapshot { filesTouched: string[]; }>; claims: ClaimEntry[]; - conflicts: Array<{ cards: string[]; paths: string[] }>; + conflicts: Array<{ + cards: string[]; + paths: string[]; + /** + * What told the two claims apart. `unproven` means neither claim carried + * a session and both resolved to the same actor, so one person holding + * two overlapping cards and two terminals racing cannot be distinguished + * — report it as possible, never as certain. + */ + basis?: "sessions-differ" | "actors-differ" | "unproven"; + }>; writing: Array<{ recordId: string | null; module: string | null }>; } diff --git a/plugins/workfile/runtime/hooks.mjs b/plugins/workfile/runtime/hooks.mjs index 6b27631..3700059 100644 --- a/plugins/workfile/runtime/hooks.mjs +++ b/plugins/workfile/runtime/hooks.mjs @@ -167,6 +167,33 @@ const actorFor = (input) => { return `${user}@${host}${suffix}`; }; +/** + * Whether a claim belongs to some process other than this one. + * + * The rule is `claimSeparation` in `modules/cards/claims.ts`: two claims are one + * process only when provably one session, and an actor is not a session. Here it + * collapses back to comparing the strings, and that is worth stating rather than + * leaving to look like a coincidence — `actorFor` writes the session + * discriminator into the tail, so for every pairing this guard can see, actor + * equality *is* session equality: + * + * - both tails present and equal, or both absent with the same actor → one + * process, or `unproven` and deliberately not prompted on. A configured + * `WORKFILE_ACTOR` is somebody declaring an identity, and interrupting them + * about their own claim is how a guard rail gets switched off. + * - tails differing, or one present and one absent → two processes. + * - no tails and different actors → two people. + * + * So this stays a string comparison, and the pinning test in + * `test/claude-surface.test.ts` drives both derivations over every case rather + * than trusting the paragraph above. What the guard cannot see is a session that + * exists only in a session file — `claimed_by` written from an explicit + * `--actor` carries no tail — and the snapshot can. That residual is LRN-0030. + */ +function separatesFromMe(claimedBy, mine) { + return claimedBy !== mine; +} + const SESSIONS = `${CACHE}/sessions`; /** @@ -368,7 +395,7 @@ async function preToolUse(input) { const conflict = board.claims.find( (claim) => claim.status === "doing" && - claim.claimedBy !== mine && + separatesFromMe(claim.claimedBy, mine) && claim.scope.length && scopeCovers(claim.scope, repoPath) ); From 543ba6792c70db8aa73af5a7af9fd7c16848c9f5 Mon Sep 17 00:00:00 2001 From: illodev Date: Fri, 7 Aug 2026 20:21:47 +0200 Subject: [PATCH 3/5] Check what callers send to the CLI, not only what each subcommand reads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `COMMAND_FLAGS` is a contract with two sides and only one was checked. `cli.test.ts` pins the table against the flags each subcommand reads, in both directions, and it caught a stale `card archive --actor` in the branch that removed `init --language`. That branch still failed in CI, because nothing checked the flags a *caller* sends — and the callers were the generated agent instructions, the docs and the package smoke test. The new check covers exactly the sources `pnpm run test` does not execute: text under `.project/agents`, `.claude`, `plugins/workfile/commands`, the docs and the README, plus `scripts/` and `test/package-smoke.ts`. The unit tests are deliberately not scanned. They are full of flags that are meant not to exist — `--bogus`, `--nonsense`, `--statuss` — because asserting the refusal path is their job, and scanning them would mean an allowlist of intentional nonsense with a real stale flag able to hide in it. They need no scanning: a test that sends a removed flag fails when the suite runs. Command-word resolution mirrors `commandKey` and reads `USAGE_ALIASES` and `DEFAULT_SUBCOMMAND` out of the source, which the survey forced — without aliases, `workfile docs create --kind` in SPEC.md looked stale and it is a valid alias for `doc`. A word that resolves to no row is reported rather than dropped. It runs in 52ms, finds 116 invocations, and asserts a floor of 90 so a regex that stops matching fails loudly instead of passing forever. Proven by mutation on both halves: dropping `--scope` from `card claim` names 10 caller sites, dropping `--yes` from `init` names 3 including `package-smoke.ts:215` — the exact command that failed in CI. What it does not cover is named in the test itself, including the route not taken: this reads text, so `check` still proves nothing about the packaged artifact. Cards: T-0182 Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01D3LTdq3mzMAQ98rwBegGjU --- ...not-exercise-the-packaged-cli-so-a-remo.md | 25 +- ...both-sides-and-the-check-has-to-sit-in-.md | 49 ++++ packages/workfile/test/cli-callers.test.ts | 269 ++++++++++++++++++ 3 files changed, 339 insertions(+), 4 deletions(-) create mode 100644 .project/memory/learnings/LRN-0031-a-contract-needs-checking-from-both-sides-and-the-check-has-to-sit-in-.md create mode 100644 packages/workfile/test/cli-callers.test.ts diff --git a/.project/cards/T-0182-check-does-not-exercise-the-packaged-cli-so-a-remo.md b/.project/cards/T-0182-check-does-not-exercise-the-packaged-cli-so-a-remo.md index b077087..5967f98 100644 --- a/.project/cards/T-0182-check-does-not-exercise-the-packaged-cli-so-a-remo.md +++ b/.project/cards/T-0182-check-does-not-exercise-the-packaged-cli-so-a-remo.md @@ -1,12 +1,19 @@ --- id: T-0182 title: check does not exercise the packaged CLI, so a removed flag fails only in CI -status: backlog +status: done type: task priority: medium area: core created: 2026-08-05 -updated: 2026-08-05 +updated: 2026-08-07 +scope: [packages/workfile/test/cli-callers.test.ts] +related: [LRN-0031, T-0220, T-0158] +verified: + at: "2026-08-07T17:18:17.691Z" + method: local + commit: 4e8da0782fecb7e52899f7916be21ad7f3d4c775 + digest: "sha256:dd11bc8e72a0a90347c9acaf3cdd819d86ad1eab09e455f3e3ee7e52e8103c64" --- `pnpm run check` is what CLAUDE.md tells an agent to run before finishing, and it does not run `smoke:package`. That belongs to `check:release` and to a separate CI job, so the branch that removed `init --language` ([[T-0158]]) was green locally through fourteen commits and failed on the packaged CLI at the first command the smoke test runs. @@ -22,5 +29,15 @@ Same class as CI running Windows: the gap is not that the test is missing, it is ## Acceptance criteria -- [ ] A flag removed from the CLI fails a check that `pnpm run check` runs -- [ ] Whichever route is taken names what it still does not cover +- [x] A flag removed from the CLI fails a check that `pnpm run check` runs +- [x] Whichever route is taken names what it still does not cover + +## Activity + +- 2026-08-07 17:10Z illodev@local#42eb42f5 · claimed +- 2026-08-07 17:18Z illodev@local#42eb42f5 · released + +## Notes + +- 2026-08-07 17:18Z illodev@local#42eb42f5 — Took the first route, and the survey changed how it was built. Scanning every caller in test/ turns up five flags that are deliberately not real — --bogus, --nonsense, --statuss, and a real flag on the wrong subcommand — because asserting the refusal path is what those tests are for. Rather than carry an allowlist of intentional nonsense, in which a genuinely stale flag could hide, the checker skips the unit tests entirely: a test that sends a removed flag already fails when the suite runs. So it covers exactly the sources pnpm run test does not execute — the generated agent instructions, the docs, scripts/, and test/package-smoke.ts — which is also where the T-0158 failure actually lived. Command-word resolution mirrors commandKey and reads USAGE_ALIASES and DEFAULT_SUBCOMMAND out of the CLI source, which the survey forced: without aliases, workfile docs create --kind in SPEC.md looked like a stale command, and it is a valid alias for doc. An invocation whose word resolves to no row is reported rather than dropped, so the reverse case would surface. The check runs in 52ms, finds 116 invocations, and asserts a floor of 90 so a regex that stops matching fails loudly instead of passing forever. Proven by mutation both halves: removing --scope from the card claim row names 10 sites across .claude, .project/agents, README, SPEC, getting-started and the plugin; removing --yes from init names 3 including package-smoke.ts:215, which is the exact file and command that failed in CI. The largest thing it still does not cover is the other route, and it is named in the test rather than left implicit: this reads text, so check still proves nothing about the packaged artifact — whether the bin ships, whether the shebang survives, whether a consumer can resolve it. Folding smoke:package into check costs ~30s on the command agents run most, which deserves its own decision alongside T-0148 since both are about a gate that only runs on a tag push. Filed as T-0220. +- 2026-08-07 17:18Z illodev@local#42eb42f5 — local verification: pnpm run check green: 465+7 tests pass (1 new), strictNullChecks held at 488. cli-callers.test.ts runs in 52ms inside pnpm run test, finds 116 caller invocations across the generated instructions, docs, scripts and package-smoke.ts, and asserts a coverage floor of 90. Mutation-proven on both extraction halves against an unmodified working tree: dropping --scope from the card claim row fails the test naming 10 caller sites, dropping --yes from init fails naming 3 including packages/workfile/test/package-smoke.ts:215 — the exact command that failed in CI for T-0158 — and bin/workfile.ts restored clean both times. doctor 0/0. diff --git a/.project/memory/learnings/LRN-0031-a-contract-needs-checking-from-both-sides-and-the-check-has-to-sit-in-.md b/.project/memory/learnings/LRN-0031-a-contract-needs-checking-from-both-sides-and-the-check-has-to-sit-in-.md new file mode 100644 index 0000000..969784a --- /dev/null +++ b/.project/memory/learnings/LRN-0031-a-contract-needs-checking-from-both-sides-and-the-check-has-to-sit-in-.md @@ -0,0 +1,49 @@ +--- +id: LRN-0031 +title: A contract needs checking from both sides, and the check has to sit in the command the protocol names +status: active +category: infra +confidence: high +related: [T-0182, T-0158, T-0220, T-0148] +tags: [ci, testing] +created: 2026-08-07 +updated: 2026-08-07 +--- + +Recorded for T-0182. `COMMAND_FLAGS` is a contract with two sides, and only one +of them was checked. `cli.test.ts` pinned the table against the flags each +subcommand *reads*, in both directions, and it worked — it caught a stale `card +archive --actor` in the very branch that broke. The branch still failed in CI, +because nothing checked the flags a caller *sends*, and the callers were the +instructions, the docs and the package smoke test. + +**The generalisation.** When a table describes an interface, ask who else states +the same thing. A flag name appears in the reader, in the table, in the generated +agent instructions, in the README, in a smoke script — and every copy outside the +code is a place the compiler cannot reach. `AGENTS.md` teaching a flag that no +longer exists is worse than a broken test: an agent will run it confidently and +the repository will refuse. + +**The second half, which is the one that actually cost fourteen green commits.** +The test that would have caught it existed as a *category* but ran in the wrong +command. `smoke:package` belongs to `check:release`; `pnpm run check` is what +CLAUDE.md tells an agent to run before finishing. A gate is only as strong as the +command the protocol names — same class as CI running Windows while local runs +POSIX, and the same class as T-0148, where `pnpm audit` only runs on a tag push so +the failure lands on a release instead of on a pull request. + +**A trick worth reusing: do not scan what already executes.** The unit tests are +full of flags that are deliberately not real — `--bogus`, `--nonsense`, +`--statuss` — because asserting the refusal path is their job. Scanning them would +have meant an allowlist of intentional nonsense, and a real stale flag could then +hide inside it. They need no scanning at all: a test that sends a removed flag +fails when the suite runs. So the checker covers exactly the sources `pnpm run +test` does *not* execute — text, `scripts/`, and the smoke test — and says so. + +**And give a text-reading check a floor.** A checker built on regexes over prose +is one refactor away from matching nothing and passing forever. `cli-callers.test.ts` +asserts it found at least 90 invocations (116 today) so a broken extractor fails +loudly instead of going quiet. Prove such a check by mutation before trusting it: +removing `--scope` from the `card claim` row names 10 caller sites, and removing +`--yes` from `init` names 3 including `package-smoke.ts` — the exact file that +failed in CI. diff --git a/packages/workfile/test/cli-callers.test.ts b/packages/workfile/test/cli-callers.test.ts new file mode 100644 index 0000000..ee88104 --- /dev/null +++ b/packages/workfile/test/cli-callers.test.ts @@ -0,0 +1,269 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import { readFile, readdir } from "node:fs/promises"; +import { join, relative, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +/** + * Every flag anything *sends* to the CLI, against the table of what it reads. + * + * T-0182. `cli.test.ts` already pins the flag table against the flags each + * subcommand reads, in both directions, and it caught a stale `card archive + * --actor` in the branch that removed `init --language`. What it cannot see is + * the other side of the call: the flags a *caller* sends. So that branch stayed + * green locally through fourteen commits and failed in CI at the first command + * the package smoke test runs — `smoke:package` belongs to `check:release`, and + * `pnpm run check`, which is the command the protocol tells an agent to run + * before finishing, does not run it. + * + * The gap is not that a test was missing. It is that the sources which teach or + * send a flag are not executed by `pnpm run test`: + * + * - the generated agent instructions and the docs, which are text: a removed + * flag still taught by `AGENTS.md` or `SKILL.md` is a command an agent will + * confidently run and the repository will refuse; + * - `test/package-smoke.ts`, which only `check:release` runs; + * - `scripts/`, which nothing runs on a pull request. + * + * The unit tests are deliberately *not* scanned. A test that passes a removed + * flag already fails when the suite runs, which is a better signal than a text + * match — and those files contain flags that are meant not to exist + * (`--bogus`, `--nonsense`, `--statuss`), because asserting the refusal path is + * their job. Scanning them would mean teaching this an allowlist of deliberate + * nonsense, and then a real stale flag could hide in it. + * + * ## What this still does not cover + * + * Flag *values*: `--status doingg` and `--area nonexistent` pass here. Only the + * flag names are compared, because the table is the only machine-readable + * contract — the legal values live in the config and in the schema. + * + * Invocations assembled at runtime. `["card", action, ...flags]` reaches the CLI + * as a real call and appears here as nothing at all, because there is no literal + * to read. The coverage floor below is the guard against that going unnoticed + * wholesale; it cannot see one call becoming dynamic. + * + * Subcommand words, except by accident. An invocation whose command word + * resolves to no table row is reported, which is how `workfile docs create` + * would surface if `docs` ever stopped aliasing `doc` — but a word that resolves + * to the wrong row still checks its flags against that row. + * + * Anything outside the roots listed in `SOURCES`. A new directory of generated + * instructions is not covered until it is named here. + * + * And the whole of the other route the card weighed, which is the largest gap: + * this reads text, so it proves nothing about the artifact that ships. Whether + * the tarball carries the bin, whether the shebang survives, whether a consumer + * can resolve the package at all — none of that is here, and `pnpm run check` + * still does not answer it. That is `smoke:package` under `check:release`, and + * folding it into `check` costs about thirty seconds on every local run, which is + * a trade worth making deliberately rather than as a side effect of this. T-0220. + */ + +const packageRoot = resolve(fileURLToPath(new URL("..", import.meta.url))); +const repoRoot = resolve(packageRoot, "../.."); + +/** Text that teaches a command, and code that sends one. */ +const SOURCES = { + text: [ + ".project/agents", + ".claude/commands", + ".claude/skills", + "plugins/workfile/commands", + "packages/workfile/docs", + "README.md" + ], + code: ["packages/workfile/test/package-smoke.ts", "scripts", "packages/workfile/scripts"] +}; + +/** + * The lowest number of invocations this may find and still be believed. + * + * A checker that reads text is one refactor away from matching nothing and + * passing forever. The floor is well under the current count so ordinary edits + * do not trip it, and far above zero so a broken extractor fails loudly. + */ +const COVERAGE_FLOOR = 90; + +async function filesUnder(root: string, extension: string) { + const absolute = join(repoRoot, root); + const stack = [absolute]; + const found: string[] = []; + while (stack.length) { + const directory = stack.pop() as string; + let entries; + try { + entries = await readdir(directory, { withFileTypes: true }); + } catch { + // A single file was named rather than a directory. + if (absolute.endsWith(extension)) return [absolute]; + return []; + } + for (const entry of entries) { + if (entry.name === "node_modules" || entry.name === "dist") continue; + const path = join(directory, entry.name); + if (entry.isDirectory()) stack.push(path); + else if (path.endsWith(extension)) found.push(path); + } + } + return found.sort(); +} + +/** `{...}` object literals in the CLI source, read as `key: "value"` pairs. */ +function pairsOf(source: string, name: string) { + const start = source.indexOf(`const ${name}`); + const text = source.slice(start, source.indexOf("\n};", start)); + return Object.fromEntries( + [...text.matchAll(/(\w+):\s*"(\w+)"/g)].map((match) => [match[1], match[2]]) + ); +} + +test("every flag a caller sends is a flag the CLI reads", async () => { + const source = await readFile(join(packageRoot, "bin/workfile.ts"), "utf8"); + + // Read from the source rather than restated here, so a new row, alias or + // bare-command default cannot leave this checking against a stale copy. + const tableStart = source.indexOf("const COMMAND_FLAGS"); + const table = source.slice(tableStart, source.indexOf("\n};", tableStart)); + const globalStart = source.indexOf("const GLOBAL_FLAGS"); + const globals = new Set( + [ + ...source + .slice(globalStart, source.indexOf("];", globalStart)) + .matchAll(/"(--?[\w-]+)"/g) + ].map((match) => match[1]) + ); + const declared: Record> = {}; + for (const entry of table.matchAll(/"([\w ]+)": \[([^\]]*)\]/g)) { + declared[entry[1]] = new Set( + [...entry[2].matchAll(/"(--?[\w-]+)"/g)].map((match) => match[1]) + ); + } + assert.ok( + Object.keys(declared).length >= 40, + "the flag table was not parsed; the slice above has drifted from the source" + ); + + const aliases = pairsOf(source, "USAGE_ALIASES"); + const defaults = pairsOf(source, "DEFAULT_SUBCOMMAND"); + const words = new Set([ + ...Object.keys(declared).map((key) => key.split(" ")[0]), + ...Object.keys(aliases) + ]); + + // `commandKey` in the CLI, over literals instead of over argv. A word with + // no row is not silently accepted: it comes back null and is reported. + const keyFor = (spoken: string[]) => { + if (!spoken.length) return null; + const word = aliases[spoken[0]] || spoken[0]; + if (spoken[1] && declared[`${word} ${spoken[1]}`]) { + return `${word} ${spoken[1]}`; + } + const fallback = defaults[word]; + if (fallback && declared[`${word} ${fallback}`]) return `${word} ${fallback}`; + return declared[word] ? word : null; + }; + + const calls: Array<{ key: string; flags: string[]; where: string }> = []; + const unattributed: string[] = []; + + const record = ( + spoken: string[], + flags: string[], + where: string + ) => { + if (!flags.length) return; + // `workfile card --help` names no subcommand and needs none: a global + // flag is accepted by all of them. + if (flags.every((flag) => globals.has(flag))) return; + const key = keyFor(spoken); + if (!key) { + unattributed.push(`${where} ${spoken.join(" ") || "(no command word)"} ${flags.join(" ")}`); + return; + } + calls.push({ key, flags, where }); + }; + + // Prose: `workfile --flags`, one invocation per line. Stops at a + // backtick or a pipe so a sentence that quotes a flag after the command does + // not get read as part of it. + for (const root of SOURCES.text) { + for (const file of await filesUnder(root, ".md")) { + const body = await readFile(file, "utf8"); + body.split("\n").forEach((line, index) => { + for (const match of line.matchAll(/\bworkfile\s+([^\n`|]*)/g)) { + const rest = match[1]; + const spoken: string[] = []; + for (const token of rest.trim().split(/\s+/).filter(Boolean)) { + if (!/^[a-z][a-z-]*$/.test(token)) break; + spoken.push(token); + } + record( + spoken, + [...rest.matchAll(/(?:^|\s)(--[a-z][\w-]*)/g)].map( + (flag) => flag[1] + ), + `${relative(repoRoot, file)}:${index + 1}` + ); + } + }); + } + } + + // Code: argv arrays whose first literal is a command word. That test is what + // separates `run(project, ["card", "create", ...])` from `run(npm, ["pack", + // "--pack-destination", ...])` without having to know which binary each + // variable holds. + for (const root of SOURCES.code) { + for (const file of await filesUnder(root, ".ts")) { + const body = await readFile(file, "utf8"); + for (const match of body.matchAll(/\[([^[\]]*)\]/g)) { + const literals = [...match[1].matchAll(/"([^"\n]*)"/g)].map( + (literal) => literal[1] + ); + if (!literals.length || !words.has(literals[0])) continue; + const spoken: string[] = []; + for (const literal of literals) { + if (!/^[a-z][a-z-]*$/.test(literal)) break; + spoken.push(literal); + } + record( + spoken, + literals.filter((literal) => /^--[a-z][\w-]*$/.test(literal)), + `${relative(repoRoot, file)}:${ + body.slice(0, match.index).split("\n").length + }` + ); + } + } + } + + assert.ok( + calls.length >= COVERAGE_FLOOR, + `only ${calls.length} invocations found, expected at least ${COVERAGE_FLOOR}. ` + + "The extractor stopped matching rather than the callers stopping sending flags." + ); + + assert.deepEqual( + unattributed.sort(), + [], + "sent to a command word with no row in the flag table:\n " + + unattributed.join("\n ") + ); + + const stale = new Set(); + for (const call of calls) { + for (const flag of call.flags) { + if (globals.has(flag) || declared[call.key].has(flag)) continue; + stale.add(`${call.where} ${call.key} ${flag}`); + } + } + assert.deepEqual( + [...stale].sort(), + [], + `${stale.size} caller(s) send a flag the CLI does not read. Either the ` + + "flag was removed and the caller was not updated, or it is missing " + + "from COMMAND_FLAGS:\n " + + [...stale].sort().join("\n ") + ); +}); From cc8d73879bb3713463fb3d668aa5b4d50e7aa076 Mon Sep 17 00:00:00 2001 From: illodev Date: Fri, 7 Aug 2026 20:22:22 +0200 Subject: [PATCH 4/5] Supply-chain gates run on pull requests, and audit the tree a consumer resolves MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit T-0148 and T-0220 asked the same question about two gates living in `check:release`, which runs on a tag push — so a failure landed on a release rather than on a pull request. v0.5.4 is what that cost: a published tag whose release did nothing, `Publish to npm` never reached, and the tag moved onto the fix. Two things turned up while answering it. T-0220 was filed on a false premise, mine. `ci.yml` has had a `smoke` job on `pull_request` since before T-0182, so a change that breaks the packaged bin already fails on a pull request; the T-0158 failure that prompted this was that job working as designed. Its body is corrected rather than left standing. And the audit gate was measuring the wrong tree. `pnpm audit` audits this workspace, where four `pnpm.overrides` had already rewritten the graph. Overrides are a workspace-install mechanism and do not travel inside a published package. Two of the four — `sharp` and `adm-zip` — sit under `@huggingface/transformers`, a `dependencies` entry of the published `@illodev/workfile-search-local`, so they made this gate green and fixed those advisories for nobody who installed it. T-0148's own body reads that distinction as reassuring; for two of the four entries it is the opposite. So `pnpm audit --audit-level=high` is now its own blocking job on pull requests, and `scripts/audit-consumer.ts` resolves what the publishable manifests declare, in a scratch root with no overrides, and fails on anything at high or above. Blocking with no allowlist is a deliberate posture, not an omission: a baseline had precedent in `doctor --accept-baseline` and was refused, because the alternative is a list that grows and a package that ships with the list as its answer. Local `pnpm run check` gains neither gate. It runs constantly, both need the network, and posture is a property of what gets published rather than of an edit. That asymmetry is stated in ADR-0021 as a trade rather than hidden. T-0148 stays in `review`: its remaining criterion asks a pull request to prove the job fails in CI, and a local run is not that. Cards: T-0220 Decisions: ADR-0021 Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01D3LTdq3mzMAQ98rwBegGjU --- .github/workflows/ci.yml | 28 +++ ...-gate-fails-on-a-devdependency-the-pack.md | 22 ++- ...s-nothing-about-the-artifact-that-ships.md | 64 +++++++ ...l-requests-and-audits-the-tree-a-consum.md | 106 +++++++++++ package.json | 3 +- scripts/audit-consumer.ts | 180 ++++++++++++++++++ 6 files changed, 398 insertions(+), 5 deletions(-) create mode 100644 .project/cards/T-0220-check-proves-nothing-about-the-artifact-that-ships.md create mode 100644 .project/memory/decisions/ADR-0021-a-supply-chain-gate-runs-on-pull-requests-and-audits-the-tree-a-consum.md create mode 100644 scripts/audit-consumer.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e1d2bbd..9fc3ed9 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -46,6 +46,34 @@ jobs: # The hosted demo build must keep compiling too. - run: pnpm run build:demo + audit: + # `pnpm audit` ran only inside `check:release`, which runs on a tag push + # — so an advisory published by the ecosystem failed a *release* instead + # of a pull request. That is what it cost for v0.5.4: a tag whose + # release did nothing, `Publish to npm` never reached, and the tag moved + # onto the fix rather than the version being burned. The advisory did not + # change; only where it surfaced did. Here a re-run replaces a retag. + # + # Blocking, and at the same threshold as the release gate. A warning + # nobody has to act on is not a gate, and the repository already fixes + # these one way — a `pnpm.overrides` entry — which is a reviewable diff. + # Read what that fix does and does not do in ADR-0021 before adding one. + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - uses: pnpm/action-setup@v6 + - uses: actions/setup-node@v7 + with: + node-version: 22 + cache: pnpm + - run: pnpm install --frozen-lockfile + - run: pnpm audit --audit-level=high + # And the tree a consumer actually resolves, which the line above + # cannot see: root `pnpm.overrides` rewrite resolution in this + # workspace only, so an override silences the gate here and reaches + # no user. Blocking with no allowlist, deliberately — ADR-0021. + - run: pnpm run audit:consumer + smoke: # Packs the real tarball and installs it in a clean consumer. Only worth # running once: it is the slowest job and platform-independent. diff --git a/.project/cards/T-0148-the-release-gate-fails-on-a-devdependency-the-pack.md b/.project/cards/T-0148-the-release-gate-fails-on-a-devdependency-the-pack.md index c935c1d..83fa1df 100644 --- a/.project/cards/T-0148-the-release-gate-fails-on-a-devdependency-the-pack.md +++ b/.project/cards/T-0148-the-release-gate-fails-on-a-devdependency-the-pack.md @@ -1,13 +1,15 @@ --- id: T-0148 title: The release gate fails on a devDependency the package never ships -status: backlog +status: review type: bug priority: medium area: infra tags: [release, audit, dependencies] created: 2026-08-03 -updated: 2026-08-03 +updated: 2026-08-07 +scope: [package.json, .github/workflows, scripts/audit-consumer.ts] +related: [ADR-0021, T-0221, T-0220] --- The `Release` workflow for `v0.5.4` failed at `pnpm run check:release`. Build, @@ -49,7 +51,19 @@ because the gate only runs on a tag push. Two options, neither obviously right: ## Acceptance criteria -- [ ] `pnpm audit --audit-level=high` passes on a clean install -- [ ] The decision above is recorded, whichever way it goes +- [x] `pnpm audit --audit-level=high` passes on a clean install +- [x] The decision above is recorded, whichever way it goes - [ ] If the audit moves into ordinary CI, a pull request proves it fails there first + +## Activity + +- 2026-08-07 17:29Z illodev@local#42eb42f5 · claimed +- 2026-08-07 17:39Z illodev@local#42eb42f5 · released + +## Notes + +- 2026-08-07 17:39Z illodev@local#42eb42f5 — Decided jointly with T-0220 as one policy, recorded in ADR-0021: gates run on pull requests, and local pnpm run check gains neither because it runs constantly and both need the network. pnpm audit --audit-level=high is now its own blocking job in ci.yml at the release gate's threshold. Blocking rather than warning, because check:release would still block at tag time, so a non-blocking pull-request job would be noise people learn to ignore. +Criterion 1 was already met before this card was worked: the four pnpm.overrides make pnpm audit --audit-level=high exit 0 on a clean install, with one moderate remaining (hono via shadcn, below the gate). +And the finding that matters more than the timing question. This card's body reads the workspace/tarball distinction as reassuring — "nothing vulnerable was ever going to reach a user" — and for two of the four overrides it is the opposite. sharp and adm-zip both sit under @huggingface/transformers, which is a dependencies entry of the published @illodev/workfile-search-local. Overrides are a workspace-install mechanism and do not travel inside a published package, so those two entries made this gate green and fixed the advisories for nobody who installed it. Measured rather than reasoned: resolving what the publishable manifests declare, with no overrides, reports four packages at high — sharp <0.35.0 with the libvips CVEs, adm-zip <0.6.0, and onnxruntime-node and transformers through them — all with no fix available, and transformers 4.2.0 is the latest release and pins sharp ^0.34.5. So a third gate exists now that neither card asked for: scripts/audit-consumer.ts audits the tree a consumer resolves, blocking, with no allowlist. That no-allowlist posture is the maintainer's explicit call over a doctor-style baseline, and its consequence is deliberate — the gate is red today and stays red until the published tree changes. Tracked as T-0221 with the routes laid out, because choosing between them is a product decision. +Criterion 3 is unchecked on purpose: the job is wired and the command fails locally with the exit code CI will see, but nothing has run in CI, and this repository does not treat a local run as proof of a pull request. It needs one push. diff --git a/.project/cards/T-0220-check-proves-nothing-about-the-artifact-that-ships.md b/.project/cards/T-0220-check-proves-nothing-about-the-artifact-that-ships.md new file mode 100644 index 0000000..28bc3fa --- /dev/null +++ b/.project/cards/T-0220-check-proves-nothing-about-the-artifact-that-ships.md @@ -0,0 +1,64 @@ +--- +id: T-0220 +title: check proves nothing about the artifact that ships +status: done +type: task +priority: medium +area: infra +tags: [ci] +effort: S +scope: [package.json, .github/workflows, scripts/audit-consumer.ts] +origin: [T-0182] +created: 2026-08-07 +updated: 2026-08-07 +related: [ADR-0021, T-0221, T-0148, T-0182] +verified: + at: "2026-08-07T17:40:10.534Z" + method: local + commit: 4e8da0782fecb7e52899f7916be21ad7f3d4c775 + digest: "sha256:c48bf3eb411de20438913d97539285597e56d947adaa787979c840c49e1463f9" +--- + +The half of T-0182 that was deliberately not taken. + +**Filed on a premise that is wrong, and the correction is the useful part.** This +card said `smoke:package` runs only under `check:release` on a tag push. It does +not: `ci.yml` has had a `smoke` job on `pull_request` since before T-0182, so a +change that breaks the packaged bin already fails on a pull request. The T-0158 +failure this came from was a CI failure, exactly as intended — what was missing +was local, not in CI. + +So what is actually left is a smaller question. `pnpm run check` is the command +CLAUDE.md tells an agent to run before finishing, and it does not pack the +tarball. Folding `smoke:package` in would cost roughly thirty seconds on every +local run, on the command run most often, to duplicate a job CI already runs on +every pull request. ADR-0021 records the decision not to, and why. + +**What working this card actually turned up** is a gate measuring the wrong tree, +which is the same class of problem one level deeper. `pnpm audit` reads the +workspace, where root `pnpm.overrides` have rewritten the dependency graph. +Overrides are a workspace-install mechanism and do not travel to consumers, so +the gate's green says nothing about what somebody who installs a published +package resolves. Two of the four overrides — `sharp` and `adm-zip` — sit under +`@huggingface/transformers`, which is a `dependencies` entry of the published +`@illodev/workfile-search-local`. Recorded in ADR-0021 and tracked as its own +card, because the exposure is real and has no upstream fix. + +## Acceptance criteria + +- [x] A change that breaks the packaged bin fails before a tag is pushed, not after. +- [x] The decision is recorded, including what it costs the command agents run most. +- [x] Whatever runs it names what it still does not cover. + +## Activity + +- 2026-08-07 17:29Z illodev@local#42eb42f5 · claimed +- 2026-08-07 17:40Z illodev@local#42eb42f5 · released + +## Notes + +- 2026-08-07 17:40Z illodev@local#42eb42f5 — Closed against a corrected premise, and the correction is most of the value. This card claimed smoke:package runs only under check:release on a tag push. It does not — ci.yml has had a smoke job on pull_request since before T-0182, so criterion 1 was already satisfied by machinery that has already fired: the T-0158 failure this whole thread came from was that job working as designed. The body has been rewritten rather than left standing, because a wrong record is worse than no record in a repository whose point is durable records. +What was actually left was the smaller question, and ADR-0021 answers it: pnpm run check does not gain smoke:package. Thirty seconds on every local run of the command CLAUDE.md tells an agent to run before finishing, to duplicate a job CI already runs on every pull request, is not worth it. The asymmetry is stated as a trade rather than hidden: an agent can close a card while the repository is red on a gate it never ran locally. +Criterion 3 is met by naming the gap in the places a reader will be: cli-callers.test.ts already says it reads text and proves nothing about the artifact, and scripts/audit-consumer.ts says it reads manifests rather than the tarball and resolves only dependencies, so an optionalDependencies path npm skipped is invisible to it. smoke:package stays the only gate that touches the real artifact. +The card also turned up the same class of problem one level deeper, which is now T-0221: pnpm audit was measuring the workspace, where overrides had rewritten the graph, so its green said nothing about what a consumer installs. That is in ADR-0021 with T-0148. +- 2026-08-07 17:40Z illodev@local#42eb42f5 — local verification: pnpm run check green: 465+7 tests pass, strictNullChecks held at 488. Criterion 1 rests on the pre-existing smoke job in ci.yml on pull_request, which has already demonstrated the behaviour by failing on the T-0158 branch. ci.yml parses and now declares four jobs (check, audit, smoke, codeql) with the audit job running the workspace audit and pnpm run audit:consumer. audit:consumer exits 1 today naming four high advisories with their GHSA/CVE ids and the dependency path that reaches each, which is the intended blocking behaviour recorded in ADR-0021. doctor 0/0, memory verify 0/0. diff --git a/.project/memory/decisions/ADR-0021-a-supply-chain-gate-runs-on-pull-requests-and-audits-the-tree-a-consum.md b/.project/memory/decisions/ADR-0021-a-supply-chain-gate-runs-on-pull-requests-and-audits-the-tree-a-consum.md new file mode 100644 index 0000000..ea6dce9 --- /dev/null +++ b/.project/memory/decisions/ADR-0021-a-supply-chain-gate-runs-on-pull-requests-and-audits-the-tree-a-consum.md @@ -0,0 +1,106 @@ +--- +id: ADR-0021 +title: A supply-chain gate runs on pull requests and audits the tree a consumer resolves, blocking with no allowlist +status: accepted +related: [T-0148, T-0220, T-0182, T-0221] +tags: [ci, security] +created: 2026-08-07 +updated: 2026-08-07 +--- + +## Context + +T-0148 and T-0220 asked the same question about two gates: `pnpm audit` and +`smoke:package` lived in `check:release`, which runs on a tag push, so a failure +landed on a release rather than on a pull request. v0.5.4 is what that cost — a +published tag whose release did nothing, `Publish to npm` never reached, and the +tag moved onto the fix. + +Two things turned up while answering it, and both matter more than the timing. + +**T-0220 was filed on a false premise.** `ci.yml` has had a `smoke` job on +`pull_request` since before T-0182, so a change that breaks the packaged bin +already fails on a pull request. The T-0158 failure that prompted all of this was +a CI failure working exactly as designed. What was missing was local, not in CI. +The card's body has been corrected rather than left standing. + +**The audit gate was measuring the wrong tree.** `pnpm audit` audits this +workspace, and this workspace has four `pnpm.overrides`. Overrides are a +workspace-install mechanism: they rewrite resolution here and do not travel +inside a published package. Two of the four — `sharp` and `adm-zip`, added for +the libvips CVEs and GHSA-xcpc-8h2w-3j85 — sit under `@huggingface/transformers`, +which is a `dependencies` entry of the published +`@illodev/workfile-search-local`. So the overrides made the gate green and fixed +those advisories for nobody but us. T-0148's own body reads the workspace/tarball +distinction as reassuring — "nothing vulnerable was ever going to reach a +user" — and for two of the four entries it is the opposite. + +Measured, not inferred: resolving what the publishable manifests declare, with no +overrides, reports four packages at high — `sharp <0.35.0`, `adm-zip <0.6.0`, and +`onnxruntime-node` and `@huggingface/transformers` through them — every one with +no fix available upstream. `@huggingface/transformers@4.2.0` is the latest release +and pins `sharp: ^0.34.5`. + +## Decision + +One policy for both gates, and a third gate that neither card asked for. + +**Gates run on pull requests.** `pnpm audit --audit-level=high` is now its own job +in `ci.yml`, blocking, at the same threshold as the release gate. An advisory the +ecosystem publishes will turn an unrelated pull request red through no fault of +its author; that is the cost, and it buys a re-run instead of a retag. A +non-blocking version was rejected: a warning nobody must act on is not a gate, and +`check:release` would still block at tag time, so the pull-request job would be +pure noise. + +**Local `pnpm run check` gains neither gate.** It is the command CLAUDE.md tells +an agent to run before finishing, it runs constantly, and both additions are slow +and need the network — `smoke:package` costs about thirty seconds to duplicate a +job CI already runs on every pull request, and the consumer audit resolves against +the registry. Supply-chain posture is a property of what gets published, not of an +edit, so it is gated where publishing is decided. This is the one place the policy +is deliberately asymmetric, and it is a trade rather than an oversight: an agent +can close a card while the repository is knowingly red on a gate it never ran. + +**The consumer tree is audited, blocking, with no allowlist.** +`scripts/audit-consumer.ts` builds a manifest from the union of every publishable +package's `dependencies`, resolves it in a scratch root with +`npm install --package-lock-only --ignore-scripts` — seconds, and no platform +binaries — and fails on anything at high or above. It runs in the pull-request +audit job and in `check:release`. + +No allowlist, and that is an explicit posture rather than an omission. A baseline +of accepted advisories was the alternative, and it has real precedent here in +`doctor --accept-baseline`. It was rejected for this gate: a known advisory in a +published dependency tree should stop the release, even when the fix is not ours +to make, because the alternative is a list that grows and a package that ships +with the list as its answer. The way out is to change what is shipped, not to +annotate what is. + +## Consequences + +**The gate was red on the day it was written, and that was the intended +behaviour.** `main` and every pull request failed the consumer audit while the +four advisories were in the published tree, and no annotation would have cleared +it. That is the whole argument for refusing an allowlist, and it held: T-0221 +resolved it by changing what is shipped. `@illodev/workfile-search-local` now +reaches the same ONNX weights through `onnxruntime-web` and +`@huggingface/tokenizers`, `@huggingface/transformers` is gone, and both trees +audit clean with nothing annotated. The `sharp` and `adm-zip` overrides were +removed with it — they had become entries pinning packages no longer in the graph. + +Worth keeping in view: a baseline would have made this a line on a list, and the +list would have been the answer for as long as nobody looked at it. + +An override is now understood as two different things depending on where the +overridden package sits. Under a devDependency it is a real fix for the only tree +that matters — nothing ships. Under a published package's `dependencies` it is a +local silence, and reaching for one there is the mistake this decision exists to +stop. `pnpm why -r` is what tells the two apart, and the script's own +comment says so. + +The consumer audit reads manifests, not the tarball, and resolves only +`dependencies`. A vulnerability reachable solely through an +`optionalDependencies` path npm skipped, or a dependency a build step adds to the +published manifest, is invisible to it. `smoke:package` remains the only gate that +touches the real artifact. diff --git a/package.json b/package.json index 7382a2f..fee0e84 100644 --- a/package.json +++ b/package.json @@ -30,7 +30,8 @@ "test": "pnpm --filter @illodev/workfile run test && pnpm --filter @illodev/workfile-search-local run test", "check": "pnpm run build && pnpm run build:plugin && pnpm run strict && pnpm run test", "smoke:package": "pnpm --filter @illodev/workfile run smoke:package", - "check:release": "pnpm run check && pnpm audit --audit-level=high && pnpm run smoke:package", + "audit:consumer": "node ./scripts/audit-consumer.ts", + "check:release": "pnpm run check && pnpm audit --audit-level=high && pnpm run audit:consumer && pnpm run smoke:package", "bench": "pnpm --filter @illodev/workfile run bench", "screenshots": "pnpm run build && node ./scripts/screenshots.ts", "doctor": "node packages/workfile/dist/bin/workfile.js doctor", diff --git a/scripts/audit-consumer.ts b/scripts/audit-consumer.ts new file mode 100644 index 0000000..0da2272 --- /dev/null +++ b/scripts/audit-consumer.ts @@ -0,0 +1,180 @@ +#!/usr/bin/env node +/** + * Audits the tree a consumer resolves, which is not the tree `pnpm audit` reads. + * + * `pnpm audit` audits this workspace. The workspace has `pnpm.overrides`, and + * overrides are a workspace-install mechanism: they rewrite resolution here and + * do not travel inside a published package. So the release gate could read zero + * high advisories while somebody running `npm i @illodev/workfile-search-local` + * resolved several — which is exactly what it did. Two of the four overrides, + * `sharp` and `adm-zip`, sit under `@huggingface/transformers`, a `dependencies` + * entry of that published package, and the overrides fixed them for nobody but + * us. + * + * This resolves what the *manifests* declare instead. The consumer manifest is + * the union of every publishable package's `dependencies`, so it describes this + * branch rather than the last release, needs nothing published, and cannot see + * the overrides because it is a different install root. + * + * `--package-lock-only` resolves without downloading, so this costs a few + * seconds and no binaries — `sharp` and `onnxruntime-node` would otherwise pull + * platform builds worth hundreds of megabytes to tell us something the lockfile + * already knows. + * + * Blocking, with no allowlist. That is a deliberate posture, recorded in + * ADR-0021: a known advisory in a published dependency tree stops the release + * rather than being carried on a list, even when the fix is not ours to make. + * The way out is to change what is shipped, not to annotate what is. + * + * What this does not cover: only `dependencies` are resolved, because that is + * what a consumer installs — a vulnerability reachable solely through an + * `optionalDependencies` path that npm skipped here is invisible. It also reads + * the manifests, not the tarball, so a dependency added to the published + * package.json by a build step would not appear. + */ +import { execFile } from "node:child_process"; +import { mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { promisify } from "node:util"; + +const run = promisify(execFile); +const repoRoot = fileURLToPath(new URL("..", import.meta.url)); +const THRESHOLD = new Set(["high", "critical"]); + +async function publishableDependencies() { + const packagesDir = join(repoRoot, "packages"); + const dependencies: Record = {}; + const sources: Record = {}; + for (const entry of await readdir(packagesDir, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const manifestPath = join(packagesDir, entry.name, "package.json"); + let manifest; + try { + manifest = JSON.parse(await readFile(manifestPath, "utf8")); + } catch { + continue; + } + // A private package ships nothing, so its dependencies reach no consumer. + if (manifest.private) continue; + for (const [name, range] of Object.entries( + (manifest.dependencies || {}) as Record + )) { + // Two publishable packages asking for different ranges of one + // dependency is a thing npm would resolve per-consumer; noted rather + // than merged silently, because the audit result could differ. + if (dependencies[name] && dependencies[name] !== range) { + console.warn( + `warning: ${name} is declared as ${dependencies[name]} and ${range}; auditing ${range}` + ); + } + dependencies[name] = range; + sources[name] = [...(sources[name] || []), manifest.name]; + } + } + return { dependencies, sources }; +} + +const { dependencies, sources } = await publishableDependencies(); +const names = Object.keys(dependencies); +if (!names.length) { + console.error( + "No publishable package declares a runtime dependency. Either every " + + "package is private, or the manifests moved and this is auditing nothing." + ); + process.exit(1); +} + +console.log( + `Auditing the consumer tree for ${names.length} declared ${ + names.length === 1 ? "dependency" : "dependencies" + }:` +); +for (const name of names) { + console.log(` ${name}@${dependencies[name]} (from ${sources[name].join(", ")})`); +} + +const scratch = await mkdtemp(join(tmpdir(), "workfile-consumer-audit-")); +try { + await writeFile( + join(scratch, "package.json"), + `${JSON.stringify( + { + name: "workfile-consumer-audit", + version: "0.0.0", + private: true, + dependencies + }, + null, + 2 + )}\n` + ); + + // Resolution only. `--ignore-scripts` because nothing here is executed, and + // a postinstall from a tree we are auditing is the last thing to run. + await run("npm", ["install", "--package-lock-only", "--ignore-scripts"], { + cwd: scratch, + maxBuffer: 32 * 1024 * 1024 + }); + + let report; + try { + const { stdout } = await run("npm", ["audit", "--json"], { + cwd: scratch, + maxBuffer: 64 * 1024 * 1024 + }); + report = JSON.parse(stdout); + } catch (error) { + // `npm audit` exits non-zero whenever it finds anything at all, so the + // findings arrive on this path in the ordinary case. + const output = `${(error as { stdout?: string }).stdout || ""}`; + if (!output.trim()) throw error; + report = JSON.parse(output); + } + + const blocking = Object.values( + (report.vulnerabilities || {}) as Record + ).filter((entry) => THRESHOLD.has(entry.severity)); + + if (!blocking.length) { + const counts = report.metadata?.vulnerabilities || {}; + console.log( + `\nConsumer tree clean at high and above. Below the threshold: ${ + Object.entries(counts) + .filter(([level, count]) => level !== "total" && Number(count) > 0) + .map(([level, count]) => `${count} ${level}`) + .join(", ") || "nothing" + }.` + ); + process.exit(0); + } + + console.error( + `\n${blocking.length} package(s) at high or above in the tree a consumer resolves:\n` + ); + for (const entry of blocking.sort((left, right) => + String(left.name).localeCompare(String(right.name)) + )) { + console.error(` ${entry.name} ${entry.range} — ${entry.severity}`); + for (const via of entry.via || []) { + if (typeof via === "string") continue; + console.error(` ${via.title}`); + if (via.url) console.error(` ${via.url}`); + } + const reachedBy = (entry.effects || []).join(", "); + if (reachedBy) console.error(` reached through: ${reachedBy}`); + console.error( + ` fix available: ${entry.fixAvailable ? "yes" : "no, not upstream"}` + ); + } + console.error( + "\nThis is what a consumer installs. `pnpm audit` does not see it: root\n" + + "pnpm.overrides rewrite resolution in this workspace only, so an override\n" + + "silences the workspace gate and reaches no user. Fixing this means\n" + + "changing what the published packages depend on — see ADR-0021.\n" + ); + process.exit(1); +} finally { + await rm(scratch, { recursive: true, force: true }); +} From 10c1e387adaa1f3e7e776311e6e6a2f891fdfca3 Mon Sep 17 00:00:00 2001 From: illodev Date: Fri, 7 Aug 2026 20:23:00 +0200 Subject: [PATCH 5/5] Drop transformers.js from search-local, clearing four advisories it shipped MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `@illodev/workfile-search-local` declared `@huggingface/transformers`, which hard-depends on `sharp` and `onnxruntime-node` — image processing and an archive extractor this package never touched. Both carry high-severity advisories with no upstream fix: the libvips CVEs in `sharp <0.35.0` (GHSA-f88m-g3jw-g9cj) and GHSA-xcpc-8h2w-3j85 in `adm-zip <0.6.0`. Anybody installing 0.8.1 or earlier got all of them, while this repository's audit read clean because overrides do not travel. The pipeline was doing four things: fetch `tokenizer.json` and the ONNX weights, tokenize, run the session, mean-pool over the attention mask and L2-normalize. Done directly over `onnxruntime-web` and `@huggingface/tokenizers` that is about 130 lines, and it audits clean with no override and no allowlist. Measured against the implementation it replaces before committing to the route, on the same q8 weights and the same texts: per-vector cosine 0.9978, unit norms, identical ranking order. The residual is the WASM and native kernels disagreeing at quantized precision, not a difference in method. The `sharp` and `adm-zip` overrides go with it — `pnpm why` finds neither package in the graph now, so those entries had become pins on nothing. `fast-uri` and `js-yaml` stay, and they are the honest kind: nothing ships. One cost, in CHG-0149: the model cache moved to `~/.cache/workfile/models`, so the first search after upgrading re-downloads about 135 MB once. Existing embedding caches stay valid, being keyed by model and content. In exchange, pointing `model` at a directory now skips the download entirely, so this runs with no network — which it could not before. Two corrections to what the card was filed with. `optionalDependencies` would not have worked: npm installs them by default. The mechanism that removes a dependency from a consumer's tree is `peerDependenciesMeta` `.optional`, verified empirically both ways. And transformers 4.2.0 declares all three heavy dependencies with no flag that omits them, so no configuration could have avoided this. All seven pre-existing tests inject `embedder` and touch none of the new code; three were added for the parts that need no model. The pooling one was mutation-checked, and it taught something worth keeping: after L2 normalization the divisor cannot matter, because dividing by the padded width is a positive scalar that normalization cancels. The first version of that test claimed otherwise in a comment, the mutation passed, and the comment was what was wrong. It pins the mask now. LRN-0032. Cards: T-0221 Changelog: CHG-0149 Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01D3LTdq3mzMAQ98rwBegGjU --- ...l-ships-four-high-advisories-with-no-up.md | 82 +++ ...e-web-clearing-four-high-advisories-it-.md | 42 ++ ...sformers-js-and-what-a-normalized-embed.md | 51 ++ README.md | 2 +- package.json | 2 - packages/search-local/README.md | 15 +- packages/search-local/index.d.ts | 33 +- packages/search-local/index.js | 218 +++++++- packages/search-local/package.json | 3 +- packages/search-local/test/provider.test.ts | 66 ++- packages/workfile/docs/cli.md | 2 +- pnpm-lock.yaml | 510 +----------------- 12 files changed, 495 insertions(+), 531 deletions(-) create mode 100644 .project/cards/T-0221-search-local-ships-four-high-advisories-with-no-up.md create mode 100644 .project/changelog/unreleased/CHG-0149-search-local-runs-on-onnxruntime-web-clearing-four-high-advisories-it-.md create mode 100644 .project/memory/learnings/LRN-0032-feature-extraction-without-transformers-js-and-what-a-normalized-embed.md diff --git a/.project/cards/T-0221-search-local-ships-four-high-advisories-with-no-up.md b/.project/cards/T-0221-search-local-ships-four-high-advisories-with-no-up.md new file mode 100644 index 0000000..36a4247 --- /dev/null +++ b/.project/cards/T-0221-search-local-ships-four-high-advisories-with-no-up.md @@ -0,0 +1,82 @@ +--- +id: T-0221 +title: search-local ships four high advisories with no upstream fix +status: done +type: bug +priority: high +area: infra +tags: [security] +effort: M +scope: [packages/search-local] +origin: [T-0148, ADR-0021] +created: 2026-08-07 +updated: 2026-08-07 +related: [ADR-0021, LRN-0032, CHG-0149] +verified: + at: "2026-08-07T18:06:34.832Z" + method: local + commit: 4e8da0782fecb7e52899f7916be21ad7f3d4c775 + digest: "sha256:f8eb3b5d5eab96282b5f745d48b22b0dcf098fe8db5406588b5148a484b3644b" +--- + +Found while deciding the gate policy in T-0148, and the reason the consumer audit +ADR-0021 added is red. + +`@illodev/workfile-search-local` declares `@huggingface/transformers: ^4.2.0` in +`dependencies`. Resolving that as a consumer does — no workspace, no overrides — +reports four packages at high, every one with no fix available: + +- `sharp <0.35.0` — inherited libvips CVEs, GHSA-f88m-g3jw-g9cj + (CVE-2026-33327, CVE-2026-33328, CVE-2026-35590, CVE-2026-35591), reached + through `@huggingface/transformers`, which pins `sharp: ^0.34.5`. +- `adm-zip <0.6.0` — GHSA-xcpc-8h2w-3j85, reached through `onnxruntime-node`, + which pins `adm-zip: ^0.5.16`. +- `onnxruntime-node` and `@huggingface/transformers` themselves, for depending on + those. + +The repository has carried `pnpm.overrides` for `sharp` and `adm-zip` since T-0023 +and T-0033, and the workspace audit has read clean ever since. Overrides do not +travel inside a published package, so they fixed this for this checkout and for +nobody who installed the package. `@huggingface/transformers@4.2.0` is the latest +release, so there is nothing to bump to. + +Reproduce with `pnpm run audit:consumer`. + +What makes this worth a card rather than a patch is that every route is a trade, +and the choice is not the gate's to make: + +- **Drop the dependency.** `sharp` is image processing and `onnxruntime-node` + carries `adm-zip` for archive extraction; this package does text embeddings + only. If transformers can be replaced by something narrower — a text-only ONNX + path, or `onnxruntime-web` — both advisories leave with it. Most work, best + outcome. +- **Make the heavy dependency optional.** Move it to `optionalDependencies` or a + peer the user installs deliberately, so the advisory is something a consumer + opts into rather than something the package hands them. Cheap, and honest only + if the package still does something useful without it. +- **Unpublish or park the package.** It is an opt-in extra; the core does lexical + search without it and the README already teaches a guarded import. Blunt, and it + reaches every existing consumer. +- **Accept and document.** Would need the allowlist ADR-0021 deliberately refused, + so it means revisiting that decision rather than working around it. + +## Acceptance criteria + +- [x] `pnpm run audit:consumer` passes with no allowlist and no override. +- [x] The route taken is recorded, including what a consumer of the current version should do. +- [x] Semantic search still works, or the card records that the capability was withdrawn and why. + +## Activity + +- 2026-08-07 17:50Z illodev@local#42eb42f5 · claimed +- 2026-08-07 18:06Z illodev@local#42eb42f5 · released + +## Notes + +- 2026-08-07 18:06Z illodev@local#42eb42f5 — Took route B: replaced @huggingface/transformers with onnxruntime-web and @huggingface/tokenizers, so nothing is asked of the consumer and the advisories leave rather than becoming opt-in. +What the pipeline was actually doing turned out to be four things: fetch tokenizer.json and the ONNX weights, tokenize, run the session, mean-pool over the attention mask and L2-normalize. About 130 lines to do it directly. Verified against the implementation it replaces on the same q8 weights and the same texts before committing to the route: per-vector cosine 0.9978, unit norms, identical ranking order. The residual is the WASM and native kernels disagreeing at quantized precision, not a difference in method. +Two corrections to what this card was filed with. optionalDependencies would not have worked at all — npm installs them by default, so the exposure would have stayed; the mechanism that removes a dependency from a consumer's tree is peerDependenciesMeta.optional, verified empirically both ways. And transformers 4.2.0 declares onnxruntime-node, onnxruntime-web and sharp in dependencies with no flag that omits them, so no configuration could have avoided this. +The sharp and adm-zip overrides were removed with the dependency: pnpm why finds neither package in the graph now, so those entries had become pins on packages that are not there. fast-uri and js-yaml stay, and they are the honest kind — devDependency-only, nothing shipped. +One cost that is real and is in CHG-0149: the model cache moved from the transformers.js cache to ~/.cache/workfile/models, so the first search after upgrading re-downloads about 135 MB once. Existing embedding caches stay valid, because they are keyed by model and content and the model is unchanged. The new modelDir option relocates it, and pointing model at a directory holding tokenizer.json and the ONNX file skips the download entirely — this now runs with no network at all, which it could not before. +The tests deserve a note. All seven pre-existing tests inject embedder, so none of them touched the new code; three were added for the parts that need no model, and the pooling one was mutation-checked. That check taught something worth keeping: after L2 normalization the pooling divisor cannot matter, because dividing by the padded width is a positive scalar and normalization cancels it. My first version of that test claimed the opposite in a comment, the mutation passed, and the comment was the thing that was wrong. What the test now pins is the mask — summing padded positions moves the direction about 45 degrees on its own data, and removing the guard fails both new tests. LRN-0032. +- 2026-08-07 18:06Z illodev@local#42eb42f5 — local verification: pnpm run audit:consumer: consumer tree clean at high and above, nothing below the threshold either, with no allowlist and the sharp/adm-zip overrides deleted. pnpm audit --audit-level=high exits 0. pnpm why sharp and pnpm why adm-zip both find nothing in the graph. pnpm run check green: 465 + 10 tests pass (3 new in search-local), strictNullChecks held at 488. pnpm run smoke:package passes against the packed tarball. Real model path driven end to end through the published entry point against Xenova/multilingual-e5-small q8: cold run 45.8s including the 135MB download, warm run 2.2-3.1s, ranking correct and byte-identical across runs. Faithfulness measured against the transformers.js implementation on the same texts: per-vector cosine 0.9978 on all three vectors, unit norms both sides, same ranking order. Pooling test mutation-checked: removing the attention-mask guard fails both new tests, restoring it returns 10/10. The workfile guarded-import test still passes. diff --git a/.project/changelog/unreleased/CHG-0149-search-local-runs-on-onnxruntime-web-clearing-four-high-advisories-it-.md b/.project/changelog/unreleased/CHG-0149-search-local-runs-on-onnxruntime-web-clearing-four-high-advisories-it-.md new file mode 100644 index 0000000..5bdab60 --- /dev/null +++ b/.project/changelog/unreleased/CHG-0149-search-local-runs-on-onnxruntime-web-clearing-four-high-advisories-it-.md @@ -0,0 +1,42 @@ +--- +id: CHG-0149 +title: search-local runs on onnxruntime-web, clearing four high advisories it used to ship +type: security +area: search +visibility: public +cards: [T-0221] +decisions: [ADR-0021] +tags: [security, dependencies] +created: 2026-08-07 +updated: 2026-08-07 +--- + +`@illodev/workfile-search-local` no longer depends on +`@huggingface/transformers`. It now reaches the same ONNX weights through +`onnxruntime-web` and `@huggingface/tokenizers`. + +**Why this matters if you installed it.** `@huggingface/transformers` depends on +`sharp` and `onnxruntime-node` — image processing and an archive extractor this +package never used — and both carry high-severity advisories with no upstream +fix: the libvips CVEs in `sharp <0.35.0` (GHSA-f88m-g3jw-g9cj) and +GHSA-xcpc-8h2w-3j85 in `adm-zip <0.6.0`, which arrives under +`onnxruntime-node`. Installing 0.8.1 or earlier put all of them in your tree. +This repository's audit had read clean the whole time, because `pnpm.overrides` +rewrote resolution here and overrides do not travel inside a published package. +Auditing the tree a consumer resolves is now its own gate. + +**What you should do.** Upgrade, then reinstall so the old transitive tree is +dropped. Nothing in the public API changed. + +**One cost, and it is not free.** The model cache moved from the transformers.js +cache to `~/.cache/workfile/models`, so the first search after upgrading +re-downloads the model — about 135 MB of tokenizer and weights, once. The new +`modelDir` option relocates it, and pointing `model` at a directory that holds +`tokenizer.json` and the ONNX file skips the download entirely, which is how this +now runs with no network at all. + +Embeddings are computed through the WASM execution provider rather than the +native binding. Verified against the implementation it replaces on the same q8 +weights: per-vector cosine 0.9978, unit norms, identical ranking order. Existing +embedding caches stay valid — they are keyed by model and content, and the model +is unchanged. diff --git a/.project/memory/learnings/LRN-0032-feature-extraction-without-transformers-js-and-what-a-normalized-embed.md b/.project/memory/learnings/LRN-0032-feature-extraction-without-transformers-js-and-what-a-normalized-embed.md new file mode 100644 index 0000000..4657272 --- /dev/null +++ b/.project/memory/learnings/LRN-0032-feature-extraction-without-transformers-js-and-what-a-normalized-embed.md @@ -0,0 +1,51 @@ +--- +id: LRN-0032 +title: Feature extraction without transformers.js, and what a normalized embedding does not care about +status: active +category: infra +confidence: high +related: [T-0221, ADR-0021] +tags: [search, embeddings] +created: 2026-08-07 +updated: 2026-08-07 +--- + +Recorded for T-0221, which replaced one `pipeline("feature-extraction")` call in +`packages/search-local/index.js` because `@huggingface/transformers` pulls `sharp` +and `onnxruntime-node` as hard dependencies and shipped their advisories to every +consumer. + +**The pipeline was doing four things, and only four.** Fetch `tokenizer.json` and +the ONNX weights; tokenize; run the session; mean-pool over the attention mask and +L2-normalize. Reimplemented over `onnxruntime-web` plus +`@huggingface/tokenizers`, that is about 130 lines, and it audits clean with no +overrides. Measured against what it replaced, on the same q8 weights and the same +texts: per-vector cosine 0.9978, unit norms, identical ranking order. The residual +is the WASM and native kernels disagreeing at quantized precision, not a +difference in method. Session creation is about 800 ms and a warm search over a +handful of records is ~2–3 s. + +**Details that are not obvious from the transformers.js API.** +`onnxruntime-web` runs in Node — the name is about the execution provider, not the +host. Threads are `ort.env.wasm.numThreads`, a global on the environment; +`intraOpNumThreads` in the session options is the native knob and does nothing +here. The tokenizer's encoding field is `ids`, not `input_ids`. `Xenova`-style +repositories spell q8 weights `onnx/model_quantized.onnx` while every other dtype +is `model_.onnx`. And the exported graph declares `token_type_ids` even for +XLM-R-derived models that never use it, so feed zeros when `session.inputNames` +asks for it and skip it when it does not. + +**The part worth remembering, because a test asserted the opposite.** After L2 +normalization the pooling *divisor* cannot matter: dividing by the padded width +instead of the real token count is a positive scalar, and normalization cancels it +exactly. A mutation test that changed `counted` to `width` passed, and it was right +to pass. What does matter is which positions are *summed* — including padded +positions moves the direction, and on the test's own data it moves it about 45 +degrees. So pin the mask, never the divisor, and do not write a comment claiming a +mutation will be caught without running it: the first version of that comment was +wrong in exactly this way. + +**How to apply.** When a dependency is mostly a convenience wrapper, price the +wrapper before accepting its tree. Here the convenience was a model downloader and +four lines of arithmetic, and the price was two high-severity advisories with no +upstream fix, handed to everyone who installed the package. diff --git a/README.md b/README.md index 81a4b66..0d7f8e2 100644 --- a/README.md +++ b/README.md @@ -488,7 +488,7 @@ content over the network by itself. ### First-party: local embeddings -`@illodev/workfile-search-local` runs embeddings on-device (transformers.js, +`@illodev/workfile-search-local` runs embeddings on-device (onnxruntime-web, ONNX on CPU, `Xenova/multilingual-e5-small` quantized) — repository content never leaves the machine. Declare it in `project.config.mjs` with a **guarded import**, because the config must also load where the package cannot resolve diff --git a/package.json b/package.json index fee0e84..a7d25a7 100644 --- a/package.json +++ b/package.json @@ -54,8 +54,6 @@ }, "pnpm": { "overrides": { - "sharp": "^0.35.0", - "adm-zip": "^0.6.0", "fast-uri": "^3.1.5", "js-yaml": "^4.3.1" } diff --git a/packages/search-local/README.md b/packages/search-local/README.md index 0edf80f..355e206 100644 --- a/packages/search-local/README.md +++ b/packages/search-local/README.md @@ -1,7 +1,9 @@ # @illodev/workfile-search-local Local embeddings semantic search provider for [Workfile](https://github.com/illodev/workfile). -Models run on-device via [transformers.js](https://github.com/huggingface/transformers.js) (ONNX on CPU) — repository content never leaves the machine. +Models run on-device via [onnxruntime-web](https://onnxruntime.ai/) and +[@huggingface/tokenizers](https://github.com/huggingface/tokenizers) (ONNX on CPU, WASM) — +repository content never leaves the machine. ## Usage @@ -59,8 +61,10 @@ record re-embeds that record only. ## Behavior - The model (default `Xenova/multilingual-e5-small`, quantized, multilingual) - is downloaded once on first use to the transformers.js cache; everything - afterwards is offline. + is downloaded once on first use into `modelDir` — about 135 MB of tokenizer + and weights — and everything afterwards is offline. Point `model` at a + directory holding `tokenizer.json` and the ONNX file to skip the download + entirely, which is how this runs on a machine with no network. - Record embeddings are cached in `~/.cache/workfile/embeddings`, keyed by content hash — editing a card re-embeds that card only. Only the first `passageChars` characters of a body are embedded, so edits beyond that @@ -77,11 +81,12 @@ record re-embeds that record only. localSearchIntegration({ id: "local-embeddings", // integration id, referenced by search.provider model: "Xenova/multilingual-e5-small", - dtype: "q8", // model quantization passed to transformers.js + dtype: "q8", // which exported ONNX weights to load cacheDir: "~/.cache/workfile/embeddings-or-null", + modelDir: "~/.cache/workfile/models", // where the model and tokenizer are kept passageChars: 2000, // body characters embedded per record embedder: null, // inject your own (texts) => vectors - numThreads: 4, // ONNX threads; default: half the cores + numThreads: 4, // ONNX WASM threads; default: half the cores batchSize: 32, // records per model call; cache persists per batch onProgress: ({ done, total }) => {} // default: stderr lines on large passes }); diff --git a/packages/search-local/index.d.ts b/packages/search-local/index.d.ts index 2f982ef..459cd9d 100644 --- a/packages/search-local/index.d.ts +++ b/packages/search-local/index.d.ts @@ -8,17 +8,27 @@ export interface LocalSearchRecord { export interface LocalSearchOptions { /** Integration id, referenced by `search.provider`. Default `local-embeddings`. */ id?: string; - /** Any feature-extraction model transformers.js can load. Default `Xenova/multilingual-e5-small`. */ + /** + * A Hugging Face repository exporting ONNX feature extraction, or a + * filesystem path to a directory holding `tokenizer.json` and the ONNX file, + * which skips the download. Default `Xenova/multilingual-e5-small`. + */ model?: string; - /** Model quantization passed to transformers.js. Default `q8`. */ + /** Which exported ONNX weights to load. Default `q8`. */ dtype?: string; /** On-disk embedding cache directory; null disables persistence. Default `~/.cache/workfile/embeddings`. */ cacheDir?: string | null; + /** + * Where the model and tokenizer are kept, separate from `cacheDir` because + * vectors are cheap to recompute and the model is a 118 MB download. Default + * `~/.cache/workfile/models`. + */ + modelDir?: string; /** Body characters embedded per record. Default 2000. */ passageChars?: number; /** Injectable embedding function (tests, custom backends). Must return one L2-normalized vector per text. */ embedder?: ((texts: string[]) => Promise) | null; - /** ONNX intra-op threads. Default: half the machine's cores, never all of them. */ + /** ONNX WASM threads. Default: half the machine's cores, never all of them. */ numThreads?: number; /** Records embedded per model call; the cache persists after each batch. Default 32. */ batchSize?: number; @@ -43,3 +53,20 @@ export interface LocalSearchIntegration { export function localSearchIntegration( options?: LocalSearchOptions ): LocalSearchIntegration; + +/** + * The ONNX file a dtype names in a Hugging Face repository. Exported because it + * is a naming convention rather than an API, and a caller passing a custom + * `dtype` needs to know what will be fetched. + */ +export function onnxFileName(dtype: string): string; + +/** + * Mean-pool hidden states over an attention mask, then L2-normalize. Exported so + * the arithmetic the ranking rests on can be checked without loading a model. + */ +export function poolAndNormalize( + hidden: ArrayLike, + mask: ArrayLike, + shape: { rows: number; width: number; size: number } +): number[][]; diff --git a/packages/search-local/index.js b/packages/search-local/index.js index a7f232d..5b0be27 100644 --- a/packages/search-local/index.js +++ b/packages/search-local/index.js @@ -1,5 +1,5 @@ import { createHash } from "node:crypto"; -import { mkdir, readFile, rename, writeFile } from "node:fs/promises"; +import { mkdir, readFile, rename, stat, writeFile } from "node:fs/promises"; import { availableParallelism, homedir } from "node:os"; import { join } from "node:path"; @@ -103,23 +103,194 @@ function createEmbeddingCache(cacheDir, model) { }; } -async function loadTransformersEmbedder(model, dtype, numThreads) { - const { pipeline } = await import("@huggingface/transformers"); - // Quantized by default: on CPU q8 is several times faster than fp32 and - // the retrieval quality difference is noise at this scale. - const extractor = await pipeline("feature-extraction", model, { - dtype, - session_options: { - intraOpNumThreads: numThreads, - interOpNumThreads: 1 +/** + * The ONNX file a dtype names, in the layout Hugging Face repositories use. + * + * `q8` is spelled `model_quantized.onnx` and everything else is + * `model_.onnx`, with `fp32` the unsuffixed original. This is a naming + * convention rather than an API, so a dtype this cannot spell is refused here + * with the list — better than a 404 from a URL the caller never wrote. + */ +export function onnxFileName(dtype) { + if (dtype === "fp32") return "model.onnx"; + if (dtype === "q8") return "model_quantized.onnx"; + if (/^(fp16|int8|uint8|q4|q4f16|bnb4)$/.test(dtype)) { + return `model_${dtype}.onnx`; + } + throw new Error( + `Unknown dtype "${dtype}". Use fp32, fp16, q8, int8, uint8, q4, q4f16 or bnb4.` + ); +} + +/** + * Mean-pool a batch's hidden states over the attention mask, then L2-normalize. + * + * Separated from the session so it can be tested without a model: this is the + * arithmetic the whole ranking rests on, and it is the part a refactor breaks + * silently — a pooled vector that divides by the padded width instead of the + * real token count still looks like a plausible embedding. + * + * Padding must be excluded rather than averaged in. It is why the mask is read + * here at all: with a batch of one, dividing by the width happens to be right, + * and every batch after that is quietly wrong in proportion to how uneven the + * texts are. + */ +export function poolAndNormalize(hidden, mask, { rows, width, size }) { + const vectors = []; + for (let row = 0; row < rows; row += 1) { + const vector = new Array(size).fill(0); + let counted = 0; + for (let column = 0; column < width; column += 1) { + if (!mask[row * width + column]) continue; + counted += 1; + const base = (row * width + column) * size; + for (let i = 0; i < size; i += 1) vector[i] += hidden[base + i]; + } + if (counted) for (let i = 0; i < size; i += 1) vector[i] /= counted; + let norm = 0; + for (const value of vector) norm += value * value; + norm = Math.sqrt(norm) || 1; + for (let i = 0; i < size; i += 1) vector[i] /= norm; + vectors.push(vector); + } + return vectors; +} + +/** + * Where the model files live, fetching them once if they are not there yet. + * + * `@huggingface/transformers` used to do this, and it is most of what it was + * here for. It also brought `sharp` and `onnxruntime-node` as hard + * dependencies — image processing and an archive extractor this package never + * touches — each carrying a high-severity advisory with no upstream fix, handed + * to everybody who installed this package. See ADR-0021 and T-0221. + * + * A `model` containing a path separator is read from disk and never fetched, so + * an air-gapped or offline install works by pointing at a directory. + */ +async function ensureModelFiles(model, dtype, modelDir) { + const onnx = onnxFileName(dtype); + if (model.includes("/") && (model.startsWith(".") || model.startsWith("/"))) { + return { + tokenizer: join(model, "tokenizer.json"), + tokenizerConfig: join(model, "tokenizer_config.json"), + onnx: join(model, onnx) + }; + } + const directory = join(modelDir, model.replace(/[^a-zA-Z0-9._-]+/g, "-")); + await mkdir(directory, { recursive: true }); + const wanted = [ + ["tokenizer.json", "tokenizer.json"], + ["tokenizer_config.json", "tokenizer_config.json"], + [`onnx/${onnx}`, onnx] + ]; + const resolved = {}; + for (const [remote, local] of wanted) { + const path = join(directory, local); + resolved[local] = path; + try { + await stat(path); + continue; + } catch { + // Not cached yet. } - }); + const url = `https://huggingface.co/${model}/resolve/main/${remote}`; + const response = await fetch(url); + if (!response.ok) { + throw new Error( + `Could not fetch ${remote} for ${model}: HTTP ${response.status} from ${url}` + ); + } + // Written under a temporary name and renamed, so an interrupted + // download can never leave a truncated model that loads and produces + // nonsense — the same reason the embedding cache does it. + const temporary = `${path}.${process.pid}.tmp`; + await writeFile(temporary, Buffer.from(await response.arrayBuffer())); + await rename(temporary, path); + } + return { + tokenizer: resolved["tokenizer.json"], + tokenizerConfig: resolved["tokenizer_config.json"], + onnx: resolved[onnx] + }; +} + +/** + * Feature extraction over `onnxruntime-web` and `@huggingface/tokenizers`. + * + * Replaces one `pipeline("feature-extraction")` call, and the reason is the + * dependency tree rather than the API: this reaches the same weights through the + * WASM execution provider, which needs neither the native binding nor its + * archive extractor. Verified against the implementation it replaces — per + * vector cosine 0.9978 on the same texts with the same q8 weights, unit norms, + * and the same ranking order. The residual is the WASM and native kernels + * disagreeing at quantized precision, not a difference in method. + */ +async function loadOnnxEmbedder(model, dtype, numThreads, modelDir) { + const [ort, { Tokenizer }] = await Promise.all([ + import("onnxruntime-web"), + import("@huggingface/tokenizers") + ]); + const files = await ensureModelFiles(model, dtype, modelDir); + const [tokenizerJson, tokenizerConfigJson] = await Promise.all([ + readFile(files.tokenizer, "utf8"), + readFile(files.tokenizerConfig, "utf8").catch(() => "{}") + ]); + const config = JSON.parse(tokenizerConfigJson); + const tokenizer = new Tokenizer(JSON.parse(tokenizerJson), config); + + // A global on the ort environment rather than a session option: for the + // WASM provider `intraOpNumThreads` is not the knob, `env.wasm.numThreads` + // is. Same intent as before — half the machine, never all of it. + ort.env.wasm.numThreads = numThreads; + const session = await ort.InferenceSession.create(files.onnx); + const wantsTokenTypes = session.inputNames.includes("token_type_ids"); + // These models truncate anyway; feeding more than the position embeddings + // cover is an error from the runtime rather than a longer read. + const limit = Number(config.model_max_length) || 512; + return async (texts) => { - const output = await extractor(texts, { - pooling: "mean", - normalize: true + const encodings = texts.map((text) => { + const encoded = tokenizer.encode(text, { + return_token_type_ids: wantsTokenTypes + }); + const ids = encoded.ids.slice(0, limit); + return { + ids, + mask: (encoded.attention_mask || ids.map(() => 1)).slice(0, limit) + }; + }); + const rows = encodings.length; + const width = Math.max(...encodings.map((encoding) => encoding.ids.length)); + const ids = new BigInt64Array(rows * width); + const mask = new BigInt64Array(rows * width); + encodings.forEach((encoding, row) => { + for (let column = 0; column < encoding.ids.length; column += 1) { + ids[row * width + column] = BigInt(encoding.ids[column]); + mask[row * width + column] = BigInt(encoding.mask[column]); + } + }); + const dims = [rows, width]; + const feeds = { + input_ids: new ort.Tensor("int64", ids, dims), + attention_mask: new ort.Tensor("int64", mask, dims) + }; + // All zeros: one segment. Present only because the exported graph + // declares the input, which XLM-R-derived models do while never using it. + if (wantsTokenTypes) { + feeds.token_type_ids = new ort.Tensor( + "int64", + new BigInt64Array(rows * width), + dims + ); + } + const output = await session.run(feeds); + const hidden = output.last_hidden_state; + return poolAndNormalize(hidden.data, mask, { + rows, + width, + size: hidden.dims[2] }); - return output.tolist(); }; } @@ -142,9 +313,10 @@ async function loadTransformersEmbedder(model, dtype, numThreads) { * } * })(); * - * The model is downloaded once on first use (to the transformers.js cache) and - * everything afterwards is offline. Repository content never leaves the - * machine — which is the reason this package exists instead of an API client. + * The model is downloaded once on first use, into `modelDir`, and everything + * afterwards is offline. Repository content never leaves the machine — which is + * the reason this package exists instead of an API client. Pointing `model` at a + * directory skips the download entirely, for an install with no network. */ export function localSearchIntegration(options = {}) { const { @@ -152,9 +324,15 @@ export function localSearchIntegration(options = {}) { model = DEFAULT_MODEL, dtype = "q8", cacheDir = join(homedir(), ".cache", "workfile", "embeddings"), + /** + * Where the model and tokenizer are kept. Separate from `cacheDir`: + * vectors are cheap to recompute and the model is a 118 MB download, so + * clearing one must not cost the other. + */ + modelDir = join(homedir(), ".cache", "workfile", "models"), passageChars = 2000, embedder = null, - /** ONNX intra-op threads. Default: half the cores, never all. */ + /** ONNX WASM threads. Default: half the cores, never all. */ numThreads = DEFAULT_THREADS, /** Records embedded per model call; the cache persists after each batch. */ batchSize = 32, @@ -166,7 +344,7 @@ export function localSearchIntegration(options = {}) { const resolveEmbedder = () => { embedderPromise ||= embedder ? Promise.resolve(embedder) - : loadTransformersEmbedder(model, dtype, numThreads); + : loadOnnxEmbedder(model, dtype, numThreads, modelDir); return embedderPromise; }; diff --git a/packages/search-local/package.json b/packages/search-local/package.json index 6225872..2502fa6 100644 --- a/packages/search-local/package.json +++ b/packages/search-local/package.json @@ -45,6 +45,7 @@ "test": "node --test test/*.test.ts" }, "dependencies": { - "@huggingface/transformers": "^4.2.0" + "@huggingface/tokenizers": "^0.1.3", + "onnxruntime-web": "^1.27.0" } } diff --git a/packages/search-local/test/provider.test.ts b/packages/search-local/test/provider.test.ts index bdedc75..5707e84 100644 --- a/packages/search-local/test/provider.test.ts +++ b/packages/search-local/test/provider.test.ts @@ -4,7 +4,11 @@ import { mkdtemp, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { localSearchIntegration } from "../index.js"; +import { + localSearchIntegration, + onnxFileName, + poolAndNormalize +} from "../index.js"; import { defineProjectIntegration, searchProjectRecordsHybrid @@ -246,3 +250,63 @@ test("empty queries and empty record sets short-circuit without embedding", asyn ); assert.equal(embed.calls.length, 0); }); + +/** + * The arithmetic and the naming the ONNX path rests on, without a model. + * + * T-0221 replaced one `pipeline("feature-extraction")` call with the tokenize → + * run → pool → normalize sequence it was doing internally. Everything above this + * line injects `embedder`, so none of it touches the new code at all: these two + * functions are the parts a refactor breaks silently, and neither needs the + * 118 MB download to check. + */ +test("a dtype names its ONNX file, and an unknown one is refused with the list", () => { + assert.equal(onnxFileName("fp32"), "model.onnx"); + // The one that is not `model_.onnx`, which is why this exists. + assert.equal(onnxFileName("q8"), "model_quantized.onnx"); + assert.equal(onnxFileName("fp16"), "model_fp16.onnx"); + assert.equal(onnxFileName("q4"), "model_q4.onnx"); + assert.throws( + () => onnxFileName("q9"), + // Refused here rather than becoming a 404 on a URL the caller never wrote. + (error: Error) => /Unknown dtype "q9"/.test(error.message) && /q8/.test(error.message) + ); +}); + +test("pooling averages only the real tokens and returns unit vectors", () => { + // Two rows, three columns, two features. Row 0 has one padded column; row 1 + // has two. Hidden states chosen so the correct mean is exact in binary. + const hidden = [ + // row 0 + 2, 0, /* col 1 */ 4, 0, /* col 2, padded */ 1000, 1000, + // row 1 + 0, 3, /* col 1, padded */ 500, 500, /* col 2, padded */ 700, 700 + ]; + const mask = [1, 1, 0, 1, 0, 0]; + const vectors = poolAndNormalize(hidden, mask, { rows: 2, width: 3, size: 2 }); + + // Row 0 pools (2,0) and (4,0) to (3,0); normalized that is (1,0). Summing + // the padded column instead would give (335.3, 333.3), about 45 degrees off, + // and that is what this pins. + // + // Note what it deliberately does *not* pin: the divisor. Dividing by the + // padded width rather than the token count is a positive scalar, and the + // normalization below cancels it exactly — so that mutation is undetectable + // here because it is undetectable anywhere. `counted` is kept for the sake of + // the value being a mean, not because the ranking can tell. + assert.deepEqual(vectors[0], [1, 0]); + assert.deepEqual(vectors[1], [0, 1]); + + for (const vector of vectors) { + const norm = Math.sqrt(vector.reduce((sum, value) => sum + value * value, 0)); + assert.ok(Math.abs(norm - 1) < 1e-12, `not a unit vector: ${norm}`); + } +}); + +test("an all-padding row is a zero vector rather than a division by zero", () => { + // Reachable only through a bug upstream, and it must not return NaN: a NaN + // score sorts unpredictably and would corrupt the whole ranking rather than + // just that record. + const vectors = poolAndNormalize([5, 5], [0], { rows: 1, width: 1, size: 2 }); + assert.deepEqual(vectors[0], [0, 0]); +}); diff --git a/packages/workfile/docs/cli.md b/packages/workfile/docs/cli.md index c68ddb9..88c6788 100644 --- a/packages/workfile/docs/cli.md +++ b/packages/workfile/docs/cli.md @@ -155,7 +155,7 @@ if the repository explicitly declares it. The first-party provider is [`@illodev/workfile-search-local`](https://github.com/illodev/workfile/tree/main/packages/search-local#readme): -on-device embeddings via transformers.js, cached by content hash, fully +on-device embeddings via onnxruntime-web, cached by content hash, fully offline after the first model download. `upgrade` is the one command to run after bumping `@illodev/workfile`: it diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 095b21d..b0392ce 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -5,8 +5,6 @@ settings: excludeLinksFromLockfile: false overrides: - sharp: ^0.35.0 - adm-zip: ^0.6.0 fast-uri: ^3.1.5 js-yaml: ^4.3.1 @@ -29,9 +27,12 @@ importers: packages/search-local: dependencies: - '@huggingface/transformers': - specifier: ^4.2.0 - version: 4.2.0(@types/node@26.1.2) + '@huggingface/tokenizers': + specifier: ^0.1.3 + version: 0.1.3 + onnxruntime-web: + specifier: ^1.27.0 + version: 1.27.0 packages/workfile: dependencies: @@ -323,9 +324,6 @@ packages: '@emnapi/core@2.0.0-alpha.3': resolution: {integrity: sha512-AZypUeJ/yByuxyS7BlSNRDOMLMlROYtjYdIAuBmJssVz1UJDSeYxLrdizhXCFYhedC5bqd/ASy8EuNXbVVXp9g==} - '@emnapi/runtime@1.11.3': - resolution: {integrity: sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA==} - '@emnapi/runtime@2.0.0-alpha.3': resolution: {integrity: sha512-hFPAhMUjJD9BSyCANEISPOogeXC9Zo9ZQl7L6vKnaVsMkCtzznaW/naYypeyl0Gv5rYfWYsZbpixTMpjDJzQeA==} @@ -359,178 +357,9 @@ packages: peerDependencies: hono: ^4 - '@huggingface/jinja@0.5.9': - resolution: {integrity: sha512-uWTG+l3VJRsl7EXxYizuL3P+cCPoc3cRqbWWRcQN0FhejRfbdq0RNhCmbY/YDtnTcz9icdLYuLDjsnz4d8JMuw==} - engines: {node: '>=18'} - '@huggingface/tokenizers@0.1.3': resolution: {integrity: sha512-8rF/RRT10u+kn7YuUbUg0OF30K8rjTc78aHpxT+qJ1uWSqxT1MHi8+9ltwYfkFYJzT/oS+qw3JVfHtNMGAdqyA==} - '@huggingface/transformers@4.2.0': - resolution: {integrity: sha512-8BRCoBMH0XsWaEIamuR0LrJGAfftgHAfb2Vrffy0VKlSAE/MnUJ5/h/zTfEP3fDIft+nk7TqB8xXEyABGitBjQ==} - - '@img/colour@1.1.0': - resolution: {integrity: sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==} - engines: {node: '>=18'} - - '@img/sharp-darwin-arm64@0.35.3': - resolution: {integrity: sha512-RMnFX7YQsMoh7lWfcM4NEHHymBX/rLuKNPVM84XE9ONPcaSCDgE7CHIHpSgPcO2xcRthgBy1HfNO319mwhIAkg==} - engines: {node: '>=20.9.0'} - cpu: [arm64] - os: [darwin] - - '@img/sharp-darwin-x64@0.35.3': - resolution: {integrity: sha512-Xo+5uFBtLN0BKqieTxiFzFPQAUlBbbH5iBKyRX/z1JrbnYsHTfKJnUfL8+p2TPXr1pXqao4eeL4Rl144uDpK9w==} - engines: {node: '>=20.9.0'} - cpu: [x64] - os: [darwin] - - '@img/sharp-freebsd-wasm32@0.35.3': - resolution: {integrity: sha512-lUxcqWIj2wMQ9BrwNjngcr1gWUr5xgaGThBRqPPalIC2n67Cqj1uPh8NnA/ZhAg8hUbKl+kVHKwgUIwe6ZYPrg==} - engines: {node: '>=20.9.0'} - os: [freebsd] - - '@img/sharp-libvips-darwin-arm64@1.3.2': - resolution: {integrity: sha512-9J6ypZFpQBj4YnePGoq/S38w6nz+vqg5WZLrLGY4YuSemdMq47GMLBPO42MzwdGwpg/agZ7xzZcFHa48xlywfg==} - cpu: [arm64] - os: [darwin] - - '@img/sharp-libvips-darwin-x64@1.3.2': - resolution: {integrity: sha512-m2pW1n6cns9VaubNwsZ+c3CRYjxNQWgJ5gPlnL1nbBcpkBvFm6SCFN5o0psFHI8w9n11NKhFkeEDns98tiqbEw==} - cpu: [x64] - os: [darwin] - - '@img/sharp-libvips-linux-arm64@1.3.2': - resolution: {integrity: sha512-dqVSFynCox4C/J8kT16V7SIFAns0IjgLwkvYT7p8LQVmJ5OS5b6tI9IGflxTeuBS//zXeFIUbwt5dwxyZ17cnA==} - cpu: [arm64] - os: [linux] - libc: [glibc] - - '@img/sharp-libvips-linux-arm@1.3.2': - resolution: {integrity: sha512-1eMLzy92I4J6rmi4mAT8yC3HxOtniyGELlzGbNMLLeqe052ahFQ0h6LFq+lh5DsDIdYViIDst08abvSbcEdLXQ==} - cpu: [arm] - os: [linux] - libc: [glibc] - - '@img/sharp-libvips-linux-ppc64@1.3.2': - resolution: {integrity: sha512-3z0NHDxD6n5I9gc05U1eW1AyRm+Gznzq3naMrthPNqE6oYykcogW0l/jfpJdjYnuNl8R7yI9pNbE1XiUeyq0Aw==} - cpu: [ppc64] - os: [linux] - libc: [glibc] - - '@img/sharp-libvips-linux-riscv64@1.3.2': - resolution: {integrity: sha512-bsb4rI+NldGOsXuej2r8OdSS8+zXDVaCWxyWrcv6kneTOlgAHtZABRzBBCwdsPiD90J4myNJuHpg6kA20ImW/w==} - cpu: [riscv64] - os: [linux] - libc: [glibc] - - '@img/sharp-libvips-linux-s390x@1.3.2': - resolution: {integrity: sha512-/ABshyj8gCpyIrNXnHn4LorDJ0HHm1VhXPBlxZ8zAtfVPAaSafXPGn+sUSIRiwaSBy0mmFjSjiXI5mkcwdChKQ==} - cpu: [s390x] - os: [linux] - libc: [glibc] - - '@img/sharp-libvips-linux-x64@1.3.2': - resolution: {integrity: sha512-ITPEtgffGJ0S6G9dRyw/366tJQqFRcHWPHhC+Stpg3Z8AEMrDrTr2lhdz4f/Y/HMbRh//7Z5mBzEpVdi62Oc3w==} - cpu: [x64] - os: [linux] - libc: [glibc] - - '@img/sharp-libvips-linuxmusl-arm64@1.3.2': - resolution: {integrity: sha512-zE9EdiUzUmg5mDT5a1rk5fYJ6GWPloTwWBYDS14naqHsL+EaMpDj1AWnpLgh3u0YCORv2Tt50wrcrpYqkP97Kw==} - cpu: [arm64] - os: [linux] - libc: [musl] - - '@img/sharp-libvips-linuxmusl-x64@1.3.2': - resolution: {integrity: sha512-m0lrLiUt+lBYnCFr8qV/65yMR4E/c7/wf78I5eKTdkEakFAlZ9QlzEM3QIhhAwVeUhLAHLcCq7a7Vszq/oFNZQ==} - cpu: [x64] - os: [linux] - libc: [musl] - - '@img/sharp-linux-arm64@0.35.3': - resolution: {integrity: sha512-QgKDspHPnrU+GQ55XPhGwyhC8acLVOOSyAvo1oVfFmrIXLkDNmGWzAfDZ4xK8oSA1qBQrALcHX0G5UZni/SuFQ==} - engines: {node: '>=20.9.0'} - cpu: [arm64] - os: [linux] - libc: [glibc] - - '@img/sharp-linux-arm@0.35.3': - resolution: {integrity: sha512-affVWCTLooy8TSxbDx2qkzuDeaWLNVBA+P//FNBirHsXpP2fuBhk5AuboYUnrDnzoXes8GFjpTx0SBFOCRg+FA==} - engines: {node: '>=20.9.0'} - cpu: [arm] - os: [linux] - libc: [glibc] - - '@img/sharp-linux-ppc64@0.35.3': - resolution: {integrity: sha512-sMd8rDxmpLOwv/7N44klFjOD5DUO7FLdjiXDI0hoxYaf7Ar262dQIEkosE98bps+5HPLtp/EvNqeqQtOycP/IA==} - engines: {node: '>=20.9.0'} - cpu: [ppc64] - os: [linux] - libc: [glibc] - - '@img/sharp-linux-riscv64@0.35.3': - resolution: {integrity: sha512-0Eob78yjlYPfL5vMNWAW55l3R9Y6BQS/gOfe0ZcP9mEz9ohhKSt4im1hayiknXgf8AWrFqMvJcKIdmLmEe7yeQ==} - engines: {node: '>=20.9.0'} - cpu: [riscv64] - os: [linux] - libc: [glibc] - - '@img/sharp-linux-s390x@0.35.3': - resolution: {integrity: sha512-KgAxQ0DxpNOq1rG2t5cgTgShJFGSuU7XO45cqC+1NVOuZnP6tlgZRuSYOfNupGkHID0o3cJOsw4DVeJpMovcGw==} - engines: {node: '>=20.9.0'} - cpu: [s390x] - os: [linux] - libc: [glibc] - - '@img/sharp-linux-x64@0.35.3': - resolution: {integrity: sha512-8pqvxubL2PGdhlPy6GLqzDYMUjyRmKAwKHYKixpdJYBUK7PJ0C029XdsnpFIdgRZG68fZiGdHVWcKPvtiPB4cA==} - engines: {node: '>=20.9.0'} - cpu: [x64] - os: [linux] - libc: [glibc] - - '@img/sharp-linuxmusl-arm64@0.35.3': - resolution: {integrity: sha512-Vz0iQjzzcSX3HCbfwFfCSG/9SCIqyO0mH2sXyiHaAYfBk0cRsCWXRyQYX0ovCK/PAQBbTzQ0dsPQHh5MAFL59w==} - engines: {node: '>=20.9.0'} - cpu: [arm64] - os: [linux] - libc: [musl] - - '@img/sharp-linuxmusl-x64@0.35.3': - resolution: {integrity: sha512-6O1NPKcDVj9QEdg7Hx549EX8U0rp6yXQERqru6yRN7fGBn32UvIRJUlWnk+8xDCiG76hXVBbX82NZ/ZKr0euIg==} - engines: {node: '>=20.9.0'} - cpu: [x64] - os: [linux] - libc: [musl] - - '@img/sharp-wasm32@0.35.3': - resolution: {integrity: sha512-cZ0XkcYGpHZkqW6iCkqTcmUC0CD9DhD5d/qeZlZkfRBn6GnHniZXLUo5+9xw8Iv76YE6LQFN9YNBlKREcCG76w==} - engines: {node: '>=20.9.0'} - - '@img/sharp-webcontainers-wasm32@0.35.3': - resolution: {integrity: sha512-2rnq7bX3NzeR2T4YWgz8qiG4h3TSdMe+vN1iQXpJleSJ3SM5zQ8Fy2SyyXAWlbxpEZ2Y+Z4u1BePgJEYbSy80Q==} - engines: {node: '>=20.9.0'} - cpu: [wasm32] - - '@img/sharp-win32-arm64@0.35.3': - resolution: {integrity: sha512-4bPwFdMbeC4JQ8L8LOyWp6nsHcboP5fxkp6iPOXz2Vg49R42TuMs2whkJ5OAP4/Ul035qOzy0AecOF9VOscn4w==} - engines: {node: '>=20.9.0'} - cpu: [arm64] - os: [win32] - - '@img/sharp-win32-ia32@0.35.3': - resolution: {integrity: sha512-r53mXsBN6lFUDiST764SvgwUdHAqM4rPAiDzAmf4fLoB6X/rkfyTrLCg6+g17wJJiCmB3JYgHuUldCWUIRFSXw==} - engines: {node: ^20.9.0} - cpu: [ia32] - os: [win32] - - '@img/sharp-win32-x64@0.35.3': - resolution: {integrity: sha512-D4y1vNeZrIIJCN+uHaWVtH86B+aCrdMYYjicy9pXHvbGZeGYLLSd3wdVuC37FxVXlU1ARsk84eKWfWMXGYEqvA==} - engines: {node: '>=20.9.0'} - cpu: [x64] - os: [win32] - '@jridgewell/gen-mapping@0.3.13': resolution: {integrity: sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA==} @@ -1659,10 +1488,6 @@ packages: resolution: {integrity: sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==} engines: {node: '>= 0.6'} - adm-zip@0.6.0: - resolution: {integrity: sha512-XleryMhbuksdKtofnWZ9Sk+4CUTbms4Mb/EU32SZwToAyZ5RgVos/ki8n+yr0LWHOGKuakbXTuuYNHLQjhddgg==} - engines: {node: '>=14.0'} - ajv-formats@2.1.1: resolution: {integrity: sha512-Wx0Kx52hxE7C18hkMEggYlEifqWZtYaRgouJor+WMdPnQyEK13vgEWyVNup7SoeeoLMsr4kf5h6dOW11I15MUA==} peerDependencies: @@ -1730,10 +1555,6 @@ packages: resolution: {integrity: sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==} engines: {node: '>=18'} - boolean@3.2.0: - resolution: {integrity: sha512-d0II/GO9uf9lfUHH2BQsjxzRJZBdsjgsBiW4BvhWk/3qoKwQFjIDVN19PfX8F2D/r9PCMTtLWjYVCFrpeYUzsw==} - deprecated: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info. - brace-expansion@5.0.9: resolution: {integrity: sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==} engines: {node: 20 || >=22} @@ -1916,10 +1737,6 @@ packages: resolution: {integrity: sha512-H9LMLr5zwIbSxrmvikGuI/5KGhZ8E2zH3stkMgM5LpOWDutGM2JZaj460Udnf1a+946zc7YBgrqEWwbk7zHvGw==} engines: {node: '>=18'} - define-data-property@1.1.4: - resolution: {integrity: sha512-rBMvIzlpA8v6E+SJZoo++HAYqsLrkg7MSfIinMPFhmkorw7X+dOXVJQs+QT69zGkzMyfDnIMN2Wid1+NbL3T+A==} - engines: {node: '>= 0.4'} - define-lazy-prop@2.0.0: resolution: {integrity: sha512-Ds09qNh8yw3khSjiJjiUInaGX9xlqZDY7JVryGxdxV7NPeuqQfplOpQ66yJFZut3jLa5zOwkXw1g9EI2uKh4Og==} engines: {node: '>=8'} @@ -1928,10 +1745,6 @@ packages: resolution: {integrity: sha512-N+MeXYoqr3pOgn8xfyRPREN7gHakLYjhsHhWGT3fWAiL4IkAt0iDw14QiiEm2bE30c5XX5q0FtAA3CK5f9/BUg==} engines: {node: '>=12'} - define-properties@1.2.1: - resolution: {integrity: sha512-8QmQKqEASLd5nx0U1B1okLElbUuuttJ/AnYmRXbbbGDWh6uS208EjD4Xqq/I9wK7u0v6O08XhTWnt5XtEbR6Dg==} - engines: {node: '>= 0.4'} - depd@2.0.0: resolution: {integrity: sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==} engines: {node: '>= 0.8'} @@ -1943,9 +1756,6 @@ packages: detect-node-es@1.1.0: resolution: {integrity: sha512-ypdmJU/TbBby2Dxibuv7ZLW3Bs1QEmM7nHjEANfohJLvE0XVujisn1qPJcZxg+qDucsr+bP6fLD1rPS3AhJ7EQ==} - detect-node@2.1.0: - resolution: {integrity: sha512-T0NIuQpnTvFDATNuHN5roPwSBG83rFsuO+MXXH9/3N1eFbn4wcPjttvjMLEPWJ0RGUYgQE7cGgS3tNxbqCGM7g==} - diff@8.0.4: resolution: {integrity: sha512-DPi0FmjiSU5EvQV0++GFDOJ9ASQUVFh5kD+OzOnYdi7n3Wpm9hWWGfB/O2blfHcMVTL5WkQXSnRiK9makhrcnw==} engines: {node: '>=0.3.1'} @@ -2005,9 +1815,6 @@ packages: es-toolkit@1.50.0: resolution: {integrity: sha512-OyZKhUVvEep9ITEiwHn8GKnMRQIVqoSIX7WnRbkWgJkllCujilqP2rD0u979tkl8wqyc8ICwlc1UBVv/Sl1G6w==} - es6-error@4.1.1: - resolution: {integrity: sha512-Um/+FxMr9CISWh0bi5Zv0iOD+4cFh5qLeks1qhAopKVAJw3drgKbKySikp7wGhDL0HPeaja0P5ULZrxLkniUVg==} - escalade@3.2.0: resolution: {integrity: sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==} engines: {node: '>=6'} @@ -2015,10 +1822,6 @@ packages: escape-html@1.0.3: resolution: {integrity: sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow==} - escape-string-regexp@4.0.0: - resolution: {integrity: sha512-TtpcNJ3XAzx3Gq8sWRzJaVajRs0uVxA2YAkdb1jm2YkPz4G6egUFAyA3n5vtEIZefPk5Wa4UXbKuS5fKkJWdgA==} - engines: {node: '>=10'} - esprima@4.0.1: resolution: {integrity: sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A==} engines: {node: '>=4'} @@ -2163,18 +1966,10 @@ packages: resolution: {integrity: sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==} engines: {node: '>= 6'} - global-agent@3.0.0: - resolution: {integrity: sha512-PT6XReJ+D07JvGoxQMkT6qji/jVNfX/h364XHZOWeRzy64sSFr+xJ5OX7LI3b4MPQzdL4H8Y8M0xzPpsVMwA8Q==} - engines: {node: '>=10.0'} - global-directory@5.0.0: resolution: {integrity: sha512-1pgFdhK3J2LeM+dVf2Pd424yHx2ou338lC0ErNP2hPx4j8eW1Sp0XqSjNxtk6Tc4Kr5wlWtSvz8cn2yb7/SG/w==} engines: {node: '>=20'} - globalthis@1.0.4: - resolution: {integrity: sha512-DpLKbNU4WylpxJykQujfCcwYWiV/Jhm50Goo0wrVILAv5jOr9d+H+UR3PhSCD2rCCEIg0uc+G+muBTwD54JhDQ==} - engines: {node: '>= 0.4'} - gopd@1.2.0: resolution: {integrity: sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==} engines: {node: '>= 0.4'} @@ -2185,9 +1980,6 @@ packages: guid-typescript@1.0.9: resolution: {integrity: sha512-Y8T4vYhEfwJOTbouREvG+3XDsjr8E3kIr7uf+JZ0BYloFsttiHU0WfvANVsR7TxNUJa/WpCnw/Ino/p+DeBhBQ==} - has-property-descriptors@1.0.2: - resolution: {integrity: sha512-55JNKuIW+vq4Ke1BjOTjM2YctQIvCT7GFzHwmfZPGo5wnrgkid0YQtnAleFSqumZm4az3n2BS+erby5ipJdgrg==} - has-symbols@1.1.0: resolution: {integrity: sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==} engines: {node: '>= 0.4'} @@ -2367,9 +2159,6 @@ packages: json-schema-typed@8.0.2: resolution: {integrity: sha512-fQhoXdcvc3V28x7C7BMs4P5+kNlgUURe2jmUT1T//oBRMDrqy1QPelJimwZGo7Hg9VPV3EQV5Bnq4hbFy2vetA==} - json-stringify-safe@5.0.1: - resolution: {integrity: sha512-ZClg6AaYvamvYEE82d3Iyd3vSSIjQ+odgjaTzRuO3s7toCdFKczob2i0zCh7JE8kWn17yvAWhUVxvqGwUalsRA==} - json5@2.2.3: resolution: {integrity: sha512-XmOWe7eyHYH14cLdVPoyg+GOH3rYX++KpzrylJwSW98t3Nk+U8XOl8FWKOgwtzdb8lXGf6zYwDUzeHMWfxasyg==} engines: {node: '>=6'} @@ -2559,10 +2348,6 @@ packages: magic-string@0.30.21: resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==} - matcher@3.0.0: - resolution: {integrity: sha512-OkeDaAZ/bQCxeFAozM55PKcKU0yJMPGifLwV4Qgjitu+5MoAfSQN4lsLJeXZ1b8w0x+/Emda6MZgXS1jvsapng==} - engines: {node: '>=10'} - math-intrinsics@1.1.0: resolution: {integrity: sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==} engines: {node: '>= 0.4'} @@ -2645,10 +2430,6 @@ packages: resolution: {integrity: sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew==} engines: {node: '>= 0.4'} - object-keys@1.1.1: - resolution: {integrity: sha512-NuAESUOUMrlIXOfHKzD6bpPu3tYt3xvjNdRIQ+FeT0lNb4K8WR70CaDxhuNguS2XG+GjkyMwOzsN5ZktImfhLA==} - engines: {node: '>= 0.4'} - object-treeify@1.1.33: resolution: {integrity: sha512-EFVjAYfzWqWsBMRHPMAXLCDIJnpMhdWAqR7xG6M6a2cs6PMFpl/+Z20w9zDW4vkxOFfddegBKq9Rehd0bxWE7A==} engines: {node: '>= 10'} @@ -2668,18 +2449,11 @@ packages: resolution: {integrity: sha512-VXJjc87FScF88uafS3JllDgvAm+c/Slfz06lorj2uAY34rlUu0Nt+v8wreiImcrgAjjIHp1rXpTDlLOGw29WwQ==} engines: {node: '>=18'} - onnxruntime-common@1.24.0-dev.20251116-b39e144322: - resolution: {integrity: sha512-BOoomdHYmNRL5r4iQ4bMvsl2t0/hzVQ3OM3PHD0gxeXu1PmggqBv3puZicEUVOA3AtHHYmqZtjMj9FOfGrATTw==} - - onnxruntime-common@1.24.3: - resolution: {integrity: sha512-GeuPZO6U/LBJXvwdaqHbuUmoXiEdeCjWi/EG7Y1HNnDwJYuk6WUbNXpF6luSUY8yASul3cmUlLGrCCL1ZgVXqA==} - - onnxruntime-node@1.24.3: - resolution: {integrity: sha512-JH7+czbc8ALA819vlTgcV+Q214/+VjGeBHDjX81+ZCD0PCVCIFGFNtT0V4sXG/1JXypKPgScQcB3ij/hk3YnTg==} - os: [win32, darwin, linux] + onnxruntime-common@1.27.0: + resolution: {integrity: sha512-3KxL5wIVqa8Ex08jxSzncm9CMgw8CjOFyOQ7SxvG9o0cVLlhTNKXyIQuTbtX4tGPJEf73OER2xrjt4HJSBL4ow==} - onnxruntime-web@1.26.0-dev.20260416-b7804b056c: - resolution: {integrity: sha512-MD6Ss4GSpQBo6zqoJzyT9LRbKYs7x/JVN23FT24EcEvlqF4VuzPOeH6X38orZPKHQDbprn7K+SBpu0/mj2CQiw==} + onnxruntime-web@1.27.0: + resolution: {integrity: sha512-ogDLsqIozHZwifPuN37OproAo0byX6t43/bP8GzeZWBWD6MOGExswFAx3up4NS/vvWBOg2u2PXomDt3rMmdQSg==} open@11.0.0: resolution: {integrity: sha512-smsWv2LzFjP03xmvFoJ331ss6h+jixfA4UUV/Bsiyuu4YJPfN+FIQGOIiv4w9/+MoHkfkJ22UIaQWRVFRfH6Vw==} @@ -2890,10 +2664,6 @@ packages: resolution: {integrity: sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw==} engines: {iojs: '>=1.0.0', node: '>=0.10.0'} - roarr@2.15.4: - resolution: {integrity: sha512-CHhPh+UNHD2GTXNYhPWLnU8ONHdI+5DI+4EYIAOaiD63rHeYlZvyh8P+in5999TTSFgUYuKUAjzRI4mdh/p+2A==} - engines: {node: '>=8.0'} - rolldown@1.2.1: resolution: {integrity: sha512-4FKJhg8d3OiyQOA6Q1Q0hoFFpW9/OoX+VsHzpECsdsIZoOArrAK90gl59YK/Z+gnDel45bgJZK03ozH/9bCqEw==} engines: {node: ^20.19.0 || >=22.12.0} @@ -2916,9 +2686,6 @@ packages: scheduler@0.27.0: resolution: {integrity: sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==} - semver-compare@1.0.0: - resolution: {integrity: sha512-YM3/ITh2MJ5MtzaM429anh+x2jiLVjqILF4m4oyQB18W7Ggea7BfqdH/wGMK7dDiMghv/6WG7znWMwUDzJiXow==} - semver@6.3.1: resolution: {integrity: sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA==} hasBin: true @@ -2932,10 +2699,6 @@ packages: resolution: {integrity: sha512-1gnZf7DFcoIcajTjTwjwuDjzuz4PPcY2StKPlsGAQ1+YH20IRVrBaXSWmdjowTJ6u8Rc01PoYOGHXfP1mYcZNQ==} engines: {node: '>= 18'} - serialize-error@7.0.1: - resolution: {integrity: sha512-8I8TjW5KMOKsZQTvoxjuSIa7foAwPWGOts+6o7sgjz41/qMD9VQHEDxi6PBvK2l0MXUmqZyNpUK+T2tQaaElvw==} - engines: {node: '>=10'} - serve-static@2.2.1: resolution: {integrity: sha512-xRXBn0pPqQTVQiC8wyQrKs2MOlX24zQ0POGaj0kultvoOCstBQM5yvOhAVSUwOMjQtTvsPWoNCHfPGwaaQJhTw==} engines: {node: '>= 18'} @@ -2948,15 +2711,6 @@ packages: engines: {node: '>=20.18.1'} hasBin: true - sharp@0.35.3: - resolution: {integrity: sha512-ej0zVHuZGHCiABXcNxeYhpRnPNPAcvbG8RMdBAhDAxLKkCRVSpK3Iyu7qbqw3JMzoj0REeM6f3tJLtVwl0023Q==} - engines: {node: '>=20.9.0'} - peerDependencies: - '@types/node': '*' - peerDependenciesMeta: - '@types/node': - optional: true - shebang-command@2.0.0: resolution: {integrity: sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==} engines: {node: '>=8'} @@ -2999,9 +2753,6 @@ packages: resolution: {integrity: sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g==} engines: {node: '>=0.10.0'} - sprintf-js@1.1.3: - resolution: {integrity: sha512-Oo+0REFV59/rz3gfJNKQiBlwfHaSESl1pcGyABQsnnIfWOFt6JNj5gCog2U6MLZ//IGYD+nA8nI+mTShREReaA==} - statuses@2.0.2: resolution: {integrity: sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw==} engines: {node: '>= 0.8'} @@ -3090,10 +2841,6 @@ packages: tw-animate-css@1.4.0: resolution: {integrity: sha512-7bziOlRqH0hJx80h/3mbicLW7o8qLsH5+RaLR2t+OHM3D0JlWGODQKQ4cxbK7WlvmUxpcj6Kgu6EKqjrGFe3QQ==} - type-fest@0.13.1: - resolution: {integrity: sha512-34R7HTnG0XIJcBSn5XhDd7nNFPRcXYRZrBB2O2jdKqYODldSzBAqzsWoZYYvduky73toYS/ESqxPvkDf/F0XMg==} - engines: {node: '>=10'} - type-is@2.1.0: resolution: {integrity: sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==} engines: {node: '>= 18'} @@ -3588,11 +3335,6 @@ snapshots: tslib: 2.8.1 optional: true - '@emnapi/runtime@1.11.3': - dependencies: - tslib: 2.8.1 - optional: true - '@emnapi/runtime@2.0.0-alpha.3': dependencies: tslib: 2.8.1 @@ -3628,126 +3370,8 @@ snapshots: dependencies: hono: 4.12.33 - '@huggingface/jinja@0.5.9': {} - '@huggingface/tokenizers@0.1.3': {} - '@huggingface/transformers@4.2.0(@types/node@26.1.2)': - dependencies: - '@huggingface/jinja': 0.5.9 - '@huggingface/tokenizers': 0.1.3 - onnxruntime-node: 1.24.3 - onnxruntime-web: 1.26.0-dev.20260416-b7804b056c - sharp: 0.35.3(@types/node@26.1.2) - transitivePeerDependencies: - - '@types/node' - - '@img/colour@1.1.0': {} - - '@img/sharp-darwin-arm64@0.35.3': - optionalDependencies: - '@img/sharp-libvips-darwin-arm64': 1.3.2 - optional: true - - '@img/sharp-darwin-x64@0.35.3': - optionalDependencies: - '@img/sharp-libvips-darwin-x64': 1.3.2 - optional: true - - '@img/sharp-freebsd-wasm32@0.35.3': - dependencies: - '@img/sharp-wasm32': 0.35.3 - optional: true - - '@img/sharp-libvips-darwin-arm64@1.3.2': - optional: true - - '@img/sharp-libvips-darwin-x64@1.3.2': - optional: true - - '@img/sharp-libvips-linux-arm64@1.3.2': - optional: true - - '@img/sharp-libvips-linux-arm@1.3.2': - optional: true - - '@img/sharp-libvips-linux-ppc64@1.3.2': - optional: true - - '@img/sharp-libvips-linux-riscv64@1.3.2': - optional: true - - '@img/sharp-libvips-linux-s390x@1.3.2': - optional: true - - '@img/sharp-libvips-linux-x64@1.3.2': - optional: true - - '@img/sharp-libvips-linuxmusl-arm64@1.3.2': - optional: true - - '@img/sharp-libvips-linuxmusl-x64@1.3.2': - optional: true - - '@img/sharp-linux-arm64@0.35.3': - optionalDependencies: - '@img/sharp-libvips-linux-arm64': 1.3.2 - optional: true - - '@img/sharp-linux-arm@0.35.3': - optionalDependencies: - '@img/sharp-libvips-linux-arm': 1.3.2 - optional: true - - '@img/sharp-linux-ppc64@0.35.3': - optionalDependencies: - '@img/sharp-libvips-linux-ppc64': 1.3.2 - optional: true - - '@img/sharp-linux-riscv64@0.35.3': - optionalDependencies: - '@img/sharp-libvips-linux-riscv64': 1.3.2 - optional: true - - '@img/sharp-linux-s390x@0.35.3': - optionalDependencies: - '@img/sharp-libvips-linux-s390x': 1.3.2 - optional: true - - '@img/sharp-linux-x64@0.35.3': - optionalDependencies: - '@img/sharp-libvips-linux-x64': 1.3.2 - optional: true - - '@img/sharp-linuxmusl-arm64@0.35.3': - optionalDependencies: - '@img/sharp-libvips-linuxmusl-arm64': 1.3.2 - optional: true - - '@img/sharp-linuxmusl-x64@0.35.3': - optionalDependencies: - '@img/sharp-libvips-linuxmusl-x64': 1.3.2 - optional: true - - '@img/sharp-wasm32@0.35.3': - dependencies: - '@emnapi/runtime': 1.11.3 - optional: true - - '@img/sharp-webcontainers-wasm32@0.35.3': - dependencies: - '@img/sharp-wasm32': 0.35.3 - optional: true - - '@img/sharp-win32-arm64@0.35.3': - optional: true - - '@img/sharp-win32-ia32@0.35.3': - optional: true - - '@img/sharp-win32-x64@0.35.3': - optional: true - '@jridgewell/gen-mapping@0.3.13': dependencies: '@jridgewell/sourcemap-codec': 1.5.5 @@ -4801,8 +4425,6 @@ snapshots: mime-types: 3.0.2 negotiator: 1.0.0 - adm-zip@0.6.0: {} - ajv-formats@2.1.1(ajv@8.20.0): optionalDependencies: ajv: 8.20.0 @@ -4858,8 +4480,6 @@ snapshots: transitivePeerDependencies: - supports-color - boolean@3.2.0: {} - brace-expansion@5.0.9: dependencies: balanced-match: 4.0.4 @@ -5022,30 +4642,16 @@ snapshots: bundle-name: 4.1.0 default-browser-id: 5.0.1 - define-data-property@1.1.4: - dependencies: - es-define-property: 1.0.1 - es-errors: 1.3.0 - gopd: 1.2.0 - define-lazy-prop@2.0.0: {} define-lazy-prop@3.0.0: {} - define-properties@1.2.1: - dependencies: - define-data-property: 1.1.4 - has-property-descriptors: 1.0.2 - object-keys: 1.1.1 - depd@2.0.0: {} detect-libc@2.1.2: {} detect-node-es@1.1.0: {} - detect-node@2.1.0: {} - diff@8.0.4: {} dot-prop@6.0.1: @@ -5094,14 +4700,10 @@ snapshots: es-toolkit@1.50.0: {} - es6-error@4.1.1: {} - escalade@3.2.0: {} escape-html@1.0.3: {} - escape-string-regexp@4.0.0: {} - esprima@4.0.1: {} etag@1.8.1: {} @@ -5284,34 +4886,16 @@ snapshots: dependencies: is-glob: 4.0.3 - global-agent@3.0.0: - dependencies: - boolean: 3.2.0 - es6-error: 4.1.1 - matcher: 3.0.0 - roarr: 2.15.4 - semver: 7.8.5 - serialize-error: 7.0.1 - global-directory@5.0.0: dependencies: ini: 6.0.0 - globalthis@1.0.4: - dependencies: - define-properties: 1.2.1 - gopd: 1.2.0 - gopd@1.2.0: {} graceful-fs@4.2.11: {} guid-typescript@1.0.9: {} - has-property-descriptors@1.0.2: - dependencies: - es-define-property: 1.0.1 - has-symbols@1.1.0: {} hasown@2.0.4: @@ -5427,8 +5011,6 @@ snapshots: json-schema-typed@8.0.2: {} - json-stringify-safe@5.0.1: {} - json5@2.2.3: {} jsonfile@6.2.1: @@ -5565,10 +5147,6 @@ snapshots: dependencies: '@jridgewell/sourcemap-codec': 1.5.5 - matcher@3.0.0: - dependencies: - escape-string-regexp: 4.0.0 - math-intrinsics@1.1.0: {} media-typer@1.1.1: {} @@ -5623,8 +5201,6 @@ snapshots: object-inspect@1.13.4: {} - object-keys@1.1.1: {} - object-treeify@1.1.33: {} on-finished@2.4.1: @@ -5643,22 +5219,14 @@ snapshots: dependencies: mimic-function: 5.0.1 - onnxruntime-common@1.24.0-dev.20251116-b39e144322: {} - - onnxruntime-common@1.24.3: {} + onnxruntime-common@1.27.0: {} - onnxruntime-node@1.24.3: - dependencies: - adm-zip: 0.6.0 - global-agent: 3.0.0 - onnxruntime-common: 1.24.3 - - onnxruntime-web@1.26.0-dev.20260416-b7804b056c: + onnxruntime-web@1.27.0: dependencies: flatbuffers: 25.9.23 guid-typescript: 1.0.9 long: 5.3.2 - onnxruntime-common: 1.24.0-dev.20251116-b39e144322 + onnxruntime-common: 1.27.0 platform: 1.3.6 protobufjs: 7.6.5 @@ -5921,15 +5489,6 @@ snapshots: reusify@1.1.0: {} - roarr@2.15.4: - dependencies: - boolean: 3.2.0 - detect-node: 2.1.0 - globalthis: 1.0.4 - json-stringify-safe: 5.0.1 - semver-compare: 1.0.0 - sprintf-js: 1.1.3 - rolldown@1.2.1: dependencies: '@oxc-project/types': 0.142.0 @@ -5971,8 +5530,6 @@ snapshots: scheduler@0.27.0: {} - semver-compare@1.0.0: {} - semver@6.3.1: {} semver@7.8.5: {} @@ -5993,10 +5550,6 @@ snapshots: transitivePeerDependencies: - supports-color - serialize-error@7.0.1: - dependencies: - type-fest: 0.13.1 - serve-static@2.2.1: dependencies: encodeurl: 2.0.0 @@ -6048,39 +5601,6 @@ snapshots: - supports-color - typescript - sharp@0.35.3(@types/node@26.1.2): - dependencies: - '@img/colour': 1.1.0 - detect-libc: 2.1.2 - semver: 7.8.5 - optionalDependencies: - '@img/sharp-darwin-arm64': 0.35.3 - '@img/sharp-darwin-x64': 0.35.3 - '@img/sharp-freebsd-wasm32': 0.35.3 - '@img/sharp-libvips-darwin-arm64': 1.3.2 - '@img/sharp-libvips-darwin-x64': 1.3.2 - '@img/sharp-libvips-linux-arm': 1.3.2 - '@img/sharp-libvips-linux-arm64': 1.3.2 - '@img/sharp-libvips-linux-ppc64': 1.3.2 - '@img/sharp-libvips-linux-riscv64': 1.3.2 - '@img/sharp-libvips-linux-s390x': 1.3.2 - '@img/sharp-libvips-linux-x64': 1.3.2 - '@img/sharp-libvips-linuxmusl-arm64': 1.3.2 - '@img/sharp-libvips-linuxmusl-x64': 1.3.2 - '@img/sharp-linux-arm': 0.35.3 - '@img/sharp-linux-arm64': 0.35.3 - '@img/sharp-linux-ppc64': 0.35.3 - '@img/sharp-linux-riscv64': 0.35.3 - '@img/sharp-linux-s390x': 0.35.3 - '@img/sharp-linux-x64': 0.35.3 - '@img/sharp-linuxmusl-arm64': 0.35.3 - '@img/sharp-linuxmusl-x64': 0.35.3 - '@img/sharp-webcontainers-wasm32': 0.35.3 - '@img/sharp-win32-arm64': 0.35.3 - '@img/sharp-win32-ia32': 0.35.3 - '@img/sharp-win32-x64': 0.35.3 - '@types/node': 26.1.2 - shebang-command@2.0.0: dependencies: shebang-regex: 3.0.0 @@ -6125,8 +5645,6 @@ snapshots: source-map@0.6.1: {} - sprintf-js@1.1.3: {} - statuses@2.0.2: {} stdin-discarder@0.2.2: {} @@ -6200,8 +5718,6 @@ snapshots: tw-animate-css@1.4.0: {} - type-fest@0.13.1: {} - type-is@2.1.0: dependencies: content-type: 2.0.0