Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1025,10 +1025,12 @@ BM25 tokenizes Greek, Cyrillic, Hebrew, Arabic, and accented Latin out of the bo

### Embedding providers

Text embeddings are opt-in. With no `EMBEDDING_PROVIDER`, search stays BM25+Graph and agentmemory does not send memory, observation, or query text to an embedding provider. For Xenova on-device embeddings, install the optional package and set `EMBEDDING_PROVIDER=local`:
Text embeddings are opt-in. With no `EMBEDDING_PROVIDER`, search stays BM25+Graph and agentmemory does not send memory, observation, or query text to an embedding provider. Xenova's local embedding package is not installed by default; for on-device embeddings, install it explicitly in the same npm prefix as agentmemory and set `EMBEDDING_PROVIDER=local`:

```bash
npm install @xenova/transformers
# If agentmemory is installed globally, use the same global prefix:
# npm install -g @xenova/transformers
```

For native Ollama embeddings, run Ollama locally and set `EMBEDDING_PROVIDER=ollama`.
Expand Down
334 changes: 334 additions & 0 deletions docs/todos/2026-06-19-issue-258-install-warning-ux/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,334 @@
# Issue 258 Install Warning UX Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Stop the default npm install graph from resolving the warning-heavy Xenova/ONNX/native embedding stack while preserving explicit opt-in local embeddings.

**Architecture:** Keep the runtime lazy-import boundary intact: BM25+Graph stays the default, and `EMBEDDING_PROVIDER=local` continues to load `@xenova/transformers` only when the user has explicitly installed it. The package contract moves `@xenova/transformers` from default-installed `optionalDependencies` to optional peer metadata and removes direct ONNX optional dependencies from the root package.

**Tech Stack:** TypeScript ESM, npm package metadata, pnpm 11 lockfile, Vitest quality gates, npm pack/install smoke, OSV/Semgrep/Gitleaks security gates.

---

## Sprint Contract Carry-Forward

- Task record: `docs/todos/2026-06-19-issue-258-install-warning-ux/todo.md`
- Spec: none; source of truth is issue #258 evidence plus arena synthesis.
- GitHub PR target: `origin/main` on `https://github.com/wbugitlab1/agentmemory.git`
- Remote boundary: no fetch, pull, push, PR creation, PR merge, issue closure, publishing, or thread archival without explicit current-turn approval.
- Terminal archival contract: terminal issue closure approval must bundle issue closure plus thread archival after successful closure; terminal PR merge approval must bundle merge into `origin/main` plus thread archival after successful merge.
- Security-sensitive surfaces: dependency metadata and lockfile. OSV and Semgrep are required before final handoff or commit; staged Gitleaks is required before commit.

## Files

- Modify: `package.json`
- Move `@xenova/transformers` from `optionalDependencies` to `peerDependencies`.
- Add `peerDependenciesMeta["@xenova/transformers"].optional = true`.
- Remove `onnxruntime-node` and `onnxruntime-web` from root `optionalDependencies`.
- Leave `@node-rs/jieba` and `tiny-segmenter` in `optionalDependencies` for this issue.
- Modify: `pnpm-lock.yaml`
- Regenerate after package metadata change with scripts disabled.
- Expected default graph shrink: remove root optional entries and unused `@xenova/transformers`, `onnxruntime-*`, `sharp`, `prebuild-install` transitive entries if no other importer needs them.
- Modify: `test/quality-gates.test.ts`
- Extend the current Anthropic install-surface guard into a default-install guard for the ML/native warning stack.
- Assert optional peer metadata exists for `@xenova/transformers`.
- Modify: `README.md`
- Clarify that the Xenova package is not installed by default and is required only for explicit local embedding opt-in.
- No source behavior files expected.

## Task 1: Add Red Package Guard

**Files:**
- Modify: `test/quality-gates.test.ts`

- [ ] **Step 1: Extend the package metadata type**

Update the `PackageJson` type only if needed. It already includes `optionalDependencies`, `peerDependencies`, `peerDependenciesMeta`, `bundledDependencies`, and `bundleDependencies`, so no type edit should be required.

- [ ] **Step 2: Replace the narrow Anthropic guard with a default-install guard**

In `test/quality-gates.test.ts`, replace the existing test named `does not auto-install Anthropic packages in the root package` with:

```ts
it("keeps opt-in provider packages out of the default install graph", () => {
const pkg = readPackageJson();
const autoInstallSurfaces = [
pkg.dependencies ?? {},
pkg.optionalDependencies ?? {},
];
const bundled = [
...(pkg.bundledDependencies ?? []),
...(pkg.bundleDependencies ?? []),
];
const defaultInstallPackages = [
"@anthropic-ai/sdk",
"@anthropic-ai/claude-agent-sdk",
"@xenova/transformers",
"onnxruntime-node",
"onnxruntime-web",
];

for (const deps of autoInstallSurfaces) {
for (const packageName of defaultInstallPackages) {
expect(deps[packageName]).toBeUndefined();
}
}
for (const packageName of defaultInstallPackages) {
expect(bundled).not.toContain(packageName);
}

expect(pkg.peerDependencies?.["@anthropic-ai/claude-agent-sdk"]).toBe("^0.3.142");
expect(pkg.peerDependenciesMeta?.["@anthropic-ai/claude-agent-sdk"]?.optional).toBe(true);
expect(pkg.peerDependencies?.["@xenova/transformers"]).toBe("^2.17.2");
expect(pkg.peerDependenciesMeta?.["@xenova/transformers"]?.optional).toBe(true);
});
```

- [ ] **Step 3: Run the focused test and confirm RED**

Run:

```bash
corepack pnpm exec vitest run test/quality-gates.test.ts --exclude test/integration.test.ts
```

Expected before implementation: fail because `@xenova/transformers`, `onnxruntime-node`, and `onnxruntime-web` are still in `optionalDependencies`, and the `@xenova/transformers` optional peer is missing.

If pnpm ignored-build hardening blocks before Vitest starts, run:

```bash
corepack pnpm install --frozen-lockfile --ignore-scripts
corepack pnpm exec vitest run test/quality-gates.test.ts --exclude test/integration.test.ts
```

Record the exact blocker or RED failure in `todo.md`.

## Task 2: Update Package Metadata And Lockfile

**Files:**
- Modify: `package.json`
- Modify: `pnpm-lock.yaml`

- [ ] **Step 1: Edit `package.json`**

Change the relevant dependency metadata to this shape:

```json
"peerDependencies": {
"@anthropic-ai/claude-agent-sdk": "^0.3.142",
"@xenova/transformers": "^2.17.2"
},
"peerDependenciesMeta": {
"@anthropic-ai/claude-agent-sdk": {
"optional": true
},
"@xenova/transformers": {
"optional": true
}
},
"optionalDependencies": {
"@node-rs/jieba": "^2.0.1",
"tiny-segmenter": "^0.2.0"
},
```

Do not change dependency versions. Do not move `@node-rs/jieba` or `tiny-segmenter` in this issue.

- [ ] **Step 2: Regenerate lockfile with scripts disabled**

Run:

```bash
corepack pnpm install --lockfile-only --ignore-scripts
```

Expected: command exits 0 and updates only lockfile metadata needed for the manifest change. If pnpm creates any build-approval scaffolding or unrelated config churn, inspect it and remove only task-caused unrelated churn.

- [ ] **Step 3: Verify lockfile no longer contains the removed ML/native packages**

Run:

```bash
rg -n "@xenova/transformers|onnxruntime-node|onnxruntime-web|sharp@0\\.32\\.6|prebuild-install" pnpm-lock.yaml package.json
```

Expected: `package.json` contains `@xenova/transformers` only in optional peer metadata; `pnpm-lock.yaml` should not contain the removed packages unless another importer still needs them. If any remain, inspect the importer path before changing more files.

- [ ] **Step 4: Run the focused package guard and confirm GREEN**

Run:

```bash
corepack pnpm exec vitest run test/quality-gates.test.ts --exclude test/integration.test.ts
```

Expected: `test/quality-gates.test.ts` passes.

## Task 3: Update Minimal README Wording

**Files:**
- Modify: `README.md`

- [ ] **Step 1: Clarify opt-in local embedding install**

In the `### Embedding providers` section, change this sentence:

```md
Text embeddings are opt-in. With no `EMBEDDING_PROVIDER`, search stays BM25+Graph and agentmemory does not send memory, observation, or query text to an embedding provider. For Xenova on-device embeddings, install the optional package and set `EMBEDDING_PROVIDER=local`:
```

to:

```md
Text embeddings are opt-in. With no `EMBEDDING_PROVIDER`, search stays BM25+Graph and agentmemory does not send memory, observation, or query text to an embedding provider. Xenova's local embedding package is not installed by default; for on-device embeddings, install it explicitly and set `EMBEDDING_PROVIDER=local`:
```

- [ ] **Step 2: Confirm no stale install wording claims Xenova is default-installed**

Run:

```bash
rg -n "Xenova|@xenova/transformers|optional package|installed by default|EMBEDDING_PROVIDER=local" README.md INSTALL_FOR_AGENTS.md .env.example
```

Expected: remaining references either document explicit opt-in or runtime configuration. Do not rewrite unrelated docs.

## Task 4: Package Install Smoke

**Files:**
- No persistent files expected unless a temporary script is useful during execution. Prefer one-off commands and record evidence in `todo.md`.

- [ ] **Step 1: Build the package before packing**

Run:

```bash
corepack pnpm run build
```

Expected: build exits 0.

- [ ] **Step 2: Run dry-run package preview**

Run:

```bash
npm pack --dry-run --json
```

Expected: exits 0 and previews expected package files.

- [ ] **Step 3: Run sanitized local tarball consumer install smoke**

Run:

```bash
tmpdir="$(mktemp -d /tmp/agentmemory-258-pack-smoke.XXXXXX)"
mkdir -p "$tmpdir/home" "$tmpdir/cache" "$tmpdir/project" "$tmpdir/pack"
HOME="$tmpdir/home" npm_config_cache="$tmpdir/cache" npm_config_userconfig="$tmpdir/missing-npmrc" npm pack --json --pack-destination "$tmpdir/pack" > "$tmpdir/pack.json"
tarball="$(jq -r '.[0].filename' "$tmpdir/pack.json")"
cd "$tmpdir/project"
HOME="$tmpdir/home" npm_config_cache="$tmpdir/cache" npm_config_userconfig="$tmpdir/missing-npmrc" npm install --package-lock-only --ignore-scripts --no-audit --no-fund "$tmpdir/pack/$tarball" 2>&1 | tee "$tmpdir/install.log"
jq -r '.packages | keys[]' package-lock.json > "$tmpdir/package-keys.txt"
```

Then run:

```bash
for name in '@anthropic-ai/sdk' '@anthropic-ai/claude-agent-sdk' '@xenova/transformers' 'onnxruntime-node' 'onnxruntime-web' 'sharp' 'prebuild-install' 'node-domexception'; do
if rg -q "(^|/)node_modules/${name}$" "$tmpdir/package-keys.txt"; then
echo "default install resolved ${name}" >&2
exit 1
fi
done
if rg -n "ERESOLVE overriding peer dependency|deprecated prebuild-install|deprecated node-domexception" "$tmpdir/install.log"; then
echo "install warning signature still present" >&2
exit 1
fi
printf 'SMOKE_DIR=%s\n' "$tmpdir/project"
```

Expected: no forbidden package entries and no reported warning signatures.

## Task 5: Final Verification And Review Notes

**Files:**
- Modify: `docs/todos/2026-06-19-issue-258-install-warning-ux/todo.md`

- [ ] **Step 1: Run targeted tests**

Run:

```bash
corepack pnpm exec vitest run test/quality-gates.test.ts test/build-package-contract.test.ts test/embedding-provider.test.ts test/reranker.test.ts --exclude test/integration.test.ts
```

Expected: all selected tests pass.

- [ ] **Step 2: Run lint**

Run:

```bash
corepack pnpm run lint
```

Expected: lint exits 0.

- [ ] **Step 3: Run dependency/security gates**

Run:

```bash
osv-scanner scan source .
semgrep scan --config p/default --error --metrics=off .
```

Expected: both commands exit 0 or findings/blockers are recorded and handled before handoff.

- [ ] **Step 4: Run diff check**

Run:

```bash
git diff --check
```

Expected: no whitespace errors.

- [ ] **Step 5: Update the task record**

Update `docs/todos/2026-06-19-issue-258-install-warning-ux/todo.md` with:

- implementation summary;
- dependency-intake decisions;
- Feature / Verification Matrix final statuses and evidence;
- verification commands and results;
- caveats and residual risks;
- remote status, including that no push, PR creation, issue closure, PR merge, publish, or thread archival ran unless separately approved.

- [ ] **Step 6: Stage only task-owned files and run staged Gitleaks if committing**

If preparing a local commit, stage only:

```bash
git add package.json pnpm-lock.yaml test/quality-gates.test.ts README.md docs/todos/2026-06-19-issue-258-install-warning-ux/todo.md docs/todos/2026-06-19-issue-258-install-warning-ux/plan.md
gitleaks protect --staged --redact
```

Expected: staged Gitleaks exits 0. Commit only after staged content is verified.

## Self-Review

Spec coverage:
- Validity evidence is already recorded in `todo.md`.
- The selected arena scope is implemented through package metadata and quality gates.
- The install-smoke command checks the original warning packages, `node-domexception`, and the remaining `prebuild-install` path through installed package entries plus npm output.
- Remote and archival approvals are preserved as explicit boundaries.

Placeholder scan:
- No `TBD`, `TODO`, or "similar to" placeholders remain in this plan.

Type consistency:
- `PackageJson` already contains the metadata fields used by the new quality gate.
- Test names and paths match existing repository files.
Loading
Loading