ContextCake · siracusa5 · Jul 30, 2026 · Jul 30, 2026 · Jul 30, 2026 · Jul 30, 2026
diff --git a/.github/workflows/app-release.yml b/.github/workflows/app-release.yml
@@ -49,6 +49,19 @@ jobs:
             exit 1
           fi
 
+      - name: Verify no release gate is open
+        run: |
+          # A gate declared only in a GitHub issue is enforceable only by
+          # someone remembering it, which is exactly how the hosted-OAuth
+          # checks in #44 came to be deferred through three releases. The
+          # gates live in docs/release-gates.md; an open one stops the build.
+          if grep -q '^## Open:' docs/release-gates.md; then
+            echo "::error::Release gate open in docs/release-gates.md:"
+            grep '^## Open:' docs/release-gates.md >&2
+            echo "Close the gate (move it under '## Closed:' with its evidence) or, to ship anyway, say so explicitly in that file." >&2
+            exit 1
+          fi
+
       - name: Setup Node
         uses: actions/setup-node@v7
         with:

diff --git a/CLAUDE.md b/CLAUDE.md
@@ -3,10 +3,19 @@
 ## Commands
 
 ```bash
-# Run all tests
+# Run all tests (the chain lives in package.json "test" — read it there rather
+# than mirroring it here, which is how the two drifted apart before)
 npm test
-# or directly (mirrors the npm test chain):
-bash packages/core/tests/smoke-test.sh && bash packages/core/tests/resolver-test.sh && bash packages/core/tests/source-test.sh && bash packages/core/tests/files-source-test.sh && bash packages/core/tests/github-source-test.sh && bash packages/core/tests/pack-test.sh && bash packages/core/tests/git-sync-test.sh && bash packages/core/tests/capture-test.sh && bash packages/core/tests/team-sync-mcp-test.sh && bash packages/core/tests/playground-test.sh && bash packages/core/tests/service-test.sh && bash packages/core/tests/mcp-respawn-test.sh && bash packages/core/tests/setup-robustness-test.sh
+
+# Run one suite while iterating
+bash packages/core/tests/resolver-test.sh
+node --test packages/core/tests/search.test.mjs
+
+# Retrieval eval — scores search.mjs against the golden question set.
+# Part of npm test: a ranking change that loses recall fails the build.
+npm run eval
+node packages/core/eval/run.mjs --verbose                    # show what each miss returned instead
+node packages/core/eval/run.mjs --record --label "why"       # accept a new number, with its reason
 
 # Run the MCP server (cascade mode)
 node mcp-server.mjs --manifest layers.json
@@ -61,6 +70,7 @@ See `docs/architecture/README.md` for the full design. Short version:
 
 - **Storage is federated** — each layer is a `source` behind a uniform adapter: an `okf-local` bundle, a plain `files` folder, a remote `github` repository, or an `mcp` foreign graph translated into OKF at read time.
 - **Reading is unified** — `resolver.mjs` stitches the sources into one effective OKF concept at read time.
+- **Finding is ranked** — `search.mjs` (BM25F over Porter-stemmed tokens) decides which concept an agent gets to resolve in the first place. Everything the cascade is good at is downstream of it.
 - **Layer precedence** — Personal (3) > Team (2) > Company (0). Higher wins per section. Levels are configurable per layer in the manifest.
 - **Section/field merge** — not whole-document replacement. A higher layer speaks to what it knows; the rest is inherited. Where layers disagree, the primary value carries per-section `conflicts[]` (dissenting layer + date) — surfaced, not hidden.
 - **MCP server** — `mcp-server.mjs` exposes the resolved graph to AI agents (search, read_file, list_concepts, get_links).
@@ -71,6 +81,9 @@ Key files:
 | File | Role |
 |------|------|
 | `packages/core/src/resolver.mjs` | Core cascade engine: section merge, precedence, provenance, conflict surfacing |
+| `packages/core/src/search.mjs` | Ranking behind the `search` and `find_captures` tools: BM25F over stemmed tokens, per-field boosts and length normalization, 7-day recency half-life for captures. Importable and side-effect-free precisely so the eval can score it |
+| `packages/core/src/stem.mjs` | Porter stemmer (1980). Deliberately the published algorithm, not a hand-tuned suffix list — see the note in the file |
+| `packages/core/eval/` | Retrieval eval: committed 3-layer corpus, golden question set, runner, and `baseline.json` with the score history |
 | `packages/core/src/service.mjs` | Embeddable HTTP service: read API, background index, sources CRUD, settings, file APIs, console mount |
 | `packages/core/src/settings.mjs` | User-configurable engine limits (manifest > env > default) + validation and the UI catalog |
 | `packages/core/src/layer-files.mjs` | Layer file explorer/editor APIs (`/api/files`, `/api/file`, `/api/section`), sandboxed to layer roots |
@@ -121,4 +134,5 @@ Key files:
 - **Aggregate reads come from a background index and are often partial.** `/api/graph` and `/api/resolve-all` answer from whatever is indexed right now and report `indexing` / `indexingSources`; clients render what they have and poll. Any assertion about completeness (tests, scripts) must pass `?wait=<ms>`. `/api/resolve` deliberately reads one concept live. An index entry is keyed by its layer config + settings, so adding a source never re-indexes the existing ones.
 - **The indexing limits are user settings, not env-only** (`settings.mjs`, `GET`/`PATCH /api/settings`, Settings → Indexing in the console). Precedence is manifest > env > default — the manifest has to win or the settings UI would silently do nothing. Env vars (`CONTEXTCAKE_MAX_DOC_FILES`, `CONTEXTCAKE_MAX_SCAN_ENTRIES`, `CONTEXTCAKE_SOURCE_BUDGET_MS`) remain the headless/CI fallback.
 - **Add-source validation is deliberately cheap**: only "folder is missing" and "that's a file" fail the form. A too-big folder is a normal thing to add — it becomes a visible source error after indexing, with a pointer at Settings. Don't reintroduce a full walk on the add path. MCP sources still probe (`tools/list`) at add time, which is bounded and catches a wrong command.
+- **Retrieval is measured, not asserted.** `npm test` ends with the eval; a ranking change that loses recall fails the build with `RETRIEVAL REGRESSED`. If a change is a deliberate trade, re-record with `--record --label "<why>"` — the superseded numbers stay in `baseline.json`'s history so the trade is visible later. Do not tune the stemmer or the field boosts against the golden questions: the set is small enough to overfit in an afternoon, and a scorer fitted to its own eval measures nothing. Add questions first, then tune.
 - **Layer file APIs live in the engine, not the playground** (`layer-files.mjs`), so the desktop app can browse and edit context files. They cover `files`-kind layers too — the playground's old copy only mapped `okf-local` roots, which made markdown folders invisible in the editor.
diff --git a/docs/release-gates.md b/docs/release-gates.md
@@ -0,0 +1,92 @@
+# Release gates
+
+A gate is a check that must pass before a release is published, recorded here so
+the answer to "was this validated?" is a file rather than a memory.
+
+**This file is enforced.** `app-release.yml` refuses to publish while any
+`## Open:` section remains below. Closing a gate means moving it under
+`## Closed:` with its evidence — not deleting it. To ship with a gate knowingly
+unmet, say so in the file; the point is that the decision is written down.
+
+## Open: hosted OAuth and settings sync (#44)
+
+**Status: breached.** PR #38 merged 2026-07-15 deferring three hosted acceptance
+checks to #44 with the note that they "remain required before publishing the
+desktop account-sync release." Three releases have shipped since:
+
+| Release | Date | Account sync reachable? |
+|---|---|---|
+| 0.1.0 | 2026-07-17 | yes |
+| 0.2.0 | 2026-07-20 | yes |
+| 0.3.0 | 2026-07-29 | yes |
+
+Reachable, not merely present: `apps/desktop/scripts/generate-supabase-config.mjs`
+throws unless `SUPABASE_URL` and `SUPABASE_ANON_KEY` are set, `app-release.yml`
+supplies both from repository secrets, and `AccountPanel` renders unconditionally
+in Settings with Sign in, Sign out, and Delete account. Every published build can
+sign a user into the hosted project.
+
+This is a process failure, not a defect report. Nothing here says the feature is
+broken — it says nobody has confirmed it works on the artifact users download.
+
+### What CI already discharges
+
+These sub-checks are covered by automated tests on every PR (`apps/desktop/test/`,
+run by the `desktop` job in `ci.yml`) and do not need to be repeated by hand:
+
+| Sub-check | Covered by |
+|---|---|
+| Callback `state` validation; only an encrypted session is written | `auth.test.mjs` — OAuth IPC smoke |
+| Cancel and retry cannot orphan or hijack an in-flight attempt | `auth.test.mjs` — duplicate sign-in, late/stale refresh, sign-out races |
+| Sign-out clears the encrypted local session, including offline | `auth.test.mjs` — sign-out with Supabase offline |
+| Paths, MCP commands/args, cache dirs and credential references never sync as runnable config | `settings-sync.test.mjs` — scrub, allowlist, quarantine, path-shaped keys |
+| A second client converges: tombstones, account switch, last-write-wins | `settings-sync.test.mjs` — dirty tombstone, shadow discard, remote-row precedence |
+
+### What still needs a human
+
+Three things cannot be simulated: a **packaged** app, a **second physical
+machine**, and the **hosted** database. Each needs John — the OAuth flow requires
+entering real credentials, which is not something an agent should do on his
+behalf.
+
+**1. Packaged OAuth and Keychain persistence**
+
+```bash
+cd apps/desktop && SUPABASE_URL=<hosted> SUPABASE_ANON_KEY=<publishable> npm run dist
+```
+
+Then, on the installed app: complete the GitHub browser flow, confirm
+`contextcake://auth/callback` returns to the app, quit and relaunch, and confirm
+the session survived. Then cancel a sign-in mid-flow and confirm nothing persists.
+
+**2. Second-machine settings roundtrip**
+
+Sign in to the same account on a second Mac or macOS user. Change a preference on
+each side and confirm both directions converge. Confirm the second machine's
+sources arrive as non-runnable pending metadata, not as executable configuration.
+
+**3. Hosted account deletion**
+
+With a disposable account, invoke Delete account. Confirm in the hosted project
+that the Auth user and the `user_settings` row are both gone, that local session
+state is cleared, and that the app still works signed out.
+
+### Evidence to record when closing
+
+Packaged version and commit SHA · macOS versions and devices used · redacted
+screenshots or logs per check · confirmation the hosted records were removed.
+
+### Why it slipped
+
+Nothing connected the open issue to the release workflow. A gate declared only in
+an issue is a gate only a person can enforce, and the person was busy shipping.
+
+Fixed by the "Verify no release gate is open" step in `app-release.yml`, added
+alongside this file: tagging `app-v0.3.1` now fails while the section above still
+says `## Open:`. That is the intended behavior, not an obstacle to route around —
+the next release is blocked until these three checks are done or the decision to
+ship without them is written here.
+
+## Closed:
+
+*(none yet — the first gate to close will be #44, above)*
diff --git a/package.json b/package.json
@@ -4,7 +4,8 @@
   "description": "Federated team knowledge with cascading layer precedence — OKF-compatible, MCP-ready.",
   "type": "module",
   "scripts": {
-    "test": "bash packages/core/tests/smoke-test.sh && bash packages/core/tests/resolver-test.sh && bash packages/core/tests/source-test.sh && bash packages/core/tests/files-source-test.sh && bash packages/core/tests/github-source-test.sh && node --test packages/core/tests/manifest.test.mjs && bash packages/core/tests/profile-runtime-test.sh && bash packages/core/tests/pack-test.sh && bash packages/core/tests/git-sync-test.sh && bash packages/core/tests/capture-test.sh && bash packages/core/tests/team-sync-mcp-test.sh && bash packages/core/tests/playground-test.sh && bash packages/core/tests/service-test.sh && bash packages/core/tests/mcp-respawn-test.sh && bash packages/core/tests/setup-robustness-test.sh",
+    "test": "bash packages/core/tests/smoke-test.sh && bash packages/core/tests/resolver-test.sh && bash packages/core/tests/source-test.sh && bash packages/core/tests/files-source-test.sh && bash packages/core/tests/github-source-test.sh && node --test packages/core/tests/manifest.test.mjs && node --test packages/core/tests/search.test.mjs && bash packages/core/tests/profile-runtime-test.sh && bash packages/core/tests/pack-test.sh && bash packages/core/tests/git-sync-test.sh && bash packages/core/tests/capture-test.sh && bash packages/core/tests/team-sync-mcp-test.sh && bash packages/core/tests/playground-test.sh && bash packages/core/tests/service-test.sh && bash packages/core/tests/mcp-respawn-test.sh && bash packages/core/tests/setup-robustness-test.sh && npm run eval",
+    "eval": "node packages/core/eval/run.mjs",
     "mcp": "node mcp-server.mjs",
     "playground": "node apps/playground/server.mjs",
     "console:live": "npm --prefix apps/console run build:live && node apps/playground/server.mjs --console apps/console/dist",

diff --git a/packages/core/eval/README.md b/packages/core/eval/README.md
@@ -0,0 +1,75 @@
+# Retrieval eval
+
+Every other suite in this repo tests mechanism: does the merge pick the right
+section, does the walk stay bounded, does the token guard hold. None of them can
+answer the question the product actually lives or dies on — **would an agent find
+the right concept from a question a person actually typed?**
+
+This measures that.
+
+```bash
+npm run eval                                              # report + regression gate
+node packages/core/eval/run.mjs --verbose                 # show what each miss returned instead
+node packages/core/eval/run.mjs --json                    # machine-readable
+node packages/core/eval/run.mjs --record --label "why"    # accept a new number
+```
+
+## What is here
+
+| Path | Role |
+|------|------|
+| `corpus/` | Three committed OKF layers — company (0), team (2), personal (3) — with deliberate cross-layer overrides |
+| `questions.json` | 38 natural-language questions, each with the concept ids that answer it and a `probes` note saying why the question exists |
+| `manifest.json` | Points the engine at the corpus; also usable by hand with `resolver.mjs` or `mcp-server.mjs` |
+| `run.mjs` | Runner and regression gate |
+| `baseline.json` | Current metrics plus the history of every superseded number |
+
+## Metrics
+
+- **recall@1** — the right concept is the first hit.
+- **recall@5** — it is in the top five, which is realistically what an agent reads.
+- **mrr** — mean reciprocal rank; moves when a hit is merely demoted rather than lost.
+- **conflict** — for questions where layers disagree, resolving the answer still
+  surfaces the dissenting layers. Finding the concept is only half the claim: if
+  the disagreement does not survive retrieval, the answer is just the top layer's
+  opinion wearing a provenance badge.
+
+## Results so far
+
+| Scorer | recall@1 | recall@5 | mrr | conflict |
+|---|---|---|---|---|
+| Substring occurrence counting | 0.263 | 0.500 | 0.348 | 1.000 |
+| BM25F over Porter-stemmed tokens | 0.895 | 1.000 | 0.947 | 1.000 |
+
+The old scorer counted `indexOf` hits with fixed field weights. Three things
+were wrong with it, and the eval separated them:
+
+1. **Substring matching, not token matching.** The query word `I` matched the
+   `i` inside *identifier*, *in*, and *with*. Natural questions carry several
+   such words, so the longest document usually won.
+2. **No inverse document frequency.** A word in every document counted as much
+   as a word in one.
+3. **No length normalization.** Repetition beat precision.
+
+`conflict` was already 1.000 before the rewrite. That is the honest read on the
+engine: the cascade and the conflict surfacing worked; retrieval was the broken
+half, and it was broken badly enough to hide the working half.
+
+## Adding questions
+
+Write the question the way someone would actually type it, *then* see what the
+ranker does. A question written after looking at the corpus tends to reuse the
+document's own vocabulary, which grades the ranker on the one case it is
+guaranteed to pass.
+
+Fill in `probes` with what the question is testing — a morphological variant, a
+distractor term, a genuine synonym gap. When a question flips from hit to miss,
+that note is what tells you whether the change was real.
+
+## Known ceilings
+
+Some questions cannot be won by lexical ranking at any amount of tuning. `q07`
+("how long do we keep logs?") asks with *keep* against a corpus that says
+*retained*; no stemmer bridges that. Those questions stay in the set on purpose
+— they are the honest measure of what a lexical ranker leaves on the table, and
+the number to beat if embeddings ever earn their dependency.
diff --git a/packages/core/eval/baseline.json b/packages/core/eval/baseline.json
@@ -0,0 +1,18 @@
+{
+  "label": "BM25F over Porter-stemmed tokens; best-layer aggregation",
+  "questions": 38,
+  "recall@1": 0.895,
+  "recall@5": 1,
+  "mrr": 0.947,
+  "conflict": 1,
+  "history": [
+    {
+      "label": "substring occurrence counting (pre-BM25)",
+      "questions": 38,
+      "recall@1": 0.263,
+      "recall@5": 0.5,
+      "mrr": 0.348,
+      "conflict": 1
+    }
+  ]
+}
diff --git a/packages/core/eval/corpus/company/decisions/data-storage.md b/packages/core/eval/corpus/company/decisions/data-storage.md
@@ -0,0 +1,23 @@
+---
+type: decision
+title: Primary datastore standard
+description: Which database a service should use for transactional state.
+tags: [database, storage, standard]
+updated: 2026-05-20
+---
+
+## Transactional Store {#transactional}
+
+PostgreSQL 16 is the default database for service state. Managed instances are provisioned through the platform portal; teams do not run their own.
+
+## Caching {#caching}
+
+Redis for ephemeral cache only. Nothing that cannot be rebuilt from the primary datastore may live in Redis.
+
+## Analytics {#analytics}
+
+Analytical queries run against the warehouse, never against a service's production database. Read replicas exist for operational debugging, not reporting.
+
+## Migrations {#migrations}
+
+Schema changes ship as forward-only migrations reviewed by the owning team. Destructive column drops require a two-release deprecation window.
diff --git a/packages/core/eval/corpus/company/decisions/service-stack.md b/packages/core/eval/corpus/company/decisions/service-stack.md
@@ -0,0 +1,23 @@
+---
+type: decision
+title: Service stack standard
+description: The org-wide default language, framework, and runtime for new services.
+tags: [platform, language, standard]
+updated: 2026-06-01
+---
+
+## Language and Framework {#language}
+
+Spring Boot with Java 21. This is the standard for all new services org-wide. Teams that need a different runtime must file an exemption with the platform group and re-confirm it annually.
+
+## Build and Packaging {#build}
+
+Gradle with the shared convention plugin. Services publish an OCI image to the internal registry; no team maintains its own base image.
+
+## Runtime Targets {#runtime}
+
+Services run on the shared Kubernetes platform. Bare EC2 and per-team clusters were retired in 2025.
+
+## Enforcement {#enforcement}
+
+New services must pass the Java 21 conformance check in company CI. Existing exemptions expire twelve months after they are granted.
diff --git a/packages/core/eval/corpus/company/index.md b/packages/core/eval/corpus/company/index.md
@@ -0,0 +1,18 @@
+---
+type: index
+title: Company knowledge
+description: Organization-wide canonical standards, policies, and runbooks.
+---
+
+# Company layer
+
+- [Service stack standard](decisions/service-stack.md)
+- [Primary datastore standard](decisions/data-storage.md)
+- [Security baseline](policies/security-baseline.md)
+- [Data retention policy](policies/data-retention.md)
+- [Incident response](policies/incident-response.md)
+- [Access control](policies/access-control.md)
+- [Standard deployment process](runbooks/deploy-standard.md)
+- [Code review expectations](standards/code-review.md)
+- [Observability standard](standards/observability.md)
+- [API design conventions](standards/api-design.md)