feat: retrieval quality and context delivery (v1.37.0)#146
Merged
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dicate and staleness barrier Ships Task N1 (t_b0c9d0a3) and carries seam 2 (the index-admission predicate at the walker/indexer touch point). What shipped: - Exact-state lane (src/core/brain/exact-state.ts): a structured overwrite-only operational-state store keyed by aspect at Brain/state/<aspect>.md. Each write replaces the aspect's canonical value with no history accumulation (one file per aspect). Over-budget values are rejected with a typed ExactStateError, never truncated. - Index-admission predicate / seam 2 (src/core/vault-scope/index-admission.ts): admitToIndex(relPath) is the single decision point for index membership. It DEFAULTS TO ADMIT and excludes only the exact-state lane, so no existing non-lane content ever leaves the index. Consulted by the walker for both directory descent and file yield; N2 will consult it for scope-aware indexing. - Retrieval-time staleness barrier (applyExactStateBarrier in result-filters.ts, wired into search.ts assemble()): drops any exact-state lane artifact a stale/older index may still hold, from any retrieval source, before any later phase can surface it. Path-based and I/O-free; returns the same array when the pool has no lane rows, so a vault that never used the lane is byte-identical. - CLI surface: o2b brain state set|get|list|clear --aspect (usage errors exit 2 via usageError; --json envelopes follow verb conventions). Acceptance evidence: - exact-state.test.ts: overwrite replaces value, exactly one file, no history sidecars; list is sorted; clear reports prior existence; over-budget rejected. - index-admission.test.ts: defaults to admit; existing Brain content (preferences, sources, pinned.md) still admitted (regression); lane dir and files excluded; name-prefix siblings (Brain/stateful, Brain/state-notes.md) NOT excluded. - exact-state-barrier.test.ts: barrier drops lane results, no-op same reference otherwise; lane never enters the index while note.md and Brain/preferences survive (regression); lane text never resurfaces through recall. Deviations from plan.md: - Lane placed at Brain/state/ (new dir) rather than modifying DEFAULT_VAULT_IGNORE_PATHS in vault-scope/defaults.ts: the design's chosen mechanism is the admission predicate (defaults to admit, excludes only the lane), not the user-facing ignore config, so defaults.ts is left unchanged. - pinned.ts is left unchanged: the lane is an additive structured sibling; touching pinned's overwrite-only path would risk the byte-identical contract of existing pinned behaviour. - Surface is CLI-only (no new MCP tool) to keep the MCP tool-count baseline at 107 per the extend-not-add guidance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…search scoping
Ships Task N2 (t_37c05a34) and carries seam 1 (the ONE composite-key
module for source-identity RRF/dedup keys and the owner/session/project
scope vocabulary). N3 consumes rrfKey / canonicalSourceSetKey from here.
What shipped:
- Composite-key module / seam 1 (src/core/scope-key.ts): owns the
owner/session/project scope vocabulary (scopeFromFrontmatter,
normalizeScopeFilter, compositeScopeKey, scopeAxisReachable) AND the
federated source-identity keys (rrfKey, canonicalSourceSetKey) so RRF/
dedup keying has exactly one implementation. Owner reuses the v1.6
agent-scope normalization; session/project reuse the session-scope slug
rule. Language-agnostic: axes are opaque identifiers, no word lists.
- Per-scope dedup (page-dedup.ts): the dedup key now folds in the page's
composite scope, so identical rule text under different owners/sessions/
projects is no longer collapsed. Additive keying: a scopeless page
yields an empty scope key, so its dedup key is byte-identical to the
pre-scope world - existing global dedup state is untouched and a rerun
over old rows never re-collapses.
- Optional search scope filter (SearchOptions.scope; applyScopeFilter in
result-filters.ts, wired into search.ts assemble()): a query may opt
into session/project filtering on top of the existing owner axis
(agentScope). Per axis, a null request does not filter and an unscoped
page is shared, so an omitted / empty ({}) filter is byte-identical.
The filter joins hasFrontmatterFilter so the overfetch already handles
dropped rows.
- Query-cache scoping (query-cache.ts): the composite scope joins the
cache key (normalized, dropped when absent) so a scoped and an unscoped
query never share a cached row, and a scopeless query keys identically
to a pre-scope cached row.
- MCP: brain_search gains optional session_scope / project_scope params
(descriptions <=160 chars); tool count unchanged at 107.
Acceptance evidence:
- scope-key.test.ts: axis normalization; empty composite key for a
scopeless page; distinct scopes -> distinct keys, same scope -> same
key, cross-axis same-slug never collides; scopeAxisReachable opt-in
semantics; rrfKey source identity (cross-origin chunk ids never
collide); canonicalSourceSetKey order-independent + deduping.
- page-dedup-scope.test.ts: different owners/sessions NOT collapsed;
identical text within one scope still collapses; scopeless identical
pages still collapse (additive / byte-identical).
- scope-filter.test.ts + search-scope-filter.test.ts (MCP): a session
filter returns that scope plus unscoped pages; omitting the filter is
byte-identical; project axis independent of session.
Deviations from plan.md:
- The search scope filter covers the two NEW axes (session, project); the
owner axis keeps its existing agentScope path (agent-scope.ts) rather
than being re-routed, to avoid two owner code paths and preserve the
byte-identical owner-filter behaviour. The composite module still owns
the whole vocabulary (dedup folds in all three axes).
- rrfKey / canonicalSourceSetKey ride here per the seam contract and are
fully unit-tested; their production wiring lands in N3 (t_09b7ccea).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dened keys Ships Task N3 (t_09b7ccea): a fourth RRF arm for relationship-shaped queries, behind a mode flag with a byte-identical default, consuming the seam-1 source-identity keys from N2 (src/core/scope-key.ts). What shipped: - Relational-query parser (relational-query.ts): detects a relationship shape STRUCTURALLY and vocabulary-driven - a wikilink seed plus a query token that is a SUBSET of the edge-type vocabulary (schema-pack link_types unioned with the default relation vocabulary). No natural-language word list anywhere; a non-relational query returns null. - relationalFanout (relational-fanout.ts): generalizes typed-edge traversal to a seed array with bounded depth (default 2), aggregating per reached node its min hop distance, edge richness, and via-link-types into a deterministic ranked node list (nearer, then richer, then doc id). - Fourth RRF arm: rrfFuse gains an optional relational lane (byte-identical when empty); rankResults feeds it and admits relational-only chunks as candidates (searchType "link") so a genuinely related node can surface. search.ts engages the arm ONLY in rrf fusion, when enabled (config.recall.relationalArmEnabled or the opts.relationalArm override), for a relationship-shaped query: it resolves seeds to document ids, fans out, maps nodes to representative chunks (deduped by the shared rrfKey source identity), and attributes each surfaced node with a "relational: via <types> (<n> hop[s])" reason. - Federation hardening: the relational lane dedups by rrfKey (source identity from the N2 module); the query cache scopes by canonicalSourceSetKey (folded into the config fingerprint) and by the relationalArm override, so a cache row is never served across a different origin set or arm setting. - Config: search_relational_arm_enabled / OPEN_SECOND_BRAIN_SEARCH_RELATIONAL_ARM (default false), mirroring the fusionMode config-flag precedent. Acceptance evidence: - relational-query.test.ts / relational-fanout.test.ts: structural detection with subset validation; depth bound; edge-type restriction; richness aggregation; seeds excluded. - relational-arm.test.ts: a relationship query surfaces the related node with attribution (arm on, rrf); depth-2 reaches a two-hop node when its edge type is named; arm off (default) does not surface it; a non-relational query is byte-identical (paths, scores, reasons) between arm on and off - the required default-off regression. Deviations from plan.md: - The arm is a config/env mode flag (plus a per-query opts override) rather than an MCP param, matching the existing fusionMode precedent (also config-only), so the MCP tool count and surface are unchanged at 107. - Candidate.searchType widened to include "link" (already valid on BrainSearchResult) so relational-only nodes carry a truthful type. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…l hardening
Task R1 (t_7b96f242). Adds a structural summary-search router to the query
plan and names the summary surface explicitly where the repo documents
search surfaces.
What buildQueryPlan ALREADY routed for summary-shaped questions (recorded
per the task): nothing at the surface level. buildQueryPlan only classified
a structural QueryIntent (neutral | exact | entity | broad) and emitted a
bounded weight profile plus the cache planHash. A summary-shaped question
was therefore only re-weighted by generic intent - a source-targeted
wikilink lookup landed on `entity`, a long exploratory summary question on
`broad` - and no code path ever selected a distinct retrieval surface. There
was no summary-surface routing to duplicate, so R1 adds the surface axis on
top of the existing intent machinery rather than forking a parallel
classifier (harden, not duplicate).
What shipped:
- QuerySurface type ("default" | "summary") and a `surface` field on
QueryPlan; new pure `routeSummarySurface(query, vocabulary)` in
query-plan.ts. Detection is structural and vocabulary-driven only:
a source-targeted `source:<x>` token (vocabulary-independent), or a
`kind:<v>`/`type:<v>` token whose value is a declared artifact kind in
the caller-supplied vocabulary (the schema pack's page_types). No
natural-language word list; the module's language-agnostic invariant note
is extended to cover the router.
- buildQueryPlan takes an optional surfaceVocabulary; omitting it keeps the
pure default provably inert (surface always "default"), so every existing
call site and the query-plan unit tests are unchanged. Surface never enters
planHash and re-weights nothing, so ranking and cache identity are
byte-identical regardless of surface.
- search.ts resolves the page_types vocabulary once from the schema pack and
echoes the verdict on SearchOutcome.surface, present only when "summary";
the MCP brain_search response mirrors it. Non-summary queries omit the
field entirely, keeping the default outcome/response shape byte-identical.
- Retrieval skill (skills/open-second-brain/SKILL.md) gains a "Search
surfaces" section and docs/mcp.md documents the router, both naming the
summary surface as the intended route for source/artifact-kind questions.
Acceptance evidence:
- tests/core/search/query-plan-surface.test.ts: unit coverage - inert pure
default, artifact-kind and source signals select summary, out-of-vocabulary
and plain queries stay default, determinism, any-script vocabulary token,
and byte-identical intent/weightProfile/planHash regardless of vocabulary.
- tests/core/search/summary-router.test.ts: integration on a temp vault
declaring page_types [note, summary] - kind: and source: queries route to
the summary surface; a plain query has no surface field and unchanged
ranked results; an unknown kind does not route.
- Gates: bun run fmt clean; bun run lint 134 warnings 0 errors; bun run
typecheck clean; bun test 6687 pass 0 fail.
Plan deviations: the design left the summary surface's concrete effect open.
R1 lands the router as an advisory routing verdict (structural detection plus
skill/docs naming) rather than auto-injecting a candidate filter, which keeps
the byte-identical-for-non-summary guarantee ironclad and matches the wave's
shadow-only posture; ranking is untouched.
Batch C / docs note: new advisory field brain_search response `surface`
("summary" only, else absent). No new config key, no new CLI verb, no new
MCP tool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Task R2 (t_267f3b4c). Adds an unsupervised, read-only diagnostic that tells
an operator whether the configured reranker actually fits this vault's
queries, hosted on the search diagnostics surface as `o2b search rerank-fit`.
What shipped:
- src/core/search/rerank-fit-check.ts: rerankFitCheck(config, deps). It
samples REAL recorded queries from the cross-query demand log, runs the
base (pre-rerank, cache-off, self-heal-off) retrieval for each, re-scores
the same candidates with the reranker, and measures the Spearman rank
correlation between the two signals per query, averaged. Verdicts:
- fits (correlation >= 0.15): the reranker tracks base retrieval; quiet.
- out_of_domain (|correlation| < 0.15): scores uncorrelated with base
relevance; recommend swapping the model or disabling.
- inverted (correlation <= -0.15): scores anti-correlated; recommend
disabling.
- inapplicable: reranker disabled, unconfigured, or too little signal
(no recorded queries / all-tied candidates) - reported explicitly.
Distinct from the existing labelled rerank-eval-gate (t_9f95ebb6): that
needs a ground-truth dataset; this is unsupervised over recorded traffic.
- CLI verb `o2b search rerank-fit` (--max-queries N --top-k K --json), a
read-only diagnostic mirroring `o2b search check`; bad flags are usage
errors (exit 2). docs/cli-reference.md documents it.
Read-only proof: base probe searches disable the query cache and self-heal
and never record access; the local reranker is offline and deterministic.
The integration test asserts the store file size, the store directory file
set, and the demand-log content are all unchanged across a run.
Acceptance evidence:
- tests/core/search/rerank-fit-check.test.ts: disabled -> inapplicable; no
recorded queries -> inapplicable; anti-correlated provider -> inverted
(recommend disable); zero-correlation provider -> out_of_domain
(recommend swap/disable); correlated provider -> fits (quiet); plus the
read-only integration test over a real index with the local reranker.
- tests/cli/search-rerank-fit-cli.test.ts: rerankerless vault -> inapplicable
JSON envelope; enabled reranker over recorded queries -> applicable
verdict; non-numeric --top-k -> exit 2.
- Gates: bun run fmt clean; bun run lint 134 warnings 0 errors; bun run
typecheck clean; bun test 6696 pass 0 fail.
Plan deviations: hosted on the search-diagnostics surface (o2b search
rerank-fit) rather than doctor --readiness. A readiness probe is a
timeout-bounded pass/fail/skipped credential/adapter check; this diagnostic
samples queries and runs searches, which does not fit that shape, and the
reranker is a search-config concern, so the search CLI is the natural
diagnostics home. Verdicts still map cleanly to the readiness taxonomy
(fits=pass, out_of_domain/inverted=fail-with-recommendation,
inapplicable=skipped) should it later be surfaced through doctor.
Batch C / docs note: new CLI verb `o2b search rerank-fit`. No new config key
(reuses search_rerank_*), no new MCP tool.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Task R3 (t_3ffb021c). Adds one read-only MCP tool, brain_retrieval_plan,
that composes four existing deterministic signals into a per-question
retrieval plan without changing any of them.
What shipped:
- src/core/brain/retrieval-plan.ts: buildRetrievalPlan(vault, question, deps)
composes:
- the query plan (src/core/search/query-plan.ts): structural intent, the
bounded weight profile, and the summary-surface route (R1) -> query
strategy;
- the context-pack density allocation (context-pack.ts +
context-density.ts): the token budget the packer would spend and the
per-item density curve -> allocation;
- the calibrated token-impact ledger (token-impact.ts): the first-pass
rate -> reliability;
- observed route latency (mcp-route-metrics.ts): the brain_search route's
success rate and p95 -> reliability.
It emits source/query strategy, token-budget allocation matching what the
packer would spend, graph-expansion advice (hops by intent), observed
reliability with p95 latency, and a marginal-value stop derived from the
density curve plus p95 latency (the density floor climbs with p95, so a
slower route stops sooner). Injectable deps make each signal unit-testable.
- MCP tool brain_retrieval_plan in RECALL_TOOLS (recall-tools.ts): inputs are
`question` (required) and optional `token_budget` - no mutating parameters;
additionalProperties:false. Description 293 chars, property descriptions
within the 160-char registry guard.
Shadow-only proof: the context pack runs without a receipt, telemetry, or
metric; token-impact and route-latency summaries are reads; the query plan is
pure. The module writes no config and no store rows and exposes no handle that
mutates ranking or weight policy. Both the core and MCP integration tests
assert an unchanged vault file listing across a call.
Tool-count baselines updated 107 -> 108 in tests/mcp/mcp.test.ts and
tests/mcp/removed-tools.test.ts; brain_retrieval_plan added to the frozen
name set in tests/mcp/brain-tools-parity.test.ts and the advertised-name
lists; docs/mcp.md documents the tool.
Acceptance evidence:
- tests/core/brain/retrieval-plan.test.ts: strategy composition (intent +
summary surface + weights); allocation matches packer spend + density
curve; graph-expansion advised for exploratory intents, not exact lookups;
reliability (success rate, p95, first-pass rate); marginal stop tightens as
p95 rises; null-reliability when no route data; read-only over a real vault.
- tests/mcp/brain-retrieval-plan.test.ts: advertised + callable + full shape;
no mutating parameters (question + token_budget only, additionalProperties
false); read-only (no vault writes); missing question is a usage error.
- Gates: bun run fmt clean; bun run lint 134 warnings 0 errors; bun run
typecheck clean; bun test 6707 pass 0 fail.
Plan deviations: none. Advisor hosted in src/core/brain (composes brain
context/ledger/latency plus the search query plan); one new MCP tool exactly.
Batch C / docs note: new MCP tool name brain_retrieval_plan (read-only,
no mutating parameters). No new config key, no new CLI verb. The MCP tool
count baseline is now 108.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Layer an additive, opt-in navigation/map tier on top of the always-on injection kernel. active-inject.ts and recall-inject.ts are untouched, so the every-turn context stays byte-identical by default; the nav tier only runs when nav_tier_enabled is set. What shipped: - hooks/lib/session-state.ts: shared per-session namespaced stamp store (osb.nav_tier.* / osb.oriented.*). Each stamp carries an explicit epoch-ms expiry; readers treat missing/malformed/expired identically (absent) and never throw; writers return false rather than throwing. One JSON file per session scope under .open-second-brain/hook-state/, mirroring the search-focus/<scope>.json precedent (no new store). - src/core/brain/navmap.ts: deterministic structural navmap built from the existing graphStats surface (vault size + top hubs by link degree); never LLM-authored. Pure renderNavmap separated from store-backed buildNavmap; content fenced as untrusted, paths neutralized. - src/core/brain/nav-inject.ts: pure inject/suppress decision core with cadence-window semantics and payload-safe audit shaping. - hooks/nav-inject.ts: UserPromptSubmit hook, registered but dormant by default (recall-inject precedent). Reads the cadence stamp, builds the navmap only when cadence is due, records every decision (when, why, added char count) via appendAuditRecord, and stamps the cadence window. - src/core/config.ts: resolveNavTierEnabled + resolveNavTierCadenceMinutes (nav_tier_enabled / OPEN_SECOND_BRAIN_NAV_TIER_ENABLED, nav_tier_cadence_minutes / OPEN_SECOND_BRAIN_NAV_TIER_CADENCE_MINUTES). Acceptance evidence: - Flag off: nav-inject emits no stdout and writes no audit (regression test), kernel unchanged -> byte-identical preamble. - Flag on, no index: suppress(empty), one audit line. - Cadence stamp live -> suppress(cadence) without building the navmap. - session-state: missing/expired/malformed/exactly-at-expiry all read as absent; per-session isolation; default scope for a missing session id. - Gates: bun run fmt clean; lint 134 warnings/0 errors; typecheck clean; bun test 6731 pass / 0 fail. Plan deviation: the plan's "existing hooks/lib session state" did not yet exist; created the minimal file-backed per-scope KV it describes (not a new general store) and shared it with D2, keeping the binding namespaced-key + epoch-ms-expiry convention. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Nudge agents to query the brain search surface before re-reading raw vault files. Default OFF and byte-identical: the PreToolUse hook is registered but emits nothing until hook_strict_enabled is set (recall-inject precedent). What shipped: - src/core/brain/pretool-orient.ts: pure decideOrient decision core. Raw vault-read detection is STRUCTURAL - the tool call's file path (file_path or path) resolving inside the configured vault root - with the tool name used only as an exact read-vs-mutate discriminator (exact allowlist, never a loose/substring match), so a Write/Edit inside the vault and a Read outside it are never blocked. Orientation refresh recognises the brain search/query tools by exact-anchored suffix. - hooks/pretool-orient.ts: PreToolUse hook. With the flag on and no orientation stamp, the first raw vault read gets permissionDecision "deny" naming the search surface, writes the osb.oriented.blocked stamp, then downgrades to a soft nudge (permissionDecision "allow" + reason). A brain search refreshes the osb.oriented.recent stamp (30 min) that suppresses the block. Reuses the D1 hooks/lib/session-state store. - hooks/hooks.json: PreToolUse matcher "*" -> pretool-orient (dormant by default). - src/core/config.ts: resolveHookStrictEnabled (hook_strict_enabled / OPEN_SECOND_BRAIN_HOOK_STRICT_ENABLED). Acceptance evidence: - Flag off: no stdout, no audit for a raw vault read (regression). - First raw read -> deny naming the search surface; second -> nudge. - Brain search refreshes orientation and suppresses the subsequent block. - Read outside the vault, and Write inside it, are allowed (no stdout). - Fail-open, one test each: non-Claude-Code harness (codex) never blocks; malformed session-state reads as absent (no crash); empty/unreadable stdin emits nothing; unresolvable vault emits nothing. - Gates: bun run fmt clean; lint 134 warnings/0 errors; typecheck clean; bun test 6751 pass / 0 fail. Plan deviation: the orientation stamp is refreshed in-hook on a brain search/query TOOL call (exact-anchored name match) rather than by instrumenting the search core, keeping the stamp write in the hook layer per the "hook stamps use the existing hooks/lib session state" convention and avoiding a core->hooks dependency. The soft nudge is delivered as an explicit allow+reason (PreToolUse cannot inject non-deciding context in this repo's conservative event allowlist); the one-time deny is the primary teaching signal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(W1)
checkCodegraph collapsed a multi-project workspace to projects[0]. It now
covers every discovered project, threading the project path into each status
query and aggregating one entry per project - while staying byte-identical
for single-project workspaces.
What shipped (src/core/partner/codegraph.ts):
- evaluateProjectStatus(project, deps): the existing per-project logic
(not-indexed / status-failed / indexed + graph-health) extracted verbatim,
so the single-project path returns exactly today's CheckResult.
- defaultDetectProjectPathSupport(): structural feature probe - reads
`codegraph status --help` and matches the documented positional `[path]`
usage token (a usage marker, not natural-language text); fails closed on
any spawn/read error. Injectable via deps.detectProjectPathSupport.
- checkCodegraph: 1 project -> evaluateProjectStatus (byte-identical, no
probe). >1 project -> probe project_path support; if supported, evaluate
every project and aggregate ("N code projects:" + one bullet per project,
ok = all ok); if unsupported, degrade to the first project and append an
explicit note naming the missing project_path support and the project count.
Bundle + skill:
- openclaw/index.js: surgical, style-matched port of the same checkCodegraph
refactor + defaultDetectProjectPathSupport + evaluateProjectStatus.
- skills/codegraph-partner/SKILL.md: documents multi-project coverage,
per-query project-path threading, and the degrade note.
Acceptance evidence (tests/core/partner/codegraph.test.ts,
tests/openclaw/bundle.test.ts):
- Single project + a project_path probe that throws if called -> byte-identical
message, no note (probe never runs).
- Multiple projects + support -> aggregate names every project and its counts.
- Multiple projects + support + one not indexed -> ok false, both named.
- Multiple projects + no support -> degrade to first project + explicit
project_path note naming the count.
- defaultDetectProjectPathSupport returns a boolean without throwing.
- Bundle substring test asserts the new multi-project surface is present.
- Gates: bun run fmt clean; lint 134 warnings/0 errors; typecheck clean;
bun test 6757 pass / 0 fail.
Plan deviation: openclaw/index.js is a build artifact last regenerated at
v1.4.0 and is deeply stale versus src/ (~30 versions of unbuilt drift, e.g.
graph-health). A full `bun run build:openclaw` would sweep all that unrelated
drift into this commit, so the bundle received a surgical, style-matched edit
of checkCodegraph only, mirroring the source. The source (with graph-health)
is the tested implementation; the bundle stays consistent with its own
pre-existing no-health baseline for single projects, keeping the single-project
output byte-identical there too. A separate bundle rebuild is left as
pre-existing debt out of this feature's scope.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four small self-review fixes, no production behavior changes except the
scope-key.ts rewrite (provably runtime-identical):
- tests/core/search/scope-filter.test.ts: the scope-filter opt-out
regression test only compared sorted path lists between omitting the
filter and passing `{}`. Strengthened to full-projection equality
(complete result objects, order preserved, no sorting) so byte-identity
is actually asserted. The clock is frozen with a Date.now() spy across
the two search() calls, since recency decay is a continuous function of
wall-clock time and two real timestamps a millisecond apart perturbed
the low-order digits of `score` / `recencyBoost`, causing an unrelated
flake under load (reproduced during the full-suite run).
- tests/core/brain/page-dedup-scope.test.ts: added a dedicated N2
idempotency test beside the existing per-scope dedup tests. Writes two
scopeless duplicate rows, runs the dedup+merge pass, then reruns it and
asserts the second pass finds the same pair (not a new/larger cluster)
and that re-merging is a pure no-op. Verified meaningful by temporarily
breaking patchWikilinks' idempotency check and watching it fail.
- tests/core/search/relational-arm.test.ts: the "byte-identical for
default-off" test only asserted the neighbor was absent. Added a test
that strips arm-attributed entries from an arm-on (rrf) run and requires
the full projection (path/score/reasons/order) of what remains to equal
the arm-off run exactly, plus a test that relationalArmEnabled:true
under fusionMode "linear" is byte-identical to flag-off, since the gate
at search.ts:461-462 only engages the arm in rrf fusion. Verified
meaningful by temporarily removing both the search.ts and ranker.ts
gates and watching the new test fail.
- src/core/scope-key.ts: replaced the literal U+0000 NUL bytes used as
the composite-key separator (lines 92/126/138) with the `\0` escape
sequence, so the file is no longer git-binary and is line-reviewable.
Runtime-identical strings (verified: 0 NUL bytes remain, full suite
passes unchanged). page-dedup.ts and other files using the same NUL
convention are untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
….37.0 CHANGELOG 1.37.0 entry with link reference, README What-is-new refresh, cli-reference and mcp reference sections for the nine units, design.md note recording the deliberate pinned.md scope call from self-review, version propagated via sync-version.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ject empty-stdin behavior Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
solaitken
enabled auto-merge (squash)
July 19, 2026 23:41
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (9)
🚧 Files skipped from review as they are similar to previous changes (7)
📝 WalkthroughWalkthroughVersion 1.37.0 adds exact-state storage, scoped and relational retrieval, deterministic summary routing, read-only diagnostics, retrieval planning, opt-in context hooks, MCP surface updates, multi-project codegraph handling, documentation, and synchronized package versions. ChangesRetrieval and context delivery
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant BrainSearch as brain_search
participant Search as search
participant Fanout as relationalFanout
participant Ranker
participant Filters as result filters
Client->>BrainSearch: submit query and optional scope
BrainSearch->>Search: forward search options
Search->>Search: route summary surface
Search->>Fanout: expand typed relations when enabled
Fanout-->>Search: relational candidates
Search->>Ranker: fuse retrieval lanes
Ranker->>Filters: apply exact-state and scope filters
Filters-->>Client: ranked results and optional surface
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@hooks/lib/session-state.ts`:
- Around line 108-129: Update writeHookStamp to synchronize the read-merge-write
sequence for each session-state file using the repository’s existing advisory
lock mechanism, or an equivalent atomic retry strategy. Acquire the lock before
readState and hold it through writeFileSync, release it on every path, and
preserve the current boolean success/failure behavior.
In `@src/cli/brain/verbs/state.ts`:
- Around line 66-69: Update the ExactStateError handling in the state command’s
catch block to check the existing json flag: return okJson({ ok: false, message:
exc.message }) for JSON output, and retain fail(...) only for non-JSON output.
Preserve rethrowing unrelated exceptions.
In `@src/core/brain/exact-state.ts`:
- Line 84: Convert invalid slug errors in src/core/brain/exact-state.ts at line
84 into ExactStateError("invalid_aspect", ...) via a shared validation helper,
and use it in writeExactState, readExactState, and clearExactState. In
src/cli/brain/verbs/state.ts lines 72-84, wrap the get subcommand in try/catch
matching set and call fail() for ExactStateError; do the same for clear at lines
86-93.
- Around line 103-104: Update readExactState to remove the existsSync pre-check
and read the file directly, catching filesystem errors from readFileSync. Return
null specifically for ENOENT, while preserving parseEntry(slug, path, contents)
for successful reads and allowing unrelated errors to propagate.
In `@src/core/search/relational-fanout.ts`:
- Around line 25-35: Resolve the mismatch between RelationalNode.score’s
documented (0, 1] range and the scoring formula used in the relational traversal
calculation. Prefer normalizing the formula near the score computation so every
valid score remains within (0, 1] while preserving nearer-node dominance and
richness ordering; otherwise update the docstring to state the actual range.
Keep the RelationalNode contract and existing ranking behavior otherwise
unchanged.
In `@src/core/search/rerank-fit-check.ts`:
- Around line 268-278: Update correlateQuery to catch failures from
fetchCandidates and provider.rerank, returning null for any per-query error so a
single failed probe does not reject rerankFitCheck’s concurrent Promise.all.
Preserve the existing null returns for insufficient candidates or mismatched
rerank scores, and keep successful queries returning their Spearman correlation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 879126a8-c38d-40f4-a7b4-528cbe12275a
📒 Files selected for processing (86)
.claude-plugin/plugin.json.codex-plugin/plugin.jsonCHANGELOG.mdREADME.mddocs/brainstorm/retrieval-quality-and-context-delivery/cli-output/claude.mddocs/brainstorm/retrieval-quality-and-context-delivery/cli-output/prompt.mddocs/brainstorm/retrieval-quality-and-context-delivery/design.mddocs/brainstorm/retrieval-quality-and-context-delivery/plan.mddocs/brainstorm/retrieval-quality-and-context-delivery/variants.mddocs/cli-reference.mddocs/mcp.mdhooks/hooks.jsonhooks/lib/session-state.tshooks/nav-inject.tshooks/pretool-orient.tsopenclaw.plugin.jsonopenclaw/index.jspackage.jsonplugin.yamlplugins/codex/.codex-plugin/plugin.jsonplugins/hermes/plugin.yamlpyproject.tomlskills/codegraph-partner/SKILL.mdskills/open-second-brain/SKILL.mdsrc/cli/brain.tssrc/cli/brain/help-text.tssrc/cli/brain/verbs/index.tssrc/cli/brain/verbs/state.tssrc/cli/search.tssrc/core/brain/exact-state.tssrc/core/brain/nav-inject.tssrc/core/brain/navmap.tssrc/core/brain/page-dedup.tssrc/core/brain/paths.tssrc/core/brain/pretool-orient.tssrc/core/brain/retrieval-plan.tssrc/core/config.tssrc/core/partner/codegraph.tssrc/core/scope-key.tssrc/core/search/fusion.tssrc/core/search/index.tssrc/core/search/query-cache.tssrc/core/search/query-plan.tssrc/core/search/ranker.tssrc/core/search/relational-fanout.tssrc/core/search/relational-query.tssrc/core/search/rerank-fit-check.tssrc/core/search/result-filters.tssrc/core/search/search.tssrc/core/search/types.tssrc/core/search/walker.tssrc/core/vault-scope/index-admission.tssrc/mcp/brain/recall-tools.tssrc/mcp/search-tools.tstests/cli/brain-state.test.tstests/cli/search-rerank-fit-cli.test.tstests/cli/search.test.tstests/core/brain/exact-state.test.tstests/core/brain/nav-inject.test.tstests/core/brain/navmap.test.tstests/core/brain/page-dedup-scope.test.tstests/core/brain/pretool-orient.test.tstests/core/brain/retrieval-plan.test.tstests/core/partner/codegraph.test.tstests/core/scope-key.test.tstests/core/search/exact-state-barrier.test.tstests/core/search/index-admission.test.tstests/core/search/query-plan-surface.test.tstests/core/search/relational-arm.test.tstests/core/search/relational-fanout.test.tstests/core/search/relational-query.test.tstests/core/search/rerank-fit-check.test.tstests/core/search/scope-filter.test.tstests/core/search/store.test.tstests/core/search/store.vec.test.tstests/core/search/summary-router.test.tstests/helpers/search-fixtures.tstests/hooks/nav-inject.test.tstests/hooks/pretool-orient.test.tstests/hooks/session-state.test.tstests/mcp/brain-retrieval-plan.test.tstests/mcp/brain-tools-parity.test.tstests/mcp/mcp.test.tstests/mcp/removed-tools.test.tstests/mcp/search-scope-filter.test.tstests/openclaw/bundle.test.ts
- exact-state: wrap slug validation in a typed ExactStateError
("invalid_aspect") via a shared helper used by write/read/clear, so an
invalid aspect surfaces as a handleable operational failure instead of a
generic Error crash. (id 3611602492, Major)
- exact-state: readExactState reads directly and catches ENOENT -> null
instead of existsSync-then-read, closing the TOCTOU window on a
concurrent delete. (id 3611602493, Minor)
- cli/state: get and clear now have the same catch handling as set, and
all three emit the repo-standard { ok: false, message } envelope under
--json (via a shared handleStateError) rather than plain fail text.
(id 3611602490, Minor; id 3611602492, Major)
- hooks/session-state: writeHookStamp now serialises its read-merge-write
under the repo's sync advisory lock (acquireLockSync) with a bounded
retry, so concurrent hook processes cannot drop each other's stamps;
lock contention degrades to a fail-open false. (id 3611602489, Minor)
- search/rerank-fit-check: correlateQuery is isolated per query inside the
concurrent Promise.all (.catch -> null) so one flaky provider/probe
rejection no longer sinks the whole diagnostic; a fully-failed sample
still maps explicitly to inapplicable/insufficient-signal. (id 3611602495, Major)
- search/relational-fanout: doc-only fix - RelationalNode.score docstring
now states the real (0, 1 + MAX_RICHNESS_BONUS] range and the
hop-dominance invariant; the formula is unchanged. (id 3611602494, Minor)
Tests added for the typed-error, TOCTOU, JSON-envelope, lock-contention,
and per-query-isolation paths.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
itechmeat
approved these changes
Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Search now answers relationship-shaped and summary-shaped questions deterministically, explains and plans its own retrieval without touching live policy, and delivers the right context to the agent while stale operational state stays out of recall. Nine units in one wave (v1.37.0): a typed-edge relational RRF arm, composite scope keys with per-scope dedup, an overwrite-only exact-state lane with a retrieval-time staleness barrier, a summary-search router, a per-store reranker fit check, a shadow-only retrieval plan advisor, a cadence-controlled navigation tier, an opt-in strict PreToolUse orientation hook, and workspace-wide codegraph partnering.
Why this matters
flowchart LR subgraph ASK["Agent asks"] Q["Relationship or summary question"] end subgraph RETRIEVE["Retrieval that explains itself"] R1["Typed-edge relational arm (opt-in RRF)"] R2["Summary-surface router"] R3["rerank-fit + brain_retrieval_plan (read-only)"] end subgraph DELIVER["Context delivery with hygiene"] D1["Kernel every turn + navmap by cadence"] D2["First raw read redirected to the index"] D3["Exact-state lane barred from recall"] end Q --> R1 --> D1 Q --> R2 --> D1 R3 -. advice only, never policy .-> RETRIEVE D2 --> RETRIEVE D3 -. stale state never resurfaces .-> D1Concrete benefits
Brain/state/lane that never enters FTS/vector/graph, and a retrieval barrier drops superseded rows.o2b search rerank-fit: fits / out_of_domain / inverted verdicts with a disable-or-swap recommendation) and what retrieval strategy a question deserves (brain_retrieval_planwith a marginal-value stop), both provably read-only.project_pathper query, degrading explicitly on older codegraph versions.What ships
search_relational_arm_enabled/OPEN_SECOND_BRAIN_SEARCH_RELATIONAL_ARM, sharedrrfKey, source-set-scoped cachesrc/core/scope-key.ts,brain_searchsession_scope/project_scopeo2b brain state(set, get, list, clear), index-admission predicate, retrieval barrierQueryPlan.surfaceverdict, retrieval skill hardeningo2b search rerank-fit(read-only diagnostic)brain_retrieval_plan(tool surface 107 -> 108)nav_tier_enabled,nav_tier_cadence_minutes, audited inclusionshook_strict_enabled, one-time deny then nudge, fail-opensrc/core/partner/codegraph.ts,openclaw/index.js, updated skillTwo shared seams carry the wave: the composite scope-key module (single implementation of source-identity and scope keys) and the index-admission predicate (single decision point for what enters the search index).
Notes for review
Brain/pinned.mddeliberately stays indexed (excluding it retroactively would remove existing searchable content and contradict the wave's own no-content-loss acceptance; the new lane is the go-forward home for state), andopenclaw/index.jsreceived a surgical, style-matched edit because the bundle is pre-existing stale debt relative tosrc/(a full rebuild would sweep unrelated drift into this PR;src/core/partner/codegraph.tsis the tested source of truth).package.jsonis canonical andscripts/sync-version.tspropagated the mirrored manifests;sync-version.ts --checkpasses in CI.Test plan
bun run fmtcleanbun run lintat the exact baseline: 134 warnings, 0 errorsbun run typecheckcleanbun testfull suite green (6760 pass / 0 fail at self-review fix commit)bun run scripts/sync-version.ts --checkall manifests at 1.37.0Summary by CodeRabbit
New Features
o2b brain state(overwrite-only exact-state lane) ando2b search rerank-fit(read-only reranker fit diagnostics).brain_retrieval_planMCP tool.Documentation