feat(recall): HOT-recall latency telemetry — measure the sub-200ms exit criterion (#308) - #313
Merged
Merged
Conversation
…it criterion (#308) The Phase-1 §13.2 exit criterion 2 has two halves: ">70% corpus demoted" (already measured as `pctWarmOrCooler`) and "HOT recall sub-200ms" — which was measured NOWHERE and admitted as a caveat. This instruments the synchronous `ask` retrieval latency (embed → doorway fan-out → RRF fusion → rerank) with a bounded process-local reservoir mirroring the blessed capture ACK-latency pattern (#73), surfaces it at GET /admin/stats and the ops exit-gate panel, and retires the latency caveat. Additive contract; no DB migration. - recall-latency.ts: new reservoir (reuses `percentile` from ack-latency.ts); strong-model reconstruction + async recall-rewrite deliberately excluded. - memory.service.ts ask(): sample the sync retrieval for ALL confidence bands. - exit-gate.ts: add `hotRecallLatencyMs: 200` target; split the combined caveat so only graph-walk recall quality remains a caveat. - admin.service.ts getStats(): surface `recallLatency` beside `captureAckLatency`. - admin api client + ops/page.tsx: extend the /admin/stats contract; new GateRow. Closes #308 Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #308.
Summary
§13.2 exit criterion 2 has two halves — ">70% corpus demoted" (already live as
pctWarmOrCooler) and "HOT recall sub-200ms", which was measured nowhere —exit-gate.tseven shipped a caveat admitting it. This times the synchronousaskretrieval (embed → doorway fan-out → RRF fusion → rerank) into a bounded process-local reservoir that mirrors the blessedack-latency.tspattern (Phase-0 gate #73), surfaces it atGET /admin/stats+ the ops exit-gate panel, and retires the caveat. Additive contract only — no DB migration.The strong-model grounded-reconstruction and the async recall-rewrite are deliberately excluded from the sample (they're per-query LLM work, not the "did the memory come back fast" signal the gate judges). Recording happens right after
retrieveCandidatesreturns, so every confidence band — including the honest-miss early return — is sampled.Files
apps/api/src/modules/memory/recall-latency.ts— the reservoir (imports, doesn't duplicate,percentilefromack-latency.ts).apps/api/src/modules/memory/recall-latency.test.ts— unit tests.apps/api/test/recall-latency.integration.test.ts— real capture→encode→ask→/admin/statsflow (M11).memory.service.ts::ask—performance.now()timing around the retrieval;recordRecallLatency(...)afterretrieveCandidates.exit-gate.ts—hotRecallLatencyMs: 200target; split the combined caveat so only the graph-walk clause remains.admin.service.ts::getStats— returnsrecallLatencybesidecaptureAckLatency.admin.service.test.ts— asserts the latency caveat is gone + target is 200.apps/admin/src/lib/api.ts+apps/admin/src/app/ops/page.tsx— client contract + the "HOT recall latency (p95)" gate row (n/auntil sampled; target read from code).Acceptance criteria
/admin/statsreturnsrecallLatency {count,p50,p95,max}; empty → zeros, never NaN — unit test over the reservoir with injected samples.retrieveCandidates, before reconstruction; verified by the integration flow.HOT recall p95row vs a 200ms target read fromEXIT_GATE_TARGETS, with pass/fail/n-a(count===0).EXIT_GATE_CAVEATS(graph-walk clause kept); admin.service.test.ts asserts it's gone andtargets.hotRecallLatencyMs===200.askthenGET /admin/statsshowsrecallLatency.count>=1and a finitep95— the new integration test (observed driving login→capture→encode→ask→stats, all 200).Verification (all green)
pnpm type-check— 7/7 taskspnpm lint— 7/7 taskspnpm build— 5/5 taskspnpm test— api 814 passed; new files independently re-run against the live DB: 3 files / 27 tests passed (integration ran, not skipped)Substrate-only telemetry: no new off-box call (no privacy-fence surface),
userId-scoping unchanged, no hard-delete,performance.now()is a perf measurement (not domain time — §2.6 N/A), no migration.Co-authored-by: Claude Opus 4.8 noreply@anthropic.com