feat(context): body-content signal + ranking fixes for /memory/context - #15
Merged
Conversation
- explain flag on /memory/context: per-entity score constituents (request-gated, default off, absent from all other callers) - content witness c (0-1): indexed full-text (ts_rank norm 33) + word_similarity per-word fuzz, explicit size damper; replaces the hard-coded 0.2 discovery fallback; seed = keyword_part + w * c (additive, boost-only; w=0 reproduces keyword-only behavior) - content-only entities share one __content__ quota bucket - retired/redirected wikis excluded from retrieval (consolidation stubs no longer occupy result slots) - literal-1.0 graph score fallback removed (honest 0) - L1 reservation ranks by match x effective_importance - final items sorted by final_rank (quota selects, value orders)
19-agent review confirmed six issues; all addressed: - content score is now body-only: c = word_similarity x damper. The full-text tier was scoring search_vector (title+content+keywords), echoing keyword tags as fake body evidence, and was ~17x under-scaled anyway (verified live: stemmed matches carry through trigrams). - admission widened to any-word full-text OR word-trigram window: multi-word queries with partial word presence are findable again (the AND + 0.6-threshold gate was narrower than the old fallback). - ORDER BY references the ws output column (single evaluation; the GREATEST sort key evaluated both scorers twice per row, measured 2x). - e.content dropped from the witness SELECT (bodies were fetched and discarded; graph expansion re-fetches whatever it returns). - retired-wiki filter reuses fetch_ext data (redundant query removed). - recall_memory budgets content per item instead of tail-truncating: the final_rank sort had pushed quota-reserved items into the 8000-char cut, silently breaking the per-query surfacing guarantee. 148/148 vs the isolated test stack; battery: 8/10 targets at #1, zero retired stubs anywhere, latency at or below main.
…al decay Each hop compounded relevance_score x importance_score AND a hardcoded 1.0/0.8/0.6 depth fade. The edge product is already <1 per hop, so it provides the distance decay on its own; the depth layer was a redundant second penalty. Removed it in both the accumulated_relevance expression and the prune check. Verified inert on real data (graph-hop results unchanged: a content_weight=0.5 + depth=1/1/1 sweep moved nothing, and the off-topic boost from cw=0.5 was reverted). Also clarified naming honesty (no API change): accumulated_relevance holds the compounded relevance x importance along the path, not just relevance - documented in the context docstring + README, fixed the stale README fade line (1.0->0.6->0.3) and a stale test comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Improves
/memory/contextranking without changing its keyword-mediated identity. Keyword (fuzzy + embedding) recall is unchanged; an entity's body now contributes a fair, additive signal, and several ranking artifacts are fixed.Scoring
seed = keyword_score + content_weight × c, wherec(0–1) measures whether the entity body matches the query — index-backed stemmed full-text + per-wordword_similarity, with a size damper so long documents must match better. It can reinforce a keyword hit or surface a keyword-missed one, never lowers a score.content_weight=0reproduces the old keyword-only behavior (default 0.3).0.2"discovery" fallback that discarded keyword/content agreement and double-penalized rescued entities.Ranking fixes
final_rank(the quota selects which, value decides order).relevance × importanceproduct — the redundant hard-coded1.0/0.8/0.6depth layer is removed (verified inert on real data).1.0graph-score fallback removed (honest0.0).Diagnostics
explainflag on the context request returns a per-entity score breakdown. Default off, absent from every other caller (agent tools / frontend never set it).Verification
recall_memoryper-item budgeting).accumulated_relevanceclarified in docs (it compounds relevance × importance along the path).