fix(memory): stop a turn recalling the question it is answering - #418
Merged
Conversation
Auto-save writes each user message to memory before recall runs, so the store
holds a verbatim copy of the question by the time the context block is built.
That copy is worthless as context — the text sits in the prompt directly below
the block — and it is actively harmful: being a perfect lexical match it takes
the top rank, and scores normalise *relative to the best hit*, so one self-echo
pushes every real fact under min_relevance_score.
Measured on a live store, same query, same config:
before keys: ["user_msg_ddbebf17-…"]
after keys: ["long_note","release_cadence","daily_deploy",
"deploy_window","roadmap_note"]
The "after" set is exactly what the same query returned with auto_save turned
off, so the curated memories were always there — the echo was hiding them.
Dropped *before* the score threshold, not after: by then the echo has already
defined the ranking everything else is measured against. The survivors are
re-ranked, so "relative to the best hit" means the best usable hit.
Fixes all three callers at once — agent, channels and the CLI loop share this
builder. An entry that merely mentions the topic is untouched; only a verbatim
echo is dropped.
PR intake checks found warnings (non-blocking)Fast safe checks found advisory issues. CI lint/test/build gates still enforce merge quality.
Action items:
Run logs: https://github.com/RantAI-dev/RantAIClaw/actions/runs/31090472574 Detected blocking line issues (sample):
Detected advisory line issues (sample):
|
The self-echo defect is a direct consequence of scores being relative, so it belongs in the section that explains them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Auto-save writes each user message to memory before recall runs, so by the
time the
[Memory context]block is built the store already holds a verbatimcopy of the question being asked.
That copy is worthless as context — the same text sits in the prompt directly
below the block — and it is actively harmful. Being a perfect lexical match it
takes the top rank, and scores are normalised relative to the best hit, so a
single self-echo pushes every real fact under
min_relevance_score.The effect, measured on a live store with the same query and the same config,
differing only in
auto_save:auto_savetrue["user_msg_ddbebf17-…"]— one entry, its own questionfalse["long_note","release_cadence","daily_deploy","deploy_window","roadmap_note"]With the default settings, curated memory was effectively never reaching the
model. Clearing the
conversationcategory did not help — the next turnre-writes its own entry before recalling.
This was invisible until #417 made injection visible.
Change
build_memory_contextdrops entries whose content is the message beinganswered, then re-ranks the survivors.
Two details carry the fix, and each is separately mutation-tested:
runs, the echo has already defined the ranking everything else is measured
against — filtering it later changes nothing.
usable hit.
One place, three callers: the agent, the channel dispatcher and the CLI loop
share this builder.
An entry that merely mentions the topic is untouched — only a verbatim echo
is dropped, matched after collapsing whitespace so a re-wrapped copy still
compares equal.
Known limit
Asking the same question verbatim N times leaves N stored echoes. Past the
small over-fetch margin the usable page shrinks by one per extra echo — at four
identical turns the block went from five entries to four. Widening the recall on
every turn to buy headroom for verbatim repeats costs more than the case is
worth; what is lost is the weakest-ranked entries, and nothing incorrect enters
the prompt. Documented on the constant.
Not changed
recall_layeredputs conversation-scoped hits ahead of shared memory, and bothtiers share one entry budget. That balance is a design decision, not a defect,
and it is left alone here.
Validation
cargo test --lib memory:: / agent:: / channels::and--test agent_e2e,--test memory_comparison— 1311 passed.the drop after the threshold fails the same 2.
auto_save = true, gateway and TUI:Stable across repeated turns (2, 3, 4).
Risk / rollback
This changes what every turn sends the model — that is the point, but it is
the blast radius. One file, one function, no config or schema change. Revert the
commit.