Skip to content

fix(memory): stop a turn recalling the question it is answering - #418

Merged
sulthannauval merged 2 commits into
mainfrom
fix/memory-self-echo-crowding
Aug 6, 2026
Merged

fix(memory): stop a turn recalling the question it is answering#418
sulthannauval merged 2 commits into
mainfrom
fix/memory-self-echo-crowding

Conversation

@sulthannauval

Copy link
Copy Markdown
Member

Problem

Auto-save writes each user message to memory before recall runs, so by the
time the [Memory context] block is built the store already holds a verbatim
copy of the question being asked.

That copy is worthless as context — the same text sits in the prompt directly
below the block — and it is actively harmful. Being a perfect lexical match it
takes the top rank, and scores are normalised relative to the best hit, so a
single self-echo pushes every real fact under min_relevance_score.

The effect, measured on a live store with the same query and the same config,
differing only in auto_save:

auto_save keys reaching the prompt
true ["user_msg_ddbebf17-…"] — one entry, its own question
false ["long_note","release_cadence","daily_deploy","deploy_window","roadmap_note"]

With the default settings, curated memory was effectively never reaching the
model. Clearing the conversation category did not help — the next turn
re-writes its own entry before recalling.

This was invisible until #417 made injection visible.

Change

build_memory_context drops entries whose content is the message being
answered, then re-ranks the survivors.

Two details carry the fix, and each is separately mutation-tested:

  • Dropped before the score threshold, not after. By the time the threshold
    runs, the echo has already defined the ranking everything else is measured
    against — filtering it later changes nothing.
  • Re-ranked after the drop, so "relative to the best hit" means the best
    usable hit.

One place, three callers: the agent, the channel dispatcher and the CLI loop
share this builder.

An entry that merely mentions the topic is untouched — only a verbatim echo
is dropped, matched after collapsing whitespace so a re-wrapped copy still
compares equal.

Known limit

Asking the same question verbatim N times leaves N stored echoes. Past the
small over-fetch margin the usable page shrinks by one per extra echo — at four
identical turns the block went from five entries to four. Widening the recall on
every turn to buy headroom for verbatim repeats costs more than the case is
worth; what is lost is the weakest-ranked entries, and nothing incorrect enters
the prompt. Documented on the constant.

Not changed

recall_layered puts conversation-scoped hits ahead of shared memory, and both
tiers share one entry budget. That balance is a design decision, not a defect,
and it is left alone here.

Validation

  • cargo test --lib memory:: / agent:: / channels:: and --test agent_e2e,
    --test memory_comparison — 1311 passed.
  • Mutation-tested both halves: skipping the re-rank fails 2 tests; moving
    the drop after the threshold fails the same 2.
  • Strict delta gate — clean.
  • Live, auto_save = true, gateway and TUI:
↺ recalled 5 memories: long_note, release_cadence, daily_deploy, +2 more

Stable across repeated turns (2, 3, 4).

Risk / rollback

This changes what every turn sends the model — that is the point, but it is
the blast radius. One file, one function, no config or schema change. Revert the
commit.

Auto-save writes each user message to memory before recall runs, so the store
holds a verbatim copy of the question by the time the context block is built.
That copy is worthless as context — the text sits in the prompt directly below
the block — and it is actively harmful: being a perfect lexical match it takes
the top rank, and scores normalise *relative to the best hit*, so one self-echo
pushes every real fact under min_relevance_score.

Measured on a live store, same query, same config:

  before  keys: ["user_msg_ddbebf17-…"]
  after   keys: ["long_note","release_cadence","daily_deploy",
                 "deploy_window","roadmap_note"]

The "after" set is exactly what the same query returned with auto_save turned
off, so the curated memories were always there — the echo was hiding them.

Dropped *before* the score threshold, not after: by then the echo has already
defined the ranking everything else is measured against. The survivors are
re-ranked, so "relative to the best hit" means the best usable hit.

Fixes all three callers at once — agent, channels and the CLI loop share this
builder. An entry that merely mentions the topic is untouched; only a verbatim
echo is dropped.
@github-actions github-actions Bot added the memory Auto scope: src/memory/** changed. label Aug 6, 2026
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

PR intake checks found warnings (non-blocking)

Fast safe checks found advisory issues. CI lint/test/build gates still enforce merge quality.

  • Missing required PR template sections: ## Summary, ## Validation Evidence (required), ## Security Impact (required), ## Privacy and Data Hygiene (required), ## Rollback Plan (required)
  • Incomplete required PR template fields: summary problem, summary why it matters, summary what changed, validation commands, security risk/mitigation, privacy status, rollback plan

Action items:

  1. Complete required PR template sections/fields.
  2. Remove tabs, trailing whitespace, and merge conflict markers from added lines.
  3. Re-run local checks before pushing:
    • ./scripts/ci/rust_quality_gate.sh
    • ./scripts/ci/rust_strict_delta_gate.sh
    • ./scripts/ci/docs_quality_gate.sh

Run logs: https://github.com/RantAI-dev/RantAIClaw/actions/runs/31090472574

Detected blocking line issues (sample):

  • none

Detected advisory line issues (sample):

  • none

@github-actions github-actions Bot added size: S Auto size: 81-250 non-doc changed lines. risk: medium Auto risk: src/** or dependency/config changes. distinguished contributor Contributor with 50+ merged PRs. memory: context Auto module: memory/context changed. and removed memory Auto scope: src/memory/** changed. labels Aug 6, 2026
The self-echo defect is a direct consequence of scores being relative, so it
belongs in the section that explains them.
@github-actions github-actions Bot added docs Auto scope: docs/markdown/template files changed. memory Auto scope: src/memory/** changed. and removed memory Auto scope: src/memory/** changed. labels Aug 6, 2026
@sulthannauval
sulthannauval merged commit 494429f into main Aug 6, 2026
17 checks passed
@sulthannauval
sulthannauval deleted the fix/memory-self-echo-crowding branch August 6, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

distinguished contributor Contributor with 50+ merged PRs. docs Auto scope: docs/markdown/template files changed. memory: context Auto module: memory/context changed. risk: medium Auto risk: src/** or dependency/config changes. size: S Auto size: 81-250 non-doc changed lines.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant