Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions crates/memory-core/src/fts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ const STOPWORDS: &[&str] = &[
"it",
"kind",
"kinds",
"like",
"many",
"of",
"on",
Expand Down Expand Up @@ -206,4 +207,10 @@ mod tests {
]
);
}

#[test]
fn query_tokens_drop_like_filler_keep_subject_anchor() {
let tokens = normalize_query_tokens("What do Melanie's kids like?");
assert_eq!(tokens, vec!["melanie".to_string(), "kids".to_string()]);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# V2-R12 LoCoMo `like` Stopword Implementation Plan

> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Improve LoCoMo retrieval with one minimal query-tokenization change by treating `like` as a stopword.

**Architecture:** Keep retrieval/rerank pipeline unchanged; update stopword list and add one unit test in `fts.rs` only.

**Tech Stack:** Rust (`memory-core`) + existing LoCoMo scripts.

---

### Task 1: Single-Knob Code Change

**Files:**
- Modify: `crates/memory-core/src/fts.rs`

- [ ] Add `like` to `STOPWORDS`.
- [ ] Add one unit test for `like` removal while preserving semantic anchors.

### Task 2: Verification

**Files:**
- No source edits

- [ ] `cargo test --workspace`
- [ ] `./.venv/bin/pytest python/tests -q`
- [ ] `corepack pnpm --dir packages/node test`
- [ ] `corepack pnpm --dir packages/mcp test`
- [ ] `REBUILD_PYTHON=0 tools/locomo/run-full-gate.sh 5`

### Task 3: Finish Round

**Files:**
- Create: `docs/superpowers/releases/2026-03-19-v2-r12-locomo-like-stopword-results.md`

- [ ] Record LoCoMo and verification outputs.
- [ ] Commit, PR, merge.
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# V2-R12 LoCoMo `like` Stopword Results

## Summary

This round intentionally applies one single-knob retrieval change:

- add `like` to query stopwords in `memory-core` FTS normalization
- add one unit test for the new token rule

No ranking formula or rerank logic changes were made.

## Files

- `crates/memory-core/src/fts.rs`
- `docs/superpowers/specs/2026-03-19-v2-r12-locomo-like-stopword-design.md`
- `docs/superpowers/plans/2026-03-19-v2-r12-locomo-like-stopword-implementation-plan.md`
- `docs/superpowers/releases/2026-03-19-v2-r12-locomo-like-stopword-results.md`

## LoCoMo Gate Result (K=5)

Command:

```bash
REBUILD_PYTHON=0 tools/locomo/run-full-gate.sh 5
```

Result:

- overall: `0.5822`
- multi-hop: `0.4255`
- open-domain: `0.3370`
- threshold checks: PASS
- drift checks: PASS
- full gate: PASS

## Required Verification Matrix

All required commands passed:

1. `cargo test --workspace`
2. `./.venv/bin/pytest python/tests -q`
3. `corepack pnpm --dir packages/node test`
4. `corepack pnpm --dir packages/mcp test`

## Outcome

- This minimal change is safe (no regression) under current guardrails.
- Measured K=5 gate metrics are unchanged in this run.
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# V2-R12 LoCoMo Single-Knob `like` Stopword Design

## Goal

Apply one minimal retrieval tweak to improve LoCoMo hit stability without broad redesign.

Primary rule for this round: one knob only.

## Baseline

Current K=5 baseline (from V2-R10/V2-R11):

- overall: `0.5822`
- multi-hop: `0.4255`
- open-domain: `0.3370`

## Brainstorming Summary

After refreshing `failure-samples-k5.json` with limit 120 and replaying candidate single-word removals:

- removing `games`: multi-hop improves but single-hop drops, overall unchanged
- removing `like`: small net gain (`overall 0.5822 -> 0.5827`, adversarial improves), no threshold-category degradation

Chosen option: add `like` into query stopwords.

## Chosen Design

1. Add one token to `STOPWORDS` in `crates/memory-core/src/fts.rs`:
- `like`

2. Add one focused unit test verifying `like` is removed while anchor tokens stay.

3. Keep all other ranking/tokenization behavior unchanged.

## Test Strategy

1. `cargo test --workspace`
2. `./.venv/bin/pytest python/tests -q`
3. `corepack pnpm --dir packages/node test`
4. `corepack pnpm --dir packages/mcp test`
5. `REBUILD_PYTHON=0 tools/locomo/run-full-gate.sh 5`

## Success Criteria

1. Single-knob change only (`like` stopword + minimal test)
2. required 4-command matrix passes
3. LoCoMo full gate passes (thresholds + drift)
Loading