Skip to content

Add semantic embedding support to MemoryRecallPipeline #7

@adam2go

Description

@adam2go

Summary

The current MemoryRecallPipeline (backend/tilo/services/memory/recall.py) uses keyword, salience, recency, and scope scoring. The _semantic_score() method exists but always returns None.

Adding real vector similarity would significantly improve memory recall quality for long-form text.

What to build

  1. Implement _semantic_score() using a local embedding model (e.g. sentence-transformers)
  2. Store embeddings in the existing Memory.embedding JSON column
  3. Compute cosine similarity at recall time
  4. Update the hybrid scoring weights when semantic score is available

The scoring with semantic:

score = semantic * 0.50 + keyword * 0.20 + salience * 0.15 + recency * 0.10 + scope * 0.05

Acceptance criteria

  • _semantic_score() returns a float when embeddings exist
  • Falls back gracefully to None when no embedding is stored (backward compatible)
  • sentence-transformers added as an optional dependency in pyproject.toml
  • Tests added in backend/tests/test_memory_lifecycle.py

Resources

  • backend/tilo/services/memory/recall.py — the file to modify
  • backend/tilo/models/domain.pyMemory.embedding column (JSON)
  • Existing test: backend/tests/test_memory_lifecycle.py

Optional: make it configurable

Add a EMBEDDING_ENABLED=true flag in .env / config.py so users can opt in.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions