Research: Compare our semantic memory with Simon Høiberg's RAG approach for OpenClaw

[NOVA-Openclaw]

Context

Simon Høiberg (@SimonHoiberg) posted a popular thread about solving OpenClaw's memory problem using RAG:

• Post: https://x.com/SimonHoiberg/status/2025918084591063322
• Engagement: 550 likes, 1232 bookmarks, 42K impressions — clearly resonating with the community

His Approach

• Replaced OpenClaw's default file-based memory (and QMD local hybrid search) with proper RAG
• Uses OpenAI embeddings (claims 50x better accuracy than QMD's small local models)
• Uses a vector database for retrieval
• Claims it's surprisingly simple to set up — had his agent build it
• Replaced QMD entirely: "qmd uses a very small (kinda dumb) local model for embeddings"

Our Current System

We already have a semantic memory system in nova_memory:

• 3,600+ embeddings in memory_embeddings table (pgvector)
• text-embedding-3-small (OpenAI, 1536 dims)
• Priority-weighted recall via memory_type_priorities table
• semantic-recall hook that auto-searches before each message
• Proactive recall script for manual queries
• Source types: entity_facts, daily_logs, agent_chat, entities, events, lessons, tasks, etc.

Research Questions

1. Gap analysis: What does Simon's approach offer that we don't already have?

   • Are we missing hybrid search (BM25 + vector)? QMD does this but we use pure vector.
   • Are we missing LLM reranking of results?
   • Is our embedding chunking strategy optimal?

2. OpenClaw native memory vs our custom system:

   • OpenClaw has built-in memory_search tool + QMD backend + file-based memory
   • We bypass most of this with our PostgreSQL/pgvector system
   • Should we integrate with OpenClaw's memory system instead of (or in addition to) our own?
   • Could we use memory.backend = "qmd" alongside our pgvector approach?

3. Retrieval quality:

   • Is our recall actually good? How would we benchmark it?
   • Are there queries where we fail to surface relevant memories?
   • Would hybrid search (keyword + vector) improve recall vs pure vector?

4. Architecture comparison:

   • Simon: OpenAI embeddings + vector DB (likely Pinecone/Weaviate/similar)
   • Us: OpenAI embeddings + pgvector (PostgreSQL)
   • QMD: Local embeddings + BM25 + vector + LLM reranking
   • What's the optimal combination?

5. Cost/performance tradeoffs:

   • Our embedding pipeline runs nightly — is that frequent enough?
   • Real-time embedding on message receipt vs batch?
   • Local embeddings (QMD) vs API embeddings (our approach) — cost vs quality

References

• OpenClaw memory docs: https://docs.openclaw.ai/concepts/memory
• QMD hybrid search: https://dev.to/chwu1946/openclaw-qmd-local-hybrid-search-for-10x-smarter-memory-4m8m
• SQLite RAG approach: https://www.pingcap.com/blog/local-first-rag-using-sqlite-ai-agent-memory-openclaw/
• QMD backend issue: memory_search tool ignores memory.backend=qmd, tries cloud embedding providers openclaw/openclaw#11278
• Our semantic-memory skill: ~/.openclaw/skills/semantic-memory/SKILL.md

Deliverable

Research report comparing approaches, identifying gaps in our system, and recommending improvements. Not a rewrite — targeted enhancements to what we already have.