Benchmark evidence and reproduction reports

[CaspianG]

This thread is for benchmark feedback and reproduction reports.

WaveMind is intentionally not positioned as a raw vector DB replacement. The main question is whether dynamic memory policy improves long-running memory behavior enough to justify the extra layer.

Current checked-in evidence

The repository currently includes reproducible artifacts for:

• dynamic memory policy checks;
• LoCoMo retrieval;
• LongMemEval retrieval;
• BEIR/SciFact retrieval;
• ANN/index latency curves;
• static agent-memory retrieval against Chroma.

Start here:

• Benchmark section: https://github.com/CaspianG/wavemind#benchmark
• Benchmark report: https://github.com/CaspianG/wavemind/blob/main/benchmarks/BENCHMARK_REPORT.md
• Result JSON files: https://github.com/CaspianG/wavemind/tree/main/benchmarks

What would be most useful

1. Re-run existing benchmarks on another machine.
2. Add service-mode Qdrant or pgvector numbers.
3. Add FAISS numbers on Linux/macOS where faiss-cpu is available.
4. Add answer-quality runs on LoCoMo or LongMemEval with a local LLM.
5. Challenge the benchmark design if it overstates the dynamic-memory advantage.

Current honest read

WaveMind is stronger in dynamic-memory scenarios: corrections, TTL, namespace isolation, stale suppression, and repeated recall.

Static vector search can still be faster. That limitation is real and should stay visible.

If you reproduce a run, please post:

OS:
Python:
CPU/RAM:
Command:
Result JSON:
Notes:

I will turn useful reproductions into checked-in benchmark artifacts or linked issues.