Skip to content

docs: compare host-native long-term memory costs #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
OLDyade wants to merge 1 commit into
base: main
Choose a base branch
from
+81 −16
Draft

docs: compare host-native long-term memory costs #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,14 +142,15 @@ evolve. See [`docs/evaluation.md`](docs/evaluation.md).

## Documentation

| Document | Content |
| ------------------------------------------------------------------ | --------------------------------------------------------------------------------- |
| [`docs/architecture.md`](docs/architecture.md) | Storage layout, recall, extraction, dream, skill loop, package boundaries |
| [`docs/integrations.md`](docs/integrations.md) | Pi, Hermes, OpenCode, OpenClaw, embedding pre-recall, SDK hooks, adapter boundary |
| [`docs/evaluation.md`](docs/evaluation.md) | LoCoMo position and local regression checks |
| [`docs/release.md`](docs/release.md) | Versioning, npm release channel, publish checklist |
| [`CHANGELOG.md`](CHANGELOG.md) | Release notes for public npm package versions |
| [`NOTICE`](NOTICE), [`THIRD_PARTY_LICENSES`](THIRD_PARTY_LICENSES) | Project notice and third-party license disclosure |
| Document | Content |
| ---------------------------------------------------------------------- | --------------------------------------------------------------------------------- |
| [`docs/architecture.md`](docs/architecture.md) | Storage layout, recall, extraction, dream, skill loop, package boundaries |
| [`docs/integrations.md`](docs/integrations.md) | Pi, Hermes, OpenCode, OpenClaw, embedding pre-recall, SDK hooks, adapter boundary |
| [`docs/native-memory-comparison.md`](docs/native-memory-comparison.md) | Host-native long-term memory capabilities and cost |
| [`docs/evaluation.md`](docs/evaluation.md) | LoCoMo position and local regression checks |
| [`docs/release.md`](docs/release.md) | Versioning, npm release channel, publish checklist |
| [`CHANGELOG.md`](CHANGELOG.md) | Release notes for public npm package versions |
| [`NOTICE`](NOTICE), [`THIRD_PARTY_LICENSES`](THIRD_PARTY_LICENSES) | Project notice and third-party license disclosure |

## Open-Source Boundary

Expand Down
17 changes: 9 additions & 8 deletions README.zh.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,14 +131,15 @@ MemFlywheel 使用面向 LoCoMo 的回归检查，让长期记忆能力在召回

## 文档

| 文档 | 内容 |
| ------------------------------------------------------------------ | ------------------------------------------------------------------------- |
| [`docs/architecture.md`](docs/architecture.md) | 存储布局、召回、提取、dream、技能闭环、包边界 |
| [`docs/integrations.md`](docs/integrations.md) | Pi、Hermes、OpenCode、OpenClaw、embedding 预召回、SDK hooks、adapter 边界 |
| [`docs/evaluation.md`](docs/evaluation.md) | LoCoMo 定位和本地回归检查 |
| [`docs/release.md`](docs/release.md) | 版本规范、npm 发布渠道、发布检查清单 |
| [`CHANGELOG.md`](CHANGELOG.md) | 公开 npm 包版本变更记录 |
| [`NOTICE`](NOTICE)、[`THIRD_PARTY_LICENSES`](THIRD_PARTY_LICENSES) | 项目版权告知和三方 license 披露 |
| 文档 | 内容 |
| ---------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| [`docs/architecture.md`](docs/architecture.md) | 存储布局、召回、提取、dream、技能闭环、包边界 |
| [`docs/integrations.md`](docs/integrations.md) | Pi、Hermes、OpenCode、OpenClaw、embedding 预召回、SDK hooks、adapter 边界 |
| [`docs/native-memory-comparison.md`](docs/native-memory-comparison.md) | 宿主原生长期记忆能力和开销对比 |
| [`docs/evaluation.md`](docs/evaluation.md) | LoCoMo 定位和本地回归检查 |
| [`docs/release.md`](docs/release.md) | 版本规范、npm 发布渠道、发布检查清单 |
| [`CHANGELOG.md`](CHANGELOG.md) | 公开 npm 包版本变更记录 |
| [`NOTICE`](NOTICE)、[`THIRD_PARTY_LICENSES`](THIRD_PARTY_LICENSES) | 项目版权告知和三方 license 披露 |

## 开源边界

Expand Down
4 changes: 4 additions & 0 deletions docs/integrations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ skill loops should fail fast instead of parsing free-form model text.
| OpenClaw | Implemented plugin path | Load the adapter as OpenClaw's memory slot; hooks provide recall, extraction, and skills |
| OpenCode | Implemented plugin path | Load the adapter as an OpenCode plugin; hooks provide recall, extraction, and skills |

For a capability and runtime cost comparison with each host's native long-term
memory, see
[Host-Native Long-Term Memory Comparison](native-memory-comparison.md).

## Large Index Pre-Recall

By default, prompt recall injects the generated `MEMORY.md` index directly. That
Expand Down
59 changes: 59 additions & 0 deletions docs/native-memory-comparison.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Host-Native Long-Term Memory Comparison

Comparison of long-term memory capabilities and cost across MemFlywheel, Pi, Hermes, OpenClaw, and OpenCode.

## Capability Comparison

| System | Native long-term memory | Storage and extraction | Recall and maintenance |
| --------------- | ----------------------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| Pi native | No | — | — |
| Hermes native | Yes | Capacity-bounded `MEMORY.md` and `USER.md`, managed by the Agent memory tool and background review | Injects a bounded snapshot into future Sessions; the Agent must replace, remove, or compress entries at capacity |
| OpenClaw native | Yes | `MEMORY.md`, dated `memory/*.md` notes, optional `DREAMS.md`, and a memory index | `memory_search` and `memory_get`; keyword, vector, or hybrid retrieval; optional dreaming |
| OpenCode native | No | — | — |
| MemFlywheel | Yes | Typed Markdown memories written by a turn-end extraction Agent, with source traces and an audit log | Index cues route the main Agent to relevant bodies; gated dream consolidation and optional learned-skill evolution |

## Long-Term Memory Cost

| Metric | Calculation |
| ----------------------- | ----------------------------------------------------------------------- |
| Total cost | `Prompt increase + extraction + consolidation + recall + index` |
| Total overhead | `long-term memory cost / host cost with long-term memory disabled` |
| Foreground prompt tax | `additional foreground input tokens / baseline foreground input tokens` |
| LLM write amplification | `memory write-side model tokens / new conversation tokens` |
| Recall amplification | `recall-path tokens / user-query tokens` |
| Maintenance cost / turn | `(skill evolution + dream) / completed turns` |
| Model cost | `Σ(input tokens × input price + output tokens × output price)` |

| System | Foreground cost | Write-side cost | Maintenance and retrieval | Dominant cost shape |
| --------------- | ------------------------------------------------------------- | -------------------------------------------------------------------- | --------------------------------------------------------------- | ----------------------------------------------------- |
| Hermes native | Approximately 1,300 bounded memory tokens accompany inference | Main Agent memory-tool writes; background review every 10 user turns | Context-resident snapshot; no separate retrieval | Fixed prompt tax + infrequent upkeep |
| OpenClaw native | Bounded `MEMORY.md`; the Codex harness uses a retrieval cue | Main Agent file writes; memory flush before compaction | On-demand BM25/vector recall; dreaming off by default | Low write cost + on-demand recall |
| MemFlywheel | Bounded index cues; bodies are read progressively | Extraction Agent after every turn, with multi-round file-tool access | Optional skill evolution; dream after five Sessions or 24 hours | **High write amplification** + periodic consolidation |

MemFlywheel uses **eager agentic consolidation**: per-turn extraction, rather than Markdown, indexing, or storage, dominates its cost.

### Normalized Workload Estimate

| Parameter | Value |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Workload | 100 turns / 10 Sessions; average 8,000 input and 800 output tokens per turn |
| Memory activity | 20% of turns write durable memory; 20% perform recall, injecting 800 tokens |
| Cache | 80% hit rate for main input; 90%–99% for stable prompts/indexes; new conversation content, tool results, and recalled bodies count as misses |
| Pricing weights | Normalized from [DeepSeek V4 Flash pricing](https://api-docs.deepseek.com/quick_start/pricing/) on 2026-07-14: cache-hit input `0.02`, cache-miss input `1`, output `2` |
| Hermes | 1,300-token snapshot; one review every 10 user turns, taking 1–3 rounds |
| OpenClaw | 1,000-token `MEMORY.md`; four compaction flushes; dreaming disabled |
| MemFlywheel Core | Approximately 3,900-token extraction prompt; one round for 80% of turns and 2–4 rounds for 20% |
| MemFlywheel Full | Core + low-frequency skill evolution + two dream passes |
| Baseline cost | `100 × [8,000 × (80% × 0.02 + 20% × 1) + 800 × 2] = 332,800 cost units` |

### Estimated Results

| System | Resident prompt and recall | Write and maintenance | Incremental cost units | Total model-cost increase |
| --------------------------- | -------------------------- | --------------------- | ---------------------- | ------------------------- |
| Hermes native | `+1%–5%` | `+3%–10%` | `12K–47K` | **`+4%–15%`** |
| OpenClaw native (Codex) | `+4%–6%` | `+1%–3%` | `19K–27K` | **`+6%–9%`** |
| OpenClaw native (non-Codex) | `+6%–9%` | `+1%–3%` | `22K–39K` | **`+7%–12%`** |
| MemFlywheel Core | `+6%–9%` | `+56%–80%` | `207K–294K` | **`+65%–90%`** |
| MemFlywheel Full loop | `+6%–9%` | `+67%–107%` | `242K–384K` | **`+75%–115%`** |

MemFlywheel spends approximately `65%–115%` more model budget on per-turn semantic extraction, in-place revision, deduplication, provenance, and continuing evolution. These are architectural estimates under normalized parameters; actual percentages should come from model usage telemetry.