Kosmos

Memory infrastructure for AI agents. Ingests events, detects episodes, and retrieves hierarchically — embeddable as a Rust library, usable from Python, or reachable over HTTP.

The macOS app (app/) is a reference implementation: a 24/7 capture workload that exercises the engine against real, messy data.

---

Architecture

capture/ (Swift)          LangGraph / custom agent
  AX traversal               Python / HTTP client
       |                            |
       | Unix socket          PyO3 / HTTP
       v                            v
  ┌─────────────────────────────────────────┐
  │              kosmos-core                │
  │  ingest → episode detect → chunk store  │
  │  FTS5 + vec KNN → hierarchical query    │
  │  suppress / forget / lineage            │
  └─────────────────────────────────────────┘
       SQLite (WAL) + sqlite-vec + Tantivy

Crates

Crate	What it does
kosmos-types	Shared types: EventId, ChunkId, EpisodeId, Scope, Event, …
kosmos-storage	SQLite backend, sqlite-vec, Tantivy, migration runner, StorageBackend trait
kosmos-tokenizer	URL / filepath / title tokenization for episode affinity scoring
kosmos-chunker	Paragraph, code, and fixed-size chunking strategies with SHA-256 dedup
kosmos-episode	Episode boundary detection: idle-gap + token-overlap + semantic drift
kosmos-embedder	fastembed-rs adapter (bge-small-en-v1.5 default, 384d); Disabled mode for tests
kosmos-worker	Async job queue + scheduler (summarizer, embed worker)
kosmos-governance	Suppress / redact / forget with lineage propagation
kosmos-retrieval	BM25 + KNN + RRF query planner (stub, wired in Phase 2)
kosmos-core	Facade: Kosmos::open(), MemoryEngine trait, ingest pipeline, query pipeline
sdk/kosmos-py	PyO3 Python extension built with maturin
sdk/kosmos-service	axum HTTP service (POST /v1/namespaces/{ns}/events, POST /v1/namespaces/{ns}/query)
app/src-tauri	Tauri macOS app — thin shell over kosmos-core
bench/	Benchmark harness: LoCoMo conditions, F1 judge, real-data smoke tests

Swift capture daemon — capture/

Reads text from each app's UI tree via the macOS Accessibility API. No pixels, no OCR.

• Per-app extraction strategies (Chrome, VS Code, Sublime Text, PyCharm, Discord)
• Triggers on focus change, window change, tab/file switch
• FNV-1a sliding-window dedup before anything crosses the bridge
• TCC revocation detection: exits cleanly if Accessibility permission is removed

kosmos-core ingest pipeline

For each event received from any source:

1. SHA-256 dedup (same content → record occurrence, skip re-index)
2. Tokenize → route to active episode via EpisodeDetector (idle-gap + token-overlap + source-continuity)
3. Chunk with source-specific strategy; fingerprint each chunk
4. Insert into SQLite + FTS5 (via Tantivy) + sqlite-vec (when embedder enabled)
5. Record lineage rows: event → chunk → episode
6. Enqueue background jobs (embed, summarize)

kosmos-core query pipeline

1. FTS5 BM25 over chunk text (SQLite chunks_fts)
2. Tantivy BM25 over same corpus
3. KNN over chunks_vec (when embedder enabled)
4. Fuse via Reciprocal Rank Fusion
5. At Episode granularity: map chunk hits → episodes via episode_members; return episodes

Governance

suppress(scope) — soft-delete, reversible, propagates to derived rows via lineage
forget(scope) — hard delete of content, writes tamper-evident tombstone, preserves lineage pointers

---

Supported capture sources

Source	Strategy
Chrome / Safari	URL host + path segments + query params
VS Code	File path segments
Sublime Text	File path via plugin
PyCharm	File path
Discord	Message text
Any HTTP client	POST /v1/namespaces/{ns}/events
Any Python agent	kosmos.ingest(Event(...))

---

Quickstart

Rust

[dependencies]
kosmos-core = { path = "crates/kosmos-core" }

use kosmos_core::{Kosmos, OpenOptions, Namespace, Event, EventKind, MemoryEngine, Query, Granularity};

let km = Kosmos::open(OpenOptions {
    path: "~/.kosmos".into(),
    namespace: Namespace("my-agent".into()),
    embedder: kosmos_embedder::EmbedderConfig::Disabled,
}).await?;

km.ingest(&ns, Event {
    kind: EventKind::Msg,
    source: "my-agent".into(),
    text: "enterprise pricing discussion".into(),
    ..Default::default()
}).await?;

let hits = km.query(&ns, Query {
    text: "pricing".into(),
    top_k: 5,
    granularity: Granularity::Episode,
}).await?;

Python

cd sdk/kosmos-py && maturin develop

from kosmos import Kosmos

km = Kosmos.open("~/.kosmos", "my-agent")
km.ingest("msg", "my-agent", "enterprise pricing discussion")
hits = km.query("pricing", top_k=5)

HTTP

cargo run -p kosmos-service -- --path ~/.kosmos --bind 0.0.0.0:3000

curl -X POST http://localhost:3000/v1/namespaces/my-agent/events \
  -H "Content-Type: application/json" \
  -d '{"kind":"msg","source":"my-agent","text":"enterprise pricing"}'

curl -X POST http://localhost:3000/v1/namespaces/my-agent/query \
  -H "Content-Type: application/json" \
  -d '{"text":"pricing","top_k":5}'

macOS app

make dev       # build Swift capture + start Tauri dev
make build     # production .app
make capture   # run capture daemon standalone
make log       # stream daemon logs

---

Running tests

cargo test --workspace --exclude kosmos-viewer --exclude kosmos-py

The embedder test (test_local_embedder_dim) is #[ignore] by default — it downloads a 30 MB model on first run. Run it explicitly with:

cargo test -p kosmos-embedder -- --include-ignored

---

Benchmark harness

# Scored run against LoCoMo (needs dataset in bench/datasets/locomo/)
cargo run -p kosmos-bench --release -- scored \
  --condition naive-rag-flat \
  --condition kosmos-episode \
  --dataset bench/datasets/locomo \
  --seeds 42 1337 9999 \
  --output bench/reports/locomo/

# Real-data smoke test against a live corpus snapshot
cargo run -p kosmos-bench -- real-data \
  --db bench/corpora/live-snapshot-YYYYMMDD.db \
  --queries bench/real_queries.toml

See bench/datasets/README.md for dataset download instructions.

---

CI

GitHub Actions runs on every push and PR:

Job	Runner	What it checks
test	ubuntu-latest	cargo check + cargo test + cargo clippy for all core crates
check-macos-app	macos-latest	cargo check -p kosmos-viewer
python-sdk	ubuntu-latest	maturin develop + pytest for the Python extension