Skip to content

Latest commit

 

History

History
290 lines (226 loc) · 10.7 KB

File metadata and controls

290 lines (226 loc) · 10.7 KB

Smriti

Shared memory layer for AI-powered engineering teams. Built on QMD.

Memory

You have Smriti. USE IT.

Start of Every Session

Before writing a single line of code, run:

smriti recall "[what we're about to work on]"

Use what comes back. If there's prior context on this topic, it changes how you approach the work. Don't ignore it.

Recognize These Moments — Then Act Immediately

No permission needed. No asking "should I save this?" Just do it.

  • Someone says "this works" or "let's go with this"
  • An approach is chosen after comparing options
  • A bug is fixed and the root cause is understood
  • A constraint is discovered that will affect future work
  • An architectural decision is made

WRONG: Continue the conversation without saving. RIGHT: Write the decision record, ingest it, tag it — then respond.

How to Save a Decision

Step 1 — Write the record:

cat > /tmp/smriti-note.md << 'EOF'
# [Title]

## Decision
[One sentence — the actual choice made.]

## Why
[The reasoning. What was considered. What was ruled out and why.]

## Context
[What problem this solves. What breaks if someone reverses this later.]
EOF

Step 2 — Ingest:

smriti ingest file /tmp/smriti-note.md \
  --title "[Title]" \
  --project smriti

Step 3 — Tag:

smriti tag [session-id] [category]

Categories

What happened Tag
Architecture choice, approach finalized decision/technical
Library or tool selected decision/tooling
Bug fixed, root cause identified bug/fix
Structural design settled architecture/design
Feature approach finalized feature/implementation
Config or setup figured out project/config

Concrete Example

User says: "Okay let's use post-filtering for vector search with filters"

WRONG:

"Got it, I'll use post-filtering."

RIGHT:

cat > /tmp/smriti-note.md << 'EOF'
# Vector search with filters: post-filter approach

## Decision
Use post-filter strategy for applying metadata filters to vector search
results, rather than combining sqlite-vec queries with EXISTS subqueries.

## Why
sqlite-vec's two-step query pattern hangs when combined with Smriti's
EXISTS subqueries on metadata tables. Post-filtering runs vector search
unfiltered (3x overfetch), then filters results against session metadata
in a separate batch lookup — no sqlite-vec changes needed.

## Context
Affects smriti recall when --project, --category, or --agent flags are
used. Without this, filtered recall loses semantic search entirely and
falls back to BM25 only.
EOF

smriti ingest file /tmp/smriti-note.md \
  --title "Vector search with filters: post-filter approach" \
  --project smriti

smriti tag [session-id] decision/technical

Then respond to the user.


Quick Reference

smriti ingest claude             # Ingest Claude Code sessions
smriti ingest codex              # Ingest Codex CLI sessions
smriti ingest cline              # Ingest Cline CLI sessions
smriti ingest copilot            # Ingest GitHub Copilot (VS Code) sessions
smriti ingest cursor --project-path ./myapp
smriti ingest all                # All known agents
smriti search "query"            # Hybrid search (BM25 + vector)
smriti recall "query"            # Smart recall with dedup
smriti recall "query" --synthesize  # Synthesize via Ollama
smriti list                      # Recent sessions
smriti show <session-id>         # View a session
smriti status                    # Memory statistics
smriti embed                     # Build vector embeddings
smriti categorize                # Auto-categorize sessions
smriti share --project myapp     # Export to .smriti/ for git
smriti sync                      # Import team knowledge

Project Structure

src/
├── index.ts              # CLI entry point
├── config.ts             # Paths, env vars, defaults
├── db.ts                 # SQLite schema + Smriti metadata tables
├── qmd.ts                # Centralized re-exports from QMD package
├── format.ts             # Output formatting (JSON, CSV, CLI)
├── ingest/
│   ├── index.ts          # Orchestrator (parser -> resolver -> store)
│   ├── parsers/          # Pure agent parsers (no DB writes)
│   ├── session-resolver.ts  # Project/session resolution + incremental state
│   ├── store-gateway.ts  # Centralized ingest persistence
│   ├── claude.ts         # Discovery + compatibility wrapper
│   ├── codex.ts          # Discovery + compatibility wrapper
│   ├── cursor.ts         # Discovery + compatibility wrapper
│   ├── cline.ts          # Discovery + compatibility wrapper
│   ├── copilot.ts        # Discovery + compatibility wrapper
│   └── generic.ts        # File import compatibility wrapper
├── search/
│   ├── index.ts          # Filtered FTS search + session listing
│   └── recall.ts         # Recall with synthesis
├── categorize/
│   ├── schema.ts         # Category tree definitions
│   └── classifier.ts     # Auto-categorization (rule-based + LLM)
└── team/
    ├── share.ts          # Export knowledge to .smriti/ directory
    ├── sync.ts           # Import team knowledge from .smriti/
    ├── formatter.ts      # Sanitization + doc formatting pipeline
    ├── reflect.ts        # LLM-powered session reflection via Ollama
    └── prompts/
        └── share-reflect.md
test/
├── ingest.test.ts        # Parser + project detection tests
├── search.test.ts        # Search + recall tests
├── db.test.ts            # Schema + metadata tests
├── categorize.test.ts    # Categorization tests
├── team.test.ts          # Share + sync tests
├── formatter.test.ts     # Formatter + sanitization tests
└── reflect.test.ts       # Reflection parsing tests
.github/workflows/
├── ci.yml                # bun test on Ubuntu, macOS, Windows
├── install-test.yml      # Full install + smoke test on all 3 OSes
├── release.yml           # GitHub Release on v*.*.* tag
└── secret-scan.yml       # Gitleaks + detect-secrets

Architecture

All QMD imports go through src/qmd.ts — a single re-export hub:

import { addMessage, recallMemories, searchMemoryFTS } from "./qmd";
import { hashContent } from "./qmd";
import { ollamaRecall } from "./qmd";

Never import from QMD directly in other files. Always go through src/qmd.ts.

Key Concepts

Project Detection

Claude Code stores sessions in ~/.claude/projects/<dir-name>/. The dir name encodes the filesystem path with - replacing / (e.g. -Users-zero8-zero8.dev-openfga).

deriveProjectPath() reconstructs the real path using greedy existsSync() matching. deriveProjectId() strips PROJECTS_ROOT (default ~/zero8.dev) to get a clean name like openfga.

Ingestion Pipeline

  1. Discover sessions (agent modules)
  2. Parse session content (pure parser layer)
  3. Resolve project/session state (resolver layer)
  4. Store message/meta/sidecars/costs (store gateway)
  5. Aggregate results and continue on per-session errors (orchestrator)

See docs/internal/ingest-architecture.md for details.

Search

  • Filtered search (searchFiltered): FTS5 with JOINs to Smriti metadata tables for category/project/agent filtering
  • Unfiltered search (searchFTS, searchVec): Delegates directly to QMD
  • Recall: Search → deduplicate by session → optionally synthesize via Ollama

Team Sharing

  • smriti share: Exports sessions as clean documentation to .smriti/knowledge/
    • Sanitizes XML noise, interrupt markers, API errors, narration filler
    • Filters noise-only sessions, merges consecutive same-role messages
    • Generates LLM reflections via Ollama by default (use --no-reflect to skip)
    • Generates .smriti/CLAUDE.md so Claude Code auto-discovers shared knowledge
    • Customizable reflection prompt at .smriti/prompts/share-reflect.md
  • smriti sync: Imports markdown files from .smriti/knowledge/ back into local DB
  • Deduplication via content hashing — same content won't import twice

Configuration

Env Var Default Description
QMD_DB_PATH ~/.cache/qmd/index.sqlite Database path
CLAUDE_LOGS_DIR ~/.claude/projects Claude Code logs
CODEX_LOGS_DIR ~/.codex Codex CLI logs
CLINE_LOGS_DIR ~/.cline/tasks Cline CLI tasks
COPILOT_STORAGE_DIR auto-detected per OS VS Code workspaceStorage root override
SMRITI_PROJECTS_ROOT ~/zero8.dev Projects root for ID derivation
OLLAMA_HOST http://127.0.0.1:11434 Ollama endpoint
QMD_MEMORY_MODEL qwen3:8b-tuned Ollama model for synthesis
SMRITI_CLASSIFY_THRESHOLD 0.5 LLM classification trigger threshold
SMRITI_AUTHOR $USER Git author for team sharing
SMRITI_DAEMON_DEBOUNCE_MS 30000 Daemon file-stability wait (v0.4.0)

Database

Smriti extends QMD's tables with its own metadata:

  • smriti_session_meta — agent_id, project_id per session
  • smriti_projects — project registry (id, path, description)
  • smriti_categories — hierarchical category tree
  • smriti_session_tags — category tags on sessions
  • smriti_message_tags — category tags on messages
  • smriti_shares — dedup tracking for team sharing

QMD's core tables: memory_sessions, memory_messages, memory_fts, content_vectors.

Development

bun install          # Install deps (QMD from github:zero8dotdev/qmd)
bun test             # Run all tests
bun --hot src/index.ts  # Dev mode with hot reload

Design Decisions

  1. Single QMD import hub (src/qmd.ts): No scattered dynamic imports, clean dependency boundary
  2. Greedy path resolution: Handles ambiguous dashes in Claude dir names via existsSync()
  3. Embeddings share QMD's tables: content_vectors + vectors_vec, no duplication
  4. Two-step vector search: Query vectors_vec first, then JOIN to avoid sqlite-vec hang
  5. Content-addressable messages: SHA256 hashing, same as QMD documents
  6. Auto-save via hooks: Claude Code conversations saved without user action