|
| 1 | +# Store Module |
| 2 | + |
| 3 | +Multi-graph storage layer built on SQLite. Manages 8 graph types with hybrid search |
| 4 | +(FTS5 + sqlite-vec), unified cross-graph edges, and optimistic locking. |
| 5 | + |
| 6 | +## Stack |
| 7 | + |
| 8 | +- **better-sqlite3** — synchronous SQLite bindings |
| 9 | +- **sqlite-vec v0.1.9** — vector similarity search (per-graph configurable dimensions, default 384) |
| 10 | +- **FTS5** — full-text search |
| 11 | + |
| 12 | +## Architecture |
| 13 | + |
| 14 | +``` |
| 15 | +Store (workspace) |
| 16 | +├── projects — ProjectsStore |
| 17 | +├── team — TeamStore |
| 18 | +├── edges — EdgeHelper (cross-graph) |
| 19 | +├── meta — MetaHelper (key-value) |
| 20 | +├── transaction — SQLite BEGIN/COMMIT/ROLLBACK |
| 21 | +└── project(id) → ProjectScopedStore |
| 22 | + ├── code — CodeStore (indexed) |
| 23 | + ├── docs — DocsStore (indexed) |
| 24 | + ├── files — FilesStore (indexed) |
| 25 | + ├── knowledge — KnowledgeStore (user-managed) |
| 26 | + ├── tasks — TasksStore (user-managed) |
| 27 | + ├── epics — EpicsStore (user-managed) |
| 28 | + ├── skills — SkillsStore (user-managed) |
| 29 | + └── attachments — AttachmentsStore |
| 30 | +``` |
| 31 | + |
| 32 | +**Indexed stores** (code, docs, files) — populated by indexers, bulk upsert by file. |
| 33 | +**User-managed stores** (knowledge, tasks, epics, skills) — CRUD with slugs, versions, |
| 34 | +tags, attachments, optimistic locking. |
| 35 | + |
| 36 | +## Directory Structure |
| 37 | + |
| 38 | +``` |
| 39 | +src/store/ |
| 40 | +├── index.ts # Module exports |
| 41 | +├── types/ |
| 42 | +│ ├── index.ts # Type re-exports |
| 43 | +│ ├── store.ts # Store, ProjectScopedStore interfaces |
| 44 | +│ ├── common.ts # SearchQuery, Edge, GraphName, VersionConflictError |
| 45 | +│ ├── code.ts # CodeStore, CodeNode, CodeFileEntry |
| 46 | +│ ├── docs.ts # DocsStore, DocNode, DocFileEntry |
| 47 | +│ ├── files.ts # FilesStore, FileNode |
| 48 | +│ ├── knowledge.ts # KnowledgeStore, NoteRecord, NoteDetail |
| 49 | +│ ├── tasks.ts # TasksStore, TaskRecord, TaskDetail |
| 50 | +│ ├── epics.ts # EpicsStore, EpicRecord, EpicDetail |
| 51 | +│ ├── skills.ts # SkillsStore, SkillRecord, SkillDetail |
| 52 | +│ ├── attachments.ts # AttachmentsStore, AttachmentMeta |
| 53 | +│ ├── projects.ts # ProjectsStore, ProjectRecord |
| 54 | +│ └── team.ts # TeamStore, TeamMemberRecord |
| 55 | +├── sqlite/ |
| 56 | +│ ├── store.ts # SqliteStore — main implementation |
| 57 | +│ ├── stores/ |
| 58 | +│ │ ├── project-scoped.ts # SqliteProjectScopedStore |
| 59 | +│ │ ├── projects.ts # SqliteProjectsStore |
| 60 | +│ │ ├── team.ts # SqliteTeamStore |
| 61 | +│ │ ├── knowledge.ts # SqliteKnowledgeStore |
| 62 | +│ │ ├── tasks.ts # SqliteTasksStore |
| 63 | +│ │ ├── epics.ts # SqliteEpicsStore |
| 64 | +│ │ ├── skills.ts # SqliteSkillsStore |
| 65 | +│ │ ├── code.ts # SqliteCodeStore |
| 66 | +│ │ ├── docs.ts # SqliteDocsStore |
| 67 | +│ │ ├── files.ts # SqliteFilesStore |
| 68 | +│ │ └── attachments.ts # SqliteAttachmentsStore |
| 69 | +│ ├── lib/ |
| 70 | +│ │ ├── db.ts # openDatabase (WAL, sqlite-vec, FK) |
| 71 | +│ │ ├── migrate.ts # runMigrations (PRAGMA user_version) |
| 72 | +│ │ ├── bigint.ts # num(), now(), likeEscape(), chunk(), assertEmbeddingDim() |
| 73 | +│ │ ├── search.ts # hybridSearch (RRF fusion) |
| 74 | +│ │ ├── meta.ts # MetaHelper (namespaced key-value) |
| 75 | +│ │ ├── edge-helper.ts # EdgeHelper (cross-graph CRUD) |
| 76 | +│ │ └── entity-helpers.ts # EntityHelpers (tags, attachments, edges batch) |
| 77 | +│ └── migrations/ |
| 78 | +│ └── v001.ts # Full schema (tables, FTS5, vec0, triggers, indexes) |
| 79 | +├── FINDINGS.md # sqlite-vec/FTS5 caveats, benchmarks |
| 80 | +├── README.md # This file |
| 81 | +├── SCHEMA.md # Database schema reference |
| 82 | +└── API.md # Public API reference |
| 83 | +``` |
| 84 | + |
| 85 | +## Key Design Decisions |
| 86 | + |
| 87 | +### Transactions are external |
| 88 | + |
| 89 | +Store methods are **not transactional** internally. The caller (orchestrator) wraps |
| 90 | +multi-step operations in `store.transaction()`: |
| 91 | + |
| 92 | +```typescript |
| 93 | +store.transaction(() => { |
| 94 | + const task = scoped.tasks.create(data, embedding); |
| 95 | + scoped.epics.linkTask(epicId, task.id); |
| 96 | +}); |
| 97 | +``` |
| 98 | + |
| 99 | +### Unified edges table |
| 100 | + |
| 101 | +All relationships (same-graph and cross-graph) go through a single `edges` table |
| 102 | +with composite PK: `(project_id, from_graph, from_id, to_graph, to_id, kind)`. |
| 103 | + |
| 104 | +### Tags as edges |
| 105 | + |
| 106 | +Tags are separate entities linked via edges (`from_graph='tags'`, `kind='tagged'`). |
| 107 | +Orphaned tags are cleaned up when the last edge is removed. |
| 108 | + |
| 109 | +### Hybrid search (RRF) |
| 110 | + |
| 111 | +Search combines FTS5 keyword ranking with sqlite-vec cosine distance using |
| 112 | +Reciprocal Rank Fusion (K=60). Three modes: `hybrid`, `keyword`, `vector`. |
| 113 | + |
| 114 | +### Optimistic locking |
| 115 | + |
| 116 | +User-managed entities have `version` field. Pass `expectedVersion` to `update()` |
| 117 | +to detect conflicts — throws `VersionConflictError` on mismatch. |
| 118 | + |
| 119 | +### Gap-based ordering |
| 120 | + |
| 121 | +Tasks and epics use REAL `order` field with 1000-gap increments. Allows |
| 122 | +drag-and-drop reordering without renumbering all rows. |
| 123 | + |
| 124 | +### Cleanup triggers |
| 125 | + |
| 126 | +SQLite triggers on entity DELETE cascade to edges, attachments, and vec0 tables. |
| 127 | +vec0 cleanup must be in triggers because virtual tables don't support CASCADE. |
| 128 | + |
| 129 | +## Usage |
| 130 | + |
| 131 | +```typescript |
| 132 | +import { SqliteStore } from './store'; |
| 133 | + |
| 134 | +const store = new SqliteStore(); |
| 135 | +store.open({ |
| 136 | + dbPath: './data.db', |
| 137 | + embeddingDims: { // optional, default 384 for all |
| 138 | + code: 1024, // use larger model for code |
| 139 | + }, |
| 140 | +}); |
| 141 | + |
| 142 | +// Workspace-level |
| 143 | +const project = store.projects.create({ slug: 'my-project', name: 'My Project', directory: '/path' }); |
| 144 | + |
| 145 | +// Project-scoped |
| 146 | +const scoped = store.project(project.id); |
| 147 | + |
| 148 | +// CRUD with embedding |
| 149 | +const note = scoped.knowledge.create( |
| 150 | + { title: 'Hello', content: 'World', tags: ['intro'] }, |
| 151 | + embedding384, |
| 152 | +); |
| 153 | + |
| 154 | +// Search |
| 155 | +const results = scoped.knowledge.search({ |
| 156 | + text: 'hello', |
| 157 | + embedding: queryEmbedding, |
| 158 | + searchMode: 'hybrid', |
| 159 | + maxResults: 10, |
| 160 | +}); |
| 161 | + |
| 162 | +// Transaction (caller-managed) |
| 163 | +store.transaction(() => { |
| 164 | + scoped.tasks.create(taskData, taskEmbedding); |
| 165 | + scoped.tasks.create(taskData2, taskEmbedding2); |
| 166 | +}); |
| 167 | + |
| 168 | +store.close(); |
| 169 | +``` |
0 commit comments