Skip to content

Commit bd2b39a

Browse files
committed
refactor(store): remove internal transactions, per-graph embedding dims, audit fixes, docs
- Remove internal db.transaction() from code/docs/files stores — caller manages atomicity - Merge v002 migration into v001 (dev phase, no deployed DBs) - Add per-graph configurable embedding dimensions via StoreOptions.embeddingDims (default 384) - Fix tags deduplication in EntityHelpers.setTags (dedupe array + INSERT OR IGNORE edge) - Fix tasks.move() timestamp type inconsistency (num(now()) vs Number(now())) - Align knowledge.update() to dynamic SET pattern matching other stores - Fix duplicate "Version conflict" section in common.ts - Add store module documentation: README.md, API.md, SCHEMA.md
1 parent 9572916 commit bd2b39a

19 files changed

Lines changed: 1437 additions & 261 deletions

File tree

src/store/API.md

Lines changed: 631 additions & 0 deletions
Large diffs are not rendered by default.

src/store/README.md

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# Store Module
2+
3+
Multi-graph storage layer built on SQLite. Manages 8 graph types with hybrid search
4+
(FTS5 + sqlite-vec), unified cross-graph edges, and optimistic locking.
5+
6+
## Stack
7+
8+
- **better-sqlite3** — synchronous SQLite bindings
9+
- **sqlite-vec v0.1.9** — vector similarity search (per-graph configurable dimensions, default 384)
10+
- **FTS5** — full-text search
11+
12+
## Architecture
13+
14+
```
15+
Store (workspace)
16+
├── projects — ProjectsStore
17+
├── team — TeamStore
18+
├── edges — EdgeHelper (cross-graph)
19+
├── meta — MetaHelper (key-value)
20+
├── transaction — SQLite BEGIN/COMMIT/ROLLBACK
21+
└── project(id) → ProjectScopedStore
22+
├── code — CodeStore (indexed)
23+
├── docs — DocsStore (indexed)
24+
├── files — FilesStore (indexed)
25+
├── knowledge — KnowledgeStore (user-managed)
26+
├── tasks — TasksStore (user-managed)
27+
├── epics — EpicsStore (user-managed)
28+
├── skills — SkillsStore (user-managed)
29+
└── attachments — AttachmentsStore
30+
```
31+
32+
**Indexed stores** (code, docs, files) — populated by indexers, bulk upsert by file.
33+
**User-managed stores** (knowledge, tasks, epics, skills) — CRUD with slugs, versions,
34+
tags, attachments, optimistic locking.
35+
36+
## Directory Structure
37+
38+
```
39+
src/store/
40+
├── index.ts # Module exports
41+
├── types/
42+
│ ├── index.ts # Type re-exports
43+
│ ├── store.ts # Store, ProjectScopedStore interfaces
44+
│ ├── common.ts # SearchQuery, Edge, GraphName, VersionConflictError
45+
│ ├── code.ts # CodeStore, CodeNode, CodeFileEntry
46+
│ ├── docs.ts # DocsStore, DocNode, DocFileEntry
47+
│ ├── files.ts # FilesStore, FileNode
48+
│ ├── knowledge.ts # KnowledgeStore, NoteRecord, NoteDetail
49+
│ ├── tasks.ts # TasksStore, TaskRecord, TaskDetail
50+
│ ├── epics.ts # EpicsStore, EpicRecord, EpicDetail
51+
│ ├── skills.ts # SkillsStore, SkillRecord, SkillDetail
52+
│ ├── attachments.ts # AttachmentsStore, AttachmentMeta
53+
│ ├── projects.ts # ProjectsStore, ProjectRecord
54+
│ └── team.ts # TeamStore, TeamMemberRecord
55+
├── sqlite/
56+
│ ├── store.ts # SqliteStore — main implementation
57+
│ ├── stores/
58+
│ │ ├── project-scoped.ts # SqliteProjectScopedStore
59+
│ │ ├── projects.ts # SqliteProjectsStore
60+
│ │ ├── team.ts # SqliteTeamStore
61+
│ │ ├── knowledge.ts # SqliteKnowledgeStore
62+
│ │ ├── tasks.ts # SqliteTasksStore
63+
│ │ ├── epics.ts # SqliteEpicsStore
64+
│ │ ├── skills.ts # SqliteSkillsStore
65+
│ │ ├── code.ts # SqliteCodeStore
66+
│ │ ├── docs.ts # SqliteDocsStore
67+
│ │ ├── files.ts # SqliteFilesStore
68+
│ │ └── attachments.ts # SqliteAttachmentsStore
69+
│ ├── lib/
70+
│ │ ├── db.ts # openDatabase (WAL, sqlite-vec, FK)
71+
│ │ ├── migrate.ts # runMigrations (PRAGMA user_version)
72+
│ │ ├── bigint.ts # num(), now(), likeEscape(), chunk(), assertEmbeddingDim()
73+
│ │ ├── search.ts # hybridSearch (RRF fusion)
74+
│ │ ├── meta.ts # MetaHelper (namespaced key-value)
75+
│ │ ├── edge-helper.ts # EdgeHelper (cross-graph CRUD)
76+
│ │ └── entity-helpers.ts # EntityHelpers (tags, attachments, edges batch)
77+
│ └── migrations/
78+
│ └── v001.ts # Full schema (tables, FTS5, vec0, triggers, indexes)
79+
├── FINDINGS.md # sqlite-vec/FTS5 caveats, benchmarks
80+
├── README.md # This file
81+
├── SCHEMA.md # Database schema reference
82+
└── API.md # Public API reference
83+
```
84+
85+
## Key Design Decisions
86+
87+
### Transactions are external
88+
89+
Store methods are **not transactional** internally. The caller (orchestrator) wraps
90+
multi-step operations in `store.transaction()`:
91+
92+
```typescript
93+
store.transaction(() => {
94+
const task = scoped.tasks.create(data, embedding);
95+
scoped.epics.linkTask(epicId, task.id);
96+
});
97+
```
98+
99+
### Unified edges table
100+
101+
All relationships (same-graph and cross-graph) go through a single `edges` table
102+
with composite PK: `(project_id, from_graph, from_id, to_graph, to_id, kind)`.
103+
104+
### Tags as edges
105+
106+
Tags are separate entities linked via edges (`from_graph='tags'`, `kind='tagged'`).
107+
Orphaned tags are cleaned up when the last edge is removed.
108+
109+
### Hybrid search (RRF)
110+
111+
Search combines FTS5 keyword ranking with sqlite-vec cosine distance using
112+
Reciprocal Rank Fusion (K=60). Three modes: `hybrid`, `keyword`, `vector`.
113+
114+
### Optimistic locking
115+
116+
User-managed entities have `version` field. Pass `expectedVersion` to `update()`
117+
to detect conflicts — throws `VersionConflictError` on mismatch.
118+
119+
### Gap-based ordering
120+
121+
Tasks and epics use REAL `order` field with 1000-gap increments. Allows
122+
drag-and-drop reordering without renumbering all rows.
123+
124+
### Cleanup triggers
125+
126+
SQLite triggers on entity DELETE cascade to edges, attachments, and vec0 tables.
127+
vec0 cleanup must be in triggers because virtual tables don't support CASCADE.
128+
129+
## Usage
130+
131+
```typescript
132+
import { SqliteStore } from './store';
133+
134+
const store = new SqliteStore();
135+
store.open({
136+
dbPath: './data.db',
137+
embeddingDims: { // optional, default 384 for all
138+
code: 1024, // use larger model for code
139+
},
140+
});
141+
142+
// Workspace-level
143+
const project = store.projects.create({ slug: 'my-project', name: 'My Project', directory: '/path' });
144+
145+
// Project-scoped
146+
const scoped = store.project(project.id);
147+
148+
// CRUD with embedding
149+
const note = scoped.knowledge.create(
150+
{ title: 'Hello', content: 'World', tags: ['intro'] },
151+
embedding384,
152+
);
153+
154+
// Search
155+
const results = scoped.knowledge.search({
156+
text: 'hello',
157+
embedding: queryEmbedding,
158+
searchMode: 'hybrid',
159+
maxResults: 10,
160+
});
161+
162+
// Transaction (caller-managed)
163+
store.transaction(() => {
164+
scoped.tasks.create(taskData, taskEmbedding);
165+
scoped.tasks.create(taskData2, taskEmbedding2);
166+
});
167+
168+
store.close();
169+
```

0 commit comments

Comments
 (0)