From 0516ac8f4b1d6bf9bf21c575acdc51de988dcffa Mon Sep 17 00:00:00 2001 From: PatrickSys Date: Wed, 28 Jan 2026 19:07:36 +0100 Subject: [PATCH 1/3] feat(memory): v1.4.0 baseline - memory system with explicit recording - Add 'remember' and 'get_memory' tools for team knowledge capture - Auto-surface memories in search_codebase and get_team_patterns - Migrate storage to .codebase-context/ folder (auto-migrates legacy) - Add startup validation for ROOT_PATH before migration - Centralize constants to prevent path drift - Remove auto-created .gitignore (document recommendation instead) - Delete unused generate-assets scripts Breaking: None (additive only) --- .gitignore | 2 + CHANGELOG.md | 43 +- README.md | 99 ++++- package.json | 2 +- scripts/generate-assets.js | 98 ---- scripts/generate-assets.ts | 88 ---- src/analyzers/angular/index.ts | 6 +- src/constants/codebase-context.ts | 9 + src/core/indexer.ts | 21 +- src/core/search.ts | 16 +- src/index.ts | 420 +++++++++++++++++- src/memory/store.ts | 129 ++++++ src/storage/types.ts | 3 +- src/types/index.ts | 40 ++ src/utils/usage-tracker.ts | 4 +- tests/memory-store.test.ts | 81 ++++ tests/memory.test.ts | 153 +++++++ tests/searcher-corruption-propagation.test.ts | 17 +- 18 files changed, 984 insertions(+), 247 deletions(-) delete mode 100644 scripts/generate-assets.js delete mode 100644 scripts/generate-assets.ts create mode 100644 src/constants/codebase-context.ts create mode 100644 src/memory/store.ts create mode 100644 tests/memory-store.test.ts create mode 100644 tests/memory.test.ts diff --git a/.gitignore b/.gitignore index ca9370a..8f69e9d 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,8 @@ node_modules/ dist/ .codebase-index/ .codebase-index.json +.codebase-context/ +.codebase/ *.log .DS_Store .env diff --git a/CHANGELOG.md b/CHANGELOG.md index 4137a54..6a44f65 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,32 +1,71 @@ # Changelog +## [Unreleased] + +## [1.4.0] - 2026-01-28 + +### Added + +- **Memory System**: New `remember` and `get_memory` tools capture team conventions, decisions, and gotchas + - **Types**: `convention` | `decision` | `gotcha` + - **Categories**: `tooling`, `architecture`, `testing`, `dependencies`, `conventions` + - **Storage**: `.codebase-context/memory.json` with content-based hash IDs (commit this) + - **Safety**: `get_memory` truncates unfiltered results to 20 most recent +- **Integration with `get_team_patterns`**: Appends relevant memories when category overlaps +- **Integration with `search_codebase`**: Surfaces `relatedMemories` via keyword match in search results + +### Changed + +- **File Structure**: All MCP files now organized in `.codebase-context/` folder for cleaner project root + - Vector DB: `.codebase-index/` → `.codebase-context/index/` + - Intelligence: `.codebase-intelligence.json` → `.codebase-context/intelligence.json` + - Keyword index: `.codebase-index.json` → `.codebase-context/index.json` + - **Migration**: Automatic on server startup (legacy JSON preserved; vector DB directory moved) + +### Fixed + +- **Startup safety**: Validates `ROOT_PATH` before running migration to avoid creating directories on typo paths + +### Why This Feature + +Patterns show "what" (97% use inject) but not "why" (standalone compatibility). AGENTS.md can't capture every hard-won lesson. Decision memory gives AI agents access to the team's battle-tested rationale. + +**Design principle**: Tool must be self-evident without AGENTS.md rules. "Is this about HOW (record) vs WHAT (don't record)" + +**Inspired by**: v1.1 Pattern Momentum (temporal dimension) + memory systems research (Copilot Memory, Gemini Memory) ## [1.3.3] - 2026-01-18 ### Fixed + - **Security**: Resolve `pnpm audit` advisories by updating `hono` to 4.11.4 and removing the vulnerable `diff` transitive dependency (replaced `ts-node` with `tsx` for `pnpm dev`). ### Changed + - **Docs**: Clarify private `internal-docs/` submodule setup, add `npx --yes` tip, document `CODEBASE_ROOT`, and list `get_indexing_status` tool. - **Submodule**: Disable automatic updates for `internal-docs` (`update = none`). ### Removed + - **Dev**: Remove local-only `test-context.cjs` helper script. ## [1.3.2] - 2026-01-16 ### Changed + - **Embeddings**: Batch embedding now uses a single Transformers.js pipeline call per batch for higher throughput. - **Dependencies**: Bump `@modelcontextprotocol/sdk` to 1.25.2. ## [1.3.1] - 2026-01-05 ### Fixed + - **Auto-Heal Semantic Search**: Detects LanceDB schema corruption (missing `vector` column), triggers re-indexing, and retries search instead of silently falling back to keyword-only results. ## [1.3.0] - 2026-01-01 ### Added + - **Workspace Detection**: Monorepo support for Nx, Turborepo, Lerna, and pnpm workspaces - New utility: `src/utils/workspace-detection.ts` - Functions: `scanWorkspacePackageJsons()`, `detectWorkspaceType()`, `aggregateWorkspaceDependencies()` @@ -36,13 +75,16 @@ - **Dependency Detection**: Added `@nx/` and `@nrwl/` prefix matching for build tools ### Fixed + - **detectMetadata() bug**: All registered analyzers now contribute to codebase metadata (previously only the first analyzer was called) - Added `mergeMetadata()` helper with proper array deduplication and layer merging ### Changed + - Updated roadmap: v1.3 is now "Extensible Architecture Foundation" ### Acknowledgements + Thanks to [@aolin480](https://github.com/aolin480) for accelerating the workspace detection roadmap and identifying the detectMetadata() limitation in their fork. ## 1.2.2 (2025-12-31) @@ -55,7 +97,6 @@ Thanks to [@aolin480](https://github.com/aolin480) for accelerating the workspac ## 1.2.1 (2025-12-31) - ### Fixed - **MCP Protocol Compatibility**: Fixed stderr output during MCP STDIO handshake for strict clients diff --git a/README.md b/README.md index 1937afb..06c189a 100644 --- a/README.md +++ b/README.md @@ -28,54 +28,107 @@ If your environment prompts on first run, use `npx --yes ...` (or `npx -y ...`) - **Testing conventions** → `Jest`: 74%, `Playwright`: 6% - **Framework patterns** → Angular signals, standalone components, etc. - **Circular dependency detection** → Find toxic import cycles between files - +- **Memory system** → Record "why" behind choices so AI doesn't repeat mistakes ## How It Works When generating code, the agent checks your patterns first: -| Without MCP | With MCP | -|-------------|----------| -| Uses `constructor(private svc: Service)` | Uses `inject()` (97% team adoption) | -| Suggests `primeng/button` directly | Uses `@mycompany/ui-toolkit` wrapper | -| Generic Jest setup | Your team's actual test utilities | +| Without MCP | With MCP | +| ---------------------------------------- | ------------------------------------ | +| Uses `constructor(private svc: Service)` | Uses `inject()` (97% team adoption) | +| Suggests `primeng/button` directly | Uses `@mycompany/ui-toolkit` wrapper | +| Generic Jest setup | Your team's actual test utilities | ### Tip: Auto-invoke in your rules Add this to your `.cursorrules`, `CLAUDE.md`, or `AGENTS.md`: ``` -When generating or reviewing code, use codebase-context tools to check team patterns first. +## Codebase Context + +**At start of each task:** Call `get_memory` to load team conventions. + +**CRITICAL:** When user says "remember this" or "record this": +- STOP immediately and call `remember` tool FIRST +- DO NOT proceed with other actions until memory is recorded +- This is a blocking requirement, not optional ``` Now the agent checks patterns automatically instead of waiting for you to ask. ## Tools -| Tool | Purpose | -|------|---------| -| `search_codebase` | Semantic + keyword hybrid search | -| `get_component_usage` | Find where a library/component is used | -| `get_team_patterns` | Pattern frequencies + canonical examples | -| `get_codebase_metadata` | Project structure overview | -| `get_indexing_status` | Indexing progress + last stats | -| `get_style_guide` | Query style guide rules | -| `detect_circular_dependencies` | Find import cycles between files | -| `refresh_index` | Re-index the codebase | +| Tool | Purpose | +| ------------------------------ | --------------------------------------------- | +| `search_codebase` | Semantic + keyword hybrid search | +| `get_component_usage` | Find where a library/component is used | +| `get_team_patterns` | Pattern frequencies + canonical examples | +| `get_codebase_metadata` | Project structure overview | +| `get_indexing_status` | Indexing progress + last stats | +| `get_style_guide` | Query style guide rules | +| `detect_circular_dependencies` | Find import cycles between files | +| `remember` | Record memory (conventions/decisions/gotchas) | +| `get_memory` | Query recorded memory by category/keyword | +| `refresh_index` | Re-index the codebase | + +## File Structure + +The MCP creates the following structure in your project: + +``` +.codebase-context/ + ├── memory.json # Team knowledge (commit this) + ├── intelligence.json # Pattern analysis (generated) + ├── index.json # Keyword index (generated) + └── index/ # Vector database (generated) +``` + +**Recommended `.gitignore`:** The vector database and generated files can be large. Add this to your `.gitignore` to keep them local while sharing team memory: +```gitignore +# Codebase Context MCP - ignore generated files, keep memory +.codebase-context/* +!.codebase-context/memory.json +``` + +### Memory System + +Patterns tell you _what_ the team does ("97% use inject"), but not _why_ ("standalone compatibility"). Use `remember` to capture rationale that prevents repeated mistakes: + +```typescript +// AI won't change this again after recording the decision +remember({ + type: 'decision', + category: 'dependencies', + memory: 'Use node-linker: hoisted, not isolated', + reason: + "Some packages don't declare transitive deps. Isolated forces manual package.json additions." +}); +``` + +Memories surface automatically in `search_codebase` results and `get_team_patterns` responses. + +**Early baseline — known quirks:** + +- Agents may bundle multiple things into one entry +- Duplicates can happen if you record the same thing twice +- Edit `.codebase-context/memory.json` directly to clean up +- Be explicit: "Remember this: use X not Y" ## Configuration -| Variable | Default | Description | -|----------|---------|-------------| -| `EMBEDDING_PROVIDER` | `transformers` | `openai` (fast, cloud) or `transformers` (local, private) | -| `OPENAI_API_KEY` | - | Required if provider is `openai` | -| `CODEBASE_ROOT` | - | Project root to index (CLI arg takes precedence) | -| `CODEBASE_CONTEXT_DEBUG` | - | Set to `1` to enable verbose logging (startup messages, analyzer registration) | +| Variable | Default | Description | +| ------------------------ | -------------- | ------------------------------------------------------------------------------ | +| `EMBEDDING_PROVIDER` | `transformers` | `openai` (fast, cloud) or `transformers` (local, private) | +| `OPENAI_API_KEY` | - | Required if provider is `openai` | +| `CODEBASE_ROOT` | - | Project root to index (CLI arg takes precedence) | +| `CODEBASE_CONTEXT_DEBUG` | - | Set to `1` to enable verbose logging (startup messages, analyzer registration) | ## Performance Note This tool runs **locally** on your machine using your hardware. + - **Initial Indexing**: The first run works hard. It may take several minutes (e.g., ~2-5 mins for 30k files) to compute embeddings for your entire codebase. - **Caching**: Subsequent queries are instant (milliseconds). - **Updates**: Currently, `refresh_index` re-scans the codebase. True incremental indexing (processing only changed files) is on the roadmap. diff --git a/package.json b/package.json index 1383e2c..c11833d 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "codebase-context", - "version": "1.3.3", + "version": "1.4.0", "description": "MCP server that helps AI agents understand your codebase - patterns, libraries, architecture, monorepo support", "type": "module", "main": "./dist/lib.js", diff --git a/scripts/generate-assets.js b/scripts/generate-assets.js deleted file mode 100644 index a244032..0000000 --- a/scripts/generate-assets.js +++ /dev/null @@ -1,98 +0,0 @@ - -import { CodebaseIndexer } from '../dist/core/indexer.js'; -import { CodebaseSearcher } from '../dist/core/search.js'; -import { analyzerRegistry } from '../dist/core/analyzer-registry.js'; -import { AngularAnalyzer } from '../dist/analyzers/angular/index.js'; -import { GenericAnalyzer } from '../dist/analyzers/generic/index.js'; -import path from 'path'; -import fs from 'fs/promises'; - -async function main() { - console.log("Registering Analyzers..."); - analyzerRegistry.register(new AngularAnalyzer()); - analyzerRegistry.register(new GenericAnalyzer()); - - const targetArg = process.argv[2]; - if (!targetArg) { - console.error("Please provide a target code path"); - process.exit(1); - } - - const targetPath = path.resolve(targetArg); - console.log(`Generating assets for: ${targetPath}`); - - // 1. Indexing (Generates .codebase-intelligence.json) - console.log("--- 1. Running Indexer ---"); - const indexer = new CodebaseIndexer({ - rootPath: targetPath, - config: { - respectGitignore: false, - include: ["**/*.ts"], - exclude: ["**/node_modules/**", "**/dist/**", "**/*.spec.ts"], - skipEmbedding: true // CRITICAL for speed - } - }); - - const stats = await indexer.index(); - console.log(`Indexed ${stats.indexedFiles} files.`); - - // 2. Read Generated Intelligence - console.log("--- 2. Reading Intelligence ---"); - const intelligencePath = path.join(targetPath, ".codebase-intelligence.json"); - const intelligenceRaw = await fs.readFile(intelligencePath, 'utf-8'); - const intelligence = JSON.parse(intelligenceRaw); - - const patterns = intelligence.patterns || {}; - const libraryUsage = intelligence.libraryUsage || {}; - - // 3. Search Simulation (Mocked, since no embeddings) - console.log("--- 3. Simulating Searches ---"); - // const searcher = new CodebaseSearcher(targetPath); - - const searchQuery = "logging utility"; - // const searchResults = await searcher.search(searchQuery, 3); - const searchResults = []; // Mocked empty results since we skipped embedding - - // 4. Output Report - const report = ` -# Marketing Assets & Validation Report -Generated for: ${targetPath} -Date: ${new Date().toISOString()} - -## 1. get_team_patterns Output (Real Data) - -\`\`\`json -${JSON.stringify({ patterns }, null, 2)} -\`\`\` - -## 2. get_component_usage ("@company/utils") - -\`\`\`json -${JSON.stringify({ - source: "@company/utils", - count: libraryUsage["@company/utils"]?.count || 0, - files: libraryUsage["@company/utils"]?.files?.slice(0, 5) - }, null, 2)} -\`\`\` - -## 3. Token Reduction Proxy Test - -**Scenario**: Search for "${searchQuery}" - -**Without MCP (Standard Search)**: -- Would return generic string matches. -- Estimate: ~10 results * ~500 tokens context + overhead = ~5000+ tokens to verify. - -**With MCP (Semantic + Usage)**: -- Tool call: \`get_component_usage("@company/utils")\` -- Result: Exact usage count and locations. -- Tokens: ~200 (Structured Response). Matches found: ${libraryUsage["@company/utils"]?.count || "Unknown"}. - -`; - - const outputPath = path.join(process.cwd(), 'internal-docs', 'marketing-assets.md'); - await fs.writeFile(outputPath, report); - console.log(`Assets written to: ${outputPath}`); -} - -main().catch(console.error); diff --git a/scripts/generate-assets.ts b/scripts/generate-assets.ts deleted file mode 100644 index e5510d4..0000000 --- a/scripts/generate-assets.ts +++ /dev/null @@ -1,88 +0,0 @@ - -import { CodebaseIndexer } from '../src/core/indexer.js'; -import { CodebaseSearcher } from '../src/core/search.js'; -import path from 'path'; -import fs from 'fs/promises'; - -async function main() { - const targetArg = process.argv[2]; - if (!targetArg) { - console.error("Please provide a target code path"); - process.exit(1); - } - - const targetPath = path.resolve(targetArg); - console.log(`Generating assets for: ${targetPath}`); - - // 1. Indexing (Generates .codebase-intelligence.json) - console.log("--- 1. Running Indexer ---"); - const indexer = new CodebaseIndexer({ rootPath: targetPath }); - const stats = await indexer.index(); - console.log(`Indexed ${stats.indexedFiles} files.`); - - // 2. Read Generated Intelligence (Simulate get_team_patterns / get_component_usage) - console.log("--- 2. Reading Intelligence ---"); - const intelligencePath = path.join(targetPath, ".codebase-intelligence.json"); - const intelligenceRaw = await fs.readFile(intelligencePath, 'utf-8'); - const intelligence = JSON.parse(intelligenceRaw); - - const patterns = intelligence.patterns || {}; - const libraryUsage = intelligence.libraryUsage || {}; - - // 3. Search Simulation (Simulate "Token Reduction") - console.log("--- 3. Simulating Searches ---"); - const searcher = new CodebaseSearcher(targetPath); - - // Scenario A: Find usage of a known internal wrapper or library - // Search for a common internal library pattern - const searchQuery = "logging utility"; - const searchResults = await searcher.search(searchQuery, 3); - - // 4. Output Report - const report = ` -# Marketing Assets & Validation Report -Generated for: ${targetPath} -Date: ${new Date().toISOString()} - -## 1. get_team_patterns Output (Real Data) - -\`\`\`json -${JSON.stringify({ patterns }, null, 2)} -\`\`\` - -## 2. get_component_usage ("@company/utils") - -\`\`\`json -${JSON.stringify({ - source: "@company/utils", - count: libraryUsage["@company/utils"]?.count || 0, - files: libraryUsage["@company/utils"]?.files?.slice(0, 5) // Truncated for display - }, null, 2)} -\`\`\` - -## 3. Token Reduction Proxy Test - -**Scenario**: Search for "${searchQuery}" - -**Without MCP (Standard Search)**: -- Would return generic string matches. -- Requires reading file content to verify. -- Estimated tokens: ~2,000 (listing hits) + ~5,000 (reading 2-3 files). - -**With MCP (Semantic + Usage)**: -- Tool call: \`get_component_usage("@company/utils")\` (if known) or \`search_codebase\` -- Search Results found: ${searchResults.length} -- Top Result Base Score: ${searchResults[0]?.score?.toFixed(2) || 'N/A'} -- Snippet Size: ~200 tokens. - -**Conclusion**: -- MCP provides structured "facts" (patterns) instantly (0 search steps). -- Wrapper detection turns "search" into "lookup" (Immediate answer). -`; - - const outputPath = path.join(process.cwd(), 'internal-docs', 'marketing-assets.md'); - await fs.writeFile(outputPath, report); - console.log(`Assets written to: ${outputPath}`); -} - -main().catch(console.error); diff --git a/src/analyzers/angular/index.ts b/src/analyzers/angular/index.ts index 53b5050..2fbc211 100644 --- a/src/analyzers/angular/index.ts +++ b/src/analyzers/angular/index.ts @@ -18,6 +18,10 @@ import { } from '../../types/index.js'; import { parse } from '@typescript-eslint/typescript-estree'; import { createChunksFromCode } from '../../utils/chunking.js'; +import { + CODEBASE_CONTEXT_DIRNAME, + KEYWORD_INDEX_FILENAME +} from '../../constants/codebase-context.js'; export class AngularAnalyzer implements FrameworkAnalyzer { readonly name = 'angular'; @@ -867,7 +871,7 @@ export class AngularAnalyzer implements FrameworkAnalyzer { // Calculate statistics from existing index if available try { - const indexPath = path.join(rootPath, '.codebase-index.json'); + const indexPath = path.join(rootPath, CODEBASE_CONTEXT_DIRNAME, KEYWORD_INDEX_FILENAME); const indexContent = await fs.readFile(indexPath, 'utf-8'); const chunks = JSON.parse(indexContent); diff --git a/src/constants/codebase-context.ts b/src/constants/codebase-context.ts new file mode 100644 index 0000000..92bbc41 --- /dev/null +++ b/src/constants/codebase-context.ts @@ -0,0 +1,9 @@ +// Centralized constants for on-disk MCP artifacts. +// Keep this module dependency-free to avoid import cycles. + +export const CODEBASE_CONTEXT_DIRNAME = '.codebase-context' as const; + +export const MEMORY_FILENAME = 'memory.json' as const; +export const INTELLIGENCE_FILENAME = 'intelligence.json' as const; +export const KEYWORD_INDEX_FILENAME = 'index.json' as const; +export const VECTOR_DB_DIRNAME = 'index' as const; diff --git a/src/core/indexer.ts b/src/core/indexer.ts index f7112af..7416587 100644 --- a/src/core/indexer.ts +++ b/src/core/indexer.ts @@ -27,6 +27,12 @@ import { FileExport } from '../utils/usage-tracker.js'; import { getFileCommitDates } from '../utils/git-dates.js'; +import { + CODEBASE_CONTEXT_DIRNAME, + INTELLIGENCE_FILENAME, + KEYWORD_INDEX_FILENAME, + VECTOR_DB_DIRNAME +} from '../constants/codebase-context.js'; export interface IndexerOptions { rootPath: string; @@ -389,23 +395,26 @@ export class CodebaseIndexer { // Phase 4: Storing this.updateProgress('storing', 75); + const contextDir = path.join(this.rootPath, CODEBASE_CONTEXT_DIRNAME); + await fs.mkdir(contextDir, { recursive: true }); + if (!this.config.skipEmbedding) { console.error(`Storing ${chunksToEmbed.length} chunks...`); // Store in LanceDB for vector search - const storagePath = path.join(this.rootPath, '.codebase-index'); + const storagePath = path.join(contextDir, VECTOR_DB_DIRNAME); const storageProvider = await getStorageProvider({ path: storagePath }); await storageProvider.clear(); // Clear existing index await storageProvider.store(chunksWithEmbeddings); } // Also save JSON for keyword search (Fuse.js) - use chunksToEmbed for consistency - const indexPath = path.join(this.rootPath, '.codebase-index.json'); + const indexPath = path.join(contextDir, KEYWORD_INDEX_FILENAME); // Write without pretty-printing to save memory await fs.writeFile(indexPath, JSON.stringify(chunksToEmbed)); // Save library usage and pattern stats - const intelligencePath = path.join(this.rootPath, '.codebase-intelligence.json'); + const intelligencePath = path.join(contextDir, INTELLIGENCE_FILENAME); const libraryStats = libraryTracker.getStats(); // Extract tsconfig paths for AI to understand import aliases @@ -594,7 +603,11 @@ export class CodebaseIndexer { // Load intelligence data if available try { - const intelligencePath = path.join(this.rootPath, '.codebase-intelligence.json'); + const intelligencePath = path.join( + this.rootPath, + CODEBASE_CONTEXT_DIRNAME, + INTELLIGENCE_FILENAME + ); const intelligenceContent = await fs.readFile(intelligencePath, 'utf-8'); const intelligence = JSON.parse(intelligenceContent); diff --git a/src/core/search.ts b/src/core/search.ts index d42e304..b543b6c 100644 --- a/src/core/search.ts +++ b/src/core/search.ts @@ -10,6 +10,12 @@ import { EmbeddingProvider, getEmbeddingProvider } from '../embeddings/index.js' import { VectorStorageProvider, getStorageProvider } from '../storage/index.js'; import { analyzerRegistry } from './analyzer-registry.js'; import { IndexCorruptedError } from '../errors/index.js'; +import { + CODEBASE_CONTEXT_DIRNAME, + INTELLIGENCE_FILENAME, + KEYWORD_INDEX_FILENAME, + VECTOR_DB_DIRNAME +} from '../constants/codebase-context.js'; export interface SearchOptions { useSemanticSearch?: boolean; @@ -46,7 +52,7 @@ export class CodebaseSearcher { constructor(rootPath: string) { this.rootPath = rootPath; - this.storagePath = path.join(rootPath, '.codebase-index'); + this.storagePath = path.join(rootPath, CODEBASE_CONTEXT_DIRNAME, VECTOR_DB_DIRNAME); } async initialize(): Promise { @@ -73,7 +79,7 @@ export class CodebaseSearcher { private async loadKeywordIndex(): Promise { try { - const indexPath = path.join(this.rootPath, '.codebase-index.json'); + const indexPath = path.join(this.rootPath, CODEBASE_CONTEXT_DIRNAME, KEYWORD_INDEX_FILENAME); const content = await fs.readFile(indexPath, 'utf-8'); this.chunks = JSON.parse(content); @@ -104,7 +110,11 @@ export class CodebaseSearcher { */ private async loadPatternIntelligence(): Promise { try { - const intelligencePath = path.join(this.rootPath, '.codebase-intelligence.json'); + const intelligencePath = path.join( + this.rootPath, + CODEBASE_CONTEXT_DIRNAME, + INTELLIGENCE_FILENAME + ); const content = await fs.readFile(intelligencePath, 'utf-8'); const intelligence = JSON.parse(content); diff --git a/src/index.ts b/src/index.ts index 5cc54c2..b8410e0 100644 --- a/src/index.ts +++ b/src/index.ts @@ -20,13 +20,32 @@ import { Resource } from '@modelcontextprotocol/sdk/types.js'; import { CodebaseIndexer } from './core/indexer.js'; -import { IndexingStats, SearchResult } from './types/index.js'; +import type { + IndexingStats, + SearchResult, + Memory, + MemoryCategory, + MemoryType +} from './types/index.js'; import { CodebaseSearcher } from './core/search.js'; import { analyzerRegistry } from './core/analyzer-registry.js'; import { AngularAnalyzer } from './analyzers/angular/index.js'; import { GenericAnalyzer } from './analyzers/generic/index.js'; import { InternalFileGraph } from './utils/usage-tracker.js'; import { IndexCorruptedError } from './errors/index.js'; +import { + CODEBASE_CONTEXT_DIRNAME, + MEMORY_FILENAME, + INTELLIGENCE_FILENAME, + KEYWORD_INDEX_FILENAME, + VECTOR_DB_DIRNAME +} from './constants/codebase-context.js'; +import { + appendMemoryFile, + readMemoriesFile, + filterMemories, + applyUnfilteredLimit +} from './memory/store.js'; analyzerRegistry.register(new AngularAnalyzer()); analyzerRegistry.register(new GenericAnalyzer()); @@ -51,6 +70,87 @@ function resolveRootPath(): string { const ROOT_PATH = resolveRootPath(); +// File paths (new structure) +const PATHS = { + baseDir: path.join(ROOT_PATH, CODEBASE_CONTEXT_DIRNAME), + memory: path.join(ROOT_PATH, CODEBASE_CONTEXT_DIRNAME, MEMORY_FILENAME), + intelligence: path.join(ROOT_PATH, CODEBASE_CONTEXT_DIRNAME, INTELLIGENCE_FILENAME), + keywordIndex: path.join(ROOT_PATH, CODEBASE_CONTEXT_DIRNAME, KEYWORD_INDEX_FILENAME), + vectorDb: path.join(ROOT_PATH, CODEBASE_CONTEXT_DIRNAME, VECTOR_DB_DIRNAME) +}; + +// Legacy paths for migration +const LEGACY_PATHS = { + // Pre-v1.5 + intelligence: path.join(ROOT_PATH, '.codebase-intelligence.json'), + keywordIndex: path.join(ROOT_PATH, '.codebase-index.json'), + vectorDb: path.join(ROOT_PATH, '.codebase-index') +}; + +/** + * Check if file/directory exists + */ +async function fileExists(filePath: string): Promise { + try { + await fs.access(filePath); + return true; + } catch { + return false; + } +} + +/** + * Migrate legacy file structure to .codebase-context/ folder. + * Idempotent, fail-safe. Rollback compatibility is not required. + */ +async function migrateToNewStructure(): Promise { + let migrated = false; + + try { + await fs.mkdir(PATHS.baseDir, { recursive: true }); + + // intelligence.json + if (!(await fileExists(PATHS.intelligence))) { + if (await fileExists(LEGACY_PATHS.intelligence)) { + await fs.copyFile(LEGACY_PATHS.intelligence, PATHS.intelligence); + migrated = true; + if (process.env.CODEBASE_CONTEXT_DEBUG) { + console.error('[DEBUG] Migrated intelligence.json'); + } + } + } + + // index.json (keyword index) + if (!(await fileExists(PATHS.keywordIndex))) { + if (await fileExists(LEGACY_PATHS.keywordIndex)) { + await fs.copyFile(LEGACY_PATHS.keywordIndex, PATHS.keywordIndex); + migrated = true; + if (process.env.CODEBASE_CONTEXT_DEBUG) { + console.error('[DEBUG] Migrated index.json'); + } + } + } + + // Vector DB directory + if (!(await fileExists(PATHS.vectorDb))) { + if (await fileExists(LEGACY_PATHS.vectorDb)) { + await fs.rename(LEGACY_PATHS.vectorDb, PATHS.vectorDb); + migrated = true; + if (process.env.CODEBASE_CONTEXT_DEBUG) { + console.error('[DEBUG] Migrated vector database'); + } + } + } + + return migrated; + } catch (error) { + if (process.env.CODEBASE_CONTEXT_DEBUG) { + console.error('[DEBUG] Migration error:', error); + } + return false; + } +} + export interface IndexState { status: 'idle' | 'indexing' | 'ready' | 'error'; lastIndexed?: Date; @@ -66,7 +166,7 @@ const indexState: IndexState = { const server: Server = new Server( { name: 'codebase-context', - version: '1.3.3' + version: '1.4.0' }, { capabilities: { @@ -237,6 +337,71 @@ const TOOLS: Tool[] = [ } } } + }, + { + name: 'remember', + description: + '📝 CALL IMMEDIATELY when user explicitly asks to remember/record something.\n\n' + + 'USER TRIGGERS:\n' + + '• "Remember this: [X]"\n' + + '• "Record this: [Y]"\n' + + '• "Save this for next time: [Z]"\n\n' + + '⚠️ DO NOT call unless user explicitly requests it.\n\n' + + 'HOW TO WRITE:\n' + + '• ONE convention per memory (if user lists 5 things, call this 5 times)\n' + + '• memory: 5-10 words (the specific rule)\n' + + '• reason: 1 sentence (why it matters)\n' + + '• Skip: one-time features, code examples, essays', + inputSchema: { + type: 'object', + properties: { + type: { + type: 'string', + enum: ['convention', 'decision', 'gotcha'], + description: 'Type of memory being recorded' + }, + category: { + type: 'string', + description: 'Broader category for filtering', + enum: ['tooling', 'architecture', 'testing', 'dependencies', 'conventions'] + }, + memory: { + type: 'string', + description: 'What to remember (concise)' + }, + reason: { + type: 'string', + description: 'Why this matters or what breaks otherwise' + } + }, + required: ['type', 'category', 'memory', 'reason'] + } + }, + { + name: 'get_memory', + description: + 'Retrieves team conventions, architectural decisions, and known gotchas.\n' + + 'CALL BEFORE suggesting patterns, libraries, or architecture.\n\n' + + 'Filters: category (tooling/architecture/testing/dependencies/conventions), type (convention/decision/gotcha), query (keyword search).', + inputSchema: { + type: 'object', + properties: { + category: { + type: 'string', + description: 'Filter by category', + enum: ['tooling', 'architecture', 'testing', 'dependencies', 'conventions'] + }, + type: { + type: 'string', + description: 'Filter by memory type', + enum: ['convention', 'decision', 'gotcha'] + }, + query: { + type: 'string', + description: 'Keyword search across memory and reason' + } + } + } } ]; @@ -261,7 +426,7 @@ server.setRequestHandler(ListResourcesRequestSchema, async () => { }); async function generateCodebaseContext(): Promise { - const intelligencePath = path.join(ROOT_PATH, '.codebase-intelligence.json'); + const intelligencePath = PATHS.intelligence; try { const content = await fs.readFile(intelligencePath, 'utf-8'); @@ -437,7 +602,7 @@ async function performIndexing(): Promise { } async function shouldReindex(): Promise { - const indexPath = path.join(ROOT_PATH, '.codebase-index.json'); + const indexPath = PATHS.keywordIndex; try { await fs.access(indexPath); return false; @@ -549,6 +714,19 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { } } + // Load memories for keyword matching + const allMemories = await readMemoriesFile(PATHS.memory); + + const findRelatedMemories = (queryTerms: string[]): Memory[] => { + return allMemories.filter((m) => { + const searchText = `${m.memory} ${m.reason}`.toLowerCase(); + return queryTerms.some((term) => searchText.includes(term)); + }); + }; + + const queryTerms = query.toLowerCase().split(/\s+/); + const relatedMemories = findRelatedMemories(queryTerms); + return { content: [ { @@ -565,11 +743,11 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { componentType: r.componentType, layer: r.layer, framework: r.framework, - // v1.2: Pattern momentum awareness trend: r.trend, patternWarning: r.patternWarning })), - totalResults: results.length + totalResults: results.length, + ...(relatedMemories.length > 0 && { relatedMemories }) }, null, 2 @@ -660,7 +838,7 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { // Load team patterns from intelligence file let teamPatterns = {}; try { - const intelligencePath = path.join(ROOT_PATH, '.codebase-intelligence.json'); + const intelligencePath = PATHS.intelligence; const intelligenceContent = await fs.readFile(intelligencePath, 'utf-8'); const intelligence = JSON.parse(intelligenceContent); @@ -814,7 +992,7 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { const { category } = args as { category?: string }; try { - const intelligencePath = path.join(ROOT_PATH, '.codebase-intelligence.json'); + const intelligencePath = PATHS.intelligence; const content = await fs.readFile(intelligencePath, 'utf-8'); const intelligence = JSON.parse(content); @@ -840,6 +1018,31 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { } } + // Load and append matching memories + try { + const allMemories = await readMemoriesFile(PATHS.memory); + + // Map pattern categories to decision categories + const categoryMap: Record = { + all: ['tooling', 'architecture', 'testing', 'dependencies', 'conventions'], + di: ['architecture', 'conventions'], + state: ['architecture', 'conventions'], + testing: ['testing'], + libraries: ['dependencies'] + }; + + const relevantCategories = categoryMap[category || 'all'] || []; + const matchingMemories = allMemories.filter((m) => + relevantCategories.includes(m.category) + ); + + if (matchingMemories.length > 0) { + result.memories = matchingMemories; + } + } catch (error) { + // No memory file yet, that's fine - don't fail the whole request + } + return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] }; @@ -867,7 +1070,7 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { const { name: componentName } = args as { name: string }; try { - const intelligencePath = path.join(ROOT_PATH, '.codebase-intelligence.json'); + const intelligencePath = PATHS.intelligence; const content = await fs.readFile(intelligencePath, 'utf-8'); const intelligence = JSON.parse(content); @@ -950,7 +1153,7 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { const { scope } = args as { scope?: string }; try { - const intelligencePath = path.join(ROOT_PATH, '.codebase-intelligence.json'); + const intelligencePath = PATHS.intelligence; const content = await fs.readFile(intelligencePath, 'utf-8'); const intelligence = JSON.parse(content); @@ -1045,6 +1248,166 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { } } + case 'remember': { + const args_typed = args as { + type?: MemoryType; + category: MemoryCategory; + memory: string; + reason: string; + }; + + const { type = 'decision', category, memory, reason } = args_typed; + + try { + const crypto = await import('crypto'); + const memoryPath = PATHS.memory; + + const hashContent = `${type}:${category}:${memory}:${reason}`; + const hash = crypto.createHash('sha256').update(hashContent).digest('hex'); + const id = hash.substring(0, 12); + + const newMemory: Memory = { + id, + type, + category, + memory, + reason, + date: new Date().toISOString() + }; + + const result = await appendMemoryFile(memoryPath, newMemory); + + if (result.status === 'duplicate') { + return { + content: [ + { + type: 'text', + text: JSON.stringify( + { + status: 'info', + message: 'This memory was already recorded.', + memory: result.memory + }, + null, + 2 + ) + } + ] + }; + } + + return { + content: [ + { + type: 'text', + text: JSON.stringify( + { + status: 'success', + message: 'Memory recorded successfully.', + memory: result.memory + }, + null, + 2 + ) + } + ] + }; + } catch (error) { + return { + content: [ + { + type: 'text', + text: JSON.stringify( + { + status: 'error', + message: 'Failed to record memory.', + error: error instanceof Error ? error.message : String(error) + }, + null, + 2 + ) + } + ] + }; + } + } + + case 'get_memory': { + const { category, type, query } = args as { + category?: MemoryCategory; + type?: MemoryType; + query?: string; + }; + + try { + const memoryPath = PATHS.memory; + const allMemories = await readMemoriesFile(memoryPath); + + if (allMemories.length === 0) { + return { + content: [ + { + type: 'text', + text: JSON.stringify( + { + status: 'success', + message: + "No team conventions recorded yet. Use 'remember' to build tribal knowledge or memory when the user corrects you over a repeatable pattern.", + memories: [], + count: 0 + }, + null, + 2 + ) + } + ] + }; + } + + const filtered = filterMemories(allMemories, { category, type, query }); + const limited = applyUnfilteredLimit(filtered, { category, type, query }, 20); + + return { + content: [ + { + type: 'text', + text: JSON.stringify( + { + status: 'success', + count: limited.memories.length, + totalCount: limited.totalCount, + truncated: limited.truncated, + message: limited.truncated + ? 'Showing 20 most recent. Use filters (category/type/query) for targeted results.' + : undefined, + memories: limited.memories + }, + null, + 2 + ) + } + ] + }; + } catch (error) { + return { + content: [ + { + type: 'text', + text: JSON.stringify( + { + status: 'error', + message: 'Failed to retrieve memories.', + error: error instanceof Error ? error.message : String(error) + }, + null, + 2 + ) + } + ] + }; + } + } + default: return { content: [ @@ -1083,18 +1446,6 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { }); async function main() { - // Server startup banner (guarded to avoid stderr during MCP STDIO handshake) - if (process.env.CODEBASE_CONTEXT_DEBUG) { - console.error('[DEBUG] Codebase Context MCP Server'); - console.error(`[DEBUG] Root: ${ROOT_PATH}`); - console.error( - `[DEBUG] Analyzers: ${analyzerRegistry - .getAll() - .map((a) => a.name) - .join(', ')}` - ); - } - // Validate root path exists and is a directory try { const stats = await fs.stat(ROOT_PATH); @@ -1109,6 +1460,31 @@ async function main() { process.exit(1); } + // Migrate legacy structure before server starts + try { + const migrated = await migrateToNewStructure(); + if (migrated && process.env.CODEBASE_CONTEXT_DEBUG) { + console.error('[DEBUG] Migrated to .codebase-context/ structure'); + } + } catch (error) { + // Non-fatal: continue with current paths + if (process.env.CODEBASE_CONTEXT_DEBUG) { + console.error('[DEBUG] Migration failed:', error); + } + } + + // Server startup banner (guarded to avoid stderr during MCP STDIO handshake) + if (process.env.CODEBASE_CONTEXT_DEBUG) { + console.error('[DEBUG] Codebase Context MCP Server'); + console.error(`[DEBUG] Root: ${ROOT_PATH}`); + console.error( + `[DEBUG] Analyzers: ${analyzerRegistry + .getAll() + .map((a) => a.name) + .join(', ')}` + ); + } + // Check for package.json to confirm it's a project root (guarded to avoid stderr during handshake) if (process.env.CODEBASE_CONTEXT_DEBUG) { try { diff --git a/src/memory/store.ts b/src/memory/store.ts new file mode 100644 index 0000000..358fbaf --- /dev/null +++ b/src/memory/store.ts @@ -0,0 +1,129 @@ +import { promises as fs } from 'fs'; +import path from 'path'; +import type { Memory, MemoryCategory, MemoryType } from '../types/index.js'; + +type RawMemory = Partial<{ + id: unknown; + type: unknown; + category: unknown; + memory: unknown; + decision: unknown; + reason: unknown; + date: unknown; +}>; + +export type MemoryFilters = { + category?: MemoryCategory; + type?: MemoryType; + query?: string; +}; + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null; +} + +export function normalizeMemory(raw: unknown): Memory | null { + if (!isRecord(raw)) return null; + const m = raw as RawMemory; + + const id = typeof m.id === 'string' ? m.id : undefined; + const type = typeof m.type === 'string' ? (m.type as MemoryType) : 'decision'; + const category = typeof m.category === 'string' ? (m.category as MemoryCategory) : undefined; + const memory = + typeof m.memory === 'string' + ? m.memory + : typeof m.decision === 'string' + ? m.decision + : undefined; + const reason = typeof m.reason === 'string' ? m.reason : undefined; + const date = typeof m.date === 'string' ? m.date : undefined; + + if (!id || !category || !memory || !reason || !date) return null; + + return { id, type, category, memory, reason, date }; +} + +export function normalizeMemories(raw: unknown): Memory[] { + if (!Array.isArray(raw)) return []; + const out: Memory[] = []; + for (const item of raw) { + const normalized = normalizeMemory(item); + if (normalized) out.push(normalized); + } + return out; +} + +export async function readMemoriesFile(memoryPath: string): Promise { + try { + const content = await fs.readFile(memoryPath, 'utf-8'); + return normalizeMemories(JSON.parse(content)); + } catch { + return []; + } +} + +export async function writeMemoriesFile(memoryPath: string, memories: Memory[]): Promise { + await fs.mkdir(path.dirname(memoryPath), { recursive: true }); + await fs.writeFile(memoryPath, JSON.stringify(memories, null, 2)); +} + +export async function appendMemoryFile( + memoryPath: string, + memory: Memory +): Promise<{ status: 'added' | 'duplicate'; memory: Memory }> { + const existing = await readMemoriesFile(memoryPath); + const found = existing.find((m) => m.id === memory.id); + if (found) return { status: 'duplicate', memory: found }; + existing.push(memory); + await writeMemoriesFile(memoryPath, existing); + return { status: 'added', memory }; +} + +export function filterMemories(memories: Memory[], filters: MemoryFilters): Memory[] { + const { category, type, query } = filters; + let filtered = memories; + + if (type) filtered = filtered.filter((m) => m.type === type); + if (category) filtered = filtered.filter((m) => m.category === category); + + if (query) { + const terms = query.toLowerCase().split(/\s+/).filter(Boolean); + if (terms.length > 0) { + filtered = filtered.filter((m) => { + const haystack = `${m.memory} ${m.reason}`.toLowerCase(); + return terms.some((t) => haystack.includes(t)); + }); + } + } + + return filtered; +} + +export function sortMemoriesByRecency(memories: Memory[]): Memory[] { + const withIndex = memories.map((m, i) => ({ m, i })); + withIndex.sort((a, b) => { + const ad = Date.parse(a.m.date); + const bd = Date.parse(b.m.date); + const aTime = Number.isFinite(ad) ? ad : 0; + const bTime = Number.isFinite(bd) ? bd : 0; + if (aTime !== bTime) return bTime - aTime; + return a.i - b.i; + }); + return withIndex.map((x) => x.m); +} + +export function applyUnfilteredLimit( + memories: Memory[], + filters: MemoryFilters, + limit: number +): { memories: Memory[]; truncated: boolean; totalCount: number } { + const totalCount = memories.length; + const hasFilters = Boolean( + filters.category || filters.type || (filters.query && filters.query.trim()) + ); + if (hasFilters || totalCount <= limit) { + return { memories, truncated: false, totalCount }; + } + const sorted = sortMemoriesByRecency(memories); + return { memories: sorted.slice(0, limit), truncated: true, totalCount }; +} diff --git a/src/storage/types.ts b/src/storage/types.ts index 80ecd4e..9e6340c 100644 --- a/src/storage/types.ts +++ b/src/storage/types.ts @@ -3,6 +3,7 @@ */ import { CodeChunk, SearchFilters } from '../types/index.js'; +import { CODEBASE_CONTEXT_DIRNAME, VECTOR_DB_DIRNAME } from '../constants/codebase-context.js'; export interface VectorStorageProvider { readonly name: string; @@ -59,5 +60,5 @@ export interface StorageConfig { export const DEFAULT_STORAGE_CONFIG: StorageConfig = { provider: 'lancedb', - path: '.codebase-index' + path: `${CODEBASE_CONTEXT_DIRNAME}/${VECTOR_DB_DIRNAME}` }; diff --git a/src/types/index.ts b/src/types/index.ts index 9cade54..9247fec 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -521,3 +521,43 @@ export interface Parameter { defaultValue?: string; decorators?: Decorator[]; } + +// Memory System + +/** + * Category of architectural/design decision + * Used for organizing and querying decisions + */ +export type MemoryCategory = + | 'tooling' // Build tools, package managers, linting, IDE config + | 'architecture' // Layers, folder structure, module boundaries + | 'testing' // Test frameworks, mocking strategies, coverage + | 'dependencies' // Library choices, wrappers, versioning, package management + | 'conventions'; // Naming, style, organization not captured in patterns + +/** + * Type of knowledge being recorded + */ +export type MemoryType = + | 'convention' // Style, naming, component preferences + | 'decision' // Architecture/tooling choices with rationale + | 'gotcha'; // Things that break and why + +/** + * A recorded architectural or design decision + * Captures the "why" behind choices to prevent AI agents from repeating mistakes + */ +export interface Memory { + /** Content-based hash ID (first 12 chars of SHA-256) */ + id: string; + /** Type of knowledge: convention, decision, or gotcha */ + type: MemoryType; + /** Category for organization and filtering */ + category: MemoryCategory; + /** Brief description of what to remember */ + memory: string; + /** Why this decision was made - the rationale/context */ + reason: string; + /** ISO 8601 date when decision was recorded */ + date: string; +} diff --git a/src/utils/usage-tracker.ts b/src/utils/usage-tracker.ts index 06935e8..2f350b4 100644 --- a/src/utils/usage-tracker.ts +++ b/src/utils/usage-tracker.ts @@ -834,7 +834,7 @@ export class InternalFileGraph { } /** - * Serialize for persistence to .codebase-intelligence.json + * Serialize for persistence to .codebase-context/intelligence.json */ toJSON(): { imports: Record; @@ -855,7 +855,7 @@ export class InternalFileGraph { } /** - * Restore from JSON (for loading from .codebase-intelligence.json) + * Restore from JSON (for loading from .codebase-context/intelligence.json) */ static fromJSON( data: { diff --git a/tests/memory-store.test.ts b/tests/memory-store.test.ts new file mode 100644 index 0000000..19696f5 --- /dev/null +++ b/tests/memory-store.test.ts @@ -0,0 +1,81 @@ +import { describe, it, expect } from 'vitest'; +import type { Memory } from '../src/types/index.js'; +import { + normalizeMemories, + filterMemories, + applyUnfilteredLimit, + sortMemoriesByRecency +} from '../src/memory/store.js'; + +describe('Memory store', () => { + it('normalizes legacy "decision" field into "memory" and defaults type', () => { + const raw = [ + { + id: 'abc123', + category: 'tooling', + decision: 'Use pnpm', + reason: 'Workspace performance', + date: '2026-01-01T00:00:00.000Z' + } + ]; + + const normalized = normalizeMemories(raw); + expect(normalized).toHaveLength(1); + expect(normalized[0]).toEqual({ + id: 'abc123', + type: 'decision', + category: 'tooling', + memory: 'Use pnpm', + reason: 'Workspace performance', + date: '2026-01-01T00:00:00.000Z' + }); + }); + + it('filters by category/type/query', () => { + const memories: Memory[] = [ + { + id: '1', + type: 'convention', + category: 'conventions', + memory: 'Use CSS tokens', + reason: 'Consistency', + date: '2026-01-01T00:00:00.000Z' + }, + { + id: '2', + type: 'gotcha', + category: 'testing', + memory: 'Avoid lodash debounce', + reason: 'Breaks zone.js', + date: '2026-01-02T00:00:00.000Z' + } + ]; + + expect(filterMemories(memories, { category: 'testing' })).toHaveLength(1); + expect(filterMemories(memories, { type: 'convention' })).toHaveLength(1); + expect(filterMemories(memories, { query: 'zone.js' })).toHaveLength(1); + }); + + it('applies unfiltered limit using recency ordering', () => { + const base: Memory[] = []; + for (let i = 0; i < 25; i++) { + base.push({ + id: String(i), + type: 'decision', + category: 'tooling', + memory: `m${i}`, + reason: 'r', + date: new Date(2026, 0, i + 1).toISOString() + }); + } + + const shuffled = [...base].reverse(); + const limited = applyUnfilteredLimit(shuffled, {}, 20); + expect(limited.truncated).toBe(true); + expect(limited.totalCount).toBe(25); + expect(limited.memories).toHaveLength(20); + + const sorted = sortMemoriesByRecency(shuffled); + expect(limited.memories[0].id).toBe(sorted[0].id); + }); +}); diff --git a/tests/memory.test.ts b/tests/memory.test.ts new file mode 100644 index 0000000..360243e --- /dev/null +++ b/tests/memory.test.ts @@ -0,0 +1,153 @@ +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { promises as fs } from 'fs'; +import path from 'path'; +import { fileURLToPath } from 'url'; +import type { Memory } from '../src/types/index.js'; +import { CODEBASE_CONTEXT_DIRNAME, MEMORY_FILENAME } from '../src/constants/codebase-context.js'; + +const __filename = fileURLToPath(import.meta.url); +const __dirname = path.dirname(__filename); + +describe('Memory System', () => { + const testDir = path.join(__dirname, 'test-workspace-memory'); + const memoryPath = path.join(testDir, CODEBASE_CONTEXT_DIRNAME, MEMORY_FILENAME); + + beforeAll(async () => { + await fs.mkdir(path.join(testDir, CODEBASE_CONTEXT_DIRNAME), { recursive: true }); + }); + + afterAll(async () => { + await fs.rm(testDir, { recursive: true, force: true }); + }); + + it('should create memory.json with valid schema', async () => { + const memory = { + id: 'test_abc123', + type: 'decision', + category: 'dependencies', + memory: 'Use hoisted mode', + reason: 'Indirect dependencies', + date: new Date().toISOString() + }; + + await fs.writeFile(memoryPath, JSON.stringify([memory], null, 2)); + + const content = await fs.readFile(memoryPath, 'utf-8'); + const memories = JSON.parse(content); + + expect(memories).toHaveLength(1); + expect(memories[0]).toHaveProperty('id'); + expect(memories[0]).toHaveProperty('type'); + expect(memories[0]).toHaveProperty('category'); + expect(memories[0]).toHaveProperty('memory'); + expect(memories[0]).toHaveProperty('reason'); + expect(memories[0]).toHaveProperty('date'); + }); + + it('should support all decision categories and types', () => { + const validCategories = ['tooling', 'architecture', 'testing', 'dependencies', 'conventions']; + const validTypes = ['convention', 'decision', 'gotcha']; + + validCategories.forEach((category) => { + validTypes.forEach((type) => { + const memory = { + id: `test_${category}_${type}`, + type, + category, + memory: `Test ${type} for ${category}`, + reason: `Test reason`, + date: new Date().toISOString() + }; + + expect(memory.category).toBe(category); + expect(memory.type).toBe(type); + }); + }); + }); + + it('should filter memories by category and type', async () => { + const memories = [ + { + id: 'test_1', + type: 'convention', + category: 'testing', + memory: 'Use Jest', + reason: 'Team standard', + date: new Date().toISOString() + }, + { + id: 'test_2', + type: 'decision', + category: 'dependencies', + memory: 'Use hoisted', + reason: 'Compatibility', + date: new Date().toISOString() + }, + { + id: 'test_3', + type: 'gotcha', + category: 'testing', + memory: 'Avoid lodash debounce', + reason: 'Breaks zone.js', + date: new Date().toISOString() + } + ]; + + await fs.writeFile(memoryPath, JSON.stringify(memories, null, 2)); + + const content = await fs.readFile(memoryPath, 'utf-8'); + const allMemories = JSON.parse(content) as Memory[]; + + // Filter by category + const testingMemories = allMemories.filter((m) => m.category === 'testing'); + expect(testingMemories).toHaveLength(2); + + // Filter by type + const conventionMemories = allMemories.filter((m) => m.type === 'convention'); + expect(conventionMemories).toHaveLength(1); + expect(conventionMemories[0].memory).toBe('Use Jest'); + + // Filter by both + const testingGotchas = allMemories.filter( + (m) => m.category === 'testing' && m.type === 'gotcha' + ); + expect(testingGotchas).toHaveLength(1); + expect(testingGotchas[0].memory).toBe('Avoid lodash debounce'); + }); + + it('should perform keyword search across memories', async () => { + const memories = [ + { + id: 'test_1', + type: 'decision', + category: 'dependencies', + memory: 'Use node-linker: hoisted', + reason: "Some packages don't declare transitive deps", + date: new Date().toISOString() + }, + { + id: 'test_2', + type: 'convention', + category: 'testing', + memory: 'Use Jest over Vitest', + reason: 'Better Angular integration', + date: new Date().toISOString() + } + ]; + + await fs.writeFile(memoryPath, JSON.stringify(memories, null, 2)); + + const content = await fs.readFile(memoryPath, 'utf-8'); + const allMemories = JSON.parse(content) as Memory[]; + + // Search for "hoisted" + const searchTerm = 'hoisted'; + const results = allMemories.filter((m) => { + const searchText = `${m.memory} ${m.reason}`.toLowerCase(); + return searchText.includes(searchTerm.toLowerCase()); + }); + + expect(results).toHaveLength(1); + expect(results[0].memory).toContain('hoisted'); + }); +}); diff --git a/tests/searcher-corruption-propagation.test.ts b/tests/searcher-corruption-propagation.test.ts index 9834ab3..50b9fff 100644 --- a/tests/searcher-corruption-propagation.test.ts +++ b/tests/searcher-corruption-propagation.test.ts @@ -3,6 +3,11 @@ import { promises as fs } from 'fs'; import os from 'os'; import path from 'path'; import { IndexCorruptedError } from '../src/errors/index.js'; +import { + CODEBASE_CONTEXT_DIRNAME, + INTELLIGENCE_FILENAME, + KEYWORD_INDEX_FILENAME +} from '../src/constants/codebase-context.js'; const deps = vi.hoisted(() => ({ getEmbeddingProvider: vi.fn(), @@ -29,8 +34,15 @@ describe('CodebaseSearcher IndexCorruptedError propagation', () => { consoleWarnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}); consoleErrorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}); - await fs.writeFile(path.join(tempDir, '.codebase-index.json'), JSON.stringify([])); - await fs.writeFile(path.join(tempDir, '.codebase-intelligence.json'), JSON.stringify({})); + await fs.mkdir(path.join(tempDir, CODEBASE_CONTEXT_DIRNAME), { recursive: true }); + await fs.writeFile( + path.join(tempDir, CODEBASE_CONTEXT_DIRNAME, KEYWORD_INDEX_FILENAME), + JSON.stringify([]) + ); + await fs.writeFile( + path.join(tempDir, CODEBASE_CONTEXT_DIRNAME, INTELLIGENCE_FILENAME), + JSON.stringify({}) + ); }); afterEach(async () => { @@ -77,4 +89,3 @@ describe('CodebaseSearcher IndexCorruptedError propagation', () => { await expect(searcher.search('test', 5)).rejects.toBeInstanceOf(IndexCorruptedError); }); }); - From c50663b9738e06f65c5ec9f8cf5e765a055d91fa Mon Sep 17 00:00:00 2001 From: PatrickSys Date: Wed, 28 Jan 2026 19:09:55 +0100 Subject: [PATCH 2/3] Fix lint --- src/index.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/index.ts b/src/index.ts index b8410e0..701f7f9 100644 --- a/src/index.ts +++ b/src/index.ts @@ -1039,7 +1039,7 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => { if (matchingMemories.length > 0) { result.memories = matchingMemories; } - } catch (error) { + } catch (_error) { // No memory file yet, that's fine - don't fail the whole request } From 9bb90eff1df8d1c7823b150fd82f4ff326d48356 Mon Sep 17 00:00:00 2001 From: PatrickSys Date: Wed, 28 Jan 2026 19:13:49 +0100 Subject: [PATCH 3/3] Fix security audit --- package.json | 2 +- pnpm-lock.yaml | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/package.json b/package.json index c11833d..7b960de 100644 --- a/package.json +++ b/package.json @@ -92,7 +92,7 @@ "@xenova/transformers": "^2.17.0", "fuse.js": "^7.0.0", "glob": "^10.3.10", - "hono": "4.11.4", + "hono": "4.11.7", "ignore": "^5.3.1", "typescript": "^5.3.3", "uuid": "^9.0.1", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 5bf2397..8b96881 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -13,7 +13,7 @@ importers: version: 0.4.20(zod@4.3.4) '@modelcontextprotocol/sdk': specifier: ^1.25.2 - version: 1.25.2(hono@4.11.4)(zod@4.3.4) + version: 1.25.2(hono@4.11.7)(zod@4.3.4) '@typescript-eslint/typescript-estree': specifier: ^7.0.0 version: 7.18.0(typescript@5.9.3) @@ -27,8 +27,8 @@ importers: specifier: ^10.3.10 version: 10.5.0 hono: - specifier: 4.11.4 - version: 4.11.4 + specifier: 4.11.7 + version: 4.11.7 ignore: specifier: ^5.3.1 version: 5.3.2 @@ -1398,8 +1398,8 @@ packages: resolution: {integrity: sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==} engines: {node: '>= 0.4'} - hono@4.11.4: - resolution: {integrity: sha512-U7tt8JsyrxSRKspfhtLET79pU8K+tInj5QZXs1jSugO1Vq5dFj3kmZsRldo29mTBfcjDRVRXrEZ6LS63Cog9ZA==} + hono@4.11.7: + resolution: {integrity: sha512-l7qMiNee7t82bH3SeyUCt9UF15EVmaBvsppY2zQtrbIhl/yzBTny+YUxsVjSjQ6gaqaeVtZmGocom8TzBlA4Yw==} engines: {node: '>=16.9.0'} http-errors@2.0.0: @@ -2539,9 +2539,9 @@ snapshots: '@eslint/core': 0.17.0 levn: 0.4.1 - '@hono/node-server@1.19.7(hono@4.11.4)': + '@hono/node-server@1.19.7(hono@4.11.7)': dependencies: - hono: 4.11.4 + hono: 4.11.7 '@huggingface/jinja@0.2.2': {} @@ -2597,9 +2597,9 @@ snapshots: - ws - zod - '@modelcontextprotocol/sdk@1.25.2(hono@4.11.4)(zod@4.3.4)': + '@modelcontextprotocol/sdk@1.25.2(hono@4.11.7)(zod@4.3.4)': dependencies: - '@hono/node-server': 1.19.7(hono@4.11.4) + '@hono/node-server': 1.19.7(hono@4.11.7) ajv: 8.17.1 ajv-formats: 3.0.1(ajv@8.17.1) content-type: 1.0.5 @@ -3814,7 +3814,7 @@ snapshots: dependencies: function-bind: 1.1.2 - hono@4.11.4: {} + hono@4.11.7: {} http-errors@2.0.0: dependencies: