feat(search): semantic_query MCP tool via TF-IDF Option A (closes #11)

README.md

@@ -2,12 +2,12 @@
 
 > **Before an AI writes a new class/id/function, CodeLens must be checked. This is not optional.**
 
-CodeLens is an AI-native code intelligence platform that gives AI agents **full visibility** into a codebase before they write any code. It prevents collision, overwrite of existing logic, security vulnerabilities, and dead code through 71 CLI commands, an MCP server with 69 tools (54 static + 15 dynamic), AST-based taint analysis, live CVE/OSV scanning, a plugin system with OWASP Top 10 + Compliance rule packs, a true graph data model (nodes + edges) for structural code queries, and token-efficient `--format compact` output for high-volume agent workflows (issue #17).
+CodeLens is an AI-native code intelligence platform that gives AI agents **full visibility** into a codebase before they write any code. It prevents collision, overwrite of existing logic, security vulnerabilities, and dead code through 73 CLI commands, an MCP server with 71 tools (55 static + 16 dynamic), AST-based taint analysis, live CVE/OSV scanning, a plugin system with OWASP Top 10 + Compliance rule packs, a true graph data model (nodes + edges) for structural code queries, and token-efficient `--format compact` output for high-volume agent workflows (issue #17).
 
 ## Features
 
-- **71 CLI Commands** — From basic scan/query to AST taint analysis, CVE scanning, plugin management, auto-fix, dashboards, CI/CD quality gates, and `graph-schema` for cheap graph-shape introspection
-- **MCP Server (69 Tools)** — Native AI agent integration via Model Context Protocol (JSON-RPC over stdio), 54 statically-defined tools + 15 dynamically discovered, every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`)
+- **73 CLI Commands** — From basic scan/query to AST taint analysis, CVE scanning, plugin management, auto-fix, dashboards, CI/CD quality gates, and `graph-schema` for cheap graph-shape introspection
+- **MCP Server (71 Tools)** — Native AI agent integration via Model Context Protocol (JSON-RPC over stdio), 55 statically-defined tools + 16 dynamically discovered, every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`)
 - **Token-Efficient Compact Output (v8.2, issue #17)** — `--format compact` produces single-char-key JSON with abbreviated types, omitted null fields, and relative paths — ~50% smaller than `json` on real trace output. Combined with `--limit`/`--offset` pagination, 5 structural queries now cost <5k tokens (down from 30-80k)
 - **AST Taint Engine** — Tree-sitter based taint analysis with return-value propagation, scope hierarchy, and branch condition refinement
 - **Live CVE/OSV Scanning** — Real-time vulnerability data from OSV.dev API with SQLite cache, 9 ecosystems (PyPI, npm, crates.io, Go, Maven, NuGet, RubyGems, Pub, Hex)
@@ -225,8 +225,8 @@ codelens/
 │   ├── changelog.md               # Older changelog (per-version highlights)
 │   └── agent-integration.md       # AI agent integration guide
 ├── scripts/
-│   ├── codelens.py                # CLI entry point (71 commands registered)
-│   ├── mcp_server.py              # MCP JSON-RPC server (69 tools)
+│   ├── codelens.py                # CLI entry point (73 commands registered)
+│   ├── mcp_server.py              # MCP JSON-RPC server (71 tools)
 │   ├── registry.py                # Registry read/write/build
 │   ├── persistent_registry.py     # SQLite persistent storage (WAL mode)
 │   ├── base_parser.py             # Base tree-sitter parser

SKILL-QUICK.md

@@ -115,7 +115,7 @@ $CLI list --limit 5 --offset 10 --format compact            # → paginated + co
 | "Cross-file taint" | `dataflow` | `taint` (taint is single-file, AST-deep) |
 | "Auto-fix issues" | `fix` | `check` (check just gates, doesn't fix) |
 
-## All 71 Commands
+## All 73 Commands
 
 ### Setup & Lifecycle (8+)
 `init` · `scan [--incremental] [--max-files N] [--full]` · `registry-validate` · `detect` · `watch [--debounce SECS] [--git-mode] [--interval SECS]` · `git-status` · `migrate` · `serve` · `lsp-status` (issue #33: `codelens --lsp-status` top-level flag is an alias of `codelens lsp-status` — both delegate to `hybrid_engine.get_lsp_status()` and return the identical payload)
@@ -147,19 +147,19 @@ $CLI list --limit 5 --offset 10 --format compact            # → paginated + co
 ### Tooling (1)
 `plugin <install|list|search|update|info|validate>`
 
-**Total: 71 commands** (auto-registered via `commands/__init__.py`; rerun `python3 scripts/sync_command_count.py --apply` after adding/removing a command)
+**Total: 73 commands** (auto-registered via `commands/__init__.py`; rerun `python3 scripts/sync_command_count.py --apply` after adding/removing a command)
 
-## MCP Server (69 Tools)
+## MCP Server (71 Tools)
 
 Start the MCP server for AI agent integration:
 
 ```bash
 python3 scripts/codelens.py serve
 ```
 
-Exposes 69 tools as `codelens_<command>` (e.g., `codelens_query`, `codelens_taint`, `codelens_graph_schema`, `codelens_architecture`, `codelens_resolve_types`, `codelens_git_status`):
+Exposes 71 tools as `codelens_<command>` (e.g., `codelens_query`, `codelens_taint`, `codelens_graph_schema`, `codelens_architecture`, `codelens_resolve_types`, `codelens_git_status`):
 - 50 statically-defined tools (full JSON schemas in `mcp_server.py`)
-- 15 dynamically-discovered tools (auto-discovered from `COMMAND_REGISTRY`; long-running `watch` and `serve` are excluded)
+- 16 dynamically-discovered tools (auto-discovered from `COMMAND_REGISTRY`; long-running `watch` and `serve` are excluded)
 - Every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`). Use `format: "compact"` for token-efficient responses (~50% smaller than `json`).
 - `watch` and `serve` itself are excluded (long-running)
 

SKILL.md

@@ -1,10 +1,10 @@
 ---
 name: codelens
 description: >
-  CodeLens — AI-Native Code Intelligence. 71 commands for AI-powered code analysis,
+  CodeLens — AI-Native Code Intelligence. 73 commands for AI-powered code analysis,
   security auditing, quality scoring, AST-based taint analysis, live CVE scanning,
   and pre-write safety checks. Supports 28+ languages with tree-sitter + regex
-  fallback parsing. MCP server exposes 69 tools for AI agent integration.
+  fallback parsing. MCP server exposes 71 tools for AI agent integration.
   For quick command reference with validated output schemas, see SKILL-QUICK.md.
   For version history, see CHANGELOG.md.
 ---

pyproject.toml

@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "codelens"
 version = "8.2.0"
-description = "Live Codebase Reference Intelligence — 71 commands for AI-powered code analysis, security auditing, and quality scoring"
+description = "Live Codebase Reference Intelligence — 73 commands for AI-powered code analysis, security auditing, and quality scoring"
 readme = "README.md"
 license = {text = "MIT"}
 requires-python = ">=3.8"

scripts/commands/semantic_query.py

@@ -0,0 +1,66 @@
+"""Semantic query command — TF-IDF symbol search (issue #11, Option A)."""
+
+import os
+from typing import Any, Dict
+
+from semantic_search_engine import semantic_query
+from commands import register_command
+
+
+def add_args(parser):
+    parser.add_argument(
+        "query",
+        help=(
+            "Natural-language or code-fragment query "
+            "(e.g. 'user authentication flow', 'parse jwt', 'error handler'). "
+            "Symbol names, signatures, kinds, and file paths are all searched."
+        ),
+    )
+    parser.add_argument(
+        "workspace",
+        nargs="?",
+        default=None,
+        help="Path to workspace root (auto-detected if omitted).",
+    )
+    parser.add_argument(
+        "--top",
+        type=int,
+        default=10,
+        metavar="N",
+        help="Maximum number of results to return (default: 10; use 0 for all).",
+    )
+    parser.add_argument(
+        "--db-path",
+        default=None,
+        metavar="PATH",
+        help="Custom path for the SQLite registry database "
+             "(default: <workspace>/.codelens/codelens.db).",
+    )
+
+
+def execute(args, workspace) -> Dict[str, Any]:
+    """Dispatch to :func:`semantic_engine.semantic_query`.
+
+    The CLI ``--top`` flag is the user-facing name; the engine takes the
+    same value as ``top_k``. We pass ``workspace`` and ``db_path`` straight
+    through so users can override the database location for ad-hoc queries
+    against a snapshot.
+    """
+    top_k = getattr(args, "top", 10)
+    if top_k is None:
+        top_k = 10
+    db_path = getattr(args, "db_path", None)
+    return semantic_query(
+        workspace=workspace,
+        query=args.query,
+        top_k=top_k,
+        db_path=db_path,
+    )
+
+
+register_command(
+    "semantic-query",
+    "Semantic symbol search via TF-IDF (find symbols by meaning, not just name)",
+    add_args,
+    execute,
+)

scripts/graph_model.py

@@ -13,7 +13,7 @@
 - New tables `graph_nodes` and `graph_edges` are additive (prefixed `graph_`
   to avoid colliding with any existing table name).
 - The flat registry tables and JSON files are untouched.
-- All 71 existing CLI commands continue to work unchanged.
+- All 73 existing CLI commands continue to work unchanged.
 
 Schema:
     graph_nodes(

scripts/mcp_server.py

@@ -470,6 +470,40 @@
             "required": ["workspace"]
         }
     },
+    "semantic-query": {
+        "description": (
+            "Semantic symbol search via TF-IDF (issue #11). Finds symbols by "
+            "meaning, not just by name — e.g. querying 'user authentication flow' "
+            "can surface a function named verify_jwt_claims. Returns ranked symbols "
+            "with cosine-similarity scores. Pure-Python, zero dependencies; reads "
+            "from the existing SQLite registry so the index is always in sync with "
+            "the last 'scan' result."
+        ),
+        "parameters": {
+            "type": "object",
+            "properties": {
+                "query": {
+                    "type": "string",
+                    "description": (
+                        "Natural-language or code-fragment query, e.g. "
+                        "'user authentication flow', 'parse jwt', 'error handler'. "
+                        "Symbol names, signatures, kinds, and file paths are all "
+                        "included in the TF-IDF document for each symbol."
+                    )
+                },
+                "workspace": {
+                    "type": "string",
+                    "description": "Path to workspace root directory"
+                },
+                "top": {
+                    "type": "integer",
+                    "description": "Maximum number of results to return (default: 10; use 0 for all).",
+                    "default": 10
+                }
+            },
+            "required": ["query", "workspace"]
+        }
+    },
     "vuln-scan": {
         "description": "Scan dependencies for known CVEs and security vulnerabilities.",
         "parameters": {