Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@

> **Before an AI writes a new class/id/function, CodeLens must be checked. This is not optional.**

CodeLens is an AI-native code intelligence platform that gives AI agents **full visibility** into a codebase before they write any code. It prevents collision, overwrite of existing logic, security vulnerabilities, and dead code through 64 CLI commands, an MCP server with 62 tools (51 static + 11 dynamic), AST-based taint analysis, live CVE/OSV scanning, a plugin system with OWASP Top 10 + Compliance rule packs, a true graph data model (nodes + edges) for structural code queries, and token-efficient `--format compact` output for high-volume agent workflows (issue #17).
CodeLens is an AI-native code intelligence platform that gives AI agents **full visibility** into a codebase before they write any code. It prevents collision, overwrite of existing logic, security vulnerabilities, and dead code through 63 CLI commands, an MCP server with 61 tools (50 static + 11 dynamic), AST-based taint analysis, live CVE/OSV scanning, a plugin system with OWASP Top 10 + Compliance rule packs, a true graph data model (nodes + edges) for structural code queries, and token-efficient `--format compact` output for high-volume agent workflows (issue #17).

## Features

- **64 CLI Commands** — From basic scan/query to AST taint analysis, CVE scanning, plugin management, auto-fix, dashboards, CI/CD quality gates, and `graph-schema` for cheap graph-shape introspection
- **MCP Server (62 Tools)** — Native AI agent integration via Model Context Protocol (JSON-RPC over stdio), 51 statically-defined tools + 11 dynamically discovered, every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`)
- **63 CLI Commands** — From basic scan/query to AST taint analysis, CVE scanning, plugin management, auto-fix, dashboards, CI/CD quality gates, and `graph-schema` for cheap graph-shape introspection
- **MCP Server (61 Tools)** — Native AI agent integration via Model Context Protocol (JSON-RPC over stdio), 50 statically-defined tools + 11 dynamically discovered, every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`)
- **Token-Efficient Compact Output (v8.2, issue #17)** — `--format compact` produces single-char-key JSON with abbreviated types, omitted null fields, and relative paths — ~50% smaller than `json` on real trace output. Combined with `--limit`/`--offset` pagination, 5 structural queries now cost <5k tokens (down from 30-80k)
- **AST Taint Engine** — Tree-sitter based taint analysis with return-value propagation, scope hierarchy, and branch condition refinement
- **Live CVE/OSV Scanning** — Real-time vulnerability data from OSV.dev API with SQLite cache, 9 ecosystems (PyPI, npm, crates.io, Go, Maven, NuGet, RubyGems, Pub, Hex)
Expand Down Expand Up @@ -82,7 +82,7 @@ python3 scripts/codelens.py query "myFunction" --lite
|---------|-------------|
| `init [workspace]` | Initialize `.codelens` config with auto-detected frameworks |
| `scan [workspace] [--incremental] [--full] [--max-files N]` | Scan workspace and build registry |
| `validate [workspace]` | Validate registry vs file system |
| `registry-validate [workspace]` | Validate registry vs file system |
| `detect [workspace]` | Detect frameworks and show recommended config |
| `watch [workspace] [--git-mode] [--interval SECS]` | Start file watcher (default: watchdog; `--git-mode` polls `git diff --name-only`) |
| `git-status [workspace]` | Show git-aware scan state: HEAD SHA, last-indexed SHA, changed files, re-scan recommendation |
Expand Down Expand Up @@ -225,8 +225,8 @@ codelens/
│ ├── changelog.md # Older changelog (per-version highlights)
│ └── agent-integration.md # AI agent integration guide
├── scripts/
│ ├── codelens.py # CLI entry point (64 commands registered)
│ ├── mcp_server.py # MCP JSON-RPC server (62 tools)
│ ├── codelens.py # CLI entry point (63 commands registered)
│ ├── mcp_server.py # MCP JSON-RPC server (61 tools)
│ ├── registry.py # Registry read/write/build
│ ├── persistent_registry.py # SQLite persistent storage (WAL mode)
│ ├── base_parser.py # Base tree-sitter parser
Expand Down
12 changes: 6 additions & 6 deletions SKILL-QUICK.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,10 @@ $CLI list --limit 5 --offset 10 --format compact # → paginated + co
| "Cross-file taint" | `dataflow` | `taint` (taint is single-file, AST-deep) |
| "Auto-fix issues" | `fix` | `check` (check just gates, doesn't fix) |

## All 64 Commands
## All 63 Commands

### Setup & Lifecycle (8+)
`init` · `scan [--incremental] [--max-files N] [--full]` · `validate` · `detect` · `watch [--debounce SECS] [--git-mode] [--interval SECS]` · `git-status` · `migrate` · `serve` · `lsp-status` (issue #33: `codelens --lsp-status` top-level flag is an alias of `codelens lsp-status` — both delegate to `hybrid_engine.get_lsp_status()` and return the identical payload)
`init` · `scan [--incremental] [--max-files N] [--full]` · `registry-validate` · `detect` · `watch [--debounce SECS] [--git-mode] [--interval SECS]` · `git-status` · `migrate` · `serve` · `lsp-status` (issue #33: `codelens --lsp-status` top-level flag is an alias of `codelens lsp-status` — both delegate to `hybrid_engine.get_lsp_status()` and return the identical payload)

### Pre-Write Safety (5)
`query "name" [--domain ...] [--fuzzy]` · `impact "name" [--action modify|delete]` · `refactor-safe "name" [--action rename|move]` · `guard (--pre|--post) --file PATH` · `check [--severity ...] [--max-findings N]`
Expand Down Expand Up @@ -146,18 +146,18 @@ $CLI list --limit 5 --offset 10 --format compact # → paginated + co
### Tooling (1)
`plugin <install|list|search|update|info|validate>`

**Total: 64 commands** (auto-registered via `commands/__init__.py`; rerun `python3 scripts/sync_command_count.py --apply` after adding/removing a command)
**Total: 63 commands** (auto-registered via `commands/__init__.py`; rerun `python3 scripts/sync_command_count.py --apply` after adding/removing a command)

## MCP Server (62 Tools)
## MCP Server (61 Tools)

Start the MCP server for AI agent integration:

```bash
python3 scripts/codelens.py serve
```

Exposes 62 tools as `codelens_<command>` (e.g., `codelens_query`, `codelens_taint`, `codelens_graph_schema`, `codelens_architecture`, `codelens_resolve_types`, `codelens_git_status`):
- 51 statically-defined tools (full JSON schemas in `mcp_server.py`)
Exposes 61 tools as `codelens_<command>` (e.g., `codelens_query`, `codelens_taint`, `codelens_graph_schema`, `codelens_architecture`, `codelens_resolve_types`, `codelens_git_status`):
- 50 statically-defined tools (full JSON schemas in `mcp_server.py`)
- 11 dynamically-discovered tools (auto-discovered from `COMMAND_REGISTRY`; long-running `watch` and `serve` are excluded)
- Every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`). Use `format: "compact"` for token-efficient responses (~50% smaller than `json`).
- `watch` and `serve` itself are excluded (long-running)
Expand Down
4 changes: 2 additions & 2 deletions SKILL.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
name: codelens
description: >
CodeLens — AI-Native Code Intelligence. 64 commands for AI-powered code analysis,
CodeLens — AI-Native Code Intelligence. 63 commands for AI-powered code analysis,
security auditing, quality scoring, AST-based taint analysis, live CVE scanning,
and pre-write safety checks. Supports 28+ languages with tree-sitter + regex
fallback parsing. MCP server exposes 62 tools for AI agent integration.
fallback parsing. MCP server exposes 61 tools for AI agent integration.
For quick command reference with validated output schemas, see SKILL-QUICK.md.
For version history, see CHANGELOG.md.
---
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "codelens"
version = "8.2.0"
description = "Live Codebase Reference Intelligence — 64 commands for AI-powered code analysis, security auditing, and quality scoring"
description = "Live Codebase Reference Intelligence — 63 commands for AI-powered code analysis, security auditing, and quality scoring"
readme = "README.md"
license = {text = "MIT"}
requires-python = ">=3.8"
Expand Down
4 changes: 2 additions & 2 deletions scripts/codelens.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
python3 codelens.py circular <workspace> # Detect circular dependencies
python3 codelens.py context <name> <workspace> # Get rich symbol context
python3 codelens.py dependents <file> <workspace> # Module-level import tracking
python3 codelens.py validate <workspace> # Validate registry vs file system
python3 codelens.py registry-validate <workspace> # Validate registry vs file system
python3 codelens.py dataflow <workspace> # Trace data flow source→sink
python3 codelens.py smell <workspace> # Detect code smells
python3 codelens.py side-effect <workspace> [--name func] # Analyze function side effects
Expand Down Expand Up @@ -1038,7 +1038,7 @@ def main():
"secrets", "dataflow", "vuln-scan", "env-check", "entrypoints",
"api-map", "state-map", "handbook", "analyze", "test-map",
"stack-trace", "config-drift", "type-infer", "ownership",
"regex-audit", "a11y", "css-deep", "diff", "ask", "validate",
"regex-audit", "a11y", "css-deep", "diff", "ask",
"taint", "dashboard", "history",
}

Expand Down
8 changes: 4 additions & 4 deletions scripts/commands/registry_validate.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
"""registry-validate command — Validate registry against file system.

Renamed from `validate` in v8.x to make room for `rule-validate` (rule YAML
validation). The old `validate` command name still works as a deprecated alias
(see ``scripts/commands/validate.py``) but prints a one-line stderr warning
and will be removed in a future release.
validation). The deprecated `validate` alias was removed in issue #100 —
use `registry-validate` for registry checks, or `rule-validate` for rule
YAML validation.
"""

import sys
Expand Down Expand Up @@ -38,7 +38,7 @@ def execute(args, workspace):

register_command(
"registry-validate",
"Validate registry against file system (renamed from `validate`)",
"Validate registry against file system",
add_args,
execute,
)
59 changes: 0 additions & 59 deletions scripts/commands/validate.py

This file was deleted.

36 changes: 0 additions & 36 deletions scripts/formatters/markdown.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,6 @@ def to_markdown(data: Any, command: str = "") -> str:
_md_diff(data, lines)
elif command == "dependents":
_md_dependents(data, lines)
elif command == "validate":
_md_validate(data, lines)
elif command == "dataflow":
_md_dataflow(data, lines)
elif command == "test-map":
Expand Down Expand Up @@ -1217,40 +1215,6 @@ def _md_dependents(data: Dict, lines: list) -> None:
lines.append("")


def _md_validate(data: Dict, lines: list) -> None:
"""Markdown for validate command."""
total = data.get("total_issues", 0)
summary = data.get("summary", {})
rec = data.get("recommendation", "")
lines.append("## Registry Validation")
lines.append("")
icon = "PASS" if total == 0 else "FAIL"
lines.append(f"**Status:** {icon} ({total} issues)")
lines.append("")
lines.append(f"- Missing files: {summary.get('missing_files', 0)}")
lines.append(f"- Unregistered files: {summary.get('unregistered_files', 0)}")
lines.append(f"- Stale references: {summary.get('stale_references', 0)}")
lines.append(f"- Orphan entries: {summary.get('orphan_entries', 0)}")
lines.append("")
issues = data.get("issues", {})
if issues:
for cat in ["missing_files", "unregistered_files", "stale_references", "orphan_entries"]:
items = issues.get(cat, [])
if items:
lines.append(f"### {cat.replace('_', ' ').title()}")
for item in items[:10]:
if isinstance(item, dict):
lines.append(f"- `{item.get('file', item.get('path', ''))}` — {item.get('reason', '')}")
else:
lines.append(f"- `{item}`")
if len(items) > 10:
lines.append(f"- ... and {len(items) - 10} more")
lines.append("")
if rec:
lines.append(f"**Recommendation:** {rec}")
lines.append("")


def _md_dataflow(data: Dict, lines: list) -> None:
"""Markdown for dataflow command."""
stats = data.get("stats", {})
Expand Down
2 changes: 1 addition & 1 deletion scripts/graph_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
- New tables `graph_nodes` and `graph_edges` are additive (prefixed `graph_`
to avoid colliding with any existing table name).
- The flat registry tables and JSON files are untouched.
- All 64 existing CLI commands continue to work unchanged.
- All 63 existing CLI commands continue to work unchanged.

Schema:
graph_nodes(
Expand Down
15 changes: 1 addition & 14 deletions scripts/mcp_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Implements the MCP specification (2025-03-26) over stdio (JSON-RPC 2.0).
Provides persistent server mode with in-memory registry caching, sub-millisecond
query latency after initial scan, and automatic tool discovery for all 64 CodeLens commands.
query latency after initial scan, and automatic tool discovery for all 63 CodeLens commands.

Usage:
python3 codelens.py serve # Start MCP server (stdio transport)
Expand Down Expand Up @@ -664,19 +664,6 @@
"required": ["workspace"]
}
},
"validate": {
"description": "Validate registry against file system. Detect stale registry entries.",
"parameters": {
"type": "object",
"properties": {
"workspace": {
"type": "string",
"description": "Path to workspace root directory"
}
},
"required": ["workspace"]
}
},
"detect": {
"description": "Detect frameworks and technologies used in the project.",
"parameters": {
Expand Down
2 changes: 1 addition & 1 deletion skill.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "codelens",
"version": "8.2.0",
"description": "Live Codebase Reference Intelligence. 64 commands for AI-powered code analysis, security auditing, quality scoring, and pre-write safety checks. Supports 28+ languages with regex+AST hybrid parsing. Must activate before writing/editing/deleting any class, id, or function.",
"description": "Live Codebase Reference Intelligence. 63 commands for AI-powered code analysis, security auditing, quality scoring, and pre-write safety checks. Supports 28+ languages with regex+AST hybrid parsing. Must activate before writing/editing/deleting any class, id, or function.",
"author": "codelens",
"command_categories": {
"setup": [
Expand Down
6 changes: 3 additions & 3 deletions tests/test_integration.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
"""
Integration smoke tests for all 64 CodeLens commands.
Integration smoke tests for all 63 CodeLens commands.

Tests that every command:
1. Runs without crash (valid JSON output)
Expand Down Expand Up @@ -50,7 +50,7 @@ def parse_json(stdout):

NO_ARGS_COMMANDS = [
"scan", "list", "handbook", "symbols", "trace", "impact",
"outline", "missing-refs", "circular", "dependents", "validate",
"outline", "missing-refs", "circular", "dependents",
"dataflow", "smell", "side-effect", "dead-code", "test-map",
"config-drift", "type-infer", "ownership", "secrets",
"entrypoints", "api-map", "state-map", "env-check", "debug-leak",
Expand Down Expand Up @@ -300,7 +300,7 @@ def test_command_registry_has_all_commands(self):
sys.path.insert(0, SCRIPT_DIR)
from commands import COMMAND_REGISTRY
# Regression sentinel — see docstring above for update procedure.
EXPECTED_COMMAND_COUNT = 64
EXPECTED_COMMAND_COUNT = 63
actual = len(COMMAND_REGISTRY)
assert actual == EXPECTED_COMMAND_COUNT, (
f"Command count drift detected: expected {EXPECTED_COMMAND_COUNT}, "
Expand Down
Loading