Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@

> **Before an AI writes a new class/id/function, CodeLens must be checked. This is not optional.**

CodeLens is an AI-native code intelligence platform that gives AI agents **full visibility** into a codebase before they write any code. It prevents collision, overwrite of existing logic, security vulnerabilities, and dead code through 63 CLI commands, an MCP server with 61 tools (50 static + 11 dynamic), AST-based taint analysis, live CVE/OSV scanning, a plugin system with OWASP Top 10 + Compliance rule packs, a true graph data model (nodes + edges) for structural code queries, and token-efficient `--format compact` output for high-volume agent workflows (issue #17).
CodeLens is an AI-native code intelligence platform that gives AI agents **full visibility** into a codebase before they write any code. It prevents collision, overwrite of existing logic, security vulnerabilities, and dead code through 65 CLI commands, an MCP server with 63 tools (50 static + 13 dynamic), AST-based taint analysis, live CVE/OSV scanning, a plugin system with OWASP Top 10 + Compliance rule packs, a true graph data model (nodes + edges) for structural code queries, and token-efficient `--format compact` output for high-volume agent workflows (issue #17).

## Features

- **63 CLI Commands** — From basic scan/query to AST taint analysis, CVE scanning, plugin management, auto-fix, dashboards, CI/CD quality gates, and `graph-schema` for cheap graph-shape introspection
- **MCP Server (61 Tools)** — Native AI agent integration via Model Context Protocol (JSON-RPC over stdio), 50 statically-defined tools + 11 dynamically discovered, every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`)
- **65 CLI Commands** — From basic scan/query to AST taint analysis, CVE scanning, plugin management, auto-fix, dashboards, CI/CD quality gates, and `graph-schema` for cheap graph-shape introspection
- **MCP Server (63 Tools)** — Native AI agent integration via Model Context Protocol (JSON-RPC over stdio), 50 statically-defined tools + 13 dynamically discovered, every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`)
- **Token-Efficient Compact Output (v8.2, issue #17)** — `--format compact` produces single-char-key JSON with abbreviated types, omitted null fields, and relative paths — ~50% smaller than `json` on real trace output. Combined with `--limit`/`--offset` pagination, 5 structural queries now cost <5k tokens (down from 30-80k)
- **AST Taint Engine** — Tree-sitter based taint analysis with return-value propagation, scope hierarchy, and branch condition refinement
- **Live CVE/OSV Scanning** — Real-time vulnerability data from OSV.dev API with SQLite cache, 9 ecosystems (PyPI, npm, crates.io, Go, Maven, NuGet, RubyGems, Pub, Hex)
Expand Down Expand Up @@ -225,8 +225,8 @@ codelens/
│ ├── changelog.md # Older changelog (per-version highlights)
│ └── agent-integration.md # AI agent integration guide
├── scripts/
│ ├── codelens.py # CLI entry point (63 commands registered)
│ ├── mcp_server.py # MCP JSON-RPC server (61 tools)
│ ├── codelens.py # CLI entry point (65 commands registered)
│ ├── mcp_server.py # MCP JSON-RPC server (63 tools)
│ ├── registry.py # Registry read/write/build
│ ├── persistent_registry.py # SQLite persistent storage (WAL mode)
│ ├── base_parser.py # Base tree-sitter parser
Expand Down
10 changes: 5 additions & 5 deletions SKILL-QUICK.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ $CLI list --limit 5 --offset 10 --format compact # → paginated + co
| "Cross-file taint" | `dataflow` | `taint` (taint is single-file, AST-deep) |
| "Auto-fix issues" | `fix` | `check` (check just gates, doesn't fix) |

## All 63 Commands
## All 65 Commands

### Setup & Lifecycle (8+)
`init` · `scan [--incremental] [--max-files N] [--full]` · `registry-validate` · `detect` · `watch [--debounce SECS] [--git-mode] [--interval SECS]` · `git-status` · `migrate` · `serve` · `lsp-status` (issue #33: `codelens --lsp-status` top-level flag is an alias of `codelens lsp-status` — both delegate to `hybrid_engine.get_lsp_status()` and return the identical payload)
Expand Down Expand Up @@ -146,19 +146,19 @@ $CLI list --limit 5 --offset 10 --format compact # → paginated + co
### Tooling (1)
`plugin <install|list|search|update|info|validate>`

**Total: 63 commands** (auto-registered via `commands/__init__.py`; rerun `python3 scripts/sync_command_count.py --apply` after adding/removing a command)
**Total: 65 commands** (auto-registered via `commands/__init__.py`; rerun `python3 scripts/sync_command_count.py --apply` after adding/removing a command)

## MCP Server (61 Tools)
## MCP Server (63 Tools)

Start the MCP server for AI agent integration:

```bash
python3 scripts/codelens.py serve
```

Exposes 61 tools as `codelens_<command>` (e.g., `codelens_query`, `codelens_taint`, `codelens_graph_schema`, `codelens_architecture`, `codelens_resolve_types`, `codelens_git_status`):
Exposes 63 tools as `codelens_<command>` (e.g., `codelens_query`, `codelens_taint`, `codelens_graph_schema`, `codelens_architecture`, `codelens_resolve_types`, `codelens_git_status`):
- 50 statically-defined tools (full JSON schemas in `mcp_server.py`)
- 11 dynamically-discovered tools (auto-discovered from `COMMAND_REGISTRY`; long-running `watch` and `serve` are excluded)
- 13 dynamically-discovered tools (auto-discovered from `COMMAND_REGISTRY`; long-running `watch` and `serve` are excluded)
- Every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`). Use `format: "compact"` for token-efficient responses (~50% smaller than `json`).
- `watch` and `serve` itself are excluded (long-running)

Expand Down
4 changes: 2 additions & 2 deletions SKILL.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
name: codelens
description: >
CodeLens — AI-Native Code Intelligence. 63 commands for AI-powered code analysis,
CodeLens — AI-Native Code Intelligence. 65 commands for AI-powered code analysis,
security auditing, quality scoring, AST-based taint analysis, live CVE scanning,
and pre-write safety checks. Supports 28+ languages with tree-sitter + regex
fallback parsing. MCP server exposes 61 tools for AI agent integration.
fallback parsing. MCP server exposes 63 tools for AI agent integration.
For quick command reference with validated output schemas, see SKILL-QUICK.md.
For version history, see CHANGELOG.md.
---
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "codelens"
version = "8.2.0"
description = "Live Codebase Reference Intelligence — 63 commands for AI-powered code analysis, security auditing, and quality scoring"
description = "Live Codebase Reference Intelligence — 65 commands for AI-powered code analysis, security auditing, and quality scoring"
readme = "README.md"
license = {text = "MIT"}
requires-python = ">=3.8"
Expand Down
168 changes: 168 additions & 0 deletions scripts/commands/export_snapshot.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
"""Export-snapshot command — Export the CodeLens graph as a compressed archive.

Issue #12: each developer previously had to ``codelens scan`` separately.
``export-snapshot`` packages the SQLite graph tables (``graph_nodes``,
``graph_edges``, ``symbols``, ``refs``, ``files``) as a gzip-compressed
archive that can be committed to the repo and shared with the team via
``codelens import-snapshot``.

The snapshot contains **graph metadata only** — file paths, symbol
names/kinds/line spans, edge relationships, content hashes, timestamps.
It NEVER contains file content.

Usage::

codelens export-snapshot [workspace] [--output path] [--db-path path]

Example output (the human-readable message is also embedded in the JSON
result under the ``message`` key and printed to stderr so stdout stays
machine-parseable)::

Snapshot exported: .codelens/snapshot.codelens.gz (1.2 MB)
"""

import os
import sys
from typing import Any, Dict, Optional

from commands import register_command
from utils import default_db_path, logger
from snapshot_io import (
DEFAULT_SNAPSHOT_FILENAME,
SNAPSHOT_TABLES,
build_snapshot,
default_snapshot_path,
format_size,
write_snapshot,
)


def add_args(parser):
"""Add export-snapshot arguments to the parser."""
parser.add_argument("workspace", nargs="?", default=None,
help="Path to workspace root (auto-detected if omitted)")
parser.add_argument("--output", default=None,
help="Output path for the snapshot archive "
"(default: .codelens/snapshot.codelens.gz)")
parser.add_argument("--db-path", default=None,
help="Custom path for the source SQLite database file")


def execute(args, workspace):
"""Execute the export-snapshot command."""
output = getattr(args, "output", None)
db_path = getattr(args, "db_path", None)
return cmd_export_snapshot(workspace, output_path=output, db_path=db_path)


def cmd_export_snapshot(
workspace: str,
output_path: Optional[str] = None,
db_path: Optional[str] = None,
) -> Dict[str, Any]:
"""Export the CodeLens graph database to a compressed snapshot archive.

Reads the SQLite graph tables (graph_nodes, graph_edges, symbols,
refs, files) and writes them as gzip-compressed JSON to
``output_path`` (default: ``<workspace>/.codelens/snapshot.codelens.gz``).

Args:
workspace: Path to the workspace root.
output_path: Optional explicit output path. If None, defaults to
``<workspace>/.codelens/snapshot.codelens.gz``.
db_path: Optional source SQLite db path. Defaults to
``<workspace>/.codelens/codelens.db``.

Returns:
Dict with keys: ``status``, ``message``, ``snapshot_path``,
``size_bytes``, ``size_human``, ``header``, ``workspace``.
On error: ``status="error"`` with an ``error`` message.
"""
workspace = os.path.abspath(workspace)

# Resolve source db path
effective_db = db_path or default_db_path(workspace)

# Resolve output path. If the user gave a relative path, resolve it
# against the workspace root so the default ``.codelens/...`` form
# works regardless of cwd.
if output_path:
effective_output = output_path if os.path.isabs(output_path) \
else os.path.join(workspace, output_path)
else:
effective_output = default_snapshot_path(workspace)

# Build the snapshot (reads the DB, raises FileNotFoundError if absent)
try:
snapshot = build_snapshot(workspace, db_path=effective_db)
except FileNotFoundError as exc:
return {
"status": "error",
"error": str(exc),
"workspace": workspace,
}
except Exception as exc: # pragma: no cover - defensive
logger.error(f"export-snapshot: build failed: {exc}", exc_info=True)
return {
"status": "error",
"error": f"Failed to build snapshot: {exc}",
"workspace": workspace,
}

header = snapshot.get("header", {})

# Write gzip-compressed JSON to disk
try:
size_bytes = write_snapshot(snapshot, effective_output)
except OSError as exc:
return {
"status": "error",
"error": f"Failed to write snapshot to {effective_output}: {exc}",
"workspace": workspace,
}

size_human = format_size(size_bytes)

# Human-readable message — also surfaced in the JSON result so
# scripted consumers can read it. Matches the issue #12 format:
# "Snapshot exported: .codelens/snapshot.codelens.gz (1.2 MB)"
# Use a workspace-relative path in the message when possible so the
# message is portable across machines (the snapshot is meant to be
# committed to the repo and shared).
try:
rel_output = os.path.relpath(effective_output, workspace)
# If the output lives outside the workspace, relpath produces
# something with '..' — fall back to the absolute path in that case.
if rel_output.startswith(".."):
display_path = effective_output
else:
display_path = rel_output
except (ValueError, OSError):
display_path = effective_output

message = f"Snapshot exported: {display_path} ({size_human})"

# Print the message to stderr so stdout (JSON) stays machine-clean,
# matching the convention used by scan's status messages.
print(message, file=sys.stderr)

return {
"status": "ok",
"message": message,
"snapshot_path": effective_output,
"display_path": display_path,
"size_bytes": size_bytes,
"size_human": size_human,
"header": header,
"workspace": workspace,
"tables": list(SNAPSHOT_TABLES),
}


register_command(
"export-snapshot",
"Export the CodeLens graph as a compressed snapshot archive (.codelens.gz) "
"for team sharing (issue #12)",
add_args,
execute,
)
Loading
Loading