Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ CodeLens is an AI-native code intelligence platform that gives AI agents **full
## Features

- **75 CLI Commands** — From basic scan/query to AST taint analysis, CVE scanning, plugin management, auto-fix, dashboards, CI/CD quality gates, and `graph-schema` for cheap graph-shape introspection
- **MCP Server (73 Tools)** — Native AI agent integration via Model Context Protocol (JSON-RPC over stdio), 56 statically-defined tools + 17 dynamically discovered, every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`)
- **MCP Server (73 Tools)** — Native AI agent integration via Model Context Protocol (JSON-RPC over stdio), 56 statically-defined tools + 17 dynamically discovered, every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`/`graphml`)
- **Token-Efficient Compact Output (v8.2, issue #17)** — `--format compact` produces single-char-key JSON with abbreviated types, omitted null fields, and relative paths — ~50% smaller than `json` on real trace output. Combined with `--limit`/`--offset` pagination, 5 structural queries now cost <5k tokens (down from 30-80k)
- **AST Taint Engine** — Tree-sitter based taint analysis with return-value propagation, scope hierarchy, and branch condition refinement
- **Live CVE/OSV Scanning** — Real-time vulnerability data from OSV.dev API with SQLite cache, 9 ecosystems (PyPI, npm, crates.io, Go, Maven, NuGet, RubyGems, Pub, Hex)
Expand Down Expand Up @@ -284,7 +284,7 @@ codelens/
│ ├── pre_commit_hook.py # Git pre-commit hook integration
│ ├── utils.py # Shared utilities (version, helpers)
│ ├── commands/ # One file per CLI command (auto-registered, 64 commands)
│ ├── formatters/ # Output formatters (markdown, sarif, compact)
│ ├── formatters/ # Output formatters (markdown, sarif, compact, graphml)
│ ├── parsers/ # Tree-sitter + fallback parsers
│ │ ├── html_parser.py, css_parser.py, js_frontend_parser.py, js_backend_parser.py
│ │ ├── rust_parser.py, python_parser.py, tsx_parser.py, ts_backend_parser.py
Expand Down Expand Up @@ -338,8 +338,9 @@ CodeLens ships with a native MCP server (55 tools) for direct AI agent integrati
python3 scripts/codelens.py serve
```

Every MCP tool accepts a `format` parameter with the enum `[json, markdown, ai, sarif, compact]`.
For high-volume agent workflows, pass `format: "compact"` to cut token usage ~50%. Example:
Every MCP tool accepts a `format` parameter with the enum `[json, markdown, ai, sarif, compact, graphml]`.
For high-volume agent workflows, pass `format: "compact"` to cut token usage ~50%.
For graph-producing commands (`scan`, `trace`, `impact`, `circular`), pass `format: "graphml"` to emit a GraphML 1.0 XML document that opens directly in Gephi/Cytoscape/yEd/Neo4j (issue #59 Phase 3). Example:

```json
// tools/call with format=compact
Expand Down Expand Up @@ -371,6 +372,12 @@ python3 scripts/codelens.py check /path/to/workspace --severity high --max-findi

# SARIF output for GitHub Advanced Security / VS Code
python3 scripts/codelens.py check /path/to/workspace --format sarif > codelens.sarif

# GraphML export — opens in Gephi/Cytoscape/yEd/Neo4j (issue #59 Phase 3)
python3 scripts/codelens.py scan /path/to/workspace --format graphml > codelens.graphml
python3 scripts/codelens.py trace main /path/to/workspace --format graphml > trace.graphml
python3 scripts/codelens.py impact my_function /path/to/workspace --format graphml > impact.graphml
python3 scripts/codelens.py circular /path/to/workspace --format graphml > cycles.graphml
```

### Plugin System
Expand Down
3 changes: 2 additions & 1 deletion SKILL-QUICK.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ $CLI list --limit 5 --offset 10 --format compact # → paginated + co
| `--format ai` | Normalized: `{stats, items[], truncated, recommendations}` |
| `--format compact` | Token-efficient: single-char keys + abbreviated types (issue #17). ~50% smaller than `json`. Best for high-volume MCP tool calls |
| `--format sarif` | SARIF v2.1.0 output for GitHub Advanced Security / VS Code |
| `--format graphml` | GraphML 1.0 XML for graph-producing commands (`scan`, `trace`, `impact`, `circular`). Opens in Gephi/Cytoscape/yEd/Neo4j. Other commands emit a single-node placeholder (issue #59 Phase 3) |
| `--limit N` / `--offset N` | Pagination on list-type commands (`list`, `search`, `trace`, `symbols`, `outline`). Default limit=20 (issue #17). `--top N` is an alias for `--limit N --offset 0` |
| `--deep` | Enable LSP-enhanced deep analysis (requires language server; check with `lsp-status`) |
| `--db-path PATH` | Custom SQLite database path (default: `.codelens/codelens.db`) |
Expand Down Expand Up @@ -160,7 +161,7 @@ python3 scripts/codelens.py serve
Exposes 73 tools as `codelens_<command>` (e.g., `codelens_query`, `codelens_taint`, `codelens_graph_schema`, `codelens_architecture`, `codelens_resolve_types`, `codelens_git_status`):
- 50 statically-defined tools (full JSON schemas in `mcp_server.py`)
- 17 dynamically-discovered tools (auto-discovered from `COMMAND_REGISTRY`; long-running `watch` and `serve` are excluded)
- Every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`). Use `format: "compact"` for token-efficient responses (~50% smaller than `json`).
- Every tool accepts a `format` parameter (`json`/`markdown`/`ai`/`sarif`/`compact`/`graphml`). Use `format: "compact"` for token-efficient responses (~50% smaller than `json`). Use `format: "graphml"` for graph-producing commands (`scan`/`trace`/`impact`/`circular`) — emits GraphML 1.0 XML that opens in Gephi/Cytoscape/yEd/Neo4j (issue #59 Phase 3).
- `watch` and `serve` itself are excluded (long-running)

See `mcp_config.json` for Claude Desktop, Cursor, VS Code Copilot, Continue.dev, and Cline configuration templates.
Expand Down
24 changes: 10 additions & 14 deletions scripts/codelens.py
Original file line number Diff line number Diff line change
Expand Up @@ -885,16 +885,12 @@ def main():
# ``["text", "json"]`` because its default human-readable output is
# a text table, not JSON. Skipping the global add here lets that
# command-specific format work without an argparse conflict.
# Issue #59 Phase 3: ``graphml`` emits a GraphML 1.0 XML document for
# graph-producing commands (scan/trace/impact/circular); other commands
# produce a single-node placeholder so the format is always valid.
if "format" not in existing_dests:
# Issue #62 Phase 1: ``affected`` command uses ``-f`` for
# ``--filter``. Avoid the ``-f`` shortcut clash by only adding
# the global ``-f`` shortcut when the command doesn't already
# claim it. The long form ``--format`` is always safe.
format_args = ["--format"]
if "-f" not in existing_option_strings:
format_args.append("-f")
sub.add_argument(*format_args, choices=["json", "markdown", "ai", "sarif", "compact"], default=None,
help="Output format: json, markdown, ai (normalized schema), sarif (GitHub/VS Code), or compact (token-efficient single-char keys)")
sub.add_argument("--format", "-f", choices=["json", "markdown", "ai", "sarif", "compact", "graphml"], default=None,
help="Output format: json, markdown, ai (normalized schema), sarif (GitHub/VS Code), compact (token-efficient single-char keys), or graphml (GraphML 1.0 XML for graph-producing commands)")

# Add AI-optimized flags to subparser ONLY if the command doesn't already have them
if "top" not in existing_dests:
Expand Down Expand Up @@ -927,8 +923,8 @@ def main():
# Global format option (works before subcommand)
# Default: "ai" if CODELENS_AI_MODE is set (for AI consumers), else "json"
_default_format = "ai" if os.environ.get("CODELENS_AI_MODE", "").lower() in ("1", "true", "yes") else "json"
parser.add_argument("--format", "-f", choices=["json", "markdown", "ai", "sarif", "compact"], default=_default_format,
help=f"Output format (default: {_default_format}. Set CODELENS_AI_MODE=1 for ai default. compact = token-efficient single-char keys)")
parser.add_argument("--format", "-f", choices=["json", "markdown", "ai", "sarif", "compact", "graphml"], default=_default_format,
help=f"Output format (default: {_default_format}. Set CODELENS_AI_MODE=1 for ai default. compact = token-efficient single-char keys. graphml = GraphML XML for graph-producing commands)")
parser.add_argument("--db-path", default=None,
help="Custom path for SQLite database (default: .codelens/codelens.db)")
# Issue #157: --diff-base <ref> restricts analysis to files changed
Expand Down Expand Up @@ -975,11 +971,11 @@ def main():
arg = sys.argv[i]
if arg in ('-f', '--format') and i + 1 < len(sys.argv):
next_arg = sys.argv[i + 1]
if next_arg in ('json', 'markdown', 'ai', 'sarif', 'compact'):
if next_arg in ('json', 'markdown', 'ai', 'sarif', 'compact', 'graphml'):
global_format = next_arg
elif arg.startswith('-f=') and arg[3:] in ('json', 'markdown', 'ai', 'sarif', 'compact'):
elif arg.startswith('-f=') and arg[3:] in ('json', 'markdown', 'ai', 'sarif', 'compact', 'graphml'):
global_format = arg[3:]
elif arg.startswith('--format=') and arg[9:] in ('json', 'markdown', 'ai', 'sarif', 'compact'):
elif arg.startswith('--format=') and arg[9:] in ('json', 'markdown', 'ai', 'sarif', 'compact', 'graphml'):
global_format = arg[9:]
elif arg == '--top' and i + 1 < len(sys.argv):
try:
Expand Down
23 changes: 14 additions & 9 deletions scripts/formatters/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,13 @@ def _normalize_to_ai(data: Any, command: str = "") -> Dict[str, Any]:

def format_output(data: Any, format_type: str = "json", command: str = "",
workspace: str = "") -> str:
"""Format output data as JSON, Markdown, AI (normalized schema), SARIF, or Compact."""
"""Format output data as JSON, Markdown, AI (normalized schema), SARIF, Compact, or GraphML.

GraphML (issue #59 Phase 3) emits a GraphML 1.0 XML document for
graph-producing commands (scan, trace, impact, circular). Non-graph
commands produce a single-node placeholder graph so the format is
always valid XML.
"""
if format_type == "ai":
normalized = _normalize_to_ai(data, command)
return json.dumps(normalized, indent=2, ensure_ascii=False)
Expand All @@ -201,12 +207,11 @@ def format_output(data: Any, format_type: str = "json", command: str = "",
if format_type == "compact":
from formatters.compact import format_compact
return format_compact(data, command, workspace)
# Default: JSON — stamp schema_version so raw JSON consumers also get the
# version signal (issue #5). Mutates a copy so the caller's dict is not
# modified in place (raw JSON should reflect what the engine returned,
# plus the version stamp).
if isinstance(data, dict):
stamped = dict(data)
stamp_schema_version(stamped)
return json.dumps(stamped, indent=2, ensure_ascii=False)
if format_type == "graphml":
# Issue #59 Phase 3 — GraphML export for graph-producing commands
# (scan/trace/impact/circular). Non-graph commands emit a single-node
# placeholder so the format is always valid, never raises.
from formatters.graphml import format_graphml
return format_graphml(data, command, workspace)
# Default: JSON
return json.dumps(data, indent=2, ensure_ascii=False)
Loading
Loading