feat(arch): get_architecture single-call overview — closes #19 by Wolfvin · Pull Request #28 · Wolfvin/CodeLens

Wolfvin · 2026-06-28T06:12:04Z

Summary

Implements issue #19 — get_architecture single-call codebase overview for AI agents.

Orientation on an unfamiliar codebase previously required 4-6 chained commands (scan → list → detect → entrypoints → api-map → read entry files), burning 10-20k tokens before any real work started. This PR collapses that into a single architecture CLI command + codelens_architecture MCP tool returning a compact overview: languages, frameworks, entry points, packages, top routes, graph hotspots, total symbol count, and an adrs placeholder (ADR feature is issue #16, Phase 3).

Files Touched

Created:

scripts/architecture_engine.py (677 lines) — engine module orchestrating framework_detect, entrypoints_engine, apimap_engine, and the graph_model hotspots query. Includes caching layer with scan-mtime invalidation.
scripts/commands/architecture.py (52 lines) — CLI command auto-registered via commands/__init__.py. Flags: --lite, --no-cache.
tests/test_architecture.py (533 lines) — 24 test cases across 7 test classes.

Modified (additive only — coordinated with parallel workers 2-a and 2-c):

scripts/mcp_server.py — added architecture entry to _TOOL_DEFINITIONS only (no other changes; 2-a owns compact-format changes in this file).
README.md — added architecture to the Architecture & Understanding command table.
SKILL-QUICK.md — added architecture [--lite] [--no-cache] to Navigation category (10 → 11); updated trigger map entry for "project overview".
CHANGELOG.md — added new ### get_architecture section under [8.2.0] Unreleased.
references/agent-integration.md — added architecture to intent-to-tool map + keyword detection matrix.

Test Results

tests/test_architecture.py ........................  [24 passed in 1.92s]

All 24 architecture tests pass. Breakdown:

TestArchitectureBasic (2) — status ok + auto-scan on fresh workspace
TestArchitectureFields (9) — all required fields present + correct types
TestArchitectureLiteMode (2) — lite omits routes/packages/hotspots, keeps core fields
TestArchitectureHotspots (3) — sorted desc, match direct SQL query, distinct files
TestArchitectureCaching (4) — cache created, reused, invalidated after re-scan, lite/full shapes not shared
TestArchitectureMCPTool (2) — tool listed in _TOOL_DEFINITIONS + end-to-end MCP call returns full data
TestArchitectureTokenBudget (2) — raw + MCP-normalised lite payloads under 4000 bytes

Broader suite (no regressions):

560 passed, 12 skipped, 4 failed in 9.26s

The 4 failures are pre-existing in test_hybrid_engine.py (confidence-field assertions on main, not caused by this PR — verified in worklog Task 1).

Token Size Measurement

Measured on the clean_app fixture (7 python files + 1 js file, 31 graph nodes, 97 edges):

Payload	Bytes	Approx Tokens (÷4)
`--lite` raw engine payload	284	~71
`--lite` MCP-normalised response	298	~75
Full raw engine payload	502	~125
Full MCP-normalised response	515	~128

All well under the 1k-token target from the issue spec.

Verification: `architecture` works on clean_app fixture

$ python3 codelens.py architecture /tmp/clean_app --lite
{
  "status": "ok",
  "workspace": "/tmp/clean_app",
  "lite": true,
  "cached": false,
  "generated_at": "2026-06-28T06:06:34.125278+00:00",
  "stats": {
    "languages": {"python": 7, "javascript": 1},
    "frameworks": [],
    "entry_points": ["main.py"],
    "total_symbols": 31,
    "adrs": []
  }
}

Verification: hotspots use the graph model

The _compute_hotspots function issues a single SQL round-trip:

SELECT gn.file, COUNT(*) AS cnt
FROM graph_edges AS ge
INNER JOIN graph_nodes AS gn ON gn.node_id = ge.target_id
WHERE ge.target_id IS NOT NULL AND ge.target_id != ''
  AND gn.file IS NOT NULL AND gn.file != ''
GROUP BY gn.file
ORDER BY cnt DESC
LIMIT 5

On clean_app this returns:

[
  "src/utils.py (15 dependents)",
  "config/settings.py (5 dependents)",
  "src/db_queries.py (4 dependents)",
  "src/system_ops.py (4 dependents)",
  "src/api_client.js (3 dependents)"
]

The test test_hotspots_match_graph_query independently re-runs this SQL and verifies the file set matches. This is the killer feature of the new graph model (PR #24, issue #8): file-level blast-radius surfaced in one call.

Design Decisions Beyond BOS Spec

Architecture-specific fields nested inside stats — The MCP _normalize_to_ai formatter preserves stats as-is but drops unknown top-level keys. To deliver the full architecture data through MCP without modifying the shared formatter (which would collide with parallel worker 2-a's compact-format changes), all architecture fields are nested inside stats. CLI consumers see the same shape. Documented in CHANGELOG and architecture_engine.get_architecture docstring.
File-level hotspots (not per-symbol) — The SQL query groups by gn.file, not ge.target_id, so each hotspot is a distinct file with its total blast radius (sum of incoming CALLS edges across all symbols in that file). Matches the issue spec example "src/models/user.py (47 dependents)" more closely than per-symbol rows would.
Three-tier language resolution — When architecture auto-scans, the scan_result's files_scanned is used directly (no re-walk). When scan was done externally (e.g. via CLI), .codelens/summary.json is read. If neither is available, a cheap stat-only extension-counting walk is used as a last-resort fallback (no file parsing, just os.walk + os.path.splitext). This honours the BOS spec "reuse existing logic, don't re-walk files" while remaining robust to all entry points.
Auto-scan on fresh workspace — If .codelens/codelens.db doesn't exist or graph tables are empty when architecture is called, the engine runs cmd_scan first. This makes the tool self-sufficient for the issue [FEATURE] get_architecture — single-call codebase overview for agents #19 use-case ("agent starts on an unfamiliar codebase").
Cache shape isolation — Lite and full payloads have different shapes (lite omits routes/packages/hotspots). The cache won't serve the wrong shape even when the db mtime is unchanged — requesting --lite after --full (or vice versa) forces a rebuild.

Commits

4 incremental commits:

feat(arch): add architecture_engine.py with overview aggregation
feat(cmd): add architecture command + codelens_architecture MCP tool
test(arch): add architecture command + caching tests
docs(arch): update README, SKILL-QUICK, CHANGELOG, agent-integration

Closes #19.

New engine module orchestrating existing engines (framework_detect, entrypoints_engine, apimap_engine, graph_model) into a single compact codebase overview payload. Addresses issue #19. Public API: - get_architecture(workspace, lite=False) -> dict Pipeline: - _compute_languages: three-tier resolution (fresh scan_result's files_scanned -> .codelens/summary.json -> cheap stat-only extension walk). Scan buckets collapsed to canonical names (js_backend + js_frontend -> javascript). - _compute_frameworks: wraps framework_detect.detect_frameworks. - _compute_entry_points: wraps entrypoints_engine.map_entrypoints, dedupes by file path, prioritises main/handler/cli types (top 10). - _compute_packages: scans src/, app/, lib/, packages/, server/, internal/ for immediate subdirs that contain source files (top 20). - _compute_routes: wraps apimap_engine.map_api_routes, normalises to {method, path, handler} (top 20). - _compute_hotspots: single SQL round-trip against graph_edges JOIN graph_nodes GROUP BY file ORDER BY count DESC LIMIT 5. Killer feature of the new graph model: files ranked by total incoming CALLS edges. - _compute_total_symbols: graph_stats(db_path).nodes. Caching: - First call writes .codelens/architecture_cache.json with timestamp. - Subsequent calls return the cache as long as .codelens/codelens.db mtime hasn't advanced (i.e. scan hasn't been re-run). - Lite and full payloads have different shapes -- the cache won't serve the wrong shape even when the db is unchanged. Auto-scan: - If .codelens/codelens.db doesn't exist or graph tables are empty when architecture is called, the engine runs cmd_scan first. This makes the tool self-sufficient for the issue #19 use-case ('agent starts on an unfamiliar codebase'). Payload shape: - Architecture-specific fields nested inside a 'stats' dict so they survive the MCP _normalize_to_ai formatter (which preserves stats as-is but drops unknown top-level keys). CLI consumers see the same shape. Adrs field is an empty list placeholder (ADR feature is issue #16, Phase 3).

CLI command (scripts/commands/architecture.py): - Auto-registered via commands/__init__.py (no manual registration needed). - Flags: --lite (omit routes/packages/hotspots for <1k token orientation), --no-cache (force rebuild by removing architecture_cache.json). - Wraps architecture_engine.get_architecture(workspace, lite=lite). MCP tool (added to _TOOL_DEFINITIONS in scripts/mcp_server.py): - Tool name: codelens_architecture - Schema: {workspace: string} required; {format: string, lite: boolean} optional. - Description documents the issue #19 use-case and the --lite flag for <1k token orientation mode. - Result cached in .codelens/architecture_cache.json and invalidated on re-scan. The MCP _execute_command path runs _normalize_to_ai on every command result. The architecture_engine nests all architecture fields inside a 'stats' dict so they survive normalization intact -- agents calling codelens_architecture get the full overview, not just an empty {stats:{}, items:[]} shell. No other changes to mcp_server.py -- only the architecture entry is added to _TOOL_DEFINITIONS, in line with the parallel-worker file ownership agreement (2-a owns the compact-format changes).

24 test cases across 7 test classes covering all eight spec verification points from issue #19: TestArchitectureBasic (2 tests): - test_returns_ok_status: architecture returns status=ok on scanned clean_app. - test_auto_scans_on_fresh_workspace: auto-scan populates db + graph on a workspace with no .codelens/ directory. TestArchitectureFields (9 tests): - test_payload_has_all_required_fields: every issue #19 field present in stats (languages, frameworks, entry_points, packages, routes, hotspots, total_symbols, adrs). - Per-field type/shape checks (dict of counts, list of paths, list of {method,path,handler} dicts, etc). - test_adrs_is_empty_list_placeholder: confirms ADR placeholder shape. - test_total_symbols_is_int: confirms graph_stats().nodes is used. TestArchitectureLiteMode (2 tests): - test_lite_omits_routes_packages_hotspots: lite mode shape. - test_lite_keeps_core_fields: languages/frameworks/entry_points/ total_symbols still present in lite mode. TestArchitectureHotspots (3 tests): - test_hotspots_sorted_by_dependents_desc: parses '(N dependents)' and verifies descending sort. - test_hotspots_match_graph_query: independent SQL query returns the same file set as the engine (proves the graph model is the source). - test_hotspots_are_distinct_files: no duplicate file paths (group-by-file SQL contract). TestArchitectureCaching (4 tests): - test_cache_file_created_on_first_call: .codelens/architecture_cache.json appears after first call. - test_cache_reused_on_second_call: second call returns cached=True and the cache file mtime is unchanged. - test_cache_invalidated_after_rescan: re-running cmd_scan makes the next architecture call rebuild (cached=False). - test_lite_and_full_caches_not_shared: requesting lite after full (or vice versa) rebuilds because the payload shapes differ. TestArchitectureMCPTool (2 tests): - test_mcp_tool_listed: codelens_architecture is in _TOOL_DEFINITIONS with workspace (required) and lite (optional) parameters. - test_mcp_tool_returns_full_architecture_via_stats: end-to-end MCP tools/call returns the full architecture data inside stats (verifies the stats-nesting workaround for _normalize_to_ai). TestArchitectureTokenBudget (2 tests): - test_lite_payload_under_4000_bytes: raw --lite engine payload stays under 4000 bytes (~1k tokens). - test_lite_mcp_response_under_4000_bytes: MCP-normalised --lite response stays under 4000 bytes. All 24 tests pass. Uses two fixtures: scanned_clean_app (full scan done before architecture call) and fresh_clean_app (no scan -- tests auto-scan path).

README.md: - Add 'architecture' to the 'Architecture & Understanding (P1)' command table with description and flag hints (--lite, --no-cache). SKILL-QUICK.md: - Add 'architecture [--lite] [--no-cache]' to the Navigation category (10 -> 11 commands). - Update the trigger map entry for 'project overview' from 'summary or handbook' to 'architecture --lite (single call, <1k tokens) or summary / handbook (deeper)'. CHANGELOG.md: - Add new '### get_architecture -- single-call codebase overview (issue #19)' section under [8.2.0] Unreleased. - Document the new architecture_engine.py module, CLI command, MCP tool, and test suite. - Document design decisions (stats-nesting, adrs placeholder, auto-scan, file-level hotspots). - Include token budget verification measurements from clean_app fixture. references/agent-integration.md: - Update the Codebase Understanding intent-to-tool map: 'How does this app work?' now starts with 'architecture --lite' before drilling into entrypoints/api-map/state-map/outline. - Add new entries: 'Give me a codebase overview' -> architecture --lite, 'What files have the most dependents?' -> architecture (hotspots field uses the graph model). - Add 'overview, orient, unfamiliar, codebase summary, what is this project' to the keyword detection matrix mapping to architecture --lite. Only additive changes to shared docs -- no rewrites of existing content, in line with the parallel-worker file ownership agreement.

chatgpt-codex-connector · 2026-06-28T06:12:09Z

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

sonarqubecloud · 2026-06-28T06:25:06Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

worker-2-a added 4 commits June 28, 2026 06:09

Wolfvin merged commit d1a4aa9 into main Jun 28, 2026
6 of 11 checks passed

Wolfvin mentioned this pull request Jun 28, 2026

[FEATURE] get_architecture — single-call codebase overview for agents #19

Closed

Wolfvin deleted the feat/issue-19-get-architecture branch June 28, 2026 06:24

Wolfvin mentioned this pull request Jun 28, 2026

Agent UX: Confidence scores, delta output, and dependency graph between findings #5

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(arch): get_architecture single-call overview — closes #19#28

feat(arch): get_architecture single-call overview — closes #19#28
Wolfvin merged 4 commits into
mainfrom
feat/issue-19-get-architecture

Wolfvin commented Jun 28, 2026

Uh oh!

chatgpt-codex-connector Bot commented Jun 28, 2026

Uh oh!

Uh oh!

sonarqubecloud Bot commented Jun 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Wolfvin commented Jun 28, 2026

Summary

Files Touched

Test Results

Token Size Measurement

Verification: architecture works on clean_app fixture

Verification: hotspots use the graph model

Design Decisions Beyond BOS Spec

Commits

Uh oh!

chatgpt-codex-connector Bot commented Jun 28, 2026

Uh oh!

Uh oh!

sonarqubecloud Bot commented Jun 28, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Verification: `architecture` works on clean_app fixture