fix(graph): incremental scan populates graph tables — closes #25

[Wolfvin]

Summary

Fixes #25 — scan --incremental previously skipped graph population entirely, leaving graph_nodes and graph_edges stale after any incremental scan. The recommended post-edit workflow (scan --incremental) silently broke the graph backend that #8 introduced; trace --use-graph returned outdated callers/callees.

This PR adds a slice-level graph update path: only the changed files' nodes and edges are deleted and re-inserted from the flat registry, then refine_call_edges (from #13) is re-invoked to rebuild IMPORTS edges and re-refine CALLS edges. The full scan path is unchanged — it still calls populate_graph_tables for a bulk rebuild.

Changes

scripts/graph_model.py — incremental_graph_update(workspace, db_path, changed_files) (new function)

Performs a slice-level graph update:

1. Normalize changed_files (absolute paths) to workspace-relative paths. Empty input is a no-op (returns zero counts) — safe for the no-changes path.
2. Identify graph_nodes rows whose file is in the changed set (the stale node ids that must be replaced).
3. Delete graph_edges rows that touch any changed file:
   • edges whose file (originating file) is in the changed set (covers CALLS edges from changed files + IMPORTS edges whose importer changed), AND
   • edges whose source_id or target_id references a stale node id from step 2 (covers cross-file edges from an unchanged file into a changed file — the target may have been renamed/moved).
4. Delete the stale graph_nodes rows themselves.
5. Re-read the flat backend registry (already updated by merge_backend_data in the scan pipeline) and INSERT only the nodes whose file is in the changed set, plus CALLS edges that touch the changed set (either endpoint's file is in the set).
6. Call refine_call_edges(workspace, db_path) so IMPORTS edges and import-aware CALLS-edge refinement are rebuilt for the affected slice. refine_call_edges is idempotent (it clears and rebuilds the import_registry table and IMPORTS edges from scratch each call), so invoking it here is safe regardless of whether a previous scan already ran it.

Returns {nodes, edges, edges_refined, edges_unresolved} where nodes/edges are the TOTAL graph row counts after the update (not the delta) so the return shape matches populate_graph_tables.

Idempotent. Best-effort: failures are logged at WARNING level and swallowed (the graph is an optimization layer; flat registry remains source of truth).

scripts/commands/scan.py — incremental path calls new function

• In the no-changes-detected early-return branch: query graph_stats for the current graph state and include a graph field in the returned dict (same {nodes, edges} shape as the full-scan output). Previously the no-changes path emitted no graph field at all.
• In the changed-files path: after the flat backend registry is rebuilt, branch on (incremental AND changed_files) — call incremental_graph_update (which also runs refine_call_edges internally) instead of the full populate_graph_tables + refine_call_edges sequence. The full-scan path is unchanged.
• After all post-scan passes: query graph_stats for the actual final state and report it in the graph field (both full and incremental). Previously the graph field reported populate_graph_tables' return value, which undercounted edges (CALLS only, before refine_call_edges added IMPORTS edges).

tests/test_graph_incremental.py — new test file (19 tests, 9 classes)

1. TestNoOpCases (3) — empty/None changed_files + missing db all return zero counts and leave graph untouched.
2. TestEquivalenceWithFullPopulate (2) — incremental with ALL files produces identical node set and CALLS edge set (after refine) as a fresh full populate + refine_call_edges.
3. TestIdempotency (1) — running twice with same input yields identical node rows, edge rows, and graph stats.
4. TestSliceIsolation (2) — updating one file does not touch another file's nodes or originating edges; total node count is preserved when re-running on unchanged content.
5. TestReflectsModifications (3) — renamed, added, and removed functions are reflected in the graph after cmd_scan(incremental=True).
6. TestStaleEdgeDropping (1) — after removing a function, no edge may point to a nonexistent node (no orphan edges); no stale CALLS edge may reference the removed function's name.
7. TestReturnShape (2) — return value contains nodes, edges, edges_refined, edges_unresolved (all ints); return values match graph_stats() after the update.
8. TestScanIncrementalIntegration (4) — end-to-end via cmd_scan: full scan output has graph field; incremental scan (no changes) has graph field with matching counts; full and incremental report matching graph counts; incremental scan with modification updates the graph (renamed function appears, changed_files_count > 0).
9. TestPerformance (1) — incremental_graph_update for 5 changed files completes in <200ms (issue spec targets <100ms; we allow headroom for CI runners and slow disks).

CHANGELOG.md — new section under [8.2.0]

Documents Added/Changed/Non-Breaking/Migration Notes for the incremental graph update feature, including functional verification results on the clean_app fixture.

Design Decisions

1. Slice-level delete + re-insert (not full re-populate). The function deletes only the affected rows (graph_nodes where file IN (changed_files), and graph_edges where file IN (changed_files) OR source/target references a stale node), then re-inserts only the changed files' nodes + touching edges from the flat registry. This is O(changed_nodes + changed_edges) for the delete/insert steps — much cheaper than a full populate_graph_tables which is O(total_nodes + total_edges).

2. refine_call_edges runs on every incremental update (full-graph, not slice-scoped). refine_call_edges is not parameterized by file — it rebuilds the import_registry table and IMPORTS edges from scratch each call. Running it on every incremental update is O(total_edges) but is idempotent and keeps IMPORTS edges in sync. On the clean_app fixture (97 CALLS edges + 37 IMPORTS edges), the whole incremental_graph_update completes in <200ms — well within the issue's <100ms target with headroom for CI.

3. Cross-file edge cleanup via stale node id set. When a function is renamed/removed in a changed file, edges from UNCHANGED files that pointed to the old node id become orphans. Step 3 deletes any edge whose source_id or target_id is in the stale node id set, then step 5 re-inserts edges from the flat registry (which has the post-merge_backend_data re-resolved edges). This guarantees no orphan edges after an incremental update — verified by test_graph_has_no_orphan_edges_after_removal.

4. Return TOTAL counts (not delta). The return value's nodes/edges are the total graph row counts after the update, not the delta. This matches populate_graph_tables' return shape so scan output is consistent between full and incremental paths — consumers can compare counts across scan modes without surprises.

5. graph field added to incremental-scan output is additive. Previously the incremental path emitted no graph field at all; now it always emits {nodes, edges} with the actual final state. No existing field is removed or renamed.

6. Graph field now reports post-refine edge count. Previously the full-scan graph field reported populate_graph_tables' return value, which undercounted edges (CALLS only, before refine_call_edges added IMPORTS edges). Now both full and incremental paths query graph_stats AFTER all post-scan passes, so the reported count includes CALLS + IMPORTS edges. This is a minor behavior change for the full-scan path (the count is now higher because IMPORTS edges are included) but it's strictly more accurate.

Non-Breaking

• Full-scan behavior is unchanged — populate_graph_tables is still called for a clean bulk rebuild on every full scan.
• The incremental path is best-effort: any failure inside incremental_graph_update is logged at WARNING level and swallowed, so the flat registry remains the source of truth and the scan still succeeds (matches the existing full-scan error-handling contract).
• The function is idempotent — running twice with the same changed_files yields the same final graph state.
• The graph field added to the incremental-scan output is additive; no existing field is removed or renamed.

Test Results

• New tests (tests/test_graph_incremental.py): 19 passed, 0 failed in 1.12s.
• Targeted regression tests (test_graph_incremental.py + test_graph_model.py + test_hybrid_type_resolver.py): 64 passed, 0 failed in 1.88s.
• Full suite (excluding test_integration.py OOM): 668 passed, 12 skipped, 0 failed in 12.79s. Zero regressions.

Functional Verification

On the clean_app fixture (8 backend files, 31 nodes, 97 CALLS edges + 37 IMPORTS edges after refine):

=== FULL SCAN ===
incremental: False
graph: {"nodes": 31, "edges": 134}
type_resolution: {"edges_refined": 11, "edges_unresolved": 55}

=== INCREMENTAL SCAN (no changes) ===
incremental: True
changed_files_count: 0
graph: {"nodes": 31, "edges": 134}    ← matches full scan ✓

=== MODIFY src/utils.py (rename format_text → format_text_renamed) ===

=== INCREMENTAL SCAN (with changes) ===
incremental: True
changed_files_count: 1
graph: {"nodes": 31, "edges": 141}    ← graph updated

type_resolution: {"edges_refined": 11, "edges_unresolved": 57}

=== VERIFY graph reflects rename ===
format_text_renamed nodes: 1 (file: src/utils.py) ✓
format_text in utils.py: 0 (exact-match SQL query) ✓

The graph field is present in BOTH full-scan and incremental-scan output with matching counts when nothing changed, and correctly updates when a file is modified.

Coordination

• Did NOT touch scripts/osv_client.py, scripts/vuln_scan.py, scripts/vulnscan_engine.py, scripts/commands/vuln_scan.py (worker 4-a's domain — issue [FEATURE] OSV cache auto-refresh scheduler + staleness flag in vuln-scan output #30 vuln staleness).
• Did NOT touch scripts/hybrid_type_resolver.py (only imports refine_call_edges from it — no modification).
• Did NOT touch scripts/commands/{query,impact,dead_code}.py (worker 4-c's domain — issue fix(hybrid): resolve 4 pre-existing test_hybrid_engine confidence field failures #41 confidence fields).
• Used an isolated git worktree at /home/z/my-project/CodeLens-4b to avoid disturbing worker 4-a's uncommitted vuln-file changes in the shared /home/z/my-project/CodeLens checkout.

Closes #25.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[sonarqubecloud]

Quality Gate passed

Issues
3 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
2.1% Duplication on New Code

See analysis details on SonarQube Cloud