Code graph overlay (shadow graph)#26
Merged
Merged
Conversation
Wires the editor-overlay manager into MCP tool dispatch so every tools/call sees the calling session's unsaved editor buffers without per-tool changes. IDE extensions can keep the graph fresh during active editing without saving the file. - Apply/revert middleware (internal/mcp/overlay.go::wrapToolHandler) evicts each overlaid file and re-indexes from caller-supplied content at tools/call start, then re-indexes from disk on return. Applies are serialised through overlayApplyMu so two in-flight overlay-active requests can't race on the same path. - Indexer.IndexFileFromContent is the new content-driven re-index path; mtime tracking is skipped so the next watcher event still restores the on-disk view. MultiIndexer.IndexFileFromContent / EvictFileByAbs / ReindexFromDisk forward per-repo. - Drift detection (overlaySHAMatches) recomputes the git blob SHA at apply time and compares to the editor-captured base_sha; mismatches surface as a structured "overlay base SHA mismatch" tool error so the client re-reads and resubmits. - OverlayManager gains RegisterWithID / Has / FileCount / SnapshotFor; the MCP session ID doubles as the overlay session ID, so query-time resolution is a single map lookup from SessionIDFromContext. - Five new MCP tools — overlay_register, overlay_push, overlay_list, overlay_delete, overlay_drop — let an MCP-native extension manage overlays without the /v1/overlay/* HTTP surface. Deletion overlays (deleted: true) tombstone a path for the session's lifetime. - HTTP transport: handleOverlayRegister accepts an explicit session_id; handleToolCall reads the active session from Mcp-Session-Id (preferred), X-Gortex-Overlay-Session, or ?session_id=. All transports (daemon socket, stdio, HTTP) share one apply/revert path. - 65 internal AddTool callsites migrated from s.mcpServer.AddTool to s.addTool so every tool picks up the wrapping; tools_overlay.go keeps the raw path for the overlay-management tools themselves (applying overlays before overlay_push would re-evict the new buffer before it ever became visible). - 14 tests: 5 daemon OverlayManager (register/snapshot ordering, idempotent re-register, fast-path gates, drift, idle TTL sweep) + 3 indexer content path (replaces graph view, no mtime poisoning, deletion equivalence) + 6 MCP end-to-end (query consumption via get_file_summary, drift surfacing, BaseSHA happy path, no-session fast-path, MCP overlay_register/push/list round-trip, deletion tombstoning). - Docs: CLAUDE.md, internal/agents/instructions.go adapter template, and README.md gain a "Live Editor Buffers (Overlay Sessions)" section; README tool count 64 -> 69.
Editor extensions push in-flight (unsaved) buffers as overlays and every subsequent tools/call from the same MCP session reads through a per-request shadow view layered on top of the immutable base graph. Base is never mutated by overlay flow, so concurrent sessions each see their own view, the file watcher's reindex passes don't race with overlay queries, and cross-file edges from non-overlaid callers into overlaid symbols are preserved. - internal/graph/reader.go introduces a Reader interface satisfied by both *Graph (base, mutable through the indexer) and *OverlaidView. Tool handlers, query.Engine, and the rerank Context all consult this interface instead of *Graph directly. - internal/graph/overlay.go defines OverlayLayer (per-session parsed overlay state: nodes, resolved edges, tombstones, name-removed set) and OverlaidView (intercepts GetNode / GetFileNodes / GetOutEdges / GetInEdges / AllNodes / AllEdges / FindNodesByName to substitute the overlay's content for overlaid files while base reads pass through unchanged). - internal/mcp/overlay_view.go is the per-request builder. The layer is parsed once per (sessionID, content-hash) tuple by running the same per-language extractors the indexer uses, applying the repo prefix to match base's ID shape, then running a local resolver pass that rewrites unresolved::* placeholders against (overlay ∪ base). The view is attached to ctx via WithOverlayView; tool handlers read it via s.readerFor(ctx) / s.engineFor(ctx). - query.Engine.WithReader returns a shallow clone that swaps the reader, so engine-level walks (FindUsages, GetCallers, CallChain, Dependencies, Dependents, hierarchies) all see the overlay without per-method changes. - internal/mcp/overlay.go's wrapToolHandler is the non-mutating middleware: builds the view per request, surfaces drift as a structured "overlay base SHA mismatch" tool error, falls through as a one-map-lookup no-op when the session has no overlay. - readLinesForCtx consults the overlay buffer before disk so get_symbol_source / get_editing_context / smart_context surface the editor view, not the saved file. - New MCP tools: overlay_register, overlay_push, overlay_list, overlay_delete, overlay_drop, and compare_with_overlay — the last runs find_usages / get_callers / get_call_chain / get_dependencies / get_dependents against base AND overlay simultaneously and returns added / removed / common ID sets, the side-by-side diff the shadow design makes possible. - 65 internal AddTool callsites migrated to s.addTool (wrapping middleware); s.engine usages in handler bodies migrated to s.engineFor(ctx); fetchAndMergeBM25 / topCallersForVerify / winnowSymbols / findTestFiles refactored to take a Reader-aware engine so they're overlay-aware without a per-callsite migration. - HTTP transport reads the session from Mcp-Session-Id (preferred), X-Gortex-Overlay-Session, or ?session_id=; the register endpoint accepts an explicit session_id so HTTP clients can bind to a known MCP session. - 16 tests under -race: 5 daemon OverlayManager tests (register/snapshot/drift/sweep) and 11 MCP end-to-end including TestOverlay_BaseGraphIsImmutable, TestOverlay_FindUsagesPreservesCrossFileCallers, TestOverlay_TwoSessionsIsolated, TestOverlay_OverlayAndBaseSessionsIsolated, drift surfacing, BaseSHA match, no-session fast-path, deletion tombstones, MCP register/push/list round-trip, compare_with_overlay diff surface. - CLAUDE.md, internal/agents/instructions.go adapter template, and README.md gain a "Live Editor Buffers (Shadow-Graph Overlay Sessions)" section.
… TTL Closes the "abandoned buffer pinned in the daemon, reachable to anyone who learns the session ID" attack surface, and fixes the related in-active-use TTL trip where a long query loop without further pushes would let the idle timer reap a session the editor still cares about. - Server.ReleaseSession (called when an MCP client disconnects) now drops the corresponding overlay synchronously and invalidates the cached parse layer. The TTL becomes a fail-safe for missed disconnects rather than the primary cleanup path. - OverlayManager.SnapshotFor (the per-request view-build read path) promotes to a write lock and bumps LastUsed. Any tools/call against a live overlay counts as activity; reading no longer races with writing for the lock either way since the lookup → bump sequence was already a tight critical section. - OverlayManager gains a Touch primitive (idempotent LastUsed refresh without altering files) and StatusFor (read-only liveness query — workspace, created, last_used, idle_seconds, idle_ttl_seconds, expires_at). StatusFor deliberately does NOT bump LastUsed so that a misconfigured editor polling overlay_list can't keep a dropped session alive forever. - daemon.DefaultOverlayIdleTTL raised 5m → 30m. New daemon.OverlayIdleTTLFromEnv(override) resolves precedence: caller-supplied non-zero > GORTEX_OVERLAY_IDLE_TTL env var (time.ParseDuration syntax) > default. Garbage env values fall through to the default; we don't fail startup over a typo'd duration. Wired at both construction sites (cmd/gortex/server.go, cmd/gortex/daemon_state.go). - New overlay_keepalive MCP tool — explicit no-op heartbeat for genuine idle gaps (debugger pause, refactor wizard, deliberation) cheaper than re-pushing buffer content. Returns fresh expires_at / idle_seconds metadata so the editor can schedule the next keepalive. - overlay_list now returns created_at / last_used_at / expires_at / idle_seconds / idle_ttl_seconds and switches to the non-LastUsed-bumping StatusFor + Files read path so polling doesn't abuse the lease. The expired-session branch surfaces expired: true so the editor can detect and recover without comparing to its own state. - overlay_push / overlay_delete / overlay_drop now invalidate the per-session parsed-overlay cache so the next tools/call re-parses with fresh buffer state. (Previously the invalidate only ran on drop; mid-session pushes could observe a stale cached parse on the very next call.) - Error surfacing on stale-session paths consolidated: overlay_keepalive returns "session has been dropped or never registered — call overlay_register before pushing" so the editor knows to recover; tools/call falls through to base (no error, no leak); overlay_push self-heals. - 8 new tests across daemon + mcp covering: SnapshotFor bumps LastUsed but StatusFor does not; Touch extends lease and errors on unknown session; OverlayIdleTTLFromEnv override/env/garbage/unset precedence; ReleaseSession drops the overlay synchronously and subsequent tools/call doesn't see it; overlay_keepalive happy path + missing-session error; overlay_list response shape includes expiry metadata. Full suite: 3846 pass under go test -race ./... - CLAUDE.md, internal/agents/instructions.go, README.md describe the new lifecycle / TTL story (session-bound; 30m default; GORTEX_OVERLAY_IDLE_TTL; keepalive; reads-bump-LastUsed).
zzet
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wires the editor-overlay manager into MCP tool dispatch so every tools/call sees the calling session's unsaved editor buffers without per-tool changes. IDE extensions can keep the graph fresh during active editing without saving the file.