Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
f7891f5
feat(cache): make persisted indexes portable
lzehrung Aug 15, 2026
0a2c038
test(cache): verify portable persisted identities
lzehrung Aug 15, 2026
75fa0b2
test(cache): cover resolver and bloom invalidation
lzehrung Aug 15, 2026
2fed86f
fix(cache): preserve duplicate consumer path contract
lzehrung Aug 15, 2026
e295e69
fix(cache): preserve portable path identities
lzehrung Aug 16, 2026
4c700af
fix(cache): confine persisted path rehydration
lzehrung Aug 16, 2026
30d1090
fix: confine persisted cache paths
lzehrung Aug 16, 2026
3e907f4
fix cache path portability validation
lzehrung Aug 16, 2026
eeb0c5e
fix cache external reexports
lzehrung Aug 16, 2026
f742370
fix cache workspace external reexports
lzehrung Aug 16, 2026
1ea7cc7
perf: remove repeated hot-path work
lzehrung Aug 15, 2026
dfeeef7
perf: collapse query candidate and fallback scans
lzehrung Aug 15, 2026
ac184fe
fix: type performance hot paths safely
lzehrung Aug 15, 2026
a6ed82e
fix: restore hot-path review regressions
lzehrung Aug 16, 2026
72ee6b9
perf(cache): batch writes and hydrate bloom sidecar
lzehrung Aug 15, 2026
32eef3f
fix(cache): validate hydrated snapshot artifacts
lzehrung Aug 16, 2026
042b059
fix(cache): preserve hydrated bloom validation
lzehrung Aug 16, 2026
7ec1fc8
fix(cache): keep diagnostics lightweight
lzehrung Aug 16, 2026
edcf2aa
fix: preserve cache location defaults
lzehrung Aug 16, 2026
19da28d
fix: preserve configured cache locations
lzehrung Aug 16, 2026
925cbc4
fix(doctor): report the effective cache anchor/layer, not the intende…
lzehrung Aug 16, 2026
275d6a1
fix: address suppressed PR review findings across cache portability, …
lzehrung Aug 16, 2026
ea40ee8
fix: address second batch of suppressed PR review findings
lzehrung Aug 16, 2026
d2c50af
fix: address third batch of review findings - manifest, session confi…
lzehrung Aug 17, 2026
b98fdb1
fix: address fourth batch of suppressed review findings
lzehrung Aug 17, 2026
ce07346
fix: prefer content-hash cacheSig over weak mtime:size sig in snapsho…
lzehrung Aug 17, 2026
5d3e50e
fix: address fifth batch of review findings on cache identity and nod…
lzehrung Aug 17, 2026
adb3ec6
fix: address sixth batch of review findings on portable cache identity
lzehrung Aug 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
- For repo-understanding flows, start with `node ./dist/cli.js doctor` and `node ./dist/cli.js orient --root . --budget small --json` when `dist` is built; build first if validating the working tree from a fresh checkout.
- For source-checkout validation and contributor examples, prefer `node ./dist/cli.js ...`; reserve bare `codegraph ...` for published/global install guidance.
- When package metadata, install scripts, optional native dependencies, or the resolved npm graph changes, update `package-lock.json` in the same change and verify with `npm ci --ignore-scripts --dry-run` unless lifecycle scripts are part of the behavior under test.
- Treat `--root` as the project boundary for config lookup, cache/manifests, path confinement, and output normalization. When `--root` is set, positional paths are include roots; for `orient` and `drift`, positional paths are always include roots.
- Treat `--root` as the project boundary for config lookup, path confinement, and output normalization. Cache/manifests may use the resolved cache anchor (`--cache-dir`/`CODEGRAPH_CACHE_DIR`, repository metadata, or project root); cached contents remain project-relative.
- Keep discovery glob guidance accurate: `codegraph.config.json` globs are project-root-relative, while CLI `--include-glob`/`--ignore-glob` values are one-off filters relative to each active scan root.
- Within any claimed cross-language capability, behavior should stay consistent across all supported languages for that capability. Avoid language-subset branches; if a limitation is intentional, document it in the parity docs and cover it with explicit tests in the same change.
- When language support changes, update `docs/language-parity.md` and `docs/scenario-catalog.md` in the same change so support claims, limitations, and fixture coverage stay aligned.
Expand Down
2 changes: 1 addition & 1 deletion codegraph-skill/codegraph/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Prefer `review` before `impact`: review is the compact reviewer handoff; impact

## Keep the Project Boundary Explicit

Use `--root` to define the boundary for config lookup, cache scope, path confinement, and output normalization.
Use `--root` to define the boundary for config lookup, path confinement, and output normalization. Cache contents use project-relative paths and may live at the resolved repository anchor; override location with `--cache-dir` or `CODEGRAPH_CACHE_DIR`.

- Positional paths are include roots inside the project boundary for `orient`, `drift`, and positional graph commands.
- `codegraph.config.json` discovery globs are project-root-relative.
Expand Down
4 changes: 4 additions & 0 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ Default workflow:
- first-turn map: `codegraph orient --root . --budget small`
- targeted follow-up: `codegraph search "<query>" --json` then `codegraph explain <handle|file|symbol>`

## Cache location

Index caches store project-relative paths, so a cache can be moved with its project. Cache selection precedence is `--cache-dir`, `CODEGRAPH_CACHE_DIR`, `cache.location` in project config (then user config), repository metadata, then the project root. `cache.location` accepts `project`, `repo`, `user`, or an absolute path; `--root` remains the project scope boundary.

## Runtime selection

The CLI defaults to `--native auto`, which uses the native Tree-sitter path when a compatible native artifact is available and falls back automatically otherwise.
Expand Down
20 changes: 20 additions & 0 deletions docs/library-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -624,6 +624,26 @@ const incremental = await buildProjectIndexIncremental(root, {

`BuildOptions.onProgress` reports index lifecycle and file progress. A rebuild emits `phase: "start"` with `mode: "build"` or `"update"`, zero or more `phase: "update"` events, and `phase: "complete"` with `elapsedMs`; a reusable snapshot emits no progress events.

## Cache location

`BuildOptions.cacheDir` and `BuildOptions.cacheLocation` control where the disk cache
(`cache: "disk"`) is written; they have no effect for `cache: "memory"` or `"off"`. Persisted
cache contents store project-relative paths, so a cache can be moved along with its project.

Anchor selection precedence: `cacheDir`, then `CODEGRAPH_CACHE_DIR`, then `cacheLocation`, then
repository metadata (nearest ancestor `.git`/`.codegraph`), then the project root. `cacheLocation`
accepts `"project"` (anchor at `projectRoot`), `"user"` (anchor at the platform user cache
directory), `"repo"` (the default repository-metadata search), or an absolute path. None of
`cacheDir`, `CODEGRAPH_CACHE_DIR`, or an absolute `cacheLocation` is the final cache directory:
each is an anchor, and the resolved cache lives in a project-namespaced subdirectory underneath
it, since one anchor can be shared by multiple projects. The project root's own default location
(no anchor configured) is the one exception, since only one project can occupy it.

`createAgentSession()` (when `useConfig` is not disabled) and `createCodeReviewSession()` both
read `cache.location` from `codegraph.config.json` (project config, falling back to user
config) the same way they merge `discovery`, `graph`, and `languages.extensions`; an explicit
`buildOptions.cacheLocation`/`cacheDir` always takes precedence over the config value.

## Project file discovery and graph building

`listProjectFiles` defaults to source files plus common project manifests and lockfiles across supported languages, for example `package.json`, `requirements.txt`, `pyproject.toml`, and `Cargo.toml`.
Expand Down
95 changes: 47 additions & 48 deletions src/agent/query-index/candidates.ts
Original file line number Diff line number Diff line change
@@ -1,46 +1,50 @@
import {
codePointLength,
escapeFtsTrigramTerm,
QUERY_INDEX_CANDIDATE_ROW_LIMIT,
type QueryIndexStore,
type StoredQueryIndexChunk,
} from "./store.js";
import { normalizeQuerySearchText } from "./content.js";
import { QUERY_INDEX_CANDIDATE_ROW_LIMIT, type QueryIndexStore, type StoredQueryIndexChunk } from "./store.js";

export const QUERY_INDEX_CANDIDATE_VERSION = 5;
export const QUERY_INDEX_CANDIDATE_VERSION = 6;

type CandidateScore = {
export type QueryIndexCandidateScore = {
score: number;
matched: string[];
exactPhrase: boolean;
proximity: boolean;
matchedTerms: number;
};

function scoreCandidateChunk(normalizedText: string, rankTerms: readonly string[]): CandidateScore {
export type QueryIndexCandidate = StoredQueryIndexChunk & {
score: QueryIndexCandidateScore;
matchedLine: number;
};

function scoreCandidateChunk(
normalizedText: string,
rankTerms: readonly string[],
normalizedRankPhrase = rankTerms.join(" "),
): QueryIndexCandidateScore {
if (!normalizedText.length || !rankTerms.length) {
return { score: 0, exactPhrase: false, proximity: false, matchedTerms: 0 };
return { score: 0, matched: [], exactPhrase: false, proximity: false, matchedTerms: 0 };
}
const words = new Set(normalizedText.split(/\s+/).filter(Boolean));
const compact = normalizedText.replace(/\s+/g, "");
const matched: string[] = [];
let score = 0;
let matchedTerms = 0;
for (const term of rankTerms) {
if (words.has(term)) {
score += 10;
matchedTerms += 1;
matched.push(term);
} else if (compact.includes(term)) {
score += 7;
matchedTerms += 1;
matched.push(term);
} else if (normalizedText.includes(term)) {
score += 4;
matchedTerms += 1;
matched.push(term);
}
}
let exactPhrase = false;
let proximity = false;
if (matchedTerms === rankTerms.length && rankTerms.length > 1) {
if (matched.length === rankTerms.length && rankTerms.length > 1) {
score += 12;
const normalizedPhrase = rankTerms.join(" ");
if (normalizedText.includes(normalizedPhrase)) {
if (normalizedText.includes(normalizedRankPhrase)) {
score += 30;
exactPhrase = true;
} else {
Expand All @@ -57,52 +61,47 @@ function scoreCandidateChunk(normalizedText: string, rankTerms: readonly string[
if (proximity) score += 10;
}
}
return { score, exactPhrase, proximity, matchedTerms };
return { score, matched, exactPhrase, proximity, matchedTerms: matched.length };
}

function compareCandidateChunks(
left: { chunk: StoredQueryIndexChunk; score: CandidateScore },
right: { chunk: StoredQueryIndexChunk; score: CandidateScore },
function firstMatchingLine(
text: string,
rankTerms: readonly string[],
normalizedRankPhrase = rankTerms.join(" "),
): number {
const lines = text.split(/\r?\n/);
const matchIndex = lines.findIndex(
(line) => scoreCandidateChunk(normalizeQuerySearchText(line), rankTerms, normalizedRankPhrase).score > 0,
);
return matchIndex >= 0 ? matchIndex : 0;
}

function compareCandidateChunks(left: QueryIndexCandidate, right: QueryIndexCandidate): number {
return (
right.score.score - left.score.score ||
Number(right.score.exactPhrase) - Number(left.score.exactPhrase) ||
Number(right.score.proximity) - Number(left.score.proximity) ||
right.score.matchedTerms - left.score.matchedTerms ||
left.chunk.path.localeCompare(right.chunk.path) ||
left.chunk.ordinal - right.chunk.ordinal
left.path.localeCompare(right.path) ||
left.ordinal - right.ordinal
);
}

export function findQueryIndexChunkCandidates(
store: QueryIndexStore,
rankTerms: readonly string[],
): StoredQueryIndexChunk[] {
const directCandidates = new Map<string, StoredQueryIndexChunk>();
normalizedRankPhrase = rankTerms.join(" "),
): QueryIndexCandidate[] {
const terms = rankTerms.filter((term) => term.length);
const eligiblePaths = store.eligibleFilePaths(terms);
const eligiblePathSet = new Set(eligiblePaths);
for (const term of terms) {
let chunks: StoredQueryIndexChunk[];
if (codePointLength(term) >= 3) {
chunks = store.ftsChunkCandidates(escapeFtsTrigramTerm(term));
} else {
chunks = store.substringChunkCandidates(term, eligiblePaths);
}
for (const chunk of chunks) {
if (eligiblePathSet.has(chunk.path)) directCandidates.set(`${chunk.path}\0${chunk.ordinal}`, chunk);
}
}

for (const term of terms) {
for (const chunk of store.compactChunkCandidates(term, eligiblePaths)) {
directCandidates.set(`${chunk.path}\0${chunk.ordinal}`, chunk);
}
}
return [...directCandidates.values()]
.map((chunk) => ({ chunk, score: scoreCandidateChunk(chunk.normalizedText, terms) }))
return store
.candidateChunksForTerms(terms, eligiblePaths)
.map((chunk) => ({
...chunk,
score: scoreCandidateChunk(chunk.normalizedText, terms, normalizedRankPhrase),
matchedLine: firstMatchingLine(chunk.text, terms, normalizedRankPhrase),
}))
.filter((candidate) => candidate.score.score > 0)
.sort(compareCandidateChunks)
.slice(0, QUERY_INDEX_CANDIDATE_ROW_LIMIT)
.map((candidate) => candidate.chunk);
.slice(0, QUERY_INDEX_CANDIDATE_ROW_LIMIT);
}
64 changes: 64 additions & 0 deletions src/agent/query-index/store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ export class QueryIndexStore {
try {
ensureQueryIndexSchema(db);
db.pragma("journal_mode = WAL");
db.pragma("synchronous = NORMAL");
db.pragma("foreign_keys = ON");
} catch (error) {
db.close();
Expand Down Expand Up @@ -321,6 +322,69 @@ export class QueryIndexStore {
}
}

candidateChunksForTerms(
terms: readonly string[],
paths: readonly string[],
limit = QUERY_INDEX_CANDIDATE_PREFETCH_LIMIT,
): StoredQueryIndexChunk[] {
if (!terms.length || !paths.length) return [];
const normalizedLimit = normalizedCandidateLimit(limit);
// Bound each term independently instead of sharing one global, path-ordered budget:
// a common term matching thousands of early-path chunks would otherwise exhaust the
// budget before a rarer term's (or a multi-term) match later in path order is read.
const perTermLimit = Math.max(1, Math.ceil(normalizedLimit / terms.length));
const candidates = new Map<string, StoredQueryIndexChunk>();
const batchSize = 500;
for (const term of terms) {
const isFtsEligible = codePointLength(term) >= 3;
const conditions: string[] = [];
const parameters: string[] = [];
if (isFtsEligible) {
conditions.push("chunks.chunk_id IN (SELECT rowid FROM fts_matches)");
} else {
conditions.push("instr(chunks.normalized_text, ?) > 0");
parameters.push(term);
}
conditions.push("instr(replace(chunks.normalized_text, ' ', ''), ?) > 0");
parameters.push(term);
const prefix = isFtsEligible
? "WITH fts_matches AS (SELECT rowid FROM chunk_search WHERE chunk_search MATCH ?)"
: "";
let termMatches = 0;
for (let offset = 0; offset < paths.length && termMatches < perTermLimit; offset += batchSize) {
const batch = paths.slice(offset, offset + batchSize);
const placeholders = batch.map(() => "?").join(", ");
const remaining = perTermLimit - termMatches;
const rows = this.db
.prepare(
`
${prefix}
SELECT files.path AS path, chunks.ordinal, chunks.kind, chunks.name,
chunks.start_line, chunks.end_line, chunks.text, chunks.normalized_text
FROM chunks
JOIN files ON files.file_id = chunks.file_id
WHERE files.path IN (${placeholders})
AND (${conditions.join(" OR ")})
ORDER BY files.path, chunks.ordinal
LIMIT ?
`,
)
.all(
...(isFtsEligible ? [escapeFtsTrigramTerm(term), ...batch, ...parameters] : [...batch, ...parameters]),
remaining,
) as Array<Record<string, unknown>>;
for (const row of rows) {
const chunk = storedCandidateChunkFromRow(row);
if (!chunk) continue;
const key = `${chunk.path}\0${chunk.ordinal}`;
if (!candidates.has(key)) termMatches += 1;
candidates.set(key, chunk);
}
}
}
return [...candidates.values()];
}

ftsChunkCandidates(query: string, limit = QUERY_INDEX_CANDIDATE_PREFETCH_LIMIT): StoredQueryIndexChunk[] {
const normalizedLimit = normalizedCandidateLimit(limit);
const rows = this.db
Expand Down
13 changes: 11 additions & 2 deletions src/agent/renamePreview.ts
Original file line number Diff line number Diff line change
Expand Up @@ -525,7 +525,11 @@ async function addScopeConflicts(
const file = normalizeAgentFilePath(snapshot.root, target.file);
let localCollision: SymbolDef | undefined;
try {
const parsed = await ensureParsedContext(target.file, snapshot.index.parsed?.get(fileIdentityKey(target.file)));
const parsed = await ensureParsedContext(
target.file,
snapshot.index.parsed?.get(fileIdentityKey(target.file)),
snapshot.index.languageExtensions,
);
const scopeIndex = getCachedScope(snapshot.index, target.file, moduleIndex, parsed);
const targetBinding = scopeIndex.all.find(
(binding) =>
Expand Down Expand Up @@ -604,6 +608,7 @@ async function addScopeConflicts(
const parsed = await ensureParsedContext(
reference.file,
snapshot.index.parsed?.get(fileIdentityKey(reference.file)),
snapshot.index.languageExtensions,
);
const scopeIndex = getCachedScope(snapshot.index, reference.file, consumer, parsed);
const activeBinding = scopeIndex.all.find((binding) => binding.import === activeImport);
Expand Down Expand Up @@ -783,7 +788,11 @@ async function collectTextualRenameEdits(
if (!loaded) continue;
let parsed;
try {
parsed = await ensureParsedContext(file, input.snapshot.index.parsed?.get(fileIdentityKey(file)));
parsed = await ensureParsedContext(
file,
input.snapshot.index.parsed?.get(fileIdentityKey(file)),
input.snapshot.index.languageExtensions,
);
} catch (error: unknown) {
input.unsafeSites.push({
location: { file: loaded.displayPath, range: zeroRange() },
Expand Down
Loading