diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 5544cb777a..f8165fbd43 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -127,6 +127,7 @@ Current stable codes and triggers: | `stale_fold_key_version` | folded rows were stamped with an older fold-key version | `cdidx backfill-fold` or full rebuild | | `stale_fold_key_fingerprint` | folded rows were stamped under an older runtime fingerprint | `cdidx backfill-fold` or full rebuild | | `fold_rows_not_restamped` | fold metadata is current but one or more folded rows were not restamped | `cdidx backfill-fold` or full rebuild | +| `fold_ready_bit_set_but_rows_incomplete` | row-level verification found NULL folded-name values even though the fold-ready bit is set | `cdidx backfill-fold` or full rebuild | | `fold_ready=false` | aggregate fold readiness bit is degraded | `cdidx backfill-fold` or full rebuild | | `sql_graph_contract_ready=false` | SQL graph rows do not match the current call-column / qualified-name contract | `cdidx index ` | | `hotspot_family_ready=false` | one or more hotspot-family languages lack current authoritative family stamps | `cdidx index ` | @@ -829,7 +830,7 @@ Process exit codes are coarse (`0` success, `1` usage, `2` not-found, `3` db, `4 - **Path-aware narrowing and ranking** — `search`, `definition`, `references`, `callers`, `callees`, `symbols`, and `files` share path include/exclude filters plus `--exclude-tests`. Read queries prefer source files over tests/docs, and full-text search boosts exact symbol-name and path matches to surface likely implementation files first. - **Compact search snippets for AI** — `search --json` and MCP `search` return match-centered snippets with explicit snippet ranges, match lines, highlights, context counts, `truncated_line_count`, and `truncation_context` instead of whole chunks. `truncation_context.char_counts` and `truncation_context.total_chars` expose the omitted character counts behind each clamped snippet line, while truncated highlights also carry `truncated_char_counts`. `--snippet-lines` lets clients trade recall for smaller payloads, and `--max-line-width` (CLI) / `maxLineWidth` (MCP) routes each snippet line through the same `LineWidthFormatter.ClampLine` contract used by `find` / `references` / `excerpt` / `inspect` so hits inside minified / transpiled / generated single-line files no longer return hundreds of KB per result unless the caller explicitly sets `0`; clamped lines carry `...(+N)...` markers and `highlights[].truncated` / `highlights[].original_line_length`. - **Repo map for first-pass orientation** — `map` aggregates languages, modules, top files, file hot spots, and likely entrypoints from indexed data so AI clients can decide where to look before issuing precise queries. Entrypoint inference now falls back to known top-level entry files when symbol extraction does not produce an explicit `Main`-style symbol. -- **Freshness metadata for trust decisions** — `status` exposes whole-workspace freshness and git state, plus trust metadata such as `sql_graph_contract_ready` / `sql_graph_contract_degraded_reason`, `hotspot_family_ready` / `hotspot_family_degraded_reason`, forward-compatibility audit fields (`index_writer_version`, `index_newer_than_reader`, `index_newer_than_reader_reason` — see "Forward-compatibility readiness audit"), and fold-only remediation fields (`fold_ready_reason`, `degraded_reason`, `recommended_action`, `alternative_action`) so AI clients can tell up front whether SQL graph/dependency/impact answers, duplicate-name hotspot families, and Unicode `--exact` are authoritative. It also carries `unknown_extension_file_count` after a current full-repository scan so extension-table coverage gaps are visible even when those files were excluded from indexing. When those fold-only remediation fields are derived from an explicit read-only `file:` DB URI, they are normalized back to a writable filesystem path for both absolute (`file:///...?...`) and relative (`file:codeindex.db?...`) forms instead of echoing the read-only URI into commands that would fail. `cdidx index` JSON/human readiness output also surfaces the same trust bits, keeping the post-index readiness summary aligned with `status`. `impact` / MCP `impact_analysis` also mirror the SQL graph-contract signal in JSON so stale SQL rows do not masquerade as authoritative zero-impact answers. `inspect` / MCP `analyze_symbol` and `references` / MCP `references` now mirror that same SQL graph-contract signal whenever SQL-backed graph reads contribute to their payloads, so stale SQL rows do not look like authoritative hits or zero-result answers there either. `map` keeps `indexed_at` / `latest_modified` scoped to the filtered result set and also exposes `workspace_indexed_at` / `workspace_latest_modified` for whole-workspace freshness. `inspect` mirrors those whole-workspace timestamps and git fields so symbol-oriented AI flows can make trust decisions without a separate `status` call. `files` exposes per-file checksum plus modified/indexed timestamps. File-column migrations are applied opportunistically for older DBs, and read paths are designed to avoid crashing if in-place migration is unavailable. CLI and MCP zero-result JSON responses for `search`, `files`, `symbols`, `definition`, `references`, `callers`, `callees`, `deps`, `unused`, `hotspots`, and `impact` include `indexed_file_count`, `indexed_at`, and `freshness_available`. `indexed_at:null` with `freshness_available=true` means the index is empty, while `freshness_available=false` means a legacy/read-only DB could not expose freshness timestamps and `freshness_degraded_reason` explains why. **HEAD-aware staleness signal**: every successful `cdidx index` full scan now stamps the captured `git HEAD` into `codeindex_meta` so subsequent runs can compare it against the workspace HEAD. When they differ and the user did not pass `--rebuild`, the CLI emits a `head_changed` warning recommending `cdidx index --rebuild` and exposes `head_changed` / `prior_indexed_head_commit` / `current_head_commit` / `head_change_notice` in `index --json`. `status --check` mirrors the same comparison through `workspace_check.head_changed` (alongside `indexed_head_commit` / `workspace_head_commit` when they differ), so AI clients that already gate on freshness can refuse to trust a default incremental scan after `git switch ` without a separate query. `--commits` / `--files` partial updates deliberately preserve the captured HEAD so the staleness signal survives until a real full scan reindexes the worktree. Non-git workspaces and legacy DBs that never captured a HEAD skip the comparison instead of false-positive flagging. +- **Freshness metadata for trust decisions** — `status` exposes whole-workspace freshness and git state, plus trust metadata such as `sql_graph_contract_ready` / `sql_graph_contract_degraded_reason`, `hotspot_family_ready` / `hotspot_family_degraded_reason`, forward-compatibility audit fields (`index_writer_version`, `index_newer_than_reader`, `index_newer_than_reader_reason` — see "Forward-compatibility readiness audit"), and fold remediation fields (`fold_ready_reason`, `degraded_reason`, `recommended_action`, `alternative_action`) so AI clients can tell up front whether SQL graph/dependency/impact answers, duplicate-name hotspot families, and Unicode `--exact` are authoritative. CLI `status --json` and MCP `status` both populate those fold remediation fields when `fold_ready=false`. It also carries `unknown_extension_file_count` after a current full-repository scan so extension-table coverage gaps are visible even when those files were excluded from indexing. When those fold remediation fields are derived from an explicit read-only `file:` DB URI, they are normalized back to a writable filesystem path for both absolute (`file:///...?...`) and relative (`file:codeindex.db?...`) forms instead of echoing the read-only URI into commands that would fail. `cdidx index` JSON/human readiness output also surfaces the same trust bits, keeping the post-index readiness summary aligned with `status`. `impact` / MCP `impact_analysis` also mirror the SQL graph-contract signal in JSON so stale SQL rows do not masquerade as authoritative zero-impact answers. `inspect` / MCP `analyze_symbol` and `references` / MCP `references` now mirror that same SQL graph-contract signal whenever SQL-backed graph reads contribute to their payloads, so stale SQL rows do not look like authoritative hits or zero-result answers there either. `map` keeps `indexed_at` / `latest_modified` scoped to the filtered result set and also exposes `workspace_indexed_at` / `workspace_latest_modified` for whole-workspace freshness. `inspect` mirrors those whole-workspace timestamps and git fields so symbol-oriented AI flows can make trust decisions without a separate `status` call. `files` exposes per-file checksum plus modified/indexed timestamps. File-column migrations are applied opportunistically for older DBs, and read paths are designed to avoid crashing if in-place migration is unavailable. CLI and MCP zero-result JSON responses for `search`, `files`, `symbols`, `definition`, `references`, `callers`, `callees`, `deps`, `unused`, `hotspots`, and `impact` include `indexed_file_count`, `indexed_at`, and `freshness_available`. `indexed_at:null` with `freshness_available=true` means the index is empty, while `freshness_available=false` means a legacy/read-only DB could not expose freshness timestamps and `freshness_degraded_reason` explains why. **HEAD-aware staleness signal**: every successful `cdidx index` full scan now stamps the captured `git HEAD` into `codeindex_meta` so subsequent runs can compare it against the workspace HEAD. When they differ and the user did not pass `--rebuild`, the CLI emits a `head_changed` warning recommending `cdidx index --rebuild` and exposes `head_changed` / `prior_indexed_head_commit` / `current_head_commit` / `head_change_notice` in `index --json`. `status --check` mirrors the same comparison through `workspace_check.head_changed` (alongside `indexed_head_commit` / `workspace_head_commit` when they differ), so AI clients that already gate on freshness can refuse to trust a default incremental scan after `git switch ` without a separate query. `--commits` / `--files` partial updates deliberately preserve the captured HEAD so the staleness signal survives until a real full scan reindexes the worktree. Non-git workspaces and legacy DBs that never captured a HEAD skip the comparison instead of false-positive flagging. - **Folded-key upgrade without reparse** — `backfill-fold` and MCP `backfill_fold` recompute `name_folded` / `*_folded` directly from existing DB rows, then stamp `FoldReadyFlag` once verification confirms no required folded values remain NULL. This gives AI clients and users a low-cost upgrade path from pre-#86 DBs without re-reading every source file, and it also rewrites all folded rows when `fold_key_version` is missing or mismatched so future `NameFold.Version` bumps cannot silently restamp stale keys. - **Bundled symbol analysis** — `inspect` and MCP `analyze_symbol` return definition, nearby symbols, references, callers, callees, file metadata, workspace trust metadata, and graph-support metadata in one request so AI clients can answer common symbol questions with fewer round-trips. - **Language-aware reference extraction** — `references`, `callers`, `callees`, and `impact` are backed by an indexed reference table built only for languages where regex-based call/reference extraction is meaningful (30 of 46 languages). Unsupported languages intentionally fall back to text search instead of returning low-confidence pseudo-graph data. When a language is removed from graph support, `PurgeUnsupportedReferences` deletes its stale `symbol_references` rows on the next indexing run, and graph read paths additionally filter by supported languages to prevent stale edges from surviving between index runs. Shell is intentionally excluded because its command-style invocations (`foo arg1 arg2`) cannot be detected by the parenthesized-call regex. **Nested generic call sites**: C#/Java constructor calls like `new Dictionary>()` and C# generic method calls like `Helper.DoWork>()` are recovered by a depth-aware fallback scanner so the outer target still reaches the reference table even though the flat regex fast-path cannot balance `>>`. **JS/TS no-paren constructors**: JavaScript / TypeScript zero-argument constructor calls that legally omit `()` — for example `new Foo;`, `new Date;`, qualified targets like `new Demo.Provider;`, and one-level generic TypeScript forms like `new Box;` — are emitted as `instantiate` edges via a dedicated language-gated path, while next-line `.bar()` / `[0]` continuations are suppressed so a line-ended `new Foo` does not become a phantom standalone instantiation. **Constructor chain calls**: C# `: this(...)` / `: base(...)` initializers and Java `this(...)` / `super(...)` first-statement calls are detected separately from the generic call regex and rewritten so the reference target is the real constructor (enclosing class/record for `this`, the parsed base type from the class signature for `base` / `super`). Cross-line C# initializers are attributed to the owning constructor rather than the enclosing class. Base-type parsing strips generics, record primary-ctor args, `where` constraints, and `global::` / dotted namespace qualifiers; Java `super.method()` stays a normal method call. **Type-position dependency edges**: C#/Java base lists, declaration types, generic constraints, `throws`, `is`/`as`/`instanceof`, and real C# XML-doc `cref` sites are indexed as `type_reference` rows so `references` / `impact` can see compile-time rename dependencies without polluting the default dynamic call graph exposed by `callers` / `callees`. C# XML-doc `cref` extraction accepts declaration-attached XML-doc comments from both `///` lines and delimited `/** ... */` blocks, including declarations that begin later on the same physical line after the closing `*/` only when no unrelated same-line code or declaration intervenes, while ordinary `//` / `////` comments, non-documenting block comments, method-body XML-doc comments that merely precede a later declaration, brace-free field/property initializer continuations, brace-free expression lambdas, intervening top-level executable statements, same-line non-target code after `*/`, other nested executable continuations, and multiline raw/verbatim string content whose line happens to start with `/**` stay excluded. Non-doc code or string content after the closing `*/` on the same physical line is still outside the doc-comment slice. Even though the regex now runs against that narrower slice, the extractor preserves `symbol_references.column` relative to the original physical source line. On the C# read path, `using static` constant-pattern suppression is token-aware around `is` / `case`, reconstructs a small indexed multi-line window when the anchor lives on a previous line, and keeps trivia-bearing forms such as `value is/*comment*/Red`, `value is\n Red or Blue`, and `case\tRed:` filtered. Same-name type rescue also honors `file` visibility so file-local types only rescue references from the same physical file; inherited protected/public/internal nested types from real base classes rescue derived-class pattern heads only after the base reference is normalized through active type and namespace aliases, and alias-expanded constructed generic bases are canonicalized again before containing-type lookup so `AliasBase = Probe.Base` resolves the same way as `Probe.Base`; implemented interfaces do not contribute inherited nested-type rescue; and same-file `using Namespace;`, project-wide `global using Namespace;`, and active type aliases all participate in the rescue set. The extractor deliberately leaves ambiguous unqualified `using static` heads such as `value is Red` in the DB, because file-local parsing alone cannot know whether another file in the same namespace declares the real `Red` type; the workspace-aware read path is responsible for suppressing the pure constant-only cases. **SQL qualified-name alignment**: SQL definitions still persist their schema-qualified symbol name (`dbo.fn_X`), but graph/`deps`/unused/hotspot readers now resolve each SQL reference row through its stored source-line context, recorded call column, and enclosing container before they compare it to definitions, so qualified `references` / `callers` / `impact` queries stay schema-scoped even when one line contains multiple qualified calls or the lookup is non-exact. Those readers fall back to the bare leaf only when the source site itself is genuinely unqualified, which keeps `deps`, `unused`, and `hotspots` aligned with qualified SQL calls without regressing bare-call support or double-counting `EXEC dbo.fn_Target; EXEC sales.fn_Target;`. Once a row already has a recorded call column, those downstream readers no longer whole-line-upgrade that row to a later qualified token, so trailing comments, string literals, or a second qualified call cannot steal the earlier unqualified edge. Exact SQL graph/dependency readers also preserve the resolved segment count, so a quoted single identifier containing a dot such as `"sales.fn_Target"` stays distinct from the real qualified name `sales.fn_Target` across exact `references` / `callers` / `impact` and aggregate `deps` / `unused` / `hotspots`. SQL CTE body source rows use the raw `cte_body_reference` kind, so `references --kind cte_body_reference` can distinguish anchor/recursive-member internals from outer-query table references. Qualified SQL `callees` queries also keep leaf fallback disabled unless the caller query itself is unqualified, so `callees sales.Caller` no longer widens to `dbo.Caller`. SQL extractors also accept optional whitespace around qualified-name dots, so definitions/calls such as `[sales] . [fn_Target]` and `[dbo] . [fn_Target]` keep their full qualified identity instead of truncating at the first segment. The same SQL no-parens extractor now preserves ANSI / PostgreSQL double-quoted call targets such as `CALL "sales"."proc_name"` and `EXEC "dbo"."fn_Target"` instead of stripping them as string literals, while true single-quoted SQL string literals remain masked. Definition-oriented readers also canonicalize quoted qualified SQL names (`[dbo].[fn_X]` → `dbo.fn_X`) before matching, and they only fall back to the leaf identifier for unqualified queries so exact qualified lookups do not widen to sibling schemas that merely share the same leaf name. Exact SQL definition matching also preserves segment count, so a quoted single identifier that contains a dot (`"sales.fn_Target"`) does not collide with a real qualified name (`sales.fn_Target`). SQL exact graph leaf fallback also stays on the Unicode folded exact path, and both quoted qualified and unqualified Unicode exact definition lookups now use the folded normalized path, so queries such as `dbo.Äpfel` / `dbo.äpfel` and bare `Äpfel` / `äpfel` keep matching leaf call/reference rows such as `äpfel` plus stored definitions such as `[dbo].[Äpfel]` or `dbo.Äpfel` instead of silently degrading to ASCII-only `NOCASE`. Exact multi-name SQL `symbols --count` lookups also bind the folded leaf parameters on that same `_foldReady` path, so Unicode leaf query sets no longer fail with missing-parameter database errors. diff --git a/changelog.d/unreleased/1964.fixed.md b/changelog.d/unreleased/1964.fixed.md new file mode 100644 index 0000000000..0df8334122 --- /dev/null +++ b/changelog.d/unreleased/1964.fixed.md @@ -0,0 +1,18 @@ +--- +category: fixed +issues: + - 1964 +affected: + - src/CodeIndex/Database/DbReader.cs + - src/CodeIndex/Database/DbReader.FilesStatus.cs + - src/CodeIndex/Database/DegradationReasonCodes.cs + - tests/CodeIndex.Tests/DbReaderTests.cs +--- + +## English + +- **Fold readiness can now verify row completeness on demand (#1964)** — when `CDIDX_VERIFY_FOLD_READY_ROWS=1` is set, `status` detects DBs whose fold-ready bit is set even though folded-name rows are incomplete, reports `fold_ready_bit_set_but_rows_incomplete`, and keeps `fold_ready=false`. + +## 日本語 + +- **fold readiness が必要時に行レベルの完全性を検証するようになりました (#1964)** — `CDIDX_VERIFY_FOLD_READY_ROWS=1` のとき、fold-ready bit が立っていても folded-name 行が未完了な DB を `status` が検出し、`fold_ready_bit_set_but_rows_incomplete` を報告して `fold_ready=false` のままにします。 diff --git a/changelog.d/unreleased/1973.fixed.md b/changelog.d/unreleased/1973.fixed.md new file mode 100644 index 0000000000..5cd596ddb5 --- /dev/null +++ b/changelog.d/unreleased/1973.fixed.md @@ -0,0 +1,17 @@ +--- +category: fixed +issues: + - 1973 +affected: + - src/CodeIndex/Mcp/McpToolHandlers.cs + - tests/CodeIndex.Tests/McpServerTests.cs + - DEVELOPER_GUIDE.md +--- + +## English + +- **MCP status now mirrors fold degradation remediation (#1973)** — when `fold_ready=false`, MCP `status` includes `degraded_reason`, `recommended_action`, and `alternative_action` alongside `fold_ready_reason`, matching CLI status guidance for Unicode exact-name readiness. + +## 日本語 + +- **MCP status が fold degradation の修復情報を CLI と揃えて返すようになりました (#1973)** — `fold_ready=false` のとき、MCP `status` は `fold_ready_reason` に加えて `degraded_reason`、`recommended_action`、`alternative_action` を返し、Unicode exact-name readiness の案内を CLI status と揃えます。 diff --git a/changelog.d/unreleased/1993.internal.md b/changelog.d/unreleased/1993.internal.md new file mode 100644 index 0000000000..50afb80aed --- /dev/null +++ b/changelog.d/unreleased/1993.internal.md @@ -0,0 +1,15 @@ +--- +category: internal +issues: + - 1993 +affected: + - tests/CodeIndex.Tests/DbReaderTests.cs +--- + +## English + +- **Expanded folded-column backfill regression coverage (#1993)** — tests now cover partial NULL states across `symbols.name_folded`, `symbol_references.symbol_name_folded`, and `symbol_references.container_name_folded`, including deterministic repeated checks. + +## 日本語 + +- **folded-column backfill の回帰テスト範囲を拡張しました (#1993)** — `symbols.name_folded`、`symbol_references.symbol_name_folded`、`symbol_references.container_name_folded` の部分的な NULL 状態と、繰り返し実行時の決定性をテストするようになりました。 diff --git a/src/CodeIndex/Database/DbReader.FilesStatus.cs b/src/CodeIndex/Database/DbReader.FilesStatus.cs index 7ccff68337..8fd296b954 100644 --- a/src/CodeIndex/Database/DbReader.FilesStatus.cs +++ b/src/CodeIndex/Database/DbReader.FilesStatus.cs @@ -419,6 +419,7 @@ public StatusResult GetStatus() var sqlGraphContractSignal = GetSqlGraphContractSignal(lang: null); var hotspotFamilySignal = GetHotspotFamilySignal(lang: null); var foldReadyReason = ResolveFoldReadyReason(); + var foldReady = _foldReady && foldReadyReason == null; // Language breakdown / 言語別内訳 // Scope the reader in an inner block so it releases its statement handle before @@ -472,7 +473,7 @@ public StatusResult GetStatus() CSharpMetadataTargetDegradedReason = csharpMetadataTargetDegradedReason, SqlGraphContractReady = sqlGraphContractSignal.Ready, SqlGraphContractDegradedReason = sqlGraphContractSignal.DegradedReason, - FoldReady = _foldReady, + FoldReady = foldReady, FoldReadyReason = foldReadyReason, IndexWriterVersion = _indexWriterVersion, IndexNewerThanReader = _indexNewerThanReader, diff --git a/src/CodeIndex/Database/DbReader.cs b/src/CodeIndex/Database/DbReader.cs index 9c277c6f3f..aae743c85f 100644 --- a/src/CodeIndex/Database/DbReader.cs +++ b/src/CodeIndex/Database/DbReader.cs @@ -30,6 +30,8 @@ public readonly record struct SqlGraphContractSignal( /// public partial class DbReader { + public const string VerifyFoldReadyRowsEnvironmentVariable = "CDIDX_VERIFY_FOLD_READY_ROWS"; + private static readonly Regex ImpactSignatureIdentifierRegex = new(@"[\p{L}_][\p{L}\p{Nd}_]*", RegexOptions.Compiled); private static readonly Regex CSharpUsingStaticImportRegex = new(@"^\s*(?:global\s+)?using\s+static\s+(?[^;]+)", RegexOptions.Compiled); private static readonly Regex CSharpUsingAliasImportRegex = new(@"^\s*(?:global\s+)?using\s+(?!static\b)(?[^\s=;]+)\s*=\s*(?[^;]+)", RegexOptions.Compiled); @@ -647,7 +649,12 @@ private static int ParseFoldVersion(SqliteConnection conn) private string? ResolveFoldReadyReason() { if (_foldReady) + { + if (ShouldVerifyFoldReadyRows() && HasIncompleteFoldRows()) + return DegradationReasonCodes.FoldReadyBitSetButRowsIncomplete; + return null; + } var storedVersion = ParseFoldVersion(_conn); var storedFingerprint = ParseFoldFingerprint(_conn); @@ -660,6 +667,32 @@ private static int ParseFoldVersion(SqliteConnection conn) return DegradationReasonCodes.FoldRowsNotRestamped; } + private static bool ShouldVerifyFoldReadyRows() + { + var value = Environment.GetEnvironmentVariable(VerifyFoldReadyRowsEnvironmentVariable); + return value is "1" || string.Equals(value, "true", StringComparison.OrdinalIgnoreCase); + } + + private bool HasIncompleteFoldRows() + { + using var cmd = _conn.CreateCommand(); + var predicates = new List + { + "EXISTS(SELECT 1 FROM symbols WHERE name IS NOT NULL AND name_folded IS NULL)", + }; + if (_hasReferencesTable + && _referenceColumns.Contains("symbol_name_folded") + && _referenceColumns.Contains("container_name_folded")) + { + predicates.Add("EXISTS(SELECT 1 FROM symbol_references WHERE symbol_name IS NOT NULL AND symbol_name_folded IS NULL)"); + predicates.Add("EXISTS(SELECT 1 FROM symbol_references WHERE container_name IS NOT NULL AND container_name_folded IS NULL)"); + } + + cmd.CommandText = $"SELECT {string.Join(" OR ", predicates)}"; + var raw = cmd.ExecuteScalar(); + return raw is long l ? l != 0 : raw is int i && i != 0; + } + private HashSet LoadIndexedHotspotFamilyLanguages() { var langs = new HashSet(StringComparer.Ordinal); diff --git a/src/CodeIndex/Database/DegradationReasonCodes.cs b/src/CodeIndex/Database/DegradationReasonCodes.cs index c7fe1d8bbb..203eb25eef 100644 --- a/src/CodeIndex/Database/DegradationReasonCodes.cs +++ b/src/CodeIndex/Database/DegradationReasonCodes.cs @@ -12,6 +12,7 @@ public static class DegradationReasonCodes public const string StaleFoldKeyVersion = "stale_fold_key_version"; public const string StaleFoldKeyFingerprint = "stale_fold_key_fingerprint"; public const string FoldRowsNotRestamped = "fold_rows_not_restamped"; + public const string FoldReadyBitSetButRowsIncomplete = "fold_ready_bit_set_but_rows_incomplete"; public const string FoldReadyNotReady = "fold_ready=false"; public const string SqlGraphContractNotReady = "sql_graph_contract_ready=false"; public const string HotspotFamilyNotReady = "hotspot_family_ready=false"; @@ -32,6 +33,7 @@ public static class DegradationReasonCodes StaleFoldKeyVersion, StaleFoldKeyFingerprint, FoldRowsNotRestamped, + FoldReadyBitSetButRowsIncomplete, FoldReadyNotReady, SqlGraphContractNotReady, HotspotFamilyNotReady, @@ -78,6 +80,7 @@ public static string NormalizeFoldReason(string? foldReadyReason) StaleFoldKeyVersion => StaleFoldKeyVersion, StaleFoldKeyFingerprint => StaleFoldKeyFingerprint, FoldRowsNotRestamped => FoldRowsNotRestamped, + FoldReadyBitSetButRowsIncomplete => FoldReadyBitSetButRowsIncomplete, _ => FoldRowsNotRestamped }; @@ -104,6 +107,11 @@ private static DegradationReasonMetadata CreateMetadata(string code) "--exact falls back to ASCII COLLATE NOCASE because some folded-name rows were not restamped under the current runtime.", "Run `cdidx backfill-fold` to restamp folded-name columns in place.", "Run `cdidx index --rebuild` for a full rebuild."), + FoldReadyBitSetButRowsIncomplete => new( + code, + "--exact falls back to ASCII COLLATE NOCASE because the fold-ready bit is set but row-level folded-name verification found incomplete rows.", + "Run `cdidx backfill-fold` to restamp folded-name columns in place.", + "Run `cdidx index --rebuild` for a full rebuild."), FoldReadyNotReady => new( code, "Unicode exact-name fold readiness is degraded.", diff --git a/src/CodeIndex/Mcp/McpToolHandlers.cs b/src/CodeIndex/Mcp/McpToolHandlers.cs index b85a640754..c1409fca03 100644 --- a/src/CodeIndex/Mcp/McpToolHandlers.cs +++ b/src/CodeIndex/Mcp/McpToolHandlers.cs @@ -1250,6 +1250,12 @@ private JsonNode ExecuteStatus(JsonNode? id) .ToList(); } status.Version = _version; + if (!status.FoldReady) + { + status.DegradedReason = DegradationReasonCodes.BuildFoldNotReadyExplanation(status.FoldReadyReason); + status.RecommendedAction = BuildFoldBackfillCommand(_dbPath, _dbPathExplicit); + status.AlternativeAction = BuildFoldRebuildRepairCommand(status.ProjectRoot, _dbPath, _dbPathExplicit); + } var structured = JsonSerializer.SerializeToNode(status, _jsonOptions)!.AsObject(); structured["hotspotFamilyReady"] = status.HotspotFamilyReady; if (status.HotspotFamilyDegradedReason != null) @@ -1261,6 +1267,45 @@ private JsonNode ExecuteStatus(JsonNode? id) }); } + private static string BuildFoldBackfillCommand(string dbPath, bool dbPathExplicit) + { + if (!dbPathExplicit) + return "cdidx backfill-fold"; + + return $"cdidx backfill-fold --db {QuoteCommandArgument(ResolveWritableDbPathOrPlaceholder(dbPath))}"; + } + + private static string BuildFoldRebuildRepairCommand(string? projectRoot, string dbPath, bool dbPathExplicit) + { + if (!dbPathExplicit) + return "cdidx index . --rebuild"; + + var resolvedDbPath = ResolveWritableDbPathOrPlaceholder(dbPath); + var targetProject = string.IsNullOrWhiteSpace(projectRoot) + ? "" + : QuoteCommandArgument(projectRoot); + return $"cdidx index {targetProject} --db {QuoteCommandArgument(resolvedDbPath)} --rebuild"; + } + + private static string ResolveWritableDbPathOrPlaceholder(string dbPath) + => DbPathResolver.TryResolveWritableMutationDbPath(dbPath, out var writableDbPath) + ? writableDbPath + : ""; + + private static string QuoteCommandArgument(string value) + { + if (value.Length >= 2 && value[0] == '<' && value[^1] == '>') + return value; + + var fullPath = DbPathResolver.NormalizeDbPath(value); + if (!fullPath.StartsWith("file:", StringComparison.OrdinalIgnoreCase)) + fullPath = Path.GetFullPath(fullPath); + + return fullPath.IndexOfAny([' ', '\t', '"']) >= 0 + ? $"\"{fullPath.Replace("\"", "\\\"", StringComparison.Ordinal)}\"" + : fullPath; + } + private JsonNode ExecuteOutline(JsonNode? id, JsonNode? args) { if (!TryReadRequiredStringParameter(args, "path", out var path, out var requiredError)) diff --git a/tests/CodeIndex.Tests/DbReaderTests.cs b/tests/CodeIndex.Tests/DbReaderTests.cs index d4a0a52ce6..affa03f339 100644 --- a/tests/CodeIndex.Tests/DbReaderTests.cs +++ b/tests/CodeIndex.Tests/DbReaderTests.cs @@ -2439,6 +2439,152 @@ public void AllFoldedColumnsBackfilled_DetectsLegacyRowsWithNullFoldedValues() } } + [Fact] + public void GetStatus_WithFoldRowVerification_DegradesWhenReadyBitRowsAreIncomplete() + { + using var env = EnvironmentVariableScope.Capture(DbReader.VerifyFoldReadyRowsEnvironmentVariable); + env.Set(DbReader.VerifyFoldReadyRowsEnvironmentVariable, "1"); + var dbPath = Path.Combine(Path.GetTempPath(), $"codeindex_fold_status_verify_{Guid.NewGuid():N}.db"); + try + { + using var db = new DbContext(dbPath); + db.InitializeSchema(); + var writer = new DbWriter(db.Connection); + var fileId = writer.UpsertFile(new FileRecord + { + Path = "src/a.py", Lang = "python", Size = 1, Lines = 1, + Modified = new DateTime(2025, 6, 1, 0, 0, 0, DateTimeKind.Utc), + }); + writer.InsertSymbols([ + new SymbolRecord { FileId = fileId, Kind = "function", Name = "authenticate", Line = 1, StartLine = 1, EndLine = 1 }, + ]); + Assert.True(writer.MarkFoldReady()); + + using (var cmd = db.Connection.CreateCommand()) + { + cmd.CommandText = "UPDATE symbols SET name_folded = NULL WHERE name = 'authenticate'"; + cmd.ExecuteNonQuery(); + } + + var status = new DbReader(db.Connection).GetStatus(); + + Assert.False(status.FoldReady); + Assert.Equal(DegradationReasonCodes.FoldReadyBitSetButRowsIncomplete, status.FoldReadyReason); + } + finally + { + Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools(); + if (File.Exists(dbPath)) File.Delete(dbPath); + } + } + + [Fact] + public void GetStatus_WithFoldRowVerification_IgnoresMissingReferenceTable() + { + using var env = EnvironmentVariableScope.Capture(DbReader.VerifyFoldReadyRowsEnvironmentVariable); + env.Set(DbReader.VerifyFoldReadyRowsEnvironmentVariable, "1"); + var dbPath = Path.Combine(Path.GetTempPath(), $"codeindex_fold_status_legacy_refs_{Guid.NewGuid():N}.db"); + try + { + using var db = new DbContext(dbPath); + db.InitializeSchema(); + var writer = new DbWriter(db.Connection); + var fileId = writer.UpsertFile(new FileRecord + { + Path = "src/a.py", Lang = "python", Size = 1, Lines = 1, + Modified = new DateTime(2025, 6, 1, 0, 0, 0, DateTimeKind.Utc), + }); + writer.InsertSymbols([ + new SymbolRecord { FileId = fileId, Kind = "function", Name = "authenticate", Line = 1, StartLine = 1, EndLine = 1 }, + ]); + Assert.True(writer.MarkFoldReady()); + + using (var cmd = db.Connection.CreateCommand()) + { + cmd.CommandText = "DROP TABLE symbol_references"; + cmd.ExecuteNonQuery(); + } + db.RefreshSchemaCache(); + + var status = new DbReader(db.Connection).GetStatus(); + + Assert.True(status.FoldReady); + Assert.Null(status.FoldReadyReason); + } + finally + { + Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools(); + if (File.Exists(dbPath)) File.Delete(dbPath); + } + } + + [Theory] + [InlineData(true, false, false)] + [InlineData(false, true, false)] + [InlineData(false, false, true)] + [InlineData(true, true, false)] + [InlineData(true, false, true)] + [InlineData(false, true, true)] + [InlineData(true, true, true)] + public void AllFoldedColumnsBackfilled_DetectsEveryPartialFoldColumnState( + bool nullSymbolName, + bool nullReferenceSymbolName, + bool nullReferenceContainerName) + { + var dbPath = Path.Combine(Path.GetTempPath(), $"codeindex_fold_partial_{Guid.NewGuid():N}.db"); + try + { + using var db = new DbContext(dbPath); + db.InitializeSchema(); + var writer = new DbWriter(db.Connection); + var fileId = writer.UpsertFile(new FileRecord + { + Path = "src/a.py", Lang = "python", Size = 1, Lines = 1, + Modified = new DateTime(2025, 6, 1, 0, 0, 0, DateTimeKind.Utc), + }); + writer.InsertSymbols([ + new SymbolRecord { FileId = fileId, Kind = "function", Name = "authenticate", Line = 1, StartLine = 1, EndLine = 1 }, + ]); + writer.InsertReferences([ + new ReferenceRecord + { + FileId = fileId, + SymbolName = "authenticate", + ReferenceKind = "call", + Line = 1, + Column = 1, + ContainerName = "login", + }, + ]); + + Assert.True(writer.AllFoldedColumnsBackfilled()); + + using (var cmd = db.Connection.CreateCommand()) + { + cmd.CommandText = $""" + UPDATE symbols + SET name_folded = CASE WHEN @nullSymbolName THEN NULL ELSE name_folded END; + UPDATE symbol_references + SET + symbol_name_folded = CASE WHEN @nullReferenceSymbolName THEN NULL ELSE symbol_name_folded END, + container_name_folded = CASE WHEN @nullReferenceContainerName THEN NULL ELSE container_name_folded END; + """; + cmd.Parameters.AddWithValue("@nullSymbolName", nullSymbolName); + cmd.Parameters.AddWithValue("@nullReferenceSymbolName", nullReferenceSymbolName); + cmd.Parameters.AddWithValue("@nullReferenceContainerName", nullReferenceContainerName); + cmd.ExecuteNonQuery(); + } + + for (var i = 0; i < 5; i++) + Assert.False(writer.AllFoldedColumnsBackfilled()); + } + finally + { + Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools(); + if (File.Exists(dbPath)) File.Delete(dbPath); + } + } + [Fact] public void GetExactGraphSupportedDefinitionLanguage_DegradesOnLegacyDbMissingContainerKind() { diff --git a/tests/CodeIndex.Tests/McpServerTests.cs b/tests/CodeIndex.Tests/McpServerTests.cs index 1c1b40963b..29f8658ba3 100644 --- a/tests/CodeIndex.Tests/McpServerTests.cs +++ b/tests/CodeIndex.Tests/McpServerTests.cs @@ -5163,6 +5163,11 @@ public void ToolsCall_Status_ReturnsCounts() Assert.NotNull(response["result"]!["structuredContent"]!["projectRoot"]); Assert.NotNull(response["result"]!["structuredContent"]!["hotspot_family_ready"]); Assert.NotNull(response["result"]!["structuredContent"]!["hotspotFamilyReady"]); + Assert.False(response["result"]!["structuredContent"]!["foldReady"]!.GetValue()); + Assert.Equal(DegradationReasonCodes.MissingFoldBackfill, response["result"]!["structuredContent"]!["fold_ready_reason"]!.GetValue()); + Assert.Contains("--exact falls back", response["result"]!["structuredContent"]!["degraded_reason"]!.GetValue()); + Assert.Equal("cdidx backfill-fold", response["result"]!["structuredContent"]!["recommended_action"]!.GetValue()); + Assert.Equal("cdidx index . --rebuild", response["result"]!["structuredContent"]!["alternative_action"]!.GetValue()); } [Fact]