Bound persisted JSON string-list metadata parsing

AGENT_GUIDE.md

@@ -137,7 +137,7 @@ CI watching must be bounded. Do not loop indefinitely.
 - `issues_table_available` reports physical `file_issues` table presence only. `file_issues_data_current` reports whether the table is also stamped current for the active index generation.
 - `index_writer_version` records the `cdidx` version that last wrote to the DB (stamped into `codeindex_meta` as `cdidx_writer_version` on every full scan, update, and MCP index). `index_newer_than_reader` flips to `true` whenever any persisted numeric contract stamp in `codeindex_meta` (or unknown `PRAGMA user_version` readiness bits) exceeds the current binary's compiled maximum, so an older CLI re-opening a DB written by a newer CLI degrades loudly with an audit trail instead of silently dropping back to text-search fallbacks. `index_newer_than_reader_reason` enumerates the specific newer-than-reader stamps.
 - `status` also surfaces indexed-HEAD freshness via `indexed_head_sha`, `indexed_head_branch`, `indexed_head_timestamp`, and `commits_ahead_of_indexed_head`. They are stamped by `cdidx index` on every successful run (full scan AND partial update, distinct from `indexed_head_commit` which is full-scan only) on a best-effort basis (never blocks an otherwise-successful index) and omitted on non-git workspaces, detached HEAD (branch only), or legacy DBs created before this contract.
-- `status` also surfaces unknown-extension scan coverage via `unknown_extension_file_count`, stamped by successful full-repository index runs (`cdidx index <projectPath>` and MCP `index_project`) as the number of non-indexed files with non-empty extensions that do not map to a known language. Current scans also stamp `unknown_extension_files` as a bounded path sample, `unknown_extension_files_truncated` when the count exceeds the sample, and `unknown_extension_file_path_limit` as the cap. These fields are omitted on legacy DBs or before a current full scan has stamped them.
+- `status` also surfaces unknown-extension scan coverage via `unknown_extension_file_count`, stamped by successful full-repository index runs (`cdidx index <projectPath>` and MCP `index_project`) as the number of non-indexed files with non-empty extensions that do not map to a known language. Current scans also stamp `unknown_extension_files` as a path sample bounded by `unknown_extension_file_path_limit` items and the string-list decoded-character budget, `unknown_extension_files_truncated` when more paths existed than were emitted for either bound, and `unknown_extension_file_path_limit` as the item cap rather than a guarantee that that many paths are returned. These fields are omitted on legacy DBs or before a current full scan has stamped them.
 - `status` also surfaces extractor plugin and pattern-config runtime diagnostics via `extractors`, including loaded counts, skipped file counts, and a bounded diagnostics list for incompatible or malformed plugin/pattern files. Diagnostic paths and messages are sanitized before output.
 - `status` also surfaces metadata-only post-extraction hook candidates and callback budgets through `hooks[]` / `hooks[].callback_budget_ms` without loading hook assemblies. Index runs still enforce `CDIDX_HOOK_CALLBACK_BUDGET_MS` (default: 5000 ms) on scratch copies, discard timed-out mutations, and disable timed-out hooks for the remainder of the current run.
 - `status` also surfaces `.cdidx` data-directory permissions via `data_dir_mode` on POSIX filesystems. New `.cdidx` data directories are forced to `0700`; the field is omitted on Windows, URI DBs, or when the directory mode cannot be inspected.

DEVELOPER_GUIDE.md

@@ -3150,7 +3150,7 @@ USER_GUIDEの[終了コード](USER_GUIDE.md#終了コード)セクションを
 - **AI向けの軽量検索スニペット** — `search --json` と MCP の `search` は、チャンク全文ではなく snippet range、match line、highlight、context count、`truncated_line_count`、`truncation_context` を持つ一致中心スニペットを返す。`truncation_context.char_counts` と `truncation_context.total_chars` はクランプされた各スニペット行の省略文字数を公開し、truncated な highlight も `truncated_char_counts` を持つ。`--snippet-lines` でペイロード量と文脈量のバランスを取れ、`--max-line-width`（CLI）/ `maxLineWidth`（MCP）は `find` / `references` / `excerpt` / `inspect` と同じ共有 `LineWidthFormatter.ClampLine` 契約で各スニペット行を最初のマッチトークン周辺にクランプするため、minified / transpiled / 生成された 1 行ファイル内の 1 ヒットで数百 KB を返さなくなる。クランプされた行はスニペットに `...(+N)...` マーカーが入り、`highlights[].truncated` と `highlights[].original_line_length` で AI クライアントがクランプを検出できる。
 - **初動向けの repo map** — `map` は、インデックス済みデータから言語、モジュール、主要ファイル、ホットスポット、推定エントリポイントを集約し、AIクライアントが精密検索前に見るべき場所を決めやすくする。シンボル抽出が `Main` 系シンボルを出さない場合でも、既知のトップレベル実行ファイルへフォールバックして入口候補を補う。
 - **信用判断のための鮮度メタデータ** — `status` はワークスペース全体の鮮度と git 状態を返す。`map` は `indexed_at` / `latest_modified` を絞り込み結果の鮮度として維持しつつ、`workspace_indexed_at` / `workspace_latest_modified` でワークスペース全体の鮮度も返す。`inspect` も同じワークスペース鮮度と git フィールドを返すため、シンボル中心の AI フローで `status` を別途呼ばずに済む。さらに `status` は `sql_graph_contract_ready` / `sql_graph_contract_degraded_reason`、`hotspot_family_ready` / `hotspot_family_degraded_reason` に加えて、forward-compatibility 監査 (`index_writer_version`、`index_newer_than_reader`、`index_newer_than_reader_reason`、詳細は「リーダー側の forward-compatibility 監査」を参照)、および fold-only remediation 用の `fold_ready_reason`、`degraded_reason`、`recommended_action`、`alternative_action` も返すため、AI クライアントは SQL graph/dependency/impact、duplicate-name hotspot family、Unicode `--exact` のどれが authoritative か、また DB が現在の binary より新しい `cdidx` で書かれていないかを最初に判断できる。現行の全体 scan 後は `unknown_extension_file_count` も返すため、未知拡張子で index 対象外になった件数を `status` から確認できる。これらの fold-only remediation field は、明示的な read-only `file:///...?...` DB URI から導出された場合でも、失敗する read-only URI をそのままコマンドへ埋め込まず、writable な filesystem path に正規化して返す。さらに `impact` / MCP `impact_analysis` に加えて、`inspect` / MCP `analyze_symbol`、`references` / `callers` / `callees`、`deps` / `unused` / `hotspots` 系も、SQL ベースの graph/dependency read が実際に結果へ関与したときだけ `sql_graph_contract_ready` / `sql_graph_contract_degraded_reason` を反映するため、stale な SQL 行が authoritative なヒットや 0 件応答に見えてしまうのを防ぎつつ、mixed-language index 内の純粋な非SQL結果を誤って degraded 扱いしない。`files` はファイルごとの checksum・modified・indexed timestamp を返す。古いDBに対する file 列の移行は可能なら自動で行い、その場移行できない場合でも読み取り経路がクラッシュしないようにする。CLI と MCP の 0 件 JSON レスポンスは `indexed_file_count`、`indexed_at`、`freshness_available` を含む。`freshness_available=true` で `indexed_at:null` なら空インデックス、`freshness_available=false` なら legacy/read-only DB で鮮度 timestamp を取得できず、理由は `freshness_degraded_reason` に入る。**HEAD 起点の stale 検知**: `cdidx index` の full scan が成功するたびに、現時点の `git HEAD` を `codeindex_meta` に stamp し、後続実行で workspace HEAD と比較できるようにする。`--rebuild` 指定なしに両者が異なる場合、CLI は `cdidx index <projectPath> --rebuild` を勧める `head_changed` 警告を表示し、`index --json` に `head_changed` / `prior_indexed_head_commit` / `current_head_commit` / `head_change_notice` を出力する。`status --check` も同じ比較を `workspace_check.head_changed` として公開し、差分時には `indexed_head_commit` / `workspace_head_commit` も併記するため、鮮度 gate ですでに `status --check` を通している AI クライアントは `git switch <branch>` 後の既定の incremental scan を別クエリなしで拒否できる。`--commits` / `--files` の部分更新は意図的に記録 HEAD を維持し、次の full scan が worktree を再インデックスするまで stale 通知が継続する。非 Git workspace と HEAD を記録していない legacy DB は比較自体をスキップし、false-positive な警告を出さない。
-`unknown_extension_files` は `unknown_extension_file_path_limit` 件までの未知拡張子 path sample で、`unknown_extension_files_truncated` は `unknown_extension_file_count` が sample 上限を超えたことを示します。
+`unknown_extension_files` は `unknown_extension_file_path_limit` 件と decoded-character budget の両方で上限付けされた未知拡張子 path sample で、`unknown_extension_files_truncated` は件数上限または decoded-character budget により未出力の path が残ったことを示します。`unknown_extension_file_path_limit` は item 上限であり、常にその件数まで返す保証ではありません。
 
 `extractors` は extractor plugin と pattern config の runtime health で、読み込み済み plugin assembly / pattern 件数、symbol/reference extractor 件数、skip されたファイル数、上限付き diagnostics list を返します。
 

README.md

@@ -218,7 +218,7 @@ The documented `status --json` trust contract covers these fields:
 
 When any readiness field is degraded, `degraded_root_cause` identifies the primary stable code and `readiness_degradations[]` lists every degraded field with `root_cause`, human `degraded_reason`, `recommended_action`, and `alternative_action`. `issues_table_available` reports physical table presence; use `file_issues_data_current` to decide whether `file_issues` rows are current for the index generation.
 
-After a current full-repository scan, `unknown_extension_file_count` reports how many skipped files had unmapped non-empty extensions, while `unknown_extension_files` lists up to `unknown_extension_file_path_limit` paths and `unknown_extension_files_truncated` marks when more paths exist.
+After a current full-repository scan, `unknown_extension_file_count` reports how many skipped files had unmapped non-empty extensions, while `unknown_extension_files` lists a path sample capped by `unknown_extension_file_path_limit` items and the decoded-character budget. `unknown_extension_files_truncated` marks when more paths existed than were emitted for either cap; `unknown_extension_file_path_limit` is the item cap, not a guarantee that the sample always reaches that count.
 
 `extractors` reports runtime extractor plugin and pattern-config diagnostics, including loaded counts, skipped file counts, and a bounded diagnostics list for load failures. Diagnostic paths and messages are sanitized before they are surfaced.
 

changelog.d/unreleased/3045.security.md

@@ -0,0 +1,20 @@
+---
+category: security
+issues:
+  - 3045
+affected:
+  - src/CodeIndex/Database/JsonStringListCodec.cs
+  - src/CodeIndex/Database/DbWriter.cs
+  - tests/CodeIndex.Tests/JsonStringListCodecTests.cs
+  - README.md
+  - DEVELOPER_GUIDE.md
+  - AGENT_GUIDE.md
+---
+
+## English
+
+- **Persisted JSON string-list metadata now enforces bounded parsing (#3045)** — string-list metadata now rejects over-deep JSON, oversized arrays, oversized raw payloads, and excessive decoded string content instead of allocating unbounded parsed values from corrupted database rows.
+
+## 日本語
+
+- **永続化 JSON string-list metadata の parse に上限を適用しました (#3045)** — string-list metadata は、壊れた database row から無制限に parse 結果を確保せず、過深な JSON、過大な配列、過大な raw payload、過大な decoded string 内容を拒否するようになりました。

src/CodeIndex/Database/DbWriter.cs

@@ -2135,9 +2135,9 @@ public void WriteUnknownExtensionFileMetadata(IReadOnlyList<string> paths)
     {
         ArgumentNullException.ThrowIfNull(paths);
 
-        var sample = paths
-            .Take(DbContext.UnknownExtensionFilePathSampleLimit)
-            .ToArray();
+        var sample = JsonStringListCodec.TakeSerializableSample(
+            paths,
+            DbContext.UnknownExtensionFilePathSampleLimit);
         SetMeta(
             DbContext.UnknownExtensionFileCountMetaKey,
             paths.Count.ToString(System.Globalization.CultureInfo.InvariantCulture));
@@ -2146,7 +2146,7 @@ public void WriteUnknownExtensionFileMetadata(IReadOnlyList<string> paths)
             JsonStringListCodec.Serialize(sample));
         SetMeta(
             DbContext.UnknownExtensionFilesTruncatedMetaKey,
-            (paths.Count > sample.Length).ToString(System.Globalization.CultureInfo.InvariantCulture));
+            (paths.Count > sample.Count).ToString(System.Globalization.CultureInfo.InvariantCulture));
         SetMeta(
             DbContext.UnknownExtensionFilePathLimitMetaKey,
             DbContext.UnknownExtensionFilePathSampleLimit.ToString(System.Globalization.CultureInfo.InvariantCulture));

src/CodeIndex/Database/JsonStringListCodec.cs

@@ -5,6 +5,18 @@ namespace CodeIndex.Database;
 
 internal static class JsonStringListCodec
 {
+    internal const int MaxRawJsonCharacters = 512 * 1024;
+    internal const int MaxJsonDepth = 4;
+    internal const int MaxArrayItems = 1024;
+    internal const int MaxDecodedStringCharacters = 64 * 1024;
+
+    private const int MaxJsonStringBytesPerDecodedChar = 6;
+
+    private static readonly JsonReaderOptions ReaderOptions = new()
+    {
+        MaxDepth = MaxJsonDepth,
+    };
+
     public static string Serialize(IReadOnlyList<string> values)
     {
         using var buffer = new MemoryStream();
@@ -19,33 +31,110 @@ public static string Serialize(IReadOnlyList<string> values)
         return Encoding.UTF8.GetString(buffer.ToArray());
     }
 
+    internal static List<string> TakeSerializableSample(IReadOnlyList<string> values, int maxItems)
+    {
+        ArgumentNullException.ThrowIfNull(values);
+        if (maxItems <= 0)
+            return [];
+
+        var itemLimit = Math.Min(maxItems, MaxArrayItems);
+        var sample = new List<string>(Math.Min(values.Count, itemLimit));
+        var decodedCharacters = 0;
+        foreach (var value in values)
+        {
+            if (sample.Count >= itemLimit)
+                break;
+
+            if (value.Length > MaxDecodedStringCharacters - decodedCharacters)
+                break;
+
+            decodedCharacters += value.Length;
+            sample.Add(value);
+        }
+
+        return sample;
+    }
+
     public static List<string>? Deserialize(string? raw)
+        => Deserialize(raw, out _);
+
+    internal static List<string>? Deserialize(string? raw, out string? diagnostic)
     {
+        diagnostic = null;
         if (string.IsNullOrWhiteSpace(raw))
             return null;
 
+        if (raw.Length > MaxRawJsonCharacters)
+            return Reject("json_string_list_raw_too_large", out diagnostic);
+
         try
         {
-            using var document = JsonDocument.Parse(raw);
-            if (document.RootElement.ValueKind != JsonValueKind.Array)
-                return null;
+            var utf8 = Encoding.UTF8.GetBytes(raw);
+            var reader = new Utf8JsonReader(utf8, ReaderOptions);
+            if (!reader.Read() || reader.TokenType != JsonTokenType.StartArray)
+                return Reject("json_string_list_not_array", out diagnostic);
 
             var values = new List<string>();
-            foreach (var element in document.RootElement.EnumerateArray())
+            var itemCount = 0;
+            var decodedCharacters = 0;
+            while (reader.Read())
             {
-                if (element.ValueKind != JsonValueKind.String)
+                if (reader.TokenType == JsonTokenType.EndArray)
+                {
+                    if (reader.Read())
+                        return Reject("json_string_list_trailing_json", out diagnostic);
+                    return values;
+                }
+
+                itemCount++;
+                if (itemCount > MaxArrayItems)
+                    return Reject("json_string_list_too_many_items", out diagnostic);
+
+                if (reader.TokenType != JsonTokenType.String)
+                {
+                    if (reader.TokenType is JsonTokenType.StartArray or JsonTokenType.StartObject)
+                        reader.Skip();
                     continue;
+                }
+
+                var remainingCharacters = MaxDecodedStringCharacters - decodedCharacters;
+                if (ExceedsStringByteBudget(ref reader, remainingCharacters))
+                    return Reject("json_string_list_too_many_characters", out diagnostic);
 
-                var value = element.GetString();
+                var value = reader.GetString();
+                if (value == null)
+                    continue;
+                if (value.Length > remainingCharacters)
+                    return Reject("json_string_list_too_many_characters", out diagnostic);
+
+                decodedCharacters += value.Length;
                 if (!string.IsNullOrWhiteSpace(value))
                     values.Add(value);
             }
 
-            return values;
+            return Reject("json_string_list_incomplete", out diagnostic);
         }
         catch (JsonException)
         {
+            diagnostic = "json_string_list_malformed";
             return null;
         }
     }
+
+    private static bool ExceedsStringByteBudget(ref Utf8JsonReader reader, int remainingCharacters)
+    {
+        if (remainingCharacters < 0)
+            return true;
+
+        var byteBudget = (long)remainingCharacters * MaxJsonStringBytesPerDecodedChar;
+        return reader.HasValueSequence
+            ? reader.ValueSequence.Length > byteBudget
+            : reader.ValueSpan.Length > byteBudget;
+    }
+
+    private static List<string>? Reject(string reason, out string? diagnostic)
+    {
+        diagnostic = reason;
+        return null;
+    }
 }