From 1915acc58e5635f3fdfee176c5f38df6ee5eebfc Mon Sep 17 00:00:00 2001 From: Widthdom Date: Sat, 20 Jun 2026 23:13:37 +0900 Subject: [PATCH 1/4] fix(index): configure watch pending path limit --- USER_GUIDE.md | 14 ++++-- changelog.d/unreleased/3726.changed.md | 1 + src/CodeIndex/Cli/CdidxConfigFile.cs | 16 +++++- src/CodeIndex/Cli/CliFlagSchema.cs | 1 + src/CodeIndex/Cli/ConsoleUi.cs | 3 +- src/CodeIndex/Cli/IndexCommandRunner.Parse.cs | 49 ++++++++++++++++++- .../Cli/IndexCommandRunner.Validation.cs | 2 +- src/CodeIndex/Cli/IndexCommandRunner.cs | 1 + src/CodeIndex/Cli/IndexWatchRunner.cs | 16 ++++-- src/CodeIndex/Cli/JsonOutputContracts.cs | 1 + tests/CodeIndex.Tests/CdidxConfigFileTests.cs | 25 +++++++++- .../IndexCommandRunnerTests.cs | 33 +++++++++++++ .../CodeIndex.Tests/IndexWatchRunnerTests.cs | 11 +++++ 13 files changed, 159 insertions(+), 14 deletions(-) create mode 100644 changelog.d/unreleased/3726.changed.md diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 361bd8540a..e303631a6a 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -551,6 +551,7 @@ Use the smallest change that reduces the expensive part of your run. | `--max-file-bytes ` / `CDIDX_MAX_FILE_BYTES` | `4MiB` | Legitimate large source files are skipped | Raising it can bloat the DB and slow snippet extraction | | `--parallelism ` / `CDIDX_INDEX_PARALLELISM` | CPU count, capped at `16` | Full-scan extraction is CPU-bound | Higher values can increase memory and IO pressure | | `--watch --debounce ` | `500` ms | Keep an active worktree fresh during editing | Long-running process; incompatible with commit/file scoped refresh flags | +| `--watch-pending-path-limit ` / `CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT` | `4096` | Watcher sees more distinct changed paths than the default queue keeps | Higher values use more memory before the safe full-rescan fallback | | `--snippet-lines` / `--max-line-width` | `8` / `512` | Query payloads are too large for AI context | Smaller snippets may hide nearby context | | `--path`, `--exclude-path`, `--exclude-tests` | off | Queries or maps are noisy | Over-filtering can hide real matches | @@ -1454,6 +1455,7 @@ same source location. | `--parallelism ` | `index` | Set full-scan extraction worker count. Defaults to CPU count capped at 16, or `CDIDX_INDEX_PARALLELISM` when set. SQLite writes stay single-consumer. | | `--watch` | `index` | After the initial scan completes, stay running and reindex incrementally as files change (FileSystemWatcher / inotify / FSEvents). Rejects `--commits`, `--changed-between`, `--files`, and `--dry-run` because the loop already drives continuous incremental updates. | | `--debounce ` | `index` (watch only) | Coalesce bursts of file events into a single update after `` of quiet (non-negative integer; default: 500). Invalid values emit a warning and are ignored. | +| `--watch-pending-path-limit ` | `index` (watch only) | Set the number of distinct changed paths the watch loop will queue before it reports an overflow and falls back to a full rescan. Defaults to `4096`, honors `CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT` and `indexing.watchPendingPathLimit`, and rejects values above `262144`. The `watching` and `overflow` JSON events include `watch_pending_path_limit`. | | `--since ` | `search`, `definition`, `symbols`, `files` | Filter to files modified since this ISO 8601 timestamp. Offsetless values (e.g. `2024-01-01T00:00:00`) are treated as UTC so the same flag resolves to the same instant in every timezone; append `Z` or an explicit offset (`+09:00`) to be explicit. | | `--no-dedup` | `search` | Disable overlapping-chunk deduplication and return every raw chunk hit; useful for debugging chunk boundaries or measuring raw match density | | `--reverse` | `deps` | Reverse lookup: show files that depend ON the matched path | @@ -1737,7 +1739,8 @@ Supported schema (top-level keys are snake_case; nested indexing kind keys keep "indexing": { "includeKinds": ["class"], // → CDIDX_INDEX_INCLUDE_SYMBOL_KINDS "excludeKinds": ["test_method"], // → CDIDX_INDEX_EXCLUDE_SYMBOL_KINDS - "generatedCodePatterns": ["src/generated/**", "*.client.ts"] // → CDIDX_INDEX_GENERATED_CODE_PATTERNS + "generatedCodePatterns": ["src/generated/**", "*.client.ts"], // → CDIDX_INDEX_GENERATED_CODE_PATTERNS + "watchPendingPathLimit": 8192 // → CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT }, "mcp": { "tools": { @@ -1753,7 +1756,7 @@ Supported schema (top-level keys are snake_case; nested indexing kind keys keep } ``` -JSON5-style line comments (`//`) and trailing commas are accepted so the file stays human-editable. The optional `$schema` key is ignored at runtime; it is honored only so editors that recognize JSON Schema references can offer completion. Setting `disable_persistent_log` to `false` is a no-op (absence already means "logging enabled") — only `true` exports `CDIDX_DISABLE_PERSISTENT_LOG=1`. Config-sourced `metrics_path` and `global_tool_log_dir` values are resolved from the config workspace root and must stay inside that workspace; use the CLI flag or a real environment variable when you intentionally need an outside destination. `stale_after` uses the same compact duration format as `status --check --stale-after`: `30m`, `2h`, or `7d`, up to `30d`. `suggestion_dedup_threshold` sets the MCP suggestion fuzzy-deduplication cutoff as a number from `0` to `1`; the built-in default is `0.85`, and `cdidx mcp --suggestion-dedup-threshold <0..1>` overrides it for one MCP session. `suggestion_max_age_days` and `suggestion_max_count` bound the live `.cdidx/suggestions-*.json` store; pruned records are appended to `.cdidx/suggestions-*.archive.jsonl`, whose active file is capped at 8 MiB and rotates up to three retained generations (`.1` through `.3`). Defaults are 365 days and 5000 records, and config-file values may not exceed 3650 days or 100000 records. Matching environment variables above those caps fall back to the defaults. `mcp.rate_limit.bucket_idle_seconds` sets the same idle bucket TTL as `CDIDX_MCP_RATE_LIMIT_BUCKET_IDLE_SECONDS`; invalid runtime values fall back to the default with a warning. String-array settings such as `indexing.includeKinds`, `indexing.excludeKinds`, `indexing.generatedCodePatterns`, `mcp.tools.allow`, and `mcp.tools.deny` are capped at 128 entries and 256 characters per item before they are joined into environment variables. `indexing.generatedCodePatterns` treats matching relative paths or basenames as extraction-suppressed generated-code sources. Matching files remain indexed for normal text search and chunk retrieval because the query-filtered `generated` flag is not set by this option; symbol/reference extraction is skipped and `file_issues` records `generated_code_extraction_skipped`. Patterns with a slash match slash-normalized relative paths, patterns without a slash match basenames, and `*`, `?`, and `**` are supported. `indexing.includeKinds` and `indexing.excludeKinds` set the default symbol-kind filter for `cdidx index`; CLI flags `--include-symbol-kind [,]` and `--exclude-symbol-kind [,]` override those env-backed defaults for a single run. +JSON5-style line comments (`//`) and trailing commas are accepted so the file stays human-editable. The optional `$schema` key is ignored at runtime; it is honored only so editors that recognize JSON Schema references can offer completion. Setting `disable_persistent_log` to `false` is a no-op (absence already means "logging enabled") — only `true` exports `CDIDX_DISABLE_PERSISTENT_LOG=1`. Config-sourced `metrics_path` and `global_tool_log_dir` values are resolved from the config workspace root and must stay inside that workspace; use the CLI flag or a real environment variable when you intentionally need an outside destination. `stale_after` uses the same compact duration format as `status --check --stale-after`: `30m`, `2h`, or `7d`, up to `30d`. `suggestion_dedup_threshold` sets the MCP suggestion fuzzy-deduplication cutoff as a number from `0` to `1`; the built-in default is `0.85`, and `cdidx mcp --suggestion-dedup-threshold <0..1>` overrides it for one MCP session. `suggestion_max_age_days` and `suggestion_max_count` bound the live `.cdidx/suggestions-*.json` store; pruned records are appended to `.cdidx/suggestions-*.archive.jsonl`, whose active file is capped at 8 MiB and rotates up to three retained generations (`.1` through `.3`). Defaults are 365 days and 5000 records, and config-file values may not exceed 3650 days or 100000 records. Matching environment variables above those caps fall back to the defaults. `mcp.rate_limit.bucket_idle_seconds` sets the same idle bucket TTL as `CDIDX_MCP_RATE_LIMIT_BUCKET_IDLE_SECONDS`; invalid runtime values fall back to the default with a warning. String-array settings such as `indexing.includeKinds`, `indexing.excludeKinds`, `indexing.generatedCodePatterns`, `mcp.tools.allow`, and `mcp.tools.deny` are capped at 128 entries and 256 characters per item before they are joined into environment variables. `indexing.watchPendingPathLimit` sets the watch pending-path queue limit and may not exceed 262144; the matching CLI flag and real environment variable override it for one run. `indexing.generatedCodePatterns` treats matching relative paths or basenames as extraction-suppressed generated-code sources. Matching files remain indexed for normal text search and chunk retrieval because the query-filtered `generated` flag is not set by this option; symbol/reference extraction is skipped and `file_issues` records `generated_code_extraction_skipped`. Patterns with a slash match slash-normalized relative paths, patterns without a slash match basenames, and `*`, `?`, and `**` are supported. `indexing.includeKinds` and `indexing.excludeKinds` set the default symbol-kind filter for `cdidx index`; CLI flags `--include-symbol-kind [,]` and `--exclude-symbol-kind [,]` override those env-backed defaults for a single run. ## How it works @@ -3082,6 +3085,7 @@ cdidx index . --duration-format seconds | `--max-file-bytes ` / `CDIDX_MAX_FILE_BYTES` | `4MiB` | 正当な大きい source file が skip される | DB が大きくなり snippet extraction も遅くなりうる | | `--parallelism ` / `CDIDX_INDEX_PARALLELISM` | CPU 数、最大 `16` | フルスキャンの抽出が CPU-bound | 大きくするとメモリと IO の圧力が増えうる | | `--watch --debounce ` | `500` ms | 編集中の worktree を live に保つ | long-running process。commit/file scoped refresh flags とは併用不可 | +| `--watch-pending-path-limit ` / `CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT` | `4096` | watcher が既定 queue を超える数の changed path を検知する | 大きくすると安全な full-rescan fallback 前に使うメモリが増える | | `--snippet-lines` / `--max-line-width` | `8` / `512` | AI context に対して query payload が大きすぎる | 小さくしすぎると周辺文脈が見えない | | `--path`, `--exclude-path`, `--exclude-tests` | off | query / map が noisy | 絞り込みすぎると実 match を隠す | @@ -4002,6 +4006,7 @@ raw match density を正確に測る、といった理由で全 raw chunk hit | `--parallelism ` | `index` | フルスキャンの抽出 worker 数を指定する。既定は CPU 数を最大 16 に丸めた値、または `CDIDX_INDEX_PARALLELISM` 設定値。SQLite 書き込みは単一 consumer のまま。 | | `--watch` | `index` | 初回スキャン完了後もプロセスを残し、ファイル変更を検知して差分更新を繰り返す（FileSystemWatcher / inotify / FSEvents）。連続的な差分更新を内蔵しているため `--commits` / `--changed-between` / `--files` / `--dry-run` との併用は拒否する。 | | `--debounce ` | `index`（`--watch` 専用） | 一連のイベントを `` の静止後に 1 つの更新へ集約する（0 以上の整数。既定: 500）。不正な値は警告を出して無視する。 | +| `--watch-pending-path-limit ` | `index`（`--watch` 専用） | watch loop が overflow を報告して full rescan へ fallback する前に保持する distinct changed path 数を設定する。既定は `4096` で、`CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT` と `indexing.watchPendingPathLimit` も使える。`262144` を超える値は拒否される。`watching` と `overflow` の JSON event には `watch_pending_path_limit` が入る。 | | `--since ` | `search`, `definition`, `symbols`, `files` | 指定タイムスタンプ以降に変更されたファイルのみ（ISO 8601）。オフセットなしの値（例: `2024-01-01T00:00:00`）は UTC として解釈されるため、どのタイムゾーンから呼び出しても同じ UTC 時点になります。明示したい場合は末尾に `Z` または `+09:00` 等のオフセットを付与してください。 | | `--no-dedup` | `search` | overlap chunk の重複排除を無効化し、全 raw chunk hit を返す。chunk 境界の debug や raw match density 計測向け | | `--reverse` | `deps` | 逆引き: 指定パスに依存しているファイルを表示 | @@ -4285,7 +4290,8 @@ MCP のレスポンスサイズ上限は、環境変数 override で guard が "indexing": { "includeKinds": ["class"], // → CDIDX_INDEX_INCLUDE_SYMBOL_KINDS "excludeKinds": ["test_method"], // → CDIDX_INDEX_EXCLUDE_SYMBOL_KINDS - "generatedCodePatterns": ["src/generated/**", "*.client.ts"] // → CDIDX_INDEX_GENERATED_CODE_PATTERNS + "generatedCodePatterns": ["src/generated/**", "*.client.ts"], // → CDIDX_INDEX_GENERATED_CODE_PATTERNS + "watchPendingPathLimit": 8192 // → CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT }, "mcp": { "tools": { @@ -4301,7 +4307,7 @@ MCP のレスポンスサイズ上限は、環境変数 override で guard が } ``` -人手で編集しやすいよう JSON5 形式の行コメント（`//`）と末尾カンマを許容します。任意の `$schema` キーはランタイムでは無視され、JSON Schema 参照をサポートするエディタが補完を提供するためだけに認識されます。`disable_persistent_log` を `false` に設定しても何も起きません（不在のままで "ログ有効" が既定）— `true` の場合のみ `CDIDX_DISABLE_PERSISTENT_LOG=1` を export します。config 由来の `metrics_path` と `global_tool_log_dir` は設定ファイルの workspace root から解決され、その workspace 内に収まる必要があります。意図的に外部の出力先を使う場合は CLI フラグまたは実際の環境変数を使ってください。`stale_after` は `status --check --stale-after` と同じ compact duration 形式（`30m` / `2h` / `7d`、最大 `30d`）です。`suggestion_dedup_threshold` は MCP suggestion の fuzzy deduplication しきい値を `0` から `1` の数値で設定します。組み込み既定値は `0.85` で、`cdidx mcp --suggestion-dedup-threshold <0..1>` は 1 回の MCP session だけこの値を上書きします。`suggestion_max_age_days` と `suggestion_max_count` は live の `.cdidx/suggestions-*.json` store の上限を設定し、prune された record は `.cdidx/suggestions-*.archive.jsonl` に追記されます。この active archive は 8 MiB で上限管理され、最大 3 世代（`.1` から `.3`）までローテーションされます。既定値は 365 日と 5000 件で、config-file 値は 3650 日または 100000 件を超えられません。同じ環境変数がこの上限を超えた場合は既定値へ戻ります。`mcp.rate_limit.bucket_idle_seconds` は `CDIDX_MCP_RATE_LIMIT_BUCKET_IDLE_SECONDS` と同じ idle bucket TTL を設定します。不正な runtime 値は警告付きで既定値へ戻ります。`indexing.includeKinds`、`indexing.excludeKinds`、`indexing.generatedCodePatterns`、`mcp.tools.allow`、`mcp.tools.deny` のような string array 設定は、環境変数へ join される前に 128 件、1 要素 256 文字までに制限されます。`indexing.generatedCodePatterns` は一致した相対パスまたはベース名を generated-code extraction の抑制対象として扱います。この設定では query filter 用の `generated` flag を立てないため、一致したファイルも通常の全文検索と chunk 取得用には引き続き index されます。symbol/reference 抽出はスキップされ、`file_issues` に `generated_code_extraction_skipped` が記録されます。スラッシュを含む pattern は slash-normalized relative path、スラッシュを含まない pattern は basename に一致し、`*`、`?`、`**` を利用できます。`indexing.includeKinds` と `indexing.excludeKinds` は `cdidx index` の symbol-kind filter 既定値を設定し、CLI フラグ `--include-symbol-kind [,]` / `--exclude-symbol-kind [,]` はその env 経由の既定値を 1 回の実行だけ上書きします。 +人手で編集しやすいよう JSON5 形式の行コメント（`//`）と末尾カンマを許容します。任意の `$schema` キーはランタイムでは無視され、JSON Schema 参照をサポートするエディタが補完を提供するためだけに認識されます。`disable_persistent_log` を `false` に設定しても何も起きません（不在のままで "ログ有効" が既定）— `true` の場合のみ `CDIDX_DISABLE_PERSISTENT_LOG=1` を export します。config 由来の `metrics_path` と `global_tool_log_dir` は設定ファイルの workspace root から解決され、その workspace 内に収まる必要があります。意図的に外部の出力先を使う場合は CLI フラグまたは実際の環境変数を使ってください。`stale_after` は `status --check --stale-after` と同じ compact duration 形式（`30m` / `2h` / `7d`、最大 `30d`）です。`suggestion_dedup_threshold` は MCP suggestion の fuzzy deduplication しきい値を `0` から `1` の数値で設定します。組み込み既定値は `0.85` で、`cdidx mcp --suggestion-dedup-threshold <0..1>` は 1 回の MCP session だけこの値を上書きします。`suggestion_max_age_days` と `suggestion_max_count` は live の `.cdidx/suggestions-*.json` store の上限を設定し、prune された record は `.cdidx/suggestions-*.archive.jsonl` に追記されます。この active archive は 8 MiB で上限管理され、最大 3 世代（`.1` から `.3`）までローテーションされます。既定値は 365 日と 5000 件で、config-file 値は 3650 日または 100000 件を超えられません。同じ環境変数がこの上限を超えた場合は既定値へ戻ります。`mcp.rate_limit.bucket_idle_seconds` は `CDIDX_MCP_RATE_LIMIT_BUCKET_IDLE_SECONDS` と同じ idle bucket TTL を設定します。不正な runtime 値は警告付きで既定値へ戻ります。`indexing.includeKinds`、`indexing.excludeKinds`、`indexing.generatedCodePatterns`、`mcp.tools.allow`、`mcp.tools.deny` のような string array 設定は、環境変数へ join される前に 128 件、1 要素 256 文字までに制限されます。`indexing.watchPendingPathLimit` は watch pending-path queue 上限を設定し、262144 を超えることはできません。対応する CLI flag と実際の環境変数は 1 回の実行でこの値を上書きします。`indexing.generatedCodePatterns` は一致した相対パスまたはベース名を generated-code extraction の抑制対象として扱います。この設定では query filter 用の `generated` flag を立てないため、一致したファイルも通常の全文検索と chunk 取得用には引き続き index されます。symbol/reference 抽出はスキップされ、`file_issues` に `generated_code_extraction_skipped` が記録されます。スラッシュを含む pattern は slash-normalized relative path、スラッシュを含まない pattern は basename に一致し、`*`、`?`、`**` を利用できます。`indexing.includeKinds` と `indexing.excludeKinds` は `cdidx index` の symbol-kind filter 既定値を設定し、CLI フラグ `--include-symbol-kind [,]` / `--exclude-symbol-kind [,]` はその env 経由の既定値を 1 回の実行だけ上書きします。 ## 動作の仕組み diff --git a/changelog.d/unreleased/3726.changed.md b/changelog.d/unreleased/3726.changed.md new file mode 100644 index 0000000000..e776bef87e --- /dev/null +++ b/changelog.d/unreleased/3726.changed.md @@ -0,0 +1 @@ +Allow `cdidx index --watch` pending-path queue limits to be configured with `--watch-pending-path-limit`, `CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT`, or `indexing.watchPendingPathLimit`, and report the effective limit in watch diagnostics. diff --git a/src/CodeIndex/Cli/CdidxConfigFile.cs b/src/CodeIndex/Cli/CdidxConfigFile.cs index 0b4b95346a..93091e5114 100644 --- a/src/CodeIndex/Cli/CdidxConfigFile.cs +++ b/src/CodeIndex/Cli/CdidxConfigFile.cs @@ -56,7 +56,7 @@ internal static class CdidxConfigFile "mcp", }; - private static readonly IReadOnlyList KnownIndexingKeys = new[] { "includeKinds", "excludeKinds", "generatedCodePatterns" }; + private static readonly IReadOnlyList KnownIndexingKeys = new[] { "includeKinds", "excludeKinds", "generatedCodePatterns", "watchPendingPathLimit" }; private static readonly IReadOnlyList KnownSearchKeys = new[] { "limit", "snippet_lines", "max_line_width" }; private static readonly IReadOnlyList KnownOutputKeys = new[] { "format", "locale" }; private static readonly IReadOnlyList KnownGraphKeys = new[] { "max_hops" }; @@ -291,6 +291,20 @@ private static void AddIndexingEnvironmentSettings(JsonElement root, string path else if (value!.Length > 0) pending.Add((IndexCommandRunner.GeneratedCodePatternsEnvironmentVariable, string.Join(",", value))); } + + if (indexing.TryGetProperty("watchPendingPathLimit", out var watchPendingPathLimit)) + { + if (!TryReadPositiveIntegerAsString(watchPendingPathLimit, "indexing.watchPendingPathLimit", path, out var value, out var err)) + errors.Add(err!); + else + { + var parsedLimit = int.Parse(value!, CultureInfo.InvariantCulture); + if (parsedLimit > IndexWatchRunner.MaxWatchPendingPathLimit) + errors.Add($"[cdidx] {path}: `indexing.watchPendingPathLimit` must be <= {IndexWatchRunner.MaxWatchPendingPathLimit}."); + else + pending.Add((IndexCommandRunner.WatchPendingPathLimitEnvironmentVariable, value!)); + } + } } private static void AddSearchEnvironmentSettings(JsonElement root, string path, List<(string EnvName, string Value)> pending, List errors) diff --git a/src/CodeIndex/Cli/CliFlagSchema.cs b/src/CodeIndex/Cli/CliFlagSchema.cs index 10b5b098a1..86d6aef4c8 100644 --- a/src/CodeIndex/Cli/CliFlagSchema.cs +++ b/src/CodeIndex/Cli/CliFlagSchema.cs @@ -384,6 +384,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--files", ValuePlaceholder = "", Description = "Update only the specified files", Commands = Set("index") }, new() { Name = "--watch", Description = "Continuous reindex on file changes (rejects --commits / --changed-between / --files / --dry-run)", Commands = Set("index") }, new() { Name = "--debounce", ValuePlaceholder = "", Description = "Watch only: coalesce file events into one update after of quiet (default 500)", Commands = Set("index") }, + new() { Name = "--watch-pending-path-limit", ValuePlaceholder = "", Description = "Watch only: changed-path queue limit before full-rescan fallback", Commands = Set("index") }, new() { Name = "--output", ShortName = "-o", ValuePlaceholder = "", Description = "Output bundle path", Commands = Set("report") }, new() { Name = "--no-log", Description = "Exclude global tool log from bundle", Commands = Set("report") }, new() { Name = "--include-args", Description = "Include args in bundle log", Commands = Set("report") }, diff --git a/src/CodeIndex/Cli/ConsoleUi.cs b/src/CodeIndex/Cli/ConsoleUi.cs index 23ad1a161f..9a22c11e99 100644 --- a/src/CodeIndex/Cli/ConsoleUi.cs +++ b/src/CodeIndex/Cli/ConsoleUi.cs @@ -80,7 +80,7 @@ public static class ConsoleUi private static readonly (string Command, string Usage)[] CommandUsageLines = [ - ("index", "cdidx index [--db ] [--rebuild] [--optimize] [--symbols-only] [--verbose] [--dry-run] [--force] [--quiet] [--json] [--memory-trace] [--duration-format ] [--notify ] [--max-file-bytes ] [--max-symbols-per-file ] [--max-references-per-file ] [--follow-symlinks ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]] [--watch [--debounce ]]"), + ("index", "cdidx index [--db ] [--rebuild] [--optimize] [--symbols-only] [--verbose] [--dry-run] [--force] [--quiet] [--json] [--memory-trace] [--duration-format ] [--notify ] [--max-file-bytes ] [--max-symbols-per-file ] [--max-references-per-file ] [--follow-symlinks ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]] [--watch [--debounce ] [--watch-pending-path-limit ]]"), ("hooks", "cdidx hooks [--project ] [--force] [--json]"), ("backfill-fold", "cdidx backfill-fold [--db ] [--dry-run] [--no-checkpoint] [--json]"), ("optimize", "cdidx optimize [--db ] [--json]"), @@ -981,6 +981,7 @@ private static void PrintFlagReference(Action WriteHelpLine) Console.WriteLine(" --files [path ...] Update only the specified files; old rename/delete paths are not purged unless also listed"); WriteHelpLine(" --watch After the initial scan, stay running and reindex on file changes (FileSystemWatcher / inotify / FSEvents); rejects --commits / --changed-between / --files / --dry-run"); Console.WriteLine($" --debounce Watch only: coalesce bursts of file events into one update after of quiet (default: {IndexWatchRunner.DefaultDebounceMs}, max {IndexWatchRunner.MaxDebounceMs})"); + WriteHelpLine($" --watch-pending-path-limit Watch only: pending changed-path queue limit before falling back to a full rescan (default: {IndexWatchRunner.DefaultWatchPendingPathLimit}, max: {IndexWatchRunner.MaxWatchPendingPathLimit}; also honors {IndexCommandRunner.WatchPendingPathLimitEnvironmentVariable})"); Console.WriteLine(" --optimize index only: optimize the existing FTS5 table for this project's DB without scanning files"); WriteHelpLine(" --color Color output: `auto` (default), `always`, or `never`; flag wins over `CLICOLOR_FORCE` / `NO_COLOR` / `CLICOLOR` env vars, which win over TTY auto-detect"); WriteHelpLine(" --palette ANSI palette: `basic` (8-color, default fallback), `256`, or `truecolor`; flag wins over `CDIDX_COLOR_PALETTE` env var, which wins over `COLORTERM` / `TERM` auto-detect"); diff --git a/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs b/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs index c2b9c56435..29085cc78d 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs @@ -13,7 +13,7 @@ public static partial class IndexCommandRunner private static readonly string[] AcceptedIndexFlags = [ "--db", "--data-dir", "--rebuild", "--verbose", "--json", "--quiet", "--dry-run", "--force", - "--yes", "--watch", "--debounce", "--duration-format", "--max-file-bytes", "--max-symbols-per-file", + "--yes", "--watch", "--debounce", "--watch-pending-path-limit", "--duration-format", "--max-file-bytes", "--max-symbols-per-file", "--max-references-per-file", "--notify", "--parallelism", "--memory-trace", "--follow-symlinks", "--symbols-only", "--commits", "--changed-between", "--files", "--solution", "--project", @@ -23,6 +23,7 @@ public static partial class IndexCommandRunner internal const string CompletionNotificationEnvironmentVariable = "CDIDX_NOTIFY"; internal const string IndexParallelismEnvironmentVariable = "CDIDX_INDEX_PARALLELISM"; + internal const string WatchPendingPathLimitEnvironmentVariable = "CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT"; internal const int MaxIndexParallelism = 16; internal const int MaxSymbolKindFilterCsvLength = 2048; internal const int MaxSymbolKindFilterCsvEntries = 128; @@ -45,6 +46,7 @@ public static IndexCommandOptions ParseArgs(string[] args) bool symbolsOnly = false; bool memoryTrace = false; int? watchDebounceMs = null; + var watchPendingPathLimit = ReadWatchPendingPathLimitFromEnvironment(); var durationFormat = DurationOutputFormat.Auto; var notifyMode = ReadCompletionNotificationModeFromEnvironment(); long? maxFileSizeBytes = ReadMaxFileSizeBytesFromEnvironment(); @@ -149,6 +151,12 @@ public static IndexCommandOptions ParseArgs(string[] args) i++; } break; + case "--watch-pending-path-limit" when i + 1 < args.Length: + watchPendingPathLimit = ParseWatchPendingPathLimit(args[++i], watchPendingPathLimit, "--watch-pending-path-limit", ref parseError); + break; + case var option when option.StartsWith("--watch-pending-path-limit=", StringComparison.Ordinal): + watchPendingPathLimit = ParseWatchPendingPathLimit(option["--watch-pending-path-limit=".Length..], watchPendingPathLimit, "--watch-pending-path-limit", ref parseError); + break; case "--duration-format" when i + 1 < args.Length: durationFormat = ParseDurationFormat(args[++i], durationFormat); break; @@ -333,6 +341,7 @@ public static IndexCommandOptions ParseArgs(string[] args) SymbolsOnly = symbolsOnly, MemoryTrace = memoryTrace, WatchDebounceMs = watchDebounceMs, + WatchPendingPathLimit = watchPendingPathLimit, DurationFormat = durationFormat, NotifyMode = notifyMode, MaxFileSizeBytes = maxFileSizeBytes, @@ -510,6 +519,44 @@ private static int ParseIndexParallelism(string value, int fallback, string sour return fallback; } + private static int ReadWatchPendingPathLimitFromEnvironment() + { + var fallback = IndexWatchRunner.DefaultWatchPendingPathLimit; + var value = CdidxEnvironment.GetEnvironmentVariable(WatchPendingPathLimitEnvironmentVariable); + if (string.IsNullOrWhiteSpace(value)) + return fallback; + + if (int.TryParse(value, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var parsed) && parsed > 0) + { + if (parsed <= IndexWatchRunner.MaxWatchPendingPathLimit) + return parsed; + + var displayValue = ConsoleUi.FormatBoundedValue(value); + CommandErrorWriter.WriteStderr($"Warning: {WatchPendingPathLimitEnvironmentVariable} value '{displayValue}' exceeds the maximum {IndexWatchRunner.MaxWatchPendingPathLimit}; using {IndexWatchRunner.MaxWatchPendingPathLimit} / {WatchPendingPathLimitEnvironmentVariable} 値 '{displayValue}' は最大 {IndexWatchRunner.MaxWatchPendingPathLimit} を超えています。{IndexWatchRunner.MaxWatchPendingPathLimit} を使用します"); + return IndexWatchRunner.MaxWatchPendingPathLimit; + } + + var invalidDisplayValue = ConsoleUi.FormatBoundedValue(value); + CommandErrorWriter.WriteStderr($"Warning: invalid {WatchPendingPathLimitEnvironmentVariable} value '{invalidDisplayValue}' (ignored; use a positive integer) / 不正な {WatchPendingPathLimitEnvironmentVariable} 値 '{invalidDisplayValue}'（無視。正の整数を指定）"); + return fallback; + } + + private static int ParseWatchPendingPathLimit(string value, int fallback, string source, ref string? parseError) + { + if (int.TryParse(value, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var parsed) && parsed > 0) + { + if (parsed <= IndexWatchRunner.MaxWatchPendingPathLimit) + return parsed; + + parseError ??= $"{source} must be less than or equal to {IndexWatchRunner.MaxWatchPendingPathLimit}"; + return fallback; + } + + var displayValue = ConsoleUi.FormatBoundedValue(value); + CommandErrorWriter.WriteStderr($"Warning: invalid {source} value '{displayValue}' (ignored; use a positive integer) / 不正な {source} 値 '{displayValue}'（無視。正の整数を指定）"); + return fallback; + } + private static long? ReadMaxFileSizeBytesFromEnvironment() { var value = Environment.GetEnvironmentVariable(FileIndexer.MaxFileSizeEnvironmentVariable); diff --git a/src/CodeIndex/Cli/IndexCommandRunner.Validation.cs b/src/CodeIndex/Cli/IndexCommandRunner.Validation.cs index bc989842a2..084a173b0f 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.Validation.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.Validation.cs @@ -156,7 +156,7 @@ public static partial class IndexCommandRunner return null; const string watchConflictSynopsis = - "`cdidx index --watch [--debounce ]` " + "`cdidx index --watch [--debounce ] [--watch-pending-path-limit ]` " + "(omit --commits / --changed-between / --files / --dry-run; the initial scan plus continuous watch handles incremental refresh)"; return WriteCommandError( options.Json, diff --git a/src/CodeIndex/Cli/IndexCommandRunner.cs b/src/CodeIndex/Cli/IndexCommandRunner.cs index 558fd14b3a..7b140575e9 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.cs @@ -1610,6 +1610,7 @@ public sealed class IndexCommandOptions public bool OptimizeOnly { get; init; } public bool SymbolsOnly { get; init; } public int? WatchDebounceMs { get; init; } + public int WatchPendingPathLimit { get; init; } = IndexWatchRunner.DefaultWatchPendingPathLimit; public DurationOutputFormat DurationFormat { get; init; } = DurationOutputFormat.Auto; public CompletionNotificationMode NotifyMode { get; init; } = CompletionNotificationMode.Auto; public long? MaxFileSizeBytes { get; init; } diff --git a/src/CodeIndex/Cli/IndexWatchRunner.cs b/src/CodeIndex/Cli/IndexWatchRunner.cs index 2794df5675..6537e1fb9a 100644 --- a/src/CodeIndex/Cli/IndexWatchRunner.cs +++ b/src/CodeIndex/Cli/IndexWatchRunner.cs @@ -16,6 +16,8 @@ internal static class IndexWatchRunner { internal const int DefaultDebounceMs = 500; internal const int MaxDebounceMs = 60_000; + internal const int DefaultWatchPendingPathLimit = 4096; + internal const int MaxWatchPendingPathLimit = 262_144; internal const int MaxHumanSummarySubRunJsonChars = 64 * 1024; internal const int MaxHumanSummaryJsonDepth = 16; internal const int BatchPathSampleLimit = 20; @@ -57,8 +59,9 @@ internal static int RunCore( CancellationToken cancellationToken) { var debounce = TimeSpan.FromMilliseconds(baseOptions.WatchDebounceMs ?? DefaultDebounceMs); + var maxPendingPaths = baseOptions.WatchPendingPathLimit; var ignoreCase = GitHelper.ResolveIgnoreCase(projectRoot, cancellationToken); - var batcher = new FileChangeBatcher(debounce, ignoreCase: ignoreCase); + var batcher = new FileChangeBatcher(debounce, ignoreCase: ignoreCase, maxPendingPaths: maxPendingPaths); var ignoreRuleRoot = GitHelper.TryGetRepositoryRoot(projectRoot, cancellationToken) ?? Path.GetFullPath(projectRoot); var fileIndexer = new FileIndexer(projectRoot, ignoreCase, ignoreRuleRoot); @@ -123,7 +126,7 @@ void Enqueue(string fullPath) watcher.EnableRaisingEvents = true; - EmitWatchStarted(baseOptions, projectRoot, resolvedDbPath, debounce); + EmitWatchStarted(baseOptions, projectRoot, resolvedDbPath, debounce, maxPendingPaths); while (!cancellationToken.IsCancellationRequested) { @@ -422,7 +425,8 @@ private static void EmitWatchStarted( IndexCommandOptions baseOptions, string projectRoot, string resolvedDbPath, - TimeSpan debounce) + TimeSpan debounce, + int maxPendingPaths) { if (baseOptions.Json) { @@ -437,12 +441,13 @@ private static void EmitWatchStarted( ProjectRoot = projectRoot, Db = resolvedDbPath, DebounceMs = (int)debounce.TotalMilliseconds, + WatchPendingPathLimit = maxPendingPaths, }, CliJsonSerializerContextFactory.Create(jsonOpts).IndexWatchEventJsonResult)); } else { CommandErrorWriter.WriteStderr(); - CommandErrorWriter.WriteStderr($"[watch] Watching {projectRoot} for changes (debounce {(int)debounce.TotalMilliseconds} ms). Press Ctrl+C to stop."); + CommandErrorWriter.WriteStderr($"[watch] Watching {projectRoot} for changes (debounce {(int)debounce.TotalMilliseconds} ms, pending path limit {maxPendingPaths.ToString("N0", CultureInfo.InvariantCulture)}). Press Ctrl+C to stop."); } } @@ -460,6 +465,7 @@ private static void EmitWatchOverflow(IndexCommandOptions baseOptions, string? r Reason = reason, Phase = "incremental", OverflowReason = reason, + WatchPendingPathLimit = baseOptions.WatchPendingPathLimit, RecoveryCommand = BuildOverflowRecoveryCommand(baseOptions, resolvedDbPath), }, CliJsonSerializerContextFactory.Create(jsonOpts).IndexWatchEventJsonResult)); } @@ -517,7 +523,7 @@ private readonly record struct WatchSubRunSummary( /// internal sealed class FileChangeBatcher { - internal const int DefaultMaxPendingPaths = 4096; + internal const int DefaultMaxPendingPaths = IndexWatchRunner.DefaultWatchPendingPathLimit; private readonly object _gate = new(); private readonly HashSet _pending; diff --git a/src/CodeIndex/Cli/JsonOutputContracts.cs b/src/CodeIndex/Cli/JsonOutputContracts.cs index 8c22b64fd7..7d3931b9e1 100644 --- a/src/CodeIndex/Cli/JsonOutputContracts.cs +++ b/src/CodeIndex/Cli/JsonOutputContracts.cs @@ -295,6 +295,7 @@ internal sealed class IndexWatchEventJsonResult public string? ProjectRoot { get; init; } public string? Db { get; init; } public int? DebounceMs { get; init; } + public int? WatchPendingPathLimit { get; init; } public int? BatchSize { get; init; } public List? BatchPathSamples { get; init; } public int? BatchPathSampleLimit { get; init; } diff --git a/tests/CodeIndex.Tests/CdidxConfigFileTests.cs b/tests/CodeIndex.Tests/CdidxConfigFileTests.cs index d9dd5e0db0..8c9a78103a 100644 --- a/tests/CodeIndex.Tests/CdidxConfigFileTests.cs +++ b/tests/CodeIndex.Tests/CdidxConfigFileTests.cs @@ -69,7 +69,8 @@ public void LoadAndApply_MaterializesKnownKeysIntoEnvironment() "indexing": { "includeKinds": ["class"], "excludeKinds": ["test_method", "generated_parser"], - "generatedCodePatterns": ["src/generated/**", "*.client.ts"] + "generatedCodePatterns": ["src/generated/**", "*.client.ts"], + "watchPendingPathLimit": 8192 }, "mcp": { "tools": { "allow": ["search", "definition"], "deny": ["index"] }, @@ -94,6 +95,7 @@ public void LoadAndApply_MaterializesKnownKeysIntoEnvironment() Assert.Equal("class", result.Settings["CDIDX_INDEX_INCLUDE_SYMBOL_KINDS"]); Assert.Equal("test_method,generated_parser", result.Settings["CDIDX_INDEX_EXCLUDE_SYMBOL_KINDS"]); Assert.Equal("src/generated/**,*.client.ts", result.Settings[IndexCommandRunner.GeneratedCodePatternsEnvironmentVariable]); + Assert.Equal("8192", result.Settings[IndexCommandRunner.WatchPendingPathLimitEnvironmentVariable]); Assert.Equal("search,definition", result.Settings["CDIDX_MCP_TOOLS_ALLOW"]); Assert.Equal("index", result.Settings["CDIDX_MCP_TOOLS_DENY"]); Assert.Equal("5", result.Settings[RateLimiterOptions.RpsEnvVar]); @@ -253,6 +255,27 @@ public void LoadAndApply_ProjectConfigJsonRejectsInvalidSearchDefaults(string js finally { TestProjectHelper.DeleteDirectory(dir); } } + [Theory] + [InlineData("""{ "indexing": { "watchPendingPathLimit": 0 } }""", "positive integer")] + [InlineData("""{ "indexing": { "watchPendingPathLimit": 262145 } }""", "<= 262144")] + public void LoadAndApply_ProjectConfigJsonRejectsInvalidWatchPendingPathLimit(string json, string expectedError) + { + var dir = CreateTempDir(); + try + { + Directory.CreateDirectory(Path.Combine(dir, ".cdidx")); + File.WriteAllText(Path.Combine(dir, ".cdidx", "config.json"), json); + + var env = new TestEnvironment(); + var result = CdidxConfigFile.Load(dir, env.Read); + + Assert.True(result.Failed); + Assert.Contains(expectedError, result.Error); + Assert.Empty(result.Settings); + } + finally { TestProjectHelper.DeleteDirectory(dir); } + } + [Fact] public void LoadAndApply_RealEnvVarWinsOverConfigFile() { diff --git a/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs b/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs index 620df80021..48d5289785 100644 --- a/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs @@ -2880,6 +2880,7 @@ public void ParseArgs_WatchFlag_SetsWatch() var options = IndexCommandRunner.ParseArgs([".", "--watch"]); Assert.True(options.Watch); Assert.Null(options.WatchDebounceMs); + Assert.Equal(IndexWatchRunner.DefaultWatchPendingPathLimit, options.WatchPendingPathLimit); } [Fact] @@ -2930,6 +2931,38 @@ public void ParseArgs_DebounceFlag_InvalidValue_IsIgnored() } } + [Fact] + public void ParseArgs_WatchPendingPathLimitFlag_ParsesValue() + { + var options = IndexCommandRunner.ParseArgs([".", "--watch", "--watch-pending-path-limit", "8192"]); + + Assert.True(options.Watch); + Assert.Equal(8192, options.WatchPendingPathLimit); + } + + [Fact] + public void ParseArgs_WatchPendingPathLimitEnv_ParsesValue() + { + using var env = EnvironmentVariableScope.Capture(IndexCommandRunner.WatchPendingPathLimitEnvironmentVariable); + Environment.SetEnvironmentVariable(IndexCommandRunner.WatchPendingPathLimitEnvironmentVariable, "6144"); + + var options = IndexCommandRunner.ParseArgs([".", "--watch"]); + + Assert.Equal(6144, options.WatchPendingPathLimit); + } + + [Fact] + public void ParseArgs_WatchPendingPathLimitFlag_RejectsValueAboveMaximum() + { + var oversized = $"{IndexWatchRunner.MaxWatchPendingPathLimit + 1}"; + + var options = IndexCommandRunner.ParseArgs([".", "--watch", "--watch-pending-path-limit", oversized]); + + Assert.Equal(IndexWatchRunner.DefaultWatchPendingPathLimit, options.WatchPendingPathLimit); + Assert.Contains("--watch-pending-path-limit", options.ParseError); + Assert.Contains($"{IndexWatchRunner.MaxWatchPendingPathLimit}", options.ParseError); + } + [Fact] public void ParseArgs_MaxFileBytesFlag_ParsesSuffixValue() { diff --git a/tests/CodeIndex.Tests/IndexWatchRunnerTests.cs b/tests/CodeIndex.Tests/IndexWatchRunnerTests.cs index df0c116def..a171ac3f26 100644 --- a/tests/CodeIndex.Tests/IndexWatchRunnerTests.cs +++ b/tests/CodeIndex.Tests/IndexWatchRunnerTests.cs @@ -338,6 +338,7 @@ public void EmitWatchOverflow_Json_EmitsStructuredRecoveryCommand() MaxFileSizeBytes = 4096, MaxSymbolsPerFile = 42, Parallelism = parallelism, + WatchPendingPathLimit = 1234, SymlinkPolicy = FileIndexer.SymlinkPolicy.All, SymbolKindFilter = SymbolKindFilter.Create(["class", "function"], ["test.method"], parseError: null), }; @@ -366,6 +367,7 @@ public void EmitWatchOverflow_Json_EmitsStructuredRecoveryCommand() Assert.Equal("overflow", doc.RootElement.GetProperty("status").GetString()); Assert.Equal("incremental", doc.RootElement.GetProperty("phase").GetString()); Assert.Equal("buffer full", doc.RootElement.GetProperty("overflow_reason").GetString()); + Assert.Equal(1234, doc.RootElement.GetProperty("watch_pending_path_limit").GetInt32()); var recovery = doc.RootElement.GetProperty("recovery_command"); Assert.Equal("cdidx", recovery.GetProperty("command").GetString()); var args = recovery.GetProperty("args").EnumerateArray().Select(static item => item.GetString()).ToList(); @@ -461,6 +463,7 @@ public void RunCore_CancellationToken_StopsImmediately() Json = true, Watch = true, WatchDebounceMs = 50, + WatchPendingPathLimit = 123, }; using var cts = new CancellationTokenSource(); @@ -509,6 +512,12 @@ public void RunCore_CancellationToken_StopsImmediately() .ToList(); Assert.Contains("watching", statuses); Assert.Contains("stopped", statuses); + + var watchingLine = capturedOut + .Split('\n', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries) + .First(line => line.Contains("\"status\":\"watching\"", StringComparison.Ordinal)); + using var watchStarted = JsonDocument.Parse(watchingLine); + Assert.Equal(123, watchStarted.RootElement.GetProperty("watch_pending_path_limit").GetInt32()); } finally { @@ -537,6 +546,7 @@ public void RunCore_EmitsHumanFriendlyStartStop_WhenJsonDisabled() Json = false, Watch = true, WatchDebounceMs = 50, + WatchPendingPathLimit = 123, }; using var cts = new CancellationTokenSource(); @@ -577,6 +587,7 @@ public void RunCore_EmitsHumanFriendlyStartStop_WhenJsonDisabled() Assert.Equal(CommandExitCodes.Success, exitCode); Assert.Contains("[watch] Watching", capturedErr); Assert.Contains("debounce 50 ms", capturedErr); + Assert.Contains("pending path limit 123", capturedErr); Assert.Contains("[watch] Stopped.", capturedErr); } finally From d55d12c78f7252a5538f98a68888555cb8533b31 Mon Sep 17 00:00:00 2001 From: Widthdom Date: Sat, 20 Jun 2026 23:18:53 +0900 Subject: [PATCH 2/4] fix(index): bound dry-run candidate processing --- USER_GUIDE.md | 4 ++ changelog.d/unreleased/3745.changed.md | 1 + src/CodeIndex/Cli/CliFlagSchema.cs | 1 + src/CodeIndex/Cli/ConsoleUi.cs | 9 ++-- .../Cli/IndexCommandRunner.DryRun.cs | 54 +++++++++++++------ src/CodeIndex/Cli/IndexCommandRunner.Parse.cs | 26 ++++++++- src/CodeIndex/Cli/IndexCommandRunner.cs | 1 + src/CodeIndex/Cli/JsonOutputContracts.cs | 4 ++ .../IndexCommandRunnerTests.cs | 54 +++++++++++++++++++ 9 files changed, 133 insertions(+), 21 deletions(-) create mode 100644 changelog.d/unreleased/3745.changed.md diff --git a/USER_GUIDE.md b/USER_GUIDE.md index e303631a6a..6b39bd00a2 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -548,6 +548,7 @@ Use the smallest change that reduces the expensive part of your run. | `--files ` | off | Editor/save hooks or known in-place edits | Does not purge old rename/delete paths unless listed | | `--commits ` | off | After normal commits | Requires git history but sees rename/delete paths | | `--changed-between ` | off | After branch switches when both refs are known | Only as accurate as the supplied refs | +| `--dry-run-path-limit ` | `100000` | Previewing a very large scan without building unbounded dry-run estimates | Truncated output reports lower-bound totals | | `--max-file-bytes ` / `CDIDX_MAX_FILE_BYTES` | `4MiB` | Legitimate large source files are skipped | Raising it can bloat the DB and slow snippet extraction | | `--parallelism ` / `CDIDX_INDEX_PARALLELISM` | CPU count, capped at `16` | Full-scan extraction is CPU-bound | Higher values can increase memory and IO pressure | | `--watch --debounce ` | `500` ms | Keep an active worktree fresh during editing | Long-running process; incompatible with commit/file scoped refresh flags | @@ -1449,6 +1450,7 @@ same source location. | `--files ` | `index` | Update only the specified files. Safe for known in-place edits or new files; old rename/delete paths are not purged unless you also list them explicitly. | | `--force` | `index` | Bypass the per-database index lock. Only use when you are sure no other `cdidx index` is active against the same DB; concurrent runs may corrupt the schema. | | `--duration-format ` | `index` | Choose human elapsed-time display for index summaries. `auto` (default) uses unit labels; `seconds` emits decimal seconds; `hms` keeps `HH:MM:SS`. JSON always keeps raw `elapsed_ms`. | +| `--dry-run-path-limit ` | `index` (`--dry-run` only) | Process at most `` dry-run candidate paths before returning truncated estimates. Defaults to `100000`; values above `1000000` are rejected. When the limit is reached, dry-run JSON sets `candidate_paths_truncated: true` and `totals_lower_bound: true`, and reports `candidate_path_limit` plus `candidate_paths_processed`. | | `--max-file-bytes ` | `index` | Override the per-file indexing limit for this run. Defaults to 4MiB, or `CDIDX_MAX_FILE_BYTES` when set. Values accept raw bytes or `K` / `M` / `G` suffixes such as `50M`. | | `--max-symbols-per-file ` | `index` | Skip file content, symbols, and references when one file emits too many symbols. Defaults to `5000`; values above `50000` are rejected. | | `--symbols-only` | `index` | Full-scan only. Build chunks, symbols, and issues while skipping reference extraction and graph finalization for a faster first pass. `search`, `definition`, `symbols`, and `map` are available; reference graph commands remain degraded until a normal `cdidx index ` run. | @@ -3082,6 +3084,7 @@ cdidx index . --duration-format seconds | `--files ` | off | editor/save hook や既知の in-place edit | rename/delete 旧 path は明示しない限り purge されない | | `--commits ` | off | 通常の commit 後 | git history が必要だが rename/delete paths も扱える | | `--changed-between ` | off | branch switch 後に両 ref が分かる | 渡した ref の正確さに依存 | +| `--dry-run-path-limit ` | `100000` | 非常に大きい scan を preview し、dry-run estimate を無制限に作らない | truncate された出力は lower-bound totals を報告する | | `--max-file-bytes ` / `CDIDX_MAX_FILE_BYTES` | `4MiB` | 正当な大きい source file が skip される | DB が大きくなり snippet extraction も遅くなりうる | | `--parallelism ` / `CDIDX_INDEX_PARALLELISM` | CPU 数、最大 `16` | フルスキャンの抽出が CPU-bound | 大きくするとメモリと IO の圧力が増えうる | | `--watch --debounce ` | `500` ms | 編集中の worktree を live に保つ | long-running process。commit/file scoped refresh flags とは併用不可 | @@ -4001,6 +4004,7 @@ raw match density を正確に測る、といった理由で全 raw chunk hit | `--files ` | `index` | 指定ファイルのみ更新。把握している in-place 編集や新規ファイル向け。rename/delete の旧パスは明示しない限り purge されない。 | | `--force` | `index` | 同一 DB に対する index ロックを bypass する。他の `cdidx index` が走っていないと確信できる場合のみ使う。並行実行は schema を破壊し得る。 | | `--duration-format ` | `index` | index summary の human 経過時間表示を選ぶ。`auto`（既定）は単位付き、`seconds` は小数秒、`hms` は `HH:MM:SS` を維持。JSON は常に raw の `elapsed_ms` を返す。 | +| `--dry-run-path-limit ` | `index`（`--dry-run` 専用） | truncate された estimate を返す前に処理する dry-run candidate path 数を指定する。既定は `100000` で、`1000000` を超える値は拒否される。上限に達した場合、dry-run JSON は `candidate_paths_truncated: true` と `totals_lower_bound: true` を設定し、`candidate_path_limit` と `candidate_paths_processed` も返す。 | | `--max-file-bytes ` | `index` | この実行で使うファイル単位の索引サイズ上限を上書きする。既定は 4MiB、または `CDIDX_MAX_FILE_BYTES` 設定値。値は raw byte 数、または `50M` のような `K` / `M` / `G` 接尾辞を受け付ける。 | | `--symbols-only` | `index` | フルスキャン専用。参照抽出と graph finalization を省き、chunks、symbols、issues だけを作ることで初回利用を速くする。`search`、`definition`、`symbols`、`map` は使えるが、reference graph 系コマンドは通常の `cdidx index ` を実行するまで degraded のまま。 | | `--parallelism ` | `index` | フルスキャンの抽出 worker 数を指定する。既定は CPU 数を最大 16 に丸めた値、または `CDIDX_INDEX_PARALLELISM` 設定値。SQLite 書き込みは単一 consumer のまま。 | diff --git a/changelog.d/unreleased/3745.changed.md b/changelog.d/unreleased/3745.changed.md new file mode 100644 index 0000000000..234aad9de1 --- /dev/null +++ b/changelog.d/unreleased/3745.changed.md @@ -0,0 +1 @@ +Bound `cdidx index --dry-run` candidate-path processing with `--dry-run-path-limit`, stream scoped dry-run candidates where feasible, and report truncation/lower-bound diagnostics in dry-run output. diff --git a/src/CodeIndex/Cli/CliFlagSchema.cs b/src/CodeIndex/Cli/CliFlagSchema.cs index 86d6aef4c8..b25d2c3bc1 100644 --- a/src/CodeIndex/Cli/CliFlagSchema.cs +++ b/src/CodeIndex/Cli/CliFlagSchema.cs @@ -371,6 +371,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--optimize", Description = "Optimize the existing FTS5 table without scanning files", Commands = Set("index") }, new() { Name = "--symbols-only", Description = "Build chunks and symbols while skipping reference graph extraction", Commands = Set("index") }, new() { Name = "--dry-run", Description = "Preview without writing", Commands = Set("index", "backfill-fold", "vacuum") }, + new() { Name = "--dry-run-path-limit", ValuePlaceholder = "", Description = "Dry run only: candidate path processing limit before truncated lower-bound estimates", Commands = Set("index") }, new() { Name = "--no-checkpoint", Description = "Skip the automatic DB checkpoint before maintenance", Commands = Set("backfill-fold") }, new() { Name = "--force", Description = "Bypass the per-database index lock", Commands = Set("index") }, new() { Name = "--duration-format", ValuePlaceholder = "", Description = "Index elapsed time display format", Commands = Set("index") }, diff --git a/src/CodeIndex/Cli/ConsoleUi.cs b/src/CodeIndex/Cli/ConsoleUi.cs index 9a22c11e99..ded9721ed2 100644 --- a/src/CodeIndex/Cli/ConsoleUi.cs +++ b/src/CodeIndex/Cli/ConsoleUi.cs @@ -80,14 +80,14 @@ public static class ConsoleUi private static readonly (string Command, string Usage)[] CommandUsageLines = [ - ("index", "cdidx index [--db ] [--rebuild] [--optimize] [--symbols-only] [--verbose] [--dry-run] [--force] [--quiet] [--json] [--memory-trace] [--duration-format ] [--notify ] [--max-file-bytes ] [--max-symbols-per-file ] [--max-references-per-file ] [--follow-symlinks ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]] [--watch [--debounce ] [--watch-pending-path-limit ]]"), + ("index", "cdidx index [--db ] [--rebuild] [--optimize] [--symbols-only] [--verbose] [--dry-run [--dry-run-path-limit ]] [--force] [--quiet] [--json] [--memory-trace] [--duration-format ] [--notify ] [--max-file-bytes ] [--max-symbols-per-file ] [--max-references-per-file ] [--follow-symlinks ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]] [--watch [--debounce ] [--watch-pending-path-limit ]]"), ("hooks", "cdidx hooks [--project ] [--force] [--json]"), ("backfill-fold", "cdidx backfill-fold [--db ] [--dry-run] [--no-checkpoint] [--json]"), ("optimize", "cdidx optimize [--db ] [--json]"), ("vacuum", "cdidx vacuum [--db ] [--dry-run] [--json]"), - ("index-commits", "cdidx index --commits [commit-ref ...] [--db ] [--verbose] [--dry-run] [--json] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), - ("index-changed-between", "cdidx index --changed-between [--db ] [--verbose] [--dry-run] [--json] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), - ("index-files", "cdidx index --files [path ...] [--db ] [--verbose] [--dry-run] [--json] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), + ("index-commits", "cdidx index --commits [commit-ref ...] [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), + ("index-changed-between", "cdidx index --changed-between [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), + ("index-files", "cdidx index --files [path ...] [--db ] [--verbose] [--dry-run [--dry-run-path-limit ]] [--json] [--memory-trace] [--duration-format ] [--max-file-bytes ] [--include-symbol-kind [,]] [--exclude-symbol-kind [,]]"), ("search", "cdidx search |--query |-- |--recipe |--list-recipes|--named-query = [--named-query = ...] [--include-query ] [--exclude-query ] [--cursor ] [--audit-scope ] [--show-excluded] [--db ] [--json[=ndjson|array]] [--pretty] [--format ] [--open-issues ] [--repo ] [--duplicate-confidence |--duplicate-threshold ] [--issue-title ] [--issue-label <label>] [--verbose] [--limit <n>|--top <n>|--max-results <n>] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--exclude-comments] [--exclude-strings] [--exclude-fixtures] [--snippet-lines <n>] [--snippet-focus <leftmost|quality|proximity>] [--max-line-width <n>] [--fts] [--exact|--exact-substring] [--prefix] [--count] [--group-by <file|symbol>] [--since <datetime>] [--no-dedup] [--no-visibility-rank] [--require-before <query>] [--require-after <query>] [--reject-before <query>] [--reject-after <query>] [--guard-window <n>] [--guard-scope <window|same-line>]"), ("definition", "cdidx definition <query>|--query <query>|-- <query> [--db <path>] [--json] [--format <text|json|count|compact|csv|tsv|lsp|qf|sarif>] [--verbose] [--limit <n>|--top <n>] [--lang <lang>] [--kind <kind>] [--visibility <v[,v]>] [--exclude-visibility <v[,v]>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--body] [--exact|--exact-name] [--count] [--since <datetime>]"), ("goto", "cdidx goto <query>|--query <query>|-- <query> [--db <path>] [--json] [--limit <n>|--top <n>] [--lang <lang>] [--kind <kind>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--exact|--exact-name] [--all]"), @@ -960,6 +960,7 @@ private static void PrintFlagReference(Action<string> WriteHelpLine) Console.WriteLine(" --rebuild Delete existing DB and rebuild from scratch"); Console.WriteLine(" --verbose Show per-file status ([OK ]/[SKIP]/[DEL ]/[ERR ])"); Console.WriteLine(" --dry-run Scan files without writing to the database"); + WriteHelpLine($" --dry-run-path-limit <n> Dry run only: process at most <n> candidate paths before returning truncated lower-bound estimates (default: {IndexCommandRunner.DefaultDryRunPathLimit}, max: {IndexCommandRunner.MaxDryRunPathLimit})"); Console.WriteLine(" --force Bypass the per-database index lock; only use when no other cdidx index is active"); WriteHelpLine(" --symbols-only Build chunks and symbols but skip reference extraction; graph queries stay degraded until a normal index run"); Console.WriteLine(" --json Output results as JSON (for AI/machine use)"); diff --git a/src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs b/src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs index 609e394f25..1a15581477 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs @@ -9,6 +9,8 @@ public static partial class IndexCommandRunner { internal const int DryRunFileSampleLimit = 100; internal const int DryRunErrorSampleLimit = 100; + internal const int DefaultDryRunPathLimit = 100_000; + internal const int MaxDryRunPathLimit = 1_000_000; private const int DryRunScanErrorKeyLimit = 2048; private static int RunDryRun( @@ -28,8 +30,8 @@ private static int RunDryRun( directoryIgnoreCaseProbe: null, symlinkPolicy: options.SymlinkPolicy, generatedCodePatterns: options.GeneratedCodePatterns); - IReadOnlyList<string> dryCandidates; - IReadOnlyList<string> dryDeleteCandidates; + IEnumerable<string> dryCandidates; + IEnumerable<string> dryDeleteCandidates; bool authoritativeFullScan; var errorSamples = new List<CliJsonMessage>(); var errorCount = 0; @@ -91,6 +93,9 @@ void RecordDryRunScanErrors(IEnumerable<FileIndexer.ScanError> scanErrors) var dryFileSamples = new List<string>(); var dryFileCount = 0; + var candidatePathsProcessed = 0; + var candidatePathsTruncated = false; + var dryRunPathLimit = options.DryRunPathLimit; var langCounts = new Dictionary<string, int>(); if (authoritativeFullScan) { @@ -100,6 +105,13 @@ void RecordDryRunScanErrors(IEnumerable<FileIndexer.ScanError> scanErrors) foreach (var f in dryCandidates) { + if (candidatePathsProcessed >= dryRunPathLimit) + { + candidatePathsTruncated = true; + break; + } + + candidatePathsProcessed++; var displayRelativePath = FileIndexer.NormalizePathSeparators(Path.GetRelativePath(projectPath, f)); var dbRelativePath = FileIndexer.NormalizeIndexPath(displayRelativePath); var pathFilter = dryIndexer.EvaluatePathFilter(f); @@ -168,12 +180,19 @@ void RecordDryRunScanErrors(IEnumerable<FileIndexer.ScanError> scanErrors) foreach (var relativePath in dryDeleteCandidates) { + if (candidatePathsProcessed >= dryRunPathLimit) + { + candidatePathsTruncated = true; + break; + } + + candidatePathsProcessed++; var dbRelativePath = FileIndexer.NormalizeIndexPath(relativePath); if (dbSnapshot.Files.ContainsKey(dbRelativePath)) projectedDeletePaths.Add(dbRelativePath); } - if (authoritativeFullScan && dbSnapshot.Files.Count > 0) + if (authoritativeFullScan && dbSnapshot.Files.Count > 0 && !candidatePathsTruncated) { AddProjectedFullScanPurges( projectedPurgePaths, @@ -204,9 +223,13 @@ void RecordDryRunScanErrors(IEnumerable<FileIndexer.ScanError> scanErrors) ProjectedFilePurges = projectedPurges, UnsupportedTotal = unsupportedTotal, UnknownExtensionTotal = unknownExtensionTotal, + CandidatePathLimit = dryRunPathLimit, + CandidatePathsProcessed = candidatePathsProcessed, + CandidatePathsTruncated = candidatePathsTruncated, + TotalsLowerBound = candidatePathsTruncated, EstimatedTableMutations = estimatedTableMutations, FileSamples = dryFileSamples.Count > 0 ? dryFileSamples : null, - FileSamplesTruncated = dryFileCount > dryFileSamples.Count, + FileSamplesTruncated = candidatePathsTruncated || dryFileCount > dryFileSamples.Count, FileSampleLimit = DryRunFileSampleLimit, Languages = langCounts, ErrorsTotal = errorCount, @@ -217,7 +240,10 @@ void RecordDryRunScanErrors(IEnumerable<FileIndexer.ScanError> scanErrors) } else { - Console.WriteLine($"Dry run: {dryFileCount} files would be indexed"); + var lowerBound = candidatePathsTruncated ? " (truncated; totals are lower bounds)" : string.Empty; + Console.WriteLine($"Dry run: {dryFileCount} files would be indexed{lowerBound}"); + if (candidatePathsTruncated) + Console.WriteLine($" candidate paths processed {candidatePathsProcessed.ToString("N0", System.Globalization.CultureInfo.InvariantCulture)} of limit {dryRunPathLimit.ToString("N0", System.Globalization.CultureInfo.InvariantCulture)}"); Console.WriteLine($" projected deletes {projectedDeletes,6}"); Console.WriteLine($" projected purges {projectedPurges,6}"); foreach (var (lang, count) in langCounts.OrderByDescending(kv => kv.Value)) @@ -233,8 +259,8 @@ private static bool TryResolveDryRunCandidates( JsonSerializerOptions jsonOptions, CancellationToken cancellationToken, Action<IEnumerable<FileIndexer.ScanError>> recordDryRunScanErrors, - out IReadOnlyList<string> dryCandidates, - out IReadOnlyList<string> dryDeleteCandidates, + out IEnumerable<string> dryCandidates, + out IEnumerable<string> dryDeleteCandidates, out bool authoritativeFullScan, out DryRunScanMetadata scanMetadata, out int exitCode) @@ -270,19 +296,17 @@ private static bool TryResolveDryRunCandidates( else { dryDeleteCandidates = updatePaths - .Where(path => !File.Exists(LongPath.EnsureWindowsPrefix(Path.Combine(projectPath, path.Replace('/', Path.DirectorySeparatorChar))))) - .ToList(); + .Where(path => !File.Exists(LongPath.EnsureWindowsPrefix(Path.Combine(projectPath, path.Replace('/', Path.DirectorySeparatorChar))))); dryCandidates = updatePaths .Select(path => Path.Combine(projectPath, path.Replace('/', Path.DirectorySeparatorChar))) - .Where(p => File.Exists(LongPath.EnsureWindowsPrefix(p))) - .ToList(); + .Where(p => File.Exists(LongPath.EnsureWindowsPrefix(p))); } } else if (options.Commits.Count > 0 || options.ChangedBetweenSpecified) { // Git update modes: files changed in commits or between refs. // Git更新モード: コミットまたはref間の変更ファイル。 - var changedFiles = new HashSet<string>(StringComparer.Ordinal); + var changedFiles = new SortedSet<string>(StringComparer.Ordinal); var relevantIgnoreFileChanged = false; var repoRoot = GitHelper.TryGetRepositoryRoot(projectPath, cancellationToken) ?? Path.GetFullPath(projectPath); try @@ -360,12 +384,10 @@ private static bool TryResolveDryRunCandidates( else { dryDeleteCandidates = changedFiles - .Where(path => !File.Exists(LongPath.EnsureWindowsPrefix(Path.Combine(projectPath, path.Replace('/', Path.DirectorySeparatorChar))))) - .ToList(); + .Where(path => !File.Exists(LongPath.EnsureWindowsPrefix(Path.Combine(projectPath, path.Replace('/', Path.DirectorySeparatorChar))))); dryCandidates = changedFiles .Select(path => Path.Combine(projectPath, path.Replace('/', Path.DirectorySeparatorChar))) - .Where(p => File.Exists(LongPath.EnsureWindowsPrefix(p))) - .ToList(); + .Where(p => File.Exists(LongPath.EnsureWindowsPrefix(p))); } } else diff --git a/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs b/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs index 29085cc78d..fb83f9ac15 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.Parse.cs @@ -12,7 +12,7 @@ public static partial class IndexCommandRunner // easter egg や random-spinner は意図的に未公開なので除外する。 private static readonly string[] AcceptedIndexFlags = [ - "--db", "--data-dir", "--rebuild", "--verbose", "--json", "--quiet", "--dry-run", "--force", + "--db", "--data-dir", "--rebuild", "--verbose", "--json", "--quiet", "--dry-run", "--dry-run-path-limit", "--force", "--yes", "--watch", "--debounce", "--watch-pending-path-limit", "--duration-format", "--max-file-bytes", "--max-symbols-per-file", "--max-references-per-file", "--notify", "--parallelism", "--memory-trace", "--follow-symlinks", "--symbols-only", @@ -38,6 +38,7 @@ public static IndexCommandOptions ParseArgs(string[] args) bool json = false; bool quiet = false; bool dryRun = false; + var dryRunPathLimit = DefaultDryRunPathLimit; bool force = false; bool readOnly = false; bool yes = false; @@ -112,6 +113,12 @@ public static IndexCommandOptions ParseArgs(string[] args) case "--dry-run": dryRun = true; break; + case "--dry-run-path-limit" when i + 1 < args.Length: + dryRunPathLimit = ParseDryRunPathLimit(args[++i], dryRunPathLimit, "--dry-run-path-limit", ref parseError); + break; + case var option when option.StartsWith("--dry-run-path-limit=", StringComparison.Ordinal): + dryRunPathLimit = ParseDryRunPathLimit(option["--dry-run-path-limit=".Length..], dryRunPathLimit, "--dry-run-path-limit", ref parseError); + break; case "--force": force = true; break; @@ -333,6 +340,7 @@ public static IndexCommandOptions ParseArgs(string[] args) ParseError = parseError ?? generatedCodePatternError, EasterEgg = easterEgg, DryRun = dryRun, + DryRunPathLimit = dryRunPathLimit, Force = force, ReadOnly = readOnly, Yes = yes, @@ -557,6 +565,22 @@ private static int ParseWatchPendingPathLimit(string value, int fallback, string return fallback; } + private static int ParseDryRunPathLimit(string value, int fallback, string source, ref string? parseError) + { + if (int.TryParse(value, System.Globalization.NumberStyles.Integer, System.Globalization.CultureInfo.InvariantCulture, out var parsed) && parsed > 0) + { + if (parsed <= MaxDryRunPathLimit) + return parsed; + + parseError ??= $"{source} must be less than or equal to {MaxDryRunPathLimit}"; + return fallback; + } + + var displayValue = ConsoleUi.FormatBoundedValue(value); + CommandErrorWriter.WriteStderr($"Warning: invalid {source} value '{displayValue}' (ignored; use a positive integer) / 不正な {source} 値 '{displayValue}'（無視。正の整数を指定）"); + return fallback; + } + private static long? ReadMaxFileSizeBytesFromEnvironment() { var value = Environment.GetEnvironmentVariable(FileIndexer.MaxFileSizeEnvironmentVariable); diff --git a/src/CodeIndex/Cli/IndexCommandRunner.cs b/src/CodeIndex/Cli/IndexCommandRunner.cs index 7b140575e9..264f115807 100644 --- a/src/CodeIndex/Cli/IndexCommandRunner.cs +++ b/src/CodeIndex/Cli/IndexCommandRunner.cs @@ -1603,6 +1603,7 @@ public sealed class IndexCommandOptions public string? ParseError { get; init; } public string? EasterEgg { get; init; } public bool DryRun { get; init; } + public int DryRunPathLimit { get; init; } = IndexCommandRunner.DefaultDryRunPathLimit; public bool Force { get; init; } public bool ReadOnly { get; init; } public bool Yes { get; init; } diff --git a/src/CodeIndex/Cli/JsonOutputContracts.cs b/src/CodeIndex/Cli/JsonOutputContracts.cs index 7d3931b9e1..d9380af31c 100644 --- a/src/CodeIndex/Cli/JsonOutputContracts.cs +++ b/src/CodeIndex/Cli/JsonOutputContracts.cs @@ -277,6 +277,10 @@ internal sealed class IndexDryRunJsonResult public int ProjectedFilePurges { get; init; } public int UnsupportedTotal { get; init; } public int UnknownExtensionTotal { get; init; } + public int CandidatePathLimit { get; init; } + public int CandidatePathsProcessed { get; init; } + public bool CandidatePathsTruncated { get; init; } + public bool TotalsLowerBound { get; init; } public Dictionary<string, long> EstimatedTableMutations { get; init; } = new(); public List<string>? FileSamples { get; init; } public bool FileSamplesTruncated { get; init; } diff --git a/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs b/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs index 48d5289785..ce3258955c 100644 --- a/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs +++ b/tests/CodeIndex.Tests/IndexCommandRunnerTests.cs @@ -2931,6 +2931,27 @@ public void ParseArgs_DebounceFlag_InvalidValue_IsIgnored() } } + [Fact] + public void ParseArgs_DryRunPathLimitFlag_ParsesValue() + { + var options = IndexCommandRunner.ParseArgs([".", "--dry-run", "--dry-run-path-limit", "42"]); + + Assert.True(options.DryRun); + Assert.Equal(42, options.DryRunPathLimit); + } + + [Fact] + public void ParseArgs_DryRunPathLimitFlag_RejectsValueAboveMaximum() + { + var oversized = $"{IndexCommandRunner.MaxDryRunPathLimit + 1}"; + + var options = IndexCommandRunner.ParseArgs([".", "--dry-run", "--dry-run-path-limit", oversized]); + + Assert.Equal(IndexCommandRunner.DefaultDryRunPathLimit, options.DryRunPathLimit); + Assert.Contains("--dry-run-path-limit", options.ParseError); + Assert.Contains($"{IndexCommandRunner.MaxDryRunPathLimit}", options.ParseError); + } + [Fact] public void ParseArgs_WatchPendingPathLimitFlag_ParsesValue() { @@ -9573,6 +9594,10 @@ public void Run_DryRun_JsonCapsFileSamples() Assert.Equal(fileCount, json.GetProperty("files_total").GetInt32()); Assert.Equal(fileCount, json.GetProperty("languages").GetProperty("csharp").GetInt32()); Assert.Equal(IndexCommandRunner.DryRunFileSampleLimit, json.GetProperty("file_sample_limit").GetInt32()); + Assert.Equal(IndexCommandRunner.DefaultDryRunPathLimit, json.GetProperty("candidate_path_limit").GetInt32()); + Assert.Equal(fileCount, json.GetProperty("candidate_paths_processed").GetInt32()); + Assert.False(json.GetProperty("candidate_paths_truncated").GetBoolean()); + Assert.False(json.GetProperty("totals_lower_bound").GetBoolean()); Assert.True(json.GetProperty("file_samples_truncated").GetBoolean()); Assert.Equal(IndexCommandRunner.DryRunFileSampleLimit, json.GetProperty("file_samples").GetArrayLength()); Assert.Equal(0, json.GetProperty("errors_total").GetInt32()); @@ -9584,6 +9609,35 @@ public void Run_DryRun_JsonCapsFileSamples() } } + [Fact] + public void Run_DryRun_PathLimitTruncatesCandidateProcessing() + { + var projectRoot = CreateTempProject(); + try + { + File.WriteAllText(Path.Combine(projectRoot, "sample001.cs"), "public class Sample001 { }\n"); + File.WriteAllText(Path.Combine(projectRoot, "sample002.cs"), "public class Sample002 { }\n"); + File.WriteAllText(Path.Combine(projectRoot, "sample003.cs"), "public class Sample003 { }\n"); + + var (exitCode, json) = RunAndCaptureJson([projectRoot, "--dry-run", "--dry-run-path-limit", "2", "--json"]); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal("dry_run", json.GetProperty("status").GetString()); + Assert.Equal(2, json.GetProperty("files_total").GetInt32()); + Assert.Equal(2, json.GetProperty("projected_file_updates").GetInt32()); + Assert.Equal(2, json.GetProperty("candidate_path_limit").GetInt32()); + Assert.Equal(2, json.GetProperty("candidate_paths_processed").GetInt32()); + Assert.True(json.GetProperty("candidate_paths_truncated").GetBoolean()); + Assert.True(json.GetProperty("totals_lower_bound").GetBoolean()); + Assert.True(json.GetProperty("file_samples_truncated").GetBoolean()); + Assert.Equal(0, json.GetProperty("errors_total").GetInt32()); + } + finally + { + DeleteDirectory(projectRoot); + } + } + [Fact] public void Run_DryRun_JsonCapsErrorSamples() { From 701749531f323caa619a89a0df49e8924463a530 Mon Sep 17 00:00:00 2001 From: Widthdom <widthdom@gmail.com> Date: Sat, 20 Jun 2026 23:53:38 +0900 Subject: [PATCH 3/4] fix(changelog): add fragment metadata for issues #3726 and #3745 --- changelog.d/unreleased/3726.changed.md | 19 ++++++++++++++++++- changelog.d/unreleased/3745.changed.md | 19 ++++++++++++++++++- 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/changelog.d/unreleased/3726.changed.md b/changelog.d/unreleased/3726.changed.md index e776bef87e..c78d9a5d7d 100644 --- a/changelog.d/unreleased/3726.changed.md +++ b/changelog.d/unreleased/3726.changed.md @@ -1 +1,18 @@ -Allow `cdidx index --watch` pending-path queue limits to be configured with `--watch-pending-path-limit`, `CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT`, or `indexing.watchPendingPathLimit`, and report the effective limit in watch diagnostics. +--- +category: changed +issues: + - 3726 +affected: + - USER_GUIDE.md + - src/CodeIndex/Cli/IndexCommandRunner.cs + - src/CodeIndex/Cli/IndexWatchRunner.cs + - tests/CodeIndex.Tests/IndexWatchRunnerTests.cs +--- + +## English + +- Allow `cdidx index --watch` pending-path queue limits to be configured with `--watch-pending-path-limit`, `CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT`, or `indexing.watchPendingPathLimit`, and report the effective limit in watch diagnostics. + +## 日本語 + +- `cdidx index --watch` の pending-path queue 上限を `--watch-pending-path-limit`、`CDIDX_INDEX_WATCH_PENDING_PATH_LIMIT`、または `indexing.watchPendingPathLimit` で設定できるようにし、watch diagnostics に有効な上限を出力するようにしました。 diff --git a/changelog.d/unreleased/3745.changed.md b/changelog.d/unreleased/3745.changed.md index 234aad9de1..2e22de327e 100644 --- a/changelog.d/unreleased/3745.changed.md +++ b/changelog.d/unreleased/3745.changed.md @@ -1 +1,18 @@ -Bound `cdidx index --dry-run` candidate-path processing with `--dry-run-path-limit`, stream scoped dry-run candidates where feasible, and report truncation/lower-bound diagnostics in dry-run output. +--- +category: changed +issues: + - 3745 +affected: + - USER_GUIDE.md + - src/CodeIndex/Cli/IndexCommandRunner.DryRun.cs + - src/CodeIndex/Cli/IndexCommandRunner.Parse.cs + - tests/CodeIndex.Tests/IndexCommandRunnerTests.cs +--- + +## English + +- Bound `cdidx index --dry-run` candidate-path processing with `--dry-run-path-limit`, stream scoped dry-run candidates where feasible, and report truncation/lower-bound diagnostics in dry-run output. + +## 日本語 + +- `cdidx index --dry-run` の candidate-path 処理を `--dry-run-path-limit` で制限し、可能な範囲で scoped dry-run candidates を stream 処理し、dry-run 出力に truncation / lower-bound diagnostics を出すようにしました。 From 78d2b1531bb08a2d45dba02a53ac18104180ceae Mon Sep 17 00:00:00 2001 From: Widthdom <widthdom@gmail.com> Date: Sun, 21 Jun 2026 01:00:57 +0900 Subject: [PATCH 4/4] test(cli): sync usage expectations for issues #3726 and #3745 --- tests/CodeIndex.Tests/ConsoleUiTests.cs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/CodeIndex.Tests/ConsoleUiTests.cs b/tests/CodeIndex.Tests/ConsoleUiTests.cs index 6fcf4fc24b..24bed398dc 100644 --- a/tests/CodeIndex.Tests/ConsoleUiTests.cs +++ b/tests/CodeIndex.Tests/ConsoleUiTests.cs @@ -118,10 +118,10 @@ public void PrintUsage_WithoutBanner_HidesAsciiArtAndEasterEggFlags() Assert.DoesNotContain("██████╗", output); Assert.Contains("Usage:", output); - Assert.Contains("cdidx index <projectPath> [--db <path>] [--rebuild] [--optimize] [--symbols-only] [--verbose] [--dry-run] [--force] [--quiet] [--json] [--memory-trace] [--duration-format <auto|seconds|hms>] [--notify <auto|bell|osc9|desktop|none>] [--max-file-bytes <bytes>] [--max-symbols-per-file <n>] [--max-references-per-file <n>] [--follow-symlinks <none|internal|all>]", output); + Assert.Contains("cdidx index <projectPath> [--db <path>] [--rebuild] [--optimize] [--symbols-only] [--verbose] [--dry-run [--dry-run-path-limit <n>]] [--force] [--quiet] [--json] [--memory-trace] [--duration-format <auto|seconds|hms>] [--notify <auto|bell|osc9|desktop|none>] [--max-file-bytes <bytes>] [--max-symbols-per-file <n>] [--max-references-per-file <n>] [--follow-symlinks <none|internal|all>] [--include-symbol-kind <kind>[,<kind>]] [--exclude-symbol-kind <kind>[,<kind>]] [--watch [--debounce <ms>] [--watch-pending-path-limit <n>]]", output); Assert.Contains("cdidx hooks <install|uninstall|status> [--project <path>] [--force] [--json]", output); - Assert.Contains("cdidx index <projectPath> --commits <commit-ref> [commit-ref ...] [--db <path>] [--verbose] [--dry-run] [--json] [--memory-trace] [--duration-format <auto|seconds|hms>]", output); - Assert.Contains("cdidx index <projectPath> --files <path> [path ...] [--db <path>] [--verbose] [--dry-run] [--json] [--memory-trace] [--duration-format <auto|seconds|hms>]", output); + Assert.Contains("cdidx index <projectPath> --commits <commit-ref> [commit-ref ...] [--db <path>] [--verbose] [--dry-run [--dry-run-path-limit <n>]] [--json] [--memory-trace] [--duration-format <auto|seconds|hms>] [--max-file-bytes <bytes>] [--include-symbol-kind <kind>[,<kind>]] [--exclude-symbol-kind <kind>[,<kind>]]", output); + Assert.Contains("cdidx index <projectPath> --files <path> [path ...] [--db <path>] [--verbose] [--dry-run [--dry-run-path-limit <n>]] [--json] [--memory-trace] [--duration-format <auto|seconds|hms>] [--max-file-bytes <bytes>] [--include-symbol-kind <kind>[,<kind>]] [--exclude-symbol-kind <kind>[,<kind>]]", output); Assert.Contains("cdidx backfill-fold [--db <path>] [--dry-run] [--no-checkpoint] [--json]", output); Assert.Contains("cdidx optimize [--db <path>] [--json]", output); Assert.Contains("cdidx license", output);