diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index b19fe1d19..e9ee364d6 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -1731,6 +1731,7 @@ access. | Grouped search totals | `search --format grouped` derives `total_matches` / `matched_count`, `total_groups`, and `total_files` from the complete bounded query rather than the displayed page. `grouped_match_count` counts rows supplied to returned groups, `emitted_match_count` counts rows left after per-file grouping limits, and `omitted_match_count`, `truncated`, `has_more`, and `continuation_action` describe incomplete output. | | Bounded high-volume responses | `search`, `definition`, `find`, `status`, `hotspots`, `references`, `callers`, `callees`, `symbols`, `files`, `languages`, `impact`, and `map` accept shared bounded-response controls where their schema exposes them. Newly emitted opaque `--cursor ` values bind the offset to the command/query/filter selection and index generation; legacy `response:v1::` cursors remain accepted for transition. Reuse with changed selection or generation fails with restart-required guidance. `search --format compact`, `symbols --format compact`, and `files --format compact` auto-select the bounded contract, while `search --json=array --json-envelope` provides the opt-in array envelope and `languages --json` selects it when paging or `--max-json-bytes` is requested. Existing compact roots and location rows remain compatible while adding shared metadata. Metadata reports `returned_count`, authoritative `total_count` where available, `omitted_count`, `remaining_count`, `cursor_offset`, `page_limit`, `has_more`, `next_cursor`, `result_stable_at`, `pagination_window_limit`, and `pagination_window_exhausted`. The safety window is 10,000 rows; exhaustion suppresses `next_cursor` rather than returning a cursor that the next request would reject. Pageable commands pass the cursor offset into their database/scan layer instead of serializing an `offset + limit` prefix. `find --all` partial scans encode the next path/line in the opaque cursor so replay continues after the last scanned line. `hotspots` and `impact` page their active primary nested collection as `results`, identify it with `metadata.primary_collection`, and retain scalar/container evidence in `metadata.response_context`; dotted fields such as `callers.path,callers.depth` select that collection and project its rows. The final newline is included in `--max-json-bytes`, and trailing whole rows are removed until the complete envelope fits. `definition` remains metadata-only by default; explicit `--body` content is retained for `body`, `body_content`, or `all`, and suppressed when the projection excludes it. `map --sections` remains its section-level projection, while dotted bounded fields page a selected array section with section-specific totals and scalar projections skip unused ranked arrays. | | Bounded outline responses | `outline` opts into the shared bounded-response contract only when `--max-json-bytes` is present. The wrapper extracts complete projected symbol rows, preserves hierarchy and deterministic order, reports authoritative returned / total / omitted counts, includes the final newline in its UTF-8 measurement, and emits a bound `response:v2` continuation cursor. An undersized minimum envelope produces typed `E010_USAGE_ERROR` diagnostics with empty stdout. Uncapped outline JSON retains its existing root shape and outline cursor contract. | +| Bounded unused responses | `unused` opts into the shared bounded-response contract only when `--max-json-bytes` is present. The wrapper extracts the canonical `symbols` rows, applies the cursor offset in the unused query layer, recomputes returned bucket / confidence / contract-domain counts after byte trimming, and includes an optional `by_bucket` view in the same whole-response UTF-8 budget. Compact mode projects smaller audit rows. Continuation cursors bind the effective audit filters, bucket mode, ordering, and index generation; an undersized one-row envelope returns typed `E010_USAGE_ERROR` diagnostics with empty stdout. Uncapped JSON, compact summaries, and the legacy unused cursor remain unchanged. | | MCP outline pages | MCP `outline` routes `fields`, `sort`, `limit`, and `cursor` through `QueryCommandRunner.BuildOutlinePage`, so its projection aliases, derived sort fields, stable tie-breakers, `page:v1` query fingerprint, and generation validation remain the CLI outline contract rather than a second MCP-specific implementation. The default page is 100 rows and the MCP-wide maximum is 200. `maxBytes` measures the fully enriched serialized `structuredContent`; a binary search rebuilds the page with fewer complete rows and therefore regenerates `next_cursor` from the actual returned count. A budget that cannot hold metadata plus one row fails instead of returning a zero-progress cursor. Default MCP symbol serialization remains backward-compatible, while explicit projection fields use the CLI snake_case names. | | Bounded response edge cases | `impact` applies the cursor offset only to the selected nested collection so definition pages do not repeat or alter caller/fallback mode. Plain `map --compact` preserves its established section arrays and truncation payload; a collection projection is rejected when `--summary-only` or an excluding `--sections` filter would remove it. Explicit definition body fields override compact defaults. Profile and verbose records are moved into `metadata.stream_control_records`, and parser/capture failures emit an error envelope only when it fits the active hard byte cap. | | `--count --json` envelope | Count-only JSON for `search`, `definition`, `references`, `callers`, `callees`, `symbols`, `files`, `find`, `impact`, and `unused` is a single automation-oriented object. It always includes `count`, applied `query_context`, freshness metadata (`indexed_file_count`, `indexed_at`, `freshness_available`), and trust flags `degraded` / `authoritative_count`; commands with matched-file totals also include `files` and the older `file_count` compatibility alias. `file_count` carries the same value as `files`, remains for compatibility, and is not scheduled for removal before the next major release. `unused --count --json` also includes `returned_bucket_counts`, `returned_contract_domain_counts`, and `summary.by_bucket` / `summary.by_confidence` / `summary.by_contract_domain`. `authoritative_count=false` means a readiness or graph/exact trust signal made the count non-authoritative, while the freshness fields describe the indexed snapshot used for the count. | @@ -5180,6 +5181,7 @@ help はすべてこのレジストリを参照します。field 名は大文字 | grouped search の総数 | `search --format grouped` の `total_matches` / `matched_count`、`total_groups`、`total_files` は、表示 page ではなく上限適用前の query 全体から算出します。`grouped_match_count` は返却 group に渡した row 数、`emitted_match_count` は file ごとの上限適用後に残った row 数を表し、`omitted_match_count`、`truncated`、`has_more`、`continuation_action` が未完了出力を示します。 | | 高ボリューム応答の bounded 契約 | `search`、`definition`、`find`、`status`、`hotspots`、`references`、`callers`、`callees`、`symbols`、`files`、`languages`、`impact`、`map` は、それぞれの schema が公開する共通 bounded-response control に対応します。新しく発行する opaque な `--cursor ` は offset を command / query / filter と index generation に束縛し、移行用に legacy の `response:v1::` も受理します。選択条件または generation を変えて再利用すると restart-required の案内付きで失敗します。`search --format compact`、`symbols --format compact`、`files --format compact` は bounded 契約を自動選択し、`search --json=array --json-envelope` は opt-in の array envelope、`languages --json` は paging または `--max-json-bytes` 指定時に同じ契約を使います。既存 compact の root と location row は維持したまま共通 metadata を追加します。metadata は `returned_count`、取得可能な場合は authoritative な `total_count`、`omitted_count`、`remaining_count`、`cursor_offset`、`page_limit`、`has_more`、`next_cursor`、`result_stable_at`、`pagination_window_limit`、`pagination_window_exhausted` を返します。safety window は 10,000 row で、上限到達時は次の request が拒否する cursor を返さず `next_cursor` を抑止します。pageable command は `offset + limit` 件を serialize せず、cursor offset を database / scan layer へ渡します。`find --all` の partial scan cursor は次の path / line を保持し、再利用時は最後に scan した line の次から継続します。`hotspots` と `impact` は active な主要 nested collection を `results` としてページングし、`metadata.primary_collection` でその名前を示し、scalar / container evidence は `metadata.response_context` に保持します。`callers.path,callers.depth` のような dotted field で collection と row field を同時に選べます。`--max-json-bytes` は最後の改行を含み、完全な envelope が収まるまで末尾の完全な row を省略します。`definition` は既定で metadata-only のままで、明示的な `--body` は `body`、`body_content`、`all` で保持し、それ以外の projection では materialize 前に抑止します。`map --sections` は section-level projection として残り、dotted な bounded field は選択した array section を section 固有の総件数付きでページングし、scalar projection は不要な ranking array を構築しません。 | | bounded outline 応答 | `outline` は `--max-json-bytes` がある場合だけ共通 bounded-response 契約を選択します。wrapper は projection 済みの完全な symbol row を抽出し、階層と決定的な順序を維持し、authoritative な返却 / 総 / 省略件数を報告します。UTF-8 計測には最後の改行を含め、束縛済みの `response:v2` continuation cursor を発行します。最小 envelope が収まらない場合は stdout を空に保ち、型付きの `E010_USAGE_ERROR` diagnostic を返します。上限なしの outline JSON は既存の root shape と outline cursor 契約を維持します。 | +| bounded unused 応答 | `unused` は `--max-json-bytes` がある場合だけ共通 bounded-response 契約を選択します。wrapper は canonical な `symbols` row を抽出し、unused query layer で cursor offset を適用し、byte trimming 後の返却 bucket / confidence / contract-domain 件数を再計算します。任意の `by_bucket` view も同じ応答全体の UTF-8 budget に含め、compact mode はより小さな audit row へ projection します。continuation cursor は有効な audit filter、bucket mode、ordering、index generation に束縛されます。1 row を含む最小 envelope が収まらない場合は stdout を空にして、型付きの `E010_USAGE_ERROR` diagnostic を返します。上限なしの JSON、compact summary、legacy unused cursor は既存契約を維持します。 | | MCP outline page | MCP `outline` は `fields`、`sort`、`limit`、`cursor` を `QueryCommandRunner.BuildOutlinePage` へ渡すため、projection alias、派生 sort field、安定した tie-breaker、`page:v1` query fingerprint、generation validation は第 2 の MCP 固有実装ではなく CLI outline 契約のままです。既定 page は 100 row、MCP 共通の上限は 200 row です。`maxBytes` は enrichment 済みの `structuredContent` 全体を serialize した byte 数で計測し、binary search で完全な row 数を減らして page を再構築するため、`next_cursor` も実際の返却件数から再生成されます。metadata と 1 row が収まらない budget は、進捗しない cursor を返さず失敗します。既定の MCP symbol serialization は後方互換を維持し、明示的な projection field は CLI の snake_case 名を使います。 | | bounded 応答の edge case | `impact` は選択された nested collection だけに cursor offset を適用するため、definition page の重複や caller / fallback mode の変化を防ぎます。通常の `map --compact` は既存の section array と truncation payload を維持し、collection projection が `--summary-only` または除外する `--sections` filter で失われる組み合わせは拒否します。明示的な definition body field は compact default より優先します。profile / verbose record は `metadata.stream_control_records` へ移し、parser / capture failure の error envelope は active な hard byte cap に収まる場合だけ出力します。 | | `--count --json` envelope | `search`、`definition`、`references`、`callers`、`callees`、`symbols`、`files`、`find`、`impact`、`unused` の count-only JSON は単一の自動化向け object です。常に `count`、適用済み `query_context`、freshness metadata（`indexed_file_count`、`indexed_at`、`freshness_available`）、trust flag の `degraded` / `authoritative_count` を含みます。matched-file total を持つ command は `files` と古い互換 alias の `file_count` も含みます。`file_count` は `files` と同じ値を持つ互換用 field として残り、少なくとも次の major release までは削除予定はありません。`unused --count --json` は `returned_bucket_counts`、`returned_contract_domain_counts`、`summary.by_bucket` / `summary.by_confidence` / `summary.by_contract_domain` も含みます。`authoritative_count=false` は readiness または graph/exact trust signal により count が authoritative ではないことを示し、freshness field は count に使った index snapshot を説明します。 | diff --git a/README.md b/README.md index e2e52846c..3cc9e9a4b 100644 --- a/README.md +++ b/README.md @@ -77,7 +77,7 @@ cdidx definition UserService cdidx references UserService --fields path,line,reference_kind --limit 20 --max-json-bytes 16384 cdidx inspect QueryCommandRunner --outline-only cdidx outline src/CodeIndex/Cli/QueryCommandRunner.cs --json --kind function --limit 50 --max-json-bytes 16384 -cdidx unused --compact --by-bucket +cdidx unused --compact --by-bucket --max-json-bytes 16384 cdidx map --compact --max-json-bytes 65536 cdidx map --format issue-drafts --limit 10 cdidx search --recipe risky-code --format compact --max-json-bytes 65536 @@ -574,7 +574,7 @@ cdidx definition UserService cdidx references UserService --fields path,line,reference_kind --limit 20 --max-json-bytes 16384 cdidx inspect QueryCommandRunner --outline-only cdidx outline src/CodeIndex/Cli/QueryCommandRunner.cs --json --kind function --limit 50 --max-json-bytes 16384 -cdidx unused --compact --by-bucket +cdidx unused --compact --by-bucket --max-json-bytes 16384 cdidx map --compact --max-json-bytes 65536 cdidx map --format issue-drafts --limit 10 cdidx search --recipe risky-code --format compact --max-json-bytes 65536 diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 490afdc78..20f689910 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -229,6 +229,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding Unused default-suppression row, JSON count, summary-only, and text count envelopes, including the `--all` count control, share one unused-symbol fixture. Unused default-suppressed and `--all` JSON cursor pagination share one unused-symbol fixture. Unused full and compact `--by-bucket` JSON envelopes share one taxonomy fixture. + Unused bounded JSON coverage shares one Unicode long-signature fixture across full, compact, and `--by-bucket` paging. Assert whole-response UTF-8 limits including the final newline, complete rows, exact cursor continuation without gaps or duplicates, recomputed response-context counts, empty/final pages, minimum-budget typed errors, help/schema exposure, and filter/generation cursor rejection. Unused bucket, minimum-confidence, and actionable confidence-alias JSON filters share one unused-symbol fixture. Unused full-summary and bucket-filtered JSON counts share one taxonomy fixture. Unused limited-page returned counts and bucket diversification share one taxonomy fixture. @@ -1178,6 +1179,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" unused default-suppressionのrow、JSON count、summary-only、text count envelopeは、`--all` count controlも含めて1つのunused-symbol fixtureを共有してください。 unusedのdefault-suppressed JSON cursor paginationと`--all` JSON cursor paginationは1つのunused-symbol fixtureを共有してください。 unusedのfull JSONとcompact `--by-bucket` JSON envelopeは1つのtaxonomy fixtureを共有してください。 + unused の bounded JSON coverage は、full、compact、`--by-bucket` paging で Unicode の長い signature を持つ1つの fixture を共有してください。最後の改行を含む応答全体の UTF-8 limit、完全な row、欠落や重複のない正確な cursor continuation、再計算された response-context 件数、empty / final page、最小 budget の型付き error、help / schema 公開、filter / generation cursor 拒否を検証してください。 unusedのbucket、minimum-confidence、actionable confidence-alias JSON filterは1つのunused-symbol fixtureを共有してください。 unusedのfull-summary JSON countとbucket-filtered JSON countは1つのtaxonomy fixtureを共有してください。 unusedのlimited-page returned countとbucket diversificationは1つのtaxonomy fixtureを共有してください。 diff --git a/USER_GUIDE.md b/USER_GUIDE.md index f5f9b2c57..0c3b879d7 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -704,6 +704,7 @@ cdidx unused --all --json --count cdidx unused --compact --bucket likely_unused_private --min-confidence medium cdidx unused --json --by-bucket cdidx unused --compact --by-bucket +cdidx unused --json --limit 50 --max-json-bytes 16384 ``` `unused` compares definitions with indexed references and groups results by @@ -733,6 +734,13 @@ When `unused` returns `next_cursor`, pass that opaque value back unchanged. The cursor is bound to the effective audit scope, filters, ordering, and index generation. Changing those inputs or refreshing the index requires restarting without `--cursor`; JSON pages also expose `result_stable_at`. +Use `--max-json-bytes ` to place a hard UTF-8 byte cap, including the final +newline, on the whole JSON response. The bounded envelope returns only complete +symbol rows, reports the actual returned and omitted counts, and resumes with a +generation-bound `response:v2` cursor. `--compact` projects smaller rows, while +`--by-bucket` keeps the bucket view inside the same byte budget. If the metadata +and one row cannot fit, `unused` returns a typed usage error with empty stdout. +Without the byte cap, the existing JSON and cursor shapes remain unchanged. For C# private members declared in partial types, `unused` aggregates use evidence across sibling files by fully qualified logical type name, including nested partial types. Same-named types in other namespaces or containing types @@ -4015,6 +4023,7 @@ cdidx unused --all --json --count cdidx unused --compact --bucket likely_unused_private --min-confidence medium cdidx unused --json --by-bucket cdidx unused --compact --by-bucket +cdidx unused --json --limit 50 --max-json-bytes 16384 ``` `unused` は definitions と indexed references を比較し、confidence ごとに結果を @@ -4040,6 +4049,12 @@ filter context だけが必要な場合は `--compact` を使ってください `unused` が `next_cursor` を返した場合は、その opaque 値を変更せず次の呼び出しへ渡してください。 cursor は有効な audit scope、filter、ordering、index generation に束縛されます。条件を変更した場合や index を更新した場合は `--cursor` なしで再開する必要があり、JSON page は `result_stable_at` も返します。 +`--max-json-bytes ` を使うと、最後の改行を含む JSON 応答全体へ UTF-8 byte の +hard cap を設定できます。bounded envelope は完全な symbol row だけを返し、実際の返却件数と +省略件数を報告して、generation に束縛された `response:v2` cursor から再開します。 +`--compact` はより小さな row へ projection し、`--by-bucket` の bucket view も同じ +byte budget 内に収めます。metadata と 1 row が収まらない場合、`unused` は stdout を空にして +型付き usage error を返します。byte cap を指定しない既存の JSON と cursor shape は変わりません。 C# の partial type で宣言された private member について、`unused` は nested partial type を含む 完全修飾された logical type 名を使い、sibling file 全体の使用 evidence を集約します。 別 namespace または別 containing type にある同名 type とは evidence を共有せず、 diff --git a/changelog.d/unreleased/4905.added.md b/changelog.d/unreleased/4905.added.md new file mode 100644 index 000000000..70a8e44ab --- /dev/null +++ b/changelog.d/unreleased/4905.added.md @@ -0,0 +1,23 @@ +--- +category: added +issues: + - 4905 +affected: + - src/CodeIndex/Cli/CliFlagSchema.cs + - src/CodeIndex/Cli/ConsoleUi.cs + - src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs + - src/CodeIndex/Cli/QueryCommandRunner.Unused.cs + - tests/CodeIndex.Tests/QueryCommandRunnerUnusedIssue4905Tests.cs + - README.md + - USER_GUIDE.md + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **`unused` now supports a hard JSON byte budget (#4905)** — `--max-json-bytes` returns complete UTF-8 symbol rows in the shared bounded envelope for normal, compact, and `--by-bucket` output, with recomputed bucket summaries and generation-bound continuation cursors. Long signatures and Unicode remain ordered across pages, undersized budgets return typed diagnostics with empty stdout, and uncapped output remains compatible. + +## 日本語 + +- **`unused` が JSON の hard byte budget に対応しました (#4905)** — `--max-json-bytes` は通常、compact、`--by-bucket` 出力で完全な UTF-8 symbol row を共通 bounded envelope により返し、bucket summary を再計算して generation に束縛された continuation cursor を提供します。長い signature と Unicode の順序は page 間で維持され、収まらない小さな budget は stdout を空にして型付き diagnostic を返し、上限なしの出力は互換性を維持します。 diff --git a/src/CodeIndex/Cli/CliFlagSchema.cs b/src/CodeIndex/Cli/CliFlagSchema.cs index 96b9fff8c..e266a089b 100644 --- a/src/CodeIndex/Cli/CliFlagSchema.cs +++ b/src/CodeIndex/Cli/CliFlagSchema.cs @@ -566,7 +566,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--env-domain", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact domain", PrimaryCommands = Set("doctor") }, new() { Name = "--env-category", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact category", PrimaryCommands = Set("doctor") }, new() { Name = "--env-sensitivity", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact sensitivity", PrimaryCommands = Set("doctor") }, - new() { Name = "--max-json-bytes", ValuePlaceholder = "", Description = "Bound emitted JSON bytes; bounded high-volume responses truncate projected rows with paging metadata", PrimaryCommands = Set("search", "definition", "find", "status", "references", "callers", "callees", "excerpt", "inspect", "outline", "impact", "recipes", "audit", "map", "files", "symbols", "deps", "hotspots", "languages", "doctor") }, + new() { Name = "--max-json-bytes", ValuePlaceholder = "", Description = "Bound emitted JSON bytes; bounded high-volume responses truncate projected rows with paging metadata", PrimaryCommands = Set("search", "definition", "find", "status", "references", "callers", "callees", "excerpt", "inspect", "outline", "impact", "recipes", "audit", "map", "files", "symbols", "deps", "hotspots", "languages", "unused", "doctor") }, new() { Name = "--next-steps", Description = "Search: print inspect/excerpt follow-up commands for top hits", PrimaryCommands = Set("search") }, new() { Name = "--exclude-comments", Description = "Search: suppress comment-only matches after origin classification", PrimaryCommands = Set("search") }, new() { Name = "--exclude-strings", Description = "Search: suppress string, regex, and help-text matches after origin classification", PrimaryCommands = Set("search") }, diff --git a/src/CodeIndex/Cli/ConsoleUi.cs b/src/CodeIndex/Cli/ConsoleUi.cs index de397103f..22a2db4d5 100644 --- a/src/CodeIndex/Cli/ConsoleUi.cs +++ b/src/CodeIndex/Cli/ConsoleUi.cs @@ -131,7 +131,7 @@ private static readonly (string Command, string Usage)[] CommandUsageLines = ("validate", "cdidx validate [--db ] [--json[=array]] [--format ] [--verbose] [--limit |--top ] [--kind ] [--severity ] [--path ]"), ("impact", "cdidx impact |--query |-- [--db ] [--json] [--redact-paths|--show-paths] [--format ] [--compact] [--fields ] [--cursor ] [--max-json-bytes ] [--verbose] [--limit |--top ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--body] [--snippet-lines ] [--max-line-width ] [--max-hops ] [--exact-name] [--count] [--with-paths] [--include-member-reads]"), ("deps", "cdidx deps [--db ] [--json] [--format ] [--summary-only] [--max-json-bytes ] [--verbose] [--limit |--top ] [--cursor ] [--graph-budget ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--reverse] [--cycles] [--suppress-noise] [--symbol ] [--symbol-family ]"), - ("unused", "cdidx unused [--db ] [--json] [--compact] [--summary-only] [--verbose] [--limit |--top ] [--cursor ] [--audit-scope ] [--kind ] [--bucket ] [--min-confidence |--confidence ] [--actionable] [--all] [--visibility ] [--exclude-visibility ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--count] [--by-bucket]"), + ("unused", "cdidx unused [--db ] [--json] [--compact] [--summary-only] [--max-json-bytes ] [--verbose] [--limit |--top ] [--cursor ] [--audit-scope ] [--kind ] [--bucket ] [--min-confidence |--confidence ] [--actionable] [--all] [--visibility ] [--exclude-visibility ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--count] [--by-bucket]"), ("hotspots", "cdidx hotspots [--db ] [--json] [--format ] [--compact] [--fields ] [--cursor ] [--summary-only] [--max-json-bytes ] [--verbose] [--limit |--top ] [--kind ] [--visibility ] [--exclude-visibility ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--count] [--group-by ] [--group-by-name]"), ("suggestions", "cdidx suggestions [list|show|export|add|update|delete] [id|description] [--db ] [--json] [--description ] [--context ] [--title ] [--evidence-path ] [--status ] [--actor ] [--reason ] [--language ] [--category ] [--since ] [--agent ] [--limit ] [--offset ] [--format ] [--output ] [--overwrite] [--open-issues ] [--repo ] [--issue-state ] [--duplicate-confidence |--duplicate-threshold ]"), ("suggestions-list", "cdidx suggestions list [--status ] [--language ] [--category ] [--since ] [--agent ] [--limit ] [--offset ] [--db ] [--json]"), diff --git a/src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs b/src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs index dcd3afbaa..8f6077bf3 100644 --- a/src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs +++ b/src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs @@ -29,7 +29,7 @@ internal static partial class JsonEnvelopeWrapper private static readonly HashSet AutoWrapByteBudgetCommands = new(StringComparer.Ordinal) { - "find", "status", "references", "callers", "callees", "languages", "outline", + "find", "status", "references", "callers", "callees", "languages", "outline", "unused", }; private static readonly HashSet AutoWrapCompactCommands = new(StringComparer.Ordinal) @@ -46,19 +46,33 @@ internal static partial class JsonEnvelopeWrapper private static readonly HashSet PageableResponseCommands = new(StringComparer.Ordinal) { "search", "definition", "find", "hotspots", "references", "callers", "callees", - "symbols", "files", "languages", "impact", "map", "outline", + "symbols", "files", "languages", "impact", "map", "outline", "unused", }; private static readonly HashSet CountableResponseCommands = new(StringComparer.Ordinal) { "search", "definition", "find", "hotspots", "references", "callers", "callees", - "symbols", "files", "languages", "impact", + "symbols", "files", "languages", "impact", "unused", }; + private static readonly string[] UnusedCompactResponseFields = + [ + "path", + "line", + "lang", + "kind", + "name", + "visibility", + "container_name", + "unused_bucket", + "unused_confidence", + "unused_contract_domain", + ]; + internal static bool ShouldAutoWrapBoundedResponse(string command, string[] args) { if (!BoundedResponseCommands.Contains(command) - && !IsOutlineByteBudgetRequest(command, args)) + && !IsStandaloneWholeRowByteBudgetRequest(command, args)) return false; if (command == "search" && IsSearchAggregateResponseRequest(args)) return false; @@ -80,7 +94,7 @@ internal static bool ShouldAutoWrapBoundedResponse(string command, string[] args private static bool IsBoundedResponseRequest(string command, string[] args) { if (!BoundedResponseCommands.Contains(command) - && !IsOutlineByteBudgetRequest(command, args)) + && !IsStandaloneWholeRowByteBudgetRequest(command, args)) return false; if (command == "search" && IsSearchAggregateResponseRequest(args)) return false; @@ -92,12 +106,12 @@ private static bool IsBoundedResponseRequest(string command, string[] args) || ShouldAutoWrapBoundedResponse(command, args); } - private static bool IsOutlineByteBudgetRequest(string command, string[] args) - => command == "outline" + private static bool IsStandaloneWholeRowByteBudgetRequest(string command, string[] args) + => command is "outline" or "unused" && HasArgument(args, "--max-json-bytes"); - private static bool HasUnsupportedOutlineBoundedControl(string command, string[] args) - => command == "outline" + private static bool HasUnsupportedStandaloneBoundedControl(string command, string[] args) + => command is "outline" or "unused" && (HasArgument(args, "--fields") || HasArgument(args, "--format") || args.Any(arg => arg.StartsWith("--json=", StringComparison.Ordinal))); @@ -183,8 +197,15 @@ private static int RunBoundedResponse( { if (!TryParseBoundedResponseControls(command, args, out var controls, out var controlError)) return WriteBoundedResponseUsageError(controlError!, "Use the command help to pass positive --limit/--max-json-bytes values and a next_cursor returned by the same query."); - if (HasUnsupportedOutlineBoundedControl(command, args)) - return RunOutlineValidationWithinBudget(args, controls.MaxJsonBytes!.Value, runInner); + if (HasUnsupportedStandaloneBoundedControl(command, args) + || command == "unused" && HasArgument(args, "--summary-only")) + { + return RunStandaloneValidationWithinBudget( + command, + args, + controls.MaxJsonBytes!.Value, + runInner); + } if (ProjectionFieldRegistry.IsDiscoveryRequest(controls.Fields)) { var discoveryJson = ProjectionFieldRegistry.CreateDiscoveryDocument(command).ToJsonString(jsonOptions); @@ -373,17 +394,23 @@ private static int RunBoundedResponse( if (envelope is null) { - var fieldsOption = command == "outline" ? "--outline-fields" : "--fields"; + var selectionHint = command switch + { + "outline" => "choose fewer --outline-fields", + "unused" => "add --compact", + _ => "choose fewer --fields", + }; return WriteBoundedResponseUsageError( $"--max-json-bytes {controls.MaxJsonBytes} is too small for the bounded response metadata and one projected row.", - $"Increase --max-json-bytes or choose fewer {fieldsOption}."); + $"Increase --max-json-bytes or {selectionHint}."); } Console.WriteLine(emittedJson); return exitCode; } - private static int RunOutlineValidationWithinBudget( + private static int RunStandaloneValidationWithinBudget( + string command, string[] args, int maxJsonBytes, Func runInner) @@ -416,8 +443,8 @@ private static int RunOutlineValidationWithinBudget( } return WriteBoundedResponseUsageError( - message ?? "outline output-selector validation failed.", - $"{hint ?? "Use only options shown in `outline --help`."} Increase --max-json-bytes to receive the structured validation error."); + message ?? $"{command} output-selector validation failed or its summary document exceeds the byte budget.", + $"{hint ?? $"Use only options shown in `{command} --help`."} Increase --max-json-bytes to receive the structured response."); } private static JsonObject? BuildBoundedEnvelopeWithinBudget( @@ -545,7 +572,16 @@ JsonObject BuildCandidate(int count) if (extraction.PrimaryCollection is not null) metadata["primary_collection"] = extraction.PrimaryCollection; if (extraction.Context is { Count: > 0 }) - metadata["response_context"] = extraction.Context.DeepClone(); + { + metadata["response_context"] = command == "unused" + ? BuildBoundedUnusedResponseContext(extraction.Context, results) + : extraction.Context.DeepClone(); + } + if (command == "unused" + && extraction.SourcePayload?["by_bucket"] is JsonObject) + { + envelope["by_bucket"] = BuildBoundedUnusedBuckets(results); + } if (controls.Compact && LegacyLocationCompactCommands.Contains(command) && extraction.SourcePayload is not null) @@ -798,6 +834,8 @@ private static ResponseExtraction ExtractResponseItems(string command, JsonArray return ExtractNestedCollection(languagesPayload, "languages"); if (command == "outline" && rawResults.FirstOrDefault() is JsonObject outlinePayload) return ExtractOutlineSymbols(outlinePayload); + if (command == "unused" && rawResults.FirstOrDefault() is JsonObject unusedPayload) + return ExtractUnusedSymbols(unusedPayload); if (command == "impact" && rawResults.FirstOrDefault() is JsonObject impactPayload) { var requestedCollection = SelectRequestedCollection(controls.Fields, "callers", "file_impacts", "definitions"); @@ -866,6 +904,93 @@ private static ResponseExtraction ExtractOutlineSymbols(JsonObject payload) return extraction; } + private static ResponseExtraction ExtractUnusedSymbols(JsonObject payload) + { + var items = payload["symbols"] as JsonArray ?? []; + var context = new JsonObject(); + foreach (var property in payload) + { + if (property.Key is "symbols" or "by_bucket" or "next_cursor" or "result_stable_at" + || property.Value is JsonArray) + { + continue; + } + context[property.Key] = property.Value?.DeepClone(); + } + return new ResponseExtraction( + new JsonArray(items.Select(item => item?.DeepClone()).ToArray()), + "symbols", + context, + payload); + } + + private static JsonObject BuildBoundedUnusedBuckets(JsonArray results) + { + var byBucket = new JsonObject(); + foreach (var bucket in QueryCommandRunner.OrderedUnusedBuckets) + byBucket[bucket] = new JsonArray(); + foreach (var result in results.OfType()) + { + var bucket = ReadString(result, "unused_bucket"); + if (bucket is null || byBucket[bucket] is not JsonArray rows) + continue; + rows.Add(result.DeepClone()); + } + return byBucket; + } + + private static JsonObject BuildBoundedUnusedResponseContext(JsonObject source, JsonArray results) + { + var context = (JsonObject)source.DeepClone(); + context["count"] = results.Count; + context["returned_bucket_counts"] = CountResponseRowsByProperty( + results, + "unused_bucket", + QueryCommandRunner.OrderedUnusedBuckets); + context["returned_contract_domain_counts"] = CountResponseRowsByProperty( + results, + "unused_contract_domain"); + if (context["summary"] is JsonObject summary) + { + summary["by_bucket"] = CountResponseRowsByProperty( + results, + "unused_bucket", + QueryCommandRunner.OrderedUnusedBuckets); + summary["by_confidence"] = CountResponseRowsByProperty( + results, + "unused_confidence"); + summary["by_contract_domain"] = CountResponseRowsByProperty( + results, + "unused_contract_domain"); + } + return context; + } + + private static JsonObject CountResponseRowsByProperty( + JsonArray results, + string propertyName, + IReadOnlyList? preferredOrder = null) + { + var counts = results + .OfType() + .Select(row => ReadString(row, propertyName)) + .Where(value => !string.IsNullOrEmpty(value)) + .GroupBy(value => value!, StringComparer.Ordinal) + .ToDictionary(group => group.Key, group => group.Count(), StringComparer.Ordinal); + var output = new JsonObject(); + if (preferredOrder is not null) + { + foreach (var value in preferredOrder) + { + if (counts.Remove(value, out var count)) + output[value] = count; + } + } + foreach (var entry in counts.OrderBy(entry => entry.Key, StringComparer.Ordinal)) + output[entry.Key] = entry.Value; + return output; + } + private static ResponseExtraction ExtractDiscoveryRows(string command, JsonArray rawResults) { var rows = new JsonArray(); @@ -1698,6 +1823,9 @@ private sealed record BoundedResponseControls( { var preserveFullDiscoveryRows = command is "search" or "languages"; var selected = Fields + ?? (command == "unused" && Compact + ? UnusedCompactResponseFields + : null) ?? ((!preserveFullDiscoveryRows || Compact) && ProjectionFieldRegistry.GetCompactFields(command) is { } defaults ? defaults diff --git a/src/CodeIndex/Cli/QueryCommandRunner.Unused.cs b/src/CodeIndex/Cli/QueryCommandRunner.Unused.cs index 1b1470921..ca2e6c621 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.Unused.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.Unused.cs @@ -228,7 +228,8 @@ UnusedCountResult CountUnusedSymbolsDetailedForCurrentQuery(Func + { + "unused", "--db", dbPath, "--json", "--all", "--lang", "csharp", "--limit", "100", + }; + var (legacyExitCode, legacyStdout, legacyStderr) = CaptureConsole(() => + ProgramRunner.Run([.. legacyArgs], _jsonOptions, "1.0.0-test")); + + Assert.Equal(CommandExitCodes.Success, legacyExitCode); + Assert.Equal(string.Empty, legacyStderr); + using var legacyDocument = JsonDocument.Parse(legacyStdout); + Assert.False(legacyDocument.RootElement.TryGetProperty("metadata", out _)); + var expectedRows = legacyDocument.RootElement + .GetProperty("symbols") + .EnumerateArray() + .Select(ReadUnusedIdentityIssue4905) + .ToArray(); + Assert.Contains( + legacyDocument.RootElement.GetProperty("symbols").EnumerateArray(), + row => row.GetProperty("name").GetString() == "未使用方法00" + && row.GetProperty("signature").GetString()!.Contains("引数23", StringComparison.Ordinal)); + + var baseArgs = new List + { + "unused", "--db", dbPath, "--json", "--all", "--lang", "csharp", + "--limit", "100", "--max-json-bytes", UnusedPageByteBudgetIssue4905.ToString(), + }; + if (compact) + baseArgs.Add("--compact"); + if (byBucket) + baseArgs.Add("--by-bucket"); + + var actualRows = new List<(string Path, int Line, string Name)>(); + string? cursor = null; + var pageCount = 0; + do + { + var args = cursor is null + ? baseArgs.ToArray() + : baseArgs.Concat(["--cursor", cursor]).ToArray(); + var (exitCode, stdout, stderr) = CaptureConsole(() => + ProgramRunner.Run(args, _jsonOptions, "1.0.0-test")); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.True( + Encoding.UTF8.GetByteCount(stdout) <= UnusedPageByteBudgetIssue4905, + $"stdout exceeded {UnusedPageByteBudgetIssue4905} UTF-8 bytes."); + using var document = JsonDocument.Parse(stdout); + var root = document.RootElement; + var metadata = root.GetProperty("metadata"); + var results = root.GetProperty("results").EnumerateArray().ToArray(); + + pageCount++; + Assert.True(pageCount <= expectedRows.Length, "unused byte-budget cursor did not make forward progress."); + Assert.NotEmpty(results); + Assert.Equal("unused", metadata.GetProperty("command").GetString()); + Assert.Equal("symbols", metadata.GetProperty("primary_collection").GetString()); + Assert.Equal(expectedRows.Length, metadata.GetProperty("total_count").GetInt32()); + Assert.True(metadata.GetProperty("total_count_authoritative").GetBoolean()); + Assert.Equal(results.Length, metadata.GetProperty("returned_count").GetInt32()); + Assert.Equal( + results.Length, + metadata.GetProperty("response_context").GetProperty("count").GetInt32()); + Assert.Equal( + results.Length, + metadata + .GetProperty("response_context") + .GetProperty("returned_bucket_counts") + .EnumerateObject() + .Sum(property => property.Value.GetInt32())); + + if (compact) + { + Assert.Equal("compact", metadata.GetProperty("format").GetString()); + Assert.All(results, row => + { + Assert.False(row.TryGetProperty("signature", out _)); + Assert.False(row.TryGetProperty("unused_reason", out _)); + Assert.True(row.TryGetProperty("unused_bucket", out _)); + }); + } + else + { + Assert.All(results, row => Assert.True(row.TryGetProperty("signature", out _))); + } + + if (byBucket) + { + var flattened = root.GetProperty("by_bucket") + .EnumerateObject() + .SelectMany(property => property.Value.EnumerateArray()) + .Select(ReadUnusedIdentityIssue4905) + .ToArray(); + Assert.Equal(results.Select(ReadUnusedIdentityIssue4905).Order(), flattened.Order()); + } + else + { + Assert.False(root.TryGetProperty("by_bucket", out _)); + } + + actualRows.AddRange(results.Select(ReadUnusedIdentityIssue4905)); + cursor = metadata.GetProperty("next_cursor").GetString(); + if (cursor is not null) + Assert.StartsWith("response:v2:", cursor, StringComparison.Ordinal); + } + while (cursor is not null); + + Assert.True(pageCount > 1); + Assert.Equal(expectedRows, actualRows); + Assert.Equal(actualRows.Count, actualRows.Distinct().Count()); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void RunUnused_MaxJsonBytesHandlesMinimumEmptyExactBoundaryAndHelp_Issue4905() + { + var (projectRoot, dbPath) = CreateUnusedByteBudgetFixtureDbIssue4905(); + try + { + var (smallExitCode, smallStdout, smallStderr) = CaptureConsole(() => + ProgramRunner.Run( + [ + "unused", "--db", dbPath, "--json", "--all", "--lang", "csharp", + "--max-json-bytes", "64", + ], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, smallExitCode); + Assert.Equal(string.Empty, smallStdout); + Assert.Contains($"Error [{CommandErrorCodes.UsageError}]", smallStderr, StringComparison.Ordinal); + Assert.Contains("bounded response metadata and one projected row", smallStderr, StringComparison.Ordinal); + + var (emptyExitCode, emptyStdout, emptyStderr) = CaptureConsole(() => + ProgramRunner.Run( + [ + "unused", "--db", dbPath, "--json", "--all", "--lang", "csharp", + "--path", "does-not-exist/**", + "--max-json-bytes", UnusedPageByteBudgetIssue4905.ToString(), + ], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.Success, emptyExitCode); + Assert.Equal(string.Empty, emptyStderr); + Assert.True(Encoding.UTF8.GetByteCount(emptyStdout) <= UnusedPageByteBudgetIssue4905); + using (var emptyDocument = JsonDocument.Parse(emptyStdout)) + { + Assert.Empty(emptyDocument.RootElement.GetProperty("results").EnumerateArray()); + Assert.Equal(0, emptyDocument.RootElement.GetProperty("metadata").GetProperty("total_count").GetInt32()); + Assert.Null(emptyDocument.RootElement.GetProperty("metadata").GetProperty("next_cursor").GetString()); + } + + const string unicodeJson = """{"results":[{"name":"未使用猫"}]}"""; + var exactBudget = Encoding.UTF8.GetByteCount(unicodeJson) + + Encoding.UTF8.GetByteCount(Environment.NewLine); + Assert.True(JsonEnvelopeWrapper.JsonFitsResponseBudget(unicodeJson, exactBudget)); + Assert.False(JsonEnvelopeWrapper.JsonFitsResponseBudget(unicodeJson, exactBudget - 1)); + + var flag = Assert.Single( + CliFlagSchema.GetCompletionFlagsForCommand("unused"), + candidate => candidate.Name == "--max-json-bytes"); + Assert.Contains("Bound emitted JSON bytes", flag.Description, StringComparison.Ordinal); + var (printed, helpStdout, helpStderr) = CaptureConsole(() => + ConsoleUi.PrintCommandUsage("unused") ? 1 : 0); + Assert.Equal(1, printed); + Assert.Equal(string.Empty, helpStderr); + Assert.Contains("--max-json-bytes ", helpStdout, StringComparison.Ordinal); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void RunUnused_MaxJsonBytesBindsCursorToFiltersAndIndexGeneration_Issue4905() + { + var (projectRoot, dbPath) = CreateUnusedByteBudgetFixtureDbIssue4905(); + try + { + var baseArgs = new[] + { + "unused", "--db", dbPath, "--json", "--all", "--lang", "csharp", + "--limit", "1", "--max-json-bytes", "20000", + }; + var (firstExitCode, firstStdout, firstStderr) = CaptureConsole(() => + ProgramRunner.Run(baseArgs, _jsonOptions, "1.0.0-test")); + + Assert.Equal(CommandExitCodes.Success, firstExitCode); + Assert.Equal(string.Empty, firstStderr); + using var firstDocument = JsonDocument.Parse(firstStdout); + var cursor = firstDocument.RootElement + .GetProperty("metadata") + .GetProperty("next_cursor") + .GetString(); + Assert.NotNull(cursor); + + var (mismatchExitCode, mismatchStdout, mismatchStderr) = CaptureConsole(() => + ProgramRunner.Run( + baseArgs.Concat(["--bucket", "likely_unused_private", "--cursor", cursor!]).ToArray(), + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, mismatchExitCode); + Assert.Equal(string.Empty, mismatchStdout); + Assert.Contains("cursor_mismatch", mismatchStderr, StringComparison.Ordinal); + + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/GenerationChange.cs", + "csharp", + "internal sealed class GenerationChange { private void NewlyUnused() { } }"); + using (var db = new DbContext(DbOpenIntent.WriteIndex, dbPath)) + new DbWriter(db.Connection).MarkGraphReady(); + + var (staleExitCode, staleStdout, staleStderr) = CaptureConsole(() => + ProgramRunner.Run( + baseArgs.Concat(["--cursor", cursor!]).ToArray(), + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, staleExitCode); + Assert.Equal(string.Empty, staleStdout); + Assert.Contains("cursor_stale", staleStderr, StringComparison.Ordinal); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + private static (string ProjectRoot, string DbPath) CreateUnusedByteBudgetFixtureDbIssue4905() + { + var (projectRoot, dbPath) = CreateUnusedFixtureDb(); + var parameters = string.Join( + ", ", + Enumerable.Range(0, 24).Select(index => $"string 引数{index:D2}")); + var methods = string.Join( + Environment.NewLine, + Enumerable.Range(0, 10).Select(index => + $"private string 未使用方法{index:D2}({parameters}) => \"値{index:D2}猫\";")); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/UnicodeUnused.cs", + "csharp", + $$""" + namespace 世界; + internal sealed class UnicodeUnused + { + {{methods}} + } + """); + using var db = new DbContext(DbOpenIntent.WriteIndex, dbPath); + new DbWriter(db.Connection).MarkGraphReady(); + return (projectRoot, dbPath); + } + + private static (string Path, int Line, string Name) ReadUnusedIdentityIssue4905(JsonElement row) + => ( + row.GetProperty("path").GetString()!, + row.GetProperty("line").GetInt32(), + row.GetProperty("name").GetString()!); +}