diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 5fbd87bb0..8c95340a5 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -1685,6 +1685,7 @@ access. | Compact location envelope | CLI `--format compact` location output uses a versioned envelope with `api_version`, returned `count`, conservative limit-based `truncated` / `truncation` metadata, applied `query_context`, and lightweight `results` rows. | | Grouped search totals | `search --format grouped` derives `total_matches` / `matched_count`, `total_groups`, and `total_files` from the complete bounded query rather than the displayed page. `grouped_match_count` counts rows supplied to returned groups, `emitted_match_count` counts rows left after per-file grouping limits, and `omitted_match_count`, `truncated`, `has_more`, and `continuation_action` describe incomplete output. | | Bounded high-volume responses | `search`, `definition`, `find`, `status`, `hotspots`, `references`, `callers`, `callees`, `symbols`, `files`, `languages`, `impact`, and `map` accept shared bounded-response controls where their schema exposes them. Newly emitted opaque `--cursor ` values bind the offset to the command/query/filter selection and index generation; legacy `response:v1::` cursors remain accepted for transition. Reuse with changed selection or generation fails with restart-required guidance. `search --format compact`, `symbols --format compact`, and `files --format compact` auto-select the bounded contract, while `search --json=array --json-envelope` provides the opt-in array envelope and `languages --json` selects it when paging or `--max-json-bytes` is requested. Existing compact roots and location rows remain compatible while adding shared metadata. Metadata reports `returned_count`, authoritative `total_count` where available, `omitted_count`, `remaining_count`, `cursor_offset`, `page_limit`, `has_more`, `next_cursor`, `result_stable_at`, `pagination_window_limit`, and `pagination_window_exhausted`. The safety window is 10,000 rows; exhaustion suppresses `next_cursor` rather than returning a cursor that the next request would reject. Pageable commands pass the cursor offset into their database/scan layer instead of serializing an `offset + limit` prefix. `find --all` partial scans encode the next path/line in the opaque cursor so replay continues after the last scanned line. `hotspots` and `impact` page their active primary nested collection as `results`, identify it with `metadata.primary_collection`, and retain scalar/container evidence in `metadata.response_context`; dotted fields such as `callers.path,callers.depth` select that collection and project its rows. The final newline is included in `--max-json-bytes`, and trailing whole rows are removed until the complete envelope fits. `definition` remains metadata-only by default; explicit `--body` content is retained for `body`, `body_content`, or `all`, and suppressed when the projection excludes it. `map --sections` remains its section-level projection, while dotted bounded fields page a selected array section with section-specific totals and scalar projections skip unused ranked arrays. | +| Bounded outline responses | `outline` opts into the shared bounded-response contract only when `--max-json-bytes` is present. The wrapper extracts complete projected symbol rows, preserves hierarchy and deterministic order, reports authoritative returned / total / omitted counts, includes the final newline in its UTF-8 measurement, and emits a bound `response:v2` continuation cursor. An undersized minimum envelope produces typed `E010_USAGE_ERROR` diagnostics with empty stdout. Uncapped outline JSON retains its existing root shape and outline cursor contract. | | Bounded response edge cases | `impact` applies the cursor offset only to the selected nested collection so definition pages do not repeat or alter caller/fallback mode. Plain `map --compact` preserves its established section arrays and truncation payload; a collection projection is rejected when `--summary-only` or an excluding `--sections` filter would remove it. Explicit definition body fields override compact defaults. Profile and verbose records are moved into `metadata.stream_control_records`, and parser/capture failures emit an error envelope only when it fits the active hard byte cap. | | `--count --json` envelope | Count-only JSON for `search`, `definition`, `references`, `callers`, `callees`, `symbols`, `files`, `find`, `impact`, and `unused` is a single automation-oriented object. It always includes `count`, applied `query_context`, freshness metadata (`indexed_file_count`, `indexed_at`, `freshness_available`), and trust flags `degraded` / `authoritative_count`; commands with matched-file totals also include `files` and the older `file_count` compatibility alias. `file_count` carries the same value as `files`, remains for compatibility, and is not scheduled for removal before the next major release. `unused --count --json` also includes `returned_bucket_counts`, `returned_contract_domain_counts`, and `summary.by_bucket` / `summary.by_confidence` / `summary.by_contract_domain`. `authoritative_count=false` means a readiness or graph/exact trust signal made the count non-authoritative, while the freshness fields describe the indexed snapshot used for the count. | | Search row selection | Row-producing plain-search and recipe paths share `ApplySearchOutputSelection`: `--first-per-file` and fixed-seed deterministic `--sample` run before the effective per-query / remaining total limit. Sample fetch envelopes are sized from at least the requested sample target. Aggregate/compact query DTOs, plain compact roots, run summaries, issue-draft source DTOs, NDJSON terminals, and bounded array-envelope stream terminals expose `source_total`, `selected_total`, `returned`, `selector_omitted_count`, and `limit_omitted_count`; `source_total_authoritative` / `source_total_lower_bound` distinguish complete populations from bounded observations. Guard filters, origin/facet post-filters, exhausted candidate windows, and recipe file-reject post-filters force lower-bound authority. Their ordered `selectors` entries preserve each stage's input/output/omission counts plus sample size, mode, and seed, while nullable `selection_reason` / `selection_omitted_count` remain compatibility summaries. Bounded plain-search selection is computed once and its selected page is reused by compact/envelope serialization. Search `query_context.row_selectors` records the applied selector configuration. Selection-only omission updates matched/omitted lower bounds without setting `truncated`, `has_more`, or `next_cursor`. When a later limit truncates selected rows, limit truncation remains visible but `next_cursor` is suppressed because raw database cursors cannot preserve selector state; incoming `--cursor` values are rejected with either selector for the same reason. Generated compact and issue-draft replay commands retain the selector. Count, aggregation, named-query, recipe-list, results-only, metadata-free array, unsupported formatted, and summary-only compact shapes reject `--first-per-file` / `--sample`, while every recipe shape rejects grouped-only `--per-file-limit`. | @@ -5053,6 +5054,7 @@ help はすべてこのレジストリを参照します。field 名は大文字 | compact location envelope | CLI の `--format compact` location output は、`api_version`、返却 `count`、limit 到達を基準にした保守的な `truncated` / `truncation` metadata、適用済み `query_context`、軽量な `results` row を持つ versioned envelope です。 | | grouped search の総数 | `search --format grouped` の `total_matches` / `matched_count`、`total_groups`、`total_files` は、表示 page ではなく上限適用前の query 全体から算出します。`grouped_match_count` は返却 group に渡した row 数、`emitted_match_count` は file ごとの上限適用後に残った row 数を表し、`omitted_match_count`、`truncated`、`has_more`、`continuation_action` が未完了出力を示します。 | | 高ボリューム応答の bounded 契約 | `search`、`definition`、`find`、`status`、`hotspots`、`references`、`callers`、`callees`、`symbols`、`files`、`languages`、`impact`、`map` は、それぞれの schema が公開する共通 bounded-response control に対応します。新しく発行する opaque な `--cursor ` は offset を command / query / filter と index generation に束縛し、移行用に legacy の `response:v1::` も受理します。選択条件または generation を変えて再利用すると restart-required の案内付きで失敗します。`search --format compact`、`symbols --format compact`、`files --format compact` は bounded 契約を自動選択し、`search --json=array --json-envelope` は opt-in の array envelope、`languages --json` は paging または `--max-json-bytes` 指定時に同じ契約を使います。既存 compact の root と location row は維持したまま共通 metadata を追加します。metadata は `returned_count`、取得可能な場合は authoritative な `total_count`、`omitted_count`、`remaining_count`、`cursor_offset`、`page_limit`、`has_more`、`next_cursor`、`result_stable_at`、`pagination_window_limit`、`pagination_window_exhausted` を返します。safety window は 10,000 row で、上限到達時は次の request が拒否する cursor を返さず `next_cursor` を抑止します。pageable command は `offset + limit` 件を serialize せず、cursor offset を database / scan layer へ渡します。`find --all` の partial scan cursor は次の path / line を保持し、再利用時は最後に scan した line の次から継続します。`hotspots` と `impact` は active な主要 nested collection を `results` としてページングし、`metadata.primary_collection` でその名前を示し、scalar / container evidence は `metadata.response_context` に保持します。`callers.path,callers.depth` のような dotted field で collection と row field を同時に選べます。`--max-json-bytes` は最後の改行を含み、完全な envelope が収まるまで末尾の完全な row を省略します。`definition` は既定で metadata-only のままで、明示的な `--body` は `body`、`body_content`、`all` で保持し、それ以外の projection では materialize 前に抑止します。`map --sections` は section-level projection として残り、dotted な bounded field は選択した array section を section 固有の総件数付きでページングし、scalar projection は不要な ranking array を構築しません。 | +| bounded outline 応答 | `outline` は `--max-json-bytes` がある場合だけ共通 bounded-response 契約を選択します。wrapper は projection 済みの完全な symbol row を抽出し、階層と決定的な順序を維持し、authoritative な返却 / 総 / 省略件数を報告します。UTF-8 計測には最後の改行を含め、束縛済みの `response:v2` continuation cursor を発行します。最小 envelope が収まらない場合は stdout を空に保ち、型付きの `E010_USAGE_ERROR` diagnostic を返します。上限なしの outline JSON は既存の root shape と outline cursor 契約を維持します。 | | bounded 応答の edge case | `impact` は選択された nested collection だけに cursor offset を適用するため、definition page の重複や caller / fallback mode の変化を防ぎます。通常の `map --compact` は既存の section array と truncation payload を維持し、collection projection が `--summary-only` または除外する `--sections` filter で失われる組み合わせは拒否します。明示的な definition body field は compact default より優先します。profile / verbose record は `metadata.stream_control_records` へ移し、parser / capture failure の error envelope は active な hard byte cap に収まる場合だけ出力します。 | | `--count --json` envelope | `search`、`definition`、`references`、`callers`、`callees`、`symbols`、`files`、`find`、`impact`、`unused` の count-only JSON は単一の自動化向け object です。常に `count`、適用済み `query_context`、freshness metadata（`indexed_file_count`、`indexed_at`、`freshness_available`）、trust flag の `degraded` / `authoritative_count` を含みます。matched-file total を持つ command は `files` と古い互換 alias の `file_count` も含みます。`file_count` は `files` と同じ値を持つ互換用 field として残り、少なくとも次の major release までは削除予定はありません。`unused --count --json` は `returned_bucket_counts`、`returned_contract_domain_counts`、`summary.by_bucket` / `summary.by_confidence` / `summary.by_contract_domain` も含みます。`authoritative_count=false` は readiness または graph/exact trust signal により count が authoritative ではないことを示し、freshness field は count に使った index snapshot を説明します。 | | search row selection | row を返す plain search / recipe path は `ApplySearchOutputSelection` を共有し、`--first-per-file` と固定 seed の決定的な `--sample` を、有効な query ごとの limit / 残り total limit より先に適用します。sample 用 fetch envelope は少なくとも要求 sample 数を基準に sizing します。aggregate / compact の query DTO、plain compact root、run summary、issue-draft の source DTO、NDJSON terminal、bounded array envelope の stream terminal は `source_total`、`selected_total`、`returned`、`selector_omitted_count`、`limit_omitted_count` を公開し、`source_total_authoritative` / `source_total_lower_bound` で完全な population と bounded な観測を区別します。guard filter、origin / facet の後段 filter、candidate window の枯渇、recipe の file-reject 後段 filter は lower-bound authority にします。適用順の `selectors` entry は各段階の input / output / omission count と sample の size / mode / seed を保持し、nullable な `selection_reason` / `selection_omitted_count` は互換用 summary として維持します。bounded plain-search selection は一度だけ計算し、その selected page を compact / envelope serialize で再利用します。search の `query_context.row_selectors` は適用済み selector 設定を記録します。selection だけによる省略は matched / omitted の lower bound を更新しますが、`truncated`、`has_more`、`next_cursor` は設定しません。後続の limit が選択済み row を truncate する場合、limit truncation は表示しますが raw database cursor は selector state を保持できないため `next_cursor` を抑止し、同じ理由で selector と受け取った `--cursor` の併用も拒否します。compact / issue-draft の生成 replay command は selector を保持します。count、aggregation、named-query、recipe-list、results-only、metadata を持たない array、非対応 formatted、summary-only compact の shape は `--first-per-file` / `--sample` を拒否し、すべての recipe shape は grouped 専用の `--per-file-limit` を拒否します。 | diff --git a/README.md b/README.md index 856c0ccb8..c41a4364d 100644 --- a/README.md +++ b/README.md @@ -76,7 +76,7 @@ cdidx search "TODO" --first-per-file --sample 25 --json=ndjson --max-json-bytes cdidx definition UserService cdidx references UserService --fields path,line,reference_kind --limit 20 --max-json-bytes 16384 cdidx inspect QueryCommandRunner --outline-only -cdidx outline src/CodeIndex/Cli/QueryCommandRunner.cs --compact --kind function --sort size --limit 10 +cdidx outline src/CodeIndex/Cli/QueryCommandRunner.cs --json --kind function --limit 50 --max-json-bytes 16384 cdidx unused --compact --by-bucket cdidx map --compact --max-json-bytes 65536 cdidx map --format issue-drafts --limit 10 @@ -110,7 +110,7 @@ Ad-hoc `search --format issue-drafts` computes `source_total_count`, `returned_c When the byte cap omits rows, these commands return partial-result exit code `11`; pass `--allow-partial` to opt into exit code `0` while retaining the same terminal metadata. Ordinary `--limit` truncation remains a successful, explicitly described stream. Array and compact outputs keep their documented whole-response behavior; check `cdidx --help` before relying on partial output. -High-volume `definition`, `find`, `status`, `hotspots`, `references`, `callers`, `callees`, `impact`, and `map` responses also support an opt-in bounded envelope through `--fields`, `--cursor`, compact output where advertised, and a total `--max-json-bytes` budget. Its metadata reports returned/total/omitted counts and an opaque `next_cursor`; replay that cursor with the same query, filters, and sort arguments. The response also exposes its 10,000-row safety window and reports when that window is exhausted instead of emitting an unusable cursor. Existing compact location responses retain their top-level keys and lightweight `file` / `line` rows while adding the shared metadata; `refs` / `stats` aliases and matching read-only `batch` children use the same envelope and hard cap. `hotspots` and `impact` page their active primary collection, while dotted projections such as `callers.path,callers.depth` select nested rows and report that collection's total. `map --sections` selects whole response sections; a bounded projection such as `--fields top_files.path` instead pages that section's rows and avoids building unrelated ranked sections. For `definition --body`, `body`, `body_content`, and `all` retain the explicit body; projections that exclude it avoid materializing body text. +High-volume `definition`, `find`, `status`, `hotspots`, `references`, `callers`, `callees`, `impact`, `map`, and `outline` responses also support an opt-in bounded envelope through `--fields`, `--cursor`, compact output where advertised, and a total `--max-json-bytes` budget. Its metadata reports returned/total/omitted counts and an opaque `next_cursor`; replay that cursor with the same query, filters, and sort arguments. The response also exposes its 10,000-row safety window and reports when that window is exhausted instead of emitting an unusable cursor. Existing compact location responses retain their top-level keys and lightweight `file` / `line` rows while adding the shared metadata; `refs` / `stats` aliases and matching read-only `batch` children use the same envelope and hard cap. `hotspots` and `impact` page their active primary collection, while dotted projections such as `callers.path,callers.depth` select nested rows and report that collection's total. `map --sections` selects whole response sections; a bounded projection such as `--fields top_files.path` instead pages that section's rows and avoids building unrelated ranked sections. For `definition --body`, `body`, `body_content`, and `all` retain the explicit body; projections that exclude it avoid materializing body text. `outline --max-json-bytes` pages complete UTF-8 symbol rows, including the final newline, and keeps hierarchy and ordering stable while honoring `--outline-fields`; uncapped outline JSON retains its established shape and cursor contract. Bounded `map --compact` keeps the established top-level section arrays and compact truncation data while adding shared metadata. A map collection projection is rejected when `--summary-only` or an excluding `--sections` selection would remove that collection. Diagnostic `--profile` / `--verbose` objects are retained as metadata control records rather than projected rows, and every hard byte cap also applies to parser or capture error output; stdout stays empty when even the bounded error envelope cannot fit. @@ -510,7 +510,7 @@ cdidx search "TODO" --first-per-file --sample 25 --json=ndjson --max-json-bytes cdidx definition UserService cdidx references UserService --fields path,line,reference_kind --limit 20 --max-json-bytes 16384 cdidx inspect QueryCommandRunner --outline-only -cdidx outline src/CodeIndex/Cli/QueryCommandRunner.cs --compact --kind function --sort size --limit 10 +cdidx outline src/CodeIndex/Cli/QueryCommandRunner.cs --json --kind function --limit 50 --max-json-bytes 16384 cdidx unused --compact --by-bucket cdidx map --compact --max-json-bytes 65536 cdidx map --format issue-drafts --limit 10 @@ -544,7 +544,7 @@ ad-hoc の `search --format issue-drafts` は、filter 済み母集団全体か byte cap により行を省略した場合、これらのコマンドは partial-result 終了コード `11` を返します。同じ終端 metadata を維持したまま終了コード `0` を明示的に許容するには `--allow-partial` を指定します。通常の `--limit` による切り詰めは、理由が明示された成功 stream のままです。array / compact 出力は文書化済みの whole-response 挙動を維持します。部分出力へ依存する前に `cdidx --help` を確認してください。 -高ボリュームな `definition`、`find`、`status`、`hotspots`、`references`、`callers`、`callees`、`impact`、`map` の応答は、`--fields`、`--cursor`、対応 command の compact 出力、応答全体に対する `--max-json-bytes` により opt-in の bounded envelope も利用できます。metadata は返却 / 総 / 省略件数と opaque な `next_cursor` を返します。次ページでは同じ query、filter、sort 引数とともにその cursor を再利用してください。応答は 10,000 row の safety window も公開し、上限到達時には利用不能な cursor を返さず、window の消費完了を報告します。既存の compact location 応答はトップレベル key と軽量な `file` / `line` row を維持したまま共通 metadata を追加し、`refs` / `stats` alias と対応する read-only `batch` 子 command にも同じ envelope と hard cap を適用します。`hotspots` と `impact` は active な主要 collection をページングし、`callers.path,callers.depth` のような dotted projection は nested row とその collection の総件数を返します。`map --sections` は section 全体を選びますが、`--fields top_files.path` のような bounded projection はその section の row をページングし、無関係な ranking section を構築しません。`definition --body` では `body`、`body_content`、`all` が明示的な body を保持し、body を除外する projection では本文を取得しません。 +高ボリュームな `definition`、`find`、`status`、`hotspots`、`references`、`callers`、`callees`、`impact`、`map`、`outline` の応答は、`--fields`、`--cursor`、対応 command の compact 出力、応答全体に対する `--max-json-bytes` により opt-in の bounded envelope も利用できます。metadata は返却 / 総 / 省略件数と opaque な `next_cursor` を返します。次ページでは同じ query、filter、sort 引数とともにその cursor を再利用してください。応答は 10,000 row の safety window も公開し、上限到達時には利用不能な cursor を返さず、window の消費完了を報告します。既存の compact location 応答はトップレベル key と軽量な `file` / `line` row を維持したまま共通 metadata を追加し、`refs` / `stats` alias と対応する read-only `batch` 子 command にも同じ envelope と hard cap を適用します。`hotspots` と `impact` は active な主要 collection をページングし、`callers.path,callers.depth` のような dotted projection は nested row とその collection の総件数を返します。`map --sections` は section 全体を選びますが、`--fields top_files.path` のような bounded projection はその section の row をページングし、無関係な ranking section を構築しません。`definition --body` では `body`、`body_content`、`all` が明示的な body を保持し、body を除外する projection では本文を取得しません。`outline --max-json-bytes` は最後の改行を含む完全な UTF-8 symbol row 単位でページングし、`--outline-fields` を尊重しながら階層と順序を維持します。上限なしの outline JSON は従来の形状と cursor 契約を維持します。 bounded な `map --compact` は、共通 metadata を追加しながら既存のトップレベル section array と compact truncation data を維持します。map collection projection と `--summary-only`、またはその collection を除外する `--sections` の組み合わせは拒否します。`--profile` / `--verbose` の diagnostic object は projected row ではなく metadata の control record として保持し、parser / capture error 出力にも hard byte cap を適用します。bounded error envelope 自体が収まらない場合、stdout は空のままです。 diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 811a6f3a1..1f58db756 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -307,7 +307,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding Search language-alias coverage may place distinct language files in one database and iterate alias filters when each filter isolates one expected result. Named-query escaping for option-looking literals reuses one indexed Probe fixture across definition, graph, symbols, files, inspect, and impact commands. Multi named-query output coverage reuses one indexed fixture for compact projection, rich JSON compatibility, per-query limits/truncation, and UTF-8 byte caps so the serializer modes stay directly comparable. - Shared bounded-response coverage reuses one graph-ready database across definition, find, status, hotspots, references, callers/callees, impact, and map; keep cursor and UTF-8 byte-budget boundary cases in a separate minimal multi-row fixture so family parity does not multiply indexing setup. Regression coverage must also exercise aliases and read-only batch dispatch, explicit definition body projections, inactive impact collections, and row-wise map-section pagination with authoritative totals. + Shared bounded-response coverage reuses one graph-ready database across definition, find, status, hotspots, references, callers/callees, impact, and map; keep cursor and UTF-8 byte-budget boundary cases in a separate minimal multi-row fixture so family parity does not multiply indexing setup. Outline keeps one focused deep-hierarchy fixture with long signatures and Unicode to verify exact newline-inclusive byte boundaries, full cursor walks without gaps or duplicates, minimum-budget diagnostics, and unchanged uncapped output. Regression coverage must also exercise aliases and read-only batch dispatch, explicit definition body projections, inactive impact collections, and row-wise map-section pagination with authoritative totals. Adversarial bounded-response coverage must also lock parser-failure byte caps, impact definition-page offsets, legacy map compact sections, conflicting map shape controls, compact explicit bodies, and profile/verbose control-record extraction. Search alias variants for JavaScript extensions, YAML, batch, and SQL dialects each reuse one language fixture and iterate casing/spelling forms in a fact. Raw FTS syntax coverage reuses one indexed source for a valid control query and all invalid query/hint variants. @@ -1235,7 +1235,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" option風literalのnamed-query escapingは、definition、graph、symbols、files、inspect、impact command全体で1つのindexed Probe fixtureを再利用してください。 impact cycle の回帰 coverage では、同じ表示名が連続する別 symbol を正規 source/target ID で区別し、構造化 shortest-path identity を検証し、未解決の上流 caller と一意でない resolved overload group を正規 cycle graph からだけ除外し、曖昧な path root に推測 ID を付けず、複数 target identity を過少計上せず集約するとともに、直接 singleton 再帰と複数 node cycle の control を維持してください。 複数 named-query の output coverage は、compact projection、rich JSON 互換性、query ごとの limit / truncation、UTF-8 byte cap に1つの indexed fixture を再利用し、serializer mode を直接比較できるようにしてください。 - 共通 bounded-response coverage は、definition、find、status、hotspots、references、callers / callees、impact、map 全体で1つの graph-ready databaseを再利用してください。cursor と UTF-8 byte-budget の境界 case は別の最小 multi-row fixture にまとめ、family parity のために indexing setup を重複させないでください。regression coverage では alias と read-only batch dispatch、明示的な definition body projection、inactive な impact collection、authoritative な総件数を持つ map section の row 単位 pagination も確認してください。 + 共通 bounded-response coverage は、definition、find、status、hotspots、references、callers / callees、impact、map 全体で1つの graph-ready databaseを再利用してください。cursor と UTF-8 byte-budget の境界 case は別の最小 multi-row fixture にまとめ、family parity のために indexing setup を重複させないでください。outline は、長い signature と Unicode を含む深い階層の focused fixture 1つを使い、最後の改行を含む正確な byte 境界、欠落や重複のない cursor 全 page 走査、最小 budget の diagnostic、上限なし出力の非変更を確認してください。regression coverage では alias と read-only batch dispatch、明示的な definition body projection、inactive な impact collection、authoritative な総件数を持つ map section の row 単位 pagination も確認してください。 adversarial な bounded-response coverage では、parser failure の byte cap、impact definition page の offset、既存 map compact section、map shape control の競合、compact と明示 body の組み合わせ、profile / verbose control record の抽出も固定してください。 JavaScript extension、YAML、batch、SQL dialectのsearch alias variantは、それぞれ1つのlanguage fixtureを再利用し、casing/spelling形式をfact内で反復してください。 raw FTS syntax coverage はvalid control queryと全invalid query/hint variantで1つのindexed sourceを再利用してください。 diff --git a/USER_GUIDE.md b/USER_GUIDE.md index 326243213..c025f12f1 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -1749,13 +1749,14 @@ Grouped `callees` rows preserve the earliest precise call site separately from t cdidx outline src/CodeIndex/Cli/GitHelper.cs cdidx outline src/CodeIndex/Cli/GitHelper.cs --json cdidx outline src/CodeIndex/Cli/GitHelper.cs --json --kind function --limit 20 --outline-fields name,line,kind,signature +cdidx outline src/CodeIndex/Cli/GitHelper.cs --json --kind function --limit 50 --max-json-bytes 16384 --outline-fields name,line,kind,signature cdidx outline src/CodeIndex/Cli/GitHelper.cs --json --cursor "$NEXT_CURSOR" --limit 20 --outline-fields name,line,kind,signature cdidx outline src/CodeIndex/Cli/QueryCommandRunner.cs --compact --kind function --sort size --limit 10 ``` Shows all symbols in a single file ordered deterministically by line, start column when available, kind, and name, with signature, visibility, and container nesting. Lets AI agents understand file structure in one call instead of reading the whole file or chaining `symbols` + `definition`. -For large files, `outline --json` supports `--kind `, `--sort `, `--limit` / `--top`, opaque `--cursor `, and `--outline-fields ` so automation can request only the symbol page and fields it needs. Use `--sort size` (alias `span`) or `--sort complexity` to jump to large bodies first, and combine it with `--compact` for bounded giant-file triage. Controlled JSON output includes `total_symbol_count`, `returned_symbol_count`, `cursor_offset`, `next_cursor`, `has_more`, and `result_stable_at`; it also reports `sort`, `kind_filter`, and `selected_fields` when those controls are used. The cursor is bound to the file path, filters, ordering, and index generation, so changing them or refreshing the index requires restarting without `--cursor`. Pass `--outline-fields all` to keep the full symbol payload while still opting into the paging metadata, or select `reference_count`, `size_lines`, `complexity_score`, and `sort_mode` for compact ranking evidence. +For large files, `outline --json` supports `--kind `, `--sort `, `--limit` / `--top`, opaque `--cursor `, `--max-json-bytes `, and `--outline-fields ` so automation can request only the symbol page and fields it needs. Use `--sort size` (alias `span`) or `--sort complexity` to jump to large bodies first, and combine it with `--compact` for bounded giant-file triage. With `--max-json-bytes`, outline returns the shared bounded envelope, counts complete UTF-8 symbol rows plus the final newline, and uses an opaque `response:v2` continuation cursor. If even the minimum envelope cannot fit, it writes no stdout and reports typed `E010_USAGE_ERROR` diagnostics. Without a byte cap, the existing outline JSON shape and cursor contract remain unchanged. Controlled uncapped JSON output includes `total_symbol_count`, `returned_symbol_count`, `cursor_offset`, `next_cursor`, `has_more`, and `result_stable_at`; it also reports `sort`, `kind_filter`, and `selected_fields` when those controls are used. The cursor is bound to the file path, filters, ordering, and index generation, so changing them or refreshing the index requires restarting without `--cursor`. Pass `--outline-fields all` to keep the full symbol payload while still opting into the paging metadata, or select `reference_count`, `size_lines`, `complexity_score`, and `sort_mode` for compact ranking evidence. ### Reconstruct a file excerpt @@ -2014,7 +2015,7 @@ same source location. | `--query ` | `search`, `definition`, `references`, `callers`, `callees`, `symbols`, `files`, `find`, `inspect`, `impact` | Pass a query literal explicitly, useful when the query starts with `-`. Query commands except `find` also accept `-- ` as a one-token query escape while continuing to parse later options. | | `--recipe ` | `search` | Run a reusable audit recipe such as `risky-code`, `json-parse-apis`, `dotnet-risk-patterns`, `unsupported-operation-boundaries`, `nullable-contracts`, `xml-parser-security`, `filesystem-traversal`, `bounded-read-evidence`, `resource-materialization-audit`, or `concurrency-state-audit`. Use `recipe/query` form, such as `risky-code/raw-diagnostic-echo`, to run one child query directly. An unknown recipe is compared with recipe names; an unknown child query is compared only with canonical names and aliases from the active recipe, and its safely quoted replay preserves that recipe and normalized filters. Recipe runs default to `--audit-scope source`, applying recipe production-code path and exclusion metadata before normal search filters and snippet controls; `--limit` / `--top` is per child query. Text, `--json` / `--format json`, `--format compact`, `--format sarif`, and `--format issue-drafts` are supported, and issue drafts include a replay command. | | `--include-query ` / `--exclude-query ` | `search --recipe ` | Include or exclude child recipe queries by name. Repeatable and comma-separated; names are listed by `cdidx search --list-recipes`. | -| `--cursor ` | `search --recipe `, `outline`, `unused` | Fetch the next page for one selected recipe child query, outline result, or unused-symbol page. Use the `next_cursor` returned by the previous JSON or compact output; outline cursors use `outline:`. | +| `--cursor ` | `search --recipe `, `outline`, `unused` | Fetch the next page for one selected recipe child query, outline result, or unused-symbol page. Use the `next_cursor` returned by the previous JSON or compact output; uncapped outline cursors use `outline:`, while `outline --max-json-bytes` returns an opaque `response:v2` cursor. | | `--audit-scope ` | `search`, `unused` | Choose audit path scope. For recipe search, `source` applies recipe production-code path and exclusion metadata. For ad hoc and named-query searches, `source` adds `src/**` when no user path was supplied, and applies default doc/test/changelog exclusions, `--exclude-tests`, and default comment / CLI help-text origin exclusions. `all` intentionally searches every indexed path unless other filters exclude it. JSON output reports the effective scope, path filters, and exclusions where applicable. | | `--source-only` | `search` | Shorthand for `--audit-scope source` on ad hoc and named searches. Use it for implementation-code searches without selecting a recipe. It also excludes comment and CLI help-text origins by default; use `--origin comment` or `--origin help_text` when those documentation-like matches are intentional evidence. | | `--show-excluded` | `search --recipe ` | Include `scope.excluded_diagnostics` in recipe output so broad audits can see which default include patterns, default exclusions, user exclusions, and test filtering were applied. | @@ -2032,7 +2033,7 @@ same source location. | `--search-fields ` | `search` | Project compact JSON fields, including recipe `query_name` and `recipe` | | `--results-only` | `search`, `symbols`, `files` | Emit result-only NDJSON without a stream terminal record for shell pipelines | | `--first-per-file` / `--sample ` / `--total-limit ` | `search` | Bound broad audit output by file, deterministic sample size, or recipe total rows | -| `--max-json-bytes ` | `search`, `definition`, `recipes`, `audit`, `deps`, `hotspots` | Fail before emitting JSON that exceeds this UTF-8 byte budget. A `definition --json` miss preflights its structured not-found object against the same cap and reports a usage error on stderr without oversized stdout when the object cannot fit. For large graph outputs, pair the cap with `deps --summary-only`, `deps --format json-graph --summary-only`, or `hotspots --summary-only`. | +| `--max-json-bytes ` | `search`, `definition`, `recipes`, `audit`, `deps`, `hotspots`, `outline` | Fail before emitting JSON that exceeds this UTF-8 byte budget. Bounded `outline` emits only complete symbol rows in the shared envelope, exposes authoritative returned / total / omitted counts and an opaque continuation cursor, and reports a typed usage error with empty stdout when the minimum envelope cannot fit. A `definition --json` miss preflights its structured not-found object against the same cap and reports a usage error on stderr without oversized stdout when the object cannot fit. For large graph outputs, pair the cap with `deps --summary-only`, `deps --format json-graph --summary-only`, or `hotspots --summary-only`. | | `--next-steps` | `search` | Emit inspect/excerpt follow-up commands for top search hits | | `--include-generated` | `search`, `definition`, `references`, `callers`, `callees`, `symbols`, `files`, `find`, `map`, `inspect`, `validate`, `deps`, `impact`, `unused`, `hotspots` | Include files detected as generated code; generated files are excluded from query results by default | | `--workspace-db ` | `deps` | Add another CodeIndex database to the file-dependency query. Repeat it for up to 7 distinct additional DBs (8 total including `--db`); JSON edges include `source_db` and `target_db` so same relative paths can be disambiguated. | @@ -4972,13 +4973,14 @@ C# の indexing は member 名が一般的な場合でも、`int.Parse`、LINQ cdidx outline src/CodeIndex/Cli/GitHelper.cs cdidx outline src/CodeIndex/Cli/GitHelper.cs --json cdidx outline src/CodeIndex/Cli/GitHelper.cs --json --kind function --limit 20 --outline-fields name,line,kind,signature +cdidx outline src/CodeIndex/Cli/GitHelper.cs --json --kind function --limit 50 --max-json-bytes 16384 --outline-fields name,line,kind,signature cdidx outline src/CodeIndex/Cli/GitHelper.cs --json --cursor "$NEXT_CURSOR" --limit 20 --outline-fields name,line,kind,signature cdidx outline src/CodeIndex/Cli/QueryCommandRunner.cs --compact --kind function --sort size --limit 10 ``` 1ファイル内の全シンボルを行、利用可能な場合は開始列、種別、名前の決定的な順序で、シグネチャ・可視性・コンテナ深さに応じたネスト付きで表示します。ファイル全体を読んだり `symbols` + `definition` をチェーンしたりする代わりに、1回でファイル構造を把握できます。 -大きなファイル向けに、`outline --json` は `--kind `、`--sort `、`--limit` / `--top`、opaque な `--cursor `、`--outline-fields ` に対応します。自動化側は必要なシンボルページとフィールドだけを取得できます。`--sort size`（`span` alias）や `--sort complexity` を使うと大きい本体を先に確認でき、`--compact` と組み合わせると巨大ファイル調査向けの上限付きペイロードになります。制御付き JSON 出力には `total_symbol_count`、`returned_symbol_count`、`cursor_offset`、`next_cursor`、`has_more`、`result_stable_at` が入り、sort、kind、field を指定した場合は `sort`、`kind_filter`、`selected_fields` も返します。cursor は file path、filter、ordering、index generation に束縛されるため、それらを変更した場合や index を更新した場合は `--cursor` なしで再開してください。`--outline-fields all` を渡すと、シンボルペイロードはフルのままページングメタデータだけを追加できます。ランキング根拠だけが必要な場合は `reference_count`、`size_lines`、`complexity_score`、`sort_mode` を選択できます。 +大きなファイル向けに、`outline --json` は `--kind `、`--sort `、`--limit` / `--top`、opaque な `--cursor `、`--max-json-bytes `、`--outline-fields ` に対応します。自動化側は必要なシンボルページとフィールドだけを取得できます。`--sort size`（`span` alias）や `--sort complexity` を使うと大きい本体を先に確認でき、`--compact` と組み合わせると巨大ファイル調査向けの上限付きペイロードになります。`--max-json-bytes` を指定すると、outline は共通 bounded envelope を返し、最後の改行を含む完全な UTF-8 symbol row 単位で計測して opaque な `response:v2` continuation cursor を使用します。最小 envelope さえ収まらない場合は stdout を空に保ち、型付きの `E010_USAGE_ERROR` diagnostic を報告します。byte cap がない場合、既存の outline JSON 形状と cursor 契約は変わりません。上限なしの制御付き JSON 出力には `total_symbol_count`、`returned_symbol_count`、`cursor_offset`、`next_cursor`、`has_more`、`result_stable_at` が入り、sort、kind、field を指定した場合は `sort`、`kind_filter`、`selected_fields` も返します。cursor は file path、filter、ordering、index generation に束縛されるため、それらを変更した場合や index を更新した場合は `--cursor` なしで再開してください。`--outline-fields all` を渡すと、シンボルペイロードはフルのままページングメタデータだけを追加できます。ランキング根拠だけが必要な場合は `reference_count`、`size_lines`、`complexity_score`、`sort_mode` を選択できます。 ### ファイル抜粋を再構成する @@ -5221,7 +5223,7 @@ raw match density を正確に測る、といった理由で全 raw chunk hit | `--query ` | `search`, `definition`, `references`, `callers`, `callees`, `symbols`, `files`, `find`, `inspect`, `impact` | クエリを明示的なリテラルとして渡す。クエリが `-` で始まる場合に有用。`find` 以外のクエリ系コマンドでは `-- ` も1トークンのクエリエスケープとして受け付け、その後のオプション解析を続ける。 | | `--recipe ` | `search` | `risky-code`、`json-parse-apis`、`dotnet-risk-patterns`、`unsupported-operation-boundaries`、`nullable-contracts`、`xml-parser-security`、`filesystem-traversal`、`bounded-read-evidence`、`resource-materialization-audit`、`concurrency-state-audit` などの再利用可能な audit recipe を実行する。`risky-code/raw-diagnostic-echo` のような `recipe/query` 形式で child query を1つだけ直接実行できる。未知の recipe は recipe 名と比較し、未知の child query は active recipe 内の canonical 名と alias だけを比較する。安全に引用された再実行コマンドは同じ recipe と正規化済み filter を保持する。Recipe 実行は既定で `--audit-scope source` になり、recipe の本番コード向け path / exclusion metadata を適用したうえで、通常の search filter と snippet control を選択された各 query に適用する。`--limit` / `--top` は child query ごとの上限になる。text、`--json` / `--format json`、`--format compact`、`--format sarif`、`--format issue-drafts` に対応し、issue draft には再実行コマンドを含める。 | | `--include-query ` / `--exclude-query ` | `search --recipe ` | recipe 内の child query を名前で含める、または除外する。繰り返し指定とカンマ区切りに対応し、名前は `cdidx search --list-recipes` で確認できる。 | -| `--cursor ` | `search --recipe `、`outline`、`unused` | 選択した recipe child query、outline 結果、unused-symbol page の次ページを取得する。直前の JSON または compact output が返す `next_cursor` を指定し、outline cursor は `outline:` 形式を使う。 | +| `--cursor ` | `search --recipe `、`outline`、`unused` | 選択した recipe child query、outline 結果、unused-symbol page の次ページを取得する。直前の JSON または compact output が返す `next_cursor` を指定する。上限なしの outline cursor は `outline:` 形式を使い、`outline --max-json-bytes` は opaque な `response:v2` cursor を返す。 | | `--audit-scope ` | `search`, `unused` | audit path scope を選ぶ。Recipe search の `source` は recipe の本番コード向け path / exclusion metadata を適用する。Ad hoc / named-query search の `source` は user path がない場合に `src/**` を追加し、既定の docs/tests/changelog exclusion、`--exclude-tests`、コメント / CLI ヘルプ文言 origin の既定除外を適用する。`all` は他の filter で除外しない限り、すべての indexed path を意図的に検索する。JSON 出力には該当する場合、有効な scope、path filter、exclusion が含まれる。 | | `--source-only` | `search` | ad hoc / named search で `--audit-scope source` を指定する shorthand。recipe を選ばずに実装コードだけを検索したい場合に使う。コメントと CLI ヘルプ文言の origin も既定で除外し、これらのドキュメント的な一致を意図的な根拠にしたい場合は `--origin comment` または `--origin help_text` を指定する。 | | `--show-excluded` | `search --recipe ` | recipe output に `scope.excluded_diagnostics` を含め、広い audit で default include pattern、default exclusion、user exclusion、test filter の適用状況を確認できるようにする。 | @@ -5239,7 +5241,7 @@ raw match density を正確に測る、といった理由で全 raw chunk hit | `--search-fields ` | `search` | recipe の `query_name` / `recipe` を含む compact JSON field を projection する | | `--results-only` | `search`、`symbols`、`files` | shell pipeline 向けに stream の終端レコードを含まない result-only NDJSON を出力する | | `--first-per-file` / `--sample ` / `--total-limit ` | `search` | file 単位、決定的 sample 数、recipe 全体の row 数で広い audit 出力を制限する | -| `--max-json-bytes ` | `search`、`definition`、`recipes`、`audit`、`deps`、`hotspots` | 指定した UTF-8 byte 上限を超える JSON を出力する前に失敗する。`definition --json` の未検出時も構造化 not-found object を同じ上限に対して事前検査し、object が収まらない場合は上限超過の stdout を出さず stderr に usage error を報告する。大きい graph 出力では `deps --summary-only`、`deps --format json-graph --summary-only`、または `hotspots --summary-only` と組み合わせる。 | +| `--max-json-bytes ` | `search`、`definition`、`recipes`、`audit`、`deps`、`hotspots`、`outline` | 指定した UTF-8 byte 上限を超える JSON を出力する前に失敗する。bounded `outline` は共通 envelope 内に完全な symbol row だけを出力し、authoritative な返却 / 総 / 省略件数と opaque な continuation cursor を公開する。最小 envelope が収まらない場合は stdout を空に保ち、型付き usage error を報告する。`definition --json` の未検出時も構造化 not-found object を同じ上限に対して事前検査し、object が収まらない場合は上限超過の stdout を出さず stderr に usage error を報告する。大きい graph 出力では `deps --summary-only`、`deps --format json-graph --summary-only`、または `hotspots --summary-only` と組み合わせる。 | | `--next-steps` | `search` | 上位 search hit に対する inspect / excerpt follow-up command を出力する | | `--include-generated` | `search`, `definition`, `references`, `callers`, `callees`, `symbols`, `files`, `find`, `map`, `inspect`, `validate`, `deps`, `impact`, `unused`, `hotspots` | 生成コードとして検出されたファイルを含める。生成ファイルは既定でクエリ結果から除外される | | `--snippet-lines ` | `search`, `references`, `callers`, `callees`, `impact` | search スニペット、または graph `--body` 抜粋の行数（デフォルト: 8、最大: 20） | diff --git a/changelog.d/unreleased/4880.added.md b/changelog.d/unreleased/4880.added.md new file mode 100644 index 000000000..99afc794b --- /dev/null +++ b/changelog.d/unreleased/4880.added.md @@ -0,0 +1,24 @@ +--- +category: added +issues: + - 4880 +affected: + - src/CodeIndex/Cli/CliFlagSchema.cs + - src/CodeIndex/Cli/ConsoleUi.cs + - src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs + - src/CodeIndex/Cli/QueryCommandRunner.Outline.cs + - tests/CodeIndex.Tests/ConsoleUiTests.cs + - tests/CodeIndex.Tests/QueryCommandRunnerOutlineIssue4880Tests.cs + - README.md + - USER_GUIDE.md + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **`outline` now supports a hard JSON byte budget (#4880)** — `--max-json-bytes` returns complete UTF-8 symbol rows in the shared bounded envelope with authoritative counts and generation-bound continuation cursors. Deep hierarchies, long signatures, and Unicode remain ordered across pages, while undersized budgets return typed diagnostics and uncapped output remains compatible. + +## 日本語 + +- **`outline` が JSON の hard byte budget に対応しました (#4880)** — `--max-json-bytes` は完全な UTF-8 symbol row を共通 bounded envelope で返し、authoritative な件数と generation に束縛された continuation cursor を提供します。深い階層、長い signature、Unicode の順序は page 間で維持され、収まらない小さな budget は型付き diagnostic を返し、上限なしの出力は互換性を維持します。 diff --git a/src/CodeIndex/Cli/CliFlagSchema.cs b/src/CodeIndex/Cli/CliFlagSchema.cs index 06083c1b7..023b865a9 100644 --- a/src/CodeIndex/Cli/CliFlagSchema.cs +++ b/src/CodeIndex/Cli/CliFlagSchema.cs @@ -555,7 +555,7 @@ private static IReadOnlyList BuildAll() new() { Name = "--env-domain", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact domain", PrimaryCommands = Set("doctor") }, new() { Name = "--env-category", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact category", PrimaryCommands = Set("doctor") }, new() { Name = "--env-sensitivity", ValuePlaceholder = "", Description = "Doctor full environment inventory: filter by exact sensitivity", PrimaryCommands = Set("doctor") }, - new() { Name = "--max-json-bytes", ValuePlaceholder = "", Description = "Bound emitted JSON bytes; bounded high-volume responses truncate projected rows with paging metadata", PrimaryCommands = Set("search", "definition", "find", "status", "references", "callers", "callees", "excerpt", "inspect", "impact", "recipes", "audit", "map", "files", "symbols", "deps", "hotspots", "languages", "doctor") }, + new() { Name = "--max-json-bytes", ValuePlaceholder = "", Description = "Bound emitted JSON bytes; bounded high-volume responses truncate projected rows with paging metadata", PrimaryCommands = Set("search", "definition", "find", "status", "references", "callers", "callees", "excerpt", "inspect", "outline", "impact", "recipes", "audit", "map", "files", "symbols", "deps", "hotspots", "languages", "doctor") }, new() { Name = "--next-steps", Description = "Search: print inspect/excerpt follow-up commands for top hits", PrimaryCommands = Set("search") }, new() { Name = "--exclude-comments", Description = "Search: suppress comment-only matches after origin classification", PrimaryCommands = Set("search") }, new() { Name = "--exclude-strings", Description = "Search: suppress string, regex, and help-text matches after origin classification", PrimaryCommands = Set("search") }, diff --git a/src/CodeIndex/Cli/ConsoleUi.cs b/src/CodeIndex/Cli/ConsoleUi.cs index dff6b8f71..4f4051b31 100644 --- a/src/CodeIndex/Cli/ConsoleUi.cs +++ b/src/CodeIndex/Cli/ConsoleUi.cs @@ -99,7 +99,7 @@ private static readonly (string Command, string Usage)[] CommandUsageLines = ("map", "cdidx map [--db ] [--json] [--format ] [--pretty] [--compact] [--fields ] [--cursor ] [--summary-only] [--verbose] [--limit |--top ] [--lang ] [--path ] [--exclude-path ] [--exclude-tests] [--bytes] [--sections ] [--depth ] [--min-entrypoint-confidence <0.0..1.0>] [--max-json-bytes ]"), ("inspect", "cdidx inspect |--query |-- [--db ] [--json] [--redact-paths|--show-paths] [--format ] [--pretty] [--compact] [--fields ] [--outline-only] [--body-only] [--cursor ] [--max-json-bytes ] [--verbose] [--limit |--top ] [--lang ] [--kind ] [--path ] [--exclude-path ] [--exclude-tests] [--body] [--body-start ] [--body-lines |--body-line-count ] [--context |--before |--after ] [--max-line-width ] [--exact|--exact-name] [--group-partials]"), ("inspect", "cdidx inspect --path --line [--end-line ] [--db ] [--json] [--redact-paths|--show-paths] [--format ] [--pretty] [--compact] [--fields ] [--outline-only] [--body-only] [--cursor ] [--max-json-bytes ] [--body] [--body-start ] [--body-lines |--body-line-count ] [--context |--before |--after ] [--max-line-width ]"), - ("outline", "cdidx outline [--db ] [--json] [--pretty] [--compact] [--verbose] [--limit |--top ] [--cursor ] [--sort ] [--kind ] [--outline-fields ]"), + ("outline", "cdidx outline [--db ] [--json] [--pretty] [--compact] [--verbose] [--limit |--top ] [--cursor ] [--max-json-bytes ] [--sort ] [--kind ] [--outline-fields ]"), ("status", "cdidx status [--db ] [--json] [--format ] [--compact] [--fields ] [--cursor ] [--max-json-bytes ] [--verbose] [--check[=workspace,fold,graph,issues,hotspot,csharp,sql,newer]] [--stale-after ] [--explain ] [--log-path] [--config [--redact-paths|--show-paths]] [--check-updates]"), ("workspace", "cdidx workspace [name-or-relative-path] [--json]"), ("workspace-list", "cdidx workspace list [--json]"), diff --git a/src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs b/src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs index 1f5d45d7f..f2ff2092f 100644 --- a/src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs +++ b/src/CodeIndex/Cli/JsonEnvelopeWrapper.Bounded.cs @@ -29,7 +29,7 @@ internal static partial class JsonEnvelopeWrapper private static readonly HashSet AutoWrapByteBudgetCommands = new(StringComparer.Ordinal) { - "find", "status", "references", "callers", "callees", "languages", + "find", "status", "references", "callers", "callees", "languages", "outline", }; private static readonly HashSet AutoWrapCompactCommands = new(StringComparer.Ordinal) @@ -46,7 +46,7 @@ internal static partial class JsonEnvelopeWrapper private static readonly HashSet PageableResponseCommands = new(StringComparer.Ordinal) { "search", "definition", "find", "hotspots", "references", "callers", "callees", - "symbols", "files", "languages", "impact", "map", + "symbols", "files", "languages", "impact", "map", "outline", }; private static readonly HashSet CountableResponseCommands = new(StringComparer.Ordinal) @@ -57,7 +57,8 @@ internal static partial class JsonEnvelopeWrapper internal static bool ShouldAutoWrapBoundedResponse(string command, string[] args) { - if (!BoundedResponseCommands.Contains(command)) + if (!BoundedResponseCommands.Contains(command) + && !IsOutlineByteBudgetRequest(command, args)) return false; if (command == "search" && IsSearchAggregateResponseRequest(args)) return false; @@ -78,7 +79,8 @@ internal static bool ShouldAutoWrapBoundedResponse(string command, string[] args private static bool IsBoundedResponseRequest(string command, string[] args) { - if (!BoundedResponseCommands.Contains(command)) + if (!BoundedResponseCommands.Contains(command) + && !IsOutlineByteBudgetRequest(command, args)) return false; if (command == "search" && IsSearchAggregateResponseRequest(args)) return false; @@ -90,6 +92,16 @@ private static bool IsBoundedResponseRequest(string command, string[] args) || ShouldAutoWrapBoundedResponse(command, args); } + private static bool IsOutlineByteBudgetRequest(string command, string[] args) + => command == "outline" + && HasArgument(args, "--max-json-bytes"); + + private static bool HasUnsupportedOutlineBoundedControl(string command, string[] args) + => command == "outline" + && (HasArgument(args, "--fields") + || HasArgument(args, "--format") + || args.Any(arg => arg.StartsWith("--json=", StringComparison.Ordinal))); + private static bool IsStandaloneFindCountContinuationRequest(string[] args) => IsFindCountResponseRequest(args) && HasArgument(args, "--cursor") @@ -167,6 +179,8 @@ private static int RunBoundedResponse( { if (!TryParseBoundedResponseControls(command, args, out var controls, out var controlError)) return WriteBoundedResponseUsageError(controlError!, "Use the command help to pass positive --limit/--max-json-bytes values and a next_cursor returned by the same query."); + if (HasUnsupportedOutlineBoundedControl(command, args)) + return RunOutlineValidationWithinBudget(args, controls.MaxJsonBytes!.Value, runInner); if (ProjectionFieldRegistry.IsDiscoveryRequest(controls.Fields)) { var discoveryJson = ProjectionFieldRegistry.CreateDiscoveryDocument(command).ToJsonString(jsonOptions); @@ -341,15 +355,53 @@ private static int RunBoundedResponse( if (envelope is null) { + var fieldsOption = command == "outline" ? "--outline-fields" : "--fields"; return WriteBoundedResponseUsageError( $"--max-json-bytes {controls.MaxJsonBytes} is too small for the bounded response metadata and one projected row.", - "Increase --max-json-bytes or choose fewer --fields."); + $"Increase --max-json-bytes or choose fewer {fieldsOption}."); } Console.WriteLine(emittedJson); return exitCode; } + private static int RunOutlineValidationWithinBudget( + string[] args, + int maxJsonBytes, + Func runInner) + { + using var captured = new BoundedStringWriter(MaxCapturedOutputChars); + int exitCode; + using (ScopedConsoleOutput.Redirect(captured)) + exitCode = runInner(args); + + var output = captured.ToString(); + if (output.Length == 0) + return exitCode; + if (Encoding.UTF8.GetByteCount(output) <= maxJsonBytes) + { + Console.Write(output); + return exitCode; + } + + string? message = null; + string? hint = null; + try + { + var error = JsonNode.Parse(output) as JsonObject; + message = ReadString(error, "message"); + hint = ReadString(error, "hint"); + } + catch (JsonException) + { + // Fall back to a bounded generic diagnostic if validation emitted malformed JSON. + } + + return WriteBoundedResponseUsageError( + message ?? "outline output-selector validation failed.", + $"{hint ?? "Use only options shown in `outline --help`."} Increase --max-json-bytes to receive the structured validation error."); + } + private static JsonObject? BuildBoundedEnvelopeWithinBudget( string command, string? queryNormalized, @@ -556,7 +608,7 @@ private static string SerializeBoundedEnvelope(JsonNode node, JsonSerializerOpti return Encoding.UTF8.GetString(stream.ToArray()); } - private static bool JsonFitsResponseBudget(string json, int maxJsonBytes) + internal static bool JsonFitsResponseBudget(string json, int maxJsonBytes) => Encoding.UTF8.GetByteCount(json) + Encoding.UTF8.GetByteCount(Environment.NewLine) <= maxJsonBytes; private static int WriteProjectionRegistryResponse( @@ -723,6 +775,8 @@ private static ResponseExtraction ExtractResponseItems(string command, JsonArray return ExtractNestedCollection(filesPayload, "files"); if (command == "languages" && rawResults.FirstOrDefault() is JsonObject languagesPayload) return ExtractNestedCollection(languagesPayload, "languages"); + if (command == "outline" && rawResults.FirstOrDefault() is JsonObject outlinePayload) + return ExtractOutlineSymbols(outlinePayload); if (command == "impact" && rawResults.FirstOrDefault() is JsonObject impactPayload) { var requestedCollection = SelectRequestedCollection(controls.Fields, "callers", "file_impacts", "definitions"); @@ -774,6 +828,23 @@ private static ResponseExtraction ExtractResponseItems(string command, JsonArray return new ResponseExtraction(rows, null, null, null); } + private static ResponseExtraction ExtractOutlineSymbols(JsonObject payload) + { + var extraction = ExtractNestedCollection(payload, "symbols"); + foreach (var pagingField in new[] + { + "returned_symbol_count", + "cursor_offset", + "next_cursor", + "has_more", + "result_stable_at", + }) + { + extraction.Context?.Remove(pagingField); + } + return extraction; + } + private static ResponseExtraction ExtractDiscoveryRows(string command, JsonArray rawResults) { var rows = new JsonArray(); @@ -1000,8 +1071,12 @@ private static string[] PrepareBoundedInnerArgs(string command, string[] args, B { var stripped = StripResponseOptions(args, stripLimit: PageableResponseCommands.Contains(command)); var bodyRequested = HasExplicitBodyProjection(controls.Fields); - if (!bodyRequested && (controls.Compact || controls.Fields is { Count: > 0 })) + if (command != "outline" + && !bodyRequested + && (controls.Compact || controls.Fields is { Count: > 0 })) + { stripped.RemoveAll(arg => string.Equals(arg, "--body", StringComparison.Ordinal)); + } if (PageableResponseCommands.Contains(command)) { stripped.Add("--limit"); diff --git a/src/CodeIndex/Cli/QueryCommandRunner.Outline.cs b/src/CodeIndex/Cli/QueryCommandRunner.Outline.cs index b731d23a7..3d3768774 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.Outline.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.Outline.cs @@ -125,17 +125,23 @@ public static int RunOutline(string[] cmdArgs, JsonSerializerOptions jsonOptions var filteredSymbols = ApplyOutlineKindFilters(outline.Symbols, kindFilters); var displaySourceSymbols = ApplyOutlineSort(filteredSymbols, outlineSortMode, includeDerivedMetadata); + var boundedLimit = JsonEnvelopeWrapper.GetBoundedResponseLimit("outline"); + var boundedOffset = boundedLimit.HasValue + ? JsonEnvelopeWrapper.GetBoundedResponseOffset("outline") + : (int?)null; + if (boundedLimit.HasValue) + JsonEnvelopeWrapper.ReportBoundedResponseTotal("outline", displaySourceSymbols.Count, authoritative: true); if (options.Json) { if (options.Compact) { - var payload = BuildOutlineJsonPayload(outline, displaySourceSymbols, kindFilters, outlineSortMode, options, cursorContext, jsonOptions, compact: true); + var payload = BuildOutlineJsonPayload(outline, displaySourceSymbols, kindFilters, outlineSortMode, options, cursorContext, jsonOptions, compact: true, boundedOffset); AddActiveSqliteDiagnostics(payload); Console.WriteLine(payload.ToJsonString(jsonOptions)); } else if (HasOutlineJsonControls(options, kindFilters)) { - var payload = BuildOutlineJsonPayload(outline, displaySourceSymbols, kindFilters, outlineSortMode, options, cursorContext, jsonOptions, compact: false); + var payload = BuildOutlineJsonPayload(outline, displaySourceSymbols, kindFilters, outlineSortMode, options, cursorContext, jsonOptions, compact: false, boundedOffset); AddActiveSqliteDiagnostics(payload); Console.WriteLine(payload.ToJsonString(jsonOptions)); } @@ -394,10 +400,11 @@ private static JsonObject BuildOutlineJsonPayload( QueryCommandOptions options, PaginationCursorContext cursorContext, JsonSerializerOptions jsonOptions, - bool compact) + bool compact, + int? boundedOffset = null) { var totalMatchingSymbols = filteredSymbols.Count; - var offset = Math.Min(options.OutlineCursorOffset ?? 0, totalMatchingSymbols); + var offset = Math.Min(boundedOffset ?? options.OutlineCursorOffset ?? 0, totalMatchingSymbols); var remainingSymbols = offset == 0 ? filteredSymbols.ToList() : filteredSymbols.Skip(offset).ToList(); diff --git a/tests/CodeIndex.Tests/ConsoleUiTests.cs b/tests/CodeIndex.Tests/ConsoleUiTests.cs index ce543dd15..7017f4b93 100644 --- a/tests/CodeIndex.Tests/ConsoleUiTests.cs +++ b/tests/CodeIndex.Tests/ConsoleUiTests.cs @@ -409,7 +409,7 @@ public void PrintUsage_WithoutBanner_HidesAsciiArtAndEasterEggFlags() Assert.Contains("cdidx references |--query |-- [--db ] [--json] [--redact-paths|--show-paths] [--format ] [--fields ] [--cursor ] [--max-json-bytes ] [--verbose] [--limit |--top ] [--lang ] [--kind ] [--path ] [--exclude-path ] [--exclude-tests] [--include-qualified-common-calls] [--body] [--snippet-lines ] [--max-line-width ] [--exact|--exact-name] [--count]", output); Assert.Contains("cdidx inspect |--query |-- [--db ] [--json] [--redact-paths|--show-paths] [--format ] [--pretty] [--compact] [--fields ] [--outline-only] [--body-only] [--cursor ] [--max-json-bytes ] [--verbose] [--limit |--top ] [--lang ] [--kind ] [--path ] [--exclude-path ] [--exclude-tests] [--body] [--body-start ] [--body-lines |--body-line-count ] [--context |--before |--after ] [--max-line-width ] [--exact|--exact-name] [--group-partials]", output); Assert.Contains("cdidx inspect --path --line [--end-line ] [--db ] [--json] [--redact-paths|--show-paths] [--format ] [--pretty] [--compact] [--fields ] [--outline-only] [--body-only] [--cursor ] [--max-json-bytes ] [--body] [--body-start ] [--body-lines |--body-line-count ] [--context |--before |--after ] [--max-line-width ]", output); - Assert.Contains("cdidx outline [--db ] [--json] [--pretty] [--compact] [--verbose] [--limit |--top ] [--cursor ] [--sort ] [--kind ] [--outline-fields ]", output); + Assert.Contains("cdidx outline [--db ] [--json] [--pretty] [--compact] [--verbose] [--limit |--top ] [--cursor ] [--max-json-bytes ] [--sort ] [--kind ] [--outline-fields ]", output); Assert.Contains("--snippet-lines ", output); Assert.Contains("Snippet length; issue-drafts accept 0 for path/line-only evidence", output); Assert.Contains("cdidx find (--path |--all)", output); diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerOutlineIssue4880Tests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerOutlineIssue4880Tests.cs new file mode 100644 index 000000000..733533154 --- /dev/null +++ b/tests/CodeIndex.Tests/QueryCommandRunnerOutlineIssue4880Tests.cs @@ -0,0 +1,280 @@ +using System.Text; +using System.Text.Json; +using CodeIndex.Cli; + +namespace CodeIndex.Tests; + +[Collection("Console sensitive")] +public sealed class QueryCommandRunnerOutlineIssue4880Tests +{ + private const int PageByteBudget = 3_000; + private readonly JsonSerializerOptions _jsonOptions = ProgramRunner.CreateDefaultJsonOptions(); + + [Fact] + public void Outline_MaxJsonBytesPagesWholeUnicodeRowsAndResumesHierarchy_Issue4880() + { + var projectRoot = TestProjectHelper.CreateTempProject("outline_byte_budget_4880"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + var parameters = string.Join( + ", ", + Enumerable.Range(0, 16).Select(index => $"string 引数{index:D2}")); + var methods = string.Join( + Environment.NewLine, + Enumerable.Range(0, 12).Select(index => + $"public string 方法{index:D2}({parameters}) => \"値{index:D2}猫\";")); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/UnicodeTree.cs", + "csharp", + $$""" + namespace 世界; + public sealed class Root + { + public sealed class LevelOne + { + public sealed class LevelTwo + { + {{methods}} + } + } + } + """); + + var (legacyExitCode, legacyStdout, legacyStderr) = ConsoleCapture.Capture(() => + ProgramRunner.Run( + ["outline", "src/UnicodeTree.cs", "--db", dbPath, "--json"], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.Success, legacyExitCode); + Assert.Equal(string.Empty, legacyStderr); + using var legacyDocument = JsonDocument.Parse(legacyStdout); + Assert.False(legacyDocument.RootElement.TryGetProperty("metadata", out _)); + var expectedRows = legacyDocument.RootElement + .GetProperty("symbols") + .EnumerateArray() + .Select(ReadOutlineIdentity) + .ToArray(); + Assert.Contains(expectedRows, row => row.Name == "方法00" && row.Depth >= 3); + Assert.Contains( + legacyDocument.RootElement.GetProperty("symbols").EnumerateArray(), + row => row.GetProperty("name").GetString() == "方法00" + && row.GetProperty("signature").GetString()!.Contains("引数15", StringComparison.Ordinal)); + + var baseArgs = new[] + { + "outline", "src/UnicodeTree.cs", "--db", dbPath, "--json", + "--limit", "50", "--max-json-bytes", PageByteBudget.ToString(), + }; + var actualRows = new List<(string Name, int Depth)>(); + string? cursor = null; + var pageCount = 0; + do + { + var args = cursor is null ? baseArgs : baseArgs.Concat(["--cursor", cursor]).ToArray(); + var (exitCode, stdout, stderr) = ConsoleCapture.Capture(() => + ProgramRunner.Run(args, _jsonOptions, "1.0.0-test")); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.True(Encoding.UTF8.GetByteCount(stdout) <= PageByteBudget); + using var document = JsonDocument.Parse(stdout); + var root = document.RootElement; + var metadata = root.GetProperty("metadata"); + var results = root.GetProperty("results").EnumerateArray().ToArray(); + + pageCount++; + Assert.True(pageCount <= expectedRows.Length, "outline byte-budget cursor did not make forward progress."); + Assert.Equal("outline", metadata.GetProperty("command").GetString()); + Assert.Equal("symbols", metadata.GetProperty("primary_collection").GetString()); + Assert.Equal(expectedRows.Length, metadata.GetProperty("total_count").GetInt32()); + Assert.True(metadata.GetProperty("total_count_authoritative").GetBoolean()); + Assert.Equal(results.Length, metadata.GetProperty("returned_count").GetInt32()); + Assert.NotEmpty(results); + Assert.Equal( + "src/UnicodeTree.cs", + metadata.GetProperty("response_context").GetProperty("path").GetString()); + Assert.Equal( + expectedRows.Length, + metadata.GetProperty("response_context").GetProperty("total_symbol_count").GetInt32()); + Assert.False(metadata.GetProperty("response_context").TryGetProperty("next_cursor", out _)); + + actualRows.AddRange(results.Select(ReadOutlineIdentity)); + cursor = metadata.GetProperty("next_cursor").GetString(); + if (cursor is not null) + Assert.StartsWith("response:v2:", cursor, StringComparison.Ordinal); + } + while (cursor is not null); + + Assert.True(pageCount > 1); + Assert.Equal(expectedRows, actualRows); + Assert.Equal(actualRows.Count, actualRows.Distinct().Count()); + + var (smallExitCode, smallStdout, smallStderr) = ConsoleCapture.Capture(() => + ProgramRunner.Run( + [ + "outline", "src/UnicodeTree.cs", "--db", dbPath, "--json", + "--kind", "function", "--outline-fields", "name,signature", + "--max-json-bytes", "64", + ], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, smallExitCode); + Assert.Equal(string.Empty, smallStdout); + Assert.Contains($"Error [{CommandErrorCodes.UsageError}]", smallStderr, StringComparison.Ordinal); + Assert.Contains("bounded response metadata and one projected row", smallStderr, StringComparison.Ordinal); + Assert.Contains("--outline-fields", smallStderr, StringComparison.Ordinal); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + [Fact] + public void Outline_JsonByteBudgetCountsUnicodeAndFinalNewlineAtExactBoundary_Issue4880() + { + const string json = """{"results":[{"name":"方法猫"}]}"""; + var exactBudget = Encoding.UTF8.GetByteCount(json) + + Encoding.UTF8.GetByteCount(Environment.NewLine); + + Assert.True(JsonEnvelopeWrapper.JsonFitsResponseBudget(json, exactBudget)); + Assert.False(JsonEnvelopeWrapper.JsonFitsResponseBudget(json, exactBudget - 1)); + } + + [Fact] + public void Outline_MaxJsonBytesIsExposedByHelpAndFlagSchema_Issue4880() + { + var flag = Assert.Single( + CliFlagSchema.GetCompletionFlagsForCommand("outline"), + candidate => candidate.Name == "--max-json-bytes"); + Assert.Contains("Bound emitted JSON bytes", flag.Description, StringComparison.Ordinal); + + var (printed, stdout, stderr) = ConsoleCapture.Capture(() => + ConsoleUi.PrintCommandUsage("outline") ? 1 : 0); + Assert.Equal(1, printed); + Assert.Equal(string.Empty, stderr); + Assert.Contains("--max-json-bytes ", stdout, StringComparison.Ordinal); + } + + [Fact] + public void Outline_MaxJsonBytesPreservesUnsupportedOutputSelectorValidation_Issue4880() + { + var projectRoot = TestProjectHelper.CreateTempProject("outline_byte_budget_validation_4880"); + try + { + var dbPath = TestProjectHelper.CreateProjectDb(projectRoot); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/Sample.cs", + "csharp", + "public sealed class Sample { }"); + + var (formatExitCode, formatStdout, formatStderr) = ConsoleCapture.Capture(() => + ProgramRunner.Run( + [ + "outline", "src/Sample.cs", "--db", dbPath, + "--max-json-bytes", PageByteBudget.ToString(), + "--format", "nonsense", + ], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, formatExitCode); + Assert.Equal(string.Empty, formatStdout); + Assert.Contains("--format is not supported for outline", formatStderr, StringComparison.Ordinal); + + var (jsonExitCode, jsonStdout, jsonStderr) = ConsoleCapture.Capture(() => + ProgramRunner.Run( + [ + "outline", "src/Sample.cs", "--db", dbPath, + "--max-json-bytes", PageByteBudget.ToString(), + "--json=nonsense", + ], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, jsonExitCode); + Assert.Equal(string.Empty, jsonStderr); + using var document = JsonDocument.Parse(jsonStdout); + Assert.Equal("error", document.RootElement.GetProperty("status").GetString()); + Assert.Contains( + "--json= is not supported by outline", + document.RootElement.GetProperty("message").GetString(), + StringComparison.Ordinal); + + foreach (var field in new[] { "nonexistent", "list" }) + { + var (fieldsExitCode, fieldsStdout, fieldsStderr) = ConsoleCapture.Capture(() => + ProgramRunner.Run( + [ + "outline", "src/Sample.cs", "--db", dbPath, "--json", + "--max-json-bytes", PageByteBudget.ToString(), + "--fields", field, + ], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, fieldsExitCode); + Assert.Equal(string.Empty, fieldsStderr); + using var fieldsDocument = JsonDocument.Parse(fieldsStdout); + Assert.Equal("error", fieldsDocument.RootElement.GetProperty("status").GetString()); + Assert.Contains( + "--fields is not supported for outline", + fieldsDocument.RootElement.GetProperty("message").GetString(), + StringComparison.Ordinal); + } + + var (bodyExitCode, bodyStdout, bodyStderr) = ConsoleCapture.Capture(() => + ProgramRunner.Run( + [ + "outline", "src/Sample.cs", "--db", dbPath, "--compact", + "--max-json-bytes", PageByteBudget.ToString(), + "--body", + ], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, bodyExitCode); + Assert.Equal(string.Empty, bodyStderr); + using (var bodyDocument = JsonDocument.Parse(bodyStdout)) + { + Assert.Equal( + "--body is not supported for outline.", + bodyDocument.RootElement + .GetProperty("metadata") + .GetProperty("error") + .GetProperty("message") + .GetString()); + } + + foreach (var invalidSelector in new[] { "--json=nonsense", "--format=nonsense", "--fields=list" }) + { + var (cappedExitCode, cappedStdout, cappedStderr) = ConsoleCapture.Capture(() => + ProgramRunner.Run( + [ + "outline", "src/Sample.cs", "--db", dbPath, "--json", + "--max-json-bytes", "1", + invalidSelector, + ], + _jsonOptions, + "1.0.0-test")); + + Assert.Equal(CommandExitCodes.UsageError, cappedExitCode); + Assert.Equal(string.Empty, cappedStdout); + Assert.Contains($"Error [{CommandErrorCodes.UsageError}]", cappedStderr, StringComparison.Ordinal); + Assert.Contains("is not supported", cappedStderr, StringComparison.Ordinal); + } + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + + private static (string Name, int Depth) ReadOutlineIdentity(JsonElement row) + => (row.GetProperty("name").GetString()!, row.GetProperty("depth").GetInt32()); +}