Standardize batch error envelopes

DEVELOPER_GUIDE.md

@@ -419,13 +419,17 @@ By default, child query commands stream their normal stdout/stderr directly. In
 `--json-summary` mode, every non-blank stdin line must instead emit one
 machine-readable batch envelope before the final summary: parsed commands use
 `record: "batch_result"` and include `line`, `command`, `arguments`,
-`exit_code`, and captured child `stderr`. The requested command/output format,
-rather than output-text sniffing, selects the projection: successful
+and `exit_code`. The requested command/output format, rather than output-text
+sniffing, selects the projection: successful
 single-document JSON is embedded as typed `result`, while successful NDJSON is
-embedded as a stable typed `results` array even when it has one row. Text and failed
-commands remain raw `stdout` text so diagnostics are not lost. Malformed or
-over-limit input lines use `record: "batch_error"` and an `error` object. Child
-output must not be written directly beside batch metadata in this mode. The entire
+embedded as a stable typed `results` array even when it has one row. Successful
+text remains `stdout`, while every failure uses one typed `error` object with a
+stable `error_code`, `category`, safe `message` / `hint`, and `scope`.
+Malformed or over-limit input lines use `record: "batch_error"` and the same
+typed error serializer. Failed records omit captured child stdout/stderr by
+default; `--include-raw-streams` explicitly adds them under a bounded
+`raw_streams` object. Child output must not be written directly beside batch
+metadata in this mode. The entire
 serialized stream—including envelopes, arguments, escaping expansion, terminal
 errors, and the final summary—uses the configured `--max-output-chars` budget
 (default 10,485,760; maximum 67,108,864). An item that exhausts it retains its
@@ -439,8 +443,13 @@ SQLite connection and thread-local batch reader, and buffer only the active
 worker window. `ScopedConsoleOutput` keeps nested JSON-envelope capture on the
 current worker's routed stdout instead of replacing another worker's process-wide
 writer. Completed records are committed to the shared output writer in input
-order; an ordinary item failure remains isolated, while caller cancellation
-stops scheduling and propagates.
+order; an ordinary item failure remains isolated. Caller cancellation is
+serialized as `batch_cancelled` for a consumed input item and in the final
+summary before batch processing stops. Serial and parallel input waits share a
+bounded pump for each input reader, so cancellation remains prompt while stdin
+is blocked and any line completed in flight stays buffered for a subsequent
+batch invocation. Cancellation during database setup still emits the typed
+final summary.
 
 Editor integrations can request standard location shapes directly. `definition`, `references`, `search`, `find`, and `validate` accept `--format <text|json|lsp|qf|sarif>`; `lsp` emits LSP `Location` arrays, `qf` emits Vim quickfix lines, and `sarif` emits SARIF 2.1.0. `goto <symbol>` returns the single unambiguous definition as one LSP `Location`, while `goto --all <symbol>` returns all matching locations without applying the default or environment-provided query limit. An explicit `--limit` or `--top` still bounds the returned location array.
 
@@ -3688,11 +3697,14 @@ path filter を受け付ける query コマンド（`search`, `definition`, `ref
 既定では child query command の通常の stdout / stderr を直接 stream する。`--json-summary`
 mode では、空白でない stdin 行ごとに final summary より前へ 1 つの machine-readable batch
 envelope を出力しなければならない。parse 済み command は `record: "batch_result"` として
-`line`、`command`、`arguments`、`exit_code`、捕捉した child `stderr` を含める。output text の
-推測ではなく requested command / output format で projection を選び、成功した単一 document JSON
+`line`、`command`、`arguments`、`exit_code` を含める。output text の推測ではなく requested
+command / output format で projection を選び、成功した単一 document JSON
 は型付き `result`、NDJSON は 1 row の場合も安定した型付き `results` array として埋め込む。
-text と失敗 command の出力は診断を失わないよう raw `stdout` text のまま
-保持する。malformed line や入力上限超過 line は `record: "batch_error"` と `error` object を使う。
+成功した text command は `stdout` のまま保持する一方、すべての失敗は安定した `error_code`、
+`category`、安全な `message` / `hint`、`scope` を持つ共通の型付き `error` object を使う。
+malformed line や入力上限超過 line は `record: "batch_error"` と同じ typed error serializer を
+使う。失敗 record は既定で捕捉した child stdout / stderr を省略し、
+`--include-raw-streams` を明示した場合だけ上限付きの `raw_streams` object に追加する。
 この mode では child output を batch metadata と並べて直接出力してはならない。envelope、
 arguments、escape 展開、terminal error、final summary を含む serialized stream 全体には
 設定された `--max-output-chars` budget（既定 10,485,760、最大 67,108,864）を適用し、
@@ -3705,7 +3717,11 @@ command ごとの bounded writer へ route し、分離した read-only SQLite c
 batch reader を使い、active worker window だけを buffer する。`ScopedConsoleOutput` は nested
 JSON-envelope capture を現在の worker の routed stdout に保ち、他 worker の process-wide writer を
 置き換えない。完了 record は入力順で共有 output writer へ commit する。通常の item failure は
-他 item から隔離し、caller cancellation は scheduling を停止して伝播する。
+他 item から隔離する。caller cancellation は、消費済み input item と final summary に
+`batch_cancelled` を記録してから後続処理を停止する。serial / parallel の input wait は input
+reader ごとの bounded pump を共有するため、stdin が block 中でも cancellation を迅速に検知し、
+同時に完成した line は後続の batch invocation 用に buffer したまま保持する。database setup
+中の cancellation でも型付き final summary を出力する。
 
 editor integration は標準的な location 形状を直接要求できる。`definition`、`references`、`search`、`find`、`validate` は `--format <text|json|lsp|qf|sarif>` を受け付け、`lsp` は LSP `Location` 配列、`qf` は Vim quickfix 行、`sarif` は SARIF 2.1.0 を出力する。`goto <symbol>` は曖昧でない単一定義を 1 つの LSP `Location` として返し、`goto --all <symbol>` は既定または環境変数由来の query limit を適用せず、一致する全 location を返す。明示的な `--limit` または `--top` を指定した場合は location 配列をその件数に制限する。
 

TESTING_GUIDE.md

@@ -193,7 +193,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding
   Doctor full-inventory coverage keeps composed filter selection, filtered summary counts, exact UTF-8 byte-budget boundaries, and structured overflow errors together in `ProgramRunnerTests`; license JSON remains a subprocess contract in `ProgramCliTests` so immediate-command dispatch and the published field names are both exercised.
   Ctags export JSON coverage reuses one seeded database for default and `--include-generated` variants, asserts the fixed skip-reason keys sum to `skipped_count`, and keeps the missing-`files.generated` degradation in a separate legacy-schema fixture.
   Dry-run JSON coverage for ambiguous `.h` files locks the bounded `language_detections` entries and their stable source/confidence codes without mutating the index.
-  Batch `--json-summary` coverage keeps side-effect-free dispatch plus successful JSON, single-row NDJSON, and JSON-looking text projection in one seeded fixture when the assertions stay distinct; include case-insensitive format values, command-specific JSON formats, and alias-injected JSON defaults so batch classification cannot drift from child parsers. Serialized-output exhaustion remains a separate boundary fixture because of its large escaped payload. Assert `result` / `results` versus raw `stdout` presence explicitly, compare `output_chars` with the actual captured stream length, and cover repeated malformed lines up to the input cap so neither syntax sniffing nor envelope overhead can bypass the transport bounds.
+  Batch `--json-summary` coverage keeps side-effect-free dispatch plus successful JSON, single-row NDJSON, and JSON-looking text projection in one seeded fixture when the assertions stay distinct; include case-insensitive format values, command-specific JSON formats, and alias-injected JSON defaults so batch classification cannot drift from child parsers. Serialized-output exhaustion remains a separate boundary fixture because of its large escaped payload. Assert `result` / `results` versus raw `stdout` presence explicitly, compare `output_chars` with the actual captured stream length, and cover repeated malformed lines up to the input cap so neither syntax sniffing nor envelope overhead can bypass the transport bounds. Typed-failure coverage reuses one mixed scalar/array/schema/policy/child-failure input across serial and parallel modes, asserts stable ordered categories, and keeps timeout, caller cancellation, and explicit `--include-raw-streams` behavior in focused fixtures. Cancellation coverage includes pre-cancelled setup, cancellation after an earlier failure and after parallel item preparation, and a blocking stdin reader that proves both prompt wake-up and preservation of an in-flight line for the next batch invocation.
   Argument-validation variants that only differ by invalid scalar input share one database fixture and iterate within a fact when no per-case state or discovery identity is required.
   Positional `files` glob coverage shares one indexed-file fixture and iterates `*`, `?`, and recursive `**` patterns in a fact, matching the exact tokens that a quoted shell argument passes to the CLI.
   Excerpt focus coverage reuses one indexed fixture for line-only leading-window behavior, the focus-length dependency, and focus-column range validation; zero and non-numeric focus-column values share one indexed Markdown fixture.
@@ -1112,7 +1112,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
   doctor full-inventory coverage では、合成 filter の選択、filtered summary 件数、UTF-8 byte budget の exact boundary、structured overflow error を `ProgramRunnerTests` にまとめます。license JSON は `ProgramCliTests` の subprocess contract として、immediate-command dispatch と公開 field 名を同時に検証します。
   ctags export JSON coverage は1つの seeded database を既定と `--include-generated` variant で再利用し、固定された skip-reason key の合計が `skipped_count` と一致することを検証します。`files.generated` がない場合の縮退は別の legacy-schema fixture に保ってください。
   曖昧な `.h` に対する dry-run JSON coverage は、index を変更せず、上限付き `language_detections` entry と安定した判定元・信頼度 code を固定します。
-  batch `--json-summary` coverage は、assertion を明確に保てる場合、副作用なし dispatch と成功時の JSON / 1 row NDJSON / JSON に見える text projection を1つの seeded fixture で共有し、case-insensitive な format 値、command 固有の JSON format、alias が内部追加する JSON default も含めて child parser と batch classification の drift を防いでください。serialized output の枯渇は escape を多く含む大きな payload の境界 fixture として分離します。test 側で `result` / `results` と raw `stdout` の有無を明示し、`output_chars` を実際に捕捉した stream 長と比較し、input cap までの malformed line 反復も検証して、syntax sniffing や envelope overhead が transport 上限を迂回しないようにしてください。
+  batch `--json-summary` coverage は、assertion を明確に保てる場合、副作用なし dispatch と成功時の JSON / 1 row NDJSON / JSON に見える text projection を1つの seeded fixture で共有し、case-insensitive な format 値、command 固有の JSON format、alias が内部追加する JSON default も含めて child parser と batch classification の drift を防いでください。serialized output の枯渇は escape を多く含む大きな payload の境界 fixture として分離します。test 側で `result` / `results` と raw `stdout` の有無を明示し、`output_chars` を実際に捕捉した stream 長と比較し、input cap までの malformed line 反復も検証して、syntax sniffing や envelope overhead が transport 上限を迂回しないようにしてください。typed failure coverage は scalar / array / schema / policy / child failure を混在させた1つの input を serial / parallel mode で共有し、安定した順序と category を検証してください。timeout、caller cancellation、明示的な `--include-raw-streams` の挙動は焦点を絞った fixture に分離します。cancellation coverage には事前 cancel 済み setup、先行 failure 後と parallel item 準備後の cancellation、blocking stdin reader を含め、型付き summary、迅速な wait 解除、同時に読み取った line が次の batch invocation 用に保持されることを決定的に検証します。
   invalid scalar input だけが異なる argument-validation variant は、case ごとの state や discovery identity が不要なら1つの database fixture を共有し、fact 内で反復してください。
   `files` の positional glob coverage は1つの indexed-file fixture を共有し、quote された shell 引数が CLI に渡す token と同じ `*`、`?`、recursive `**` pattern を fact 内で反復してください。
   excerpt の focus coverage は、line-only 時の先頭側 window、focus-length の依存関係、focus-column の範囲検証を1つの indexed fixture で共有してください。focus-column の zero / non-numeric value も1つの indexed Markdown fixture を再利用してください。

changelog.d/unreleased/4871.fixed.md

@@ -0,0 +1,22 @@
+---
+category: fixed
+issues:
+  - 4871
+affected:
+  - src/CodeIndex/Cli/CliFlagSchema.cs
+  - src/CodeIndex/Cli/ConsoleUi.cs
+  - src/CodeIndex/Cli/QueryCommandRunner.Batch.cs
+  - tests/CodeIndex.Tests/QueryCommandRunnerBatchIssue4723Tests.cs
+  - tests/CodeIndex.Tests/QueryCommandRunnerBatchIssue4871Tests.cs
+  - tests/CodeIndex.Tests/QueryCommandRunnerTests.cs
+  - DEVELOPER_GUIDE.md
+  - TESTING_GUIDE.md
+---
+
+## English
+
+- **Standardized typed error envelopes for every batch failure (#4871)** — `batch --json-summary` now gives malformed input, policy rejection, child failure, timeout, and cancellation the same stable error-code/category contract in serial and parallel modes, while bounded raw child streams require explicit `--include-raw-streams`.
+
+## 日本語
+
+- **すべての batch failure で型付き error envelope を統一しました (#4871)** — `batch --json-summary` は malformed input、policy rejection、child failure、timeout、cancellation に対して serial / parallel mode 共通の安定した error-code / category 契約を返し、上限付きの raw child stream は `--include-raw-streams` を明示した場合だけ含めるようになりました。

src/CodeIndex/Cli/CliFlagSchema.cs

@@ -403,6 +403,7 @@ private static IReadOnlyList<CliFlag> BuildAll()
             new() { Name = "--data-dir", ValuePlaceholder = "<dir>", Description = "Directory containing codeindex.db; overrides CDIDX_DATA_DIR/XDG/workspace defaults", PrimaryCommands = Set(DataDirCommands), Safety = CliOptionSafety.Scope },
             new() { Name = "--json", Description = "JSON output; search/symbols/files/validate also accept --json=array for a single JSON array", PrimaryCommands = Set(JsonCommands.Concat(["hooks"]).ToArray()) },
             new() { Name = "--json-summary", Description = "Batch: emit one typed result/error record per input plus a final summary", PrimaryCommands = Set("batch") },
+            new() { Name = "--include-raw-streams", Description = "Batch JSON-summary: attach bounded child stdout/stderr to failed records", PrimaryCommands = Set("batch") },
             new() { Name = "--max-input-lines", ValuePlaceholder = "<n>", Description = $"Batch: input-line budget (default {QueryCommandRunner.BatchDefaultInputLines}, max {QueryCommandRunner.BatchMaxInputLines})", PrimaryCommands = Set("batch") },
             new() { Name = "--max-output-chars", ValuePlaceholder = "<n>", Description = $"Batch JSON-summary output budget (default {QueryCommandRunner.BatchDefaultTotalOutputChars}, max {QueryCommandRunner.BatchMaxTotalOutputChars})", PrimaryCommands = Set("batch") },
             new() { Name = "--parallel", ValuePlaceholder = "<n>", Description = $"Batch JSON-summary worker count (default 1, max {QueryCommandRunner.BatchMaxParallelism}); results retain input order", PrimaryCommands = Set("batch") },

src/CodeIndex/Cli/ConsoleUi.cs

@@ -146,7 +146,7 @@ private static readonly (string Command, string Usage)[] CommandUsageLines =
         ("export-ctags", "cdidx export ctags [--output <path>] [--db <path>] [--json] [--lang <lang>] [--path <glob>] [--exclude-path <glob>] [--exclude-tests] [--include-generated]"),
         ("import", "cdidx import <archive> [--db <path>] [--prune-paths] [--no-backup] [--dry-run|--check] [--limit <n<=10000>] [--offset <n>] [--json]"),
         ("languages", "cdidx languages [--db <path>] [--json] [--format <text|json|count>] [--summary-only] [--limit <n>|--top <n>] [--cursor <next_cursor>] [--max-json-bytes <n>] [--indexed-only] [--language <lang>|--extension <ext>|--alias <alias>] [--capability <all|none|graph|references|symbols|missing-any|missing-graph|missing-references|missing-symbols|search-only>]"),
-        ("batch", "cdidx batch [--db <path>] [--json-summary] [--max-input-lines <n>] [--max-output-chars <n>] [--parallel <n>]  # stdin is JSON Lines; --json-summary embeds typed child JSON plus a final summary"),
+        ("batch", "cdidx batch [--db <path>] [--json-summary] [--include-raw-streams] [--max-input-lines <n>] [--max-output-chars <n>] [--parallel <n>]  # stdin is JSON Lines; --json-summary embeds typed child JSON plus a final summary"),
         ("hooks-install", "cdidx hooks install [--project <path>] [--force] [--dry-run] [--json]"),
         ("hooks-uninstall", "cdidx hooks uninstall [--project <path>] [--force] [--json]"),
         ("hooks-status", "cdidx hooks status [--project <path>] [--json]"),