From d9ed5f349c3fd9ad3d293698044477b40e87b0d1 Mon Sep 17 00:00:00 2001 From: Widthdom Date: Sat, 25 Jul 2026 00:42:31 +0900 Subject: [PATCH] Stabilize console restoration assertions (#4800) --- TESTING_GUIDE.md | 4 ++-- changelog.d/unreleased/4800.fixed.md | 16 +++++++++++++++ tests/CodeIndex.Tests/ConsoleCaptureTests.cs | 21 ++++++++++++++------ 3 files changed, 33 insertions(+), 8 deletions(-) create mode 100644 changelog.d/unreleased/4800.fixed.md diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index b33a0c5a1..7c4663fee 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -25,7 +25,7 @@ Use the full suite by default. Use targeted filters only while iterating locally - Common direct test-only packages: `Microsoft.NET.Test.Sdk`, `xunit`, `xunit.runner.visualstudio`, `coverlet.collector`, `Microsoft.Data.Sqlite`, `FsCheck.Xunit` - These test-only packages are separate from the production dependency rule in `src/CodeIndex`, which still allows only `Microsoft.Data.Sqlite` at runtime. - `FsCheck.Xunit` is reserved for property-based tests that assert universal invariants (never-throws contracts, idempotence, "output is parseable by downstream consumer") across randomly generated inputs. Use it to complement, not replace, the example-based `[Fact]` / `[Theory]` tests — pick FsCheck when the property is a universally quantified claim, and an example test when a specific concrete case is the contract. -- Test parallelism: enabled by default across independent test classes. Tests that touch process-global state such as SQLite pool resets, environment variables, or current-directory overrides must use an explicit non-parallel collection. Console-sensitive classes share one xUnit collection so they remain serial with each other, but the collection may run beside independent classes; use `ConsoleCapture` for ordinary capture and keep every direct `Console.Out` / `Console.Error` swap under `TestConsoleLock.Gate`. That gate aliases the production `ConsoleStreamOwnership` gate so console synchronization and scoped production redirects cannot retain a test writer after its capture ends. +- Test parallelism: enabled by default across independent test classes. Tests that touch process-global state such as SQLite pool resets, environment variables, or current-directory overrides must use an explicit non-parallel collection. Console-sensitive classes share one xUnit collection so they remain serial with each other, but the collection may run beside independent classes; use `ConsoleCapture` for ordinary capture and keep every direct `Console.Out` / `Console.Error` swap under `TestConsoleLock.Gate`. Snapshot and assert global console writers under the same gate so another test cannot replace a writer between capture completion and the assertion. That gate aliases the production `ConsoleStreamOwnership` gate so console synchronization and scoped production redirects cannot retain a test writer after its capture ends. - Markdown unused-audit coverage indexes one real Markdown fixture containing common backtick and tilde fence-language markers. Keep default suppression, `documentation_surface` totals, reason tags, and `--all` recovery in that shared fixture. - Full-scan CLI and MCP no-op coverage treats one repository-wide reusable-stat snapshot read and one folded-readiness verification as performance contracts. Keep assertions for one snapshot read, one stat lookup per candidate, one folded verification, and no content load for unchanged files when changing incremental indexing. - Reference-graph refresh coverage treats graph-neutral indexing as a performance contract across incremental full scan, scoped `--files` update, and MCP indexing. Keep zero-refresh assertions for new and modified source files without symbols/references, plus a single batched refresh assertion when existing or new graph identity rows change. A healthy incremental generation must restrict identity/candidate/recursion work to transaction-committed dirty files, old and new `(language, folded name)` dependencies, and their old/new reciprocal edges; retain C#/Python language-transition and unchanged-target parity with a subsequent full refresh, rolled-back file batches, cancellation/retry, orphan-candidate cleanup, and the controlled 4,100-of-4,100 broad-scope fallback. Fresh/rebuild runs, missing identity contracts, and dirty sets of at least 4,096 references covering at least 50% of the graph must keep the full-refresh path. Query-plan coverage must keep all four scoped update phases and all ten candidate inserts on dirty-table-driven reference primary-key seeks, keep C# instantiate grouping on lookup names plus `idx_symbols_name_folded`, and prove that a sub-4,096 dirty set does not count the whole reference table without an explicit diagnostic hook. @@ -870,7 +870,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" - 主な直接参照の test-only package: `Microsoft.NET.Test.Sdk`、`xunit`、`xunit.runner.visualstudio`、`coverlet.collector`、`Microsoft.Data.Sqlite`、`FsCheck.Xunit` - これらの test-only package は `src/CodeIndex` の本番依存ルールとは別であり、runtime 側は引き続き `Microsoft.Data.Sqlite` のみを許容する。 - `FsCheck.Xunit` はランダム生成入力に対する普遍的不変条件（never-throws、idempotence、"出力が downstream consumer で parse 可能" 等）を表明する property-based テスト専用です。例ベースの `[Fact]` / `[Theory]` を置き換えるのではなく補完するもので、普遍量化された主張なら FsCheck、特定の具体ケースが契約なら例ベースという形で使い分けてください。 -- テスト並列実行: 独立したテストクラス間ではデフォルトで有効です。SQLite pool の解放、環境変数の変更、カレントディレクトリの上書きのような process-global 状態を触るテストは、明示的な non-parallel collection に入れてください。console-sensitive class は同じ xUnit collection 内で互いに直列実行しますが、独立 class とは並列に実行できます。通常の capture には `ConsoleCapture` を使い、`Console.Out` / `Console.Error` を直接差し替える場合は `TestConsoleLock.Gate` で保護してください。この gate は本番の `ConsoleStreamOwnership` gate と同一なので、console 同期処理や scoped redirect が capture 終了後も test writer を保持することを防ぎます。 +- テスト並列実行: 独立したテストクラス間ではデフォルトで有効です。SQLite pool の解放、環境変数の変更、カレントディレクトリの上書きのような process-global 状態を触るテストは、明示的な non-parallel collection に入れてください。console-sensitive class は同じ xUnit collection 内で互いに直列実行しますが、独立 class とは並列に実行できます。通常の capture には `ConsoleCapture` を使い、`Console.Out` / `Console.Error` を直接差し替える場合は `TestConsoleLock.Gate` で保護してください。global console writer の snapshot 取得と assertion も同じ gate 内で行い、capture 完了から assertion までの間に別のテストが writer を差し替えないようにします。この gate は本番の `ConsoleStreamOwnership` gate と同一なので、console 同期処理や scoped redirect が capture 終了後も test writer を保持することを防ぎます。 - Markdown の unused audit coverage は、一般的な backtick / tilde fence の language marker を含む実 Markdown fixture を1回 index します。同じ fixture で既定抑制、`documentation_surface` totals、reason tag、`--all` による復元を維持してください。 - full-scan CLI と MCP の no-op coverage は、リポジトリ全体の reusable-stat snapshot read と folded-readiness verification がそれぞれ 1 回であることを performance contract とします。incremental indexing を変更するときは、snapshot read が 1 回、候補ごとの stat lookup が 1 回、folded verification が 1 回、unchanged file の content load が 0 回という assertion を維持してください。 - reference-graph refresh coverage は、incremental full scan、scoped `--files` update、MCP indexing を横断する graph-neutral indexing を performance contract とします。symbol/reference を持たない新規・変更 source file では refresh 0 回を維持し、既存または新規の graph identity 行が変化する場合は batch 全体で refresh 1 回を assertion してください。健全な incremental generation では identity / candidate / recursion 処理を transaction commit 済みの dirty file、旧・新の `(language, folded name)` 依存、旧・新の逆辺に限定します。C# / Python の言語遷移、未変更targetを参照する新規callerと後続full refreshのparity、rollback file batch、cancel後retry、孤立candidate cleanup、4,100件中4,100件をdirtyにする制御broad-scope fallbackを維持してください。fresh/rebuild、identity契約欠落、または4,096件以上かつgraphの50%以上を占めるdirty集合ではfull-refresh経路を維持します。query-plan coverageでは、scoped updateの4 phaseとcandidate INSERT 10本をdirty table起点のreference主キーseekに保ち、C# instantiate groupingをlookup nameと`idx_symbols_name_folded`起点にし、明示的なdiagnostic hookがない4,096件未満のdirty集合ではreference table全件COUNTを行わないことを検証してください。 diff --git a/changelog.d/unreleased/4800.fixed.md b/changelog.d/unreleased/4800.fixed.md new file mode 100644 index 000000000..9c54679d7 --- /dev/null +++ b/changelog.d/unreleased/4800.fixed.md @@ -0,0 +1,16 @@ +--- +category: fixed +issues: + - 4800 +affected: + - tests/CodeIndex.Tests/ConsoleCaptureTests.cs + - TESTING_GUIDE.md +--- + +## English + +- **Console capture restoration assertions no longer race parallel tests (#4800)** — the timeout regression test now snapshots and verifies `Console.Out` and `Console.Error` while holding the shared console ownership gate, preventing another test from replacing a writer during the assertion. + +## 日本語 + +- **console capture の復元 assertion が並列テストと競合しなくなりました (#4800)** — timeout 回帰テストは共有 console ownership gate を保持した状態で `Console.Out` と `Console.Error` の snapshot 取得と検証を行うため、assertion 中に別のテストが writer を差し替える競合を防止します。 diff --git a/tests/CodeIndex.Tests/ConsoleCaptureTests.cs b/tests/CodeIndex.Tests/ConsoleCaptureTests.cs index 15b587253..1d562a6ac 100644 --- a/tests/CodeIndex.Tests/ConsoleCaptureTests.cs +++ b/tests/CodeIndex.Tests/ConsoleCaptureTests.cs @@ -195,8 +195,14 @@ await ConsoleCapture.CaptureAsync( [Fact] public async Task CaptureAsync_TimeoutRestoresOnlyAfterCallbackObservesCancellation_Issue4749() { - var originalOut = Console.Out; - var originalError = Console.Error; + TextWriter originalOut; + TextWriter originalError; + lock (TestConsoleLock.Gate) + { + originalOut = Console.Out; + originalError = Console.Error; + } + using var stdout = new StringWriter(); using var stderr = new StringWriter(); var callbackExited = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); @@ -220,9 +226,12 @@ public async Task CaptureAsync_TimeoutRestoresOnlyAfterCallbackObservesCancellat Assert.Contains("within", exception.Message, StringComparison.Ordinal); Assert.True(callbackExited.Task.IsCompletedSuccessfully); - Assert.Same(originalOut, Console.Out); - Assert.Same(originalError, Console.Error); - Assert.Null(Record.Exception(() => Console.Out.Flush())); - Assert.Null(Record.Exception(() => Console.Error.Flush())); + lock (TestConsoleLock.Gate) + { + Assert.Same(originalOut, Console.Out); + Assert.Same(originalError, Console.Error); + Assert.Null(Record.Exception(() => Console.Out.Flush())); + Assert.Null(Record.Exception(() => Console.Error.Flush())); + } } }