diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index b24c47251..6e96e357d 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -1765,6 +1765,7 @@ access. | Recipe SARIF | `search --recipe --format sarif` emits one result per bounded recipe result. Rule IDs use `recipe/query`; standard `fingerprints.cdidx/v1` values are derived from the normalized source location; result properties preserve recipe/query identity, severity, confidence, and per-query truncation; run properties preserve scope, applied result limits, aggregate counts, and conservative omitted-result metadata. `--max-json-bytes` preflights the complete schema-valid document and exact UTF-8 byte count, including escaping and the final newline, through a counting writer before materializing only the selected prefix. If the full document does not fit, serialization omits only whole trailing results and adds run/query source, emitted, omitted, byte-strategy, and replay metadata; emitted rules and locations remain intact. This truncation exits `11` unless `--allow-partial` is set. A cap below the zero-result document minimum emits no SARIF and reports the required bytes; explicit `--json` may place a versioned error object on stdout when that object fits the cap. Replay metadata removes the byte cap when a complete report exceeds the parser's maximum supported cap. Bound SARIF with `--limit` / `--total-limit`; row selectors such as `--sample`, `--first-per-file`, and `--per-file-limit` are rejected instead of being silently ignored. Recipe severity maps `critical` / `high` to `error`, `medium` to `warning`, and `low` / `info` to `note`. | | Recipe classifier output | Recipe run JSON may add `audit_classifications` to individual `CompactSearchResult` rows when a recipe classifier can classify the hit, and query/count payloads may add `classifier_counts` when classified rows are present. These fields are additive; use them to separate triage domains such as DTO/result-wrapper `.Result` properties versus Task/ValueTask blocking waits without changing the raw search query. | | NDJSON terminal records | Default NDJSON for `search`, `symbols`, and `files` appends one final `terminal_record` after result rows; search also emits it for zero-result responses, while raw `symbols` and `files` keep zero-result NDJSON empty. Recipe/audit search row streams share the same writer. Terminals report returned and observed total counts, `total_count_authoritative` / `total_count_lower_bound`, selection or interruption reason, applied limits, omitted rows, and recovery guidance. `--max-json-bytes` covers the complete stdout stream, including newlines and this terminal record; when additive selector-accounting fields prevent the terminal from fitting, the writer omits those optional fields before declaring the terminal impossible. A cap that still cannot fit the terminal fails before stdout. Capped output rejects `--profile`, `--verbose`, and `--json-envelope`. Byte-cap partial output exits with `CommandExitCodes.PartialResult` (`11`) unless `--allow-partial` explicitly opts into exit `0`. `--results-only` is the explicit terminal-record opt-out for these NDJSON row streams and is rejected with array, compact, summary, or count output. | +| C# outline callable display | `DbSymbolReader.Outline` derives `display_name` only at read time and never changes canonical `symbols.name`, qualified paths, folded identity, or exact-query aliases. A complete C# generic method signature normally uses arity placeholders (`` or ``); if one would collide with a concrete parameter type, it deterministically selects collision-free `TArg` placeholders. Replacement applies only to unqualified method-type-parameter references, preserving qualified concrete types and escaped-keyword distinctions. Literal-aware scanning keeps delimiters in attributes and default values from changing parameter boundaries. The display omits `where` constraints, drops non-identity `this` / `params` / `scoped`, and retains overload-significant `ref` / `out` / `in` (including `ref readonly`). Non-generic and non-C# formatting stays on the existing path. Missing, truncated, or syntactically incomplete persisted signatures retain the legacy `Name@line` fallback for old-index compatibility. | | `outline` / `unused` cursor binding | `outline --json` accepts `--kind `, `--limit` / `--top`, opaque `--cursor `, and `--outline-fields ` for bounded machine output. Controlled outline responses keep the normal envelope and add `total_symbol_count`, `returned_symbol_count`, `cursor_offset`, `next_cursor`, `has_more`, and `result_stable_at`, plus `kind_filter` and `selected_fields` when active. Projection parsing canonicalizes aliases and removes duplicates before validation; unknown field names are reported together as one terminal usage error with valid candidates, while the empty-selection error is reserved for deliberately empty CSV input. `outline` and `unused` cursors bind their offset to the normalized path/scope, filters, ordering, and index generation; reuse after changing those inputs or refreshing the index fails with explicit restart-required guidance. Legacy `outline:` / `unused:` inputs remain accepted for transition, but every newly emitted cursor is opaque and bound. | | `hotspots --json` grouping semantics | `hotspots` and MCP `symbol_hotspots` emit `grouped_by`, `grouping_unit`, `count_kind`, `limit_applies_to`, `score_fields`, `ranking_fields`, and matching `query_context` fields. `--limit` applies to returned symbols, files, name/kind groups, or SQL statements; `--count` ignores `--limit` and reports total groups. Explicit `statement` grouping is SQL-only (`--lang sql` / `lang: "sql"`). | | `--json-envelope` commands | Applies to `search`, `definition`, `references`, `callers`, `callees`, `symbols`, `files`, `find`, `excerpt`, `map`, `inspect`, `outline`, `status`, `validate`, `languages`, `impact`, `deps`, `unused`, and `hotspots`. | @@ -5239,6 +5240,7 @@ help はすべてこのレジストリを参照します。field 名は大文字 | Recipe SARIF | `search --recipe --format sarif` は、上限付き recipe result ごとに result を1件出力します。rule ID は `recipe/query` を使い、標準の `fingerprints.cdidx/v1` は正規化済み source location から導出します。result properties は recipe/query identity、severity、confidence、query ごとの truncation を保持し、run properties は scope、適用済み result limit、集計 count、保守的な omitted-result metadata を保持します。`--max-json-bytes` は escape と末尾改行を含む schema-valid な完全 document と正確な UTF-8 byte 数を counting writer で検査し、選択した prefix だけを materialize します。完全な document が収まらない場合、末尾の result だけを1件単位で省略し、run / query に source、emitted、omitted、byte strategy、replay metadata を追加します。出力済み result の rule と location は維持し、この truncation は `--allow-partial` がなければ `11` を返します。result 0件の document の最小値より小さい cap では SARIF を出力せず必要 byte 数を報告し、明示的な `--json` では error object 自体が cap に収まる場合に version 付き error を stdout へ出力することがあります。完全な report が parser の対応可能な最大 cap を超える場合、replay metadata は byte cap を外します。SARIF の上限には `--limit` / `--total-limit` も使い、`--sample`、`--first-per-file`、`--per-file-limit` のような row selector は黙って無視せず拒否します。recipe severity は `critical` / `high` を `error`、`medium` を `warning`、`low` / `info` を `note` に対応付けます。 | | Recipe classifier output | recipe classifier が hit を分類できる場合、recipe run JSON は個別の `CompactSearchResult` row に `audit_classifications` を追加することがあり、分類済み row がある query / count payload は `classifier_counts` を追加することがあります。これらは additive field です。raw search query を変えずに、DTO / result-wrapper の `.Result` property と Task / ValueTask の blocking wait などの triage domain を分離するために使います。 | | NDJSON terminal record | `search`、`symbols`、`files` の既定 NDJSON は result row の後に最後の `terminal_record` を 1 件追加します。`search` は 0 件応答にも終端を出力しますが、raw `symbols` / `files` の 0 件 NDJSON は空のままです。recipe / audit search の row stream も同じ writer を使います。終端は返却件数と観測済み総件数、`total_count_authoritative` / `total_count_lower_bound`、selection または中断理由、適用上限、省略行数、復旧案内を報告します。`--max-json-bytes` は改行と終端レコードを含む stdout stream 全体を対象にし、追加 selector-accounting field が原因で終端が収まらない場合は、終端自体を不可能と判定する前にそれらの任意 field を省略します。それでも終端が収まらない cap は stdout 出力前に失敗します。上限付き出力は `--profile`、`--verbose`、`--json-envelope` を拒否します。byte cap による部分出力は、`--allow-partial` で終了コード `0` を明示許可しない限り `CommandExitCodes.PartialResult`（`11`）を返します。`--results-only` はこれらの NDJSON row stream から終端レコードを明示的に除外するための option であり、array / compact / summary / count 出力との組み合わせは拒否されます。 | +| C# outline callable 表示 | `DbSymbolReader.Outline` は read 時にだけ `display_name` を導出し、canonical な `symbols.name`、qualified path、folded identity、完全一致 query alias は変更しません。完全な C# generic method signature は通常 arity placeholder（`` または ``）で表示し、具体的な parameter type と衝突する場合は決定的で衝突しない `TArg` placeholder を選びます。置換対象は修飾されていない method type parameter の参照だけであり、修飾された具体型と escaped keyword の区別は保持します。literal-aware な走査により、attribute と既定値内の区切り文字は parameter 境界を変えません。`where` constraint と identity に影響しない `this` / `params` / `scoped` は省略し、overload を区別する `ref` / `out` / `in`（`ref readonly` を含む）は保持します。非 generic および C# 以外の formatting は既存経路のままです。永続 signature が欠落、切り詰め、または構文的に不完全な場合は、旧 index 互換のため legacy `Name@line` fallback を維持します。 | | `outline` / `unused` cursor の束縛 | `outline --json` は bounded な機械向け出力として `--kind `、`--limit` / `--top`、opaque な `--cursor `、`--outline-fields ` を受け付けます。制御付き outline 応答は通常の envelope を維持し、`total_symbol_count`、`returned_symbol_count`、`cursor_offset`、`next_cursor`、`has_more`、`result_stable_at` を追加し、該当時は `kind_filter` と `selected_fields` も返します。projection parser は検証前に alias を canonicalize して重複を除きます。未知field名はvalid候補を伴う1つの終端usage errorにまとめ、empty-selection errorは意図的に空のCSV入力にだけ使います。`outline` と `unused` の cursor は offset を正規化済み path/scope、filter、ordering、index generation に束縛するため、条件変更後または index 更新後の再利用は restart-required の明示案内付きで失敗します。移行用に legacy の `outline:` / `unused:` 入力は受理しますが、新しく出力する cursor はすべて opaque かつ束縛済みです。 | | `hotspots --json` grouping semantics | `hotspots` と MCP `symbol_hotspots` は `grouped_by`、`grouping_unit`、`count_kind`、`limit_applies_to`、`score_fields`、`ranking_fields` と、対応する `query_context` field を返します。`--limit` は返却される symbol、file、name/kind group、SQL statement に適用されます。`--count` は `--limit` を無視し、total group 数を返します。明示的な `statement` grouping は SQL 専用です（`--lang sql` / `lang: "sql"`）。 | | `--json-envelope` 対象 command | `search`、`definition`、`references`、`callers`、`callees`、`symbols`、`files`、`find`、`excerpt`、`map`、`inspect`、`outline`、`status`、`validate`、`languages`、`impact`、`deps`、`unused`、`hotspots`。 | diff --git a/README.md b/README.md index c55ebb1b0..7cf890e8a 100644 --- a/README.md +++ b/README.md @@ -196,6 +196,16 @@ edges, reports structured `file_errors`, and exits with partial-result code `11` Use `--allow-partial` only when automation deliberately accepts exit `0` for that incomplete generation; JSON still reports `status: "partial"`. +`outline` derives callable display names from persisted signatures without changing +canonical symbol names, paths, or exact-query aliases. C# generic methods normally use +stable arity placeholders (`` or ``), choosing deterministic +collision-free `TArg` placeholders when a concrete type name would be ambiguous. The +reader substitutes method-type-parameter references without rewriting qualified types, +parses attributes and default-value literals lexically, omits constraints, and retains +overload-significant `ref` / `out` / `in` modifiers. Older indexes whose callable +signature is missing, truncated, or incomplete retain the compatibility fallback +`Name@line` until they are reindexed. + ## Highlights | Area | What to use | @@ -714,6 +724,15 @@ commit し、構造化 `file_errors` を返して partial-result 終了コード 不完全 generation の終了コード `0` を automation が意図的に許容するときだけ `--allow-partial` を指定してください。JSON の `status: "partial"` は維持されます。 +`outline` は canonical symbol name、path、完全一致 query alias を変更せず、永続化済み +signature から callable の表示名を導出します。C# generic method は通常、安定した arity +placeholder（`` または ``）を使い、具体的な型名と衝突する場合は決定的で +衝突しない `TArg` placeholder を選びます。method type parameter の参照だけを置換して修飾型は +書き換えず、attribute と既定値 literal は字句として解析します。constraint は省略しつつ +overload を区別する `ref` / `out` / `in` modifier は保持します。callable signature が欠落、 +切り詰め、または不完全な旧 index は、再 index されるまで互換 fallback の `Name@line` を +維持します。 + ## 特長 | 分野 | 使うもの | diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 7c6a16fbb..3027d1e7d 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -475,6 +475,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding - No-argument parameter attributes on methods, delegates, and lambdas share one C# fixture because all three exercise the same section-local parenthesis-depth rule. - Argument-bearing parameter attributes share one method fixture for inline and line-broken declaration layouts. - C# parameter and argument-list modifier coverage keeps multiline declaration and call sites for `out` / `ref` / `in` / `params` / `this` / `scoped`, multiple modifier fragments on one continuation line, final `)`-closing fragments, nested lambda arguments, `out var`, following generic types, ref returns, and ref structs in one extractor fixture. The indexed reader fixture separately proves raw `type_reference` queries exclude modifiers while resolved following-type edges remain graph-queryable. +- C# generic outline display coverage stays in one `DbReaderTests` fixture that shares extraction and reader setup across generic/non-generic overloads, extension/local/explicit-interface methods, nested generic tuple/nullable/array parameter types, constraints, overload-significant modifiers, qualified/concrete-name collisions, escaped and Unicode identifiers, and delimiters inside attribute/default literals. Keep the paired moved-line fixture byte-identical except for leading newlines, assert canonical names/paths remain unchanged, and retain one manually persisted incomplete-signature row for the old-index `Name@line` fallback (#4916). - Direct and `global::` static type qualifiers share one C# fixture while retaining per-container reference assertions. - Static qualifiers in using statements and field access share one consumer fixture and extraction pass. - Namespace-qualified and Pascal-cased instance-member chains share one qualifier fixture with a rightmost static type reference, so positive and negative qualifier outcomes are checked after one parse. @@ -1042,6 +1043,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" YAML sequence hierarchy の fixture では、sequence item を path 専用のまま保ち、empty / scalar / alias item を安定した index に数え、anchor と block scalar の境界を維持し、論理 qualified path と最寄りの出力済み symbol parent を分けて検証し、旧 extractor contract が stamp された未変更 YAML row を通常の full scan が更新することも確認してください。 C# の declaration boundary に関する regression では、extractor を直接呼ぶ fixture と、実 index に対する `symbols --exact-name` query を組み合わせてください。呼び出し・parameter の continuation と、正当な複数行 method、constructor、delegate、local function を同居させ、false positive の拒否と宣言 range の両方を観測可能にします。 C# の明示的 interface identity coverage では、method、property、event、indexer の extractor assertion と、永続化後の修飾 / 非修飾の完全一致 query、fold rewrite 検証、inspect / outline、LSP の definition / reference scope を組み合わせます。複数および継承 interface、generic arity、同名 public member に加え、明示的 interface qualifier と誤認してはならない修飾 return type を含めてください。 + C# generic outline 表示の coverage は、generic / 非 generic overload、extension / local / explicit-interface method、nested generic の tuple / nullable / array parameter type、constraint、overload を区別する modifier、修飾型 / 具体型名との衝突、escaped / Unicode identifier、attribute / 既定値 literal 内の区切り文字を、抽出と reader setup を共有する1つの `DbReaderTests` fixture に維持します。line 移動を検証する対の fixture は先頭改行以外を同一にし、canonical name / path が不変であることを assertion に含め、旧 index の `Name@line` fallback 用に不完全 signature を手動永続化した row も1件残してください（#4916）。 C# の callable containment fixture では、block body の test method、local / nested local function、named lambda、expression-bodied member、nested type を同居させ、symbol の親と call reference の container の両方を検証してください。 repository metadata の coverage は `SymbolExtractorRepositoryMetadataTests.cs` と `ReferenceExtractorRepositoryMetadataTests.cs` に置きます。TOML、JSON Lines、ignore / attributes、EditorConfig、`.rules`、application manifest の capability assertion を、保守的な local-path 抽出と malformed-record control に同期させてください。 未対応言語を必要とする capability regression fixture には明示的な `text` placeholder または ambiguity bucket を使い、認識済み repository metadata 形式を未対応 control に使わないでください。 diff --git a/changelog.d/unreleased/4916.fixed.md b/changelog.d/unreleased/4916.fixed.md new file mode 100644 index 000000000..8769f568a --- /dev/null +++ b/changelog.d/unreleased/4916.fixed.md @@ -0,0 +1,19 @@ +--- +category: fixed +issues: + - 4916 +affected: + - src/CodeIndex/Database/DbSymbolReader.Outline.cs + - tests/CodeIndex.Tests/DbReaderTests.cs + - README.md + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **C# generic methods now have stable readable outline signatures (#4916)** — `outline` normalizes generic arity and unqualified type-parameter uses with collision-free placeholders while retaining concrete parameter types and overload-significant modifiers; literal-aware parsing keeps attribute and default-value delimiters stable, so moving a method no longer changes a generic callable display from a line-based fallback, while canonical names and exact navigation remain unchanged. + +## 日本語 + +- **C# generic method の outline signature が安定した読みやすい表示になりました (#4916)** — `outline` は generic arity と未修飾の type-parameter 使用箇所を衝突しない placeholder で正規化し、具体的な parameter type と overload を区別する modifier を保持します。literal-aware な解析により attribute と既定値内の区切り文字も安定するため、method の行を移動しても generic callable の表示が行番号 fallback によって変化しません。canonical name と完全一致 navigation は従来どおりです。 diff --git a/src/CodeIndex/Database/DbSymbolReader.Outline.cs b/src/CodeIndex/Database/DbSymbolReader.Outline.cs index 3e4fe46a2..0af0d99b0 100644 --- a/src/CodeIndex/Database/DbSymbolReader.Outline.cs +++ b/src/CodeIndex/Database/DbSymbolReader.Outline.cs @@ -1,3 +1,5 @@ +using System.Globalization; +using System.Text; using System.Text.RegularExpressions; using CodeIndex.Indexer; using Microsoft.Data.Sqlite; @@ -218,25 +220,100 @@ private static bool IsCallableOutlineSymbol(string kind) if (string.IsNullOrWhiteSpace(signature)) return null; - var openParen = FindCallableParameterOpenParen(signature, name); + var normalizedSignature = lang == "csharp" + ? NormalizeCSharpSignatureForOutline(signature) + : signature; + var openParen = FindCallableParameterOpenParen( + normalizedSignature, + name, + lang, + out var csharpTypeParameters); if (openParen < 0) return null; - var closeParen = FindMatchingParen(signature, openParen); + var closeParen = lang == "csharp" + ? FindMatchingCSharpDelimiter(normalizedSignature, openParen, '(', ')') + : FindMatchingParen(normalizedSignature, openParen); if (closeParen < 0) return null; - var parameters = signature.Substring(openParen + 1, closeParen - openParen - 1); - var parameterLabels = SplitTopLevelParameters(parameters) - .Select(parameter => SimplifyParameterForOutline(parameter, lang)) + var parameters = normalizedSignature.Substring(openParen + 1, closeParen - openParen - 1); + var splitParameters = (lang == "csharp" + ? SplitTopLevelCSharpParameters(parameters) + : SplitTopLevelParameters(parameters)) + .ToList(); + if (csharpTypeParameters == null) + { + var ordinaryParameterLabels = splitParameters + .Select(parameter => SimplifyParameterForOutline(parameter, lang)) + .Where(parameter => !string.IsNullOrWhiteSpace(parameter)); + return $"{name}({string.Join(", ", ordinaryParameterLabels)})"; + } + + var simplifiedParameters = splitParameters + .Select(SimplifyCSharpGenericParameterForOutline) .Where(parameter => !string.IsNullOrWhiteSpace(parameter)) .ToList(); + var normalizedTypeParameters = BuildNormalizedCSharpTypeParameterMap( + csharpTypeParameters, + simplifiedParameters); + var parameterLabels = simplifiedParameters + .Select(parameter => ReplaceCSharpTypeParameterTokens( + parameter, + csharpTypeParameters, + normalizedTypeParameters)) + .ToList(); + var displayName = name + BuildNormalizedCSharpTypeParameterSuffix( + csharpTypeParameters, + normalizedTypeParameters); + return $"{displayName}({string.Join(", ", parameterLabels)})"; + } + + private readonly record struct CSharpTypeParameterName(string Name, bool RequiresEscape); + + private static string NormalizeCSharpSignatureForOutline(string signature) + { + var builder = new StringBuilder(signature.Length); + var segmentStart = 0; + var index = 0; + while (index < signature.Length) + { + if (!TryGetCSharpLexicalRegionEnd(signature, index, out var regionEnd)) + { + index++; + continue; + } + + AppendNormalizedCSharpSignatureSegment(builder, signature, segmentStart, index); + builder.Append(signature, index, regionEnd - index); + index = regionEnd; + segmentStart = regionEnd; + } - return $"{name}({string.Join(", ", parameterLabels)})"; + AppendNormalizedCSharpSignatureSegment(builder, signature, segmentStart, signature.Length); + return builder.ToString(); } - private static int FindCallableParameterOpenParen(string signature, string name) + private static void AppendNormalizedCSharpSignatureSegment( + StringBuilder builder, + string signature, + int start, + int end) { + if (end <= start) + return; + + var segment = signature[start..end]; + builder.Append(ExactSourceSearchNormalizer.NormalizeCSharpUnicodeEscapes(segment, out _)); + } + + private static int FindCallableParameterOpenParen( + string signature, + string name, + string? lang, + out IReadOnlyList? csharpTypeParameters) + { + csharpTypeParameters = null; var searchStart = 0; while (searchStart < signature.Length) { @@ -245,19 +322,701 @@ private static int FindCallableParameterOpenParen(string signature, string name) return -1; var afterName = nameIndex + name.Length; + var tokenStart = nameIndex > 0 && signature[nameIndex - 1] == '@' + ? nameIndex - 1 + : nameIndex; + var hasIdentifierBoundary = + (tokenStart == 0 || !IsCSharpIdentifierPart(signature[tokenStart - 1])) + && (afterName >= signature.Length || !IsCSharpIdentifierPart(signature[afterName])); + if (!hasIdentifierBoundary) + { + searchStart = afterName; + continue; + } + var cursor = afterName; - while (cursor < signature.Length && char.IsWhiteSpace(signature[cursor])) - cursor++; + if (lang == "csharp") + { + cursor = SkipCSharpWhitespaceAndComments(signature, cursor); + } + else + { + while (cursor < signature.Length && char.IsWhiteSpace(signature[cursor])) + cursor++; + } if (cursor < signature.Length && signature[cursor] == '(') return cursor; + if (lang == "csharp" && cursor < signature.Length && signature[cursor] == '<') + { + var closeAngle = FindMatchingCSharpDelimiter(signature, cursor, '<', '>'); + if (closeAngle > cursor + && TryReadCSharpTypeParameterNames( + signature[(cursor + 1)..closeAngle], + out var typeParameters)) + { + var parameterOpen = SkipCSharpWhitespaceAndComments(signature, closeAngle + 1); + + if (parameterOpen < signature.Length && signature[parameterOpen] == '(') + { + csharpTypeParameters = typeParameters; + return parameterOpen; + } + } + } + searchStart = afterName; } return -1; } + private static int SkipCSharpWhitespaceAndComments(string value, int start) + { + var cursor = start; + while (cursor < value.Length) + { + while (cursor < value.Length && char.IsWhiteSpace(value[cursor])) + cursor++; + + if (cursor + 1 >= value.Length + || value[cursor] != '/' + || value[cursor + 1] is not ('/' or '*') + || !TryGetCSharpLexicalRegionEnd(value, cursor, out var commentEnd)) + { + break; + } + + cursor = commentEnd; + } + + return cursor; + } + + private static int FindMatchingCSharpDelimiter( + string value, + int openIndex, + char openDelimiter, + char closeDelimiter) + { + var depth = 0; + var squareBracketDepth = 0; + for (var i = openIndex; i < value.Length; i++) + { + if (TryGetCSharpLexicalRegionEnd(value, i, out var regionEnd)) + { + i = regionEnd - 1; + continue; + } + + if (openDelimiter == '<') + { + if (value[i] == '[') + { + squareBracketDepth++; + continue; + } + + if (value[i] == ']' && squareBracketDepth > 0) + { + squareBracketDepth--; + continue; + } + + if (squareBracketDepth > 0) + continue; + } + + if (value[i] == openDelimiter) + { + depth++; + } + else if (value[i] == closeDelimiter) + { + depth--; + if (depth == 0) + return i; + } + } + + return -1; + } + + private static bool TryGetCSharpLexicalRegionEnd(string value, int start, out int end) + { + end = start; + if (start + 1 < value.Length && value[start] == '/') + { + if (value[start + 1] == '/') + { + var newline = value.IndexOf('\n', start + 2); + end = newline < 0 ? value.Length : newline + 1; + return true; + } + + if (value[start + 1] == '*') + { + var close = value.IndexOf("*/", start + 2, StringComparison.Ordinal); + end = close < 0 ? value.Length : close + 2; + return true; + } + } + + if (value[start] == '\'') + { + end = FindCSharpQuotedLiteralEnd(value, start, '\'', verbatim: false); + return true; + } + + var quoteStart = -1; + var verbatim = false; + var interpolated = false; + if (value[start] == '"') + { + quoteStart = start; + } + else if (value[start] == '@' && start + 1 < value.Length && value[start + 1] == '"') + { + quoteStart = start + 1; + verbatim = true; + } + else if (value[start] == '$') + { + interpolated = true; + var cursor = start; + while (cursor < value.Length && value[cursor] == '$') + cursor++; + if (cursor < value.Length && value[cursor] == '@') + { + verbatim = true; + cursor++; + } + + if (cursor < value.Length && value[cursor] == '"') + quoteStart = cursor; + } + else if (value[start] == '@' + && start + 2 < value.Length + && value[start + 1] == '$' + && value[start + 2] == '"') + { + quoteStart = start + 2; + verbatim = true; + interpolated = true; + } + + if (quoteStart < 0) + return false; + + var quoteCount = 1; + while (quoteStart + quoteCount < value.Length && value[quoteStart + quoteCount] == '"') + quoteCount++; + end = quoteCount >= 3 + ? FindCSharpRawStringEnd(value, quoteStart, quoteCount) + : interpolated + ? FindCSharpInterpolatedStringEnd(value, quoteStart, verbatim) + : FindCSharpQuotedLiteralEnd(value, quoteStart, '"', verbatim); + return true; + } + + private static int FindCSharpInterpolatedStringEnd( + string value, + int quoteStart, + bool verbatim) + { + var interpolationDepth = 0; + for (var i = quoteStart + 1; i < value.Length; i++) + { + if (interpolationDepth > 0 + && TryGetCSharpLexicalRegionEnd(value, i, out var regionEnd)) + { + i = regionEnd - 1; + continue; + } + + if (interpolationDepth == 0) + { + if (verbatim && value[i] == '"' && i + 1 < value.Length && value[i + 1] == '"') + { + i++; + continue; + } + + if (!verbatim && value[i] == '\\') + { + i++; + continue; + } + + if (value[i] == '"') + return i + 1; + + if (value[i] == '{') + { + if (i + 1 < value.Length && value[i + 1] == '{') + { + i++; + continue; + } + + interpolationDepth = 1; + } + + continue; + } + + if (value[i] == '{') + { + interpolationDepth++; + } + else if (value[i] == '}') + { + interpolationDepth--; + } + } + + return value.Length; + } + + private static int FindCSharpQuotedLiteralEnd( + string value, + int quoteStart, + char quote, + bool verbatim) + { + for (var i = quoteStart + 1; i < value.Length; i++) + { + if (verbatim && value[i] == quote && i + 1 < value.Length && value[i + 1] == quote) + { + i++; + continue; + } + + if (!verbatim && value[i] == '\\') + { + i++; + continue; + } + + if (value[i] == quote) + return i + 1; + } + + return value.Length; + } + + private static int FindCSharpRawStringEnd(string value, int quoteStart, int quoteCount) + { + for (var i = quoteStart + quoteCount; i < value.Length; i++) + { + if (value[i] != '"') + continue; + + var runLength = 1; + while (i + runLength < value.Length && value[i + runLength] == '"') + runLength++; + if (runLength >= quoteCount) + return i + quoteCount; + + i += runLength - 1; + } + + return value.Length; + } + + private static bool TryReadCSharpTypeParameterNames( + string typeParameterList, + out IReadOnlyList typeParameters) + { + var parsed = new List(); + var seen = new HashSet(StringComparer.Ordinal); + foreach (var parameter in SplitTopLevelCSharpParameters(typeParameterList)) + { + var name = ReadTrailingCSharpIdentifier(parameter); + if (name == null || !seen.Add(name.Value.Name)) + { + typeParameters = Array.Empty(); + return false; + } + + parsed.Add(name.Value); + } + + typeParameters = parsed; + return parsed.Count > 0; + } + + private static CSharpTypeParameterName? ReadTrailingCSharpIdentifier(string value) + { + var end = value.Length; + while (end > 0 && char.IsWhiteSpace(value[end - 1])) + end--; + + for (var start = 0; start < end;) + { + if (!TryReadCSharpIdentifierToken( + value, + start, + out var tokenEnd, + out var identifier, + out var escaped)) + { + start++; + continue; + } + + if (tokenEnd == end) + return new CSharpTypeParameterName( + identifier, + escaped && IsCSharpReservedKeyword(identifier)); + + start = tokenEnd; + } + + return null; + } + + private static bool IsCSharpReservedKeyword(string value) + { + return value is + "abstract" or "as" or "base" or "bool" or "break" or "byte" or + "case" or "catch" or "char" or "checked" or "class" or "const" or "continue" or + "decimal" or "default" or "delegate" or "do" or "double" or + "else" or "enum" or "event" or "explicit" or "extern" or + "false" or "finally" or "fixed" or "float" or "for" or "foreach" or + "goto" or "if" or "implicit" or "in" or "int" or "interface" or "internal" or "is" or + "lock" or "long" or "namespace" or "new" or "null" or + "object" or "operator" or "out" or "override" or + "params" or "private" or "protected" or "public" or + "readonly" or "ref" or "return" or + "sbyte" or "sealed" or "short" or "sizeof" or "stackalloc" or "static" or + "string" or "struct" or "switch" or + "this" or "throw" or "true" or "try" or "typeof" or + "uint" or "ulong" or "unchecked" or "unsafe" or "ushort" or "using" or + "virtual" or "void" or "volatile" or "while"; + } + + private static Dictionary BuildNormalizedCSharpTypeParameterMap( + IReadOnlyList typeParameters, + IReadOnlyList parameterTypes) + { + var typeParameterLookup = typeParameters.ToDictionary( + parameter => parameter.Name, + StringComparer.Ordinal); + var reservedNames = new HashSet(StringComparer.Ordinal); + foreach (var parameterType in parameterTypes) + { + for (var index = 0; index < parameterType.Length;) + { + if (!TryReadCSharpIdentifierToken( + parameterType, + index, + out var tokenEnd, + out var identifier, + out var escaped)) + { + index++; + continue; + } + + if (!IsCSharpTypeParameterReference( + parameterType, + index, + tokenEnd, + identifier, + escaped, + typeParameterLookup)) + { + reservedNames.Add(identifier); + } + + index = tokenEnd; + } + } + + var placeholders = ChooseCSharpTypeParameterPlaceholders(typeParameters.Count, reservedNames); + + var normalized = new Dictionary(typeParameters.Count, StringComparer.Ordinal); + for (var i = 0; i < typeParameters.Count; i++) + normalized[typeParameters[i].Name] = placeholders[i]; + + return normalized; + } + + private static IReadOnlyList ChooseCSharpTypeParameterPlaceholders( + int arity, + IReadOnlySet reservedNames) + { + for (var attempt = 0; ; attempt++) + { + var prefix = attempt switch + { + 0 => "T", + 1 => "TArg", + _ => $"TArg{attempt}", + }; + var candidates = Enumerable.Range(1, arity) + .Select(index => arity == 1 && attempt == 0 ? prefix : $"{prefix}{index}") + .ToList(); + if (candidates.All(candidate => !reservedNames.Contains(candidate))) + return candidates; + } + } + + private static string BuildNormalizedCSharpTypeParameterSuffix( + IReadOnlyList typeParameters, + IReadOnlyDictionary normalizedTypeParameters) + { + return $"<{string.Join(", ", typeParameters.Select( + parameter => normalizedTypeParameters[parameter.Name]))}>"; + } + + private static string SimplifyCSharpGenericParameterForOutline(string parameter) + { + var cleaned = RemoveCSharpDefaultValue(parameter).Trim(); + while (cleaned.Length > 0 && cleaned[0] == '[') + { + var attributeEnd = FindMatchingCSharpDelimiter(cleaned, 0, '[', ']'); + if (attributeEnd < 0) + return string.Empty; + cleaned = cleaned[(attributeEnd + 1)..].TrimStart(); + } + + if (cleaned.Length == 0) + return string.Empty; + + var overloadModifiers = new List(2); + while (TryReadLeadingWord(cleaned, out var modifier, out var remainder)) + { + if (modifier is "ref" or "out" or "in") + { + overloadModifiers.Add(modifier); + } + else if (modifier == "readonly" + && overloadModifiers.Count > 0 + && overloadModifiers[^1] == "ref") + { + overloadModifiers.Add(modifier); + } + else if (modifier is not ("this" or "params" or "scoped")) + { + break; + } + + cleaned = remainder.TrimStart(); + } + + var parts = cleaned.Split(' ', StringSplitOptions.RemoveEmptyEntries); + var typeName = parts.Length <= 1 + ? cleaned + : string.Join(" ", parts.Take(parts.Length - 1)); + if (typeName.Length == 0) + return string.Empty; + + return overloadModifiers.Count == 0 + ? typeName + : $"{string.Join(" ", overloadModifiers)} {typeName}"; + } + + private static string RemoveCSharpDefaultValue(string parameter) + { + var angleDepth = 0; + var parenDepth = 0; + var bracketDepth = 0; + var braceDepth = 0; + for (var i = 0; i < parameter.Length; i++) + { + if (TryGetCSharpLexicalRegionEnd(parameter, i, out var regionEnd)) + { + i = regionEnd - 1; + continue; + } + + switch (parameter[i]) + { + case '<' when bracketDepth == 0: + angleDepth++; + break; + case '>' when bracketDepth == 0: + if (angleDepth > 0) angleDepth--; + break; + case '(': + parenDepth++; + break; + case ')': + if (parenDepth > 0) parenDepth--; + break; + case '[': + bracketDepth++; + break; + case ']': + if (bracketDepth > 0) bracketDepth--; + break; + case '{': + braceDepth++; + break; + case '}': + if (braceDepth > 0) braceDepth--; + break; + case '=' when angleDepth == 0 + && parenDepth == 0 + && bracketDepth == 0 + && braceDepth == 0: + return parameter[..i]; + } + } + + return parameter; + } + + private static bool TryReadLeadingWord(string value, out string word, out string remainder) + { + var end = 0; + while (end < value.Length && IsCSharpIdentifierPart(value[end])) + end++; + + if (end == 0 || (end < value.Length && !char.IsWhiteSpace(value[end]))) + { + word = string.Empty; + remainder = value; + return false; + } + + word = value[..end]; + remainder = value[end..]; + return true; + } + + private static string ReplaceCSharpTypeParameterTokens( + string typeName, + IReadOnlyList typeParameters, + IReadOnlyDictionary normalizedTypeParameters) + { + var typeParameterLookup = typeParameters.ToDictionary( + parameter => parameter.Name, + StringComparer.Ordinal); + var builder = new StringBuilder(typeName.Length); + for (var i = 0; i < typeName.Length;) + { + if (!TryReadCSharpIdentifierToken( + typeName, + i, + out var tokenEnd, + out var identifier, + out var escaped)) + { + builder.Append(typeName[i]); + i++; + continue; + } + + if (IsCSharpTypeParameterReference( + typeName, + i, + tokenEnd, + identifier, + escaped, + typeParameterLookup) + && normalizedTypeParameters.TryGetValue(identifier, out var normalized)) + { + builder.Append(normalized); + } + else + { + builder.Append(typeName, i, tokenEnd - i); + } + + i = tokenEnd; + } + + return builder.ToString(); + } + + private static bool IsCSharpTypeParameterReference( + string value, + int tokenStart, + int tokenEnd, + string identifier, + bool escaped, + IReadOnlyDictionary typeParameters) + { + if (!typeParameters.TryGetValue(identifier, out var typeParameter) + || (typeParameter.RequiresEscape && !escaped)) + { + return false; + } + + var previous = tokenStart - 1; + while (previous >= 0 && char.IsWhiteSpace(value[previous])) + previous--; + if (previous >= 0 && value[previous] is '.' or ':') + return false; + + var next = tokenEnd; + while (next < value.Length && char.IsWhiteSpace(value[next])) + next++; + return next + 1 >= value.Length || value[next] != ':' || value[next + 1] != ':'; + } + + private static bool TryReadCSharpIdentifierToken( + string value, + int start, + out int end, + out string identifier, + out bool escaped) + { + end = start; + identifier = string.Empty; + escaped = false; + if (start >= value.Length) + return false; + + var identifierStart = start; + if (value[identifierStart] == '@') + { + escaped = true; + identifierStart++; + } + + if (identifierStart >= value.Length || !IsCSharpIdentifierStart(value[identifierStart])) + return false; + + end = identifierStart + 1; + while (end < value.Length && IsCSharpIdentifierPart(value[end])) + end++; + + identifier = value[identifierStart..end]; + return true; + } + + private static bool IsCSharpIdentifierStart(char value) + { + if (value == '_' || char.IsSurrogate(value)) + return true; + + return char.GetUnicodeCategory(value) is + UnicodeCategory.UppercaseLetter or + UnicodeCategory.LowercaseLetter or + UnicodeCategory.TitlecaseLetter or + UnicodeCategory.ModifierLetter or + UnicodeCategory.OtherLetter or + UnicodeCategory.LetterNumber; + } + + private static bool IsCSharpIdentifierPart(char value) + { + if (IsCSharpIdentifierStart(value)) + return true; + + return char.GetUnicodeCategory(value) is + UnicodeCategory.DecimalDigitNumber or + UnicodeCategory.ConnectorPunctuation or + UnicodeCategory.NonSpacingMark or + UnicodeCategory.SpacingCombiningMark or + UnicodeCategory.Format; + } + private static int FindMatchingParen(string value, int openParen) { var depth = 0; @@ -319,6 +1078,70 @@ private static IEnumerable SplitTopLevelParameters(string parameters) yield return last; } + private static IEnumerable SplitTopLevelCSharpParameters(string parameters) + { + var start = 0; + var angleDepth = 0; + var parenDepth = 0; + var bracketDepth = 0; + var braceDepth = 0; + var inDefaultValue = false; + for (var i = 0; i < parameters.Length; i++) + { + if (TryGetCSharpLexicalRegionEnd(parameters, i, out var regionEnd)) + { + i = regionEnd - 1; + continue; + } + + switch (parameters[i]) + { + case '<' when !inDefaultValue && bracketDepth == 0: + angleDepth++; + break; + case '>' when !inDefaultValue && bracketDepth == 0: + if (angleDepth > 0) angleDepth--; + break; + case '(': + parenDepth++; + break; + case ')': + if (parenDepth > 0) parenDepth--; + break; + case '[': + bracketDepth++; + break; + case ']': + if (bracketDepth > 0) bracketDepth--; + break; + case '{': + braceDepth++; + break; + case '}': + if (braceDepth > 0) braceDepth--; + break; + case '=' when angleDepth == 0 + && parenDepth == 0 + && bracketDepth == 0 + && braceDepth == 0: + inDefaultValue = true; + break; + case ',' when angleDepth == 0 + && parenDepth == 0 + && bracketDepth == 0 + && braceDepth == 0: + yield return parameters[start..i].Trim(); + start = i + 1; + inDefaultValue = false; + break; + } + } + + var last = parameters[start..].Trim(); + if (last.Length > 0) + yield return last; + } + private static string SimplifyParameterForOutline(string parameter, string? lang) { var cleaned = Regex.Replace(parameter, @"\s*=\s*.*$", "").Trim(); diff --git a/tests/CodeIndex.Tests/DbReaderTests.cs b/tests/CodeIndex.Tests/DbReaderTests.cs index 2bd368cc8..be38b71c5 100644 --- a/tests/CodeIndex.Tests/DbReaderTests.cs +++ b/tests/CodeIndex.Tests/DbReaderTests.cs @@ -7183,6 +7183,210 @@ public void Process(int count, CancellationToken cancellationToken = default) { Assert.Equal("Worker.Process", overloads[1].Path); } + [Fact] + public void GetOutline_CSharpGenericCallablesUseStableReadableSignatures_Issue4916() + { + const string source = """" + using System; + using System.Collections.Generic; + + [AttributeUsage(AttributeTargets.Parameter)] + public sealed class MarkerAttribute : Attribute + { + public MarkerAttribute(string text) { } + public MarkerAttribute(bool value) { } + } + + public sealed class T { } + public sealed class U { } + + public static class Outer + { + public sealed class U { } + } + + public interface IWorker + { + TResult Convert(TSource source, ref TResult result) + where TSource : class; + } + + public static class WorkerExtensions + { + public static TResult Project( + this Worker worker, + IReadOnlyDictionary> values, + Func<(TSource, int[]), TResult?[]> selector) + where TSource : class + where TResult : class => default!; + } + + public sealed class Worker : IWorker + { + public void Convert(int count) { } + + public void NonGenericLiteral(string closing = "\u0029", int count = 0) { } + + public T Convert(T value) + where T : notnull => value; + + public void Qualified(global::U one, Outer.U two, U three) { } + + public void Collision(U value) { } + + public void Collision(T value) { } + + public void Keyword<@string>(string value) { } + + public void Keyword<@string>(@string value) { } + + public void EscapedOrdinary<@U>(U value) { } + + public void Combining(T́ value) { } + + public void Defaults( + string closing = ")", + string comma = "a,b", + char close = ')', + T value = default!) { } + + public void RawDefault(string value = """a,b)""", T other = default!) { } + + public void InterpolatedDefault(string value = $"{"a,b)"}", T other = default!) { } + + public void AttributeParam([Marker("x]")] T value, int count) { } + + public void AttributeExpression([Marker(1 < 2)] T value, int count) { } + + public void CommentedName /* between name and type parameters */ (T value) { } + + public void CommentedParameters /* before parameters */ (T value) { } + + public void Escaped<\u0055>(\u0055 value) { } + + TResult IWorker.Convert(TSource source, ref TResult result) + where TSource : class => result; + + public void Use() + { + TLocal Local(TLocal?[] values) + where TLocal : struct => values[0].Value; + } + } + """"; + + InsertIndexedFile("src/generic-before.cs", "csharp", source); + InsertIndexedFile("src/generic-after.cs", "csharp", "\n\n\n" + source); + + var before = _reader.GetOutline("src/generic-before.cs"); + var after = _reader.GetOutline("src/generic-after.cs"); + + Assert.NotNull(before); + Assert.NotNull(after); + var beforeCallables = before!.Symbols + .Where(symbol => symbol.Kind == "function") + .OrderBy(symbol => symbol.Signature, StringComparer.Ordinal) + .Select(symbol => (symbol.Name, symbol.Path, symbol.Signature, symbol.DisplayName)) + .ToList(); + var afterCallables = after!.Symbols + .Where(symbol => symbol.Kind == "function") + .OrderBy(symbol => symbol.Signature, StringComparer.Ordinal) + .Select(symbol => (symbol.Name, symbol.Path, symbol.Signature, symbol.DisplayName)) + .ToList(); + + Assert.Equal(beforeCallables, afterCallables); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Convert" + && symbol.DisplayName == "Convert(int)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Convert" + && symbol.Path == "Worker.Convert" + && symbol.DisplayName == "Convert(T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Convert" + && symbol.Signature!.Contains("IWorker.Convert", StringComparison.Ordinal) + && symbol.DisplayName == "Convert(T1, ref T2)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Project" + && symbol.DisplayName == + "Project(Worker, IReadOnlyDictionary>, Func<(T1, int[]), T2?[]>)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Local" + && symbol.DisplayName == "Local(T?[])"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "NonGenericLiteral" + && symbol.DisplayName == "NonGenericLiteral(string, int)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Qualified" + && symbol.DisplayName == "Qualified(global::U, Outer.U, T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Collision" + && symbol.DisplayName == "Collision(T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Collision" + && symbol.DisplayName == "Collision(T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Keyword" + && symbol.DisplayName == "Keyword(string)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Keyword" + && symbol.DisplayName == "Keyword(T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "EscapedOrdinary" + && symbol.DisplayName == "EscapedOrdinary(T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Combining" + && symbol.DisplayName == "Combining(T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Defaults" + && symbol.DisplayName == "Defaults(string, string, char, T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "RawDefault" + && symbol.DisplayName == "RawDefault(string, T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "InterpolatedDefault" + && symbol.DisplayName == "InterpolatedDefault(string, T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "AttributeParam" + && symbol.DisplayName == "AttributeParam(T, int)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "AttributeExpression" + && symbol.DisplayName == "AttributeExpression(T, int)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "CommentedName" + && symbol.DisplayName == "CommentedName(T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "CommentedParameters" + && symbol.DisplayName == "CommentedParameters(T)"); + Assert.Contains(beforeCallables, symbol => + symbol.Name == "Escaped" + && symbol.DisplayName == "Escaped(T)"); + Assert.DoesNotContain(beforeCallables, symbol => + symbol.DisplayName!.Contains('@', StringComparison.Ordinal)); + + var legacyFileId = _writer.UpsertFile(new FileRecord + { + Path = "src/legacy-generic.cs", + Lang = "csharp", + Size = 32, + Lines = 40, + Modified = new DateTime(2025, 6, 1, 0, 0, 0, DateTimeKind.Utc), + }); + _writer.InsertSymbols([new SymbolRecord + { + FileId = legacyFileId, + Kind = "function", + Name = "Incomplete", + Signature = "T Incomplete