diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index efe043126..ed0d87468 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -1802,6 +1802,7 @@ Process exit codes are coarse (`0` success including valid zero-row queries, `1` - **Extractor regex backtracking policy** — Built-in symbol and reference extractors must not use unbounded regular expression matching on repository-controlled file content. Backtracking regexes use `BoundedRegex.DefaultMatchTimeout`, while `RegexOptions.NonBacktracking` is allowed for patterns that are compatible with the non-backtracking engine. Patterns that deliberately remain backtracking-only, such as lookaround-heavy or balancing-group extractors, are acceptable only because the shared timeout audit covers them. If a future extractor must use `System.Text.RegularExpressions.Regex` directly, it must pass an explicit timeout and document why `BoundedRegex` or `NonBacktracking` is not suitable. - **Hybrid symbol extraction** — No AST parsers and no heavyweight language-specific dependencies. Most languages still use compiled regex patterns, while JavaScript/TypeScript add a lightweight lexer/state machine for class-body method extraction, private-scope filtering, synthetic class-expression binding detection, and JS/TS-specific range resolution that regex alone could not handle reliably. The trade-off still favors speed and portability over full parser accuracy, but the index stores richer symbol metadata such as definition ranges, optional body ranges, signatures, enclosing symbols, qualified container paths, authoritative family keys, visibility, and return types when the language patterns or JS/TS state machine can infer them. Visual Basic patterns also treat `Namespace ... End Namespace` as a real container and allow implicit-visibility declarations plus leading modifiers (`Shared`, `Overrides`, `Partial`, etc.), so VB projects expose the same top-level orientation and member coverage that other class-based languages already get. Visual Basic container patterns use case-insensitive `VisualBasicEnd` range tracking so cross-file partial families still get stable body ranges and can participate in hotspot-family grouping. **Pattern externalization**: Language patterns are currently defined inline in `SymbolExtractor.cs` using compiled `Regex` objects. This keeps the extraction pipeline self-contained and allows compile-time validation, but means adding a new language requires a code change and rebuild. A future iteration could externalize patterns to JSON/TOML files (loaded at startup), which would lower the barrier for community contributions and enable hot-reload during development. The trade-off is losing compile-time safety and slightly increasing startup cost. If externalized, patterns should include: language name, kind (function/class/import/namespace), regex string, body style (brace/indent/ruby-end/none), and optional capture group names for visibility and return type. - **Nested C# interpolation state** — The C# lexical masker keeps immutable parent frames when an interpolated regular, verbatim, or raw string starts inside another interpolation hole. Closing the nested string restores the complete outer mode, delimiter, dollar-count, and brace-depth state; expression-bodied property calls remain excluded from declaration patterns, and C# extractor contract bumps force existing indexes to refresh affected files. +- **C# static-lambda declaration gating** — The declaration scanner treats a candidate name inside a confirmed `static`, `static async`, or `async static` lambda header as expression context, including when multiline property-header composition prepends call arguments. Real static members, local functions, and assigned-lambda symbols remain eligible. C# extractor contract v6 makes a normal index refresh re-extract stale C# symbols (#4830; regression of #4453). - **Authoritative hotspot-family trust** — `hotspots` only promotes duplicate-name families back to codebase-wide counts when the persisted `symbols.container_qualified_name` / `symbols.family_key` were produced under the current per-language `hotspot_family_version_*` contract. These readiness stamps and marker fingerprints live in `codeindex_meta`, so legacy, mixed, or partially refreshed DBs degrade explicitly instead of silently reusing stale cross-file family identities. - **Authoritative C# metadata-target trust** — `deps` / `impact` metadata-attribute edges (linking `[Foo]` usage to the defining `FooAttribute` class) are promoted from a signature-shape heuristic to an authoritative resolver whenever `is_metadata_target` is persisted under the current `metadata_target_version_csharp` contract. The resolver walks C# class base lists with fixed-point transitive resolution through same-DB class rows and falls back to the BCL `Attribute` suffix convention only for unresolved external bases. Readiness lives in `codeindex_meta`, and the reader uses a three-way branch: (1) ready → `is_metadata_target = 1`; (2) column present but not stamped (legacy row) → `signature LIKE '%: %'`; (3) column missing → naming-only fallback. This fixes non-attribute impostors (`class FooAttribute : BaseService`) silently dropping edges when they shared names with real `FooAttribute : Attribute` classes (#435). - **Human-readable default** — All commands default to human-readable output. `--json` for AI/machine consumption. @@ -4988,6 +4989,7 @@ USER_GUIDEの[終了コード](USER_GUIDE.md#終了コード)セクションを - **extractor regex の backtracking policy** — built-in symbol/reference extractor は repository-controlled な file content に対して unbounded regex match を使わない。backtracking regex は `BoundedRegex.DefaultMatchTimeout` を使い、`RegexOptions.NonBacktracking` は non-backtracking engine と互換な pattern で使ってよい。lookaround-heavy な extractor や balancing-group を使う extractor など、意図的に backtracking-only のまま残す pattern は、共有 timeout audit の対象になる場合だけ許容する。将来の extractor が `System.Text.RegularExpressions.Regex` を直接使う必要がある場合は、明示 timeout を渡し、`BoundedRegex` や `NonBacktracking` が適さない理由を文書化すること。 - **ハイブリッドなシンボル抽出** — ASTパーサーも重量級の言語固有依存も追加しない方針。大半の言語はコンパイル済み正規表現で処理し、JavaScript / TypeScript だけは class body の method 抽出、private-scope filtering、synthetic class expression の binding 判定、JS/TS 固有の range 解決など、正規表現だけでは壊れやすい箇所を軽量 lexer / state machine で補う。引き続き精度より速度とポータビリティを優先しつつ、言語パターンや JS/TS state machine から推論できる範囲で定義範囲、本体範囲、シグネチャ、親シンボル、修飾付きコンテナ経路、正式なグループキー、可視性、戻り値型も保存する。Visual Basic では `Namespace ... End Namespace` も実コンテナとして扱い、implicit visibility の宣言や `Shared` / `Overrides` / `Partial` など visibility 以外の先行修飾子も受理するようにしたため、他のクラス系言語と同じようにトップレベル構造とメンバーを取りこぼしにくくなった。Visual Basic のコンテナパターンは `VisualBasicEnd` ベースの範囲追跡を大文字小文字非依存で扱うため、partial 型ファミリーでも安定した本体範囲と `hotspots` 集計用メタデータを維持できる。**パターン外部化**: 言語パターンは現在 `SymbolExtractor.cs` 内にコンパイル済み `Regex` として定義。抽出パイプラインが自己完結し、コンパイル時検証が効くが、言語追加にはコード変更と再ビルドが必要。将来的にはJSON/TOMLファイルに外部化し（起動時読み込み）、コミュニティ貢献の敷居を下げ、開発時のホットリロードも可能にできる。トレードオフはコンパイル時安全性の喪失と起動コストの微増。外部化時のスキーマ: 言語名、種別（function/class/import/namespace）、正規表現文字列、本体スタイル（brace/indent/ruby-end/none）、可視性・戻り値型のキャプチャグループ名。 - **C# の nested interpolation state** — C# lexical masker は、別の interpolation hole 内で interpolated regular / verbatim / raw string が始まると immutable な親 frame を保持する。nested string を閉じると外側の mode、delimiter、dollar count、brace depth を完全に復元し、expression-bodied property 内の call を declaration pattern から除外する。C# extractor contract の更新により、既存 index の対象ファイルも再抽出される。 +- **C# static lambda の宣言ゲート** — 宣言 scanner は、確認済みの `static`、`static async`、`async static` lambda header 内にある候補名を式コンテキストとして扱います。複数行 property-header の結合によって呼び出し引数が前置された場合も同様です。本物の static member、local function、代入済み lambda symbol は引き続き抽出対象です。C# extractor contract v6 により、通常の index 更新で古い C# symbol が再抽出されます（#4830、#4453 の回帰）。 - **`hotspots` の正式な family trust** — `hotspots` が重名グループをコードベース全体の件数へ昇格させるのは、永続化済み `symbols.container_qualified_name` / `symbols.family_key` が現行の言語別 `hotspot_family_version_*` 契約で生成されたときだけ。readiness stamp と marker fingerprint は `codeindex_meta` に置き、旧形式・混在・部分更新直後の DB は古いファイル横断グループ識別子を黙って再利用せず、明示的に縮退する。 - **C# metadata-target の正式な trust** — `deps` / `impact` の metadata attribute edge（`[Foo]` 使用と定義側 `FooAttribute` クラスの紐付け）は、永続化済み `is_metadata_target` が現行の `metadata_target_version_csharp` 契約で stamp されている DB ではシグネチャ形状ヒューリスティックではなく authoritative resolver の判定結果を使う。resolver は C# クラスの base list を同 DB 内の class 行で fixed-point 展開して解決し、未解決の外部基底のみ BCL 規約（`Attribute` サフィックス）へフォールバックする。readiness は `codeindex_meta` に置き、reader は (1) ready → `is_metadata_target = 1`、(2) 列はあるが stamp 未完（legacy 行）→ `signature LIKE '%: %'`、(3) 列すらない → 命名のみ、の 3 way 分岐で縮退する。これにより、`class FooAttribute : BaseService` のような非 attribute 同名 impostor が真の `FooAttribute : Attribute` と同居したときにエッジを黙ってドロップする挙動を修正した（#435）。 - **人間向けがデフォルト** — 全コマンドのデフォルト出力は人間向け。`--json`でAI/機械向け出力。 diff --git a/TESTING_GUIDE.md b/TESTING_GUIDE.md index 018c8040d..54e94e152 100644 --- a/TESTING_GUIDE.md +++ b/TESTING_GUIDE.md @@ -426,6 +426,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding - C# reflection-name extraction coverage keeps literal, constant-concatenation, dynamic, comment, and string-decoy cases in one source fixture so those parser boundaries share one symbol/reference pass. - C# BOM extraction keeps a simple leading-BOM import fixture plus one mixed-newline fixture that simultaneously covers leading and mid-file BOM handling across CRLF, bare CR, and LF boundaries; do not repeat separate extraction passes for newline subsets already present in the mixed fixture. - C# lambda-capture coverage keeps positive enclosing-local capture, parameter shadowing, and same-named-method isolation in one source fixture; a single capture assertion proves the negative regions did not leak. +- C# static-lambda declaration regression coverage keeps stateful, typed/untyped, explicit-return, both async-modifier orders, Unicode/escaped-identifier, multiline, nested, and argument-position forms in one extractor pass. Preserve real static members/local functions, including generic, constructor, explicit-interface, and verbatim-type-name forms, and an assigned-lambda range/container assertion in that fixture, plus one CLI `symbols` corpus fixture for phantom-name checks (#4830). - Escaped-brace coverage for regular and verbatim interpolated C# strings shares one extraction fixture because both variants assert the same phantom-call exclusion. - Direct and nested interpolations inside C# raw strings share one source fixture and one extraction pass while retaining distinct container assertions. - C# nested-interpolation regression coverage keeps raw SQL, a completion template containing another interpolated string, following expression-bodied properties, same-line siblings, and enclosing-class range assertions in one fixture and extraction pass. A reader integration fixture reuses that lexical context around a nullable-generic `out`-parameter method and verifies its definition, outline, resolved reference, caller, and callee surfaces together. @@ -1310,6 +1311,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests" - C# reflection-name 抽出 coverage は、literal、定数連結、dynamic、comment、string decoy を1つの source fixture にまとめ、これらの parser boundary で1回の symbol/reference pass を共有します。 - C# BOM 抽出は、単純な先頭 BOM import fixture と、CRLF・bare CR・LF 境界で先頭/mid-file BOM を同時に扱う1つの混在改行 fixture を維持します。混在 fixture に含まれる改行 subset ごとに抽出 pass を重複させないでください。 - C# lambda capture coverage は、外側 local の正例、parameter shadowing、同名 method 間の分離を1つの source fixture にまとめます。capture が1件だけである assertion により、negative region からの漏れも同時に検証します。 +- C# static lambda の宣言回帰 coverage は、stateful、型あり/型なし、明示的戻り値型、両方の async modifier 順、Unicode/escape 識別子、複数行、入れ子、引数位置の各形式を1回の extractor pass にまとめます。同じ fixture で generic、constructor、明示的 interface、verbatim 型名を含む本物の static member / local function と、代入済み lambda の range / container assertion を維持し、phantom 名の確認には CLI `symbols` corpus fixture を1つ追加します（#4830）。 - C# interpolated string の escaped-brace coverage は、通常形式と逐語形式で同じ phantom call 除外を検証するため、1回の抽出 fixture を共有します。 - C# raw string 内の direct interpolation と nested interpolation は1つの source fixture と抽出 pass を共有し、container assertion は個別に維持します。 - C# nested interpolation の regression coverage は、raw SQL、別の interpolated string を含む completion template、後続の expression-bodied property、same-line sibling、enclosing class range の assertion を1つの fixture と抽出 pass にまとめます。reader integration fixture では、その lexical context の後に nullable generic の `out` parameter method を置き、definition、outline、resolved reference、caller、callee の各 surface をまとめて検証します。 diff --git a/changelog.d/unreleased/4830.fixed.md b/changelog.d/unreleased/4830.fixed.md new file mode 100644 index 000000000..7197f8fe3 --- /dev/null +++ b/changelog.d/unreleased/4830.fixed.md @@ -0,0 +1,22 @@ +--- +category: fixed +issues: + - 4830 +affected: + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPatterns.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs + - tests/CodeIndex.Tests/SymbolExtractorIssue4830Tests.cs + - tests/CodeIndex.Tests/QueryCommandRunnerIssue4830Tests.cs + - DEVELOPER_GUIDE.md + - TESTING_GUIDE.md +--- + +## English + +- **C# static lambdas no longer emit phantom declarations (#4830; regression of #4453)** — Declaration scanning now rejects function and property candidates whose names fall inside a confirmed static-lambda header while preserving real static members, local functions, and assigned-lambda symbols. The C# extractor contract advances so normal index refreshes replace stale rows. + +## 日本語 + +- **C# static lambda が phantom 宣言を生成しなくなりました（#4830、#4453 の回帰）** — 宣言 scan は、確認済み static-lambda header 内に名前がある function / property 候補を除外し、本物の static member、local function、代入済み lambda symbol を維持します。C# extractor contract を更新したため、通常の index 更新で古い行が置き換わります。 diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPatterns.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPatterns.cs index 46fabbac9..e505ee4e6 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPatterns.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPatterns.cs @@ -82,6 +82,12 @@ public static partial class SymbolExtractor private static readonly Regex CSharpConstOrStaticReadonlyFieldRegex = new( @"(?:\bconst\b|\bstatic\b[^=;]*\breadonly\b|\breadonly\b[^=;]*\bstatic\b)", RegexOptions.Compiled | RegexOptions.CultureInvariant); + private static readonly Regex CSharpExplicitLambdaReturnTypeRegex = new( + $@"^(?:{CSharpTypePattern})$", + RegexOptions.Compiled | RegexOptions.CultureInvariant); + private static readonly Regex CSharpStaticConstructorHeaderPrefixRegex = new( + $@"^(?:unsafe\s+)?static\s+(?:unsafe\s+)?(?{CSharpIdentifierPattern})\s*\(\s*$", + RegexOptions.Compiled | RegexOptions.CultureInvariant); private static bool IsCSharpConstOrStaticReadonlyField(string signature) => CSharpConstOrStaticReadonlyFieldRegex.IsMatch(signature); diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs index c11510a38..0ee5bbdd8 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs @@ -2086,6 +2086,408 @@ private static bool IsCSharpFunctionMatchInsideExpressionBody(string matchLine, return expressionBraceDepth.HasValue; } + private static bool IsCSharpStaticLambdaHeaderCandidate( + string matchLine, + int nameIndex, + string? enclosingTypeName) + { + var boundedNameIndex = Math.Min(Math.Max(0, nameIndex), matchLine.Length); + var searchIndex = 0; + while (searchIndex < matchLine.Length) + { + var arrowIndex = matchLine.IndexOf("=>", searchIndex, StringComparison.Ordinal); + if (arrowIndex < 0) + return false; + + if (TryGetCSharpStaticLambdaHeaderRange( + matchLine, + arrowIndex, + enclosingTypeName, + out var headerStart, + out var headerEnd) + && boundedNameIndex >= headerStart + && boundedNameIndex < headerEnd) + { + return true; + } + + searchIndex = arrowIndex + 2; + } + + return false; + } + + private static bool TryGetCSharpStaticLambdaHeaderRange( + string matchLine, + int arrowIndex, + string? enclosingTypeName, + out int headerStart, + out int headerEnd) + { + headerStart = -1; + headerEnd = -1; + + var cursor = arrowIndex - 1; + SkipCSharpWhitespaceBackward(matchLine, ref cursor); + if (cursor < 0) + return false; + + int parameterStart; + var hasParenthesizedParameters = matchLine[cursor] == ')'; + if (hasParenthesizedParameters) + { + var depth = 1; + cursor--; + while (cursor >= 0 && depth > 0) + { + if (matchLine[cursor] == ')') + depth++; + else if (matchLine[cursor] == '(') + depth--; + + cursor--; + } + + if (depth != 0) + return false; + + parameterStart = cursor + 1; + } + else + { + var parameterEnd = cursor + 1; + if (!TryReadCSharpLambdaIdentifierBackward( + matchLine, + ref cursor, + out parameterStart, + out var identifierEnd) + || identifierEnd != parameterEnd) + { + return false; + } + } + + cursor = parameterStart - 1; + SkipCSharpWhitespaceBackward(matchLine, ref cursor); + + if (TryReadCSharpStaticLambdaModifiersBackward( + matchLine, + cursor, + out var modifierStart)) + { + headerStart = modifierStart; + headerEnd = arrowIndex + 2; + return true; + } + + if (!hasParenthesizedParameters + || !TryFindCSharpExplicitReturnStaticLambdaModifier( + matchLine, + parameterStart, + enclosingTypeName, + out modifierStart)) + { + return false; + } + + headerStart = modifierStart; + headerEnd = arrowIndex + 2; + return true; + } + + private static bool TryReadCSharpStaticLambdaModifiersBackward( + string text, + int cursor, + out int modifierStart) + { + var hasStaticModifier = false; + modifierStart = -1; + for (var modifierCount = 0; modifierCount < 2 && cursor >= 0; modifierCount++) + { + if (!TryReadCSharpLambdaIdentifierBackward( + text, + ref cursor, + out var tokenStart, + out var tokenEnd)) + { + break; + } + + var token = text.AsSpan(tokenStart, tokenEnd - tokenStart); + if (!token.SequenceEqual("static".AsSpan()) + && !token.SequenceEqual("async".AsSpan())) + { + break; + } + + hasStaticModifier |= token.SequenceEqual("static".AsSpan()); + modifierStart = tokenStart; + SkipCSharpWhitespaceBackward(text, ref cursor); + } + + return hasStaticModifier; + } + + private static bool TryFindCSharpExplicitReturnStaticLambdaModifier( + string text, + int parameterStart, + string? enclosingTypeName, + out int modifierStart) + { + modifierStart = -1; + var searchBefore = parameterStart; + while (searchBefore > 0) + { + var staticIndex = text.LastIndexOf( + "static", + searchBefore - 1, + StringComparison.Ordinal); + if (staticIndex < 0) + return false; + + searchBefore = staticIndex; + if (!IsCSharpStandaloneKeywordAt(text, staticIndex, "static")) + continue; + + var returnTypeStart = SkipWhitespace(text, staticIndex + "static".Length); + if (IsCSharpStandaloneKeywordAt(text, returnTypeStart, "async")) + returnTypeStart = SkipWhitespace(text, returnTypeStart + "async".Length); + + var candidateModifierStart = staticIndex; + var precedingCursor = staticIndex - 1; + SkipCSharpWhitespaceBackward(text, ref precedingCursor); + if (TryReadCSharpLambdaIdentifierBackward( + text, + ref precedingCursor, + out var precedingTokenStart, + out var precedingTokenEnd) + && text.AsSpan(precedingTokenStart, precedingTokenEnd - precedingTokenStart) + .SequenceEqual("async".AsSpan())) + { + candidateModifierStart = precedingTokenStart; + } + + if (IsCSharpRealStaticDeclarationHeader( + text, + staticIndex, + parameterStart, + enclosingTypeName)) + { + continue; + } + + if (!IsCSharpExplicitLambdaReturnType( + text, + returnTypeStart, + parameterStart)) + { + continue; + } + + modifierStart = candidateModifierStart; + return true; + } + + return false; + } + + private static bool IsCSharpRealStaticDeclarationHeader( + string text, + int staticIndex, + int parameterStart, + string? enclosingTypeName) + { + var headerEnd = Math.Min(text.Length, parameterStart + 1); + if (staticIndex < 0 || staticIndex >= headerEnd) + return false; + + var header = text[staticIndex..headerEnd]; + if (CSharpMethodHeaderPrefixRegex.IsMatch(header)) + return true; + + var precedingCursor = staticIndex - 1; + SkipCSharpWhitespaceBackward(text, ref precedingCursor); + + // A static constructor has no return type, so its lexical prefix is otherwise + // indistinguishable from a zero-parameter explicit-return lambda. Preserve the + // constructor only at a declaration boundary; lambdas follow `=`, `,`, `(`, or + // another expression token. Generic methods and explicit-interface implementations + // are recognized by CSharpMethodHeaderPrefixRegex above. + // static constructor は戻り値型を持たないため、字句上は parameter 0 個の明示的 + // 戻り値型 lambda と区別できない。宣言境界にある constructor だけを維持し、 + // `=`、`,`、`(`、その他の式 token に続く lambda は維持しない。generic method と + // 明示的 interface 実装は上の CSharpMethodHeaderPrefixRegex で識別する。 + if (precedingCursor >= 0 && text[precedingCursor] is not ('{' or '}' or ';')) + return false; + + var constructorMatch = CSharpStaticConstructorHeaderPrefixRegex.Match(header); + if (!constructorMatch.Success || string.IsNullOrWhiteSpace(enclosingTypeName)) + return false; + + var constructorName = constructorMatch.Groups["name"].ValueSpan.Trim(); + if (constructorName.Length > 0 && constructorName[0] == '@') + constructorName = constructorName[1..]; + + return constructorName.SequenceEqual(enclosingTypeName.AsSpan()); + } + + private static string? FindCSharpEnclosingTypeName( + IReadOnlyList symbols, + int line) + { + for (var index = symbols.Count - 1; index >= 0; index--) + { + var candidate = symbols[index]; + if (candidate.Kind is not ("class" or "struct") + || candidate.BodyStartLine is null + || candidate.BodyEndLine is null + || candidate.BodyStartLine > line + || candidate.BodyEndLine < line) + { + continue; + } + + return candidate.Name; + } + + return null; + } + + private static bool IsCSharpExplicitLambdaReturnType( + string text, + int start, + int end) + { + while (start < end && char.IsWhiteSpace(text[start])) + start++; + while (end > start && char.IsWhiteSpace(text[end - 1])) + end--; + + if (IsCSharpStandaloneKeywordAt(text, start, "scoped")) + { + start = SkipWhitespace(text, start + "scoped".Length); + } + if (IsCSharpStandaloneKeywordAt(text, start, "ref")) + { + start = SkipWhitespace(text, start + "ref".Length); + if (IsCSharpStandaloneKeywordAt(text, start, "readonly")) + start = SkipWhitespace(text, start + "readonly".Length); + } + + if (start >= end) + return false; + + var returnType = text[start..end]; + return CSharpExplicitLambdaReturnTypeRegex.IsMatch(returnType); + } + + private static bool IsCSharpStandaloneKeywordAt( + string text, + int start, + string keyword) + { + if (start < 0 || start + keyword.Length > text.Length) + return false; + if (!text.AsSpan(start, keyword.Length).SequenceEqual(keyword.AsSpan())) + return false; + + var before = start - 1; + var after = start + keyword.Length; + return (before < 0 + || text[before] != '@' + && !IsCSharpLambdaIdentifierPart(text[before])) + && (after >= text.Length || !IsCSharpLambdaIdentifierPart(text[after])); + } + + private static bool TryReadCSharpLambdaIdentifierBackward( + string text, + ref int cursor, + out int tokenStart, + out int tokenEnd) + { + tokenEnd = cursor + 1; + while (cursor >= 0) + { + if (TrySkipCSharpUnicodeEscapeBackward(text, ref cursor)) + continue; + + var decodeStatus = Rune.DecodeLastFromUtf16( + text.AsSpan(0, cursor + 1), + out var rune, + out var charsConsumed); + if (decodeStatus != System.Buffers.OperationStatus.Done + || !IsCSharpLambdaIdentifierPart(rune)) + { + break; + } + + cursor -= charsConsumed; + } + + if (cursor >= 0 && text[cursor] == '@') + cursor--; + + tokenStart = cursor + 1; + return tokenStart < tokenEnd; + } + + private static bool TrySkipCSharpUnicodeEscapeBackward( + string text, + ref int cursor) + { + if (TrySkipCSharpUnicodeEscapeBackward(text, ref cursor, 'U', 8)) + return true; + + return TrySkipCSharpUnicodeEscapeBackward(text, ref cursor, 'u', 4); + } + + private static bool TrySkipCSharpUnicodeEscapeBackward( + string text, + ref int cursor, + char prefix, + int digitCount) + { + var slashIndex = cursor - digitCount - 1; + if (slashIndex < 0 + || text[slashIndex] != '\\' + || text[slashIndex + 1] != prefix) + { + return false; + } + + for (var i = slashIndex + 2; i <= cursor; i++) + { + if (!Uri.IsHexDigit(text[i])) + return false; + } + + cursor = slashIndex - 1; + return true; + } + + private static void SkipCSharpWhitespaceBackward(string text, ref int cursor) + { + while (cursor >= 0 && char.IsWhiteSpace(text[cursor])) + cursor--; + } + + private static bool IsCSharpLambdaIdentifierPart(char ch) => + char.IsSurrogate(ch) || IsCSharpLambdaIdentifierPart(new Rune(ch)); + + private static bool IsCSharpLambdaIdentifierPart(Rune rune) => + Rune.GetUnicodeCategory(rune) is + System.Globalization.UnicodeCategory.UppercaseLetter or + System.Globalization.UnicodeCategory.LowercaseLetter or + System.Globalization.UnicodeCategory.TitlecaseLetter or + System.Globalization.UnicodeCategory.ModifierLetter or + System.Globalization.UnicodeCategory.OtherLetter or + System.Globalization.UnicodeCategory.LetterNumber or + System.Globalization.UnicodeCategory.NonSpacingMark or + System.Globalization.UnicodeCategory.SpacingCombiningMark or + System.Globalization.UnicodeCategory.DecimalDigitNumber or + System.Globalization.UnicodeCategory.ConnectorPunctuation or + System.Globalization.UnicodeCategory.Format; + private static bool IsCSharpMultilineExpressionBodiedMember(string[] lines, int startLineIndex, int startColumn) { var lexState = new CSharpLexState(); diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs index a92c6b43a..a84c494bf 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs @@ -5,7 +5,7 @@ public static partial class SymbolExtractor public const int DefaultContractVersion = 1; public const int ExpandedLanguageContractVersion = 2; public const int PythonContractVersion = 2; - public const int CSharpContractVersion = 5; + public const int CSharpContractVersion = 6; public const int DockerfileContractVersion = 2; public const int MakefileContractVersion = 2; public const int StyleAndXamlContractVersion = 2; diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs index 4da960b56..9a6449f7d 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs @@ -442,6 +442,29 @@ private static List ExtractCore( continue; } + if (lang == "csharp" + && (pattern.Kind is "function" or "property") + && match.Groups["name"].Success + && IsCSharpStaticLambdaHeaderCandidate( + patternMatchLine, + lineOffset + match.Groups["name"].Index, + FindCSharpEnclosingTypeName(symbols, i + 1))) + { + // Multi-line call arguments are composed into the same candidate + // string as the following lambda. A declaration-shaped regex can then + // reinterpret the state argument as a return type and `static` (or a + // parameter) as a member name. Reject only names that occupy a + // confirmed static-lambda header; real static members and local + // functions remain eligible. Closes #4830; regression of #4453. + // 複数行の呼び出し引数と後続 lambda は同じ候補文字列に結合されるため、 + // 宣言形 regex が state 引数を戻り値型、`static`（または parameter）を + // member 名として再解釈し得る。確認済み static-lambda header 内の名前 + // だけを除外し、本物の static member / local function は維持する。 + // Closes #4830; regression of #4453. + lineOffset = absoluteStartColumn + Math.Max(1, match.Length); + continue; + } + if (lang == "csharp" && pattern.Kind == "function" && HasCSharpTokenBeforeIndex(matchLine, "when", absoluteStartColumn + match.Groups["name"].Index)) diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerIssue4830Tests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerIssue4830Tests.cs new file mode 100644 index 000000000..09216152e --- /dev/null +++ b/tests/CodeIndex.Tests/QueryCommandRunnerIssue4830Tests.cs @@ -0,0 +1,55 @@ +using CodeIndex.Cli; + +namespace CodeIndex.Tests; + +public partial class QueryCommandRunnerTests +{ + [Fact] + public void RunSymbols_CSharpStaticLambdaCorpusHasNoPhantomDeclarations_Issue4830() + { + using var project = TestProjectHelper.CreateTempProjectScope("cdidx_symbols_static_lambda_4830"); + var dbPath = TestProjectHelper.CreateProjectDb(project.Root); + TestProjectHelper.InsertIndexedFile( + dbPath, + "src/StaticLambdaCorpus.cs", + "csharp", + """ + using System; + + public static class StaticLambdaCorpus + { + public static string Fold(string value) + { + return string.Create( + value.Length, + (Value: value, Offset: 0), + static ( + Span destination, + (string Value, int Offset) state) => + { + state.Value.AsSpan().CopyTo(destination); + }); + } + } + """); + + var (exitCode, stdout, stderr) = CaptureConsole(() => QueryCommandRunner.RunSymbols( + ["--db", dbPath, "--json", "--lang", "csharp"], + _jsonOptions)); + var rows = ParseJsonLines(stdout); + var symbols = rows + .Select(row => ( + Kind: row.RootElement.GetProperty("kind").GetString(), + Name: row.RootElement.GetProperty("name").GetString())) + .ToList(); + + Assert.Equal(CommandExitCodes.Success, exitCode); + Assert.Equal(string.Empty, stderr); + Assert.DoesNotContain(symbols, symbol => symbol.Kind == "function" && symbol.Name == "static"); + Assert.DoesNotContain( + symbols, + symbol => (symbol.Kind is "function" or "property") + && (symbol.Name is "destination" or "state")); + Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == "Fold"); + } +} diff --git a/tests/CodeIndex.Tests/SymbolExtractorIssue4830Tests.cs b/tests/CodeIndex.Tests/SymbolExtractorIssue4830Tests.cs new file mode 100644 index 000000000..acf6a71e9 --- /dev/null +++ b/tests/CodeIndex.Tests/SymbolExtractorIssue4830Tests.cs @@ -0,0 +1,160 @@ +using CodeIndex.Indexer; + +namespace CodeIndex.Tests; + +public partial class SymbolExtractorTests +{ + [Fact] + public void Extract_CSharp_StaticLambdaHeadersDoNotCreateFunctionOrPropertySymbols_Issue4830() + { + const string content = """ + using System; + using System.Threading.Tasks; + + public class StaticLambdaSamples + { + public static int RealStaticMember(int value) => value; + + public Func ExistingLambda = value => value + 1; + + public string Format(string value) + { + static int RealStaticLocal(int item) => item + 1; + + var stateful = string.Create( + value.Length, + (Value: value, Offset: 0), + static (destination, state) => + { + state.Value.AsSpan().CopyTo(destination); + }); + + var typed = Apply( + static ( + int typedValue, + string typedLabel) => + typedValue + typedLabel.Length); + + var untyped = Apply(static item => item + 1); + var asyncStatic = Apply(static async (int asyncValue) => + { + await Task.Yield(); + return asyncValue; + }); + var asyncThenStatic = Apply(async static (int secondAsyncValue) => + { + await Task.Yield(); + return secondAsyncValue; + }); + var explicitReturn = Apply( + static int (int explicitValue) => explicitValue + 1); + var explicitAsyncReturn = Apply( + async static Task (int asyncExplicitValue) => + { + await Task.Yield(); + return asyncExplicitValue; + }); + Func explicitCustomZero = + static LambdaResult () => new(); + var combiningIdentifier = Apply( + static café => café + 1); + var escapedIdentifier = Apply( + static \u0061 => \u0061 + 1); + var nested = Apply( + static outerValue => + Apply(static innerValue => outerValue + innerValue)); + + return stateful + RealStaticLocal( + typed(1, "x") + + untyped(1) + + asyncStatic(1).Result + + asyncThenStatic(1).Result + + explicitReturn(1) + + explicitAsyncReturn(1).Result + + explicitCustomZero().Value + + combiningIdentifier(1) + + escapedIdentifier(1) + + nested(1)); + } + + static StaticLambdaSamples() => RealStaticMember(0); + + private static T RealStaticGeneric(T value) => value; + + static int IStaticContract.Transform(int value) => value; + + private @static RealVerbatimTypeMember(int value) => new(); + + private static @static RealStaticVerbatimTypeMember(int value) => new(); + + private static int RealStaticGenericOwner(int value) + { + static T RealStaticGenericLocal(T item) => item; + return RealStaticGenericLocal(value); + } + + private static Func Apply(Func callback) => callback; + } + + public class @static + { + } + + public sealed class LambdaResult + { + public int Value { get; } + } + + public interface IStaticContract + where TSelf : IStaticContract + { + static abstract int Transform(int value); + } + """; + + var symbols = SymbolExtractor.Extract(1, "csharp", content); + var forbiddenNames = new HashSet(StringComparer.Ordinal) + { + "static", + "destination", + "state", + "typedValue", + "typedLabel", + "item", + "asyncValue", + "secondAsyncValue", + "int", + "Task", + "LambdaResult", + "explicitValue", + "asyncExplicitValue", + "café", + "a", + @"\u0061", + "outerValue", + "innerValue", + }; + + Assert.DoesNotContain( + symbols, + symbol => (symbol.Kind is "function" or "property") + && forbiddenNames.Contains(symbol.Name)); + Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == "RealStaticMember"); + Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == "RealStaticLocal"); + Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == "StaticLambdaSamples"); + Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == "RealStaticGeneric"); + Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == "RealStaticGenericLocal"); + Assert.Contains( + symbols, + symbol => symbol.Kind == "function" + && symbol.Name == "Transform" + && symbol.ContainerName == "StaticLambdaSamples"); + Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == "RealVerbatimTypeMember"); + Assert.Contains(symbols, symbol => symbol.Kind == "function" && symbol.Name == "RealStaticVerbatimTypeMember"); + + var lambda = Assert.Single(symbols.Where(symbol => symbol.Kind == "lambda" && symbol.Name == "ExistingLambda")); + Assert.Equal("StaticLambdaSamples", lambda.ContainerName); + Assert.Equal(8, lambda.StartLine); + Assert.Equal(8, lambda.EndLine); + } +}