Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1802,6 +1802,7 @@ Process exit codes are coarse (`0` success including valid zero-row queries, `1`
- **Extractor regex backtracking policy** — Built-in symbol and reference extractors must not use unbounded regular expression matching on repository-controlled file content. Backtracking regexes use `BoundedRegex.DefaultMatchTimeout`, while `RegexOptions.NonBacktracking` is allowed for patterns that are compatible with the non-backtracking engine. Patterns that deliberately remain backtracking-only, such as lookaround-heavy or balancing-group extractors, are acceptable only because the shared timeout audit covers them. If a future extractor must use `System.Text.RegularExpressions.Regex` directly, it must pass an explicit timeout and document why `BoundedRegex` or `NonBacktracking` is not suitable.
- **Hybrid symbol extraction** — No AST parsers and no heavyweight language-specific dependencies. Most languages still use compiled regex patterns, while JavaScript/TypeScript add a lightweight lexer/state machine for class-body method extraction, private-scope filtering, synthetic class-expression binding detection, and JS/TS-specific range resolution that regex alone could not handle reliably. The trade-off still favors speed and portability over full parser accuracy, but the index stores richer symbol metadata such as definition ranges, optional body ranges, signatures, enclosing symbols, qualified container paths, authoritative family keys, visibility, and return types when the language patterns or JS/TS state machine can infer them. Visual Basic patterns also treat `Namespace ... End Namespace` as a real container and allow implicit-visibility declarations plus leading modifiers (`Shared`, `Overrides`, `Partial`, etc.), so VB projects expose the same top-level orientation and member coverage that other class-based languages already get. Visual Basic container patterns use case-insensitive `VisualBasicEnd` range tracking so cross-file partial families still get stable body ranges and can participate in hotspot-family grouping. **Pattern externalization**: Language patterns are currently defined inline in `SymbolExtractor.cs` using compiled `Regex` objects. This keeps the extraction pipeline self-contained and allows compile-time validation, but means adding a new language requires a code change and rebuild. A future iteration could externalize patterns to JSON/TOML files (loaded at startup), which would lower the barrier for community contributions and enable hot-reload during development. The trade-off is losing compile-time safety and slightly increasing startup cost. If externalized, patterns should include: language name, kind (function/class/import/namespace), regex string, body style (brace/indent/ruby-end/none), and optional capture group names for visibility and return type.
- **Nested C# interpolation state** — The C# lexical masker keeps immutable parent frames when an interpolated regular, verbatim, or raw string starts inside another interpolation hole. Closing the nested string restores the complete outer mode, delimiter, dollar-count, and brace-depth state; expression-bodied property calls remain excluded from declaration patterns, and C# extractor contract bumps force existing indexes to refresh affected files.
- **C# static-lambda declaration gating** — The declaration scanner treats a candidate name inside a confirmed `static`, `static async`, or `async static` lambda header as expression context, including when multiline property-header composition prepends call arguments. Real static members, local functions, and assigned-lambda symbols remain eligible. C# extractor contract v6 makes a normal index refresh re-extract stale C# symbols (#4830; regression of #4453).
- **Authoritative hotspot-family trust** — `hotspots` only promotes duplicate-name families back to codebase-wide counts when the persisted `symbols.container_qualified_name` / `symbols.family_key` were produced under the current per-language `hotspot_family_version_*` contract. These readiness stamps and marker fingerprints live in `codeindex_meta`, so legacy, mixed, or partially refreshed DBs degrade explicitly instead of silently reusing stale cross-file family identities.
- **Authoritative C# metadata-target trust** — `deps` / `impact` metadata-attribute edges (linking `[Foo]` usage to the defining `FooAttribute` class) are promoted from a signature-shape heuristic to an authoritative resolver whenever `is_metadata_target` is persisted under the current `metadata_target_version_csharp` contract. The resolver walks C# class base lists with fixed-point transitive resolution through same-DB class rows and falls back to the BCL `Attribute` suffix convention only for unresolved external bases. Readiness lives in `codeindex_meta`, and the reader uses a three-way branch: (1) ready → `is_metadata_target = 1`; (2) column present but not stamped (legacy row) → `signature LIKE '%: %'`; (3) column missing → naming-only fallback. This fixes non-attribute impostors (`class FooAttribute : BaseService`) silently dropping edges when they shared names with real `FooAttribute : Attribute` classes (#435).
- **Human-readable default** — All commands default to human-readable output. `--json` for AI/machine consumption.
Expand Down Expand Up @@ -4988,6 +4989,7 @@ USER_GUIDEの[終了コード](USER_GUIDE.md#終了コード)セクションを
- **extractor regex の backtracking policy** — built-in symbol/reference extractor は repository-controlled な file content に対して unbounded regex match を使わない。backtracking regex は `BoundedRegex.DefaultMatchTimeout` を使い、`RegexOptions.NonBacktracking` は non-backtracking engine と互換な pattern で使ってよい。lookaround-heavy な extractor や balancing-group を使う extractor など、意図的に backtracking-only のまま残す pattern は、共有 timeout audit の対象になる場合だけ許容する。将来の extractor が `System.Text.RegularExpressions.Regex` を直接使う必要がある場合は、明示 timeout を渡し、`BoundedRegex` や `NonBacktracking` が適さない理由を文書化すること。
- **ハイブリッドなシンボル抽出** — ASTパーサーも重量級の言語固有依存も追加しない方針。大半の言語はコンパイル済み正規表現で処理し、JavaScript / TypeScript だけは class body の method 抽出、private-scope filtering、synthetic class expression の binding 判定、JS/TS 固有の range 解決など、正規表現だけでは壊れやすい箇所を軽量 lexer / state machine で補う。引き続き精度より速度とポータビリティを優先しつつ、言語パターンや JS/TS state machine から推論できる範囲で定義範囲、本体範囲、シグネチャ、親シンボル、修飾付きコンテナ経路、正式なグループキー、可視性、戻り値型も保存する。Visual Basic では `Namespace ... End Namespace` も実コンテナとして扱い、implicit visibility の宣言や `Shared` / `Overrides` / `Partial` など visibility 以外の先行修飾子も受理するようにしたため、他のクラス系言語と同じようにトップレベル構造とメンバーを取りこぼしにくくなった。Visual Basic のコンテナパターンは `VisualBasicEnd` ベースの範囲追跡を大文字小文字非依存で扱うため、partial 型ファミリーでも安定した本体範囲と `hotspots` 集計用メタデータを維持できる。**パターン外部化**: 言語パターンは現在 `SymbolExtractor.cs` 内にコンパイル済み `Regex` として定義。抽出パイプラインが自己完結し、コンパイル時検証が効くが、言語追加にはコード変更と再ビルドが必要。将来的にはJSON/TOMLファイルに外部化し(起動時読み込み)、コミュニティ貢献の敷居を下げ、開発時のホットリロードも可能にできる。トレードオフはコンパイル時安全性の喪失と起動コストの微増。外部化時のスキーマ: 言語名、種別(function/class/import/namespace)、正規表現文字列、本体スタイル(brace/indent/ruby-end/none)、可視性・戻り値型のキャプチャグループ名。
- **C# の nested interpolation state** — C# lexical masker は、別の interpolation hole 内で interpolated regular / verbatim / raw string が始まると immutable な親 frame を保持する。nested string を閉じると外側の mode、delimiter、dollar count、brace depth を完全に復元し、expression-bodied property 内の call を declaration pattern から除外する。C# extractor contract の更新により、既存 index の対象ファイルも再抽出される。
- **C# static lambda の宣言ゲート** — 宣言 scanner は、確認済みの `static`、`static async`、`async static` lambda header 内にある候補名を式コンテキストとして扱います。複数行 property-header の結合によって呼び出し引数が前置された場合も同様です。本物の static member、local function、代入済み lambda symbol は引き続き抽出対象です。C# extractor contract v6 により、通常の index 更新で古い C# symbol が再抽出されます(#4830、#4453 の回帰)。
- **`hotspots` の正式な family trust** — `hotspots` が重名グループをコードベース全体の件数へ昇格させるのは、永続化済み `symbols.container_qualified_name` / `symbols.family_key` が現行の言語別 `hotspot_family_version_*` 契約で生成されたときだけ。readiness stamp と marker fingerprint は `codeindex_meta` に置き、旧形式・混在・部分更新直後の DB は古いファイル横断グループ識別子を黙って再利用せず、明示的に縮退する。
- **C# metadata-target の正式な trust** — `deps` / `impact` の metadata attribute edge(`[Foo]` 使用と定義側 `FooAttribute` クラスの紐付け)は、永続化済み `is_metadata_target` が現行の `metadata_target_version_csharp` 契約で stamp されている DB ではシグネチャ形状ヒューリスティックではなく authoritative resolver の判定結果を使う。resolver は C# クラスの base list を同 DB 内の class 行で fixed-point 展開して解決し、未解決の外部基底のみ BCL 規約(`Attribute` サフィックス)へフォールバックする。readiness は `codeindex_meta` に置き、reader は (1) ready → `is_metadata_target = 1`、(2) 列はあるが stamp 未完(legacy 行)→ `signature LIKE '%: %'`、(3) 列すらない → 命名のみ、の 3 way 分岐で縮退する。これにより、`class FooAttribute : BaseService` のような非 attribute 同名 impostor が真の `FooAttribute : Attribute` と同居したときにエッジを黙ってドロップする挙動を修正した(#435)。
- **人間向けがデフォルト** — 全コマンドのデフォルト出力は人間向け。`--json`でAI/機械向け出力。
Expand Down
2 changes: 2 additions & 0 deletions TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,7 @@ Use `docs/test-doc-maintenance-plan.md` before moving oversized suites or adding
- C# reflection-name extraction coverage keeps literal, constant-concatenation, dynamic, comment, and string-decoy cases in one source fixture so those parser boundaries share one symbol/reference pass.
- C# BOM extraction keeps a simple leading-BOM import fixture plus one mixed-newline fixture that simultaneously covers leading and mid-file BOM handling across CRLF, bare CR, and LF boundaries; do not repeat separate extraction passes for newline subsets already present in the mixed fixture.
- C# lambda-capture coverage keeps positive enclosing-local capture, parameter shadowing, and same-named-method isolation in one source fixture; a single capture assertion proves the negative regions did not leak.
- C# static-lambda declaration regression coverage keeps stateful, typed/untyped, explicit-return, both async-modifier orders, Unicode/escaped-identifier, multiline, nested, and argument-position forms in one extractor pass. Preserve real static members/local functions, including generic, constructor, explicit-interface, and verbatim-type-name forms, and an assigned-lambda range/container assertion in that fixture, plus one CLI `symbols` corpus fixture for phantom-name checks (#4830).
- Escaped-brace coverage for regular and verbatim interpolated C# strings shares one extraction fixture because both variants assert the same phantom-call exclusion.
- Direct and nested interpolations inside C# raw strings share one source fixture and one extraction pass while retaining distinct container assertions.
- C# nested-interpolation regression coverage keeps raw SQL, a completion template containing another interpolated string, following expression-bodied properties, same-line siblings, and enclosing-class range assertions in one fixture and extraction pass. A reader integration fixture reuses that lexical context around a nullable-generic `out`-parameter method and verifies its definition, outline, resolved reference, caller, and callee surfaces together.
Expand Down Expand Up @@ -1310,6 +1311,7 @@ dotnet test --filter "FullyQualifiedName~GitHelperTests"
- C# reflection-name 抽出 coverage は、literal、定数連結、dynamic、comment、string decoy を1つの source fixture にまとめ、これらの parser boundary で1回の symbol/reference pass を共有します。
- C# BOM 抽出は、単純な先頭 BOM import fixture と、CRLF・bare CR・LF 境界で先頭/mid-file BOM を同時に扱う1つの混在改行 fixture を維持します。混在 fixture に含まれる改行 subset ごとに抽出 pass を重複させないでください。
- C# lambda capture coverage は、外側 local の正例、parameter shadowing、同名 method 間の分離を1つの source fixture にまとめます。capture が1件だけである assertion により、negative region からの漏れも同時に検証します。
- C# static lambda の宣言回帰 coverage は、stateful、型あり/型なし、明示的戻り値型、両方の async modifier 順、Unicode/escape 識別子、複数行、入れ子、引数位置の各形式を1回の extractor pass にまとめます。同じ fixture で generic、constructor、明示的 interface、verbatim 型名を含む本物の static member / local function と、代入済み lambda の range / container assertion を維持し、phantom 名の確認には CLI `symbols` corpus fixture を1つ追加します(#4830)。
- C# interpolated string の escaped-brace coverage は、通常形式と逐語形式で同じ phantom call 除外を検証するため、1回の抽出 fixture を共有します。
- C# raw string 内の direct interpolation と nested interpolation は1つの source fixture と抽出 pass を共有し、container assertion は個別に維持します。
- C# nested interpolation の regression coverage は、raw SQL、別の interpolated string を含む completion template、後続の expression-bodied property、same-line sibling、enclosing class range の assertion を1つの fixture と抽出 pass にまとめます。reader integration fixture では、その lexical context の後に nullable generic の `out` parameter method を置き、definition、outline、resolved reference、caller、callee の各 surface をまとめて検証します。
Expand Down
22 changes: 22 additions & 0 deletions changelog.d/unreleased/4830.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
category: fixed
issues:
- 4830
affected:
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.Contracts.cs
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpPatterns.cs
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.ExtractCore.cs
- tests/CodeIndex.Tests/SymbolExtractorIssue4830Tests.cs
- tests/CodeIndex.Tests/QueryCommandRunnerIssue4830Tests.cs
- DEVELOPER_GUIDE.md
- TESTING_GUIDE.md
---

## English

- **C# static lambdas no longer emit phantom declarations (#4830; regression of #4453)** — Declaration scanning now rejects function and property candidates whose names fall inside a confirmed static-lambda header while preserving real static members, local functions, and assigned-lambda symbols. The C# extractor contract advances so normal index refreshes replace stale rows.

## 日本語

- **C# static lambda が phantom 宣言を生成しなくなりました(#4830、#4453 の回帰)** — 宣言 scan は、確認済み static-lambda header 内に名前がある function / property 候補を除外し、本物の static member、local function、代入済み lambda symbol を維持します。C# extractor contract を更新したため、通常の index 更新で古い行が置き換わります。
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,12 @@ public static partial class SymbolExtractor
private static readonly Regex CSharpConstOrStaticReadonlyFieldRegex = new(
@"(?:\bconst\b|\bstatic\b[^=;]*\breadonly\b|\breadonly\b[^=;]*\bstatic\b)",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private static readonly Regex CSharpExplicitLambdaReturnTypeRegex = new(
$@"^(?:{CSharpTypePattern})$",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private static readonly Regex CSharpStaticConstructorHeaderPrefixRegex = new(
$@"^(?:unsafe\s+)?static\s+(?:unsafe\s+)?(?<name>{CSharpIdentifierPattern})\s*\(\s*$",
RegexOptions.Compiled | RegexOptions.CultureInvariant);

private static bool IsCSharpConstOrStaticReadonlyField(string signature)
=> CSharpConstOrStaticReadonlyFieldRegex.IsMatch(signature);
Expand Down
Loading
Loading