diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index cc4167bf7a..9a7b8fe0e5 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -1236,6 +1236,7 @@ Process exit codes are coarse (`0` success including valid zero-row queries, `1` - **Language-aware reference extraction** — `references`, `callers`, `callees`, and `impact` are backed by an indexed reference table built only for languages where regex-based call/reference extraction is meaningful (30 of 46 languages). Unsupported languages intentionally fall back to text search instead of returning low-confidence pseudo-graph data. When a language is removed from graph support, `PurgeUnsupportedReferences` deletes its stale `symbol_references` rows on the next indexing run, and graph read paths additionally filter by supported languages to prevent stale edges from surviving between index runs. Shell is intentionally excluded because its command-style invocations (`foo arg1 arg2`) cannot be detected by the parenthesized-call regex. **Nested generic call sites**: C#/Java constructor calls like `new Dictionary>()` and C# generic method calls like `Helper.DoWork>()` are recovered by a depth-aware fallback scanner so the outer target still reaches the reference table even though the flat regex fast-path cannot balance `>>`. **JS/TS no-paren constructors**: JavaScript / TypeScript zero-argument constructor calls that legally omit `()` — for example `new Foo;`, `new Date;`, qualified targets like `new Demo.Provider;`, and one-level generic TypeScript forms like `new Box;` — are emitted as `instantiate` edges via a dedicated language-gated path, while next-line `.bar()` / `[0]` continuations are suppressed so a line-ended `new Foo` does not become a phantom standalone instantiation. **Constructor chain calls**: C# `: this(...)` / `: base(...)` initializers and Java `this(...)` / `super(...)` first-statement calls are detected separately from the generic call regex and rewritten so the reference target is the real constructor (enclosing class/record for `this`, the parsed base type from the class signature for `base` / `super`). Cross-line C# initializers are attributed to the owning constructor rather than the enclosing class. Base-type parsing strips generics, record primary-ctor args, `where` constraints, and `global::` / dotted namespace qualifiers; Java `super.method()` stays a normal method call. **Type-position dependency edges**: C#/Java base lists, declaration types, generic constraints, `throws`, `is`/`as`/`instanceof`, and real C# XML-doc `cref` sites are indexed as `type_reference` rows so `references` / `impact` can see compile-time rename dependencies without polluting the default dynamic call graph exposed by `callers` / `callees`. C# XML-doc `cref` extraction accepts declaration-attached XML-doc comments from both `///` lines and delimited `/** ... */` blocks, including declarations that begin later on the same physical line after the closing `*/` only when no unrelated same-line code or declaration intervenes, while ordinary `//` / `////` comments, non-documenting block comments, method-body XML-doc comments that merely precede a later declaration, brace-free field/property initializer continuations, brace-free expression lambdas, intervening top-level executable statements, same-line non-target code after `*/`, other nested executable continuations, and multiline raw/verbatim string content whose line happens to start with `/**` stay excluded. Non-doc code or string content after the closing `*/` on the same physical line is still outside the doc-comment slice. Even though the regex now runs against that narrower slice, the extractor preserves `symbol_references.column` relative to the original physical source line. On the C# read path, `using static` constant-pattern suppression is token-aware around `is` / `case`, reconstructs a small indexed multi-line window when the anchor lives on a previous line, and keeps trivia-bearing forms such as `value is/*comment*/Red`, `value is\n Red or Blue`, and `case\tRed:` filtered. Same-name type rescue also honors `file` visibility so file-local types only rescue references from the same physical file; inherited protected/public/internal nested types from real base classes rescue derived-class pattern heads only after the base reference is normalized through active type and namespace aliases, and alias-expanded constructed generic bases are canonicalized again before containing-type lookup so `AliasBase = Probe.Base` resolves the same way as `Probe.Base`; implemented interfaces do not contribute inherited nested-type rescue; and same-file `using Namespace;`, project-wide `global using Namespace;`, and active type aliases all participate in the rescue set. The extractor deliberately leaves ambiguous unqualified `using static` heads such as `value is Red` in the DB, because file-local parsing alone cannot know whether another file in the same namespace declares the real `Red` type; the workspace-aware read path is responsible for suppressing the pure constant-only cases. **SQL qualified-name alignment**: SQL definitions still persist their schema-qualified symbol name (`dbo.fn_X`), but graph/`deps`/unused/hotspot readers now resolve each SQL reference row through its stored source-line context, recorded call column, and enclosing container before they compare it to definitions, so qualified `references` / `callers` / `impact` queries stay schema-scoped even when one line contains multiple qualified calls or the lookup is non-exact. Those readers fall back to the bare leaf only when the source site itself is genuinely unqualified, which keeps `deps`, `unused`, and `hotspots` aligned with qualified SQL calls without regressing bare-call support or double-counting `EXEC dbo.fn_Target; EXEC sales.fn_Target;`. Once a row already has a recorded call column, those downstream readers no longer whole-line-upgrade that row to a later qualified token, so trailing comments, string literals, or a second qualified call cannot steal the earlier unqualified edge. Exact SQL graph/dependency readers also preserve the resolved segment count, so a quoted single identifier containing a dot such as `"sales.fn_Target"` stays distinct from the real qualified name `sales.fn_Target` across exact `references` / `callers` / `impact` and aggregate `deps` / `unused` / `hotspots`. SQL CTE body source rows use the raw `cte_body_reference` kind, so `references --kind cte_body_reference` can distinguish anchor/recursive-member internals from outer-query table references. Qualified SQL `callees` queries also keep leaf fallback disabled unless the caller query itself is unqualified, so `callees sales.Caller` no longer widens to `dbo.Caller`. SQL extractors also accept optional whitespace around qualified-name dots, so definitions/calls such as `[sales] . [fn_Target]` and `[dbo] . [fn_Target]` keep their full qualified identity instead of truncating at the first segment. The same SQL no-parens extractor now preserves ANSI / PostgreSQL double-quoted call targets such as `CALL "sales"."proc_name"` and `EXEC "dbo"."fn_Target"` instead of stripping them as string literals, while true single-quoted SQL string literals remain masked. Definition-oriented readers also canonicalize quoted qualified SQL names (`[dbo].[fn_X]` → `dbo.fn_X`) before matching, and they only fall back to the leaf identifier for unqualified queries so exact qualified lookups do not widen to sibling schemas that merely share the same leaf name. Exact SQL definition matching also preserves segment count, so a quoted single identifier that contains a dot (`"sales.fn_Target"`) does not collide with a real qualified name (`sales.fn_Target`). SQL exact graph leaf fallback also stays on the Unicode folded exact path, and both quoted qualified and unqualified Unicode exact definition lookups now use the folded normalized path, so queries such as `dbo.Äpfel` / `dbo.äpfel` and bare `Äpfel` / `äpfel` keep matching leaf call/reference rows such as `äpfel` plus stored definitions such as `[dbo].[Äpfel]` or `dbo.Äpfel` instead of silently degrading to ASCII-only `NOCASE`. Exact multi-name SQL `symbols --count` lookups also bind the folded leaf parameters on that same `_foldReady` path, so Unicode leaf query sets no longer fail with missing-parameter database errors. - **Language-aware reference extraction** — `references`, `callers`, `callees`, and `impact` are backed by an indexed reference table built only for languages where regex-based call/reference extraction is meaningful (30 of 46 languages). Unsupported languages intentionally fall back to text search instead of returning low-confidence pseudo-graph data. When a language is removed from graph support, `PurgeUnsupportedReferences` deletes its stale `symbol_references` rows on the next indexing run, and graph read paths additionally filter by supported languages to prevent stale edges from surviving between index runs. Shell is intentionally excluded because its command-style invocations (`foo arg1 arg2`) cannot be detected by the parenthesized-call regex. **Nested generic call sites**: C#/Java constructor calls like `new Dictionary>()` and C# generic method calls like `Helper.DoWork>()` are recovered by a depth-aware fallback scanner so the outer target still reaches the reference table even though the flat regex fast-path cannot balance `>>`. **JS/TS no-paren constructors**: JavaScript / TypeScript zero-argument constructor calls that legally omit `()` — for example `new Foo;`, `new Date;`, qualified targets like `new Demo.Provider;`, and one-level generic TypeScript forms like `new Box;` — are emitted as `instantiate` edges via a dedicated language-gated path, while next-line `.bar()` / `[0]` continuations are suppressed so a line-ended `new Foo` does not become a phantom standalone instantiation. **Constructor chain calls**: C# `: this(...)` / `: base(...)` initializers and Java `this(...)` / `super(...)` first-statement calls are detected separately from the generic call regex and rewritten so the reference target is the real constructor (enclosing class/record for `this`, the parsed base type from the class signature for `base` / `super`). Cross-line C# initializers are attributed to the owning constructor rather than the enclosing class. Base-type parsing strips generics, record primary-ctor args, `where` constraints, and `global::` / dotted namespace qualifiers; Java `super.method()` stays a normal method call. **Type-position dependency edges**: C#/Java base lists, declaration types, generic constraints, `throws`, `is`/`as`/`instanceof`, and real C# XML-doc `cref` sites are indexed as `type_reference` rows so `references` / `impact` can see compile-time rename dependencies without polluting the default dynamic call graph exposed by `callers` / `callees`. C# XML-doc `cref` extraction accepts declaration-attached XML-doc comments from both `///` lines and delimited `/** ... */` blocks, including declarations that begin later on the same physical line after the closing `*/` only when no unrelated same-line code or declaration intervenes, while ordinary `//` / `////` comments, non-documenting block comments, method-body XML-doc comments that merely precede a later declaration, brace-free field/property initializer continuations, brace-free expression lambdas, intervening top-level executable statements, same-line non-target code after `*/`, other nested executable continuations, and multiline raw/verbatim string content whose line happens to start with `/**` stay excluded. Non-doc code or string content after the closing `*/` on the same physical line is still outside the doc-comment slice. Even though the regex now runs against that narrower slice, the extractor preserves `symbol_references.column` relative to the original physical source line. On the C# read path, `using static` constant-pattern suppression is token-aware around `is` / `case`, reconstructs an anchor-aware indexed multi-line window when the anchor lives on a previous line, and keeps trivia-bearing forms such as `value is/*comment*/Red`, `value is\n Red or Blue`, `value is\n // comment\n Red`, `case\n // comment\n Point:`, long `case` / `or` chains, and `case\tRed:` filtered or rescued correctly. Qualified constant/member patterns stay qualifier-driven on that exact-name read path, so an unrelated same-name type such as `class Red {}` no longer cancels suppression for `case Color.Red or Color.Blue:` just because the leaf name matches. The extractor-side pending type-pattern carry now also survives trivia-only separator lines, standalone continuation-line `not`, and multiline `case` heads/logical continuations, so comment-only or `not`-only continuation lines no longer drop the later type head before the real token arrives. Non-type `case` labels such as `case > 0:` and `case not > 0:` do not arm that pending carry, so the next-line call/identifier token stays out of `type_reference`. Same-name type rescue also honors `file` visibility so file-local types only rescue references from the same physical file; inherited protected/public/internal nested types from real base classes rescue derived-class pattern heads only after the base reference is normalized through active type and namespace aliases, and alias-expanded constructed generic bases are canonicalized again before containing-type lookup so `AliasBase = Probe.Base` resolves the same way as `Probe.Base`; implemented interfaces do not contribute inherited nested-type rescue; and same-file `using Namespace;`, project-wide `global using Namespace;`, and active type aliases all participate in the rescue set. The extractor deliberately leaves ambiguous unqualified `using static` heads such as `value is Red` in the DB, because file-local parsing alone cannot know whether another file in the same namespace declares the real `Red` type; the workspace-aware read path is responsible for suppressing the pure constant-only cases. **SQL qualified-name alignment**: SQL definitions still persist their schema-qualified symbol name (`dbo.fn_X`), but graph/`deps`/unused/hotspot readers now resolve each SQL reference row through its stored source-line context, recorded call column, and enclosing container before they compare it to definitions, so qualified `references` / `callers` / `impact` queries stay schema-scoped even when one line contains multiple qualified calls or the lookup is non-exact. Those readers fall back to the bare leaf only when the source site itself is genuinely unqualified, which keeps `deps`, `unused`, and `hotspots` aligned with qualified SQL calls without regressing bare-call support or double-counting `EXEC dbo.fn_Target; EXEC sales.fn_Target;`. Once a row already has a recorded call column, those downstream readers no longer whole-line-upgrade that row to a later qualified token, so trailing comments, string literals, or a second qualified call cannot steal the earlier unqualified edge. Exact SQL graph/dependency readers also preserve the resolved segment count, so a quoted single identifier containing a dot such as `"sales.fn_Target"` stays distinct from the real qualified name `sales.fn_Target` across exact `references` / `callers` / `impact` and aggregate `deps` / `unused` / `hotspots`. SQL CTE body source rows use the raw `cte_body_reference` kind, so `references --kind cte_body_reference` can distinguish anchor/recursive-member internals from outer-query table references. Qualified SQL `callees` queries also keep leaf fallback disabled unless the caller query itself is unqualified, so `callees sales.Caller` no longer widens to `dbo.Caller`. SQL extractors also accept optional whitespace around qualified-name dots, so definitions/calls such as `[sales] . [fn_Target]` and `[dbo] . [fn_Target]` keep their full qualified identity instead of truncating at the first segment. The same SQL no-parens extractor now preserves ANSI / PostgreSQL double-quoted call targets such as `CALL "sales"."proc_name"` and `EXEC "dbo"."fn_Target"` instead of stripping them as string literals, while true single-quoted SQL string literals remain masked. Definition-oriented readers also canonicalize quoted qualified SQL names (`[dbo].[fn_X]` → `dbo.fn_X`) before matching, and they only fall back to the leaf identifier for unqualified queries so exact qualified lookups do not widen to sibling schemas that merely share the same leaf name. Exact SQL definition matching also preserves segment count, so a quoted single identifier that contains a dot (`"sales.fn_Target"`) does not collide with a real qualified name (`sales.fn_Target`). SQL exact graph leaf fallback also stays on the Unicode folded exact path, and both quoted qualified and unqualified Unicode exact definition lookups now use the folded normalized path, so queries such as `dbo.Äpfel` / `dbo.äpfel` and bare `Äpfel` / `äpfel` keep matching leaf call/reference rows such as `äpfel` plus stored definitions such as `[dbo].[Äpfel]` or `dbo.Äpfel` instead of silently degrading to ASCII-only `NOCASE`. Exact multi-name SQL `symbols --count` lookups also bind the folded leaf parameters on that same `_foldReady` path, so Unicode leaf query sets no longer fail with missing-parameter database errors. - **Transitive impact analysis** — `impact` and MCP `impact_analysis` compute the transitive caller chain of a symbol using BFS. Design constraints refined through adversarial review: caller matching uses case-insensitive exact match (`lower() = lower()`) to avoid both substring expansion and case-sensitivity brittleness; symbol names are pre-resolved through definitions with exact-case preference; the read path filters to graph-supported languages to prevent stale edges from removed languages; the definition set used for heuristic fallback must also respect active `--lang` / `--path` / `--exclude-path` / `--exclude-tests` filters and graph-supported languages so out-of-scope or unsupported duplicates do not suppress in-scope hints; fallback eligibility is keyed off class-like definitions only, so same-name namespace/import siblings do not block a single resolved class / struct / interface target, while pure non-callable `namespace` / `import` queries surface `non_callable_symbol_kind` guidance; heuristic file-level hints still return a successful result and encode their non-authoritative status via `impact_mode`, `heuristic`, `hint_count`, and `truncated`; caller rows include `result_kind: "graph"` and heuristic `file_impacts` rows include `result_kind: "file_heuristic"` so clients can distinguish authoritative hop-depth graph results from boundary fallback hints without inferring from list position or depth values; when `truncated` is `true`, the JSON / MCP payload also exposes `truncated_reason` so callers can distinguish actionable cases from runaway-graph cases — `user_limit` means the caller-supplied `--limit` was reached and raising `--limit` will return more results, while `safety_cap` means an internal per-symbol BFS fetch-iteration cap fired (the graph is likely pathological / cyclic and raising `--limit` alone will not help). `impact` / MCP `impact_analysis` also expose `termination_reason` (`completed`, `max_depth_reached`, `cycle_detected`, `row_limit_truncated`, `safety_cap`, or `cancelled`), `cycle_detected`, and `cycles` so caller cycles are distinguishable from natural traversal completion or limit/depth termination (#1883). `safety_cap` outranks `user_limit` whenever both are encountered, and the heuristic file-level hints path is `user_limit`-only because hint truncation is always driven by the caller's `--limit`. The field is omitted whenever `truncated` is `false`. (#1533) `count` / `file_count` now describe the visible returned set while `confirmed_count` / `confirmed_file_count` preserve symbol-level caller totals for heuristic-success payloads, and `impact --json --count` uses the same `*_count` field names as the full payload; to reduce general-name collisions, a file only qualifies for type fallback if it both references one of the candidate member names and also exposes same-file evidence anchoring the source/target pair — either a `call` / `instantiate` reference to the resolved target name (the call-graph itself authoritatively pins the relationship, so this path runs before the metadata-attribute bypass and does not depend on the looser ambiguity guard) or structured type evidence through indexed symbol metadata such as signatures or return types — rather than raw comment/string text matches. The call/instantiate anchor matches the resolved name exactly with no suffix-strip alias, because callable references already carry the authoritative identifier and applying the C# `[Foo]` → `FooAttribute` alias there would let unrelated `Foo()` method calls falsely anchor `impact FooAttribute` (#1881); the metadata bypass keeps the C# `Attribute` suffix alias because attribute use sites legitimately abbreviate the target name. The signature evidence path is Unicode-aware so fullwidth/accented identifiers are tokenized consistently with exact-name resolution; hint `reference_count` reflects the real number of matching reference rows while the symbol list stays deduplicated; only multiple class-like definitions are treated as fallback ambiguity, even when they share one file; and `PurgeUnsupportedReferences` runs in all three indexing paths (CLI full scan, CLI update mode, MCP index). +- **Extractor regex backtracking policy** — Built-in symbol and reference extractors must not use unbounded regular expression matching on repository-controlled file content. Backtracking regexes use `BoundedRegex.DefaultMatchTimeout`, while `RegexOptions.NonBacktracking` is allowed for patterns that are compatible with the non-backtracking engine. Patterns that deliberately remain backtracking-only, such as lookaround-heavy or balancing-group extractors, are acceptable only because the shared timeout audit covers them. If a future extractor must use `System.Text.RegularExpressions.Regex` directly, it must pass an explicit timeout and document why `BoundedRegex` or `NonBacktracking` is not suitable. - **Hybrid symbol extraction** — No AST parsers and no heavyweight language-specific dependencies. Most languages still use compiled regex patterns, while JavaScript/TypeScript add a lightweight lexer/state machine for class-body method extraction, private-scope filtering, synthetic class-expression binding detection, and JS/TS-specific range resolution that regex alone could not handle reliably. The trade-off still favors speed and portability over full parser accuracy, but the index stores richer symbol metadata such as definition ranges, optional body ranges, signatures, enclosing symbols, qualified container paths, authoritative family keys, visibility, and return types when the language patterns or JS/TS state machine can infer them. Visual Basic patterns also treat `Namespace ... End Namespace` as a real container and allow implicit-visibility declarations plus leading modifiers (`Shared`, `Overrides`, `Partial`, etc.), so VB projects expose the same top-level orientation and member coverage that other class-based languages already get. Visual Basic container patterns use case-insensitive `VisualBasicEnd` range tracking so cross-file partial families still get stable body ranges and can participate in hotspot-family grouping. **Pattern externalization**: Language patterns are currently defined inline in `SymbolExtractor.cs` using compiled `Regex` objects. This keeps the extraction pipeline self-contained and allows compile-time validation, but means adding a new language requires a code change and rebuild. A future iteration could externalize patterns to JSON/TOML files (loaded at startup), which would lower the barrier for community contributions and enable hot-reload during development. The trade-off is losing compile-time safety and slightly increasing startup cost. If externalized, patterns should include: language name, kind (function/class/import/namespace), regex string, body style (brace/indent/ruby-end/none), and optional capture group names for visibility and return type. - **Authoritative hotspot-family trust** — `hotspots` only promotes duplicate-name families back to codebase-wide counts when the persisted `symbols.container_qualified_name` / `symbols.family_key` were produced under the current per-language `hotspot_family_version_*` contract. These readiness stamps and marker fingerprints live in `codeindex_meta`, so legacy, mixed, or partially refreshed DBs degrade explicitly instead of silently reusing stale cross-file family identities. - **Authoritative C# metadata-target trust** — `deps` / `impact` metadata-attribute edges (linking `[Foo]` usage to the defining `FooAttribute` class) are promoted from a signature-shape heuristic to an authoritative resolver whenever `is_metadata_target` is persisted under the current `metadata_target_version_csharp` contract. The resolver walks C# class base lists with fixed-point transitive resolution through same-DB class rows and falls back to the BCL `Attribute` suffix convention only for unresolved external bases. Readiness lives in `codeindex_meta`, and the reader uses a three-way branch: (1) ready → `is_metadata_target = 1`; (2) column present but not stamped (legacy row) → `signature LIKE '%: %'`; (3) column missing → naming-only fallback. This fixes non-attribute impostors (`class FooAttribute : BaseService`) silently dropping edges when they shared names with real `FooAttribute : Attribute` classes (#435). @@ -2018,6 +2019,7 @@ The following categories ride the standard JSON-RPC codes: | `-32602` | `tool_unknown` | `false` | `tools/call` received an MCP tool name the server does not implement (typo or version mismatch). `data.tool` carries the unknown name. | | `-32602` | `missing_parameter` | `false` | `tools/call` request omitted the required `params.name` string. | | `-32602` | `invalid_argument` | `false` | Argument shape rejected by the tool (also covers the protocol-version handshake mismatch from #1554, which exposes `data.requestedVersion` / `data.supportedVersions`). | +| `-32602` | `regex_timeout` | `true` | A user-supplied regex exceeded the bounded match timeout while executing, for example `find_in_file` regex scans. `data.error_code` carries the CLI-aligned stable code. | | `-32603` | `internal_error` | `false` | Unhandled exception path (fallback bucket). The wire message stays generic per #1530 sanitization; stderr carries the exception type. | The classifier `McpErrorEnvelope.ClassifyException(ex)` maps unhandled @@ -3469,6 +3471,7 @@ USER_GUIDEの[終了コード](USER_GUIDE.md#終了コード)セクションを - **MCPサーバー instructions** — `initialize` レスポンスにツール選択ガイダンスの `instructions` 文字列を含め、AIクライアントが初回接続時に適切なツールを選べるようにする。 - **デプロイ単位での MCP ツール有効化** — `cdidx mcp` が 2 つの環境変数を尊重し、コード変更なしに公開ツールを絞れるようにする (#1561)。`CDIDX_MCP_TOOLS_ALLOW=` は厳格な allowlist で、指定された場合はそのツールだけが `tools/list` に現れ `tools/call` で dispatch される。`CDIDX_MCP_TOOLS_DENY=` は既定の全有効集合から個別ツールを除外する。両方指定された場合は allow を優先。既知ツール名の真実の源は `McpToolFilter.KnownToolNames` に集約し、`tools/list` 側の filter、`tools/call` 側のゲート、`batch_query` の slot ガードのいずれもここを参照する。`BuildInstructions` もゲート対応で、scoped デプロイの `initialize` instructions では無効化されたツールを推奨しなくなり、案内と公開面が一致する。トップレベル `tools/call` で無効化された既知ツールを呼ぶと `-32601 Tool not enabled: ` を返し、`batch_query` 自体はエンベロープとして成功するが各無効化スロットに `code: -32601` が `error` 文字列と並んで載るため、クライアントは prose を parsing せず code で分岐できる。サーバーに無い名前は既存の `-32602 Unknown tool` に流すことでオペレータ無効化と typo を区別できる。ツール名比較は大小文字無視、env var 内の未知名は既知集合で filter（typo で未知名のみの allowlist は意図的に何も公開しないため、ゲートが silent に外れない）。env var を一切設定しない既定挙動は全ツール有効なので、既存デプロイへの影響はない。 - **トリガー付きコンテンツ外部参照FTS5** — `chunks`テーブルを参照しコピーを保存しないことでストレージ倍増を回避。データベーストリガーでFTSインデックスを自動同期。 +- **extractor regex の backtracking policy** — built-in symbol/reference extractor は repository-controlled な file content に対して unbounded regex match を使わない。backtracking regex は `BoundedRegex.DefaultMatchTimeout` を使い、`RegexOptions.NonBacktracking` は non-backtracking engine と互換な pattern で使ってよい。lookaround-heavy な extractor や balancing-group を使う extractor など、意図的に backtracking-only のまま残す pattern は、共有 timeout audit の対象になる場合だけ許容する。将来の extractor が `System.Text.RegularExpressions.Regex` を直接使う必要がある場合は、明示 timeout を渡し、`BoundedRegex` や `NonBacktracking` が適さない理由を文書化すること。 - **ハイブリッドなシンボル抽出** — ASTパーサーも重量級の言語固有依存も追加しない方針。大半の言語はコンパイル済み正規表現で処理し、JavaScript / TypeScript だけは class body の method 抽出、private-scope filtering、synthetic class expression の binding 判定、JS/TS 固有の range 解決など、正規表現だけでは壊れやすい箇所を軽量 lexer / state machine で補う。引き続き精度より速度とポータビリティを優先しつつ、言語パターンや JS/TS state machine から推論できる範囲で定義範囲、本体範囲、シグネチャ、親シンボル、修飾付きコンテナ経路、正式なグループキー、可視性、戻り値型も保存する。Visual Basic では `Namespace ... End Namespace` も実コンテナとして扱い、implicit visibility の宣言や `Shared` / `Overrides` / `Partial` など visibility 以外の先行修飾子も受理するようにしたため、他のクラス系言語と同じようにトップレベル構造とメンバーを取りこぼしにくくなった。Visual Basic のコンテナパターンは `VisualBasicEnd` ベースの範囲追跡を大文字小文字非依存で扱うため、partial 型ファミリーでも安定した本体範囲と `hotspots` 集計用メタデータを維持できる。**パターン外部化**: 言語パターンは現在 `SymbolExtractor.cs` 内にコンパイル済み `Regex` として定義。抽出パイプラインが自己完結し、コンパイル時検証が効くが、言語追加にはコード変更と再ビルドが必要。将来的にはJSON/TOMLファイルに外部化し（起動時読み込み）、コミュニティ貢献の敷居を下げ、開発時のホットリロードも可能にできる。トレードオフはコンパイル時安全性の喪失と起動コストの微増。外部化時のスキーマ: 言語名、種別（function/class/import/namespace）、正規表現文字列、本体スタイル（brace/indent/ruby-end/none）、可視性・戻り値型のキャプチャグループ名。 - **`hotspots` の正式な family trust** — `hotspots` が重名グループをコードベース全体の件数へ昇格させるのは、永続化済み `symbols.container_qualified_name` / `symbols.family_key` が現行の言語別 `hotspot_family_version_*` 契約で生成されたときだけ。readiness stamp と marker fingerprint は `codeindex_meta` に置き、旧形式・混在・部分更新直後の DB は古いファイル横断グループ識別子を黙って再利用せず、明示的に縮退する。 - **C# metadata-target の正式な trust** — `deps` / `impact` の metadata attribute edge（`[Foo]` 使用と定義側 `FooAttribute` クラスの紐付け）は、永続化済み `is_metadata_target` が現行の `metadata_target_version_csharp` 契約で stamp されている DB ではシグネチャ形状ヒューリスティックではなく authoritative resolver の判定結果を使う。resolver は C# クラスの base list を同 DB 内の class 行で fixed-point 展開して解決し、未解決の外部基底のみ BCL 規約（`Attribute` サフィックス）へフォールバックする。readiness は `codeindex_meta` に置き、reader は (1) ready → `is_metadata_target = 1`、(2) 列はあるが stamp 未完（legacy 行）→ `signature LIKE '%: %'`、(3) 列すらない → 命名のみ、の 3 way 分岐で縮退する。これにより、`class FooAttribute : BaseService` のような非 attribute 同名 impostor が真の `FooAttribute : Attribute` と同居したときにエッジを黙ってドロップする挙動を修正した（#435）。 @@ -3871,6 +3874,7 @@ JSON-RPC 2.0 は `-32700` と `-32600..-32603` を仕様自身、`-32000..-32099 | `-32602` | `tool_unknown` | `false` | `tools/call` がサーバー未実装の MCP ツール名を指定した（typo またはバージョン不整合）。`data.tool` に未知の名前を含める。 | | `-32602` | `missing_parameter` | `false` | `tools/call` リクエストに必須 `params.name` 文字列が無い。 | | `-32602` | `invalid_argument` | `false` | ツールが引数 shape を拒否した（#1554 のプロトコルバージョン交渉ミスマッチもここで、`data.requestedVersion` / `data.supportedVersions` を併載）。 | +| `-32602` | `regex_timeout` | `true` | ユーザー指定 regex が実行中に bounded match timeout を超えた場合。例: `find_in_file` の regex scan。`data.error_code` に CLI と揃えた stable code を含める。 | | `-32603` | `internal_error` | `false` | 未処理例外の fallback バケット。ワイヤメッセージは #1530 の sanitization に従い汎用のまま、stderr に例外型を出す。 | 分類器 `McpErrorEnvelope.ClassifyException(ex)` は未処理例外を例外型と一部の `SqliteException.Message` サブストリングから `index_stale` / `index_corrupted` / `request_cancelled` / `internal_error` にマッピングする — 生メッセージはワイヤに乗らない（#1530）。`ProcessFrame` の JSON-RPC catch-all と `tools/call` の catch-all が同じ分類器を使うため、ツール呼び出し途中で `SqliteException` が起きても `error.data` でも `result.structuredContent` でも同じ `index_stale` envelope が surface する。 diff --git a/USER_GUIDE.md b/USER_GUIDE.md index ef2af552e2..71f2459611 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -1413,11 +1413,12 @@ If a query itself begins with `-`, pass it as `--query ` or `-- `. | `7` | Invalid argument value (for example invalid `--kind`, `--color`, or `--metrics`) | | `8` | Cancelled by signal / Ctrl-C (`SIGINT` / `SIGTERM`-style cancellation path) | | `9` | Install or upgrade installer failure (for example a failed `install.sh` start, timeout, download, checksum, or preparation step) | +| `10` | Runtime error from bounded query/index execution (for example regex match timeout or extraction stall) | | `99` | Unhandled exception after command dispatch; run `cdidx report` and inspect the lifecycle log | ### Error codes -For scripts and AI agents that need to classify failures without substring-matching the human prose, every CLI error carries a stable machine-readable code. Human stderr prefixes the code in brackets (`Error [E001_DB_NOT_FOUND]: database not found at …`) and CLI `--json` envelopes add an optional `error_code` field (omitted when not applicable, so existing JSON consumers see no schema break). MCP tool errors today surface as `isError: true` text content without a structured `error_code` field, and the bracketed CLI constant is not guaranteed to appear in the MCP message text — see [Troubleshooting](#troubleshooting) for the documented MCP message per failure mode that MCP clients should match. Codes never get renamed or reused once published — retired codes simply stop being emitted. +For scripts and AI agents that need to classify failures without substring-matching the human prose, every CLI error carries a stable machine-readable code. Human stderr prefixes the code in brackets (`Error [E001_DB_NOT_FOUND]: database not found at …`) and CLI `--json` envelopes add an optional `error_code` field (omitted when not applicable, so existing JSON consumers see no schema break). MCP tool errors usually surface as `isError: true` text content, while newer failure modes can also expose stable fields under `structuredContent`; the bracketed CLI constant is not guaranteed to appear in MCP message text. See [Troubleshooting](#troubleshooting) for the MCP message text and structured fields each failure mode expects clients to match. Codes never get renamed or reused once published — retired codes simply stop being emitted. | Code | When emitted | |---|---| @@ -1433,6 +1434,8 @@ For scripts and AI agents that need to classify failures without substring-match | `E010_USAGE_ERROR` | Argument parse error, conflicting flags, or unknown subcommand | | `E011_DIRECTORY_NOT_FOUND` | Project / target directory passed to `cdidx index` does not exist | | `E012_INTERRUPTED` | The user interrupted the command with Ctrl-C / signal cancellation | +| `E013_INDEX_EXTRACTION_STALLED` | Index extraction made no forward progress within the bounded stall timeout | +| `E014_REGEX_MATCH_TIMEOUT` | A user-supplied regular expression exceeded the bounded match timeout while executing | ### Debugging reader errors @@ -2369,7 +2372,7 @@ Suggestion history readers can query the local store by lifecycle status, create ## Troubleshooting -This section catalogs the failure modes you are most likely to hit while running `cdidx` and the concrete recovery steps for each one. Most coded CLI errors carry a stable code from the `E001`–`E011` taxonomy: human stderr prefixes the constant in brackets (for example `Error [E002_DB_LOCKED]: ...`) and the CLI `--json` envelope adds an optional `error_code` field. The canonical taxonomy lives in [Error codes](#error-codes) under `## Options` above; entries below that have a stable CLI error code tag it in the heading so CLI scripts can branch on it without parsing prose. Other entries cover warning, status-field, or verbose scan-message conditions that do not carry an `error_code` — those document the exact symptom string or `status --json` field to watch instead. MCP tool errors today surface as `isError: true` text content with no structured `error_code` field, and the bracketed CLI constant is not guaranteed to appear in the MCP message text (the E001 entry below is one such case). Until MCP exposes a structured error field, MCP clients should match the documented MCP message text where an entry records one (only the entries below that explicitly describe an MCP symptom), and otherwise rely on the CLI / `status --json` / `--verbose` symptom the entry documents. +This section catalogs the failure modes you are most likely to hit while running `cdidx` and the concrete recovery steps for each one. Most coded CLI errors carry a stable code from the `E001`–`E014` taxonomy: human stderr prefixes the constant in brackets (for example `Error [E002_DB_LOCKED]: ...`) and the CLI `--json` envelope adds an optional `error_code` field. The canonical taxonomy lives in [Error codes](#error-codes) under `## Options` above; entries below that have a stable CLI error code tag it in the heading so CLI scripts can branch on it without parsing prose. Other entries cover warning, status-field, or verbose scan-message conditions that do not carry an `error_code` — those document the exact symptom string or `status --json` field to watch instead. MCP tool errors usually surface as `isError: true` text content, but newer entries can also expose structured fields under `structuredContent`; where such fields exist, the entry below records the stable keys. MCP clients should match the documented MCP message text where an entry records one (only the entries below that explicitly describe an MCP symptom), and otherwise rely on the CLI / `status --json` / `--verbose` symptom the entry documents. ### Common failure modes @@ -2426,32 +2429,37 @@ When SQLite returns permission-style errors such as `SQLITE_AUTH`, `SQLITE_PERM` - Cause: the raw FTS5 string failed to parse — usually unbalanced quotes, an unsupported operator combo, a trailing `NEAR/OR`, or a column qualifier other than `content:`. - Recovery: drop `--fts` to use the default tokenizer, or fix the FTS5 expression. For prefix matching of a single token, prefer trailing `*` (e.g. `auth*`) without `--fts`. -10. **Files indexed with replacement characters (non-UTF-8 input)** +10. **Regex match timeout** (`E014_REGEX_MATCH_TIMEOUT`) + - Symptom: CLI `find --regex` exits `10` with `Error [E014_REGEX_MATCH_TIMEOUT]: ...`; `--json` responses include `error_code: "E014_REGEX_MATCH_TIMEOUT"` and `category: "regex_timeout"`. MCP `find_in_file` returns `isError: true` with `structuredContent.category: "regex_timeout"`, `retry_safe: true`, `error_code: "E014_REGEX_MATCH_TIMEOUT"`, and `timeout_ms`. + - Cause: the user-supplied regular expression exceeded the bounded match timeout while scanning indexed file contents. + - Recovery: simplify the pattern, narrow the scan with `--path` / `--lang`, or omit `--regex` when searching for literal text. + +11. **Files indexed with replacement characters (non-UTF-8 input)** - Symptom: `cdidx index --verbose` shows `[OK]` lines but the warning `: contains invalid UTF-8 bytes (replaced with U+FFFD)` is recorded. `cdidx validate` later reports `Likely non-UTF8 encoding (N U+FFFD over M chars, X.X%); source may be SHIFT_JIS, GBK, ISO-8859-1, or UTF-16 without BOM` for the same files. - Cause: the file is encoded in something other than UTF-8 (UTF-16 LE/BE files with BOM are decoded losslessly). To preserve indexability `cdidx` falls back to UTF-8 with replacement, but symbol names and snippets are corrupted at the offending bytes. - Recovery: re-save the file as UTF-8 (or add a UTF-16 BOM if you must keep UTF-16) and re-index — a normal `cdidx index .` will pick up the fixed file. Run `cdidx validate` to enumerate every affected file in one pass. -11. **Files skipped: permission denied mid-scan** +12. **Files skipped: permission denied mid-scan** - Symptom: `Could not scan directory due to permissions.` or `Could not probe file for indexability/language.` in `--verbose` output; the file is absent from search results. - Cause: the indexing process lacks read permission on the directory or file — common with system directories, other users' homes, or files locked by an editor. - Recovery: fix file/directory permissions, or exclude the path via `.cdidxignore`. The index keeps running across the rest of the tree; no rebuild is required after permissions are fixed — a normal `cdidx index .` will pick up the now-readable files. -12. **File rejected: too large** +13. **File rejected: too large** - Symptom: `validate --kind file_too_large` reports `File too large (N MiB > M MiB limit). Override with --max-file-bytes or CDIDX_MAX_FILE_BYTES= when this source file is intentionally indexable.` The file is listed in `files`, but no chunks, symbols, or references are indexed for it, so it does not appear in search. - Cause: the file exceeds the configured per-file size limit. Indexing huge generated files would waste tokens and bloat the DB. - Recovery: shrink or split the file, add it to `.cdidxignore`, or raise the limit with `cdidx index . --max-file-bytes 50M` / `CDIDX_MAX_FILE_BYTES=50M` when the file is legitimate source. Generated artifacts should generally be gitignored too. -13. **Feature unavailable on trimmed / AOT build** (`E009_FEATURE_UNAVAILABLE`) +14. **Feature unavailable on trimmed / AOT build** (`E009_FEATURE_UNAVAILABLE`) - Symptom: `Error [E009_FEATURE_UNAVAILABLE]: ...` when invoking flags such as `--json` on a build that lacks the required code paths. - Cause: the binary was produced with trimming or AOT settings that stripped the requested feature. - Recovery: use the standard published build, or rebuild without aggressive trimming. Check `cdidx --version` and the release notes for the feature matrix. -14. **Argument or usage error** (`E010_USAGE_ERROR`) +15. **Argument or usage error** (`E010_USAGE_ERROR`) - Symptom: `Error [E010_USAGE_ERROR]: ...` with a brief explanation of the offending flag combination, unknown subcommand, or missing argument. - Cause: conflicting flags (e.g. `--fts` with `--exact-substring`), an unknown option, or a literal starting with `--` mistaken for a flag. - Recovery: consult `cdidx --help`. For literals that begin with `--`, pass them via `--query "--path"` or quote them after `--`. -15. **Project directory missing** (`E011_DIRECTORY_NOT_FOUND`) +16. **Project directory missing** (`E011_DIRECTORY_NOT_FOUND`) - Symptom: `Error [E011_DIRECTORY_NOT_FOUND]: ...` with the requested path. - Cause: the project / target directory does not exist on disk, or the path was typed for a different host. - Recovery: pass an existing absolute path. `cdidx` does not create the project directory on your behalf. @@ -3866,11 +3874,12 @@ raw match density を正確に測る、といった理由で全 raw chunk hit | `7` | 引数値が不正（例: 不正な `--kind`、`--color`、`--metrics`） | | `8` | シグナル / Ctrl-C によるキャンセル（`SIGINT` / `SIGTERM` 系のキャンセル経路） | | `9` | install / upgrade installer の失敗（例: `install.sh` 起動失敗、timeout、download、checksum、準備処理の失敗） | +| `10` | 制限付きのクエリ／インデックス実行で発生した実行時エラー（regex match timeout や抽出停止など） | | `99` | コマンド dispatch 後の想定外例外。`cdidx report` とライフサイクルログを確認 | ### エラーコード -スクリプトや AI エージェントが人間向け文言の部分一致なしで失敗を分類できるよう、CLI のエラーには安定した機械可読コードが付与されます。人間向け stderr ではコードを角括弧で前置し（`Error [E001_DB_NOT_FOUND]: database not found at …`）、CLI `--json` エンベロープには任意フィールド `error_code` を追加します（該当しない場合は省略されるので、既存 JSON 利用者にスキーマ破壊なし）。MCP ツールエラーは現状 `isError: true` のテキストコンテンツとして返り、構造化された `error_code` フィールドを持たず、本文に CLI 側の角括弧付き定数が必ず含まれる保証もありません。MCP クライアントが照合すべき各失敗モードの MCP メッセージ本文は [トラブルシューティング](#トラブルシューティング) を参照してください。一度公開したコードは renaming / 使い回しをせず、廃止する場合も新規 emission を止めるだけです。 +スクリプトや AI エージェントが人間向け文言の部分一致なしで失敗を分類できるよう、CLI のエラーには安定した機械可読コードが付与されます。人間向け stderr ではコードを角括弧で前置し（`Error [E001_DB_NOT_FOUND]: database not found at …`）、CLI `--json` エンベロープには任意フィールド `error_code` を追加します（該当しない場合は省略されるので、既存 JSON 利用者にスキーマ破壊なし）。MCP ツールエラーは通常 `isError: true` のテキストコンテンツとして返りますが、新しい失敗モードでは `structuredContent` に安定フィールドを持つこともあります。本文に CLI 側の角括弧付き定数が必ず含まれる保証はありません。MCP クライアントが照合すべき各失敗モードの MCP メッセージ本文と構造化フィールドは [トラブルシューティング](#トラブルシューティング) を参照してください。一度公開したコードは renaming / 使い回しをせず、廃止する場合も新規 emission を止めるだけです。 | コード | 発行条件 | |---|---| @@ -3886,6 +3895,8 @@ raw match density を正確に測る、といった理由で全 raw chunk hit | `E010_USAGE_ERROR` | 引数のパースエラー、フラグの競合、未知のサブコマンド | | `E011_DIRECTORY_NOT_FOUND` | `cdidx index` に渡したプロジェクト / 対象ディレクトリが存在しない | | `E012_INTERRUPTED` | Ctrl-C / signal cancellation でユーザーがコマンドを中断した | +| `E013_INDEX_EXTRACTION_STALLED` | 制限付きの停止判定時間内に index 抽出が前進しなかった | +| `E014_REGEX_MATCH_TIMEOUT` | ユーザー指定の正規表現が実行中に制限付き match timeout を超えた | ### reader エラーのデバッグ @@ -4807,7 +4818,7 @@ cdidx には、AI エージェントがギャップや不具合に気づいた ## トラブルシューティング -`cdidx` を使っているときに遭遇しやすい代表的な失敗モードと、その具体的な復旧手順をまとめています。コード付きの CLI エラーには `E001`〜`E011` の安定コードが付与され、人間向け stderr では定数を角括弧でくるんで（例: `Error [E002_DB_LOCKED]: ...`）、CLI `--json` の envelope では任意フィールド `error_code` として付加されます。正準な分類表は上の `## オプション一覧` 内 [エラーコード](#エラーコード) にあるので、安定 CLI エラーコードを持つ項目では見出しで併記し、CLI スクリプトが文面 grep 不要で分岐できるようにしています。コードを持たない警告・ステータスフィールド・`--verbose` スキャン診断などの項目では、代わりに監視対象となる具体的なメッセージ文字列や `status --json` フィールドを記載しています。MCP ツールエラーは現状 `isError: true` のテキストコンテンツとして返り、構造化された `error_code` フィールドは持ちません。また、MCP メッセージ本文に CLI 側の角括弧付き定数が必ず含まれる保証もありません（下記 E001 の項目がその一例です）。MCP 側に構造化フィールドが追加されるまで、MCP クライアントは MCP の症状を明示記載している該当項目では記録済みの MCP メッセージ本文と照合し、それ以外の項目では各項目に記載した CLI / `status --json` / `--verbose` の症状を参照してください。 +`cdidx` を使っているときに遭遇しやすい代表的な失敗モードと、その具体的な復旧手順をまとめています。コード付きの CLI エラーには `E001`〜`E014` の安定コードが付与され、人間向け stderr では定数を角括弧でくるんで（例: `Error [E002_DB_LOCKED]: ...`）、CLI `--json` の envelope では任意フィールド `error_code` として付加されます。正準な分類表は上の `## オプション一覧` 内 [エラーコード](#エラーコード) にあるので、安定 CLI エラーコードを持つ項目では見出しで併記し、CLI スクリプトが文面 grep 不要で分岐できるようにしています。コードを持たない警告・ステータスフィールド・`--verbose` スキャン診断などの項目では、代わりに監視対象となる具体的なメッセージ文字列や `status --json` フィールドを記載しています。MCP ツールエラーは通常 `isError: true` のテキストコンテンツとして返りますが、新しい項目では `structuredContent` に構造化フィールドを持つこともあります。そのようなフィールドがある場合は、下記の項目で安定キーを記録しています。MCP クライアントは MCP の症状を明示記載している該当項目では記録済みの MCP メッセージ本文と照合し、それ以外の項目では各項目に記載した CLI / `status --json` / `--verbose` の症状を参照してください。 ### よくある失敗モード @@ -4873,32 +4884,37 @@ SELinux profile が取れれば confinement-aware hint を追加します。 - 原因: 生の FTS5 文字列のパースに失敗した。引用符の不整合、サポートされない演算子の組み合わせ、末尾の `NEAR/OR`、または `content:` 以外の列修飾子などが多い。 - 復旧: `--fts` を外してデフォルトトークナイザを使うか、FTS5 表現を直す。単一トークンのプレフィックスマッチなら `--fts` を使わずに `auth*` のような末尾 `*` で十分。 -10. **置換文字付きで索引化される（非 UTF-8 入力）** +10. **正規表現の match timeout**（`E014_REGEX_MATCH_TIMEOUT`） + - 症状: CLI の `find --regex` が `Error [E014_REGEX_MATCH_TIMEOUT]: ...` を出して終了コード `10` で終了する。`--json` 応答には `error_code: "E014_REGEX_MATCH_TIMEOUT"` と `category: "regex_timeout"` が含まれる。MCP の `find_in_file` は `isError: true` を返し、`structuredContent.category: "regex_timeout"`、`retry_safe: true`、`error_code: "E014_REGEX_MATCH_TIMEOUT"`、`timeout_ms` を含む。 + - 原因: ユーザー指定の正規表現が、索引済みファイル内容の走査中に制限付き match timeout を超えた。 + - 復旧: 正規表現を単純化する、`--path` / `--lang` で走査範囲を絞る、またはリテラル検索では `--regex` を外す。 + +11. **置換文字付きで索引化される（非 UTF-8 入力）** - 症状: `cdidx index --verbose` の出力は `[OK]` だが、`: contains invalid UTF-8 bytes (replaced with U+FFFD)` という警告が記録される。あとで `cdidx validate` を実行すると同じファイルに対し `Likely non-UTF8 encoding (N U+FFFD over M chars, X.X%); source may be SHIFT_JIS, GBK, ISO-8859-1, or UTF-16 without BOM` を報告する。 - 原因: ファイルが UTF-8 ではない（BOM 付き UTF-16 LE/BE は損失なく decode される）。索引化を継続するために `cdidx` は U+FFFD への置換付き UTF-8 にフォールバックするが、該当バイト位置のシンボル名やスニペットは壊れる。 - 復旧: ファイルを UTF-8 で保存し直す（UTF-16 を維持する場合は BOM を付ける）と、通常の `cdidx index .` で取り込まれる。`cdidx validate` を使えば対象ファイルを一括で列挙できる。 -11. **ファイルがスキップされる: 走査中に権限エラー** +12. **ファイルがスキップされる: 走査中に権限エラー** - 症状: `--verbose` 出力に `Could not scan directory due to permissions.` や `Could not probe file for indexability/language.` が出て、当該ファイルが検索結果に現れない。 - 原因: インデックスプロセスがディレクトリ／ファイルの読み取り権限を持っていない。システムディレクトリ、他ユーザーのホーム、エディタが排他保持しているファイルなどで起きやすい。 - 復旧: ファイル／ディレクトリ権限を直すか、`.cdidxignore` で除外する。インデックスはツリーの他の部分は走査を続けるので、権限修正後は通常の `cdidx index .` で取り込まれ、`--rebuild` は不要。 -12. **ファイルが拒否される: サイズ超過** +13. **ファイルが拒否される: サイズ超過** - 症状: `validate --kind file_too_large` が `File too large (N MiB > M MiB limit). Override with --max-file-bytes or CDIDX_MAX_FILE_BYTES= when this source file is intentionally indexable.` を報告する。対象 file は `files` に載るが、chunk、symbol、reference は index されないため search には現れない。 - 原因: ファイルが設定された 1 ファイルあたりサイズ上限を超えている。巨大な生成ファイルを索引化するとトークンを浪費し DB が肥大化する。 - 復旧: ファイルを縮小／分割する、`.cdidxignore` に追加する、または正当な source file なら `cdidx index . --max-file-bytes 50M` / `CDIDX_MAX_FILE_BYTES=50M` で上限を上げる。生成物は基本的に `.gitignore` 対象でもあるはず。 -13. **トリム / AOT ビルドで機能が無い**（`E009_FEATURE_UNAVAILABLE`） +14. **トリム / AOT ビルドで機能が無い**（`E009_FEATURE_UNAVAILABLE`） - 症状: `--json` などのフラグで `Error [E009_FEATURE_UNAVAILABLE]: ...` が出る。 - 原因: trimming / AOT の設定で必要なコードパスが落とされたバイナリ。 - 復旧: 公式の通常ビルドを使う、または積極的なトリムなしで再ビルドする。`cdidx --version` と各リリースの機能マトリクスを確認すること。 -14. **引数 / 利用エラー**（`E010_USAGE_ERROR`） +15. **引数 / 利用エラー**（`E010_USAGE_ERROR`） - 症状: `Error [E010_USAGE_ERROR]: ...` で衝突したフラグ、未知のサブコマンド、または不足引数の短い説明が出る。 - 原因: 競合するフラグ（例: `--fts` と `--exact-substring`）、未知のオプション、または `--` で始まるリテラルをフラグと誤認した。 - 復旧: `cdidx --help` を確認する。`--` で始まるリテラルは `--query "--path"` のように渡すか、`--` の後にクォートして渡す。 -15. **プロジェクトディレクトリが存在しない**（`E011_DIRECTORY_NOT_FOUND`） +16. **プロジェクトディレクトリが存在しない**（`E011_DIRECTORY_NOT_FOUND`） - 症状: 指定パスを伴う `Error [E011_DIRECTORY_NOT_FOUND]: ...`。 - 原因: プロジェクト／対象ディレクトリがディスク上に無い、または別ホスト用のパスを打っている。 - 復旧: 実在する絶対パスを渡す。`cdidx` は対象ディレクトリを勝手に作らない。 diff --git a/changelog.d/unreleased/3438.fixed.md b/changelog.d/unreleased/3438.fixed.md new file mode 100644 index 0000000000..e52749e69d --- /dev/null +++ b/changelog.d/unreleased/3438.fixed.md @@ -0,0 +1,16 @@ +--- +category: fixed +issues: + - 3438 +affected: + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.TypeScriptPathAliases.cs + - tests/CodeIndex.Tests/SymbolExtractorTests.cs +--- + +## English + +- **TypeScript path alias config warnings now preserve failure categories (#3438)** - unreadable configs, invalid JSON, and excessive extends depth now emit stable diagnostic codes while keeping the existing human-readable warning text. + +## 日本語 + +- **TypeScript path alias config の警告で失敗カテゴリを保持するようにしました (#3438)** - 読み取れない config、不正な JSON、過剰な extends 深度では、既存の人間向け警告文を保ったまま安定した診断コードを出力します。 diff --git a/changelog.d/unreleased/3479.fixed.md b/changelog.d/unreleased/3479.fixed.md new file mode 100644 index 0000000000..8de60c61e3 --- /dev/null +++ b/changelog.d/unreleased/3479.fixed.md @@ -0,0 +1,17 @@ +--- +category: fixed +issues: + - 3479 +affected: + - DEVELOPER_GUIDE.md + - tests/CodeIndex.Tests/SymbolExtractorTests.cs + - tests/CodeIndex.Tests/ReferenceExtractorTests.cs +--- + +## English + +- **Extractor regex backtracking policy is now documented and audited (#3479)** - built-in symbol and reference extractor regexes must either use `RegexOptions.NonBacktracking` or the shared `BoundedRegex.DefaultMatchTimeout`. + +## 日本語 + +- **extractor regex の backtracking policy を文書化し audit するようになりました (#3479)** - built-in symbol/reference extractor regex は `RegexOptions.NonBacktracking` または共有 `BoundedRegex.DefaultMatchTimeout` を使う必要があります。 diff --git a/changelog.d/unreleased/3510.fixed.md b/changelog.d/unreleased/3510.fixed.md new file mode 100644 index 0000000000..918a3811ec --- /dev/null +++ b/changelog.d/unreleased/3510.fixed.md @@ -0,0 +1,17 @@ +--- +category: fixed +issues: + - 3510 +affected: + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs + - tests/CodeIndex.Tests/SymbolExtractorTests.cs +--- + +## English + +- **Shell symbol extraction now ignores heredoc bodies and bounds function ranges (#3510)** - shell heredoc contents no longer emit phantom functions, and shell function ranges stop at their matching body close instead of stretching to EOF. + +## 日本語 + +- **shell のシンボル抽出で heredoc 本文を無視し、関数範囲を制限しました (#3510)** - shell heredoc 内の内容が偽の関数として出力されなくなり、shell 関数範囲は EOF ではなく対応する本体の閉じ位置で止まります。 diff --git a/changelog.d/unreleased/3516.fixed.md b/changelog.d/unreleased/3516.fixed.md new file mode 100644 index 0000000000..734e464ea9 --- /dev/null +++ b/changelog.d/unreleased/3516.fixed.md @@ -0,0 +1,23 @@ +--- +category: fixed +issues: + - 3516 +affected: + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs + - src/CodeIndex/Indexer/Symbols/SymbolExtractor.Sql.cs + - src/CodeIndex/Indexer/References/ReferenceExtractor.cs + - src/CodeIndex/Indexer/References/Languages/CssReferenceExtractor.cs + - src/CodeIndex/Indexer/References/Languages/DockerfileReferenceExtractor.cs + - src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.Patterns.cs + - src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.cs + - tests/CodeIndex.Tests/SymbolExtractorTests.cs + - tests/CodeIndex.Tests/ReferenceExtractorTests.cs +--- + +## English + +- **Case-insensitive extractor regexes now use culture-invariant matching consistently (#3516)** - built-in symbol and reference extractor regex audits now fail if an `IgnoreCase` regex omits `CultureInvariant`. + +## 日本語 + +- **case-insensitive な extractor regex が一貫して culture-invariant matching を使うようになりました (#3516)** - built-in symbol/reference extractor の regex audit は、`IgnoreCase` regex が `CultureInvariant` を省いた場合に失敗します。 diff --git a/changelog.d/unreleased/3559.fixed.md b/changelog.d/unreleased/3559.fixed.md new file mode 100644 index 0000000000..b92d117294 --- /dev/null +++ b/changelog.d/unreleased/3559.fixed.md @@ -0,0 +1,23 @@ +--- +category: fixed +issues: + - 3559 +affected: + - src/CodeIndex/Database/DbReader.FilesStatus.cs + - src/CodeIndex/Cli/QueryCommandRunner.cs + - src/CodeIndex/Cli/CommandErrorCodes.cs + - src/CodeIndex/Cli/CommandErrorWriter.cs + - src/CodeIndex/Cli/CommandExitCodes.cs + - src/CodeIndex/Mcp/McpErrorEnvelope.cs + - src/CodeIndex/Mcp/McpToolHandlers.cs + - tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs + - DEVELOPER_GUIDE.md +--- + +## English + +- **`find --regex` timeouts now use the shared bounded regex policy (#3559)** - CLI and MCP `find` regex scans distinguish match timeouts from invalid regex syntax and emit machine-readable `regex_timeout` diagnostics. + +## 日本語 + +- **`find --regex` の timeout が共有 bounded regex policy を使うようになりました (#3559)** - CLI と MCP の `find` regex scan は match timeout と regex 構文エラーを区別し、機械可読な `regex_timeout` 診断を出します。 diff --git a/src/CodeIndex/Cli/CommandErrorCodes.cs b/src/CodeIndex/Cli/CommandErrorCodes.cs index 2c33191a50..f36ad85fd1 100644 --- a/src/CodeIndex/Cli/CommandErrorCodes.cs +++ b/src/CodeIndex/Cli/CommandErrorCodes.cs @@ -53,4 +53,7 @@ internal static class CommandErrorCodes /// Index extraction made no forward progress within the bounded stall timeout. public const string IndexExtractionStalled = "E013_INDEX_EXTRACTION_STALLED"; + + /// A user-supplied regular expression exceeded the bounded match timeout while executing. + public const string RegexMatchTimeout = "E014_REGEX_MATCH_TIMEOUT"; } diff --git a/src/CodeIndex/Cli/CommandErrorWriter.cs b/src/CodeIndex/Cli/CommandErrorWriter.cs index 2ff066088e..915d3947f1 100644 --- a/src/CodeIndex/Cli/CommandErrorWriter.cs +++ b/src/CodeIndex/Cli/CommandErrorWriter.cs @@ -43,12 +43,13 @@ internal static int WriteJsonOrHuman( int exitCode, string? hint = null, string? usage = null, - string? errorCode = null) + string? errorCode = null, + string? category = null) { if (json) { WriteStdout(JsonSerializer.Serialize( - new CommandErrorJsonResult("error", message, hint, errorCode), + new CommandErrorJsonResult("error", message, hint, errorCode, Category: category), CliJsonSerializerContextFactory.Create(jsonOptions).CommandErrorJsonResult)); return exitCode; } diff --git a/src/CodeIndex/Cli/CommandExitCodes.cs b/src/CodeIndex/Cli/CommandExitCodes.cs index 3c7daa47b6..ee7b1fce68 100644 --- a/src/CodeIndex/Cli/CommandExitCodes.cs +++ b/src/CodeIndex/Cli/CommandExitCodes.cs @@ -16,6 +16,7 @@ public static class CommandExitCodes public const int InvalidArgument = 7; public const int CancelledBySignal = 8; public const int InstallError = 9; + public const int RuntimeError = 10; public const int UnhandledException = 99; public const int ExUsage = 64; public const int Interrupted = CancelledBySignal; diff --git a/src/CodeIndex/Cli/QueryCommandRunner.cs b/src/CodeIndex/Cli/QueryCommandRunner.cs index e4b0b6b767..ecf52c6d0e 100644 --- a/src/CodeIndex/Cli/QueryCommandRunner.cs +++ b/src/CodeIndex/Cli/QueryCommandRunner.cs @@ -3984,8 +3984,9 @@ public static int RunFind(string[] cmdArgs, JsonSerializerOptions jsonOptions) } catch (Exception ex) when (options.Regex && (ex is ArgumentException || ex is RegexMatchTimeoutException)) { - Console.Error.WriteLine($"Error: invalid regular expression: {ex.Message}"); - return CommandExitCodes.UsageError; + return ex is RegexMatchTimeoutException timeout + ? WriteFindRegexTimeoutError(timeout, jsonOptions, options.Json) + : WriteFindInvalidRegexError(ex); } if (counts.Count == 0) { @@ -4037,13 +4038,11 @@ public static int RunFind(string[] cmdArgs, JsonSerializerOptions jsonOptions) } catch (ArgumentException ex) when (options.Regex) { - Console.Error.WriteLine($"Error: invalid regular expression: {ex.Message}"); - return CommandExitCodes.UsageError; + return WriteFindInvalidRegexError(ex); } catch (RegexMatchTimeoutException ex) when (options.Regex) { - Console.Error.WriteLine($"Error: invalid regular expression: {ex.Message}"); - return CommandExitCodes.UsageError; + return WriteFindRegexTimeoutError(ex, jsonOptions, options.Json); } var results = findResults.Results; if (results.Count == 0) @@ -4126,6 +4125,32 @@ public static int RunFind(string[] cmdArgs, JsonSerializerOptions jsonOptions) }); } + private static int WriteFindInvalidRegexError(Exception ex) + { + Console.Error.WriteLine($"Error: invalid regular expression: {ex.Message}"); + return CommandExitCodes.UsageError; + } + + internal static int WriteFindRegexTimeoutError(RegexMatchTimeoutException ex, JsonSerializerOptions jsonOptions, bool json) + { + var timeout = FormatRegexMatchTimeout(ex.MatchTimeout); + return CommandErrorWriter.WriteJsonOrHuman( + json, + jsonOptions, + $"regular expression timed out after {timeout} while scanning indexed file contents.", + CommandExitCodes.RuntimeError, + hint: "Simplify the pattern, narrow the scan with --path/--lang, or omit --regex for literal text.", + errorCode: CommandErrorCodes.RegexMatchTimeout, + category: "regex_timeout"); + } + + internal static string FormatRegexMatchTimeout(TimeSpan timeout) + { + if (timeout.TotalMilliseconds < 1000) + return timeout.TotalMilliseconds.ToString("0.###", CultureInfo.InvariantCulture) + "ms"; + return timeout.TotalSeconds.ToString("0.###", CultureInfo.InvariantCulture) + "s"; + } + private static string? ValidateFindArgs(string[] args) { var (allowedWithValues, allowedFlags) = CliFlagSchema.GetParserFlagsPartitionedByValueBearing("find"); diff --git a/src/CodeIndex/Database/DbReader.FilesStatus.cs b/src/CodeIndex/Database/DbReader.FilesStatus.cs index dc8a1a0ef1..2fb92bf0c3 100644 --- a/src/CodeIndex/Database/DbReader.FilesStatus.cs +++ b/src/CodeIndex/Database/DbReader.FilesStatus.cs @@ -18,7 +18,7 @@ public FindResults FindInFiles(string query, int limit, string? lang = null, IRe maxLineWidth = LineWidthFormatter.ClampMaxLineWidth(maxLineWidth); var comparison = exact ? StringComparison.Ordinal : StringComparison.OrdinalIgnoreCase; var regexMatcher = regex - ? new Regex(query, exact ? RegexOptions.None : RegexOptions.IgnoreCase, TimeSpan.FromMilliseconds(500)) + ? CreateFindRegexMatcher(query, exact) : null; using var fileCmd = _conn.CreateCommand(); @@ -167,7 +167,7 @@ public FindCountResult CountFindInFiles(string query, string? lang = null, IRead var comparison = exact ? StringComparison.Ordinal : StringComparison.OrdinalIgnoreCase; var regexMatcher = regex - ? new Regex(query, exact ? RegexOptions.None : RegexOptions.IgnoreCase, TimeSpan.FromMilliseconds(500)) + ? CreateFindRegexMatcher(query, exact) : null; using var fileCmd = _conn.CreateCommand(); var sql = "SELECT f.id, f.path, f.lang, f.lines FROM files f WHERE 1=1"; @@ -351,6 +351,14 @@ private static bool TryCreateFindLineMatch(int matchColumn, int matchLength, int return !focusColumn.HasValue || (focusColumn.Value >= rawMatchColumn + 1 && focusColumn.Value <= rawMatchColumn + rawMatchLength); } + private static Regex CreateFindRegexMatcher(string query, bool exact) + { + var options = RegexOptions.CultureInvariant; + if (!exact) + options |= RegexOptions.IgnoreCase; + return new Regex(query, options, BoundedRegex.DefaultMatchTimeout); + } + private static void AddLineToFindWindow(IndexedLine indexedLine, Queue snippetWindow, Dictionary snippetLinesByNumber) { snippetWindow.Enqueue(indexedLine); diff --git a/src/CodeIndex/Indexer/References/Languages/CssReferenceExtractor.cs b/src/CodeIndex/Indexer/References/Languages/CssReferenceExtractor.cs index cdd6162a2c..d41c87ef73 100644 --- a/src/CodeIndex/Indexer/References/Languages/CssReferenceExtractor.cs +++ b/src/CodeIndex/Indexer/References/Languages/CssReferenceExtractor.cs @@ -20,9 +20,9 @@ internal static class CssReferenceExtractor @"@include\s+(?[A-Za-z_][\w-]*)", RegexOptions.Compiled); - private static readonly Regex CssCustomPropertyReferenceRegex = new(@"\bvar\(\s*--(?[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase); - private static readonly Regex CssAnimationNameValueRegex = new(@"\banimation-name\s*:\s*(?[^;{}]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase); - private static readonly Regex CssAnimationShorthandValueRegex = new(@"\banimation\s*:\s*(?[^;{}]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase); + private static readonly Regex CssCustomPropertyReferenceRegex = new(@"\bvar\(\s*--(?[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); + private static readonly Regex CssAnimationNameValueRegex = new(@"\banimation-name\s*:\s*(?[^;{}]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); + private static readonly Regex CssAnimationShorthandValueRegex = new(@"\banimation\s*:\s*(?[^;{}]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex CssClassSelectorReferenceRegex = new(@"\.(?[\w-]+)", RegexOptions.Compiled); // First char restricted to letter/`_`/`-` so numeric hex colors like `#336699` // do not match. Letter-only hex colors (`#fff`) are still ambiguous; the @@ -33,7 +33,7 @@ internal static class CssReferenceExtractor private static readonly Regex CssIdSelectorReferenceRegex = new(@"#(?[A-Za-z_-][\w-]*)", RegexOptions.Compiled); private static readonly Regex CssImportReferenceRegex = new( @"@import\s+(?:url\(\s*)?(?:""(?[^""]+)""|'(?[^']+)'|(?[^\s)""';]+))", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex CssInlineBlockCommentRegex = new(@"/\*.*?\*/", RegexOptions.Compiled); private static readonly ReferencePattern[] CssReferencePatterns = diff --git a/src/CodeIndex/Indexer/References/Languages/DockerfileReferenceExtractor.cs b/src/CodeIndex/Indexer/References/Languages/DockerfileReferenceExtractor.cs index b255ba7045..57757a119e 100644 --- a/src/CodeIndex/Indexer/References/Languages/DockerfileReferenceExtractor.cs +++ b/src/CodeIndex/Indexer/References/Languages/DockerfileReferenceExtractor.cs @@ -8,15 +8,15 @@ internal static class DockerfileReferenceExtractor { private static readonly Regex StageReferenceRegex = new( @"^\s*FROM\s+(?:--platform=\S+\s+)?(?[A-Za-z0-9_.-]+)\s+AS\s+[A-Za-z0-9_.-]+(?:\s+#.*)?\s*$", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex CopyFromReferenceRegex = new( @"^\s*(?:ONBUILD\s+)?(?:COPY|ADD)\b.*?--from=[""']?(?[A-Za-z0-9_.-]+)(?![:/@])\b[""']?", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex RunMountFromReferenceRegex = new( @"(?:^|,)from=[""']?(?[A-Za-z0-9_.-]+)(?![:/@])\b[""']?", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex UnbracedVariableReferenceRegex = new( @"(?[A-Za-z_][A-Za-z0-9_]*)", diff --git a/src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.Patterns.cs b/src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.Patterns.cs index a9273b8467..ec5b0c6a4b 100644 --- a/src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.Patterns.cs +++ b/src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.Patterns.cs @@ -40,269 +40,269 @@ internal static partial class SqlReferenceExtractor @"(?:(?:" + QuotedIdentifierPattern + "|" + BareIdentifierPattern + @")\s*\.\s*)*(?" + QuotedIdentifierPattern + "|" + BareIdentifierPattern + @")\s*\.\s*(?:" + QuotedIdentifierPattern + "|" + BareIdentifierPattern + @")"; private static readonly Regex CteDefinitionRegex = new( $@"(?{QuotedIdentifierPattern}|{BareIdentifierPattern})(?:\s*\([^)]*\))?\s+AS\s*\(", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex ProcCallRegex = new( @"(?" + ProcCallIdentifierPattern + @")", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex SystemVariableReferenceRegex = new( @"(?@@[_\p{L}][\p{L}\p{Mn}\p{Mc}\p{Nd}\p{Pc}$]*(?:\s*\.\s*[_\p{L}][\p{L}\p{Mn}\p{Mc}\p{Nd}\p{Pc}$]*)?)", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex FromSourceListRegex = new( $@"(?[\s\S]*?)(?=(?\((?:[^()]|\([^()]*\))*\))?(?:\s+VALUES\s*(?\((?:[^()]|\([^()]*\))*\)))?", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex MergeOnClauseRegex = new( @"(?[\s\S]*?)(?=(?{QuotedIdentifierPattern}|{BareIdentifierPattern})", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex MergeUsingPrefixRegex = new( $@"(?TOP)\s*\(", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex AccessMethodCallSuppressionRegex = new( $@"(?{QuotedIdentifierPattern}|{BareIdentifierPattern})(?=\s*\()", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex CreateIndexOnTargetRegex = new( $@"(?{TempIdentifierPattern})", - RegexOptions.Compiled | RegexOptions.IgnoreCase); - private static readonly Regex UsingKeywordRegex = new(@"(?{TempIdentifierPattern})", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex GeneratedColumnMarkerRegex = new( @"\b(?:GENERATED\s+(?:ALWAYS\s+)?AS|NEXT\s+VALUE\s+FOR)\b|(?{QualifiedIdentifierNoCapturePattern})", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex SqlExpressionIdentifierRegex = new( $@"(?{QualifiedIdentifierNoCapturePattern})", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex TrailingTempIdentifierRegex = new( $@"^(?:(?:ONLY)\b\s+)?(?(?:{TempIdentifierPattern}|{QualifiedIdentifierNoCapturePattern}))(?:\s+(?:AS\s+)?(?:{QuotedIdentifierPattern}|{BareIdentifierPattern}))?\s*$", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex MergeTargetHintContinuationPrefixRegex = new( $@"(?ROWS|RANGE|GROUPS|BETWEEN|UNBOUNDED|PRECEDING|FOLLOWING|CURRENT|ROW|EXCLUDE|TIES|OTHERS|NO)\b", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); } diff --git a/src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.cs b/src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.cs index f1c827a18c..fea026048a 100644 --- a/src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.cs +++ b/src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.cs @@ -2466,8 +2466,8 @@ private static bool IsGeneratedColumnDependencyKeyword(string name) private static bool IsLikelyComputedColumnAsExpression(string statement, int asIndex) { var prefix = statement[..asIndex]; - return Regex.IsMatch(prefix, @"(?` / ``。 private static readonly Regex CSharpDocCrefRegex = new( @"<(?:see|seealso)\s+cref\s*=\s*""(?[^""]+)""", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); // Javadoc / KDoc cross-reference links (`{@link Foo#bar}`, `@see Foo`, `[Foo.bar]`). // Javadoc / KDoc の cross-reference link。 private static readonly Regex JvmDocInlineLinkRegex = new( diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs index 15eada0356..95d6f37672 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs @@ -1435,6 +1435,7 @@ private static int FindSameLineBraceEndColumn(string line, int startColumn, stri "css" => FindCssSameLineBraceEndColumn(line, startColumn), "csharp" => FindCSharpSameLineBraceEndColumn(line, startColumn), "java" => FindJavaSameLineBraceEndColumn(line, startColumn), + "shell" => FindShellSameLineBraceEndColumn(line, startColumn), _ => -1, }; } diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Sql.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Sql.cs index 4b26fa94cf..d90edfa9fd 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Sql.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.Sql.cs @@ -11,7 +11,7 @@ public static partial class SymbolExtractor // Regex helpers for SQL procedure body scanning / SQL プロシージャ本体走査用の正規表現ヘルパー private static readonly Regex SqlGoSeparatorRegex = new( @"^\s*GO\s*(?:;[\s;]*)?(?:--.*)?$", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); // Only close a SQL proc body when the next top-level statement looks like another proc-like // header (`CREATE|ALTER|DROP PROCEDURE|PROC|FUNCTION|TRIGGER`, optionally with `OR REPLACE` for // PostgreSQL or `OR ALTER` for T-SQL / SQL Server 2016+). Body-internal `CREATE TABLE` / @@ -25,7 +25,7 @@ public static partial class SymbolExtractor // `CREATE OR ALTER PROCEDURE` の隣接宣言でも前の body 範囲を確実に終端させる。issue #429 参照。 private static readonly Regex SqlTopLevelDdlStartRegex = new( @"^\s*(?:CREATE|ALTER|DROP)\s+(?:OR\s+(?:REPLACE|ALTER)\s+)?(?:PROCEDURE|PROC|FUNCTION|TRIGGER)\b", - RegexOptions.Compiled | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); // Dollar-quoted body tags: `$$` or `$tagname$` (PostgreSQL). Tag must be empty or an identifier. // Dollar-quoted の本体タグ: `$$` または `$タグ名$`（PostgreSQL）。タグは空か識別子のみ。 private static readonly Regex SqlDollarTagRegex = new( diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.TypeScriptPathAliases.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.TypeScriptPathAliases.cs index 5794f75ef6..f7e9def6f1 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.TypeScriptPathAliases.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.TypeScriptPathAliases.cs @@ -16,6 +16,10 @@ public static partial class SymbolExtractor private const int MaxTypeScriptPathAliasTargetLength = 1024; private const int MaxTypeScriptPathAliasModuleSpecifierLength = 4096; private const int MaxTypeScriptPathAliasSubstitutedTargetLength = 4096; + private const string TypeScriptPathAliasDiagnosticReadFailed = "tsconfig_read_failed"; + private const string TypeScriptPathAliasDiagnosticJsonInvalid = "tsconfig_json_invalid"; + private const string TypeScriptPathAliasDiagnosticSizeLimit = "tsconfig_size_limit"; + private const string TypeScriptPathAliasDiagnosticDepthLimit = "path_alias_depth_limit"; private static readonly object TypeScriptPathAliasWarningLock = new(); private static readonly HashSet TypeScriptPathAliasReportedWarnings = new(StringComparer.Ordinal); private static readonly JsonDocumentOptions TypeScriptPathAliasConfigJsonOptions = new() @@ -29,6 +33,8 @@ private sealed record TypeScriptPathAliasConfig(string ConfigPath, string Projec private sealed record TypeScriptPathAliasRule(string Pattern, string BaseDirectory, IReadOnlyList Targets); + private readonly record struct TypeScriptPathAliasConfigSkippedReason(string Code, string Reason); + private static string ResolveJavaScriptTypeScriptModuleSpecifier(string lang, string? filePath, string? projectRoot, string moduleName) { if (lang is not ("typescript" or "javascript") || string.IsNullOrWhiteSpace(filePath)) @@ -97,7 +103,7 @@ private static string ResolveJavaScriptTypeScriptModuleSpecifier(string lang, st foreach (var configFileName in new[] { "tsconfig.json", "jsconfig.json" }) { var configPath = Path.Combine(directory, configFileName); - if (File.Exists(configPath)) + if (File.Exists(configPath) || Directory.Exists(configPath)) { var totalConfigBytesRead = 0L; return ParseTypeScriptPathAliasConfig( @@ -130,7 +136,10 @@ private static string ResolveJavaScriptTypeScriptModuleSpecifier(string lang, st if (depth > MaxTypeScriptPathAliasExtendsDepth) { ReportTypeScriptPathAliasWarningOnce( - $"Skipped TypeScript path alias config {configPath} because the extends depth exceeds {MaxTypeScriptPathAliasExtendsDepth}."); + FormatTypeScriptPathAliasConfigSkippedMessage( + configPath, + TypeScriptPathAliasDiagnosticDepthLimit, + $"the extends depth exceeds {MaxTypeScriptPathAliasExtendsDepth}")); return null; } @@ -144,7 +153,7 @@ private static string ResolveJavaScriptTypeScriptModuleSpecifier(string lang, st out var skippedReason)) { ReportTypeScriptPathAliasWarningOnce( - $"Skipped TypeScript path alias config {configPath} because {skippedReason}."); + FormatTypeScriptPathAliasConfigSkippedMessage(configPath, skippedReason)); return null; } @@ -155,11 +164,19 @@ private static string ResolveJavaScriptTypeScriptModuleSpecifier(string lang, st catch (JsonException) { ReportTypeScriptPathAliasWarningOnce( - $"Skipped TypeScript path alias config {configPath} because it could not be parsed as JSON within the {MaxTypeScriptPathAliasConfigJsonDepth}-level depth limit."); + FormatTypeScriptPathAliasConfigSkippedMessage( + configPath, + TypeScriptPathAliasDiagnosticJsonInvalid, + $"it could not be parsed as JSON within the {MaxTypeScriptPathAliasConfigJsonDepth}-level depth limit")); return null; } catch { + ReportTypeScriptPathAliasWarningOnce( + FormatTypeScriptPathAliasConfigSkippedMessage( + configPath, + TypeScriptPathAliasDiagnosticReadFailed, + "it could not be read")); return null; } @@ -284,10 +301,10 @@ private static bool TryReadTypeScriptPathAliasConfigText( string configPath, ref long totalConfigBytesRead, out string text, - out string skippedReason) + out TypeScriptPathAliasConfigSkippedReason skippedReason) { text = string.Empty; - skippedReason = string.Empty; + skippedReason = default; try { @@ -301,13 +318,13 @@ private static bool TryReadTypeScriptPathAliasConfigText( if (stream.Length > MaxTypeScriptPathAliasConfigBytes) { - skippedReason = $"it exceeds {MaxTypeScriptPathAliasConfigBytes} bytes"; + skippedReason = new(TypeScriptPathAliasDiagnosticSizeLimit, $"it exceeds {MaxTypeScriptPathAliasConfigBytes} bytes"); return false; } if (totalConfigBytesRead + stream.Length > MaxTypeScriptPathAliasTotalConfigBytes) { - skippedReason = $"the extends chain exceeds {MaxTypeScriptPathAliasTotalConfigBytes} bytes"; + skippedReason = new(TypeScriptPathAliasDiagnosticSizeLimit, $"the extends chain exceeds {MaxTypeScriptPathAliasTotalConfigBytes} bytes"); return false; } @@ -320,14 +337,14 @@ private static bool TryReadTypeScriptPathAliasConfigText( fileBytesRead += read; if (fileBytesRead > MaxTypeScriptPathAliasConfigBytes) { - skippedReason = $"it exceeds {MaxTypeScriptPathAliasConfigBytes} bytes"; + skippedReason = new(TypeScriptPathAliasDiagnosticSizeLimit, $"it exceeds {MaxTypeScriptPathAliasConfigBytes} bytes"); return false; } totalConfigBytesRead += read; if (totalConfigBytesRead > MaxTypeScriptPathAliasTotalConfigBytes) { - skippedReason = $"the extends chain exceeds {MaxTypeScriptPathAliasTotalConfigBytes} bytes"; + skippedReason = new(TypeScriptPathAliasDiagnosticSizeLimit, $"the extends chain exceeds {MaxTypeScriptPathAliasTotalConfigBytes} bytes"); return false; } @@ -339,18 +356,24 @@ private static bool TryReadTypeScriptPathAliasConfigText( text = text[1..]; return true; } - catch (IOException) - { - skippedReason = "it could not be read"; - return false; - } - catch (UnauthorizedAccessException) + catch (Exception ex) when (IsTypeScriptPathAliasConfigReadException(ex)) { - skippedReason = "it could not be read"; + skippedReason = new(TypeScriptPathAliasDiagnosticReadFailed, "it could not be read"); return false; } } + private static string FormatTypeScriptPathAliasConfigSkippedMessage( + string configPath, + TypeScriptPathAliasConfigSkippedReason reason) => + FormatTypeScriptPathAliasConfigSkippedMessage(configPath, reason.Code, reason.Reason); + + private static string FormatTypeScriptPathAliasConfigSkippedMessage(string configPath, string code, string reason) => + $"Skipped TypeScript path alias config {configPath} [{code}] because {reason}."; + + private static bool IsTypeScriptPathAliasConfigReadException(Exception exception) => + exception is IOException or UnauthorizedAccessException or NotSupportedException; + private static void ReportTypeScriptPathAliasWarningOnce(string message) { lock (TypeScriptPathAliasWarningLock) diff --git a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs index aa8a05e959..4c2972e24a 100644 --- a/src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs +++ b/src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs @@ -42,6 +42,9 @@ public static partial class SymbolExtractor private static readonly Regex JavaScriptTypeScriptModuleDocRegex = new( @"@module(?:\s+(?[^\s*]+))?", RegexOptions.Compiled | RegexOptions.CultureInvariant); + private static readonly Regex ShellHeredocRedirectRegex = new( + @"(?[^""]+)""|'(?[^']+)'|\\?(?[A-Za-z_][A-Za-z0-9_]*))", + RegexOptions.Compiled | RegexOptions.CultureInvariant); public static int GetContractVersion(string? lang) { @@ -212,16 +215,16 @@ public static int GetContractVersion(string? lang) RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex PhpGroupUseRegex = new( @"^\s*use\s+(?:(?function|const)\s+)?(?[\w\\]+\\)\{\s*(?[^{}]+?)\s*\}\s*;", - RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex PhpUseRegex = new( @"^\s*use\s+(?:(?function|const)\s+)?(?[\w\\]+)(?:\s+as\s+(?\w+))?\s*;", - RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex PhpRequireIncludeRegex = new( @"^\s*(?:require|include)(?:_once)?\s*\(?\s*(?:'(?[^']+)'|""(?[^""]+)"")\s*\)?\s*;", - RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); private static readonly Regex PhpPrefixedRequireIncludeRegex = new( @"^\s*(?:require|include)(?:_once)?\s*\(?\s*(?(?:(?:__DIR__|__FILE__|dirname\s*\(\s*__FILE__\s*\))\s*\.\s*)+)\s*(?:'(?[^']+)'|""(?[^""]+)"")\s*\)?\s*;", - RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase); + RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant); // `delegate` is a non-type keyword only when it is NOT followed by `*` — `delegate*<...>` is a valid return type. // `delegate` は `*` を伴わないときだけ非型キーワード扱い。`delegate*<...>` は戻り値型として有効。 private const string CSharpNonTypeKeywordPattern = @"(?:(?:public|private|protected|internal|static|sealed|partial|readonly|unsafe|extern|virtual|override|abstract|async|new|file|required|ref)\b|delegate\b(?!\s*\*))"; @@ -856,8 +859,8 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult // Keep the extraction deliberately small and conservative: one symbol per program. // COBOL は brace ではなく program ID 単位で構成されるため、抽出は保守的に // program ひとつにつき 1 symbol に絞る。 - new("class", new Regex(@"^\s*(?:IDENTIFICATION\s+DIVISION\.\s*)?(?:PROGRAM|CLASS)-ID\.\s*(?[A-Z0-9][A-Z0-9-]*)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("function", new Regex(@"^\s*METHOD-ID\.\s*(?:""(?[^""]+)""|'(?[^']+)'|(?[A-Z0-9][A-Z0-9-]*))", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("class", new Regex(@"^\s*(?:IDENTIFICATION\s+DIVISION\.\s*)?(?:PROGRAM|CLASS)-ID\.\s*(?[A-Z0-9][A-Z0-9-]*)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("function", new Regex(@"^\s*METHOD-ID\.\s*(?:""(?[^""]+)""|'(?[^']+)'|(?[A-Z0-9][A-Z0-9-]*))", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), ], ["javascript"] = [ @@ -1602,7 +1605,7 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult new("function", new Regex(@"^\s*\$(?\w+)\s*=\s*(?:static\s+)?function\s*\(", RegexOptions.Compiled), BodyStyle.Brace), new("function", new Regex(@"^\s*\$(?\w+)\s*=\s*(?:static\s+)?fn\s*\(", RegexOptions.Compiled), BodyStyle.None), // Const declaration / 定数宣言 - new("function", new Regex(@"^\s*define\s*\(\s*['""](?[A-Za-z_]\w*)['""]\s*,", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("function", new Regex(@"^\s*define\s*\(\s*['""](?[A-Za-z_]\w*)['""]\s*,", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), new("function", new Regex(@"^\s*(?:(?public|private|protected)\s+)?const\s+(?\??[A-Za-z_\\][\w\\]*(?:\s*[|&]\s*\??[A-Za-z_\\][\w\\]*)*)\s+(?\w+)\s*=", RegexOptions.Compiled), BodyStyle.None, "visibility", ReturnTypeGroup: "returnType"), new("function", new Regex(@"^\s*(?:(?public|private|protected)\s+)?const\s+(?\w+)\s*=", RegexOptions.Compiled), BodyStyle.None, "visibility"), // Class property declarations / クラスプロパティ宣言 @@ -1704,21 +1707,21 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult ], ["vb"] = [ - new("namespace", new Regex(@"^\s*Namespace\s+(?(?:Global\.)?" + VbIdentifierPattern + @"(?:\." + VbIdentifierPattern + @")*)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.VisualBasicEnd), - new("delegate", new Regex(@$"^\s*(?:(?:{VbMemberModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?Delegate\s+(?:Sub|Function)\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None, "visibility"), - new("function", new Regex(@$"^\s*(?:(?:{VbMemberModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbMemberModifierPattern})\s+)*(?:Sub|Function)\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None, "visibility"), - new("operator", new Regex(@$"^\s*(?:(?:{VbOperatorModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbOperatorModifierPattern})\s+)*(?Operator\s+[^\s(]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.VisualBasicEnd, "visibility"), - new("property", new Regex(@$"^\s*(?:(?:Shared|Shadows)\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:Shared|Shadows)\s+)*Const\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None, "visibility"), - new("property", new Regex(@$"^\s*(?:(?:Shared|Shadows|ReadOnly|WithEvents)\s+)*(?{VbVisibilityPattern})\s+(?:(?:Shared|Shadows|ReadOnly|WithEvents)\s+)*(?{VbIdentifierPattern})\s+As\s+", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None, "visibility"), - new("property", new Regex(@$"^\s*(?:(?:{VbPropertyModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbPropertyModifierPattern})\s+)*Property\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None, "visibility"), - new("event", new Regex(@$"^\s*(?:(?:{VbEventModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbEventModifierPattern})\s+)*Event\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None, "visibility"), - new("interface", new Regex(@$"^\s*(?:(?:{VbTypeModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbTypeModifierPattern})\s+)*Interface\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.VisualBasicEnd, "visibility"), - new("enum", new Regex(@$"^\s*(?:(?{VbVisibilityPattern})\s+)?Enum\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.VisualBasicEnd, "visibility"), - new("struct", new Regex(@$"^\s*(?:(?:Partial)\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:Partial)\s+)*Structure\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.VisualBasicEnd, "visibility"), - new("class", new Regex(@$"^\s*(?:(?:{VbTypeModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbTypeModifierPattern})\s+)*(?:Class|Module)\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.VisualBasicEnd, "visibility"), - new("import", new Regex(@"^\s*Imports\s+<\s*xmlns:(?[A-Za-z_][\w.-]*)\s*=", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("import", new Regex(@$"^\s*Imports\s+(?{VbIdentifierPattern})\s*=", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("import", new Regex(@"^\s*Imports\s+(?.+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("namespace", new Regex(@"^\s*Namespace\s+(?(?:Global\.)?" + VbIdentifierPattern + @"(?:\." + VbIdentifierPattern + @")*)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.VisualBasicEnd), + new("delegate", new Regex(@$"^\s*(?:(?:{VbMemberModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?Delegate\s+(?:Sub|Function)\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None, "visibility"), + new("function", new Regex(@$"^\s*(?:(?:{VbMemberModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbMemberModifierPattern})\s+)*(?:Sub|Function)\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None, "visibility"), + new("operator", new Regex(@$"^\s*(?:(?:{VbOperatorModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbOperatorModifierPattern})\s+)*(?Operator\s+[^\s(]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.VisualBasicEnd, "visibility"), + new("property", new Regex(@$"^\s*(?:(?:Shared|Shadows)\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:Shared|Shadows)\s+)*Const\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None, "visibility"), + new("property", new Regex(@$"^\s*(?:(?:Shared|Shadows|ReadOnly|WithEvents)\s+)*(?{VbVisibilityPattern})\s+(?:(?:Shared|Shadows|ReadOnly|WithEvents)\s+)*(?{VbIdentifierPattern})\s+As\s+", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None, "visibility"), + new("property", new Regex(@$"^\s*(?:(?:{VbPropertyModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbPropertyModifierPattern})\s+)*Property\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None, "visibility"), + new("event", new Regex(@$"^\s*(?:(?:{VbEventModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbEventModifierPattern})\s+)*Event\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None, "visibility"), + new("interface", new Regex(@$"^\s*(?:(?:{VbTypeModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbTypeModifierPattern})\s+)*Interface\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.VisualBasicEnd, "visibility"), + new("enum", new Regex(@$"^\s*(?:(?{VbVisibilityPattern})\s+)?Enum\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.VisualBasicEnd, "visibility"), + new("struct", new Regex(@$"^\s*(?:(?:Partial)\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:Partial)\s+)*Structure\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.VisualBasicEnd, "visibility"), + new("class", new Regex(@$"^\s*(?:(?:{VbTypeModifierPattern})\s+)*(?:(?{VbVisibilityPattern})\s+)?(?:(?:{VbTypeModifierPattern})\s+)*(?:Class|Module)\s+(?{VbIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.VisualBasicEnd, "visibility"), + new("import", new Regex(@"^\s*Imports\s+<\s*xmlns:(?[A-Za-z_][\w.-]*)\s*=", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("import", new Regex(@$"^\s*Imports\s+(?{VbIdentifierPattern})\s*=", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("import", new Regex(@"^\s*Imports\s+(?.+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), ], ["scala"] = [ @@ -1849,17 +1852,17 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult ["msbuild"] = [], ["dockerfile"] = [ - new("build_arg", new Regex(@"^\s*ARG\s+(?[A-Za-z_][A-Za-z0-9_]*)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("environment", new Regex(@"^\s*ENV\s+(?[A-Za-z_][A-Za-z0-9_]*)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("label", new Regex(@"^\s*LABEL\s+(?[A-Za-z0-9_.-]+)\s*=", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("label", new Regex(@"^\s*LABEL\s+(?[A-Za-z0-9_.-]+)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("expose", new Regex(@"^\s*EXPOSE\s+(?\d+(?:/(?:tcp|udp))?)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("user", new Regex(@"^\s*USER\s+(?[A-Za-z0-9_][A-Za-z0-9_.-]*(?::[A-Za-z0-9_][A-Za-z0-9_.-]*)?)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("workdir", new Regex(@"^\s*WORKDIR\s+(?\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("volume", new Regex(@"^\s*VOLUME\s+(?(?!\[)\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("stopsignal", new Regex(@"^\s*STOPSIGNAL\s+(?\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("stage", new Regex(@"^\s*FROM\s+(?:--platform=\S+\s+)?\S+\s+(?:AS|as)\s+(?[A-Za-z0-9_.-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), // Named stage / 名前付きステージ - new("base_image", new Regex(@"^\s*FROM\s+(?:--platform=\S+\s+)?(?\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), // Base image / ベースイメージ + new("build_arg", new Regex(@"^\s*ARG\s+(?[A-Za-z_][A-Za-z0-9_]*)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("environment", new Regex(@"^\s*ENV\s+(?[A-Za-z_][A-Za-z0-9_]*)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("label", new Regex(@"^\s*LABEL\s+(?[A-Za-z0-9_.-]+)\s*=", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("label", new Regex(@"^\s*LABEL\s+(?[A-Za-z0-9_.-]+)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("expose", new Regex(@"^\s*EXPOSE\s+(?\d+(?:/(?:tcp|udp))?)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("user", new Regex(@"^\s*USER\s+(?[A-Za-z0-9_][A-Za-z0-9_.-]*(?::[A-Za-z0-9_][A-Za-z0-9_.-]*)?)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("workdir", new Regex(@"^\s*WORKDIR\s+(?\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("volume", new Regex(@"^\s*VOLUME\s+(?(?!\[)\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("stopsignal", new Regex(@"^\s*STOPSIGNAL\s+(?\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("stage", new Regex(@"^\s*FROM\s+(?:--platform=\S+\s+)?\S+\s+(?:AS|as)\s+(?[A-Za-z0-9_.-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // Named stage / 名前付きステージ + new("base_image", new Regex(@"^\s*FROM\s+(?:--platform=\S+\s+)?(?\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // Base image / ベースイメージ ], ["protobuf"] = [ @@ -1890,7 +1893,7 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult //（schema.name、[dbo].[sp_X]、"s"."n"）。 // CREATE TABLE / VIEW — Postgres TEMP/UNLOGGED + MATERIALIZED VIEW, T-SQL `CREATE OR ALTER` (2016+) // CREATE TABLE / VIEW — Postgres の TEMP/UNLOGGED や MATERIALIZED VIEW、T-SQL の `CREATE OR ALTER`（2016+）に対応 - new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+(?:REPLACE|ALTER)\s+)?(?:(?:(?:GLOBAL|LOCAL)\s+)?(?:TEMP|TEMPORARY)\s+|UNLOGGED\s+)?(?:TABLE|(?:MATERIALIZED\s+)?VIEW)\s+(?:IF\s+NOT\s+EXISTS\s+)?(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+(?:REPLACE|ALTER)\s+)?(?:(?:(?:GLOBAL|LOCAL)\s+)?(?:TEMP|TEMPORARY)\s+|UNLOGGED\s+)?(?:TABLE|(?:MATERIALIZED\s+)?VIEW)\s+(?:IF\s+NOT\s+EXISTS\s+)?(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // CREATE PROCEDURE / PROC / FUNCTION / TRIGGER — Postgres `OR REPLACE` and T-SQL `OR ALTER` / `PROC` short form // Uses BodyStyle.SqlProcBody so the body range covers the BEGIN...END / dollar-quoted body, // letting ReferenceExtractor.ResolveContainerForCall attribute calls inside the body to the @@ -1898,49 +1901,49 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult // CREATE PROCEDURE / PROC / FUNCTION / TRIGGER — Postgres の `OR REPLACE` と T-SQL の `OR ALTER` / 短縮形 `PROC` に対応 // BodyStyle.SqlProcBody により BEGIN...END / dollar-quoted の本体範囲を求め、ReferenceExtractor の // ResolveContainerForCall が本体内の呼び出しを外側のプロシージャに帰属させられるようにする（issue #429）。 - new("function", new Regex($@"^\s*CREATE\s+(?:OR\s+(?:REPLACE|ALTER)\s+)?(?:PROCEDURE|PROC|FUNCTION|TRIGGER)\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.SqlProcBody), + new("function", new Regex($@"^\s*CREATE\s+(?:OR\s+(?:REPLACE|ALTER)\s+)?(?:PROCEDURE|PROC|FUNCTION|TRIGGER)\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.SqlProcBody), // SQL Server aggregate definitions are callable search anchors too, but they do not have // a statement body to scan, so they stay on the BodyStyle.None path. // SQL Server の aggregate 定義も検索アンカーとして有用だが、走査すべき statement body は // 持たないため BodyStyle.None のまま扱う。 - new("function", new Regex($@"^\s*CREATE\s+AGGREGATE\b\s+(?{SqlQualifiedIdentifierPattern})\s*\(", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("enum", new Regex($@"^\s*CREATE\s+TYPE\s+(?{SqlQualifiedIdentifierPattern})\s+AS\s+ENUM\b", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("function", new Regex($@"^\s*CREATE\s+AGGREGATE\b\s+(?{SqlQualifiedIdentifierPattern})\s*\(", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("enum", new Regex($@"^\s*CREATE\s+TYPE\s+(?{SqlQualifiedIdentifierPattern})\s+AS\s+ENUM\b", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // Oracle: CREATE [OR REPLACE] TYPE BODY and CREATE [OR REPLACE] PACKAGE [BODY] . // These must precede the bare CREATE TYPE / CREATE PACKAGE rows so the `BODY` keyword is // not absorbed as the object name. // Oracle: CREATE [OR REPLACE] TYPE BODY と CREATE [OR REPLACE] PACKAGE [BODY] 。 // 裸の CREATE TYPE / CREATE PACKAGE 行より前に置き、`BODY` キーワードを name として // 飲み込まないようにする。 - new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?TYPE\s+BODY\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:EDITIONABLE\s+|NONEDITIONABLE\s+)?PACKAGE\s+BODY\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:EDITIONABLE\s+|NONEDITIONABLE\s+)?PACKAGE\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?TYPE\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?TYPE\s+BODY\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:EDITIONABLE\s+|NONEDITIONABLE\s+)?PACKAGE\s+BODY\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:EDITIONABLE\s+|NONEDITIONABLE\s+)?PACKAGE\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?TYPE\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // SQL Server legacy scalar-object definitions still appear in older T-SQL codebases. // The `AS ` tail is part of the definition, not a body to track. // SQL Server の legacy な scalar-object 定義は古い T-SQL コードベースに残っている。 // 末尾の `AS ` は定義の一部であり、追跡すべき body ではない。 - new("class", new Regex($@"^\s*CREATE\s+(?:RULE|DEFAULT)\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("namespace", new Regex($@"^\s*CREATE\s+SCHEMA\s+(?:IF\s+NOT\s+EXISTS\s+)?(?:(?(?!AUTHORIZATION\b){SqlQualifiedIdentifierPattern})|AUTHORIZATION\s+(?{SqlQualifiedIdentifierPattern}))", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("class", new Regex($@"^\s*CREATE\s+(?:SEQUENCE|DOMAIN)\s+(?:IF\s+NOT\s+EXISTS\s+)?(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("import", new Regex($@"^\s*CREATE\s+EXTENSION\s+(?:IF\s+NOT\s+EXISTS\s+)?(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:RULE|DEFAULT)\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("namespace", new Regex($@"^\s*CREATE\s+SCHEMA\s+(?:IF\s+NOT\s+EXISTS\s+)?(?:(?(?!AUTHORIZATION\b){SqlQualifiedIdentifierPattern})|AUTHORIZATION\s+(?{SqlQualifiedIdentifierPattern}))", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:SEQUENCE|DOMAIN)\s+(?:IF\s+NOT\s+EXISTS\s+)?(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("import", new Regex($@"^\s*CREATE\s+EXTENSION\s+(?:IF\s+NOT\s+EXISTS\s+)?(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // T-SQL SYNONYM (also Oracle / DB2) - new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:PUBLIC\s+)?SYNONYM\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:PUBLIC\s+)?SYNONYM\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // Oracle: CREATE [SHARED] [PUBLIC] DATABASE LINK — must precede the bare CREATE DATABASE row // so the `LINK` token is not taken as a name. SHARED and PUBLIC may appear together in that order. // Oracle: CREATE [SHARED] [PUBLIC] DATABASE LINK — 裸の CREATE DATABASE 行より前に置き、 // `LINK` を name として飲み込まないようにする。SHARED と PUBLIC はこの順で 2 語並ぶことがある。 - new("class", new Regex($@"^\s*CREATE\s+(?:SHARED\s+)?(?:PUBLIC\s+)?DATABASE\s+LINK\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:SHARED\s+)?(?:PUBLIC\s+)?DATABASE\s+LINK\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // T-SQL server-level / database-level principals and objects, plus Oracle-only DIRECTORY / CONTEXT / PROFILE. // Include T-SQL SECURITY POLICY so row-level-security policy definitions are discoverable. // T-SQL のサーバ/データベースレベルのプリンシパル・オブジェクトと、Oracle 固有の DIRECTORY / CONTEXT / PROFILE。 // T-SQL の SECURITY POLICY も含め、行レベルセキュリティポリシー定義を検索可能にする。 - new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:DATABASE|LOGIN|USER|ROLE|CERTIFICATE|DIRECTORY|CONTEXT|PROFILE|ASSEMBLY|XML\s+SCHEMA\s+COLLECTION|SECURITY\s+POLICY)\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:DATABASE|LOGIN|USER|ROLE|CERTIFICATE|DIRECTORY|CONTEXT|PROFILE|ASSEMBLY|XML\s+SCHEMA\s+COLLECTION|SECURITY\s+POLICY)\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // T-SQL partitioning and full-text catalogs // T-SQL のパーティション関連と全文検索カタログ - new("function", new Regex($@"^\s*CREATE\s+PARTITION\s+FUNCTION\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("class", new Regex($@"^\s*CREATE\s+PARTITION\s+SCHEME\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("class", new Regex($@"^\s*CREATE\s+FULLTEXT\s+CATALOG\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:UNIQUE\s+)?INDEX\s+(?:IF\s+NOT\s+EXISTS\s+)?(?!ON\b)(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("function", new Regex($@"^\s*CREATE\s+PARTITION\s+FUNCTION\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+PARTITION\s+SCHEME\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+FULLTEXT\s+CATALOG\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("class", new Regex($@"^\s*CREATE\s+(?:OR\s+REPLACE\s+)?(?:UNIQUE\s+)?INDEX\s+(?:IF\s+NOT\s+EXISTS\s+)?(?!ON\b)(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // ALTER covers the same object kinds we create above, so migration scripts remain visible. // Kinds are split to match the CREATE side (procedure-like → function, schema → namespace, // extension → import, everything else → class) so `symbols --kind` / `definition` / `inspect` @@ -1955,11 +1958,11 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult // ALTER PROCEDURE / PROC / FUNCTION / TRIGGER は CREATE と同じ本体形状を持つため // BodyStyle.SqlProcBody を使う。ALTER PARTITION FUNCTION は本体を持たない // （パーティション境界の変更のみ）ため、下の別パターンで BodyStyle.None のままにする。 - new("function", new Regex($@"^\s*ALTER\s+(?:PROCEDURE|PROC|FUNCTION|TRIGGER)\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.SqlProcBody), - new("function", new Regex($@"^\s*ALTER\s+AGGREGATE\b\s+(?{SqlQualifiedIdentifierPattern})\s*\(", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("function", new Regex($@"^\s*ALTER\s+PARTITION\s+FUNCTION\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("namespace", new Regex($@"^\s*ALTER\s+SCHEMA\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("import", new Regex($@"^\s*ALTER\s+EXTENSION\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("function", new Regex($@"^\s*ALTER\s+(?:PROCEDURE|PROC|FUNCTION|TRIGGER)\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.SqlProcBody), + new("function", new Regex($@"^\s*ALTER\s+AGGREGATE\b\s+(?{SqlQualifiedIdentifierPattern})\s*\(", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("function", new Regex($@"^\s*ALTER\s+PARTITION\s+FUNCTION\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("namespace", new Regex($@"^\s*ALTER\s+SCHEMA\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("import", new Regex($@"^\s*ALTER\s+EXTENSION\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // Oracle: ALTER DATABASE LINK — must precede the bare ALTER DATABASE row so `LINK` // is not absorbed as the object name. Real Oracle body compilation is expressed as // `ALTER PACKAGE COMPILE BODY` / `ALTER TYPE COMPILE BODY` and falls through @@ -1968,8 +1971,8 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult // として飲み込まないようにする。Oracle の body コンパイルは実際には // `ALTER PACKAGE COMPILE BODY` / `ALTER TYPE COMPILE BODY` の形で、下の // generic ALTER 行で拾う。`ALTER PACKAGE BODY ` のような構文は Oracle に存在しない。 - new("class", new Regex($@"^\s*ALTER\s+DATABASE\s+LINK\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), - new("class", new Regex($@"^\s*ALTER\s+(?:TABLE|(?:MATERIALIZED\s+)?VIEW|SEQUENCE|SYNONYM|LOGIN|USER|ROLE|DATABASE|CERTIFICATE|INDEX|PACKAGE|TYPE|DOMAIN|DIRECTORY|PROFILE|ASSEMBLY|XML\s+SCHEMA\s+COLLECTION|PARTITION\s+SCHEME|FULLTEXT\s+CATALOG|SECURITY\s+POLICY)\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("class", new Regex($@"^\s*ALTER\s+DATABASE\s+LINK\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), + new("class", new Regex($@"^\s*ALTER\s+(?:TABLE|(?:MATERIALIZED\s+)?VIEW|SEQUENCE|SYNONYM|LOGIN|USER|ROLE|DATABASE|CERTIFICATE|INDEX|PACKAGE|TYPE|DOMAIN|DIRECTORY|PROFILE|ASSEMBLY|XML\s+SCHEMA\s+COLLECTION|PARTITION\s+SCHEME|FULLTEXT\s+CATALOG|SECURITY\s+POLICY)\b\s+(?{SqlQualifiedIdentifierPattern})", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), ], ["terraform"] = [ @@ -2045,16 +2048,16 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult ["powershell"] = [ // DSC configuration / workflow declarations / DSC 構成・workflow 宣言 - new("function", new Regex(@"^\s*configuration\s+(?[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.Brace), - new("function", new Regex(@"^\s*workflow\s+(?[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.Brace), + new("function", new Regex(@"^\s*configuration\s+(?[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.Brace), + new("function", new Regex(@"^\s*workflow\s+(?[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.Brace), // Function/filter declarations with optional scope prefixes / scope プレフィックス付き関数・フィルタ宣言 - new("function", new Regex(@"^\s*(?:function|filter)\s+(?:(?:script|global|local|private):)?(?[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.Brace), + new("function", new Regex(@"^\s*(?:function|filter)\s+(?:(?:script|global|local|private):)?(?[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.Brace), // PowerShell class members / PowerShell クラスメンバー // Return-typed methods and modifiers such as `static` / `hidden` / `static hidden` // stay on the function path. // 戻り値付き method と `static` / `hidden` / `static hidden` のような修飾子は // function パスで扱う。 - new("function", new Regex(@"^\s*(?:(?:static|hidden)\s+)*(?:\[[^\]]+\]\s+)+(?[\w-]+)\s*\(", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.Brace), + new("function", new Regex(@"^\s*(?:(?:static|hidden)\s+)*(?:\[[^\]]+\]\s+)+(?[\w-]+)\s*\(", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.Brace), // Constructors are bare class-name declarations inside a class body, so the // PascalCase gate keeps most cmdlet-style calls out while still catching the // canonical PS5+ shape. @@ -2062,17 +2065,17 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult // PascalCase の条件で cmdlet 風の呼び出しを大半弾きつつ、PS5+ の標準形を拾う。 new("function", new Regex(@"^\s*(?[A-Z]\w*)\s*\(", RegexOptions.Compiled), BodyStyle.Brace), // Alias definitions / エイリアス定義 - new("alias", new Regex(@"^\s*(?:Set-Alias|New-Alias)\s+(?:-Name\s+)?(?[\w-]+)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("alias", new Regex(@"^\s*(?:Set-Alias|New-Alias)\s+(?:-Name\s+)?(?[\w-]+)\b", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // Attributes and typed properties / 属性付きプロパティと型付きプロパティ - new("property", new Regex(@"^\s*(?:(?:static|hidden)\s+)*(?:\[[^\]]+\]\s*)+\$(?\w+)\s*(?:=|$)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("property", new Regex(@"^\s*(?:(?:static|hidden)\s+)*(?:\[[^\]]+\]\s*)+\$(?\w+)\s*(?:=|$)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // Class (PowerShell 5+) / クラス (PowerShell 5+) - new("class", new Regex(@"^\s*class\s+(?\w+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.Brace), + new("class", new Regex(@"^\s*class\s+(?\w+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.Brace), // Enum (PowerShell 5+) / enum (PowerShell 5+) - new("enum", new Regex(@"^\s*enum\s+(?\w+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.Brace), + new("enum", new Regex(@"^\s*enum\s+(?\w+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.Brace), // Enum values / enum 値 new("enum", new Regex(@"^\s{2,}(?[\w-]+)\s*(?:=\s*[^#\r\n]+)?\s*$", RegexOptions.Compiled), BodyStyle.None), // Import-Module / using module / using namespace / using assembly / モジュールインポート - new("import", new Regex(@"^\s*(?:Import-Module|using\s+(?:module|namespace|assembly))\s+(?\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("import", new Regex(@"^\s*(?:Import-Module|using\s+(?:module|namespace|assembly))\s+(?\S+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), ], ["batch"] = [ @@ -2089,7 +2092,7 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult // `:EOF` は `goto :EOF` / `call :EOF` 用の予約ターゲットであってユーザー定義ラベルではないため除外するが、 // 除外するのは名前全体が `eof` のときだけ。`:eof2` / `:eofish` / `:end-of-file` / `:eof.x` のように // 単に `eof` で始まるだけのラベルは通す必要があるため、`\b` ではなく名前終端文字を見る negative lookahead を使う。 - new("function", new Regex(@"^\s*:(?!eof(?![\w.-]))(?[\w.\-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("function", new Regex(@"^\s*:(?!eof(?![\w.-]))(?[\w.\-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), // Variable assignment — set VAR=value, set /a VAR=expr, set /p VAR=prompt, set "VAR=value". // Also handles `@set VAR=...` (echo suppression prefix), `set /a VAR+=1` (compound // assignment operators), `if ... set VAR=...` (inline assignment inside a one-line @@ -2111,7 +2114,7 @@ private enum JavaScriptTypeScriptFunctionHeaderConsumeResult // `rem` / `@rem` / `::` コメント行にもこれらの境界トークンが入りうる // (`REM & set FAKE=1` 等) ため、この正規表現が走る前に `IsBatchCommentLine` で // 行ごと早期スキップしている — 境界 alternation だけではコメント本文を弾ききれない。 - new("property", new Regex(@"(?:(?:^|&|\()\s*|(?:\belse|\bdo)\s+)(?:@\s*)?(?:if\s+.+?\s+)?set\s+(?:/[aApP]\s+)?""?(?[A-Za-z_][\w]*)\s*(?:[+\-*/%&^|]|<<|>>)?=", RegexOptions.Compiled | RegexOptions.IgnoreCase), BodyStyle.None), + new("property", new Regex(@"(?:(?:^|&|\()\s*|(?:\belse|\bdo)\s+)(?:@\s*)?(?:if\s+.+?\s+)?set\s+(?:/[aApP]\s+)?""?(?[A-Za-z_][\w]*)\s*(?:[+\-*/%&^|]|<<|>>)?=", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant), BodyStyle.None), ], // Assembly uses a dedicated line scanner because label body ranges extend until // the next label/section rather than a brace or indentation boundary. @@ -2359,6 +2362,9 @@ public static List Extract(long fileId, string? lang, string conte var cssScannerLines = lang == "css" ? MaskCssScannerLines(lines) : null; + var shellScannerLines = lang == "shell" + ? MaskShellHeredocLines(lines) + : null; int[]?[] csharpMatchColumnToRaw = null!; var csharpMatchLines = lang == "csharp" ? BuildCSharpMatchLines(lines, out csharpMatchColumnToRaw) @@ -2433,6 +2439,7 @@ public static List Extract(long fileId, string? lang, string conte var structuralLine = structuralLines[i]; var cssScannerLine = cssScannerLines?[i]; + var shellScannerLine = shellScannerLines?[i]; var matchLine = structuralLine; if (lang == "css" && cssScannerLine != null) { @@ -2443,6 +2450,10 @@ public static List Extract(long fileId, string? lang, string conte // 保持する。brace/depth 判定だけ別の scanner line を使う。 matchLine = line; } + else if (lang == "shell" && shellScannerLine != null) + { + matchLine = shellScannerLine; + } else if (lang == "csharp") { matchLine = csharpMatchLines![i]; @@ -2930,6 +2941,8 @@ public static List Extract(long fileId, string? lang, string conte var rangeLines = lang == "css" && cssScannerLines != null ? cssScannerLines + : lang == "shell" && shellScannerLines != null + ? shellScannerLines : structuralLines; var scalaBracelessClassEndLine = lang == "scala" && pattern.Kind == "class" ? TryFindScalaBracelessClassEndLine(lines, i, absoluteStartColumn) @@ -5774,6 +5787,7 @@ private static (int EndLine, int? BodyStartLine, int? BodyEndLine) ResolveRange( BodyStyle.Brace when lang is "javascript" or "typescript" => FindJavaScriptBraceRange(lines, startIndex, lang, startColumn), BodyStyle.Brace when lang == "csharp" => FindCSharpBraceRange(lines, startIndex, startColumn), BodyStyle.Brace when lang == "java" => FindJavaBraceRange(lines, startIndex, startColumn), + BodyStyle.Brace when lang == "shell" => FindShellFunctionRange(lines, startIndex, startColumn), BodyStyle.Brace => FindBraceRange(lines, startIndex, startColumn, lang), BodyStyle.Indent => FindIndentRange(lines, startIndex), BodyStyle.RubyEnd => FindRubyRange(lines, startIndex), @@ -5787,6 +5801,268 @@ private static (int EndLine, int? BodyStartLine, int? BodyEndLine) ResolveRange( }; } + private readonly record struct ShellHeredocTerminator(string Delimiter, bool StripLeadingTabs); + + private static string[] MaskShellHeredocLines(string[] lines) + { + var maskedLines = (string[])lines.Clone(); + var pendingTerminators = new Queue(); + ShellHeredocTerminator? activeTerminator = null; + + for (var i = 0; i < lines.Length; i++) + { + var line = lines[i]; + if (activeTerminator is { } terminator) + { + maskedLines[i] = string.Empty; + var terminatorLine = terminator.StripLeadingTabs + ? line.TrimStart('\t') + : line; + terminatorLine = terminatorLine.TrimEnd('\r'); + if (string.Equals(terminatorLine, terminator.Delimiter, StringComparison.Ordinal)) + { + activeTerminator = pendingTerminators.Count > 0 + ? pendingTerminators.Dequeue() + : null; + } + + continue; + } + + foreach (var heredocTerminator in EnumerateShellHeredocTerminators(line)) + pendingTerminators.Enqueue(heredocTerminator); + + if (pendingTerminators.Count > 0) + activeTerminator = pendingTerminators.Dequeue(); + } + + return maskedLines; + } + + private static IEnumerable EnumerateShellHeredocTerminators(string line) + { + var ignored = BuildShellIgnoredCharacterMask(line); + foreach (Match match in ShellHeredocRedirectRegex.Matches(line)) + { + if (match.Index < ignored.Length && ignored[match.Index]) + continue; + + var delimiter = match.Groups["dq"].Success + ? match.Groups["dq"].Value + : match.Groups["sq"].Success + ? match.Groups["sq"].Value + : match.Groups["bare"].Value; + if (delimiter.Length == 0) + continue; + + var stripLeadingTabs = match.Index + 2 < line.Length && line[match.Index + 2] == '-'; + yield return new ShellHeredocTerminator(delimiter, stripLeadingTabs); + } + } + + private static bool[] BuildShellIgnoredCharacterMask(string line) + { + var ignored = new bool[line.Length]; + var inSingleQuote = false; + var inDoubleQuote = false; + + for (var i = 0; i < line.Length; i++) + { + var c = line[i]; + if (inSingleQuote) + { + ignored[i] = true; + if (c == '\'') + inSingleQuote = false; + continue; + } + + if (inDoubleQuote) + { + ignored[i] = true; + if (c == '\\' && i + 1 < line.Length) + { + ignored[++i] = true; + continue; + } + + if (c == '"') + inDoubleQuote = false; + continue; + } + + if (c == '\\' && i + 1 < line.Length) + { + i++; + continue; + } + + if (c == '\'') + { + ignored[i] = true; + inSingleQuote = true; + continue; + } + + if (c == '"') + { + ignored[i] = true; + inDoubleQuote = true; + continue; + } + + if (c == '#' && IsShellCommentStart(line, i)) + { + Array.Fill(ignored, true, i, line.Length - i); + break; + } + } + + return ignored; + } + + private static (int EndLine, int? BodyStartLine, int? BodyEndLine) FindShellFunctionRange(string[] lines, int startIndex, int startColumn) + { + var depth = 0; + var opened = false; + int? bodyStartLine = null; + var inSingleQuote = false; + var inDoubleQuote = false; + + for (var i = startIndex; i < lines.Length; i++) + { + var scanLine = i == startIndex && startColumn > 0 && startColumn < lines[i].Length + ? lines[i][startColumn..] + : i == startIndex && startColumn >= lines[i].Length + ? string.Empty + : lines[i]; + + var closeColumn = ScanShellBraceLine( + scanLine, + ref depth, + ref opened, + ref bodyStartLine, + i + 1, + ref inSingleQuote, + ref inDoubleQuote); + if (closeColumn >= 0) + return (i + 1, bodyStartLine, i + 1); + } + + if (!opened) + return (startIndex + 1, null, null); + + var boundedEndLine = bodyStartLine.HasValue + ? Math.Max(startIndex + 1, bodyStartLine.Value) + : startIndex + 1; + return (boundedEndLine, bodyStartLine, boundedEndLine); + } + + private static int FindShellSameLineBraceEndColumn(string line, int startColumn) + { + var depth = 0; + var opened = false; + int? bodyStartLine = null; + var inSingleQuote = false; + var inDoubleQuote = false; + return ScanShellBraceLine( + startColumn > 0 && startColumn < line.Length + ? line[startColumn..] + : startColumn >= line.Length + ? string.Empty + : line, + ref depth, + ref opened, + ref bodyStartLine, + 1, + ref inSingleQuote, + ref inDoubleQuote) is var relativeCloseColumn && relativeCloseColumn >= 0 + ? startColumn + relativeCloseColumn + : -1; + } + + private static int ScanShellBraceLine( + string line, + ref int depth, + ref bool opened, + ref int? bodyStartLine, + int currentLine, + ref bool inSingleQuote, + ref bool inDoubleQuote) + { + for (var i = 0; i < line.Length; i++) + { + var c = line[i]; + if (inSingleQuote) + { + if (c == '\'') + inSingleQuote = false; + continue; + } + + if (inDoubleQuote) + { + if (c == '\\' && i + 1 < line.Length) + { + i++; + continue; + } + + if (c == '"') + inDoubleQuote = false; + continue; + } + + if (c == '\\' && i + 1 < line.Length) + { + i++; + continue; + } + + if (c == '\'') + { + inSingleQuote = true; + continue; + } + + if (c == '"') + { + inDoubleQuote = true; + continue; + } + + if (c == '#' && IsShellCommentStart(line, i)) + break; + + if (c == '{') + { + depth++; + if (!opened) + { + opened = true; + bodyStartLine = currentLine; + } + } + else if (c == '}' && opened) + { + depth--; + if (depth <= 0) + return i; + } + } + + return -1; + } + + private static bool IsShellCommentStart(string line, int index) + { + if (index == 0) + return true; + + var previous = line[index - 1]; + return char.IsWhiteSpace(previous) || previous is ';' or '|' or '&' or '(' or '{'; + } + private static (int EndLine, int? BodyStartLine, int? BodyEndLine) FindBraceRange(string[] lines, int startIndex, int startColumn = 0, string? lang = null) { diff --git a/src/CodeIndex/Mcp/McpErrorEnvelope.cs b/src/CodeIndex/Mcp/McpErrorEnvelope.cs index 5366dcd657..6937ba98a7 100644 --- a/src/CodeIndex/Mcp/McpErrorEnvelope.cs +++ b/src/CodeIndex/Mcp/McpErrorEnvelope.cs @@ -48,6 +48,7 @@ internal static class McpErrorEnvelope public const string CategoryIndexMissing = "index_missing"; public const string CategoryIndexStale = "index_stale"; public const string CategoryIndexCorrupted = "index_corrupted"; + public const string CategoryRegexTimeout = "regex_timeout"; public const string CategoryInternalError = "internal_error"; /// diff --git a/src/CodeIndex/Mcp/McpToolHandlers.cs b/src/CodeIndex/Mcp/McpToolHandlers.cs index db0364d979..ba99f5a2d8 100644 --- a/src/CodeIndex/Mcp/McpToolHandlers.cs +++ b/src/CodeIndex/Mcp/McpToolHandlers.cs @@ -3635,7 +3635,21 @@ private JsonNode ExecuteFindInFile(JsonNode? id, JsonNode? args) { results = reader.FindInFiles(query, limit, lang, pathPatterns, excludePaths, excludeTests, before, after, exact, maxLineWidth, focusLine, focusColumn, regex).Results; } - catch (Exception ex) when (regex && (ex is ArgumentException || ex is RegexMatchTimeoutException)) + catch (RegexMatchTimeoutException ex) when (regex) + { + return CreateToolErrorResponse( + id, + $"regular expression timed out after {QueryCommandRunner.FormatRegexMatchTimeout(ex.MatchTimeout)} while scanning indexed file contents.", + category: McpErrorEnvelope.CategoryRegexTimeout, + suggestion: "Simplify the pattern, narrow the scan with path/lang filters, or disable regex mode for literal text.", + retrySafe: true, + extraData: new JsonObject + { + ["error_code"] = CommandErrorCodes.RegexMatchTimeout, + ["timeout_ms"] = ex.MatchTimeout.TotalMilliseconds, + }); + } + catch (ArgumentException) when (regex) { return CreateToolErrorResponse(id, "invalid regular expression. Check regex syntax and retry."); } diff --git a/tests/CodeIndex.Tests/CommandErrorCodesTests.cs b/tests/CodeIndex.Tests/CommandErrorCodesTests.cs index 306eab50fb..5ed68c8694 100644 --- a/tests/CodeIndex.Tests/CommandErrorCodesTests.cs +++ b/tests/CodeIndex.Tests/CommandErrorCodesTests.cs @@ -106,6 +106,38 @@ public void Search_MissingDb_StderrIncludesDbNotFoundCode() Assert.Contains("[E001_DB_NOT_FOUND]", stderr); } + [Fact] + public void Find_RegexTimeout_StderrIncludesBracketedCode_Issue3559() + { + var timeout = new System.Text.RegularExpressions.RegexMatchTimeoutException( + "aaaaaaaaaaaaaaaa!", + "^(a+)+$", + TimeSpan.FromMilliseconds(25)); + + var (exitCode, _, stderr) = CaptureStreams(() => + QueryCommandRunner.WriteFindRegexTimeoutError(timeout, _jsonOptions, json: false)); + + Assert.Equal(CommandExitCodes.RuntimeError, exitCode); + Assert.Contains("[E014_REGEX_MATCH_TIMEOUT]", stderr); + Assert.Contains("regular expression timed out", stderr, StringComparison.Ordinal); + } + + [Fact] + public void Find_RegexTimeout_JsonIncludesRegexTimeoutCode_Issue3559() + { + var timeout = new System.Text.RegularExpressions.RegexMatchTimeoutException( + "aaaaaaaaaaaaaaaa!", + "^(a+)+$", + TimeSpan.FromMilliseconds(25)); + + var (exitCode, json) = RunFindRegexTimeoutCapturingJson(timeout); + + Assert.Equal(CommandExitCodes.RuntimeError, exitCode); + Assert.Equal("error", json.GetProperty("status").GetString()); + Assert.Equal("E014_REGEX_MATCH_TIMEOUT", json.GetProperty("error_code").GetString()); + Assert.Equal("regex_timeout", json.GetProperty("category").GetString()); + } + [Fact] public void Symbols_InvalidKind_ReturnsInvalidArgumentExitCode() { @@ -177,6 +209,14 @@ public void KindFilteredCommands_InvalidKind_ReturnInvalidArgumentExitCode(strin return (exitCode, capture.Out!.ToString()!, capture.Error!.ToString()!); } + private (int ExitCode, JsonElement Json) RunFindRegexTimeoutCapturingJson(System.Text.RegularExpressions.RegexMatchTimeoutException timeout) + { + using var capture = ConsoleCapture.Start(captureOut: true); + var exitCode = QueryCommandRunner.WriteFindRegexTimeoutError(timeout, _jsonOptions, json: true); + using var document = JsonDocument.Parse(capture.Out!.ToString()!); + return (exitCode, document.RootElement.Clone()); + } + private static (int ExitCode, string StdOut, string StdErr) CaptureStreams(Func run) { using var capture = ConsoleCapture.Start(captureOut: true, captureError: true); diff --git a/tests/CodeIndex.Tests/McpServerTests.cs b/tests/CodeIndex.Tests/McpServerTests.cs index 9c9defb565..1d7a48708f 100644 --- a/tests/CodeIndex.Tests/McpServerTests.cs +++ b/tests/CodeIndex.Tests/McpServerTests.cs @@ -15132,6 +15132,27 @@ public void BuildData_ExtraDataCannotShadowCanonicalKeys() Assert.Equal("search", data["tool"]!.GetValue()); } + [Fact] + public void BuildData_RegexTimeoutCarriesStructuredPayload_Issue3559() + { + var extra = new JsonObject + { + ["error_code"] = CommandErrorCodes.RegexMatchTimeout, + ["timeout_ms"] = 500.0, + }; + + var data = McpErrorEnvelope.BuildData( + McpErrorEnvelope.CategoryRegexTimeout, + "Simplify the pattern.", + retrySafe: true, + extra); + + Assert.Equal(McpErrorEnvelope.CategoryRegexTimeout, data["category"]!.GetValue()); + Assert.True(data["retry_safe"]!.GetValue()); + Assert.Equal(CommandErrorCodes.RegexMatchTimeout, data["error_code"]!.GetValue()); + Assert.Equal(500.0, data["timeout_ms"]!.GetValue()); + } + private static void AssertJsonNullId(JsonNode node) { var obj = Assert.IsType(node); diff --git a/tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs b/tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs index f688bf414f..9487c6065f 100644 --- a/tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs +++ b/tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs @@ -1,3 +1,5 @@ +using System.Reflection; +using System.Text.RegularExpressions; using System.Text.Json; using CodeIndex.Cli; using CodeIndex.Database; @@ -5132,6 +5134,48 @@ public void RunFind_RegexMatchesAnchors() } } + [Fact] + public void RunFind_RegexMatcherUsesSharedTimeoutAndCultureInvariant_Issue3559() + { + var method = typeof(DbReader).GetMethod("CreateFindRegexMatcher", BindingFlags.NonPublic | BindingFlags.Static); + Assert.NotNull(method); + + var defaultTimeout = typeof(CodeIndex.Indexer.SymbolExtractor).Assembly + .GetType("CodeIndex.Indexer.BoundedRegex")! + .GetField("DefaultMatchTimeout", BindingFlags.NonPublic | BindingFlags.Static)! + .GetValue(null); + + var insensitive = Assert.IsType(method.Invoke(null, ["needle", false])); + Assert.Equal(defaultTimeout, insensitive.MatchTimeout); + Assert.True((insensitive.Options & RegexOptions.IgnoreCase) != 0); + Assert.True((insensitive.Options & RegexOptions.CultureInvariant) != 0); + + var exact = Assert.IsType(method.Invoke(null, ["needle", true])); + Assert.Equal(defaultTimeout, exact.MatchTimeout); + Assert.False((exact.Options & RegexOptions.IgnoreCase) != 0); + Assert.True((exact.Options & RegexOptions.CultureInvariant) != 0); + } + + [Fact] + public void RunFind_RegexTimeoutWritesRuntimeErrorJsonMetadata_Issue3559() + { + var timeout = new RegexMatchTimeoutException("aaaaaaaaaaaaaaaa!", "^(a+)+$", TimeSpan.FromMilliseconds(25)); + + var (exitCode, stdout, stderr) = CaptureConsole(() => + QueryCommandRunner.WriteFindRegexTimeoutError(timeout, _jsonOptions, json: true)); + + Assert.Equal(CommandExitCodes.RuntimeError, exitCode); + Assert.Equal(string.Empty, stderr); + using var document = ParseJsonOutput(stdout); + var json = document.RootElement; + Assert.Equal("error", json.GetProperty("status").GetString()); + Assert.Equal("E014_REGEX_MATCH_TIMEOUT", json.GetProperty("error_code").GetString()); + Assert.Equal("regex_timeout", json.GetProperty("category").GetString()); + Assert.Contains("timed out", json.GetProperty("message").GetString()); + Assert.DoesNotContain("invalid regular expression", json.GetProperty("message").GetString()); + Assert.Contains("--regex", json.GetProperty("hint").GetString()); + } + [Fact] public void RunFind_CountOnlyRegexAndFocusUseSameMatchingSemantics() { diff --git a/tests/CodeIndex.Tests/ReferenceExtractorTests.cs b/tests/CodeIndex.Tests/ReferenceExtractorTests.cs index d79a6e052b..711b578a0a 100644 --- a/tests/CodeIndex.Tests/ReferenceExtractorTests.cs +++ b/tests/CodeIndex.Tests/ReferenceExtractorTests.cs @@ -44,6 +44,47 @@ public void BuiltInReferenceRegexes_HaveBoundedMatchTimeouts() "Built-in reference regexes must have explicit match timeouts: " + string.Join(", ", infiniteTimeouts)); } + [Fact] + public void BuiltInReferenceRegexes_UseDefaultBacktrackingPolicy_Issue3479() + { + var regexes = EnumerateStaticRegexValues( + typeof(ReferenceExtractor).Assembly.GetTypes().Where(IsReferenceRegexOwnerType)) + .ToList(); + + Assert.NotEmpty(regexes); + + var backtrackingRegexesWithCustomTimeouts = regexes + .Where(item => (item.Regex.Options & RegexOptions.NonBacktracking) == 0) + .Where(item => item.Regex.MatchTimeout != BoundedRegex.DefaultMatchTimeout) + .Select(item => item.Path) + .ToList(); + + Assert.True( + backtrackingRegexesWithCustomTimeouts.Count == 0, + "Built-in reference regexes that use backtracking must use BoundedRegex.DefaultMatchTimeout: " + + string.Join(", ", backtrackingRegexesWithCustomTimeouts)); + } + + [Fact] + public void BuiltInReferenceRegexes_WithIgnoreCaseUseCultureInvariant_Issue3516() + { + var regexes = EnumerateStaticRegexValues( + typeof(ReferenceExtractor).Assembly.GetTypes().Where(IsReferenceRegexOwnerType)) + .ToList(); + + Assert.NotEmpty(regexes); + + var cultureSensitive = regexes + .Where(item => (item.Regex.Options & RegexOptions.IgnoreCase) != 0) + .Where(item => (item.Regex.Options & RegexOptions.CultureInvariant) == 0) + .Select(item => item.Path) + .ToList(); + + Assert.True( + cultureSensitive.Count == 0, + "Built-in reference regexes with IgnoreCase must use CultureInvariant: " + string.Join(", ", cultureSensitive)); + } + [Fact] public void Extract_BuiltInReferenceRegexes_AdversarialLongLinesDoNotThrow() { diff --git a/tests/CodeIndex.Tests/SymbolExtractorTests.cs b/tests/CodeIndex.Tests/SymbolExtractorTests.cs index 72467a3900..09196c3469 100644 --- a/tests/CodeIndex.Tests/SymbolExtractorTests.cs +++ b/tests/CodeIndex.Tests/SymbolExtractorTests.cs @@ -46,6 +46,47 @@ public void BuiltInSymbolRegexes_HaveBoundedMatchTimeouts() "Built-in symbol regexes must have explicit match timeouts: " + string.Join(", ", infiniteTimeouts)); } + [Fact] + public void BuiltInSymbolRegexes_UseDefaultBacktrackingPolicy_Issue3479() + { + var regexes = EnumerateStaticRegexValues( + typeof(SymbolExtractor).Assembly.GetTypes().Where(IsSymbolRegexOwnerType)) + .ToList(); + + Assert.NotEmpty(regexes); + + var backtrackingRegexesWithCustomTimeouts = regexes + .Where(item => (item.Regex.Options & RegexOptions.NonBacktracking) == 0) + .Where(item => item.Regex.MatchTimeout != BoundedRegex.DefaultMatchTimeout) + .Select(item => item.Path) + .ToList(); + + Assert.True( + backtrackingRegexesWithCustomTimeouts.Count == 0, + "Built-in symbol regexes that use backtracking must use BoundedRegex.DefaultMatchTimeout: " + + string.Join(", ", backtrackingRegexesWithCustomTimeouts)); + } + + [Fact] + public void BuiltInSymbolRegexes_WithIgnoreCaseUseCultureInvariant_Issue3516() + { + var regexes = EnumerateStaticRegexValues( + typeof(SymbolExtractor).Assembly.GetTypes().Where(IsSymbolRegexOwnerType)) + .ToList(); + + Assert.NotEmpty(regexes); + + var cultureSensitive = regexes + .Where(item => (item.Regex.Options & RegexOptions.IgnoreCase) != 0) + .Where(item => (item.Regex.Options & RegexOptions.CultureInvariant) == 0) + .Select(item => item.Path) + .ToList(); + + Assert.True( + cultureSensitive.Count == 0, + "Built-in symbol regexes with IgnoreCase must use CultureInvariant: " + string.Join(", ", cultureSensitive)); + } + [Fact] public void Extract_CSharp_BraceBodiedFunctionSignatureStopsAtDeclarationHeader() { @@ -6693,6 +6734,83 @@ function setup() { Assert.Contains(symbols, s => s.Kind == "alias" && s.Name == "G"); } + [Fact] + public void Extract_Shell_IgnoresHeredocBodies_Issue3510() + { + var content = """ + setup() { + python3 <<'PY' + def main(): + pass + main() { + echo not-shell + } + PY + cat <<-EOF + function fake_heredoc_function() { + } + EOF + echo done + } + real_after() { echo done; } + """; + var symbols = SymbolExtractor.Extract(1, "shell", content); + + Assert.Contains(symbols, s => s.Kind == "function" && s.Name == "setup"); + Assert.Contains(symbols, s => s.Kind == "function" && s.Name == "real_after"); + Assert.DoesNotContain(symbols, s => s.Kind == "function" && s.Name == "main"); + Assert.DoesNotContain(symbols, s => s.Kind == "function" && s.Name == "fake_heredoc_function"); + } + + [Fact] + public void Extract_Shell_BoundsFunctionRanges_Issue3510() + { + var content = """ + info() { printf '%s\n' "$*"; } + warn() { printf '%s\n' "$*"; } + extract_release_tag_name() { + local tag + tag="${1#refs/tags/}" + printf '%s\n' "$tag" + } + verify_payload_manifest() { + cat <<'PY' + main() { + pass + } + PY + echo done + } + after() { echo done; } + """; + var symbols = SymbolExtractor.Extract(1, "shell", content); + + var info = Assert.Single(symbols, s => s.Kind == "function" && s.Name == "info"); + Assert.Equal(1, info.EndLine); + Assert.Equal(1, info.BodyStartLine); + Assert.Equal(1, info.BodyEndLine); + + var warn = Assert.Single(symbols, s => s.Kind == "function" && s.Name == "warn"); + Assert.Equal(2, warn.EndLine); + Assert.Equal(2, warn.BodyStartLine); + Assert.Equal(2, warn.BodyEndLine); + + var extractTag = Assert.Single(symbols, s => s.Kind == "function" && s.Name == "extract_release_tag_name"); + Assert.Equal(7, extractTag.EndLine); + Assert.Equal(3, extractTag.BodyStartLine); + Assert.Equal(7, extractTag.BodyEndLine); + + var verifyManifest = Assert.Single(symbols, s => s.Kind == "function" && s.Name == "verify_payload_manifest"); + Assert.Equal(15, verifyManifest.EndLine); + Assert.Equal(8, verifyManifest.BodyStartLine); + Assert.Equal(15, verifyManifest.BodyEndLine); + + var after = Assert.Single(symbols, s => s.Kind == "function" && s.Name == "after"); + Assert.Equal(16, after.EndLine); + Assert.Equal(16, after.BodyStartLine); + Assert.Equal(16, after.BodyEndLine); + } + [Fact] public void Extract_ShellLargeAliasSet_CompletesWithinPracticalBudget() { @@ -18550,6 +18668,7 @@ public void Extract_TypeScript_MalformedTsconfigSkipsPathAliasesWithWarning() Assert.Contains(symbols, s => s.Kind == "import" && s.Name == "@/components/Button"); Assert.DoesNotContain(symbols, s => s.Kind == "import" && s.Name == "src/components/Button.tsx"); Assert.Contains("Skipped TypeScript path alias config", stderr, StringComparison.Ordinal); + Assert.Contains("tsconfig_json_invalid", stderr, StringComparison.Ordinal); Assert.Contains("could not be parsed as JSON", stderr, StringComparison.Ordinal); } finally @@ -18558,6 +18677,32 @@ public void Extract_TypeScript_MalformedTsconfigSkipsPathAliasesWithWarning() } } + [Fact] + public void Extract_TypeScript_UnreadableTsconfigSkipsPathAliasesWithReadFailedWarning_Issue3438() + { + var projectRoot = TestProjectHelper.CreateTempProject("tsconfig_alias_unreadable_symbols"); + try + { + Directory.CreateDirectory(Path.Combine(projectRoot, "tsconfig.json")); + WriteFile(projectRoot, "src/components/Button.tsx", "export const Button = 1;\n"); + var sourcePath = WriteFile(projectRoot, "src/main.ts", "import { Button } from \"@/components/Button\";\n"); + + List symbols = []; + var stderr = ConsoleCapture.CaptureError(() => + symbols = SymbolExtractor.Extract(1, "typescript", File.ReadAllText(sourcePath), sourcePath)); + + Assert.Contains(symbols, s => s.Kind == "import" && s.Name == "@/components/Button"); + Assert.DoesNotContain(symbols, s => s.Kind == "import" && s.Name == "src/components/Button.tsx"); + Assert.Contains("Skipped TypeScript path alias config", stderr, StringComparison.Ordinal); + Assert.Contains("tsconfig_read_failed", stderr, StringComparison.Ordinal); + Assert.Contains("could not be read", stderr, StringComparison.Ordinal); + } + finally + { + TestProjectHelper.DeleteDirectory(projectRoot); + } + } + [Fact] public void Extract_TypeScript_DeepTsconfigJsonSkipsPathAliasesWithWarning() { @@ -18579,6 +18724,7 @@ public void Extract_TypeScript_DeepTsconfigJsonSkipsPathAliasesWithWarning() Assert.Contains(symbols, s => s.Kind == "import" && s.Name == "@/components/Button"); Assert.DoesNotContain(symbols, s => s.Kind == "import" && s.Name == "src/components/Button.tsx"); Assert.Contains("Skipped TypeScript path alias config", stderr, StringComparison.Ordinal); + Assert.Contains("tsconfig_json_invalid", stderr, StringComparison.Ordinal); Assert.Contains("32-level depth limit", stderr, StringComparison.Ordinal); } finally @@ -18615,6 +18761,7 @@ public void Extract_TypeScript_ExcessiveTsconfigExtendsDepthSkipsInheritedPathAl Assert.Contains(symbols, s => s.Kind == "import" && s.Name == "~lib/math"); Assert.DoesNotContain(symbols, s => s.Kind == "import" && s.Name == "lib/math.ts"); + Assert.Contains("path_alias_depth_limit", stderr, StringComparison.Ordinal); Assert.Contains("extends depth", stderr, StringComparison.Ordinal); } finally