Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1223,7 +1223,7 @@ Process exit codes are coarse (`0` success including valid zero-row queries, `1`

- **Cross-compiled linux-arm64 without runtime smoke test** — The `release.yml` workflow cross-compiles `linux-arm64` on an x64 runner (`dotnet publish -r linux-arm64 --self-contained`). Tests are skipped because the runner cannot execute ARM binaries natively. Ideally, a QEMU-based smoke test (`cdidx --version`) would run before publishing, but GitHub Actions free-tier runners do not include QEMU or ARM runners. Adding a QEMU setup step is possible but increases CI complexity and wall-clock time for every release. .NET's cross-compilation is an officially supported and widely used feature, so the risk of a broken artifact is low in practice. If ARM-specific failures are reported in the future, adding `docker run --platform linux/arm64` with QEMU should be the first mitigation step.
- **CLI / MCP only — no public library API (#1557)** — The `cdidx` assembly is shipped as `OutputType=Exe` with `PackAsTool=true` and is published as a .NET global tool, not as a referenceable library. The supported, versioned surfaces are the `cdidx` CLI (including its `--json` output) and the `cdidx mcp` JSON-RPC server. `public` types on the assembly (for example `CodeIndex.Database.DbReader` and DTOs in `CodeIndex.Models` / `CodeIndex.Database`) exist to satisfy CLI / MCP composition and the `CodeIndex.Tests` `InternalsVisibleTo` boundary — they are implementation details that may change, move, or become `internal` without a deprecation cycle. Embedders are expected to depend on the CLI / MCP / JSON surfaces, not on the assembly. See [INTEGRATION_POLICY.md — API Surface and Library Use](INTEGRATION_POLICY.md#api-surface-and-library-use). If a real library API is ever justified, it will be carved out as a separate package with its own interface and versioning contract rather than being implied by whatever happens to be `public` on this assembly.
- **Extractor plugins (#1937)** — `CodeIndex.Indexer.Extensibility.ISymbolExtractor` and `IReferenceExtractor` are the only supported assembly-extension surface. `cdidx` discovers trusted plugin DLLs in workspace `.cdidx/plugins/` and user `~/.cdidx/plugins/`. A plugin assembly must declare `[assembly: CdidxPlugin(minApiVersion: 1, maxApiVersion: 1)]` and expose a public parameterless type implementing one or both interfaces. Set `FileExtensions` when the plugin owns new file extensions so `FileIndexer` can route those files to the plugin language. Plugins run inside the `cdidx` process and are not sandboxed; install only trusted local DLLs. This narrow contract lets teams add DSL-specific symbols/references without forking CodeIndex, but it is not a general library/SDK embedding API.
- **Extractor plugins (#1937)** — `CodeIndex.Indexer.Extensibility.ISymbolExtractor` and `IReferenceExtractor` are the only supported assembly-extension surface. `cdidx` discovers trusted plugin DLLs in the user-owned `~/.cdidx/plugins/` directory by default. Workspace `.cdidx/plugins/` DLL discovery is fail-closed unless the process sets `CDIDX_TRUST_WORKSPACE_PLUGINS=1` (also accepts `true`, `yes`, or `on`), because loading a workspace DLL executes checkout-provided code inside the `cdidx` process. A plugin assembly must declare `[assembly: CdidxPlugin(minApiVersion: 1, maxApiVersion: 1)]` and expose a public parameterless type implementing one or both interfaces. Set `FileExtensions` when the plugin owns new file extensions so `FileIndexer` can route those files to the plugin language. Plugins run inside the `cdidx` process and are not sandboxed; install only trusted local DLLs. This narrow contract lets teams add DSL-specific symbols/references without forking CodeIndex, but it is not a general library/SDK embedding API.

<a id="reference-kind-filtering-matrix"></a>

Expand Down Expand Up @@ -3251,7 +3251,10 @@ Downstream users can add lightweight language support without rebuilding
workspace ancestor `.cdidx-langmap.yaml`; workspace entries override user
entries;
- regex-backed symbol patterns are read from `.cdidx/patterns/*.yaml` and
`~/.config/cdidx/patterns/*.yaml`;
`~/.config/cdidx/patterns/*.yaml`; sidecars must be regular files under
non-symlink pattern directories, each file is capped at 64 KiB / 128 rules,
the process loads at most 128 configured rules total, and regex matches use a
100 ms timeout;
- `cdidx test-extractor --language <lang> --file <path> --json` runs symbol
extraction without building an index, and `--expect-symbols <json>` compares
the extracted JSON to a fixture.
Expand All @@ -3276,8 +3279,9 @@ patterns:
```

Each configured regex should expose a named `name` capture. If it does not,
`cdidx` uses the full match text as the symbol name. Invalid sidecar files are
ignored so a broken local experiment does not prevent indexing.
`cdidx` uses the full match text as the symbol name. Invalid, symlinked,
oversized, or over-budget sidecar files are skipped with a stderr diagnostic so
a broken local experiment does not prevent indexing.

## カスタム言語抽出

Expand All @@ -3286,10 +3290,16 @@ ignored so a broken local experiment does not prevent indexing.
- 拡張子 alias は `~/.config/cdidx/langmap.yaml` と、最初に見つかった workspace
祖先の `.cdidx-langmap.yaml` から読み込まれ、workspace 側が user 側を上書きします。
- regex ベースのシンボルパターンは `.cdidx/patterns/*.yaml` と
`~/.config/cdidx/patterns/*.yaml` から読み込まれます。
`~/.config/cdidx/patterns/*.yaml` から読み込まれます。sidecar は symlink ではない
pattern directory 配下の通常ファイルのみが対象で、各ファイルは 64 KiB / 128 ルール、
プロセス全体では configured rule 128 件に制限され、regex match には 100 ms の timeout が付きます。
- `cdidx test-extractor --language <lang> --file <path> --json` は index を作らずに
symbol extraction だけを実行し、`--expect-symbols <json>` で fixture JSON と比較できます。

各 regex は `name` という名前付き capture を公開することを推奨します。存在しない場合、
`cdidx` は match 全体の文字列を symbol 名として使います。無効、symlink、過大、または
上限超過の sidecar は stderr の診断付きで skip されるため、壊れたローカル実験が indexing を止めません。

## SQLite reader のデバッグ

`Database/DbDebug.cs` は `ExecuteTrackedReader` / `TrackedRead` の最後に流れた SQL、パラメーター、行ごとの状態を記録し、ループ途中で `SqliteException` が発生した場合に再現に十分な文脈を stderr へダンプする。インデックス済みのソースバイトが想定外の経路に漏れないよう、ダンプ経路はゲート制御されている:
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,9 @@ cdidx lsp --db .cdidx/codeindex.db
Custom language loops can stay out of tree: put extension aliases in
`.cdidx-langmap.yaml`, put regex symbol patterns in `.cdidx/patterns/*.yaml`,
and run `cdidx test-extractor --language <lang> --file <path> --json` to test
an extractor fixture without building a full index. See
an extractor fixture without building a full index. Pattern sidecars are
limited to regular files under non-symlink pattern directories, size/count
bounded per file and per process, and regex matches are time-limited. See
[Custom Language Extraction](DEVELOPER_GUIDE.md#custom-language-extraction).

After the first command, use these cues and follow-up commands:
Expand Down Expand Up @@ -358,7 +360,9 @@ cdidx lsp --db .cdidx/codeindex.db
カスタム言語の開発ループは out-of-tree で回せます。拡張子 alias は
`.cdidx-langmap.yaml`、regex シンボルパターンは `.cdidx/patterns/*.yaml` に置き、
`cdidx test-extractor --language <lang> --file <path> --json` で full index を作らずに
extractor fixture を確認できます。詳細は
extractor fixture を確認できます。pattern sidecar は symlink ではない pattern directory
配下の通常ファイルだけが対象で、size / count は file 単位と process 単位で制限され、
regex match には timeout が付きます。詳細は
[Custom Language Extraction](DEVELOPER_GUIDE.md#custom-language-extraction) を参照してください。

初回実行後は、次の見方と追加コマンドをよく使います。
Expand Down
17 changes: 17 additions & 0 deletions changelog.d/unreleased/2863.security.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
category: security
issues:
- 2863
affected:
- src/CodeIndex/Indexer/Extensibility/ExtractorPluginRegistry.cs
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
- DEVELOPER_GUIDE.md
---

## English

- **Workspace extractor plugin DLLs now require explicit trust (#2863)** — `cdidx` no longer discovers workspace `.cdidx/plugins/*.dll` files by default; set `CDIDX_TRUST_WORKSPACE_PLUGINS=1` only for checkouts whose plugin code you trust.

## 日本語

- **workspace extractor plugin DLL は明示的な trust が必要になりました (#2863)** — `cdidx` は既定で workspace の `.cdidx/plugins/*.dll` を discovery しません。信頼できる checkout の plugin code に限り `CDIDX_TRUST_WORKSPACE_PLUGINS=1` を設定してください。
19 changes: 19 additions & 0 deletions changelog.d/unreleased/2864.security.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
category: security
issues:
- 2864
affected:
- src/CodeIndex/Indexer/Extensibility/ConfiguredSymbolExtractor.cs
- src/CodeIndex/Indexer/Extensibility/ExtractorPluginRegistry.cs
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
- README.md
- DEVELOPER_GUIDE.md
---

## English

- **Workspace pattern extractors are now bounded and time-limited (#2864)** — pattern sidecars must be regular files under non-symlink pattern directories, configs are capped by file size plus per-file and total rule count, invalid or over-budget configs emit clear stderr diagnostics, and timed-out configured regexes are disabled.

## 日本語

- **workspace pattern extractor に上限と timeout を追加しました (#2864)** — pattern sidecar は symlink ではない pattern directory 配下の通常ファイルに限定され、config は file size と file 単位 / total の rule count で制限され、無効または上限超過の config は stderr に明確な診断を出し、timeout した configured regex は無効化されます。
41 changes: 40 additions & 1 deletion src/CodeIndex/Indexer/Extensibility/ConfiguredSymbolExtractor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ internal sealed class ConfiguredSymbolExtractor(
IReadOnlyCollection<string> fileExtensions,
IReadOnlyList<ConfiguredSymbolExtractor.PatternRule> patterns) : ISymbolExtractor
{
private readonly object timeoutGate = new();
private readonly HashSet<PatternRule> disabledTimeoutPatterns = [];
private readonly HashSet<string> timeoutWarnings = new(StringComparer.Ordinal);

internal sealed record PatternRule(string Kind, Regex Regex);

public string Language { get; } = language;
Expand All @@ -23,7 +27,20 @@ public IReadOnlyList<SymbolRecord> Extract(long fileId, string source, Extractio
lineNumber++;
foreach (var pattern in patterns)
{
var match = pattern.Regex.Match(line);
if (IsPatternDisabled(pattern))
continue;

Match match;
try
{
match = pattern.Regex.Match(line);
}
catch (RegexMatchTimeoutException)
{
DisablePatternAfterTimeout(pattern);
continue;
}

if (!match.Success)
continue;

Expand All @@ -47,4 +64,26 @@ public IReadOnlyList<SymbolRecord> Extract(long fileId, string source, Extractio

return symbols;
}

private bool IsPatternDisabled(PatternRule pattern)
{
lock (timeoutGate)
return disabledTimeoutPatterns.Contains(pattern);
}

private void DisablePatternAfterTimeout(PatternRule pattern)
{
var shouldReport = false;
lock (timeoutGate)
{
disabledTimeoutPatterns.Add(pattern);
shouldReport = timeoutWarnings.Add(pattern.Kind + "\0" + pattern.Regex);
}

if (!shouldReport)
return;

Console.Error.WriteLine(
$"[cdidx] Pattern extractor for language '{Language}' kind '{pattern.Kind}' timed out after {(int)ExtractorPluginRegistry.PatternRegexTimeout.TotalMilliseconds}ms; skipped this pattern.");
}
}
Loading
Loading