Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,97 @@ Query commands that accept path filters (`search`, `definition`, `references`, `

Do not add mutable static caches, shared `StringBuilder` instances, reused `MatchCollection` enumerators, or singleton scanner state to extractor code. If a future extractor needs cross-call memoization, use an explicit thread-safe collection and add a targeted parallel regression test that proves deterministic output under concurrent calls.

### Symbol Kind Taxonomy

`symbols.kind`, `symbols.container_kind`, and `symbol_references.container_kind` use the public symbol kind taxonomy below. New extractors must register new kind values in `SymbolKindCatalog` before writing them so schema checks, writer validation, CLI filters, and downstream JSON consumers stay aligned.

| Kind | Current producers / meaning | Graph behavior |
|---|---|---|
| `accessor` | Accessor declarations when extracted separately from their owning property | Search/filter symbol |
| `annotation` | Annotation declarations or annotation-like language constructs | Metadata/search symbol |
| `async_function` | JavaScript/TypeScript async function declarations | Callable definition; participates in callers/callees through reference rows |
| `async_generator` | JavaScript/TypeScript async generator declarations | Callable definition; participates in callers/callees through reference rows |
| `attribute` | Razor attributes and metadata-like declarations | Context/search symbol; not a call edge by itself |
| `associatedtype` | Swift associated type declarations | Type-like definition target |
| `class` | Class declarations across object-oriented languages | Definition target and container |
| `class_hook` | Python class hook methods such as dunder hooks reclassified from functions | Callable/search symbol |
| `code` | Markdown fenced or structured code blocks | Search/outline symbol |
| `constant` | Constant declarations where the language distinguishes them | Search/filter symbol |
| `delegate` | C# / F# delegate declarations | Callable type definition and container-like target |
| `enum` | Enum declarations | Definition target and container |
| `event` | Event declarations | Search/filter symbol |
| `field` | Field declarations where distinct from properties | Search/filter symbol |
| `file_module` | File-scoped module/package declarations | Namespace-like context symbol |
| `function` | Functions, methods, constructors, delegates, tasks, and callable bindings that do not have a narrower kind | Primary callable definition; participates in callers/callees through reference rows |
| `generator` | JavaScript/TypeScript generator declarations | Callable definition; participates in callers/callees through reference rows |
| `heading` | Markdown headings | Outline symbol |
| `hook` | JavaScript/TypeScript React custom hook bindings | Callable-like search/filter symbol |
| `implements` | Razor `@implements` directives | Context/search symbol |
| `import` | Imports, using directives, aliases, and package includes | Search/filter symbol |
| `interface` | Interface declarations | Definition target and container |
| `lambda` | Named lambda/arrow bindings | Callable definition; participates in callers/callees through reference rows |
| `layout` | Razor layout directives | Context/search symbol |
| `method` | Languages or hooks that explicitly distinguish methods from functions | Callable definition; participates in callers/callees through reference rows |
| `module` | Module declarations | Definition target and container |
| `namespace` | Namespace declarations | Definition target and container |
| `operator` | C# operator overload and conversion operator declarations | Callable definition; participates in callers/callees through reference rows |
| `object` | Object-literal/object container context used by nested extracted symbols | Container context |
| `package` | Package declarations | Namespace-like context symbol |
| `property` | Properties and property-like fields | Definition target; not treated as a call edge by itself |
| `procedure` | Procedure declarations in languages such as Fortran | Callable definition |
| `program` | Program block declarations in languages such as Fortran | Definition target and container |
| `protocol` | Protocol declarations in languages that distinguish protocols from interfaces | Definition target and container |
| `reference` | Secondary extracted symbolic references, such as HTML classes or metadata keys | Search/filter symbol |
| `rule` | CSS/SCSS rule container context used by nested references | Container context |
| `route` | Razor route directives | Context/search symbol |
| `service` | Service declarations in IDL/protobuf-like languages | Definition target and container |
| `specialization` | C++ template specialization declarations | Definition target for specialized type/function forms |
| `struct` | Struct declarations | Definition target and container |
| `submodule` | Fortran submodule declarations | Namespace/module-like definition target |
| `subroutine` | Fortran subroutine declarations | Callable definition |
| `test.method` | Test methods detected by test-aware extraction | Callable definition; participates in callers/callees through reference rows |
| `trait` | Trait declarations in languages that distinguish traits from interfaces | Definition target and container |
| `type` | Type declarations where a narrower class/interface/struct/enum kind is not available | Definition target |
| `typealias` | Type alias declarations | Definition target for alias names |
| `union` | Union declarations | Definition target and container |
| `block data` | Fortran block data declarations | Definition target |
| `variable` | Variable bindings | Search/filter symbol |

`symbol_references.reference_kind` uses this separate reference taxonomy:

| Reference kind | Meaning |
|---|---|
| `annotation` | Annotation usage in languages that distinguish annotations from attributes |
| `attribute` | Metadata/attribute usage |
| `augmentation` | TypeScript declaration/interface merge edge |
| `call` | Function, method, operator, macro, or command call |
| `capture` | Captured callback/delegate relationship used by impact analysis |
| `column_reference` | SQL column reference in a statement-specific context |
| `consumes_hook` | React hook consumption relationship |
| `const_assertion` | TypeScript `as const` assertion edge |
| `const_generic_reference` | Rust const generic argument reference |
| `copy_from` | Dockerfile `COPY --from=<stage>` stage dependency |
| `cte_body_reference` | SQL common table expression body reference |
| `decorator` | Python decorator usage |
| `extends` | Inheritance or type-extension relationship |
| `from` | Dockerfile `FROM <stage>` dependency |
| `friend` | C++ friend declaration relationship |
| `generic_type_argument` | Generic type argument attached to an explicit invocation |
| `implement` | Interface implementation relationship |
| `implicit_implementation` | C# implicit interface implementation relationship |
| `import` | Import/include/reference through a module system |
| `instantiate` | Constructor or object creation |
| `join_condition_reference` | SQL join/merge condition column reference |
| `lifetime_reference` | Rust/C#-style lifetime or lifetime-like type reference |
| `metadata` | Metadata-only reference |
| `reference` | Generic persisted reference row used by fixtures or extractors without a narrower edge kind |
| `razor_event_binding` | Razor event binding relationship |
| `stage` | Build-stage relationship |
| `subscribe` | Event subscription relationship |
| `type_reference` | Type annotation, generic constraint, or other type-position reference |
| `unsubscribe` | Event unsubscription relationship |
| `use` | Generic usage relationship when no narrower reference kind applies |

### Status freshness age threshold

`status --check` keeps the DB/worktree checksum comparison in `IndexFreshnessChecker`, but the user-facing age hint threshold is resolved in `QueryCommandRunner`: CLI `--stale-after <duration>` wins over `CDIDX_STALE_AFTER`, which wins over `.cdidxrc.json`'s `stale_after`, then the 24-hour default. Supported duration suffixes are `m`, `h`, and `d`. JSON output includes `stale_after_seconds` and `index_age_seconds` only for `--check`, so clients can confirm which threshold was applied without inferring it from text.
Expand Down
17 changes: 17 additions & 0 deletions changelog.d/unreleased/1691.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
category: fixed
issues:
- 1691
affected:
- src/CodeIndex/Database/DbContext.cs
- src/CodeIndex/Database/DbWriter.cs
- tests/CodeIndex.Tests/DatabaseTests.cs
---

## English

- **Rejected malformed persisted kind values (#1691)** — new schemas add CHECK constraints for symbol/reference kind columns, and writer paths fail fast when extractors try to persist unregistered kind values.

## 日本語

- **不正な kind 値の永続化を拒否するようにしました (#1691)** — 新規 schema が symbol/reference kind 列に CHECK 制約を追加し、extractor が未登録 kind 値を永続化しようとした場合は writer path が早期に失敗します。
16 changes: 16 additions & 0 deletions changelog.d/unreleased/1762.docs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
category: docs
issues:
- 1762
affected:
- DEVELOPER_GUIDE.md
- src/CodeIndex/Models/SymbolKindCatalog.cs
---

## English

- **Documented the symbol kind taxonomy (#1762)** — `DEVELOPER_GUIDE.md` now lists the public symbol/reference kind values, and `SymbolKindCatalog` centralizes the registered values for code paths that need to enforce the taxonomy.

## 日本語

- **symbol kind taxonomy を文書化しました (#1762)** — `DEVELOPER_GUIDE.md` に公開 symbol/reference kind 値を一覧化し、taxonomy を検証するコードが参照できるよう `SymbolKindCatalog` に登録値を集約しました。
16 changes: 16 additions & 0 deletions changelog.d/unreleased/1965.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
category: fixed
issues:
- 1965
affected:
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
---

## English

- **C# operator overloads now use the `operator` symbol kind (#1965)** — arithmetic, comparison, conversion, checked, and static abstract interface operators are no longer indexed as generic `function` symbols.

## 日本語

- **C# operator overload が `operator` symbol kind を使うようになりました (#1965)** — 算術、比較、変換、checked、static abstract interface の各 operator が汎用 `function` symbol として index されなくなりました。
2 changes: 1 addition & 1 deletion src/CodeIndex/Cli/IndexCommandRunner.cs
Original file line number Diff line number Diff line change
Expand Up @@ -4636,7 +4636,7 @@ private static int CountConsecutiveQuotes(char[] chars, int index)
}

private static bool IsCSharpStaticInterfaceContractSymbol(SymbolRecord symbol)
=> symbol.Kind is "function" or "property"
=> symbol.Kind is "function" or "operator" or "property"
&& symbol.ContainerKind == "interface"
&& !string.IsNullOrWhiteSpace(symbol.Signature)
&& ContainsCSharpWord(symbol.Signature!, "static")
Expand Down
12 changes: 8 additions & 4 deletions src/CodeIndex/Database/DbContext.cs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
using CodeIndex.Cli;
using CodeIndex.Indexer;
using CodeIndex.Models;
using Microsoft.Data.Sqlite;
using System.Globalization;

Expand Down Expand Up @@ -1316,12 +1317,15 @@ file_id INTEGER NOT NULL REFERENCES files(id) ON DELETE CASCADE,
UNIQUE(file_id, line, context)
)");

var symbolKindCheck = SymbolKindCatalog.ToSqlCheckInList(SymbolKindCatalog.SymbolKinds);
var referenceKindCheck = SymbolKindCatalog.ToSqlCheckInList(SymbolKindCatalog.ReferenceKinds);

// Symbols table / シンボルテーブル
Execute(@"
CREATE TABLE IF NOT EXISTS symbols (
id INTEGER PRIMARY KEY AUTOINCREMENT,
file_id INTEGER NOT NULL REFERENCES files(id) ON DELETE CASCADE,
kind TEXT,
kind TEXT CHECK (kind IN (" + symbolKindCheck + @")),
sub_kind TEXT,
name TEXT,
line INTEGER,
Expand All @@ -1331,7 +1335,7 @@ file_id INTEGER NOT NULL REFERENCES files(id) ON DELETE CASCADE,
body_start_line INTEGER,
body_end_line INTEGER,
signature TEXT,
container_kind TEXT,
container_kind TEXT CHECK (container_kind IS NULL OR container_kind IN (" + symbolKindCheck + @")),
container_name TEXT,
container_qualified_name TEXT,
family_key TEXT,
Expand All @@ -1346,12 +1350,12 @@ CREATE TABLE IF NOT EXISTS symbol_references (
id INTEGER PRIMARY KEY AUTOINCREMENT,
file_id INTEGER NOT NULL REFERENCES files(id) ON DELETE CASCADE,
symbol_name TEXT,
reference_kind TEXT,
reference_kind TEXT CHECK (reference_kind IN (" + referenceKindCheck + @")),
line INTEGER,
column_number INTEGER,
context TEXT,
reference_line_id INTEGER REFERENCES reference_lines(id),
container_kind TEXT,
container_kind TEXT CHECK (container_kind IS NULL OR container_kind IN (" + symbolKindCheck + @")),
container_name TEXT
)");

Expand Down
2 changes: 1 addition & 1 deletion src/CodeIndex/Database/DbReader.CSharpResolution.cs
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ SELECT s.container_qualified_name
FROM symbols s
JOIN files f ON s.file_id = f.id
WHERE f.lang = 'csharp'
AND s.kind IN ('function', 'property')
AND s.kind IN ('function', 'operator', 'property')
AND s.container_qualified_name IS NOT NULL
AND s.container_qualified_name != ''
AND s.name = @memberName COLLATE NOCASE
Expand Down
2 changes: 1 addition & 1 deletion src/CodeIndex/Database/DbSymbolReader.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1485,7 +1485,7 @@ private static string BuildOutlineSymbolPath(string? containerQualifiedName, str

private static bool IsCallableOutlineSymbol(string kind)
{
return kind is "function" or "method" or "constructor";
return kind is "function" or "operator" or "method" or "constructor";
}

private static string? TryBuildCompactCallableSignature(string name, string? signature, string? lang)
Expand Down
24 changes: 22 additions & 2 deletions src/CodeIndex/Database/DbWriter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -866,6 +866,7 @@ INSERT INTO symbols (
for (int j = start; j < end; j++)
{
var symbol = symbols[j];
ValidateSymbolKinds(symbol);
var startLine = symbol.StartLine > 0 ? symbol.StartLine : symbol.Line;
var endLine = symbol.EndLine > 0 ? symbol.EndLine : startLine;
if (j > start)
Expand Down Expand Up @@ -929,7 +930,7 @@ FROM symbols s
s.kind = 'interface'
OR (
s.container_kind = 'interface'
AND s.kind IN ('function', 'property')
AND s.kind IN ('function', 'operator', 'property')
AND s.signature LIKE '%static%'
AND (s.signature LIKE '%abstract%' OR s.signature LIKE '%virtual%')
)
Expand Down Expand Up @@ -979,7 +980,7 @@ FROM symbols s
JOIN files f ON f.id = s.file_id
WHERE f.lang = 'csharp'
AND s.container_kind = 'interface'
AND s.kind IN ('function', 'property')
AND s.kind IN ('function', 'operator', 'property')
AND s.signature LIKE '%static%'
AND (s.signature LIKE '%abstract%' OR s.signature LIKE '%virtual%')";

Expand Down Expand Up @@ -1026,6 +1027,7 @@ INSERT INTO symbol_references (
for (int j = i; j < end; j++)
{
var reference = references[j];
ValidateReferenceKinds(reference);
var referenceLineId = referenceLineIds[(reference.FileId, reference.Line, reference.Context)];

if (j > i)
Expand Down Expand Up @@ -1060,6 +1062,24 @@ INSERT INTO symbol_references (
RefreshMutualRecursionFlags();
}

private static void ValidateSymbolKinds(SymbolRecord symbol)
{
if (!SymbolKindCatalog.IsValidSymbolKind(symbol.Kind))
throw new ArgumentException($"Unknown symbol kind '{symbol.Kind}'. Register the kind in {nameof(SymbolKindCatalog)} before writing it.", nameof(symbol));

if (symbol.ContainerKind != null && !SymbolKindCatalog.IsValidSymbolKind(symbol.ContainerKind))
throw new ArgumentException($"Unknown symbol container kind '{symbol.ContainerKind}'. Register the kind in {nameof(SymbolKindCatalog)} before writing it.", nameof(symbol));
}

private static void ValidateReferenceKinds(ReferenceRecord reference)
{
if (!SymbolKindCatalog.IsValidReferenceKind(reference.ReferenceKind))
throw new ArgumentException($"Unknown reference kind '{reference.ReferenceKind}'. Register the kind in {nameof(SymbolKindCatalog)} before writing it.", nameof(reference));

if (reference.ContainerKind != null && !SymbolKindCatalog.IsValidSymbolKind(reference.ContainerKind))
throw new ArgumentException($"Unknown reference container kind '{reference.ContainerKind}'. Register the kind in {nameof(SymbolKindCatalog)} before writing it.", nameof(reference));
}

private Dictionary<(long FileId, int Line, string Context), long> UpsertReferenceLines(IReadOnlyList<ReferenceRecord> references, int start, int end)
{
var contextsByLine = new Dictionary<(long FileId, int Line, string Context), string>();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -495,7 +495,7 @@ public static void EmitQualifiedEnumMemberReferences(
{
foreach (var candidate in candidates)
{
if (candidate.Kind != "function")
if (candidate.Kind is not ("function" or "operator"))
continue;
if (candidate.StartLine <= lineNumber && candidate.BodyEndLine!.Value >= lineNumber)
return candidate;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ private static void EmitCSharpStaticInterfaceMemberImplementationReferences(

var interfaceGenericParameters = BuildCSharpInterfaceGenericParameterLookup(workspaceSymbols);
var staticMembersByContainer = symbols
.Where(symbol => symbol.Kind is "function" or "property"
.Where(symbol => symbol.Kind is "function" or "operator" or "property"
&& !string.IsNullOrWhiteSpace(symbol.ContainerName)
&& !string.IsNullOrWhiteSpace(symbol.Signature)
&& ContainsCSharpWord(symbol.Signature!, "static"))
Expand Down
2 changes: 1 addition & 1 deletion src/CodeIndex/Indexer/References/ReferenceExtractor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ internal sealed class CSharpWhereConstraintState
};

private static bool IsFunctionLikeSymbolKind(string kind)
=> kind is "function" or "lambda" or "async_function" or "generator" or "async_generator";
=> kind is "function" or "operator" or "lambda" or "async_function" or "generator" or "async_generator";

private static readonly Dictionary<string, HashSet<string>> LanguageSpecificIgnoredCallNames = new(StringComparer.Ordinal)
{
Expand Down
Loading
Loading