Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,7 @@ Do not add mutable static caches, shared `StringBuilder` instances, reused `Matc
| `file_module` | File-scoped module/package declarations | Namespace-like context symbol |
| `function` | Functions, methods, constructors, delegates, tasks, and callable bindings that do not have a narrower kind | Primary callable definition; participates in callers/callees through reference rows |
| `generator` | JavaScript/TypeScript generator declarations | Callable definition; participates in callers/callees through reference rows |
| `heading` | Markdown headings | Outline symbol |
| `heading` | Markdown headings and language section markers such as C# regions, Python module docstrings, and JavaScript/TypeScript `@module` docblocks | Outline symbol |
| `hook` | JavaScript/TypeScript React custom hook bindings | Callable-like search/filter symbol |
| `implements` | Razor `@implements` directives | Context/search symbol |
| `import` | Imports, using directives, aliases, and package includes | Search/filter symbol |
Expand All @@ -251,12 +251,12 @@ Do not add mutable static caches, shared `StringBuilder` instances, reused `Matc
| `operator` | C# operator overload and conversion operator declarations | Callable definition; participates in callers/callees through reference rows |
| `object` | Object-literal/object container context used by nested extracted symbols | Container context |
| `package` | Package declarations | Namespace-like context symbol |
| `property` | Properties and property-like fields | Definition target; not treated as a call edge by itself |
| `property` | Properties, property-like fields, and GraphQL input fields | Definition target; not treated as a call edge by itself |
| `procedure` | Procedure declarations in languages such as Fortran | Callable definition |
| `program` | Program block declarations in languages such as Fortran | Definition target and container |
| `protocol` | Protocol declarations in languages that distinguish protocols from interfaces | Definition target and container |
| `protocol_impl` | Elixir `defimpl` protocol implementation declarations | Definition target and container for implementation blocks |
| `reference` | Secondary extracted symbolic references, such as HTML classes or metadata keys | Search/filter symbol |
| `reference` | Secondary extracted symbolic references, such as HTML classes, metadata keys, or GraphQL union variants | Search/filter symbol |
| `rule` | CSS/SCSS rule container context used by nested references | Container context |
| `route` | Razor route directives | Context/search symbol |
| `service` | Service declarations in IDL/protobuf-like languages | Definition target and container |
Expand Down
17 changes: 17 additions & 0 deletions changelog.d/unreleased/1636.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
category: fixed
issues:
- 1636
affected:
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
- DEVELOPER_GUIDE.md
---

## English

- **Outline now surfaces language section markers (#1636)** — C# `#region`, Python module docstrings, and JavaScript/TypeScript JSDoc `@module` markers are emitted as `heading` symbols for table-of-contents style outline views.

## 日本語

- **outline が言語ごとの section marker を表示するようになりました (#1636)** — C# `#region`、Python module docstring、JavaScript/TypeScript JSDoc `@module` marker を `heading` symbol として出力し、目次型の outline で扱えるようにしました。
17 changes: 17 additions & 0 deletions changelog.d/unreleased/1825.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
category: fixed
issues:
- 1825
affected:
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
- DEVELOPER_GUIDE.md
---

## English

- **GraphQL input fields and union variants are now indexed as anchors (#1825)** — `input` members are emitted as `property` symbols and `union` variants as `reference` symbols, including `extend` and multiline union declarations.

## 日本語

- **GraphQL input field と union variant を anchor として index するようになりました (#1825)** — `input` member は `property` symbol、`union` variant は `reference` symbol として出力され、`extend` と複数行 union 宣言にも対応します。
16 changes: 16 additions & 0 deletions changelog.d/unreleased/2016.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
category: fixed
issues:
- 2016
affected:
- src/CodeIndex/Database/DbSymbolReader.cs
- tests/CodeIndex.Tests/DbReaderTests.cs
---

## English

- **Outline depth now disambiguates same-named containers by qualified path (#2016)** — nested symbols now attach to the matching qualified parent, preventing duplicate class names in different scopes from skewing outline depth.

## 日本語

- **outline depth が同名 container を qualified path で区別するようになりました (#2016)** — nested symbol は一致する qualified parent に紐づくため、別 scope の同名 class が outline depth を誤らせなくなりました。
17 changes: 17 additions & 0 deletions src/CodeIndex/Database/DbSymbolReader.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1662,13 +1662,19 @@ private static int GetOutlineDepth(List<OutlineSymbol> symbols, int index, Dicti
private static int FindOutlineContainerIndex(List<OutlineSymbol> symbols, int childIndex, string containerName, string? containerKind)
{
var child = symbols[childIndex];
var expectedContainerPath = GetOutlineContainerPath(child);
for (var i = childIndex - 1; i >= 0; i--)
{
var candidate = symbols[i];
if (!string.Equals(candidate.Name, containerName, StringComparison.Ordinal))
continue;
if (containerKind != null && !string.Equals(candidate.Kind, containerKind, StringComparison.Ordinal))
continue;
if (expectedContainerPath != null
&& !string.Equals(candidate.Path, expectedContainerPath, StringComparison.Ordinal))
{
continue;
}
if (candidate.Line > child.Line)
continue;
if (IsOutlineContainerMatch(candidate, child.Line))
Expand All @@ -1678,6 +1684,17 @@ private static int FindOutlineContainerIndex(List<OutlineSymbol> symbols, int ch
return -1;
}

private static string? GetOutlineContainerPath(OutlineSymbol symbol)
{
if (string.IsNullOrWhiteSpace(symbol.Path) || string.IsNullOrWhiteSpace(symbol.Name))
return null;

var suffix = "." + symbol.Name;
return symbol.Path.EndsWith(suffix, StringComparison.Ordinal)
? symbol.Path[..^suffix.Length]
: null;
}

private static bool IsOutlineContainerMatch(OutlineSymbol candidate, int childLine)
{
if (candidate.StartLine <= childLine && candidate.EndLine >= childLine)
Expand Down
242 changes: 242 additions & 0 deletions src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,30 @@ public static partial class SymbolExtractor
{
public const int DefaultContractVersion = 1;
public const int CSharpContractVersion = 2;
private static readonly Regex GraphQLInputBlockRegex = new(
@"^\s*(?:extend\s+)?input\s+(?<name>\w+)[^{]*\{(?<body>.*?)^\s*\}",
RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.Multiline | RegexOptions.Singleline);
private static readonly Regex GraphQLInputFieldRegex = new(
@"^\s*(?<name>[_A-Za-z]\w*)\s*:",
RegexOptions.Compiled | RegexOptions.CultureInvariant | RegexOptions.Multiline);
private static readonly Regex GraphQLUnionDeclarationRegex = new(
@"^\s*(?:extend\s+)?union\s+(?<name>\w+)(?:\s+@\w+(?:\([^)]*\))?)*\s*=\s*(?<variants>.*)$",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private static readonly Regex GraphQLUnionHeaderRegex = new(
@"^\s*(?:extend\s+)?union\s+(?<name>\w+)(?:\s+@\w+(?:\([^)]*\))?)*\s*$",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private static readonly Regex GraphQLUnionVariantRegex = new(
@"\|?\s*(?<name>[_A-Za-z]\w*)\b",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private static readonly Regex GraphQLDeclarationStartRegex = new(
@"^\s*(?:extend\s+)?(?:type|interface|input|enum|union|scalar|schema|query|mutation|subscription|fragment|directive)\b",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private static readonly Regex CSharpRegionRegex = new(
@"^\s*#region(?:\s+(?<name>.*\S))?\s*$",
RegexOptions.Compiled | RegexOptions.CultureInvariant);
private static readonly Regex JavaScriptTypeScriptModuleDocRegex = new(
@"@module(?:\s+(?<name>[^\s*]+))?",
RegexOptions.Compiled | RegexOptions.CultureInvariant);

public static int GetContractVersion(string? lang)
{
Expand Down Expand Up @@ -4044,6 +4068,10 @@ public static List<SymbolRecord> Extract(long fileId, string? lang, string conte
ExtractSqlRoutineResultColumnSymbols(fileId, lines, sqlSyntheticSymbolLines, symbols);
ExtractSqlGeneratedColumnSymbols(fileId, lines, sqlSyntheticSymbolLines, symbols);
}
if (lang == "graphql")
ExtractGraphQLMemberSymbols(fileId, lines, symbols);
if (lang is "csharp" or "python" or "javascript" or "typescript")
ExtractSectionHeadingSymbols(fileId, lang, lines, symbols);
if (IsRazorLanguage(originalLang) || IsRazorFilePath(filePath))
ExtractRazorDirectiveSymbols(fileId, lines, symbols);
AssignContainers(symbols, lines, csharpLineStartStates);
Expand Down Expand Up @@ -4150,6 +4178,220 @@ private static int GetLineNumberFromOffset(List<int> lineStarts, int offset)
return ~index;
}

private static void ExtractGraphQLMemberSymbols(long fileId, string[] lines, List<SymbolRecord> symbols)
{
var content = string.Join('\n', lines);
var lineStarts = BuildLineStarts(content);
foreach (Match inputMatch in GraphQLInputBlockRegex.Matches(content))
{
var inputName = inputMatch.Groups["name"].Value;
var body = inputMatch.Groups["body"];
foreach (Match fieldMatch in GraphQLInputFieldRegex.Matches(body.Value))
{
var fieldGroup = fieldMatch.Groups["name"];
var absoluteIndex = body.Index + fieldGroup.Index;
var lineNumber = GetLineNumberFromOffset(lineStarts, absoluteIndex);
AddSymbolRecord(
symbols,
null,
lineNumber,
new SymbolRecord
{
FileId = fileId,
Kind = "property",
Name = fieldGroup.Value,
Line = lineNumber,
StartLine = lineNumber,
StartColumn = absoluteIndex - lineStarts[lineNumber - 1],
EndLine = lineNumber,
Signature = lines[lineNumber - 1].Trim(),
ContainerKind = "class",
ContainerName = inputName,
},
lines[lineNumber - 1]);
}
}

for (var lineIndex = 0; lineIndex < lines.Length; lineIndex++)
{
var match = GraphQLUnionDeclarationRegex.Match(lines[lineIndex]);
if (match.Success)
{
var unionName = match.Groups["name"].Value;
AddGraphQLUnionVariantSymbols(fileId, lines, lineIndex, match.Groups["variants"].Value, match.Groups["variants"].Index, unionName, symbols);
for (var continuationIndex = lineIndex + 1; continuationIndex < lines.Length; continuationIndex++)
{
var continuation = lines[continuationIndex];
if (string.IsNullOrWhiteSpace(continuation) || GraphQLDeclarationStartRegex.IsMatch(continuation))
break;

AddGraphQLUnionVariantSymbols(fileId, lines, continuationIndex, continuation, 0, unionName, symbols);
}

continue;
}

var headerMatch = GraphQLUnionHeaderRegex.Match(lines[lineIndex]);
if (!headerMatch.Success)
continue;

var headerUnionName = headerMatch.Groups["name"].Value;
for (var continuationIndex = lineIndex + 1; continuationIndex < lines.Length; continuationIndex++)
{
var continuation = lines[continuationIndex];
if (string.IsNullOrWhiteSpace(continuation) || GraphQLDeclarationStartRegex.IsMatch(continuation))
break;

var equalsIndex = continuation.IndexOf('=', StringComparison.Ordinal);
if (equalsIndex >= 0)
{
AddGraphQLUnionVariantSymbols(fileId, lines, continuationIndex, continuation[(equalsIndex + 1)..], equalsIndex + 1, headerUnionName, symbols);
for (var variantIndex = continuationIndex + 1; variantIndex < lines.Length; variantIndex++)
{
var variantContinuation = lines[variantIndex];
if (string.IsNullOrWhiteSpace(variantContinuation) || GraphQLDeclarationStartRegex.IsMatch(variantContinuation))
break;

AddGraphQLUnionVariantSymbols(fileId, lines, variantIndex, variantContinuation, 0, headerUnionName, symbols);
}

lineIndex = continuationIndex;
break;
}
}
}
}

private static void AddGraphQLUnionVariantSymbols(
long fileId,
string[] lines,
int lineIndex,
string variantText,
int baseColumn,
string unionName,
List<SymbolRecord> symbols)
{
variantText = StripGraphQLUnionVariantTrivia(variantText);
if (string.IsNullOrWhiteSpace(variantText))
return;

foreach (Match variantMatch in GraphQLUnionVariantRegex.Matches(variantText))
{
var variantName = variantMatch.Groups["name"].Value;
if (variantName == "extend" || variantName == "union")
continue;

AddSymbolRecord(
symbols,
null,
lineIndex + 1,
new SymbolRecord
{
FileId = fileId,
Kind = "reference",
Name = variantName,
Line = lineIndex + 1,
StartLine = lineIndex + 1,
StartColumn = baseColumn + variantMatch.Groups["name"].Index,
EndLine = lineIndex + 1,
Signature = lines[lineIndex].Trim(),
ContainerKind = "class",
ContainerName = unionName,
},
lines[lineIndex]);
}
}

private static string StripGraphQLUnionVariantTrivia(string text)
{
var commentIndex = text.IndexOf('#', StringComparison.Ordinal);
if (commentIndex >= 0)
text = text[..commentIndex];

var directiveIndex = text.IndexOf('@', StringComparison.Ordinal);
if (directiveIndex >= 0)
text = text[..directiveIndex];

return text;
}

private static void ExtractSectionHeadingSymbols(long fileId, string lang, string[] lines, List<SymbolRecord> symbols)
{
if (lang == "csharp")
{
for (var i = 0; i < lines.Length; i++)
{
var match = CSharpRegionRegex.Match(lines[i]);
if (!match.Success)
continue;

AddHeadingSymbol(fileId, lines, symbols, i, match.Groups["name"].Value.Trim(), "#region");
}
}
else if (lang == "python")
{
TryAddPythonModuleDocstringHeading(fileId, lines, symbols);
}
else
{
for (var i = 0; i < lines.Length; i++)
{
var match = JavaScriptTypeScriptModuleDocRegex.Match(lines[i]);
if (!match.Success)
continue;

AddHeadingSymbol(fileId, lines, symbols, i, match.Groups["name"].Value.Trim(), "@module");
}
}
}

private static void TryAddPythonModuleDocstringHeading(long fileId, string[] lines, List<SymbolRecord> symbols)
{
for (var i = 0; i < lines.Length; i++)
{
var trimmed = lines[i].TrimStart();
if (trimmed.Length == 0 || trimmed.StartsWith("#", StringComparison.Ordinal))
continue;

var quote = trimmed.StartsWith("\"\"\"", StringComparison.Ordinal) ? "\"\"\"" :
trimmed.StartsWith("'''", StringComparison.Ordinal) ? "'''" : null;
if (quote == null)
return;

var name = trimmed[quote.Length..].Trim();
if (name.EndsWith(quote, StringComparison.Ordinal))
name = name[..^quote.Length].Trim();
AddHeadingSymbol(fileId, lines, symbols, i, name, "module docstring");
return;
}
}

private static void AddHeadingSymbol(
long fileId,
string[] lines,
List<SymbolRecord> symbols,
int lineIndex,
string name,
string fallbackName)
{
var lineNumber = lineIndex + 1;
AddSymbolRecord(
symbols,
null,
lineNumber,
new SymbolRecord
{
FileId = fileId,
Kind = "heading",
Name = string.IsNullOrWhiteSpace(name) ? fallbackName : name,
Line = lineNumber,
StartLine = lineNumber,
EndLine = lineNumber,
Signature = lines[lineIndex].Trim(),
},
lines[lineIndex]);
}

private static void ExtractSqlGeneratedColumnSymbols(long fileId, string[] lines, string[] structuralLines, List<SymbolRecord> symbols)
{
var structuralContent = string.Join('\n', structuralLines);
Expand Down
Loading
Loading