Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions DEVELOPER_GUIDE.md

Large diffs are not rendered by default.

48 changes: 32 additions & 16 deletions USER_GUIDE.md

Large diffs are not rendered by default.

16 changes: 16 additions & 0 deletions changelog.d/unreleased/3438.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
category: fixed
issues:
- 3438
affected:
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.TypeScriptPathAliases.cs
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
---

## English

- **TypeScript path alias config warnings now preserve failure categories (#3438)** - unreadable configs, invalid JSON, and excessive extends depth now emit stable diagnostic codes while keeping the existing human-readable warning text.

## 日本語

- **TypeScript path alias config の警告で失敗カテゴリを保持するようにしました (#3438)** - 読み取れない config、不正な JSON、過剰な extends 深度では、既存の人間向け警告文を保ったまま安定した診断コードを出力します。
17 changes: 17 additions & 0 deletions changelog.d/unreleased/3479.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
category: fixed
issues:
- 3479
affected:
- DEVELOPER_GUIDE.md
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
- tests/CodeIndex.Tests/ReferenceExtractorTests.cs
---

## English

- **Extractor regex backtracking policy is now documented and audited (#3479)** - built-in symbol and reference extractor regexes must either use `RegexOptions.NonBacktracking` or the shared `BoundedRegex.DefaultMatchTimeout`.

## 日本語

- **extractor regex の backtracking policy を文書化し audit するようになりました (#3479)** - built-in symbol/reference extractor regex は `RegexOptions.NonBacktracking` または共有 `BoundedRegex.DefaultMatchTimeout` を使う必要があります。
17 changes: 17 additions & 0 deletions changelog.d/unreleased/3510.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
category: fixed
issues:
- 3510
affected:
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.CSharpScanner.cs
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
---

## English

- **Shell symbol extraction now ignores heredoc bodies and bounds function ranges (#3510)** - shell heredoc contents no longer emit phantom functions, and shell function ranges stop at their matching body close instead of stretching to EOF.

## 日本語

- **shell のシンボル抽出で heredoc 本文を無視し、関数範囲を制限しました (#3510)** - shell heredoc 内の内容が偽の関数として出力されなくなり、shell 関数範囲は EOF ではなく対応する本体の閉じ位置で止まります。
23 changes: 23 additions & 0 deletions changelog.d/unreleased/3516.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
category: fixed
issues:
- 3516
affected:
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.cs
- src/CodeIndex/Indexer/Symbols/SymbolExtractor.Sql.cs
- src/CodeIndex/Indexer/References/ReferenceExtractor.cs
- src/CodeIndex/Indexer/References/Languages/CssReferenceExtractor.cs
- src/CodeIndex/Indexer/References/Languages/DockerfileReferenceExtractor.cs
- src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.Patterns.cs
- src/CodeIndex/Indexer/References/Languages/SqlReferenceExtractor.cs
- tests/CodeIndex.Tests/SymbolExtractorTests.cs
- tests/CodeIndex.Tests/ReferenceExtractorTests.cs
---

## English

- **Case-insensitive extractor regexes now use culture-invariant matching consistently (#3516)** - built-in symbol and reference extractor regex audits now fail if an `IgnoreCase` regex omits `CultureInvariant`.

## 日本語

- **case-insensitive な extractor regex が一貫して culture-invariant matching を使うようになりました (#3516)** - built-in symbol/reference extractor の regex audit は、`IgnoreCase` regex が `CultureInvariant` を省いた場合に失敗します。
23 changes: 23 additions & 0 deletions changelog.d/unreleased/3559.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
category: fixed
issues:
- 3559
affected:
- src/CodeIndex/Database/DbReader.FilesStatus.cs
- src/CodeIndex/Cli/QueryCommandRunner.cs
- src/CodeIndex/Cli/CommandErrorCodes.cs
- src/CodeIndex/Cli/CommandErrorWriter.cs
- src/CodeIndex/Cli/CommandExitCodes.cs
- src/CodeIndex/Mcp/McpErrorEnvelope.cs
- src/CodeIndex/Mcp/McpToolHandlers.cs
- tests/CodeIndex.Tests/QueryCommandRunnerSearchTests.cs
- DEVELOPER_GUIDE.md
---

## English

- **`find --regex` timeouts now use the shared bounded regex policy (#3559)** - CLI and MCP `find` regex scans distinguish match timeouts from invalid regex syntax and emit machine-readable `regex_timeout` diagnostics.

## 日本語

- **`find --regex` の timeout が共有 bounded regex policy を使うようになりました (#3559)** - CLI と MCP の `find` regex scan は match timeout と regex 構文エラーを区別し、機械可読な `regex_timeout` 診断を出します。
3 changes: 3 additions & 0 deletions src/CodeIndex/Cli/CommandErrorCodes.cs
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,7 @@ internal static class CommandErrorCodes

/// <summary>Index extraction made no forward progress within the bounded stall timeout.</summary>
public const string IndexExtractionStalled = "E013_INDEX_EXTRACTION_STALLED";

/// <summary>A user-supplied regular expression exceeded the bounded match timeout while executing.</summary>
public const string RegexMatchTimeout = "E014_REGEX_MATCH_TIMEOUT";
}
5 changes: 3 additions & 2 deletions src/CodeIndex/Cli/CommandErrorWriter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,13 @@ internal static int WriteJsonOrHuman(
int exitCode,
string? hint = null,
string? usage = null,
string? errorCode = null)
string? errorCode = null,
string? category = null)
{
if (json)
{
WriteStdout(JsonSerializer.Serialize(
new CommandErrorJsonResult("error", message, hint, errorCode),
new CommandErrorJsonResult("error", message, hint, errorCode, Category: category),
CliJsonSerializerContextFactory.Create(jsonOptions).CommandErrorJsonResult));
return exitCode;
}
Expand Down
1 change: 1 addition & 0 deletions src/CodeIndex/Cli/CommandExitCodes.cs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ public static class CommandExitCodes
public const int InvalidArgument = 7;
public const int CancelledBySignal = 8;
public const int InstallError = 9;
public const int RuntimeError = 10;
public const int UnhandledException = 99;
public const int ExUsage = 64;
public const int Interrupted = CancelledBySignal;
Expand Down
37 changes: 31 additions & 6 deletions src/CodeIndex/Cli/QueryCommandRunner.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3984,8 +3984,9 @@ public static int RunFind(string[] cmdArgs, JsonSerializerOptions jsonOptions)
}
catch (Exception ex) when (options.Regex && (ex is ArgumentException || ex is RegexMatchTimeoutException))
{
Console.Error.WriteLine($"Error: invalid regular expression: {ex.Message}");
return CommandExitCodes.UsageError;
return ex is RegexMatchTimeoutException timeout
? WriteFindRegexTimeoutError(timeout, jsonOptions, options.Json)
: WriteFindInvalidRegexError(ex);
}
if (counts.Count == 0)
{
Expand Down Expand Up @@ -4037,13 +4038,11 @@ public static int RunFind(string[] cmdArgs, JsonSerializerOptions jsonOptions)
}
catch (ArgumentException ex) when (options.Regex)
{
Console.Error.WriteLine($"Error: invalid regular expression: {ex.Message}");
return CommandExitCodes.UsageError;
return WriteFindInvalidRegexError(ex);
}
catch (RegexMatchTimeoutException ex) when (options.Regex)
{
Console.Error.WriteLine($"Error: invalid regular expression: {ex.Message}");
return CommandExitCodes.UsageError;
return WriteFindRegexTimeoutError(ex, jsonOptions, options.Json);
}
var results = findResults.Results;
if (results.Count == 0)
Expand Down Expand Up @@ -4126,6 +4125,32 @@ public static int RunFind(string[] cmdArgs, JsonSerializerOptions jsonOptions)
});
}

private static int WriteFindInvalidRegexError(Exception ex)
{
Console.Error.WriteLine($"Error: invalid regular expression: {ex.Message}");
return CommandExitCodes.UsageError;
}

internal static int WriteFindRegexTimeoutError(RegexMatchTimeoutException ex, JsonSerializerOptions jsonOptions, bool json)
{
var timeout = FormatRegexMatchTimeout(ex.MatchTimeout);
return CommandErrorWriter.WriteJsonOrHuman(
json,
jsonOptions,
$"regular expression timed out after {timeout} while scanning indexed file contents.",
CommandExitCodes.RuntimeError,
hint: "Simplify the pattern, narrow the scan with --path/--lang, or omit --regex for literal text.",
errorCode: CommandErrorCodes.RegexMatchTimeout,
category: "regex_timeout");
}

internal static string FormatRegexMatchTimeout(TimeSpan timeout)
{
if (timeout.TotalMilliseconds < 1000)
return timeout.TotalMilliseconds.ToString("0.###", CultureInfo.InvariantCulture) + "ms";
return timeout.TotalSeconds.ToString("0.###", CultureInfo.InvariantCulture) + "s";
}

private static string? ValidateFindArgs(string[] args)
{
var (allowedWithValues, allowedFlags) = CliFlagSchema.GetParserFlagsPartitionedByValueBearing("find");
Expand Down
12 changes: 10 additions & 2 deletions src/CodeIndex/Database/DbReader.FilesStatus.cs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ public FindResults FindInFiles(string query, int limit, string? lang = null, IRe
maxLineWidth = LineWidthFormatter.ClampMaxLineWidth(maxLineWidth);
var comparison = exact ? StringComparison.Ordinal : StringComparison.OrdinalIgnoreCase;
var regexMatcher = regex
? new Regex(query, exact ? RegexOptions.None : RegexOptions.IgnoreCase, TimeSpan.FromMilliseconds(500))
? CreateFindRegexMatcher(query, exact)
: null;

using var fileCmd = _conn.CreateCommand();
Expand Down Expand Up @@ -167,7 +167,7 @@ public FindCountResult CountFindInFiles(string query, string? lang = null, IRead

var comparison = exact ? StringComparison.Ordinal : StringComparison.OrdinalIgnoreCase;
var regexMatcher = regex
? new Regex(query, exact ? RegexOptions.None : RegexOptions.IgnoreCase, TimeSpan.FromMilliseconds(500))
? CreateFindRegexMatcher(query, exact)
: null;
using var fileCmd = _conn.CreateCommand();
var sql = "SELECT f.id, f.path, f.lang, f.lines FROM files f WHERE 1=1";
Expand Down Expand Up @@ -351,6 +351,14 @@ private static bool TryCreateFindLineMatch(int matchColumn, int matchLength, int
return !focusColumn.HasValue || (focusColumn.Value >= rawMatchColumn + 1 && focusColumn.Value <= rawMatchColumn + rawMatchLength);
}

private static Regex CreateFindRegexMatcher(string query, bool exact)
{
var options = RegexOptions.CultureInvariant;
if (!exact)
options |= RegexOptions.IgnoreCase;
return new Regex(query, options, BoundedRegex.DefaultMatchTimeout);
}

private static void AddLineToFindWindow(IndexedLine indexedLine, Queue<IndexedLine> snippetWindow, Dictionary<int, string> snippetLinesByNumber)
{
snippetWindow.Enqueue(indexedLine);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ internal static class CssReferenceExtractor
@"@include\s+(?<name>[A-Za-z_][\w-]*)",
RegexOptions.Compiled);

private static readonly Regex CssCustomPropertyReferenceRegex = new(@"\bvar\(\s*--(?<name>[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase);
private static readonly Regex CssAnimationNameValueRegex = new(@"\banimation-name\s*:\s*(?<value>[^;{}]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase);
private static readonly Regex CssAnimationShorthandValueRegex = new(@"\banimation\s*:\s*(?<value>[^;{}]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase);
private static readonly Regex CssCustomPropertyReferenceRegex = new(@"\bvar\(\s*--(?<name>[\w-]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
private static readonly Regex CssAnimationNameValueRegex = new(@"\banimation-name\s*:\s*(?<value>[^;{}]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
private static readonly Regex CssAnimationShorthandValueRegex = new(@"\banimation\s*:\s*(?<value>[^;{}]+)", RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
private static readonly Regex CssClassSelectorReferenceRegex = new(@"\.(?<name>[\w-]+)", RegexOptions.Compiled);
// First char restricted to letter/`_`/`-` so numeric hex colors like `#336699`
// do not match. Letter-only hex colors (`#fff`) are still ambiguous; the
Expand All @@ -33,7 +33,7 @@ internal static class CssReferenceExtractor
private static readonly Regex CssIdSelectorReferenceRegex = new(@"#(?<name>[A-Za-z_-][\w-]*)", RegexOptions.Compiled);
private static readonly Regex CssImportReferenceRegex = new(
@"@import\s+(?:url\(\s*)?(?:""(?<name>[^""]+)""|'(?<name>[^']+)'|(?<name>[^\s)""';]+))",
RegexOptions.Compiled | RegexOptions.IgnoreCase);
RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);
private static readonly Regex CssInlineBlockCommentRegex = new(@"/\*.*?\*/", RegexOptions.Compiled);

private static readonly ReferencePattern[] CssReferencePatterns =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,15 @@ internal static class DockerfileReferenceExtractor
{
private static readonly Regex StageReferenceRegex = new(
@"^\s*FROM\s+(?:--platform=\S+\s+)?(?<name>[A-Za-z0-9_.-]+)\s+AS\s+[A-Za-z0-9_.-]+(?:\s+#.*)?\s*$",
RegexOptions.Compiled | RegexOptions.IgnoreCase);
RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);

private static readonly Regex CopyFromReferenceRegex = new(
@"^\s*(?:ONBUILD\s+)?(?:COPY|ADD)\b.*?--from=[""']?(?<name>[A-Za-z0-9_.-]+)(?![:/@])\b[""']?",
RegexOptions.Compiled | RegexOptions.IgnoreCase);
RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);

private static readonly Regex RunMountFromReferenceRegex = new(
@"(?:^|,)from=[""']?(?<name>[A-Za-z0-9_.-]+)(?![:/@])\b[""']?",
RegexOptions.Compiled | RegexOptions.IgnoreCase);
RegexOptions.Compiled | RegexOptions.IgnoreCase | RegexOptions.CultureInvariant);

private static readonly Regex UnbracedVariableReferenceRegex = new(
@"(?<![\$\\])\$(?<name>[A-Za-z_][A-Za-z0-9_]*)",
Expand Down
Loading
Loading