Skip to content

[codex] Improve extractor hot path performance - #3607

Merged
Widthdom merged 2 commits into
mainfrom
fix-extractor-hot-path-performance
Jun 13, 2026
Merged

[codex] Improve extractor hot path performance#3607
Widthdom merged 2 commits into
mainfrom
fix-extractor-hot-path-performance

Conversation

@Widthdom

Copy link
Copy Markdown
Owner

Summary

  • Fix the C# reference-extraction slowdown by avoiding repeated value-receiver body rescans and expensive duplicate checks.
  • Reduce non-C# extractor hot-path overhead by replacing candidate-by-candidate scans over growing symbol/reference lists with keyed HashSet/cache lookups.
  • Add large-fixture runaway guards for C#, Rust, Shell, Go, Java, Kotlin, C++, Dockerfile, JavaScript, TypeScript, and Swift paths.
  • Document the performance issue and prevention guidance in changelog/developer documentation.

Root Cause

Large generated files could trigger super-linear extraction work when supplemental extractors repeatedly scanned already accumulated references, symbols, or scan targets while processing each new candidate. The observed severe regression was in C# reference extraction; the follow-up audit found similar theoretical hot paths in several non-C# symbol/reference extractors.

Validation

  • dotnet build src/CodeIndex/CodeIndex.csproj --no-restore -p:UseSharedCompilation=false -m:1 -v:minimal
  • dotnet build tests/CodeIndex.Tests/CodeIndex.Tests.csproj --no-restore -p:UseSharedCompilation=false -m:1 -v:minimal
  • Reflection harness for targeted extractor performance/correctness tests: 19 PASS
  • dotnet run --project tools/CodeIndex.Changelog -- check
  • git diff --check
  • cdidx index ... --json
  • cdidx searches confirmed no remaining symbols.Any, symbols.First, targets.Any, references.Any, or references.First hot-path matches in extractor code.

Note: dotnet test could not run in this environment because VSTest/MSBuild socket or pipe creation failed with Permission denied; targeted tests were run through the reflection harness instead.

@Widthdom
Widthdom marked this pull request as ready for review June 13, 2026 07:45
@Widthdom
Widthdom merged commit ef5524c into main Jun 13, 2026
10 checks passed
@Widthdom
Widthdom deleted the fix-extractor-hot-path-performance branch June 13, 2026 07:45
Widthdom added a commit that referenced this pull request Jun 18, 2026
* Optimize cdidx full scan after HEAD changes

* Optimize fresh full-scan indexing

* Limit reference line lookups to insert batch

* Optimize C# reference line-state scanning

* Add symbols-only indexing mode (#3607)

* Fix symbols-only graph reuse marker (#3655)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant