Propagate index cancellation through scan and extraction

[Widthdom]

Summary

• Propagates cancellation tokens through CLI/MCP index scanning, raw file reads, symbol extraction, and reference extraction.
• Normalizes scan-time cancellation into the existing interrupted JSON/exit-code contract.
• Adds regression coverage for fresh-index cancellation, dry-run scan cancellation, scanner cancellation, and extractor cancelled-token entry points.

Fixes #1841
Fixes #1818

Validation

• dotnet build CodeIndex.sln
• dotnet test tests/CodeIndex.Tests/CodeIndex.Tests.csproj --filter "FullyQualifiedName~FileIndexerTests|FullyQualifiedName~SymbolExtractorTests|FullyQualifiedName~ReferenceExtractorTests|FullyQualifiedName~IndexCommandRunnerTests.Run_CancelDuringFreshIndex_ReturnsInterruptedJson|FullyQualifiedName~IndexCommandRunnerTests.Run_CancelDuringDryRunScan_ReturnsInterruptedJson|FullyQualifiedName~IndexCommandRunnerTests.Run_CancelBeforeFreshScan_ReturnsInterruptedJson"
• dotnet run --project tools/CodeIndex.Changelog -- check
• git diff --check
• Codex adversarial review: No blocking/actionable issues found.

Documentation and changelog

• TESTING_GUIDE.md
• changelog.d/unreleased/1818.fixed.md
• changelog.d/unreleased/1841.fixed.md

Notes

• cdidx index can self-interrupt during full rebuild and leave the local index empty #2642 was excluded because it already had a work-started comment: Work started on branch codex/fix-issue2642.
• Follow-up: Index refresh can hang on large extraction phase #2665 tracks the separate long-running extraction/index refresh behavior found during validation.