Fix DbReader schema cache reuse for issue #1701

DEVELOPER_GUIDE.md

@@ -143,6 +143,8 @@ Current stable codes and triggers:
 
 Under WAL, `NORMAL` avoids per-commit fsync pressure during 500-row indexing batches while preserving database consistency after crashes. `DbWriter` runs `PRAGMA wal_checkpoint(PASSIVE)` after each outer transaction commit, and SQLite may also checkpoint automatically after the configured 1000-page threshold. Both checkpoint paths are opportunistic: active readers are not blocked, and an uncheckpointed WAL is expected state rather than corruption. If the process is killed after SQLite has committed a transaction but before checkpointing, the next normal opener rolls the WAL forward; no manual recovery step is required. If the process dies before a transaction commits, that transaction is rolled back by SQLite.
 
+`DbReader` schema discovery uses a process-level cache keyed by the normalized DB path. The cache stores `PRAGMA table_info`, `PRAGMA index_list`, and `sqlite_master` table-existence results, and checks `PRAGMA schema_version` before serving a lookup so SQLite DDL performed by cdidx or an external `sqlite3` session invalidates stale snapshots. Manual schema edits outside cdidx are still unsupported operationally; run `cdidx validate` after such edits before trusting query output.
+
 Index write batches also stamp `codeindex_meta.batch_in_progress=true` before starting a mutation transaction and clear it inside the transaction that commits the matching rows and readiness metadata. If the indexer crashes after the marker is written but before the commit clears it, the next writable DB open demotes readiness bits and warns: `Last batch did not complete; run cdidx index --rebuild to re-index from a known clean state.` Gracefully handled per-file errors clear the marker after rollback; orphaned markers are reserved for interrupted or crashed batches whose trust metadata should not be treated as clean.
 
 Read-only fallback uses an immutable SQLite URI when the normal writable open cannot create or lock journal/WAL side files, so query commands can still read a DB from read-only or sandboxed storage. That fallback intentionally skips writable pragmas, migrations, and WAL recovery writes; if a WAL is present and must be observed, copy the `.db`, `.db-wal`, and `.db-shm` files together to a writable location or use a SQLite backup from an environment that can open the full WAL set. `status --json` exposes the resolved connection values under `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`) for automation and support diagnostics. `cdidx vacuum` runs `PRAGMA incremental_vacuum` against writable incremental-auto-vacuum DBs, and performs a one-time `PRAGMA auto_vacuum=INCREMENTAL` plus full `VACUUM` conversion for legacy no-autovacuum DBs.
@@ -1750,6 +1752,8 @@ path filter を受け付ける query コマンド（`search`, `definition`, `ref
 
 WAL では `NORMAL` により 500 行単位の indexing batch ごとの fsync 負荷を避けつつ、crash 後の database consistency を保つ。`DbWriter` は outer transaction commit 後に `PRAGMA wal_checkpoint(PASSIVE)` を実行し、SQLite も設定済みの 1000 page threshold を超えると自動 checkpoint する場合がある。どちらの checkpoint path も opportunistic であり、active reader は block されず、未 checkpoint の WAL は corruption ではなく期待される状態である。SQLite が transaction を commit した後、checkpoint 前に process が kill された場合、次の通常 open が WAL を roll forward するため手動 recovery は不要。commit 前に process が終了した transaction は SQLite により rollback される。
 
+`DbReader` の schema discovery は正規化済み DB path を key にした process-level cache を使う。この cache は `PRAGMA table_info`、`PRAGMA index_list`、`sqlite_master` の table existence 結果を保持し、lookup 前に `PRAGMA schema_version` を確認するため、cdidx や外部 `sqlite3` session による SQLite DDL は stale snapshot を invalidate する。ただし cdidx 外での手動 schema edit は運用上 unsupported であり、その後は query output を信頼する前に `cdidx validate` を実行すること。
+
 通常の writable open が journal/WAL side file を作成または lock できない場合、read-only fallback は immutable SQLite URI を使うため、query command は read-only / sandboxed storage 上の DB でも読み取りを継続できる。この fallback は意図的に writable pragma、migration、WAL recovery write を skip する。WAL が存在し、その内容を観測する必要がある場合は、`.db` / `.db-wal` / `.db-shm` をまとめて writable location に copy するか、full WAL set を open できる環境で SQLite backup を使う。`status --json` は automation / support diagnostics 用に、解決済みの接続値を `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`) で公開する。`cdidx vacuum` は incremental-auto-vacuum DB では `PRAGMA incremental_vacuum` を実行し、legacy no-autovacuum DB では初回のみ `PRAGMA auto_vacuum=INCREMENTAL` と full `VACUUM` で変換する。
 
 ## データベーススキーマ

changelog.d/unreleased/1701.fixed.md

@@ -0,0 +1,18 @@
+---
+category: fixed
+issues:
+  - 1701
+affected:
+  - src/CodeIndex/Database/DbContext.cs
+  - src/CodeIndex/Database/DbSchemaCache.cs
+  - tests/CodeIndex.Tests/DbSchemaCacheTests.cs
+  - DEVELOPER_GUIDE.md
+---
+
+## English
+
+- **DbReader now reuses schema snapshots across DbContext instances for the same database (#1701)** — schema discovery caches `PRAGMA table_info`, `PRAGMA index_list`, and table-existence results by normalized DB path, reducing repeated reader startup work while invalidating on SQLite schema-version changes.
+
+## 日本語
+
+- **同じ database の DbContext 間で DbReader が schema snapshot を再利用するようになりました (#1701)** — schema discovery は正規化済み DB path ごとに `PRAGMA table_info`、`PRAGMA index_list`、table existence 結果を cache し、SQLite schema-version 変更時には invalidate します。

src/CodeIndex/Database/DbContext.cs

@@ -29,6 +29,7 @@ public class DbContext : IDisposable
 
     private readonly SqliteConnection _connection;
     private readonly bool _isReadOnly;
+    private readonly string? _schemaCacheKey;
     private SqliteTransaction? _activeMigrationTransaction;
     private DbSchemaCache? _schemaCache;
     private PreparedCommandCache? _preparedCommands;
@@ -46,13 +47,13 @@ public static string GetSymbolExtractorVersionMetaKey(string lang)
         => SymbolExtractorVersionMetaPrefix + lang;
 
     /// <summary>
-    /// Connection-scoped schema cache. Created lazily so a `DbContext` that
-    /// never opens a reader pays nothing. Subsequent `DbReader` instances on
-    /// the same `DbContext` reuse the cached `PRAGMA table_info` /
-    /// `PRAGMA index_list` / `sqlite_master` results instead of re-running
-    /// the scan on every construction (issue #1565).
+    /// DB-path-scoped schema cache. Created lazily so a `DbContext` that
+    /// never opens a reader pays nothing. Subsequent `DbReader` instances for
+    /// the same database reuse cached `PRAGMA table_info` / `PRAGMA index_list`
+    /// / `sqlite_master` results instead of re-running the scan on every
+    /// construction (issues #1565 / #1701).
     /// </summary>
-    public DbSchemaCache SchemaCache => _schemaCache ??= new DbSchemaCache(_connection);
+    public DbSchemaCache SchemaCache => _schemaCache ??= new DbSchemaCache(_connection, _schemaCacheKey);
 
     /// <summary>
     /// Drop cached schema state so subsequent reads observe DDL that ran
@@ -164,6 +165,8 @@ internal static bool TryValidateExistingCodeIndexDb(
 
     public DbContext(string dbPath)
     {
+        _schemaCacheKey = TryCreateSchemaCacheKey(dbPath);
+
         // Explicit URI form (file:///abs/path?immutable=1 etc.) — the user has opted into
         // a read-only open with SQLite-specific URI flags. Skip the writable-open attempt
         // and all write-oriented pragmas. This is the CLI escape hatch for sandboxes where
@@ -203,7 +206,10 @@ public DbContext(string dbPath)
             // immutable/mode=ro 指定のない file: URI はローカルパスに戻して通常経路で開く。
             var normalized = TryGetLocalPath(dbPath);
             if (normalized != null)
+            {
                 dbPath = normalized;
+                _schemaCacheKey = TryCreateSchemaCacheKey(dbPath);
+            }
         }
 
         // Use SqliteConnectionStringBuilder to prevent connection string injection
@@ -274,6 +280,29 @@ public DbContext(string dbPath)
         _suppressWriteWorkTracking = false;
     }
 
+    private static string? TryCreateSchemaCacheKey(string dbPath)
+    {
+        if (string.IsNullOrWhiteSpace(dbPath))
+            return null;
+
+        if (dbPath.StartsWith("file:", StringComparison.OrdinalIgnoreCase))
+        {
+            var localPath = TryGetLocalPath(dbPath);
+            if (localPath == null)
+                return null;
+            dbPath = localPath;
+        }
+
+        try
+        {
+            return Path.GetFullPath(dbPath);
+        }
+        catch (Exception ex) when (ex is ArgumentException or NotSupportedException or PathTooLongException)
+        {
+            return null;
+        }
+    }
+
     private void WarnIfBatchInProgress()
     {
         var raw = GetMetaString(BatchInProgressMetaKey);

src/CodeIndex/Database/DbSchemaCache.cs

@@ -1,16 +1,17 @@
 using Microsoft.Data.Sqlite;
+using System.Collections.Concurrent;
 
 namespace CodeIndex.Database;
 
 /// <summary>
-/// Connection-scoped cache for SQLite schema discovery (`PRAGMA table_info`,
+/// Process-level cache for SQLite schema discovery (`PRAGMA table_info`,
 /// `PRAGMA index_list`, and `sqlite_master` table existence checks).
 ///
 /// `DbReader` discovered these on every construction, so MCP sessions that
-/// reuse a `DbContext` across tool calls were re-running the same PRAGMAs per
-/// invocation (issue #1565). Caching them at the `DbContext` level pays the
-/// scan once per session and serves subsequent `DbReader` instances from
-/// memory.
+/// create or reuse `DbContext` instances for the same DB path were re-running
+/// the same PRAGMAs per invocation (issues #1565 / #1701). Caching them at a
+/// DB-path level pays the scan once per process and serves subsequent
+/// `DbReader` instances from memory.
 ///
 /// In-process migrations (`InitializeSchema`, `TryMigrateForRead`, `DropAll`)
 /// call <see cref="Refresh"/> directly. To also catch DDL run through a
@@ -23,7 +24,20 @@ namespace CodeIndex.Database;
 /// </summary>
 public sealed class DbSchemaCache
 {
+    private sealed class SharedState
+    {
+        public readonly object Lock = new();
+        public readonly Dictionary<string, HashSet<string>> Columns = new(StringComparer.OrdinalIgnoreCase);
+        public readonly Dictionary<string, HashSet<string>> Indexes = new(StringComparer.OrdinalIgnoreCase);
+        public readonly Dictionary<string, bool> TableExists = new(StringComparer.OrdinalIgnoreCase);
+        public long? LastSchemaVersion;
+        public bool VersionStale;
+    }
+
+    private static readonly ConcurrentDictionary<string, SharedState> SharedStates = new(StringComparer.Ordinal);
+
     private readonly SqliteConnection _connection;
+    private readonly SharedState? _sharedState;
     private readonly object _lock = new();
     private readonly Dictionary<string, HashSet<string>> _columns = new(StringComparer.OrdinalIgnoreCase);
     private readonly Dictionary<string, HashSet<string>> _indexes = new(StringComparer.OrdinalIgnoreCase);
@@ -40,16 +54,29 @@ public sealed class DbSchemaCache
     // この sentinel が立っている間に追加されたエントリは次の成功時に必ず破棄する。
     private bool _versionStale;
 
-    public DbSchemaCache(SqliteConnection connection)
+    public DbSchemaCache(SqliteConnection connection, string? sharedCacheKey = null)
     {
         _connection = connection ?? throw new ArgumentNullException(nameof(connection));
+        if (!string.IsNullOrWhiteSpace(sharedCacheKey))
+            _sharedState = SharedStates.GetOrAdd(sharedCacheKey, static _ => new SharedState());
     }
 
     public HashSet<string> GetColumns(string tableName)
     {
         lock (_lock)
         {
             EnsureFreshUnlocked();
+            if (_sharedState != null)
+            {
+                lock (_sharedState.Lock)
+                {
+                    if (_sharedState.Columns.TryGetValue(tableName, out var sharedCached))
+                        return sharedCached;
+                    var sharedFresh = LoadColumns(_connection, tableName);
+                    _sharedState.Columns[tableName] = sharedFresh;
+                    return sharedFresh;
+                }
+            }
             if (_columns.TryGetValue(tableName, out var cached))
                 return cached;
             var fresh = LoadColumns(_connection, tableName);
@@ -63,6 +90,17 @@ public HashSet<string> GetIndexes(string tableName)
         lock (_lock)
         {
             EnsureFreshUnlocked();
+            if (_sharedState != null)
+            {
+                lock (_sharedState.Lock)
+                {
+                    if (_sharedState.Indexes.TryGetValue(tableName, out var sharedCached))
+                        return sharedCached;
+                    var sharedFresh = LoadIndexes(_connection, tableName, HasTableUnlocked(tableName));
+                    _sharedState.Indexes[tableName] = sharedFresh;
+                    return sharedFresh;
+                }
+            }
             if (_indexes.TryGetValue(tableName, out var cached))
                 return cached;
             var fresh = LoadIndexes(_connection, tableName, HasTableUnlocked(tableName));
@@ -94,6 +132,13 @@ public void Refresh()
 
     private void ClearUnlocked()
     {
+        if (_sharedState != null)
+        {
+            lock (_sharedState.Lock)
+            {
+                ClearSharedUnlocked(_sharedState);
+            }
+        }
         _columns.Clear();
         _indexes.Clear();
         _tableExists.Clear();
@@ -132,10 +177,32 @@ private void EnsureFreshUnlocked()
             // DB mid-DDL and would otherwise get version-stamped as current.
             // PRAGMA schema_version 取得に失敗した場合は安全側に倒し、現エントリを破棄して
             // sentinel を立てる。失敗中に読み込んだ値は次回成功時に必ず破棄される。
-            _columns.Clear();
-            _indexes.Clear();
-            _tableExists.Clear();
-            _versionStale = true;
+            if (_sharedState != null)
+            {
+                lock (_sharedState.Lock)
+                {
+                    _sharedState.Columns.Clear();
+                    _sharedState.Indexes.Clear();
+                    _sharedState.TableExists.Clear();
+                    _sharedState.VersionStale = true;
+                }
+            }
+            else
+            {
+                _columns.Clear();
+                _indexes.Clear();
+                _tableExists.Clear();
+                _versionStale = true;
+            }
+            return;
+        }
+
+        if (_sharedState != null)
+        {
+            lock (_sharedState.Lock)
+            {
+                EnsureSharedFreshUnlocked(_sharedState, current);
+            }
             return;
         }
 
@@ -172,10 +239,35 @@ private void EnsureFreshUnlocked()
     /// failure window. Production code never calls this; only the regression
     /// test for the stale-sentinel path uses it.
     /// </summary>
-    internal void MarkVersionStaleForTest() { lock (_lock) { _versionStale = true; } }
+    internal void MarkVersionStaleForTest()
+    {
+        lock (_lock)
+        {
+            if (_sharedState != null)
+            {
+                lock (_sharedState.Lock)
+                {
+                    _sharedState.VersionStale = true;
+                }
+                return;
+            }
+            _versionStale = true;
+        }
+    }
 
     private bool HasTableUnlocked(string tableName)
     {
+        if (_sharedState != null)
+        {
+            lock (_sharedState.Lock)
+            {
+                if (_sharedState.TableExists.TryGetValue(tableName, out var sharedCached))
+                    return sharedCached;
+                var sharedExists = QueryHasTable(_connection, tableName);
+                _sharedState.TableExists[tableName] = sharedExists;
+                return sharedExists;
+            }
+        }
         if (_tableExists.TryGetValue(tableName, out var cached))
             return cached;
         var exists = QueryHasTable(_connection, tableName);
@@ -218,4 +310,40 @@ internal static bool QueryHasTable(SqliteConnection conn, string tableName)
         cmd.Parameters.AddWithValue("@name", tableName);
         return cmd.ExecuteScalar() != null;
     }
+
+    private static void EnsureSharedFreshUnlocked(SharedState state, long current)
+    {
+        if (state.VersionStale)
+        {
+            state.Columns.Clear();
+            state.Indexes.Clear();
+            state.TableExists.Clear();
+            state.LastSchemaVersion = current;
+            state.VersionStale = false;
+            return;
+        }
+
+        if (state.LastSchemaVersion is null)
+        {
+            state.LastSchemaVersion = current;
+            return;
+        }
+
+        if (state.LastSchemaVersion.Value == current)
+            return;
+
+        state.Columns.Clear();
+        state.Indexes.Clear();
+        state.TableExists.Clear();
+        state.LastSchemaVersion = current;
+    }
+
+    private static void ClearSharedUnlocked(SharedState state)
+    {
+        state.Columns.Clear();
+        state.Indexes.Clear();
+        state.TableExists.Clear();
+        state.LastSchemaVersion = null;
+        state.VersionStale = false;
+    }
 }

tests/CodeIndex.Tests/DbSchemaCacheTests.cs

@@ -4,8 +4,8 @@
 namespace CodeIndex.Tests;
 
 /// <summary>
-/// Tests for the connection-scoped schema cache that backs DbReader's
-/// PRAGMA table_info / PRAGMA index_list / sqlite_master lookups (issue #1565).
+/// Tests for the DB-path-scoped schema cache that backs DbReader's
+/// PRAGMA table_info / PRAGMA index_list / sqlite_master lookups (issues #1565 / #1701).
 /// </summary>
 [Collection("SQLite pool sensitive")]
 public sealed class DbSchemaCacheTests : IDisposable
@@ -129,6 +129,23 @@ public void DbReader_FromDbContext_ReusesSharedSchemaCacheInstances()
         Assert.Same(fromCacheBefore, fromCacheAfter);
     }
 
+    [Fact]
+    public void DbReader_SeparateDbContextsForSamePath_ReuseSharedSchemaCacheInstances()
+    {
+        // A writable open may run SQLite maintenance that changes
+        // schema_version. Once two separately-opened contexts observe the
+        // same generation, they should reuse the process-level snapshot.
+        using var secondDb = new DbContext(_dbPath);
+        _ = new DbReader(secondDb);
+        var secondContextColumns = secondDb.SchemaCache.GetColumns("files");
+
+        using var thirdDb = new DbContext(_dbPath);
+        _ = new DbReader(thirdDb);
+        var thirdContextColumns = thirdDb.SchemaCache.GetColumns("files");
+
+        Assert.Same(secondContextColumns, thirdContextColumns);
+    }
+
     [Fact]
     public void GetColumns_AutoRefreshesWhenSecondConnectionMutatesSchema()
     {