Harden private writes and database diagnostics

DEVELOPER_GUIDE.md

@@ -48,7 +48,7 @@ Development contracts:
 | Read-only database queries | `cdidx status --db /artifacts/codeindex.db --read-only --json`; `cdidx search AuthService --db /artifacts/codeindex.db --immutable` | Query commands accept `--read-only` (alias `--immutable`) to open an existing CodeIndex database through SQLite's immutable read-only URI mode. Use this for CI artifacts, mounted caches, and sandboxes where creating or updating `codeindex.db-wal` / `codeindex.db-shm` sidecars is not allowed. |
 | Mutating commands | `index`, `backfill-fold`, `optimize`, `vacuum` | These require writable storage and reject read-only database opens. |
 | Reusable index artifact | `cdidx export codeindex.cdidx.zip`; `cdidx import codeindex.cdidx.zip --db <path>`; `cdidx import codeindex.cdidx.zip --dry-run --json` | Run export after indexing and upload the archive. Consumers import before query commands, or use `--dry-run` / `--check` to validate the archive without replacing the destination DB. Use `--prune-paths` when the archive comes from another checkout and the restored DB should advertise the import target project root; imports targeting `.../.cdidx/codeindex.db` use the sibling project directory, while other DB paths fall back to the process current directory. The archive contains `manifest.json` plus `codeindex.db`; the manifest carries bounded summary/readiness metadata including row counts, readiness bits, writer/indexed-head metadata, schema contract stamps, and unknown-extension summary when available. Import validates manifest format, manifest `user_version`, `database_sha256`, present summary counts, and the embedded SQLite file as a CodeIndex database before replacing the destination DB. Import rejects archive `codeindex.db` entries whose compressed or uncompressed metadata exceeds 8 GiB, and the extraction stream is also capped at 8 GiB. |
-| Maintenance checkpoint | `cdidx db checkpoint <name>`; `cdidx db restore <name>` | Checkpoint snapshots `codeindex.db` plus existing WAL/SHM sidecars before risky maintenance. Restore rolls back and keeps pre-restore files under `<db>.restore-backup-<timestamp>/`. Checkpoints live next to the DB under `<db>.checkpoints/<name>/`. `backfill-fold` creates an automatic checkpoint before it mutates rows unless `--no-checkpoint` is passed. |
+| Maintenance checkpoint | `cdidx db checkpoint <name>`; `cdidx db restore <name>`; `cdidx db restore-backups --list|--prune --keep <n>` | Checkpoint snapshots `codeindex.db` plus existing WAL/SHM sidecars before risky maintenance. Checkpoint manifests record only the database file name, not the local absolute DB path. Restore rolls back and keeps pre-restore files under `<db>.restore-backup-<timestamp>/`; use `restore-backups --list` to inspect retained backups and `restore-backups --prune --keep <n>` to apply retention. Checkpoints live next to the DB under `<db>.checkpoints/<name>/`. `backfill-fold` creates an automatic checkpoint before it mutates rows unless `--no-checkpoint` is passed. |
 | Binary compatibility | [COMPATIBILITY.md](COMPATIBILITY.md) | Database compatibility across `cdidx` binary upgrades and downgrades is documented there. Keep that policy updated whenever readiness bits, `codeindex_meta` contract stamps, or rebuild requirements change. |
 | Fold backfill preview and recovery | `backfill-fold --dry-run`; MCP `backfill_fold` with `dry_run: true` or `force: true` | Dry-run previews folded-key rows without mutating the DB or stamping FoldReady. MCP accepts the same preview and can force rewriting all folded keys when an operator needs to recover from suspicious fold metadata or row state even though the stored version/fingerprint appears current. Non-dry-run row rewrites are resumable after interruption: completed row updates remain durable, and final FoldReady metadata is stamped only after verification succeeds. MCP responses include `progress.rows_done`, `progress.rows_total`, and `progress.fraction` so clients can report and retry long backfills. |
 
@@ -59,7 +59,7 @@ Development contracts:
 | `.cdidx/` | Created with mode `0700`. |
 | `codeindex.db` plus WAL/SHM sidecars | Mode `0600` is applied when the files exist. |
 | `suggestions-*.json` suggestion stores | Written atomically with owner-only mode `0600` on POSIX. |
-| Atomic file writes | `AtomicFileWriter` writes to a sibling temp file, applies the requested POSIX mode before replacement, flushes file contents, renames over the target, and fsyncs the parent directory on Unix. If the parent directory flush fails after replacement, the command fails explicitly so callers know the file was replaced but directory durability was not confirmed. Windows skips directory fsync because the helper only promises it on supported Unix platforms. |
+| Atomic file writes | `AtomicFileWriter` writes to a sibling temp file, applies the requested POSIX mode before replacement, flushes file contents, renames over the target, and fsyncs the parent directory on Unix. Callers must use the `Sensitive` write profile for local state, caches, suggestions, checkpoints, and other private payloads; user-requested exports and reports use the default `Public` profile unless their content is explicitly private. If the parent directory flush fails after replacement, the command fails explicitly so callers know the file was replaced but directory durability was not confirmed. Windows skips directory fsync because the helper only promises it on supported Unix platforms. |
 | Index lock metadata sidecars and active workspace `active.json` | Written as owner-only files and read through small bounded buffers so stale or corrupted diagnostics cannot expose local paths more broadly or force unbounded allocation. |
 | Checkpoint roots, snapshot directories, manifest files, copied DB/WAL/SHM snapshots, and restore staging/backup directories | Forced owner-only on POSIX. |
 | `status --json` | Reports `data_dir_mode` and `db_file_mode` when the platform exposes Unix file modes. |
@@ -2257,7 +2257,7 @@ net9 CI lane に合わせる場合は `FRAMEWORK=net9.0 make test` を使いま
 | read-only database query | `cdidx status --db /artifacts/codeindex.db --read-only --json`; `cdidx search AuthService --db /artifacts/codeindex.db --immutable` | query コマンドは `--read-only`（alias: `--immutable`）を受け付け、既存の CodeIndex database を SQLite の immutable read-only URI mode で開けます。CI artifact、mounted cache、`codeindex.db-wal` / `codeindex.db-shm` sidecar を作成・更新できない sandbox で使います。 |
 | 変更系コマンド | `index`、`backfill-fold`、`optimize`、`vacuum` | 書き込み可能な storage を必要とし、read-only database open を拒否します。 |
 | 再利用可能な index artifact | `cdidx export codeindex.cdidx.zip`; `cdidx import codeindex.cdidx.zip --db <path>`; `cdidx import codeindex.cdidx.zip --dry-run --json` | CI job では index 後に export して archive を upload します。利用側は query コマンドの前に import でき、`--dry-run` / `--check` で destination DB を置き換えず archive を検証できます。別 checkout 由来の archive を import 先 project root として扱いたい場合は `--prune-paths` を使います。`.../.cdidx/codeindex.db` を import 先にした場合は sibling の project directory を使い、それ以外の DB path では process current directory に fallback します。archive は `manifest.json` と `codeindex.db` を含み、manifest は row count、readiness bit、writer / indexed-head metadata、schema contract stamp、利用可能な unknown-extension summary などの bounded summary/readiness metadata を持ちます。import は manifest format、manifest `user_version`、`database_sha256`、存在する summary count、embedded SQLite file が CodeIndex database であることを検証してから destination DB を置き換えます。archive の `codeindex.db` entry は compressed / uncompressed metadata と extraction stream の双方で 8 GiB を上限に拒否されます。 |
-| maintenance checkpoint | `cdidx db checkpoint <name>`; `cdidx db restore <name>` | 危険な maintenance の前に `codeindex.db` と既存 WAL/SHM sidecar の filesystem snapshot を作成し、restore で戻します。checkpoint は DB の隣の `<db>.checkpoints/<name>/` に置かれ、restore は pre-restore file を `<db>.restore-backup-<timestamp>/` に保持します。`backfill-fold` は `--no-checkpoint` を渡さない限り、row mutation 前に automatic checkpoint を作ります。 |
+| maintenance checkpoint | `cdidx db checkpoint <name>`; `cdidx db restore <name>`; `cdidx db restore-backups --list|--prune --keep <n>` | 危険な maintenance の前に `codeindex.db` と既存 WAL/SHM sidecar の filesystem snapshot を作成し、restore で戻します。checkpoint manifest は database file name だけを記録し、local absolute DB path は記録しません。checkpoint は DB の隣の `<db>.checkpoints/<name>/` に置かれ、restore は pre-restore file を `<db>.restore-backup-<timestamp>/` に保持します。保持された backup は `restore-backups --list` で確認し、`restore-backups --prune --keep <n>` で retention を適用できます。`backfill-fold` は `--no-checkpoint` を渡さない限り、row mutation 前に automatic checkpoint を作ります。 |
 | binary compatibility | [COMPATIBILITY.md](COMPATIBILITY.md) | `cdidx` binary の upgrade / downgrade をまたぐ database compatibility を記載します。readiness bit、`codeindex_meta` contract stamp、rebuild requirement を変える場合は、この policy も更新してください。 |
 | Fold backfill の preview / recovery | `backfill-fold --dry-run`; MCP `backfill_fold` の `dry_run: true` または `force: true` | dry-run は DB を変更せず FoldReady stamp も書かずに、rewrite 対象の folded-key row をプレビューします。MCP も同じ preview を受け付け、stored version / fingerprint が current に見える場合でも suspicious な fold metadata や row state を復旧するため `force: true` を受け付けます。non-dry-run rewrite は中断後に resume でき、完了済み row update は durable に残り、最終 FoldReady metadata は verification 成功後にだけ stamp されます。MCP response は `progress.rows_done`、`progress.rows_total`、`progress.fraction` を含みます。 |
 
@@ -2268,7 +2268,7 @@ net9 CI lane に合わせる場合は `FRAMEWORK=net9.0 make test` を使いま
 | `.cdidx/` | mode `0700` で作成。 |
 | `codeindex.db` と WAL/SHM sidecar | ファイルが存在する場合は mode `0600` を適用。 |
 | `suggestions-*.json` suggestion store | POSIX では owner-only の mode `0600` で atomic write します。 |
-| atomic file write | `AtomicFileWriter` は sibling temp file に書き込み、要求された POSIX mode を置換前に適用し、file content を flush してから target へ rename し、Unix では parent directory を fsync します。置換後に parent directory flush が失敗した場合、file は置換済みだが directory durability を確認できていないことが caller に分かるよう command は明示的に失敗します。Windows では、この helper の directory fsync 保証は supported Unix platform に限定されるため skip します。 |
+| atomic file write | `AtomicFileWriter` は sibling temp file に書き込み、要求された POSIX mode を置換前に適用し、file content を flush してから target へ rename し、Unix では parent directory を fsync します。local state、cache、suggestion、checkpoint など private payload には `Sensitive` write profile を使い、user-requested export や report は内容が明示的に private でない限り既定の `Public` profile を使います。置換後に parent directory flush が失敗した場合、file は置換済みだが directory durability を確認できていないことが caller に分かるよう command は明示的に失敗します。Windows では、この helper の directory fsync 保証は supported Unix platform に限定されるため skip します。 |
 | index lock metadata sidecar と active workspace の `active.json` | owner-only file として書き、stale / corrupt diagnostic が local path を広く漏らしたり unbounded allocation を強制したりしないよう小さな bounded buffer で読みます。 |
 | database checkpoint root、snapshot directory、manifest file、copy された DB/WAL/SHM snapshot、restore staging/backup directory | POSIX では owner-only に固定。 |
 | `status --json` | platform が Unix file mode を公開する場合、`data_dir_mode` と `db_file_mode` を報告。 |

changelog.d/unreleased/3688.security.md

@@ -0,0 +1,21 @@
+---
+category: security
+issues:
+  - 3688
+affected:
+  - src/CodeIndex/Cli/AtomicFileWriter.cs
+  - src/CodeIndex/Cli/DataDirectorySecurity.cs
+  - src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs
+  - src/CodeIndex/Cli/SuggestionStore.cs
+  - src/CodeIndex/Cli/UpdateChecker.cs
+  - tests/CodeIndex.Tests/AtomicFileWriterTests.cs
+  - DEVELOPER_GUIDE.md
+---
+
+## English
+
+- **Sensitive atomic writes now use an explicit private write profile (#3688)** — local state JSON files such as scan checkpoints, update caches, and suggestion stores are written through a sensitive profile that creates POSIX temp files as owner-only before the atomic replacement and still reports parent-directory durability failures.
+
+## 日本語
+
+- **sensitive な atomic write が明示的な private write profile を使うようになりました (#3688)** — scan checkpoint、update cache、suggestion store などの local state JSON は sensitive profile 経由で書き込まれ、atomic replace 前の POSIX temp file 生成時点から owner-only になり、parent directory durability の失敗も引き続き報告します。

changelog.d/unreleased/3725.security.md

@@ -0,0 +1,18 @@
+---
+category: security
+issues:
+  - 3725
+affected:
+  - src/CodeIndex/Cli/GlobalToolLog.cs
+  - src/CodeIndex/Database/DbDebug.cs
+  - tests/CodeIndex.Tests/GlobalToolLogTests.cs
+  - tests/CodeIndex.Tests/DbDebugTests.cs
+---
+
+## English
+
+- **Persistent logs and DB debug dumps now bound exception chains (#3725)** — exception-chain diagnostics are capped and truncated with an explicit marker while continuing to classify messages instead of persisting raw paths, SQL, or secret-like values.
+
+## 日本語
+
+- **persistent log と DB debug dump の exception chain に上限を設けました (#3725)** — exception-chain diagnostic は明示的な marker 付きで truncation され、raw path、SQL、secret 風の値を保存せず分類名として記録する動作を維持します。

changelog.d/unreleased/3833.security.md

@@ -0,0 +1,19 @@
+---
+category: security
+issues:
+  - 3833
+affected:
+  - src/CodeIndex/Cli/DbCommandRunner.cs
+  - src/CodeIndex/Cli/JsonOutputContracts.cs
+  - src/CodeIndex/Cli/ConsoleUi.cs
+  - tests/CodeIndex.Tests/DbCommandRunnerTests.cs
+  - DEVELOPER_GUIDE.md
+---
+
+## English
+
+- **Database checkpoint and restore diagnostics are now privacy-hardened (#3833)** — checkpoint manifests omit absolute DB paths, checkpoint creation reports recoverable file-list diagnostics without failing success responses, restore errors are sanitized, rollback failures are exposed in JSON, and retained restore backups can now be listed or pruned.
+
+## 日本語
+
+- **database checkpoint / restore diagnostic の privacy hardening を行いました (#3833)** — checkpoint manifest は absolute DB path を記録せず、checkpoint 作成後の file list 失敗は成功レスポンスを落とさず recoverable diagnostic として返し、restore error は sanitized され、rollback failure は JSON に構造化され、保持された restore backup は list / prune できるようになりました。

src/CodeIndex/Cli/AtomicFileWriter.cs

@@ -9,6 +9,12 @@ internal static class AtomicFileWriter
 {
     internal static Action<string>? FlushParentDirectoryForTesting { get; set; }
 
+    public enum WriteProfile
+    {
+        Public,
+        Sensitive,
+    }
+
     public static void WriteText(string path, string contents, Encoding encoding, Action<string>? applyFileMode = null)
     {
         Write(
@@ -22,12 +28,45 @@ public static void WriteText(string path, string contents, Encoding encoding, Ac
             applyFileMode);
     }
 
+    public static void WriteText(string path, string contents, Encoding encoding, WriteProfile profile)
+    {
+        Write(
+            path,
+            stream =>
+            {
+                using var writer = new StreamWriter(stream, encoding, bufferSize: 1024, leaveOpen: true);
+                writer.Write(contents);
+                writer.Flush();
+            },
+            profile);
+    }
+
     public static void WriteJson<T>(string path, T value, JsonSerializerOptions? options = null, Action<string>? applyFileMode = null)
     {
         Write(path, stream => JsonSerializer.Serialize(stream, value, options), applyFileMode);
     }
 
+    public static void WriteJson<T>(string path, T value, JsonSerializerOptions? options, WriteProfile profile)
+    {
+        Write(path, stream => JsonSerializer.Serialize(stream, value, options), profile);
+    }
+
+    public static void WriteJson<T>(string path, T value, WriteProfile profile)
+    {
+        WriteJson(path, value, options: null, profile);
+    }
+
     public static void Write(string path, Action<Stream> writeContents, Action<string>? applyFileMode = null)
+        => WriteCore(path, writeContents, applyFileMode, WriteProfile.Public);
+
+    public static void Write(string path, Action<Stream> writeContents, WriteProfile profile)
+        => WriteCore(path, writeContents, ResolveProfileModeCallback(profile), profile);
+
+    private static void WriteCore(
+        string path,
+        Action<Stream> writeContents,
+        Action<string>? applyFileMode,
+        WriteProfile profile)
     {
         ArgumentNullException.ThrowIfNull(writeContents);
 
@@ -38,7 +77,7 @@ public static void Write(string path, Action<Stream> writeContents, Action<strin
 
         try
         {
-            using (var stream = new FileStream(ioTempPath, FileMode.CreateNew, FileAccess.Write, FileShare.None))
+            using (var stream = CreateTempFile(ioTempPath, profile))
             {
                 applyFileMode?.Invoke(ioTempPath);
                 writeContents(stream);
@@ -57,6 +96,25 @@ public static void Write(string path, Action<Stream> writeContents, Action<strin
         }
     }
 
+    private static FileStream CreateTempFile(string path, WriteProfile profile)
+    {
+        if (profile != WriteProfile.Sensitive || OperatingSystem.IsWindows())
+            return new FileStream(path, FileMode.CreateNew, FileAccess.Write, FileShare.None);
+
+        return new FileStream(
+            path,
+            new FileStreamOptions
+            {
+                Mode = FileMode.CreateNew,
+                Access = FileAccess.Write,
+                Share = FileShare.None,
+                UnixCreateMode = DataDirectorySecurity.PrivateFileMode,
+            });
+    }
+
+    private static Action<string>? ResolveProfileModeCallback(WriteProfile profile)
+        => profile == WriteProfile.Sensitive ? DataDirectorySecurity.ApplyPrivateFileMode : null;
+
     private static void FlushParentDirectory(string path)
     {
         var directory = Path.GetDirectoryName(Path.GetFullPath(path));