Expose status diagnostics and bound hooks

AGENT_GUIDE.md

@@ -131,12 +131,14 @@ CI watching must be bounded. Do not loop indefinitely.
 
 ## Status Contract
 
-- `status --json` and related JSON/MCP payloads currently expose the trust fields documented in `README.md` and `DEVELOPER_GUIDE.md`, including `fold_ready`, `fold_ready_reason`, `graph_table_available`, `issues_table_available`, `file_issues_data_current`, `migration_in_progress`, `sql_graph_contract_ready`, `sql_graph_contract_degraded_reason`, `hotspot_family_ready`, `hotspot_family_degraded_reason`, `language_readiness`, `csharp_symbol_name_ready`, `csharp_metadata_target_ready`, `csharp_metadata_target_degraded_reason`, `indexed_head_commit`, `worktree_head_changed`, `index_writer_version`, `index_newer_than_reader`, `index_newer_than_reader_reason`, `unknown_extension_file_count`, `path_case_sensitive`, `data_dir`, `data_dir_source`, `data_dir_mode`, `mac_profile`, `db_size_bytes`, `wal_size_bytes`, `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`), `symbols_by_language`, `process`, `last_index_run`, `hooks`, MCP-only `mcp_session`, and the `status --check`-only `stale_after_seconds` / `index_age_seconds` threshold audit fields.
+- `status --json` and related JSON/MCP payloads currently expose the trust fields documented in `README.md` and `DEVELOPER_GUIDE.md`, including `fold_ready`, `fold_ready_reason`, `graph_table_available`, `issues_table_available`, `file_issues_data_current`, `migration_in_progress`, `sql_graph_contract_ready`, `sql_graph_contract_degraded_reason`, `hotspot_family_ready`, `hotspot_family_degraded_reason`, `language_readiness`, `csharp_symbol_name_ready`, `csharp_metadata_target_ready`, `csharp_metadata_target_degraded_reason`, `indexed_head_commit`, `worktree_head_changed`, `index_writer_version`, `index_newer_than_reader`, `index_newer_than_reader_reason`, `unknown_extension_file_count`, `unknown_extension_files`, `unknown_extension_files_truncated`, `unknown_extension_file_path_limit`, `extractors`, `path_case_sensitive`, `data_dir`, `data_dir_source`, `data_dir_mode`, `mac_profile`, `db_size_bytes`, `wal_size_bytes`, `db_pragma_settings` (`journal_mode`, `synchronous`, `wal_autocheckpoint`, `page_count`, `freelist_count`, `page_size`), `symbols_by_language`, `process`, `last_index_run`, `hooks`, MCP-only `mcp_session`, and the `status --check`-only `stale_after_seconds` / `index_age_seconds` threshold audit fields.
 - When any readiness field is degraded, the CLI adds `degraded_root_cause`, `degraded_reason`, `recommended_action`, `alternative_action`, and `readiness_degradations[]`. `degraded_root_cause` is the primary stable machine code; `readiness_degradations[]` lists every degraded field with `root_cause`, human reason, and remediation strings.
 - `issues_table_available` reports physical `file_issues` table presence only. `file_issues_data_current` reports whether the table is also stamped current for the active index generation.
 - `index_writer_version` records the `cdidx` version that last wrote to the DB (stamped into `codeindex_meta` as `cdidx_writer_version` on every full scan, update, and MCP index). `index_newer_than_reader` flips to `true` whenever any persisted numeric contract stamp in `codeindex_meta` (or unknown `PRAGMA user_version` readiness bits) exceeds the current binary's compiled maximum, so an older CLI re-opening a DB written by a newer CLI degrades loudly with an audit trail instead of silently dropping back to text-search fallbacks. `index_newer_than_reader_reason` enumerates the specific newer-than-reader stamps.
 - `status` also surfaces indexed-HEAD freshness via `indexed_head_sha`, `indexed_head_branch`, `indexed_head_timestamp`, and `commits_ahead_of_indexed_head`. They are stamped by `cdidx index` on every successful run (full scan AND partial update, distinct from `indexed_head_commit` which is full-scan only) on a best-effort basis (never blocks an otherwise-successful index) and omitted on non-git workspaces, detached HEAD (branch only), or legacy DBs created before this contract.
-- `status` also surfaces unknown-extension scan coverage via `unknown_extension_file_count`, stamped by successful full-repository index runs (`cdidx index <projectPath>` and MCP `index_project`) as the number of non-indexed files with non-empty extensions that do not map to a known language. It is omitted on legacy DBs or before a current full scan has stamped the value.
+- `status` also surfaces unknown-extension scan coverage via `unknown_extension_file_count`, stamped by successful full-repository index runs (`cdidx index <projectPath>` and MCP `index_project`) as the number of non-indexed files with non-empty extensions that do not map to a known language. Current scans also stamp `unknown_extension_files` as a bounded path sample, `unknown_extension_files_truncated` when the count exceeds the sample, and `unknown_extension_file_path_limit` as the cap. These fields are omitted on legacy DBs or before a current full scan has stamped them.
+- `status` also surfaces extractor plugin and pattern-config runtime diagnostics via `extractors`, including loaded counts, skipped file counts, and a bounded diagnostics list for incompatible or malformed plugin/pattern files.
+- `status` also surfaces post-extraction hook callback budgets through `hooks[].callback_budget_ms`. Index runs enforce `CDIDX_HOOK_CALLBACK_BUDGET_MS` (default: 5000 ms) on scratch copies, discard timed-out mutations, and disable timed-out hooks for the remainder of the current run.
 - `status` also surfaces `.cdidx` data-directory permissions via `data_dir_mode` on POSIX filesystems. New `.cdidx` data directories are forced to `0700`; the field is omitted on Windows, URI DBs, or when the directory mode cannot be inspected.
 - `status` also surfaces filesystem case-sensitivity via `path_case_sensitive`, stamped on every successful `cdidx index` run (full scan AND partial update, plus MCP-driven indexes) from `core.ignorecase` + a live filesystem probe. `true` means the volume is case-sensitive (`Foo.cs` and `foo.cs` are distinct); `false` means case-insensitive. Omitted on legacy DBs that predate the stamp. Use it to audit path-equality decisions on case-sensitive APFS, WSL NTFS / dev-drive, and ReFS mounts where the prior OS-keyed heuristic could mis-classify the workspace (#1546).
 - `status` also surfaces Linux mandatory-access-control context via `mac_profile` when `/proc/self/attr/current` or `/proc/self/attr/exec` indicates an AppArmor or SELinux profile. It is omitted on non-Linux hosts, unconstrained processes, or unreadable proc attributes (#1768).

README.md

@@ -193,16 +193,23 @@ The documented `status --json` trust contract covers these fields:
 <tr><td><code>language_readiness</code></td><td><code>csharp_symbol_name_ready</code></td><td><code>csharp_metadata_target_ready</code></td><td><code>csharp_metadata_target_degraded_reason</code></td></tr>
 <tr><td><code>indexed_head_commit</code></td><td><code>worktree_head_changed</code></td><td><code>indexed_head_sha</code></td><td><code>indexed_head_branch</code></td></tr>
 <tr><td><code>indexed_head_timestamp</code></td><td><code>commits_ahead_of_indexed_head</code></td><td><code>index_writer_version</code></td><td><code>index_newer_than_reader</code></td></tr>
-<tr><td><code>index_newer_than_reader_reason</code></td><td><code>unknown_extension_file_count</code></td><td><code>path_case_sensitive</code></td><td><code>data_dir</code></td></tr>
-<tr><td><code>data_dir_source</code></td><td><code>data_dir_mode</code></td><td><code>mac_profile</code></td><td><code>db_size_bytes</code></td></tr>
-<tr><td><code>wal_size_bytes</code></td><td><code>db_pragma_settings</code></td><td><code>symbols_by_language</code></td><td><code>process</code></td></tr>
-<tr><td><code>last_index_run</code></td><td><code>hooks</code></td><td><code>stale_after_seconds</code></td><td><code>index_age_seconds</code></td></tr>
-<tr><td><code>degraded_reason</code></td><td><code>recommended_action</code></td><td><code>alternative_action</code></td><td><code>mcp_session</code></td></tr>
+<tr><td><code>index_newer_than_reader_reason</code></td><td><code>unknown_extension_file_count</code></td><td><code>unknown_extension_files</code></td><td><code>unknown_extension_files_truncated</code></td></tr>
+<tr><td><code>unknown_extension_file_path_limit</code></td><td><code>path_case_sensitive</code></td><td><code>data_dir</code></td><td><code>data_dir_source</code></td></tr>
+<tr><td><code>data_dir_mode</code></td><td><code>mac_profile</code></td><td><code>db_size_bytes</code></td><td><code>wal_size_bytes</code></td></tr>
+<tr><td><code>db_pragma_settings</code></td><td><code>symbols_by_language</code></td><td><code>process</code></td><td><code>last_index_run</code></td></tr>
+<tr><td><code>hooks</code></td><td><code>stale_after_seconds</code></td><td><code>index_age_seconds</code></td><td><code>degraded_reason</code></td></tr>
+<tr><td><code>recommended_action</code></td><td><code>alternative_action</code></td><td><code>mcp_session</code></td><td><code>extractors</code></td></tr>
 </tbody>
 </table>
 
 When any readiness field is degraded, `degraded_root_cause` identifies the primary stable code and `readiness_degradations[]` lists every degraded field with `root_cause`, human `degraded_reason`, `recommended_action`, and `alternative_action`. `issues_table_available` reports physical table presence; use `file_issues_data_current` to decide whether `file_issues` rows are current for the index generation.
 
+After a current full-repository scan, `unknown_extension_file_count` reports how many skipped files had unmapped non-empty extensions, while `unknown_extension_files` lists up to `unknown_extension_file_path_limit` paths and `unknown_extension_files_truncated` marks when more paths exist.
+
+`extractors` reports runtime extractor plugin and pattern-config diagnostics, including loaded counts, skipped file counts, and a bounded diagnostics list for load failures.
+
+`hooks[]` includes `callback_budget_ms`. `CDIDX_HOOK_CALLBACK_BUDGET_MS` bounds each post-extraction hook callback in milliseconds (default: 5000); callbacks that exceed the budget emit index warnings, drop timed-out mutations, and disable that hook for the current index run.
+
 For MCP `status`, `mcp_session` is session-scoped diagnostic data rather than persisted index state. It includes `log_level`, `roots`, optional `client_info`, and optional `client_capabilities`.
 
 `process` is captured at status-call time and includes heap, GC collection, and working-set counters. `last_index_run` is persisted by successful CLI and MCP index runs with the run mode, duration, file counts, byte count, row-change counts, and optional peak-memory summary from CLI `--memory-trace`.
@@ -447,16 +454,23 @@ upgrade / downgrade 後はインストール済み補完 script を再生成し
 <tr><td><code>language_readiness</code></td><td><code>csharp_symbol_name_ready</code></td><td><code>csharp_metadata_target_ready</code></td><td><code>csharp_metadata_target_degraded_reason</code></td></tr>
 <tr><td><code>indexed_head_commit</code></td><td><code>worktree_head_changed</code></td><td><code>indexed_head_sha</code></td><td><code>indexed_head_branch</code></td></tr>
 <tr><td><code>indexed_head_timestamp</code></td><td><code>commits_ahead_of_indexed_head</code></td><td><code>index_writer_version</code></td><td><code>index_newer_than_reader</code></td></tr>
-<tr><td><code>index_newer_than_reader_reason</code></td><td><code>unknown_extension_file_count</code></td><td><code>path_case_sensitive</code></td><td><code>data_dir</code></td></tr>
-<tr><td><code>data_dir_source</code></td><td><code>data_dir_mode</code></td><td><code>mac_profile</code></td><td><code>db_size_bytes</code></td></tr>
-<tr><td><code>wal_size_bytes</code></td><td><code>db_pragma_settings</code></td><td><code>symbols_by_language</code></td><td><code>process</code></td></tr>
-<tr><td><code>last_index_run</code></td><td><code>hooks</code></td><td><code>stale_after_seconds</code></td><td><code>index_age_seconds</code></td></tr>
-<tr><td><code>degraded_reason</code></td><td><code>recommended_action</code></td><td><code>alternative_action</code></td><td><code>mcp_session</code></td></tr>
+<tr><td><code>index_newer_than_reader_reason</code></td><td><code>unknown_extension_file_count</code></td><td><code>unknown_extension_files</code></td><td><code>unknown_extension_files_truncated</code></td></tr>
+<tr><td><code>unknown_extension_file_path_limit</code></td><td><code>path_case_sensitive</code></td><td><code>data_dir</code></td><td><code>data_dir_source</code></td></tr>
+<tr><td><code>data_dir_mode</code></td><td><code>mac_profile</code></td><td><code>db_size_bytes</code></td><td><code>wal_size_bytes</code></td></tr>
+<tr><td><code>db_pragma_settings</code></td><td><code>symbols_by_language</code></td><td><code>process</code></td><td><code>last_index_run</code></td></tr>
+<tr><td><code>hooks</code></td><td><code>stale_after_seconds</code></td><td><code>index_age_seconds</code></td><td><code>degraded_reason</code></td></tr>
+<tr><td><code>recommended_action</code></td><td><code>alternative_action</code></td><td><code>mcp_session</code></td><td><code>extractors</code></td></tr>
 </tbody>
 </table>
 
 readiness field のいずれかが degraded の場合、`degraded_root_cause` は primary の安定コードを示し、`readiness_degradations[]` は degraded な各 field と `root_cause`、人間向け `degraded_reason`、`recommended_action`、`alternative_action` を列挙します。`issues_table_available` は物理 table の有無を表し、`file_issues` 行が現在の index generation に対して current かどうかは `file_issues_data_current` を使って判定します。
 
+現行の全体 scan 後、`unknown_extension_file_count` は未知の非空拡張子で skip された件数を返し、`unknown_extension_files` は `unknown_extension_file_path_limit` 件までの path sample、`unknown_extension_files_truncated` は sample より多くの path があることを示します。
+
+`extractors` は extractor plugin と pattern config の runtime 診断で、読み込み済み件数、skip されたファイル数、読み込み失敗の上限付き diagnostics list を含みます。
+
+`hooks[]` は `callback_budget_ms` を含みます。`CDIDX_HOOK_CALLBACK_BUDGET_MS` は post-extraction hook callback ごとの上限ミリ秒を指定します（既定値: 5000）。上限を超えた callback は index warning を出し、timeout した変更を捨て、その index run 中は該当 hook を無効化します。
+
 MCP `status` の `mcp_session` は永続化された index 状態ではなく、セッション単位の診断情報です。`log_level`、`roots`、任意の `client_info`、任意の `client_capabilities` を含みます。
 
 `process` は status 呼び出し時点の heap、GC collection、working-set counters です。`last_index_run` は成功した CLI / MCP index 実行が永続化し、run mode、duration、file counts、byte count、row-change counts、CLI `--memory-trace` 由来の任意の peak-memory summary を含みます。

changelog.d/unreleased/2822.fixed.md

@@ -0,0 +1,18 @@
+---
+category: fixed
+issues:
+  - 2822
+affected:
+  - src/CodeIndex/Database/DbContext.cs
+  - src/CodeIndex/Database/DbReader.FilesStatus.cs
+  - src/CodeIndex/Database/DbWriter.cs
+  - src/CodeIndex/Models/QueryResults.cs
+---
+
+## English
+
+- **Status now exposes unknown-extension path samples (#2822)** — `status --json` and MCP `status` now include `unknown_extension_files`, `unknown_extension_files_truncated`, and `unknown_extension_file_path_limit` alongside the existing count so users can identify skipped files that need a language mapping or ignore rule.
+
+## 日本語
+
+- **status が未知拡張子の path sample を返すようになりました (#2822)** — `status --json` と MCP `status` は既存の件数に加えて `unknown_extension_files`、`unknown_extension_files_truncated`、`unknown_extension_file_path_limit` を返すため、言語マッピングや ignore rule が必要な skip ファイルを特定できます。

changelog.d/unreleased/2865.fixed.md

@@ -0,0 +1,18 @@
+---
+category: fixed
+issues:
+  - 2865
+affected:
+  - src/CodeIndex/Indexer/Extensibility/ExtractorPluginRegistry.cs
+  - src/CodeIndex/Models/QueryResults.cs
+  - src/CodeIndex/Cli/QueryCommandRunner.cs
+  - src/CodeIndex/Mcp/McpToolHandlers.cs
+---
+
+## English
+
+- **Status now reports extractor plugin and pattern diagnostics (#2865)** — `status --json` and MCP `status` include an `extractors` object with plugin/pattern load counts, skipped file counts, and a bounded diagnostic summary for incompatible or malformed extractor files.
+
+## 日本語
+
+- **status が extractor plugin / pattern 診断を返すようになりました (#2865)** — `status --json` と MCP `status` は `extractors` object を含み、plugin / pattern の読み込み件数、skip されたファイル数、互換性のないまたは壊れた extractor ファイルの上限付き diagnostics summary を返します。

changelog.d/unreleased/2877.fixed.md

@@ -0,0 +1,18 @@
+---
+category: fixed
+issues:
+  - 2877
+affected:
+  - src/CodeIndex/Indexer/Hooks/PostExtractionHooks.cs
+  - src/CodeIndex/Models/QueryResults.cs
+  - src/CodeIndex/Cli/QueryCommandRunner.cs
+  - src/CodeIndex/Mcp/McpToolHandlers.cs
+---
+
+## English
+
+- **Post-extraction hooks now have a bounded callback budget (#2877)** — `cdidx index` now enforces `CDIDX_HOOK_CALLBACK_BUDGET_MS` for each post-extraction hook callback, drops timed-out mutations, disables the timed-out hook for the current run, and reports `callback_budget_ms` in `status --json` and MCP `status`.
+
+## 日本語
+
+- **post-extraction hook callback に上限時間を設けました (#2877)** — `cdidx index` は post-extraction hook callback ごとに `CDIDX_HOOK_CALLBACK_BUDGET_MS` を強制し、timeout した変更を破棄し、その run 中は timeout した hook を無効化し、`status --json` と MCP `status` で `callback_budget_ms` を報告します。

src/CodeIndex/Cli/DiffCommandRunner.cs

@@ -228,6 +228,9 @@ OR key LIKE 'symbol_extractor_version_%'
                     OR key LIKE 'metadata_target_version_%'
                     OR key = 'workspace_path_case_sensitive'
                     OR key = 'unknown_extension_file_count'
+                    OR key = 'unknown_extension_file_paths_json'
+                    OR key = 'unknown_extension_files_truncated'
+                    OR key = 'unknown_extension_file_path_limit'
                     OR key = 'cdidx_writer_version'
                 ORDER BY
                     key,

src/CodeIndex/Cli/IndexCommandRunner.FullScan.cs

@@ -1321,9 +1321,7 @@ void StopJsonHeartbeat()
             // metadata ahead of the success markers.
             // no-op full-scan の explicit DB root backfill は readiness stamp 後に限定する。
             WriteProjectRootOnce();
-            writer.SetMeta(
-                DbContext.UnknownExtensionFileCountMetaKey,
-                scanResult.UnknownExtensionFiles.Count.ToString(System.Globalization.CultureInfo.InvariantCulture));
+            writer.WriteUnknownExtensionFileMetadata(scanResult.UnknownExtensionFiles);
             // Persist the current HEAD only after the run is fully successful (errors == 0).
             // We deliberately only stamp on full scans (rebuild or default incremental). Update
             // mode (`--commits` / `--files`) leaves the captured HEAD untouched so the next

src/CodeIndex/Cli/JsonOutputContracts.cs

@@ -2,6 +2,7 @@
 using System.Text.Json;
 using System.Text.Json.Serialization;
 using CodeIndex.Database;
+using CodeIndex.Indexer.Extensibility;
 using CodeIndex.Models;
 
 namespace CodeIndex.Cli;
@@ -434,6 +435,8 @@ internal sealed record VersionInfoJsonResult(
 [JsonSerializable(typeof(SearchResult))]
 [JsonSerializable(typeof(SearchTermOccurrence))]
 [JsonSerializable(typeof(SearchTruncationContext))]
+[JsonSerializable(typeof(ExtractorRegistryDiagnostic))]
+[JsonSerializable(typeof(ExtractorRegistryStatus))]
 [JsonSerializable(typeof(StatusResult))]
 [JsonSerializable(typeof(StatusReadinessDegradation))]
 [JsonSerializable(typeof(StatusDbPragmaSettings))]