Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
bd5f2e9
Add dynamic language reference graphs (#4746)
Widthdom Jul 24, 2026
e1bc74e
Harden dynamic reference graph extraction (#4746)
Widthdom Jul 24, 2026
a079695
Tighten dynamic graph lexical handling (#4746)
Widthdom Jul 24, 2026
7d0c7e5
Close dynamic graph review gaps (#4746)
Widthdom Jul 24, 2026
c7a8f7c
Address dynamic graph adversarial findings (#4746)
Widthdom Jul 24, 2026
4eacd13
Refine dynamic reference graph semantics (#4746)
Widthdom Jul 24, 2026
7aed1ce
Close dynamic graph review findings (#4746)
Widthdom Jul 24, 2026
d4b2ff5
Resolve dynamic graph adversarial cases (#4746)
Widthdom Jul 24, 2026
fef710f
Complete dynamic graph edge cases (#4746)
Widthdom Jul 24, 2026
c99a90a
Harden dynamic graph upgrade paths (#4746)
Widthdom Jul 24, 2026
8ddf0e7
Fix dynamic graph lexical edge cases (#4746)
Widthdom Jul 24, 2026
6371c1c
Resolve final dynamic graph review findings (#4746)
Widthdom Jul 24, 2026
4390ea9
Close dynamic graph adversarial gaps (#4746)
Widthdom Jul 24, 2026
d33ee37
Resolve final Prolog and Tcl review cases (#4746)
Widthdom Jul 24, 2026
7055444
Close final dynamic graph review gaps (#4746)
Widthdom Jul 24, 2026
33d6580
Resolve final dynamic graph review findings (#4746)
Widthdom Jul 24, 2026
e8e76f6
Resolve dynamic graph review regressions (#4746)
Widthdom Jul 24, 2026
c1bd36e
Handle Tcl expressions and Crystal regexes (#4746)
Widthdom Jul 24, 2026
cc697c1
Resolve dynamic graph contract review findings (#4746)
Widthdom Jul 24, 2026
3047a3d
Resolve final dynamic graph review gaps (#4746)
Widthdom Jul 24, 2026
61a2012
Resolve dynamic graph adversarial findings (#4746)
Widthdom Jul 24, 2026
c1e4b0f
Fix ambiguous Prolog graph edge cases (#4746)
Widthdom Jul 24, 2026
994c895
Resolve dynamic graph final review findings (#4746)
Widthdom Jul 24, 2026
bdf39dd
Fix Tcl namespace and subcommand graph edges (#4746)
Widthdom Jul 24, 2026
1418bec
Resolve final dynamic graph review findings (#4746)
Widthdom Jul 24, 2026
518595d
Merge origin/main into fix-issue4746 (#4746)
Widthdom Jul 24, 2026
7af6b00
Merge remote-tracking branch 'origin/main' into fix-issue4746
Widthdom Jul 24, 2026
ab3a83e
Fix CI capability fixtures and Prolog scan (#4746)
Widthdom Jul 24, 2026
75425f4
Merge remote-tracking branch 'origin/main' into fix-issue4746
Widthdom Jul 24, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENT_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ CI watching must be bounded. Do not loop indefinitely.
- When any readiness field is degraded, the CLI adds `degraded_root_cause`, `degraded_reason`, `recommended_action`, `alternative_action`, and `readiness_degradations[]`. `degraded_root_cause` is the primary stable machine code; `readiness_degradations[]` lists every degraded field with `root_cause`, human reason, and remediation strings.
- `hotspot_family_degraded_reason` currently uses `hotspot_family_support_not_indexed`, `hotspot_family_metadata_stale`, `hotspot_family_disabled_at_index_time`, `partial_family_key_population`, and `hotspot_family_marker_fingerprint_incomplete`; the incomplete marker fingerprint code means marker traversal hit safety caps and should stay synchronized with README / developer-guide recovery notes.
- `issues_table_available` reports physical `file_issues` table presence only. `file_issues_data_current` reports whether the table is also stamped current for the active index generation.
- `graph_table_available` reports a queryable persisted reference generation, while `graph_data_current`, `reference_graph_complete`, and `index_complete` report current-generation coverage. Reference extraction is bounded at 50,000 lookup symbols, 20,000 lookup lines, 512 names per line, and 20,000 container candidates; `reference_extraction_limits`, `reference_graph_incomplete_reasons`, and `reference_extraction_cap_hits` publish cap state, and `last_index_run.reference_extraction_cap_hits` snapshots it per run. Cap hits persist per file and propagate degraded, non-authoritative absence semantics to callers, callees, deps, and impact. A per-file extraction failure keeps successful graph rows queryable, stamps completeness false with bounded `last_failed_or_partial_index_run.file_errors`, and returns exit `11` unless `index --allow-partial` explicitly opts into exit `0`. While such file failures remain unresolved, a later scoped update automatically uses the normal incremental full-scan path so unrelated targets cannot clear the failure and successful recovery can restore every workspace-wide readiness contract without `--rebuild`.
- `graph_table_available` reports a queryable persisted reference generation, while `graph_data_current`, `reference_graph_complete`, and `index_complete` report current-generation coverage. Reference extraction is bounded at 50,000 lookup symbols, 20,000 lookup lines, 512 names per line, and 20,000 container candidates; `reference_extraction_limits`, `reference_graph_incomplete_reasons`, and `reference_extraction_cap_hits` publish cap state, and `last_index_run.reference_extraction_cap_hits` snapshots it per run. Cap hits persist per file and propagate degraded, non-authoritative absence semantics to callers, callees, deps, and impact. Indexed Crystal, Groovy, Tcl, Prolog, or `ambiguous_pl` rows with a missing or stale extractor stamp add `dynamic_reference_graph_contract_stale` to `reference_graph_incomplete_reasons` and keep graph readiness false until a normal index refresh rewrites them. A per-file extraction failure keeps successful graph rows queryable, stamps completeness false with bounded `last_failed_or_partial_index_run.file_errors`, and returns exit `11` unless `index --allow-partial` explicitly opts into exit `0`. While such file failures remain unresolved, a later scoped update automatically uses the normal incremental full-scan path so unrelated targets cannot clear the failure and successful recovery can restore every workspace-wide readiness contract without `--rebuild`.
- `index_writer_version` records the `cdidx` version that last wrote to the DB (stamped into `codeindex_meta` as `cdidx_writer_version` on every full scan, update, and MCP index). `index_newer_than_reader` flips to `true` whenever any persisted numeric contract stamp in `codeindex_meta` (or unknown `PRAGMA user_version` readiness bits) exceeds the current binary's compiled maximum, so an older CLI re-opening a DB written by a newer CLI degrades loudly with an audit trail instead of silently dropping back to text-search fallbacks. `index_newer_than_reader_reason` enumerates the specific newer-than-reader stamps.
- `status` also surfaces indexed-HEAD freshness via `indexed_head_sha`, `indexed_head_branch`, `indexed_head_timestamp`, `commits_ahead_of_indexed_head`, and the compact `head_freshness` summary. They are stamped by `cdidx index` on every successful run (full scan AND partial update, distinct from `indexed_head_commit` which is full-scan only) on a best-effort basis (never blocks an otherwise-successful index) and omitted on non-git workspaces, detached HEAD (branch only), or legacy DBs created before this contract. `worktree_head_changed` compares runtime HEAD with this latest stamp when available and falls back to `indexed_head_commit` only for legacy DBs. `head_freshness.state=fresh` requires `status --check` to match the workspace, `fresh_but_incomplete` keeps matching-workspace freshness distinct from incomplete extraction coverage, and `state=head_current` only means the runtime HEAD matches the `indexed_head` selected by `indexed_head_source`.
- `status` also surfaces unknown-extension scan coverage via `unknown_extension_file_count`, stamped by successful full-repository index runs (`cdidx index <projectPath>` and MCP `index_project`) as the number of non-indexed files with non-empty extensions that do not map to a known language. Current scans also stamp `unknown_extension_files` as a path sample bounded by `unknown_extension_file_path_limit` items and the string-list decoded-character budget, `unknown_extension_files_truncated` when more paths existed than were emitted for either bound, and `unknown_extension_file_path_limit` as the item cap rather than a guarantee that that many paths are returned. Newer scans also expose `unknown_extension_extension_counts`, `unknown_extension_category_counts`, and `unknown_extension_groups`; groups classify common non-code buckets such as repository metadata, licenses, binary assets, configuration, structural metadata, and language-support candidates, and include `recommended_action` values of `ignore_configuration`, `first_class_structural_extraction`, or `language_support`. These fields are omitted on legacy DBs or before a current full scan has stamped them.
Expand Down
6 changes: 4 additions & 2 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1641,7 +1641,8 @@ Process exit codes are coarse (`0` success including valid zero-row queries, `1`
## Design decisions

- **Language capability patterns remain typed at the integration boundary** — CLI/MCP `languages` rows expose suffix-only `extensions`, literal `exact_filenames`, and `<suffix>`-rendered `filename_prefix_patterns`. `legacy_patterns` preserves the former combined list during deprecation, and `pattern_provenance` identifies built-in, plugin/pattern, and language-map override ownership. Round-trip tests feed every advertised typed pattern back through `FileIndexer.DetectLanguage` (#4617).
- **Ambiguous source extensions stay explicit** — `.m` and `.pl` are not assigned to Objective-C and Perl by default. `FileIndexer` checks an authoritative recognized shebang, then a 64 KiB bounded prefix for strong mutually exclusive Objective-C/MATLAB or Perl/Prolog markers, then at most 256 entries per ancestor directory for conservative project markers. Conflicting or weak evidence is indexed as `ambiguous_m` / `ambiguous_pl`; unresolved `.m` files run the bounded MATLAB and Objective-C symbol/reference paths after a shared position-preserving comment mask, while Prolog deliberately remains without reference/graph support (#4612, #4738).
- **Ambiguous source extensions stay explicit** — `.m` and `.pl` are not assigned to Objective-C and Perl by default. `FileIndexer` checks an authoritative recognized shebang, then a 64 KiB bounded prefix for strong mutually exclusive Objective-C/MATLAB or Perl/Prolog markers, then at most 256 entries per ancestor directory for conservative project markers. Conflicting or weak evidence is indexed as `ambiguous_m` / `ambiguous_pl`; unresolved `.m` files run the bounded MATLAB and Objective-C symbol/reference paths after a shared position-preserving comment mask, while Prolog and `ambiguous_pl` advertise conservative reference/graph support and the ambiguous `.pl` bucket uses union symbol/reference rules without changing content-based classification (#4612, #4738, #4746).
- **Dynamic reference-graph readiness follows extractor contracts** — when indexed Crystal, Groovy, Tcl, Prolog, or `ambiguous_pl` rows have a missing or stale symbol-extractor version stamp, status reports `dynamic_reference_graph_contract_stale` and keeps `reference_graph_complete` / `graph_data_current` false until a normal index refresh rewrites those rows (#4746).
- **Hotspot marker fingerprints share one bounded tree traversal** — full/update CLI and MCP indexing compute C#, VB, F#, and MSBuild marker fingerprints together instead of walking the directory tree once per language. Each distinct marker glob retains the platform filesystem's matching behavior and is enumerated once per visited directory, while child directories are enumerated once; marker sets, budgets, truncation sentinels, and warning order remain isolated per language. The single-language API delegates to the same engine, preserving ignore rules, nested-repository/submodule boundaries, and MCP authorized-read failures.
- **Lock-file dependency graphs model package relationships** — `packages.lock.json`, `package-lock.json`, and `npm-shrinkwrap.json` keep package declarations as symbols, but emit `dependency` references only for explicit parent-package to child-package entries. The parent package is stored as the reference container, so `callers` can identify which package requires a child and `deps` does not infer lock-file-to-lock-file similarity merely because two files contain the same resolved package set (#4409).
- **Dependency-cycle audits separate analysis from display** — CLI `deps --cycles` and MCP `deps` with `cycles=true` analyze a deterministic, path-ordered edge set up to the independent `--graph-budget` / `graphBudget` before computing and stably ranking strongly connected components. `--limit` / `limit` only paginates that ranked SCC set, and opaque cursors are bound to the filters, graph budget, and indexed graph that produced them. Machine-readable responses expose `analysis_complete`, graph edge count/budget, stable ranking mode, authoritative total-cycle status, and continuation metadata; exhausting the graph budget is reported as an explicitly incomplete analysis rather than a complete cycle audit (#4731).
Expand Down Expand Up @@ -4670,7 +4671,8 @@ USER_GUIDEの[終了コード](USER_GUIDE.md#終了コード)セクションを
## 設計判断

- **integration boundary では language capability pattern の型を維持** — CLI/MCP の `languages` 行は suffix のみの `extensions`、literal な `exact_filenames`、`<suffix>` 表記の `filename_prefix_patterns` を公開します。`legacy_patterns` は deprecation 中に従来の combined list を保持し、`pattern_provenance` は built-in、plugin/pattern、language-map override の所有元を示します。round-trip test は広告した全 typed pattern を `FileIndexer.DetectLanguage` に戻して検証します(#4617)。
- **曖昧な source extension は曖昧なまま明示** — `.m` と `.pl` を既定で Objective-C / Perl に割り当てません。`FileIndexer` は authoritative な認識済み shebang、64 KiB 上限 prefix 内の相互排他的で強い Objective-C/MATLAB または Perl/Prolog marker、各 ancestor directory 最大 256 entry の保守的な project marker の順に確認します。競合または弱い証拠は `ambiguous_m` / `ambiguous_pl` として index し、未確定の `.m` は位置を保つ共通コメントマスクの後で上限付きの MATLAB / Objective-C symbol・reference 経路を実行します。Prolog は意図的に reference / graph 未対応のままです(#4612、#4738)。
- **曖昧な source extension は曖昧なまま明示** — `.m` と `.pl` を既定で Objective-C / Perl に割り当てません。`FileIndexer` は authoritative な認識済み shebang、64 KiB 上限 prefix 内の相互排他的で強い Objective-C/MATLAB または Perl/Prolog marker、各 ancestor directory 最大 256 entry の保守的な project marker の順に確認します。競合または弱い証拠は `ambiguous_m` / `ambiguous_pl` として index し、未確定の `.m` は位置を保つ共通コメントマスクの後で上限付きの MATLAB / Objective-C symbol・reference 経路を実行します。一方、Prolog と `ambiguous_pl` は保守的な reference / graph 対応を広告し、曖昧な `.pl` bucket は content-based classification を変えずに symbol / reference rule の和集合を使います(#4612、#4738、#4746)。
- **動的言語の reference-graph readiness は extractor contract に従う** — index 済みの Crystal、Groovy、Tcl、Prolog、`ambiguous_pl` row で symbol-extractor version stamp が欠落または古い場合、status は `dynamic_reference_graph_contract_stale` を報告し、通常の index refresh が対象 row を更新するまで `reference_graph_complete` / `graph_data_current` を false に保ちます(#4746)。
- **hotspot marker fingerprint は上限付きtree traversalを1回共有** — full/update CLIとMCP indexingは、directory treeを言語ごとに歩かず、C#、VB、F#、MSBuildのmarker fingerprintをまとめて計算します。各directoryでは固有marker globごとにplatform filesystemのmatching挙動を保って1回ずつ列挙し、child directoryも1回だけ列挙する一方、marker集合、budget、truncation sentinel、warning順は言語別に分離します。single-language APIも同じengineへ委譲し、ignore rule、nested repository/submodule境界、MCP authorized read failureを維持します。
- **lock file の依存グラフは package 間の関係をモデル化** — `packages.lock.json`、`package-lock.json`、`npm-shrinkwrap.json` は package 宣言を symbol として保持しますが、`dependency` reference は明示された親 package → 子 package の項目だけに出力します。親 package を reference container に保存するため、`callers` はどの package が子 package を必要としているかを特定でき、`deps` は同じ resolved package 集合を持つだけの lock file 同士を類似関係として推論しません(#4409)。
- **依存サイクル監査では解析と表示を分離** — CLI の `deps --cycles` と MCP `deps` の `cycles=true` は、独立した `--graph-budget` / `graphBudget` まで path 順で決定的な edge 集合を解析してから、強連結成分を安定順位付けします。`--limit` / `limit` はその SCC 順位集合をページ分割するだけで、不透明 cursor は生成時の filter、graph budget、indexed graph に結び付けます。machine-readable 応答は `analysis_complete`、graph edge 件数/予算、安定 ranking mode、authoritative な総 cycle 件数かどうか、continuation metadata を公開し、graph budget 枯渇時は完全な cycle 監査を装わず明示的な未完了解析として報告します(#4731)。
Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,11 @@ last successful run in `reference_extraction_cap_hits`. In that state
`reference_graph_complete=false`; CLI/MCP callers, callees, deps, and impact
responses carry the same stable reasons, and a zero result is not an
authoritative absence.
Indexed Crystal, Groovy, Tcl, Prolog, or `ambiguous_pl` files with a missing or
older extractor stamp similarly report
`dynamic_reference_graph_contract_stale` and keep
`reference_graph_complete=false` / `graph_data_current=false` until
`cdidx index <projectPath>` refreshes their graph rows.

Database Unix-mode hardening defaults to `database_permission_policy=best_effort`.
If a filesystem permits SQLite I/O but rejects mode reads or changes, cdidx continues,
Expand Down Expand Up @@ -624,6 +629,10 @@ CLI の `languages --json` / `status --json` と対応する MCP response は
`reference_extraction_cap_hits` に集約します。この状態では
`reference_graph_complete=false` となり、CLI/MCP の callers / callees / deps / impact response にも
同じ stable reason が伝播するため、0件という結果は authoritative な不在を意味しません。
index 済みの Crystal、Groovy、Tcl、Prolog、`ambiguous_pl` file で extractor
stamp が欠落または古い場合も `dynamic_reference_graph_contract_stale` を報告し、
`cdidx index <projectPath>` が graph row を更新するまで
`reference_graph_complete=false` / `graph_data_current=false` を維持します。

database の Unix mode hardening は既定で
`database_permission_policy=best_effort` です。SQLite I/O は可能でも mode の
Expand Down
Loading
Loading