Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 20 additions & 6 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -465,8 +465,8 @@ Operators can override the defaults with environment variables:

| Variable | Default | Meaning |
|---|---:|---|
| `CDIDX_SQLITE_CACHE_KB` | `65536` | Positive cache size in KiB; cdidx applies it as a negative SQLite `cache_size` value so SQLite interprets it as KiB. |
| `CDIDX_SQLITE_MMAP_BYTES` | `268435456` | Non-negative memory-map window in bytes on 64-bit processes. Use `0` to disable mmap. |
| `CDIDX_SQLITE_CACHE_KB` | `65536` | Positive cache size in KiB, up to `1048576`; cdidx applies it as a negative SQLite `cache_size` value so SQLite interprets it as KiB. Invalid or oversized values fall back to the default. |
| `CDIDX_SQLITE_MMAP_BYTES` | `268435456` | Non-negative memory-map window in bytes on 64-bit processes, up to `1073741824`. Use `0` to disable mmap. Invalid or oversized values fall back to the default. |

After a successful `cdidx index` run, the writer refreshes SQLite planner statistics so large repositories do not rely on default selectivity estimates for `search`, `references`, `callers`, and related joins. A brand-new index database runs full `ANALYZE` once after the initial population; later successful index runs use SQLite's lighter `PRAGMA optimize`. This maintenance is best-effort and never changes the schema contract.

Expand Down Expand Up @@ -1115,6 +1115,8 @@ The `suggest_improvement` MCP tool allows AI agents to report gaps or errors.

`SuggestionStore` first checks the SHA256 hash, then compares the candidate against the most recent suggestions in the same category and language using normalized-token Jaccard similarity. The default fuzzy threshold is `0.85`; `cdidx mcp --suggestion-dedup-threshold`, `CDIDX_SUGGESTION_DEDUP_THRESHOLD`, or `.cdidxrc.json` `suggestion_dedup_threshold` can override it with a value from `0` to `1`. Fuzzy matches are returned as duplicates before GitHub submission and log the matched hash plus score to stderr for auditability.

Local suggestion retention is bounded by `CDIDX_SUGGESTION_MAX_AGE_DAYS` and `CDIDX_SUGGESTION_MAX_COUNT`, also available as `.cdidxrc.json` `suggestion_max_age_days` and `suggestion_max_count`. The built-in defaults are 365 days and 5000 records; accepted values are capped at 3650 days and 100000 records. Non-positive, non-numeric, overflowing, or larger environment values fall back to the defaults, while larger config-file values are rejected during config validation.

### Local lifecycle fields

Local suggestion records use the `status` lifecycle field instead of a binary submitted flag. New records start as `draft`; successful GitHub submission moves them to `submitted_pending_triage` and stamps `upstream_url`, `upstream_issue_number`, and `last_synced_at` when known. Every GitHub submission attempt also stamps `last_submit_attempt`, increments `submit_attempt_count`, and records `last_submit_error` on failure; success clears the last error. GitHub rate-limit responses also stamp `next_retry_at`, and duplicate unsubmitted suggestions are not retried until that timestamp has passed. The remaining additive states are reserved for follow-up sync/listing flows: `open_in_upstream`, `resolved_in_upstream`, `wont_fix`, `duplicate`, and `superseded`. Older records containing `submitted_to_github` / `github_issue_url` are normalized on read to the new lifecycle fields.
Expand All @@ -1125,7 +1127,7 @@ Local suggestion records use the `status` lifecycle field instead of a binary su

Before creating an upstream Issue, `GitHubIssueReporter` checks whether an Issue with the same SHA256 suggestion hash already exists. It first queries GitHub Search for the hash in issue bodies, then falls back to listing Issues with the existing repository labels cdidx applies (`enhancement` for ordinary suggestions, `bug` for crash/error reports) and matching the hash in each body. The fallback avoids GitHub Search indexing latency, so a retry immediately after a lost create response can still find the just-created Issue and avoid a duplicate POST. Lookup failures remain best-effort: if both checks fail because GitHub is unavailable, the reporter proceeds to the normal create path instead of blocking a legitimate first submission.

The shared GitHub HTTP client uses an explicit 10-second submission timeout by default, configurable with `CDIDX_GITHUB_SUBMIT_TIMEOUT_SECONDS`, and the platform default proxy (`HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY` through .NET default proxy handling). Create failures mention proxy environment variables in their diagnostic hint. `429` responses and `403` responses with `x-ratelimit-remaining: 0` are treated as rate limits; `Retry-After` wins, then `x-ratelimit-reset`, then a one-minute fallback retry window.
The shared GitHub HTTP client uses an explicit 10-second submission timeout by default, configurable up to 300 seconds with `CDIDX_GITHUB_SUBMIT_TIMEOUT_SECONDS`, and the platform default proxy (`HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY` through .NET default proxy handling). Non-positive, non-numeric, and larger timeout values fall back to the 10-second default. Create failures mention proxy environment variables in their diagnostic hint. `429` responses and `403` responses with `x-ratelimit-remaining: 0` are treated as rate limits; `Retry-After` wins, then `x-ratelimit-reset`, then a one-minute fallback retry window.

### What is NOT included in the payload by design

Expand Down Expand Up @@ -1365,7 +1367,12 @@ logger commits to it, so read-only state/cache/runtime mounts fall through to
the next candidate instead of losing the first log write. The file name is
`stderr-YYYYMMDD.log`, timestamps inside the file are ISO-8601 UTC
(`yyyy-MM-ddTHH:mm:ss.fffZ`) using invariant culture, and the logger keeps
only the newest 30 daily files. Repository-local development runs from
only the newest 30 daily files. `CDIDX_LOG_FORMAT` / `--log-format` switch
between text and JSONL, `CDIDX_LOG_RETAIN` / `--log-retain-count` set retained
file count, and `CDIDX_LOG_MAX_SIZE_MB` / `--log-max-size-mb` or
`CDIDX_GLOBAL_TOOL_LOG_MAX_BYTES` set the size-rotation cap. The default size
cap is 50 MiB and accepted values are capped at 1024 MiB / 1 GiB.
Repository-local development runs from
`src/CodeIndex/bin/...` and `tests/.../bin/...` are excluded by default so
ordinary build/test cycles do not accumulate persistent logs. Set
`CDIDX_DISABLE_PERSISTENT_LOG=1` to opt out entirely; the toggle accepts `1`,
Expand Down Expand Up @@ -2740,6 +2747,12 @@ Unlist しても exact version restore は不可能になりません。これ
- attribution メタデータ: `created_by_agent`、`session_id`、`client_version`、`mcp_client_name`、`mcp_client_version`、および任意の `tool_invocation_context`
- SHA256 提案ハッシュ(重複排除用)

### 重複排除とローカル保持

`SuggestionStore` はまず SHA256 ハッシュを確認し、その後、同じ category / language の直近提案と正規化 token の Jaccard 類似度で比較する。fuzzy しきい値の既定は `0.85` で、`cdidx mcp --suggestion-dedup-threshold`、`CDIDX_SUGGESTION_DEDUP_THRESHOLD`、または `.cdidxrc.json` の `suggestion_dedup_threshold` で `0` から `1` の値へ上書きできる。fuzzy match は GitHub 送信前に重複として返され、監査用に一致先 hash と score を stderr に記録する。

ローカル提案の保持は `CDIDX_SUGGESTION_MAX_AGE_DAYS` と `CDIDX_SUGGESTION_MAX_COUNT` で制限され、`.cdidxrc.json` では `suggestion_max_age_days` と `suggestion_max_count` として設定できる。組み込み既定値は 365 日と 5000 件で、受け付ける値の上限は 3650 日と 100000 件。0 以下、数値以外、overflow、または上限を超える環境変数値は既定値へ戻り、上限を超える config-file 値は config validation 時に拒否される。

### ローカルライフサイクルフィールド

ローカルの提案レコードは、送信済みかどうかの二値フラグではなく `status` ライフサイクルフィールドを使う。新規レコードは `draft` で始まり、GitHub への送信が成功すると `submitted_pending_triage` へ移行し、判明している範囲で `upstream_url`、`upstream_issue_number`、`last_synced_at` を記録する。GitHub の rate-limit 応答では `next_retry_at` も記録し、未送信の重複提案はその時刻を過ぎるまで再送しない。残りの追加状態は後続の sync / listing フロー向けに予約されている: `open_in_upstream`、`resolved_in_upstream`、`wont_fix`、`duplicate`、`superseded`。`submitted_to_github` / `github_issue_url` を含む古いレコードは、読み取り時に新しいライフサイクルフィールドへ正規化される。
Expand All @@ -2748,7 +2761,7 @@ Unlist しても exact version restore は不可能になりません。これ

upstream Issue を作成する前に、`GitHubIssueReporter` は同じ SHA256 提案ハッシュを持つ Issue が既に存在するか確認する。まず GitHub Search で Issue 本文内のハッシュを検索し、その後 backstop として `ai-suggestion` Issue を直接一覧取得して各本文内のハッシュを照合する。この fallback により GitHub Search の indexing 遅延を回避できるため、作成レスポンスが失われた直後の再試行でも、作成済み Issue を検出して重複 POST を防げる。lookup 失敗時の扱いは引き続きベストエフォートであり、GitHub 側の障害などで両方の確認に失敗した場合は、正規の初回送信をブロックせず通常の作成経路へ進む。

共有 GitHub HTTP クライアントは 15 秒 timeout と platform default proxy(.NET の既定 proxy 処理を通じた `HTTPS_PROXY`、`HTTP_PROXY`、`ALL_PROXY`、`NO_PROXY`)を使う。作成失敗の診断には proxy 環境変数の確認ヒントを含める。`429` 応答と `x-ratelimit-remaining: 0` 付きの `403` 応答は rate limit として扱い、`Retry-After`、`x-ratelimit-reset`、1 分の fallback retry window の順で再試行時刻を決める。
共有 GitHub HTTP クライアントは既定で 10 秒 timeout と platform default proxy(.NET の既定 proxy 処理を通じた `HTTPS_PROXY`、`HTTP_PROXY`、`ALL_PROXY`、`NO_PROXY`)を使う。`CDIDX_GITHUB_SUBMIT_TIMEOUT_SECONDS` で最大 300 秒まで設定でき、0 以下、数値以外、または上限を超える値は 10 秒の既定値へ戻る。作成失敗の診断には proxy 環境変数の確認ヒントを含める。`429` 応答と `x-ratelimit-remaining: 0` 付きの `403` 応答は rate limit として扱い、`Retry-After`、`x-ratelimit-reset`、1 分の fallback retry window の順で再試行時刻を決める。

### ペイロードに設計上含まれないもの

Expand Down Expand Up @@ -2907,7 +2920,8 @@ Linux では `~/.local/state/cdidx/logs/` を使う。ファイル名はプロ
`--log-retain-count` は保持ファイル数、`CDIDX_LOG_MAX_SIZE_MB` /
`--log-max-size-mb` または `CDIDX_GLOBAL_TOOL_LOG_MAX_BYTES` は日次ファイルの
サイズローテーション上限を指定する。サイズ上限の既定は 50 MiB、保持世代の
既定は新しい 30 ファイルまで。通常の開発/テストサイクルで
既定は新しい 30 ファイルまでで、サイズ上限は最大 1024 MiB / 1 GiB。
通常の開発/テストサイクルで
ワークツリー直下に永続ログが増えないよう、`src/CodeIndex/bin/...`
と `tests/.../bin/...` からのリポジトリ内開発実行は既定で対象外として
いる。完全に無効化したい場合は `CDIDX_DISABLE_PERSISTENT_LOG=1`、
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ Output controls:

| Need | Option |
|---|---|
| Owner-only persistent stderr logs on POSIX | Global tool stderr logs are forced to `0600` permissions on every open, including existing date/process-stamped log files. Use `--log-format text|json`, `--log-retain-count <N>`, `--log-max-size-mb <N>`, `CDIDX_LOG_*`, or `CDIDX_GLOBAL_TOOL_LOG_MAX_BYTES` to make lifecycle logs JSONL-friendly and rotate them for aggregation. |
| Owner-only persistent stderr logs on POSIX | Global tool stderr logs are forced to `0600` permissions on every open, including existing date/process-stamped log files. Use `--log-format text|json`, `--log-retain-count <N>`, `--log-max-size-mb <N>`, `CDIDX_LOG_*`, or `CDIDX_GLOBAL_TOOL_LOG_MAX_BYTES` to make lifecycle logs JSONL-friendly and rotate them for aggregation. Log size caps are limited to 1024 MiB / 1 GiB. |
| Checked-in configuration | Use `.cdidx/config.json` for repository defaults such as `search.limit`, `search.snippet_lines`, and `search.max_line_width`; run `cdidx validate-config` to validate the discovered file and `cdidx config show` to inspect precedence. |
| Workspaces | Use `cdidx.workspace.json` or `.cdidx-workspace.json` to declare monorepo members, `cdidx workspace list` to inspect them, and `cdidx workspace use <name>` / `cdidx workspace current` for a persisted active workspace. |
| ASCII-only terminal output | Use `--ascii`, `CDIDX_ASCII=1`, `NO_UNICODE`, `TERM=dumb`, accessibility env hints, or a non-UTF-8 locale. Spinners use pipe, slash, dash, and backslash frames; progress bars use `#` / `-`; very narrow terminals fall back to percentage-only progress. Use `--no-progress`, `CDIDX_DISABLE_PROGRESS=1`, or `PREFERS_REDUCED_MOTION` to keep static progress text without animation. |
Expand Down
Loading
Loading