Skip to content

Cap report schema summary work #3284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Widthdom merged 2 commits into from
Jun 5, 2026
Merged

Cap report schema summary work #3284

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3a5e31b
Cap report schema summary work (#3146)
Widthdom Jun 5, 2026
855ac17
Clarify report schema row content wording (#3146)
Widthdom Jun 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions USER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1121,12 +1121,12 @@ cdidx report --output report.tgz
cdidx report --output report.tgz --json
```

`cdidx report --output <path>` packages a redacted `.tar.gz` you can attach to a GitHub issue. The bundle includes the cdidx version, .NET runtime, OS / process architecture, and a `schema.txt` listing each SQLite table with its row count (no user content). It also tails the recent cdidx lifecycle log (`stderr-yyyyMMdd.log`), with the database path, lifecycle-log source directory, `process_path=`, `base_dir=`, `cwd=`, `db=`, `path=`, and `args=` lines replaced by `[redacted]` so local filesystem paths and literal query strings never leave your machine.
`cdidx report --output <path>` packages a redacted `.tar.gz` you can attach to a GitHub issue. The bundle includes the cdidx version, .NET runtime, OS / process architecture, and a `schema.txt` with a capped SQLite table list plus bounded row counts (no table row contents). It also tails the recent cdidx lifecycle log (`stderr-yyyyMMdd.log`), with the database path, lifecycle-log source directory, `process_path=`, `base_dir=`, `cwd=`, `db=`, `path=`, and `args=` lines replaced by `[redacted]` so local filesystem paths and literal query strings never leave your machine.

| Flag | Default | Effect |
|---|---|---|
| `--output <path>` / `-o <path>` | (required) | Destination `.tar.gz`. The directory is created if missing; on POSIX, the archive and tar entries are owner-readable/writable only. |
| `--db <path>` | `.cdidx/codeindex.db` | Override the database whose schema is summarized. If absent, `schema.txt` records that no DB was found. |
| `--db <path>` | `.cdidx/codeindex.db` | Override the database whose schema is summarized. If absent, `schema.txt` records that no DB was found. Schema summaries cap table entries at 64, displayed table names at 96 characters, and row-count scans at 1000 rows per table. |
| `--log-lines <n>` | `200` | How many trailing lifecycle-log lines to include (`0` disables the tail; values above `2000` are clamped). Each log file contributes from a bounded 1,048,576-byte tail window instead of being loaded fully. |
| `--no-log` | | Skip the lifecycle log entirely. |
| `--include-args` | | Keep literal `cwd=` and `args=` values in the log tail (opt-in; share only with trusted recipients). |
Expand Down Expand Up @@ -3330,12 +3330,12 @@ cdidx report --output report.tgz
cdidx report --output report.tgz --json
```

`cdidx report --output <path>` は GitHub Issue に添付できる匿名化済み `.tar.gz` を生成します。バンドルには cdidx のバージョン、.NET ランタイム、OS / プロセスアーキテクチャ、各 SQLite テーブル名と整数の行数のみを記録した `schema.txt`（ユーザコンテンツは含まれません）が入ります。さらに直近のライフサイクルログ（`stderr-yyyyMMdd.log`）の末尾も含まれますが、DB パス、ライフサイクルログの source directory、`process_path=`、`base_dir=`、`cwd=`、`db=`、`path=`、`args=` 行は `[redacted]` に置換されるため、ローカルファイルシステムのパスや具体的なクエリ文字列が端末から外に出ることはありません。
`cdidx report --output <path>` は GitHub Issue に添付できる匿名化済み `.tar.gz` を生成します。バンドルには cdidx のバージョン、.NET ランタイム、OS / プロセスアーキテクチャ、上限付きの SQLite テーブル一覧と bounded な行数を記録した `schema.txt`（table の行内容は含まれません）が入ります。さらに直近のライフサイクルログ（`stderr-yyyyMMdd.log`）の末尾も含まれますが、DB パス、ライフサイクルログの source directory、`process_path=`、`base_dir=`、`cwd=`、`db=`、`path=`、`args=` 行は `[redacted]` に置換されるため、ローカルファイルシステムのパスや具体的なクエリ文字列が端末から外に出ることはありません。

| フラグ | 既定値 | 効果 |
|---|---|---|
| `--output <path>` / `-o <path>` | （必須） | 出力先 `.tar.gz`。親ディレクトリが無ければ作成します。POSIX では archive と tar entry は owner の読み書きのみになります。 |
| `--db <path>` | `.cdidx/codeindex.db` | スキーマ要約対象の DB を上書きします。存在しなければ `schema.txt` に「DB が見つからなかった」旨が記録されます。 |
| `--db <path>` | `.cdidx/codeindex.db` | スキーマ要約対象の DB を上書きします。存在しなければ `schema.txt` に「DB が見つからなかった」旨が記録されます。スキーマ要約は table entry を 64 件、表示 table 名を 96 文字、行数 scan を table ごとに 1000 行までに制限します。 |
| `--log-lines <n>` | `200` | ライフサイクルログ末尾を何行含めるか（`0` で末尾を含めません。`2000` を超える値は clamp されます）。各ログファイルは全体を読み込まず、末尾 1,048,576 byte の範囲から収集します。 |
| `--no-log` | | ライフサイクルログを完全に省略します。 |
| `--include-args` | | ログ末尾の `cwd=` / `args=` 値を伏字化せずそのまま含めます（信頼できる相手にだけ使用してください）。 |
Expand Down
17 changes: 17 additions & 0 deletions changelog.d/unreleased/3146.security.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
category: security
issues:
- 3146
affected:
- src/CodeIndex/Cli/ReportCommandRunner.cs
- tests/CodeIndex.Tests/ReportCommandRunnerTests.cs
- USER_GUIDE.md
---

## English

- **Report schema summaries now bound arbitrary SQLite table work (#3146)** — `cdidx report --db` caps schema table entries, displayed table-name length, and per-table row-count scans so malformed or oversized databases cannot force unbounded schema-summary collection.

## 日本語

- **report の schema summary が任意 SQLite DB で行う table 処理に上限を設けました (#3146)** — `cdidx report --db` は schema table entry、表示 table 名の長さ、table ごとの行数 scan を制限し、壊れた DB や巨大 DB が schema summary 収集を無制限に走らせないようにしました。
48 changes: 39 additions & 9 deletions src/CodeIndex/Cli/ReportCommandRunner.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,12 @@ public static class ReportCommandRunner
internal const int DefaultLogLines = 200;
internal const int MaxLogLines = 2000;
internal const int MaxLogFileTailBytes = 1024 * 1024;
internal const int MaxSchemaTables = 64;
internal const int MaxSchemaTableNameDisplayChars = 96;
internal const int MaxSchemaRowCountScanRows = 1000;
internal const string RedactedPlaceholder = "[redacted]";
internal const UnixFileMode BundleFileMode = UnixFileMode.UserRead | UnixFileMode.UserWrite;
private const string TruncatedTableNameSuffix = "...[truncated]";

public static int Run(string[] cmdArgs, JsonSerializerOptions jsonOptions, string? appVersion = null)
{
Expand Down Expand Up @@ -167,7 +171,7 @@ internal static string BuildReadme(string version, bool includeLog, bool include
sb.AppendLine("- `metadata.json` — version, OS, .NET runtime info (machine-readable).");
sb.AppendLine("- `version.txt` — cdidx version only.");
sb.AppendLine("- `env.txt` — human-readable OS / runtime summary.");
sb.AppendLine("- `schema.txt` — list of SQLite tables and row counts (no user content).");
sb.AppendLine("- `schema.txt` — capped SQLite table list and bounded row counts (no table row contents).");
if (includeLog)
{
sb.AppendLine("- `log/stderr-recent.log` — last N lines of the cdidx lifecycle log");
Expand All @@ -184,7 +188,7 @@ internal static string BuildReadme(string version, bool includeLog, bool include
sb.AppendLine();
sb.AppendLine("- Indexed source content, file paths, query strings, and `args=` lines are not included by default.");
sb.AppendLine("- Path-bearing lifecycle fields such as `process_path=`, `base_dir=`, `cwd=`, `db=`, and `path=` are redacted by default.");
sb.AppendLine("- Schema reporting only emits table names and integer row counts.");
sb.AppendLine($"- Schema reporting emits at most {MaxSchemaTables} table names, capped at {MaxSchemaTableNameDisplayChars} display characters, with row counts bounded at {MaxSchemaRowCountScanRows} scanned rows per table.");
return sb.ToString();
}

Expand All @@ -209,40 +213,66 @@ internal static (string Text, List<ReportSchemaTable> Tables, string? DbPath, bo
var tableNames = new List<string>();
using (var listCmd = connection.CreateCommand())
{
listCmd.CommandText = "SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name";
listCmd.CommandText = $"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' LIMIT {MaxSchemaTables + 1}";
using var reader = listCmd.ExecuteReader();
while (reader.Read())
tableNames.Add(reader.GetString(0));
}

var tableListTruncated = tableNames.Count > MaxSchemaTables;
if (tableListTruncated)
tableNames.RemoveRange(MaxSchemaTables, tableNames.Count - MaxSchemaTables);

foreach (var name in tableNames)
{
long rowCount;
var rowCountTruncated = false;
try
{
using var countCmd = connection.CreateCommand();
countCmd.CommandText = $"SELECT COUNT(*) FROM \"{name.Replace("\"", "\"\"")}\"";
rowCount = Convert.ToInt64(countCmd.ExecuteScalar());
countCmd.CommandText = $"SELECT COUNT(*) FROM (SELECT 1 FROM \"{name.Replace("\"", "\"\"")}\" LIMIT {MaxSchemaRowCountScanRows + 1})";
var cappedCount = Convert.ToInt64(countCmd.ExecuteScalar());
rowCountTruncated = cappedCount > MaxSchemaRowCountScanRows;
rowCount = rowCountTruncated ? MaxSchemaRowCountScanRows : cappedCount;
}
catch (SqliteException)
{
rowCount = -1;
}
tables.Add(new ReportSchemaTable(name, rowCount));
tables.Add(new ReportSchemaTable(FormatSchemaTableName(name), rowCount, rowCountTruncated));
}

var sb = new StringBuilder();
sb.AppendLine($"database: {RedactedPlaceholder}");
sb.AppendLine($"tables : {tables.Count}");
sb.AppendLine(tableListTruncated
? $"tables : {tables.Count} (capped; additional tables omitted)"
: $"tables : {tables.Count}");
sb.AppendLine($"limits : table entries <= {MaxSchemaTables}, table name chars <= {MaxSchemaTableNameDisplayChars}, row count scan rows <= {MaxSchemaRowCountScanRows}");
sb.AppendLine();
sb.AppendLine("name | row_count");
sb.AppendLine("-----|----------");
foreach (var t in tables)
sb.AppendLine($"{t.Name} | {(t.RowCount < 0 ? "(unreadable)" : t.RowCount.ToString())}");
sb.AppendLine($"{t.Name} | {FormatSchemaRowCount(t)}");

return (sb.ToString(), tables, dbPath, true);
}

private static string FormatSchemaTableName(string name)
{
if (name.Length <= MaxSchemaTableNameDisplayChars)
return name;

return name[..(MaxSchemaTableNameDisplayChars - TruncatedTableNameSuffix.Length)] + TruncatedTableNameSuffix;
}

private static string FormatSchemaRowCount(ReportSchemaTable table)
{
if (table.RowCount < 0)
return "(unreadable)";

return table.RowCountTruncated ? $">={table.RowCount}" : table.RowCount.ToString();
}

internal static string BuildRecentLogTail(int maxLines, bool includeArgs, out int linesIncluded)
{
linesIncluded = 0;
Expand Down Expand Up @@ -498,7 +528,7 @@ public void AddText(string name, string content) =>
Files.Add((name, Encoding.UTF8.GetBytes(content)));
}

internal sealed record ReportSchemaTable(string Name, long RowCount);
internal sealed record ReportSchemaTable(string Name, long RowCount, bool RowCountTruncated = false);

internal sealed record ReportMetadata(
string Version,
Expand Down
89 changes: 89 additions & 0 deletions tests/CodeIndex.Tests/ReportCommandRunnerTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,93 @@ public void Run_WithRealDb_SchemaTxtListsTablesAndRowCounts()
}
}

[Fact]
public void BuildSchemaSummary_CapsTableEntries_Issue3146()
{
var workDir = CreateWorkDir();
var dbPath = Path.Combine(workDir, "many-tables.db");
try
{
using (var connection = new SqliteConnection(new SqliteConnectionStringBuilder { DataSource = dbPath }.ConnectionString))
{
connection.Open();
for (var i = 0; i < ReportCommandRunner.MaxSchemaTables + 3; i++)
{
using var cmd = connection.CreateCommand();
cmd.CommandText = $"CREATE TABLE {QuoteIdentifier($"table_{i:D3}")}(value INTEGER)";
cmd.ExecuteNonQuery();
}
}

var (schemaText, tables, _, dbIncluded) = ReportCommandRunner.BuildSchemaSummary(dbPath);

Assert.True(dbIncluded);
Assert.Equal(ReportCommandRunner.MaxSchemaTables, tables.Count);
Assert.Contains($"tables : {ReportCommandRunner.MaxSchemaTables} (capped; additional tables omitted)", schemaText);
Assert.Contains($"limits : table entries <= {ReportCommandRunner.MaxSchemaTables}", schemaText);
Assert.Contains("table_063 | 0", schemaText);
Assert.DoesNotContain("table_064 |", schemaText);
}
finally
{
SqliteConnection.ClearAllPools();
TryDeleteDirectory(workDir);
}
}

[Fact]
public void BuildSchemaSummary_CapsDisplayedTableNamesAndRowCountScans_Issue3146()
{
var workDir = CreateWorkDir();
var dbPath = Path.Combine(workDir, "large-table.db");
var longName = "table_" + new string('a', ReportCommandRunner.MaxSchemaTableNameDisplayChars + 20);
try
{
using (var connection = new SqliteConnection(new SqliteConnectionStringBuilder { DataSource = dbPath }.ConnectionString))
{
connection.Open();
using (var createCmd = connection.CreateCommand())
{
createCmd.CommandText = $"CREATE TABLE {QuoteIdentifier(longName)}(value INTEGER)";
createCmd.ExecuteNonQuery();
}

using var transaction = connection.BeginTransaction();
using var insertCmd = connection.CreateCommand();
insertCmd.Transaction = transaction;
insertCmd.CommandText = $"INSERT INTO {QuoteIdentifier(longName)}(value) VALUES ($value)";
var valueParameter = insertCmd.CreateParameter();
valueParameter.ParameterName = "$value";
insertCmd.Parameters.Add(valueParameter);

for (var i = 0; i < ReportCommandRunner.MaxSchemaRowCountScanRows + 5; i++)
{
valueParameter.Value = i;
insertCmd.ExecuteNonQuery();
}

transaction.Commit();
}

var (schemaText, tables, _, dbIncluded) = ReportCommandRunner.BuildSchemaSummary(dbPath);
var table = Assert.Single(tables);

Assert.True(dbIncluded);
Assert.Equal(ReportCommandRunner.MaxSchemaRowCountScanRows, table.RowCount);
Assert.True(table.RowCountTruncated);
Assert.True(table.Name.Length <= ReportCommandRunner.MaxSchemaTableNameDisplayChars);
Assert.Contains("[truncated]", table.Name);
Assert.Contains($">={ReportCommandRunner.MaxSchemaRowCountScanRows}", schemaText);
Assert.DoesNotContain(longName, schemaText);
Assert.DoesNotContain((ReportCommandRunner.MaxSchemaRowCountScanRows + 5).ToString(), schemaText);
}
finally
{
SqliteConnection.ClearAllPools();
TryDeleteDirectory(workDir);
}
}

[Fact]
public void Run_WithLogDirOverride_IncludesRedactedTail()
{
Expand Down Expand Up @@ -552,6 +639,8 @@ private static string CreateWorkDir()
return path;
}

private static string QuoteIdentifier(string value) => "\"" + value.Replace("\"", "\"\"") + "\"";

private static void TryDeleteDirectory(string path)
{
try
Expand Down
Loading