Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: AxeForging/reviewforge@main
continue-on-error: true
with:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
AI_PROVIDER: gemini
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,8 @@ seedstorm gaps \
- **Enum coverage** — every enum value appears at least `--rows` times, independently per column
- **AI enrichment** — Gemini rewrites faker hints for domain-meaningful data; supply `--prompt` for richer context
- **Gap analysis** — `gaps` shows which tables are empty with row counts and FK context; `--fill` seeds only the empty ones
- **Interactive TUI** — wizard for table selection, config, and review before seeding
- **Web UI** — `seedstorm serve` exposes an interactive graph workspace with click-to-select tables, live SSE job logs, multi-DB session switcher, and connection presets in `localStorage`
- **Interactive TUI** — wizard for table selection, global config, per-table row volumes, and review before seeding
- **Web UI** — `seedstorm serve` exposes an interactive graph workspace with click-to-select tables, per-table row overrides, live SSE job logs, multi-DB session switcher, and connection presets in `localStorage`
- **Dry-run** — preview the seed plan and INSERT SQL without touching the database
- **Export** — generate fake data as YAML, JSON, or SQL without a live connection

Expand Down
8 changes: 7 additions & 1 deletion docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,8 @@ seedstorm seed \
--interactive
```

The interactive TUI includes a **Volumes** step after global config. Each selected table starts with the `--rows` value, and you can override individual tables before review, dry-run, or execution.

<img src="gifs/seed-interactive.gif" alt="seed interactive TUI demo" width="720" />

| Flag | Default | Description |
Expand Down Expand Up @@ -168,6 +170,8 @@ seedstorm gaps \
--interactive
```

Interactive gap fill also includes the **Volumes** step, so empty child tables can receive higher or lower row counts than their auto-required parents.

Sample output (gap analysis report):

```
Expand Down Expand Up @@ -214,6 +218,8 @@ seedstorm generate --schema schema.yaml --rows 20 --format yaml
seedstorm generate --schema schema.yaml --interactive
```

In interactive mode, the **Volumes** step can override row counts per selected table while `--rows` remains the default.

<img src="gifs/generate.gif" alt="generate demo" width="720" />

| Flag | Default | Description |
Expand Down Expand Up @@ -253,7 +259,7 @@ SEEDSTORM_ADDR=127.0.0.1:9000 seedstorm serve

What the UI gives you:

- **Workspace** — Cytoscape DAG of every table; click to select, non-nullable parents auto-lock as a dependency closure (mirrors the TUI). Three run modes: **Seed**, **Fill empty**, **Generate (no DB write)**. Live SSE log stream + status pill.
- **Workspace** — Cytoscape DAG of every table; click to select, non-nullable parents auto-lock as a dependency closure (mirrors the TUI). The selected-table panel lets you override row counts per table for **Seed**, **Fill empty**, and workspace **Generate** runs while `Rows` remains the default. Live SSE log stream + status pill.
- **Connection management** — multi-session: hold several DBs open in one browser and switch from a topbar dropdown. Saved connection presets in `localStorage` with optional password (eye-icon reveal, closed by default). Passwords are kept in process memory only on the server.
- **Standalone tools** — `/generate`, `/enrich`, `/export` mirror the CLI commands as forms.

Expand Down
22 changes: 17 additions & 5 deletions internal/faker/faker.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,12 @@ func Generate(s *schema.Schema, sortedTables []string, rows, enumRows int, conn
// Use this when you only want to seed a subset of tables (e.g. empty ones)
// while still being able to resolve FK references to already-populated parents.
func GenerateFiltered(s *schema.Schema, allTables, targetTables []string, rows, enumRows int, conn *sql.DB, dbType string) (map[string][]map[string]interface{}, error) {
return GenerateFilteredWithCounts(s, allTables, targetTables, rows, enumRows, nil, conn, dbType)
}

// GenerateFilteredWithCounts is like GenerateFiltered, but tableRows can
// override the default row count for individual target tables.
func GenerateFilteredWithCounts(s *schema.Schema, allTables, targetTables []string, rows, enumRows int, tableRows map[string]int, conn *sql.DB, dbType string) (map[string][]map[string]interface{}, error) {
data := make(map[string][]map[string]interface{})
generatedPKs := make(map[string][]interface{})

Expand All @@ -46,21 +52,27 @@ func GenerateFiltered(s *schema.Schema, allTables, targetTables []string, rows,
for _, tableName := range sortedTables {
table := s.Tables[tableName]
data[tableName] = nil
tableRowCount := rows
_, hasRowOverride := tableRows[tableName]
if override := tableRows[tableName]; override > 0 {
tableRowCount = override
}

enumCol, enumVals := findEnumColumn(table)

if enumCol != "" && enumRows > 0 {
if enumCol != "" && enumRows > 0 && !hasRowOverride {
if err := generateEnumRows(data, generatedPKs, table, tableName, enumCol, enumVals, enumRows); err != nil {
return nil, fmt.Errorf("table %s: %w", tableName, err)
}
} else {
if err := generateStandardRows(data, generatedPKs, table, tableName, rows); err != nil {
if err := generateStandardRows(data, generatedPKs, table, tableName, tableRowCount); err != nil {
return nil, fmt.Errorf("table %s: %w", tableName, err)
}
// Guarantee every enum value appears at least `rows` times, independently per column.
// Guarantee every enum value appears at least the requested table
// row count, independently per column.
enumCols := findAllEnumColumns(table)
if len(enumCols) > 0 {
if err := topUpEnumCoverage(data, generatedPKs, table, tableName, enumCols, rows); err != nil {
if len(enumCols) > 0 && !hasRowOverride {
if err := topUpEnumCoverage(data, generatedPKs, table, tableName, enumCols, tableRowCount); err != nil {
return nil, fmt.Errorf("table %s enum top-up: %w", tableName, err)
}
}
Expand Down
102 changes: 102 additions & 0 deletions internal/faker/faker_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -732,6 +732,108 @@ func TestGenerate_reproducibleWithSeed(t *testing.T) {
}
}

func TestGenerateFilteredWithCountsUsesGlobalRowsByDefault(t *testing.T) {
s := &schema.Schema{
Tables: map[string]schema.Table{
"users": {
Columns: map[string]schema.Column{
"id": {Type: "integer", PK: true},
"name": {Type: "varchar", Faker: "name"},
},
},
"orders": {
Columns: map[string]schema.Column{
"id": {Type: "integer", PK: true},
"user_id": {Type: "integer", FK: "users.id"},
},
},
},
}

data, err := GenerateFilteredWithCounts(s, []string{"users", "orders"}, []string{"users", "orders"}, 3, 0, nil, nil, "pgx")
if err != nil {
t.Fatalf("GenerateFilteredWithCounts: %v", err)
}
if got := len(data["users"]); got != 3 {
t.Fatalf("users rows = %d, want 3", got)
}
if got := len(data["orders"]); got != 3 {
t.Fatalf("orders rows = %d, want 3", got)
}
}

func TestGenerateFilteredWithCountsOverridesIndividualTables(t *testing.T) {
s := &schema.Schema{
Tables: map[string]schema.Table{
"users": {
Columns: map[string]schema.Column{
"id": {Type: "integer", PK: true},
"name": {Type: "varchar", Faker: "name"},
},
},
"orders": {
Columns: map[string]schema.Column{
"id": {Type: "integer", PK: true},
"user_id": {Type: "integer", FK: "users.id"},
},
},
},
}

data, err := GenerateFilteredWithCounts(s, []string{"users", "orders"}, []string{"users", "orders"}, 2, 0, map[string]int{
"orders": 5,
}, nil, "pgx")
if err != nil {
t.Fatalf("GenerateFilteredWithCounts: %v", err)
}
if got := len(data["users"]); got != 2 {
t.Fatalf("users rows = %d, want 2", got)
}
if got := len(data["orders"]); got != 5 {
t.Fatalf("orders rows = %d, want 5", got)
}
}

func TestGenerateFilteredWithCountsUsesOverrideAsExactEnumTableVolume(t *testing.T) {
s := &schema.Schema{
Tables: map[string]schema.Table{
"tickets": makeEnumTable(map[string][]string{
"status": {"open", "closed"},
}),
},
}

data, err := GenerateFilteredWithCounts(s, []string{"tickets"}, []string{"tickets"}, 1, 0, map[string]int{
"tickets": 3,
}, nil, "pgx")
if err != nil {
t.Fatalf("GenerateFilteredWithCounts: %v", err)
}
if got := len(data["tickets"]); got != 3 {
t.Fatalf("tickets rows = %d, want exact table override 3", got)
}
}

func TestGenerateFilteredWithCountsOverrideWinsOverEnumRows(t *testing.T) {
s := &schema.Schema{
Tables: map[string]schema.Table{
"tickets": makeEnumTable(map[string][]string{
"status": {"open", "closed"},
}),
},
}

data, err := GenerateFilteredWithCounts(s, []string{"tickets"}, []string{"tickets"}, 1, 2, map[string]int{
"tickets": 7,
}, nil, "pgx")
if err != nil {
t.Fatalf("GenerateFilteredWithCounts: %v", err)
}
if got := len(data["tickets"]); got != 7 {
t.Fatalf("tickets rows = %d, want table override 7 instead of enumRows total 4", got)
}
}

func TestGenerate_differentSeedsDifferentOutput(t *testing.T) {
s := &schema.Schema{
Tables: map[string]schema.Table{
Expand Down
4 changes: 2 additions & 2 deletions internal/tui/execute.go
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,7 @@ func startSeed(ctx context.Context, s *seedParams) tea.Cmd {
}
}

data, err := faker.Generate(s.schema, s.tables, s.rows, s.enumRows, conn, s.dbType)
data, err := faker.GenerateFilteredWithCounts(s.schema, s.tables, s.tables, s.rows, s.enumRows, s.tableRows, conn, s.dbType)
if err != nil {
return seedDoneMsg{err: fmt.Errorf("data generation failed: %w", err)}
}
Expand Down Expand Up @@ -317,7 +317,7 @@ func startSeed(ctx context.Context, s *seedParams) tea.Cmd {
// startDryRun returns a tea.Cmd that generates data and builds a summary.
func startDryRun(s *seedParams) tea.Cmd {
return func() tea.Msg {
data, err := faker.Generate(s.schema, s.tables, s.rows, s.enumRows, nil, s.dbType)
data, err := faker.GenerateFilteredWithCounts(s.schema, s.tables, s.tables, s.rows, s.enumRows, s.tableRows, nil, s.dbType)
if err != nil {
return dryRunDoneMsg{err: fmt.Errorf("data generation failed: %w", err)}
}
Expand Down
34 changes: 34 additions & 0 deletions internal/tui/execute_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -452,3 +452,37 @@ func TestStartDryRun_multipleTables(t *testing.T) {
t.Errorf("total = %d, want 6", dm.total)
}
}

func TestStartDryRun_usesPerTableRowOverrides(t *testing.T) {
s := makeSchema(map[string]map[string]schema.Column{
"users": {
"id": {Type: "integer", PK: true},
},
"orders": {
"id": {Type: "integer", PK: true},
"user_id": {Type: "integer", FK: "users.id"},
},
})
params := &seedParams{
schema: s,
tables: []string{"users", "orders"},
rows: 2,
tableRows: map[string]int{"orders": 5},
dbType: "pgx",
}
msg := startDryRun(params)()
dm := msg.(dryRunDoneMsg)
if dm.err != nil {
t.Fatalf("unexpected error: %v", dm.err)
}
if dm.total != 7 {
t.Fatalf("total = %d, want 7", dm.total)
}
got := map[string]int{}
for _, table := range dm.tables {
got[table.name] = table.rows
}
if got["users"] != 2 || got["orders"] != 5 {
t.Fatalf("rows by table = %+v, want users=2 orders=5", got)
}
}
45 changes: 37 additions & 8 deletions internal/tui/gaps.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ type gapsStep int
const (
gapsStepPicker gapsStep = iota
gapsStepConfig
gapsStepRows
gapsStepReview
gapsStepExecute
)
Expand All @@ -39,6 +40,7 @@ type GapsModel struct {

picker tablePickerModel
config configModel
volumes tableRowsModel
review reviewModel
execute executeModel

Expand Down Expand Up @@ -125,6 +127,7 @@ func (m GapsModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
m.width = msg.Width
m.height = msg.Height
m.picker.height = msg.Height
m.volumes.height = msg.Height
case tea.KeyMsg:
if msg.String() == "ctrl+c" {
m.quitting = true
Expand All @@ -137,6 +140,8 @@ func (m GapsModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
return m.updatePicker(msg)
case gapsStepConfig:
return m.updateConfig(msg)
case gapsStepRows:
return m.updateRows(msg)
case gapsStepReview:
return m.updateReview(msg)
case gapsStepExecute:
Expand Down Expand Up @@ -202,12 +207,33 @@ func (m GapsModel) updateConfig(msg tea.Msg) (tea.Model, tea.Cmd) {
}
resolved, _ := ResolveDeps(m.graph, cleanSelected, m.sortedAll)

m.volumes = newTableRows(resolved, m.config.Rows(), nil, m.height)
m.step = gapsStepRows
}
return m, cmd
}

func (m GapsModel) updateRows(msg tea.Msg) (tea.Model, tea.Cmd) {
var cmd tea.Cmd
m.volumes, cmd = m.volumes.Update(msg)

if m.volumes.quitting {
m.quitting = true
return m, tea.Quit
}
if m.volumes.back {
m.volumes.back = false
m.config.done = false
m.step = gapsStepConfig
return m, nil
}
if m.volumes.done {
parents := make(map[string][]string)
for _, t := range resolved {
for _, t := range m.volumes.tables {
parents[t] = m.graph.Parents(t)
}
m.review = newReview(resolved, parents,
m.config.Rows(), m.config.EnumRows(), m.config.BatchSize(), false)
m.review = newReview(m.volumes.tables, parents,
m.config.Rows(), m.config.EnumRows(), m.config.BatchSize(), false, m.volumes.TableRows())
m.step = gapsStepReview
}
return m, cmd
Expand All @@ -223,8 +249,8 @@ func (m GapsModel) updateReview(msg tea.Msg) (tea.Model, tea.Cmd) {
}
if m.review.back {
m.review.back = false
m.config.done = false
m.step = gapsStepConfig
m.volumes.done = false
m.step = gapsStepRows
return m, nil
}
if m.review.done {
Expand All @@ -233,6 +259,7 @@ func (m GapsModel) updateReview(msg tea.Msg) (tea.Model, tea.Cmd) {
tables: m.review.tables,
rows: m.review.rows,
enumRows: m.review.enumRows,
tableRows: m.review.tableRows,
batchSize: m.review.batch,
truncate: false, // gaps never truncates
dbType: m.dbType,
Expand Down Expand Up @@ -261,7 +288,7 @@ func (m GapsModel) updateExecute(msg tea.Msg) (tea.Model, tea.Cmd) {
func (m GapsModel) View() string {
var sb strings.Builder

steps := []string{"Tables", "Config", "Review", "Execute"}
steps := []string{"Tables", "Config", "Volumes", "Review", "Execute"}
var breadcrumbs []string
for i, name := range steps {
if gapsStep(i) == m.step {
Expand All @@ -282,6 +309,8 @@ func (m GapsModel) View() string {
sb.WriteString(m.picker.View())
case gapsStepConfig:
sb.WriteString(m.config.View())
case gapsStepRows:
sb.WriteString(m.volumes.View())
case gapsStepReview:
sb.WriteString(m.review.View())
case gapsStepExecute:
Expand Down Expand Up @@ -310,7 +339,7 @@ func startGapsFill(ctx context.Context, s *seedParams, allSorted []string) tea.C
return seedDoneMsg{err: fmt.Errorf("failed to ping database: %w", err)}
}

data, err := faker.GenerateFiltered(s.schema, allSorted, s.tables, s.rows, s.enumRows, conn, s.dbType)
data, err := faker.GenerateFilteredWithCounts(s.schema, allSorted, s.tables, s.rows, s.enumRows, s.tableRows, conn, s.dbType)
if err != nil {
return seedDoneMsg{err: fmt.Errorf("data generation failed: %w", err)}
}
Expand Down Expand Up @@ -345,7 +374,7 @@ func startGapsFill(ctx context.Context, s *seedParams, allSorted []string) tea.C
// startGapsDryRun generates data for gap tables and returns a preview.
func startGapsDryRun(s *seedParams, allSorted []string) tea.Cmd {
return func() tea.Msg {
data, err := faker.GenerateFiltered(s.schema, allSorted, s.tables, s.rows, s.enumRows, nil, s.dbType)
data, err := faker.GenerateFilteredWithCounts(s.schema, allSorted, s.tables, s.rows, s.enumRows, s.tableRows, nil, s.dbType)
if err != nil {
return dryRunDoneMsg{err: fmt.Errorf("data generation failed: %w", err)}
}
Expand Down
Loading
Loading