Skip to content

Avoid mixed SQLite access during TUI startup#417

Merged
wesm merged 2 commits into
mainfrom
flossy-pufferfish
Jun 25, 2026
Merged

Avoid mixed SQLite access during TUI startup#417
wesm merged 2 commits into
mainfrom
flossy-pufferfish

Conversation

@wesm

@wesm wesm commented Jun 25, 2026

Copy link
Copy Markdown
Member

Issue #401 exposed a TUI startup crash while msgvault sync was active: the process could rebuild the Parquet cache through DuckDB while also holding go-sqlite3 connections and starting background FTS maintenance. That repeats the same mixed-SQLite-library risk already documented for daemon cache rebuilds, where DuckDB's bundled SQLite should not attach the live database inside a process that also owns normal msgvault SQLite connections.

This changes interactive startup so the automatic cache build runs before the long-lived TUI store is opened, and FTS maintenance starts only after the query engine is initialized. The DuckDB-backed TUI engine also defaults to direct SQLite fallback for detail reads instead of attaching the live database through sqlite_scanner, preserving fast Parquet aggregates without keeping two SQLite implementations attached to the same WAL database in the interactive process.

The regression coverage locks in that TUI DuckDB options keep sqlite_scanner disabled by default. Standard Go formatting, vetting, and full tagged tests pass locally.

Issue #401 shows TUI startup rebuilding the analytics cache while another writer is active, with DuckDB and go-sqlite3 both in the crash stack. The TUI should not overlap its long-lived Store or background FTS maintenance with DuckDB sqlite_scanner access to the same database file.

Run the automatic cache rebuild before opening the TUI Store, start background FTS only after the query engine is initialized, and default the interactive DuckDB engine to the direct SQLite fallback for detail reads. This matches the daemon-side constraint that keeps DuckDB's bundled SQLite from attaching a live msgvault database inside a process that also owns go-sqlite3 connections.

Generated with Codex
Co-authored-by: Codex <codex@openai.com>
@roborev-ci

roborev-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown

roborev: Combined Review (c0d5292)

Medium issue found in the TUI startup/cache ordering; no security-impacting issues were reported.

Medium

  • cmd/msgvault/cmd/tui.go:86 and cmd/msgvault/cmd/tui.go:123
    The new pre-open cache build runs before InitSchema / runStartupMigrations, while the old post-open build remains. On fresh or upgraded databases, the first build can fail on missing migrated columns or write cache state from pre-migration data. After migrations, the second block can either skip because staleness does not track participant-only migrations, leaving stale Parquet, or retry buildCache while s.DB() is open, reintroducing the mixed DuckDB/go-sqlite3 access this change is trying to avoid.
    Fix: Run schema/migrations first on a short-lived store, close it, build the cache once, then reopen for the TUI; alternatively use the existing subprocess cache build path. Remove the duplicate post-open cache build.

Panel: ci_default_security | Synthesis: codex, 9s | Members: codex_default (codex/default, done, 2m14s), codex_security (codex/security, done, 13s) | Total: 2m36s

The first pass for issue #401 moved automatic cache export before the long-lived TUI Store existed, but it also moved the export ahead of InitSchema/runStartupMigrations and left a second post-open build block behind. That could fail on upgraded SQLite databases whose cache-export columns are added by startup migrations, or retry the DuckDB export while the TUI Store was already open.

Keep the mixed-SQLite avoidance invariant by using a short-lived migrated store first, closing it before any cache export, and reopening the migrated store for the interactive TUI. The TUI now has one cache-build point, after migrations and before the long-lived Store is used by the query engine.

Generated with Codex
Co-authored-by: Codex <codex@openai.com>
@wesm

wesm commented Jun 25, 2026

Copy link
Copy Markdown
Member Author

Addressed the review finding in a28897d. TUI startup now opens a short-lived store to run InitSchema/startup migrations, closes it before any DuckDB cache export, performs at most one cache build, then reopens the migrated store for the interactive TUI.

The linked CI job failed while exporting Docker Buildx cache (error writing layer blob: not_found), not in Go validation. Local go fmt ./..., tagged go vet ./..., and make test all pass after this change.

@roborev-ci

roborev-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown

roborev: Combined Review (a28897d)

No issues found.


Panel: ci_default_security | Synthesis: codex | Members: codex_default (codex/default, done, 4m8s), codex_security (codex/security, done, 14s) | Total: 4m22s

@wesm wesm merged commit 94d80de into main Jun 25, 2026
13 checks passed
@wesm wesm deleted the flossy-pufferfish branch June 25, 2026 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant