Skip to content

Replace PostgreSQL with SQLite#3

Open
james-langridge wants to merge 2 commits intomainfrom
feat/sqlite
Open

Replace PostgreSQL with SQLite#3
james-langridge wants to merge 2 commits intomainfrom
feat/sqlite

Conversation

@james-langridge
Copy link
Owner

Summary

  • Replace PostgreSQL with SQLite (via better-sqlite3) as the database backend
  • Remove the separate db service from Docker Compose — the app is now a single container
  • Simplify deployment: no connection strings, no database passwords, just a file path

SQLite is a better fit for Glimpse since it's single-admin with low write concurrency. This eliminates the need for a separate database service while keeping the same query API surface (sql, query, withTransaction).

Key changes

  • src/lib/db.ts: Full rewrite using better-sqlite3 with WAL mode, foreign keys, busy timeout. Keeps the same async API so callers need minimal changes. Converts $N placeholders to ? and normalizes JS booleans to 0/1.
  • src/db/schema.ts: Clean SQLite DDL (no incremental ALTER TABLE migrations). All columns defined inline in CREATE TABLE statements.
  • Query files (analytics.ts, downloads.ts, links.ts, photos.ts, settings.ts, cleanup.ts, download-token route): PostgreSQL syntax converted to SQLite — NOW()datetime('now'), INTERVALdatetime() modifiers, TRUE/FALSE1/0, removed ::text casts.
  • docker-compose.yml: Removed PostgreSQL service, postgres_data volume, and depends_on. Single app_data volume for both DB and photos.
  • Dockerfile.compose: Added python3 make g++ for native addon compilation.
  • .env.example: DATABASE_URL/POSTGRES_PASSWORD replaced with DATABASE_PATH.
  • README.md: Updated all deployment instructions and tech stack for SQLite.

Test plan

  • npm install — confirm better-sqlite3 installs
  • npm run build — confirm Next.js builds cleanly (verified)
  • npm run dev — confirm schema initializes, creates SQLite file at DATABASE_PATH
  • Upload a photo — exercises insertPhoto, ON CONFLICT deduplication
  • Create a share link with multiple photos — exercises withTransaction
  • Visit share link — exercises getLinkByCode, insertView (RETURNING), analytics
  • Check admin dashboard — exercises getAllPhotosWithStats, getLinkCounts, analytics queries
  • Create download token and download — exercises atomic token consumption in withTransaction
  • Wait for or trigger cleanup — exercises INTERVAL date arithmetic
  • Duplicate a link — exercises withTransaction with client.query
  • docker compose up -d --build — confirm single-container deployment works

Single-file database removes the need for a separate PostgreSQL
service, simplifying deployment to a single container. The app is
single-admin with low write concurrency, making SQLite a better fit.

Core changes:
- Rewrite src/lib/db.ts with better-sqlite3 backend (WAL mode,
  foreign keys, busy timeout), keeping the same async API surface
- Rewrite src/db/schema.ts as clean SQLite DDL
- Convert all PG-specific SQL across query files: NOW() to
  datetime('now'), INTERVAL to datetime() modifiers, TRUE/FALSE
  to 1/0, remove ::text casts
- Simplify docker-compose.yml to single app service
- Update Dockerfile, .env.example, README for SQLite
Copy link
Owner Author

@james-langridge james-langridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overview

  • PR: #3 - Replace PostgreSQL with SQLite
  • Author: james-langridge
  • Branch: feat/sqlite -> main
  • Scope: 16 files changed, +603/-366 lines
  • Status: Ready | CI: No checks configured
  • Linked Issues: None

PR Hygiene

Description Quality

Excellent. The PR description clearly explains the "why" (SQLite is a better fit for single-admin, low write concurrency), provides a thorough summary of key changes per file, and includes a comprehensive manual test plan.

Scope Assessment

Appropriately sized for a database backend swap. All changes are directly related to the migration. The lockfile changes account for most of the diff.

Commit Structure

Single commit ("Replace PostgreSQL with SQLite via better-sqlite3") -- appropriate for an atomic backend swap.


Code Review Findings

[MAJOR] data/ directory not in .gitignore -- SQLite database can be accidentally committed

File: .gitignore

Problem: The default DATABASE_PATH is ./data/glimpse.db. Running the app locally creates data/glimpse.db, data/glimpse.db-shm, and data/glimpse.db-wal. The data/ directory is not gitignored, so these files could be accidentally committed (they already show as untracked: ?? data/). A committed database could contain admin sessions, IP hashes, emails, and download tokens.

Recommendation:
Add /data/ to .gitignore.


[MAJOR] SQLite returns integers (0/1) for boolean columns, but TypeScript interfaces declare boolean

File: src/db/links.ts (line 8, 10), src/db/download-tokens.ts (line 36, 38), src/db/analytics.ts (line 286), src/db/photos.ts (line 130)

Problem: The ShareLink interface declares revoked: boolean and allow_downloads: boolean. The DownloadToken interface declares link_revoked: boolean and allow_downloads: boolean. SQLite stores these as INTEGER (0/1) and better-sqlite3 returns them as JavaScript numbers, not booleans.

Currently this works by coincidence because:

  • if (link.revoked) -- truthiness of 1 is true, 0 is false
  • !link.allow_downloads -- !0 is true, !1 is false
  • No strict equality checks (=== true/false) exist

However:

  1. The types are lying -- typeof link.revoked is "number", not "boolean"
  2. JSON serialization sends 0/1 to the client, where client components also declare boolean types
  3. The toggle in app/admin/links/[id]/page.tsx:204 sends !link.allow_downloads back. Since !0 is true (boolean) and !1 is false (boolean), this works but only by coincidence
  4. Future code using === true or === false will silently fail

Recommendation: Either:

  • (A) Change the TypeScript interfaces to number (0 | 1), or
  • (B) Add a row-mapping step in execQuery or the sql helper that converts known integer-boolean columns, or
  • (C) At minimum, add a comment in db.ts documenting that SQLite returns 0/1 for boolean columns and callers must use truthiness checks

[MINOR] Vercel deployment instructions removed entirely

File: README.md (lines removed from original)

Problem: The Vercel deployment section was deleted. While SQLite doesn't work well on Vercel's serverless platform (ephemeral filesystem), the README still references BLOB_READ_WRITE_TOKEN for Vercel Blob storage in the manual setup section and in the architecture docs. This creates confusion -- Vercel Blob is still supported for photo storage, but there's no guidance on whether Vercel deployment is still viable and under what constraints.

Recommendation: Add a brief note explaining that Vercel deployment is no longer supported due to the SQLite migration (serverless functions don't have persistent filesystem), or clarify the path forward if Vercel support is planned.


[MINOR] withTransaction uses async callbacks but SQLite operations are synchronous

File: src/lib/db.ts (lines 91-109)

Problem: The withTransaction function accepts an async callback and awaits its result. But execQuery is synchronous (it calls better-sqlite3's synchronous API). The async/await layer exists for API compatibility with the old PostgreSQL code, which is fine, but there's a subtle correctness issue: if the callback does any real async work (like file I/O) between BEGIN and COMMIT, other synchronous queries from the same process could interleave and see the uncommitted transaction state. This is acceptable since SQLite is single-connection and the app is single-admin, but worth documenting.

Recommendation: Add a brief comment noting that the transaction assumes no concurrent database access within the async callback, since all operations share a single synchronous connection.


[SUGGESTION] Consider adding a migration note for existing PostgreSQL users

File: README.md

Problem: There's no guidance for users who have an existing PostgreSQL deployment and want to migrate to SQLite. Since the schema is clean and there are no migration scripts, existing data would be lost.

Recommendation: Consider adding a note in the PR description or changelog about this being a breaking change for existing deployments, with guidance to export/reimport data if needed.


Summary

Verdict: COMMENT

Severity Count
Critical 0
Major 2
Minor 2
Suggestion 1

Top Priorities

  1. Add /data/ to .gitignore to prevent accidental commit of database files containing sensitive data
  2. Address the boolean/integer type mismatch between SQLite returns and TypeScript interfaces

What's Good

The migration is clean and thorough. The db.ts abstraction layer is well designed -- it preserves the same API surface (sql, query, withTransaction) so callers need minimal changes. The placeholder conversion ($N -> ?) and boolean normalization are smart compatibility shims. The schema consolidation (removing incremental ALTER TABLE migrations in favor of clean CREATE TABLE DDL) is a significant improvement. The PR description and test plan are exemplary.

- Add /data/ to .gitignore to prevent committing SQLite DB files
- Coerce SQLite integer columns (0/1) back to JS booleans for
  revoked, allow_downloads, link_revoked via BOOLEAN_COLUMNS set
- Add Vercel deprecation note (ephemeral filesystem incompatible
  with SQLite) and migration guide for existing PostgreSQL users
- Document withTransaction async/sync caveat
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant