Skip to content

fix(backend): per-wire-connection sessions for real transaction isolation - #6

Merged
luthermonson merged 1 commit into
mainfrom
fix/per-connection-backend
Jul 10, 2026
Merged

fix(backend): per-wire-connection sessions for real transaction isolation#6
luthermonson merged 1 commit into
mainfrom
fix/per-connection-backend

Conversation

@luthermonson

Copy link
Copy Markdown
Contributor

Stacked on top of perf/translate-cache-and-prepare (which is stacked on compat/session-statements-and-ci). Merge order: compat -> perf -> this. If perf is rebased on main before merge, rebase this too -- the overlap is crates/litewire-backend/src/rusqlite_backend.rs and the per-crate handler.rs/lib.rs pairs, and I already resolved that overlap into this branch.

The bug

crates/litewire-backend/src/rusqlite_backend.rs (pre-PR): one Arc<Mutex<Connection>> per Rusqlite instance, shared across every accepted wire connection. crates/litewire-mysql/src/handler.rs (and its Postgres / TDS peers) tracked in_transaction: bool per wire connection in Rust, but the actual BEGIN ran on the shared rusqlite handle. Consequences:

  • Correctness: client A's BEGIN opened a transaction on the shared handle, so client B's next statement landed inside A's transaction. Either client's COMMIT finalized the other's writes.
  • Concurrency: the process-wide Mutex serialized every backend call. SQLite WAL's concurrent-reader property was defeated by the Rust lock before it ever reached the storage layer.

Additionally, Hrana: HranaClient::execute_pipeline sent baton: None on every call, so all wire clients collided on sqld's default stream and inherited the same transaction-scoping problem.

The fix

Split Backend into a factory for per-session handles:

#[async_trait]
pub trait Backend: Send + Sync + 'static {
    async fn connect(&self) -> Result<Box<dyn BackendConn>, BackendError>;
    // stateless shims kept as defaults for stateless callers:
    async fn query(&self, ...) -> ... { self.connect().await?.query(...).await }
    async fn execute(&self, ...) -> ... { self.connect().await?.execute(...).await }
    async fn describe_columns(&self, ...) -> ... { self.connect().await?.describe_columns(...).await }
}

#[async_trait]
pub trait BackendConn: Send + Sync {
    async fn query(&self, ...) -> ...;
    async fn execute(&self, ...) -> ...;
    async fn describe_columns(&self, ...) -> ... { /* default: LIMIT 0 probe */ }
}

Each LiteWireHandler / PostgresHandler / TDS handle_connection calls Backend::connect() at session start and holds the Box<dyn BackendConn> for the lifetime of the client.

rusqlite backend

  • Each connect() opens a fresh rusqlite::Connection against the same DB file.
  • SQLite's own file locking coordinates writers between connections; WAL readers run concurrently.
  • busy_timeout (default 5000ms, configurable via RusqliteBuilder) handles the SQLITE_BUSY retries that used to be silently masked by the process-wide Mutex.
  • synchronous=NORMAL (WAL-appropriate) is the default; configurable.
  • prepare_cached (from the perf branch) now lives naturally on the per-session RusqliteConn -- its LRU is per-connection in rusqlite anyway.

In-memory database caveat (called out in the module docs)

rusqlite::Connection::open_in_memory() gives every connection a distinct database, which breaks the whole model. The obvious workaround -- shared-cache URI (file:name?mode=memory&cache=shared) -- defeats WAL and reintroduces SQLITE_LOCKED under writers, which is exactly the regression this PR is fixing. I proved that with a failing test before switching approaches.

Chosen fix: Rusqlite::memory() transparently backs itself with a per-process temp file (deleted on Rusqlite drop, via a TempOwner handle that also removes the -wal / -shm sidecars). Consumers keep the exact API. This gives real WAL semantics and real per-connection isolation for tests.

Hrana backend

HranaConn carries the sqld-issued baton across calls so all statements from one wire client stay on the same sqld stream (sqld does its own per-stream transaction isolation). Drop sends a best-effort Close request so sqld reclaims the stream promptly -- the runtime handle guard means this is safe to construct outside a runtime (unit tests).

Frontend changes

  • MySQL (crates/litewire-mysql): LiteWireHandler::new is now async and takes out the session; on failure the accept task logs and returns without serving.
  • Postgres (crates/litewire-postgres): previously one Arc<PostgresHandler> was reused across every pgwire client via the factory. Now each accept spins up its own PostgresHandler inside a per-connection LiteWireHandlerFactory.
  • TDS (crates/litewire-tds): handle_connection opens the session after Login7 and threads &dyn BackendConn through the sub-handlers instead of &SharedBackend.

Tests

Unit (litewire-backend):

  • per_conn_transaction_isolation -- A's uncommitted INSERT invisible to B; B's rogue COMMIT doesn't touch A's tx; after A commits both see the row. This is the exact scenario the old code corrupted.
  • per_conn_rollback_stays_local -- A rolls back, no residue anywhere.
  • per_conn_concurrent_readers -- 16 tasks * 50 SELECTs on distinct sessions complete without SQLITE_BUSY.
  • per_conn_shared_memory_visibility -- two sessions on Rusqlite::memory() see each other's committed schema/rows (proves the temp-file redirection preserves API semantics).
  • per_conn_beats_reopen_on_hot_selects -- 200 point-selects via the stateless shortcut vs one held BackendConn.

E2E (crates/litewire/tests/mysql_e2e.rs, real TCP + mysql_async):

  • two_client_transaction_isolation -- drives the whole stack. B does not see A's uncommitted row.
  • two_client_independent_transactions -- both clients issue BEGIN concurrently. Under the old shared-conn bug B's BEGIN would fail with 'cannot start a transaction within a transaction'; here it succeeds and stays isolated.

Full workspace: 283 tests, 0 failures. Clippy -D warnings clean.

Concurrency measurement

per_conn_beats_reopen_on_hot_selects running in release mode on the dev box:

iters=200 stateless=315.6462ms per_conn=6.1016ms ratio=51.73x

The stateless shortcut (fresh conn per call, mimicking the old post-Mutex-drop hot path) does 200 point-selects in ~316ms; a held BackendConn does the same in ~6ms. That's ~52x -- driven by both the removed re-open cost per call and by prepare_cached LRU hits on the held session. The test asserts only 'per_conn <= stateless * 2' so machine noise doesn't flake CI.

Cross-crate impact (ephpm/ephpm repo)

ephpm-server/src/tracked_backend.rs overrides Backend::query / execute (not connect), so it now transparently opens a throw-away session per call via the trait defaults. That preserves the existing 'wrap for stats' semantics without change. When ephpm wants transaction correctness for wrapped backends, it will need to expose a TrackedBackendConn -- follow-up PR.

What's deliberately not in scope

  • Backend connection pooling (would defeat per-conn transaction isolation).
  • Following sqld base_url redirects on the Hrana side (accepted, not acted on -- noted in the struct).
  • Postgres single-writer-lock retry policy tuning (SQLite's busy_timeout handles this; the wire code just surfaces BackendError).

@luthermonson
luthermonson force-pushed the perf/translate-cache-and-prepare branch from 00e3997 to c00b26d Compare July 10, 2026 04:15
…tion

Before this change every wire frontend shared a single Arc<dyn Backend>
whose only handle was one rusqlite::Connection (or one un-batoned Hrana
stream). Two consequences:

1. Correctness: client A's BEGIN was visible to every other client;
   client B's statements landed inside A's open transaction; either
   client's COMMIT finalized both. The Mutex serialized calls but did
   nothing to isolate per-session transaction state -- that's a
   property of the *underlying connection*, not the Rust lock.
2. Performance: WAL's concurrent-reader model was defeated by the
   process-wide Mutex on the shared Connection.

Fix: split Backend into a factory that hands out per-session
BackendConn instances. Each MySQL / Postgres / TDS handler calls
Backend::connect() at session start and drops the conn on disconnect.
Backend::query/execute/describe_columns stay on the trait as
stateless shims that open-and-drop a fresh conn per call, so callers
like TrackedBackend that don't need transactions keep working with a
one-line diff (none needed, actually -- they use the default impls).

- Rusqlite: one Connection per session against the same DB file; SQLite
  WAL + busy_timeout handles cross-conn coordination without a Rust-side
  lock. prepare_cached (from the perf branch this is stacked on) is
  now naturally per-session, which is where its LRU wants to live.
- Rusqlite in-memory: memory DBs back themselves with an internally-
  managed temp file (deleted on Rusqlite drop). Shared-cache in-memory
  mode was the obvious alternative but it *defeats* WAL and reintroduces
  the LOCKED-under-writer regression this PR is fixing. Documented in
  the module header.
- Hrana: HranaConn carries the sqld-issued baton across calls so all
  statements from one wire client stay on the same sqld stream (sqld's
  own transaction isolation is per-stream). Drop sends a best-effort
  Close so sqld reclaims stream state promptly.
- MySQL / PG / TDS frontends: switched to per-connection factory
  pattern. Postgres previously reused one Arc<PostgresHandler> across
  all pgwire clients via the factory; that's fixed too.

Isolation tests:

- per_conn_transaction_isolation (rusqlite unit): A's uncommitted
  INSERT is invisible to B; B's rogue COMMIT doesn't touch A's tx;
  after A commits both see the row. This is the exact scenario the
  old code corrupted.
- per_conn_rollback_stays_local: A's rolled-back tx leaves no residue
  on either A or B.
- per_conn_concurrent_readers: 16 tasks * 50 SELECTs against distinct
  sessions complete without SQLITE_BUSY / SQLITE_LOCKED.
- per_conn_shared_memory_visibility: two sessions on Rusqlite::memory()
  see each other's committed schema and rows -- proves the temp-file
  redirection preserves the API.
- per_conn_beats_reopen_on_hot_selects: measures 200 point-selects
  via the stateless shortcut vs one held BackendConn. On this box
  (release build): stateless=315ms, per_conn=6ms, ratio ~52x. Asserts
  only that per_conn is not slower.
- two_client_transaction_isolation (MySQL e2e via mysql_async):
  drives the whole stack -- two real TCP connections, MySQL wire in,
  SQLite out. B does not see A's uncommitted row; A can still commit.
- two_client_independent_transactions (MySQL e2e): both clients open
  distinct transactions concurrently, which would have failed under
  the shared-conn bug with 'cannot start a transaction within a
  transaction'.

Cross-crate impact:
- BackendConn is a new trait; Backend::connect() is required on new
  Backend impls. Existing callers of Backend::query/execute do not
  need changes.
- ephpm's TrackedBackend (in the ephpm/ephpm repo) only overrides
  Backend::query/execute; it now transparently opens a fresh session
  per stateless call. If ephpm ever wants transaction correctness for
  the wrapped backend it should call TrackedBackend::connect() through
  the trait and wrap the returned BackendConn -- follow-up PR.
@luthermonson
luthermonson force-pushed the fix/per-connection-backend branch from 817f373 to 19093f9 Compare July 10, 2026 04:18
@luthermonson
luthermonson changed the base branch from perf/translate-cache-and-prepare to main July 10, 2026 04:18
@luthermonson
luthermonson merged commit 9c92404 into main Jul 10, 2026
3 checks passed
@luthermonson
luthermonson deleted the fix/per-connection-backend branch July 10, 2026 04:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant