Skip to content

perf: LRU translate cache + prepare_cached + kill LIMIT-0 probe + utf8 fix - #5

Merged
luthermonson merged 1 commit into
mainfrom
perf/translate-cache-and-prepare
Jul 10, 2026
Merged

perf: LRU translate cache + prepare_cached + kill LIMIT-0 probe + utf8 fix#5
luthermonson merged 1 commit into
mainfrom
perf/translate-cache-and-prepare

Conversation

@luthermonson

Copy link
Copy Markdown
Contributor

Summary

Builds on PR #4 (compat/session-statements-and-ci) -- merge that first. This PR is a pure perf + correctness follow-up:

  • Bounded LRU cache in front of the sqlparser parse + rewrite pipeline. Measured 87x speedup in debug, 140x in release on a WordPress-shaped query.
  • Switch the rusqlite backend to prepare_cached so identical repeat statements avoid sqlite3_prepare_v2.
  • Enable rusqlite's column_decltype feature so empty-result SELECTs carry real column types on the wire.
  • New additive Backend::describe_columns trait method; MySQL / PG frontends use it to answer COM_STMT_PREPARE / Describe without a LIMIT 0 probe round trip.
  • extract_value on ValueRef::Text now falls back to Value::Blob for invalid UTF-8 instead of lossily replacing bytes with U+FFFD.

Base branch: compat/session-statements-and-ci. Diff shown against that branch; when PR #4 lands, retarget or rebase onto main.

Benchmark

translate_cache_speedup (5000 iterations of a WP-shaped SELECT ... INNER JOIN ... WHERE post_status = ? AND term_taxonomy_id IN (?, ?, ?) ORDER BY post_date DESC LIMIT 10):

profile cold (uncached) warm (cached) speedup
debug 191 us / call 2.2 us / call 87x
release 39 us / call 277 ns / call 140x

Cold path is dominated by sqlparser::Parser::parse_sql + expression tree rewriting; the LRU eliminates it after the first hit. Run yourself: cargo test --release -p litewire-translate translate_cache_speedup -- --nocapture.

Changes

Translate cache (litewire-translate::cache)

  • TranslateCache = parking_lot::Mutex<lru::LruCache<(Dialect, String), Vec<TranslateResult>>>. Default capacity 1024; documented in the module.
  • translate_cached(&cache, sql, dialect) is the new entry point. Errors are not cached; a repeatedly-bad statement re-parses each time (avoids negative caching stampedes).
  • Dialect gets #[derive(Hash)]; TranslateResult gets Clone so the cache can return owned copies.
  • Per-frontend Arc<TranslateCache> created in MysqlFrontend::serve / PostgresFrontend::serve and cloned into every accepted connection.
  • lru = 0.12 added as an explicit dep on litewire-translate (already in the workspace lockfile transitively).

prepare_cached

  • Rusqlite::query() and Rusqlite::execute() both switched to conn.prepare_cached(&sql). rusqlite maintains a per-connection LRU of parsed statements; second-and-later identical SQL avoids sqlite3_prepare_v2 cost entirely. Complements the translate cache: translate cache eliminates parsing on the litewire side, prepare_cached eliminates it on the SQLite side.

Kill the LIMIT 0 probe

  • Enabled rusqlite feature column_decltype in the workspace Cargo.toml. Comment records why: Statement::columns() needs it.
  • New default trait method on Backend:
    async fn describe_columns(&self, sql: &str) -> Result<Vec<Column>, BackendError>
    Default impl falls back to the old LIMIT 0 probe (so third-party backends keep working). rusqlite overrides it to read metadata off the prepared statement without executing.
  • LiteWireHandler::on_prepare: for INSERT/UPDATE/DELETE, skip the probe entirely (they have no result set); for SELECT, call describe_columns.
  • PostgresHandler::probe_columns: describe_columns first; only fall back to LIMIT 1 if any column comes back untyped (e.g. SELECT 1 + 2).

utf8 fallback (extract_value)

  • ValueRef::Text was String::from_utf8_lossy(s).into_owned() -- silently corrupts non-UTF-8 bytes with U+FFFD. Now: std::str::from_utf8 -> Value::Text on success, Value::Blob(bytes) on failure. Wire frontends can then round-trip the raw bytes back to the client unmodified.

Fallout / wire-format regression fix

Enabling column_decltype exposed a latent bug in the MySQL row writer. It was stringifying integers/floats (i.to_string()), which only worked because every column arrived at the wire as MYSQL_TYPE_VAR_STRING (decltype was always None on the old path). With real decltypes flowing, integer columns come out as MYSQL_TYPE_LONGLONG, and opensrv-mysql's binary encoder refuses a String payload for a LONGLONG column ("tried to use [49] as MYSQL_TYPE_LONGLONG" -- caught by prepared_select_with_param). Row writer now hands native i64/f64 to write_col.

Test plan

  • cargo test --workspace -- 281 passing, 0 failed (was 271; added 6 backend tests for describe_columns + invalid UTF-8 blob fallback, and 4 cache-behaviour tests + 1 timing test).
  • cargo clippy --workspace --all-targets -- -D warnings -- clean.
  • cargo fmt --all -- --check -- clean.
  • Micro-benchmark result recorded in the PR body (cargo test --release translate_cache_speedup -- --nocapture).
  • Suggest running the release-mode mysql_e2e on a real MySQL client (PDO end-to-end) after merging -- I ran the in-process mysql_e2e suite which exercises the binary prepared-statement protocol end-to-end and everything passes, but a real pdo_mysql -> litewire link is worth verifying before tagging a release.

@luthermonson
luthermonson changed the base branch from compat/session-statements-and-ci to main July 10, 2026 02:39
…8 fix

Follows compat/session-statements-and-ci; that PR ships the CI + clippy
cleanup and any conflicts should be resolved by merging PR A first.

Translate LRU cache
- New crate::cache module: bounded LRU keyed by (Dialect, SQL text) with
  default capacity 1024 -- enough to hold every distinct prepared
  statement WordPress + Woocommerce issue on a single page, small enough
  that worst-case memory is a few MB.
- lib.rs exposes translate_cached(cache, sql, dialect) alongside
  translate(); errors are not stored so a repeatedly-bad statement always
  re-parses.
- MysqlFrontend and PostgresFrontend each own an Arc<TranslateCache>
  shared across all accepted connections.
- Micro-benchmark test (translate_cache_speedup) times cold vs cached on
  a WordPress-shaped SELECT with joins/ORDER BY/LIMIT/IN(?,?,?):
    debug   : cold=191us/call warm=2.2us/call  =>  87x speedup
    release : cold=39us/call  warm=277ns/call  => 140x speedup

prepare_cached in rusqlite backend
- query()/execute() switched to conn.prepare_cached(), which interns the
  parsed statement in rusqlite's per-connection LRU. Second and later
  identical SQL avoid the sqlite3_prepare_v2 cost entirely.

column_decltype + kill the LIMIT 0 probe
- Enable the rusqlite `column_decltype` feature so Statement::columns()
  exposes decl_type().
- New Backend::describe_columns(sql) trait method with a default LIMIT-0
  probe implementation; rusqlite overrides it to read metadata off the
  prepared statement without executing.
- MySQL on_prepare: skip describe entirely for INSERT/UPDATE/DELETE (they
  have no result set); for SELECTs, use describe_columns instead of the
  old `SELECT ... LIMIT 0` probe.
- PostgreSQL probe_columns: use describe_columns first; only fall back to
  a LIMIT 1 probe if any column is untyped (`SELECT 1 + 2` style).
- Empty-result-set SELECTs now carry real column types on the wire,
  which fixes Laravel Query::exists() and PDO::getColumnMeta().

utf8 fallback in rusqlite backend
- extract_value on ValueRef::Text used to lossy-decode via
  String::from_utf8_lossy, which silently corrupts non-UTF-8 bytes with
  U+FFFD. Invalid UTF-8 now surfaces as Value::Blob instead so the wire
  frontend can round-trip the raw bytes unchanged.

Wire-format regression fix (uncovered by column_decltype flip)
- MySQL row writer previously stringified integers/floats. That worked
  only because every column arrived at the wire as VAR_STRING. With
  real decltypes flowing through, integer columns become MYSQL_TYPE_LONGLONG
  and opensrv-mysql rejects a String payload for a LONGLONG column
  ("tried to use [49] as MYSQL_TYPE_LONGLONG"). Row writer now writes
  Value::Integer/Value::Float natively.

Dialect gains Hash so it can be used as a HashMap/LruCache key.
@luthermonson
luthermonson force-pushed the perf/translate-cache-and-prepare branch from 00e3997 to c00b26d Compare July 10, 2026 04:15
@luthermonson
luthermonson merged commit dba459b into main Jul 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant