perf: LRU translate cache + prepare_cached + kill LIMIT-0 probe + utf8 fix - #5
Merged
Merged
Conversation
luthermonson
changed the base branch from
compat/session-statements-and-ci
to
main
July 10, 2026 02:39
…8 fix
Follows compat/session-statements-and-ci; that PR ships the CI + clippy
cleanup and any conflicts should be resolved by merging PR A first.
Translate LRU cache
- New crate::cache module: bounded LRU keyed by (Dialect, SQL text) with
default capacity 1024 -- enough to hold every distinct prepared
statement WordPress + Woocommerce issue on a single page, small enough
that worst-case memory is a few MB.
- lib.rs exposes translate_cached(cache, sql, dialect) alongside
translate(); errors are not stored so a repeatedly-bad statement always
re-parses.
- MysqlFrontend and PostgresFrontend each own an Arc<TranslateCache>
shared across all accepted connections.
- Micro-benchmark test (translate_cache_speedup) times cold vs cached on
a WordPress-shaped SELECT with joins/ORDER BY/LIMIT/IN(?,?,?):
debug : cold=191us/call warm=2.2us/call => 87x speedup
release : cold=39us/call warm=277ns/call => 140x speedup
prepare_cached in rusqlite backend
- query()/execute() switched to conn.prepare_cached(), which interns the
parsed statement in rusqlite's per-connection LRU. Second and later
identical SQL avoid the sqlite3_prepare_v2 cost entirely.
column_decltype + kill the LIMIT 0 probe
- Enable the rusqlite `column_decltype` feature so Statement::columns()
exposes decl_type().
- New Backend::describe_columns(sql) trait method with a default LIMIT-0
probe implementation; rusqlite overrides it to read metadata off the
prepared statement without executing.
- MySQL on_prepare: skip describe entirely for INSERT/UPDATE/DELETE (they
have no result set); for SELECTs, use describe_columns instead of the
old `SELECT ... LIMIT 0` probe.
- PostgreSQL probe_columns: use describe_columns first; only fall back to
a LIMIT 1 probe if any column is untyped (`SELECT 1 + 2` style).
- Empty-result-set SELECTs now carry real column types on the wire,
which fixes Laravel Query::exists() and PDO::getColumnMeta().
utf8 fallback in rusqlite backend
- extract_value on ValueRef::Text used to lossy-decode via
String::from_utf8_lossy, which silently corrupts non-UTF-8 bytes with
U+FFFD. Invalid UTF-8 now surfaces as Value::Blob instead so the wire
frontend can round-trip the raw bytes unchanged.
Wire-format regression fix (uncovered by column_decltype flip)
- MySQL row writer previously stringified integers/floats. That worked
only because every column arrived at the wire as VAR_STRING. With
real decltypes flowing through, integer columns become MYSQL_TYPE_LONGLONG
and opensrv-mysql rejects a String payload for a LONGLONG column
("tried to use [49] as MYSQL_TYPE_LONGLONG"). Row writer now writes
Value::Integer/Value::Float natively.
Dialect gains Hash so it can be used as a HashMap/LruCache key.
luthermonson
force-pushed
the
perf/translate-cache-and-prepare
branch
from
July 10, 2026 04:15
00e3997 to
c00b26d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on PR #4 (
compat/session-statements-and-ci) -- merge that first. This PR is a pure perf + correctness follow-up:prepare_cachedso identical repeat statements avoidsqlite3_prepare_v2.column_decltypefeature so empty-result SELECTs carry real column types on the wire.Backend::describe_columnstrait method; MySQL / PG frontends use it to answerCOM_STMT_PREPARE/Describewithout aLIMIT 0probe round trip.extract_valueonValueRef::Textnow falls back toValue::Blobfor invalid UTF-8 instead of lossily replacing bytes with U+FFFD.Base branch:
compat/session-statements-and-ci. Diff shown against that branch; when PR #4 lands, retarget or rebase ontomain.Benchmark
translate_cache_speedup(5000 iterations of a WP-shapedSELECT ... INNER JOIN ... WHERE post_status = ? AND term_taxonomy_id IN (?, ?, ?) ORDER BY post_date DESC LIMIT 10):Cold path is dominated by
sqlparser::Parser::parse_sql+ expression tree rewriting; the LRU eliminates it after the first hit. Run yourself:cargo test --release -p litewire-translate translate_cache_speedup -- --nocapture.Changes
Translate cache (
litewire-translate::cache)TranslateCache=parking_lot::Mutex<lru::LruCache<(Dialect, String), Vec<TranslateResult>>>. Default capacity 1024; documented in the module.translate_cached(&cache, sql, dialect)is the new entry point. Errors are not cached; a repeatedly-bad statement re-parses each time (avoids negative caching stampedes).#[derive(Hash)];TranslateResultgetsCloneso the cache can return owned copies.Arc<TranslateCache>created inMysqlFrontend::serve/PostgresFrontend::serveand cloned into every accepted connection.lru = 0.12added as an explicit dep onlitewire-translate(already in the workspace lockfile transitively).prepare_cachedRusqlite::query()andRusqlite::execute()both switched toconn.prepare_cached(&sql). rusqlite maintains a per-connection LRU of parsed statements; second-and-later identical SQL avoidssqlite3_prepare_v2cost entirely. Complements the translate cache: translate cache eliminates parsing on the litewire side,prepare_cachedeliminates it on the SQLite side.Kill the LIMIT 0 probe
rusqlitefeaturecolumn_decltypein the workspaceCargo.toml. Comment records why:Statement::columns()needs it.Backend:LIMIT 0probe (so third-party backends keep working). rusqlite overrides it to read metadata off the prepared statement without executing.LiteWireHandler::on_prepare: forINSERT/UPDATE/DELETE, skip the probe entirely (they have no result set); forSELECT, calldescribe_columns.PostgresHandler::probe_columns:describe_columnsfirst; only fall back toLIMIT 1if any column comes back untyped (e.g.SELECT 1 + 2).utf8 fallback (
extract_value)ValueRef::TextwasString::from_utf8_lossy(s).into_owned()-- silently corrupts non-UTF-8 bytes withU+FFFD. Now:std::str::from_utf8->Value::Texton success,Value::Blob(bytes)on failure. Wire frontends can then round-trip the raw bytes back to the client unmodified.Fallout / wire-format regression fix
Enabling
column_decltypeexposed a latent bug in the MySQL row writer. It was stringifying integers/floats (i.to_string()), which only worked because every column arrived at the wire asMYSQL_TYPE_VAR_STRING(decltype was alwaysNoneon the old path). With real decltypes flowing, integer columns come out asMYSQL_TYPE_LONGLONG, andopensrv-mysql's binary encoder refuses aStringpayload for aLONGLONGcolumn ("tried to use [49] as MYSQL_TYPE_LONGLONG" -- caught byprepared_select_with_param). Row writer now hands nativei64/f64towrite_col.Test plan
cargo test --workspace-- 281 passing, 0 failed (was 271; added 6 backend tests fordescribe_columns+ invalid UTF-8 blob fallback, and 4 cache-behaviour tests + 1 timing test).cargo clippy --workspace --all-targets -- -D warnings-- clean.cargo fmt --all -- --check-- clean.cargo test --release translate_cache_speedup -- --nocapture).mysql_e2eon a real MySQL client (PDO end-to-end) after merging -- I ran the in-processmysql_e2esuite which exercises the binary prepared-statement protocol end-to-end and everything passes, but a realpdo_mysql-> litewire link is worth verifying before tagging a release.