Skip to content

dbutil: rewrite query placeholders without regexp#37

Merged
tulir merged 2 commits into
mautrix:mainfrom
jlucaso1:dbutil-query-rewrite
Jun 11, 2026
Merged

dbutil: rewrite query placeholders without regexp#37
tulir merged 2 commits into
mautrix:mainfrom
jlucaso1:dbutil-query-rewrite

Conversation

@jlucaso1

@jlucaso1 jlucaso1 commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

With the SQLite dialect, mutateQuery runs positionalParamPattern.ReplaceAllString on every query that goes through LoggingExecable, so the regexp engine cost is paid on every single Exec/Query/QueryRow. This replaces it with a hand-rolled scan doing the same $N to ?N rewrite: queries without placeholders return the input string with zero allocations, and queries with placeholders cost one allocation for the rewritten string.

The new test checks the function against the old regexp on every combination of up to 4 characters from a relevant alphabet plus some realistic queries, so the behavior is provably identical. Benchmark on a typical INSERT:

BenchmarkReplacePositionalParams/scanner-12    166.4 ns/op    160 B/op    1 allocs/op
BenchmarkReplacePositionalParams/regexp-12     578.9 ns/op    514 B/op    7 allocs/op

I also considered caching rewritten queries, but dynamically built queries (IN clauses, mass inserts) would make the cache unbounded, and the scanner already gets it down to the single unavoidable allocation.

Before/after heap profiles from a whatsmeow ping/pong benchmark that uses this library with SQLite, viewable in speedscope: before, after. mutateQuery drops from 111MB to 22MB of allocations over the run, with the regexp machinery replaced by the single allocation of the rewritten string.

Comment thread dbutil/database.go Outdated
Comment thread dbutil/database.go
@jlucaso1
jlucaso1 requested a review from tulir June 11, 2026 11:42
@tulir
tulir merged commit 4cb0682 into mautrix:main Jun 11, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants