From d4776946b51d425037307de615faf76b41e6eee0 Mon Sep 17 00:00:00 2001 From: Luther Monson Date: Fri, 10 Jul 2026 19:24:30 -0700 Subject: [PATCH] perf(mysql): coalesce result-set response into one write to kill 40ms stall The MySQL frontend stalled ~44ms on every query that returns a result set (point SELECT: 44.0ms p50 via PHP PDO) while single-OK-packet responses (INSERT: 1.3ms) were unaffected. Root cause is a Nagle + delayed-ACK deadlock on the multi-packet response path: opensrv-mysql emits a result set as several separate packets (column count, one column-def per column, EOF, one packet per row, EOF) and writes each to the socket individually, calling flush() only once after the whole response. On a raw socket each packet is its own TCP segment. PHP's mysqlnd client does NOT set TCP_NODELAY, so its Nagle withholds the ACK of the first segment while it waits to coalesce more data to send, and Linux delayed-ACK holds that ACK ~40ms. The server-side set_nodelay added in 54b5fce does not help because the stall is driven by the *client's* Nagle, not the server's; a client that sets nodelay (mysql_async, the mysql CLI) never sees it, which is why it reproduced only through PHP. Wrapping the write half in a BufWriter makes opensrv's single trailing flush() emit the entire result set as one write -> one TCP segment, so there is no intermediate packet for the client to sit on. The handshake greeting and every command response are already flushed at their protocol boundaries, so buffering changes only how many segments a response spans, not protocol semantics. Larger-than-buffer responses still flush in chunks, which remains correct. Measured via ePHPm single-node SQLite (PHP PDO point SELECT): 44.0ms -> sub-millisecond p50. Postgres uses pgwire's Framed writer (already buffered) and TDS is a placeholder, so no change is needed there. --- crates/litewire-mysql/src/lib.rs | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/crates/litewire-mysql/src/lib.rs b/crates/litewire-mysql/src/lib.rs index 1153a32..0b941d6 100644 --- a/crates/litewire-mysql/src/lib.rs +++ b/crates/litewire-mysql/src/lib.rs @@ -82,6 +82,26 @@ impl MysqlFrontend { } }; let (reader, writer) = stream.into_split(); + // Coalesce the whole response into a single write. + // + // A result set is emitted by opensrv-mysql as several distinct + // packets (column-count, one column-def per column, EOF, one + // packet per row, EOF), each written to the socket separately; + // opensrv only calls `flush()` once, after the full response. + // On a raw socket every packet becomes its own TCP segment, so + // a client whose Nagle is enabled (PHP mysqlnd does NOT set + // TCP_NODELAY) withholds its ACK of the first segment while it + // waits for more data, and Linux delayed-ACK holds that ACK for + // ~40ms, stalling every result-set round trip (measured 44ms + // p50 per point-SELECT via PDO, vs 1.3ms for an INSERT, whose + // response is a single OK packet). Server-side `set_nodelay` + // alone does not cure it because the deadlock is driven by the + // client's Nagle, not the server's. Buffering makes opensrv's + // single trailing `flush()` emit the entire result set as one + // segment, so there is no intermediate packet for the client to + // sit on. The buffer is sized for the common small result set; + // larger responses flush in chunks, which is still correct. + let writer = tokio::io::BufWriter::with_capacity(64 * 1024, writer); if let Err(e) = opensrv_mysql::AsyncMysqlIntermediary::run_on(handler, reader, writer).await {