exp 168: synchronous fast path for uncontended writer mutex

[danReynolds]

Hypothesis

Exp 159 made the writer's request path synchronous through SendPort.send and pipelined concurrent standalone writes by releasing the write lock at send time. It still relied on await _mutex.lock() to acquire the mutex, even when no transaction was holding it. Mutex.lock is async, so its implicit Future hops once before the caller resumes — every uncontended db.execute / db.executeBatch paid that microtask hop on its way to SendPort.send.

Adding Mutex.lockSync() (returns null when the lock can be claimed without waiting) and rewriting Writer.execute / Writer.executeBatch as non-async should remove that hop. Dart is single-threaded, so checking the completer and claiming the slot in one synchronous call is safe.

Approach

• Mutex.lockSync() — additive, opt-in. Returns null (lock acquired sync) or the Future<void> from lock() (caller should await it). Existing lock() / unlock() / run() are unchanged.
• Writer.execute / Writer.executeBatch — no longer async. Fast path: synchronous lock, _closed check, executeInTransaction / executeBatchInTransaction send, unlock, return the reply Future. Synchronous throws are converted to Future.error so the public contract — "always returns a Future" — is preserved.
• Slow path: new _executeSlow / _executeBatchSlow async helpers that await the waiter, then mirror the original try/finally body bit for bit.
• Writer.locked (transactions) is intentionally unchanged. Transactions hold the lock across BEGIN/body/COMMIT, where the microtask hop is dominated by the round-trips. Leaving this path on the async lock keeps writer_pipelining's transaction-guardrail as a clean control.

See experiments/168-uncontended-write-mutex-fast-path.md for the full reasoning and prior-experiment context.

Results

Focused benchmark/experiments/writer_pipelining.dart, three paired passes with stash/pop side-flipping (medians, ms):

shape	baseline	candidate	delta
sequential-awaited (2000 writes)	32.686	30.868	-5.6%
concurrent-burst (10 × 200 writes)	25.272	24.607	-2.6%
transaction-guardrail (50 tx × 10)	4.282	4.240	-1.0%

Per-pass:

shape	pass 1	pass 2	pass 3
sequential-awaited	-4.0%	-9.0%	-5.0%
concurrent-burst	-4.4%	-2.6%	-4.8%
transaction-guardrail	+1.6%	0.0%	-1.5%

The transaction guardrail stays inside ±2% across all three passes — the expected null result for the untouched code path. Full per-round timings: benchmark/profile/results/exp-168-uncontended-write-mutex-fast-path.md.

Outcome

In Review (accept-shaped). The change is a focused overhead removal: monotonic direction on the two paths it touches (sequential-awaited and concurrent-burst), neutral on the unchanged path (transactions), no public API change. The Mutex contract is unchanged; lockSync() is additive. Reopen if exp 161's release rows (Single Inserts (100 sequential), Concurrent Single Inserts (100 concurrent)) regress during the soak window, or if any future workload shows the missing fast path on Writer.locked is now material.

One behavioral change worth calling out: writes submitted synchronously before close() (e.g. db.execute(A); db.execute(B); db.close();) now succeed instead of sometimes throwing ResqliteConnectionException. The old behavior was an artifact of the async lock wait; A and B's ExecuteRequests now reach the worker's port FIFO before close() enqueues CloseRequest. The original close-vs-queued-writer test is updated to force the queue with a long-running transaction (preserving the original assertion), and a new test pins the new fast-path semantics.

Test plan

• dart pub get
• dart analyze lib/src/mutex.dart lib/src/writer/writer.dart (no issues)
• dart test test/transaction_test.dart test/database_test.dart test/profile_counters_test.dart test/stream_test.dart test/stream_invalidation_coalescing_test.dart test/stream_dependency_shapes_test.dart test/stream_cache_hit_reliability_test.dart test/stream_overflow_fallback_test.dart test/stream_trigger_cascade_test.dart test/reader_pool_test.dart test/query_decoder_test.dart test/diagnostics_test.dart (all pass)
• Focused A/B: dart run benchmark/experiments/writer_pipelining.dart — three paired passes
• dart run benchmark/finalize_experiment.dart --experiment=experiments/168-uncontended-write-mutex-fast-path.md (green; history.json regenerated)
• CI / release-suite soak — watch Single Inserts (100 sequential) and Concurrent Single Inserts (100 concurrent) rows for the public-lane confirmation of this focused signal

🤖 Generated with Claude Code

[danReynolds]

Review cleanup pushed (459433b)

Collapsed the fast/slow × execute/batch structure (4 functions, the lock/check/send/unlock body written 4×, unlock in 2 sites per non-async path) into one generic helper:

Future<ExecuteResponse> execute(String sql, [params, id]) {
  Future<ExecuteResponse> send() => executeInTransaction(sql, params, id);
  if (_mutex.tryLock()) return _sendUnderLock(send);          // fast: non-async, no hop
  return _mutex.lock().then((_) => _sendUnderLock(send));     // contended: one line
}

Future<T> _sendUnderLock<T>(Future<T> Function() send) {
  try {
    if (_closed) throw ResqliteConnectionException('Database is closed.');
    return send();                       // finally runs AFTER this → unlock at send time
  } catch (e, st) {
    return Future.error(e, st);          // preserve 'never throws synchronously'
  } finally {
    _mutex.unlock();                     // one unlock site for both paths
  }
}

Why the shape is irreducible, not arbitrary: the win requires execute be non-async (an async wrapper re-adds the exact microtask hop being removed), and a non-async function can't await, so the contended branch must hand back _mutex.lock().then(...). That branch stays — but as one line, not a second function. The try/catch/finally unifies what was a hand-rolled catch→Future.error (fast) + try/finally (slow): finally runs after return send() evaluates, so early-unlock-at-send-time is preserved with a single unlock site.

Verification: dart analyze clean; 109 tests pass (incl. the close() contention guards); fast path is structurally identical (tryLock → send → unlock), the only addition being one send closure/call (~0.3% of the 2000-write lane, below the win and below noise). Net −45 lines. Focused A/B vs the prior version is neutral; the public-lane win confirmation is the release-suite soak as before.