Skip to content

Lock-free MPSC/SPSC inter-thread queue + reactor mailbox (#81)#84

Merged
EdmondDantes merged 3 commits into
mainfrom
81-performant-lock-free-inter-thread-message-queue-mpscspsc-reactor-integration
Jun 4, 2026
Merged

Lock-free MPSC/SPSC inter-thread queue + reactor mailbox (#81)#84
EdmondDantes merged 3 commits into
mainfrom
81-performant-lock-free-inter-thread-message-queue-mpscspsc-reactor-integration

Conversation

@EdmondDantes

Copy link
Copy Markdown
Contributor

Closes #81.

Implements the non-blocking inter-thread message queue + reactor integration from #81, using the queues selected by the #81 micro-benchmark. This is the handoff foundation that unblocks cross-worker HTTP/3 (#72) and will back WebSocket outbound/offload (#2).

Design — two layers

L1 thread_queue (C++ TU, C-ABI)src/core/thread_queue.cc + include/core/thread_queue.h

  • Bounded thread_mpsc_t (moodycamel ConcurrentQueue) and thread_spsc_t (moodycamel ReaderWriterQueue) carrying a void* payload.
  • An atomic length counter is the authoritative cap: producers reserve before pushing, so enqueue fails cleanly when full (deterministic backpressure, no allocation past the cap) and the reservation drives the empty→non-empty edge via the was_empty out-param.
  • Batch drain (*_drain). Aligned operator new so the cache-line-padded queue members are correctly aligned on every C++ standard.

L2 thread_mailbox (C, ZEND_ASYNC)src/core/thread_mailbox.c + include/core/thread_mailbox.h

  • MPSC queue wired to the consumer's loop via a zend_async_trigger_event_t (uv_async). thread_mailbox_post() is callable from any thread and never blocks (clean backpressure on full), so it is safe on a reactor / poll_cb path that must not stall ACKs (cf. HTTP/3: don't block the transport/reactor thread with business logic (ACK-timing budget) #80).
  • Lost-wakeup-safe: the producer signals only on the empty→non-empty edge, the enqueue (release) happens-before the signal, and the consumer drains to empty before returning. uv_async coalescing + drain-to-empty means no item is stranded.
  • The mailbox pointer lives in the trigger event's persistent extra area (not the ZMM-allocated callback); ZEND_ASYNC_EVENT_SET_CLOSED before dispose closes the door on late producer signals.

Vendored dependency

deps/concurrentqueue/concurrentqueue.h / readerwriterqueue.h / atomicops.h (Simplified BSD), with UPSTREAM.md recording the #81 benchmark decision (ConcurrentQueue is the only ready-made MPSC that holds throughput under producer contention; ReaderWriterQueue is the maintained SPSC).

Build

  • PHP_REQUIRE_CXX() + shared-module link via $(CXX) (libstdc++). Wired into config.m4, CMakeLists.txt (C → C CXX), and config.w32 (/EHsc).
  • Adds a C++ toolchain requirement to the build (the issue explicitly allows C++).

Tests

  • tests/unit/core/test_thread_queue.c (cmocka): single-thread correctness, bound rejection + recovery, batch drain, was_empty edges, and a 4-producer × 50k-item multi-producer stress that asserts every item arrives exactly once with the count returning to 0.
  • Validated: builds the full .so (loads into PHP), unit test green directly + under ASan/UBSan (×5) + via ctest, and the existing http1parser + multipart phpt suites pass 30/30 on the rebuilt module.

Scope

This PR is the reusable primitive only. Wiring it into the cross-worker H3 forward/inject path (#72) and WebSocket queues (#2) is downstream follow-up. Note the mailbox carries a raw void*, so #72 no longer needs the php-async ThreadChannel / IS_PTR zval path for the packet handoff.

Note: ThreadSanitizer is not used here — moodycamel synchronizes via atomic_thread_fence, which TSAN cannot instrument (it warns -Wtsan), producing false positives. Correctness is covered by the exactly-once stress test under ASan/UBSan instead.

…#81)

Vendor the moodycamel queues chosen by the #81 benchmark and wrap them in a
C-ABI primitive plus a reactor-integrated mailbox — the non-blocking handoff
foundation for cross-worker HTTP/3 (#72) and WebSocket (#2).

- deps/concurrentqueue: vendored concurrentqueue.h / readerwriterqueue.h /
  atomicops.h (Simplified BSD); UPSTREAM.md records the #81 decision.
- thread_queue (C++ TU, C-ABI): bounded MPSC (ConcurrentQueue) and SPSC
  (ReaderWriterQueue) over a void* payload. An atomic length counter is the
  authoritative cap and drives the empty->non-empty wakeup edge; batch drain.
- thread_mailbox (C): MPSC + zend_async_trigger_event_t (uv_async) wakeup.
  Producer signals only on the empty edge, consumer drains to empty —
  lost-wakeup-safe; clean backpressure on full, OOM-safe by construction.
- build: PHP_REQUIRE_CXX + link via $(CXX); wired in config.m4, CMake, config.w32.
- tests: cmocka unit test (single-thread correctness, bound, batch drain,
  4-producer 200k-item exactly-once stress). Clean under ASan/UBSan.
…ee-inter-thread-message-queue-mpscspsc-reactor-integration

# Conflicts:
#	CHANGELOG.md
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Coverage

Total lines: 81.42% → 81.16% (-0.26 pp)

File Baseline Current Δ Touched
src/core/thread_mailbox.c 0.00% 0.00% +0.00 pp
src/http3/http3_callbacks.c 80.43% 81.02% +0.59 pp

The PHP/autoconf build sets no C++ standard, relying on the compiler default.
g++ on Linux defaults to >= C++17, but AppleClang on the macOS CI runner
defaults to C++98, so moodycamel (and the C++11 #error guard) failed to compile.
Probe newest-first and pin -std=gnu++17..c++11 on CXXFLAGS, which only reaches
the C++ TU — the C sources keep using CFLAGS untouched.
@EdmondDantes EdmondDantes merged commit f7439f0 into main Jun 4, 2026
8 checks passed
@EdmondDantes EdmondDantes deleted the 81-performant-lock-free-inter-thread-message-queue-mpscspsc-reactor-integration branch June 4, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performant lock-free inter-thread message queue (MPSC/SPSC) + reactor integration

1 participant