Lock-free MPSC/SPSC inter-thread queue + reactor mailbox (#81)#84
Merged
EdmondDantes merged 3 commits intoJun 4, 2026
Conversation
…#81) Vendor the moodycamel queues chosen by the #81 benchmark and wrap them in a C-ABI primitive plus a reactor-integrated mailbox — the non-blocking handoff foundation for cross-worker HTTP/3 (#72) and WebSocket (#2). - deps/concurrentqueue: vendored concurrentqueue.h / readerwriterqueue.h / atomicops.h (Simplified BSD); UPSTREAM.md records the #81 decision. - thread_queue (C++ TU, C-ABI): bounded MPSC (ConcurrentQueue) and SPSC (ReaderWriterQueue) over a void* payload. An atomic length counter is the authoritative cap and drives the empty->non-empty wakeup edge; batch drain. - thread_mailbox (C): MPSC + zend_async_trigger_event_t (uv_async) wakeup. Producer signals only on the empty edge, consumer drains to empty — lost-wakeup-safe; clean backpressure on full, OOM-safe by construction. - build: PHP_REQUIRE_CXX + link via $(CXX); wired in config.m4, CMake, config.w32. - tests: cmocka unit test (single-thread correctness, bound, batch drain, 4-producer 200k-item exactly-once stress). Clean under ASan/UBSan.
4 tasks
…ee-inter-thread-message-queue-mpscspsc-reactor-integration # Conflicts: # CHANGELOG.md
Contributor
CoverageTotal lines: 81.42% → 81.16% (-0.26 pp)
|
The PHP/autoconf build sets no C++ standard, relying on the compiler default. g++ on Linux defaults to >= C++17, but AppleClang on the macOS CI runner defaults to C++98, so moodycamel (and the C++11 #error guard) failed to compile. Probe newest-first and pin -std=gnu++17..c++11 on CXXFLAGS, which only reaches the C++ TU — the C sources keep using CFLAGS untouched.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #81.
Implements the non-blocking inter-thread message queue + reactor integration from #81, using the queues selected by the #81 micro-benchmark. This is the handoff foundation that unblocks cross-worker HTTP/3 (#72) and will back WebSocket outbound/offload (#2).
Design — two layers
L1
thread_queue(C++ TU, C-ABI) —src/core/thread_queue.cc+include/core/thread_queue.hthread_mpsc_t(moodycamel ConcurrentQueue) andthread_spsc_t(moodycamel ReaderWriterQueue) carrying avoid*payload.was_emptyout-param.*_drain). Alignedoperator newso the cache-line-padded queue members are correctly aligned on every C++ standard.L2
thread_mailbox(C, ZEND_ASYNC) —src/core/thread_mailbox.c+include/core/thread_mailbox.hzend_async_trigger_event_t(uv_async).thread_mailbox_post()is callable from any thread and never blocks (clean backpressure on full), so it is safe on a reactor /poll_cbpath that must not stall ACKs (cf. HTTP/3: don't block the transport/reactor thread with business logic (ACK-timing budget) #80).uv_asynccoalescing + drain-to-empty means no item is stranded.ZEND_ASYNC_EVENT_SET_CLOSEDbefore dispose closes the door on late producer signals.Vendored dependency
deps/concurrentqueue/—concurrentqueue.h/readerwriterqueue.h/atomicops.h(Simplified BSD), withUPSTREAM.mdrecording the #81 benchmark decision (ConcurrentQueue is the only ready-made MPSC that holds throughput under producer contention; ReaderWriterQueue is the maintained SPSC).Build
PHP_REQUIRE_CXX()+ shared-module link via$(CXX)(libstdc++). Wired intoconfig.m4,CMakeLists.txt(C → C CXX), andconfig.w32(/EHsc).Tests
tests/unit/core/test_thread_queue.c(cmocka): single-thread correctness, bound rejection + recovery, batch drain,was_emptyedges, and a 4-producer × 50k-item multi-producer stress that asserts every item arrives exactly once with the count returning to 0..so(loads into PHP), unit test green directly + under ASan/UBSan (×5) + via ctest, and the existing http1parser + multipart phpt suites pass 30/30 on the rebuilt module.Scope
This PR is the reusable primitive only. Wiring it into the cross-worker H3 forward/inject path (#72) and WebSocket queues (#2) is downstream follow-up. Note the mailbox carries a raw
void*, so #72 no longer needs the php-async ThreadChannel /IS_PTRzval path for the packet handoff.