Skip to content

Added lockfree bounded MPMC queue (Vyukov/Strauss-style) with unique_ptr wrapper, GTest suite, and sanitizer builds - #11

Merged
toshit3q34 merged 3 commits into
CPP-CodingClubIITG:mainfrom
Aryan810:main
Jun 21, 2026
Merged

Added lockfree bounded MPMC queue (Vyukov/Strauss-style) with unique_ptr wrapper, GTest suite, and sanitizer builds#11
toshit3q34 merged 3 commits into
CPP-CodingClubIITG:mainfrom
Aryan810:main

Conversation

@Aryan810

@Aryan810 Aryan810 commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a new lockfree bounded MPMC queue (tsfqueue::impl::lockfree_mpmc_bounded<T, N>)
under include/lockfree_mpmc_bounded/, a unique_ptr wrapper around it, a 9-test
GTest suite, and TSAN/ASAN/UBSAN build options.

  1. Added: lockfree_mpmc_bounded<T, N>

A bounded, lock-free, multi-producer / multi-consumer ring buffer in the spirit
of Vyukov's MPMC bounded queue + Erez Strauss's packed-entry optimization.

Each slot is a single 64-bit entry packing both {value, index}, so the slot's
state can be CAS'd atomically in one shot - no per-slot mutex, no separate
sequence counter word.

Algorithm in brief:

Two atomic counters: head_index (consumer cursor) and tail_index (producer cursor).
Each slot stores {value, index} packed into a single atomic word.

Push:

  1. Load tail_index -> t. Read slot S = arr[t & (N-1)].
  2. If S.index == t -> slot is empty for this lap -> CAS the slot to {value, t},
    then CAS-bump tail_index from t to t+1 (helping rule).
  3. If S.index == t+1 -> another producer already wrote here and we are stale ->
    help-bump tail_index and retry.
  4. Otherwise -> queue is full for this lap -> return false (or spin, depending
    on API).

Pop:

  1. Load head_index -> h. Read slot S = arr[h & (N-1)].
  2. If S.index == h+1 -> slot holds a value for this lap -> CAS the slot to
    {empty, h+N} (mark it free for the next lap), then bump head_index to h+1.
  3. If S.index == h -> producer hasn't written yet -> queue empty -> return false.
  4. Otherwise -> another consumer raced us -> help-bump head_index and retry.

Single-word slot CAS + lap-encoded indices give wait-free progress per operation
and no ABA risk (the lap stride is N, so indices advance monotonically modulo
the full index_type range).

N must be a power of two so & (N-1) replaces % N.

  1. unique_ptr Wrapper

lockfree_mpmc_bounded_unique_ptr<T, N> wraps the raw queue so callers can move
heap-owned objects across threads safely. Internally it stores the released raw
pointer as uint64_t. push releases on success, pop reconstructs a
std::unique_ptr via reset, and the destructor drains any remaining entries
so nothing leaks on queue teardown.

  1. Tests (tests/test_mpmc_bounded_lockfree.cpp)

9 GTest cases across correctness, linearizability, memory safety, and lifecycle:

  1. BasicFIFO_SingleProducerSingleConsumer - Correctness
  2. EdgeCases_EmptyPop_FullPush_WrapAround - Edge cases
  3. MultiProducerMultiConsumer_NoLossNoDuplicates - MPMC correctness
  4. Linearizability_PerProducerMonotonicOrder - Linearizability sniff
  5. MemorySafety_DrainOnDestroy - Wrapper drains on dtor
  6. MemorySafety_PushPopRoundtrip - constructed == destroyed
  7. MemorySafety_MPMC_NoLeak - Concurrent leak check
  8. Lifecycle_RepeatedConstructDestruct - 2000 build/tear cycles
  9. Lifecycle_NonCopyableNonMovable - Type-trait guarantees

All 9 pass under: normal build, TSAN, and ASAN.

A short test-running cheat sheet is included at
tests/guide_to_run_tests/TEST_MPMC_BOUNDED_LOCKFREE.md.

  1. CMake / Sanitizer Options

New CMake options (TSAN and ASAN are mutually exclusive):

cmake -DENABLE_TSAN=ON .. # ThreadSanitizer (data-race detection)
cmake -DENABLE_ASAN=ON .. # AddressSanitizer (UAF, OOB, leaks)
cmake -DENABLE_UBSAN=ON .. # UndefinedBehaviorSanitizer

Tests auto-scale iteration counts down under TSAN (via TSFQUEUE_TSAN_BUILD)
so the suite still finishes in seconds.

  1. Workflow change

.github/workflows/tests.yaml:

  • Removed the MPSC step. tests/test_mpsc.cpp is currently empty (no code), so
    there is no test_mpsc binary to run - keeping the step would fail CI. Removed
    for now so the workflow stays green; can be re-added when an MPSC test file
    is actually written.

  • Replaced the now-stale ./test_mpmc invocation with the two new binaries
    (./test_mpmc_unbounded_blocking, ./test_mpmc_bounded_lockfree) and updated
    the changed-file detection to match the new test filenames.

Aryan810 added 2 commits June 4, 2026 00:11
Removed MPSC test execution and added unbounded and bounded MPMC tests.
@Aryan810

Aryan810 commented Jun 3, 2026

Copy link
Copy Markdown
Contributor Author

Build is failing because, This queue required 128 bit CAS Operations.

rviz190606 pushed a commit to rviz190606/ThreadsafeQueueLib that referenced this pull request Jun 5, 2026
@toshit3q34

toshit3q34 commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

Asked AI and suggested to try this. Try and then lmk if this works:

  1. Check whether linking against libatomic (-latomic) fixes the issue.
  2. Verify if we are using legacy GCC intrinsics (__sync_bool_compare_and_swap_16, __sync_val_compare_and_swap_16) and consider migrating to __atomic_* or std::atomic.
  3. Check whether the target is being compiled with -mcx16 (required for CMPXCHG16B instructions).
  4. Verify the CI runner CPU supports CMPXCHG16B (cx16 flag in /proc/cpuinfo).
  5. Confirm the CAS object/entry is exactly 16 bytes (sizeof(entry) == 16).
  6. Confirm 16-byte alignment (alignof(entry) >= 16 or alignas(16)).
  7. Check whether 16-byte atomics are reported as lock-free (__atomic_always_lock_free(16, nullptr) or std::atomic<T>::is_lock_free()).
  8. Create a minimal standalone program using __sync_*_16 to see whether the issue is with the toolchain or our implementation. (Can try this in your own repo)

@Aryan810

Aryan810 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor Author

Added -mcx16 Flag. Now works on Github workflows.

@toshit3q34 toshit3q34 closed this Jun 21, 2026
@toshit3q34 toshit3q34 reopened this Jun 21, 2026

@toshit3q34 toshit3q34 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@toshit3q34
toshit3q34 merged commit 92e431a into CPP-CodingClubIITG:main Jun 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants