core(mailbox): rebuild from kameo reference — queue + Signal + backpressure

[joeldsouzax]

Part of the core-rebuild epic — follows the shared rigor contract there. First card.

Component

src/mailbox.rs (~1,151 LOC). The actor's in-memory queue: the Signal enum (message + control/stop/link-died), bounded vs unbounded, backpressure (block) vs drop. The run-loop pulls from it; ActorRef pushes to it. Everything else in the runtime stands on this.

Learning & discussion

Carries Box<dyn DynMessage> — typed, in-memory, zero-serialize: this is the local tier of the two-tier model (see #66 design note). Signals are heterogeneous; the loop selects over them. We keep the local typed queue; we rebuild the impl to the god-level bar on a vetted primitive + our own Signal layer.

Existing crates

tokio::sync::mpsc (bounded/unbounded base), flume, thingbuf (bounded lock-free ring / keep-latest), async-channel, crossbeam-channel; tokio-util CancellationToken for stop (#55). Decide the primitive for bounded-block vs ring/keep-latest.

Research — best algorithm/approach

Vyukov intrusive MPSC (1024cores), LMAX Disruptor ring-buffer + backpressure, thingbuf design. Capacity-hit returns Result, never panics (rule 4 / nexus #22). No commitment before the survey.

Testing — hard as fuck

• Existing net (Test coverage: write tests for each section of the surviving kameo core #74) stays green (the spec).
• Cross-cutting: sequence (enqueue/dequeue ordering under interleave), lifecycle (close/drop; senders after receiver dropped), defensive boundary (cap 0, usize::MAX), linearizability (N senders + 1 receiver, FIFO-per-sender).
• Property (proptest): boundary caps 0,1,MAX-1,MAX.
• DST: seedable scheduler over send/recv/close/drop interleavings (madsim / shuttle), replay on failure.
• loom: Signal/close ordering — no lost-wakeup, no lost-message (ties test(core): loom model for the std-atomic concurrency primitives (id counter first) #88).
• Mutation: cargo-mutants, zero surviving mutants.

Related

#19 (Zenoh channel-handler strategy — remote tier), #55 (stop/abort).

[joeldsouzax]

Knock-on from #114 — mailbox is a concrete mpsc<Signal<A>>, no Box<dyn>

The message-model finalization (#114) fixes the mailbox shape:

• Element type = Signal<A>, a concrete closed envelope enum { Message(A::Msg), Stop, LinkDied{..}, /* system arms */ } — not kameo's Box<dyn DynMessage<A>>. This is the substrate that makes tell zero-alloc (move the value into the slot; no per-message heap box).
• Channel = bounded tokio::mpsc<Signal<A>> (bounded for backpressure — kameo/actix bound; coerce's unbounded is a memory footgun). Monomorphized per actor A, so no type erasure at the queue.
• Large-variant discipline: the queue stores by value, so every slot costs size_of::<Signal<A>>() = largest variant. Keep clippy large_enum_variant on; Box only the cold fat variants (core(message): rebuild from kameo reference — Message trait + Context dispatch #114).
• Signal<A> is where the fixed system arms live (Stop / LinkDied / supervision) alongside the domain Message(A::Msg) — both closed, so no Box<dyn> at either layer.

Zero-alloc holds only while the send path is monomorphic (ActorRef<A> holds mpsc::Sender<Signal<A>>); ref-erasure (#117) must hold a typed sender to preserve it. Related: #19 (Fifo/Ring strategies), #55 (cancellation). Cross-links: #114, #116, #115.

[joeldsouzax]

⚠️ Implementation watch-outs (from the adversarial review #122)

In addition to the MaybeSend bound + Store trait seam already noted:

• Large-enum-variant trap. mpsc<Signal<A>> queues by value, so every slot costs size_of::<Signal<A>>() = the largest variant. Keep clippy large_enum_variant ON; Box only the cold fat variants (by measured frequency×size), never the whole enum — else a tiny message pays the biggest message's footprint on every slot.
• Bounded, not unbounded. Bounded mailbox → MailboxFull(M) backpressure (the deadlock-detection lever, feat: messaging over Zenoh query/reply #4/core(request): rebuild from kameo reference — ask/tell builders #118). Unbounded is a memory footgun (coerce's mistake).
• Signal<A> is a concrete envelope enum ({Message(Msg), Stop, LinkDied, …}) — no Box<dyn>; both layers closed so the loop is a match.

[joeldsouzax]

🧪 Test cases to write first (TDD — implement with this card)

• large_enum_variant_gate (lint/defensive) — clippy large_enum_variant stays enabled; a deliberately-fat Msg variant triggers it, forcing a Box on the cold variant only. Guards the "every slot = largest variant" trap. (Ref epic: rebuild the runtime core part-by-part (kameo = reference oracle) #122 / core(message): rebuild from kameo reference — Message trait + Context dispatch #114.)
• bounded_mailbox_backpressures (defensive) — a full bounded mailbox yields MailboxFull (try) / awaits (send); never grows unbounded.
• signal_envelope_has_no_box_dyn (defensive/compile) — the mailbox element is the concrete Signal<A> enum; a tell is a move (0 alloc — assert allocation count).
• loom: send_close_race (linearizability/loom) — concurrent Sender::send and Receiver drop; assert no lost-signal / no UB (loom over the Arc/mpsc path, now loom-visible post-reversal). (Ref epic: rebuild the runtime core part-by-part (kameo = reference oracle) #122-design: link-death — per-actor liveliness vs per-node + explicit death #8.)
• DST: deterministic_store_replays (linearizability/DST, seeded) — the deterministic Store/Registry impl (behind the trait seam) replays bit-identically under a fixed seed. This is the seam that carries DST/loom, since prod papaya is loom-opaque. (Ref epic: rebuild the runtime core part-by-part (kameo = reference oracle) #122-design: link-death — per-actor liveliness vs per-node + explicit death #8.)

[joeldsouzax]

Implementation landed — standalone bombay-core::mailbox

Built standalone (approved bootstrap: the zero-box Signal<A> needs the #114 associated-Msg, so the mailbox is built against a 3-line Mailboxed scaffold seam that #114/#116 later subsume). New crate bombay-core, born under the god-level bar (no #61 quarantine).

Delivered (TDD, 16 tests, clippy --all-features clean):

• Signal<A> = Message(A::Msg) | Stop | LinkDied(Box<LinkDied>) — concrete, closed, no Box<dyn>; tell is a move.
• Capacity(NonZeroUsize) validated 1..=MAX_PERMITS → bounded infallible (both illegal caps unrepresentable; tokio panics on 0 and > usize::MAX>>3, verified empirically).
• bounded send/try_send/recv/close; SendError/TrySendError::{Full,Closed} hand the signal back.
• WeakMailboxSender non-pinning liveness (the death-watch primitive — keeps kameo's WeakSender, per the epic: rebuild the runtime core part-by-part (kameo = reference oracle) #122 arena reversal).
• large_enum_variant discipline: LinkDied boxed; size_of::<Signal<Probe>>() pinned at 16 bytes.

Rigor: sequence/FIFO, lifecycle, defensive boundary (proptest 0,1,MAX-1,MAX,usize::MAX), an 8-thread Barrier linearizability test (FIFO-per-sender + no loss/dup), single-sender FIFO proptest. Criterion bench: tell enqueue ≈ 5.6 ns, send+recv ≈ 21.4 ns. Mutation (nix build .#mutants, pinned/reproducible): 0 missed.

loom/shuttle DST — deferred to #116/#120 (decision)

loom and shuttle only model-check code compiled against their primitives; the real tokio::sync::mpsc this mailbox wraps is opaque to both (loom docs, shuttle). The mailbox delegates all synchronization to tokio, so a loom/shuttle send_close_race here would either explore nothing or test a reimplemented channel (violates "test the actual SUT"). The first bombay-owned concurrency is the run-loop select over signals (#116) and the death-watch push (#120) — that is where loom/shuttle land. Until then the 8-thread linearizability test is the mailbox's concurrency coverage. Matches this card's own note: "the seam that carries DST/loom, since prod papaya is loom-opaque."

Everything reproducible via the pinned flake (nix build .#mutants, bench via nix develop). Branch: feat/112-mailbox-rebuild.