[Feature] Graceful member removal is one best-effort broadcast — no exiting phase, no acknowledgment, no leader-sequenced handoff (survey)

[pathosDev]

Problem

Leaving the cluster is a single best-effort broadcast with no protocol behind it. leave() marks itself leaving, fires one LeaveMessage at each currently-reachable peer, cancels its timers and stops — no acknowledgment, no retry, no leader involvement, and no exiting status in the member-state machine (Protocol.ts knows joining/weakly-up/up/leaving/down/removed).

Consequences, each independently annoying:

• A lost frame turns a graceful leave into a failure. Any peer that misses the broadcast (backpressured socket, momentary unreachability — precisely likely during a rolling deploy) never learns of the leave and discovers the node via the failure detector instead, with the full unreachableAfterMs/downAfterMs latency and an "unreachable" event in place of a clean removal — the operational difference between "deploy" and "incident" in every dashboard that watches membership.
• Nothing sequences the leave with workload handoff. There is no exiting phase during which the leader (or the coordinator) can move singletons and shards before the node disappears; leave() is instantaneous from the cluster's perspective. Shard handoff on shutdown therefore rides MemberRemoved/timeout paths rather than an orderly "this node is exiting, relocate first, then remove" sequence ([Docs] The docs say the cluster shutdown phases wire themselves up automatically while the only three addTask sites in src/ hit service-unbind and actor-system-terminate, so a rolling deploy neither leaves the cluster nor hands off shards #993 documents the wiring gap on the shutdown side).
• leaving is terminal-but-undefined: the status exists on the wire, but no component reacts to it with any behaviour distinct from up — it is a label, not a phase.

The tracked decision: define the graceful-removal handshake. Minimum useful shape: leave() retries the broadcast until at least one peer acknowledges (or a deadline passes); the leader, on seeing leaving, relocates singletons/shards, then gossips the member to removed; the leaver waits for that echo (or the deadline) before dropping its transport — turning "hope the frame arrived" into a bounded two-step.

Evidence

src/cluster/Cluster.ts:574-590 — the whole mechanism:

  /** Gracefully leave the cluster (broadcast `leave`, stop transport). */
  async leave(): Promise<void> {
    ...
    const leaveMessage: LeaveMessage = { kind: 'leave', node: this.selfAddress.toJSON() };
    const peers = this.reachableMembers().filter(...);
    for (const member of peers) this.transport.send(member.address, leaveMessage);
    this.gossipTimer?.cancel();
    ...

src/cluster/Protocol.ts:27-33 — the status list; no exiting.

Verification status

CONFIRMED-BY-READ — quoted from the v0.15.0 tree. From the independent production-readiness pass (2026-08-14), second batch (design decisions). Related: #549 (wiring leave() into CoordinatedShutdown — the caller side of this), #993 (docs claim the phases self-wire), #994 (membership transitions invisible at default log level).