-
Notifications
You must be signed in to change notification settings - Fork 39
Closed
Description
Problem
When two or more MPCClient instances connect to the same NATS cluster, they all subscribe to the same result work queues:
| Operation | Result Topic |
|---|---|
| Keygen | mpc.mpc_keygen_result.* |
| Signing | mpc.mpc_signing_result.complete |
| Reshare | mpc.mpc_reshare_result.* |
Because the result streams use WorkQueuePolicy (message deleted on ACK), consumers on the same stream compete for messages. Client A's keygen result may be delivered to Client B, and vice versa.
Architecture Diagram — Current (Broken with Multiple Clients)
┌──────────────┐ ┌──────────────┐
│ Client A │ │ Client B │
│ (Backend 1) │ │ (Backend 2) │
└──────┬───────┘ └──────┬───────┘
│ CreateWallet("w1") │ CreateWallet("w2")
▼ ▼
┌─────────────────────────────────────────┐
│ NATS JetStream │
│ │
│ keygen_request stream │
│ ┌─────────┐ ┌─────────┐ │
│ │ req: w1 │ │ req: w2 │ │
│ └────┬────┘ └────┬────┘ │
│ └──────┬─────┘ │
│ ▼ │
│ MPC Cluster processes both │
│ │ │
│ ▼ │
│ keygen_result stream (WorkQueuePolicy) │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ result: w1 │ │ result: w2 │ │
│ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌────────────────────────────────┐ │
│ │ consumer: "mpc-keygen-consumer"│ │
│ │ filter: mpc.mpc_keygen_result.*│ │
│ │ (SHARED by both clients!) │ │
│ └────────────┬───────────────────┘ │
└───────────────┼─────────────────────────┘
│
┌───────┴───────┐
▼ ▼
Client A Client B
gets w2 ✗ gets w1 ✗ ← RACE! Wrong results delivered
Reproduction Scenario
- Client A (backend service 1) calls
CreateWallet("wallet-A")and registersOnWalletCreationResult(callbackA). - Client B (backend service 2) calls
CreateWallet("wallet-B")and registersOnWalletCreationResult(callbackB). - MPC cluster completes keygen for
wallet-Aand publishes result tompc.mpc_keygen_result.wallet-A. - Client B receives
wallet-A's result (because both share the same durable consumer onWorkQueuePolicy). - Client B's callback processes a wallet it never requested → unexpected behavior.
- Client A never receives its own result → timeout / lost event.
The same issue affects signing results (even worse since mpc.mpc_signing_result.complete is an exact single topic shared by all clients) and reshare results.
Proposed Solution — Client-Specific Result Routing
Introduce an optional clientID when constructing MPCClient. When set, the client subscribes to client-scoped result topics, and the MPC cluster routes results to the correct client-specific stream.
Architecture Diagram — Proposed (With Client ID)
┌───────────────────┐ ┌───────────────────┐
│ Client A │ │ Client B │
│ clientID: "svcA" │ │ clientID: "svcB" │
└──────┬────────────┘ └──────┬────────────┘
│ CreateWallet("w1") │ CreateWallet("w2")
│ (header: clientID=svcA) │ (header: clientID=svcB)
▼ ▼
┌──────────────────────────────────────────────────────┐
│ NATS JetStream │
│ │
│ keygen_request stream │
│ ┌──────────────────┐ ┌──────────────────┐ │
│ │ req: w1 │ │ req: w2 │ │
│ │ hdr: clientID=A │ │ hdr: clientID=B │ │
│ └────────┬─────────┘ └────────┬─────────┘ │
│ └──────────┬──────────┘ │
│ ▼ │
│ MPC Cluster processes both │
│ │ │
│ ┌──────────┴──────────┐ │
│ ▼ ▼ │
│ Result for w1 Result for w2 │
│ topic: mpc.mpc_keygen_ topic: mpc.mpc_keygen_ │
│ result.svcA.w1 result.svcB.w2 │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────────┐ ┌─────────────────────┐ │
│ │consumer: keygen-svcA│ │consumer: keygen-svcB│ │
│ │filter: ...result. │ │filter: ...result. │ │
│ │ svcA.* │ │ svcB.* │ │
│ └──────────┬──────────┘ └──────────┬──────────┘ │
└─────────────┼─────────────────────────┼──────────────┘
▼ ▼
Client A Client B
gets w1 ✓ gets w2 ✓ ← Correct!
Topic Scheme
| Operation | Without clientID (backward compat) | With clientID |
|---|---|---|
| Keygen result | mpc.mpc_keygen_result.<walletID> |
mpc.mpc_keygen_result.<clientID>.<walletID> |
| Signing result | mpc.mpc_signing_result.complete |
mpc.mpc_signing_result.<clientID>.complete |
| Reshare result | mpc.mpc_reshare_result.<sessionID> |
mpc.mpc_reshare_result.<clientID>.<sessionID> |
Backward Compatibility
| Scenario | Behavior |
|---|---|
NewMPCClient(signer, nats) — no clientID |
Works exactly as today. Same topics, same consumers. Zero breaking changes. |
NewMPCClient(signer, nats, WithClientID("svcA")) — with clientID |
Client subscribes to mpc.mpc_keygen_result.svcA.* etc. Requests include clientID header. Results routed to client-specific topics. |
| Mixed deployment (some clients with ID, some without) | Each operates independently. Legacy clients get legacy topics, ID-aware clients get scoped topics. |
Implementation Scope
pkg/client/client.go— AddWithClientID(id string)option. When set, modify topic subscriptions and includeclientIDheader in all requests.pkg/event/— Add topic builder functions that accept optional clientID (e.g.,KeygenResultTopic(clientID, walletID) string).pkg/eventconsumer/event_consumer.go— When publishing results, check forclientIDheader on the original request. If present, publish to client-scoped topic. If absent, publish to legacy topic.pkg/messaging/message_queue.go— Support dynamic consumer/stream creation for client-scoped result queues.- TypeScript client (
mpcium-client-ts) — Mirror the sameclientIDoption.
Impact
- No changes required for existing single-client deployments.
- Enables multi-tenant and multi-instance deployments safely.
- Signing results benefit the most since all clients currently compete on the single
mpc.mpc_signing_result.completetopic.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels