Skip to content

perf(relay): index relay table for O(1) hot-path lookups - #83

Open
krazyjakee wants to merge 2 commits into
foxssake:mainfrom
krazyjakee:perf/relay-hotpath-index
Open

perf(relay): index relay table for O(1) hot-path lookups#83
krazyjakee wants to merge 2 commits into
foxssake:mainfrom
krazyjakee:perf/relay-hotpath-index

Conversation

@krazyjakee

@krazyjakee krazyjakee commented Jul 23, 2026

Copy link
Copy Markdown

Summary

The UDP relay hot path (UDPRelayHandler.relay) did two Array.prototype.find linear scans over the relay table on every packet — one to resolve the sender, one for the target. Cost is O(N) in active relays, so per-packet throughput collapses as the relay count grows.

This PR replaces those scans with maintained indexes, keeping the hot path O(1) regardless of table size.

Changes

  • _byAddress — nested Map (address → port → entry), so the hot path looks up the sender by raw address + port with no per-packet string concatenation.
  • _byPortMap (port → entry) for the target lookup.
  • Indexes are kept in sync in createRelay / freeRelay / clear (cost paid once per relay, not per packet).
  • Split relay() into a thin NetAddress wrapper over a new allocation-free relayRaw(); the socket data callback now calls relayRaw directly, removing the per-packet new NetAddress. A NetAddress is still built on the (cold) drop path to preserve the drop event contract.

Benchmark

bun scripts/bench.relay.ts (added). Measures the real relay/relayRaw path (metrics + event emit intact) with a no-op socket injected, so it isolates relay-logic CPU cost rather than loopback syscalls.

Active relays Before (linear scan) After (indexed) Speedup
10 1,723,328 /s 1,737,344 /s ~1x
1,000 468,080 /s 1,466,026 /s ~3x
5,000 90,678 /s 1,382,513 /s ~15x
10,000 38,710 /s 1,291,839 /s ~33x

Before: ~44x throughput collapse from 10 → 10,000 relays. After: flat ~1.3–1.7M ops/s.

Note: this microbenchmark measures relay-logic CPU cost with a no-op socket, so absolute numbers are a ceiling, not end-to-end pps (real socket.send() syscalls bound throughput lower). The point is that the bound is now independent of relay count.

Testing

  • All existing spec/unit tests pass (bun test test/spec → 144 pass, 3 pre-existing skips).
  • End-to-end forwarding verified through real loopback UDP sockets via the new relayRaw path.
  • Added unit coverage for the new index bookkeeping in test/spec/relay/udp.relay.handler.test.ts:
    • direct relayRaw() transmit — exercises the shipped hot path itself, not just the relay(NetAddress) wrapper.
    • drop after target free — freeing a target removes it from _byPort, so a subsequent relayRaw() to that port drops (no send, drop emitted).
    • shared-address bucket integrity — two relays on one address: freeing one keeps the sibling routable; freeing the last cleans up the empty _byAddress bucket (the ports.size === 0 branch).
    • drop-event address contract — the drop event's sender is a value-equal NetAddress, which dynamic relaying keys off.

The UDP relay hot path (`UDPRelayHandler.relay`) previously did two
`Array.prototype.find` linear scans over the relay table on *every*
packet — one to resolve the sender, one for the target. Cost grew
O(N) in active relays, so per-packet throughput collapsed as the
relay count rose (~44x slower from 10 to 10,000 relays in the
included benchmark).

Add two indexes maintained alongside the relay table:
  - `_byAddress`: nested Map (address -> port -> entry), so the hot
    path can look up by the raw sender address + port with no
    per-packet string concatenation
  - `_byPort`: Map (port -> entry) for the target lookup

Also split `relay()` into a thin NetAddress wrapper over a new
allocation-free `relayRaw()`, and have the socket data callback call
`relayRaw` directly — removing the per-packet `new NetAddress`. A
NetAddress is still constructed on the (cold) drop path to preserve
the `drop` event contract.

Result: hot-path throughput stays flat (~1.3-1.7M ops/s) regardless
of table size instead of collapsing — ~33x at 10,000 relays. All
existing spec/unit tests pass unchanged.

Adds `scripts/bench.relay.ts` to reproduce the measurements.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@krazyjakee
krazyjakee marked this pull request as ready for review July 23, 2026 09:40
Add unit coverage for the maintained relay indexes introduced alongside
relayRaw():
- direct relayRaw() transmit (the shipped hot path, not just the
  relay(NetAddress) wrapper)
- drop after a target is freed (deindex from _byPort)
- shared-address bucket integrity: freeing one port keeps the sibling
  routable; freeing the last port cleans up the empty _byAddress bucket

Also assert the drop event's sender is a value-equal NetAddress, pinning
the contract dynamic relaying keys off.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant