perf(relay): index relay table for O(1) hot-path lookups - #83
Open
krazyjakee wants to merge 2 commits into
Open
Conversation
The UDP relay hot path (`UDPRelayHandler.relay`) previously did two
`Array.prototype.find` linear scans over the relay table on *every*
packet — one to resolve the sender, one for the target. Cost grew
O(N) in active relays, so per-packet throughput collapsed as the
relay count rose (~44x slower from 10 to 10,000 relays in the
included benchmark).
Add two indexes maintained alongside the relay table:
- `_byAddress`: nested Map (address -> port -> entry), so the hot
path can look up by the raw sender address + port with no
per-packet string concatenation
- `_byPort`: Map (port -> entry) for the target lookup
Also split `relay()` into a thin NetAddress wrapper over a new
allocation-free `relayRaw()`, and have the socket data callback call
`relayRaw` directly — removing the per-packet `new NetAddress`. A
NetAddress is still constructed on the (cold) drop path to preserve
the `drop` event contract.
Result: hot-path throughput stays flat (~1.3-1.7M ops/s) regardless
of table size instead of collapsing — ~33x at 10,000 relays. All
existing spec/unit tests pass unchanged.
Adds `scripts/bench.relay.ts` to reproduce the measurements.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
krazyjakee
marked this pull request as ready for review
July 23, 2026 09:40
Add unit coverage for the maintained relay indexes introduced alongside relayRaw(): - direct relayRaw() transmit (the shipped hot path, not just the relay(NetAddress) wrapper) - drop after a target is freed (deindex from _byPort) - shared-address bucket integrity: freeing one port keeps the sibling routable; freeing the last port cleans up the empty _byAddress bucket Also assert the drop event's sender is a value-equal NetAddress, pinning the contract dynamic relaying keys off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The UDP relay hot path (
UDPRelayHandler.relay) did twoArray.prototype.findlinear scans over the relay table on every packet — one to resolve the sender, one for the target. Cost is O(N) in active relays, so per-packet throughput collapses as the relay count grows.This PR replaces those scans with maintained indexes, keeping the hot path O(1) regardless of table size.
Changes
_byAddress— nestedMap(address → port → entry), so the hot path looks up the sender by raw address + port with no per-packet string concatenation._byPort—Map(port → entry) for the target lookup.createRelay/freeRelay/clear(cost paid once per relay, not per packet).relay()into a thinNetAddresswrapper over a new allocation-freerelayRaw(); the socketdatacallback now callsrelayRawdirectly, removing the per-packetnew NetAddress. ANetAddressis still built on the (cold) drop path to preserve thedropevent contract.Benchmark
bun scripts/bench.relay.ts(added). Measures the realrelay/relayRawpath (metrics + event emit intact) with a no-op socket injected, so it isolates relay-logic CPU cost rather than loopback syscalls.Before: ~44x throughput collapse from 10 → 10,000 relays. After: flat ~1.3–1.7M ops/s.
Testing
bun test test/spec→ 144 pass, 3 pre-existing skips).relayRawpath.test/spec/relay/udp.relay.handler.test.ts:relayRaw()transmit — exercises the shipped hot path itself, not just therelay(NetAddress)wrapper._byPort, so a subsequentrelayRaw()to that port drops (no send,dropemitted)._byAddressbucket (theports.size === 0branch).dropevent's sender is a value-equalNetAddress, which dynamic relaying keys off.