Add optional request governor for concurrency and rate limiting (1.2.0) - #2
Merged
Conversation
Introduces a client-side governor so consumers can stay within an upstream API's concurrency and rate limits during large or bursty workloads such as long paginated pulls. New FetchEnhConfig options (all optional, also accepted by setConfig): - concurrency: cap logical requests in flight via a fair FIFO semaphore, applied once per logical request so a slot is not held during retry backoff. - maxRps: cap requests started per second (minimum spacing between starts). - minIntervalMs: explicit minimum spacing; takes precedence over maxRps. Both gates honour a request's AbortSignal (a cancelled queued request rejects promptly and never runs), and the governor is a zero-overhead pass-through when no option is set. Adds a governor test suite; full suite 261 passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional client-side request governor (1.2.0, purely additive) so consumers can stay within an upstream API's concurrency and rate limits during large or bursty workloads — e.g. the long paginated pulls this was built to support. No behavioural change when the new options are omitted.
Builds on #1 (merged).
New
FetchEnhConfigoptionsAll optional; all also accepted by
setConfig(...):concurrency— caps logical requests in flight via a fair FIFO semaphore. Applied once per logical request, not per retry attempt, so a request does not hold a slot while it sleeps between retries.maxRps— caps requests started per second (minimum spacing of1000 / maxRpsms between starts).minIntervalMs— explicit minimum spacing between starts; takes precedence overmaxRps.Both gates honour a request's
AbortSignal— a cancelled request waiting in the concurrency queue rejects promptly with anAbortErrorand never runs. With no option set, the governor is a zero-overhead pass-through.Implementation
src/core/governor.ts(RequestGovernor): FIFO semaphore + virtual "next allowed start" clock for rate spacing, no background timers._requestaround the logical-request promise, outside the retry loop, preserving the existing pre-awaitdedup-tracking guarantee (deduplicated callers share the single governed request).Tests
New
tests/fetchEnh.governor.test.ts— 9 tests: concurrency ceiling never exceeded, FIFO ordering, slot released on rejection, abort-while-queued,maxRps/minIntervalMsspacing, no-op fast path, and two FetchEnh integration tests (constructor +setConfig). Full suite: 261 passing across 11 suites.Notes
1.2.0;CHANGELOG.mdupdated. Not yet published to npm.