Harden pagination, retries, and interceptor error handling (1.1.0) - #1
Merged
Conversation
Pagination no longer silently truncates at the built-in maxPages safety cap. Reaching the default cap while the server still reports more data now throws PaginationLimitError instead of returning a partial result set; an explicit maxPages (including Infinity) remains an opt-in that stops silently. This converts a silent data-loss footgun into a loud, actionable failure. Also: - Add RetryConfig.maxBackoffMs to make the exponential-backoff cap configurable (default 2000, unchanged) for riding out sustained 429 bursts. - Errors thrown by a response interceptor are no longer misclassified as transient network failures: they now propagate untouched instead of being retried and wrapped in RetryError, matching InterceptorAbortError semantics. New tests cover the throw-on-overflow behaviour, the explicit-maxPages opt-in, natural termination at the cap boundary, the configurable backoff cap, and interceptor-throw propagation. Full suite: 252 passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Data-integrity and resilience release (1.1.0). One behavioural change to pagination plus two additive config options. No public type or method removed or renamed.
Surfaced while building a large (~100K-record) Keap audit consumer on top of this library.
Changed — pagination no longer silently truncates⚠️
Page-based and cursor-based pagination previously stopped at the built-in
maxPagessafety cap (default 100) and returned a truncated array with no signal. At common page sizes this silently dropped data — e.g. paging 100 000 records at 1 000/page hits exactly 100 pages, and anything larger thanpageSize × 100came back short with no error.Now, reaching the default cap while the server still reports more data throws the new
PaginationLimitErrorinstead of returning a partial result.Opt-in escape hatch (unchanged from 1.0.0 for existing callers): passing an explicit
maxPages— any number, ormaxPages: Infinityfor unbounded — is treated as a deliberate limit and stops silently without throwing.Migration: callers who never set
maxPagesand relied on the implicit 100-page stop should either pass an explicitmaxPagesto restore the old behaviour, or (recommended) usemaxPages: Infinityand page the full set. Callers already passingmaxPagesare unaffected.Added
PaginationLimitError(exported from root;code: 'EPAGINATION_LIMIT',maxPages). Not retried by the retry loop, likeInterceptorAbortError.RetryConfig.maxBackoffMs— configurable exponential-backoff cap (default 2000, unchanged). Does not affect aRetry-Aftervalue (capped separately at 60s) or a customBackoffStrategy.Fixed
RetryError; it now propagates untouched (no retry, no wrapping), consistent withInterceptorAbortError.Tests
New coverage for: throw-on-overflow, the explicit-
maxPagesopt-in, natural termination exactly at the cap boundary, the configurable backoff cap, and interceptor-throw propagation. Full suite: 252 passing across 10 suites.Notes
CHANGELOG.mdupdated with a[1.1.0]section and a prominent migration note.1.1.0inpackage.json/package-lock.json; not yet published to npm.