Introduce common pagination utility#4732
Merged
Merged
Conversation
pritishpai
had a problem deploying
to
account-admin
GitHub Actions
Failure
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4732 +/- ##
==========================================
+ Coverage 87.77% 87.80% +0.02%
==========================================
Files 123 123
Lines 17569 17595 +26
Branches 3713 3717 +4
==========================================
+ Hits 15422 15449 +27
Misses 1458 1458
+ Partials 689 688 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
✅ 66/66 passed, 10 flaky, 6 skipped, 5h16m53s total Flaky tests:
Running from acceptance #9029 |
FastLee
requested changes
Mar 9, 2026
Introduce paginated_fetch_offset (SCIM startIndex/count) and paginated_fetch_cursor (token-based) helpers in framework/utils.py to eliminate duplicated pagination boilerplate across the codebase.
Replace inline pagination loop in AccountGroupLookup._list_account_groups with the shared paginated_fetch_offset utility, adding proper SCIM offset pagination and deduplication to the previously unpaginated call.
Replace inline cursor-based pagination loop in feature_store_listing with the shared paginated_fetch_cursor utility.
Rewrite multi-line list comprehension as for loop (R8923), prefix unused arguments with underscore (W0613), use implicit booleaness for empty list checks (C1803).
Rename inner fetch functions to be more descriptive (fetch_page -> fetch_feature_tables, fetch_account_groups). Redesign paginated_fetch_offset callable signature to accept (start_index, count) as ints instead of a mutable dict, so callers own query construction. Add unit tests for feature_store_listing multi-page collection and _list_account_groups duplicate-ID deduplication to close coverage gap.
pritishpai
force-pushed
the
chore/pagination_utility
branch
from
7a4545d to
dfe3ed1
Compare
pritishpai
had a problem deploying
to
account-admin
GitHub Actions
Failure
pritishpai
had a problem deploying
to
account-admin
GitHub Actions
Failure
FastLee
approved these changes
Jun 4, 2026
FastLee
left a comment
FastLee
left a comment
Contributor
There was a problem hiding this comment.
LGTM, I would suggest spliting the push.yml change into a separate PR
Contributor
Author
|
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Introduce shared pagination utilities in framework/utils.py and refactor all manual raw-API pagination loops to use them, eliminating duplicated boilerplate and reducing the risk of pagination bugs (like the missing pagination that caused customer issues with account groups listing).
Linked issues
Closes #4731
Related to #4730
Functionality
paginated_fetch_offset -- Generic SCIM-style offset pagination (startIndex/count). Accepts a callable with signature
(start_index: int, count: int) -> dictso callers own query construction. Handles page advancement, empty-page termination, and partial-page detection.paginated_fetch_cursor -- Generic cursor/token-based pagination (page_token/next_page_token). Handles token forwarding and missing-token termination.
Refactored sites
AccountGroupLookup._list_account_groups (workspace_access/groups.py) -- Previously made a single unpaginated API call. Now uses paginated_fetch_offset with deduplication, matching the pagination logic in the SDK's AccountGroupsAPI.list(). This fixes the root cause where not all account groups were fetched for large accounts. Inner fetch function renamed from fetch_page to fetch_account_groups.
feature_store_listing (workspace_access/generic.py) -- Previously had an inline while True / next_page_token loop. Now delegates to paginated_fetch_cursor. Inner fetch function renamed from fetch_page to fetch_feature_tables.
Tests
Unit tests added: