[bulk][GH#90] Employ shuffle scans to diffuse read traffic across partitions - #30

Closed

relentlesscol wants to merge 23 commits into

mainfrom

polecat/bu-i90

Commits on Jun 26, 2026

ci: GitHub Actions CI pipeline for bulk_executor (#195)
relentlesscol
committed
e8a7da7

Commits on Jun 29, 2026

fix(diff): skip writing empty diff segment files to S3 (bu-1a1) (#16)
relentlesscol
authored
a99bcf1
[bulk_executor] fix: exclude __pycache__ and dev cruft from python_modules.zip (#17)
relentlesscol
authored
18c0288
fix(bulk_executor): require PITR enabled for load command (bu-598) (#18)
relentlesscol
authored
ae37c49
[bulk_executor] feat: warn if custom --XRole lacks minimum permissions (#19)
relentlesscol
authored
32e186f
[bulk_executor] feat: early-exit on non-recoverable systemic errors (#20)
relentlesscol
authored
bed6be5
fix(rate_limiter): downgrade noisy Initializing log to debug (#181) (#24)
relentlesscol
authored
11c12da
feat(load): report write rate at load start (bu-p8y) (#25)
relentlesscol
authored
55c3ff9
fix(find): preserve DynamoDB types in find-to-S3 output (bu-waj) (#26)
relentlesscol
authored
b2e44b7
Remove redundant TableName from scan_kwargs in update module (#5)
relentlesscol
authored
386bb71
Show clean error messages for bad parameters instead of stack traces (#6)
relentlesscol
authored
2560d5b
Add dedicated unit tests for rate_limiter modules (#7)
relentlesscol
authored
0e1ded6
Force built-in role refresh on version mismatch (#8)
relentlesscol
authored
9f287fd
Add --XIdleTimeout for Glue cost optimization (#9)
relentlesscol
authored
8cd70bc
Add rate validation warnings for XMaxReadRate/XMaxWriteRate (#10)
relentlesscol
authored
30d1174
Install faker only for fill verb to speed worker startup (#11)
relentlesscol
authored
eb919d5
Add --XExistingBucket parameter to bootstrap (#13)
relentlesscol
authored
c2044ff
fix(tests): resolve rate_limiter import shadow breaking test collection (#27)
relentlesscol
authored
7a0e9cc
docs: finalize fork-based PR workflow rules (max 3 upstream, domain-grouped) (#28)
relentlesscol
authored
8dbcbe0

Navigation Menu

[bulk][GH#90] Employ shuffle scans to diffuse read traffic across partitions - #30

[bulk][GH#90] Employ shuffle scans to diffuse read traffic across partitions#30
relentlesscol wants to merge 23 commits into
mainfrom
polecat/bu-i90

Commits on Jun 26, 2026

ci: GitHub Actions CI pipeline for bulk_executor (#195)

Commits on Jun 29, 2026

fix(diff): skip writing empty diff segment files to S3 (bu-1a1) (#16)

[bulk_executor] fix: exclude pycache and dev cruft from python_modules.zip (#17)

fix(bulk_executor): require PITR enabled for load command (bu-598) (#18)

[bulk_executor] feat: warn if custom --XRole lacks minimum permissions (#19)

[bulk_executor] feat: early-exit on non-recoverable systemic errors (#20)

fix(rate_limiter): downgrade noisy Initializing log to debug (#181) (#24)

feat(load): report write rate at load start (bu-p8y) (#25)

fix(find): preserve DynamoDB types in find-to-S3 output (bu-waj) (#26)

Remove redundant TableName from scan_kwargs in update module (#5)

Show clean error messages for bad parameters instead of stack traces (#6)

Add dedicated unit tests for rate_limiter modules (#7)

Force built-in role refresh on version mismatch (#8)

Add --XIdleTimeout for Glue cost optimization (#9)

Add rate validation warnings for XMaxReadRate/XMaxWriteRate (#10)

Install faker only for fill verb to speed worker startup (#11)

Add --XExistingBucket parameter to bootstrap (#13)

fix(tests): resolve rate_limiter import shadow breaking test collection (#27)

docs: finalize fork-based PR workflow rules (max 3 upstream, domain-grouped) (#28)

Commits on Jun 30, 2026

feat(scancount): add --per-segment flag to reveal data skew (bu-i92)

feat(scancount): add --segments flag to control parallel scan segment count

Commits on Jul 1, 2026

feat: shuffle scan segments to diffuse read traffic across partitions (bu-i90)

fix(test): adapt segments assertion for shuffle after rebase onto --segments flag