Flaky page loading in tests because of staging CD - introducing better retry - #1172
Flaky page loading in tests because of staging CD - introducing better retry #1172piotrwitek with Copilot wants to merge 9 commits into
Conversation
|
@copilot the problem was page didn't load because staging server is low on resources and also there are many commits going in during the day with auto deployments, the solution here is really to retry loading page and not skipping the test, think how to enable it globally for all tests, also retry should be 5 times with 3 seconds wait with quadratic increase each turn |
… increase fixture timeout
|
@copilot where you added exponential backoff on retries and where you defined 5 retries? I don't see you added that |
Updated |
The implementation is in async openPage() {
// Quadratic backoff: 3*n² seconds per retry → 3s, 12s, 27s, 48s, 75s
const retryIntervals = [1, 2, 3, 4, 5].map((n) => 3_000 * n * n);
await expect(async () => {
await this.page.goto('', { timeout: expectDefaultTimeout * 2 });
await this.waitForAppToBeStable();
}).toPass({
intervals: retryIntervals,
timeout: openPageTimeout,
});
}
The beach club header test in |
There was a problem hiding this comment.
Pull request overview
This PR refactors Earn Protocol test setup by centralizing “open app + stabilize” logic into a new App.openPage() helper with retry/backoff, and updates multiple wallet/no-wallet fixtures to use that helper with a shared timeout constant.
Changes:
- Added
openPageTimeoutto shared config and applied it as the fixture setup timeout across Earn Protocol fixtures. - Introduced
App.openPage()with retry intervals and moved setup navigation/stabilization into this helper. - Updated Earn Protocol fixtures to call
app.openPage()instead ofpage.goto('')+waitForAppToBeStable().
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| utils/config.ts | Adds a new shared timeout constant for page-opening/setup logic. |
| srcEarnProtocol/app.ts | Adds openPage() helper using expect().toPass() retry/backoff around navigation + stability checks. |
| srcEarnProtocol/fixtures/withTestWalletBase.ts | Uses app.openPage() and switches fixture timeout to openPageTimeout. |
| srcEarnProtocol/fixtures/withTestWalletArbitrum.ts | Uses app.openPage() and switches fixture timeout to openPageTimeout. |
| srcEarnProtocol/fixtures/withRealWalletBase.ts | Uses app.openPage() and switches fixture timeout to openPageTimeout. |
| srcEarnProtocol/fixtures/withRealWalletArbitrum.ts | Uses app.openPage() and switches fixture timeout to openPageTimeout. |
| srcEarnProtocol/fixtures/withRealWalletSonic.ts | Uses app.openPage() and switches fixture timeout to openPageTimeout. |
| srcEarnProtocol/fixtures/withRealWalletHyperliquid.ts | Uses app.openPage() and switches fixture timeout to openPageTimeout. |
| srcEarnProtocol/fixtures/noWallet.ts | Uses app.openPage(), removes stale commented code, and switches fixture timeout to openPageTimeout. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This pull request refactors how the Earn Protocol test suite opens and stabilizes the application page across all fixtures, centralizing the logic into a new
openPagemethod and standardizing timeouts. This improves reliability, maintainability, and consistency for E2E tests by ensuring all test setups use the same robust page loading logic with quadratic backoff retries and configurable timeouts.Test setup improvements:
openPagemethod to theAppclass, which navigates to the root page using a quadratic backoff retry strategy and waits for the app to stabilize. This replaces direct calls topage.gotoand manual stabilization in all fixtures.noWallet,withRealWallet*,withTestWallet*) now useapp.openPage()instead of duplicating navigation and stabilization logic. [1] [2] [3] [4] [5] [6] [7]Timeout configuration:
openPageTimeoutinutils/config.tsto provide environment-aware timeouts for opening the page, and updated all relevant fixture timeouts to use this value for consistency. [1] [2] [3] [4] [5] [6] [7] [8]openPageTimeoutplus a buffer, replacing hardcoded values. [1] [2] [3] [4] [5] [6] [7]Code cleanup:
openPagenow handles page readiness. [1] [2]These changes make the test setup more robust and easier to maintain by centralizing page load logic and standardizing configuration.