Skip to content

test(e2e): size MR and token-rotation budgets above their nested waits - #276

Open
jmrplens wants to merge 1 commit into
mainfrom
fix/e2e-timeout-budgets
Open

test(e2e): size MR and token-rotation budgets above their nested waits#276
jmrplens wants to merge 1 commit into
mainfrom
fix/e2e-timeout-budgets

Conversation

@jmrplens

@jmrplens jmrplens commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes three CE e2e tests whose context budget was smaller than the sum of the waits nested inside it, so the context expired mid-poll instead of letting the assertion run. Found while validating make test-e2e-docker before tagging 2.6.3.

This is a pre-existing test-infrastructure bug, not a regression: the failure mode is a timeout waiting on GitLab's own async processing, and none of the recent dependency updates touch MR diffs or token rotation.

The arithmetic

Test Old budget Nested waits Measured on a green run
TestIndividual_MRRawDiffs 180s ~20s setup + 120s waitForMRReady + 60s raw-diff poll = 200s 173s
TestMeta_MRReviewChanges 180s ~30s setup + 120s waitForMRReady + 120s diff-version deadline = 270s 159s (subtest)
TestIndividual_ProjectAccessTokenRotateSelf 240s 404 propagation window outlasted 8 retries 249s

waitForMRReady only logs when it times out — by design, so the following poll still gets its chance. An exactly-sized parent budget denies it that chance, which is why MRRawDiffs failed deterministically at ~181s on two consecutive full-suite runs.

New budgets: 300s / 360s (600s / 720s enterprise), and 12 retries for the token rotation. Every measured duration above exceeded its old budget, so these are the sizes the tests actually need — not padding.

Scope

Only the waits changed. No assertion was modified, relaxed, or removed.

Verification

Full make test-e2e-docker against an ephemeral GitLab CE + runner:

DONE 1445 tests, 17 skipped in 836.016s

JUnit report: tests=1445 failures=0 errors=0 skipped=17. The two prior runs on the same code (before this fix) reported 4 and 1 failures respectively.

go test -tags e2e -c compiles and golangci-lint run --build-tags e2e ./test/e2e/suite/ is clean.

Review in cubic

Summary by CodeRabbit

  • Tests
    • Improved end-to-end test reliability by allowing more time for access-token rotation and merge request operations.
    • Extended waiting periods for merge request setup, readiness checks, raw diffs, and review changes.
    • Added additional retry attempts where needed to accommodate longer processing delays.

Three CE tests failed against the Docker GitLab because their context
budget was smaller than the sum of the waits nested inside it, so the
context expired mid-poll instead of letting the assertion run:

- TestIndividual_MRRawDiffs: 180s budget vs ~20s setup + 120s
  waitForMRReady + 60s raw-diff poll. waitForMRReady only logs when it
  times out, precisely so the following poll still gets its chance,
  which an exactly-sized budget denies. Now 300s (600s enterprise);
  measured 173s on a green run.
- TestMeta_MRReviewChanges: 180s budget vs ~30s setup + 120s
  waitForMRReady + 120s diff-version deadline in DiffVersionsList.
  Now 360s (720s enterprise); the subtest measured 159s.
- TestIndividual_ProjectAccessTokenRotateSelf: the self-rotate 404
  propagation window was observed exceeding two minutes under load,
  outlasting the 8 retries. Now 12 retries within a 300s budget;
  measured 249s against the previous 240s budget.

Only the waits changed; every assertion is unchanged. Verified with a
full make test-e2e-docker run: 1445 tests, 17 skipped, 0 failures.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @jmrplens, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@sourcery-ai

sourcery-ai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Adjusts e2e test context timeouts and retry budgets so that parent test contexts outlast their nested waits, preventing premature context expiration during MR readiness and token rotation polling.

File-Level Changes

Change Details Files
Increase timeout and retry budget for project access token self-rotation polling so the test can tolerate longer 404 propagation windows under full-suite load.
  • Extend test-level context timeout from 240s to 300s for the ProjectAccessTokenRotateSelf e2e test.
  • Increase retry attempts in the self-rotate polling helper from 8 to 12 while keeping the 5s backoff interval.
  • Update the test comment to document the observed >2 minute 404 window and rationale for the higher retry budget.
test/e2e/suite/accessextras_ce_test.go
Resize MR raw diff e2e test context budget to exceed cumulative setup, MR readiness wait, and raw diff polling duration, and use shared e2e timeout helper.
  • Replace fixed 180s context.WithTimeout with e2eTimeoutContext using CE/EE-aware budgets (300s CE, 600s EE).
  • Add inline comment explaining the need for the parent budget to exceed nested waits and the behavior of waitForMRReady on timeout.
test/e2e/suite/miscextras_ce_test.go
Increase MR review changes meta test context budget so its subtest can complete both MR readiness and diff-version polling without expiring mid-poll.
  • Increase e2eTimeoutContext budgets from 180s/480s to 360s/720s for CE/EE runs.
  • Add inline comment clarifying the composition of nested waits (setup, waitForMRReady, diff-version deadline) and the failure mode when the budget is exactly sized.
test/e2e/suite/mrreview_meta_ce_test.go

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b17d8319-379a-44fe-a98c-019220190d0f

📥 Commits

Reviewing files that changed from the base of the PR and between 6697237 and f873211.

📒 Files selected for processing (3)
  • test/e2e/suite/accessextras_ce_test.go
  • test/e2e/suite/miscextras_ce_test.go
  • test/e2e/suite/mrreview_meta_ce_test.go

📝 Walkthrough

Walkthrough

The E2E tests now use longer timeout and retry windows for access-token rotation and merge request readiness and diff polling.

Changes

E2E timeout budget updates

Layer / File(s) Summary
Extend asynchronous test wait budgets
test/e2e/suite/accessextras_ce_test.go, test/e2e/suite/miscextras_ce_test.go, test/e2e/suite/mrreview_meta_ce_test.go
Access-token rotation allows 300 seconds and 12 retries. Merge request tests use extended readiness and diff polling contexts.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Mergeability Score: ⚪ Minimal · up to f8732

This PR only adjusts end-to-end test timeout and retry budgets without changing assertions or production behavior; no actionable merge-blocking risk remains after normal checks and review.

Possibly related PRs

Poem

A rabbit checks the clock with care,
More time for tokens in the air.
Merge requests wait, diffs appear,
Retry paws hop past old fear.
Longer bounds make tests run bright!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: increasing MR and token-rotation wait budgets for E2E tests.
Description check ✅ Passed The description clearly explains the issue, affected tests, scope, rationale, and verification results, despite not using every template heading.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/e2e-timeout-budgets

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant