ci: retry the Codecov upload, and never let it fail a green run - #6012
Merged
Conversation
CI-mysql84-gr-g8 failed on #5998 with every test passing: SUMMARY: 'tests' PASS 1/402 : FAIL 0/402 : SKIP 401/402 SUMMARY: ret_rc = [0] ##[error]Codecov: Failed to get OIDC token with url: https://codecov.io. Error Message: Request timeout: /37//idtoken/... The only failing step was 'Upload coverage to Codecov'. fail_ci_if_error: false does not cover this: the timeout happens while minting the GitHub OIDC token, before any upload is attempted, so the action fails the step and the step fails the job. A transient token-endpoint blip therefore reds a run whose tests were entirely green -- which is exactly the kind of false red that trains people to ignore CI. Two changes to all 47 workflows that upload coverage: - Three attempts, chained on steps.codecov_N.outcome == 'failure', with 15s and 45s waits between them. The observed failure is a transient request timeout, so a retry is very likely to succeed. - continue-on-error: true on each attempt, so even three failures leave the job green. Coverage reporting must never gate correctness. A ::warning:: annotation is emitted instead, so the loss is visible without being fatal. outcome (not conclusion) is the right field to branch on: with continue-on-error the conclusion is rewritten to success, while outcome keeps the real result. The explanatory comment block is kept only on the first attempt; the retries carry the identical 'with:' block without it.
📝 WalkthroughWalkthroughCI workflows now pin Codecov actions, gate uploads on coverage and cancellation state where configured, retry failed uploads after 15 and 45 seconds, and warn after three failures. Upload failures remain non-fatal to test jobs. ChangesCodecov upload resilience
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci-basictests.yml:
- Around line 321-323: Configure every codecov_1, codecov_2, and codecov_3
upload step to set fail_ci_if_error to true while retaining continue-on-error:
true, so upload failures produce failure outcomes and trigger retries without
failing the job. Apply this change in
.github/workflows/ci-basictests.yml:321-323,
.github/workflows/ci-legacy-clickhouse-g1.yml:237-239,
.github/workflows/ci-legacy-g1.yml:304-306,
.github/workflows/ci-legacy-g2-genai.yml:250-252,
.github/workflows/ci-mysql84-g4.yml:304-306,
.github/workflows/ci-mysql84-g5.yml:234-236,
.github/workflows/ci-mysql84-g6.yml:236-238,
.github/workflows/ci-mysql84-g7.yml:236-238,
.github/workflows/ci-mysql84-g8.yml:236-238,
.github/workflows/ci-mysql84-g9.yml:236-238,
.github/workflows/ci-mysql84-gr-g1.yml:229-231, and
.github/workflows/ci-mysql84-gr-g2.yml:229-231.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a5edd882-11a8-4f57-8584-93d285b88cc8
📒 Files selected for processing (47)
.github/workflows/ci-basictests.yml.github/workflows/ci-legacy-clickhouse-g1.yml.github/workflows/ci-legacy-g1.yml.github/workflows/ci-legacy-g2-genai.yml.github/workflows/ci-legacy-g2.yml.github/workflows/ci-legacy-g3.yml.github/workflows/ci-legacy-g4.yml.github/workflows/ci-legacy-g5.yml.github/workflows/ci-legacy-g6.yml.github/workflows/ci-legacy-g7.yml.github/workflows/ci-legacy-g8.yml.github/workflows/ci-legacy-g9.yml.github/workflows/ci-mariadb10-galera-g1.yml.github/workflows/ci-mariadb10-galera-g2.yml.github/workflows/ci-mariadb10-galera-g3.yml.github/workflows/ci-mariadb10-galera-g4.yml.github/workflows/ci-mariadb10-galera-g5.yml.github/workflows/ci-mariadb10-galera-g6.yml.github/workflows/ci-mariadb10-galera-g7.yml.github/workflows/ci-mariadb10-galera-g8.yml.github/workflows/ci-mariadb10-galera-g9.yml.github/workflows/ci-mysql56-single-g1.yml.github/workflows/ci-mysql84-g1.yml.github/workflows/ci-mysql84-g2.yml.github/workflows/ci-mysql84-g3.yml.github/workflows/ci-mysql84-g4.yml.github/workflows/ci-mysql84-g5.yml.github/workflows/ci-mysql84-g6.yml.github/workflows/ci-mysql84-g7.yml.github/workflows/ci-mysql84-g8.yml.github/workflows/ci-mysql84-g9.yml.github/workflows/ci-mysql84-gr-g1.yml.github/workflows/ci-mysql84-gr-g2.yml.github/workflows/ci-mysql84-gr-g3.yml.github/workflows/ci-mysql84-gr-g4.yml.github/workflows/ci-mysql84-gr-g5.yml.github/workflows/ci-mysql84-gr-g6.yml.github/workflows/ci-mysql84-gr-g7.yml.github/workflows/ci-mysql84-gr-g8.yml.github/workflows/ci-mysql84-gr-g9.yml.github/workflows/ci-mysql90-gr-g1.yml.github/workflows/ci-mysql93-gr-g1.yml.github/workflows/ci-mysql95-gr-g1.yml.github/workflows/ci-no-infra-g1.yml.github/workflows/ci-pgsql-socket-g1.yml.github/workflows/ci-set_parser_algorithm_3-g1.yml.github/workflows/ci-taptests-pgsql-cluster.yml
📜 Review details
🔇 Additional comments (35)
.github/workflows/ci-legacy-g2.yml (1)
309-311: LGTM!Also applies to: 327-375
.github/workflows/ci-legacy-g3.yml (1)
304-306: LGTM!Also applies to: 353-401
.github/workflows/ci-legacy-g4.yml (1)
305-307: LGTM!Also applies to: 354-402
.github/workflows/ci-legacy-g5.yml (1)
302-304: LGTM!Also applies to: 351-399
.github/workflows/ci-legacy-g6.yml (1)
304-306: LGTM!Also applies to: 353-401
.github/workflows/ci-mysql84-gr-g3.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql84-gr-g4.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql84-gr-g5.yml (1)
227-229: LGTM!Also applies to: 276-324
.github/workflows/ci-mysql84-gr-g6.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql84-gr-g7.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql84-gr-g8.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql84-gr-g9.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-legacy-g7.yml (1)
304-306: LGTM!Also applies to: 353-401
.github/workflows/ci-legacy-g8.yml (1)
236-238: LGTM!Also applies to: 285-333
.github/workflows/ci-legacy-g9.yml (1)
236-238: LGTM!Also applies to: 285-333
.github/workflows/ci-mariadb10-galera-g1.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mariadb10-galera-g2.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql90-gr-g1.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql93-gr-g1.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql95-gr-g1.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-no-infra-g1.yml (1)
303-305: LGTM!Also applies to: 352-400
.github/workflows/ci-pgsql-socket-g1.yml (1)
236-238: LGTM!Also applies to: 285-333
.github/workflows/ci-set_parser_algorithm_3-g1.yml (1)
236-238: LGTM!Also applies to: 285-333
.github/workflows/ci-taptests-pgsql-cluster.yml (1)
266-282: LGTM!Also applies to: 284-314, 330-332
.github/workflows/ci-mariadb10-galera-g3.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mariadb10-galera-g4.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mariadb10-galera-g5.yml (1)
227-229: LGTM!Also applies to: 276-324
.github/workflows/ci-mariadb10-galera-g6.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mariadb10-galera-g7.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mariadb10-galera-g8.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mariadb10-galera-g9.yml (1)
229-231: LGTM!Also applies to: 278-326
.github/workflows/ci-mysql56-single-g1.yml (1)
242-244: LGTM!Also applies to: 291-339
.github/workflows/ci-mysql84-g1.yml (1)
304-306: LGTM!Also applies to: 353-401
.github/workflows/ci-mysql84-g2.yml (1)
304-306: LGTM!Also applies to: 353-401
.github/workflows/ci-mysql84-g3.yml (1)
304-306: LGTM!Also applies to: 353-401
Review catch, and it would have made most of the previous commit inert. With 'fail_ci_if_error: false' the action swallows upload errors and the step outcome stays 'success', so 'steps.codecov_N.outcome == "failure"' never matches and retries 1 and 2 are dead code. Only the OIDC token failure that motivated this work fails the step regardless -- that one is raised before any upload is attempted -- so the retries would have covered exactly one failure mode and silently skipped every other. Setting it true makes every Codecov failure visible to the retry chain. The job is still kept green by 'continue-on-error: true' on all three attempts, which is the property that actually matters: coverage reporting must never gate correctness. After three failures the run stays green and emits a ::warning:: annotation. 185 occurrences across 47 workflows, plus the stale rationale comment on each first attempt, which still credited fail_ci_if_error for the never-fail-the-job behaviour that continue-on-error now provides.
SonarCloud reported 94 new githubactions:S7637 issues on this branch ('Use
full commit SHA hash for this dependency'), all of them the same thing:
duplicating the upload step for the retry chain tripled the number of
references to an already-unpinned action, so 47 files x 2 new copies became
94 new findings.
Waiving them would be the wrong call. The rule is real -- codecov-action
runs with the job token, and a mutable tag can be repointed at arbitrary
code -- and the duplication is mine, so the noise is mine to clear.
Pinning all 141 references to the commit v4 already resolves to fixes both
the 94 new findings and the 47 pre-existing ones, leaving zero.
Behaviourally this is a no-op today: v4 currently points at v4.6.0, which
is b9fd7d16f6d7d1b5d2bec1a2887e65ceed900238, so the same code runs before
and after. What changes is that a future repoint of the v4 tag no longer
silently changes what executes with our token. The version is kept visible
as a trailing '# v4.6.0' comment so bumps stay deliberate and reviewable.
The remaining 'codecov-action@v4' matches in these files are prose inside
the permissions comment, not action references.
|
This was referenced Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI-mysql84-gr-g8failed on #5998 with every test passing:The only failing step was
Upload coverage to Codecov.fail_ci_if_error: falsedoes not cover this — the timeout happens while minting the GitHub OIDC token, before any upload — so the action fails the step and the step fails the job. A transient blip reds a run whose tests were entirely green.Changes (47 workflows)
steps.codecov_N.outcome == 'failure', with 15 s / 45 s backoff.continue-on-error: trueon each, so even three failures leave the job green; a::warning::is emitted instead. Coverage must never gate correctness.fail_ci_if_error: true— review catch, and without it most of this PR was inert: with itfalse, the action swallows upload errors,outcomestayssuccess, and retries 1–2 are dead code. Only the OIDC failure fails the step regardless, so the retries would have covered exactly one failure mode.codecov-actionpinned to a full commit SHA — SonarCloud flagged 94 newgithubactions:S7637, all from duplicating an already-unpinned reference. Pinning all 141 refs clears the 94 new findings and the 47 pre-existing ones. Behaviourally a no-op today (v4→v4.6.0→b9fd7d16…), but a future repoint of the tag can no longer change what runs with our token.outcome(notconclusion) is the field the retries branch on —continue-on-errorrewritesconclusiontosuccess, which would make them dead code.All 47 files re-parse as valid YAML; the generated step chain was verified structurally.
Summary by CodeRabbit