ci: increase flux-local helm retry budget to 5×30s#601
Closed
ExtraToast wants to merge 1 commit into
Closed
Conversation
flux-local passes --repository-cache /tmp/<random> to every helm template invocation, bypassing HELM_CACHE_HOME and actions/cache entirely — chart tarballs are always downloaded fresh. GitHub release CDN 504s (the observed failure mode: external-dns-1.21.1.tgz) typically clear within 60-90 seconds, but the previous retry 3×10s gave only 20 s of retry window, which was not enough. 5 attempts × 30 s delay gives ~2 minutes of retry budget, covering the observed failure mode without burning excessive CI minutes on a real template/schema error.
Owner
Author
|
Approach rejected — adding retry time is not the right fix. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
flux-local --enable-helmpasses--repository-cache /tmp/<random>to every internalhelm templateinvocation. That temp dir is created fresh each run, which meansHELM_CACHE_HOME,actions/cacheon~/.cache/helm, and any pre-warming of Helm's default repo cache are all completely bypassed — chart tarballs are always downloaded from the remote on every CI run.GitHub's release CDN (used by several charts, most recently
external-dns-1.21.1.tgz) periodically returns 504s that clear within ~60-90 seconds. The previousretry 3 10budget (3 attempts, 10 s delay = ~20 s total retry window) was not enough to outlast these hiccups, causing Platform Validate to fail and requiring a manual re-run.retry 5 30(5 attempts, 30 s delay = ~2 min total retry window) covers the observed failure duration without meaningfully extending CI run time on a real template or schema error, which fails on the first attempt.The comment in the script documents why actions/cache and HELM_CACHE_HOME are not viable alternatives, so the next person doesn't re-investigate.