Skip to content

fix: retry Helm downloads and extend install timeout - #171

Draft
bugkeep wants to merge 1 commit into
fix/app-store-prerequisites-reviewfrom
fix/helm-install-reliability-review
Draft

fix: retry Helm downloads and extend install timeout#171
bugkeep wants to merge 1 commit into
fix/app-store-prerequisites-reviewfrom
fix/helm-install-reliability-review

Conversation

@bugkeep

@bugkeep bugkeep commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Problem

App Store installs still fail unnecessarily when a Helm repository or OCI registry has a transient timeout, truncated response, HTTP 408/429, or HTTP 5xx response. The same index or chart can also be downloaded repeatedly while opening the install dialog and submitting the install.

The install path also had three related deadline and recovery problems: Helm used a fixed 10-minute readiness timeout, an active task could be treated as stale after 11 minutes even while a larger chart was still installing, and a failed install could leave a failed release and partial resources behind.

What changed

  • retry classic Helm index/chart downloads and OCI tag/pull operations only for transient failures, with bounded per-attempt deadlines and context-aware backoff
  • cache downloaded indexes and chart archives in process for 15 minutes with entry and memory limits; OCI chart archives use the same bounded cache
  • add helmInstallTimeout, defaulting to 20 minutes, while preserving an explicit bound for the complete detached install operation
  • enable Helm Wait, WaitForJobs, and Atomic together so timeout and install failures clean up the failed release and resources
  • refresh active task timestamps once per minute so long-running installs are not mistaken for abandoned tasks; after a process failure the existing 11-minute stale-task recovery remains effective

Scope and dependency

This fork PR is intentionally based on fix/app-store-prerequisites-review, which combines the current #109 and #114 heads and resolves their existing store/helm.go conflict. The PR itself contains one functional commit.

The intended upstream order is #109, then #114, then this change.

Validation

  • go test ./...
  • go vet ./...
  • go build ./...
  • local heartbeat test passed 20 consecutive runs and was removed before commit, per the repository rule against uploading Go test files
  • Airflow OCI chart: first load 5.00s, cached load 62ms
  • hello-world chart: installed successfully in 43.7s, reached deployed, exposed its endpoint, and uninstalled cleanly
  • Airflow timeout path: diagnostics were recorded and atomic cleanup removed the release resources, namespace, and provisioned volume
  • GitLab default chart: application preflight failure was reported before resource creation, confirming that application-specific required values remain outside this backend change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant