Skip to content

feat(backend): Unify labels for pods - #236

Open
wnevis-cmyk wants to merge 50 commits into
v2from
wnevis/unifyLabels
Open

feat(backend): Unify labels for pods#236
wnevis-cmyk wants to merge 50 commits into
v2from
wnevis/unifyLabels

Conversation

@wnevis-cmyk

Copy link
Copy Markdown
Contributor
  • Standardizes labels for pods
  • Tested with wandb verify
  • Adds documentation describing labeling scheme

wandb-kc and others added 13 commits November 20, 2025 13:24
* chore: Upgrade go to 1.25.3

* chore: Upgrade go to 1.25.3
### [1.21.3](v1.21.2...v1.21.3) (2025-12-04)

### Bug Fixes

* Upgrade helm to 3.19.2 ([#106](#106)) ([aa4ca21](aa4ca21))
…un in v2 branch (#127)

* chore: Need to create the workflow in main so it can be updated and run in v2 branch

* chore: Add newline
* feat: Add OCI Helm chart registry support and upgrade to Helm v4

Enable the operator to pull Helm charts from OCI-based registries via
a new OCIRelease chart type, and upgrade the Helm SDK from v3.19.2 to
v4.1.3 to align with the current Helm ecosystem.

Helm v4 upgrade:
- Update all import paths (chart/v2, release/v1, repo/v1, etc.)
- Adapt to v4 API changes: Config.Init log param removed,
  Uninstall.Wait replaced by WaitStrategy, Entry field rename
- Remove dead DownloadChart() code
- Regenerate counterfeiter fakes

OCI support:
- New OCIRelease type in pkg/wandb/spec/charts/oci.go using Helm's
  native registry.Client.Pull() with in-memory chart loading
- Custom ociurl validator using registry.IsOCI()
- Supports authentication via direct credentials or K8s Secrets
- PlainHTTP option for insecure/local registries
- Registered in chart dispatcher between LocalRelease and RepoRelease
- 21 new Ginkgo test specs covering validation, dispatcher routing,
  credential secrets, and error paths

* fix: require OCI chart version and guard against nil pull result

Reject tagless OCI refs at validation time instead of letting them fail
at runtime on every reconcile. Also add a nil check on the pull result
to prevent a panic on corrupted registry responses.

* fix: move nil check before debug dereference, reject oci:// in RepoRelease

- Move result.Chart nil guard above the debug log that dereferences it
  to prevent a nil-pointer panic when Debug is enabled.
- Add nonociurl validator to RepoRelease.URL so oci:// URLs are
  explicitly rejected at validation time, not just by dispatcher ordering.
- Fix misleading test: include name field so the test actually verifies
  scheme-level rejection rather than passing due to missing required field.

* fix: use Accessor interface in isInstalled and parallel-safe env in tests

Use release.NewAccessor() instead of type-asserting to *releasev1.Release
in isInstalled(), making it resilient to future Helm release types.
Replace os.Setenv with GinkgoT().Setenv() for parallel-safe test execution.

* fix: add ContentCache to ChartDownloader for Helm v4

Helm v4's ChartDownloader requires the ContentCache field to be set,
separate from RepositoryCache. Without it, DownloadTo() fails with
"content cache must be set". Pass settings.ContentCache through from
the cli.EnvSettings, which defaults to $HELM_CONTENT_CACHE or
~/.cache/helm/content.

* fix: disable SSA for Helm v4 to preserve client-side apply semantics

Helm v4 defaults to Server-Side Apply (SSA) for new installs (HIP-0023).
SSA uses field-level merge instead of full object replacement, which
breaks when upgrading charts that change a Deployment's strategy.

Specifically: when a chart creates a Deployment without specifying
strategy (defaulting to RollingUpdate), the API server auto-populates
strategy.rollingUpdate with defaults that no field manager owns. A
subsequent SSA upgrade setting strategy.type=Recreate cannot remove the
unowned rollingUpdate field, and the API server rejects the result:
"spec.strategy.rollingUpdate: Forbidden: may not be specified when
strategy type is 'Recreate'"

Disable SSA (ServerSideApply=false) on Install, Upgrade, and Rollback
to use client-side apply (full object replacement), which cleans up
stale fields and matches Helm v3 behavior.

Use HookOnlyStrategy instead of StatusWatcherStrategy/LegacyStrategy.
The operator's own reconcile loop already monitors resource readiness.
Having Helm also wait via StatusWatcherStrategy causes the reconciler
to re-enter while a Helm wait is in progress, producing "another
operation (install/upgrade/rollback) is in progress" errors.
HookOnlyStrategy waits only for pre/post hooks, then returns control
to the operator.

* chore: update test manifest for Helm v4 testing

Fix app.resources.memory indentation (was sibling of requests, not
child). Bump chart to v0.41.3 (v0.32.4 has duplicate env keys rejected
by Helm v4). Add glue strategy and resource limits. Disable
settingsMigrationJob.

* chore: add orbstack to Tiltfile allowed contexts

* chore: go mod tidy and update dependencies

Update controller-runtime v0.23.1→v0.23.3, validator v10.26.0→v10.30.2,
gomega v1.39.0→v1.39.1, and transitive dependencies.

* fix: use LegacyStrategy for Uninstall to wait for resource deletion

HookOnlyStrategy's WaitForDelete() is a no-op, which means Uninstall
returns success while resources are still terminating. This could cause
Prune() to report success prematurely. Use LegacyStrategy for Uninstall
to actually wait for resources to be deleted, while keeping
HookOnlyStrategy for Install/Upgrade/Rollback where the operator's
reconcile loop handles readiness.

* cleanup

* feat: add local OCI registry testing infrastructure

Add Makefile targets for local OCI registry with basic auth:
- make local-registry: starts registry:2 on port 5050 with htpasswd auth
- make local-registry-push: pulls chart from charts.wandb.ai and pushes
  to local registry (version read from oci.yaml manifest)
- make local-registry-secret: creates K8s secret with registry creds
- make local-registry-stop: tears down the registry

Add hack/testing-manifests/wandb/oci.yaml for testing OCI chart pulls
with credential secret authentication.

* fix: upgrade google.golang.org/grpc to v1.79.3 for CVE-2026-33186

Fixes GO-2026-4762 / GHSA-p77j-4mvh-x3m3: authorization bypass in
gRPC-Go caused by improper validation of the HTTP/2 :path pseudo-header.
All versions before v1.79.3 are affected.

* fix: use scheme-based TLS verification instead of unconditional bypass

The getter option WithInsecureSkipVerifyTLS was hard-coded to true,
disabling TLS certificate validation for all chart downloads including
HTTPS. Use the already-computed entry.InsecureSkipTLSVerify which is
true only for HTTP URLs, preserving TLS verification for HTTPS.

* fix: validate credential secret keys exist before using them

When loading credentials from a Kubernetes secret, check that the
expected keys exist and are non-empty. Previously, missing keys would
silently produce empty username/password strings, leading to opaque
auth failures when pulling from the registry.

* fix: handle stopped registry containers and explicit namespace for secret

Remove stopped containers before starting a new registry to prevent
name conflicts. Add explicit namespace (WANDB_NAMESPACE, default:
default) to the credential secret creation.

* fix: reject partial OCI auth credentials (username xor password)

When only one of username or password is provided, the code silently
falls back to anonymous pull. Fail fast with a clear error so
misconfiguration is immediately visible.

* chore: add orbstack to docs and sample tilt settings

Add orbstack to the allowed Kubernetes contexts in README and
tilt-settings.sample.json. Minor formatting cleanup in README.

* fix(chart): Use helm to validate references for versions and digest refs

* fix(dependency): Bump helm go module for security fix

* fix(dependency): Bump golang.opentelemetry.io packages for security fix

* fix(deps): tidy go.sum to drop superseded module versions

Removes stale entries for older grpc-gateway, otelhttp, otel, and
x/net/oauth2 versions that were no longer referenced after recent
dependency bumps, resolving the Dependency Check CI job.

---------
Co-authored-by: casey-coreweave <cabernathy@coreweave.com>
## [1.22.0](v1.21.3...v1.22.0) (2026-04-30)

### Features

* Add OCI Helm chart registry support and upgrade to Helm v4 ([#147](#147)) ([fcb72b8](fcb72b8))
Co-authored-by: wandb-renovate[bot] <285832833+wandb-renovate[bot]@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9364aa9f-c974-4b20-bbe9-6981094fc893

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch wnevis/unifyLabels

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@wnevis-cmyk wnevis-cmyk changed the title Unify labels for pods feat(backend): unify labels for pods Jul 8, 2026
@wnevis-cmyk wnevis-cmyk changed the title feat(backend): unify labels for pods feat(backend): Unify labels for pods Jul 8, 2026
@wnevis-cmyk

Copy link
Copy Markdown
Contributor Author

Totally happy to scrap the doc, but I thought it'd be a little easier than digging through the code

Comment on lines +47 to +55
var appComponentRoles = map[string]string{
"executor": RoleWorker,
"parquet": RoleWorker,
"weave-trace-worker": RoleWorker,
"weave-trace-evaluate-model-worker": RoleWorker,
"flat-run-fields-updater": RoleWorker,
"metric-observer": RoleWorker,
"nginx-proxy": RoleProxy,
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid hard-coding these?

jthakkar04 and others added 12 commits July 14, 2026 16:04
Co-authored-by: Daniel Panzella <daniel.panzella@wandb.com>
Co-authored-by: jonathan meeks <jonathan.meeks@wandb.com>
Co-authored-by: Kevin Chen <140976422+wandb-kc@users.noreply.github.com>
Co-authored-by: Jonathan Meeks <jonathanmeeks@gmail.com>
Co-authored-by: Jonathan Meeks <jmeeks@coreweave.com>
Co-authored-by: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
Co-authored-by: amwarrier <aravind.mohanakumar.warrier@gmail.com>
Co-authored-by: Zachary Blasczyk <77289967+zacharyblasczyk@users.noreply.github.com>
Co-authored-by: semantic-release-bot <semantic-release-bot@martynus.net>
Co-authored-by: wnevis-cmyk <wnevis@coreweave.com>
Co-authored-by: Jagat Thakkar <32109558+jthakkar04@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Collin Olander <colander@coreweave.com>
dependabot Bot and others added 25 commits July 21, 2026 12:35
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…2 to 6.12.2 (#255)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
….33 (#264)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… to 10.30.3 (#258)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…tlptracehttp from 1.35.0 to 1.43.0 (#265)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
@wnevis-cmyk
wnevis-cmyk requested a review from a team as a code owner August 5, 2026 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants