Skip to content

Consolidate ArgoCD node pinning into chart global.affinity#55

Merged
venkatamutyala merged 6 commits into
mainfrom
unify-argocd-node-affinity
Jul 3, 2026
Merged

Consolidate ArgoCD node pinning into chart global.affinity#55
venkatamutyala merged 6 commits into
mainfrom
unify-argocd-node-affinity

Conversation

@venkatamutyala

@venkatamutyala venkatamutyala commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What

Pin every ArgoCD pod to the glueops.dev/role: glueops-platform nodes using the chart's global.affinity preset instead of repeating a nodeSelector + nodeAffinity block on each component. Targets the deployed chart 9.x (verified on 9.3.7 and 9.7.1). Net −36 lines.

Change (single file: argocd.yaml.tpl)

  • global: add
    affinity:
      podAntiAffinity: soft
      nodeAffinity:
        type: hard              # -> requiredDuringSchedulingIgnoredDuringExecution
        matchExpressions:
          - {key: glueops.dev/role, operator: In, values: [glueops-platform]}
    (global.tolerations already carried the glueops-platform toleration.)
  • Removed the per-component nodeSelector + affinity from server, applicationSet, notifications, repoServer, and deleted the redisSecretInit block — they now inherit global.

Left overriding global on purpose

  • controller — keeps its affinity (dedicated glueops-platform-argocd-app-controller pool preferred + glueops-platform fallback) and two-taint tolerations.
  • redis-ha — subchart, not reachable by global; keeps nodeSelector + tolerations + additionalAffinities (also pins its helm-test pods).
  • gatekeeper — raw extraObjects manifest; keeps its own nodeSelector + tolerations.

Why not global.nodeSelector

It would AND onto the controller (whose nodeSelector is empty) and exclude its dedicated pool — a nodeSelector can't express the OR the controller's nodeAffinity does. So pinning is via global.affinity.nodeAffinity.

Verification (helm template)

11/11 pod-producing workloads pinned, 0 unpinned on both 9.3.7 (deployed) and 9.7.1; controller keeps req=[app-controller, glueops-platform], pref=[app-controller], both tolerations. Independently reviewed by a Helm expert (approved).

Behavior notes

  • server/applicationSet/notifications gain a soft own-pod host-spread (from the global preset) — preferred-only, good for HA.
  • repoServer loses the moot weight:50 "spread off app-controller" term (controller lives on its own pool); own-pod spread preserved via the global soft preset.
  • Existing pods won't migraterequiredDuringSchedulingIgnoredDuringExecution is schedule-time only. After merge + Argo sync, kubectl -n glueops-core rollout restart the affected workloads so they reschedule. Ensure the glueops-platform pool has ≥3 nodes (redis-ha's required host anti-affinity).
  • metrics.enabled stays per-component (there is no global.metrics). No redis metrics changes.

Follow-up (separate, not in this PR)

README.md still documents installing chart 5.50.0 while the cluster runs 9.3.7+ — worth a doc update.

🤖 Generated with Claude Code

venkatamutyala and others added 3 commits July 2, 2026 07:57
The application controller was the only component pinned to a dedicated
`glueops-platform-argocd-app-controller` node pool (with `glueops-platform`
as fallback) plus an extra matching toleration. Every other component
(server, applicationSet, repoServer, notifications, redis-ha) targets
`glueops.dev/role: glueops-platform` and inherits the `glueops-platform`
toleration from `global.tolerations`.

Align the controller with the rest so all pods the chart deploys use the
same node selector/affinity rules: it now requires `glueops-platform` and
inherits `global.tolerations` like the other components.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The redis-ha server and haproxy pods were already pinned via
additionalAffinities, but the subchart's helm-test pods
(redis-ha-configmap-test, redis-ha-service-test) rendered with an empty
nodeSelector and no affinity, so they were not constrained to the
glueops-platform nodes. Those test pods only honor `redis-ha.nodeSelector`,
not `additionalAffinities`.

Set `redis-ha.nodeSelector: glueops.dev/role=glueops-platform`. Verified
with `helm template` (argo-cd 5.50.0): the test pods now carry the
nodeSelector, and the server/haproxy pods gain a matching (redundant with
their existing required nodeAffinity) nodeSelector. All pod-producing
workloads now target glueops-platform.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per review feedback, restore the application controller to its original
configuration: it keeps its dedicated `glueops-platform-argocd-app-controller`
node pool (preferred, with `glueops-platform` as fallback) and tolerations for
both taints. The dedicated-pool setup is the preferred arrangement.

Also add an explicit `repoServer.nodeSelector: glueops.dev/role=glueops-platform`
alongside its existing required nodeAffinity, guaranteeing repo-server pods
stay on the platform regardless of the podAntiAffinity spreading hints.

Verified with `helm template` (argo-cd 5.50.0): every pod-producing workload
targets glueops-platform; the app-controller again prefers its dedicated pool.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@venkatamutyala venkatamutyala changed the title Unify node affinity/tolerations across all ArgoCD components Keep all ArgoCD redis-ha and repo-server pods on the platform nodes Jul 2, 2026
venkatamutyala and others added 3 commits July 2, 2026 08:44
Extend the nodeSelector treatment (already on repoServer and redis-ha) to the
remaining argo-cd components so every pod that can be pinned via nodeSelector
is, in addition to its existing required nodeAffinity.

The application controller intentionally keeps NO nodeSelector: a nodeSelector
is a hard glueops-platform requirement that would exclude its dedicated
glueops-platform-argocd-app-controller pool. It continues to rely on its
nodeAffinity (dedicated pool preferred, glueops-platform fallback).

Verified with `helm template` (argo-cd 5.50.0).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rendered the current values against argo-cd chart 5.50.0 (pinned), 8.6.4,
and 9.7.1. All existing per-component keys (nodeSelector on server,
applicationSet, notifications, repoServer, redis-ha; controller affinity)
render identically across all three majors -- no schema breakage.

v8/v9 introduce a new `argocd-redis-secret-init` Job (a pre-install/
pre-upgrade helm hook, run by Argo CD as a PreSync hook) that inherits
global.tolerations but has no node targeting, so it would schedule on any
node. Set `redisSecretInit.nodeSelector` to keep it on the platform.

Harmless no-op on 5.50.0 (no redisSecretInit component; no strict values
schema in any of the three charts). Verified: 0 unpinned pod-producing
workloads on 5.50.0, 8.6.4, and 9.7.1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the per-component nodeSelector + nodeAffinity boilerplate on server,
applicationSet, notifications, repoServer, and redisSecretInit with a single
chart-level global.affinity preset (podAntiAffinity: soft, nodeAffinity:
type=hard for glueops.dev/role=glueops-platform). global.tolerations already
carried the glueops-platform toleration.

How the chart merges (argo-cd.affinity helper, verified in templates/_common.tpl):
components without their own affinity inherit global.affinity; components that
set affinity override it entirely; nodeSelector/tolerations are
`component | default global`.

Left overriding global on purpose:
- controller: keeps its affinity (dedicated glueops-platform-argocd-app-controller
  pool preferred + glueops-platform fallback) and two-taint tolerations.
- redis-ha: subchart, not reachable by global; keeps nodeSelector + tolerations +
  additionalAffinities (also pins its helm-test pods).
- gatekeeper: raw extraObjects manifest; keeps its own nodeSelector + tolerations.

global.nodeSelector is intentionally NOT used: it would AND onto the controller
and exclude its dedicated pool (nodeSelector can't express the OR the controller's
nodeAffinity does), so pinning is via global.affinity.nodeAffinity instead.

Verified with `helm template` on 9.3.7 (deployed) and 9.7.1: 11/11 pod-producing
workloads pinned to glueops-platform, 0 unpinned, controller dedicated pool +
both tolerations intact. Independently reviewed by a Helm expert (approved).

Note: existing pods won't migrate (nodeAffinity is IgnoredDuringExecution) --
roll the workloads after sync. metrics.enabled stays per-component (no
global.metrics exists). No redis metrics changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@venkatamutyala venkatamutyala changed the title Keep all ArgoCD redis-ha and repo-server pods on the platform nodes Consolidate ArgoCD node pinning into chart global.affinity Jul 3, 2026
@venkatamutyala venkatamutyala merged commit d3ed075 into main Jul 3, 2026
1 of 2 checks passed
@venkatamutyala venkatamutyala deleted the unify-argocd-node-affinity branch July 3, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant