Skip to content

fix(imagehub): stop rotating image registry robot secrets on every reconcile - #88

Draft
pranludi wants to merge 1 commit into
developfrom
fix/image-registry-secret-unnecessary-rotation
Draft

fix(imagehub): stop rotating image registry robot secrets on every reconcile#88
pranludi wants to merge 1 commit into
developfrom
fix/image-registry-secret-unnecessary-rotation

Conversation

@pranludi

Copy link
Copy Markdown
Contributor

Summary

ImageRegistrySecretReconciler called refreshSecret() (via AipubDockerConfigJsonResolver) unconditionally every time it reconciled a project's .dockerconfigjson Secret — not just at first creation. This fires on:

  • every project-controller restart (informer resync re-lists all Projects, bypassing the update filter)
  • any change to spec.binding.imageHubs, including adding a second, unrelated ImageHub to an already-connected project

Harbor robot permissions are updated independently of the robot's secret (PUT /robots/{id} vs PUT /robots/{id}/refreshsecret), so this rotation was never actually necessary once a working Secret already existed — it just invalidated whatever password was currently in use, with no warning.

Confirmed live on cluster12: created a project, connected one ImageHub, docker login with the resulting secret succeeded. Added a second, unrelated ImageHub binding to the same project (no restart, no crash) — the previously-working password immediately stopped authenticating, while the new one (silently rotated in) worked fine.

Fix

Track which robot's secret is currently reflected in the K8s Secret via a new project.aipub.ten1010.io/image-registry-robot-id annotation. Only call refreshSecret() when:

  • the Secret doesn't exist yet (first creation), or
  • the project's current robot id differs from what's recorded on the existing Secret (i.e. the robot was actually deleted/recreated)

Otherwise, reuse the existing Secret's .dockerconfigjson data as-is — permission-only changes no longer touch the password at all.

Note on rollout

Secrets created before this change won't have the annotation yet, so each will get one extra (harmless) rotation the first time it's reconciled after this ships. After that they settle into the new behavior permanently and stop rotating on unrelated changes.

Follow-up (separate, cross-repo — not in this PR)

Even after this fix, first-time robot creation still costs two Harbor round-trips (create, then refresh-secret) because aipub-backend discards the plaintext secret Harbor already returns at creation time, at three layers (backend-api's Harbor client call, the create gRPC response, and the gateway REST controller returning Void). Propagating that through would let project-controller seed the Secret directly from the create response. Tracked separately since it requires aipub-backend changes too — marking this PR as draft until that's scoped/decided, and this one is deliberately self-contained without a testing plan.

Test plan

  • ./gradlew compileJava / ./gradlew build -x test pass
  • Manually verify: connect ImageHub → login works → add second ImageHub → login with the same password still works (previously would have broken)
  • Verify a pre-existing Secret (created before this PR) gets the one-time backfill rotation + annotation on its next reconcile, then stops rotating on subsequent unrelated changes

🤖 Generated with Claude Code

…concile

ImageRegistrySecretReconciler called AipubDockerConfigJsonResolver's
refreshSecret() unconditionally every time it reconciled a project's
.dockerconfigjson Secret — including on every project-controller restart
(informer resync re-lists all Projects) and any unrelated change to
spec.binding.imageHubs (e.g. adding a second, unrelated ImageHub). Harbor
robot permissions are updated independently of the robot's secret, so
this rotation was never actually necessary once a working Secret already
existed; it only invalidated whatever was previously working.

Confirmed live on cluster12: adding a second ImageHub binding to an
already-connected project immediately broke docker login with the
previously-valid robot password, even with zero restarts/crashes.

Fix: track which robot's secret is currently reflected in the K8s Secret
via a new project.aipub.ten1010.io/image-registry-robot-id annotation.
Only call refreshSecret() when the project's current robot id differs
from what's recorded (i.e. the robot was actually recreated) or when the
Secret doesn't exist yet. Otherwise reuse the existing Secret data as-is.

Note: pre-existing Secrets created before this change won't have the
annotation yet, so they'll get one extra (harmless) rotation the first
time they're reconciled after this fix ships, after which they settle
into the new behavior permanently.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@pranludi
pranludi changed the base branch from main to develop August 6, 2026 04:44
@pranludi pranludi self-assigned this Aug 6, 2026
@pranludi pranludi added this to the v5.1.0 milestone Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant