test: measure image build with registry layer cache

[marians]

Not for merge

Measurement branch for giantswarm/architect-orb#871.

push-to-registries has been taking a flat ~5m32s here (5m31s / 5m32s / 5m34s on #2008 / #2010 / #2011 — 3 seconds of variance across three runs is the tell that nothing is cached). It is not multi-arch emulation: both push-to-registries and push-to-registries-release pin platforms: linux/amd64, so no RUN step was ever emulated.

The cause is that the orb had no way to persist BuildKit's layer cache. Each job gets a fresh setup_remote_docker VM and a fresh docker-container builder, so every build re-runs the whole Dockerfile: apt-get install build-essential, pip install mkdocs-techdocs-core, and a full yarn workspaces focus --all --production including node-gyp native compiles. The --mount=type=cache directives in packages/backend/Dockerfile only ever helped local rebuilds.

Changes (both test-only)

• .circleci/workflows.yml: orb → giantswarm/architect@dev:buildkit-registry-cache, and cache: registry on push-to-registries.

Note this touches a generated file (devctl gen circleci). Landing it for real needs a gen.ci.image change in devctl — custom.yml can override the orbs: map, but workflows.build.jobs is a list that the yq *+ merge appends to, so it cannot add a parameter to an existing job entry.

Expected

• This run (cold): ~5m30s or slightly more — the cache ref doesn't exist yet, so it's a full build plus a cache export.
• Next run (warm): substantially faster. skeleton.tar.gz is written with portable: true, noMtime: true (@backstage/cli-module-build, createDistWorkspace.cjs.js:150-151), so it's byte-identical unless a workspace package.json changes — which means the layer above yarn workspaces focus should actually hit.

Caveat that will remain even once this lands: a yarn.lock or package.json change misses that layer and re-runs the install against a cold cache mount, so dependency bumps still cost full price.

---

Results

run	push-to-registries
baseline (main, orb 9.6.0, no cache)	~5m32s
cold (cache written, QEMU skipped)	5m15s
warm (cache hit)	2m03s

3m29s faster — 2.7×. The prediction held: skeleton.tar.gz being byte-stable means the yarn workspaces focus --all --production layer hits on a source-only change.

The cold run beat baseline by 17s despite uploading 554 MB of cache — that's the QEMU gating in the same orb PR, since the tonistiigi/binfmt pull it now skips cost about what the cache export added.

Verified the speedup is real rather than a skipped step: the cache landed as a proper application/vnd.oci.image.manifest.v1+json / vnd.buildkit.cacheconfig.v0 artifact (24 blobs, 554 MB), and the warm run pushed a genuine image (…11-53-40.hc44a292).

Remaining caveat

A yarn.lock or package.json change invalidates that layer and re-runs the install against a cold cache mount, so dependency bumps still cost the full ~5m30s. That's inherent — cache mounts aren't exportable, only layers.

To land this for real

This branch edits the devctl-generated .circleci/workflows.yml, so it is not mergeable as-is. custom.yml can override the orbs: map (it's a map), but workflows.build.jobs is a list that the yq *+ merge appends to, so it cannot add cache: registry to an existing job entry. Landing it needs a gen.ci.image option in devctl, plus a released architect-orb containing giantswarm/architect-orb#871.

---

Updated results (reviewed orb revision)

run	push-to-registries
baseline (main, orb 9.6.0, no cache)	~5m32s
cold, cache written	5m15s
warm	2m03s
re-run after base-image republish	5m40s (cold — confounded)
warm, reviewed orb, stable base	2m08s

The 5m40s run was not a regression: node:24-trixie-slim was republished at 2026-08-05T08:39:10Z, which moves the FROM digest and invalidates every layer beneath it.

This repo should pin its base image

packages/backend/Dockerfile uses the floating node:24-trixie-slim. Every upstream republish (Debian security rebuilds land in -slim tags often) produces a fully cold build regardless of the cache. Pinning by digest with Renovate managing the bump would make cache hits consistent — and is worth doing for reproducibility independently of this work. Not included here, since this branch is measurement-only.

Note on the release job

cache: registry is set on the branch job only, and that is now the orb's documented recommendation rather than an accident: release-tag builds are cosign-signed and carry a provenance attestation, and should not be assembled from a cache any repo sharing the push credentials can write. See the trust-model section in the orb docs.