Pin package-manager dep stores as mirrored Sources — kill in-build network fetches (pilot: next)

[bryan-minimal]

Motivation — data from the first fully-instrumented world rebuild (2026-07-14, pkgs#402)

The sqlite-cascade world rebuild ran with the new observability stack live (15s host metrics, per-package spans, phase strip). The per-instance CPU picture:

instance	profile during the run
res-server-amd64 (c4-standard-96)	pegged 100% for ~40 min — genuinely CPU-saturated during the parallel bulk
res-server-arm64 (c4a-standard-72)	99% bursts separated by ~20 min at 3–15% (≈3–10 busy cores of 72)

The arm64 trough is named from the build timeline: next (22:48→22:58, ~10 min) then rust (22:58→23:10, ~11 min) — near-serial, network-bound work (pnpm install fetching from the registry; toolchain repackaging), during which the plan had no other ready work. Conclusion: bigger instances don't buy back the tail — more cores just idle harder through the troughs. Lifting the in-build network fetches out of the build does.

75 packages declare needs.internet today; next is the worst measured offender.

The pattern — pin the dep store as a mirrored Source

At version-bump time, warm the package manager's store once; pin it content-addressed on the mirror; build offline:

1. pnpm install --ignore-scripts --store-dir ./pnpm-store in the package source (install, NOT pnpm fetch — see pitfall below)
2. tar | zstd the store → upload to gs://minimal-staging-archives/... with sha256
3. build.ncl gains the store tarball as a pinned Source; build.sh runs pnpm install --offline --frozen-lockfile --store-dir ...
4. drop needs.internet from the package

The res-server then prefetches the store like any Source — content-addressed, at GCS line speed, in parallel with everything else. (The Gentoo --fetchonly move, made structural: fetches belong to declared inputs, not build execution.)

Feasibility — validated twice

A. Local pilot (2026-07-14, next.js v16.2.6, the exact pinned source):

phase	result
pnpm fetch from lockfile	40s, 2.2GB store — but PARTIAL: misses the 4 git-pinned deps (ERR_PNPM_NO_OFFLINE_TARBALL)
warm via online pnpm install --ignore-scripts	completes the store (git deps + the packageManager binary itself)
offline install, pristine tree	✅ 3,822 packages, downloaded 0, 17.7s
negative control (--offline + empty store)	fails correctly — the pass above is real
mirror artifact	513MB tar.zst (in family with chromium-bin's 225MB / jdk's 200MB objects)

B. Production precedent — gominimal/minimermetic already ships this (fetcher/ecosystem_cache.go, 2026-05-19): pnpm + bun ecosystem-cache resolvers doing exactly steps 1–3, uploading to gs://<mirror>/pnpm-store/<pkg>-<version>.tar.zst + a sha256/<hex> CAS path, with the hermetic builder hydrating by tarball sha (PNPM_STORE_SHA256 via task manifests). Its code comment documents the identical pitfall the pilot hit independently:

"pnpm install --no-frozen-lockfile --ignore-scripts, NOT pnpm fetch: fetch leaves a PARTIAL store (skips git-source deps …) — install populates the content-addressable store COMPLETELY."

Also borrowable from minimermetic: goresolver.go (go.sum → module set with h1 hash-chain verification) and the cargo-vendor / npm-cache builder-side wiring; and sandbox2 already carries the offline-cache-bundle hardlink machinery (the MINIMAL_INTERNAL_CS_BUILD cs-mirror path), proving the idiom in this exact sandbox.

Proposed adaptation for pkgs (differs from minimermetic in one way)

Express the store tarball as a build.ncl Source rather than env-var hydration: it rides the existing Source/mirror machinery, spec_hash captures it automatically (store bump ⇒ rebuild, correctly), and the corresponding-source check (pkgs#415) sees it like any other archive.

• Warm on linux (minimermetic's fetcher runs on a Linux VM for the same reason) or set supportedArchitectures: {os: [linux], cpu: [x64, arm64]} in pnpm-workspace.yaml (pnpm 11 no longer reads the package.json pnpm field) so one store serves both arches.
• Warm at version-bump time via pkgmgr — natural fit with gominimal/pkgmgr-rs#513 (mirror-sync on update PRs).

Pilot plan

1. next first (worst measured offender): add the store Source + --offline build, drop needs.internet.
2. Validate locally before CI: minimal package next (arm64-native sandbox build on a Mac) proves the offline build end-to-end.
3. Watch for: install-scripts that fetch at build time (modern sharp ships prebuilts as npm packages, so expected-fine — but this is what step 2 verifies).
4. After the pkgs#402 flamegraph review, extend to the next candidates (rust/go/bun ecosystems have the same shape and minimermetic has resolvers for each).

Expected payoff: ~8 min/arch per world rebuild from next alone. Side effects, all free: one fewer needs.internet package (hermeticity), the store tarball is the embedded-deps corpus for attribution/SBOM (inbox#282 phase 2), and corresponding source for the JS deps (inbox#283).

Refs: inbox#282 · inbox#283 · gominimal/pkgmgr-rs#513 · minimermetic fetcher/ecosystem_cache.go