Skip to content

perf(ci): reuse build results across workflows - #810

Open
donbeave wants to merge 215 commits into
mainfrom
perf/subminute-ci
Open

perf(ci): reuse build results across workflows#810
donbeave wants to merge 215 commits into
mainfrom
perf/subminute-ci

Conversation

@donbeave

@donbeave donbeave commented Jul 16, 2026

Copy link
Copy Markdown
Member

Summary

This PR makes jackin❯ CI reuse previously completed work instead of downloading and recompiling unchanged dependencies, then standardizes the complete workflow estate on Velnor-default execution with an explicit GitHub Ubuntu 26.04 parity lane. Maintainers retain one independently readable complete test job per affected crate while exact-result proofs skip unchanged producers, prepared artifacts tolerate measured producer latency, and writer gates keep publication single-owner.

What ships

  • The CI workflow again produces complete jackin-role validator archives for both Linux targets on the Velnor-default lane; PRs exercise the read-only build, main publishes one writer-owned artifact per target, and checksum sidecars satisfy the consumer’s sha256sum --check contract.

  • Each crate and feature mode owns one stable v7 Cargo target seed with a consistent debugless, non-incremental profile; a warm dependency seed initializes the first complete crate job, which publishes the canonical result.

  • The prepared jackin-xtask binary owns target discovery, validation, restoration, packing, publication inputs, workflow performance auditing, semantic Docs contracts, and prepared toolchain repair.

  • Prepared-artifact consumers wait across the complete observed producer-latency envelope and report their true remaining deadline; action-level tests cover immediate, delayed, invalid, absent, and timed-out artifacts.

  • Documentation examples are parser-audited: runnable Rust fences are rejected unless their executable proof lives in nextest, keeping all Rust test execution on the nextest contract. Docs tooling is resolved through the repository's mise configuration.

  • CI, Docs, Construct, Preview, Release, and jackin-dev preserve exact-result reuse while automatic and omitted dispatch selection now resolve to the Velnor lane.

  • Every runner-selectable dispatch orders velnor, github, and both; GitHub records use ubuntu-26.04, and all workflow utility jobs use the same pinned Ubuntu image.

  • The standalone native-macOS hygiene job is removed. Existing release and preview Apple artifacts remain Linux cross-builds through the repository's cargo-zigbuild archive machinery.

  • The duplicate Rust target-cache layer is removed across compile workflows. The shared Cargo registry cache and prepared sccache v0.16.0 tool set remain the runner-neutral cache contract with a 20 GiB cap.

  • Explicit sccache-statistics workflow steps and their composite are removed; cache reporting belongs to setup/post behavior rather than a build step.

  • Every workflow now has concurrency control, every concrete job has a timeout, and every retained full-history checkout documents its consumer.

  • Workflow guidance records Velnor-default selection, Ubuntu 26.04, the three-lane contract, and the single compiler-cache stack.

Behavior changes

  • Pull requests, pushes, and dispatches without a lane input run on Velnor instead of GitHub-hosted runners.
  • lanes=github remains the explicit GitHub-hosted verification path; lanes=both retains lane parity with one writer.
  • Input-identical workflow results continue to skip expensive producers entirely. The previously measured final CI repeat completed end to end in 19 seconds with all warmup, policy, audit, format, routing, and crate-test jobs structurally skipped.
  • A consumer now waits up to 120 seconds for a concurrently produced prepared artifact, covering the observed 69-second producer publication without changing missing-artifact failure semantics.
  • The previously measured dual-lane run completed every job below one minute: warmup took 12 seconds on GitHub and 36 seconds on Velnor, with no dependency, toolchain, or registry downloads and no dependency compilation in raw logs.
  • Cached Preview parity retains one complete job per crate and lane: GitHub capsule completed in 19 seconds, GitHub jackin❯ in 20 seconds, Velnor capsule in 28 seconds, and Velnor jackin❯ in 40 seconds.

What this addresses

  • Removes repeated dependency downloads and third-party compilation from input-identical runs while preserving precise crate-scoped failures.
  • Eliminates the previous GitHub-default, mixed Ubuntu, macOS-runner, double-cache, and build-step cache-reporting drift.
  • Prevents fast consumers from declaring a prepared artifact absent while its valid producer is still publishing.
  • Replaces opaque target batches, pointer artifacts, and sequential log processing with Rust-owned orchestration and one archive operation per crate result.
  • Makes performance regressions visible at the job, step, and log-marker level instead of treating a green but slow pipeline as acceptable.

Hard rule: complete crate jobs with Rust-owned orchestration

Every affected crate remains one independently readable job that runs its complete applicable suite. CI optimizes reuse without splitting a crate into shards, batches, or numbered parts, and workflow run blocks call prepared Rust tooling or a direct tool command rather than implementing CI decisions, parsing, cache contracts, or artifact transport as Bash programs.

Verification evidence

Exact head f928092efde4305501c9276449df16cbeff94b63:

  • Prepared-artifact action tests — 6 passed; the bundled action was regenerated from the tested source.
  • CI run 29874561294 — passed on GitHub, including the complete Rust and policy graph.
  • Construct run 29874563229 — passed on GitHub.
  • Docs run 29875308785 — passed on GitHub, including repository links, documentation links, spelling, and the required aggregator.
  • REUSE run 29875795245 — passed on GitHub.
  • Renovate Validate run 29875838995 — passed on GitHub.

Historical timing remains useful for the unchanged reuse graph: the no-change CI proof completed in 19 seconds end to end; the prior dual-lane warmup took 12 seconds on GitHub and 36 seconds on Velnor; cached Preview crate jobs completed in 19–20 seconds on GitHub and 28–40 seconds on Velnor. Fresh Velnor and Preview parity evidence remains pending the signed-apt fleet migration and the separately approval-gated RFC3161 attestation capability; no workflow was weakened to manufacture that proof.

Deviations

  • PR perf(ci): reuse build results across workflows #810's prepared-tool artifact and result-reuse machinery remains intact as required. It already pins sccache v0.16.0 in mise.toml and transports that prepared binary without per-job installs; replacing it with unconditional setup actions would regress the proven no-change path by installing tools in structurally skipped jobs. The standard is therefore applied through the existing Rust-owned preparation boundary rather than duplicating setup actions.
  • Apple artifact names and Homebrew formula keys retain macos identifiers because they describe supported artifact targets, not runner operating systems. No workflow job runs on macOS.
  • .github/actionlint.yaml declares ubuntu-26.04 and ubuntu-26.04-arm because actionlint v1.7.12 predates those GitHub-hosted labels; this suppresses only its stale label catalog, not workflow syntax or expression checks.

Not included

  • The PR does not expand the strict attestation capability. RFC3161 TSA trust, inputs, network behavior, and verification remain subject to separate explicit operator approval.
  • A remote sccache backend remains outside the approved local-only compiler-cache contract.

Verify locally

Checkout

Paste this first to bypass the tirith paste scanner for the rest of the session:

export TIRITH=0

Then paste the checkout block:

jackin-dev pr sync 810
cd "$(jackin-dev pr path 810)/jackin"
source "$(jackin-dev pr path 810)/env.sh"
which jackin

Static checks

actionlint .github/workflows/*.yml
cargo fmt --check
cargo clippy -p jackin-xtask --all-targets -- -D warnings

Rust tests

cargo nextest run -p jackin-xtask

Docs checks

(
  cd docs
  bun install --frozen-lockfile
  bun run build
  cargo xtask docs repo-links
  cargo xtask roadmap audit
  bunx tsc --noEmit
  bun test
)

User smoke

cargo run -p jackin-xtask --locked -- lint ratchet

Confirm the Rust-owned CI contracts pass without numbered shards or batches.

jackin-capsule smoke

jackin load the-architect . --debug

Inside the container, verify:

  • Row 0 status bar is visible: jackin❯ [the-architect].
  • Agent TUI starts and renders correctly below the status bar.
  • Ctrl+\\ opens the command palette.
  • Mouse clicks, arrow keys, and paste reach the agent unmodified.
  • OSC policy and daemon subsystem behavior remain unchanged after the CI-focused test updates.

Documentation

(
  cd docs
  bun install --frozen-lockfile
  bun run dev
)

Vite serves at http://localhost:3000/. Pages to walk:

http://localhost:3000/reference/research/ci/performance/ci-speed-roadmap/
UPDATED. Confirm the compiler-cache and exact-result reuse contract matches the workflow implementation.

http://localhost:3000/roadmap/infrastructure/shared-ci-compiler-cache/
UPDATED. Confirm roadmap status and local-only cache boundaries render correctly.

Migration notes

None.

donbeave and others added 8 commits July 16, 2026 21:19
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
@donbeave donbeave changed the title perf(ci): collapse crate target transport perf(ci): reuse crate targets and docs proofs Jul 16, 2026
donbeave and others added 7 commits July 16, 2026 22:07
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Version the debugless Cargo profile as target contract v7 so incompatible build generations cannot accumulate in one archive. Share the warmup producer's dependency outputs with the complete per-crate test job.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Use attempt-scoped GitHub job APIs with validated retries so transient HTML 503 responses cannot break required aggregators. Fetch job logs concurrently to keep the complete audit below the one-minute budget.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
GitHub returns 404 for skipped jobs because no log archive exists. Keep missing logs fatal for executed jobs while excluding semantically logless skipped jobs from the concurrent audit.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
@donbeave donbeave changed the title perf(ci): reuse crate targets and docs proofs perf(ci): reuse build results across workflows Jul 16, 2026
donbeave and others added 13 commits July 16, 2026 23:26
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Co-authored-by: Codex <codex@openai.com>
@donbeave

Copy link
Copy Markdown
Member Author

Final exact-head GitHub evidence for acaabf0db7353e70622065629b3a9ecc7fc26d79:

CI:

jackin-dev cross-release:

All four cross targets restore distinct exact-SHA target caches. No dependency compilation occurs on no-change; each target recompiles only the workspace jackin-dev crate after fresh-checkout mtime invalidation. CI uses nextest and all aggregators pass. Actionlint and estate audit are clean; no review threads remain.

Velnor/both proof awaits signed-apt fleet access. GHA-backed compiler cache remains the explicit human decision for eliminating workspace recompilation on ephemeral GitHub hosts.

donbeave and others added 24 commits July 22, 2026 17:50
Resolve workflow conflicts favoring PR CI reuse structure
(rustup-v2, download-ci-xtask, mold/sccache, ci-toolchain)
while taking main's mise-action v4.2.1 pins and native-macos
smoke job. Preserve default: velnor lanes.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Resolve post-#816 conflicts: take main product/tests, keep #810 CI reuse
paths with Velnor lane default.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Use the pinned user-space CLI instead of expanding the runner capability manifest for a third-party Docker action.

Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Keep CI reuse work; align lane defaults with main (velnor).

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Duplicate build-jackin-capsule job broke actionlint. Re-apply Velnor
lane defaults only.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
The warm-CI rewrite removed the producer while jackin-role-action kept resolving latest-build artifacts. Restore the Velnor-default producer and emit sha256sum-compatible sidecars.

Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Unblocks Velnor lane jobs when the cross-ref prepared artifact is
not yet published for the contract key.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Velnor nextest was failing with missing toolchain manifest / cargo
component conflict from polluted rustup-v2 caches.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Parallel zigbuild/mold for multi-target release-archives hit
ProcessFdQuotaExceeded on multi-slot Velnor hosts. Build targets
serially with full job count each; raise soft nofile in the
validator CI step.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Restore performance audit script dropped during main reconciliation and pass the required artifact token.

Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Avoids concurrent capacity peak holds across many Velnor slots on
the shared host (capacity_backpressure / pre_execution rejects).

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Job env cannot use runner.*; actionlint fails closed on PR CI.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Shared Velnor slots leave partial toolchains; uninstall alone leaves
component conflicts. Hard-remove trees + update-hashes and retry once.

Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Use an explicit drop for ignored removal errors and collapse the nested update-hash guard required by the upgraded lint set.

Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
@donbeave
donbeave force-pushed the perf/subminute-ci branch 5 times, most recently from 3f0e8ac to e0a70cc Compare July 25, 2026 04:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant