perf(ci): reuse build results across workflows - #810
Open
donbeave wants to merge 215 commits into
Open
Conversation
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Version the debugless Cargo profile as target contract v7 so incompatible build generations cannot accumulate in one archive. Share the warmup producer's dependency outputs with the complete per-crate test job. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Use attempt-scoped GitHub job APIs with validated retries so transient HTML 503 responses cannot break required aggregators. Fetch job logs concurrently to keep the complete audit below the one-minute budget. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
GitHub returns 404 for skipped jobs because no log archive exists. Keep missing logs fatal for executed jobs while excluding semantically logless skipped jobs from the concurrent audit. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com> Co-authored-by: Codex <codex@openai.com>
Member
Author
|
Final exact-head GitHub evidence for CI:
jackin-dev cross-release:
All four cross targets restore distinct exact-SHA target caches. No dependency compilation occurs on no-change; each target recompiles only the workspace Velnor/both proof awaits signed-apt fleet access. GHA-backed compiler cache remains the explicit human decision for eliminating workspace recompilation on ephemeral GitHub hosts. |
Resolve workflow conflicts favoring PR CI reuse structure (rustup-v2, download-ci-xtask, mold/sccache, ci-toolchain) while taking main's mise-action v4.2.1 pins and native-macos smoke job. Preserve default: velnor lanes. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Use the pinned user-space CLI instead of expanding the runner capability manifest for a third-party Docker action. Co-authored-by: Codex <codex@openai.com> Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Keep CI reuse work; align lane defaults with main (velnor). Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Duplicate build-jackin-capsule job broke actionlint. Re-apply Velnor lane defaults only. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
The warm-CI rewrite removed the producer while jackin-role-action kept resolving latest-build artifacts. Restore the Velnor-default producer and emit sha256sum-compatible sidecars. Co-authored-by: Codex <codex@openai.com> Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Unblocks Velnor lane jobs when the cross-ref prepared artifact is not yet published for the contract key. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Velnor nextest was failing with missing toolchain manifest / cargo component conflict from polluted rustup-v2 caches. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Parallel zigbuild/mold for multi-target release-archives hit ProcessFdQuotaExceeded on multi-slot Velnor hosts. Build targets serially with full job count each; raise soft nofile in the validator CI step. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Restore performance audit script dropped during main reconciliation and pass the required artifact token. Co-authored-by: Codex <codex@openai.com> Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Avoids concurrent capacity peak holds across many Velnor slots on the shared host (capacity_backpressure / pre_execution rejects). Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Job env cannot use runner.*; actionlint fails closed on PR CI. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Shared Velnor slots leave partial toolchains; uninstall alone leaves component conflicts. Hard-remove trees + update-hashes and retry once. Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Use an explicit drop for ignored removal errors and collapse the nested update-hash guard required by the upgraded lint set. Co-authored-by: Codex <codex@openai.com> Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
Signed-off-by: Alexey Zhokhov <alexey@zhokhov.com>
donbeave
force-pushed
the
perf/subminute-ci
branch
5 times, most recently
from
July 25, 2026 04:48
3f0e8ac to
e0a70cc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR makes jackin❯ CI reuse previously completed work instead of downloading and recompiling unchanged dependencies, then standardizes the complete workflow estate on Velnor-default execution with an explicit GitHub Ubuntu 26.04 parity lane. Maintainers retain one independently readable complete test job per affected crate while exact-result proofs skip unchanged producers, prepared artifacts tolerate measured producer latency, and writer gates keep publication single-owner.
What ships
The CI workflow again produces complete
jackin-rolevalidator archives for both Linux targets on the Velnor-default lane; PRs exercise the read-only build, main publishes one writer-owned artifact per target, and checksum sidecars satisfy the consumer’ssha256sum --checkcontract.Each crate and feature mode owns one stable v7 Cargo target seed with a consistent debugless, non-incremental profile; a warm dependency seed initializes the first complete crate job, which publishes the canonical result.
The prepared
jackin-xtaskbinary owns target discovery, validation, restoration, packing, publication inputs, workflow performance auditing, semantic Docs contracts, and prepared toolchain repair.Prepared-artifact consumers wait across the complete observed producer-latency envelope and report their true remaining deadline; action-level tests cover immediate, delayed, invalid, absent, and timed-out artifacts.
Documentation examples are parser-audited: runnable Rust fences are rejected unless their executable proof lives in nextest, keeping all Rust test execution on the nextest contract. Docs tooling is resolved through the repository's mise configuration.
CI, Docs, Construct, Preview, Release, and jackin-dev preserve exact-result reuse while automatic and omitted dispatch selection now resolve to the Velnor lane.
Every runner-selectable dispatch orders
velnor,github, andboth; GitHub records useubuntu-26.04, and all workflow utility jobs use the same pinned Ubuntu image.The standalone native-macOS hygiene job is removed. Existing release and preview Apple artifacts remain Linux cross-builds through the repository's
cargo-zigbuildarchive machinery.The duplicate Rust target-cache layer is removed across compile workflows. The shared Cargo registry cache and prepared sccache v0.16.0 tool set remain the runner-neutral cache contract with a 20 GiB cap.
Explicit sccache-statistics workflow steps and their composite are removed; cache reporting belongs to setup/post behavior rather than a build step.
Every workflow now has concurrency control, every concrete job has a timeout, and every retained full-history checkout documents its consumer.
Workflow guidance records Velnor-default selection, Ubuntu 26.04, the three-lane contract, and the single compiler-cache stack.
Behavior changes
lanes=githubremains the explicit GitHub-hosted verification path;lanes=bothretains lane parity with one writer.What this addresses
Hard rule: complete crate jobs with Rust-owned orchestration
Every affected crate remains one independently readable job that runs its complete applicable suite. CI optimizes reuse without splitting a crate into shards, batches, or numbered parts, and workflow run blocks call prepared Rust tooling or a direct tool command rather than implementing CI decisions, parsing, cache contracts, or artifact transport as Bash programs.
Verification evidence
Exact head
f928092efde4305501c9276449df16cbeff94b63:Historical timing remains useful for the unchanged reuse graph: the no-change CI proof completed in 19 seconds end to end; the prior dual-lane warmup took 12 seconds on GitHub and 36 seconds on Velnor; cached Preview crate jobs completed in 19–20 seconds on GitHub and 28–40 seconds on Velnor. Fresh Velnor and Preview parity evidence remains pending the signed-apt fleet migration and the separately approval-gated RFC3161 attestation capability; no workflow was weakened to manufacture that proof.
Deviations
mise.tomland transports that prepared binary without per-job installs; replacing it with unconditional setup actions would regress the proven no-change path by installing tools in structurally skipped jobs. The standard is therefore applied through the existing Rust-owned preparation boundary rather than duplicating setup actions.macosidentifiers because they describe supported artifact targets, not runner operating systems. No workflow job runs on macOS..github/actionlint.yamldeclaresubuntu-26.04andubuntu-26.04-armbecause actionlint v1.7.12 predates those GitHub-hosted labels; this suppresses only its stale label catalog, not workflow syntax or expression checks.Not included
Verify locally
Checkout
Paste this first to bypass the
tirithpaste scanner for the rest of the session:export TIRITH=0Then paste the checkout block:
Static checks
actionlint .github/workflows/*.yml cargo fmt --check cargo clippy -p jackin-xtask --all-targets -- -D warningsRust tests
Docs checks
User smoke
Confirm the Rust-owned CI contracts pass without numbered shards or batches.
jackin-capsule smoke
jackin load the-architect . --debugInside the container, verify:
jackin❯ [the-architect].Ctrl+\\opens the command palette.Documentation
( cd docs bun install --frozen-lockfile bun run dev )Vite serves at
http://localhost:3000/. Pages to walk:http://localhost:3000/reference/research/ci/performance/ci-speed-roadmap/
UPDATED. Confirm the compiler-cache and exact-result reuse contract matches the workflow implementation.
http://localhost:3000/roadmap/infrastructure/shared-ci-compiler-cache/
UPDATED. Confirm roadmap status and local-only cache boundaries render correctly.
Migration notes
None.