Skip to content

Optimize CI workflows: add caching, reduce redundant steps#1

Open
devin-ai-integration[bot] wants to merge 1 commit into
masterfrom
devin/1775932373-optimize-ci-workflows
Open

Optimize CI workflows: add caching, reduce redundant steps#1
devin-ai-integration[bot] wants to merge 1 commit into
masterfrom
devin/1775932373-optimize-ci-workflows

Conversation

@devin-ai-integration

Copy link
Copy Markdown

Summary

Adds dependency caching and reduces step overhead across artifact.yaml and unusual.yaml CI workflows. No changes to build logic, test commands, or artifacts produced.

Caching additions:

  • Swatinem/rust-cache@v2 added to all jobs that install Rust directly (sdist, macos_aarch64, macos_universal2, windows_amd64, windows_aarch64, unusual). Skipped for container-based jobs (manylinux, musllinux) where caching wouldn't persist.
  • cache: 'pip' added to actions/setup-python in sdist, windows_amd64, windows_aarch64, and unusual jobs.

Redundancy removal:

  • sdist & unusual: Two separate pip install calls consolidated into one (test/integration deps installed upfront alongside maturin, removing a second pip resolution pass).
  • sdist: Four integration test steps → one combined step.
  • macos jobs: Three integration test steps each → one step (eliminates repeated source .venv/bin/activate).
  • unusual: Three integration test steps → one step.

Toolchain modernization (unusual.yaml):

  • actions/setup-python updated from v5 → v6.
  • Manual curl https://sh.rustup.rs replaced with dtolnay/rust-toolchain@master (removes need for manual PATH prepend in build step).

Step reordering:

  • actions/checkout moved earlier in sdist and unusual jobs so cache actions (rust-cache, setup-python pip cache) can read Cargo.lock and requirements*.txt for cache key computation.

Review & Testing Checklist for Human

  • Verify unusual.yaml build step works without explicit PATH="$HOME/.cargo/bin:$PATH"dtolnay/rust-toolchain should add cargo to GITHUB_PATH automatically, but this is a behavioral change. Trigger the unusual workflow and confirm the maturin build step finds cargo.
  • Verify Swatinem/rust-cache works on windows_aarch64 — Rust is installed via manual PowerShell script, and CARGO_HOME is set via GITHUB_ENV (available in subsequent steps). Confirm the cache step finds the cargo registry.
  • Verify combined pip install in sdist doesn't cause resolution conflicts — Test deps (pytest, numpy, etc.) are now installed alongside maturin before the build instead of after. Confirm no version conflicts arise.
  • Run both workflows end-to-end and compare total job durations against a recent baseline run to confirm caching provides the expected speedup on subsequent runs. First run will be a cache miss (expect similar or slightly slower times); second run should show improvement.

Notes

  • Container-based jobs (manylinux, manylinux_cross, musllinux_amd64, musllinux_aarch64) are intentionally unchanged since builds happen inside ephemeral containers where host-level caching doesn't apply.
  • Integration tests remain sequential within each combined step (not parallelized with &) since the http test starts a daemon on a fixed port that could conflict with other tests.
  • The cache-on-failure: true option on rust-cache ensures partial compilation results are cached even on build failures, improving retry speed.

Link to Devin session: https://app.devin.ai/sessions/d3a95b8122444b47a6baa069678ebb49
Requested by: @lalo458

artifact.yaml:
- sdist: replace curl rustup with dtolnay/rust-toolchain action
- sdist: add Swatinem/rust-cache for Cargo registry/target caching
- sdist: add pip caching via actions/setup-python cache option
- sdist: combine pip install steps (one resolution pass instead of two)
- sdist: combine integration test steps to reduce step overhead
- macos_aarch64: add Swatinem/rust-cache for Cargo caching
- macos_aarch64: combine integration test steps (removes repeated venv activation)
- macos_universal2_aarch64: add Swatinem/rust-cache for Cargo caching
- macos_universal2_aarch64: combine integration test steps
- windows_amd64: add Swatinem/rust-cache and pip caching
- windows_aarch64: add Swatinem/rust-cache and pip caching

unusual.yaml:
- Update actions/setup-python v5 to v6 with pip caching
- Replace curl rustup with dtolnay/rust-toolchain action
- Add Swatinem/rust-cache for Cargo registry/target caching
- Combine pip install steps (one resolution pass instead of two)
- Combine integration test steps into single step
- Move checkout before toolchain setup for proper cache key resolution

Co-Authored-By: laloesparza458 <laloesparza458@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant