ci: add riscv64 manylinux wheels with native RISE runners#234
ci: add riscv64 manylinux wheels with native RISE runners#234gounthar wants to merge 13 commits intopython-cffi:mainfrom
Conversation
fix: add checkout to release job and use DISPATCH_TOKEN
fix: set GH_REPO for gh release commands
|
Correction: the linked CI run was from the fork's own scheduled build, not RISE runners. However, the build succeeds on native riscv64. build time: ~2m39s (https://github.com/gounthar/cffi/actions/runs/23103252300). Also verified on BananaPi F3 hardware. The PR's native runner approach (ubuntu-24.04-riscv) is ready for when RISE runners are installed on the cffi org. |
|
Successful riscv64 build on native RISE runners (ubuntu-24.04-riscv): https://github.com/gounthar/cffi/actions/runs/23706627020 Build, wheel verification, and release all passed. Build time under 3 minutes on native hardware. |
20b754e to
a68dec8
Compare
2399473 to
5a09612
Compare
.github/workflows/ci.yaml
Outdated
| - { spec: cp314t-manylinux_s390x, arch: s390x, omit: ${{ env.skip_slow_jobs }} } | ||
|
|
||
| # riscv64 manylinux | ||
| - { spec: cp39-manylinux_riscv64, arch: riscv64, test_args: '{package}/src/c', omit: ${{ env.skip_slow_jobs }} } |
There was a problem hiding this comment.
I guess we can remove ${{ env.skip_slow_jobs }} now it runs on native runners
There was a problem hiding this comment.
Good catch. Removed skip_slow_jobs from all riscv64 entries. Kept skip_ci_redundant_jobs for the intermediate Python versions, matching the pattern of the other architectures.
There was a problem hiding this comment.
Also other non emulated targets don't specify test_args: '{package}/src/c' which seems to run only a subset of the test suite. Maybe we should remove it here also?
I started an action on my fork, and with the full test suite, riscv64 build takes 34 minutes (see https://github.com/justeph/cffi/actions/runs/23733528422/job/69141241075) compared to x86 which succeeded in 4 minutes..
For what it worth, this is still better than the 1 hour with emulation :) https://github.com/justeph/cffi/actions/runs/21716002341/job/62632136017
So makes me wondering if we should remove the skip_slow_jobs or running the full test suite?
But then, do we need native runners? Emulated riscv64 job with test_args: '{package}/src/c' runs in 5 minutes https://github.com/justeph/cffi/actions/runs/21716002341/job/62632136043 when it runs in 6 minutes on native runner https://github.com/justeph/cffi/actions/runs/23731763287/job/69126983344...
Maybe that will be a decision for @DimitriPapadopoulos or @mattip (or someone else 😅 ) ?
There was a problem hiding this comment.
Makes sense. Removed test_args so riscv64 runs the full test suite like the other native targets. 34 minutes is reasonable for a native build, and much better than the hour under emulation.
4f54944 to
f65814a
Compare
Signed-off-by: Bruno Verachten <gounthar@gmail.com>
433c715 to
84162b7
Compare
Summary
Add riscv64 manylinux wheel builds using native RISE RISC-V runners instead of QEMU emulation.
Based on the excellent work by @justeph in #227. This PR builds on that approach with native runner support for significantly faster builds.
Changes
ubuntu-24.04-riscvnative runner (no QEMU needed)docker/setup-qemu-actionfor riscv64CIBW_MANYLINUX_RISCV64_IMAGEdefaulting tomanylinux_2_39(first manylinux with riscv64)Why native runners over QEMU
QEMU emulation (as in #227) works but is slow. Native RISE runners provide real riscv64 hardware at no cost to the project, with build times comparable to other architectures.
Evidence
Credits
Supersedes #227