Enable build on aarch64#509
Conversation
JunHe77
requested review from
chenhengqi,
fslongjin,
lisongqian,
ls-ggg,
tinklone and
up2wing
as code owners
Signed-off-by: Jun He <jun.he@arm.com> Change-Id: I8148fa1c3eddff106f5e4223404ed1b3f5c37a52
CubeMaster previously pulled in both the unversioned gomonkey module and gomonkey/v2 at the same time. The v1 module is unmaintained and only kept alive in go.sum because a single integration test helper still imported it, which inflated the dependency graph and made the two versions easy to mix up in future contributions. This change consolidates the codebase on gomonkey/v2, the version already used by the rest of CubeMaster, so that only one monkey- patching library is shipped and audited. No functional or test-behavior change is intended. Assisted-by: Anthropic:claude-opus-4-7 Signed-off-by: Like Xu <likexu@tencent.com> Change-Id: If8a836df9f6e040977d433f8ccee44f9d27416cf
Signed-off-by: Jun He <jun.he@arm.com> Change-Id: I499da5f135ae4dba76ec542e88e54cd440fb7ef8
The guest agent was hard-wired to x86_64 in two places: the Makefile always built against x86_64-unknown-linux-musl, and the RPC startup path used a PIO write to signal the shim that the vsock server is ready, which is not available on aarch64. As a result the agent could neither be built for nor run inside an ARM64 sandbox. This commit lifts both assumptions. The musl build now follows the host architecture by default while still allowing an explicit TRIPLE override, and the readiness notification on aarch64 is delivered via the SysCtrl MMIO region exposed by the hypervisor, matching what the shim already listens for. A small print-target-path helper is added so the surrounding build scripts can locate the produced binary without duplicating the triple logic. The x86_64 path is unchanged. Assisted-by: Anthropic:claude-opus-4-7 Signed-off-by: Like Xu <likexu@tencent.com> Change-Id: Iac42d7f405fe0faa8c2c3fe42200f3d0ee67a01f
Refactor the way to generate architecture dependent localgw/mvmtap/nodenic bpf code. Add header file, vmlinux.h, for arm64. Update build deps and steps accrodingly. Signed-off-by: Jun He <jun.he@arm.com> Change-Id: Ia712c5515f4e67208edc05dfe935406277b72ffe
After rebasing the hypervisor on a newer cloud-hypervisor revision, the aarch64 path accumulated a number of regressions that prevented MicroVMs from being built and booted on ARM64 hosts. This commit consolidates the aarch64-only fixes required to bring the platform back to a working state. It realigns the KVM, snapshot and migration call sites with the upstream API changes, exposes the SysCtrl device over MMIO so the guest can still signal shutdown and reboot to the shim, and tightens the cross-arch cfg gates so the crate compiles cleanly without warnings. The x86_64 behavior is intentionally left unchanged. Assisted-by: Anthropic:claude-opus-4-7 Signed-off-by: Like Xu <likexu@tencent.com> Change-Id: Ied3d75b301a06869bddfd20d4cd022485ebfdd41
CubeShim was previously assuming an x86_64 host when launching the guest VM and when installing seccomp rules for the hypervisor and snapshot workers. As a result the shim could neither boot a guest nor pass syscall filtering on aarch64 hosts. This change generalizes the host-architecture assumptions so that the shim works on both x86_64 and aarch64. The default guest kernel cmdline now picks an appropriate console and drops x86-only mitiga- tion knobs on ARM64, and the seccomp allow-lists account for the syscall numbering differences between the two architectures. No functional change is intended on x86_64. Assisted-by: Anthropic:claude-opus-4-7 Signed-off-by: Like Xu <likexu@tencent.com> Change-Id: Ie65a897403ad1b8c1d2710855c9bc51c3fb6e1bf
Change hardcoded architecture name to dynamically detect one. Add aarch64 dependent machine name and bios params. Signed-off-by: Jun He <jun.he@arm.com> Change-Id: I8053c73b3466382a154b36926551f42e8564f861
Signed-off-by: Jun He <jun.he@arm.com> Change-Id: Idb1d099a00e8b1afa9bbf1e6b7be59bd73685dd4
Signed-off-by: Jun He <jun.he@arm.com> Change-Id: If395689ca16cb5a213098c1df82b51babceddc97
Signed-off-by: Jun He <jun.he@arm.com> Change-Id: I2402cdb8c046a0fb1700cf88fae67255b924576a
As based image has downgraded to Ubuntu 20.04, the linker here is updated to align with this change Signed-off-by: Jun He <jun.he@arm.com> Change-Id: I9c3b0b956233be5d9ccffcde569729568ace3a73
![cubesandboxbot[bot]](https://avatars.githubusercontent.com/u/20101770?s=60&v=4){kind=small}
| cube_hypervisor::set_runtime_seccomp_rules(vec![ | ||
| #[cfg(target_arch = "x86_64")] | ||
| (libc::SYS_mkdir, vec![]), | ||
| #[cfg(target_arch = "aarch64")] | ||
| (libc::SYS_mkdirat, vec![]), | ||
| (libc::SYS_getsockopt, vec![]), | ||
| (libc::SYS_setsockopt, vec![]), | ||
| ]); |
There was a problem hiding this comment.
Seccomp rule inconsistency: The snapshot launch_vmm() is missing (libc::SYS_faccessat2, vec![]) from its seccomp allowlist, while the equivalent call in cube_hypervisor.rs (line 86) includes it on both x86_64 and aarch64.
On aarch64, if Rust's standard library or any internal dependency calls faccessat2 (used for file access checks), the seccomp filter will kill the snapshot worker process. Please add SYS_faccessat2 here to match cube_hypervisor.rs.
|
@JunHe77 Thanks for your contributions. Could you please provide more test environments and the results of the corresponding test cases? May I ask which aarch64 hardware and host kernel versions you have actually tested Could you help check whether this PR is actually helpful for aarch64 support? For instance, in this PR we are still missing (at least):
If the default page size on the arm machine is not 4KB, will Cloud Hypervisor (CH) still work properly? Besides, being able to run CubeSandbox on Apple M-series chips would make cube and ARM much more popular. |
There was a problem hiding this comment.
We clearly don't need these reproducible binaries.
There was a problem hiding this comment.
For commit "docker/Dockerfile.builder: support multi-arch builder image"
Overall the arch-mapping is mostly correct (Go via dpkg --print-architecture, protoc aarch_64, rustup musl target, libseccomp --host, openssl multiarch dir all map correctly). But there is one confirmed regression that defeats part of the patch's own goal, plus a few smaller items.
High — OpenSSL lib-dir override is now dead on both arches
docker/Dockerfile.builder:28-29
UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR=/usr/lib/${HOST_ARCH}-linux-gnu \
UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR=/usr/lib/${HOST_ARCH}-linux-gnu \openssl-sys only reads env vars prefixed with the full uppercased target triple (e.g. X86_64_UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR, AARCH64_UNKNOWN_LINUX_MUSL_OPENSSL_LIB_DIR) or the unprefixed OPENSSL_LIB_DIR. The renamed UNKNOWN_LINUX_GNU_OPENSSL_LIB_DIR matches no target, so it's silently never read — on both x86_64 and aarch64. The original block already listed all four arch-prefixed vars (so it was already arch-agnostic), and hypervisor/resources/Dockerfile:87-90 still uses the correct names. Fix: revert to the four arch-prefixed vars (no HOST_ARCH interpolation needed — they already covered both arches).
Medium — World-writable Rust toolchain
docker/Dockerfile.builder:133
chmod -R a+rwX /usr/local/rustupa+rwX makes every toolchain binary world-writable (tampering surface / least-privilege violation). a+rX is sufficient for compilation; the write bit is only needed for runtime rustup install/update. Impact is bounded (ephemeral --rm container, non-root --user), but recommend dropping the w bit unless runtime rustup mutation is intended.
Low — HOST_ARCH default can silently produce a broken mixed-arch image
HOST_ARCH defaults to x86_64, but the Go download independently uses $(dpkg --print-architecture). A bare docker build (no --build-arg) on an arm host — which is how CI invokes it (build-builder-image.yml doesn't pass HOST_ARCH) — would fetch an arm64 Go but x86_64 rustup/libseccomp/openssl. Only the make builder-image path auto-detects the arch. Consider sourcing a single arch via buildx TARGETARCH, or at least documenting that direct docker build on non-x86_64 requires --build-arg HOST_ARCH=.
Low — The aarch64 path has zero CI coverage
All workflows run on x86_64 only (no QEMU/linux/arm64/arm runners). Every new HOST_ARCH=aarch64 branch is unexercised, so the patch's primary purpose is unvalidated, and the OpenSSL rename above wouldn't be caught by any existing check. Worth an arm64 build job (QEMU + buildx) building the image with HOST_ARCH=aarch64 and at least one musl Rust component.
Low — Stale docs
CONTRIBUTING.md:36hardcodes thex86_64-unknown-linux-musltarget prerequisite, now arch-dependent.docker/README.mdis the natural place to document the newHOST_ARCHarg / multi-arch capability.
Dismissed (raised by reviewers, but not real issues)
- Removing the blank line between the two
RUNs does not merge layers (continuation depends on trailing\). - The
chmod -Rdoes not create a duplicate/oversized layer — it's in the sameRUNas the rustup install. - Unverified downloads (go/protoc/libseccomp/rustup) and unquoted
HOST_ARCHinterpolation are pre-existing / trusted-input, not introduced or worsened here.
Recommendation: address the High OpenSSL finding before merge; the Medium chmod is a quick safe improvement; the rest are optional follow-ups.
There was a problem hiding this comment.
Please add .gitignore entries for these auto-generated files.
chenhengqi
left a comment
chenhengqi
left a comment
There was a problem hiding this comment.
Networking-related changes looks good to me. Thanks.
3 participants
This PR enables build on aarch64 paltform. Tested with both
make alland ./deploy/one-click/build-release-bundle-builder.sh.