bootstrap: add bootstrap step to run stdarch-gen checks in CI by xonx4l · Pull Request #156674 · rust-lang/rust

xonx4l · 2026-05-17T18:22:18Z

This PR ports stdarch-gen checks into the bootstrap test runner as a step, so that we can run the stdarch-gen checks via x.py test.

Changes -:
-> Adds a new StdarchGenCheck bootstrap step.
-> Mirrors https://github.com/rust-lang/stdarch/blob/f71ac102d30d5d1957277cbffc6ba631b67bffd1/.github/workflows/main.yml#L309-L333 (check-stdarch-gen job)
-> stdarch-gen-arm uses a nightly only feature (#![feature(pattern)]) but bootstrap's cargo runs on the beta toolchain thus we set RUSTC_BOOTSTRAP=1 on the child process so it builds.
-> The generators call out to rustfmt to format their output so rustfmt needs to be on PATH. So added bootstrap's rustfmt to PATH before each run. If rustfmt isn't available, the step fails early and tells the user to set rustfmt = true in bootstrap.toml .

r? @Kobzol

rustbot · 2026-05-17T18:22:24Z

Kobzol is not on the review rotation at the moment.
They may take a while to respond.

rustbot · 2026-05-26T13:35:15Z

stdarch is developed in its own repository. If possible, consider making this change to rust-lang/stdarch instead.

cc @Amanieu, @folkertdev, @sayantn

Kobzol

Thank you! I left some comments, the most important one is the question about why things are copied over.

View changes since this review

Kobzol · 2026-05-27T07:30:06Z

+        });
+
+        let mut path_dirs: Vec<PathBuf> = Vec::new();
+        if let Some(cargo_dir) = builder.initial_cargo.parent() {


Why does cargo need to be in PATH?

Because cargo_dir is providing rustc . As cargo run build looks up rustc on PATH .

In that case we should set RUSTC to point to the rustc that we want to invoke directly, I think.

Okay makes sense can do.

Kobzol · 2026-05-27T07:33:09Z

+        let work_dir = builder.out.join("stdarch-gen-check");
+        let cargo_target_dir = work_dir.join("target");
+
+        // Writable copy of `crates/core_arch` that the generators write into.


Suggested change

// Writable copy of `crates/core_arch` that the generators write into.

// Writable copy of `crates/core_arch` that the generators write into when blessing.

Kobzol · 2026-05-27T07:35:00Z

+        if out_core_arch.exists() {
+            t!(fs::remove_dir_all(&out_core_arch));
+        }
+        cp_writable_r(&src_core_arch, &out_core_arch);


Why do we need to copy things over? My mental model of how this should work is:

When running the test, we generate the output code into some temporary directory on disk

We then check if the generated files are the same as the files committed in the repo.

If not, and we are not blessing, the test fails.

If not, and we are blessing, the generated files override the committed files (we also need to ensure that any leftover files are deleted, so the committed dir should probably be emptied first).

Your mental model is right it's just the copy is mainly there for the arm generator. It writes its output files directly into existing folders under core_arch/src but doesn't create those folders itself so they have to already exist. Copying core_arch over gives it those folders. Having the full copy also let run a single git diff over the whole folder instead of listing every generated file by hand. It's the same thing stdarch's own check-stdarch-gen CI does.

I'd rather edit the generator to create those directories and make it so that it doesn't write to the source dirs, than do hacks with cp_writable_r.

In fact, to ensure that we run the same logic, it might be better if the check + bless logic actually lived within the generators themselves, and maybe we could also unify them under a single crate, it looks like they are separate right now? I wonder if there's an opportunity to do some refactoring there to make the whole setup less hacky.

I approached with cp_writable_r to keep the changes minimal and strictly around bootstrap step only but if you want then I think we can edit the generator and can get rid of cp_writable_r . Yes they are separate right now and yeah by unifying it will make the whole setup less hacky for sure .

Yeah I'd rather also do targeted refactorings to stdarch to make the overall testing infra better, rather than to just try hacking bootstrap around whatever stdarch is currently doing.

Such changes are welcome. You can experiment here, and once you're happy we can see how to sync the changes back into stdarch

Kobzol · 2026-05-27T07:36:16Z

+
+/// Recursively copy src to dst clearing the read-only bit so the destination can be
+/// regenerated even when src lives on a read-only filesystem.
+fn cp_writable_r(src: &Path, dst: &Path) {


This shouldn't be needed. We shouldn't ever bless on CI, nor copy any files on CI, only locally.

The read-only handling isn't for bless it only runs locally never on CI. It's there because we copy core_arch into the work folder at the start of every run and on CI the original files are read-only so the copies come out read-only too.

Kobzol · 2026-05-27T07:39:02Z

+        run_gen("-p", "stdarch-gen-hexagon", &[hexagon_out.as_os_str()], None);
+        run_gen("-p", "stdarch-gen-hexagon-scalar", &[hexagon_out.as_os_str()], None);
+
+        let mut git = helpers::git(Some(&stdarch_root));


Hmm, I was kinda thinking that we would check the files using Rust code, but I checked bootstrap and it doesn't seem like we already do any diffing at the moment, so git diff seems fine.

rust-bors · 2026-06-01T15:40:23Z

☔ The latest upstream changes (presumably #153957) made this pull request unmergeable. Please resolve the merge conflicts.

port stdarch-gen checks into bootstrap

775ac40

rustbot assigned Kobzol May 17, 2026

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 17, 2026

This comment has been minimized.

Sign in to view

use helpers::git

adf95df

This comment has been minimized.

Sign in to view

fix: set CARGO_TARGET_DIR for read-only CI

009df0f

This comment has been minimized.

Sign in to view

xonx4l added 2 commits May 26, 2026 13:17

fix: redirect generator output to a writable dir

3174875

add optional output-dir argument

a594176

Kobzol reviewed May 27, 2026

View reviewed changes

	// Writable copy of `crates/core_arch` that the generators write into.
	// Writable copy of `crates/core_arch` that the generators write into when blessing.

Uh oh!

Conversation

xonx4l commented May 17, 2026

Uh oh!

rustbot commented May 17, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rustbot commented May 26, 2026

Uh oh!

Kobzol left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rust-bors Bot commented Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kobzol left a comment •

edited by rustbot

Loading