From 5750d605848e3187eb4d426db324bcb07d444f1b Mon Sep 17 00:00:00 2001 From: Bas van Gijzel Date: Mon, 20 Jul 2026 18:49:28 +0000 Subject: [PATCH] ci: extract the standard-config toolchain prologue into build-toolchain MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The submodules → SFW → msys → macOS-disk → LLVM → solc prologue existed in five near-identical copies (test.yaml cargo-checks and build-and-test, cache-warmup warm-llvm and warm-solc, slang-tests). Their LLVM/solc build params define the standard-config cache keys, so any drift between copies is a silent cache miss — and step-level drift is just as real: slang-tests never picked up free-disk-space-macos, so its macOS legs died with ENOSPC whenever the runner image shipped tight on space (run 29762518487: the arm64 leg hit `ld: write() failed, errno=28` at 98% disk). The new build-toolchain composite action encodes the prologue exactly once; call sites only choose which tools to build (llvm/solc toggles). The composite also absorbs the libstdc++ workaround from #550 (fix option 2), applying it to every standard-config Windows consumer: the bundled static libstdc++.a is dropped from the restored LLVM tree, so test builds link libstdc++ shared. This defuses the landmine #550 documents for test.yaml — its Windows leg was green only while the cargo cache kept llvm-sys Fresh, one fingerprint change away from an unlinkable fresh compile. The drop runs after build-llvm's mid-action cache save, so the shared cached tree keeps the archive; release binaries keep static linking (release.yaml builds LLVM itself and does not use this action). The rust-cache steps move after the composite rather than before it: prepare-msys (now inside the composite) switches the rustup host, which is part of the Swatinem cache key, so only the post-msys position keeps hitting the caches already saved on main. The late restore costs nothing on artifact-cache hits — build-llvm/build-solc skip their internal solx-dev builds entirely. Other consequences of the consolidation: - slang-tests' and cache-warmup's macOS legs now run free-disk-space-macos — previously missing exactly where the coldest LLVM builds happen. - cache-warmup's warm jobs ran msys before SFW; they now get test.yaml's proven SFW-first order. - warm-solc's explicit boost-version '1.83.0' dropped (it is the build-solc default, so the solc cache key is unchanged). - the inline submodule-checkout scripts are replaced by the existing checkout-submodules action (same update, plus --init). The specialty configs (sanitizer, coverage, integration, release) stay on direct build-llvm/build-solc calls: each is a one-producer/one-consumer pair with distinct params, and folding them in would turn the composite into a switch bag. Extracted from the cross-os-tester branch (#553), which will rebase onto this and add its cross-os-tests.yaml as a consumer. One deliberate difference from the #553 version: the composite passes optional: 'true' to setup-sfw, preserving what every current call site does — setup-sfw defaults to mandatory, and making a socket.dev outage fail all five jobs is not a change this extraction should smuggle in. --- .github/actions/build-toolchain/action.yml | 67 ++++++++++++++++++++++ .github/workflows/cache-warmup.yaml | 45 ++------------- .github/workflows/slang-tests.yaml | 25 ++------ .github/workflows/test.yaml | 59 +++---------------- 4 files changed, 86 insertions(+), 110 deletions(-) create mode 100644 .github/actions/build-toolchain/action.yml diff --git a/.github/actions/build-toolchain/action.yml b/.github/actions/build-toolchain/action.yml new file mode 100644 index 000000000..67865d091 --- /dev/null +++ b/.github/actions/build-toolchain/action.yml @@ -0,0 +1,67 @@ +name: Build toolchain +description: >- + Shared CI prologue for the standard toolchain configuration: submodule + checkout, Socket Firewall, platform prep (MSYS2 on Windows, disk cleanup + on macOS), then the LLVM and solc builds. The build parameters below + define the standard-config cache keys shared by test.yaml, + slang-tests.yaml, and cache-warmup.yaml — changing them here re-keys all + of those workflows together. Workflows needing a different LLVM + configuration (sanitizer, coverage, integration, release) call + build-llvm/build-solc directly and pair with their own cache-warmup job. + +inputs: + llvm: + description: Whether to build LLVM + required: false + default: 'true' + solc: + description: Whether to build solc + required: false + default: 'true' + +runs: + using: composite + steps: + - name: Checkout submodules + uses: ./.github/actions/checkout-submodules + + - name: Setup SFW (optional) + uses: ./.github/actions/setup-sfw + with: + optional: 'true' + + - name: Prepare Windows env + if: runner.os == 'Windows' + uses: ./.github/actions/prepare-msys + + - name: Free disk space (macOS) + if: runner.os == 'macOS' + uses: ./.github/actions/free-disk-space-macos + + - name: Build LLVM + if: inputs.llvm == 'true' + uses: ./.github/actions/build-llvm + with: + build-type: RelWithDebInfo + enable-assertions: 'true' + enable-mlir: 'true' + enable-utils: 'true' + + # Fresh llvm-sys objects under MSYS2 GCC 16 cannot link against the + # static libstdc++.a that solx-dev bundles into the LLVM tree; dropping + # it resolves -lstdc++ to the shared libstdc++.dll.a instead (#550, fix + # option 2). Test builds link libstdc++ shared; release binaries keep + # static linking (release.yaml builds LLVM itself and does not use this + # action). Runs after build-llvm's mid-action cache save, so the shared + # cached tree keeps the archive. + - name: Drop bundled static libstdc++ (Windows) + if: runner.os == 'Windows' && inputs.llvm == 'true' + shell: bash + run: rm -v target-llvm/target-final/lib/libstdc++.a + + - name: Build solc + if: inputs.solc == 'true' + uses: ./.github/actions/build-solc + with: + cmake-build-type: RelWithDebInfo + working-dir: 'solx-solidity' diff --git a/.github/workflows/cache-warmup.yaml b/.github/workflows/cache-warmup.yaml index 64a6a24bb..75e8b7f31 100644 --- a/.github/workflows/cache-warmup.yaml +++ b/.github/workflows/cache-warmup.yaml @@ -68,27 +68,10 @@ jobs: if: runner.os == 'Linux' && matrix.image != '' uses: ./.github/actions/free-disk-space-linux - - name: Checkout submodules - run: | - git config --global --add safe.directory '*' - git submodule update --force --depth=1 --recursive --checkout - - - name: Prepare Windows env - if: runner.os == 'Windows' - uses: ./.github/actions/prepare-msys - - - name: Setup SFW (optional) - uses: ./.github/actions/setup-sfw + - name: Build toolchain + uses: ./.github/actions/build-toolchain with: - optional: 'true' - - - name: Build LLVM - uses: ./.github/actions/build-llvm - with: - build-type: RelWithDebInfo - enable-assertions: 'true' - enable-mlir: 'true' - enable-utils: 'true' + solc: 'false' # When the artifact cache is warm, build-llvm skips ccache entirely, # so the ccache entry's LRU timer isn't refreshed. Touch it explicitly @@ -144,26 +127,10 @@ jobs: if: runner.os == 'Linux' && matrix.image != '' uses: ./.github/actions/free-disk-space-linux - - name: Checkout submodules - run: | - git config --global --add safe.directory '*' - git submodule update --force --depth=1 --recursive --checkout - - - name: Prepare Windows env - if: runner.os == 'Windows' - uses: ./.github/actions/prepare-msys - - - name: Setup SFW (optional) - uses: ./.github/actions/setup-sfw + - name: Build toolchain + uses: ./.github/actions/build-toolchain with: - optional: 'true' - - - name: Build solc - uses: ./.github/actions/build-solc - with: - cmake-build-type: RelWithDebInfo - boost-version: '1.83.0' - working-dir: 'solx-solidity' + llvm: 'false' - name: Touch solc ccache if: always() diff --git a/.github/workflows/slang-tests.yaml b/.github/workflows/slang-tests.yaml index 10ea8e198..fc511c1b9 100644 --- a/.github/workflows/slang-tests.yaml +++ b/.github/workflows/slang-tests.yaml @@ -74,27 +74,12 @@ jobs: with: submodules: true - # This step is required to checkout submodules - # that are disabled in .gitmodules config - - name: Checkout submodules - uses: ./.github/actions/checkout-submodules - - - name: Setup SFW (optional) - uses: ./.github/actions/setup-sfw - with: - optional: 'true' - - - name: Prepare Windows env - if: runner.os == 'Windows' - uses: ./.github/actions/prepare-msys - - - name: Build LLVM - uses: ./.github/actions/build-llvm + # solc is built separately below: the lit tests need a Release build + # against a synced solx-solidity, not the standard config. + - name: Build toolchain + uses: ./.github/actions/build-toolchain with: - build-type: RelWithDebInfo - enable-assertions: 'true' - enable-mlir: 'true' - enable-utils: 'true' + solc: 'false' # TODO: remove this step and the Build solc step below once the lit # tests stop running against solc. diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml index 969cdb96f..035edacd0 100644 --- a/.github/workflows/test.yaml +++ b/.github/workflows/test.yaml @@ -112,17 +112,10 @@ jobs: - name: Free disk space (Linux) uses: ./.github/actions/free-disk-space-linux - # This step is required to checkout submodules - # that are disabled in .gitmodules config - - name: Checkout submodules - run: | - git config --global --add safe.directory '*' - git submodule update --force --depth=1 --recursive --checkout - - - name: Setup SFW (optional) - uses: ./.github/actions/setup-sfw + - name: Build toolchain + uses: ./.github/actions/build-toolchain with: - optional: 'true' + solc: 'false' - name: Cache cargo artifacts uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1 @@ -131,14 +124,6 @@ jobs: cache-on-failure: true save-if: ${{ github.ref == 'refs/heads/main' }} - - name: Build LLVM - uses: ./.github/actions/build-llvm - with: - build-type: RelWithDebInfo - enable-assertions: 'true' - enable-mlir: 'true' - enable-utils: 'true' - - name: Cargo checks uses: ./.github/actions/cargo-check @@ -234,26 +219,12 @@ jobs: if: runner.os == 'Linux' && matrix.image != '' uses: ./.github/actions/free-disk-space-linux - # This step is required to checkout submodules - # that are disabled in .gitmodules config - - name: Checkout submodules - run: | - git config --global --add safe.directory '*' - git submodule update --force --depth=1 --recursive --checkout - - - name: Setup SFW (optional) - uses: ./.github/actions/setup-sfw - with: - optional: 'true' - - - name: Prepare Windows env - if: runner.os == 'Windows' - uses: ./.github/actions/prepare-msys - - - name: Free disk space (macOS) - if: runner.os == 'macOS' - uses: ./.github/actions/free-disk-space-macos + - name: Build toolchain + uses: ./.github/actions/build-toolchain + # Positioned after build-toolchain: prepare-msys switches the rustup + # host, which is part of the cache key, so only the post-msys position + # matches the keys already saved on main. - name: Cache cargo artifacts uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1 with: @@ -261,20 +232,6 @@ jobs: cache-on-failure: true save-if: ${{ github.ref == 'refs/heads/main' }} - - name: Build LLVM - uses: ./.github/actions/build-llvm - with: - build-type: RelWithDebInfo - enable-assertions: 'true' - enable-mlir: 'true' - enable-utils: 'true' - - - name: Building solc - uses: ./.github/actions/build-solc - with: - cmake-build-type: RelWithDebInfo - working-dir: 'solx-solidity' - - name: Free disk space (remove LLVM source and build artifacts) shell: bash run: |