refactor: shared build runner across platforms#2907
Draft
henryiii wants to merge 7 commits into
Draft
Conversation
Adds cibuildwheel/platforms/runner.py: a Builder ABC (generic over the wheel path type, since Linux wheels live in the container as PurePosixPath) whose methods are the platform-specific build steps, and run_builds(), a single flat loop that owns everything identical across platforms: logging, compatible-wheel reuse, built/repaired wheel validation, the test gate, output bookkeeping, and cleanup. HostBuilder provides the step bodies shared by the five non-Linux platforms. run_before_all() and fatal_on_called_process_error() replace the per-platform copies of the before_all hook and the CalledProcessError-to-FatalError wrapper. No platform uses this yet; subsequent commits port them one at a time. Assisted-by: ClaudeCode:claude-fable-5
The per-wheel loop in build() is replaced by a PyodideBuilder driven by
runner.run_builds(). Behavior changes:
- a compatible-wheel (abi3) reuse no longer crashes with a NameError
(the old loop assigned built_wheel but read repaired_wheel; it was
unreachable in practice since pyodide wheels are never reused)
- a repair command producing zero wheels now raises
RepairStepProducedNoWheelError instead of a raw StopIteration, and
producing several is now an error, like on Linux
- before_test no longer receives the undocumented {wheel} placeholder;
test_command now does, matching the other platforms
- the per-identifier temp dir is now removed after each build
Assisted-by: ClaudeCode:claude-fable-5
The per-wheel loop becomes a MacOSBuilder; the per-architecture test loop (universal2 testing both halves, Rosetta emulation, arch-specific test-skip identifiers) stays inside test_wheel(), unchanged. Behavior changes: - a repair command producing several wheels is now an error, as on Linux - a missing uv now raises FatalError instead of AssertionError - the per-identifier temp dir is removed with ignore_errors, like Windows Assisted-by: ClaudeCode:claude-fable-5
The per-wheel loop becomes a WindowsBuilder. The ARM64-on-non-ARM64 test skip moves inside test_wheel(), so its warning now only appears when a test command is actually configured. A repair command producing several wheels is now an error, as on Linux. Assisted-by: ClaudeCode:claude-fable-5
The per-wheel loop becomes an IOSBuilder; the testbed-based test flow, the non-simulator/non-native skip steps, and the python-module test command validation stay inside test_wheel(). Behavior changes: - a test-suite failure now raises FatalError instead of calling sys.exit(1) directly (same exit code, consistent error reporting) - a repair command producing several wheels is now an error, as on Linux - the compatible-wheel reuse message now matches the other platforms - the per-identifier temp dir is removed with ignore_errors Assisted-by: ClaudeCode:claude-fable-5
The BuildState dataclass and its step functions fold into an AndroidBuilder; setup_target_python/setup_env and the other environment helpers are unchanged. Behavior changes: - test-environment (CIBW_TEST_ENVIRONMENT_ANDROID) is now applied to the test steps (before-test, wheel install, testbed invocation); it was documented but previously ignored on Android - a wheel name colliding with an earlier build now raises AlreadyBuiltWheelError, like every other platform - the redundant post-repair none-any.whl check is dropped (the built wheel is still checked, like every other platform) - the compatible-wheel reuse message now matches the other platforms, and the moved-elsewhere warning from move_file is now emitted - a missing uv now raises FatalError instead of AssertionError Assisted-by: ClaudeCode:claude-fable-5
The per-config loop in build_in_container() becomes a LinuxBuilder (generic over PurePosixPath — wheels stay container-side, and the existing copy-out and host-side audit behavior is preserved as overridden steps). build() keeps the container grouping, engine check, and OCIContainer lifecycle; the CalledProcessError wrapper keeps its per-container scope and troubleshoot() hook via fatal_on_called_process_error. build_in_container() keeps its signature, since tests mock it. Also fixes a latent bug: the host-side per-identifier temp dir was never created, so dependency-versions with packages: would have crashed writing constraints.txt on Linux. Assisted-by: ClaudeCode:claude-fable-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Alternative to #2903, this time using Claude Fable 5 instead of Claude Opus 4.8. This is not just a one to one; this one doesn't have nearly as much restraining it, and I didn't tell it where I wanted to go later (
--stage). But otherwise it's also an unedited output of a plan.🤖 AI text below 🤖
What
Extracts the per-wheel build loop that was duplicated across all six platform modules into a shared runner (
cibuildwheel/platforms/runner.py), driven by a per-platform builder class with clearly-named steps:Builder(ABC, Generic[PathT])— one method per step. Generic over the wheel-path type because Linux wheels live in the container asPurePosixPath; everywhere else they're hostPaths.run_builds()— a single flat loop that owns everything that was previously copy-pasted six times:log.build_start/build_end/step accounting, compatible-wheel (abi3) reuse, thenone-any.whlcheck, exactly-one-repaired-wheel validation,AlreadyBuiltWheelError, the test gate, output bookkeeping, and cleanup. The loop body reads top-to-bottom like the old per-platform loops — there's no step registry or hook indirection.HostBuilder— the step bodies shared by the five non-Linux platforms (before_build, default repair, audit, move-to-output, cleanup).run_before_all()/fatal_on_called_process_error()— replace the per-platform copies of the before_all hook and theCalledProcessError → FatalErrorwrapper (Linux keeps its per-container scope andtroubleshoot()hook).Each platform's
build()shrinks to ~15 lines; platform-specific structure stays where it was and stays visible: macOS keeps its per-arch test loop (universal2/Rosetta), iOS its testbed flow, Android its dual build/android environments (the previousBuildStatepattern was the prototype for this design), pyodide itspyodide venvtesting, and Linux keepsget_build_steps()grouping and theOCIContainerlifecycle, invoking the runner once per container.build_in_container()keeps its signature (it's mocked inoption_prepare_test.py), and thePlatformModuleinterface is unchanged.Behavior changes
The duplication had let the platforms drift; this unifies the unjustified differences (one commit per platform, each itemized in its commit message):
Bugs fixed
NameError(assignedbuilt_wheel, readrepaired_wheel) — unreachable today only because pyodide wheels are never reusedStopIterationinstead ofRepairStepProducedNoWheelErrordependency-versionswithpackages:crashed because the host-side per-identifier temp dir was never createdtest-environment/CIBW_TEST_ENVIRONMENT_ANDROIDwas documented but silently ignored; it's now applied to before-test, the wheel install, and the testbed invocationUnified behavior
RepairStepProducedMultipleWheelsError, previously Linux/Android-only)AlreadyBuiltWheelErrorduplicate-name check; its redundant post-repairnone-any.whlcheck is dropped (the built wheel is still checked, like the other platforms)FatalErrorinstead of callingsys.exit(1)(same exit code)before_testno longer receives an undocumented{wheel}placeholder;test_commandnow receives{wheel}like Linux/macOS/WindowsAssertionError("uv not found")→FatalError(macOS, Android)log.step_end()accounting, host temp dirs removed withignore_errors=True(pyodide previously never removed them), Android gains the "was moved to … instead of" warningTesting
unit_test/runner_test.py(new): a recordingFakeBuildercovers step ordering, compatible-wheel reuse (build/repair/audit/move skipped, test still runs), 0/2-wheel repair errors, duplicate-name and none-any errors, test gating, and the error wrapper + troubleshoot hook.build()monkeypatch seams andoption_prepare_test.py), doctests, ruff, strict mypy (3.11 + 3.14), pylint.test_troubleshooting+test_before_all(11 passed) andtest_custom_repair_wheel+test_abi_variants(6 passed, covering repair validation and compatible-wheel reuse). macOS CPython/Windows/iOS/Android rely on this PR's CI.🤖 Generated with Claude Code