Skip to content

refactor: shared build runner across platforms#2907

Draft
henryiii wants to merge 7 commits into
pypa:mainfrom
henryiii:refactor/shared-build-runner
Draft

refactor: shared build runner across platforms#2907
henryiii wants to merge 7 commits into
pypa:mainfrom
henryiii:refactor/shared-build-runner

Conversation

@henryiii

@henryiii henryiii commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Alternative to #2903, this time using Claude Fable 5 instead of Claude Opus 4.8. This is not just a one to one; this one doesn't have nearly as much restraining it, and I didn't tell it where I wanted to go later (--stage). But otherwise it's also an unedited output of a plan.

🤖 AI text below 🤖

What

Extracts the per-wheel build loop that was duplicated across all six platform modules into a shared runner (cibuildwheel/platforms/runner.py), driven by a per-platform builder class with clearly-named steps:

setup → before_build → build_wheel → repair_wheel → audit_wheel
      → test_wheel → move_to_output → cleanup
  • Builder(ABC, Generic[PathT]) — one method per step. Generic over the wheel-path type because Linux wheels live in the container as PurePosixPath; everywhere else they're host Paths.
  • run_builds() — a single flat loop that owns everything that was previously copy-pasted six times: log.build_start/build_end/step accounting, compatible-wheel (abi3) reuse, the none-any.whl check, exactly-one-repaired-wheel validation, AlreadyBuiltWheelError, the test gate, output bookkeeping, and cleanup. The loop body reads top-to-bottom like the old per-platform loops — there's no step registry or hook indirection.
  • HostBuilder — the step bodies shared by the five non-Linux platforms (before_build, default repair, audit, move-to-output, cleanup).
  • run_before_all() / fatal_on_called_process_error() — replace the per-platform copies of the before_all hook and the CalledProcessError → FatalError wrapper (Linux keeps its per-container scope and troubleshoot() hook).

Each platform's build() shrinks to ~15 lines; platform-specific structure stays where it was and stays visible: macOS keeps its per-arch test loop (universal2/Rosetta), iOS its testbed flow, Android its dual build/android environments (the previous BuildState pattern was the prototype for this design), pyodide its pyodide venv testing, and Linux keeps get_build_steps() grouping and the OCIContainer lifecycle, invoking the runner once per container. build_in_container() keeps its signature (it's mocked in option_prepare_test.py), and the PlatformModule interface is unchanged.

Behavior changes

The duplication had let the platforms drift; this unifies the unjustified differences (one commit per platform, each itemized in its commit message):

Bugs fixed

  • pyodide: compatible-wheel reuse hit a latent NameError (assigned built_wheel, read repaired_wheel) — unreachable today only because pyodide wheels are never reused
  • pyodide: a repair command producing no wheel raised a raw StopIteration instead of RepairStepProducedNoWheelError
  • Linux: dependency-versions with packages: crashed because the host-side per-identifier temp dir was never created
  • Android: test-environment / CIBW_TEST_ENVIRONMENT_ANDROID was documented but silently ignored; it's now applied to before-test, the wheel install, and the testbed invocation

Unified behavior

  • repair must produce exactly one wheel on every platform (macOS/Windows/iOS gain RepairStepProducedMultipleWheelsError, previously Linux/Android-only)
  • Android gains the AlreadyBuiltWheelError duplicate-name check; its redundant post-repair none-any.whl check is dropped (the built wheel is still checked, like the other platforms)
  • Windows: the "ARM64 wheels cannot be tested" warning now only appears when a test command is actually configured
  • iOS: a test-suite failure raises FatalError instead of calling sys.exit(1) (same exit code)
  • pyodide: before_test no longer receives an undocumented {wheel} placeholder; test_command now receives {wheel} like Linux/macOS/Windows
  • AssertionError("uv not found")FatalError (macOS, Android)
  • cosmetics: one compatible-wheel reuse message everywhere, uniform log.step_end() accounting, host temp dirs removed with ignore_errors=True (pyodide previously never removed them), Android gains the "was moved to … instead of" warning

Testing

  • unit_test/runner_test.py (new): a recording FakeBuilder covers step ordering, compatible-wheel reuse (build/repair/audit/move skipped, test still runs), 0/2-wheel repair errors, duplicate-name and none-any errors, test gating, and the error wrapper + troubleshoot hook.
  • Existing suites pass unchanged: 802 unit tests (including the build() monkeypatch seams and option_prepare_test.py), doctests, ruff, strict mypy (3.11 + 3.14), pylint.
  • Integration run locally: pyodide (4 passed, end-to-end builds) and Linux in Docker — test_troubleshooting + test_before_all (11 passed) and test_custom_repair_wheel + test_abi_variants (6 passed, covering repair validation and compatible-wheel reuse). macOS CPython/Windows/iOS/Android rely on this PR's CI.

🤖 Generated with Claude Code

henryiii added 7 commits June 9, 2026 17:14
Adds cibuildwheel/platforms/runner.py: a Builder ABC (generic over the
wheel path type, since Linux wheels live in the container as
PurePosixPath) whose methods are the platform-specific build steps, and
run_builds(), a single flat loop that owns everything identical across
platforms: logging, compatible-wheel reuse, built/repaired wheel
validation, the test gate, output bookkeeping, and cleanup.

HostBuilder provides the step bodies shared by the five non-Linux
platforms. run_before_all() and fatal_on_called_process_error() replace
the per-platform copies of the before_all hook and the
CalledProcessError-to-FatalError wrapper.

No platform uses this yet; subsequent commits port them one at a time.

Assisted-by: ClaudeCode:claude-fable-5
The per-wheel loop in build() is replaced by a PyodideBuilder driven by
runner.run_builds(). Behavior changes:

- a compatible-wheel (abi3) reuse no longer crashes with a NameError
  (the old loop assigned built_wheel but read repaired_wheel; it was
  unreachable in practice since pyodide wheels are never reused)
- a repair command producing zero wheels now raises
  RepairStepProducedNoWheelError instead of a raw StopIteration, and
  producing several is now an error, like on Linux
- before_test no longer receives the undocumented {wheel} placeholder;
  test_command now does, matching the other platforms
- the per-identifier temp dir is now removed after each build

Assisted-by: ClaudeCode:claude-fable-5
The per-wheel loop becomes a MacOSBuilder; the per-architecture test
loop (universal2 testing both halves, Rosetta emulation, arch-specific
test-skip identifiers) stays inside test_wheel(), unchanged.

Behavior changes:
- a repair command producing several wheels is now an error, as on Linux
- a missing uv now raises FatalError instead of AssertionError
- the per-identifier temp dir is removed with ignore_errors, like Windows

Assisted-by: ClaudeCode:claude-fable-5
The per-wheel loop becomes a WindowsBuilder. The ARM64-on-non-ARM64
test skip moves inside test_wheel(), so its warning now only appears
when a test command is actually configured. A repair command producing
several wheels is now an error, as on Linux.

Assisted-by: ClaudeCode:claude-fable-5
The per-wheel loop becomes an IOSBuilder; the testbed-based test flow,
the non-simulator/non-native skip steps, and the python-module test
command validation stay inside test_wheel().

Behavior changes:
- a test-suite failure now raises FatalError instead of calling
  sys.exit(1) directly (same exit code, consistent error reporting)
- a repair command producing several wheels is now an error, as on Linux
- the compatible-wheel reuse message now matches the other platforms
- the per-identifier temp dir is removed with ignore_errors

Assisted-by: ClaudeCode:claude-fable-5
The BuildState dataclass and its step functions fold into an
AndroidBuilder; setup_target_python/setup_env and the other
environment helpers are unchanged.

Behavior changes:
- test-environment (CIBW_TEST_ENVIRONMENT_ANDROID) is now applied to
  the test steps (before-test, wheel install, testbed invocation); it
  was documented but previously ignored on Android
- a wheel name colliding with an earlier build now raises
  AlreadyBuiltWheelError, like every other platform
- the redundant post-repair none-any.whl check is dropped (the built
  wheel is still checked, like every other platform)
- the compatible-wheel reuse message now matches the other platforms,
  and the moved-elsewhere warning from move_file is now emitted
- a missing uv now raises FatalError instead of AssertionError

Assisted-by: ClaudeCode:claude-fable-5
The per-config loop in build_in_container() becomes a LinuxBuilder
(generic over PurePosixPath — wheels stay container-side, and the
existing copy-out and host-side audit behavior is preserved as
overridden steps). build() keeps the container grouping, engine check,
and OCIContainer lifecycle; the CalledProcessError wrapper keeps its
per-container scope and troubleshoot() hook via
fatal_on_called_process_error. build_in_container() keeps its
signature, since tests mock it.

Also fixes a latent bug: the host-side per-identifier temp dir was
never created, so dependency-versions with packages: would have
crashed writing constraints.txt on Linux.

Assisted-by: ClaudeCode:claude-fable-5
@henryiii henryiii marked this pull request as draft June 9, 2026 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant