Skip to content

Parallelize Python test suite with pytest-xdist (31s -> 12s) - #51

Merged
Aztec03hub merged 1 commit into
mainfrom
perf/parallelize-pytest-xdist
Jun 26, 2026
Merged

Parallelize Python test suite with pytest-xdist (31s -> 12s)#51
Aztec03hub merged 1 commit into
mainfrom
perf/parallelize-pytest-xdist

Conversation

@Aztec03hub

Copy link
Copy Markdown
Owner

What

Parallelize the Python unit-test suite with pytest-xdist. The 1526-test suite was running serially on a single core (~63% CPU on an 8-core box). This defaults the run to -n auto --dist=worksteal, a pure wall-clock win with identical results.

Results (8-core box)

wall clock test time CPU
Before (serial) ~31s ~26s 61%
After (-n auto) ~12s cold, ~6-9s warm ~9s 307%

~2.6x faster cold wall-clock, same 1526 passed, zero skips added.

Changes

  • Add pytest-xdist>=3.0 to the dev optional-dependency group.
  • Add addopts = "-n auto --dist=worksteal" to [tool.pytest.ini_options] (existing testpaths and filterwarnings untouched). worksteal rebalances pending tests onto idle workers, which matters because the textual TUI tests are much slower than the rest.

Parallel safety

No production or test logic needed changing — the suite was already parallel-safe, verified by audit:

  • Real socket binding (test_mqtt_ws_bridge.py, test_doctor.py) already uses OS-assigned ephemeral ports via socket.bind((host, 0)) / getsockname().
  • Broker / publisher lifecycle tests (test_broker.py, test_publisher.py) fully mock amqtt / aiomqtt, so no real ports are opened.
  • Filesystem / env / home / cwd state flows through per-test tmp_path, monkeypatch.setenv, and patched Path.home(). Each xdist worker is its own process with its own tmp base, so 8 workers never collide. No test writes to a real ~/.claude-comms or a repo-relative path.
  • The fixed ports (1883 / 9001 / 9920 / 9921) in conftest's tmp_config / e2e_config are config-dict values only and are never bound to a socket.

CI

The Test job runs plain pytest --tb=short --junitxml=... with no addopts override and no -p no:xdist, so it inherits -n auto automatically. pytest-xdist ships via the [all,dev] extra the job already installs. Verified the exact CI command works under xdist (junitxml records tests="1526", exit 0).

Flake gate — 5 consecutive parallel runs

Run 1: exit=0 | 1526 passed in 6.31s
Run 2: exit=0 | 1526 passed in 6.67s
Run 3: exit=0 | 1526 passed in 8.21s
Run 4: exit=0 | 1526 passed in 8.64s
Run 5: exit=0 | 1526 passed in 9.75s

All 1526 passed, all exit 0 — no parallel flakiness. ruff check, ruff format --check, and basedpyright (src 0/0, tests 0/0) all remain clean.

🤖 Generated with Claude Code

Add pytest-xdist to the dev dependency group and default the pytest run
to '-n auto --dist=worksteal' so the 1526-test unit suite runs across all
CPU cores instead of serially on one.

The suite was already parallel-safe: every test that binds a real socket
(test_mqtt_ws_bridge, test_doctor) uses an OS-assigned ephemeral port via
socket.bind((host, 0)); the broker/publisher lifecycle tests fully mock
amqtt/aiomqtt so no real ports are opened; and all filesystem and env
state flows through per-test tmp_path / monkeypatch. Each xdist worker is
its own process with its own tmp base, so 8 workers never collide. The
fixed ports in conftest's tmp_config/e2e_config are config-dict values
only and are never bound.

worksteal rebalances pending tests onto idle workers, which matters
because the textual TUI tests are much slower than the rest.

CI's Test job runs plain 'pytest ...' with no addopts override, so it
inherits -n auto automatically; pytest-xdist ships via the dev extra the
job already installs ([all,dev]).

Result: ~31s -> ~12s cold wall (2.6x), ~6-9s warm. 1526 passed, verified
stable across 5 consecutive parallel runs (all exit 0).
@Aztec03hub
Aztec03hub merged commit 496eb18 into main Jun 26, 2026
8 checks passed
@Aztec03hub
Aztec03hub deleted the perf/parallelize-pytest-xdist branch June 26, 2026 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant