Skip to content

Deflake the issue-1363 tests: wait for lifespan startup instead of sleeping#2879

Merged
maxisbey merged 1 commit into
mainfrom
maxisbey/deflake-1363-startup-handshake
Jun 15, 2026
Merged

Deflake the issue-1363 tests: wait for lifespan startup instead of sleeping#2879
maxisbey merged 1 commit into
mainfrom
maxisbey/deflake-1363-startup-handshake

Conversation

@maxisbey

Copy link
Copy Markdown
Contributor

The three tests in tests/issues/test_1363_race_condition_streamable_http.py synchronize with their server thread via a fixed await anyio.sleep(0.1). On a loaded CI runner the thread sometimes isn't ready within that window, and the test fails with RuntimeError: Task group is not initialized. Make sure to use run(). This replaces the fixed sleep with a readiness handshake.

Motivation and Context

These tests (added in #1384) start a ServerThread that spins up its own event loop, enters the Starlette lifespan, and only then sets the session manager's task group. The test waits a fixed 0.1s and then sends requests from its own event loop via httpx.ASGITransport. When the thread loses the scheduling race — easy under pytest -n auto plus coverage on a 4-vCPU runner — the first request reaches handle_request() while _task_group is still None and the test fails.

This has flaked 6 times since 2026-05-27, on both Ubuntu and Windows, across unrelated PRs — e.g. 3.14/locked/windows and 3.14/locked/ubuntu. Because the test job is continue-on-error, these never turn a run red, so they're easy to miss.

The fix: the server thread sets a threading.Event once lifespan startup has completed (at that point the task group is guaranteed to exist), and the tests wait on it (bounded at 5s, via anyio.to_thread.run_sync) instead of sleeping. The trailing anyio.sleep(0.2) that gives the original #1363 race its detection window is deliberately left untouched — the tests still exercise exactly the same request paths and log checks.

How Has This Been Tested?

  • The three tests pass, and pass 150/150 with --flake-finder --flake-runs=50 -n 4.
  • Failure-mode A/B with an artificial 0.3s delay injected at the top of ServerThread.run(): the old tests reproduce the exact CI error, the fixed tests pass. Repeated on a windows-latest runner (Python 3.14): unpatched + delay fails all three tests with the same RuntimeError; patched + delay passes; patched 50× per test under -n 4 passes 150/150.
  • ./scripts/test passes at 100% coverage; ruff/pyright clean.

Breaking Changes

None — test-only change.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

All three tests in the file shared the same fixed-sleep pattern; only test_race_condition_invalid_accept_headers happened to be the one observed failing in CI, but the delayed-thread experiment shows the other two fail the same way, so all three call sites are converted.

AI Disclaimer

…eeping

The three tests in test_1363_race_condition_streamable_http.py waited a
fixed 0.1s for the ServerThread to start the app lifespan before sending
requests. On a loaded CI runner the thread is sometimes not ready in time,
so the first request reaches handle_request() before the session manager's
task group exists and the test fails with "RuntimeError: Task group is not
initialized" (seen intermittently on both Ubuntu and Windows jobs).

Replace the fixed sleep with a threading.Event that the server thread sets
once lifespan startup has completed; the tests wait for it (bounded at 5s)
before sending the first request.
@maxisbey maxisbey marked this pull request as ready for review June 15, 2026 15:48
@maxisbey maxisbey enabled auto-merge (squash) June 15, 2026 15:49
@maxisbey maxisbey merged commit ddb7b78 into main Jun 15, 2026
30 checks passed
@maxisbey maxisbey deleted the maxisbey/deflake-1363-startup-handshake branch June 15, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants