ci: test every interpreter the project claims to support - #1089
Closed
doublewhy wants to merge 2 commits into
Closed
ci: test every interpreter the project claims to support#1089doublewhy wants to merge 2 commits into
doublewhy wants to merge 2 commits into
Conversation
`asyncio.get_event_loop()` outside a running loop has warned since 3.12 and raises `RuntimeError: There is no current event loop` on 3.14, so the whole module fails there — 20+ tests — while only emitting a DeprecationWarning on the interpreters CI currently pins. `asyncio.run` is the supported spelling for driving a coroutine from sync test code. Verified by running the module under `-W error::DeprecationWarning`, which reproduces the 3.14 failure: the previous spelling errors out, this one passes all 85 tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`requires-python = ">=3.11"` is open-ended and the trove classifiers
advertise 3.11 and 3.12, but every workflow pins 3.12, so the project
publishes to interpreters nothing exercises. That is how the
`asyncio.get_event_loop()` call fixed in the parent commit stayed
invisible: it was latent on every version above 3.12 and only surfaced
when uv happened to select 3.14.
A new `interpreters` job runs the hermetic unit suite on 3.11, 3.12, 3.13
and 3.14, with `fail-fast: false` so one version failing still reports the
others. `verify` deliberately stays single-version: it owns the coverage
artifact Sonar consumes and the Isabelle replay, and neither varies by
interpreter.
Classifiers now list 3.13 and 3.14, which the file's own comment asks for
("Keep this list in step with `requires-python`").
Verified locally before wiring it up: the hermetic suite is 6313 passed on
3.11.15 and on 3.14.4, both with the full locked dependency set including
the governed z3-solver 4.16.0.0 pin.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
|
Please note in the comments or body which issue this PR closes. If no issue exists, please create one and link it. Thank you! |
Author
Author
|
Correction after reconciling the accepted issue scope: this basic matrix is only part of #1097, so I changed the body from |
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Plain-language summary
Related issues
Part of #1097
Depends on #1087. The accepted #1097 scope also requires an explicit
<3.15bound, exact-runtime selection, distribution build and clean-install smoke, and a separately labelled 3.14t preview; those are not yet in this PR.GitHub currently targets this branch directly at
dev, so its diff also includes #1087's MCP test commit. It remains blocked and does not satisfy all #1097 acceptance criteria; the focused qualification successor will replace this combined branch.What breaks
The project publishes to interpreters nothing tests.
requires-python = ">=3.11"is open-ended, sopip install raessucceeds on 3.13, 3.14, and 3.15 when it arrives. Every workflow pins 3.12. So a user on 3.13 installs a version whose test suite has never been run on their interpreter.That is not hypothetical: it is exactly how the bug in #1087 survived.
asyncio.get_event_loop()has been deprecated since 3.12 and raises on 3.14, sotests/test_mcp_server.pywas already broken on every version above the pin — invisibly, untiluvhappened to pick 3.14 on a machine of mine and ~20 tests failed at once.The trove classifiers advertise 3.11 and 3.12 only, which contradicts
requires-pythonin the other direction. The comment directly above them already asks for the two to agree:# Keep this list in step with `requires-python`.The fix
A new
interpretersjob runs the hermetic unit suite on 3.11, 3.12, 3.13 and 3.14, withfail-fast: falseso one version failing still reports the rest. Classifiers gain 3.13 and 3.14 to match.Why not just matrix
verify, or move the pinverifydeliberately stays single-version. It owns thecoverage-reportartifact that thesonarjob downloads, and three jobs writing one artifact name collide — Sonar would ingest coverage from an arbitrary interpreter. It also acquires and replays Isabelle, whose result cannot vary by interpreter, and it is pinned toubuntu-22.04specifically so bubblewrap can enforce the proof sandbox. Fanning all of that out costs CI time and risks the governed parts for no signal.Raising the pin instead of adding a matrix would just relocate the blind spot to whichever versions the new pin excludes.
How I know
I ran the hermetic suite on each end of the range before wiring it up, using the locked dependency set:
uv sync --frozenresolved every dependency on both 3.11 and 3.14, including the governedz3-solver==4.16.0.0pin that reproducible satisfiability evidence depends on.3.14 passes only with #1087 applied; on
devtoday it fails.One risk worth naming
Python 3.14 makes PEP 649 deferred annotation evaluation the default, and this project has ~158 files of Pydantic contracts that lean on annotation introspection. The runs above are reassuring, but they are one commit. This job is what keeps that true as the contract layer grows — which is the argument for adding it now rather than at the point someone wants to bump the pin.
If you would rather start advisory, adding
continue-on-error: trueto the job makes it non-blocking in one line.🤖 Generated with Claude Code