Release validation #11
|
What sort of testing / what things should be verified before each binary release? |
Replies: 2 comments 4 replies
|
If we end up doing this and it's all doable from the CPython repo itself it would probably be part of general CI. We can also have a buildbot help as well (see #11 (reply in thread)). |
|
I worry that current CI doesn't validate well "here's a fresh install of <platform> does downloading and doing the steps on the webpage result in a working python". Two of the issues I've run into around that on similar projects:
👍 that these should be automated / in CI / a tool to run which does it for the most part. |
|
PBS has a small pile of Rust code to validate built distributions. Lots of spot tests for loading extensions and verifying certain functionality works. (Like optional SQLite features.) It also enforces an allow list of library dependencies, glibc symbol versions, and spot checks other symbol properties. Various checks were added after regressions were accidentally shipped. It's important to do this at CI time at the latest. |
|
Yeah I think checks similar to what PBS does would be good. I also think we can have install buildbots like https://github.com/python/buildmaster-config/blob/a3bd9128393e49f48c9cf97dc18966adf12a9051/master/custom/factories.py#L191 |
|
FWIW, I've got a similar set of "smoke tests" for Apple Standalone in BeeWare. I've had occasions where CPython compiles fine, but the specific mechanics of the redistributable redistributable artefact then fail - sometimes because optional features in the build weren't autodetected, and sometimes because of formatting/configuration issue. |
|
To be clear, I don't expect there to be instructions on python.org on how to download and use these builds. I personally view them as for workflow tools to use or advanced users. |
If we end up doing this and it's all doable from the CPython repo itself it would probably be part of general CI. We can also have a buildbot help as well (see #11 (reply in thread)).