bug: uv writes non-canonical dist-info dir names (oslo_cache vs oslo.cache), breaking pkg_resources lookups for dotted-name distributions

[larainema]

Summary

The Dockerfile installs all OpenStack dependencies with uv pip install. uv follows PEP 625 and writes *.dist-info directories with project names normalized so that dots become underscores — e.g. oslo.cache lands on disk as oslo_cache-3.10.2.dist-info. Plain pip historically preserved the dot (oslo.cache-3.10.2.dist-info).

pkg_resources (still embedded in the OpenStack runtime via keystonemiddleware, magnum-cluster-api, staffeln, etc.) cannot resolve Requirement.parse("oslo.cache>=...") against an oslo_cache-*.dist-info directory. The result is pkg_resources.DistributionNotFound at import time inside any consumer that calls pkg_resources.require() against a dotted-name distribution.

When such a consumer is loaded via stevedore (which swallows import errors silently), the affected entry points are dropped from the registry without any operator-visible warning.

Reproduction

In any image built from this repo (verified on ghcr.io/vexxhost/magnum:2025.1@sha256:31e5dd3995981666aea54ee15b2e2b3cd51aa630cd1ff61c09475a39c58952ef):

$ ls /var/lib/openstack/lib/python3.12/site-packages/ | grep -i oslo.cache
oslo_cache-3.10.2.dist-info       # underscore — written by uv
oslo_cache/

$ cat /var/lib/openstack/lib/python3.12/site-packages/oslo_cache-3.10.2.dist-info/INSTALLER
uv

$ /var/lib/openstack/bin/python3 -c "
import pkg_resources
pkg_resources.require('oslo.cache>=1.26.0')
"
Traceback (most recent call last):
  ...
pkg_resources.DistributionNotFound: The 'oslo.cache>=1.26.0' distribution was not found and is required by the application

Real-world impact

Hit in production on Atmosphere 2025.1. End-user symptom on the Magnum API:

HTTP 400 ClusterTypeNotSupported: Cluster type (vm, ubuntu, kubernetes) not supported

Conductor logs every 10 s:

File "/var/lib/openstack/lib/python3.12/site-packages/magnum/drivers/common/driver.py", line 138, in get_driver
    raise exception.ClusterTypeNotSupported(...)

Root cause: stevedore tried to load each magnum.drivers entry point from magnum-cluster-api; every load chain transitively imported keystonemiddleware, which raised DistributionNotFound; stevedore caught the exception and silently dropped all four CAPI drivers (k8s_cluster_api_ubuntu, _ubuntu_focal, _flatcar, _rockylinux) from the registry. Only the in-tree k8s_fedora_coreos_v1 driver remained, so any cluster_template referencing coe=kubernetes / server_type=vm / os_distro=ubuntu resolved to no driver.

Scope

I surveyed every running OpenStack pod on the affected cluster. All 12 vexxhost OpenStack images (barbican, cinder, glance, heat, ironic, keystone, magnum, manila, neutron, nova, octavia, placement) ship oslo_cache-*.dist-info with INSTALLER=uv. The bug is latent in all of them; the failure only becomes operator-visible when something in the import chain makes a pkg_resources.require() call against a dotted-name distribution. Today that is mostly Magnum (via certain magnum-cluster-api versions) and staffeln; tomorrow it could be anything else that adds such a call.

Other dotted-name distributions installed under non-canonical names in the same images:

• oslo.cache → oslo_cache-…dist-info
• oslo.config, oslo.context, oslo.db, oslo.i18n, oslo.log, oslo.messaging, oslo.middleware, oslo.policy, oslo.privsep, oslo.reports, oslo.serialization, oslo.service, oslo.upgradecheck, oslo.utils, oslo.versionedobjects, oslo.vmware — all oslo_*-…dist-info
• dogpile.cache → dogpile_cache-…dist-info
• Various XStatic.* packages in horizon

Affected branches

image.yml in vexxhost/atmosphere runs the same Earthly pipeline for every push to main and stable/*. Every Atmosphere image rebuilt after this repo switched to uv is affected. Image digests pinned in roles/defaults/vars/main.yml before that switch are safe.

Proposed fix

Two complementary options; suggesting both:

1. Pin uv to a version that preserves canonical dist-info names, or pass an option that does. (Needs verification — I have not yet found a uv flag that disables PEP 625 normalization; this may require an upstream astral-sh/uv ask.)

2. Post-install rename pass in the Dockerfile, immediately after uv pip install, that rewrites any *.dist-info directory whose METADATA Name: field contains a dot back into the dotted form. Single Python snippet, deterministic, idempotent.

   RUN <<'EOF' bash -xe
   /var/lib/openstack/bin/python3 - <<'PY'
   from pathlib import Path
   site = Path("/var/lib/openstack/lib/python3.12/site-packages")
   for d in site.glob("*.dist-info"):
       try:
           meta = (d / "METADATA").read_text(errors="replace").splitlines()
       except FileNotFoundError:
           continue
       name = next((l.split(":", 1)[1].strip()
                    for l in meta if l.lower().startswith("name:")), None)
       if not name or "." not in name:
           continue
       version = d.name.rsplit("-", 1)[1].removesuffix(".dist-info")
       canonical = site / f"{name}-{version}.dist-info"
       if d.name != canonical.name and not canonical.exists():
           d.rename(canonical)
   PY
   EOF