Dependency reproducibility: fresh installs drift off the tested lockfile

[arelchan]

Summary

A fresh install can silently drift onto untested upstream dependency versions, because the wheel only carries loose ranges and neither the wheel nor uv tool install consults uv.lock. This is the class of problem behind #113 (missing orjson). We have a temporary fix; this issue tracks the long-term, systematic approach and should not be rushed.

What happened (concrete instance)

• pyproject.toml pins litellm>=1.82.1,<2.0.0 (a range); uv.lock pins litellm==1.85.0.
• uv tool install / wheel installs ignore uv.lock and resolve the newest in-range version at install time -> litellm 1.92.0.
• litellm 1.92's request path imports orjson at runtime; 1.85 did not. orjson is only declared under litellm's proxy extra (which raven does not install), so it was absent from the dependency graph entirely.
• Result: fresh installs on the newer litellm crash every turn with APIConnectionError: OpenrouterException - No module named 'orjson', surfaced in the TUI as the generic turn_failed (code=-32099).

Root problem in one line: uv.lock protects dev/CI only; the versions users actually get are unpinned and drift with upstream releases.

Temporary fix (in flight)

• Missing 'orjson' dependency causes APIConnectionError on fresh install #113 -> fix: declare orjson as a direct dependency #116: declare orjson as a direct dependency (mirrors litellm's own >=3.11.6,<4.0). Unblocks users now.

This is a point fix for one symptom. It does not stop the next silent upstream change from doing the same thing.

Long-term options to evaluate (not decided yet)

1. Distribution honors the lock. Export the locked set from uv.lock at release time and install with --constraint, so a fresh install gets exactly the versions CI tested. A draft implementation is in build: pin fresh-install deps to the release lockfile #117 (release exports requirements.txt; install.sh pins against it). Verified locally: unpinned install drifts litellm to 1.92.0; with the exported constraint it resolves the locked 1.85.0.

   • Open questions: PyPI path (wheel metadata still loose), pip install users, sdist, keeping the constraint in sync per release.

2. CI guard: fresh-install + real-turn smoke. Today CI mocks the LLM and never installs the wheel fresh and runs a real turn, so Missing 'orjson' dependency causes APIConnectionError on fresh install #113 sailed through green. Add a job that installs the wheel into a clean env (fresh resolution, like a user) and drives a real request/response round trip against a local stub server (no secrets, no cost). Note: litellm's mock_response does NOT exercise the response-parse path where orjson is imported, so the guard must hit a real HTTP round trip to be meaningful.

3. Version policy for volatile deps + bots. Cap the highest-risk deps (e.g. litellm to a minor: >=1.85,<1.86) so the wheel itself constrains drift, and use Dependabot/Renovate to propose bumps as PRs that run the full (incl. fresh-install) CI. Trade-off: tighter control vs. manual maintenance of security/bug-fix updates.

Ask

• Decide the combination (my lean: 1 + 2 as the durable fix, 3 as a lightweight complement for a couple of high-risk deps).
• Assign a release/CI owner, since 1 and 2 touch release.yml, install.sh, and ci.yml.

Refs

• Root symptom: Missing 'orjson' dependency causes APIConnectionError on fresh install #113
• Temp fix: fix: declare orjson as a direct dependency #116
• Draft for option 1: build: pin fresh-install deps to the release lockfile #117