Migrate from pip to uv and upgrade Python 3.9 to 3.11#76
Open
harshavemula-ua wants to merge 7 commits intomainfrom
Open
Migrate from pip to uv and upgrade Python 3.9 to 3.11#76harshavemula-ua wants to merge 7 commits intomainfrom
harshavemula-ua wants to merge 7 commits intomainfrom
Conversation
- Replace pip with uv in Dockerfile.datastream-deps and Dockerfile.datastream - Install python3.11 instead of python3.9 in deps image - Add python3->python3.11 symlink in datastream image - Update setup.cfg python_requires to >=3.11 - Update 5 Python test workflows to use Python 3.11 - Remove buildx from x86 build/integration workflows - Fix ARM workflow tmpfs issue (docker prune + runner.temp paths)
Buildx stores images in its own cache, not the local Docker daemon, causing builds to pull old images from Docker Hub instead of using locally-built ones (results in uv: command not found).
Workflows that only built datastream were pulling the old datastream-deps image from Docker Hub (no uv/python3.11).
Only build datastream-deps from source when its Dockerfile has changed. Otherwise pull from Docker Hub to save CI time, especially important for ARM builds with compiled deps.
Use PR-aware git diff (base.sha...head.sha) with fetch-depth: 0 instead of origin/main...HEAD which fails in shallow clones. Also upgrade checkout v2 to v4 in ngiab and teehr workflows.
The git diff used HEAD~1 which fails in shallow clones, causing sed to remove ALL Docker build commands from the execution JSON. The Step Function then ran with no builds and falsely reported success. Replace with PR-aware git diff SHAs and add fetch-depth: 0.
github.ref_name returns '76/merge' for PRs which is not a valid git branch. Use github.head_ref to get the actual source branch name so the EC2 instance can clone the correct branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan