Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 43 additions & 20 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ on:
jobs:
cpu:
runs-on: linux-amd64-cpu8
strategy:
fail-fast: false
matrix:
# pinned: install from uv.lock (exact locked versions)
# minimal: re-resolve with --resolution lowest-direct (lowest
# versions allowed by pyproject.toml bounds, e.g. torch 2.9)
resolution: [pinned, minimal]
container:
image: nvidia/cuda:13.2.1-cudnn-devel-ubuntu24.04
env:
Expand All @@ -30,12 +37,16 @@ jobs:
id: get-pr-info
uses: nv-gha-runners/get-pr-info@main

- name: Install lint system dependencies
- name: Install system dependencies
run: |
apt-get update -qq
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \
python3 python3-dev python3-venv \
git curl ca-certificates
ffmpeg \
gcc g++ ninja-build \
libnccl-dev \
git curl ca-certificates unzip
rm -rf /var/lib/apt/lists/*

- name: Checkout
uses: actions/checkout@v4
Expand All @@ -47,38 +58,39 @@ jobs:
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-suffix: "cpu"
cache-suffix: "cpu-${{ matrix.resolution }}"

- name: Install lint dependencies
- name: Install dependencies
env:
# Skip compiling CUDA kernels for block-sparse-attn on CPU runners;
# the Python interface is enough for CPU-only unit tests.
BLOCK_SPARSE_ATTN_SKIP_CUDA_BUILD: "TRUE"
run: |
# ty needs third-party packages to resolve imports. Skip packages
# that cannot be installed on CPU-only runners:
# - transformer-engine-torch: source-only, requires GPU arch to compile
uv venv --clear
uv sync --extra dev --group lint \
--no-install-package transformer-engine-torch
if [ "${{ matrix.resolution }}" = "minimal" ]; then
uv sync --resolution lowest-direct --extra dev --group lint \
--no-install-package transformer-engine-torch
else
Comment on lines +70 to +73

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The minimal CPU cell installs --group lint (which pulls in pre-commit and ty) even though the "Run linter checks" step is skipped for that cell. Only --group test is needed to satisfy pytest-asyncio and imageio-ffmpeg for the unit-test step, saving two unnecessary package installs per minimal run.

Suggested change
if [ "${{ matrix.resolution }}" = "minimal" ]; then
uv sync --resolution lowest-direct --extra dev --group lint \
--no-install-package transformer-engine-torch
else
if [ "${{ matrix.resolution }}" = "minimal" ]; then
uv sync --resolution lowest-direct --extra dev --group test \
--no-install-package transformer-engine-torch
else

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

# ty needs third-party packages to resolve imports. Skip packages
# that cannot be installed on CPU-only runners:
# - transformer-engine-torch: source-only, requires GPU arch to compile
uv sync --extra dev --group lint \
--no-install-package transformer-engine-torch
fi

- name: Run linter checks
# Type-checking (ty) validates against installed stub versions;
# minimal resolution pulls older stubs with different signatures
# that produce false positives. Only lint against pinned versions.
if: matrix.resolution == 'pinned'
run: |
# Container runs as root but the workspace is owned by the runner
# uid -- git refuses to operate without safe.directory.
git config --global --add safe.directory "$(pwd)"
uv run --no-sync pre-commit run -a

- name: Install remaining system dependencies
run: |
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends \
ffmpeg \
gcc g++ ninja-build \
libnccl-dev \
unzip
rm -rf /var/lib/apt/lists/*

- name: Build flashdreams wheel
if: matrix.resolution == 'pinned'
run: uv build --wheel --package flashdreams

- name: Run CPU unit tests
Expand All @@ -89,6 +101,13 @@ jobs:
gpu:
runs-on: linux-amd64-gpu-rtxpro6000-latest-2
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
# pinned: install from uv.lock (exact locked versions)
# minimal: re-resolve with --resolution lowest-direct (lowest
# versions allowed by pyproject.toml bounds, e.g. torch 2.9)
resolution: [pinned, minimal]
container:
image: nvidia/cuda:13.2.1-cudnn-devel-ubuntu24.04
options: --gpus all
Expand Down Expand Up @@ -139,7 +158,7 @@ jobs:
uses: astral-sh/setup-uv@v6
with:
enable-cache: true
cache-suffix: "gpu-sm${{ steps.gpu-arch.outputs.arch }}"
cache-suffix: "gpu-sm${{ steps.gpu-arch.outputs.arch }}-${{ matrix.resolution }}"
# Disable default prune --ci which removes pre-built wheels but also
# invalidates git-sourced builds (timestamps change on cache restore).
# Instead we selectively trim the cache in a post step below.
Expand All @@ -151,7 +170,11 @@ jobs:
BLOCK_SPARSE_ATTN_CUDA_ARCHS: ${{ steps.gpu-arch.outputs.arch }}
run: |
uv venv --clear
uv sync --extra dev
if [ "${{ matrix.resolution }}" = "minimal" ]; then
uv sync --resolution lowest-direct --extra dev
else
uv sync --extra dev
fi

- name: Verify GPU availability
run: |
Expand Down
13 changes: 3 additions & 10 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,16 +307,9 @@ The `flashdreams/pyproject.toml` declares minimum version bounds for all
runtime dependencies. These bounds reflect the oldest versions we believe
are compatible based on API analysis.

**CI tests run against the pinned versions in `uv.lock`**, not against
the declared minimums. This means:

- We guarantee correctness at the locked versions.
- We expect the package to work at the declared minimum bounds, but do
not continuously validate this in CI.
- If you encounter breakage with a version that satisfies the declared
bounds but differs from the lock file, please
[open an issue](https://github.com/NVIDIA/flashdreams/issues). We will
either fix compatibility or bump the bound in `pyproject.toml`.
CI tests both the **pinned versions** (from `uv.lock`) and the **declared
minimums** (via `uv sync --resolution lowest-direct`). If you change a
dependency bound, both resolution strategies must pass.

## Speeding up local builds

Expand Down
32 changes: 2 additions & 30 deletions flashdreams/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,17 +50,6 @@ dependencies = [
# no_restore_buffers=) and set_rotate_method in
# torch.distributed.tensor.experimental require PyTorch 2.9+.
"torch>=2.9",
# ``torch.compile`` lowers conv/elementwise kernels through Inductor +
# Triton. On Windows the upstream PyPI ``triton`` package is Linux-only,
# so without ``triton-windows`` Inductor crashes deep inside its
# template heuristics with ``TypeError: object() takes no arguments``.
# The maintainers ship a Windows-compatible drop-in under the
# ``triton-windows`` package; pulling it on win32 keeps every
# ``use_compile=True`` config path (encoder, decoder, DiT) working
# the same way it does on Linux. This is the same fix carried before
# on dev/truesight_alpa, scoped to the workspace member that owns
# the ``use_compile`` knob.
"triton-windows>=3.5.0; sys_platform == 'win32'",
]

[project.scripts]
Expand Down Expand Up @@ -92,7 +81,7 @@ dev = [
# main; the canonical extra is ``runners`` (same package set).
examples = [
"mediapy>=1.1",
"opencv-python-headless>=4.5",
"opencv-python-headless>=4.6",
"scipy>=1.11",
]
# I/O dependencies the ``flashdreams-run`` CLI runners lazy-import
Expand All @@ -101,7 +90,7 @@ examples = [
# pipeline (e.g. ``integrations/lingbot``) doesn't pull in ffmpeg / opencv.
runners = [
"mediapy>=1.1",
"opencv-python-headless>=4.5",
"opencv-python-headless>=4.6",
"scipy>=1.11",
]
serving = [
Expand All @@ -119,22 +108,5 @@ version = {attr = "flashdreams._version.__version__"}
include = ["flashdreams*"]
exclude = ["tests", "flashdreams._pytest_plugins*"]

# PyPI's default ``torch`` is the CPU build on Windows. The truesight CUDA
# extensions (built by ``uv run build-truesight``) need a CUDA-enabled
# PyTorch so PyTorch's ``cpp_extension`` resolves ``CUDA_HOME`` from the
# env (the gate is ``torch.version.cuda is not None``). On Linux the
# default wheel is already CUDA-enabled; on Windows we have to point at
# NVIDIA's PyTorch index. Constrain by marker so the override only fires
# on Windows -- Linux / NGC users keep the upstream wheel.
[[tool.uv.index]]
name = "pytorch-cu130"
url = "https://download.pytorch.org/whl/cu130"
explicit = true

[tool.uv.sources]
torch = [
{ index = "pytorch-cu130", marker = "sys_platform == 'win32'" },
]

[tool.uv]
managed = true
2 changes: 1 addition & 1 deletion integrations/causal_forcing/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ requires-python = ">=3.10"
dependencies = [
"flashdreams",
"mediapy>=1.1",
"opencv-python-headless>=4.5",
"opencv-python-headless>=4.6",
]

[tool.uv.sources]
Expand Down
4 changes: 2 additions & 2 deletions integrations/lingbot/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ dependencies = [
"aiohttp>=3.9",
"aiortc>=1.9",
"mediapy>=1.1",
"opencv-python-headless>=4.5",
"opencv-python-headless>=4.6",
"scipy>=1.11",
]

Expand All @@ -38,7 +38,7 @@ flashdreams = { workspace = true }
[project.optional-dependencies]
dev = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"pytest-asyncio>=0.24",
]

# Each entry registers one ``runner_name`` slug with ``flashdreams-run``.
Expand Down
4 changes: 2 additions & 2 deletions integrations/omnidreams/ludus-renderer/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ authors = [
requires-python = ">=3.10"
dependencies = [
"numpy>=1.24",
"torch>=2.11",
"torch>=2.9",
"pandas>=2.0",
"pyarrow>=14.0",
"scipy>=1.11",
Expand All @@ -41,7 +41,7 @@ dependencies = [
dev = [
"pytest>=8.0",
"imageio-ffmpeg>=0.5",
"opencv-python>=4.5",
"opencv-python>=4.6",
]

[build-system]
Expand Down
6 changes: 3 additions & 3 deletions integrations/omnidreams/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ dependencies = [
"grpcio-tools>=1.50",
"grpcio>=1.50",
"nvidia-cudnn-frontend==1.22.1",
"opencv-python-headless>=4.5",
"opencv-python-headless>=4.6",
"shapely>=2.0",
# Runtime deps for the ``omnidreams.interactive_drive`` desktop demo
# subpackage. Most are already transitive via ``flashdreams[serving]``
Expand All @@ -44,8 +44,8 @@ dependencies = [
"Pillow>=10.0",
"PyYAML>=6.0",
"pyarrow>=16.0",
"torch>=2.11",
"torchvision>=0.26",
"torch>=2.9",
"torchvision>=0.24",
"einops>=0.8",
"safetensors>=0.5",
"tqdm>=4.67",
Expand Down
2 changes: 1 addition & 1 deletion integrations/wan21/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ requires-python = ">=3.10"
dependencies = [
"flashdreams",
"mediapy>=1.1",
"opencv-python-headless>=4.5",
"opencv-python-headless>=4.6",
]

[tool.uv.sources]
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ markers = [
# flashdreams/_pytest_plugins.
test = [
"pytest>=8.0",
"pytest-asyncio>=0.23",
"pytest-asyncio>=0.24",
"pytest-manual-marker>=2.0",
"imageio-ffmpeg>=0.5",
]
Expand Down
Loading
Loading