Skip to content

perf(ci): cut CUDA build time and archive size #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
worthant merged 2 commits into from
Aug 5, 2026
Merged

perf(ci): cut CUDA build time and archive size #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dc1fb5d
:zap: perf(ci): cut CUDA build time and archive size
worthant Aug 5, 2026
6f2460a
:bug: fix(ci): keep the windows job names off the required-check list
worthant Aug 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 54 additions & 14 deletions .github/workflows/dev-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ jobs:
find build/bin -name "*.so*" -exec cp -P {} release/build/bin/ \; 2>/dev/null || true
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-x64-vulkan.zip .
zip -ry ../llama-turboquant-linux-x64-vulkan.zip .
tar -czf ../llama-turboquant-linux-x64-vulkan.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -147,9 +147,10 @@ jobs:

- name: Build
# Consumer SASS: RTX 30 (86), RTX 40 (89), RTX 50 / Blackwell (120).
# PTX floor at Turing (75-virtual) so Turing/older consumer and all
# server cards (A100 80, H100/H200 90, B200 100) JIT at first run --
# we don't ship their SASS, which is what bloated the archive. Runner
# Two PTX floors: 75-virtual for Turing, 80-virtual so the server cards
# (A100 80, H100 90, B200 100) JIT Ampere-class code instead of Turing
# code -- cp.async and the Ampere MMA path are gated on __CUDA_ARCH__
# >= 800, so a compute_75 PTX fallback quietly cost them both. Runner
# has no GPU: build only, backend is a dlopen'd libggml-cuda.so.
run: |
cmake -B build \
Expand All @@ -161,7 +162,8 @@ jobs:
-DGGML_CPU_ALL_VARIANTS=ON \
-DGGML_CUDA=ON \
-DGGML_CUDA_CUB_3DOT2=ON \
-DCMAKE_CUDA_ARCHITECTURES="75-virtual;86-real;89-real;120-real" \
-DCMAKE_CUDA_ARCHITECTURES="75-virtual;80-virtual;86-real;89-real;120-real" \
-DCMAKE_CUDA_FLAGS=-compress-mode=size \
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \
-DLLAMA_CURL=OFF \
-DLLAMA_OPENSSL=OFF \
Expand Down Expand Up @@ -192,7 +194,7 @@ jobs:
cp -P /usr/local/cuda/lib64/libcublasLt.so* release/build/bin/
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-x64-cuda-13.3.zip .
zip -ry ../llama-turboquant-linux-x64-cuda-13.3.zip .
tar -czf ../llama-turboquant-linux-x64-cuda-13.3.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -262,6 +264,7 @@ jobs:
-DGGML_CUDA=ON \
-DGGML_CUDA_CUB_3DOT2=ON \
-DCMAKE_CUDA_ARCHITECTURES="90-virtual;121-real" \
-DCMAKE_CUDA_FLAGS=-compress-mode=size \
-DCMAKE_CUDA_HOST_COMPILER=g++-14 \
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \
-DLLAMA_CURL=OFF \
Expand Down Expand Up @@ -298,7 +301,7 @@ jobs:
cp -P "$CUDA_LIB"/libcublasLt.so* release/build/bin/
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-arm64-cuda-13.3.zip .
zip -ry ../llama-turboquant-linux-arm64-cuda-13.3.zip .
tar -czf ../llama-turboquant-linux-arm64-cuda-13.3.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -365,7 +368,7 @@ jobs:
find build/bin -name "*.so*" -exec cp -P {} release/build/bin/ \; 2>/dev/null || true
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-x64-cpu.zip .
zip -ry ../llama-turboquant-linux-x64-cpu.zip .
tar -czf ../llama-turboquant-linux-x64-cpu.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -447,7 +450,7 @@ jobs:
cp -P /usr/local/cuda/lib64/libcublasLt.so* release/build/bin/
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-x64-cuda-12.4.zip .
zip -ry ../llama-turboquant-linux-x64-cuda-12.4.zip .
tar -czf ../llama-turboquant-linux-x64-cuda-12.4.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -561,7 +564,7 @@ jobs:
MI200/MI300). Older GCN GPUs: use the Vulkan build.
EOF
cd release
zip -r ../llama-turboquant-linux-x64-rocm.zip .
zip -ry ../llama-turboquant-linux-x64-rocm.zip .
tar -czf ../llama-turboquant-linux-x64-rocm.tar.gz .

- name: Upload archive
Expand All @@ -584,6 +587,7 @@ jobs:
cmake_flags: ''
- variant: vulkan
cmake_flags: '-DGGML_VULKAN=ON'
# Do not add flags here -- see the CUDA_EXTRA note in the Build step.
- variant: cuda-12.4
cuda: '12.4'
cmake_flags: '-DGGML_CUDA=ON -DGGML_CUDA_CUB_3DOT2=ON'
Expand All @@ -601,7 +605,10 @@ jobs:
uses: ggml-org/ccache-action@v1.2.21
with:
key: turboquant-windows-x64-${{ matrix.variant }}
evict-old-files: 1d
# 500M (the action default) cannot hold a CUDA build, so the cache
# thrashed and every run was effectively cold.
max-size: ${{ startsWith(matrix.variant, 'cuda') && '2G' || '500M' }}
evict-old-files: 7d

- name: Install CUDA Toolkit
if: ${{ startsWith(matrix.variant, 'cuda') }}
Expand All @@ -621,11 +628,35 @@ jobs:
- name: Install Ninja
run: choco install ninja -y

# Defender scans every object nvcc writes, and the CUDA variants write
# tens of thousands of them. Best-effort: never fail the build over it.
- name: Exclude build tree from Defender
shell: pwsh
run: |
try {
Add-MpPreference -ExclusionPath "${{ github.workspace }}", "$env:RUNNER_TEMP"
Add-MpPreference -ExclusionProcess "nvcc.exe", "cl.exe", "ninja.exe", "cicc.exe", "ptxas.exe", "cudafe++.exe"
} catch {
Write-Host "Defender exclusions unavailable: $_"
}

- name: Build
shell: cmd
# CUDA_EXTRA carries the pinned arch lists (the ggml default builds 8
# targets on 12.4 and 7 on 13.3, including DGX Spark SASS on Windows)
# and -compress-mode=size, which needs CTK >= 12.8 so 12.4 cannot have
# it. These live here rather than in the matrix because a matrix value
# becomes part of the job's display name, and those names are master's
# required status checks -- renaming one leaves the required check
# permanently "Expected" and the PR unmergeable.
# Nothing that runs today loses support: only 90-virtual on 12.4
# (Hopper still JITs from 80-virtual) and 121a-real on 13.3 are dropped.
# NOTE: GGML_CUDA_CUB_3DOT2 can be dropped once CCCL 3.2 ships in the CTK used here.
run: |
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
set CUDA_EXTRA=
if "${{ matrix.variant }}"=="cuda-12.4" set CUDA_EXTRA=-DCMAKE_CUDA_ARCHITECTURES="50-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-real;89-real"
if "${{ matrix.variant }}"=="cuda-13.3" set CUDA_EXTRA=-DCMAKE_CUDA_ARCHITECTURES="75-virtual;80-virtual;86-real;89-real;90-virtual;120a-real" -DCMAKE_CUDA_FLAGS=-compress-mode=size
cmake -S . -B build -G "Ninja Multi-Config" ^
-DGGML_NATIVE=OFF ^
-DGGML_BACKEND_DL=ON ^
Expand All @@ -637,8 +668,8 @@ jobs:
-DLLAMA_BUILD_TOOLS=ON ^
-DLLAMA_BUILD_TESTS=OFF ^
-DLLAMA_BUILD_EXAMPLES=OFF ^
${{ matrix.cmake_flags }}
set /A NINJA_JOBS=%NUMBER_OF_PROCESSORS%-1
${{ matrix.cmake_flags }} %CUDA_EXTRA%
set NINJA_JOBS=%NUMBER_OF_PROCESSORS%
cmake --build build --config Release -j %NINJA_JOBS% -t ggml
cmake --build build --config Release -j %NINJA_JOBS%

Expand Down Expand Up @@ -678,7 +709,16 @@ jobs:
New-Item -ItemType Directory -Force -Path release\build\bin | Out-Null
Copy-Item .\build\bin\Release\* release\build\bin\ -Recurse -Force
Copy-Item .\LICENSE release\build\bin\ -ErrorAction SilentlyContinue
Compress-Archive -Path release\build -DestinationPath llama-turboquant-windows-x64-${{ matrix.variant }}.zip -Force
$zip = "llama-turboquant-windows-x64-${{ matrix.variant }}.zip"
# Compress-Archive is single-threaded and needs double-digit minutes
# on the CUDA archives. 7-Zip ships with the runner image.
if (Get-Command 7z -ErrorAction SilentlyContinue) {
Push-Location release
7z a -tzip -mx=5 -mmt=on "..\$zip" build | Out-Null
Pop-Location
} else {
Compress-Archive -Path release\build -DestinationPath $zip -Force
}

- name: Upload archive
uses: actions/upload-artifact@v4
Expand Down
68 changes: 54 additions & 14 deletions .github/workflows/release-turboquant.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ jobs:
find build/bin -name "*.so*" -exec cp -P {} release/build/bin/ \; 2>/dev/null || true
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-x64-vulkan.zip .
zip -ry ../llama-turboquant-linux-x64-vulkan.zip .
tar -czf ../llama-turboquant-linux-x64-vulkan.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -150,9 +150,10 @@ jobs:

- name: Build
# Consumer SASS: RTX 30 (86), RTX 40 (89), RTX 50 / Blackwell (120).
# PTX floor at Turing (75-virtual) so Turing/older consumer and all
# server cards (A100 80, H100/H200 90, B200 100) JIT at first run --
# we don't ship their SASS, which is what bloated the archive. Runner
# Two PTX floors: 75-virtual for Turing, 80-virtual so the server cards
# (A100 80, H100 90, B200 100) JIT Ampere-class code instead of Turing
# code -- cp.async and the Ampere MMA path are gated on __CUDA_ARCH__
# >= 800, so a compute_75 PTX fallback quietly cost them both. Runner
# has no GPU: build only, backend is a dlopen'd libggml-cuda.so.
run: |
cmake -B build \
Expand All @@ -164,7 +165,8 @@ jobs:
-DGGML_CPU_ALL_VARIANTS=ON \
-DGGML_CUDA=ON \
-DGGML_CUDA_CUB_3DOT2=ON \
-DCMAKE_CUDA_ARCHITECTURES="75-virtual;86-real;89-real;120-real" \
-DCMAKE_CUDA_ARCHITECTURES="75-virtual;80-virtual;86-real;89-real;120-real" \
-DCMAKE_CUDA_FLAGS=-compress-mode=size \
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \
-DLLAMA_CURL=OFF \
-DLLAMA_OPENSSL=OFF \
Expand Down Expand Up @@ -195,7 +197,7 @@ jobs:
cp -P /usr/local/cuda/lib64/libcublasLt.so* release/build/bin/
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-x64-cuda-13.3.zip .
zip -ry ../llama-turboquant-linux-x64-cuda-13.3.zip .
tar -czf ../llama-turboquant-linux-x64-cuda-13.3.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -266,6 +268,7 @@ jobs:
-DGGML_CUDA=ON \
-DGGML_CUDA_CUB_3DOT2=ON \
-DCMAKE_CUDA_ARCHITECTURES="90-virtual;121-real" \
-DCMAKE_CUDA_FLAGS=-compress-mode=size \
-DCMAKE_CUDA_HOST_COMPILER=g++-14 \
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache \
-DLLAMA_CURL=OFF \
Expand Down Expand Up @@ -302,7 +305,7 @@ jobs:
cp -P "$CUDA_LIB"/libcublasLt.so* release/build/bin/
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-arm64-cuda-13.3.zip .
zip -ry ../llama-turboquant-linux-arm64-cuda-13.3.zip .
tar -czf ../llama-turboquant-linux-arm64-cuda-13.3.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -370,7 +373,7 @@ jobs:
find build/bin -name "*.so*" -exec cp -P {} release/build/bin/ \; 2>/dev/null || true
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-x64-cpu.zip .
zip -ry ../llama-turboquant-linux-x64-cpu.zip .
tar -czf ../llama-turboquant-linux-x64-cpu.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -453,7 +456,7 @@ jobs:
cp -P /usr/local/cuda/lib64/libcublasLt.so* release/build/bin/
cp LICENSE release/build/bin/ 2>/dev/null || true
cd release
zip -r ../llama-turboquant-linux-x64-cuda-12.4.zip .
zip -ry ../llama-turboquant-linux-x64-cuda-12.4.zip .
tar -czf ../llama-turboquant-linux-x64-cuda-12.4.tar.gz .

- name: Upload archive
Expand Down Expand Up @@ -562,7 +565,7 @@ jobs:
MI200/MI300). Older GCN GPUs: use the Vulkan build.
EOF
cd release
zip -r ../llama-turboquant-linux-x64-rocm.zip .
zip -ry ../llama-turboquant-linux-x64-rocm.zip .
tar -czf ../llama-turboquant-linux-x64-rocm.tar.gz .

- name: Upload archive
Expand All @@ -586,6 +589,7 @@ jobs:
cmake_flags: ''
- variant: vulkan
cmake_flags: '-DGGML_VULKAN=ON'
# Do not add flags here -- see the CUDA_EXTRA note in the Build step.
- variant: cuda-12.4
cuda: '12.4'
cmake_flags: '-DGGML_CUDA=ON -DGGML_CUDA_CUB_3DOT2=ON'
Expand All @@ -603,7 +607,10 @@ jobs:
uses: ggml-org/ccache-action@v1.2.21
with:
key: turboquant-windows-x64-${{ matrix.variant }}
evict-old-files: 1d
# 500M (the action default) cannot hold a CUDA build, so the cache
# thrashed and every run was effectively cold.
max-size: ${{ startsWith(matrix.variant, 'cuda') && '2G' || '500M' }}
evict-old-files: 7d

- name: Install CUDA Toolkit
if: ${{ startsWith(matrix.variant, 'cuda') }}
Expand All @@ -623,10 +630,34 @@ jobs:
- name: Install Ninja
run: choco install ninja -y

# Defender scans every object nvcc writes, and the CUDA variants write
# tens of thousands of them. Best-effort: never fail the build over it.
- name: Exclude build tree from Defender
shell: pwsh
run: |
try {
Add-MpPreference -ExclusionPath "${{ github.workspace }}", "$env:RUNNER_TEMP"
Add-MpPreference -ExclusionProcess "nvcc.exe", "cl.exe", "ninja.exe", "cicc.exe", "ptxas.exe", "cudafe++.exe"
} catch {
Write-Host "Defender exclusions unavailable: $_"
}

- name: Build
shell: cmd
# CUDA_EXTRA carries the pinned arch lists (the ggml default builds 8
# targets on 12.4 and 7 on 13.3, including DGX Spark SASS on Windows)
# and -compress-mode=size, which needs CTK >= 12.8 so 12.4 cannot have
# it. These live here rather than in the matrix because a matrix value
# becomes part of the job's display name, and those names are master's
# required status checks -- renaming one leaves the required check
# permanently "Expected" and the PR unmergeable.
# Nothing that runs today loses support: only 90-virtual on 12.4
# (Hopper still JITs from 80-virtual) and 121a-real on 13.3 are dropped.
run: |
call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64
set CUDA_EXTRA=
if "${{ matrix.variant }}"=="cuda-12.4" set CUDA_EXTRA=-DCMAKE_CUDA_ARCHITECTURES="50-virtual;61-virtual;70-virtual;75-virtual;80-virtual;86-real;89-real"
if "${{ matrix.variant }}"=="cuda-13.3" set CUDA_EXTRA=-DCMAKE_CUDA_ARCHITECTURES="75-virtual;80-virtual;86-real;89-real;90-virtual;120a-real" -DCMAKE_CUDA_FLAGS=-compress-mode=size
cmake -S . -B build -G "Ninja Multi-Config" ^
-DGGML_NATIVE=OFF ^
-DGGML_BACKEND_DL=ON ^
Expand All @@ -638,8 +669,8 @@ jobs:
-DLLAMA_BUILD_TOOLS=ON ^
-DLLAMA_BUILD_TESTS=OFF ^
-DLLAMA_BUILD_EXAMPLES=OFF ^
${{ matrix.cmake_flags }}
set /A NINJA_JOBS=%NUMBER_OF_PROCESSORS%-1
${{ matrix.cmake_flags }} %CUDA_EXTRA%
set NINJA_JOBS=%NUMBER_OF_PROCESSORS%
cmake --build build --config Release -j %NINJA_JOBS% -t ggml
cmake --build build --config Release -j %NINJA_JOBS%

Expand Down Expand Up @@ -675,7 +706,16 @@ jobs:
New-Item -ItemType Directory -Force -Path release\build\bin | Out-Null
Copy-Item .\build\bin\Release\* release\build\bin\ -Recurse -Force
Copy-Item .\LICENSE release\build\bin\ -ErrorAction SilentlyContinue
Compress-Archive -Path release\build -DestinationPath llama-turboquant-windows-x64-${{ matrix.variant }}.zip -Force
$zip = "llama-turboquant-windows-x64-${{ matrix.variant }}.zip"
# Compress-Archive is single-threaded and needs double-digit minutes
# on the CUDA archives. 7-Zip ships with the runner image.
if (Get-Command 7z -ErrorAction SilentlyContinue) {
Push-Location release
7z a -tzip -mx=5 -mmt=on "..\$zip" build | Out-Null
Pop-Location
} else {
Compress-Archive -Path release\build -DestinationPath $zip -Force
}

- name: Upload archive
uses: actions/upload-artifact@v4
Expand Down
Loading