Skip to content

fix: slim container image #314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
roivanov wants to merge 7 commits into
base: main
Choose a base branch
from
Open

fix: slim container image #314

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8acd812
fix: use slim container image
roivanov Apr 3, 2026
e02c25a
debug
roivanov Apr 3, 2026
4a0bc51
cleanup
roivanov Apr 3, 2026
9e8e741
pin on tags
roivanov Apr 3, 2026
c207957
pin on tags
roivanov Apr 3, 2026
cea1693
move env vars
roivanov Apr 3, 2026
1935931
bump
roivanov Apr 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 44 additions & 14 deletions .github/workflows/conda-python-build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,21 +57,21 @@ jobs:

# Please keep the matrices sorted in ascending order by the following:
#
# [ARCH, PY_VER, CUDA_VER, LINUX_VER]
# [ARCH, PY_VER]
#
export MATRIX="
# amd64
- { ARCH: 'amd64', PY_VER: '3.10', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'amd64', PY_VER: '3.11', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'amd64', PY_VER: '3.12', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'amd64', PY_VER: '3.13', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'amd64', PY_VER: '3.14', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'amd64', PY_VER: '3.10' }
- { ARCH: 'amd64', PY_VER: '3.11' }
- { ARCH: 'amd64', PY_VER: '3.12' }
- { ARCH: 'amd64', PY_VER: '3.13' }
- { ARCH: 'amd64', PY_VER: '3.14' }
# arm64
- { ARCH: 'arm64', PY_VER: '3.10', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'arm64', PY_VER: '3.11', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'arm64', PY_VER: '3.12', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'arm64', PY_VER: '3.13', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'arm64', PY_VER: '3.14', CUDA_VER: '12.9.1', LINUX_VER: 'rockylinux8' }
- { ARCH: 'arm64', PY_VER: '3.10' }
- { ARCH: 'arm64', PY_VER: '3.11' }
- { ARCH: 'arm64', PY_VER: '3.12' }
- { ARCH: 'arm64', PY_VER: '3.13' }
- { ARCH: 'arm64', PY_VER: '3.14' }
"

MATRIX="$(
Expand All @@ -80,16 +80,34 @@ jobs:
)"

echo "MATRIX=${MATRIX}" | tee --append "${GITHUB_OUTPUT}"
fetch-gha-tools:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
repository: rapidsai/gha-tools
ref: v0.0.198
path: gha-tools
fetch-depth: 1
- uses: actions/upload-artifact@v7
with:
name: rapids-gha-tools
path: gha-tools
Comment on lines +92 to +95
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== workflow excerpt ==="
sed -n '86,132p' .github/workflows/conda-python-build.yaml

echo
echo "=== direct RAPIDS tool invocations ==="
sed -n '1,14p' ci/upload_conda.sh

upload_doc=$(mktemp)
download_doc=$(mktemp)
gha_readme=$(mktemp)
trap 'rm -f "$upload_doc" "$download_doc" "$gha_readme"' EXIT

curl -fsSL https://raw.githubusercontent.com/actions/upload-artifact/main/README.md > "$upload_doc"
curl -fsSL https://raw.githubusercontent.com/actions/download-artifact/main/README.md > "$download_doc"
curl -fsSL https://raw.githubusercontent.com/rapidsai/gha-tools/v0.0.198/README.md > "$gha_readme"

echo
echo "=== upload-artifact permission semantics ==="
grep -nE 'Permission Loss|File permissions are not maintained|tar all of your files together' "$upload_doc"

echo
echo "=== download-artifact extraction semantics ==="
grep -nE 'Destination path|single artifact .*directly to the specified path|individual named directories' "$download_doc"

echo
echo "=== upstream gha-tools install guidance ==="
grep -nE 'git clone .*gha-tools|tools.tar.gz|GITHUB_PATH' "$gha_readme"

echo
echo "=== gha-tools repo root at v0.0.198 ==="
python - <<'PY'
import json, urllib.request
url = "https://api.github.com/repos/rapidsai/gha-tools/contents?ref=v0.0.198"
with urllib.request.urlopen(url) as resp:
    data = json.load(resp)
for item in data:
    print(item["name"])
PY

Repository: NVIDIA/numbast

Length of output: 6094

Package only the tools/ payload, not the whole gha-tools checkout.

GitHub artifacts do not preserve file permissions: single-artifact downloads extract directly to the requested path with all files set to 644 and directories to 755. With line 130 adding ${GITHUB_WORKSPACE}/tools to PATH, the current setup downloads the entire external repo into the workspace root, where rapidsai/gha-tools has top-level README.md, LICENSE, CONTRIBUTING.md, and other files that can overwrite the checked-out source tree. More critically, tools like rapids-find-anaconda-uploads.py and rapids-retry (invoked directly in ci/upload_conda.sh) lose their executable bit after the artifact round-trip, causing invocation failures.

🔧 Suggested fix 
   fetch-gha-tools:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v6
         with:
           repository: rapidsai/gha-tools
           ref: v0.0.198
           path: gha-tools
           fetch-depth: 1
+      - name: Archive RAPIDS gha-tools
+        run: tar -C gha-tools -czf rapids-gha-tools-tools.tgz tools
       - uses: actions/upload-artifact@v7
         with:
           name: rapids-gha-tools
-          path: gha-tools
+          path: rapids-gha-tools-tools.tgz
           if-no-files-found: error
...
       - uses: actions/download-artifact@v6
         with:
           name: rapids-gha-tools
-          path: ${{ github.workspace }}
+          path: ${{ github.workspace }}/gha-tools
+      - name: Extract RAPIDS gha-tools
+        run: |
+          mkdir -p "${GITHUB_WORKSPACE}/gha-tools"
+          tar -xzf "${GITHUB_WORKSPACE}/gha-tools/rapids-gha-tools-tools.tgz" -C "${GITHUB_WORKSPACE}/gha-tools"
       - name: Add RAPIDS gha-tools to PATH
-        run: echo "${GITHUB_WORKSPACE}/tools" | tee -a "${GITHUB_PATH}"
+        run: echo "${GITHUB_WORKSPACE}/gha-tools/tools" | tee -a "${GITHUB_PATH}"
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/conda-python-build.yaml around lines 92 - 95, The workflow
currently uploads the entire gha-tools checkout artifact (name:
rapids-gha-tools, path: gha-tools), which when re-downloaded overwrites repo
files and strips execute permissions for scripts referenced on PATH; change the
upload to only package the tools payload (e.g., tools/ subdirectory) so
consumers add ${GITHUB_WORKSPACE}/tools to PATH safely and scripts like
rapids-find-anaconda-uploads.py and rapids-retry retain their executable
bits—update the actions/upload-artifact step to point its path to the tools
subdirectory and adjust any downstream download/restore steps accordingly.

if-no-files-found: error
build:
needs: compute-matrix
needs:
- compute-matrix
- fetch-gha-tools
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.compute-matrix.outputs.MATRIX) }}
runs-on: "linux-${{ matrix.ARCH }}-${{ inputs.node_type }}"
env:
RAPIDS_ARTIFACTS_DIR: ${{ github.workspace }}/artifacts
RAPIDS_CONDA_BLD_OUTPUT_DIR: /tmp/conda-bld-output
RAPIDS_CONDA_BLD_ROOT_DIR: /tmp/conda-bld-workspace
container:
image: rapidsai/ci-conda:cuda${{ matrix.CUDA_VER }}-${{ matrix.LINUX_VER }}-py${{ matrix.PY_VER }}
image: condaforge/miniforge3:26.1.1-3
env:
RAPIDS_BUILD_TYPE: ${{ inputs.build_type }}
steps:
Expand All @@ -103,6 +121,15 @@ jobs:
repository: ${{ inputs.repo }}
ref: ${{ inputs.sha }}
fetch-depth: 0
# One clone in fetch-gha-tools; matrix jobs only download the artifact into the mounted workspace.
- uses: actions/download-artifact@v6
with:
name: rapids-gha-tools
path: ${{ github.workspace }}
- name: Add RAPIDS gha-tools to PATH
run: echo "${GITHUB_WORKSPACE}/tools" | tee -a "${GITHUB_PATH}"
- name: Mark workspace safe for the git command
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
- name: Standardize repository information
run: |
echo "RAPIDS_REPOSITORY=${{ inputs.repo || github.repository }}" >> "${GITHUB_ENV}"
Expand All @@ -123,13 +150,16 @@ jobs:
- name: Build Conda Package
run: ${{ inputs.script }}
env:
# renovate: datasource=github-releases depName=rapidsai/sccache
SCCACHE_VER: 0.14.0-rapids.1
GH_TOKEN: ${{ github.token }}
PYTHON_VERSION: ${{ matrix.PY_VER }}
- name: Upload conda repo
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
with:
name: conda-repo-${{ matrix.ARCH }}-${{ matrix.PY_VER }}
path: "/tmp/conda-bld-output"
path: ${{ env.RAPIDS_CONDA_BLD_OUTPUT_DIR }}
- name: Publish Conda Package
if: inputs.upload_to_conda
run: "ci/upload_conda.sh"
Expand Down
26 changes: 26 additions & 0 deletions ci/build_conda_python.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,32 @@

set -euo pipefail

chmod +x tools/rapids-*

apt update
apt install -y jq gh unzip curl gettext

curl -fsSL "https://raw.githubusercontent.com/rapidsai/ci-imgs/v26.06.00a/context/condarc.tmpl" | \
envsubst | tee ~/.condarc

SCCACHE_VERSION="${SCCACHE_VER}" rapids-install-sccache

rapids-mamba-retry update --all -y -n base
conda install -y conda-build

# install expected Python version
PYTHON_MAJOR_VERSION=${PYTHON_VERSION%%.*}
PYTHON_MINOR_VERSION=${PYTHON_VERSION#*.}
PYTHON_UPPER_BOUND="${PYTHON_MAJOR_VERSION}.$((PYTHON_MINOR_VERSION+1)).0a0"
PYTHON_MINOR_PADDED=$(printf "%02d" "$PYTHON_MINOR_VERSION")
PYTHON_VERSION_PADDED="${PYTHON_MAJOR_VERSION}.${PYTHON_MINOR_PADDED}"
if [[ "$PYTHON_VERSION_PADDED" > "3.12" ]]; then
PYTHON_ABI_TAG="cp${PYTHON_MAJOR_VERSION}${PYTHON_MINOR_VERSION}"
else
PYTHON_ABI_TAG="cpython"
fi
rapids-mamba-retry install -y -n base "python>=${PYTHON_VERSION},<${PYTHON_UPPER_BOUND}=*_${PYTHON_ABI_TAG}"
Comment on lines +26 to +31
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Bash string comparison for version numbers is fragile.

The > operator in [[ ... ]] performs lexicographic (string) comparison, not numeric comparison. While "3.12" > "3.13" happens to work correctly, this pattern will break for edge cases like comparing "3.9" (which would incorrectly sort after "3.12" lexicographically since "9" > "1").

Given that the matrix includes versions 3.10–3.14, this currently works, but the logic is error-prone if 3.9 or earlier is ever added.

🔧 Proposed fix using numeric comparison 
-PYTHON_MINOR_PADDED=$(printf "%02d" "$PYTHON_MINOR_VERSION")
-PYTHON_VERSION_PADDED="${PYTHON_MAJOR_VERSION}.${PYTHON_MINOR_PADDED}"
-if [[ "$PYTHON_VERSION_PADDED" > "3.12" ]]; then
+if [[ "$PYTHON_MAJOR_VERSION" -gt 3 ]] || { [[ "$PYTHON_MAJOR_VERSION" -eq 3 ]] && [[ "$PYTHON_MINOR_VERSION" -gt 12 ]]; }; then
     PYTHON_ABI_TAG="cp${PYTHON_MAJOR_VERSION}${PYTHON_MINOR_VERSION}"
 else
     PYTHON_ABI_TAG="cpython"
 fi
🧰 Tools
🪛 Shellcheck (0.11.0)

[error] 26-26: Decimals are not supported. Either use integers only, or use bc or awk to compare.

(SC2072)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@ci/build_conda_python.sh` around lines 26 - 31, The conditional using string
comparison (if [[ "$PYTHON_VERSION_PADDED" > "3.12" ]]) is fragile; change it to
perform numeric comparison by comparing integer major/minor parts: parse
PYTHON_MAJOR_VERSION and PYTHON_MINOR_VERSION (or split PYTHON_VERSION_PADDED)
and compare PYTHON_MAJOR_VERSION first then PYTHON_MINOR_VERSION against 3 and
12 respectively, then set PYTHON_ABI_TAG as before
(PYTHON_ABI_TAG="cp${PYTHON_MAJOR_VERSION}${PYTHON_MINOR_VERSION}" else
"cpython"); ensure the install line using PYTHON_ABI_TAG remains unchanged.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

conda install version constraint syntax with build string selector

💡 Result:

The syntax for conda install with version constraint and build string selector uses the MatchSpec format: conda install package==version=build_string for exact matches, or variations like package=version=build_string (single = before build in some contexts). For patterns, use wildcards like package=version=py3* or bracket notation package[build=py3*]. Examples: - Exact: conda install numpy==1.23.1=py39hc836922_0 - Version with build pattern: conda install numpy=1.23.1=py39* - Build selector: conda install numpy[build=py39*] - Exclude build: conda install numpy[build!=hc292b87_0] This works in conda install, conda search, and environment files. Use quotes for complex specs with spaces or operators: conda install "numpy>=1.23=py39*".

Citations:

🏁 Script executed:

cd ci && wc -l build_conda_python.sh

Repository: NVIDIA/numbast

Length of output: 82

🏁 Script executed:

cd ci && sed -n '25,35p' build_conda_python.sh

Repository: NVIDIA/numbast

Length of output: 453

🏁 Script executed:

rg "rapids-mamba-retry install" -A 2 -B 2

Repository: NVIDIA/numbast

Length of output: 356

🌐 Web query:

conda MatchSpec syntax multiple version constraints build string >= < operators

💡 Result:

Conda MatchSpec syntax supports multiple version constraints using operators like >=, <, <=, >, ==, !=, combined with logical operators , (AND) and | (OR). Build strings can be specified exactly or with wildcards/patterns. Key syntax elements: - Positional form: name version build - Examples: - numpy==1.21.0=py39h1234567_0 (exact version and build) - foo 1.0 py27_0 → foo==1.0=py27_0 - >=1.0 , < 2.0 py34_0 → version '>=1.0,<2.0' with build py34_0 - Bracket notation for explicit constraints: - conda-forge::foo[version='>=1.0'] - numpy[version='>=1.7,<2.0', build=py3*] - Version specifiers support: - Single: >=1.0, <2.0, ==1.2.3, 1.2.* (fuzzy) - Multiple: >=1.0,<2.0 (AND), >1.8,<2|==1.7 (AND then OR) - Note: Spaces around operators are ignored or normalized; no parentheses for grouping (may cause parsing issues). - Build string: Exact like =py27_0 or patterns like py2*, specified after version with = or in brackets [build=py2*]. Canonical form: (channel(/subdir):)name(version(build))[key=value,...] Usage examples: - conda install "numpy>=1.20,<1.22 py3*" - conda search conda-forge::python[version='>=3.9',build='h*'] This is based on conda's official MatchSpec documentation (conda 26.1.1) and source code, which confirm parsing of complex version+build specs like ">=1.0 , < 2.0 py34_0" and bracket syntax for operators.

Citations:

🏁 Script executed:

# Search for similar patterns in the codebase
rg "python>=" --type sh -A 1 -B 1

Repository: NVIDIA/numbast

Length of output: 245

Fix the conda version constraint syntax to properly specify the build string.

The version constraint "python>=${PYTHON_VERSION},<${PYTHON_UPPER_BOUND}=*_${PYTHON_ABI_TAG}" does not follow standard conda MatchSpec format. The build pattern =*_${PYTHON_ABI_TAG} is ambiguously attached only to the upper bound. Use standard syntax instead:

"python>=${PYTHON_VERSION},<${PYTHON_UPPER_BOUND} *_${PYTHON_ABI_TAG}"

Or use bracket notation for explicit constraints:

"python[version='${PYTHON_VERSION},<${PYTHON_UPPER_BOUND}', build='*_${PYTHON_ABI_TAG}']"
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@ci/build_conda_python.sh` at line 31, The conda MatchSpec in the
rapids-mamba-retry install line is malformed; update the package spec used in
the command that references PYTHON_VERSION, PYTHON_UPPER_BOUND and
PYTHON_ABI_TAG to use valid conda syntax — either separate the build string from
the version range (e.g., "python>=${PYTHON_VERSION},<${PYTHON_UPPER_BOUND}
*_${PYTHON_ABI_TAG}") or use bracket notation with explicit version and build
fields (e.g.,
"python[version='>=${PYTHON_VERSION},<${PYTHON_UPPER_BOUND}',build='*_${PYTHON_ABI_TAG}']"),
and replace the current
`"python>=${PYTHON_VERSION},<${PYTHON_UPPER_BOUND}=*_${PYTHON_ABI_TAG}"` token
with one of these valid forms in the rapids-mamba-retry install invocation.


source rapids-configure-sccache

source rapids-date-string
Expand Down
Loading