Add conda build option and CUDA target includes by JanuszL · Pull Request #6316 · NVIDIA/DALI

JanuszL · 2026-04-27T12:13:09Z

Adds a conda packaging build option and switches CUDA-header consumers to
the CMake CUDAToolkit target directory.

It absorbs https://github.com/conda-forge/nvidia-dali-python-feedstock/tree/main/recipe/patches.

Category:

Other (Build / Configuration)

Description:

This change adds BUILD_FOR_CONDA, enables it from the conda native-libs and
Python-bindings recipes, and keeps the default source and wheel build behavior
unchanged when the option is disabled.

When BUILD_FOR_CONDA=ON, the build:

uses conda's lib layout for nvImageCodec runtime lookup and RPATH;
prefers the shared libtar library instead of forcing libtar.a;
emits conda install guidance for dynamically loaded CUDA toolkit libraries;
keeps conda-provided CUDA architecture settings intact.

The change also requires CMake's CUDAToolkit package and uses
CUDAToolkit_TARGET_DIR/include directly for CUDA-family generated stubs and
include directories. This covers the CUDA, cuFile, cuFFT, NPP, nvImageCodec,
NVML, and nvCUVID stub generation paths without depending on the trimmed
CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES value.

For the split conda package build, bundled FFTS and cocoapi static targets are
not rebuilt when PREBUILD_DALI_LIBS=ON.

Additional information:

Affected modules and functionalities:

CMake configuration: CMakeLists.txt
Dependency lookup: cmake/Dependencies.common.cmake
Conda recipes:
- conda/dali_native_libs/recipe/build.sh
- conda/dali_python_bindings/recipe/build.sh
CUDA-header stub generation and include paths:
- dali/core/CMakeLists.txt
- dali/kernels/signal/fft/CMakeLists.txt
- dali/npp/CMakeLists.txt
- dali/nvimgcodec/CMakeLists.txt
- dali/operators/video/dynlink_nvcuvid/CMakeLists.txt
- dali/util/CMakeLists.txt
Dynamic loader diagnostics:
- dali/core/dynlink_cufile.cc
- dali/core/dynlink_nvcomp.cc
- dali/kernels/signal/fft/cufft_wrap.cc
- dali/npp/npp_wrap.cc
- dali/nvimgcodec/nvimgcodec_wrap.cc
- dali/operators/operators.cc
Minor typo fix in dali/util/nvml_wrap.cc

Key points relevant for the review:

BUILD_FOR_CONDA defaults to OFF; default builds should retain existing
behavior.
The option is propagated through propagate_option(BUILD_FOR_CONDA), which
produces FOR_CONDA_ENABLED for the C++ loader diagnostics.
find_package(CUDAToolkit REQUIRED) is now used so the build can consistently
reference CUDAToolkit_TARGET_DIR/include.
The obsolete nvJPEG2K conda-forge patch was intentionally not carried over.

Tests:

Previously configured successfully with:

default reduced CMake configuration
BUILD_FOR_CONDA=ON
BUILD_FOR_CONDA=ON with explicit CUDAToolkit_TARGET_DIR

For this metadata refresh, git diff --check upstream/main...HEAD passes.

Checklist

Documentation

DALI team only

Requirements

Implements new requirements
Affects existing requirements
N/A

REQ IDs: N/A

JIRA TASK: DALI-4681

greptile-apps · 2026-04-27T12:16:33Z

Greptile Summary

This PR introduces a BUILD_FOR_CONDA=OFF CMake option that adjusts the DALI build for conda-forge packaging: it switches nvimgcodec's runtime lookup path to lib/, prefers the shared libtar, emits conda-flavoured install guidance when dynamic libraries fail to load, and guards FFTS/cocoapi from being rebuilt when PREBUILD_DALI_LIBS=ON. It also migrates all CUDA-header stub generators from the fragile CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES extraction to the canonical CUDAToolkit_TARGET_DIR/include path via a new find_package(CUDAToolkit REQUIRED).

BUILD_FOR_CONDA defaults to OFF; standard wheel/source builds are unaffected.
All stub-generator custom_command invocations now use ${CUDAToolkit_TARGET_DIR}/include directly; only the nvcomp stub in dali/core/CMakeLists.txt still uses the old CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES_DIRECTIVE path, which is fed correctly by the root-level override when BUILD_FOR_CONDA=ON.
Conda recipes for both native-libs and python-bindings packages now pass -DBUILD_FOR_CONDA=ON; the python-bindings recipe pairs this with PREBUILD_DALI_LIBS=ON.

Confidence Score: 5/5

Safe to merge; all changes are build-system and diagnostic-message adjustments with no effect on non-conda builds.

The logic is straightforward: BUILD_FOR_CONDA defaults to OFF, so existing wheel and source builds are entirely unaffected. The conda-path changes are compile-time switches, not runtime data paths, so a misconfiguration would surface immediately as a build error rather than a silent runtime regression. No correctness or memory-safety concerns were found.

The CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES_DIRECTIVE computation blocks left in dali/npp/CMakeLists.txt, dali/nvimgcodec/CMakeLists.txt, dali/kernels/signal/fft/CMakeLists.txt, dali/operators/video/dynlink_nvcuvid/CMakeLists.txt, and dali/util/CMakeLists.txt are now dead code — a cleanup pass would tidy these up.

Important Files Changed

Filename	Overview
CMakeLists.txt	Adds BUILD_FOR_CONDA option, find_package(CUDAToolkit REQUIRED), per-config nvimgcodec rpath, and propagate_option(BUILD_FOR_CONDA); logic is correct and default behavior is unchanged.
cmake/Dependencies.common.cmake	Guards FFTS and cocoapi builds with NOT PREBUILD_DALI_LIBS; switches libtar search order for conda builds. Changes are scoped correctly.
dali/core/CMakeLists.txt	cuda.h and cufile.h stubs switched to CUDAToolkit_TARGET_DIR/include; target_include_directories updated similarly. The existing CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES_DIRECTIVE block is still needed here for the nvcomp stub.
dali/kernels/signal/fft/CMakeLists.txt	cufft stub migrated to CUDAToolkit_TARGET_DIR/include; the CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES_DIRECTIVE computation block (lines 28-37) is now dead code and should be removed.
dali/nvimgcodec/nvimgcodec_wrap.cc	Adds NVIMGCODEC_DEFAULT_LIBRARY_DIR macro and conda-specific dlopen error message; conda branch omits the attempts path list from the error, reducing debuggability.
dali/operators/operators.cc	Adds conda upgrade guidance with a properly single-quoted version spec in the error message; non-conda path updated to inline CUDA_VERSION / 1000.
dali/util/nvml_wrap.cc	Fixes pre-existing typo 'dirver' -> 'driver' in the dlopen failure message.
conda/dali_native_libs/recipe/build.sh	Adds -DBUILD_FOR_CONDA=ON to the native-libs CMake invocation.
conda/dali_python_bindings/recipe/build.sh	Adds -DBUILD_FOR_CONDA=ON to the python-bindings CMake invocation (paired with PREBUILD_DALI_LIBS=ON).

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[CMake configure] --> B{BUILD_FOR_CONDA?}
    B -- OFF --> C[Standard wheel/source build]
    B -- ON --> D{PREBUILD_DALI_LIBS?}
    C --> E[FFTS & cocoapi built from source]
    C --> F[libtar.a preferred]
    C --> G[nvimgcodec rpath: lib64/]
    C --> H[pip install guidance in errors]
    D -- OFF --> I[native-libs recipe: FFTS & cocoapi built]
    D -- ON --> J[python-bindings recipe: FFTS & cocoapi skipped]
    I --> K[libtar shared preferred]
    J --> K
    K --> L[nvimgcodec rpath: lib/]
    L --> M[conda install guidance in errors]
    M --> N[CUDAToolkit_TARGET_DIR/include for all stub generators]
    E --> O[CUDAToolkit_TARGET_DIR/include for all stub generators]

_{Reviews (4): Last reviewed commit: "Review fix" | Re-trigger Greptile}

JanuszL · 2026-04-27T12:25:39Z

@greptile review

dali-automaton · 2026-04-27T16:56:02Z

CI MESSAGE: [49621159]: BUILD STARTED

dali-automaton · 2026-04-27T18:01:17Z

CI MESSAGE: [49621159]: BUILD FAILED

dali-automaton · 2026-04-27T20:00:50Z

CI MESSAGE: [49621159]: BUILD PASSED

dali-automaton · 2026-04-27T21:23:11Z

CI MESSAGE: [49643983]: BUILD STARTED

dali-automaton · 2026-04-28T00:51:34Z

CI MESSAGE: [49643983]: BUILD FAILED

dali-automaton · 2026-04-28T06:42:21Z

CI MESSAGE: [49679151]: BUILD STARTED

dali-automaton · 2026-04-28T08:39:53Z

CI MESSAGE: [49679151]: BUILD FAILED

dali-automaton · 2026-04-28T11:55:07Z

CI MESSAGE: [49679151]: BUILD PASSED

dali-automaton · 2026-05-29T18:04:56Z

CI MESSAGE: [53088284]: BUILD STARTED

dali-automaton · 2026-05-29T18:29:13Z

CI MESSAGE: [53088284]: BUILD FAILED

dali-automaton · 2026-05-29T20:44:39Z

CI MESSAGE: [53103732]: BUILD STARTED

dali-automaton · 2026-05-30T21:03:59Z

CI MESSAGE: [53103732]: BUILD PASSED

Add BUILD_FOR_CONDA and pass it from the conda recipes so conda-forge packaging can opt into its own dependency layout without changing default builds. When enabled, prefer shared libtar, use conda's nvImageCodec lib directory in runtime lookup and RPATH, and emit conda install guidance for dynamically loaded CUDA toolkit libraries. Require CUDAToolkit package discovery and use CUDAToolkit_TARGET_DIR/include directly for generated CUDA-family stubs and CUDA include directories. This avoids relying on the trimmed CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES value while still honoring explicit CUDA target directories. Also skip bundled FFTS and cocoapi static targets when prebuilt DALI libs are used, and fix a typo in the NVML loader diagnostic. Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

jantonguirao · 2026-06-08T13:52:42Z

+
 # more recent CMake puts both cuda/targets/x86_64-linux/include and cuda/targets/x86_64-linux/include/cccl into the path
 # while we usually need only the former
 string(FIND "${CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES}" ";" SEMICOLON_POS)


[Minor] SEMICOLON_POS is set here but is no longer consumed — the if/else/endif block that used it to derive CUDA_TOOLKIT_INCLUDE_MAJOR_DIRECTORY was removed by this PR. The string(FIND ...) call is now dead code and can be deleted together with the two-line comment above it.

(The sub-CMakeLists.txt files still have their own copies of this pattern for the nvcomp stub, so they are unaffected.)

jantonguirao · 2026-06-08T14:03:27Z

+#if FOR_CONDA_ENABLED
+#define NVIMGCODEC_DEFAULT_LIBRARY_DIR "lib"
+#else
+#define NVIMGCODEC_DEFAULT_LIBRARY_DIR "lib64"
+#endif
+


just wondering: don't we link directly with the shared-object in Conda (without the dynlink wrapper I mean)?

I think it is possible but more substantial change to do. Now it is either the wrapper or bundling together.

Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

dali-automaton · 2026-06-08T15:34:34Z

CI MESSAGE: [54069946]: BUILD STARTED

dali-automaton · 2026-06-08T15:40:42Z

CI MESSAGE: [54069946]: BUILD STARTED

dali-automaton · 2026-06-08T16:19:49Z

CI MESSAGE: [54067531]: BUILD FAILED

dali-automaton · 2026-06-08T16:51:41Z

CI MESSAGE: [54067531]: BUILD STARTED

dali-automaton · 2026-06-09T14:49:49Z

CI MESSAGE: [54199753]: BUILD STARTED

dali-automaton · 2026-06-09T18:13:15Z

CI MESSAGE: [54199753]: BUILD FAILED

dali-automaton · 2026-06-09T18:36:36Z

CI MESSAGE: [54225421]: BUILD STARTED

dali-automaton · 2026-06-09T20:28:45Z

CI MESSAGE: [54225421]: BUILD FAILED

dali-automaton · 2026-06-10T08:40:16Z

CI MESSAGE: [54225421]: BUILD PASSED

JanuszL marked this pull request as draft April 27, 2026 12:13

greptile-apps Bot reviewed Apr 27, 2026

View reviewed changes

Comment thread dali/operators/operators.cc Outdated

JanuszL force-pushed the incorporate_conda_changes branch from 5666d1a to c985677 Compare April 27, 2026 12:20

JanuszL force-pushed the incorporate_conda_changes branch from c985677 to 71390d0 Compare April 27, 2026 16:53

JanuszL mentioned this pull request Apr 27, 2026

BLD: Split recipe into nvidia-dali-native + nvidia-dali-python outputs, bumps to v2.1.0 conda-forge/nvidia-dali-python-feedstock#65

Merged

5 tasks

JanuszL force-pushed the incorporate_conda_changes branch from 71390d0 to 832f595 Compare April 27, 2026 21:21

JanuszL force-pushed the incorporate_conda_changes branch from 832f595 to 2b83881 Compare April 28, 2026 06:41

JanuszL force-pushed the incorporate_conda_changes branch from 2b83881 to 8a6a7ef Compare May 29, 2026 18:03

JanuszL marked this pull request as ready for review June 1, 2026 11:19

JanuszL force-pushed the incorporate_conda_changes branch from 8a6a7ef to e2a4ea1 Compare June 1, 2026 11:24

JanuszL changed the title ~~Add conda-forge build option~~ Add conda build option and CUDA target includes Jun 1, 2026

dali-automaton assigned mdabek-nvidia and jantonguirao Jun 2, 2026

mdabek-nvidia approved these changes Jun 2, 2026

View reviewed changes

jantonguirao reviewed Jun 8, 2026

View reviewed changes

jantonguirao approved these changes Jun 8, 2026

View reviewed changes

Review fix

a92aa4a

Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>

NVIDIA deleted a comment from dali-automaton Jun 8, 2026

Conversation

JanuszL commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Category:

Description:

Additional information:

Affected modules and functionalities:

Key points relevant for the review:

Tests:

Checklist

Documentation

DALI team only

Requirements

Uh oh!

greptile-apps Bot commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

Uh oh!

JanuszL commented Apr 27, 2026

Uh oh!

dali-automaton commented Apr 27, 2026

Uh oh!

dali-automaton commented Apr 27, 2026

Uh oh!

dali-automaton commented Apr 27, 2026

Uh oh!

dali-automaton commented Apr 27, 2026

Uh oh!

dali-automaton commented Apr 28, 2026

Uh oh!

dali-automaton commented Apr 28, 2026

Uh oh!

dali-automaton commented Apr 28, 2026

Uh oh!

dali-automaton commented Apr 28, 2026

Uh oh!

dali-automaton commented May 29, 2026

Uh oh!

dali-automaton commented May 29, 2026

Uh oh!

dali-automaton commented May 29, 2026

Uh oh!

dali-automaton commented May 30, 2026

Uh oh!

jantonguirao Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

JanuszL Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

jantonguirao Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

JanuszL Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

dali-automaton commented Jun 8, 2026

Uh oh!

dali-automaton commented Jun 8, 2026

Uh oh!

dali-automaton commented Jun 8, 2026

Uh oh!

dali-automaton commented Jun 8, 2026

Uh oh!

dali-automaton commented Jun 9, 2026

Uh oh!

dali-automaton commented Jun 9, 2026

Uh oh!

dali-automaton commented Jun 9, 2026

Uh oh!

dali-automaton commented Jun 9, 2026

Uh oh!

dali-automaton commented Jun 10, 2026

Uh oh!

Reviewers

Assignees

JanuszL commented Apr 27, 2026 •

edited

Loading

greptile-apps Bot commented Apr 27, 2026 •

edited

Loading