[codex] Add macOS build and PR CI support#1930
Conversation
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Require kind labelWonderful, this rule succeeded.
🟢 Require version labelWonderful, this rule succeeded.
|
There was a problem hiding this comment.
Code Review
This pull request introduces support for macOS (arm64) and provides a unified dependency installation script. Build configurations in CMake and the Makefile were updated to handle Apple-specific dependencies, such as Homebrew's OpenMP and OpenBLAS. To ensure compatibility with ARM architecture, direct pointer casting in HNSW and quantization modules has been replaced with std::memcpy and unaligned access helpers. Additionally, the IO module was updated to support multiple parameter types, and documentation was expanded to cover the new platform support. Feedback focuses on addressing a layering violation in the IO path resolution logic.
There was a problem hiding this comment.
Pull request overview
Adds macOS arm64 parity for the core C++ build/test workflow and aligns CI/build tooling so macOS follows the same “install deps script + make targets + artifact reuse” pattern as Linux.
Changes:
- Introduces platform-aware dependency installation via
scripts/deps/install_deps.sh(Linux distro detection + macOS Homebrew path). - Adds macOS arm64 PR CI jobs (ASan build + artifact-based unit/functional tests + C++ examples) and a macOS daily test job.
- Fixes macOS-specific build/runtime issues (OpenMP wiring, alignment-safe loads/stores, optional dataset fields, compatibility asset handling) and updates docs accordingly.
Reviewed changes
Copilot reviewed 41 out of 41 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tools/eval/CMakeLists.txt | Prefer vsag_openmp target when available; fallback to -fopenmp. |
| tools/check_compatibility/check_compatibility.cpp | Make compatibility assets directory configurable; add file validation and better errors. |
| tests/test_index_old.cpp | Adjust FreshHNSW ef_search parameter for test stability. |
| tests/test_index.cpp | Guard optional dataset fields; improve concurrent-destruction test behavior. |
| tests/test_diskann.cpp | Guard optional sparse vector field access. |
| src/simd/normalize_test.cpp | Make float comparisons more stable across platforms/toolchains. |
| src/simd/CMakeLists.txt | Avoid Apple-specific OpenMP flags; link vsag_openmp when present. |
| src/quantization/scalar_quantization/sq8_uniform_quantizer.cpp | Avoid unaligned type-punning; handle diff_==0 safely. |
| src/quantization/scalar_quantization/sq4_uniform_quantizer.cpp | Avoid unaligned type-punning; handle diff_==0 safely. |
| src/quantization/quantizer_test.h | Use <cmath> and stabilize test value generation. |
| src/io/buffer_io.cpp | Support AsyncIOParameter for path extraction; handle zero-size direct reads. |
| src/impl/blas/CMakeLists.txt | Link vsag_openmp into BLAS object target when present. |
| src/dataset_impl_test.cpp | Ensure deep-copy checks cover std::string internals. |
| src/datacell/sparse_term_datacell.cpp | Harden term-list scanning against missing/empty term storage. |
| src/algorithm/hnswlib/hnswalg.h | Use memcpy-based accessors for list metadata/neighbors (alignment-safe). |
| src/algorithm/hnswlib/hnswalg.cpp | Switch neighbor reads/writes to new alignment-safe accessors. |
| src/CMakeLists.txt | Link vsag_openmp into main libraries when present. |
| scripts/download_compatibility_assets.sh | New: download + verify compatibility assets via gh. |
| scripts/deps/install_deps_ubuntu.sh | Add strict bash mode; improve arch handling and errors. |
| scripts/deps/install_deps_macos.sh | New: Homebrew-based macOS dependency installer. |
| scripts/deps/install_deps_centos.sh | Add strict bash mode. |
| scripts/deps/install_deps.sh | New: unified deps entrypoint (macOS + Linux distro detection + sudo handling). |
| scripts/check_compatibility.sh | Add strict bash mode; validate required inputs; pass COMPATIBILITY_INDEX_DIR. |
| extern/roaringbitmap/roaringbitmap.cmake | Disable UBSan for roaring on Clang/GNU to avoid platform-specific UB reports. |
| extern/openblas/openblas.cmake | Expand Homebrew search paths; adjust Apple/non-Apple BLAS linkage behavior. |
| extern/diskann/diskann.cmake | Avoid Apple-incompatible flags; link vsag_openmp when present. |
| docs/docs/zh/src/guide/installation.md | Update supported platforms and deps workflow (Linux + macOS arm64). |
| docs/docs/zh/src/development/building.md | Document unified deps script and validated macOS targets. |
| docs/docs/en/src/guide/installation.md | Document macOS support and unified deps flow. |
| docs/docs/en/src/development/building.md | Document macOS support, outputs, and unified deps flow. |
| cmake/VSAGDependencyTargets.cmake | Propagate vsag_openmp via shared interface target when present. |
| cmake/VSAGCompilerConfig.cmake | Treat AppleClang as Clang; adjust libc++/C++ standard selection logic. |
| cmake/DarwinDep.cmake | Introduce vsag_openmp interface target; prefer Homebrew libomp fallback. |
| Makefile | Darwin defaults (disable libaio, use system OpenBLAS); add test tag filtering knob. |
| DEVELOPMENT.md | Document macOS support + unified deps/install workflow. |
| CMakeLists.txt | Enable mockimpl on macOS when ENABLE_MOCKIMPL is set. |
| .github/workflows/python_build_and_test.yml | Use unified deps installer in Python CI jobs. |
| .github/workflows/pr-ci.yml | Add macOS arm64 PR CI build/test/example jobs; use unified deps installer; refactor compatibility asset download. |
| .github/workflows/daily_test.yml | Add macOS arm64 daily job; use unified deps installer. |
| .github/workflows/check_compatibility.yml | Use new compatibility asset download script. |
| .github/workflows/asan_build_and_test.yml | Use unified deps installer. |
49d2875 to
e6cb30d
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
extern/openblas/openblas.cmake:135
- In the non-system OpenBLAS path,
BLAS_LIBRARIESprependsomp/gompbased only on the compiler ID. On macOS this will add-lompwhen using AppleClang (because the ID matchesClang), but there’s no corresponding-L$(brew --prefix libomp)/lib, so links can fail. Since macOS OpenMP is handled via thevsag_openmptarget, consider skipping theomp/gomptokens on APPLE (or usingOpenMP::OpenMP_CXX/vsag_openmpinstead of raw library names).
if (APPLE AND DEFINED GFORTRAN_LIB AND EXISTS "${GFORTRAN_LIB}")
set (BLAS_LIBRARIES ${install_dir}/lib/libopenblas.a "${GFORTRAN_LIB}")
else ()
set (BLAS_LIBRARIES ${install_dir}/lib/libopenblas.a gfortran)
endif ()
if (CMAKE_CXX_COMPILER_ID MATCHES "Clang")
list (PREPEND BLAS_LIBRARIES omp)
else ()
list (PREPEND BLAS_LIBRARIES gomp)
endif ()
a25db48 to
10fea3f
Compare
10fea3f to
5c2b0e0
Compare
5c2b0e0 to
085ec31
Compare
085ec31 to
90fa6a8
Compare
Signed-off-by: JiangChao <jacllovey@qq.com>
90fa6a8 to
6359264
Compare
What
Add macOS arm64 development, build, and PR CI support while keeping the workflow aligned with the existing Linux/x86 flow.
Key changes:
scripts/deps/install_deps.shmaketargetsWhy
macOS previously only had partial developer workflow support. This reduces platform-specific mental overhead by making the macOS path follow the same high-level flow as Linux: install dependencies with a script, then use
maketargets for build and test.Validation
CMAKE_GENERATOR=Ninja make asan VSAG_ENABLE_EXAMPLES=ON COMPILE_JOBS=4ASAN_OPTIONS=detect_container_overflow=0 UBSAN_OPTIONS=halt_on_error=1 ./scripts/testing/test_parallel_by_name.sh unittestsNote:
ASAN_OPTIONS=detect_container_overflow=0is used only for macOS test execution because the enabled check reports a container-overflow in ANTLR/libc++ runtime code, not in VSAG business logic. ASan itself remains enabled.Out Of Scope