Skip to content

[codex] Add macOS build and PR CI support#1930

Draft
jac0626 wants to merge 1 commit into
antgroup:mainfrom
jac0626:codex/macos-build-baseline
Draft

[codex] Add macOS build and PR CI support#1930
jac0626 wants to merge 1 commit into
antgroup:mainfrom
jac0626:codex/macos-build-baseline

Conversation

@jac0626
Copy link
Copy Markdown
Collaborator

@jac0626 jac0626 commented Apr 25, 2026

What

Add macOS arm64 development, build, and PR CI support while keeping the workflow aligned with the existing Linux/x86 flow.

Key changes:

  • add platform-aware dependency installation through scripts/deps/install_deps.sh
  • support macOS local builds through the existing make targets
  • add macOS PR CI jobs for ASan build, release build, unit tests, functional tests, and C++ examples
  • make macOS test jobs consume the ASan build artifact, matching the x86 artifact flow
  • fix macOS-specific build/test issues found while enabling the CI path
  • document macOS build prerequisites and workflow

Why

macOS previously only had partial developer workflow support. This reduces platform-specific mental overhead by making the macOS path follow the same high-level flow as Linux: install dependencies with a script, then use make targets for build and test.

Validation

  • Local macOS ASan build: CMAKE_GENERATOR=Ninja make asan VSAG_ENABLE_EXAMPLES=ON COMPILE_JOBS=4
  • Local macOS unit tests: ASAN_OPTIONS=detect_container_overflow=0 UBSAN_OPTIONS=halt_on_error=1 ./scripts/testing/test_parallel_by_name.sh unittests
  • PR CI passed on fork: https://github.com/jac0626/vsag/actions/runs/24882026779

Note: ASAN_OPTIONS=detect_container_overflow=0 is used only for macOS test execution because the enabled check reports a container-overflow in ANTLR/libc++ runtime code, not in VSAG business logic. ASan itself remains enabled.

Out Of Scope

  • macOS Python wheels
  • lint cleanup, which is handled separately

Copilot AI review requested due to automatic review settings April 25, 2026 05:11
@jac0626 jac0626 added kind/improvement Code improvements (variable/function renaming, refactoring, etc. ) version/1.0 labels Apr 25, 2026
@jac0626 jac0626 added the created-by-AI The issue is found and created by AI Agent label Apr 25, 2026
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Apr 25, 2026

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Require kind label

Wonderful, this rule succeeded.
  • label~=^kind/

🟢 Require version label

Wonderful, this rule succeeded.
  • label~=^version/

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for macOS (arm64) and provides a unified dependency installation script. Build configurations in CMake and the Makefile were updated to handle Apple-specific dependencies, such as Homebrew's OpenMP and OpenBLAS. To ensure compatibility with ARM architecture, direct pointer casting in HNSW and quantization modules has been replaced with std::memcpy and unaligned access helpers. Additionally, the IO module was updated to support multiple parameter types, and documentation was expanded to cover the new platform support. Feedback focuses on addressing a layering violation in the IO path resolution logic.

Comment thread src/io/buffer_io.cpp Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds macOS arm64 parity for the core C++ build/test workflow and aligns CI/build tooling so macOS follows the same “install deps script + make targets + artifact reuse” pattern as Linux.

Changes:

  • Introduces platform-aware dependency installation via scripts/deps/install_deps.sh (Linux distro detection + macOS Homebrew path).
  • Adds macOS arm64 PR CI jobs (ASan build + artifact-based unit/functional tests + C++ examples) and a macOS daily test job.
  • Fixes macOS-specific build/runtime issues (OpenMP wiring, alignment-safe loads/stores, optional dataset fields, compatibility asset handling) and updates docs accordingly.

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tools/eval/CMakeLists.txt Prefer vsag_openmp target when available; fallback to -fopenmp.
tools/check_compatibility/check_compatibility.cpp Make compatibility assets directory configurable; add file validation and better errors.
tests/test_index_old.cpp Adjust FreshHNSW ef_search parameter for test stability.
tests/test_index.cpp Guard optional dataset fields; improve concurrent-destruction test behavior.
tests/test_diskann.cpp Guard optional sparse vector field access.
src/simd/normalize_test.cpp Make float comparisons more stable across platforms/toolchains.
src/simd/CMakeLists.txt Avoid Apple-specific OpenMP flags; link vsag_openmp when present.
src/quantization/scalar_quantization/sq8_uniform_quantizer.cpp Avoid unaligned type-punning; handle diff_==0 safely.
src/quantization/scalar_quantization/sq4_uniform_quantizer.cpp Avoid unaligned type-punning; handle diff_==0 safely.
src/quantization/quantizer_test.h Use <cmath> and stabilize test value generation.
src/io/buffer_io.cpp Support AsyncIOParameter for path extraction; handle zero-size direct reads.
src/impl/blas/CMakeLists.txt Link vsag_openmp into BLAS object target when present.
src/dataset_impl_test.cpp Ensure deep-copy checks cover std::string internals.
src/datacell/sparse_term_datacell.cpp Harden term-list scanning against missing/empty term storage.
src/algorithm/hnswlib/hnswalg.h Use memcpy-based accessors for list metadata/neighbors (alignment-safe).
src/algorithm/hnswlib/hnswalg.cpp Switch neighbor reads/writes to new alignment-safe accessors.
src/CMakeLists.txt Link vsag_openmp into main libraries when present.
scripts/download_compatibility_assets.sh New: download + verify compatibility assets via gh.
scripts/deps/install_deps_ubuntu.sh Add strict bash mode; improve arch handling and errors.
scripts/deps/install_deps_macos.sh New: Homebrew-based macOS dependency installer.
scripts/deps/install_deps_centos.sh Add strict bash mode.
scripts/deps/install_deps.sh New: unified deps entrypoint (macOS + Linux distro detection + sudo handling).
scripts/check_compatibility.sh Add strict bash mode; validate required inputs; pass COMPATIBILITY_INDEX_DIR.
extern/roaringbitmap/roaringbitmap.cmake Disable UBSan for roaring on Clang/GNU to avoid platform-specific UB reports.
extern/openblas/openblas.cmake Expand Homebrew search paths; adjust Apple/non-Apple BLAS linkage behavior.
extern/diskann/diskann.cmake Avoid Apple-incompatible flags; link vsag_openmp when present.
docs/docs/zh/src/guide/installation.md Update supported platforms and deps workflow (Linux + macOS arm64).
docs/docs/zh/src/development/building.md Document unified deps script and validated macOS targets.
docs/docs/en/src/guide/installation.md Document macOS support and unified deps flow.
docs/docs/en/src/development/building.md Document macOS support, outputs, and unified deps flow.
cmake/VSAGDependencyTargets.cmake Propagate vsag_openmp via shared interface target when present.
cmake/VSAGCompilerConfig.cmake Treat AppleClang as Clang; adjust libc++/C++ standard selection logic.
cmake/DarwinDep.cmake Introduce vsag_openmp interface target; prefer Homebrew libomp fallback.
Makefile Darwin defaults (disable libaio, use system OpenBLAS); add test tag filtering knob.
DEVELOPMENT.md Document macOS support + unified deps/install workflow.
CMakeLists.txt Enable mockimpl on macOS when ENABLE_MOCKIMPL is set.
.github/workflows/python_build_and_test.yml Use unified deps installer in Python CI jobs.
.github/workflows/pr-ci.yml Add macOS arm64 PR CI build/test/example jobs; use unified deps installer; refactor compatibility asset download.
.github/workflows/daily_test.yml Add macOS arm64 daily job; use unified deps installer.
.github/workflows/check_compatibility.yml Use new compatibility asset download script.
.github/workflows/asan_build_and_test.yml Use unified deps installer.

Comment thread tools/check_compatibility/check_compatibility.cpp
@jac0626 jac0626 force-pushed the codex/macos-build-baseline branch from 49d2875 to e6cb30d Compare April 29, 2026 03:02
Copilot AI review requested due to automatic review settings April 29, 2026 04:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.

Comment thread src/io/buffer_io.cpp Outdated
Comment thread src/quantization/scalar_quantization/sq8_uniform_quantizer.cpp Outdated
Comment thread src/quantization/scalar_quantization/sq4_uniform_quantizer.cpp Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

extern/openblas/openblas.cmake:135

  • In the non-system OpenBLAS path, BLAS_LIBRARIES prepends omp/gomp based only on the compiler ID. On macOS this will add -lomp when using AppleClang (because the ID matches Clang), but there’s no corresponding -L$(brew --prefix libomp)/lib, so links can fail. Since macOS OpenMP is handled via the vsag_openmp target, consider skipping the omp/gomp tokens on APPLE (or using OpenMP::OpenMP_CXX/vsag_openmp instead of raw library names).
    if (APPLE AND DEFINED GFORTRAN_LIB AND EXISTS "${GFORTRAN_LIB}")
        set (BLAS_LIBRARIES ${install_dir}/lib/libopenblas.a "${GFORTRAN_LIB}")
    else ()
        set (BLAS_LIBRARIES ${install_dir}/lib/libopenblas.a gfortran)
    endif ()
    if (CMAKE_CXX_COMPILER_ID MATCHES "Clang")
        list (PREPEND BLAS_LIBRARIES omp)
    else ()
        list (PREPEND BLAS_LIBRARIES gomp)
    endif ()

Comment thread cmake/DarwinDep.cmake
Comment thread cmake/VSAGDependencyTargets.cmake Outdated
@jac0626 jac0626 force-pushed the codex/macos-build-baseline branch from a25db48 to 10fea3f Compare April 29, 2026 05:16
Copilot AI review requested due to automatic review settings April 29, 2026 05:42
@jac0626 jac0626 force-pushed the codex/macos-build-baseline branch from 10fea3f to 5c2b0e0 Compare April 29, 2026 05:42
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Comment thread src/io/CMakeLists.txt Outdated
@jac0626 jac0626 force-pushed the codex/macos-build-baseline branch from 5c2b0e0 to 085ec31 Compare April 29, 2026 06:19
Copilot AI review requested due to automatic review settings April 29, 2026 06:42
@jac0626 jac0626 force-pushed the codex/macos-build-baseline branch from 085ec31 to 90fa6a8 Compare April 29, 2026 06:42
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

Signed-off-by: JiangChao <jacllovey@qq.com>
@jac0626 jac0626 force-pushed the codex/macos-build-baseline branch from 90fa6a8 to 6359264 Compare May 19, 2026 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

created-by-AI The issue is found and created by AI Agent kind/improvement Code improvements (variable/function renaming, refactoring, etc. ) module/docs module/simd module/testing module/thirdparty module/tools size/L version/1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants