GH-49921: [C++] Bump xsimd to 14.2.0 by AntoinePrv · Pull Request #49922 · apache/arrow

AntoinePrv · 2026-05-05T13:09:35Z

Rationale for this change

Fix for the current SVE ODR violation in the latest version
Fix SVE dynamic dispatch ODR violation xtensor-stack/xsimd#1311
Pre-requisit to [C++] Use existing library for CPU detection #45331

What changes are included in this PR?

Bump version and cleanup backports.

Are these changes tested?

Yes, in CI.

Are there any user-facing changes?

No

GitHub Issue: [C++] Bump xsimd to 14.2.0 #49921

github-actions · 2026-05-05T13:10:25Z

⚠️ GitHub issue #49921 has been automatically assigned in GitHub to PR creator.

pitrou · 2026-05-06T09:11:20Z

@AntoinePrv Can you take a look at the CI failures? They seem related.

AntoinePrv · 2026-05-06T09:12:36Z

Yes, I am still waiting for the Msys2 build of xsimd to happen...

pitrou · 2026-05-06T09:14:37Z

There are failures on non-MSys2 builds, though:

AntoinePrv · 2026-05-06T09:30:33Z

Perhaps it's this check that does not work on MSVC?

https://github.com/xtensor-stack/xsimd/blob/80c23624ce008d937da7e845e528e82ce0cbf4e0/include/xsimd/types/xsimd_batch_constant.hpp#L500

Either we could add /Zc:__cplusplus or re-introduce the backport.

pitrou · 2026-05-06T09:49:52Z

I think we should both 1) fix xsimd upstream anyway and 2) pass /Zc:__cplusplus when building Arrow C++.

It seems xsimd should do something like this:

#ifdef _MSVC_LANG
#define XSIMD_CPP_LEVEL _MSVC_LANG
#else
#define XSIMD_CPP_LEVEL __cplusplus
#endif

and then use the XSIMD_CPP_LEVEL macro everywhere instead of __cplusplus.

(but xsimd could also use find-grained feature testing).

AntoinePrv · 2026-05-06T09:51:33Z

Already sending a patch, but not sure I want to go through another round of xsimd releases

xtensor-stack/xsimd#1342

pitrou · 2026-05-06T09:59:38Z

Agreed, hence "pass /Zc:__cplusplus when building Arrow C++" as well 😉

pitrou · 2026-05-06T10:41:57Z

Bloody protobuf fails building with /Zc:__cplusplus.
https://github.com/apache/arrow/actions/runs/25429128655/job/74590408553?pr=49922#step:9:2218

pitrou · 2026-05-06T11:24:58Z

Tiny syntax error that needs fixing:

  #17 27.36 Failed to load manifest from directory /arrow/ci/vcpkg
  #17 27.36 /arrow/ci/vcpkg/vcpkg.json:24:26: error: Trailing comma in an object
  #17 27.36   on expression:       "version>=": "14.2",

AntoinePrv · 2026-05-06T12:43:05Z

@pitrou I'm not very proactively working on this until the msys2 xsimd build is live

pitrou · 2026-05-06T14:19:52Z

@pitrou I'm not very proactively working on this until the msys2 xsimd build is live

@kou Do you know how we can make that happen?

pitrou · 2026-05-06T14:20:30Z

@github-actions crossbow submit -g cpp

github-actions · 2026-05-06T14:23:15Z

Revision: 89a1f7e

Submitted crossbow builds: ursacomputing/crossbow @ actions-9c434593fe

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

raulcd · 2026-05-06T14:41:25Z

Do you know how we can make that happen?

It's just a matter of waiting for their queue to update it:
https://packages.msys2.org/queue

Based on experience when uploading Arrow it takes between 6 and 12 hours. Their CI seems to run several times a day and xsimd is already listed on the queue so it should be available "soon".

kou · 2026-05-14T00:20:01Z

In general, already uploaded packages should not be rebuilt. They should be downloaded instead of built and uploaded.

AntoinePrv · 2026-05-19T08:20:41Z

seems now this function is not found:

arrow/cpp/src/arrow/flight/transport/grpc/util_internal.cc

Lines 334 to 335 in 8b19449

    
           #if GRPC_CPP_VERSION_CHECK(1, 80, 0) 
        
           Status FromAbslStatus(const ::absl::Status& absl_status) {

Setup Msys2 downloading 1.8.0 in the logs

 mingw-w64-ucrt-x86_64-grpc-1.80.0-1-any downloading...

But CMake finds otherwise

-- ARROW_GRPC_BUILD_VERSION: v1.76.0
-- ARROW_GRPC_BUILD_SHA256_CHECKSUM: 0af37b800953130b47c075b56683ee60bdc3eda3c37fc6004193f5b569758204

AntoinePrv · 2026-05-19T11:33:36Z

@pitrou @raulcd how does the CI look to you?

raulcd · 2026-05-19T15:53:20Z

@github-actions crossbow submit wheelcp314

github-actions · 2026-05-19T15:56:04Z

Revision: 2b5a233

Submitted crossbow builds: ursacomputing/crossbow @ actions-d5749ce93b

Task	Status
wheel-macos-monterey-cp314-cp314-amd64
wheel-macos-monterey-cp314-cp314-arm64
wheel-macos-monterey-cp314-cp314t-amd64
wheel-macos-monterey-cp314-cp314t-arm64
wheel-manylinux-2-28-cp314-cp314-amd64
wheel-manylinux-2-28-cp314-cp314-arm64
wheel-manylinux-2-28-cp314-cp314t-amd64
wheel-manylinux-2-28-cp314-cp314t-arm64
wheel-musllinux-1-2-cp314-cp314-amd64
wheel-musllinux-1-2-cp314-cp314-arm64
wheel-musllinux-1-2-cp314-cp314t-amd64
wheel-musllinux-1-2-cp314-cp314t-arm64
wheel-windows-cp314-cp314-amd64
wheel-windows-cp314-cp314t-amd64

raulcd · 2026-05-19T16:41:26Z

Some of the failures are unrelated (ODBC failures, debian-forky) but there's something I still don't understand and is the vcpkg nuget caching issues, example the JNI failures or the wheel failures. @kou how are those cached packages updated? Is the problem that the vcpkg baseline used doesn't seem to be a vcpkg release?
77826283b4676c526da3a5c59e6717e3775ba722
Do we have to wait until xsimd is on a vcpkg release (tag)? https://github.com/microsoft/vcpkg/releases

To be fair I don't understand how the cache for vcpkg works on those jobs we should document how are those updated.

kou

Some of the failures are unrelated (ODBC failures, debian-forky) but there's something I still don't understand and is the vcpkg nuget caching issues, example the JNI failures or the wheel failures. @kou how are those cached packages updated? Is the problem that the vcpkg baseline used doesn't seem to be a vcpkg release? 77826283b4676c526da3a5c59e6717e3775ba722 Do we have to wait until xsimd is on a vcpkg release (tag)? https://github.com/microsoft/vcpkg/releases

vcpkg cache with NuGet is done automatically. If there is not a cached package, vcpkg builds the package and push the built package to cache by NuGet. If there is a cached package, vcpkg just downloads and uses the cached package.

It's configured by:

arrow/ci/scripts/install_vcpkg.sh

Lines 60 to 89 in 603eeec

    
           if [ -n "${GITHUB_TOKEN:-}" ] && \ 
        
                [ -n "${GITHUB_REPOSITORY_OWNER:-}" ] && \ 
        
                [ "${VCPKG_BINARY_SOURCES:-}" = "clear;nuget,GitHub,readwrite" ] ; then 
        
             if type dnf 2>/dev/null; then 
        
               dnf install -y epel-release 
        
               dnf install -y mono-complete 
        
             fi 
        
             if type mono 2>/dev/null; then 
        
               curl \ 
        
                 --location \ 
        
                 --output "${vcpkg_destination}/nuget" \ 
        
                 https://dist.nuget.org/win-x86-commandline/latest/nuget.exe 
        
             fi 
        
             PATH="${vcpkg_destination}:${PATH}" 
        
             nuget_url="https://nuget.pkg.github.com/${GITHUB_REPOSITORY_OWNER}/index.json" 
        
             nuget="$(vcpkg fetch nuget | tail -n 1)" 
        
             if type mono 2>/dev/null; then 
        
               nuget="mono ${nuget}" 
        
             fi 
        
             ${nuget} \ 
        
               sources add \ 
        
               -source "${nuget_url}" \ 
        
               -storepasswordincleartext \ 
        
               -name "GitHub" \ 
        
               -username "${GITHUB_REPOSITORY_OWNER}" \ 
        
               -password "${GITHUB_TOKEN}" 
        
             ${nuget} \ 
        
               setapikey "${GITHUB_TOKEN}" \ 
        
               -source "${nuget_url}" 
        
           fi

JNI jobs: We use NuGet repository for the "apache" organization on PR. So we can't push built packages to cache on PR. (They are pushed when these jobs run on main.) vcpkg ignores push failures. So it doesn't fail CI jobs but CI jobs may take a long time because they need to build not cached packages.

Wheel jobs: It's failed because Apache Thrift build is failed:

https://github.com/ursacomputing/crossbow/actions/runs/26108848996/job/76780185321#step:8:2494

  #18 163.7 FAILED: compiler/cpp/thrift/thrifty.cc compiler/cpp/thrift/thrifty.hh /opt/vcpkg/buildtrees/thrift/amd64-linux-static-release-rel/compiler/cpp/thrift/thrifty.cc /opt/vcpkg/buildtrees/thrift/amd64-linux-static-release-rel/compiler/cpp/thrift/thrifty.hh 
  #18 163.7 cd /opt/vcpkg/buildtrees/thrift/amd64-linux-static-release-rel/compiler/cpp && /usr/bin/bison "--file-prefix-map=/opt/vcpkg/buildtrees/thrift/amd64-linux-static-release-rel=''" "--file-prefix-map=/opt/vcpkg/buildtrees/thrift/src/thrift-0-7fb4443e4a.clean=''" -d -o /opt/vcpkg/buildtrees/thrift/amd64-linux-static-release-rel/compiler/cpp/thrift/thrifty.cc /opt/vcpkg/buildtrees/thrift/src/thrift-0-7fb4443e4a.clean/compiler/cpp/src/thrift/thrifty.yy
  #18 163.7 /usr/bin/bison: unrecognized option '--file-prefix-map=/opt/vcpkg/buildtrees/thrift/amd64-linux-static-release-rel='''
  #18 163.7 Try '/usr/bin/bison --help' for more information.

It seems that Bison is old in the wheel image. It seems that we need Bison 3.7 or later https://lwn.net/Articles/826988/ for --file-prefix-map=.

To be fair I don't understand how the cache for vcpkg works on those jobs we should document how are those updated.

I agree with you. We should document it. Sorry for not writing it...

kou · 2026-05-20T01:18:32Z


 #pragma once

+#include <grpcpp/grpcpp.h>


Is this for #49991 ?

Looks like it was the same issue indeed. I can rebase instead.

AntoinePrv requested review from assignUser, jonkeane, kou and raulcd as code owners May 5, 2026 13:09

github-actions Bot added Component: C++ awaiting review Awaiting review labels May 5, 2026

AntoinePrv changed the title ~~GH-49921: [C++] Bunp xsimd to 14.2.0~~ GH-49921: [C++] Bump xsimd to 14.2.0 May 5, 2026

AntoinePrv force-pushed the main branch from 20b6061 to 474dc03 Compare May 6, 2026 07:21

AntoinePrv force-pushed the main branch from 7df3806 to 18d7351 Compare May 6, 2026 10:29

pitrou added the CI: Extra: C++ Run extra C++ CI label May 6, 2026

github-actions Bot removed the CI: Extra: C++ Run extra C++ CI label May 6, 2026

pitrou added the CI: Extra: C++ Run extra C++ CI label May 6, 2026

AntoinePrv mentioned this pull request May 6, 2026

GH-45331: Use xsimd for CPU feature detection #49940

Draft

github-actions Bot removed the CI: Extra: C++ Run extra C++ CI label May 6, 2026

pitrou added the CI: Extra: C++ Run extra C++ CI label May 7, 2026

AntoinePrv force-pushed the main branch from f30ce6e to f86060d Compare May 18, 2026 09:28

github-actions Bot removed the CI: Extra: C++ Run extra C++ CI label May 18, 2026

AntoinePrv force-pushed the main branch from b176da5 to 2b5a233 Compare May 19, 2026 09:11

AntoinePrv requested a review from lidavidm as a code owner May 19, 2026 09:11

github-actions Bot added the Component: FlightRPC label May 19, 2026

raulcd added CI: Extra: C++ Run extra C++ CI CI: Extra: Package: Linux Run extra Linux Packages CI CI: Extra: R Run extra R CI labels May 19, 2026

kou reviewed May 20, 2026

View reviewed changes

github-actions Bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels May 20, 2026

AntoinePrv added 7 commits May 20, 2026 09:56

Bump xsimd to 14.2.0

c96b4b9

Clean up xsimd bpacking

9182644

Back to using array_to_batch_constant

164ea1f

Fix vcpkg

2364204

Bump VCPKG

50bdff4

Adapt VCPKG port

ce917ba

Fully qualify xsimd version

4a8e74b

AntoinePrv force-pushed the main branch from 2b5a233 to 4a8e74b Compare May 20, 2026 07:56

github-actions Bot added awaiting change review Awaiting change review and removed Component: FlightRPC CI: Extra: C++ Run extra C++ CI CI: Extra: Package: Linux Run extra Linux Packages CI awaiting changes Awaiting changes labels May 20, 2026

	if [ -n "${GITHUB_TOKEN:-}" ] && \
	[ -n "${GITHUB_REPOSITORY_OWNER:-}" ] && \
	[ "${VCPKG_BINARY_SOURCES:-}" = "clear;nuget,GitHub,readwrite" ] ; then
	if type dnf 2>/dev/null; then
	dnf install -y epel-release
	dnf install -y mono-complete
	fi
	if type mono 2>/dev/null; then
	curl \
	--location \
	--output "${vcpkg_destination}/nuget" \
	https://dist.nuget.org/win-x86-commandline/latest/nuget.exe
	fi
	PATH="${vcpkg_destination}:${PATH}"
	nuget_url="https://nuget.pkg.github.com/${GITHUB_REPOSITORY_OWNER}/index.json"
	nuget="$(vcpkg fetch nuget \| tail -n 1)"
	if type mono 2>/dev/null; then
	nuget="mono ${nuget}"
	fi
	${nuget} \
	sources add \
	-source "${nuget_url}" \
	-storepasswordincleartext \
	-name "GitHub" \
	-username "${GITHUB_REPOSITORY_OWNER}" \
	-password "${GITHUB_TOKEN}"
	${nuget} \
	setapikey "${GITHUB_TOKEN}" \
	-source "${nuget_url}"
	fi

Conversation

AntoinePrv commented May 5, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions Bot commented May 5, 2026

Uh oh!

pitrou commented May 6, 2026

Uh oh!

AntoinePrv commented May 6, 2026

Uh oh!

pitrou commented May 6, 2026

Uh oh!

AntoinePrv commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pitrou commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AntoinePrv commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pitrou commented May 6, 2026

Uh oh!

pitrou commented May 6, 2026

Uh oh!

pitrou commented May 6, 2026

Uh oh!

AntoinePrv commented May 6, 2026

Uh oh!

pitrou commented May 6, 2026

Uh oh!

pitrou commented May 6, 2026

Uh oh!

github-actions Bot commented May 6, 2026

Uh oh!

raulcd commented May 6, 2026

Uh oh!

kou commented May 14, 2026

Uh oh!

AntoinePrv commented May 19, 2026

Uh oh!

AntoinePrv commented May 19, 2026

Uh oh!

raulcd commented May 19, 2026

Uh oh!

github-actions Bot commented May 19, 2026

Uh oh!

raulcd commented May 19, 2026

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

kou May 20, 2026

Choose a reason for hiding this comment

Uh oh!

AntoinePrv May 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AntoinePrv commented May 5, 2026 •

edited by github-actions Bot

Loading

AntoinePrv commented May 6, 2026 •

edited

Loading

pitrou commented May 6, 2026 •

edited

Loading

AntoinePrv commented May 6, 2026 •

edited

Loading