vkshmem is an open source, vendor neutral, distributed shared memory runtime for GPUs.
vkshmem treats GPU accelerators, and their respective host CPUs, as unified computing
resources. vkshmem creates a parallel, heterogeneous, GPU and CPU machine from a set, or
cluster, of computers.
vkshmem allows GPUs to communicate with local and remote CPUs and GPUs over shared memory or a network with zero copy and CPU by-pass. The network becomes and extension of the memory system, providing the benefits and challenges that come with weak memory consistency.
OpenSHMEM provides a distributed shared memory communications mechansim built using a Partitioned Global Address Space (PGAS). OpenSHMEM imposes a symmetric property to the heap as part of it's PGAS specification.
Vulkan is an open standard for 3d graphics processing and computing. Vulkan allows users the ability to implement computing kernels (shaders) using either GLSL (OpenGL shading langauge) or HLSL (High-Level shader language). User provided computing kernels are compiled into SPIR-V. At runtime the SPIR-V byte code is passed to the GPU driver which performs the necessary translation steps into GPU machine code (Just In Time Compilation).
Vulkan provides low CPU overhead which is great for performance. In addition to lower CPU overhead, Vulkan grants lower level access to GPU hardware. Vulkan's lower level access to GPU resources includes mechanisms to directy engage with the GPU memory system.
The Vulkan External Memory API (specifically VK_KHR_external_memory and VK_KHR_external_memory_fd) allows VkDeviceMemory to be exported as a standard dma-buf (file descriptor). Vulkan device memory, exported as a dma-buf can be used to create a GPU memory mapped file. The GPU memory mapped file provides the CPU direct memory access (DMA) to GPU memory.
Additionally the dma-buf allows Vulkan device memory to be registered with other devices that have access to a DMA engine. vkshmem can register Vulkan device memory with RDMA NICs. Vulkan device memory registered with RDMA NICs provides vkshmem with zero-copy and CPU bypass memory operations across a network - think GPUDirect.
Vulkan combined with dma-buf allows GPU memory to be treated as a standard segment of a process's heap and as an extension of OpenSHMEM's symmetric heap. Vulkan allows sharing CPU host-pinned memory with both the GPU and compute kernels. This provides a shared memory mechansim to bridge loads, stores, and communications between the GPU and CPU.
vkshmem integrates Vulkan and OpenSHMEM enabling partitioned global address space communications as defined in the OpenSHMEM specification across these pairs of devices:
From\To | CPU | GPU | --------|-----------+ CPU | x | x | --------|-----------+ GPU | x | x | --------------------+
vkshmem interacting with NICs that have RDMA (PCIe P2P, RDMA, GPUDirect, RoCE, etc) are capable of performing zero-copy without host CPU intervention. This capability is enabled by memory registeration routines offered by ucx and libfabric.
In the situation where P2P or RDMA NICs are not available, the software falls back on a CPU intermediary for GPU network operations (Signal Orchestrate). vkshmem allows users to explicitly state the device to communicate with (CPU or GPU) or users can choose to perform memory allocations using a flat, symmetric, unfied address space, in which case, the vkshmem runtime acts as a virtual memory controller in user space and selects the appropriate device specific functions to perform the communication options.
vkshmem in multi-rail scenarios presents a unique architectural opportunity to provide exclusive and explicit networking capabilities for GPUs.
- Boost Version 1.0 (2022-)
Christopher Taylor, Hidden Layer, LLC (copyright 2026)
- The OpenSHMEM developers and committee
- The Khronos Group for Vulkan