Skip to content

AMD GPU support: HIP/ROCm via single-source backend_gpu_compat.h (+ WMMA dispatch gate)#339

Open
noobdev-ph wants to merge 1 commit into
JustVugg:devfrom
noobdev-ph:feat/hip-single-source
Open

AMD GPU support: HIP/ROCm via single-source backend_gpu_compat.h (+ WMMA dispatch gate)#339
noobdev-ph wants to merge 1 commit into
JustVugg:devfrom
noobdev-ph:feat/hip-single-source

Conversation

@noobdev-ph

Copy link
Copy Markdown

Summary

The AMD half of #112, rebuilt against dev as requested β€” one thing only: the existing CUDA backend compiles unchanged for AMD GPUs via ROCm (make HIP=1), through a 62-line mapping header, the same one-shim pattern compat.h uses for the Windows port. 5 files, +218/βˆ’15; backend_cuda.cu itself changes by 7 lines; glm.c untouched.

  • backend_gpu_compat.h: under nvcc, a pass-through to cuda_runtime.h+mma.h β€” the NVIDIA path's preprocessed source is byte-identical to today's, which your CI can verify. Under hipcc, the CUDA runtime surface maps to HIP 1:1.
  • WMMA gating (COLI_GPU_HAS_WMMA) β€” this part matters beyond AMD: the tensor-core kernel bodies are __CUDA_ARCH__ >= 700-guarded, but the host dispatch checks compute_major >= 7 at runtime. gfx GPUs report compute_major = 12, so without the compile-time gate the dispatch selects empty kernel bodies and silently returns garbage on any non-CUDA target. HIP uses the portable kernels; rocWMMA is a natural follow-up.
  • Makefile: HIP=1/HIP_ARCH, hip-test, gpu-compile (CI compile-only), a GPUCC indirection so all GPU rules stay single-path, and HIP flags joined to your new .build-config stamp so HIP↔CPU flag flips relink correctly (Benchmark + Windows GPU-build fixes: GLM-5.2 on Core Ultra 9 285K + RTX 5080 (sm_120)Β #306's fix extended, not bypassed).
  • CI: an engine-hip-syntax job mirroring your engine-cuda-syntax (rocm/dev container β€” tag pinned and disk pre-freed; latest no longer fits hosted runners, details in the job comments).

On "verified rather than trusted" β€” the honest position

Hosted CI proves compilation; runtime verification necessarily comes from community AMD hardware β€” the same epistemology as the README's benchmark table. What this PR offers concretely: (1) the CUDA path is provably unchanged (byte-identical preprocessing β€” verifiable by anyone); (2) hip-test runs the full upstream test suite (incl. tensor_update) on my RX 9070 XT β€” pass, 5/5 repeat runs; (3) end-to-end: GLM-5.2 at 0.33 tok/s / 60% expert hit with CUDA_RELEASE_HOST=1 on this dev rebase (CPU: 0.22–0.27 same machine) β€” full setup documented in GPU_BACKENDS.md so any AMD owner can reproduce; (4) I commit to running hip-test + an end-to-end check against dev on request and on releases, and to first-responding AMD issues. If you'd rather see a second independent AMD datapoint before merging, keeping this open until one arrives is a reasonable call β€” the branch is maintained either way.

Validation

  • make -C c check β€” CPU untouched
  • hip-test on gfx1201 (ROCm 7.2.4): full suite pass
  • nvcc path: byte-identical preprocessed source (pass-through include)
  • engine-hip-syntax runs in your CI structure

Compatibility

  • Default CPU build dependency-free; HIP=1 opt-in, Linux-only, mutually exclusive with CUDA=1, errors early
  • No model files, binaries, or artifacts

Companion PR: #338 (vendor-neutral backend hardening β€” independently mergeable).

…ckend_gpu_compat.h

backend_cuda.cu compiles unchanged for both vendors: under nvcc the new
header passes through to cuda_runtime.h + mma.h (preprocessed source is
byte-identical to before this commit); under hipcc (make HIP=1) it maps
the CUDA runtime surface onto HIP 1:1. Same one-shim-header pattern
compat.h uses for the Windows port.

The WMMA tensor-core dispatch is compile-gated via COLI_GPU_HAS_WMMA:
gfx GPUs report compute_major >= 7 (gfx1201 reports 12), so the runtime
check alone would select the __CUDA_ARCH__-guarded kernels' EMPTY bodies
under HIP and return garbage silently. HIP uses the portable kernels;
rocWMMA matrix-core support is a natural follow-up.

Build: make HIP=1 [HIP_ARCH=native|gfxXXXX], make hip-test, make
gpu-compile (CI compile-only target). HIP joins the .build-config stamp
so HIP<->CPU flag flips relink correctly. CI: engine-hip-syntax job
mirrors engine-cuda-syntax (rocm/dev container, pinned tag).

Verified on RX 9070 XT (gfx1201, ROCm 7.2.4): hip-test passes the full
upstream suite incl. tensor_update; GLM-5.2 end-to-end measures 0.32
tok/s at 61% expert hit with CUDA_RELEASE_HOST=1 (CPU: 0.22-0.27).
Runtime details and hardware matrix: GPU_BACKENDS.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant