AMD GPU support: HIP/ROCm via single-source backend_gpu_compat.h (+ WMMA dispatch gate)#339
Open
noobdev-ph wants to merge 1 commit into
Open
AMD GPU support: HIP/ROCm via single-source backend_gpu_compat.h (+ WMMA dispatch gate)#339noobdev-ph wants to merge 1 commit into
noobdev-ph wants to merge 1 commit into
Conversation
β¦ckend_gpu_compat.h backend_cuda.cu compiles unchanged for both vendors: under nvcc the new header passes through to cuda_runtime.h + mma.h (preprocessed source is byte-identical to before this commit); under hipcc (make HIP=1) it maps the CUDA runtime surface onto HIP 1:1. Same one-shim-header pattern compat.h uses for the Windows port. The WMMA tensor-core dispatch is compile-gated via COLI_GPU_HAS_WMMA: gfx GPUs report compute_major >= 7 (gfx1201 reports 12), so the runtime check alone would select the __CUDA_ARCH__-guarded kernels' EMPTY bodies under HIP and return garbage silently. HIP uses the portable kernels; rocWMMA matrix-core support is a natural follow-up. Build: make HIP=1 [HIP_ARCH=native|gfxXXXX], make hip-test, make gpu-compile (CI compile-only target). HIP joins the .build-config stamp so HIP<->CPU flag flips relink correctly. CI: engine-hip-syntax job mirrors engine-cuda-syntax (rocm/dev container, pinned tag). Verified on RX 9070 XT (gfx1201, ROCm 7.2.4): hip-test passes the full upstream suite incl. tensor_update; GLM-5.2 end-to-end measures 0.32 tok/s at 61% expert hit with CUDA_RELEASE_HOST=1 (CPU: 0.22-0.27). Runtime details and hardware matrix: GPU_BACKENDS.md.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The AMD half of #112, rebuilt against
devas requested β one thing only: the existing CUDA backend compiles unchanged for AMD GPUs via ROCm (make HIP=1), through a 62-line mapping header, the same one-shim patterncompat.huses for the Windows port. 5 files, +218/β15;backend_cuda.cuitself changes by 7 lines;glm.cuntouched.backend_gpu_compat.h: under nvcc, a pass-through tocuda_runtime.h+mma.hβ the NVIDIA path's preprocessed source is byte-identical to today's, which your CI can verify. Under hipcc, the CUDA runtime surface maps to HIP 1:1.COLI_GPU_HAS_WMMA) β this part matters beyond AMD: the tensor-core kernel bodies are__CUDA_ARCH__ >= 700-guarded, but the host dispatch checkscompute_major >= 7at runtime. gfx GPUs reportcompute_major = 12, so without the compile-time gate the dispatch selects empty kernel bodies and silently returns garbage on any non-CUDA target. HIP uses the portable kernels; rocWMMA is a natural follow-up.HIP=1/HIP_ARCH,hip-test,gpu-compile(CI compile-only), aGPUCCindirection so all GPU rules stay single-path, and HIP flags joined to your new.build-configstamp so HIPβCPU flag flips relink correctly (Benchmark + Windows GPU-build fixes: GLM-5.2 on Core Ultra 9 285K + RTX 5080 (sm_120)Β #306's fix extended, not bypassed).engine-hip-syntaxjob mirroring yourengine-cuda-syntax(rocm/dev container β tag pinned and disk pre-freed;latestno longer fits hosted runners, details in the job comments).On "verified rather than trusted" β the honest position
Hosted CI proves compilation; runtime verification necessarily comes from community AMD hardware β the same epistemology as the README's benchmark table. What this PR offers concretely: (1) the CUDA path is provably unchanged (byte-identical preprocessing β verifiable by anyone); (2)
hip-testruns the full upstream test suite (incl.tensor_update) on my RX 9070 XT β pass, 5/5 repeat runs; (3) end-to-end: GLM-5.2 at 0.33 tok/s / 60% expert hit withCUDA_RELEASE_HOST=1on thisdevrebase (CPU: 0.22β0.27 same machine) β full setup documented inGPU_BACKENDS.mdso any AMD owner can reproduce; (4) I commit to runninghip-test+ an end-to-end check againstdevon request and on releases, and to first-responding AMD issues. If you'd rather see a second independent AMD datapoint before merging, keeping this open until one arrives is a reasonable call β the branch is maintained either way.Validation
make -C c checkβ CPU untouchedhip-teston gfx1201 (ROCm 7.2.4): full suite passengine-hip-syntaxruns in your CI structureCompatibility
HIP=1opt-in, Linux-only, mutually exclusive withCUDA=1, errors earlyCompanion PR: #338 (vendor-neutral backend hardening β independently mergeable).