fix(omp): park idle workers under PASSIVE policy (was spinning all cores at idle) #341#349
Open
woolcoxm wants to merge 2 commits into
Open
fix(omp): park idle workers under PASSIVE policy (was spinning all cores at idle) #341#349woolcoxm wants to merge 2 commits into
woolcoxm wants to merge 2 commits into
Conversation
β¦res at idle) (JustVugg#341) The hot-thread tuning seeded OMP_WAIT_POLICY=active so libgomp's worker team stays hot across the back-to-back per-expert matmul regions. But keeps the team spinning in userspace FOREVER between regions: once a forward pass spins the team up, the engine burns ~100% per core while it sits blocked in getline() waiting for the next prompt. On a many-core host that is the '3000% CPU after the model finished the answer' from JustVugg#341. Switch to PASSIVE. GOMP_SPINCOUNT=200000 still keeps the team hot across the microsecond gaps between adjacent matmul regions (preserving the 66.9s -> 20.9s win from JustVugg#77), but idle workers now park on a futex once the spin window lapses, so between-turn idle drops to 0%. Same fix applied to the win32 path in the coli launcher (env_for), plus the env-defaults test and the ENVIRONMENT.md doc. Metal/CUDA builds were already excluded from this block, so they are unaffected. test_stops still fails to build on Windows, but that is a pre-existing mkdtemp portability issue unrelated to this change (fails identically on pristine origin/dev).
# Conflicts: # c/glm.c
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #341 β `glm` stays at ~3000% CPU after the model finishes its answer.
Root cause
The hot-thread tuning (added in #77) seeds `OMP_WAIT_POLICY=active` so libgomp's worker team stays hot across the back-to-back per-expert matmul regions. But `active` keeps the team spinning in userspace forever between regions: once a forward pass spins the team up, every worker core keeps busy-waiting even while the engine sits blocked in `getline()` waiting for the next prompt. On a many-core host that is the sustained ~3000% CPU the reporter sees at idle.
The original comment claimed `GOMP_SPINCOUNT=200000` would "spin briefly, then yield" β but that only holds under the passive policy. Under `active`, the spincount does not actually park the threads during long idle gaps.
This was independently corroborated by `docs/METAL-M5MAX-PERF-REPORT.md`, which already diagnosed the active-spin as harmful on Apple Silicon and worked around it with `COLI_NO_OMP_TUNE`. This PR generalizes that fix to the CPU path.
The fix
Switch the wait policy to `passive` while keeping `GOMP_SPINCOUNT=200000`:
Changes
Metal/CUDA builds were already excluded from the tuning block (`!getenv("COLI_METAL") && !getenv("COLI_CUDA")`), so they are unaffected.
Verification
I could not reproduce the 3000% figure locally (my CPU does not exhibit the same spin behavior), so the reporter's confirmation on their FreeBSD host would be the definitive test.
Closes #341.