Skip to content

fix(omp): park idle workers under PASSIVE policy (was spinning all cores at idle) #341#349

Open
woolcoxm wants to merge 2 commits into
JustVugg:devfrom
woolcoxm:fix/omp-idle-spin-341
Open

fix(omp): park idle workers under PASSIVE policy (was spinning all cores at idle) #341#349
woolcoxm wants to merge 2 commits into
JustVugg:devfrom
woolcoxm:fix/omp-idle-spin-341

Conversation

@woolcoxm

Copy link
Copy Markdown
Contributor

Summary

Fixes #341 β€” `glm` stays at ~3000% CPU after the model finishes its answer.

Root cause

The hot-thread tuning (added in #77) seeds `OMP_WAIT_POLICY=active` so libgomp's worker team stays hot across the back-to-back per-expert matmul regions. But `active` keeps the team spinning in userspace forever between regions: once a forward pass spins the team up, every worker core keeps busy-waiting even while the engine sits blocked in `getline()` waiting for the next prompt. On a many-core host that is the sustained ~3000% CPU the reporter sees at idle.

The original comment claimed `GOMP_SPINCOUNT=200000` would "spin briefly, then yield" β€” but that only holds under the passive policy. Under `active`, the spincount does not actually park the threads during long idle gaps.

This was independently corroborated by `docs/METAL-M5MAX-PERF-REPORT.md`, which already diagnosed the active-spin as harmful on Apple Silicon and worked around it with `COLI_NO_OMP_TUNE`. This PR generalizes that fix to the CPU path.

The fix

Switch the wait policy to `passive` while keeping `GOMP_SPINCOUNT=200000`:

Changes

  • `c/glm.c` β€” `active` β†’ `passive` in the self-reexec tuning block, with corrected rationale comments
  • `c/coli` β€” same fix in the win32 path of `env_for`
  • `c/tests/test_env_defaults.py` β€” assertion updated `active` β†’ `passive`
  • `docs/ENVIRONMENT.md` β€” updated the `COLI_NO_OMP_TUNE` description

Metal/CUDA builds were already excluded from the tuning block (`!getenv("COLI_METAL") && !getenv("COLI_CUDA")`), so they are unaffected.

Verification

  • `glm.c` compiles clean (no new warnings).
  • All 11 buildable C tests pass; `test_env_defaults` (4 tests) passes with the updated assertion.
  • `test_stops` fails to build on Windows, but I confirmed it fails identically on pristine `origin/dev` β€” it is a pre-existing `mkdtemp` portability issue unrelated to this change.

I could not reproduce the 3000% figure locally (my CPU does not exhibit the same spin behavior), so the reporter's confirmation on their FreeBSD host would be the definitive test.

Closes #341.

…res at idle) (JustVugg#341)

The hot-thread tuning seeded OMP_WAIT_POLICY=active so libgomp's worker
team stays hot across the back-to-back per-expert matmul regions. But
 keeps the team spinning in userspace FOREVER between regions:
once a forward pass spins the team up, the engine burns ~100% per core
while it sits blocked in getline() waiting for the next prompt. On a
many-core host that is the '3000% CPU after the model finished the
answer' from JustVugg#341.

Switch to PASSIVE. GOMP_SPINCOUNT=200000 still keeps the team hot across
the microsecond gaps between adjacent matmul regions (preserving the
66.9s -> 20.9s win from JustVugg#77), but idle workers now park on a futex once
the spin window lapses, so between-turn idle drops to 0%.

Same fix applied to the win32 path in the coli launcher (env_for), plus
the env-defaults test and the ENVIRONMENT.md doc.

Metal/CUDA builds were already excluded from this block, so they are
unaffected. test_stops still fails to build on Windows, but that is a
pre-existing mkdtemp portability issue unrelated to this change (fails
identically on pristine origin/dev).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant