Skip to content

refactor policies to per-process scheduling granularity - #30

Open
kylin1019 wants to merge 1 commit into
XpuOS:mainfrom
kylin1019:main
Open

refactor policies to per-process scheduling granularity#30
kylin1019 wants to merge 1 commit into
XpuOS:mainfrom
kylin1019:main

Conversation

@kylin1019

Copy link
Copy Markdown

Summary

Refactor UP, CFS, MLFQ, KEDF, and LAX policies from per-XQueue to per-process scheduling; add SwitchProcess() to suspend/resume all XQueues of a process as a unit; trigger immediate Sched()+ExecuteOperations() on policy switch in LocalScheduler::SetPolicy().

Motivation

LLM serving frameworks (llama.cpp, vLLM) create multiple CUDA streams per process. Per-XQueue policies can suspend one stream while another stream of the same process is running, causing a circular wait: the running stream blocks on an inter-stream dependency that requires the suspended stream to resume, but the scheduler waits for the running stream to idle before rescheduling. Per-process scheduling eliminates this deadlock by treating all XQueues of a process as an indivisible unit.

Scope

Only scheduler policy refactoring and the SetPolicy() fix. No new policies, hint types, HTTP API changes, tests, or docs.

Validation

  • Builds with -Wall -Wextra -Werror;
  • Tested with llama.cpp dual-instance serving: KEDF serializes correctly, CFS respects weights, UP enforces utilization direction;
  • No deadlock observed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant