Skip to content

Update SGLang to v0.5.17 and rebase Miles - #131

Merged
nanjiangwill merged 24 commits into
mainfrom
update/sglang-v0.5.17-validation
Aug 10, 2026
Merged

Update SGLang to v0.5.17 and rebase Miles#131
nanjiangwill merged 24 commits into
mainfrom
update/sglang-v0.5.17-validation

Conversation

@nanjiangwill

@nanjiangwill nanjiangwill commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • pin one unified SGLang v0.5.17 runtime for all cookbook models, including Kimi K3
  • pin the rebased Miles fully-async runtime and migrate the GLM-4.7/GLM-5.2 recipes to Session v2
  • retain disk, CPU/RAM, and CPU/NVMe weight-update paths with bounded staging lifetime and NUMA-interleaved canonical host memory
  • add trajectory affinity and load-aware routing for dynamic rollout fleets
  • validate FP8, ModelOpt NVFP4, mixed NVFP4/BF16, and Blackwell MXFP4 updates
  • propagate explicit client cancellations through SGLang's structured abort lifecycle

Validation

  • Stitch: 110 tests passed; Ruff lint and format checks passed
  • SGLang: pre-commit passed; focused weight-loading suite passed with 48 tests and 2 skips
  • all 12 model/update-path rows in the root README passed checksum and generation correctness checks
  • GLM-5.2 CPU/RAM on a 3.17 TB host: 116.4 s preparation, 3.30 s engine pause, exact post-update text, tokens, and logprobs
  • GLM-4.7 five-step E2E: four 8-GPU H200 trainer nodes, 48 one-GPU rollout replicas, logprob absolute difference 0.0172–0.0185, deltas v1–v5, and fleet catch-up through v5

SGLang is pinned at 0c79627e857eec795298a372adadf649209cdf2f on modal-projects/sglang:stitch-sglang-v0.5.17.

Miles is pinned at f83a68b296ca6f42d04e77e2b42c070c1ab02c70 on modal-projects/miles:stitch-miles-fully-async-swe.

@nanjiangwill nanjiangwill changed the title Track SGLang v0.5.17 migration Port weight staging to SGLang v0.5.17 Aug 9, 2026
@nanjiangwill
nanjiangwill marked this pull request as ready for review August 9, 2026 07:37
@nanjiangwill
nanjiangwill force-pushed the update/sglang-v0.5.17-validation branch from 38701dd to c48f550 Compare August 9, 2026 17:29
@nanjiangwill nanjiangwill changed the title Port weight staging to SGLang v0.5.17 Update SGLang to v0.5.17 and rebase Miles Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant