Skip to content

Improve BLIS blog: technical rigor, structure, and diagrams#363

Merged
davidgs merged 10 commits into
mainfrom
codex/pr-344
Jul 1, 2026
Merged

Improve BLIS blog: technical rigor, structure, and diagrams#363
davidgs merged 10 commits into
mainfrom
codex/pr-344

Conversation

@chcost

@chcost chcost commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Editorial and structural improvements to the BLIS blog post (builds on #344):

  • Tone: Tightened prose to match the technical register of published llm-d blogs (KV-cache, Intelligent Scheduling)
  • Structure: Added :::tip key takeaways, :::info fidelity validation scope callout, numbered figure captions (FIGURE 1–5), BLIS-vs-cluster comparison table, admission control results table, and standard "Get Involved with llm-d" closing section
  • Diagrams: Replaced original PNG diagrams with purpose-built SVGs verified against the inference-sim codebase:
    • BLIS architecture (inputs → simulator internals → outputs)
    • AI-native inner/outer loop pipeline
    • Admission control latency overlay chart
  • Content: Added "What's next" roadmap, "Further reading" links, and "Limitations" section refinements

Builds on top of #344 — intended as a complementary PR with editorial improvements.

@netlify

netlify Bot commented Jun 15, 2026

Copy link
Copy Markdown

Deploy Preview for elaborate-kangaroo-25e1ee ready!

Name Link
🔨 Latest commit e0f1da5
🔍 Latest deploy log https://app.netlify.com/projects/elaborate-kangaroo-25e1ee/deploys/6a4537b38f03a100081b4e87
😎 Deploy Preview https://deploy-preview-363--elaborate-kangaroo-25e1ee.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions

Copy link
Copy Markdown

Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

toslali-ibm and others added 8 commits June 15, 2026 16:52
Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Align the BLIS blog post with the quality conventions of published
llm-d blogs. Tighten tone, add callout boxes, numbered figure captions,
results table, and standard closing sections. Replace original PNG
diagrams with purpose-built SVGs: BLIS architecture (verified against
inference-sim codebase), AI-native inner/outer loop pipeline, and
admission control latency overlay chart.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
@chcost

chcost commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

@ahg-g @robertgshaw2-redhat PTAL

@petecheslock petecheslock added the Blog Post This PR is a blog post label Jun 17, 2026

@mtoslalibu mtoslalibu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two suggestions:


1. Clarify the code-generation paragraph (line ~147)

The last sentence is hard to parse — "stabilizing the disaggregation fraction near 50%" doesn't clearly convey what's happening or why that matters. Suggested rewrite:

Code generation: 30K-token prefix, approximately 1,500 uncached tokens per turn, 15 turns per session. At N=16, the threshold fires for 100% of requests, routing all of them to the prefill pool. This can saturate prefill and inflate TTFT by up to 20× relative to always-local. EDPP avoids this: when disaggregated requests start missing the TTFT SLO, its feedback loop reduces the disaggregation rate — in this workload, settling near 50% — so the prefill pool stays below saturation and TTFT remains bounded.

The key change: make explicit what the feedback loop does (reduces disaggregation rate), why (TTFT SLO misses), and what the outcome is (prefill stays below saturation), rather than stating the steady-state number without context.


2. Fix "The physics of high-fidelity distributed inference platform simulation" link

The current link points to Medium. It should point to the BLIS project blog:

- [The physics of high-fidelity distributed inference platform simulation](https://medium.com/modeling-distributed-inference/the-physics-of-high-fidelity-distributed-inference-platform-simulation-28fe27b59da2)
+ [The physics of high-fidelity distributed inference platform simulation](https://inference-sim.github.io/inference-sim/latest/blog/2026/04/09/the-physics-of-high-fidelity-distributed-inference-platform-simulation/)

Comment thread blog/2026-06-05_blis-evolving-llm-d-at-simulation-speed.md Outdated
- **BLIS project:** [inference-sim.github.io/inference-sim](https://inference-sim.github.io/inference-sim/latest/)
- [Why simulate before you scale](https://inference-sim.github.io/inference-sim/latest/blog/2026/03/05/why-simulate-before-you-scale/)
- [The physics of high-fidelity distributed inference platform simulation](https://medium.com/modeling-distributed-inference/the-physics-of-high-fidelity-distributed-inference-platform-simulation-28fe27b59da2)
- [From simulation to production: the admission controller case study](https://ai-native-systems-research.github.io/ai-native-systems-research/blog/2026/05/13/from-simulation-to-production-how-an-ai-native-pipeline-discovered-a-better-admission-controller-for-llm-d/)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link should point to the BLIS project blog, not Medium:

Suggested change
- [From simulation to production: the admission controller case study](https://ai-native-systems-research.github.io/ai-native-systems-research/blog/2026/05/13/from-simulation-to-production-how-an-ai-native-pipeline-discovered-a-better-admission-controller-for-llm-d/)
- [The physics of high-fidelity distributed inference platform simulation](https://inference-sim.github.io/inference-sim/latest/blog/2026/04/09/the-physics-of-high-fidelity-distributed-inference-platform-simulation/)

@ahg-g

ahg-g commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Non blocking questions:

  1. I know that simulation makes it faster to iterate over a wide range of configurations. But for the configurations that turned out to be good candidates, did we run actual experiments to cross check and validate?
  2. How does this compare with Dynamo's AI Configurator?

chcost and others added 2 commits June 30, 2026 21:01
Co-authored-by: mtoslalibu <44147008+mtoslalibu@users.noreply.github.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Keep both sets of author entries: existing main authors and new BLIS blog authors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Carlos H. Andrade Costa <chcost@us.ibm.com>

@davidgs davidgs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@davidgs
davidgs merged commit 90fe9c4 into main Jul 1, 2026
11 checks passed
@davidgs
davidgs deleted the codex/pr-344 branch July 1, 2026 15:56
@davidgs
davidgs restored the codex/pr-344 branch July 1, 2026 16:31
chcost added a commit to kfirtoledo/llm-d.github.io that referenced this pull request Jul 1, 2026
…e authors.yml conflict

Keep both BLIS blog authors (from llm-d#363) and hybrid-models blog authors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Carlos H. Andrade Costa <chcost@us.ibm.com>
davidgs pushed a commit that referenced this pull request Jul 9, 2026
* Add blis blogpost

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Add limitations section

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Remove distributed-serving section, tighten intro tone

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Adopt technical tone throughout

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Add precise admission control results with config details

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Add fidelity validation numbers and speedup claim

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Expand PD disaggregation section with EDPP details and workload results

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Improve BLIS blog: technical rigor, structure, and diagrams

Align the BLIS blog post with the quality conventions of published
llm-d blogs. Tighten tone, add callout boxes, numbered figure captions,
results table, and standard closing sections. Replace original PNG
diagrams with purpose-built SVGs: BLIS architecture (verified against
inference-sim codebase), AI-native inner/outer loop pipeline, and
admission control latency overlay chart.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Apply suggestion from @mtoslalibu

Co-authored-by: mtoslalibu <44147008+mtoslalibu@users.noreply.github.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

---------

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Signed-off-by: Carlos H. Andrade Costa <chcost@us.ibm.com>
Co-authored-by: Mert Toslali <toslali@ibm.com>
Co-authored-by: Carlos H. Andrade Costa <chcost@Carloss-Mac-mini.local>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: mtoslalibu <44147008+mtoslalibu@users.noreply.github.com>
davidgs pushed a commit that referenced this pull request Jul 9, 2026
* Add blis blogpost

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Add limitations section

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Remove distributed-serving section, tighten intro tone

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Adopt technical tone throughout

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Add precise admission control results with config details

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Add fidelity validation numbers and speedup claim

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Expand PD disaggregation section with EDPP details and workload results

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Improve BLIS blog: technical rigor, structure, and diagrams

Align the BLIS blog post with the quality conventions of published
llm-d blogs. Tighten tone, add callout boxes, numbered figure captions,
results table, and standard closing sections. Replace original PNG
diagrams with purpose-built SVGs: BLIS architecture (verified against
inference-sim codebase), AI-native inner/outer loop pipeline, and
admission control latency overlay chart.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

* Apply suggestion from @mtoslalibu

Co-authored-by: mtoslalibu <44147008+mtoslalibu@users.noreply.github.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>

---------

Signed-off-by: Mert Toslali <toslali@ibm.com>
Signed-off-by: Carlos H. A. Costa <chcost@us.ibm.com>
Signed-off-by: Carlos H. Andrade Costa <chcost@us.ibm.com>
Co-authored-by: Mert Toslali <toslali@ibm.com>
Co-authored-by: Carlos H. Andrade Costa <chcost@Carloss-Mac-mini.local>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: mtoslalibu <44147008+mtoslalibu@users.noreply.github.com>
Signed-off-by: David G. Simmons <santafen@mac.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Blog Post This PR is a blog post

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants