Skip to content

int2/int4 per-token per-head KV cache quantization - #9

Open
JartX wants to merge 564 commits into
feature/int4_per_tokenfrom
feature/int2_int4_per_token_head
Open

int2/int4 per-token per-head KV cache quantization#9
JartX wants to merge 564 commits into
feature/int4_per_tokenfrom
feature/int2_int4_per_token_head

Conversation

@JartX

@JartX JartX commented Apr 6, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Owner

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

qiching and others added 30 commits April 14, 2026 14:43
@qiching Albert Cheng (Engrg-Hardware 1)
@mgoin @gemini-code-assist
add warning when FP8 KV cache misses prefill query quantization (vllm…
dc8df11 
…-project#39752)

Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Albert Cheng (Engrg-Hardware 1) <albecheng@login-lyris02.lyris.clusters.nvidia.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@danielafrimi root
[Kernel] Support TRTLLM GEN NVFP4 MoE for non-512-aligned hidden dims…
240f263 
… via weight padding (vllm-project#39510)

Signed-off-by: root <root@lyris0017.lyris.clusters.nvidia.com>
Signed-off-by: Daniel Afrimi <dafrimi@nvidia.com>
Co-authored-by: root <root@lyris0017.lyris.clusters.nvidia.com>
@ZhanqiuHu
[CI][KVConnector][Metrics] Update multi KV connector edge case accord…
3067931 
…ing to prefill stats changes (vllm-project#39808)

Signed-off-by: Zhanqiu Hu <zhu@redhat.com>
@roikoren755
[Mamba] Flashinfer selective_state_update (vllm-project#36162)
ecf5ff7 
Signed-off-by: Roi Koren <roik@nvidia.com>
@Jackmin801 @robertgshaw2-redhat
[MoE] Move GPT OSS Triton kernel experts into fused_moe/experts/ (vll…
1a9353b 
…m-project#39007)

Signed-off-by: Robert Shaw <robertgshaw2@gmail.com>
Signed-off-by: Jackmin801 <ongjackm@gmail.com>
Co-authored-by: Robert Shaw <robertgshaw2@gmail.com>
@bnellnm
[MoE Refactor] Refactor ZeroExpertFusedMoE into new framework (vllm-p…
19ec9a0 
…roject#35549)

Signed-off-by: Bill Nell <bnell@redhat.com>
@rishitdholakia13 @mergify @aarnphm
[Reasoning][Frontend] Add model config to adjust_request in reasoning…
70ed015 
… parser (vllm-project#37848)

Signed-off-by: rishitdholakia13 <rishit+github@cohere.com>
Signed-off-by: rishitdholakia13 <123388671+rishitdholakia13@users.noreply.github.com>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
@maobaolong
fix(lmcache): correct store for cached requests while enable prefix c…
b2f749d 
…ache (vllm-project#39719)

Signed-off-by: baoloongmao <baoloongmao@tencent.com>
@yewentao256
[Bug] Fix batch invariance nvfp4 support (vllm-project#39820)
2ad1029 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
@zhewenl Zhewen Li
[Bugfix][Mooncake] Fix thread-local CUDA context for NVLink transfers…
1696c86 
… in _send_blocks (vllm-project#39548)

Signed-off-by: Zhewen Li <zhewenli@inferact.ai>
Co-authored-by: Zhewen Li <zhewenli@inferact.ai>
@fuscof-ibm
[Hybrid] Simplify accepted token counting in spec decode for hybrid m…
507df79 
…odels (vllm-project#38372)
@mgoin @claude
[Bugfix] Disable FlashInfer CUTLASS MoE on SM121 (DGX Spark) (vllm-pr…
65b9808 
…oject#39825)

Signed-off-by: mgoin <mgoin64@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
@atalman
[CI] Add PyTorch nightly build and test pipeline (vllm-project#37226)
b569620 
Signed-off-by: atalman <atalman@fb.com>
@TheEpicDolphin
[Model Runner V2] Disable piecewise cudagraph mode fallback for eagle…
3bfe55a 
… draft decodes (vllm-project#39773)

Signed-off-by: Giancarlo Delfin <gdelfin@inferact.ai>
@vibhavagarwal5 @xinyu-intel @mgoin
[Attention Backend] TurboQuant: 2-bit KV cache compression with 4x ca…
f4b42df 
…pacity (vllm-project#38479)

Signed-off-by: vibhavagarwal5 <vibhavagarwal5@gmail.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
@wincent8
[Test] Refactor hard coded device string in test files under compile/…
3abf858 
…quantization/models/model_executor folders (vllm-project#38901)

Signed-off-by: Liao, Wei <wei.liao@intel.com>
@he-yufeng @Isotr0py
[Bugfix] Respect VLLM_WEIGHT_OFFLOADING_DISABLE_PIN_MEMORY in prefetc…
bcc2306 
…h offloader (vllm-project#37699)

Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@ZhanqiuHu
[CI][NIXL] Fix PD CI breakage: pin nixl-cu{12,13} versions (vllm-proj…
799973a 
…ect#39851)

Signed-off-by: ZhanqiuHu <zhu@redhat.com>
@JartX
refactor Thomas
f51ccb3 
Signed-off-by: JartX <sagformas@epdcenter.es>
@wojciech-wais @mergify
@mgoin @xinyu-intel @ekagra-ranjan @puririshi98 @zzaebok @DarkLight1337 @Oasis-Git yuwei @arpera @ibrahim1023 @ChuanLi1101 @chaunceyjiang @jikunshang @ganeshr10 @LucasWilkinson @robertgshaw2-redhat @lkm2835 @ronensc @WorldExplored vnadathur vnadathur @Isotr0py @Elm8116 @yma11 @njhill @jackcfwang @xuechendi @injaeryou @zou3519 @milesial @elvircrn @claude @whx-sjtu @amd-lalithnc @gemini-code-assist @ProExpertProg @PatchouliTIS PatchouliTaisa @jatseng-ai @mgehre-amd @xaguilar-amd @dondetir @NickLucche @petern48 @noooop @izhuhaoran @jeejeelee @tjtanaa @jefp @efortin @ZhanqiuHu @yzong-rh @fynnsu @rahul-tuli @hmellor @benchislett @gty111 @elwhyjay @z1ying @bigPYJ1151 @vibhavagarwal5 vibhav-agarwal @ShubyM @wzhao18 @Etelis @EtelisIBM @EdalatiAli @AndreasKaratzas @r266-tech @ywang96 @hickeyma @orozery @markmc @Young-Leo @anijain2305 @zyongye @zhxchen17 @EricccYang @maobaolong @1643661061leo @sfeng33 @he-yufeng @zufangzhu @TihoElek @yiliu30 @yewentao256 @santiramos27 @haosdent @JartX @Yuyi-Ao @tlrmchlsmth @mukesh-hai @pedramr
[Bugfix] Fix tool_calls Iterable consumed when debug logging is enabl…
431cea3 
…ed (vllm-project#34844)

Signed-off-by: Wojciech Wais <wojciech.wais@gmail.com>
Signed-off-by: mgoin <mgoin64@gmail.com>
Signed-off-by: Xinyu Chen <xinyu1.chen@intel.com>
Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Signed-off-by: Rishi Puri <riship@nvidia.com>
Signed-off-by: Jaebok Lee <jaebok9541@naver.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Signed-off-by: yuwei <yuwei@dev.local>
Signed-off-by: Artem Perevedentsev <aperevedents@nvidia.com>
Signed-off-by: Ibrahim Arshad <38925737+ibrahim1023@users.noreply.github.com>
Signed-off-by: Li <chuali@amd.com>
Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
Signed-off-by: Kunshang Ji <kunshang.ji@intel.com>
Signed-off-by: Kunshang Ji <jikunshang95@gmail.com>
Signed-off-by: R <Ganesh.R@amd.com>
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
Signed-off-by: lkm2835 <lkm2835@gmail.com>
Signed-off-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Signed-off-by: vnadathur <glvikramn@gmail.com>
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: Elham Harirpoush <elham.harirpoush@arm.com>
Signed-off-by: Yan Ma <yan.ma@intel.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: jackcfwang <jackcfwang@tencent.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Injae Ryou <injaeryou@gmail.com>
Signed-off-by: Richard Zou <zou3519@gmail.com>
Signed-off-by: milesial <milesial@users.noreply.github.com>
Signed-off-by: Elvir Crncevic <elvircrn@gmail.com>
Signed-off-by: whx-sjtu <2952154980@qq.com>
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Signed-off-by: PatchouliTaisa <patchychen@tencent.com>
Signed-off-by: jatseng-ai <jatseng@amd.com>
Signed-off-by: jatseng-ai <janet.tseng@amd.com>
Signed-off-by: Matthias Gehre <matthias.gehre@amd.com>
Signed-off-by: xaguilar-amd <xaguilar@amd.com>
Signed-off-by: rdondeti <ravitez.dondeti@gmail.com>
Signed-off-by: Ravitez Dondeti <ravitez.dondeti@gmail.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: Peter Nguyen <petern0408@gmail.com>
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Signed-off-by: Jesus Federico <jefp@amazon.com>
Signed-off-by: manu <fortin.emmanuel@gmail.com>
Signed-off-by: ZhanqiuHu <zhu@redhat.com>
Signed-off-by: Yifan Zong <yzong@redhat.com>
Signed-off-by: Rahul-Tuli <rtuli@redhat.com>
Signed-off-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Michael Goin <mgoin64@gmail.com>
Signed-off-by: Benjamin Chislett <bchislett@nvidia.com>
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: leeyongjun <jqueen.astro@gmail.com>
Signed-off-by: Ziying Tao <tzzying@outlook.com>
Signed-off-by: jiang1.li <jiang1.li@intel.com>
Signed-off-by: Vibhav Agarwal <vibhavagarwal5@gmail.com>
Signed-off-by: ShubyM <shubymishra20@gmail.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Signed-off-by: Itay Etelis <itay.etelis@ibm.com>
Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: r266-tech <r266.tech@gmail.com>
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
Signed-off-by: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Animesh Jain <anijain@umich.edu>
Signed-off-by: Yongye Zhu <zyy1102000@gmail.com>
Signed-off-by: zhxchen17 <zhxchen17@fb.com>
Signed-off-by: EricccYang <yangyang4991@gmail.com>
Signed-off-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com>
Signed-off-by: baoloongmao <baoloongmao@tencent.com>
Signed-off-by: sihao.li <sihao.li@intel.com>
Signed-off-by: sfeng33 <4florafeng@gmail.com>
Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
Signed-off-by: Tihomir Elek <tiho.elek@gmail.com>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Santino Ramos <santinor@inferact.ai>
Signed-off-by: haosdent <haosdent@gmail.com>
Signed-off-by: JartX <sagformas@epdcenter.es>
Signed-off-by: George-ao <yuyiao772@gmail.com>
Signed-off-by: Yuyi Ao <yuyiao772@gmail.com>
Signed-off-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
Signed-off-by: Mukesh Baphna <mukesh@hippocraticai.com>
Signed-off-by: Pedram Razavi <pedram.razavi@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Michael Goin <mgoin64@gmail.com>
Co-authored-by: Xinyu Chen <xinyu1.chen@intel.com>
Co-authored-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: Rishi Puri <riship@nvidia.com>
Co-authored-by: zzaebok <44357534+zzaebok@users.noreply.github.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Yuwei An <ayw.sirius19@gmail.com>
Co-authored-by: yuwei <yuwei@dev.local>
Co-authored-by: Artem Perevedentsev <aperevedents@nvidia.com>
Co-authored-by: Ibrahim Arshad <38925737+ibrahim1023@users.noreply.github.com>
Co-authored-by: Chuan (Richard) Li <chuali@amd.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
Co-authored-by: Kunshang Ji <kunshang.ji@intel.com>
Co-authored-by: Ganesh R <ganesh.r@amd.com>
Co-authored-by: Lucas Wilkinson <LucasWilkinson@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Kyungmin Lee <30465912+lkm2835@users.noreply.github.com>
Co-authored-by: Ronen Schaffer <ronen.schaffer@ibm.com>
Co-authored-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Co-authored-by: vnadathur <glvikramn@gmail.com>
Co-authored-by: vnadathur <236933696+vnadathur@users.noreply.github.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Elham <elham.harirpoush@arm.com>
Co-authored-by: Yan Ma <yan.ma@intel.com>
Co-authored-by: Nick Hill <nickhill123@gmail.com>
Co-authored-by: Chaofan Wang <jackcfwang@tencent.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
Co-authored-by: Injae Ryou <injaeryou@gmail.com>
Co-authored-by: Richard Zou <zou3519@users.noreply.github.com>
Co-authored-by: milesial <milesial@users.noreply.github.com>
Co-authored-by: Elvir Crnčević <elvircrn@gmail.com>
Co-authored-by: Claude Sonnet 4 <noreply@anthropic.com>
Co-authored-by: Hexiang Wang <56632993+whx-sjtu@users.noreply.github.com>
Co-authored-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
Co-authored-by: PatchyTIS <58251192+PatchouliTIS@users.noreply.github.com>
Co-authored-by: PatchouliTaisa <patchychen@tencent.com>
Co-authored-by: jatseng-ai <janet.tseng@amd.com>
Co-authored-by: Matthias Gehre <matthias.gehre@amd.com>
Co-authored-by: xaguilar-amd <xavier.aguilarfruto@amd.com>
Co-authored-by: Ravitez Dondeti <dondetir@users.noreply.github.com>
Co-authored-by: Nicolò Lucchesi <nlucches@redhat.com>
Co-authored-by: Peter Nguyen <petern0408@gmail.com>
Co-authored-by: wang.yuqi <yuqi.wang@daocloud.io>
Co-authored-by: zhrrr <43847754+izhuhaoran@users.noreply.github.com>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: TJian <tunjian.tan@embeddedllm.com>
Co-authored-by: Jesus Federico <14651+jefp@users.noreply.github.com>
Co-authored-by: Manu <efortin@users.noreply.github.com>
Co-authored-by: zhanqiuhu <49648934+ZhanqiuHu@users.noreply.github.com>
Co-authored-by: yzong-rh <yzong@redhat.com>
Co-authored-by: Fynn Schmitt-Ulms <fschmitt@redhat.com>
Co-authored-by: Rahul-Tuli <rtuli@redhat.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Benjamin Chislett <bchislett@nvidia.com>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Co-authored-by: Lee Yongjun <35302114+elwhyjay@users.noreply.github.com>
Co-authored-by: z1ying <55220715+z1ying@users.noreply.github.com>
Co-authored-by: Li, Jiang <jiang1.li@intel.com>
Co-authored-by: Vibhav Agarwal <vibhavagarwal5@gmail.com>
Co-authored-by: vibhav-agarwal <vibhav.agarwal@glance.com>
Co-authored-by: ShubyM <shubymishra20@gmail.com>
Co-authored-by: Wei Zhao <51183510+wzhao18@users.noreply.github.com>
Co-authored-by: Itay Etelis <92247226+Etelis@users.noreply.github.com>
Co-authored-by: Itay Etelis <itay.etelis@ibm.com>
Co-authored-by: EdalatiAli <aliedalati@cohere.com>
Co-authored-by: Andreas Karatzas <akaratza@amd.com>
Co-authored-by: r266-tech <r2668940489@gmail.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
Co-authored-by: Martin Hickey <martin.hickey@ie.ibm.com>
Co-authored-by: Or Ozeri <or@ozery.com>
Co-authored-by: Mark McLoughlin <markmc@redhat.com>
Co-authored-by: Le Yang <562593859@qq.com>
Co-authored-by: Animesh Jain <anijain@umich.edu>
Co-authored-by: Yongye Zhu <zyy1102000@gmail.com>
Co-authored-by: Zhengxu Chen <zhxchen17@fb.com>
Co-authored-by: Kaicheng Yang <53411596+EricccYang@users.noreply.github.com>
Co-authored-by: maobaolong <baoloongmao@tencent.com>
Co-authored-by: sihao_li <165983188+1643661061leo@users.noreply.github.com>
Co-authored-by: Flora Feng <4florafeng@gmail.com>
Co-authored-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>
Co-authored-by: zofia <110436990+zufangzhu@users.noreply.github.com>
Co-authored-by: Tihomir Elek <tiho.elek@gmail.com>
Co-authored-by: Yi Liu <yi4.liu@intel.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: Santino Ramos <51103228+santiramos27@users.noreply.github.com>
Co-authored-by: haosdent <haosdent@gmail.com>
Co-authored-by: JartX <sagformas@epdcenter.es>
Co-authored-by: Yuyi Ao <yuyiao772@gmail.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: mukesh-hai <mukesh@hippocraticai.com>
Co-authored-by: Pedram Razavi <pedram@sierra.ai>
@JartX
scaling triton kernels
eeb3484 
Signed-off-by: JartX <sagformas@epdcenter.es>
@JartX
scaling triton kernels
b368f19 
Signed-off-by: JartX <sagformas@epdcenter.es>
@orozery
[kv_offload+HMA][3/N]: Remove block_size from KVEvents (vllm-project#…
235e1f9 
…36644)

Signed-off-by: Or Ozeri <oro@il.ibm.com>
@yma11
fix online fp8 for MiniCPM models (vllm-project#39862)
29e5d10 
Signed-off-by: Yan Ma <yan.ma@intel.com>
@Zhenzhong1
[Quantization][Autoround][CPU] Add W4A16 Support (vllm-project#38192)
60995c0 
Signed-off-by: Zhenzhong1 <zhenzhong.xu@intel.com>
Signed-off-by: Zhenzhong Xu <zhenzhong.xu@intel.com>
@Csrayz
[Metrics] Add request_id to FinishedRequestStats to enable correlatio…
68be0f8 
…n between metrics and requests (vllm-project#39710)

Enables external `StatLogger` plugins to correlate per-request metrics
with request-level context. Also, this is a pre-requisite for Prometheus
exemplars in vllm-project#30972.

Signed-off-by: Csrayz <33659823+Csrayz@users.noreply.github.com>
@JartX
precommit
78f9f48 
Signed-off-by: JartX <sagformas@epdcenter.es>
@zufangzhu
[XPU][MXFP4] add mxfp4 quant op for XPU (vllm-project#39857)
fc701c8 
Signed-off-by: Zhu, Zufang <zufang.zhu@intel.com>
@JartX
refactor packed_per_token_head
eb087e5 
Signed-off-by: JartX <sagformas@epdcenter.es>
@JartX
refactor packed_per_token_head
86a153b 
Signed-off-by: JartX <sagformas@epdcenter.es>
fergusfinn and others added 26 commits April 21, 2026 18:31
@fergusfinn @chaunceyjiang
[Performance] Add is_reasoning_end_streaming() override to GptOssReas…
5544f8c 
…oningParser (vllm-project#35745)

Signed-off-by: Fergus <fergus.barratt00@gmail.com>
Signed-off-by: fergus barratt <fergus.barratt00@gmail.com>
Co-authored-by: Chauncey <chaunceyjiang@gmail.com>
@jzakrzew
[Bugfix][Kernel] nvfp4 cutlass MoE: fix nvfp4 experts quant out-of-bo…
6fbec8e 
…unds read for expert counts not divisible by 4 or 16 (vllm-project#40351)

Signed-off-by: Jakub Zakrzewski <jzakrzewski@nvidia.com>
@yewentao256
[Perf] Optimize batch invariant with fused rms norm, 2.1% E2E latency…
16688b2 
… improvement (vllm-project#40413)

Signed-off-by: yewentao256 <zhyanwentao@126.com>
@yewentao256
[Refactor] Remove unused param (vllm-project#39750)
1842447 
Signed-off-by: yewentao256 <zhyanwentao@126.com>
@bnellnm
[MoE Refactor] Remove SharedFusedMoE class (vllm-project#35782)
5e584ce 
Signed-off-by: Bill Nell <bnell@redhat.com>
@bnellnm @robertgshaw2-redhat
[MoE Refactor] Add more MoE layer tests (vllm-project#39349)
9db4650 
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-redhat@users.noreply.github.com>
@MatthewBonanni @tlrmchlsmth
[Startup][UX] Enable CUDAGraph memory profiling by default (vllm-proj…
96a85c5 
…ect#38284)

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>
Co-authored-by: Tyler Michael Smith <tlrmchlsmth@gmail.com>
@tjtanaa
[ROCm] [Wheel] [Bugfix] [Critical] Remove any packages installed from…
583e6f2 
… github from rocm.txt e.g `fastsafetensors` as it is incompatible with `uv pip` (vllm-project#40461)

Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
@khushali9
[Bugfix] avoid warmup if text only expectation in multi_modal run (vl…
6ff8dea 
…lm-project#40409)

Signed-off-by: khushali9 <khushali.desai9@gmail.com>
@jhaotingc
test: add nan/inf clamp regression test for fused_topk_bias (vllm-pro…
4679495 
…ject#40553)

Signed-off-by: Jhao-Ting Chen <jhaotingc@nvidia.com>
@rasmith
[ROCm][P/D][MORI][BugFix] Ensure correct api is used when making requ…
cefa528 
…ests to prefill / decode nodes (vllm-project#39835)

Signed-off-by: Randall Smith <Randall.Smith@amd.com>
@skavulya
[NIXL][XPU]Fix nixl import on XPU (vllm-project#40430)
f90aa44 
Signed-off-by: Soila Kavulya <soila.p.kavulya.intel.com>
@EdalatiAli @gemini-code-assist @mergify
[Bugfix] Fix W4A8_FP8 MoE tp>1 correctness and view() TypeError (vllm…
f946659 
…-project#40310)

Signed-off-by: EdalatiAli <aliedalati@cohere.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@rasmith
[AMD][CI][BugFix] Override normalize_e4m3fn_to_e4m3fnuz for fnuz mach…
2463f00 
…ines in test_moe_layer_no_parallel (vllm-project#40550)

Signed-off-by: Randall Smith <Randall.Smith@amd.com>
@jaseelmohd2 @Isotr0py
[Multimodal] Add PyAV video backend for concurrent video decoding (vl…
6f2c71b 
…lm-project#39986)

Signed-off-by: Jaseel Muhammad <jaseel.muhammad@mbzuai.ac.ae>
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
@hickeyma
[MyPy] Enable mypy for vllm/model_executor/layers/ (vllm-project#40159
3951d3e 
)

Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
@rishaps @mergify
[Bugfix] Fix quantized model initialization failure with prefetch off…
9b60e2f 
…loading (vllm-project#40432)

Signed-off-by: Rishapveer Singh <singhrishapveer@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@carlyou @mergify
[compile] mla + group fp8 fusion (vllm-project#38877)
4506319 
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@micah-wil
[ROCm] Support non-causal attention in ROCM_ATTN (vllm-project#40176)
6d09769 
Signed-off-by: Micah Williamson <micah.williamson@amd.com>
@ekagra-ranjan @gemini-code-assist @DarkLight1337
[Audio] Bundle get_generation_prompt() params into `SpeechToTextPar…
4eafc72 
…ams` (vllm-project#36268)

Signed-off-by: Ekagra Ranjan <3116519+ekagra-ranjan@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@BugenZhao
[Bugfix] Pass effective chat template kwargs to reasoning parsers (vl…
0210024 
…lm-project#40460)

Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@orozery @mergify
[kv_offload+HMA][8/N]: Support multi-group worker transfer (vllm-proj…
aad88f8 
…ect#38453)

Signed-off-by: Or Ozeri <oro@il.ibm.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@carlyou
[fix] flaky test_mla_attn_quant_fusion.py (vllm-project#40530)
4254aeb 
Signed-off-by: Carl You <4531192+carlyou@users.noreply.github.com>
@philip-essential
[Model] Add block-local attention and YaRN for local layers to Gemma3 (…
1236748 
…vllm-project#39823)

Signed-off-by: Philip Monk <169196560+philip-essential@users.noreply.github.com>
@chaunceyjiang
[Bugfix] [Reasoning] Add reasoning_start_str/reasoning_end_str proper…
a2bd09c 
…ties to reasoning parsers (vllm-project#40566)

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
@JartX
Merge branch 'main' into feature/int2_int4_per_token_head
7878dc2
@JartX
JartX force-pushed the feature/int2_int4_per_token_head branch from 67bf95c to 7878dc2 Compare April 22, 2026 07:42
storyicon and others added 3 commits April 22, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.