Skip to content

init int4 per token - #8

Open
JartX wants to merge 18 commits into
mainfrom
feature/int4_per_token
Open

init int4 per token#8
JartX wants to merge 18 commits into
mainfrom
feature/int4_per_token

Conversation

@JartX

@JartX JartX commented Apr 2, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Owner

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

YM2132 and others added 12 commits April 2, 2026 09:29
@YM2132 @claude
@yewentao256 yusuf
[Bugfix] Enable batch-invariant Triton matmul on all Ampere GPUs (SM …
16a65e4 
…8x) (vllm-project#38427)

Signed-off-by: yusuf <yusufmohammad@live.com>
Signed-off-by: yusuf <yusuf@deeplearningmachine.mynet>
Signed-off-by: Yusuf Mohammad <79484377+YM2132@users.noreply.github.com>
Signed-off-by: <>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Wentao Ye <44945378+yewentao256@users.noreply.github.com>
Co-authored-by: yusuf <yusuf@deeplearningmachine.mynet>
@koush root @DarkLight1337
Triton MLA perf fixes (vllm-project#33529)
d9408ff 
Signed-off-by: Koushik Dutta <koushd@gmail.com>
Co-authored-by: root <root@ubuntu-nvidia.localdomain>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
@noooop
[Frontend] Re-enable running MaxSim on GPU (vllm-project#38620)
a9b4f07 
Signed-off-by: wang.yuqi <yuqi.wang@daocloud.io>
@BowenBao
[CI][ROCm] Add gpt-oss w4a8 in CI (vllm-project#38292)
82a006b 
Signed-off-by: Bowen Bao <bowenbao@amd.com>
@LucasWilkinson
[FA4] Update flash-attention to latest upstream FA4 (vllm-project#38690)
cb3935a 
Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>
@stecasta @claude
[Bugfix] Fix test mocks after SM100 restriction in vllm-project#38730 (…
58262de 
…vllm-project#38791)

Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Claude <noreply@anthropic.com>
@gmagogsfm @claude
Bump helion dependency from 0.3.2 to 0.3.3 (vllm-project#38062)
ecd5443 
Signed-off-by: Yanan Cao <gmagogsfm@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@lucianommartins @Isotr0py
feat(models): implement Google Gemma 4 architecture support (MoE, Mul…
08ed2b9 
…timodal, Reasoning, Tool-Use) (vllm-project#38826)

Signed-off-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Signed-off-by: Luciano Martins <lucianomartins@google.com>
Co-authored-by: Luciano Martins <lucianommartins@users.noreply.github.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
@stecasta @ProExpertProg
[CI] Add flashinfer.py to attention test source deps (vllm-project#38792
188defb 
)

Signed-off-by: Stefano Castagnetta <scastagnetta@nvidia.com>
Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>
@ZhanqiuHu
[CI] Fix: pass string cache_dtype in test_register_kv_caches (vllm-pr…
7b743ba 
…oject#38836)
@hospedales
[Bugfix]: Fix Gemma4ToolParser.__init__() missing tools parameter (v…
bb39382 
…llm-project#38847)

Signed-off-by: Michael Hospedales <hospedales@me.com>
@JartX
init int4 per token
60e0ba0 
Signed-off-by: JartX <sagformas@epdcenter.es>
@JartX
JartX force-pushed the feature/int4_per_token branch from 7b4549c to 60e0ba0 Compare April 2, 2026 21:54
@JartX
precommit
4d1ac5e 
Signed-off-by: JartX <sagformas@epdcenter.es>
@JartX
JartX force-pushed the feature/int4_per_token branch from e33ef4b to 4d1ac5e Compare April 2, 2026 23:41
JartX added 5 commits April 3, 2026 01:41
@JartX
Merge branch 'main' into feature/int4_per_token
3ac2d18
@JartX
remove ls
573aaa7 
Signed-off-by: JartX <sagformas@epdcenter.es>
@JartX
refactor to nf4
97ec9d8 
Signed-off-by: JartX <sagformas@epdcenter.es>
@JartX
rollback
7807988 
Signed-off-by: JartX <sagformas@epdcenter.es>
@JartX
change comment
ee15230 
Signed-off-by: JartX <sagformas@epdcenter.es>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.