Skip to content

init: first impliment for async GPU connector - #239

Draft
specture724 wants to merge 2 commits into
vllm-project:mainfrom
specture724:AsyncGPU
Draft

init: first impliment for async GPU connector#239
specture724 wants to merge 2 commits into
vllm-project:mainfrom
specture724:AsyncGPU

Conversation

@specture724

@specture724 specture724 commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Purpose

Impliment an async GPU connector based on NVSHMEM.

Issue

Scope

  • In scope: GPU connectors, model runners, models
  • Out of scope: NPU connectors

Test Plan

test_async_gpu_connector.py is the main unit test added. async_gpu_connector_e2e.py is for e2e GPU test. 1a1f and 2a2f eager recipe added.

Test Result

TBD

Docs Impact

  • Files updated: TODO add docs for design
  • If none, reason:

Essential PR Checklist
  • Purpose is clear and linked to public context when possible.
  • Scope is bounded.
  • Compatibility with vLLM v0.26.0 is considered.
  • No changes are made to the vLLM source checkout.
  • Plugin-owned classes or explicit dotted class paths are preferred over monkey patches.
  • Any compat shim or monkey patch is isolated, idempotent, version-guarded, documented, and tested.
  • Imports remain CPU-safe; CUDA-heavy work is delayed or GPU-gated.
  • Validation evidence is included, including skipped GPU tests when applicable.
  • Documentation impact is stated.

Signed-off-by: specture724 <specture724@gmail.com>
@specture724
specture724 marked this pull request as draft August 13, 2026 08:31
Signed-off-by: specture724 <specture724@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC]: Async GPU Connector

1 participant