ci: add vLLM GPU hello-world smoke (T4 / Qwen3-0.6B) by skamenan7 · Pull Request #640 · praxis-proxy/ai

skamenan7 · 2026-07-31T19:36:44Z

Summary

Adds a workflow_dispatch GPU smoke workflow for Praxis issue Add vLLM integration tests on GPU and expand CPU coverage #541
Runs the existing test_stateless_request HELLO-PRAXIS check against CUDA vLLM (Qwen/Qwen3-0.6B, max 4096 tokens)
Defaults to Ken’s offered gpu-t4-4-core larger-runner label; runner label is overridable
Avoids the OGX ec2-github-runner / personal-PAT pattern

Test plan

Confirm gpu-t4-4-core (or equivalent) exists on praxis-proxy
Actions → vLLM GPU Smoke → Run workflow
Job shows nvidia-smi, vLLM becomes healthy, test_stateless_request passes
Failure path uploads vLLM/OGX logs

praxis-bot-app · 2026-07-31T19:36:57Z

Unsigned commits: ffbecfa. Please sign your commits.

praxis-bot-app · 2026-07-31T19:36:58Z

Missing Signed-off-by: ffbecfa. All commits require sign-off (via git commit --signoff).

skamenan7 · 2026-07-31T19:38:23Z

Dispatched vLLM GPU Smoke on this branch (runner=gpu-t4-4-core).

The job stayed queued with no runner picked up, so praxis-proxy does not appear to have a gpu-t4-4-core larger-runner yet (Ken offered that shape from another org).

Once that runner exists on this org (or we get the exact label), re-run:
Actions → vLLM GPU Smoke → gpu-t4-4-core (or override).

Smoke itself is the existing test_stateless_request / HELLO-PRAXIS check against CUDA vLLM Qwen/Qwen3-0.6B.

Wire a workflow_dispatch job for Qwen/Qwen3-0.6B that runs the existing stateless HELLO-PRAXIS responses check on gpu-t4-4-core (or an override label), without cloning the OGX ec2-github-runner PAT pattern. Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>

praxis-bot

Review: vLLM GPU Smoke Workflow

Clean workflow with good security practices throughout: minimal permissions, SHA-pinned actions with version comments, digest-pinned container image, proper cleanup, and failure log collection. The workflow_dispatch-only trigger is appropriate for manual GPU testing. The step ordering efficiently parallelizes the Rust build with vLLM model loading.

Consistent with the existing vllm-integration.yaml patterns.

Severity	Count
Critical	0
Large	0
Medium	1

Reviewed by praxis-bot

nathan-weinberg · 2026-08-03T18:03:42Z

+# Does not use the OGX ec2-github-runner / personal-PAT pattern.
+
+on:
+  workflow_dispatch:


Is this just for testing? Presume we will want to also run this on a schedule/PRs/etc.

nathan-weinberg · 2026-08-03T18:07:37Z

+
+env:
+  CARGO_TERM_COLOR: always
+  VLLM_IMAGE: "docker.io/vllm/vllm-openai:v0.22.1-cu129@sha256:e1668bce9790a4b86682f8fcc99678153a13e12dc70e05348d8e239ffa474b05"


@leseb @franciscojavierarceo hey guys, @skamenan7 and I were talking and wanted to loop y'all in on an idea

For the OGX distro, we have automation in place in the repo for building and pushing our own vLLM CPU images with baked-in models, including Dependabot configurations for automatically updating the vLLM base image via a PR. This prevents us from being reliant on HuggingFace credentials/uptime for our testing.

I could replicate those workflows here or in another repo somewhere in Praxis if we were interested in using the same strategy for the testing here, but with CUDA-based GPU images rather than CPU

Let me know your thoughts - if y'all agree I can open a new tracker issue and assign myself for this work - it shouldn't be a heavy lift

A PoC for this idea can be found here: https://github.com/praxis-proxy/ai/tree/vllm-gpu

That's a good idea, once this merges.

Opened an issue to track and assigned myself: #692

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>

praxis-bot

Review: vLLM GPU Smoke Workflow (round 2)

The Docker/Podman consolidation from the previous review is properly incorporated. The workflow is well-structured: SHA-pinned actions, digest-pinned container image, minimal permissions, proper if: always() cleanup, and good parallelization of service startup with the Rust build.

One remaining item:

Severity	Count
Critical	0
Large	0
Medium	1

Reviewed by praxis-bot

praxis-bot · 2026-08-06T14:40:58Z

+
+      - name: Start OGX
+        run: |
+          uv run --with 'ogx[starter]' ogx run starter --insecure > /tmp/ogx.log 2>&1 &


[Medium] ogx[starter] is not version-pinned. A breaking OGX release will fail this workflow with no corresponding diff to bisect.

Pin to the version you're currently testing against:

uv run --with 'ogx[starter]==<version>' ogx run starter --insecure > /tmp/ogx.log 2>&1 &

The existing vllm-integration.yaml has the same gap, but a new workflow shouldn't carry it forward.

leseb

A few thoughts:

We should use a more capable model, otherwise the value is low
We need to use this instance with the existing vLLM integration test

leseb · 2026-08-10T08:23:02Z

+env:
+  CARGO_TERM_COLOR: always
+  VLLM_IMAGE: "docker.io/vllm/vllm-openai:v0.22.1-cu129@sha256:e1668bce9790a4b86682f8fcc99678153a13e12dc70e05348d8e239ffa474b05"
+  VLLM_MODEL: "Qwen/Qwen3-0.6B"


Can't we use a more capable model?

Perhaps Qwen/Qwen3.5-9B?

leseb · 2026-08-10T08:26:21Z

+            --reasoning-parser deepseek_r1 \
+            --gpu-memory-utilization 0.7
+
+      - name: Start OGX


leseb · 2026-08-10T08:26:58Z

+
+env:
+  CARGO_TERM_COLOR: always
+  VLLM_IMAGE: "docker.io/vllm/vllm-openai:v0.22.1-cu129@sha256:e1668bce9790a4b86682f8fcc99678153a13e12dc70e05348d8e239ffa474b05"


That's a good idea, once this merges.

skamenan7 requested review from a team and leseb July 31, 2026 19:36

skamenan7 force-pushed the feat/541-vllm-gpu-smoke branch 2 times, most recently from d1d6ee2 to bbfccbd Compare July 31, 2026 19:45

skamenan7 closed this Jul 31, 2026

skamenan7 force-pushed the feat/541-vllm-gpu-smoke branch from bbfccbd to fe8ecee Compare July 31, 2026 19:45

skamenan7 reopened this Jul 31, 2026

skamenan7 marked this pull request as draft July 31, 2026 19:55

skamenan7 force-pushed the feat/541-vllm-gpu-smoke branch from 8e3df81 to 27f2e6c Compare July 31, 2026 20:04

nathan-weinberg self-requested a review August 3, 2026 14:36

praxis-bot reviewed Aug 3, 2026

View reviewed changes

Comment thread .github/workflows/vllm-gpu-smoke.yaml Outdated

nathan-weinberg reviewed Aug 3, 2026

View reviewed changes

skamenan7 added 2 commits August 4, 2026 15:35

ci: deduplicate vLLM container startup

371da83

Signed-off-by: Sumanth Kamenani <skamenan@redhat.com>

Merge branch 'main' into feat/541-vllm-gpu-smoke

8ae5977

praxis-bot reviewed Aug 6, 2026

View reviewed changes

leseb reviewed Aug 10, 2026

View reviewed changes

Uh oh!

Conversation

skamenan7 commented Jul 31, 2026

Summary

Test plan

Uh oh!

praxis-bot-app Bot commented Jul 31, 2026

Uh oh!

praxis-bot-app Bot commented Jul 31, 2026

Uh oh!

skamenan7 commented Jul 31, 2026

Uh oh!

praxis-bot left a comment

Choose a reason for hiding this comment

Review: vLLM GPU Smoke Workflow

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nathan-weinberg Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

praxis-bot left a comment

Choose a reason for hiding this comment

Review: vLLM GPU Smoke Workflow (round 2)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leseb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nathan-weinberg Aug 3, 2026 •

edited

Loading