Skip to content

ci: prototype vllm-vcr backend for faster Responses integration tests - #719

Draft
leseb wants to merge 6 commits into
praxis-proxy:mainfrom
leseb:leseb/evaluate-vllm-vcr
Draft

ci: prototype vllm-vcr backend for faster Responses integration tests#719
leseb wants to merge 6 commits into
praxis-proxy:mainfrom
leseb:leseb/evaluate-vllm-vcr

Conversation

@leseb

@leseb leseb commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Prototype a vllm-vcr-backed CI lane to measure the speedup from
eliminating the ~8 minute vLLM CPU model startup penalty. The
branch-scoped workflow runs the vLLM frontend with vllm-vcr random
tokens instead of real inference. The test file gains a
VLLM_TEST_BACKEND=vcr-random mode that runs 3 protocol-safe tests
(smoke, store/retrieve, streaming) and skips 7 that need real model
inference or captured tool-call traces.

Related issue

N/A — exploratory prototype to evaluate CI runtime savings.

Validation

  • make lint
  • make build
  • CI prototype workflow (will run on push to this branch)

Checklist

  • I reviewed every changed line and can explain the change.
  • New capabilities include an example config and functional example test.
  • User-facing behavior and generated documentation are updated.
  • Performance-sensitive changes include appropriate benchmark or load-test evidence.
  • Commits are signed and include a Signed-off-by trailer.

Breaking changes

None — branch-scoped prototype workflow, no changes to existing CI or test behavior.

leseb added 6 commits August 12, 2026 11:28
Add a branch-scoped CI workflow that replaces the real vLLM CPU engine
with vllm-vcr random tokens, eliminating the ~8 minute model startup
penalty. The test file gains a VLLM_TEST_BACKEND=vcr-random mode that
runs 3 protocol-safe tests (smoke, store/retrieve, streaming) while
skipping 7 that require real inference or captured tool-call traces.

Signed-off-by: Sébastien Han <seb@redhat.com>
Signed-off-by: Sébastien Han <seb@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant