test: add E2E tests for payload processor with Kind cluster CI by asaadbalum · Pull Request #31 · llm-d/llm-d-inference-payload-processor

asaadbalum · 2026-05-04T11:31:21Z

Summary

Add end-to-end tests that deploy a complete Envoy + Payload Processor + model-server-simulator stack on a Kind cluster and validate core functionality through the actual ext_proc gRPC pipeline.

Base model routing: Verifies that model field extraction from /v1/chat/completions and /v1/completions bodies routes Llama and DeepSeek requests to the correct pools via X-Gateway-Base-Model-Name header.
LoRA adapter routing: Verifies that adapter names are resolved to base models through ConfigMap reconciliation and routed to the correct pool.
Streaming: Verifies that streaming requests ("stream": true) return SSE text/event-stream chunks through the full Envoy → Payload Processor → model-server pipeline.
Metrics: Verifies that ipp_info and ipp_success_total Prometheus metrics are populated after traffic flows.
CI integration: E2E job added to ci-pr-checks.yaml, skipping docs-only changes. Removed unused python-lint and container-build jobs.

Manifest structure

Kubernetes manifests live under deploy/ following the llm-d-router pattern: shared components (deploy/components/) and environment-specific infrastructure (deploy/environments/dev/e2e-infra/). Each component directory includes a kustomization.yaml. Test code references these manifests via relative paths with ${VAR} substitution, enabling reuse for both E2E tests and local Kind development.

The E2E Envoy configuration mirrors production:

Envoy v1.33 with request_body_mode: FULL_DUPLEX_STREAMED
allow_mode_override: true, failure_mode_allow: false
Payload Processor runs in streaming-only mode (matching upstream after PR refactor(extproc): remove buffered mode, use streaming only #38)
request_trailer_mode: SEND, response_trailer_mode: SKIP

New files

File	Purpose
`deploy/components/ipp/deployment.yaml`	Payload Processor Deployment (parameterized)
`deploy/components/ipp/service.yaml`	Payload Processor Service
`deploy/components/ipp/rbac.yaml`	ServiceAccount + ClusterRole + ClusterRoleBinding
`deploy/components/ipp/kustomization.yaml`	Kustomize resource list for IPP component
`deploy/components/model-server/llama/deployment.yaml`	Llama simulator + Service + adapter ConfigMap
`deploy/components/model-server/llama/kustomization.yaml`	Kustomize resource list
`deploy/components/model-server/deepseek/deployment.yaml`	DeepSeek simulator + Service + adapter ConfigMap
`deploy/components/model-server/deepseek/kustomization.yaml`	Kustomize resource list
`deploy/environments/dev/e2e-infra/envoy.yaml`	Envoy proxy with ext_proc filter config
`deploy/environments/dev/e2e-infra/client.yaml`	Curl client pod for in-cluster requests
`test/e2e/e2e_suite_test.go`	Ginkgo BeforeSuite/AfterSuite — deploys stack, waits for readiness
`test/e2e/e2e_test.go`	7 test cases covering all scenarios above
`test/e2e/README.md`	Developer quickstart guide for running E2E locally
`test/e2e/TROUBLESHOOTING.md`	Troubleshooting guide for common issues
`hack/test-e2e.sh`	Shell script orchestrating Kind + image build + test run

Modified files

File	Change
`.github/workflows/ci-pr-checks.yaml`	Added `e2e` job, removed `python-lint` and `container-build`
`Makefile`	Added `test-e2e`, `image-build-local`, `image-kind` targets

Test plan

All 7 E2E tests pass on Kind (7 Passed | 0 Failed)
All existing unit/integration tests pass (go test ./...)
Lint clean (go vet + build tags)
Manual verification with kubectl exec curl for each scenario
CI workflow validated locally (make image-kind && make test-e2e)

Closes #14

github-actions · 2026-05-04T11:31:31Z

Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

aradhalevy

Looks good, some minor comments (and we will need the new added check to run and pass first)

aradhalevy · 2026-05-05T14:15:02Z

+  name: llama-adapters
+  namespace: $E2E_NS
+  labels:
+    inference.llm-d.io/ipp-managed: "true"


I believe it supposed to be llm-d.ai instead of llm-d.io as per #28.

After this fix the test pass for me locally

aradhalevy · 2026-05-05T14:15:39Z

+  name: deepseek-adapters
+  namespace: $E2E_NS
+  labels:
+    inference.llm-d.io/ipp-managed: "true"


same llm-d.ai instead of llm-d.io

aradhalevy · 2026-05-05T14:41:14Z

+| Streaming routing | SSE chunks returned |
+| Metrics | `bbr_info`, `bbr_success_total` |
+
+## Troubleshooting


When I had a couple of other clusters set up in kind, Envoy tried to route requests to them. Please add a suggestion / troubleshooting to use kind delete clusters --all first to clean your kind environment first.

aradhalevy · 2026-05-05T18:28:08Z

+      containers:
+        - name: payload-processor
+          image: $E2E_IMAGE
+          imagePullPolicy: Never


This should be IfNotPresent if we want to test on a different cluster other than kind. But that requires pushing an image to ghcr.io and might require some more changes, and can be dealt with in another issue / PR if you prefer to keep this PR for kind only

Keeping it for now, will address it in a follow-up pr

aradhalevy · 2026-05-05T18:32:35Z

+
+      - name: Run E2E tests
+        run: |
+          E2E_IMAGE=ghcr.io/llm-d/llm-d-inference-payload-processor:e2e \


You don't use the Makefile / script here, I think it would be better to use them to have a single source of truth.

github-actions · 2026-05-06T07:57:19Z

⚠️ Large PR detected

Your PR is large. Please consider breaking it into multiple PRs.

The do-not-merge/hold label has been added and can be removed by the reviewers based on their judgement.

aradhalevy

LGTM.
I think this is fine even tough it is a large PR as all the code is needed and relevant to this minimal e2e test suite.

nirrozenbaum · 2026-05-06T12:23:44Z

+
+var (
+	testConfig *testutils.TestConfig
+	ppImage    string


nit: it would be better to align the name on ipp rather than pp (that was the agreed acronym).

nirrozenbaum · 2026-05-06T12:27:21Z

+| Base model routing | Pool routing via header |
+| LoRA adapter routing | ConfigMap adapter lookup |
+| Streaming routing | SSE chunks returned |
+| Metrics | `bbr_info`, `bbr_success_total` |


as a follow up, we should update all metrics to be named ipp instead of bbr.

not a blocker

nirrozenbaum · 2026-05-06T12:28:52Z

+| Streaming routing | SSE chunks returned |
+| Metrics | `bbr_info`, `bbr_success_total` |
+
+## Troubleshooting


this should probably go to a separate troubleshot guide.
quickstart guide should be quick, and simple :)
in other words, the simplest explanation of the green path.

nirrozenbaum · 2026-05-06T12:31:55Z

@@ -0,0 +1,164 @@
+# Llama model server simulator
+apiVersion: apps/v1


can you explain the separation between e2e-deployment and deepseek-model-server?
I see deepseek has deployment + svc.
here I see deployment + svc for a llama plus adapter of deepseek + llama + many other CRs.
not sure I understand the separation.

nirrozenbaum · 2026-05-06T12:36:58Z

+              - '!**/*.md'
+              - '!LICENSE'
+              - '!OWNERS'
+


can you move this logic to the file "ci-pr-checks.yaml" (and on the way to clean from it the lint python and build at the end)?

github-actions · 2026-05-11T07:13:25Z

⚠️ Large PR detected

Your PR is large. Please consider breaking it into multiple PRs.

The do-not-merge/hold label has been added and can be removed by the reviewers based on their judgement.

nirrozenbaum · 2026-05-13T05:57:40Z

@shmuelk can you please review this PR when you have time?
would be good to validate the tests structure is aligned with scheduler.

shmuelk · 2026-05-13T12:22:38Z

@nirrozenbaum I took a very quick look at this PR.

I don't like it's structure. This E2E test looks a lot more like the old IGW E2E test and not like the scheduler's E2E test.

@roytman restructured the End to End test and the development environment on Kind to use the same K8S YAML and config YAML files where possible. Following that idea here will make it easier to put together a development environment on Kind.

nirrozenbaum · 2026-05-13T21:13:06Z

@nirrozenbaum I took a very quick look at this PR.

I don't like it's structure. This E2E test looks a lot more like the old IGW E2E test and not like the scheduler's E2E test.

@roytman restructured the End to End test and the development environment on Kind to use the same K8S YAML and config YAML files where possible. Following that idea here will make it easier to put together a development environment on Kind.

@asaadbalum can you please take a look on @shmuelk's feedback and work towards setting the e2e to work like they do in llm-d scheduler? (or the new name llm-d router).

Adds end-to-end tests that deploy a complete stack on a Kind cluster: Envoy proxy (v1.33, FULL_DUPLEX_STREAMED ext_proc), Payload Processor, Llama and DeepSeek model-server simulators, and adapter ConfigMaps. Kubernetes manifests live under deploy/ following the llm-d-router pattern: shared components (deploy/components/) and environment-specific infrastructure (deploy/environments/dev/e2e-infra/). Test code references these manifests via relative paths with ${VAR} substitution. Tests cover base-model routing, LoRA adapter resolution, streaming requests, and ipp_* metrics exposure. Signed-off-by: Asaad Balum <asaad.balum@gmail.com>

github-actions · 2026-05-17T07:12:24Z

⚠️ Large PR detected

Your PR is large. Please consider breaking it into multiple PRs.

The do-not-merge/hold label has been added and can be removed by the reviewers based on their judgement.

nirrozenbaum · 2026-05-18T21:18:43Z

cc for another pair of eyes: @noyitz

asaadbalum force-pushed the feat/issue-14-add-e2e-tests branch 3 times, most recently from 9da84d3 to 1b2b395 Compare May 4, 2026 11:54

aradhalevy reviewed May 5, 2026

View reviewed changes

asaadbalum force-pushed the feat/issue-14-add-e2e-tests branch from 1b2b395 to 9de9955 Compare May 6, 2026 07:57

github-actions Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels May 6, 2026

nirrozenbaum mentioned this pull request May 6, 2026

add size label automatically on incoming PRs llm-d/llm-d-router#996

Closed

asaadbalum requested a review from aradhalevy May 6, 2026 08:08

aradhalevy approved these changes May 6, 2026

View reviewed changes

nirrozenbaum reviewed May 6, 2026

View reviewed changes

asaadbalum force-pushed the feat/issue-14-add-e2e-tests branch from 9de9955 to 3c561d1 Compare May 11, 2026 07:13

asaadbalum requested a review from nirrozenbaum May 11, 2026 07:17

asaadbalum force-pushed the feat/issue-14-add-e2e-tests branch from 3c561d1 to 2a0668c Compare May 17, 2026 07:12

		@@ -0,0 +1,164 @@
		# Llama model server simulator
		apiVersion: apps/v1

Conversation

asaadbalum commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Manifest structure

New files

Modified files

Test plan

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

aradhalevy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 6, 2026

Uh oh!

aradhalevy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented May 11, 2026

Uh oh!

nirrozenbaum commented May 13, 2026

Uh oh!

shmuelk commented May 13, 2026

Uh oh!

nirrozenbaum commented May 13, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

nirrozenbaum commented May 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

asaadbalum commented May 4, 2026 •

edited

Loading