Skip to content

[Test] UI - Models: Add E2E tests for Add Model flow#25590

Merged
yuneng-berri merged 4 commits intomainfrom
litellm_add_model_e2e_tests
Apr 14, 2026
Merged

[Test] UI - Models: Add E2E tests for Add Model flow#25590
yuneng-berri merged 4 commits intomainfrom
litellm_add_model_e2e_tests

Conversation

@yuneng-berri
Copy link
Copy Markdown
Collaborator

Summary

Adds E2E tests for the Add Model page covering test connection failure, adding specific models, adding wildcard routes, and verifying models appear in the All Models table. Added data-testid attributes to support stable test selectors.

Testing

All 4 tests pass 3/3 consecutive runs with zero flakiness.

Type

✅ Test

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 12, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 14, 2026 0:55am

Request Review

@codspeed-hq
Copy link
Copy Markdown
Contributor

codspeed-hq bot commented Apr 12, 2026

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_add_model_e2e_tests (9b74ff3) with main (8427534)

Open in CodSpeed

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 12, 2026

Greptile Summary

Adds four Playwright E2E tests for the Add Model flow (connection failure, adding a specific model, adding a wildcard route, verifying results in the All Models table) and instruments the relevant UI components with data-testid attributes to support stable selectors. All production code changes are additive data-testid annotations with no logic impact.

  • The selectProvider helper uses page: any (loses Playwright type safety) and fixed waitForTimeout delays rather than condition-based waits, which slows the suite and can mask timing races.
  • The "Test connection with bad credentials" test explicitly makes a real outbound call to Anthropic; if CI blocks egress or the provider throttles, that test will time out after 30 s.

Confidence Score: 5/5

Safe to merge — production code changes are purely additive data-testid annotations, all findings are P2 style suggestions.

All five changed files either add data-testid attributes to existing elements with no logic changes, or add new E2E tests. The only findings are P2 style suggestions (untyped page param, fixed timeouts, undocumented external network dependency). No correctness, security, or data-integrity issues were found.

ui/litellm-dashboard/e2e_tests/tests/modelsPage/addModel.spec.ts — minor type-safety and timing robustness improvements suggested.

Important Files Changed

Filename Overview
ui/litellm-dashboard/e2e_tests/tests/modelsPage/addModel.spec.ts New E2E test file covering four Add Model scenarios; minor type-safety gap (page: any) and fixed waitForTimeout delays that could slow CI; one test explicitly depends on outbound internet access to Anthropic.
ui/litellm-dashboard/src/app/(dashboard)/models-and-endpoints/components/AllModelsTab.tsx Adds data-testid="model-search-input" and data-testid="models-results-count" attributes to support stable test selectors; no logic changes.
ui/litellm-dashboard/src/components/add_model/AddModelForm.tsx Adds data-testid attributes to "Test Connect" and "Add Model" buttons; no logic changes.
ui/litellm-dashboard/src/components/add_model/litellm_model_name.tsx Adds data-testid="model-name-select" to the model multi-select; no logic changes.
ui/litellm-dashboard/src/components/add_model/model_connection_test.tsx Adds data-testid attributes to the connection success and failure text nodes; no logic changes.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Navigate to Models Page] --> B[Click 'Add Model' tab]
    B --> C[selectProvider helper\nfill + Enter + waitForTimeout]
    C --> D{Test scenario}
    D -->|Test Connection| E[Select model & enter bad key]
    E --> F[Click 'Test Connect' button]
    F --> G[LiteLLM Proxy calls\nexternal API with bad key]
    G --> H[UI shows 'Connection Test Results' modal]
    H --> I{API response}
    I -->|401 from Anthropic| J[✅ 'Connection to X failed' visible]
    D -->|Add Specific Model| K[Select claude-haiku-4-5\nenter API key]
    K --> L[Click 'Add Model' button]
    L --> M[POST /model/new]
    M --> N[✅ 'created successfully' toast]
    N --> O[Navigate to 'All Models' tab]
    O --> P[Search 'claude-haiku-4-5']
    P --> Q[✅ Results count > 0 & model in table]
    D -->|Add Wildcard Route| R[Select 'All Cohere Models Wildcard'\nenter API key]
    R --> S[Click 'Add Model' button]
    S --> T[✅ 'created successfully' toast]
    T --> U[Search 'cohere' in All Models]
    U --> V[✅ 'cohere/' appears in table]
Loading

Reviews (5): Last reviewed commit: "remove unnecessary cleanup helper" | Re-trigger Greptile

@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:21 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:22 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:22 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:34 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:35 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:35 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:50 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:51 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 12, 2026 03:51 — with GitHub Actions Inactive
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)

Added data-testid attributes to UI components to support stable test selectors.

Tests verified passing 3/3 consecutive runs with zero flakiness.
Add cleanup helper to delete models created during tests, preventing
stale data accumulation across repeated test runs.
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
The database is freshly seeded on every test run via seed.sql,
so per-test cleanup is not needed.
@yuneng-berri yuneng-berri force-pushed the litellm_add_model_e2e_tests branch from 5c80cd8 to 9b74ff3 Compare April 14, 2026 00:53
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 14, 2026 00:53 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 14, 2026 00:53 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri temporarily deployed to integration-postgres April 14, 2026 00:53 — with GitHub Actions Inactive
@yuneng-berri yuneng-berri merged commit e64d98f into main Apr 14, 2026
100 of 107 checks passed
@yuneng-berri yuneng-berri deleted the litellm_add_model_e2e_tests branch April 14, 2026 02:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants