-
Notifications
You must be signed in to change notification settings - Fork 608
test(e2e): regression suite + form-state refactor (fixes #3293, #3342, #3344, #3370) #3391
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
moonming
wants to merge
3
commits into
master
Choose a base branch
from
feat/e2e-regression-suite-and-fixes
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| /** | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| // Bulk D-01: routes list page must render correctly with 100 rows. | ||
| // Data is seeded directly via the Admin API to keep the bulk-render | ||
| // assertion decoupled from the add-form behavior. | ||
|
|
||
| import { routesPom } from '@e2e/pom/routes'; | ||
| import { | ||
| bulkCreateRoutes, | ||
| bulkDeleteRoutesByPrefix, | ||
| } from '@e2e/utils/bulk'; | ||
| import { test } from '@e2e/utils/test'; | ||
| import { expect } from '@playwright/test'; | ||
|
|
||
| const PREFIX = 'bulk100'; | ||
| const COUNT = 100; | ||
|
|
||
| test.describe.configure({ mode: 'serial', timeout: 120_000 }); | ||
|
|
||
| test.beforeAll(async () => { | ||
| await bulkDeleteRoutesByPrefix(PREFIX); | ||
| await bulkCreateRoutes({ count: COUNT, prefix: PREFIX }); | ||
| }); | ||
|
|
||
| test.afterAll(async () => { | ||
| await bulkDeleteRoutesByPrefix(PREFIX); | ||
| }); | ||
|
|
||
| test('routes list shows correct total and renders default page', async ({ | ||
| page, | ||
| }) => { | ||
| const t0 = Date.now(); | ||
| await routesPom.toIndex(page); | ||
| await routesPom.isIndexPage(page); | ||
|
|
||
| // Routes are listed in server-defined order (not insertion order), so | ||
| // bulk100-0 isn't guaranteed to be on page 1. Just require *some* | ||
| // seeded row to render. | ||
| const someSeededRow = page | ||
| .getByRole('row', { name: new RegExp(`${PREFIX}-\\d+`) }) | ||
| .first(); | ||
| await expect(someSeededRow).toBeVisible({ timeout: 5000 }); | ||
| const renderMs = Date.now() - t0; | ||
| test | ||
| .info() | ||
| .annotations.push({ type: 'perf', description: `LCP-ish: ${renderMs}ms` }); | ||
| expect(renderMs, 'list page should render in < 5s with 100 rows').toBeLessThan( | ||
| 5000 | ||
| ); | ||
| }); | ||
|
|
||
| test('pagination shows ≥ 10 pages at default page_size=10', async ({ | ||
| page, | ||
| }) => { | ||
| await routesPom.toIndex(page); | ||
| await routesPom.isIndexPage(page); | ||
|
|
||
| // page=10 must be reachable (100 rows / 10 per page = 10 pages). | ||
| const page10 = page.getByRole('listitem', { name: '10' }); | ||
| await expect(page10).toBeVisible(); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| /** | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| /* eslint-disable playwright/no-wait-for-timeout -- regression test stabilization */ | ||
|
|
||
| // Bulk D-03: with 1000 routes the list page must: | ||
| // - render the first page within a reasonable budget | ||
| // - allow URL-driven jump to a far page (e.g. ?page=50) | ||
| // - not flood the Admin API with per-row requests | ||
| // | ||
| // Marked as `bulk` — exclude from default CI runs. | ||
|
|
||
| import { routesPom } from '@e2e/pom/routes'; | ||
| import { | ||
| bulkCreateRoutes, | ||
| bulkDeleteRoutesByPrefix, | ||
| } from '@e2e/utils/bulk'; | ||
| import { test } from '@e2e/utils/test'; | ||
| import { expect } from '@playwright/test'; | ||
|
|
||
| const PREFIX = 'bulk1k'; | ||
| const COUNT = 1000; | ||
|
|
||
| test.describe.configure({ mode: 'serial', timeout: 300_000 }); | ||
|
|
||
| test.beforeAll(async () => { | ||
| await bulkDeleteRoutesByPrefix(PREFIX); | ||
| await bulkCreateRoutes({ count: COUNT, prefix: PREFIX }); | ||
| }); | ||
|
|
||
| test.afterAll(async () => { | ||
| await bulkDeleteRoutesByPrefix(PREFIX); | ||
| }); | ||
|
|
||
| test('first page renders within 5s with 1000 rows in etcd', async ({ | ||
| page, | ||
| }) => { | ||
| const t0 = Date.now(); | ||
| await routesPom.toIndex(page); | ||
| await routesPom.isIndexPage(page); | ||
|
|
||
| await expect( | ||
| page | ||
| .getByRole('row', { name: new RegExp(`${PREFIX}-\\d+`) }) | ||
| .first() | ||
| ).toBeVisible({ timeout: 8000 }); | ||
| const ms = Date.now() - t0; | ||
| test | ||
| .info() | ||
| .annotations.push({ type: 'perf', description: `first-page render: ${ms}ms` }); | ||
| expect(ms).toBeLessThan(8000); | ||
| }); | ||
|
|
||
| test('admin API call volume on list page load is bounded (≤ 5 GETs)', async ({ | ||
| page, | ||
| }) => { | ||
| const adminCalls: string[] = []; | ||
| page.on('request', (req) => { | ||
| if (req.method() === 'GET' && req.url().includes('/apisix/admin/routes')) { | ||
| adminCalls.push(req.url()); | ||
| } | ||
| }); | ||
|
|
||
| await routesPom.toIndex(page); | ||
| await routesPom.isIndexPage(page); | ||
| await expect( | ||
| page | ||
| .getByRole('row', { name: new RegExp(`${PREFIX}-\\d+`) }) | ||
| .first() | ||
| ).toBeVisible({ timeout: 8000 }); | ||
| // Settle for any deferred queries. | ||
| await page.waitForTimeout(800); | ||
|
|
||
| test.info().annotations.push({ | ||
| type: 'perf', | ||
| description: `admin /routes GETs: ${adminCalls.length}`, | ||
| }); | ||
| expect( | ||
| adminCalls.length, | ||
| 'list page should not issue per-row GETs (≤ 5 total)' | ||
| ).toBeLessThanOrEqual(5); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| /** | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| /* eslint-disable playwright/no-wait-for-timeout -- regression test stabilization */ | ||
|
|
||
| // Edge: typing syntactically-invalid JSON in the plugin config editor must | ||
| // either prevent submission or surface a visible error — never round-trip a | ||
| // broken config to the Admin API. | ||
|
|
||
| import { routesPom } from '@e2e/pom/routes'; | ||
| import { e2eReq } from '@e2e/utils/req'; | ||
| import { test } from '@e2e/utils/test'; | ||
| import { | ||
| uiFillMonacoEditor, | ||
| uiGetMonacoEditor, | ||
| } from '@e2e/utils/ui'; | ||
| import { expect } from '@playwright/test'; | ||
|
|
||
| import { deleteAllRoutes } from '@/apis/routes'; | ||
|
|
||
| const pluginName = 'cors'; | ||
| const invalidJson = '{ "allow_origins": "*", '; // missing closing brace + trailing comma | ||
|
|
||
| test.beforeAll(async () => { | ||
| await deleteAllRoutes(e2eReq); | ||
| }); | ||
|
|
||
| test.afterAll(async () => { | ||
| await deleteAllRoutes(e2eReq); | ||
| }); | ||
|
|
||
| test('invalid JSON in plugin editor cannot be saved', async ({ page }) => { | ||
| await routesPom.toAdd(page); | ||
| await routesPom.isAddPage(page); | ||
|
|
||
| await page.getByRole('button', { name: 'Select Plugins' }).click(); | ||
| const selectDialog = page.getByRole('dialog', { name: 'Select Plugins' }); | ||
| await selectDialog.getByPlaceholder('Search').fill(pluginName); | ||
| await selectDialog | ||
| .getByTestId(`plugin-${pluginName}`) | ||
| .getByRole('button', { name: 'Add' }) | ||
| .click(); | ||
|
|
||
| const addPluginDialog = page.getByRole('dialog', { name: 'Add Plugin' }); | ||
| const editor = await uiGetMonacoEditor(page, addPluginDialog, false); | ||
| await uiFillMonacoEditor(page, editor, invalidJson); | ||
|
|
||
| await addPluginDialog.getByRole('button', { name: 'Add' }).click(); | ||
|
|
||
| // The dialog must remain open OR an error must surface — what we forbid | ||
| // is silently accepting invalid JSON and closing the dialog as if it | ||
| // were valid. | ||
| await page.waitForTimeout(1000); | ||
| const dialogStillOpen = await addPluginDialog | ||
| .isVisible() | ||
| .catch(() => false); | ||
| const errorVisible = await page | ||
| .getByText(/(invalid|json|parse|syntax)/i) | ||
| .first() | ||
| .isVisible() | ||
| .catch(() => false); | ||
|
|
||
| expect( | ||
| dialogStillOpen || errorVisible, | ||
| 'invalid JSON must NOT silently close the editor as if accepted' | ||
| ).toBe(true); | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.