Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
883b916
feat(e2e): harden browser lifecycle, smoke gate, and naming
schultzp2020 Jun 23, 2026
f017ad0
feat(e2e): complete hardening — POMs, fixtures, naming, a11y gate
schultzp2020 Jun 23, 2026
7dfd006
feat(e2e): replace fixed waits with pollUntil and project timeouts
schultzp2020 Jun 23, 2026
46b0ce3
chore(e2e): apply oxfmt after rebase conflict resolution
schultzp2020 Jun 24, 2026
c92a8fd
feat(e2e): unify browser fixtures and auth provider harness
schultzp2020 Jun 24, 2026
5334255
refactor(e2e): collapse page-objects stack and deepen core POMs
schultzp2020 Jun 24, 2026
a896d3c
refactor(e2e): move residual spec locators into POMs
schultzp2020 Jun 24, 2026
4e936fd
test(e2e): add Vitest unit tests for poll-until helpers
schultzp2020 Jun 24, 2026
37dbc4a
chore(e2e): harden E2E vs unit test separation and lint plugins
schultzp2020 Jun 24, 2026
f9c7887
chore(e2e): scope oxlint test plugins to spec and test globs
schultzp2020 Jun 24, 2026
ff18d95
chore(e2e): reformat after hardening rebase onto Oxfmt defaults
schultzp2020 Jun 24, 2026
44ddbb2
fix(e2e): validate audit logs from raw JSON without Log defaults
schultzp2020 Jun 24, 2026
bb1504d
fix(e2e): restore import paths after rebase onto main
schultzp2020 Jun 25, 2026
39b7d10
fix(e2e): ignore TLS errors in global setup health check
schultzp2020 Jun 25, 2026
d630e97
fix(e2e): worker fixture video paths and known shell a11y exclusions
schultzp2020 Jun 25, 2026
9b0e66c
refactor(e2e): consolidate auth and runtime test seams
schultzp2020 Jun 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/e2e-tests-lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ jobs:
working-directory: ./e2e-tests
run: yarn test:list

- name: Run unit tests
working-directory: ./e2e-tests
run: yarn test:unit

- name: Run ShellCheck
working-directory: ./e2e-tests
run: yarn shellcheck
Expand Down
12 changes: 11 additions & 1 deletion docs/e2e-tests/CONTRIBUTING.MD
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,24 @@ These principles are valid for new contributions. Some parts of the codebase may

We follow Playwright best practices, including the use of fixtures. Adhering to these practices ensures that our tests are reliable, efficient, and maintainable. Please refer to the [Playwright Best Practices](https://playwright.dev/docs/best-practices) and [Fixtures](https://playwright.dev/docs/test-fixtures) documentation for guidance.

- Prefer the shared fixtures from `@support/coverage/test` instead of re-implementing login/setup in each spec:
- Use `guestPage` for ordinary specs that need a fresh guest login per test.
- Use `rhdhGuestPage` only when a describe block intentionally shares one browser context/page across tests.
- If you use `rhdhPage`, `rhdhGuestPage`, `rhdhContext`, or other worker-scoped fixtures across multiple tests in one file, make the suite serial so state sharing is explicit.

3. **Avoid Using `uiHelper` in Spec Files**

- The `uiHelper` utility should not be used directly in spec files. The reason for that is that some methods in this class are too generic and sometimes it is difficult to point what they are intended. Idellay, they shall be called from inside a POM that states what thay are looking for.
- Use `uiHelper` methods only within the Page Object Model (POM) classes.
- When working with tests that directly use `uiHelper` in spec files, refactor them to move `uiHelper` usage into POM classes.
- This ensures that all UI interactions are encapsulated within page objects, promoting cleaner and more maintainable test code.

4. **Use External Sources for Validation**
4. **Blocked Flows**

- If a flow is blocked by a known product issue, keep it out of the default `*.spec.ts` Playwright discovery path until it can run deterministically again.
- Preserve the Jira or bug reference in the filename comments and/or documentation so the test can be restored once the blocker is fixed.

5. **Use External Sources for Validation**

- **Avoid Hardcoded Data in Tests**

Expand Down
13 changes: 12 additions & 1 deletion docs/e2e-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
| `e2e-tests/playwright/e2e` | Contains all the end-to-end (E2E) test suites and test cases |
| `e2e-tests/playwright/e2e/plugins` | Contains all the dynamic plugins E2E test suites and test cases |
| `e2e-tests/playwright/utils` | Utilities for easier test development, from UI interaction tasks to network requests |
| `e2e-tests/unit/**/*.test.ts` | Vitest unit tests for shared helpers (run via `yarn test:unit`) |
| `e2e-tests/vitest.config.ts` | Vitest configuration for unit tests |
| `e2e-tests/playwright/support` | Contains helper files for Playwright, like custom commands and page objects |
| `e2e-tests/playwright-report/index.html` | HTML report of the test execution |
| `e2e-tests/test-results` | Contains video recordings of the executed test cases |
Expand All @@ -35,6 +37,8 @@ yarn playwright install chromium
## Adding a Test

To incorporate a new test case, create a file with a `.spec.ts` extension in the `e2e-tests/playwright/e2e` directory.

Unit tests for shared helpers (for example `poll-until.ts`) live in `e2e-tests/unit/` as `*.test.ts` and run with `yarn test:unit` (Vitest). E2E specs use `*.spec.ts` under `playwright/e2e/`.
The tests within a spec file can run in parallel (by default) or sequentially if using the `.serial` modifier like in [these examples](../../e2e-tests/playwright/e2e/). Note that sequential execution is considered a bad practice and is strongly discouraged.
To add or edit a test, you should adhere to the [contribution guidelines](./CONTRIBUTING.MD).

Expand Down Expand Up @@ -83,7 +87,14 @@ The currently supported environment variables are:

### Running the Tests

The Playwright command line supports many options; see them [here](https://playwright.dev/docs/test-cli). Flags like `--ui` or `--headed` are very useful when debugging. You can also specify a specific test to run:
Unit tests (Vitest) do not need a deployed RHDH instance:

```bash
cd e2e-tests
yarn test:unit
```

E2E tests (Playwright) require `BASE_URL` and a running application. The Playwright command line supports many options; see them [here](https://playwright.dev/docs/test-cli). Flags like `--ui` or `--headed` are very useful when debugging. You can also specify a specific test to run:

```bash
yarn playwright test e2e-tests/playwright/e2e/your-test-file.spec.ts
Expand Down
31 changes: 19 additions & 12 deletions e2e-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ letting CI or a local run target a subset.
| `@smoke` | Fast, high-signal check suitable to run on every PR. |
| `@ga-plugin` | Exercises a generally-available (GA) plugin. |
| `@non-ga-plugin` | Exercises a tech-preview / dev-preview (non-GA) plugin. |
| `@blocked` | Blocked by a known issue; tests are skipped with a Jira reference. |

```bash
# Run only smoke-tagged tests
Expand Down Expand Up @@ -404,18 +405,24 @@ This opens an interactive UI where you can select individual tests, watch them r

After running `local-test-setup.sh`, these variables are set:

| Variable | Description |
| --------------------------- | --------------------------------------------- |
| `BASE_URL` | URL of the deployed RHDH instance |
| `SHOWCASE_URL` | Showcase deployment URL |
| `SHOWCASE_RBAC_URL` | Showcase RBAC deployment URL |
| `K8S_CLUSTER_URL` | OpenShift API server URL |
| `K8S_CLUSTER_TOKEN` | Service account token (48-hour duration) |
| `JOB_NAME` | Selected job name |
| `IMAGE_REGISTRY` | Image registry (default: `quay.io`) |
| `IMAGE_REPO` | Image repository (fallback: `QUAY_REPO`) |
| `TAG_NAME` | Image tag |
| Plus all secrets from Vault | (exported with `-`, `.`, `/` replaced by `_`) |
| Variable | Description |
| --------------------------- | ---------------------------------------------------------------------------------------------- |
| `BASE_URL` | URL of the deployed RHDH instance (Playwright uses this as the test base URL) |
| `SHOWCASE_URL` | Legacy name for the standard RHDH deployment URL (same value as `BASE_URL` for showcase tests) |
| `SHOWCASE_RBAC_URL` | Legacy name for the RBAC deployment URL |
| `K8S_CLUSTER_URL` | OpenShift API server URL |
| `K8S_CLUSTER_TOKEN` | Service account token (48-hour duration) |
| `JOB_NAME` | Selected job name |
| `IMAGE_REGISTRY` | Image registry (default: `quay.io`) |
| `IMAGE_REPO` | Image repository (fallback: `QUAY_REPO`) |
| `TAG_NAME` | Image tag |
| Plus all secrets from Vault | (exported with `-`, `.`, `/` replaced by `_`) |

> **Note:** `BASE_URL` is the canonical Playwright base URL for the RHDH instance under test.
> `SHOWCASE_URL` and `SHOWCASE_RBAC_URL` are legacy names retained by `local-test-setup.sh` and
> deployment scripts; `local-test-setup.sh` sets `BASE_URL` from the appropriate legacy URL.
> Yarn scripts such as `yarn showcase`, `yarn showcase-rbac`, and `yarn test:stability` still use
> the `showcase` Playwright project name for historical reasons.

### Artifacts and Logs

Expand Down
186 changes: 115 additions & 71 deletions e2e-tests/oxlint.config.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,87 @@
import { defineConfig } from "oxlint";

/** POM and helper methods that perform assertions on behalf of E2E specs. */
const playwrightAssertFunctions = [
"expect",
"toPass",
"verifyHeading",
"verifyWelcomeHeading",
"verifyGuestProfile",
"verifySignInPageTitle",
"verifyProfileHeading",
"verifySignInError",
"verifyQuickAccess",
"verifyLink",
"verifyRowsInTable",
"verifyRowInTableByUniqueText",
"verifyDivHasText",
"verifyComponentInCatalog",
"verifyComponentsInCatalog",
"verifyParagraph",
"verifyText",
"verifyTextinCard",
"verifyTextInCard",
"verifyVisitedCardContent",
"verifyAboutCardIsDisplayed",
"verifyPRStatisticsRendered",
"verifyPRRows",
"verifyPRRowsPerPage",
"waitForEntityPath",
"clickPullRequestFilter",
"verifyGithubUserProfile",
"verifySignInButtonVisible",
"verifyTemplateHeading",
"verifyTableCell",
"verifyDependencyResource",
"verifySharedCardCount",
"incrementFirstCardCounter",
"waitForOpenInCatalogLink",
"verifyComponentNameVisible",
"verifyLinkHidden",
"clearSearchIfVisible",
"sortCreatedAtDescending",
"verifyFirstRowCreatedAtNotEmpty",
"openLicensedUsersCatalog",
"verifyTestPageContent",
"verifyContextOneCard",
"verifyContextTwoCard",
"verifyTemplatesHeading",
"verifyDocumentationHeading",
"verifyDocHeading",
"verifyCreateReactAppReviewTableWithGroupOwner",
"verifyDependencyGraphLabels",
"launchTemplateAndVerifyIntro",
"runHttpRequestTemplateFlow",
"inspectEntityAndVerifyYaml",
"registerExistingComponent",
"runAccessibilityTests",
"validateLog",
"validateLogEvent",
"validateRbacLogEvent",
"checkRbacResponse",
"verifyTextInSelector",
"verifyPartialTextInSelector",
"verifyTextVisible",
"verifyLanguageToggleList",
"verifyLanguageSelectShowsOptions",
"verifyLanguageOptionsList",
"verifySelectedLanguage",
"verifySignOutMenuLabel",
"verifySidebarMenuItemHidden",
"verifyLocalizedUserSettingsLabelsWithOwnership",
"verifyBuildInfoCardVisible",
"verifyBuildInfoText",
"verifyGuestSignInMethodNotListed",
"verifyInactivityLogoutMessageHidden",
"verifyRhdhMetadata",
"verifyMenuItemInSection",
"verifyLearningPathLinksOpenInNewTab",
"verifyMainHeadingVisible",
"loginAsGuest",
"restartDeployment",
"waitForTitle",
];

export default defineConfig({
plugins: ["eslint", "typescript", "unicorn", "oxc", "import", "node", "promise"],
categories: {
Expand All @@ -11,7 +93,7 @@ export default defineConfig({
typeAware: true,
typeCheck: true,
},
jsPlugins: ["eslint-plugin-playwright", "eslint-plugin-check-file"],
jsPlugins: ["eslint-plugin-check-file"],
ignorePatterns: [
"node_modules/**",
"playwright-report/**",
Expand Down Expand Up @@ -44,48 +126,23 @@ export default defineConfig({
"**": "KEBAB_CASE",
},
],
"playwright/no-wait-for-timeout": "error",
"playwright/no-force-option": "error",
"playwright/expect-expect": "error",
"playwright/valid-expect": "error",
"playwright/prefer-native-locators": "error",
"playwright/no-raw-locators": [
"error",
{
allowed: [],
},
],
"playwright/no-skipped-test": [
"error",
{
allowConditional: true,
},
],
},
overrides: [
{
// Auth-provider specs deploy RHDH in beforeAll and use async Playwright hooks.
// strict-void-return and no-misused-promises produce false positives on those
// describe/beforeAll callbacks without improving test safety.
files: ["playwright/e2e/auth-providers/**/*.spec.ts"],
rules: {
"typescript/strict-void-return": "off",
"typescript/no-misused-promises": "off",
},
},
{
// Spec files orchestrate multi-step E2E flows; length limits target production
// code readability, not test scenarios that must stay in one file for clarity.
files: ["**/*.spec.ts", "**/*.test.ts"],
rules: {
"eslint/max-lines": "off",
"eslint/max-lines-per-function": "off",
},
},
{
// Shared infrastructure (utils, support, data, e2e helpers) is split into
// modules but still contains cohesive orchestration (kube waits, deployment
// setup, log parsing). Complexity limits would force artificial fragmentation.
files: [
"playwright/utils/**/*.ts",
"playwright/support/**/*.ts",
Expand All @@ -99,68 +156,55 @@ export default defineConfig({
},
},
{
// Facade modules aggregate many submodules by design (e.g. KubeClient re-exports,
// rhdh-deployment orchestration, locale translation maps). A flat import count
// does not reflect coupling when each import is a focused submodule.
files: ["playwright/utils/**/*.ts", "playwright/e2e/localization/**/*.ts"],
rules: {
"import/max-dependencies": "off",
},
},
{
// valid-title / valid-describe-callback: existing suite uses legacy naming
// patterns that do not match the plugin's strict conventions.
// no-wait-for-selector: replaced with expect() and locator.waitFor() per
// hardening guidelines; rule would flag intentional migration patterns.
// expect-expect + assertFunctionNames: POM verify* helpers and loginAsGuest
// perform assertions on behalf of the spec; register them so specs are not
// forced to duplicate expect() calls after every helper invocation.
files: ["**/*.spec.ts", "**/*.test.ts", "playwright/**/*.ts"],
// E2E: *.spec.ts only. Playwright jsPlugin is not loaded for other files.
files: ["**/*.spec.ts"],
jsPlugins: ["eslint-plugin-playwright"],
rules: {
// Playwright requires object destructuring for hook/test callbacks that take
// testInfo as a second argument (e.g. async ({}, testInfo) =>). Oxlint's
// no-empty-pattern rejects {}; disable it here so lint and runtime agree.
"playwright/no-wait-for-timeout": "error",
"playwright/no-force-option": "error",
"playwright/valid-expect": "error",
"playwright/prefer-native-locators": "error",
"playwright/no-raw-locators": [
"error",
{
allowed: [],
},
],
"playwright/no-skipped-test": [
"error",
{
allowConditional: true,
},
],
"eslint/no-empty-pattern": "off",
"playwright/valid-title": "off",
"playwright/valid-describe-callback": "off",
"playwright/no-wait-for-selector": "off",
"playwright/expect-expect": [
"error",
{
assertFunctionNames: [
"expect",
"toPass",
"verifyHeading",
"verifyQuickAccess",
"verifyLink",
"verifyRowsInTable",
"verifyRowInTableByUniqueText",
"verifyDivHasText",
"verifyComponentInCatalog",
"verifyParagraph",
"verifyText",
"verifyTextinCard",
"verifyVisitedCardContent",
"verifyAboutCardIsDisplayed",
"verifyPRStatisticsRendered",
"verifyPRRows",
"verifyPRRowsPerPage",
"registerExistingComponent",
"inspectEntityAndVerifyYaml",
"runAccessibilityTests",
"validateLog",
"validateLogEvent",
"validateRbacLogEvent",
"checkRbacResponse",
"verifyTextInSelector",
"verifyPartialTextInSelector",
"loginAsGuest",
"restartDeployment",
"waitForTitle",
],
assertFunctionNames: playwrightAssertFunctions,
},
],
},
},
{
// Unit: *.test.ts only. Vitest plugin is not loaded for other files.
files: ["**/*.test.ts"],
plugins: ["vitest"],
rules: {
"vitest/expect-expect": "error",
"vitest/valid-expect": "off",
"vitest/no-conditional-in-test": "off",
"vitest/no-conditional-tests": "off",
"eslint/no-empty-pattern": "off",
},
},
],
});
6 changes: 4 additions & 2 deletions e2e-tests/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"private": true,
"type": "module",
"scripts": {
"test:stability": "playwright test --project=showcase --retries=0",
"showcase": "playwright test --project=showcase",
"showcase-rbac": "playwright test --project=showcase-rbac",
"showcase-k8s": "playwright test --project=showcase-k8s",
Expand All @@ -23,6 +24,7 @@
"lint": "oxlint .",
"lint:fix": "oxlint --fix .",
"test:list": "playwright test --list",
"test:unit": "vitest run",
"fmt": "oxfmt .",
"fmt:check": "oxfmt --check .",
"postinstall": "playwright install chromium",
Expand All @@ -49,7 +51,6 @@
"@playwright/test": "1.59.1",
"@types/js-yaml": "4.0.9",
"@types/node": "24.13.2",
"@types/node-fetch": "2.6.13",
"@types/pg": "8.20.0",
"eslint-plugin-check-file": "3.3.1",
"eslint-plugin-playwright": "2.10.4",
Expand All @@ -60,7 +61,8 @@
"oxlint": "1.71.0",
"oxlint-tsgolint": "0.23.0",
"shellcheck": "4.1.0",
"typescript": "6.0.3"
"typescript": "6.0.3",
"vitest": "^4.1.9"
},
"engines": {
"node": "24"
Expand Down
Loading
Loading