Skip to content

Improve pr builder around playwright tests and product build - #4339

Merged
ThaminduDilshan merged 1 commit into
thunder-id:mainfrom
ThaminduDilshan:thamindu-fix-2
Jul 25, 2026
Merged

Improve pr builder around playwright tests and product build#4339
ThaminduDilshan merged 1 commit into
thunder-id:mainfrom
ThaminduDilshan:thamindu-fix-2

Conversation

@ThaminduDilshan

@ThaminduDilshan ThaminduDilshan commented Jul 24, 2026

Copy link
Copy Markdown
Member

Purpose

This pull request makes several significant improvements to the GitHub Actions CI workflows and Playwright E2E test configuration. The main goals are to optimize CI performance, improve reliability (especially for Playwright tests), and simplify Playwright browser setup and caching. The most important changes are grouped below.

  • A new test-backend-unit job is added to run backend unit tests in isolation, and allow product build to finish early to unblock several other jobs that depends on it.
  • E2E and frontend test jobs now run inside the official Playwright container (mcr.microsoft.com/playwright:v1.60.0-noble), with appropriate environment and shell settings for improved reliability and speed. Native build tools and utilities are installed as needed.
  • The custom .github/actions/setup-playwright action and the dedicated warm-playwright-cache.yml workflow have been removed, in favor of using the official Playwright Docker image and its preinstalled browsers for all E2E and frontend test jobs.
  • Node and pnpm setup in container jobs now uses corepack (shipped with the Playwright image), reducing setup time and noise.
  • The default number of Playwright workers is increased from 1 to 6 in both the workflow and playwright.config.ts, matching GitHub's standard runner capacity for faster test execution.

Attempt 1: 16m 58s

Screenshot 2026-07-25 at 1 54 57 AM

Attempt 2: 15m 0s
Screenshot 2026-07-25 at 2 11 44 AM

Related Issues

  • N/A

Related PRs

  • N/A

Checklist

  • Followed the contribution guidelines.
  • Manual test round performed and verified.
  • Documentation provided. (Add links if there are any)
    • Ran Vale and fixed all errors and warnings
  • Tests provided. (Add links if there are any)
    • Unit Tests
    • Integration Tests
  • Breaking changes. (Fill if applicable)
    • Breaking changes section filled.
    • breaking change label added.

Security checks

  • Followed secure coding standards in WSO2 Secure Coding Guidelines
  • Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets.

Summary by CodeRabbit

Summary by CodeRabbit

  • Bug Fixes
    • Improved end-to-end test reliability by running shared-state specs sequentially across browsers.
  • Tests
    • Increased default end-to-end parallelism from 1 to 6 workers.
    • Updated CI to run Playwright-based tests in the official Playwright container for more consistent browser setup and more robust test server lifecycle handling.
    • Tightened patch checks to require backend unit test coverage.
  • Chores
    • Removed the dedicated Playwright cache warm-up workflow to simplify CI setup.

@ThaminduDilshan ThaminduDilshan added skip-changelog Skip generating changelog for a particular PR trigger-pr-builder Add when the PR is ready for CI; starts the PR Builder for this and all later pushes labels Jul 24, 2026
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 06bb209d-7dc0-440c-9969-6c6b6a53885e

📥 Commits

Reviewing files that changed from the base of the PR and between aa42a44 and 078ba20.

📒 Files selected for processing (5)
  • .github/actions/setup-playwright/action.yml
  • .github/workflows/pr-builder.yml
  • .github/workflows/warm-playwright-cache.yml
  • tests/e2e/playwright.config.ts
  • turbo.json
💤 Files with no reviewable changes (2)
  • .github/workflows/warm-playwright-cache.yml
  • .github/actions/setup-playwright/action.yml
🚧 Files skipped from review as they are similar to previous changes (3)
  • turbo.json
  • tests/e2e/playwright.config.ts
  • .github/workflows/pr-builder.yml

📝 Walkthrough

Walkthrough

Backend coverage and unit-test gating now use build scripts. Frontend and E2E jobs run in Playwright containers, with detached application processes and six workers. Shared-state E2E specs run through ordered browser projects, and obsolete Playwright setup workflows were removed.

Changes

CI test pipeline

Layer / File(s) Summary
Backend coverage and unit-test gating
.github/workflows/pr-builder.yml
Backend coverage and unit tests use build.sh; patch coverage requires backend unit-test success, and integration tests no longer download unit coverage artifacts.
Playwright container test jobs
.github/workflows/pr-builder.yml, .github/actions/setup-playwright/action.yml, .github/workflows/warm-playwright-cache.yml
Frontend package and E2E jobs use Playwright containers with container tooling and image-provided browsers; obsolete Playwright setup and cache workflows were removed.
E2E process lifecycle and worker defaults
.github/workflows/pr-builder.yml
Server and sample-app processes use detached sessions across workflow steps, and the default worker count is six.
Serialized E2E project execution
tests/e2e/playwright.config.ts
Shared-state specs are excluded from parallel browser projects and run sequentially across Chromium, Firefox, and WebKit.
Playwright environment propagation
turbo.json
PLAYWRIGHT_BROWSERS_PATH is passed through Turbo test tasks.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CI
  participant PlaywrightContainer
  participant Applications
  participant PlaywrightTests
  CI->>PlaywrightContainer: Start frontend and E2E jobs
  PlaywrightContainer->>Applications: Start detached server and sample app
  PlaywrightContainer->>PlaywrightTests: Run tests with six workers
  PlaywrightTests->>Applications: Execute browser scenarios
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main CI and Playwright-related changes in the pull request.
Description check ✅ Passed The description covers purpose, related items, checklist, and security checks; only the Approach section is missing.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

tests/e2e/playwright.config.ts

Parsing error: error TS5012: Cannot read file '/tsconfig.json': ENOENT: no such file or directory, open '/tsconfig.json'.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ThaminduDilshan ThaminduDilshan added trigger-pr-builder Add when the PR is ready for CI; starts the PR Builder for this and all later pushes and removed trigger-pr-builder Add when the PR is ready for CI; starts the PR Builder for this and all later pushes labels Jul 24, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (1)
.github/workflows/pr-builder.yml (1)

739-741: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Align the Playwright E2E container image with the locked @playwright/test version.

tests/e2e locks @playwright/test to 1.60.0 via the frontend catalog, but .github/workflows/pr-builder.yml installs Playwright packages from the registry inside the mcr.microsoft.com/playwright:v1.60.0-noble container. Use a catalog-derived lock value or document why this image is intentionally decoupled from the package lock to keep E2E results trustworthy.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/pr-builder.yml around lines 739 - 741, Align the
Playwright E2E container image in the workflow with the catalog-locked
`@playwright/test` 1.60.0 version by deriving the image tag from the same lock
source where workflow interpolation supports it; otherwise document the
intentional decoupling and its rationale near the container configuration.
Preserve the existing noble image variant and root-user option.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/pr-builder.yml:
- Around line 379-380: Update the “📥 Checkout Code” actions/checkout step to
set persist-credentials to false, matching the other checkout steps in the
workflow while leaving its existing pinned action version unchanged.
- Around line 764-765: Update the “📥 Checkout Code” step using actions/checkout
to set persist-credentials to false, ensuring subsequent package installation
and system commands cannot access persisted checkout credentials.
- Around line 263-264: Update the actions/checkout step in the workflow to set
persist-credentials to false, preventing the GitHub token from being retained in
.git/config for the remainder of the job.

---

Nitpick comments:
In @.github/workflows/pr-builder.yml:
- Around line 739-741: Align the Playwright E2E container image in the workflow
with the catalog-locked `@playwright/test` 1.60.0 version by deriving the image
tag from the same lock source where workflow interpolation supports it;
otherwise document the intentional decoupling and its rationale near the
container configuration. Preserve the existing noble image variant and root-user
option.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 455b8f33-03a6-4958-bbbd-bd60daf0a66f

📥 Commits

Reviewing files that changed from the base of the PR and between 0dc6476 and aa42a44.

📒 Files selected for processing (5)
  • .github/actions/setup-playwright/action.yml
  • .github/workflows/pr-builder.yml
  • .github/workflows/warm-playwright-cache.yml
  • tests/e2e/playwright.config.ts
  • turbo.json
💤 Files with no reviewable changes (2)
  • .github/workflows/warm-playwright-cache.yml
  • .github/actions/setup-playwright/action.yml

Comment thread .github/workflows/pr-builder.yml
Comment thread .github/workflows/pr-builder.yml
Comment thread .github/workflows/pr-builder.yml
@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@coderabbitai coderabbitai Bot mentioned this pull request Jul 24, 2026
12 tasks
@ThaminduDilshan
ThaminduDilshan added this pull request to the merge queue Jul 25, 2026
Merged via the queue into thunder-id:main with commit 746a38a Jul 25, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Skip generating changelog for a particular PR trigger-pr-builder Add when the PR is ready for CI; starts the PR Builder for this and all later pushes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants