Shard console app tests across parallel jobs

[DonOmalVindula]

Purpose

Console App Tests is the longest job in the PR Builder: ~11.7 min of vitest in a single run, against ~2.7 min of fixed setup, for a ~14.5 min job. Everything else finishes well before it, so it sets the floor for how fast a PR run can be.

This splits the suite across 3 shards that run concurrently, taking the job to roughly 6.5 min and moving it off the critical path (which becomes Build Product to Playwright E2E at ~13 min).

Phase 2 of #4268.

Approach

test-frontend-console-app-shards is a 3-way matrix running vitest run --coverage --shard=N/3, with fail-fast: false so one broken shard does not mask the others.

Because each shard only sees its own tests, coverage arrives in three partial LCOV reports. test-frontend-console-app becomes a small aggregator that merges them back into a single console-coverage artifact, so upload-frontend-coverage and the patch coverage gate are untouched and keep consuming exactly what they do today. The merge itself is a new merge-lcov composite action: an awk script that sums per-line and per-function hit counts and recomputes the LF/LH/FNF/FNH totals. That follows the precedent already set by go-cover-to-lcov.awk in the patch coverage gate, which deliberately avoids third-party coverage converters.

Two details that are load-bearing:

• The aggregator keeps the job name 🧪 Console App Tests with Coverage, which is a required status check. It also runs with always() plus an explicit shard-result check rather than relying on needs alone, because a skipped required check counts as passing for branch protection: without that, a failing shard would skip the aggregator and quietly satisfy the requirement.
• Shard artifacts are named console-lcov-shard-N, with no "coverage" in the name, so the patch coverage gate's *coverage* artifact download only ever picks up the merged report and never a partial one.

Verification

The merge was validated end to end against real sharded output rather than by inspection. Running the console flows/hooks tests as --shard=1/2 and --shard=2/2 produced two genuine istanbul reports (25 files/452 lines and 19 files/450 lines); merging them yields the union of 37 files and 860 lines, and every single line's merged hit count equals the sum of its per-shard counts (0 mismatches). The merged report was then parsed with the same diff_cover==10.4.0 the gate uses, to confirm it is inside that parser's LCOV grammar.

Two parser constraints found during that check and handled: FNDA must carry exactly two fields, and FN may carry either two or three, so the merger keeps the leading FN fields verbatim and takes the function name after the last comma.

Trade-off

Runner-minutes for this job go from ~14.5 to ~20 (each shard repeats the ~2.7 min of install and frontend build). 3 shards rather than 2 is deliberate: --shard splits by file count, not duration, so an uneven split could otherwise leave one shard close to the old runtime.

Related Issues

• Refs Reduce PR Builder and merge queue turnaround time #4268

Related PRs

• Make preflight checks the root of the PR builder job graph #4370 also edits this workflow; whichever merges second needs a trivial rebase.

Checklist

• Followed the contribution guidelines.
• Manual test round performed and verified.
• Documentation provided. (Add links if there are any)
  • Ran Vale and fixed all errors and warnings
• Tests provided. (Add links if there are any)
  • Unit Tests
  • Integration Tests
• Breaking changes. (Fill if applicable)
  • Breaking changes section filled.
  • breaking change label added.

Security checks

• Followed secure coding standards in WSO2 Secure Coding Guidelines
• Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets.

Summary by CodeRabbit

• Tests
  • Frontend console tests now run in parallel across three shards, reducing CI execution time.
  • Coverage reports from all test shards are consolidated into one complete report.
  • Coverage validation now fails when any test shard fails or no coverage reports are produced.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 26f24d4f-de09-463f-a948-3098ea602d7c

📥 Commits

Reviewing files that changed from the base of the PR and between f117301 and 2a51e44.

📒 Files selected for processing (3)

• .github/actions/merge-lcov/action.yml
• .github/actions/merge-lcov/merge-lcov.awk
• .github/workflows/pr-builder.yml

---

📝 Walkthrough

Walkthrough

Console coverage tests now run in three shards. Each shard uploads an LCOV report, and a new composite action merges those reports into one file before downstream coverage processing.

Changes

Console coverage aggregation

Layer / File(s)	Summary
LCOV record aggregation .github/actions/merge-lcov/merge-lcov.awk	The AWK merger deduplicates source files, sums function and line hits across reports, recomputes coverage totals, and emits consolidated LCOV records.
Reusable LCOV merge action .github/actions/merge-lcov/action.yml	The composite action finds sorted .info files, fails when none exist, creates the output directory, runs the merger, and logs the source count.
Sharded workflow integration .github/workflows/pr-builder.yml	Console tests run across three matrix shards, upload separate LCOV artifacts, verify shard completion, download reports, and generate the consolidated console coverage file.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ShardJobs as Console shard jobs
  participant Artifacts as LCOV artifacts
  participant MergeAction as merge-lcov action
  participant AWK as merge-lcov.awk

  ShardJobs->>Artifacts: Upload shard LCOV reports
  Artifacts->>MergeAction: Download reports
  MergeAction->>AWK: Pass sorted .info tracefiles
  AWK-->>MergeAction: Write merged lcov.info
  MergeAction-->>ShardJobs: Provide consolidated coverage report

Loading

Possibly related PRs

• thunder-id/thunderid#4245: Updates the same workflow’s downstream Codecov patch-coverage handling.
• thunder-id/thunderid#4265: Coordinates Codecov processing with frontend coverage uploads in the same workflow.

Suggested reviewers: brionmario

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: sharding Console App tests across parallel jobs.
Description check	✅ Passed	The PR description follows the template with Purpose, Approach, Related Issues/PRs, Checklist, and Security checks.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!