Skip to content

ci: shard + cache for faster runs#47

Open
Anmol1696 wants to merge 3 commits into
mainfrom
ci/speedup
Open

ci: shard + cache for faster runs#47
Anmol1696 wants to merge 3 commits into
mainfrom
ci/speedup

Conversation

@Anmol1696
Copy link
Copy Markdown
Contributor

Summary

Reduces CI wall-clock time and runner cost across three workflows by eliminating duplicate work, parallelising sequential steps, and skipping CI on non-code changes.

What changed and why

1. Remove redundant fn-init-e2e job (test.yaml) — saves 1 runner per trigger

The fn-init-e2e job ran pnpm exec jest tests/integration/fn-init.test.ts — exactly the same test file that the integration job already runs as part of pnpm test:integration (--testPathPatterns='tests/integration'). It also duplicated the six-package toolkit build that the toolkit job performs. Every CI trigger was spinning up a fourth parallel runner to do no new work.

  • Measured timing (runs 26075627160, 26071351252): fn-init-e2e consistently completed in 44–52 s.
  • Expected saving: ~44–52 s of parallel compute eliminated per trigger; wall-clock unchanged (other jobs are the bottleneck).

2. Split Build & Lint into parallel build + lint jobs (ci.yaml) — saves ~7–10 s wall-clock

Previously a single sequential job ran: generate → install → build (22 s)lint (~8 s). Lint was blocked behind the full build even though it doesn't need build artifacts.

Splitting into two independent parallel jobs each doing their own generate + install means the wall-clock is now max(build, lint) ≈ 46 s instead of build + lint ≈ 53 s.

  • Measured timing (step timestamps from run 26071351252): pnpm build step took 22.3 s; pnpm lint step is independent.
  • Expected saving: ~7–10 s off the CI workflow's wall-clock per trigger.

3. paths-ignore on ci.yaml, test.yaml, docker.yaml — full skip on docs/k8s-only PRs

All three workflows previously ran on every PR regardless of what changed. Changes to docs/**, k8s/**, or markdown files don't affect build, lint, unit tests, integration tests, toolkit tests, or Docker images.

  • ci.yaml + test.yaml: ignore docs/**, k8s/**, **/*.md.
  • docker.yaml: additionally ignores tests/** (changing tests doesn't affect what images are built). Note: docker.yaml already rebuilds only images whose source changed via matrix; this filter handles the "nothing to rebuild" case.
  • test-k8s-deployment.yaml already had fine-grained paths: filters — untouched.
  • Expected saving: full skip of ~50 s build + ~52 s tests + ~2 min Docker on docs/k8s-only PRs.

4. --frozen-lockfile consistency (test.yaml)

ci.yaml already used pnpm install --frozen-lockfile; test.yaml did not. Added the flag to all three pnpm install calls in test.yaml so CI can't silently succeed with a stale lockfile.

What was NOT changed

  • No product code, test code, or package versions touched.
  • test-k8s-deployment.yaml left intact — already has path filters and per-function matrix parallelism.
  • docker.yaml build matrix and GHA layer cache (cache-from: type=gha) left intact.
  • actions/setup-node@v4 pin in docker.yaml preserved (intentional — comment explains v5 breaks when pnpm store is absent).

Test plan

  • Verify CI and Tests workflows trigger on a code-only PR and all jobs pass.
  • Verify CI and Tests workflows are skipped on a docs-only PR (e.g., a README.md change).
  • Confirm tests/integration/fn-init.test.ts still runs (covered by integration job's pnpm test:integration).
  • Check that build and lint jobs in CI run concurrently, not sequentially.

🤖 Generated with Claude Code

Anmol1696 and others added 3 commits May 20, 2026 08:03
tests/integration/fn-init.test.ts is already executed by the
integration job via `pnpm test:integration` (--testPathPatterns
'tests/integration'). fn-init-e2e ran only that one file after
doing the same toolkit build as the toolkit job — pure duplicate
work. Removing it saves one full parallel runner (~44-52 s compute)
per CI trigger.

Also adds --frozen-lockfile to every pnpm install in this workflow
to match ci.yaml and prevent non-reproducible installs on CI.

Measured: fn-init-e2e consistently completes in 44-52 s across the
last 10 runs (same wall-clock as the other parallel jobs). The
integration job already covers the test.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously the CI workflow had a single sequential job that ran
generate → install (22 s) → build (22 s) → lint (~8 s) for a
total of ~50-55 s wall-clock.

Splitting into parallel `build` and `lint` jobs lets them both do
generate + install once on their own runner and then run their step
concurrently. The wall-clock is now max(build, lint) ≈ 46 s instead
of build + lint ≈ 53 s — roughly 7 s saved per CI trigger.

Measured from recent runs: `pnpm build` step ~22 s, `pnpm lint`
step ~8 s; sequential overhead was therefore the full lint time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ci.yaml and test.yaml: ignore docs/**, k8s/**, and **/*.md changes.
Build, lint, and unit/integration/toolkit tests don't need to run
when only documentation or k8s manifests are updated. K8s changes
are already gated by test-k8s-deployment.yaml with its own path
filters.

docker.yaml: additionally ignores tests/** — changing tests doesn't
affect what Docker images are built.

Expected savings: full CI skip (~50 s build + ~50 s tests + ~2 min
Docker) on docs-only or k8s-manifest-only PRs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 20, 2026 08:05
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes GitHub Actions CI execution by skipping workflows on non-code-only changes, removing redundant test execution, and parallelizing build vs lint to reduce wall-clock time and runner usage.

Changes:

  • Add paths-ignore filters to ci.yaml, test.yaml, and docker.yaml to skip CI runs for docs/k8s/markdown-only changes (and tests-only changes for Docker publishing).
  • Split the CI “Build & Lint” job into separate parallel build and lint jobs.
  • Remove the redundant fn-init-e2e job and enforce pnpm install --frozen-lockfile consistently in test.yaml.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
.github/workflows/test.yaml Adds path-based skipping, enforces frozen lockfile installs, and removes duplicate fn init e2e job.
.github/workflows/docker.yaml Adds path-based skipping (including tests/**) to avoid unnecessary image builds on non-image-affecting changes.
.github/workflows/ci.yaml Adds path-based skipping and splits build/lint into parallel jobs for faster CI wall-clock.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants