ci: shard + cache for faster runs#47
Open
Anmol1696 wants to merge 3 commits into
Open
Conversation
tests/integration/fn-init.test.ts is already executed by the integration job via `pnpm test:integration` (--testPathPatterns 'tests/integration'). fn-init-e2e ran only that one file after doing the same toolkit build as the toolkit job — pure duplicate work. Removing it saves one full parallel runner (~44-52 s compute) per CI trigger. Also adds --frozen-lockfile to every pnpm install in this workflow to match ci.yaml and prevent non-reproducible installs on CI. Measured: fn-init-e2e consistently completes in 44-52 s across the last 10 runs (same wall-clock as the other parallel jobs). The integration job already covers the test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously the CI workflow had a single sequential job that ran generate → install (22 s) → build (22 s) → lint (~8 s) for a total of ~50-55 s wall-clock. Splitting into parallel `build` and `lint` jobs lets them both do generate + install once on their own runner and then run their step concurrently. The wall-clock is now max(build, lint) ≈ 46 s instead of build + lint ≈ 53 s — roughly 7 s saved per CI trigger. Measured from recent runs: `pnpm build` step ~22 s, `pnpm lint` step ~8 s; sequential overhead was therefore the full lint time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ci.yaml and test.yaml: ignore docs/**, k8s/**, and **/*.md changes. Build, lint, and unit/integration/toolkit tests don't need to run when only documentation or k8s manifests are updated. K8s changes are already gated by test-k8s-deployment.yaml with its own path filters. docker.yaml: additionally ignores tests/** — changing tests doesn't affect what Docker images are built. Expected savings: full CI skip (~50 s build + ~50 s tests + ~2 min Docker) on docs-only or k8s-manifest-only PRs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR optimizes GitHub Actions CI execution by skipping workflows on non-code-only changes, removing redundant test execution, and parallelizing build vs lint to reduce wall-clock time and runner usage.
Changes:
- Add
paths-ignorefilters toci.yaml,test.yaml, anddocker.yamlto skip CI runs for docs/k8s/markdown-only changes (and tests-only changes for Docker publishing). - Split the CI “Build & Lint” job into separate parallel
buildandlintjobs. - Remove the redundant
fn-init-e2ejob and enforcepnpm install --frozen-lockfileconsistently intest.yaml.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| .github/workflows/test.yaml | Adds path-based skipping, enforces frozen lockfile installs, and removes duplicate fn init e2e job. |
| .github/workflows/docker.yaml | Adds path-based skipping (including tests/**) to avoid unnecessary image builds on non-image-affecting changes. |
| .github/workflows/ci.yaml | Adds path-based skipping and splits build/lint into parallel jobs for faster CI wall-clock. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reduces CI wall-clock time and runner cost across three workflows by eliminating duplicate work, parallelising sequential steps, and skipping CI on non-code changes.
What changed and why
1. Remove redundant
fn-init-e2ejob (test.yaml) — saves 1 runner per triggerThe
fn-init-e2ejob ranpnpm exec jest tests/integration/fn-init.test.ts— exactly the same test file that theintegrationjob already runs as part ofpnpm test:integration(--testPathPatterns='tests/integration'). It also duplicated the six-package toolkit build that thetoolkitjob performs. Every CI trigger was spinning up a fourth parallel runner to do no new work.fn-init-e2econsistently completed in 44–52 s.2. Split
Build & Lintinto parallelbuild+lintjobs (ci.yaml) — saves ~7–10 s wall-clockPreviously a single sequential job ran: generate → install → build (22 s) → lint (~8 s). Lint was blocked behind the full build even though it doesn't need build artifacts.
Splitting into two independent parallel jobs each doing their own generate + install means the wall-clock is now
max(build, lint) ≈ 46 sinstead ofbuild + lint ≈ 53 s.pnpm buildstep took 22.3 s;pnpm lintstep is independent.3.
paths-ignoreonci.yaml,test.yaml,docker.yaml— full skip on docs/k8s-only PRsAll three workflows previously ran on every PR regardless of what changed. Changes to
docs/**,k8s/**, or markdown files don't affect build, lint, unit tests, integration tests, toolkit tests, or Docker images.ci.yaml+test.yaml: ignoredocs/**,k8s/**,**/*.md.docker.yaml: additionally ignorestests/**(changing tests doesn't affect what images are built). Note:docker.yamlalready rebuilds only images whose source changed via matrix; this filter handles the "nothing to rebuild" case.test-k8s-deployment.yamlalready had fine-grainedpaths:filters — untouched.4.
--frozen-lockfileconsistency (test.yaml)ci.yamlalready usedpnpm install --frozen-lockfile;test.yamldid not. Added the flag to all threepnpm installcalls intest.yamlso CI can't silently succeed with a stale lockfile.What was NOT changed
test-k8s-deployment.yamlleft intact — already has path filters and per-function matrix parallelism.docker.yamlbuild matrix and GHA layer cache (cache-from: type=gha) left intact.actions/setup-node@v4pin indocker.yamlpreserved (intentional — comment explains v5 breaks when pnpm store is absent).Test plan
README.mdchange).tests/integration/fn-init.test.tsstill runs (covered byintegrationjob'spnpm test:integration).buildandlintjobs in CI run concurrently, not sequentially.🤖 Generated with Claude Code