feat: prewarm git repo from Bitrise Build Cache#249
Draft
zsolt-marta-bitrise wants to merge 7 commits into
Draft
feat: prewarm git repo from Bitrise Build Cache#249zsolt-marta-bitrise wants to merge 7 commits into
zsolt-marta-bitrise wants to merge 7 commits into
Conversation
Prepend a tarball-restore step to the git clone step that, on Bitrise runs, attempts to restore a `.git` directory tarball from the Build Cache under key `gitrepo-$BITRISE_APP_SLUG-main` and prepares the working tree the same way the `cached-tar` workflow does in hackathon-2026-git-prewarm: extract, resolve default branch via `git symbolic-ref --short HEAD`, add origin, run `git checkout BR -- .`. On cache miss or any error, the step falls back transparently to a normal clone. Uses the new `pkg/file` API from bitrise-io/bitrise-build-cache-cli#314 (pinned to PR head while it's still draft) and bumps the module to go 1.24 to satisfy that dep. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each phase of CheckoutState (setup, sparse-checkout, working-tree clean check, fetch+checkout, submodule update) and the surrounding step.Run/ExportOutputs now log a TInfof start marker plus "phase took Xms" line, mirroring the prewarm logging style. Allows side-by-side comparison of the prewarm-on / prewarm-off paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The original prewarm mirrored the hackathon `cached-tar` workflow and ran `git checkout BR -- .` after extracting the tarball — populating the working tree with master's files (~1.7s on the bazel repo). This turned out to be wasted work for both PR and push builds: the existing CheckoutState then checks out the actual target ref, rewriting the working tree. Removing the prewarm checkout naively isn't enough — the existing `isWorkingTreeClean` check would see the index pointing at master files but no working tree, mark the tree dirty, and fire a `git reset --hard HEAD` that does the same wasted work. Fix: replace `git checkout BR -- .` with `git read-tree --empty`. Both index and working tree are then empty, status reports clean, and the existing flow's checkout populates the working tree exactly once. Expected ~1.5-1.7s shaved off git-clone wall time when the prewarm hits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous attempt (3443705) cleared the index but left HEAD pointing at master. CheckoutState's `git status --porcelain` then reported every file in master's tree as " D <path>" (deleted-from-working-tree relative to the index — wait, actually, deleted-from-index relative to HEAD: the file is in HEAD's tree but absent from the now-empty index). Either way, status came back dirty, the existing flow ran `git reset --hard HEAD`, and we wrote 13314 files — exactly what we were trying to avoid. Fix: also point HEAD at a fresh, nonexistent branch (`refs/heads/__prewarm_uninit__`). With HEAD having no commit, no index entries, and no working tree files, `git status --porcelain` is empty, isWorkingTreeClean returns true, and the existing fetch + checkout writes the working tree exactly once. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wraps each git operation in `checkoutPRMergeRef.performCheckout` (delete-ref ×2, fetch merge ref, fetch head ref, checkout merge ref) with a wall-clock + duration log line, so we can isolate which sub-op dominates the "Fetch and checkout took Xs" total. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…M env override New step input `enable_repo_prewarm` (default "yes") gates the prewarm restore. Setting it to "no" — or setting the `BITRISE_GIT_CLONE_ENABLE_PREWARM` env var to false/no/0/off — bypasses PrewarmRepoFromBuildCache entirely while still using the same step binary, giving a clean apples-to-apples comparison of prewarm-on vs prewarm-off paths through the rest of CheckoutState. The env var takes precedence over the input so reviewers can flip prewarm off via build trigger params (`environments[]` array) without editing the workflow YAML. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When early_exit is yes/true/1/on (read directly from env, before ProcessConfig), main returns Success immediately. The Bitrise CLI still downloads + builds the step binary first; the early-exit just skips all git work after. Lets the workflow precede the real git-clone with a no-op invocation that warms Bitrise's stepman cache, so the real invocation pays no step boot cost. No more dummy clone of git-clone-test, no more submodule failure, no is_skippable hack. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.gitdirectory tarball from the Build Cache under keygitrepo-$BITRISE_APP_SLUG-mainand prepares the working tree the same way thecached-tarworkflow does in hackathon-2026-git-prewarm: extract, resolve default branch viagit symbolic-ref --short HEAD, add origin, rungit checkout BR -- ..pkg/fileAPI from bitrise-io/bitrise-build-cache-cli#314 (pinned to PR head while it's still draft).1.24to satisfy the build-cache-cli dep.Context: Hackathon 2026 — Git repo prewarm.
Notes
cfg.RepositoryURL, which keeps the existingisOriginPresentcheck happy and lets the rest of the clone flow continue unchanged.Test plan
go vet ./...cleango test ./...clean (gitclone, bitriseapi, step packages all green)gitrepo-<slug>-maintarball — verify HIT path logs and timings🤖 Generated with Claude Code