triggered-integration-test: refs/pull/N/merge can lag live main by up to 12h, causing spurious 'base changed' failures

[jordigilh]

Context

Follow-up to #62, discovered while validating that fix's landed behavior live against PR #63.

#62 made resolve-source's two ref lookups (refs/pull/<n>/merge and refs/heads/main) atomic — a single git ls-remote call instead of two sequential ones — closing the race where main could advance between the two calls. That fix is correct and is merged.

What I found

refs/pull/<n>/merge itself is a commit GitHub computes and caches asynchronously, and reading it atomically alongside refs/heads/main doesn't guarantee the merge ref's content reflects that same live main tip — the ref can already be stale by the time both are atomically read.

Per GitHub's own changelog, Changes to test merge commit generation for pull requests (2026-02-19), refs/pull/N/merge is now only regenerated when:

• a push lands on the PR branch,
• the merge-base between the PR and its base branch changes, or
• the existing test-merge commit turns 12 hours old.

Viewing the PR page no longer forces a refresh (an assumption some older tooling/advice relies on).

Live reproduction (2026-08-14, ~17:00-17:46 UTC)

Right after #62 merged (main advanced to 7ebabdb), I checked out PR #63's merge ref (refs/pull/63/merge = b4a5ae0) and ran the exact "Verify PR merge checkout" logic from triggered-integration-test.yaml against it:

Expected base:    7ebabdbc49ed5e71fffe832a620590f46b37fad1  (live main)
Merge parent 1:   282a33753af0065ad4c474f8f5783376df2973ce  (stale -- main's previous tip)
FAIL: Checked-out merge does not combine the expected base and PR head

49+ minutes later, including after an explicit GET /pulls/63 call that returned mergeable: true, refs/pull/63/merge still pointed at the same stale commit -- the mergeable boolean and the physical merge-ref content are refreshed independently, and re-querying mergeability doesn't force the ref itself to regenerate.

Impact

Any workflow_dispatch run of triggered-integration-test.yaml against an open PR, triggered shortly after any push to main, will hit this and fail the "Verify PR merge checkout" step with a "base changed" error -- correctly (fail-closed is the right behavior for that check), but for a transient, unrelated-to-the-PR reason. Given GitHub's stated 12-hour worst case, a short retry/poll loop would only reduce the failure rate, not bound it.

Proposed fix (spiked and validated live)

Stop depending on refs/pull/N/merge for the tested content entirely. resolve-source already resolves head_sha (from the PR REST API) and base_sha (live refs/heads/main) as two pinned, immutable commit SHAs -- construct the merge locally from those two SHAs instead of trusting GitHub's cached merge commit:

git fetch origin "${BASE_SHA}" "${HEAD_SHA}"
git checkout "${BASE_SHA}"
git merge --no-ff --no-edit "${HEAD_SHA}"   # --no-ff: force a 2-parent commit even if FF is possible

Validated live against PR #63 while the staleness above was still in effect -- this merges cleanly against the current main tip in under 2 seconds, with no dependency on GitHub's cache state:

$ git fetch origin 7ebabdbc... 50720c9d... && git checkout 7ebabdbc... && git merge --no-edit 50720c9d...
Merge made by the 'ort' strategy.
 crdt/src/gcounter.rs        |  48 ++
 crdt/src/grid_state.rs      | 156 ++++++++
 swim/src/state_broadcast.rs |  74 +++
exit=0

Both resolve-source (to fail fast on real conflicts before the 120-minute glb-e2e job runs) and glb-e2e's checkout step (to get the actual tested tree) would run the identical two commands against the same pinned base_sha/head_sha outputs already produced today -- no new cross-job artifact sharing, no contents: write permission bump, no polling/timeout tuning.

Why not a retry loop instead

Considered and rejected: bounded retry/poll of refs/pull/N/merge until its parent matches live main. Rejected because GitHub's own changelog states a 12-hour worst-case regeneration bound, so a short (minutes-scale) retry only reduces -- doesn't eliminate -- the false-failure rate, and a 12-hour-scale retry isn't practical for a CI job.