Stabilize release UI validation and Xcode selection - #31
Merged
Conversation
added 2 commits
July 18, 2026 12:58
Give each hosted UI smoke shard one bounded retry on a newly cloned simulator so transient XCUITest accessibility-service stalls can recover without masking deterministic failures. Keep CI and release settings aligned, document the seven-shard policy, and add a shell regression test for recovery, exhaustion, and invalid configuration.
Prefer the runner image's maintained stable Xcode instead of the oldest versioned install, keep prereleases as last-resort fallbacks, and regression-test candidate ordering. Expand the outer CI and release job budgets while retaining the strict per-shard watchdog and bounded retry policy.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
/Applications/Xcode.appinstead of the first lexicographic versioned installPINES_XCODE_PATHoverrideRoot cause
The initial release workflow ran the full UI target because it omitted the CI smoke-mode environment; #30 fixed that drift.
The first run of this PR then exposed two more infrastructure defects. Six smoke shards passed, while
testArtifactsResearchComposerFlowhitFailed to get matching snapshots: Timed out while evaluating UI queryon both fresh clones. The outer 90-minute job limit canceled the step just after the second failure, before the script could emit its final retry-exhausted error.The job log also showed that
select-xcode.shchose Xcode 26.0.1. GitHub macOS images install several Xcode 26 versions and maintain the current stable release at/Applications/Xcode.app, but the old script accepted the firstXcode*.appglob entry. That silently pinned CI, CodeQL, and release validation to the oldest available toolchain.No smoke journey was removed or weakened. A deterministic product failure must still fail independently on both clones, and each Xcode test process keeps its 720-second guard.
Validation
bash -n scripts/ci/*.shruby -c scripts/ci/check-xcode-workflow-contracts.rbbash scripts/ci/test-select-xcode.shbash scripts/ci/test-xcode-ui-shard-retry.shbash scripts/ci/check-xcode-workflow-contracts.shbash scripts/ci/check-public-hygiene.shgit diff --check