e2e: enable demo_app on iOS (draft, iterating)#3253
Draft
amanjeetsingh150 wants to merge 7 commits intomainfrom
Draft
e2e: enable demo_app on iOS (draft, iterating)#3253amanjeetsingh150 wants to merge 7 commits intomainfrom
amanjeetsingh150 wants to merge 7 commits intomainfrom
Conversation
The OOM concern that motivated this skip predates the move to macos-26 runners. Remove the dispatcher-level short-circuit so demo_app's passing-tagged flows run on iOS via the existing test-e2e workflow; android-only flows are still excluded via tag filtering.
Maestro's CLI ships a pinned applesimutils that it unpacks to ~/.maestro/deps/applesimutils. LocalSimulatorUtils falls back to a PATH-based applesimutils if the pinned binary fails. Install via wix/brew so that fallback path actually has something to call.
- Remove the simulator screen-recording start/stop/upload steps from the test-ios job. The xctest runner logs and ~/.maestro artifacts carry enough debugging context for failure analysis. - Skip wikipedia on iOS in run_tests (mirroring the existing android skip) and skip downloading wikipedia.zip in download_apps. Wikipedia is no longer part of the iOS demo_app focus on this branch.
iPhone 17 Pro is @3x DPR. The Flutter cropped_screenshot fixture's physical/DPR math doesn't round-trip cleanly through DPR=3 (e.g. 400/3=133.33, then snapped back to 399 or 400 depending on path), producing a 1-pixel size mismatch against the Android-generated baseline. iPhone SE (3rd gen) is @2x, so 400/2/320/2/etc. are all integer-clean and the existing baseline works on both platforms. Bump runtime to iOS 26.2 alongside.
The iPhone SE (3rd gen) switch fixed the assertScreenshot dimension round-trip but its smaller screen caused the Flutter counter widget to fall below the viewport on the demo_app home page, breaking several non-screenshot flows (repeat, runFlow, retry, copyTextFrom). Going back to iPhone 17 Pro keeps the larger viewport; we will address the screenshot DPR issue separately. Runtime stays on iOS 26.2.
The cropped_screenshot fixture renders 400x320 on Android (DPR=1.5)
but 399x321 on iPhone Pro (DPR=3) — Flutter's physical/DPR math
doesn't round-trip cleanly across both DPRs. Rather than touching
the fixture, ship a separate iOS baseline.
- Tag existing assertScreenshotCropped{,JPG}.yaml as android-only;
baselines unchanged.
- Add assertScreenshotCropped{,JPG}_ios.yaml mirroring the same
flow steps, tagged ios-only, against new iOS-Pro baselines
captured locally on iPhone 17 Pro / iOS 26.0.
amanjeetsingh150
force-pushed
the
enable-e2e-demo-app
branch
from
db9b9a0 to
ee8764e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
demo_appon iOS ine2e/run_tests(single-app and full-suite branches) plus the stale# demo_app has OOM issues on GHAcomment.test-e2e.yamlalready buildsdemo_app.appfor the simulator and calls./run_tests ios. Tag-filtering inrun_testsalready excludes android-only flows on the iOS path, so this is the entire enablement diff.Why draft
Iterating to green: each
test-iosfailure will be diagnosed with thediagnose-maestro-failureagent, reported back, and the disposition (fix vs. tag android-only vs. defer) decided per flow. PR will flip to ready-for-review oncetest-iosis green.Disposition log
Test plan
test-e2e.yamltest-iosjob runs end-to-end without the runner being killed (no OOM).passing-tagged demo_app flows pass on iOS._run_failingblock on iOS still reports the failing-tagged flows as expected failures.test-androidortest-webjobs.