Add feature to demo_app to expose assertScreenshot threshold issue

[simon-gilmurray]

Proposed changes

Doximity are experiencing issues with assertScreenshot where it is returning a failure on assertion, but the threshold is reporting back as being above the specified limit, e.g. a default 95 threshold erroring with a >96 threshold:

Comparison error: Assert screenshot matches ../screenshots/assertScreenshot_threshold.png (threshold: 95.0%) - threshold not met, current: 96.18651%

This PR creates:

• a new screen accessible through assertScreenshot Threshold button
• this screen has an alert on it with grey background and content I've mimicked from Doximity
• I've added a button that toggles the position of the alert upwards/downwards 10px
• A Maestro flow to assertScreenshot that compares the previous position alert golden screenshot with the position once it is shifted downwards 10px and generates the above error

My suspicion here is that perhaps because the majority of the screen content is the same, just repositioned slightly, it is therefore failing but with a higher than expected threshold.

Slack thread

Resulting diff screenshot:

Testing

issue_assertScreenshotThreshold flow added

Issues fixed

[proksh]

Hey @simon-gilmurray I tried moving the test in e2e but now I also see that app file is not apk. How are planning to run the test? Should we build the app and then place it? Or should we have a similar test for android case?

[simon-gilmurray]

Hey @simon-gilmurray I tried moving the test in e2e but now I also see that app file is not apk. How are planning to run the test? Should we build the app and then place it? Or should we have a similar test for android case?

@proksh Can we mark e2e test with an iOS tag and make a note to add an Android one at a later time?

[simon-gilmurray]

@proksh Can we mark e2e test with an iOS tag and make a note to add an Android one at a later time?

Sure, but I dont see ios app file in this PR as well. What if we build this test on wikipedia app? I dont wanna add a new app for just this e2e test.

We can use this screenshot for assertion which wont match for the new wikipedia version of app - https://github.com/mobile-dev-inc/Maestro/pull/3261/changes#diff-e30efa982303fc49f8567ef8ae887b25f156f55c58d22c98131eade080d5621a

Have DMed @proksh and explained that this PR has the feature for demo_app used by e2e

[proksh]

Ok, just checked and its running fine on android. For ios, we are skipping demo app currently. Amanjeet is working on fixing that. So this PR is fine to be merged.