We currently don't have an open source "Eval" suite to verify basic interactions. There is a test suite that is not public, but it would be good to create a public test suite to ensure evaluations are run on open source changes.
This change relies on #48 being done first.
Examples:
Level 1 - Basic Screen Interaction
- Tap on UI elements
- Verify that a string exists or does not exist on screen
- Enter text into a text field
- Verify up to 2-3 commands in a prompt
Level 2 - Screen Navigation and Basic Logic
- Manipulate the correct screen or transition to the next screen:
- Scroll to find something in a list
- Tap on a button (e.g., Review Sale, Charge)
- Perform basic math verifications:
- Example: Stock quantity increases correctly after an action
- Handle use of the back button
- Navigate a ten-key (numeric keypad)
- Fail gracefully if:
- Element is impossible to tap
- Text is impossible to enter
- Edit existing text
- Verify up to 5 commands in a prompt
Level 3 - Complex Interaction and Timing-Based Verification
- Locate and select the correct text input field when multiple fields exist
- Navigate screens with large number of interactive elements
- Verify math or data consistency on a busy screen with many updates
- Verify up to 10 commands in a prompt
We currently don't have an open source "Eval" suite to verify basic interactions. There is a test suite that is not public, but it would be good to create a public test suite to ensure evaluations are run on open source changes.
This change relies on #48 being done first.
Examples:
Level 1 - Basic Screen Interaction
Level 2 - Screen Navigation and Basic Logic
Level 3 - Complex Interaction and Timing-Based Verification