Functional test generated by RoostGPT#70
Open
Divyeshzb wants to merge 1 commit into
Open
Conversation
Using AI Model gpt-5
There was a problem hiding this comment.
Pull request overview
Adds a new RoostGPT-generated functional test pack (functional-test-generation) to the functional_tests/ suite, including human-readable specs and multiple export formats.
Changes:
- Added a comprehensive end-to-end + functional coverage set in JSON and Gherkin (
.feature) formats. - Added supporting exports (
.csv,.xlsx,.docx) and Roost metadata for traceability. - Updated
functional_tests/README.mdto register the new generated test bundle.
Reviewed changes
Copilot reviewed 4 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| functional_tests/functional-test-generation/functional-test-generation.json | New test-case bundle (JSON) describing scenarios, steps, and expectations |
| functional_tests/functional-test-generation/functional-test-generation.feature | New Gherkin feature file covering E2E/security/transactions/payments/WS/compliance flows |
| functional_tests/functional-test-generation/functional-test-generation.csv | New scenario index list for the generated pack |
| functional_tests/functional-test-generation/functional-test-generation.xlsx | Spreadsheet export of the generated tests |
| functional_tests/functional-test-generation/functional-test-generation.docx | Document export of the generated tests |
| functional_tests/functional-test-generation/.roost/roost_metadata.json | Generation metadata (inputs, timestamps, source URIs) |
| functional_tests/README.md | Appended entry referencing the new generated pack outputs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | VALID | A | PAYMENT_REMINDER | IN_APP | INFO | Invalid alert | 11111111-2222-4333-8444-555555555555 | 400 | INVALID_ALERT_TYPE | | ||
| | VALID | A | STATEMENT_READY | FAX | INFO | Invalid channel | 11111111-2222-4333-8444-555555555556 | 400 | INVALID_ALERT_TYPE | | ||
| | VALID | A | STATEMENT_READY | IN_APP | INFO | Missing idempotency key | | 400 | VALIDATION_ERROR | | ||
| | MISSING | A | STATEMENT_READY | IN_APP | INFO | Unauthorized | 11111111-2222-4333-8444-555555555557 | 401 | FORBIDDEN | |
There was a problem hiding this comment.
In this examples row the step is described as "Unauthorized" with HTTP 401, but the expected error code is 'FORBIDDEN' (typically associated with 403). This mismatch will make the test ambiguous/flaky—please align the status and error code (e.g., 401 + UNAUTHORIZED/INVALID_AUTH, or 403 + FORBIDDEN).
Suggested change
| | MISSING | A | STATEMENT_READY | IN_APP | INFO | Unauthorized | 11111111-2222-4333-8444-555555555557 | 401 | FORBIDDEN | | |
| | MISSING | A | STATEMENT_READY | IN_APP | INFO | Unauthorized | 11111111-2222-4333-8444-555555555557 | 401 | UNAUTHORIZED | |
Comment on lines
+1
to
+3
| [ | ||
| { | ||
| "type": "end-to-end", |
There was a problem hiding this comment.
This JSON file’s top-level value is an array, but other Roost-generated functional_tests/*.json files in this repo use an object wrapper with a 'scenarios' array (e.g., { "scenarios": [...] }). If any tooling expects the existing schema, this new format will break parsing—please conform to the established JSON structure.
Comment on lines
+1
to
+33
| Approved end-to-end: Registration to rescind with card controls, FX transaction, payments, notifications, and rescind window | ||
| Application session expiration with save/resume and Step 3 signature validation | ||
| Application decision boundaries for FICO thresholds | ||
| Registration invalid field validations and weak password handling | ||
| Registration valid then duplicate email rejection | ||
| Transactions validation errors and CSRF enforcement | ||
| Essential buffer boundary and FX fee precision | ||
| Transaction frequency rate limiting with step-up MFA retry | ||
| Cross-account/card owner-only enforcement returns 403 with no leakage | ||
| Owner can access own summary while non-owner cannot | ||
| Notifications webhook invalid inputs and authorization checks | ||
| Notifications webhook idempotency scoping per account and across channels | ||
| Notifications webhook message length and channel-scoped idempotency per account | ||
| Email verification lifecycle with blocked pre-verification login and resend | ||
| Refresh token rotation under multi-tab concurrency with CSRF continuity | ||
| Transactions history category filters, date-range validation, and paging | ||
| Login lockout, per-IP rate limiting, refresh rotation, CSRF cross-site protection, inactivity timeout, and PAN masking | ||
| Authorized subscription to own account, forbidden cross-account, schema validation, reconnect and dedupe | ||
| WebSocket unauthorized handshake and subscribe/unsubscribe lifecycle | ||
| Payment scheduling boundaries: min amount, past-date rejection, same-day immediate, and FULL_BALANCE | ||
| MINIMUM and STATEMENT_BALANCE payments and precision enforcement | ||
| Report lost/stolen irreversible flow, OTP failures, invalid transitions, and PIN format enforcement | ||
| Report lost/stolen with delivery address override validation and replacement confirmation | ||
| Refresh token TTL expiry, 401 on refresh, re-authentication, and CSRF continuity | ||
| Summary include_rewards toggle, rewards floor verification, and owner-only enforcement | ||
| Step 2 idempotency and cross-application session token misuse | ||
| CSRF token binding and invalid-token rejection across endpoints | ||
| Essential over-limit buffer lifecycle with recovery after payment | ||
| Transactions maximum amount and FX exchange_rate precision with REQ-006 rounding | ||
| Trusted device remember_me 30-day TTL and MFA suppression on known device | ||
| Rescind on exact Day 14 with CSRF enforcement and post-closure behavior | ||
| Ensure no PII leakage on error payloads and UI across modules | ||
| Draft auto-save at 60s, sanitized localStorage, resume, submit clears draft, and inactivity warning No newline at end of file |
There was a problem hiding this comment.
This CSV doesn’t follow the formatting used by other generated functional_tests/*.csv files in the repo (they wrap each entry in quotes and often include a consistent prefix like "Scenario:" or "[API]/[UI]"). For consistency and to avoid CSV parsing issues in downstream tooling, consider matching the existing quoting/prefix convention here as well.
Suggested change
| Approved end-to-end: Registration to rescind with card controls, FX transaction, payments, notifications, and rescind window | |
| Application session expiration with save/resume and Step 3 signature validation | |
| Application decision boundaries for FICO thresholds | |
| Registration invalid field validations and weak password handling | |
| Registration valid then duplicate email rejection | |
| Transactions validation errors and CSRF enforcement | |
| Essential buffer boundary and FX fee precision | |
| Transaction frequency rate limiting with step-up MFA retry | |
| Cross-account/card owner-only enforcement returns 403 with no leakage | |
| Owner can access own summary while non-owner cannot | |
| Notifications webhook invalid inputs and authorization checks | |
| Notifications webhook idempotency scoping per account and across channels | |
| Notifications webhook message length and channel-scoped idempotency per account | |
| Email verification lifecycle with blocked pre-verification login and resend | |
| Refresh token rotation under multi-tab concurrency with CSRF continuity | |
| Transactions history category filters, date-range validation, and paging | |
| Login lockout, per-IP rate limiting, refresh rotation, CSRF cross-site protection, inactivity timeout, and PAN masking | |
| Authorized subscription to own account, forbidden cross-account, schema validation, reconnect and dedupe | |
| WebSocket unauthorized handshake and subscribe/unsubscribe lifecycle | |
| Payment scheduling boundaries: min amount, past-date rejection, same-day immediate, and FULL_BALANCE | |
| MINIMUM and STATEMENT_BALANCE payments and precision enforcement | |
| Report lost/stolen irreversible flow, OTP failures, invalid transitions, and PIN format enforcement | |
| Report lost/stolen with delivery address override validation and replacement confirmation | |
| Refresh token TTL expiry, 401 on refresh, re-authentication, and CSRF continuity | |
| Summary include_rewards toggle, rewards floor verification, and owner-only enforcement | |
| Step 2 idempotency and cross-application session token misuse | |
| CSRF token binding and invalid-token rejection across endpoints | |
| Essential over-limit buffer lifecycle with recovery after payment | |
| Transactions maximum amount and FX exchange_rate precision with REQ-006 rounding | |
| Trusted device remember_me 30-day TTL and MFA suppression on known device | |
| Rescind on exact Day 14 with CSRF enforcement and post-closure behavior | |
| Ensure no PII leakage on error payloads and UI across modules | |
| Draft auto-save at 60s, sanitized localStorage, resume, submit clears draft, and inactivity warning | |
| "Scenario: Approved end-to-end: Registration to rescind with card controls, FX transaction, payments, notifications, and rescind window" | |
| "Scenario: Application session expiration with save/resume and Step 3 signature validation" | |
| "Scenario: Application decision boundaries for FICO thresholds" | |
| "Scenario: Registration invalid field validations and weak password handling" | |
| "Scenario: Registration valid then duplicate email rejection" | |
| "Scenario: Transactions validation errors and CSRF enforcement" | |
| "Scenario: Essential buffer boundary and FX fee precision" | |
| "Scenario: Transaction frequency rate limiting with step-up MFA retry" | |
| "Scenario: Cross-account/card owner-only enforcement returns 403 with no leakage" | |
| "Scenario: Owner can access own summary while non-owner cannot" | |
| "Scenario: Notifications webhook invalid inputs and authorization checks" | |
| "Scenario: Notifications webhook idempotency scoping per account and across channels" | |
| "Scenario: Notifications webhook message length and channel-scoped idempotency per account" | |
| "Scenario: Email verification lifecycle with blocked pre-verification login and resend" | |
| "Scenario: Refresh token rotation under multi-tab concurrency with CSRF continuity" | |
| "Scenario: Transactions history category filters, date-range validation, and paging" | |
| "Scenario: Login lockout, per-IP rate limiting, refresh rotation, CSRF cross-site protection, inactivity timeout, and PAN masking" | |
| "Scenario: Authorized subscription to own account, forbidden cross-account, schema validation, reconnect and dedupe" | |
| "Scenario: WebSocket unauthorized handshake and subscribe/unsubscribe lifecycle" | |
| "Scenario: Payment scheduling boundaries: min amount, past-date rejection, same-day immediate, and FULL_BALANCE" | |
| "Scenario: MINIMUM and STATEMENT_BALANCE payments and precision enforcement" | |
| "Scenario: Report lost/stolen irreversible flow, OTP failures, invalid transitions, and PIN format enforcement" | |
| "Scenario: Report lost/stolen with delivery address override validation and replacement confirmation" | |
| "Scenario: Refresh token TTL expiry, 401 on refresh, re-authentication, and CSRF continuity" | |
| "Scenario: Summary include_rewards toggle, rewards floor verification, and owner-only enforcement" | |
| "Scenario: Step 2 idempotency and cross-application session token misuse" | |
| "Scenario: CSRF token binding and invalid-token rejection across endpoints" | |
| "Scenario: Essential over-limit buffer lifecycle with recovery after payment" | |
| "Scenario: Transactions maximum amount and FX exchange_rate precision with REQ-006 rounding" | |
| "Scenario: Trusted device remember_me 30-day TTL and MFA suppression on known device" | |
| "Scenario: Rescind on exact Day 14 with CSRF enforcement and post-closure behavior" | |
| "Scenario: Ensure no PII leakage on error payloads and UI across modules" | |
| "Scenario: Draft auto-save at 60s, sanitized localStorage, resume, submit clears draft, and inactivity warning" |
Comment on lines
+4
to
+6
| Given the API base URL is 'https://api.aegiscard.com/v2' | ||
| And the Portal URL is 'https://portal.aegiscard.com' | ||
| And the WebSocket URL is 'wss://realtime.aegiscard.com/v2/stream' |
There was a problem hiding this comment.
This feature hardcodes the API/Portal/WebSocket base URLs. Other functional_tests/*.feature files set the API base URL from an environment variable (e.g., BASE_URL), which keeps tests portable across environments. Consider switching these URLs to env/config-driven values (and avoid embedding the '/v2' path segment in the base if the step definitions already add it).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RoostGPT Logs