Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
196 changes: 38 additions & 158 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,186 +1,66 @@
# 🤖 RoostGPT: AI-Generated Playwright Test Suite
# Playwright Test Suite

**Automatic UI Test Generation for ATID Store Demo Website**
## RoostGpt Generated playwright test

## 📋 Overview
This project contains Playwright-based automated tests located in the `playwright_tests` directory.

This repository contains sample test runs demonstrating **RoostGPT's automatic UI test generation** capabilities using Playwright for the [ATID Store](https://atid.store) demo website. The ATID Store is a demo e-commerce website specifically designed **For Practicing QA & Test Automation Only!**
## 📁 Directory Structure

RoostGPT showcases two different approaches to AI-powered test generation:
- **Fully Automated**: RoostGPT-generated tests without any user input
- **User-Guided**: RoostGPT-generated tests based on user-provided scenarios

## 🌐 Target Website

**Website**: [atid.store](https://atid.store)
**Purpose**: Demo e-commerce site for QA practice and test automation learning



## 📁 Repository Structure

```
├── ai-generated-tests/
│ ├── tests/ # RoostGPT-generated test files
│ ├── scenarios/ # Generated test scenarios
│ ├── test-results/ # Test execution results
│ ├── package.json # Project dependencies
│ ├── package-lock.json # Locked dependencies
│ └── playwright.config.js # Playwright configuration
├── user-input-tests/
│ ├── tests/ # User-guided RoostGPT test files
│ ├── scenarios/ # User-defined test scenarios
│ ├── test-results/ # Test execution results
│ ├── package.json # Project dependencies
│ ├── package-lock.json # Locked dependencies
│ └── playwright.config.js # Playwright configuration
└── README.md # This file
```

## 🔍 Sample Test Scenarios

### RoostGPT Auto-Generated Example
```javascript
// Auto-generated product search test by RoostGPT
test('Add Product to Shopping Cart', async ({ page }) => {
// Step 1: Navigate to the Store page
const storePageUrl = 'https://atid.store/store/';
await page.goto(storePageUrl);

// Verify the Store page loaded correctly
await expect(page).toHaveURL(storePageUrl);

// Step 2: Locate a product listing and click on it to view its details
const productSelector = "//a[@href='https://atid.store/product/anchor-bracelet/' and contains(@class, 'woocommerce-LoopProduct-link')]";
const productUrl = 'https://atid.store/product/anchor-bracelet/';
await page.locator(productSelector).click();

// Step 3: Verify the product detail page loaded
await page.waitForURL(productUrl);
await expect(page).toHaveURL(productUrl);
// Additional RoostGPT-generated assertions...
});
```

### User-Guided RoostGPT Example
```javascript
test('Verify E-Commerce Checkout Process with Validation Errors', async ({ page }) => {
// Step 1: Navigate to the e-commerce website
await page.goto('https://atid.store/');
await expect(page).toHaveURL(/.*atid\.store\/$/);

// Step 2: Click on the 'SHOP NOW' button in the 'Latest Eyewear For You' category
const shopNowButton = page.locator('html > body > div:nth-of-type(1) > div:nth-of-type(1) > div > div > main > article > div > div > section:nth-of-type(1) > div:nth-of-type(2) > div > div > div:nth-of-type(4) > div > div > a > span');
await shopNowButton.click();

// Step 3: Verify navigation to the products page
await expect(page).toHaveURL(/.*atid\.store\/store\/$/);

// Step 4: Navigate to page 3 by clicking on '3'
const pageThreeButton = page.locator('html > body > div:nth-of-type(1) > div:nth-of-type(1) > div > div:nth-of-type(2) > main > div > nav:nth-of-type(2) > ul > li:nth-of-type(3) > a.page-numbers');
await pageThreeButton.click();
```bash
playwright_tests/
├── scenarios/
├── tests/
├── package.json
├── playwright.config.js
```

## 📝 User Requirements Workflow (User-Input Tests)
## 🧩 Prerequisites

The `user-input-tests/` folder follows a structured workflow from user requirements to RoostGPT test generation:
Make sure you have the following installed:

### 1. Requirements Documentation
```
- *אפיון*.docx - Original user requirements/scenarios document (Hebrew specification)
- Contains detailed business requirements and user scenarios provided by stakeholders
```
### 2. Feature Specification
- **`user_scenario.feature`** - Gherkin-formatted feature file generated from the requirements document
- Structured in Given-When-Then format for RoostGPT consumption
- Serves as the bridge between business requirements and RoostGPT implementation
- [Node.js](https://nodejs.org/) (version 18 or higher recommended)
- npm (comes with Node.js)

### 3. RoostGPT Test Generation Flow
```
*אפיון*.docx → user_scenario.feature → RoostGPT → Playwright Tests
```
## 📦 Setup

This workflow ensures traceability from original user requirements through to automated test generation via RoostGPT.
1. Navigate to the Playwright tests directory:

## 📋 Scenario Documentation
2. Install dependencies:

Each `scenarios/` folder contains a `scenario_summary.md` file providing comprehensive documentation of all RoostGPT-generated test scenarios. These files include:
```bash
npm install
```

### Scenario Details
- **Metadata**: Complexity level, priority, risk assessment, and execution time estimates
- **Test Steps**: Detailed step-by-step instructions for each scenario
- **Element Selectors**: CSS/XPath selectors used for element targeting
- **Expected Results**: Success criteria and validation points
- **Tags**: Categorization labels (form-submission, navigation, accessibility, etc.)
## 🚀 Running Tests

### Example Scenario Structure
```markdown
### 1. Verify User Can Successfully Submit the Contact Form
**Complexity**: high | **Priority**: high | **Risk Level**: high
**Tags**: form-submission, error-handling, accessibility
**Est. Execution Time**: 50 seconds | **Flakiness Potential**: medium
To execute all Playwright tests:

#### Steps:
- Navigate to the Contact Us page
- Fill in required form fields with valid data
- Submit the form and verify confirmation message
```bash
npx playwright test
```

The scenario documentation helps developers and QA engineers understand RoostGPT's test coverage, prioritize maintenance efforts, and plan execution schedules based on complexity and risk assessments.

## 🧪 Test Implementation Documentation
You can also run a specific test file:

Each `tests/` folder contains a `test_summary.md` file that provides a concise overview of all RoostGPT-implemented Playwright test files. This documentation bridges the gap between scenario planning and actual test implementation.

### Test Summary Format
```markdown
## Verify Navigation to 'Return to Shop' from Empty Cart
**Description:** Ensure that users can navigate to the shop page from the empty cart page using the 'RETURN TO SHOP' button.
**Priority**: high | **Complexity**: high
**Tags**: accessibility, navigation, cross-browser, ui-test
**Test File:** [verify_navigation_to__return_to_shop__from_empty_cart.spec.js](./verify_navigation_to__return_to_shop__from_empty_cart.spec.js)
```bash
npx playwright test tests/example.spec.js
```

### Key Features
- **Direct File Links**: Clickable links to actual RoostGPT-generated Playwright test files
- **Implementation Focus**: Bridges scenario documentation to executable code
- **Quick Reference**: Fast overview of all available tests
- **Tag-Based Filtering**: Easy identification of test types and coverage areas
- **Priority Mapping**: Helps prioritize test execution and maintenance

> Replace `example.spec.js` with the actual test file you want to run.

## 🚀 Test Execution
## 📂 Test Reports

### Prerequisites
- Node.js (version 14 or higher)
- npm or yarn package manager
After the tests run, Playwright will generate a report. To view the report:

### Installation & Setup

1. **Navigate to test folder**: Choose either `ai-generated-tests/` or `user-input-tests/`
```bash
cd ai-generated-tests/
# OR
cd user-input-tests/
```
```bash
npx playwright show-report
```

2. **Install dependencies**:
```bash
npm install
```
## ⚙️ Configuration

3. **Install Playwright browsers** (first time only):
```bash
npx playwright install
```
Test configuration is defined in `playwright.config.js`.

### Running Tests
You can modify settings like test directory, timeout etc., in this file.

#### Basic Test Execution
```bash
# Run all tests
npx playwright test
## 📘 More Info

```
For detailed Playwright documentation, visit: [https://playwright.dev](https://playwright.dev)
18 changes: 18 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
name: "production-playwright-tests",
version: "1.0.0",
description: "A robust suite of end-to-end tests using Playwright.",
scripts: {
test: "npx playwright test",
"test:headed": "npx playwright test --headed",
report: "npx playwright show-report",
lint: "eslint . --ext .js,.ts",
},
keywords: ["playwright", "testing", "e2e", "automation"],
author: "",
license: "ISC",
devDependencies: {
"@playwright/test": "^1.54.1",
eslint: "^8.57.0",
},
}
47 changes: 47 additions & 0 deletions playwright.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@

// @ts-check
const { defineConfig, devices } = require('@playwright/test');

/**
* @see https://playwright.dev/docs/test-configuration
*/
module.exports = defineConfig({
testDir: './playwright_tests',
/* Run tests in files in parallel */
fullyParallel: true,
/* Fail the build on CI if you accidentally left test.only in the source code. */
forbidOnly: !!process.env.CI,
/* Retry on CI only */
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
/* Reporter to use. See https://playwright.dev/docs/test-reporters */
reporter: 'html',
/* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
use: {
/* Base URL to use in actions like `await page.goto('/')`. */
// baseURL: 'http://127.0.0.1:3000',

/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: 'on-first-retry',
},

/* Configure projects for major browsers */
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},

{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},

{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
},
],
});

Loading