Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
227959c
feat: add initial loadtest to testbench
caalador Mar 23, 2026
5f958dd
Fix delete concurrency issue in demo.
caalador Mar 24, 2026
5b42387
Add duration treshold and Exception check.
caalador Mar 24, 2026
4712b80
Make metrics a bit more readable
caalador Mar 24, 2026
5cdff45
Add start and stop mojos
caalador Mar 24, 2026
9cfefff
Add metrics to renamed module
caalador Mar 24, 2026
4a3c0e6
Add helper class for setting up the proxy for tests.
caalador Mar 25, 2026
8656215
Add value collection to csv that can be extended to get random input …
caalador Mar 26, 2026
0ef837b
Enable configuring Tresholds for request duration (P95, P99) and abor…
caalador Mar 26, 2026
8c58aa2
Fix server logging when server started with plugin start-server goal
caalador Mar 27, 2026
df41b3e
Add licenseChecker with tests.
caalador Mar 27, 2026
27da7fd
Merge branch 'main' into feat/loadtest
caalador Mar 27, 2026
0e40cd1
Merge branch 'main' into feat/loadtest
caalador Mar 27, 2026
5c304b8
Update loadtest to 10.2
caalador Mar 27, 2026
dfb434d
format
caalador Mar 27, 2026
95b314e
remove extra version properties
caalador Mar 27, 2026
3119dff
match versions
caalador Mar 27, 2026
c6c78ca
add missing import
caalador Mar 27, 2026
6ef35c9
add javadoc
caalador Mar 27, 2026
6f93f5d
javadocs
caalador Mar 27, 2026
ba52b80
fix javadoc error
caalador Mar 27, 2026
21d4abc
format
caalador Mar 27, 2026
b128b7e
use flow maven plugin instead of vaadin maven plugin
caalador Mar 27, 2026
d90276f
Fix version targets.
caalador Mar 27, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@
<module>vaadin-testbench-unit-junit5</module>
<module>vaadin-testbench-unit-junit6</module>
<module>vaadin-testbench-unit-quarkus</module>
<module>vaadin-testbench-loadtest</module>
</modules>
<repositories>
<repository>
Expand Down
3 changes: 3 additions & 0 deletions vaadin-testbench-loadtest/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*/src/main/frontend/generated
*/target
src/main/frontend/generated
14 changes: 14 additions & 0 deletions vaadin-testbench-loadtest/.mcp.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"mcpServers": {
"Vaadin": {
"type": "http",
"url": "https://mcp.vaadin.com/docs"
},
"playwright": {
"command": "npx",
"args": [
"@playwright/mcp@latest"
]
}
}
}
90 changes: 90 additions & 0 deletions vaadin-testbench-loadtest/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# AI TOOL GUIDANCE

This file provides guidance when working with code in this repository.

## Technology Stack

This is a Vaadin application built with:
- Java
- Spring Boot
- Spring Data JPA with H2 database
- Maven build system

## Development Commands

### Running the Application
```bash
./mvnw # Start in development mode (default goal: spring-boot:run)
./mvnw spring-boot:run # Explicit development mode
```

The application will be available at http://localhost:8080

### Building for Production
```bash
./mvnw -Pproduction package # Build production JAR
docker build -t my-application:latest . # Build Docker image
```

### Testing
```bash
./mvnw test # Run all tests
./mvnw test -Dtest=TaskServiceTest # Run a single test class
./mvnw test -Dtest=TaskServiceTest#tasks_are_stored_in_the_database_with_the_current_timestamp # Run a single test method
```

## Architecture

This project follows a **feature-based package structure** rather than traditional layered architecture. Code is organized by functional units (features), not by technical layers.

### Package Structure

- **`com.example.application.base`**: Reusable components and base classes for all features
- `base.ui.MainLayout`: AppLayout with drawer navigation using SideNav, automatically populated from @Menu annotations
- `base.ui.component.ViewToolbar`: Reusable toolbar component for views

- **`com.example.application.examplefeature`**: Example feature demonstrating the structure
- `Task.java`: JPA entity with validation
- `TaskRepository.java`: Spring Data JPA repository
- `TaskService.java`: Service layer with @Transactional methods
- `ui.TaskListView.java`: Vaadin Flow view component (server-side UI)
- `TaskServiceTest.java`: Integration test using @SpringBootTest

- **`Application.java`**: Main entry point, annotated with @SpringBootApplication and @Theme("default")

### Key Architecture Patterns

1. **Feature Packages**: Each feature is self-contained with its own UI, business logic, data access, and tests
2. **Navigation**: Views use `@Route` and `@Menu` annotations. MainLayout automatically builds navigation from menu entries
3. **Service Layer**: Use `@Transactional` for write operations and `@Transactional(readOnly = true)` for read operations
4. **Validation**: Domain validation in entity setters (see Task.setDescription)
5. **Dependency Injection**: Constructor injection throughout (no @Autowired on fields)

## Adding New Features

When creating a new feature:
1. Create a new package under `com.example.application` (e.g., `com.example.application.myfeature`)
2. Include: Entity, Repository, Service, and UI view classes
3. Use the `examplefeature` package as a reference
4. Once your features are complete, **delete the `examplefeature` package entirely**

## Vaadin-Specific Notes

- **Server-side rendering**: UI components are Java classes extending Vaadin components
- **Grid lazy loading**: Use `VaadinSpringDataHelpers.toSpringPageRequest(query)` for pagination
- **Themes**: Located in `src/main/frontend/themes/default/`, based on Lumo theme
- **Routing**: `@Route("")` for root path, `@Route("path")` for specific paths
- **Menu**: `@Menu` annotation controls navigation items (order, icon, title)

## Database

- H2 in-memory database for development
- JPA entities use `@GeneratedValue(strategy = GenerationType.SEQUENCE)`
- Entity equality based on ID (see Task.equals/hashCode pattern)

## Testing

- k6 tests are located in `src/test/k6`
- k6 tests can be run with `k6 run src/test/k6/...`
- https://github.com/johannest/k6-demo/blob/main/book-store.js is a good example of a k6 test
- https://grafana.com/docs/k6/latest/ is a good resource for learning k6
252 changes: 252 additions & 0 deletions vaadin-testbench-loadtest/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,252 @@
# Vaadin LoadTestKit k6 - record real browser interactions and replay them at scale

This is a PoC of a tool that utilizes Vaadin TestBench E2E tests as "user stories" for k6 based load testing. Technically the tooling starts a local instance of your server, records the web traffic of E2E test cases (you can select n+1 from your app), converts them into k6 scripts (taking details of Vaadin client-server communications, such as csrf preventation mechanism into account) and provides an easy way to then generate k6 load test against your test server (doesn't have to, and for meaningful numbers, shouldn't be the same server.)


## Project Structure

```
k6-testbench-recorder/
├── testbench-converter-plugin/ # Maven plugin for k6 recording and conversion
├── demo-web-app/ # Sample Vaadin application with TestBench tests
└── demo-web-app-loadtest/ # Integration test module demonstrating the workflow
```

## Prerequisites

| Tool | Version | Installation |
|------|---------|--------------|
| Java | 21+ | [Download](https://adoptium.net/) |
| Maven | 3.9+ | [Download](https://maven.apache.org/) |
| k6 | latest | `brew install k6` (macOS) or [Download](https://grafana.com/docs/k6/) |
| Chrome | latest | [Download](https://www.google.com/chrome/) |

## Quick Start

The modules are not pushed to Maven central yet, so make a clean install first:

```bash
mvn install
```

Builds both the tooling and a simple demo web app with two Vaadin TestBench E2E tests.

### Run the Demo (Local)

*Note, you should not do this for anything else but to test the setup without external server*

```bash

# Run the complete workflow (start app, record, run load test)
mvn verify -pl demo-web-app-loadtest
```

### Option 3: Remote Load Testing

Run pre-recorded tests against a server running on another machine:

First deploy the test app to a remote server. The next snippet assumes the remote server is staging.example.com (replace to yours):

```bash
# Test against a staging server
mvn verify -pl demo-web-app-loadtest -Premote \
-Dk6.appHost=staging.example.com \
-Dk6.appPort=8080 \
-Dk6.vus=100 \
-Dk6.duration=5m
```

This now re-builds the k6 tests against local server deployment if source tests (TestBench) have changed, and then executes load test with k6 against the defined appHost.

### Using the plugin "manually"

The `testbench-converter-plugin` plugin provides three goals, these are used in the loadtest module, but can be in theory used standalone as well.

```bash
# Convert an existing HAR file to k6
mvn k6:convert -Dk6.harFile=recording.har

# Record a TestBench test and convert to k6
mvn k6:record -Dk6.testClass=HelloWorldIT

# Run a k6 load test
mvn k6:run -Dk6.testFile=k6/tests/hello-world.js -Dk6.vus=50 -Dk6.duration=1m
```

### JBang App (Alternative to Maven Plugin)

For quick experimentation or when you don't want to use Maven for load testing orchestration, there's a standalone JBang app that provides the same recording functionality.

Check [jbang/README.md](jbang/README.md) for documentation & demo walkthrough.

The JBang app also serves as a **reference implementation** for creating your own custom tooling.

## Creating TestBench Tests

Standard TestBench integration tests define user workflows. The same that you use
already to ensure the functionality doesn't break. If you update the app and/or
your test pattern, your load tests are automatically updated 🥳

In practice you probably want to select certain case from your apps E2E test battery, or craft a special case(s) re-using some page-object classes of your E2E
tests.

The k6:record goal runs these through a proxy to capture HTTP traffic for load testing. This is automated in the demo-web-app-loadtest module.

*In the current PoC, we need a slight hacks to the superclass, but final version of the project should require no special things in your app (we'll find a workaround or add support to TestBench superclass).*

### Example Test

```java
public class HelloWorldIT extends AbstractIT {

@BrowserTest
public void helloWorldWorkflow() {
// Enter name in text field
TextFieldElement nameField = $(TextFieldElement.class).first();
nameField.setValue("Test User");

// Click button
$(ButtonElement.class).first().click();

// Verify result
$(NotificationElement.class).waitForFirst();
}

@Override
public String getViewName() {
return ""; // Root path
}
}
```

## Running k6 Tests

Running k6 tests is automated in the demo-web-app-loadtest module and you can
tune parameter via Maven build file. The Maven example also displays a bit for
server health metrics after the execution, collected using Spring Boot Actuator.
Below you can see an example output:




The k6 scripts are also available in target directory if you want to execute them manually.

### Basic Execution

```bash
k6 run k6/tests/hello-world.js
```

### Load Testing

```bash
# 50 virtual users for 30 seconds
k6 run --vus 50 --duration 30s k6/tests/hello-world.js

# Against a different server
k6 run -e APP_IP=192.168.1.100 -e APP_PORT=8080 k6/tests/hello-world.js
```

### Understanding k6 Output

```
/\ |‾‾| /‾‾/ /‾‾/
/\ / \ | |/ / / /
/ \/ \ | ( / ‾‾\
/ \ | |\ \ | (‾) |
/ __________ \ |__| \__\ \_____/

scenarios: (100.00%) 1 scenario, 50 max VUs, 1m30s max duration

✓ page load status equals 200
✓ vaadin init status equals 200

http_req_duration..............: avg=45.23ms min=12.34ms max=234.56ms
http_req_failed................: 0.00% ✓ 0 ✗ 1234
http_reqs......................: 1234 41.13/s
```

Key metrics:
- **http_req_duration**: Response time (avg, min, max)
- **http_req_failed**: Percentage of failed requests
- **http_reqs**: Total requests and throughput

## Module Documentation

- [testbench-converter-plugin](testbench-converter-plugin/README.md) - Maven plugin documentation
- [demo-web-app](demo-web-app/README.md) - Sample application and scenarios
- [demo-web-app-loadtest](demo-web-app-loadtest/README.md) - Integration test workflow

## How It Works

The plugin uses pure Java utilities (no Node.js required) to:

1. **Record** - BrowserMob Proxy captures browser traffic as HAR
2. **Filter** - Removes external requests (Google, analytics, etc.)
3. **Convert** - Generates k6 script from HAR
4. **Refactor** - Adds Vaadin-specific session handling:
- Dynamic JSESSIONID extraction
- CSRF token handling
- UI ID and Push ID management
- Configurable target server
- Realistic think time between user actions

## Realistic User Simulation

By default, the generated k6 scripts include realistic "think time" delays to simulate actual user behavior:

- **Page read delay**: 2-5 seconds after page loads (user reading the page)
- **Interaction delay**: 0.5-2 seconds between user actions (thinking time)

### How It Works

The plugin intelligently analyzes HAR content and timing to identify user actions:

1. **User action detection**: Analyzes UIDL request content to detect user interactions:
- Click events (button clicks, selections)
- Text input events (typing in fields)
2. **Page load detection**: Identifies v-r=init requests followed by resource loading as "page load" sequences
3. **Smart delay placement**:
- After page load completes: page read delay (user reading the page)
- After each user action: interaction delay (user thinking before next action)
4. **HAR timing awareness**: If the recorded HAR already has large gaps (> 500ms) - for example from `Thread.sleep()` or TestBench wait methods - no additional delay is added for that action

### Configuration

Configure think times via Maven properties:

```xml
<configuration>
<!-- Enable/disable think times (default: true) -->
<thinkTimeEnabled>true</thinkTimeEnabled>

<!-- Base delay after page load in seconds (default: 2.0) -->
<!-- Actual: baseDelay + random(0, baseDelay * 1.5) -->
<pageReadDelay>2.0</pageReadDelay>

<!-- Base delay after user interaction in seconds (default: 0.5) -->
<!-- Actual: baseDelay + random(0, baseDelay * 3) -->
<interactionDelay>0.5</interactionDelay>
</configuration>
```

Or via command line:

```bash
# Disable think times for maximum throughput testing
mvn k6:record -Dk6.thinkTime.enabled=false

# Custom delays
mvn k6:record -Dk6.thinkTime.pageReadDelay=3.0 -Dk6.thinkTime.interactionDelay=1.0
```

### When to Disable Think Times

Disable think times (`-Dk6.thinkTime.enabled=false`) when:
- **Maximum throughput testing**: You want to stress test the server at maximum request rate
- **TestBench tests with realistic sleeps**: If your TestBench tests already include realistic pauses using `Thread.sleep()` or TestBench wait methods, the HAR recording captures these delays. The plugin respects existing delays, but you may want to disable additional think times entirely

## Useful Links

- [Vaadin TestBench Documentation](https://vaadin.com/docs/latest/flow/testing)
- [k6 Documentation](https://grafana.com/docs/k6/latest/)
Loading
Loading