Skip to content

Commit ca18f58

Browse files
vbilopavvbilopav
committed
latest results link
1 parent 08b7d42 commit ca18f58

56 files changed

Lines changed: 3437 additions & 745 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎README.md‎

Lines changed: 156 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,183 @@
1-
# Load Performance Testing for Web APIs Returning a Single PostgreSQL Function
1+
# Load Performance Testing for Web APIs Returning PostgreSQL Functions
22

3-
This project performs load performance testing for Web APIs on different tech stacks that will execute a single [PostgreSQL function](https://github.com/vb-consulting/pg_function_load_tests/blob/master/src/postgres/init.sql) and return the result.
3+
This project performs load performance testing for Web APIs on different tech stacks that execute PostgreSQL functions and return the results. The goal is to **measure framework overhead**, not database performance - all test functions use `generate_series()` for constant, predictable database response times.
44

5-
## Running Manually
5+
## Frameworks Tested
66

7-
```
7+
| Framework | Version | Port | Language |
8+
|-----------|---------|------|----------|
9+
| Django | 6.0.1 | 8000 | Python |
10+
| FastAPI | 0.128.0 | 8001 | Python |
11+
| Fastify | 5.7.1 | 3101 | Node.js |
12+
| Bun | 1.3.3 | 3104 | Bun/TypeScript |
13+
| Go (net/http) | 1.25 | 5200 | Go |
14+
| Spring Boot | 4.0.1 | 5400 | Java 24 |
15+
| Actix-web | 1.91.1 | 5300 | Rust |
16+
| Swoole | 6.0 | 3103 | PHP |
17+
| PostgREST | 14.3 | 3000 | Haskell |
18+
| .NET 9 Minimal API (EF) | 9.0 | 5002 | C# |
19+
| .NET 10 Minimal API (EF) | 10.0 | 5003 | C# |
20+
| .NET 10 Minimal API (Dapper) | 10.0 | 5004 | C# |
21+
| NpgsqlRest (AOT) | 3.4.5 | 5005 | C# |
22+
| NpgsqlRest (JIT) | 3.4.5 | 5006 | C# |
23+
24+
## Benchmark Scenarios
25+
26+
All scenarios use memory-only PostgreSQL functions (no table I/O) to isolate framework overhead:
27+
28+
| Scenario | Endpoint | Tests |
29+
|----------|----------|-------|
30+
| **perf_test** | `GET /api/perf-test` | Comprehensive data type serialization (23 types) |
31+
| **perf_minimal** | `GET /api/perf-minimal` | Pure routing overhead baseline |
32+
| **perf_post** | `POST /api/perf-post` | JSON request body parsing |
33+
| **perf_nested** | `GET /api/perf-nested` | Nested JSON object serialization |
34+
| **perf_large_payload** | `GET /api/perf-large-payload` | Large response streaming/buffering |
35+
| **perf_many_params** | `GET /api/perf-many-params` | Query string parsing (20 parameters) |
36+
37+
## Quick Start
38+
39+
### Prerequisites
40+
41+
- Docker and Docker Compose
42+
- Git
43+
44+
### Clone and Build
45+
46+
```bash
847
git clone https://github.com/vb-consulting/pg_function_load_tests.git
9-
cd pg_function_load_tests
10-
cd src
48+
cd pg_function_load_tests/src
1149
docker-compose down && docker-compose up --build --detach
1250
```
1351

14-
Please wait a few seconds for all services to initialize properly (health check isn't implemented yet), and then run from the same directory:
52+
Wait for health checks to pass (all services depend on PostgreSQL being ready).
53+
54+
### Verify Services
55+
56+
Test all 14 services and 84 endpoints:
57+
58+
```bash
59+
./test-services.sh
60+
```
61+
62+
Or test against a remote server:
1563

64+
```bash
65+
./test-services.sh your-server-ip
1666
```
17-
docker-compose exec test /bin/sh /scripts/run.sh
67+
68+
## Running Benchmarks
69+
70+
### Test Profiles
71+
72+
The benchmark runner supports three profiles:
73+
74+
| Profile | Purpose | Duration | Use Case |
75+
|---------|---------|----------|----------|
76+
| **minimal** | Markdown validation | ~2 min | Test output format before deploying |
77+
| **local** | Development testing | ~30 min | Validate changes locally |
78+
| **server** | Production benchmarks | ~4 hours | Full benchmark on dedicated server |
79+
80+
### Local Testing
81+
82+
**Quick markdown validation** (minimal profile):
83+
```bash
84+
docker-compose exec test /bin/sh -c "PROFILE=minimal /scripts/run-all.sh"
1885
```
1986

20-
Keep it running on server:
87+
**Development testing** (local profile - default):
88+
```bash
89+
docker-compose exec test /bin/sh /scripts/run-all.sh
2190
```
22-
tmux # Start a new tmux session
23-
# Run your command
24-
docker-compose exec test /bin/sh /scripts/run.sh
25-
# Press Ctrl+B, then D to detach
91+
92+
**Run specific scenario only**:
93+
```bash
94+
docker-compose exec test /bin/sh -c "SCENARIO=minimal /scripts/run-all.sh"
2695
```
2796

28-
Results will be saved to the `src/k6/results/` directory under the current timestamp subdirectory.
97+
Available scenarios: `all`, `perf-test`, `minimal`, `post`, `nested`, `large`, `params`
98+
99+
### Server Testing
100+
101+
For production benchmarks on a dedicated server (e.g., Hetzner CCX33):
102+
103+
```bash
104+
# Start a tmux session (survives SSH disconnection)
105+
tmux
106+
107+
# Run full benchmark suite
108+
docker-compose exec test /bin/sh -c "PROFILE=server /scripts/run-all.sh"
109+
110+
# Detach: Press Ctrl+B, then D
111+
# Reattach later: tmux attach
112+
```
113+
114+
**Server profile settings:**
115+
- 60s test duration per combination
116+
- VUs: 1, 50, 100, 200 (up to 500 for minimal baseline)
117+
- 30s sleep between tests (TCP TIME_WAIT clearance)
118+
- JIT warmup phase before benchmarks
119+
120+
### Results
121+
122+
Results are saved to `src/_k6/results/<timestamp>/`:
123+
- `<timestamp>_all.md` - Unified summary with all results
124+
- Individual test summaries for each service/scenario combination
125+
126+
## Architecture
127+
128+
### PostgreSQL Configuration
129+
130+
```yaml
131+
postgres:
132+
image: postgres:17.2-alpine
133+
command: postgres -c 'max_connections=2000'
134+
```
135+
136+
**Connection calculation:**
137+
- Tests are serialized (one service at a time)
138+
- Active service: 100 connections (pool max)
139+
- 13 idle services: ~200 connections
140+
- Total during test: ~300 connections
141+
- Setting 2000 provides 6x headroom
142+
143+
### Test Isolation
144+
145+
Tests are **serialized** - only one service is tested at a time. This ensures:
146+
- No resource contention between services
147+
- Clean baseline for each test
148+
- Accurate framework overhead measurement
149+
150+
The 30s sleep between tests (server profile) allows:
151+
- TCP TIME_WAIT connections to clear
152+
- Connection pools to stabilize
153+
- CPU/memory to return to baseline
154+
- JIT-compiled code to cool down
155+
156+
## Manual Testing
157+
158+
Use `test.http` with VS Code REST Client or similar tools to manually test individual endpoints.
159+
160+
Example:
161+
```http
162+
### Django - perf_test
163+
GET http://localhost:8000/api/perf-test?_records=1&_text=test&...
164+
165+
### PostgREST - perf_minimal
166+
GET http://localhost:3000/rpc/perf_minimal
167+
```
29168

30169
## Latest Results
31170

171+
- [PostgreSQL REST API Benchmark 2025](https://npgsqlrest.github.io/blog/postgresql-rest-api-benchmark-2025.html)
32172
- [Test Branch](https://github.com/vb-consulting/pg_function_load_tests/tree/202412302119)
33173
- [Test Results Raw Output](https://github.com/vb-consulting/pg_function_load_tests/blob/202412302119/src/_k6/results/202412302119.md)
34-
- [Parsed Tests Results Disucssion Thread](https://github.com/vb-consulting/pg_function_load_tests/discussions/8)
174+
- [Parsed Tests Results Discussion Thread](https://github.com/vb-consulting/pg_function_load_tests/discussions/8)
35175
- [Interactive Chart](https://vb-consulting.github.io/blog/npgsqlrest/load-test/)
36176

37177
## License
38178

39179
MIT
40180

41-
## 🤝 Contributing
181+
## Contributing
42182

43183
Contributions are welcome! Please feel free to submit a Pull Request.

0 commit comments

Comments
 (0)