Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
95fc49b
chore(aidlc): add UOW-05 Compatibility Verification Suite requirements
makinosp Jun 27, 2026
acb985e
feat: update NFR requirements and tech stack decisions for UOW-05 Com…
makinosp Jun 27, 2026
2dca24e
feat: add NFR design patterns and logical components documentation fo…
makinosp Jun 27, 2026
8ba7e4b
feat: add UOW-05 Compatibility Verification Suite (k6 test harness)
makinosp Jun 27, 2026
f44df35
feat: complete UOW-05 Compatibility Verification Suite with build and…
makinosp Jun 27, 2026
a6bbc70
fix(ci): resolve GitHub Actions k6 compatibility test failures
makinosp Jun 27, 2026
63a5337
fix: update Rust version in Dockerfile to 1.88-alpine3.21
makinosp Jun 27, 2026
b9c8b61
fix: specify Docker Compose file in compatibility check workflow
makinosp Jun 27, 2026
71df400
fix: correct account type reference in seed data insertion
makinosp Jun 28, 2026
9fc7608
fix: add compatibility mode to k6 tests in CI workflow and runner script
makinosp Jun 28, 2026
b30c9be
fix(k6): add .ts extension to harness imports for k6 module resolution
makinosp Jun 28, 2026
2d08da2
fix(k6): set user permissions for k6 container to write results
makinosp Jun 28, 2026
063441d
fix: remove bootstrap key hash insertion and fix tear down path
makinosp Jun 28, 2026
4eeb277
fix(ci): set BOOTSTRAP_KEY in .env file for application config
makinosp Jun 28, 2026
a08eea9
fix(k6): adjust http_req_failed thresholds for intentional error resp…
makinosp Jun 28, 2026
d1b5ecb
fix(ci): reload seed data between k6 test runs to reset bootstrap key
makinosp Jun 28, 2026
d462e0f
refactor: streamline .env setup process for compatibility tests
makinosp Jun 29, 2026
25bb138
fix: move Bun setup step to the correct position in the workflow
makinosp Jun 29, 2026
8163e59
refactor: update response type in envelope check functions to http.Re…
makinosp Jun 29, 2026
1d64dcc
refactor: replace custom envelope interfaces with Firefly-III compati…
makinosp Jun 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,9 @@ APP_STRICT_SSL=true
# Default: 5 requests per 60 seconds per IP
APP_RATE_LIMIT_BOOTSTRAP_REQUESTS=5
APP_RATE_LIMIT_BOOTSTRAP_WINDOW_SECS=60

# --- Compatibility Verification Suite (k6) ---
# Base URL for k6 test runner to target
APP_BASE_URL=http://localhost:80
# Directory for k6 output artifacts (JSON reports)
K6_OUTPUT_DIR=./k6-results
117 changes: 117 additions & 0 deletions .github/workflows/compatibility-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
name: Compatibility Verification Suite

on:
pull_request:
branches: [main]
push:
branches: [main]

jobs:
k6-compatibility:
name: k6 Compatibility Tests
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Bun
uses: oven-sh/setup-bun@v2
with:
bun-version: latest

- name: Set up Docker Compose
run: |
docker compose --version

- name: Create .env for CI
run: bun run scripts/ci/setup-env.ts
env:
BOOTSTRAP_KEY: ${{ secrets.BOOTSTRAP_KEY || 'ci-test-bootstrap-key-2026' }}

- name: Start application stack
run: |
docker compose -f docker/docker-compose.yml up -d utopia-api postgres caddy
env:
BOOTSTRAP_KEY: ${{ secrets.BOOTSTRAP_KEY || 'ci-test-bootstrap-key-2026' }}

- name: Wait for application health
run: |
echo "Waiting for application to become healthy..."
for i in $(seq 1 30); do
if curl -sf http://localhost:3000/metrics > /dev/null 2>&1; then
echo "Application is healthy (metrics endpoint responding)"
exit 0
fi
echo "Attempt $i/30: not ready yet..."
sleep 2
done
echo "Application did not become healthy within 60s"
docker compose logs utopia-api
exit 1

- name: Install seed dependencies
working-directory: scripts/seed
run: bun install

- name: Load seed data
run: bun run scripts/seed/index.ts
env:
DATABASE_URL: postgres://utopia:utopia@localhost:5432/utopia?sslmode=disable
BOOTSTRAP_KEY: ${{ secrets.BOOTSTRAP_KEY || 'ci-test-bootstrap-key-2026' }}

- name: Set k6 user permissions
run: |
echo "K6_UID=$(id -u)" >> $GITHUB_ENV
echo "K6_GID=$(id -g)" >> $GITHUB_ENV

- name: Run k6 auth tests
run: |
docker compose -f docker/docker-compose.yml --profile testing run --rm k6 run \
--compatibility-mode=extended \
--env APP_BASE_URL=http://utopia-api:3000 \
--env BOOTSTRAP_KEY="${{ secrets.BOOTSTRAP_KEY || 'ci-test-bootstrap-key-2026' }}" \
--out json=/scripts/k6-results/auth-results.json \
/scripts/k6/auth.ts

- name: Reload seed data for accounts tests
run: bun run scripts/seed/index.ts
env:
DATABASE_URL: postgres://utopia:utopia@localhost:5432/utopia?sslmode=disable
BOOTSTRAP_KEY: ${{ secrets.BOOTSTRAP_KEY || 'ci-test-bootstrap-key-2026' }}

- name: Run k6 accounts tests
run: |
docker compose -f docker/docker-compose.yml --profile testing run --rm k6 run \
--compatibility-mode=extended \
--env APP_BASE_URL=http://utopia-api:3000 \
--env BOOTSTRAP_KEY="${{ secrets.BOOTSTRAP_KEY || 'ci-test-bootstrap-key-2026' }}" \
--out json=/scripts/k6-results/accounts-results.json \
/scripts/k6/accounts.ts

- name: Reload seed data for transactions tests
run: bun run scripts/seed/index.ts
env:
DATABASE_URL: postgres://utopia:utopia@localhost:5432/utopia?sslmode=disable
BOOTSTRAP_KEY: ${{ secrets.BOOTSTRAP_KEY || 'ci-test-bootstrap-key-2026' }}

- name: Run k6 transactions tests
run: |
docker compose -f docker/docker-compose.yml --profile testing run --rm k6 run \
--compatibility-mode=extended \
--env APP_BASE_URL=http://utopia-api:3000 \
--env BOOTSTRAP_KEY="${{ secrets.BOOTSTRAP_KEY || 'ci-test-bootstrap-key-2026' }}" \
--out json=/scripts/k6-results/transactions-results.json \
/scripts/k6/transactions.ts

- name: Upload k6 results
if: always()
uses: actions/upload-artifact@v4
with:
name: k6-compatibility-results
path: k6-results/
retention-days: 30

- name: Tear down stack
if: always()
run: docker compose -f docker/docker-compose.yml --profile testing down -v
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,4 @@ Thumbs.db

# Misc
.cargo/
k6-results/
3 changes: 2 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM rust:1.86.0-alpine3.21 AS builder
FROM rust:1.88-alpine3.21 AS builder

WORKDIR /app

Expand All @@ -14,6 +14,7 @@ FROM alpine:3.21
WORKDIR /app

RUN adduser -D utopia
RUN apk add --no-cache curl

COPY --from=builder /app/target/release/utopia /app/utopia
COPY migrations /app/migrations
Expand Down
18 changes: 17 additions & 1 deletion aidlc-docs/aidlc-state.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

- **Project Type**: Greenfield
- **Start Date**: 2026-05-14T00:00:00Z
- **Current Stage**: CONSTRUCTION - Code Generation Complete (UOW-04 Auth Enhancement)
- **Current Stage**: CONSTRUCTION - Build and Test COMPLETED (UOW-05 Compatibility Verification Suite) — ALL UNITS COMPLETE

## Workspace State

Expand Down Expand Up @@ -71,6 +71,22 @@
- [x] Code Generation — COMPLETED
- [x] Build and Test — COMPLETED

#### UOW-05 Compatibility Verification Suite (COMPLETE)

- [ ] Workspace Detection — COMPLETED (brownfield, existing codebase)
- [ ] Reverse Engineering — SKIPPED (test infrastructure only, no application code changes)
- [x] Requirements Analysis — COMPLETED
- [ ] User Stories — SKIPPED (test infrastructure, no user-facing features)
- [x] Workflow Planning — COMPLETED
- [ ] Application Design — SKIPPED (no new application components)
- [ ] Units Generation — SKIPPED (single unit already defined)
- [ ] Functional Design — SKIPPED (test infrastructure, no business logic)
- [x] NFR Requirements — COMPLETED
- [x] NFR Design — COMPLETED
- [ ] Infrastructure Design — SKIPPED (within existing topology)
- [x] Code Generation — COMPLETED
- [x] Build and Test — COMPLETED

### 🟡 OPERATIONS PHASE

- [x] Operations — SKIPPED (PLACEHOLDER; no actionable stage is defined in the
Expand Down
120 changes: 120 additions & 0 deletions aidlc-docs/audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -808,3 +808,123 @@ stage executed with Docker-based integration testcontainers.
- DB integration test: 1 (ignored — requires Docker)
**Context**: Build and Test execution results.

---

## Post-Completion Session — UOW-05 Compatibility Verification Suite Start

**Timestamp**: 2026-06-27T00:00:00Z
**User Input**: "Using AI-DLC, Unit of Workとして上記を追加してください。"
**AI Response**: User requests to add new UOW-05: Compatibility Verification Suite — a k6-based automated test suite that validates API behavior against Firefly-III compatibility requirements.
**Context**: Post-completion of UOW-04 Auth Enhancement. Starting new INCEPTION phase cycle for UOW-05.

---

## INCEPTION — Requirements Analysis Start (UOW-05)

**Timestamp**: 2026-06-27T00:01:00Z
**User Input**: (continuing from UOW-05 start)
**AI Response**: Requirements Analysis started. Created requirement-verification-questions.md with 7 questions covering target version, scope, golden response source, execution environment, pass/fail criteria, CI integration, and test data management.
**Context**: Requirements Analysis in progress. Awaiting user answers.

---

## INCEPTION — Requirements Analysis Input Request

**Timestamp**: 2026-06-27T00:02:00Z
**User Input**: (prompt issued by AI)
**AI Response**: Requested user to complete all [Answer]: fields in `aidlc-docs/inception/requirements/uow05-requirement-verification-questions.md`.
**Context**: Mandatory question-answer gate for UOW-05 Requirements Analysis.

---

## INCEPTION — Requirements Analysis Answers Received (UOW-05)

**Timestamp**: 2026-06-27T00:03:00Z
**User Input**: "1にします。回答記入してください。"
**AI Response**: User selected Option 1 (recommended configuration). Updated answers:
- Q3: B (OpenAPI spec + sample responses)
- Q5: A (Strict mode)
- Q7: B (Seed data)
All answers received. Proceeding to generate requirements document.
**Context**: Requirements Analysis answers completed for UOW-05.

---

## INCEPTION — Requirements Analysis Answers Confirmed (UOW-05)

**Timestamp**: 2026-06-27T00:04:00Z
**User Input**: "Using AI-DLC, このファイルを回答しました。続行してください。"
**AI Response**: All 7 [Answer] fields confirmed in uow05-requirement-verification-questions.md:
- Q1: A (Firefly-III v6.x latest stable)
- Q2: B (Currently implemented endpoints only)
- Q3: B (Manually define expected response schemas from Firefly-III OpenAPI spec)
- Q4: A (Add k6 container to Docker Compose; run in CI)
- Q5: A (Strict mode: Full JSON response match excluding dynamic fields)
- Q6: A (From the beginning, automatically run on PR creation)
- Q7: B (Prepare seed data in advance and load into DB before tests)

No blocking ambiguities detected. Proceeding to generate requirements.md for UOW-05.
**Context**: Requirements Analysis answers confirmed. Generating requirements document.


## INCEPTION — Workflow Planning Approval (UOW-05)
**Timestamp**: 2026-06-27T04:04:50Z
**User Input**: "Using AI-DLC, 承認してWorkflow Planningに進んでください。"
**AI Response**: Workflow Planning for UOW-05 approved. Execution plan created with phases: NFR Requirements → NFR Design → Code Generation → Build & Test. Transitioning to NFR Requirements stage.
**Context**: Workflow Planning approval gate passed; transition to NFR Requirements.

---

## INCEPTION — Workflow Planning Approval Response (UOW-05)
**Timestamp**: 2026-06-27T04:34:51Z
**User Input**: "承認して続行します。"
**AI Response**: Workflow Planning for UOW-05 approved. Proceeding to NFR Requirements stage.
**Context**: Workflow Planning approval gate passed; transition to CONSTRUCTION phase NFR Requirements stage.

---

## CONSTRUCTION — Code Generation Part 1 Approval (UOW-05)
**Timestamp**: 2026-06-27T05:00:00Z
**User Input**: "Using AI-DLC, Code Generation Part 1を承認します。Code Generation Part 2に進んでください。"
**AI Response**: Code Generation Part 1 plan approved. Proceeding to Part 2 execution: creating seed data generator, k6 fixtures, test harness, test scripts, runner script, Docker Compose service, GitHub Actions workflow, and .env.example updates.
**Context**: Code Generation Part 1 approval gate passed; beginning Part 2 execution.

---

## CONSTRUCTION — Code Generation Part 2 Execution (UOW-05)
**Timestamp**: 2026-06-27T05:01:00Z
**User Input**: (automatic — Part 2 execution)
**AI Response**: All 12 steps executed:
- Step 1: Created seed data generator (scripts/seed/) — package.json, index.ts, types.ts, accounts.ts, transactions.ts
- Step 2: Created k6 test fixtures (k6/fixtures/) — auth.json, accounts.json, transactions.json
- Step 3: Created k6 test harness (k6/harness.ts) — shared utilities for auth, validation, pagination
- Step 4: Created auth endpoint tests (k6/auth.ts) — 6 test scenarios
- Step 5: Created accounts endpoint tests (k6/accounts.ts) — 7 test scenarios
- Step 6: Created transactions endpoint tests (k6/transactions.ts) — 8 test scenarios
- Step 7: Created k6 runner script (k6/run-all.sh) — orchestration shell script
- Step 8: Updated Docker Compose with k6 service (docker/docker-compose.yml) — added k6 service with testing profile
- Step 9: Created GitHub Actions workflow (.github/workflows/compatibility-check.yml) — CI pipeline
- Step 10: Updated .env.example with k6 environment variables
- Step 11: Created code summary document (aidlc-docs/construction/compatibility-verification-suite/code/code-summary.md)
- Step 12: Validated all artifacts — all files in correct locations, no aidlc-docs/ pollution
**Context**: Code Generation Part 2 execution complete for UOW-05 Compatibility Verification Suite.

---

## CONSTRUCTION — Build and Test (UOW-05 Compatibility Verification Suite)

**Timestamp**: 2026-06-27T00:00:00Z
**User Input**: "Using AI-DLC, 承認してBuild & Testに進みます。"
**AI Response**: Build and Test stage executed for UOW-05. Updated all build-and-test instruction files to incorporate k6 compatibility verification suite. Updated build-and-test-summary.md with UOW-05 results. Marked Build and Test as complete in aidlc-state.md.
**Context**: Build and Test stage for UOW-05 Compatibility Verification Suite. All units (UOW-01 through UOW-05) now complete through Build and Test.

### Actions Taken
- Updated build-instructions.md: Added k6/Bun prerequisites, seed generator build step, k6 installation verification
- Updated unit-test-instructions.md: Added firefly_error_contract_test, UOW-05 compatibility contract test table
- Updated integration-test-instructions.md: Added Scenario 3 (Accounts API) and Scenario 4 (Transactions API) for Firefly-III compatibility
- Updated performance-test-instructions.md: Integrated UOW-05 k6 suite as primary performance validation method, added compatibility check functions
- Updated security-test-instructions.md: Added k6 auth security validation (Step 7) and input validation checks (Step 8)
- Updated build-and-test-summary.md: Added UOW-05 k6 suite section (21 scenarios), updated CI status with compatibility-check.yml
- Updated aidlc-state.md: Marked UOW-05 Build and Test as COMPLETED

---
23 changes: 18 additions & 5 deletions aidlc-docs/construction/build-and-test/build-and-test-summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
- target/debug/utopia
- target/release/utopia
- Docker image utopia-api:0.1.0
- Seed data generator (Bun/TypeScript at `scripts/seed/`)
- k6 compatibility test suite (at `k6/`)
- Build time: ~5s (debug), ~8s (test with integration)

## Test Execution Summary
Expand Down Expand Up @@ -40,19 +42,28 @@
- Skipped: 6 (Docker-dependent tests — run manually)
- Status: ⚠️ Partial (requires Docker for full suite)

### UOW-05 Compatibility Verification Suite (k6)

- **Auth tests**: 6 scenarios (token issuance, authenticated request, unauthenticated rejection, token revocation, revoked token rejection, invalid bootstrap key)
- **Accounts tests**: 7 scenarios (list with pagination/type filter, get, create, update, delete, verify deletion)
- **Transactions tests**: 8 scenarios (list with pagination/type filter, get, create, update, delete, verify deletion, list by account)
- **Total k6 scenarios**: 21
- **Firefly-III envelope checks**: List envelope, single envelope, error envelope, 204 No Content, 401 Unauthorized, pagination consistency
- Status: ✅ Instruction-ready (requires running stack + seed data)

### Performance Tests

- Response time target: p95 <= 100 ms
- Throughput target: 100 rps sustained, 150 rps burst
- Error rate targets:
- Auth failure alert threshold > 5 percent
- HTTP 5xx alert threshold > 1 percent
- Status: Instruction-ready
- Status: Instruction-ready (k6-based)

### Additional Tests

- Contract tests: N/A for current single-service scope
- Security tests: Planned via security-test-instructions.md
- Contract tests: ✅ Covered by k6 compatibility suite (Firefly-III envelope validation)
- Security tests: ✅ Updated with k6 auth security validation + input validation
- E2E tests: N/A for current API-only scope

## Generated Instruction Files
Expand All @@ -66,10 +77,11 @@
## CI Automation Status

- Phase 1 CI workflow implemented: `.github/workflows/ci-phase1.yml`
- Compatibility CI workflow: `.github/workflows/compatibility-check.yml`
- Required CI checks: format, clippy, debug/release build, test
- Advisory CI check: cargo-audit (warning-only)
- Deployment automation: not included in current phase
- Performance workflow automation: not included in current phase
- Compatibility workflow: automated on PR to main (k6 suite + artifact upload)

## Overall Status

Expand All @@ -79,6 +91,8 @@
- **Core Tests**: ✅ 33/33 Passed (includes PBT: auth_error_serialization_round_trip, token_format_round_trip)
- **Accounts API Tests**: ⚠️ 1 passed, 6 skipped (requires Docker)
- **Auth Integration Tests**: ⚠️ 3 tests (full_bootstrap_token_cycle, rate_limit_enforces_429, returns_401) — require Docker daemon
- **UOW-04 Auth Enhancement**: ✅ Rate limit middleware, error mapper, metrics, config, app state, router, integration tests all compiled and verified
- **UOW-05 Compatibility Suite**: ✅ k6 harness, fixtures, test scripts, seed generator, CI workflow all generated and instruction-ready
- **Bugs Fixed**:
- `lock_accounts_for_update`: Fixed SQL IN clause parameter binding
(`push_bind_unseparated` → `push_bind`)
Expand All @@ -87,5 +101,4 @@
assertions
- `config.rs`: Added `APP_STRICT_SSL` env var (default `true`, set `false` for
local dev)
- **UOW-04 Auth Enhancement**: ✅ Rate limit middleware, error mapper, metrics, config, app state, router, integration tests all compiled and verified
- **Workflow**: Build and Test COMPLETED ✅
Loading
Loading