Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .upstream-version-tea
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v1.2.4
v1.3.1
5 changes: 5 additions & 0 deletions plugins/bmad/_shared/tea-index.csv
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,10 @@ burn-in,Burn-in Runner,"Smart test selection, git diff for CI optimization","ci,
network-error-monitor,Network Error Monitor,"HTTP 4xx/5xx detection for UI tests","monitoring,playwright-utils,ui",extended,knowledge/network-error-monitor.md
fixtures-composition,Fixtures Composition,"mergeTests composition patterns for combining utilities","fixtures,playwright-utils",extended,knowledge/fixtures-composition.md
api-testing-patterns,API Testing Patterns,"Pure API test patterns without browser: service testing, microservices, GraphQL","api,backend,service-testing,api-testing,microservices,graphql,no-browser",specialized,knowledge/api-testing-patterns.md
pactjs-utils-overview,Pact.js Utils Overview,"Installation, contract testing flows, utility table","pactjs-utils,contract-testing,pact,api,backend,microservices",specialized,knowledge/pactjs-utils-overview.md
pactjs-utils-consumer-helpers,Pact.js Utils Consumer Helpers,"createProviderState, toJsonMap for type-safe provider states","pactjs-utils,consumer,contract-testing,pact,api",specialized,knowledge/pactjs-utils-consumer-helpers.md
pactjs-utils-provider-verifier,Pact.js Utils Provider Verifier,"buildVerifierOptions, buildMessageVerifierOptions for provider verification","pactjs-utils,provider,contract-testing,pact,api,backend,ci",specialized,knowledge/pactjs-utils-provider-verifier.md
pactjs-utils-request-filter,Pact.js Utils Request Filter,"createRequestFilter, noOpRequestFilter for auth injection","pactjs-utils,auth,contract-testing,pact",specialized,knowledge/pactjs-utils-request-filter.md
pact-mcp,Pact MCP Server,"SmartBear MCP for PactFlow: generate tests, review, can-i-deploy, provider states","pact,mcp,pactflow,contract-testing,broker",specialized,knowledge/pact-mcp.md
adr-quality-readiness-checklist,ADR Quality Readiness Checklist,"8-category 29-criteria framework for ADR testability and NFR assessment","nfr,testability,adr,quality,assessment,checklist",extended,knowledge/adr-quality-readiness-checklist.md
playwright-cli,Playwright CLI,"Token-efficient CLI for AI coding agents: element refs, sessions, snapshots, browser automation","cli,browser,agent,automation,snapshot",core,knowledge/playwright-cli.md
4 changes: 2 additions & 2 deletions plugins/bmad/skills/atdd/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description:
Generate failing acceptance tests using TDD cycle. Use when the user says 'l
ets write acceptance tests' or 'I want to do ATDD'
Generate failing acceptance tests using TDD cycle. Use when the user says "l
ets write acceptance tests" or "I want to do ATDD"
user-invocable: true
---

Expand Down
30 changes: 30 additions & 0 deletions plugins/bmad/skills/atdd/steps-c/step-01-preflight-and-context.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ If any are missing: **HALT** and notify the user.
From `{config_source}`:

- `tea_use_playwright_utils`
- `tea_use_pactjs_utils`
- `tea_pact_mcp`
- `tea_browser_automation`
- `test_stack_type`

Expand Down Expand Up @@ -111,6 +113,22 @@ Load fragments based on their `tier` classification in `tea-index.csv`:

**Detection**: Scan `{test_dir}` for files containing `page.goto` or `page.locator`. If none found, use API-only profile.

### Pact.js Utils Loading

**If `tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`, or microservices indicators detected):

Load: `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`

**If `tea_use_pactjs_utils` is disabled** but contract testing is relevant:

Load: `contract-testing.md`

### Pact MCP Loading

**If `tea_pact_mcp` is `"mcp"`:**

Load: `pact-mcp.md`

## 5. Load Knowledge Base Fragments

Use `{knowledgeIndex}` to load:
Expand Down Expand Up @@ -150,6 +168,18 @@ Use `{knowledgeIndex}` to load:
- `test-priorities-matrix.md`
- `ci-burn-in.md`

**Pact.js Utils (if enabled):**

- `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`

**Contract Testing (if pactjs-utils disabled but relevant):**

- `contract-testing.md`

**Pact MCP (if tea_pact_mcp is "mcp"):**

- `pact-mcp.md`

---

## 6. Confirm Inputs
Expand Down
4 changes: 2 additions & 2 deletions plugins/bmad/skills/automate/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description:
Expand test automation coverage for codebase. Use when the user says 'lets e
xpand test coverage' or 'I want to automate tests'
Expand test automation coverage for codebase. Use when the user says "lets e
xpand test coverage" or "I want to automate tests"
user-invocable: true
---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ If missing: **HALT** with message "Run `framework` workflow first."
### Read TEA Config Flags

- From `{config_source}` read `tea_use_playwright_utils`
- From `{config_source}` read `tea_use_pactjs_utils`
- From `{config_source}` read `tea_pact_mcp`
- From `{config_source}` read `tea_browser_automation`
- From `{config_source}` read `test_stack_type`

Expand Down Expand Up @@ -123,6 +125,24 @@ Load fragments based on their `tier` classification in `tea-index.csv`:

**Detection**: Scan `{test_dir}` for files containing `page.goto` or `page.locator`. If none found, use API-only profile.

### Pact.js Utils Loading

**If `tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`, or microservices indicators detected):

Load: `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md` (~800 lines)

**If `tea_use_pactjs_utils` is disabled** but contract testing is relevant (microservices architecture detected, existing Pact config found):

Load: `contract-testing.md` (~960 lines)

**Detection**: Scan `{project-root}` for Pact indicators: `pact/` directory, `@pact-foundation/pact` in `package.json`, `pactUrls` in test files, `PACT_BROKER` in env files.

### Pact MCP Loading

**If `tea_pact_mcp` is `"mcp"`:**

Load: `pact-mcp.md` (~150 lines) — enables agent to use SmartBear MCP tools for fetching provider states and generating pact tests during automation.

## 4. Load Knowledge Base Fragments

Use `{knowledgeIndex}` and load only what is required.
Expand All @@ -145,6 +165,18 @@ Use `{knowledgeIndex}` and load only what is required.
- `fixture-architecture.md`
- `network-first.md`

**Pact.js Utils (if enabled):**

- `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`

**Contract Testing (if pactjs-utils disabled but relevant):**

- `contract-testing.md`

**Pact MCP (if tea_pact_mcp is "mcp"):**

- `pact-mcp.md`

**Healing (if auto-heal enabled):**

- `test-healing-patterns.md`
Expand Down
15 changes: 15 additions & 0 deletions plugins/bmad/skills/automate/steps-c/step-03-generate-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@ const subprocessContext = {
config: {
test_framework: config.test_framework,
use_playwright_utils: config.tea_use_playwright_utils,
use_pactjs_utils: config.tea_use_pactjs_utils,
pact_mcp: config.tea_pact_mcp, // "mcp" | "none"
browser_automation: config.tea_browser_automation, // "auto" | "cli" | "mcp" | "none"
detected_stack: '{detected_stack}' // "frontend" | "backend" | "fullstack"
},
Expand All @@ -78,6 +80,19 @@ const subprocessContext = {

---

### Contract Test Generation Note

When `use_pactjs_utils` is enabled, the API test generation subprocess (step-03a) also generates:

- **Consumer contract tests**: Using `createProviderState` for type-safe provider states
- **Provider verification tests**: Using `buildVerifierOptions` for one-call verifier setup
- **Message contract tests**: Using `buildMessageVerifierOptions` if async/Kafka patterns detected
- **Helper files**: Request filter setup with `createRequestFilter`, shared state constants

When `pact_mcp` is `"mcp"`, the subprocess can use SmartBear MCP tools to fetch existing provider states and generate tests informed by broker data.

---

### 3. Launch Subprocess A: API Test Generation (always)

**Launch subprocess in parallel:**
Expand Down
12 changes: 11 additions & 1 deletion plugins/bmad/skills/automate/steps-c/step-03a-subprocess-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,13 @@ This is an **isolated subprocess** running in parallel with E2E test generation.

- Target features/components identified in Step 2
- Knowledge fragments loaded: api-request, data-factories, api-testing-patterns
- Config: test framework, Playwright Utils enabled/disabled
- Config: test framework, Playwright Utils enabled/disabled, Pact.js Utils enabled/disabled, Pact MCP mode
- Coverage plan: which API endpoints need testing

**Your task:** Generate API tests ONLY (not E2E, not fixtures, not other test types).

**If `use_pactjs_utils` is enabled:** Also generate consumer contract tests and provider verification tests alongside API tests. Use the loaded pactjs-utils fragments (`pactjs-utils-overview`, `pactjs-utils-consumer-helpers`, `pactjs-utils-provider-verifier`, `pactjs-utils-request-filter`) for patterns. If `pact_mcp` is `"mcp"`, use SmartBear MCP tools (Fetch Provider States, Generate Pact Tests) to inform test generation.

---

## MANDATORY EXECUTION RULES
Expand Down Expand Up @@ -88,6 +90,14 @@ test.describe('[Feature] API Tests', () => {
- ✅ Use proper TypeScript types
- ✅ Deterministic assertions (no timing dependencies)

**If Pact.js Utils enabled (from `subprocessContext.config.use_pactjs_utils`):**

- ✅ Generate consumer contract tests in `pact/http/consumer/` using `createProviderState({ name, params })` pattern
- ✅ Generate provider verification tests in `pact/http/provider/` using `buildVerifierOptions({ provider, port, includeMainAndDeployed, stateHandlers })` pattern
- ✅ Generate request filter helpers in `pact/http/helpers/` using `createRequestFilter({ tokenGenerator: () => string })`
- ✅ Generate shared state constants in `pact/http/helpers/states.ts`
- ✅ If async/message patterns detected, generate message consumer tests in `pact/message/` using `buildMessageVerifierOptions`

### 3. Track Fixture Needs

Identify fixtures needed for API tests:
Expand Down
2 changes: 1 addition & 1 deletion plugins/bmad/skills/ci/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description:
Scaffold CI/CD quality pipeline with test execution. Use when the user says
'lets setup CI pipeline' or 'I want to create quality gates'
"lets setup CI pipeline" or "I want to create quality gates"
user-invocable: true
---

Expand Down
46 changes: 46 additions & 0 deletions plugins/bmad/skills/ci/steps-c/step-02-generate-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ Include stages:

- lint
- test (parallel shards)
- contract-test (if `tea_use_pactjs_utils` enabled)
- burn-in (flaky detection)
- report (aggregate + publish)

Expand All @@ -80,6 +81,51 @@ Write the selected pipeline configuration to the resolved output path from step
- **Backend (C#/.NET)**: Use `dotnet test` with coverage, restore NuGet packages
- **Backend (Ruby)**: Use `bundle exec rspec` with coverage, cache `vendor/bundle`

### Contract Testing Pipeline (if `tea_use_pactjs_utils` enabled)

When `tea_use_pactjs_utils` is enabled, add a `contract-test` stage after `test`:

**Required env block** (add to the generated pipeline):

```yaml
env:
PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}
PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
GITHUB_SHA: ${{ github.sha }} # auto-set by GitHub Actions
GITHUB_BRANCH: ${{ github.head_ref || github.ref_name }} # NOT auto-set — must be defined explicitly
```

> **Note:** `GITHUB_SHA` is auto-set by GitHub Actions, but `GITHUB_BRANCH` is **not** — it must be derived from `github.head_ref` (for PRs) or `github.ref_name` (for pushes). The pactjs-utils library reads both from `process.env`.

1. **Consumer test + publish**: Run consumer contract tests, then publish pacts to broker
- `npm run test:contract:consumer`
- `npx pact-broker publish ./pacts --consumer-app-version=$GITHUB_SHA --branch=$GITHUB_BRANCH`
- Only publish on PR and main branch pushes

2. **Provider verification**: Run provider verification against published pacts
- `npm run test:contract:provider`
- `buildVerifierOptions` auto-reads `PACT_BROKER_BASE_URL`, `PACT_BROKER_TOKEN`, `GITHUB_SHA`, `GITHUB_BRANCH`
- Verification results published to broker when `CI=true`

3. **Can-I-Deploy gate**: Block deployment if contracts are incompatible
- `npx pact-broker can-i-deploy --pacticipant=<ServiceName> --version=$GITHUB_SHA --to-environment=production`
- Add `--retry-while-unknown 6 --retry-interval 10` for async verification

4. **Webhook job**: Add `repository_dispatch` trigger for `pact_changed` event
- Provider verification runs when consumers publish new pacts
- Ensures compatibility is checked on both consumer and provider changes

5. **Breaking change handling**: When `PACT_BREAKING_CHANGE=true` env var is set:
- Provider test passes `includeMainAndDeployed: false` to `buildVerifierOptions` — verifies only matching branch
- Coordinate with consumer team before removing the flag

6. **Record deployment**: After successful deployment, record version in broker
- `npx pact-broker record-deployment --pacticipant=<ServiceName> --version=$GITHUB_SHA --environment=production`

Required CI secrets: `PACT_BROKER_BASE_URL`, `PACT_BROKER_TOKEN`

**If `tea_pact_mcp` is `"mcp"`:** Reference the SmartBear MCP `Can I Deploy` and `Matrix` tools for pipeline guidance in `pact-mcp.md`.

---

### 4. Save Progress
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,13 @@ Define:
- Fail CI on critical test failures
- Optional: require traceability or nfr-assess output before release

**Contract testing gate** (if `tea_use_pactjs_utils` is enabled):

- **can-i-deploy must pass** before any deployment to staging or production
- Block the deployment pipeline if contract verification fails
- Treat consumer pact publishing failures as CI failures (contracts must stay up-to-date)
- Provider verification must pass for all consumer pacts before merge

---

## 3. Notifications
Expand Down
2 changes: 1 addition & 1 deletion plugins/bmad/skills/framework/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description:
Initialize test framework with Playwright or Cypress. Use when the user says
'lets setup test framework' or 'I want to initialize testing framework'
"lets setup test framework" or "I want to initialize testing framework"
user-invocable: true
---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,15 @@ Create the idiomatic test directory for the detected language:
- **Ruby (RSpec)**: `spec/` with `spec/unit/`, `spec/integration/`, `spec/api/`, `spec/support/`
- **Rust**: `tests/` for integration tests, inline `#[cfg(test)]` modules for unit tests

**If `config.tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`):

Create contract testing directory structure:

- `pact/http/consumer/` — consumer contract tests
- `pact/http/provider/` — provider verification tests and state handlers
- `pact/http/helpers/` — shared helpers (request filter, state constants)
- `pact/message/` — message/Kafka contract tests (if async patterns detected)

---

## 2. Generate Framework Config
Expand Down Expand Up @@ -122,6 +131,19 @@ Read `{config_source}` and use `{knowledgeIndex}` to load fragments based on `co

- `fixture-architecture.md`, `data-factories.md`, `network-first.md`, `playwright-config.md`, `test-quality.md`

**If Pact.js Utils enabled** (`config.tea_use_pactjs_utils`):

- `pactjs-utils-overview.md`, `pactjs-utils-consumer-helpers.md`, `pactjs-utils-provider-verifier.md`, `pactjs-utils-request-filter.md`, `contract-testing.md`
- Recommend installing `@seontechnologies/pactjs-utils` and `@pact-foundation/pact`

**If Pact.js Utils disabled but contract testing relevant:**

- `contract-testing.md`

**If Pact MCP enabled** (`config.tea_pact_mcp` is `"mcp"`):

- `pact-mcp.md`

Implement:

- Fixture index with `mergeTests`
Expand Down Expand Up @@ -158,6 +180,16 @@ Create helpers for:
- Auth helpers
- Test data factories (language-idiomatic patterns)

**If `config.tea_use_pactjs_utils` is enabled** (and `{detected_stack}` is `backend` or `fullstack`):

Create contract test samples in `pact/` directory:

- **Consumer test**: Example using `PactV3` + `createProviderState` for type-safe provider states
- **Provider verification test**: Example using `buildVerifierOptions` + `createRequestFilter`
- **Helpers**: Request filter setup (`pact/http/helpers/request-filter.ts`), shared state constants (`pact/http/helpers/states.ts`)
- **Vitest configs** (if vitest detected): `vitest.consumer.config.mts` and `vitest.provider.config.mts` for separated test execution
- **package.json scripts**: `test:contract:consumer`, `test:contract:provider`, `pact:publish`, `pact:can-deploy`

---

### 6. Save Progress
Expand Down
2 changes: 1 addition & 1 deletion plugins/bmad/skills/nfr-assess/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description:
Assess NFRs like performance security and reliability. Use when the user say
s 'lets assess NFRs' or 'I want to evaluate non-functional requirements'
s "lets assess NFRs" or "I want to evaluate non-functional requirements"
user-invocable: true
---

Expand Down
2 changes: 1 addition & 1 deletion plugins/bmad/skills/teach-me-testing/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description:
Teach testing progressively through structured sessions. Use when user says
'lets learn testing' or 'I want to study test practices'
"lets learn testing" or "I want to study test practices"
user-invocable: true
---

Expand Down
4 changes: 2 additions & 2 deletions plugins/bmad/skills/test-design/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
description:
Create system-level or epic-level test plans. Use when the user says 'lets d
esign test plan' or 'I want to create test strategy'
Create system-level or epic-level test plans. Use when the user says "lets d
esign test plan" or "I want to create test strategy"
user-invocable: true
---

Expand Down
Loading