Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Copilot Instructions

This is a Test-Agent project with a defined Skill → Agent → Utils execution framework.

## When generating code or answering questions:

1. **Read Skill docs first** — Check `skills/<task>.md` to understand the workflow before generating any test code or calling agents.
2. **Follow the framework** — Do not skip the Skill workflow layer. Agent definitions are in `agents/`, utils in `utils/`.
3. **Use existing utils** — The project has 79 utility modules in `utils/`. Prefer using them over generating new code.
4. **Match the project structure** — Test outputs go to `workspace/`, config in `config/`, source code in `src/`.
5. **Config first** — Check `.env.example` for required environment variables before generating code that depends on them.

## Standalone CLI

The project has a CLI at `runtime/cli/main.py` (entry point: `tagent`):
```bash
tagent run "path/to/prd.md"
tagent catalog
tagent doctor
```

## Key directories

| What | Where |
|------|-------|
| Agent definitions | `agents/` |
| Skill workflow docs | `skills/` |
| Python utils | `utils/` |
| Runtime (CLI + MCP) | `runtime/` |
| Config templates | `config/` |
| Test outputs | `workspace/` |
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ jobs:
echo "Agents=$AGENTS Skills=$SKILLS Utils=$UTILS"
[ "$AGENTS" = "16" ] || { echo "❌ Agents count mismatch (expected 16, got $AGENTS)"; exit 1; }
[ "$SKILLS" -eq "32" ] || { echo "❌ Skills count mismatch (expected 32, got $SKILLS)"; exit 1; }
[ "$UTILS" -eq "78" ] || { echo "❌ Utils count mismatch (expected 78, got $UTILS)"; exit 1; }
[ "$UTILS" -eq "79" ] || { echo "❌ Utils count mismatch (expected 79, got $UTILS)"; exit 1; }
echo "✅ File counts correct"

- name: 验证 .gitignore 排除源 MD
Expand Down
51 changes: 48 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,57 @@ htmlcov/

# workspace 测试产出(敏感 + 体积大)
workspace/测试数据/
workspace/automotive/
workspace/测试用例/
workspace/测试计划/
workspace/需求分析/
workspace/执行日志/
workspace/测试报告/
workspace/audit/
workspace/_outputs/
workspace/feedback/
workspace/自动化脚本/

# 但保留 workspace/执行日志/baselines/(性能基线需提交)
!workspace/执行日志/baselines/
# 测试报告 — 排除产物,保留 baselines/
# 根级直接文件(临时报告、自检产物等)
workspace/测试报告/*.md
workspace/测试报告/*.docx
workspace/测试报告/*.json
# 旧式子目录(v1.42 前,无项目名层级)
workspace/测试报告/allure-results/
workspace/测试报告/allure-report/
workspace/测试报告/jmeter-results/
workspace/测试报告/jmeter-report/
workspace/测试报告/decisions/
workspace/测试报告/evidence/
workspace/测试报告/reports/
workspace/测试报告/security/
workspace/测试报告/history/
workspace/测试报告/quarantine/
workspace/测试报告/soak/
workspace/测试报告/ai-*/
workspace/测试报告/web-vitals/
workspace/测试报告/silent-failures/
workspace/测试报告/audit/
workspace/测试报告/screenshots/
# 按 run_id 隔离的产出目录(时间戳格式: YYYYMMDD_HHMMSS_xxxxxx)
workspace/测试报告/*/2*/
# 项目级产出子目录({项目名}/ 下)
workspace/测试报告/*/screenshots/
workspace/测试报告/*/allure-results/
workspace/测试报告/*/allure-report/
workspace/测试报告/*/jmeter-results/
workspace/测试报告/*/jmeter-report/
workspace/测试报告/*/decisions/
workspace/测试报告/*/evidence/
workspace/测试报告/*/reports/
workspace/测试报告/*/security/
workspace/测试报告/*/history/
workspace/测试报告/*/quarantine/
workspace/测试报告/*/soak/
workspace/测试报告/*/ai-*/
workspace/测试报告/*/web-vitals/
workspace/测试报告/*/silent-failures/
workspace/测试报告/*/audit/

# ===== 敏感配置 =====
.env
Expand All @@ -41,6 +84,8 @@ workspace/自动化脚本/
Thumbs.db

# ===== Claude Code =====
.claude/agents/
.claude/skills/
.claude/cache/
.claude/sessions/

Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ repos:
# 文件统计校验(防误删 agent/skill/utils)
- id: file-count-check
name: Agents/Skills/Utils 数量校验
entry: bash -c 'A=$(ls agents/[0-9]*.md 2>/dev/null | wc -l); S=$(ls skills/*.md 2>/dev/null | grep -v README | wc -l); U=$(find utils -name "*.py" ! -name "__init__.py" 2>/dev/null | wc -l); [ "$A" = "16" ] || { echo "❌ Agents count mismatch (expected 16, got $A)"; exit 1; }; [ "$S" -eq "32" ] || { echo "❌ Skills count mismatch (expected 32, got $S)"; exit 1; }; [ "$U" -eq "78" ] || { echo "❌ Utils count mismatch (expected 78, got $U)"; exit 1; }; echo "✅ File counts correct"'
entry: bash -c 'A=$(ls agents/[0-9]*.md 2>/dev/null | wc -l); S=$(ls skills/*.md 2>/dev/null | grep -v README | wc -l); U=$(find utils -name "*.py" ! -name "__init__.py" 2>/dev/null | wc -l); [ "$A" = "16" ] || { echo "❌ Agents count mismatch (expected 16, got $A)"; exit 1; }; [ "$S" -eq "32" ] || { echo "❌ Skills count mismatch (expected 32, got $S)"; exit 1; }; [ "$U" -eq "79" ] || { echo "❌ Utils count mismatch (expected 79, got $U)"; exit 1; }; echo "✅ File counts correct"'
language: system
pass_filenames: false
always_run: true
Expand Down
14 changes: 0 additions & 14 deletions .vscode/extensions.json

This file was deleted.

53 changes: 0 additions & 53 deletions .vscode/launch.json

This file was deleted.

39 changes: 0 additions & 39 deletions .vscode/settings.json

This file was deleted.

78 changes: 78 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# AGENTS.md

## AI AGENT — STOP. READ THIS FIRST.

**This is a Test-Agent project. It has a defined execution framework. You MUST follow it.**

- Do NOT call or spawn sub-agents without reading the relevant Skill doc first.
- Do NOT skip the Skill workflow layer.
- Do NOT execute test workflows directly — read the Skill doc and follow its agent call sequence.

**Correct flow:**
1. Read `skills/<task>.md` to understand the workflow
2. Follow the Skill doc's step-by-step agent call sequence
3. Use Agent tools ONLY when and how the Skill doc says

**If no Skill doc matches the task:** fall back to `skills/test-coordinator.md`.

---

## Two Ways to Use Test-Agent

### Way 1: Standalone CLI (no AI required)

```bash
cd project-dir
pip install -e runtime/ # first time only
tagent run "path/to/prd.md" # router + orchestrator end-to-end
tagent catalog # list 16 experts + 32 skills
tagent doctor # health check
```

### Way 2: AI Agent Collaboration

```bash
cd project-dir
# Read skills/smoke-test.md first, then follow workflow
# Call sub-agents per Skill doc sequence (requirements-analyst → testcase-designer → ...)
# Outputs in workspace/
```

---

## Architecture

```
Standalone CLI (tagent) AI Agent Mode
│ │
▼ ▼
runtime/router Skill docs (skills/*.md)
│ │
▼ ▼
runtime/orchestrator Agent defs (agents/*.md)
│ │
▼ ▼
utils/*.py (79 modules) utils/*.py (79 modules)
```

Both paths converge at the utils execution layer.

## Directory Map

| What | Where |
|------|-------|
| Agent definitions | `agents/` (16 agents) |
| Skill workflow docs | `skills/` (35 skills) |
| Python utils | `utils/` (79 modules) |
| Runtime (CLI + orchestrator + MCP) | `runtime/` |
| Config templates | `config/` (incl. `.env.example`) |
| Test outputs | `workspace/` |
| CI pipelines | `ci/` |
| Docs | `docs/` |

## Design Notes

- Slash commands like `/smoke-test` are Skill workflow documents, not registered commands.
- Agent definitions contain Python import hints as prompts for AI agents — not actual executable code.
- Pentest workflows require `tagent.yml` with `pentest.authorized: true`.
- MCP server (`python -m runtime.mcp.test_orchestrator.server`) provides catalog/plan/run/status/report tools.
8 changes: 4 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -320,8 +320,8 @@ _后续累积变更入此节;切版本时移到下方版本节。_
- `workspace/需求分析/requirements_summary.json`(需求摘要 + P0/P1 + 风险区)
- `workspace/测试计划/automation_scripts_plan.json`(脚本规划 + fixture 复用)
- `workspace/测试计划/execution_plan.json`(4 阶段 + 失败 4 分类 + Flaky 规则)
- `workspace/测试报告/bug_drafts.json`(BugTracker-ready Bug 草案 + severity 1-4)
- `workspace/测试报告/decisions/final_verdict_*.json`(test-lead go/no-go 决策,标 `requires_human_signoff: true`)
- `workspace/测试报告/{项目名}/bug_drafts.json`(BugTracker-ready Bug 草案 + severity 1-4)
- `workspace/测试报告/{项目名}/decisions/final_verdict_*.json`(test-lead go/no-go 决策,标 `requires_human_signoff: true`)
- **录制脚本**(`scripts/`):
- `_demo-commands.sh`:实际 demo 命令序列(被 record-demo-* 调)
- `record-demo-asciinema.sh`:`asciinema rec` 自动录,产 .cast 可上传 asciinema.org 或转 GIF/SVG
Expand Down Expand Up @@ -382,7 +382,7 @@ _后续累积变更入此节;切版本时移到下方版本节。_
- `00-项目导航.md` · `agents/{01,07,08,09}.md` · `agents/README.md` · `skills/{README,test-coordinator,zentao-bug-submission}.md` · `config/mcp-server-impl.md` · `utils/{README.md,api_retry_util.py}` · `ci/{INDEX,CICD集成说明}.md` · `docs/getting-started/{交付物清单,使用手册,配置清单}.md` · `examples/web-demo/README.md` · `CONTRIBUTING.md` · `FULL_GUIDE.md`
- **adapter 修 V1.10 n7 bug**:`runtime/orchestrator/adapters/experts.py` 加 `SCRIPT_DEFAULT_ARGS` + `_ensure_fixture()` 通用机制
- 现 `tagent selftest --e2e --strict` **100% PASS 8/8**(原 88% 7/8)
- generate_report.py 默认注入 `--data=workspace/测试报告/_selftest_summary.json`,fixture 自动生成
- generate_report.py 默认注入 `--data=workspace/测试报告/{项目名}/_selftest_summary.json`,fixture 自动生成
- **主宪章扩**:
- §36 多端通知 canon(扩 §6,6 渠道权威清单 + env 字段 + 业务语言铁律)
- §37 BugTracker canon(扩 §12,6 adapter 权威清单 + measurement env + 措辞规范)
Expand Down Expand Up @@ -542,7 +542,7 @@ _后续累积变更入此节;切版本时移到下方版本节。_
- `runtime/mcp/defect_tracker/`:工单桥 5 工具(create/get/update/query_bugs/list_trackers),默认 zentao + 预留扩展位(主宪章 §12 契约)
- `runtime/mcp/knowledge_base/`:pgvector 向量检索 4 工具(embed/index_case/index_defect/search_similar),LiteLLM embedding + stub 兜底
- `runtime/mcp/compliance_checker/`:行业合规规则 3 工具(list_profiles/get_profile/check_compliance);10 框架 profile 起步空载(SOC2/PCI-DSS/HIPAA/IEC 62304/IEC 61508/ISO 26262/DO-178C/GDPR/PIPL/CCPA)
- 共享基类 `runtime/mcp/base.py`:make_server / run_stdio / @tool_decision_logged(决策落 `workspace/测试报告/decisions/` 符合主宪章 §18-12)
- 共享基类 `runtime/mcp/base.py`:make_server / run_stdio / @tool_decision_logged(决策落 `workspace/测试报告/{项目名}/decisions/` 符合主宪章 §18-12)
- **行业合规规则插槽** `profiles/compliance/`:10 框架空载示例 YAML,真规则由领域专家+test-lead 双签签字后入库
- **飞轮回灌路由**(M2-9):`runtime/router/retrieval.py` 历史相似用例 → LLM prompt few-shot;router 透明集成,无 KB 时降级
- **真模型路由测试套件**(M2-7):`runtime/tests/test_router_real.py` 20 样本(4 类型 × 5)真模型测试;门槛单模型 ≥85%、双模型投票 ≥95%;无 API key 自动 skip;失败自动落 decisions/ 含 seed+模型版本+输入快照(主宪章 §21 横切准则)
Expand Down
Loading
Loading