11# CLI E2E 测试规范
22
3+ ## 架构分层
4+
5+ | 层级 | 路径 | 测什么 |
6+ | --------------- | ----------------------------------------------------- | --------------------------------------------------------------- |
7+ | ** 共享基建** | ` packages/e2e ` | gating、子进程 runner、output、globalSetup(` private ` ,不发布) |
8+ | ** 命令 E2E** | ` packages/commands/tests/e2e ` | help、缺参、dry-run、live(gated);harness ` binName: "bl" ` |
9+ | ** bl smoke** | ` packages/cli/tests/e2e/registry.smoke.e2e.test.ts ` | 产品 map 全部 path ` --help ` 、分组 help、根 help |
10+ | ** kscli smoke** | ` packages/kscli/tests/e2e/registry.smoke.e2e.test.ts ` | 6 条 flat path ` --help ` ;` search --help ` 与 harness 一致 |
11+ | ** runtime** | ` packages/runtime/tests ` | ` proxy.e2e ` 、console 跨域 flag 拒绝 |
12+
13+ ** 依赖边界** :` e2e ` → ` core ` ;` commands/tests ` → ` e2e ` + ` commands/src ` ;产品 tests → ` e2e ` + 各自 ` src ` 。** 禁止** 产品 import ` commands/tests/** ` (子进程 spawn harness 路径除外)。
14+
315## 触发条件
416
517- 新增/修改 ` packages/commands/src/commands ` 下的 command 实现
618- 新增/修改 ` packages/cli/src/commands.ts ` 的 ` bl ` 命令路径 map
7- - 新建或扩展 ` packages/cli /tests/e2e/* .e2e.test.ts ` 用例
8- - 为命令补 help / 缺参 / dry-run / 真实集成测试
19+ - 新建或扩展 ` packages/commands /tests/e2e/<topic> .e2e.test.ts `
20+ - 新增 bl 产品 path → 同步 ` registry.smoke ` (自动覆盖 leaf path)与 ` harness/e2e-command-map.ts `
921
10- 以上情况必须同步维护 ` packages/cli/tests/e2e/<topic>.e2e.test.ts ` 。 跑测与环境变量见 ` .cursor/skills/bailian-cli-e2e/SKILL.md ` 。
22+ 跑测与环境变量见 ` .cursor/skills/bailian-cli-e2e/SKILL.md ` 。
1123
1224## 文件与工具
1325
14- - 路径:` packages/cli/tests/e2e/<kebab-topic>.e2e.test.ts `
15- - 框架:` vite-plus/test ` ;子进程跑 CLI:` runCli ` from ` ./helpers.ts `
26+ ### commands E2E
27+
28+ - 路径:` packages/commands/tests/e2e/<kebab-topic>.e2e.test.ts `
29+ - 子进程:` runCommandE2e ` from ` ./helpers.ts ` (spawn ` harness/main.ts ` )
30+ - fixtures:` packages/commands/tests/e2e/fixtures/ `
31+ - map:` harness/e2e-command-map.ts ` (自维护,初始从 ` cli/commands.ts ` 复制)
32+
33+ ### 产品 smoke
34+
35+ - bl:` runCli ` from ` packages/cli/tests/e2e/helpers.ts `
36+ - kscli:` runKscli ` from ` packages/kscli/tests/e2e/helpers.ts `
37+
38+ ### 共享
39+
40+ - gating / output / runner:` e2e/gating ` 、` e2e/output ` 、` e2e/runner `
41+ - globalSetup:根 ` vite.config.ts ` → ` packages/e2e/src/global-setup.ts `
1642- 解析 JSON stdout:` parseStdoutJson ` ;输出目录:` makeE2eOutputDir(e2eLabelFromMetaUrl(import.meta.url)) `
1743- 长任务:` cliTimeoutPrefix() ` ;视频用例加 ` test(..., 3_600_000) ` 等显式超时
1844
@@ -33,24 +59,31 @@ describe.skipIf(<ready>)("e2e: <topic>(DashScope …)", () => {
3359});
3460```
3561
36- ## skip 条件(helpers.ts )
62+ ## skip 条件(` e2e/gating ` ,commands helpers re-export )
3763
38- | 场景 | 条件 |
39- | ------------------- | ----------------------------------------------------- |
40- | 文本/搜索/记忆/配置 | ` isDashScopeE2EReady() ` |
41- | 图像/语音 | ` isBailianE2EMediaEnabled() && isDashScopeE2EReady() ` |
42- | 视频 | ` isBailianE2EVideoEnabled() && isDashScopeE2EReady() ` |
43- | 知识库 | ` isKnowledgeE2EReady() ` |
44- | 视频 download/task | 另需 ` BAILIAN_E2E_VIDEO_TASK_ID ` |
64+ | 场景 | 条件 |
65+ | ----------------------- | ----------------------------------------------------- ----------------------------------------------------- |
66+ | 文本/搜索/记忆/配置 | ` isDashScopeE2EReady() ` |
67+ | 图像/语音 | ` isBailianE2EMediaEnabled() && isDashScopeE2EReady() ` |
68+ | 视频 | ` isBailianE2EVideoEnabled() && isDashScopeE2EReady() ` |
69+ | 视频 download/task | 另需 ` BAILIAN_E2E_VIDEO_TASK_ID ` |
70+ | 知识库 chat/search live | ` isChatE2EReady() ` / ` isSearchE2EReady() ` ( ` knowledge chat/search ` ,需 ` BAILIAN_WORKSPACE_ID ` + agent ID) |
4571
4672## 用例类型
4773
48- 1 . ** 分组 help** :` runCli (["image"])` → ` exitCode === 0 ` ,stdout+stderr 含子命令名
49- 2 . ** --help** :` runCli ([..., "--help"])` → stderr 含主要 flags
74+ 1 . ** 分组 help** :` runCommandE2e (["image"])` → ` exitCode === 0 ` ,stdout+stderr 含子命令名
75+ 2 . ** --help** :` runCommandE2e ([..., "--help"])` → stderr 含主要 flags
50763 . ** 缺参** :带一个无害全局 flag(如 ` --quiet ` )且不传 required flag → ` exitCode === 2 ` ,stderr 匹配 ` --flag|Missing required argument `
51774 . ** --dry-run** :仅当实现在联网/上传/写盘** 之前** 返回;断言 stdout JSON/文本,不入网
52785 . ** 真实集成** :保留既有用例名称与断言;放在 skip 块** 末尾**
5379
80+ ## 契约与防漂移
81+
82+ - ` harness/e2e-command-map.contract.test.ts ` :path 唯一、合法 export、白名单、测试 argv 前缀在 map 中
83+ - Advisory(不阻塞 CI):` node tools/compare-e2e-command-map.ts ` 对比 ` cli/commands.ts ` 与 harness map
84+
85+ 增删命令须同步三处:** 实现 export** + ** ` cli/commands.ts ` ** + ** ` e2e-command-map.ts ` ** (见 [ command-add-remove.md] ( command-add-remove.md ) )。
86+
5487## 安全与例外
5588
5689- ** 禁止真实破坏性操作** :` auth logout ` 只用 ` --dry-run ` ;` config set ` 只用 ` --dry-run `
@@ -60,24 +93,35 @@ describe.skipIf(<ready>)("e2e: <topic>(DashScope …)", () => {
6093
6194## 新增 command 检查清单
6295
63- - [ ] ` packages/commands/src/index.ts ` 导出 + ` packages/cli/src/commands.ts ` 暴露路径 + ` tests/e2e/<topic>.e2e.test.ts ` (新建或扩展)
96+ - [ ] ` packages/commands/src/index.ts ` 导出 + ` packages/cli/src/commands.ts ` 暴露路径 + ` harness/e2e-command-map.ts ` 登记
97+ - [ ] ` packages/commands/tests/e2e/<topic>.e2e.test.ts ` (新建或扩展)
6498- [ ] 若改了 ` usageArgs ` / ` flags ` / ` exampleArgs ` ,跑 ` pnpm --filter bailian-cli run generate:reference ` 更新 ` skills/bailian-cli/reference/ ` 并提交
6599- [ ] 顶层:分组 help + 子命令 ` --help ` (多子命令则各一条 help)
66100- [ ] skip 块:每个 required flag 缺参;可 dry-run 则加一条
67101- [ ] 至少一条真实集成(或说明为何仅 smoke);不破坏已有集成用例顺序
68- - [ ] ` pnpm test packages/cli/tests/e2e/<file> ` 通过
102+ - [ ] ` vp test packages/commands/tests/e2e/<file> ` 通过
103+
104+ ## 调试命令
105+
106+ ``` sh
107+ pnpm --filter bailian-cli-commands exec vp test packages/commands/tests/e2e/text-chat.e2e.test.ts
108+ pnpm --filter bailian-cli exec vp test packages/cli/tests/e2e/registry.smoke.e2e.test.ts
109+ pnpm --filter knowledge-studio-cli exec vp test packages/kscli/tests/e2e/registry.smoke.e2e.test.ts
110+ pnpm --filter bailian-cli-runtime exec vp test packages/runtime/tests/proxy.e2e.test.ts
111+ node tools/compare-e2e-command-map.ts
112+ ```
69113
70114## 示例片段
71115
72116``` ts
73117test (" foo bar 缺少 --prompt 时退出为用法错误 (2)" , async () => {
74- const { stderr, exitCode } = await runCli ([" foo" , " bar" , " --quiet" ]);
118+ const { stderr, exitCode } = await runCommandE2e ([" foo" , " bar" , " --quiet" ]);
75119 expect (exitCode ).toBe (2 );
76120 expect (stderr ).toMatch (/ --prompt| Missing required argument/ i );
77121});
78122
79123test (" foo bar --dry-run 仅输出计划" , async () => {
80- const { stdout, stderr, exitCode } = await runCli ([
124+ const { stdout, stderr, exitCode } = await runCommandE2e ([
81125 " foo" ,
82126 " bar" ,
83127 " --dry-run" ,
@@ -97,4 +141,4 @@ test("foo bar --dry-run 仅输出计划", async () => {
97141- ** E2E** :单条/少量调用、断言固定、可进 ` vp test ` (见上文 skip 条件)
98142- ** 批量压测** :` packages/cli/tests/stress/run.mjs ` + ` targets/*.mjs ` ,并发 + 报告,** 仅手动** ` pnpm run test:stress -- <target> `
99143
100- 勿把压测并入 E2E 或默认 CI。详见 [ stress-batch-tests.md] ( stress-batch-tests.md ) 。
144+ 勿把压测并入 E2E 或默认 CI。详见 [ stress-batch-tests.md] ( stress-batch-tests.md ) .
0 commit comments