|
| 1 | +--- |
| 2 | +name: Cloud Provider Regression Test |
| 3 | +description: Connectivity, chat, JSON & streaming regression tests for all enabled cloud LLM providers |
| 4 | +version: 1.0.0 |
| 5 | +category: analysis |
| 6 | +runtime: node |
| 7 | +entry: scripts/run-regression.cjs |
| 8 | +install: npm |
| 9 | +requirements: |
| 10 | + node: ">=18" |
| 11 | + npm_install: false |
| 12 | + platforms: ["linux", "macos", "windows"] |
| 13 | +--- |
| 14 | + |
| 15 | +# Cloud Provider Regression Test |
| 16 | + |
| 17 | +Tests every enabled cloud provider for connectivity, chat completion, JSON output, and SSE streaming. |
| 18 | +Reads keys from `~/.aegis-ai/llm-config.json`. |
| 19 | + |
| 20 | +## Standalone |
| 21 | + |
| 22 | +```bash |
| 23 | +node scripts/run-regression.cjs # all providers |
| 24 | +node scripts/run-regression.cjs --provider glm,xai # specific |
| 25 | +node scripts/run-regression.cjs --verbose # full responses |
| 26 | +``` |
| 27 | + |
| 28 | +## Protocol |
| 29 | + |
| 30 | +```jsonl |
| 31 | +{"event":"ready","providers":8} |
| 32 | +{"event":"test_result","suite":"GLM","test":"chat","status":"pass","timeMs":1930} |
| 33 | +{"event":"complete","passed":14,"failed":1,"total":15,"timeMs":38000} |
| 34 | +``` |
| 35 | + |
| 36 | +## Tests Per Provider |
| 37 | + |
| 38 | +| Test | Verifies | |
| 39 | +|------|----------| |
| 40 | +| Chat | Connectivity, auth, URL construction, param compat | |
| 41 | +| JSON | Structured output (JSON instruction following) | |
| 42 | +| Stream | SSE streaming, chunks received | |
| 43 | + |
| 44 | +Results saved to `~/.aegis-ai/regression-tests/`. |
0 commit comments