|
| 1 | +# Test Migration Playbook: Legacy → regressionTest |
| 2 | + |
| 3 | +Migrate `packages/bridge/test/legacy/*.test.ts` to the `regressionTest` framework. |
| 4 | + |
| 5 | +## Prerequisites |
| 6 | + |
| 7 | +- Read `packages/bridge/test/utils/regression.ts` (the framework — DO NOT EDIT) |
| 8 | +- Read `packages/bridge/test/utils/bridge-tools.ts` (test multitools) |
| 9 | +- Study `packages/bridge/test/coalesce-cost.test.ts` as the gold-standard example |
| 10 | + |
| 11 | +## Step-by-step process |
| 12 | + |
| 13 | +### 1. Categorise every test in the legacy file |
| 14 | + |
| 15 | +Read the file and sort each test into one of these buckets: |
| 16 | + |
| 17 | +| Bucket | Action | |
| 18 | +| ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------- | |
| 19 | +| **Parser-only** (parses AST, checks wire structure) | DELETE — regressionTest's `parse → serialise → parse` covers this automatically | |
| 20 | +| **Serializer roundtrip** (parse → serialize → parse) | DELETE — regressionTest does this automatically | |
| 21 | +| **Runtime execution** (runs bridge, asserts data/errors) | MIGRATE to `regressionTest` scenarios | |
| 22 | +| **Non-runtime tests** (class constructors, pure unit tests) | MOVE to the corresponding package test dir (e.g. `bridge-core/test/`, `bridge-parser/test/`) | |
| 23 | +| **Tests requiring custom execution** (AbortSignal, custom contexts) | Keep using `forEachEngine` in the new file | |
| 24 | + |
| 25 | +### 2. Design bridges for regressionTest |
| 26 | + |
| 27 | +Group related runtime-execution tests into **logical regressionTest blocks**. Each block has: |
| 28 | + |
| 29 | +```typescript |
| 30 | +regressionTest("descriptive name", { |
| 31 | + bridge: ` |
| 32 | + version 1.5 |
| 33 | + bridge Operation.field { |
| 34 | + with test.multitool as a |
| 35 | + with input as i |
| 36 | + with output as o |
| 37 | + // ... wires |
| 38 | + } |
| 39 | + `, |
| 40 | + tools, // import { tools } from "./utils/bridge-tools.ts" |
| 41 | + scenarios: { |
| 42 | + "Operation.field": { |
| 43 | + "scenario name": { input: {...}, assertData: {...}, assertTraces: N }, |
| 44 | + }, |
| 45 | + }, |
| 46 | +}); |
| 47 | +``` |
| 48 | + |
| 49 | +**Design rules:** |
| 50 | + |
| 51 | +- One regressionTest can have **multiple bridges** (multiple operations in scenarios) |
| 52 | +- Group by **feature/behavior** (e.g. "throw control flow", "continue/break in arrays") |
| 53 | +- Each bridge needs enough scenarios to achieve **traversal coverage** (all non-error paths hit) |
| 54 | +- Keep bridge definitions minimal — test one concept per wire |
| 55 | + |
| 56 | +### 3. Use test.multitool everywhere possible |
| 57 | + |
| 58 | +The multitool (`with test.multitool as a`) is a passthrough: input → output (minus `_`-prefixed keys). |
| 59 | + |
| 60 | +**Capabilities:** |
| 61 | + |
| 62 | +- `_error`: `input: { a: { _error: "boom" } }` → tool throws `Error("boom")` |
| 63 | +- `_delay`: `input: { a: { _delay: 100, name: "A" } }` → delays 100ms, returns `{ name: "A" }` |
| 64 | +- All other `_` keys are stripped from output |
| 65 | +- Correctly handles nested objects and arrays |
| 66 | + |
| 67 | +**Wiring pattern:** |
| 68 | + |
| 69 | +``` |
| 70 | +a <- i.a // sends i.a as input to tool, tool returns cleaned copy |
| 71 | +o.x <- a.y // reads .y from tool output |
| 72 | +``` |
| 73 | + |
| 74 | +**Only use custom tool definitions when:** |
| 75 | + |
| 76 | +- You need a tool that transforms data (not passthrough) |
| 77 | +- You need AbortSignal handling on the tool side |
| 78 | +- You need `ctx.signal` inspection |
| 79 | + |
| 80 | +### 4. Write scenarios |
| 81 | + |
| 82 | +Each scenario needs: |
| 83 | + |
| 84 | +| Field | Required | Description | |
| 85 | +| ---------------- | -------- | ----------------------------------------------------------------------- | |
| 86 | +| `input` | Yes | Input object passed to bridge | |
| 87 | +| `assertTraces` | Yes | Number of tool calls (or function for custom check) | |
| 88 | +| `assertData` | No | Expected output data (object or function) | |
| 89 | +| `assertError` | No | Expected error (regex or function) — mutually exclusive with assertData | |
| 90 | +| `fields` | No | Restrict which output fields are resolved | |
| 91 | +| `context` | No | Context values (for `with context as ctx`) | |
| 92 | +| `tools` | No | Per-scenario tool overrides | |
| 93 | +| `allowDowngrade` | No | Set `true` if compiler can't handle this bridge feature | |
| 94 | +| `assertGraphql` | No | GraphQL-specific expectations (object or function) | |
| 95 | +| `assertLogs` | No | Log assertions | |
| 96 | + |
| 97 | +**assertData shorthand:** For simple cases, use object literal: |
| 98 | + |
| 99 | +```typescript |
| 100 | +assertData: { name: "Alice", age: 30 } |
| 101 | +``` |
| 102 | + |
| 103 | +**assertError with regex:** Matches against `${error.name} ${error.message}`: |
| 104 | + |
| 105 | +```typescript |
| 106 | +assertError: /BridgeRuntimeError/; // matches error name |
| 107 | +assertError: /name is required/; // matches error message |
| 108 | +assertError: /BridgePanicError.*fatal/; // matches both |
| 109 | +``` |
| 110 | + |
| 111 | +**assertError with function** (for instanceof checks): |
| 112 | + |
| 113 | +```typescript |
| 114 | +assertError: (err: any) => { |
| 115 | + assert.ok(err instanceof BridgePanicError); |
| 116 | + assert.equal(err.message, "fatal"); |
| 117 | +}; |
| 118 | +``` |
| 119 | + |
| 120 | +**fields for isolating wires:** When one wire throws but others don't, use `fields` to test them separately: |
| 121 | + |
| 122 | +```typescript |
| 123 | +"error on fieldA only": { |
| 124 | + input: { ... }, |
| 125 | + fields: ["fieldA"], // only resolve this field |
| 126 | + assertError: /message/, |
| 127 | + assertTraces: 0, |
| 128 | +}, |
| 129 | +``` |
| 130 | + |
| 131 | +### 5. Handle traversal coverage |
| 132 | + |
| 133 | +The framework automatically checks that all non-error traversal paths are covered. Common uncovered paths: |
| 134 | + |
| 135 | +- **empty-array**: Add a scenario with an empty array: `input: { a: { items: [] } }` |
| 136 | +- **Fallback paths**: Add a scenario where each fallback fires |
| 137 | +- **Short-circuit paths**: Add scenarios for each branch of ||/?? chains |
| 138 | + |
| 139 | +If traversal coverage fails, the error message tells you exactly which paths are missing. |
| 140 | + |
| 141 | +### 6. Handle compiler downgrade |
| 142 | + |
| 143 | +The compiled engine doesn't support all features. When the compiler downgrades, add `allowDowngrade: true` to the scenario. Common triggers: |
| 144 | + |
| 145 | +- `?.` (safe execution modifier) without `catch` |
| 146 | +- Some complex expressions |
| 147 | +- Certain nested array patterns |
| 148 | + |
| 149 | +**Important:** `allowDowngrade` applies per-scenario, but the bridge is shared. If ANY wire in the bridge triggers downgrade, ALL scenarios need `allowDowngrade: true`. |
| 150 | + |
| 151 | +### 7. Handle errors in GraphQL |
| 152 | + |
| 153 | +as graphql has partial errors then we need to assert it separately |
| 154 | + |
| 155 | +```typescript |
| 156 | +assertGraphql: { |
| 157 | + fieldA: /error message/i, // expect GraphQL error for this field |
| 158 | + fieldB: "fallback-value", // expect this value |
| 159 | +} |
| 160 | +``` |
| 161 | + |
| 162 | +### 8. Move non-runtime tests |
| 163 | + |
| 164 | +Tests that don't invoke the bridge execution engine belong in the corresponding package: |
| 165 | + |
| 166 | +| Test type | Target | |
| 167 | +| ------------------------ | -------------------------------------------------- | |
| 168 | +| Error class constructors | `packages/bridge-core/test/execution-tree.test.ts` | |
| 169 | +| Parser AST structure | `packages/bridge-parser/test/` | |
| 170 | +| Serializer output format | `packages/bridge-parser/test/` | |
| 171 | +| Type definitions | `packages/bridge-types/test/` | |
| 172 | + |
| 173 | +### 9. Final verification |
| 174 | + |
| 175 | +```bash |
| 176 | +pnpm build # 0 type errors |
| 177 | +pnpm lint # 0 lint errors |
| 178 | +pnpm test # 0 failures |
| 179 | +``` |
| 180 | + |
| 181 | +Run the specific test file first for fast iteration: |
| 182 | + |
| 183 | +```bash |
| 184 | +node --experimental-transform-types --test packages/bridge/test/<new-file>.test.ts |
| 185 | +``` |
| 186 | + |
| 187 | +## Migration checklist template |
| 188 | + |
| 189 | +For each legacy test file: |
| 190 | + |
| 191 | +- [ ] Read and categorise all tests |
| 192 | +- [ ] Delete parser-only and roundtrip tests (covered by regressionTest) |
| 193 | +- [ ] Design bridges using test.multitool |
| 194 | +- [ ] Write scenarios with correct assertions |
| 195 | +- [ ] Ensure traversal coverage (add empty-array, fallback scenarios) |
| 196 | +- [ ] Add `allowDowngrade: true` where compiler downgrades |
| 197 | +- [ ] Handle GraphQL replay bugs with `assertGraphql: () => {}` |
| 198 | +- [ ] Move non-runtime tests to corresponding package |
| 199 | +- [ ] Keep tests needing custom execution (AbortSignal) using `forEachEngine` |
| 200 | +- [ ] Verify: `pnpm build && pnpm lint && pnpm test` |
| 201 | +- [ ] Don't delete the legacy file until confirmation |
| 202 | + |
| 203 | +## Files remaining to migrate |
| 204 | + |
| 205 | +``` |
| 206 | +packages/bridge/test/legacy/ # check for remaining legacy tests |
| 207 | +packages/bridge/test/expressions.test.ts # if still using forEachEngine |
| 208 | +packages/bridge/test/infinite-loop-protection.test.ts # if still using forEachEngine |
| 209 | +``` |
0 commit comments