diff --git a/.agents/skills/ship/SKILL.md b/.agents/skills/ship/SKILL.md index 54940fd9e58..8d30b1b63d2 100644 --- a/.agents/skills/ship/SKILL.md +++ b/.agents/skills/ship/SKILL.md @@ -100,6 +100,16 @@ improvement(scope): description for enhancements chore(scope): description for maintenance ``` +## What to Omit + +The repo is public. Keep the title and description to the code change and its reasoning — never: + +- Customer, company, or user names; workspace/user/org IDs; email addresses +- Prod or staging operational data: log lines, DB rows, metrics, timestamps, incident details, canary/alert output +- Infrastructure specifics: hostnames, ARNs, internal URLs, env var values, secret names + +Describe the bug by its mechanism, not by how you found it. "Expired OAuth credentials fail to refresh in the worker" — not "the Sheets canary failed at 16:31Z for workspace abc-123". + ## PR Description Format Use this exact template in the user's voice (concise, bullet points): diff --git a/.agents/skills/tool-registry-boundary/SKILL.md b/.agents/skills/tool-registry-boundary/SKILL.md index a1463f1c8fa..6e1caaf0a64 100644 --- a/.agents/skills/tool-registry-boundary/SKILL.md +++ b/.agents/skills/tool-registry-boundary/SKILL.md @@ -68,6 +68,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/.agents/skills/v2-api-conventions/SKILL.md b/.agents/skills/v2-api-conventions/SKILL.md new file mode 100644 index 00000000000..55da39cd239 --- /dev/null +++ b/.agents/skills/v2-api-conventions/SKILL.md @@ -0,0 +1,233 @@ +--- +name: v2-api-conventions +description: The response, error, pagination, and validation contract every `/api/v2` endpoint must satisfy. Use when adding or changing a route under `apps/sim/app/api/v2/`, or when auditing one for conformance. +argument-hint: +--- + +# v2 API Conventions + +The v2 surface makes one promise: **every response is the same two shapes, and a caller-supplied value can never produce a 500.** + +``` +success (single) { "data": {...} } +success (collection) { "data": [...], "nextCursor": "..." | null } +failure (always) { "error": { "code": "...", "message": "...", "details"?: ... } } +``` + +Nothing else at the top level. No `success: true`, no bare `{ "error": "string" }`, no HTML. + +That promise is worth stating as a rule because it has been broken five separate ways, each time by a route or a builder taking a shortcut that looked local: + +- `GET /workflows?limit=1.5` returned **500**. The contract was copied from a sibling and lost its `.int()`, so a fractional limit passed validation and reached Postgres as `LIMIT 2.5`. +- A malformed JSON body returned **`{"error":"Request body must be valid JSON"}`** — a bare string. The envelope was a per-route opt-in that only 8 routes remembered. +- `GET /api/v2/nonexistent` returned a **full HTML 404 document**, because no route file matched and the request fell through to the app's global not-found page. +- Four collections returned `nextCursor` while **silently discarding** any `limit` the caller sent, because Zod strips unknown keys unless the schema is `.strict()`. +- Handing back a `nextCursor` from any timestamp-sorted list and passing it straight in returned **500**. The value was validated and bound — but bound with no SQL type, into `date_trunc`, which is overloaded, so Postgres could resolve no overload. Validation was never the missing half; the type was. + +Each was one line. The rules below are the generalisations. + +## Where the machinery lives + +| Concern | File | +|---|---| +| Envelope + error codes + cursor codecs | `apps/sim/app/api/v2/lib/response.ts` | +| Rollout gate (`v2-api` flag) | `apps/sim/app/api/v2/lib/gate.ts` | +| Cross-tenant concealment | `apps/sim/lib/api/server/routes/resource-concealment.ts` | +| Route builder | `apps/sim/lib/api/server/routes/v2-json-route.ts` | +| Contracts | `apps/sim/lib/api/contracts/v2/**` | +| Shared list/keyset helpers | `apps/sim/lib/api/list-query.ts` | + +## Rule 1 — the envelope is produced by helpers, never by hand + +`v2Data`, `v2CursorList`, and `v2Error` in `response.ts` are the only things that build a v2 body. They also set `Cache-Control: private, no-store`, which every v2 response needs because every v2 response is authed per-caller data. + +A route built with `defineV2JsonRoute` gets this for free: its `present` returns the *body shape* and the builder renders it. Never call `NextResponse.json` from a v2 route. + +**The envelope must hold for every failure mode, including the ones that happen before your handler runs.** That is what the four bugs above have in common. Defaults for the transport-level failures live on the builder — `v2PayloadTooLargeResponse` (413) and `v2InvalidJsonResponse` (400) — precisely so a route cannot forget them. + +## Rule 2 — status codes mean specific things + +| Status | `code` | Meaning | +|---|---|---| +| 200 / 201 | — | Success. 201 only for a created resource. | +| 400 | `BAD_REQUEST` | Contract validation. Carries field-level `details` from `serializeZodIssues`. | +| 401 | `UNAUTHORIZED` | No/!valid API key. **Runs before the rollout gate.** | +| 403 | `FORBIDDEN` | Authenticated, same tenant, insufficient rights. Where the cause is one a caller can act on it is named in `details.code`, from the closed set in `lib/core/application/forbidden.ts` (e.g. `INSUFFICIENT_WORKSPACE_ROLE`, `PERSONAL_API_KEYS_DISABLED`). A few domain refusals still reach the wire without one. | +| 404 | `NOT_FOUND` | Not found, **and** cross-tenant concealment, **and** the rollout gate, **and** an unknown path. | +| 409 | `CONFLICT` | Uniqueness/state conflict, human-readable message. | +| 413 | `PAYLOAD_TOO_LARGE` | Body over the route's `maxBodyBytes` — **and** a collection the response must materialize that is over *its* ceiling. Fourteen bodyless `GET`/`DELETE` operations publish it for the folder-tree cap (`FolderCollectionLimitExceededError`) or the rendered-artifact cap. | +| 429 | `RATE_LIMITED` | With `Retry-After` and `X-RateLimit-*`. | +| 500 | `INTERNAL_ERROR` | Genuine server fault only. Message is always generic. | + +Two of these carry real design weight: + +**404 is deliberately overloaded.** A workspace the caller cannot reach answers `404 "Workspace not found"`, never 403 — a 403 would confirm the resource exists. `createV2ResourceConcealmentPolicy` does this by mapping a cross-tenant authorization failure to `v2Error('NOT_FOUND', ...)`. The rollout gate answers the same way for the same reason (`gate.ts`: "an ungated caller cannot distinguish 'not in the rollout cohort' from 'no such endpoint'"), and so does the unknown-path catch-all at `app/api/v2/[[...segments]]/route.ts` — its body is byte-identical to the gate's on purpose. + +**500 is never caller-reachable.** Any input a caller can send must be rejected at the contract boundary with a 400. If you can construct a query string or body that produces a 500, that is a bug in the contract, not something to wrap in a `try`/`catch`. `v2ErrorForOrchestration` also replaces the message on an unclassified failure with a generic one, so internal detail never leaks. A caller-reachable 500 has shipped three times — a fractional `limit` reaching `LIMIT 2.5`, a plain `HEAD` tripping the builder's method guard, and a keyset cursor's timestamp reaching `date_trunc` as an untyped placeholder — so treat "a well-formed request produced a 500" as the highest-severity class of defect on this surface. + +**Validating a value is only half of it; the value also has to reach SQL with a type.** A bound parameter arrives as `unknown` and takes its type from context. Against a typed column (`sort_order > $1`) that inference always succeeds, which is why the gap stays invisible almost everywhere — but as an argument to an overloaded function it can resolve to nothing at all. So: **if a bound value is an argument to a SQL function rather than one side of a comparison, write its type down** (`lib/api/list-query.ts`, `timestampKey`, casts from the column). + +And this class survives a green test suite — `keysetAfter` returned well-formed SQL and every assertion passed; only Postgres's parser rejected it. When a change alters the *shape* of generated SQL rather than its values, execute it somewhere before believing the suite. + +**Which of 403 and 404 an operation documents follows from its authorization, not from whether it is a read.** `requirePermission` throws two different failures: no workspace access at all is `NoWorkspaceAccessError`, which `createV2ResourceConcealmentPolicy` conceals as 404; access below the operation's `minimumRole` is `InsufficientWorkspacePermissionsError`, which stays a 403. So: + +- An operation whose `minimumRole` is `write` or `admin` can always 403 — a member with a lower role hits it. Document 403. +- An operation whose `minimumRole` is `read` cannot 403 *that* way, because `read` is the floor of the `read < write < admin` ordering and anyone without access is concealed as 404 instead. It can still 403 through `PersonalApiKeysDisabledError` (a personal API key against a workspace whose organization disabled them) or `WorkspaceApiKeyAuthorizationError` (`workspaceApiKey: 'deny'`), and every v2 operation is reachable by a personal API key. **So in practice every workspace-scoped v2 operation documents 403**, and the reads that omitted it were wrong, not principled. + +**A 403 a caller can act on names its cause in `error.details.code` — but not every 403 does yet.** One status covers several different remedies — raise a member's role, issue a personal key instead of a workspace-scoped one, re-point a workspace key, buy an enterprise plan, delete a resource to get under a quota — and prose is not branchable, so a client that must tell them apart was string-matching messages, which turns every reword into a silent break. A handful of domain refusals still throw a bare `OrchestrationError('forbidden', …)` and reach the wire with no code, so **write client code that treats `details.code` as optional**, and read `openapi/shared.ts`'s `FORBIDDEN_DESCRIPTION` for the current position rather than assuming the sweep is finished. For code you are *writing*, the rule below is unconditional. + +The vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES` in `lib/core/application/forbidden.ts`, with a `Record` of descriptions beside it that the generated OpenAPI 403 description is built from. Adding a member fails to compile until it is documented, so a code cannot reach the wire unpublished. Do not invent a code at a route: throw `ForbiddenOperationError(code, message)` from the domain and let `v2CaughtOrchestrationError` — the function every v2 error policy falls through to — attach it. `InsufficientWorkspacePermissionsError`, `PersonalApiKeysDisabledError`, `WorkspaceApiKeyAuthorizationError`, and `PrincipalKindAuthorizationError` already carry theirs. + +The cross-tenant refusals (`NoWorkspaceAccessError`, `WorkspaceApiKeyScopeAuthorizationError`, `DelegatedWorkspaceAuthorizationError`) deliberately carry **no** code. They are concealed as 404, and naming their cause would hand back the resource-existence signal the concealment exists to withhold. + +Use the shared sets in `contracts/v2/openapi/shared.ts` — `RESOURCE_ERRORS`, `RESOURCE_CONFLICT_ERRORS`, `RESOURCE_MUTATION_ERRORS` — rather than assembling a per-operation list; all three already include `Forbidden`, and hand-assembled lists are how three knowledge reads and three upload operations quietly lost it. + +**HEAD is answered by the `GET` handler, not rejected.** Next aliases a missing `HEAD` export onto `GET` and drops the body when sending, so a route's `GET` legitimately runs with `request.method === 'HEAD'`. The builders' method guard accepts that pairing via `methodMatchesContract`; any other mismatch stays a hard error. Never hand-write a `HEAD` export to "fix" this. + +**A `GET` with side effects must declare `headSafe: false`.** The aliasing above is only sound because RFC 9110 §9.2.1 defines `HEAD` as safe. A `GET` that writes a row or opens an outbound connection is not, and an uptime monitor or link checker walking the documented URL list would drive those effects on every probe — `GET /files/{fileId}` records a `FILE_DOWNLOADED` audit event, so a `HEAD` used to fabricate a download that never happened. Both the JSON and binary builders take `headSafe`: a route that sets it `false` still authenticates and rate-limits a `HEAD`, then answers `v2HeadNoEffect()` — a bodiless 200 — before parsing or executing. Nothing observable is lost, because `HEAD` carries no body either way. Audit this whenever a read acquires an audit projection or an outbound call. + +## Rule 3 — a collection that returns `nextCursor` must accept `limit` + `cursor`, and must apply them + +Every list returns `{ data, nextCursor }`. Whether it *pages* is a separate, pinned decision — see `lib/api/contracts/v2/__tests__/list-pagination.test.ts`, which enumerates both sets and fails when a new list is in neither. + +Build the query slice from the shared helper, never by hand: + +```ts +...v2PaginationFields({ description: 'Maximum widgets to return per page.' }) +``` + +That gives `limit` (integer, 1..`V2_MAX_PAGE_SIZE`, defaulting to `V2_DEFAULT_PAGE_SIZE` = 50) and an opaque `cursor`. Re-declaring `limit: z.coerce.number()...` inline is how the 500 happened; there is one schema so the family cannot drift again. + +Three cursor schemes exist. Two are the shared codecs in `response.ts`, both opaque base64-JSON, and which of them you use is decided by what the read can express, not by taste: + +- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort AND the filters are stamped into the cursor and re-checked on replay, so changing `sortBy` or any filter mid-pagination is a 400, not a silently skipped page. +- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results. + +Both take the same two stamps: `cursorSortKey(sortBy, sortOrder)` for the ordering, and `cursorFilterScope({ ... })` for every param that filters the sequence. **`limit` is never a stamp** — it selects how much of the sequence to return, not what the sequence is, and binding it strands every cursor the moment a caller changes page size. Params that only shape the response body are out for the same reason. + +The third is **per-domain**: a list whose read predates the shared codecs, or whose page boundary is not expressible as one, mints its own — a bare `encodeCursor({ version })` on `GET /workflows/{id}/versions` and `encodeCursor({ email })` on the workspace member list, the local codecs in `lib/audit-logs/query.ts`, `lib/logs/list-logs.ts`, and `lib/table/rows/cursor.ts`, and a usage-event id passed straight through by `GET /billing/logs`. Those tokens stay opaque and untouched, but a domain-minted cursor on a list a caller can re-filter is wrapped at the surface with `encodeScopedCursor(cursorFilterScope({...}), token)` and unwrapped with `readScopedCursor`, so it carries the same binding as the shared schemes. **A new list picks one of the two shared schemes.** Do not add a fourth. + +Every paged list's binding is declared in `lib/api/contracts/v2/__tests__/list-pagination.test.ts` and checked against what the contract actually accepts, in both directions. A new list, or a new filter on an existing one, fails that test until its binding is declared or the param is explicitly recorded as unable to change the sequence. + +**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure. + +Return `nextCursor: null` on the last page and only then. Never construct a cursor client-side. + +**Ordering is `sortBy` + `sortOrder`, except where there is nothing to sort by.** Fourteen lists take the pair. Two — `GET /logs` and `GET /workflows/{id}/runs` — have exactly one sortable column (start time), so there is no `sortBy` to pair with and the direction rides on a single `order` param; `sortBy`/`sortOrder` are not accepted there. That split is documented in both contracts and is the *only* sanctioned deviation. A new list picks the pair. Do not "fix" the two by accepting `sortOrder` as an alias: an alias is a second spelling of one thing with undefined precedence when both arrive, which is its own inconsistency, and renaming `order` would break every shipped caller. + +**A boolean query param is a real boolean**, declared with `booleanQueryFlagSchema` from `contracts/primitives.ts`. It coerces `'true'`/`'1'` and `'false'`/`'0'`/`''`, so it is a strict widening of a `z.enum(['true','false'])` — which is what two v2 params used to be, purely by inheritance from the internal shapes they reused. Reusing an internal `.shape.x` inherits the internal spelling; re-declare instead when the internal one is not the v2 convention. + +## Rule 4 — reject what you do not implement + +**Every contract declares a `query`, even when the endpoint takes none** — `query: noInputSchema` (`z.object({}).strict()`), never omission. `parseRequest` validates the query slice only when the contract declares one, so an omitted `query` means "never look at the query string", not "takes no query params". The two were indistinguishable, which is how 69 contracts ended up accepting anything without anyone deciding they should: `GET /workflows/{id}?bogus=1` answered 200 while every list answered 400 for the same shape. `query-declaration.test.ts` sweeps the tree so contract 70 fails at authoring time rather than shipping unvalidated. + +Declaring them is a **deliberate tightening** of endpoints that previously ignored an unknown param. It was weighed and kept: the v2 body slice on those same endpoints was already strict, so the split was arbitrary rather than a promise to callers; a mistyped param that is silently dropped is the bug class this rule exists to prevent; and no first-party caller sends an undeclared v2 query param (the two SDKs send only `includeOutput`/`selectedOutputs`, both declared; the UI makes no v2 calls at all; `requestJson` appends nothing implicitly and no v2 cache buster exists). Third-party callers appending a tracking tag or cache buster do break, which is why `api-reference/getting-started.mdx` documents the behavior rather than leaving it to be discovered from a 400. + +Query and body schemas are **`.strict()`** — and `.strict()` binds the **top level only**. A strict body containing a non-strict nested object still drops unknown keys one level down, which is the headline `filter` bug at a smaller scale: `sort: [{ field, direction, nulls: 'last' }]` answered 200 and ordered by the default. Strictness belongs on the shared nested schema (`sortSpecSchema`'s element, `tableViewConfigSchema`), not restated per body. + +Before tightening a schema that is **also** a response or a stored blob, make the read canonical first. `table_views.config` is schemaless JSONB, so a legacy row carrying a retired key would fail a newly strict response parse and become a 500; `normalizeStoredViewConfig` projects the stored blob onto the declared keys so the tightening is safe in both directions. Zod strips unknown keys by default, so a non-strict schema answers `?limit=1` with 200 and the whole set — the caller believes it bounded the response and it did not. That is a contract lie, and on an uncapped list it is also an unbounded-response risk. + +Error messages name the field and, where there is one, the escape hatch: + +``` +limit must be a whole number +limit cannot exceed 100 +search cannot be empty +sortBy: expected one of "name" | "createdAt" | "updatedAt" +Limit cannot exceed 1000; use limit=0 to stream all rows, or create an export +``` + +That last one is the standard to aim for. A message that only says `Invalid input` fails this rule — the caller cannot act on it. + +## Rule 5 — contract first, then use case, then route + +Order matters because each layer is checked against the one before it. + +1. **Contract** in `lib/api/contracts/v2/.ts` via `defineRouteContract`. Response schemas are `.parse`d on the way out, so a field the producer does not actually emit becomes a 500 on a successful read — assert only what you can prove. +2. **Application use case** owns canonical loading, authorization, business behavior, and audit. The route's `present` receives the use-case result **and the parsed request**, so a presenter reads request params (the active `sortBy`/`sortOrder` and filters it stamps into a cursor) straight from `query`/`params` rather than making the use case carry an HTTP concern back out. +3. **Route** with `defineV2JsonRoute`, declaring `contract`, `auth: v2ApiKeyAuth`, `operation`, `rateLimit`, `errorPolicy`, `mapInput`, `useCase`, `present`. Auth and rate limiting run before parsing. +4. **OpenAPI description** in `lib/api/contracts/v2/openapi/.ts`, then `bun run generate:openapi`. A description that claims behaviour the route does not have is the same class of bug as a wrong schema. + +## Rule 6 — a transient failure says when to come back + +A response the caller is *expected* to retry must say how long to wait. Two statuses qualify, and both are wired: + +| Status | Source of the value | Where | +|---|---|---| +| 429 | The caller's own token bucket (`retryAfterMs`, else `resetAt - now`) | `v2RateLimitError` | +| 503 | A fixed floor, `RETRY_AFTER_SECONDS_BY_STATUS` | `v2Error`, applied automatically | + +The 503 default is applied by `v2Error` keyed on the response *status* — `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees — so every 503 the surface can emit carries it — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse`'s `infra` failures. A route with a better number passes `headers: { 'Retry-After': … }` and wins. + +Do not add a default for any other code. 400/403/404/409 are not fixed by waiting, and 402 (`USAGE_LIMIT_EXCEEDED`) is resolved by a billing change, not by time. + +**Where a policy already knows the wait, carry it — do not re-guess it at the transport.** The admission descriptors in `lib/core/admission/transient-failure` declare `retryAfterSeconds` per denial. That value used to be dropped when the descriptor was mapped onto a preprocess error, so a concurrency denial arrived as a bare 429 with no `Retry-After` even though the policy had named the wait. It now travels `descriptor.retryAfterSeconds → PreprocessExecutionError.retryAfterMs → ExecuteWorkflowServiceFailure.retryAfterMs → serviceFailureResponse`. The `v2Error` default is the floor for paths with *no* policy signal, not the source of truth. + +**A failure whose outcome is unknown must not advise a retry.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim. Telling that caller to come back in 5 seconds invites a client with no `X-Run-Id` to start and bill a second run. It passes `omitRetryAfter: true` and returns the run id so the caller reconciles instead. Any future "we don't know if it happened" failure does the same. + +RFC 9110 §10.2.3 gives 503 this field's clearest meaning — "how long the service is expected to be unavailable to the client". Note the requirement level is only `MAY`, on 503 (§15.6.4) and, via RFC 6585 §4, on 429. It is `SHOULD` on exactly one status, 413, and only when the condition is temporary; none of Sim's 413s are temporary — they are fixed ceilings, on the request body and on the collections a response must materialize — so it correctly sends none. + +## Deliberate non-adoptions + +Audited against the primary specs and against Stripe, GitHub, and Google's AIPs. Each is a considered "no", not an oversight. Re-open one only with new evidence. + +| Practice | Verdict | Why | +|---|---|---| +| **RFC 9457 `application/problem+json`** | No | 9457 §4 steers APIs with an existing format toward keeping it: "Problem details are intended to avoid the necessity of establishing new 'fault' or 'error' document formats, **not to replace existing domain-specific formats**." Nothing in it is a `MUST` to adopt, and none of Stripe, GitHub, or Google use it. Our envelope is load-bearing for every client. **The default error shape does not change.** | +| **`RateLimit`/`RateLimit-Policy` (IETF draft)** | No | Still an unpublished draft (`-11`, May 2026), returned "Not ready" at HTTPDIR review, and on its **third mutually incompatible wire format** — anything built against `-07` or earlier is already broken. None of the three surveyed APIs emit it; GitHub uses `x-ratelimit-*`, as we do. | +| **Renaming `X-RateLimit-*` per RFC 6648** | No | 6648 is a `SHOULD NOT` binding *creators of new* parameters, and §1 item 4 "**makes no recommendation as to whether existing 'X-' parameters ought to remain in use or be migrated**". Appendix B argues the migration is itself the interoperability harm. A rename is a client-visible break bought with nothing. | +| **`X-RateLimit-Reset` as delta-seconds** | No | It is an absolute ISO 8601 timestamp, so it is clock-skew sensitive — but the response where timing actually decides behaviour (429) also carries `Retry-After`, which is skew-free. The absolute value stays useful for scheduling. | +| **422 for semantic validation** | No | RFC 9110 §15.5.21 defines 422, but Appendix B.3 records that 9110 **deleted** RFC 4918's clause saying 400 was inappropriate. 400 covers "cannot or will not process… perceived to be a client error". The split is convention, not requirement — GitHub splits, Stripe and Google do not. Our machine-readable `error.code` already carries the distinction, and restatusing now breaks clients. | +| **`Location` on 201** | No | §9.3.3 makes this a `SHOULD` **for POST**; the status code itself (§15.3.2) requires nothing and defines the fallback — absent `Location`, the target URI identifies the resource. Declined knowingly: several 201 responses (signed upload sessions, table exports, knowledge folders) have no canonical single-resource GET, so a `Location` would 404, and adopting it on some of the 19 is worse for a client than on none. Every 201 returns the full representation including its `id`. Revisit per-route if one gains a canonical GET. | +| **ETag / `If-None-Match` / `If-Match`** | No | Every v2 response is `Cache-Control: private, no-store` per-caller data, so `If-None-Match` buys nothing. For writes, `If-Match` needs a **strong** validator: §8.8.3.2's strong comparison fails if *either* tag is weak, so a weak ETag silently makes every `If-Match` fail. None of the three surveyed APIs does HTTP optimistic concurrency — Google does the semantics via a resource `etag` **field** (AIP-154), deliberately not the header. If Sim needs optimistic concurrency, do it that way. | +| **`Deprecation` / `Sunset` on v1** | Not yet | RFC 9745 (Standards Track) and RFC 8594 (Informational) both apply, and GitHub emits both. But `Sunset` is a timestamp and 9745 §4 makes `Sunset >= Deprecation` a `MUST`, so emitting either commits Sim to a v1 retirement date — a product decision, not an engineering one. When that date exists: `Deprecation` is an RFC 9651 Structured Field **Date** (`@1688169599`); `Sunset` is an **HTTP-date** (`Sat, 31 Dec 2033 23:59:59 GMT`). Two encodings in one response — the most common implementation error here. | +| **`application/merge-patch+json`** | No | v2 PATCH bodies are merge-patch *shaped* — absent means unchanged, `null` clears — but they are `.strict()`, so unknown members are rejected where RFC 7396 §2 would merge them, and nested objects are replaced wholesale rather than merged. Advertising the media type would over-claim. Document the semantics per contract instead. | + +## Idempotency: at-most-once, not replay + +`POST /workflows/{id}/execute` accepts `X-Run-Id`, a caller-supplied run identifier claimed through the `idempotency_key` table (`execution-id-claim.ts`). It is a **uniqueness claim, not an idempotency key**, and the distinction is deliberate and already published in the operation description: + +- First use wins and runs. +- Any reuse returns **409** with `error.details.code: "RUN_ID_CONFLICT"`, the run id in `error.details.runId`, and an `X-Run-Id` response header. It never replays the earlier run's result — the client recovers it by polling the runs resource. +- Claims are durable tombstones, so deleting execution logs cannot make an id reusable. + +That makes the money path safe against double-execution **for callers that opt in**. What it is not: a Stripe-style `Idempotency-Key` that stores and replays the original status and body. Building that means a request fingerprint, a retention window, an in-flight-vs-completed distinction (the expired IETF draft would have these be 422 and 409 respectively), and somewhere to put a large synchronous execution body. It is a designed piece of work, not an increment — do not half-build it by aliasing the header name, which would invite clients written against Stripe semantics to treat our 409 as a hard failure. + +## Cursors are opaque, not trusted + +The base64-JSON cursor is **not signed**, and does not need to be. Tampering is bounded by construction, and that is a property to preserve: + +- Every key value is re-validated by its `KeysetKey.bind`, which returns `null` for a wrong-typed or unparseable value and becomes a 400. A forged cursor cannot reach SQL as `NaN` or an `Invalid Date`. +- The sort and a fingerprint of the filters are stamped into the cursor and re-checked (`decodeSortedCursor`), so a cursor from a differently-sorted or differently-filtered query is a 400, not a silently skipped page. The filters are hashed (SHA-256, via `lib/api/cursor-binding.ts`) rather than embedded, so the token stays short and a caller cannot cheaply construct a filter that collides with another sequence's stamp. +- The offset codec rejects anything that is not a non-negative integer. +- Authorization is **never** carried in the cursor. Every list re-derives its workspace scope from the authenticated principal, so a cursor lifted from another query — or another tenant — can only move the caller within their own authorized result set. + +The consequence to keep true: **never put a resource id, filter, or scope into a cursor and then trust it on the way back.** A cursor is a position hint, never an input to an access decision. + +## Checklist + +Run this against any new or changed v2 endpoint. + +- [ ] Success body is exactly `{data}` or `{data, nextCursor}`; failures are exactly `{error:{code,message,details?}}`. +- [ ] Route uses a shared builder; no hand-built `NextResponse.json`. +- [ ] Query and body schemas are `.strict()`. +- [ ] The contract declares a `query` — `noInputSchema` when the endpoint takes no query params, never omission. +- [ ] No caller-supplied value can produce a 500 — check every numeric param reaches SQL as a validated integer, and that any bound value passed as an argument to a SQL function carries an explicit type. +- [ ] `limit` comes from `v2PaginationFields`, not a hand-written `z.coerce.number()`. +- [ ] If the response carries `nextCursor`, the query accepts `limit` + `cursor` and the query actually applies them. +- [ ] The cursor is bound to every param that filters or orders the sequence, and to none that do not (never `limit`), with the binding declared in `list-pagination.test.ts`. +- [ ] Keyset sorts end in a unique `id` key. +- [ ] The list is classified in `list-pagination.test.ts`. +- [ ] Cross-tenant access answers 404, never 403 — and carries `Cache-Control: private, no-store`, because RFC 9110 §15.5.5 makes 404 heuristically cacheable and an authorization-dependent 404 must never be stored. `v2Error` sets this unconditionally; do not build a v2 response any other way. +- [ ] A retryable failure says when: 429 and 503 carry `Retry-After`. No other status invents one. +- [ ] 403s carry a machine-readable `details.code` from `FORBIDDEN_DETAIL_CODES`, thrown as `ForbiddenOperationError` in the domain rather than attached at the route. +- [ ] Nested objects inside a `.strict()` body are strict too — `.strict()` does not recurse. +- [ ] Ordering uses `sortBy` + `sortOrder`; boolean query params use `booleanQueryFlagSchema`. +- [ ] Validation messages name the field and echo the valid set. +- [ ] Response schema matches every field the route actually emits. +- [ ] OpenAPI description regenerated and truthful about pagination. +- [ ] `bun run type-check`, `bun run check:api-validation`, `bun run check:openapi` pass. + +## Known gap + +A 405 on a path that *does* have a route file but does not export that verb is generated by Next.js before any Sim code runs: zero-byte body, no `content-type`, and no `Allow` header, which RFC 9110 §15.5.6 requires. Fixing it means either exporting explicit rejecting handlers from every v2 route file or intercepting in `apps/sim/proxy.ts` with a static path→methods table. Neither is done. Unknown *paths* are handled — the catch-all covers those. diff --git a/.claude/commands/ship.md b/.claude/commands/ship.md index 5380cbd23f0..5eb95285a76 100644 --- a/.claude/commands/ship.md +++ b/.claude/commands/ship.md @@ -99,6 +99,16 @@ improvement(scope): description for enhancements chore(scope): description for maintenance ``` +## What to Omit + +The repo is public. Keep the title and description to the code change and its reasoning — never: + +- Customer, company, or user names; workspace/user/org IDs; email addresses +- Prod or staging operational data: log lines, DB rows, metrics, timestamps, incident details, canary/alert output +- Infrastructure specifics: hostnames, ARNs, internal URLs, env var values, secret names + +Describe the bug by its mechanism, not by how you found it. "Expired OAuth credentials fail to refresh in the worker" — not "the Sheets canary failed at 16:31Z for workspace abc-123". + ## PR Description Format Use this exact template in the user's voice (concise, bullet points): diff --git a/.claude/commands/tool-registry-boundary.md b/.claude/commands/tool-registry-boundary.md index 676fa256cbc..da6758efe3e 100644 --- a/.claude/commands/tool-registry-boundary.md +++ b/.claude/commands/tool-registry-boundary.md @@ -67,6 +67,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/.claude/commands/v2-api-conventions.md b/.claude/commands/v2-api-conventions.md new file mode 100644 index 00000000000..c4d86a251dc --- /dev/null +++ b/.claude/commands/v2-api-conventions.md @@ -0,0 +1,232 @@ +--- +description: The response, error, pagination, and validation contract every `/api/v2` endpoint must satisfy. Use when adding or changing a route under `apps/sim/app/api/v2/`, or when auditing one for conformance. +argument-hint: +--- + +# v2 API Conventions + +The v2 surface makes one promise: **every response is the same two shapes, and a caller-supplied value can never produce a 500.** + +``` +success (single) { "data": {...} } +success (collection) { "data": [...], "nextCursor": "..." | null } +failure (always) { "error": { "code": "...", "message": "...", "details"?: ... } } +``` + +Nothing else at the top level. No `success: true`, no bare `{ "error": "string" }`, no HTML. + +That promise is worth stating as a rule because it has been broken five separate ways, each time by a route or a builder taking a shortcut that looked local: + +- `GET /workflows?limit=1.5` returned **500**. The contract was copied from a sibling and lost its `.int()`, so a fractional limit passed validation and reached Postgres as `LIMIT 2.5`. +- A malformed JSON body returned **`{"error":"Request body must be valid JSON"}`** — a bare string. The envelope was a per-route opt-in that only 8 routes remembered. +- `GET /api/v2/nonexistent` returned a **full HTML 404 document**, because no route file matched and the request fell through to the app's global not-found page. +- Four collections returned `nextCursor` while **silently discarding** any `limit` the caller sent, because Zod strips unknown keys unless the schema is `.strict()`. +- Handing back a `nextCursor` from any timestamp-sorted list and passing it straight in returned **500**. The value was validated and bound — but bound with no SQL type, into `date_trunc`, which is overloaded, so Postgres could resolve no overload. Validation was never the missing half; the type was. + +Each was one line. The rules below are the generalisations. + +## Where the machinery lives + +| Concern | File | +|---|---| +| Envelope + error codes + cursor codecs | `apps/sim/app/api/v2/lib/response.ts` | +| Rollout gate (`v2-api` flag) | `apps/sim/app/api/v2/lib/gate.ts` | +| Cross-tenant concealment | `apps/sim/lib/api/server/routes/resource-concealment.ts` | +| Route builder | `apps/sim/lib/api/server/routes/v2-json-route.ts` | +| Contracts | `apps/sim/lib/api/contracts/v2/**` | +| Shared list/keyset helpers | `apps/sim/lib/api/list-query.ts` | + +## Rule 1 — the envelope is produced by helpers, never by hand + +`v2Data`, `v2CursorList`, and `v2Error` in `response.ts` are the only things that build a v2 body. They also set `Cache-Control: private, no-store`, which every v2 response needs because every v2 response is authed per-caller data. + +A route built with `defineV2JsonRoute` gets this for free: its `present` returns the *body shape* and the builder renders it. Never call `NextResponse.json` from a v2 route. + +**The envelope must hold for every failure mode, including the ones that happen before your handler runs.** That is what the four bugs above have in common. Defaults for the transport-level failures live on the builder — `v2PayloadTooLargeResponse` (413) and `v2InvalidJsonResponse` (400) — precisely so a route cannot forget them. + +## Rule 2 — status codes mean specific things + +| Status | `code` | Meaning | +|---|---|---| +| 200 / 201 | — | Success. 201 only for a created resource. | +| 400 | `BAD_REQUEST` | Contract validation. Carries field-level `details` from `serializeZodIssues`. | +| 401 | `UNAUTHORIZED` | No/!valid API key. **Runs before the rollout gate.** | +| 403 | `FORBIDDEN` | Authenticated, same tenant, insufficient rights. Where the cause is one a caller can act on it is named in `details.code`, from the closed set in `lib/core/application/forbidden.ts` (e.g. `INSUFFICIENT_WORKSPACE_ROLE`, `PERSONAL_API_KEYS_DISABLED`). A few domain refusals still reach the wire without one. | +| 404 | `NOT_FOUND` | Not found, **and** cross-tenant concealment, **and** the rollout gate, **and** an unknown path. | +| 409 | `CONFLICT` | Uniqueness/state conflict, human-readable message. | +| 413 | `PAYLOAD_TOO_LARGE` | Body over the route's `maxBodyBytes` — **and** a collection the response must materialize that is over *its* ceiling. Fourteen bodyless `GET`/`DELETE` operations publish it for the folder-tree cap (`FolderCollectionLimitExceededError`) or the rendered-artifact cap. | +| 429 | `RATE_LIMITED` | With `Retry-After` and `X-RateLimit-*`. | +| 500 | `INTERNAL_ERROR` | Genuine server fault only. Message is always generic. | + +Two of these carry real design weight: + +**404 is deliberately overloaded.** A workspace the caller cannot reach answers `404 "Workspace not found"`, never 403 — a 403 would confirm the resource exists. `createV2ResourceConcealmentPolicy` does this by mapping a cross-tenant authorization failure to `v2Error('NOT_FOUND', ...)`. The rollout gate answers the same way for the same reason (`gate.ts`: "an ungated caller cannot distinguish 'not in the rollout cohort' from 'no such endpoint'"), and so does the unknown-path catch-all at `app/api/v2/[[...segments]]/route.ts` — its body is byte-identical to the gate's on purpose. + +**500 is never caller-reachable.** Any input a caller can send must be rejected at the contract boundary with a 400. If you can construct a query string or body that produces a 500, that is a bug in the contract, not something to wrap in a `try`/`catch`. `v2ErrorForOrchestration` also replaces the message on an unclassified failure with a generic one, so internal detail never leaks. A caller-reachable 500 has shipped three times — a fractional `limit` reaching `LIMIT 2.5`, a plain `HEAD` tripping the builder's method guard, and a keyset cursor's timestamp reaching `date_trunc` as an untyped placeholder — so treat "a well-formed request produced a 500" as the highest-severity class of defect on this surface. + +**Validating a value is only half of it; the value also has to reach SQL with a type.** A bound parameter arrives as `unknown` and takes its type from context. Against a typed column (`sort_order > $1`) that inference always succeeds, which is why the gap stays invisible almost everywhere — but as an argument to an overloaded function it can resolve to nothing at all. So: **if a bound value is an argument to a SQL function rather than one side of a comparison, write its type down** (`lib/api/list-query.ts`, `timestampKey`, casts from the column). + +And this class survives a green test suite — `keysetAfter` returned well-formed SQL and every assertion passed; only Postgres's parser rejected it. When a change alters the *shape* of generated SQL rather than its values, execute it somewhere before believing the suite. + +**Which of 403 and 404 an operation documents follows from its authorization, not from whether it is a read.** `requirePermission` throws two different failures: no workspace access at all is `NoWorkspaceAccessError`, which `createV2ResourceConcealmentPolicy` conceals as 404; access below the operation's `minimumRole` is `InsufficientWorkspacePermissionsError`, which stays a 403. So: + +- An operation whose `minimumRole` is `write` or `admin` can always 403 — a member with a lower role hits it. Document 403. +- An operation whose `minimumRole` is `read` cannot 403 *that* way, because `read` is the floor of the `read < write < admin` ordering and anyone without access is concealed as 404 instead. It can still 403 through `PersonalApiKeysDisabledError` (a personal API key against a workspace whose organization disabled them) or `WorkspaceApiKeyAuthorizationError` (`workspaceApiKey: 'deny'`), and every v2 operation is reachable by a personal API key. **So in practice every workspace-scoped v2 operation documents 403**, and the reads that omitted it were wrong, not principled. + +**A 403 a caller can act on names its cause in `error.details.code` — but not every 403 does yet.** One status covers several different remedies — raise a member's role, issue a personal key instead of a workspace-scoped one, re-point a workspace key, buy an enterprise plan, delete a resource to get under a quota — and prose is not branchable, so a client that must tell them apart was string-matching messages, which turns every reword into a silent break. A handful of domain refusals still throw a bare `OrchestrationError('forbidden', …)` and reach the wire with no code, so **write client code that treats `details.code` as optional**, and read `openapi/shared.ts`'s `FORBIDDEN_DESCRIPTION` for the current position rather than assuming the sweep is finished. For code you are *writing*, the rule below is unconditional. + +The vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES` in `lib/core/application/forbidden.ts`, with a `Record` of descriptions beside it that the generated OpenAPI 403 description is built from. Adding a member fails to compile until it is documented, so a code cannot reach the wire unpublished. Do not invent a code at a route: throw `ForbiddenOperationError(code, message)` from the domain and let `v2CaughtOrchestrationError` — the function every v2 error policy falls through to — attach it. `InsufficientWorkspacePermissionsError`, `PersonalApiKeysDisabledError`, `WorkspaceApiKeyAuthorizationError`, and `PrincipalKindAuthorizationError` already carry theirs. + +The cross-tenant refusals (`NoWorkspaceAccessError`, `WorkspaceApiKeyScopeAuthorizationError`, `DelegatedWorkspaceAuthorizationError`) deliberately carry **no** code. They are concealed as 404, and naming their cause would hand back the resource-existence signal the concealment exists to withhold. + +Use the shared sets in `contracts/v2/openapi/shared.ts` — `RESOURCE_ERRORS`, `RESOURCE_CONFLICT_ERRORS`, `RESOURCE_MUTATION_ERRORS` — rather than assembling a per-operation list; all three already include `Forbidden`, and hand-assembled lists are how three knowledge reads and three upload operations quietly lost it. + +**HEAD is answered by the `GET` handler, not rejected.** Next aliases a missing `HEAD` export onto `GET` and drops the body when sending, so a route's `GET` legitimately runs with `request.method === 'HEAD'`. The builders' method guard accepts that pairing via `methodMatchesContract`; any other mismatch stays a hard error. Never hand-write a `HEAD` export to "fix" this. + +**A `GET` with side effects must declare `headSafe: false`.** The aliasing above is only sound because RFC 9110 §9.2.1 defines `HEAD` as safe. A `GET` that writes a row or opens an outbound connection is not, and an uptime monitor or link checker walking the documented URL list would drive those effects on every probe — `GET /files/{fileId}` records a `FILE_DOWNLOADED` audit event, so a `HEAD` used to fabricate a download that never happened. Both the JSON and binary builders take `headSafe`: a route that sets it `false` still authenticates and rate-limits a `HEAD`, then answers `v2HeadNoEffect()` — a bodiless 200 — before parsing or executing. Nothing observable is lost, because `HEAD` carries no body either way. Audit this whenever a read acquires an audit projection or an outbound call. + +## Rule 3 — a collection that returns `nextCursor` must accept `limit` + `cursor`, and must apply them + +Every list returns `{ data, nextCursor }`. Whether it *pages* is a separate, pinned decision — see `lib/api/contracts/v2/__tests__/list-pagination.test.ts`, which enumerates both sets and fails when a new list is in neither. + +Build the query slice from the shared helper, never by hand: + +```ts +...v2PaginationFields({ description: 'Maximum widgets to return per page.' }) +``` + +That gives `limit` (integer, 1..`V2_MAX_PAGE_SIZE`, defaulting to `V2_DEFAULT_PAGE_SIZE` = 50) and an opaque `cursor`. Re-declaring `limit: z.coerce.number()...` inline is how the 500 happened; there is one schema so the family cannot drift again. + +Three cursor schemes exist. Two are the shared codecs in `response.ts`, both opaque base64-JSON, and which of them you use is decided by what the read can express, not by taste: + +- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort AND the filters are stamped into the cursor and re-checked on replay, so changing `sortBy` or any filter mid-pagination is a 400, not a silently skipped page. +- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results. + +Both take the same two stamps: `cursorSortKey(sortBy, sortOrder)` for the ordering, and `cursorFilterScope({ ... })` for every param that filters the sequence. **`limit` is never a stamp** — it selects how much of the sequence to return, not what the sequence is, and binding it strands every cursor the moment a caller changes page size. Params that only shape the response body are out for the same reason. + +The third is **per-domain**: a list whose read predates the shared codecs, or whose page boundary is not expressible as one, mints its own — a bare `encodeCursor({ version })` on `GET /workflows/{id}/versions` and `encodeCursor({ email })` on the workspace member list, the local codecs in `lib/audit-logs/query.ts`, `lib/logs/list-logs.ts`, and `lib/table/rows/cursor.ts`, and a usage-event id passed straight through by `GET /billing/logs`. Those tokens stay opaque and untouched, but a domain-minted cursor on a list a caller can re-filter is wrapped at the surface with `encodeScopedCursor(cursorFilterScope({...}), token)` and unwrapped with `readScopedCursor`, so it carries the same binding as the shared schemes. **A new list picks one of the two shared schemes.** Do not add a fourth. + +Every paged list's binding is declared in `lib/api/contracts/v2/__tests__/list-pagination.test.ts` and checked against what the contract actually accepts, in both directions. A new list, or a new filter on an existing one, fails that test until its binding is declared or the param is explicitly recorded as unable to change the sequence. + +**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure. + +Return `nextCursor: null` on the last page and only then. Never construct a cursor client-side. + +**Ordering is `sortBy` + `sortOrder`, except where there is nothing to sort by.** Fourteen lists take the pair. Two — `GET /logs` and `GET /workflows/{id}/runs` — have exactly one sortable column (start time), so there is no `sortBy` to pair with and the direction rides on a single `order` param; `sortBy`/`sortOrder` are not accepted there. That split is documented in both contracts and is the *only* sanctioned deviation. A new list picks the pair. Do not "fix" the two by accepting `sortOrder` as an alias: an alias is a second spelling of one thing with undefined precedence when both arrive, which is its own inconsistency, and renaming `order` would break every shipped caller. + +**A boolean query param is a real boolean**, declared with `booleanQueryFlagSchema` from `contracts/primitives.ts`. It coerces `'true'`/`'1'` and `'false'`/`'0'`/`''`, so it is a strict widening of a `z.enum(['true','false'])` — which is what two v2 params used to be, purely by inheritance from the internal shapes they reused. Reusing an internal `.shape.x` inherits the internal spelling; re-declare instead when the internal one is not the v2 convention. + +## Rule 4 — reject what you do not implement + +**Every contract declares a `query`, even when the endpoint takes none** — `query: noInputSchema` (`z.object({}).strict()`), never omission. `parseRequest` validates the query slice only when the contract declares one, so an omitted `query` means "never look at the query string", not "takes no query params". The two were indistinguishable, which is how 69 contracts ended up accepting anything without anyone deciding they should: `GET /workflows/{id}?bogus=1` answered 200 while every list answered 400 for the same shape. `query-declaration.test.ts` sweeps the tree so contract 70 fails at authoring time rather than shipping unvalidated. + +Declaring them is a **deliberate tightening** of endpoints that previously ignored an unknown param. It was weighed and kept: the v2 body slice on those same endpoints was already strict, so the split was arbitrary rather than a promise to callers; a mistyped param that is silently dropped is the bug class this rule exists to prevent; and no first-party caller sends an undeclared v2 query param (the two SDKs send only `includeOutput`/`selectedOutputs`, both declared; the UI makes no v2 calls at all; `requestJson` appends nothing implicitly and no v2 cache buster exists). Third-party callers appending a tracking tag or cache buster do break, which is why `api-reference/getting-started.mdx` documents the behavior rather than leaving it to be discovered from a 400. + +Query and body schemas are **`.strict()`** — and `.strict()` binds the **top level only**. A strict body containing a non-strict nested object still drops unknown keys one level down, which is the headline `filter` bug at a smaller scale: `sort: [{ field, direction, nulls: 'last' }]` answered 200 and ordered by the default. Strictness belongs on the shared nested schema (`sortSpecSchema`'s element, `tableViewConfigSchema`), not restated per body. + +Before tightening a schema that is **also** a response or a stored blob, make the read canonical first. `table_views.config` is schemaless JSONB, so a legacy row carrying a retired key would fail a newly strict response parse and become a 500; `normalizeStoredViewConfig` projects the stored blob onto the declared keys so the tightening is safe in both directions. Zod strips unknown keys by default, so a non-strict schema answers `?limit=1` with 200 and the whole set — the caller believes it bounded the response and it did not. That is a contract lie, and on an uncapped list it is also an unbounded-response risk. + +Error messages name the field and, where there is one, the escape hatch: + +``` +limit must be a whole number +limit cannot exceed 100 +search cannot be empty +sortBy: expected one of "name" | "createdAt" | "updatedAt" +Limit cannot exceed 1000; use limit=0 to stream all rows, or create an export +``` + +That last one is the standard to aim for. A message that only says `Invalid input` fails this rule — the caller cannot act on it. + +## Rule 5 — contract first, then use case, then route + +Order matters because each layer is checked against the one before it. + +1. **Contract** in `lib/api/contracts/v2/.ts` via `defineRouteContract`. Response schemas are `.parse`d on the way out, so a field the producer does not actually emit becomes a 500 on a successful read — assert only what you can prove. +2. **Application use case** owns canonical loading, authorization, business behavior, and audit. The route's `present` receives the use-case result **and the parsed request**, so a presenter reads request params (the active `sortBy`/`sortOrder` and filters it stamps into a cursor) straight from `query`/`params` rather than making the use case carry an HTTP concern back out. +3. **Route** with `defineV2JsonRoute`, declaring `contract`, `auth: v2ApiKeyAuth`, `operation`, `rateLimit`, `errorPolicy`, `mapInput`, `useCase`, `present`. Auth and rate limiting run before parsing. +4. **OpenAPI description** in `lib/api/contracts/v2/openapi/.ts`, then `bun run generate:openapi`. A description that claims behaviour the route does not have is the same class of bug as a wrong schema. + +## Rule 6 — a transient failure says when to come back + +A response the caller is *expected* to retry must say how long to wait. Two statuses qualify, and both are wired: + +| Status | Source of the value | Where | +|---|---|---| +| 429 | The caller's own token bucket (`retryAfterMs`, else `resetAt - now`) | `v2RateLimitError` | +| 503 | A fixed floor, `RETRY_AFTER_SECONDS_BY_STATUS` | `v2Error`, applied automatically | + +The 503 default is applied by `v2Error` keyed on the response *status* — `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees — so every 503 the surface can emit carries it — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse`'s `infra` failures. A route with a better number passes `headers: { 'Retry-After': … }` and wins. + +Do not add a default for any other code. 400/403/404/409 are not fixed by waiting, and 402 (`USAGE_LIMIT_EXCEEDED`) is resolved by a billing change, not by time. + +**Where a policy already knows the wait, carry it — do not re-guess it at the transport.** The admission descriptors in `lib/core/admission/transient-failure` declare `retryAfterSeconds` per denial. That value used to be dropped when the descriptor was mapped onto a preprocess error, so a concurrency denial arrived as a bare 429 with no `Retry-After` even though the policy had named the wait. It now travels `descriptor.retryAfterSeconds → PreprocessExecutionError.retryAfterMs → ExecuteWorkflowServiceFailure.retryAfterMs → serviceFailureResponse`. The `v2Error` default is the floor for paths with *no* policy signal, not the source of truth. + +**A failure whose outcome is unknown must not advise a retry.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim. Telling that caller to come back in 5 seconds invites a client with no `X-Run-Id` to start and bill a second run. It passes `omitRetryAfter: true` and returns the run id so the caller reconciles instead. Any future "we don't know if it happened" failure does the same. + +RFC 9110 §10.2.3 gives 503 this field's clearest meaning — "how long the service is expected to be unavailable to the client". Note the requirement level is only `MAY`, on 503 (§15.6.4) and, via RFC 6585 §4, on 429. It is `SHOULD` on exactly one status, 413, and only when the condition is temporary; none of Sim's 413s are temporary — they are fixed ceilings, on the request body and on the collections a response must materialize — so it correctly sends none. + +## Deliberate non-adoptions + +Audited against the primary specs and against Stripe, GitHub, and Google's AIPs. Each is a considered "no", not an oversight. Re-open one only with new evidence. + +| Practice | Verdict | Why | +|---|---|---| +| **RFC 9457 `application/problem+json`** | No | 9457 §4 steers APIs with an existing format toward keeping it: "Problem details are intended to avoid the necessity of establishing new 'fault' or 'error' document formats, **not to replace existing domain-specific formats**." Nothing in it is a `MUST` to adopt, and none of Stripe, GitHub, or Google use it. Our envelope is load-bearing for every client. **The default error shape does not change.** | +| **`RateLimit`/`RateLimit-Policy` (IETF draft)** | No | Still an unpublished draft (`-11`, May 2026), returned "Not ready" at HTTPDIR review, and on its **third mutually incompatible wire format** — anything built against `-07` or earlier is already broken. None of the three surveyed APIs emit it; GitHub uses `x-ratelimit-*`, as we do. | +| **Renaming `X-RateLimit-*` per RFC 6648** | No | 6648 is a `SHOULD NOT` binding *creators of new* parameters, and §1 item 4 "**makes no recommendation as to whether existing 'X-' parameters ought to remain in use or be migrated**". Appendix B argues the migration is itself the interoperability harm. A rename is a client-visible break bought with nothing. | +| **`X-RateLimit-Reset` as delta-seconds** | No | It is an absolute ISO 8601 timestamp, so it is clock-skew sensitive — but the response where timing actually decides behaviour (429) also carries `Retry-After`, which is skew-free. The absolute value stays useful for scheduling. | +| **422 for semantic validation** | No | RFC 9110 §15.5.21 defines 422, but Appendix B.3 records that 9110 **deleted** RFC 4918's clause saying 400 was inappropriate. 400 covers "cannot or will not process… perceived to be a client error". The split is convention, not requirement — GitHub splits, Stripe and Google do not. Our machine-readable `error.code` already carries the distinction, and restatusing now breaks clients. | +| **`Location` on 201** | No | §9.3.3 makes this a `SHOULD` **for POST**; the status code itself (§15.3.2) requires nothing and defines the fallback — absent `Location`, the target URI identifies the resource. Declined knowingly: several 201 responses (signed upload sessions, table exports, knowledge folders) have no canonical single-resource GET, so a `Location` would 404, and adopting it on some of the 19 is worse for a client than on none. Every 201 returns the full representation including its `id`. Revisit per-route if one gains a canonical GET. | +| **ETag / `If-None-Match` / `If-Match`** | No | Every v2 response is `Cache-Control: private, no-store` per-caller data, so `If-None-Match` buys nothing. For writes, `If-Match` needs a **strong** validator: §8.8.3.2's strong comparison fails if *either* tag is weak, so a weak ETag silently makes every `If-Match` fail. None of the three surveyed APIs does HTTP optimistic concurrency — Google does the semantics via a resource `etag` **field** (AIP-154), deliberately not the header. If Sim needs optimistic concurrency, do it that way. | +| **`Deprecation` / `Sunset` on v1** | Not yet | RFC 9745 (Standards Track) and RFC 8594 (Informational) both apply, and GitHub emits both. But `Sunset` is a timestamp and 9745 §4 makes `Sunset >= Deprecation` a `MUST`, so emitting either commits Sim to a v1 retirement date — a product decision, not an engineering one. When that date exists: `Deprecation` is an RFC 9651 Structured Field **Date** (`@1688169599`); `Sunset` is an **HTTP-date** (`Sat, 31 Dec 2033 23:59:59 GMT`). Two encodings in one response — the most common implementation error here. | +| **`application/merge-patch+json`** | No | v2 PATCH bodies are merge-patch *shaped* — absent means unchanged, `null` clears — but they are `.strict()`, so unknown members are rejected where RFC 7396 §2 would merge them, and nested objects are replaced wholesale rather than merged. Advertising the media type would over-claim. Document the semantics per contract instead. | + +## Idempotency: at-most-once, not replay + +`POST /workflows/{id}/execute` accepts `X-Run-Id`, a caller-supplied run identifier claimed through the `idempotency_key` table (`execution-id-claim.ts`). It is a **uniqueness claim, not an idempotency key**, and the distinction is deliberate and already published in the operation description: + +- First use wins and runs. +- Any reuse returns **409** with `error.details.code: "RUN_ID_CONFLICT"`, the run id in `error.details.runId`, and an `X-Run-Id` response header. It never replays the earlier run's result — the client recovers it by polling the runs resource. +- Claims are durable tombstones, so deleting execution logs cannot make an id reusable. + +That makes the money path safe against double-execution **for callers that opt in**. What it is not: a Stripe-style `Idempotency-Key` that stores and replays the original status and body. Building that means a request fingerprint, a retention window, an in-flight-vs-completed distinction (the expired IETF draft would have these be 422 and 409 respectively), and somewhere to put a large synchronous execution body. It is a designed piece of work, not an increment — do not half-build it by aliasing the header name, which would invite clients written against Stripe semantics to treat our 409 as a hard failure. + +## Cursors are opaque, not trusted + +The base64-JSON cursor is **not signed**, and does not need to be. Tampering is bounded by construction, and that is a property to preserve: + +- Every key value is re-validated by its `KeysetKey.bind`, which returns `null` for a wrong-typed or unparseable value and becomes a 400. A forged cursor cannot reach SQL as `NaN` or an `Invalid Date`. +- The sort and a fingerprint of the filters are stamped into the cursor and re-checked (`decodeSortedCursor`), so a cursor from a differently-sorted or differently-filtered query is a 400, not a silently skipped page. The filters are hashed (SHA-256, via `lib/api/cursor-binding.ts`) rather than embedded, so the token stays short and a caller cannot cheaply construct a filter that collides with another sequence's stamp. +- The offset codec rejects anything that is not a non-negative integer. +- Authorization is **never** carried in the cursor. Every list re-derives its workspace scope from the authenticated principal, so a cursor lifted from another query — or another tenant — can only move the caller within their own authorized result set. + +The consequence to keep true: **never put a resource id, filter, or scope into a cursor and then trust it on the way back.** A cursor is a position hint, never an input to an access decision. + +## Checklist + +Run this against any new or changed v2 endpoint. + +- [ ] Success body is exactly `{data}` or `{data, nextCursor}`; failures are exactly `{error:{code,message,details?}}`. +- [ ] Route uses a shared builder; no hand-built `NextResponse.json`. +- [ ] Query and body schemas are `.strict()`. +- [ ] The contract declares a `query` — `noInputSchema` when the endpoint takes no query params, never omission. +- [ ] No caller-supplied value can produce a 500 — check every numeric param reaches SQL as a validated integer, and that any bound value passed as an argument to a SQL function carries an explicit type. +- [ ] `limit` comes from `v2PaginationFields`, not a hand-written `z.coerce.number()`. +- [ ] If the response carries `nextCursor`, the query accepts `limit` + `cursor` and the query actually applies them. +- [ ] The cursor is bound to every param that filters or orders the sequence, and to none that do not (never `limit`), with the binding declared in `list-pagination.test.ts`. +- [ ] Keyset sorts end in a unique `id` key. +- [ ] The list is classified in `list-pagination.test.ts`. +- [ ] Cross-tenant access answers 404, never 403 — and carries `Cache-Control: private, no-store`, because RFC 9110 §15.5.5 makes 404 heuristically cacheable and an authorization-dependent 404 must never be stored. `v2Error` sets this unconditionally; do not build a v2 response any other way. +- [ ] A retryable failure says when: 429 and 503 carry `Retry-After`. No other status invents one. +- [ ] 403s carry a machine-readable `details.code` from `FORBIDDEN_DETAIL_CODES`, thrown as `ForbiddenOperationError` in the domain rather than attached at the route. +- [ ] Nested objects inside a `.strict()` body are strict too — `.strict()` does not recurse. +- [ ] Ordering uses `sortBy` + `sortOrder`; boolean query params use `booleanQueryFlagSchema`. +- [ ] Validation messages name the field and echo the valid set. +- [ ] Response schema matches every field the route actually emits. +- [ ] OpenAPI description regenerated and truthful about pagination. +- [ ] `bun run type-check`, `bun run check:api-validation`, `bun run check:openapi` pass. + +## Known gap + +A 405 on a path that *does* have a route file but does not export that verb is generated by Next.js before any Sim code runs: zero-byte body, no `content-type`, and no `Allow` header, which RFC 9110 §15.5.6 requires. Fixing it means either exporting explicit rejecting handlers from every v2 route file or intercepting in `apps/sim/proxy.ts` with a static path→methods table. Neither is done. Unknown *paths* are handled — the catch-all covers those. diff --git a/.claude/rules/sim-architecture.md b/.claude/rules/sim-architecture.md index 6886710a61c..a8b25498eae 100644 --- a/.claude/rules/sim-architecture.md +++ b/.claude/rules/sim-architecture.md @@ -57,6 +57,27 @@ Use the `migrate-application-operation` skill before creating or migrating a pro Every export of a `'use client'` module becomes a *client reference* on the server — server-evaluated code (RSC pages/layouts, `prefetch.ts`, route handlers, block definitions, triggers) can only *render* it as a component or pass it as a prop, never *call* it (doing so throws at runtime, e.g. `tableKeys.list is not a function`; `next build` does not catch it). Keep server-importable query primitives (key factories, fetchers, mappers, constants) in non-`'use client'` modules — see `.claude/rules/sim-queries.md`. Enforced by `scripts/check-client-boundary-imports.ts`. +## The app/worker runtime boundary + +Server code runs in two runtimes with **different environments**. The app container loads the +full env from `SIM_ENV_SECRET_ID` (Secrets Manager). Trigger.dev workers — which execute +workflows, so every block handler and every tool call — get their env from the Trigger.dev +dashboard, and `trigger.config.ts` syncs only `DB_APP_NAME`. The repo cannot see what the +dashboard holds. + +So before replacing a worker's HTTP call to our own API with an in-process call, ask what env +that work reads *on the app side*. Anything gated by a `require*Capability` helper is the sharp +case: those **throw** when the variable is absent (`requireOAuthClientCapability` → +`EnvCapabilityConfigurationError`), and the throw may be caught and reported as something +unrelated. OAuth token refresh is the known example — moving it into the worker turns every +expired credential into `Failed to refresh access token`, while a still-valid token hides the +bug entirely, so it surfaces hours later and only for whoever's token lapsed first. + +An in-process conversion is safe when the same work already runs in that runtime (the agent +block has always called `executeProviderRequest` in-process, so router and evaluator joining it +is proven), or when the caller and the callee are both the app (a route calling a lib module, an +RSC prefetch reading the data layer). It is not safe on reasoning alone. + ## Feature Organization Features live under `app/workspace/[workspaceId]/`: diff --git a/.claude/rules/sim-queries.md b/.claude/rules/sim-queries.md index 14acca8f2cb..a71ee448dde 100644 --- a/.claude/rules/sim-queries.md +++ b/.claude/rules/sim-queries.md @@ -143,6 +143,22 @@ const handler = useCallback(() => { }, [data]) ``` +## Server prefetching + +A server prefetch fills the *same* cache key a client hook fills, so it must be indistinguishable from a client fetch. Five rules: + +1. **Read the data layer, never our own API over HTTP.** A server-to-server call to `/api/...` costs a round trip and a second authentication for data the process can already read. Where the route runs an application use case, call that same use case with a principal from the same auth policy the route declares — not a manager underneath it. +2. **Match the wire shape the hook caches.** The hook's data is whatever `requestJson(contract, …)` produced, so the seed must equal it. Two traps: a contract field declared `z.coerce.date()` means the hook holds a `Date` where raw route JSON holds a string; a passthrough response schema (`z.custom`) means the hook caches route JSON *verbatim*, so seeding raw rows leaks `Date`s and server-only fields. When the route projects before responding, share that projection — have the route and the prefetch call one function. +3. **Prove the viewer.** Data-layer reads carry no authorization; the route used to provide it. Resolve the viewer (`getWorkspaceHostContextForViewer`, already `cache`d by the layout so it costs nothing) and return early on failure, caching nothing — the client fetch then reaches the route for the real 403. Never widen what a viewer can see. +4. **Always `await`.** Only a settled query is dehydrated, so an unawaited prefetch is silently dropped from the payload and the pane waterfalls anyway. +5. **Don't repeat what the layout already seeded.** `getQueryClient()` builds a new client per server call, so a page re-seeding a layout key is a genuine second read — and `HydrationBoundary` defers an already-seen query to an effect, which SSR never runs, so it never reaches the server render either. + +Reuse the hook's exported `staleTime` constant and its key factory; `dehydrate` carries neither options nor `staleTime`, and freshness is per-observer. + +Seed with `setQueryData` only when the prefetch must be able to *decline* to create an entry (an empty list that has to fall through to a route's creation path). `prefetchQuery` and `ensureQueryData` always create one. + +Keep prefetch imports light. A page prefetch's imports land in that route's server graph, so pulling a barrel to reach one function can drag thousands of modules behind it — `bun run check:tool-registry-boundary` gates this per page. + ## Boundary Types - Hooks import named type aliases from `@/lib/api/contracts/**` (e.g., `import { listEntitiesContract, type EntityList } from '@/lib/api/contracts/entities'`). Never write `z.input<...>` / `z.output<...>` in hooks, and never `import { z } from 'zod'` in client code. diff --git a/.cursor/commands/ship.md b/.cursor/commands/ship.md index 7421187736d..bb437b52ccc 100644 --- a/.cursor/commands/ship.md +++ b/.cursor/commands/ship.md @@ -94,6 +94,16 @@ improvement(scope): description for enhancements chore(scope): description for maintenance ``` +## What to Omit + +The repo is public. Keep the title and description to the code change and its reasoning — never: + +- Customer, company, or user names; workspace/user/org IDs; email addresses +- Prod or staging operational data: log lines, DB rows, metrics, timestamps, incident details, canary/alert output +- Infrastructure specifics: hostnames, ARNs, internal URLs, env var values, secret names + +Describe the bug by its mechanism, not by how you found it. "Expired OAuth credentials fail to refresh in the worker" — not "the Sheets canary failed at 16:31Z for workspace abc-123". + ## PR Description Format Use this exact template in the user's voice (concise, bullet points): diff --git a/.cursor/commands/tool-registry-boundary.md b/.cursor/commands/tool-registry-boundary.md index d560d4f40e5..62fb47f53fd 100644 --- a/.cursor/commands/tool-registry-boundary.md +++ b/.cursor/commands/tool-registry-boundary.md @@ -63,6 +63,10 @@ If it fails, do not add the entry to an allowlist — there isn't one. Find the Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. +The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. This catches bloat the registry rule misses — a prefetch importing `listTables` cost the Tables page 444 modules without ever touching `@/tools/registry`. + +Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. + ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: diff --git a/.cursor/commands/v2-api-conventions.md b/.cursor/commands/v2-api-conventions.md new file mode 100644 index 00000000000..7456c295e20 --- /dev/null +++ b/.cursor/commands/v2-api-conventions.md @@ -0,0 +1,227 @@ +# v2 API Conventions + +The v2 surface makes one promise: **every response is the same two shapes, and a caller-supplied value can never produce a 500.** + +``` +success (single) { "data": {...} } +success (collection) { "data": [...], "nextCursor": "..." | null } +failure (always) { "error": { "code": "...", "message": "...", "details"?: ... } } +``` + +Nothing else at the top level. No `success: true`, no bare `{ "error": "string" }`, no HTML. + +That promise is worth stating as a rule because it has been broken five separate ways, each time by a route or a builder taking a shortcut that looked local: + +- `GET /workflows?limit=1.5` returned **500**. The contract was copied from a sibling and lost its `.int()`, so a fractional limit passed validation and reached Postgres as `LIMIT 2.5`. +- A malformed JSON body returned **`{"error":"Request body must be valid JSON"}`** — a bare string. The envelope was a per-route opt-in that only 8 routes remembered. +- `GET /api/v2/nonexistent` returned a **full HTML 404 document**, because no route file matched and the request fell through to the app's global not-found page. +- Four collections returned `nextCursor` while **silently discarding** any `limit` the caller sent, because Zod strips unknown keys unless the schema is `.strict()`. +- Handing back a `nextCursor` from any timestamp-sorted list and passing it straight in returned **500**. The value was validated and bound — but bound with no SQL type, into `date_trunc`, which is overloaded, so Postgres could resolve no overload. Validation was never the missing half; the type was. + +Each was one line. The rules below are the generalisations. + +## Where the machinery lives + +| Concern | File | +|---|---| +| Envelope + error codes + cursor codecs | `apps/sim/app/api/v2/lib/response.ts` | +| Rollout gate (`v2-api` flag) | `apps/sim/app/api/v2/lib/gate.ts` | +| Cross-tenant concealment | `apps/sim/lib/api/server/routes/resource-concealment.ts` | +| Route builder | `apps/sim/lib/api/server/routes/v2-json-route.ts` | +| Contracts | `apps/sim/lib/api/contracts/v2/**` | +| Shared list/keyset helpers | `apps/sim/lib/api/list-query.ts` | + +## Rule 1 — the envelope is produced by helpers, never by hand + +`v2Data`, `v2CursorList`, and `v2Error` in `response.ts` are the only things that build a v2 body. They also set `Cache-Control: private, no-store`, which every v2 response needs because every v2 response is authed per-caller data. + +A route built with `defineV2JsonRoute` gets this for free: its `present` returns the *body shape* and the builder renders it. Never call `NextResponse.json` from a v2 route. + +**The envelope must hold for every failure mode, including the ones that happen before your handler runs.** That is what the four bugs above have in common. Defaults for the transport-level failures live on the builder — `v2PayloadTooLargeResponse` (413) and `v2InvalidJsonResponse` (400) — precisely so a route cannot forget them. + +## Rule 2 — status codes mean specific things + +| Status | `code` | Meaning | +|---|---|---| +| 200 / 201 | — | Success. 201 only for a created resource. | +| 400 | `BAD_REQUEST` | Contract validation. Carries field-level `details` from `serializeZodIssues`. | +| 401 | `UNAUTHORIZED` | No/!valid API key. **Runs before the rollout gate.** | +| 403 | `FORBIDDEN` | Authenticated, same tenant, insufficient rights. Where the cause is one a caller can act on it is named in `details.code`, from the closed set in `lib/core/application/forbidden.ts` (e.g. `INSUFFICIENT_WORKSPACE_ROLE`, `PERSONAL_API_KEYS_DISABLED`). A few domain refusals still reach the wire without one. | +| 404 | `NOT_FOUND` | Not found, **and** cross-tenant concealment, **and** the rollout gate, **and** an unknown path. | +| 409 | `CONFLICT` | Uniqueness/state conflict, human-readable message. | +| 413 | `PAYLOAD_TOO_LARGE` | Body over the route's `maxBodyBytes` — **and** a collection the response must materialize that is over *its* ceiling. Fourteen bodyless `GET`/`DELETE` operations publish it for the folder-tree cap (`FolderCollectionLimitExceededError`) or the rendered-artifact cap. | +| 429 | `RATE_LIMITED` | With `Retry-After` and `X-RateLimit-*`. | +| 500 | `INTERNAL_ERROR` | Genuine server fault only. Message is always generic. | + +Two of these carry real design weight: + +**404 is deliberately overloaded.** A workspace the caller cannot reach answers `404 "Workspace not found"`, never 403 — a 403 would confirm the resource exists. `createV2ResourceConcealmentPolicy` does this by mapping a cross-tenant authorization failure to `v2Error('NOT_FOUND', ...)`. The rollout gate answers the same way for the same reason (`gate.ts`: "an ungated caller cannot distinguish 'not in the rollout cohort' from 'no such endpoint'"), and so does the unknown-path catch-all at `app/api/v2/[[...segments]]/route.ts` — its body is byte-identical to the gate's on purpose. + +**500 is never caller-reachable.** Any input a caller can send must be rejected at the contract boundary with a 400. If you can construct a query string or body that produces a 500, that is a bug in the contract, not something to wrap in a `try`/`catch`. `v2ErrorForOrchestration` also replaces the message on an unclassified failure with a generic one, so internal detail never leaks. A caller-reachable 500 has shipped three times — a fractional `limit` reaching `LIMIT 2.5`, a plain `HEAD` tripping the builder's method guard, and a keyset cursor's timestamp reaching `date_trunc` as an untyped placeholder — so treat "a well-formed request produced a 500" as the highest-severity class of defect on this surface. + +**Validating a value is only half of it; the value also has to reach SQL with a type.** A bound parameter arrives as `unknown` and takes its type from context. Against a typed column (`sort_order > $1`) that inference always succeeds, which is why the gap stays invisible almost everywhere — but as an argument to an overloaded function it can resolve to nothing at all. So: **if a bound value is an argument to a SQL function rather than one side of a comparison, write its type down** (`lib/api/list-query.ts`, `timestampKey`, casts from the column). + +And this class survives a green test suite — `keysetAfter` returned well-formed SQL and every assertion passed; only Postgres's parser rejected it. When a change alters the *shape* of generated SQL rather than its values, execute it somewhere before believing the suite. + +**Which of 403 and 404 an operation documents follows from its authorization, not from whether it is a read.** `requirePermission` throws two different failures: no workspace access at all is `NoWorkspaceAccessError`, which `createV2ResourceConcealmentPolicy` conceals as 404; access below the operation's `minimumRole` is `InsufficientWorkspacePermissionsError`, which stays a 403. So: + +- An operation whose `minimumRole` is `write` or `admin` can always 403 — a member with a lower role hits it. Document 403. +- An operation whose `minimumRole` is `read` cannot 403 *that* way, because `read` is the floor of the `read < write < admin` ordering and anyone without access is concealed as 404 instead. It can still 403 through `PersonalApiKeysDisabledError` (a personal API key against a workspace whose organization disabled them) or `WorkspaceApiKeyAuthorizationError` (`workspaceApiKey: 'deny'`), and every v2 operation is reachable by a personal API key. **So in practice every workspace-scoped v2 operation documents 403**, and the reads that omitted it were wrong, not principled. + +**A 403 a caller can act on names its cause in `error.details.code` — but not every 403 does yet.** One status covers several different remedies — raise a member's role, issue a personal key instead of a workspace-scoped one, re-point a workspace key, buy an enterprise plan, delete a resource to get under a quota — and prose is not branchable, so a client that must tell them apart was string-matching messages, which turns every reword into a silent break. A handful of domain refusals still throw a bare `OrchestrationError('forbidden', …)` and reach the wire with no code, so **write client code that treats `details.code` as optional**, and read `openapi/shared.ts`'s `FORBIDDEN_DESCRIPTION` for the current position rather than assuming the sweep is finished. For code you are *writing*, the rule below is unconditional. + +The vocabulary is a closed set, `FORBIDDEN_DETAIL_CODES` in `lib/core/application/forbidden.ts`, with a `Record` of descriptions beside it that the generated OpenAPI 403 description is built from. Adding a member fails to compile until it is documented, so a code cannot reach the wire unpublished. Do not invent a code at a route: throw `ForbiddenOperationError(code, message)` from the domain and let `v2CaughtOrchestrationError` — the function every v2 error policy falls through to — attach it. `InsufficientWorkspacePermissionsError`, `PersonalApiKeysDisabledError`, `WorkspaceApiKeyAuthorizationError`, and `PrincipalKindAuthorizationError` already carry theirs. + +The cross-tenant refusals (`NoWorkspaceAccessError`, `WorkspaceApiKeyScopeAuthorizationError`, `DelegatedWorkspaceAuthorizationError`) deliberately carry **no** code. They are concealed as 404, and naming their cause would hand back the resource-existence signal the concealment exists to withhold. + +Use the shared sets in `contracts/v2/openapi/shared.ts` — `RESOURCE_ERRORS`, `RESOURCE_CONFLICT_ERRORS`, `RESOURCE_MUTATION_ERRORS` — rather than assembling a per-operation list; all three already include `Forbidden`, and hand-assembled lists are how three knowledge reads and three upload operations quietly lost it. + +**HEAD is answered by the `GET` handler, not rejected.** Next aliases a missing `HEAD` export onto `GET` and drops the body when sending, so a route's `GET` legitimately runs with `request.method === 'HEAD'`. The builders' method guard accepts that pairing via `methodMatchesContract`; any other mismatch stays a hard error. Never hand-write a `HEAD` export to "fix" this. + +**A `GET` with side effects must declare `headSafe: false`.** The aliasing above is only sound because RFC 9110 §9.2.1 defines `HEAD` as safe. A `GET` that writes a row or opens an outbound connection is not, and an uptime monitor or link checker walking the documented URL list would drive those effects on every probe — `GET /files/{fileId}` records a `FILE_DOWNLOADED` audit event, so a `HEAD` used to fabricate a download that never happened. Both the JSON and binary builders take `headSafe`: a route that sets it `false` still authenticates and rate-limits a `HEAD`, then answers `v2HeadNoEffect()` — a bodiless 200 — before parsing or executing. Nothing observable is lost, because `HEAD` carries no body either way. Audit this whenever a read acquires an audit projection or an outbound call. + +## Rule 3 — a collection that returns `nextCursor` must accept `limit` + `cursor`, and must apply them + +Every list returns `{ data, nextCursor }`. Whether it *pages* is a separate, pinned decision — see `lib/api/contracts/v2/__tests__/list-pagination.test.ts`, which enumerates both sets and fails when a new list is in neither. + +Build the query slice from the shared helper, never by hand: + +```ts +...v2PaginationFields({ description: 'Maximum widgets to return per page.' }) +``` + +That gives `limit` (integer, 1..`V2_MAX_PAGE_SIZE`, defaulting to `V2_DEFAULT_PAGE_SIZE` = 50) and an opaque `cursor`. Re-declaring `limit: z.coerce.number()...` inline is how the 500 happened; there is one schema so the family cannot drift again. + +Three cursor schemes exist. Two are the shared codecs in `response.ts`, both opaque base64-JSON, and which of them you use is decided by what the read can express, not by taste: + +- **Keyset** (`readSortedCursor` in, `encodeSortedCursor` out) — the default. Requires the page to come from one ordered SQL read. The sort AND the filters are stamped into the cursor and re-checked on replay, so changing `sortBy` or any filter mid-pagination is a 400, not a silently skipped page. +- **Offset** (`decodeOffsetCursor` / `encodeOffsetCursor`) — only when a keyset is impossible. Two lists qualify: `GET /skills` merges a static in-code registry with DB rows and re-sorts in JS, and `GET /knowledge/{id}/documents` sits on a limit/offset query. A bare offset replayed against a re-sorted or re-filtered sequence names a different row, which skips or repeats results. + +Both take the same two stamps: `cursorSortKey(sortBy, sortOrder)` for the ordering, and `cursorFilterScope({ ... })` for every param that filters the sequence. **`limit` is never a stamp** — it selects how much of the sequence to return, not what the sequence is, and binding it strands every cursor the moment a caller changes page size. Params that only shape the response body are out for the same reason. + +The third is **per-domain**: a list whose read predates the shared codecs, or whose page boundary is not expressible as one, mints its own — a bare `encodeCursor({ version })` on `GET /workflows/{id}/versions` and `encodeCursor({ email })` on the workspace member list, the local codecs in `lib/audit-logs/query.ts`, `lib/logs/list-logs.ts`, and `lib/table/rows/cursor.ts`, and a usage-event id passed straight through by `GET /billing/logs`. Those tokens stay opaque and untouched, but a domain-minted cursor on a list a caller can re-filter is wrapped at the surface with `encodeScopedCursor(cursorFilterScope({...}), token)` and unwrapped with `readScopedCursor`, so it carries the same binding as the shared schemes. **A new list picks one of the two shared schemes.** Do not add a fourth. + +Every paged list's binding is declared in `lib/api/contracts/v2/__tests__/list-pagination.test.ts` and checked against what the contract actually accepts, in both directions. A new list, or a new filter on an existing one, fails that test until its binding is declared or the param is explicitly recorded as unable to change the sequence. + +**A keyset's key list must end in a unique column (`id`).** A non-unique trailing key cannot separate tied rows, so the page boundary either repeats or drops them. `lib/api/list-keyset-paging.test.ts` demonstrates the failure. + +Return `nextCursor: null` on the last page and only then. Never construct a cursor client-side. + +**Ordering is `sortBy` + `sortOrder`, except where there is nothing to sort by.** Fourteen lists take the pair. Two — `GET /logs` and `GET /workflows/{id}/runs` — have exactly one sortable column (start time), so there is no `sortBy` to pair with and the direction rides on a single `order` param; `sortBy`/`sortOrder` are not accepted there. That split is documented in both contracts and is the *only* sanctioned deviation. A new list picks the pair. Do not "fix" the two by accepting `sortOrder` as an alias: an alias is a second spelling of one thing with undefined precedence when both arrive, which is its own inconsistency, and renaming `order` would break every shipped caller. + +**A boolean query param is a real boolean**, declared with `booleanQueryFlagSchema` from `contracts/primitives.ts`. It coerces `'true'`/`'1'` and `'false'`/`'0'`/`''`, so it is a strict widening of a `z.enum(['true','false'])` — which is what two v2 params used to be, purely by inheritance from the internal shapes they reused. Reusing an internal `.shape.x` inherits the internal spelling; re-declare instead when the internal one is not the v2 convention. + +## Rule 4 — reject what you do not implement + +**Every contract declares a `query`, even when the endpoint takes none** — `query: noInputSchema` (`z.object({}).strict()`), never omission. `parseRequest` validates the query slice only when the contract declares one, so an omitted `query` means "never look at the query string", not "takes no query params". The two were indistinguishable, which is how 69 contracts ended up accepting anything without anyone deciding they should: `GET /workflows/{id}?bogus=1` answered 200 while every list answered 400 for the same shape. `query-declaration.test.ts` sweeps the tree so contract 70 fails at authoring time rather than shipping unvalidated. + +Declaring them is a **deliberate tightening** of endpoints that previously ignored an unknown param. It was weighed and kept: the v2 body slice on those same endpoints was already strict, so the split was arbitrary rather than a promise to callers; a mistyped param that is silently dropped is the bug class this rule exists to prevent; and no first-party caller sends an undeclared v2 query param (the two SDKs send only `includeOutput`/`selectedOutputs`, both declared; the UI makes no v2 calls at all; `requestJson` appends nothing implicitly and no v2 cache buster exists). Third-party callers appending a tracking tag or cache buster do break, which is why `api-reference/getting-started.mdx` documents the behavior rather than leaving it to be discovered from a 400. + +Query and body schemas are **`.strict()`** — and `.strict()` binds the **top level only**. A strict body containing a non-strict nested object still drops unknown keys one level down, which is the headline `filter` bug at a smaller scale: `sort: [{ field, direction, nulls: 'last' }]` answered 200 and ordered by the default. Strictness belongs on the shared nested schema (`sortSpecSchema`'s element, `tableViewConfigSchema`), not restated per body. + +Before tightening a schema that is **also** a response or a stored blob, make the read canonical first. `table_views.config` is schemaless JSONB, so a legacy row carrying a retired key would fail a newly strict response parse and become a 500; `normalizeStoredViewConfig` projects the stored blob onto the declared keys so the tightening is safe in both directions. Zod strips unknown keys by default, so a non-strict schema answers `?limit=1` with 200 and the whole set — the caller believes it bounded the response and it did not. That is a contract lie, and on an uncapped list it is also an unbounded-response risk. + +Error messages name the field and, where there is one, the escape hatch: + +``` +limit must be a whole number +limit cannot exceed 100 +search cannot be empty +sortBy: expected one of "name" | "createdAt" | "updatedAt" +Limit cannot exceed 1000; use limit=0 to stream all rows, or create an export +``` + +That last one is the standard to aim for. A message that only says `Invalid input` fails this rule — the caller cannot act on it. + +## Rule 5 — contract first, then use case, then route + +Order matters because each layer is checked against the one before it. + +1. **Contract** in `lib/api/contracts/v2/.ts` via `defineRouteContract`. Response schemas are `.parse`d on the way out, so a field the producer does not actually emit becomes a 500 on a successful read — assert only what you can prove. +2. **Application use case** owns canonical loading, authorization, business behavior, and audit. The route's `present` receives the use-case result **and the parsed request**, so a presenter reads request params (the active `sortBy`/`sortOrder` and filters it stamps into a cursor) straight from `query`/`params` rather than making the use case carry an HTTP concern back out. +3. **Route** with `defineV2JsonRoute`, declaring `contract`, `auth: v2ApiKeyAuth`, `operation`, `rateLimit`, `errorPolicy`, `mapInput`, `useCase`, `present`. Auth and rate limiting run before parsing. +4. **OpenAPI description** in `lib/api/contracts/v2/openapi/.ts`, then `bun run generate:openapi`. A description that claims behaviour the route does not have is the same class of bug as a wrong schema. + +## Rule 6 — a transient failure says when to come back + +A response the caller is *expected* to retry must say how long to wait. Two statuses qualify, and both are wired: + +| Status | Source of the value | Where | +|---|---|---| +| 429 | The caller's own token bucket (`retryAfterMs`, else `resetAt - now`) | `v2RateLimitError` | +| 503 | A fixed floor, `RETRY_AFTER_SECONDS_BY_STATUS` | `v2Error`, applied automatically | + +The 503 default is applied by `v2Error` keyed on the response *status* — `Retry-After` is defined against the status, and the status is the only half of the code/status pair a client sees — so every 503 the surface can emit carries it — the three route builders' `unhandledErrorResponse`, the execute and resume routes, and `serviceFailureResponse`'s `infra` failures. A route with a better number passes `headers: { 'Retry-After': … }` and wins. + +Do not add a default for any other code. 400/403/404/409 are not fixed by waiting, and 402 (`USAGE_LIMIT_EXCEEDED`) is resolved by a billing change, not by time. + +**Where a policy already knows the wait, carry it — do not re-guess it at the transport.** The admission descriptors in `lib/core/admission/transient-failure` declare `retryAfterSeconds` per denial. That value used to be dropped when the descriptor was mapped onto a preprocess error, so a concurrency denial arrived as a bare 429 with no `Retry-After` even though the policy had named the wait. It now travels `descriptor.retryAfterSeconds → PreprocessExecutionError.retryAfterMs → ExecuteWorkflowServiceFailure.retryAfterMs → serviceFailureResponse`. The `v2Error` default is the floor for paths with *no* policy signal, not the source of truth. + +**A failure whose outcome is unknown must not advise a retry.** `ASYNC_ENQUEUE_AMBIGUOUS` is a 503 whose enqueue may have succeeded — it deliberately retains its execution-ID claim. Telling that caller to come back in 5 seconds invites a client with no `X-Run-Id` to start and bill a second run. It passes `omitRetryAfter: true` and returns the run id so the caller reconciles instead. Any future "we don't know if it happened" failure does the same. + +RFC 9110 §10.2.3 gives 503 this field's clearest meaning — "how long the service is expected to be unavailable to the client". Note the requirement level is only `MAY`, on 503 (§15.6.4) and, via RFC 6585 §4, on 429. It is `SHOULD` on exactly one status, 413, and only when the condition is temporary; none of Sim's 413s are temporary — they are fixed ceilings, on the request body and on the collections a response must materialize — so it correctly sends none. + +## Deliberate non-adoptions + +Audited against the primary specs and against Stripe, GitHub, and Google's AIPs. Each is a considered "no", not an oversight. Re-open one only with new evidence. + +| Practice | Verdict | Why | +|---|---|---| +| **RFC 9457 `application/problem+json`** | No | 9457 §4 steers APIs with an existing format toward keeping it: "Problem details are intended to avoid the necessity of establishing new 'fault' or 'error' document formats, **not to replace existing domain-specific formats**." Nothing in it is a `MUST` to adopt, and none of Stripe, GitHub, or Google use it. Our envelope is load-bearing for every client. **The default error shape does not change.** | +| **`RateLimit`/`RateLimit-Policy` (IETF draft)** | No | Still an unpublished draft (`-11`, May 2026), returned "Not ready" at HTTPDIR review, and on its **third mutually incompatible wire format** — anything built against `-07` or earlier is already broken. None of the three surveyed APIs emit it; GitHub uses `x-ratelimit-*`, as we do. | +| **Renaming `X-RateLimit-*` per RFC 6648** | No | 6648 is a `SHOULD NOT` binding *creators of new* parameters, and §1 item 4 "**makes no recommendation as to whether existing 'X-' parameters ought to remain in use or be migrated**". Appendix B argues the migration is itself the interoperability harm. A rename is a client-visible break bought with nothing. | +| **`X-RateLimit-Reset` as delta-seconds** | No | It is an absolute ISO 8601 timestamp, so it is clock-skew sensitive — but the response where timing actually decides behaviour (429) also carries `Retry-After`, which is skew-free. The absolute value stays useful for scheduling. | +| **422 for semantic validation** | No | RFC 9110 §15.5.21 defines 422, but Appendix B.3 records that 9110 **deleted** RFC 4918's clause saying 400 was inappropriate. 400 covers "cannot or will not process… perceived to be a client error". The split is convention, not requirement — GitHub splits, Stripe and Google do not. Our machine-readable `error.code` already carries the distinction, and restatusing now breaks clients. | +| **`Location` on 201** | No | §9.3.3 makes this a `SHOULD` **for POST**; the status code itself (§15.3.2) requires nothing and defines the fallback — absent `Location`, the target URI identifies the resource. Declined knowingly: several 201 responses (signed upload sessions, table exports, knowledge folders) have no canonical single-resource GET, so a `Location` would 404, and adopting it on some of the 19 is worse for a client than on none. Every 201 returns the full representation including its `id`. Revisit per-route if one gains a canonical GET. | +| **ETag / `If-None-Match` / `If-Match`** | No | Every v2 response is `Cache-Control: private, no-store` per-caller data, so `If-None-Match` buys nothing. For writes, `If-Match` needs a **strong** validator: §8.8.3.2's strong comparison fails if *either* tag is weak, so a weak ETag silently makes every `If-Match` fail. None of the three surveyed APIs does HTTP optimistic concurrency — Google does the semantics via a resource `etag` **field** (AIP-154), deliberately not the header. If Sim needs optimistic concurrency, do it that way. | +| **`Deprecation` / `Sunset` on v1** | Not yet | RFC 9745 (Standards Track) and RFC 8594 (Informational) both apply, and GitHub emits both. But `Sunset` is a timestamp and 9745 §4 makes `Sunset >= Deprecation` a `MUST`, so emitting either commits Sim to a v1 retirement date — a product decision, not an engineering one. When that date exists: `Deprecation` is an RFC 9651 Structured Field **Date** (`@1688169599`); `Sunset` is an **HTTP-date** (`Sat, 31 Dec 2033 23:59:59 GMT`). Two encodings in one response — the most common implementation error here. | +| **`application/merge-patch+json`** | No | v2 PATCH bodies are merge-patch *shaped* — absent means unchanged, `null` clears — but they are `.strict()`, so unknown members are rejected where RFC 7396 §2 would merge them, and nested objects are replaced wholesale rather than merged. Advertising the media type would over-claim. Document the semantics per contract instead. | + +## Idempotency: at-most-once, not replay + +`POST /workflows/{id}/execute` accepts `X-Run-Id`, a caller-supplied run identifier claimed through the `idempotency_key` table (`execution-id-claim.ts`). It is a **uniqueness claim, not an idempotency key**, and the distinction is deliberate and already published in the operation description: + +- First use wins and runs. +- Any reuse returns **409** with `error.details.code: "RUN_ID_CONFLICT"`, the run id in `error.details.runId`, and an `X-Run-Id` response header. It never replays the earlier run's result — the client recovers it by polling the runs resource. +- Claims are durable tombstones, so deleting execution logs cannot make an id reusable. + +That makes the money path safe against double-execution **for callers that opt in**. What it is not: a Stripe-style `Idempotency-Key` that stores and replays the original status and body. Building that means a request fingerprint, a retention window, an in-flight-vs-completed distinction (the expired IETF draft would have these be 422 and 409 respectively), and somewhere to put a large synchronous execution body. It is a designed piece of work, not an increment — do not half-build it by aliasing the header name, which would invite clients written against Stripe semantics to treat our 409 as a hard failure. + +## Cursors are opaque, not trusted + +The base64-JSON cursor is **not signed**, and does not need to be. Tampering is bounded by construction, and that is a property to preserve: + +- Every key value is re-validated by its `KeysetKey.bind`, which returns `null` for a wrong-typed or unparseable value and becomes a 400. A forged cursor cannot reach SQL as `NaN` or an `Invalid Date`. +- The sort and a fingerprint of the filters are stamped into the cursor and re-checked (`decodeSortedCursor`), so a cursor from a differently-sorted or differently-filtered query is a 400, not a silently skipped page. The filters are hashed (SHA-256, via `lib/api/cursor-binding.ts`) rather than embedded, so the token stays short and a caller cannot cheaply construct a filter that collides with another sequence's stamp. +- The offset codec rejects anything that is not a non-negative integer. +- Authorization is **never** carried in the cursor. Every list re-derives its workspace scope from the authenticated principal, so a cursor lifted from another query — or another tenant — can only move the caller within their own authorized result set. + +The consequence to keep true: **never put a resource id, filter, or scope into a cursor and then trust it on the way back.** A cursor is a position hint, never an input to an access decision. + +## Checklist + +Run this against any new or changed v2 endpoint. + +- [ ] Success body is exactly `{data}` or `{data, nextCursor}`; failures are exactly `{error:{code,message,details?}}`. +- [ ] Route uses a shared builder; no hand-built `NextResponse.json`. +- [ ] Query and body schemas are `.strict()`. +- [ ] The contract declares a `query` — `noInputSchema` when the endpoint takes no query params, never omission. +- [ ] No caller-supplied value can produce a 500 — check every numeric param reaches SQL as a validated integer, and that any bound value passed as an argument to a SQL function carries an explicit type. +- [ ] `limit` comes from `v2PaginationFields`, not a hand-written `z.coerce.number()`. +- [ ] If the response carries `nextCursor`, the query accepts `limit` + `cursor` and the query actually applies them. +- [ ] The cursor is bound to every param that filters or orders the sequence, and to none that do not (never `limit`), with the binding declared in `list-pagination.test.ts`. +- [ ] Keyset sorts end in a unique `id` key. +- [ ] The list is classified in `list-pagination.test.ts`. +- [ ] Cross-tenant access answers 404, never 403 — and carries `Cache-Control: private, no-store`, because RFC 9110 §15.5.5 makes 404 heuristically cacheable and an authorization-dependent 404 must never be stored. `v2Error` sets this unconditionally; do not build a v2 response any other way. +- [ ] A retryable failure says when: 429 and 503 carry `Retry-After`. No other status invents one. +- [ ] 403s carry a machine-readable `details.code` from `FORBIDDEN_DETAIL_CODES`, thrown as `ForbiddenOperationError` in the domain rather than attached at the route. +- [ ] Nested objects inside a `.strict()` body are strict too — `.strict()` does not recurse. +- [ ] Ordering uses `sortBy` + `sortOrder`; boolean query params use `booleanQueryFlagSchema`. +- [ ] Validation messages name the field and echo the valid set. +- [ ] Response schema matches every field the route actually emits. +- [ ] OpenAPI description regenerated and truthful about pagination. +- [ ] `bun run type-check`, `bun run check:api-validation`, `bun run check:openapi` pass. + +## Known gap + +A 405 on a path that *does* have a route file but does not export that verb is generated by Next.js before any Sim code runs: zero-byte body, no `content-type`, and no `Allow` header, which RFC 9110 §15.5.6 requires. Fixing it means either exporting explicit rejecting handlers from every v2 route file or intercepting in `apps/sim/proxy.ts` with a static path→methods table. Neither is done. Unknown *paths* are handled — the catch-all covers those. diff --git a/.gitattributes b/.gitattributes index 8347b118c47..ffd7862b0be 100644 --- a/.gitattributes +++ b/.gitattributes @@ -21,6 +21,15 @@ Dockerfile* text eol=lf .gitignore text eol=lf .gitattributes text eol=lf +# Source files always diff as text. Git otherwise classifies a whole file as +# binary the moment it contains a NUL byte, hiding every line of it from review. +*.ts diff +*.tsx diff +*.js diff +*.jsx diff +*.json diff +*.md diff + # Denote all files that are truly binary and should not be modified *.png binary *.jpg binary diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index cf2f27d7836..746c341cae8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -697,7 +697,15 @@ jobs: desktop-release: name: Desktop Release needs: [create-release, check-desktop-signing, detect-version] - if: needs.check-desktop-signing.outputs.configured == 'true' + # Suppress the implicit success() check: check-desktop-signing has an + # intentionally skipped transitive dependency on main, which would + # otherwise cascade-skip this job even when every direct need succeeded. + if: >- + !cancelled() && + needs.create-release.result == 'success' && + needs.check-desktop-signing.result == 'success' && + needs.detect-version.result == 'success' && + needs.check-desktop-signing.outputs.configured == 'true' permissions: contents: write uses: ./.github/workflows/desktop-release.yml @@ -707,14 +715,13 @@ jobs: secrets: inherit # Per-env desktop prereleases: a dev/staging push that touches shell code - # publishes an environment-tagged GitHub prerelease (vX.Y.Z-dev.N from dev, - # vX.Y.Z-staging.N from staging). Each environment's /api/desktop/update feed - # offers only its stream, so dev-pointed shells pick up dev builds, - # staging-pointed shells staging builds, and prod-pointed shells stable - # releases — independently. Unlike stable releases, prereleases build even + # publishes an environment-tagged GitHub prerelease to the public, + # release-only simstudioai/sim-desktop-releases repository. Keeping these + # builds out of this source repository prevents its followers from receiving + # every internal shell release. Each environment's /api/desktop/update feed + # still offers only its own stream. Unlike stable releases, prereleases build # before the Apple signing secrets exist — unsigned, so the update pipeline - # is testable end to end; installed shells detect the missing Developer ID - # and offer a manual download instead of a Squirrel install. + # remains testable end to end with a manual download. create-desktop-prerelease: name: Create Desktop Prerelease runs-on: blacksmith-4vcpu-ubuntu-2404 @@ -724,7 +731,7 @@ jobs: # cancelled") so a probe failure can't produce a release with no build. if: ${{ !cancelled() && needs.detect-desktop-changes.outputs.changed == 'true' && needs.check-desktop-signing.result == 'success' }} permissions: - contents: write + contents: read outputs: version: ${{ steps.version.outputs.version }} steps: @@ -734,10 +741,16 @@ jobs: - name: Compute prerelease version and create draft release id: version env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - GH_REPO: ${{ github.repository }} + DESKTOP_RELEASE_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }} + GH_TOKEN: ${{ github.token }} + PRERELEASE_REPOSITORY: simstudioai/sim-desktop-releases + SOURCE_REPOSITORY: ${{ github.repository }} SIGNED: ${{ needs.check-desktop-signing.outputs.configured }} run: | + if [ -z "$DESKTOP_RELEASE_TOKEN" ]; then + echo "::error::DESKTOP_RELEASE_TOKEN is required to publish desktop prereleases." + exit 1 + fi if [ "$GITHUB_REF" = "refs/heads/dev" ]; then CHANNEL=dev; APP_NAME="Sim Dev"; else CHANNEL=staging; APP_NAME="Sim Staging"; fi # Prerelease core = next patch after the latest stable release, so # channel builds always outrank the stable they are built on top of @@ -747,7 +760,7 @@ jobs: # Fail loudly if the query itself fails: silently falling back to # v0.0.0 would publish a channel build that sorts below the shipped # stable, and installed shells would never see it as an update. - if ! LATEST="$(gh release list --exclude-pre-releases --limit 1 --json tagName --jq '.[0].tagName')"; then + if ! LATEST="$(gh release list --repo "$SOURCE_REPOSITORY" --exclude-pre-releases --limit 1 --json tagName --jq '.[0].tagName')"; then echo "::error::Could not query the latest stable release." exit 1 fi @@ -767,12 +780,14 @@ jobs: fi # Draft until the build uploads its artifacts: drafts are invisible # to the update feed, so a failed or in-flight build can never take - # the channel down with an assetless release. Publishing later also - # defers tag creation, so failed builds strand no tags. - gh release create "$TAG" \ + # the channel down with an assetless release. The release-only repo + # has no source commit for this SHA, so its tag intentionally targets + # that repository's main branch; the notes retain the source SHA. + GH_TOKEN="$DESKTOP_RELEASE_TOKEN" gh release create "$TAG" \ + --repo "$PRERELEASE_REPOSITORY" \ --draft \ --prerelease \ - --target "$GITHUB_SHA" \ + --target main \ --title "$TAG" \ --notes "$NOTES" echo "version=$TAG" >> "$GITHUB_OUTPUT" @@ -781,6 +796,9 @@ jobs: desktop-prerelease: name: Desktop Prerelease Build needs: [create-desktop-prerelease, check-desktop-signing] + # The reusable workflow declares contents: write for its stable-release + # path. GitHub cannot elevate a caller's token, even though this prerelease + # path uses the dedicated cross-repository token for its actual upload. permissions: contents: write uses: ./.github/workflows/desktop-release.yml @@ -799,14 +817,19 @@ jobs: timeout-minutes: 5 needs: [create-desktop-prerelease, desktop-prerelease] permissions: - contents: write + contents: read env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - GH_REPO: ${{ github.repository }} + GH_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }} + GH_REPO: simstudioai/sim-desktop-releases TAG: ${{ needs.create-desktop-prerelease.outputs.version }} steps: - name: Publish the draft release - run: gh release edit "$TAG" --draft=false + run: | + if [ -z "$GH_TOKEN" ]; then + echo "::error::DESKTOP_RELEASE_TOKEN is required to publish desktop prereleases." + exit 1 + fi + gh release edit "$TAG" --draft=false # Keep the release list tidy: per channel, retain the newest 5 prereleases # and delete the rest (with their tags, so dev force-resets don't strand @@ -818,13 +841,17 @@ jobs: timeout-minutes: 5 needs: [publish-desktop-prerelease] permissions: - contents: write + contents: read env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - GH_REPO: ${{ github.repository }} + GH_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }} + GH_REPO: simstudioai/sim-desktop-releases steps: - name: Delete stale prereleases run: | + if [ -z "$GH_TOKEN" ]; then + echo "::error::DESKTOP_RELEASE_TOKEN is required to prune desktop prereleases." + exit 1 + fi if [ "$GITHUB_REF" = "refs/heads/dev" ]; then CHANNELS='(dev|alpha)'; else CHANNELS='(staging|beta)'; fi gh release list --limit 100 --json tagName,isPrerelease,isDraft,createdAt \ --jq "[.[] | select(.isPrerelease and (.isDraft | not) and (.tagName | test(\"-${CHANNELS}\\\\.\")))] | sort_by(.createdAt) | reverse | .[5:] | .[].tagName" | diff --git a/.github/workflows/desktop-e2e.yml b/.github/workflows/desktop-e2e.yml index 7632b0bea66..7b87a84bc8d 100644 --- a/.github/workflows/desktop-e2e.yml +++ b/.github/workflows/desktop-e2e.yml @@ -10,6 +10,9 @@ name: Desktop E2E on: workflow_dispatch: +permissions: + contents: read + concurrency: group: desktop-e2e-${{ github.ref }} cancel-in-progress: true diff --git a/.github/workflows/desktop-release.yml b/.github/workflows/desktop-release.yml index a160d22c37f..ed014c85039 100644 --- a/.github/workflows/desktop-release.yml +++ b/.github/workflows/desktop-release.yml @@ -1,10 +1,10 @@ name: Desktop Release (macOS) # Builds, signs, notarizes, and uploads the desktop app to an existing GitHub -# release. Ordering is load-bearing: scripts/create-single-release.ts skips -# creation when the tag already exists, so this workflow must never create the -# release itself — it only uploads assets after create-release ran (wired via -# workflow_call from ci.yml with needs: [create-release]). +# release. Stable releases live in this source repository; dev and staging +# releases live in simstudioai/sim-desktop-releases. Ordering is load-bearing: +# scripts/create-single-release.ts skips creation when the stable tag already +# exists, so this workflow must never create a release itself. on: workflow_call: @@ -54,6 +54,46 @@ jobs: - name: Checkout code uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + # Prerelease versions carry their environment in the tag: -dev.N is a + # dev build, -staging.N a staging build. Legacy -alpha/-beta tags remain + # accepted while already-published builds age out. The channel decides the app's + # identity (name/bundle id — a separate app per environment, installable + # side by side) and the default origin baked into the bundle, which in + # turn selects the update feed the installed app polls. + - name: Resolve channel identity + id: channel + env: + VERSION: ${{ inputs.version }} + run: | + case "$VERSION" in + *-dev.*|*-alpha.*) + NAME='Sim Dev'; APP_ID=ai.sim.desktop.dev; ORIGIN=https://www.dev.sim.ai; RELEASE_REPOSITORY=simstudioai/sim-desktop-releases; TOKEN_KIND=prerelease ;; + *-staging.*|*-beta.*) + NAME='Sim Staging'; APP_ID=ai.sim.desktop.staging; ORIGIN=https://www.staging.sim.ai; RELEASE_REPOSITORY=simstudioai/sim-desktop-releases; TOKEN_KIND=prerelease ;; + *) + NAME='Sim'; APP_ID=ai.sim.desktop; ORIGIN=''; RELEASE_REPOSITORY="$GITHUB_REPOSITORY"; TOKEN_KIND=stable ;; + esac + { + echo "name=$NAME" + echo "app_id=$APP_ID" + echo "origin=$ORIGIN" + echo "release_repository=$RELEASE_REPOSITORY" + echo "token_kind=$TOKEN_KIND" + } >> "$GITHUB_OUTPUT" + echo "Building $NAME ($APP_ID) for $RELEASE_REPOSITORY; default origin: ${ORIGIN:-production}" + + - name: Validate release authentication + if: ${{ inputs.publish }} + env: + DESKTOP_RELEASE_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }} + RELEASE_REPOSITORY: ${{ steps.channel.outputs.release_repository }} + TOKEN_KIND: ${{ steps.channel.outputs.token_kind }} + run: | + if [ "$TOKEN_KIND" = prerelease ] && [ -z "$DESKTOP_RELEASE_TOKEN" ]; then + echo "::error::DESKTOP_RELEASE_TOKEN is required to publish prereleases to $RELEASE_REPOSITORY." + exit 1 + fi + - name: Setup Bun uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 with: @@ -91,32 +131,6 @@ jobs: exit 1 fi - # Prerelease versions carry their environment in the tag: -dev.N is a - # dev build, -staging.N a staging build. Legacy -alpha/-beta tags remain - # accepted while already-published builds age out. The channel decides the app's - # identity (name/bundle id — a separate app per environment, installable - # side by side) and the default origin baked into the bundle, which in - # turn selects the update feed the installed app polls. - - name: Resolve channel identity - id: channel - env: - VERSION: ${{ inputs.version }} - run: | - case "$VERSION" in - *-dev.*|*-alpha.*) - NAME='Sim Dev'; APP_ID=ai.sim.desktop.dev; ORIGIN=https://www.dev.sim.ai ;; - *-staging.*|*-beta.*) - NAME='Sim Staging'; APP_ID=ai.sim.desktop.staging; ORIGIN=https://www.staging.sim.ai ;; - *) - NAME='Sim'; APP_ID=ai.sim.desktop; ORIGIN='' ;; - esac - { - echo "name=$NAME" - echo "app_id=$APP_ID" - echo "origin=$ORIGIN" - } >> "$GITHUB_OUTPUT" - echo "Building $NAME ($APP_ID) default origin: ${ORIGIN:-production}" - - name: Bundle main and preload working-directory: apps/desktop env: @@ -177,9 +191,24 @@ jobs: - name: Upload artifacts to the release if: ${{ inputs.publish }} env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + DESKTOP_RELEASE_TOKEN: ${{ secrets.DESKTOP_RELEASE_TOKEN }} + RELEASE_REPOSITORY: ${{ steps.channel.outputs.release_repository }} + SOURCE_RELEASE_TOKEN: ${{ github.token }} + TOKEN_KIND: ${{ steps.channel.outputs.token_kind }} VERSION: ${{ inputs.version }} run: | + case "$TOKEN_KIND" in + prerelease) GH_TOKEN="$DESKTOP_RELEASE_TOKEN" ;; + stable) GH_TOKEN="$SOURCE_RELEASE_TOKEN" ;; + *) + echo "::error::Unknown desktop release token kind: $TOKEN_KIND" + exit 1 ;; + esac + if [ -z "$GH_TOKEN" ]; then + echo "::error::No GitHub token is available to publish to $RELEASE_REPOSITORY." + exit 1 + fi + export GH_TOKEN # electron-builder's GitHub provider always names the manifest # latest-mac.yml (channels are a generic-provider concept), and the # update feed expects exactly that asset name on every release — @@ -198,6 +227,7 @@ jobs: apps/desktop/release/*.zip \ apps/desktop/release/*.blockmap \ apps/desktop/release/latest-mac.yml \ + --repo "$RELEASE_REPOSITORY" \ --clobber - name: Upload artifacts to the workflow run diff --git a/apps/desktop/README.md b/apps/desktop/README.md index eab15594e1f..766873fab65 100644 --- a/apps/desktop/README.md +++ b/apps/desktop/README.md @@ -104,7 +104,7 @@ Pre-release share (no Developer ID yet): `SIM_DESKTOP_DEFAULT_ORIGIN=https://www The build also derives the app icon from `SIM_DESKTOP_DEFAULT_ORIGIN`. Every channel uses the exact production icon with its white background and black `sim` mark. Non-production channels add a thin outline using existing platform colors: dev uses orange, staging uses Loop blue, and localhost uses Workflow violet. The macOS menu-bar icon also carries a compact `D`, `S`, or `L` subscript for those environments; production remains unmarked. Native Icon Composer assets live in `build/`; `scripts/build.ts` copies the selected variant to the ignored `build/generated-icon.icon` path consumed by electron-builder. Electron-builder compiles it to `Assets.car` and derives the legacy `.icns` fallback from the same source. Matching 512px PNGs in `static/` provide the Dock icon for unpackaged runs. CI (`.github/workflows/desktop-release.yml`, wired into `ci.yml`): -- Runs only after `create-release` on a `vX.Y.Z:` commit to main — **never before**: `scripts/create-single-release.ts` skips creation if the tag exists, so a desktop job publishing first would eat the changelog. The job builds `--publish never` and uploads assets with `gh release upload --clobber` (idempotent re-runs). +- Stable builds run only after `create-release` on a `vX.Y.Z:` commit to main — **never before**: `scripts/create-single-release.ts` skips creation if the tag exists, so a desktop job publishing first would eat the changelog. Stable assets remain on `simstudioai/sim`; dev/staging assets publish to the public `simstudioai/sim-desktop-releases` repository so source-repository followers are not notified for internal shell builds. The job builds `--publish never` and uploads assets with `gh release upload --clobber` (idempotent re-runs). - **Secrets gate**: `check-desktop-signing` in `ci.yml` probes the six Apple secrets and skips the desktop job with a warning until they exist — releases never fail on a missing Apple account, and the first release after the secrets land ships desktop artifacts automatically. Manual/one-off builds: Actions → "Desktop Release (macOS)" → Run workflow with a `vX.Y.Z` version (`publish: false` uploads artifacts to the run instead of the release). - The product semver is **injected** from the release tag into `apps/desktop/package.json` at build time (repo package versions are placeholders). A mismatch guard fails the build. - Fuses are flipped at package time (`electronFuses` in `electron-builder.yml`): runAsNode off, NODE_OPTIONS off, inspect args off, ASAR-only + integrity validation, cookie encryption on, `strictlyRequireAllFuses` so new fuses fail loudly on Electron bumps. @@ -120,6 +120,7 @@ Required repo secrets (owner: whoever holds the Apple Developer account; calenda | `APPLE_API_KEY_ID` | API key ID | | `APPLE_API_ISSUER` | API issuer ID | | `APPLE_TEAM_ID` | Developer team ID | +| `DESKTOP_RELEASE_TOKEN` | Fine-grained GitHub token with `Contents: write` on only `simstudioai/sim-desktop-releases`; used to create, upload, publish, and prune dev/staging releases | ## Desktop-only features (how to add them cleanly) @@ -168,7 +169,7 @@ Raw local file bytes are never exposed through the preload bridge and cannot be ## Auto-update, channels, rollout, rollback -- `electron-updater` reads the GitHub Releases feed (`publish` is pinned to `simstudioai/sim`); deltas via `.zip.blockmap`. Install is prompt-based (Restart Now / Later; Later installs on quit) — never forced mid-session. +- `electron-updater` reads the deployment's `/api/desktop/update` feed; production resolves stable releases from `simstudioai/sim`, while dev/staging resolve prereleases from `simstudioai/sim-desktop-releases`. Artifact downloads go directly to GitHub and deltas use `.zip.blockmap`. Install is prompt-based (Restart Now / Later; Later installs on quit) — never forced mid-session. - Streams: production follows stable `X.Y.Z` releases, dev follows `-dev.N`, and staging follows `-staging.N`. The feed still recognizes legacy `-alpha.N`/`-beta.N` releases during migration. - Staged rollout: after publishing, edit `stagingPercentage: 10` into the release's `latest-mac.yml`, then raise as crash metrics stay clean. - Rollback: a pulled release must be superseded by a **higher** version — users on the broken build will not reinstall an equal one. (A blocked-versions kill-switch was removed as unwired dead code; reintroduce it in `updater.ts` if a remote config source ever exists to feed it.) diff --git a/apps/desktop/src/main/browser-agent/session.test.ts b/apps/desktop/src/main/browser-agent/session.test.ts index 5e96931f06f..f846cb90749 100644 --- a/apps/desktop/src/main/browser-agent/session.test.ts +++ b/apps/desktop/src/main/browser-agent/session.test.ts @@ -21,6 +21,7 @@ interface MockView { setPermissionCheckHandler: ReturnType } on: ReturnType + setUserAgent: ReturnType setWindowOpenHandler: ReturnType loadURL: ReturnType reload: ReturnType @@ -169,6 +170,18 @@ describe('browser-agent session', () => { expect(onTabNavigated).toHaveBeenCalledWith(contents, true) }) + it('gives every tab a user agent with no Electron token in it', () => { + const first = session.ensureTab() + const second = session.addTab() + + for (const tab of [first, second]) { + const contents = (tab.view as unknown as MockView).webContents + const agent = contents.setUserAgent.mock.calls.at(-1)?.[0] as string | undefined + expect(agent).toMatch(/^Mozilla\/5\.0 \(.+\) .*Chrome\/\d+\.0\.0\.0 Safari\/537\.36$/) + expect(agent).not.toMatch(/Electron|Sim\//) + } + }) + it('settles the tab spinner when only subresources are still loading', () => { const tab = session.ensureTab() const contents = (tab.view as unknown as MockView).webContents @@ -1757,7 +1770,7 @@ describe('browser-agent session', () => { expect(event.preventDefault).toHaveBeenCalledOnce() }) - it('permission handlers deny every request on the agent partition', () => { + it('permission handlers deny every request on the agent partition but the copy button', () => { const tab = session.ensureTab() const ses = (tab.view as unknown as MockView).webContents.session const requestHandler = ses.setPermissionRequestHandler.mock.calls[0][0] as ( @@ -1765,12 +1778,26 @@ describe('browser-agent session', () => { permission: string, callback: (granted: boolean) => void ) => void - const callback = vi.fn() - requestHandler(null, 'media', callback) - expect(callback).toHaveBeenCalledWith(false) + const checkHandler = ses.setPermissionCheckHandler.mock.calls[0][0] as ( + wc: unknown, + permission: string + ) => boolean + + // Reading the clipboard would leak whatever the user last copied anywhere + // else, so it stays denied alongside everything a page could spy through. + for (const permission of ['media', 'geolocation', 'notifications', 'clipboard-read']) { + const callback = vi.fn() + requestHandler(null, permission, callback) + expect(callback).toHaveBeenCalledWith(false) + expect(checkHandler(null, permission)).toBe(false) + } - const checkHandler = ses.setPermissionCheckHandler.mock.calls[0][0] as () => boolean - expect(checkHandler()).toBe(false) + // Chromium routes navigator.clipboard.writeText through this one; denying + // it silently broke every copy button that does not use execCommand. + const writeCallback = vi.fn() + requestHandler(null, 'clipboard-sanitized-write', writeCallback) + expect(writeCallback).toHaveBeenCalledWith(true) + expect(checkHandler(null, 'clipboard-sanitized-write')).toBe(true) }) it('leaves nothing of the signed-out user behind in the browser profile', async () => { diff --git a/apps/desktop/src/main/browser-agent/session.ts b/apps/desktop/src/main/browser-agent/session.ts index 3d1b1589e14..fbedb905a3e 100644 --- a/apps/desktop/src/main/browser-agent/session.ts +++ b/apps/desktop/src/main/browser-agent/session.ts @@ -62,6 +62,7 @@ import { isBlockedSubresourceUrl, subresourceNeedsResolution, } from '@/main/browser-agent/url-guard' +import { browserUserAgent } from '@/main/browser-agent/user-agent' import type { BrowserSessionSnapshot } from '@/main/desktop-chat-session-store' import { suggestedFilename, uniqueDownloadPath } from '@/main/downloads' import { @@ -809,16 +810,40 @@ export async function importAgentCookies( return { imported, failed } } +/** + * The single site permission a browsing surface cannot withhold: the one every + * "Copy" button on the web goes through. Blanket-denying it made + * `navigator.clipboard.writeText` reject with `NotAllowedError`, so those + * buttons did nothing at all — no error, no copied text — while the legacy + * `document.execCommand('copy')` path kept working, which is why only some + * sites looked broken. + * + * Granting it hands the page no reach it lacked: Chromium still requires the + * document to be focused and to hold a transient user activation, and a + * sanitized write only places text the page already renders onto the clipboard. + * Reading stays denied — that is the direction that would leak whatever the + * user last copied from anywhere else. + */ +const ALLOWED_SITE_PERMISSIONS = new Set(['clipboard-sanitized-write']) + /** * Default-deny hardening for the agent partition. Site permissions remain - * denied, while uploads use Chromium's native file chooser and downloads are - * saved into the device-level browser download directory. + * denied apart from ALLOWED_SITE_PERMISSIONS, while uploads use Chromium's + * native file chooser and downloads are saved into the device-level browser + * download directory. */ function configureAgentPartition(ses: Session): void { if (configuredPartitions.has(ses)) return configuredPartitions.add(ses) - ses.setPermissionRequestHandler((_wc, _permission, callback) => callback(false)) - ses.setPermissionCheckHandler(() => false) + ses.setPermissionRequestHandler((_wc, permission, callback) => + callback(ALLOWED_SITE_PERMISSIONS.has(permission)) + ) + ses.setPermissionCheckHandler((_wc, permission) => ALLOWED_SITE_PERMISSIONS.has(permission)) + // Service workers do not inherit a tab's user agent. With only the tab's set, + // the document request carries the browser string while the worker's own + // script request still announces Electron — and on a site that routes its + // fetches through a worker, that is the one the server sees. + ses.setUserAgent(browserUserAgent()) // SSRF choke point for the agent partition. Document navigations (top-level + // iframes) get the full DNS-resolving check — the one seam every navigation // passes through, including page-initiated ones the driver never sees (server @@ -1101,6 +1126,10 @@ function createTabView(): WebContentsView { const contents = view.webContents registerAgentWebContents(contents) configureAgentPartition(contents.session) + // The session default does not reach a WebContents that already exists, and + // the first tab is what brings the session into being, so each tab sets its + // own as well — otherwise tab one browses as Electron and the rest as Chrome. + contents.setUserAgent(browserUserAgent()) attachAgentContextMenu(contents, { addToChat: (text) => withBrowserScope(scopeId, () => addPageSelectionToChat(contents, text)), openTab: (url) => withBrowserScope(scopeId, () => openTabWithUrl(url, false)), diff --git a/apps/desktop/src/main/browser-agent/user-agent.test.ts b/apps/desktop/src/main/browser-agent/user-agent.test.ts new file mode 100644 index 00000000000..db63efa7141 --- /dev/null +++ b/apps/desktop/src/main/browser-agent/user-agent.test.ts @@ -0,0 +1,43 @@ +import { app } from 'electron' +import { describe, expect, it, vi } from 'vitest' +import { browserUserAgent, stockChromeUserAgent } from '@/main/browser-agent/user-agent' + +vi.mock('electron', () => import('@/test/electron-mock')) + +const ELECTRON_DEFAULT = + 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Sim/1.0.0 Chrome/140.0.7339.207 Electron/43.1.1 Safari/537.36' + +describe('stockChromeUserAgent', () => { + it('drops the application and Electron tokens a browser allowlist rejects', () => { + const agent = stockChromeUserAgent(ELECTRON_DEFAULT) + expect(agent).not.toMatch(/Electron/) + expect(agent).not.toMatch(/Sim\//) + }) + + it('reproduces the desktop string Chrome sends under user-agent reduction', () => { + expect(stockChromeUserAgent(ELECTRON_DEFAULT)).toBe( + 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36' + ) + }) + + it('keeps the platform token of the machine it is running on', () => { + const windowsDefault = + 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Sim/1.0.0 Chrome/140.0.7339.207 Electron/43.1.1 Safari/537.36' + expect(stockChromeUserAgent(windowsDefault)).toContain('(Windows NT 10.0; Win64; x64)') + }) + + it('passes through a string that is not a Chromium user agent', () => { + expect(stockChromeUserAgent('curl/8.4.0')).toBe('curl/8.4.0') + expect(stockChromeUserAgent('')).toBe('') + }) +}) + +describe('browserUserAgent', () => { + it('derives from the string Electron would otherwise have sent', () => { + app.userAgentFallback = ELECTRON_DEFAULT + + expect(browserUserAgent()).toBe( + 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36' + ) + }) +}) diff --git a/apps/desktop/src/main/browser-agent/user-agent.ts b/apps/desktop/src/main/browser-agent/user-agent.ts new file mode 100644 index 00000000000..9e988c67987 --- /dev/null +++ b/apps/desktop/src/main/browser-agent/user-agent.ts @@ -0,0 +1,46 @@ +/** + * The user agent the browser resource presents to sites. + * + * Electron's default string carries two tokens no browser sends — + * `Sim/` and `Electron/`. Chromium's own token sits right + * beside them, but that does not save it: the detection libraries sites gate on + * test for Electron BEFORE Chrome (bowser matches `/electron/i` several + * descriptors ahead of its Chrome one, ua-parser-js reports `Electron` as the + * browser name), so the browser reads as "Electron", which is on nobody's + * supported list. Ashby warns "Ashby does not support this browser"; stricter + * sites refuse to render at all. + * + * Reporting stock Chrome is accurate rather than a disguise — the engine is the + * Chromium build the token already names, and Electron's user-agent client + * hints (`Sec-CH-UA`, `navigator.userAgentData`) only ever carried a Chromium + * brand, so dropping the token makes the header and the hints agree instead of + * contradicting each other. + */ +import { app } from 'electron' + +/** Platform token, then the Chromium major version, in the order a Chromium user agent lists them. */ +const CHROMIUM_USER_AGENT = /^Mozilla\/5\.0 \(([^)]*)\).* Chrome\/(\d+)\./ + +/** + * Rebuilds the default user agent as the string Chrome itself sends. Chrome's + * user-agent reduction fixes the desktop form at + * `Mozilla/5.0 () AppleWebKit/537.36 (KHTML, like Gecko) Chrome/.0.0.0 Safari/537.36`, + * so keeping the platform token and the Chromium major version — and zeroing + * the rest — reproduces it exactly, with no room left for an application or + * Electron token. A string that is not a Chromium user agent is returned + * unchanged rather than replaced with a guess. + */ +export function stockChromeUserAgent(defaultUserAgent: string): string { + const match = defaultUserAgent.match(CHROMIUM_USER_AGENT) + if (!match) return defaultUserAgent + const [, platform, chromeMajor] = match + return `Mozilla/5.0 (${platform}) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/${chromeMajor}.0.0.0 Safari/537.36` +} + +/** + * Derived from the string Electron would otherwise have sent, so the reported + * Chromium version tracks whatever Chromium the app actually ships. + */ +export function browserUserAgent(): string { + return stockChromeUserAgent(app.userAgentFallback) +} diff --git a/apps/desktop/src/main/downloads.test.ts b/apps/desktop/src/main/downloads.test.ts index e8308633b60..92721bfa706 100644 Binary files a/apps/desktop/src/main/downloads.test.ts and b/apps/desktop/src/main/downloads.test.ts differ diff --git a/apps/desktop/src/main/updater.test.ts b/apps/desktop/src/main/updater.test.ts index 6feece68361..5f00197d3c2 100644 --- a/apps/desktop/src/main/updater.test.ts +++ b/apps/desktop/src/main/updater.test.ts @@ -391,15 +391,15 @@ describe('initUpdater state machine', () => { }) }) -function manifest(version: string): string { +function manifest(version: string, repository = 'simstudioai/sim'): string { return [ `version: ${version}`, 'files:', - ` - url: https://github.com/simstudioai/sim/releases/download/v${version}/Sim-${version}-universal-mac.zip`, + ` - url: https://github.com/${repository}/releases/download/v${version}/Sim-${version}-universal-mac.zip`, ' sha512: abc', - ` - url: https://github.com/simstudioai/sim/releases/download/v${version}/Sim-${version}-universal.dmg`, + ` - url: https://github.com/${repository}/releases/download/v${version}/Sim-${version}-universal.dmg`, ' sha512: def', - `path: https://github.com/simstudioai/sim/releases/download/v${version}/Sim-${version}-universal-mac.zip`, + `path: https://github.com/${repository}/releases/download/v${version}/Sim-${version}-universal-mac.zip`, "releaseDate: '2026-07-23T00:00:00.000Z'", ].join('\n') } @@ -453,6 +453,26 @@ describe('initUpdater manual mode (no Developer ID signature)', () => { expect(shell.openExternal).toHaveBeenCalledTimes(2) }) + it('offers prerelease-repository assets as manual downloads', async () => { + const fetchManifest = vi.fn(async () => + manifest('9.9.9-dev.1', 'simstudioai/sim-desktop-releases') + ) + const { handle } = await createManualUpdater(fetchManifest) + + handle.check() + await vi.advanceTimersByTimeAsync(0) + expect(handle.getState()).toEqual({ + status: 'available', + version: '9.9.9-dev.1', + manual: true, + }) + + handle.check() + expect(shell.openExternal).toHaveBeenCalledWith( + 'https://github.com/simstudioai/sim-desktop-releases/releases/download/v9.9.9-dev.1/Sim-9.9.9-dev.1-universal.dmg' + ) + }) + it('refuses a manifest whose download urls are not http(s)', async () => { const hostile = [ 'version: 9.9.9', @@ -499,6 +519,19 @@ describe('initUpdater manual mode (no Developer ID signature)', () => { expect(shell.openExternal).not.toHaveBeenCalled() }) + it('refuses assets from other repositories on github.com', async () => { + const offRepository = manifest('9.9.9', 'simstudioai/not-desktop-releases') + const { handle } = await createManualUpdater(async () => offRepository) + + handle.check() + await vi.advanceTimersByTimeAsync(0) + + expect(handle.getState()).toMatchObject({ status: 'error', manual: true }) + handle.check() + handle.install() + expect(shell.openExternal).not.toHaveBeenCalled() + }) + it('skips an unusable url but still offers a safe one from the same manifest', async () => { const mixed = [ 'version: 9.9.9', diff --git a/apps/desktop/src/main/updater.ts b/apps/desktop/src/main/updater.ts index 31333eb1403..a6702d0090b 100644 --- a/apps/desktop/src/main/updater.ts +++ b/apps/desktop/src/main/updater.ts @@ -43,7 +43,10 @@ export function feedUrlForOrigin(origin: string): string | null { * host cannot get a bundle in front of the user's Download button. */ const RELEASE_ASSET_ORIGIN = 'https://github.com' -const RELEASE_ASSET_PATH = '/simstudioai/sim/releases/download/' +const RELEASE_ASSET_PATHS = [ + '/simstudioai/sim/releases/download/', + '/simstudioai/sim-desktop-releases/releases/download/', +] as const /** Whether a manifest url is one of our own release assets. */ function isReleaseAssetUrl(rawUrl: string): boolean { @@ -53,7 +56,10 @@ function isReleaseAssetUrl(rawUrl: string): boolean { // Compared on the parsed origin and the parsed pathname, never by prefix on // the raw string: `https://github.com.evil.example/…` must not pass, and // `URL` has already normalized away any `..` segments by this point. - return url.origin === RELEASE_ASSET_ORIGIN && url.pathname.startsWith(RELEASE_ASSET_PATH) + return ( + url.origin === RELEASE_ASSET_ORIGIN && + RELEASE_ASSET_PATHS.some((path) => url.pathname.startsWith(path)) + ) } catch { return false } diff --git a/apps/desktop/src/test/electron-mock.ts b/apps/desktop/src/test/electron-mock.ts index fafa34ee4ac..476e391d9bb 100644 --- a/apps/desktop/src/test/electron-mock.ts +++ b/apps/desktop/src/test/electron-mock.ts @@ -11,6 +11,8 @@ import { vi } from 'vitest' export const app = { name: 'Sim', isPackaged: false, + userAgentFallback: + 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Sim/1.0.0 Chrome/140.0.7339.207 Electron/43.1.1 Safari/537.36', getVersion: vi.fn(() => '1.0.0'), getName: vi.fn(() => 'Sim'), setName: vi.fn(), @@ -152,6 +154,7 @@ function createWebContentsMock() { findInPage: vi.fn(() => 1), stopFindInPage: vi.fn(), setBackgroundThrottling: vi.fn(), + setUserAgent: vi.fn(), setIgnoreMenuShortcuts: vi.fn(), getZoomFactor: vi.fn(() => 1), setZoomFactor: vi.fn(), @@ -185,6 +188,7 @@ function createWebContentsMock() { session: { setPermissionRequestHandler: vi.fn(), setPermissionCheckHandler: vi.fn(), + setUserAgent: vi.fn(), webRequest: { onBeforeRequest: vi.fn() }, on: vi.fn(), }, diff --git a/apps/docs/app/[lang]/[[...slug]]/page.tsx b/apps/docs/app/[lang]/[[...slug]]/page.tsx index 36c31389949..a4ffafdad87 100644 --- a/apps/docs/app/[lang]/[[...slug]]/page.tsx +++ b/apps/docs/app/[lang]/[[...slug]]/page.tsx @@ -16,7 +16,7 @@ import { CodeBlock } from '@/components/ui/code-block' import { Heading } from '@/components/ui/heading' import { ResponseSection } from '@/components/ui/response-section' import { i18n } from '@/lib/i18n' -import { getApiSpecContent, openapi } from '@/lib/openapi' +import { getApiSpecContent, getAuthenticatedCodeSamples, openapi } from '@/lib/openapi' import { type PageData, source } from '@/lib/source' import { DOCS_BASE_URL } from '@/lib/urls' @@ -71,6 +71,7 @@ function stripLocalePrefix(url: string, lang: string): string { const APIPage = createAPIPage(openapi, { playground: { enabled: false }, + generateCodeSamples: getAuthenticatedCodeSamples, client: { operation: { APIExampleSelector }, }, diff --git a/apps/docs/app/global.css b/apps/docs/app/global.css index 6373ec36d75..6414f5797e3 100644 --- a/apps/docs/app/global.css +++ b/apps/docs/app/global.css @@ -42,6 +42,13 @@ body { --text-small: 13px; --text-base: 15px; --text-md: 16px; + + /* Code-token size for the API reference — a deliberate sixth step, between + --text-caption and --text-small, because the mono face reads small at 12px. */ + --text-code: 0.78125rem; + + --font-mono-stack: var(--font-geist-mono), ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, + "Liberation Mono", "Courier New", monospace; } /* Pure white light mode background */ @@ -134,7 +141,6 @@ body { --selection-dark: #264f78; --highlight-search-active: #f6ad55; --scrollbar-thumb-color: #c0c0c0; - --scrollbar-thumb-hover-color: #a8a8a8; --shadow-subtle: 0 2px 4px 0 rgba(0, 0, 0, 0.08); --shadow-medium: 0 4px 12px rgba(0, 0, 0, 0.1); --shadow-overlay: 0 10px 30px rgba(0, 0, 0, 0.11); @@ -216,34 +222,18 @@ body { --code-line-number: #a8a8a8; --selection-bg: #264f78; --scrollbar-thumb-color: #5a5a5a; - --scrollbar-thumb-hover-color: #6a6a6a; --shadow-overlay: 0 10px 30px rgba(0, 0, 0, 0.3); } -/* Scrollbars — platform thumb tokens, transparent track */ +/* Scrollbars — platform thumb tokens, transparent track. A non-auto + `scrollbar-width`/`scrollbar-color` makes Chromium ignore every + `::-webkit-scrollbar*` rule on the element, so no webkit block here. Hover + shading is not expressible through the standard properties. */ * { scrollbar-width: thin; scrollbar-color: var(--scrollbar-thumb-color) transparent; } -*::-webkit-scrollbar { - width: 8px; - height: 8px; -} - -*::-webkit-scrollbar-track { - background: transparent; -} - -*::-webkit-scrollbar-thumb { - background-color: var(--scrollbar-thumb-color); - border-radius: 9999px; -} - -*::-webkit-scrollbar-thumb:hover { - background-color: var(--scrollbar-thumb-hover-color); -} - /* Font family utilities */ .font-sans { font-family: var(--font-geist-sans), ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, @@ -251,8 +241,7 @@ body { } .font-mono { - font-family: var(--font-geist-mono), ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, - "Liberation Mono", "Courier New", monospace; + font-family: var(--font-mono-stack); } /* Platform UI font — Season Sans, used by the chip chrome to match the main app */ @@ -672,8 +661,7 @@ aside[data-sidebar], code, pre, pre code { - font-family: var(--font-geist-mono), ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, - "Liberation Mono", "Courier New", monospace; + font-family: var(--font-mono-stack); } /* Inline code — neutral colors aligned with sim design system */ @@ -912,16 +900,18 @@ video { display: none !important; } -/* Ensure API reference pages use the same font as the rest of the docs */ +/* Ensure API reference pages use the same font as the rest of the docs. + `.font-mono` is excluded: this selector (id + element) outranks the + `.font-mono` class rule, so without it every code identifier renders sans. */ #nd-page:has(.api-page-header), #nd-page:has(.api-page-header) h2, #nd-page:has(.api-page-header) h3, #nd-page:has(.api-page-header) h4, -#nd-page:has(.api-page-header) p, -#nd-page:has(.api-page-header) span, -#nd-page:has(.api-page-header) div, -#nd-page:has(.api-page-header) label, -#nd-page:has(.api-page-header) button { +#nd-page:has(.api-page-header) p:not(.font-mono), +#nd-page:has(.api-page-header) span:not(.font-mono), +#nd-page:has(.api-page-header) div:not(.font-mono), +#nd-page:has(.api-page-header) label:not(.font-mono), +#nd-page:has(.api-page-header) button:not(.font-mono) { font-family: var(--font-geist-sans), ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif; } @@ -1162,23 +1152,45 @@ div.flex.flex-row.items-start.bg-fd-secondary.border.rounded-lg.text-xs { position: relative; } +/* API-reference metadata face — the status trigger, the content-type label, the + `required` / `header` markers, and the status-code tabs. Defined once; each + consumer below adds only its own colour, content, and order. The `code.text-xs` + label further down needs `!important` to beat fumadocs and stays separate. */ +#nd-page:has(.api-page-header) button.response-section-dropdown-trigger, +.response-section-dropdown-trigger, +#nd-page:has(.api-page-header) span.response-section-content-type, +.response-section-content-type, +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose:has(span.text-red-400)::after, +#nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose::before, +#nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose::after, +#nd-page:has(.api-page-header) .flex.gap-3\.5.overflow-x-auto.not-prose > button { + font-size: var(--text-code); + line-height: 1.25rem; + font-weight: 400; + font-family: var(--font-mono-stack); +} + +/* Status-code trigger — matches the content-type label beside it. */ +#nd-page:has(.api-page-header) button.response-section-dropdown-trigger, .response-section-dropdown-trigger { display: flex; align-items: center; gap: 0.25rem; - padding: 0.125rem 0.25rem; - font-size: 0.875rem; - font-weight: 500; - color: var(--color-fd-muted-foreground); + height: 1.25rem; + padding: 0 0.25rem; + color: var(--text-secondary); background: none; border: none; cursor: pointer; border-radius: 0.375rem; transition: color 0.15s; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; } +/* Carries the same id-qualified prefix as the base rule above; without it the + base rule outranks this one and the trigger never changes colour on hover. */ +#nd-page:has(.api-page-header) button.response-section-dropdown-trigger:hover, .response-section-dropdown-trigger:hover { - color: var(--color-fd-foreground); + color: var(--text-primary); } .response-section-chevron { @@ -1226,7 +1238,7 @@ div.flex.flex-row.items-start.bg-fd-secondary.border.rounded-lg.text-xs { color: var(--text-primary); } .response-section-dropdown-item-selected { - color: var(--color-fd-foreground); + color: var(--text-primary); } .response-section-check { @@ -1234,10 +1246,15 @@ div.flex.flex-row.items-start.bg-fd-secondary.border.rounded-lg.text-xs { height: 0.875rem; } +/* Content-type label. The Response header renders this class; the Request Body + header renders a fumadocs `code.text-xs`. Keep the two in sync — the same + string at different weights reads as one being lighter than the other. */ +#nd-page:has(.api-page-header) span.response-section-content-type, .response-section-content-type { - font-size: 0.875rem; - color: var(--color-fd-muted-foreground); - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; + color: var(--text-secondary); + background: none; + border: none; + padding: 0; } /* Response schema container — remove border to match Path Parameters style */ @@ -1262,25 +1279,80 @@ div.flex.flex-row.items-start.bg-fd-secondary.border.rounded-lg.text-xs { order: 1; } -/* Type badge — order 2, grey pill */ +/* Type token — order 2. Covers every shape the slot takes: scalar span, union + wrapper, schema-reference button, and the auth row's `::after` label. Reuses + the docs inline-code recipe, so a type reads as code wherever it appears; the + explicit 20px height keeps a union level with a scalar, which its nested + links would otherwise push to 26px. */ #nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose - > span.text-sm.font-mono.text-fd-muted-foreground { + > span.text-sm.font-mono.text-fd-muted-foreground, +#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > button, +#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > span:has(> button), +#nd-page:has(.api-page-header) + div.my-4 + > .flex.flex-wrap.items-center.gap-3.not-prose + > span.text-sm.font-mono.text-fd-muted-foreground::after { order: 2; - background-color: var(--surface-5); - color: var(--text-secondary); - padding: 0.1875rem 0.5rem; + display: inline-flex; + align-items: center; + height: 1.25rem; + /* No gap: an `array` slot holds its brackets as bare text nodes, which + become anonymous flex items, so any gap here would prise `array<` and `>` + away from the type they wrap. The union separator spaces itself instead. */ + gap: 0; + background-color: var(--surface-4); + border: 1px solid var(--border-1); + color: var(--text-body); + padding: 0 0.3125rem; border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; + font-size: var(--text-code); + line-height: 1; + font-weight: 400; + font-family: var(--font-mono-stack); +} + +/* Everything inside a type token inherits the token's own face, size, and ink. + Applied to every descendant, not just the links: a union's `|` separator is a + classless `span`, so the page-wide `span:not(.font-mono)` rule assigned it the + body sans face and one chip rendered in two faces. Anything fumadocs nests in + here later is covered by the same reset. + Underline is deferred to hover so links don't read heavier than a plain scalar + in the same box. The button that *is* the slot needs its own rule below: it + cannot `inherit`, which would pull the row's 14px sans back in. */ +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose + > span.text-sm.font-mono.text-fd-muted-foreground + * { + text-decoration: none; + color: inherit; + font-size: inherit; + font-family: inherit; } -html.dark - #nd-page:has(.api-page-header) +#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose - > span.text-sm.font-mono.text-fd-muted-foreground { - background-color: var(--surface-4); + > button.text-sm.font-mono.text-fd-muted-foreground { + text-decoration: none; +} +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose + > span.text-sm.font-mono.text-fd-muted-foreground + :is(a, button):hover, +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose + > button.text-sm.font-mono.text-fd-muted-foreground:hover { + text-decoration: underline; + text-underline-offset: 2px; +} + +/* Union separator — dimmed one step, no further: `string | null` started + reading as `string null` on the chip fill. Own margin; the slot has no gap. */ +#nd-page:has(.api-page-header) + .flex.flex-wrap.items-center.gap-3.not-prose + > span.text-sm.font-mono.text-fd-muted-foreground + > span { + margin: 0 0.375rem; + color: var(--text-muted); } /* Hide the "*" inside the name span — we'll add "required" as a ::after on the flex row */ @@ -1288,21 +1360,15 @@ html.dark display: none; } -/* Required badge — order 3, red pill */ +/* Required marker — order 3. Error text colour but no fill: eight required + params on one page should not read as eight alarms. */ #nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose:has(span.text-red-400)::after { content: "required"; order: 3; display: inline-flex; align-items: center; - background-color: var(--badge-error-bg); color: var(--badge-error-text); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; } /* Optional "?" indicator — hide it */ #nd-page:has(.api-page-header) @@ -1326,79 +1392,52 @@ html.dark > span.font-medium.font-mono.text-fd-primary { order: 1; } +/* Auth rows collapse the real `` text to zero and draw the chip in the + `::after` below, so this span is a bare wrapper: it must drop the type-token + box it matches, or the chip renders inside a second, empty bordered box. */ #nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose > span.text-sm.font-mono.text-fd-muted-foreground { order: 2; font-size: 0; - padding: 0 !important; - background: none !important; + padding: 0; + background: none; + border: none; + height: auto; line-height: 0; } +/* Only the label — the box comes from the shared type-token rule above, which + this pseudo-element is a member of. */ #nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose > span.text-sm.font-mono.text-fd-muted-foreground::after { content: "string"; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; - background-color: var(--surface-5); - color: var(--text-secondary); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - display: inline-flex; - align-items: center; -} -html.dark - #nd-page:has(.api-page-header) - div.my-4 - > .flex.flex-wrap.items-center.gap-3.not-prose - > span.text-sm.font-mono.text-fd-muted-foreground::after { - background-color: var(--surface-4); } -/* "header" badge via ::before on the auth flex row */ +/* "header" location via ::before on the auth flex row — uncontained metadata, + matching the `required` marker rather than the type token. */ #nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose::before { content: "header"; order: 3; display: inline-flex; align-items: center; - background-color: var(--surface-5); color: var(--text-secondary); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; -} -html.dark - #nd-page:has(.api-page-header) - div.my-4 - > .flex.flex-wrap.items-center.gap-3.not-prose::before { - background-color: var(--surface-4); } -/* "required" badge via ::after on the auth flex row — red pill */ +/* "required" marker via ::after on the auth flex row */ #nd-page:has(.api-page-header) div.my-4 > .flex.flex-wrap.items-center.gap-3.not-prose::after { content: "required"; order: 4; display: inline-flex; align-items: center; - background-color: var(--badge-error-bg); color: var(--badge-error-text); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; } -/* Hide "In: header" text below auth property — redundant with the header badge */ -#nd-page:has(.api-page-header) div.my-4 .prose-no-margin p:has(> code) { +/* Hide the trailing "In: header" line — redundant with the header marker. + Matched by position, not shape: descriptions contain a `code` too (status + codes), so a bare `p:has(> code)` also hid the API-key description. */ +#nd-page:has(.api-page-header) div.my-4 .prose-no-margin > p:last-child:has(> code) { display: none !important; } @@ -1425,36 +1464,18 @@ html.dark border-color: var(--surface-active); } -/* Body/Callback section "application/json" label — remove inline code styling */ +/* Body/Callback "application/json" label — strip inline-code chrome and keep in + sync with `.response-section-content-type`; same string, two headers. */ #nd-page:has(.api-page-header) .flex.gap-2.items-center.justify-between p.not-prose code.text-xs, #nd-page:has(.api-page-header) .flex.justify-between.gap-2.items-end p.not-prose code.text-xs { background: none !important; border: none !important; padding: 0 !important; - color: var(--color-fd-muted-foreground) !important; - font-size: 0.875rem !important; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif !important; -} - -/* Object/array type triggers in property rows — order 2 + badge chip styling */ -#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > button, -#nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > span:has(> button) { - order: 2; - background-color: var(--surface-5); - color: var(--text-secondary); - padding: 0.1875rem 0.5rem; - border-radius: 0.375rem; - font-size: var(--text-xs); - line-height: 1.125rem; - font-weight: 500; - font-family: var(--font-geist-sans), ui-sans-serif, system-ui, sans-serif; -} -html.dark #nd-page:has(.api-page-header) .flex.flex-wrap.items-center.gap-3.not-prose > button, -html.dark - #nd-page:has(.api-page-header) - .flex.flex-wrap.items-center.gap-3.not-prose - > span:has(> button) { - background-color: var(--surface-4); + color: var(--text-secondary) !important; + font-size: var(--text-code) !important; + line-height: 1.25rem !important; + font-weight: 400 !important; + font-family: var(--font-mono-stack) !important; } /* Section headings (Authorization, Path Parameters, etc.) — consistent top spacing */ @@ -1463,15 +1484,15 @@ html.dark margin-bottom: 0.25rem !important; } -/* Code examples in right column — wrap long lines instead of horizontal scroll */ -#nd-page:has(.api-page-header) pre { - white-space: pre-wrap !important; - word-break: break-all !important; -} -#nd-page:has(.api-page-header) pre code { - width: 100% !important; - word-break: break-all !important; - overflow-wrap: break-word !important; +/* Example-panel code overflows rather than wraps: a wrapped line restarts at + column zero and misreports the JSON nesting depth. */ + +/* fumadocs' own lucide glyphs (heading anchor, copy button) ship at stroke-width + 2 while emcn strokes at 1.55, so they read heavier than everything near them. + Layout-wide on purpose: one icon weight across the docs. Retired once + createAPIPage is given renderHeading/renderCodeBlock. */ +#nd-docs-layout svg[class*="lucide"] { + stroke-width: 1.55; } /* Callout/alert — transparent background, no shadow, hide colored bar, add padding */ @@ -1497,7 +1518,7 @@ div.not-prose.rounded-md.border.bg-fd-card.p-2 { div.rounded-xl.border.bg-fd-card.shadow-md:has(> [role="none"]) > svg { fill: none !important; color: var(--color-fd-foreground) !important; - stroke-width: 1.75 !important; + stroke-width: 1.55 !important; flex-shrink: 0; width: 1rem !important; height: 1rem !important; diff --git a/apps/docs/components/icons.tsx b/apps/docs/components/icons.tsx index 447d38bb28f..ad17849bd0a 100644 --- a/apps/docs/components/icons.tsx +++ b/apps/docs/components/icons.tsx @@ -1518,12 +1518,12 @@ export function InputIcon(props: SVGProps) { export function StartIcon(props: SVGProps) { return ( ) { ) } +export function WindchillIcon(props: SVGProps) { + return ( + + + + + + + + + + + + + ) +} + export function MintlifyIcon(props: SVGProps) { return ( @@ -8599,6 +8625,17 @@ export function NewRelicIcon(props: SVGProps) { ) } +export function NetSuiteIcon(props: SVGProps) { + return ( + + + + ) +} + export function WizaIcon(props: SVGProps) { return ( @@ -9009,6 +9046,23 @@ export function LogfireIcon(props: SVGProps) { ) } +export function LogRocketIcon(props: SVGProps) { + return ( + + + + + ) +} + export function SmartleadIcon(props: SVGProps) { return ( = { datadog: DatadogIcon, datagma: DatagmaIcon, daytona: DaytonaIcon, - deployments: SimDeploymentsIcon, + deployments: Rocket, devin: DevinIcon, discord: DiscordIcon, docusign: DocuSignIcon, @@ -413,6 +415,7 @@ export const blockTypeToIconMap: Record = { linkup: LinkupIcon, linq: LinqIcon, logfire: LogfireIcon, + logrocket: LogRocketIcon, logs: Library, logs_v2: Library, loops: LoopsIcon, @@ -438,6 +441,7 @@ export const blockTypeToIconMap: Record = { mongodb: MongoDBIcon, mysql: MySQLIcon, neo4j: Neo4jIcon, + netsuite: NetSuiteIcon, neverbounce: NeverBounceIcon, new_relic: NewRelicIcon, notion: NotionIcon, @@ -538,6 +542,7 @@ export const blockTypeToIconMap: Record = { webflow: WebflowIcon, whatsapp: WhatsAppIcon, wikipedia: WikipediaIcon, + windchill: WindchillIcon, wiza: WizaIcon, wordpress: WordpressIcon, workday: WorkdayIcon, diff --git a/apps/docs/content/docs/de/api-reference/getting-started.mdx b/apps/docs/content/docs/de/api-reference/getting-started.mdx index 55c4503f38b..4f5ae08008f 100644 --- a/apps/docs/content/docs/de/api-reference/getting-started.mdx +++ b/apps/docs/content/docs/de/api-reference/getting-started.mdx @@ -171,6 +171,24 @@ The API uses standard HTTP status codes. v2 errors include a stable code and hum | `404` | Resource not found | Verify the ID exists and belongs to your workspace | | `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header | +### Unrecognized fields are rejected + +Every v2 endpoint validates the request against its published schema — path parameters, query string, and body — and answers `400` for any field it does not declare. A misspelled parameter is an error rather than a silent no-op, so `?limt=20` fails instead of quietly returning an unbounded list. + +This holds for endpoints that declare no query parameters at all. Do not append tracking tags, cache busters, or other extra parameters to a v2 URL; send only what the endpoint documents. + +```json +{ + "error": { + "code": "BAD_REQUEST", + "message": "Invalid request", + "details": [ + { "code": "unrecognized_keys", "keys": ["limt"], "path": [], "message": "Unrecognized key: \"limt\"" } + ] + } +} +``` + Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage. diff --git a/apps/docs/content/docs/en/api-reference/(generated)/workflows/meta.json b/apps/docs/content/docs/en/api-reference/(generated)/workflows/meta.json index fb32a86e414..1017e55e280 100644 --- a/apps/docs/content/docs/en/api-reference/(generated)/workflows/meta.json +++ b/apps/docs/content/docs/en/api-reference/(generated)/workflows/meta.json @@ -9,6 +9,7 @@ "getWorkflowVersionV2", "exportWorkflow", "importWorkflow", + "getWorkflowDeployment", "deployWorkflow", "undeployWorkflow", "rollbackWorkflow", diff --git a/apps/docs/content/docs/en/api-reference/getting-started.mdx b/apps/docs/content/docs/en/api-reference/getting-started.mdx index 8136cec9df3..73b72490885 100644 --- a/apps/docs/content/docs/en/api-reference/getting-started.mdx +++ b/apps/docs/content/docs/en/api-reference/getting-started.mdx @@ -171,6 +171,24 @@ The API uses standard HTTP status codes. v2 errors include a stable code and hum | `404` | Resource not found | Verify the ID exists and belongs to your workspace | | `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header | +### Unrecognized fields are rejected + +Every v2 endpoint validates the request against its published schema — path parameters, query string, and body — and answers `400` for any field it does not declare. A misspelled parameter is an error rather than a silent no-op, so `?limt=20` fails instead of quietly returning an unbounded list. + +This holds for endpoints that declare no query parameters at all. Do not append tracking tags, cache busters, or other extra parameters to a v2 URL; send only what the endpoint documents. + +```json +{ + "error": { + "code": "BAD_REQUEST", + "message": "Invalid request", + "details": [ + { "code": "unrecognized_keys", "keys": ["limt"], "path": [], "message": "Unrecognized key: \"limt\"" } + ] + } +} +``` + Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage. diff --git a/apps/docs/content/docs/en/integrations/logrocket.mdx b/apps/docs/content/docs/en/integrations/logrocket.mdx new file mode 100644 index 00000000000..b986ace5ccb --- /dev/null +++ b/apps/docs/content/docs/en/integrations/logrocket.mdx @@ -0,0 +1,191 @@ +--- +title: LogRocket +description: Summarize sessions, manage users, and tag releases in LogRocket +--- + +import { BlockInfoCard } from "@/components/ui/block-info-card" + + + +{/* MANUAL-CONTENT-START:intro */} +[LogRocket](https://logrocket.com/) is a session replay and product analytics platform for frontend applications. It records what users actually did — clicks, navigation, network requests, console output, and application state — so teams can see the exact path that led to a bug, a drop-off, or a support ticket instead of asking the customer to reproduce it. + +With LogRocket, you can: + +- **Summarize sessions with AI**: Galileo Highlights reads a user's sessions and answers a plain-English question about them, returning a markdown summary plus per-session breakdowns. +- **Identify users**: Attach names, emails, and custom traits to a user so sessions can be segmented by plan, lifecycle stage, or account value. +- **Export session data**: Pull download URLs for JSON Lines session exports and load them into a warehouse. +- **Audit session access**: Page through the audit log to see who viewed which sessions and when. +- **Tag releases**: Register a deployed version so uploaded source maps decode its stack traces. + +Sim's LogRocket integration lets your agents run these operations programmatically. Use it to enrich a support ticket with a session summary, turn a vague bug report into reproduction steps, keep user traits in sync with your CRM, or register a release as part of a deploy workflow. + +Highlights requests are asynchronous: **Request Highlights** returns an ID and **Get Highlights** reports `PENDING` until generation finishes (typically one to three minutes), then `READY` or `FAILED`. Poll it rather than expecting the summary immediately. + +LogRocket has no REST API for querying sessions, issues, or metrics directly — that data is available through their [MCP server](https://docs.logrocket.com/docs/mcp) at `https://mcp.logrocket.com/mcp`, which you can add to Sim as an MCP server connection alongside this block. +{/* MANUAL-CONTENT-END */} + + +## Usage Instructions + +Integrate LogRocket into your workflow to request AI session highlights for a user, poll for the result, list exported session files, read the audit log, create or update user profiles, and register releases so source maps decode their stack traces. + + + +## Actions + +### LogRocket Request Highlights + +Start a Galileo session highlights job for a user. Returns a request ID to poll with LogRocket's Get Highlights operation. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `apiKey` | string | Yes | LogRocket API key | +| `orgId` | string | Yes | LogRocket organization ID | +| `appId` | string | Yes | LogRocket project \(app\) ID | +| `apiHost` | string | No | API host override for Private Cloud deployments | +| `userEmail` | string | No | Email of the user to summarize. Required if no user ID is given. | +| `userID` | string | No | ID of the user to summarize. Required if no email is given. | +| `question` | string | No | Question to focus the highlights on | +| `startMs` | string | No | Start of the session window, in milliseconds since epoch | +| `endMs` | string | No | End of the session window, in milliseconds since epoch | +| `webhookURL` | string | No | URL notified when the highlights job finishes | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `id` | string | Request ID used to retrieve the highlights result | + +### LogRocket Get Highlights + +Retrieve the result of a Galileo session highlights request. Status is PENDING until generation finishes, then READY or FAILED. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `apiKey` | string | Yes | LogRocket API key | +| `orgId` | string | Yes | LogRocket organization ID | +| `appId` | string | Yes | LogRocket project \(app\) ID | +| `apiHost` | string | No | API host override for Private Cloud deployments | +| `id` | string | Yes | Request ID returned by the Request Highlights operation | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | string | Job status: PENDING, READY, or FAILED | +| `requestID` | string | ID of the highlights request | +| `appID` | string | LogRocket project the request belongs to | +| `highlights` | string | Markdown summary across the matched sessions. Present when status is READY. | +| `sessions` | array | Per-session highlights | +| ↳ `recordingID` | string | LogRocket recording ID | +| ↳ `sessionID` | number | Session number within the recording | +| ↳ `highlights` | string | Highlights for this session | + +### LogRocket List Exported Sessions + +List exported session files from the LogRocket Data Export API. Returns download URLs for JSON Lines exports, oldest first, plus a cursor for the next page. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `apiKey` | string | Yes | LogRocket API key | +| `orgId` | string | Yes | LogRocket organization ID | +| `appId` | string | Yes | LogRocket project \(app\) ID | +| `apiHost` | string | No | API host override for Private Cloud deployments | +| `cursor` | string | No | Cursor from a previous page, used to fetch newer sessions | +| `limit` | string | No | Results per page. Defaults to 10, maximum 100. | +| `date` | string | No | Unix timestamp in milliseconds to start listing from | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `sessions` | array | Exported session files | +| ↳ `url` | string | Download URL for the JSON Lines export file | +| `cursor` | string | Opaque cursor for the next page of results | + +### LogRocket Get Audit Logs + +Export audit log entries from LogRocket, recording who viewed sessions and what actions were taken. Paginated with a cursor. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `apiKey` | string | Yes | LogRocket API key | +| `orgId` | string | Yes | LogRocket organization ID | +| `appId` | string | Yes | LogRocket project \(app\) ID | +| `apiHost` | string | No | API host override for Private Cloud deployments | +| `limit` | string | No | Number of audit log records to return | +| `cursor` | string | No | Cursor from a previous page, used to fetch the next page | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `logs` | array | Audit log entries | +| ↳ `time` | string | Formatted timestamp of the action | +| ↳ `createdDate` | string | ISO 8601 timestamp of the action | +| ↳ `user` | string | Email or system ID of the actor | +| ↳ `action` | string | Action taken, e.g. Viewed session | +| ↳ `description` | string | Action details, e.g. the session ID | +| `cursor` | string | Opaque cursor for the next page of results | +| `hasNext` | boolean | Whether more audit logs exist beyond this page | + +### LogRocket Identify User + +Create or update a LogRocket user profile with demographic, financial, and engagement traits. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `apiKey` | string | Yes | LogRocket API key | +| `orgId` | string | Yes | LogRocket organization ID | +| `appId` | string | Yes | LogRocket project \(app\) ID | +| `apiHost` | string | No | API host override for Private Cloud deployments | +| `userId` | string | Yes | ID of the user to create or update | +| `name` | string | No | Display name of the user. Maximum 1024 characters. | +| `email` | string | No | Email of the user. Maximum 1024 characters. | +| `timestamp` | string | No | Unix timestamp in milliseconds describing when the submitted data was true | +| `traits` | string | No | JSON object of custom traits. Each key and value is limited to 1024 characters and stored as a string. | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `userID` | string | ID of the created or updated user | +| `name` | string | Display name stored on the profile | +| `email` | string | Email stored on the profile | +| `traits` | json | Custom traits stored on the profile, with every value coerced to a string | + +### LogRocket Create Release + +Register a release version in LogRocket so uploaded source maps can decode stack traces for it. Fails with a conflict if the version already exists. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `apiKey` | string | Yes | LogRocket API key | +| `orgId` | string | Yes | LogRocket organization ID | +| `appId` | string | Yes | LogRocket project \(app\) ID | +| `apiHost` | string | No | API host override for Private Cloud deployments | +| `version` | string | Yes | Release version to register, e.g. 1.2.3 or a commit SHA | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `version` | string | Release version that was registered | + + diff --git a/apps/docs/content/docs/en/integrations/meta.json b/apps/docs/content/docs/en/integrations/meta.json index da4fbf044e5..868cf803849 100644 --- a/apps/docs/content/docs/en/integrations/meta.json +++ b/apps/docs/content/docs/en/integrations/meta.json @@ -143,6 +143,7 @@ "linkup", "linq", "logfire", + "logrocket", "logs", "loops", "luma", @@ -164,6 +165,8 @@ "mongodb", "mysql", "neo4j", + "netsuite", + "netsuite-service-account", "neverbounce", "new_relic", "notion", @@ -259,6 +262,7 @@ "webflow-service-account", "whatsapp", "wikipedia", + "windchill", "wiza", "wordpress", "workday", diff --git a/apps/docs/content/docs/en/integrations/netsuite-service-account.mdx b/apps/docs/content/docs/en/integrations/netsuite-service-account.mdx new file mode 100644 index 00000000000..fdf3b82011b --- /dev/null +++ b/apps/docs/content/docs/en/integrations/netsuite-service-account.mdx @@ -0,0 +1,86 @@ +--- +title: Oracle NetSuite Service Account +description: Configure certificate-based OAuth 2.0 client credentials once and reuse them across NetSuite blocks +--- + +import { Callout } from 'fumadocs-ui/components/callout' +import { Step, Steps } from 'fumadocs-ui/components/steps' +import { FAQ } from '@/components/ui/faq' + +Oracle NetSuite authenticates SuiteTalk machine-to-machine clients with a signed JWT and a certificate mapping. Sim stores the SuiteTalk URL, Client ID, Certificate ID, and private key as one encrypted service-account credential. Every NetSuite block stores only that credential's ID; Sim signs the assertion and injects the short-lived access token on the server. + +## Prerequisites + +- A dedicated NetSuite integration role with **REST Web Services** and **Log in using OAuth 2.0 Access Tokens**, plus the record and SuiteAnalytics permissions your workflows require. +- An integration record with **Client Credentials (Machine to Machine) Grant** and the **REST Web Services** scope enabled. +- A 3072- or 4096-bit RSA key pair, or a P-256, P-384, or P-521 EC key pair, and a public certificate generated through your organization's certificate process. +- Access to **OAuth 2.0 Client Credentials (M2M) Setup** and **Company URLs** in the target NetSuite environment. + + +Create and map credentials separately in production, sandbox, and Release Preview. A sandbox refresh removes its OAuth 2.0 client-credential mappings, and each environment has a different authoritative SuiteTalk URL. + + +## Configure NetSuite + + + + In **Setup → Company → Enable Features**, enable **REST Web Services** and **OAuth 2.0**. Enable **SuiteAnalytics Workbook** if workflows will use datasets. + + + Create a dedicated integration role and grant only the record, transaction, subsidiary, and analytics permissions the workflows need. Avoid using Administrator. + + + Under **Setup → Integration → Manage Integrations**, create or edit an integration, enable the machine-to-machine client-credentials grant and REST Web Services scope, then save its **Client ID**. + + + Upload only the public certificate under **OAuth 2.0 Client Credentials (M2M) Setup**. Map it to the integration, entity, and dedicated role, then save the generated **Certificate ID**. Keep the private key outside NetSuite. + + + Under **Setup → Company → Company Information → Company URLs**, copy the complete **SuiteTalk (SOAP and REST Web Services)** URL for this environment. + + + +Oracle documents the [role setup](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_157771510070.html), [integration record](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_157771733782.html), [certificate requirements](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/subsect_162755332391.html), and [client-credential mapping](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_162686838198.html). + +## Add the Credential to Sim + + + + Add an **Oracle NetSuite** block to a workflow and open the **NetSuite Account** dropdown. + + + Choose to add a credential, then enter the authoritative SuiteTalk URL, Client ID, Certificate ID, and PEM private key that matches the uploaded certificate. + + + Save the credential. Sim validates the URL and key policy, signs a client assertion, and performs a real token exchange before storing the encrypted credential. + + + +The private key is encrypted at rest and is never returned through the token endpoint or injected into a workflow tool. At execution time, Sim resolves the selected credential to a short-lived bearer token and the normalized SuiteTalk origin. + +## Use Pickers and Manual Values + +Selecting the credential enables these account-backed fields: + +| Field | Lists | Additional scope | +| --- | --- | --- | +| Record Type | Up to 1,000 record types visible in the metadata catalog | credential | +| Async Task | Up to 100 tasks belonging to a known batch job | job ID | + +Picker results reflect the selected role's permissions. Switch any picker to Advanced mode to type an identifier or reference an upstream output. Enter SuiteAnalytics dataset IDs manually after finding them with **List SuiteAnalytics Datasets**. Record IDs, job IDs, transform targets, actions, fields, forms, subresources, and relationship IDs also remain manual because NetSuite does not expose bounded universal listings that would make those choices complete and reliable. + +**Create Record** without `replace` returns HTTP 204 with no response body; with `replace`, it returns HTTP 201 and the created record object. Both responses expose NetSuite's validated `location`. The `replace` option applies to create and update, not upsert. + +## Rotate or Revoke + +To rotate a certificate, create and upload the replacement certificate and create its new NetSuite mapping. Then reconnect the existing Sim credential by re-entering all four required fields: SuiteTalk URL, Client ID, the new Certificate ID, and the replacement private key. Reconnecting changes the encrypted credential fingerprint, so later executions mint against the new material. + +After confirming workflows succeed, remove the old certificate mapping in NetSuite so the previous certificate can no longer mint tokens. Deleting a Sim credential removes its workflow bindings but does not revoke the corresponding NetSuite certificate mapping. + + diff --git a/apps/docs/content/docs/en/integrations/netsuite.mdx b/apps/docs/content/docs/en/integrations/netsuite.mdx new file mode 100644 index 00000000000..1a074d4d178 --- /dev/null +++ b/apps/docs/content/docs/en/integrations/netsuite.mdx @@ -0,0 +1,653 @@ +--- +title: Oracle NetSuite +description: Manage NetSuite records, queries, datasets, batches, and async jobs +--- + +import { BlockInfoCard } from "@/components/ui/block-info-card" + + + +{/* MANUAL-CONTENT-START:intro */} +[Oracle NetSuite](https://www.netsuite.com/) is a cloud ERP platform for financials, order management, inventory, procurement, CRM, and analytics. Sim connects through SuiteTalk REST Web Services using NetSuite's OAuth 2.0 client-credentials flow; it does not require a RESTlet or a user-interactive login. + +## Before you connect + +1. In **Setup → Company → Enable Features**, enable **REST Web Services** and **OAuth 2.0**. Enable **SuiteAnalytics Workbook** if you will list or execute datasets. +2. Use a dedicated integration role. Grant it **REST Web Services** and **Log in using OAuth 2.0 Access Tokens**, plus the record, transaction, and subsidiary permissions needed by your workflows. Dataset access also requires **SuiteAnalytics Workbook** permission and access to the selected datasets. Oracle recommends a purpose-built role instead of Administrator. See [Set Up OAuth 2.0 Roles](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_157771510070.html) and [Prerequisites and Setup for REST Web Services](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/article_5085602973.html). +3. Create or edit an integration record under **Setup → Integration → Manage Integrations**. Enable **Client Credentials (Machine to Machine) Grant** and the **REST Web Services** OAuth 2.0 scope, then save the **Client ID**. See [Create Integration Records for Applications to Use OAuth 2.0](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_157771733782.html). +4. Create either a 3072- or 4096-bit RSA key pair or a P-256, P-384, or P-521 EC key pair and certificate using your organization's certificate process. Sim signs RSA assertions with PS256 and selects ES256, ES384, or ES512 for the corresponding EC curve. Keep the PEM private key secure; upload only the public certificate to NetSuite, and plan renewal because NetSuite limits certificate validity to two years. See [OAuth 2.0 Client Credentials Certificate Conditions](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/subsect_162755332391.html). +5. Go to **Setup → Integration → Manage Authentication → OAuth 2.0 Client Credentials (M2M) Setup**. Create a mapping for the integration's entity, role, application, and public certificate, then save the generated **Certificate ID**. Oracle requires this mapping separately in production, sandbox, and Release Preview, and a sandbox refresh clears it. See [OAuth 2.0 Client Credentials Setup](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_162686838198.html). +6. In **Setup → Company → Company Information → Company URLs**, copy the **SuiteTalk (SOAP and REST Web Services)** URL for the current environment. Production, sandbox, and Release Preview environments each have their own authoritative URL. +7. Add a NetSuite block, open **NetSuite Account**, and create a reusable credential with the SuiteTalk URL, Client ID, Certificate ID, and matching private key. See [Oracle NetSuite service-account setup](/integrations/netsuite-service-account) for the complete setup and rotation workflow. + +## Usage notes + +- Record fields and supported actions vary by account, enabled features, custom records, forms, role, and permissions. Use **List Record Types** and **Get Record Metadata** before constructing create, update, upsert, action, or transform bodies. Sim intentionally accepts JSON for these dynamic record shapes instead of guessing a fixed schema. +- Select the stored NetSuite account once per block. Record Type and known-job Async Task fields use account-backed pickers; switch a field to Advanced mode to type or reference an identifier that is not present in the bounded picker result. Enter SuiteAnalytics dataset IDs manually after finding them with **List SuiteAnalytics Datasets**. +- **Create Record** without `replace` returns NetSuite's HTTP 204 response with no body; with `replace`, it returns HTTP 201 and the created record object. Both cases expose the validated `location` returned by NetSuite. The `replace` option applies to create and update, not upsert. +- Paged operations return one page only. The default limit is 100, the maximum is 1,000, and the offset must be a non-negative multiple of the limit. Sim never fetches later pages automatically. Requests must stay within NetSuite's first 100,000 results and first 1,000 pages. +- Homogeneous batch operations accept 1–100 records of one record type and always run asynchronously. NetSuite processes records in parallel, and individual tasks can fail independently; submission is not an all-or-none transaction. Preserve the returned `location` or `jobId`, use **Get Async Status** with **Job Status** until the job completes, choose **List Tasks** to collect task IDs, check each ID with **Task Status**, then use completed IDs with **Get Async Operation Result**. Canceling or timing out the local Sim request does not cancel a batch that NetSuite has already accepted. +- Sim gives the OAuth token exchange and each SuiteTalk request up to 30 seconds. A timed-out request fails locally, but a mutation that NetSuite already accepted may still finish remotely. +- Sim limits each materialized request body and successful SuiteTalk response to 16 MiB. Request JSON is also limited to 100 levels of nesting and 100,000 JSON values. Split work into smaller pages or batches when a request or response would exceed these ceilings, even if NetSuite would otherwise accept the payload. +- When attaching a contact with a role, provide either the role's internal ID or external ID, not both. File attachments do not use a contact role. +- **Attach/Detach**, homogeneous batch operations, **Get Record Form**, and **Get Select Options** require a NetSuite 2026.1-compatible account. Oracle introduced these SuiteTalk REST capabilities in [NetSuite 2026.1](https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_N3950559.html). +- **Get Governance Limits** returns data only for roles allowed by NetSuite; Oracle documents Administrator access for that operation. +- This integration does not include a trigger. SuiteTalk has no generic API for registering record-change webhooks; polling or customer-deployed SuiteScript requires a separate design. +{/* MANUAL-CONTENT-END */} + + +## Usage Instructions + +Connect a reusable Oracle NetSuite service-account credential to SuiteTalk REST Web Services. Read and write account-specific records, execute SuiteQL and SuiteAnalytics datasets, run asynchronous record batches, inspect metadata, and monitor async jobs. + + + +## Actions + +### NetSuite List/Search Records + +List one page of a NetSuite record collection, optionally filtered with a q expression. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `q` | string | No | NetSuite record collection filter expression | +| `limit` | number | No | Results to return in this page \(1-1000; default 100\) | +| `offset` | number | No | Zero-based result offset; must be divisible by limit and stay within the first 100,000 results and 1,000 pages | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | One documented NetSuite collection page | +| ↳ `links` | array | Oracle HATEOAS links for the response | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `items` | array | Matching record references in this page | +| ↳ `id` | string | NetSuite record ID | +| ↳ `links` | array | Oracle HATEOAS links for the record | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `count` | number | Number of items in this page | +| ↳ `hasMore` | boolean | Whether another page is available | +| ↳ `offset` | number | Offset of this page | +| ↳ `totalResults` | number | Total number of matching items | + +### NetSuite Get Record + +Retrieve one NetSuite record by internal or external ID. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | Yes | NetSuite internal ID or an external-ID reference beginning with eid: | +| `fields` | string | No | Comma-separated record fields to return | +| `expand` | string | No | Comma-separated resources to expand when supported by the record metadata | +| `expandSubResources` | boolean | No | Whether to expand sublists and subrecords in the response | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | NetSuite response body; record fields are account-specific and dynamic | + +### NetSuite Create Record + +Create a NetSuite record using the account-specific record metadata schema. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `body` | json | Yes | Record fields matching the account-specific NetSuite metadata schema | +| `replace` | string | No | Comma-separated sublists whose default lines should be replaced | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for standard HTTP 204 creation; replacement creation can return the documented HTTP 201 post-state object | +| `location` | string | Newly created record URL from the Location response header | + +### NetSuite Update Record + +Update fields on an existing NetSuite record with PATCH. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | Yes | NetSuite internal ID or an external-ID reference beginning with eid: | +| `body` | json | Yes | Record fields matching the account-specific NetSuite metadata schema | +| `replace` | string | No | Comma-separated sublists whose existing lines should be replaced | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 204 No Content response | +| `location` | string | Updated record URL from the Location response header | + +### NetSuite Upsert Record + +Create or update a NetSuite record by external ID with PUT. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `externalId` | string | Yes | External ID without the eid: prefix | +| `body` | json | Yes | Record fields matching the account-specific NetSuite metadata schema | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 204 No Content response | +| `location` | string | URL of the created or updated record, when NetSuite returns a Location header | + +### NetSuite Delete Record + +Delete one NetSuite record by internal or external ID. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | Yes | NetSuite internal ID or an external-ID reference beginning with eid: | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 204 No Content response | + +### NetSuite Get Subresource + +Retrieve a record sublist, subrecord, referenced record, or nested subresource. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | Yes | NetSuite internal ID or an external-ID reference beginning with eid: | +| `subresourcePath` | string | Yes | Slash-separated subresource path, such as item or item/1/inventoryDetail | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | NetSuite response body; record fields are account-specific and dynamic | + +### NetSuite Get Record Form + +Return a prepopulated create form, or an edit form when a record ID is supplied. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | No | Existing record ID; omit to request a create form | +| `body` | json | No | Record fields matching the account-specific NetSuite metadata schema | +| `fields` | string | No | Comma-separated record fields to return | +| `expand` | string | No | Comma-separated resources to expand when supported by the record metadata | +| `expandSubResources` | boolean | No | Whether to expand sublists and subrecords in the response | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | NetSuite response body; record fields are account-specific and dynamic | + +### NetSuite Get Select Options + +Retrieve valid select values for one or more fields on a new or existing record. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | No | Existing record ID; omit to evaluate options for a new record | +| `fields` | string | Yes | Comma-separated select field IDs | +| `q` | string | No | Optional select-option filter using CONTAIN, IS, or START_WITH | +| `body` | json | No | Record fields matching the account-specific NetSuite metadata schema | +| `limit` | number | No | Results to return in this page \(1-1000; default 100\) | +| `offset` | number | No | Zero-based result offset; must be divisible by limit and stay within the first 100,000 results and 1,000 pages | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Select-options response keyed by requested field ID; each dynamic field contains an _selectOptions object with links, items, count, offset, hasMore, and totalResults | +| ↳ `links` | array | Oracle HATEOAS links for the response | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | + +### NetSuite Attach Record or File + +Attach a contact or file to another NetSuite record. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | Yes | NetSuite internal ID or an external-ID reference beginning with eid: | +| `relatedType` | string | Yes | Related resource type: contact or file | +| `relatedId` | string | Yes | Internal ID, or external ID prefixed with eid:, of the contact or file | +| `roleId` | string | No | Optional contact role internal ID | +| `roleExternalId` | string | No | Optional contact role external ID | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 204 No Content response | + +### NetSuite Detach Record or File + +Detach a contact or file from another NetSuite record. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | Yes | NetSuite internal ID or an external-ID reference beginning with eid: | +| `relatedType` | string | Yes | Related resource type: contact or file | +| `relatedId` | string | Yes | Internal ID, or external ID prefixed with eid:, of the contact or file | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 204 No Content response | + +### NetSuite Execute Record Action + +Execute a supported NetSuite record action such as approve, reject, or confirm. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | Yes | NetSuite internal ID or an external-ID reference beginning with eid: | +| `action` | string | Yes | NetSuite record action ID without the @ prefix | +| `body` | json | No | Parameters accepted by the selected NetSuite record action | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Documented NetSuite record-action response | +| ↳ `result` | boolean | True when NetSuite completed the record action | + +### NetSuite Transform Record + +Transform a supported source record into another NetSuite record type. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `recordId` | string | Yes | NetSuite internal ID or an external-ID reference beginning with eid: | +| `targetRecordType` | string | Yes | Target record type supported by the source record metadata | +| `body` | json | No | Record fields matching the account-specific NetSuite metadata schema | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 204 No Content response | +| `location` | string | URL of the transformed record, when NetSuite returns a Location header | + +### NetSuite Batch Get Records + +Submit an asynchronous request to retrieve up to 100 records of one type. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `ids` | string | Yes | Up to 100 comma-separated internal IDs or eid: external-ID references | +| `fields` | string | No | Comma-separated record fields to return | +| `expand` | string | No | Comma-separated resources to expand when supported by the record metadata | +| `expandSubResources` | boolean | No | Whether to expand sublists and subrecords in the response | +| `idempotencyKey` | string | No | Optional unique idempotency key for retrying the asynchronous request | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 202 Accepted submission response | +| `location` | string | Asynchronous job URL from the Location response header | +| `jobId` | string | Asynchronous job ID parsed from the Location header | + +### NetSuite Batch Create Records + +Submit an asynchronous batch that creates up to 100 records of one type. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `items` | array | Yes | Array of 1-100 records matching the account-specific metadata schema | +| `idempotencyKey` | string | No | Optional unique idempotency key for retrying the batch | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 202 Accepted submission response | +| `location` | string | Asynchronous job URL from the Location response header | +| `jobId` | string | Asynchronous job ID parsed from the Location header | + +### NetSuite Batch Update Records + +Submit an asynchronous batch that updates up to 100 records of one type. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `items` | array | Yes | Array of 1-100 records; every item must include an internal or external ID | +| `idempotencyKey` | string | No | Optional unique idempotency key for retrying the batch | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 202 Accepted submission response | +| `location` | string | Asynchronous job URL from the Location response header | +| `jobId` | string | Asynchronous job ID parsed from the Location header | + +### NetSuite Batch Upsert Records + +Submit an asynchronous batch that creates or updates up to 100 records by external ID. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `items` | array | Yes | Array of 1-100 records; every item must include externalId | +| `idempotencyKey` | string | No | Optional unique idempotency key for retrying the batch | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 202 Accepted submission response | +| `location` | string | Asynchronous job URL from the Location response header | +| `jobId` | string | Asynchronous job ID parsed from the Location header | + +### NetSuite Batch Delete Records + +Submit an asynchronous request to delete up to 100 records of one type. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `ids` | string | Yes | Up to 100 comma-separated internal IDs or eid: external-ID references | +| `idempotencyKey` | string | No | Optional unique idempotency key for retrying the batch | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Empty for the documented HTTP 202 Accepted submission response | +| `location` | string | Asynchronous job URL from the Location response header | +| `jobId` | string | Asynchronous job ID parsed from the Location header | + +### NetSuite Execute SuiteQL + +Execute one page of a SuiteQL query through SuiteTalk REST web services. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `query` | string | Yes | SuiteQL SELECT query; use a complete unique ORDER BY when retrieving multiple pages | +| `limit` | number | No | Results to return in this page \(1-1000; default 100\) | +| `offset` | number | No | Zero-based result offset; must be divisible by limit and stay within the first 100,000 results and 1,000 pages | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | One documented NetSuite collection page | +| ↳ `links` | array | Oracle HATEOAS links for the response | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `items` | array | Items in this page; item fields depend on the record, query, or dataset | +| ↳ `count` | number | Number of items in this page | +| ↳ `hasMore` | boolean | Whether another page is available | +| ↳ `offset` | number | Offset of this page | +| ↳ `totalResults` | number | Total number of matching items | + +### NetSuite List SuiteAnalytics Datasets + +List one page of SuiteAnalytics Workbook datasets available to the authenticated role. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `limit` | number | No | Results to return in this page \(1-1000; default 100\) | +| `offset` | number | No | Zero-based result offset; must be divisible by limit and stay within the first 100,000 results and 1,000 pages | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | One documented NetSuite collection page | +| ↳ `links` | array | Oracle HATEOAS links for the response | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `items` | array | Items in this page; item fields depend on the record, query, or dataset | +| ↳ `count` | number | Number of items in this page | +| ↳ `hasMore` | boolean | Whether another page is available | +| ↳ `offset` | number | Offset of this page | +| ↳ `totalResults` | number | Total number of matching items | + +### NetSuite Execute SuiteAnalytics Dataset + +Execute one page of a standard or custom SuiteAnalytics Workbook dataset. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `datasetId` | string | Yes | SuiteAnalytics dataset script ID | +| `limit` | number | No | Results to return in this page \(1-1000; default 100\) | +| `offset` | number | No | Zero-based result offset; must be divisible by limit and stay within the first 100,000 results and 1,000 pages | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | One documented NetSuite collection page | +| ↳ `links` | array | Oracle HATEOAS links for the response | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `items` | array | Items in this page; item fields depend on the record, query, or dataset | +| ↳ `count` | number | Number of items in this page | +| ↳ `hasMore` | boolean | Whether another page is available | +| ↳ `offset` | number | Offset of this page | +| ↳ `totalResults` | number | Total number of matching items | + +### NetSuite List Record Types + +List record types exposed to the authenticated role by the REST metadata catalog. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | NetSuite REST metadata catalog | +| ↳ `links` | array | Oracle HATEOAS links for the response | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `items` | array | Record types exposed to the authenticated role | +| ↳ `name` | string | REST record type script ID | +| ↳ `links` | array | Oracle HATEOAS links for the response | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `mediaType` | string | Media type advertised for the linked metadata resource | + +### NetSuite Get Record Metadata + +Retrieve account-specific metadata for one NetSuite record type. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `recordType` | string | Yes | NetSuite REST record type script ID, such as customer or salesOrder | +| `format` | string | No | Metadata representation: default, openapi, or json_schema | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | NetSuite response body; record fields are account-specific and dynamic | + +### NetSuite Get Async Status + +Retrieve job status, list job tasks, or retrieve one task status. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `jobId` | string | Yes | Asynchronous job ID | +| `view` | string | No | Retrieve job status, list tasks for the job, or retrieve one task status | +| `taskId` | string | No | Task ID; required when view is task | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Documented NetSuite asynchronous job, task collection, or task status | +| ↳ `completed` | boolean | Whether processing has completed | +| ↳ `endTime` | string | Task completion time | +| ↳ `id` | string | Asynchronous job or task ID | +| ↳ `progress` | string | Current task progress state | +| ↳ `startTime` | string | Task start time | +| ↳ `count` | number | Number of task collection entries returned | +| ↳ `items` | array | Collection entries containing links to one or more asynchronous tasks | +| ↳ `links` | array | Links to individual tasks | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `links` | array | HATEOAS links for the job or task collection | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | +| ↳ `task` | object | Link container for the tasks belonging to this asynchronous job | +| ↳ `links` | array | Links to the job task collection | +| ↳ `rel` | string | Link relationship | +| ↳ `href` | string | Link target | + +### NetSuite Get Async Operation Result + +Retrieve the provider response for one task within a completed asynchronous job. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `jobId` | string | Yes | Asynchronous job ID | +| `taskId` | string | Yes | Task ID within the asynchronous job | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Result payload for the submitted asynchronous operation; record fields are account-specific and dynamic | + +### NetSuite Get Server Time + +Retrieve the current UTC time from the NetSuite server. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | NetSuite server time response | +| ↳ `serverTime` | string | Current NetSuite server time in UTC | + +### NetSuite Get Governance Limits + +Retrieve REST web-services concurrency limits for the NetSuite account and integration; NetSuite requires an Administrator role. + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `status` | number | HTTP status returned by NetSuite | +| `data` | json | Documented NetSuite governance limits | +| ↳ `accountConcurrencyLimit` | number | Account concurrency limit | +| ↳ `accountUnallocatedConcurrencyLimit` | number | Account concurrency not allocated to integrations | +| ↳ `integrationConcurrencyLimit` | number | Concurrency allocated to this integration | +| ↳ `integrationLimitType` | string | Limit assignment: integrationSpecific, accountLimit, or internal | diff --git a/apps/docs/content/docs/en/integrations/windchill.mdx b/apps/docs/content/docs/en/integrations/windchill.mdx new file mode 100644 index 00000000000..7f2b4dbfab1 --- /dev/null +++ b/apps/docs/content/docs/en/integrations/windchill.mdx @@ -0,0 +1,937 @@ +--- +title: Windchill +description: Manage documents, revisions, and content in PTC Windchill +--- + +import { BlockInfoCard } from "@/components/ui/block-info-card" + + + +{/* MANUAL-CONTENT-START:intro */} +[PTC Windchill](https://www.ptc.com/en/products/windchill) is the product lifecycle management system manufacturers use as the system of record for engineering data. Documents in Windchill are controlled objects: each one carries a number, a revision and iteration, a lifecycle state, folder placement, security labels, and a checkout status that decides who is allowed to change it right now. + +This integration talks to Windchill REST Services (WRS) 2.7 over OData — the query protocol Windchill exposes its data through — using a Basic-authenticated service account. Point it at a complete versioned service root — `https://your-host/Windchill/servlet/odata/v6` — and your agents can: + +- **Find and read documents**: list documents with an OData filter, sort order, field selection, page size, and a latest-version-only switch; fetch a single document by its object identifier (OID); and walk a document's structure through its usage links to see child documents with their versions and states. +- **Create and update**: create one document or a batch of them in a container and optional folder, and patch editable attributes on one or many documents. Name, Number, and Organization are rejected here and have their own operation, because Windchill changes those through a separate action and refuses it while a document is checked out. +- **Run the version and lifecycle cycle**: check documents out and back in with notes, undo a checkout, revise to the next revision, read the lifecycle states a document is actually allowed to move to, and transition it to one of them. +- **Move files**: download a document's primary content, or a specific attachment by its OID, into a Sim file — and upload files as primary content or attachments. Sim handles Windchill's CSRF token and its multi-step upload handshake for you. + +Bulk actions are atomic on Windchill's side: PTC documents that if the action fails for any object in the collection, the entire action is rolled back and nothing changes. + +Two limits are worth knowing before you build. Windchill identifies everything by OID (`OR:wt.doc.WTDocument:48796581`), so most operations need an OID you got from a list or get call rather than a document number. And this integration supports Basic authentication only — Windchill deployments fronted by OAuth are not currently supported. +{/* MANUAL-CONTENT-END */} + + +## Usage Instructions + +Integrate PTC Windchill REST Services 2.7 document management into your workflow using Basic authentication. Read and update document metadata, perform version and lifecycle actions, and transfer primary content and attachments. Windchill OAuth deployments are not currently supported. + + + +## Actions + +### Windchill List Documents + +List documents with an OData query, sorting, and pagination + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `select` | string | No | Comma-separated normalized document properties to return | +| `filter` | string | No | OData $filter expression | +| `orderBy` | string | No | OData $orderby expression | +| `top` | number | No | Maximum documents in the OData result set \($top\), from 1 to 2000 | +| `skip` | number | No | Documents to skip | +| `count` | boolean | No | Ask Windchill to include the total matching count | +| `latestVersion` | boolean | No | Return only the latest version of matching documents | +| `nextLink` | string | No | Verified @odata.nextLink from a previous list response | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `documents` | array | Windchill documents | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | +| `pageInfo` | object | OData pagination information | +| ↳ `count` | number | Number of items returned in this page | +| ↳ `totalCount` | number | Total matching items | +| ↳ `nextLink` | string | URL returned by Windchill for the next page | + +### Windchill Get Document + +Get a WT.Document by OID + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `select` | string | No | Comma-separated normalized document properties to return | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `document` | object | Windchill document | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Get Document Structure + +Retrieve recursive document usage links and their parent and child documents + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `structureDepth` | number | No | Document structure expansion depth, from 1 to 3 | +| `nextLink` | string | No | Verified @odata.nextLink from a previous structure response | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `structure` | array | Document usage links, including recursively expanded child links | +| ↳ `id` | string | Document usage link OID | +| ↳ `parent` | object | Parent document | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | +| ↳ `child` | object | Child document | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | +| ↳ `children` | array | Nested child usage links with the same recursive shape | +| `pageInfo` | object | OData pagination information | +| ↳ `count` | number | Number of items returned in this page | +| ↳ `totalCount` | number | Total matching items | +| ↳ `nextLink` | string | URL returned by Windchill for the next page | + +### Windchill Get Valid State Transitions + +Get lifecycle states a document can transition to from its current state + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `states` | array | Valid lifecycle transitions | +| ↳ `value` | string | Internal state value | +| ↳ `display` | string | Displayed state value | + +### Windchill Get Primary Content + +Get primary-content metadata for a document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `content` | object | Primary-content metadata | +| ↳ `id` | string | Content object identifier | +| ↳ `fileName` | string | Content file name | +| ↳ `description` | string | Content description | +| ↳ `format` | string | Windchill content format | +| ↳ `mimeType` | string | Content MIME type | +| ↳ `fileSize` | number | Content size in bytes | +| ↳ `contentType` | string | Windchill OData content entity type | +| ↳ `displayName` | string | Displayed content name | +| ↳ `urlLocation` | string | URL-data location | +| ↳ `externalLocation` | string | External-storage location | + +### Windchill List Attachments + +List attachment metadata for a document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `nextLink` | string | No | Verified @odata.nextLink from a previous attachment response | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `attachments` | array | Document attachments | +| ↳ `id` | string | Content object identifier | +| ↳ `fileName` | string | Content file name | +| ↳ `description` | string | Content description | +| ↳ `format` | string | Windchill content format | +| ↳ `mimeType` | string | Content MIME type | +| ↳ `fileSize` | number | Content size in bytes | +| ↳ `contentType` | string | Windchill OData content entity type | +| ↳ `displayName` | string | Displayed content name | +| ↳ `urlLocation` | string | URL-data location | +| ↳ `externalLocation` | string | External-storage location | +| `pageInfo` | object | OData pagination information | +| ↳ `count` | number | Number of items returned in this page | +| ↳ `totalCount` | number | Total matching items | +| ↳ `nextLink` | string | URL returned by Windchill for the next page | + +### Windchill Create Document + +Create one WT.Document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `name` | string | Yes | Document name | +| `containerOid` | string | Yes | Container OID in which to create the document | +| `number` | string | No | Optional document number when manual numbering is enabled | +| `title` | string | No | Document title | +| `description` | string | No | Document description | +| `folderOid` | string | No | Optional folder OID for the new document | +| `attributes` | json | No | Optional installed Windchill document attributes as a JSON object | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Create Documents + +Create several documents in one atomic Windchill request + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documents` | array | Yes | Document inputs as a JSON array; each item requires name and containerOid | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Update Document + +Update one document's editable attributes + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `attributes` | json | Yes | Editable attributes as a JSON object. Name, Number, and Organization require the Update Common Properties operation and are not supported here. | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Update Common Properties + +Update a document's Name, Number, and other common properties + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581. The document must not be checked out. | +| `commonProperties` | json | Yes | Common properties as a JSON object, for example \{"Name":"New name","Number":"NEW-001"\}. Enumerated properties take a value/display pair. | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Update Documents + +Update several documents' editable attributes in one atomic request + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documents` | array | Yes | Document updates as a JSON array; each item requires id and the editable attributes to set. Name, Number, and Organization are not supported. | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Delete Document + +Delete one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | + +### Windchill Delete Documents + +Delete multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | + +### Windchill Check Out Document + +Check out one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `checkOutNote` | string | No | Checkout note | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Check Out Documents + +Check out multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | +| `checkOutNote` | string | No | Checkout note | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Check In Document + +Check in one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `checkInNote` | string | No | Check-in note | +| `keepCheckedOut` | boolean | No | Keep the document checked out after checking it in | +| `checkOutNote` | string | No | Checkout note | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Check In Documents + +Check in multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | +| `checkInNote` | string | No | Check-in note | +| `keepCheckedOut` | boolean | No | Keep the document checked out after checking it in | +| `checkOutNote` | string | No | Checkout note | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Undo Check Out Document + +Undo checkout for one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Undo Check Out Documents + +Undo checkout for multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Revise Document + +Create a new revision of one document + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `versionId` | string | No | Optional target revision identifier when override-on-revise is enabled | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Revise Documents + +Create new revisions of multiple documents atomically + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOids` | array | Yes | WT.Document OIDs to process atomically | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Set Lifecycle State + +Transition a document to a valid lifecycle state + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `stateValue` | string | Yes | Internal value of the target lifecycle state | +| `stateDisplay` | string | Yes | Display value of the target lifecycle state | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `document` | object | Document returned by Windchill when the operation returns one | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Update Document Security Labels + +Update installed security-label attributes for one or more documents + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `securityLabelUpdates` | array | Yes | Array of document IDs and installed security-label values | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the operation | +| `documents` | array | Documents returned by Windchill when the operation returns them | +| ↳ `id` | string | Windchill object identifier | +| ↳ `name` | string | Document name | +| ↳ `number` | string | Document number | +| ↳ `title` | string | Document title | +| ↳ `description` | string | Document description | +| ↳ `state` | string | Internal life cycle state value | +| ↳ `stateDisplay` | string | Displayed life cycle state value | +| ↳ `versionId` | string | Version identifier | +| ↳ `revision` | string | Revision identifier | +| ↳ `version` | string | Version and iteration | +| ↳ `latest` | boolean | Whether this is the latest version | +| ↳ `checkoutState` | string | Checkout state | +| ↳ `folderName` | string | Folder name | +| ↳ `folderLocation` | string | Folder path | + +### Windchill Download Primary Content + +Download primary content into a canonical UserFile + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `fileName` | string | No | Optional downloaded file name override | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `file` | file | Downloaded content stored as a canonical UserFile | +| `fileName` | string | Downloaded file name | +| `mimeType` | string | Downloaded content MIME type | + +### Windchill Upload Primary Content + +Upload a primary-content file to a document that has none + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `primaryFile` | file | Yes | Primary content file to upload | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the upload | +| `uploadedFileNames` | array | Names of files accepted by Windchill | + +### Windchill Download Attachment + +Download a document attachment into a canonical UserFile + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `attachmentOid` | string | Yes | Windchill attachment content OID | +| `fileName` | string | No | Optional downloaded file name override | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `file` | file | Downloaded content stored as a canonical UserFile | +| `fileName` | string | Downloaded file name | +| `mimeType` | string | Downloaded content MIME type | + +### Windchill Upload Attachments + +Upload one or more files as document attachments + +#### Input + +| Parameter | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `baseUrl` | string | Yes | Complete WRS 2.7 versioned service root using Basic authentication, for example https://host/Windchill/servlet/odata/v6 | +| `username` | string | Yes | Windchill service-account username | +| `password` | string | Yes | Windchill service-account password | +| `documentOid` | string | Yes | WT.Document OID, for example OR:wt.doc.WTDocument:48796581 | +| `attachmentFiles` | file[] | Yes | Attachment files to upload | + +#### Output + +| Parameter | Type | Description | +| --------- | ---- | ----------- | +| `operation` | string | Windchill operation that was executed | +| `affectedIds` | array | Document identifiers affected by the upload | +| `uploadedFileNames` | array | Names of files accepted by Windchill | + + diff --git a/apps/docs/content/docs/en/platform/credentials.mdx b/apps/docs/content/docs/en/platform/credentials.mdx index 7b03b1a72d9..ef1425bd3c8 100644 --- a/apps/docs/content/docs/en/platform/credentials.mdx +++ b/apps/docs/content/docs/en/platform/credentials.mdx @@ -124,8 +124,16 @@ When a workspace secret and a personal secret share the same key name, the **wor When a workflow runs, secrets resolve in this order: -1. **Workspace secrets** are checked first -2. **Personal secrets** are used as a fallback — from the user who triggered the run (manual) or the workflow owner (automated runs via API, webhook, or schedule) +1. **Workspace secrets** are checked first, and always resolve against the identity running the workflow — the caller when one can be identified, otherwise the workspace's billing account. A run only sees the workspace secrets that identity is allowed to use. +2. **Personal secrets** are used as a fallback, from whichever identity is running: + +| Run started by | Personal secrets come from | +| --- | --- | +| Clicking Run, or a personal API key | The person running it | +| A workspace API key, schedule, or webhook | The workflow owner | +| A public API URL with no authentication | Nobody — personal secrets do not resolve | + +The workflow owner is the fallback only where nobody can be identified but somebody in the workspace set the trigger up, since those workflows are usually built against the owner's own keys. A public URL can be called by anyone, so it never borrows a person's keys at all — put every secret such a workflow needs in **Workspace**. ## Best Practices @@ -138,7 +146,7 @@ When a workflow runs, secrets resolve in this order: { question: "Are my secrets encrypted at rest?", answer: "Yes. Values saved under Secrets are encrypted before being stored in the database." }, { question: "Can a saved secret still appear in a workflow result?", answer: "Yes. Functional workflow data is not rewritten, so the raw value can still reach downstream blocks and tools and can appear in workflow execution responses, streams, or callbacks if your workflow deliberately returns or prints it. Log-facing views and read APIs receive a protected copy after a successful {{KEY}} resolution. Before content is sent to a model, exact values from the run's authorized secret catalog are replaced with placeholders, but encoded or otherwise transformed values remain outside that protection." }, { question: "What happens if both a workspace secret and a personal secret have the same key name?", answer: "Among secrets available to the execution actor, the workspace secret takes precedence and the personal secret is the fallback. An inaccessible workspace secret does not shadow an authorized personal value." }, - { question: "Who determines which personal secret is used for automated runs?", answer: "For manual runs, the personal secrets of the user who clicked Run are used as fallback. For automated runs triggered by API, webhook, or schedule, the personal secrets of the workflow owner are used instead." }, + { question: "Who determines which personal secret is used for automated runs?", answer: "Whoever is running it, when that can be identified. Clicking Run or calling with a personal API key uses that person's personal secrets. A workspace API key, schedule, or webhook has no identifiable caller, so it falls back to the workflow owner's — those triggers are set up inside the workspace and the workflow is usually built against the owner's own keys. A public API URL with no authentication can be called by anyone, so no personal secrets resolve at all — those workflows run on workspace secrets only." }, { question: "Can I import secrets from a .env file?", answer: "Yes. Paste .env-style content (KEY=VALUE format) into any key or value field and the secrets will be auto-populated. The parser supports export KEY=VALUE, quoted values, and inline comments." }, { question: "What happens if I delete a secret that is used in a workflow?", answer: "The workflow will fail at any block that references the deleted secret during execution because the value cannot be resolved. Update any references before deleting a secret." }, ]} /> diff --git a/apps/docs/content/docs/en/tables/using-in-workflows.mdx b/apps/docs/content/docs/en/tables/using-in-workflows.mdx index 7445a4c2f00..b690f3a6a70 100644 --- a/apps/docs/content/docs/en/tables/using-in-workflows.mdx +++ b/apps/docs/content/docs/en/tables/using-in-workflows.mdx @@ -125,7 +125,7 @@ After the run, the table holds the enriched rows. The next run queries them agai **Iterate row by row.** Wrap a Query → process → update cycle in a [Loop block](/workflows/blocks/loop) to handle one row at a time. This runs sequentially, slower than a batch update but useful when each row needs its own multi-step logic. Inside the loop the Agent reads the current row and an Update Row by ID writes its result. -**Paginate large reads.** Query Rows returns at most 1000 rows. When `totalCount` exceeds your **Limit**, increase **Offset** on each pass (0, then 100, then 200) to walk through the whole table, typically inside a Loop. +**Paginate large reads.** Query Rows returns at most 1000 rows, and a page can also end early once its rows reach the response size budget — so a page may come back shorter than your **Limit** even when more rows match. Advance **Offset** by the `rowCount` you actually received, not by the Limit you asked for, and keep going while `nextCursor` is set. Stop when `nextCursor` is null. Stepping by the Limit instead skips whatever a short page left behind. ## Inspecting reads and writes diff --git a/apps/docs/content/docs/en/workflows/blocks/agent.mdx b/apps/docs/content/docs/en/workflows/blocks/agent.mdx index d4ce50bcacc..1a6fb899444 100644 --- a/apps/docs/content/docs/en/workflows/blocks/agent.mdx +++ b/apps/docs/content/docs/en/workflows/blocks/agent.mdx @@ -115,6 +115,7 @@ Live tool-call chips stream for **OpenAI, Anthropic, Azure Anthropic, Google, Ve | Google | Summaries only | `gemini-3.6-flash`, `gemini-3.5-flash-lite`, `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3.1-flash-lite`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` | | Vertex AI | Summaries only | `vertex/gemini-3.5-flash`, `vertex/gemini-3.1-pro-preview`, `vertex/gemini-3.1-flash-lite`, `vertex/gemini-3-flash-preview`, `vertex/gemini-2.5-pro`, `vertex/gemini-2.5-flash`, `vertex/gemini-2.5-flash-lite` | | DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-v4-flash`, `deepseek-reasoner` | +| xAI | Full thinking deltas | `grok-4.6`, `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-0309` | | Groq | Full thinking deltas | `groq/openai/gpt-oss-120b`, `groq/openai/gpt-oss-20b`, `groq/openai/gpt-oss-safeguard-20b`, `groq/qwen/qwen3.6-27b` | | Meta | Not streamed | `muse-spark-1.1` | | Kimi | Full thinking deltas | `kimi-k2.6` | @@ -142,7 +143,7 @@ The Agent reads the message from Start with `` and returns a result { question: "What are the memory options for the Agent block?", answer: "Four modes: None (no memory, each run is independent), Conversation (full history keyed by a conversation ID), Sliding window by messages (the N most recent messages), and Sliding window by tokens (messages up to a token budget). Memory needs a conversation ID to persist across runs." }, { question: "What is the difference between the tool usage controls (Auto, Force, None)?", answer: "In Auto, the model decides when to call a tool based on context. In Force, the model must call the tool on every run. In None, the tool is hidden from the model and never sent, which disables it without removing it from the block." }, { question: "How does the Response Format work?", answer: "It enforces structured output by providing a JSON Schema. When set, the model's response is constrained to match the schema exactly, and each field is read directly by downstream blocks using . Without a response format, the agent returns its standard outputs: content, model, tokens, and toolCalls." }, - { question: "What does the Reasoning Effort / Thinking Level setting do?", answer: "They appear only for models that support extended reasoning. Reasoning Effort (OpenAI o-series and GPT-5 models) and Thinking Level (Anthropic Claude and Gemini models with thinking) control how much compute the model spends reasoning before responding. Higher levels produce more thorough answers but cost more tokens and take longer." }, + { question: "What does the Reasoning Effort / Thinking Level setting do?", answer: "They appear only for models that support extended reasoning. Reasoning Effort (OpenAI, Azure OpenAI, xAI Grok, DeepSeek, Groq, Meta, and Z.ai models that accept an effort level) and Thinking Level (Anthropic Claude and Gemini models with thinking) control how much compute the model spends reasoning before responding. Higher levels produce more thorough answers but cost more tokens and take longer." }, { question: "When should I turn on Prompt Caching?", answer: "Turn it on when the same agent runs repeatedly with a large, stable system prompt or tool set — cached input bills at a tenth of the normal input rate. Leave it off for one-off runs, because writing the cache costs 1.25x and nothing reads it back. The setting appears only for Anthropic Claude models; OpenAI and Gemini cache automatically with no setting and no write fee. Anthropic only caches a prefix of at least 1,024 tokens (2,048 on Haiku), and entries expire after five minutes of no use." }, { question: "How does max output tokens work with Anthropic models?", answer: "The Agent block uses each Anthropic model's full max output token limit by default (for example, 64,000 tokens). You can override this with the Max Output Tokens setting. For non-streaming requests that exceed the SDK's internal threshold, the provider automatically uses internal streaming to avoid timeouts." }, { question: "Can I use the Agent block with a custom or self-hosted model?", answer: "Yes. Use any Ollama or VLLM-compatible model by typing the model name directly into the model combobox, as long as it exposes a compatible API endpoint." }, diff --git a/apps/docs/content/docs/es/api-reference/getting-started.mdx b/apps/docs/content/docs/es/api-reference/getting-started.mdx index 8136cec9df3..73b72490885 100644 --- a/apps/docs/content/docs/es/api-reference/getting-started.mdx +++ b/apps/docs/content/docs/es/api-reference/getting-started.mdx @@ -171,6 +171,24 @@ The API uses standard HTTP status codes. v2 errors include a stable code and hum | `404` | Resource not found | Verify the ID exists and belongs to your workspace | | `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header | +### Unrecognized fields are rejected + +Every v2 endpoint validates the request against its published schema — path parameters, query string, and body — and answers `400` for any field it does not declare. A misspelled parameter is an error rather than a silent no-op, so `?limt=20` fails instead of quietly returning an unbounded list. + +This holds for endpoints that declare no query parameters at all. Do not append tracking tags, cache busters, or other extra parameters to a v2 URL; send only what the endpoint documents. + +```json +{ + "error": { + "code": "BAD_REQUEST", + "message": "Invalid request", + "details": [ + { "code": "unrecognized_keys", "keys": ["limt"], "path": [], "message": "Unrecognized key: \"limt\"" } + ] + } +} +``` + Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage. diff --git a/apps/docs/content/docs/fr/api-reference/getting-started.mdx b/apps/docs/content/docs/fr/api-reference/getting-started.mdx index 8136cec9df3..73b72490885 100644 --- a/apps/docs/content/docs/fr/api-reference/getting-started.mdx +++ b/apps/docs/content/docs/fr/api-reference/getting-started.mdx @@ -171,6 +171,24 @@ The API uses standard HTTP status codes. v2 errors include a stable code and hum | `404` | Resource not found | Verify the ID exists and belongs to your workspace | | `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header | +### Unrecognized fields are rejected + +Every v2 endpoint validates the request against its published schema — path parameters, query string, and body — and answers `400` for any field it does not declare. A misspelled parameter is an error rather than a silent no-op, so `?limt=20` fails instead of quietly returning an unbounded list. + +This holds for endpoints that declare no query parameters at all. Do not append tracking tags, cache busters, or other extra parameters to a v2 URL; send only what the endpoint documents. + +```json +{ + "error": { + "code": "BAD_REQUEST", + "message": "Invalid request", + "details": [ + { "code": "unrecognized_keys", "keys": ["limt"], "path": [], "message": "Unrecognized key: \"limt\"" } + ] + } +} +``` + Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage. diff --git a/apps/docs/content/docs/ja/api-reference/getting-started.mdx b/apps/docs/content/docs/ja/api-reference/getting-started.mdx index 8136cec9df3..73b72490885 100644 --- a/apps/docs/content/docs/ja/api-reference/getting-started.mdx +++ b/apps/docs/content/docs/ja/api-reference/getting-started.mdx @@ -171,6 +171,24 @@ The API uses standard HTTP status codes. v2 errors include a stable code and hum | `404` | Resource not found | Verify the ID exists and belongs to your workspace | | `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header | +### Unrecognized fields are rejected + +Every v2 endpoint validates the request against its published schema — path parameters, query string, and body — and answers `400` for any field it does not declare. A misspelled parameter is an error rather than a silent no-op, so `?limt=20` fails instead of quietly returning an unbounded list. + +This holds for endpoints that declare no query parameters at all. Do not append tracking tags, cache busters, or other extra parameters to a v2 URL; send only what the endpoint documents. + +```json +{ + "error": { + "code": "BAD_REQUEST", + "message": "Invalid request", + "details": [ + { "code": "unrecognized_keys", "keys": ["limt"], "path": [], "message": "Unrecognized key: \"limt\"" } + ] + } +} +``` + Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage. diff --git a/apps/docs/content/docs/zh/api-reference/getting-started.mdx b/apps/docs/content/docs/zh/api-reference/getting-started.mdx index 8136cec9df3..73b72490885 100644 --- a/apps/docs/content/docs/zh/api-reference/getting-started.mdx +++ b/apps/docs/content/docs/zh/api-reference/getting-started.mdx @@ -171,6 +171,24 @@ The API uses standard HTTP status codes. v2 errors include a stable code and hum | `404` | Resource not found | Verify the ID exists and belongs to your workspace | | `429` | Rate limit exceeded | Wait for the duration in the `Retry-After` header | +### Unrecognized fields are rejected + +Every v2 endpoint validates the request against its published schema — path parameters, query string, and body — and answers `400` for any field it does not declare. A misspelled parameter is an error rather than a silent no-op, so `?limt=20` fails instead of quietly returning an unbounded list. + +This holds for endpoints that declare no query parameters at all. Do not append tracking tags, cache busters, or other extra parameters to a v2 URL; send only what the endpoint documents. + +```json +{ + "error": { + "code": "BAD_REQUEST", + "message": "Invalid request", + "details": [ + { "code": "unrecognized_keys", "keys": ["limt"], "path": [], "message": "Unrecognized key: \"limt\"" } + ] + } +} +``` + Use [Get Billing Status](/api-reference/billing/getBillingStatus) to inspect current credit and storage usage. diff --git a/apps/docs/lib/openapi-code-samples-client.ts b/apps/docs/lib/openapi-code-samples-client.ts new file mode 100644 index 00000000000..f983da91e43 --- /dev/null +++ b/apps/docs/lib/openapi-code-samples-client.ts @@ -0,0 +1,37 @@ +'use client' + +import type { CodeUsageGeneratorFn } from 'fumadocs-openapi/requests/generators' +import { createCodeUsageGeneratorRegistry } from 'fumadocs-openapi/requests/generators' +import { registerDefault } from 'fumadocs-openapi/requests/generators/all' + +/** + * Context handed to {@link generateWithAuth} by the server: which built-in + * generator to delegate to, and the auth headers the sample must send. + */ +export interface AuthCodeSampleContext { + generatorId: string + headers: Record +} + +const generators = createCodeUsageGeneratorRegistry() +registerDefault(generators) + +/** + * Wraps a built-in code-usage generator so the sample carries the operation's + * security headers. Fumadocs builds request data from declared parameters only, + * so an operation's security requirement never reaches the generated snippet. + */ +export const generateWithAuth: CodeUsageGeneratorFn = (url, data, context) => { + const { generatorId, headers } = context.server as AuthCodeSampleContext + const generator = generators.get(generatorId) + if (!generator) { + throw new Error(`[docs] Unknown code usage generator: ${generatorId}`) + } + + const authHeaders: Record = {} + for (const [name, value] of Object.entries(headers)) { + authHeaders[name] = { value } + } + + return generator.generate(url, { ...data, header: { ...authHeaders, ...data.header } }, context) +} diff --git a/apps/docs/lib/openapi-code-samples.ts b/apps/docs/lib/openapi-code-samples.ts new file mode 100644 index 00000000000..ebfbf764bee --- /dev/null +++ b/apps/docs/lib/openapi-code-samples.ts @@ -0,0 +1,21 @@ +import type { InlineCodeUsageGenerator } from 'fumadocs-openapi/requests/generators' +import { createCodeUsageGeneratorRegistry } from 'fumadocs-openapi/requests/generators' +import { registerDefault } from 'fumadocs-openapi/requests/generators/all' +import { generateWithAuth } from '@/lib/openapi-code-samples-client' + +const generators = createCodeUsageGeneratorRegistry() +registerDefault(generators) + +/** + * Replace every built-in language sample with one that prepends `headers`, + * preserving the built-in tab order, language, and label. + */ +export function buildAuthCodeSamples(headers: Record): InlineCodeUsageGenerator[] { + return Array.from(generators.map().entries()).map(([id, generator]) => ({ + id, + lang: generator.lang, + label: generator.label, + source: generateWithAuth, + serverContext: { generatorId: id, headers }, + })) +} diff --git a/apps/docs/lib/openapi.ts b/apps/docs/lib/openapi.ts index f67d60db719..7841fd863ad 100644 --- a/apps/docs/lib/openapi.ts +++ b/apps/docs/lib/openapi.ts @@ -1,6 +1,9 @@ import { readFileSync } from 'node:fs' import { join } from 'node:path' +import type { MethodInformation } from 'fumadocs-openapi' +import type { InlineCodeUsageGenerator } from 'fumadocs-openapi/requests/generators' import { createOpenAPI } from 'fumadocs-openapi/server' +import { buildAuthCodeSamples } from '@/lib/openapi-code-samples' import { OPENAPI_SPEC_FILES } from '@/lib/openapi-specs' export const openapi = createOpenAPI({ @@ -75,6 +78,109 @@ function getSpecs(): Record[] { return cachedSpecs } +type SecurityRequirement = Record + +interface SecurityScheme { + type?: string + in?: string + name?: string + scheme?: string +} + +interface SharedSecurity { + security: SecurityRequirement[] + schemes: Record +} + +const AUTH_SAMPLE_VALUE = 'YOUR_API_KEY' + +let cachedSharedSecurity: SharedSecurity | null = null + +/** + * Document-level security shared by every rendered spec. Code samples are + * generated from an operation alone, with no handle on the document that owns + * it, so the specs must agree on their default security — a spec that diverges + * would silently get another document's auth in its samples. + */ +function getSharedSecurity(): SharedSecurity { + if (cachedSharedSecurity) return cachedSharedSecurity + + let shared: SharedSecurity | undefined + let sharedFile: string | undefined + + getSpecs().forEach((spec, index) => { + const file = OPENAPI_SPEC_FILES[index] + const current: SharedSecurity = { + security: (spec.security as SecurityRequirement[] | undefined) ?? [], + schemes: + ((spec.components as Record | undefined)?.securitySchemes as + | Record + | undefined) ?? {}, + } + + if (!shared) { + shared = current + sharedFile = file + return + } + + if (JSON.stringify(current) !== JSON.stringify(shared)) { + throw new Error( + `[docs] ${file} declares different default security than ${sharedFile}. Every OpenAPI spec must share one security scheme so generated code samples stay correct.` + ) + } + }) + + cachedSharedSecurity = shared ?? { security: [], schemes: {} } + return cachedSharedSecurity +} + +/** + * Resolve a security requirement to the request headers a sample must send. + * The first non-empty alternative wins — an empty one means the operation also + * accepts anonymous callers, which is not what a reference example should show. + */ +function resolveAuthHeaders( + security: SecurityRequirement[], + schemes: Record +): Record { + const requirement = security.find((item) => Object.keys(item).length > 0) + if (!requirement) return {} + + const headers: Record = {} + for (const name of Object.keys(requirement)) { + const scheme = schemes[name] + if (!scheme) { + throw new Error(`[docs] Operation references undefined security scheme "${name}"`) + } + if (scheme.type === 'apiKey' && scheme.in === 'header' && scheme.name) { + headers[scheme.name] = AUTH_SAMPLE_VALUE + continue + } + if (scheme.type === 'http' && scheme.scheme === 'bearer') { + headers.Authorization = `Bearer ${AUTH_SAMPLE_VALUE}` + continue + } + throw new Error( + `[docs] Security scheme "${name}" (type ${scheme.type}) cannot be rendered as a request header in code samples` + ) + } + return headers +} + +/** + * Code samples for an operation, with its authentication header included. + * Fumadocs derives sample requests from declared parameters only, so without + * this every endpoint documents an unauthenticated call that returns `401`. + */ +export function getAuthenticatedCodeSamples(method: MethodInformation): InlineCodeUsageGenerator[] { + const shared = getSharedSecurity() + const security = (method.security as SecurityRequirement[] | undefined) ?? shared.security + const headers = resolveAuthHeaders(security, shared.schemes) + if (Object.keys(headers).length === 0) return [] + return buildAuthCodeSamples(headers) +} + /** * Locate an operation by path + method across every rendered spec, returning the * operation together with the spec that owns it so `$ref`s resolve within the diff --git a/apps/docs/openapi-v2-billing.json b/apps/docs/openapi-v2-billing.json index 05e85b78dd7..5b513c1c03d 100644 --- a/apps/docs/openapi-v2-billing.json +++ b/apps/docs/openapi-v2-billing.json @@ -36,7 +36,7 @@ "get": { "operationId": "getBillingStatus", "summary": "Get Billing Status", - "description": "Return the current plan, billing standing, credit allowance, and storage quota. `credits` and `storage` report the payer's pooled allowances and are null unless the caller can manage that payer's billing; they are always null for a workspace API key. Billing history lives at `GET /api/v2/billing/logs`. Without a Stripe subscription — notably on the free plan — there is no real billing period: `period` is the open interval 1970-01-01 to 9999-12-31 and `credits.used` is lifetime consumption, not consumption since a period start.", + "description": "Return the current plan, billing standing, credit allowance, and storage quota. `credits` and `storage` report the payer's pooled allowances and are null unless the caller can manage that payer's billing; they are always null for a workspace API key. Billing history lives at `GET /api/v2/billing/logs`.", "tags": ["Billing"], "parameters": [ { @@ -101,7 +101,7 @@ "get": { "operationId": "listBillingLogs", "summary": "List Billing Logs", - "description": "List the credit-denominated billing ledger with source filtering and opaque cursor pagination. `period` defaults to `30d`, so an unqualified request covers only the last 30 days: paginating to `nextCursor: null` exhausts that window, not the whole ledger. Pass `period=all` for full history, or `period=custom` with `startDate` and `endDate` for a specific range.", + "description": "List the credit-denominated billing ledger with source filtering and opaque cursor pagination. `period` defaults to `30d`, so an unqualified request covers only the last 30 days: paginating to `nextCursor: null` exhausts that window, not the whole ledger. An inverted custom window is a 400 rather than an empty page.", "tags": ["Billing"], "parameters": [ { @@ -140,10 +140,10 @@ "name": "period", "in": "query", "required": false, - "description": "Relative window, all history, or a custom date range.", + "description": "Relative window, all history, or a custom date range. `startDate` and `endDate` are accepted only with `custom`; every other value computes its own window.", "schema": { "default": "30d", - "description": "Relative window, all history, or a custom date range.", + "description": "Relative window, all history, or a custom date range. `startDate` and `endDate` are accepted only with `custom`; every other value computes its own window.", "type": "string", "enum": ["1d", "7d", "30d", "all", "custom"] } @@ -152,32 +152,34 @@ "name": "startDate", "in": "query", "required": false, - "description": "Start of a custom window as a Date-parseable string.", + "description": "Only include usage events recorded at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. Requires `period=custom`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant.", "schema": { - "description": "Start of a custom window as a Date-parseable string.", "type": "string", - "minLength": 1 + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include usage events recorded at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. Requires `period=custom`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant." } }, { "name": "endDate", "in": "query", "required": false, - "description": "End of a custom window as a Date-parseable string; defaults to now.", + "description": "Only include usage events recorded at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. Requires `period=custom`, and defaults to now when omitted. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant.", "schema": { - "description": "End of a custom window as a Date-parseable string; defaults to now.", "type": "string", - "minLength": 1 + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include usage events recorded at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. Requires `period=custom`, and defaults to now when omitted. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant." } }, { "name": "limit", "in": "query", "required": false, - "description": "Maximum usage events per page, from 1 to 100.", + "description": "Maximum usage events per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum usage events per page, from 1 to 100.", + "description": "Maximum usage events per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -187,9 +189,9 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "type": "string", "minLength": 1 } @@ -248,7 +250,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings, then API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -283,13 +285,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -304,7 +306,7 @@ }, "responses": { "BadRequest": { - "description": "The request is invalid.", + "description": "The request is invalid. This includes a query parameter sent with no value (`?limit=`, `?search=`), which is rejected rather than read as zero, empty, or the parameter default — omit the parameter instead.", "content": { "application/json": { "schema": { @@ -334,7 +336,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. When the cause is one a caller can act on, `error.details.code` names it. A resource in a workspace the caller cannot reach at all answers `404` instead, so absence and denial are indistinguishable.", "content": { "application/json": { "schema": { @@ -364,7 +366,7 @@ } }, "RunIdConflict": { - "description": "The run cannot be started. Two causes share this status, distinguished by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already associated with a different request, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has already reached the maximum workflow-to-workflow call depth.", + "description": "The run cannot be started, for one of two causes named by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already claimed, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has reached the maximum workflow-to-workflow call depth.", "headers": { "X-Run-Id": { "$ref": "#/components/headers/X-Run-Id" @@ -378,18 +380,8 @@ } } }, - "Gone": { - "description": "The requested generated resource has expired.", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/V2Error" - } - } - } - }, "PayloadTooLarge": { - "description": "The request, or a resource collection it must materialize, exceeds the allowed size. Besides an oversized request body, this covers a generated artifact that renders past the download ceiling and a workspace folder tree too large to load in full.", + "description": "The request, or a resource collection it must materialize, exceeds the allowed size: an oversized request body, a generated artifact past the download ceiling, or a workspace folder tree too large to load in full.", "content": { "application/json": { "schema": { @@ -434,7 +426,7 @@ } }, "ClientClosedRequest": { - "description": "The client closed the connection before the response was produced.", + "description": "The client closed the connection before the response was produced. An abort can leave the run going, so `error.details.runId` carries the run id — reconcile against the runs resource rather than starting another run.", "content": { "application/json": { "schema": { @@ -454,7 +446,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. `Retry-After` carries the seconds to wait; treat it as a floor and add jitter. The header is omitted when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, because the run may already have started — reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -480,7 +477,7 @@ "description": "Human-readable explanation of the error." }, "details": { - "description": "Optional structured error details." + "description": "Structured error details. On a `403` whose cause a caller can act on, this carries a `code` from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `SECRET_ADMIN_ACCESS_REQUIRED` — The caller can write in the workspace but is not an admin of this secret. Ask a workspace admin, or someone holding admin on the secret, to grant access or set the value.\n- `WORKSPACE_RESOURCE_LIMIT_REACHED` — The workspace already holds the maximum number of resources of this kind. Delete one, or contact Sim to raise the limit; the message names the ceiling.\n- `PUBLIC_SHARING_NOT_ALLOWED` — The workspace's organization does not permit sharing this resource publicly. An organization admin controls the policy.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address." } }, "required": ["code", "message"], @@ -749,7 +746,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], diff --git a/apps/docs/openapi-v2-files-audit.json b/apps/docs/openapi-v2-files-audit.json index 53404f27332..e842423e106 100644 --- a/apps/docs/openapi-v2-files-audit.json +++ b/apps/docs/openapi-v2-files-audit.json @@ -2,7 +2,7 @@ "openapi": "3.1.0", "info": { "title": "Sim API v2 — Files & Audit Logs", - "description": "Version 2 of the Sim REST API for workspace files and organization audit logs. Lists use opaque cursors, and rate-limit state is returned in response headers. Download File streams raw bytes as `application/octet-stream`; every other response uses the canonical v2 data, cursor-list, or error envelope.", + "description": "Version 2 of the Sim REST API for workspace files, resumable uploads, public shares, and organization audit logs.", "version": "2.0.0", "contact": { "name": "Sim Support", @@ -40,7 +40,7 @@ "get": { "operationId": "listFiles", "summary": "List Files", - "description": "List workspace files with search, sorting, folder filtering, and opaque cursor pagination.", + "description": "List workspace files with search, sorting, folder filtering, and opaque cursor pagination. Defaults to active files; pass `scope=archived` to page over soft-deleted ones. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Files"], "parameters": [ { @@ -58,10 +58,22 @@ "name": "folderPath", "in": "query", "required": false, - "description": "Restrict results to files directly inside this folder.", + "description": "Restrict results to files directly inside this folder. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", "schema": { - "description": "Restrict results to files directly inside this folder.", - "type": "string" + "description": "Restrict results to files directly inside this folder. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", + "$ref": "#/components/schemas/FolderPathInput" + } + }, + { + "name": "scope", + "in": "query", + "required": false, + "description": "Which lifecycle set to list: `active` (default) for live files, `archived` for files a `DELETE` soft-deleted. `folderPath` resolves against active folders only, so pairing it with `scope=archived` returns an empty page when the containing folder was archived too.", + "schema": { + "default": "active", + "description": "Which lifecycle set to list: `active` (default) for live files, `archived` for files a `DELETE` soft-deleted. `folderPath` resolves against active folders only, so pairing it with `scope=archived` returns an empty page when the containing folder was archived too.", + "type": "string", + "enum": ["active", "archived"] } }, { @@ -80,10 +92,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "uploadedAt", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "size", "uploadedAt", "updatedAt"] } @@ -104,20 +116,20 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum files per page, clamped to 1–1000.", + "description": "Maximum files per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", "schema": { - "description": "Maximum files per page, clamped to 1–1000.", - "default": 100, - "type": "number" + "description": "Maximum files per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", + "type": "integer", + "default": 100 } }, { "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "type": "string", "minLength": 1 } @@ -157,6 +169,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -287,6 +302,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -368,6 +386,9 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -466,12 +487,18 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -553,6 +580,9 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -575,7 +605,7 @@ "get": { "operationId": "downloadFile", "summary": "Download File", - "description": "Download the current file bytes from a workspace. A generated document is served as its compiled artifact, so it returns `409` while that artifact is still compiling and `413` if it renders past the size ceiling.", + "description": "Download the current file bytes from a workspace. A generated document is served as its compiled artifact, so it answers `409` while that artifact is still compiling and `413` if it renders past the size ceiling. Downloading records an audit event, so it is not a safe read. A `HEAD` skips the effect but is authorized exactly as the `GET` is, so it answers `400`, `401`, `403`, or `404` wherever the `GET` would and an empty `200` otherwise. Skipping the effect means skipping the read that produces the payload, so that `200` carries none of the response headers documented below — it answers whether the `GET` would be allowed, not what the `GET` would return. In particular a `HEAD` does not report `Content-Length`, so it cannot be used to size a download in advance; read the size from the file resource instead.", "tags": ["Files"], "parameters": [ { @@ -667,7 +697,7 @@ "delete": { "operationId": "deleteFile", "summary": "Delete File", - "description": "Archive a workspace file. This is a soft delete: the row is retained with a deletion timestamp, the file stops appearing in listings and is no longer readable through the API, and its stored bytes are never removed. An archived file can be restored from the workspace Recently Deleted settings; the v2 API exposes no restore operation.", + "description": "Archive a workspace file. This is a soft delete: the file stops appearing in the default listing and is no longer readable through the API, but its stored bytes are never removed. Archiving an already-archived file is a `404`, not a no-op. List archived files with `GET /files?scope=archived`, and reverse the delete with `POST /files/{fileId}/restore`.", "tags": ["Files"], "parameters": [ { @@ -729,9 +759,6 @@ "404": { "$ref": "#/components/responses/NotFound" }, - "409": { - "$ref": "#/components/responses/Conflict" - }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -811,6 +838,93 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, + "/api/v2/files/{fileId}/restore": { + "post": { + "operationId": "restoreFile", + "summary": "Restore File", + "description": "Reverse a soft delete and return the file to the workspace. Not a pure undo: the file comes back at the workspace root, and gains a `_restored` suffix when another file there already holds its name, so read `folderPath` and `name` off the response. Restoring an already-active file returns it unchanged, so a retry is safe. An archived workspace is a `400`, and a name the restore could not free is a `409`.", + "tags": ["Files"], + "parameters": [ + { + "name": "fileId", + "in": "path", + "required": true, + "description": "File identifier.", + "schema": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "pattern": "^[A-Za-z0-9_-]+$", + "description": "File identifier." + } + } + ], + "requestBody": { + "required": true, + "description": "Workspace scope for the archived file.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/RestoreFileRequest" + } + } + } + }, + "responses": { + "200": { + "description": "The file as it exists after the restore.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2RestoreFileResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "409": { + "$ref": "#/components/responses/Conflict" + }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -905,7 +1019,7 @@ "get": { "operationId": "listAuditLogs", "summary": "List Audit Logs", - "description": "List an organization audit trail with filters and opaque cursor pagination. Requires an Enterprise subscription and organization admin or owner access. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "description": "List an organization audit trail with filters and opaque cursor pagination. Requires an Enterprise subscription and organization admin or owner access. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Audit Logs"], "parameters": [ { @@ -922,9 +1036,9 @@ "name": "resourceType", "in": "query", "required": false, - "description": "Filter by exact resource type.", + "description": "Filter by resource type. Accepts a comma-separated set; members are trimmed and deduplicated, and member order affects neither the result nor the cursor.", "schema": { - "description": "Filter by exact resource type.", + "description": "Filter by resource type. Accepts a comma-separated set; members are trimmed and deduplicated, and member order affects neither the result nor the cursor.", "type": "string" } }, @@ -945,27 +1059,32 @@ "description": "Filter to actions in one workspace.", "schema": { "description": "Filter to actions in one workspace.", - "type": "string" + "type": "string", + "minLength": 1 } }, { "name": "startDate", "in": "query", "required": false, - "description": "Inclusive ISO 8601 start timestamp.", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant.", "schema": { - "description": "Inclusive ISO 8601 start timestamp.", - "type": "string" + "type": "string", + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant." } }, { "name": "endDate", "in": "query", "required": false, - "description": "Inclusive ISO 8601 end timestamp.", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant.", "schema": { - "description": "Inclusive ISO 8601 end timestamp.", - "type": "string" + "type": "string", + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant." } }, { @@ -975,20 +1094,18 @@ "description": "Include actions by users who have left the organization.", "schema": { "description": "Include actions by users who have left the organization.", - "default": "false", - "type": "string", - "enum": ["true", "false"] + "type": "boolean" } }, { "name": "limit", "in": "query", "required": false, - "description": "Maximum entries per page, from 1 to 100.", + "description": "Maximum audit entries to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum entries per page, from 1 to 100.", - "type": "number", + "description": "Maximum audit entries to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", "minimum": 1, "maximum": 100 } @@ -997,10 +1114,11 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque cursor returned by the previous page.", - "type": "string" + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 } }, { @@ -1058,9 +1176,6 @@ "403": { "$ref": "#/components/responses/Forbidden" }, - "404": { - "$ref": "#/components/responses/NotFound" - }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1077,7 +1192,7 @@ "get": { "operationId": "getAuditLog", "summary": "Get Audit Log", - "description": "Return one organization audit-log entry. Requires an Enterprise subscription and organization admin or owner access. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "description": "Return one organization audit-log entry. Requires an Enterprise subscription and organization admin or owner access. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Audit Logs"], "parameters": [ { @@ -1203,6 +1318,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1295,7 +1413,7 @@ "patch": { "operationId": "upsertFileShare", "summary": "Enable or Disable File Share", - "description": "Create or partially update a server-tokenized public share. Only isActive is required, and an omitted authType keeps the stored auth mode. What happens to password and allowedEmails depends on the resulting mode, because enabling a share always rewrites the credentials the chosen mode does not use: 'public' clears the stored password and empties allowedEmails; 'password' keeps the stored password when password is omitted but empties allowedEmails; 'email' and 'sso' clear the stored password and keep the stored allowedEmails when the field is omitted. Only disabling with isActive false preserves the whole access configuration untouched — it also retains the token, so re-enabling restores the share as it was. Two enabling combinations are rejected outright with a 400 instead of being partially applied: 'password' when neither a password is supplied nor one is already stored, and 'email' or 'sso' when the resulting allowedEmails would be empty because none was supplied and none is stored. On a file that has never been shared there is nothing stored to fall back on, so enabling any mode other than 'public' must carry its credential in the same request. A workspace API key cannot call this operation. Because unauthorized resources are concealed, the rejection is reported as `404` rather than `403`; use a personal API key.", + "description": "Create or partially update a server-tokenized public share. Only `isActive` is required; each other field states what enabling a mode does to it. Enabling any mode other than `public` on a file that has never been shared must carry its credential in the same request. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Files"], "parameters": [ { @@ -1357,6 +1475,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1504,6 +1625,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1520,7 +1644,7 @@ "get": { "operationId": "listFilesFolders", "summary": "List Folders", - "description": "List workspace file folders with optional parent-path filtering and sorting. The bounded set is returned in one page with `nextCursor` always null; there is no second page to fetch.", + "description": "List workspace file folders with optional parent-path filtering and sorting. The bounded set is returned in one page; `nextCursor` is always null.", "tags": ["Files"], "parameters": [ { @@ -1541,7 +1665,7 @@ "description": "Restrict results to direct children of this parent path.", "schema": { "description": "Restrict results to direct children of this parent path.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, { @@ -1560,10 +1684,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "name", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -1679,6 +1803,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1743,6 +1870,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1778,16 +1908,30 @@ "description": "Path of the folder to delete.", "schema": { "description": "Path of the folder to delete.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, { "name": "recursive", "in": "query", "required": false, - "description": "Delete nested files and folders when true.", + "description": "Delete the folder's nested files and folders too. An empty folder deletes either way; a non-empty one needs this. The listed spellings are the whole accepted vocabulary and are case-sensitive; any other value is rejected.", "schema": { - "description": "Delete nested files and folders when true.", + "description": "Delete the folder's nested files and folders too. An empty folder deletes either way; a non-empty one needs this. The listed spellings are the whole accepted vocabulary and are case-sensitive; any other value is rejected.", + "enum": [ + "true", + "1", + "yes", + "on", + "y", + "enabled", + "false", + "0", + "no", + "off", + "n", + "disabled" + ], "default": "false", "type": "string" } @@ -1849,7 +1993,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings, then API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -1909,13 +2053,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -1930,7 +2074,7 @@ }, "responses": { "BadRequest": { - "description": "The request is invalid.", + "description": "The request is invalid. This includes a query parameter sent with no value (`?limit=`, `?search=`), which is rejected rather than read as zero, empty, or the parameter default — omit the parameter instead.", "content": { "application/json": { "schema": { @@ -1960,7 +2104,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. When the cause is one a caller can act on, `error.details.code` names it. A resource in a workspace the caller cannot reach at all answers `404` instead, so absence and denial are indistinguishable.", "content": { "application/json": { "schema": { @@ -1990,7 +2134,7 @@ } }, "RunIdConflict": { - "description": "The run cannot be started. Two causes share this status, distinguished by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already associated with a different request, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has already reached the maximum workflow-to-workflow call depth.", + "description": "The run cannot be started, for one of two causes named by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already claimed, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has reached the maximum workflow-to-workflow call depth.", "headers": { "X-Run-Id": { "$ref": "#/components/headers/X-Run-Id" @@ -2004,18 +2148,8 @@ } } }, - "Gone": { - "description": "The requested generated resource has expired.", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/V2Error" - } - } - } - }, "PayloadTooLarge": { - "description": "The request, or a resource collection it must materialize, exceeds the allowed size. Besides an oversized request body, this covers a generated artifact that renders past the download ceiling and a workspace folder tree too large to load in full.", + "description": "The request, or a resource collection it must materialize, exceeds the allowed size: an oversized request body, a generated artifact past the download ceiling, or a workspace folder tree too large to load in full.", "content": { "application/json": { "schema": { @@ -2060,7 +2194,7 @@ } }, "ClientClosedRequest": { - "description": "The client closed the connection before the response was produced.", + "description": "The client closed the connection before the response was produced. An abort can leave the run going, so `error.details.runId` carries the run id — reconcile against the runs resource rather than starting another run.", "content": { "application/json": { "schema": { @@ -2080,7 +2214,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. `Retry-After` carries the seconds to wait; treat it as a floor and add jitter. The header is omitted when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, because the run may already have started — reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -2106,7 +2245,7 @@ "description": "Human-readable explanation of the error." }, "details": { - "description": "Optional structured error details." + "description": "Structured error details. On a `403` whose cause a caller can act on, this carries a `code` from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `SECRET_ADMIN_ACCESS_REQUIRED` — The caller can write in the workspace but is not an admin of this secret. Ask a workspace admin, or someone holding admin on the secret, to grant access or set the value.\n- `WORKSPACE_RESOURCE_LIMIT_REACHED` — The workspace already holds the maximum number of resources of this kind. Delete one, or contact Sim to raise the limit; the message names the ceiling.\n- `PUBLIC_SHARING_NOT_ALLOWED` — The workspace's organization does not permit sharing this resource publicly. An organization admin controls the policy.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address." } }, "required": ["code", "message"], @@ -2127,6 +2266,12 @@ } ] }, + "FolderPathInput": { + "title": "Folder path input", + "description": "Folder path. A missing leading slash is normalized before validation. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096, + "type": "string" + }, "V2File": { "type": "object", "properties": { @@ -2143,12 +2288,12 @@ "size": { "type": "number", "minimum": 0, - "description": "Size in bytes of the stored file. For a generated document (docx, pptx, pdf, xlsx) the stored file is the generation source rather than the rendered document, so this does not predict how many bytes `GET /files/{fileId}` returns — that endpoint serves the compiled artifact, which is typically much larger.", + "description": "Size in bytes of the stored file. For a generated document (docx, pptx, pdf, xlsx) this is the generation source, not the rendered document, so it does not predict how many bytes `GET /files/{fileId}` returns.", "examples": [1024] }, "type": { "type": "string", - "description": "MIME type of the stored file. For a generated document (docx, pptx, pdf, xlsx) the stored file is the generation source, so this describes the source and not what `GET /files/{fileId}` serves — that endpoint returns the compiled artifact under the rendered document type.", + "description": "MIME type of the stored file. For a generated document (docx, pptx, pdf, xlsx) this is the generation source type, not the rendered document type `GET /files/{fileId}` serves.", "examples": ["text/csv"] }, "key": { @@ -2158,7 +2303,9 @@ }, "folderPath": { "type": "string", - "description": "Canonical containing-folder path. `/` is the workspace root." + "title": "Folder path", + "description": "Canonical containing-folder path. `/` is the workspace root.", + "maxLength": 4096 }, "uploadedByEmail": { "type": "string", @@ -2178,6 +2325,19 @@ "description": "ISO 8601 timestamp of the last content or metadata write.", "format": "date-time", "examples": ["2026-01-15T10:30:00Z"] + }, + "deletedAt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "ISO 8601 timestamp when the file was archived by `DELETE /files/{fileId}`, or null while the file is active. Only `GET /files?scope=archived` returns files with a non-null value.", + "format": "date-time", + "examples": ["2026-01-16T09:00:00Z"] } }, "required": [ @@ -2189,7 +2349,8 @@ "folderPath", "uploadedByEmail", "uploadedAt", - "updatedAt" + "updatedAt", + "deletedAt" ], "additionalProperties": false, "title": "Workspace file", @@ -2214,7 +2375,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -2233,7 +2394,8 @@ "folderPath": "/Engineering", "uploadedByEmail": "jane@example.com", "uploadedAt": "2026-01-15T10:30:00Z", - "updatedAt": "2026-01-15T10:30:00Z" + "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null } ], "nextCursor": null @@ -2263,7 +2425,8 @@ "folderPath": "/Engineering", "uploadedByEmail": "jane@example.com", "uploadedAt": "2026-01-15T10:30:00Z", - "updatedAt": "2026-01-15T10:30:00Z" + "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null } } ] @@ -2290,11 +2453,11 @@ }, "folderPath": { "description": "Canonical containing-folder path. Omit for the workspace root.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" }, "content": { "default": "", - "description": "Initial file content. Omit or send an empty string for a zero-byte file. The 70,000,000-character bound is a JSON-envelope guard, not the file-size limit: the decoded bytes must be at most 50 MiB, so a longer base64 payload is admitted here and then rejected with 413. Use an upload session for anything larger.", + "description": "Initial file content. Omit or send an empty string for a zero-byte file. The 70,000,000-character bound guards the JSON envelope; the decoded bytes must be at most 50 MiB, and a longer base64 payload is rejected with `413`. Use an upload session for anything larger.", "type": "string", "maxLength": 70000000 }, @@ -2390,7 +2553,7 @@ "url": { "type": "string", "format": "uri", - "description": "Signed URL to which the file bytes are uploaded." + "description": "Signed URL to which the file bytes are uploaded. Send the bytes with `PUT` to this URL, including exactly the headers in `headers` and nothing that alters the body. Success is `204` with an empty body. A failure is the same `{ \"error\": { \"code\", \"message\" } }` envelope as every other v2 response: `400` when the body does not match the size or content type the session was created for, `403` when the token is invalid, expired, or belongs to another session, and `409` when the session is no longer accepting bytes. The URL is signed and self-describing — construct it from this field only, never by hand." }, "headers": { "type": "object", @@ -2510,7 +2673,7 @@ }, "folderPath": { "description": "Canonical destination folder path. Omit for the workspace root.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["workspaceId", "name", "contentType", "size"], @@ -2543,7 +2706,7 @@ "url": { "type": "string", "format": "uri", - "description": "Signed URL for this upload part." + "description": "Signed URL for this upload part. Send the bytes with `PUT` to this URL, including exactly the headers in `headers` and nothing that alters the body. Success is `204` with an empty body. A failure is the same `{ \"error\": { \"code\", \"message\" } }` envelope as every other v2 response: `400` when the body does not match the size or content type the session was created for, `403` when the token is invalid, expired, or belongs to another session, and `409` when the session is no longer accepting bytes. The URL is signed and self-describing — construct it from this field only, never by hand." }, "headers": { "type": "object", @@ -2679,6 +2842,54 @@ } ] }, + "V2RestoreFileResponse": { + "type": "object", + "properties": { + "data": { + "description": "Response data.", + "$ref": "#/components/schemas/V2File" + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Restore file response", + "description": "The restored workspace file, at the root and under its post-restore name.", + "examples": [ + { + "data": { + "id": "wf_V1StGXR8z5jdHi6BmyT91", + "name": "data_restored.csv", + "size": 1024, + "type": "text/csv", + "key": "workspace/example/data.csv", + "folderPath": "/", + "uploadedByEmail": "jane@example.com", + "uploadedAt": "2026-01-15T10:30:00Z", + "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null + } + } + ] + }, + "RestoreFileRequest": { + "type": "object", + "properties": { + "workspaceId": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the archived file." + } + }, + "required": ["workspaceId"], + "additionalProperties": false, + "title": "Restore file request", + "description": "Workspace scope for the archived file.", + "examples": [ + { + "workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64" + } + ] + }, "V2FileShare": { "type": "object", "properties": { @@ -2759,12 +2970,12 @@ "size": { "type": "number", "minimum": 0, - "description": "Size in bytes of the stored file. For a generated document (docx, pptx, pdf, xlsx) the stored file is the generation source rather than the rendered document, so this does not predict how many bytes `GET /files/{fileId}` returns — that endpoint serves the compiled artifact, which is typically much larger.", + "description": "Size in bytes of the stored file. For a generated document (docx, pptx, pdf, xlsx) this is the generation source, not the rendered document, so it does not predict how many bytes `GET /files/{fileId}` returns.", "examples": [1024] }, "type": { "type": "string", - "description": "MIME type of the stored file. For a generated document (docx, pptx, pdf, xlsx) the stored file is the generation source, so this describes the source and not what `GET /files/{fileId}` serves — that endpoint returns the compiled artifact under the rendered document type.", + "description": "MIME type of the stored file. For a generated document (docx, pptx, pdf, xlsx) this is the generation source type, not the rendered document type `GET /files/{fileId}` serves.", "examples": ["text/csv"] }, "key": { @@ -2774,7 +2985,9 @@ }, "folderPath": { "type": "string", - "description": "Canonical containing-folder path. `/` is the workspace root." + "title": "Folder path", + "description": "Canonical containing-folder path. `/` is the workspace root.", + "maxLength": 4096 }, "uploadedByEmail": { "type": "string", @@ -2795,6 +3008,19 @@ "format": "date-time", "examples": ["2026-01-15T10:30:00Z"] }, + "deletedAt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "ISO 8601 timestamp when the file was archived by `DELETE /files/{fileId}`, or null while the file is active. Only `GET /files?scope=archived` returns files with a non-null value.", + "format": "date-time", + "examples": ["2026-01-16T09:00:00Z"] + }, "share": { "anyOf": [ { @@ -2817,6 +3043,7 @@ "uploadedByEmail", "uploadedAt", "updatedAt", + "deletedAt", "share" ], "additionalProperties": false, @@ -2847,6 +3074,7 @@ "uploadedByEmail": "jane@example.com", "uploadedAt": "2026-01-15T10:30:00Z", "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null, "share": null } }, @@ -2861,6 +3089,7 @@ "uploadedByEmail": "jane@example.com", "uploadedAt": "2026-01-15T10:30:00Z", "updatedAt": "2026-01-15T10:30:00Z", + "deletedAt": null, "share": { "id": "shr_8Hf3kL9wQ2mNpXr6Tz1Vb", "token": "share-token-example", @@ -3008,7 +3237,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -3137,7 +3366,7 @@ }, "targetFolderPath": { "description": "Destination folder path. Omit to move files to the workspace root.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["workspaceId", "fileIds"], @@ -3221,21 +3450,21 @@ }, "isActive": { "type": "boolean", - "description": "Whether the share should resolve." + "description": "Whether the share should resolve. Disabling preserves the token and the whole access configuration, so re-enabling restores the share as it was; enabling rewrites the credentials the resulting mode does not use." }, "authType": { - "description": "How access to the share is gated.", + "description": "How access to the share is gated. The stored mode is kept when omitted. Enabling `public` clears the stored password and empties `allowedEmails`; `password` empties `allowedEmails`; `email` and `sso` clear the stored password.", "type": "string", "enum": ["public", "password", "email", "sso"] }, "password": { - "description": "Password for a password-gated share.", + "description": "Password for a password-gated share. Kept when omitted; enabling `password` with neither a supplied nor a stored password is a 400.", "type": "string", "minLength": 1, "maxLength": 1024 }, "allowedEmails": { - "description": "Allowed addresses or @domain patterns for email and SSO shares.", + "description": "Allowed addresses or `@domain` patterns for email and SSO shares. Kept when omitted; enabling `email` or `sso` with an empty resulting list is a 400.", "maxItems": 200, "type": "array", "items": { @@ -3272,7 +3501,7 @@ "content": { "type": "string", "maxLength": 70000000, - "description": "Complete replacement content for the file. The 70,000,000-character bound is a JSON-envelope guard, not the file-size limit: the decoded bytes must be at most 50 MiB, so a longer base64 payload is admitted here and then rejected with 413." + "description": "Complete replacement content for the file. The 70,000,000-character bound guards the JSON envelope; the decoded bytes must be at most 50 MiB, and a longer base64 payload is rejected with `413`." }, "encoding": { "default": "utf-8", @@ -3376,11 +3605,15 @@ }, "path": { "type": "string", - "description": "Canonical folder path used as the public folder identifier." + "title": "Non-root folder path", + "description": "Canonical folder path used as the public folder identifier.", + "maxLength": 4096 }, "parentPath": { "type": "string", - "description": "Canonical parent path; `/` is the root." + "title": "Folder path", + "description": "Canonical parent path; `/` is the root.", + "maxLength": 4096 }, "createdAt": { "type": "string", @@ -3417,7 +3650,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Always `null` — this list has no `cursor` or `limit` param and returns its whole bounded set in one page. Present so the list can gain pages later without a shape change." } }, "required": ["data", "nextCursor"], @@ -3438,6 +3671,12 @@ "title": "File folder response", "description": "A single workspace file folder." }, + "NonRootFolderPathInput": { + "title": "Non-root folder path input", + "description": "Non-root folder path. A missing leading slash is normalized before validation. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096, + "type": "string" + }, "CreateFileFolderRequest": { "type": "object", "properties": { @@ -3448,7 +3687,7 @@ }, "path": { "description": "Path of the folder to create.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, "required": ["workspaceId", "path"], @@ -3466,11 +3705,11 @@ }, "path": { "description": "Current folder path.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" }, "destinationPath": { "description": "New full path for the folder and its descendants.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, "required": ["workspaceId", "path", "destinationPath"], @@ -3483,7 +3722,9 @@ "properties": { "path": { "type": "string", - "description": "Deleted folder path." + "title": "Folder path", + "description": "Deleted folder path.", + "maxLength": 4096 }, "deleted": { "type": "boolean", diff --git a/apps/docs/openapi-v2-knowledge.json b/apps/docs/openapi-v2-knowledge.json index 37a2238c614..24157c46f4e 100644 --- a/apps/docs/openapi-v2-knowledge.json +++ b/apps/docs/openapi-v2-knowledge.json @@ -36,7 +36,7 @@ "get": { "operationId": "listKnowledgeBases", "summary": "List Knowledge Bases", - "description": "List knowledge bases in a workspace with folder filtering, search, sorting, and the canonical cursor envelope. The bounded workspace set is returned in one page with `nextCursor` always null; there is no second page to fetch. An unknown `folderPath` is a 404. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "List knowledge bases in a workspace with folder filtering, search, sorting, and opaque cursor pagination. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Knowledge Bases"], "parameters": [ { @@ -54,19 +54,19 @@ "name": "folderPath", "in": "query", "required": false, - "description": "Restrict results to knowledge bases in this folder.", + "description": "Restrict results to knowledge bases in this folder. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", "schema": { - "description": "Restrict results to knowledge bases in this folder.", - "type": "string" + "description": "Restrict results to knowledge bases in this folder. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", + "$ref": "#/components/schemas/FolderPathInput" } }, { "name": "search", "in": "query", "required": false, - "description": "Case-insensitive substring search on the resource name.", + "description": "Case-insensitive substring match against the resource name.", "schema": { - "description": "Case-insensitive substring search on the resource name.", + "description": "Case-insensitive substring match against the resource name.", "type": "string", "minLength": 1, "maxLength": 200 @@ -76,10 +76,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "createdAt", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -95,6 +95,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum knowledge bases to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum knowledge bases to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "schema": { + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -148,7 +172,7 @@ "post": { "operationId": "createKnowledgeBase", "summary": "Create Knowledge Base", - "description": "Create a knowledge base in a workspace with optional folder placement and chunking configuration. An unknown `folderPath` is a 404. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Create a knowledge base in a workspace with optional folder placement and chunking configuration. An unknown `folderPath` is a `404`. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Knowledge Bases"], "requestBody": { "required": true, @@ -217,7 +241,7 @@ "get": { "operationId": "getKnowledgeBase", "summary": "Get Knowledge Base", - "description": "Retrieve a knowledge base by identifier. Inaccessible knowledge bases are reported as not found. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Retrieve a knowledge base by identifier. Inaccessible knowledge bases are reported as not found. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Knowledge Bases"], "parameters": [ { @@ -271,6 +295,9 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -291,7 +318,7 @@ "patch": { "operationId": "updateKnowledgeBase", "summary": "Update Knowledge Base", - "description": "Update a knowledge base name, description, chunking configuration, or folder placement. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Update a knowledge base name, description, chunking configuration, or folder placement. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Knowledge Bases"], "parameters": [ { @@ -447,7 +474,7 @@ "post": { "operationId": "searchKnowledge", "summary": "Search Knowledge", - "description": "Search one or more knowledge bases with semantic vector retrieval, optional hybrid full-text retrieval, and structured tag filters. The request body is capped at 2 MiB; a larger body is a 413.", + "description": "Search one or more knowledge bases with semantic vector retrieval, optional hybrid full-text retrieval, and structured tag filters. Every result names the `knowledgeBaseId` it came from. A request body over 2 MiB is a `413`.", "tags": ["Knowledge Bases"], "requestBody": { "required": true, @@ -512,11 +539,87 @@ } } }, + "/api/v2/knowledge/{id}/tags": { + "get": { + "operationId": "listKnowledgeTags", + "summary": "List Tags", + "description": "List the knowledge base's tag vocabulary: each tag's display name, the slot it is stored in, and its field type. Filters and document reads use display names; document writes address slots. The bounded set is returned in one page; `nextCursor` is always null.", + "tags": ["Knowledge Bases"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "Unique knowledge base identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique knowledge base identifier." + } + }, + { + "name": "workspaceId", + "in": "query", + "required": true, + "description": "Workspace that owns the knowledge base.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the knowledge base." + } + } + ], + "responses": { + "200": { + "description": "The knowledge base tag vocabulary.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2KnowledgeTagListResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, "/api/v2/knowledge/{id}/documents": { "get": { "operationId": "listKnowledgeDocuments", "summary": "List Documents", - "description": "List documents in a knowledge base with filename search, state filtering, sorting, and opaque cursor pagination.", + "description": "List documents in a knowledge base with filename search, state filtering, tag filtering, sorting, and opaque cursor pagination. Tag values are keyed by display name; resolve those to write slots with `GET /api/v2/knowledge/{id}/tags`.", "tags": ["Knowledge Bases"], "parameters": [ { @@ -545,10 +648,10 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum documents to return, between 1 and 100.", + "description": "Maximum documents to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum documents to return, between 1 and 100.", + "description": "Maximum documents to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -558,10 +661,12 @@ "name": "search", "in": "query", "required": false, - "description": "Case-insensitive filename search.", + "description": "Case-insensitive substring match against the document filename.", "schema": { - "description": "Case-insensitive filename search.", - "type": "string" + "description": "Case-insensitive substring match against the document filename.", + "type": "string", + "minLength": 1, + "maxLength": 200 } }, { @@ -580,10 +685,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Document field used to sort results.", + "description": "Field used to sort the result. Sorting by `filename` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "uploadedAt", - "description": "Document field used to sort results.", + "description": "Field used to sort the result. Sorting by `filename` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": [ "filename", @@ -612,12 +717,25 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "type": "string", "minLength": 1 } + }, + { + "name": "tagFilters", + "in": "query", + "required": false, + "description": "A JSON-encoded array of at most 10 tag filters, using the same display-name shape as knowledge search: `[{\"tagName\":\"category\",\"operator\":\"eq\",\"value\":\"billing\"}]`. Every filter must hold, including two that name the same tag. A name that is not defined in this knowledge base is rejected, never ignored.", + "schema": { + "description": "A JSON-encoded array of at most 10 tag filters, using the same display-name shape as knowledge search: `[{\"tagName\":\"category\",\"operator\":\"eq\",\"value\":\"billing\"}]`. Every filter must hold, including two that name the same tag. A name that is not defined in this knowledge base is rejected, never ignored.", + "examples": [ + "[{\"tagName\":\"category\",\"operator\":\"eq\",\"value\":\"billing\"}]" + ], + "type": "string" + } } ], "responses": { @@ -648,9 +766,89 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + }, + "patch": { + "operationId": "bulkUpdateKnowledgeDocuments", + "summary": "Bulk Enable or Disable Documents", + "description": "Enable or disable many documents in one request, either by identifier or, with `selectAll`, every document in the knowledge base. Bulk delete is not offered; delete documents one at a time with `DELETE /api/v2/knowledge/{id}/documents/{documentId}`. A workspace API key is rejected with `403`; use a personal API key.", + "tags": ["Knowledge Bases"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "Unique knowledge base identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique knowledge base identifier." + } + } + ], + "requestBody": { + "required": true, + "description": "Operation and the documents it applies to.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/BulkUpdateKnowledgeDocumentsRequest" + } + } + } + }, + "responses": { + "200": { + "description": "The number and identifiers of the documents that changed.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2BulkKnowledgeDocumentsResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1043,6 +1241,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1228,6 +1429,9 @@ "401": { "$ref": "#/components/responses/Unauthorized" }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, "404": { "$ref": "#/components/responses/NotFound" }, @@ -1242,10 +1446,10 @@ } } }, - "delete": { - "operationId": "deleteKnowledgeDocument", - "summary": "Delete Document", - "description": "Remove one document from a knowledge base. What that means depends on the document. A directly uploaded document is deleted outright along with its indexed chunks. A connector-backed document is instead excluded: its row survives, marked excluded and disabled so it stops being searchable and a later connector sync does not re-add it, and its embeddings are not deleted. Either way the document no longer appears in listings or search results.", + "patch": { + "operationId": "updateKnowledgeDocument", + "summary": "Update Document", + "description": "Rename a document, enable or disable it for search, set any of its 17 tag slots, or requeue it for processing. Absent fields are unchanged, and derived indexing state is read-only. Resolve a tag display name to its slot with `GET /api/v2/knowledge/{id}/tags`. The returned document omits the connector provenance the detail read carries. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Knowledge Bases"], "parameters": [ { @@ -1269,22 +1473,22 @@ "minLength": 1, "description": "Unique knowledge document identifier." } - }, - { - "name": "workspaceId", - "in": "query", - "required": true, - "description": "Workspace that owns the knowledge base.", - "schema": { - "type": "string", - "minLength": 1, - "description": "Workspace that owns the knowledge base." - } } ], + "requestBody": { + "required": true, + "description": "Filename, search state, tag slot values, or a processing retry.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/UpdateKnowledgeDocumentRequest" + } + } + } + }, "responses": { "200": { - "description": "Knowledge document deletion acknowledgement.", + "description": "The updated document, or the requeue acknowledgement.", "headers": { "X-RateLimit-Limit": { "$ref": "#/components/headers/X-RateLimit-Limit" @@ -1299,7 +1503,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/V2KnowledgeDeleteResponse" + "$ref": "#/components/schemas/V2UpdateKnowledgeDocumentResponse" } } } @@ -1316,6 +1520,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1326,56 +1533,141 @@ "$ref": "#/components/responses/ServiceUnavailable" } } - } - }, - "/api/v2/knowledge/folders": { - "get": { - "operationId": "listKnowledgeFolders", - "summary": "List Folders", - "description": "List folders in the knowledge-base folder tree with filtering and sorting. The bounded set is returned in one page with `nextCursor` always null; there is no second page to fetch. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + }, + "delete": { + "operationId": "deleteKnowledgeDocument", + "summary": "Delete Document", + "description": "Remove one document from a knowledge base. An uploaded document is deleted outright with its indexed chunks. A connector-backed document is instead excluded — its row and embeddings survive, but it stops being searchable and a later sync does not re-add it. Either way it no longer appears in listings or search results.", "tags": ["Knowledge Bases"], "parameters": [ { - "name": "workspaceId", - "in": "query", + "name": "id", + "in": "path", "required": true, - "description": "Workspace whose folders should be listed.", + "description": "Unique knowledge base identifier.", "schema": { "type": "string", "minLength": 1, - "description": "Workspace whose folders should be listed." + "description": "Unique knowledge base identifier." } }, { - "name": "parentPath", - "in": "query", - "required": false, - "description": "Restrict results to direct children of this parent path.", + "name": "documentId", + "in": "path", + "required": true, + "description": "Unique knowledge document identifier.", "schema": { - "description": "Restrict results to direct children of this parent path.", - "type": "string" + "type": "string", + "minLength": 1, + "description": "Unique knowledge document identifier." } }, { - "name": "search", + "name": "workspaceId", "in": "query", - "required": false, - "description": "Case-insensitive substring match against the folder name.", + "required": true, + "description": "Workspace that owns the knowledge base.", "schema": { - "description": "Case-insensitive substring match against the folder name.", "type": "string", "minLength": 1, - "maxLength": 200 + "description": "Workspace that owns the knowledge base." } - }, - { - "name": "sortBy", + } + ], + "responses": { + "200": { + "description": "Knowledge document deletion acknowledgement.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2KnowledgeDeleteResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, + "/api/v2/knowledge/folders": { + "get": { + "operationId": "listKnowledgeFolders", + "summary": "List Folders", + "description": "List folders in the knowledge-base folder tree with filtering and sorting. The bounded set is returned in one page; `nextCursor` is always null. A workspace folder tree over 10,000 folders is a `413`.", + "tags": ["Knowledge Bases"], + "parameters": [ + { + "name": "workspaceId", + "in": "query", + "required": true, + "description": "Workspace whose folders should be listed.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Workspace whose folders should be listed." + } + }, + { + "name": "parentPath", + "in": "query", + "required": false, + "description": "Restrict results to direct children of this parent path.", + "schema": { + "description": "Restrict results to direct children of this parent path.", + "$ref": "#/components/schemas/FolderPathInput" + } + }, + { + "name": "search", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Case-insensitive substring match against the folder name.", + "schema": { + "description": "Case-insensitive substring match against the folder name.", + "type": "string", + "minLength": 1, + "maxLength": 200 + } + }, + { + "name": "sortBy", + "in": "query", + "required": false, + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "name", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -1444,7 +1736,7 @@ "post": { "operationId": "createKnowledgeFolder", "summary": "Create Folder", - "description": "Create a folder in the knowledge-base folder tree. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Create a folder in the knowledge-base folder tree. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Knowledge Bases"], "requestBody": { "required": true, @@ -1511,7 +1803,7 @@ "patch": { "operationId": "relocateKnowledgeFolder", "summary": "Rename or Move Folder", - "description": "Rename or move a folder and atomically rewrite descendant paths. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Rename or move a folder and atomically rewrite descendant paths. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Knowledge Bases"], "requestBody": { "required": true, @@ -1599,16 +1891,30 @@ "description": "Path of the folder to delete.", "schema": { "description": "Path of the folder to delete.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, { "name": "recursive", "in": "query", "required": false, - "description": "Delete nested files and folders when true.", + "description": "Delete the folder's nested files and folders too. An empty folder deletes either way; a non-empty one needs this. The listed spellings are the whole accepted vocabulary and are case-sensitive; any other value is rejected.", "schema": { - "description": "Delete nested files and folders when true.", + "description": "Delete the folder's nested files and folders too. An empty folder deletes either way; a non-empty one needs this. The listed spellings are the whole accepted vocabulary and are case-sensitive; any other value is rejected.", + "enum": [ + "true", + "1", + "yes", + "on", + "y", + "enabled", + "false", + "0", + "no", + "off", + "n", + "disabled" + ], "default": "false", "type": "string" } @@ -1673,7 +1979,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings, then API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -1708,13 +2014,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -1729,7 +2035,7 @@ }, "responses": { "BadRequest": { - "description": "The request is invalid.", + "description": "The request is invalid. This includes a query parameter sent with no value (`?limit=`, `?search=`), which is rejected rather than read as zero, empty, or the parameter default — omit the parameter instead.", "content": { "application/json": { "schema": { @@ -1759,7 +2065,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. When the cause is one a caller can act on, `error.details.code` names it. A resource in a workspace the caller cannot reach at all answers `404` instead, so absence and denial are indistinguishable.", "content": { "application/json": { "schema": { @@ -1789,7 +2095,7 @@ } }, "RunIdConflict": { - "description": "The run cannot be started. Two causes share this status, distinguished by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already associated with a different request, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has already reached the maximum workflow-to-workflow call depth.", + "description": "The run cannot be started, for one of two causes named by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already claimed, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has reached the maximum workflow-to-workflow call depth.", "headers": { "X-Run-Id": { "$ref": "#/components/headers/X-Run-Id" @@ -1803,18 +2109,8 @@ } } }, - "Gone": { - "description": "The requested generated resource has expired.", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/V2Error" - } - } - } - }, "PayloadTooLarge": { - "description": "The request, or a resource collection it must materialize, exceeds the allowed size. Besides an oversized request body, this covers a generated artifact that renders past the download ceiling and a workspace folder tree too large to load in full.", + "description": "The request, or a resource collection it must materialize, exceeds the allowed size: an oversized request body, a generated artifact past the download ceiling, or a workspace folder tree too large to load in full.", "content": { "application/json": { "schema": { @@ -1859,7 +2155,7 @@ } }, "ClientClosedRequest": { - "description": "The client closed the connection before the response was produced.", + "description": "The client closed the connection before the response was produced. An abort can leave the run going, so `error.details.runId` carries the run id — reconcile against the runs resource rather than starting another run.", "content": { "application/json": { "schema": { @@ -1879,7 +2175,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. `Retry-After` carries the seconds to wait; treat it as a floor and add jitter. The header is omitted when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, because the run may already have started — reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -1905,7 +2206,7 @@ "description": "Human-readable explanation of the error." }, "details": { - "description": "Optional structured error details." + "description": "Structured error details. On a `403` whose cause a caller can act on, this carries a `code` from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `SECRET_ADMIN_ACCESS_REQUIRED` — The caller can write in the workspace but is not an admin of this secret. Ask a workspace admin, or someone holding admin on the secret, to grant access or set the value.\n- `WORKSPACE_RESOURCE_LIMIT_REACHED` — The workspace already holds the maximum number of resources of this kind. Delete one, or contact Sim to raise the limit; the message names the ceiling.\n- `PUBLIC_SHARING_NOT_ALLOWED` — The workspace's organization does not permit sharing this resource publicly. An organization admin controls the policy.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address." } }, "required": ["code", "message"], @@ -1926,6 +2227,12 @@ } ] }, + "FolderPathInput": { + "title": "Folder path input", + "description": "Folder path. A missing leading slash is normalized before validation. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096, + "type": "string" + }, "V2KnowledgeBase": { "type": "object", "properties": { @@ -2003,7 +2310,9 @@ }, "folderPath": { "type": "string", + "title": "Folder path", "description": "Canonical containing-folder path; `/` is the workspace root.", + "maxLength": 4096, "examples": ["/Product"] } }, @@ -2102,7 +2411,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -2186,7 +2495,7 @@ }, "folderPath": { "description": "Containing folder path; omission creates the knowledge base at the root.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["workspaceId", "name"], @@ -2221,7 +2530,7 @@ }, "folderPath": { "description": "New containing-folder path.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["workspaceId"], @@ -2264,6 +2573,11 @@ "V2KnowledgeSearchResult": { "type": "object", "properties": { + "knowledgeBaseId": { + "type": "string", + "description": "Knowledge base the matching chunk came from; a search may span up to 20.", + "examples": ["7c9e6679-7425-40de-944b-e07fc1f90ae7"] + }, "documentId": { "type": "string", "description": "Identifier of the document containing the matching chunk.", @@ -2324,9 +2638,15 @@ "type": "number", "description": "Similarity score for vector search; tag-only matches use 1.", "examples": [0.8423] + }, + "rerankerScore": { + "description": "Relevance score assigned by the reranker, present only on results a reranker ordered. Results are ordered by this score when it is present, which is why it can disagree with `similarity`.", + "examples": [0.9312], + "type": "number" } }, "required": [ + "knowledgeBaseId", "documentId", "documentName", "sourceUrl", @@ -2375,9 +2695,22 @@ "maximum": 9007199254740991, "description": "Number of results returned.", "examples": [4] + }, + "rerankerStatus": { + "type": "string", + "enum": ["not_requested", "skipped", "unavailable", "applied"], + "description": "What the reranker did on this search. `applied` means it ordered the results, which carry `rerankerScore`. `unavailable` means it was attempted but could not complete, so results are in vector order with no `rerankerScore` — the search still succeeded, and is worth retrying. `skipped` means there was nothing to rank. `not_requested` means `rerankerEnabled` was absent or false.", + "examples": ["applied"] } }, - "required": ["results", "query", "knowledgeBaseIds", "topK", "totalResults"], + "required": [ + "results", + "query", + "knowledgeBaseIds", + "topK", + "totalResults", + "rerankerStatus" + ], "additionalProperties": false, "title": "Knowledge search data", "description": "Results and execution context for a knowledge search." @@ -2410,9 +2743,22 @@ }, "operator": { "default": "eq", - "description": "Comparison operator; valid operators depend on the field type.", + "description": "Comparison operator; valid operators depend on the field type. Text tags accept eq, neq, contains, not_contains, starts_with, ends_with; number and date tags accept eq, neq, gt, gte, lt, lte, between; boolean tags accept eq, neq. An operator the tag's field type does not implement is rejected, never ignored.", "examples": ["eq"], - "type": "string" + "type": "string", + "enum": [ + "eq", + "neq", + "contains", + "not_contains", + "starts_with", + "ends_with", + "gt", + "gte", + "lt", + "lte", + "between" + ] }, "value": { "anyOf": [ @@ -2430,7 +2776,7 @@ "examples": ["billing"] }, "valueTo": { - "description": "Upper bound for the `between` operator.", + "description": "Upper bound for the `between` operator, and required whenever that operator is used.", "anyOf": [ { "type": "string" @@ -2442,6 +2788,7 @@ } }, "required": ["tagName", "value"], + "additionalProperties": false, "title": "Knowledge search tag filter", "description": "A structured tag filter applied to knowledge search." }, @@ -2473,9 +2820,10 @@ "examples": [["7c9e6679-7425-40de-944b-e07fc1f90ae7"]] }, "query": { - "description": "Natural-language query; required when tag filters are omitted.", + "description": "Natural-language query; required when tag filters are omitted. At most 32768 characters — longer text exceeds the embedding model's per-input token ceiling and would be truncated before the billed search ran.", "examples": ["How do I reset my password?"], - "type": "string" + "type": "string", + "maxLength": 32768 }, "topK": { "default": 10, @@ -2485,7 +2833,8 @@ "maximum": 100 }, "tagFilters": { - "description": "Structured tag filters. Supported across multiple knowledge bases, but each filtered tag must resolve to the same slot and field type in every knowledge base selected; a tag missing from one of them, or defined inconsistently across them, is rejected and those knowledge bases must be searched separately. With a single knowledge base, an unknown tag name is simply ignored.", + "description": "Structured tag filters, at most 10 of them. Every filter must hold, including two that name the same tag: repeating one tag narrows the result rather than widening it, matching `GET /api/v2/knowledge/{id}/documents`. To match either of two values for one tag, issue a search per value. Each filtered tag must resolve to the same slot and field type in every knowledge base selected; one missing from any of them, or defined inconsistently across them, is rejected rather than ignored, and those knowledge bases must be searched separately. List the available names with `GET /api/v2/knowledge/{id}/tags`.", + "maxItems": 10, "type": "array", "items": { "$ref": "#/components/schemas/V2KnowledgeSearchTagFilter" @@ -2503,13 +2852,81 @@ "type": "null" } ] + }, + "rerankerEnabled": { + "description": "Re-order retrieved chunks with a reranking model before truncating to `topK`. Ignored for a tag-only search, and billed as an additional search unit. Reranking is best-effort — a provider failure falls back to vector ordering, so check `rerankerStatus` on the response.", + "type": "boolean" + }, + "rerankerModel": { + "default": "rerank-v4.0-fast", + "description": "Reranking model to use when `rerankerEnabled` is true. Defaults to `rerank-v4.0-fast`.", + "type": "string", + "enum": ["rerank-v4.0-pro", "rerank-v4.0-fast", "rerank-v3.5"] + }, + "rerankerInputCount": { + "description": "How many candidate chunks to retrieve before reranking. Defaults to four times `topK`, capped at 100. A larger pool costs more retrieval work but gives the reranker more to choose from.", + "type": "integer", + "minimum": 1, + "maximum": 100 } }, "required": ["workspaceId", "knowledgeBaseIds"], + "additionalProperties": false, "title": "Search knowledge request", "description": "Knowledge bases, query, result limit, retrieval mode, and optional tag filters." }, - "V2KnowledgeDocumentSummary": { + "V2KnowledgeTag": { + "type": "object", + "properties": { + "displayName": { + "type": "string", + "description": "Display name used by tag filters and by tag values on document reads.", + "examples": ["category"] + }, + "tagSlot": { + "type": "string", + "description": "Storage slot the tag occupies. Document writes set tag values by slot (`tag1`..`tag7`).", + "examples": ["tag1"] + }, + "fieldType": { + "type": "string", + "description": "Value type stored in the slot; it determines the valid filter operators.", + "examples": ["text"] + } + }, + "required": ["displayName", "tagSlot", "fieldType"], + "additionalProperties": false, + "title": "Knowledge tag", + "description": "A tag defined on a knowledge base, and the slot it is stored in." + }, + "V2KnowledgeTagListResponse": { + "type": "object", + "properties": { + "data": { + "type": "array", + "items": { + "$ref": "#/components/schemas/V2KnowledgeTag" + }, + "description": "Items in the current page." + }, + "nextCursor": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Always `null` — this list has no `cursor` or `limit` param and returns its whole bounded set in one page. Present so the list can gain pages later without a shape change." + } + }, + "required": ["data", "nextCursor"], + "additionalProperties": false, + "title": "Knowledge tag list response", + "description": "The full tag vocabulary of one knowledge base." + }, + "V2KnowledgeTaggedDocument": { "type": "object", "properties": { "id": { @@ -2575,6 +2992,36 @@ "description": "ISO 8601 timestamp when the document was uploaded, or null.", "format": "date-time", "examples": ["2025-06-18T16:45:00Z"] + }, + "tags": { + "type": "object", + "propertyNames": { + "type": "string" + }, + "additionalProperties": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "description": "Tag value; dates are ISO 8601 strings and an unset tag is null." + }, + "description": "Document tag values keyed by tag display name. Writes address the same tags by slot (`tag1`..`tag7`); resolve names to slots with GET /api/v2/knowledge/{id}/tags.", + "examples": [ + { + "category": "billing", + "priority": 2 + } + ] } }, "required": [ @@ -2588,11 +3035,12 @@ "tokenCount", "characterCount", "enabled", - "createdAt" + "createdAt", + "tags" ], "additionalProperties": false, - "title": "Knowledge document summary", - "description": "Summary returned by document lists and upload acknowledgements." + "title": "Knowledge document list item", + "description": "Document summary with the document tag values keyed by display name." }, "V2KnowledgeDocumentListResponse": { "type": "object", @@ -2600,7 +3048,7 @@ "data": { "type": "array", "items": { - "$ref": "#/components/schemas/V2KnowledgeDocumentSummary" + "$ref": "#/components/schemas/V2KnowledgeTaggedDocument" }, "description": "Items in the current page." }, @@ -2613,7 +3061,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -2621,6 +3069,178 @@ "title": "Knowledge document list response", "description": "A cursor-paginated page of knowledge documents." }, + "V2BulkKnowledgeDocumentsData": { + "type": "object", + "properties": { + "operation": { + "type": "string", + "enum": ["enable", "disable"], + "description": "Operation that was applied." + }, + "updatedCount": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991, + "description": "Number of documents the operation changed.", + "examples": [42] + }, + "documentIds": { + "description": "Identifiers of the documents the operation changed. Present only for an explicit `documentIds` request, which is bounded to 100 documents; a `selectAll` request omits it because the selection is unbounded, and reports `updatedCount` instead.", + "type": "array", + "items": { + "type": "string" + } + } + }, + "required": ["operation", "updatedCount"], + "additionalProperties": false, + "title": "Bulk knowledge document update data", + "description": "Outcome of a bulk enable or disable across knowledge documents." + }, + "V2BulkKnowledgeDocumentsResponse": { + "type": "object", + "properties": { + "data": { + "description": "Response data.", + "$ref": "#/components/schemas/V2BulkKnowledgeDocumentsData" + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Bulk knowledge document response", + "description": "Outcome of a bulk enable or disable." + }, + "BulkUpdateKnowledgeDocumentsRequest": { + "type": "object", + "properties": { + "workspaceId": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the knowledge base." + }, + "operation": { + "type": "string", + "enum": ["enable", "disable"], + "description": "Whether the selected documents become enabled or disabled for search." + }, + "documentIds": { + "description": "Documents to update, by identifier.", + "minItems": 1, + "maxItems": 100, + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "selectAll": { + "description": "Update every document in the knowledge base instead of an explicit list, narrowed by `enabledFilter`.", + "type": "boolean", + "const": true + }, + "enabledFilter": { + "description": "With `selectAll`, restrict the update to documents in this state.", + "type": "string", + "enum": ["all", "enabled", "disabled"] + } + }, + "required": ["workspaceId", "operation"], + "additionalProperties": false, + "title": "Bulk knowledge document request", + "description": "Operation and the documents it applies to.", + "examples": [ + { + "workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64", + "operation": "disable", + "documentIds": ["b2d4f8a0-1c3e-4a5b-9d7c-2e6f0a8b4c12"] + } + ] + }, + "V2KnowledgeDocumentSummary": { + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Unique document identifier.", + "examples": ["b2d4f8a0-1c3e-4a5b-9d7c-2e6f0a8b4c12"] + }, + "knowledgeBaseId": { + "type": "string", + "description": "Knowledge base to which the document belongs.", + "examples": ["7c9e6679-7425-40de-944b-e07fc1f90ae7"] + }, + "filename": { + "type": "string", + "description": "Original filename of the uploaded document.", + "examples": ["getting-started.pdf"] + }, + "fileSize": { + "type": "number", + "description": "File size in bytes.", + "examples": [248913] + }, + "mimeType": { + "type": "string", + "description": "MIME type of the document file.", + "examples": ["application/pdf"] + }, + "processingStatus": { + "type": "string", + "enum": ["pending", "processing", "completed", "failed"], + "description": "Current document processing state.", + "examples": ["completed"] + }, + "chunkCount": { + "type": "number", + "description": "Number of indexed chunks; zero until processing completes.", + "examples": [24] + }, + "tokenCount": { + "type": "number", + "description": "Total tokens extracted from the document.", + "examples": [8123] + }, + "characterCount": { + "type": "number", + "description": "Total characters extracted from the document.", + "examples": [41205] + }, + "enabled": { + "type": "boolean", + "description": "Whether the document is enabled for search.", + "examples": [true] + }, + "createdAt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "ISO 8601 timestamp when the document was uploaded, or null.", + "format": "date-time", + "examples": ["2025-06-18T16:45:00Z"] + } + }, + "required": [ + "id", + "knowledgeBaseId", + "filename", + "fileSize", + "mimeType", + "processingStatus", + "chunkCount", + "tokenCount", + "characterCount", + "enabled", + "createdAt" + ], + "additionalProperties": false, + "title": "Knowledge document summary", + "description": "Summary returned by document lists and upload acknowledgements." + }, "V2KnowledgeDocumentSummaryResponse": { "type": "object", "properties": { @@ -2758,7 +3378,7 @@ "url": { "type": "string", "format": "uri", - "description": "Signed URL to which the file bytes are uploaded." + "description": "Signed URL to which the file bytes are uploaded. Send the bytes with `PUT` to this URL, including exactly the headers in `headers` and nothing that alters the body. Success is `204` with an empty body. A failure is the same `{ \"error\": { \"code\", \"message\" } }` envelope as every other v2 response: `400` when the body does not match the size or content type the session was created for, `403` when the token is invalid, expired, or belongs to another session, and `409` when the session is no longer accepting bytes. The URL is signed and self-describing — construct it from this field only, never by hand." }, "headers": { "type": "object", @@ -2961,7 +3581,7 @@ "url": { "type": "string", "format": "uri", - "description": "Signed URL for this upload part." + "description": "Signed URL for this upload part. Send the bytes with `PUT` to this URL, including exactly the headers in `headers` and nothing that alters the body. Success is `204` with an empty body. A failure is the same `{ \"error\": { \"code\", \"message\" } }` envelope as every other v2 response: `400` when the body does not match the size or content type the session was created for, `403` when the token is invalid, expired, or belongs to another session, and `409` when the session is no longer accepting bytes. The URL is signed and self-describing — construct it from this field only, never by hand." }, "headers": { "type": "object", @@ -3107,6 +3727,36 @@ "format": "date-time", "examples": ["2025-06-18T16:45:00Z"] }, + "tags": { + "type": "object", + "propertyNames": { + "type": "string" + }, + "additionalProperties": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "number" + }, + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "description": "Tag value; dates are ISO 8601 strings and an unset tag is null." + }, + "description": "Document tag values keyed by tag display name. Writes address the same tags by slot (`tag1`..`tag7`); resolve names to slots with GET /api/v2/knowledge/{id}/tags.", + "examples": [ + { + "category": "billing", + "priority": 2 + } + ] + }, "processingError": { "anyOf": [ { @@ -3190,6 +3840,7 @@ "characterCount", "enabled", "createdAt", + "tags", "processingError", "processingStartedAt", "processingCompletedAt", @@ -3214,6 +3865,167 @@ "title": "Knowledge document response", "description": "Full knowledge document detail." }, + "V2KnowledgeDocumentProcessing": { + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Identifier of the requeued document." + }, + "queued": { + "type": "boolean", + "const": true, + "description": "Confirms that processing was requeued." + }, + "processingStatus": { + "type": "string", + "description": "Processing state the document was moved to.", + "examples": ["pending"] + }, + "message": { + "type": "string", + "description": "Human-readable outcome of the requeue." + } + }, + "required": ["id", "queued", "processingStatus", "message"], + "additionalProperties": false, + "title": "Knowledge document processing acknowledgement", + "description": "Acknowledgement returned when a document is requeued for processing." + }, + "V2UpdateKnowledgeDocumentResponse": { + "type": "object", + "properties": { + "data": { + "anyOf": [ + { + "$ref": "#/components/schemas/V2KnowledgeTaggedDocument" + }, + { + "$ref": "#/components/schemas/V2KnowledgeDocumentProcessing" + } + ], + "description": "Response data." + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Update knowledge document response", + "description": "The updated document, or the processing requeue acknowledgement." + }, + "UpdateKnowledgeDocumentRequest": { + "type": "object", + "properties": { + "workspaceId": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the knowledge base." + }, + "filename": { + "description": "New filename for the document.", + "examples": ["getting-started-v2.pdf"], + "type": "string", + "minLength": 1, + "maxLength": 255 + }, + "enabled": { + "description": "Whether the document participates in search. Disabling keeps it indexed.", + "type": "boolean" + }, + "tag1": { + "description": "New value for tag slot 1.", + "type": "string", + "maxLength": 1000 + }, + "tag2": { + "description": "New value for tag slot 2.", + "type": "string", + "maxLength": 1000 + }, + "tag3": { + "description": "New value for tag slot 3.", + "type": "string", + "maxLength": 1000 + }, + "tag4": { + "description": "New value for tag slot 4.", + "type": "string", + "maxLength": 1000 + }, + "tag5": { + "description": "New value for tag slot 5.", + "type": "string", + "maxLength": 1000 + }, + "tag6": { + "description": "New value for tag slot 6.", + "type": "string", + "maxLength": 1000 + }, + "tag7": { + "description": "New value for tag slot 7.", + "type": "string", + "maxLength": 1000 + }, + "number1": { + "description": "New value for number tag slot 1.", + "type": "number" + }, + "number2": { + "description": "New value for number tag slot 2.", + "type": "number" + }, + "number3": { + "description": "New value for number tag slot 3.", + "type": "number" + }, + "number4": { + "description": "New value for number tag slot 4.", + "type": "number" + }, + "number5": { + "description": "New value for number tag slot 5.", + "type": "number" + }, + "date1": { + "description": "New value for date tag slot 1, formatted YYYY-MM-DD.", + "type": "string", + "pattern": "^\\d{4}-\\d{2}-\\d{2}$" + }, + "date2": { + "description": "New value for date tag slot 2, formatted YYYY-MM-DD.", + "type": "string", + "pattern": "^\\d{4}-\\d{2}-\\d{2}$" + }, + "boolean1": { + "description": "New value for boolean tag slot 1.", + "type": "boolean" + }, + "boolean2": { + "description": "New value for boolean tag slot 2.", + "type": "boolean" + }, + "boolean3": { + "description": "New value for boolean tag slot 3.", + "type": "boolean" + }, + "retryProcessing": { + "description": "Requeue a failed or stuck document for processing. Send it alone — no other field may accompany it — and it answers with a queue acknowledgement rather than the document.", + "type": "boolean", + "const": true + } + }, + "required": ["workspaceId"], + "additionalProperties": false, + "title": "Update knowledge document request", + "description": "Filename, search state, tag slot values, or a processing retry.", + "examples": [ + { + "workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64", + "enabled": false, + "tag1": "billing" + } + ] + }, "V2Folder": { "type": "object", "properties": { @@ -3223,11 +4035,15 @@ }, "path": { "type": "string", - "description": "Canonical folder path used as the public folder identifier." + "title": "Non-root folder path", + "description": "Canonical folder path used as the public folder identifier.", + "maxLength": 4096 }, "parentPath": { "type": "string", - "description": "Canonical parent path; `/` is the root." + "title": "Folder path", + "description": "Canonical parent path; `/` is the root.", + "maxLength": 4096 }, "createdAt": { "type": "string", @@ -3264,13 +4080,13 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Always `null` — this list has no `cursor` or `limit` param and returns its whole bounded set in one page. Present so the list can gain pages later without a shape change." } }, "required": ["data", "nextCursor"], "additionalProperties": false, "title": "Knowledge folder list response", - "description": "A cursor-paginated page of knowledge-base folders." + "description": "The whole bounded set of knowledge-base folders, in one page." }, "V2KnowledgeFolderResponse": { "type": "object", @@ -3285,6 +4101,12 @@ "title": "Knowledge folder response", "description": "A single knowledge-base folder." }, + "NonRootFolderPathInput": { + "title": "Non-root folder path input", + "description": "Non-root folder path. A missing leading slash is normalized before validation. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096, + "type": "string" + }, "CreateKnowledgeFolderRequest": { "type": "object", "properties": { @@ -3295,7 +4117,7 @@ }, "path": { "description": "Path of the folder to create.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, "required": ["workspaceId", "path"], @@ -3313,11 +4135,11 @@ }, "path": { "description": "Current folder path.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" }, "destinationPath": { "description": "New full path for the folder and its descendants.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, "required": ["workspaceId", "path", "destinationPath"], @@ -3330,7 +4152,9 @@ "properties": { "path": { "type": "string", - "description": "Canonical path of the deleted folder." + "title": "Folder path", + "description": "Canonical path of the deleted folder.", + "maxLength": 4096 }, "deleted": { "type": "boolean", diff --git a/apps/docs/openapi-v2-logs.json b/apps/docs/openapi-v2-logs.json index d4df4c2b0df..0a64c9f581f 100644 --- a/apps/docs/openapi-v2-logs.json +++ b/apps/docs/openapi-v2-logs.json @@ -36,7 +36,7 @@ "get": { "operationId": "listLogs", "summary": "List Logs", - "description": "List workflow execution logs for a workspace with filters, selectable detail, and opaque cursor pagination. This list predates the shared sort convention: it has no `sortBy` (the sort column is fixed to execution start time) and spells the direction `order` rather than `sortOrder`. Trace spans are stored separately from the log row and are pruned on their own retention schedule: `includeTraceSpans=true` on a run whose stored spans have aged out returns `traceSpans: []` rather than an error, so an empty array does not mean the run recorded no spans.", + "description": "List workflow execution logs for a workspace with filters, selectable detail, and opaque cursor pagination. Runs are hard-deleted once they pass the payer's log retention window, so an older run is simply absent rather than reported as removed. The window is 30 days from run start on the free plan, unbounded on Pro and Team, and set per organization on Enterprise with an optional per-workspace override.", "tags": ["Logs"], "parameters": [ { @@ -54,20 +54,20 @@ "name": "workflowIds", "in": "query", "required": false, - "description": "Comma-separated workflow identifiers to include.", + "description": "Comma-separated workflow identifiers to include. An empty entry is rejected.", "schema": { "type": "string", - "description": "Comma-separated workflow identifiers to include." + "description": "Comma-separated workflow identifiers to include. An empty entry is rejected." } }, { "name": "triggers", "in": "query", "required": false, - "description": "Comma-separated trigger types to include.", + "description": "Comma-separated trigger types to include. An empty entry is rejected. Values are matched exactly and are case-sensitive — every recorded trigger is lowercase, so `API` matches nothing while `api` matches. The vocabulary is open: it covers the core trigger types (`manual`, `api`, `schedule`, `chat`, `webhook`, `mcp`, `copilot`, `workflow`, `custom_block`) and the provider id of any webhook trigger (`slack`, `gmail`, `github`, …), so an unrecognized member is not rejected — it selects no runs. The literal value `all` is a sentinel that disables this filter entirely, so a list containing it returns runs of every trigger type; no real trigger type is named `all`.", "schema": { "type": "string", - "description": "Comma-separated trigger types to include." + "description": "Comma-separated trigger types to include. An empty entry is rejected. Values are matched exactly and are case-sensitive — every recorded trigger is lowercase, so `API` matches nothing while `api` matches. The vocabulary is open: it covers the core trigger types (`manual`, `api`, `schedule`, `chat`, `webhook`, `mcp`, `copilot`, `workflow`, `custom_block`) and the provider id of any webhook trigger (`slack`, `gmail`, `github`, …), so an unrecognized member is not rejected — it selects no runs. The literal value `all` is a sentinel that disables this filter entirely, so a list containing it returns runs of every trigger type; no real trigger type is named `all`." } }, { @@ -85,60 +85,72 @@ "name": "startDate", "in": "query", "required": false, - "description": "Only include runs started at or after this ISO 8601 timestamp.", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant.", "schema": { "type": "string", - "description": "Only include runs started at or after this ISO 8601 timestamp." + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant." } }, { "name": "endDate", "in": "query", "required": false, - "description": "Only include runs started at or before this ISO 8601 timestamp.", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant.", "schema": { "type": "string", - "description": "Only include runs started at or before this ISO 8601 timestamp." + "format": "date-time", + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant." } }, { "name": "minDurationMs", "in": "query", "required": false, - "description": "Minimum total execution duration in milliseconds.", + "description": "Minimum total execution duration in milliseconds. Whole milliseconds from 0 to 2147483647; the stored duration is a 32-bit integer, so a fractional or out-of-range bound is rejected.", "schema": { - "type": "number", - "description": "Minimum total execution duration in milliseconds." + "type": "integer", + "minimum": 0, + "maximum": 2147483647, + "description": "Minimum total execution duration in milliseconds. Whole milliseconds from 0 to 2147483647; the stored duration is a 32-bit integer, so a fractional or out-of-range bound is rejected." } }, { "name": "maxDurationMs", "in": "query", "required": false, - "description": "Maximum total execution duration in milliseconds.", + "description": "Maximum total execution duration in milliseconds. Whole milliseconds from 0 to 2147483647; the stored duration is a 32-bit integer, so a fractional or out-of-range bound is rejected.", "schema": { - "type": "number", - "description": "Maximum total execution duration in milliseconds." + "type": "integer", + "minimum": 0, + "maximum": 2147483647, + "description": "Maximum total execution duration in milliseconds. Whole milliseconds from 0 to 2147483647; the stored duration is a 32-bit integer, so a fractional or out-of-range bound is rejected." } }, { "name": "minCost", "in": "query", "required": false, - "description": "Minimum execution cost in USD.", + "description": "Minimum execution cost in USD, from 0 to 1000000. A run is never charged a negative amount, so a negative bound is rejected rather than treated as a filter that matches every run.", "schema": { "type": "number", - "description": "Minimum execution cost in USD." + "minimum": 0, + "maximum": 1000000, + "description": "Minimum execution cost in USD, from 0 to 1000000. A run is never charged a negative amount, so a negative bound is rejected rather than treated as a filter that matches every run." } }, { "name": "maxCost", "in": "query", "required": false, - "description": "Maximum execution cost in USD.", + "description": "Maximum execution cost in USD, from 0 to 1000000. A run is never charged a negative amount, so a negative bound is rejected rather than treated as a filter that matches every run.", "schema": { "type": "number", - "description": "Maximum execution cost in USD." + "minimum": 0, + "maximum": 1000000, + "description": "Maximum execution cost in USD, from 0 to 1000000. A run is never charged a negative amount, so a negative bound is rejected rather than treated as a filter that matches every run." } }, { @@ -155,21 +167,21 @@ "name": "details", "in": "query", "required": false, - "description": "Response detail level.", + "description": "Response detail level. `full` adds the `workflow` summary to every item. `includeTraceSpans=true` and `includeFinalOutput=true` each imply `full`, so either one adds `workflow` even when `details=basic` is sent explicitly.", "schema": { "default": "basic", "type": "string", "enum": ["basic", "full"], - "description": "Response detail level." + "description": "Response detail level. `full` adds the `workflow` summary to every item. `includeTraceSpans=true` and `includeFinalOutput=true` each imply `full`, so either one adds `workflow` even when `details=basic` is sent explicitly." } }, { "name": "includeTraceSpans", "in": "query", "required": false, - "description": "Whether to include block-level trace spans.", + "description": "Whether to include block-level trace spans. Implies `details=full`. Spans are pruned on their own retention schedule, so a run whose spans have aged out returns `traceSpans: []` rather than an error.", "schema": { - "description": "Whether to include block-level trace spans.", + "description": "Whether to include block-level trace spans. Implies `details=full`. Spans are pruned on their own retention schedule, so a run whose spans have aged out returns `traceSpans: []` rather than an error.", "type": "boolean" } }, @@ -177,9 +189,9 @@ "name": "includeFinalOutput", "in": "query", "required": false, - "description": "Whether to include the final workflow output.", + "description": "Whether to include the final workflow output. Implies `details=full`, so the `workflow` summary is present regardless of what `details` is set to.", "schema": { - "description": "Whether to include the final workflow output.", + "description": "Whether to include the final workflow output. Implies `details=full`, so the `workflow` summary is present regardless of what `details` is set to.", "type": "boolean" } }, @@ -187,33 +199,34 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum log entries per page, clamped to 1–1000.", + "description": "Maximum log entries per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", "schema": { - "description": "Maximum log entries per page, clamped to 1–1000.", - "default": 100, - "type": "number" + "description": "Maximum log entries per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", + "type": "integer", + "default": 100 } }, { "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by a previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "type": "string", - "description": "Opaque cursor returned by a previous page." + "minLength": 1 } }, { "name": "order", "in": "query", "required": false, - "description": "Sort order by execution start time.", + "description": "Sort direction by execution start time. This list is sortable only by execution start time, so it takes `order` in place of `sortBy`/`sortOrder`, which it rejects.", "schema": { "default": "desc", + "description": "Sort direction by execution start time. This list is sortable only by execution start time, so it takes `order` in place of `sortBy`/`sortOrder`, which it rejects.", "type": "string", - "enum": ["desc", "asc"], - "description": "Sort order by execution start time." + "enum": ["asc", "desc"] } }, { @@ -224,6 +237,8 @@ "schema": { "type": "string", "minLength": 1, + "maxLength": 128, + "pattern": "^[A-Za-z0-9._:-]+$", "description": "Exact run identifier to match." } }, @@ -231,10 +246,10 @@ "name": "folderPaths", "in": "query", "required": false, - "description": "Comma-separated workflow folder paths to include.", + "description": "Comma-separated workflow folder paths to include. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", "schema": { "type": "string", - "description": "Comma-separated workflow folder paths to include." + "description": "Comma-separated workflow folder paths to include. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error." } } ], @@ -288,18 +303,20 @@ "get": { "operationId": "getLog", "summary": "Get Log", - "description": "Retrieve the diagnostic representation of a run, including its workflow snapshot, trace spans, final output, and cost. The returned `workflowState` snapshot has credential values redacted: OAuth credential references and secret (`password`) sub-block values are null, while `{{VAR}}` environment-variable references are preserved so consecutive snapshots stay diffable. Trace spans are stored separately from the log row and are pruned on their own retention schedule: a run whose stored spans have aged out returns `traceSpans: []` rather than an error, so an empty array does not mean the run recorded no spans.", + "description": "Retrieve the diagnostic representation of a run, including its workflow snapshot, trace spans, final output, and cost. Trace spans are pruned on their own retention schedule, so an empty `traceSpans` array does not mean the run recorded none.", "tags": ["Logs"], "parameters": [ { "name": "runId", "in": "path", "required": true, - "description": "The unique run identifier shared by lifecycle and diagnostic resources.", + "description": "Unique workflow run identifier.", "schema": { "type": "string", "minLength": 1, - "description": "The unique run identifier shared by lifecycle and diagnostic resources." + "maxLength": 128, + "pattern": "^[A-Za-z0-9._:-]+$", + "description": "Unique workflow run identifier." } } ], @@ -356,7 +373,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings, then API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -391,13 +408,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -412,7 +429,7 @@ }, "responses": { "BadRequest": { - "description": "The request is invalid.", + "description": "The request is invalid. This includes a query parameter sent with no value (`?limit=`, `?search=`), which is rejected rather than read as zero, empty, or the parameter default — omit the parameter instead.", "content": { "application/json": { "schema": { @@ -442,7 +459,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. When the cause is one a caller can act on, `error.details.code` names it. A resource in a workspace the caller cannot reach at all answers `404` instead, so absence and denial are indistinguishable.", "content": { "application/json": { "schema": { @@ -472,7 +489,7 @@ } }, "RunIdConflict": { - "description": "The run cannot be started. Two causes share this status, distinguished by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already associated with a different request, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has already reached the maximum workflow-to-workflow call depth.", + "description": "The run cannot be started, for one of two causes named by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already claimed, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has reached the maximum workflow-to-workflow call depth.", "headers": { "X-Run-Id": { "$ref": "#/components/headers/X-Run-Id" @@ -486,18 +503,8 @@ } } }, - "Gone": { - "description": "The requested generated resource has expired.", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/V2Error" - } - } - } - }, "PayloadTooLarge": { - "description": "The request, or a resource collection it must materialize, exceeds the allowed size. Besides an oversized request body, this covers a generated artifact that renders past the download ceiling and a workspace folder tree too large to load in full.", + "description": "The request, or a resource collection it must materialize, exceeds the allowed size: an oversized request body, a generated artifact past the download ceiling, or a workspace folder tree too large to load in full.", "content": { "application/json": { "schema": { @@ -542,7 +549,7 @@ } }, "ClientClosedRequest": { - "description": "The client closed the connection before the response was produced.", + "description": "The client closed the connection before the response was produced. An abort can leave the run going, so `error.details.runId` carries the run id — reconcile against the runs resource rather than starting another run.", "content": { "application/json": { "schema": { @@ -562,7 +569,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. `Retry-After` carries the seconds to wait; treat it as a floor and add jitter. The header is omitted when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, because the run may already have started — reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -588,7 +600,7 @@ "description": "Human-readable explanation of the error." }, "details": { - "description": "Optional structured error details." + "description": "Structured error details. On a `403` whose cause a caller can act on, this carries a `code` from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `SECRET_ADMIN_ACCESS_REQUIRED` — The caller can write in the workspace but is not an admin of this secret. Ask a workspace admin, or someone holding admin on the secret, to grant access or set the value.\n- `WORKSPACE_RESOURCE_LIMIT_REACHED` — The workspace already holds the maximum number of resources of this kind. Delete one, or contact Sim to raise the limit; the message names the ceiling.\n- `PUBLIC_SHARING_NOT_ALLOWED` — The workspace's organization does not permit sharing this resource publicly. An organization admin controls the policy.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address." } }, "required": ["code", "message"], @@ -649,7 +661,7 @@ "failed", "cancelled" ], - "description": "Current execution status. `redacting` is transient while run output is scrubbed. `paused` is reported when a resume attempt did not run to completion and the run is waiting to be resumed again." + "description": "Current execution status, reported as persisted. `redacting` is transient while run output is scrubbed. `paused` is reported only when a resume attempt did not complete; a run held at a human-in-the-loop pause point reads `pending` here, and `paused` on the workflow run resources. Use those when the pause state matters." }, "level": { "type": "string", @@ -975,7 +987,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -1045,7 +1057,7 @@ "failed", "cancelled" ], - "description": "Current execution status. `redacting` is transient while run output is scrubbed. `paused` is reported when a resume attempt did not run to completion and the run is waiting to be resumed again." + "description": "Current execution status, reported as persisted. `redacting` is transient while run output is scrubbed. `paused` is reported only when a resume attempt did not complete; a run held at a human-in-the-loop pause point reads `pending` here, and `paused` on the workflow run resources. Use those when the pause state matters." }, "level": { "type": "string", @@ -1132,13 +1144,15 @@ "anyOf": [ { "type": "string", - "description": "Canonical slash-prefixed folder path. `/` is the workspace root." + "title": "Folder path", + "description": "Canonical slash-prefixed folder path. `/` is the workspace root. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096 }, { "type": "null" } ], - "description": "Workflow folder path, or null when unavailable." + "description": "Canonical folder path of the workflow, in the same form `folderPaths` accepts as a filter: `/` for a workflow at the workspace root. Null only when the path cannot be resolved — the folder has been deleted, or the workflow itself no longer exists." }, "ownerEmail": { "anyOf": [ @@ -1222,7 +1236,7 @@ "type": "null" } ], - "description": "Workflow graph snapshot captured for the run, with credential values redacted: `oauth-input`, `password: true`, and table sub-block values are null; sensitive nested tool parameters and every parameter without authoritative codec metadata are null; and `{{VAR}}` references in non-opaque fields are preserved. Null when no snapshot is retained." + "description": "Workflow graph snapshot captured for the run, or null when none is retained. Credential-bearing values are redacted to null: `oauth-input`, `password: true`, table sub-block values, sensitive nested tool parameters, and any parameter without authoritative codec metadata. `{{VAR}}` references in non-opaque fields are preserved." }, "traceSpans": { "type": "array", diff --git a/apps/docs/openapi-v2-resources.json b/apps/docs/openapi-v2-resources.json index 920147e91ce..51e85c8125b 100644 --- a/apps/docs/openapi-v2-resources.json +++ b/apps/docs/openapi-v2-resources.json @@ -139,10 +139,10 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum members to return. Defaults to 50 and cannot exceed 100.", + "description": "Maximum members to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum members to return. Defaults to 50 and cannot exceed 100.", + "description": "Maximum members to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -152,9 +152,9 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by the preceding page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque cursor returned by the preceding page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "type": "string", "minLength": 1 } @@ -210,7 +210,7 @@ "get": { "operationId": "listMcpServers", "summary": "List MCP Servers", - "description": "List MCP servers registered in a workspace. Request-header values and OAuth client secrets are never returned. The bounded workspace set uses the standard cursor envelope with `nextCursor` always null; there is no second page to fetch.", + "description": "List MCP servers registered in a workspace. Request-header values and OAuth client secrets are never returned. The discovery fields stay at their registration defaults until `GET /api/v2/mcp-servers/{id}/tools` runs a discovery.", "tags": ["MCP Servers"], "parameters": [ { @@ -240,10 +240,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "createdAt", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -259,6 +259,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum MCP servers to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum MCP servers to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "schema": { + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -309,7 +333,7 @@ "post": { "operationId": "createMcpServer", "summary": "Create MCP Server", - "description": "Register an MCP server in a workspace. The endpoint URL determines server identity, must be absolute HTTP or HTTPS, and cannot contain environment-variable references. Header values and OAuth client secrets are write-only. `transport`, `timeout`, `retries`, and `enabled` are applied server-side when omitted; the effective values are in the response.", + "description": "Register an MCP server in a workspace. The endpoint URL is the server identity, so a URL already registered here is a `409` — reconfigure that server with `PATCH /api/v2/mcp-servers/{id}` instead. Registration never connects to the endpoint: the server comes back `disconnected` and stays unavailable until `GET /api/v2/mcp-servers/{id}/tools` succeeds.", "tags": ["MCP Servers"], "requestBody": { "required": true, @@ -359,6 +383,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -382,11 +409,11 @@ "name": "id", "in": "path", "required": true, - "description": "MCP server to retrieve, update, or delete.", + "description": "Unique MCP server identifier.", "schema": { "type": "string", "minLength": 1, - "description": "MCP server to retrieve, update, or delete." + "description": "Unique MCP server identifier." } }, { @@ -449,18 +476,18 @@ "patch": { "operationId": "updateMcpServer", "summary": "Update MCP Server", - "description": "Update the supplied MCP server fields. The URL is immutable because it determines server identity; delete and recreate the server to change endpoints. Two fields do not follow the omitted-fields-are-retained rule. `headers` is replaced wholesale rather than merged: sending it drops every stored header it does not repeat, and the only way to keep a header is to resend it. Changing `oauthClientId`, or sending `oauthClientSecret` as null or a new value, revokes the stored OAuth grant and forces reauthorization; switching away from OAuth authentication revokes it too.", + "description": "Update the supplied MCP server fields. Omitted fields are retained, except where a field says otherwise. Any change that invalidates authentication revokes the stored OAuth grant, resets `connectionStatus` to `disconnected`, and clears `lastConnected` and `lastError`, so the server must be rediscovered.", "tags": ["MCP Servers"], "parameters": [ { "name": "id", "in": "path", "required": true, - "description": "MCP server to retrieve, update, or delete.", + "description": "Unique MCP server identifier.", "schema": { "type": "string", "minLength": 1, - "description": "MCP server to retrieve, update, or delete." + "description": "Unique MCP server identifier." } } ], @@ -509,6 +536,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -530,11 +560,11 @@ "name": "id", "in": "path", "required": true, - "description": "MCP server to retrieve, update, or delete.", + "description": "Unique MCP server identifier.", "schema": { "type": "string", "minLength": 1, - "description": "MCP server to retrieve, update, or delete." + "description": "Unique MCP server identifier." } }, { @@ -595,11 +625,100 @@ } } }, + "/api/v2/mcp-servers/{id}/tools": { + "get": { + "operationId": "listMcpServerTools", + "summary": "List MCP Server Tools", + "description": "Connect to a registered MCP server and return the tools it exposes. This read has side effects: it opens a live connection to the third-party server and writes `connectionStatus`, `toolCount`, `lastError`, and `lastToolsRefresh`. A `HEAD` skips the effect but is authorized exactly as the `GET` is, so it answers `400`, `401`, `403`, or `404` wherever the `GET` would and an empty `200` otherwise. Skipping the effect means skipping the read that produces the payload, so that `200` carries none of the response headers documented below — it answers whether the `GET` would be allowed, not what the `GET` would return. Discovery is bounded at 1,000 tools and 5 MB of tool payload per server. The bounded set is returned in one page; `nextCursor` is always null. An unreachable, slow, or cooling-down server is a `503`; a stored OAuth grant that no longer works is a `409` with `error.details.code` `MCP_SERVER_REAUTHORIZATION_REQUIRED`, which only a human reauthorizing in Sim can clear. A workspace API key is rejected with `403`; use a personal API key.", + "tags": ["MCP Servers"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "Unique MCP server identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique MCP server identifier." + } + }, + { + "name": "workspaceId", + "in": "query", + "required": true, + "description": "Workspace that owns the MCP server.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Workspace that owns the MCP server." + } + }, + { + "name": "refresh", + "in": "query", + "required": false, + "description": "Bypass the short-lived per-workspace tool cache and reconnect under your own credentials. A cached result reflects whichever workspace member last ran discovery, so this is the only way to pick up a tool added since then; it costs a live round trip.", + "schema": { + "description": "Bypass the short-lived per-workspace tool cache and reconnect under your own credentials. A cached result reflects whichever workspace member last ran discovery, so this is the only way to pick up a tool added since then; it costs a live round trip.", + "type": "boolean" + } + } + ], + "responses": { + "200": { + "description": "Tools exposed by the MCP server.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ListMcpServerToolsResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "409": { + "$ref": "#/components/responses/Conflict" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, "/api/v2/skills": { "get": { "operationId": "listSkills", "summary": "List Skills", - "description": "List workspace and built-in skills. Built-ins are marked read-only. The list omits skill bodies and uses the standard cursor envelope with `nextCursor` always null, so there is no second page to fetch; fetch one skill to read its content.", + "description": "List workspace and built-in skills with opaque cursor pagination. Built-ins are marked read-only. The list omits skill bodies; fetch one skill to read its content.", "tags": ["Skills"], "parameters": [ { @@ -629,10 +748,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "createdAt", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -648,6 +767,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum skills to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum skills to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "schema": { + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -698,7 +841,7 @@ "post": { "operationId": "createSkill", "summary": "Create Skill", - "description": "Create one skill in a workspace. Its kebab-case name must be unique and cannot be reserved by a built-in skill. Note that a workspace API key may create a skill but may not later update or delete it.", + "description": "Create one skill in a workspace. Its kebab-case name must be unique and cannot be reserved by a built-in skill. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Skills"], "requestBody": { "required": true, @@ -748,6 +891,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -771,11 +917,11 @@ "name": "id", "in": "path", "required": true, - "description": "Skill to retrieve, update, or delete. Built-in skills use their name as the id.", + "description": "Unique skill identifier. A built-in skill is `builtin-` followed by its name, for example `builtin-research`.", "schema": { "type": "string", "minLength": 1, - "description": "Skill to retrieve, update, or delete. Built-in skills use their name as the id." + "description": "Unique skill identifier. A built-in skill is `builtin-` followed by its name, for example `builtin-research`." } }, { @@ -838,18 +984,18 @@ "patch": { "operationId": "updateSkill", "summary": "Update Skill", - "description": "Update the supplied fields on a workspace skill. Omitted fields retain their stored values. Built-in skills are read-only. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "description": "Update the supplied fields on a workspace skill. Omitted fields retain their stored values. Built-in skills are read-only. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Skills"], "parameters": [ { "name": "id", "in": "path", "required": true, - "description": "Skill to retrieve, update, or delete. Built-in skills use their name as the id.", + "description": "Unique skill identifier. A built-in skill is `builtin-` followed by its name, for example `builtin-research`.", "schema": { "type": "string", "minLength": 1, - "description": "Skill to retrieve, update, or delete. Built-in skills use their name as the id." + "description": "Unique skill identifier. A built-in skill is `builtin-` followed by its name, for example `builtin-research`." } } ], @@ -901,6 +1047,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -915,18 +1064,18 @@ "delete": { "operationId": "deleteSkill", "summary": "Delete Skill", - "description": "Delete a workspace skill. Built-in skills are read-only and cannot be deleted. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "description": "Delete a workspace skill. Built-in skills are read-only and cannot be deleted. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Skills"], "parameters": [ { "name": "id", "in": "path", "required": true, - "description": "Skill to retrieve, update, or delete. Built-in skills use their name as the id.", + "description": "Unique skill identifier. A built-in skill is `builtin-` followed by its name, for example `builtin-research`.", "schema": { "type": "string", "minLength": 1, - "description": "Skill to retrieve, update, or delete. Built-in skills use their name as the id." + "description": "Unique skill identifier. A built-in skill is `builtin-` followed by its name, for example `builtin-research`." } }, { @@ -991,7 +1140,7 @@ "get": { "operationId": "listCustomTools", "summary": "List Custom Tools", - "description": "List code-backed custom tools defined in a workspace. Legacy personal tools are excluded. The bounded workspace set uses the standard cursor envelope with `nextCursor` always null; there is no second page to fetch.", + "description": "List code-backed custom tools defined in a workspace, with opaque cursor pagination. Legacy personal tools are excluded.", "tags": ["Custom Tools"], "parameters": [ { @@ -1040,6 +1189,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum custom tools to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum custom tools to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "schema": { + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -1140,6 +1313,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1163,11 +1339,11 @@ "name": "id", "in": "path", "required": true, - "description": "Custom tool to retrieve, update, or delete.", + "description": "Unique custom tool identifier.", "schema": { "type": "string", "minLength": 1, - "description": "Custom tool to retrieve, update, or delete." + "description": "Unique custom tool identifier." } }, { @@ -1237,11 +1413,11 @@ "name": "id", "in": "path", "required": true, - "description": "Custom tool to retrieve, update, or delete.", + "description": "Unique custom tool identifier.", "schema": { "type": "string", "minLength": 1, - "description": "Custom tool to retrieve, update, or delete." + "description": "Unique custom tool identifier." } } ], @@ -1293,6 +1469,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1314,11 +1493,11 @@ "name": "id", "in": "path", "required": true, - "description": "Custom tool to retrieve, update, or delete.", + "description": "Unique custom tool identifier.", "schema": { "type": "string", "minLength": 1, - "description": "Custom tool to retrieve, update, or delete." + "description": "Unique custom tool identifier." } }, { @@ -1383,7 +1562,7 @@ "get": { "operationId": "listCredentials", "summary": "List Credentials", - "description": "List OAuth and service-account connections visible to the caller. Secret material is never returned. Credential mutations and single-resource reads are intentionally not exposed. The bounded set uses the standard cursor envelope with `nextCursor` always null; there is no second page to fetch.", + "description": "List OAuth and service-account connections visible to the caller. Secret material is never returned. Credential mutations and single-resource reads are not exposed.", "tags": ["Credentials"], "parameters": [ { @@ -1454,6 +1633,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum credentials to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum credentials to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "schema": { + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -1506,7 +1709,7 @@ "get": { "operationId": "listSecrets", "summary": "List Secrets", - "description": "List workspace and caller-owned personal secret metadata. Only names, scope, role, and timestamps are returned; secret values are never read or returned. The bounded set uses the standard cursor envelope with `nextCursor` always null; there is no second page to fetch. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "description": "List workspace and caller-owned personal secret metadata with opaque cursor pagination. Only names, scope, role, and timestamps are returned; secret values are never returned. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Secrets"], "parameters": [ { @@ -1547,10 +1750,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "name", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -1566,6 +1769,30 @@ "type": "string", "enum": ["asc", "desc"] } + }, + { + "name": "limit", + "in": "query", + "required": false, + "description": "Maximum secrets to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "schema": { + "default": 50, + "description": "Maximum secrets to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", + "minimum": 1, + "maximum": 100 + } + }, + { + "name": "cursor", + "in": "query", + "required": false, + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "schema": { + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -1618,7 +1845,7 @@ "put": { "operationId": "setSecret", "summary": "Set Secret", - "description": "Create or replace a workspace or caller-owned personal secret. The value is encrypted at rest, is write-only, and is never included in the response. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "description": "Create or replace a workspace or caller-owned personal secret. The value is encrypted at rest, is write-only, and is never included in the response. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Secrets"], "parameters": [ { @@ -1701,6 +1928,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1715,7 +1945,7 @@ "delete": { "operationId": "deleteSecret", "summary": "Delete Secret", - "description": "Delete a workspace or caller-owned personal secret without reading or returning its stored value. A workspace API key cannot call this operation and is rejected with `403`; use a personal API key.", + "description": "Delete a workspace or caller-owned personal secret without reading or returning its stored value. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Secrets"], "parameters": [ { @@ -1735,22 +1965,22 @@ "name": "workspaceId", "in": "query", "required": true, - "description": "Workspace in which the secret is available.", + "description": "Workspace the request is authorized against. A workspace secret is deleted from it; a personal secret is deleted for the caller in all of their workspaces.", "schema": { "type": "string", "minLength": 1, - "description": "Workspace in which the secret is available." + "description": "Workspace the request is authorized against. A workspace secret is deleted from it; a personal secret is deleted for the caller in all of their workspaces." } }, { "name": "scope", "in": "query", "required": true, - "description": "Whether the secret belongs to the workspace or the caller.", + "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace.", "schema": { "type": "string", "enum": ["workspace", "personal"], - "description": "Whether the secret belongs to the workspace or the caller." + "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace." } } ], @@ -1807,7 +2037,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings, then API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -1842,13 +2072,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -1863,7 +2093,7 @@ }, "responses": { "BadRequest": { - "description": "The request is invalid.", + "description": "The request is invalid. This includes a query parameter sent with no value (`?limit=`, `?search=`), which is rejected rather than read as zero, empty, or the parameter default — omit the parameter instead.", "content": { "application/json": { "schema": { @@ -1893,7 +2123,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. When the cause is one a caller can act on, `error.details.code` names it. A resource in a workspace the caller cannot reach at all answers `404` instead, so absence and denial are indistinguishable.", "content": { "application/json": { "schema": { @@ -1923,7 +2153,7 @@ } }, "RunIdConflict": { - "description": "The run cannot be started. Two causes share this status, distinguished by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already associated with a different request, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has already reached the maximum workflow-to-workflow call depth.", + "description": "The run cannot be started, for one of two causes named by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already claimed, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has reached the maximum workflow-to-workflow call depth.", "headers": { "X-Run-Id": { "$ref": "#/components/headers/X-Run-Id" @@ -1937,18 +2167,8 @@ } } }, - "Gone": { - "description": "The requested generated resource has expired.", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/V2Error" - } - } - } - }, "PayloadTooLarge": { - "description": "The request, or a resource collection it must materialize, exceeds the allowed size. Besides an oversized request body, this covers a generated artifact that renders past the download ceiling and a workspace folder tree too large to load in full.", + "description": "The request, or a resource collection it must materialize, exceeds the allowed size: an oversized request body, a generated artifact past the download ceiling, or a workspace folder tree too large to load in full.", "content": { "application/json": { "schema": { @@ -1993,7 +2213,7 @@ } }, "ClientClosedRequest": { - "description": "The client closed the connection before the response was produced.", + "description": "The client closed the connection before the response was produced. An abort can leave the run going, so `error.details.runId` carries the run id — reconcile against the runs resource rather than starting another run.", "content": { "application/json": { "schema": { @@ -2013,7 +2233,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. `Retry-After` carries the seconds to wait; treat it as a floor and add jitter. The header is omitted when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, because the run may already have started — reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -2039,7 +2264,7 @@ "description": "Human-readable explanation of the error." }, "details": { - "description": "Optional structured error details." + "description": "Structured error details. On a `403` whose cause a caller can act on, this carries a `code` from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `SECRET_ADMIN_ACCESS_REQUIRED` — The caller can write in the workspace but is not an admin of this secret. Ask a workspace admin, or someone holding admin on the secret, to grant access or set the value.\n- `WORKSPACE_RESOURCE_LIMIT_REACHED` — The workspace already holds the maximum number of resources of this kind. Delete one, or contact Sim to raise the limit; the message names the ceiling.\n- `PUBLIC_SHARING_NOT_ALLOWED` — The workspace's organization does not permit sharing this resource publicly. An organization admin controls the policy.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address." } }, "required": ["code", "message"], @@ -2168,7 +2393,7 @@ }, "isExternal": { "type": "boolean", - "description": "Whether the member belongs to a different organization than the workspace. True for an explicitly granted member whose own organization differs from the workspace's; false for the workspace owner and for a member sharing the workspace organization. Inherited organization-administrator access is always reported as false, so this is not a signal that access came from outside the explicit member list." + "description": "Whether the member belongs to a different organization than the workspace. True only for an explicitly granted member whose own organization differs; inherited organization-administrator access is always reported as false, so this does not detect every outside caller." }, "joinedAt": { "type": "string", @@ -2201,7 +2426,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -2267,12 +2492,12 @@ "description": "Whether the server tools are available to workflows." }, "connectionStatus": { - "description": "Result of the most recent connection attempt.", + "description": "Result of the most recent connection attempt. Registration and re-registration establish no connection — the auth-type probe they may send does not count as one — so a server begins, and returns to, `disconnected` until a tool discovery runs.", "type": "string", "enum": ["connected", "disconnected", "error"] }, "lastError": { - "description": "Message from the most recent failed connection, or null when absent.", + "description": "Message from the most recent failed connection, or null when absent. A re-registration clears it, since the configuration it described no longer applies.", "anyOf": [ { "type": "string" @@ -2293,7 +2518,7 @@ "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$" }, "lastConnected": { - "description": "ISO 8601 timestamp of the most recent successful connection.", + "description": "ISO 8601 timestamp of the most recent successful connection. Absent until the server completes one; registering a server does not set it.", "type": "string", "format": "date-time", "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$" @@ -2365,7 +2590,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -2425,11 +2650,9 @@ "timeout": 30000, "retries": 3, "enabled": true, - "connectionStatus": "connected", + "connectionStatus": "disconnected", "lastError": null, - "toolCount": 7, - "lastToolsRefresh": "2026-06-20T14:02:11.000Z", - "lastConnected": "2026-06-20T14:02:11.000Z", + "toolCount": 0, "createdAt": "2026-06-01T09:14:00.000Z", "updatedAt": "2026-06-20T14:02:11.000Z", "hasHeaders": true, @@ -2468,15 +2691,15 @@ "type": "string", "minLength": 1, "maxLength": 2048, - "description": "Absolute HTTP or HTTPS endpoint URL without `{{ENV_VAR}}` references." + "description": "Absolute HTTP or HTTPS endpoint URL without `{{ENV_VAR}}` references. It determines server identity and is immutable: delete and recreate the server to change endpoints." }, "authType": { - "description": "Authentication method. Sim detects it from the server when omitted.", + "description": "Authentication method. When omitted, and no `headers` are sent, registration probes the endpoint once to classify it, falling back to `headers` when the probe fails or the server does not advertise OAuth. A server publishing RFC 9728 metadata is therefore stored as `oauth`, and headers configured afterwards will not authenticate — send this field explicitly to pin the method.", "type": "string", "enum": ["none", "headers", "oauth"] }, "headers": { - "description": "Write-only request headers sent to the server.", + "description": "Write-only request headers sent to the server. Replaced wholesale rather than merged on update: sending this field drops every stored header it does not repeat.", "writeOnly": true, "type": "object", "propertyNames": { @@ -2508,7 +2731,7 @@ "type": "boolean" }, "oauthClientId": { - "description": "Pre-registered OAuth client identifier.", + "description": "Pre-registered OAuth client identifier. Changing it on update revokes the stored OAuth grant and forces reauthorization.", "anyOf": [ { "type": "string", @@ -2520,7 +2743,7 @@ ] }, "oauthClientSecret": { - "description": "Write-only pre-registered OAuth client secret.", + "description": "Write-only pre-registered OAuth client secret. Sending it on update as null or a new value revokes the stored OAuth grant and forces reauthorization, as does switching away from OAuth authentication.", "writeOnly": true, "anyOf": [ { @@ -2657,12 +2880,12 @@ "maxLength": 2048 }, "authType": { - "description": "Authentication method. Sim detects it from the server when omitted.", + "description": "Authentication method. When omitted, and no `headers` are sent, registration probes the endpoint once to classify it, falling back to `headers` when the probe fails or the server does not advertise OAuth. A server publishing RFC 9728 metadata is therefore stored as `oauth`, and headers configured afterwards will not authenticate — send this field explicitly to pin the method.", "type": "string", "enum": ["none", "headers", "oauth"] }, "headers": { - "description": "Write-only request headers sent to the server.", + "description": "Write-only request headers sent to the server. Replaced wholesale rather than merged on update: sending this field drops every stored header it does not repeat.", "writeOnly": true, "type": "object", "propertyNames": { @@ -2694,7 +2917,7 @@ "type": "boolean" }, "oauthClientId": { - "description": "Pre-registered OAuth client identifier.", + "description": "Pre-registered OAuth client identifier. Changing it on update revokes the stored OAuth grant and forces reauthorization.", "anyOf": [ { "type": "string", @@ -2706,7 +2929,7 @@ ] }, "oauthClientSecret": { - "description": "Write-only pre-registered OAuth client secret.", + "description": "Write-only pre-registered OAuth client secret. Sending it on update as null or a new value revokes the stored OAuth grant and forces reauthorization, as does switching away from OAuth authentication.", "writeOnly": true, "anyOf": [ { @@ -2769,12 +2992,120 @@ } ] }, + "V2McpTool": { + "type": "object", + "properties": { + "name": { + "type": "string", + "description": "Tool name, as the MCP server reports it." + }, + "description": { + "description": "Tool description reported by the server.", + "type": "string" + }, + "inputSchema": { + "type": "object", + "properties": { + "type": { + "type": "string", + "const": "object", + "description": "JSON Schema type of the argument object. MCP requires `object`." + }, + "properties": { + "description": "Argument schemas keyed by argument name.", + "type": "object", + "propertyNames": { + "type": "string" + }, + "additionalProperties": { + "description": "Server-defined JSON Schema for one tool argument." + } + }, + "required": { + "description": "Names of the arguments the tool requires.", + "type": "array", + "items": { + "type": "string", + "description": "Name of a required argument." + } + } + }, + "required": ["type"], + "additionalProperties": { + "description": "Additional JSON Schema keyword reported by the server." + }, + "description": "JSON Schema for the tool's arguments, as reported by the server." + }, + "serverId": { + "type": "string", + "description": "Identifier of the MCP server exposing the tool." + }, + "serverName": { + "type": "string", + "description": "Display name of the MCP server exposing the tool." + } + }, + "required": ["name", "inputSchema", "serverId", "serverName"], + "additionalProperties": false, + "title": "MCP tool", + "description": "A tool exposed by a registered MCP server." + }, + "ListMcpServerToolsResponse": { + "type": "object", + "properties": { + "data": { + "type": "array", + "items": { + "$ref": "#/components/schemas/V2McpTool" + }, + "description": "Items in the current page." + }, + "nextCursor": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Always `null` — this list has no `cursor` or `limit` param and returns its whole bounded set in one page. Present so the list can gain pages later without a shape change." + } + }, + "required": ["data", "nextCursor"], + "additionalProperties": false, + "title": "List MCP server tools response", + "description": "Tools exposed by the MCP server.", + "examples": [ + { + "data": [ + { + "name": "search_docs", + "description": "Search the internal documentation", + "inputSchema": { + "type": "object", + "properties": { + "query": { + "type": "string", + "description": "Search terms" + } + }, + "required": ["query"] + }, + "serverId": "mcp-3f7a9c21", + "serverName": "Docs server" + } + ], + "nextCursor": null + } + ] + }, "V2SkillSummary": { "type": "object", "properties": { "id": { "type": "string", - "description": "Unique skill identifier. Built-in skills use their name as the id." + "description": "Unique skill identifier. A built-in skill is `builtin-` followed by its name, for example `builtin-research`." }, "name": { "type": "string", @@ -2825,7 +3156,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -2853,7 +3184,7 @@ "properties": { "id": { "type": "string", - "description": "Unique skill identifier. Built-in skills use their name as the id." + "description": "Unique skill identifier. A built-in skill is `builtin-` followed by its name, for example `builtin-research`." }, "name": { "type": "string", @@ -3203,7 +3534,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -3715,7 +4046,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -3755,7 +4086,7 @@ "scope": { "type": "string", "enum": ["workspace", "personal"], - "description": "Whether the secret belongs to the workspace or the caller." + "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace." }, "role": { "type": "string", @@ -3799,7 +4130,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -3851,12 +4182,12 @@ "workspaceId": { "type": "string", "minLength": 1, - "description": "Workspace in which the secret is available." + "description": "Workspace the request is authorized against. A workspace secret is written to it; a personal secret is written to the caller and is available in all of their workspaces." }, "scope": { "type": "string", "enum": ["workspace", "personal"], - "description": "Whether the secret belongs to the workspace or the caller." + "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace." }, "value": { "type": "string", @@ -3891,7 +4222,7 @@ "scope": { "type": "string", "enum": ["workspace", "personal"], - "description": "Whether the secret belongs to the workspace or the caller." + "description": "Whether the secret belongs to the workspace or to the caller. A personal secret belongs to the caller across every workspace, not to one workspace." }, "deleted": { "type": "boolean", diff --git a/apps/docs/openapi-v2-tables.json b/apps/docs/openapi-v2-tables.json index 20c7ec7bb62..b1bff5d7222 100644 --- a/apps/docs/openapi-v2-tables.json +++ b/apps/docs/openapi-v2-tables.json @@ -2,7 +2,7 @@ "openapi": "3.1.0", "info": { "title": "Sim Tables API v2", - "description": "Manage tables, typed columns, rows, saved views, workflow groups, folders, imports, and exports through the public v2 API. Row data is keyed by column name.", + "description": "Version 2 of the Sim REST API for tables, typed columns, rows, saved views, workflow groups, folders, imports, and exports. Row data is keyed by column name.", "version": "2.0.0", "contact": { "name": "Sim Support", @@ -36,7 +36,7 @@ "get": { "operationId": "listTables", "summary": "List Tables", - "description": "List tables in a workspace with optional folder filtering, search, sorting, and an opaque cursor envelope. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "List tables in a workspace with optional folder filtering, search, sorting, and an opaque cursor envelope. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Tables"], "parameters": [ { @@ -54,19 +54,19 @@ "name": "folderPath", "in": "query", "required": false, - "description": "Restrict results to tables in this folder.", + "description": "Restrict results to tables in this folder. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", "schema": { - "description": "Restrict results to tables in this folder.", - "type": "string" + "description": "Restrict results to tables in this folder. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", + "$ref": "#/components/schemas/FolderPathInput" } }, { "name": "search", "in": "query", "required": false, - "description": "Case-insensitive substring search on the resource name.", + "description": "Case-insensitive substring match against the resource name.", "schema": { - "description": "Case-insensitive substring search on the resource name.", + "description": "Case-insensitive substring match against the resource name.", "type": "string", "minLength": 1, "maxLength": 200 @@ -76,10 +76,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "createdAt", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -100,12 +100,10 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum tables to return (1-1000). Fractional or out-of-range values are truncated and clamped into that range rather than rejected.", + "description": "Maximum tables to return per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", "schema": { - "description": "Maximum tables to return (1-1000). Fractional or out-of-range values are truncated and clamped into that range rather than rejected.", + "description": "Maximum tables to return per page. Values outside 1–1000 are truncated and clamped into that range rather than rejected. Defaults to 100.", "type": "integer", - "minimum": 1, - "maximum": 1000, "default": 100 } }, @@ -113,9 +111,9 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor from the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque cursor from the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "type": "string", "minLength": 1 } @@ -241,7 +239,7 @@ "get": { "operationId": "getTable", "summary": "Get Table", - "description": "Retrieve a table with its metadata, column schema, locks, and current job. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Retrieve a table with its metadata, column schema, locks, and current job. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Tables"], "parameters": [ { @@ -395,7 +393,7 @@ "patch": { "operationId": "updateTable", "summary": "Update Table", - "description": "Rename a table, edit its description, or move it to a canonical folder. At least one mutable field is required; lock flags remain read-only.\n\nThis operation is NOT atomic. The name, description, and folder changes are written independently in that order, so a failure part-way through leaves the earlier writes committed — a 4xx does NOT mean nothing changed. When at least one field landed before the failure, the error body carries `details.applied`: the list of fields (`name`, `description`, `folderPath`) that were successfully written. Re-read the table, or retry with only the fields missing from `details.applied`.\n\nA workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Rename a table, edit its description, or move it to a canonical folder. At least one mutable field is required; lock flags remain read-only.\n\nNOT atomic: name, description, and folder are written independently, so a 4xx does not mean nothing changed. The error body carries `details.applied` naming the fields that landed — retry with only the ones missing from it.\n\nA workspace folder tree over 10,000 folders is a `413`.", "tags": ["Tables"], "parameters": [ { @@ -504,7 +502,7 @@ } }, "responses": { - "200": { + "201": { "description": "The updated table columns.", "headers": { "X-RateLimit-Limit": { @@ -537,6 +535,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -614,6 +615,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -691,6 +695,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -739,10 +746,10 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum rows to return in the current page.", + "description": "Maximum rows to return per page. Must be a whole number from 1 to 1000. Defaults to 100.", "schema": { "default": 100, - "description": "Maximum rows to return in the current page.", + "description": "Maximum rows to return per page. Must be a whole number from 1 to 1000. Defaults to 100.", "type": "integer", "minimum": 1, "maximum": 1000 @@ -752,9 +759,9 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque cursor returned by the previous page.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "type": "string", "minLength": 1 } @@ -835,7 +842,7 @@ } }, "responses": { - "200": { + "201": { "description": "The inserted row or rows.", "headers": { "X-RateLimit-Limit": { @@ -868,6 +875,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -945,6 +955,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -1022,6 +1035,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -1197,6 +1213,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -1304,7 +1323,7 @@ "post": { "operationId": "upsertTableRow", "summary": "Upsert Row", - "description": "Insert a row or update the existing row that conflicts on a selected unique column.\n\nWARNING — the update branch REPLACES the row, it does not merge. `data` is treated as the complete new row value, so every column you omit is cleared on the matched row. Upserting 2 of 10 columns blanks the other 8. This differs from `PATCH /api/v2/tables/{tableId}/rows/{rowId}`, which merges the patch into the existing row data. Send the full row here, or use PATCH when you only mean to change a subset.", + "description": "Insert a row or update the existing row that conflicts on a selected unique column.\n\nWARNING — the update branch REPLACES the row, it does not merge. `data` is the complete new row value, so every column you omit is cleared on the matched row. Send the full row here, or use `PATCH /api/v2/tables/{tableId}/rows/{rowId}` to change a subset.", "tags": ["Tables"], "parameters": [ { @@ -1364,6 +1383,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -1383,7 +1405,7 @@ "post": { "operationId": "queryTableRows", "summary": "Query Rows", - "description": "Query rows with a typed predicate, ordered sort specification, and opaque cursor pagination. Bounded pages are capped at 5MB by default and may contain fewer rows than the requested limit; continue until nextCursor is null.", + "description": "Query rows with a typed predicate, ordered sort specification, and opaque cursor pagination. Bounded pages are capped at 5MB by default and may contain fewer rows than the requested limit; continue until nextCursor is null. A predicate larger than the request-body ceiling is a `413`.", "tags": ["Tables"], "parameters": [ { @@ -1443,6 +1465,88 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, + "/api/v2/tables/{tableId}/query/count": { + "post": { + "operationId": "countTableRows", + "summary": "Count Rows", + "description": "Count the rows matching a typed predicate across the entire table. The paged reads carry no total, and `rowCount` on the table resource counts every row rather than the matches. Omit the predicate to count the whole table. A predicate larger than the request-body ceiling is a `413`.", + "tags": ["Tables"], + "parameters": [ + { + "name": "tableId", + "in": "path", + "required": true, + "description": "Unique table identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique table identifier." + } + } + ], + "requestBody": { + "required": true, + "description": "Workspace scope and the optional predicate whose matches are counted.", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/CountTableRowsRequest" + } + } + } + }, + "responses": { + "200": { + "description": "The number of matching table rows.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/V2CountTableRowsResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1459,7 +1563,7 @@ "get": { "operationId": "listTableViews", "summary": "List Views", - "description": "List the bounded set of saved table views, with references to removed columns pruned on read. The bounded set is returned in one page with `nextCursor` always null; there is no second page to fetch.", + "description": "List the bounded set of saved table views, with references to removed columns pruned on read. The bounded set is returned in one page; `nextCursor` is always null.", "tags": ["Tables"], "parameters": [ { @@ -1593,6 +1697,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1765,6 +1872,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -1866,7 +1976,7 @@ "get": { "operationId": "listTableWorkflowGroups", "summary": "List Workflow Groups", - "description": "List the workflow and enrichment groups that can be dispatched for a table. The bounded set is returned in one page with `nextCursor` always null; there is no second page to fetch.", + "description": "List the workflow and enrichment groups that can be dispatched for a table. The bounded set is returned in one page; `nextCursor` is always null.", "tags": ["Tables"], "parameters": [ { @@ -2000,6 +2110,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -2017,7 +2130,7 @@ "patch": { "operationId": "updateTableWorkflowGroup", "summary": "Update Workflow Group", - "description": "Restructure a workflow group, its producer, outputs, or execution behavior.\n\nOutput leaf types are resolved against the group’s workflow outside the write lock. If the group is repointed at a different workflow concurrently, that snapshot is invalidated and the request returns `409` — retry the update.", + "description": "Restructure a workflow group, its producer, outputs, or execution behavior. Repointing the group at a different workflow concurrently invalidates the resolved output types and returns `409` — retry the update.", "tags": ["Tables"], "parameters": [ { @@ -2080,6 +2193,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -2157,6 +2273,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "423": { "$ref": "#/components/responses/Locked" }, @@ -2236,6 +2355,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -2334,6 +2456,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -2410,6 +2535,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -2498,7 +2626,7 @@ "get": { "operationId": "getTableImport", "summary": "Get Table Import", - "description": "Read progress and terminal state for a durable table import.", + "description": "Read progress and terminal state for a durable table import.\n\nAn upload-backed import has no durable record until its upload completes, so send the signed upload control token to read it during the `uploading` phase; without the token that phase is a `404`.", "tags": ["Tables"], "parameters": [ { @@ -2522,6 +2650,17 @@ "minLength": 1, "description": "Workspace that owns the transfer resource." } + }, + { + "name": "upload-token", + "in": "header", + "required": false, + "description": "Signed upload control token returned when an upload-backed import was created.", + "schema": { + "description": "Signed upload control token returned when an upload-backed import was created.", + "type": "string", + "minLength": 1 + } } ], "responses": { @@ -2572,7 +2711,7 @@ "delete": { "operationId": "cancelTableImport", "summary": "Cancel Table Import", - "description": "Cancel an upload or processing import without rolling back committed row batches.\n\nCanceling an import that is not in a cancelable state returns `409` naming the current status, and that includes an expired import — `expired` is a terminal import status, not a `410`. An import id that never existed, or one whose retention window already purged the record, returns `404`.", + "description": "Cancel an upload or processing import without rolling back committed row batches.\n\nAn import that is not in a cancelable state, including an `expired` one, is a `409` naming the current status. An unknown or already-purged import id is a `404`.", "tags": ["Tables"], "parameters": [ { @@ -2662,7 +2801,7 @@ "post": { "operationId": "createTableImportPartUrls", "summary": "Create Table Import Part URLs", - "description": "Issue short-lived signed PUT URLs for a bounded set of multipart part numbers.\n\nThe import must still be in the `uploading` state. An import that has moved on — including one that has `expired` — returns `409` naming the current status; a purged or unknown import id returns `404`.", + "description": "Issue short-lived signed PUT URLs for a bounded set of multipart part numbers.\n\nThe import must still be `uploading`; one that has moved on, including an `expired` one, is a `409` naming the current status. An unknown or already-purged import id is a `404`.", "tags": ["Tables"], "parameters": [ { @@ -2747,6 +2886,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -2763,7 +2905,7 @@ "post": { "operationId": "completeTableImportUpload", "summary": "Complete Table Import Upload", - "description": "Verify or assemble the uploaded CSV and begin processing with the same import id.\n\nCompleting an import that is no longer awaiting an upload — including one that has `expired` — returns `409` naming the current status; a purged or unknown import id returns `404`.", + "description": "Verify or assemble the uploaded CSV and begin processing with the same import id.\n\nAn import no longer awaiting an upload, including an `expired` one, is a `409` naming the current status. An unknown or already-purged import id is a `404`.", "tags": ["Tables"], "parameters": [ { @@ -2919,6 +3061,9 @@ "409": { "$ref": "#/components/responses/Conflict" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -3088,7 +3233,7 @@ "get": { "operationId": "downloadTableExport", "summary": "Download Table Export", - "description": "Return a short-lived signed download URL for a completed table export.\n\nThe export must have reached the `completed` status. An export still processing, or one that failed or was canceled, returns `409` naming the current status. An export whose generated file is no longer available — the retention window elapsed, or the object was purged — returns `404` (`Export file is no longer available`), not `410`.", + "description": "Return a short-lived signed download URL for a completed table export.\n\nThe export must have reached `completed`; one still processing, failed, or canceled is a `409` naming the current status. An export whose file is no longer available is a `404`, not a `410`.", "tags": ["Tables"], "parameters": [ { @@ -3227,6 +3372,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "413": { + "$ref": "#/components/responses/PayloadTooLarge" + }, "429": { "$ref": "#/components/responses/RateLimited" }, @@ -3243,7 +3391,7 @@ "get": { "operationId": "listTablesFolders", "summary": "List Folders", - "description": "List table folders, optionally restricting the result to direct children of a canonical parent path. The bounded set is returned in one page with `nextCursor` always null; there is no second page to fetch.", + "description": "List table folders, optionally restricting the result to direct children of a canonical parent path. The bounded set is returned in one page; `nextCursor` is always null.", "tags": ["Tables"], "parameters": [ { @@ -3264,7 +3412,7 @@ "description": "Restrict results to direct children of this parent path.", "schema": { "description": "Restrict results to direct children of this parent path.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, { @@ -3283,10 +3431,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "name", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -3510,16 +3658,30 @@ "description": "Path of the folder to delete.", "schema": { "description": "Path of the folder to delete.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, { "name": "recursive", "in": "query", "required": false, - "description": "Delete nested files and folders when true.", + "description": "Delete the folder's nested files and folders too. An empty folder deletes either way; a non-empty one needs this. The listed spellings are the whole accepted vocabulary and are case-sensitive; any other value is rejected.", "schema": { - "description": "Delete nested files and folders when true.", + "description": "Delete the folder's nested files and folders too. An empty folder deletes either way; a non-empty one needs this. The listed spellings are the whole accepted vocabulary and are case-sensitive; any other value is rejected.", + "enum": [ + "true", + "1", + "yes", + "on", + "y", + "enabled", + "false", + "0", + "no", + "off", + "n", + "disabled" + ], "default": "false", "type": "string" } @@ -3587,7 +3749,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings, then API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -3622,13 +3784,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -3643,7 +3805,7 @@ }, "responses": { "BadRequest": { - "description": "The request is invalid.", + "description": "The request is invalid. This includes a query parameter sent with no value (`?limit=`, `?search=`), which is rejected rather than read as zero, empty, or the parameter default — omit the parameter instead.", "content": { "application/json": { "schema": { @@ -3673,7 +3835,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. When the cause is one a caller can act on, `error.details.code` names it. A resource in a workspace the caller cannot reach at all answers `404` instead, so absence and denial are indistinguishable.", "content": { "application/json": { "schema": { @@ -3703,7 +3865,7 @@ } }, "RunIdConflict": { - "description": "The run cannot be started. Two causes share this status, distinguished by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already associated with a different request, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has already reached the maximum workflow-to-workflow call depth.", + "description": "The run cannot be started, for one of two causes named by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already claimed, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has reached the maximum workflow-to-workflow call depth.", "headers": { "X-Run-Id": { "$ref": "#/components/headers/X-Run-Id" @@ -3717,18 +3879,8 @@ } } }, - "Gone": { - "description": "The requested generated resource has expired.", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/V2Error" - } - } - } - }, "PayloadTooLarge": { - "description": "The request, or a resource collection it must materialize, exceeds the allowed size. Besides an oversized request body, this covers a generated artifact that renders past the download ceiling and a workspace folder tree too large to load in full.", + "description": "The request, or a resource collection it must materialize, exceeds the allowed size: an oversized request body, a generated artifact past the download ceiling, or a workspace folder tree too large to load in full.", "content": { "application/json": { "schema": { @@ -3773,7 +3925,7 @@ } }, "ClientClosedRequest": { - "description": "The client closed the connection before the response was produced.", + "description": "The client closed the connection before the response was produced. An abort can leave the run going, so `error.details.runId` carries the run id — reconcile against the runs resource rather than starting another run.", "content": { "application/json": { "schema": { @@ -3793,7 +3945,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. `Retry-After` carries the seconds to wait; treat it as a floor and add jitter. The header is omitted when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, because the run may already have started — reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -3819,7 +3976,7 @@ "description": "Human-readable explanation of the error." }, "details": { - "description": "Optional structured error details." + "description": "Structured error details. On a `403` whose cause a caller can act on, this carries a `code` from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `SECRET_ADMIN_ACCESS_REQUIRED` — The caller can write in the workspace but is not an admin of this secret. Ask a workspace admin, or someone holding admin on the secret, to grant access or set the value.\n- `WORKSPACE_RESOURCE_LIMIT_REACHED` — The workspace already holds the maximum number of resources of this kind. Delete one, or contact Sim to raise the limit; the message names the ceiling.\n- `PUBLIC_SHARING_NOT_ALLOWED` — The workspace's organization does not permit sharing this resource publicly. An organization admin controls the policy.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address." } }, "required": ["code", "message"], @@ -3840,6 +3997,12 @@ } ] }, + "FolderPathInput": { + "title": "Folder path input", + "description": "Folder path. A missing leading slash is normalized before validation. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096, + "type": "string" + }, "V2ApiTable": { "type": "object", "properties": { @@ -3961,7 +4124,9 @@ }, "folderPath": { "type": "string", - "description": "Canonical slash-prefixed folder path. `/` is the workspace root." + "title": "Folder path", + "description": "Canonical slash-prefixed folder path. `/` is the workspace root. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096 }, "locks": { "type": "object", @@ -4100,7 +4265,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -4167,6 +4332,10 @@ "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], "description": "Column data type." }, + "required": { + "description": "Whether inserts must supply a value for this column.", + "type": "boolean" + }, "unique": { "description": "Whether values in the column must be unique.", "type": "boolean" @@ -4201,10 +4370,6 @@ "description": "ISO 4217 code for currency columns.", "type": "string", "pattern": "^[A-Za-z]{3}$" - }, - "workflowGroupId": { - "description": "Workflow group initially associated with the column.", - "type": "string" } }, "required": ["name", "type"], @@ -4219,7 +4384,7 @@ }, "folderPath": { "description": "Folder in which to create the table.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["name", "workspaceId", "schema"], @@ -4313,8 +4478,7 @@ "description": "Replacement table description, or null to clear it." }, "folderPath": { - "description": "Folder path. A missing leading slash is normalized before validation.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["workspaceId"], @@ -4445,6 +4609,10 @@ "enum": ["string", "number", "currency", "boolean", "date", "json", "select"], "description": "Column data type." }, + "required": { + "description": "Whether inserts must supply a value for this column.", + "type": "boolean" + }, "unique": { "description": "Whether values in the column must be unique.", "type": "boolean" @@ -4536,6 +4704,10 @@ "type": "string", "enum": ["string", "number", "currency", "boolean", "date", "json", "select"] }, + "required": { + "description": "Whether inserts must supply a value for this column.", + "type": "boolean" + }, "unique": { "description": "Whether values in the column must be unique.", "type": "boolean" @@ -4607,6 +4779,7 @@ } }, "required": ["workspaceId", "columnName"], + "additionalProperties": false, "title": "Delete table column request", "description": "Workspace scope and column name to delete.", "examples": [ @@ -4682,7 +4855,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -4768,7 +4941,8 @@ "description": "Rows to insert, with cells keyed by column name." } }, - "required": ["workspaceId", "rows"] + "required": ["workspaceId", "rows"], + "additionalProperties": false }, { "type": "object", @@ -4793,7 +4967,8 @@ "minLength": 1 } }, - "required": ["workspaceId", "data"] + "required": ["workspaceId", "data"], + "additionalProperties": false } ], "title": "Create table rows request", @@ -4841,6 +5016,145 @@ "title": "Update table rows response", "description": "Updated row count and identifiers." }, + "TablePredicate": { + "title": "Table predicate", + "description": "Recursive predicate tree. Each group node is exactly one non-empty `all` or `any` array whose members are further groups or `{ field, op, value }` conditions; the root must be a group, not a bare condition. At most 100 members per group, 10 levels of nesting, and 500 nodes in total. The negating operators include nulls: `ne`, `nin`, `ncontains`, `nlike`, and `nilike` match rows whose column is null or absent, so \"not X\" is not the complement of \"X\" over a nullable column. That holds for every column type, multi-select included. To exclude nulls, `all`-combine the negation with `isNotEmpty` (multi-select) or `isNotNull`. Comparison: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`. Membership: `in`, `nin` (array operand). Emptiness: `isEmpty`, `isNotEmpty`, `isNull`, `isNotNull` (no operand). Substring, always case-insensitive, operand matched literally: `contains`, `ncontains`, `startsWith`, `endsWith`. Pattern: `like`/`nlike` (case-sensitive), `ilike`/`nilike` (case-insensitive). **`*` is the only wildcard** and stands for any run of characters; `%`, `_`, and backslash match themselves. Use `like: \"Hi*\"`, not `like: \"Hi%\"`. A `select` column compares by option id and restricts its operators: single-select accepts `eq`, `ne`, `in`, `nin`; multi-select accepts `contains`, `ncontains`. Option names are accepted as operands and resolved to ids.", + "type": "object", + "oneOf": [ + { + "type": "object", + "description": "Matches a row when every member matches.", + "properties": { + "all": { + "type": "array", + "minItems": 1, + "maxItems": 100, + "description": "Members combined with AND. An empty group is rejected, because it would compile to no filter at all.", + "items": { + "description": "A nested group, or a single condition.", + "anyOf": [ + { + "$ref": "#/components/schemas/TablePredicate" + }, + { + "type": "object", + "title": "Predicate condition", + "description": "One column comparison.", + "properties": { + "field": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "description": "Column name to compare, or one of the system fields `id`, `createdAt`, `updatedAt`." + }, + "op": { + "type": "string", + "enum": [ + "eq", + "ne", + "gt", + "gte", + "lt", + "lte", + "in", + "nin", + "contains", + "ncontains", + "startsWith", + "endsWith", + "like", + "ilike", + "nlike", + "nilike", + "isEmpty", + "isNotEmpty", + "isNull", + "isNotNull" + ], + "description": "Comparison operator. The `TablePredicate` schema description carries the grammar for all of them." + }, + "value": { + "description": "Operand. A scalar for the comparison operators, an array of at most 1000 entries for `in`/`nin`, a pattern for the matching operators, and omitted for `isEmpty`/`isNotEmpty`/`isNull`/`isNotNull`." + } + }, + "required": ["field", "op"], + "additionalProperties": false + } + ] + } + } + }, + "required": ["all"], + "additionalProperties": false + }, + { + "type": "object", + "description": "Matches a row when at least one member matches.", + "properties": { + "any": { + "type": "array", + "minItems": 1, + "maxItems": 100, + "description": "Members combined with OR. An empty group is rejected, because it would compile to no filter at all.", + "items": { + "description": "A nested group, or a single condition.", + "anyOf": [ + { + "$ref": "#/components/schemas/TablePredicate" + }, + { + "type": "object", + "title": "Predicate condition", + "description": "One column comparison.", + "properties": { + "field": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "description": "Column name to compare, or one of the system fields `id`, `createdAt`, `updatedAt`." + }, + "op": { + "type": "string", + "enum": [ + "eq", + "ne", + "gt", + "gte", + "lt", + "lte", + "in", + "nin", + "contains", + "ncontains", + "startsWith", + "endsWith", + "like", + "ilike", + "nlike", + "nilike", + "isEmpty", + "isNotEmpty", + "isNull", + "isNotNull" + ], + "description": "Comparison operator. The `TablePredicate` schema description carries the grammar for all of them." + }, + "value": { + "description": "Operand. A scalar for the comparison operators, an array of at most 1000 entries for `in`/`nin`, a pattern for the matching operators, and omitted for `isEmpty`/`isNotEmpty`/`isNull`/`isNotNull`." + } + }, + "required": ["field", "op"], + "additionalProperties": false + } + ] + } + } + }, + "required": ["any"], + "additionalProperties": false + } + ] + }, "UpdateTableRowsRequest": { "type": "object", "properties": { @@ -4850,7 +5164,7 @@ "description": "Unique workspace identifier." }, "filter": { - "description": "Recursive predicate tree with exactly one non-empty `all` or `any` group at each group node." + "$ref": "#/components/schemas/TablePredicate" }, "data": { "description": "Row-data patch applied to every matching row.", @@ -4864,6 +5178,7 @@ } }, "required": ["workspaceId", "filter", "data"], + "additionalProperties": false, "title": "Update table rows request", "description": "Workspace scope, typed predicate, and row-data patch." }, @@ -4920,7 +5235,7 @@ "description": "Unique workspace identifier." }, "filter": { - "description": "Recursive predicate tree with exactly one non-empty `all` or `any` group at each group node." + "$ref": "#/components/schemas/TablePredicate" }, "limit": { "description": "Maximum matching rows to delete.", @@ -4940,6 +5255,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Delete table rows request", "description": "Workspace scope and exactly one of a predicate or row identifier list.", "examples": [ @@ -4976,6 +5292,7 @@ } }, "required": ["workspaceId", "data"], + "additionalProperties": false, "title": "Update table row request", "description": "Workspace scope and row-data patch keyed by column name.", "examples": [ @@ -5058,7 +5375,7 @@ "description": "Unique workspace identifier." }, "data": { - "description": "Complete set of row cells keyed by column name. On the update branch this REPLACES the matched row: any column not present here is cleared, unlike the merging PATCH /rows/{rowId}.", + "description": "Complete set of row cells keyed by column name. On the update branch this REPLACES the matched row: any column not present here is cleared, unlike the merging `PATCH /api/v2/tables/{tableId}/rows/{rowId}`.", "$ref": "#/components/schemas/V2TableRowData" }, "conflictTarget": { @@ -5068,6 +5385,7 @@ } }, "required": ["workspaceId", "data"], + "additionalProperties": false, "title": "Upsert table row request", "description": "Workspace scope, row data, and optional unique-column conflict target.", "examples": [ @@ -5100,7 +5418,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -5117,7 +5435,7 @@ "description": "Unique workspace identifier." }, "predicate": { - "description": "Recursive predicate tree with exactly one non-empty `all` or `any` group at each group node." + "$ref": "#/components/schemas/TablePredicate" }, "sort": { "description": "Ordered table-row sort specification.", @@ -5138,7 +5456,8 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } }, "limit": { @@ -5154,6 +5473,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Query table rows request", "description": "Workspace scope, optional predicate and sort, and cursor pagination controls.", "examples": [ @@ -5178,6 +5498,65 @@ } ] }, + "V2QueryRowsCountData": { + "type": "object", + "properties": { + "totalCount": { + "type": "integer", + "minimum": 0, + "maximum": 9007199254740991, + "description": "Number of rows matching the predicate across the entire table." + } + }, + "required": ["totalCount"], + "additionalProperties": false, + "title": "Query rows count data", + "description": "Total number of table rows matching a predicate." + }, + "V2CountTableRowsResponse": { + "type": "object", + "properties": { + "data": { + "description": "Response data.", + "$ref": "#/components/schemas/V2QueryRowsCountData" + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Count table rows response", + "description": "The total number of table rows matching the predicate." + }, + "CountTableRowsRequest": { + "type": "object", + "properties": { + "workspaceId": { + "type": "string", + "minLength": 1, + "description": "Unique workspace identifier." + }, + "predicate": { + "$ref": "#/components/schemas/TablePredicate" + } + }, + "required": ["workspaceId"], + "additionalProperties": false, + "title": "Count table rows request", + "description": "Workspace scope and the optional predicate whose matches are counted.", + "examples": [ + { + "workspaceId": "a91c4b2e-6d3f-4e8a-b5c7-0d9e2f1a8c64", + "predicate": { + "all": [ + { + "field": "status", + "op": "eq", + "value": "active" + } + ] + } + } + ] + }, "V2ApiTableView": { "type": "object", "properties": { @@ -5245,7 +5624,7 @@ "type": "object", "properties": { "columnWidths": { - "description": "Column widths keyed by stable column identifier.", + "description": "Column widths keyed by column name.", "type": "object", "propertyNames": { "type": "string" @@ -5256,21 +5635,21 @@ } }, "columnOrder": { - "description": "Stable column identifiers in display order.", + "description": "Column names in display order.", "type": "array", "items": { "type": "string" } }, "pinnedColumns": { - "description": "Stable identifiers of pinned columns.", + "description": "Names of pinned columns.", "type": "array", "items": { "type": "string" } }, "hiddenColumns": { - "description": "Stable identifiers of hidden columns.", + "description": "Names of hidden columns.", "type": "array", "items": { "type": "string" @@ -5341,7 +5720,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Always `null` — this list has no `cursor` or `limit` param and returns its whole bounded set in one page. Present so the list can gain pages later without a shape change." } }, "required": ["data", "nextCursor"], @@ -5362,6 +5741,188 @@ "title": "Create table view response", "description": "The created saved view." }, + "TablePredicateInput": { + "title": "Table predicate input", + "description": "A single `{ field, op, value }` condition or a group, normalized to a grouped predicate after validation. Same grammar and limits as `TablePredicate`.", + "oneOf": [ + { + "type": "object", + "description": "Matches a row when every member matches.", + "properties": { + "all": { + "type": "array", + "minItems": 1, + "maxItems": 100, + "description": "Members combined with AND. An empty group is rejected, because it would compile to no filter at all.", + "items": { + "description": "A nested group, or a single condition.", + "anyOf": [ + { + "$ref": "#/components/schemas/TablePredicateInput" + }, + { + "type": "object", + "title": "Predicate condition", + "description": "One column comparison.", + "properties": { + "field": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "description": "Column name to compare, or one of the system fields `id`, `createdAt`, `updatedAt`." + }, + "op": { + "type": "string", + "enum": [ + "eq", + "ne", + "gt", + "gte", + "lt", + "lte", + "in", + "nin", + "contains", + "ncontains", + "startsWith", + "endsWith", + "like", + "ilike", + "nlike", + "nilike", + "isEmpty", + "isNotEmpty", + "isNull", + "isNotNull" + ], + "description": "Comparison operator. The `TablePredicate` schema description carries the grammar for all of them." + }, + "value": { + "description": "Operand. A scalar for the comparison operators, an array of at most 1000 entries for `in`/`nin`, a pattern for the matching operators, and omitted for `isEmpty`/`isNotEmpty`/`isNull`/`isNotNull`." + } + }, + "required": ["field", "op"], + "additionalProperties": false + } + ] + } + } + }, + "required": ["all"], + "additionalProperties": false + }, + { + "type": "object", + "description": "Matches a row when at least one member matches.", + "properties": { + "any": { + "type": "array", + "minItems": 1, + "maxItems": 100, + "description": "Members combined with OR. An empty group is rejected, because it would compile to no filter at all.", + "items": { + "description": "A nested group, or a single condition.", + "anyOf": [ + { + "$ref": "#/components/schemas/TablePredicateInput" + }, + { + "type": "object", + "title": "Predicate condition", + "description": "One column comparison.", + "properties": { + "field": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "description": "Column name to compare, or one of the system fields `id`, `createdAt`, `updatedAt`." + }, + "op": { + "type": "string", + "enum": [ + "eq", + "ne", + "gt", + "gte", + "lt", + "lte", + "in", + "nin", + "contains", + "ncontains", + "startsWith", + "endsWith", + "like", + "ilike", + "nlike", + "nilike", + "isEmpty", + "isNotEmpty", + "isNull", + "isNotNull" + ], + "description": "Comparison operator. The `TablePredicate` schema description carries the grammar for all of them." + }, + "value": { + "description": "Operand. A scalar for the comparison operators, an array of at most 1000 entries for `in`/`nin`, a pattern for the matching operators, and omitted for `isEmpty`/`isNotEmpty`/`isNull`/`isNotNull`." + } + }, + "required": ["field", "op"], + "additionalProperties": false + } + ] + } + } + }, + "required": ["any"], + "additionalProperties": false + }, + { + "type": "object", + "title": "Predicate condition", + "description": "One column comparison.", + "properties": { + "field": { + "type": "string", + "minLength": 1, + "maxLength": 128, + "description": "Column name to compare, or one of the system fields `id`, `createdAt`, `updatedAt`." + }, + "op": { + "type": "string", + "enum": [ + "eq", + "ne", + "gt", + "gte", + "lt", + "lte", + "in", + "nin", + "contains", + "ncontains", + "startsWith", + "endsWith", + "like", + "ilike", + "nlike", + "nilike", + "isEmpty", + "isNotEmpty", + "isNull", + "isNotNull" + ], + "description": "Comparison operator. The `TablePredicate` schema description carries the grammar for all of them." + }, + "value": { + "description": "Operand. A scalar for the comparison operators, an array of at most 1000 entries for `in`/`nin`, a pattern for the matching operators, and omitted for `isEmpty`/`isNotEmpty`/`isNull`/`isNotNull`." + } + }, + "required": ["field", "op"], + "additionalProperties": false + } + ] + }, "CreateTableViewRequest": { "type": "object", "properties": { @@ -5379,7 +5940,7 @@ "type": "object", "properties": { "columnWidths": { - "description": "Column widths keyed by stable column identifier.", + "description": "Column widths keyed by column name or stable column identifier.", "type": "object", "propertyNames": { "type": "string" @@ -5390,21 +5951,21 @@ } }, "columnOrder": { - "description": "Stable column identifiers in display order.", + "description": "Columns in display order, by name or stable identifier.", "type": "array", "items": { "type": "string" } }, "pinnedColumns": { - "description": "Stable identifiers of pinned columns.", + "description": "Pinned columns, by name or stable identifier.", "type": "array", "items": { "type": "string" } }, "hiddenColumns": { - "description": "Stable identifiers of hidden columns.", + "description": "Hidden columns, by name or stable identifier.", "type": "array", "items": { "type": "string" @@ -5414,7 +5975,7 @@ "description": "Saved row predicate, or null when the view is unfiltered.", "anyOf": [ { - "description": "Recursive predicate condition or group, normalized to a grouped predicate after validation." + "$ref": "#/components/schemas/TablePredicateInput" }, { "type": "null" @@ -5442,7 +6003,8 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } }, { @@ -5451,10 +6013,12 @@ ] } }, + "additionalProperties": false, "description": "Saved filter, sort, and column-layout configuration." } }, "required": ["workspaceId", "name", "config"], + "additionalProperties": false, "title": "Create table view request", "description": "Workspace scope, name, and saved filter, sort, and layout configuration." }, @@ -5489,7 +6053,7 @@ "type": "object", "properties": { "columnWidths": { - "description": "Column widths keyed by stable column identifier.", + "description": "Column widths keyed by column name or stable column identifier.", "type": "object", "propertyNames": { "type": "string" @@ -5500,21 +6064,21 @@ } }, "columnOrder": { - "description": "Stable column identifiers in display order.", + "description": "Columns in display order, by name or stable identifier.", "type": "array", "items": { "type": "string" } }, "pinnedColumns": { - "description": "Stable identifiers of pinned columns.", + "description": "Pinned columns, by name or stable identifier.", "type": "array", "items": { "type": "string" } }, "hiddenColumns": { - "description": "Stable identifiers of hidden columns.", + "description": "Hidden columns, by name or stable identifier.", "type": "array", "items": { "type": "string" @@ -5524,7 +6088,7 @@ "description": "Saved row predicate, or null when the view is unfiltered.", "anyOf": [ { - "description": "Recursive predicate condition or group, normalized to a grouped predicate after validation." + "$ref": "#/components/schemas/TablePredicateInput" }, { "type": "null" @@ -5552,7 +6116,8 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } }, { @@ -5560,14 +6125,15 @@ } ] } - } + }, + "additionalProperties": false }, "configPatch": { "description": "Saved-view configuration fields to shallow-merge.", "type": "object", "properties": { "columnWidths": { - "description": "Column widths keyed by stable column identifier.", + "description": "Column widths keyed by column name or stable column identifier.", "type": "object", "propertyNames": { "type": "string" @@ -5578,21 +6144,21 @@ } }, "columnOrder": { - "description": "Stable column identifiers in display order.", + "description": "Columns in display order, by name or stable identifier.", "type": "array", "items": { "type": "string" } }, "pinnedColumns": { - "description": "Stable identifiers of pinned columns.", + "description": "Pinned columns, by name or stable identifier.", "type": "array", "items": { "type": "string" } }, "hiddenColumns": { - "description": "Stable identifiers of hidden columns.", + "description": "Hidden columns, by name or stable identifier.", "type": "array", "items": { "type": "string" @@ -5602,7 +6168,7 @@ "description": "Saved row predicate, or null when the view is unfiltered.", "anyOf": [ { - "description": "Recursive predicate condition or group, normalized to a grouped predicate after validation." + "$ref": "#/components/schemas/TablePredicateInput" }, { "type": "null" @@ -5630,7 +6196,8 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } }, { @@ -5638,7 +6205,8 @@ } ] } - } + }, + "additionalProperties": false }, "isDefault": { "description": "Whether to promote this view to the table default.", @@ -5646,6 +6214,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Update table view request", "description": "Workspace scope and one or more saved-view changes." }, @@ -5737,7 +6306,7 @@ }, "columnName": { "type": "string", - "description": "Table column receiving the output." + "description": "Name of the table column receiving the output." } }, "required": ["blockId", "path", "columnName"], @@ -5757,7 +6326,7 @@ }, "columnName": { "type": "string", - "description": "Source table column name." + "description": "Name of the source table column." } }, "required": ["inputName", "columnName"], @@ -5798,7 +6367,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Always `null` — this list has no `cursor` or `limit` param and returns its whole bounded set in one page. Present so the list can gain pages later without a shape change." } }, "required": ["data", "nextCursor"], @@ -5923,9 +6492,9 @@ "minLength": 1 }, "workflowId": { - "default": "", - "description": "Backing workflow identifier for a manual group.", - "type": "string" + "description": "Backing workflow identifier. Required when `type` is `manual` (which is also the default when `type` is omitted); omit it for an `enrichment` group.", + "type": "string", + "minLength": 1 }, "enrichmentId": { "description": "Registry enrichment identifier.", @@ -6042,7 +6611,8 @@ "type": "boolean" } }, - "required": ["name", "type"] + "required": ["name", "type"], + "additionalProperties": false }, "description": "Columns created for producer outputs." }, @@ -6181,7 +6751,8 @@ "type": "boolean" } }, - "required": ["name", "type"] + "required": ["name", "type"], + "additionalProperties": false } }, "mappingUpdates": { @@ -6235,7 +6806,7 @@ "enum": ["live", "deployed"] }, "type": { - "description": "Replacement workflow-group producer type.", + "description": "Workflow-group producer type. Must match the group's stored type — a group's producer cannot be changed after creation.", "type": "string", "enum": ["manual", "enrichment"] }, @@ -6445,6 +7016,7 @@ "rowIds": { "description": "Explicit row subset to run.", "minItems": 1, + "maxItems": 1000000, "type": "array", "items": { "type": "string", @@ -6452,7 +7024,7 @@ } }, "filter": { - "description": "Recursive predicate tree with exactly one non-empty `all` or `any` group at each group node." + "$ref": "#/components/schemas/TablePredicate" }, "excludeRowIds": { "description": "Rows excluded from a select-all run scope.", @@ -6483,6 +7055,7 @@ } }, "required": ["workspaceId", "groupIds"], + "additionalProperties": false, "title": "Run table columns request", "description": "Workspace scope, producer groups, execution mode, and optional row scope.", "examples": [ @@ -6515,6 +7088,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Run row enrichment request", "description": "Workspace scope for the row enrichment.", "examples": [ @@ -6548,15 +7122,16 @@ "type": "object", "properties": { "matches": { + "maxItems": 1000, "type": "array", "items": { "$ref": "#/components/schemas/V2TableRowMatch" }, - "description": "Matching table cells." + "description": "Matching table cells, at most 1000." }, "truncated": { "type": "boolean", - "description": "Whether more matches exist beyond the server cap." + "description": "Whether more than 1000 cells matched, so the list was cut." } }, "required": ["matches", "truncated"], @@ -6588,10 +7163,11 @@ "q": { "type": "string", "minLength": 1, + "maxLength": 200, "description": "Case-insensitive cell substring to find." }, "predicate": { - "description": "Recursive predicate tree with exactly one non-empty `all` or `any` group at each group node." + "$ref": "#/components/schemas/TablePredicate" }, "sort": { "description": "Ordered table-row sort specification.", @@ -6612,11 +7188,13 @@ "description": "Sort direction for this column." } }, - "required": ["field", "direction"] + "required": ["field", "direction"], + "additionalProperties": false } } }, "required": ["workspaceId", "q"], + "additionalProperties": false, "title": "Find table rows request", "description": "Workspace scope, substring query, and optional predicate and sort.", "examples": [ @@ -6680,15 +7258,7 @@ }, "status": { "type": "string", - "enum": [ - "uploading", - "queued", - "processing", - "completed", - "failed", - "canceled", - "expired" - ], + "enum": ["uploading", "processing", "completed", "failed", "canceled", "expired"], "description": "Current import lifecycle state." }, "source": { @@ -6713,8 +7283,7 @@ "description": "Name of the table to create." }, "folderPath": { - "description": "Folder path. A missing leading slash is normalized before validation.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["type", "name"], @@ -6827,7 +7396,7 @@ "url": { "type": "string", "format": "uri", - "description": "Signed URL to which the file bytes are uploaded." + "description": "Signed URL to which the file bytes are uploaded. Send the bytes with `PUT` to this URL, including exactly the headers in `headers` and nothing that alters the body. Success is `204` with an empty body. A failure is the same `{ \"error\": { \"code\", \"message\" } }` envelope as every other v2 response: `400` when the body does not match the size or content type the session was created for, `403` when the token is invalid, expired, or belongs to another session, and `409` when the session is no longer accepting bytes. The URL is signed and self-describing — construct it from this field only, never by hand." }, "headers": { "type": "object", @@ -6909,15 +7478,7 @@ }, "status": { "type": "string", - "enum": [ - "uploading", - "queued", - "processing", - "completed", - "failed", - "canceled", - "expired" - ], + "enum": ["uploading", "processing", "completed", "failed", "canceled", "expired"], "description": "Current import lifecycle state." }, "source": { @@ -6942,8 +7503,7 @@ "description": "Name of the table to create." }, "folderPath": { - "description": "Folder path. A missing leading slash is normalized before validation.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["type", "name"], @@ -7083,11 +7643,11 @@ }, "uploadToken": { "type": "null", - "description": "Always null for workspace-file imports." + "description": "Always null; a workspace-file import has no upload to authorize." }, "transfer": { "type": "null", - "description": "Always null for workspace-file imports." + "description": "Always null; a workspace-file import has no bytes to transfer." } }, "required": ["session", "uploadToken", "transfer"], @@ -7147,8 +7707,7 @@ "description": "Name of the table to create." }, "folderPath": { - "description": "Folder path. A missing leading slash is normalized before validation.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["type", "name"], @@ -7233,15 +7792,7 @@ }, "status": { "type": "string", - "enum": [ - "uploading", - "queued", - "processing", - "completed", - "failed", - "canceled", - "expired" - ], + "enum": ["uploading", "processing", "completed", "failed", "canceled", "expired"], "description": "Current import lifecycle state." }, "source": { @@ -7273,8 +7824,7 @@ "description": "Name of the table to create." }, "folderPath": { - "description": "Folder path. A missing leading slash is normalized before validation.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["type", "name"], @@ -7414,7 +7964,7 @@ "url": { "type": "string", "format": "uri", - "description": "Signed URL for this upload part." + "description": "Signed URL for this upload part. Send the bytes with `PUT` to this URL, including exactly the headers in `headers` and nothing that alters the body. Success is `204` with an empty body. A failure is the same `{ \"error\": { \"code\", \"message\" } }` envelope as every other v2 response: `400` when the body does not match the size or content type the session was created for, `403` when the token is invalid, expired, or belongs to another session, and `409` when the session is no longer accepting bytes. The URL is signed and self-describing — construct it from this field only, never by hand." }, "headers": { "type": "object", @@ -7619,6 +8169,7 @@ } }, "required": ["workspaceId"], + "additionalProperties": false, "title": "Create table export request", "description": "Workspace scope and export format.", "examples": [ @@ -7736,7 +8287,7 @@ "minLength": 1 }, "filter": { - "description": "Recursive predicate tree with exactly one non-empty `all` or `any` group at each group node." + "$ref": "#/components/schemas/TablePredicate" }, "excludeRowIds": { "description": "Rows excluded from an all-scope cancellation.", @@ -7749,6 +8300,7 @@ } }, "required": ["workspaceId", "scope"], + "additionalProperties": false, "title": "Cancel table runs request", "description": "Workspace scope, cancellation scope, and optional predicate or producer groups.", "examples": [ @@ -7768,11 +8320,15 @@ }, "path": { "type": "string", - "description": "Canonical folder path used as the public folder identifier." + "title": "Non-root folder path", + "description": "Canonical folder path used as the public folder identifier.", + "maxLength": 4096 }, "parentPath": { "type": "string", - "description": "Canonical parent path; `/` is the root." + "title": "Folder path", + "description": "Canonical parent path; `/` is the root.", + "maxLength": 4096 }, "createdAt": { "type": "string", @@ -7809,7 +8365,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Always `null` — this list has no `cursor` or `limit` param and returns its whole bounded set in one page. Present so the list can gain pages later without a shape change." } }, "required": ["data", "nextCursor"], @@ -7830,6 +8386,12 @@ "title": "Create table folder response", "description": "The created table folder." }, + "NonRootFolderPathInput": { + "title": "Non-root folder path input", + "description": "Non-root folder path. A missing leading slash is normalized before validation. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096, + "type": "string" + }, "CreateTableFolderRequest": { "type": "object", "properties": { @@ -7840,7 +8402,7 @@ }, "path": { "description": "Path of the folder to create.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, "required": ["workspaceId", "path"], @@ -7871,11 +8433,11 @@ }, "path": { "description": "Current folder path.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" }, "destinationPath": { "description": "New full path for the folder and its descendants.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, "required": ["workspaceId", "path", "destinationPath"], @@ -7888,7 +8450,9 @@ "properties": { "path": { "type": "string", - "description": "Canonical path of the deleted folder." + "title": "Folder path", + "description": "Canonical path of the deleted folder.", + "maxLength": 4096 }, "deleted": { "type": "boolean", diff --git a/apps/docs/openapi-v2-workflows.json b/apps/docs/openapi-v2-workflows.json index 722d9688c06..6d9fdd9da01 100644 --- a/apps/docs/openapi-v2-workflows.json +++ b/apps/docs/openapi-v2-workflows.json @@ -40,7 +40,7 @@ "get": { "operationId": "listWorkflows", "summary": "List Workflows", - "description": "List workflows in a workspace with folder and deployment filters, search, sorting, and opaque cursor pagination. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "List workflows in a workspace with folder and deployment filters, search, sorting, and opaque cursor pagination. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "parameters": [ { @@ -58,10 +58,10 @@ "name": "folderPath", "in": "query", "required": false, - "description": "Restrict results to workflows in this folder path.", + "description": "Restrict results to workflows in this folder path. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", "schema": { - "description": "Restrict results to workflows in this folder path.", - "type": "string" + "description": "Restrict results to workflows in this folder path. A path that names no folder narrows the result to nothing, so the response is an empty page rather than an error.", + "$ref": "#/components/schemas/FolderPathInput" } }, { @@ -78,11 +78,11 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum workflows to return per page.", + "description": "Maximum workflows to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum workflows to return per page.", - "type": "number", + "description": "Maximum workflows to return per page. Must be a whole number from 1 to 100. Defaults to 50.", + "type": "integer", "minimum": 1, "maximum": 100 } @@ -91,19 +91,20 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque pagination cursor returned by a previous request.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque pagination cursor returned by a previous request.", - "type": "string" + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 } }, { "name": "search", "in": "query", "required": false, - "description": "Case-insensitive substring search on the resource name.", + "description": "Case-insensitive substring match against the resource name.", "schema": { - "description": "Case-insensitive substring search on the resource name.", + "description": "Case-insensitive substring match against the resource name.", "type": "string", "minLength": 1, "maxLength": 200 @@ -113,10 +114,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "position", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["position", "name", "createdAt", "updatedAt", "runCount"] } @@ -185,7 +186,7 @@ "post": { "operationId": "createWorkflowV2", "summary": "Create Workflow", - "description": "Create a workflow in a workspace root or canonical workflow folder. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Create a workflow in a workspace root or canonical workflow folder. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "requestBody": { "required": true, @@ -257,7 +258,7 @@ "get": { "operationId": "getWorkflow", "summary": "Get Workflow", - "description": "Get a workflow with its variables and deployed API-trigger inputs. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Get a workflow with its variables and deployed API-trigger inputs. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "parameters": [ { @@ -324,7 +325,7 @@ "patch": { "operationId": "updateWorkflowV2", "summary": "Update Workflow", - "description": "Rename, describe, or move a workflow to a canonical folder path. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Rename, describe, or move a workflow to a canonical folder path. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "parameters": [ { @@ -496,10 +497,10 @@ "name": "limit", "in": "query", "required": false, - "description": "Maximum deployment versions to return per page.", + "description": "Maximum deployment versions to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum deployment versions to return per page.", + "description": "Maximum deployment versions to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -509,10 +510,11 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque pagination cursor returned by a previous request.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque pagination cursor returned by a previous request.", - "type": "string" + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", + "type": "string", + "minLength": 1 } } ], @@ -588,7 +590,7 @@ "schema": { "type": "integer", "exclusiveMinimum": 0, - "maximum": 9007199254740991, + "maximum": 2147483647, "description": "Numeric deployment version.", "examples": [3] } @@ -640,11 +642,77 @@ } } }, + "/api/v2/workflows/{id}/deployment": { + "get": { + "operationId": "getWorkflowDeployment", + "summary": "Get Workflow Deployment", + "description": "Read the current deployment state of a workflow: whether a version is live, when it went live, the most recent deployment attempt with its readiness and failure payload, and whether the editable draft has since diverged from the live version. This is the only operation that publishes `needsRedeployment`.", + "tags": ["Workflows"], + "parameters": [ + { + "name": "id", + "in": "path", + "required": true, + "description": "Unique workflow identifier.", + "schema": { + "type": "string", + "minLength": 1, + "description": "Unique workflow identifier.", + "examples": ["3b1f7c92-8d4e-4a6b-9c0d-5e2f8a714b36"] + } + } + ], + "responses": { + "200": { + "description": "The current deployment state.", + "headers": { + "X-RateLimit-Limit": { + "$ref": "#/components/headers/X-RateLimit-Limit" + }, + "X-RateLimit-Remaining": { + "$ref": "#/components/headers/X-RateLimit-Remaining" + }, + "X-RateLimit-Reset": { + "$ref": "#/components/headers/X-RateLimit-Reset" + } + }, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/WorkflowDeploymentResponse" + } + } + } + }, + "400": { + "$ref": "#/components/responses/BadRequest" + }, + "401": { + "$ref": "#/components/responses/Unauthorized" + }, + "403": { + "$ref": "#/components/responses/Forbidden" + }, + "404": { + "$ref": "#/components/responses/NotFound" + }, + "429": { + "$ref": "#/components/responses/RateLimited" + }, + "500": { + "$ref": "#/components/responses/InternalError" + }, + "503": { + "$ref": "#/components/responses/ServiceUnavailable" + } + } + } + }, "/api/v2/workflows/{id}/deploy": { "post": { "operationId": "deployWorkflow", "summary": "Deploy Workflow", - "description": "Create and asynchronously activate a deployment version. This request is not idempotent: it accepts no idempotency key and every call mints a new deployment version, so retrying after a timeout creates a second version rather than returning the first. The response carries `latestDeploymentAttempt` for the accepted attempt, but `GET /workflows/{id}` does not expose that field — poll activation with `isDeployed` and `deployedAt` on the workflow, or with `isActive` on `GET /workflows/{id}/versions`. Returns 409 when the deployment would conflict with an existing webhook path. A workspace API key cannot call this operation. Because unauthorized resources are concealed, the rejection is reported as `404` rather than `403`; use a personal API key.", + "description": "Create and asynchronously activate a deployment version. Not idempotent: every call mints a new version, so a retry after a timeout creates a second one. A deployment that would conflict with an existing webhook path is a `409`. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Workflows"], "parameters": [ { @@ -728,7 +796,7 @@ "delete": { "operationId": "undeployWorkflow", "summary": "Undeploy Workflow", - "description": "Deactivate the currently serving workflow version. A workspace API key cannot call this operation. Because unauthorized resources are concealed, the rejection is reported as `404` rather than `403`; use a personal API key.", + "description": "Deactivate the currently serving workflow version. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Workflows"], "parameters": [ { @@ -797,7 +865,7 @@ "post": { "operationId": "rollbackWorkflow", "summary": "Rollback Workflow", - "description": "Asynchronously reactivate a previous deployment version, selecting the preceding active version when no version is supplied. A workspace API key cannot call this operation. Because unauthorized resources are concealed, the rejection is reported as `404` rather than `403`; use a personal API key.", + "description": "Asynchronously reactivate a previous deployment version, selecting the preceding active version when no version is supplied. A workspace API key is rejected with `403`; use a personal API key.", "tags": ["Workflows"], "parameters": [ { @@ -858,6 +926,9 @@ "404": { "$ref": "#/components/responses/NotFound" }, + "409": { + "$ref": "#/components/responses/Conflict" + }, "413": { "$ref": "#/components/responses/PayloadTooLarge" }, @@ -880,7 +951,7 @@ "get": { "operationId": "exportWorkflow", "summary": "Export Workflow", - "description": "Export a portable, secret-sanitized workflow. Workspace-scoped bindings must be selected again after import. A workspace whose folder tree exceeds 10,000 folders is a 413, because the response needs the whole tree to render folder paths.", + "description": "Export a portable, secret-sanitized workflow. Workspace-scoped bindings must be selected again after import. Exporting records an audit event, so it is not a safe read. A `HEAD` skips the effect but is authorized exactly as the `GET` is, so it answers `400`, `401`, `403`, or `404` wherever the `GET` would and an empty `200` otherwise. Skipping the effect means skipping the read that produces the payload, so that `200` carries none of the response headers documented below — it answers whether the `GET` would be allowed, not what the `GET` would return. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "parameters": [ { @@ -949,7 +1020,7 @@ "post": { "operationId": "importWorkflow", "summary": "Import Workflow", - "description": "Create a workflow from a portable export object, bare state, or JSON string.", + "description": "Create a workflow from a portable export object, bare state, or JSON string. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "requestBody": { "required": true, @@ -1021,7 +1092,7 @@ "post": { "operationId": "executeWorkflowV2", "summary": "Execute Workflow", - "description": "Execute a deployed workflow synchronously, asynchronously, or as Server-Sent Events. Public workflows permit anonymous synchronous and streaming execution; asynchronous execution requires an API key. A synchronous run that exceeds its execution timeout returns HTTP 200 with `status: \"failed\"` and `error.code: \"TIMEOUT\"` rather than an HTTP error, so branch on `status`. The optional `X-Run-Id` header is a one-shot uniqueness claim, not an idempotency key: reusing a value returns 409 with `error.details.code: \"RUN_ID_CONFLICT\"` and never replays the earlier run. Option constraints — each is a 400: (1) `async: true` requires an API key; anonymous public-workflow callers may only execute synchronously or as a stream. (2) `async` and `stream` cannot both be true. (3) `executionTimeoutSeconds` is accepted only when `async: true`. (4) `async: true` rejects every streaming and output-shaping option — `selectedOutputs`, `includeThinking`, `includeToolCalls`, `includeFileBase64`, and `base64MaxBytes`. (5) `includeThinking` and `includeToolCalls` require `stream: true`. (6) `includeThinking` and `includeToolCalls` require the `X-Sim-Stream-Protocol: agent-events-v1` request header, which declares that the client understands agent-event frames.", + "description": "Execute a deployed workflow synchronously, asynchronously, or as Server-Sent Events. Public workflows permit anonymous synchronous and streaming execution; asynchronous execution requires an API key. A synchronous run that exceeds its execution timeout returns HTTP 200 with `status: \"failed\"` and `error.code: \"TIMEOUT\"` rather than an HTTP error, so branch on `status`. Each option carries the modes it requires and the modes that reject it; a violated combination is a 400.", "tags": ["Workflows"], "security": [ { @@ -1046,9 +1117,9 @@ "name": "x-run-id", "in": "header", "required": false, - "description": "Caller-supplied run identifier, available only to API-key callers. This is a one-shot uniqueness claim, NOT an idempotency key: the first request to use a value starts a run, and any later request reusing it fails with 409 and `error.details.code: \"RUN_ID_CONFLICT\"` instead of replaying the original result. To retry safely, generate a fresh value per attempt and reconcile duplicates yourself, or omit the header and let the server allocate the run identifier.", + "description": "Caller-supplied run identifier, available only to API-key callers. A one-shot uniqueness claim, NOT an idempotency key: reusing a value fails with `409` and `error.details.code: \"RUN_ID_CONFLICT\"` rather than replaying the original result. To retry safely, send a fresh value per attempt, or omit the header and let the server allocate one.", "schema": { - "description": "Caller-supplied run identifier, available only to API-key callers. This is a one-shot uniqueness claim, NOT an idempotency key: the first request to use a value starts a run, and any later request reusing it fails with 409 and `error.details.code: \"RUN_ID_CONFLICT\"` instead of replaying the original result. To retry safely, generate a fresh value per attempt and reconcile duplicates yourself, or omit the header and let the server allocate the run identifier.", + "description": "Caller-supplied run identifier, available only to API-key callers. A one-shot uniqueness claim, NOT an idempotency key: reusing a value fails with `409` and `error.details.code: \"RUN_ID_CONFLICT\"` rather than replaying the original result. To retry safely, send a fresh value per attempt, or omit the header and let the server allocate one.", "type": "string", "minLength": 1, "maxLength": 128, @@ -1060,16 +1131,16 @@ "name": "x-sim-via", "in": "header", "required": false, - "description": "Comma-separated workflow identifiers describing the workflow-to-workflow call chain that led to this request. Each hop appends its own workflow id, and Sim sets it automatically when one workflow calls another; supply it yourself only when relaying an existing chain. A chain already at the maximum depth is rejected with 409 and `error.details.code: \"CALL_CHAIN_DEPTH_EXCEEDED\"`, which is how runaway recursion between workflows is stopped.", + "description": "Comma-separated workflow identifiers naming the workflow-to-workflow call chain that led to this request. Each hop appends its own workflow id, and Sim sets it automatically; supply it yourself only when relaying an existing chain. A chain at the maximum depth is rejected with `409` and `error.details.code: \"CALL_CHAIN_DEPTH_EXCEEDED\"`.", "schema": { - "description": "Comma-separated workflow identifiers describing the workflow-to-workflow call chain that led to this request. Each hop appends its own workflow id, and Sim sets it automatically when one workflow calls another; supply it yourself only when relaying an existing chain. A chain already at the maximum depth is rejected with 409 and `error.details.code: \"CALL_CHAIN_DEPTH_EXCEEDED\"`, which is how runaway recursion between workflows is stopped.", + "description": "Comma-separated workflow identifiers naming the workflow-to-workflow call chain that led to this request. Each hop appends its own workflow id, and Sim sets it automatically; supply it yourself only when relaying an existing chain. A chain at the maximum depth is rejected with `409` and `error.details.code: \"CALL_CHAIN_DEPTH_EXCEEDED\"`.", "type": "string" } } ], "requestBody": { "required": true, - "description": "Input and execution-mode options for a deployed workflow. Option constraints — each is a 400: (1) `async: true` requires an API key; anonymous public-workflow callers may only execute synchronously or as a stream. (2) `async` and `stream` cannot both be true. (3) `executionTimeoutSeconds` is accepted only when `async: true`. (4) `async: true` rejects every streaming and output-shaping option — `selectedOutputs`, `includeThinking`, `includeToolCalls`, `includeFileBase64`, and `base64MaxBytes`. (5) `includeThinking` and `includeToolCalls` require `stream: true`. (6) `includeThinking` and `includeToolCalls` require the `X-Sim-Stream-Protocol: agent-events-v1` request header, which declares that the client understands agent-event frames.", + "description": "Input and execution-mode options for a deployed workflow. Each option carries the modes it requires and the modes that reject it; a violated combination is a 400.", "content": { "application/json": { "schema": { @@ -1172,7 +1243,7 @@ "get": { "operationId": "listWorkflowRunsV2", "summary": "List Workflow Runs", - "description": "List recorded runs of a workflow with filtering and opaque cursor pagination. Ordering deviates from the v2 `sortBy` + `sortOrder` convention: runs are sortable only by start time, so direction is carried by the single `order` param.", + "description": "List recorded runs of a workflow with filtering and opaque cursor pagination. Runs are hard-deleted once they pass the payer's log retention window, so an older run is simply absent rather than reported as removed. The window is 30 days from run start on the free plan, unbounded on Pro and Team, and set per organization on Enterprise with an optional per-workspace override.", "tags": ["Workflow Runs"], "parameters": [ { @@ -1213,34 +1284,34 @@ "name": "startDate", "in": "query", "required": false, - "description": "Include runs started at or after this ISO 8601 timestamp.", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant.", "schema": { - "description": "Include runs started at or after this ISO 8601 timestamp.", "type": "string", "format": "date-time", - "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$" + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or after this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant." } }, { "name": "endDate", "in": "query", "required": false, - "description": "Include runs started at or before this ISO 8601 timestamp.", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant.", "schema": { - "description": "Include runs started at or before this ISO 8601 timestamp.", "type": "string", "format": "date-time", - "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$" + "pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d(?::[0-5]\\d(?:\\.\\d+)?)?(?:Z))$", + "description": "Only include runs started at or before this UTC ISO 8601 timestamp, e.g. `2026-08-06T00:00:00Z`. A date without a time, or a timestamp carrying a UTC offset instead of `Z`, is rejected, as is year `0000`, which names no storable instant." } }, { "name": "limit", "in": "query", "required": false, - "description": "Maximum workflow runs to return per page.", + "description": "Maximum workflow runs to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "schema": { "default": 50, - "description": "Maximum workflow runs to return per page.", + "description": "Maximum workflow runs to return per page. Must be a whole number from 1 to 100. Defaults to 50.", "type": "integer", "minimum": 1, "maximum": 100 @@ -1250,9 +1321,9 @@ "name": "cursor", "in": "query", "required": false, - "description": "Opaque pagination cursor returned by a previous request.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "schema": { - "description": "Opaque pagination cursor returned by a previous request.", + "description": "Opaque cursor from the previous page. Send it back with the same sort and filters; only `limit` may change. Change anything else and pagination must restart without a cursor.", "type": "string", "minLength": 1 } @@ -1261,10 +1332,10 @@ "name": "order", "in": "query", "required": false, - "description": "Sort direction by run start time. This operation deviates from the v2 `sortBy` + `sortOrder` convention: runs are sortable only by start time, so the direction is carried by this single `order` param and `sortBy`/`sortOrder` are not accepted.", + "description": "Sort direction by run start time. This list is sortable only by run start time, so it takes `order` in place of `sortBy`/`sortOrder`, which it rejects.", "schema": { "default": "desc", - "description": "Sort direction by run start time. This operation deviates from the v2 `sortBy` + `sortOrder` convention: runs are sortable only by start time, so the direction is carried by this single `order` param and `sortBy`/`sortOrder` are not accepted.", + "description": "Sort direction by run start time. This list is sortable only by run start time, so it takes `order` in place of `sortBy`/`sortOrder`, which it rejects.", "type": "string", "enum": ["asc", "desc"] } @@ -1352,20 +1423,19 @@ "name": "includeOutput", "in": "query", "required": false, - "description": "Include final and block outputs when true.", + "description": "Include the final workflow output when true. It does not gate `blockOutputs`, which `selectedOutputs` selects on its own.", "schema": { - "description": "Include final and block outputs when true.", - "type": "string", - "enum": ["true", "false"] + "description": "Include the final workflow output when true. It does not gate `blockOutputs`, which `selectedOutputs` selects on its own.", + "type": "boolean" } }, { "name": "selectedOutputs", "in": "query", "required": false, - "description": "Comma-separated block output references to include.", + "description": "Comma-separated block output references to include, as `blockId` or `blockId.path`. Block *names* are not resolved here — unlike the execute request, this resource reads a recorded run and matches ids only, so a name selects nothing and yields an empty `blockOutputs`.", "schema": { - "description": "Comma-separated block output references to include.", + "description": "Comma-separated block output references to include, as `blockId` or `blockId.path`. Block *names* are not resolved here — unlike the execute request, this resource reads a recorded run and matches ids only, so a name selects nothing and yields an empty `blockOutputs`.", "type": "string" } } @@ -1549,7 +1619,7 @@ "post": { "operationId": "cancelRunV2", "summary": "Cancel Workflow Run", - "description": "Request cancellation of a running, queued, or paused workflow run. Cancelling a run that has already reached a terminal state succeeds with no effect rather than returning an error. The `reason` field is present on every response, including full successes — `recorded` is the success value; it is not a partial-failure marker. A run produced by a table workflow group is a 409 when its cell can no longer accept the cancellation, because the run and its cell must reach the cancelled state together.", + "description": "Request cancellation of a running, queued, or paused workflow run. Cancelling a run already in a terminal state succeeds with no effect. A run produced by a table workflow group is a `409` when its cell can no longer accept the cancellation.", "tags": ["Workflow Runs"], "parameters": [ { @@ -1631,7 +1701,7 @@ "get": { "operationId": "listWorkflowsFolders", "summary": "List Workflow Folders", - "description": "List canonical workflow folders in a workspace. The bounded set is returned in one page with `nextCursor` always null; there is no second page to fetch.", + "description": "List canonical workflow folders in a workspace. The bounded set is returned in one page; `nextCursor` is always null. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "parameters": [ { @@ -1652,7 +1722,7 @@ "description": "Restrict results to direct children of this parent path.", "schema": { "description": "Restrict results to direct children of this parent path.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, { @@ -1671,10 +1741,10 @@ "name": "sortBy", "in": "query", "required": false, - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "schema": { "default": "name", - "description": "Field used to sort the result.", + "description": "Field used to sort the result. Sorting by `name` is case-sensitive and follows the storage collation, so do not rely on a case-insensitive order.", "type": "string", "enum": ["name", "createdAt", "updatedAt"] } @@ -1743,7 +1813,7 @@ "post": { "operationId": "createWorkflowsFolder", "summary": "Create Workflow Folder", - "description": "Create a canonical workflow folder in a workspace.", + "description": "Create a canonical workflow folder in a workspace. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "requestBody": { "required": true, @@ -1813,7 +1883,7 @@ "patch": { "operationId": "relocateWorkflowsFolder", "summary": "Rename or Move Workflow Folder", - "description": "Rename or move a workflow folder and its descendants to a canonical path.", + "description": "Rename or move a workflow folder and its descendants to a canonical path. A workspace folder tree over 10,000 folders is a `413`.", "tags": ["Workflows"], "requestBody": { "required": true, @@ -1904,16 +1974,30 @@ "description": "Path of the folder to delete.", "schema": { "description": "Path of the folder to delete.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, { "name": "recursive", "in": "query", "required": false, - "description": "Delete nested files and folders when true.", + "description": "Delete the folder's nested files and folders too. An empty folder deletes either way; a non-empty one needs this. The listed spellings are the whole accepted vocabulary and are case-sensitive; any other value is rejected.", "schema": { - "description": "Delete nested files and folders when true.", + "description": "Delete the folder's nested files and folders too. An empty folder deletes either way; a non-empty one needs this. The listed spellings are the whole accepted vocabulary and are case-sensitive; any other value is rejected.", + "enum": [ + "true", + "1", + "yes", + "on", + "y", + "enabled", + "false", + "0", + "no", + "off", + "n", + "disabled" + ], "default": "false", "type": "string" } @@ -1981,7 +2065,7 @@ "type": "apiKey", "in": "header", "name": "X-API-Key", - "description": "Your Sim API key, personal or workspace-scoped. Generate one from the Sim dashboard under Settings > API Keys. A workspace API key is not accepted everywhere: operations that act on behalf of a specific human — administrative reads, secret access, and irreversible or governance-affecting writes — always reject it, whatever role the key carries. Each such operation says so in its own description, and the rejection surfaces as `403` unless the operation conceals unauthorized resources, in which case it is reported as `404`. Use a personal API key for those." + "description": "Your Sim API key, personal or workspace-scoped. Generate one under Settings, then API Keys. Operations that reject workspace keys say so in their own description." } }, "headers": { @@ -2016,13 +2100,13 @@ } }, "Retry-After": { - "description": "Seconds to wait before retrying a rate-limited request.", + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset.", "schema": { "type": "integer", "minimum": 0, "maximum": 9007199254740991, "title": "Retry after", - "description": "Seconds to wait before retrying a rate-limited request." + "description": "Seconds to wait before retrying, sent on `429` and `503`. Add jitter rather than retrying at exactly this offset." } }, "X-Run-Id": { @@ -2037,7 +2121,7 @@ }, "responses": { "BadRequest": { - "description": "The request is invalid.", + "description": "The request is invalid. This includes a query parameter sent with no value (`?limit=`, `?search=`), which is rejected rather than read as zero, empty, or the parameter default — omit the parameter instead.", "content": { "application/json": { "schema": { @@ -2067,7 +2151,7 @@ } }, "Forbidden": { - "description": "The caller lacks access to the resource.", + "description": "The caller lacks the rights this operation requires. When the cause is one a caller can act on, `error.details.code` names it. A resource in a workspace the caller cannot reach at all answers `404` instead, so absence and denial are indistinguishable.", "content": { "application/json": { "schema": { @@ -2097,7 +2181,7 @@ } }, "RunIdConflict": { - "description": "The run cannot be started. Two causes share this status, distinguished by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already associated with a different request, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has already reached the maximum workflow-to-workflow call depth.", + "description": "The run cannot be started, for one of two causes named by `error.details.code`: `RUN_ID_CONFLICT` when the supplied `X-Run-Id` is already claimed, and `CALL_CHAIN_DEPTH_EXCEEDED` when the incoming `X-Sim-Via` chain has reached the maximum workflow-to-workflow call depth.", "headers": { "X-Run-Id": { "$ref": "#/components/headers/X-Run-Id" @@ -2111,18 +2195,8 @@ } } }, - "Gone": { - "description": "The requested generated resource has expired.", - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/V2Error" - } - } - } - }, "PayloadTooLarge": { - "description": "The request, or a resource collection it must materialize, exceeds the allowed size. Besides an oversized request body, this covers a generated artifact that renders past the download ceiling and a workspace folder tree too large to load in full.", + "description": "The request, or a resource collection it must materialize, exceeds the allowed size: an oversized request body, a generated artifact past the download ceiling, or a workspace folder tree too large to load in full.", "content": { "application/json": { "schema": { @@ -2167,7 +2241,7 @@ } }, "ClientClosedRequest": { - "description": "The client closed the connection before the response was produced.", + "description": "The client closed the connection before the response was produced. An abort can leave the run going, so `error.details.runId` carries the run id — reconcile against the runs resource rather than starting another run.", "content": { "application/json": { "schema": { @@ -2187,7 +2261,12 @@ } }, "ServiceUnavailable": { - "description": "A required service is temporarily unavailable.", + "description": "A required service is temporarily unavailable. `Retry-After` carries the seconds to wait; treat it as a floor and add jitter. The header is omitted when `error.details.code` is `ASYNC_ENQUEUE_AMBIGUOUS`, because the run may already have started — reconcile against the returned run id instead of retrying.", + "headers": { + "Retry-After": { + "$ref": "#/components/headers/Retry-After" + } + }, "content": { "application/json": { "schema": { @@ -2213,7 +2292,7 @@ "description": "Human-readable explanation of the error." }, "details": { - "description": "Optional structured error details." + "description": "Structured error details. On a `403` whose cause a caller can act on, this carries a `code` from a closed set:\n- `INSUFFICIENT_WORKSPACE_ROLE` — The caller has access to the workspace but its role is below the one this operation requires.\n- `PERSONAL_API_KEYS_DISABLED` — The workspace's organization does not allow personal API keys. Use a workspace API key.\n- `WORKSPACE_KEY_OPERATION_NOT_PERMITTED` — This operation is not available to a workspace-scoped API key. Use a personal API key.\n- `PRINCIPAL_KIND_NOT_PERMITTED` — This operation does not accept the caller’s kind of API key.\n- `ORGANIZATION_MEMBERSHIP_REQUIRED` — The caller is not a member of the organization it named.\n- `ORGANIZATION_ADMIN_REQUIRED` — The caller is a member of the organization but not an admin or owner.\n- `ENTERPRISE_PLAN_REQUIRED` — The organization has no active enterprise subscription.\n- `AUDIT_LOGS_DISABLED` — Audit logging is not enabled for this deployment.\n- `SKILL_EDITOR_ACCESS_REQUIRED` — The caller can write in the workspace but is not an editor of this skill.\n- `SECRET_ADMIN_ACCESS_REQUIRED` — The caller can write in the workspace but is not an admin of this secret. Ask a workspace admin, or someone holding admin on the secret, to grant access or set the value.\n- `WORKSPACE_RESOURCE_LIMIT_REACHED` — The workspace already holds the maximum number of resources of this kind. Delete one, or contact Sim to raise the limit; the message names the ceiling.\n- `PUBLIC_SHARING_NOT_ALLOWED` — The workspace's organization does not permit sharing this resource publicly. An organization admin controls the policy.\n- `MCP_SERVER_URL_NOT_ALLOWED` — The supplied MCP server URL is outside the allowed domains or resolves to an internal address." } }, "required": ["code", "message"], @@ -2234,6 +2313,12 @@ } ] }, + "FolderPathInput": { + "title": "Folder path input", + "description": "Folder path. A missing leading slash is normalized before validation. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096, + "type": "string" + }, "WorkflowListItem": { "type": "object", "properties": { @@ -2260,7 +2345,9 @@ }, "folderPath": { "type": "string", + "title": "Folder path", "description": "Canonical containing-folder path; `/` is the workspace root.", + "maxLength": 4096, "examples": ["/Operations"] }, "workspaceId": { @@ -2287,7 +2374,7 @@ "type": "integer", "minimum": 0, "maximum": 9007199254740991, - "description": "Total recorded workflow runs." + "description": "Runs that finished successfully. Failed, cancelled, and paused runs are not counted, and the counter is never reduced when a run ages out of log retention — so it does not match the size of `GET /api/v2/workflows/{id}/runs`, in either direction." }, "lastRunAt": { "anyOf": [ @@ -2298,7 +2385,7 @@ "type": "null" } ], - "description": "ISO 8601 timestamp of the latest run, or null when never run.", + "description": "ISO 8601 timestamp of the latest run counted by `runCount`, or null when none has been. Stamped by the same successful-run path, so a workflow whose only runs failed reports null here.", "format": "date-time" }, "createdAt": { @@ -2348,7 +2435,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -2433,8 +2520,7 @@ ] }, "folderPath": { - "description": "Folder path. A missing leading slash is normalized before validation.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "required": ["workspaceId", "name"], @@ -2489,7 +2575,9 @@ }, "folderPath": { "type": "string", + "title": "Folder path", "description": "Canonical containing-folder path; `/` is the workspace root.", + "maxLength": 4096, "examples": ["/Operations"] }, "workspaceId": { @@ -2516,7 +2604,7 @@ "type": "integer", "minimum": 0, "maximum": 9007199254740991, - "description": "Total recorded workflow runs." + "description": "Runs that finished successfully. Failed, cancelled, and paused runs are not counted, and the counter is never reduced when a run ages out of log retention — so it does not match the size of `GET /api/v2/workflows/{id}/runs`, in either direction." }, "lastRunAt": { "anyOf": [ @@ -2527,7 +2615,7 @@ "type": "null" } ], - "description": "ISO 8601 timestamp of the latest run, or null when never run.", + "description": "ISO 8601 timestamp of the latest run counted by `runCount`, or null when none has been. Stamped by the same successful-run path, so a workflow whose only runs failed reports null here.", "format": "date-time" }, "createdAt": { @@ -2662,7 +2750,7 @@ }, "folderPath": { "description": "Destination folder path; `/` moves the workflow to the workspace root.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" } }, "additionalProperties": false, @@ -2800,7 +2888,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -2876,7 +2964,7 @@ "format": "date-time" }, "state": { - "description": "Deployed workflow graph snapshot pinned by this version. Credential-bearing values are redacted: `oauth-input`, `password: true`, and table sub-block values are null; sensitive nested tool parameters and every parameter without authoritative codec metadata are null.", + "description": "Deployed workflow graph snapshot pinned by this version, with credential-bearing values redacted to null: `oauth-input`, `password: true`, table sub-block values, sensitive nested tool parameters, and any parameter without authoritative codec metadata.", "$ref": "#/components/schemas/DeployedWorkflowState" } }, @@ -3061,6 +3149,123 @@ "title": "Deployment operation error", "description": "Failure details for a deployment lifecycle operation." }, + "WorkflowDeployment": { + "type": "object", + "properties": { + "id": { + "type": "string", + "description": "Unique workflow identifier.", + "examples": ["3b1f7c92-8d4e-4a6b-9c0d-5e2f8a714b36"] + }, + "isDeployed": { + "type": "boolean", + "description": "Whether a workflow version is currently live and available for API execution." + }, + "deployedAt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "ISO 8601 timestamp associated with the deployment, or null when unavailable.", + "format": "date-time", + "examples": ["2026-06-12T10:30:00.000Z"] + }, + "warnings": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Non-fatal synchronization warnings. Empty when there is nothing to report." + }, + "activeDeployment": { + "anyOf": [ + { + "$ref": "#/components/schemas/ActiveDeploymentSummary" + }, + { + "type": "null" + } + ], + "description": "Currently live deployment version, or null while no version is active." + }, + "latestDeploymentAttempt": { + "anyOf": [ + { + "$ref": "#/components/schemas/DeploymentOperationSummary" + }, + { + "type": "null" + } + ], + "description": "Most recent deployment lifecycle attempt, or null when none is available." + }, + "needsRedeployment": { + "type": "boolean", + "description": "Whether the editable draft has diverged from the live deployment version. False while a deployment attempt is still preparing or activating, and false when nothing is deployed." + } + }, + "required": [ + "id", + "isDeployed", + "deployedAt", + "warnings", + "activeDeployment", + "latestDeploymentAttempt", + "needsRedeployment" + ], + "additionalProperties": false, + "title": "Workflow deployment", + "description": "Current deployment state of a workflow, including draft-versus-live drift and the most recent deployment attempt." + }, + "WorkflowDeploymentResponse": { + "type": "object", + "properties": { + "data": { + "description": "Response data.", + "$ref": "#/components/schemas/WorkflowDeployment" + } + }, + "required": ["data"], + "additionalProperties": false, + "title": "Workflow deployment response", + "description": "Current deployment state, including draft-versus-live drift.", + "examples": [ + { + "data": { + "id": "3b1f7c92-8d4e-4a6b-9c0d-5e2f8a714b36", + "isDeployed": true, + "needsRedeployment": true, + "deployedAt": "2026-06-12T10:30:00.000Z", + "warnings": [], + "activeDeployment": { + "deploymentVersionId": "depver_01J8ZK3QW4M6X2R9T7B5C0V2", + "version": 3, + "deployedAt": "2026-06-12T10:30:00.000Z" + }, + "latestDeploymentAttempt": { + "id": "depop_01J8ZK3QW4M6X2R9T7B5C0V1", + "deploymentVersionId": "depver_01J8ZK3QW4M6X2R9T7B5C0V2", + "version": 3, + "action": "deploy", + "status": "active", + "isCurrent": true, + "readiness": { + "webhooks": "ready", + "schedules": "ready", + "mcp": "not_applicable" + }, + "requestedAt": "2026-06-12T10:29:58.000Z", + "activatedAt": "2026-06-12T10:30:00.000Z", + "error": null + } + } + } + ] + }, "DeployResult": { "type": "object", "properties": { @@ -3132,7 +3337,7 @@ ], "additionalProperties": false, "title": "Deploy result", - "description": "Deployment attempt accepted for processing. Activation is asynchronous; `latestDeploymentAttempt` on this response is the attempt handle. The request is NOT idempotent — every POST mints a new deployment version, so a retry after a timeout creates a second version rather than returning the first. `latestDeploymentAttempt` is returned only here: `GET /workflows/{id}` does not carry it, so poll activation with `isDeployed` and `deployedAt` on the workflow, or with `isActive` on `GET /workflows/{id}/versions`." + "description": "Deployment attempt accepted for processing. Activation is asynchronous, and `latestDeploymentAttempt` is the attempt handle — returned only here. Poll activation with `isDeployed` and `deployedAt` on the workflow, or `isActive` on `GET /workflows/{id}/versions`." }, "DeployWorkflowResponse": { "type": "object", @@ -3205,7 +3410,8 @@ } ] } - } + }, + "additionalProperties": false }, "UndeployResult": { "type": "object", @@ -3431,7 +3637,8 @@ "minimum": 1, "maximum": 2147483647 } - } + }, + "additionalProperties": false }, "WorkflowExportPayload": { "type": "object", @@ -3481,7 +3688,9 @@ }, "folderPath": { "type": "string", - "description": "Canonical containing-folder path; `/` is the workspace root." + "title": "Folder path", + "description": "Canonical containing-folder path; `/` is the workspace root.", + "maxLength": 4096 } }, "required": ["id", "name", "description", "workspaceId", "folderPath"], @@ -3559,7 +3768,9 @@ }, "folderPath": { "type": "string", - "description": "Canonical containing-folder path." + "title": "Folder path", + "description": "Canonical containing-folder path.", + "maxLength": 4096 }, "createdAt": { "type": "string", @@ -3636,7 +3847,7 @@ }, "folderPath": { "description": "Destination folder path; omit for the workspace root.", - "type": "string" + "$ref": "#/components/schemas/FolderPathInput" }, "name": { "description": "Override for the imported workflow name.", @@ -3671,21 +3882,20 @@ "INVALID_INPUT", "BLOCK_EXECUTION_FAILED", "CHILD_WORKFLOW_FAILED", - "OUTPUT_TOO_LARGE", "EXECUTION_FAILED" ], - "description": "Stable machine-readable execution failure code." + "description": "Stable machine-readable execution failure code. `BLOCK_EXECUTION_FAILED` and `CHILD_WORKFLOW_FAILED` are reported only where block attribution is available; elsewhere a block-level failure is reported as `EXECUTION_FAILED`." }, "blockId": { - "description": "Identifier of the failing block, when attributable.", + "description": "Identifier of the failing block. Present on the synchronous execute response only; the polled run resource and the resume response cannot attribute a block.", "type": "string" }, "blockName": { - "description": "Display name of the failing block.", + "description": "Display name of the failing block. Present on the synchronous execute response only.", "type": "string" }, "blockType": { - "description": "Integration or block type that failed.", + "description": "Integration or block type that failed. Present on the synchronous execute response only.", "type": "string" } }, @@ -3747,7 +3957,7 @@ "required": ["runId", "workflowId", "status", "output", "error"], "additionalProperties": false, "title": "Workflow run result", - "description": "Synchronous workflow run output and in-band execution status. Run failures are reported in band, not as HTTP errors — a synchronous run that exceeds its execution timeout returns HTTP 200 with `status: \"failed\"` and `error.code: \"TIMEOUT\"`, so always branch on `status` rather than on the HTTP status alone." + "description": "Synchronous workflow run output and in-band execution status. Run failures are reported in band, not as HTTP errors — a run that exceeds its execution timeout returns HTTP 200 with `status: \"failed\"` and `error.code: \"TIMEOUT\"`, so branch on `status`." }, "ExecuteWorkflowSyncResponse": { "type": "object", @@ -3840,7 +4050,7 @@ "type": "boolean" }, "executionTimeoutSeconds": { - "description": "Requested server-side timeout for an asynchronous run, in seconds. This is an upper bound on the request, not the effective timeout: the run uses the smaller of this value and the account plan's execution timeout, so requesting more than the plan allows silently yields the plan timeout with no warning. Rejected with 400 unless `async` is true.", + "description": "Requested server-side timeout for an asynchronous run, in seconds. An upper bound, not the effective timeout: the run uses the smaller of this value and the plan's execution timeout, so requesting more than the plan allows silently yields the plan timeout. Rejected with `400` unless `async` is true.", "type": "integer", "minimum": 1, "maximum": 604800 @@ -3870,11 +4080,11 @@ "type": "boolean" }, "includeFileBase64": { - "description": "Inline eligible output files as base64 content.", + "description": "Inline eligible output files as base64 content. Rejected when `async` is true.", "type": "boolean" }, "base64MaxBytes": { - "description": "Maximum total bytes of file content to inline as base64.", + "description": "Maximum total bytes of file content to inline as base64. Rejected when `async` is true.", "type": "integer", "exclusiveMinimum": 0, "maximum": 10485760 @@ -3882,7 +4092,7 @@ }, "additionalProperties": false, "title": "Execute workflow request", - "description": "Input and execution-mode options for a deployed workflow. Option constraints — each is a 400: (1) `async: true` requires an API key; anonymous public-workflow callers may only execute synchronously or as a stream. (2) `async` and `stream` cannot both be true. (3) `executionTimeoutSeconds` is accepted only when `async: true`. (4) `async: true` rejects every streaming and output-shaping option — `selectedOutputs`, `includeThinking`, `includeToolCalls`, `includeFileBase64`, and `base64MaxBytes`. (5) `includeThinking` and `includeToolCalls` require `stream: true`. (6) `includeThinking` and `includeToolCalls` require the `X-Sim-Stream-Protocol: agent-events-v1` request header, which declares that the client understands agent-event frames.", + "description": "Input and execution-mode options for a deployed workflow. Each option carries the modes it requires and the modes that reject it; a violated combination is a 400.", "examples": [ { "input": { @@ -3929,7 +4139,7 @@ "failed", "cancelled" ], - "description": "Current or terminal run status. `redacting` is transient, reported while the output of a finished run is being scrubbed. `paused` means the run is not executing and is waiting to be resumed: either held at a human-in-the-loop pause point, or left paused because a resume attempt did not run to completion. The status alone does not say which. On the single-run response `paused.automaticResumeWaitingReason` distinguishes them: it is recorded whenever a resume attempt fails and cleared once a resume succeeds, so a null value means the run is waiting on human input. When the failure is not retryable or the automatic retries are exhausted, the reason is prefixed `Automatic resume requires manual intervention: `. Run-list items carry no `paused` object, so the two cases are indistinguishable there." + "description": "Current or terminal run status. `redacting` is transient, reported while a finished run's output is being scrubbed. `paused` means the run is waiting to be resumed — either held at a human-in-the-loop pause point, or left paused by a resume attempt that did not complete. Only the single-run response distinguishes the two, through `paused.automaticResumeWaitingReason`." }, "trigger": { "type": "string", @@ -4016,7 +4226,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Opaque cursor for the next page. Send it back as `cursor`; `null` means there is nothing further to fetch. Never construct one yourself." } }, "required": ["data", "nextCursor"], @@ -4070,7 +4280,7 @@ "cancelled", "queued" ], - "description": "Current or terminal run status. `redacting` is transient, reported while the output of a finished run is being scrubbed. `paused` means the run is not executing and is waiting to be resumed: either held at a human-in-the-loop pause point, or left paused because a resume attempt did not run to completion. The status alone does not say which. On the single-run response `paused.automaticResumeWaitingReason` distinguishes them: it is recorded whenever a resume attempt fails and cleared once a resume succeeds, so a null value means the run is waiting on human input. When the failure is not retryable or the automatic retries are exhausted, the reason is prefixed `Automatic resume requires manual intervention: `. Run-list items carry no `paused` object, so the two cases are indistinguishable there." + "description": "Current or terminal run status. `redacting` is transient, reported while a finished run's output is being scrubbed. `paused` means the run is waiting to be resumed — either held at a human-in-the-loop pause point, or left paused by a resume attempt that did not complete. Only the single-run response distinguishes the two, through `paused.automaticResumeWaitingReason`." }, "trigger": { "anyOf": [ @@ -4081,7 +4291,7 @@ "type": "null" } ], - "description": "Trigger type, or null before the run is recorded." + "description": "Trigger type that started the run. Backfilled as `api` for a run that is still queued, so it is populated from the first poll." }, "startedAt": { "anyOf": [ @@ -4092,7 +4302,7 @@ "type": "null" } ], - "description": "ISO 8601 start timestamp, or null while queued.", + "description": "ISO 8601 start timestamp. A queued run reports the time it was enqueued, so it is populated from the first poll.", "format": "date-time" }, "endedAt": { @@ -4185,7 +4395,7 @@ "type": "null" } ], - "description": "Reason automatic resume is waiting, or null when it is not waiting." + "description": "Why automatic resume is waiting, or null when it is not — on a paused run, null means it is waiting on human input. Recorded whenever a resume attempt fails and cleared once one succeeds. A non-retryable or exhausted failure is prefixed `Automatic resume requires manual intervention: `." }, "pausePointCount": { "type": "number", @@ -4242,7 +4452,7 @@ "type": "null" } ], - "description": "Structured execution failure, or null when none occurred." + "description": "Structured execution failure, or null when none occurred. Reclassified from the persisted error message, so `blockId`/`blockName`/`blockType` are absent and a block-level failure reports `EXECUTION_FAILED` here even when the same run reported `BLOCK_EXECUTION_FAILED` on its synchronous execute response." }, "output": { "anyOf": [ @@ -4270,7 +4480,7 @@ "type": "null" } ], - "description": "Selected block outputs when requested, otherwise null." + "description": "Outputs of the blocks named by `selectedOutputs`, or null when none were requested. Gated by `selectedOutputs` alone — `includeOutput` governs `output` only." } }, "required": [ @@ -4461,7 +4671,7 @@ "description": "Whether a paused execution was cancelled." }, "reason": { - "description": "Machine-readable cancellation outcome. Present on every cancellation, including full successes — it is not a partial-failure marker. `recorded` means cancellation was durably recorded (the normal success value). `redis_unavailable` and `redis_write_failed` mean the distributed cancellation signal could not be written, so an already-running execution may not observe the cancellation. `paused_event_publish_failed` and `paused_database_cancel_failed` name the failing step when cancelling a paused human-in-the-loop run.", + "description": "Machine-readable cancellation outcome, present on every cancellation including full successes. `recorded` is the success value. `redis_unavailable` and `redis_write_failed` mean the distributed cancellation signal was not written, so an already-running execution may not observe the cancellation. `paused_event_publish_failed` and `paused_database_cancel_failed` name the failing step for a paused run.", "type": "string", "enum": [ "recorded", @@ -4482,7 +4692,7 @@ ], "additionalProperties": false, "title": "Cancel workflow run result", - "description": "Outcome of a workflow run cancellation request. Cancelling a run that has already reached a terminal state (completed, failed, or cancelled) succeeds with no effect rather than returning an error — treat this endpoint as best-effort and poll the run to observe the final state." + "description": "Outcome of a workflow run cancellation request. Cancellation is best-effort: a run already in a terminal state succeeds with no effect, so poll the run to observe its final state." }, "CancelWorkflowRunResponse": { "type": "object", @@ -4519,11 +4729,15 @@ }, "path": { "type": "string", - "description": "Canonical folder path used as the public folder identifier." + "title": "Non-root folder path", + "description": "Canonical folder path used as the public folder identifier.", + "maxLength": 4096 }, "parentPath": { "type": "string", - "description": "Canonical parent path; `/` is the root." + "title": "Folder path", + "description": "Canonical parent path; `/` is the root.", + "maxLength": 4096 }, "createdAt": { "type": "string", @@ -4564,7 +4778,7 @@ "type": "null" } ], - "description": "Opaque cursor for the next page: send it back as `cursor` to continue, and stop when it is null. Most v2 lists page, so null means the last page was reached. A few are full-set lists that return their whole bounded result in one response and therefore always report null; those say so in the operation description. Either way, null means there is nothing further to fetch — never construct a cursor yourself." + "description": "Always `null` — this list has no `cursor` or `limit` param and returns its whole bounded set in one page. Present so the list can gain pages later without a shape change." } }, "required": ["data", "nextCursor"], @@ -4612,6 +4826,12 @@ } ] }, + "NonRootFolderPathInput": { + "title": "Non-root folder path input", + "description": "Non-root folder path. A missing leading slash is normalized before validation. Segments are percent-encoded, so a folder shown as \"New folder\" is `/New%20folder`: everything outside `A-Z a-z 0-9 - _ . ~` is escaped as uppercase hex, and only that exact encoding is accepted. A trailing slash, an empty segment, and a literal `.` or `..` segment are rejected. At most 64 segments and 4096 encoded bytes.", + "maxLength": 4096, + "type": "string" + }, "CreateWorkflowFolderRequest": { "type": "object", "properties": { @@ -4622,7 +4842,7 @@ }, "path": { "description": "Path of the folder to create.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, "required": ["workspaceId", "path"], @@ -4665,11 +4885,11 @@ }, "path": { "description": "Current folder path.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" }, "destinationPath": { "description": "New full path for the folder and its descendants.", - "type": "string" + "$ref": "#/components/schemas/NonRootFolderPathInput" } }, "required": ["workspaceId", "path", "destinationPath"], @@ -4682,7 +4902,9 @@ "properties": { "path": { "type": "string", - "description": "Path of the deleted workflow folder." + "title": "Folder path", + "description": "Path of the deleted workflow folder.", + "maxLength": 4096 }, "deleted": { "type": "boolean", diff --git a/apps/realtime/src/database/operations.ts b/apps/realtime/src/database/operations.ts index 1cf601ae965..4e7baa85d78 100644 --- a/apps/realtime/src/database/operations.ts +++ b/apps/realtime/src/database/operations.ts @@ -8,6 +8,7 @@ import { workflowEdges, workflowSubflows, } from '@sim/db' +import { withUtcTimestamps } from '@sim/db/timestamps' import { createLogger } from '@sim/logger' import { getActiveWorkflowContext } from '@sim/platform-authz/workflow' import { @@ -222,16 +223,20 @@ const connectionString = // Realtime process footprint = this socketDb pool + the shared @sim/db pool. const socketDb = drizzle( instrumentPoolClient( - postgres(connectionString, { - prepare: false, - // See `packages/db/db.ts` — skips the per-connection pg_type roundtrip. - fetch_types: false, - idle_timeout: 10, - connect_timeout: 20, - max: 10, - onnotice: () => {}, - connection: { application_name: process.env.DB_APP_NAME ?? 'sim-realtime' }, - }), + postgres( + connectionString, + // `withUtcTimestamps` — see `packages/db/timestamps.ts`. + withUtcTimestamps({ + prepare: false, + // See `packages/db/db.ts` — skips the per-connection pg_type roundtrip. + fetch_types: false, + idle_timeout: 10, + connect_timeout: 20, + max: 10, + onnotice: () => {}, + connection: { application_name: process.env.DB_APP_NAME ?? 'sim-realtime' }, + }) + ), 'socketDb' ), { schema } diff --git a/apps/realtime/src/handlers/file-doc-store.test.ts b/apps/realtime/src/handlers/file-doc-store.test.ts index 1662a8a3426..f5159c10cbc 100644 --- a/apps/realtime/src/handlers/file-doc-store.test.ts +++ b/apps/realtime/src/handlers/file-doc-store.test.ts @@ -1,6 +1,7 @@ /** * @vitest-environment node */ +import { sleep } from '@sim/utils/helpers' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import * as Y from 'yjs' @@ -15,6 +16,16 @@ interface Backing { seq: number /** Number of upcoming xAdd calls to fail with a transient error (to exercise publish retry). */ failXAdd: number + /** Set to fail every xRead the way node-redis does once a client has been closed. */ + readerClosed: boolean + /** Failed reads served, so a test can prove the loop is not spinning at the read cadence. */ + reads: number + /** When each FAILED read was attempted, so a test can measure one backoff interval exactly. */ + failedReadTimes: number[] + /** Reads that returned (the idle steady state) — the event that ends a failure streak. */ + idleReads: number + /** `connect()` calls, so a test can prove a closed reader is re-opened rather than abandoned. */ + connects: number } const state = vi.hoisted(() => ({ backing: null as Backing | null })) @@ -27,7 +38,11 @@ function makeClient(): any { return state.backing } const client: any = { - connect: async () => {}, + isOpen: true, + connect: async () => { + client.isOpen = true + b().connects++ + }, quit: async () => {}, on: () => client, duplicate: () => makeClient(), @@ -52,14 +67,24 @@ function makeClient(): any { ) }, xRead: async (streams: { key: string; id: string }[]) => { + b().reads++ + if (b().readerClosed) { + b().failedReadTimes.push(Date.now()) + client.isOpen = false + throw new Error('The client is closed') + } const res: { name: string; messages: { id: string; message: Record }[] }[] = [] for (const { key, id } of streams) { const after = (b().streams.get(key) ?? []).filter((e) => seqOf(e.id) > seqOf(id)) if (after.length) res.push({ name: key, messages: after.map((e) => ({ ...e })) }) } - if (res.length) return res - await new Promise((r) => setTimeout(r, 5)) + if (res.length) { + b().idleReads++ + return res + } + await sleep(5) + b().idleReads++ return null }, set: async (key: string, val: string, opts?: { NX?: boolean }) => { @@ -129,7 +154,17 @@ async function newStore(): Promise { describe('FileDocStore', () => { beforeEach(() => { - state.backing = { streams: new Map(), kv: new Map(), seq: 0, failXAdd: 0 } + state.backing = { + streams: new Map(), + kv: new Map(), + seq: 0, + failXAdd: 0, + readerClosed: false, + reads: 0, + failedReadTimes: [], + idleReads: 0, + connects: 0, + } stores = [] }) @@ -137,6 +172,85 @@ describe('FileDocStore', () => { await Promise.all(stores.map((s) => s.shutdown())) }) + /** + * A connection that stops serving reads used to spin the tailer at the read cadence — two attempts a + * second, one warning each, forever — while the task quietly stopped converging with every other one. + * The loop must back off instead, and re-open a client that was closed rather than reading a dead one. + */ + it('backs off and re-opens the reader when its connection is closed, instead of spinning', async () => { + const store = await newStore() + const doc = new Y.Doc() + await store.attachRoom(NAME, doc) + state.backing!.readerClosed = true + + state.backing!.connects = 0 // ignore the two `init` connects; count only recovery attempts + const before = state.backing!.reads + await sleep(3000) + const attempts = state.backing!.reads - before + + // A fixed 500ms retry manages 6–7 attempts in this window; backing off (500 → 1s → 2s → …) manages + // about 3. Exact counts are timing-dependent, so assert the property — it slowed down — not a number. + expect(attempts).toBeGreaterThan(0) + expect(attempts).toBeLessThanOrEqual(4) + // …and it tried to bring the connection back rather than leaving the tailer dead forever. + expect(state.backing!.connects).toBeGreaterThan(0) + doc.destroy() + }) + + /** + * The streak has to end on a read that RETURNS, not on one that carries messages: a blocking read + * timing out with nothing new is the idle steady state. Counting only message-bearing reads would + * keep a healed outage's streak alive through normal polling, so the next unrelated blip would open + * at the backoff cap — minutes of unnecessary split-brain — and log a count it never earned. + */ + it('ends the failure streak on an idle read, so a later blip starts over', async () => { + const store = await newStore() + const doc = new Y.Doc() + await store.attachRoom(NAME, doc) + + // Build a streak of two failures (the retries back off ~0.5s, then ~1s). + state.backing!.readerClosed = true + await vi.waitFor( + () => expect(state.backing!.failedReadTimes.length).toBeGreaterThanOrEqual(2), + { + timeout: 5000, + interval: 25, + } + ) + + // Redis comes back. Wait for a read to actually RETURN — waiting a fixed span instead is a race: + // the pending backoff can outlast it, no idle read lands, and the streak survives into the phase + // below, which then measures the wrong backoff and fails. That is an event, so wait on the event. + state.backing!.readerClosed = false + const idleBefore = state.backing!.idleReads + await vi.waitFor(() => expect(state.backing!.idleReads).toBeGreaterThan(idleBefore), { + timeout: 5000, + interval: 25, + }) + + // A fresh blip must retry at the START of the backoff curve, not partway up it. Assert the DELAY + // itself: counting attempts inside a fixed window cannot tell the two apart, because the jittered + // delay for a carried streak (1.6–2.4s) overlaps any window wide enough to catch a reset one. + // Measure FAILURE to FAILURE so the sample is exactly one backoff — a straggler successful read + // landing just after the flag flips would otherwise become the first sample and pass trivially. + state.backing!.readerClosed = true + state.backing!.failedReadTimes.length = 0 + await vi.waitFor( + () => expect(state.backing!.failedReadTimes.length).toBeGreaterThanOrEqual(2), + { + timeout: 6000, + interval: 25, + } + ) + const [first, second] = state.backing!.failedReadTimes + + // Streak reset ⇒ the first delay is 500ms ±20% ⇒ at most 600ms. Streak carried over ⇒ it is the + // third delay, 2000ms ±20% ⇒ at least 1600ms. The bound sits between them with room on both + // sides, so a loaded machine stretching the short sleep does not flip the verdict. + expect(second - first).toBeLessThan(1200) + doc.destroy() + }) + it('elects exactly one seeder across tasks (no split-brain seed)', async () => { const a = await newStore() const b = await newStore() diff --git a/apps/realtime/src/handlers/file-doc-store.ts b/apps/realtime/src/handlers/file-doc-store.ts index f7b8fc5180a..537f7f4db12 100644 --- a/apps/realtime/src/handlers/file-doc-store.ts +++ b/apps/realtime/src/handlers/file-doc-store.ts @@ -18,10 +18,12 @@ * client receives each update exactly once, from its own task's local broadcast — no adapter * amplification, and every task's doc stays converged. (Awareness/presence stay on the adapter: they * are ephemeral and need no convergence or replay.) - * - {@link attachRoom} does a synchronous catch-up read from the head of the stream when a task first - * opens a file, so a late-joining task (the normal case under autoscaling) loads the current shared - * state before its first client syncs. Catch-up + tail are seamless: the tailer resumes from the - * exact id catch-up stopped at. + * - {@link attachRoom} reads the stream from the head when a task first opens a file, and the relay + * AWAITS it before attaching a client, so a late-joining task (the normal case under autoscaling) + * holds the current shared state before its first client syncs — a client must never watch the + * catch-up land entry by entry, which is the document's edit history replaying on screen. Catch-up + + * tail are seamless: the tailer resumes from the exact id catch-up stopped at, and {@link catchUp} + * can re-run at any time for a caller that must converge without waiting on the tailer. * - The one-time seed is written via the atomic {@link seedIfEmpty} (append-iff-empty in one Redis * step), so exactly one task ever writes the seed cluster-wide (the fix for split-brain) — even if two * tasks race. {@link shouldSeed} is a Redis lock + empty-stream check layered on top ONLY as an @@ -158,6 +160,12 @@ const SEED_LOCK_TTL_MS = FILE_DOC_TIMEOUTS.seedRequestMs + 4_000 const STREAM_TTL_SEC = 600 /** Refresh every occupied stream's TTL on this cadence, so a live doc's stream never expires. */ const HEARTBEAT_MS = 60_000 +/** Cap on the delay between reconnection attempts — the strategy retries indefinitely (see `init`). */ +const RECONNECT_MAX_DELAY_MS = 3_000 +/** Cap on the reader's own retry backoff after a failed read. */ +const READER_RETRY_MAX_MS = 10_000 +/** After the first failure of a streak, log one reader failure in this many. */ +const READER_ERROR_LOG_EVERY = 20 const streamKey = (name: string) => `${STREAM_PREFIX}${name}` @@ -185,6 +193,18 @@ function applyEntryToDoc( } } +/** + * Whether stream id `id` sorts after `than`. A Redis stream id is `-`, so a lexicographic + * compare is wrong the moment the millisecond part changes digit length (`'9999-0' > '10000-0'`); + * compare the two parts numerically instead. The initial `'0'` (nothing applied) has no `-seq` part, + * which reads as sequence 0 — before every real entry. + */ +function isAfterStreamId(id: string, than: string): boolean { + const [ms, seq = '0'] = id.split('-') + const [thanMs, thanSeq = '0'] = than.split('-') + return Number(ms) === Number(thanMs) ? Number(seq) > Number(thanSeq) : Number(ms) > Number(thanMs) +} + /** Whether a doc carries the seed flag (mirrors the relay's `isDocSeeded`), so the store can tell the * one-time seed transition from a real post-seed edit without re-implementing the check divergently. */ function isDocSeeded(doc: Y.Doc): boolean { @@ -232,10 +252,17 @@ export class FileDocStore { const options = { url: this.redisUrl, socket: { - reconnectStrategy: (retries: number) => { - if (retries > 10) return new Error('FileDocStore Redis reconnection failed') - return Math.min(retries * 100, 3000) - }, + /** + * Never stop reconnecting. Returning an `Error` here tells node-redis to give up and CLOSE the + * client — and a closed client rejects every command with "The client is closed" for the rest of + * the process's life. So an outage longer than the retry budget does not degrade this task, it + * takes it out silently: its rooms stop receiving other tasks' updates, its own edits stop + * reaching the shared stream, seeds and locks fail, and the only symptom is a warning per retry. + * This process holds live documents whose sole convergence path is this connection, so a + * connection it can rebuild is always worth rebuilding. + */ + reconnectStrategy: (retries: number) => + backoffWithJitter(retries + 1, null, { baseMs: 100, maxMs: RECONNECT_MAX_DELAY_MS }), }, } this.write = createClient(options) @@ -260,10 +287,9 @@ export class FileDocStore { } /** - * Register a locally-opened room and load the shared state into its doc: read the whole stream from - * the head, apply every entry (origin {@link REDIS_ORIGIN}), and remember the last id so the tailer - * resumes exactly after it. A brand-new file has an empty stream and loads nothing (it is seeded - * shortly after, via {@link shouldSeed}). No-op when disabled. + * Register a locally-opened room and load the shared state into its doc ({@link catchUp}). A + * brand-new file has an empty stream and loads nothing (it is seeded shortly after, via + * {@link shouldSeed}). No-op when disabled. */ async attachRoom(name: string, doc: Y.Doc): Promise { if (!this.enabled || !this.write) return @@ -277,12 +303,31 @@ export class FileDocStore { realEdited: false, } this.rooms.set(name, room) + await this.catchUp(name) + } + + /** + * PULL the shared state into a registered room: read the stream and apply every entry the doc has + * not integrated yet (origin {@link REDIS_ORIGIN}), advancing `lastId` so the tailer resumes exactly + * after it. This is the ONLY way a room loads shared state, so a caller that must not depend on the + * tailer's asynchronous push — the join, which may not serve a client a half-assembled document — + * can converge on demand. Idempotent and safe to call repeatedly; no-op when disabled or the room is + * not registered (a fast open→close detached it). Never throws. + */ + async catchUp(name: string): Promise { + if (!this.enabled || !this.write) return + const room = this.rooms.get(name) + if (!room) return try { const entries = await this.write.xRange(streamKey(name), '-', '+') for (const entry of entries) { - // The room can be detached + its doc destroyed while catch-up is in flight (a fast open→close); - // stop touching it the moment that happens. + // The room can be detached + its doc destroyed while the read is in flight (a fast + // open→close); stop touching it the moment that happens. if (this.rooms.get(name) !== room) return + // Applying a Yjs update twice is a no-op, but `applyEntry`'s bookkeeping is not: re-applying + // the SEED after `seededObserved` latched would count it as a post-seed edit and let a + // compaction snapshot claim content no user ever typed. Skip what this room already holds. + if (!isAfterStreamId(entry.id, room.lastId)) continue this.applyEntry(room, entry.id, entry.message) } await this.write.expire(streamKey(name), STREAM_TTL_SEC) @@ -619,6 +664,7 @@ export class FileDocStore { * apply new entries. One blocking connection for the whole process regardless of open-file count. */ private async runReader(): Promise { + let failures = 0 while (this.running && this.read) { const snapshot = new Map(this.rooms) if (snapshot.size === 0) { @@ -630,6 +676,12 @@ export class FileDocStore { [...snapshot].map(([name, room]) => ({ key: streamKey(name), id: room.lastId })), { BLOCK: READ_BLOCK_MS, COUNT: READ_COUNT } ) + // The streak ends HERE, on the read returning at all — not further down once entries are + // applied. A blocking read that times out with nothing new is the idle steady state, and it + // proves the connection works just as well as one carrying messages; leaving the streak + // standing through it would keep an old outage's count alive indefinitely, so the next + // unrelated blip would open at the backoff cap and log a failure count it never earned. + failures = 0 if (!res) continue for (const stream of res) { const name = stream.name.slice(STREAM_PREFIX.length) @@ -642,12 +694,40 @@ export class FileDocStore { } } catch (error) { if (!this.running) break - logger.warn('FileDocStore reader error; retrying', { error: getErrorMessage(error) }) - await sleep(500) + await this.recoverReader(++failures, error) } } } + /** + * A failed read is either a transient blip or a connection that is gone, and this loop cannot tell + * them apart — so it backs off instead of retrying at the read cadence. Without that, a connection + * that cannot serve reads spins this loop forever at two attempts a second, one warning each, which + * is how an outage turns into thousands of identical log lines that bury the reason for it. + * + * It also re-opens a CLOSED client. node-redis reconnects a client that merely dropped, but never one + * it has closed; the strategy above no longer closes one, so this covers a client closed some other + * way (an explicit disconnect, a shutdown that raced a read) rather than leaving the tailer dead. + * + * Logs the first failure of a streak and then one in every {@link READER_ERROR_LOG_EVERY}, carrying + * the streak length, so a real outage stays visible without filling the log. + */ + private async recoverReader(failures: number, error: unknown): Promise { + if (failures === 1 || failures % READER_ERROR_LOG_EVERY === 0) { + logger.warn(`FileDocStore reader failed ${failures}x in a row; retrying`, { + error: getErrorMessage(error), + }) + } + await sleep(backoffWithJitter(failures, null, { baseMs: 500, maxMs: READER_RETRY_MAX_MS })) + if (this.running && this.read && !this.read.isOpen) { + await this.read.connect().catch((reconnectError) => { + logger.warn('FileDocStore could not re-open the reader connection', { + error: getErrorMessage(reconnectError), + }) + }) + } + } + /** * Snapshot-then-trim compaction: append a full-state snapshot and drop the older deltas it subsumes, * so the stream stays bounded while a fresh task can still catch up from the head. Lock-guarded so diff --git a/apps/realtime/src/handlers/file-doc.join-readiness.test.ts b/apps/realtime/src/handlers/file-doc.join-readiness.test.ts new file mode 100644 index 00000000000..9b7b6a1c7ec --- /dev/null +++ b/apps/realtime/src/handlers/file-doc.join-readiness.test.ts @@ -0,0 +1,305 @@ +/** + * @vitest-environment node + * + * The join's readiness contract, with the shared store ENABLED (`file-doc.test.ts` runs it disabled). + * + * A room loads its document from the file's Redis stream one entry at a time, into the same `Y.Doc` + * that fans every update out to the room. So a client attached while that is happening is not sent the + * document — it is sent the document's history, and it watches the history replay on screen (reload + * right after moving a block and the block moves again in front of you). These tests pin the fix: the + * join waits for the room to hold the whole document, so the client's first sync is authoritative. + */ +import { + FILE_DOC_EVENTS, + FILE_DOC_MESSAGE_TYPE, + FILE_DOC_SEED, +} from '@sim/realtime-protocol/file-doc' +import * as decoding from 'lib0/decoding' +import * as encoding from 'lib0/encoding' +import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest' +import * as syncProtocol from 'y-protocols/sync' +import * as Y from 'yjs' +import type { IRoomManager } from '@/rooms' + +const { mockAuthorizeRoom, mockFetchFileDocSeed } = vi.hoisted(() => ({ + mockAuthorizeRoom: vi.fn(), + mockFetchFileDocSeed: vi.fn(), +})) + +vi.mock('@sim/platform-authz/rooms', () => ({ authorizeRoom: mockAuthorizeRoom })) + +vi.mock('@/handlers/file-doc-app', () => ({ + fetchFileDocSeed: mockFetchFileDocSeed, + fetchFileDocMerge: vi.fn(), + fetchFileDocPersist: vi.fn().mockResolvedValue({ status: 'persisted', version: 1 }), +})) + +/** One in-memory Redis backing per test — only the stream/lock ops the store actually uses. */ +const backing = vi.hoisted(() => ({ + streams: new Map }[]>(), + kv: new Map(), + seq: 0, + /** Ticks of event-loop delay each xRange takes, modelling a remote (cross-region) Redis. */ + readDelayTicks: 0, +})) + +const seqOf = (id: string) => Number(id.split('-')[0]) + +vi.mock('redis', () => { + const makeClient = (): Record => { + const client: Record = { + connect: async () => {}, + quit: async () => {}, + on: () => client, + duplicate: () => makeClient(), + xAdd: async (key: string, _star: string, fields: Record) => { + const id = `${++backing.seq}-0` + const arr = backing.streams.get(key) ?? [] + arr.push({ id, message: { ...fields } }) + backing.streams.set(key, arr) + return id + }, + xRange: async (key: string) => { + for (let i = 0; i < backing.readDelayTicks; i++) await Promise.resolve() + return (backing.streams.get(key) ?? []).map((e) => ({ ...e })) + }, + xLen: async (key: string) => (backing.streams.get(key) ?? []).length, + xRead: async (streams: { key: string; id: string }[]) => { + const res: { name: string; messages: { id: string; message: Record }[] }[] = + [] + for (const { key, id } of streams) { + const after = (backing.streams.get(key) ?? []).filter((e) => seqOf(e.id) > seqOf(id)) + if (after.length) res.push({ name: key, messages: after.map((e) => ({ ...e })) }) + } + if (res.length) return res + await new Promise((r) => setTimeout(r, 5)) + return null + }, + set: async (key: string, val: string, opts?: { NX?: boolean }) => { + if (opts?.NX && backing.kv.has(key)) return null + backing.kv.set(key, val) + return 'OK' + }, + eval: async (script: string, opts: { keys: string[]; arguments: string[] }) => { + const [key] = opts.keys + if (script.includes('xlen')) { + const [field, value] = opts.arguments + const arr = backing.streams.get(key) ?? [] + if (arr.length > 0) return 0 + arr.push({ id: `${++backing.seq}-0`, message: { [field]: value } }) + backing.streams.set(key, arr) + return 1 + } + const [token] = opts.arguments + if (backing.kv.get(key) === token) { + backing.kv.delete(key) + return 1 + } + return 0 + }, + expire: async () => 1, + get: async (key: string) => backing.kv.get(key) ?? null, + exists: async (key: string) => (backing.kv.has(key) ? 1 : 0), + } + return client + } + return { createClient: () => makeClient() } +}) + +import { cleanupFileDocForSocket, setupWorkspaceFileDocHandlers } from '@/handlers/file-doc' +import { getFileDocStore, initFileDocStore } from '@/handlers/file-doc-store' + +const FILE_ID = 'file-1' +const ROOM_NAME = `workspace-file-doc:${FILE_ID}` +const STREAM_KEY = `filedoc:stream:${ROOM_NAME}` +const FIELD = 'default' + +type Handler = (payload?: unknown) => Promise | void + +interface FakeSocket { + id: string + emit: (event: string, payload: unknown) => void + rooms: Set +} + +/** + * An `io` that actually DELIVERS: a room emit reaches every socket that joined that room, so a frame + * the relay fans out mid-assembly lands on the joiner's `emit` exactly as it would in the browser. + * Recording the emits without routing them would hide the very thing these tests are about. + */ +function createIo(sockets: FakeSocket[]) { + const emitTo = (target: string, except: string | null, event: string, payload: unknown) => { + for (const socket of sockets) { + if (socket.id === except || !socket.rooms.has(target)) continue + socket.emit(event, payload) + } + } + const to = vi.fn((target: string) => ({ + except: (exclude: string) => ({ + emit: (event: string, payload: unknown) => emitTo(target, exclude, event, payload), + }), + emit: (event: string, payload: unknown) => emitTo(target, null, event, payload), + })) + return { + to, + in: vi.fn(() => ({ socketsLeave: () => {} })), + local: { to }, + } as unknown as IRoomManager['io'] +} + +function setup(id: string, sockets: FakeSocket[]) { + const handlers: Record = {} + const rooms = new Set() + const socket = { + id, + userId: 'user-1', + userName: 'Test User', + userImage: 'avatar.png', + disconnected: false, + rooms, + on: vi.fn((event: string, handler: Handler) => { + handlers[event] = handler + }), + emit: vi.fn(), + join: vi.fn((name: string) => rooms.add(name)), + leave: vi.fn((name: string) => rooms.delete(name)), + } + sockets.push(socket as unknown as FakeSocket) + setupWorkspaceFileDocHandlers( + socket as unknown as Parameters[0], + { isReady: () => true, io: createIo(sockets) } as unknown as IRoomManager + ) + return { socket, handlers } +} + +/** Append a Yjs update to the file's stream, exactly as `publish`/`seedIfEmpty` would. */ +function appendToStream(update: Uint8Array): void { + const arr = backing.streams.get(STREAM_KEY) ?? [] + arr.push({ id: `${++backing.seq}-0`, message: { u: Buffer.from(update).toString('base64') } }) + backing.streams.set(STREAM_KEY, arr) +} + +/** + * A warm room's history: the seed, then a later edit — the "I moved a block, then reloaded" case. + * Returns the markdown-equivalent text of each state. + */ +function seedWarmStreamHistory(): { intermediate: string; final: string } { + const doc = new Y.Doc() + doc.getText(FIELD).insert(0, 'AAA') + doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.flag, true) + appendToStream(Y.encodeStateAsUpdate(doc)) + const afterSeed = Y.encodeStateVector(doc) + doc.getText(FIELD).insert(0, 'BBB') + appendToStream(Y.encodeStateAsUpdate(doc, afterSeed)) + doc.destroy() + return { intermediate: 'AAA', final: 'BBBAAA' } +} + +/** Every document state this socket was ever shown, in order. */ +function statesDeliveredTo(socket: { emit: ReturnType }): string[] { + const clientDoc = new Y.Doc() + const states: string[] = [] + for (const [event, payload] of socket.emit.mock.calls) { + if (event !== FILE_DOC_EVENTS.MESSAGE || !(payload instanceof Uint8Array)) continue + const decoder = decoding.createDecoder(payload) + if (decoding.readVarUint(decoder) !== FILE_DOC_MESSAGE_TYPE.SYNC) continue + syncProtocol.readSyncMessage(decoder, encoding.createEncoder(), clientDoc, null) + const text = clientDoc.getText(FIELD).toString() + if (text !== (states.at(-1) ?? '')) states.push(text) + } + clientDoc.destroy() + return states +} + +/** Let anything the join left running (a catch-up, a seed) settle, so a frame it fans out afterwards + * is counted — that late delivery IS the replay these tests exist to rule out. */ +async function flushPendingWork(): Promise { + for (let i = 0; i < 20; i++) await Promise.resolve() +} + +/** Ask the server for its state the way a client does after the join ack. */ +function requestSyncStep2(handlers: Record): void { + const encoder = encoding.createEncoder() + encoding.writeVarUint(encoder, FILE_DOC_MESSAGE_TYPE.SYNC) + syncProtocol.writeSyncStep1(encoder, new Y.Doc()) + handlers[FILE_DOC_EVENTS.MESSAGE](encoding.toUint8Array(encoder)) +} + +describe('file-doc join readiness (shared store enabled)', () => { + /** Every socket the test created, so a room emit can be routed to its members. */ + const sockets: FakeSocket[] = [] + + // One store for the whole file: `initFileDocStore` is idempotent once enabled, so a per-test + // shutdown would leave every later test running against a store with closed clients. + beforeAll(async () => { + await initFileDocStore('redis://fake') + }) + + afterAll(async () => { + await getFileDocStore().shutdown() + }) + + beforeEach(() => { + vi.clearAllMocks() + backing.streams.clear() + backing.kv.clear() + backing.seq = 0 + backing.readDelayTicks = 0 + mockAuthorizeRoom.mockResolvedValue({ + allowed: true, + status: 200, + workspaceId: 'ws-1', + workspacePermission: 'write', + }) + mockFetchFileDocSeed.mockResolvedValue(null) + }) + + afterEach(() => { + cleanupFileDocForSocket('socket-1', createIo(sockets), true) + sockets.length = 0 + }) + + it('hands a joiner the final document, never the room history it was rebuilt from', async () => { + const { intermediate, final } = seedWarmStreamHistory() + // The catch-up read is not instantaneous — the case that made this visible is a cross-region Redis. + backing.readDelayTicks = 6 + const { socket, handlers } = setup('socket-1', sockets) + + await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: FILE_ID, clientId: 1 }) + requestSyncStep2(handlers) + await flushPendingWork() + + // One state, and it is the final one: the client never saw the pre-move document. + expect(statesDeliveredTo(socket)).toEqual([final]) + expect(statesDeliveredTo(socket)).not.toContain(intermediate) + }) + + it('does not fetch a seed for a room the stream can already reconstruct', async () => { + seedWarmStreamHistory() + const { handlers } = setup('socket-1', sockets) + + await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: FILE_ID, clientId: 1 }) + + expect(mockFetchFileDocSeed).not.toHaveBeenCalled() + }) + + it('pulls a seed another writer put in the stream instead of waiting for the tailer to push it', async () => { + // The seed lock is held by a writer whose room has since been dropped (a fast open→close on a + // freshly created file), and its seed lands in the stream. Waiting to be told about it is what + // left a new file un-editable until the client's readiness deadline lapsed; the join reads it. + backing.kv.set(`filedoc:seedlock:${ROOM_NAME}`, 'held-by-a-writer-that-is-gone') + const doc = new Y.Doc() + doc.getText(FIELD).insert(0, 'seeded by the writer that held the lock') + doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.flag, true) + appendToStream(Y.encodeStateAsUpdate(doc)) + doc.destroy() + + const { socket, handlers } = setup('socket-1', sockets) + await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: FILE_ID, clientId: 1 }) + requestSyncStep2(handlers) + await flushPendingWork() + + expect(mockFetchFileDocSeed).not.toHaveBeenCalled() + expect(statesDeliveredTo(socket)).toEqual(['seeded by the writer that held the lock']) + }) +}) diff --git a/apps/realtime/src/handlers/file-doc.test.ts b/apps/realtime/src/handlers/file-doc.test.ts index 938092d4484..4ba34c83de1 100644 --- a/apps/realtime/src/handlers/file-doc.test.ts +++ b/apps/realtime/src/handlers/file-doc.test.ts @@ -570,22 +570,42 @@ describe('setupWorkspaceFileDocHandlers', () => { expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toBe('# From server') }) - it('seeds the document only once from the server across concurrent joiners of the same file', async () => { + it('seeds once across concurrent joiners, and every one of them waits for that seed', async () => { // Keep the first seed fetch IN FLIGHT so the doc is still unseeded when the second socket joins: - // that forces the dedup onto `serverSeedStarted` (the in-flight guard) rather than `isDocSeeded`. + // that forces the dedup onto the in-flight seed rather than `isDocSeeded`. Both joins must WAIT + // for it — a joiner answered before the seed would be handed an empty document and would then + // watch the content arrive as a live update. let resolveSeed: (v: { update: Uint8Array; version: number } | null) => void = () => {} mockFetchFileDocSeed.mockReturnValueOnce(new Promise((resolve) => (resolveSeed = resolve))) const { io } = createIo() const a = setup('socket-a', io) const b = setup('socket-b', io) - await a.handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) - await b.handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 2 }) - // Second join happened with the fetch still pending; only after this does the seed land. + const joinA = a.handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) + const joinB = b.handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 2 }) + await flushMicrotasks() + + // The second join found the seed already in flight, so it does not start another one — and + // neither join has been answered yet. expect(mockFetchFileDocSeed).toHaveBeenCalledTimes(1) + expect(joinSuccessFileId(a.socket)).toBeUndefined() + expect(joinSuccessFileId(b.socket)).toBeUndefined() + resolveSeed(seedResult('# From server')) - await flushMicrotasks() + await Promise.all([joinA, joinB]) expect(mockFetchFileDocSeed).toHaveBeenCalledTimes(1) + + // The joiner that never triggered the fetch is served the seeded document all the same. + b.socket.emit.mockClear() + b.handlers[FILE_DOC_EVENTS.MESSAGE]( + frame(FILE_DOC_MESSAGE_TYPE.SYNC, (e) => syncProtocol.writeSyncStep1(e, new Y.Doc())) + ) + const reply = b.socket.emit.mock.calls.find( + ([event, payload]) => event === FILE_DOC_EVENTS.MESSAGE && payload instanceof Uint8Array + ) + const clientDoc = new Y.Doc() + applySyncReply(reply?.[1] as Uint8Array, clientDoc) + expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toBe('# From server') }) it('marks an empty/absent-file doc seeded so clients still reach readiness', async () => { @@ -640,43 +660,58 @@ describe('setupWorkspaceFileDocHandlers', () => { expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toBe('# Recovered') }) - it('does not seed a room that was dropped while the seed fetch was in flight', async () => { + it('does not seed a room the joiner abandoned while the seed fetch was in flight', async () => { let resolveSeed: (v: { update: Uint8Array; version: number } | null) => void = () => {} mockFetchFileDocSeed.mockReturnValueOnce(new Promise((resolve) => (resolveSeed = resolve))) const { io } = createIo() - const { handlers } = setup('socket-1', io) + const { socket, handlers } = setup('socket-1', io) - await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) - // The only owner leaves → the room (and its doc) is destroyed while the fetch is still pending. - cleanupFileDocForSocket('socket-1', io, true) + const joining = handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) + await flushMicrotasks() + // The client leaves before the room finished assembling → the join aborts and drops the room it + // was preparing (nothing else owns it). + handlers[FILE_DOC_EVENTS.LEAVE]({ fileId: 'file-1' }) // Resolving now must not touch the destroyed doc or throw (liveness re-check after the await). resolveSeed(seedResult('# Too late')) - await expect(flushMicrotasks()).resolves.toBeUndefined() + await expect(joining).resolves.toBeUndefined() + expect(joinSuccessFileId(socket)).toBeUndefined() + expect(socket.join).not.toHaveBeenCalled() }) - it('still seeds when content was synced into the doc before the seed returned', async () => { - // Defensive: the guard is `isDocSeeded`, NOT doc-emptiness. In practice a fresh client never - // writes ahead of the seed (@tiptap/y-tiptap suppresses the empty-paragraph placeholder and real - // edits are readiness-gated), but even if some update landed content in the doc before the seed - // fetch resolved, the seed must still apply and set the flag — or the client's - // `synced && initialContentLoaded` gate would never open. + it('attaches a client only once the document is whole — no empty sync, no frames before it', async () => { + // The room assembles itself into the same doc that fans updates out to its room, so a socket + // attached mid-assembly receives the document's history rather than the document. Nothing about + // the client exists in the room until the seed has landed: no membership, no sync, and any frame + // it sends meanwhile is not applied. let resolveSeed: (v: { update: Uint8Array; version: number } | null) => void = () => {} mockFetchFileDocSeed.mockReturnValueOnce(new Promise((resolve) => (resolveSeed = resolve))) const { io } = createIo() const { socket, handlers } = setup('socket-1', io) - await handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) + const joining = handlers[FILE_DOC_EVENTS.JOIN]({ fileId: 'file-1', clientId: 1 }) + await flushMicrotasks() - // The client syncs a placeholder update — content in the doc, but no seed flag. - const placeholder = new Y.Doc() - placeholder.getText(FILE_DOC_FIELD).insert(0, 'x') + expect(socket.join).not.toHaveBeenCalled() + expect(joinSuccessFileId(socket)).toBeUndefined() + expect( + socket.emit.mock.calls.some( + ([event, payload]) => event === FILE_DOC_EVENTS.MESSAGE && payload instanceof Uint8Array + ) + ).toBe(false) + + // A document frame sent before the join was answered reaches an unbound socket and is dropped. + const early = new Y.Doc() + early.getText(FILE_DOC_FIELD).insert(0, 'too early') handlers[FILE_DOC_EVENTS.MESSAGE]( frame(FILE_DOC_MESSAGE_TYPE.SYNC, (e) => - syncProtocol.writeUpdate(e, Y.encodeStateAsUpdate(placeholder)) + syncProtocol.writeUpdate(e, Y.encodeStateAsUpdate(early)) ) ) + resolveSeed(seedResult('# Seeded')) - await flushMicrotasks() + await joining + expect(joinSuccessFileId(socket)).toBe('file-1') + // The first thing the client is served is the finished document — content and seed flag together. socket.emit.mockClear() handlers[FILE_DOC_EVENTS.MESSAGE]( frame(FILE_DOC_MESSAGE_TYPE.SYNC, (e) => syncProtocol.writeSyncStep1(e, new Y.Doc())) @@ -687,7 +722,7 @@ describe('setupWorkspaceFileDocHandlers', () => { const clientDoc = new Y.Doc() applySyncReply(reply?.[1] as Uint8Array, clientDoc) expect(clientDoc.getMap(FILE_DOC_SEED.configMap).get(FILE_DOC_SEED.flag)).toBe(true) - expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toContain('# Seeded') + expect(clientDoc.getText(FILE_DOC_FIELD).toString()).toBe('# Seeded') }) it('merges a copilot edit into a seeded live room and relays it to editors', async () => { diff --git a/apps/realtime/src/handlers/file-doc.ts b/apps/realtime/src/handlers/file-doc.ts index a0152fd85f6..1e29f476de9 100644 --- a/apps/realtime/src/handlers/file-doc.ts +++ b/apps/realtime/src/handlers/file-doc.ts @@ -131,9 +131,13 @@ interface FileDocRoom { /** socketId → (clientId → its presence ownership). A socket owns one entry per collaborative provider * it mounted for this file (see {@link FileDocOwner}); an empty inner map is never kept. */ owners: Map> - /** True once the server-side seed fetch has started, so concurrent joins don't each fetch. - * Reset on a fetch FAILURE so a later join can retry (a genuinely empty file stays empty). */ - serverSeedStarted: boolean + /** + * The in-flight server seed for this room, or `null`. Concurrent joins await THIS promise rather + * than each starting a fetch — and, unlike a "started" boolean, awaiting it is what lets a second + * joiner be served a document that is already seeded instead of an empty one. Cleared when it + * settles, so a failed seed is re-attempted by a later join (a genuinely empty file stays empty). + */ + seeding: Promise | null /** The workspace this file belongs to, captured at join — needed to persist back to markdown. */ workspaceId: string | null /** The last collaborator to edit here, for persist attribution (blob metadata) only. */ @@ -170,6 +174,17 @@ interface FileDocRoom { * {@link FileDocStore.isAgentStreaming} flag. `0` when no agent stream is active. */ agentStreamingUntil: number + /** + * Resolves once this room's doc reflects the file's shared stream (see {@link FileDocStore.catchUp}). + * Never rejects — the catch-up logs and gives up — so awaiting it can never fail a join. + */ + hydrated: Promise + /** + * How many joins are currently preparing this room. A room is created by the first join and has no + * owner until that join commits, so without this a concurrent last-leave would tear down the very + * document being assembled. A room with a join in flight is not idle. + */ + pendingJoins: number } /** Live documents keyed by Socket.IO room name. Module-global: one Y.Doc per file. */ @@ -360,7 +375,12 @@ async function flushPersist(name: string, room: FileDocRoom, final: boolean): Pr } if (result.status === 'persisted') { room.syncedVersion = Math.max(room.syncedVersion ?? 0, result.version) - void store.setSyncedVersion(name, result.version) + // AWAITED, unlike every other version write: the room's own copy dies with the room, so this + // cluster key is the only record that survives a teardown or a process restart. Fire-and-forget + // here means a task that exits in the moments after a write comes back holding a version older + // than the file's, and — since a conflict neither writes nor advances the token — never persists + // that document again. One round trip after a blob write is not a cost worth that. + await store.setSyncedVersion(name, result.version) return } // status === 'conflict': the durable file advanced out-of-band since our If-Match token. We do NOT @@ -412,6 +432,12 @@ function isDocSeeded(doc: Y.Doc): boolean { return doc.getMap(FILE_DOC_SEED.configMap).get(FILE_DOC_SEED.flag) === true } +/** The identity of the document this doc holds ({@link FILE_DOC_SEED.docIdKey}), if it carries one. */ +function docIdOf(doc: Y.Doc): string | undefined { + const docId = doc.getMap(FILE_DOC_SEED.configMap).get(FILE_DOC_SEED.docIdKey) + return typeof docId === 'string' ? docId : undefined +} + /** * Decode the client IDs an awareness update carries, without applying it, to * check a frame only touches its sender's own presence. Mirrors the wire format @@ -435,10 +461,13 @@ function awarenessUpdateClientIds(update: Uint8Array): number[] { * memory. Before dropping, flush the converged doc back to durable markdown (the last collaborator on * this task leaving) and detach from the shared stream. A later joiner re-creates it — catching up * from the stream if the doc is still live on another task, or re-seeding from markdown otherwise. + * + * A room being PREPARED for a join is not idle even though it has no owners yet: tearing it down there + * would drop the hydration/seed that join is waiting on, and the join would have to start over. */ function destroyRoomIfIdle(name: string) { const room = fileDocRooms.get(name) - if (!room || room.owners.size > 0) return + if (!room || room.owners.size > 0 || room.pendingJoins > 0) return room.persistDeadline = null if (room.persistTimer) { clearTimeout(room.persistTimer) @@ -469,40 +498,83 @@ export async function flushAllFileDocRooms(): Promise { } /** - * Seed a room's document server-side, once, on the first join: ask the app to build the seed (the - * file's current markdown → Yjs, through the exact editor engine) and apply it, which relays the - * content to every connected client via `doc.on('update')`. No client is elected to import content. + * Bring a room's document to its AUTHORITATIVE state — reflecting the file's shared stream and + * carrying its seed — so the join can attach a client to a document that is already whole. Never + * rejects: a room that cannot be seeded is served unseeded, which the client's readiness deadline + * turns into its read-only fallback, exactly as an unreachable relay does. + */ +async function ensureRoomReady( + name: string, + room: FileDocRoom, + workspaceId: string | null +): Promise { + await room.hydrated + // The room can be dropped and re-created while the catch-up is in flight (a fast open→close); the + // join re-checks identity after this and abandons a stale room rather than serving from it. + if (fileDocRooms.get(name) !== room || !workspaceId) return + await ensureServerSeed(name, room, workspaceId) +} + +/** + * Seed a room's document server-side, once: ask the app to build the seed (the file's current markdown + * → Yjs, through the exact editor engine) and apply it. No client is elected to import content. + * + * MEMOIZED on the room, so concurrent joins await the same seed instead of the second one being served + * an empty document while the first one's fetch is still in flight. Cleared when it settles: a failed + * seed is re-attempted by the next join (a genuinely empty file stays empty and needs no retry). * * `isDocSeeded` is the sufficient guard: content only ever reaches the doc alongside the seed flag * (this seed, or a client's offline fallback), so an unseeded doc is genuinely empty and safe to seed. * A genuinely empty/missing file returns `null` (a read error throws instead), so still set the flag — - * an empty doc must reach readiness, not wait forever. After the fetch, re-check the room is still - * live and unseeded (an owner may have left, or a client seeded it, while the fetch was in flight). - * - * Recovery on failure is deliberately simple — no in-room retry loop: a single attempt bounded by a - * timeout shorter than the client's readiness deadline, then release the guard. A transient failure - * is re-attempted by the next join/reconnect; a persistent one lets the connected client's readiness - * deadline lapse into its read-only fallback. (An in-room backoff retry can outlast that client - * deadline, so it would keep trying a doc the client has already given up on — worse, not better.) + * an empty doc must reach readiness, not wait forever. */ -async function ensureServerSeed( +function ensureServerSeed(name: string, room: FileDocRoom, workspaceId: string): Promise { + if (isDocSeeded(room.doc)) return Promise.resolve() + room.seeding ??= runServerSeed(name, room, workspaceId).finally(() => { + room.seeding = null + }) + return room.seeding +} + +/** + * Whichever task wins the seed lock writes the seed; the others must end up holding the SAME seed + * before they serve anyone. They pull it, on this cadence, rather than waiting for the tailer to push + * it: a join's readiness may not depend on an asynchronous subscriber, because when that delivery is + * late or lost the client sits on an empty document until its readiness deadline lapses and the file + * opens read-only. Bounded by the longest a legitimate seed can take (the winner's own fetch bound), + * which stays inside the client's readiness deadline — see {@link FILE_DOC_TIMEOUTS}. + */ +const SEED_WAIT_RETRY_MS = 150 + +async function runServerSeed(name: string, room: FileDocRoom, workspaceId: string): Promise { + const store = getFileDocStore() + const deadline = Date.now() + FILE_DOC_TIMEOUTS.seedRequestMs + while (fileDocRooms.get(name) === room && !isDocSeeded(room.doc)) { + // Exactly one task across the cluster builds the seed; the others receive it via the stream (the + // fix for split-brain seeding). Returns a lock token here (single-pod: a sentinel token). + const token = await store.shouldSeed(name) + if (token) { + await seedUnderLock(name, room, workspaceId, token) + return + } + // No token: a peer holds the lock with its fetch in flight, or the stream is already seeded (which + // includes a PRIOR room for this same file whose seed landed after we read the stream). Either way + // the seed can only appear in the stream, so read it rather than wait to be told. + await store.catchUp(name) + if (isDocSeeded(room.doc) || Date.now() >= deadline) return + await sleep(SEED_WAIT_RETRY_MS) + } +} + +/** Fetch, publish, and apply the seed while holding the cluster's seed lock for this file. */ +async function seedUnderLock( name: string, room: FileDocRoom, - workspaceId: string + workspaceId: string, + token: string ): Promise { - if (room.serverSeedStarted || isDocSeeded(room.doc)) return - room.serverSeedStarted = true const store = getFileDocStore() - // Exactly one task across the cluster builds the seed; the others receive it via the stream (the fix - // for split-brain seeding). Returns a lock token here (single-pod: a sentinel token). - const token = await store.shouldSeed(name) - if (!token) { - // A peer is seeding (or already did). Release our guard so a later join can retry if the seed never - // arrives (e.g. the seeder died); the stream / this doc being seeded makes a retry safe. - room.serverSeedStarted = false - return - } - // We hold the seed lock — release it on EVERY exit from here (one `finally`, impossible to leak). + // Release the lock on EVERY exit from here (one `finally`, impossible to leak). try { const seed = await fetchFileDocSeed(workspaceId, room.fileId) if (fileDocRooms.get(name) !== room || isDocSeeded(room.doc)) return @@ -533,15 +605,13 @@ async function ensureServerSeed( if (didSeed) { Y.applyUpdate(room.doc, seedUpdate, SEED_ORIGIN) } else { - // A peer seeded first: its seed arrives via the tailer, so we must NOT apply our own — a second, - // different-client-id seed IS the split-brain. Clear the guard so a later join can retry if that - // peer seed somehow never lands (e.g. a fail-closed `xLen` error made `shouldSeed` skip a genuinely - // empty stream); a real peer-seed makes the retry a no-op. - room.serverSeedStarted = false + // A peer won the atomic append: we must NOT apply our own — a second, different-client-id seed IS + // the split-brain. Read THEIRS out of the stream instead of waiting for the tailer to deliver it, + // so this room is seeded by the time the caller is told it is ready. + await store.catchUp(name) } } catch (error) { logger.warn(`Server seed failed for file ${room.fileId} (workspace ${workspaceId})`, error) - room.serverSeedStarted = false } finally { await store.releaseSeedLock(name, token) } @@ -725,12 +795,14 @@ function getOrCreateRoom(io: Server, ref: RoomRef): FileDocRoom { // The server holds no cursor of its own; it only relays clients' awareness. awareness.setLocalState(null) + // Started BEFORE the room is registered so no join can observe a room without its hydration handle. + const hydrated = getFileDocStore().attachRoom(name, doc) const room: FileDocRoom = { fileId: ref.id, doc, awareness, owners: new Map(), - serverSeedStarted: false, + seeding: null, workspaceId: null, lastEditorUserId: null, edited: false, @@ -739,6 +811,8 @@ function getOrCreateRoom(io: Server, ref: RoomRef): FileDocRoom { persistDeadline: null, syncedVersion: null, agentStreamingUntil: 0, + hydrated, + pendingJoins: 0, } // Register synchronously BEFORE the async catch-up so a concurrent join sees this room, not a second. fileDocRooms.set(name, room) @@ -818,10 +892,6 @@ function getOrCreateRoom(io: Server, ref: RoomRef): FileDocRoom { broadcast(io, name, encoding.toUint8Array(encoder), originSocketId(origin)) }) - // Load the shared state into the doc and start tailing the stream (fire-and-forget: content streams - // in via `doc.on('update')` as it lands, mirroring the fire-and-forget seed below). Disabled → no-op. - void getFileDocStore().attachRoom(name, doc) - return room } @@ -1091,117 +1161,143 @@ export function setupWorkspaceFileDocHandlers( // awareness). Resolved here so the generation guard below also covers this await. const avatarUrl = await resolveAvatarUrl(socket, userId) - // Re-check access immediately before registering, mirroring the workflow join: the - // access re-validation sweep records a revocation BEFORE it evicts, so a join that - // authorized just before the revocation must not complete afterwards and re-bind - // the socket to the document. This RE-RESOLVES rather than peeking the cache — a - // peek treats an expired entry as unknown and fails open, which a join stalled - // longer than the cache TTL would slip straight through. Normally a cache hit (this - // join's own authorize just warmed it), so it costs no extra query. - const currentPermission = await resolveCurrentRoomPermission(userId, room, FILE_DOC_ACTION) - if (!satisfiesRoomMembership(currentPermission, ROOM_TYPES.WORKSPACE_FILE_DOC)) { - logger.warn(`User ${userId} lost write access to file ${fileId} before the join completed`) - emitJoinError(socket, fileId, 'Access denied to file', 'ACCESS_DENIED', false) - return - } - - // Abort a JOIN superseded during authorization/identity resolution: the socket - // disconnected, or a newer JOIN (a document switch) bumped the generation. Registering - // here would leak a dead socket's room or bind the socket to the wrong document. - // Last await before the commit, so nothing can interleave between the access - // re-check above and the registration below. - if (socket.disconnected || joinGeneration.get(socket.id) !== generation) return - const entry = getOrCreateRoom(io, room) + // The workspace the server-side persist writes back to — and what the seed is built from, so it + // must be captured BEFORE the room is prepared below. + if (authorized.workspaceId) entry.workspaceId = authorized.workspaceId - // A client id must be owned by at most one user, or a peer could bind an active - // collaborator's id and pass the per-frame ownership check to spoof/clear its caret. - // Distinguish a reconnect from a spoof by the owning user: the same user reclaiming its - // own client id (a dropped socket reconnecting reuses the Yjs client id, and its prior - // socket may not be cleaned up yet) takes over the stale binding; a DIFFERENT user is - // rejected. This runs BEFORE any teardown of the socket's current binding below, so a - // rejected rebind — even during a document switch — leaves the socket's existing document - // and caret untouched. - for (const [otherSid, clientMap] of entry.owners) { - if (otherSid === socket.id) continue - const owner = clientMap.get(clientId) - if (owner === undefined) continue - if (owner.userId !== userId) { - emitJoinError(socket, fileId, 'Client id already in use', 'CLIENT_ID_IN_USE', false) + // Hold the room open across the awaits below: it has no owner until this join commits, so a + // concurrent last-leave would otherwise tear down the very document being prepared. + entry.pendingJoins += 1 + try { + // A client is attached to a WHOLE document or to nothing. A room assembles itself from the + // shared stream and the server seed, and both land in the same Y.Doc that fans every update out + // to its room — so a socket attached mid-assembly is not sent the document, it is sent the + // document's history, and it watches that replay on screen (reload right after moving a block + // and the block moves again in front of you). Waiting here is what makes the handshake below + // authoritative: the client's first sync IS the finished document, in one message. + await ensureRoomReady(name, entry, entry.workspaceId) + + // Re-check access immediately before registering, mirroring the workflow join: the + // access re-validation sweep records a revocation BEFORE it evicts, so a join that + // authorized just before the revocation must not complete afterwards and re-bind + // the socket to the document. This RE-RESOLVES rather than peeking the cache — a + // peek treats an expired entry as unknown and fails open, which a join stalled + // longer than the cache TTL would slip straight through. Normally a cache hit (this + // join's own authorize just warmed it), so it costs no extra query. + const currentPermission = await resolveCurrentRoomPermission(userId, room, FILE_DOC_ACTION) + if (!satisfiesRoomMembership(currentPermission, ROOM_TYPES.WORKSPACE_FILE_DOC)) { + logger.warn( + `User ${userId} lost write access to file ${fileId} before the join completed` + ) + emitJoinError(socket, fileId, 'Access denied to file', 'ACCESS_DENIED', false) return } - // Same user reclaiming its client id on a stale prior socket: evict just THAT clientID's binding - // + caret from the old socket. If that leaves the old socket with no providers, also drop its - // room mapping + Socket.IO membership so it can no longer send document (sync) frames - // (handleMessage's SYNC path gates on socketToRoomName, not owners); an old socket that still - // hosts OTHER providers keeps them. Done inline rather than via cleanupFileDocForSocket, which - // could destroyRoomIfIdle the room we're joining. - clientMap.delete(clientId) - awarenessProtocol.removeAwarenessStates(entry.awareness, [clientId], null) - if (clientMap.size === 0) { - entry.owners.delete(otherSid) - socketToRoomName.delete(otherSid) - io.in(otherSid).socketsLeave(name) - } - } - // Only now that the rebind is guaranteed to succeed, leave a previously-joined document if - // switching (a socket edits at most one). A duplicate join of the SAME room falls through - // and simply re-runs the sync handshake, idempotently. - const currentName = socketToRoomName.get(socket.id) - if (currentName && currentName !== name) { - socket.leave(currentName) - cleanupFileDocForSocket(socket.id, io) - } + // Abort a JOIN superseded while the room was being prepared: the socket disconnected, a newer + // JOIN (a document switch) bumped the generation, or the room was dropped and re-created. + // Registering here would leak a dead socket's room, bind the socket to the wrong document, or + // attach it to a doc no longer registered. Last await before the commit, so nothing can + // interleave between the access re-check above and the registration below. + if ( + socket.disconnected || + joinGeneration.get(socket.id) !== generation || + fileDocRooms.get(name) !== entry + ) + return - // ADD this provider's clientID to the socket's ownership set (do NOT overwrite a sibling provider - // on the same socket — that lone-owner overwrite is exactly what dropped the chat preview's - // awareness when the Files editor co-mounted). A re-JOIN of the same clientID is idempotent. A - // single provider that later unmounts clears its own caret via its awareness removal; the whole - // set is dropped on the socket's LEAVE/disconnect (client emits LEAVE only after its LAST provider - // for the file tears down). - let clientMap = entry.owners.get(socket.id) - if (clientMap === undefined) { - clientMap = new Map() - entry.owners.set(socket.id, clientMap) - } - clientMap.set(clientId, { clientId, userId, userName, avatarUrl }) - socketToRoomName.set(socket.id, name) - socket.join(name) + // A client id must be owned by at most one user, or a peer could bind an active + // collaborator's id and pass the per-frame ownership check to spoof/clear its caret. + // Distinguish a reconnect from a spoof by the owning user: the same user reclaiming its + // own client id (a dropped socket reconnecting reuses the Yjs client id, and its prior + // socket may not be cleaned up yet) takes over the stale binding; a DIFFERENT user is + // rejected. This runs BEFORE any teardown of the socket's current binding below, so a + // rejected rebind — even during a document switch — leaves the socket's existing document + // and caret untouched. + for (const [otherSid, clientMap] of entry.owners) { + if (otherSid === socket.id) continue + const owner = clientMap.get(clientId) + if (owner === undefined) continue + if (owner.userId !== userId) { + emitJoinError(socket, fileId, 'Client id already in use', 'CLIENT_ID_IN_USE', false) + return + } + // Same user reclaiming its client id on a stale prior socket: evict just THAT clientID's + // binding + caret from the old socket. If that leaves the old socket with no providers, also + // drop its room mapping + Socket.IO membership so it can no longer send document (sync) frames + // (handleMessage's SYNC path gates on socketToRoomName, not owners); an old socket that still + // hosts OTHER providers keeps them. Done inline rather than via cleanupFileDocForSocket, which + // could destroyRoomIfIdle the room we're joining. + clientMap.delete(clientId) + awarenessProtocol.removeAwarenessStates(entry.awareness, [clientId], null) + if (clientMap.size === 0) { + entry.owners.delete(otherSid) + socketToRoomName.delete(otherSid) + io.in(otherSid).socketsLeave(name) + } + } - // Capture what the server-side persist needs: the workspace to write back to, and the current - // user for attribution (refreshed to the actual editor on each edit in `handleMessage`). - if (authorized.workspaceId) entry.workspaceId = authorized.workspaceId - entry.lastEditorUserId = userId - - socket.emit(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId }) - // Server-authenticated roster → everyone in the room, including this joiner. - broadcastFileDocPresence(io, name, entry) - - // Begin the sync handshake: send the server's state (sync step 1). The - // client replies with its updates and requests the server's in return. - const syncEncoder = encoding.createEncoder() - encoding.writeVarUint(syncEncoder, FILE_DOC_MESSAGE_TYPE.SYNC) - syncProtocol.writeSyncStep1(syncEncoder, entry.doc) - socket.emit(FILE_DOC_EVENTS.MESSAGE, encoding.toUint8Array(syncEncoder)) - - // Send existing awareness so the new client immediately sees others' carets. - const states = entry.awareness.getStates() - if (states.size > 0) { - const awarenessEncoder = encoding.createEncoder() - encoding.writeVarUint(awarenessEncoder, FILE_DOC_MESSAGE_TYPE.AWARENESS) - encoding.writeVarUint8Array( - awarenessEncoder, - awarenessProtocol.encodeAwarenessUpdate(entry.awareness, Array.from(states.keys())) - ) - socket.emit(FILE_DOC_EVENTS.MESSAGE, encoding.toUint8Array(awarenessEncoder)) - } + // Only now that the rebind is guaranteed to succeed, leave a previously-joined document if + // switching (a socket edits at most one). A duplicate join of the SAME room falls through + // and simply re-runs the sync handshake, idempotently. + const currentName = socketToRoomName.get(socket.id) + if (currentName && currentName !== name) { + socket.leave(currentName) + cleanupFileDocForSocket(socket.id, io) + } - // Seed the document server-side (once). Fire-and-forget: the join completes immediately and - // the seed relays to this socket via `doc.on('update')` the moment it lands. - if (authorized.workspaceId) void ensureServerSeed(name, entry, authorized.workspaceId) + // ADD this provider's clientID to the socket's ownership set (do NOT overwrite a sibling + // provider on the same socket — that lone-owner overwrite is exactly what dropped the chat + // preview's awareness when the Files editor co-mounted). A re-JOIN of the same clientID is + // idempotent. A single provider that later unmounts clears its own caret via its awareness + // removal; the whole set is dropped on the socket's LEAVE/disconnect (the client emits LEAVE + // only after its LAST provider for the file tears down). + let clientMap = entry.owners.get(socket.id) + if (clientMap === undefined) { + clientMap = new Map() + entry.owners.set(socket.id, clientMap) + } + clientMap.set(clientId, { clientId, userId, userName, avatarUrl }) + socketToRoomName.set(socket.id, name) + socket.join(name) + + // Attribution for the server-side persist, refreshed to the actual editor on each edit in + // `handleMessage`. + entry.lastEditorUserId = userId + + // Name the document this room holds, so a client that still carries a DIFFERENT one (its room + // outlived by a document rebuilt in its place) can refuse to merge instead of unioning two + // documents into the file twice over. Read after readiness — before it, the room has no doc yet. + socket.emit(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId, docId: docIdOf(entry.doc) }) + // Server-authenticated roster → everyone in the room, including this joiner. + broadcastFileDocPresence(io, name, entry) + + // Begin the sync handshake: send the server's state (sync step 1). The + // client replies with its updates and requests the server's in return. + const syncEncoder = encoding.createEncoder() + encoding.writeVarUint(syncEncoder, FILE_DOC_MESSAGE_TYPE.SYNC) + syncProtocol.writeSyncStep1(syncEncoder, entry.doc) + socket.emit(FILE_DOC_EVENTS.MESSAGE, encoding.toUint8Array(syncEncoder)) + + // Send existing awareness so the new client immediately sees others' carets. + const states = entry.awareness.getStates() + if (states.size > 0) { + const awarenessEncoder = encoding.createEncoder() + encoding.writeVarUint(awarenessEncoder, FILE_DOC_MESSAGE_TYPE.AWARENESS) + encoding.writeVarUint8Array( + awarenessEncoder, + awarenessProtocol.encodeAwarenessUpdate(entry.awareness, Array.from(states.keys())) + ) + socket.emit(FILE_DOC_EVENTS.MESSAGE, encoding.toUint8Array(awarenessEncoder)) + } - logger.info(`User ${userId} joined file-doc room ${fileId}`) + logger.info(`User ${userId} joined file-doc room ${fileId}`) + } finally { + entry.pendingJoins -= 1 + // A join that returned without registering may have left behind the room it created; drop it + // if nothing else claimed it. A no-op once this join committed (the room then has an owner). + destroyRoomIfIdle(name) + } } catch (error) { logger.error('Error joining file-doc room:', error) try { diff --git a/apps/sim/app/(landing)/components/navbar/components/sim-wordmark/sim-wordmark.tsx b/apps/sim/app/(landing)/components/navbar/components/sim-wordmark/sim-wordmark.tsx index 0043641a3e0..37c883552dd 100644 --- a/apps/sim/app/(landing)/components/navbar/components/sim-wordmark/sim-wordmark.tsx +++ b/apps/sim/app/(landing)/components/navbar/components/sim-wordmark/sim-wordmark.tsx @@ -1,7 +1,11 @@ +import { WORDMARK_PATHS, WORDMARK_VIEW_BOX } from '@/lib/branding/wordmark' + /** * Inline "sim" brand logotype (wordmark, no separate icon mark) - the paths * from the v1.0 brand guide's `simLogotype--dark.svg`, inlined so the logo - * ships as zero-request server-rendered HTML. + * ships as zero-request server-rendered HTML. They live in + * `@/lib/branding/wordmark` because the email header rasterizes the same + * outlines. * * Filled with a single solid `var(--text-body)` - the navbar's own text color * (the same token its nav-link chips use) - so the wordmark reads as one solid @@ -14,7 +18,7 @@ export function SimWordmark() { return ( - - - - + {WORDMARK_PATHS.map((d) => ( + + ))} ) diff --git a/apps/sim/app/_shell/providers/get-query-client.ts b/apps/sim/app/_shell/providers/get-query-client.ts index 681fd4ca84f..7fe869b9212 100644 --- a/apps/sim/app/_shell/providers/get-query-client.ts +++ b/apps/sim/app/_shell/providers/get-query-client.ts @@ -1,4 +1,4 @@ -import { defaultShouldDehydrateQuery, isServer, QueryClient } from '@tanstack/react-query' +import { isServer, QueryClient } from '@tanstack/react-query' import { isDesktopApp } from '@/lib/desktop' export function makeQueryClient() { @@ -6,7 +6,6 @@ export function makeQueryClient() { defaultOptions: { queries: { staleTime: 30 * 1000, - gcTime: 5 * 60 * 1000, // The desktop app window lives for days, so cross-session changes — // an admin upgrading your org/workspace role, a workspace you were // auto-added to, seat/entitlement changes — would otherwise stay @@ -18,16 +17,19 @@ export function makeQueryClient() { // frequent and noisy. Per-query overrides (e.g. useWorkspaceSchedules // pins this off) always win over this default. refetchOnWindowFocus: isDesktopApp(), - retry: 1, + /** + * Query core already defaults retries to 0 on the server and 3 in the browser; + * only the browser number is ours to change. Stating one value for both would + * silently opt server prefetches into a retry, and because the layout awaits + * them that spends a retry backoff of document latency on a read whose failure + * the client recovers from on its own. + */ + retry: isServer ? 0 : 1, retryOnMount: false, }, mutations: { retry: false, }, - dehydrate: { - shouldDehydrateQuery: (query) => - defaultShouldDehydrateQuery(query) || query.state.status === 'pending', - }, }, }) } diff --git a/apps/sim/app/_shell/providers/posthog-provider.tsx b/apps/sim/app/_shell/providers/posthog-provider.tsx index 368bb3fc913..17e3f8c040b 100644 --- a/apps/sim/app/_shell/providers/posthog-provider.tsx +++ b/apps/sim/app/_shell/providers/posthog-provider.tsx @@ -42,6 +42,28 @@ export function PostHogProvider({ children }: { children: React.ReactNode }) { password: true, email: false, }, + /** + * None of these nodes are painted, so replay fidelity is + * unchanged, while each full snapshot serializes fewer nodes on + * the main thread and ships a smaller payload. + * + * Enumerated rather than `true`/`'all'` on purpose — those + * presets also enable `headTitleMutations`, which would drop + * `document.title` changes and lose the page identity a replay + * viewer reads while scrubbing. + */ + slimDOMOptions: { + script: true, + comment: true, + headFavicon: true, + headWhitespace: true, + headMetaDescKeywords: true, + headMetaSocial: true, + headMetaRobots: true, + headMetaHttpEquiv: true, + headMetaAuthorship: true, + headMetaVerification: true, + }, recordCrossOriginIframes: false, recordHeaders: false, recordBody: false, diff --git a/apps/sim/app/_styles/fonts/season/season.ts b/apps/sim/app/_styles/fonts/season/season.ts index b778b47e985..eff2a3cec31 100644 --- a/apps/sim/app/_styles/fonts/season/season.ts +++ b/apps/sim/app/_styles/fonts/season/season.ts @@ -3,13 +3,26 @@ import localFont from 'next/font/local' /** * Season Sans variable font configuration * Uses variable font file to support any weight from 300-800 + * + * `display: 'block'`, not `swap`: this is the document font, so a swap is not a cosmetic change of + * typeface — the fallback's glyph advances differ, so paragraphs re-wrap and everything below them + * moves. In long-form prose (the Files editor) that reads as the line and paragraph spacing visibly + * correcting itself a beat after the text appears, on every hard refresh (a normal reload serves the + * font from cache and never swaps). `swap` is the setting that says "painting the wrong font first is + * fine"; for a brand face it is not. + * + * The block period costs nothing here because delivery is already optimal: `preload` emits a + * `Link: rel=preload` RESPONSE header, so the fetch starts before the HTML is parsed, and the file is + * one same-origin, immutably-cached 87KB woff2. The metric-adjusted Arial below stays as the safety + * net for the >3s tail, where the browser gives up blocking and swaps — i.e. the worst case is + * today's behavior, not a regression. */ export const season = localFont({ src: [ // Variable font - supports all weights from 300 to 800 { path: './SeasonSansUprightsVF.woff2', weight: '300 800', style: 'normal' }, ], - display: 'swap', + display: 'block', preload: true, variable: '--font-season', fallback: ['system-ui', 'Segoe UI', 'Roboto', 'Helvetica Neue', 'Arial', 'Noto Sans'], diff --git a/apps/sim/app/_styles/globals.css b/apps/sim/app/_styles/globals.css index 14508122541..ae53a936065 100644 --- a/apps/sim/app/_styles/globals.css +++ b/apps/sim/app/_styles/globals.css @@ -11,7 +11,7 @@ */ :root { --sidebar-width: 0px; /* 0 outside workspace; blocking script always sets actual value on workspace pages */ - --sidebar-collapsed-width: 51px; /* icon rail on web; desktop overrides to 0 before first paint */ + --sidebar-collapsed-width: 48px; /* icon rail on web; desktop overrides to 0 before first paint */ --sidebar-expanded-width: 238px; /* SIDEBAR_WIDTH.DEFAULT; the width to restore to, held even while collapsed */ --desktop-title-bar-height: 0px; /* macOS traffic-light lane; desktop overrides before first paint */ --workspace-content-title-bar-inset: 0px; /* lane the content pane must leave clear; only non-zero when the pane, not the sidebar, sits under it */ diff --git a/apps/sim/app/api/auth/oauth/token/route.test.ts b/apps/sim/app/api/auth/oauth/token/route.test.ts index b1c07e96b68..c149d1909b0 100644 --- a/apps/sim/app/api/auth/oauth/token/route.test.ts +++ b/apps/sim/app/api/auth/oauth/token/route.test.ts @@ -24,6 +24,7 @@ vi.mock('@/lib/oauth/credential-service', () => ({ vi.mock('@/lib/auth/credential-access', () => ({ authorizeCredentialUse: mockAuthorizeCredentialUse, + authorizeCredentialUseForAuth: mockAuthorizeCredentialUse, })) import { TokenServiceAccountValidationError } from '@/lib/credentials/token-service-accounts/errors' @@ -201,6 +202,36 @@ describe('OAuth Token API Routes', () => { }) describe('service account path', () => { + it('threads the NetSuite SuiteTalk instance URL into the token response', async () => { + const instanceUrl = 'https://1234567.suitetalk.api.netsuite.com' + authOAuthUtilsMockFns.mockResolveOAuthAccountId.mockResolvedValueOnce({ + accountId: '', + credentialId: 'netsuite-credential-id', + credentialType: 'service_account', + providerId: 'netsuite-service-account', + workspaceId: 'workspace-id', + usedCredentialTable: true, + }) + mockAuthorizeCredentialUse.mockResolvedValueOnce({ + ok: true, + authType: 'session', + requesterUserId: 'test-user-id', + workspaceId: 'workspace-id', + }) + mockResolveServiceAccountToken.mockResolvedValueOnce({ + accessToken: 'netsuite-token', + instanceUrl, + }) + + const response = await POST( + createMockRequest('POST', { credentialId: 'netsuite-credential-id' }) + ) + const data = await response.json() + + expect(response.status).toBe(200) + expect(data).toMatchObject({ accessToken: 'netsuite-token', instanceUrl }) + }) + it('should thread authStyle from the resolver into the response', async () => { authOAuthUtilsMockFns.mockResolveOAuthAccountId.mockResolvedValueOnce({ accountId: '', diff --git a/apps/sim/app/api/auth/oauth/token/route.ts b/apps/sim/app/api/auth/oauth/token/route.ts index cc66068135a..c3e1744dc1f 100644 --- a/apps/sim/app/api/auth/oauth/token/route.ts +++ b/apps/sim/app/api/auth/oauth/token/route.ts @@ -11,17 +11,9 @@ import { authorizeCredentialUse } from '@/lib/auth/credential-access' import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid' import { generateRequestId } from '@/lib/core/utils/request' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' -import { TokenServiceAccountValidationError } from '@/lib/credentials/token-service-accounts/errors' -import { - getCredential, - getOAuthToken, - refreshTokenIfNeeded, - resolveOAuthAccountId, - resolveServiceAccountToken, -} from '@/lib/oauth/credential-service' -import { extractSalesforceInstanceUrl, isSalesforceOAuthProviderId } from '@/lib/oauth/salesforce' +import { getCredential, getOAuthToken } from '@/lib/oauth/credential-service' +import { completeOAuthCredentialToken, resolveCredentialToken } from '@/lib/oauth/token-resolution' import { captureServerEvent } from '@/lib/posthog/server' -import { extractZohoDeskBaseFromScope } from '@/tools/zoho_desk/host-allowlist' export const dynamic = 'force-dynamic' @@ -123,194 +115,25 @@ export const POST = withRouteHandler(async (request: NextRequest) => { } } - if (!credentialId) { - return NextResponse.json({ error: 'Credential ID is required' }, { status: 400 }) - } - - const resolved = await resolveOAuthAccountId(credentialId) - if (resolved?.credentialType === 'service_account' && resolved.credentialId) { - const authz = await authorizeCredentialUse(request, { - credentialId, - workflowId: workflowId ?? undefined, - requireWorkflowIdForInternal: false, - callerUserId, - }) - if (!authz.ok) { - return NextResponse.json({ error: authz.error || 'Unauthorized' }, { status: 403 }) - } - - const saActorId = authz.requesterUserId - const saWorkspaceId = resolved.workspaceId ?? authz.workspaceId ?? null - const emitServiceAccountAccess = () => { - if (!saActorId) return - recordAudit({ - workspaceId: saWorkspaceId, - actorId: saActorId, - action: AuditAction.CREDENTIAL_ACCESSED, - resourceType: AuditResourceType.CREDENTIAL, - resourceId: resolved.credentialId ?? credentialId, - description: `Accessed service account credential for provider ${resolved.providerId ?? 'unknown'}`, - metadata: { - provider: resolved.providerId, - credentialType: 'service_account', - }, - request, - }) - captureServerEvent( - saActorId, - 'credential_used', - { - credential_type: 'service_account', - provider_id: resolved.providerId ?? 'unknown', - ...(saWorkspaceId ? { workspace_id: saWorkspaceId } : {}), - }, - saWorkspaceId ? { groups: { workspace: saWorkspaceId } } : undefined - ) - } - - try { - const result = await resolveServiceAccountToken( - resolved.credentialId, - resolved.providerId, - scopes ?? [], - impersonateEmail - ) - emitServiceAccountAccess() - return NextResponse.json( - { - accessToken: result.accessToken, - cloudId: result.cloudId, - domain: result.domain, - instanceUrl: result.instanceUrl, - apiDomain: result.apiDomain, - authStyle: result.authStyle, - }, - { status: 200 } - ) - } catch (error) { - logger.error(`[${requestId}] Service account token error:`, error) - if (error instanceof TokenServiceAccountValidationError) { - // Classified provider outages are infra failures, not bad credentials. - if (error.code === 'provider_unavailable') { - return NextResponse.json( - { error: 'Credential provider is temporarily unavailable' }, - { status: 502 } - ) - } - // A stored host that no longer resolves is a configuration failure — - // surface the code so runtime consumers can say "check the host" - // instead of a generic auth error. - if (error.code === 'site_not_found') { - return NextResponse.json( - { - code: error.code, - error: 'Credential host not found — reconnect the credential with a valid host', - }, - { status: 400 } - ) - } - // A revoked/rotated-away or misconfigured stored secret — surface the - // code so runtime consumers can prompt to reconnect the credential - // rather than showing a generic auth failure. - if (error.code === 'invalid_credentials') { - return NextResponse.json( - { - code: error.code, - error: 'Credential rejected by the provider — reconnect the credential', - }, - { status: 401 } - ) - } - } - return NextResponse.json({ error: 'Failed to get service account token' }, { status: 401 }) - } - } - - const authz = await authorizeCredentialUse(request, { + const auth = await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) + const result = await resolveCredentialToken(auth, { + requestId, credentialId, workflowId: workflowId ?? undefined, - requireWorkflowIdForInternal: false, + scopes, + impersonateEmail, callerUserId, + auditRequest: request, }) - if (!authz.ok || !authz.credentialOwnerUserId) { - return NextResponse.json({ error: authz.error || 'Unauthorized' }, { status: 403 }) - } - - const resolvedCredentialId = authz.resolvedCredentialId || credentialId - const credential = await getCredential( - requestId, - resolvedCredentialId, - authz.credentialOwnerUserId - ) - - if (!credential) { - return NextResponse.json({ error: 'Credential not found' }, { status: 404 }) - } - - const oauthActorId = authz.requesterUserId - const oauthWorkspaceId = authz.workspaceId ?? null - - try { - const { accessToken } = await refreshTokenIfNeeded( - requestId, - credential, - resolvedCredentialId - ) - - if (oauthActorId) { - recordAudit({ - workspaceId: oauthWorkspaceId, - actorId: oauthActorId, - action: AuditAction.CREDENTIAL_ACCESSED, - resourceType: AuditResourceType.CREDENTIAL, - resourceId: resolvedCredentialId, - description: `Accessed OAuth credential for provider ${credential.providerId}`, - metadata: { - provider: credential.providerId, - credentialType: 'oauth', - }, - request, - }) - captureServerEvent( - oauthActorId, - 'credential_used', - { - credential_type: 'oauth', - provider_id: credential.providerId, - ...(oauthWorkspaceId ? { workspace_id: oauthWorkspaceId } : {}), - }, - oauthWorkspaceId ? { groups: { workspace: oauthWorkspaceId } } : undefined - ) - } - - const instanceUrl = isSalesforceOAuthProviderId(credential.providerId) - ? extractSalesforceInstanceUrl(credential.scope) - : undefined - - // Zoho Desk persists its data-center-specific REST base URL in the scope - // string (derived from the token response api_domain) so callers never - // assume a host. Surface it as apiDomain for tool param injection. - let apiDomain: string | undefined - if (credential.providerId === 'zoho-desk' && credential.scope) { - // Use the shared extractor, not a local regex: it also enforces https + - // the Zoho apex allowlist. This value is injected into EVERY tool call, - // so an unvalidated host here would receive the OAuth token. - apiDomain = extractZohoDeskBaseFromScope(credential.scope) - } + if (!result.ok) { return NextResponse.json( - { - accessToken, - idToken: credential.idToken || undefined, - ...(instanceUrl && { instanceUrl }), - ...(apiDomain && { apiDomain }), - }, - { status: 200 } + { ...(result.code ? { code: result.code } : {}), error: result.error }, + { status: result.status } ) - } catch (error) { - logger.error(`[${requestId}] Failed to refresh access token:`, error) - return NextResponse.json({ error: 'Failed to refresh access token' }, { status: 401 }) } + + return NextResponse.json(result.token, { status: 200 }) } catch (error) { logger.error(`[${requestId}] Error getting access token`, error) return NextResponse.json({ error: 'Internal server error' }, { status: 500 }) @@ -366,70 +189,20 @@ export const GET = withRouteHandler(async (request: NextRequest) => { return NextResponse.json({ error: 'No access token available' }, { status: 400 }) } - const actorId = authz.requesterUserId - const workspaceId = authz.workspaceId ?? null - - try { - const { accessToken } = await refreshTokenIfNeeded( - requestId, - credential, - resolvedCredentialId - ) - - if (actorId) { - recordAudit({ - workspaceId, - actorId, - action: AuditAction.CREDENTIAL_ACCESSED, - resourceType: AuditResourceType.CREDENTIAL, - resourceId: resolvedCredentialId, - description: `Accessed OAuth credential for provider ${credential.providerId}`, - metadata: { - provider: credential.providerId, - credentialType: 'oauth', - }, - request, - }) - captureServerEvent( - actorId, - 'credential_used', - { - credential_type: 'oauth', - provider_id: credential.providerId, - ...(workspaceId ? { workspace_id: workspaceId } : {}), - }, - workspaceId ? { groups: { workspace: workspaceId } } : undefined - ) - } - - const instanceUrl = isSalesforceOAuthProviderId(credential.providerId) - ? extractSalesforceInstanceUrl(credential.scope) - : undefined - - // Zoho Desk persists its data-center-specific REST base URL in the scope - // string (derived from the token response api_domain) so callers never - // assume a host. Surface it as apiDomain for tool param injection. - let apiDomain: string | undefined - if (credential.providerId === 'zoho-desk' && credential.scope) { - // Use the shared extractor, not a local regex: it also enforces https + - // the Zoho apex allowlist. This value is injected into EVERY tool call, - // so an unvalidated host here would receive the OAuth token. - apiDomain = extractZohoDeskBaseFromScope(credential.scope) - } + const result = await completeOAuthCredentialToken({ + requestId, + credential, + resolvedCredentialId, + actorId: authz.requesterUserId, + workspaceId: authz.workspaceId ?? null, + auditRequest: request, + }) - return NextResponse.json( - { - accessToken, - idToken: credential.idToken || undefined, - ...(instanceUrl && { instanceUrl }), - ...(apiDomain && { apiDomain }), - }, - { status: 200 } - ) - } catch (error) { - logger.error(`[${requestId}] Failed to refresh access token:`, error) - return NextResponse.json({ error: 'Failed to refresh access token' }, { status: 401 }) + if (!result.ok) { + return NextResponse.json({ error: result.error }, { status: result.status }) } + + return NextResponse.json(result.token, { status: 200 }) } catch (error) { logger.error(`[${requestId}] Error fetching access token`, error) return NextResponse.json({ error: 'Internal server error' }, { status: 500 }) diff --git a/apps/sim/app/api/auth/oauth/utils.test.ts b/apps/sim/app/api/auth/oauth/utils.test.ts index e7cee6bbc33..a5d76c1bc5a 100644 --- a/apps/sim/app/api/auth/oauth/utils.test.ts +++ b/apps/sim/app/api/auth/oauth/utils.test.ts @@ -26,7 +26,10 @@ vi.mock('@/lib/credentials/client-credential-accounts/server', () => ({ import { db } from '@sim/db' import { __resetCoalesceLocallyForTests } from '@/lib/concurrency/singleflight' -import { ZOOM_SERVICE_ACCOUNT_PROVIDER_ID } from '@/lib/credentials/client-credential-accounts/descriptors' +import { + NETSUITE_SERVICE_ACCOUNT_PROVIDER_ID, + ZOOM_SERVICE_ACCOUNT_PROVIDER_ID, +} from '@/lib/credentials/client-credential-accounts/descriptors' import { refreshOAuthToken } from '@/lib/oauth' import { getCredential, @@ -526,6 +529,33 @@ describe('OAuth Utils', () => { expect(mockMinter).toHaveBeenCalledTimes(1) }) + it('forwards NetSuite certificate material and caches its SuiteTalk instance URL', async () => { + const credId = 'ccsa-netsuite-certificate' + const fields = { + clientId: 'netsuite-client', + certificateId: 'certificate-id', + orgId: 'https://1234567.suitetalk.api.netsuite.com', + privateKey: 'private-key', + } + mockDecryptSecret.mockResolvedValueOnce({ decrypted: JSON.stringify(fields) }) + mockCredentialRow(ENCRYPTED_KEY_A) + mockMinter.mockResolvedValueOnce({ + accessToken: 'netsuite-token', + expiresInSeconds: 3600, + instanceUrl: fields.orgId, + }) + + const first = await resolveServiceAccountToken(credId, NETSUITE_SERVICE_ACCOUNT_PROVIDER_ID) + + expect(first).toEqual({ accessToken: 'netsuite-token', instanceUrl: fields.orgId }) + expect(mockMinter).toHaveBeenCalledWith(fields, { skipIdentity: true }) + + mockCredentialRow(ENCRYPTED_KEY_A) + const cached = await resolveServiceAccountToken(credId, NETSUITE_SERVICE_ACCOUNT_PROVIDER_ID) + expect(cached).toEqual(first) + expect(mockMinter).toHaveBeenCalledTimes(1) + }) + it('re-mints when remaining validity is below the 5-minute serve floor', async () => { const credId = 'ccsa-ttl-floor' mockCredentialRow(ENCRYPTED_KEY_A) diff --git a/apps/sim/app/api/copilot/checkpoints/revert/route.test.ts b/apps/sim/app/api/copilot/checkpoints/revert/route.test.ts index 785718e9ba2..621256b8ac3 100644 --- a/apps/sim/app/api/copilot/checkpoints/revert/route.test.ts +++ b/apps/sim/app/api/copilot/checkpoints/revert/route.test.ts @@ -17,12 +17,23 @@ import { import { NextRequest } from 'next/server' import { afterAll, afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -const { mockGetAccessibleCopilotChat } = vi.hoisted(() => ({ +const { + mockGetAccessibleCopilotChat, + mockParseWorkflowStateForPersistence, + mockSaveWorkflowNormalizedState, +} = vi.hoisted(() => ({ mockGetAccessibleCopilotChat: vi.fn(), + mockParseWorkflowStateForPersistence: vi.fn(), + mockSaveWorkflowNormalizedState: vi.fn(), })) vi.mock('@/lib/workflows/utils', () => workflowsUtilsMock) +vi.mock('@/lib/workflows/persistence/save-normalized-state', () => ({ + parseWorkflowStateForPersistence: mockParseWorkflowStateForPersistence, + saveWorkflowNormalizedState: mockSaveWorkflowNormalizedState, +})) + vi.mock('@/lib/copilot/chat/lifecycle', () => ({ getAccessibleCopilotChat: mockGetAccessibleCopilotChat, getAccessibleCopilotChatAuth: mockGetAccessibleCopilotChat, @@ -38,13 +49,21 @@ describe('Copilot Checkpoints Revert API Route', () => { authMockFns.mockGetSession.mockResolvedValue(null) + /** Authorization is the route's workflow read, so an allowed result always carries one. */ workflowAuthzMockFns.mockAuthorizeWorkflowByWorkspacePermission.mockResolvedValue({ allowed: true, status: 200, + workflow: { id: 'b2c3d4e5-f6a7-4b89-a0d1-e2f3a4b5c6d7', workspaceId: 'ws-123' }, }) mockGetAccessibleCopilotChat.mockResolvedValue({ id: 'chat-123', userId: 'user-123' }) + mockParseWorkflowStateForPersistence.mockImplementation((value: unknown) => ({ + success: true, + data: value, + })) + mockSaveWorkflowNormalizedState.mockResolvedValue({ success: true, warnings: [] }) + global.fetch = vi.fn() vi.spyOn(Date, 'now').mockReturnValue(1640995200000) @@ -184,7 +203,12 @@ describe('Copilot Checkpoints Revert API Route', () => { } queueTableRows(schemaMock.workflowCheckpoints, [mockCheckpoint]) - queueTableRows(schemaMock.workflow, []) + /** Authorization performs the workflow read, so a missing workflow surfaces through it. */ + workflowAuthzMockFns.mockAuthorizeWorkflowByWorkspacePermission.mockResolvedValueOnce({ + allowed: false, + status: 404, + workflow: null, + }) const req = new NextRequest('http://localhost:3000/api/copilot/checkpoints/revert', { method: 'POST', @@ -197,6 +221,7 @@ describe('Copilot Checkpoints Revert API Route', () => { expect(response.status).toBe(404) const responseData = await response.json() expect(responseData.error).toBe('Workflow not found') + expect(mockSaveWorkflowNormalizedState).not.toHaveBeenCalled() }) it('should return 401 when workflow belongs to different user', async () => { @@ -220,6 +245,7 @@ describe('Copilot Checkpoints Revert API Route', () => { workflowAuthzMockFns.mockAuthorizeWorkflowByWorkspacePermission.mockResolvedValueOnce({ allowed: false, status: 403, + workflow: { id: 'b2c3d4e5-f6a7-4b89-a0d1-e2f3a4b5c6d7', workspaceId: 'ws-123' }, }) const req = new NextRequest('http://localhost:3000/api/copilot/checkpoints/revert', { @@ -297,24 +323,19 @@ describe('Copilot Checkpoints Revert API Route', () => { }, }) - // Verify fetch was called with correct parameters - expect(global.fetch).toHaveBeenCalledWith( - 'http://localhost:3000/api/workflows/c3d4e5f6-a7b8-4c09-a1e2-f3a4b5c6d7e8/state', - { - method: 'PUT', - headers: { - 'Content-Type': 'application/json', - Cookie: 'session=test-session', - }, - body: JSON.stringify({ + expect(mockSaveWorkflowNormalizedState).toHaveBeenCalledWith( + expect.objectContaining({ + workflowId: 'c3d4e5f6-a7b8-4c09-a1e2-f3a4b5c6d7e8', + userId: 'user-123', + state: { blocks: { block1: { type: 'start' } }, edges: [{ from: 'block1', to: 'block2' }], loops: {}, parallels: {}, isDeployed: true, lastSaved: 1640995200000, - }), - } + }, + }) ) }) @@ -452,7 +473,7 @@ describe('Copilot Checkpoints Revert API Route', () => { }) }) - it('should return 500 when state API call fails', async () => { + it('should return 500 when the state write fails', async () => { setAuthenticated() const mockCheckpoint = { @@ -470,9 +491,10 @@ describe('Copilot Checkpoints Revert API Route', () => { queueTableRows(schemaMock.workflowCheckpoints, [mockCheckpoint]) queueTableRows(schemaMock.workflow, [mockWorkflow]) - ;(global.fetch as any).mockResolvedValue({ - ok: false, - text: () => Promise.resolve('State validation failed'), + mockSaveWorkflowNormalizedState.mockResolvedValueOnce({ + success: false, + status: 500, + error: 'Failed to save workflow state', }) const req = new NextRequest('http://localhost:3000/api/copilot/checkpoints/revert', { @@ -488,6 +510,36 @@ describe('Copilot Checkpoints Revert API Route', () => { expect(responseData.error).toBe('Failed to revert workflow to checkpoint') }) + it('should return 500 when the checkpoint state fails validation', async () => { + setAuthenticated() + + const mockCheckpoint = { + id: 'checkpoint-123', + workflowId: 'a7b8c9d0-e1f2-4a34-b5c6-d7e8f9a0b1c2', + userId: 'user-123', + workflowState: { blocks: {}, edges: [] }, + } + + queueTableRows(schemaMock.workflowCheckpoints, [mockCheckpoint]) + queueTableRows(schemaMock.workflow, [{ id: mockCheckpoint.workflowId, userId: 'user-123' }]) + + mockParseWorkflowStateForPersistence.mockReturnValueOnce({ + success: false, + error: { issues: [{ message: 'blocks: invalid' }] }, + }) + + const req = new NextRequest('http://localhost:3000/api/copilot/checkpoints/revert', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ checkpointId: 'checkpoint-123' }), + }) + + const response = await POST(req) + + expect(response.status).toBe(500) + expect(mockSaveWorkflowNormalizedState).not.toHaveBeenCalled() + }) + it('should handle database errors during checkpoint lookup', async () => { setAuthenticated() @@ -519,8 +571,9 @@ describe('Copilot Checkpoints Revert API Route', () => { } dbChainMockFns.where.mockReturnValueOnce(Promise.resolve([mockCheckpoint])) - dbChainMockFns.where.mockReturnValueOnce( - Promise.reject(new Error('Database error during workflow lookup')) + /** Authorization performs the workflow read, so a failed lookup surfaces through it. */ + workflowAuthzMockFns.mockAuthorizeWorkflowByWorkspacePermission.mockRejectedValueOnce( + new Error('Database error during workflow lookup') ) const req = new NextRequest('http://localhost:3000/api/copilot/checkpoints/revert', { @@ -536,7 +589,7 @@ describe('Copilot Checkpoints Revert API Route', () => { expect(responseData.error).toBe('Failed to revert to checkpoint') }) - it('should handle fetch network errors', async () => { + it('should handle unexpected errors from the state write', async () => { setAuthenticated() const mockCheckpoint = { @@ -554,7 +607,7 @@ describe('Copilot Checkpoints Revert API Route', () => { queueTableRows(schemaMock.workflowCheckpoints, [mockCheckpoint]) queueTableRows(schemaMock.workflow, [mockWorkflow]) - ;(global.fetch as any).mockRejectedValue(new Error('Network error')) + mockSaveWorkflowNormalizedState.mockRejectedValueOnce(new Error('Network error')) const req = new NextRequest('http://localhost:3000/api/copilot/checkpoints/revert', { method: 'POST', @@ -587,7 +640,7 @@ describe('Copilot Checkpoints Revert API Route', () => { expect(responseData.error).toBe('Failed to revert to checkpoint') }) - it('should forward cookies to state API call', async () => { + it('should apply the state in-process instead of re-authenticating over HTTP', async () => { setAuthenticated() const mockCheckpoint = { @@ -623,17 +676,15 @@ describe('Copilot Checkpoints Revert API Route', () => { await POST(req) - expect(global.fetch).toHaveBeenCalledWith( - 'http://localhost:3000/api/workflows/d0e1f2a3-b4c5-4d67-a8f9-a0b1c2d3e4f5/state', - { - method: 'PUT', - headers: { - 'Content-Type': 'application/json', - Cookie: 'session=test-session; auth=token123', - }, - body: expect.any(String), - } + expect(mockSaveWorkflowNormalizedState).toHaveBeenCalledWith( + expect.objectContaining({ + workflowId: 'd0e1f2a3-b4c5-4d67-a8f9-a0b1c2d3e4f5', + userId: 'user-123', + }) ) + for (const call of (global.fetch as any).mock.calls) { + expect(String(call[0])).not.toContain('/state') + } }) it('should handle missing cookies gracefully', async () => { @@ -673,16 +724,11 @@ describe('Copilot Checkpoints Revert API Route', () => { const response = await POST(req) expect(response.status).toBe(200) - expect(global.fetch).toHaveBeenCalledWith( - 'http://localhost:3000/api/workflows/e1f2a3b4-c5d6-4e78-a9a0-b1c2d3e4f5a6/state', - { - method: 'PUT', - headers: { - 'Content-Type': 'application/json', - Cookie: '', // Empty string when no cookies - }, - body: expect.any(String), - } + expect(mockSaveWorkflowNormalizedState).toHaveBeenCalledWith( + expect.objectContaining({ + workflowId: 'e1f2a3b4-c5d6-4e78-a9a0-b1c2d3e4f5a6', + userId: 'user-123', + }) ) }) diff --git a/apps/sim/app/api/copilot/checkpoints/revert/route.ts b/apps/sim/app/api/copilot/checkpoints/revert/route.ts index f784dc48d84..1543372f773 100644 --- a/apps/sim/app/api/copilot/checkpoints/revert/route.ts +++ b/apps/sim/app/api/copilot/checkpoints/revert/route.ts @@ -1,5 +1,5 @@ import { db } from '@sim/db' -import { workflowCheckpoints, workflow as workflowTable } from '@sim/db/schema' +import { workflowCheckpoints } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { authorizeWorkflowByWorkspacePermission } from '@sim/platform-authz/workflow' import { and, eq } from 'drizzle-orm' @@ -15,8 +15,11 @@ import { createRequestTracker, createUnauthorizedResponse, } from '@/lib/copilot/request/http' -import { getInternalApiBaseUrl } from '@/lib/core/utils/urls' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { + parseWorkflowStateForPersistence, + saveWorkflowNormalizedState, +} from '@/lib/workflows/persistence/save-normalized-state' import { isUuidV4 } from '@/executor/constants' const logger = createLogger('CheckpointRevertAPI') @@ -62,21 +65,15 @@ export const POST = withRouteHandler(async (request: NextRequest) => { return createNotFoundResponse('Checkpoint not found or access denied') } - const workflowData = await db - .select() - .from(workflowTable) - .where(eq(workflowTable.id, checkpoint.workflowId)) - .then((rows) => rows[0]) - - if (!workflowData) { - return createNotFoundResponse('Workflow not found') - } - + /** Authorization already loads the workflow, so its absence is the not-found signal. */ const authorization = await authorizeWorkflowByWorkspacePermission({ workflowId: checkpoint.workflowId, userId, action: 'write', }) + if (!authorization.workflow) { + return createNotFoundResponse('Workflow not found') + } if (!authorization.allowed) { return createUnauthorizedResponse() } @@ -121,28 +118,40 @@ export const POST = withRouteHandler(async (request: NextRequest) => { return NextResponse.json({ error: 'Invalid workflow ID format' }, { status: 400 }) } - const stateResponse = await fetch( - `${getInternalApiBaseUrl()}/api/workflows/${checkpoint.workflowId}/state`, - { - method: 'PUT', - headers: { - 'Content-Type': 'application/json', - Cookie: request.headers.get('Cookie') || '', - }, - body: JSON.stringify(cleanedState), - } - ) + /** + * The checkpoint blob is persisted JSONB, so it goes through the same + * schema the PUT state contract applies before it is written back — the + * validation the removed HTTP hop used to provide. + */ + const parsedState = parseWorkflowStateForPersistence(cleanedState) + if (!parsedState.success) { + logger.error( + `[${tracker.requestId}] Checkpoint state failed validation`, + parsedState.error.issues + ) + return NextResponse.json( + { error: 'Failed to revert workflow to checkpoint' }, + { status: 500 } + ) + } + + const saveResult = await saveWorkflowNormalizedState({ + requestId: tracker.requestId, + workflowId: checkpoint.workflowId, + userId, + state: parsedState.data, + /** Already resolved above; re-deriving it would repeat 2-3 sequential reads. */ + authorization, + }) - if (!stateResponse.ok) { - const errorData = await stateResponse.text() - logger.error(`[${tracker.requestId}] Failed to apply checkpoint state: ${errorData}`) + if (!saveResult.success) { + logger.error(`[${tracker.requestId}] Failed to apply checkpoint state: ${saveResult.error}`) return NextResponse.json( { error: 'Failed to revert workflow to checkpoint' }, { status: 500 } ) } - const result = await stateResponse.json() logger.info( `[${tracker.requestId}] Successfully reverted workflow ${checkpoint.workflowId} to checkpoint ${checkpointId}` ) diff --git a/apps/sim/app/api/credentials/[id]/route.ts b/apps/sim/app/api/credentials/[id]/route.ts index e82dca826df..3ff1de37444 100644 --- a/apps/sim/app/api/credentials/[id]/route.ts +++ b/apps/sim/app/api/credentials/[id]/route.ts @@ -96,6 +96,7 @@ export const PUT = withRouteHandler( domain: body.domain, clientId: body.clientId, clientSecret: body.clientSecret, + certificateId: body.certificateId, orgId: body.orgId, dataCenter: body.dataCenter, authMethod: body.authMethod, diff --git a/apps/sim/app/api/credentials/route.test.ts b/apps/sim/app/api/credentials/route.test.ts index 0e5f1c21a51..3a105b71e57 100644 --- a/apps/sim/app/api/credentials/route.test.ts +++ b/apps/sim/app/api/credentials/route.test.ts @@ -188,6 +188,58 @@ describe('POST /api/credentials', () => { ) }) + it('threads NetSuite certificate credentials through the create contract', async () => { + mockVerifyAndBuildServiceAccountSecret.mockResolvedValueOnce({ + providerId: 'netsuite-service-account', + encryptedServiceAccountKey: 'encrypted-netsuite-blob', + displayName: 'Oracle NetSuite 1234567', + auditMetadata: { principalKind: 'tenant', principalId: '1234567' }, + principal: { kind: 'tenant', id: '1234567' }, + }) + queueTableRows(credential, []) + queueTableRows(credential, []) + queueTableRows(credential, [ + { + id: 'credential-netsuite', + workspaceId: WORKSPACE_ID, + type: 'service_account', + displayName: 'Oracle NetSuite 1234567', + description: null, + providerId: 'netsuite-service-account', + accountId: null, + envKey: null, + envOwnerUserId: null, + encryptedServiceAccountKey: 'encrypted-netsuite-blob', + createdBy: 'user-1', + createdAt: new Date('2026-08-11T00:00:00.000Z'), + updatedAt: new Date('2026-08-11T00:00:00.000Z'), + }, + ]) + + const response = await POST( + createMockRequest('POST', { + workspaceId: WORKSPACE_ID, + type: 'service_account', + providerId: 'netsuite-service-account', + orgId: 'https://1234567.suitetalk.api.netsuite.com', + clientId: 'netsuite-client-id', + certificateId: 'netsuite-certificate-id', + privateKey: '-----BEGIN PRIVATE KEY-----key', + }) + ) + + expect(response.status).toBe(201) + expect(mockVerifyAndBuildServiceAccountSecret).toHaveBeenCalledWith( + 'netsuite-service-account', + expect.objectContaining({ + orgId: 'https://1234567.suitetalk.api.netsuite.com', + clientId: 'netsuite-client-id', + certificateId: 'netsuite-certificate-id', + privateKey: '-----BEGIN PRIVATE KEY-----key', + }) + ) + }) + it('maps a verification failure to a 400 with the validation code', async () => { mockVerifyAndBuildServiceAccountSecret.mockRejectedValueOnce( new TokenServiceAccountValidationError('invalid_credentials', 400, { diff --git a/apps/sim/app/api/credentials/route.ts b/apps/sim/app/api/credentials/route.ts index 20b4a4bcac0..69ec1fb54e2 100644 --- a/apps/sim/app/api/credentials/route.ts +++ b/apps/sim/app/api/credentials/route.ts @@ -222,7 +222,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => { types: type ? [type] : undefined, providerId, }) - const credentials = visible.map(({ hasServiceAccountKey: _hasKey, ...rest }) => rest) + const credentials = visible.data.map(({ hasServiceAccountKey: _hasKey, ...rest }) => rest) return NextResponse.json({ credentials }) } catch (error) { diff --git a/apps/sim/app/api/cron/cleanup-stale-executions/route.test.ts b/apps/sim/app/api/cron/cleanup-stale-executions/route.test.ts index a1422424ef7..a58e0bfa8ec 100644 --- a/apps/sim/app/api/cron/cleanup-stale-executions/route.test.ts +++ b/apps/sim/app/api/cron/cleanup-stale-executions/route.test.ts @@ -37,6 +37,21 @@ function flattenConditions(condition: unknown): MockCondition[] { return [node, ...(node.conditions?.flatMap((child) => flattenConditions(child)) ?? [])] } +function hasToSQL(value: unknown): value is { toSQL: () => { sql: string; params: unknown[] } } { + return typeof value === 'object' && value !== null && 'toSQL' in value +} + +/** + * Collects the leaves of a nested `sql` expression. The duration expression is + * built by a shared helper, so the values it binds sit one level below the + * fragment this route assembles rather than directly in its own params. + */ +function flattenSqlParams(expression: { sql: string; params: unknown[] }): unknown[] { + return expression.params.flatMap((param) => + hasToSQL(param) ? flattenSqlParams(param.toSQL()) : [param] + ) +} + function createRequest() { return createMockRequest( 'GET', @@ -96,11 +111,7 @@ describe('stale execution cleanup deadline grace', () => { 'toSQL' in value && value.toSQL().sql.includes('EXTRACT(EPOCH') ) - const totalDurationExpression = update.totalDurationMs.toSQL() - const cleanupTimestamp = totalDurationExpression.params.find( - (value): value is { toSQL: () => { sql: string; params: unknown[] } } => - typeof value === 'object' && value !== null && 'toSQL' in value - ) + const totalDurationLeaves = flattenSqlParams(update.totalDurationMs.toSQL()) expect(errorExpression.sql).toContain('CASE') expect(errorExpression.sql).toContain('IS NOT NULL') @@ -111,11 +122,9 @@ describe('stale execution cleanup deadline grace', () => { ) expect(staleDurationExpression?.toSQL().sql).toContain('ROUND') expect(staleDurationExpression?.toSQL().params).toContain(workflowExecutionLogs.startedAt) - expect(totalDurationExpression.sql).toContain('LEAST') - expect(totalDurationExpression.sql).toContain('ROUND') - expect(totalDurationExpression.params).toContain(2_147_483_647) - expect(totalDurationExpression.params).toContain(workflowExecutionLogs.startedAt) - expect(cleanupTimestamp?.toSQL().params).toEqual([new Date('2026-08-03T12:10:00.000Z')]) + expect(totalDurationLeaves).toContain(2_147_483_647) + expect(totalDurationLeaves).toContain(workflowExecutionLogs.startedAt) + expect(totalDurationLeaves).toContainEqual(new Date('2026-08-03T12:10:00.000Z')) expect(update.endedAt).toEqual(new Date('2026-08-03T12:10:00.000Z')) } finally { vi.useRealTimers() diff --git a/apps/sim/app/api/cron/cleanup-stale-executions/route.ts b/apps/sim/app/api/cron/cleanup-stale-executions/route.ts index fa3dabf0a03..f77e35a0741 100644 --- a/apps/sim/app/api/cron/cleanup-stale-executions/route.ts +++ b/apps/sim/app/api/cron/cleanup-stale-executions/route.ts @@ -25,6 +25,7 @@ import { } from '@/lib/core/execution-limits' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import type { DbTransaction } from '@/lib/db/types' +import { elapsedDurationMsSql } from '@/lib/logs/execution/duration' import { deleteFile } from '@/lib/uploads/core/storage-service' const logger = createLogger('CleanupStaleExecutions') @@ -33,7 +34,6 @@ const STALE_THRESHOLD_MS = getExecutionReservationTtlMs() const STALE_THRESHOLD_MINUTES = Math.ceil(STALE_THRESHOLD_MS / 60000) const GENERIC_STALE_PROCESSING_ERROR = `Job terminated: stuck in processing for more than ${STALE_THRESHOLD_MINUTES} minutes` const EXECUTION_DEADLINE_ERROR = getTimeoutErrorMessage(undefined) -const MAX_INT32 = 2_147_483_647 /** * Table jobs run as detached workers with progress heartbeats, independently of workflow timeout * policy. Preserve their historical 90-minute task window plus five-minute cleanup grace. @@ -154,10 +154,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => { const staleDurationMinutes = sql`ROUND( EXTRACT(EPOCH FROM (${cleanupTimestamp} - ${workflowExecutionLogs.startedAt})) / 60 )::integer` - const totalDurationMs = sql`LEAST( - ${MAX_INT32}, - ROUND(EXTRACT(EPOCH FROM (${cleanupTimestamp} - ${workflowExecutionLogs.startedAt})) * 1000) - )::integer` + const totalDurationMs = elapsedDurationMsSql(now) let workflowRowsConsidered = 0 while (workflowRowsConsidered < WORKFLOW_EXECUTION_MAX_ROWS_PER_RUN) { const limit = Math.min( diff --git a/apps/sim/app/api/desktop/update/latest-mac.yml/route.test.ts b/apps/sim/app/api/desktop/update/latest-mac.yml/route.test.ts index 7750053970f..754b5fff8b4 100644 --- a/apps/sim/app/api/desktop/update/latest-mac.yml/route.test.ts +++ b/apps/sim/app/api/desktop/update/latest-mac.yml/route.test.ts @@ -4,10 +4,15 @@ import { setEnv } from '@sim/testing' import { NextRequest } from 'next/server' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { MANIFEST_ASSET_NAME } from '@/lib/desktop/update-feed' +import { + DESKTOP_PRERELEASE_REPOSITORY, + DESKTOP_STABLE_RELEASE_REPOSITORY, + MANIFEST_ASSET_NAME, +} from '@/lib/desktop/update-feed' import { GET } from '@/app/api/desktop/update/latest-mac.yml/route' -const RELEASES_URL = 'https://api.github.com/repos/simstudioai/sim/releases?per_page=30' +const STABLE_RELEASES_URL = `https://api.github.com/repos/${DESKTOP_STABLE_RELEASE_REPOSITORY}/releases?per_page=30` +const PRERELEASE_RELEASES_URL = `https://api.github.com/repos/${DESKTOP_PRERELEASE_REPOSITORY}/releases?per_page=30` const FEED_STATUS_HEADER = 'x-sim-desktop-update-feed' function release(tag: string) { @@ -49,36 +54,38 @@ describe('desktop update manifest route', () => { }) it.each([ - ['dev', 'v1.2.0-dev.4', '1.2.0-dev.4'], - ['staging', 'v1.2.0-staging.5', '1.2.0-staging.5'], - ['production', 'v1.1.0', '1.1.0'], - ])('serves the newest release for the %s deployment', async (environment, tag, version) => { - setEnv({ APPCONFIG_ENVIRONMENT: environment }) - fetchMock.mockImplementation(async (input: string | URL | Request) => { - const url = String(input) - if (url === RELEASES_URL) { - return Response.json([ - release('v1.2.0-dev.4'), - release('v1.2.0-staging.5'), - release('v1.1.0'), - ]) - } - if (url === `https://downloads.example/${tag}/${MANIFEST_ASSET_NAME}`) { - return new Response(manifest(version)) - } - return new Response(null, { status: 404 }) - }) + ['dev', 'v1.2.0-dev.4', '1.2.0-dev.4', DESKTOP_PRERELEASE_REPOSITORY], + ['staging', 'v1.2.0-staging.5', '1.2.0-staging.5', DESKTOP_PRERELEASE_REPOSITORY], + ['production', 'v1.1.0', '1.1.0', DESKTOP_STABLE_RELEASE_REPOSITORY], + ])( + 'serves the newest release for the %s deployment', + async (environment, tag, version, repository) => { + setEnv({ APPCONFIG_ENVIRONMENT: environment }) + fetchMock.mockImplementation(async (input: string | URL | Request) => { + const url = String(input) + if (url === PRERELEASE_RELEASES_URL) { + return Response.json([release('v1.2.0-dev.4'), release('v1.2.0-staging.5')]) + } + if (url === STABLE_RELEASES_URL) { + return Response.json([release('v1.1.0')]) + } + if (url === `https://downloads.example/${tag}/${MANIFEST_ASSET_NAME}`) { + return new Response(manifest(version)) + } + return new Response(null, { status: 404 }) + }) - const response = await getFeed('internal.service.local') - const body = await response.text() + const response = await getFeed('internal.service.local') + const body = await response.text() - expect(response.status).toBe(200) - expect(response.headers.get(FEED_STATUS_HEADER)).toBe('release') - expect(body).toContain(`version: ${version}`) - expect(body).toContain( - `https://github.com/simstudioai/sim/releases/download/${tag}/Sim-${version}-universal-mac.zip` - ) - }) + expect(response.status).toBe(200) + expect(response.headers.get(FEED_STATUS_HEADER)).toBe('release') + expect(body).toContain(`version: ${version}`) + expect(body).toContain( + `https://github.com/${repository}/releases/download/${tag}/Sim-${version}-universal-mac.zip` + ) + } + ) it.each([ ['dev', 'www.staging.sim.ai:443', 'v1.2.0-dev.4', '1.2.0-dev.4'], @@ -90,12 +97,11 @@ describe('desktop update manifest route', () => { setEnv({ APPCONFIG_ENVIRONMENT: environment }) fetchMock.mockImplementation(async (input: string | URL | Request) => { const url = String(input) - if (url === RELEASES_URL) { - return Response.json([ - release('v1.2.0-dev.4'), - release('v1.2.0-staging.5'), - release('v1.1.0'), - ]) + if (url === PRERELEASE_RELEASES_URL) { + return Response.json([release('v1.2.0-dev.4'), release('v1.2.0-staging.5')]) + } + if (url === STABLE_RELEASES_URL) { + return Response.json([release('v1.1.0')]) } if (url === `https://downloads.example/${tag}/${MANIFEST_ASSET_NAME}`) { return new Response(manifest(version)) @@ -118,8 +124,8 @@ describe('desktop update manifest route', () => { it('defaults self-hosted deployments to the stable channel', async () => { fetchMock.mockImplementation(async (input: string | URL | Request) => { const url = String(input) - if (url === RELEASES_URL) { - return Response.json([release('v1.2.0-dev.4'), release('v1.1.0')]) + if (url === STABLE_RELEASES_URL) { + return Response.json([release('v1.1.0')]) } if (url === `https://downloads.example/v1.1.0/${MANIFEST_ASSET_NAME}`) { return new Response(manifest('1.1.0')) @@ -134,6 +140,7 @@ describe('desktop update manifest route', () => { expect(response.status).toBe(200) expect(await response.text()).toContain('version: 1.1.0') + expect(fetchMock).toHaveBeenCalledWith(STABLE_RELEASES_URL, expect.any(Object)) }) it('reports an authoritative no-release result for production with only prereleases', async () => { @@ -159,5 +166,6 @@ describe('desktop update manifest route', () => { expect(response.status).toBe(502) expect(await response.json()).toMatchObject({ error: 'Release manifest unavailable' }) + expect(fetchMock).toHaveBeenNthCalledWith(1, PRERELEASE_RELEASES_URL, expect.any(Object)) }) }) diff --git a/apps/sim/app/api/desktop/update/latest-mac.yml/route.ts b/apps/sim/app/api/desktop/update/latest-mac.yml/route.ts index e3b483a210d..a731ba3be6f 100644 --- a/apps/sim/app/api/desktop/update/latest-mac.yml/route.ts +++ b/apps/sim/app/api/desktop/update/latest-mac.yml/route.ts @@ -4,9 +4,9 @@ import { env } from '@/lib/core/config/env' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { channelForDeploymentEnvironment, - DESKTOP_RELEASE_REPO, type DesktopReleaseCandidate, MANIFEST_ASSET_NAME, + releaseRepositoryForChannel, rewriteManifestUrls, selectReleaseForChannel, } from '@/lib/desktop/update-feed' @@ -20,8 +20,6 @@ const logger = createLogger('DesktopUpdateFeedAPI') const REVALIDATE_SECONDS = 300 const FEED_STATUS_HEADER = 'x-sim-desktop-update-feed' -const RELEASES_API_URL = `https://api.github.com/repos/${DESKTOP_RELEASE_REPO}/releases?per_page=30` - /** * The per-environment desktop update feed (see `lib/desktop/update-feed.ts`). * @@ -38,11 +36,13 @@ export const GET = withRouteHandler(async (_request: NextRequest): Promise ({ })) vi.mock('@/lib/uploads/contexts/workspace/workspace-file-manager', () => ({ - parseWorkspaceFileKey: vi.fn().mockReturnValue(undefined), + parseWorkspaceFileKey: mockParseWorkspaceFileKey, +})) + +vi.mock('@/lib/workspace-files/api', () => ({ + internalWorkspaceFileServeAuth: { authenticate: mockAuthenticateWorkspaceFile }, +})) + +vi.mock('@/lib/workspace-files/application/read-workspace-file-content-by-key', () => ({ + readWorkspaceFileContentByKey: { execute: mockReadWorkspaceFileContentByKey }, +})) + +vi.mock('@/lib/copilot/tools/server/files/doc-compile', () => ({ + resolveServableDocBytes: mockResolveServableDocBytes, })) vi.mock('@/app/api/files/utils', () => ({ @@ -109,7 +129,27 @@ describe('File Serve API Route', () => { mockReadFile.mockResolvedValue(Buffer.from('test content')) mockIsUsingCloudStorage.mockReturnValue(false) storageServiceMockFns.mockHasCloudStorage.mockReturnValue(true) - mockInferContextFromKey.mockReturnValue('workspace') + mockInferContextFromKey.mockReturnValue('mothership') + mockParseWorkspaceFileKey.mockReturnValue(undefined) + mockAuthenticateWorkspaceFile.mockResolvedValue({ + kind: 'session', + userId: 'test-user-id', + sessionId: 'session-1', + }) + mockReadWorkspaceFileContentByKey.mockResolvedValue({ + file: { + id: 'file-1', + workspaceId: 'test-workspace-id', + name: 'report.pdf', + }, + content: Buffer.from('generated source'), + }) + mockResolveServableDocBytes.mockImplementation( + async ({ rawBuffer, fileName }: { rawBuffer: Buffer; fileName: string }) => ({ + buffer: rawBuffer, + contentType: mockGetContentType(fileName), + }) + ) mockGetContentType.mockReturnValue('text/plain') mockFindLocalFile.mockReturnValue('/test/uploads/test-file.txt') mockCreateFileResponse.mockImplementation( @@ -181,8 +221,59 @@ describe('File Serve API Route', () => { expect(storageServiceMockFns.mockDownloadFile).toHaveBeenCalledWith({ key: 'workspace/test-workspace-id/1234567890-image.png', - context: 'workspace', + context: 'mothership', + }) + }) + + it('serves a workspace document through the authorized use case and preserves the Principal', async () => { + const principal = { + kind: 'delegated' as const, + serviceId: 'executor' as const, + subjectUserId: 'test-user-id', + workspaceId: 'test-workspace-id', + delegationId: 'delegation-1', + audience: 'sim:workspace-files', + issuedAt: new Date('2026-08-01T00:00:00Z'), + expiresAt: new Date('2026-08-01T01:00:00Z'), + delegationContext: { + kind: 'workflow_execution' as const, + workflowId: 'workflow-1', + }, + } + mockInferContextFromKey.mockReturnValue('workspace') + mockParseWorkspaceFileKey.mockReturnValue('test-workspace-id') + mockAuthenticateWorkspaceFile.mockResolvedValue(principal) + mockResolveServableDocBytes.mockResolvedValue({ + buffer: Buffer.from('%PDF-compiled'), + contentType: 'application/pdf', }) + + const req = new NextRequest( + 'http://localhost:3000/api/files/serve/workspace/test-workspace-id/report.pdf' + ) + const response = await GET(req, { + params: Promise.resolve({ + path: ['workspace', 'test-workspace-id', 'report.pdf'], + }), + }) + + expect(response.status).toBe(200) + expect(mockReadWorkspaceFileContentByKey).toHaveBeenCalledWith({ + principal, + input: { + key: 'workspace/test-workspace-id/report.pdf', + assertedWorkspaceId: 'test-workspace-id', + }, + request: req, + }) + expect(mockResolveServableDocBytes).toHaveBeenCalledWith( + expect.objectContaining({ + workspaceId: 'test-workspace-id', + filePrincipal: principal, + }) + ) + expect(hybridAuthMockFns.mockCheckSessionOrInternalAuth).not.toHaveBeenCalled() + expect(mockVerifyFileAccess).not.toHaveBeenCalled() }) it('should return 404 when file not found', async () => { diff --git a/apps/sim/app/api/files/serve/[...path]/route.ts b/apps/sim/app/api/files/serve/[...path]/route.ts index 7ba54de8e06..0899cdd0dfa 100644 --- a/apps/sim/app/api/files/serve/[...path]/route.ts +++ b/apps/sim/app/api/files/serve/[...path]/route.ts @@ -1,11 +1,17 @@ import { readFile } from 'fs/promises' +import { type Principal, requirePrincipalSubjectUserId } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import type { NextRequest } from 'next/server' import { NextResponse } from 'next/server' import { fileServeParamsSchema, fileServeQuerySchema } from '@/lib/api/contracts/storage-transfer' +import { + concealCrossTenantResourceError, + InternalUnauthenticatedError, +} from '@/lib/api/server/routes' import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' import { resolveServableDocBytes } from '@/lib/copilot/tools/server/files/doc-compile' import { DocCompileUserError } from '@/lib/copilot/tools/server/files/doc-compile-error' +import { asOrchestrationError } from '@/lib/core/orchestration/types' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { CopilotFiles, isUsingCloudStorage } from '@/lib/uploads' import type { StorageContext } from '@/lib/uploads/config' @@ -13,6 +19,8 @@ import { parseWorkspaceFileKey } from '@/lib/uploads/contexts/workspace/workspac import { downloadFile } from '@/lib/uploads/core/storage-service' import { resolveServableImageBytes } from '@/lib/uploads/server/image-derivative' import { inferContextFromKey } from '@/lib/uploads/utils/file-utils' +import { internalWorkspaceFileServeAuth } from '@/lib/workspace-files/api' +import { readWorkspaceFileContentByKey } from '@/lib/workspace-files/application/read-workspace-file-content-by-key' import { verifyFileAccess } from '@/app/api/files/authorization' import { createErrorResponse, @@ -66,9 +74,11 @@ async function resolveServableBytes(params: { workspaceId: string | undefined options: ServeOptions ownerKey: string | undefined + filePrincipal?: Principal signal: AbortSignal | undefined }): Promise<{ buffer: Buffer; contentType: string }> { - const { buffer, filename, storageKey, workspaceId, options, ownerKey, signal } = params + const { buffer, filename, storageKey, workspaceId, options, ownerKey, filePrincipal, signal } = + params if (options.raw) return { buffer, contentType: getContentType(filename) } if (options.preview) { @@ -82,6 +92,7 @@ async function resolveServableBytes(params: { rawBuffer: buffer, fileName: filename, workspaceId, + filePrincipal, ownerKey, signal, }) @@ -154,6 +165,23 @@ export const GET = withRouteHandler( return await handleLocalFilePublic(fullPath) } + const storageContext = inferContextFromKey(cloudKey) + const workspacePrincipal = + storageContext === 'workspace' + ? await internalWorkspaceFileServeAuth.authenticate(request, { path }) + : undefined + const legacyAuthResult = workspacePrincipal + ? undefined + : await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) + + if (legacyAuthResult && (!legacyAuthResult.success || !legacyAuthResult.userId)) { + logger.warn('Unauthorized file access attempt', { + path, + error: legacyAuthResult.error || 'Missing userId', + }) + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + } + const query = fileServeQuerySchema.parse({ raw: request.nextUrl.searchParams.get('raw'), preview: request.nextUrl.searchParams.get('preview'), @@ -165,17 +193,12 @@ export const GET = withRouteHandler( versioned: query.v != null, } - const authResult = await checkSessionOrInternalAuth(request, { requireWorkflowId: false }) - - if (!authResult.success || !authResult.userId) { - logger.warn('Unauthorized file access attempt', { - path, - error: authResult.error || 'Missing userId', - }) - return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + if (workspacePrincipal) { + return await handleWorkspaceFile(cloudKey, workspacePrincipal, options, request) } - const userId = authResult.userId + const userId = legacyAuthResult?.userId + if (!userId) throw new Error('Authenticated file serve request is missing a user ID') if (isUsingCloudStorage()) { return await handleCloudProxy(cloudKey, userId, options, request.signal) @@ -183,6 +206,11 @@ export const GET = withRouteHandler( return await handleLocalFile(cloudKey, userId, options, request.signal) } catch (error) { + if (error instanceof InternalUnauthenticatedError) { + logger.warn('Unauthorized file access attempt', { error: error.message }) + return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + } + // An in-progress/incomplete doc source fails to compile — this is expected // mid-generation, not a server fault. Return 409 (not 500) so it isn't an // alarming error; the client re-fetches once the doc finishes (the serve @@ -194,6 +222,15 @@ export const GET = withRouteHandler( return NextResponse.json({ error: 'Document is still being generated' }, { status: 409 }) } + const orchestrationError = asOrchestrationError( + concealCrossTenantResourceError(error, 'File not found') + ) + if (orchestrationError?.code === 'not_found') { + const notFound = new FileNotFoundError('File not found') + logServeFailure('Error serving file:', notFound) + return createErrorResponse(notFound) + } + logServeFailure('Error serving file:', error) if (error instanceof FileNotFoundError) { @@ -205,6 +242,45 @@ export const GET = withRouteHandler( } ) +async function handleWorkspaceFile( + key: string, + principal: Principal, + options: ServeOptions, + request: NextRequest +): Promise { + const workspaceId = getWorkspaceIdForCompile(key) + if (!workspaceId) throw new FileNotFoundError(`File not found: ${key}`) + + const { file, content } = await readWorkspaceFileContentByKey.execute({ + principal, + input: { key, assertedWorkspaceId: workspaceId }, + request, + }) + const ownerKey = `user:${requirePrincipalSubjectUserId(principal)}` + const resolved = await resolveServableBytes({ + buffer: content, + filename: file.name, + storageKey: key, + workspaceId, + options, + ownerKey, + filePrincipal: principal, + signal: request.signal, + }) + + logger.info('Workspace file served', { + fileId: file.id, + workspaceId, + size: resolved.buffer.length, + }) + return createFileResponse({ + buffer: resolved.buffer, + contentType: resolved.contentType, + filename: file.name, + cacheControl: resolveServeCacheControl(options.versioned, 'workspace'), + }) +} + async function handleLocalFile( filename: string, userId: string, diff --git a/apps/sim/app/api/help/route.ts b/apps/sim/app/api/help/route.ts index b5c25a9c5c3..3bbb7fef636 100644 --- a/apps/sim/app/api/help/route.ts +++ b/apps/sim/app/api/help/route.ts @@ -6,6 +6,7 @@ import { validationErrorResponse } from '@/lib/api/server' import { getSession } from '@/lib/auth' import { generateRequestId } from '@/lib/core/utils/request' import { + isMultipartFieldValidationError, isPayloadSizeLimitError, MAX_MULTIPART_OVERHEAD_BYTES, readFormDataWithLimit, @@ -145,6 +146,9 @@ ${message} { status: 200 } ) } catch (error) { + if (isMultipartFieldValidationError(error)) { + return NextResponse.json({ error: error.message }, { status: 400 }) + } if (isPayloadSizeLimitError(error)) { logger.warn(`[${requestId}] Help request form data too large`, { message: error.message }) return NextResponse.json( diff --git a/apps/sim/app/api/mcp/oauth/callback/route.ts b/apps/sim/app/api/mcp/oauth/callback/route.ts index 9d63191fb32..ff76af5b2a3 100644 --- a/apps/sim/app/api/mcp/oauth/callback/route.ts +++ b/apps/sim/app/api/mcp/oauth/callback/route.ts @@ -208,7 +208,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => { try { // forceRefresh: skip any stale cache from before re-auth. await timedStep('discoverServerTools', 60_000, () => - mcpService.discoverServerTools(session.user.id, server.id, server.workspaceId, true) + mcpService.discoverServerTools(session.user.id, server.id, server.workspaceId, 'force') ) } catch (e) { logger.warn('Post-auth tools refresh failed', toError(e).message) diff --git a/apps/sim/app/api/mcp/servers/[id]/refresh/route.test.ts b/apps/sim/app/api/mcp/servers/[id]/refresh/route.test.ts index e31560a561d..55eb87f4ce7 100644 --- a/apps/sim/app/api/mcp/servers/[id]/refresh/route.test.ts +++ b/apps/sim/app/api/mcp/servers/[id]/refresh/route.test.ts @@ -98,6 +98,35 @@ describe('MCP server refresh route', () => { ) }) + /** + * `updatedAt` means "when the server's configuration last changed" and is one + * of the public list's keyset sorts, so a refresh must not stamp it. The + * service's discovery status write already holds that invariant; this route + * writes the same row from the UI's refresh button, and stamping it here moves + * the row to the head of `sortBy=updatedAt` under an in-flight v2 page, which + * duplicates some servers across pages and skips others. Liveness is published + * through `lastToolsRefresh`, `lastConnected`, and `lastError`. + */ + it('records the refresh without stamping updatedAt', async () => { + mockDiscoverServerTools.mockResolvedValueOnce([]) + + const request = new Request('http://localhost/api/mcp/servers/server-1/refresh', { + method: 'POST', + }) as NextRequest + await POST(request, { params: Promise.resolve({ id: 'server-1' }) }) + + const refreshWrites = dbChainMockFns.set.mock.calls.filter( + ([values]) => (values as Record)?.lastToolsRefresh !== undefined + ) + expect(refreshWrites.length).toBeGreaterThan(0) + for (const [values] of refreshWrites) { + expect( + (values as Record).updatedAt, + 'the refresh route stamped updatedAt, corrupting the updatedAt keyset page' + ).toBeUndefined() + } + }) + it('reports the discovery failure when status persistence leaves a stale connected row', async () => { const reflectedSecret = 'Bearer reflected-static-token' mockDiscoverServerTools.mockRejectedValueOnce( diff --git a/apps/sim/app/api/mcp/servers/[id]/refresh/route.ts b/apps/sim/app/api/mcp/servers/[id]/refresh/route.ts index b1ceda9d016..550aa2f77d3 100644 --- a/apps/sim/app/api/mcp/servers/[id]/refresh/route.ts +++ b/apps/sim/app/api/mcp/servers/[id]/refresh/route.ts @@ -196,7 +196,7 @@ export const POST = withRouteHandler( userId, serverId, workspaceId, - true + 'force' ) logger.info( `[${requestId}] Discovered ${discoveredTools.length} tools from server ${serverId}` @@ -229,11 +229,20 @@ export const POST = withRouteHandler( const now = new Date() + /** + * Deliberately leaves `updatedAt` alone, matching the invariant + * `McpService.updateServerStatus` holds: `updatedAt` means "when the + * server's configuration last changed", and it is one of the public + * list's keyset sorts. A refresh stamping it moves the row to the head + * of `sortBy=updatedAt` under an in-flight page, so a caller walking the + * list while anyone presses this button sees servers duplicated across + * pages and others skipped. Refresh liveness is already published + * through `lastToolsRefresh`, `lastConnected`, and `lastError`. + */ const [refreshedServer] = await db .update(mcpServers) .set({ lastToolsRefresh: now, - updatedAt: now, }) .where( and( diff --git a/apps/sim/app/api/mcp/tools/discover/route.ts b/apps/sim/app/api/mcp/tools/discover/route.ts index 84acdad0b3d..a592dc116bb 100644 --- a/apps/sim/app/api/mcp/tools/discover/route.ts +++ b/apps/sim/app/api/mcp/tools/discover/route.ts @@ -65,8 +65,17 @@ export const GET = withRouteHandler( logger.info(`[${requestId}] Discovering MCP tools`, { serverId, workspaceId, forceRefresh }) const tools = serverId - ? await mcpService.discoverServerTools(userId, serverId, workspaceId, forceRefresh) - : await mcpService.discoverTools(userId, workspaceId, forceRefresh) + ? await mcpService.discoverServerTools( + userId, + serverId, + workspaceId, + forceRefresh ? 'force' : 'cache-aside' + ) + : await mcpService.discoverTools( + userId, + workspaceId, + forceRefresh ? 'force' : 'cache-aside' + ) const byServer: Record = {} for (const tool of tools) { @@ -115,7 +124,7 @@ export const POST = withRouteHandler( serverIds, MCP_REFRESH_DISCOVERY_CONCURRENCY, async (serverId: string) => { - const tools = await mcpService.discoverServerTools(userId, serverId, workspaceId, true) + const tools = await mcpService.discoverServerTools(userId, serverId, workspaceId, 'force') return { serverId, toolCount: tools.length } } ) diff --git a/apps/sim/app/api/mcp/tools/execute/route.ts b/apps/sim/app/api/mcp/tools/execute/route.ts index d045b407a54..cd371580e42 100644 --- a/apps/sim/app/api/mcp/tools/execute/route.ts +++ b/apps/sim/app/api/mcp/tools/execute/route.ts @@ -157,7 +157,7 @@ export const POST = withRouteHandler( userId, serverId, workspaceId, - false, + 'cache-aside', recordProvenance ) tool = tools.find((t) => t.name === toolName) ?? null diff --git a/apps/sim/app/api/pinned-items/route.ts b/apps/sim/app/api/pinned-items/route.ts index bf31285fb61..376bf510f60 100644 --- a/apps/sim/app/api/pinned-items/route.ts +++ b/apps/sim/app/api/pinned-items/route.ts @@ -2,40 +2,21 @@ import { db, pinnedItem } from '@sim/db' import { createLogger } from '@sim/logger' import { getPostgresErrorCode } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' -import { and, eq, ne } from 'drizzle-orm' import { type NextRequest, NextResponse } from 'next/server' import { createPinnedItemContract, listPinnedItemsContract, type PinnedItemApi, - pinnedResourceTypeSchema, } from '@/lib/api/contracts' import { parseRequest } from '@/lib/api/server' import { getSession } from '@/lib/auth' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' -import { filterToActiveResources, pinnableResourceExists } from '@/lib/pinned-items/resources' +import { listPinnedItemsForUser } from '@/lib/pinned-items/queries' +import { pinnableResourceExists } from '@/lib/pinned-items/resources' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' const logger = createLogger('PinnedItemsAPI') -/** - * Narrows a stored row to the wire shape, dropping any row whose `resourceType` this build does - * not recognise. - * - * `pinned_item.resource_type` is plain `text` — deliberately, so the set of pinnable kinds can - * grow — while the contract is a closed enum. During a rolling deploy an older pod can therefore - * read a pin a newer one wrote. Returning it would fail response validation and take the WHOLE - * list down rather than the single row, so the unknown kind is skipped instead. - * - * `filterToActiveResources` already drops these as a side effect of not having a table to look - * them up in; this makes the guarantee explicit and compiler-checked at the wire boundary. - */ -function toPinnedItemApi(row: typeof pinnedItem.$inferSelect): PinnedItemApi | null { - const resourceType = pinnedResourceTypeSchema.safeParse(row.resourceType) - if (!resourceType.success) return null - return { ...row, resourceType: resourceType.data, pinnedAt: row.pinnedAt.toISOString() } -} - /** Lists the session user's pinned items in a workspace, optionally filtered to one `resourceType`. */ export const GET = withRouteHandler(async (request: NextRequest) => { const session = await getSession() @@ -52,30 +33,7 @@ export const GET = withRouteHandler(async (request: NextRequest) => { return NextResponse.json({ error: 'Access denied to this workspace' }, { status: 403 }) } - const rows = await db - .select() - .from(pinnedItem) - .where( - and( - eq(pinnedItem.userId, session.user.id), - eq(pinnedItem.workspaceId, workspaceId), - /** - * A `workspace` pin stores `workspaceId === resourceId`, so it would otherwise - * appear in this workspace's unscoped listing as a resource *inside* itself. - * It is read from the workspace-list payload instead, so it is excluded here - * rather than left for a future unscoped caller to mistake for a real resource. - */ - resourceType - ? eq(pinnedItem.resourceType, resourceType) - : ne(pinnedItem.resourceType, 'workspace') - ) - ) - - const activeRows = await filterToActiveResources(rows, workspaceId) - - const pinnedItems = activeRows - .map(toPinnedItemApi) - .filter((item): item is PinnedItemApi => item !== null) + const pinnedItems = await listPinnedItemsForUser(session.user.id, workspaceId, resourceType) return NextResponse.json({ pinnedItems }) }) diff --git a/apps/sim/app/api/table/[tableId]/columns/route.test.ts b/apps/sim/app/api/table/[tableId]/columns/route.test.ts index 6223c12bff6..24830309efc 100644 --- a/apps/sim/app/api/table/[tableId]/columns/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/columns/route.test.ts @@ -49,10 +49,12 @@ vi.mock('@/lib/table/columns/service', () => ({ updateColumnOptions: mockUpdateColumnOptions, updateColumnType: mockUpdateColumnType, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (c: unknown) => c, +})) vi.mock('@/app/api/table/utils', () => ({ accessError: () => new Response('denied', { status: 403 }), checkAccess: mockCheckAccess, - normalizeColumn: (c: unknown) => c, orchestrationOutcomeErrorResponse: ( outcome: { error?: string; errorCode?: OrchestrationErrorCode }, fallback: string diff --git a/apps/sim/app/api/table/[tableId]/columns/route.ts b/apps/sim/app/api/table/[tableId]/columns/route.ts index 54ca6de54e8..2b2aa60c131 100644 --- a/apps/sim/app/api/table/[tableId]/columns/route.ts +++ b/apps/sim/app/api/table/[tableId]/columns/route.ts @@ -13,10 +13,10 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { addTableColumn, deleteColumn } from '@/lib/table' import { signalTableSchemaChanged } from '@/lib/table/events' import { performUpdateTableColumn } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, rootErrorMessage, tableLockErrorResponse, diff --git a/apps/sim/app/api/table/[tableId]/groups/route.test.ts b/apps/sim/app/api/table/[tableId]/groups/route.test.ts index cad09be8b65..7d628cfdeb3 100644 --- a/apps/sim/app/api/table/[tableId]/groups/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/groups/route.test.ts @@ -54,7 +54,7 @@ vi.mock('@/lib/table/application/groups', () => ({ updateTableGroupUseCase: mocks.useCases.update, })) -vi.mock('@/app/api/table/utils', () => ({ +vi.mock('@/lib/table/wire', () => ({ normalizeColumn: vi.fn(), })) diff --git a/apps/sim/app/api/table/[tableId]/groups/route.ts b/apps/sim/app/api/table/[tableId]/groups/route.ts index b1f9a1c4749..f8f14909ac4 100644 --- a/apps/sim/app/api/table/[tableId]/groups/route.ts +++ b/apps/sim/app/api/table/[tableId]/groups/route.ts @@ -12,7 +12,7 @@ import { } from '@/lib/table/application/groups' import { tableOperations } from '@/lib/table/application/operations' import type { TableDefinition } from '@/lib/table/types' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' const rateLimit = internalRateLimits.none({ reason: 'Existing authenticated table group mutations have no request-rate policy', diff --git a/apps/sim/app/api/table/[tableId]/query/route.ts b/apps/sim/app/api/table/[tableId]/query/route.ts index 9d156eeb656..ebfb0507504 100644 --- a/apps/sim/app/api/table/[tableId]/query/route.ts +++ b/apps/sim/app/api/table/[tableId]/query/route.ts @@ -10,7 +10,7 @@ import type { Sort, TableSchema } from '@/lib/table' import { buildIdByName, sortSpecNamesToIds } from '@/lib/table/column-keys' import { TableQueryValidationError } from '@/lib/table/errors' import { validatePredicate, validateSortSpec } from '@/lib/table/query-builder/validate' -import { assertCursorSortBinding, decodeCursor } from '@/lib/table/rows/cursor' +import { assertCursorQueryBinding, decodeCursor } from '@/lib/table/rows/cursor' import { queryRows } from '@/lib/table/rows/service' import { predicateToStorage } from '@/lib/table/select-values' import { createTableRowsResponse } from '@/app/api/table/row-secret-provenance' @@ -84,7 +84,7 @@ export const POST = withRouteHandler(async (request: NextRequest, context: RowQu // Cursor↔sort binding: keyset cursors are default-order only; an offset // cursor must be replayed under the exact sort it was minted with. - if (cursor) assertCursorSortBinding(cursor, sort) + if (cursor) assertCursorQueryBinding(cursor, { sort, predicate }) const result = await queryRows( table, diff --git a/apps/sim/app/api/table/[tableId]/route.test.ts b/apps/sim/app/api/table/[tableId]/route.test.ts index 43cbf68ae83..6e1b9a957c9 100644 --- a/apps/sim/app/api/table/[tableId]/route.test.ts +++ b/apps/sim/app/api/table/[tableId]/route.test.ts @@ -51,9 +51,11 @@ vi.mock('@/lib/workspaces/permissions/utils', () => ({ vi.mock('@/app/api/table/utils', () => ({ accessError: () => new Response('denied', { status: 403 }), checkAccess: mockCheckAccess, - normalizeColumn: (column: unknown) => column, tableLockErrorResponse: () => null, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (column: unknown) => column, +})) import { GET, PATCH } from '@/app/api/table/[tableId]/route' diff --git a/apps/sim/app/api/table/[tableId]/route.ts b/apps/sim/app/api/table/[tableId]/route.ts index e14a3bdf775..4f61ce4ea12 100644 --- a/apps/sim/app/api/table/[tableId]/route.ts +++ b/apps/sim/app/api/table/[tableId]/route.ts @@ -18,11 +18,11 @@ import { performUpdateTableLocks, } from '@/lib/table/orchestration' import { TABLE_LOCK_FLAGS, TABLE_LOCK_KINDS } from '@/lib/table/types' +import { normalizeColumn } from '@/lib/table/wire' import { getWorkspaceWithOwner } from '@/lib/workspaces/permissions/utils' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, tableLockErrorResponse, } from '@/app/api/table/utils' diff --git a/apps/sim/app/api/table/[tableId]/rows/[rowId]/route.ts b/apps/sim/app/api/table/[tableId]/rows/[rowId]/route.ts index 67a293081c0..66ec90870f8 100644 --- a/apps/sim/app/api/table/[tableId]/rows/[rowId]/route.ts +++ b/apps/sim/app/api/table/[tableId]/rows/[rowId]/route.ts @@ -3,6 +3,7 @@ import { userTableRows } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { and, eq } from 'drizzle-orm' import { type NextRequest, NextResponse } from 'next/server' +import { readClientId } from '@/lib/api/client-id' import { deleteTableRowContract, getTableQuerySchema, @@ -14,7 +15,7 @@ import { generateRequestId } from '@/lib/core/utils/request' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import type { RowData, TableSchema } from '@/lib/table' import { updateRow } from '@/lib/table' -import { signalTableRowsChanged } from '@/lib/table/events' +import { signalTableRowsChangedByActor } from '@/lib/table/events' import { performDeleteTableRow } from '@/lib/table/orchestration' import { createTableRowsResponse, @@ -172,7 +173,7 @@ export const PATCH = withRouteHandler(async (request: NextRequest, context: RowR ) // Live-collab: tell open viewers the change landed so they refetch. - signalTableRowsChanged(tableId) + signalTableRowsChangedByActor(tableId, readClientId(request)) // Only `null` when a `cancellationGuard` is supplied and the SQL guard // rejects the write — this route doesn't pass one, so reaching null is a bug. if (!updatedRow) throw new Error('updateRow returned null without a cancellationGuard') @@ -251,7 +252,7 @@ export const DELETE = withRouteHandler(async (request: NextRequest, context: Row } // Live-collab: tell open viewers the change landed so they refetch. - signalTableRowsChanged(tableId) + signalTableRowsChangedByActor(tableId, readClientId(request)) return NextResponse.json({ success: true, diff --git a/apps/sim/app/api/table/[tableId]/rows/route.ts b/apps/sim/app/api/table/[tableId]/rows/route.ts index 461e8040b19..869869c18b6 100644 --- a/apps/sim/app/api/table/[tableId]/rows/route.ts +++ b/apps/sim/app/api/table/[tableId]/rows/route.ts @@ -1,5 +1,6 @@ import { createLogger } from '@sim/logger' import { type NextRequest, NextResponse } from 'next/server' +import { readClientId } from '@/lib/api/client-id' import { type BatchInsertTableRowsBodyInput, batchUpdateTableRowsBodySchema, @@ -26,7 +27,7 @@ import { validateRowSize, } from '@/lib/table' import { TableQueryValidationError } from '@/lib/table/errors' -import { signalTableRowsChanged } from '@/lib/table/events' +import { signalTableRowsChanged, signalTableRowsChangedByActor } from '@/lib/table/events' import { isTablePredicate, predicateToFilter } from '@/lib/table/query-builder/converters' import { validatePredicateShape, @@ -254,7 +255,9 @@ export const POST = withRouteHandler( table, requestId ) - signalTableRowsChanged(tableId) + // Attributed unlike the batch path above: the acting tab's insert deliberately avoids + // invalidating the rows root to prevent flicker, which an unattributed echo would undo. + signalTableRowsChangedByActor(tableId, readClientId(request)) const responseBody = { success: true, diff --git a/apps/sim/app/api/table/import-csv/route.test.ts b/apps/sim/app/api/table/import-csv/route.test.ts index dae8f0c3d63..dea46a06a3a 100644 --- a/apps/sim/app/api/table/import-csv/route.test.ts +++ b/apps/sim/app/api/table/import-csv/route.test.ts @@ -34,7 +34,6 @@ vi.mock('@/app/api/table/utils', async () => { const { asOrchestrationError, messageForOrchestrationError, statusForOrchestrationError } = await import('@/lib/core/orchestration/types') return { - normalizeColumn: (column: unknown) => column, csvProxyBodyCapResponse: () => null, multipartErrorResponse: (error: { code: string; message: string }) => NextResponse.json( diff --git a/apps/sim/app/api/table/route.ts b/apps/sim/app/api/table/route.ts index 28714885cb5..be28a064fd1 100644 --- a/apps/sim/app/api/table/route.ts +++ b/apps/sim/app/api/table/route.ts @@ -15,8 +15,9 @@ import { type TableSchema, type TableScope, } from '@/lib/table' +import { normalizeColumn, toTableListItem, toWireTimestamp } from '@/lib/table/wire' import { getUserEntityPermissions } from '@/lib/workspaces/permissions/utils' -import { normalizeColumn, orchestrationErrorResponse } from '@/app/api/table/utils' +import { orchestrationErrorResponse } from '@/app/api/table/utils' const logger = createLogger('TableAPI') @@ -140,14 +141,8 @@ export const POST = withRouteHandler(async (request: NextRequest) => { maxRows: table.maxRows, folderId: table.folderId ?? null, locks: table.locks, - createdAt: - table.createdAt instanceof Date - ? table.createdAt.toISOString() - : String(table.createdAt), - updatedAt: - table.updatedAt instanceof Date - ? table.updatedAt.toISOString() - : String(table.updatedAt), + createdAt: toWireTimestamp(table.createdAt), + updatedAt: toWireTimestamp(table.updatedAt), }, message: 'Table created successfully', }, @@ -198,41 +193,10 @@ export const GET = withRouteHandler(async (request: NextRequest) => { logger.info(`[${requestId}] Listed ${tables.length} tables in workspace ${params.workspaceId}`) - const responseTables = tables.map((t) => { - const schemaData = t.schema as TableSchema - return { - id: t.id, - name: t.name, - description: t.description, - schema: { - columns: schemaData.columns.map(normalizeColumn), - }, - rowCount: t.rowCount, - maxRows: t.maxRows, - locks: t.locks, - workspaceId: t.workspaceId, - folderId: t.folderId ?? null, - createdBy: t.createdBy, - createdAt: t.createdAt instanceof Date ? t.createdAt.toISOString() : String(t.createdAt), - updatedAt: t.updatedAt instanceof Date ? t.updatedAt.toISOString() : String(t.updatedAt), - archivedAt: - t.archivedAt instanceof Date - ? t.archivedAt.toISOString() - : t.archivedAt - ? String(t.archivedAt) - : null, - jobStatus: t.jobStatus ?? null, - jobId: t.jobId ?? null, - jobType: t.jobType ?? null, - jobError: t.jobError ?? null, - jobRowsProcessed: t.jobRowsProcessed ?? 0, - } - }) - return NextResponse.json({ success: true, data: { - tables: responseTables, + tables: tables.map(toTableListItem), totalCount: tables.length, }, }) diff --git a/apps/sim/app/api/table/utils.ts b/apps/sim/app/api/table/utils.ts index e049b1a3f68..037c6c83cc8 100644 --- a/apps/sim/app/api/table/utils.ts +++ b/apps/sim/app/api/table/utils.ts @@ -17,7 +17,6 @@ import { import type { MultipartError } from '@/lib/core/utils/multipart' import type { ColumnDefinition, Filter, TableDefinition, TablePredicate } from '@/lib/table' import { buildFilterClause, getTableById, TableQueryValidationError } from '@/lib/table' -import { typeMetadataOf } from '@/lib/table/column-types' import { USER_TABLE_ROWS_SQL_NAME } from '@/lib/table/constants' import { TableLockedError } from '@/lib/table/mutation-locks' import { isTablePredicate } from '@/lib/table/query-builder/converters' @@ -358,21 +357,3 @@ export function serverErrorResponse(message = 'Internal server error') { export const CreateColumnSchema = createTableColumnBodySchema export const UpdateColumnSchema = updateTableColumnBodySchema export const DeleteColumnSchema = deleteTableColumnBodySchema - -export function normalizeColumn( - col: ColumnDefinition -): ColumnDefinition & { required: boolean; unique: boolean } { - return { - // Preserve the stable column id — it's the row-data storage key, so dropping - // it makes clients fall back to `name` and miss id-keyed cell values. - ...(col.id ? { id: col.id } : {}), - name: col.name, - type: col.type, - required: col.required ?? false, - unique: col.unique ?? false, - ...(col.workflowGroupId ? { workflowGroupId: col.workflowGroupId } : {}), - // Type-specific metadata is forwarded generically: naming keys here meant a - // new type's metadata was stored server-side but silently never returned. - ...typeMetadataOf(col), - } -} diff --git a/apps/sim/app/api/tools/netsuite/objects/route.test.ts b/apps/sim/app/api/tools/netsuite/objects/route.test.ts new file mode 100644 index 00000000000..a709cf52da0 --- /dev/null +++ b/apps/sim/app/api/tools/netsuite/objects/route.test.ts @@ -0,0 +1,462 @@ +/** + * @vitest-environment node + */ +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' +import { TokenServiceAccountValidationError } from '@/lib/credentials/token-service-accounts/errors' + +const { + mockAuthorizeCredentialUse, + mockCheckSessionOrInternalAuth, + mockGetAsyncStatus, + mockListRecordTypes, + mockResolveCredentialAccessToken, + mockResolveOAuthAccountId, +} = vi.hoisted(() => ({ + mockAuthorizeCredentialUse: vi.fn(), + mockCheckSessionOrInternalAuth: vi.fn(), + mockGetAsyncStatus: vi.fn(), + mockListRecordTypes: vi.fn(), + mockResolveCredentialAccessToken: vi.fn(), + mockResolveOAuthAccountId: vi.fn(), +})) + +vi.mock('@/lib/auth/credential-access', () => ({ + authorizeCredentialUse: mockAuthorizeCredentialUse, +})) +vi.mock('@/lib/auth/hybrid', () => ({ + checkSessionOrInternalAuth: mockCheckSessionOrInternalAuth, +})) +vi.mock('@/lib/oauth/credential-service', () => ({ + resolveCredentialAccessToken: mockResolveCredentialAccessToken, + resolveOAuthAccountId: mockResolveOAuthAccountId, +})) +vi.mock('@/tools/netsuite/get_async_status', () => ({ + netsuiteGetAsyncStatusTool: { directExecution: mockGetAsyncStatus }, +})) +vi.mock('@/tools/netsuite/list_record_types', () => ({ + netsuiteListRecordTypesTool: { directExecution: mockListRecordTypes }, +})) + +import { POST } from '@/app/api/tools/netsuite/objects/route' + +const URL = 'http://localhost:3000/api/tools/netsuite/objects' +const ORIGIN = 'https://1234567.suitetalk.api.netsuite.com' +const RECORD_TYPES_BODY = { + credential: 'credential-1', + workflowId: 'workflow-1', + kind: 'record_types', +} as const + +function request( + body: unknown, + signal?: AbortSignal, + headers: Record = {} +): NextRequest { + return new NextRequest(URL, { + method: 'POST', + headers: { 'content-type': 'application/json', ...headers }, + body: typeof body === 'string' ? body : JSON.stringify(body), + signal, + }) +} + +async function json(response: Response): Promise> { + return (await response.json()) as Record +} + +function success(data: unknown) { + return { success: true, output: { status: 200, data } } +} + +function failure(status?: number) { + return { success: false, output: { status, data: null, error: 'provider secret' } } +} + +describe('POST /api/tools/netsuite/objects', () => { + beforeEach(() => { + vi.clearAllMocks() + mockCheckSessionOrInternalAuth.mockResolvedValue({ success: true, userId: 'user-1' }) + mockAuthorizeCredentialUse.mockResolvedValue({ + ok: true, + credentialOwnerUserId: 'owner-1', + resolvedCredentialId: 'resolved-credential-1', + credentialType: 'service_account', + }) + mockResolveOAuthAccountId.mockResolvedValue({ + credentialType: 'service_account', + providerId: 'netsuite-service-account', + }) + mockResolveCredentialAccessToken.mockResolvedValue({ + accessToken: 'short-lived-token', + instanceUrl: ORIGIN, + }) + mockListRecordTypes.mockResolvedValue(success({ items: [{ name: 'customer' }] })) + mockGetAsyncStatus.mockResolvedValue(success({ items: [] })) + }) + + it.each([ + ['unauthenticated malformed input', '{not-json', {}, 'Unauthorized'], + [ + 'API-key caller', + RECORD_TYPES_BODY, + { 'x-api-key': 'external-api-key' }, + 'API key access not allowed for this endpoint', + ], + ])('authenticates before parsing and rejects an %s', async (_label, body, headers, error) => { + mockCheckSessionOrInternalAuth.mockResolvedValueOnce({ success: false, error }) + + const response = await POST(request(body, undefined, headers), {}) + + expect(response.status).toBe(401) + expect(await json(response)).toMatchObject({ error }) + expect(mockCheckSessionOrInternalAuth).toHaveBeenCalledWith(expect.any(NextRequest), { + requireWorkflowId: true, + }) + expect(mockAuthorizeCredentialUse).not.toHaveBeenCalled() + }) + + it.each([ + ['invalid JSON', '{not-json', 400], + ['removed dataset selector kind', { ...RECORD_TYPES_BODY, kind: 'datasets' }, 400], + [ + 'missing async job', + { credential: 'credential-1', workflowId: 'workflow-1', kind: 'async_tasks' }, + 400, + ], + ['unexpected record-type job', { ...RECORD_TYPES_BODY, jobId: 'job-1' }, 400], + ['oversized body', { ...RECORD_TYPES_BODY, padding: 'x'.repeat(17 * 1024) }, 413], + ])('rejects %s', async (_label, body, expectedStatus) => { + const response = await POST(request(body), {}) + + expect(response.status).toBe(expectedStatus) + expect(mockAuthorizeCredentialUse).not.toHaveBeenCalled() + }) + + it('authorizes the exact credential and injects only resolved provider authentication', async () => { + const controller = new AbortController() + const discoveryRequest = request(RECORD_TYPES_BODY, controller.signal) + const response = await POST(discoveryRequest, {}) + + expect(response.status).toBe(200) + expect(mockAuthorizeCredentialUse).toHaveBeenCalledWith(expect.any(NextRequest), { + credentialId: 'credential-1', + workflowId: 'workflow-1', + callerUserId: 'user-1', + }) + expect(mockResolveOAuthAccountId).toHaveBeenCalledWith('resolved-credential-1') + expect(mockResolveCredentialAccessToken).toHaveBeenCalledWith( + 'resolved-credential-1', + 'owner-1', + expect.any(String) + ) + expect(mockListRecordTypes).toHaveBeenCalledWith( + { + oauthCredential: 'resolved-credential-1', + accessToken: 'short-lived-token', + instanceUrl: ORIGIN, + }, + discoveryRequest.signal + ) + }) + + it.each([ + [ + 'credential authorization', + () => mockAuthorizeCredentialUse.mockResolvedValueOnce({ ok: false, error: 'Forbidden' }), + 403, + ], + [ + 'non-service-account credential', + () => + mockAuthorizeCredentialUse.mockResolvedValueOnce({ + ok: true, + credentialOwnerUserId: 'owner-1', + credentialType: 'oauth', + }), + 400, + ], + [ + 'wrong service-account provider', + () => + mockResolveOAuthAccountId.mockResolvedValueOnce({ + credentialType: 'service_account', + providerId: 'snowflake-service-account', + }), + 400, + ], + ])('fails closed on invalid %s', async (_label, arrange, expectedStatus) => { + arrange() + + const response = await POST(request(RECORD_TYPES_BODY), {}) + + expect(response.status).toBe(expectedStatus) + expect(mockListRecordTypes).not.toHaveBeenCalled() + }) + + it('dispatches async task discovery with the exact job, view, auth, and signal', async () => { + const body = { ...RECORD_TYPES_BODY, kind: 'async_tasks', jobId: 'job 1' } as const + const task2 = '/services/rest/async/v1/job/job%201/task/task-2' + const task1 = `${ORIGIN}/services/rest/async/v1/job/job%201/task/task-1` + mockGetAsyncStatus.mockResolvedValueOnce( + success({ + items: [ + { links: [{ rel: 'self', href: task2 }] }, + { + links: [ + { rel: 'self', href: task1 }, + { rel: 'self', href: task1 }, + ], + }, + ], + }) + ) + + const discoveryRequest = request(body) + const response = await POST(discoveryRequest, {}) + + expect(response.status).toBe(200) + expect(mockGetAsyncStatus).toHaveBeenCalledWith( + { + oauthCredential: 'resolved-credential-1', + accessToken: 'short-lived-token', + instanceUrl: ORIGIN, + jobId: 'job 1', + view: 'tasks', + }, + discoveryRequest.signal + ) + expect(await json(response)).toEqual({ + objects: [ + { id: 'task-1', label: 'task-1', detail: null }, + { id: 'task-2', label: 'task-2', detail: null }, + ], + }) + }) + + it('skips non-self task link relationships instead of failing discovery', async () => { + const href = `${ORIGIN}/services/rest/async/v1/job/job-1/task/task-1` + mockGetAsyncStatus.mockResolvedValueOnce( + success({ + items: [ + { + links: [ + { rel: 'canonical', href: `${ORIGIN}/services/rest/async/v1/job/job-1` }, + { rel: 'self', href }, + ], + }, + ], + }) + ) + + const response = await POST( + request({ ...RECORD_TYPES_BODY, kind: 'async_tasks', jobId: 'job-1' }), + {} + ) + + expect(response.status).toBe(200) + expect(await json(response)).toEqual({ + objects: [{ id: 'task-1', label: 'task-1', detail: null }], + }) + }) + + it('fails discovery when a task entry has no self link', async () => { + mockGetAsyncStatus.mockResolvedValueOnce( + success({ + items: [ + { links: [{ rel: 'canonical', href: `${ORIGIN}/services/rest/async/v1/job/job-1` }] }, + ], + }) + ) + + const response = await POST( + request({ ...RECORD_TYPES_BODY, kind: 'async_tasks', jobId: 'job-1' }), + {} + ) + + expect(response.status).toBe(502) + }) + + it('normalizes, deduplicates, and sorts up to 1,000 unique record types', async () => { + const items = Array.from({ length: 1_000 }, (_, index) => ({ + name: `record_${String(999 - index).padStart(4, '0')}`, + })) + items.push({ name: 'record_0000' }, { name: 'record_0999' }) + mockListRecordTypes.mockResolvedValueOnce(success({ items })) + + const response = await POST(request(RECORD_TYPES_BODY), {}) + const body = await json(response) + + expect(response.status).toBe(200) + expect(body.objects).toHaveLength(1_000) + expect((body.objects as { id: string }[]).at(0)?.id).toBe('record_0000') + expect((body.objects as { id: string }[]).at(-1)?.id).toBe('record_0999') + }) + + it('fails closed instead of returning a partial record-type catalog', async () => { + mockListRecordTypes.mockResolvedValueOnce( + success({ + items: Array.from({ length: 1_001 }, (_, index) => ({ name: `record_${index}` })), + }) + ) + + const response = await POST(request(RECORD_TYPES_BODY), {}) + + expect(response.status).toBe(502) + expect(await json(response)).toEqual({ + error: 'NetSuite returned an invalid object-discovery response.', + }) + }) + + it('fails closed on a malformed provider envelope', async () => { + mockListRecordTypes.mockResolvedValueOnce(success({ items: [{ name: 42 }] })) + + const response = await POST(request(RECORD_TYPES_BODY), {}) + + expect(response.status).toBe(502) + expect(await json(response)).toEqual({ + error: 'NetSuite returned an invalid object-discovery response.', + }) + }) + + it.each([ + ['foreign origin', 'https://evil.example/services/rest/async/v1/job/job-1/task/task-1'], + ['wrong job', `${ORIGIN}/services/rest/async/v1/job/job-2/task/task-1`], + ['query string', `${ORIGIN}/services/rest/async/v1/job/job-1/task/task-1?secret=x`], + ['fragment', `${ORIGIN}/services/rest/async/v1/job/job-1/task/task-1#fragment`], + ['noncanonical encoding', `${ORIGIN}/services/rest/async/v1/job/job%2D1/task/task-1`], + ])('rejects a %s task link', async (_label, href) => { + mockGetAsyncStatus.mockResolvedValueOnce( + success({ items: [{ links: [{ rel: 'self', href }] }] }) + ) + + const response = await POST( + request({ ...RECORD_TYPES_BODY, kind: 'async_tasks', jobId: 'job-1' }), + {} + ) + + expect(response.status).toBe(502) + expect(await json(response)).toEqual({ + error: 'NetSuite returned an invalid object-discovery response.', + }) + }) + + it('maps unexpected discovery failures to the generic route error', async () => { + mockListRecordTypes.mockRejectedValueOnce(new Error('secret provider detail')) + + const response = await POST(request(RECORD_TYPES_BODY), {}) + const body = await json(response) + + expect(response.status).toBe(500) + expect(body.error).toBe('Internal server error') + expect(JSON.stringify(body)).not.toContain('secret provider detail') + }) + + it('rejects malformed task relationships and collections above the provider ceiling', async () => { + const taskItems = Array.from({ length: 101 }, (_, index) => ({ + links: [ + { + rel: 'self', + href: `/services/rest/async/v1/job/job-1/task/task-${index}`, + }, + ], + })) + for (const items of [ + [{ links: [{ rel: 'alternate', href: taskItems[0].links[0].href }] }], + taskItems, + [{ links: 'not-an-array' }], + ]) { + mockGetAsyncStatus.mockResolvedValueOnce(success({ items })) + const response = await POST( + request({ ...RECORD_TYPES_BODY, kind: 'async_tasks', jobId: 'job-1' }), + {} + ) + expect(response.status).toBe(502) + } + }) + + it('applies the async-task ceiling after duplicate task links are removed', async () => { + const href = '/services/rest/async/v1/job/job-1/task/task-1' + mockGetAsyncStatus.mockResolvedValueOnce( + success({ + items: Array.from({ length: 101 }, () => ({ links: [{ rel: 'self', href }] })), + }) + ) + + const response = await POST( + request({ ...RECORD_TYPES_BODY, kind: 'async_tasks', jobId: 'job-1' }), + {} + ) + + expect(response.status).toBe(200) + expect(await json(response)).toEqual({ + objects: [{ id: 'task-1', label: 'task-1', detail: null }], + }) + }) + + it.each([ + [401, 401, true], + [403, 403, undefined], + [404, 400, undefined], + [500, 502, undefined], + [undefined, 502, undefined], + ])( + 'maps provider status %s without reflecting its error', + async (providerStatus, status, authRequired) => { + mockListRecordTypes.mockResolvedValueOnce(failure(providerStatus)) + + const response = await POST(request(RECORD_TYPES_BODY), {}) + const body = await json(response) + + expect(response.status).toBe(status) + expect(body.authRequired).toBe(authRequired) + expect(JSON.stringify(body)).not.toContain('provider secret') + } + ) + + it.each([ + [new TokenServiceAccountValidationError('invalid_credentials', 401), 401, true], + [new TokenServiceAccountValidationError('provider_unavailable', 502), 502, undefined], + [null, 401, true], + ])( + 'maps credential resolution failures without exposing details', + async (error, status, authRequired) => { + if (error === null) { + mockResolveCredentialAccessToken.mockResolvedValueOnce(null) + } else { + mockResolveCredentialAccessToken.mockRejectedValueOnce(error) + } + + const response = await POST(request(RECORD_TYPES_BODY), {}) + const body = await json(response) + + expect(response.status).toBe(status) + expect(body.authRequired).toBe(authRequired) + } + ) + + it.each([ + ['missing access token', { instanceUrl: ORIGIN }], + ['missing instance URL', { accessToken: 'short-lived-token' }], + ])('rejects a resolved credential with %s', async (_label, token) => { + mockResolveCredentialAccessToken.mockResolvedValueOnce(token) + + const response = await POST(request(RECORD_TYPES_BODY), {}) + + expect(response.status).toBe(401) + expect(await json(response)).toMatchObject({ authRequired: true }) + expect(mockListRecordTypes).not.toHaveBeenCalled() + }) + + it('returns 499 when the caller cancels after provider dispatch', async () => { + const controller = new AbortController() + mockListRecordTypes.mockImplementationOnce(async () => { + controller.abort() + return success({ items: [{ name: 'customer' }] }) + }) + + const response = await POST(request(RECORD_TYPES_BODY, controller.signal), {}) + + expect(response.status).toBe(499) + }) +}) diff --git a/apps/sim/app/api/tools/netsuite/objects/route.ts b/apps/sim/app/api/tools/netsuite/objects/route.ts new file mode 100644 index 00000000000..b6183e13a28 --- /dev/null +++ b/apps/sim/app/api/tools/netsuite/objects/route.ts @@ -0,0 +1,355 @@ +import { createLogger } from '@sim/logger' +import { isPlainRecord } from '@sim/utils/object' +import { type NextRequest, NextResponse } from 'next/server' +import { + type NetSuiteObjectsSelectorBody, + netsuiteObjectsSelectorContract, +} from '@/lib/api/contracts/selectors/netsuite' +import { getValidationErrorMessage, parseRequest } from '@/lib/api/server' +import { authorizeCredentialUse } from '@/lib/auth/credential-access' +import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' +import { generateRequestId } from '@/lib/core/utils/request' +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { NETSUITE_SERVICE_ACCOUNT_PROVIDER_ID } from '@/lib/credentials/client-credential-accounts/descriptors' +import { TokenServiceAccountValidationError } from '@/lib/credentials/token-service-accounts/errors' +import { resolveCredentialAccessToken, resolveOAuthAccountId } from '@/lib/oauth/credential-service' +import { netsuiteGetAsyncStatusTool } from '@/tools/netsuite/get_async_status' +import { netsuiteListRecordTypesTool } from '@/tools/netsuite/list_record_types' +import type { NetSuiteAuthParams } from '@/tools/netsuite/types' +import { normalizeSuiteTalkUrl } from '@/tools/netsuite/utils' +import type { ToolResponse } from '@/tools/types' + +const logger = createLogger('NetSuiteObjectsAPI') + +export const dynamic = 'force-dynamic' + +/** + * This session/internal-only metadata route intentionally has no separate rate + * limiter: it reuses read-only NetSuite tools whose deadlines and bounded + * result sets constrain each provider call, matching Snowflake's picker route. + */ +const SELECTOR_REQUEST_MAX_BYTES = 16 * 1024 +const MAX_RECORD_TYPES = 1_000 +const MAX_ASYNC_TASKS = 100 +const MAX_ID_LENGTH = 512 + +interface NetSuiteSelectorObject { + id: string + label: string + detail: string | null +} + +function throwIfAborted(signal: AbortSignal): void { + if (!signal.aborted) return + throw signal.reason instanceof Error + ? signal.reason + : new DOMException('NetSuite selector request was cancelled', 'AbortError') +} + +function requireString(value: unknown, label: string, maxLength: number): string { + if (typeof value !== 'string' || !value.trim()) { + throw new Error(`NetSuite returned an invalid ${label}`) + } + const normalized = value.trim() + if (normalized.length > maxLength) { + throw new Error(`NetSuite returned an oversized ${label}`) + } + return normalized +} + +function requireItems(data: unknown, label: string): Record[] { + if (!isPlainRecord(data) || !Array.isArray(data.items)) { + throw new Error(`NetSuite returned an invalid ${label} response`) + } + if (!data.items.every(isPlainRecord)) { + throw new Error(`NetSuite returned malformed ${label} entries`) + } + return data.items +} + +function dedupeAndSort(objects: NetSuiteSelectorObject[]): NetSuiteSelectorObject[] { + const unique = new Map() + for (const object of objects) { + if (!unique.has(object.id)) unique.set(object.id, object) + } + return [...unique.values()].sort( + (left, right) => left.label.localeCompare(right.label) || left.id.localeCompare(right.id) + ) +} + +function normalizeRecordTypes(data: unknown): NetSuiteSelectorObject[] { + const objects: NetSuiteSelectorObject[] = [] + const names = new Set() + for (const item of requireItems(data, 'record-type catalog')) { + const name = requireString(item.name, 'record type name', MAX_ID_LENGTH) + if (!names.has(name)) { + if (names.size >= MAX_RECORD_TYPES) { + throw new Error('NetSuite returned too many record types') + } + names.add(name) + objects.push({ id: name, label: name, detail: null }) + } + } + return dedupeAndSort(objects) +} + +function taskIdFromHref(href: unknown, origin: string, jobId: string): string { + const hrefValue = requireString(href, 'asynchronous task link', 4_096) + let url: URL + try { + url = new URL(hrefValue, origin) + } catch { + throw new Error('NetSuite returned a malformed task link') + } + if ( + url.protocol !== 'https:' || + url.origin !== origin || + url.username || + url.password || + url.search || + url.hash + ) { + throw new Error('NetSuite returned an unsafe task link') + } + + const match = url.pathname.match(/^\/services\/rest\/async\/v1\/job\/([^/]+)\/task\/([^/]+)$/) + if (!match?.[1] || !match[2]) { + throw new Error('NetSuite returned an unexpected task link') + } + + let linkedJobId: string + let taskId: string + try { + linkedJobId = decodeURIComponent(match[1]) + taskId = decodeURIComponent(match[2]) + } catch { + throw new Error('NetSuite returned a malformed task link') + } + if (linkedJobId !== jobId || !taskId || taskId.length > MAX_ID_LENGTH) { + throw new Error('NetSuite returned a task link outside the requested job') + } + + const canonicalPath = `/services/rest/async/v1/job/${encodeURIComponent(linkedJobId)}/task/${encodeURIComponent(taskId)}` + if ( + url.pathname !== canonicalPath || + (hrefValue !== canonicalPath && hrefValue !== `${origin}${canonicalPath}`) + ) { + throw new Error('NetSuite returned a noncanonical task link') + } + return taskId +} + +function normalizeAsyncTasks( + data: unknown, + instanceUrl: string, + jobId: string +): NetSuiteSelectorObject[] { + const items = requireItems(data, 'asynchronous task collection') + const origin = normalizeSuiteTalkUrl(instanceUrl) + const objects = new Map() + + for (const item of items) { + if (!Array.isArray(item.links) || item.links.length === 0 || !item.links.every(isPlainRecord)) { + throw new Error('NetSuite returned malformed asynchronous task links') + } + // Oracle documents a `self` link per task but never guarantees it is the + // only relationship on the entry, so additional rels are skipped rather + // than failing the whole picker. + const selfLinks = item.links.filter((link) => link.rel === 'self') + if (selfLinks.length === 0) { + throw new Error('NetSuite returned an asynchronous task without a self link') + } + for (const link of selfLinks) { + const id = taskIdFromHref(link.href, origin, jobId) + if (!objects.has(id)) { + if (objects.size >= MAX_ASYNC_TASKS) { + throw new Error('NetSuite returned too many asynchronous tasks') + } + objects.set(id, { id, label: id, detail: null }) + } + } + } + return dedupeAndSort([...objects.values()]) +} + +async function executeDiscoveryTool( + body: NetSuiteObjectsSelectorBody, + auth: NetSuiteAuthParams, + signal: AbortSignal +): Promise { + throwIfAborted(signal) + switch (body.kind) { + case 'record_types': { + const execute = netsuiteListRecordTypesTool.directExecution + if (!execute) throw new Error('NetSuite record-type tool is not executable') + return execute(auth, signal) + } + case 'async_tasks': { + const execute = netsuiteGetAsyncStatusTool.directExecution + if (!execute) throw new Error('NetSuite asynchronous-status tool is not executable') + return execute({ ...auth, jobId: body.jobId, view: 'tasks' }, signal) + } + } +} + +function failedDiscoveryResponse(result: ToolResponse): NextResponse { + const providerStatus = + typeof result.output.status === 'number' && Number.isInteger(result.output.status) + ? result.output.status + : 0 + if (providerStatus === 401) { + return NextResponse.json( + { + error: 'NetSuite rejected this credential. Reconnect it and try again.', + authRequired: true, + }, + { status: 401 } + ) + } + if (providerStatus === 403) { + return NextResponse.json( + { error: 'NetSuite denied access to object discovery for this credential.' }, + { status: 403 } + ) + } + if (providerStatus >= 400 && providerStatus < 500) { + return NextResponse.json( + { error: 'NetSuite could not list objects for this request.' }, + { status: 400 } + ) + } + return NextResponse.json({ error: 'NetSuite object discovery failed.' }, { status: 502 }) +} + +function credentialFailureResponse(error: unknown): NextResponse { + if (error instanceof TokenServiceAccountValidationError) { + if (error.code !== 'provider_unavailable') { + return NextResponse.json( + { + error: 'Could not resolve the NetSuite credential. Reconnect it and try again.', + authRequired: true, + }, + { status: 401 } + ) + } + return NextResponse.json( + { error: 'The NetSuite credential service is temporarily unavailable.' }, + { status: 502 } + ) + } + return NextResponse.json( + { + error: 'Could not resolve the NetSuite credential. Reconnect it and try again.', + authRequired: true, + }, + { status: 401 } + ) +} + +/** + * Lists the bounded NetSuite objects used by the block's record-type and + * asynchronous-task pickers. Like Snowflake's selector endpoint, this + * route owns authentication, credential resolution, provider access, and + * response normalization directly; short-lived tokens never reach the client. + */ +export const POST = withRouteHandler(async (request: NextRequest) => { + const requestId = generateRequestId() + + const caller = await checkSessionOrInternalAuth(request, { requireWorkflowId: true }) + if (!caller.success || !caller.userId) { + return NextResponse.json({ error: caller.error || 'Authentication required' }, { status: 401 }) + } + + const parsed = await parseRequest( + netsuiteObjectsSelectorContract, + request, + {}, + { + maxBodyBytes: SELECTOR_REQUEST_MAX_BYTES, + validationErrorResponse: (error) => + NextResponse.json( + { error: getValidationErrorMessage(error, 'Invalid request') }, + { status: 400 } + ), + } + ) + if (!parsed.success) return parsed.response + const body = parsed.data.body + const { credential, workflowId, kind } = body + + const authorization = await authorizeCredentialUse(request, { + credentialId: credential, + workflowId, + callerUserId: caller.userId, + }) + if (!authorization.ok || !authorization.credentialOwnerUserId) { + return NextResponse.json({ error: authorization.error || 'Unauthorized' }, { status: 403 }) + } + + const resolvedCredentialId = authorization.resolvedCredentialId ?? credential + const resolvedCredential = await resolveOAuthAccountId(resolvedCredentialId) + if ( + authorization.credentialType !== 'service_account' || + resolvedCredential?.credentialType !== 'service_account' || + resolvedCredential.providerId !== NETSUITE_SERVICE_ACCOUNT_PROVIDER_ID + ) { + return NextResponse.json( + { error: 'Select a NetSuite client-credentials service account.' }, + { status: 400 } + ) + } + + throwIfAborted(request.signal) + let token + try { + token = await resolveCredentialAccessToken( + resolvedCredentialId, + authorization.credentialOwnerUserId, + requestId + ) + } catch (error) { + throwIfAborted(request.signal) + logger.warn('Failed to resolve NetSuite selector credential', { + credentialId: resolvedCredentialId, + kind, + errorType: error instanceof Error ? error.name : 'unknown', + }) + if (error instanceof TokenServiceAccountValidationError) { + return credentialFailureResponse(error) + } + throw error + } + throwIfAborted(request.signal) + if (!token?.accessToken || !token.instanceUrl) { + return credentialFailureResponse(null) + } + + const auth: NetSuiteAuthParams = { + oauthCredential: resolvedCredentialId, + accessToken: token.accessToken, + instanceUrl: token.instanceUrl, + } + + const result: ToolResponse = await executeDiscoveryTool(body, auth, request.signal) + throwIfAborted(request.signal) + if (!result.success) return failedDiscoveryResponse(result) + + try { + const data = result.output.data as unknown + const objects = + body.kind === 'record_types' + ? normalizeRecordTypes(data) + : normalizeAsyncTasks(data, token.instanceUrl, body.jobId) + return NextResponse.json({ objects }) + } catch (error) { + logger.error('NetSuite selector response was invalid', { + credentialId: resolvedCredentialId, + kind, + errorType: error instanceof Error ? error.name : 'unknown', + }) + return NextResponse.json( + { error: 'NetSuite returned an invalid object-discovery response.' }, + { status: 502 } + ) + } +}) diff --git a/apps/sim/app/api/tools/windchill/route.test.ts b/apps/sim/app/api/tools/windchill/route.test.ts new file mode 100644 index 00000000000..7504e7e606c --- /dev/null +++ b/apps/sim/app/api/tools/windchill/route.test.ts @@ -0,0 +1,836 @@ +/** + * @vitest-environment node + */ +import { createMockRequest as createTestingRequest, resetEnvMock } from '@sim/testing' +import { NextResponse } from 'next/server' +import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest' +import { PayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' + +const { + MockInvalidBindingError, + MockWindchillProviderError, + mockAssertToolFileAccess, + mockBindDelegation, + mockCreateWindchillSession, + mockDownloadServableFileFromStorage, + mockDownloadWindchillContent, + mockGetSession, + mockResolveWindchillContentUrl, + mockProcessFilesToUserFiles, + mockUploadCopilotFile, + mockUploadExecutionFile, + mockUploadWindchillContent, + mockWindchillMutationRequest, +} = vi.hoisted(() => { + class MockInvalidBindingError extends Error {} + class MockWindchillProviderError extends Error { + constructor( + message: string, + readonly status: number + ) { + super(message) + this.name = 'WindchillProviderError' + } + } + + return { + MockInvalidBindingError, + MockWindchillProviderError, + mockAssertToolFileAccess: vi.fn(), + mockBindDelegation: vi.fn(), + mockCreateWindchillSession: vi.fn(), + mockDownloadServableFileFromStorage: vi.fn(), + mockDownloadWindchillContent: vi.fn(), + mockGetSession: vi.fn(), + mockResolveWindchillContentUrl: vi.fn(), + mockProcessFilesToUserFiles: vi.fn(), + mockUploadCopilotFile: vi.fn(), + mockUploadExecutionFile: vi.fn(), + mockUploadWindchillContent: vi.fn(), + mockWindchillMutationRequest: vi.fn(), + } +}) + +vi.mock('@/lib/auth', () => ({ getSession: mockGetSession })) +vi.mock('@/lib/auth/internal-delegation', () => ({ + bindInternalExecutorDelegation: mockBindDelegation, + InvalidInternalDelegationBindingError: MockInvalidBindingError, +})) +vi.unmock('@/lib/auth/internal') + +vi.mock('@/app/api/files/authorization', () => ({ + assertToolFileAccess: mockAssertToolFileAccess, +})) +vi.mock('@/lib/uploads/utils/file-utils', () => ({ + processFilesToUserFiles: mockProcessFilesToUserFiles, +})) +vi.mock('@/lib/uploads/utils/file-utils.server', () => ({ + downloadServableFileFromStorage: mockDownloadServableFileFromStorage, +})) +vi.mock('@/lib/uploads/utils/servable-file-response', () => ({ + docNotReadyResponse: vi.fn().mockReturnValue(null), +})) +vi.mock('@/lib/uploads/contexts/copilot', () => ({ + uploadCopilotFile: mockUploadCopilotFile, +})) +vi.mock('@/lib/uploads/contexts/execution', () => ({ + uploadExecutionFile: mockUploadExecutionFile, +})) +vi.mock('@/tools/windchill/utils.server', () => ({ + createWindchillSession: mockCreateWindchillSession, + downloadWindchillContent: mockDownloadWindchillContent, + resolveWindchillContentUrl: mockResolveWindchillContentUrl, + sanitizeWindchillError: (message: string) => message.replace(/https?:\/\/\S+/g, '[redacted URL]'), + uploadWindchillContent: mockUploadWindchillContent, + windchillDocumentUrl: (baseUrl: string, documentOid: string) => + `${baseUrl}/DocMgmt/Documents('${encodeURIComponent(documentOid)}')`, + windchillMutationRequest: mockWindchillMutationRequest, + WindchillProviderError: MockWindchillProviderError, +})) + +import { generateInternalDelegationToken, generateInternalToken } from '@/lib/auth/internal' +import { POST } from '@/app/api/tools/windchill/route' + +const BASE_BODY = { + baseUrl: 'https://windchill.example.com/Windchill/servlet/odata/v6', + username: 'windchill-user', + password: 'not-a-real-password', +} + +const DOCUMENT_OID = 'OR:wt.doc.WTDocument:1' +const SECOND_DOCUMENT_OID = 'OR:wt.doc.WTDocument:2' +let delegationToken = '' +let legacyInternalToken = '' + +function createMockRequest(method: string, body: unknown, headers: Record = {}) { + return createTestingRequest(method, body, { + authorization: `Bearer ${delegationToken}`, + ...headers, + }) +} + +const MUTATION_CASES = [ + { + operation: 'windchill_create_document', + input: { name: 'Specification', containerOid: 'OR:wt.pdmlink.PDMLinkProduct:1' }, + url: '/DocMgmt/Documents', + method: 'POST', + }, + { + operation: 'windchill_create_documents', + input: { + documents: [{ name: 'Specification', containerOid: 'OR:wt.pdmlink.PDMLinkProduct:1' }], + }, + url: '/DocMgmt/CreateDocuments', + method: 'POST', + }, + { + operation: 'windchill_update_document', + input: { documentOid: DOCUMENT_OID, attributes: { Title: 'Updated' } }, + url: '/DocMgmt/Documents(', + method: 'PATCH', + }, + { + operation: 'windchill_update_documents', + input: { documents: [{ id: DOCUMENT_OID, attributes: { Title: 'Updated' } }] }, + url: '/DocMgmt/UpdateDocuments', + method: 'POST', + }, + { + operation: 'windchill_update_common_properties', + input: { documentOid: DOCUMENT_OID, commonProperties: { Name: 'Renamed' } }, + url: '/PTC.DocMgmt.UpdateCommonProperties', + method: 'POST', + }, + { + operation: 'windchill_delete_document', + input: { documentOid: DOCUMENT_OID }, + url: '/DocMgmt/Documents(', + method: 'DELETE', + }, + { + operation: 'windchill_delete_documents', + input: { documentOids: [DOCUMENT_OID, SECOND_DOCUMENT_OID] }, + url: '/DocMgmt/DeleteDocuments', + method: 'POST', + }, + { + operation: 'windchill_check_out_document', + input: { documentOid: DOCUMENT_OID, checkOutNote: 'Editing' }, + url: '/PTC.DocMgmt.CheckOut', + method: 'POST', + }, + { + operation: 'windchill_check_out_documents', + input: { documentOids: [DOCUMENT_OID], checkOutNote: 'Editing' }, + url: '/DocMgmt/CheckOutDocuments', + method: 'POST', + }, + { + operation: 'windchill_check_in_document', + input: { documentOid: DOCUMENT_OID, checkInNote: 'Done', keepCheckedOut: false }, + url: '/PTC.DocMgmt.CheckIn', + method: 'POST', + }, + { + operation: 'windchill_check_in_documents', + input: { documentOids: [DOCUMENT_OID], checkInNote: 'Done' }, + url: '/DocMgmt/CheckInDocuments', + method: 'POST', + }, + { + operation: 'windchill_undo_check_out_document', + input: { documentOid: DOCUMENT_OID }, + url: '/PTC.DocMgmt.UndoCheckOut', + method: 'POST', + }, + { + operation: 'windchill_undo_check_out_documents', + input: { documentOids: [DOCUMENT_OID] }, + url: '/DocMgmt/UndoCheckOutDocuments', + method: 'POST', + }, + { + operation: 'windchill_revise_document', + input: { documentOid: DOCUMENT_OID, versionId: 'B' }, + url: '/PTC.DocMgmt.Revise', + method: 'POST', + }, + { + operation: 'windchill_revise_documents', + input: { documentOids: [DOCUMENT_OID] }, + url: '/DocMgmt/ReviseDocuments', + method: 'POST', + }, + { + operation: 'windchill_set_lifecycle_state', + input: { documentOid: DOCUMENT_OID, stateValue: 'RELEASED', stateDisplay: 'Released' }, + url: '/PTC.DocMgmt.SetState', + method: 'POST', + }, + { + operation: 'windchill_update_document_security_labels', + input: { + securityLabelUpdates: [{ id: DOCUMENT_OID, labels: { EXPORT_CONTROL: 'L1' } }], + }, + url: '/DocMgmt/EditDocumentsSecurityLabels', + method: 'POST', + }, +] as const + +const MUTATION_PAYLOAD_CASES = [ + { + operation: 'windchill_check_out_documents', + input: { documentOids: [DOCUMENT_OID], checkOutNote: 'Editing' }, + body: { Documents: [{ ID: DOCUMENT_OID }], CheckOutNote: 'Editing' }, + }, + { + operation: 'windchill_check_in_document', + input: { + documentOid: DOCUMENT_OID, + checkInNote: 'Done', + keepCheckedOut: false, + checkOutNote: 'Continue editing', + }, + body: { + CheckInNote: 'Done', + KeepCheckedOut: false, + CheckOutNote: 'Continue editing', + }, + }, + { + operation: 'windchill_revise_document', + input: { documentOid: DOCUMENT_OID, versionId: 'B' }, + body: { VersionId: 'B' }, + }, + { + operation: 'windchill_update_common_properties', + input: { + documentOid: DOCUMENT_OID, + commonProperties: { Name: 'Renamed', Number: 'DOC-001' }, + }, + body: { Updates: { Name: 'Renamed', Number: 'DOC-001' } }, + }, + { + operation: 'windchill_revise_documents', + input: { documentOids: [DOCUMENT_OID] }, + body: { Documents: [{ ID: DOCUMENT_OID }] }, + }, + { + operation: 'windchill_set_lifecycle_state', + input: { documentOid: DOCUMENT_OID, stateValue: 'RELEASED', stateDisplay: 'Released' }, + body: { State: { Display: 'Released', Value: 'RELEASED' } }, + }, + { + operation: 'windchill_update_document_security_labels', + input: { + securityLabelUpdates: [{ id: DOCUMENT_OID, labels: { EXPORT_CONTROL: 'L1' } }], + }, + body: { Documents: [{ EXPORT_CONTROL: 'L1', ID: DOCUMENT_OID }] }, + }, +] as const + +beforeAll(async () => { + delegationToken = await generateInternalDelegationToken({ + subjectUserId: 'user-1', + workflowId: '550e8400-e29b-41d4-a716-446655440001', + }) + legacyInternalToken = await generateInternalToken() +}) + +afterAll(resetEnvMock) + +beforeEach(() => { + vi.clearAllMocks() + mockGetSession.mockResolvedValue(null) + mockBindDelegation.mockImplementation(async (delegation, options) => ({ + kind: 'delegated', + serviceId: 'executor', + subjectUserId: delegation.subjectUserId, + workspaceId: '550e8400-e29b-41d4-a716-446655440000', + delegationId: delegation.delegationId, + audience: options.audience, + issuedAt: delegation.issuedAt, + expiresAt: delegation.expiresAt, + delegationContext: { + kind: 'workflow_execution', + workflowId: delegation.workflowId, + executionId: delegation.executionId, + }, + })) + mockCreateWindchillSession.mockResolvedValue({ + nonceHeader: 'CSRF_NONCE', + nonceValue: 'nonce-value', + cookie: 'JSESSIONID=session-value', + }) + mockWindchillMutationRequest.mockResolvedValue({ value: [{ ID: DOCUMENT_OID }] }) + mockAssertToolFileAccess.mockResolvedValue(null) + mockProcessFilesToUserFiles.mockReturnValue([ + { + key: 'workspace/workspace-1/specification.pdf', + name: 'specification.pdf', + size: 3, + type: 'application/pdf', + }, + ]) + mockDownloadServableFileFromStorage.mockResolvedValue({ + buffer: Buffer.from('pdf'), + contentType: 'application/pdf', + }) + mockUploadWindchillContent.mockResolvedValue(['specification.pdf']) + mockResolveWindchillContentUrl.mockImplementation( + async ({ contentPath }: { contentPath: string }) => + `https://windchill.example.com/Windchill/servlet/WindchillGW/download?from=${encodeURIComponent(contentPath)}` + ) + mockDownloadWindchillContent.mockResolvedValue({ + buffer: Buffer.from('pdf'), + contentType: 'application/pdf', + contentDisposition: 'attachment; filename="specification.pdf"', + }) + mockUploadCopilotFile.mockResolvedValue({ + id: 'file-1', + name: 'specification.pdf', + url: '/api/files/serve?key=copilot/specification.pdf', + size: 3, + type: 'application/pdf', + key: 'copilot/specification.pdf', + }) +}) + +describe('POST /api/tools/windchill', () => { + it('authenticates before parsing the request body', async () => { + const response = await POST(createTestingRequest('POST', { operation: 'not-valid' })) + + expect(response.status).toBe(401) + expect(await response.json()).toEqual({ success: false, error: 'Unauthorized' }) + expect(mockCreateWindchillSession).not.toHaveBeenCalled() + }) + + it('binds executor identity and scope through the canonical delegation path', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_update_document', + documentOid: DOCUMENT_OID, + attributes: { Title: 'Updated' }, + }) + ) + + expect(response.status).toBe(200) + expect(mockBindDelegation).toHaveBeenCalledWith(expect.any(Object), { + audience: 'sim:windchill', + resourceScope: undefined, + }) + }) + + it('rejects browser sessions and legacy internal tokens', async () => { + mockGetSession.mockResolvedValueOnce({ + user: { id: 'user-1' }, + session: { id: 'session-1' }, + }) + + const sessionResponse = await POST(createTestingRequest('POST', BASE_BODY)) + const legacyResponse = await POST( + createTestingRequest('POST', BASE_BODY, { + authorization: `Bearer ${legacyInternalToken}`, + }) + ) + + expect(sessionResponse.status).toBe(401) + expect(legacyResponse.status).toBe(401) + expect(mockBindDelegation).not.toHaveBeenCalled() + }) + + it('rejects malformed operation inputs at the shared contract boundary', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_update_document', + documentOid: DOCUMENT_OID, + attributes: {}, + }) + ) + + expect(response.status).toBe(400) + expect(mockCreateWindchillSession).not.toHaveBeenCalled() + }) + + it('rejects an invalid service root before reading a protected upload', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + baseUrl: `${BASE_BODY.baseUrl}?token=secret`, + operation: 'windchill_upload_primary_content', + documentOid: DOCUMENT_OID, + primaryFile: { + key: 'workspace/workspace-1/specification.pdf', + name: 'specification.pdf', + size: 3, + type: 'application/pdf', + }, + }) + ) + + expect(response.status).toBe(400) + expect(mockAssertToolFileAccess).not.toHaveBeenCalled() + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + }) + + it.each(MUTATION_CASES)( + 'dispatches $operation through one CSRF-protected transaction', + async ({ operation, input, url, method }) => { + const response = await POST(createMockRequest('POST', { ...BASE_BODY, operation, ...input })) + + expect(response.status).toBe(200) + expect((await response.json()).success).toBe(true) + expect(mockCreateWindchillSession).toHaveBeenCalledTimes(1) + expect(mockWindchillMutationRequest).toHaveBeenCalledTimes(1) + expect(mockWindchillMutationRequest.mock.calls[0][0].url).toContain(url) + expect(mockWindchillMutationRequest.mock.calls[0][0].method).toBe(method) + } + ) + + it.each(MUTATION_PAYLOAD_CASES)( + 'encodes the exact $operation action payload', + async ({ operation, input, body }) => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation, + ...input, + }) + ) + + expect(response.status).toBe(200) + expect(mockWindchillMutationRequest.mock.calls[0][0].body).toEqual(body) + } + ) + + it('maps create bindings and custom attributes without allowing them to replace bindings', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_create_document', + name: 'Specification', + containerOid: 'OR:wt.pdmlink.PDMLinkProduct:1', + folderOid: 'OR:wt.folder.SubFolder:2', + attributes: { CustomString: 'value' }, + }) + ) + + expect(response.status).toBe(200) + expect((await response.json()).output.affectedIds).toEqual([DOCUMENT_OID]) + expect(mockWindchillMutationRequest.mock.calls[0][0].body).toEqual({ + CustomString: 'value', + Name: 'Specification', + 'Context@odata.bind': "Containers('OR%3Awt.pdmlink.PDMLinkProduct%3A1')", + 'Folder@odata.bind': "Folders('OR%3Awt.folder.SubFolder%3A2')", + }) + }) + + it('returns operation-specific single, bulk, and delete mutation shapes', async () => { + const singleResponse = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_update_document', + documentOid: DOCUMENT_OID, + attributes: { Title: 'Updated' }, + }) + ) + const singleOutput = (await singleResponse.json()).output + expect(singleOutput.document).toMatchObject({ id: DOCUMENT_OID }) + expect(singleOutput).not.toHaveProperty('documents') + + const bulkResponse = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_update_documents', + documents: [{ id: DOCUMENT_OID, attributes: { Title: 'Updated' } }], + }) + ) + const bulkOutput = (await bulkResponse.json()).output + expect(bulkOutput.documents).toEqual([expect.objectContaining({ id: DOCUMENT_OID })]) + expect(bulkOutput).not.toHaveProperty('document') + + const deleteResponse = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_delete_document', + documentOid: DOCUMENT_OID, + }) + ) + const deleteOutput = (await deleteResponse.json()).output + expect(deleteOutput.affectedIds).toEqual([DOCUMENT_OID]) + expect(deleteOutput).not.toHaveProperty('document') + expect(deleteOutput).not.toHaveProperty('documents') + }) + + it('authorizes and reads a UserFile before starting the upload transaction', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_primary_content', + documentOid: DOCUMENT_OID, + primaryFile: { + key: 'workspace/workspace-1/specification.pdf', + name: 'specification.pdf', + size: 3, + type: 'application/pdf', + }, + }) + ) + + expect(response.status).toBe(200) + expect(mockAssertToolFileAccess).toHaveBeenCalledWith( + 'workspace/workspace-1/specification.pdf', + 'user-1', + expect.any(String), + expect.anything() + ) + expect(mockDownloadServableFileFromStorage).toHaveBeenCalledTimes(1) + expect(mockUploadWindchillContent).toHaveBeenCalledWith( + expect.objectContaining({ + documentOid: DOCUMENT_OID, + primaryContent: true, + files: [ + expect.objectContaining({ + name: 'specification.pdf', + mimeType: 'application/pdf', + size: 3, + }), + ], + }) + ) + }) + + it('uploads multiple authorized files as attachments', async () => { + mockProcessFilesToUserFiles.mockReturnValueOnce([ + { + key: 'workspace/workspace-1/one.txt', + name: 'one.txt', + size: 3, + type: 'text/plain', + }, + { + key: 'workspace/workspace-1/two.txt', + name: 'two.txt', + size: 3, + type: 'text/plain', + }, + ]) + mockDownloadServableFileFromStorage.mockResolvedValue({ + buffer: Buffer.from('txt'), + contentType: 'text/plain', + }) + mockUploadWindchillContent.mockResolvedValueOnce(['one.txt', 'two.txt']) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_attachments', + documentOid: DOCUMENT_OID, + attachmentFiles: [ + { key: 'workspace/workspace-1/one.txt', name: 'one.txt', size: 3 }, + { key: 'workspace/workspace-1/two.txt', name: 'two.txt', size: 3 }, + ], + }) + ) + + expect(response.status).toBe(200) + expect(mockAssertToolFileAccess).toHaveBeenCalledTimes(2) + expect(mockDownloadServableFileFromStorage).toHaveBeenCalledTimes(2) + expect(mockUploadWindchillContent).toHaveBeenCalledWith( + expect.objectContaining({ primaryContent: false }) + ) + expect(mockDownloadServableFileFromStorage.mock.calls[0][3]).toEqual({ + maxBytes: MAX_FILE_SIZE, + }) + expect(mockDownloadServableFileFromStorage.mock.calls[1][3]).toEqual({ + maxBytes: MAX_FILE_SIZE - 3, + }) + }) + + it('rejects attachment counts above the contract limit before reading storage', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_attachments', + documentOid: DOCUMENT_OID, + attachmentFiles: Array.from({ length: 11 }, (_, index) => ({ + key: `workspace/workspace-1/${index}.txt`, + name: `${index}.txt`, + size: 1, + })), + }) + ) + + expect(response.status).toBe(400) + expect(mockAssertToolFileAccess).not.toHaveBeenCalled() + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + }) + + it('rejects declared aggregate upload size before reading storage', async () => { + mockProcessFilesToUserFiles.mockReturnValueOnce([ + { + key: 'workspace/workspace-1/oversized.bin', + name: 'oversized.bin', + size: MAX_FILE_SIZE + 1, + type: 'application/octet-stream', + }, + ]) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_primary_content', + documentOid: DOCUMENT_OID, + primaryFile: { + key: 'workspace/workspace-1/oversized.bin', + name: 'oversized.bin', + size: MAX_FILE_SIZE + 1, + }, + }) + ) + + expect(response.status).toBe(413) + expect(mockAssertToolFileAccess).not.toHaveBeenCalled() + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + }) + + it('stops an under-reported upload at the remaining aggregate byte budget', async () => { + mockProcessFilesToUserFiles.mockReturnValueOnce([ + { key: 'workspace/workspace-1/one.txt', name: 'one.txt', size: 1, type: 'text/plain' }, + { key: 'workspace/workspace-1/two.txt', name: 'two.txt', size: 1, type: 'text/plain' }, + { + key: 'workspace/workspace-1/three.txt', + name: 'three.txt', + size: 1, + type: 'text/plain', + }, + ]) + mockDownloadServableFileFromStorage + .mockResolvedValueOnce({ buffer: Buffer.from('one'), contentType: 'text/plain' }) + .mockRejectedValueOnce( + new PayloadSizeLimitError({ + label: 'Uploaded file', + maxBytes: MAX_FILE_SIZE - 3, + observedBytes: MAX_FILE_SIZE - 2, + }) + ) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_attachments', + documentOid: DOCUMENT_OID, + attachmentFiles: [ + { key: 'workspace/workspace-1/one.txt', name: 'one.txt', size: 1 }, + { key: 'workspace/workspace-1/two.txt', name: 'two.txt', size: 1 }, + { key: 'workspace/workspace-1/three.txt', name: 'three.txt', size: 1 }, + ], + }) + ) + + expect(response.status).toBe(413) + expect(mockDownloadServableFileFromStorage).toHaveBeenCalledTimes(2) + expect(mockDownloadServableFileFromStorage.mock.calls[1][3]).toEqual({ + maxBytes: MAX_FILE_SIZE - 3, + }) + expect(mockUploadWindchillContent).not.toHaveBeenCalled() + }) + + it('stops before storage or Windchill when file ownership is denied', async () => { + mockAssertToolFileAccess.mockResolvedValueOnce( + NextResponse.json({ success: false, error: 'File not found' }, { status: 404 }) + ) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_upload_primary_content', + documentOid: DOCUMENT_OID, + primaryFile: { + key: 'workspace/other/specification.pdf', + name: 'specification.pdf', + size: 3, + type: 'application/pdf', + }, + }) + ) + + expect(response.status).toBe(404) + expect(mockDownloadServableFileFromStorage).not.toHaveBeenCalled() + expect(mockUploadWindchillContent).not.toHaveBeenCalled() + }) + + it('stores downloads as a UserFile instead of returning inline bytes', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_download_primary_content', + documentOid: DOCUMENT_OID, + }) + ) + const data = await response.json() + + expect(response.status).toBe(200) + expect(mockResolveWindchillContentUrl).toHaveBeenCalledWith( + expect.objectContaining({ + contentPath: expect.stringContaining('/PrimaryContent'), + }) + ) + expect(mockResolveWindchillContentUrl.mock.calls[0][0].contentPath).not.toContain('$value') + expect(mockDownloadWindchillContent).toHaveBeenCalledWith( + expect.objectContaining({ + url: expect.stringContaining('/WindchillGW/download'), + }) + ) + expect(mockUploadCopilotFile).toHaveBeenCalledWith( + expect.objectContaining({ + buffer: Buffer.from('pdf'), + fileName: 'specification.pdf', + contentType: 'application/pdf', + userId: 'user-1', + }) + ) + expect(data.output.file).toMatchObject({ key: 'copilot/specification.pdf' }) + expect(data.output.content).toBeUndefined() + }) + + it('downloads an attachment through its document-scoped content path', async () => { + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_download_attachment', + documentOid: DOCUMENT_OID, + attachmentOid: 'OR:wt.content.ApplicationData:2', + }) + ) + + expect(response.status).toBe(200) + expect(mockResolveWindchillContentUrl).toHaveBeenCalledWith( + expect.objectContaining({ + contentPath: expect.stringContaining("/Attachments('OR%3Awt.content.ApplicationData%3A2')"), + }) + ) + expect(mockDownloadWindchillContent).toHaveBeenCalledWith( + expect.objectContaining({ + url: expect.stringContaining('/WindchillGW/download'), + }) + ) + }) + + it('uses execution storage derived from the bound delegation principal', async () => { + mockBindDelegation.mockResolvedValueOnce({ + kind: 'delegated', + serviceId: 'executor', + subjectUserId: 'user-1', + workspaceId: '550e8400-e29b-41d4-a716-446655440000', + delegationId: 'delegation-1', + audience: 'sim:windchill', + issuedAt: new Date('2026-01-01T00:00:00.000Z'), + expiresAt: new Date('2027-01-01T00:00:00.000Z'), + delegationContext: { + kind: 'workflow_execution', + workflowId: '550e8400-e29b-41d4-a716-446655440001', + executionId: 'execution-1', + }, + }) + mockUploadExecutionFile.mockResolvedValueOnce({ + id: 'file-2', + name: 'specification.pdf', + url: '/api/files/serve?key=execution/specification.pdf', + size: 3, + type: 'application/pdf', + key: 'execution/specification.pdf', + }) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_download_primary_content', + documentOid: DOCUMENT_OID, + workspaceId: 'forged-workspace', + workflowId: 'forged-workflow', + executionId: 'forged-execution', + }) + ) + + expect(response.status).toBe(200) + expect(mockUploadExecutionFile).toHaveBeenCalledWith( + { + workspaceId: '550e8400-e29b-41d4-a716-446655440000', + workflowId: '550e8400-e29b-41d4-a716-446655440001', + executionId: 'execution-1', + }, + Buffer.from('pdf'), + 'specification.pdf', + 'application/pdf', + 'user-1' + ) + expect(mockUploadCopilotFile).not.toHaveBeenCalled() + }) + + it('preserves sanitized provider status codes', async () => { + mockWindchillMutationRequest.mockRejectedValueOnce( + new MockWindchillProviderError('Windchill rejected the transition', 409) + ) + + const response = await POST( + createMockRequest('POST', { + ...BASE_BODY, + operation: 'windchill_set_lifecycle_state', + documentOid: DOCUMENT_OID, + stateValue: 'RELEASED', + stateDisplay: 'Released', + }) + ) + + expect(response.status).toBe(409) + expect(await response.json()).toEqual({ + success: false, + error: 'Windchill rejected the transition', + }) + }) +}) diff --git a/apps/sim/app/api/tools/windchill/route.ts b/apps/sim/app/api/tools/windchill/route.ts new file mode 100644 index 00000000000..21eaee259ff --- /dev/null +++ b/apps/sim/app/api/tools/windchill/route.ts @@ -0,0 +1,641 @@ +import type { WorkflowExecutionDelegatedPrincipal } from '@sim/auth/principal' +import { createLogger } from '@sim/logger' +import { getErrorMessage } from '@sim/utils/errors' +import { type NextRequest, NextResponse } from 'next/server' +import type { + WindchillOperationBody, + WindchillOperationResponse, +} from '@/lib/api/contracts/tools/windchill' +import { windchillOperationContract } from '@/lib/api/contracts/tools/windchill' +import { getValidationErrorMessage, parseRequest } from '@/lib/api/server' +import { + createInternalSessionOrExecutorAuth, + InternalUnauthenticatedError, +} from '@/lib/api/server/routes' +import { generateRequestId } from '@/lib/core/utils/request' +import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { uploadCopilotFile } from '@/lib/uploads/contexts/copilot' +import { uploadExecutionFile } from '@/lib/uploads/contexts/execution' +import type { RawFileInput } from '@/lib/uploads/utils/file-schemas' +import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' +import { downloadServableFileFromStorage } from '@/lib/uploads/utils/file-utils.server' +import { docNotReadyResponse } from '@/lib/uploads/utils/servable-file-response' +import { MAX_FILE_SIZE } from '@/lib/uploads/utils/validation' +import { assertToolFileAccess } from '@/app/api/files/authorization' +import { sanitizeFileName } from '@/executor/constants' +import type { UserFile } from '@/executor/types' +import { + encodeWindchillOid, + normalizeServiceRoot, + normalizeWindchillDocument, + normalizeWindchillDocuments, + sanitizeWindchillError, +} from '@/tools/windchill/utils' +import { + createWindchillSession, + downloadWindchillContent, + resolveWindchillContentUrl, + uploadWindchillContent, + WindchillProviderError, + type WindchillUploadFile, + windchillDocumentUrl, + windchillMutationRequest, +} from '@/tools/windchill/utils.server' + +export const dynamic = 'force-dynamic' +export const maxDuration = 900 + +const logger = createLogger('WindchillAPI') +const windchillSessionOrExecutorAuth = createInternalSessionOrExecutorAuth({ + audience: 'sim:windchill', +}) + +async function authenticateWindchillExecutor( + request: NextRequest +): Promise { + const principal = await windchillSessionOrExecutorAuth.authenticate(request, {}) + if ( + principal.kind !== 'delegated' || + principal.serviceId !== 'executor' || + !('delegationContext' in principal) + ) { + throw new InternalUnauthenticatedError('Authentication required') + } + return principal +} + +type WindchillRouteOutput = Extract['output'] +type MutationOperation = Exclude< + WindchillRouteOutput['operation'], + | 'windchill_download_attachment' + | 'windchill_download_primary_content' + | 'windchill_upload_attachments' + | 'windchill_upload_primary_content' +> + +const BULK_RESULT_OPERATIONS = [ + 'windchill_create_documents', + 'windchill_update_documents', + 'windchill_check_out_documents', + 'windchill_check_in_documents', + 'windchill_undo_check_out_documents', + 'windchill_revise_documents', + 'windchill_update_document_security_labels', +] as const satisfies readonly MutationOperation[] + +const DELETE_OPERATIONS = [ + 'windchill_delete_document', + 'windchill_delete_documents', +] as const satisfies readonly MutationOperation[] + +type BulkResultOperation = (typeof BULK_RESULT_OPERATIONS)[number] +type DeleteOperation = (typeof DELETE_OPERATIONS)[number] + +function isBulkResultOperation(operation: MutationOperation): operation is BulkResultOperation { + return BULK_RESULT_OPERATIONS.includes(operation as BulkResultOperation) +} + +function isDeleteOperation(operation: MutationOperation): operation is DeleteOperation { + return DELETE_OPERATIONS.includes(operation as DeleteOperation) +} + +function successResponse(output: WindchillRouteOutput) { + const body = { success: true, output } satisfies WindchillOperationResponse + return NextResponse.json(body) +} + +function failureResponse(error: string, status: number) { + const body = { + success: false, + error: sanitizeWindchillError(error), + } satisfies WindchillOperationResponse + return NextResponse.json(body, { status }) +} + +function documentsById(documentOids: string[]) { + return documentOids.map((ID) => ({ ID })) +} + +/** Keeps the media type and drops any `; charset=...` parameters Windchill cannot use. */ +function safeMimeType(value: string | undefined): string { + const mediaType = value?.split(';', 1)[0]?.trim() + if (mediaType && /^[A-Za-z0-9!#$&^_.+-]+\/[A-Za-z0-9!#$&^_.+-]+$/.test(mediaType)) { + return mediaType + } + return 'application/octet-stream' +} + +function mutationOutput( + operation: MutationOperation, + data: unknown, + fallbackIds: string[] +): WindchillRouteOutput { + const documents = normalizeWindchillDocuments(data) + const document = documents[0] ?? normalizeWindchillDocument(data) + const collectionIds = documents + .map((item) => item.id) + .filter((id): id is string => typeof id === 'string') + const returnedIds = + document?.id && !collectionIds.includes(document.id) + ? [document.id, ...collectionIds] + : collectionIds + const affectedIds = returnedIds.length > 0 ? returnedIds : fallbackIds + if (isDeleteOperation(operation)) return { operation, affectedIds: fallbackIds } + if (isBulkResultOperation(operation)) { + return { + operation, + affectedIds, + ...(documents.length > 0 ? { documents } : {}), + } + } + return { + operation, + affectedIds, + ...(document ? { document } : {}), + } +} + +async function executeMutation( + body: Exclude< + WindchillOperationBody, + | { operation: 'windchill_download_primary_content' } + | { operation: 'windchill_upload_primary_content' } + | { operation: 'windchill_download_attachment' } + | { operation: 'windchill_upload_attachments' } + >, + signal: AbortSignal +): Promise { + const session = await createWindchillSession(body, signal) + const root = normalizeServiceRoot(body.baseUrl) + + switch (body.operation) { + case 'windchill_create_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/Documents`, + method: 'POST', + body: { + ...(body.attributes ?? {}), + Name: body.name, + ...(body.number ? { Number: body.number } : {}), + ...(body.title ? { Title: body.title } : {}), + ...(body.description ? { Description: body.description } : {}), + 'Context@odata.bind': `Containers('${encodeWindchillOid(body.containerOid)}')`, + ...(body.folderOid + ? { 'Folder@odata.bind': `Folders('${encodeWindchillOid(body.folderOid)}')` } + : {}), + }, + signal, + }) + return mutationOutput(body.operation, data, []) + } + case 'windchill_create_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/CreateDocuments`, + method: 'POST', + body: { + Documents: body.documents.map((document) => ({ + ...(document.attributes ?? {}), + Name: document.name, + ...(document.number ? { Number: document.number } : {}), + ...(document.title ? { Title: document.title } : {}), + ...(document.description ? { Description: document.description } : {}), + 'Context@odata.bind': `Containers('${encodeWindchillOid(document.containerOid)}')`, + ...(document.folderOid + ? { 'Folder@odata.bind': `Folders('${encodeWindchillOid(document.folderOid)}')` } + : {}), + })), + }, + signal, + }) + return mutationOutput(body.operation, data, []) + } + case 'windchill_update_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: windchillDocumentUrl(root, body.documentOid), + method: 'PATCH', + body: body.attributes, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_update_common_properties': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.UpdateCommonProperties`, + method: 'POST', + body: { Updates: body.commonProperties }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_update_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/UpdateDocuments`, + method: 'POST', + body: { + Documents: body.documents.map((document) => ({ + ...document.attributes, + ID: document.id, + })), + }, + signal, + }) + return mutationOutput( + body.operation, + data, + body.documents.map((document) => document.id) + ) + } + case 'windchill_delete_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: windchillDocumentUrl(root, body.documentOid), + method: 'DELETE', + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_delete_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/DeleteDocuments`, + method: 'POST', + body: { Documents: documentsById(body.documentOids) }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_check_out_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.CheckOut`, + method: 'POST', + body: { ...(body.checkOutNote ? { CheckOutNote: body.checkOutNote } : {}) }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_check_out_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/CheckOutDocuments`, + method: 'POST', + body: { + Documents: documentsById(body.documentOids), + ...(body.checkOutNote ? { CheckOutNote: body.checkOutNote } : {}), + }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_check_in_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.CheckIn`, + method: 'POST', + body: { + ...(body.checkInNote ? { CheckInNote: body.checkInNote } : {}), + ...(body.keepCheckedOut !== undefined ? { KeepCheckedOut: body.keepCheckedOut } : {}), + ...(body.checkOutNote ? { CheckOutNote: body.checkOutNote } : {}), + }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_check_in_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/CheckInDocuments`, + method: 'POST', + body: { + Documents: documentsById(body.documentOids), + ...(body.checkInNote ? { CheckInNote: body.checkInNote } : {}), + ...(body.keepCheckedOut !== undefined ? { KeepCheckedOut: body.keepCheckedOut } : {}), + ...(body.checkOutNote ? { CheckOutNote: body.checkOutNote } : {}), + }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_undo_check_out_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.UndoCheckOut`, + method: 'POST', + body: {}, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_undo_check_out_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/UndoCheckOutDocuments`, + method: 'POST', + body: { Documents: documentsById(body.documentOids) }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_revise_document': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.Revise`, + method: 'POST', + body: { ...(body.versionId ? { VersionId: body.versionId } : {}) }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_revise_documents': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/ReviseDocuments`, + method: 'POST', + body: { + Documents: documentsById(body.documentOids), + }, + signal, + }) + return mutationOutput(body.operation, data, body.documentOids) + } + case 'windchill_set_lifecycle_state': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${windchillDocumentUrl(root, body.documentOid)}/PTC.DocMgmt.SetState`, + method: 'POST', + body: { State: { Display: body.stateDisplay, Value: body.stateValue } }, + signal, + }) + return mutationOutput(body.operation, data, [body.documentOid]) + } + case 'windchill_update_document_security_labels': { + const data = await windchillMutationRequest({ + params: body, + session, + url: `${root}/DocMgmt/EditDocumentsSecurityLabels`, + method: 'POST', + body: { + Documents: body.securityLabelUpdates.map((update) => ({ + ...update.labels, + ID: update.id, + })), + }, + signal, + }) + return mutationOutput( + body.operation, + data, + body.securityLabelUpdates.map((update) => update.id) + ) + } + } +} + +async function loadUploadFiles( + inputs: RawFileInput[], + userId: string, + requestId: string +): Promise { + let userFiles: UserFile[] + try { + userFiles = processFilesToUserFiles(inputs, requestId, logger) + } catch (error) { + return failureResponse(getErrorMessage(error, 'Invalid file input'), 400) + } + if (userFiles.length !== inputs.length) return failureResponse('Invalid file input', 400) + + const declaredTotal = userFiles.reduce((total, file) => total + file.size, 0) + if (declaredTotal > MAX_FILE_SIZE) { + return failureResponse('Combined Windchill upload exceeds the maximum file size', 413) + } + + const files: WindchillUploadFile[] = [] + let actualTotal = 0 + for (const userFile of userFiles) { + const denied = await assertToolFileAccess(userFile.key, userId, requestId, logger) + if (denied) return denied + try { + const servable = await downloadServableFileFromStorage(userFile, requestId, logger, { + maxBytes: MAX_FILE_SIZE - actualTotal, + }) + actualTotal += servable.buffer.length + if (actualTotal > MAX_FILE_SIZE) { + return failureResponse('Combined Windchill upload exceeds the maximum file size', 413) + } + files.push({ + name: sanitizeFileName(userFile.name), + mimeType: safeMimeType(servable.contentType || userFile.type), + size: servable.buffer.length, + buffer: servable.buffer, + }) + } catch (error) { + const notReady = docNotReadyResponse(error) + if (notReady) return notReady + return failureResponse( + getErrorMessage(error, 'Failed to read uploaded file'), + isPayloadSizeLimitError(error) ? 413 : 400 + ) + } + } + return files +} + +function contentDispositionFileName(value: string | null): string | null { + if (!value) return null + const encoded = value.match(/filename\*=UTF-8''([^;]+)/i)?.[1] + if (encoded) { + try { + return decodeURIComponent(encoded) + } catch { + return encoded + } + } + return ( + value.match(/filename\s*=\s*"([^"]+)"/i)?.[1] ?? + value.match(/filename\s*=\s*([^;]+)/i)?.[1]?.trim() ?? + null + ) +} + +async function storeDownloadedFile({ + principal, + buffer, + fileName, + contentType, +}: { + principal: WorkflowExecutionDelegatedPrincipal + buffer: Buffer + fileName: string + contentType: string +}): Promise { + const { workflowId, executionId } = principal.delegationContext + if (executionId) { + return uploadExecutionFile( + { + workspaceId: principal.workspaceId, + workflowId, + executionId, + }, + buffer, + fileName, + contentType, + principal.subjectUserId + ) + } + return uploadCopilotFile({ + buffer, + fileName, + contentType, + userId: principal.subjectUserId, + }) +} + +async function executeDownload( + body: Extract< + WindchillOperationBody, + | { operation: 'windchill_download_primary_content' } + | { operation: 'windchill_download_attachment' } + >, + principal: WorkflowExecutionDelegatedPrincipal, + signal: AbortSignal +): Promise { + const documentUrl = windchillDocumentUrl(body.baseUrl, body.documentOid) + const contentPath = + body.operation === 'windchill_download_primary_content' + ? `${documentUrl}/PrimaryContent` + : `${documentUrl}/Attachments('${encodeWindchillOid(body.attachmentOid)}')` + const contentUrl = await resolveWindchillContentUrl({ + params: body, + contentPath, + signal, + }) + const downloaded = await downloadWindchillContent({ + params: body, + url: contentUrl, + maxBytes: MAX_FILE_SIZE, + signal, + }) + const fallback = + body.operation === 'windchill_download_primary_content' + ? 'windchill-primary-content.bin' + : 'windchill-attachment.bin' + const fileName = sanitizeFileName( + body.fileName || contentDispositionFileName(downloaded.contentDisposition) || fallback + ) + const mimeType = safeMimeType(downloaded.contentType) + const file = await storeDownloadedFile({ + principal, + buffer: downloaded.buffer, + fileName, + contentType: mimeType, + }) + return { + operation: body.operation, + file: { ...file }, + fileName, + mimeType, + } +} + +export const POST = withRouteHandler( + async (request: NextRequest) => { + const requestId = generateRequestId() + let principal: WorkflowExecutionDelegatedPrincipal + try { + principal = await authenticateWindchillExecutor(request) + } catch (error) { + if (error instanceof InternalUnauthenticatedError) { + return failureResponse(error.message, 401) + } + throw error + } + + const parsed = await parseRequest( + windchillOperationContract, + request, + {}, + { + validationErrorResponse: (error) => + failureResponse(getValidationErrorMessage(error, 'Invalid Windchill request'), 400), + invalidJsonResponse: () => + failureResponse('Windchill request body must be valid JSON', 400), + payloadTooLargeResponse: () => failureResponse('Windchill request body is too large', 413), + } + ) + if (!parsed.success) return parsed.response + const body = parsed.data.body + + try { + if ( + body.operation === 'windchill_download_primary_content' || + body.operation === 'windchill_download_attachment' + ) { + return successResponse(await executeDownload(body, principal, request.signal)) + } + + if ( + body.operation === 'windchill_upload_primary_content' || + body.operation === 'windchill_upload_attachments' + ) { + const inputs = + body.operation === 'windchill_upload_primary_content' + ? [body.primaryFile] + : body.attachmentFiles + const files = await loadUploadFiles(inputs, principal.subjectUserId, requestId) + if (files instanceof NextResponse) return files + const uploadedFileNames = await uploadWindchillContent({ + params: body, + documentOid: body.documentOid, + files, + primaryContent: body.operation === 'windchill_upload_primary_content', + signal: request.signal, + }) + return successResponse({ + operation: body.operation, + affectedIds: [body.documentOid], + uploadedFileNames, + }) + } + + return successResponse(await executeMutation(body, request.signal)) + } catch (error) { + logger.error('Windchill operation failed', { + operation: body.operation, + error: sanitizeWindchillError(getErrorMessage(error, 'Windchill operation failed')), + }) + if (error instanceof WindchillProviderError) { + const status = error.status >= 400 && error.status <= 599 ? error.status : 502 + return failureResponse(error.message, status) + } + return failureResponse( + getErrorMessage(error, 'Windchill operation failed'), + isPayloadSizeLimitError(error) ? 413 : 500 + ) + } + }, + { + unhandledErrorResponse: () => failureResponse('Windchill operation failed', 500), + } +) diff --git a/apps/sim/app/api/users/me/usage-logs/cursor-route.test.ts b/apps/sim/app/api/users/me/usage-logs/cursor-route.test.ts new file mode 100644 index 00000000000..5d747a37d0c --- /dev/null +++ b/apps/sim/app/api/users/me/usage-logs/cursor-route.test.ts @@ -0,0 +1,50 @@ +/** + * @vitest-environment node + * + * The unresolvable-cursor rejection, end to end on the session-only ledger route. + * + * Deliberately a separate file from `route.test.ts`: that suite replaces + * `@/lib/billing/core/usage-log` with mocks, which is exactly the seam this case + * has to cross. Here the real query runs against the shared `@sim/db` chain mock, + * so the assertion covers the throw in billing core, `withRouteHandler`'s typed-error + * projection, and the message the caller reads — the path that answered 500 while the + * rejection was an `OrchestrationError` alone. + */ +import { authMockFns, createMockRequest, dbChainMockFns, resetDbChainMock } from '@sim/testing' +import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' +import { UNKNOWN_CURSOR_MESSAGE } from '@/lib/billing/core/usage-log' +import { GET } from '@/app/api/users/me/usage-logs/route' + +afterAll(() => { + resetDbChainMock() +}) + +describe('GET /api/users/me/usage-logs cursor rejection', () => { + beforeEach(() => { + vi.clearAllMocks() + resetDbChainMock() + authMockFns.mockGetSession.mockResolvedValue({ user: { id: 'user-1' } }) + }) + + it('answers 400 when the cursor names no usage event', async () => { + dbChainMockFns.limit.mockResolvedValueOnce([]) + + const response = await GET( + createMockRequest( + 'GET', + undefined, + {}, + 'http://localhost:3000/api/users/me/usage-logs?cursor=log-from-another-ledger' + ) + ) + + expect(response.status).toBe(400) + await expect(response.json()).resolves.toMatchObject({ error: UNKNOWN_CURSOR_MESSAGE }) + }) + + it('answers 200 for a request carrying no cursor', async () => { + const response = await GET(createMockRequest('GET')) + + expect(response.status).toBe(200) + }) +}) diff --git a/apps/sim/app/api/v1/admin/responses.ts b/apps/sim/app/api/v1/admin/responses.ts index 9308df895dc..3ecc5353b29 100644 --- a/apps/sim/app/api/v1/admin/responses.ts +++ b/apps/sim/app/api/v1/admin/responses.ts @@ -51,9 +51,7 @@ export function errorResponse( return NextResponse.json(body, { status }) } -// ============================================================================= // Common Error Responses -// ============================================================================= export function unauthorizedResponse(message = 'Authentication required'): NextResponse { return errorResponse('UNAUTHORIZED', message, 401) diff --git a/apps/sim/app/api/v1/admin/types.ts b/apps/sim/app/api/v1/admin/types.ts index 4256076d457..a6062ca8eee 100644 --- a/apps/sim/app/api/v1/admin/types.ts +++ b/apps/sim/app/api/v1/admin/types.ts @@ -20,9 +20,7 @@ import type { InferSelectModel } from 'drizzle-orm' import type { Edge } from 'reactflow' import type { BlockState, Loop, Parallel } from '@/stores/workflows/workflow/types' -// ============================================================================= // Database Model Types (inferred from schema) -// ============================================================================= export type DbUser = InferSelectModel export type DbWorkspace = InferSelectModel @@ -33,9 +31,7 @@ export type DbSubscription = InferSelectModel export type DbMember = InferSelectModel export type DbUserStats = InferSelectModel -// ============================================================================= // Pagination -// ============================================================================= export interface PaginationParams { limit: number @@ -74,9 +70,7 @@ export function createPaginationMeta(total: number, limit: number, offset: numbe } } -// ============================================================================= // API Response Types -// ============================================================================= export interface AdminListResponse { data: T[] @@ -95,9 +89,7 @@ export interface AdminErrorResponse { } } -// ============================================================================= // User Types -// ============================================================================= export interface AdminUser { id: string @@ -121,9 +113,7 @@ export function toAdminUser(dbUser: DbUser): AdminUser { } } -// ============================================================================= // Workspace Types -// ============================================================================= export interface AdminWorkspace { id: string @@ -148,9 +138,7 @@ export function toAdminWorkspace(dbWorkspace: DbWorkspace): AdminWorkspace { } } -// ============================================================================= // Folder Types -// ============================================================================= export interface AdminFolder { id: string @@ -179,9 +167,7 @@ export function toAdminFolder(dbFolder: DbWorkflowFolder): AdminFolder { } } -// ============================================================================= // Workflow Types -// ============================================================================= export interface AdminWorkflow { id: string @@ -233,9 +219,7 @@ export function toAdminWorkflow(dbWorkflow: AdminWorkflowSource): AdminWorkflow } } -// ============================================================================= // Workflow Variable Types -// ============================================================================= export type VariableType = 'string' | 'number' | 'boolean' | 'object' | 'array' | 'plain' @@ -246,9 +230,7 @@ export interface WorkflowVariable { value: unknown } -// ============================================================================= // Export/Import Types -// ============================================================================= export interface WorkflowExportState { blocks: Record @@ -296,9 +278,7 @@ export interface WorkspaceExportPayload { folders: FolderExportPayload[] } -// ============================================================================= // Import Types -// ============================================================================= export interface WorkflowImportRequest { workspaceId: string @@ -328,9 +308,7 @@ export interface WorkspaceImportResponse { results: ImportResult[] } -// ============================================================================= // Utility Functions -// ============================================================================= /** * Extract workflow metadata from various export formats. @@ -384,9 +362,7 @@ function getNestedString(obj: Record, path: string): string | u return typeof current === 'string' ? current : undefined } -// ============================================================================= // Organization Types -// ============================================================================= export interface AdminOrganization { id: string @@ -432,9 +408,7 @@ export function toAdminOrganization(dbOrg: AdminOrganizationSource): AdminOrgani } } -// ============================================================================= // Subscription Types -// ============================================================================= export interface AdminSubscription { id: string @@ -470,9 +444,7 @@ export function toAdminSubscription(dbSub: DbSubscription): AdminSubscription { } } -// ============================================================================= // Member Types -// ============================================================================= export interface AdminMember { id: string @@ -492,9 +464,7 @@ export interface AdminMemberDetail extends AdminMember { billingBlocked: boolean } -// ============================================================================= // Workspace Member Types -// ============================================================================= export interface AdminWorkspaceMember { id: string @@ -508,9 +478,7 @@ export interface AdminWorkspaceMember { userImage: string | null } -// ============================================================================= // User Billing Types -// ============================================================================= interface AdminUserBilling { userId: string @@ -539,9 +507,7 @@ export interface AdminUserBillingWithSubscription extends AdminUserBilling { }> } -// ============================================================================= // Organization Billing Summary Types -// ============================================================================= export interface AdminOrganizationBillingSummary { organizationId: string @@ -587,9 +553,7 @@ export interface AdminDeploymentVersion { deployedByName: string | null } -// ============================================================================= // Audit Log Types -// ============================================================================= export type DbAuditLog = InferSelectModel diff --git a/apps/sim/app/api/v1/auth.test.ts b/apps/sim/app/api/v1/auth.test.ts new file mode 100644 index 00000000000..90f0f2f1d76 --- /dev/null +++ b/apps/sim/app/api/v1/auth.test.ts @@ -0,0 +1,86 @@ +/** + * @vitest-environment node + */ + +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + authenticateApiKey: vi.fn(), + updateLastUsed: vi.fn(), +})) + +vi.mock('@/lib/core/config/env-flags', () => ({ isAuthDisabled: false })) +vi.mock('@/lib/api-key/service', () => ({ + authenticateApiKeyFromHeader: mocks.authenticateApiKey, + updateApiKeyLastUsed: mocks.updateLastUsed, +})) + +import { authenticateV1Request } from '@/app/api/v1/auth' + +describe('v1 API key authentication', () => { + beforeEach(() => { + vi.clearAllMocks() + }) + + it('constructs a personal API-key Principal from canonical key identity', async () => { + mocks.authenticateApiKey.mockResolvedValue({ + success: true, + userId: 'user-1', + keyId: 'key-1', + keyType: 'personal', + }) + + await expect( + authenticateV1Request( + new NextRequest('http://localhost/api/v1/files', { + headers: { 'x-api-key': 'secret' }, + }) + ) + ).resolves.toMatchObject({ + authenticated: true, + principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' }, + }) + }) + + it('constructs a workspace API-key Principal without borrowing the creator identity', async () => { + mocks.authenticateApiKey.mockResolvedValue({ + success: true, + userId: 'creator-1', + keyId: 'key-1', + keyType: 'workspace', + workspaceId: 'workspace-1', + }) + + const result = await authenticateV1Request( + new NextRequest('http://localhost/api/v1/files', { + headers: { 'x-api-key': 'secret' }, + }) + ) + + expect(result.principal).toEqual({ + kind: 'workspace_api_key', + workspaceId: 'workspace-1', + keyId: 'key-1', + }) + expect(result.principal).not.toHaveProperty('userId') + }) + + it('fails closed when authenticated key identity is incomplete', async () => { + mocks.authenticateApiKey.mockResolvedValue({ + success: true, + userId: 'creator-1', + keyId: 'key-1', + keyType: 'workspace', + }) + + await expect( + authenticateV1Request( + new NextRequest('http://localhost/api/v1/files', { + headers: { 'x-api-key': 'secret' }, + }) + ) + ).resolves.toEqual({ authenticated: false, error: 'Authentication failed' }) + expect(mocks.updateLastUsed).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/v1/auth.ts b/apps/sim/app/api/v1/auth.ts index 0f391889005..78c68e1f9dd 100644 --- a/apps/sim/app/api/v1/auth.ts +++ b/apps/sim/app/api/v1/auth.ts @@ -1,3 +1,4 @@ +import type { PersonalApiKeyPrincipal, WorkspaceApiKeyPrincipal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import type { NextRequest } from 'next/server' import { authenticateApiKeyFromHeader, updateApiKeyLastUsed } from '@/lib/api-key/service' @@ -11,6 +12,7 @@ export interface AuthResult { userId?: string workspaceId?: string keyType?: 'personal' | 'workspace' + principal?: PersonalApiKeyPrincipal | WorkspaceApiKeyPrincipal error?: string } @@ -20,6 +22,11 @@ export async function authenticateV1Request(request: NextRequest): Promise ({ mockCheckRateLimit: vi.fn(), mockValidateWorkspaceAccess: vi.fn(), mockGetWorkspaceFile: vi.fn(), - mockFetchServableWorkspaceFileBuffer: vi.fn(), + mockDownloadWorkspaceFileStream: vi.fn(), })) vi.mock('@/app/api/v1/middleware', () => ({ checkRateLimit: mockCheckRateLimit, createRateLimitResponse: () => new Response('rate limited', { status: 429 }), + requireRateLimitPrincipal: (rateLimit: { principal: unknown }) => rateLimit.principal, validateWorkspaceAccess: mockValidateWorkspaceAccess, v1ValidationErrorResponse: (e: { issues: unknown[] }) => NextResponse.json({ error: 'Validation error', details: e.issues }, { status: 400 }), })) vi.mock('@/lib/uploads/contexts/workspace', () => ({ getWorkspaceFile: mockGetWorkspaceFile, - fetchServableWorkspaceFileBuffer: mockFetchServableWorkspaceFileBuffer, +})) +vi.mock('@/lib/workspace-files/application/download-workspace-file', () => ({ + downloadWorkspaceFileStream: { execute: mockDownloadWorkspaceFileStream }, })) vi.mock('@/lib/workspace-files/orchestration', () => ({ performDeleteWorkspaceFileItems: vi.fn(), @@ -37,7 +40,7 @@ vi.mock('@sim/audit', () => ({ })) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: vi.fn() })) -import { DocCompileUserError } from '@/lib/copilot/tools/server/files/doc-compile-error' +import { OrchestrationError } from '@/lib/core/orchestration/types' import { GET } from '@/app/api/v1/files/[fileId]/route' const WORKSPACE_ID = 'ws-1' @@ -45,6 +48,11 @@ const FILE_ID = 'file-1' const context = { params: Promise.resolve({ fileId: FILE_ID }) } const DOCX_MIME = 'application/vnd.openxmlformats-officedocument.wordprocessingml.document' +const PRINCIPAL = { + kind: 'personal_api_key' as const, + userId: 'user-1', + keyId: 'key-1', +} function request() { return createMockRequest( @@ -71,16 +79,31 @@ function generatedDocument(name = 'report.docx') { } } +function renderedDownload(buffer: Buffer) { + return { + file: generatedDocument(), + stream: new ReadableStream({ + start(controller) { + controller.enqueue(buffer) + controller.close() + }, + }), + contentLength: buffer.length, + contentType: DOCX_MIME, + } +} + describe('v1 file download', () => { beforeEach(() => { vi.clearAllMocks() - mockCheckRateLimit.mockResolvedValue({ allowed: true, userId: 'user-1' }) + mockCheckRateLimit.mockResolvedValue({ + allowed: true, + userId: 'user-1', + principal: PRINCIPAL, + }) mockValidateWorkspaceAccess.mockResolvedValue(null) mockGetWorkspaceFile.mockResolvedValue(generatedDocument()) - mockFetchServableWorkspaceFileBuffer.mockResolvedValue({ - buffer: Buffer.from('PKrendered'), - contentType: DOCX_MIME, - }) + mockDownloadWorkspaceFileStream.mockResolvedValue(renderedDownload(Buffer.from('PKrendered'))) }) it('serves the rendered bytes and the rendered content type', async () => { @@ -104,10 +127,7 @@ describe('v1 file download', () => { it('reports Content-Length from the rendered bytes, not the declared source size', async () => { const rendered = Buffer.alloc(50_000) - mockFetchServableWorkspaceFileBuffer.mockResolvedValue({ - buffer: rendered, - contentType: DOCX_MIME, - }) + mockDownloadWorkspaceFileStream.mockResolvedValue(renderedDownload(rendered)) const response = await GET(request(), context) @@ -115,8 +135,8 @@ describe('v1 file download', () => { }) it('returns a retryable 409 while the artifact is still compiling', async () => { - mockFetchServableWorkspaceFileBuffer.mockRejectedValue( - new DocCompileUserError('Document is still being generated') + mockDownloadWorkspaceFileStream.mockRejectedValue( + new OrchestrationError('conflict', 'Document is still being generated') ) const response = await GET(request(), context) @@ -127,11 +147,17 @@ describe('v1 file download', () => { }) it('404s a file that does not exist', async () => { - mockGetWorkspaceFile.mockResolvedValue(null) + mockDownloadWorkspaceFileStream.mockRejectedValue( + new OrchestrationError('not_found', 'File not found') + ) const response = await GET(request(), context) expect(response.status).toBe(404) - expect(mockFetchServableWorkspaceFileBuffer).not.toHaveBeenCalled() + expect(mockDownloadWorkspaceFileStream).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: { fileId: FILE_ID, assertedWorkspaceId: WORKSPACE_ID }, + request: expect.anything(), + }) }) }) diff --git a/apps/sim/app/api/v1/files/[fileId]/route.ts b/apps/sim/app/api/v1/files/[fileId]/route.ts index 1e9b084e680..eb767ac5f3c 100644 --- a/apps/sim/app/api/v1/files/[fileId]/route.ts +++ b/apps/sim/app/api/v1/files/[fileId]/route.ts @@ -1,20 +1,18 @@ -import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit' import { createLogger } from '@sim/logger' import { type NextRequest, NextResponse } from 'next/server' import { v1DeleteFileContract, v1DownloadFileContract } from '@/lib/api/contracts/v1/files' import { parseRequest } from '@/lib/api/server' +import { asOrchestrationError, statusForOrchestrationError } from '@/lib/core/orchestration/types' import { generateRequestId } from '@/lib/core/utils/request' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { captureServerEvent } from '@/lib/posthog/server' -import { - fetchServableWorkspaceFileBuffer, - getWorkspaceFile, -} from '@/lib/uploads/contexts/workspace' -import { docNotReadyMessage, isDocNotReadyError } from '@/lib/uploads/utils/doc-not-ready' +import { getWorkspaceFile } from '@/lib/uploads/contexts/workspace' +import { downloadWorkspaceFileStream } from '@/lib/workspace-files/application/download-workspace-file' import { performDeleteWorkspaceFileItems } from '@/lib/workspace-files/orchestration' import { checkRateLimit, createRateLimitResponse, + requireRateLimitPrincipal, v1ValidationErrorResponse, validateWorkspaceAccess, } from '@/app/api/v1/middleware' @@ -38,7 +36,6 @@ export const GET = withRouteHandler(async (request: NextRequest, context: FileRo return createRateLimitResponse(rateLimit) } - const userId = rateLimit.userId! const parsed = await parseRequest(v1DownloadFileContract, request, context, { validationErrorResponse: v1ValidationErrorResponse, }) @@ -47,64 +44,43 @@ export const GET = withRouteHandler(async (request: NextRequest, context: FileRo const { fileId } = parsed.data.params const { workspaceId } = parsed.data.query - const accessError = await validateWorkspaceAccess(rateLimit, userId, workspaceId) - if (accessError) return accessError - - const fileRecord = await getWorkspaceFile(workspaceId, fileId) - if (!fileRecord) { - return NextResponse.json({ error: 'File not found' }, { status: 404 }) + const principal = requireRateLimitPrincipal(rateLimit) + const { file, stream, contentLength, contentType } = await downloadWorkspaceFileStream.execute({ + principal, + input: { fileId, assertedWorkspaceId: workspaceId }, + request, + }) + if (principal.kind === 'personal_api_key') { + captureServerEvent( + principal.userId, + 'file_downloaded', + { workspace_id: workspaceId, is_bulk: false, file_count: 1 }, + { groups: { workspace: workspaceId } } + ) } - // Generated docs store their generation source; serve the rendered artifact. - // Its content type is the rendered one, not the source MIME on the record. - const { buffer, contentType } = await fetchServableWorkspaceFileBuffer(fileRecord) - - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FILE_DOWNLOADED, - resourceType: AuditResourceType.FILE, - resourceId: fileRecord.id, - resourceName: fileRecord.name, - description: `Downloaded file "${fileRecord.name}" via API`, - metadata: { - fileId: fileRecord.id, - fileName: fileRecord.name, - bytes: buffer.length, - source: 'api_v1', + return new Response(stream, { + status: 200, + headers: { + 'Content-Type': contentType || file.type || 'application/octet-stream', + 'Content-Disposition': `attachment; filename="${file.name.replace(/[^\w.-]/g, '_')}"; filename*=UTF-8''${encodeURIComponent(file.name)}`, + 'Content-Length': String(contentLength), + 'X-File-Id': file.id, + 'X-File-Name': encodeURIComponent(file.name), + 'X-Uploaded-At': + file.uploadedAt instanceof Date ? file.uploadedAt.toISOString() : String(file.uploadedAt), }, - request, }) - captureServerEvent( - userId, - 'file_downloaded', - { workspace_id: workspaceId, is_bulk: false, file_count: 1 }, - { groups: { workspace: workspaceId } } - ) - - // View, not copy — a second full copy would double peak memory for a large file. - return new Response( - new Uint8Array(buffer.buffer as ArrayBuffer, buffer.byteOffset, buffer.byteLength), - { - status: 200, - headers: { - 'Content-Type': contentType || fileRecord.type || 'application/octet-stream', - 'Content-Disposition': `attachment; filename="${fileRecord.name.replace(/[^\w.-]/g, '_')}"; filename*=UTF-8''${encodeURIComponent(fileRecord.name)}`, - 'Content-Length': String(buffer.length), - 'X-File-Id': fileRecord.id, - 'X-File-Name': encodeURIComponent(fileRecord.name), - 'X-Uploaded-At': - fileRecord.uploadedAt instanceof Date - ? fileRecord.uploadedAt.toISOString() - : String(fileRecord.uploadedAt), - }, - } - ) } catch (error) { - // A generated doc whose artifact is still compiling is retryable, not a fault: - // without this the caller sees a 500 and has no reason to try again. - if (isDocNotReadyError(error)) { - return NextResponse.json({ error: docNotReadyMessage() }, { status: 409 }) + const orchestrationError = asOrchestrationError(error) + if (orchestrationError && orchestrationError.code !== 'internal') { + return NextResponse.json( + { + error: + orchestrationError.code === 'not_found' ? 'File not found' : orchestrationError.message, + }, + { status: statusForOrchestrationError(orchestrationError.code) } + ) } logger.error(`[${requestId}] Error downloading file:`, error) return NextResponse.json({ error: 'Failed to download file' }, { status: 500 }) diff --git a/apps/sim/app/api/v1/files/route.ts b/apps/sim/app/api/v1/files/route.ts index 7cc3f0fbd51..36adcb392e4 100644 --- a/apps/sim/app/api/v1/files/route.ts +++ b/apps/sim/app/api/v1/files/route.ts @@ -6,6 +6,7 @@ import { v1ListFilesContract, v1UploadFileFormFieldsSchema } from '@/lib/api/con import { getValidationErrorMessage, parseRequest } from '@/lib/api/server' import { generateRequestId } from '@/lib/core/utils/request' import { + isMultipartFieldValidationError, isPayloadSizeLimitError, MAX_MULTIPART_OVERHEAD_BYTES, readFileToBufferWithLimit, @@ -106,6 +107,9 @@ export const POST = withRouteHandler(async (request: NextRequest) => { if (isPayloadSizeLimitError(error)) { return NextResponse.json({ error: error.message }, { status: 413 }) } + if (isMultipartFieldValidationError(error)) { + return NextResponse.json({ error: error.message }, { status: 400 }) + } return NextResponse.json( { error: 'Request body must be valid multipart form data' }, { status: 400 } diff --git a/apps/sim/app/api/v1/knowledge/[id]/documents/route.ts b/apps/sim/app/api/v1/knowledge/[id]/documents/route.ts index 94999266c7a..2a5ddb92f2c 100644 --- a/apps/sim/app/api/v1/knowledge/[id]/documents/route.ts +++ b/apps/sim/app/api/v1/knowledge/[id]/documents/route.ts @@ -14,6 +14,7 @@ import { statusForOrchestrationError, } from '@/lib/core/orchestration/types' import { + isMultipartFieldValidationError, isPayloadSizeLimitError, MAX_MULTIPART_OVERHEAD_BYTES, readFormDataWithLimit, @@ -117,6 +118,9 @@ export const POST = withRouteHandler( if (isPayloadSizeLimitError(error)) { return NextResponse.json({ error: error.message }, { status: 413 }) } + if (isMultipartFieldValidationError(error)) { + return NextResponse.json({ error: error.message }, { status: 400 }) + } return NextResponse.json( { error: 'Request body must be valid multipart form data' }, { status: 400 } diff --git a/apps/sim/app/api/v1/middleware.test.ts b/apps/sim/app/api/v1/middleware.test.ts index 94c0790b274..2a8e598a7b6 100644 --- a/apps/sim/app/api/v1/middleware.test.ts +++ b/apps/sim/app/api/v1/middleware.test.ts @@ -61,6 +61,7 @@ describe('checkRateLimit', () => { authenticated: true, userId: 'user-1', keyType: 'personal', + principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' }, }) mockGetSubscription.mockResolvedValue({ plan: 'team' }) mockGetRateLimit.mockReturnValue(TEAM_BUCKET) @@ -78,6 +79,16 @@ describe('checkRateLimit', () => { expect(result.limit).not.toBe(TEAM_BUCKET.refillRate) }) + it('preserves the authenticated API-key Principal for application operations', async () => { + const result = await checkRateLimit(request(), 'workflows') + + expect(result.principal).toEqual({ + kind: 'personal_api_key', + userId: 'user-1', + keyId: 'key-1', + }) + }) + it('never reports more remaining than the limit', async () => { const result = await checkRateLimit(request(), 'workflows') @@ -196,6 +207,7 @@ describe('rate-limit snapshot context', () => { authenticated: true, userId: 'user-1', keyType: 'personal', + principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' }, }) mockGetSubscription.mockResolvedValue({ plan: 'team' }) mockGetRateLimit.mockReturnValue(TEAM_BUCKET) diff --git a/apps/sim/app/api/v1/middleware.ts b/apps/sim/app/api/v1/middleware.ts index 6a9db50ec44..3f8d4878119 100644 --- a/apps/sim/app/api/v1/middleware.ts +++ b/apps/sim/app/api/v1/middleware.ts @@ -1,3 +1,4 @@ +import type { PersonalApiKeyPrincipal, WorkspaceApiKeyPrincipal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { type PermissionType, permissionSatisfies } from '@sim/platform-authz/workspace' import { type NextRequest, NextResponse } from 'next/server' @@ -63,6 +64,7 @@ export interface RateLimitResult { userId?: string workspaceId?: string keyType?: 'personal' | 'workspace' + principal?: PersonalApiKeyPrincipal | WorkspaceApiKeyPrincipal error?: string } @@ -82,6 +84,18 @@ export function requireRateLimitUserId(rateLimit: RateLimitResult): string { return rateLimit.userId } +export function requireRateLimitPrincipal( + rateLimit: RateLimitResult +): PersonalApiKeyPrincipal | WorkspaceApiKeyPrincipal { + if (!rateLimit.allowed) { + throw new Error('Cannot authorize a denied public API request') + } + if (!rateLimit.principal) { + throw new Error('Allowed public API request is missing its Principal') + } + return rateLimit.principal +} + export async function checkRateLimit( request: NextRequest, endpoint: ApiEndpoint = 'logs' @@ -144,6 +158,7 @@ export async function checkRateLimit( userId, workspaceId: auth.workspaceId, keyType: auth.keyType, + principal: auth.principal, } } catch (error) { logger.error('Rate limit check error', { error }) diff --git a/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts b/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts index aa3f74d8157..ae8c00affab 100644 --- a/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts +++ b/apps/sim/app/api/v1/tables/[tableId]/columns/route.ts @@ -12,10 +12,10 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { addTableColumn, deleteColumn } from '@/lib/table' import { signalTableSchemaChanged } from '@/lib/table/events' import { performUpdateTableColumn } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationErrorResponse, orchestrationOutcomeErrorResponse, tableLockErrorResponse, diff --git a/apps/sim/app/api/v1/tables/[tableId]/route.ts b/apps/sim/app/api/v1/tables/[tableId]/route.ts index caaf87d8be7..5d46bdf619b 100644 --- a/apps/sim/app/api/v1/tables/[tableId]/route.ts +++ b/apps/sim/app/api/v1/tables/[tableId]/route.ts @@ -6,10 +6,10 @@ import { generateRequestId } from '@/lib/core/utils/request' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import type { TableSchema } from '@/lib/table' import { performDeleteTable } from '@/lib/table/orchestration' +import { normalizeColumn } from '@/lib/table/wire' import { accessError, checkAccess, - normalizeColumn, orchestrationOutcomeErrorResponse, tableLockErrorResponse, } from '@/app/api/table/utils' diff --git a/apps/sim/app/api/v1/tables/route.test.ts b/apps/sim/app/api/v1/tables/route.test.ts index ded5f484e8d..f12bceb2334 100644 --- a/apps/sim/app/api/v1/tables/route.test.ts +++ b/apps/sim/app/api/v1/tables/route.test.ts @@ -35,9 +35,11 @@ vi.mock('@/app/api/v1/middleware', () => ({ })) vi.mock('@/app/api/table/utils', () => ({ - normalizeColumn: (column: unknown) => column, orchestrationErrorResponse: mocks.orchestrationErrorResponse, })) +vi.mock('@/lib/table/wire', () => ({ + normalizeColumn: (column: unknown) => column, +})) vi.mock('@/lib/table', () => ({ createTable: mocks.createTable, diff --git a/apps/sim/app/api/v1/tables/route.ts b/apps/sim/app/api/v1/tables/route.ts index e8a13eb9090..ecd742efb29 100644 --- a/apps/sim/app/api/v1/tables/route.ts +++ b/apps/sim/app/api/v1/tables/route.ts @@ -12,7 +12,8 @@ import { TableConflictError, type TableSchema, } from '@/lib/table' -import { normalizeColumn, orchestrationErrorResponse } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' +import { orchestrationErrorResponse } from '@/app/api/table/utils' import { checkRateLimit, createRateLimitResponse, diff --git a/apps/sim/app/api/v2/[[...segments]]/route.test.ts b/apps/sim/app/api/v2/[[...segments]]/route.test.ts new file mode 100644 index 00000000000..4d4e14da713 --- /dev/null +++ b/apps/sim/app/api/v2/[[...segments]]/route.test.ts @@ -0,0 +1,50 @@ +/** + * @vitest-environment node + */ +import { NextRequest } from 'next/server' +import { describe, expect, it } from 'vitest' +import { DELETE, GET, PATCH, POST, PUT } from '@/app/api/v2/[[...segments]]/route' + +/** + * An unknown path under `/api/v2` used to fall through to the app's global + * `not-found` page, so a mistyped URL handed an API client a full HTML document + * — the one v2 response a JSON-parsing caller cannot read. + * + * The body must stay byte-identical to the rollout gate's 404 + * (`v2ApiGateError`), which answers 404 so an ungated caller cannot tell "not in + * the cohort" from "no such endpoint". A different body here would give that + * distinction straight back. + */ +describe('unknown /api/v2 path', () => { + const EXPECTED = { error: { code: 'NOT_FOUND', message: 'Not found' } } + + function probe(method: string) { + return new NextRequest('http://localhost/api/v2/nonexistent', { method }) + } + + it('answers JSON, not an HTML document', async () => { + const response = await GET(probe('GET'), undefined) + + expect(response.status).toBe(404) + expect(response.headers.get('content-type')).toContain('application/json') + expect(await response.json()).toEqual(EXPECTED) + }) + + it.each([ + ['POST', POST], + ['PUT', PUT], + ['PATCH', PATCH], + ['DELETE', DELETE], + ])('answers the same envelope for %s', async (method, handler) => { + const response = await handler(probe(method), undefined) + + expect(response.status).toBe(404) + expect(await response.json()).toEqual(EXPECTED) + }) + + it('does not require an API key, so probing a typo cannot become a 401', async () => { + const response = await GET(probe('GET'), undefined) + + expect(response.status).toBe(404) + }) +}) diff --git a/apps/sim/app/api/v2/[[...segments]]/route.ts b/apps/sim/app/api/v2/[[...segments]]/route.ts new file mode 100644 index 00000000000..893f455a7bc --- /dev/null +++ b/apps/sim/app/api/v2/[[...segments]]/route.ts @@ -0,0 +1,40 @@ +import { withRouteHandler } from '@/lib/core/utils/with-route-handler' +import { v2Error } from '@/app/api/v2/lib/response' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * JSON 404 for any `/api/v2` path that matches no route file. + * + * Without it a mistyped path falls through to the app's global `not-found` + * page and hands an API client a full HTML document, which is the one v2 + * response a JSON-parsing caller cannot read. Every other v2 failure — including + * the rollout gate's own 404 — is the `{ error: { code, message } }` envelope. + * + * The body is deliberately byte-identical to `v2ApiGateError`'s. The gate + * answers 404 so an ungated caller cannot distinguish "not in the rollout + * cohort" from "no such endpoint"; a different body here would reintroduce + * exactly that distinction. + * + * This is a documented raw-`withRouteHandler` route rather than a contract + * builder: it has no contract, no operation, and no authentication, because a + * caller probing an unknown path must get the same answer whether or not it + * holds a key — requiring auth first would turn the 404 into a 401 and confirm + * that the path is special. + * + * Next.js only routes a request here when no literal segment matches, so every + * real v2 route file is unaffected, however many there are. The optional form (`[[...segments]]`) also + * covers bare `/api/v2`. It cannot fix a 405 on a path that *does* have a route + * file but does not export that verb — Next generates that response itself, + * before any handler runs. + */ +const notFound = () => v2Error('NOT_FOUND', 'Not found') + +export const GET = withRouteHandler(notFound) +export const POST = withRouteHandler(notFound) +export const PUT = withRouteHandler(notFound) +export const PATCH = withRouteHandler(notFound) +export const DELETE = withRouteHandler(notFound) +export const HEAD = withRouteHandler(notFound) +export const OPTIONS = withRouteHandler(notFound) diff --git a/apps/sim/app/api/v2/audit-logs/route.test.ts b/apps/sim/app/api/v2/audit-logs/route.test.ts index ec46e6fe9d6..59f7e6107bc 100644 --- a/apps/sim/app/api/v2/audit-logs/route.test.ts +++ b/apps/sim/app/api/v2/audit-logs/route.test.ts @@ -29,6 +29,7 @@ vi.mock('@/lib/audit-logs/application/get-audit-log', () => ({ getAuditLog: { operation: { id: 'audit_logs.read_detail' }, execute: mocks.get }, })) +import { REFILTERED_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { OrchestrationError } from '@/lib/core/orchestration/types' import { GET as getDetail } from '@/app/api/v2/audit-logs/[id]/route' import { GET as listLogs } from '@/app/api/v2/audit-logs/route' @@ -84,7 +85,12 @@ describe('v2 audit-log routes', () => { const response = await listLogs(request) expect(response.status).toBe(200) - expect(await response.json()).toMatchObject({ data: [{ id: 'audit-1' }], nextCursor: 'next-1' }) + const body = await response.json() + expect(body).toMatchObject({ data: [{ id: 'audit-1' }] }) + /** The domain token travels inside the query-bound wrapper, not bare. */ + expect(JSON.parse(Buffer.from(body.nextCursor, 'base64').toString())).toMatchObject({ + inner: 'next-1', + }) expect(mocks.list).toHaveBeenCalledWith({ principal: auth.principal, input: expect.objectContaining({ @@ -96,6 +102,132 @@ describe('v2 audit-log routes', () => { expect(response.headers.get('x-ratelimit-limit')).toBe('100') }) + /** + * Pins the binding end-to-end — the mint in `present` and the read in + * `mapInput` — because the contract-level sweep only checks a hand-maintained + * map of param names and stays green when a route drops the stamp entirely. + */ + it('refuses a cursor minted under a different filter', async () => { + const minted = await listLogs( + new NextRequest( + 'http://localhost:3000/api/v2/audit-logs?organizationId=org-1&actorEmail=ada%40example.com' + ) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.list.mockClear() + const replayed = await listLogs( + new NextRequest( + `http://localhost:3000/api/v2/audit-logs?organizationId=org-1&actorEmail=bob%40example.com&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.list).not.toHaveBeenCalled() + }) + + /** + * A window bound selects by instant, and the query schema admits every + * sub-second spelling of one, so the same window written a different way must + * resume rather than 400. + */ + it('resumes a cursor whose window bound is respelled to the same instant', async () => { + const minted = await listLogs( + new NextRequest( + 'http://localhost:3000/api/v2/audit-logs?organizationId=org-1&startDate=2026-01-01T00%3A00%3A00Z' + ) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.list.mockClear() + const resumed = await listLogs( + new NextRequest( + `http://localhost:3000/api/v2/audit-logs?organizationId=org-1&startDate=2026-01-01T00%3A00%3A00.000Z&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(resumed.status).toBe(200) + expect(mocks.list).toHaveBeenCalled() + }) + + it('resumes a cursor replayed under the filters it was minted with', async () => { + const minted = await listLogs( + new NextRequest( + 'http://localhost:3000/api/v2/audit-logs?organizationId=org-1&actorEmail=ada%40example.com' + ) + ) + const { nextCursor } = await minted.json() + + mocks.list.mockClear() + const resumed = await listLogs( + new NextRequest( + `http://localhost:3000/api/v2/audit-logs?organizationId=org-1&actorEmail=ada%40example.com&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(resumed.status).toBe(200) + expect(mocks.list).toHaveBeenCalledWith({ + principal: auth.principal, + input: expect.objectContaining({ + filters: expect.objectContaining({ actorEmail: 'ada@example.com' }), + cursor: 'next-1', + }), + request: expect.anything(), + }) + }) + + /** + * `resourceType` is split into an `inArray` downstream, so its spelling is a + * set the query acts on rather than the exact string the caller sent. The + * cursor must bind the members, not the text. + */ + it.each([ + ['reordered', 'workflow,file'], + ['respaced', 'file,%20workflow'], + ['repeated', 'file,workflow,file'], + ])('resumes a cursor whose resourceType set is %s', async (_label, respelled) => { + const minted = await listLogs( + new NextRequest( + 'http://localhost:3000/api/v2/audit-logs?organizationId=org-1&resourceType=file,workflow' + ) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.list.mockClear() + const resumed = await listLogs( + new NextRequest( + `http://localhost:3000/api/v2/audit-logs?organizationId=org-1&resourceType=${respelled}&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(resumed.status).toBe(200) + expect(mocks.list).toHaveBeenCalled() + }) + + it('still refuses a cursor replayed under a different resourceType set', async () => { + const minted = await listLogs( + new NextRequest( + 'http://localhost:3000/api/v2/audit-logs?organizationId=org-1&resourceType=file,workflow' + ) + ) + const { nextCursor } = await minted.json() + + mocks.list.mockClear() + const replayed = await listLogs( + new NextRequest( + `http://localhost:3000/api/v2/audit-logs?organizationId=org-1&resourceType=file,knowledge&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.list).not.toHaveBeenCalled() + }) + it('projects typed admin-policy failures without leaking internals', async () => { mocks.list.mockRejectedValueOnce(new OrchestrationError('forbidden', 'Admin required')) diff --git a/apps/sim/app/api/v2/audit-logs/route.ts b/apps/sim/app/api/v2/audit-logs/route.ts index b9bc8743819..207251b3145 100644 --- a/apps/sim/app/api/v2/audit-logs/route.ts +++ b/apps/sim/app/api/v2/audit-logs/route.ts @@ -1,4 +1,10 @@ import { v2ListAuditLogsContract } from '@/lib/api/contracts/v2/audit-logs' +import { + cursorRoute, + cursorScopeKey, + instantScopePart, + unorderedScopePart, +} from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, @@ -8,6 +14,32 @@ import { import { listAuditLogs } from '@/lib/audit-logs/application/list-audit-logs' import { auditLogOperations } from '@/lib/audit-logs/application/operations' import { formatV2AuditLogEntry } from '@/app/api/v2/audit-logs/format' +import { encodeScopedCursor, readScopedCursor } from '@/app/api/v2/lib/response' + +/** Every param that changes which audit entries, in which order, this list returns. */ +function auditLogCursorFilters(query: { + organizationId: string + includeDeparted: boolean + action?: string + resourceType?: string + resourceId?: string + workspaceId?: string + actorEmail?: string + startDate?: string + endDate?: string +}) { + return cursorScopeKey(cursorRoute(v2ListAuditLogsContract), { + organizationId: query.organizationId, + includeDeparted: query.includeDeparted, + action: query.action, + resourceType: unorderedScopePart(query.resourceType), + resourceId: query.resourceId, + workspaceId: query.workspaceId, + actorEmail: query.actorEmail, + startDate: instantScopePart(query.startDate), + endDate: instantScopePart(query.endDate), + }) +} export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -38,11 +70,11 @@ export const GET = defineV2JsonRoute({ endDate: query.endDate, }, limit: query.limit, - cursor: query.cursor, + cursor: readScopedCursor(query.cursor, auditLogCursorFilters(query)), }), useCase: listAuditLogs, - present: ({ data, nextCursor }) => ({ + present: ({ data, nextCursor }, { query }) => ({ data: data.map(formatV2AuditLogEntry), - nextCursor: nextCursor ?? null, + nextCursor: nextCursor ? encodeScopedCursor(auditLogCursorFilters(query), nextCursor) : null, }), }) diff --git a/apps/sim/app/api/v2/billing/logs/route.test.ts b/apps/sim/app/api/v2/billing/logs/route.test.ts index 0c5e5f79387..c929a42215a 100644 --- a/apps/sim/app/api/v2/billing/logs/route.test.ts +++ b/apps/sim/app/api/v2/billing/logs/route.test.ts @@ -24,7 +24,20 @@ vi.mock('@/lib/billing/application/list-billing-logs', () => ({ listBillingLogs: { operation: { id: 'billing.logs.list' }, execute: mocks.execute }, })) +import { v2ListBillingLogsContract } from '@/lib/api/contracts/v2/billing' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' +import { UNKNOWN_CURSOR_MESSAGE } from '@/lib/billing/core/usage-log' +import { OrchestrationError } from '@/lib/core/orchestration/types' import { GET } from '@/app/api/v2/billing/logs/route' +import { encodeScopedCursor } from '@/app/api/v2/lib/response' + +/** A ledger cursor exactly as the route mints one, for the filters given. */ +function ledgerCursor( + inner: string, + filters: { source?: string; workspaceId?: string; period?: string } +): string { + return encodeScopedCursor(cursorScopeKey(cursorRoute(v2ListBillingLogsContract), filters), inner) +} const auth = { principal: { kind: 'personal_api_key' as const, userId: 'user-1', keyId: 'key-1' }, @@ -95,6 +108,94 @@ describe('GET /api/v2/billing/logs', () => { }) }) + it('projects an unresolvable cursor as a 400 rather than an unpositioned first page', async () => { + mocks.execute.mockRejectedValueOnce( + new OrchestrationError('validation', UNKNOWN_CURSOR_MESSAGE) + ) + const cursor = ledgerCursor('log-from-another-ledger', { period: '30d' }) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/billing/logs?cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: UNKNOWN_CURSOR_MESSAGE }, + }) + }) + + /** + * The ledger cursor is a usage-event id, so it names a row rather than an + * ordinal — but which rows follow it depends entirely on the window and source + * filters, so replaying one across a changed filter walks a different ledger + * and never reaches the entries the caller narrowed to. + */ + it('rejects a cursor replayed under a different filter without reaching the ledger', async () => { + const cursor = ledgerCursor('usage-1', { period: '30d' }) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/billing/logs?source=workflow&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('requested filters') }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + /** + * An empty inner token reads as falsy in the ledger reader, so no cursor + * condition is applied and the caller walks the first page again — the very + * failure {@link UNKNOWN_CURSOR_MESSAGE} exists to make visible. + */ + it('rejects a cursor whose inner token is empty instead of restarting at page one', async () => { + const cursor = ledgerCursor('', { period: 'all' }) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/billing/logs?period=all&limit=1&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + /** This operation takes neither param, so naming them sends the caller nowhere. */ + it('names the params a rejected cursor is actually bound to', async () => { + const response = await GET( + new NextRequest('http://localhost:3000/api/v2/billing/logs?cursor=not-a-cursor') + ) + + const body = await response.json() + expect(body.error.message).not.toContain('sortBy') + expect(body.error.message).not.toContain('sortOrder') + }) + + /** + * `0000` satisfies the published `\d{4}` date-time pattern but names no + * instant Postgres can store, so the value has to be refused before + * `resolveDateRange` turns it into a bind parameter. + */ + it('rejects a year-0000 custom range bound before it can reach the ledger', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/billing/logs?period=custom&startDate=${encodeURIComponent('0000-01-01T00:00:00Z')}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('startDate') }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + it('authenticates before rejecting invalid custom ranges', async () => { const response = await GET( new NextRequest('http://localhost:3000/api/v2/billing/logs?period=custom') @@ -104,4 +205,81 @@ describe('GET /api/v2/billing/logs', () => { expect(v2RouteMocks.authenticate).toHaveBeenCalled() expect(mocks.execute).not.toHaveBeenCalled() }) + + it('rejects a window bound the effective period would discard', async () => { + const response = await GET( + new NextRequest( + 'http://localhost:3000/api/v2/billing/logs?startDate=2030-01-01T00:00:00Z&limit=100' + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { + code: 'BAD_REQUEST', + message: expect.stringContaining('period=custom'), + }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('rejects an endDate paired with an explicit relative period', async () => { + const response = await GET( + new NextRequest( + 'http://localhost:3000/api/v2/billing/logs?period=7d&endDate=2026-07-01T00:00:00Z' + ) + ) + + expect(response.status).toBe(400) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('rejects a window bound that is not a UTC ISO 8601 timestamp', async () => { + const response = await GET( + new NextRequest( + 'http://localhost:3000/api/v2/billing/logs?period=custom&startDate=2026-08-01' + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('UTC ISO 8601') }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('rejects an inverted custom range instead of answering with an empty page', async () => { + const response = await GET( + new NextRequest( + 'http://localhost:3000/api/v2/billing/logs?period=custom&startDate=2026-08-06T00:00:00Z&endDate=2026-08-05T00:00:00Z' + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { + code: 'BAD_REQUEST', + message: expect.stringContaining('startDate must be before or equal to endDate'), + }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('forwards a valid custom range to the ledger read', async () => { + const response = await GET( + new NextRequest( + 'http://localhost:3000/api/v2/billing/logs?period=custom&startDate=2026-07-01T00:00:00Z&endDate=2026-07-31T00:00:00Z' + ) + ) + + expect(response.status).toBe(200) + expect(mocks.execute).toHaveBeenCalledWith({ + principal: auth.principal, + input: expect.objectContaining({ + startDate: new Date('2026-07-01T00:00:00Z'), + endDate: new Date('2026-07-31T00:00:00Z'), + }), + request: expect.anything(), + }) + }) }) diff --git a/apps/sim/app/api/v2/billing/logs/route.ts b/apps/sim/app/api/v2/billing/logs/route.ts index 03d02a8d715..61f1e4acd80 100644 --- a/apps/sim/app/api/v2/billing/logs/route.ts +++ b/apps/sim/app/api/v2/billing/logs/route.ts @@ -1,25 +1,53 @@ import { v2ListBillingLogsContract } from '@/lib/api/contracts/v2/billing' -import { - defineV2JsonRoute, - v2ApiKeyAuth, - v2OrchestrationErrorPolicy, - v2RateLimits, -} from '@/lib/api/server/routes' +import { cursorRoute, cursorScopeKey, instantScopePart } from '@/lib/api/cursor-binding' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2BillingErrorPolicies } from '@/lib/billing/api/route-policies' import { listBillingLogs } from '@/lib/billing/application/list-billing-logs' import { billingOperations } from '@/lib/billing/application/operations' import { toBillingUsageLogSource, toInternalUsageLogSources } from '@/lib/billing/usage-sources' import { resolveDateRange } from '@/app/api/users/me/usage-logs/shared' +import { encodeScopedCursor, readScopedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** + * Every param that changes which ledger entries, in which order, this list + * returns. + * + * The raw params are stamped, not the range `resolveDateRange` derives from + * them: a relative `period` resolves against the clock, so hashing the resolved + * window would produce a different stamp on every request and reject each next + * page. `period=30d` and an explicit custom range covering the same days are + * therefore two scopes, which is right — one is a moving window. + * + * The explicit bounds still bind by instant rather than spelling. That is a + * pure function of the caller's own text, so it collapses `…00Z` and `…00.000Z` + * without resolving anything against the clock. + */ +function billingLogCursorFilters(query: { + source?: string + workspaceId?: string + period?: string + startDate?: string + endDate?: string +}) { + return cursorScopeKey(cursorRoute(v2ListBillingLogsContract), { + source: query.source, + workspaceId: query.workspaceId, + period: query.period, + startDate: instantScopePart(query.startDate), + endDate: instantScopePart(query.endDate), + }) +} + /** Cursor-paged, credit-denominated billing ledger. */ export const GET = defineV2JsonRoute({ contract: v2ListBillingLogsContract, auth: v2ApiKeyAuth, operation: billingOperations.listLogs, rateLimit: v2RateLimits.publicApi, - errorPolicy: v2OrchestrationErrorPolicy, + errorPolicy: v2BillingErrorPolicies.concealWorkspaceAuthorization, mapInput: ({ query }) => { const dateRange = resolveDateRange(query.period, query.startDate, query.endDate) return { @@ -28,11 +56,11 @@ export const GET = defineV2JsonRoute({ startDate: dateRange.startDate, endDate: dateRange.endDate, limit: query.limit, - cursor: query.cursor, + cursor: readScopedCursor(query.cursor, billingLogCursorFilters(query)), } }, useCase: listBillingLogs, - present: ({ usage, creditsByLogId }) => ({ + present: ({ usage, creditsByLogId }, { query }) => ({ data: usage.logs.map((log) => ({ id: log.id, createdAt: log.createdAt, @@ -42,6 +70,9 @@ export const GET = defineV2JsonRoute({ runId: log.executionId ?? null, creditCost: creditsByLogId[log.id] ?? 0, })), - nextCursor: usage.pagination.hasMore ? (usage.pagination.nextCursor ?? null) : null, + nextCursor: + usage.pagination.hasMore && usage.pagination.nextCursor + ? encodeScopedCursor(billingLogCursorFilters(query), usage.pagination.nextCursor) + : null, }), }) diff --git a/apps/sim/app/api/v2/billing/status/route.test.ts b/apps/sim/app/api/v2/billing/status/route.test.ts index e664f896b36..bc82837a17b 100644 --- a/apps/sim/app/api/v2/billing/status/route.test.ts +++ b/apps/sim/app/api/v2/billing/status/route.test.ts @@ -24,7 +24,10 @@ vi.mock('@/lib/billing/application/get-billing-status', () => ({ getBillingStatus: { operation: { id: 'billing.status.read' }, execute: mocks.execute }, })) -import { OrchestrationError } from '@/lib/core/orchestration/types' +import { + PersonalApiKeysDisabledError, + WorkspaceApiKeyScopeAuthorizationError, +} from '@/lib/core/application' import { GET } from '@/app/api/v2/billing/status/route' const auth = { @@ -80,17 +83,47 @@ describe('GET /api/v2/billing/status', () => { expect(await response.json()).toEqual({ data: { ...result, credits: null, storage: null } }) }) - it('projects typed workspace-policy errors', async () => { - mocks.execute.mockRejectedValueOnce( - new OrchestrationError('forbidden', 'API key is not authorized for this workspace') + it.each(['workspaceID', 'workspace_id', 'workspace'])( + 'rejects %s rather than silently answering for the account payer', + async (key) => { + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/billing/status?${key}=workspace-1`) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ error: { code: 'BAD_REQUEST' } }) + expect(mocks.execute).not.toHaveBeenCalled() + } + ) + + it('names the cause of an actionable workspace-policy refusal', async () => { + mocks.execute.mockRejectedValueOnce(new PersonalApiKeysDisabledError()) + + const response = await GET( + new NextRequest('http://localhost:3000/api/v2/billing/status?workspaceId=workspace-1') ) + expect(response.status).toBe(403) + expect(await response.json()).toMatchObject({ + error: { code: 'FORBIDDEN', details: { code: 'PERSONAL_API_KEYS_DISABLED' } }, + }) + }) + + /** + * A workspace key naming another workspace must not learn that the workspace + * exists, so this refusal is answered exactly as an unknown workspace id is. + */ + it('conceals a cross-tenant workspace-key refusal as a not-found workspace', async () => { + mocks.execute.mockRejectedValueOnce(new WorkspaceApiKeyScopeAuthorizationError()) + const response = await GET( new NextRequest('http://localhost:3000/api/v2/billing/status?workspaceId=workspace-2') ) - expect(response.status).toBe(403) - expect(await response.json()).toMatchObject({ error: { code: 'FORBIDDEN' } }) + expect(response.status).toBe(404) + expect(await response.json()).toMatchObject({ + error: { code: 'NOT_FOUND', message: 'Workspace not found' }, + }) }) it('hides unknown billing infrastructure errors', async () => { diff --git a/apps/sim/app/api/v2/billing/status/route.ts b/apps/sim/app/api/v2/billing/status/route.ts index b5a7fd95b5f..50c7e8bd95d 100644 --- a/apps/sim/app/api/v2/billing/status/route.ts +++ b/apps/sim/app/api/v2/billing/status/route.ts @@ -1,10 +1,6 @@ import { v2GetBillingStatusContract } from '@/lib/api/contracts/v2/billing' -import { - defineV2JsonRoute, - v2ApiKeyAuth, - v2OrchestrationErrorPolicy, - v2RateLimits, -} from '@/lib/api/server/routes' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2BillingErrorPolicies } from '@/lib/billing/api/route-policies' import { getBillingStatus } from '@/lib/billing/application/get-billing-status' import { billingOperations } from '@/lib/billing/application/operations' @@ -17,7 +13,7 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, operation: billingOperations.readStatus, rateLimit: v2RateLimits.publicApi, - errorPolicy: v2OrchestrationErrorPolicy, + errorPolicy: v2BillingErrorPolicies.concealWorkspaceAuthorization, mapInput: ({ query }) => ({ workspaceId: query.workspaceId }), useCase: getBillingStatus, present: (data) => ({ data }), diff --git a/apps/sim/app/api/v2/credentials/route.test.ts b/apps/sim/app/api/v2/credentials/route.test.ts index a1987f1b5af..465d2cbd6be 100644 --- a/apps/sim/app/api/v2/credentials/route.test.ts +++ b/apps/sim/app/api/v2/credentials/route.test.ts @@ -27,6 +27,8 @@ vi.mock('@/lib/credentials/application/list-workspace-credentials', () => ({ }, })) +import { V2_DEFAULT_PAGE_SIZE } from '@/lib/api/contracts/v2/shared' +import { REFILTERED_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { GET } from '@/app/api/v2/credentials/route' const WORKSPACE_ID = '11111111-2222-4333-8444-555555555555' @@ -65,7 +67,12 @@ describe('GET /api/v2/credentials', () => { v2RouteMocks.gate.mockResolvedValue(null) v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) - mocks.execute.mockResolvedValue({ credentials: [credential] }) + mocks.execute.mockResolvedValue({ + credentials: [credential], + nextCursorKeys: null, + sortBy: 'createdAt', + sortOrder: 'desc', + }) }) it('authenticates and charges before validating workspace input', async () => { @@ -93,11 +100,80 @@ describe('GET /api/v2/credentials', () => { search: undefined, sortBy: 'createdAt', sortOrder: 'desc', + limit: V2_DEFAULT_PAGE_SIZE, + cursor: undefined, + cursorKeys: undefined, }, request, }) }) + /** + * Pins the binding end-to-end — the mint in `present` and the read in + * `mapInput` — because the contract-level sweep only checks a hand-maintained + * map of param names and stays green when a route drops the stamp entirely. + */ + it('refuses a cursor minted under a different filter', async () => { + mocks.execute.mockResolvedValue({ + credentials: [credential], + nextCursorKeys: ['2026-01-01T00:00:00.000Z', 'credential-1'], + sortBy: 'createdAt', + sortOrder: 'desc', + }) + + const minted = await GET( + new NextRequest( + `http://localhost:3000/api/v2/credentials?workspaceId=${WORKSPACE_ID}&search=zoom` + ) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.execute.mockClear() + const replayed = await GET( + new NextRequest( + `http://localhost:3000/api/v2/credentials?workspaceId=${WORKSPACE_ID}&search=slack&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('resumes a cursor replayed under the filters it was minted with', async () => { + mocks.execute.mockResolvedValue({ + credentials: [credential], + nextCursorKeys: ['2026-01-01T00:00:00.000Z', 'credential-1'], + sortBy: 'createdAt', + sortOrder: 'desc', + }) + + const minted = await GET( + new NextRequest( + `http://localhost:3000/api/v2/credentials?workspaceId=${WORKSPACE_ID}&search=zoom` + ) + ) + const { nextCursor } = await minted.json() + + mocks.execute.mockClear() + const resumed = await GET( + new NextRequest( + `http://localhost:3000/api/v2/credentials?workspaceId=${WORKSPACE_ID}&search=zoom&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(resumed.status).toBe(200) + expect(mocks.execute).toHaveBeenCalledWith({ + principal: auth.principal, + input: expect.objectContaining({ + search: 'zoom', + cursorKeys: ['2026-01-01T00:00:00.000Z', 'credential-1'], + }), + request: expect.anything(), + }) + }) + it('projects credential metadata field by field without secret material', async () => { const response = await GET( new NextRequest(`http://localhost:3000/api/v2/credentials?workspaceId=${WORKSPACE_ID}`) diff --git a/apps/sim/app/api/v2/credentials/route.ts b/apps/sim/app/api/v2/credentials/route.ts index 0312ea3a957..b0dde7a39ab 100644 --- a/apps/sim/app/api/v2/credentials/route.ts +++ b/apps/sim/app/api/v2/credentials/route.ts @@ -1,4 +1,6 @@ +import type { V2Credential } from '@/lib/api/contracts/v2/credentials' import { v2ListCredentialsContract } from '@/lib/api/contracts/v2/credentials' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, @@ -7,11 +9,47 @@ import { } from '@/lib/api/server/routes' import { listWorkspaceCredentials } from '@/lib/credentials/application/list-workspace-credentials' import { credentialOperations } from '@/lib/credentials/application/operations' -import { toV2Credential } from '@/app/api/v2/credentials/utils' +import type { VisibleWorkspaceCredential } from '@/lib/credentials/queries' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Serialize connection metadata field by field so encrypted columns can never reach the wire. */ +function toV2Credential(row: VisibleWorkspaceCredential): V2Credential { + if (row.type !== 'oauth' && row.type !== 'service_account') { + throw new Error(`Secret credential type ${row.type} reached the credentials API`) + } + + return { + id: row.id, + type: row.type, + displayName: row.displayName, + description: row.description, + providerId: row.providerId, + accountId: row.accountId, + hasServiceAccountKey: row.hasServiceAccountKey, + role: row.role, + createdAt: row.createdAt.toISOString(), + updatedAt: row.updatedAt.toISOString(), + } +} + +/** Every param that changes which credentials, in which order, this list returns. */ +function credentialCursorFilters(query: { + workspaceId: string + type?: string + providerId?: string + search?: string +}) { + return cursorScopeKey(cursorRoute(v2ListCredentialsContract), { + workspaceId: query.workspaceId, + type: query.type, + providerId: query.providerId, + search: query.search, + }) +} + /** GET /api/v2/credentials — List the credentials the caller can see in a workspace. */ export const GET = defineV2JsonRoute({ contract: v2ListCredentialsContract, @@ -19,10 +57,23 @@ export const GET = defineV2JsonRoute({ operation: credentialOperations.listConnections, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => ({ + ...query, + cursorKeys: readSortedCursor( + query.cursor, + query.sortBy, + query.sortOrder, + credentialCursorFilters(query) + ), + }), useCase: listWorkspaceCredentials, - present: ({ credentials }) => ({ + present: ({ credentials, nextCursorKeys }, { query }) => ({ data: credentials.map(toV2Credential), - nextCursor: null, + nextCursor: writeSortedCursor( + nextCursorKeys, + query.sortBy, + query.sortOrder, + credentialCursorFilters(query) + ), }), }) diff --git a/apps/sim/app/api/v2/credentials/utils.ts b/apps/sim/app/api/v2/credentials/utils.ts deleted file mode 100644 index e186a4f1558..00000000000 --- a/apps/sim/app/api/v2/credentials/utils.ts +++ /dev/null @@ -1,22 +0,0 @@ -import type { V2Credential } from '@/lib/api/contracts/v2/credentials' -import type { VisibleWorkspaceCredential } from '@/lib/credentials/queries' - -/** Serialize connection metadata field by field so encrypted columns can never reach the wire. */ -export function toV2Credential(row: VisibleWorkspaceCredential): V2Credential { - if (row.type !== 'oauth' && row.type !== 'service_account') { - throw new Error(`Secret credential type ${row.type} reached the credentials API`) - } - - return { - id: row.id, - type: row.type, - displayName: row.displayName, - description: row.description, - providerId: row.providerId, - accountId: row.accountId, - hasServiceAccountKey: row.hasServiceAccountKey, - role: row.role, - createdAt: row.createdAt.toISOString(), - updatedAt: row.updatedAt.toISOString(), - } -} diff --git a/apps/sim/app/api/v2/custom-tools/[id]/route.test.ts b/apps/sim/app/api/v2/custom-tools/[id]/route.test.ts index b55abafc968..ca4784712e4 100644 --- a/apps/sim/app/api/v2/custom-tools/[id]/route.test.ts +++ b/apps/sim/app/api/v2/custom-tools/[id]/route.test.ts @@ -93,18 +93,22 @@ const tool = { } const context = { params: Promise.resolve({ id: tool.id }) } +/** + * The read and delete verbs scope themselves with `?workspaceId=`; the write + * verb carries `workspaceId` in its body. Sending the query copy on a write is + * now a 400 rather than a silently dropped key, so the helper only appends it + * where the contract declares it. + */ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { - return new NextRequest( - `http://localhost:3000/api/v2/custom-tools/${tool.id}?workspaceId=${WORKSPACE_ID}`, - { - method, - headers: { - 'x-api-key': 'key', - ...(body === undefined ? {} : { 'Content-Type': 'application/json' }), - }, - ...(body === undefined ? {} : { body: JSON.stringify(body) }), - } - ) + const query = method === 'PATCH' ? '' : `?workspaceId=${WORKSPACE_ID}` + return new NextRequest(`http://localhost:3000/api/v2/custom-tools/${tool.id}${query}`, { + method, + headers: { + 'x-api-key': 'key', + ...(body === undefined ? {} : { 'Content-Type': 'application/json' }), + }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }), + }) } describe('/api/v2/custom-tools/[id]', () => { @@ -130,6 +134,25 @@ describe('/api/v2/custom-tools/[id]', () => { }) }) + /** + * Every list in this family rejects a query param it does not implement, so + * the single-resource reads must too. A caller who mistypes a flag otherwise + * gets a 200 that silently ignored it, which reads as confirmation the flag + * exists and does nothing. + */ + it('rejects a query param it does not implement', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/custom-tools/${tool.id}?workspaceId=${WORKSPACE_ID}&includeCodes=true`, + { method: 'GET', headers: { 'x-api-key': 'key' } } + ), + context + ) + + expect(response.status).toBe(400) + expect(mocks.get).not.toHaveBeenCalled() + }) + it('updates a custom tool through its semantic update operation', async () => { const response = await PATCH( request('PATCH', { workspaceId: WORKSPACE_ID, code: 'return 2' }), diff --git a/apps/sim/app/api/v2/custom-tools/route.test.ts b/apps/sim/app/api/v2/custom-tools/route.test.ts index f0d113e8961..3b32d07ccab 100644 --- a/apps/sim/app/api/v2/custom-tools/route.test.ts +++ b/apps/sim/app/api/v2/custom-tools/route.test.ts @@ -54,6 +54,8 @@ vi.mock('@/lib/custom-tools/application/use-cases', () => ({ }, })) +import { V2_DEFAULT_PAGE_SIZE } from '@/lib/api/contracts/v2/shared' +import { REFILTERED_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { GET, POST } from '@/app/api/v2/custom-tools/route' const WORKSPACE_ID = 'workspace-1' @@ -121,9 +123,10 @@ describe('/api/v2/custom-tools', () => { principal: PRINCIPAL, input: { workspaceId: WORKSPACE_ID, - search: undefined, sortBy: 'createdAt', sortOrder: 'desc', + limit: V2_DEFAULT_PAGE_SIZE, + cursorKeys: undefined, }, request: expect.anything(), }) @@ -133,6 +136,66 @@ describe('/api/v2/custom-tools', () => { ) }) + /** + * Pins the binding end-to-end — the mint in `present` and the read in + * `mapInput` — because the contract-level sweep only checks a hand-maintained + * map of param names and stays green when a route drops the stamp entirely. + */ + it('refuses a cursor minted under a different filter', async () => { + mocks.list.mockResolvedValue({ + tools: [tool], + nextCursorKeys: ['2026-01-01T00:00:00.000Z', 'tool-1'], + }) + + const minted = await GET( + request('GET', `/api/v2/custom-tools?workspaceId=${WORKSPACE_ID}&search=lookup`) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.list.mockClear() + const replayed = await GET( + request( + 'GET', + `/api/v2/custom-tools?workspaceId=${WORKSPACE_ID}&search=refund&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.list).not.toHaveBeenCalled() + }) + + it('resumes a cursor replayed under the filters it was minted with', async () => { + mocks.list.mockResolvedValue({ + tools: [tool], + nextCursorKeys: ['2026-01-01T00:00:00.000Z', 'tool-1'], + }) + + const minted = await GET( + request('GET', `/api/v2/custom-tools?workspaceId=${WORKSPACE_ID}&search=lookup`) + ) + const { nextCursor } = await minted.json() + + mocks.list.mockClear() + const resumed = await GET( + request( + 'GET', + `/api/v2/custom-tools?workspaceId=${WORKSPACE_ID}&search=lookup&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(resumed.status).toBe(200) + expect(mocks.list).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: expect.objectContaining({ + search: 'lookup', + cursorKeys: ['2026-01-01T00:00:00.000Z', 'tool-1'], + }), + request: expect.anything(), + }) + }) + it('creates exactly one custom tool with the v2 source and status', async () => { const response = await POST( request('POST', '/api/v2/custom-tools', { diff --git a/apps/sim/app/api/v2/custom-tools/route.ts b/apps/sim/app/api/v2/custom-tools/route.ts index 0da00d8ce8f..4cbfafb5492 100644 --- a/apps/sim/app/api/v2/custom-tools/route.ts +++ b/apps/sim/app/api/v2/custom-tools/route.ts @@ -2,6 +2,7 @@ import { v2CreateCustomToolContract, v2ListCustomToolsContract, } from '@/lib/api/contracts/v2/custom-tools' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, @@ -14,10 +15,19 @@ import { listWorkspaceCustomToolsUseCase, } from '@/lib/custom-tools/application/use-cases' import { toV2CustomTool } from '@/app/api/v2/custom-tools/utils' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Every param that changes which custom tools, in which order, this list returns. */ +function customToolCursorFilters(query: { workspaceId: string; search?: string }) { + return cursorScopeKey(cursorRoute(v2ListCustomToolsContract), { + workspaceId: query.workspaceId, + search: query.search, + }) +} + /** GET /api/v2/custom-tools — List custom tools in a workspace. */ export const GET = defineV2JsonRoute({ contract: v2ListCustomToolsContract, @@ -25,9 +35,25 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => ({ + ...query, + cursorKeys: readSortedCursor( + query.cursor, + query.sortBy, + query.sortOrder, + customToolCursorFilters(query) + ), + }), useCase: listWorkspaceCustomToolsUseCase, - present: ({ tools }) => ({ data: tools.map(toV2CustomTool), nextCursor: null }), + present: ({ tools, nextCursorKeys }, { query }) => ({ + data: tools.map(toV2CustomTool), + nextCursor: writeSortedCursor( + nextCursorKeys, + query.sortBy, + query.sortOrder, + customToolCursorFilters(query) + ), + }), }) /** POST /api/v2/custom-tools — Create a custom tool. */ diff --git a/apps/sim/app/api/v2/custom-tools/utils.ts b/apps/sim/app/api/v2/custom-tools/utils.ts index 516101065ad..d3f1181a10e 100644 --- a/apps/sim/app/api/v2/custom-tools/utils.ts +++ b/apps/sim/app/api/v2/custom-tools/utils.ts @@ -1,33 +1,8 @@ import type { customTools } from '@sim/db/schema' -import { getErrorMessage, getPostgresErrorCode } from '@sim/utils/errors' -import type { NextResponse } from 'next/server' import type { V2CustomTool } from '@/lib/api/contracts/v2/custom-tools' -import { v2Error } from '@/app/api/v2/lib/response' /** Shared serialization + error mapping for the v2 custom tool surface. */ -/** - * Classifies a title collision as a conflict so it surfaces as 409 rather than a - * generic 500. Two distinct failures reach here and both must be covered: - * - * - `upsertCustomTools` throws its own message when its in-transaction duplicate - * `SELECT` finds one. - * - Under a concurrent create or rename, both callers pass that `SELECT` too, and - * the loser is rejected by `custom_tools_workspace_title_unique` as a raw - * Postgres `23505` — whose message matches nothing, which is exactly the race - * the message check alone cannot see. - */ -export function v2CustomToolWriteError(error: unknown): NextResponse | null { - if (getPostgresErrorCode(error) === '23505') { - return v2Error('CONFLICT', 'A custom tool with that title already exists in this workspace') - } - const message = getErrorMessage(error, '') - if (/already exists in this workspace/i.test(message)) { - return v2Error('CONFLICT', message) - } - return null -} - type CustomToolRow = typeof customTools.$inferSelect /** diff --git a/apps/sim/app/api/v2/files/[fileId]/content/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/content/route.test.ts index acef3a88a5f..db308de20b6 100644 --- a/apps/sim/app/api/v2/files/[fileId]/content/route.test.ts +++ b/apps/sim/app/api/v2/files/[fileId]/content/route.test.ts @@ -1,15 +1,21 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ admit: vi.fn(), updateContent: vi.fn(), - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), getUserEmailsByIds: vi.fn(), })) @@ -25,22 +31,9 @@ vi.mock('@/lib/workspace-files/application/update-workspace-file-content', () => }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ - v2ApiGateError: vi.fn().mockResolvedValue(null), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/users/queries', () => ({ getUserEmailsByIds: mocks.getUserEmailsByIds, @@ -94,17 +87,10 @@ const callPut = (body: unknown, contentLength?: number) => describe('PUT /api/v2/files/[fileId]/content', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.admit.mockResolvedValue(undefined) mocks.updateContent.mockResolvedValue({ file: record }) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'ada@example.com']])) @@ -120,6 +106,15 @@ describe('PUT /api/v2/files/[fileId]/content', () => { expect(mocks.updateContent).not.toHaveBeenCalled() }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await callPut({ workspaceId: WORKSPACE_ID, content: 'id,name\n' }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('validates body fields after admission', async () => { const response = await callPut({ workspaceId: WORKSPACE_ID }) @@ -159,6 +154,7 @@ describe('PUT /api/v2/files/[fileId]/content', () => { uploadedByEmail: 'ada@example.com', uploadedAt: '2024-01-01T00:00:00.000Z', updatedAt: '2024-01-03T00:00:00.000Z', + deletedAt: null, }, }) expect(mocks.updateContent).toHaveBeenCalledWith({ @@ -171,7 +167,7 @@ describe('PUT /api/v2/files/[fileId]/content', () => { }, request, }) - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledWith( + expect(v2RouteMocks.operationRate).toHaveBeenCalledWith( 'v2:files.update_content:api-key:key-1', expect.anything() ) diff --git a/apps/sim/app/api/v2/files/[fileId]/content/route.ts b/apps/sim/app/api/v2/files/[fileId]/content/route.ts index 48d35dd94b7..c2a64201d23 100644 --- a/apps/sim/app/api/v2/files/[fileId]/content/route.ts +++ b/apps/sim/app/api/v2/files/[fileId]/content/route.ts @@ -8,7 +8,6 @@ import { } from '@/lib/workspace-files/application/update-workspace-file-content' import { MAX_WORKSPACE_FILE_INLINE_BODY_BYTES } from '@/lib/workspace-files/orchestration' import { toV2File } from '@/app/api/v2/files/utils' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -21,7 +20,6 @@ export const PUT = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2FileErrorPolicies.concealResourceAuthorization, parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), maxBodyBytes: MAX_WORKSPACE_FILE_INLINE_BODY_BYTES, }, beforeParse: async ({ principal, params }) => { diff --git a/apps/sim/app/api/v2/files/[fileId]/metadata/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/metadata/route.test.ts index 3c26fadfaad..c947dff15a9 100644 --- a/apps/sim/app/api/v2/files/[fileId]/metadata/route.test.ts +++ b/apps/sim/app/api/v2/files/[fileId]/metadata/route.test.ts @@ -1,14 +1,20 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ readMetadata: vi.fn(), - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), getUserEmailsByIds: vi.fn(), })) @@ -19,22 +25,9 @@ vi.mock('@/lib/workspace-files/application/read-workspace-file-metadata', () => }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ - v2ApiGateError: vi.fn().mockResolvedValue(null), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/users/queries', () => ({ getUserEmailsByIds: mocks.getUserEmailsByIds, @@ -89,17 +82,10 @@ const callGet = (query: string) => describe('GET /api/v2/files/[fileId]/metadata', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.readMetadata.mockResolvedValue({ file: buildRecord(), share: SHARE }) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'ada@example.com']])) }) @@ -108,11 +94,20 @@ describe('GET /api/v2/files/[fileId]/metadata', () => { const response = await callGet('') expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalled() - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.readMetadata).not.toHaveBeenCalled() }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await callGet(`workspaceId=${WORKSPACE_ID}`) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('conceals cross-workspace authorization as not found', async () => { mocks.readMetadata.mockRejectedValue(new NoWorkspaceAccessError()) @@ -137,6 +132,7 @@ describe('GET /api/v2/files/[fileId]/metadata', () => { uploadedByEmail: 'ada@example.com', uploadedAt: '2024-01-01T00:00:00.000Z', updatedAt: '2024-01-02T00:00:00.000Z', + deletedAt: null, share: SHARE, }, }) diff --git a/apps/sim/app/api/v2/files/[fileId]/restore/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/restore/route.test.ts new file mode 100644 index 00000000000..926e0166a28 --- /dev/null +++ b/apps/sim/app/api/v2/files/[fileId]/restore/route.test.ts @@ -0,0 +1,156 @@ +/** + * @vitest-environment node + */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + restoreFile: vi.fn(), + getUserEmailsByIds: vi.fn(), +})) + +vi.mock('@/lib/workspace-files/application/restore-workspace-file', () => ({ + restoreWorkspaceFileOperation: { + operation: { id: 'files.restore', minimumRole: 'write', workspaceApiKey: 'allow' }, + execute: mocks.restoreFile, + }, +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +vi.mock('@/lib/users/queries', () => ({ + getUserEmailsByIds: mocks.getUserEmailsByIds, + requireResolvedUserEmail: (emails: Map, userId: string) => emails.get(userId)!, +})) + +import { OrchestrationError } from '@/lib/core/orchestration/types' +import { POST } from '@/app/api/v2/files/[fileId]/restore/route' + +const WORKSPACE_ID = 'workspace-1' +const FILE_ID = 'wf_1' + +const auth = { + principal: { + kind: 'workspace_api_key' as const, + workspaceId: WORKSPACE_ID, + keyId: 'key-1', + }, + rolloutUserId: 'billing-owner-1', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, + rateLimitSubscription: null, + keyType: 'workspace' as const, +} + +/** The post-restore record: renamed away from the taken name, back at the root. */ +const RESTORED_FILE = { + id: FILE_ID, + workspaceId: WORKSPACE_ID, + name: 'notes_restored.md', + key: `workspace/${WORKSPACE_ID}/notes.md`, + path: '/api/files/serve/notes.md?context=workspace', + size: 12, + type: 'text/markdown', + uploadedBy: 'user-1', + folderId: null, + folderPath: null, + deletedAt: null, + uploadedAt: new Date('2026-08-04T00:00:00.000Z'), + updatedAt: new Date('2026-08-07T00:00:00.000Z'), +} + +function restoreRequest(body: unknown): NextRequest { + return new NextRequest(`http://localhost:3000/api/v2/files/${FILE_ID}/restore`, { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify(body), + }) +} + +function post(body: unknown) { + return POST(restoreRequest(body), { params: Promise.resolve({ fileId: FILE_ID }) }) +} + +describe('POST /api/v2/files/[fileId]/restore', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.restoreFile.mockResolvedValue({ restored: true, file: RESTORED_FILE }) + mocks.getUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'ada@example.com']])) + }) + + it('returns the post-restore record so the caller sees the new name and root placement', async () => { + const response = await post({ workspaceId: WORKSPACE_ID }) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { + id: FILE_ID, + name: 'notes_restored.md', + size: 12, + type: 'text/markdown', + key: RESTORED_FILE.key, + folderPath: '/', + uploadedByEmail: 'ada@example.com', + uploadedAt: '2026-08-04T00:00:00.000Z', + updatedAt: '2026-08-07T00:00:00.000Z', + deletedAt: null, + }, + }) + expect(mocks.restoreFile).toHaveBeenCalledWith({ + principal: auth.principal, + input: { fileId: FILE_ID, assertedWorkspaceId: WORKSPACE_ID }, + request: expect.anything(), + }) + }) + + it('conceals a file in another workspace as 404 rather than confirming it exists', async () => { + mocks.restoreFile.mockRejectedValueOnce(new OrchestrationError('not_found', 'File not found')) + + const response = await post({ workspaceId: WORKSPACE_ID }) + + expect(response.status).toBe(404) + expect((await response.json()).error).toMatchObject({ + code: 'NOT_FOUND', + message: 'File not found', + }) + }) + + it('rejects an unknown body key instead of ignoring it', async () => { + const response = await post({ workspaceId: WORKSPACE_ID, folderPath: '/Engineering' }) + + expect(response.status).toBe(400) + expect(mocks.restoreFile).not.toHaveBeenCalled() + }) + + it('authenticates and charges before validating the body', async () => { + const response = await post({}) + + expect(response.status).toBe(400) + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) + expect(mocks.restoreFile).not.toHaveBeenCalled() + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await post({ workspaceId: WORKSPACE_ID }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) +}) diff --git a/apps/sim/app/api/v2/files/[fileId]/restore/route.ts b/apps/sim/app/api/v2/files/[fileId]/restore/route.ts new file mode 100644 index 00000000000..050a49e0ccf --- /dev/null +++ b/apps/sim/app/api/v2/files/[fileId]/restore/route.ts @@ -0,0 +1,35 @@ +import { v2RestoreFileContract } from '@/lib/api/contracts/v2/files' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2FileErrorPolicies } from '@/lib/workspace-files/api' +import { fileOperations } from '@/lib/workspace-files/application/operations' +import { restoreWorkspaceFileOperation } from '@/lib/workspace-files/application/restore-workspace-file' +import { toV2File } from '@/app/api/v2/files/utils' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * POST /api/v2/files/[fileId]/restore — Bring an archived file back. + * + * `DELETE /api/v2/files/[fileId]` is a soft delete; this reverses it. Find the + * ids to pass here with `GET /api/v2/files?scope=archived`. + * + * Restore is not a pure undo: the file returns to the workspace root regardless + * of the folder it was deleted from, and it is renamed when its original name + * is no longer free. The response is therefore the post-restore record, not the + * one the caller deleted. Restoring an already-active file is a no-op that + * returns that file, so a retried request is safe. + */ +export const POST = defineV2JsonRoute({ + contract: v2RestoreFileContract, + auth: v2ApiKeyAuth, + operation: fileOperations.restore, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2FileErrorPolicies.concealResourceAuthorization, + mapInput: ({ params, body }) => ({ + fileId: params.fileId, + assertedWorkspaceId: body.workspaceId, + }), + useCase: restoreWorkspaceFileOperation, + present: async ({ file }) => ({ data: await toV2File(file) }), +}) diff --git a/apps/sim/app/api/v2/files/[fileId]/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/route.test.ts index a63dbb5c50a..973435bcc52 100644 --- a/apps/sim/app/api/v2/files/[fileId]/route.test.ts +++ b/apps/sim/app/api/v2/files/[fileId]/route.test.ts @@ -1,16 +1,23 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ download: vi.fn(), + authorizeDownload: vi.fn(), rename: vi.fn(), deleteFile: vi.fn(), - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), getUserEmailsByIds: vi.fn(), })) @@ -18,6 +25,7 @@ vi.mock('@/lib/workspace-files/application/download-workspace-file', () => ({ downloadWorkspaceFileStream: { operation: { id: 'files.download', minimumRole: 'read', workspaceApiKey: 'allow' }, execute: mocks.download, + authorize: mocks.authorizeDownload, }, })) @@ -35,20 +43,9 @@ vi.mock('@/lib/workspace-files/application/delete-workspace-file', () => ({ }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: vi.fn().mockResolvedValue(null) })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/users/queries', () => ({ getUserEmailsByIds: mocks.getUserEmailsByIds, @@ -77,6 +74,12 @@ const auth = { keyType: 'workspace' as const, } +function headRequest(query: string): NextRequest { + return new NextRequest(`http://localhost:3000/api/v2/files/${FILE_ID}?${query}`, { + method: 'HEAD', + }) +} + function fileRecord(overrides: Record = {}) { return { id: FILE_ID, @@ -97,17 +100,10 @@ function fileRecord(overrides: Record = {}) { describe('v2 single-file routes', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2024-01-01T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.download.mockResolvedValue({ file: fileRecord(), stream: new Blob(['id,name\n']).stream(), @@ -121,6 +117,49 @@ describe('v2 single-file routes', () => { deleted: true, }) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'ada@example.com']])) + mocks.authorizeDownload.mockResolvedValue(undefined) + }) + + /** + * A download `HEAD` answered before the use case's workspace-scoped file + * resolution is an existence oracle: any valid API key draws a bodiless 200 + * for a file id whose `GET` answers 404. These pin the probe to the answer the + * download gives, and to still not auditing one. + */ + it('answers an authorized HEAD bodiless without auditing a download', async () => { + const response = await GET(headRequest(`workspaceId=${WORKSPACE_ID}`), context) + + expect(response.status).toBe(200) + expect(await response.text()).toBe('') + expect(mocks.download).not.toHaveBeenCalled() + expect(mocks.authorizeDownload).toHaveBeenCalledOnce() + }) + + it('does not confirm a file the caller cannot reach', async () => { + mocks.authorizeDownload.mockRejectedValueOnce(new NoWorkspaceAccessError()) + + const response = await GET(headRequest('workspaceId=someone-elses-workspace'), context) + + expect(response.status).toBe(404) + expect(mocks.download).not.toHaveBeenCalled() + }) + + it('does not confirm a file id that does not exist', async () => { + mocks.authorizeDownload.mockRejectedValueOnce( + new OrchestrationError('not_found', 'File not found') + ) + + const response = await GET(headRequest(`workspaceId=${WORKSPACE_ID}`), context) + + expect(response.status).toBe(404) + expect(mocks.download).not.toHaveBeenCalled() + }) + + it('rejects a HEAD missing the required workspaceId instead of answering 200', async () => { + const response = await GET(headRequest(''), context) + + expect(response.status).toBe(400) + expect(mocks.authorizeDownload).not.toHaveBeenCalled() }) it('downloads bytes through the binary adapter with operation rate headers', async () => { @@ -141,6 +180,18 @@ describe('v2 single-file routes', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/files/${FILE_ID}?workspaceId=${WORKSPACE_ID}`), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('encodes special characters in the extended download filename', async () => { mocks.download.mockResolvedValueOnce({ file: fileRecord({ name: "it's (final)* café.pdf" }), @@ -226,7 +277,11 @@ describe('v2 single-file routes', () => { expect(response.status).toBe(403) expect(await response.json()).toEqual({ - error: { code: 'FORBIDDEN', message: 'Insufficient workspace permissions' }, + error: { + code: 'FORBIDDEN', + message: 'Insufficient workspace permissions', + details: { code: 'INSUFFICIENT_WORKSPACE_ROLE' }, + }, }) }) diff --git a/apps/sim/app/api/v2/files/[fileId]/route.ts b/apps/sim/app/api/v2/files/[fileId]/route.ts index 0a7bbc896bc..eb0df4f5f66 100644 --- a/apps/sim/app/api/v2/files/[fileId]/route.ts +++ b/apps/sim/app/api/v2/files/[fileId]/route.ts @@ -28,10 +28,14 @@ export const revalidate = 0 * Lookups are workspace-scoped (IDOR-safe): a file in another workspace 404s. * * A generated doc whose artifact is still compiling renders `CONFLICT`; retry. + * + * `headSafe: false` because downloading records a `FILE_DOWNLOADED` audit event + * and pulls the bytes out of object storage. */ export const GET = defineV2BinaryRoute({ contract: v2DownloadFileContract, auth: v2ApiKeyAuth, + headSafe: false, operation: fileOperations.download, rateLimit: v2RateLimits.publicApi, errorPolicy: v2FileErrorPolicies.concealResourceAuthorization, diff --git a/apps/sim/app/api/v2/files/[fileId]/share/route.test.ts b/apps/sim/app/api/v2/files/[fileId]/share/route.test.ts index 42d6c30a8a2..7d33f8c91f9 100644 --- a/apps/sim/app/api/v2/files/[fileId]/share/route.test.ts +++ b/apps/sim/app/api/v2/files/[fileId]/share/route.test.ts @@ -1,46 +1,26 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error { - constructor(message = 'Invalid API key') { - super(message) - this.name = 'V2ApiKeyUnauthenticatedError' - } - } - - return { - mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), - getShare: vi.fn(), - updateShare: vi.fn(), - }, - MockV2ApiKeyUnauthenticatedError, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, +const mocks = vi.hoisted(() => ({ + getShare: vi.fn(), + updateShare: vi.fn(), })) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: vi.fn().mockReturnValue({ - maxTokens: 100, - refillRate: 100, - refillIntervalMs: 60_000, - }), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), @@ -52,8 +32,6 @@ vi.mock('@/lib/core/utils/request', () => ({ getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) - vi.mock('@/lib/workspace-files/application/share-workspace-file', () => ({ getWorkspaceFileShare: { operation: { id: 'files.share.read', minimumRole: 'read', workspaceApiKey: 'allow' }, @@ -67,6 +45,7 @@ vi.mock('@/lib/workspace-files/application/share-workspace-file', () => ({ import { OrchestrationError } from '@/lib/core/orchestration/types' import { GET, PATCH } from '@/app/api/v2/files/[fileId]/share/route' +import { v2Error } from '@/app/api/v2/lib/response' const WORKSPACE_ID = 'workspace-1' const FILE_ID = 'wf_1' @@ -74,16 +53,16 @@ const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_I const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'] as const, + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE_LIMIT_OK = { - allowed: true, +const RATE_LIMIT_DENIED = { + allowed: false, limit: 100, - remaining: 99, + remaining: 0, resetAt: new Date('2024-01-01T01:00:00Z'), - retryAfterMs: 0, + retryAfterMs: 1000, } const SHARE = { id: 'shr_1', @@ -121,15 +100,15 @@ function callPatch(body: unknown) { describe('GET /api/v2/files/[fileId]/share', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) mocks.getShare.mockResolvedValue({ share: SHARE }) }) it('authenticates and rate-limits before parsing or executing', async () => { - mocks.authenticate.mockRejectedValueOnce( + v2RouteMocks.authenticate.mockRejectedValueOnce( new MockV2ApiKeyUnauthenticatedError('API key required') ) @@ -137,12 +116,11 @@ describe('GET /api/v2/files/[fileId]/share', () => { expect(response.status).toBe(401) expect(mocks.getShare).not.toHaveBeenCalled() - expect(mocks.operationRate).not.toHaveBeenCalled() + expect(v2RouteMocks.operationRate).not.toHaveBeenCalled() }) it('returns 404 when the v2 API surface flag is off', async () => { - const { v2Error } = await import('@/app/api/v2/lib/response') - mocks.gate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) + v2RouteMocks.gate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) const response = await callGet() @@ -180,7 +158,7 @@ describe('GET /api/v2/files/[fileId]/share', () => { }) it('returns the rate-limit response when denied', async () => { - mocks.operationRate.mockResolvedValueOnce({ ...RATE_LIMIT_OK, allowed: false, remaining: 0 }) + v2RouteMocks.operationRate.mockResolvedValueOnce(RATE_LIMIT_DENIED) const response = await callGet() @@ -193,10 +171,10 @@ describe('GET /api/v2/files/[fileId]/share', () => { describe('PATCH /api/v2/files/[fileId]/share', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) mocks.updateShare.mockResolvedValue({ share: SHARE }) }) @@ -261,7 +239,7 @@ describe('PATCH /api/v2/files/[fileId]/share', () => { }) it('returns the rate-limit response when denied', async () => { - mocks.operationRate.mockResolvedValueOnce({ ...RATE_LIMIT_OK, allowed: false, remaining: 0 }) + v2RouteMocks.operationRate.mockResolvedValueOnce(RATE_LIMIT_DENIED) const response = await callPatch({ workspaceId: WORKSPACE_ID, isActive: true }) diff --git a/apps/sim/app/api/v2/files/bulk-delete/route.test.ts b/apps/sim/app/api/v2/files/bulk-delete/route.test.ts index a66f490f6c2..01206253399 100644 --- a/apps/sim/app/api/v2/files/bulk-delete/route.test.ts +++ b/apps/sim/app/api/v2/files/bulk-delete/route.test.ts @@ -1,35 +1,25 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockPreauth, mockOperationRate, mockGate, mockExecute } = vi.hoisted(() => ({ - mockPreauth: vi.fn(), - mockOperationRate: vi.fn(), - mockGate: vi.fn(), +const { mockExecute } = vi.hoisted(() => ({ mockExecute: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: vi.fn().mockResolvedValue({ - principal: { kind: 'workspace_api_key', workspaceId: 'workspace-1', keyId: 'key-1' }, - rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'], - rateLimitSubscription: null, - keyType: 'workspace', - }), - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mockPreauth - checkRateLimitDirectOrThrow = mockOperationRate - }, - getRateLimit: vi - .fn() - .mockReturnValue({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -38,7 +28,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mockGate })) vi.mock('@/lib/workspace-files/application/archive-workspace-file-items', () => ({ archiveWorkspaceFileItemsOperation: { operation: { id: 'files.delete', minimumRole: 'write', workspaceApiKey: 'allow' }, @@ -46,13 +35,22 @@ vi.mock('@/lib/workspace-files/application/archive-workspace-file-items', () => }, })) +import { OrchestrationError } from '@/lib/core/orchestration/types' import { POST } from '@/app/api/v2/files/bulk-delete/route' +import { v2Error } from '@/app/api/v2/lib/response' const WS = 'workspace-1' -const RATE_LIMIT_OK = { - allowed: true, +const AUTH = { + principal: { kind: 'workspace_api_key' as const, workspaceId: WS, keyId: 'key-1' }, + rolloutUserId: 'owner-1', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WS}`] as const, + rateLimitSubscription: null, + keyType: 'workspace' as const, +} +const RATE_LIMIT_DENIED = { + allowed: false, limit: 100, - remaining: 99, + remaining: 0, resetAt: new Date('2024-01-01T01:00:00Z'), retryAfterMs: 0, } @@ -69,15 +67,15 @@ const callDelete = (body: unknown) => describe('POST /api/v2/files/bulk-delete', () => { beforeEach(() => { vi.clearAllMocks() - mockPreauth.mockResolvedValue(RATE_LIMIT_OK) - mockOperationRate.mockResolvedValue(RATE_LIMIT_OK) - mockGate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mockExecute.mockResolvedValue({ deletedItems: { files: 3, folders: 0 } }) }) it('returns 404 when the v2 API surface flag is off', async () => { - const { v2Error } = await import('@/app/api/v2/lib/response') - mockGate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) + v2RouteMocks.gate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(404) expect(mockExecute).not.toHaveBeenCalled() @@ -91,19 +89,26 @@ describe('POST /api/v2/files/bulk-delete', () => { }) it('surfaces a forbidden collection operation', async () => { - const { OrchestrationError } = await import('@/lib/core/orchestration/types') mockExecute.mockRejectedValue(new OrchestrationError('forbidden', 'Access denied')) const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(403) }) it('returns the rate-limit response when denied', async () => { - mockPreauth.mockResolvedValue({ ...RATE_LIMIT_OK, allowed: false, remaining: 0 }) + v2RouteMocks.preauthRate.mockResolvedValue(RATE_LIMIT_DENIED) const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(429) expect((await res.json()).error.code).toBe('RATE_LIMITED') }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) + expect(res.status).toBe(401) + expect((await res.json()).error.code).toBe('UNAUTHORIZED') + expect(mockExecute).not.toHaveBeenCalled() + }) + it('deletes the selection and reports the file count', async () => { const res = await callDelete({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(200) @@ -114,7 +119,6 @@ describe('POST /api/v2/files/bulk-delete', () => { }) it('maps a not-found failure to 404', async () => { - const { OrchestrationError } = await import('@/lib/core/orchestration/types') mockExecute.mockRejectedValue(new OrchestrationError('not_found', 'File not found')) const res = await callDelete({ workspaceId: WS, fileIds: ['wf_missing'] }) expect(res.status).toBe(404) diff --git a/apps/sim/app/api/v2/files/folders/route.test.ts b/apps/sim/app/api/v2/files/folders/route.test.ts index 50dd9b0c98c..8c42e0ebd66 100644 --- a/apps/sim/app/api/v2/files/folders/route.test.ts +++ b/apps/sim/app/api/v2/files/folders/route.test.ts @@ -1,41 +1,28 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error {} - return { - mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), - listFolders: vi.fn(), - createFolder: vi.fn(), - updateFolder: vi.fn(), - deleteFolder: vi.fn(), - }, - MockV2ApiKeyUnauthenticatedError, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: vi.fn().mockReturnValue({ - maxTokens: 100, - refillRate: 100, - refillIntervalMs: 60_000, - }), +const mocks = vi.hoisted(() => ({ + listFolders: vi.fn(), + createFolder: vi.fn(), + updateFolder: vi.fn(), + deleteFolder: vi.fn(), })) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -44,7 +31,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) vi.mock('@/lib/workspace-files/application/workspace-file-folders', () => ({ listWorkspaceFileFoldersOperation: { operation: { id: 'files.folders.list', minimumRole: 'read', workspaceApiKey: 'allow' }, @@ -75,17 +61,10 @@ const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_I const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'] as const, + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE_LIMIT_OK = { - allowed: true, - limit: 100, - remaining: 99, - resetAt: new Date('2024-01-01T01:00:00Z'), - retryAfterMs: 0, -} const folder = { id: 'folder-1', workspaceId: WORKSPACE_ID, @@ -114,10 +93,10 @@ function request(method: 'GET' | 'POST' | 'PATCH' | 'DELETE', url: string, body? describe('/api/v2/files/folders', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listFolders.mockResolvedValue({ folders: [folder] }) mocks.createFolder.mockResolvedValue({ folder }) mocks.updateFolder.mockResolvedValue({ folder }) @@ -293,12 +272,38 @@ describe('/api/v2/files/folders', () => { expect((await response.json()).error.code).toBe('NOT_FOUND') }) + it('rejects a percent-encoded NUL in a canonical path before the write reaches Postgres', async () => { + const created = await POST( + request('POST', '/api/v2/files/folders', { + workspaceId: WORKSPACE_ID, + path: '/apitest_%00x', + }), + context + ) + const relocated = await PATCH( + request('PATCH', '/api/v2/files/folders', { + workspaceId: WORKSPACE_ID, + path: '/Reports', + destinationPath: '/apitest_%00b', + }), + context + ) + + expect(created.status).toBe(400) + expect((await created.json()).error.code).toBe('BAD_REQUEST') + expect(relocated.status).toBe(400) + expect((await relocated.json()).error.code).toBe('BAD_REQUEST') + expect(mocks.createFolder).not.toHaveBeenCalled() + expect(mocks.updateFolder).not.toHaveBeenCalled() + }) + it('authenticates before parsing folder input', async () => { - mocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) const response = await POST(request('POST', '/api/v2/files/folders', {}), context) expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.createFolder).not.toHaveBeenCalled() }) }) diff --git a/apps/sim/app/api/v2/files/move/route.test.ts b/apps/sim/app/api/v2/files/move/route.test.ts index a311ffcb614..9f8d95138f4 100644 --- a/apps/sim/app/api/v2/files/move/route.test.ts +++ b/apps/sim/app/api/v2/files/move/route.test.ts @@ -1,35 +1,25 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockPreauth, mockOperationRate, mockGate, mockExecute } = vi.hoisted(() => ({ - mockPreauth: vi.fn(), - mockOperationRate: vi.fn(), - mockGate: vi.fn(), +const { mockExecute } = vi.hoisted(() => ({ mockExecute: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: vi.fn().mockResolvedValue({ - principal: { kind: 'workspace_api_key', workspaceId: 'workspace-1', keyId: 'key-1' }, - rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'], - rateLimitSubscription: null, - keyType: 'workspace', - }), - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mockPreauth - checkRateLimitDirectOrThrow = mockOperationRate - }, - getRateLimit: vi - .fn() - .mockReturnValue({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -38,7 +28,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mockGate })) vi.mock('@/lib/workspace-files/application/move-workspace-file-items', () => ({ moveWorkspaceFileItemsOperation: { operation: { id: 'files.move', minimumRole: 'write', workspaceApiKey: 'allow' }, @@ -46,18 +35,26 @@ vi.mock('@/lib/workspace-files/application/move-workspace-file-items', () => ({ }, })) +import { OrchestrationError } from '@/lib/core/orchestration/types' import { WorkspaceFileMoveConflictError } from '@/lib/uploads/contexts/workspace/workspace-file-folder-manager' import { POST } from '@/app/api/v2/files/move/route' +import { v2Error } from '@/app/api/v2/lib/response' const WS = 'workspace-1' -const RATE_LIMIT_OK = { - allowed: true, +const auth = { + principal: { kind: 'workspace_api_key' as const, workspaceId: WS, keyId: 'key-1' }, + rolloutUserId: 'owner-1', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WS}`] as const, + rateLimitSubscription: null, + keyType: 'workspace' as const, +} +const RATE_LIMIT_DENIED = { + allowed: false, limit: 100, - remaining: 99, + remaining: 0, resetAt: new Date('2024-01-01T01:00:00Z'), - retryAfterMs: 0, + retryAfterMs: 1000, } -const RATE_LIMIT_DENIED = { ...RATE_LIMIT_OK, allowed: false, remaining: 0, retryAfterMs: 1000 } const callMove = (body: unknown) => POST( @@ -71,15 +68,22 @@ const callMove = (body: unknown) => describe('POST /api/v2/files/move', () => { beforeEach(() => { vi.clearAllMocks() - mockPreauth.mockResolvedValue(RATE_LIMIT_OK) - mockOperationRate.mockResolvedValue(RATE_LIMIT_OK) - mockGate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mockExecute.mockResolvedValue({ movedItems: { files: 2, folders: 0 } }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + const res = await callMove({ workspaceId: WS, fileIds: ['wf_1'] }) + expect(res.status).toBe(401) + expect((await res.json()).error.code).toBe('UNAUTHORIZED') + }) + it('returns 404 when the v2 API surface flag is off', async () => { - const { v2Error } = await import('@/app/api/v2/lib/response') - mockGate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) + v2RouteMocks.gate.mockResolvedValueOnce(v2Error('NOT_FOUND', 'Not found')) const res = await callMove({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(404) expect(mockExecute).not.toHaveBeenCalled() @@ -93,7 +97,6 @@ describe('POST /api/v2/files/move', () => { }) it('surfaces a forbidden collection operation', async () => { - const { OrchestrationError } = await import('@/lib/core/orchestration/types') mockExecute.mockRejectedValue(new OrchestrationError('forbidden', 'Access denied')) const res = await callMove({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(403) @@ -101,7 +104,7 @@ describe('POST /api/v2/files/move', () => { }) it('returns the rate-limit response when denied', async () => { - mockPreauth.mockResolvedValue(RATE_LIMIT_DENIED) + v2RouteMocks.preauthRate.mockResolvedValue(RATE_LIMIT_DENIED) const res = await callMove({ workspaceId: WS, fileIds: ['wf_1'] }) expect(res.status).toBe(429) expect((await res.json()).error.code).toBe('RATE_LIMITED') diff --git a/apps/sim/app/api/v2/files/route.test.ts b/apps/sim/app/api/v2/files/route.test.ts index 8d62db0876c..a1c162f1319 100644 --- a/apps/sim/app/api/v2/files/route.test.ts +++ b/apps/sim/app/api/v2/files/route.test.ts @@ -1,17 +1,22 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), createFile: vi.fn(), queryFiles: vi.fn(), getUserEmailsByIds: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/workspace-files/application/create-workspace-file', () => ({ @@ -28,20 +33,9 @@ vi.mock('@/lib/workspace-files/application/list-workspace-files', () => ({ }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/users/queries', () => ({ getUserEmailsByIds: mocks.getUserEmailsByIds, @@ -89,22 +83,13 @@ function createRequest(body: unknown): NextRequest { describe('/api/v2/files', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-04T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-04T01:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.queryFiles.mockResolvedValue({ files: [FILE], nextKeys: undefined, - cursorSort: 'name:asc', }) mocks.createFile.mockResolvedValue({ file: FILE }) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'ada@example.com']])) @@ -114,11 +99,52 @@ describe('/api/v2/files', () => { const response = await GET(new NextRequest('http://localhost:3000/api/v2/files')) expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalled() - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.queryFiles).not.toHaveBeenCalled() }) + /** + * `?limit=` is not `limit` omitted. `Number('') === 0`, and this list clamps + * out-of-range values, so an unrejected blank reaches the query as `LIMIT 1` + * and returns a single row where the omitted param returns a hundred — a + * silently wrong page, not an error. Whitespace-only is the same value. + */ + it.each(['limit=', 'limit=%20', 'sortBy=', 'cursor='])( + 'rejects the blank query value %s instead of coercing it', + async (param) => { + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&${param}`) + ) + + expect(response.status).toBe(400) + expect((await response.json()).error.code).toBe('BAD_REQUEST') + expect(mocks.queryFiles).not.toHaveBeenCalled() + } + ) + + it('still applies the documented default when limit is omitted entirely', async () => { + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}`) + ) + + expect(response.status).toBe(200) + expect(mocks.queryFiles).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ limit: 100 }) }) + ) + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}`) + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('lists through the shared use case and v2 presenter', async () => { const request = new NextRequest( `http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&sortBy=name` @@ -138,6 +164,7 @@ describe('/api/v2/files', () => { uploadedByEmail: 'ada@example.com', uploadedAt: '2026-08-04T00:00:00.000Z', updatedAt: '2026-08-05T00:00:00.000Z', + deletedAt: null, }, ], nextCursor: null, @@ -146,6 +173,7 @@ describe('/api/v2/files', () => { principal: auth.principal, input: expect.objectContaining({ workspaceId: WORKSPACE_ID, + scope: 'active', sortBy: 'name', sortOrder: 'asc', limit: 100, @@ -154,11 +182,39 @@ describe('/api/v2/files', () => { }) }) + it('pages the archived set and dates each soft delete when asked for it', async () => { + mocks.queryFiles.mockResolvedValueOnce({ + files: [{ ...FILE, deletedAt: new Date('2026-08-06T00:00:00.000Z') }], + nextKeys: undefined, + }) + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&scope=archived` + ) + ) + + expect(response.status).toBe(200) + expect((await response.json()).data[0].deletedAt).toBe('2026-08-06T00:00:00.000Z') + expect(mocks.queryFiles).toHaveBeenCalledWith({ + principal: auth.principal, + input: expect.objectContaining({ scope: 'archived' }), + request: expect.anything(), + }) + }) + + it('rejects an unimplemented scope instead of silently listing the active set', async () => { + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&scope=all`) + ) + + expect(response.status).toBe(400) + expect(mocks.queryFiles).not.toHaveBeenCalled() + }) + it('preserves escaped slashes in the containing folder path', async () => { mocks.queryFiles.mockResolvedValueOnce({ files: [{ ...FILE, folderId: 'folder-1', folderPath: 'Finance\\/Legal' }], nextKeys: undefined, - cursorSort: 'name:asc', }) const response = await GET( new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}`) @@ -168,6 +224,63 @@ describe('/api/v2/files', () => { expect((await response.json()).data[0].folderPath).toBe('/Finance%2FLegal') }) + /** + * A keyset cursor stays *coherent* under a changed filter, which is what makes + * it dangerous: replaying it under a narrowed `search` returns a correctly + * ordered page of the new matches that happen to sort after the old position, + * and silently omits every match before it. The caller sees an opaque token + * and a short page, and reads that as "almost nothing matched". + */ + it.each([ + ['search', 'search=quarterly'], + ['scope', 'scope=archived'], + ['folderPath', 'folderPath=/Finance'], + ])('refuses a cursor replayed under a different %s', async (_filter, param) => { + mocks.queryFiles.mockResolvedValueOnce({ files: [FILE], nextKeys: ['notes.md', FILE.id] }) + const firstPage = await ( + await GET(new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}`)) + ).json() + expect(firstPage.nextCursor).toEqual(expect.any(String)) + mocks.queryFiles.mockClear() + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&${param}&cursor=${encodeURIComponent(firstPage.nextCursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('requested filters') }, + }) + expect(mocks.queryFiles).not.toHaveBeenCalled() + }) + + /** + * `limit` is not part of the binding: it selects how much of the sequence to + * return, not what the sequence is. + */ + it('resumes a cursor under an unchanged filter and a changed page size', async () => { + mocks.queryFiles.mockResolvedValueOnce({ files: [FILE], nextKeys: ['notes.md', FILE.id] }) + const firstPage = await ( + await GET(new NextRequest(`http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}`)) + ).json() + mocks.queryFiles.mockResolvedValueOnce({ files: [FILE], nextKeys: undefined }) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/files?workspaceId=${WORKSPACE_ID}&limit=5&cursor=${encodeURIComponent(firstPage.nextCursor)}` + ) + ) + + expect(response.status).toBe(200) + expect(mocks.queryFiles).toHaveBeenLastCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ limit: 5, after: ['notes.md', FILE.id] }), + }) + ) + }) + it('rejects malformed cursors before the application service', async () => { const response = await GET( new NextRequest( @@ -216,8 +329,8 @@ describe('/api/v2/files', () => { ) expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalled() - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.createFile).not.toHaveBeenCalled() }) diff --git a/apps/sim/app/api/v2/files/route.ts b/apps/sim/app/api/v2/files/route.ts index 9b905ca9342..e544f4bf231 100644 --- a/apps/sim/app/api/v2/files/route.ts +++ b/apps/sim/app/api/v2/files/route.ts @@ -3,9 +3,8 @@ import { v2CreateFileContract, v2ListFilesContract, } from '@/lib/api/contracts/v2/files' -import { INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' -import { OrchestrationError } from '@/lib/core/orchestration/types' import { getFileExtension, getMimeTypeFromExtension } from '@/lib/uploads/utils/file-utils' import { v2FileErrorPolicies } from '@/lib/workspace-files/api' import { createWorkspaceFile } from '@/lib/workspace-files/application/create-workspace-file' @@ -13,16 +12,26 @@ import { queryWorkspaceFilePage } from '@/lib/workspace-files/application/list-w import { fileOperations } from '@/lib/workspace-files/application/operations' import { MAX_WORKSPACE_FILE_INLINE_BODY_BYTES } from '@/lib/workspace-files/orchestration' import { toV2File, toV2Files } from '@/app/api/v2/files/utils' -import { - cursorSortKey, - decodeSortedCursor, - encodeSortedCursor, - v2Error, -} from '@/app/api/v2/lib/response' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Every param that changes which files, in which order, this list returns. */ +function fileCursorFilters(query: { + workspaceId: string + scope?: string + folderPath?: string + search?: string +}) { + return cursorScopeKey(cursorRoute(v2ListFilesContract), { + workspaceId: query.workspaceId, + scope: query.scope, + folderPath: query.folderPath, + search: query.search, + }) +} + /** GET /api/v2/files — List files with search, sort, and cursor pagination. */ export const GET = defineV2JsonRoute({ contract: v2ListFilesContract, @@ -30,27 +39,28 @@ export const GET = defineV2JsonRoute({ operation: fileOperations.list, rateLimit: v2RateLimits.publicApi, errorPolicy: v2FileErrorPolicies.default, - mapInput: ({ query }) => { - const cursorSort = cursorSortKey(query.sortBy, query.sortOrder) - const decoded = decodeSortedCursor(query.cursor, cursorSort) - if (decoded.status === 'invalid') { - throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) - } - return { - workspaceId: query.workspaceId, - folderPath: query.folderPath, - search: query.search, - sortBy: query.sortBy, - sortOrder: query.sortOrder, - limit: query.limit, - after: decoded.status === 'ok' ? decoded.keys : undefined, - cursorSort, - } - }, + mapInput: ({ query }) => ({ + workspaceId: query.workspaceId, + scope: query.scope, + folderPath: query.folderPath, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + limit: query.limit, + after: readSortedCursor(query.cursor, query.sortBy, query.sortOrder, fileCursorFilters(query)), + }), useCase: queryWorkspaceFilePage, - present: async ({ files, nextKeys, cursorSort }) => { + present: async ({ files, nextKeys }, { query }) => { const items: V2File[] = await toV2Files(files) - return { data: items, nextCursor: nextKeys ? encodeSortedCursor(cursorSort, nextKeys) : null } + return { + data: items, + nextCursor: writeSortedCursor( + nextKeys, + query.sortBy, + query.sortOrder, + fileCursorFilters(query) + ), + } }, }) @@ -62,7 +72,6 @@ export const POST = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2FileErrorPolicies.default, parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), maxBodyBytes: MAX_WORKSPACE_FILE_INLINE_BODY_BYTES, }, mapInput: ({ body }) => ({ diff --git a/apps/sim/app/api/v2/files/uploads/[uploadId]/route.test.ts b/apps/sim/app/api/v2/files/uploads/[uploadId]/route.test.ts index 8df882b0064..193aeee9c65 100644 --- a/apps/sim/app/api/v2/files/uploads/[uploadId]/route.test.ts +++ b/apps/sim/app/api/v2/files/uploads/[uploadId]/route.test.ts @@ -1,14 +1,20 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ abort: vi.fn(), - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), })) vi.mock('@/lib/uploads/upload-session/application', () => ({ @@ -18,20 +24,9 @@ vi.mock('@/lib/uploads/upload-session/application', () => ({ }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: vi.fn().mockResolvedValue(null) })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/files/uploads/utils', () => ({ toV2FileUpload: vi.fn(async () => ({ @@ -78,17 +73,10 @@ function abortRequest() { describe('DELETE /api/v2/files/uploads/[uploadId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(AUTH) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-04T21:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-04T21:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.abort.mockResolvedValue({ id: UPLOAD_ID }) }) @@ -99,6 +87,16 @@ describe('DELETE /api/v2/files/uploads/[uploadId]', () => { expect(await response.json()).toMatchObject({ data: { id: UPLOAD_ID, status: 'aborted' } }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await DELETE(abortRequest(), context) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + expect(mocks.abort).not.toHaveBeenCalled() + }) + it('conceals a cross-tenant reach as a missing upload session', async () => { mocks.abort.mockRejectedValueOnce(new NoWorkspaceAccessError()) @@ -130,7 +128,11 @@ describe('DELETE /api/v2/files/uploads/[uploadId]', () => { expect(response.status).toBe(403) expect(await response.json()).toEqual({ - error: { code: 'FORBIDDEN', message: 'Insufficient workspace permissions' }, + error: { + code: 'FORBIDDEN', + message: 'Insufficient workspace permissions', + details: { code: 'INSUFFICIENT_WORKSPACE_ROLE' }, + }, }) }) }) diff --git a/apps/sim/app/api/v2/files/uploads/route.test.ts b/apps/sim/app/api/v2/files/uploads/route.test.ts index cfc19a29766..f1b6df19ad9 100644 --- a/apps/sim/app/api/v2/files/uploads/route.test.ts +++ b/apps/sim/app/api/v2/files/uploads/route.test.ts @@ -1,15 +1,20 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), createUpload: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/uploads/upload-session/application', () => ({ @@ -19,20 +24,9 @@ vi.mock('@/lib/uploads/upload-session/application', () => ({ }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/files/uploads/utils', () => ({ toV2FileUpload: vi.fn(async () => ({ @@ -86,18 +80,10 @@ function request(body: Record) { describe('POST /api/v2/files/uploads', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(AUTH) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-04T21:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-04T21:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.createUpload.mockResolvedValue(UPLOAD_SESSION) }) @@ -139,8 +125,23 @@ describe('POST /api/v2/files/uploads', () => { const response = await request({ workspaceId: WORKSPACE_ID }).response expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalledTimes(1) - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalledTimes(1) + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) + expect(mocks.createUpload).not.toHaveBeenCalled() + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await request({ + workspaceId: WORKSPACE_ID, + name: 'file.csv', + contentType: 'text/csv', + size: 10, + }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.createUpload).not.toHaveBeenCalled() }) @@ -152,7 +153,7 @@ describe('POST /api/v2/files/uploads', () => { size: 0, }).response - expect(mocks.authenticateV2ApiKey).toHaveBeenCalledTimes(1) + expect(v2RouteMocks.authenticate).toHaveBeenCalledTimes(1) expect(mocks.createUpload).toHaveBeenCalledWith( expect.objectContaining({ principal: PRINCIPAL }) ) diff --git a/apps/sim/app/api/v2/files/utils.ts b/apps/sim/app/api/v2/files/utils.ts index d21514036e4..bff1477af38 100644 --- a/apps/sim/app/api/v2/files/utils.ts +++ b/apps/sim/app/api/v2/files/utils.ts @@ -30,6 +30,7 @@ function serializeV2File(record: WorkspaceFileRecord, uploadedByEmail: string): uploadedByEmail, uploadedAt: record.uploadedAt.toISOString(), updatedAt: record.updatedAt.toISOString(), + deletedAt: record.deletedAt?.toISOString() ?? null, } } diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.test.ts new file mode 100644 index 00000000000..272d20c5f52 --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.test.ts @@ -0,0 +1,252 @@ +/** + * @vitest-environment node + */ +import { + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockReadDocument, mockUpdateDocument, mockDeleteDocument, mockCapture } = vi.hoisted( + () => ({ + mockReadDocument: vi.fn(), + mockUpdateDocument: vi.fn(), + mockDeleteDocument: vi.fn(), + mockCapture: vi.fn(), + }) +) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +vi.mock('@/lib/knowledge/application/documents', () => ({ + readKnowledgeDocument: { + operation: { id: 'knowledge.documents.read' }, + execute: mockReadDocument, + }, + updateKnowledgeDocument: { + operation: { id: 'knowledge.documents.update' }, + execute: mockUpdateDocument, + }, + deleteKnowledgeDocument: { + operation: { id: 'knowledge.documents.delete' }, + execute: mockDeleteDocument, + }, +})) + +vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mockCapture })) + +import { GET, PATCH } from '@/app/api/v2/knowledge/[id]/documents/[documentId]/route' + +const WORKSPACE_ID = 'workspace-1' +const PRINCIPAL = { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' } as const +const UPLOADED_AT = new Date('2025-06-18T16:45:00Z') + +const TAG_DEFINITIONS = [ + { + id: 'tag-def-1', + knowledgeBaseId: 'kb-1', + tagSlot: 'tag1', + displayName: 'category', + fieldType: 'text', + createdAt: UPLOADED_AT, + updatedAt: UPLOADED_AT, + }, + { + id: 'tag-def-2', + knowledgeBaseId: 'kb-1', + tagSlot: 'number1', + displayName: 'priority', + fieldType: 'number', + createdAt: UPLOADED_AT, + updatedAt: UPLOADED_AT, + }, +] + +const DOCUMENT_ROW = { + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'support.txt', + fileSize: 5, + mimeType: 'text/plain', + processingStatus: 'completed' as const, + processingError: null, + processingStartedAt: UPLOADED_AT, + processingCompletedAt: UPLOADED_AT, + chunkCount: 2, + tokenCount: 10, + characterCount: 40, + enabled: true, + connectorId: null, + connectorType: null, + sourceUrl: null, + uploadedAt: UPLOADED_AT, + tag1: 'billing', + tag2: null, + number1: 2, + date1: null, + boolean1: null, + tag6: 'orphaned-slot-value', +} + +const context = { params: Promise.resolve({ id: 'kb-1', documentId: 'doc-1' }) } + +function buildGetRequest() { + return new NextRequest( + `http://localhost/api/v2/knowledge/kb-1/documents/doc-1?workspaceId=${WORKSPACE_ID}`, + { headers: { 'x-api-key': 'secret' } } + ) +} + +function buildPatchRequest(body: unknown) { + return new NextRequest('http://localhost/api/v2/knowledge/kb-1/documents/doc-1', { + method: 'PATCH', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify(body), + }) +} + +describe('/api/v2/knowledge/[id]/documents/[documentId]', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue({ + principal: PRINCIPAL, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:key-1', 'user:user-1'], + rateLimitSubscription: null, + keyType: 'personal', + }) + mockReadDocument.mockResolvedValue({ + document: DOCUMENT_ROW, + tagDefinitions: TAG_DEFINITIONS, + workspaceId: WORKSPACE_ID, + }) + mockUpdateDocument.mockResolvedValue({ + kind: 'updated', + document: { ...DOCUMENT_ROW, filename: 'renamed.txt', enabled: false }, + tagDefinitions: TAG_DEFINITIONS, + updatedFields: ['filename', 'enabled'], + }) + }) + + it('keys document tag values by display name, falling back to the raw slot', async () => { + const response = await GET(buildGetRequest(), context) + + expect(response.status).toBe(200) + const body = await response.json() + expect(body.data.tags).toEqual({ + category: 'billing', + priority: 2, + tag6: 'orphaned-slot-value', + }) + }) + + it('updates the whitelisted fields and returns the updated document with its tags', async () => { + const response = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + filename: 'renamed.txt', + enabled: false, + tag1: 'support', + }), + context + ) + + expect(response.status).toBe(200) + expect(mockUpdateDocument).toHaveBeenCalledWith( + expect.objectContaining({ + input: { + knowledgeBaseId: 'kb-1', + documentId: 'doc-1', + assertedWorkspaceId: WORKSPACE_ID, + updates: { filename: 'renamed.txt', enabled: false, tag1: 'support' }, + source: 'api', + }, + }) + ) + const body = await response.json() + expect(body.data).toEqual( + expect.objectContaining({ + id: 'doc-1', + filename: 'renamed.txt', + enabled: false, + tags: { category: 'billing', priority: 2, tag6: 'orphaned-slot-value' }, + }) + ) + }) + + it('acknowledges a processing retry without claiming settled indexing state', async () => { + mockUpdateDocument.mockResolvedValueOnce({ + kind: 'processing', + documentId: 'doc-1', + status: 'pending', + message: 'Document processing restarted', + }) + + const response = await PATCH( + buildPatchRequest({ workspaceId: WORKSPACE_ID, retryProcessing: true }), + context + ) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { + id: 'doc-1', + queued: true, + processingStatus: 'pending', + message: 'Document processing restarted', + }, + }) + expect(mockUpdateDocument).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ retryProcessing: true }), + }) + ) + expect(mockUpdateDocument.mock.calls[0][0].input).not.toHaveProperty('updates') + }) + + it('refuses to let a caller assert derived indexing state', async () => { + for (const body of [ + { workspaceId: WORKSPACE_ID, processingStatus: 'completed' }, + { workspaceId: WORKSPACE_ID, chunkCount: 99 }, + { workspaceId: WORKSPACE_ID, tokenCount: 99 }, + { workspaceId: WORKSPACE_ID, processingError: null }, + { workspaceId: WORKSPACE_ID, markFailedDueToTimeout: true }, + ]) { + const response = await PATCH(buildPatchRequest(body), context) + expect(response.status).toBe(400) + } + expect(mockUpdateDocument).not.toHaveBeenCalled() + }) + + it('rejects a retry combined with field updates instead of silently dropping them', async () => { + const response = await PATCH( + buildPatchRequest({ workspaceId: WORKSPACE_ID, retryProcessing: true, enabled: false }), + context + ) + + expect(response.status).toBe(400) + expect(await response.json()).toEqual({ + error: expect.objectContaining({ + message: expect.stringContaining('retryProcessing cannot be combined with enabled'), + }), + }) + expect(mockUpdateDocument).not.toHaveBeenCalled() + }) + + it('rejects an update that changes nothing', async () => { + const response = await PATCH(buildPatchRequest({ workspaceId: WORKSPACE_ID }), context) + + expect(response.status).toBe(400) + expect(mockUpdateDocument).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.ts index 2695d8e3984..3d9d8543e08 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/[documentId]/route.ts @@ -1,29 +1,56 @@ import { + V2_WRITABLE_TAG_SLOTS, + type V2UpdateKnowledgeDocumentBody, v2DeleteKnowledgeDocumentContract, v2GetKnowledgeDocumentContract, + v2UpdateKnowledgeDocumentContract, } from '@/lib/api/contracts/v2/knowledge' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { deleteKnowledgeDocument, readKnowledgeDocument, + type UpdateKnowledgeDocumentInput, + updateKnowledgeDocument, } from '@/lib/knowledge/application/documents' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { captureServerEvent } from '@/lib/posthog/server' import { serializeDate } from '@/app/api/v1/knowledge/utils' +import { + toV2DocumentSummary, + toV2DocumentTags, + toV2TaggedDocument, +} from '@/app/api/v2/knowledge/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 -function toProcessingStatus(status: string): 'pending' | 'processing' | 'completed' | 'failed' { - switch (status) { - case 'pending': - case 'processing': - case 'completed': - case 'failed': - return status - default: - throw new Error(`Unexpected knowledge document processing status: ${status}`) +type V2DocumentUpdates = Omit + +type UpdateKnowledgeDocumentUpdates = NonNullable + +/** + * Serializes the typed tag slots for the document writer. + * + * The wire takes each slot in its natural JSON type — a number for a number + * slot, `true`/`false` for a boolean one — because that is how a document read + * projects them. The writer's `convertTagValue` takes strings and parses back to + * the storage column's type, so the boundary hands it the canonical spelling. + * The contract has already rejected anything those parsers would answer `null` + * for, so nothing reaches storage silently cleared. + */ +function toTagSlotUpdates(updates: V2DocumentUpdates): UpdateKnowledgeDocumentUpdates { + const { filename, enabled, ...slots } = updates + const serialized: Record = {} + for (const slot of V2_WRITABLE_TAG_SLOTS) { + const value = slots[slot] + if (value === undefined) continue + serialized[slot] = typeof value === 'string' ? value : String(value) + } + return { + ...(filename === undefined ? {} : { filename }), + ...(enabled === undefined ? {} : { enabled }), + ...serialized, } } @@ -40,29 +67,60 @@ export const GET = defineV2JsonRoute({ assertedWorkspaceId: query.workspaceId, }), useCase: readKnowledgeDocument, - present: ({ document }) => ({ + present: ({ document, tagDefinitions }) => ({ data: { - id: document.id, - knowledgeBaseId: document.knowledgeBaseId, - filename: document.filename, - fileSize: document.fileSize, - mimeType: document.mimeType, - processingStatus: toProcessingStatus(document.processingStatus), + ...toV2DocumentSummary(document), + tags: toV2DocumentTags(document, tagDefinitions), processingError: document.processingError, processingStartedAt: serializeDate(document.processingStartedAt), processingCompletedAt: serializeDate(document.processingCompletedAt), - chunkCount: document.chunkCount, - tokenCount: document.tokenCount, - characterCount: document.characterCount, - enabled: document.enabled, connectorId: document.connectorId, connectorType: document.connectorType, sourceUrl: document.sourceUrl, - createdAt: serializeDate(document.uploadedAt), }, }), }) +/** + * PATCH /api/v2/knowledge/[id]/documents/[documentId] — Update a document. + * + * Renames, enables or disables, retags, or requeues processing. Derived + * indexing state is not writable; the contract records why. + * + * The updated document is returned without connector provenance because the + * update writes and returns the document row alone. A caller that needs the full + * detail re-reads it with GET. + */ +export const PATCH = defineV2JsonRoute({ + contract: v2UpdateKnowledgeDocumentContract, + auth: v2ApiKeyAuth, + operation: knowledgeOperations.updateDocument, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, + mapInput: ({ params, body }) => { + const { workspaceId, retryProcessing, ...updates } = body + return { + knowledgeBaseId: params.id, + documentId: params.documentId, + assertedWorkspaceId: workspaceId, + ...(retryProcessing ? { retryProcessing } : { updates: toTagSlotUpdates(updates) }), + source: 'api', + } + }, + useCase: updateKnowledgeDocument, + present: (result) => + result.kind === 'processing' + ? { + data: { + id: result.documentId, + queued: true as const, + processingStatus: result.status, + message: result.message, + }, + } + : { data: toV2TaggedDocument(result.document, result.tagDefinitions) }, +}) + /** DELETE /api/v2/knowledge/[id]/documents/[documentId] — Delete a document. */ export const DELETE = defineV2JsonRoute({ contract: v2DeleteKnowledgeDocumentContract, diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/collection.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/collection.test.ts new file mode 100644 index 00000000000..6f018f33d1b --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/collection.test.ts @@ -0,0 +1,327 @@ +/** + * @vitest-environment node + * + * Covers the JSON halves of the documents collection route (list and bulk + * update). The multipart upload half is covered in `route.test.ts`, which mocks + * the stream-limit helpers the JSON body parser also uses. + */ +import { + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockListDocuments, mockBulkUpdate } = vi.hoisted(() => ({ + mockListDocuments: vi.fn(), + mockBulkUpdate: vi.fn(), +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +vi.mock('@/lib/knowledge/application/documents', () => ({ + listKnowledgeDocuments: { + operation: { id: 'knowledge.documents.list' }, + execute: mockListDocuments, + }, + bulkUpdateKnowledgeDocuments: { + operation: { id: 'knowledge.documents.bulk' }, + execute: mockBulkUpdate, + }, + admitKnowledgeDocumentUpload: { + operation: { id: 'knowledge.documents.upload' }, + execute: vi.fn(), + }, + uploadKnowledgeDocument: { + operation: { id: 'knowledge.documents.upload' }, + execute: vi.fn(), + }, +})) + +vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: vi.fn() })) + +import { GET, PATCH } from '@/app/api/v2/knowledge/[id]/documents/route' + +const WORKSPACE_ID = 'workspace-1' +const PRINCIPAL = { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' } as const +const UPLOADED_AT = new Date('2025-06-18T16:45:00Z') + +const TAG_DEFINITIONS = [ + { + id: 'tag-def-1', + knowledgeBaseId: 'kb-1', + tagSlot: 'tag1', + displayName: 'category', + fieldType: 'text', + createdAt: UPLOADED_AT, + updatedAt: UPLOADED_AT, + }, +] + +const DOCUMENT = { + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'support.txt', + fileSize: 5, + mimeType: 'text/plain', + processingStatus: 'completed' as const, + chunkCount: 2, + tokenCount: 10, + characterCount: 40, + enabled: true, + uploadedAt: UPLOADED_AT, + tag1: 'billing', + tag2: null, +} + +const context = { params: Promise.resolve({ id: 'kb-1' }) } + +function buildListRequest(query: string) { + return new NextRequest(`http://localhost/api/v2/knowledge/kb-1/documents${query}`, { + headers: { 'x-api-key': 'secret' }, + }) +} + +function buildPatchRequest(body: unknown) { + return new NextRequest('http://localhost/api/v2/knowledge/kb-1/documents', { + method: 'PATCH', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify(body), + }) +} + +function authenticateAsPersonalKey() { + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue({ + principal: PRINCIPAL, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:key-1', 'user:user-1'], + rateLimitSubscription: null, + keyType: 'personal', + }) +} + +describe('GET /api/v2/knowledge/[id]/documents', () => { + beforeEach(() => { + vi.clearAllMocks() + authenticateAsPersonalKey() + mockListDocuments.mockResolvedValue({ + documents: [DOCUMENT], + tagDefinitions: TAG_DEFINITIONS, + pagination: { total: 1, limit: 50, offset: 0, hasMore: false }, + workspaceId: WORKSPACE_ID, + }) + }) + + it('returns each document with its tag values keyed by display name', async () => { + const response = await GET(buildListRequest(`?workspaceId=${WORKSPACE_ID}`), context) + + expect(response.status).toBe(200) + const body = await response.json() + expect(body.data[0]).toEqual( + expect.objectContaining({ id: 'doc-1', tags: { category: 'billing' } }) + ) + expect(body.nextCursor).toBeNull() + }) + + it('forwards display-named tag filters to the application use case', async () => { + const tagFilters = JSON.stringify([{ tagName: 'category', operator: 'eq', value: 'billing' }]) + + const response = await GET( + buildListRequest(`?workspaceId=${WORKSPACE_ID}&tagFilters=${encodeURIComponent(tagFilters)}`), + context + ) + + expect(response.status).toBe(200) + expect(mockListDocuments).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + tagNameFilters: [{ tagName: 'category', operator: 'eq', value: 'billing' }], + }), + }) + ) + }) + + it('stamps the tag filters into the cursor so a replayed cursor cannot cross filters', async () => { + const tagFilters = JSON.stringify([{ tagName: 'category', operator: 'eq', value: 'billing' }]) + mockListDocuments.mockResolvedValue({ + documents: [DOCUMENT], + tagDefinitions: TAG_DEFINITIONS, + pagination: { total: 4, limit: 2, offset: 0, hasMore: true }, + workspaceId: WORKSPACE_ID, + }) + + const unfiltered = await ( + await GET(buildListRequest(`?workspaceId=${WORKSPACE_ID}`), context) + ).json() + const filtered = await ( + await GET( + buildListRequest( + `?workspaceId=${WORKSPACE_ID}&tagFilters=${encodeURIComponent(tagFilters)}` + ), + context + ) + ).json() + + expect(unfiltered.nextCursor).not.toEqual(filtered.nextCursor) + + const replayed = await GET( + buildListRequest( + `?workspaceId=${WORKSPACE_ID}&tagFilters=${encodeURIComponent(tagFilters)}&cursor=${encodeURIComponent(unfiltered.nextCursor)}` + ), + context + ) + + expect(replayed.status).toBe(400) + }) + + it('rejects malformed and wrongly shaped tag filters with a 400', async () => { + const malformed = await GET( + buildListRequest(`?workspaceId=${WORKSPACE_ID}&tagFilters=not-json`), + context + ) + const wrongShape = await GET( + buildListRequest( + `?workspaceId=${WORKSPACE_ID}&tagFilters=${encodeURIComponent(JSON.stringify([{ tagSlot: 'tag1' }]))}` + ), + context + ) + + expect(malformed.status).toBe(400) + expect(await malformed.json()).toEqual({ + error: expect.objectContaining({ + message: 'tagFilters must be a JSON-encoded array of tag filters', + }), + }) + expect(wrongShape.status).toBe(400) + expect(mockListDocuments).not.toHaveBeenCalled() + }) +}) + +describe('PATCH /api/v2/knowledge/[id]/documents', () => { + beforeEach(() => { + vi.clearAllMocks() + authenticateAsPersonalKey() + mockBulkUpdate.mockResolvedValue({ + operation: 'disable', + successCount: 2, + updatedDocuments: [ + { id: 'doc-1', enabled: false }, + { id: 'doc-2', enabled: false }, + ], + selectAll: false, + }) + }) + + /** + * `documentIds` is bounded by the request; `selectAll` is bounded by nothing. + * Echoing the identifiers for a knowledge base of 100k documents is a + * multi-megabyte array the caller never asked for, materialized and then + * element-wise validated by the response schema. + */ + it('omits the identifier echo for an unbounded selectAll update', async () => { + mockBulkUpdate.mockResolvedValueOnce({ + operation: 'disable', + successCount: 100_000, + updatedDocuments: Array.from({ length: 100_000 }, (_, index) => ({ + id: `doc-${index}`, + enabled: false, + })), + selectAll: true, + }) + + const response = await PATCH( + buildPatchRequest({ workspaceId: WORKSPACE_ID, operation: 'disable', selectAll: true }), + context + ) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { operation: 'disable', updatedCount: 100_000 }, + }) + }) + + it('disables the named documents and answers with one object, not a page', async () => { + const response = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + operation: 'disable', + documentIds: ['doc-1', 'doc-2'], + }), + context + ) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { operation: 'disable', updatedCount: 2, documentIds: ['doc-1', 'doc-2'] }, + }) + expect(mockBulkUpdate).toHaveBeenCalledWith( + expect.objectContaining({ + input: { + knowledgeBaseId: 'kb-1', + assertedWorkspaceId: WORKSPACE_ID, + operation: 'disable', + documentIds: ['doc-1', 'doc-2'], + selectAll: undefined, + enabledFilter: undefined, + }, + }) + ) + }) + + it('does not expose an unaudited bulk delete', async () => { + const response = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + operation: 'delete', + documentIds: ['doc-1'], + }), + context + ) + + expect(response.status).toBe(400) + expect(await response.json()).toEqual({ + error: expect.objectContaining({ + message: expect.stringContaining('operation: expected one of "enable" | "disable"'), + }), + }) + expect(mockBulkUpdate).not.toHaveBeenCalled() + }) + + it('requires exactly one selection and bounds an explicit list', async () => { + const neither = await PATCH( + buildPatchRequest({ workspaceId: WORKSPACE_ID, operation: 'enable' }), + context + ) + const both = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + operation: 'enable', + documentIds: ['doc-1'], + selectAll: true, + }), + context + ) + const tooMany = await PATCH( + buildPatchRequest({ + workspaceId: WORKSPACE_ID, + operation: 'enable', + documentIds: Array.from({ length: 101 }, (_, index) => `doc-${index}`), + }), + context + ) + + expect(neither.status).toBe(400) + expect(both.status).toBe(400) + expect(tooMany.status).toBe(400) + expect(mockBulkUpdate).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/route.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/route.test.ts index 1b03a060bbe..e85abb1cce8 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/route.test.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/route.test.ts @@ -20,7 +20,10 @@ const { mockPlatformUploaded, mockCapture, mockIsPayloadSizeLimitError, + mockIsMultipartFieldValidationError, + mockListDocuments, } = vi.hoisted(() => ({ + mockListDocuments: vi.fn(), mockAdmitUpload: vi.fn(), mockUploadDocument: vi.fn(), mockReadFormData: vi.fn(), @@ -28,6 +31,7 @@ const { mockPlatformUploaded: vi.fn(), mockCapture: vi.fn(), mockIsPayloadSizeLimitError: vi.fn(), + mockIsMultipartFieldValidationError: vi.fn(), })) vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) @@ -37,6 +41,10 @@ vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/knowledge/application/documents', () => ({ listKnowledgeDocuments: { operation: { id: 'knowledge.documents.list' }, + execute: mockListDocuments, + }, + bulkUpdateKnowledgeDocuments: { + operation: { id: 'knowledge.documents.bulk' }, execute: vi.fn(), }, admitKnowledgeDocumentUpload: { @@ -52,6 +60,7 @@ vi.mock('@/lib/knowledge/application/documents', () => ({ vi.mock('@/lib/core/utils/stream-limits', () => ({ MAX_MULTIPART_OVERHEAD_BYTES: 1024 * 1024, isPayloadSizeLimitError: mockIsPayloadSizeLimitError, + isMultipartFieldValidationError: mockIsMultipartFieldValidationError, readFormDataWithLimit: mockReadFormData, readFileToBufferWithLimit: mockReadFile, })) @@ -62,11 +71,12 @@ vi.mock('@/lib/core/telemetry', () => ({ vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mockCapture })) +import { REFILTERED_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { OrchestrationError } from '@/lib/core/orchestration/types' import { KnowledgeUsageLimitExceededError } from '@/lib/knowledge/application/billing' import { MAX_KNOWLEDGE_DOCUMENT_FILE_SIZE } from '@/lib/uploads/shared/types' import { validateFileType } from '@/lib/uploads/utils/validation' -import { POST } from '@/app/api/v2/knowledge/[id]/documents/route' +import { GET, POST } from '@/app/api/v2/knowledge/[id]/documents/route' const WORKSPACE_ID = 'workspace-1' const PRINCIPAL = { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' } as const @@ -85,6 +95,7 @@ describe('POST /api/v2/knowledge/[id]/documents', () => { v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) v2RouteMocks.gate.mockResolvedValue(null) mockIsPayloadSizeLimitError.mockReturnValue(false) + mockIsMultipartFieldValidationError.mockReturnValue(false) v2RouteMocks.authenticate.mockResolvedValue({ principal: PRINCIPAL, rolloutUserId: 'user-1', @@ -211,6 +222,24 @@ describe('POST /api/v2/knowledge/[id]/documents', () => { expect(mockPlatformUploaded).not.toHaveBeenCalled() }) + it('surfaces an unstorable multipart field as its own bad request', async () => { + const error = new Error( + 'Multipart file name for field "file" cannot contain a NUL character (U+0000)' + ) + mockReadFormData.mockRejectedValueOnce(error) + mockIsMultipartFieldValidationError.mockImplementation( + (candidate: unknown) => candidate === error + ) + + const response = await POST(buildRequest(), { params: Promise.resolve({ id: 'kb-1' }) }) + + expect(response.status).toBe(400) + expect(await response.json()).toEqual({ + error: { code: 'BAD_REQUEST', message: error.message }, + }) + expect(mockUploadDocument).not.toHaveBeenCalled() + }) + it('preserves bounded multipart rejection and stops before the upload operation', async () => { const error = new Error('knowledge document upload body exceeds maximum size') mockReadFormData.mockRejectedValueOnce(error) @@ -301,3 +330,121 @@ describe('POST /api/v2/knowledge/[id]/documents', () => { expect(mockCapture).not.toHaveBeenCalled() }) }) + +describe('GET /api/v2/knowledge/[id]/documents', () => { + const document = { + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'support.txt', + fileUrl: 's3://workspace/support.txt', + fileSize: 5, + mimeType: 'text/plain', + processingStatus: 'completed', + chunkCount: 1, + tokenCount: 2, + characterCount: 5, + enabled: true, + uploadedAt: new Date('2024-01-01T00:00:00Z'), + } + + function listRequest(query: string) { + return new NextRequest(`http://localhost/api/v2/knowledge/kb-1/documents?${query}`, { + headers: { 'x-api-key': 'secret' }, + }) + } + + function list(query: string) { + return GET(listRequest(query), { params: Promise.resolve({ id: 'kb-1' }) }) + } + + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue({ + principal: PRINCIPAL, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:key-1', 'user:user-1'], + rateLimitSubscription: null, + keyType: 'personal', + }) + mockListDocuments.mockResolvedValue({ + documents: [document], + tagDefinitions: [], + pagination: { hasMore: true, offset: 0, limit: 1 }, + }) + }) + + /** + * An offset cursor is the weaker scheme: replayed under a different filter it + * names an ordinal in an unrelated sequence. Pins the binding end-to-end — the + * mint in `present` and the read in `mapInput` — because the contract-level + * sweep only checks a hand-maintained map of param names and stays green when + * a route drops the stamp entirely. + */ + it('refuses a cursor minted under a different filter', async () => { + const minted = await list(`workspaceId=${WORKSPACE_ID}&limit=1&search=support`) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mockListDocuments.mockClear() + const replayed = await list( + `workspaceId=${WORKSPACE_ID}&limit=1&search=billing&cursor=${encodeURIComponent(nextCursor)}` + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mockListDocuments).not.toHaveBeenCalled() + }) + + /** + * `tagFilters` binds through the contract's parser, so spellings that parse to + * one filter share one scope. The schema defaults `operator` to `eq` and AND + * is commutative, so omitting the operator, stating it, and reordering the + * clauses all name the same sequence and must all resume. + */ + it.each([ + [ + 'the default operator stated explicitly', + '[{"tagName":"a","value":"1","operator":"eq"},{"tagName":"b","value":"2","operator":"eq"}]', + ], + [ + 'a fieldType the resolver overrides with the stored definition', + '[{"tagName":"a","value":"1","fieldType":"text"},{"tagName":"b","value":"2","fieldType":"text"}]', + ], + ['the clauses reordered', '[{"tagName":"b","value":"2"},{"tagName":"a","value":"1"}]'], + ])('resumes a tag-filter cursor with %s', async (_label, replayFilters) => { + const mintFilters = '[{"tagName":"a","value":"1"},{"tagName":"b","value":"2"}]' + const minted = await list( + `workspaceId=${WORKSPACE_ID}&limit=1&tagFilters=${encodeURIComponent(mintFilters)}` + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mockListDocuments.mockClear() + const resumed = await list( + `workspaceId=${WORKSPACE_ID}&limit=1&tagFilters=${encodeURIComponent(replayFilters)}&cursor=${encodeURIComponent(nextCursor)}` + ) + + expect(resumed.status).toBe(200) + expect(mockListDocuments).toHaveBeenCalled() + }) + + it('resumes a cursor replayed under the filters it was minted with', async () => { + const minted = await list(`workspaceId=${WORKSPACE_ID}&limit=1&search=support`) + const { nextCursor } = await minted.json() + + mockListDocuments.mockClear() + const resumed = await list( + `workspaceId=${WORKSPACE_ID}&limit=1&search=support&cursor=${encodeURIComponent(nextCursor)}` + ) + + expect(resumed.status).toBe(200) + expect(mockListDocuments).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: expect.objectContaining({ search: 'support', offset: 1 }), + request: expect.anything(), + }) + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/route.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/route.ts index 6ba62a1a941..436567fe845 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/route.ts @@ -1,8 +1,11 @@ +import { omit } from '@sim/utils/object' import { - type V2KnowledgeDocumentSummary, + parseV2KnowledgeTagFiltersParam, + v2BulkUpdateKnowledgeDocumentsContract, v2ListKnowledgeDocumentsContract, v2UploadKnowledgeDocumentContract, } from '@/lib/api/contracts/v2/knowledge' +import { cursorRoute, cursorScopeKey, unorderedScopeOf } from '@/lib/api/cursor-binding' import { defineV2BodyLifecycleRoute, defineV2JsonRoute, @@ -12,6 +15,7 @@ import { import { OrchestrationError } from '@/lib/core/orchestration/types' import { PlatformEvents } from '@/lib/core/telemetry' import { + isMultipartFieldValidationError, isPayloadSizeLimitError, MAX_MULTIPART_OVERHEAD_BYTES, readFileToBufferWithLimit, @@ -20,6 +24,7 @@ import { import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { admitKnowledgeDocumentUpload, + bulkUpdateKnowledgeDocuments, listKnowledgeDocuments, uploadKnowledgeDocument, } from '@/lib/knowledge/application/documents' @@ -28,40 +33,40 @@ import { KnowledgeDocumentUnsupportedMediaTypeError } from '@/lib/knowledge/appl import { captureServerEvent } from '@/lib/posthog/server' import { MAX_KNOWLEDGE_DOCUMENT_FILE_SIZE } from '@/lib/uploads/shared/types' import { validateFileType } from '@/lib/uploads/utils/validation' -import { serializeDate } from '@/app/api/v1/knowledge/utils' -import { decodeOffsetCursor, encodeCursor } from '@/app/api/v2/lib/response' +import { toV2DocumentSummary, toV2TaggedDocument } from '@/app/api/v2/knowledge/utils' +import { cursorSortKey, decodeOffsetCursor, encodeOffsetCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 const MAX_FILE_SIZE = MAX_KNOWLEDGE_DOCUMENT_FILE_SIZE -function toV2DocumentSummary(document: { - id: string - knowledgeBaseId: string - filename: string - fileSize: number - mimeType: string - processingStatus?: 'pending' | 'processing' | 'completed' | 'failed' - chunkCount: number - tokenCount: number - characterCount: number - enabled: boolean - uploadedAt: Date -}): V2KnowledgeDocumentSummary { - return { - id: document.id, - knowledgeBaseId: document.knowledgeBaseId, - filename: document.filename, - fileSize: document.fileSize, - mimeType: document.mimeType, - processingStatus: document.processingStatus ?? 'pending', - chunkCount: document.chunkCount, - tokenCount: document.tokenCount, - characterCount: document.characterCount, - enabled: document.enabled, - createdAt: serializeDate(document.uploadedAt), - } +/** + * Every param that changes which documents, in which order, this list returns. + * + * `tagFilters` binds through the contract's parser, not the raw query text: the + * schema defaults `operator` to `eq`, so `{tagName}` and `{tagName, operator}` + * are one filter to the query and must be one scope to the cursor. An + * unparseable value binds raw — that request is about to 400 anyway. + * + * `fieldType` is dropped: `resolveKnowledgeTagFilters` builds every structured + * filter with the stored definition's type and never reads the caller's, so + * stating it or omitting it selects the same documents. A scope part the query + * ignores refuses a cursor for a page that did not move. + */ +function documentCursorFilters( + knowledgeBaseId: string, + query: { workspaceId: string; enabledFilter?: string; search?: string; tagFilters?: string } +) { + const parsed = parseV2KnowledgeTagFiltersParam(query.tagFilters) + return cursorScopeKey(cursorRoute(v2ListKnowledgeDocumentsContract, { id: knowledgeBaseId }), { + workspaceId: query.workspaceId, + enabledFilter: query.enabledFilter, + search: query.search, + tagFilters: parsed.success + ? unorderedScopeOf(parsed.filters?.map((filter) => omit(filter, ['fieldType']))) + : query.tagFilters, + }) } /** GET /api/v2/knowledge/[id]/documents — List documents in a knowledge base. */ @@ -71,25 +76,86 @@ export const GET = defineV2JsonRoute({ operation: knowledgeOperations.listDocuments, rateLimit: v2RateLimits.publicApi, errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, - mapInput: ({ params, query }) => ({ - knowledgeBaseId: params.id, - assertedWorkspaceId: query.workspaceId, - enabledFilter: query.enabledFilter, - search: query.search, - limit: query.limit, - offset: decodeOffsetCursor(query.cursor), - sortBy: query.sortBy, - sortOrder: query.sortOrder, - }), + mapInput: ({ params, query }) => { + const tagFilters = parseV2KnowledgeTagFiltersParam(query.tagFilters) + if (!tagFilters.success) { + throw new OrchestrationError('validation', tagFilters.message) + } + return { + knowledgeBaseId: params.id, + assertedWorkspaceId: query.workspaceId, + enabledFilter: query.enabledFilter, + search: query.search, + limit: query.limit, + offset: decodeOffsetCursor( + query.cursor, + cursorSortKey(query.sortBy, query.sortOrder), + documentCursorFilters(params.id, query) + ), + sortBy: query.sortBy, + sortOrder: query.sortOrder, + tagNameFilters: tagFilters.filters, + } + }, useCase: listKnowledgeDocuments, - present: ({ documents, pagination }) => ({ - data: documents.map(toV2DocumentSummary), + present: ({ documents, tagDefinitions, pagination }, { params, query }) => ({ + data: documents.map((document) => toV2TaggedDocument(document, tagDefinitions)), nextCursor: pagination.hasMore - ? encodeCursor({ offset: pagination.offset + pagination.limit }) + ? encodeOffsetCursor( + cursorSortKey(query.sortBy, query.sortOrder), + documentCursorFilters(params.id, query), + pagination.offset + pagination.limit + ) : null, }), }) +/** + * PATCH /api/v2/knowledge/[id]/documents — Enable or disable many documents. + * + * Enable and disable only. Bulk delete is deliberately not offered: the bulk + * operation records no semantic audit, so a public bulk delete would empty a + * knowledge base leaving no `DOCUMENT_DELETED` entries, while the per-document + * DELETE audits every one. + */ +export const PATCH = defineV2JsonRoute({ + contract: v2BulkUpdateKnowledgeDocumentsContract, + auth: v2ApiKeyAuth, + operation: knowledgeOperations.bulkDocuments, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, + mapInput: ({ params, body }) => ({ + knowledgeBaseId: params.id, + assertedWorkspaceId: body.workspaceId, + operation: body.operation, + documentIds: body.documentIds, + selectAll: body.selectAll, + enabledFilter: body.enabledFilter, + }), + useCase: bulkUpdateKnowledgeDocuments, + present: (result) => { + if (result.operation === 'delete') { + throw new Error('Bulk knowledge document delete is not exposed on the public API') + } + /** + * `documentIds` is echoed only for an explicit-list request, which the body + * bounds. A `selectAll` request has no such bound: a knowledge base with + * 100k documents would otherwise materialize and element-wise validate a + * multi-megabyte identifier array nobody asked for. That caller reads + * `updatedCount` and re-lists if it needs the identifiers. + */ + return { + data: { + operation: result.operation, + updatedCount: result.successCount, + documentIds: result.selectAll + ? undefined + : result.updatedDocuments.map((document) => document.id), + }, + } + }, +}) + /** POST /api/v2/knowledge/[id]/documents — Upload a document to a knowledge base. */ export const POST = defineV2BodyLifecycleRoute({ contract: v2UploadKnowledgeDocumentContract, @@ -113,6 +179,9 @@ export const POST = defineV2BodyLifecycleRoute({ }) } catch (error) { if (isPayloadSizeLimitError(error)) throw error + if (isMultipartFieldValidationError(error)) { + throw new OrchestrationError('validation', error.message) + } throw new OrchestrationError('validation', 'Request body must be valid multipart form data') } diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route.test.ts index e1db73c90fe..96936470e88 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route.test.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route.test.ts @@ -69,7 +69,6 @@ vi.mock('@/app/api/v2/knowledge/[id]/documents/uploads/utils', () => ({ } : null, }), - v2KnowledgeDocumentUploadError: vi.fn(() => null), })) import { POST } from '@/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route' diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route.ts index aa9bf3e6751..930e3ec1d73 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route.ts @@ -1,20 +1,18 @@ import { v2CompleteKnowledgeDocumentUploadContract } from '@/lib/api/contracts/v2/knowledge' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { PlatformEvents } from '@/lib/core/telemetry' +import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { completeKnowledgeDocumentUpload } from '@/lib/knowledge/application/upload-sessions' import { captureServerEvent } from '@/lib/posthog/server' -import { - toV2KnowledgeDocumentUpload, - v2KnowledgeDocumentUploadError, -} from '@/app/api/v2/knowledge/[id]/documents/uploads/utils' +import { toV2KnowledgeDocumentUpload } from '@/app/api/v2/knowledge/[id]/documents/uploads/utils' export const POST = defineV2JsonRoute({ contract: v2CompleteKnowledgeDocumentUploadContract, auth: v2ApiKeyAuth, operation: knowledgeOperations.uploadComplete, rateLimit: v2RateLimits.publicApi, - errorPolicy: { render: v2KnowledgeDocumentUploadError }, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseUploadAuthorization, mapInput: ({ params, query, headers }) => ({ knowledgeBaseId: params.id, assertedWorkspaceId: query.workspaceId, diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/parts/route.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/parts/route.ts index 6640972b06b..8c7121c2af2 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/parts/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/parts/route.ts @@ -1,15 +1,15 @@ import { v2CreateKnowledgeDocumentUploadPartUrlsContract } from '@/lib/api/contracts/v2/knowledge' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { issueKnowledgeDocumentUploadParts } from '@/lib/knowledge/application/upload-sessions' -import { v2KnowledgeDocumentUploadError } from '@/app/api/v2/knowledge/[id]/documents/uploads/utils' export const POST = defineV2JsonRoute({ contract: v2CreateKnowledgeDocumentUploadPartUrlsContract, auth: v2ApiKeyAuth, operation: knowledgeOperations.uploadParts, rateLimit: v2RateLimits.publicApi, - errorPolicy: { render: v2KnowledgeDocumentUploadError }, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseUploadAuthorization, mapInput: ({ params, query, headers, body }) => ({ knowledgeBaseId: params.id, assertedWorkspaceId: query.workspaceId, diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/route.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/route.ts index 87194ec5d55..710eae7b8e5 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/route.ts @@ -1,18 +1,16 @@ import { v2AbortKnowledgeDocumentUploadContract } from '@/lib/api/contracts/v2/knowledge' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { cancelKnowledgeDocumentUpload } from '@/lib/knowledge/application/upload-sessions' -import { - toV2KnowledgeDocumentUpload, - v2KnowledgeDocumentUploadError, -} from '@/app/api/v2/knowledge/[id]/documents/uploads/utils' +import { toV2KnowledgeDocumentUpload } from '@/app/api/v2/knowledge/[id]/documents/uploads/utils' export const DELETE = defineV2JsonRoute({ contract: v2AbortKnowledgeDocumentUploadContract, auth: v2ApiKeyAuth, operation: knowledgeOperations.uploadCancel, rateLimit: v2RateLimits.publicApi, - errorPolicy: { render: v2KnowledgeDocumentUploadError }, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseUploadAuthorization, mapInput: ({ params, query, headers }) => ({ knowledgeBaseId: params.id, assertedWorkspaceId: query.workspaceId, diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/concealment.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/concealment.test.ts new file mode 100644 index 00000000000..b16d4788c37 --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/concealment.test.ts @@ -0,0 +1,194 @@ +/** + * @vitest-environment node + */ +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + authenticateV2ApiKey: vi.fn(), + cancel: vi.fn(), + checkRateLimitDirect: vi.fn(), + checkRateLimitDirectOrThrow: vi.fn(), + complete: vi.fn(), + create: vi.fn(), + gate: vi.fn(), + parts: vi.fn(), +})) + +function operation(id: string) { + return { id, minimumRole: 'write', workspaceApiKey: 'allow' } +} + +vi.mock('@/lib/knowledge/application/upload-sessions', () => ({ + KnowledgeDocumentUnsupportedMediaTypeError: class KnowledgeDocumentUnsupportedMediaTypeError extends Error {}, + createKnowledgeDocumentUpload: { + operation: operation('knowledge.documents.upload.create'), + execute: mocks.create, + }, + cancelKnowledgeDocumentUpload: { + operation: operation('knowledge.documents.upload.cancel'), + execute: mocks.cancel, + }, + issueKnowledgeDocumentUploadParts: { + operation: operation('knowledge.documents.upload.parts'), + execute: mocks.parts, + }, + completeKnowledgeDocumentUpload: { + operation: operation('knowledge.documents.upload.complete'), + execute: mocks.complete, + }, +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ + authenticateV2ApiKey: mocks.authenticateV2ApiKey, + V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, +})) + +vi.mock('@/lib/core/rate-limiter', () => ({ + getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), + RateLimiter: class RateLimiter { + checkRateLimitDirect = mocks.checkRateLimitDirect + checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow + }, +})) + +vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: vi.fn() })) + +import { + InsufficientWorkspacePermissionsError, + NoWorkspaceAccessError, +} from '@/lib/core/application' +import { POST as COMPLETE } from '@/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/complete/route' +import { POST as PARTS } from '@/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/parts/route' +import { DELETE as CANCEL } from '@/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/route' +import { POST as CREATE } from '@/app/api/v2/knowledge/[id]/documents/uploads/route' + +const WORKSPACE_ID = '6fc7631d-88cd-46f8-9f0a-d4764daef7f8' +const BASE = `http://localhost:3000/api/v2/knowledge/kb-1/documents/uploads` + +function context() { + return { params: Promise.resolve({ id: 'kb-1', uploadId: 'upload-1' }) } +} + +function controlHeaders() { + return { 'upload-token': 'token', 'x-api-key': 'secret' } +} + +/** + * Each entry pairs the route handler with the mocked use case behind it, so a + * case can make that one operation refuse and read the status the route + * renders. + */ +const routes = [ + { + name: 'create upload session', + useCase: mocks.create, + call: () => + CREATE( + new NextRequest(BASE, { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify({ + workspaceId: WORKSPACE_ID, + name: 'guide.pdf', + contentType: 'application/pdf', + size: 1024, + }), + }), + context() + ), + }, + { + name: 'abort upload session', + useCase: mocks.cancel, + call: () => + CANCEL( + new NextRequest(`${BASE}/upload-1?workspaceId=${WORKSPACE_ID}`, { + method: 'DELETE', + headers: controlHeaders(), + }), + context() + ), + }, + { + name: 'issue part urls', + useCase: mocks.parts, + call: () => + PARTS( + new NextRequest(`${BASE}/upload-1/parts?workspaceId=${WORKSPACE_ID}`, { + method: 'POST', + headers: { ...controlHeaders(), 'content-type': 'application/json' }, + body: JSON.stringify({ partNumbers: [1] }), + }), + context() + ), + }, + { + name: 'complete upload session', + useCase: mocks.complete, + call: () => + COMPLETE( + new NextRequest(`${BASE}/upload-1/complete?workspaceId=${WORKSPACE_ID}`, { + method: 'POST', + headers: controlHeaders(), + }), + context() + ), + }, +] as const + +/** + * The four knowledge upload routes are the only knowledge routes naming a + * knowledge base whose failures were not concealed, and their ordering made the + * gap an oracle: the use case resolves the knowledge-base context — which throws + * `not_found` when the base is absent *or* lives in another workspace — before + * workspace authorization runs. So an unconcealed 403 meant "this base exists in + * a workspace you cannot reach" and a 404 meant "it does not exist", while + * `GET /api/v2/knowledge/{id}` answers 404 to both. + */ +describe('v2 knowledge upload resource concealment', () => { + beforeEach(() => { + vi.clearAllMocks() + mocks.authenticateV2ApiKey.mockResolvedValue({ + principal: { kind: 'personal_api_key' as const, userId: 'user-1', keyId: 'key-1' }, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:key-1', 'user:user-1'], + rateLimitSubscription: null, + keyType: 'personal', + }) + mocks.gate.mockResolvedValue(null) + for (const limiter of [mocks.checkRateLimitDirect, mocks.checkRateLimitDirectOrThrow]) { + limiter.mockResolvedValue({ + allowed: true, + remaining: 99, + resetAt: new Date('2026-08-04T21:00:00.000Z'), + }) + } + }) + + it.each(routes)( + '$name reports a cross-tenant refusal as a missing knowledge base', + async ({ useCase, call }) => { + useCase.mockRejectedValue(new NoWorkspaceAccessError()) + + const response = await call() + + expect(response.status).toBe(404) + await expect(response.json()).resolves.toEqual({ + error: { code: 'NOT_FOUND', message: 'Knowledge base not found' }, + }) + } + ) + + it.each(routes)( + '$name still reports a same-workspace role denial as forbidden', + async ({ useCase, call }) => { + useCase.mockRejectedValue(new InsufficientWorkspacePermissionsError()) + + const response = await call() + + expect(response.status).toBe(403) + } + ) +}) diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/control-routes.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/control-routes.test.ts index 8f90f4b030b..8a28f5c99c0 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/control-routes.test.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/control-routes.test.ts @@ -58,7 +58,6 @@ vi.mock('@/app/api/v2/knowledge/[id]/documents/uploads/utils', () => ({ error: null, document: null, }), - v2KnowledgeDocumentUploadError: vi.fn(() => null), })) import { POST as PARTS } from '@/app/api/v2/knowledge/[id]/documents/uploads/[uploadId]/parts/route' diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/route.test.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/route.test.ts index 3f2e59f50c0..b8ab41afce9 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/route.test.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/route.test.ts @@ -50,7 +50,6 @@ vi.mock('@/app/api/v2/knowledge/[id]/documents/uploads/utils', () => ({ error: null, document: null, }), - v2KnowledgeDocumentUploadError: vi.fn(() => null), })) import { POST } from '@/app/api/v2/knowledge/[id]/documents/uploads/route' diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/route.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/route.ts index 03f1ea7289d..aaf70318147 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/route.ts @@ -1,18 +1,16 @@ import { v2CreateKnowledgeDocumentUploadContract } from '@/lib/api/contracts/v2/knowledge' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { createKnowledgeDocumentUpload } from '@/lib/knowledge/application/upload-sessions' -import { - toV2KnowledgeDocumentUpload, - v2KnowledgeDocumentUploadError, -} from '@/app/api/v2/knowledge/[id]/documents/uploads/utils' +import { toV2KnowledgeDocumentUpload } from '@/app/api/v2/knowledge/[id]/documents/uploads/utils' export const POST = defineV2JsonRoute({ contract: v2CreateKnowledgeDocumentUploadContract, auth: v2ApiKeyAuth, operation: knowledgeOperations.uploadCreate, rateLimit: v2RateLimits.publicApi, - errorPolicy: { render: v2KnowledgeDocumentUploadError }, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseUploadAuthorization, mapInput: ({ params, body }) => { const { workspaceId, name, contentType, size, ...metadata } = body return { diff --git a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/utils.ts b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/utils.ts index fbf8e030418..aedafc82f46 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/utils.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/documents/uploads/utils.ts @@ -1,42 +1,7 @@ -import type { NextResponse } from 'next/server' -import type { - V2KnowledgeDocumentSummary, - V2KnowledgeDocumentUpload, -} from '@/lib/api/contracts/v2/knowledge' -import { KnowledgeUsageLimitExceededError } from '@/lib/knowledge/application/billing' -import { KnowledgeDocumentUnsupportedMediaTypeError } from '@/lib/knowledge/application/upload-sessions' +import type { V2KnowledgeDocumentUpload } from '@/lib/api/contracts/v2/knowledge' import type { CreatedKnowledgeDocument } from '@/lib/knowledge/orchestration/documents' import type { UploadSessionRecord } from '@/lib/uploads/upload-session/service' -import { serializeDate } from '@/app/api/v1/knowledge/utils' -import { v2CaughtOrchestrationError, v2Error } from '@/app/api/v2/lib/response' - -export function v2KnowledgeDocumentUploadError(error: unknown): NextResponse | null { - if (error instanceof KnowledgeDocumentUnsupportedMediaTypeError) { - return v2Error('UNSUPPORTED_MEDIA_TYPE', error.message) - } - if (error instanceof KnowledgeUsageLimitExceededError) { - return v2Error('USAGE_LIMIT_EXCEEDED', error.message) - } - return v2CaughtOrchestrationError(error) -} - -export function toV2KnowledgeDocumentSummary( - document: CreatedKnowledgeDocument -): V2KnowledgeDocumentSummary { - return { - id: document.id, - knowledgeBaseId: document.knowledgeBaseId, - filename: document.filename, - fileSize: document.fileSize, - mimeType: document.mimeType, - processingStatus: document.processingStatus ?? 'pending', - chunkCount: document.chunkCount, - tokenCount: document.tokenCount, - characterCount: document.characterCount, - enabled: document.enabled, - createdAt: serializeDate(document.uploadedAt), - } -} +import { toV2DocumentSummary } from '@/app/api/v2/knowledge/utils' export function toV2KnowledgeDocumentUpload( session: UploadSessionRecord, @@ -54,6 +19,6 @@ export function toV2KnowledgeDocumentUpload( size: session.fileSize, expiresAt: session.expiresAt.toISOString(), error: session.error, - document: document ? toV2KnowledgeDocumentSummary(document) : null, + document: document ? toV2DocumentSummary(document) : null, } } diff --git a/apps/sim/app/api/v2/knowledge/[id]/route.ts b/apps/sim/app/api/v2/knowledge/[id]/route.ts index b9bcee78298..6e2fdf65ba3 100644 --- a/apps/sim/app/api/v2/knowledge/[id]/route.ts +++ b/apps/sim/app/api/v2/knowledge/[id]/route.ts @@ -13,7 +13,6 @@ import { } from '@/lib/knowledge/application/knowledge-bases' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { toV2KnowledgeBase } from '@/app/api/v2/knowledge/utils' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -42,9 +41,6 @@ export const PATCH = defineV2JsonRoute({ operation: knowledgeOperations.update, rateLimit: v2RateLimits.publicApi, errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, - parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), - }, mapInput: ({ params, body }) => ({ knowledgeBaseId: params.id, assertedWorkspaceId: body.workspaceId, diff --git a/apps/sim/app/api/v2/knowledge/[id]/tags/route.test.ts b/apps/sim/app/api/v2/knowledge/[id]/tags/route.test.ts new file mode 100644 index 00000000000..d6ab7343cc9 --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/tags/route.test.ts @@ -0,0 +1,107 @@ +/** + * @vitest-environment node + */ +import { + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mockListTags } = vi.hoisted(() => ({ + mockListTags: vi.fn(), +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +vi.mock('@/lib/knowledge/application/tags', () => ({ + listKnowledgeTags: { operation: { id: 'knowledge.tags.list' }, execute: mockListTags }, +})) + +import { knowledgeOperations } from '@/lib/knowledge/application/operations' +import { GET } from '@/app/api/v2/knowledge/[id]/tags/route' + +const WORKSPACE_ID = 'workspace-1' +const PRINCIPAL = { kind: 'workspace_api_key', workspaceId: WORKSPACE_ID, keyId: 'key-1' } as const + +function buildRequest(query = `?workspaceId=${WORKSPACE_ID}`) { + return new NextRequest(`http://localhost/api/v2/knowledge/kb-1/tags${query}`, { + headers: { 'x-api-key': 'secret' }, + }) +} + +const context = { params: Promise.resolve({ id: 'kb-1' }) } + +describe('GET /api/v2/knowledge/[id]/tags', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue({ + principal: PRINCIPAL, + rolloutUserId: 'billing-owner', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], + rateLimitSubscription: null, + keyType: 'workspace', + }) + mockListTags.mockResolvedValue({ + tagDefinitions: [ + { + id: 'tag-def-1', + knowledgeBaseId: 'kb-1', + tagSlot: 'tag1', + displayName: 'category', + fieldType: 'text', + createdAt: new Date('2025-01-10T09:00:00Z'), + updatedAt: new Date('2025-01-10T09:00:00Z'), + }, + ], + }) + }) + + it('returns the tag vocabulary as a full-set list', async () => { + const response = await GET(buildRequest(), context) + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: [{ displayName: 'category', tagSlot: 'tag1', fieldType: 'text' }], + nextCursor: null, + }) + expect(mockListTags).toHaveBeenCalledWith( + expect.objectContaining({ + principal: PRINCIPAL, + input: { knowledgeBaseId: 'kb-1', assertedWorkspaceId: WORKSPACE_ID }, + }) + ) + expect(response.headers.get('cache-control')).toBe('private, no-store') + }) + + it('does not publish the tag definition identifier or its timestamps', async () => { + const response = await GET(buildRequest(), context) + + const [tag] = (await response.json()).data + expect(Object.keys(tag).sort()).toEqual(['displayName', 'fieldType', 'tagSlot']) + }) + + it('requires the workspace scope', async () => { + const response = await GET(buildRequest(''), context) + + expect(response.status).toBe(400) + expect(mockListTags).not.toHaveBeenCalled() + }) + + it('is reachable by a workspace API key, like its sibling knowledge reads', () => { + expect(knowledgeOperations.listTags.workspaceApiKey).toBe('allow') + expect(knowledgeOperations.listTags.principalKinds).toContain('workspace_api_key') + expect(knowledgeOperations.listTags.workspaceApiKey).toBe( + knowledgeOperations.listDocuments.workspaceApiKey + ) + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/[id]/tags/route.ts b/apps/sim/app/api/v2/knowledge/[id]/tags/route.ts new file mode 100644 index 00000000000..fadf77fe83a --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/[id]/tags/route.ts @@ -0,0 +1,35 @@ +import { v2ListKnowledgeTagsContract } from '@/lib/api/contracts/v2/knowledge' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' +import { knowledgeOperations } from '@/lib/knowledge/application/operations' +import { listKnowledgeTags } from '@/lib/knowledge/application/tags' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * GET /api/v2/knowledge/[id]/tags — List the knowledge base's tag vocabulary. + * + * Full-set list: a knowledge base has a fixed number of tag slots, so the whole + * vocabulary is one page and `nextCursor` is always null. + */ +export const GET = defineV2JsonRoute({ + contract: v2ListKnowledgeTagsContract, + auth: v2ApiKeyAuth, + operation: knowledgeOperations.listTags, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseAuthorization, + mapInput: ({ params, query }) => ({ + knowledgeBaseId: params.id, + assertedWorkspaceId: query.workspaceId, + }), + useCase: listKnowledgeTags, + present: ({ tagDefinitions }) => ({ + data: tagDefinitions.map((definition) => ({ + displayName: definition.displayName, + tagSlot: definition.tagSlot, + fieldType: definition.fieldType, + })), + nextCursor: null, + }), +}) diff --git a/apps/sim/app/api/v2/knowledge/folders/route.ts b/apps/sim/app/api/v2/knowledge/folders/route.ts index 45887f4c4b9..8392f343e68 100644 --- a/apps/sim/app/api/v2/knowledge/folders/route.ts +++ b/apps/sim/app/api/v2/knowledge/folders/route.ts @@ -18,7 +18,6 @@ import { relocateKnowledgeFolder, } from '@/lib/knowledge/application/folders' import { knowledgeOperations } from '@/lib/knowledge/application/operations' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -49,9 +48,6 @@ export const POST = defineV2JsonRoute({ operation: knowledgeOperations.createFolder, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), - }, mapInput: ({ body }) => ({ workspaceId: body.workspaceId, path: body.path, source: 'api' }), useCase: createKnowledgeFolder, present: ({ folder }) => ({ data: toFolderPathView(folder, folder.path) }), @@ -63,9 +59,6 @@ export const PATCH = defineV2JsonRoute({ operation: knowledgeOperations.relocateFolder, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), - }, mapInput: ({ body }) => ({ workspaceId: body.workspaceId, path: body.path, diff --git a/apps/sim/app/api/v2/knowledge/route.test.ts b/apps/sim/app/api/v2/knowledge/route.test.ts index 22084ec7aca..8e2abe97bcb 100644 --- a/apps/sim/app/api/v2/knowledge/route.test.ts +++ b/apps/sim/app/api/v2/knowledge/route.test.ts @@ -59,6 +59,8 @@ vi.mock('@/lib/users/queries', () => ({ requireResolvedUserEmail: (emails: Map, userId: string) => emails.get(userId)!, })) +import { V2_DEFAULT_PAGE_SIZE } from '@/lib/api/contracts/v2/shared' +import { REFILTERED_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { GET, POST } from '@/app/api/v2/knowledge/route' const WORKSPACE_ID = 'workspace-1' @@ -104,6 +106,9 @@ describe('/api/v2/knowledge route composition', () => { mockGetUserEmailsByIds.mockResolvedValue(new Map([['user-1', 'owner@example.com']])) mockList.mockResolvedValue({ knowledgeBases: [{ knowledgeBase: buildKnowledgeBase(), folderPath: '/' }], + nextCursorKeys: null, + sortBy: 'name', + sortOrder: 'desc', }) mockCreate.mockResolvedValue({ knowledgeBase: buildKnowledgeBase(), folderPath: '/' }) }) @@ -125,6 +130,8 @@ describe('/api/v2/knowledge route composition', () => { search: 'support', sortBy: 'name', sortOrder: 'desc', + limit: V2_DEFAULT_PAGE_SIZE, + cursorKeys: undefined, }, request, }) @@ -142,6 +149,76 @@ describe('/api/v2/knowledge route composition', () => { }) }) + /** + * Pins the binding end-to-end — the mint in `present` and the read in + * `mapInput` — because the contract-level sweep only checks a hand-maintained + * map of param names and stays green when a route drops the stamp entirely. + */ + it('refuses a cursor minted under a different filter', async () => { + mockList.mockResolvedValue({ + knowledgeBases: [{ knowledgeBase: buildKnowledgeBase(), folderPath: '/' }], + nextCursorKeys: ['Support docs', 'kb-1'], + sortBy: 'name', + sortOrder: 'desc', + }) + + const minted = await GET( + new NextRequest( + `http://localhost/api/v2/knowledge?workspaceId=${WORKSPACE_ID}&search=support`, + { headers: { 'x-api-key': 'secret' } } + ) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mockList.mockClear() + const replayed = await GET( + new NextRequest( + `http://localhost/api/v2/knowledge?workspaceId=${WORKSPACE_ID}&search=billing&cursor=${encodeURIComponent(nextCursor)}`, + { headers: { 'x-api-key': 'secret' } } + ) + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mockList).not.toHaveBeenCalled() + }) + + it('resumes a cursor replayed under the filters it was minted with', async () => { + mockList.mockResolvedValue({ + knowledgeBases: [{ knowledgeBase: buildKnowledgeBase(), folderPath: '/' }], + nextCursorKeys: ['Support docs', 'kb-1'], + sortBy: 'name', + sortOrder: 'desc', + }) + + const minted = await GET( + new NextRequest( + `http://localhost/api/v2/knowledge?workspaceId=${WORKSPACE_ID}&search=support`, + { headers: { 'x-api-key': 'secret' } } + ) + ) + const { nextCursor } = await minted.json() + + mockList.mockClear() + const resumed = await GET( + new NextRequest( + `http://localhost/api/v2/knowledge?workspaceId=${WORKSPACE_ID}&search=support&cursor=${encodeURIComponent(nextCursor)}`, + { headers: { 'x-api-key': 'secret' } } + ) + ) + + expect(resumed.status).toBe(200) + expect(mockList).toHaveBeenCalledWith({ + principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-1' }, + input: expect.objectContaining({ + search: 'support', + cursorKeys: ['Support docs', 'kb-1'], + }), + request: expect.anything(), + }) + }) + it('returns 201 and keeps human analytics on the personal-key actor', async () => { const request = new NextRequest('http://localhost/api/v2/knowledge', { method: 'POST', diff --git a/apps/sim/app/api/v2/knowledge/route.ts b/apps/sim/app/api/v2/knowledge/route.ts index f8f0ee2e4a5..8bfaabf2fcd 100644 --- a/apps/sim/app/api/v2/knowledge/route.ts +++ b/apps/sim/app/api/v2/knowledge/route.ts @@ -2,6 +2,7 @@ import { v2CreateKnowledgeBaseContract, v2ListKnowledgeBasesContract, } from '@/lib/api/contracts/v2/knowledge' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, @@ -16,11 +17,24 @@ import { import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { captureServerEvent } from '@/lib/posthog/server' import { toV2KnowledgeBase, toV2KnowledgeBases } from '@/app/api/v2/knowledge/utils' -import { v2Error } from '@/app/api/v2/lib/response' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Every param that changes which knowledge bases, in which order, this list returns. */ +function knowledgeCursorFilters(query: { + workspaceId: string + folderPath?: string + search?: string +}) { + return cursorScopeKey(cursorRoute(v2ListKnowledgeBasesContract), { + workspaceId: query.workspaceId, + folderPath: query.folderPath, + search: query.search, + }) +} + /** GET /api/v2/knowledge — List knowledge bases in a workspace. */ export const GET = defineV2JsonRoute({ contract: v2ListKnowledgeBasesContract, @@ -34,11 +48,23 @@ export const GET = defineV2JsonRoute({ search: query.search, sortBy: query.sortBy, sortOrder: query.sortOrder, + limit: query.limit, + cursorKeys: readSortedCursor( + query.cursor, + query.sortBy, + query.sortOrder, + knowledgeCursorFilters(query) + ), }), useCase: listKnowledgeBases, - present: async ({ knowledgeBases }) => ({ + present: async ({ knowledgeBases, nextCursorKeys }, { query }) => ({ data: await toV2KnowledgeBases(knowledgeBases), - nextCursor: null, + nextCursor: writeSortedCursor( + nextCursorKeys, + query.sortBy, + query.sortOrder, + knowledgeCursorFilters(query) + ), }), }) @@ -49,9 +75,6 @@ export const POST = defineV2JsonRoute({ operation: knowledgeOperations.create, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - parseOptions: { - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), - }, mapInput: ({ body }) => ({ workspaceId: body.workspaceId, name: body.name, diff --git a/apps/sim/app/api/v2/knowledge/search/route.test.ts b/apps/sim/app/api/v2/knowledge/search/route.test.ts index cb7816ec799..dcba7d0d593 100644 --- a/apps/sim/app/api/v2/knowledge/search/route.test.ts +++ b/apps/sim/app/api/v2/knowledge/search/route.test.ts @@ -25,6 +25,7 @@ vi.mock('@/lib/knowledge/application/search', () => ({ })) import { KnowledgeUsageLimitExceededError } from '@/lib/knowledge/application/billing' +import { DEFAULT_RERANKER_MODEL } from '@/lib/knowledge/reranker-models' import { POST, V2_KNOWLEDGE_SEARCH_MAX_BODY_BYTES } from '@/app/api/v2/knowledge/search/route' const WORKSPACE_ID = 'workspace-1' @@ -53,19 +54,23 @@ describe('POST /api/v2/knowledge/search', () => { mockSearch.mockResolvedValue({ results: [ { + embeddingId: 'embedding-1', + knowledgeBaseId: 'kb-1', documentId: 'doc-1', documentName: 'support.txt', sourceUrl: null, content: 'hello', chunkIndex: 0, - metadata: {}, + metadata: { category: 'billing' }, similarity: 0.9, + rerankerScore: 0.42, }, ], query: 'hello', knowledgeBaseIds: ['kb-1'], topK: 10, totalResults: 1, + rerankerStatus: 'applied', }) }) @@ -92,6 +97,9 @@ describe('POST /api/v2/knowledge/search', () => { topK: 10, tagFilters: undefined, searchMode: 'hybrid', + rerankerEnabled: undefined, + rerankerModel: DEFAULT_RERANKER_MODEL, + rerankerInputCount: undefined, }, request, }) @@ -102,6 +110,198 @@ describe('POST /api/v2/knowledge/search', () => { expect(response.headers.get('x-ratelimit-limit')).toBe('100') }) + it('names the source knowledge base and the reranker score on every result', async () => { + const response = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1', 'kb-2'], + query: 'hello', + topK: 10, + }) + ) + ) + + expect(response.status).toBe(200) + const body = await response.json() + expect(body.data.results[0]).toEqual({ + knowledgeBaseId: 'kb-1', + documentId: 'doc-1', + documentName: 'support.txt', + sourceUrl: null, + content: 'hello', + chunkIndex: 0, + metadata: { category: 'billing' }, + similarity: 0.9, + rerankerScore: 0.42, + }) + }) + + it('forwards reranker options and never a caller-supplied reranker key', async () => { + const response = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-fast', + rerankerInputCount: 40, + }) + ) + ) + + expect(response.status).toBe(200) + expect(mockSearch).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-fast', + rerankerInputCount: 40, + }), + }) + ) + const [{ input }] = mockSearch.mock.calls[0] + expect(input).not.toHaveProperty('rerankerApiKey') + expect(input).not.toHaveProperty('skipUsageBilling') + }) + + /** + * Without the default, `rerankerEnabled` alone satisfies the schema, fails the + * use case's model guard, and answers 200 in plain vector order — after paying + * for the widened candidate retrieval. + */ + it('defaults the reranker model so enabling reranking is enough to run it', async () => { + const response = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + }) + ) + ) + + expect(response.status).toBe(200) + expect(mockSearch).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + rerankerEnabled: true, + rerankerModel: DEFAULT_RERANKER_MODEL, + }), + }) + ) + }) + + it('reports on the wire that a requested reranker did not run', async () => { + mockSearch.mockResolvedValueOnce({ + results: [ + { + embeddingId: 'embedding-1', + knowledgeBaseId: 'kb-1', + documentId: 'doc-1', + documentName: 'support.txt', + sourceUrl: null, + content: 'hello', + chunkIndex: 0, + metadata: {}, + similarity: 0.9, + }, + ], + query: 'hello', + knowledgeBaseIds: ['kb-1'], + topK: 5, + totalResults: 1, + rerankerStatus: 'unavailable', + }) + + const response = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-pro', + }) + ) + ) + + expect(response.status).toBe(200) + const body = await response.json() + expect(body.data.rerankerStatus).toBe('unavailable') + expect(body.data.results[0]).not.toHaveProperty('rerankerScore') + }) + + it('rejects an unsupported reranker model and an out-of-range candidate pool', async () => { + const unsupportedModel = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-does-not-exist', + }) + ) + ) + const oversizedPool = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-fast', + rerankerInputCount: 101, + }) + ) + ) + + expect(unsupportedModel.status).toBe(400) + expect(oversizedPool.status).toBe(400) + expect(await oversizedPool.json()).toEqual({ + error: expect.objectContaining({ + code: 'BAD_REQUEST', + message: expect.stringContaining('rerankerInputCount cannot exceed 100'), + }), + }) + expect(mockSearch).not.toHaveBeenCalled() + }) + + /** + * The search body is strict, so an undeclared key is refused rather than + * stripped. That matters most for a bring-your-own reranker key: dropping it + * silently left the caller believing the secret it sent was in use. It + * matters for an ordinary mis-spelling too: a stripped `rerankerenabled` is a + * 200 with reranking off, and a stripped `topk` leaves `topK` at its default — + * both change what the search is billed. + */ + it('refuses a caller-supplied reranker key instead of silently dropping it', async () => { + const response = await POST( + buildRequest( + JSON.stringify({ + workspaceId: WORKSPACE_ID, + knowledgeBaseIds: ['kb-1'], + query: 'hello', + topK: 5, + rerankerEnabled: true, + rerankerModel: 'rerank-v4.0-fast', + rerankerApiKey: 'secret-byok-key', + }) + ) + ) + + expect(response.status).toBe(400) + expect(mockSearch).not.toHaveBeenCalled() + }) + it('forwards an opted-in hybrid search mode to the application use case', async () => { const response = await POST( buildRequest( diff --git a/apps/sim/app/api/v2/knowledge/search/route.ts b/apps/sim/app/api/v2/knowledge/search/route.ts index 070e1342015..9ee4f6c9244 100644 --- a/apps/sim/app/api/v2/knowledge/search/route.ts +++ b/apps/sim/app/api/v2/knowledge/search/route.ts @@ -3,7 +3,6 @@ import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/ import { v2KnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies' import { knowledgeOperations } from '@/lib/knowledge/application/operations' import { searchKnowledge } from '@/lib/knowledge/application/search' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -24,7 +23,6 @@ export const POST = defineV2JsonRoute({ errorPolicy: v2KnowledgeErrorPolicies.concealKnowledgeBaseUsageAuthorization, parseOptions: { maxBodyBytes: V2_KNOWLEDGE_SEARCH_MAX_BODY_BYTES, - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), }, mapInput: ({ body }) => ({ workspaceId: body.workspaceId, @@ -35,7 +33,26 @@ export const POST = defineV2JsonRoute({ topK: body.topK, tagFilters: body.tagFilters, searchMode: body.searchMode, + rerankerEnabled: body.rerankerEnabled, + rerankerModel: body.rerankerModel, + rerankerInputCount: body.rerankerInputCount, }), useCase: searchKnowledge, - present: (result) => ({ data: result }), + /** + * Projected field by field rather than spread. The use-case result also + * carries `userId`, `workspaceId`, a `cost` breakdown with pricing internals, + * and a live resolved-secret trace registry; only Zod's default key-stripping + * keeps them off the wire today, so a single loosened or opaque field in the + * response schema would ship them. + */ + present: (result) => ({ + data: { + results: result.results, + query: result.query, + knowledgeBaseIds: result.knowledgeBaseIds, + topK: result.topK, + totalResults: result.totalResults, + rerankerStatus: result.rerankerStatus, + }, + }), }) diff --git a/apps/sim/app/api/v2/knowledge/utils.test.ts b/apps/sim/app/api/v2/knowledge/utils.test.ts new file mode 100644 index 00000000000..9ccf8818312 --- /dev/null +++ b/apps/sim/app/api/v2/knowledge/utils.test.ts @@ -0,0 +1,92 @@ +/** + * @vitest-environment node + */ + +import { describe, expect, it } from 'vitest' +import { v2KnowledgeTaggedDocumentSchema } from '@/lib/api/contracts/v2/knowledge' +import type { DocumentTagDefinition } from '@/lib/knowledge/tags/types' +import { + toV2DocumentSummary, + toV2DocumentTags, + toV2TaggedDocument, +} from '@/app/api/v2/knowledge/utils' + +const uploadedAt = new Date('2026-08-01T00:00:00.000Z') + +const documentRow = { + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'invoice.pdf', + fileSize: 1024, + mimeType: 'application/pdf', + processingStatus: 'completed', + chunkCount: 4, + tokenCount: 512, + characterCount: 2048, + enabled: true, + uploadedAt, + tag1: 'billing', + number1: 7, +} + +const tagDefinitions: DocumentTagDefinition[] = [ + { + id: 'def-1', + knowledgeBaseId: 'kb-1', + tagSlot: 'tag1', + displayName: 'category', + fieldType: 'text', + createdAt: uploadedAt, + updatedAt: uploadedAt, + } as DocumentTagDefinition, +] + +describe('toV2DocumentSummary', () => { + it('serializes the shared document fields', () => { + expect(toV2DocumentSummary(documentRow)).toEqual({ + id: 'doc-1', + knowledgeBaseId: 'kb-1', + filename: 'invoice.pdf', + fileSize: 1024, + mimeType: 'application/pdf', + processingStatus: 'completed', + chunkCount: 4, + tokenCount: 512, + characterCount: 2048, + enabled: true, + createdAt: '2026-08-01T00:00:00.000Z', + }) + }) + + it('returns a null createdAt for a document with no upload timestamp', () => { + expect(toV2DocumentSummary({ ...documentRow, uploadedAt: null }).createdAt).toBeNull() + }) + + it('reads an absent processing status as pending', () => { + expect(toV2DocumentSummary({ ...documentRow, processingStatus: null }).processingStatus).toBe( + 'pending' + ) + }) +}) + +describe('toV2TaggedDocument', () => { + it('produces a contract-valid list item with tags keyed by display name', () => { + const projected = toV2TaggedDocument(documentRow, tagDefinitions) + expect(v2KnowledgeTaggedDocumentSchema.parse(projected)).toEqual(projected) + expect(projected.tags).toEqual({ category: 'billing', number1: 7 }) + }) + + it('does not throw when the document has no upload timestamp', () => { + const projected = toV2TaggedDocument({ ...documentRow, uploadedAt: null }, tagDefinitions) + expect(projected.createdAt).toBeNull() + expect(v2KnowledgeTaggedDocumentSchema.parse(projected)).toEqual(projected) + }) +}) + +describe('toV2DocumentTags', () => { + it('serializes a date-valued slot as an ISO string', () => { + expect(toV2DocumentTags({ date1: new Date('2026-08-02T00:00:00.000Z') }, [])).toEqual({ + date1: '2026-08-02T00:00:00.000Z', + }) + }) +}) diff --git a/apps/sim/app/api/v2/knowledge/utils.ts b/apps/sim/app/api/v2/knowledge/utils.ts index 373f7b2dbd4..583368db64b 100644 --- a/apps/sim/app/api/v2/knowledge/utils.ts +++ b/apps/sim/app/api/v2/knowledge/utils.ts @@ -1,6 +1,113 @@ -import type { V2KnowledgeBase } from '@/lib/api/contracts/v2/knowledge' +import type { + V2KnowledgeBase, + V2KnowledgeDocumentSummary, + V2KnowledgeTaggedDocument, +} from '@/lib/api/contracts/v2/knowledge' +import { ALL_TAG_SLOTS, type AllTagSlot } from '@/lib/knowledge/constants' +import type { DocumentTagDefinition } from '@/lib/knowledge/tags/types' import type { KnowledgeBaseWithCounts } from '@/lib/knowledge/types' import { getUserEmailsByIds, requireResolvedUserEmail } from '@/lib/users/queries' +import { serializeDate } from '@/app/api/v1/knowledge/utils' + +/** + * Projects a document's tag slots onto a map keyed by tag display name, the same + * projection knowledge search applies to its result `metadata`. A slot holding a + * value with no definition keeps its raw slot name rather than disappearing. + */ +export function toV2DocumentTags( + document: Partial>, + tagDefinitions: readonly DocumentTagDefinition[] +): Record { + const displayNameBySlot = new Map( + tagDefinitions.map((definition) => [definition.tagSlot, definition.displayName]) + ) + const tags: Record = {} + for (const slot of ALL_TAG_SLOTS) { + const value = document[slot] + if (value === null || value === undefined) continue + const key = displayNameBySlot.get(slot) ?? slot + if (value instanceof Date) { + tags[key] = value.toISOString() + } else if ( + typeof value === 'string' || + typeof value === 'number' || + typeof value === 'boolean' + ) { + tags[key] = value + } + } + return tags +} + +const PROCESSING_STATUSES = ['pending', 'processing', 'completed', 'failed'] as const + +type V2DocumentProcessingStatus = (typeof PROCESSING_STATUSES)[number] + +/** + * Narrows a stored processing status onto the published enum. An absent value + * reads as `pending`, matching the column default; an unrecognised one is a + * producer bug rather than a caller-reachable failure, so it throws. + */ +function toProcessingStatus(status: string | null | undefined): V2DocumentProcessingStatus { + if (status === null || status === undefined) return 'pending' + const known = PROCESSING_STATUSES.find((candidate) => candidate === status) + if (!known) throw new Error(`Unexpected knowledge document processing status: ${status}`) + return known +} + +/** + * The document columns every v2 document projection reads. `uploadedAt` is + * accepted as nullable because the column is nullable in storage. + */ +interface V2DocumentSummarySource { + id: string + knowledgeBaseId: string + filename: string + fileSize: number + mimeType: string + processingStatus?: string | null + chunkCount: number + tokenCount: number + characterCount: number + enabled: boolean + uploadedAt: Date | string | null | undefined +} + +/** + * The single v2 document summary projection. Every v2 document response — list + * item, upload acknowledgement, detail — is this shape plus its own extras, so + * the shared field set is serialized in exactly one place. + */ +export function toV2DocumentSummary(document: V2DocumentSummarySource): V2KnowledgeDocumentSummary { + return { + id: document.id, + knowledgeBaseId: document.knowledgeBaseId, + filename: document.filename, + fileSize: document.fileSize, + mimeType: document.mimeType, + processingStatus: toProcessingStatus(document.processingStatus), + chunkCount: document.chunkCount, + tokenCount: document.tokenCount, + characterCount: document.characterCount, + enabled: document.enabled, + createdAt: serializeDate(document.uploadedAt), + } +} + +interface V2TaggedDocumentSource + extends V2DocumentSummarySource, + Partial> {} + +/** Serializes a document summary with its tag values keyed by display name. */ +export function toV2TaggedDocument( + document: V2TaggedDocumentSource, + tagDefinitions: readonly DocumentTagDefinition[] +): V2KnowledgeTaggedDocument { + return { + ...toV2DocumentSummary(document), + tags: toV2DocumentTags(document, tagDefinitions), + } +} interface KnowledgeBaseWithFolder { knowledgeBase: KnowledgeBaseWithCounts diff --git a/apps/sim/app/api/v2/lib/folders.ts b/apps/sim/app/api/v2/lib/folders.ts index 294bb00771e..12991cb897d 100644 --- a/apps/sim/app/api/v2/lib/folders.ts +++ b/apps/sim/app/api/v2/lib/folders.ts @@ -1,53 +1,12 @@ import type { folder } from '@sim/db/schema' -import type { NextResponse } from 'next/server' -import type { FolderResourceType } from '@/lib/api/contracts/folders' -import type { OrchestrationErrorCode } from '@/lib/core/orchestration/types' -import { withFolderTreeLock } from '@/lib/folders/locks' import { type FolderPathIndex, isFolderPathEffectivelyLocked, - ROOT_FOLDER_PATH, toFolderPathView, } from '@/lib/folders/paths' -import { loadActiveFolderPathIndex } from '@/lib/folders/queries' -import { v2ErrorForOrchestration } from '@/app/api/v2/lib/response' type FolderRow = typeof folder.$inferSelect -export function resolveFolderPathId( - index: FolderPathIndex, - path: string -): string | null | undefined { - return path === ROOT_FOLDER_PATH ? null : index.idByPath.get(path) -} - -export type ResolvedFolderPathIdentity = - | { found: false } - | { found: true; folderId: string | null; index: FolderPathIndex } - -/** Resolves a path to its stable internal identity under a short-lived folder tree lock. */ -export async function resolveFolderPathIdentity(params: { - workspaceId: string - resourceType: FolderResourceType - path: string -}): Promise { - return withFolderTreeLock(params.workspaceId, params.resourceType, async (tx) => { - const index = await loadActiveFolderPathIndex(params.workspaceId, params.resourceType, tx) - const folderId = resolveFolderPathId(index, params.path) - return folderId === undefined ? { found: false } : { found: true, folderId, index } - }) -} - -export function folderPathForId( - index: FolderPathIndex, - folderId: string | null | undefined -): string { - if (!folderId) return ROOT_FOLDER_PATH - const path = index.pathById.get(folderId) - if (!path) throw new Error('Resource references an inactive or missing folder') - return path -} - export function toV2PathFolder( row: FolderRow, index: FolderPathIndex, @@ -58,10 +17,3 @@ export function toV2PathFolder( const base = toFolderPathView(row, path) return includeLocked ? { ...base, locked: isFolderPathEffectivelyLocked(index, row.id) } : base } - -export function v2FolderPathMutationError( - errorCode: OrchestrationErrorCode | undefined, - message: string -): NextResponse { - return v2ErrorForOrchestration(errorCode, message) -} diff --git a/apps/sim/app/api/v2/lib/response.test.ts b/apps/sim/app/api/v2/lib/response.test.ts new file mode 100644 index 00000000000..f84dd84a203 --- /dev/null +++ b/apps/sim/app/api/v2/lib/response.test.ts @@ -0,0 +1,48 @@ +/** + * @vitest-environment node + */ +import { describe, expect, it } from 'vitest' +import { v2Error } from '@/app/api/v2/lib/response' + +describe('v2Error retry guidance', () => { + it('sends Retry-After on 503 so a client does not retry a degraded dependency immediately', () => { + const response = v2Error('SERVICE_UNAVAILABLE', 'Service temporarily unavailable') + + expect(response.status).toBe(503) + const retryAfter = response.headers.get('Retry-After') + expect(retryAfter).not.toBeNull() + expect(Number(retryAfter)).toBeGreaterThan(0) + expect(Number.isInteger(Number(retryAfter))).toBe(true) + }) + + it('lets a caller-supplied Retry-After win over the default', () => { + const response = v2Error('SERVICE_UNAVAILABLE', 'Service temporarily unavailable', { + headers: { 'Retry-After': '30' }, + }) + + expect(response.headers.get('Retry-After')).toBe('30') + }) + + it('does not invent Retry-After for failures a retry cannot fix', () => { + for (const code of ['BAD_REQUEST', 'NOT_FOUND', 'FORBIDDEN', 'CONFLICT'] as const) { + expect(v2Error(code, 'nope').headers.get('Retry-After')).toBeNull() + } + }) + + it('does not default Retry-After on 429, whose wait comes from the token bucket', () => { + expect(v2Error('RATE_LIMITED', 'API rate limit exceeded').headers.get('Retry-After')).toBeNull() + }) + + it('stays silent on retrying when the outcome is unknown rather than absent', () => { + const response = v2Error( + 'SERVICE_UNAVAILABLE', + 'Async execution queue acceptance unconfirmed', + { + omitRetryAfter: true, + } + ) + + expect(response.status).toBe(503) + expect(response.headers.get('Retry-After')).toBeNull() + }) +}) diff --git a/apps/sim/app/api/v2/lib/response.ts b/apps/sim/app/api/v2/lib/response.ts index c0567024d65..8f5f61215b5 100644 --- a/apps/sim/app/api/v2/lib/response.ts +++ b/apps/sim/app/api/v2/lib/response.ts @@ -1,14 +1,17 @@ import { NextResponse } from 'next/server' import type { ZodError } from 'zod' +import { REFILTERED_CURSOR_MESSAGE, UNREADABLE_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { type CursorKey, INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' import { getValidationErrorMessage, serializeZodIssues } from '@/lib/api/server' +import { ADMISSION_RETRY_AFTER_SECONDS } from '@/lib/core/admission/transient-failure' +import { forbiddenErrorDetails } from '@/lib/core/application' import { asOrchestrationError, OrchestrationError, type OrchestrationErrorCode, } from '@/lib/core/orchestration/types' import type { HttpError } from '@/lib/core/utils/http-error' -import type { RateLimitResult, WorkspaceAccessError } from '@/app/api/v1/middleware' +import type { RateLimitResult } from '@/app/api/v1/middleware' /** * Runtime response helpers for the v2 API surface. Every v2 route renders its @@ -59,9 +62,47 @@ const V2_CODE_BY_HTTP_STATUS: Partial> = Object.from */ const PRIVATE_NO_STORE = { 'Cache-Control': 'private, no-store' } as const +/** + * Seconds a caller should wait before retrying a transient v2 failure, for the + * statuses whose response carries no other timing signal. + * + * Keyed on the response status rather than the v2 error code because + * `Retry-After` is defined against the status, and the status is the only half + * of the pair a client actually sees. `v2Error` lets a caller override the + * status independently of the code, so keying on the code would let the two + * disagree. + * + * RFC 9110 §10.2.3 singles out 503 as the status whose `Retry-After` means "how + * long the service is expected to be unavailable to the client", and §15.6.4 + * permits one. Note the requirement level is `MAY`, so this is a deliberate + * improvement on the baseline rather than a conformance fix: without it a + * client's only defensible policy on a 503 is an immediate retry, which is + * exactly the traffic a degraded dependency cannot absorb. Sim raises 503 when + * the API-key store, the rollout gate, the rate-limit backend, or + * execution-identity allocation is briefly unavailable, and all four are made + * worse by an unthrottled retry storm. + * + * 429 is deliberately absent because every 429 already knows its own wait: the + * throttle path measures it from the caller's token bucket + * ({@link v2RateLimitError}), and an admission denial carries the descriptor's + * declared `retryAfterSeconds` through to the route. Defaulting it here would + * paper over a path that had simply dropped its value — which is exactly the + * bug that used to leave a concurrency denial with no `Retry-After` at all. + * + * The value is Sim's one transient-failure floor, shared with the admission + * descriptors so the execute route's capacity 429 and every other surface's 503 + * cannot drift apart. It is a floor, not a schedule: a fleet that retries at + * exactly this offset re-converges into a single burst, so callers should still + * add jitter — `backoffWithJitter` from `@sim/utils/retry` is what Sim's own + * clients use. + */ +const RETRY_AFTER_SECONDS_BY_STATUS: Partial> = { + 503: ADMISSION_RETRY_AFTER_SECONDS, +} + type RateLimitHeaderSource = Pick -export function rateLimitHeaders(rateLimit?: RateLimitHeaderSource): Record { +function rateLimitHeaders(rateLimit?: RateLimitHeaderSource): Record { if (!rateLimit) return {} return { 'X-RateLimit-Limit': rateLimit.limit.toString(), @@ -80,6 +121,23 @@ function successHeaders(options: V2SuccessOptions): Record { return { ...PRIVATE_NO_STORE, ...rateLimitHeaders(options.rateLimit), ...options.headers } } +/** + * The bodiless 200 a `HEAD` receives from a route whose `GET` is not safe, once + * that `HEAD` has been authorized. + * + * RFC 9110 §9.3.2 lets Next alias `HEAD` onto `GET` only because §9.2.1 defines + * `HEAD` as safe — "essentially read-only". A `GET` that opens an outbound + * connection or writes a row breaks that assumption, and an uptime monitor + * walking the documented URL list would drive those effects on every probe. + * + * The 200 is unconditional **by construction**: callers must only reach this + * after `useCase.authorize` has resolved, or it becomes the existence oracle + * the `headSafe` option on the v2 route builders documents. + */ +export function v2HeadNoEffect(options: V2SuccessOptions = {}): NextResponse { + return new NextResponse(null, { status: options.status ?? 200, headers: successHeaders(options) }) +} + /** `{ data }` (+ rate-limit headers). */ export function v2Data(data: T, options: V2SuccessOptions = {}): NextResponse { return NextResponse.json( @@ -88,22 +146,23 @@ export function v2Data(data: T, options: V2SuccessOptions = {}): NextResponse ) } -/** `{ data, nextCursor }` (+ rate-limit headers). */ -export function v2CursorList( - data: T[], - nextCursor: string | null, - options: V2SuccessOptions = {} -): NextResponse { - return NextResponse.json( - { data, nextCursor }, - { status: options.status ?? 200, headers: successHeaders(options) } - ) -} - interface V2ErrorOptions { status?: number details?: unknown headers?: Record + /** + * Suppresses the code's default `Retry-After` for a failure whose outcome is + * *unknown* rather than *absent*. + * + * A 503 normally means the work did not happen, so "come back in 5 seconds" + * is safe advice. The async enqueue that could not be confirmed + * (`ASYNC_ENQUEUE_AMBIGUOUS`) is the exception: it deliberately retains its + * execution-ID claim because a job may already exist. Telling that caller to + * retry invites a client with no `X-Run-Id` to start a second run of the same + * workflow, which bills twice. It must reconcile against the run id the + * response returns instead, so the response stays silent on retrying. + */ + omitRetryAfter?: boolean } /** `{ error: { code, message, details? } }`. */ @@ -114,11 +173,19 @@ export function v2Error( ): NextResponse { const error: { code: V2ErrorCode; message: string; details?: unknown } = { code, message } if (options.details !== undefined) error.details = options.details + const status = options.status ?? STATUS_BY_CODE[code] + const retryAfterSeconds = options.omitRetryAfter + ? undefined + : RETRY_AFTER_SECONDS_BY_STATUS[status] return NextResponse.json( { error }, { - status: options.status ?? STATUS_BY_CODE[code], - headers: { ...PRIVATE_NO_STORE, ...options.headers }, + status, + headers: { + ...PRIVATE_NO_STORE, + ...(retryAfterSeconds === undefined ? {} : { 'Retry-After': retryAfterSeconds.toString() }), + ...options.headers, + }, } ) } @@ -130,6 +197,18 @@ export function v2HttpError(error: HttpError): NextResponse { return v2Error(code, error.message) } +/** + * The 500 of the local-storage upload data plane, in the canonical envelope. + * + * `PUT /api/v2/uploads/{uploadId}` and its `/parts/{partNumber}` sibling are + * undocumented but still v2 routes, and they do not run `admitV2Request`, so + * they cannot reuse the JSON builder's handler — this is the one piece of it + * they need. + */ +export function v2UploadDataPlaneError(): NextResponse { + return v2Error('INTERNAL_ERROR', 'Internal server error') +} + /** Render a contract `ZodError` as the v2 error envelope. */ export function v2ValidationError(error: ZodError): NextResponse { return v2Error('BAD_REQUEST', getValidationErrorMessage(error, 'Invalid request'), { @@ -137,11 +216,6 @@ export function v2ValidationError(error: ZodError): NextResponse { }) } -/** Render a shared {@link WorkspaceAccessError} as the v2 error envelope. */ -export function v2WorkspaceAccessError(failure: WorkspaceAccessError): NextResponse { - return v2Error(failure.code, failure.message, { status: failure.status }) -} - /** * Render a v1 rate-limit/auth failure (`checkRateLimit` result) as the v2 error * envelope: an auth failure becomes 401, a throttle becomes 429 with @@ -174,18 +248,50 @@ export function decodeCursor>(cursor: string): T | n } } +interface OffsetCursorPayload { + /** The ordering the offset counts positions within. */ + sort: string + /** Fingerprint of the list and filters the offset counts positions within. */ + filter: string + offset: number +} + +/** An offset cursor stamped with the sort and scope that produced it. */ +export function encodeOffsetCursor(sort: string, filter: string, offset: number): string { + return encodeCursor({ sort, filter, offset } satisfies OffsetCursorPayload) +} + /** - * Reads back an offset cursor minted by `encodeCursor({ offset })`. + * Reads back an offset cursor, refusing one minted under a different sort or + * different filters. * * An absent cursor means page one. A cursor that is not valid base64-JSON, or * that does not carry a non-negative integer `offset`, is rejected rather than * coerced to 0: silently restarting at page one while the caller believes it is - * paging forward makes a paging client loop over the first page forever. The v2 - * error policies render the thrown validation error as the canonical 400. + * paging forward makes a paging client loop over the first page forever. + * + * An offset is the weaker of the two schemes here: unlike a keyset it names an + * ordinal, not a position, so replaying it against a re-filtered or re-sorted + * sequence lands at an unrelated point in it — skipping rows, repeating them, or + * landing past the end and returning an empty page the caller reads as "no more + * matches". The v2 error policies render the thrown validation error as the + * canonical 400. */ -export function decodeOffsetCursor(cursor: string | undefined): number { +export function decodeOffsetCursor( + cursor: string | undefined, + sort: string, + filter: string +): number { if (!cursor) return 0 - const offset = decodeCursor<{ offset?: unknown }>(cursor)?.offset + const decoded = decodeCursor>(cursor) + if (!decoded) throw new OrchestrationError('validation', UNREADABLE_CURSOR_MESSAGE) + if (decoded.sort !== sort) { + throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) + } + if ((decoded.filter ?? undefined) !== filter) { + throw new OrchestrationError('validation', REFILTERED_CURSOR_MESSAGE) + } + const { offset } = decoded if (typeof offset !== 'number' || !Number.isInteger(offset) || offset < 0) { throw new OrchestrationError('validation', 'Invalid cursor') } @@ -204,46 +310,157 @@ export function cursorSortKey(sortBy: string, sortOrder: string): string { interface SortedCursorPayload { sort: string keys: CursorKey[] + /** Fingerprint of the list and filters the page was read under. */ + filter: string } /** - * A keyset cursor stamped with the sort that produced it. The keys are only - * meaningful under that exact ordering, so the stamp travels with them. + * A keyset cursor stamped with the sort AND the scope that produced it. The + * keys are only meaningful under that exact ordering, and only name a useful + * position within that exact row set, so both stamps travel with them. */ -export function encodeSortedCursor(sort: string, keys: CursorKey[]): string { - return encodeCursor({ sort, keys } satisfies SortedCursorPayload) +export function encodeSortedCursor(sort: string, keys: CursorKey[], filter: string): string { + return encodeCursor({ sort, keys, filter } satisfies SortedCursorPayload) } -export type DecodedSortedCursor = +type DecodedSortedCursor = | { status: 'absent' } | { status: 'ok'; keys: CursorKey[] } - /** Malformed, or minted under a different sort — the page cannot be resumed. */ + /** Not a pagination cursor at all — it does not decode into one. */ + | { status: 'unreadable' } + /** Minted under a different sort — the keys compare the wrong column. */ | { status: 'invalid' } + /** Minted under a different list or different filters — another sequence. */ + | { status: 'refiltered' } /** * Reads a keyset cursor back, refusing one that does not belong to the - * requested sort. Resuming a `name`-ordered cursor under `createdAt` would - * compare the wrong column and silently duplicate or skip rows, so a mismatch - * is a client error rather than a best-effort page. A cursor that isn't valid - * base64-JSON is rejected for the same reason: ignoring it would restart from - * page one while the caller believes it is paging forward. + * requested query. + * + * Resuming a `name`-ordered cursor under `createdAt` would compare the wrong + * column and silently duplicate or skip rows, so a sort mismatch is a client + * error rather than a best-effort page. A cursor that isn't valid base64-JSON + * is rejected for the same reason: ignoring it would restart from page one + * while the caller believes it is paging forward. + * + * A filter mismatch is rejected too, even though a keyset does not corrupt the + * way an offset does: `(sortKey, id)` names an absolute position, so replaying + * it under a narrower filter returns a coherent, duplicate-free page that is + * silently missing every new match sorting before that position. The token is + * opaque, so a caller cannot tell that truncated page from a complete one. * * This checks the envelope only. The key VALUES are caller-controlled too, and * are type-checked against the sort's keys by `keysetAfter`, which is where a * bad arity or an unparseable timestamp is caught. */ -export function decodeSortedCursor(cursor: string | undefined, sort: string): DecodedSortedCursor { +export function decodeSortedCursor( + cursor: string | undefined, + sort: string, + filter: string +): DecodedSortedCursor { if (!cursor) return { status: 'absent' } const decoded = decodeCursor>(cursor) - if (!decoded || decoded.sort !== sort || !Array.isArray(decoded.keys)) { - return { status: 'invalid' } + if (!decoded || typeof decoded.sort !== 'string' || !Array.isArray(decoded.keys)) { + return { status: 'unreadable' } } + if (decoded.sort !== sort) return { status: 'invalid' } + if ((decoded.filter ?? undefined) !== filter) return { status: 'refiltered' } return { status: 'ok', keys: decoded.keys } } -/** The 400 for a cursor that cannot be resumed under the request's sort. */ -export function v2CursorSortError(): NextResponse { - return v2Error('BAD_REQUEST', INVALID_CURSOR_MESSAGE) +/** + * The keyset a paged list should resume from, or `undefined` for page one. + * + * This is the `mapInput` half of every keyset list: it stamps the request's + * sort and filters, reads the cursor back under them, and turns a cursor minted + * under a different query into the canonical 400 rather than letting mismatched + * keys reach `keysetAfter` or a stale position reach a re-filtered read. Sharing + * it is what keeps "a bad cursor is a 400" from being re-decided per route. + * + * Build `filter` with `cursorScopeKey` from the same params on both + * sides of the request. A list with no filters at all passes nothing. + */ +export function readSortedCursor( + cursor: string | undefined, + sortBy: string, + sortOrder: string, + filter: string +): CursorKey[] | undefined { + const decoded = decodeSortedCursor(cursor, cursorSortKey(sortBy, sortOrder), filter) + if (decoded.status === 'unreadable') { + throw new OrchestrationError('validation', UNREADABLE_CURSOR_MESSAGE) + } + if (decoded.status === 'invalid') { + throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) + } + if (decoded.status === 'refiltered') { + throw new OrchestrationError('validation', REFILTERED_CURSOR_MESSAGE) + } + return decoded.status === 'ok' ? decoded.keys : undefined +} + +/** + * The next page's cursor, or `null` on the last page. + * + * The `present` half of the pair {@link readSortedCursor} opens: it stamps the + * response token with the same sort and filters the request was read under, so + * a list cannot mint a token its own reader would reject. Pass the identical + * `sortBy`/`sortOrder`/`filter` triple both sides. + */ +export function writeSortedCursor( + keys: CursorKey[] | null | undefined, + sortBy: string, + sortOrder: string, + filter: string +): string | null { + return keys ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), keys, filter) : null +} + +interface ScopedCursorPayload { + /** Fingerprint of the list, filters, and sort the inner token was minted under. */ + scope: string + /** The domain codec's own opaque token, passed through untouched. */ + inner: string +} + +/** + * Binds a cursor minted by a domain codec to the query it was minted under. + * + * `GET /logs`, `GET /audit-logs`, and `GET /billing/logs` page through readers + * that predate the shared v2 codecs and mint their own tokens, so the stamp + * cannot live inside the payload the way it does for {@link encodeSortedCursor}. + * Wrapping keeps the domain token opaque and untouched while still giving those + * lists the same binding as the rest of the surface — one rule for v2 callers + * rather than "some lists notice, some don't". + */ +export function encodeScopedCursor(scope: string, inner: string): string { + return encodeCursor({ scope, inner } satisfies ScopedCursorPayload) +} + +/** + * Unwraps a {@link encodeScopedCursor} token, yielding the domain codec's own + * cursor, or `undefined` for page one. A token that is malformed or was minted + * under a different query is the canonical 400 — the domain codec never sees it. + * + * An empty inner token is malformed, not "page one". Only an absent `cursor` + * param means page one; a present-but-empty inner passed the old + * `typeof === 'string'` envelope check and then read as falsy in every domain + * reader downstream, so no cursor condition was applied and the caller was + * handed page one again — with a `nextCursor` telling it to keep going. That is + * exactly the loop `UNKNOWN_CURSOR_MESSAGE` describes on the billing ledger, + * reached through the wrapper instead of through the token, and it slipped past + * the unresolvable-cursor 400 that exists to stop it. + */ +export function readScopedCursor(cursor: string | undefined, scope: string): string | undefined { + if (!cursor) return undefined + const decoded = decodeCursor>(cursor) + if (!decoded || typeof decoded.inner !== 'string' || decoded.inner.length === 0) { + throw new OrchestrationError('validation', UNREADABLE_CURSOR_MESSAGE) + } + if ((decoded.scope ?? undefined) !== scope) { + throw new OrchestrationError('validation', REFILTERED_CURSOR_MESSAGE) + } + return decoded.inner } const V2_CODE_BY_ORCHESTRATION_ERROR: Record = { @@ -278,9 +495,19 @@ export function v2ErrorForOrchestration( * Renders a thrown domain failure in the v2 envelope, or `null` when the error * carries no classification and the caller should log it and return its own * generic 500. The v2 counterpart of `orchestrationErrorResponse`. + * + * A refusal that names its cause carries it through as `error.details.code`. + * That projection lives here, on the one function every v2 error policy + * ultimately falls through to, rather than at each throw site — a route cannot + * then forget it, and the code cannot be attached to a status other than the + * one its failure class maps to. */ export function v2CaughtOrchestrationError(error: unknown): NextResponse | null { const classified = asOrchestrationError(error) if (!classified) return null - return v2ErrorForOrchestration(classified.code, classified.message) + return v2ErrorForOrchestration( + classified.code, + classified.message, + forbiddenErrorDetails(classified) + ) } diff --git a/apps/sim/app/api/v2/logs/route.test.ts b/apps/sim/app/api/v2/logs/route.test.ts index 74b5755c412..36917bedbab 100644 --- a/apps/sim/app/api/v2/logs/route.test.ts +++ b/apps/sim/app/api/v2/logs/route.test.ts @@ -24,7 +24,10 @@ vi.mock('@/lib/logs/application/list-public-logs', () => ({ listPublicLogs: { operation: { id: 'logs.list' }, execute: mocks.execute }, })) +import { v2ListLogsContract } from '@/lib/api/contracts/v2/logs' +import { cursorRoute, cursorScopeKey, UNREADABLE_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { OrchestrationError } from '@/lib/core/orchestration/types' +import { encodeScopedCursor } from '@/app/api/v2/lib/response' import { GET } from '@/app/api/v2/logs/route' const WORKSPACE_ID = '6fc7631d-88cd-46f8-9f0a-d4764daef7f8' @@ -116,6 +119,71 @@ describe('GET /api/v2/logs', () => { expect(body.data[0]).toMatchObject({ runId: 'run-1', status: 'paused' }) }) + /** + * The run-log cursor is minted by the domain codec, so it carries only its own + * `(startedAt, id)` position and the requested order. Binding it to the filters + * is what stops a cursor taken from an unfiltered walk from resuming inside a + * `level=error` read at an unrelated point in that shorter sequence. + */ + it('refuses a cursor replayed under a different filter', async () => { + mocks.execute.mockResolvedValueOnce({ + items: [{ log, executionData: null }], + nextCursor: Buffer.from( + JSON.stringify({ startedAt: log.startedAt.toISOString(), id: 'run-1', order: 'desc' }) + ).toString('base64'), + includeFullDetails: false, + includeFinalOutput: false, + includeTraceSpans: false, + }) + const firstPage = await ( + await GET(new NextRequest(`http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}`)) + ).json() + expect(firstPage.nextCursor).toEqual(expect.any(String)) + mocks.execute.mockClear() + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&level=error&cursor=${encodeURIComponent(firstPage.nextCursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('requested filters') }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + /** + * These three decide how much of each row is rendered, not which rows are in + * the sequence, so they must stay out of the binding. + */ + it.each([['details=full'], ['includeTraceSpans=true'], ['includeFinalOutput=true']])( + 'resumes a cursor across a changed %s', + async (param) => { + mocks.execute.mockResolvedValueOnce({ + items: [{ log, executionData: null }], + nextCursor: Buffer.from( + JSON.stringify({ startedAt: log.startedAt.toISOString(), id: 'run-1', order: 'desc' }) + ).toString('base64'), + includeFullDetails: false, + includeFinalOutput: false, + includeTraceSpans: false, + }) + const firstPage = await ( + await GET(new NextRequest(`http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}`)) + ).json() + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&${param}&cursor=${encodeURIComponent(firstPage.nextCursor)}` + ) + ) + + expect(response.status).toBe(200) + } + ) + it('rejects malformed cursors after admission and before protected reads', async () => { const response = await GET( new NextRequest( @@ -128,6 +196,221 @@ describe('GET /api/v2/logs', () => { expect(mocks.execute).not.toHaveBeenCalled() }) + /** + * An empty inner token reads as falsy in the domain codec, so no cursor + * condition is applied and the caller silently gets page one back, with a + * `nextCursor` inviting it to do the same thing forever. + */ + it('rejects a cursor whose inner token is empty instead of restarting at page one', async () => { + const cursor = encodeScopedCursor( + cursorScopeKey(cursorRoute(v2ListLogsContract), { workspaceId: WORKSPACE_ID, order: 'desc' }), + '' + ) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&limit=1&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + /** + * An undecodable token says nothing about which param changed, and this + * operation declares neither `sortBy` nor `sortOrder` under a `.strict()` + * query schema — so the sort-mismatch message would answer one 400 with + * advice that earns a second. The message is asserted exactly rather than by + * absence: "does not say sortBy" is satisfied by almost any wording, including + * one that tells the caller nothing at all. + */ + it('names the params a rejected cursor is actually bound to', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&cursor=not-a-cursor` + ) + ) + + const body = await response.json() + expect(body.error.message).toBe(UNREADABLE_CURSOR_MESSAGE) + expect(body.error.message).toContain('Restart pagination without a cursor') + expect(body.error.message).not.toContain('sortBy') + expect(body.error.message).not.toContain('sortOrder') + }) + + /** + * `total_duration_ms` is an `integer` column, so a value that is not + * representable as int4 is rejected by Postgres itself — the request has to + * fail at the contract instead of reaching the query. + */ + it.each([ + ['minDurationMs', '1.5'], + ['maxDurationMs', '1.5'], + ['maxDurationMs', '-0.5'], + ['minDurationMs', '1e30'], + ['minDurationMs', '2147483648'], + ['minDurationMs', '999999999999999999999'], + ['maxDurationMs', '-1'], + ])('rejects %s=%s before it can reach the query', async (field, value) => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&${field}=${encodeURIComponent(value)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining(field) }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it.each([ + ['minDurationMs', '0'], + ['maxDurationMs', '1000000'], + ['minDurationMs', '2147483647'], + ])('accepts %s=%s', async (field, value) => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&${field}=${value}` + ) + ) + + expect(response.status).toBe(200) + }) + + /** + * `0000` satisfies the published `\d{4}` date-time pattern but names no + * instant Postgres can store — the proleptic Gregorian calendar has no year + * zero — so the value has to be refused before it becomes a bind parameter. + */ + it.each([['startDate'], ['endDate']])( + 'rejects a year-0000 %s before it can reach the query', + async (field) => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&${field}=${encodeURIComponent('0000-01-01T00:00:00Z')}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining(field) }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + } + ) + + it('accepts the earliest storable year', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&startDate=${encodeURIComponent('0001-01-01T00:00:00Z')}` + ) + ) + + expect(response.status).toBe(200) + }) + + /** + * `folderPaths=/,` was already a 400 while the sibling comma lists dropped + * the empty entry, so one endpoint answered two ways to the same mistake. + */ + it.each([ + ['workflowIds', 'workflow-1,,workflow-2'], + ['workflowIds', 'workflow-1,'], + ['triggers', 'manual,'], + ['folderPaths', '/,'], + ])('rejects an empty entry in %s=%s', async (field, value) => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&${field}=${encodeURIComponent(value)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining(field) }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + /** A repeated param arrives as an array, which every v2 schema reads as a missing value. */ + it('names duplication when a query param is sent twice', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&workspaceId=${WORKSPACE_ID}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('workspaceId was sent') }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it.each([ + ['abc', 'startDate'], + ['2026-08-06', 'startDate'], + ['2026-08-06T00:00:00+02:00', 'startDate'], + ])('rejects %s as a window bound before it can reach the query', async (value, field) => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&${field}=${encodeURIComponent(value)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('startDate') }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('rejects an unparseable endDate', async () => { + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&endDate=abc`) + ) + + expect(response.status).toBe(400) + expect(mocks.execute).not.toHaveBeenCalled() + }) + + it('forwards a UTC window bound as a Date', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&startDate=2026-08-06T00:00:00Z` + ) + ) + + expect(response.status).toBe(200) + expect(mocks.execute).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + filters: expect.objectContaining({ startDate: new Date('2026-08-06T00:00:00Z') }), + }), + }) + ) + }) + + it('rejects an inverted window instead of answering with an empty page', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/logs?workspaceId=${WORKSPACE_ID}&startDate=2026-08-06T00:00:00Z&endDate=2026-08-05T00:00:00Z` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { + code: 'BAD_REQUEST', + message: expect.stringContaining('startDate must be before or equal to endDate'), + }, + }) + expect(mocks.execute).not.toHaveBeenCalled() + }) + it('projects typed folder errors', async () => { mocks.execute.mockRejectedValueOnce(new OrchestrationError('not_found', 'Folder not found')) diff --git a/apps/sim/app/api/v2/logs/route.ts b/apps/sim/app/api/v2/logs/route.ts index ea37bd87e30..74123a88266 100644 --- a/apps/sim/app/api/v2/logs/route.ts +++ b/apps/sim/app/api/v2/logs/route.ts @@ -4,16 +4,66 @@ import { v2ListLogsContract, v2LogStatusSchema, } from '@/lib/api/contracts/v2/logs' +import { + cursorRoute, + cursorScopeKey, + instantScopePart, + parseUnorderedList, + UNREADABLE_CURSOR_MESSAGE, + unorderedScopePart, +} from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { OrchestrationError } from '@/lib/core/orchestration/types' import { v2LogErrorPolicies } from '@/lib/logs/api/route-policies' import { listPublicLogs } from '@/lib/logs/application/list-public-logs' import { logOperations } from '@/lib/logs/application/operations' import { decodePublicLogCursor } from '@/lib/logs/public-queries' +import { encodeScopedCursor, readScopedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** + * Every param that changes which logs, in which order, this list returns. + * + * `details`, `includeFinalOutput`, and `includeTraceSpans` are deliberately + * absent: they decide how much of each row is rendered, not which rows are in + * the sequence, so a caller may turn them on mid-walk. + */ +function logCursorFilters(query: { + workspaceId: string + workflowIds?: string + triggers?: string + level?: string + startDate?: string + endDate?: string + runId?: string + minDurationMs?: number + maxDurationMs?: number + minCost?: number + maxCost?: number + model?: string + folderPaths?: string + order?: string +}) { + return cursorScopeKey(cursorRoute(v2ListLogsContract), { + workspaceId: query.workspaceId, + workflowIds: unorderedScopePart(query.workflowIds), + triggers: unorderedScopePart(query.triggers), + level: query.level, + startDate: instantScopePart(query.startDate), + endDate: instantScopePart(query.endDate), + runId: query.runId, + minDurationMs: query.minDurationMs, + maxDurationMs: query.maxDurationMs, + minCost: query.minCost, + maxCost: query.maxCost, + model: query.model, + folderPaths: unorderedScopePart(query.folderPaths), + order: query.order, + }) +} + export const GET = defineV2JsonRoute({ contract: v2ListLogsContract, auth: v2ApiKeyAuth, @@ -21,17 +71,16 @@ export const GET = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2LogErrorPolicies.default, mapInput: ({ query }) => { - const decodedCursor = query.cursor - ? decodePublicLogCursor(query.cursor, query.order ?? 'desc') - : null - if (query.cursor && !decodedCursor) { - throw new OrchestrationError('validation', 'Invalid cursor') + const inner = readScopedCursor(query.cursor, logCursorFilters(query)) + const decodedCursor = inner ? decodePublicLogCursor(inner, query.order ?? 'desc') : null + if (inner && !decodedCursor) { + throw new OrchestrationError('validation', UNREADABLE_CURSOR_MESSAGE) } return { workspaceId: query.workspaceId, filters: { - workflowIds: query.workflowIds?.split(',').filter(Boolean), - triggers: query.triggers?.split(',').filter(Boolean), + workflowIds: parseUnorderedList(query.workflowIds), + triggers: parseUnorderedList(query.triggers), level: query.level, startDate: query.startDate ? new Date(query.startDate) : undefined, endDate: query.endDate ? new Date(query.endDate) : undefined, @@ -44,7 +93,7 @@ export const GET = defineV2JsonRoute({ cursor: decodedCursor ?? undefined, order: query.order, }, - folderPaths: query.folderPaths?.split(',').filter(Boolean), + folderPaths: parseUnorderedList(query.folderPaths), limit: query.limit, includeFullDetails: query.details === 'full' || query.includeFinalOutput || query.includeTraceSpans, @@ -53,7 +102,10 @@ export const GET = defineV2JsonRoute({ } }, useCase: listPublicLogs, - present: ({ items, nextCursor, includeFullDetails, includeFinalOutput, includeTraceSpans }) => ({ + present: ( + { items, nextCursor, includeFullDetails, includeFinalOutput, includeTraceSpans }, + { query } + ) => ({ data: items.map(({ log, executionData }): V2LogListItem => { const item: V2LogListItem = { runId: log.executionId, @@ -86,6 +138,6 @@ export const GET = defineV2JsonRoute({ } return item }), - nextCursor, + nextCursor: nextCursor ? encodeScopedCursor(logCursorFilters(query), nextCursor) : null, }), }) diff --git a/apps/sim/app/api/v2/mcp-servers/[id]/route.test.ts b/apps/sim/app/api/v2/mcp-servers/[id]/route.test.ts index 01b9dffcb7e..536f5b0adb4 100644 --- a/apps/sim/app/api/v2/mcp-servers/[id]/route.test.ts +++ b/apps/sim/app/api/v2/mcp-servers/[id]/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ import type { mcpServers } from '@sim/db/schema' +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' import { @@ -9,38 +18,16 @@ import { NoWorkspaceAccessError, } from '@/lib/core/application' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error {} - return { - mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), - get: vi.fn(), - update: vi.fn(), - remove: vi.fn(), - capture: vi.fn(), - }, - MockV2ApiKeyUnauthenticatedError, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: vi.fn().mockReturnValue({ - maxTokens: 100, - refillRate: 100, - refillIntervalMs: 60_000, - }), +const mocks = vi.hoisted(() => ({ + get: vi.fn(), + update: vi.fn(), + remove: vi.fn(), + capture: vi.fn(), })) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -49,7 +36,6 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mocks.capture })) vi.mock('@/lib/mcp/application/use-cases', () => ({ getMcpServerUseCase: { operation: { id: 'mcp_servers.read' }, execute: mocks.get }, @@ -65,17 +51,10 @@ const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_I const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'] as const, + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE_LIMIT_OK = { - allowed: true, - limit: 100, - remaining: 99, - resetAt: new Date('2026-01-01T00:00:00Z'), - retryAfterMs: 0, -} const server = { id: 'mcp-server-1', workspaceId: WORKSPACE_ID, @@ -105,27 +84,31 @@ const server = { } as McpServerRow const context = { params: Promise.resolve({ id: server.id }) } +/** + * The read and delete verbs scope themselves with `?workspaceId=`; the write + * verb carries `workspaceId` in its body. Sending the query copy on a write is + * now a 400 rather than a silently dropped key, so the helper only appends it + * where the contract declares it. + */ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { - return new NextRequest( - `http://localhost:3000/api/v2/mcp-servers/${server.id}?workspaceId=${WORKSPACE_ID}`, - { - method, - headers: { - 'x-api-key': 'key', - ...(body === undefined ? {} : { 'Content-Type': 'application/json' }), - }, - ...(body === undefined ? {} : { body: JSON.stringify(body) }), - } - ) + const query = method === 'PATCH' ? '' : `?workspaceId=${WORKSPACE_ID}` + return new NextRequest(`http://localhost:3000/api/v2/mcp-servers/${server.id}${query}`, { + method, + headers: { + 'x-api-key': 'key', + ...(body === undefined ? {} : { 'Content-Type': 'application/json' }), + }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }), + }) } describe('/api/v2/mcp-servers/[id]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.get.mockResolvedValue({ server }) mocks.update.mockResolvedValue({ server }) mocks.remove.mockResolvedValue({ server }) @@ -142,6 +125,25 @@ describe('/api/v2/mcp-servers/[id]', () => { }) }) + /** + * Every list in this family rejects a query param it does not implement, so + * the single-resource reads must too. A caller who mistypes a flag otherwise + * gets a 200 that silently ignored it, which reads as confirmation the flag + * exists and does nothing. + */ + it('rejects a query param it does not implement', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/mcp-servers/${server.id}?workspaceId=${WORKSPACE_ID}&includeTools=true`, + { method: 'GET', headers: { 'x-api-key': 'key' } } + ), + context + ) + + expect(response.status).toBe(400) + expect(mocks.get).not.toHaveBeenCalled() + }) + it('updates an MCP server through the strict semantic update operation', async () => { const response = await PATCH( request('PATCH', { workspaceId: WORKSPACE_ID, name: 'New docs' }), @@ -175,11 +177,12 @@ describe('/api/v2/mcp-servers/[id]', () => { }) it('authenticates before parsing an invalid update body', async () => { - mocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) const response = await PATCH(request('PATCH', {}), context) expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.update).not.toHaveBeenCalled() }) diff --git a/apps/sim/app/api/v2/mcp-servers/[id]/route.ts b/apps/sim/app/api/v2/mcp-servers/[id]/route.ts index 20027c2f474..1612b2269b0 100644 --- a/apps/sim/app/api/v2/mcp-servers/[id]/route.ts +++ b/apps/sim/app/api/v2/mcp-servers/[id]/route.ts @@ -3,12 +3,7 @@ import { v2GetMcpServerContract, v2UpdateMcpServerContract, } from '@/lib/api/contracts/v2/mcp-servers' -import { - createV2ResourceConcealmentPolicy, - defineV2JsonRoute, - v2ApiKeyAuth, - v2RateLimits, -} from '@/lib/api/server/routes' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { mcpServerOperations } from '@/lib/mcp/application/operations' import { deleteMcpServerUseCase, @@ -16,15 +11,11 @@ import { updateMcpServerUseCase, } from '@/lib/mcp/application/use-cases' import { captureServerEvent } from '@/lib/posthog/server' -import { toV2McpServer } from '@/app/api/v2/mcp-servers/utils' +import { mcpServerResourceErrorPolicy, toV2McpServer } from '@/app/api/v2/mcp-servers/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 -const mcpServerResourceErrorPolicy = createV2ResourceConcealmentPolicy({ - notFoundMessage: 'MCP server not found', -}) - /** GET /api/v2/mcp-servers/[id] — Fetch a single MCP server. */ export const GET = defineV2JsonRoute({ contract: v2GetMcpServerContract, diff --git a/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.test.ts b/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.test.ts new file mode 100644 index 00000000000..e577d2428f5 --- /dev/null +++ b/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.test.ts @@ -0,0 +1,306 @@ +/** + * @vitest-environment node + */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + discover: vi.fn(), + authorizeDiscover: vi.fn(), +})) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) +vi.mock('@/lib/api/server/rate-limit-context', () => ({ + recordRateLimitSnapshot: vi.fn(), + getRateLimitHeaders: vi.fn().mockReturnValue(null), +})) +vi.mock('@/lib/core/utils/request', () => ({ + generateRequestId: vi.fn().mockReturnValue('request-1'), + getClientIp: vi.fn().mockReturnValue('127.0.0.1'), +})) +vi.mock('@/lib/mcp/application/use-cases', () => ({ + discoverMcpServerToolsUseCase: { + operation: { id: 'mcp_servers.tools.discover' }, + execute: mocks.discover, + authorize: mocks.authorizeDiscover, + }, +})) + +import { NoWorkspaceAccessError, WorkspaceApiKeyAuthorizationError } from '@/lib/core/application' +import { OrchestrationError } from '@/lib/core/orchestration/types' +import { + McpConnectionError, + McpOauthAuthorizationRequiredError, + McpServerCooldownError, +} from '@/lib/mcp/types' +import { GET } from '@/app/api/v2/mcp-servers/[id]/tools/route' + +const WORKSPACE_ID = 'workspace-1' +const SERVER_ID = 'mcp-3f7a9c21' +const PRINCIPAL = { kind: 'personal_api_key' as const, userId: 'user-1', keyId: 'key-1' } +const AUTH = { + principal: PRINCIPAL, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:key-1', 'user:user-1'] as const, + rateLimitSubscription: null, + keyType: 'personal' as const, +} +const TOOL = { + name: 'search_docs', + description: 'Search the internal documentation', + inputSchema: { + type: 'object' as const, + properties: { query: { type: 'string' } }, + required: ['query'], + }, + serverId: SERVER_ID, + serverName: 'Docs server', +} + +function request(query: string, method = 'GET') { + return new NextRequest(`http://localhost:3000/api/v2/mcp-servers/${SERVER_ID}/tools?${query}`, { + method, + headers: { 'x-api-key': 'key' }, + }) +} + +const context = { params: Promise.resolve({ id: SERVER_ID }) } + +describe('/api/v2/mcp-servers/[id]/tools', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.discover.mockResolvedValue({ tools: [TOOL] }) + mocks.authorizeDiscover.mockResolvedValue(undefined) + }) + + it('returns a server tool inventory as a single page', async () => { + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), context) + const body = await response.json() + + expect(response.status).toBe(200) + expect(body).toEqual({ data: [TOOL], nextCursor: null }) + expect(mocks.discover).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: { workspaceId: WORKSPACE_ID, serverId: SERVER_ID, refresh: false }, + request: expect.anything(), + }) + }) + + it('forwards an explicit refresh so a caller can bypass the tool cache', async () => { + await GET(request(`workspaceId=${WORKSPACE_ID}&refresh=true`), { ...context }) + + expect(mocks.discover).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ refresh: true }) }) + ) + }) + + /** + * Next aliases a missing `HEAD` export onto `GET`, and RFC 9110 §9.2.1 defines + * `HEAD` as safe. Discovery is not: it opens a live connection to a + * third-party endpoint and writes the outcome onto the server row. An uptime + * monitor or link checker walking the documented URL list would otherwise + * drive both on every probe, invisibly. + */ + it('answers HEAD without connecting to the server or writing its status', async () => { + const response = await GET(request(`workspaceId=${WORKSPACE_ID}&refresh=true`, 'HEAD'), { + ...context, + }) + + expect(response.status).toBe(200) + expect(await response.text()).toBe('') + expect(mocks.discover).not.toHaveBeenCalled() + expect(mocks.authorizeDiscover).toHaveBeenCalledOnce() + }) + + /** + * A `HEAD` answered before the use case's resource authorization is an + * existence oracle: any valid API key draws a bodiless 200 for a server id in + * a workspace it cannot read, for one that does not exist, and for a principal + * kind this operation refuses outright. These four pin the probe to the answer + * the `GET` gives. + */ + it('does not confirm a server to a principal kind the operation refuses', async () => { + mocks.authorizeDiscover.mockRejectedValueOnce(new WorkspaceApiKeyAuthorizationError()) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`, 'HEAD'), { ...context }) + + expect(response.status).toBe(403) + expect(mocks.discover).not.toHaveBeenCalled() + }) + + it('does not confirm a server id that does not exist', async () => { + mocks.authorizeDiscover.mockRejectedValueOnce( + new OrchestrationError('not_found', 'MCP server not found') + ) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`, 'HEAD'), { ...context }) + + expect(response.status).toBe(404) + expect(mocks.discover).not.toHaveBeenCalled() + }) + + it('does not confirm a server in a workspace the caller cannot read', async () => { + mocks.authorizeDiscover.mockRejectedValueOnce(new NoWorkspaceAccessError()) + + const response = await GET(request('workspaceId=someone-elses-workspace', 'HEAD'), { + ...context, + }) + + expect(response.status).toBe(404) + expect(mocks.discover).not.toHaveBeenCalled() + }) + + it('rejects a HEAD missing the required workspaceId instead of answering 200', async () => { + const response = await GET(request('', 'HEAD'), { ...context }) + + expect(response.status).toBe(400) + expect(mocks.authorizeDiscover).not.toHaveBeenCalled() + expect(mocks.discover).not.toHaveBeenCalled() + }) + + it('rejects a query param it does not implement', async () => { + const response = await GET(request(`workspaceId=${WORKSPACE_ID}&limit=10`), { ...context }) + + expect(response.status).toBe(400) + expect(mocks.discover).not.toHaveBeenCalled() + }) + + it('reports an unreachable server as a retryable 503, not a server fault', async () => { + mocks.discover.mockRejectedValueOnce(new McpConnectionError('ECONNREFUSED', 'Docs server')) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(503) + expect(body.error.code).toBe('SERVICE_UNAVAILABLE') + expect(response.headers.get('Retry-After')).not.toBeNull() + expect(JSON.stringify(body)).not.toContain('ECONNREFUSED') + }) + + it('reports a stale OAuth grant as a 409 a client can branch on, never as a Sim credential failure', async () => { + mocks.discover.mockRejectedValueOnce( + new McpOauthAuthorizationRequiredError(SERVER_ID, 'Docs server') + ) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(409) + expect(body.error.code).toBe('CONFLICT') + expect(body.error.details).toEqual({ code: 'MCP_SERVER_REAUTHORIZATION_REQUIRED' }) + }) + + it('does not blame the caller for an upstream protocol fault', async () => { + mocks.discover.mockRejectedValueOnce(new Error('MCP error -32602: Invalid params')) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(500) + expect(body.error.code).toBe('INTERNAL_ERROR') + expect(JSON.stringify(body)).not.toContain('Invalid params') + }) + + it('does not report a Sim-side response-schema defect as the caller`s bad request', async () => { + mocks.discover.mockResolvedValueOnce({ + tools: [{ ...TOOL, inputSchema: { ...TOOL.inputSchema, type: 'string' } }], + }) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(500) + expect(body.error.code).toBe('INTERNAL_ERROR') + }) + + /** + * `inputSchema` below the `object` wrapper is authored by the third-party + * server, and the MCP SDK's own `ToolSchema` does not declare `description` + * there — its `.catchall(z.unknown())` lets any value through, so a server + * serializing an absent description as JSON `null` (what a Python `None` + * produces) reaches Sim unvalidated. Declaring the key more tightly than the + * upstream schema does made the builder's outbound `.parse()` throw, and + * discovery answered a bare 500 for a payload the protocol permits. + */ + it('publishes a tool whose server reported a non-string inputSchema description', async () => { + mocks.discover.mockResolvedValueOnce({ + tools: [ + { + ...TOOL, + inputSchema: { type: 'object' as const, description: null, properties: {} }, + }, + ], + }) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(200) + expect(body.data[0].inputSchema).toEqual({ + type: 'object', + description: null, + properties: {}, + }) + }) + + /** + * `McpConnectionError` interpolates the server's display name into its + * message, so selecting the 503 wording by searching that message for + * `cooldown` hands a server named after the word the negative-cache wording + * for a cooldown it was never in. + */ + it('does not read cooldown wording out of a server display name', async () => { + mocks.discover.mockRejectedValueOnce(new McpConnectionError('ECONNREFUSED', 'Cooldown Docs')) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(503) + expect(body.error.message).toBe('The MCP server could not be reached') + }) + + it('reports a server inside the discovery cooldown with its own wording', async () => { + mocks.discover.mockRejectedValueOnce(new McpServerCooldownError(SERVER_ID)) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(503) + expect(body.error.message).toBe('The MCP server recently failed and is in cooldown') + }) + + it('rejects a workspace API key, which cannot supply the caller`s OAuth grant', async () => { + mocks.discover.mockRejectedValueOnce(new WorkspaceApiKeyAuthorizationError()) + + const response = await GET(request(`workspaceId=${WORKSPACE_ID}`), { ...context }) + const body = await response.json() + + expect(response.status).toBe(403) + expect(body.error.code).toBe('FORBIDDEN') + }) + + it('authenticates before parsing', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request(''), { ...context }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + expect(mocks.discover).not.toHaveBeenCalled() + }) +}) diff --git a/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.ts b/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.ts new file mode 100644 index 00000000000..11343abb137 --- /dev/null +++ b/apps/sim/app/api/v2/mcp-servers/[id]/tools/route.ts @@ -0,0 +1,34 @@ +import { v2ListMcpServerToolsContract } from '@/lib/api/contracts/v2/mcp-servers' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { mcpServerOperations } from '@/lib/mcp/application/operations' +import { discoverMcpServerToolsUseCase } from '@/lib/mcp/application/use-cases' +import { v2McpToolDiscoveryErrorPolicy } from '@/app/api/v2/mcp-servers/utils' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * GET /api/v2/mcp-servers/[id]/tools — List the tools a registered MCP server exposes. + * + * The path segment is static, so it can never shadow a server id: ids are minted + * as `mcp-` from the workspace and endpoint URL, and the registration + * contract requires a URL. + * + * `headSafe: false` because discovery opens a live connection to the registered + * endpoint and records the outcome on the server row. + */ +export const GET = defineV2JsonRoute({ + contract: v2ListMcpServerToolsContract, + operation: mcpServerOperations.discoverTools, + auth: v2ApiKeyAuth, + headSafe: false, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2McpToolDiscoveryErrorPolicy, + mapInput: ({ params, query }) => ({ + workspaceId: query.workspaceId, + serverId: params.id, + refresh: query.refresh, + }), + useCase: discoverMcpServerToolsUseCase, + present: ({ tools }) => ({ data: tools, nextCursor: null }), +}) diff --git a/apps/sim/app/api/v2/mcp-servers/route.test.ts b/apps/sim/app/api/v2/mcp-servers/route.test.ts index cf158e4cec2..92b2d8cc184 100644 --- a/apps/sim/app/api/v2/mcp-servers/route.test.ts +++ b/apps/sim/app/api/v2/mcp-servers/route.test.ts @@ -2,40 +2,27 @@ * @vitest-environment node */ import type { mcpServers } from '@sim/db/schema' +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error {} - return { - mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), - list: vi.fn(), - create: vi.fn(), - capture: vi.fn(), - }, - MockV2ApiKeyUnauthenticatedError, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: vi.fn().mockReturnValue({ - maxTokens: 100, - refillRate: 100, - refillIntervalMs: 60_000, - }), +const mocks = vi.hoisted(() => ({ + list: vi.fn(), + create: vi.fn(), + capture: vi.fn(), })) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/api/server/rate-limit-context', () => ({ recordRateLimitSnapshot: vi.fn(), getRateLimitHeaders: vi.fn().mockReturnValue(null), @@ -44,13 +31,13 @@ vi.mock('@/lib/core/utils/request', () => ({ generateRequestId: vi.fn().mockReturnValue('request-1'), getClientIp: vi.fn().mockReturnValue('127.0.0.1'), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mocks.capture })) vi.mock('@/lib/mcp/application/use-cases', () => ({ listMcpServersUseCase: { operation: { id: 'mcp_servers.list' }, execute: mocks.list }, createMcpServerUseCase: { operation: { id: 'mcp_servers.create' }, execute: mocks.create }, })) +import { REFILTERED_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { GET, POST } from '@/app/api/v2/mcp-servers/route' type McpServerRow = typeof mcpServers.$inferSelect @@ -59,17 +46,10 @@ const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_I const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: ['workspace:workspace-1'] as const, + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`] as const, rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE_LIMIT_OK = { - allowed: true, - limit: 100, - remaining: 99, - resetAt: new Date('2026-01-01T00:00:00Z'), - retryAfterMs: 0, -} const server = { id: 'mcp-server-1', workspaceId: WORKSPACE_ID, @@ -112,11 +92,16 @@ function request(method: 'GET' | 'POST', url: string, body?: unknown) { describe('/api/v2/mcp-servers', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) - mocks.gate.mockResolvedValue(null) - mocks.list.mockResolvedValue({ servers: [server] }) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.list.mockResolvedValue({ + servers: [server], + nextCursorKeys: null, + sortBy: 'createdAt', + sortOrder: 'desc', + }) mocks.create.mockResolvedValue({ server, updated: false }) }) @@ -134,11 +119,151 @@ describe('/api/v2/mcp-servers', () => { search: undefined, sortBy: 'createdAt', sortOrder: 'desc', + limit: 50, + cursor: undefined, + cursorKeys: undefined, }, request: expect.anything(), }) }) + it('bounds the server list by the requested limit', async () => { + await GET(request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=2`)) + + expect(mocks.list).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ limit: 2 }) }) + ) + }) + + it('mints a resumable cursor and replays it against the same sort', async () => { + mocks.list.mockResolvedValueOnce({ + servers: [server], + nextCursorKeys: [server.createdAt.toISOString(), server.id], + sortBy: 'createdAt', + sortOrder: 'desc', + }) + + const first = await GET( + request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1`) + ) + const { nextCursor } = await first.json() + + expect(nextCursor).toEqual(expect.any(String)) + + const second = await GET( + request( + 'GET', + `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(second.status).toBe(200) + expect(mocks.list).toHaveBeenLastCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + cursorKeys: [server.createdAt.toISOString(), server.id], + }), + }) + ) + }) + + it('rejects a cursor minted under a different sort', async () => { + mocks.list.mockResolvedValueOnce({ + servers: [server], + nextCursorKeys: [server.createdAt.toISOString(), server.id], + sortBy: 'createdAt', + sortOrder: 'desc', + }) + + const first = await GET( + request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1`) + ) + const { nextCursor } = await first.json() + + const response = await GET( + request( + 'GET', + `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1&sortBy=name&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(response.status).toBe(400) + }) + + /** + * The sort case above is a separate stamp. This pins the filter half of the + * binding end-to-end — the mint in `present` and the read in `mapInput` — + * because the contract-level sweep only checks a hand-maintained map of param + * names and stays green when a route drops the stamp entirely. + */ + it('refuses a cursor minted under a different filter', async () => { + mocks.list.mockResolvedValue({ + servers: [server], + nextCursorKeys: [server.createdAt.toISOString(), server.id], + sortBy: 'createdAt', + sortOrder: 'desc', + }) + + const minted = await GET( + request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&search=docs`) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.list.mockClear() + const replayed = await GET( + request( + 'GET', + `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&search=tickets&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.list).not.toHaveBeenCalled() + }) + + it('resumes a cursor replayed under the filters it was minted with', async () => { + mocks.list.mockResolvedValue({ + servers: [server], + nextCursorKeys: [server.createdAt.toISOString(), server.id], + sortBy: 'createdAt', + sortOrder: 'desc', + }) + + const minted = await GET( + request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&search=docs`) + ) + const { nextCursor } = await minted.json() + + mocks.list.mockClear() + const resumed = await GET( + request( + 'GET', + `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&search=docs&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(resumed.status).toBe(200) + expect(mocks.list).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: expect.objectContaining({ + search: 'docs', + cursorKeys: [server.createdAt.toISOString(), server.id], + }), + request: expect.anything(), + }) + }) + + it('rejects a fractional limit rather than paging on a fractional LIMIT', async () => { + const response = await GET( + request('GET', `/api/v2/mcp-servers?workspaceId=${WORKSPACE_ID}&limit=1.5`) + ) + + expect(response.status).toBe(400) + expect(mocks.list).not.toHaveBeenCalled() + }) + it('strictly creates an MCP server with the v2 source and status', async () => { const response = await POST( request('POST', '/api/v2/mcp-servers', { @@ -163,9 +288,10 @@ describe('/api/v2/mcp-servers', () => { }) it('keeps product analytics surface-specific for personal API keys', async () => { - mocks.authenticate.mockResolvedValueOnce({ + v2RouteMocks.authenticate.mockResolvedValueOnce({ ...AUTH, principal: { kind: 'personal_api_key', userId: 'user-1', keyId: 'key-personal' }, + rateLimitSubjectIds: ['api-key:key-personal', 'user:user-1'], keyType: 'personal', }) @@ -187,11 +313,12 @@ describe('/api/v2/mcp-servers', () => { }) it('authenticates before parsing create input', async () => { - mocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) const response = await POST(request('POST', '/api/v2/mcp-servers', {})) expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.create).not.toHaveBeenCalled() }) }) diff --git a/apps/sim/app/api/v2/mcp-servers/route.ts b/apps/sim/app/api/v2/mcp-servers/route.ts index 5d31b41f9b9..91ea160afc2 100644 --- a/apps/sim/app/api/v2/mcp-servers/route.ts +++ b/apps/sim/app/api/v2/mcp-servers/route.ts @@ -2,6 +2,7 @@ import { v2CreateMcpServerContract, v2ListMcpServersContract, } from '@/lib/api/contracts/v2/mcp-servers' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, @@ -11,11 +12,20 @@ import { import { mcpServerOperations } from '@/lib/mcp/application/operations' import { createMcpServerUseCase, listMcpServersUseCase } from '@/lib/mcp/application/use-cases' import { captureServerEvent } from '@/lib/posthog/server' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' import { toV2McpServer } from '@/app/api/v2/mcp-servers/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Every param that changes which MCP servers, in which order, this list returns. */ +function mcpServerCursorFilters(query: { workspaceId: string; search?: string }) { + return cursorScopeKey(cursorRoute(v2ListMcpServersContract), { + workspaceId: query.workspaceId, + search: query.search, + }) +} + /** GET /api/v2/mcp-servers — List MCP servers in a workspace. */ export const GET = defineV2JsonRoute({ contract: v2ListMcpServersContract, @@ -23,9 +33,29 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => ({ + workspaceId: query.workspaceId, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + limit: query.limit, + cursorKeys: readSortedCursor( + query.cursor, + query.sortBy, + query.sortOrder, + mcpServerCursorFilters(query) + ), + }), useCase: listMcpServersUseCase, - present: ({ servers }) => ({ data: servers.map(toV2McpServer), nextCursor: null }), + present: ({ servers, nextCursorKeys }, { query }) => ({ + data: servers.map(toV2McpServer), + nextCursor: writeSortedCursor( + nextCursorKeys, + query.sortBy, + query.sortOrder, + mcpServerCursorFilters(query) + ), + }), }) /** POST /api/v2/mcp-servers — Register a new MCP server. */ diff --git a/apps/sim/app/api/v2/mcp-servers/utils.ts b/apps/sim/app/api/v2/mcp-servers/utils.ts index 7d186ff8770..c8ef47af52b 100644 --- a/apps/sim/app/api/v2/mcp-servers/utils.ts +++ b/apps/sim/app/api/v2/mcp-servers/utils.ts @@ -1,7 +1,16 @@ -import type { NextResponse } from 'next/server' +import { UnauthorizedError } from '@modelcontextprotocol/sdk/client/auth.js' +import { StreamableHTTPError } from '@modelcontextprotocol/sdk/client/streamableHttp.js' +import { McpError as McpSdkError } from '@modelcontextprotocol/sdk/types.js' import { type V2McpServer, v2McpServerSchema } from '@/lib/api/contracts/v2/mcp-servers' +import { createV2ResourceConcealmentPolicy, type V2ErrorPolicy } from '@/lib/api/server/routes' +import { isTimeoutError } from '@/lib/core/execution-limits' import { projectMcpHeaders } from '@/lib/mcp/projection' import type { McpServerRow } from '@/lib/mcp/queries' +import { + McpConnectionError, + McpOauthAuthorizationRequiredError, + McpServerCooldownError, +} from '@/lib/mcp/types' import { v2Error } from '@/app/api/v2/lib/response' /** @@ -26,25 +35,89 @@ export function toV2McpServer(row: McpServerRow): V2McpServer { }) } +export const mcpServerResourceErrorPolicy = createV2ResourceConcealmentPolicy({ + notFoundMessage: 'MCP server not found', +}) + +/** + * `error.details.code` on the 409 a stale MCP OAuth grant produces. + * + * 409 carries more than one cause across the v2 surface, so the discriminator is + * what lets a client branch without matching on prose. It is published in the + * operation description. + */ +export const MCP_SERVER_REAUTHORIZATION_REQUIRED = 'MCP_SERVER_REAUTHORIZATION_REQUIRED' + /** - * Renders an MCP orchestration failure in the v2 error envelope. + * Caller-safe wording for a third-party server that did not answer usefully. + * + * Every branch returns a constant, so an upstream message — which may quote a + * hostname, a token endpoint, or a stack — never reaches the caller. * - * `forbidden` is the domain-allowlist / SSRF rejection and keeps its 403. - * `bad_gateway` is a DNS failure on the caller-supplied hostname — the caller's - * input is at fault, so it surfaces as a 400 rather than implying a Sim outage. + * Selection is typed, never matched on message text: `McpConnectionError` + * interpolates the server's display name into its message, so a server named + * after the word `cooldown` would select the cooldown branch it is not in. */ -export function v2McpOrchestrationError( - errorCode: string | undefined, - message: string -): NextResponse { - switch (errorCode) { - case 'not_found': - return v2Error('NOT_FOUND', 'MCP server not found') - case 'forbidden': - return v2Error('FORBIDDEN', message) - case 'bad_gateway': - return v2Error('BAD_REQUEST', message) - default: - return v2Error('INTERNAL_ERROR', 'Internal server error') +function unreachableServerMessage(error: unknown): string { + if (isTimeoutError(error)) return 'The MCP server took too long to respond' + if (error instanceof McpServerCooldownError) { + return 'The MCP server recently failed and is in cooldown' } + return 'The MCP server could not be reached' } + +/** + * Renders a tool-discovery failure. + * + * Discovery talks to a server the caller registered, so its failures are + * ordinary operating conditions rather than Sim faults: an unreachable, slow, or + * cooling-down server is a retryable 503 (`v2Error` stamps it with + * `Retry-After`), and a server whose stored OAuth grant no longer works is a 409 + * — the registration exists but its grant no longer does, which is a state + * conflict a human resolves by reauthorizing. Answering all of those with a bare + * 500 would make the endpoint that completes MCP onboarding indistinguishable + * from a Sim outage. + * + * The reauthorization case deliberately does **not** reuse 401. On this surface + * 401 means exactly one thing — the Sim API key is missing or invalid — and the + * published response description says so; a client that reacted to it by + * rotating or refreshing its Sim key would loop forever without touching the + * actual problem. It is also not a 403: the caller's rights on the Sim resource + * are fine. + * + * Classification is a typed dispatch over the MCP error families rather than + * `categorizeError`'s substring fallback. That fallback reaches 400 on any + * message containing `invalid`, which misattributed two different faults to the + * caller: an upstream JSON-RPC `Invalid params`, and — because the builder + * `.parse`s the response on the way out — a Sim-side response-schema defect, + * whose `ZodError` message carries `invalid_type`. The second is the worse of + * the two: answering it here suppressed the builder's 500 and its + * unhandled-error logging on the one v2 endpoint whose payload shape is authored + * by a third party. Anything unrecognised now returns `null` and keeps that + * generic 500. + */ +export const v2McpToolDiscoveryErrorPolicy = { + render(error) { + const orchestrated = mcpServerResourceErrorPolicy.render(error) + if (orchestrated) return orchestrated + + if (error instanceof McpOauthAuthorizationRequiredError || error instanceof UnauthorizedError) { + return v2Error( + 'CONFLICT', + 'The MCP server must be reauthorized in Sim before its tools can be listed', + { details: { code: MCP_SERVER_REAUTHORIZATION_REQUIRED } } + ) + } + + if ( + isTimeoutError(error) || + error instanceof McpConnectionError || + error instanceof McpSdkError || + error instanceof StreamableHTTPError + ) { + return v2Error('SERVICE_UNAVAILABLE', unreachableServerMessage(error)) + } + + return null + }, +} satisfies V2ErrorPolicy diff --git a/apps/sim/app/api/v2/secrets/[name]/route.test.ts b/apps/sim/app/api/v2/secrets/[name]/route.test.ts index d77c511e782..7f49d462fb8 100644 --- a/apps/sim/app/api/v2/secrets/[name]/route.test.ts +++ b/apps/sim/app/api/v2/secrets/[name]/route.test.ts @@ -86,19 +86,22 @@ const secret = { } const context = { params: Promise.resolve({ name: SECRET_NAME }) } +/** + * The read and delete verbs scope themselves with `?workspaceId=`; the write + * verb carries `workspaceId` in its body. Sending the query copy on a write is + * now a 400 rather than a silently dropped key, so the helper only appends it + * where the contract declares it. + */ function request(method: 'PUT' | 'DELETE', body?: unknown) { - const scope = method === 'DELETE' ? '&scope=workspace' : '' - return new NextRequest( - `http://localhost:3000/api/v2/secrets/${SECRET_NAME}?workspaceId=${WORKSPACE_ID}${scope}`, - { - method, - headers: { - 'x-api-key': 'key', - ...(body === undefined ? {} : { 'Content-Type': 'application/json' }), - }, - ...(body === undefined ? {} : { body: JSON.stringify(body) }), - } - ) + const query = method === 'DELETE' ? `?workspaceId=${WORKSPACE_ID}&scope=workspace` : '' + return new NextRequest(`http://localhost:3000/api/v2/secrets/${SECRET_NAME}${query}`, { + method, + headers: { + 'x-api-key': 'key', + ...(body === undefined ? {} : { 'Content-Type': 'application/json' }), + }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }), + }) } describe('/api/v2/secrets/[name]', () => { @@ -157,6 +160,25 @@ describe('/api/v2/secrets/[name]', () => { }) }) + /** + * The secrets list rejects a query param it does not implement, so the delete + * must too. A caller who mistypes `scope` otherwise gets a 400 for the missing + * required param — but a caller who adds a param that does not exist at all + * would have had it silently ignored. + */ + it('rejects a query param it does not implement', async () => { + const response = await DELETE( + new NextRequest( + `http://localhost:3000/api/v2/secrets/${SECRET_NAME}?workspaceId=${WORKSPACE_ID}&scope=workspace&scopes=personal`, + { method: 'DELETE', headers: { 'x-api-key': 'key' } } + ), + context + ) + + expect(response.status).toBe(400) + expect(mocks.remove).not.toHaveBeenCalled() + }) + it('renders typed application errors without leaking raw errors', async () => { mocks.remove.mockRejectedValueOnce(new OrchestrationError('not_found', 'stored detail')) diff --git a/apps/sim/app/api/v2/secrets/route.test.ts b/apps/sim/app/api/v2/secrets/route.test.ts index 5c173b4d883..ef740920143 100644 --- a/apps/sim/app/api/v2/secrets/route.test.ts +++ b/apps/sim/app/api/v2/secrets/route.test.ts @@ -46,6 +46,8 @@ vi.mock('@/lib/secrets/application/use-cases', () => ({ listSecretsUseCase: { operation: { id: 'secrets.list' }, execute: mocks.list }, })) +import { V2_DEFAULT_PAGE_SIZE } from '@/lib/api/contracts/v2/shared' +import { REFILTERED_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { GET } from '@/app/api/v2/secrets/route' const WORKSPACE_ID = 'workspace-1' @@ -88,7 +90,13 @@ describe('GET /api/v2/secrets', () => { mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) mocks.gate.mockResolvedValue(null) - mocks.list.mockResolvedValue({ secrets: [secret], userId: 'user-1' }) + mocks.list.mockResolvedValue({ + secrets: [secret], + userId: 'user-1', + nextCursorKeys: null, + sortBy: 'name', + sortOrder: 'asc', + }) }) it('lists secret metadata without exposing values', async () => { @@ -121,11 +129,86 @@ describe('GET /api/v2/secrets', () => { search: undefined, sortBy: 'name', sortOrder: 'asc', + limit: V2_DEFAULT_PAGE_SIZE, + cursor: undefined, + cursorKeys: undefined, }, request: expect.anything(), }) }) + /** + * Pins the binding end-to-end — the mint in `present` and the read in + * `mapInput` — because the contract-level sweep only checks a hand-maintained + * map of param names and stays green when a route drops the stamp entirely. + */ + it('refuses a cursor minted under a different filter', async () => { + mocks.list.mockResolvedValue({ + secrets: [secret], + userId: 'user-1', + nextCursorKeys: ['STRIPE_API_KEY', 'secret-1'], + sortBy: 'name', + sortOrder: 'asc', + }) + + const minted = await GET( + new NextRequest( + `http://localhost:3000/api/v2/secrets?workspaceId=${WORKSPACE_ID}&search=stripe`, + { headers: { 'x-api-key': 'key' } } + ) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.list.mockClear() + const replayed = await GET( + new NextRequest( + `http://localhost:3000/api/v2/secrets?workspaceId=${WORKSPACE_ID}&search=twilio&cursor=${encodeURIComponent(nextCursor)}`, + { headers: { 'x-api-key': 'key' } } + ) + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.list).not.toHaveBeenCalled() + }) + + it('resumes a cursor replayed under the filters it was minted with', async () => { + mocks.list.mockResolvedValue({ + secrets: [secret], + userId: 'user-1', + nextCursorKeys: ['STRIPE_API_KEY', 'secret-1'], + sortBy: 'name', + sortOrder: 'asc', + }) + + const minted = await GET( + new NextRequest( + `http://localhost:3000/api/v2/secrets?workspaceId=${WORKSPACE_ID}&search=stripe`, + { headers: { 'x-api-key': 'key' } } + ) + ) + const { nextCursor } = await minted.json() + + mocks.list.mockClear() + const resumed = await GET( + new NextRequest( + `http://localhost:3000/api/v2/secrets?workspaceId=${WORKSPACE_ID}&search=stripe&cursor=${encodeURIComponent(nextCursor)}`, + { headers: { 'x-api-key': 'key' } } + ) + ) + + expect(resumed.status).toBe(200) + expect(mocks.list).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: expect.objectContaining({ + search: 'stripe', + cursorKeys: ['STRIPE_API_KEY', 'secret-1'], + }), + request: expect.anything(), + }) + }) + it('authenticates before validating list input', async () => { mocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) diff --git a/apps/sim/app/api/v2/secrets/route.ts b/apps/sim/app/api/v2/secrets/route.ts index 62a1685dc88..bd9a44c1d39 100644 --- a/apps/sim/app/api/v2/secrets/route.ts +++ b/apps/sim/app/api/v2/secrets/route.ts @@ -1,4 +1,5 @@ import { v2ListSecretsContract } from '@/lib/api/contracts/v2/secrets' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, @@ -7,11 +8,21 @@ import { } from '@/lib/api/server/routes' import { secretOperations } from '@/lib/secrets/application/operations' import { listSecretsUseCase } from '@/lib/secrets/application/use-cases' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' import { toV2Secret } from '@/app/api/v2/secrets/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Every param that changes which secrets, in which order, this list returns. */ +function secretCursorFilters(query: { workspaceId: string; scope?: string; search?: string }) { + return cursorScopeKey(cursorRoute(v2ListSecretsContract), { + workspaceId: query.workspaceId, + scope: query.scope, + search: query.search, + }) +} + /** GET /api/v2/secrets — List secret names and metadata without reading their values. */ export const GET = defineV2JsonRoute({ contract: v2ListSecretsContract, @@ -19,10 +30,23 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => ({ + ...query, + cursorKeys: readSortedCursor( + query.cursor, + query.sortBy, + query.sortOrder, + secretCursorFilters(query) + ), + }), useCase: listSecretsUseCase, - present: ({ secrets, userId }) => ({ + present: ({ secrets, userId, nextCursorKeys }, { query }) => ({ data: secrets.map((secret) => toV2Secret(secret, userId)), - nextCursor: null, + nextCursor: writeSortedCursor( + nextCursorKeys, + query.sortBy, + query.sortOrder, + secretCursorFilters(query) + ), }), }) diff --git a/apps/sim/app/api/v2/secrets/utils.ts b/apps/sim/app/api/v2/secrets/utils.ts index e92a1d472ce..498040cd920 100644 --- a/apps/sim/app/api/v2/secrets/utils.ts +++ b/apps/sim/app/api/v2/secrets/utils.ts @@ -1,4 +1,4 @@ -import type { V2Secret, V2SecretScope } from '@/lib/api/contracts/v2/secrets' +import type { V2Secret } from '@/lib/api/contracts/v2/secrets' import type { VisibleWorkspaceCredential } from '@/lib/credentials/queries' /** Serialize environment credential metadata as a secret without exposing its stored value. */ @@ -18,9 +18,3 @@ export function toV2Secret(row: VisibleWorkspaceCredential, userId: string): V2S updatedAt: row.updatedAt.toISOString(), } } - -export function secretCredentialTypes(scope?: V2SecretScope) { - if (scope === 'workspace') return ['env_workspace'] as const - if (scope === 'personal') return ['env_personal'] as const - return ['env_workspace', 'env_personal'] as const -} diff --git a/apps/sim/app/api/v2/skills/[id]/route.test.ts b/apps/sim/app/api/v2/skills/[id]/route.test.ts index 814159913b0..734d4b7da28 100644 --- a/apps/sim/app/api/v2/skills/[id]/route.test.ts +++ b/apps/sim/app/api/v2/skills/[id]/route.test.ts @@ -86,18 +86,22 @@ const skill = { } const context = { params: Promise.resolve({ id: skill.id }) } +/** + * The read and delete verbs scope themselves with `?workspaceId=`; the write + * verb carries `workspaceId` in its body. Sending the query copy on a write is + * now a 400 rather than a silently dropped key, so the helper only appends it + * where the contract declares it. + */ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { - return new NextRequest( - `http://localhost:3000/api/v2/skills/${skill.id}?workspaceId=${WORKSPACE_ID}`, - { - method, - headers: { - 'x-api-key': 'key', - ...(body === undefined ? {} : { 'Content-Type': 'application/json' }), - }, - ...(body === undefined ? {} : { body: JSON.stringify(body) }), - } - ) + const query = method === 'PATCH' ? '' : `?workspaceId=${WORKSPACE_ID}` + return new NextRequest(`http://localhost:3000/api/v2/skills/${skill.id}${query}`, { + method, + headers: { + 'x-api-key': 'key', + ...(body === undefined ? {} : { 'Content-Type': 'application/json' }), + }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }), + }) } describe('/api/v2/skills/[id]', () => { @@ -124,6 +128,25 @@ describe('/api/v2/skills/[id]', () => { }) }) + /** + * Every list in this family rejects a query param it does not implement, so + * the single-resource reads must too. A caller who mistypes a flag otherwise + * gets a 200 that silently ignored it, which reads as confirmation the flag + * exists and does nothing. + */ + it('rejects a query param it does not implement', async () => { + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/skills/${skill.id}?workspaceId=${WORKSPACE_ID}&includeContents=true`, + { method: 'GET', headers: { 'x-api-key': 'key' } } + ), + context + ) + + expect(response.status).toBe(400) + expect(mocks.get).not.toHaveBeenCalled() + }) + it('updates a skill and emits only surface analytics', async () => { const response = await PATCH( request('PATCH', { workspaceId: WORKSPACE_ID, content: '# Updated' }), diff --git a/apps/sim/app/api/v2/skills/route.test.ts b/apps/sim/app/api/v2/skills/route.test.ts index 6d7770e05a0..40b8668b169 100644 --- a/apps/sim/app/api/v2/skills/route.test.ts +++ b/apps/sim/app/api/v2/skills/route.test.ts @@ -50,9 +50,36 @@ vi.mock('@/lib/skills/application/use-cases', () => ({ createSkillUseCase: { operation: { id: 'skills.create' }, execute: mocks.create }, })) +import { v2ListSkillsContract } from '@/lib/api/contracts/v2/skills' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' +import { PrincipalKindAuthorizationError } from '@/lib/core/application' +import { cursorSortKey, encodeOffsetCursor } from '@/app/api/v2/lib/response' import { GET, POST } from '@/app/api/v2/skills/route' const WORKSPACE_ID = 'workspace-1' + +/** + * A cursor exactly as this route mints one, built from the shared codec so the + * test exercises the real binding rather than a restatement of it. `search` is + * the only filter the skills list takes beyond its workspace. + */ +function skillCursor({ + offset, + search, + sortBy = 'createdAt', + sortOrder = 'desc', +}: { + offset: number + search?: string + sortBy?: string + sortOrder?: string +}): string { + return encodeOffsetCursor( + cursorSortKey(sortBy, sortOrder), + cursorScopeKey(cursorRoute(v2ListSkillsContract), { workspaceId: WORKSPACE_ID, search }), + offset + ) +} const PRINCIPAL = { kind: 'workspace_api_key' as const, workspaceId: WORKSPACE_ID, keyId: 'key-1' } const AUTH = { principal: PRINCIPAL, @@ -79,7 +106,7 @@ const skill = { updatedAt: new Date('2026-01-02T00:00:00Z'), } -function request(method: 'GET' | 'POST', url: string, body?: unknown) { +function request(method: 'GET' | 'POST' | 'HEAD', url: string, body?: unknown) { return new NextRequest(`http://localhost:3000${url}`, { method, headers: { @@ -97,7 +124,7 @@ describe('/api/v2/skills', () => { mocks.preauthRate.mockResolvedValue(RATE_LIMIT_OK) mocks.operationRate.mockResolvedValue(RATE_LIMIT_OK) mocks.gate.mockResolvedValue(null) - mocks.list.mockResolvedValue({ skills: [skill] }) + mocks.list.mockResolvedValue({ skills: [skill], hasMore: false, offset: 0, limit: 50 }) mocks.create.mockResolvedValue({ skill }) }) @@ -105,7 +132,9 @@ describe('/api/v2/skills', () => { const response = await GET(request('GET', `/api/v2/skills?workspaceId=${WORKSPACE_ID}`)) expect(response.status).toBe(200) - expect((await response.json()).data[0]).not.toHaveProperty('content') + const body = await response.json() + expect(body.data[0]).not.toHaveProperty('content') + expect(body.nextCursor).toBeNull() expect(mocks.list).toHaveBeenCalledWith({ principal: PRINCIPAL, input: { @@ -113,12 +142,129 @@ describe('/api/v2/skills', () => { search: undefined, sortBy: 'createdAt', sortOrder: 'desc', + limit: 50, + cursor: undefined, + offset: 0, }, request: expect.anything(), }) }) + it('resumes from the offset cursor and mints the next one while pages remain', async () => { + mocks.list.mockResolvedValueOnce({ + skills: [skill], + hasMore: true, + offset: 2, + limit: 2, + }) + const cursor = skillCursor({ offset: 2 }) + + const response = await GET( + request( + 'GET', + `/api/v2/skills?workspaceId=${WORKSPACE_ID}&limit=2&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(200) + expect((await response.json()).nextCursor).toBe(skillCursor({ offset: 4 })) + expect(mocks.list).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ limit: 2, offset: 2 }) }) + ) + }) + + /** + * An offset means nothing against a sequence it was not counted in, so a + * cursor minted under one sort must not silently resume under another. + */ + it('rejects a cursor replayed under a different sort', async () => { + const cursor = skillCursor({ offset: 2 }) + + const response = await GET( + request( + 'GET', + `/api/v2/skills?workspaceId=${WORKSPACE_ID}&sortBy=name&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(mocks.list).not.toHaveBeenCalled() + }) + + /** + * `search` and `sortOrder` change the sequence the offset counts positions in + * just as `sortBy` does, so both are stamped into the scope and both must + * invalidate a replayed cursor. + */ + it.each([ + ['search', 'search=other'], + ['sortOrder', 'sortOrder=asc'], + ])('rejects a cursor replayed under a different %s', async (_field, param) => { + const cursor = skillCursor({ offset: 2 }) + + const response = await GET( + request( + 'GET', + `/api/v2/skills?workspaceId=${WORKSPACE_ID}&${param}&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(mocks.list).not.toHaveBeenCalled() + }) + + /** + * `limit` is deliberately absent from the scope: it selects how much of the + * sequence to return, not what the sequence is. Stamping it would strand every + * cursor the moment a caller changed page size, for no correctness gain. + */ + it('resumes a cursor minted under a different page size', async () => { + mocks.list.mockResolvedValueOnce({ skills: [skill], hasMore: false, offset: 2, limit: 5 }) + const cursor = skillCursor({ offset: 2 }) + + const response = await GET( + request( + 'GET', + `/api/v2/skills?workspaceId=${WORKSPACE_ID}&limit=5&cursor=${encodeURIComponent(cursor)}` + ) + ) + + expect(response.status).toBe(200) + expect(mocks.list).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ limit: 5, offset: 2 }) }) + ) + }) + + /** + * The guard itself is unit-tested in `definition.test.ts`; this proves the + * pairing end-to-end, on a real v2 read that used to reply 500 to a plain HEAD. + */ + it('serves HEAD through the GET handler instead of throwing', async () => { + const response = await GET(request('HEAD', `/api/v2/skills?workspaceId=${WORKSPACE_ID}`)) + + expect(response.status).toBe(200) + expect(mocks.list).toHaveBeenCalled() + }) + + it('rejects a malformed cursor rather than silently restarting at page one', async () => { + const response = await GET( + request('GET', `/api/v2/skills?workspaceId=${WORKSPACE_ID}&cursor=not-a-cursor`) + ) + + expect(response.status).toBe(400) + expect(mocks.list).not.toHaveBeenCalled() + }) + + /** + * A personal key, not the suite's default workspace key. `skills.create` denies + * a workspace key like every other skill write: the per-skill editor row that + * authorizes an update or a delete resolves against a human subject a workspace + * key cannot supply, so allowing it to create left rows it could never remove. + */ it('creates a skill with the v2 source and status', async () => { + const principal = { kind: 'personal_api_key' as const, userId: 'user-1', keyId: 'key-personal' } + mocks.authenticate.mockResolvedValueOnce({ ...AUTH, principal, keyType: 'personal' as const }) + const response = await POST( request('POST', '/api/v2/skills', { workspaceId: WORKSPACE_ID, @@ -131,7 +277,7 @@ describe('/api/v2/skills', () => { expect(response.status).toBe(201) expect((await response.json()).data.id).toBe(skill.id) expect(mocks.create).toHaveBeenCalledWith({ - principal: PRINCIPAL, + principal, input: { workspaceId: WORKSPACE_ID, name: skill.name, @@ -141,7 +287,6 @@ describe('/api/v2/skills', () => { }, request: expect.anything(), }) - expect(mocks.capture).not.toHaveBeenCalled() }) it('keeps skill analytics on the personal-key v2 surface', async () => { @@ -169,6 +314,30 @@ describe('/api/v2/skills', () => { ) }) + /** + * `skills.create` denies a workspace key outright, so what this pins is the + * surface's half: the refusal reaches the caller as the operation's own 403, + * and a create that never happened emits no analytics. + */ + it('refuses a workspace-key create and records no analytics for it', async () => { + mocks.create.mockRejectedValueOnce( + new PrincipalKindAuthorizationError('workspace_api_key', 'skills.create') + ) + + const response = await POST( + request('POST', '/api/v2/skills', { + workspaceId: WORKSPACE_ID, + name: skill.name, + description: skill.description, + content: skill.content, + }) + ) + + expect(response.status).toBe(403) + expect(mocks.create).toHaveBeenCalledWith(expect.objectContaining({ principal: PRINCIPAL })) + expect(mocks.capture).not.toHaveBeenCalled() + }) + it('authenticates before parsing skill input', async () => { mocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) diff --git a/apps/sim/app/api/v2/skills/route.ts b/apps/sim/app/api/v2/skills/route.ts index b49c685fe90..42de2765968 100644 --- a/apps/sim/app/api/v2/skills/route.ts +++ b/apps/sim/app/api/v2/skills/route.ts @@ -1,4 +1,5 @@ import { v2CreateSkillContract, v2ListSkillsContract } from '@/lib/api/contracts/v2/skills' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, @@ -8,8 +9,17 @@ import { import { captureServerEvent } from '@/lib/posthog/server' import { skillOperations } from '@/lib/skills/application/operations' import { createSkillUseCase, listSkillsUseCase } from '@/lib/skills/application/use-cases' +import { cursorSortKey, decodeOffsetCursor, encodeOffsetCursor } from '@/app/api/v2/lib/response' import { toV2Skill, toV2SkillSummary } from '@/app/api/v2/skills/utils' +/** Every param that changes which skills, in which order, this list returns. */ +function skillCursorFilters(query: { workspaceId: string; search?: string }) { + return cursorScopeKey(cursorRoute(v2ListSkillsContract), { + workspaceId: query.workspaceId, + search: query.search, + }) +} + export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -20,9 +30,33 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => query, + mapInput: ({ query }) => { + /** + * The offset counts positions in the merged, filtered, sorted sequence, so + * every param that changes that sequence is stamped into the cursor and + * re-checked here. `limit` is deliberately absent — it selects how much of + * the sequence to return, not what the sequence is. + */ + return { + ...query, + offset: decodeOffsetCursor( + query.cursor, + cursorSortKey(query.sortBy, query.sortOrder), + skillCursorFilters(query) + ), + } + }, useCase: listSkillsUseCase, - present: ({ skills }) => ({ data: skills.map(toV2SkillSummary), nextCursor: null }), + present: ({ skills, hasMore, offset, limit }, { query }) => ({ + data: skills.map(toV2SkillSummary), + nextCursor: hasMore + ? encodeOffsetCursor( + cursorSortKey(query.sortBy, query.sortOrder), + skillCursorFilters(query), + offset + limit + ) + : null, + }), }) /** POST /api/v2/skills — Create a skill. */ diff --git a/apps/sim/app/api/v2/skills/utils.ts b/apps/sim/app/api/v2/skills/utils.ts index a1cc30ceb02..eb07440b91e 100644 --- a/apps/sim/app/api/v2/skills/utils.ts +++ b/apps/sim/app/api/v2/skills/utils.ts @@ -1,18 +1,16 @@ import type { skill } from '@sim/db/schema' -import type { NextResponse } from 'next/server' import type { V2Skill, V2SkillSummary } from '@/lib/api/contracts/v2/skills' -import type { SkillOrchestrationErrorCode } from '@/lib/skills/orchestration' import { isBuiltinSkillId } from '@/lib/workflows/skills/builtin-skills' -import { v2Error } from '@/app/api/v2/lib/response' - -/** - * Shared serialization + error mapping for the v2 skills surface. - */ +import type { SkillSummaryRow } from '@/lib/workflows/skills/operations' +/** Shared serialization for the v2 skills surface. */ type SkillRow = typeof skill.$inferSelect -/** List projection — no `content`; skill bodies are fetched per skill. */ -export function toV2SkillSummary(row: SkillRow): V2SkillSummary { +/** + * List projection — no `content`; skill bodies are fetched per skill. It takes + * the body-less row so the list query never has to load one. + */ +export function toV2SkillSummary(row: SkillSummaryRow): V2SkillSummary { return { id: row.id, name: row.name, @@ -27,22 +25,3 @@ export function toV2SkillSummary(row: SkillRow): V2SkillSummary { export function toV2Skill(row: SkillRow): V2Skill { return { ...toV2SkillSummary(row), content: row.content } } - -/** Renders a skill orchestration failure in the v2 error envelope. */ -export function v2SkillOrchestrationError( - errorCode: SkillOrchestrationErrorCode | undefined, - message: string -): NextResponse { - switch (errorCode) { - case 'validation': - return v2Error('BAD_REQUEST', message) - case 'forbidden': - return v2Error('FORBIDDEN', message) - case 'not_found': - return v2Error('NOT_FOUND', 'Skill not found') - case 'conflict': - return v2Error('CONFLICT', message) - default: - return v2Error('INTERNAL_ERROR', 'Internal server error') - } -} diff --git a/apps/sim/app/api/v2/tables/[tableId]/cancel-runs/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/cancel-runs/route.test.ts index 17d013606d6..4ed78156035 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/cancel-runs/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/cancel-runs/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,28 +18,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), cancelRuns: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, })) @@ -49,16 +45,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} function call(body: unknown) { const request = new NextRequest('http://localhost/api/v2/tables/table-1/cancel-runs', { @@ -75,10 +65,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/cancel-runs', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.cancelRuns.mockResolvedValue({ table: { id: 'table-1' }, cancelled: 4 }) }) @@ -146,4 +136,13 @@ describe('POST /api/v2/tables/[tableId]/cancel-runs', () => { expect(contradictory.status).toBe(400) expect(mocks.cancelRuns).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID, scope: 'all' }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/columns/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/columns/route.test.ts index 4ca1f73bf67..58675d8e48f 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/columns/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/columns/route.test.ts @@ -2,31 +2,27 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), add: vi.fn(), update: vi.fn(), remove: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/columns', () => ({ addTableColumnUseCase: { operation: { id: 'tables.columns.add' }, execute: mocks.add }, updateTableColumnUseCase: { operation: { id: 'tables.columns.update' }, execute: mocks.update }, @@ -45,16 +41,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const table = { id: 'table-1', name: 'Contacts', @@ -77,10 +67,10 @@ function request(method: 'POST' | 'PATCH' | 'DELETE', body: unknown) { describe('/api/v2/tables/[tableId]/columns', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.add.mockResolvedValue({ table }) mocks.update.mockResolvedValue({ table, changed: false }) mocks.remove.mockResolvedValue({ table }) @@ -93,7 +83,7 @@ describe('/api/v2/tables/[tableId]/columns', () => { }) const response = await POST(req, context) - expect(response.status).toBe(200) + expect(response.status).toBe(201) expect((await response.json()).data.columns).toEqual([ { id: 'col-1', name: 'Name', type: 'string', required: false, unique: false }, ]) @@ -108,6 +98,49 @@ describe('/api/v2/tables/[tableId]/columns', () => { }) }) + it('forwards required on both the add and the update column write', async () => { + await POST( + request('POST', { + workspaceId: WORKSPACE_ID, + column: { name: 'Name', type: 'string', required: true }, + }), + context + ) + await PATCH( + request('PATCH', { + workspaceId: WORKSPACE_ID, + columnName: 'Name', + updates: { required: false }, + }), + context + ) + + expect(mocks.add).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + column: { name: 'Name', type: 'string', required: true }, + }), + }) + ) + expect(mocks.update).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ updates: { required: false } }) }) + ) + }) + + it('rejects an unrecognized key on the column delete body', async () => { + const response = await DELETE( + request('DELETE', { + workspaceId: WORKSPACE_ID, + columnName: 'Other', + columnNames: ['Other'], + }), + context + ) + + expect(response.status).toBe(400) + expect(mocks.remove).not.toHaveBeenCalled() + }) + it('maps typed application validation failures without inspecting messages', async () => { mocks.update.mockRejectedValueOnce(new OrchestrationError('validation', 'Invalid column')) @@ -133,4 +166,16 @@ describe('/api/v2/tables/[tableId]/columns', () => { expect(response.status).toBe(200) expect(mocks.remove).toHaveBeenCalledOnce() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await POST( + request('POST', { workspaceId: WORKSPACE_ID, column: { name: 'Name', type: 'string' } }), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts b/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts index 03030a64729..4440d408b74 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/columns/route.ts @@ -12,7 +12,7 @@ import { updateTableColumnUseCase, } from '@/lib/table/application/columns' import { tableOperations } from '@/lib/table/application/operations' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' export const dynamic = 'force-dynamic' export const revalidate = 0 diff --git a/apps/sim/app/api/v2/tables/[tableId]/columns/run/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/columns/run/route.test.ts index 83b77f98923..e9257648ab5 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/columns/run/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/columns/run/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,28 +18,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), startRun: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, })) @@ -49,16 +45,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} function call(body: unknown) { const request = new NextRequest('http://localhost/api/v2/tables/table-1/columns/run', { @@ -75,10 +65,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/columns/run', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.startRun.mockResolvedValue({ table: { id: 'table-1' }, dispatchId: 'dispatch-1' }) }) @@ -143,4 +133,13 @@ describe('POST /api/v2/tables/[tableId]/columns/run', () => { expect(response.status).toBe(400) expect(mocks.startRun).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID, groupIds: ['group-1'] }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/exports/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/exports/route.test.ts index ad7a2d1bc2f..b2c700871c9 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/exports/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/exports/route.test.ts @@ -2,31 +2,27 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), create: vi.fn(), read: vi.fn(), download: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/tables/presenters', () => ({ presentV2TableExport: (tableExport: unknown) => ({ data: tableExport }), })) @@ -53,16 +49,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const tableExport = { id: 'export-1', tableId: 'table-1', @@ -79,10 +69,10 @@ const tableExport = { describe('v2 table exports', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) }) it('creates an export through the authorized use case', async () => { @@ -140,4 +130,18 @@ describe('v2 table exports', () => { request: downloadRequest, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + const request = new NextRequest('http://localhost:3000/api/v2/tables/table-1/exports', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ workspaceId: WORKSPACE_ID, format: 'csv' }), + }) + + const response = await POST(request, { params: Promise.resolve({ tableId: 'table-1' }) }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/groups/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/groups/route.test.ts index 35acd0b7739..1d8c2b1fe43 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/groups/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/groups/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), list: vi.fn(), create: vi.fn(), update: vi.fn(), remove: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/groups', () => ({ listTableGroupsUseCase: { operation: { id: 'tables.groups.list' }, execute: mocks.list }, createTableGroupUseCase: { operation: { id: 'tables.groups.create' }, execute: mocks.create }, @@ -47,16 +43,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const group = { id: 'group-1', workflowId: 'workflow-1', @@ -93,11 +83,11 @@ function writeRequest(method: 'POST' | 'PATCH' | 'DELETE', body: unknown) { describe('/api/v2/tables/[tableId]/groups', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) - mocks.list.mockResolvedValue({ groups: [group] }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.list.mockResolvedValue({ table, groups: [group] }) mocks.create.mockResolvedValue({ table, group }) mocks.update.mockResolvedValue({ table, group, changed: true, startAutoRun: false }) mocks.remove.mockResolvedValue({ table, groupId: 'group-1' }) @@ -110,7 +100,10 @@ describe('/api/v2/tables/[tableId]/groups', () => { const response = await GET(req, context) expect(response.status).toBe(200) - expect(await response.json()).toEqual({ data: [group], nextCursor: null }) + expect(await response.json()).toEqual({ + data: [{ ...group, outputs: [{ ...group.outputs[0], columnName: 'Result' }] }], + nextCursor: null, + }) expect(mocks.list).toHaveBeenCalledWith({ principal, input: { tableId: 'table-1', workspaceId: WORKSPACE_ID }, @@ -118,6 +111,20 @@ describe('/api/v2/tables/[tableId]/groups', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/tables/table-1/groups?workspaceId=${WORKSPACE_ID}` + ), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('defaults create autoRun off and delegates all execution initiation to the application layer', async () => { const req = writeRequest('POST', { workspaceId: WORKSPACE_ID, @@ -132,6 +139,25 @@ describe('/api/v2/tables/[tableId]/groups', () => { expect(response.status).toBe(201) expect((await response.json()).data.group.id).toBe('group-1') + + /** + * `columnName` is sent as a column name and stored as a column id; reading + * back the id under the same field made the value un-round-trippable and + * unmatched by anything else on a surface that is otherwise name-keyed. + */ + const created = await POST( + writeRequest('POST', { + workspaceId: WORKSPACE_ID, + group: { + workflowId: 'workflow-1', + type: 'manual', + outputs: [{ blockId: 'block-1', path: 'result', columnName: 'Result' }], + }, + outputColumns: [{ name: 'Result', type: 'string' }], + }), + context + ) + expect((await created.json()).data.group.outputs[0].columnName).toBe('Result') expect(mocks.create).toHaveBeenCalledWith({ principal, input: expect.objectContaining({ diff --git a/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts b/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts index 8910958f55a..178e9564382 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/groups/route.ts @@ -13,7 +13,8 @@ import { updateTableGroupUseCase, } from '@/lib/table/application/groups' import { tableOperations } from '@/lib/table/application/operations' -import { normalizeColumn } from '@/app/api/table/utils' +import { normalizeColumn } from '@/lib/table/wire' +import { presentV2WorkflowGroup } from '@/app/api/v2/tables/presenters' export const dynamic = 'force-dynamic' export const revalidate = 0 @@ -26,7 +27,10 @@ export const GET = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2TableErrorPolicies.concealTableAuthorization, mapInput: ({ params, query }) => ({ tableId: params.tableId, workspaceId: query.workspaceId }), - present: ({ groups }) => ({ data: groups, nextCursor: null }), + present: ({ table, groups }) => ({ + data: groups.map((group) => presentV2WorkflowGroup(group, table.schema)), + nextCursor: null, + }), }) export const POST = defineV2JsonRoute({ @@ -38,7 +42,10 @@ export const POST = defineV2JsonRoute({ errorPolicy: v2TableErrorPolicies.concealTableAuthorization, mapInput: ({ params, body }) => ({ tableId: params.tableId, ...body }), present: ({ table, group }) => ({ - data: { group, columns: table.schema.columns.map(normalizeColumn) }, + data: { + group: presentV2WorkflowGroup(group, table.schema), + columns: table.schema.columns.map(normalizeColumn), + }, }), }) @@ -51,7 +58,10 @@ export const PATCH = defineV2JsonRoute({ errorPolicy: v2TableErrorPolicies.concealTableAuthorization, mapInput: ({ params, body }) => ({ tableId: params.tableId, ...body }), present: ({ table, group }) => ({ - data: { group, columns: table.schema.columns.map(normalizeColumn) }, + data: { + group: presentV2WorkflowGroup(group, table.schema), + columns: table.schema.columns.map(normalizeColumn), + }, }), }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/query/count/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/query/count/route.test.ts new file mode 100644 index 00000000000..576b3925505 --- /dev/null +++ b/apps/sim/app/api/v2/tables/[tableId]/query/count/route.test.ts @@ -0,0 +1,181 @@ +/** + * @vitest-environment node + */ + +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { + class MockTableRowsValidationError extends Error { + constructor( + message: string, + readonly details?: unknown + ) { + super(message) + } + } + return { + mocks: { + queryRows: vi.fn(), + }, + MockTableRowsValidationError, + } +}) + +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) +vi.mock('@/lib/table/application/rows', () => ({ + TableRowsValidationError: MockTableRowsValidationError, + queryTableRows: { operation: { id: 'tables.rows.query' }, execute: mocks.queryRows }, +})) + +import { POST } from '@/app/api/v2/tables/[tableId]/query/count/route' + +const WORKSPACE_ID = 'workspace-1' +const PRINCIPAL = { + kind: 'workspace_api_key' as const, + workspaceId: WORKSPACE_ID, + keyId: 'key-1', +} +const AUTH = { + principal: PRINCIPAL, + rolloutUserId: 'owner-1', + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], + rateLimitSubscription: null, + keyType: 'workspace' as const, +} +const TABLE = { + id: 'table-1', + workspaceId: WORKSPACE_ID, + schema: { columns: [{ id: 'column-name', name: 'name', type: 'string' as const }] }, +} +const ROW = { + id: 'row-1', + data: { 'column-name': 'Ada' }, + createdAt: new Date('2026-01-01T00:00:00Z'), + updatedAt: new Date('2026-01-02T00:00:00Z'), +} + +function call(body: unknown) { + const request = new NextRequest('http://localhost/api/v2/tables/table-1/query/count', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify(body), + }) + return { + request, + response: POST(request, { params: Promise.resolve({ tableId: 'table-1' }) }), + } +} + +describe('POST /api/v2/tables/[tableId]/query/count', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.queryRows.mockResolvedValue({ + table: TABLE, + rows: [ROW], + rowCount: 1, + totalCount: 4321, + nextCursor: 'cursor-1', + }) + }) + + it('counts the predicate matches across the whole table, not the page', async () => { + const predicate = { all: [{ field: 'name', op: 'eq', value: 'Ada' }] } + const invocation = call({ workspaceId: WORKSPACE_ID, predicate }) + const response = await invocation.response + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ data: { totalCount: 4321 } }) + expect(mocks.queryRows).toHaveBeenCalledWith({ + principal: PRINCIPAL, + input: { + tableId: 'table-1', + assertedWorkspaceId: WORKSPACE_ID, + predicate, + limit: 1, + includeTotal: true, + }, + request: invocation.request, + }) + }) + + it('counts the whole table when no predicate is sent', async () => { + mocks.queryRows.mockResolvedValue({ + table: TABLE, + rows: [], + rowCount: 0, + totalCount: 0, + nextCursor: null, + }) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ data: { totalCount: 0 } }) + expect(mocks.queryRows).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ predicate: undefined }) }) + ) + }) + + it('rejects the paging controls a count has no use for', async () => { + const response = await call({ workspaceId: WORKSPACE_ID, limit: 10, cursor: 'x' }).response + + expect(response.status).toBe(400) + expect(mocks.queryRows).not.toHaveBeenCalled() + }) + + it('keeps a malformed predicate as a structured 400', async () => { + mocks.queryRows.mockRejectedValue( + new MockTableRowsValidationError('Unknown column "nope"', { code: 'INVALID_PREDICATE' }) + ) + + const response = await call({ + workspaceId: WORKSPACE_ID, + predicate: { all: [{ field: 'nope', op: 'eq', value: 1 }] }, + }).response + + expect(response.status).toBe(400) + expect((await response.json()).error.details).toEqual({ code: 'INVALID_PREDICATE' }) + }) + + it('never presents a fabricated zero when no total was computed', async () => { + mocks.queryRows.mockResolvedValue({ + table: TABLE, + rows: [ROW], + rowCount: 1, + totalCount: null, + nextCursor: null, + }) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(500) + expect(await response.json()).toEqual({ + error: { code: 'INTERNAL_ERROR', message: 'Internal server error' }, + }) + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) +}) diff --git a/apps/sim/app/api/v2/tables/[tableId]/query/count/route.ts b/apps/sim/app/api/v2/tables/[tableId]/query/count/route.ts new file mode 100644 index 00000000000..f71712224dc --- /dev/null +++ b/apps/sim/app/api/v2/tables/[tableId]/query/count/route.ts @@ -0,0 +1,45 @@ +import { TABLE_QUERY_MAX_BODY_BYTES } from '@/lib/api/contracts/tables' +import { v2QueryRowsCountContract } from '@/lib/api/contracts/v2/tables' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2TableRowsErrorPolicy } from '@/lib/table/api/row-route-policies' +import { tableOperations } from '@/lib/table/application/operations' +import { queryTableRows } from '@/lib/table/application/rows' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * Counts the rows a predicate matches. + * + * The same `queryTableRows` read the paged endpoints use, asked for its total + * instead of its page: `includeTotal` runs a COUNT over the full predicate view + * (not the page's keyset window), and `limit: 1` keeps the row drain that runs + * alongside it to a single row rather than a full default page. + * + * `totalCount` is `number | null` on the use-case result because callers may ask + * for a page without a total. This route always asks for one, so a null here is + * a broken invariant rather than a reachable outcome — it fails loudly instead + * of being coerced into a plausible-looking zero. + */ +export const POST = defineV2JsonRoute({ + contract: v2QueryRowsCountContract, + operation: tableOperations.queryRows, + auth: v2ApiKeyAuth, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2TableRowsErrorPolicy, + parseOptions: { maxBodyBytes: TABLE_QUERY_MAX_BODY_BYTES }, + mapInput: ({ params, body }) => ({ + tableId: params.tableId, + assertedWorkspaceId: body.workspaceId, + predicate: body.predicate, + limit: 1, + includeTotal: true, + }), + useCase: queryTableRows, + present: ({ totalCount }) => { + if (totalCount === null) { + throw new Error('Table row count requested with includeTotal but no total was computed') + } + return { data: { totalCount } } + }, +}) diff --git a/apps/sim/app/api/v2/tables/[tableId]/query/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/query/route.test.ts index 88ab015bf3c..e271d8eee6c 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/query/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/query/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -16,35 +25,30 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { } return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), queryRows: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, queryTableRows: { operation: { id: 'tables.rows.query' }, execute: mocks.queryRows }, })) +import { v2QueryRowsContract } from '@/lib/api/contracts/v2/tables' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' +import { encodeScopedCursor } from '@/app/api/v2/lib/response' import { POST } from '@/app/api/v2/tables/[tableId]/query/route' +/** A query cursor exactly as the route mints one, for the table given. */ +function queryCursor(tableId: string, inner: string): string { + return encodeScopedCursor(cursorScopeKey(cursorRoute(v2QueryRowsContract, { tableId })), inner) +} + const WORKSPACE_ID = 'workspace-1' const PRINCIPAL = { kind: 'workspace_api_key' as const, @@ -54,16 +58,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -91,10 +89,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/query', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.queryRows.mockResolvedValue({ table: TABLE, rows: [ROW], nextCursor: null }) }) @@ -138,12 +136,22 @@ describe('POST /api/v2/tables/[tableId]/query', () => { ) }) + it('rejects a v1-shaped filter key instead of answering with an unfiltered page', async () => { + const response = await call({ + workspaceId: WORKSPACE_ID, + filter: { name: { $eq: 'Ada' } }, + }).response + + expect(response.status).toBe(400) + expect(mocks.queryRows).not.toHaveBeenCalled() + }) + it('rejects an invalid page limit after admission and before delegation', async () => { const response = await call({ workspaceId: WORKSPACE_ID, limit: 5000 }).response expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalledOnce() - expect(mocks.operationRate).toHaveBeenCalledOnce() + expect(v2RouteMocks.authenticate).toHaveBeenCalledOnce() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(AUTH.rateLimitSubjectIds.length) expect(mocks.queryRows).not.toHaveBeenCalled() }) @@ -152,12 +160,31 @@ describe('POST /api/v2/tables/[tableId]/query', () => { new MockTableRowsValidationError('Invalid cursor', { code: 'INVALID_CURSOR' }) ) - const response = await call({ workspaceId: WORKSPACE_ID, cursor: 'malformed' }).response + const response = await call({ + workspaceId: WORKSPACE_ID, + cursor: queryCursor('table-1', 'malformed'), + }).response expect(response.status).toBe(400) expect((await response.json()).error.details).toEqual({ code: 'INVALID_CURSOR' }) }) + /** + * The row codec binds the predicate and sort a page was produced under but + * carries no table identity, so an unfiltered token from one table decoded + * cleanly against another and answered 200 with that other table's rows. + */ + it('refuses a query cursor minted on a different table', async () => { + const response = await call({ + workspaceId: WORKSPACE_ID, + cursor: queryCursor('table-2', 'native-row-cursor'), + }).response + + expect(response.status).toBe(400) + expect((await response.json()).error.message).toMatch(/requested filters/) + expect(mocks.queryRows).not.toHaveBeenCalled() + }) + it('enforces the one MiB body cap before delegation', async () => { const response = await call({ workspaceId: WORKSPACE_ID, cursor: 'x'.repeat(1024 * 1024) }) .response @@ -165,4 +192,13 @@ describe('POST /api/v2/tables/[tableId]/query', () => { expect(response.status).toBe(413) expect(mocks.queryRows).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/query/route.ts b/apps/sim/app/api/v2/tables/[tableId]/query/route.ts index 76805afa9b5..5b9d0516056 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/query/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/query/route.ts @@ -1,15 +1,29 @@ import { TABLE_QUERY_MAX_BODY_BYTES } from '@/lib/api/contracts/tables' import { V2_DEFAULT_ROW_LIMIT, v2QueryRowsContract } from '@/lib/api/contracts/v2/tables' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { v2TableRowsErrorPolicy } from '@/lib/table/api/row-route-policies' import { tableOperations } from '@/lib/table/application/operations' import { queryTableRows } from '@/lib/table/application/rows' import { namedRowMapper } from '@/lib/table/cell-format' +import { encodeScopedCursor, readScopedCursor } from '@/app/api/v2/lib/response' import { toApiRow } from '@/app/api/v2/tables/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** + * The sequence a query cursor names a position in: this list, on THIS table. + * + * The row codec binds the predicate and sort a page was produced under, but not + * the table — so an unfiltered token from one table decoded cleanly against + * another and answered 200 with that other table's rows. The table id lives in + * the path, so the route is the only place that knows it. + */ +function queryRowCursorScope(tableId: string): string { + return cursorScopeKey(cursorRoute(v2QueryRowsContract, { tableId })) +} + export const POST = defineV2JsonRoute({ contract: v2QueryRowsContract, operation: tableOperations.queryRows, @@ -22,17 +36,19 @@ export const POST = defineV2JsonRoute({ assertedWorkspaceId: body.workspaceId, predicate: body.predicate, sort: body.sort, - cursor: body.cursor, + cursor: readScopedCursor(body.cursor, queryRowCursorScope(params.tableId)), limit: body.limit === undefined ? V2_DEFAULT_ROW_LIMIT : body.limit === 0 ? undefined : body.limit, includeTotal: false, }), useCase: queryTableRows, - present: ({ table, rows, nextCursor }) => { + present: ({ table, rows, nextCursor }, { params }) => { const toNamedRow = namedRowMapper(table.schema.columns) return { data: rows.map((row) => toApiRow(row, toNamedRow)), - nextCursor, + nextCursor: nextCursor + ? encodeScopedCursor(queryRowCursorScope(params.tableId), nextCursor) + : null, } }, }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/route.test.ts index da221f15235..5060a47a762 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/route.test.ts @@ -2,14 +2,19 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), read: vi.fn(), update: vi.fn(), remove: vi.fn(), @@ -18,18 +23,9 @@ const mocks = vi.hoisted(() => ({ getMaxRowsPerTable: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mocks.capture })) vi.mock('@/lib/table/application/tables', () => ({ readTableUseCase: { operation: { id: 'tables.read' }, execute: mocks.read }, @@ -57,16 +53,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const table = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -107,10 +97,10 @@ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { describe('/api/v2/tables/[tableId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['owner-1', 'owner@example.com']])) mocks.getMaxRowsPerTable.mockResolvedValue(5000) mocks.read.mockResolvedValue({ table, folderPath: '/' }) @@ -147,6 +137,15 @@ describe('/api/v2/tables/[tableId]', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request('GET'), context) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('preserves a successful no-op PATCH response', async () => { const response = await PATCH( request('PATCH', { workspaceId: WORKSPACE_ID, name: 'Contacts' }), diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]/route.test.ts index c0e1306fe6a..31671eb4fa4 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,28 +18,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), startRun: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, })) @@ -50,16 +46,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} function call(body: unknown) { const request = new NextRequest( @@ -81,10 +71,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.startRun.mockResolvedValue({ table: { id: 'table-1' }, dispatchId: 'dispatch-1' }) }) @@ -116,6 +106,15 @@ describe('POST /api/v2/tables/[tableId]/rows/[rowId]/enrichment/[groupId]', () = expect(await response.json()).toEqual({ data: { dispatchId: null } }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('rejects a missing workspace before delegation', async () => { const response = await call({}).response diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.test.ts index 8a2395c073a..387473a3935 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,10 +18,6 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), readRow: vi.fn(), updateRow: vi.fn(), deleteRow: vi.fn(), @@ -21,18 +26,9 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, readTableRow: { operation: { id: 'tables.rows.read' }, execute: mocks.readRow }, @@ -53,16 +49,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -93,10 +83,10 @@ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { describe('/api/v2/tables/[tableId]/rows/[rowId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.readRow.mockResolvedValue({ table: TABLE, row: ROW }) mocks.updateRow.mockResolvedValue({ table: TABLE, row: ROW, changed: true }) mocks.deleteRow.mockResolvedValue({ table: TABLE, deletedRowId: ROW.id }) @@ -120,6 +110,15 @@ describe('/api/v2/tables/[tableId]/rows/[rowId]', () => { }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request('GET'), CONTEXT) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('updates through the shared use case with the exact patch', async () => { const req = request('PATCH', { workspaceId: WORKSPACE_ID, data: { name: 'Ada' } }) const response = await PATCH(req, CONTEXT) @@ -132,6 +131,7 @@ describe('/api/v2/tables/[tableId]/rows/[rowId]', () => { rowId: 'row-1', assertedWorkspaceId: WORKSPACE_ID, data: { name: 'Ada' }, + strictWrite: true, }, request: req, }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.ts index 446ecf0ee97..465577e099a 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/[rowId]/route.ts @@ -41,6 +41,7 @@ export const PATCH = defineV2JsonRoute({ rowId: params.rowId, assertedWorkspaceId: body.workspaceId, data: body.data, + strictWrite: true, }), useCase: updateTableRow, present: ({ table, row }) => ({ diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/find/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/find/route.test.ts index e86f657c272..293935428a1 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/find/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/find/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,33 +18,21 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), findRows: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, findTableRows: { operation: { id: 'tables.rows.find' }, execute: mocks.findRows }, })) +import { v2Error } from '@/app/api/v2/lib/response' import { POST } from '@/app/api/v2/tables/[tableId]/rows/find/route' const WORKSPACE_ID = 'workspace-1' @@ -47,16 +44,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -78,10 +69,10 @@ function call(body: unknown) { describe('POST /api/v2/tables/[tableId]/rows/find', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.findRows.mockResolvedValue({ table: TABLE, matches: [{ ordinal: 3, rowId: 'row-1', column: 'column-name' }], @@ -119,17 +110,25 @@ describe('POST /api/v2/tables/[tableId]/rows/find', () => { const response = await call({ workspaceId: WORKSPACE_ID, q: '' }).response expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalledOnce() + expect(v2RouteMocks.authenticate).toHaveBeenCalledOnce() expect(mocks.findRows).not.toHaveBeenCalled() }) it('stops at the rollout gate before the shared use case', async () => { - const { v2Error } = await import('@/app/api/v2/lib/response') - mocks.gate.mockResolvedValue(v2Error('NOT_FOUND', 'Not found')) + v2RouteMocks.gate.mockResolvedValue(v2Error('NOT_FOUND', 'Not found')) const response = await call({ workspaceId: WORKSPACE_ID, q: 'ada' }).response expect(response.status).toBe(404) expect(mocks.findRows).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await call({ workspaceId: WORKSPACE_ID, q: 'ada' }).response + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/route.test.ts index 9c4eaa74426..1f0b8a3b5e7 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,10 +18,6 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), listRows: vi.fn(), createRows: vi.fn(), updateRows: vi.fn(), @@ -22,18 +27,9 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, listTableRows: { operation: { id: 'tables.rows.list' }, execute: mocks.listRows }, @@ -42,8 +38,19 @@ vi.mock('@/lib/table/application/rows', () => ({ deleteTableRows: { operation: { id: 'tables.rows.delete_many' }, execute: mocks.deleteRows }, })) +import { v2ListTableRowsContract } from '@/lib/api/contracts/v2/tables' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' +import { encodeScopedCursor } from '@/app/api/v2/lib/response' import { DELETE, GET, PATCH, POST } from '@/app/api/v2/tables/[tableId]/rows/route' +/** A row cursor exactly as the route mints one, for the table given. */ +function rowCursor(tableId: string, inner: string): string { + return encodeScopedCursor( + cursorScopeKey(cursorRoute(v2ListTableRowsContract, { tableId })), + inner + ) +} + const WORKSPACE_ID = 'workspace-1' const PRINCIPAL = { kind: 'workspace_api_key' as const, @@ -53,16 +60,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -90,10 +91,10 @@ function request(method: 'GET' | 'POST' | 'PATCH' | 'DELETE', body?: unknown, qu describe('/api/v2/tables/[tableId]/rows', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listRows.mockResolvedValue({ table: TABLE, rows: [ROW], nextCursor: null }) mocks.createRows.mockResolvedValue({ kind: 'single', table: TABLE, row: ROW }) mocks.updateRows.mockResolvedValue({ @@ -112,7 +113,7 @@ describe('/api/v2/tables/[tableId]/rows', () => { }) it('passes the opaque native row cursor through the route unchanged', async () => { - const cursor = 'native-row-cursor' + const cursor = rowCursor('table-1', 'native-row-cursor') mocks.listRows.mockResolvedValue({ table: TABLE, rows: [ROW], @@ -132,16 +133,52 @@ describe('/api/v2/tables/[tableId]/rows', () => { tableId: 'table-1', assertedWorkspaceId: WORKSPACE_ID, limit: 25, - cursor, + cursor: 'native-row-cursor', }, request: req, }) - expect((await response.json()).nextCursor).toBe('next-native-cursor') + expect((await response.json()).nextCursor).toBe(rowCursor('table-1', 'next-native-cursor')) + }) + + /** + * The row codec binds the sort and predicate a page was produced under but + * carries no table identity, so an unfiltered token from one table decoded + * cleanly against another and answered 200 with that other table's rows. + */ + it('refuses a row cursor minted on a different table', async () => { + const foreign = rowCursor('table-2', 'native-row-cursor') + const response = await GET( + request( + 'GET', + undefined, + `?workspaceId=${WORKSPACE_ID}&limit=25&cursor=${encodeURIComponent(foreign)}` + ), + CONTEXT + ) + + expect(response.status).toBe(400) + expect((await response.json()).error.message).toMatch(/requested filters/) + expect(mocks.listRows).not.toHaveBeenCalled() + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + request('GET', undefined, `?workspaceId=${WORKSPACE_ID}&limit=25`), + CONTEXT + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') }) it('delegates single and batch creation through one semantic use case', async () => { const single = request('POST', { workspaceId: WORKSPACE_ID, data: { name: 'Ada' } }) - expect((await (await POST(single, CONTEXT)).json()).data.id).toBe('row-1') + const singleResponse = await POST(single, CONTEXT) + // 201 on both arms: every v2 create answers the same status, batch included. + expect(singleResponse.status).toBe(201) + expect((await singleResponse.json()).data.id).toBe('row-1') expect(mocks.createRows).toHaveBeenLastCalledWith({ principal: PRINCIPAL, input: { @@ -149,13 +186,19 @@ describe('/api/v2/tables/[tableId]/rows', () => { tableId: 'table-1', assertedWorkspaceId: WORKSPACE_ID, data: { name: 'Ada' }, + // v2 alone opts into the strict write contract: an unknown column name + // or a value the column cannot hold is a 400, not a dropped key or a + // nulled cell. Every first-party surface leaves this unset. + strictWrite: true, }, request: single, }) mocks.createRows.mockResolvedValue({ kind: 'batch', table: TABLE, rows: [ROW] }) const batch = request('POST', { workspaceId: WORKSPACE_ID, rows: [{ name: 'Ada' }] }) - expect((await (await POST(batch, CONTEXT)).json()).data.insertedCount).toBe(1) + const batchResponse = await POST(batch, CONTEXT) + expect(batchResponse.status).toBe(201) + expect((await batchResponse.json()).data.insertedCount).toBe(1) expect(mocks.createRows).toHaveBeenLastCalledWith({ principal: PRINCIPAL, input: { @@ -163,6 +206,7 @@ describe('/api/v2/tables/[tableId]/rows', () => { tableId: 'table-1', assertedWorkspaceId: WORKSPACE_ID, rows: [{ name: 'Ada' }], + strictWrite: true, }, request: batch, }) @@ -198,4 +242,38 @@ describe('/api/v2/tables/[tableId]/rows', () => { }, }) }) + /** + * A table cell is `z.unknown()` on the wire — its type is decided by the + * column, not the contract — so no string schema guards it. A `U+0000` in a + * cell value or a predicate value therefore travelled all the way to the + * driver and came back as `500 INTERNAL_ERROR`. + */ + describe('NUL bytes in table values', () => { + const NUL = '\u0000' + + it('rejects a NUL in a cell value before the row use case runs', async () => { + const response = await POST( + request('POST', { workspaceId: WORKSPACE_ID, data: { name: `a${NUL}b` } }), + CONTEXT + ) + + expect(response.status).toBe(400) + expect((await response.json()).error.code).toBe('BAD_REQUEST') + expect(mocks.createRows).not.toHaveBeenCalled() + }) + + it('rejects a NUL in a predicate value on the update-by-filter path', async () => { + const response = await PATCH( + request('PATCH', { + workspaceId: WORKSPACE_ID, + filter: { all: [{ field: 'name', op: 'contains', value: `a${NUL}b` }] }, + data: { name: 'Grace' }, + }), + CONTEXT + ) + + expect(response.status).toBe(400) + expect(mocks.updateRows).not.toHaveBeenCalled() + }) + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/route.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/route.ts index 2d7d9592135..4727a7bba6a 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/route.ts @@ -4,6 +4,7 @@ import { v2ListTableRowsContract, v2UpdateRowsByFilterContract, } from '@/lib/api/contracts/v2/tables' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { v2TableRowsErrorPolicy } from '@/lib/table/api/row-route-policies' import { tableOperations } from '@/lib/table/application/operations' @@ -14,11 +15,24 @@ import { updateTableRows, } from '@/lib/table/application/rows' import { namedRowMapper } from '@/lib/table/cell-format' +import { encodeScopedCursor, readScopedCursor } from '@/app/api/v2/lib/response' import { toApiRow } from '@/app/api/v2/tables/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** + * The sequence a row cursor names a position in: this list, on THIS table. + * + * The row codec binds a token to the sort and predicate it was minted under but + * carries no table identity, so an unfiltered token from one table decoded + * cleanly against another and answered 200 with that other table's rows. The + * table id lives in the path, so the route is the only place that knows it. + */ +function tableRowCursorScope(tableId: string): string { + return cursorScopeKey(cursorRoute(v2ListTableRowsContract, { tableId })) +} + export const GET = defineV2JsonRoute({ contract: v2ListTableRowsContract, operation: tableOperations.listRows, @@ -29,14 +43,16 @@ export const GET = defineV2JsonRoute({ tableId: params.tableId, assertedWorkspaceId: query.workspaceId, limit: query.limit, - cursor: query.cursor, + cursor: readScopedCursor(query.cursor, tableRowCursorScope(params.tableId)), }), useCase: listTableRows, - present: ({ table, rows, nextCursor }) => { + present: ({ table, rows, nextCursor }, { params }) => { const toNamedRow = namedRowMapper(table.schema.columns) return { data: rows.map((row) => toApiRow(row, toNamedRow)), - nextCursor, + nextCursor: nextCursor + ? encodeScopedCursor(tableRowCursorScope(params.tableId), nextCursor) + : null, } }, }) @@ -54,6 +70,7 @@ export const POST = defineV2JsonRoute({ tableId: params.tableId, assertedWorkspaceId: body.workspaceId, rows: body.rows, + strictWrite: true, } : { kind: 'single' as const, @@ -62,6 +79,7 @@ export const POST = defineV2JsonRoute({ data: body.data, afterRowId: body.afterRowId, beforeRowId: body.beforeRowId, + strictWrite: true, }, useCase: createTableRows, present: (result) => { @@ -89,6 +107,7 @@ export const PATCH = defineV2JsonRoute({ filter: body.filter, data: body.data, limit: body.limit, + strictWrite: true, }), useCase: updateTableRows, present: ({ affectedCount, affectedRowIds }) => ({ diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.test.ts index ddcd5106649..e17f2760632 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.test.ts @@ -2,6 +2,15 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' @@ -9,28 +18,15 @@ const { mocks, MockTableRowsValidationError } = vi.hoisted(() => { class MockTableRowsValidationError extends Error {} return { mocks: { - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), upsertRow: vi.fn(), }, MockTableRowsValidationError, } }) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/rows', () => ({ TableRowsValidationError: MockTableRowsValidationError, upsertTableRow: { operation: { id: 'tables.rows.upsert' }, execute: mocks.upsertRow }, @@ -47,16 +43,10 @@ const PRINCIPAL = { const AUTH = { principal: PRINCIPAL, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const RATE = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00Z'), - retryAfterMs: 0, -} const TABLE = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -72,10 +62,10 @@ const ROW = { describe('POST /api/v2/tables/[tableId]/rows/upsert', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(AUTH) - mocks.preauthRate.mockResolvedValue(RATE) - mocks.operationRate.mockResolvedValue(RATE) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(AUTH) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.upsertRow.mockResolvedValue({ table: TABLE, row: ROW, operation: 'update' }) }) @@ -112,11 +102,32 @@ describe('POST /api/v2/tables/[tableId]/rows/upsert', () => { assertedWorkspaceId: WORKSPACE_ID, data: { email: 'ada@example.com' }, conflictTarget: 'email', + strictWrite: true, }, request, }) }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const request = new NextRequest('http://localhost/api/v2/tables/table-1/rows/upsert', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify({ + workspaceId: WORKSPACE_ID, + data: { email: 'ada@example.com' }, + conflictTarget: 'email', + }), + }) + const response = await POST(request, { + params: Promise.resolve({ tableId: 'table-1' }), + }) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('rejects an empty conflict target before delegation', async () => { const request = new NextRequest('http://localhost/api/v2/tables/table-1/rows/upsert', { method: 'POST', diff --git a/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.ts b/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.ts index 6c2d84285de..26550374d43 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/rows/upsert/route.ts @@ -20,6 +20,7 @@ export const POST = defineV2JsonRoute({ assertedWorkspaceId: body.workspaceId, data: body.data, conflictTarget: body.conflictTarget, + strictWrite: true, }), useCase: upsertTableRow, present: ({ table, row, operation }) => ({ diff --git a/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.test.ts index fa253c093f4..c680afffa9c 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), read: vi.fn(), update: vi.fn(), remove: vi.fn(), email: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/views', () => ({ readTableViewUseCase: { operation: { id: 'tables.views.read' }, execute: mocks.read }, updateTableViewUseCase: { operation: { id: 'tables.views.update' }, execute: mocks.update }, @@ -46,21 +42,23 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} +const columns = [ + { id: 'col_a', name: 'Status', type: 'text' as const }, + { id: 'col_b', name: 'Email', type: 'text' as const }, +] const view = { id: 'view-1', tableId: 'table-1', name: 'Active', - config: {}, + config: { + hiddenColumns: ['col_b'], + sort: [{ field: 'col_a', direction: 'desc' as const }], + filter: { all: [{ field: 'col_a', op: 'eq' as const, value: 'open' }] }, + }, isDefault: true, createdBy: 'user-1', createdAt: new Date('2026-01-01T00:00:00.000Z'), @@ -82,12 +80,12 @@ function request(method: 'GET' | 'PATCH' | 'DELETE', body?: unknown) { describe('/api/v2/tables/[tableId]/views/[viewId]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) - mocks.read.mockResolvedValue({ view }) - mocks.update.mockResolvedValue({ view, changed: false }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.read.mockResolvedValue({ view, columns }) + mocks.update.mockResolvedValue({ view, columns, changed: false }) mocks.remove.mockResolvedValue({ viewId: 'view-1' }) mocks.email.mockResolvedValue('user@example.com') }) @@ -105,6 +103,20 @@ describe('/api/v2/tables/[tableId]/views/[viewId]', () => { }) }) + /** + * Storage keys on stable column ids; this surface reads and writes column + * names, so a `col_…` id must never reach the caller. + */ + it('presents the saved config keyed by column name', async () => { + const response = await GET(request('GET'), context) + + expect((await response.json()).data.config).toEqual({ + hiddenColumns: ['Email'], + sort: [{ field: 'Status', direction: 'desc' }], + filter: { all: [{ field: 'Status', op: 'eq', value: 'open' }] }, + }) + }) + it('preserves no-op PATCH response compatibility', async () => { const response = await PATCH( request('PATCH', { workspaceId: WORKSPACE_ID, name: 'Active' }), @@ -122,4 +134,13 @@ describe('/api/v2/tables/[tableId]/views/[viewId]', () => { expect(await response.json()).toEqual({ data: { id: 'view-1', deleted: true } }) expect(mocks.remove).toHaveBeenCalledOnce() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request('GET'), context) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.ts b/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.ts index 9f9ccada5c4..e7d4a2eaed2 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/views/[viewId]/route.ts @@ -17,10 +17,17 @@ import { toApiView } from '@/app/api/v2/tables/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 -async function presentView(result: { view: Parameters[0] }) { - const { view } = result +async function presentView(result: { + view: Parameters[0] + columns: Parameters[2] +}) { + const { view, columns } = result return { - data: toApiView(view, view.createdBy ? await getRequiredUserEmail(view.createdBy) : null), + data: toApiView( + view, + view.createdBy ? await getRequiredUserEmail(view.createdBy) : null, + columns + ), } } diff --git a/apps/sim/app/api/v2/tables/[tableId]/views/route.test.ts b/apps/sim/app/api/v2/tables/[tableId]/views/route.test.ts index 3f2e6a20288..006383eac23 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/views/route.test.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/views/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), list: vi.fn(), create: vi.fn(), emails: vi.fn(), email: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/views', () => ({ listTableViewsUseCase: { operation: { id: 'tables.views.list' }, execute: mocks.list }, createTableViewUseCase: { operation: { id: 'tables.views.create' }, execute: mocks.create }, @@ -49,21 +45,16 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} +const columns = [{ id: 'col_a', name: 'Status', type: 'text' as const }] const view = { id: 'view-1', tableId: 'table-1', name: 'Active', - config: {}, + config: { sort: [{ field: 'col_a', direction: 'desc' as const }] }, isDefault: false, createdBy: 'user-1', createdAt: new Date('2026-01-01T00:00:00.000Z'), @@ -74,12 +65,12 @@ const context = { params: Promise.resolve({ tableId: 'table-1' }) } describe('/api/v2/tables/[tableId]/views', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) - mocks.list.mockResolvedValue({ views: [view] }) - mocks.create.mockResolvedValue({ view }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.list.mockResolvedValue({ views: [view], columns }) + mocks.create.mockResolvedValue({ view, columns }) mocks.emails.mockResolvedValue(new Map([['user-1', 'user@example.com']])) mocks.email.mockResolvedValue('user@example.com') }) @@ -97,7 +88,7 @@ describe('/api/v2/tables/[tableId]/views', () => { id: 'view-1', tableId: 'table-1', name: 'Active', - config: {}, + config: { sort: [{ field: 'Status', direction: 'desc' }] }, isDefault: false, createdByEmail: 'user@example.com', createdAt: '2026-01-01T00:00:00.000Z', @@ -129,4 +120,18 @@ describe('/api/v2/tables/[tableId]/views', () => { request: req, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest( + `http://localhost:3000/api/v2/tables/table-1/views?workspaceId=${WORKSPACE_ID}` + ), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/[tableId]/views/route.ts b/apps/sim/app/api/v2/tables/[tableId]/views/route.ts index e4eb50539e9..0daa6815561 100644 --- a/apps/sim/app/api/v2/tables/[tableId]/views/route.ts +++ b/apps/sim/app/api/v2/tables/[tableId]/views/route.ts @@ -21,7 +21,7 @@ export const GET = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2TableErrorPolicies.concealTableAuthorization, mapInput: ({ params, query }) => ({ tableId: params.tableId, workspaceId: query.workspaceId }), - present: async ({ views }) => { + present: async ({ views, columns }) => { const emailByUserId = await getUserEmailsByIds( views.flatMap((view) => (view.createdBy ? [view.createdBy] : [])) ) @@ -29,7 +29,8 @@ export const GET = defineV2JsonRoute({ data: views.map((view) => toApiView( view, - view.createdBy ? requireResolvedUserEmail(emailByUserId, view.createdBy) : null + view.createdBy ? requireResolvedUserEmail(emailByUserId, view.createdBy) : null, + columns ) ), nextCursor: null, @@ -45,7 +46,11 @@ export const POST = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2TableErrorPolicies.concealTableAuthorization, mapInput: ({ params, body }) => ({ tableId: params.tableId, ...body }), - present: async ({ view }) => ({ - data: toApiView(view, view.createdBy ? await getRequiredUserEmail(view.createdBy) : null), + present: async ({ view, columns }) => ({ + data: toApiView( + view, + view.createdBy ? await getRequiredUserEmail(view.createdBy) : null, + columns + ), }), }) diff --git a/apps/sim/app/api/v2/tables/folders/route.test.ts b/apps/sim/app/api/v2/tables/folders/route.test.ts index 866438a058c..c0f2f244068 100644 --- a/apps/sim/app/api/v2/tables/folders/route.test.ts +++ b/apps/sim/app/api/v2/tables/folders/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), list: vi.fn(), create: vi.fn(), update: vi.fn(), remove: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/folders', () => ({ listTableFoldersUseCase: { operation: { id: 'tables.folders.list' }, execute: mocks.list }, createTableFolderUseCase: { operation: { id: 'tables.folders.create' }, execute: mocks.create }, @@ -46,16 +42,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const folder = { id: 'folder-1', workspaceId: WORKSPACE_ID, @@ -85,10 +75,10 @@ function request(method: 'GET' | 'POST' | 'PATCH' | 'DELETE', url: string, body? describe('/api/v2/tables/folders', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.list.mockResolvedValue({ folders: [folder], index }) mocks.create.mockResolvedValue({ folder, index, path: '/Reports' }) mocks.update.mockResolvedValue({ @@ -155,4 +145,13 @@ describe('/api/v2/tables/folders', () => { }, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET(request('GET', `/api/v2/tables/folders?workspaceId=${WORKSPACE_ID}`)) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/imports/[importId]/complete/route.test.ts b/apps/sim/app/api/v2/tables/imports/[importId]/complete/route.test.ts index 6964d818d6b..e9e12f184a1 100644 --- a/apps/sim/app/api/v2/tables/imports/[importId]/complete/route.test.ts +++ b/apps/sim/app/api/v2/tables/imports/[importId]/complete/route.test.ts @@ -2,29 +2,25 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), complete: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/tables/presenters', () => ({ presentV2TableImport: (tableImport: unknown) => ({ data: tableImport }), })) @@ -46,24 +42,18 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} describe('POST /api/v2/tables/imports/[importId]/complete', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) }) it('delegates idempotent completion to the authorized import use case', async () => { @@ -101,4 +91,19 @@ describe('POST /api/v2/tables/imports/[importId]/complete', () => { request, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await POST( + new NextRequest( + `http://localhost:3000/api/v2/tables/imports/import-1/complete?workspaceId=${WORKSPACE_ID}`, + { method: 'POST', headers: { 'upload-token': 'signed-upload-token' } } + ), + { params: Promise.resolve({ importId: 'import-1' }) } + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/imports/[importId]/route.ts b/apps/sim/app/api/v2/tables/imports/[importId]/route.ts index 7092782c602..5aa45c2a899 100644 --- a/apps/sim/app/api/v2/tables/imports/[importId]/route.ts +++ b/apps/sim/app/api/v2/tables/imports/[importId]/route.ts @@ -17,9 +17,10 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2TableErrorPolicies.concealImportAuthorization, - mapInput: ({ params, query }) => ({ + mapInput: ({ params, query, headers }) => ({ importId: params.importId, workspaceId: query.workspaceId, + uploadToken: headers['upload-token'], }), useCase: readTableImportUseCase, present: ({ import: tableImport }) => presentV2TableImport(tableImport), diff --git a/apps/sim/app/api/v2/tables/imports/route.test.ts b/apps/sim/app/api/v2/tables/imports/route.test.ts index fd1f7672193..8c25906f209 100644 --- a/apps/sim/app/api/v2/tables/imports/route.test.ts +++ b/apps/sim/app/api/v2/tables/imports/route.test.ts @@ -2,29 +2,25 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), create: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/app/api/v2/tables/presenters', () => ({ presentV2CreateTableImport: (tableImport: unknown) => ({ data: tableImport }), })) @@ -43,25 +39,19 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const timestamp = '2026-01-01T00:00:00.000Z' describe('POST /api/v2/tables/imports', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) }) it.each([ @@ -98,7 +88,7 @@ describe('POST /api/v2/tables/imports', () => { session: { id: 'import-1', workspaceId: WORKSPACE_ID, - status: 'queued', + status: 'processing', source: { type: 'workspace_file', fileId: 'file-1' }, target: { type: 'new', name: 'imported_data' }, tableId: 'table-1', @@ -145,8 +135,27 @@ describe('POST /api/v2/tables/imports', () => { ) expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalled() - expect(mocks.operationRate).toHaveBeenCalled() + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalled() expect(mocks.create).not.toHaveBeenCalled() }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await POST( + new NextRequest('http://localhost:3000/api/v2/tables/imports', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ + workspaceId: WORKSPACE_ID, + source: { type: 'workspace_file', fileId: 'file-1' }, + target: { type: 'new', name: 'imported_data' }, + }), + }) + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/tables/presenters.test.ts b/apps/sim/app/api/v2/tables/presenters.test.ts index da5f27bca2c..7b08ee059ac 100644 --- a/apps/sim/app/api/v2/tables/presenters.test.ts +++ b/apps/sim/app/api/v2/tables/presenters.test.ts @@ -3,10 +3,12 @@ */ import { describe, expect, it } from 'vitest' +import type { TableSchema, WorkflowGroup } from '@/lib/table/types' import { presentV2CreateTableImport, presentV2TableExport, presentV2TableImport, + presentV2WorkflowGroup, } from '@/app/api/v2/tables/presenters' const createdAt = new Date('2026-08-01T00:00:00.000Z') @@ -80,3 +82,59 @@ describe('v2 table presenters', () => { }) }) }) + +/** + * A group is created with column **names** and was read back with stored column + * **ids** under the same `columnName` field, on a surface every other row/data + * endpoint keys by name. The value could not be round-tripped into another + * create, and named nothing the caller could see elsewhere. + */ +describe('presentV2WorkflowGroup', () => { + const schema: TableSchema = { + columns: [ + { id: 'col_score', name: 'score', type: 'number' }, + { id: 'col_input', name: 'website', type: 'string' }, + ], + } + + const stored = { + id: 'group-1', + workflowId: 'workflow-1', + outputs: [{ blockId: 'block-1', path: 'result', columnName: 'col_score' }], + dependencies: { columns: ['col_input'] }, + inputMappings: [{ inputName: 'url', columnName: 'col_input' }], + } as WorkflowGroup + + it('presents every column reference as the column name', () => { + const presented = presentV2WorkflowGroup(stored, schema) + + expect(presented.outputs[0].columnName).toBe('score') + expect(presented.dependencies?.columns).toEqual(['website']) + expect(presented.inputMappings?.[0].columnName).toBe('website') + }) + + it('leaves the stored group untouched', () => { + presentV2WorkflowGroup(stored, schema) + expect(stored.outputs[0].columnName).toBe('col_score') + }) + + it('passes a reference naming no current column through unchanged', () => { + const orphaned = { + ...stored, + outputs: [{ blockId: 'block-1', path: 'result', columnName: 'col_deleted' }], + } as WorkflowGroup + + expect(presentV2WorkflowGroup(orphaned, schema).outputs[0].columnName).toBe('col_deleted') + }) + + it('leaves a legacy name-keyed group alone', () => { + const legacy = { + ...stored, + outputs: [{ blockId: 'block-1', path: 'result', columnName: 'score' }], + dependencies: undefined, + inputMappings: undefined, + } as unknown as WorkflowGroup + + expect(presentV2WorkflowGroup(legacy, schema).outputs[0].columnName).toBe('score') + }) +}) diff --git a/apps/sim/app/api/v2/tables/presenters.ts b/apps/sim/app/api/v2/tables/presenters.ts index d194cb5c3fa..10a624e0980 100644 --- a/apps/sim/app/api/v2/tables/presenters.ts +++ b/apps/sim/app/api/v2/tables/presenters.ts @@ -1,3 +1,4 @@ +import { buildNameById, remapGroupColumnRefs } from '@/lib/table/column-keys' import { type TableExportRecord, toV2TableExport } from '@/lib/table/orchestration/export-resource' import { type CreateTableImportResult, @@ -5,6 +6,7 @@ import { toV2CreateTableImport, toV2TableImport, } from '@/lib/table/orchestration/import-resource' +import type { TableSchema, WorkflowGroup } from '@/lib/table/types' export function presentV2CreateTableImport(result: CreateTableImportResult) { return { data: toV2CreateTableImport(result) } @@ -17,3 +19,20 @@ export function presentV2TableImport(record: TableImportResource) { export function presentV2TableExport(record: TableExportRecord, queued = false) { return { data: toV2TableExport(record, queued) } } + +/** + * A workflow group with its column references presented as column **names**. + * + * Groups store `outputs[].columnName`, `dependencies.columns[]`, and + * `inputMappings[].columnName` as stable column **ids** so a rename cannot + * orphan them — but the field is named for, documented as, and accepted on + * create as a name, and every other v2 row surface is keyed by name. Reading + * back a `col_…` id under `columnName` meant a group could not be round-tripped + * into a create, and the value did not correspond to anything else the caller + * could see. `remapGroupColumnRefs` is the same rewrite the write path uses, + * driven by the inverse map; a ref naming no current column is left as-is, so a + * legacy name-keyed group and a ref to a since-deleted column both survive. + */ +export function presentV2WorkflowGroup(group: WorkflowGroup, schema: TableSchema): WorkflowGroup { + return remapGroupColumnRefs(group, buildNameById(schema)) +} diff --git a/apps/sim/app/api/v2/tables/route.test.ts b/apps/sim/app/api/v2/tables/route.test.ts index 831a3c69d9c..89c64f6b4bd 100644 --- a/apps/sim/app/api/v2/tables/route.test.ts +++ b/apps/sim/app/api/v2/tables/route.test.ts @@ -2,32 +2,28 @@ * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - preauthRate: vi.fn(), - operationRate: vi.fn(), - gate: vi.fn(), list: vi.fn(), create: vi.fn(), getUserEmailsByIds: vi.fn(), getMaxRowsPerTable: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - RateLimiter: class { - checkRateLimitDirect = mocks.preauthRate - checkRateLimitDirectOrThrow = mocks.operationRate - }, - getRateLimit: () => ({ maxTokens: 100, refillRate: 100, refillIntervalMs: 60_000 }), -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/table/application/tables', () => ({ listTablesUseCase: { operation: { id: 'tables.list' }, execute: mocks.list }, createTableUseCase: { operation: { id: 'tables.create' }, execute: mocks.create }, @@ -40,6 +36,8 @@ vi.mock('@/lib/table/billing', () => ({ getMaxRowsPerTable: mocks.getMaxRowsPerTable, })) +import { REFILTERED_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' +import { ForbiddenOperationError } from '@/lib/core/application/forbidden' import { GET, POST } from '@/app/api/v2/tables/route' const WORKSPACE_ID = 'workspace-1' @@ -51,16 +49,10 @@ const principal = { const auth = { principal, rolloutUserId: 'owner-1', - rateLimitSubjectIds: [`workspace:${WORKSPACE_ID}`], + rateLimitSubjectIds: ['api-key:key-1', `workspace:${WORKSPACE_ID}`], rateLimitSubscription: null, keyType: 'workspace' as const, } -const rate = { - allowed: true, - remaining: 99, - resetAt: new Date('2026-01-01T01:00:00.000Z'), - retryAfterMs: 0, -} const table = { id: 'table-1', workspaceId: WORKSPACE_ID, @@ -89,10 +81,10 @@ const table = { describe('/api/v2/tables', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.preauthRate.mockResolvedValue(rate) - mocks.operationRate.mockResolvedValue(rate) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.getUserEmailsByIds.mockResolvedValue(new Map([['owner-1', 'owner@example.com']])) mocks.getMaxRowsPerTable.mockResolvedValue(5000) mocks.list.mockResolvedValue({ @@ -104,6 +96,72 @@ describe('/api/v2/tables', () => { mocks.create.mockResolvedValue({ table, folderPath: '/' }) }) + /** + * The cursor a page mints is bound to the filters that produced it, so + * resuming it under a different `search` or `folderPath` is a 400 rather than + * a page silently sequenced against rows the new filter excludes. Pins the + * binding end-to-end — both the mint in `present` and the read in `mapInput` — + * because the contract-level sweep only checks a hand-maintained map of param + * names and stays green when a route drops the stamp entirely. + */ + it('refuses a cursor minted under a different filter', async () => { + mocks.list.mockResolvedValue({ + tables: [{ table, folderPath: '/' }], + nextKeys: ['Contacts', 'table-1'], + sortBy: 'name', + sortOrder: 'asc', + }) + + const minted = await GET( + new NextRequest( + `http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25&search=alpha` + ) + ) + const { nextCursor } = await minted.json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.list.mockClear() + const replayed = await GET( + new NextRequest( + `http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25&search=beta&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.list).not.toHaveBeenCalled() + }) + + it('resumes a cursor replayed under the filters it was minted with', async () => { + mocks.list.mockResolvedValue({ + tables: [{ table, folderPath: '/' }], + nextKeys: ['Contacts', 'table-1'], + sortBy: 'name', + sortOrder: 'asc', + }) + + const minted = await GET( + new NextRequest( + `http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25&search=alpha` + ) + ) + const { nextCursor } = await minted.json() + + mocks.list.mockClear() + const resumed = await GET( + new NextRequest( + `http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25&search=alpha&cursor=${encodeURIComponent(nextCursor)}` + ) + ) + + expect(resumed.status).toBe(200) + expect(mocks.list).toHaveBeenCalledWith({ + principal, + input: expect.objectContaining({ after: ['Contacts', 'table-1'] }), + request: expect.anything(), + }) + }) + it('lists through the semantic use case and preserves the cursor envelope', async () => { const request = new NextRequest( `http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25` @@ -134,13 +192,24 @@ describe('/api/v2/tables', () => { const response = await GET(new NextRequest('http://localhost:3000/api/v2/tables')) expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalled() - expect(mocks.operationRate).toHaveBeenCalled() + expect(v2RouteMocks.authenticate).toHaveBeenCalled() + expect(v2RouteMocks.operationRate).toHaveBeenCalled() expect(mocks.list).not.toHaveBeenCalled() }) + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25`) + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + it('maps operation rate-limit infrastructure failures to service unavailable', async () => { - mocks.operationRate.mockRejectedValueOnce(new Error('rate store unavailable')) + v2RouteMocks.operationRate.mockRejectedValueOnce(new Error('rate store unavailable')) const response = await GET( new NextRequest(`http://localhost:3000/api/v2/tables?workspaceId=${WORKSPACE_ID}&limit=25`) @@ -181,7 +250,7 @@ describe('/api/v2/tables', () => { }) }) - it('rejects required in a table column before calling the use case', async () => { + it('forwards required on a table column to the use case', async () => { const request = new NextRequest('http://localhost:3000/api/v2/tables', { method: 'POST', headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, @@ -193,6 +262,59 @@ describe('/api/v2/tables', () => { }) const response = await POST(request) + expect(response.status).toBe(201) + expect(mocks.create).toHaveBeenCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ + schema: { columns: [{ name: 'Name', type: 'string', required: true }] }, + }), + }) + ) + }) + + /** + * A quota ceiling and a permission refusal share the `403` status but demand + * opposite caller behaviour — delete something and retry, versus stop and + * escalate — so the ceiling names itself rather than leaving a client to + * string-match the message. + */ + it('names a workspace table-quota refusal in error.details.code', async () => { + mocks.create.mockRejectedValueOnce( + new ForbiddenOperationError( + 'WORKSPACE_RESOURCE_LIMIT_REACHED', + 'Workspace has reached maximum table limit (100)' + ) + ) + + const request = new NextRequest('http://localhost:3000/api/v2/tables', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify({ + workspaceId: WORKSPACE_ID, + name: 'Contacts', + schema: { columns: [{ name: 'Name', type: 'string', required: true }] }, + }), + }) + const response = await POST(request) + + expect(response.status).toBe(403) + expect(await response.json()).toMatchObject({ + error: { code: 'FORBIDDEN', details: { code: 'WORKSPACE_RESOURCE_LIMIT_REACHED' } }, + }) + }) + + it('rejects an unrecognized key in a table column before calling the use case', async () => { + const request = new NextRequest('http://localhost:3000/api/v2/tables', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify({ + workspaceId: WORKSPACE_ID, + name: 'Contacts', + schema: { columns: [{ name: 'Name', type: 'string', requried: true }] }, + }), + }) + const response = await POST(request) + expect(response.status).toBe(400) expect(mocks.create).not.toHaveBeenCalled() }) diff --git a/apps/sim/app/api/v2/tables/route.ts b/apps/sim/app/api/v2/tables/route.ts index b9515fdb91a..a91b9c4ef61 100644 --- a/apps/sim/app/api/v2/tables/route.ts +++ b/apps/sim/app/api/v2/tables/route.ts @@ -1,16 +1,24 @@ import { v2CreateTableContract, v2ListTablesContract } from '@/lib/api/contracts/v2/tables' -import { INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' -import { OrchestrationError } from '@/lib/core/orchestration/types' import { v2TableErrorPolicies } from '@/lib/table/api' import { tableOperations } from '@/lib/table/application/operations' import { createTableUseCase, listTablesUseCase } from '@/lib/table/application/tables' -import { cursorSortKey, decodeSortedCursor, encodeSortedCursor } from '@/app/api/v2/lib/response' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' import { toApiTable, toApiTables } from '@/app/api/v2/tables/utils' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Every param that changes which tables, in which order, this list returns. */ +function tableCursorFilters(query: { workspaceId: string; folderPath?: string; search?: string }) { + return cursorScopeKey(cursorRoute(v2ListTablesContract), { + workspaceId: query.workspaceId, + folderPath: query.folderPath, + search: query.search, + }) +} + export const GET = defineV2JsonRoute({ contract: v2ListTablesContract, operation: tableOperations.list, @@ -18,25 +26,23 @@ export const GET = defineV2JsonRoute({ auth: v2ApiKeyAuth, rateLimit: v2RateLimits.publicApi, errorPolicy: v2TableErrorPolicies.default, - mapInput: ({ query }) => { - const sort = cursorSortKey(query.sortBy, query.sortOrder) - const decoded = decodeSortedCursor(query.cursor, sort) - if (decoded.status === 'invalid') { - throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) - } - return { - workspaceId: query.workspaceId, - folderPath: query.folderPath, - search: query.search, - sortBy: query.sortBy, - sortOrder: query.sortOrder, - limit: query.limit, - after: decoded.status === 'ok' ? decoded.keys : undefined, - } - }, - present: async ({ tables, nextKeys, sortBy, sortOrder }) => ({ + mapInput: ({ query }) => ({ + workspaceId: query.workspaceId, + folderPath: query.folderPath, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + limit: query.limit, + after: readSortedCursor(query.cursor, query.sortBy, query.sortOrder, tableCursorFilters(query)), + }), + present: async ({ tables, nextKeys }, { query }) => ({ data: await toApiTables(tables), - nextCursor: nextKeys ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), nextKeys) : null, + nextCursor: writeSortedCursor( + nextKeys, + query.sortBy, + query.sortOrder, + tableCursorFilters(query) + ), }), }) diff --git a/apps/sim/app/api/v2/tables/utils.ts b/apps/sim/app/api/v2/tables/utils.ts index 04d02c3264e..8a480854ca7 100644 --- a/apps/sim/app/api/v2/tables/utils.ts +++ b/apps/sim/app/api/v2/tables/utils.ts @@ -1,22 +1,11 @@ -import type { NextResponse } from 'next/server' import type { V2ApiTable } from '@/lib/api/contracts/v2/tables' -import type { OrchestrationErrorCode } from '@/lib/core/orchestration/types' -import type { MultipartError } from '@/lib/core/utils/multipart' -import type { RowData, TableDefinition, TablePredicate, TableSchema } from '@/lib/table' +import type { RowData, TableDefinition, TableSchema } from '@/lib/table' import { getMaxRowsPerTable } from '@/lib/table/billing' -import { getColumnId } from '@/lib/table/column-keys' -import { TableLockedError } from '@/lib/table/mutation-locks' -import { predicateToFilter } from '@/lib/table/query-builder/converters' -import { - validatePredicateShape, - validateStoragePredicate, -} from '@/lib/table/query-builder/validate' -import { predicateToStorage } from '@/lib/table/select-values' -import type { Filter, TableLockKind } from '@/lib/table/types' +import { buildColumnNameById, remapViewConfigColumnRefs } from '@/lib/table/column-keys' +import type { ColumnDefinition } from '@/lib/table/types' import type { TableView } from '@/lib/table/views/service' +import { normalizeColumn } from '@/lib/table/wire' import { getUserEmailsByIds, requireResolvedUserEmail } from '@/lib/users/queries' -import { CSV_IMPORT_PROXY_BODY_CAP_BYTES, normalizeColumn } from '@/app/api/table/utils' -import { v2Error, v2ErrorForOrchestration } from '@/app/api/v2/lib/response' /** * Shared serialization + error helpers for the v2 tables surface. Every v2 @@ -49,21 +38,6 @@ function requireMaxRows( return maxRows } -/** - * Resolves a public v2 bulk-op predicate to the storage-id-keyed legacy `Filter` - * the row runners consume. The public wire is column-NAME-keyed: shape-check - * first (keying-agnostic), translate names → storage ids (including select - * operand names → option ids), then validate the RESULT against storage keys — - * on a destructive path an unresolved field must 400, not silently match - * nothing. - */ -export function v2BulkPredicateToFilter(predicate: TablePredicate, schema: TableSchema): Filter { - validatePredicateShape(predicate) - const translated = predicateToStorage(predicate, schema) - validateStoragePredicate(translated, schema.columns) - return predicateToFilter(translated) -} - /** * Normalized public table shape — the same subset of fields the v1 surface * exposes, with timestamps serialized to ISO strings. Shared by every v2 table @@ -143,15 +117,27 @@ export async function toApiTables( } /** - * Normalized public view shape. Identical to the stored view except that the - * timestamps are ISO strings, matching every other v2 payload. + * Normalized public view shape: ISO timestamps, and a `config` whose column + * references are presented as column **names**. + * + * A view stores every column reference as a stable id so a rename cannot orphan + * it — but the v2 surface is name-keyed everywhere else (row `data`, query + * predicates, sort fields, and workflow groups via `presentV2WorkflowGroup`), + * and a caller who never sees a `col_…` id cannot round-trip a config it reads + * back into a create. The write path translates in the other direction, so the + * pair is symmetric. A ref naming no current column (a since-deleted column in + * a saved filter) is left as-is. */ -export function toApiView(view: TableView, createdByEmail: string | null) { +export function toApiView( + view: TableView, + createdByEmail: string | null, + columns: ColumnDefinition[] +) { return { id: view.id, tableId: view.tableId, name: view.name, - config: view.config, + config: remapViewConfigColumnRefs(view.config, buildColumnNameById(columns)), isDefault: view.isDefault, createdByEmail, createdAt: toIso(view.createdAt), @@ -165,7 +151,7 @@ export function toApiView(view: TableView, createdByEmail: string | null) { * row `data`. Falls back to the id for a column that no longer exists. */ export function columnNameById(schema: TableSchema): (columnId: string) => string { - const nameById = new Map(schema.columns.map((column) => [getColumnId(column), column.name])) + const nameById = buildColumnNameById(schema.columns) return (columnId) => nameById.get(columnId) ?? columnId } @@ -194,105 +180,3 @@ export function toApiRow(row: ApiRowInput, toNamedRow: (data: RowData) => RowDat updatedAt: toIso(row.updatedAt), } } - -/** - * Maps a {@link MultipartError} from the streaming CSV reader to the v2 - * envelope. Mirrors v1's {@link multipartErrorResponse} — same classification, - * different envelope. - */ -export function v2MultipartError(error: MultipartError): NextResponse { - if (error.code === 'FILE_TOO_LARGE') { - return v2Error('PAYLOAD_TOO_LARGE', 'CSV import file exceeds maximum size') - } - return error.code === 'NO_FILE' - ? v2Error('BAD_REQUEST', 'CSV file is required') - : v2Error('BAD_REQUEST', `Invalid CSV upload: ${error.message}`) -} - -/** - * 413 when a synchronous CSV upload would exceed the proxy's body cap; `null` - * otherwise. Next buffers the request body for the proxy and silently - * TRUNCATES it past the cap, so an unchecked oversize upload imports a partial - * file and reports success — the failure this exists to prevent. - */ -export function v2CsvBodyCapError(request: { headers: Headers }): NextResponse | null { - const contentLength = Number(request.headers.get('content-length') ?? 0) - if (contentLength <= CSV_IMPORT_PROXY_BODY_CAP_BYTES) return null - return v2Error( - 'PAYLOAD_TOO_LARGE', - 'File too large to import through the server. Upload it to workspace storage and use the async import instead.' - ) -} - -/** - * Maps a delete/write rejected by a table lock to the v2 `LOCKED` envelope, - * mirroring v1's {@link tableLockErrorResponse}. Returns `null` for anything - * else so the caller falls through to its own classification. - * - * `details.lock` names the flag that rejected the write. A table carries four - * independent locks, so "locked" on its own does not tell a caller which one to - * clear — every 423 on the surface reports it. - */ -export function v2TableLockError( - error: unknown, - /** Merged into `details` — e.g. which operations of a composite write landed. */ - extraDetails?: Record -): NextResponse | null { - if (error instanceof TableLockedError) { - return v2Error('LOCKED', error.message, { details: { lock: error.lock, ...extraDetails } }) - } - return null -} - -/** The failure half of any `lib/table/orchestration` result. */ -export interface OrchestrationOutcome { - errorCode?: OrchestrationErrorCode - error?: string - lock?: TableLockKind -} - -/** - * Renders a `lib/table/orchestration` failure in the v2 envelope, naming the - * lock when one caused it. - * - * A lock rejection reaches a route two different ways — thrown and caught at - * the boundary ({@link v2TableLockError}), or returned as a classified - * `errorCode: 'locked'` outcome — and both must produce the same body. Plain - * {@link v2ErrorForOrchestration} cannot, because the `lock` kind lives on the - * outcome rather than the code, so every table route that renders an - * orchestration result goes through this instead. - */ -export function v2TableOrchestrationError( - outcome: OrchestrationOutcome, - fallback: string, - /** Merged into `details` — e.g. which operations of a composite write landed. */ - extraDetails?: Record -): NextResponse { - // `lock` is omitted rather than sent as null when the kind is unknown — a - // caller branching on `details.lock` should see absence, not a phantom value. - const details = { - ...(outcome.errorCode === 'locked' && outcome.lock ? { lock: outcome.lock } : {}), - ...extraDetails, - } - return v2ErrorForOrchestration( - outcome.errorCode, - outcome.error ?? fallback, - Object.keys(details).length > 0 ? details : undefined - ) -} - -/** - * Adapts a failed-row validation from the shared `validateRowData` / - * `validateBatchRows` helpers — which bake a v1-shaped `{ error, details }` 400 - * response — into the canonical v2 error envelope while preserving the - * structured `details` (per-field / per-row). The validators expose the failure - * only as a rendered response, so the body is read back rather than - * re-implementing the size/schema/unique checks. - */ -export async function v2RowValidationError(response: NextResponse): Promise { - const body = (await response - .clone() - .json() - .catch(() => ({}))) as { error?: string; details?: unknown } - return v2Error('BAD_REQUEST', body.error ?? 'Invalid row data', { details: body.details }) -} diff --git a/apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.test.ts b/apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.test.ts index f5cc527da7d..4a9e4ce0b9c 100644 --- a/apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.test.ts +++ b/apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.test.ts @@ -29,6 +29,7 @@ vi.mock('@/lib/uploads/upload-session/service', () => ({ verifyUploadSessionToken: mockVerifyUploadSessionToken, })) +import { OrchestrationError } from '@/lib/core/orchestration/types' import { PUT } from '@/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route' const SESSION = { @@ -37,6 +38,8 @@ const SESSION = { method: 'multipart', status: 'uploading', expiresAt: new Date('2999-01-01T00:00:00.000Z'), + partSize: 3, + partCount: 2, } as const describe('PUT /api/v2/uploads/[uploadId]/parts/[partNumber]', () => { @@ -70,7 +73,7 @@ describe('PUT /api/v2/uploads/[uploadId]/parts/[partNumber]', () => { expect(response.status).toBe(400) await expect(response.json()).resolves.toMatchObject({ - error: 'Part 1 has 2 bytes; expected 3', + error: { code: 'BAD_REQUEST', message: 'Part 1 has 2 bytes; expected 3' }, }) }) @@ -84,6 +87,41 @@ describe('PUT /api/v2/uploads/[uploadId]/parts/[partNumber]', () => { expect(mockWriteLocalMultipartPart).not.toHaveBeenCalled() }) + /** + * The part number is a path segment of a session-scoped signed URL, so any + * holder of a legitimate part URL can address a part the session does not + * have. `expectedUploadPartSize` classifies that as a validation failure; + * `service.test.ts` pins that classification on the real implementation, + * which this suite mocks away. + */ + it('maps an out-of-range part number to the documented 400', async () => { + mockExpectedUploadPartSize.mockImplementation(() => { + throw new OrchestrationError('validation', 'partNumber must be between 1 and 2') + }) + + const response = await request({ partNumber: '99' }) + + expect(response.status).toBe(400) + await expect(response.json()).resolves.toEqual({ + error: { code: 'BAD_REQUEST', message: 'partNumber must be between 1 and 2' }, + }) + expect(mockWriteLocalMultipartPart).not.toHaveBeenCalled() + }) + + it('still renders an unclassified part-size failure as a generic 500', async () => { + mockExpectedUploadPartSize.mockImplementation(() => { + throw new Error('unexpected') + }) + + const response = await request() + + expect(response.status).toBe(500) + await expect(response.json()).resolves.toEqual({ + error: { code: 'INTERNAL_ERROR', message: 'Internal server error' }, + }) + expect(mockWriteLocalMultipartPart).not.toHaveBeenCalled() + }) + it('rejects expired upload sessions before writing the part', async () => { mockVerifyUploadSessionToken.mockReturnValue({ ...SESSION, @@ -93,23 +131,29 @@ describe('PUT /api/v2/uploads/[uploadId]/parts/[partNumber]', () => { const response = await request() expect(response.status).toBe(409) - await expect(response.json()).resolves.toEqual({ error: 'Upload session has expired' }) + await expect(response.json()).resolves.toEqual({ + error: { code: 'CONFLICT', message: 'Upload session has expired' }, + }) expect(mockExpectedUploadPartSize).not.toHaveBeenCalled() expect(mockWriteLocalMultipartPart).not.toHaveBeenCalled() }) }) -function request(options?: { contentLength?: string | null }) { +function request(options?: { contentLength?: string | null; partNumber?: string }) { const headers = new Headers({ 'Content-Type': 'application/octet-stream' }) if (options?.contentLength !== null) { headers.set('Content-Length', options?.contentLength ?? '3') } + const partNumber = options?.partNumber ?? '1' return PUT( - new NextRequest('http://localhost:3000/api/v2/uploads/upload-1/parts/1?token=signed-token', { - method: 'PUT', - headers, - body: new Uint8Array([1, 2, 3]), - }), - { params: Promise.resolve({ uploadId: 'upload-1', partNumber: '1' }) } + new NextRequest( + `http://localhost:3000/api/v2/uploads/upload-1/parts/${partNumber}?token=signed-token`, + { + method: 'PUT', + headers, + body: new Uint8Array([1, 2, 3]), + } + ), + { params: Promise.resolve({ uploadId: 'upload-1', partNumber }) } ) } diff --git a/apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.ts b/apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.ts index 934e64d839e..7a3aa46a8a3 100644 --- a/apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.ts +++ b/apps/sim/app/api/v2/uploads/[uploadId]/parts/[partNumber]/route.ts @@ -1,6 +1,7 @@ import { type NextRequest, NextResponse } from 'next/server' import { localUploadPartContract } from '@/lib/api/contracts/upload-sessions' import { parseRequest } from '@/lib/api/server' +import { V2_PARSE_DEFAULTS } from '@/lib/api/server/routes' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { LocalUploadBodyError, @@ -11,6 +12,12 @@ import { type UploadSessionRecord, verifyUploadSessionToken, } from '@/lib/uploads/upload-session/service' +import { + v2CaughtOrchestrationError, + v2Error, + v2HttpError, + v2UploadDataPlaneError, +} from '@/app/api/v2/lib/response' interface LocalPartRouteParams { params: Promise<{ uploadId: string; partNumber: string }> @@ -19,6 +26,15 @@ interface LocalPartRouteParams { /** * Local-storage data plane for signed multipart PUT URLs. Cloud deployments return provider URLs * instead, so this route is never in the cloud byte path. + * + * Raw `withRouteHandler` rather than a v2 builder, for the same reason as the + * whole-object PUT beside it: a signed token credential and a streamed body, + * with no `Principal` or semantic operation for a builder to act on. + * + * Absent from the public OpenAPI documents by design — see + * `UNDOCUMENTED_V2_ROUTES` in `scripts/check-openapi-specs.ts` — but it answers + * in the canonical `{ error: { code, message } }` envelope like the rest of the + * surface, for the reason given on the whole-object PUT beside it. */ export const PUT = withRouteHandler( async (request: NextRequest, context: LocalPartRouteParams): Promise => { @@ -28,45 +44,59 @@ export const PUT = withRouteHandler( try { session = await verifyUploadSessionToken(token) } catch { - return NextResponse.json({ error: 'Invalid or expired upload token' }, { status: 403 }) + return v2Error('FORBIDDEN', 'Invalid or expired upload token') } - const parsed = await parseRequest(localUploadPartContract, request, context) + const parsed = await parseRequest(localUploadPartContract, request, context, { + ...V2_PARSE_DEFAULTS, + }) if (!parsed.success) return parsed.response if (session.id !== uploadId || session.storageProvider !== 'local') { - return NextResponse.json({ error: 'Upload URL does not match this session' }, { status: 403 }) + return v2Error('FORBIDDEN', 'Upload URL does not match this session') } if (session.status !== 'uploading') { - return NextResponse.json({ error: `Upload session is ${session.status}` }, { status: 409 }) + return v2Error('CONFLICT', `Upload session is ${session.status}`) } if (session.expiresAt.getTime() <= Date.now()) { - return NextResponse.json({ error: 'Upload session has expired' }, { status: 409 }) + return v2Error('CONFLICT', 'Upload session has expired') } if (session.method !== 'multipart') { - return NextResponse.json({ error: 'PUT upload sessions do not have parts' }, { status: 409 }) + return v2Error('CONFLICT', 'PUT upload sessions do not have parts') } const { partNumber } = parsed.data.params - const expectedSize = expectedUploadPartSize(session, partNumber) + let expectedSize: number + try { + expectedSize = expectedUploadPartSize(session, partNumber) + } catch (error) { + // The part number is a path segment of a session-scoped signed URL, so a + // caller can address a part this session does not have. That refusal is a + // classified domain failure, and the data plane's generic 500 tail would + // otherwise render it as an internal error. + const classified = v2CaughtOrchestrationError(error) + if (classified) return classified + throw error + } const contentLength = request.headers.get('content-length') if (contentLength !== null && Number(contentLength) !== expectedSize) { - return NextResponse.json( - { error: `Part ${partNumber} must contain exactly ${expectedSize} bytes` }, - { status: 400 } - ) + return v2Error('BAD_REQUEST', `Part ${partNumber} must contain exactly ${expectedSize} bytes`) } if (!request.body) { - return NextResponse.json({ error: 'Upload part body is required' }, { status: 400 }) + return v2Error('BAD_REQUEST', 'Upload part body is required') } try { await writeLocalMultipartPart({ uploadId, partNumber, body: request.body, expectedSize }) } catch (error) { if (error instanceof LocalUploadBodyError) { - return NextResponse.json({ error: error.message }, { status: 400 }) + return v2Error('BAD_REQUEST', error.message) } throw error } return new NextResponse(null, { status: 204 }) + }, + { + typedErrorResponse: ({ error }) => v2HttpError(error), + unhandledErrorResponse: () => v2UploadDataPlaneError(), } ) diff --git a/apps/sim/app/api/v2/uploads/[uploadId]/route.test.ts b/apps/sim/app/api/v2/uploads/[uploadId]/route.test.ts index 71d92a6d22d..31cd0e3c254 100644 --- a/apps/sim/app/api/v2/uploads/[uploadId]/route.test.ts +++ b/apps/sim/app/api/v2/uploads/[uploadId]/route.test.ts @@ -99,17 +99,27 @@ describe('PUT /api/v2/uploads/[uploadId]', () => { expect(response.status).toBe(400) await expect(response.json()).resolves.toMatchObject({ - error: 'Upload must contain exactly 3 bytes', + error: { code: 'BAD_REQUEST', message: 'Upload must contain exactly 3 bytes' }, }) expect(mockWriteLocalPut).not.toHaveBeenCalled() }) - it('rejects a URL whose token names a non-local or multipart session', async () => { + /** + * This route is deliberately absent from the OpenAPI documents, which is a + * statement about addressability rather than about behaviour. It used to + * answer with a bare `{ error: string }`, which made the one step of an + * upload that actually moves the bytes the one step a caller could not parse + * with its v2 error handling. + */ + it('rejects a URL whose token names a non-local or multipart session, in the v2 envelope', async () => { mockGetOwnedUploadSession.mockReturnValue({ ...SESSION, method: 'multipart' }) const response = await request() expect(response.status).toBe(403) + await expect(response.json()).resolves.toEqual({ + error: { code: 'FORBIDDEN', message: 'Upload URL does not match this session' }, + }) expect(mockWriteLocalPut).not.toHaveBeenCalled() }) @@ -119,7 +129,9 @@ describe('PUT /api/v2/uploads/[uploadId]', () => { const response = await request({ contentLength: null }) expect(response.status).toBe(400) - await expect(response.json()).resolves.toMatchObject({ error: 'Upload exceeds 3 bytes' }) + await expect(response.json()).resolves.toMatchObject({ + error: { code: 'BAD_REQUEST', message: 'Upload exceeds 3 bytes' }, + }) }) }) diff --git a/apps/sim/app/api/v2/uploads/[uploadId]/route.ts b/apps/sim/app/api/v2/uploads/[uploadId]/route.ts index 52ec40a4cdb..58ba867229b 100644 --- a/apps/sim/app/api/v2/uploads/[uploadId]/route.ts +++ b/apps/sim/app/api/v2/uploads/[uploadId]/route.ts @@ -1,21 +1,39 @@ import { type NextRequest, NextResponse } from 'next/server' import { localPutUploadContract } from '@/lib/api/contracts/upload-sessions' import { parseRequest } from '@/lib/api/server' +import { V2_PARSE_DEFAULTS } from '@/lib/api/server/routes' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' import { LocalUploadBodyError, writeLocalPutObject } from '@/lib/uploads/upload-session/provider' import { getOwnedUploadSession, uploadSessionObjectMetadata, } from '@/lib/uploads/upload-session/service' +import { v2Error, v2HttpError, v2UploadDataPlaneError } from '@/app/api/v2/lib/response' interface LocalPutRouteParams { params: Promise<{ uploadId: string }> } -/** Local-storage data plane for a signed whole-object PUT upload session. */ +/** + * Local-storage data plane for a signed whole-object PUT upload session. + * + * Raw `withRouteHandler` rather than a v2 builder: the signed `upload-token` + * header is the credential, so there is no API key, `Principal`, or semantic + * operation for a builder to authenticate and authorize against, and the body + * is streamed straight to storage rather than parsed. + * + * Absent from the public OpenAPI documents by design — see + * `UNDOCUMENTED_V2_ROUTES` in `scripts/check-openapi-specs.ts` — but the error + * envelope is not part of that exemption. This is the one step that moves the + * bytes, and a caller that cannot parse its failures the way it parses every + * other v2 failure has to special-case the whole upload flow, so it renders the + * canonical `{ error: { code, message } }` like the rest of the surface. + */ export const PUT = withRouteHandler( async (request: NextRequest, context: LocalPutRouteParams): Promise => { - const parsed = await parseRequest(localPutUploadContract, request, context) + const parsed = await parseRequest(localPutUploadContract, request, context, { + ...V2_PARSE_DEFAULTS, + }) if (!parsed.success) return parsed.response let session @@ -25,35 +43,29 @@ export const PUT = withRouteHandler( uploadToken: parsed.data.headers['upload-token'], }) } catch { - return NextResponse.json({ error: 'Invalid or expired upload token' }, { status: 403 }) + return v2Error('FORBIDDEN', 'Invalid or expired upload token') } if (session.storageProvider !== 'local' || session.method !== 'put') { - return NextResponse.json({ error: 'Upload URL does not match this session' }, { status: 403 }) + return v2Error('FORBIDDEN', 'Upload URL does not match this session') } if (session.status !== 'uploading') { - return NextResponse.json({ error: `Upload session is ${session.status}` }, { status: 409 }) + return v2Error('CONFLICT', `Upload session is ${session.status}`) } if (session.expiresAt.getTime() <= Date.now()) { - return NextResponse.json({ error: 'Upload session has expired' }, { status: 409 }) + return v2Error('CONFLICT', 'Upload session has expired') } const contentType = request.headers.get('content-type') if (contentType !== session.contentType) { - return NextResponse.json( - { error: `Content-Type must be ${session.contentType}` }, - { status: 400 } - ) + return v2Error('BAD_REQUEST', `Content-Type must be ${session.contentType}`) } const contentLength = request.headers.get('content-length') if (contentLength !== null && Number(contentLength) !== session.fileSize) { - return NextResponse.json( - { error: `Upload must contain exactly ${session.fileSize} bytes` }, - { status: 400 } - ) + return v2Error('BAD_REQUEST', `Upload must contain exactly ${session.fileSize} bytes`) } if (!request.body) { - return NextResponse.json({ error: 'Upload body is required' }, { status: 400 }) + return v2Error('BAD_REQUEST', 'Upload body is required') } try { @@ -67,10 +79,14 @@ export const PUT = withRouteHandler( }) } catch (error) { if (error instanceof LocalUploadBodyError) { - return NextResponse.json({ error: error.message }, { status: 400 }) + return v2Error('BAD_REQUEST', error.message) } throw error } return new NextResponse(null, { status: 204 }) + }, + { + typedErrorResponse: ({ error }) => v2HttpError(error), + unhandledErrorResponse: () => v2UploadDataPlaneError(), } ) diff --git a/apps/sim/app/api/v2/workflows/[id]/deploy/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/deploy/route.test.ts index 8e77d94b1dd..21e6ad41708 100644 --- a/apps/sim/app/api/v2/workflows/[id]/deploy/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/deploy/route.test.ts @@ -30,6 +30,12 @@ import { workflowOperations } from '@/lib/workflows/application/operations' import { DELETE, POST } from '@/app/api/v2/workflows/[id]/deploy/route' describe('/api/v2/workflows/[id]/deploy route definitions', () => { + /** + * Both the malformed-body 400 and the oversized-body 413 are v2 builder + * defaults, so neither belongs on the route. The envelope they produce is + * asserted once against the builder in + * `lib/api/server/routes/v2-error-envelope.test.ts`. + */ it('keeps an omitted deploy body valid and binds the authorized deployment use case', async () => { expect(v2DeployWorkflowContract.body?.parse(undefined)).toEqual({}) expect(POST).toMatchObject({ @@ -46,15 +52,7 @@ describe('/api/v2/workflows/[id]/deploy route definitions', () => { }) ) - const invalidJsonResponse = Reflect.get( - Reflect.get(POST, 'parseOptions'), - 'invalidJsonResponse' - )() - expect(invalidJsonResponse.status).toBe(400) - expect(await invalidJsonResponse.json()).toEqual({ - error: { code: 'BAD_REQUEST', message: 'Request body must be valid JSON' }, - }) - + expect(Reflect.get(Reflect.get(POST, 'parseOptions'), 'invalidJsonResponse')).toBeUndefined() expect( Reflect.get(Reflect.get(POST, 'parseOptions'), 'payloadTooLargeResponse') ).toBeUndefined() diff --git a/apps/sim/app/api/v2/workflows/[id]/deploy/route.ts b/apps/sim/app/api/v2/workflows/[id]/deploy/route.ts index a72e3838f7f..93f0e6a8cdb 100644 --- a/apps/sim/app/api/v2/workflows/[id]/deploy/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/deploy/route.ts @@ -8,7 +8,6 @@ import { captureServerEvent } from '@/lib/posthog/server' import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' import { deployWorkflow, undeployWorkflow } from '@/lib/workflows/application/deployments' import { workflowOperations } from '@/lib/workflows/application/operations' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const runtime = 'nodejs' @@ -22,7 +21,6 @@ export const POST = defineV2JsonRoute({ errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, parseOptions: { optionalJsonBody: true, - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), }, mapInput: ({ params, body }) => ({ workflowId: params.id, diff --git a/apps/sim/app/api/v2/workflows/[id]/deployment/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/deployment/route.test.ts new file mode 100644 index 00000000000..54f7bc7664d --- /dev/null +++ b/apps/sim/app/api/v2/workflows/[id]/deployment/route.test.ts @@ -0,0 +1,207 @@ +/** + * @vitest-environment node + */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' +import { NextRequest } from 'next/server' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +const mocks = vi.hoisted(() => ({ + resolvePermission: vi.fn(), + resolveWorkflowContext: vi.fn(), + getWorkflowDeploymentSummary: vi.fn(), + checkNeedsRedeployment: vi.fn(), +})) + +vi.mock('@sim/platform-authz/workspace', () => ({ + permissionSatisfies: (actual: string | null, required: string) => { + const rank = { read: 1, write: 2, admin: 3 } as const + return ( + actual !== null && rank[actual as keyof typeof rank] >= rank[required as keyof typeof rank] + ) + }, + resolveEffectiveWorkspacePermission: mocks.resolvePermission, +})) +vi.mock('@/lib/workflows/application/context', () => ({ + resolveActiveWorkflowApplicationContext: mocks.resolveWorkflowContext, +})) +vi.mock('@/lib/workflows/orchestration/deploy', () => ({ + getWorkflowDeploymentSummary: mocks.getWorkflowDeploymentSummary, + performActivateVersion: vi.fn(), + performFullDeploy: vi.fn(), + performFullUndeploy: vi.fn(), + performRevertToVersion: vi.fn(), +})) +vi.mock('@/lib/workflows/deployment-status', () => ({ + checkNeedsRedeployment: mocks.checkNeedsRedeployment, +})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) + +import { GET } from '@/app/api/v2/workflows/[id]/deployment/route' + +const auth = { + principal: { + kind: 'personal_api_key' as const, + userId: 'user-1', + keyId: 'personal-key-1', + }, + rolloutUserId: 'user-1', + rateLimitSubjectIds: ['api-key:personal-key-1', 'user:user-1'] as const, + rateLimitSubscription: null, + keyType: 'personal' as const, +} + +const activeDeployment = { + deploymentVersionId: 'depver-2', + version: 2, + deployedAt: '2026-08-01T00:00:00.000Z', +} + +const latestDeploymentAttempt = { + id: 'op-2', + deploymentVersionId: 'depver-2', + version: 2, + action: 'deploy' as const, + status: 'active' as const, + isCurrent: true, + readiness: { + webhooks: 'not_applicable' as const, + schedules: 'not_applicable' as const, + mcp: 'not_applicable' as const, + }, + requestedAt: '2026-08-01T00:00:00.000Z', + activatedAt: '2026-08-01T00:00:01.000Z', + error: null, +} + +/** + * `workflow.deployedAt` carries a stale timestamp from a deployment that was + * later undeployed — the presenter must never fall back to it. + */ +const workflowContext = { + workspaceId: 'workspace-1', + workspaceOrganizationId: null, + allowPersonalApiKeys: true, + billedAccountUserId: 'billing-owner-1', + workflowId: 'workflow-1', + workflow: { + id: 'workflow-1', + workspaceId: 'workspace-1', + deployedAt: new Date('2025-01-01T00:00:00.000Z'), + }, +} + +async function get() { + const request = new NextRequest('http://localhost/api/v2/workflows/workflow-1/deployment') + return GET(request, { params: Promise.resolve({ id: 'workflow-1' }) }) +} + +describe('GET /api/v2/workflows/[id]/deployment', () => { + beforeEach(() => { + vi.clearAllMocks() + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) + mocks.resolvePermission.mockResolvedValue('read') + mocks.resolveWorkflowContext.mockResolvedValue(workflowContext) + mocks.getWorkflowDeploymentSummary.mockResolvedValue({ + activeDeployment, + latestDeploymentAttempt, + warnings: undefined, + }) + mocks.checkNeedsRedeployment.mockResolvedValue(true) + }) + + it('publishes draft-versus-live drift and the latest attempt after canonical authorization', async () => { + const response = await get() + + expect(response.status).toBe(200) + expect(await response.json()).toEqual({ + data: { + id: 'workflow-1', + isDeployed: true, + needsRedeployment: true, + deployedAt: '2026-08-01T00:00:00.000Z', + warnings: [], + activeDeployment, + latestDeploymentAttempt, + }, + }) + expect(mocks.resolveWorkflowContext).toHaveBeenCalledBefore(mocks.getWorkflowDeploymentSummary) + }) + + it('carries the failed attempt error payload when nothing is live', async () => { + mocks.getWorkflowDeploymentSummary.mockResolvedValue({ + activeDeployment: null, + latestDeploymentAttempt: { + ...latestDeploymentAttempt, + status: 'failed' as const, + activatedAt: null, + error: { + code: 'webhook_conflict', + message: 'Webhook path already in use', + retryable: false, + }, + }, + warnings: ['Deployment attempt failed'], + }) + + const response = await get() + const body = await response.json() + + expect(response.status).toBe(200) + expect(body.data.isDeployed).toBe(false) + expect(body.data.needsRedeployment).toBe(false) + expect(body.data.deployedAt).toBeNull() + expect(body.data.warnings).toEqual(['Deployment attempt failed']) + expect(body.data.latestDeploymentAttempt.error).toEqual({ + code: 'webhook_conflict', + message: 'Webhook path already in use', + retryable: false, + }) + expect(mocks.checkNeedsRedeployment).not.toHaveBeenCalled() + }) + + it('never reports a deploy time from the stale workflow column once nothing is live', async () => { + mocks.getWorkflowDeploymentSummary.mockResolvedValue({ + activeDeployment: null, + latestDeploymentAttempt: null, + warnings: undefined, + }) + + const response = await get() + const body = await response.json() + + expect(response.status).toBe(200) + expect(body.data.deployedAt).toBeNull() + }) + + it('conceals a workflow the caller cannot reach as 404', async () => { + mocks.resolvePermission.mockResolvedValue(null) + + const response = await get() + + expect(response.status).toBe(404) + expect((await response.json()).error.code).toBe('NOT_FOUND') + expect(mocks.getWorkflowDeploymentSummary).not.toHaveBeenCalled() + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await get() + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) +}) diff --git a/apps/sim/app/api/v2/workflows/[id]/deployment/route.ts b/apps/sim/app/api/v2/workflows/[id]/deployment/route.ts new file mode 100644 index 00000000000..6999c325e3c --- /dev/null +++ b/apps/sim/app/api/v2/workflows/[id]/deployment/route.ts @@ -0,0 +1,47 @@ +import { v2GetWorkflowDeploymentContract } from '@/lib/api/contracts/v2/workflows' +import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' +import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' +import { readWorkflowDeploymentStatus } from '@/lib/workflows/application/deployments' +import { workflowOperations } from '@/lib/workflows/application/operations' + +export const dynamic = 'force-dynamic' +export const revalidate = 0 + +/** + * GET /api/v2/workflows/[id]/deployment — Read current deployment state. + * + * The deploy, undeploy, and rollback responses are the only other place this + * state is published, so a caller that lost one — or that polls from a + * different process — had no way to ask. `needsRedeployment` is exposed here + * only: it compares the draft against the live version, so it is meaningless on + * the response of the mutation that just made them equal. + * + * `deployedAt` comes from the active deployment version, which always carries + * one. The workflow's own `deployed_at` column is deliberately not used as a + * fallback: it retains the timestamp of a deployment that has since been + * undeployed, so reading it would report a deploy time alongside + * `isDeployed: false`. + * + * Deliberately head-safe despite the migrate-on-read write, for the reasons on + * `GET /api/v2/workflows/[id]`. + */ +export const GET = defineV2JsonRoute({ + contract: v2GetWorkflowDeploymentContract, + auth: v2ApiKeyAuth, + operation: workflowOperations.read, + rateLimit: v2RateLimits.publicApi, + errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, + mapInput: ({ params }) => ({ workflowId: params.id }), + useCase: readWorkflowDeploymentStatus, + present: (result) => ({ + data: { + id: result.workflow.id, + isDeployed: result.isDeployed, + needsRedeployment: result.needsRedeployment, + deployedAt: result.activeDeployment?.deployedAt ?? null, + warnings: result.warnings ?? [], + activeDeployment: result.activeDeployment ?? null, + latestDeploymentAttempt: result.latestDeploymentAttempt ?? null, + }, + }), +}) diff --git a/apps/sim/app/api/v2/workflows/[id]/execute/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/execute/route.test.ts index 9d5f5aeeb17..8d2abfda872 100644 --- a/apps/sim/app/api/v2/workflows/[id]/execute/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/execute/route.test.ts @@ -536,6 +536,39 @@ describe('POST /api/v2/workflows/[id]/execute', () => { expect((await res.json()).error.code).toBe('RATE_LIMITED') }) + it('tells a client how long to wait when a dependency is briefly unavailable', async () => { + mockPreprocessExecution.mockResolvedValue({ + success: false, + error: { + message: 'Workflow execution identity is temporarily unavailable', + statusCode: 503, + }, + }) + + const res = await callExecute({ input: {} }) + + expect(res.status).toBe(503) + expect(Number(res.headers.get('Retry-After'))).toBeGreaterThan(0) + }) + + it('never advises a retry when an enqueue may already have started a run', async () => { + mockPreprocessExecution.mockResolvedValue({ + success: false, + error: { + message: 'Async execution queue acceptance could not be confirmed', + statusCode: 503, + code: 'ASYNC_ENQUEUE_AMBIGUOUS', + }, + }) + + const res = await callExecute({ input: {} }) + + expect(res.status).toBe(503) + // Retrying without X-Run-Id would start, and bill, a second run of the same workflow. + expect(res.headers.get('Retry-After')).toBeNull() + expect((await res.json()).error.details.code).toBe('ASYNC_ENQUEUE_AMBIGUOUS') + }) + it('runs the anonymous public path sync but refuses async', async () => { dbChainMockFns.limit.mockReset() dbChainMockFns.limit.mockResolvedValueOnce([ diff --git a/apps/sim/app/api/v2/workflows/[id]/execute/route.ts b/apps/sim/app/api/v2/workflows/[id]/execute/route.ts index b9fe5eab1e9..1ac2c88dbec 100644 --- a/apps/sim/app/api/v2/workflows/[id]/execute/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/execute/route.ts @@ -12,6 +12,7 @@ import { import { parseRequest } from '@/lib/api/server' import { admitOptionalV2Request, + V2_PARSE_DEFAULTS, V2RouteInfrastructureError, v2ApiKeyAuth, v2RateLimits, @@ -19,6 +20,7 @@ import { import type { V2ApiKeyPrincipal } from '@/lib/api/server/routes/v2-api-key-auth' import { tryAdmit } from '@/lib/core/admission/gate' import { ADMISSION_ERROR_DESCRIPTOR } from '@/lib/core/admission/transient-failure' +import type { ForbiddenDetailCode } from '@/lib/core/application' import { generateRequestId } from '@/lib/core/utils/request' import { getBaseUrl } from '@/lib/core/utils/urls' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' @@ -43,7 +45,7 @@ import { hasAgentStreamPolicy, } from '@/lib/workflows/streaming/agent-stream-protocol' import { v2ApiGateError } from '@/app/api/v2/lib/gate' -import { type V2ErrorCode, v2Data, v2Error, v2ValidationError } from '@/app/api/v2/lib/response' +import { type V2ErrorCode, v2Data, v2Error } from '@/app/api/v2/lib/response' import { PublicApiNotAllowedError, validatePublicApiAllowed, @@ -82,6 +84,8 @@ function serviceFailureResponse(failure: ExecuteWorkflowServiceFailure) { return v2Error(code, isRunIdConflict ? 'Run ID has already been used' : failure.message, { status: failure.statusCode, headers, + /** An unconfirmed enqueue may already have started a run — reconcile on `runId`, never retry blind. */ + omitRetryAfter: failure.code === 'ASYNC_ENQUEUE_AMBIGUOUS', details: detailCode || failure.executionId ? { @@ -185,8 +189,8 @@ export const POST = withRouteHandler( try { const parsed = await parseRequest(v2ExecuteWorkflowContract, req, context, { + ...V2_PARSE_DEFAULTS, maxBodyBytes: 10 * 1024 * 1024, - validationErrorResponse: v2ValidationError, }) if (!parsed.success) return parsed.response const body = parsed.data.body @@ -270,7 +274,9 @@ export const POST = withRouteHandler( return v2Error('NOT_FOUND', 'Workflow not found') } if (workflowAuthorization.status === 403) { - return v2Error('FORBIDDEN', 'Insufficient workspace permissions') + return v2Error('FORBIDDEN', 'Insufficient workspace permissions', { + details: { code: 'INSUFFICIENT_WORKSPACE_ROLE' satisfies ForbiddenDetailCode }, + }) } throw new Error( `Unexpected workflow authorization status: ${workflowAuthorization.status}` @@ -279,6 +285,7 @@ export const POST = withRouteHandler( result = await executeWorkflowService({ workflowId, userId, + isPublicApiAccess, input: body.input ?? {}, triggerType: 'api', requestId, diff --git a/apps/sim/app/api/v2/workflows/[id]/export/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/export/route.test.ts index 96b5468d343..872932e6ba1 100644 --- a/apps/sim/app/api/v2/workflows/[id]/export/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/export/route.test.ts @@ -29,4 +29,26 @@ describe('/api/v2/workflows/[id]/export route definition', () => { errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, }) }) + + /** + * Next aliases a missing `HEAD` export onto `GET`, and RFC 9110 §9.2.1 defines + * `HEAD` as safe. This `GET` is not: the use case projects a + * `WORKFLOW_EXPORTED` audit event, so an uptime monitor or link checker + * probing the documented URL would file an export that never handed anyone + * the workflow. + */ + it('does not run the audited export for a HEAD probe', () => { + expect(GET).toMatchObject({ headSafe: false }) + }) + + /** + * Not running the export is only half of it. The `HEAD` must still resolve the + * workflow and check access, or the probe answers 200 for an id the caller's + * `GET` would conceal as a 404 — an existence oracle over every workspace's + * workflow ids. The builder refuses at definition time to pair + * `headSafe: false` with a use case that cannot answer that on its own. + */ + it('exposes an authorization phase the HEAD probe can run without exporting', () => { + expect(typeof exportWorkflow.authorize).toBe('function') + }) }) diff --git a/apps/sim/app/api/v2/workflows/[id]/export/route.ts b/apps/sim/app/api/v2/workflows/[id]/export/route.ts index d4011a4f64c..ad3af5d0518 100644 --- a/apps/sim/app/api/v2/workflows/[id]/export/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/export/route.ts @@ -7,11 +7,17 @@ import { workflowOperations } from '@/lib/workflows/application/operations' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** + * `headSafe: false` because the use case projects a `WORKFLOW_EXPORTED` audit + * event. Letting Next alias `HEAD` onto this `GET` would record an export that + * handed the caller no bytes. + */ export const GET = defineV2JsonRoute({ contract: v2ExportWorkflowContract, auth: v2ApiKeyAuth, operation: workflowOperations.export, rateLimit: v2RateLimits.publicApi, + headSafe: false, errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, mapInput: ({ params }) => ({ workflowId: params.id }), useCase: exportWorkflow, diff --git a/apps/sim/app/api/v2/workflows/[id]/rollback/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/rollback/route.test.ts index 2bd018a3df9..e06621ad0cc 100644 --- a/apps/sim/app/api/v2/workflows/[id]/rollback/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/rollback/route.test.ts @@ -25,6 +25,12 @@ import { workflowOperations } from '@/lib/workflows/application/operations' import { POST } from '@/app/api/v2/workflows/[id]/rollback/route' describe('/api/v2/workflows/[id]/rollback route definition', () => { + /** + * Both the malformed-body 400 and the oversized-body 413 are v2 builder + * defaults, so neither belongs on the route. The envelope they produce is + * asserted once against the builder in + * `lib/api/server/routes/v2-error-envelope.test.ts`. + */ it('keeps an omitted rollback body valid and delegates version selection to the use case', async () => { expect(v2RollbackWorkflowContract.body?.parse(undefined)).toEqual({}) expect(POST).toMatchObject({ @@ -41,15 +47,7 @@ describe('/api/v2/workflows/[id]/rollback route definition', () => { }) ) - const invalidJsonResponse = Reflect.get( - Reflect.get(POST, 'parseOptions'), - 'invalidJsonResponse' - )() - expect(invalidJsonResponse.status).toBe(400) - expect(await invalidJsonResponse.json()).toEqual({ - error: { code: 'BAD_REQUEST', message: 'Request body must be valid JSON' }, - }) - + expect(Reflect.get(Reflect.get(POST, 'parseOptions'), 'invalidJsonResponse')).toBeUndefined() expect( Reflect.get(Reflect.get(POST, 'parseOptions'), 'payloadTooLargeResponse') ).toBeUndefined() diff --git a/apps/sim/app/api/v2/workflows/[id]/rollback/route.ts b/apps/sim/app/api/v2/workflows/[id]/rollback/route.ts index c0b9b6c20e0..3c40d92b4ba 100644 --- a/apps/sim/app/api/v2/workflows/[id]/rollback/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/rollback/route.ts @@ -4,7 +4,6 @@ import { generateRequestId } from '@/lib/core/utils/request' import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' import { activateWorkflowVersion } from '@/lib/workflows/application/deployments' import { workflowOperations } from '@/lib/workflows/application/operations' -import { v2Error } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const runtime = 'nodejs' @@ -18,7 +17,6 @@ export const POST = defineV2JsonRoute({ errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, parseOptions: { optionalJsonBody: true, - invalidJsonResponse: () => v2Error('BAD_REQUEST', 'Request body must be valid JSON'), }, mapInput: ({ params, body }) => ({ workflowId: params.id, diff --git a/apps/sim/app/api/v2/workflows/[id]/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/route.test.ts index d16cabfc429..72003b5574f 100644 --- a/apps/sim/app/api/v2/workflows/[id]/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/route.test.ts @@ -1,17 +1,22 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), readWorkflow: vi.fn(), updateWorkflow: vi.fn(), deleteWorkflow: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/workflows/application/read-workflow', () => ({ @@ -23,18 +28,9 @@ vi.mock('@/lib/workflows/application/update-workflow', () => ({ vi.mock('@/lib/workflows/application/delete-workflow', () => ({ deleteWorkflow: { operation: { id: 'workflows.delete' }, execute: mocks.deleteWorkflow }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) import { NoWorkspaceAccessError, PersonalApiKeysDisabledError } from '@/lib/core/application' import { DELETE, GET, PATCH } from '@/app/api/v2/workflows/[id]/route' @@ -71,18 +67,10 @@ const routeContext = { params: Promise.resolve({ id: WORKFLOW_ID }) } describe('/api/v2/workflows/[id]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.readWorkflow.mockResolvedValue({ workflow, workspaceId: WORKSPACE_ID, @@ -176,4 +164,16 @@ describe('/api/v2/workflows/[id]', () => { request, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost/api/v2/workflows/${WORKFLOW_ID}`), + routeContext + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/workflows/[id]/route.ts b/apps/sim/app/api/v2/workflows/[id]/route.ts index 43c40be983f..9e1d6be489e 100644 --- a/apps/sim/app/api/v2/workflows/[id]/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/route.ts @@ -12,6 +12,18 @@ import { updateWorkflow } from '@/lib/workflows/application/update-workflow' export const revalidate = 0 +/** + * Deliberately head-safe despite issuing a write. + * + * Reading a workflow can trigger a migrate-on-read `workflow_blocks` update when + * `applyBlockMigrations` upgrades a stored block. That write is convergent: it is + * conditional on a migration actually applying, idempotent, and would be issued by + * the next ordinary read regardless, so a `HEAD` only brings it forward. + * + * Declaring `headSafe: false` would also cost real capability: the bodiless + * `200` is unconditional, so a `HEAD` could no longer distinguish a workflow + * that exists from one that does not. + */ export const GET = defineV2JsonRoute({ contract: v2GetWorkflowContract, auth: v2ApiKeyAuth, diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.test.ts index 2d726aa208d..87d14c90168 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.test.ts @@ -45,6 +45,7 @@ vi.mock('@/lib/api/server/routes', () => { V2RouteInfrastructureError, v2ApiKeyAuth: { kind: 'v2-api-key' }, v2RateLimits: { publicApi: { kind: 'public-api' } }, + V2_PARSE_DEFAULTS: {}, v2OrchestrationErrorPolicy: { render: renderOrchestrationError }, } }) diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.ts b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.ts index cc0938d8232..d3e3410117e 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/resume/route.ts @@ -8,6 +8,7 @@ import { import { parseRequest } from '@/lib/api/server' import { admitV2Request, + V2_PARSE_DEFAULTS, V2RouteInfrastructureError, v2ApiKeyAuth, v2RateLimits, @@ -18,7 +19,7 @@ import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' import { workflowOperations } from '@/lib/workflows/application/operations' import { resumeWorkflowRun } from '@/lib/workflows/application/resume-run' import { ResumeWorkflowExecutionError } from '@/lib/workflows/executor/resume-execution' -import { type V2ErrorCode, v2Data, v2Error, v2ValidationError } from '@/app/api/v2/lib/response' +import { type V2ErrorCode, v2Data, v2Error } from '@/app/api/v2/lib/response' import { classifyExecutionError } from '@/executor/utils/errors' const logger = createLogger('V2WorkflowResumeAPI') @@ -52,8 +53,8 @@ export const POST = withRouteHandler( if (!admission.success) return admission.response const parsed = await parseRequest(v2ResumeWorkflowContract, request, context, { + ...V2_PARSE_DEFAULTS, maxBodyBytes: 10 * 1024 * 1024, - validationErrorResponse: v2ValidationError, }) if (!parsed.success) return parsed.response const { id: workflowId, runId } = parsed.data.params diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/route.test.ts index 14ca04fbc5d..e67b9ddf739 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/[runId]/route.test.ts @@ -1,40 +1,27 @@ /** * @vitest-environment node */ -import { createMockRequest } from '@sim/testing' +import { + createMockRequest, + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mocks, MockV2ApiKeyUnauthenticatedError } = vi.hoisted(() => { - class MockV2ApiKeyUnauthenticatedError extends Error {} - return { - MockV2ApiKeyUnauthenticatedError, - mocks: { - authenticate: vi.fn(), - cancel: vi.fn(), - capture: vi.fn(), - checkOperationRate: vi.fn(), - checkPreAuthRate: vi.fn(), - readRun: vi.fn(), - }, - } -}) - -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: MockV2ApiKeyUnauthenticatedError, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkPreAuthRate - checkRateLimitDirectOrThrow = mocks.checkOperationRate - }, +const mocks = vi.hoisted(() => ({ + cancel: vi.fn(), + capture: vi.fn(), + readRun: vi.fn(), })) -vi.mock('@/app/api/v2/lib/gate', () => ({ - v2ApiGateError: vi.fn().mockResolvedValue(null), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/posthog/server', () => ({ captureServerEvent: mocks.capture })) @@ -98,6 +85,17 @@ const baseStatus = { blockOutputs: null, } +/** + * Local denial fixture — the harness only publishes the allowed shapes, and the + * cancel adapter must surface `retryAfterMs` as a `Retry-After` header. + */ +const OPERATION_RATE_LIMIT_DENIED = { + allowed: false, + remaining: 0, + resetAt: new Date('2026-08-05T01:00:00Z'), + retryAfterMs: 5_000, +} as const + const successfulCancellation = { success: true, executionId: 'run-1', @@ -113,17 +111,10 @@ const successfulCancellation = { describe('v2 run detail and cancel adapters', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.checkPreAuthRate.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) - mocks.checkOperationRate.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.readRun.mockResolvedValue(baseStatus) mocks.cancel.mockResolvedValue(successfulCancellation) }) @@ -223,13 +214,14 @@ describe('v2 run detail and cancel adapters', () => { }) it('rejects missing API keys before reading the run', async () => { - mocks.authenticate.mockRejectedValueOnce( + v2RouteMocks.authenticate.mockRejectedValueOnce( new MockV2ApiKeyUnauthenticatedError('API key required') ) const response = await callStatus() expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') expect(mocks.readRun).not.toHaveBeenCalled() }) @@ -249,8 +241,8 @@ describe('v2 run detail and cancel adapters', () => { input: { workflowId: 'workflow-1', runId: 'run-1' }, request: expect.anything(), }) - expect(mocks.checkOperationRate).toHaveBeenCalledTimes(2) - expect(mocks.checkOperationRate).toHaveBeenCalledWith( + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.operationRate).toHaveBeenCalledWith( 'v2:workflows.runs.cancel:api-key:key-1', expect.anything() ) @@ -258,18 +250,9 @@ describe('v2 run detail and cancel adapters', () => { }) it('keeps cancellation request-rate admission separate from run control', async () => { - mocks.checkOperationRate - .mockResolvedValueOnce({ - allowed: false, - remaining: 0, - resetAt: new Date('2026-08-05T01:00:00Z'), - retryAfterMs: 5_000, - }) - .mockResolvedValueOnce({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) + v2RouteMocks.operationRate + .mockResolvedValueOnce(OPERATION_RATE_LIMIT_DENIED) + .mockResolvedValueOnce(V2_OPERATION_RATE_LIMIT_ALLOWED) const response = await cancelPost(createMockRequest('POST', undefined, {}), { params: Promise.resolve({ id: 'workflow-1', runId: 'run-1' }), @@ -296,7 +279,7 @@ describe('v2 run detail and cancel adapters', () => { }) it('projects cancellation analytics only after a successful personal-key result', async () => { - mocks.authenticate.mockResolvedValueOnce({ + v2RouteMocks.authenticate.mockResolvedValueOnce({ ...auth, principal: { kind: 'personal_api_key', userId: 'key-user', keyId: 'personal-key' }, rolloutUserId: 'key-user', diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/runs/route.test.ts index f3714d3f563..0d01548f713 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/route.test.ts @@ -1,32 +1,25 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticate: vi.fn(), - checkPreAuthRate: vi.fn(), - checkOperationRate: vi.fn(), listRuns: vi.fn(), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticate, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkPreAuthRate - checkRateLimitDirectOrThrow = mocks.checkOperationRate - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ - v2ApiGateError: vi.fn().mockResolvedValue(null), -})) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) vi.mock('@/lib/workflows/application/list-workflow-runs', () => ({ listWorkflowRuns: { @@ -35,6 +28,7 @@ vi.mock('@/lib/workflows/application/list-workflow-runs', () => ({ }, })) +import { REFILTERED_CURSOR_MESSAGE, UNREADABLE_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { NoWorkspaceAccessError, PersonalApiKeysDisabledError } from '@/lib/core/application' import { GET } from '@/app/api/v2/workflows/[id]/runs/route' @@ -85,17 +79,10 @@ const EXECUTIONS = [ describe('GET /api/v2/workflows/[id]/runs', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticate.mockResolvedValue(auth) - mocks.checkPreAuthRate.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) - mocks.checkOperationRate.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-05T01:00:00Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listRuns.mockResolvedValue({ data: EXECUTIONS, nextCursor: null, @@ -153,15 +140,71 @@ describe('GET /api/v2/workflows/[id]/runs', () => { expect(JSON.parse(Buffer.from(body.nextCursor, 'base64').toString())).toEqual({ sort: 'startedAt:asc', keys: ['2026-08-05T00:01:00.000Z', 'row-1'], + filter: expect.any(String), + }) + }) + + /** + * Resuming a cursor under a different filter is a 400, not a page sequenced + * against rows the new filter excludes. The assertion above pins that a filter + * is stamped at all; this pins that the stamp is read back and enforced. + */ + it('refuses a cursor minted under a different filter', async () => { + mocks.listRuns.mockResolvedValueOnce({ + data: EXECUTIONS, + nextCursor: { startedAt: EXECUTIONS[1].startedAt, rowId: 'row-1' }, + workflowId: 'workflow-1', + order: 'asc', + }) + + const { nextCursor } = await (await callGet('?order=asc&status=completed')).json() + expect(nextCursor).toEqual(expect.any(String)) + + mocks.listRuns.mockClear() + const replayed = await callGet( + `?order=asc&status=failed&cursor=${encodeURIComponent(nextCursor)}` + ) + + expect(replayed.status).toBe(400) + expect((await replayed.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.listRuns).not.toHaveBeenCalled() + }) + + /** + * This list orders by the single `order` param — its query schema is + * `.strict()` and declares no `sortBy` — so the sort-mismatch wording would + * answer one 400 with advice that earns a second. + */ + it('names a cursor with unusable keys unreadable rather than blaming sortBy', async () => { + mocks.listRuns.mockResolvedValueOnce({ + data: EXECUTIONS, + nextCursor: { startedAt: EXECUTIONS[1].startedAt, rowId: 'row-1' }, + workflowId: 'workflow-1', + order: 'desc', }) + + const { nextCursor } = await (await callGet()).json() + const payload = JSON.parse(Buffer.from(nextCursor, 'base64').toString()) + const tampered = Buffer.from( + JSON.stringify({ ...payload, keys: ['not-a-date', 'row-1'] }) + ).toString('base64') + + mocks.listRuns.mockClear() + const response = await callGet(`?cursor=${encodeURIComponent(tampered)}`) + + expect(response.status).toBe(400) + const { error } = await response.json() + expect(error.message).toBe(UNREADABLE_CURSOR_MESSAGE) + expect(error.message).not.toMatch(/sortBy/) + expect(mocks.listRuns).not.toHaveBeenCalled() }) it('rejects an invalid cursor after API-key admission without calling the use case', async () => { const response = await callGet('?cursor=not-a-cursor') expect(response.status).toBe(400) - expect(mocks.authenticate).toHaveBeenCalledOnce() - expect(mocks.checkOperationRate).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalledOnce() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) expect(mocks.listRuns).not.toHaveBeenCalled() }) @@ -242,4 +285,13 @@ describe('GET /api/v2/workflows/[id]/runs', () => { error: { code: 'INTERNAL_ERROR', message: 'Internal server error' }, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await callGet() + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/workflows/[id]/runs/route.ts b/apps/sim/app/api/v2/workflows/[id]/runs/route.ts index 6893f79f822..e504143b335 100644 --- a/apps/sim/app/api/v2/workflows/[id]/runs/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/runs/route.ts @@ -3,17 +3,35 @@ import { v2ListWorkflowRunsContract, v2WorkflowRunListStatusValueSchema, } from '@/lib/api/contracts/v2/workflows' -import { INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' +import { + cursorRoute, + cursorScopeKey, + instantScopePart, + UNREADABLE_CURSOR_MESSAGE, +} from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { OrchestrationError } from '@/lib/core/orchestration/types' import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' import { listWorkflowRuns } from '@/lib/workflows/application/list-workflow-runs' import { workflowOperations } from '@/lib/workflows/application/operations' -import { cursorSortKey, decodeSortedCursor, encodeSortedCursor } from '@/app/api/v2/lib/response' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Every param that changes which runs, in which order, this list returns. */ +function runCursorFilters( + workflowId: string, + query: { status?: string; trigger?: string; startDate?: string; endDate?: string } +) { + return cursorScopeKey(cursorRoute(v2ListWorkflowRunsContract, { id: workflowId }), { + status: query.status, + trigger: query.trigger, + startDate: instantScopePart(query.startDate), + endDate: instantScopePart(query.endDate), + }) +} + /** List the durable runs belonging to one workflow. */ export const GET = defineV2JsonRoute({ contract: v2ListWorkflowRunsContract, @@ -23,21 +41,22 @@ export const GET = defineV2JsonRoute({ errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, mapInput: ({ params, query }) => { const { status, trigger, startDate, endDate, limit, cursor, order } = query - const sort = cursorSortKey('startedAt', order) - const decodedCursor = decodeSortedCursor(cursor, sort) - if (decodedCursor.status === 'invalid') { - throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) - } - const [cursorStartedAt, cursorRowId] = decodedCursor.status === 'ok' ? decodedCursor.keys : [] + const cursorKeys = readSortedCursor( + cursor, + 'startedAt', + order, + runCursorFilters(params.id, query) + ) + const [cursorStartedAt, cursorRowId] = cursorKeys ?? [] const cursorDate = typeof cursorStartedAt === 'string' ? new Date(cursorStartedAt) : null if ( - decodedCursor.status === 'ok' && - (decodedCursor.keys.length !== 2 || + cursorKeys && + (cursorKeys.length !== 2 || !cursorDate || Number.isNaN(cursorDate.getTime()) || typeof cursorRowId !== 'string') ) { - throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) + throw new OrchestrationError('validation', UNREADABLE_CURSOR_MESSAGE) } return { @@ -48,14 +67,14 @@ export const GET = defineV2JsonRoute({ endDate: endDate ? new Date(endDate) : undefined, limit, cursor: - decodedCursor.status === 'ok' && cursorDate && typeof cursorRowId === 'string' + cursorKeys && cursorDate && typeof cursorRowId === 'string' ? { startedAt: cursorDate, rowId: cursorRowId } : undefined, order, } }, useCase: listWorkflowRuns, - present: (result) => { + present: (result, { params, query }) => { const data: V2WorkflowRunListItem[] = result.data.map((row) => ({ runId: row.executionId, workflowId: row.workflowId ?? result.workflowId, @@ -66,13 +85,14 @@ export const GET = defineV2JsonRoute({ durationMs: row.durationMs, cost: row.costTotal != null ? { total: Number(row.costTotal) } : null, })) - const sort = cursorSortKey('startedAt', result.order) - const nextCursor = result.nextCursor - ? encodeSortedCursor(sort, [ - result.nextCursor.startedAt.toISOString(), - result.nextCursor.rowId, - ]) - : null + const nextCursor = writeSortedCursor( + result.nextCursor + ? [result.nextCursor.startedAt.toISOString(), result.nextCursor.rowId] + : null, + 'startedAt', + result.order, + runCursorFilters(params.id, query) + ) return { data, nextCursor } }, }) diff --git a/apps/sim/app/api/v2/workflows/[id]/versions/[version]/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/versions/[version]/route.test.ts index 909c823e864..fd19cec176f 100644 --- a/apps/sim/app/api/v2/workflows/[id]/versions/[version]/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/versions/[version]/route.test.ts @@ -1,17 +1,22 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), resolvePermission: vi.fn(), resolveWorkflowContext: vi.fn(), readVersion: vi.fn(), - gate: vi.fn(), })) vi.mock('@sim/platform-authz/workspace', () => ({ @@ -60,18 +65,9 @@ vi.mock('@/blocks/registry', () => ({ outputs: {}, }), })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) import { GET } from '@/app/api/v2/workflows/[id]/versions/[version]/route' @@ -136,20 +132,12 @@ function versionState() { describe('GET /api/v2/workflows/[id]/versions/[version]', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.gate.mockResolvedValue(null) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.resolvePermission.mockResolvedValue('admin') mocks.resolveWorkflowContext.mockResolvedValue(workflowContext) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) mocks.readVersion.mockResolvedValue({ id: 'version-2', version: 2, @@ -194,4 +182,13 @@ describe('GET /api/v2/workflows/[id]/versions/[version]', () => { expect(JSON.stringify(subBlocks)).not.toContain('sk-tool-plaintext-secret') expect(JSON.stringify(subBlocks)).not.toContain('table-plaintext-secret') }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await get() + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) }) diff --git a/apps/sim/app/api/v2/workflows/[id]/versions/route.test.ts b/apps/sim/app/api/v2/workflows/[id]/versions/route.test.ts index aca18ee8ede..d9762d98578 100644 --- a/apps/sim/app/api/v2/workflows/[id]/versions/route.test.ts +++ b/apps/sim/app/api/v2/workflows/[id]/versions/route.test.ts @@ -1,15 +1,21 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' +import { REFILTERED_CURSOR_MESSAGE, UNREADABLE_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), listVersions: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/workflows/application/list-workflow-versions', () => ({ @@ -18,18 +24,9 @@ vi.mock('@/lib/workflows/application/list-workflow-versions', () => ({ execute: mocks.listVersions, }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) import { GET } from '@/app/api/v2/workflows/[id]/versions/route' @@ -46,21 +43,57 @@ const auth = { } const context = { params: Promise.resolve({ id: 'workflow-1' }) } +function contextFor(workflowId: string) { + return { params: Promise.resolve({ id: workflowId }) } +} + +function listVersions(workflowId: string, query = '') { + return GET( + new NextRequest(`http://localhost/api/v2/workflows/${workflowId}/versions${query}`), + contextFor(workflowId) + ) +} + +/** + * A cursor as the route itself mints it, rather than a payload hand-built by + * the test. The binding a cursor carries is the route's to compute, so a test + * that reconstructs it would pass against a route that stopped applying one. + */ +async function mintCursor(workflowId: string): Promise { + mocks.listVersions.mockResolvedValueOnce({ + versions: [ + { + id: 'version-5', + version: 5, + name: 'Production', + description: null, + isActive: true, + createdAt: new Date('2026-08-01T00:00:00.000Z'), + deployedByName: 'Ada', + latestOperationStatus: 'active', + }, + ], + hasMore: true, + }) + const { nextCursor } = await (await listVersions(workflowId)).json() + expect(typeof nextCursor).toBe('string') + return nextCursor +} + +/** A minted cursor's binding, carrying a forged payload inside it. */ +async function forgeInsideBinding(workflowId: string, payload: unknown): Promise { + const { scope } = JSON.parse(Buffer.from(await mintCursor(workflowId), 'base64').toString()) + const inner = Buffer.from(JSON.stringify(payload)).toString('base64') + return Buffer.from(JSON.stringify({ scope, inner })).toString('base64') +} + describe('GET /api/v2/workflows/[id]/versions', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(auth) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(auth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listVersions.mockResolvedValue({ versions: [ { @@ -115,5 +148,82 @@ describe('GET /api/v2/workflows/[id]/versions', () => { expect(response.status).toBe(400) expect(mocks.listVersions).not.toHaveBeenCalled() + /** + * The undecodable-token message, not the sort-mismatch one: this list + * declares no `sortBy`/`sortOrder`, so naming them would answer a 400 with + * advice that earns a second. + */ + expect((await response.json()).error.message).toBe(UNREADABLE_CURSOR_MESSAGE) + }) + + /** + * A cursor is caller-controlled bytes, so its decoded payload is validated + * like any request field. `version` is compared against an `integer` column, + * where an out-of-range value overflows the comparison and 500s instead of + * returning an empty page. + */ + it.each([ + ['out of the integer range', { version: 2147483648 }], + ['at zero', { version: 0 }], + ['non-numeric', { version: 'two' }], + ['carrying an unknown key', { version: 2, sort: 'name' }], + ['missing its key', {}], + ])('rejects a forged cursor %s', async (_case, payload) => { + const cursor = await forgeInsideBinding('workflow-1', payload) + mocks.listVersions.mockClear() + + const response = await listVersions('workflow-1', `?cursor=${encodeURIComponent(cursor)}`) + + expect(response.status).toBe(400) + expect(mocks.listVersions).not.toHaveBeenCalled() + }) + + /** + * The regression guard for the binding: a list still pages through its OWN + * cursor. A token that no route accepts binds nothing — it just breaks + * pagination. + */ + it('resumes from the cursor it minted', async () => { + const cursor = await mintCursor('workflow-1') + mocks.listVersions.mockClear() + + const response = await listVersions('workflow-1', `?cursor=${encodeURIComponent(cursor)}`) + + expect(response.status).toBe(200) + expect(mocks.listVersions).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ afterVersion: 5 }) }) + ) + }) + + /** + * A `version` is an ordinal every workflow's history numbers from 1, so an + * unbound token from a sibling workflow decoded cleanly and resumed from a + * position in a history the caller never walked — silently skipping versions. + */ + it('refuses a cursor minted on another workflow', async () => { + const cursor = await mintCursor('workflow-1') + mocks.listVersions.mockClear() + + const response = await listVersions('workflow-2', `?cursor=${encodeURIComponent(cursor)}`) + + expect(response.status).toBe(400) + expect((await response.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.listVersions).not.toHaveBeenCalled() + }) + + it('mints a cursor bound to the workflow that answered', async () => { + expect(await mintCursor('workflow-1')).not.toBe(await mintCursor('workflow-2')) + }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest('http://localhost/api/v2/workflows/workflow-1/versions?limit=10'), + context + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') }) }) diff --git a/apps/sim/app/api/v2/workflows/[id]/versions/route.ts b/apps/sim/app/api/v2/workflows/[id]/versions/route.ts index 1fbb169fe33..30cff126cec 100644 --- a/apps/sim/app/api/v2/workflows/[id]/versions/route.ts +++ b/apps/sim/app/api/v2/workflows/[id]/versions/route.ts @@ -1,17 +1,36 @@ import type { V2WorkflowVersion } from '@/lib/api/contracts/v2/workflows' -import { v2ListWorkflowVersionsContract } from '@/lib/api/contracts/v2/workflows' +import { + v2ListWorkflowVersionsContract, + v2WorkflowVersionCursorSchema, +} from '@/lib/api/contracts/v2/workflows' +import { cursorRoute, cursorScopeKey, UNREADABLE_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, v2RateLimits } from '@/lib/api/server/routes' import { OrchestrationError } from '@/lib/core/orchestration/types' import { v2WorkflowErrorPolicies } from '@/lib/workflows/api' import { listWorkflowVersions } from '@/lib/workflows/application/list-workflow-versions' import { workflowOperations } from '@/lib/workflows/application/operations' -import { decodeCursor, encodeCursor } from '@/app/api/v2/lib/response' +import { + decodeCursor, + encodeCursor, + encodeScopedCursor, + readScopedCursor, +} from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 -interface WorkflowVersionCursor { - version: number +/** + * The sequence a version cursor names a position in: this list, on THIS + * workflow. + * + * The payload is a bare `version` ordinal and every workflow numbers its + * history from 1, so an unscoped token minted on one workflow decoded cleanly + * against another and answered 200 from a position the caller never reached. + * The workflow id lives in the path, so the route is the only place that knows + * which history the ordinal counts within. + */ +function versionCursorScope(workflowId: string): string { + return cursorScopeKey(cursorRoute(v2ListWorkflowVersionsContract, { id: workflowId })) } export const GET = defineV2JsonRoute({ @@ -21,18 +40,19 @@ export const GET = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2WorkflowErrorPolicies.concealWorkflowAuthorization, mapInput: ({ params, query }) => { - const after = query.cursor ? decodeCursor(query.cursor) : null - if (query.cursor && (!after || !Number.isInteger(after.version) || after.version < 1)) { - throw new OrchestrationError('validation', 'Invalid cursor') + const inner = readScopedCursor(query.cursor, versionCursorScope(params.id)) + const decoded = inner ? v2WorkflowVersionCursorSchema.safeParse(decodeCursor(inner)) : undefined + if (decoded && !decoded.success) { + throw new OrchestrationError('validation', UNREADABLE_CURSOR_MESSAGE) } return { workflowId: params.id, limit: query.limit, - afterVersion: after?.version, + afterVersion: decoded?.data.version, } }, useCase: listWorkflowVersions, - present: ({ versions, hasMore }) => { + present: ({ versions, hasMore }, { params }) => { const data: V2WorkflowVersion[] = versions.map((version) => ({ id: version.id, version: version.version, @@ -48,7 +68,10 @@ export const GET = defineV2JsonRoute({ data, nextCursor: hasMore && data.length > 0 - ? encodeCursor({ version: data[data.length - 1].version }) + ? encodeScopedCursor( + versionCursorScope(params.id), + encodeCursor({ version: data[data.length - 1].version }) + ) : null, } }, diff --git a/apps/sim/app/api/v2/workflows/route.test.ts b/apps/sim/app/api/v2/workflows/route.test.ts index 6a3e094b23c..0eea5e18ef4 100644 --- a/apps/sim/app/api/v2/workflows/route.test.ts +++ b/apps/sim/app/api/v2/workflows/route.test.ts @@ -1,16 +1,21 @@ /** * @vitest-environment node */ +import { + MockV2ApiKeyUnauthenticatedError, + V2_OPERATION_RATE_LIMIT_ALLOWED, + V2_PREAUTH_RATE_LIMIT_ALLOWED, + v2ApiKeyAuthModuleMock, + v2GateModuleMock, + v2RateLimiterModuleMock, + v2RouteMocks, +} from '@sim/testing' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - authenticateV2ApiKey: vi.fn(), - checkRateLimitDirect: vi.fn(), - checkRateLimitDirectOrThrow: vi.fn(), createWorkflow: vi.fn(), listWorkflows: vi.fn(), - gate: vi.fn(), })) vi.mock('@/lib/workflows/application/create-workflow', () => ({ @@ -21,20 +26,9 @@ vi.mock('@/lib/workflows/application/list-workflows', () => ({ listWorkflows: { operation: { id: 'workflows.list' }, execute: mocks.listWorkflows }, })) -vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => ({ - authenticateV2ApiKey: mocks.authenticateV2ApiKey, - V2ApiKeyUnauthenticatedError: class V2ApiKeyUnauthenticatedError extends Error {}, -})) - -vi.mock('@/lib/core/rate-limiter', () => ({ - getRateLimit: () => ({ maxTokens: 100, refillRate: 50, refillIntervalMs: 60_000 }), - RateLimiter: class RateLimiter { - checkRateLimitDirect = mocks.checkRateLimitDirect - checkRateLimitDirectOrThrow = mocks.checkRateLimitDirectOrThrow - }, -})) - -vi.mock('@/app/api/v2/lib/gate', () => ({ v2ApiGateError: mocks.gate })) +vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) +vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) +vi.mock('@/app/api/v2/lib/gate', () => v2GateModuleMock) import { GET, POST } from '@/app/api/v2/workflows/route' @@ -82,18 +76,10 @@ const personalAuth = { describe('/api/v2/workflows', () => { beforeEach(() => { vi.clearAllMocks() - mocks.authenticateV2ApiKey.mockResolvedValue(workspaceAuth) - mocks.gate.mockResolvedValue(null) - mocks.checkRateLimitDirect.mockResolvedValue({ - allowed: true, - remaining: 599, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) - mocks.checkRateLimitDirectOrThrow.mockResolvedValue({ - allowed: true, - remaining: 99, - resetAt: new Date('2026-08-01T01:00:00.000Z'), - }) + v2RouteMocks.authenticate.mockResolvedValue(workspaceAuth) + v2RouteMocks.gate.mockResolvedValue(null) + v2RouteMocks.preauthRate.mockResolvedValue(V2_PREAUTH_RATE_LIMIT_ALLOWED) + v2RouteMocks.operationRate.mockResolvedValue(V2_OPERATION_RATE_LIMIT_ALLOWED) mocks.listWorkflows.mockResolvedValue({ workflows: [WORKFLOW], nextCursorKeys: null, @@ -107,11 +93,76 @@ describe('/api/v2/workflows', () => { const response = await GET(new NextRequest('http://localhost/api/v2/workflows')) expect(response.status).toBe(400) - expect(mocks.authenticateV2ApiKey).toHaveBeenCalledOnce() - expect(mocks.checkRateLimitDirectOrThrow).toHaveBeenCalledTimes(2) + expect(v2RouteMocks.authenticate).toHaveBeenCalledOnce() + expect(v2RouteMocks.operationRate).toHaveBeenCalledTimes(2) + expect(mocks.listWorkflows).not.toHaveBeenCalled() + }) + + /** + * The reported defect: a cursor from an unfiltered page was accepted under + * `deployedOnly=true` or a changed `search`, and answered with whatever + * matched the new filter *after* the old position — every earlier match + * silently missing behind an opaque token. + */ + it.each([ + ['deployedOnly', 'deployedOnly=true'], + ['search', 'search=billing'], + ['folderPath', 'folderPath=/Ops'], + ])('refuses a cursor replayed under a different %s', async (_filter, param) => { + mocks.listWorkflows.mockResolvedValueOnce({ + workflows: [WORKFLOW], + nextCursorKeys: [1, WORKFLOW.id], + sortBy: 'position', + sortOrder: 'asc', + }) + const firstPage = await ( + await GET(new NextRequest(`http://localhost/api/v2/workflows?workspaceId=${WORKSPACE_ID}`)) + ).json() + expect(firstPage.nextCursor).toEqual(expect.any(String)) + mocks.listWorkflows.mockClear() + + const response = await GET( + new NextRequest( + `http://localhost/api/v2/workflows?workspaceId=${WORKSPACE_ID}&${param}&cursor=${encodeURIComponent(firstPage.nextCursor)}` + ) + ) + + expect(response.status).toBe(400) + expect(await response.json()).toMatchObject({ + error: { code: 'BAD_REQUEST', message: expect.stringContaining('requested filters') }, + }) expect(mocks.listWorkflows).not.toHaveBeenCalled() }) + it('resumes a cursor whose filters are unchanged', async () => { + mocks.listWorkflows.mockResolvedValueOnce({ + workflows: [WORKFLOW], + nextCursorKeys: [1, WORKFLOW.id], + sortBy: 'position', + sortOrder: 'asc', + }) + const firstPage = await ( + await GET( + new NextRequest( + `http://localhost/api/v2/workflows?workspaceId=${WORKSPACE_ID}&deployedOnly=true` + ) + ) + ).json() + + const response = await GET( + new NextRequest( + `http://localhost/api/v2/workflows?workspaceId=${WORKSPACE_ID}&deployedOnly=true&cursor=${encodeURIComponent(firstPage.nextCursor)}` + ) + ) + + expect(response.status).toBe(200) + expect(mocks.listWorkflows).toHaveBeenLastCalledWith( + expect.objectContaining({ + input: expect.objectContaining({ cursorKeys: [1, WORKFLOW.id] }), + }) + ) + }) + it('lists through the workspace principal and preserves rate headers', async () => { const request = new NextRequest( `http://localhost/api/v2/workflows?workspaceId=${WORKSPACE_ID}`, @@ -148,7 +199,7 @@ describe('/api/v2/workflows', () => { }) it('creates through a personal-key principal with the exact 201 contract', async () => { - mocks.authenticateV2ApiKey.mockResolvedValue(personalAuth) + v2RouteMocks.authenticate.mockResolvedValue(personalAuth) const request = new NextRequest('http://localhost/api/v2/workflows', { method: 'POST', headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, @@ -176,4 +227,69 @@ describe('/api/v2/workflows', () => { error: { code: 'INTERNAL_ERROR', message: 'Internal server error' }, }) }) + + it('rejects an unauthenticated request', async () => { + v2RouteMocks.authenticate.mockRejectedValueOnce(new MockV2ApiKeyUnauthenticatedError()) + + const response = await GET( + new NextRequest(`http://localhost/api/v2/workflows?workspaceId=${WORKSPACE_ID}`) + ) + + expect(response.status).toBe(401) + expect((await response.json()).error.code).toBe('UNAUTHORIZED') + }) + + /** + * A `U+0000` in caller text is a driver-level throw on the way to a `text` + * column, and an unclassified throw is a `500 INTERNAL_ERROR`. The read case + * needed no write at all — a search term was enough — so it is asserted here + * against the real route, not only against the parser. + */ + describe('NUL bytes in caller text', () => { + const NUL = '\u0000' + + it('rejects a NUL search term with the v2 validation envelope, not a 500', async () => { + const response = await GET( + new NextRequest( + `http://localhost/api/v2/workflows?workspaceId=${WORKSPACE_ID}&search=${encodeURIComponent(`a${NUL}b`)}`, + { headers: { 'x-api-key': 'secret' } } + ) + ) + + expect(response.status).toBe(400) + expect((await response.json()).error.code).toBe('BAD_REQUEST') + expect(mocks.listWorkflows).not.toHaveBeenCalled() + }) + + it('rejects a NUL workflow name before the create use case runs', async () => { + const response = await POST( + new NextRequest('http://localhost/api/v2/workflows', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify({ name: `a${NUL}b`, workspaceId: WORKSPACE_ID }), + }) + ) + + expect(response.status).toBe(400) + expect((await response.json()).error.code).toBe('BAD_REQUEST') + expect(mocks.createWorkflow).not.toHaveBeenCalled() + }) + + it('rejects a NUL description on the same body', async () => { + const response = await POST( + new NextRequest('http://localhost/api/v2/workflows', { + method: 'POST', + headers: { 'content-type': 'application/json', 'x-api-key': 'secret' }, + body: JSON.stringify({ + name: 'Daily digest', + description: `notes${NUL}`, + workspaceId: WORKSPACE_ID, + }), + }) + ) + + expect(response.status).toBe(400) + expect(mocks.createWorkflow).not.toHaveBeenCalled() + }) + }) }) diff --git a/apps/sim/app/api/v2/workflows/route.ts b/apps/sim/app/api/v2/workflows/route.ts index f4dc34c3ef7..d291c231a6c 100644 --- a/apps/sim/app/api/v2/workflows/route.ts +++ b/apps/sim/app/api/v2/workflows/route.ts @@ -1,46 +1,58 @@ import type { V2WorkflowListItem } from '@/lib/api/contracts/v2/workflows' import { v2CreateWorkflowContract, v2ListWorkflowsContract } from '@/lib/api/contracts/v2/workflows' -import { INVALID_CURSOR_MESSAGE } from '@/lib/api/list-query' +import { cursorRoute, cursorScopeKey } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, v2OrchestrationErrorPolicy, v2RateLimits, } from '@/lib/api/server/routes' -import { OrchestrationError } from '@/lib/core/orchestration/types' import { createWorkflow } from '@/lib/workflows/application/create-workflow' import { listWorkflows } from '@/lib/workflows/application/list-workflows' import { workflowOperations } from '@/lib/workflows/application/operations' -import { cursorSortKey, decodeSortedCursor, encodeSortedCursor } from '@/app/api/v2/lib/response' +import { readSortedCursor, writeSortedCursor } from '@/app/api/v2/lib/response' export const dynamic = 'force-dynamic' export const revalidate = 0 +/** Every param that changes which workflows, in which order, this list returns. */ +function workflowCursorFilters(query: { + workspaceId: string + folderPath?: string + deployedOnly: boolean + search?: string +}) { + return cursorScopeKey(cursorRoute(v2ListWorkflowsContract), { + workspaceId: query.workspaceId, + folderPath: query.folderPath, + deployedOnly: query.deployedOnly, + search: query.search, + }) +} + export const GET = defineV2JsonRoute({ contract: v2ListWorkflowsContract, auth: v2ApiKeyAuth, operation: workflowOperations.list, rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, - mapInput: ({ query }) => { - const sort = cursorSortKey(query.sortBy, query.sortOrder) - const decoded = decodeSortedCursor(query.cursor, sort) - if (decoded.status === 'invalid') { - throw new OrchestrationError('validation', INVALID_CURSOR_MESSAGE) - } - return { - workspaceId: query.workspaceId, - folderPath: query.folderPath, - deployedOnly: query.deployedOnly, - search: query.search, - sortBy: query.sortBy, - sortOrder: query.sortOrder, - cursorKeys: decoded.status === 'ok' ? decoded.keys : undefined, - limit: query.limit, - } - }, + mapInput: ({ query }) => ({ + workspaceId: query.workspaceId, + folderPath: query.folderPath, + deployedOnly: query.deployedOnly, + search: query.search, + sortBy: query.sortBy, + sortOrder: query.sortOrder, + cursorKeys: readSortedCursor( + query.cursor, + query.sortBy, + query.sortOrder, + workflowCursorFilters(query) + ), + limit: query.limit, + }), useCase: listWorkflows, - present: ({ workflows, nextCursorKeys, sortBy, sortOrder }) => ({ + present: ({ workflows, nextCursorKeys }, { query }) => ({ data: workflows.map( (workflow): V2WorkflowListItem => ({ id: workflow.id, @@ -56,9 +68,12 @@ export const GET = defineV2JsonRoute({ updatedAt: workflow.updatedAt.toISOString(), }) ), - nextCursor: nextCursorKeys - ? encodeSortedCursor(cursorSortKey(sortBy, sortOrder), nextCursorKeys) - : null, + nextCursor: writeSortedCursor( + nextCursorKeys, + query.sortBy, + query.sortOrder, + workflowCursorFilters(query) + ), }), }) diff --git a/apps/sim/app/api/v2/workspaces/[workspaceId]/members/route.ts b/apps/sim/app/api/v2/workspaces/[workspaceId]/members/route.ts index b07898ffdb1..190d930ca5f 100644 --- a/apps/sim/app/api/v2/workspaces/[workspaceId]/members/route.ts +++ b/apps/sim/app/api/v2/workspaces/[workspaceId]/members/route.ts @@ -2,6 +2,7 @@ import { v2ListWorkspaceMembersContract, v2WorkspaceMemberCursorSchema, } from '@/lib/api/contracts/v2/workspaces' +import { cursorRoute, cursorScopeKey, UNREADABLE_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { defineV2JsonRoute, v2ApiKeyAuth, @@ -11,7 +12,26 @@ import { import { OrchestrationError } from '@/lib/core/orchestration/types' import { listPublicWorkspaceMembers } from '@/lib/workspaces/application/list-public-workspace-members' import { workspaceOperations } from '@/lib/workspaces/application/operations' -import { decodeCursor, encodeCursor } from '@/app/api/v2/lib/response' +import { + decodeCursor, + encodeCursor, + encodeScopedCursor, + readScopedCursor, +} from '@/app/api/v2/lib/response' + +/** + * The sequence a member cursor names a position in: this roster, on THIS + * workspace. + * + * The payload is a bare email, and the same person is a member of every + * workspace they belong to, so an unscoped token minted on one roster decoded + * cleanly against another and resumed from an email that names a different + * position in it. The workspace id lives in the path, so the route is the only + * place that knows which roster the email indexes. + */ +function memberCursorScope(workspaceId: string): string { + return cursorScopeKey(cursorRoute(v2ListWorkspaceMembersContract, { workspaceId })) +} /** GET /api/v2/workspaces/[workspaceId]/members — Effective member roster. */ export const GET = defineV2JsonRoute({ @@ -21,11 +41,10 @@ export const GET = defineV2JsonRoute({ rateLimit: v2RateLimits.publicApi, errorPolicy: v2OrchestrationErrorPolicy, mapInput: ({ params, query }) => { - const decoded = query.cursor - ? v2WorkspaceMemberCursorSchema.safeParse(decodeCursor(query.cursor)) - : undefined + const inner = readScopedCursor(query.cursor, memberCursorScope(params.workspaceId)) + const decoded = inner ? v2WorkspaceMemberCursorSchema.safeParse(decodeCursor(inner)) : undefined if (decoded && !decoded.success) { - throw new OrchestrationError('validation', 'Invalid cursor') + throw new OrchestrationError('validation', UNREADABLE_CURSOR_MESSAGE) } return { workspaceId: params.workspaceId, @@ -34,7 +53,7 @@ export const GET = defineV2JsonRoute({ } }, useCase: listPublicWorkspaceMembers, - present: ({ page }) => ({ + present: ({ page }, { params }) => ({ data: page.members.map((member) => ({ email: member.email, name: member.name, @@ -43,6 +62,11 @@ export const GET = defineV2JsonRoute({ isExternal: member.isExternal, joinedAt: member.joinedAt.toISOString(), })), - nextCursor: page.nextEmail ? encodeCursor({ email: page.nextEmail }) : null, + nextCursor: page.nextEmail + ? encodeScopedCursor( + memberCursorScope(params.workspaceId), + encodeCursor({ email: page.nextEmail }) + ) + : null, }), }) diff --git a/apps/sim/app/api/v2/workspaces/route.test.ts b/apps/sim/app/api/v2/workspaces/route.test.ts index b4ad92a9872..e57d7679e56 100644 --- a/apps/sim/app/api/v2/workspaces/route.test.ts +++ b/apps/sim/app/api/v2/workspaces/route.test.ts @@ -35,6 +35,7 @@ vi.mock('@/lib/workspaces/application/list-public-workspace-members', () => ({ }, })) +import { REFILTERED_CURSOR_MESSAGE, UNREADABLE_CURSOR_MESSAGE } from '@/lib/api/cursor-binding' import { OrchestrationError } from '@/lib/core/orchestration/types' import { GET as listMembers } from '@/app/api/v2/workspaces/[workspaceId]/members/route' import { GET as getWorkspace } from '@/app/api/v2/workspaces/[workspaceId]/route' @@ -51,7 +52,34 @@ const auth = { rateLimitSubscription: null, keyType: 'workspace' as const, } -const context = () => ({ params: Promise.resolve({ workspaceId: WORKSPACE_ID }) }) +const OTHER_WORKSPACE_ID = 'b1c1a2f5-1f4b-4a2e-9a2f-1d0a5f1c9e77' +const context = (workspaceId: string = WORKSPACE_ID) => ({ + params: Promise.resolve({ workspaceId }), +}) + +function requestMembers(workspaceId: string, query = '') { + return listMembers( + new NextRequest(`http://localhost:3000/api/v2/workspaces/${workspaceId}/members${query}`), + context(workspaceId) + ) +} + +/** + * A cursor as the route itself mints it, rather than a payload hand-built by + * the test. The binding a cursor carries is the route's to compute, so a test + * that reconstructed it would pass against a route that stopped applying one. + */ +async function mintMemberCursor(workspaceId: string): Promise { + const { nextCursor } = await (await requestMembers(workspaceId, '?limit=1')).json() + expect(typeof nextCursor).toBe('string') + return nextCursor +} + +/** The payload inside a minted cursor's binding envelope. */ +function innerPayload(cursor: string): unknown { + const { inner } = JSON.parse(Buffer.from(cursor, 'base64').toString()) + return JSON.parse(Buffer.from(inner, 'base64').toString()) +} describe('v2 workspace routes', () => { beforeEach(() => { @@ -122,18 +150,72 @@ describe('v2 workspace routes', () => { isExternal: false, joinedAt: '2026-01-01T00:00:00.000Z', }) - expect(JSON.parse(Buffer.from(body.nextCursor, 'base64').toString())).toEqual({ - email: 'ada@example.com', - }) + expect(innerPayload(body.nextCursor)).toEqual({ email: 'ada@example.com' }) }) it('rejects malformed cursors before the application read', async () => { - const response = await listMembers( - new NextRequest( - `http://localhost:3000/api/v2/workspaces/${WORKSPACE_ID}/members?cursor=not-a-cursor` - ), - context() + const response = await requestMembers(WORKSPACE_ID, '?cursor=not-a-cursor') + + expect(response.status).toBe(400) + expect(mocks.listMembers).not.toHaveBeenCalled() + expect((await response.json()).error.message).toBe(UNREADABLE_CURSOR_MESSAGE) + }) + + /** + * The regression guard for the cursor's binding: the roster still pages + * through its OWN cursor. A token no route accepts binds nothing — it just + * breaks pagination. + */ + it('resumes from the cursor it minted', async () => { + const cursor = await mintMemberCursor(WORKSPACE_ID) + mocks.listMembers.mockClear() + + const response = await requestMembers(WORKSPACE_ID, `?cursor=${encodeURIComponent(cursor)}`) + + expect(response.status).toBe(200) + expect(mocks.listMembers).toHaveBeenCalledWith( + expect.objectContaining({ input: expect.objectContaining({ afterEmail: 'ada@example.com' }) }) + ) + }) + + /** + * The same person is a member of every workspace they belong to, so an + * unbound email token from one roster decoded cleanly against another and + * resumed from a position that silently skipped members. + */ + it('refuses a members cursor minted on another workspace', async () => { + const cursor = await mintMemberCursor(WORKSPACE_ID) + mocks.listMembers.mockClear() + + const response = await requestMembers( + OTHER_WORKSPACE_ID, + `?cursor=${encodeURIComponent(cursor)}` + ) + + expect(response.status).toBe(400) + expect((await response.json()).error.message).toBe(REFILTERED_CURSOR_MESSAGE) + expect(mocks.listMembers).not.toHaveBeenCalled() + }) + + it('mints a members cursor bound to the workspace that answered', async () => { + expect(await mintMemberCursor(WORKSPACE_ID)).not.toBe( + await mintMemberCursor(OTHER_WORKSPACE_ID) ) + }) + + it.each([ + ['non-email', { email: 'not-an-email' }], + ['carrying an unknown key', { email: 'ada@example.com', role: 'admin' }], + ['missing its key', {}], + ])('rejects a members cursor forged inside a valid binding %s', async (_case, payload) => { + const { scope } = JSON.parse( + Buffer.from(await mintMemberCursor(WORKSPACE_ID), 'base64').toString() + ) + const inner = Buffer.from(JSON.stringify(payload)).toString('base64') + const cursor = Buffer.from(JSON.stringify({ scope, inner })).toString('base64') + mocks.listMembers.mockClear() + + const response = await requestMembers(WORKSPACE_ID, `?cursor=${encodeURIComponent(cursor)}`) expect(response.status).toBe(400) expect(mocks.listMembers).not.toHaveBeenCalled() diff --git a/apps/sim/app/api/workflows/[id]/execute/route.ts b/apps/sim/app/api/workflows/[id]/execute/route.ts index 25704a10308..53040cb8ed0 100644 --- a/apps/sim/app/api/workflows/[id]/execute/route.ts +++ b/apps/sim/app/api/workflows/[id]/execute/route.ts @@ -385,6 +385,8 @@ type AsyncExecutionParams = { executionId: string copilotToolCallId?: string callChain?: string[] + enforceCredentialAccess?: boolean + isPublicApiAccess?: boolean executionTimeoutMs: number trustedInitialResolvedSecretTraceProvenance?: ResolvedSecretTraceProvenanceV1 } @@ -1245,6 +1247,8 @@ async function handleExecutePost( executionId, copilotToolCallId, callChain, + enforceCredentialAccess: useAuthenticatedUserAsActor, + isPublicApiAccess, executionTimeoutMs: preprocessResult.executionTimeout.async, trustedInitialResolvedSecretTraceProvenance, }) @@ -1382,6 +1386,7 @@ async function handleExecutePost( startTime: new Date().toISOString(), isClientSession, enforceCredentialAccess: useAuthenticatedUserAsActor, + isPublicApiAccess, workflowStateOverride: effectiveWorkflowStateOverride, largeValueExecutionIds, largeValueKeys, @@ -1627,7 +1632,13 @@ async function handleExecutePost( const streamVariables = cachedWorkflowData?.variables ?? (workflow as any).variables const streamWorkflow = { id: workflow.id, - userId: actorUserId, + /** + * The owner, not the actor: `executeWorkflow` reads this one field to set + * `workflowUserId`, which is the personal-environment fallback for runs with + * no identifiable caller. Passing the actor here made the streaming path + * resolve the actor where the JSON path resolves the owner. + */ + userId: workflow.userId, workspaceId, isDeployed: workflow.isDeployed, variables: streamVariables, @@ -1698,6 +1709,8 @@ async function handleExecutePost( base64MaxBytes, abortSignal, executionMode: 'stream', + enforceCredentialAccess: useAuthenticatedUserAsActor, + isPublicApiAccess, billingAttribution, largeValueKeys, fileKeys, @@ -2104,6 +2117,7 @@ async function handleExecutePost( startTime: new Date().toISOString(), isClientSession, enforceCredentialAccess: useAuthenticatedUserAsActor, + isPublicApiAccess, workflowStateOverride: effectiveWorkflowStateOverride, largeValueExecutionIds, largeValueKeys, diff --git a/apps/sim/app/api/workflows/[id]/executions/[executionId]/cancel/route.test.ts b/apps/sim/app/api/workflows/[id]/executions/[executionId]/cancel/route.test.ts index a8591a5f2ca..84ba7ec5d3a 100644 --- a/apps/sim/app/api/workflows/[id]/executions/[executionId]/cancel/route.test.ts +++ b/apps/sim/app/api/workflows/[id]/executions/[executionId]/cancel/route.test.ts @@ -1320,6 +1320,7 @@ describe('POST /api/workflows/[id]/executions/[executionId]/cancel', () => { expect(mockSet).toHaveBeenCalledWith({ status: 'cancelled', endedAt: expect.any(Date), + totalDurationMs: expect.anything(), executionDeadlineAt: null, }) }) @@ -1340,6 +1341,7 @@ describe('POST /api/workflows/[id]/executions/[executionId]/cancel', () => { expect(mockSet).toHaveBeenCalledWith({ status: 'cancelled', endedAt: expect.any(Date), + totalDurationMs: expect.anything(), executionDeadlineAt: null, }) }) diff --git a/apps/sim/app/api/workflows/[id]/executions/[executionId]/cancel/route.ts b/apps/sim/app/api/workflows/[id]/executions/[executionId]/cancel/route.ts index 9848732d8d3..d696a5e6f90 100644 --- a/apps/sim/app/api/workflows/[id]/executions/[executionId]/cancel/route.ts +++ b/apps/sim/app/api/workflows/[id]/executions/[executionId]/cancel/route.ts @@ -24,6 +24,7 @@ import { } from '@/lib/execution/cancellation' import { createExecutionEventWriter, readExecutionMetaState } from '@/lib/execution/event-buffer' import { abortManualExecution } from '@/lib/execution/manual-cancellation' +import { cancelledExecutionLogFields } from '@/lib/logs/execution/cancellation' import { workflowExecutionOriginSql } from '@/lib/logs/execution-origin' import { captureServerEvent } from '@/lib/posthog/server' import { @@ -215,7 +216,7 @@ async function claimExecutionLogCancellation(args: { const now = new Date() const [cancelledExecution] = await db .update(workflowExecutionLogs) - .set({ status: 'cancelled', endedAt: now, executionDeadlineAt: null }) + .set(cancelledExecutionLogFields(now)) .where( and( eq(workflowExecutionLogs.executionId, args.executionId), diff --git a/apps/sim/app/api/workflows/[id]/state/route.ts b/apps/sim/app/api/workflows/[id]/state/route.ts index 209cfd226e8..27ef1e72c2d 100644 --- a/apps/sim/app/api/workflows/[id]/state/route.ts +++ b/apps/sim/app/api/workflows/[id]/state/route.ts @@ -1,28 +1,17 @@ import { db } from '@sim/db' import { workflow } from '@sim/db/schema' import { createLogger } from '@sim/logger' -import { - assertWorkflowMutable, - authorizeWorkflowByWorkspacePermission, - WorkflowLockedError, -} from '@sim/platform-authz/workflow' +import { authorizeWorkflowByWorkspacePermission } from '@sim/platform-authz/workflow' import { toError } from '@sim/utils/errors' import { eq, sql } from 'drizzle-orm' import { type NextRequest, NextResponse } from 'next/server' import { putWorkflowNormalizedStateContract } from '@/lib/api/contracts/workflows' import { parseRequest } from '@/lib/api/server' import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid' -import { env } from '@/lib/core/config/env' import { generateRequestId } from '@/lib/core/utils/request' -import { getSocketServerUrl } from '@/lib/core/utils/urls' import { withRouteHandler } from '@/lib/core/utils/with-route-handler' -import { extractAndPersistCustomTools } from '@/lib/workflows/persistence/custom-tools-persistence' -import { prepareWorkflowStateForPersistence } from '@/lib/workflows/persistence/prepare-state' -import { - loadWorkflowFromNormalizedTables, - saveWorkflowToNormalizedTables, -} from '@/lib/workflows/persistence/utils' -import type { BlockState, WorkflowState } from '@/stores/workflows/workflow/types' +import { saveWorkflowNormalizedState } from '@/lib/workflows/persistence/save-normalized-state' +import { loadWorkflowFromNormalizedTables } from '@/lib/workflows/persistence/utils' const logger = createLogger('WorkflowStateAPI') @@ -117,164 +106,29 @@ export const PUT = withRouteHandler( const parsed = await parseRequest(putWorkflowNormalizedStateContract, request, context) if (!parsed.success) return parsed.response - const state = parsed.data.body - const authorization = await authorizeWorkflowByWorkspacePermission({ + const result = await saveWorkflowNormalizedState({ + requestId, workflowId, userId, - action: 'write', + state: parsed.data.body, }) - const workflowData = authorization.workflow - - if (!workflowData) { - logger.warn(`[${requestId}] Workflow ${workflowId} not found for state update`) - return NextResponse.json({ error: 'Workflow not found' }, { status: 404 }) - } - const canUpdate = authorization.allowed - - if (!canUpdate) { - logger.warn( - `[${requestId}] User ${userId} denied permission to update workflow state ${workflowId}` - ) + if (!result.success) { return NextResponse.json( - { error: authorization.message || 'Access denied' }, - { status: authorization.status || 403 } - ) - } - - await assertWorkflowMutable(workflowId) - - // Note: prior versions cross-checked that each variable's `workflowId` - // equalled the path param. The write contract does not carry `workflowId` - // per variable (the path param is the source of truth), so the check - // is unreachable and was removed. - - const { state: preparedState, warnings: preparationWarnings } = - prepareWorkflowStateForPersistence({ - blocks: state.blocks as Record, - edges: state.edges as WorkflowState['edges'], - }) - - const workflowState = { - ...preparedState, - lastSaved: state.lastSaved || Date.now(), - isDeployed: state.isDeployed || false, - deployedAt: state.deployedAt, - } - - const saveResult = await db.transaction(async (tx) => { - await tx - .select({ id: workflow.id }) - .from(workflow) - .where(eq(workflow.id, workflowId)) - .limit(1) - .for('update') - - const result = await saveWorkflowToNormalizedTables( - workflowId, - workflowState as WorkflowState, - tx - ) - - if (!result.success) return result - - // Update workflow's lastSynced timestamp and variables if provided - const updateData: { - lastSynced: Date - updatedAt: Date - variables?: typeof state.variables - } = { - lastSynced: new Date(), - updatedAt: new Date(), - } - - // If variables are provided in the state, update them in the workflow record - if (state.variables !== undefined) { - updateData.variables = state.variables - } - - await tx.update(workflow).set(updateData).where(eq(workflow.id, workflowId)) - - return result - }) - - if (!saveResult.success) { - logger.error( - `[${requestId}] Failed to save workflow ${workflowId} state:`, - saveResult.error - ) - return NextResponse.json( - { error: 'Failed to save workflow state', details: saveResult.error }, - { status: 500 } + { + error: result.error, + ...(result.details !== undefined ? { details: result.details } : {}), + }, + { status: result.status } ) } - // Extract and persist custom tools to database - try { - const workspaceId = workflowData.workspaceId - if (workspaceId) { - const { saved, errors } = await extractAndPersistCustomTools( - workflowState, - workspaceId, - userId - ) - - if (saved > 0) { - logger.info(`[${requestId}] Persisted ${saved} custom tool(s) to database`, { - workflowId, - }) - } - - if (errors.length > 0) { - logger.warn(`[${requestId}] Some custom tools failed to persist`, { - errors, - workflowId, - }) - } - } else { - logger.warn( - `[${requestId}] Workflow has no workspaceId, skipping custom tools persistence`, - { - workflowId, - } - ) - } - } catch (error) { - logger.error(`[${requestId}] Failed to persist custom tools`, { error, workflowId }) - } - const elapsed = Date.now() - startTime logger.info(`[${requestId}] Successfully saved workflow ${workflowId} state in ${elapsed}ms`) - try { - const notifyResponse = await fetch(`${getSocketServerUrl()}/api/workflow-updated`, { - method: 'POST', - headers: { - 'Content-Type': 'application/json', - 'x-api-key': env.INTERNAL_API_SECRET, - }, - body: JSON.stringify({ workflowId }), - }) - - if (!notifyResponse.ok) { - logger.warn( - `[${requestId}] Failed to notify Socket.IO server about workflow ${workflowId} update` - ) - } - } catch (notificationError) { - logger.warn( - `[${requestId}] Error notifying Socket.IO server about workflow ${workflowId} update`, - notificationError - ) - } - - return NextResponse.json({ success: true, warnings: preparationWarnings }, { status: 200 }) + return NextResponse.json({ success: true, warnings: result.warnings }, { status: 200 }) } catch (error: any) { - if (error instanceof WorkflowLockedError) { - return NextResponse.json({ error: error.message }, { status: error.status }) - } - const elapsed = Date.now() - startTime logger.error( `[${requestId}] Error saving workflow ${workflowId} state after ${elapsed}ms`, diff --git a/apps/sim/app/api/workspaces/[id]/files/[fileId]/content/route.test.ts b/apps/sim/app/api/workspaces/[id]/files/[fileId]/content/route.test.ts index 698f524a4ce..5be59775eea 100644 --- a/apps/sim/app/api/workspaces/[id]/files/[fileId]/content/route.test.ts +++ b/apps/sim/app/api/workspaces/[id]/files/[fileId]/content/route.test.ts @@ -135,9 +135,9 @@ describe('PUT /api/workspaces/[id]/files/[fileId]/content', () => { }) }) - it('allows JSON bodies above the default 50 MiB cap for base64 expansion', async () => { + it('allows a base64 JSON body up to what the proxy forwards intact', async () => { const response = await PUT( - createRequest({ content: 'TQ==', encoding: 'base64' }, 60 * 1024 * 1024), + createRequest({ content: 'TQ==', encoding: 'base64' }, 10 * 1024 * 1024), routeContext ) @@ -145,8 +145,13 @@ describe('PUT /api/workspaces/[id]/files/[fileId]/content', () => { expect(mocks.updateContent).toHaveBeenCalled() }) - it('rejects a JSON body above the inline-content cap after admission', async () => { - const response = await PUT(createRequest({ content: '' }, 70 * 1024 * 1024 + 1), routeContext) + /** + * The route declares a 70 MB inline cap, but Next's proxy truncates a client + * body past 10 MiB, so the parser clamps to that ceiling and answers 413 + * rather than letting a truncated prefix surface as malformed JSON. + */ + it('rejects a JSON body above the proxy ceiling after admission', async () => { + const response = await PUT(createRequest({ content: '' }, 10 * 1024 * 1024 + 1), routeContext) expect(response.status).toBe(413) expect(mocks.admit).toHaveBeenCalled() diff --git a/apps/sim/app/api/workspaces/[id]/files/inline/route.test.ts b/apps/sim/app/api/workspaces/[id]/files/inline/route.test.ts index 495044427bb..2b95cd7e007 100644 --- a/apps/sim/app/api/workspaces/[id]/files/inline/route.test.ts +++ b/apps/sim/app/api/workspaces/[id]/files/inline/route.test.ts @@ -30,6 +30,7 @@ describe('GET /api/workspaces/[id]/files/inline', () => { mockReadInline.mockResolvedValue({ file: { name: 'photo.png', type: 'image/png', size: PNG.length }, stream: new Blob([new Uint8Array(PNG)]).stream(), + contentAddressed: false, }) }) @@ -43,6 +44,7 @@ describe('GET /api/workspaces/[id]/files/inline', () => { input: { workspaceId: 'ws-1', fileId: 'wf_abc' }, }) ) + // A file id names the FILE, whose bytes move under it on every edit — so it must revalidate. expect(res.headers.get('Cache-Control')).toBe('private, no-cache, must-revalidate') expect(res.headers.get('Content-Disposition')).toBe('inline; filename="photo.png"') }) @@ -58,6 +60,24 @@ describe('GET /api/workspaces/[id]/files/inline', () => { }) }) + /** + * A storage key names one object and a content write never rewrites one, so these bytes can never + * change. Revalidating them meant re-downloading every embedded image on every open — a document is + * rendered by two editors (the read-only placeholder, then the live one) and each renders the image + * twice, so the image was fetched again on every one of those passes. + */ + it('lets the browser keep an image whose URL names the object that was streamed', async () => { + mockReadInline.mockResolvedValue({ + file: { name: 'photo.png', type: 'image/png', size: PNG.length }, + stream: new Blob([new Uint8Array(PNG)]).stream(), + contentAddressed: true, + }) + + const res = await GET(req('key=workspace%2Fws-1%2Fphoto.png'), params) + + expect(res.headers.get('Cache-Control')).toBe('private, max-age=31536000, immutable') + }) + it('returns the concealed 404 response for an unauthorized or missing file', async () => { mockReadInline.mockRejectedValue( new OrchestrationError('forbidden', 'Insufficient permissions') diff --git a/apps/sim/app/api/workspaces/[id]/files/inline/route.ts b/apps/sim/app/api/workspaces/[id]/files/inline/route.ts index ad3780eb4be..0a3f20bf4ec 100644 --- a/apps/sim/app/api/workspaces/[id]/files/inline/route.ts +++ b/apps/sim/app/api/workspaces/[id]/files/inline/route.ts @@ -10,6 +10,24 @@ import { encodeFilenameForHeader, getSecureFileHeaders } from '@/app/api/files/u export const dynamic = 'force-dynamic' +/** + * How long the browser may reuse an embedded image, decided by whether the URL names the exact object + * that was streamed (see {@link ReadWorkspaceInlineFileResult.contentAddressed}). + * + * A content write never rewrites a storage object, so a URL that names one addresses bytes that can + * never change and the browser needs no round trip — which is the difference between an embedded image + * reappearing instantly and being downloaded again. Every document render asks for the same image at + * least twice (ProseMirror's own DOM, then the React node view) and every editor mounts twice (the + * read-only placeholder, then the live editor), so revalidating each time meant re-fetching the whole + * image on every open and reload — measured at ~1 MB per open on a real document, with the image area + * blank until it landed. `private` keeps it out of shared caches: the bytes are authorized per user. + * + * Anything else — a request that names the FILE, whose bytes move under it, or one whose object was + * rotated away mid-request — keeps revalidating. + */ +const IMMUTABLE_CACHE_CONTROL = 'private, max-age=31536000, immutable' +const REVALIDATE_CACHE_CONTROL = 'private, no-cache, must-revalidate' + /** * GET /api/workspaces/[id]/files/inline?key=|fileId= * @@ -29,12 +47,12 @@ export const GET = defineInternalBinaryRoute({ fileId: query.fileId, }), useCase: readWorkspaceInlineFile, - present: ({ file, stream }) => { + present: ({ file, stream, contentAddressed }) => { const secure = getSecureFileHeaders(file.name, file.type) const headers = new Headers({ 'Content-Type': secure.contentType, 'Content-Disposition': `${secure.disposition}; ${encodeFilenameForHeader(file.name)}`, - 'Cache-Control': 'private, no-cache, must-revalidate', + 'Cache-Control': contentAddressed ? IMMUTABLE_CACHE_CONTROL : REVALIDATE_CACHE_CONTROL, 'X-Content-Type-Options': 'nosniff', }) if (secure.contentType === 'image/svg+xml') { diff --git a/apps/sim/app/invite/[id]/invite.tsx b/apps/sim/app/invite/[id]/invite.tsx index 232dfbdb884..ffc75e40866 100644 --- a/apps/sim/app/invite/[id]/invite.tsx +++ b/apps/sim/app/invite/[id]/invite.tsx @@ -15,7 +15,7 @@ import { InviteLayout, InviteStatusCard } from '@/app/invite/components' import { useInvitationDetails } from '@/hooks/queries/invitations' import { organizationKeys } from '@/hooks/queries/organization' import { refreshSessionQuery } from '@/hooks/queries/session' -import { subscriptionKeys } from '@/hooks/queries/subscription' +import { subscriptionKeys } from '@/hooks/queries/utils/subscription-keys' import { workspaceKeys } from '@/hooks/queries/workspace' const logger = createLogger('InviteById') diff --git a/apps/sim/app/layout.tsx b/apps/sim/app/layout.tsx index 6f5d5ff5705..21688e8470e 100644 --- a/apps/sim/app/layout.tsx +++ b/apps/sim/app/layout.tsx @@ -75,7 +75,7 @@ export default function RootLayout({ children }: { children: React.ReactNode }) // The macOS desktop shell overlays native traffic lights on the // workspace. Mark it before first paint so the sidebar reserves // its inset title-bar lane without a post-hydration layout shift. - var collapsedSidebarWidth = 51; + var collapsedSidebarWidth = 48; try { if (window.simDesktop && /Mac/i.test(navigator.userAgent)) { document.documentElement.setAttribute('data-sim-desktop-title-bar', 'inset'); diff --git a/apps/sim/app/workspace/[workspaceId]/chat/[chatId]/page.tsx b/apps/sim/app/workspace/[workspaceId]/chat/[chatId]/page.tsx index 2c56d1d9837..999260b02e7 100644 --- a/apps/sim/app/workspace/[workspaceId]/chat/[chatId]/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/chat/[chatId]/page.tsx @@ -1,8 +1,13 @@ import { Suspense } from 'react' +import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' +import { notFound } from 'next/navigation' import { getSession } from '@/lib/auth' +import { isChatEnabled } from '@/lib/core/config/env-flags' +import { getQueryClient } from '@/app/_shell/providers/get-query-client' import { Home } from '@/app/workspace/[workspaceId]/home/home' import { HomeFallback } from '@/app/workspace/[workspaceId]/home/home-fallback' +import { prefetchHomeSurface } from '@/app/workspace/[workspaceId]/home/prefetch' import { resolveTableViewsEnabled } from '@/app/workspace/[workspaceId]/home/resolve-table-views-flag' export const metadata: Metadata = { @@ -17,18 +22,30 @@ interface ChatPageProps { } export default async function ChatPage({ params }: ChatPageProps) { + // The layout 404s too, but pages and layouts resolve concurrently — without this + // the prefetch below still fires on its way out. + if (!isChatEnabled) { + notFound() + } + const [{ workspaceId, chatId }, session] = await Promise.all([params, getSession()]) const userId = session?.user?.id - const tableViewsEnabled = await resolveTableViewsEnabled(workspaceId, userId) + const queryClient = getQueryClient() + const [tableViewsEnabled] = await Promise.all([ + resolveTableViewsEnabled(workspaceId, userId), + prefetchHomeSurface(queryClient, workspaceId, userId), + ]) return ( - }> - - + + }> + + + ) } diff --git a/apps/sim/app/workspace/[workspaceId]/files/[fileId]/loading.tsx b/apps/sim/app/workspace/[workspaceId]/files/[fileId]/loading.tsx new file mode 100644 index 00000000000..da22416378e --- /dev/null +++ b/apps/sim/app/workspace/[workspaceId]/files/[fileId]/loading.tsx @@ -0,0 +1,35 @@ +'use client' + +import { File as FileIcon } from '@sim/emcn/icons' +import { noop } from '@sim/utils/helpers' +import { + type BreadcrumbItem, + ResourceChromeFallback, +} from '@/app/workspace/[workspaceId]/components' +import { FOLDERED_RESOURCE_HEADERS } from '@/app/workspace/[workspaceId]/components/folders/foldered-resources' + +const FILES_HEADER = FOLDERED_RESOURCE_HEADERS.file + +/** + * Transcribes the trail the loaded page shows while its record resolves (`loadingBreadcrumbs` in + * `files.tsx`): the root crumb plus a terminal `…`, with no icon on the leaf — so the fallback and + * the page paint the same two crumbs and only the label changes. + */ +const BREADCRUMBS: BreadcrumbItem[] = [ + { label: FILES_HEADER.rootLabel, icon: FileIcon, onClick: noop }, + { label: '…', terminal: true }, +] + +/** + * Fallback for the file DETAIL route. Without it the segment inherits the Files list fallback, which + * paints an options bar and a table header row that a document page does not have — chrome that has + * to be torn down a frame later. A detail page is header + body, so this is the header alone. + * + * Header actions are deliberately omitted: they are a function of the open file (a previewable + * non-markdown file gets a mode toggle, an editable one gets Share/Delete), which is exactly what is + * not yet known here. Chips appearing beside the title reads as content arriving; chips appearing + * and then changing reads as a glitch. + */ +export default function FilesFileLoading() { + return +} diff --git a/apps/sim/app/workspace/[workspaceId]/files/[fileId]/page.tsx b/apps/sim/app/workspace/[workspaceId]/files/[fileId]/page.tsx index 590e94b0816..b4808ec50dd 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/[fileId]/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/files/[fileId]/page.tsx @@ -1,17 +1,48 @@ import { Suspense } from 'react' +import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' +import { getSession } from '@/lib/auth' +import { getQueryClient } from '@/app/_shell/providers/get-query-client' +import FilesFileLoading from '@/app/workspace/[workspaceId]/files/[fileId]/loading' import { Files } from '@/app/workspace/[workspaceId]/files/files' -import FilesLoading from '@/app/workspace/[workspaceId]/files/loading' +import { prefetchFilesBrowser } from '@/app/workspace/[workspaceId]/files/prefetch' export const metadata: Metadata = { title: 'Files', robots: { index: false }, } -export default function FilesFilePage() { +/** + * File detail entry. `Files` resolves the open file out of the workspace file LIST, so this route + * needs the same prefetch its sibling list page does — without it the server can only ever render + * the "resolving the record" spinner, and the real header (breadcrumbs, actions) has to pop in a + * frame later on the client. + * + * It also removes a whole class of hydration mismatch: which branch `Files` renders is decided by + * whether that list is in the cache, so a server render without it and a client render with it + * disagree on the header's markup (a static `…` crumb vs. the file's dropdown crumb). Prefetching + * here makes both sides read the same cache and pick the same branch by construction. + * + * `Files` reads URL query params via nuqs (`useSearchParams` internally), so it must sit under a + * Suspense boundary; the fallback is the detail chrome, matching the route's own `loading.tsx`. + */ +export default async function FilesFilePage({ + params, +}: { + params: Promise<{ workspaceId: string; fileId: string }> +}) { + const [{ workspaceId }, session] = await Promise.all([params, getSession()]) + + const queryClient = getQueryClient() + if (session?.user?.id) { + await prefetchFilesBrowser(queryClient, workspaceId, session.user.id) + } + return ( - }> - - + + }> + + + ) } diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/mermaid-diagram.tsx b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/mermaid-diagram.tsx index 2085b046840..1eb77d5bafe 100644 Binary files a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/mermaid-diagram.tsx and b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/mermaid-diagram.tsx differ diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.ts index 88f54d6c0b7..708bb00444c 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/apply-streamed-markdown.ts @@ -2,10 +2,8 @@ import type { Editor } from '@tiptap/core' import { Node as PMNode } from '@tiptap/pm/model' import { initProseMirrorDoc, updateYFragment, ySyncPluginKey } from '@tiptap/y-tiptap' import * as Y from 'yjs' -import { parseMarkdownToDoc } from '../markdown-parse' - -/** The Yjs fragment name TipTap's Collaboration extension binds to (its default `field`). */ -const COLLAB_DOC_FIELD = 'default' +import { COLLAB_DOC_FIELD } from '@/lib/collab-doc/field' +import { editorNormalForm } from '../markdown-parse' /** * Transaction origin for agent-streamed writes into a live collaborative doc. It is deliberately NOT @@ -63,7 +61,11 @@ export function applyAgentStreamFrame( ): boolean { const binding = ySyncPluginKey.getState(editor.state)?.binding if (!binding) return false - const target = PMNode.fromJSON(editor.schema, parseMarkdownToDoc(body)) + // Through the editor's normal form, like every other writer to the shared document. A frame whose + // body ends on a list, heading, table, or rule parses WITHOUT the editor's trailing paragraph, so + // reconciling toward the bare parse deletes the one the seed put there — and the next client to bind + // writes it back, which is the divergence this normalization exists to prevent. + const target = PMNode.fromJSON(editor.schema, editorNormalForm(body)) let delta: Uint8Array | null = null const capture = (update: Uint8Array, origin: unknown) => { if (origin === AGENT_STREAM_ORIGIN) delta = update diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/collab-streaming-integration.test.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/collab-streaming-integration.test.ts index b8996800923..46183927ec1 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/collab-streaming-integration.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/collab-streaming-integration.test.ts @@ -164,7 +164,7 @@ describe('collab streaming integration — moving pieces', () => { reopened.destroy() }) - it('EMPTY-COLLAPSE ON THE STREAM PATH: an agent body with a huge blank run does not strand empties', () => { + it('EMPTY-BOUND ON THE STREAM PATH: an agent body with a huge blank run does not strand empties', () => { const A = makeCollabEditor() A.editor.commands.setContent(parseMarkdownToDoc('# Title\n\nintro'), { contentType: 'json' }) const session = beginAgentStream(A.editor)! @@ -173,9 +173,11 @@ describe('collab streaming integration — moving pieces', () => { endAgentStream(session) console.log( - `\n[STREAM-COLLAPSE] text=${JSON.stringify(A.editor.state.doc.textContent)} empty=${emptyParas(A.editor)}` + `\n[STREAM-BOUND] text=${JSON.stringify(A.editor.state.doc.textContent)} empty=${emptyParas(A.editor)}` ) - expect(emptyParas(A.editor)).toBe(0) // collapse protects the live streaming path, not just static open + // ~200 blank paragraphs' worth of run arrives; the parse bound caps what reaches the live doc, so the + // streaming path is protected exactly like a static open — no unbounded node explosion in the CRDT. + expect(emptyParas(A.editor)).toBe(20) expect(A.editor.state.doc.textContent).toContain('tail paragraph') }) @@ -197,4 +199,45 @@ describe('collab streaming integration — moving pieces', () => { expect(D.editor.state.doc.textContent).toContain('streamed body') expect(emptyParas(D.editor)).toBe(0) }) + + /** + * Every writer to the shared document has to produce the editor's normal form, or the one that does + * not silently removes what the others add. A frame whose body ends on a list, heading, table, or rule + * parses WITHOUT the trailing paragraph the seed puts there — reconciling toward that bare parse + * deleted it from the live room, and the next client to bind wrote it back, reopening the + * placeholder-vs-live divergence the seed normalization exists to close. + */ + it.each([ + ['ends on a list', ['# T\n\nintro\n\n- a', '# T\n\nintro\n\n- a\n- b']], + ['ends on a heading', ['# T\n\nintro\n\n## Sec', '# T\n\nintro\n\n## Section']], + ['ends on a table', ['# T\n\n| a |\n| --- |\n| 1 |']], + ])('AGENT STREAM KEEPS THE EDITOR NORMAL FORM: %s', (_label, frames) => { + const doc = markdownToYDoc('# T\n\nintro\n\n- seed') + const awareness = new Awareness(doc) + const editor = new Editor({ + extensions: createMarkdownEditorExtensions({ + placeholder: '', + collaboration: { + doc, + awareness, + user: { name: 'U', color: '#fff', clientId: doc.clientID }, + }, + }), + }) + const trailingIsEmptyParagraph = () => { + const fragment = doc.getXmlFragment('default') + const last = fragment.get(fragment.length - 1) + return last instanceof Y.XmlElement && last.nodeName === 'paragraph' && last.length === 0 + } + expect(trailingIsEmptyParagraph()).toBe(true) + + const session = beginAgentStream(editor)! + for (const frame of frames) applyAgentStreamFrame(editor, session, frame) + endAgentStream(session) + + expect(trailingIsEmptyParagraph()).toBe(true) + editor.destroy() + awareness.destroy() + doc.destroy() + }) }) diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.test.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.test.ts index 689c2ee6c77..baa4d97b592 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.test.ts @@ -98,6 +98,50 @@ describe('FileDocProvider', () => { expect(emittedMessages(emit)).toHaveLength(0) }) + /** + * A tab that outlived its room can be offered a DIFFERENT document for the same file. Yjs would union + * the two — the file twice, on both sides, and the relay persists it — and there is no un-merge. So + * the sync must not happen at all; the fatal path leaves the editor read-only on what it already + * shows, and a reload binds a fresh document. + */ + it('refuses to sync into a document it does not recognize', () => { + const { provider, doc, emit, fire } = createProvider(true) + doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.docIdKey, 'doc-original') + const joinError = vi.fn() + provider.on('join-error', joinError) + emit.mockClear() + + fire(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId: 'file-1', docId: 'doc-rebuilt' }) + + expect(emittedMessages(emit)).toHaveLength(0) + expect(provider.synced).toBe(false) + expect(provider.joinError).toMatchObject({ code: 'DOCUMENT_REPLACED', retryable: false }) + expect(joinError).toHaveBeenCalledTimes(1) + }) + + it('syncs when the room holds the document it already has', () => { + const { doc, emit, fire } = createProvider(true) + doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.docIdKey, 'doc-original') + emit.mockClear() + + fire(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId: 'file-1', docId: 'doc-original' }) + + expect(emittedMessages(emit).length).toBeGreaterThan(0) + }) + + it('syncs when either side carries no identity (a fresh doc, or a room seeded before identities)', () => { + const fresh = createProvider(true) + fresh.emit.mockClear() + fresh.fire(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId: 'file-1', docId: 'doc-rebuilt' }) + expect(emittedMessages(fresh.emit).length).toBeGreaterThan(0) + + const unnamedRoom = createProvider(true) + unnamedRoom.doc.getMap(FILE_DOC_SEED.configMap).set(FILE_DOC_SEED.docIdKey, 'doc-original') + unnamedRoom.emit.mockClear() + unnamedRoom.fire(FILE_DOC_EVENTS.JOIN_SUCCESS, { fileId: 'file-1' }) + expect(emittedMessages(unnamedRoom.emit).length).toBeGreaterThan(0) + }) + it('applies a server sync step 2 and becomes synced', () => { const { provider, doc, fire } = createProvider(true) const synced = vi.fn() diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.ts index 84bab0733bc..79884c1e81c 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/file-doc-provider.ts @@ -174,19 +174,11 @@ export class FileDocProvider extends ObservableV2 { */ private handleReadinessDeadline = () => { this.readinessTimer = null - if ((this.synced && this.isSeeded()) || this.fatal || this.disposed) return - const error: JoinFileDocError = { - fileId: this.fileId, - error: 'Realtime document was not ready in time', - code: 'READINESS_TIMEOUT', - retryable: false, - } - this.fatal = true - this.joinError = error - // Drop `synced` so the editor's `synced && seeded` gate stays closed → the fallback renders the - // stored content read-only rather than becoming editable on a doc the server never seeded. - this.setSynced(false) - this.emit('join-error', [error]) + if (this.synced && this.isSeeded()) return + // Dropping `synced` (see {@link failFatally}) is what keeps the editor's `synced && seeded` gate + // closed, so the fallback renders the stored content read-only rather than becoming editable on a + // document the server never seeded. + this.failFatally('Realtime document was not ready in time', 'READINESS_TIMEOUT') } private clearReadinessTimer() { @@ -215,13 +207,55 @@ export class FileDocProvider extends ObservableV2 { /** * Handle the join ack. The server registers the room before acking, so an earlier * send could be dropped — the initial sync + local awareness exchange begins here. + * + * Unless the room holds a DIFFERENT document than ours. Two documents built from the same markdown + * are not the same document to Yjs — their items carry different client ids — so syncing one into the + * other appends the file to itself, on both sides, and the server persists the result. A document is + * rebuilt only when the room AND the shared stream are both gone (a tab that slept through it), which + * is precisely when a stale tab reconnects. There is no way to un-merge afterwards, so the sync never + * happens: take the fatal path, which leaves the editor read-only on the content it already shows. + * A reload binds a fresh document and recovers. */ private handleJoinSuccess = (data: JoinFileDocSuccess) => { if (data.fileId !== this.fileId) return + const local = this.docId() + if (local !== undefined && data.docId !== undefined && data.docId !== local) { + this.failFatally( + 'This document was reloaded on the server; refresh to continue editing', + 'DOCUMENT_REPLACED' + ) + return + } this.sendSyncStep1() this.sendLocalAwareness() } + /** The identity of the document we hold, once the server seed has named one. */ + private docId(): string | undefined { + const docId = this.doc.getMap(FILE_DOC_SEED.configMap).get(FILE_DOC_SEED.docIdKey) + return typeof docId === 'string' ? docId : undefined + } + + /** + * Give up on this document, non-retryably: latch fatal so nothing more is applied or relayed, drop + * `synced` so the editor's gate closes, and surface the rejection to the owner (which falls back to a + * read-only view of the stored content). + */ + private failFatally(message: string, code: string) { + if (this.fatal || this.disposed) return + const error: JoinFileDocError = { + fileId: this.fileId, + error: message, + code, + retryable: false, + } + this.fatal = true + this.joinError = error + this.clearReadinessTimer() + this.setSynced(false) + this.emit('join-error', [error]) + } + /** * Handle a join rejection. A non-retryable rejection (access denied, invalid) * won't succeed on retry, so latch {@link fatal} to stop (re)joining and let the @@ -247,18 +281,7 @@ export class FileDocProvider extends ObservableV2 { */ private handleAccessRevoked = (data: RoomAccessRevokedBroadcast) => { if (data.room?.type !== ROOM_TYPES.WORKSPACE_FILE_DOC || data.room.id !== this.fileId) return - if (this.fatal || this.disposed) return - const error: JoinFileDocError = { - fileId: this.fileId, - error: data.message, - code: 'ACCESS_REVOKED', - retryable: false, - } - this.fatal = true - this.joinError = error - this.clearReadinessTimer() - this.setSynced(false) - this.emit('join-error', [error]) + this.failFatally(data.message, 'ACCESS_REVOKED') } private handleMessage = (data: unknown) => { diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.test.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.test.ts index 39bb68590fc..667638a78ea 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.test.ts @@ -4,11 +4,20 @@ import { describe, expect, it } from 'vitest' import { type CollabReadinessInputs, nextCollabReadiness } from './readiness' +/** An observation, with the healthy defaults filled in so each case states only what it exercises. */ +const at = (input: Partial): CollabReadinessInputs => ({ + synced: false, + seeded: false, + offlineSeed: false, + fatal: false, + ...input, +}) + /** Drive a sequence of observations through the latch, returning the readiness at each step. */ -function run(steps: CollabReadinessInputs[]): boolean[] { +function run(steps: Partial[]): boolean[] { let syncedOnce = false - return steps.map((input) => { - const next = nextCollabReadiness(syncedOnce, input) + return steps.map((step) => { + const next = nextCollabReadiness(syncedOnce, at(step)) syncedOnce = next.syncedOnce return next.ready }) @@ -16,21 +25,27 @@ function run(steps: CollabReadinessInputs[]): boolean[] { describe('nextCollabReadiness', () => { it('is not ready before syncing or seeding', () => { - const { syncedOnce, ready } = nextCollabReadiness(false, { - synced: false, - seeded: false, - offlineSeed: false, - }) + const { syncedOnce, ready } = nextCollabReadiness( + false, + at({ + synced: false, + seeded: false, + offlineSeed: false, + }) + ) expect(syncedOnce).toBe(false) expect(ready).toBe(false) }) it('is not ready when synced but not yet seeded', () => { - const { syncedOnce, ready } = nextCollabReadiness(false, { - synced: true, - seeded: false, - offlineSeed: false, - }) + const { syncedOnce, ready } = nextCollabReadiness( + false, + at({ + synced: true, + seeded: false, + offlineSeed: false, + }) + ) expect(syncedOnce).toBe(true) // latched expect(ready).toBe(false) // waits for the seed }) @@ -50,11 +65,14 @@ describe('nextCollabReadiness', () => { it('opens even if the seed lands before we ever observed synced (server seed proves a sync)', () => { // If the flap beat our first observation, the seed flag alone (not the offline fallback) proves a // completed sync happened. - const { syncedOnce, ready } = nextCollabReadiness(false, { - synced: false, - seeded: true, - offlineSeed: false, - }) + const { syncedOnce, ready } = nextCollabReadiness( + false, + at({ + synced: false, + seeded: true, + offlineSeed: false, + }) + ) expect(syncedOnce).toBe(true) expect(ready).toBe(true) }) @@ -74,4 +92,33 @@ describe('nextCollabReadiness', () => { ]) expect(readiness).toEqual([true, true]) }) + /** + * The reported bug. A brand-new file syncs EMPTY (latching `syncedOnce`), its server seed never + * lands, and the readiness deadline fires: the provider goes fatal and drops `synced` precisely so + * this gate closes. The offline fallback then seeds locally — and the sticky latch used to re-open + * the gate on that, handing back an editable editor bound to a document the provider had abandoned. + * Every keystroke was dropped (the provider ignores frames and never rejoins) and client autosave + * stayed off (collaboration is nominally on), so the edits vanished on reload with no error shown. + */ + it('stays read-only after the readiness deadline goes fatal, even though a sync was latched', () => { + const readiness = run([ + { synced: false }, + { synced: true }, // initial EMPTY sync — latches syncedOnce + { synced: false, fatal: true }, // deadline: provider drops synced and gives up + { seeded: true, offlineSeed: true, fatal: true }, // fallback seeds locally + ]) + expect(readiness).toEqual([false, false, false, false]) + }) + + /** + * The same revocation on an ALREADY-ready doc: access is withdrawn mid-session, the provider goes + * fatal, and readiness must be taken back rather than left latched open. + */ + it('revokes readiness when a live document turns fatal', () => { + const readiness = run([ + { synced: true, seeded: true }, // ready + { synced: false, seeded: true, fatal: true }, // access revoked mid-session + ]) + expect(readiness).toEqual([true, false]) + }) }) diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.ts index 343b301b394..9974de6ed43 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/collaboration/readiness.ts @@ -22,16 +22,32 @@ export interface CollabReadinessInputs { seeded: boolean /** Whether the seed flag was set by the offline fallback (no server sync) rather than the server. */ offlineSeed: boolean + /** + * Whether the provider has GIVEN UP on this document — a non-retryable rejection, an access + * revocation, or the readiness deadline lapsing. A fatal provider ignores every inbound frame and + * never rejoins, so nothing typed after this point reaches the server. + */ + fatal: boolean } /** * Pure transition for the readiness latch. `syncedOnce` is the sticky prior state — pass the returned * `syncedOnce` back in on the next call. `ready` is whether the doc is synced-and-seeded. + * + * `fatal` overrides the latch, and that override is the whole reason it is an input. The latch is + * sticky on purpose, but stickiness must not outlive the document: a doc that syncs empty and never + * receives its server seed trips the readiness deadline, and the provider answers by dropping `synced` + * so this gate closes. The latch ignored that — `syncedOnce` was already set by the empty sync — so the + * offline fallback's seed flag re-opened the gate and handed back an EDITABLE editor on a document the + * provider had already abandoned. Nothing typed into it could persist: the provider drops every frame + * and never rejoins, and the client's own autosave stays gated off because collaboration is nominally + * on. The user types, sees no error, and loses the edits on reload. Revoking readiness on `fatal` is + * what makes the fallback what it is documented to be — a READ-ONLY view of the stored content. */ export function nextCollabReadiness( syncedOnce: boolean, input: CollabReadinessInputs ): { syncedOnce: boolean; ready: boolean } { const next = syncedOnce || input.synced || (input.seeded && !input.offlineSeed) - return { syncedOnce: next, ready: next && input.seeded } + return { syncedOnce: next, ready: next && input.seeded && !input.fatal } } diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-fidelity.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-fidelity.ts index 43466f49861..4470187fefa 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-fidelity.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-fidelity.ts @@ -173,10 +173,11 @@ function stripEmptyListItemLines(markdown: string): string { * round-trip ({@link stripEmptyListItemLines}), restores callout markers the serializer * backslash-escapes (`> \[!NOTE\]` → `> [!NOTE]`), and collapses trailing blank lines to a single * newline. Interior blank runs are NOT collapsed here — blank lines inside a fenced code block (or a - * verbatim raw-markdown-snippet) are significant, and a global collapse would corrupt them. Spurious - * interior blank runs between top-level blocks are removed upstream instead, by - * {@link parseMarkdownToDoc} stripping empty paragraphs, so a doc that has been through the editor - * never serializes with an interior blank run outside code in the first place. The table serializer's + * verbatim raw-markdown-snippet) are significant, and a global collapse would corrupt them. An interior + * run between top-level blocks is significant too: it is how an empty paragraph is written, and + * {@link parseMarkdownToDoc} reads exactly the count back out, so collapsing it here would delete the + * document's spacing. Only the TRAILING run is collapsed — it can carry no paragraph (see + * `clampEmptyParagraphs`) and would otherwise churn the file on every save. The table serializer's * spurious surrounding blank lines are trimmed at the source (PipeSafeTable), so no global * leading-newline strip is needed here — avoiding clobbering content that legitimately begins with * whitespace. diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts index 0067ef31f47..94c645fc2e2 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.test.ts @@ -91,45 +91,101 @@ describe('parseMarkdownToDoc (chunked)', () => { expect(splitMarkdownBlocks('\n\n \n')).toEqual([]) }) - // Asserts the collapse documented on `stripEmptyParagraphs` — at document edges, between blocks, for - // one or many blank lines, and around lists. (Blank runs are insignificant in markdown, so a collapsed - // file renders identically everywhere it's viewed; the pathological case is a run of thousands.) - describe('collapses blank-line runs to markdown-standard spacing', () => { - /** Block-type shape of a doc after `parseMarkdownToDoc`, `∅` for any surviving empty paragraph. */ - function shapeOf(md: string): string { - return (parseMarkdownToDoc(md).content ?? []) - .map((n) => (isEmptyPara(n) ? '∅' : n.type)) - .join(',') - } + /** Block-type shape of a doc after `parseMarkdownToDoc`, `∅` for each empty paragraph. */ + function shapeOf(md: string): string { + return (parseMarkdownToDoc(md).content ?? []) + .map((n) => (isEmptyPara(n) ? '∅' : n.type)) + .join(',') + } + // A blank line an author left between two blocks is part of the document, so parse must read back the + // exact count the serializer wrote (`blocks.join('\n\n')` ⇒ an empty paragraph costs TWO blank lines, + // the first separator is free). Getting this wrong is visible: the static placeholder is built from + // markdown while the live collaborative doc is the CRDT, so any drift shows up as the doc reflowing + // its spacing a beat after the file appears. + describe('preserves authored blank lines', () => { it.each([ - ['one blank gap between paragraphs', 'a\n\n\n\nb', 'paragraph,paragraph'], - ['many blank lines between paragraphs', 'a\n\n\n\n\n\n\n\nb', 'paragraph,paragraph'], - ['leading blank lines', '\n\n\n\na', 'paragraph'], - ['leading + interior', '\n\n\na\n\n\n\nb', 'paragraph,paragraph'], - ['blank gap between a heading and text', '# H\n\n\n\ntext', 'heading,paragraph'], - ['blank gap after a tight list', '- a\n- b\n\n\n\ntext', 'bulletList,paragraph'], - ['blank gap before a tight list', 'text\n\n\n\n- a\n- b', 'paragraph,bulletList'], - // Line-ending variants normalize first, so `\r`-only / CRLF blank runs collapse identically. - ['CRLF between blocks', 'a\r\n\r\n\r\n\r\nb', 'paragraph,paragraph'], - ['CR-only (classic Mac) between blocks', 'a\r\r\r\rb', 'paragraph,paragraph'], - ])('collapses to no empty paragraphs: %s', (_label, md, expected) => { + ['single separator — no empty paragraph', 'a\n\nb', 'paragraph,paragraph'], + ['odd blank line is insignificant', 'a\n\n\nb', 'paragraph,paragraph'], + ['one authored blank line', 'a\n\n\n\nb', 'paragraph,∅,paragraph'], + ['three authored blank lines', 'a\n\n\n\n\n\n\n\nb', 'paragraph,∅,∅,∅,paragraph'], + ['leading blank lines', '\n\n\n\na', '∅,∅,paragraph'], + ['leading + interior', '\n\n\na\n\n\n\nb', '∅,paragraph,∅,paragraph'], + ['between a heading and text', '# H\n\n\n\ntext', 'heading,∅,paragraph'], + ['after a tight list', '- a\n- b\n\n\n\ntext', 'bulletList,∅,paragraph'], + ['before a tight list', 'text\n\n\n\n- a\n- b', 'paragraph,∅,bulletList'], + // Line-ending variants normalize first, so `\r`-only / CRLF runs count identically. + ['CRLF between blocks', 'a\r\n\r\n\r\n\r\nb', 'paragraph,∅,paragraph'], + ['CR-only (classic Mac) between blocks', 'a\r\r\r\rb', 'paragraph,∅,paragraph'], + ])('%s', (_label, md, expected) => { + expect(shapeOf(md)).toBe(expected) + }) + + // A loose list's own internal blank lines are absorbed into its merged block, so they stay list + // spacing rather than becoming top-level paragraphs that would split the list in two. + it('a loose list keeps its internal blank lines as one list', () => { + expect(shapeOf('- a\n\n- b\n\n- c')).toBe('bulletList') + }) + + // …but a gap WIDE enough to carry an empty paragraph is a top-level block boundary: the serializer + // only writes one by emitting the two sides as separate blocks, so re-merging them made parse stop + // inverting serialize. That swallowed the paragraph, fused the two blocks, and — because the file + // then never reached a fixpoint — silently opened it READ-ONLY. + it.each([ + ['between two bullet lists', '- a\n\n\n\n- b', 'bulletList,∅,bulletList'], + ['between two blockquotes', '> a\n\n\n\n> b', 'blockquote,∅,blockquote'], + ['before an indented continuation', 'a\n\n\n\n indented', 'paragraph,∅,paragraph'], + ])('a gap that carries a paragraph breaks the merge: %s', (_label, md, expected) => { expect(shapeOf(md)).toBe(expected) }) it('a pathological blank run does not explode into empty paragraph nodes', () => { // The production incident: an agent/paste artifact with a huge blank run became ~1959 empty - // paragraphs baked into the doc. Collapsing on parse neutralizes any such source. + // paragraphs baked into the doc. The run is bounded on parse, so no source can reach that. const body = `Para A${'\n'.repeat(4000)}Para B` const content = parseMarkdownToDoc(body).content ?? [] - expect(content.filter(isEmptyPara).length).toBe(0) - expect(content.map((n) => n.type)).toEqual(['paragraph', 'paragraph']) + expect(content.filter(isEmptyPara).length).toBe(20) + expect(content.length).toBe(22) + }) + + // The per-gap ceiling bounds one run; the realistic artifact shape is a moderate run between EVERY + // paragraph, which scales with file size. Without a document budget an 86KB body produced ~40k empty + // paragraphs — twenty times the incident the per-gap ceiling exists to prevent. + it('many blank runs cannot explode the document either', () => { + const body = `${'x'.padEnd(1)}${`${'\n'.repeat(42)}x`.repeat(2000)}` + const content = parseMarkdownToDoc(body).content ?? [] + expect(content.filter(isEmptyPara).length).toBe(500) + }) + + // The bounds have to be fixpoints too, or a clamped file would churn on every save. + it.each([ + ['one huge run', `Para A${'\n'.repeat(4000)}Para B`], + ['many runs past the document budget', `x${`${'\n'.repeat(42)}x`.repeat(2000)}`], + ])('a bounded document re-serializes to itself: %s', (_label, md) => { + const once = serializeMarkdownBody(md) + expect(serializeMarkdownBody(once)).toBe(once) + }) + + // The whole-document path hands blank runs to @tiptap/markdown, which keeps them after a paragraph + // but swallows them after a heading/ordered list/table. Preserving only some would break the fixpoint + // for the same document, so that path keeps none — consistently zero, which IS a fixpoint. + it.each([ + ['block HTML', '# H\n\n\n\ntext\n\n
x
', 'heading,paragraph,rawHtmlBlock'], + [ + 'a reference definition', + '# H\n\n\n\nsee [y][r]\n\n[r]: https://e.com', + 'heading,paragraph', + ], + ])('a document that must parse whole keeps no empty paragraphs: %s', (_label, md, expected) => { + expect(shapeOf(md)).toBe(expected) + const once = serializeMarkdownBody(md) + expect(serializeMarkdownBody(once)).toBe(once) }) }) - // Regression: a file with blank lines (leading, interior, or trailing) must stay EDITABLE. Collapsing - // blank runs keeps serialize→parse idempotent, so the round-trip-safety probe reaches a fixed point - // instead of flipping the file read-only. + // Regression: a file with blank lines (leading, interior, or trailing) must stay EDITABLE — parse and + // serialize have to agree on the blank count, or the round-trip-safety probe never reaches a fixed + // point and the file silently opens read-only. describe('blank lines stay editable (regression)', () => { it.each([ ['plain paragraph', 'abc\n\n'], @@ -137,15 +193,27 @@ describe('parseMarkdownToDoc (chunked)', () => { ['three trailing newlines', 'hello\n\n\n'], ['two paragraphs', 'para one\n\npara two\n\n'], ['interior blank run + trailing', 'a\n\n\n\nb\n\n'], + ['many interior blank runs', '# T\n\n\n\na\n\n\n\n\n\nb\n\n\n\n- x\n- y\n\n'], + ['leading blank run', '\n\n\n\nabc\n'], + // These regressed to read-only when a gap carrying a paragraph was still merged away: the merge + // fused the two blocks, so the second pass produced different markdown from the first. + ['gap before a list glued to a lead-in line', 'text\n1. one\n\n\n\n- bullet'], + ['gap between two glued list kinds', 'text\n- bullet\n\n\n\n1. one'], + ['gap between two blockquotes after a lead-in', 'text\n> a\n\n\n\n> b'], + [ + 'changelog shape', + '## v2\n\nHighlights:\n1. faster\n2. smaller\n\n\n\n- also: fixed a crash\n', + ], ])('a file with blank lines is round-trip-safe: %s', (_label, md) => { expect(isRoundTripSafe(md)).toBe(true) }) - it('removes only structurally-empty paragraphs — a paragraph with content survives', () => { - // The shape suite above already proves leading/interior/trailing blank runs collapse to zero empty - // paragraphs; this pins the complementary guarantee — a real (non-empty) paragraph is never dropped. + // Trailing empties are the one kind that cannot round-trip: `postProcessSerializedMarkdown` + // collapses trailing blank lines, so keeping them would make the doc differ from its own output. + it('drops trailing empty paragraphs', () => { + expect(shapeOf('abc\n\n')).toBe('paragraph') + expect(shapeOf('abc\n\n\n\n\n\n')).toBe('paragraph') const trailing = parseMarkdownToDoc('abc\n\n').content ?? [] - expect(trailing.at(-1)?.type).toBe('paragraph') expect(isEmptyPara(trailing.at(-1) ?? {})).toBe(false) }) }) @@ -247,16 +315,25 @@ const FUZZ_BLOCKS: Array<(r: () => number) => string> = [ () => 'See [the docs][ref].\n\n[ref]: https://example.com/docs', ] -function buildFuzzDoc(seed: number): string { +/** + * `blankRuns` widens the separator from a single blank line to a run of up to three, so the corpus + * exercises authored spacing. The single-separator corpus structurally could not: every document it + * built was `parts.join('\n\n')`, which is exactly the one gap width that carries no empty paragraph — + * so the whole blank-line design was invisible to the property test that claims to cover any input. + */ +function buildFuzzDoc(seed: number, blankRuns: boolean): string { const r = rng(seed) const count = 2 + Math.floor(r() * 8) const parts: string[] = [] - for (let i = 0; i < count; i++) parts.push(FUZZ_BLOCKS[Math.floor(r() * FUZZ_BLOCKS.length)](r)) - return parts.join('\n\n') + for (let i = 0; i < count; i++) { + if (i > 0) parts.push('\n'.repeat(blankRuns ? 2 + Math.floor(r() * 4) : 2)) + parts.push(FUZZ_BLOCKS[Math.floor(r() * FUZZ_BLOCKS.length)](r)) + } + return parts.join('') } describe('chunked parse — property test over randomized documents', () => { - it('chunked === one-shot for every document, and idempotent for every editable one', () => { + it('chunked === one-shot on single-separator documents, and idempotent for every editable one', () => { const failures: Array<{ seed: number; kind: string }> = [] // Compare modulo trailing whitespace: `parseMarkdownToDoc` strips trailing empty paragraphs (they // can't be serialized stably — postProcess collapses trailing newlines — so keeping them would flip @@ -264,17 +341,48 @@ describe('chunked parse — property test over randomized documents', () => { // intended and invisible after save; interior/leading fidelity is still compared exactly. const trimEnd = (md: string) => md.replace(/\n+$/, '') for (let seed = 1; seed <= 400; seed++) { - const body = buildFuzzDoc(seed) + const body = buildFuzzDoc(seed, false) const chunked = serializeMarkdownBody(body) - // Fidelity is the load-bearing invariant — chunked must never diverge from the whole-document - // parse, for ANY input; idempotency only needs to hold where the doc is editable (raw HTML is - // non-idempotent in the underlying editor regardless of chunking, which is why it opens read-only). + // On documents with no authored blank run the two paths must still agree exactly. They are allowed + // to differ once a gap carries an empty paragraph: the chunked path reconstructs it and the + // whole-document path deliberately keeps none (see `parseMarkdownToDoc`), and only ONE path ever + // runs for a given document. Idempotency is the invariant that must hold for both, and it is + // asserted for every editable document in the blank-run corpus below. if (trimEnd(chunked) !== trimEnd(oneShot(body))) failures.push({ seed, kind: 'fidelity' }) else if (isRoundTripSafe(body) && serializeMarkdownBody(chunked) !== chunked) { failures.push({ seed, kind: 'idempotency' }) } } expect(failures).toEqual([]) - // 400 docs each parsed+serialized twice — generous timeout so it can't flake under parallel load. - }, 30000) + // 400 docs each parsed+serialized twice. Measured ~10s alone; the whole-suite run gives each worker + // a fraction of a core, and at 30s BOTH property tests in this file timed out there while passing + // standalone. Sized off the loaded number, not the isolated one. + }, 60000) + + /** + * Idempotency is what keeps a file editable: `isRoundTripSafe` opens a document read-only unless + * serializing twice is byte-identical. Preserving blank lines put every gap width on that path, and a + * merge rule that swallowed a gap silently flipped ordinary documents (a changelog, a lead-in line + * followed by a list) to read-only. Fuzz the separator width so that class cannot come back. + * + * Gated on `isRoundTripSafe` for the same reason the single-separator test above is: a document the + * probe rejects opens read-only and is never re-serialized, so its instability is contained by design. + * This corpus does surface such documents — a blank run INSIDE a loose list parses to an empty + * paragraph nested in a list item, which `getMarkdown` writes as an indented `' '` marker line rather + * than a blank one, and that does not round-trip. That defect predates blank-line preservation (it + * reproduces identically with the empty-paragraph strip in place) and is only reachable through a gap + * width the old corpus could not generate; the probe correctly holds those files read-only. + */ + it('stays idempotent with authored blank runs of every width', () => { + const failures: Array<{ seed: number; body: string }> = [] + for (let seed = 1; seed <= 400; seed++) { + const body = buildFuzzDoc(seed, true) + if (!isRoundTripSafe(body)) continue + const once = serializeMarkdownBody(body) + if (serializeMarkdownBody(once) !== once) failures.push({ seed, body }) + } + expect(failures).toEqual([]) + // Same budget as the corpus above, for the same reason — this is the second ~10s property test in + // the file, and adding it is what pushed both past 30s under whole-suite parallelism. + }, 60000) }) diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.ts index 6cdeeabf4ae..fd8cb706a34 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/markdown-parse.ts @@ -47,14 +47,64 @@ const FENCE_CLOSE = /^ {0,3}(`{3,}|~{3,})[ \t]*$/ const LIST_MARKER = /^[ ]{0,3}(?:[-*+]|\d+[.)])\s/ const BLOCKQUOTE = /^[ ]{0,3}>/ +/** + * Ceiling on the empty paragraphs one gap may carry. Deliberate spacing is a handful of blank lines; a + * run of thousands is an agent/paste artifact, and baking a node per blank would put thousands of empty + * paragraphs in the document forever (the reported incident: ~1959 nodes from one 4000-newline run). + * Well past any spacing a person types, low enough that no single gap can explode. + */ +const MAX_CONSECUTIVE_EMPTY_PARAGRAPHS = 20 + +/** + * Ceiling on a document's TOTAL empty paragraphs, enforced by {@link boundEmptyParagraphs}. The per-gap + * ceiling alone bounds nothing at document scale — the realistic artifact shape is a moderate blank run + * between every paragraph, not one giant run, and that scales linearly with file size. Generous enough + * that no hand-spaced document reaches it, finite so a machine-generated one cannot grow the node count + * without limit. + */ +const MAX_EMPTY_PARAGRAPHS_PER_DOC = 500 + +/** + * How many empty paragraphs a run of `blankLines` between two blocks carries. + * + * The serializer joins top-level blocks with a blank line (`blocks.join('\n\n')`), so an empty + * paragraph costs TWO blank lines — its own, plus the separator that follows it — while the first + * separator is free. Inverting that join is the whole rule: an interior gap of `b` blank lines carries + * `(b - 1) / 2` empty paragraphs, a leading gap (no preceding block, so no free separator) carries + * `b / 2`, and both round down. A hand-authored odd blank line is insignificant in markdown and + * collapses, exactly as every standard renderer shows it; a gap the editor itself wrote reconstructs + * exactly, which is what makes parse ∘ serialize a fixed point. + * + * The count is computed here rather than delegated to `@tiptap/markdown`, whose own blank-run handling + * is not self-consistent: after a paragraph, list, blockquote, code fence, rule, or image it follows the + * same `(b - 1) / 2`, but after a heading, an ordered list, or a table the token swallows the whole run + * and yields nothing. Delegating would mean a blank line after a heading could never survive a save. + * + * Bounded here as well as in {@link clampEmptyParagraphs} so a pathological run is never materialized + * in the first place — a megabyte of newlines would otherwise allocate half a million throwaway nodes + * on its way to being clamped back down to {@link MAX_CONSECUTIVE_EMPTY_PARAGRAPHS}. + */ +function emptyBlockCount(blankLines: number, leading: boolean): number { + const count = Math.floor((blankLines - (leading ? 0 : 1)) / 2) + return Math.max(0, Math.min(count, MAX_CONSECUTIVE_EMPTY_PARAGRAPHS)) +} + /** * Split a markdown body into top-level blocks that can each be parsed independently and reassembled - * without changing meaning. Blank lines separate candidate groups (fenced code blocks stay atomic), - * then adjacent groups are merged back together whenever they could form one logical block: any - * indented (continuation) group, and consecutive list/blockquote groups (which would otherwise be a - * single loose list/quote). Merging is intentionally conservative — over-merging only yields a larger - * chunk, whereas under-merging would shatter a structure — and every block is parsed by - * `@tiptap/markdown`'s own lexer, so block boundaries always match the parser. + * (by `join('\n\n')`) without changing meaning. Blank lines separate candidate groups (fenced code + * blocks stay atomic), then adjacent groups are merged back together whenever they could form one + * logical block: any indented (continuation) group, and consecutive list/blockquote groups (which + * would otherwise be a single loose list/quote). Merging is intentionally conservative — over-merging + * only yields a larger chunk, whereas under-merging would shatter a structure — and every non-empty + * block is parsed by `@tiptap/markdown`'s own lexer, so block boundaries always match the parser. + * + * An EMPTY string in the result is a blank line the author left between two blocks — the exact inverse + * of the serializer's block join (see {@link emptyBlockCount}), so a document's deliberate spacing + * survives the round-trip instead of being silently dropped. {@link parseMarkdownToDoc} turns each into + * an empty paragraph; a run is bounded there by {@link clampEmptyParagraphs}. Gaps are measured before + * merging, so blank lines absorbed INTO a merged block (a loose list's own internal spacing) never + * become paragraphs — only gaps between the final top-level blocks do. Trailing blank lines carry + * nothing: the serializer collapses them to a single newline, so keeping them would never round-trip. * * The indent-merge rule is load-bearing for fenced code indented past 3 spaces (e.g. inside a list * item): {@link FENCE_OPEN} only tracks fences at the document margin, so a nested fence's interior @@ -67,10 +117,17 @@ export function splitMarkdownBlocks(body: string): string[] { // block (defeating the chunker). The editor normalizes `\r` on parse anyway, so meaning is unchanged. const lines = body.replace(/\r\n?/g, '\n').split('\n') const groups: string[] = [] + /** Blank lines immediately preceding `groups[i]`, parallel to it. */ + const gaps: number[] = [] + let blanks = 0 let current: string[] = [] let fence: string | null = null const flush = () => { - if (current.length > 0) groups.push(current.join('\n')) + if (current.length > 0) { + groups.push(current.join('\n')) + gaps.push(blanks) + blanks = 0 + } current = [] } for (const line of lines) { @@ -87,7 +144,9 @@ export function splitMarkdownBlocks(body: string): string[] { continue } if (line.trim() === '') { + // Flush BEFORE counting: `blanks` is the gap that preceded the group being closed here. flush() + blanks++ continue } current.push(line) @@ -97,18 +156,33 @@ export function splitMarkdownBlocks(body: string): string[] { // Build continuation runs and join each once — concatenating onto the growing block per group would be // O(n²) for one long loose list. A group continues the run when indented, or when its first line and the // group open the same marker kind (list or blockquote) — i.e. they form one loose list/quote. - const runs: string[][] = [] - for (const group of groups) { - const head = runs.length > 0 ? runs[runs.length - 1][0] : null + const runs: Array<{ empties: number; parts: string[] }> = [] + for (let index = 0; index < groups.length; index++) { + const group = groups[index] + const previous = runs.length > 0 ? runs[runs.length - 1] : null + const head = previous?.parts[0] ?? null + // A gap wide enough to carry an empty paragraph IS a top-level block boundary: the serializer only + // writes one by emitting the two sides as separate blocks, so merging across it swallowed the + // paragraph AND fused the two blocks (`- a` ∅ `- b` became one list, `> a` ∅ `> b` one quote, an + // indented continuation absorbed the gap). Parse then stopped inverting serialize, so the file never + // reached a fixpoint and silently opened READ-ONLY on the next open. + const empties = emptyBlockCount(gaps[index], index === 0) const continues = head !== null && + empties === 0 && (/^\s/.test(group) || (LIST_MARKER.test(head) && LIST_MARKER.test(group)) || (BLOCKQUOTE.test(head) && BLOCKQUOTE.test(group))) - if (continues) runs[runs.length - 1].push(group) - else runs.push([group]) + if (continues) previous?.parts.push(group) + else runs.push({ empties, parts: [group] }) } - return runs.map((run) => run.join('\n\n')) + + const blocks: string[] = [] + for (const run of runs) { + for (let n = run.empties; n > 0; n--) blocks.push('') + blocks.push(run.parts.join('\n\n')) + } + return blocks } /** @@ -121,11 +195,18 @@ export function splitMarkdownBlocks(body: string): string[] { * Documents whose constructs span blocks ({@link NON_CHUNKABLE}) parse whole, and any failure falls * back to a single whole-document parse, so correctness never depends on the splitter. * - * Runs of blank lines take the fast chunked path too: the chunker parses each block stripped of the - * blank lines between them, which drops the empty paragraphs `@tiptap/markdown` reconstructs from a - * blank run — exactly what {@link stripEmptyParagraphs} does to the whole-parse output anyway. A blank - * run between blocks is insignificant in markdown, so collapsing it is the intended normalization (see - * {@link stripEmptyParagraphs}), and both parse paths converge on the same empty-paragraph-free result. + * A blank line the author left between two blocks is part of the document, not noise: the chunker hands + * it back as an empty block (see {@link splitMarkdownBlocks}) and it becomes an empty paragraph here, so + * the editor renders the spacing that is actually in the file — on the very first paint, with no reflow + * once a collaborative doc settles. + * + * The whole-document path CANNOT do that. It hands blank runs to `@tiptap/markdown`, whose handling is + * not self-consistent (see {@link emptyBlockCount}), so a blank line there survives after a paragraph but + * is swallowed after a heading, an ordered list, or a table. Preserving it on only some of those would + * make parse stop inverting serialize for the same document — the file would never reach a fixpoint and + * would open read-only. So that path keeps NO empty paragraphs: consistently zero is a fixpoint, and a + * document whose spacing cannot be represented is better rendered the way every other markdown renderer + * shows it than rendered one way and saved another. */ export function parseMarkdownToDoc(body: string): JSONContent { const manager = markdownManager() @@ -133,22 +214,23 @@ export function parseMarkdownToDoc(body: string): JSONContent { // the chunker and parser do — a classic `\r`-only body would otherwise slip past the reference-def / // block-HTML guard and be chunked, shattering a construct that must parse whole. const normalized = body.replace(/\r\n?/g, '\n') - let doc: JSONContent - if (NON_CHUNKABLE.test(normalized)) { - doc = manager.parse(normalized) - } else { - try { - const content: JSONContent[] = [] - for (const block of splitMarkdownBlocks(normalized)) { - // `MarkdownManager.parse` always returns a doc node with a `content` array; spread its blocks. - content.push(...(manager.parse(block).content ?? [])) + if (NON_CHUNKABLE.test(normalized)) return boundEmptyParagraphs(manager.parse(normalized), 0) + try { + const content: JSONContent[] = [] + for (const block of splitMarkdownBlocks(normalized)) { + // An empty block is the chunker's marker for an authored blank line, and + // `MarkdownManager.parse('')` yields a doc with no blocks — so materialize the node directly. + if (block === '') { + content.push({ type: 'paragraph' }) + continue } - doc = { type: 'doc', content } - } catch { - doc = manager.parse(normalized) + // `MarkdownManager.parse` always returns a doc node with a `content` array; spread its blocks. + content.push(...(manager.parse(block).content ?? [])) } + return boundEmptyParagraphs({ type: 'doc', content }, MAX_EMPTY_PARAGRAPHS_PER_DOC) + } catch { + return boundEmptyParagraphs(manager.parse(normalized), 0) } - return stripEmptyParagraphs(doc) } /** An empty paragraph node — the shape a blank line reconstructs to (no content, or `content: []`). */ @@ -157,26 +239,72 @@ function isEmptyParagraph(node: JSONContent): boolean { } /** - * Drop ALL top-level empty paragraphs from a parsed doc — leading, interior, and trailing. In markdown - * a run of blank lines between blocks is insignificant (CommonMark collapses it), so `@tiptap/markdown` - * reconstructing each blank as an empty paragraph node is not fidelity: it makes the editor render the - * file differently from every standard renderer (GitHub, the download, our own static preview), and a - * pathological blank run (an agent/paste artifact) explodes into thousands of empty nodes that persist - * forever and reflow the doc on open. Collapsing them here keeps normal single-blank-line block spacing - * while removing the spurious gaps, and stays idempotent so the round-trip-safety probe still passes: a - * doc parsed this way has no empty paragraphs, so re-serializing it never re-emits an interior blank run - * (the serializer is intentionally left alone — a blank line inside a fenced code block IS significant), - * and a second parse is a fixed point. Only TOP-LEVEL paragraphs are touched, so blank lines that carry - * meaning inside a construct (e.g. a loose list) are left to the block parser. TipTap re-adds its own - * trailing filler paragraph on `setContent`, so the editor still has a place to type. + * Bound the top-level empty paragraphs of a parsed doc to `budget` in total, and drop trailing ones + * entirely. `budget` is 0 for the whole-document path, which cannot represent them at all. + * + * The per-gap ceiling in {@link emptyBlockCount} bounds one run; this bounds the DOCUMENT. Without it the + * ceiling buys nothing against the shape a real artifact takes — an export that puts a moderate blank run + * between every paragraph, rather than one giant run. Measured before this budget existed: an 86KB body + * of `x` + 42 newlines produced 39,980 empty paragraphs, twenty times the incident the ceiling cites. + * + * Trailing empties cannot round-trip — `postProcessSerializedMarkdown` collapses trailing blank lines to + * a single newline, so a trailing empty paragraph would be re-serialized away and the doc would differ + * from its own output, flipping the file read-only. Dropping them here is what keeps the probe stable + * (TipTap re-adds its own trailing filler paragraph on `setContent`, so there is still somewhere to + * type). Interior and leading empties DO round-trip exactly, so they are kept. + * + * Only TOP-LEVEL paragraphs are considered — blank lines that carry meaning inside a construct (a loose + * list, a blockquote) live below the doc root and belong to the block parser. Returns the doc untouched, + * with no array copy, when nothing needs bounding (the overwhelmingly common case). */ -function stripEmptyParagraphs(doc: JSONContent): JSONContent { +function boundEmptyParagraphs(doc: JSONContent, budget: number): JSONContent { const content = doc.content if (!content || content.length === 0) return doc - // The dominant (chunked) parse already emits no top-level empty paragraphs, so scan before allocating: - // return the doc untouched — no array copy — unless there is actually something to strip. + // Most documents carry no empty paragraph at all, so scan before allocating anything. if (!content.some(isEmptyParagraph)) return doc - return { ...doc, content: content.filter((node) => !isEmptyParagraph(node)) } + let end = content.length + while (end > 0 && isEmptyParagraph(content[end - 1])) end-- + const kept: JSONContent[] = [] + let remaining = budget + for (let index = 0; index < end; index++) { + const node = content[index] + if (!isEmptyParagraph(node)) { + kept.push(node) + continue + } + if (remaining > 0) { + remaining-- + kept.push(node) + } + } + return kept.length === content.length ? doc : { ...doc, content: kept } +} + +/** + * The markdown parse in the form the EDITOR settles on — the only shape that may enter the shared + * document. + * + * ProseMirror appends an empty paragraph to any document that does not end in one, so a parse ending on + * a list, heading, table, or rule is NOT what a bound editor holds. Seeding the CRDT with the + * un-normalized shape means the first client to bind writes that paragraph back into the SHARED + * document — and because a trailing blank line does not survive serialization + * (`postProcessSerializedMarkdown` collapses it) the file never records it, so nothing reconciles the + * two and a client that seeds without seeing another's contribution adds one more. Measured on a + * heavily-reopened document: 18 stacked empty paragraphs in the live doc against the placeholder's 1 — + * the pane growing several hundred pixels the instant the live editor took over. + * + * Opt-in rather than folded into {@link parseMarkdownToDoc}, because only a writer to the SHARED + * document has to agree with the editor. Every other consumer of the parse (paste, the round-trip + * probe, the read-only placeholder) is rendered through a real editor that applies this itself, and + * baking it into the parse changes what those surfaces assert. Every CRDT writer — the seed, the agent + * merge, and the streaming frame reconciler — must go through here, or the one that does not silently + * removes what the others add. + */ +export function editorNormalForm(markdown: string): JSONContent { + const json = parseMarkdownToDoc(markdown) + const content = json.content ?? [] + if (content[content.length - 1]?.type === 'paragraph') return json + return { ...json, content: [...content, { type: 'paragraph' }] } } /** diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-editor.tsx b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-editor.tsx index 4bc6285eedb..0a53b21cce5 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-editor.tsx +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/rich-markdown-editor/rich-markdown-editor.tsx @@ -721,6 +721,15 @@ export function LoadedRichMarkdownEditor({ * is latched, so a fatal rejection that fired before this subscription is not missed. */ useEffect(() => { + /** + * Readiness is a protocol fact, never a timing guess: the relay attaches a client only once its + * room holds the whole document (it awaits the shared-stream catch-up and the server seed before + * answering a join), so a completed sync IS the finished document and revealing on it cannot show + * an intermediate state. This deliberately does NOT wait for the document to "stop moving" — a + * quiet-frame gate was tried and it is unsound in both directions: it delays the reveal of a + * document that was already correct, and it opens mid-flight anyway whenever the updates arrive + * more than a frame apart (which is what a remote Redis and a long room history produce). + */ const setReady = (ready: boolean) => { // Child-local: gates editability (a user must never type into an unsynced/unseeded doc). setCollabReady(ready) @@ -766,12 +775,21 @@ export function LoadedRichMarkdownEditor({ const report = () => { const synced = provider.synced const seeded = config.get(FILE_DOC_SEED.flag) === true - const next = nextCollabReadiness(syncedOnce, { synced, seeded, offlineSeed }) + // `joinError` is latched ONLY on the provider's fatal paths (non-retryable rejection, access + // revocation, readiness deadline), so it is exactly "this document is abandoned". + const fatal = provider.joinError !== null + const next = nextCollabReadiness(syncedOnce, { synced, seeded, offlineSeed, fatal }) syncedOnce = next.syncedOnce setReady(next.ready) } + /** + * Re-report unconditionally, not just when the fallback seeds. A fatal that arrives on an ALREADY + * seeded doc (access revoked mid-session) leaves `seedFromLoaded` a no-op, so nothing else would + * fire an observer and the editor would stay editable on a document the provider has abandoned. + */ const onJoinError = (error: JoinFileDocError) => { if (error.retryable === false) seedFromLoaded() + report() } // A server edit that changes ONLY the frontmatter (e.g. copilot) updates the config map but not diff --git a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/use-editable-file-content.ts b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/use-editable-file-content.ts index f8dac76bbdd..db6206c43b1 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/use-editable-file-content.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/components/file-viewer/use-editable-file-content.ts @@ -178,7 +178,15 @@ export function useEditableFileContent({ file.id, file.key, GENERATED_SOURCE_FILE_TYPES.has(file.type), - { refetchInterval: reconcileRefetchInterval } + { + refetchInterval: reconcileRefetchInterval, + // `canAutosave: false` on this surface means a server-side owner holds durability — the + // collaborative relay, which projects the live document to markdown itself and merges + // external writes INTO that document. There is nothing a focus refetch of the durable bytes + // can teach the editor that the shared document does not already have; all it does is + // re-read a storage key the relay's last save has already rotated away from. + refetchOnWindowFocus: canAutosave, + } ) /** diff --git a/apps/sim/app/workspace/[workspaceId]/files/page.tsx b/apps/sim/app/workspace/[workspaceId]/files/page.tsx index 389b4d17ded..11dc9fd450e 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/files/page.tsx @@ -23,9 +23,7 @@ export default async function FilesPage({ params }: { params: Promise<{ workspac const [{ workspaceId }, session] = await Promise.all([params, getSession()]) const queryClient = getQueryClient() - if (session?.user?.id) { - await prefetchFilesBrowser(queryClient, workspaceId, session.user.id) - } + await prefetchFilesBrowser(queryClient, workspaceId, session?.user?.id) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts index 250a6e5f713..dd08f5fc925 100644 --- a/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/files/prefetch.ts @@ -1,53 +1,58 @@ import type { QueryClient } from '@tanstack/react-query' +import { listWorkspaceFileFoldersContract } from '@/lib/api/contracts/workspace-file-folders' import { listWorkspaceFileFolders } from '@/lib/uploads/contexts/workspace/workspace-file-folder-manager' -import { listWorkspaceFilesWithShares } from '@/lib/workspace-files/queries' import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' import { prefetchResourceListChrome } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome' +import { seedWorkspaceFiles } from '@/app/workspace/[workspaceId]/lib/seed-workspace-files' import { WORKSPACE_FILE_FOLDERS_STALE_TIME, workspaceFileFolderKeys, } from '@/hooks/queries/workspace-file-folders' -import { - WORKSPACE_FILES_LIST_STALE_TIME, - workspaceFilesKeys, -} from '@/hooks/queries/workspace-files' /** - * Prefetches everything the Files browser needs to paint a complete, correctly-ordered - * first frame: workspace files, file folders, and (via {@link prefetchResourceListChrome}) - * the pinned ids that drive row order plus the members behind the Owner column — - * under the same query keys their client hooks (`useWorkspaceFiles`, - * `useWorkspaceFileFolders`) use (scope `active`), so the browser paints - * populated on first render. + * Prefetches what the Files browser needs on top of the workspace layout's own prefetch, so the + * first frame is complete and correctly ordered: file folders, and (via + * {@link prefetchResourceListChrome}) the pinned ids that drive row order plus the members behind + * the Owner column — under the same query keys their client hooks (`useWorkspaceFileFolders`) use + * (scope `active`), so the browser paints populated on first render. + * + * The file list is seeded here rather than in the layout so only the routes that render it pay for + * it. See {@link seedWorkspaceFiles} for why a large workspace seeds nothing at all. * - * Files and folders read the data layer; both payloads are shaped to their route contract so - * a hydrated entry matches a client fetch. Everything else still goes through its route — - * see {@link prefetchInternalJson}. + * Folders and the chrome reads all go through the data layer, shaped to their route contracts so a + * hydrated entry matches a client fetch. * - * Those two reads carry no authorization of their own, so the viewer is proved first. This - * reuses the layout's `cache`d host-context lookup rather than re-deriving the permission, - * so it costs no additional queries; a viewer without access caches nothing and the client - * fetch reaches the route for the real 403. + * That read carries no authorization of its own, so the viewer is proved first. This reuses the + * layout's `cache`d host-context lookup rather than re-deriving the permission, so it costs no + * additional queries; a viewer without access caches nothing and the client fetch reaches the + * route for the real 403. */ export async function prefetchFilesBrowser( queryClient: QueryClient, workspaceId: string, - userId: string + userId: string | undefined ): Promise { + if (!userId) return const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) if (!hostContext) return await Promise.all([ - queryClient.prefetchQuery({ - queryKey: workspaceFilesKeys.list(workspaceId, 'active'), - queryFn: () => listWorkspaceFilesWithShares(workspaceId, 'active'), - staleTime: WORKSPACE_FILES_LIST_STALE_TIME, - }), queryClient.prefetchQuery({ queryKey: workspaceFileFolderKeys.list(workspaceId, 'active'), - queryFn: () => listWorkspaceFileFolders(workspaceId, { scope: 'active' }), + /** + * Parsed through the route's own response schema rather than seeded raw. The + * manager's record type and `workspaceFileFolderSchema` are two independent + * declarations that happen to agree today; without this parse, adding a column + * to one silently seeds a shape a client fetch would have stripped — the exact + * divergence that put ISO strings under `workspaceFilesKeys.list`. + */ + queryFn: async () => { + const folders = await listWorkspaceFileFolders(workspaceId, { scope: 'active' }) + return listWorkspaceFileFoldersContract.response.schema.shape.folders.parse(folders) + }, staleTime: WORKSPACE_FILE_FOLDERS_STALE_TIME, }), - prefetchResourceListChrome(queryClient, workspaceId, 'file'), + prefetchResourceListChrome(queryClient, workspaceId, 'file', userId), + seedWorkspaceFiles(queryClient, workspaceId), ]) } diff --git a/apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-tabs/resource-tab-controls.ts b/apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-tabs/resource-tab-controls.ts index cf8f02ac5ab..5d5697d7be9 100644 --- a/apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-tabs/resource-tab-controls.ts +++ b/apps/sim/app/workspace/[workspaceId]/home/components/mothership-view/components/resource-tabs/resource-tab-controls.ts @@ -7,13 +7,18 @@ export const RESOURCE_TAB_ICON_CLASS = 'size-[16px] text-[var(--text-icon)]' /** Shared geometry for the resource header and controls positioned over it. */ export const RESOURCE_HEADER_CLASSES = { layout: - '[--resource-header-controls-height:43px] [--resource-header-end-inset:16px] [--resource-header-fixed-reserve:54px]', + '[--resource-header-controls-height:43px] [--resource-header-end-inset:16px] [--resource-header-fixed-reserve:54px] [--resource-header-toggle-size:30px]', bar: 'h-[calc(var(--resource-header-controls-height)_+_1px)]', - controls: 'h-[var(--resource-header-controls-height)]', - contentTop: 'top-[8.5px]', + overlay: 'absolute top-0 flex h-[var(--resource-header-controls-height)] items-center', startPadding: 'pl-[var(--resource-header-end-inset)]', endPadding: 'pr-[var(--resource-header-fixed-reserve)]', endPosition: 'right-[var(--resource-header-end-inset)]', - adjacentEndPosition: 'right-[var(--resource-header-fixed-reserve)]', + /** + * Sits a control 1px clear of the overlaid 30px collapse toggle — the same + * chip-to-chip gap the sidebar header cluster uses (`gap-[1px]`), so the + * credits chip and the toggle read as one cluster across both surfaces. + */ + adjacentEndPosition: + 'right-[calc(var(--resource-header-end-inset)_+_var(--resource-header-toggle-size)_+_1px)]', emptyAddOffset: '-translate-x-1.5', } as const diff --git a/apps/sim/app/workspace/[workspaceId]/home/components/user-input/user-input.tsx b/apps/sim/app/workspace/[workspaceId]/home/components/user-input/user-input.tsx index d8bf4142639..1b2c6467340 100644 --- a/apps/sim/app/workspace/[workspaceId]/home/components/user-input/user-input.tsx +++ b/apps/sim/app/workspace/[workspaceId]/home/components/user-input/user-input.tsx @@ -46,6 +46,16 @@ export type { FileAttachmentForApi } from '@/app/workspace/[workspaceId]/home/ty const logger = createLogger('UserInput') +/** + * Whether the element is somewhere the user could be typing. Focusing the composer on mount + * must not steal focus from another field, but may take it from a link or button — opening a + * chat leaves the sidebar link focused, and the composer should win. + */ +function isTextEntry(element: HTMLElement): boolean { + const tag = element.tagName + return tag === 'INPUT' || tag === 'TEXTAREA' || tag === 'SELECT' || element.isContentEditable +} + interface UserInputProps { defaultValue?: string draftScopeKey?: string @@ -453,12 +463,10 @@ const UserInputImpl = forwardRef(function UserI useEffect(() => { const raf = window.requestAnimationFrame(() => { + if (!document.hasFocus()) return const active = document.activeElement - const pageHasFocus = document.hasFocus() - const hasNeutralFocus = active === document.body || active === document.documentElement - if (pageHasFocus && hasNeutralFocus) { - textareaRef.current?.focus() - } + if (active instanceof HTMLElement && isTextEntry(active)) return + textareaRef.current?.focus() }) return () => window.cancelAnimationFrame(raf) }, [textareaRef]) diff --git a/apps/sim/app/workspace/[workspaceId]/home/home.tsx b/apps/sim/app/workspace/[workspaceId]/home/home.tsx index dd5d02da37d..eba97d8274e 100644 --- a/apps/sim/app/workspace/[workspaceId]/home/home.tsx +++ b/apps/sim/app/workspace/[workspaceId]/home/home.tsx @@ -283,21 +283,32 @@ export function Home({ chatId, userName, userId, tableViewsEnabled }: HomeProps) setIsResourceCollapsed(true) }, [clearWidth]) + const clearResourceActivity = useCallback((resourceId: string) => { + setResourceActivityIds((current) => { + if (!current.has(resourceId)) return current + const next = new Set(current) + next.delete(resourceId) + return next + }) + }, []) + + const expandResource = () => { + userOwnsResourceViewRef.current = true + const activeResourceId = activeResourceParamRef.current + if (activeResourceId) clearResourceActivity(activeResourceId) + setIsResourceCollapsed(false) + } + const selectResourceFromUser = useCallback( (resourceId: string) => { userOwnsResourceViewRef.current = true - setResourceActivityIds((current) => { - if (!current.has(resourceId)) return current - const next = new Set(current) - next.delete(resourceId) - return next - }) + clearResourceActivity(resourceId) if (effectiveActiveResourceIdRef.current === resourceId) return effectiveActiveResourceIdRef.current = resourceId activeResourceParamRef.current = resourceId setActiveResourceId(resourceId) }, - [setActiveResourceId] + [setActiveResourceId, clearResourceActivity] ) const addResourceFromUser = useCallback( @@ -581,9 +592,17 @@ export function Home({ chatId, userName, userId, tableViewsEnabled }: HomeProps) {showEmptyState && (
@@ -690,34 +709,14 @@ export function Home({ chatId, userName, userId, tableViewsEnabled }: HomeProps)
+ + +

Center the canvas on a block when you click it

+ + + +
+ +
+
diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/enrichment-details/enrichment-details.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/enrichment-details/enrichment-details.tsx index c8995487beb..677cd5eedf1 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/enrichment-details/enrichment-details.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/enrichment-details/enrichment-details.tsx @@ -7,10 +7,10 @@ import type { EnrichmentProviderOutcome, EnrichmentRunDetail } from '@/lib/table import { adjustBgForContrast, getBlockIconAndColor, - iconColorClass, } from '@/app/workspace/[workspaceId]/logs/components/log-details/utils' import { useLogDetailsResize } from '@/app/workspace/[workspaceId]/logs/hooks' import { formatDate } from '@/app/workspace/[workspaceId]/logs/utils' +import { getTileIconColorClass } from '@/blocks/icon-color' import { useEnrichmentDetail } from '@/hooks/queries/tables' import { formatCost } from '@/providers/utils' import { useLogDetailsUIStore } from '@/stores/logs/store' @@ -255,7 +255,9 @@ function EnrichmentDetailsContent({ style={{ background: bgColor }} > {ProviderIcon && ( - + )}
diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/workflow-sidebar/workflow-sidebar.tsx b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/workflow-sidebar/workflow-sidebar.tsx index 9dbb32fce28..d2d486561c5 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/workflow-sidebar/workflow-sidebar.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/components/workflow-sidebar/workflow-sidebar.tsx @@ -1,6 +1,5 @@ 'use client' -import type React from 'react' import { useMemo, useState } from 'react' import { Button, @@ -18,7 +17,7 @@ import { Tooltip, toast, } from '@sim/emcn' -import { ArrowLeft, ChevronDown, Repeat, Split, SquareArrowUpRight, X } from '@sim/emcn/icons' +import { ArrowLeft, ChevronDown, SquareArrowUpRight, X } from '@sim/emcn/icons' import { toError } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' import { useMutation, useQueryClient } from '@tanstack/react-query' @@ -57,8 +56,7 @@ import { RequiredLabel, } from '@/app/workspace/[workspaceId]/tables/[tableId]/components/sidebar-fields' import { PreviewWorkflow } from '@/app/workspace/[workspaceId]/w/components/preview' -import { getBlock } from '@/blocks' -import { getTileIconColorClass } from '@/blocks/icon-color' +import { BlockTile } from '@/blocks/block-tile' import { useAddWorkflowGroup, useUpdateColumn, @@ -140,8 +138,6 @@ interface BlockOutputGroup { blockId: string blockName: string blockType: string - blockIcon: string | React.ComponentType<{ className?: string }> - blockColor: string paths: string[] } @@ -164,25 +160,6 @@ function tableColumnTypeToInputType(colType: ColumnDefinition['type'] | undefine return columnTypeById(colType).workflowInputType } -const TagIcon: React.FC<{ - icon: string | React.ComponentType<{ className?: string }> - color: string -}> = ({ icon, color }) => ( -
- {typeof icon === 'string' ? ( - {icon} - ) : ( - (() => { - const IconComponent = icon - return - })() - )} -
-) - /** * Right-edge sidebar for workflow group configuration. Three flows: * - create a new group (workflow + outputs + deps), @@ -468,20 +445,10 @@ export function WorkflowSidebarBody({ for (const f of flat) { let group = groupsByBlockId.get(f.blockId) if (!group) { - const blockConfig = getBlock(f.blockType) - const blockColor = blockConfig?.bgColor || '#2F55FF' - let blockIcon: string | React.ComponentType<{ className?: string }> = f.blockName - .charAt(0) - .toUpperCase() - if (blockConfig?.icon) blockIcon = blockConfig.icon - else if (f.blockType === 'loop') blockIcon = Repeat - else if (f.blockType === 'parallel') blockIcon = Split group = { blockId: f.blockId, blockName: f.blockName, blockType: f.blockType, - blockIcon, - blockColor, paths: [], } groupsByBlockId.set(f.blockId, group) @@ -504,7 +471,11 @@ export function WorkflowSidebarBody({ section: group.blockName, sectionElement: (
- + {group.blockName}
), diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table-event-stream.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table-event-stream.ts index 781f6c76441..c36f87792e8 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table-event-stream.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/hooks/use-table-event-stream.ts @@ -5,6 +5,7 @@ import { toast } from '@sim/emcn' import { createLogger } from '@sim/logger' import { backoffWithJitter } from '@sim/utils/retry' import { useQueryClient } from '@tanstack/react-query' +import { getClientFingerprint } from '@/lib/api/client-id' import type { ActiveDispatch } from '@/lib/api/contracts/tables' import type { RowData, @@ -245,6 +246,30 @@ export function useTableEventStream({ }, ROWS_INVALIDATE_DEBOUNCE_MS) } + /** + * This tab's fingerprint as it appears on a broadcast it caused. Resolved once, asynchronously; + * until it lands `applyEdit` simply takes the refetch path, which is the pre-existing behavior. + */ + let ownFingerprint: string | undefined + void getClientFingerprint().then((fingerprint) => { + ownFingerprint = fingerprint + }) + + /** + * A manual row edit landed. Refetch the rows so the winning last-write value shows live — + * unless this tab is the one that made it. + * + * The signal names its originator only for writes whose mutation hook already applies the + * server's answer to every cached rows query, active or not (single-row create, update, + * delete). For those the refetch is pure duplication: on a scrolled table it re-fetches every + * loaded page, and on delete it races the refetch the hook itself issued. Other tabs see + * someone else's fingerprint and refetch normally; an unattributed edit refetches everywhere. + */ + const applyEdit = (event: Extract): void => { + if (event.originatorId && event.originatorId === ownFingerprint) return + scheduleRowsInvalidate() + } + const applyCell = (event: Extract): void => { void snapshotAndMutateRows(queryClient, tableId, (row) => applyCellEventToRow(row, event), { cancelInFlight: false, @@ -445,9 +470,7 @@ export function useTableEventStream({ else if (entry.event?.kind === 'dispatch') applyDispatch(entry.event) else if (entry.event?.kind === 'job') applyJob(entry.event) else if (entry.event?.kind === 'usageLimitReached') applyUsageLimit(entry.event) - // A collaborator's manual edit: refetch rows (debounced) so the winning - // last-write value shows live, in this client's own wire format. - else if (entry.event?.kind === 'edit') scheduleRowsInvalidate() + else if (entry.event?.kind === 'edit') applyEdit(entry.event) // A collaborator changed the table structure: mirror the local // invalidateTableSchema set — the definition (exact, so rows stay on the // debounce), the run-state + enrichment sibling queries under detail (a group diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.test.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.test.ts index 5945be6c388..146fb11cc23 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.test.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.test.ts @@ -2,6 +2,7 @@ * @vitest-environment node */ import { describe, expect, it } from 'vitest' +import { columnTypeOf } from '@/lib/table/column-types' import { cleanCellValue, dateValueToLocalParts, @@ -157,6 +158,32 @@ describe('cleanCellValue', () => { expect(cleanCellValue('Bug, Bug', column)).toEqual(['opt_a']) expect(cleanCellValue('Nope', column)).toEqual([]) }) + + /** + * The grid writes through a first-party route, which runs the `null` policy — + * a member the paste names that resolves to no option is dropped, and the ones + * that do resolve are kept. Erasing the cell instead would lose two live + * options over one deleted one. The registry pairing is asserted rather than + * described so a helper that stops consulting `salvage` fails here. + */ + it('keeps the members of a partial multiselect paste that still resolve', () => { + const column = { + name: 'tags', + type: 'select', + multiple: true, + options: [ + { id: 'opt_a', name: 'Bug' }, + { id: 'opt_b', name: 'Docs' }, + ], + } as const + + expect(columnTypeOf(column).coerce('Bug, Nope', column)).toEqual({ ok: false }) + expect(columnTypeOf(column).salvage?.('Bug, Nope', column)).toEqual({ + ok: true, + value: ['opt_a'], + }) + expect(cleanCellValue('Bug, Nope', column)).toEqual(['opt_a']) + }) }) describe('formatValueForInput', () => { diff --git a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.ts b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.ts index d4c3fc5b3ce..69f7722d11d 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/[tableId]/utils.ts @@ -19,8 +19,16 @@ export function generateColumnName(columns: ReadonlyArray<{ name: string }>): st } /** - * Coerce a raw input value to the appropriate type for a column. - * Throws on invalid JSON. + * Coerce a value a person typed or pasted into a cell to that column's type. + * Throws on invalid JSON, and answers `null` for everything else the column + * type can read nothing from. + * + * The result is what the server would store for the same value, which is the + * point: the optimistic cache and the row that comes back agree. The grid + * writes through a first-party route, which runs the `null` policy — so a + * refused value falls back to `ColumnTypeDefinition.salvage` here exactly as it + * does there, and a multiselect paste naming one live option and one deleted + * one keeps the live one instead of erasing the cell. */ export function cleanCellValue( value: unknown, @@ -46,8 +54,11 @@ export function cleanCellValue( // Everything else runs the SAME coercion the server will run, so the // optimistic cache holds exactly the value that gets persisted. - const coerced = columnTypeOf(column).coerce(value as JsonValue, column) - return coerced.ok ? coerced.value : null + const columnType = columnTypeOf(column) + const coerced = columnType.coerce(value as JsonValue, column) + if (coerced.ok) return coerced.value + const salvaged = columnType.salvage?.(value as JsonValue, column) + return salvaged?.ok ? salvaged.value : null } /** diff --git a/apps/sim/app/workspace/[workspaceId]/tables/page.tsx b/apps/sim/app/workspace/[workspaceId]/tables/page.tsx index 0e9390a5d95..1e9cb1f0592 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/page.tsx +++ b/apps/sim/app/workspace/[workspaceId]/tables/page.tsx @@ -1,6 +1,7 @@ import { Suspense } from 'react' import { dehydrate, HydrationBoundary } from '@tanstack/react-query' import type { Metadata } from 'next' +import { getSession } from '@/lib/auth' import { getQueryClient } from '@/app/_shell/providers/get-query-client' import TablesLoading from '@/app/workspace/[workspaceId]/tables/loading' import { prefetchTables } from '@/app/workspace/[workspaceId]/tables/prefetch' @@ -17,10 +18,9 @@ export const metadata: Metadata = { * route-level `loading.tsx` covers the navigation/chunk-load transition. */ export default async function TablesPage({ params }: { params: Promise<{ workspaceId: string }> }) { - const { workspaceId } = await params - + const [{ workspaceId }, session] = await Promise.all([params, getSession()]) const queryClient = getQueryClient() - await prefetchTables(queryClient, workspaceId) + await prefetchTables(queryClient, workspaceId, session?.user?.id) return ( diff --git a/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts b/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts index 5a548885511..a937a26e753 100644 --- a/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts +++ b/apps/sim/app/workspace/[workspaceId]/tables/prefetch.ts @@ -1,9 +1,9 @@ import type { QueryClient } from '@tanstack/react-query' -import type { FolderApi } from '@/lib/api/contracts/folders' -import type { TableDefinition } from '@/lib/table' -import { prefetchInternalJson } from '@/app/workspace/[workspaceId]/lib/prefetch-internal-fetch' +import { listTables } from '@/lib/table/service' +import { toTableListItem } from '@/lib/table/wire' +import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' +import { prefetchResourceFolders } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-folders' import { prefetchResourceListChrome } from '@/app/workspace/[workspaceId]/lib/prefetch-resource-list-chrome' -import { FOLDER_LIST_STALE_TIME, folderKeys, mapFolder } from '@/hooks/queries/utils/folder-keys' import { TABLE_LIST_STALE_TIME, tableKeys } from '@/hooks/queries/utils/table-keys' /** @@ -14,33 +14,36 @@ import { TABLE_LIST_STALE_TIME, tableKeys } from '@/hooks/queries/utils/table-ke * only placed correctly relative to the folder rows it sits beside, so * prefetching one without the other still flashes an ungrouped list. * - * Table definitions carry `Date` fields, so the list goes through the - * `/api/table` route and caches the serialized wire shape — see - * {@link prefetchInternalJson}. Folders are mapped with the same `mapFolder` the - * hook applies so the hydrated entry matches a client fetch exactly. + * The list goes through {@link toTableListItem}, the projection `GET /api/table` itself + * returns, because `listTablesContract`'s response schema is a passthrough that neither + * coerces nor strips — the client caches the route's JSON verbatim, so seeding raw rows + * would put `Date` objects and the server-only `metadata` field under that key. + * + * The read carries no authorization of its own, so the viewer is proved first. + * `getWorkspaceHostContextForViewer` resolves the same effective workspace permission the + * route's own check does (both bottom out in `checkWorkspaceAccess`), and it is `cache`d and + * already resolved by the layout for this request, so it costs no additional queries. A viewer + * without access caches nothing and the client fetch reaches the route for the real 403. */ -export async function prefetchTables(queryClient: QueryClient, workspaceId: string): Promise { +export async function prefetchTables( + queryClient: QueryClient, + workspaceId: string, + userId: string | undefined +): Promise { + if (!userId) return + const hostContext = await getWorkspaceHostContextForViewer(workspaceId, userId) + if (!hostContext) return + await Promise.all([ queryClient.prefetchQuery({ queryKey: tableKeys.list(workspaceId, 'active'), queryFn: async () => { - const response = await prefetchInternalJson<{ data: { tables: TableDefinition[] } }>( - `/api/table?workspaceId=${workspaceId}&scope=active` - ) - return response.data.tables + const tables = await listTables(workspaceId, { scope: 'active' }) + return tables.map(toTableListItem) }, staleTime: TABLE_LIST_STALE_TIME, }), - queryClient.prefetchQuery({ - queryKey: folderKeys.list(workspaceId, 'active', 'table'), - queryFn: async () => { - const { folders } = await prefetchInternalJson<{ folders?: FolderApi[] }>( - `/api/folders?workspaceId=${workspaceId}&scope=active&resourceType=table` - ) - return (folders ?? []).map(mapFolder) - }, - staleTime: FOLDER_LIST_STALE_TIME, - }), - prefetchResourceListChrome(queryClient, workspaceId, 'table'), + prefetchResourceFolders(queryClient, workspaceId, 'table', userId), + prefetchResourceListChrome(queryClient, workspaceId, 'table', userId), ]) } diff --git a/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.test.tsx b/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.test.tsx index 188e3a6523f..f2079037598 100644 --- a/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.test.tsx +++ b/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.test.tsx @@ -120,6 +120,22 @@ describe('useUpgradeState', () => { }) }) + it('shows checkout admission failures through the standard error toast', async () => { + mockHandleUpgrade.mockRejectedValueOnce( + new Error('Your subscription payment is still processing.') + ) + + await act(async () => { + root.render() + }) + + await act(async () => { + await currentState?.doUpgrade('team', 25000) + }) + + expect(mockToastError).toHaveBeenCalledWith('Your subscription payment is still processing.') + }) + it('includes the routed workspace when switching the host billing interval', async () => { await act(async () => { root.render() diff --git a/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.ts b/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.ts index 1792360d7d6..15152b4a57b 100644 --- a/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.ts +++ b/apps/sim/app/workspace/[workspaceId]/upgrade/hooks/use-upgrade-state.ts @@ -9,6 +9,8 @@ import type { WorkspaceHostContext } from '@/lib/api/contracts/workspaces' import { useSubscriptionUpgrade } from '@/lib/billing/client/upgrade' import { CREDIT_TIERS } from '@/lib/billing/constants' import { getPlanTierCredits, isEnterprise, isFree, isPro, isTeam } from '@/lib/billing/plan-helpers' +import { invalidateWorkspaceUsage } from '@/hooks/queries/utils/invalidate-usage' +import { subscriptionKeys } from '@/hooks/queries/utils/subscription-keys' import { workspaceHostKeys } from '@/hooks/queries/workspace-host' const PRO_TIER = CREDIT_TIERS[0] @@ -89,8 +91,22 @@ export function useUpgradeState({ } }, [ownerBilling.billingInterval, subscription.isPaid]) - const refreshHostContext = useCallback( - () => queryClient.invalidateQueries({ queryKey: workspaceHostKeys.detail(workspaceId) }), + /** + * A non-redirect plan switch settles server-side immediately, so every read that + * describes the plan has to be refetched — the host context the page renders from, + * the subscription/usage reads the billing surfaces share, the proration invoice the + * switch just produced, and the workspace credit availability that drives the credits + * chip and the run gate. + */ + const refreshBillingState = useCallback( + () => + Promise.all([ + queryClient.invalidateQueries({ queryKey: workspaceHostKeys.detail(workspaceId) }), + queryClient.invalidateQueries({ queryKey: subscriptionKeys.users() }), + queryClient.invalidateQueries({ queryKey: subscriptionKeys.usage() }), + queryClient.invalidateQueries({ queryKey: subscriptionKeys.invoicesAll() }), + invalidateWorkspaceUsage(queryClient), + ]), [queryClient, workspaceId] ) @@ -123,9 +139,9 @@ export function useUpgradeState({ await requestJson(billingSwitchPlanContract, { body: { targetPlanName: subscription.plan, interval, workspaceId }, }) - await refreshHostContext() + await refreshBillingState() }, - [isLegacyPlan, refreshHostContext, subscription.plan, workspaceId] + [isLegacyPlan, refreshBillingState, subscription.plan, workspaceId] ) const currentCredits = getPlanTierCredits(subscription.plan) @@ -154,11 +170,11 @@ export function useUpgradeState({ workspaceId, }, }) - await refreshHostContext() + await refreshBillingState() } catch (e) { toast.error(getErrorMessage(e, 'Failed to upgrade')) } - }, [subscription.isTeam, isAnnual, refreshHostContext, workspaceId]) + }, [subscription.isTeam, isAnnual, refreshBillingState, workspaceId]) const onUpgradeToOtherTier = useCallback(async () => { const onMax = @@ -170,11 +186,11 @@ export function useUpgradeState({ await requestJson(billingSwitchPlanContract, { body: { targetPlanName, workspaceId }, }) - await refreshHostContext() + await refreshBillingState() } catch (e) { toast.error(getErrorMessage(e, 'Failed to switch plan')) } - }, [subscription.plan, subscription.isTeam, refreshHostContext, workspaceId]) + }, [subscription.plan, subscription.isTeam, refreshBillingState, workspaceId]) return { isLoading: false, diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar.tsx index c2882134000..c053ec1b50e 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/action-bar/action-bar.tsx @@ -66,10 +66,30 @@ const ACTION_BUTTON_STYLES = [ * 25.11px of period. Writing 24/26 directly would render ~3.5% wide and drift * out of the squares' rhythm across the row. * + * Each edge ramps over 0.75px rather than switching colour at a single offset, + * which is why the stops come in pairs 0.375px either side of the mark's two + * edges. A gradient is sampled once per pixel with no coverage term, so a hard + * stop on a 15°-off-vertical edge can only ever land wholly on one side or the + * other — the marks came out visibly stepped, which is the one thing a shape + * this thin cannot hide. Ramping across roughly a device pixel gives the + * rasterizer the intermediate values antialiasing would have produced, and + * measured edge deviation drops from 0.28 device px (pure quantization) to 0.05. + * + * The period runs centre-of-mark to centre-of-mark (11.59 → 36.7) rather than + * starting at an edge, because a repeating gradient truncates at its own wrap: + * anchored at 0, the ramp leaving the mark would have run 24.735 → 25.485 and + * been cut at 25.11, so that edge got half the feather and the gap came out + * 1.93 → 1.75px. Both ramps have to sit strictly inside the period. The list + * still tiles backwards from its first stop, so the marks land exactly where + * anchoring at 0 put them — same 26px pitch, same phase against the squares. + * + * Widening the feather further would keep smoothing, but the gap is only 1.93px + * of stop, so it comes straight out of the mark's dark core. + * * `--surface-2` is the same fill the slots used; only where it is painted moved. */ const RUNNING_FILL = - 'bg-[repeating-linear-gradient(75deg,var(--surface-2)_0_23.18px,transparent_23.18px_25.11px)]' + 'bg-[repeating-linear-gradient(75deg,var(--surface-2)_11.59px_22.805px,transparent_23.555px_24.735px,var(--surface-2)_25.485px_36.7px)]' /** Left edge of the fill: clears the run/stop button, which stays live mid-run. */ const RUNNING_FILL_INSET_SWELL = 'left-[42px]' @@ -84,12 +104,14 @@ const RUNNING_FILL_INSET_PLAIN = 'left-[26px]' * inside it at the bottom — the fill visibly ran off the block. The per-slot * version never did, because each button's own clip contained it. * - * Same taper, read off that path: the edge sits 16.67px in from the row's right - * at the overlay's top (y=4) and 3.33px at its bottom (y=20), a slope of 20/24. - * Changing the end silhouette means changing these two numbers with it. + * Same taper, read off that path. Its straight run — (22.4, 2.88) to + * (36.59, 19.9) in the slot's own 40×24 box — has a slope of 20/24, so across + * the full row it moves from 20px in at the top to flush at the bottom. The + * overlay spans the row, so those are its two numbers; they are the slot's own + * edge continued, which is what puts the hatch's end exactly where a hovered + * slot's fill ends. Changing the end silhouette means changing them with it. */ -const RUNNING_FILL_END_TAPER = - '[clip-path:polygon(0_0,calc(100%_-_16.67px)_0,calc(100%_-_3.33px)_100%,0_100%)]' +const RUNNING_FILL_END_TAPER = '[clip-path:polygon(0_0,calc(100%_-_20px)_0,100%_100%,0_100%)]' const ICON_SIZE = 'size-[14px]' @@ -415,7 +437,11 @@ export const ActionBar = memo(