refactor(tables): make lib/table/orchestration the single implementation (#6134)

* refactor(orchestration): move the shared error contract out of lib/workflows

OrchestrationErrorCode and statusForOrchestrationError are the contract every
lib/[resource]/orchestration module returns against, but they lived inside the
workflows module, so resource-neutral code (lib/folders) already had to import
from a workflow path. Moved to lib/core/orchestration/types.

Adds a 'locked' class mapping to 423. Both tables and workflows have a lock
that forbids a mutation, and each caller was translating that to a status
itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(tables): make lib/table/orchestration the single implementation

Column update was implemented four times — the UI route, v1, v2, and the
copilot table tool — each calling the same column services but owning its own
guards, error mapping, and audit. The copies had drifted, and the drift was the
bug: v2 was missing both guards, only the copilot copy minted stable option
ids, and only v1/v2 audited.

performUpdateTableColumn, performDeleteTable, and performDeleteTableRow now own
that logic; all ten call sites reduce to auth, parse, call, render. The guards
are asserted once in lib/table/orchestration rather than four times against
four routes.

Behavior this consolidates, previously true on only some paths:

- The typeChanging guard. updateColumnType early-returns on an unchanged type
  and drops any options sent with it, so restating the current type alongside
  new options silently discarded them. v2 had no guard at all and, since its
  contract shares v1's body schema, accepted options and ignored them.
- The select-unique guard. Each write is its own locked transaction, so a
  rename or type change paired with a constraint write that is going to fail
  commits first and then throws, half-applying the schema change.
- Stable select-option ids. Cells reference the option id, so an edit that
  re-sends an option by name has to reuse it or every cell holding it is
  orphaned. Only the copilot path did this; normalizeSelectOptionsInput moves to
  lib/table/select-options and now covers every caller. It preserves a supplied
  id, so it is a no-op for the fully-formed options the HTTP contracts accept.
- required forwarded into the type and options writes, so a conversion
  validates against the constraint the same request is setting.
- An audit on every successful update. The UI route and the copilot tool
  emitted none.
- Single-row delete through the row service. v2 did a raw db.delete, skipping
  assertRowDelete and deleteOrderedRow, so a delete-locked table returned 200
  and the row-count bookkeeping never ran.
- The delete actor handed to deleteTable, which audits only when a row was
  actually archived. v1 and v2 omitted it and audited themselves outside that
  check, emitting TABLE_DELETED for a no-op delete of an archived table.

Failure classes come back as OrchestrationErrorCode; v2 renders them through a
new v2ErrorForOrchestration, mirroring statusForOrchestrationError on the v1
and UI surfaces, so a given failure maps to the same status everywhere.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(tables): bind the column-update tests to the orchestration function

The base's route tests assert which column service each payload reaches — the
behavior that now lives in performUpdateTableColumn. They mocked the `@/lib/table`
barrel; the orchestration module imports the service directly, so they mock that
too and keep asserting the same thing through the extracted implementation.

The orchestration tests move onto the base's semantics: writes address the
stable column id, a rename rides inside the write it accompanies rather than
running first, and the currency guards replace the non-select options guard the
service now owns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(copilot): drop the column-type import the delegation made dead

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(tables): move the audit log out of the table service

`lib/table/service.ts` wrote its own audit rows, so whether an operation was
audited depended on which function a caller reached for rather than on a user
having performed it. That is what let v1 and v2 audit a no-op delete, and what
made `deleteTable`'s optional `actingUserId` double as an audit opt-out flag.

Worse, most sites fell back to `actingUserId ?? createdBy`, so an unattributed
call was logged against the table's *creator*. The copilot `mv` path passed no
actor at all: renaming someone else's table recorded them as the renamer.

Audit now lives in the orchestration functions — performDeleteTable,
performRenameTable, performMoveTableToFolder, performUpdateTableLocks — and
the services just write. Internal callers (folder cascade, import rollback)
keep calling the service and are silent by construction rather than by
remembering to omit an argument.

Two services now return what the audit needs: `deleteTable` reports whether it
actually archived a row, so a repeat delete logs nothing; `updateTableLocks`
returns the before/after locks, since only the locked write can observe the
transition its description names.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(tables): restore audit provenance and conflict status in orchestration

Moving the audits into the orchestration functions dropped three things the
routes had been carrying, and added one the orchestration now owns twice.

- The v1 and v2 column-update routes passed `request` to `recordAudit`, so
  their audit rows recorded the caller's IP and user-agent. The orchestration
  function had no way to receive it. Every table orchestration function now
  takes an optional `OrchestrationRequestContext` and every HTTP route
  forwards it; the copilot and VFS callers, which have no request, omit it.
- `classifyTableMutation` matched `TableConflictError` on "already exists"
  appearing in the message and reported it as `validation`, turning the UI
  route's 409 on a duplicate table rename into a 400. It now matches the type,
  the way `performRestoreTable` already did.
- `captureServerEvent` ran on every delete while the audit was gated on a row
  actually being archived, so a repeat delete of an archived table still
  reported `table_deleted`. Both now hang off the same evidence.
- The copilot delete path kept its own `captureServerEvent` from when the
  service did not emit one, double-counting every copilot table delete.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a

* fix(tables): say which type a no-op column update restated

A copilot `update_column` payload whose only content was the column's current
type used to return success with the live schema, while the v1, v2, and UI
routes rejected the same payload with "No updates specified". Delegating to
`performUpdateTableColumn` unified them onto the routes' rejection — correct,
but the message tells the caller its request was empty when it named a type.

The orchestration function now reports the same thing `updateColumnType` reports
when it loses this race concurrently: the column is already that type, re-issue
without the type change. An empty payload still reads "No updates specified".

Drops the copilot's `outcome.table ?? tableForUpdate` fallback with it — the
comment described the no-op that can no longer reach that line, and a success
always carries a table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a

* refactor(tables): classify failures by type instead of by message text

The table module decided HTTP statuses by searching error messages for
phrases. `VALIDATION_MESSAGE_FRAGMENTS` and `ROW_WRITE_ERROR_PATTERNS` held 32
substrings between them, and fifteen more lists were inlined in routes — 83
matchers over 17 files, each its own copy of the guesswork and already drifted
apart. It made message wording load-bearing: `TableRowLimitError`'s own doc
comment noted that its text had to contain "row limit" for a route to answer
400, and adding "already exists" to a rename message silently demoted a 409 to
a 400 (the bug fixed one commit ago, by adding another special case).

Services now throw `OrchestrationError`, which carries the transport-neutral
`OrchestrationErrorCode` the layers above already speak. Classification is one
`instanceof` in `orchestrationErrorResponse` (UI + v1) and
`v2CaughtOrchestrationError` (v2). Every pattern list is gone. Wording is free
to change; an unclassified error still becomes a generic 500, which is what an
unexpected fault should be.

`asOrchestrationError` walks the `cause` chain rather than testing the caught
value directly: drizzle wraps a throw raised inside a transaction callback in a
`DrizzleQueryError` whose own message is the failed SQL, so a bare `instanceof`
would drop every failure raised inside `withLockedTable`. That is the same
reason `rootErrorMessage` had to dig for a root cause before.

Three throws stay bare `Error` deliberately — `Table ID mismatch`, `Workspace
ID mismatch`, and `Failed to build upsert conflict predicate` are internal
invariants no consumer classified, and they keep falling through to a 500.
`Insufficient capacity` was in the pattern list with no producer anywhere in
the codebase.

Status changes, all deliberate:

- `'forbidden'` joins the code union so the table-row-limit ceiling keeps its
  403; without it this refactor would have flattened it to 400.
- import-async's table-limit rejection: 400 -> 403, matching the two other
  create routes it had drifted from.
- Renaming a table to an invalid name: 500 -> 400. `validateTableName`
  messages don't contain "Invalid", so no matcher ever caught them.
- Restoring a table that isn't archived, or into an archived workspace:
  500 -> 400.
- A duplicate *column* name stays `validation`/400 rather than becoming a 409
  like a duplicate table name. Both v1 and the orchestration have always
  answered 400 for it; changing a published status is not this refactor's job.

The twelve tests that changed were asserting the substring mechanism itself,
constructing plain `Error`s with magic strings. They now assert the real
contract, plus new cases pinning that identical wording carrying no
classification stays internal and keeps its message off the wire.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YGzbVDZpe2dEALbu2BUU8a

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Author: TheodoreSpeaks

apps/sim/app/api/table/[tableId]/columns/route.test.ts

@@ -42,6 +42,13 @@ vi.mock('@/lib/table', () => ({
   updateColumnOptions: mockUpdateColumnOptions,
   updateColumnType: mockUpdateColumnType,
 }))
+vi.mock('@/lib/table/columns/service', () => ({
+  renameColumn: mockRenameColumn,
+  updateColumnConstraints: mockUpdateColumnConstraints,
+  updateColumnCurrency: mockUpdateColumnCurrency,
+  updateColumnOptions: mockUpdateColumnOptions,
+  updateColumnType: mockUpdateColumnType,
+}))
 vi.mock('@/app/api/table/utils', () => ({
   accessError: () => new Response('denied', { status: 403 }),
   checkAccess: mockCheckAccess,
@@ -50,6 +57,7 @@ vi.mock('@/app/api/table/utils', () => ({
   tableLockErrorResponse: () => null,
 }))
 
+import { OrchestrationError } from '@/lib/core/orchestration/types'
 import { PATCH } from '@/app/api/table/[tableId]/columns/route'
 
 const WORKSPACE_ID = '11111111-1111-4111-8111-111111111111'
@@ -159,7 +167,10 @@ describe('PATCH /api/table/[tableId]/columns — pre-flight guards', () => {
     // Stands in for the race the guards cannot close: the column stopped being
     // a currency between the snapshot the guards read and this write.
     mockUpdateColumnCurrency.mockRejectedValue(
-      new Error('Cannot set currency on column "amount" of type "string"')
+      new OrchestrationError(
+        'validation',
+        'Cannot set currency on column "amount" of type "string"'
+      )
     )
 
     const response = await patch({ name: 'renamed', currencyCode: 'USD' })

apps/sim/app/api/table/[tableId]/columns/route.ts

@@ -8,20 +8,11 @@ import {
 import { parseRequest } from '@/lib/api/server'
 import { isZodError, validationErrorResponse } from '@/lib/api/server/validation'
 import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
+import { statusForOrchestrationError } from '@/lib/core/orchestration/types'
 import { generateRequestId } from '@/lib/core/utils/request'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
-import {
-  addTableColumn,
-  deleteColumn,
-  renameColumn,
-  updateColumnConstraints,
-  updateColumnCurrency,
-  updateColumnOptions,
-  updateColumnType,
-} from '@/lib/table'
-import { columnMatchesRef, getColumnId } from '@/lib/table/column-keys'
-import { columnTypeById } from '@/lib/table/column-types'
-import { isSupportedCurrencyCode } from '@/lib/table/currency'
+import { addTableColumn, deleteColumn } from '@/lib/table'
+import { performUpdateTableColumn } from '@/lib/table/orchestration'
 import {
   accessError,
   checkAccess,
@@ -120,215 +111,32 @@ export const PATCH = withRouteHandler(async (request: NextRequest, context: Colu
       return NextResponse.json({ error: 'Invalid workspace ID' }, { status: 400 })
     }
 
-    const { updates } = validated
-    let updatedTable = null
-
-    // A payload that repeats the current type must not go through
-    // `updateColumnType` — it early-returns on an unchanged type and would drop
-    // any `options` alongside it. Only a real type change routes there; an
-    // unchanged type with options routes to the options-only update.
-    const currentColumn = table.schema.columns.find((c) =>
-      columnMatchesRef(c, validated.columnName)
-    )
-    // Address every write below by the stable id, not the name: a rename folded
-    // into one of them must not break the next one's lookup.
-    const columnRef = currentColumn ? getColumnId(currentColumn) : validated.columnName
-    // The constraints write below is a separate, unconditional step, so it is
-    // the last one whenever it runs — that is the write the rename rides on.
-    const typeChanging = updates.type !== undefined && updates.type !== currentColumn?.type
-    if (!currentColumn) {
-      return NextResponse.json(
-        { error: `Column "${validated.columnName}" not found` },
-        { status: 404 }
-      )
-    }
-
-    // A retype applies and validates the constraints itself, so the separate
-    // constraint write only runs when the type is unchanged. The rename rides
-    // whichever write actually runs last.
-    const typedWriteRuns =
-      typeChanging ||
-      updates.currencyCode !== undefined ||
-      updates.options !== undefined ||
-      updates.multiple !== undefined
-    const constraintsWriteRuns =
-      !typedWriteRuns && (updates.required !== undefined || updates.unique !== undefined)
-    const renameWithTypedWrite =
-      updates.name && !constraintsWriteRuns ? { newName: updates.name } : {}
-
-    // Every write below is its own locked transaction, so one that is going to
-    // fail leaves the earlier ones committed. These guards reject the knowable
-    // cases up front, before any write at all.
-    // Gate on the type the column ENDS UP with, not on whether the type is
-    // changing: an options-only update on an existing select column carries the
-    // same hazard as a conversion does.
-    const resultingType = updates.type ?? currentColumn?.type
-    if (updates.currencyCode !== undefined) {
-      if (resultingType !== 'currency') {
-        return NextResponse.json(
-          {
-            error: `Cannot set currency on column "${validated.columnName}" of type "${resultingType}"`,
-          },
-          { status: 400 }
-        )
-      }
-      if (!isSupportedCurrencyCode(updates.currencyCode)) {
-        return NextResponse.json(
-          {
-            error: `Invalid currency code "${updates.currencyCode}". Use an ISO 4217 code, e.g. USD`,
-          },
-          { status: 400 }
-        )
-      }
-    }
-    // The rename runs last (see below), so a name already taken would fail after
-    // the typed write committed. This is the only rename failure a caller can
-    // cause; catching it here leaves just the concurrent-collision race, which
-    // no pre-flight check can close.
-    if (
-      updates.name &&
-      table.schema.columns.some(
-        (c) =>
-          c.name.toLowerCase() === updates.name?.toLowerCase() &&
-          !columnMatchesRef(c, validated.columnName)
-      )
-    ) {
-      return NextResponse.json(
-        { error: `Column "${updates.name}" already exists` },
-        { status: 400 }
-      )
-    }
-    if (
-      currentColumn?.workflowGroupId &&
-      (updates.required !== undefined || updates.unique !== undefined)
-    ) {
-      return NextResponse.json(
-        {
-          error: `Cannot change constraints on workflow-output column "${currentColumn.name}". Constraints aren't applicable to columns whose values come from workflow execution.`,
-        },
-        { status: 400 }
-      )
-    }
-    if (updates.unique === true && !columnTypeById(resultingType).supportsUnique) {
+    const outcome = await performUpdateTableColumn({
+      table,
+      columnName: validated.columnName,
+      userId: authResult.userId,
+      updates: validated.updates,
+      requestId,
+      request,
+    })
+    if (!outcome.success || !outcome.table) {
       return NextResponse.json(
-        { error: `Cannot set a ${resultingType} column as unique` },
-        { status: 400 }
+        { error: outcome.error ?? 'Failed to update column' },
+        { status: statusForOrchestrationError(outcome.errorCode) }
       )
     }
 
-    if (typeChanging) {
-      updatedTable = await updateColumnType(
-        {
-          tableId,
-          columnName: columnRef,
-          newType: updates.type as NonNullable<typeof updates.type>,
-          ...(updates.options !== undefined ? { options: updates.options } : {}),
-          ...(updates.multiple !== undefined ? { multiple: updates.multiple } : {}),
-          ...(updates.currencyCode !== undefined ? { currencyCode: updates.currencyCode } : {}),
-          // Forwarded so the conversion validates against the constraint this
-          // same request is about to set, not the column's current one.
-          ...(updates.required !== undefined ? { required: updates.required } : {}),
-          ...(updates.unique !== undefined ? { unique: updates.unique } : {}),
-          ...renameWithTypedWrite,
-        },
-        requestId
-      )
-    } else if (updates.currencyCode !== undefined) {
-      // Re-denominating an existing currency column: schema-only, no cell
-      // rewrite. Reached only when the type is unchanged — a conversion INTO
-      // currency carries the code through `updateColumnType` above.
-      updatedTable = await updateColumnCurrency(
-        {
-          tableId,
-          columnName: columnRef,
-          currencyCode: updates.currencyCode,
-          ...(updates.required !== undefined ? { required: updates.required } : {}),
-          ...(updates.unique !== undefined ? { unique: updates.unique } : {}),
-          ...renameWithTypedWrite,
-        },
-        requestId
-      )
-    } else if (updates.options !== undefined || updates.multiple !== undefined) {
-      updatedTable = await updateColumnOptions(
-        {
-          tableId,
-          columnName: columnRef,
-          options: updates.options ?? currentColumn?.options ?? [],
-          ...(updates.multiple !== undefined ? { multiple: updates.multiple } : {}),
-          // Forwarded so the removal guard validates against the constraint this
-          // same request is about to set, not the column's current one.
-          ...(updates.required !== undefined ? { required: updates.required } : {}),
-          ...(updates.unique !== undefined ? { unique: updates.unique } : {}),
-          ...renameWithTypedWrite,
-        },
-        requestId
-      )
-    }
-
-    // Skipped whenever a typed write ran: that write already applied and
-    // validated these, in one transaction with the change they accompany.
-    if (constraintsWriteRuns) {
-      updatedTable = await updateColumnConstraints(
-        {
-          tableId,
-          columnName: columnRef,
-          ...(updates.required !== undefined ? { required: updates.required } : {}),
-          ...(updates.unique !== undefined ? { unique: updates.unique } : {}),
-          ...(updates.name ? { newName: updates.name } : {}),
-        },
-        requestId
-      )
-    }
-
-    // A rename rides along with the LAST write above, inside that write's
-    // transaction — a rename is metadata-only (rows key on the stable column
-    // id), so nothing forces it to be its own write, and folding it in is what
-    // stops a combined request from committing one half and then failing. Only
-    // a rename with nothing to ride on runs standalone.
-    if (updates.name && !updatedTable) {
-      updatedTable = await renameColumn(
-        { tableId, oldName: columnRef, newName: updates.name },
-        requestId
-      )
-    }
-
-    if (!updatedTable) {
-      return NextResponse.json({ error: 'No updates specified' }, { status: 400 })
-    }
-
     return NextResponse.json({
       success: true,
       data: {
-        columns: updatedTable.schema.columns.map(normalizeColumn),
+        columns: outcome.table.schema.columns.map(normalizeColumn),
       },
     })
   } catch (error) {
-    const lockError = tableLockErrorResponse(error)
-    if (lockError) return lockError
     if (isZodError(error)) {
       return validationErrorResponse(error, 'Invalid request data')
     }
 
-    const msg = rootErrorMessage(error)
-    if (msg.includes('not found') || msg.includes('Table not found')) {
-      return NextResponse.json({ error: msg }, { status: 404 })
-    }
-    if (
-      msg.includes('already exists') ||
-      msg.includes('Cannot delete the last column') ||
-      msg.includes('Cannot set column') ||
-      msg.includes('Cannot set unique column') ||
-      msg.includes('Invalid column') ||
-      msg.includes('exceeds maximum') ||
-      msg.includes('incompatible') ||
-      msg.includes('duplicate') ||
-      msg.includes('option') ||
-      msg.includes('currency') ||
-      msg.includes('is already type')
-    ) {
-      return NextResponse.json({ error: msg }, { status: 400 })
-    }
-
     logger.error(`[${requestId}] Error updating column in table ${tableId}:`, error)
     return NextResponse.json({ error: 'Failed to update column' }, { status: 500 })
   }

apps/sim/app/api/table/[tableId]/columns/run/route.ts

@@ -8,7 +8,12 @@ import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { TableQueryValidationError } from '@/lib/table/errors'
 import { toLegacyFilter } from '@/lib/table/query-builder/converters'
 import { runWorkflowColumn } from '@/lib/table/workflow-columns'
-import { accessError, checkAccess, tableFilterError } from '@/app/api/table/utils'
+import {
+  accessError,
+  checkAccess,
+  orchestrationErrorResponse,
+  tableFilterError,
+} from '@/app/api/table/utils'
 
 const logger = createLogger('TableRunColumnAPI')
 
@@ -66,9 +71,8 @@ export const POST = withRouteHandler(async (request: NextRequest, { params }: Ro
     if (error instanceof TableQueryValidationError) {
       return NextResponse.json({ error: error.message }, { status: 400 })
     }
-    if (error instanceof Error && error.message === 'Invalid workspace ID') {
-      return NextResponse.json({ error: 'Invalid workspace ID' }, { status: 400 })
-    }
+    const classified = orchestrationErrorResponse(error)
+    if (classified) return classified
     logger.error(`run-column failed:`, error)
     return NextResponse.json({ error: 'Failed to run columns' }, { status: 500 })
   }

apps/sim/app/api/table/[tableId]/import/route.test.ts

@@ -79,6 +79,7 @@ vi.mock('@/lib/table/billing', () => ({
     limit >= 0 && current + added > limit,
 }))
 
+import { OrchestrationError } from '@/lib/core/orchestration/types'
 import { TableLockedError } from '@/lib/table/mutation-locks'
 import { POST } from '@/app/api/table/[tableId]/import/route'
 
@@ -372,7 +373,10 @@ describe('POST /api/table/[tableId]/import', () => {
 
   it('surfaces unique violations from importAppendRows as 400', async () => {
     mockImportAppendRows.mockRejectedValueOnce(
-      new Error('Row 1: Column "name" must be unique. Value "Alice" already exists in row row_xxx')
+      new OrchestrationError(
+        'validation',
+        'Row 1: Column "name" must be unique. Value "Alice" already exists in row row_xxx'
+      )
     )
     const response = await callPost(
       createFormData(createCsvFile('name,age\nAlice,30'), { mode: 'append' })
@@ -516,7 +520,9 @@ describe('POST /api/table/[tableId]/import', () => {
     })
 
     it('surfaces column-creation failures from importAppendRows as 400', async () => {
-      mockImportAppendRows.mockRejectedValueOnce(new Error('Column "email" already exists'))
+      mockImportAppendRows.mockRejectedValueOnce(
+        new OrchestrationError('validation', 'Column "email" already exists')
+      )
       const response = await callPost(
         createFormData(createCsvFile('name,age,email\nAlice,30,a@x.io'), {
           mode: 'append',
@@ -529,7 +535,9 @@ describe('POST /api/table/[tableId]/import', () => {
     })
 
     it('surfaces row insert failures without success when schema was mutated', async () => {
-      mockImportAppendRows.mockRejectedValueOnce(new Error('must be unique'))
+      mockImportAppendRows.mockRejectedValueOnce(
+        new OrchestrationError('validation', 'must be unique')
+      )
       const response = await callPost(
         createFormData(createCsvFile('name,age,email\nAlice,30,a@x.io'), {
           mode: 'append',