Skip to content

Commit 3726bc0

Browse files
committed
docs(ai-chat): document custom agents and complete the createSession reference
Adds a dedicated Custom agents page covering chat.customAgent (agent registration + session binding) and both loop styles: the managed createSession iterator and the hand-rolled primitives loop. Includes the patterns the managed lifecycle otherwise covers for you: continuation history seeding, persisting the user message before streaming, racing totalUsage after a stop, and the slim wire shape. The Backend page leads with a decision table across the three abstraction levels and now focuses on chat.agent, routing to the new page. Completes the ChatSessionOptions and ChatTurn reference tables (compaction, pendingMessages, usage fields, setMessages, prepareStep) and fixes stale examples that read a plural messages field off the wire payload.
1 parent 954ee5c commit 3726bc0

7 files changed

Lines changed: 428 additions & 270 deletions

File tree

docs/ai-chat/backend.mdx

Lines changed: 23 additions & 238 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,22 @@ import RcBanner from "/snippets/ai-chat-rc-banner.mdx";
88

99
<RcBanner />
1010

11+
There are three abstraction levels for a chat backend. All three speak the same wire protocol, so the [frontend transport](/ai-chat/frontend) works unchanged whichever you pick.
12+
13+
| Capability | `chat.agent()` | `chat.createSession()` | Raw primitives |
14+
| ------------------------------------- | -------------- | ------------------------------------------------------------- | -------------- |
15+
| Turn loop, stop signals, accumulation | Managed | Managed | You write it |
16+
| Lifecycle hooks | Yes | No — inline code per turn | No |
17+
| Continuation recovery on new runs | Automatic | [Manual seeding](/ai-chat/custom-agents#continuation-runs-and-history-seeding) | Manual seeding |
18+
| Compaction / steering | Built-in | Built-in | Manual |
19+
| Head Start, actions, tool approvals | Yes | No | No |
20+
| Custom stream conversion | No | Limited | Full control |
21+
| Agent dashboard visibility | Yes | Yes (via `customAgent`) | Yes |
22+
23+
The raw-primitives column assumes [`chat.customAgent()`](/ai-chat/custom-agents) as the wrapper, which is what makes the task visible to the agent dashboard.
24+
25+
Start with `chat.agent()`. Drop to `chat.createSession()` when you want to own the per-turn code (model routing, persistence, custom telemetry) without rebuilding the turn loop. Drop to raw primitives only when you need full control over stream conversion or a custom protocol.
26+
1127
## chat.agent()
1228

1329
The highest-level approach. Handles message accumulation, stop signals, turn lifecycle, and auto-piping automatically.
@@ -119,7 +135,7 @@ writer.write({
119135
</Info>
120136

121137
<Note>
122-
`chat.response` and the `writer` accumulation behavior work with `chat.agent` and `chat.createSession`. If you're using [`chat.customAgent`](#raw-task-with-primitives), you own the accumulator — see the raw-task example for the manual pattern.
138+
`chat.response` and the `writer` accumulation behavior work with `chat.agent` and `chat.createSession`. If you're using [`chat.customAgent`](/ai-chat/custom-agents), you own the accumulator — see the raw-task example for the manual pattern.
123139
</Note>
124140

125141
### Raw streaming with `chat.stream`
@@ -750,7 +766,7 @@ See [ChatUIMessageStreamOptions](/ai-chat/reference#chatuimessagestreamoptions)
750766
<Note>
751767
`onFinish` is managed internally for response capture and cannot be overridden here. Use
752768
`streamText`'s `onFinish` callback for custom finish handling, or use [raw task
753-
mode](#raw-task-with-primitives) for full control over `toUIMessageStream()`.
769+
mode](/ai-chat/custom-agents) for full control over `toUIMessageStream()`.
754770
</Note>
755771

756772
### Manual mode with task()
@@ -787,241 +803,10 @@ export const manualChat = task({
787803

788804
---
789805

790-
## chat.createSession()
791-
792-
A middle ground between `chat.agent()` and raw primitives. You get an async iterator that yields `ChatTurn` objects — each turn handles stop signals, message accumulation, and turn-complete signaling automatically. You control initialization, model/tool selection, persistence, and any custom per-turn logic.
793-
794-
Use `chat.createSession()` inside a standard `task()`:
795-
796-
```ts
797-
import { task } from "@trigger.dev/sdk";
798-
import { chat, type ChatTaskWirePayload } from "@trigger.dev/sdk/ai";
799-
import { streamText } from "ai";
800-
import { anthropic } from "@ai-sdk/anthropic";
801-
802-
export const myChat = task({
803-
id: "my-chat",
804-
run: async (payload: ChatTaskWirePayload, { signal }) => {
805-
// One-time initialization — just code, no hooks
806-
const clientData = payload.metadata as { userId: string };
807-
await db.chat.create({ data: { id: payload.chatId, userId: clientData.userId } });
808-
809-
const session = chat.createSession(payload, {
810-
signal,
811-
idleTimeoutInSeconds: 60,
812-
timeout: "1h",
813-
});
814-
815-
for await (const turn of session) {
816-
const result = streamText({
817-
model: anthropic("claude-sonnet-4-5"),
818-
messages: turn.messages,
819-
abortSignal: turn.signal,
820-
stopWhen: stepCountIs(15),
821-
});
822-
823-
// Pipe, capture, accumulate, and signal turn-complete — all in one call
824-
await turn.complete(result);
825-
826-
// Persist after each turn
827-
await db.chat.update({
828-
where: { id: turn.chatId },
829-
data: { messages: turn.uiMessages },
830-
});
831-
}
832-
},
833-
});
834-
```
835-
836-
### ChatSessionOptions
806+
## Custom agents
837807

838-
| Option | Type | Default | Description |
839-
| ---------------------- | ------------- | -------- | ------------------------------------------- |
840-
| `signal` | `AbortSignal` | required | Run-level cancel signal (from task context) |
841-
| `idleTimeoutInSeconds` | `number` | `30` | Seconds to stay idle between turns |
842-
| `timeout` | `string` | `"1h"` | Duration string for suspend timeout |
843-
| `maxTurns` | `number` | `100` | Max turns before ending |
808+
Both lower levels — `chat.createSession()` (managed turn iterator, your turn body) and `chat.customAgent()` with raw primitives (hand-rolled loop, full stream-conversion control) — are covered together on the Custom agents page, including the `ChatTurn` surface, the continuation-seeding pattern, and the hand-rolled-loop checklist:
844809

845-
### ChatTurn
846-
847-
Each turn yielded by the iterator provides:
848-
849-
| Field | Type | Description |
850-
| -------------- | ---------------- | ------------------------------------------------------ |
851-
| `number` | `number` | Turn number (0-indexed) |
852-
| `chatId` | `string` | Chat session ID |
853-
| `trigger` | `string` | What triggered this turn |
854-
| `clientData` | `unknown` | Client data from the transport |
855-
| `messages` | `ModelMessage[]` | Full accumulated model messages — pass to `streamText` |
856-
| `uiMessages` | `UIMessage[]` | Full accumulated UI messages — use for persistence |
857-
| `signal` | `AbortSignal` | Combined stop+cancel signal (fresh each turn) |
858-
| `stopped` | `boolean` | Whether the user stopped generation this turn |
859-
| `continuation` | `boolean` | Whether this is a continuation run |
860-
861-
| Method | Description |
862-
| ---------------------------- | ------------------------------------------------------------------- |
863-
| `turn.complete(source)` | Pipe stream, capture response, accumulate, and signal turn-complete |
864-
| `turn.done()` | Just signal turn-complete (when you've piped manually) |
865-
| `turn.addResponse(response)` | Add a response to the accumulator manually |
866-
867-
### turn.complete() vs manual control
868-
869-
`turn.complete(result)` is the easy path — it handles piping, capturing the response, accumulating messages, cleaning up aborted parts, and writing the turn-complete chunk.
870-
871-
For more control, you can do each step manually:
872-
873-
```ts
874-
for await (const turn of session) {
875-
const result = streamText({
876-
model: anthropic("claude-sonnet-4-5"),
877-
messages: turn.messages,
878-
abortSignal: turn.signal,
879-
stopWhen: stepCountIs(15),
880-
});
881-
882-
// Manual: pipe and capture separately
883-
const response = await chat.pipeAndCapture(result, { signal: turn.signal });
884-
885-
if (response) {
886-
// Custom processing before accumulating
887-
await turn.addResponse(response);
888-
}
889-
890-
// Custom persistence, analytics, etc.
891-
await db.chat.update({ ... });
892-
893-
// Must call done() when not using complete()
894-
await turn.done();
895-
}
896-
```
897-
898-
---
899-
900-
## Raw task with primitives
901-
902-
For full control, use a standard `task()` with the composable primitives from the `chat` namespace. You manage everything: the turn loop, stop signals, message accumulation, and turn-complete signaling.
903-
904-
Raw task mode also lets you call `.toUIMessageStream()` yourself with any options — including `onFinish` and `originalMessages`. This is the right choice when you need complete control over the stream conversion beyond what `chat.setUIMessageStreamOptions()` provides.
905-
906-
### Primitives
907-
908-
| Primitive | Description |
909-
| ------------------------------- | ------------------------------------------------------------------------------------------- |
910-
| `chat.messages` | Input stream for incoming messages — use `.waitWithIdleTimeout()` to wait for the next turn |
911-
| `chat.createStopSignal()` | Create a managed stop signal wired to the stop input stream |
912-
| `chat.pipeAndCapture(result)` | Pipe a `StreamTextResult` to the chat stream and capture the response |
913-
| `chat.writeTurnComplete()` | Signal the frontend that the current turn is complete |
914-
| `chat.MessageAccumulator` | Accumulates conversation messages across turns |
915-
| `chat.pipe(stream)` | Pipe a stream to the frontend (no response capture) |
916-
| `chat.cleanupAbortedParts(msg)` | Clean up incomplete parts from a stopped response |
917-
918-
### Example
919-
920-
```ts
921-
import { task } from "@trigger.dev/sdk";
922-
import { chat, type ChatTaskWirePayload } from "@trigger.dev/sdk/ai";
923-
import { streamText } from "ai";
924-
import { anthropic } from "@ai-sdk/anthropic";
925-
926-
export const myChat = task({
927-
id: "my-chat-raw",
928-
run: async (payload: ChatTaskWirePayload, { signal: runSignal }) => {
929-
let currentPayload = payload;
930-
931-
// Handle preload — wait for the first real message
932-
if (currentPayload.trigger === "preload") {
933-
const result = await chat.messages.waitWithIdleTimeout({
934-
idleTimeoutInSeconds: 60,
935-
timeout: "1h",
936-
spanName: "waiting for first message",
937-
});
938-
if (!result.ok) return;
939-
currentPayload = result.output;
940-
}
941-
942-
const stop = chat.createStopSignal();
943-
const conversation = new chat.MessageAccumulator();
944-
945-
for (let turn = 0; turn < 100; turn++) {
946-
stop.reset();
947-
948-
const messages = await conversation.addIncoming(
949-
currentPayload.messages,
950-
currentPayload.trigger,
951-
turn
952-
);
953-
954-
const combinedSignal = AbortSignal.any([runSignal, stop.signal]);
955-
956-
const result = streamText({
957-
model: anthropic("claude-sonnet-4-5"),
958-
messages,
959-
abortSignal: combinedSignal,
960-
stopWhen: stepCountIs(15),
961-
});
962-
963-
let response;
964-
try {
965-
response = await chat.pipeAndCapture(result, { signal: combinedSignal });
966-
} catch (error) {
967-
if (error instanceof Error && error.name === "AbortError") {
968-
if (runSignal.aborted) break;
969-
// Stop — fall through to accumulate partial
970-
} else {
971-
throw error;
972-
}
973-
}
974-
975-
if (response) {
976-
const cleaned =
977-
stop.signal.aborted && !runSignal.aborted ? chat.cleanupAbortedParts(response) : response;
978-
await conversation.addResponse(cleaned);
979-
}
980-
981-
if (runSignal.aborted) break;
982-
983-
// Persist, analytics, etc.
984-
await db.chat.update({
985-
where: { id: currentPayload.chatId },
986-
data: { messages: conversation.uiMessages },
987-
});
988-
989-
await chat.writeTurnComplete();
990-
991-
// Wait for the next message
992-
const next = await chat.messages.waitWithIdleTimeout({
993-
idleTimeoutInSeconds: 60,
994-
timeout: "1h",
995-
spanName: "waiting for next message",
996-
});
997-
if (!next.ok) break;
998-
currentPayload = next.output;
999-
}
1000-
1001-
stop.cleanup();
1002-
},
1003-
});
1004-
```
1005-
1006-
### MessageAccumulator
1007-
1008-
The `MessageAccumulator` handles the transport protocol automatically:
1009-
1010-
- Turn 0: replaces messages (full history from frontend)
1011-
- Subsequent turns: appends new messages (frontend only sends the new user message)
1012-
- Regenerate: replaces messages (full history minus last assistant message)
1013-
1014-
```ts
1015-
const conversation = new chat.MessageAccumulator();
1016-
1017-
// Returns full accumulated ModelMessage[] for streamText
1018-
const messages = await conversation.addIncoming(payload.messages, payload.trigger, turn);
1019-
1020-
// After piping, add the response
1021-
const response = await chat.pipeAndCapture(result);
1022-
if (response) await conversation.addResponse(response);
1023-
1024-
// Access accumulated messages for persistence
1025-
conversation.uiMessages; // UIMessage[]
1026-
conversation.modelMessages; // ModelMessage[]
1027-
```
810+
<Card title="Custom agents" icon="screwdriver-wrench" href="/ai-chat/custom-agents">
811+
Build agents without the managed lifecycle — createSession or raw primitives.
812+
</Card>

0 commit comments

Comments
 (0)