feat(cloudflare): AI Gateway LanguageModel + DO-backed chat persistence

[sam-goodwin]

Wire Workers AI through the AI Gateway as an Effect LanguageModel, with a Durable Object–backed BackingPersistence so chat history lives in state.storage and survives across DO invocations.

// 1. Declare the gateway as a resource
export const Gateway = Cloudflare.AiGateway("Gateway", {
  cacheTtl: 60,
  collectLogs: true,
});

// 2. Inside a Durable Object, bind it and turn it into a LanguageModel
export default class ChatAgent extends Cloudflare.DurableObjectNamespace<ChatAgent>()(
  "ChatAgent",
  Effect.gen(function* () {
    const ai = yield* Cloudflare.AiGateway.bind(Gateway);
    const model = ai.model({
      client: ai,
      model: "@cf/moonshotai/kimi-k2.6",
      parameters: { temperature: 0.3, maxTokens: 1024 },
    });

    return Effect.gen(function* () {
      const persistence = yield* Chat.Persistence;
      return {
        send: (threadId: string, prompt: string) =>
          Effect.gen(function* () {
            const chat = yield* persistence.getOrCreate(threadId);
            return yield* chat.generateText({ prompt, toolkit: ChatToolkit });
          }).pipe(Effect.provide(model), Effect.provide(ChatToolkitLayer)),
      };
    }).pipe(
      // Persist chat history into the DO's own storage
      Effect.provide(Chat.layerPersisted({ storeId: "chat" })),
      Effect.provide(Cloudflare.DurableObjectChatPersistence),
    );
  }),
) {}

A POST /api/chat on the Worker that owns this DO returns the assistant reply and re-reads its history on subsequent turns — the same getByName(id) instance keeps its Chat state across separate HTTP requests, because the underlying BackingPersistence writes to state.storage.

What's in here

• Cloudflare.AiGatewayLanguageModel — LanguageModel.LanguageModel provider for Workers AI via the AI Gateway. Supports generateText and streamText (full SSE pipeline, text + tool-call deltas).
• Cloudflare.DurableObjectChatPersistence — Effect BackingPersistence Layer backed by a DO's state.storage. Drop in under Chat.layerPersisted({ storeId }) for fully-persistent chat sessions; nothing else to wire.
• Slim Agent — toolkit and handlerLayer are now static class fields. The old chat / coding / lsp / fs subtrees are removed; you compose Effect's AI primitives directly.
• Integration tests deploy a fixture Worker + DO and verify (a) generateText, (b) streamText over real SSE, and (c) a two-turn conversation where turn 2 recalls a fact from turn 1 across separate HTTP requests (different DO invocations).

Example app

examples/cloudflare-chat-bot/ — full stack: Worker + DO ChatAgent (this PR) + Vite React SPA. Routes:

• POST /api/chat?id=:sessionId — send a prompt, get the assistant reply + full conversation
• GET /api/messages?id=:sessionId — fetch existing thread
• POST /api/reset?id=:sessionId — clear thread

One getByName(sessionId) per session = one persisted chat thread per user. Tools (get_current_time, calculate, roll_dice) demonstrate Toolkit.toLayer end-to-end.

Docs

• New tutorial: tutorial/cloudflare/agent walks from "bind the gateway" → "wrap as LanguageModel" → "persist in a DO" step by step.

[alchemy-version-bot]

Website Preview Deployed

URL: https://alchemyeffectwebsite-worker-pr-293-24sz5eo6hk5t6245.testing-2b2.workers.dev

Built from commit 9b69603.

---

This comment updates automatically with each push.

[alchemy-version-bot]

Install the packages built from this commit:

alchemy

bun add alchemy@https://pkg.ing/alchemy/dd39a24

@alchemy.run/better-auth

bun add @alchemy.run/better-auth@https://pkg.ing/@alchemy.run/better-auth/dd39a24

@alchemy.run/pr-package

bun add @alchemy.run/pr-package@https://pkg.ing/@alchemy.run/pr-package/dd39a24