Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,9 +187,12 @@ const welcomeEmail = await step.run({ name: "welcome-email" }, async () => {
All steps are executed synchronously by the worker. When a worker encounters a
new step:

1. It creates a `step_attempt` record with status `running`.
2. It executes the step function inline.
3. Upon completion, it updates the `step_attempt` to status `completed` with
1. It resolves the step's durable key for this execution pass. The first
occurrence keeps its base name; later collisions are auto-indexed as
`name:1`, `name:2`, and so on.
Comment thread
jamescmartinez marked this conversation as resolved.
2. It creates a `step_attempt` record with status `running`.
3. It executes the step function inline.
4. Upon completion, it updates the `step_attempt` to status `completed` with
the result.

Workers can be configured with a high concurrency limit (e.g., 100 or more) to
Expand Down Expand Up @@ -224,6 +227,11 @@ await step.sleep("wait-one-hour", "1h");
it durably. When the timeout is reached (default 7d), the parent step fails but
the child workflow continues running independently.

All step APIs (`step.run`, `step.sleep`, and `step.invokeWorkflow`) share the
same collision logic for durable keys. If duplicate base names are encountered
in one execution pass, OpenWorkflow auto-indexes them as `name`, `name:1`,
`name:2`, and so on so each step call maps to a distinct step attempt.

## 4. Error Handling & Retries

### 4.1. Step Failures & Retries
Expand Down
52 changes: 19 additions & 33 deletions packages/docs/docs/dynamic-steps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@ description: Run a variable number of steps based on runtime data

Sometimes you don't know how many steps a workflow needs until it runs. You
might need to fetch data for each item in a list, process rows from a query,
or fan out across a set of IDs from an API response. OpenWorkflow handles
this — you can create steps inside loops and maps, as long as each step has a
deterministic name.
or fan out across a set of IDs from an API response.

OpenWorkflow handles this automatically. When multiple steps share the same
name, they're disambiguated in order (`fetch-data`, `fetch-data:1`,
`fetch-data:2`, ...). You don't need to generate unique names yourself.
Comment thread
jamescmartinez marked this conversation as resolved.

## Basic Pattern

Expand All @@ -16,49 +18,33 @@ Map over your data and create a step per item using `Promise.all`:
```ts
const results = await Promise.all(
input.items.map((item) =>
step.run({ name: `fetch-data:${item.id}` }, async () => {
step.run({ name: "fetch-data" }, async () => {
return await thirdPartyApi.fetch(item.id);
}),
),
);
```

Each step is individually memoized. If the workflow restarts, completed steps
return their cached results and only the remaining steps re-execute.

The most important rule: **step names must be deterministic across replays**.
Use a stable identifier from the data itself — like a database ID, a slug, or
a unique key:

```ts
// Good — stable ID from the data
step.run({ name: `process-order:${order.id}` }, ...)
step.run({ name: `send-email:${user.email}` }, ...)

// Bad — non-deterministic, different on every run
step.run({ name: `task-${Date.now()}` }, ...)
step.run({ name: `task-${crypto.randomUUID()}` }, ...)
```

<Warning>
Non-deterministic names (timestamps, random values, request IDs) break replay.
Completed steps won't be found in history, causing them to re-execute.
</Warning>
Every step uses the same name — OpenWorkflow appends `:1`, `:2`, etc.
automatically. Each step is individually memoized, so if the workflow restarts,
completed steps return their cached results and only the remaining steps
re-execute.

### Falling Back to Array Indexes
## Stable IDs for Mutable Collections

When no stable ID exists, you can use the array index:
If items can be added, removed, or reordered between retries, include a stable
ID from the data in the step name instead of relying on auto-indexing:

```ts
const results = await Promise.all(
input.items.map((item, index) =>
step.run({ name: `fetch-data:${index}` }, async () => {
return await thirdPartyApi.fetch(item.lookupKey);
input.orders.map((order) =>
step.run({ name: `process-order:${order.id}` }, async () => {
return await processOrder(order);
}),
),
);
```

This is safe only if the array order is identical between the original run and
any replay. If the order changes, cached results get returned for the wrong
items.
This way, each step is tied to a specific item regardless of its position in
the array. Use any stable identifier — a database ID, a slug, or a unique key
from the data itself.
17 changes: 6 additions & 11 deletions packages/docs/docs/steps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,22 +65,20 @@ The customer is charged exactly once.

## Step Names

Step names must be unique within a workflow. They identify steps during replay
and should be stable across code changes.
Step names identify checkpoints during replay and should be stable across code
changes. Use descriptive names that reflect what the step does:

```ts
// Good - descriptive, stable names
await step.run({ name: "fetch-user" }, ...);
await step.run({ name: "send-welcome-email" }, ...);
await step.run({ name: "update-user-status" }, ...);

// Bad - generic names that could conflict
await step.run({ name: "step-1" }, ...);
await step.run({ name: "step-2" }, ...);
```

If you need to create a dynamic number of steps from runtime data (like
mapping over an array), see [Dynamic Steps](/docs/dynamic-steps).
If two steps share the same name in a single execution, OpenWorkflow
automatically disambiguates them by appending `:1`, `:2`, and so on in
encounter order. This is most useful for [dynamic steps](/docs/dynamic-steps)
where the number of steps isn't known ahead of time.
Comment thread
jamescmartinez marked this conversation as resolved.

<Warning>
Changing step names after workflows are in-flight can cause replay errors.
Expand Down Expand Up @@ -150,9 +148,6 @@ const childOutput = await step.invokeWorkflow("generate-report", {
});
```

Comment thread
jamescmartinez marked this conversation as resolved.
If `timeout` is reached, the parent step fails, but the child workflow keeps
running independently.

## Retry Policy (Optional)

Control backoff and retry limits for an individual step:
Expand Down
Loading
Loading