Summary
Request an opt-in asynchronous (dataflow) execution mode in which a downstream executor fires as soon as its direct predecessor completes — without waiting for the current superstep to quiesce. BSP (the current model) would remain the default.
Background
The .NET workflow runtime executes as Bulk Synchronous Parallel (BSP): executors fire in supersteps, and the next superstep begins only after the current one fully quiesces (per the execution-model docs and #1444). This is a sound default — it gives determinism, a simple concurrency model, and clean checkpoint boundaries.
However, as called out in #3840 (and experienced here), BSP has a latency cost in one specific but common graph shape, and there is currently no way to opt out of it.
The gap
When a fan-out has siblings of unequal latency, a downstream chain of the fast sibling waits for the slow sibling, even though it has no data dependency on it:
A ──fan-out──→ B (fast) ──→ D ← D depends only on B
└──→ C (slow)
- B finishes quickly, but D is one superstep deeper than B.
- D's superstep cannot start until the current superstep quiesces — i.e. until C finishes.
- So D is gated by C, despite D not depending on C.
WithIntermediateOutputFrom lets a completed executor stream its output early within a superstep, but it cannot make a not-yet-scheduled executor start sooner. Sub-workflows do not help either (see #3840): their inner supersteps share the parent scheduler and are subject to the same barrier.
Minimal repro
Synthetic workflow where "C" is gated by a TaskCompletionSource to make the barrier visible (executor bodies omitted for brevity — B and D return instantly, C awaits the gate):
// A → fan-out[B, C]; B → D
var a = new SourceExecutor(); // emits a value, then completes
var b = new FastExecutor("B");
var c = new GatedExecutor("C", gate); // blocks until gate.SetResult()
var d = new FastExecutor("D");
var wf = new WorkflowBuilder(a)
.AddFanOutEdge(a, [b, c])
.AddEdge(b, d)
.WithOutputFrom(d)
.Build();
await using var run = await InProcessExecution.RunStreamingAsync(wf, input);
await run.TrySendMessageAsync(new TurnToken(true));Measured (InProcessExecution / OffThread, .NET 10, Microsoft.Agents.AI.Workflows 1.15.0):
| Executor |
Completes at |
Notes |
| B |
~10 ms |
fast; its output (D's only input) is ready |
| C |
~10000 ms |
gated, released manually |
| D |
~10000 ms |
depends on B only — expected ~10 ms, actually waits for C |
D starts only after C finishes, because D lives in the superstep after B, and that superstep is gated by C. The same effect occurs whether the B→D chain is expressed as an edge or wrapped as a sub-workflow (BindAsExecutor).
Proposal
An opt-in mode where edges fire eagerly — a downstream executor is scheduled as soon as its direct predecessor emits, independent of parallel siblings:
- Default remains BSP, preserving determinism, checkpoint semantics, and existing tests.
- An opt-in "asynchronous / dataflow" mode (per-workflow, e.g. on
WorkflowBuilder or InProcessExecution) would trade the global barrier for lower tail latency in graphs where independent chains should not gate each other.
- A lighter alternative that would already cover the common case: a per-edge "eager" flag, opting specific edges out of the barrier while leaving the rest of the workflow BSP-synchronized.
Use case
Progressive-result / streaming pipelines (e.g. multi-arm recommendation or retrieval workflows): several independent arms compute concurrently, and each arm should surface to the client the moment it finishes. A fast arm that ends in a short refinement chain (e.g. produce candidates → judge them) currently waits for a slower sibling arm before its refinement step can even start, adding the sibling's latency to the fast arm's time-to-first-result.
Related
Summary
Request an opt-in asynchronous (dataflow) execution mode in which a downstream executor fires as soon as its direct predecessor completes — without waiting for the current superstep to quiesce. BSP (the current model) would remain the default.
Background
The .NET workflow runtime executes as Bulk Synchronous Parallel (BSP): executors fire in supersteps, and the next superstep begins only after the current one fully quiesces (per the execution-model docs and #1444). This is a sound default — it gives determinism, a simple concurrency model, and clean checkpoint boundaries.
However, as called out in #3840 (and experienced here), BSP has a latency cost in one specific but common graph shape, and there is currently no way to opt out of it.
The gap
When a fan-out has siblings of unequal latency, a downstream chain of the fast sibling waits for the slow sibling, even though it has no data dependency on it:
WithIntermediateOutputFromlets a completed executor stream its output early within a superstep, but it cannot make a not-yet-scheduled executor start sooner. Sub-workflows do not help either (see #3840): their inner supersteps share the parent scheduler and are subject to the same barrier.Minimal repro
Synthetic workflow where "C" is gated by a
TaskCompletionSourceto make the barrier visible (executor bodies omitted for brevity — B and D return instantly, C awaits the gate):Measured (
InProcessExecution/ OffThread, .NET 10,Microsoft.Agents.AI.Workflows1.15.0):D starts only after C finishes, because D lives in the superstep after B, and that superstep is gated by C. The same effect occurs whether the B→D chain is expressed as an edge or wrapped as a sub-workflow (
BindAsExecutor).Proposal
An opt-in mode where edges fire eagerly — a downstream executor is scheduled as soon as its direct predecessor emits, independent of parallel siblings:
WorkflowBuilderorInProcessExecution) would trade the global barrier for lower tail latency in graphs where independent chains should not gate each other.Use case
Progressive-result / streaming pipelines (e.g. multi-arm recommendation or retrieval workflows): several independent arms compute concurrently, and each arm should surface to the client the moment it finishes. A fast arm that ends in a short refinement chain (e.g. produce candidates → judge them) currently waits for a slower sibling arm before its refinement step can even start, adding the sibling's latency to the fast arm's time-to-first-result.
Related