Skip to content

Python: Improve the handling of intermediate outputs for workflows and orchestrations - #5623

Merged
Evan Mattson (moonbox3) merged 23 commits into
microsoft:mainfrom
moonbox3:intermediate-workflow-outputs
May 19, 2026
Merged

Python: Improve the handling of intermediate outputs for workflows and orchestrations#5623
Evan Mattson (moonbox3) merged 23 commits into
microsoft:mainfrom
moonbox3:intermediate-workflow-outputs

Conversation

@moonbox3

@moonbox3 Evan Mattson (moonbox3) commented May 4, 2026

Copy link
Copy Markdown
Contributor

Motivation and Context

Workflows had no first-class way to distinguish a designated terminal output from observational intermediate emissions. Both surfaced as type='output' events with no marker, so consumers could not separate "the workflow's answer" from "progress / reasoning along the way" without out-of-band knowledge of the graph. The same gap existed in orchestrations: participant replies and the manager / aggregator / terminator answer were all output, indistinguishable to the caller.

The original kwarg names (output_executors / intermediate_executors) also read ambiguously. They could be parsed as "the executors that produce output," when their real meaning is "which executors emit which kind of output."

This PR introduces an explicit intermediate output type, designates per-executor finality at build time, and renames the builder kwargs to match how the API is actually read.

Important

This breaking change is scoped to the experimental agent_framework_orchestrations package only; the core agent_framework kwarg rename ships with deprecation aliases.

Description

Core changes:

  • New WorkflowEvent.type == "intermediate" discriminator alongside "output". Legacy "data" becomes a deprecated alias of "intermediate".
  • Per-executor designation through WorkflowBuilder(final_output_from=[...], intermediate_output_from=[...]). Designation is fixed at build time; ctx.yield_output(...) has no per-emission flag and an executor cannot vary the label per yield (documented on the yield_output docstring).
  • WorkflowRunResult exposes get_intermediate_outputs() alongside get_outputs().
  • The workflow.as_agent() boundary forwards both output and intermediate events via an explicit allowlist (AGENT_FORWARDED_EVENT_TYPES); intermediates render as text_reasoning content, so existing .text accessors continue to return only the terminal answer.
  • WorkflowExecutor (sub-workflow embedding) now pipes child intermediate emissions up through the parent's event stream, attributed to the WorkflowExecutor's own id rather than the child's internal executor ids. This preserves encapsulation and keeps the intermediate label intact across the boundary regardless of how the parent designates the WorkflowExecutor.

Builder kwarg rename:

  • Core: output_executors -> final_output_from, intermediate_executors -> intermediate_output_from. Both old names are still accepted with DeprecationWarning via a small _coalesce_renamed_kwarg helper. Supplying both the old and new name for the same slot raises TypeError. The serialized form (Workflow.to_dict()) deliberately keeps the old wire keys output_executors / intermediate_executors for checkpoint compatibility, with a test to lock that asymmetry in place.
  • Orchestrations: output_participants -> final_output_from, intermediate_participants -> intermediate_output_from. Clean break, no shim, because the orchestrations package is still pre-1.0.

Orchestrations fix:

  • HandoffBuilder defaults final_output_from to every participant, and SequentialBuilder defaults to the last participant. Before this fix, supplying intermediate_output_from=[X] where X was in the default-final list triggered the "Participants cannot be both output and intermediate designated" overlap rejection, contradicting the documented contract that intermediate_output_from is usable on its own. The shared _resolve_participant_output_config helper now subtracts the explicit intermediates from the default-final list, so intermediate designation implicitly demotes the listed participants out of the default-final set. Regression tests added for both Handoff and Sequential.

Samples and docs:

  • samples/03-workflows/control-flow/intermediate_vs_terminal_outputs.py walks the full path: explicit designation, streaming consumption, WorkflowRunResult accessors, as_agent() translation, and now sub-workflow embedding via WorkflowExecutor showing child intermediates bubbling up.
  • samples/03-workflows/README.md and orchestration sample docstrings updated to the new kwarg names.
  • Docstrings on ctx.yield_output, WorkflowEvent.output, and WorkflowEvent.intermediate make the per-executor (not per-yield) invariant explicit, so future contributors do not propose a ctx.yield_intermediate(...) API.

Test coverage added:

  • WorkflowExecutor intermediate propagation across the sub-workflow boundary.
  • Workflow.to_dict() preserves the legacy wire keys after the kwarg rename.
  • WorkflowBuilder raises TypeError when both the deprecated alias and the new canonical kwarg are supplied (both the output and intermediate pairs).
  • HandoffBuilder(intermediate_output_from=[X]) and SequentialBuilder(intermediate_output_from=[last_participant]) do not collide with the default-final list.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

@moonbox3 Evan Mattson (moonbox3) added the python Usage: [Issues, PRs], Target: Python label May 4, 2026
Copilot AI review requested due to automatic review settings May 4, 2026 11:07
@moonbox3 Evan Mattson (moonbox3) added agent orchestration workflows Usage: [Issues, PRs], Target: Workflows labels May 4, 2026
@moonbox3 Evan Mattson (moonbox3) added the documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs label May 4, 2026
@moonbox3

Evan Mattson (moonbox3) commented May 4, 2026

Copy link
Copy Markdown
Contributor Author

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/azurefunctions/agent_framework_azurefunctions
   _app.py52914572%287, 295, 299, 314–320, 334–335, 337, 374–375, 383, 396–397, 422–423, 425–426, 428, 431, 434, 436, 438, 440–442, 446–449, 451, 453–454, 456, 467–469, 473–475, 477, 479–480, 482, 493, 499–504, 512, 514, 520–522, 528–529, 531–532, 534–537, 541, 545, 551, 562–564, 575, 674–675, 783, 791–792, 812–814, 820–822, 828–830, 863–864, 924–925, 974–975, 980, 1062, 1065, 1074–1076, 1078–1080, 1082, 1084, 1095, 1097–1100, 1102, 1104–1105, 1107, 1114–1115, 1117–1118, 1120–1121, 1123, 1127, 1137–1139, 1141–1142, 1144–1146, 1153, 1155–1156, 1158, 1179, 1184, 1196, 1268, 1358, 1373–1376, 1401
   _context.py72691%100, 104, 155, 163–164, 172
packages/core/agent_framework/_workflows
   _agent.py3627180%67, 75–81, 117–118, 211, 272, 285, 352, 363, 365, 425, 431, 448–449, 456, 458, 464, 530–531, 540, 579, 612–614, 616, 618, 620, 625, 630, 689, 719, 736, 764, 775–778, 784, 790, 794–795, 798–804, 808–809, 817, 878, 885, 891–892, 903, 935, 942, 963, 972, 976, 978–980, 987
   _agent_executor.py2001692%166, 190, 231, 255, 275–276, 356–358, 360, 370–371, 490–491, 563, 569
   _edge_runner.py1621391%58, 63, 76, 148–149, 153, 160, 221, 226, 379–380, 384, 429
   _events.py1522186%91–92, 240, 244, 246, 278, 360, 375, 390, 405, 417–419, 431–433, 435–436, 438–439, 443
   _functional.py4903193%397–398, 464, 487, 496–498, 501, 531, 712–713, 720, 968, 1000, 1003–1004, 1012–1014, 1089–1090, 1096, 1479–1480, 1482, 1489–1492, 1513, 1538
   _runner_context.py1581491%66, 80–81, 83–84, 86, 381, 400, 409, 411–412, 453, 466, 470
   _validation.py161795%131, 152, 247, 328, 331, 365, 393
   _workflow.py3382492%59, 61, 66, 90, 95, 156, 192, 400–402, 404–405, 429, 463, 627, 838, 859, 907, 919, 925, 930, 950–952
   _workflow_builder.py2101393%224, 501–505, 507, 610, 625, 654, 670, 678, 713
   _workflow_context.py1661889%57–58, 66, 70, 74–77, 79–81, 83–85, 160, 185, 297, 441
   _workflow_executor.py1943084%95, 405, 465, 488, 490, 498–499, 504, 506, 511, 513, 609–615, 619–621, 629, 634, 645, 655, 659, 665, 669, 679, 683
packages/orchestrations/agent_framework_orchestrations
   _concurrent.py1472483%59, 68–69, 97–98, 103, 121, 126, 131–132, 153, 163, 170, 244, 260, 263, 320, 350, 352–353, 355, 360, 373, 377
   _group_chat.py3327078%183, 352, 359, 392, 403–404, 410, 415, 436, 440, 453–454, 467, 482–483, 485, 501, 528–533, 535, 569–572, 574, 579–583, 677, 680, 719, 722, 725, 728, 736, 748–749, 751–752, 754–755, 757, 762, 765, 774, 780, 824–825, 829–830, 844–845, 847–848, 879–880, 946, 965, 973, 978–980, 987, 997
   _handoff.py3395284%112–113, 115, 168–178, 180, 182, 184, 189, 320, 345, 372, 398, 462, 505, 513, 517–518, 549–551, 556–558, 688, 691, 704, 766, 771, 778, 788, 790, 809, 811, 893–894, 926–927, 1039, 1046, 1118–1119, 1121
   _magentic.py5959184%73–82, 87, 91–102, 267, 278, 282, 302, 363, 372, 374, 416, 433, 442–443, 445–447, 449, 460, 603, 605, 645, 695, 731–733, 735, 745, 753–754, 821–824, 915, 921, 927, 969, 1007, 1039, 1056, 1067, 1124–1125, 1129–1131, 1155, 1179–1180, 1193, 1213, 1236, 1281–1282, 1320–1321, 1490, 1493, 1502, 1505, 1510, 1561–1562, 1603–1604, 1652, 1682, 1740, 1754, 1765
   _orchestration_request_info.py66198%162
   _participant_output_config.py84692%34, 40, 52, 55, 111, 165
   _sequential.py83692%56, 132, 143, 149, 197, 225
TOTAL34410390988% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
6908 30 💤 0 ❌ 0 🔥 1m 53s ⏱️

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 3 | Confidence: 89%

✓ Security Reliability

This PR introduces a three-state output_executors contract (None/legacy, []/strict-no-terminals, [X,...]/strict) for distinguishing terminal vs intermediate workflow events. The changes are well-structured: the _mark_msg closure correctly forwards all six Message fields (role, contents, author_name, message_id, additional_properties, raw_representation), the _LIFECYCLE_EVENT_TYPES frozenset correctly enumerates all framework-internal event types, and the _to_text_reasoning rewriter only transforms text content while passing non-text items through unchanged. The _workflow back-reference introduces a reference cycle (RunerContext→Workflow→RunerContext) but Python's cyclic GC handles this, and the single-assignment-at-build-time pattern avoids race conditions. The validation call output_executors_for_workflow or [] correctly colapses both None (legacy) and [] (strict-no-terminals) to the empty list for graph validation purposes, which matches the pre-existing behavior. No security, injection, secret-leakage, resource-leak, or unhandled-failure-mode issues were found.

✓ Test Coverage

This PR introduces a three-state output_executors contract (None/[]/[X,...]) for distinguishing intermediate vs terminal workflow outputs, with text_reasoning content mapping through as_agent(). The new test files cover the core contract well — event factories, strict-mode labeling, WorkflowAgent text_reasoning mapping, and deprecation paths. However, the orchestration test file (test_orchestration_intermediate_vs_terminal.py) only covers Sequential and Concurrent, leaving GroupChat, Handoff, Magentic, and OrchestrationRequestInfo untested despite all four having builder changes in this PR. Additionally, the _to_text_reasoning function's non-text content passthrough (function_call, data, uri, etc.) lacks a focused test, though the function is simple enough that this is lower risk.

✗ Design Approach

The main design issue is that the new forwarding rule switches from an allowlist (output/request_info) to a lifecycle blacklist, which makes workflow.as_agent() ingest orchestration bookkeeping events as if they were user-facing intermediate output. That leaks lower-layer implementation details like group_chat, handoff_sent, and magentic_orchestrator across the agent boundary, and those payloads will be stringified by the generic fallback path instead of staying on their dedicated event channel.


Automated review by moonbox3's agents

Comment thread python/packages/core/agent_framework/_workflows/_agent.py

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a stricter Python workflow output contract so orchestrations can distinguish terminal outputs from intermediate progress, and updates workflow.as_agent()/DevUI handling around those events.

Changes:

  • Adds explicit output_executors semantics to workflows: legacy unset mode, strict empty mode, and strict designated-output mode.
  • Updates orchestration builders (sequential, concurrent, group chat, magentic, handoff) to opt into explicit output designation.
  • Adds tests/samples around intermediate-vs-terminal labeling and text_reasoning translation in workflow.as_agent().

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
python/samples/03-workflows/README.md Adds sample entry for intermediate vs terminal outputs.
python/samples/03-workflows/control-flow/intermediate_vs_terminal_outputs.py New sample showing strict output designation and as_agent() behavior.
python/packages/orchestrations/tests/test_orchestration_intermediate_vs_terminal.py Adds orchestration tests for intermediate/terminal labeling.
python/packages/orchestrations/tests/test_magentic.py Updates Magentic test comments for new labeling semantics.
python/packages/orchestrations/agent_framework_orchestrations/_sequential.py Makes sequential builder explicitly designate terminal outputs.
python/packages/orchestrations/agent_framework_orchestrations/_orchestration_request_info.py Sets explicit inner-workflow output executors for approval flows.
python/packages/orchestrations/agent_framework_orchestrations/_magentic.py Makes Magentic builder explicitly designate manager/participant outputs.
python/packages/orchestrations/agent_framework_orchestrations/_handoff.py Makes handoff builder explicitly designate all participant outputs.
python/packages/orchestrations/agent_framework_orchestrations/_group_chat.py Makes group chat builder explicitly designate orchestrator/participant outputs.
python/packages/orchestrations/agent_framework_orchestrations/_concurrent.py Makes concurrent builder explicitly designate aggregator/participant outputs.
python/packages/devui/tests/devui/test_mapper.py Updates mapper tests for intermediate events.
python/packages/devui/agent_framework_devui/_mapper.py Adds partial DevUI support for intermediate workflow events.
python/packages/core/tests/workflow/test_workflow_event_factories.py Adds tests for WorkflowEvent.intermediate() and emit() deprecation.
python/packages/core/tests/workflow/test_workflow_agent_intermediate.py Adds workflow-agent tests for intermediate-to-reasoning translation.
python/packages/core/tests/workflow/test_validation.py Updates validation expectations for explicit empty output executors.
python/packages/core/tests/workflow/test_strict_mode_event_labeling.py Adds strict-mode event-labeling tests.
python/packages/core/tests/workflow/test_output_executors_contract.py Adds tests for the three-state output_executors contract.
python/packages/core/tests/workflow/test_functional_workflow.py Updates functional workflow test to use intermediate() instead of emit().
python/packages/core/tests/workflow/test_agent_run_event_typing.py Updates typing tests to use WorkflowEvent.intermediate().
python/packages/core/agent_framework/_workflows/_workflow.py Stores nullable output-executor state and updates strict-mode behavior.
python/packages/core/agent_framework/_workflows/_workflow_context.py Labels yielded outputs as output or intermediate at emission time.
python/packages/core/agent_framework/_workflows/_workflow_builder.py Implements the three-state output_executors contract and warning.
python/packages/core/agent_framework/_workflows/_runner_context.py Adds workflow back-reference and intermediate-label decision helper.
python/packages/core/agent_framework/_workflows/_events.py Adds intermediate event type/factory and deprecates emit()/data.
python/packages/core/agent_framework/_workflows/_agent.py Translates intermediate workflow events into text_reasoning on agent surfaces.

Comment thread python/packages/core/agent_framework/_workflows/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_agent.py Outdated
Comment thread python/packages/devui/agent_framework_devui/_mapper.py
Comment thread python/packages/core/agent_framework/_workflows/_workflow_builder.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_workflow.py Outdated
- Switch workflow.as_agent() forwarding to an explicit allowlist of {output,
  intermediate, data, request_info} so orchestration-internal events
  (group_chat, handoff_sent, magentic_orchestrator) stay inside the workflow
  instead of leaking into agent responses via str(data) coercion.
- Stop raising on intermediate AgentResponseUpdate in non-streaming run();
  surface the partial as a Message with text_reasoning content. The defensive
  raise still applies to terminal output events, where Update payloads would
  corrupt message ordering.
- Extend the DevUI workflow-event mapper so intermediate yields wrapping
  plain strings, Messages, and list[Message] render as visible output items
  instead of generic completed-trace events.
- Add orchestration coverage for GroupChat, Handoff, and Magentic builders
  (default vs intermediate_outputs=True; structural where end-to-end is heavy).
Replace the ``Workflow._output_executors`` list and the
``RunnerContext.should_label_as_intermediate`` Protocol method with a single
immutable ``OutputDesignation`` value type owned by ``Workflow``. Thread the
designation as a parameter through the existing call chain (Runner ->
EdgeRunner -> Executor -> WorkflowContext) so ``yield_output`` consults the
threaded snapshot directly rather than calling back into the runner context.

Removes the ``InProcRunnerContext._workflow`` back-reference and the
``WorkflowBuilder.build()`` assignment that wired it up. Adds the public
predicate ``Workflow.is_terminal_executor(executor_id)`` for external
observers; ``OutputDesignation`` itself stays package-internal.

Key decisions
- ``OutputDesignation.designated`` is ``frozenset[str] | None`` -- ``None``
  preserves legacy "every yield is type='output'" behavior, any frozenset
  (including empty) opts into strict mode. The ``DeprecationWarning`` for
  legacy mode at build time is unchanged.
- ``output_designation`` is an optional parameter on ``Runner``,
  ``EdgeRunner.send_message``, ``EdgeRunner._execute_on_target``,
  ``Executor.execute``, ``Executor._create_context_for_handler``, and
  ``WorkflowContext.__init__``. Each defaults to legacy ``OutputDesignation()``
  so direct callers (Azure Functions ``CapturingRunnerContext``,
  ``test_runner`` recording fixtures) keep working without ceremony.
- The workflow-level filter in ``_run_core`` reads ``self._output_designation``
  live, preserving today's semantics where mutating the designation after
  build still affects subsequent runs (used by two existing tests).
- ``Workflow.to_dict()`` continues to emit ``"output_executors":
  list[str] | None`` (sorted from the frozenset). Checkpoint format unchanged.

Files changed
- _workflow.py: add ``OutputDesignation`` dataclass; replace
  ``_output_executors`` with ``_output_designation``; add
  ``is_terminal_executor``; delete ``_should_yield_output_event``.
- _runner_context.py: drop ``should_label_as_intermediate`` Protocol method
  and ``InProcRunnerContext`` impl; drop ``_workflow`` back-reference.
- _workflow_builder.py: remove ``context._workflow = workflow`` assignment.
- _runner.py, _edge_runner.py, _executor.py, _workflow_context.py: thread
  ``output_designation`` parameter through the call chain.
- tests/workflow/test_output_designation.py (new): three-state coverage of
  the value type plus the public predicate delegation.
- tests/workflow/test_workflow_builder.py, test_validation.py,
  test_workflow.py, test_runner.py and
  orchestrations/tests/test_orchestration_intermediate_vs_terminal.py:
  switch probes from ``_output_executors`` set checks to
  ``get_output_executors`` / ``is_terminal_executor``; update two
  post-build mutation tests to set ``_output_designation`` instead.

Verification
- core/tests/workflow/, orchestrations/tests/, azurefunctions/tests/:
  1119 passed, 42 skipped, 2 xfailed.
- ``uv run poe lint``: clean.
- ``uv run poe typing``: only the pre-existing
  ``_AGENT_FORWARDED_EVENT_TYPES`` pyright warning from 394bcd6 remains.

Notes for next iteration
- The builder's own ``_output_executors`` attribute (``list[Executor |
  SupportsAgentRun]``) is intentionally untouched; the issue scoped the
  rename to the workflow attribute.
- Adjacent review candidates (twin ``WorkflowAgent`` translators,
  ``_AGENT_FORWARDED_EVENT_TYPES`` kind classifier,
  ``_event_origin_context`` ContextVar removal, ``WorkflowEvent`` ADT
  split, legacy-mode removal) remain out of scope.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 31 out of 31 changed files in this pull request and generated 4 comments.

Comment thread python/packages/core/agent_framework/_workflows/_workflow.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_agent.py
Comment thread python/packages/core/tests/workflow/test_workflow.py
Comment thread python/packages/core/agent_framework/_workflows/_agent.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_workflow_builder.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_workflow_context.py Outdated
Key decisions

- Extend the internal OutputDesignation value type from terminal-only membership to output/intermediate/hidden classification. Legacy mode remains outputs=None, so workflows built without output_executors or intermediate_executors still label every yield_output as type='output'.

- WorkflowBuilder now accepts intermediate_executors. Providing either designation enters explicit mode; output executors emit output, intermediate executors emit intermediate, and unlisted yield_output payloads are hidden from caller-facing events while remaining in executor_completed data.

- Empty explicit designation, duplicate entries, overlaps, unknown executors, and designated executors without workflow output annotations fail build validation. Existing orchestration builders pass intermediate-capable participants through intermediate_executors to preserve current intermediate_outputs behavior until participant-oriented designation lands.

Files changed

- packages/core/agent_framework/_workflows/_workflow.py, _workflow_builder.py, _workflow_context.py, _validation.py, _events.py

- packages/core/tests/workflow/test_output_designation.py, test_output_executors_contract.py, test_strict_mode_event_labeling.py, test_validation.py, test_workflow.py, test_workflow_agent_intermediate.py

- packages/orchestrations/agent_framework_orchestrations/_sequential.py, _concurrent.py, _group_chat.py, _magentic.py

- packages/core/AGENTS.md

Verification

- uv run pytest packages/core/tests/workflow packages/orchestrations/tests packages/devui/tests/devui/test_mapper.py -q

- uv run pytest packages/azurefunctions/tests -q

- uv run poe lint

- uv run poe typing fails only on pre-existing packages/core/agent_framework/_workflows/_agent.py _AGENT_FORWARDED_EVENT_TYPES private-use pyright error.

Notes for next iteration

- issues/03-core-workflow-explicit-designation.md was moved to issues/done but issues/ remains untracked and intentionally excluded from this commit.

- Slice 4 should tighten workflow.as_agent() mapping for hidden emissions and streaming-only update payloads; Slice 5 should replace orchestration intermediate_outputs with participant-oriented designation.
Key decisions

- Treat AgentResponseUpdate as a streaming-only payload across the workflow.as_agent() adapter, so non-streaming agent runs now reject both terminal output and intermediate workflow events carrying updates.
- Keep streaming classification behavior explicit: terminal update payloads remain normal text content, while intermediate update payloads are rewritten to text_reasoning content.
- Add explicit-mode coverage proving hidden yield_output emissions do not appear in non-streaming AgentResponse messages or streaming AgentResponseUpdate chunks.

Files changed

- packages/core/agent_framework/_workflows/_agent.py
- packages/core/tests/workflow/test_workflow_agent_intermediate.py

Verification

- uv run pytest packages/core/tests/workflow/test_workflow_agent_intermediate.py -q
- uv run pytest packages/core/tests/workflow/test_workflow_agent.py packages/core/tests/workflow/test_workflow_agent_intermediate.py -q
- uv run pytest packages/core/tests/workflow packages/orchestrations/tests packages/devui/tests/devui/test_mapper.py -q
- uv run poe lint
- uv run poe typing fails only on the pre-existing packages/core/agent_framework/_workflows/_agent.py _AGENT_FORWARDED_EVENT_TYPES private-use pyright error.

Blockers or notes for next iteration

- issues/04-workflow-as-agent-output-mapping.md was moved to issues/done/ but issues/ remains untracked and intentionally excluded from this commit.
- Slice 5 should replace orchestration intermediate_outputs with participant-oriented designation.
@moonbox3
Evan Mattson (moonbox3) force-pushed the intermediate-workflow-outputs branch from 709d506 to 8199693 Compare May 12, 2026 06:10
Key decisions

- Replace orchestration intermediate_outputs with participant-oriented output_participants and intermediate_participants across Sequential, Concurrent, GroupChat, Magentic, and Handoff builders.
- Keep synthetic final executors terminal by default for Concurrent, GroupChat, and Magentic; keep Sequential's final participant terminal by default; keep Handoff participants terminal by default.
- Centralize participant designation validation for empty explicit designation, duplicates, overlaps, and unknown participants, then map validated participants to workflow output/intermediate executors.

Files changed

- packages/orchestrations/agent_framework_orchestrations/_participant_designation.py
- packages/orchestrations/agent_framework_orchestrations/_sequential.py
- packages/orchestrations/agent_framework_orchestrations/_concurrent.py
- packages/orchestrations/agent_framework_orchestrations/_group_chat.py
- packages/orchestrations/agent_framework_orchestrations/_magentic.py
- packages/orchestrations/agent_framework_orchestrations/_handoff.py
- packages/orchestrations/tests/test_orchestration_intermediate_vs_terminal.py
- packages/orchestrations/tests/test_magentic.py

Blockers or notes for next iteration

- issues/05-orchestration-participant-designation.md was moved to issues/done/ but issues/ remains untracked and intentionally excluded from this commit.
- Slice 7 should migrate samples and docs away from intermediate_outputs to the new participant designation API.
- uv run poe typing still fails only on the pre-existing packages/core/agent_framework/_workflows/_agent.py _AGENT_FORWARDED_EVENT_TYPES private-use pyright error.
Key decisions

- Replace sample usage of the removed orchestration intermediate_outputs boolean with participant-oriented intermediate_participants designation.
- Update raw workflow guidance to show output_executors together with intermediate_executors, and document that unlisted yields are hidden in explicit designation mode.
- Keep orchestration final outputs terminal while streaming designated participant responses as intermediate progress, including workflow.as_agent() samples where intermediates map to text_reasoning content.
- Refresh workflow and orchestration README guidance plus the changelog reference so public docs no longer point users at intermediate_outputs.

Files changed

- CHANGELOG.md
- packages/orchestrations/README.md
- samples/README.md
- samples/03-workflows/README.md
- samples/03-workflows/control-flow/intermediate_vs_terminal_outputs.py
- samples/03-workflows/orchestrations/README.md
- samples/03-workflows/orchestrations/group_chat_agent_manager.py
- samples/03-workflows/orchestrations/group_chat_philosophical_debate.py
- samples/03-workflows/orchestrations/group_chat_simple_selector.py
- samples/03-workflows/orchestrations/magentic.py
- samples/03-workflows/orchestrations/magentic_human_plan_review.py
- samples/03-workflows/orchestrations/sequential_chain_only_agent_responses.py
- samples/03-workflows/agents/group_chat_workflow_as_agent.py
- samples/03-workflows/agents/magentic_workflow_as_agent.py
- samples/03-workflows/agents/sequential_workflow_as_agent.py
- samples/semantic-kernel-migration/orchestrations/group_chat.py
- samples/semantic-kernel-migration/orchestrations/magentic.py

Blockers or notes for next iteration

- issues/07-samples-and-docs-explicit-output-designation.md was moved to issues/done/ but issues/ remains untracked and intentionally excluded from this commit.
- issues/06-devui-intermediate-event-rendering.md remains present and appears already satisfied by existing DevUI mapper/tests from the prior implementation slice.
- PRD-explicit-workflow-output-designation.md remains untracked and intentionally excluded from this commit.
Key decisions

- Preserve workflow output designation metadata on visible DevUI output messages and text deltas so intermediate/data emissions remain distinguishable from terminal output.
- Render intermediate workflow message items in the execution timeline using executor metadata, while excluding them from the final workflow result aggregation.
- Keep terminal output message rendering unchanged and retain legacy data events on the intermediate compatibility path.

Files changed

- packages/devui/agent_framework_devui/_mapper.py
- packages/devui/frontend/src/components/features/workflow/execution-timeline.tsx
- packages/devui/frontend/src/components/features/workflow/workflow-view.tsx
- packages/devui/frontend/src/types/openai.ts
- packages/devui/tests/devui/test_mapper.py

Blockers or notes for next iteration

- issues/06-devui-intermediate-event-rendering.md was moved to issues/done/ but issues/ remains untracked and intentionally excluded from this commit.
- PRD-explicit-workflow-output-designation.md remains untracked and intentionally excluded from this commit.
- uv run poe typing still fails only on the pre-existing packages/core/agent_framework/_workflows/_agent.py _AGENT_FORWARDED_EVENT_TYPES private-use pyright error.
output_participants -> final_output_from, intermediate_participants ->
intermediate_output_from. The old names read like categories of
participant; the new names make it clear the kwarg designates which
participants' outputs surface as final vs. intermediate events.
Adds final_output_from / intermediate_output_from as canonical kwargs on
Workflow and WorkflowBuilder. Old output_executors / intermediate_executors
kwargs continue to work but emit DeprecationWarning via a shared coalesce
helper that also rejects supplying both. Wire-format keys in to_dict()
stay as output_executors / intermediate_executors so checkpoint
compatibility is preserved.

Internal call sites in orchestrations and samples updated to the new
names so users following sample code learn the canonical vocabulary;
legacy callers still work with a one-shot warning.
Comment thread python/packages/core/agent_framework/_workflows/_runner.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_workflow.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_workflow.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_workflow.py Outdated
Comment thread python/packages/core/agent_framework/_workflows/_workflow.py Outdated
Key decisions:\n- Make output_from the canonical workflow-output allow-list and keep output_executors/final_output_from as deprecated compatibility aliases.\n- Treat empty output_from/intermediate_output_from lists as explicit selections and keep validation responsible for empty, duplicate, overlap, and unknown selections.\n- Remove the branch-only public intermediate_executors WorkflowBuilder kwarg while preserving legacy wire keys in to_dict().\n\nFiles changed:\n- packages/core/agent_framework/_workflows/_workflow.py\n- packages/core/agent_framework/_workflows/_workflow_builder.py\n- packages/core/agent_framework/_workflows/_workflow_context.py\n- packages/core/agent_framework/_workflows/_agent.py\n- packages/core/agent_framework/_workflows/_agent_executor.py\n- packages/core/tests/workflow/* output-selection coverage updates\n- packages/core/AGENTS.md\n- issues/done/001-canonical-list-based-output-selection.md\n\nBlockers/notes:\n- Orchestration builders still pass final_output_from internally; follow-up issue 004 should migrate them to output_from.\n- Legacy omitted-selection behavior and explicit all/all_other literals are left for issues 002 and 003.
Key decisions:
- Treat output_from='all' as an explicit workflow-output selection sentinel and expand it at build time to executors with declared workflow output types.
- Keep omitted output selections in legacy all-output mode with a deprecation warning that names output_from and intermediate_output_from and points to output_from='all'.
- Reject intermediate_output_from='all' at construction because the all-output literal is output-only for this issue.

Files changed:
- packages/core/agent_framework/_workflows/_workflow_builder.py
- packages/core/tests/workflow/test_output_executors_contract.py
- issues/done/002-explicit-all-output-and-legacy-migration.md

Blockers/notes:
- all_other intermediate-output selection remains for issue 003.
- Workflow-as-agent/orchestration parity remains for issue 004.
Key decisions:
- Treat intermediate_output_from='all_other' as an explicit intermediate-output selection sentinel and expand it at build time after the workflow graph is complete.
- Expand all_other to output-capable executors not selected by output_from; omitted or empty output_from selects no workflow outputs, while output_from='all' leaves an empty intermediate selection.
- Keep output_from='all_other' invalid so all_other remains intermediate-output-only and runtime classification still receives concrete executor-id sets.

Files changed:
- packages/core/agent_framework/_workflows/_workflow_builder.py
- packages/core/tests/workflow/test_output_executors_contract.py
- issues/done/003-all-other-intermediate-output-selection.md

Blockers/notes:
- Workflow-as-agent and orchestration parity remains for issue 004.
- Full documentation updates remain for issue 005.
Key decisions:
- Expose output_from on sequential, concurrent, group chat, handoff, and magentic builders while keeping final_output_from as a deprecated compatibility alias.
- Resolve orchestration participant selections through the same explicit rules as workflows: output_from='all', intermediate_output_from='all_other', hidden unselected participant payloads, and overlap/duplicate/unknown/invalid-literal validation.
- Continue preserving documented orchestration defaults by always designating each pattern's terminal internal executor where applicable.

Files changed:
- packages/orchestrations/agent_framework_orchestrations/_participant_output_config.py
- packages/orchestrations/agent_framework_orchestrations/_sequential.py
- packages/orchestrations/agent_framework_orchestrations/_concurrent.py
- packages/orchestrations/agent_framework_orchestrations/_group_chat.py
- packages/orchestrations/agent_framework_orchestrations/_handoff.py
- packages/orchestrations/agent_framework_orchestrations/_magentic.py
- packages/orchestrations/agent_framework_orchestrations/_orchestration_request_info.py
- packages/orchestrations/tests/test_orchestration_intermediate_vs_terminal.py
- issues/done/004-workflow-as-agent-and-orchestration-parity.md

Blockers/notes:
- Full documentation and sample migration wording remains for issue 005.
- Existing tests that intentionally use final_output_from now emit the new deprecation warning.
Key decisions:
- Use Workflow Output and Intermediate Output as the developer-facing terms for selected caller-facing emissions.
- Document output_from and intermediate_output_from as the canonical API, with output_from as an allow-list and unselected payloads hidden unless explicitly selected as intermediate.
- Add scenario and invalid-selection tables for workflow and orchestration docs, including legacy omission warnings, output_from='all', intermediate_output_from='all_other', list selections, invalid literals, overlap, duplicates, unknown selections, and empty explicit selections.
- Migrate samples away from final_output_from and output_executors except where compatibility aliases are explicitly documented.

Files changed:
- packages/core/AGENTS.md
- packages/orchestrations/README.md
- packages/orchestrations/agent_framework_orchestrations/_handoff.py
- packages/orchestrations/agent_framework_orchestrations/_sequential.py
- samples/03-workflows/README.md
- samples/03-workflows/control-flow/intermediate_vs_terminal_outputs.py
- samples/03-workflows/human-in-the-loop/agents_with_approval_requests.py
- samples/03-workflows/orchestrations/README.md
- samples/04-hosting/foundry-hosted-agents/responses/05_workflows/main.py
- scripts/sample_validation/create_dynamic_workflow_executor.py
- issues/done/005-document-output-selection-contract.md

Blockers/notes:
- Direct full Ruff on scripts/sample_validation/create_dynamic_workflow_executor.py still reports pre-existing docstring/print/line-length issues outside this docs migration; syntax-focused checks for changed files pass.
- No remaining AFK issue files are present under issues/.
@moonbox3
Evan Mattson (moonbox3) requested a review from a team as a code owner May 15, 2026 08:56
Comment thread python/issues/done/001-canonical-list-based-output-selection.md Outdated
Comment thread python/samples/03-workflows/orchestrations/group_chat_philosophical_debate.py Outdated
@moonbox3
Evan Mattson (moonbox3) force-pushed the intermediate-workflow-outputs branch from 9567f34 to 9e796d8 Compare May 18, 2026 09:12
@moonbox3
Evan Mattson (moonbox3) added this pull request to the merge queue May 19, 2026
Merged via the queue into microsoft:main with commit 3bbc815 May 19, 2026
37 checks passed
@github-project-automation github-project-automation Bot moved this from In Review to Done in Agent Framework May 19, 2026
@moonbox3 Evan Mattson (moonbox3) added orchestration Usage: [Issues, PRs], Target: multi-agent orchestration (high-level patterns) and removed agent orchestration labels Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs orchestration Usage: [Issues, PRs], Target: multi-agent orchestration (high-level patterns) python Usage: [Issues, PRs], Target: Python workflows Usage: [Issues, PRs], Target: Workflows

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants