Skip to content

Python: [BREAKING] Standardize model selection on model - #4999

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 11 commits into
microsoft:mainfrom
eavanvalkenburg:anthropic_clients
Apr 1, 2026
Merged

Python: [BREAKING] Standardize model selection on model#4999
Eduard van Valkenburg (eavanvalkenburg) merged 11 commits into
microsoft:mainfrom
eavanvalkenburg:anthropic_clients

Conversation

@eavanvalkenburg

@eavanvalkenburg Eduard van Valkenburg (eavanvalkenburg) commented Mar 31, 2026

Copy link
Copy Markdown
Member

Motivation and Context

The Python codebase still mixed model_id, deployment_name, and other legacy model-selection aliases across core types, provider clients, env vars, telemetry sourcing, docs, and samples. The Anthropic package was the most visible gap because it still exposed model_id and lacked first-class Foundry, Bedrock, and Vertex wrapper clients, but the inconsistency extended well beyond Anthropic.

This PR standardizes AF-owned Python model-selection surfaces on model, updates the related configuration and documentation surfaces, and adds the Anthropic transport wrappers. This is a breaking change because callers and configuration now need to use the new model-based API and env-var names.

Description

  • standardizes AF-owned Python model-selection surfaces on model across shared core types/runtime, provider clients, helper packages, and provider settings/env vars
  • adds slim Anthropic wrapper clients for Foundry, Bedrock, and Vertex, and exports them through agent_framework.anthropic, agent_framework.foundry, agent_framework.amazon, and agent_framework.google
  • updates telemetry model sourcing to read from model while keeping the existing spec-defined emitted attribute keys unchanged
  • removes AF-owned model_id / deployment-style aliases from docs, samples, README and .env.example files, including the follow-up Foundry-backed evaluation and workflow sample env-var cleanup
  • preserves only externally required wire keys where third-party contracts still require names like deployment_name or model_name

Fixes #5004

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

Copilot AI review requested due to automatic review settings March 31, 2026 15:40
@markwallace-microsoft Mark Wallace (markwallace-microsoft) added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python labels Mar 31, 2026
@github-actions github-actions Bot changed the title [BREAKING] Refactor Anthropic model option and provider clients Python: [BREAKING] Refactor Anthropic model option and provider clients Mar 31, 2026
@markwallace-microsoft

Mark Wallace (markwallace-microsoft) commented Mar 31, 2026

Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/ag-ui/agent_framework_ag_ui
   _types.py420100% 
packages/anthropic/agent_framework_anthropic
   _bedrock_client.py30196%153
   _chat_client.py4373591%446, 449, 530, 623, 625, 768, 795–796, 874, 904–905, 950, 966–967, 974–976, 980–982, 986–989, 1103, 1113, 1165, 1286, 1313–1314, 1331, 1344, 1357, 1382–1383
   _foundry_client.py34197%153
   _vertex_client.py28196%146
packages/azure-ai-search/agent_framework_azure_ai_search
   _context_provider.py3561097%120–121, 524, 575–576, 699–700, 878–879, 926
packages/azure-ai/agent_framework_azure_ai
   _embedding_client.py1131289%152, 167–170, 174, 178, 228, 238, 249, 258, 284
   _shared.py60100% 
packages/bedrock/agent_framework_bedrock
   _chat_client.py38010073%298–299, 315–324, 329, 345–351, 354–355, 363, 380, 389, 400, 402, 404, 422–423, 444, 457, 469, 472, 480–481, 484–485, 487–488, 493–495, 497, 507–508, 530, 537, 546–547, 549–550, 552–554, 556, 558–559, 565–567, 570–571, 577–580, 586–596, 599, 618, 623, 665–666, 679, 705, 717, 722, 731, 735, 743–744, 748, 750–757
   _embedding_client.py791383%129–140, 152
packages/core/agent_framework
   _agents.py4165287%461, 470, 525, 1020, 1072, 1146–1150, 1210, 1238, 1275, 1296, 1316–1317, 1322, 1369, 1411, 1433, 1435, 1448, 1454, 1499, 1501, 1510–1515, 1520, 1522, 1528–1529, 1536, 1538–1539, 1547–1548, 1551–1553, 1563–1568, 1572, 1577, 1579
   _clients.py138794%324, 376, 532–535, 650
   _mcp.py6113095%153, 294, 354–355, 484, 546, 559, 561–564, 583–584, 597–600, 602–603, 607, 670–672, 1031, 1144, 1196–1197, 1200, 1218, 1671
   _middleware.py3661695%61, 64, 69, 798, 814, 816, 818, 951, 954, 981, 983, 1114, 1118, 1300, 1304, 1372
   _serialization.py119595%388, 515, 532, 542, 610
   _settings.py1161587%92, 99, 115, 126, 130, 141–143, 145, 148, 249–250, 258–259, 264
   _types.py10648592%58, 67–68, 122, 127, 146, 148, 152, 156, 158, 160, 162, 180, 184, 210, 232, 237, 242, 246, 276, 686–687, 846–847, 1232, 1304, 1339, 1359, 1369, 1421, 1553–1555, 1745, 1836–1841, 1866, 2040, 2052, 2075, 2315, 2339, 2434, 2665, 2871, 2940, 2951, 2953–2957, 2959, 2962–2970, 2980, 3184–3186, 3189–3191, 3195, 3200, 3204, 3288–3290, 3319, 3396–3400
   observability.py7498488%377, 379–380, 383, 386, 389–390, 395–396, 402–403, 409–410, 417, 419–421, 424–426, 431–432, 438–439, 445–446, 453, 610–611, 739, 743–745, 747, 751–752, 756, 794, 796, 807–809, 811–813, 817, 825, 949–950, 1112, 1349–1350, 1448–1453, 1460–1463, 1467–1475, 1482, 1547–1551, 1604–1605, 1739, 1831, 2028, 2246, 2248
packages/declarative/agent_framework_declarative
   _loader.py2535877%574, 576, 603–613, 655–658, 661–663, 666, 668–670, 673, 681, 698–703, 755–760, 778–783, 791, 793, 798, 801–804, 807–808, 811–812, 826–827, 857
packages/foundry/agent_framework_foundry
   _chat_client.py1361986%80, 82–84, 88–89, 93, 186, 217, 295, 356, 358, 446, 450–451, 453–456
packages/foundry_local/agent_framework_foundry_local
   _foundry_local_client.py46589%197, 201–204
packages/ollama/agent_framework_ollama
   _chat_client.py208995%393, 395, 406, 410–411, 419, 430, 509, 515
   _embedding_client.py53296%117, 149
packages/openai/agent_framework_openai
   _chat_client.py85512885%520–523, 527–528, 533–534, 544–545, 552, 567–573, 594, 602, 625, 743, 842, 901, 903, 905, 907, 973, 987, 1067, 1077, 1082, 1125, 1202, 1219, 1232, 1289, 1382, 1387, 1391–1393, 1397–1398, 1464, 1493, 1499, 1509, 1515, 1520, 1526, 1531–1532, 1593, 1615–1616, 1631–1632, 1650–1651, 1692–1695, 1857, 1895–1896, 1912, 1914, 1994–2002, 2124, 2159, 2174, 2194–2204, 2217, 2228–2232, 2246, 2260–2271, 2280, 2312–2315, 2323–2324, 2326–2328, 2342–2344, 2354–2355, 2361, 2376
   _chat_completion_client.py3562892%428, 524–525, 529, 666, 749–756, 758–761, 771, 849, 851, 868, 889, 917, 930, 954, 974, 1014, 1289
   _embedding_client.py77396%238, 250, 284
   _shared.py1351489%239–241, 253, 263, 271, 275, 281, 300–301, 320, 339–340, 342
TOTAL26989318388% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
5332 20 💤 0 ❌ 0 🔥 1m 25s ⏱️

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aligns the Python Anthropic integration with newer framework naming by hard-renaming model_idmodel, adds first-class wrapper clients for Anthropic-hosted transports (Foundry/Bedrock/Vertex), and exposes these clients consistently via agent_framework.* provider namespaces. It also updates core option handling and observability to recognize model, and refreshes docs/samples/tests accordingly.

Changes:

  • Breaking rename of Anthropic configuration and options surface from model_id to model (including env var updates and validation).
  • Added Anthropic wrapper clients for Azure AI Foundry, Amazon Bedrock, and Google Vertex AI, plus new agent_framework.google namespace export.
  • Updated core Agent option handling and observability to accept model, with corresponding tests and doc updates.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
python/samples/02-agents/providers/anthropic/README.md Updates sample env var docs to ANTHROPIC_CHAT_MODEL and Foundry vars.
python/samples/02-agents/providers/anthropic/anthropic_skills.py Adds typing ignores to keep sample checks passing with current SDK stubs.
python/samples/02-agents/providers/anthropic/anthropic_foundry.py Switches sample to new AnthropicFoundryClient wrapper.
python/packages/core/tests/core/test_observability.py Adds coverage that telemetry captures options["model"].
python/packages/core/tests/core/test_agents.py Adds coverage for agent default model selection via client .model.
python/packages/core/agent_framework/observability.py Expands model resolution to prefer model while still accepting model_id.
python/packages/core/agent_framework/google/init.py New lazy-loading namespace exporting Vertex Anthropic clients.
python/packages/core/agent_framework/google/init.pyi Type stub for the new agent_framework.google namespace.
python/packages/core/agent_framework/foundry/init.py Adds lazy exports for Foundry Anthropic wrapper clients.
python/packages/core/agent_framework/foundry/init.pyi Adds type exports for Foundry Anthropic wrapper clients.
python/packages/core/agent_framework/anthropic/init.py Adds lazy exports for Bedrock/Foundry/Vertex wrapper clients.
python/packages/core/agent_framework/anthropic/init.pyi Adds stub exports for new Anthropic wrapper/raw clients.
python/packages/core/agent_framework/amazon/init.py Adds lazy exports for Bedrock Anthropic wrapper/raw clients.
python/packages/core/agent_framework/amazon/init.pyi Adds stub exports for Bedrock Anthropic wrapper/raw clients.
python/packages/core/agent_framework/_types.py Adds model to common ChatOptions base typing.
python/packages/core/agent_framework/_agents.py Updates Agent default/runtime option building to prefer model.
python/packages/anthropic/tests/test_anthropic_provider_clients.py New tests for wrapper/raw client construction and layer ordering.
python/packages/anthropic/tests/test_anthropic_client.py Updates tests for model rename and validates model_id rejection.
python/packages/anthropic/tests/conftest.py Updates Anthropic env fixture to ANTHROPIC_CHAT_MODEL.
python/packages/anthropic/README.md Documents new transport wrapper clients.
python/packages/anthropic/AGENTS.md Updates docs to use model= in examples and lists new clients.
python/packages/anthropic/agent_framework_anthropic/_chat_client.py Implements model rename and rejects model_id in Anthropic options.
python/packages/anthropic/agent_framework_anthropic/_foundry_client.py Adds Foundry wrapper/raw client.
python/packages/anthropic/agent_framework_anthropic/_bedrock_client.py Adds Bedrock wrapper/raw client.
python/packages/anthropic/agent_framework_anthropic/_vertex_client.py Adds Vertex wrapper/raw client.
python/packages/anthropic/agent_framework_anthropic/init.py Re-exports new wrapper/raw clients from the Anthropic package.

Comment thread python/packages/core/agent_framework/_agents.py Outdated
Comment thread python/packages/core/agent_framework/_agents.py Outdated
Comment thread python/packages/core/agent_framework/_types.py Outdated
Comment thread python/packages/core/tests/core/test_agents.py

@TaoChenOSU Tao Chen (TaoChenOSU) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 4 | Confidence: 79%

✗ Correctness

The PR adds Anthropic Bedrock, Foundry, and Vertex provider clients and renames model_id to model across the Anthropic package. However, there is a critical runtime bug: the _process_message and _process_stream_event methods now pass model= when constructing ChatResponse and ChatResponseUpdate, but neither class accepts a model keyword argument—they only accept model_id. This will cause a TypeError at runtime on every non-streaming and streaming response. The core _merge_options, Agent.__init__, and observability changes look correct and properly handle the model/model_id mutual-exclusion logic.

✓ Security Reliability

✓ Test Coverage

The PR adds three new Anthropic provider clients (Bedrock, Foundry, Vertex), renames model_id to model across the Anthropic package, and updates _merge_options / Agent to handle model/model_id mutual exclusion. Test coverage is generally good: the new provider clients have construction and validation tests, the model_idmodel rename is tested, and _merge_options mutual-exclusion is partially tested. However, there are two notable coverage gaps: (1) the Foundry client's base_url code path is untested, and (2) the initial _merge_options cleanup when the base dict itself contains both model and model_id has no test.

✗ Design Approach

This PR migrates the Anthropic client family from model_id to model and introduces three new provider clients (Foundry, Bedrock, Vertex). The provider client structure, mutual-exclusion logic in _merge_options, and namespace layout are all sound. One significant design issue stands out: removing "model_id": "model" from OPTION_TRANSLATIONS and replacing it with a hard raise ValueError in _prepare_options is a breaking change that contradicts the shared ChatOptions contract. _ChatOptionsBase still declares model_id: str as a valid key (other clients still consume it), so the type system tells users the key is valid but the Anthropic runtime throws. Users calling the Anthropic client directly with options={"model_id": ".."} get an opaque runtime error with no deprecation period; the old translation approach was the correct design. A secondary, minor inconsistency exists in _vertex_client.py where access_token bypasses load_settings while all other credentials in Bedrock and Foundry clients go through it.

Flagged Issues

  • ChatResponse.__init__ and ChatResponseUpdate.__init__ do not accept a model keyword argument (only model_id). The Anthropic client now passes model= to both constructors, which will crash with TypeError at runtime on every non-streaming and streaming response.
  • _prepare_options hard-raises ValueError when model_id appears in runtime options, but _ChatOptionsBase still declares model_id: str as a valid key and other framework clients consume it. This contract violation means the type system permits model_id while the Anthropic runtime rejects it. The original OPTION_TRANSLATIONS = {"model_id": "model"} pattern was the correct design—it translated silently (or could emit a DeprecationWarning) without breaking calers.

Suggestions

  • Add a test for RawAnthropicFoundryClient when base_url is provided instead of resource—the Foundry client has two distinct construction branches and only the resource path is currently covered.
  • Add a _merge_options test where the base dict already contains both model and model_id to cover the cleanup logic (e.g., _merge_options({'model': 'a', 'model_id': 'b'}, {}) should drop model_id).
  • Consider adding an end-to-end test exercising Agent.run(model='override-model') to verify _prepare_run_context model/model_id handling at runtime, not just at Agent.__init__ time.
  • In _vertex_client.py, access_token is resolved from the constructor argument directly and never loaded from settings/env, unlike credentials in the Bedrock and Foundry clients. Consider adding it to AnthropicVertexSettings for consistency, or document that access_token is constructor-only.
  • The agent_framework.google namespace currently contains only Anthropic Vertex clients. Consider adding a docstring note clarifying this is limited to the Anthropic SDK's Vertex transport, to avoid misleading users expecting native Gemini/VertexAI support.

Automated review by TaoChenOSU's agents

Comment thread python/packages/anthropic/agent_framework_anthropic/_chat_client.py
Comment thread python/packages/anthropic/agent_framework_anthropic/_chat_client.py
Comment thread python/packages/anthropic/agent_framework_anthropic/_chat_client.py
Comment thread python/packages/anthropic/tests/test_anthropic_provider_clients.py
Comment thread python/packages/core/tests/core/test_agents.py Outdated
@eavanvalkenburg Eduard van Valkenburg (eavanvalkenburg) changed the title Python: [BREAKING] Refactor Anthropic model option and provider clients Python: [BREAKING] Standardize model selection on model Apr 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 1, 2026
Rename the Anthropic client model option from model_id to model, add provider-specific Anthropic wrappers for Foundry, Bedrock, and Vertex, and expose them through the Anthropic, Foundry, Amazon, and Google namespaces. Update core option handling, docs, samples, and tests accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cast the Anthropic beta client to Any in the skills sample so the pre-commit sample pyright check no longer fails on beta skills and files endpoints that are not exposed by the current SDK stubs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Retrigger PR validation after an unrelated Copilot review workflow SAML failure and a transient external tau2 git fetch failure in the Windows Python test setup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Apr 1, 2026
Merged via the queue into microsoft:main with commit 6acab3d Apr 1, 2026
43 checks passed
@eavanvalkenburg
Eduard van Valkenburg (eavanvalkenburg) deleted the anthropic_clients branch June 30, 2026 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: move all providers to use model instead of model_id

6 participants