Skip to content

Python: skip hosted tools test on transient upstream MCP errors #5296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Evan Mattson (moonbox3) merged 1 commit into from
Apr 16, 2026
Merged

Python: skip hosted tools test on transient upstream MCP errors #5296

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions python/packages/anthropic/tests/test_anthropic_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1504,9 +1504,7 @@ async def test_anthropic_client_integration_function_calling() -> None:
@skip_if_anthropic_integration_tests_disabled
async def test_anthropic_client_integration_hosted_tools() -> None:
"""Integration test for hosted tools."""
local_mcp_url = os.environ.get("LOCAL_MCP_URL", "")
if not local_mcp_url or not local_mcp_url.startswith(("http://", "https://")):
pytest.skip("LOCAL_MCP_URL not set or not an HTTP URL; skipping hosted tools test")
import anthropic

Comment on lines +1507 to 1508

Copilot AI Apr 16, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import anthropic is added inside the test function, but this module is already required at import-time in this file (e.g., from anthropic.types.beta import ... near the top). Consider moving this import to the module import section (or importing only the needed exception types) for consistent import ordering and to avoid a redundant local import.

Copilot uses AI. Check for mistakes.
client = AnthropicClient()

Expand All @@ -1515,15 +1513,23 @@ async def test_anthropic_client_integration_hosted_tools() -> None:
AnthropicClient.get_web_search_tool(),
AnthropicClient.get_code_interpreter_tool(),
AnthropicClient.get_mcp_tool(
name="local-mcp",
url=local_mcp_url,
name="example-mcp",
url="https://learn.microsoft.com/api/mcp",
),
]

response = await client.get_response(
messages=messages,
options={"tools": tools, "max_tokens": 100},
)
try:
response = await client.get_response(
messages=messages,
options={"tools": tools, "max_tokens": 100},
)
except (
anthropic.BadRequestError,
Comment on lines +1526 to +1527

Copilot AI Apr 16, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Catching anthropic.BadRequestError and skipping can mask real regressions (a 400 typically indicates an invalid request/tool schema rather than a transient upstream outage). Consider limiting the skip behavior to clearly-transient failures (e.g., connection/timeout/5xx) or inspecting the error details and only skipping when the 400 is explicitly caused by the hosted MCP call failing; otherwise re-raise so genuine test failures still fail CI.

Suggested change
except (
anthropic.BadRequestError,
except anthropic.BadRequestError as e:
error_message = str(e)
error_body = getattr(e, "body", None)
if isinstance(error_body, dict):
error_details = error_body.get("error")
if isinstance(error_details, dict):
detail_message = error_details.get("message")
if isinstance(detail_message, str) and detail_message:
error_message = f"{error_message} {detail_message}".strip()
normalized_error_message = error_message.lower()
is_hosted_mcp_failure = "mcp" in normalized_error_message and any(
marker in normalized_error_message
for marker in ("unavailable", "failed", "failure", "timeout", "timed out")
)
if is_hosted_mcp_failure:
pytest.skip(f"Upstream MCP server unavailable: {e}")
raise
except (

Copilot uses AI. Check for mistakes.
anthropic.InternalServerError,
anthropic.APIConnectionError,
anthropic.APITimeoutError,
) as e:
pytest.skip(f"Upstream MCP server unavailable: {e}")

assert response is not None
assert response.text is not None
Expand Down
Loading