CancelledError when connecting to MCP servers via streamable HTTP transport

[thinkingmik]

CancelledError when connecting to MCP servers via streamable HTTP transport

Description

When attempting to connect an OpenAgents agent to an MCP server using the streamable_http transport type, the connection fails during session initialization with an asyncio.exceptions.CancelledError.

Environment

• OpenAgents version: 0.8.6
• Python version: 3.12.0
• OS: Windows 10/11
• MCP library version: 1.15.0 (also tested with 1.20.0)

Configuration

Agent configuration (webhook_agent.yaml):

type: "openagents.agents.collaborator_agent.CollaboratorAgent"
agent_id: "webhook-agent-test"

config:
  model_name: "gpt-4o-mini"
  provider: "openai"

  instruction: |
    You are an agent that receives messages from external webhooks.
    You have access to tools from the MCP server.

  react_to_all_messages: true
  max_iterations: 5

  mcps:
    - name: "arcade-demo"
      type: "streamable_http"
      url: "https://mcp-http-demo.arcade.dev/mcp"
      timeout: 10
      retry_attempts: 3

mods:
  - name: "openagents.mods.workspace.messaging"
    enabled: true

connection:
  host: "localhost"
  port: 8700
  transport: "grpc"

Steps to Reproduce

1. Create an agent with MCP server configuration using streamable_http transport
2. Start the OpenAgents network: openagents network start deployments
3. Start the agent: openagents agent start deployments/agents/webhook_agent.yaml
4. Observe the error during MCP client initialization

Expected Behavior

The agent should successfully connect to the MCP server, initialize the session, and load available tools.

Actual Behavior

The connection fails with CancelledError during session.initialize():

INFO     Connecting to streamable HTTP MCP server 'arcade-demo' at https://mcp-http-demo.arcade.dev/mcp (timeout: 10s)
INFO     Initializing MCP session for 'arcade-demo'...

Traceback (most recent call last):
  File "C:\temp\openagents\src\openagents\utils\mcp_connector.py", line 200, in _setup_streamable_http_mcp_client
    await asyncio.wait_for(session.initialize(), timeout=timeout)
  File "C:\Users\M.Andreoli\.conda\envs\py3.12.0\Lib\asyncio\tasks.py", line 510, in wait_for
    return await fut
           ^^^^^^^^^
  File "C:\Users\M.Andreoli\.conda\envs\py3.12.0\Lib\site-packages\mcp\client\session.py", line 171, in initialize
    result = await self.send_request(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\M.Andreoli\.conda\envs\py3.12.0\Lib\site-packages\mcp\shared\session.py", line 292, in send_request
    response_or_error = await response_stream_reader.receive()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\M.Andreoli\.conda\envs\py3.12.0\Lib\site-packages\anyio\streams\memory.py", line 115, in receive
    await checkpoint()
  File "C:\Users\M.Andreoli\.conda\envs\py3.12.0\Lib\site-packages\anyio\lowlevel.py", line 38, in checkpoint
    await get_async_backend().checkpoint()
  File "C:\Users\M.Andreoli\.conda\envs\py3.12.0\Lib\site-packages\anyio\_backends\_asyncio.py", line 2358, in checkpoint
    await sleep(0)
  File "C:\Users\M.Andreoli\.conda\envs\py3.12.0\Lib\asyncio\tasks.py", line 646, in sleep
    await __sleep0()
  File "C:\Users\M.Andreoli\.conda\envs\py3.12.0\Lib\asyncio\tasks.py", line 640, in __sleep0
    yield
asyncio.exceptions.CancelledError: Cancelled via cancel scope 13d666feed0 by <Task pending name='Task-13' coro=<<async_generator_athrow without __name__>()>>

Additional Context

1. The issue occurs with both custom MCP servers and public test servers (e.g., https://mcp-http-demo.arcade.dev/mcp)
2. Manual testing with curl shows that the MCP servers respond correctly to initialization requests:

   curl -X POST https://mcp-http-demo.arcade.dev/mcp \
     -H "Content-Type: application/json" \
     -H "Accept: application/json" \
     -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"test","version":"1.0"}}}'

3. The error occurs in the upstream mcp library's streamablehttp_client during session initialization
4. Adding timeout handling in src/openagents/utils/mcp_connector.py (lines 175-200) makes the timeout explicit but doesn't resolve the underlying cancellation issue

Relevant Code

The issue occurs in src/openagents/utils/mcp_connector.py in the _setup_streamable_http_mcp_client method:

async def _setup_streamable_http_mcp_client(self, mcp_config: MCPServerConfig):
    """Setup a streamable HTTP-based MCP client."""
    if not mcp_config.url:
        raise ValueError(f"URL is required for streamable_http MCP server '{mcp_config.name}'")

    timeout = getattr(mcp_config, 'timeout', 30)
    
    try:
        logger.info(f"Connecting to streamable HTTP MCP server '{mcp_config.name}' at {mcp_config.url} (timeout: {timeout}s)")
        transport = await asyncio.wait_for(
            streamablehttp_client(mcp_config.url).__aenter__(),
            timeout=timeout
        )
        read_stream, write_stream, get_session_id = transport
        
        session = ClientSession(read_stream, write_stream)
        await asyncio.wait_for(session.__aenter__(), timeout=timeout)
        
        # Initialize the session with timeout
        logger.info(f"Initializing MCP session for '{mcp_config.name}'...")
        await asyncio.wait_for(session.initialize(), timeout=timeout)  # <-- Fails here
        
        # ... rest of the code

[linear]

OADEV-40