Add Timeout Configuration to Foundry OpenAI Client

[nzthiago]

Summary

The azurefunctions-agents-runtime 0.1.0b1 builds the Foundry chat client without configuring HTTP timeouts on the underlying Azure SDK transport. When the Foundry endpoint has transient latency on a Responses API roundtrip, the underlying socket read blocks indefinitely with no exception raised, causing agent execution to hang silently.

Fix proposed in PR #70.

Environment

• Runtime version: azurefunctions-agents-runtime==0.1.0b1
• Python version: 3.13
• Function plan: Flex Consumption (Linux)
• Region: westus3
• Foundry model: gpt-5 (GlobalStandard, 900K TPM capacity) — reproduced on gpt-5.4, gpt-5.4-mini too
• Foundry endpoint: …/api/projects/{project}/openai/v1/responses (stateful Responses API)

Symptom

The agent runs cleanly for ~2 minutes, making multiple Foundry roundtrips for multi-turn reasoning. Then:

1. Agent POSTs a tool result to the Foundry /responses endpoint.
2. Request sends successfully (headers + body transmitted).
3. The httpx state machine reaches receive_response_headers.started.
4. No response. No exception. No log. The Python worker waits indefinitely.
5. The host eventually kills the worker after functionTimeout (30 min on Flex), but the invocation is silently lost — no error surfaces to the caller, no retry signal, no error log.

This manifests as "the agent randomly stops working after ~10 requests" — but it's actually: transient Foundry latency + missing httpx timeout → indefinite hang on a socket read.

Evidence

• Operation_Id: 0ebc620e2d44cedfa3abb2b66560915f
• App Insights: appi-lqwx5rxrxq4xk (RG azure-functions-reports-agent-rg)
• DEBUG-level traces show exact hang point:

19:52:52.227 UTC  HTTP Request: POST .../openai/v1/responses
19:52:52.228 UTC  send_request_headers.started
19:52:52.228 UTC  send_request_headers.complete
19:52:52.228 UTC  send_request_body.started
19:52:52.228 UTC  send_request_body.complete
19:52:52.228 UTC  receive_response_headers.started   ← LAST TRACE
[SILENCE — no response, no exception, no timeout]

Previous Foundry roundtrips in the same invocation all succeeded (200 OK at 19:52:30, 19:52:33, 19:52:44, 19:52:49).

Worker stack at the hung state

Captured by a sys._current_frames() watchdog thread (added to function_app.py purely for this diagnostic, routed via print() so it reaches the Function App's stdout pipeline):

[watchdog 2026-06-15T21:22:51Z] Thread <main> frames:
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/azure-functions-host/workers/python/3.13/LINUX/X64/proxy_worker/__main__.py", line 6, in <module>
    start_worker.start()
  File "/azure-functions-host/workers/python/3.13/LINUX/X64/proxy_worker/start_worker.py", line 65, in start
    return asyncio.run(start_async(
  File "/opt/python/3/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
  File "/opt/python/3/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)

Main thread blocked in run_until_complete(task) — event loop is alive, but the awaited coroutine (the Foundry call chain) is suspended indefinitely on a pending socket read. The hang lives inside the coroutine, behind await client.create(...) for the Responses API.

Alternates ruled out

• ❌ Worker SIGKILL / OOM — zero WorkerProcessExitException in 8h; MemoryWorkingSet peak ~650 MB (~15% of available on Flex).
• ❌ Worker segfault — zero exit-code-139 events.
• ❌ Foundry rate-limit at the deployment level — usage 5–11% of 900 K TPM, zero 429 errors in App Insights.
• ❌ Model-specific bug — reproduced across gpt-5, gpt-5.4, gpt-5.4-mini (5-model investigation; same hang signature regardless of model choice).

Root cause

In src/azure_functions_agents/client_manager.py lines 198-202, _build_foundry passes project_endpoint (a string) to FoundryChatClient:

return FoundryChatClient(
    project_endpoint=endpoint,
    model=model,
    credential=build_async_credential(),
)

When given just an endpoint string, FoundryChatClient internally constructs azure.ai.projects.aio.AIProjectClient(endpoint=…, credential=…) using the Azure SDK pipeline's default transport. The default transport has no httpx.Timeout configured for read operations, so a stuck socket read on a Responses API roundtrip blocks forever and no exception ever fires.

Proposed fix (implemented in PR #70)

Have _build_foundry construct AIProjectClient itself with an explicit AioHttpTransport carrying connect/read timeouts, then pass the pre-built project_client to FoundryChatClient:

@classmethod
def _build_foundry(cls, model: str) -> Any:
    from agent_framework.foundry import FoundryChatClient
    from azure.ai.projects.aio import AIProjectClient
    from azure.core.pipeline.transport import AioHttpTransport

    endpoint = cls._env("FOUNDRY_PROJECT_ENDPOINT")
    if not endpoint:
        raise RuntimeError(
            "AZURE_FUNCTIONS_AGENTS_PROVIDER=foundry requires "
            "FOUNDRY_PROJECT_ENDPOINT to be set."
        )
    read_timeout = float(cls._env("FOUNDRY_HTTP_READ_TIMEOUT") or 180)
    connect_timeout = float(cls._env("FOUNDRY_HTTP_CONNECT_TIMEOUT") or 10)
    transport = AioHttpTransport(
        connection_timeout=connect_timeout,
        read_timeout=read_timeout,
    )
    project_client = AIProjectClient(
        endpoint=endpoint,
        credential=build_async_credential(),
        transport=transport,
    )
    return FoundryChatClient(
        project_client=project_client,
        model=model,
    )

Env-var knobs (defaults match Azure SDK examples)

• FOUNDRY_HTTP_READ_TIMEOUT — default 180.0 (seconds). Generous enough for the longest legitimate Responses-API call; short enough that a stuck connection fails an invocation in roughly 1/10th of the platform timeout window.
• FOUNDRY_HTTP_CONNECT_TIMEOUT — default 10.0 (seconds).

Both overridable via app settings without touching code.

Why mcp.json timeouts don't fix this

Users may try configuring timeouts via mcp.json:

{
  "mcpServers": { "kusto": { "timeout": { "connect": 10, "read": 120 } } }
}

That does not help. mcp.json timeouts apply only to MCP tool calls (Kusto, GitHub, etc.). The Foundry chat client that orchestrates the agent loop is a separate client with separate transport configuration — and that's the one that hangs.

┌─────────────────────────────────────────────────┐
│ Azure Functions Agent (Python worker)           │
├─────────────────────────────────────────────────┤
│  Foundry chat client (orchestrator)             │ ← NO timeout (BUG)
│  └─ POST /openai/v1/responses (every reasoning  │ ← HANGS HERE
│     turn)                                       │
│                                                 │
│  MCP clients (tools)                            │ ← mcp.json timeout applies
│  ├─ Kusto connector (120s read)                 │
│  └─ GitHub connector (60s read)                 │
└─────────────────────────────────────────────────┘

Expected behaviour after fix

When Foundry is slow to respond:

1. The Azure SDK transport waits up to FOUNDRY_HTTP_READ_TIMEOUT seconds (default 180s).
2. If no response: raises an azure.core.exceptions.ServiceResponseTimeoutError (which wraps the underlying httpx.ReadTimeout).
3. The exception surfaces to Application Insights with a real stack trace.
4. Function execution fails cleanly instead of hanging silently.
5. Users can implement retry logic or investigate Foundry-side latency from observable failures.

Impact if not fixed

• Production blocker for long-running agents. Agents using >10 Foundry roundtrips per invocation hang probabilistically.
• Silent failures. No error logs at default trace level; debugging requires DEBUG-level instrumentation + a thread-frame watchdog.
• User cannot work around in user code. The runtime instantiates the client internally; there's no hook to override transport config without forking.
• Architectural workarounds are costly. Splitting agents into shorter functions (so each stays under the hang threshold) adds significant complexity and is itself a workaround for a runtime bug.

Out of scope (follow-up)

The same timeout gap exists in _build_openai (lines 153-159) and _build_azure_openai (lines 162-186) — OpenAIChatClient accepts a pre-built async_client parameter (AsyncOpenAI / AsyncAzureOpenAI), both of which take http_client=httpx.AsyncClient(timeout=httpx.Timeout(...)). Same pattern as this PR. Worth fixing in a follow-up.

[nzthiago]

Following up with sharper evidence + a concrete Option A fix.

Exact code location

src/azure_functions_agents/client_manager.py lines 198-202 — _build_foundry:

return FoundryChatClient(
    project_endpoint=endpoint,
    model=model,
    credential=build_async_credential(),
)

When project_endpoint is passed as a string, FoundryChatClient internally constructs azure.ai.projects.aio.AIProjectClient(endpoint=..., credential=...) using the Azure SDK pipeline's default transport — which has no httpx.Timeout configured for read operations. When Foundry has any transient latency on a Responses API roundtrip, the underlying socket read blocks indefinitely with no exception raised.

Ruled-out alternates (Application Insights, last 8h, prod Function App on Flex Consumption Linux)

• ❌ Worker SIGKILL / OOM — zero WorkerProcessExitException; MemoryWorkingSet peak ~650 MB (~15% of available)
• ❌ Worker segfault — zero exit-code-139 events
• ❌ Foundry rate-limit at deployment level — usage 5–11% of 900 K TPM, zero 429s
• ❌ Model-specific bug — reproduced across gpt-5, gpt-5.4, gpt-5.4-mini (5-model investigation; same hang signature regardless of model)

Live worker stack trace at the active hung state

Captured via a sys._current_frames() watchdog thread + print() to stdout (which the Function App's Python worker forwards to Application Insights). Repro: agent making ~10+ Foundry roundtrips in a single timer invocation; hang materializes probabilistically around the 2-minute mark.

[watchdog 2026-06-15T21:22:51Z] Thread <main> frames:
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/azure-functions-host/workers/python/3.13/LINUX/X64/proxy_worker/__main__.py", line 6, in <module>
    start_worker.start()
  File "/azure-functions-host/workers/python/3.13/LINUX/X64/proxy_worker/start_worker.py", line 65, in start
    return asyncio.run(start_async(
  File "/opt/python/3/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
  File "/opt/python/3/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)

[watchdog 2026-06-15T21:22:51Z] Thread <grpc-channel-spin> frames:
  File "/azure-functions-host/workers/python/3.13/LINUX/X64/grpc/_channel.py", line 1751, in channel_spin
    event = state.channel.next_call_event()

[watchdog 2026-06-15T21:22:51Z] Thread <grpc-poll> frames:
  File "/azure-functions-host/workers/python/3.13/LINUX/X64/proxy_worker/dispatcher.py", line 294, in __poll_grpc
    for req in grpc_req_stream:
  ...
  File "/opt/python/3/lib/python3.13/threading.py", line 369, in wait
    if not gotit:

The main worker thread is blocked in self._loop.run_until_complete(task) — the asyncio event loop is alive, but the awaited task (the agent's Foundry call chain) is suspended indefinitely on a pending socket read. The hang lives inside the coroutine, behind the await client.create(...) for Foundry's Responses API. No exception is ever raised because there's no httpx.Timeout to fire one.

Concrete fix (Option A — runtime owns the timeout)

@classmethod
def _build_foundry(cls, model: str) -> Any:
    from agent_framework.foundry import FoundryChatClient
    from azure.ai.projects.aio import AIProjectClient
    from azure.core.pipeline.transport import AioHttpTransport

    endpoint = cls._env("FOUNDRY_PROJECT_ENDPOINT")
    if not endpoint:
        raise RuntimeError(
            "AZURE_FUNCTIONS_AGENTS_PROVIDER=foundry requires "
            "FOUNDRY_PROJECT_ENDPOINT to be set."
        )
    # 180s read default; configurable via env so operators can tune
    read_timeout = float(cls._env("FOUNDRY_HTTP_READ_TIMEOUT") or 180)
    connect_timeout = float(cls._env("FOUNDRY_HTTP_CONNECT_TIMEOUT") or 10)
    transport = AioHttpTransport(
        connection_timeout=connect_timeout,
        read_timeout=read_timeout,
    )
    project_client = AIProjectClient(
        endpoint=endpoint,
        credential=build_async_credential(),
        transport=transport,
    )
    return FoundryChatClient(
        project_client=project_client,
        model=model,
    )

Sensible defaults (180 s / 10 s) match what Foundry's own SDK examples use. The env-var overrides let operators tune without code changes.

The same gap exists at _build_openai (lines 153-159) and _build_azure_openai (lines 162-186) — OpenAIChatClient likewise has no timeout passthrough. Worth fixing in the same change with the equivalent http_client=httpx.AsyncClient(timeout=httpx.Timeout(...)) parameter (the OpenAI SDK supports it directly).

PR incoming.