MCP OAuth: track upstream single-flight refresh and harden Netclaw diagnostics

[Aaronontheweb]

Status: blocked on an upstream MCP SDK fix

Netclaw should not replace the MCP SDK's OAuth transport or become the owner of
bearer injection, request replay, reactive 401 handling, and token refresh.
Those responsibilities belong in the SDK's ClientOAuthProvider.

As of 2026-07-21:

• Netclaw uses ModelContextProtocol.Core 1.4.1, which is the latest stable
  release and does not serialize concurrent OAuth refreshes.
• 2.0.0-preview.3 is the latest prerelease and also does not contain the fix.
• Upstream issue
  modelcontextprotocol/csharp-sdk#1595
  tracks the thread-safety defect.
• Upstream PR
  modelcontextprotocol/csharp-sdk#1708
  adds single-flight token acquisition inside ClientOAuthProvider. It is open,
  targets main and the 2.x milestone, and still has unresolved review
  feedback. No 1.x backport has been published or proposed.

This issue must remain open until the corrected upstream implementation is
available in a published package and Netclaw has upgraded to it.

Incident

Two independently operated instances had their Notion MCP grants rejected in
the same time window, with no Netclaw OAuth code changes in that window.
Forensics on one instance showed that the rejected refresh token was current,
correctly persisted, and had not previously been reused. The immediate trigger
was therefore likely provider-side, but the incident exposed reliability and
diagnostic gaps in the client path.

Confirmed defects

Upstream SDK ownership

1. Concurrent refresh is not single-flight in 1.4.1. Concurrent MCP
   requests can independently observe an expired token or receive a 401, then
   redeem the same rotating refresh token concurrently. The SDK's public
   ITokenCache abstraction only surrounds token reads and writes; it cannot
   serialize the intervening token-endpoint request. This is the defect tracked
   by upstream issue Token/metadata caches in auth providers are not thread-safe modelcontextprotocol/csharp-sdk#1595 and PR Make auth provider token caches thread-safe (#1595) modelcontextprotocol/csharp-sdk#1708.
2. Refresh failures lack actionable detail. In 1.4.1,
   ClientOAuthProvider.RefreshTokensAsync returns null on a non-success token
   response and falls through to interactive authorization. A headless daemon
   consequently reports a generic authentication failure without preserving the
   token endpoint's error, such as invalid_grant or a transient server error.
   Upstream PR Make auth provider token caches thread-safe (#1595) modelcontextprotocol/csharp-sdk#1708 does not address this diagnostic gap. Closed upstream PR
   #1501
   previously proposed status-code logging but was not merged.

Netclaw ownership

3. Reconnect attempts need coalescing. Concurrent invocation failures must
   not tear down or replace the same per-server MCP client multiple times.
   This is client-lifecycle coordination, separate from OAuth refresh ownership.
4. Non-refreshable token sets need an explicit operator signal. When an MCP
   OAuth token set has no refresh token, Netclaw should report that
   reauthorization will be required instead of allowing a generic headless auth
   failure to be the first indication.

Decision

• Keep the MCP SDK as the sole runtime owner of OAuth request authentication and
  refresh.
• Do not add a Netclaw authorization handler that duplicates bearer injection,
  request cloning/replay, 401 handling, or token-endpoint protocol behavior.
• Do not run an independent Netclaw refresh loop alongside the SDK provider;
  two uncoordinated refresh owners recreate the same rotating-token race.
• Keep Netclaw changes limited to its own boundaries: durable token-cache
  integration, lifecycle/reconnect coordination, status, diagnostics, and
  operator guidance.

Completion criteria

• Upstream Token/metadata caches in auth providers are not thread-safe modelcontextprotocol/csharp-sdk#1595 is fixed and included in a published MCP SDK package.
• Decide whether to consume a 1.x backport or perform the separately scoped
  2.x migration; do not adopt an unreleased source commit.
• Upgrade Netclaw to the released package containing the fix.
• Retain the SDK OAuth provider and remove any duplicate Netclaw HTTP auth
  handler or token-endpoint implementation from the runtime path.
• Add an integration regression test using the real SDK HTTP transport that
  proves concurrent 401 responses result in one refresh-token redemption
  and all waiting calls reuse the winning token.
• Add or track an upstream solution for actionable refresh-failure
  diagnostics without taking ownership of OAuth transport behavior.
• Handle Netclaw-owned reconnect coalescing and missing-refresh-token
  diagnostics with focused tests.

Related Netclaw work

• fix(mcp): logged, single-flight, proactive OAuth refresh for MCP servers #1697 explored making Netclaw the OAuth token authority. That approach was
  rejected because it duplicated SDK responsibilities and substantially
  increased Netclaw's protocol and HTTP retry surface.

[Aaronontheweb]

Netclaw-owned lifecycle hardening is implemented locally under the simplify-mcp-oauth-lifecycle change:

• reconnects coalesce by observed client generation
• replacement candidates initialize before publication
• in-flight calls drain before retired clients are disposed
• cancellation and application/tool errors no longer tear down healthy clients
• ambiguous transport failures reconnect for later calls without replaying the failed tool
• missing-refresh-token and OAuth status diagnostics are explicit

The implementation has deterministic reconnect, replacement, shutdown, and disposal race coverage. The SDK remains the sole owner of OAuth discovery, DCR, PKCE, bearer injection, token exchange, and refresh.

This issue should remain open: ModelContextProtocol.Core is still pinned to 1.4.1 and the upstream single-flight refresh fix has not shipped in a stable package.