fix(mcp): logged, single-flight, proactive OAuth refresh for MCP servers - #1697
fix(mcp): logged, single-flight, proactive OAuth refresh for MCP servers#1697Aaronontheweb wants to merge 5 commits into
Conversation
…h rejections Parse the RFC 6749 error body at the token endpoint (4xx only) and surface invalid_grant, invalid_client, and unauthorized_client as a typed OAuthTokenRefreshRejectedException. McpOAuthService handles all three in one shared terminal path (clear + persist + alert with the code as reason); client errors additionally drop the cached DCR client_id so re-auth performs a fresh registration instead of reusing the dead one. 5xx, network errors, and unrecognized/unparsable bodies stay transient.
|
Addressed review: |
Yeah, this is the real killer. I noticed that my Notion MCP connections were needing to be reset every couple of weeks or so, and this is why. |
Aaronontheweb
left a comment
There was a problem hiding this comment.
Rejected. This is massively over-engineered for the problem we were trying to solve.
| // servers — e.g. an OAuth token with no refresh token, which | ||
| // will hard-fail at expiry with no chance of silent | ||
| // recovery. See McpClientManager.BuildOAuthAdvisory. | ||
| statusMessages.Add(string.IsNullOrEmpty(error) |
| [InlineData("invalid_grant", HttpStatusCode.BadRequest)] | ||
| [InlineData("invalid_client", HttpStatusCode.Unauthorized)] | ||
| [InlineData("unauthorized_client", HttpStatusCode.BadRequest)] | ||
| public async Task RefreshToken_TerminalOAuthError_ThrowsRejectedWithErrorCode( |
There was a problem hiding this comment.
Forcibly tests the types of error codes we get back on token exchange to ensure that there's a visible error that appears in the logs this time around
| var results = await Task.WhenAll(reconnects); | ||
|
|
||
| Assert.All(results, Assert.True); | ||
| Assert.Equal(1, Volatile.Read(ref reconnectCount)); |
| ct); | ||
| } | ||
|
|
||
| private McpReconnectGate GetReconnectGate(McpServerName name) |
There was a problem hiding this comment.
populates the McpReconnectGate for this MCP server on-demand
| /// state so `netclaw doctor` and the daemon status API can show it ahead | ||
| /// of time (see McpServersDoctorCheck's "Connected" case). | ||
| /// </summary> | ||
| private static string? BuildOAuthAdvisory(McpServerName serverName, McpOAuthTokenSet? tokenSet) |
There was a problem hiding this comment.
LGTM - give user a heads up that they'll need to log back in in N days.
| client = null; | ||
| _statuses[name] = new McpServerStatus(name, McpConnectionState.Connected, tools.Count, null); | ||
|
|
||
| // Advance warning for OAuth-managed servers whose token can never |
| // refresh path before constructing the HTTP transport. Runtime | ||
| // requests use McpOAuthAuthorizationHandler, so the SDK never | ||
| // receives or independently redeems the refresh token. | ||
| await _oauthService.GetValidTokenAsync(name, entry, ct); |
There was a problem hiding this comment.
Refresh happens internally inside here otherwise
| /// refresh. The MCP SDK's OAuth provider cannot participate here because it | ||
| /// redeems cached refresh tokens outside Netclaw's per-server single-flight. | ||
| /// </summary> | ||
| internal sealed class McpOAuthAuthorizationHandler : DelegatingHandler |
There was a problem hiding this comment.
So this looks like we basically ripped out what was previously supported by the ModelContextProtocol SDK and started doing it ourselves to work around, what is fundamentally a race condition? That seems:
- A bad idea
- Are you sure we aren't solving a problem that the ModelContextProtocol SDK already handles perfectly fine on its own?
Going to lock this PR down until we figure that out definitively. This stinks.
|
Closing after revisiting the OAuth ownership boundary. This PR made Netclaw responsible for bearer injection, request replay, reactive 401 handling, and token refresh in order to provide single-flight behavior. The MCP SDK maintainers have confirmed that concurrency defect in modelcontextprotocol/csharp-sdk#1595 and are implementing the fix inside ClientOAuthProvider in modelcontextprotocol/csharp-sdk#1708, which is the correct ownership layer. No published SDK version contains that fix yet. Netclaw #1696 has been converted into an upstream dependency tracker and now records the remaining Netclaw-owned lifecycle and diagnostic work. We will upgrade to a published SDK fix and validate it through the real SDK transport rather than maintain a parallel OAuth stack. The branch is being retained for auditability. |
Defects (from #1696)
ClientOAuthProvider, which refreshes silently and falls back to a headless-fatal interactive flow; Netclaw's own logged refresh path (McpOAuthService.GetValidTokenAsync) had zero callers.Fix
McpOAuthServiceis now the sole runtime token authority. A Netclaw-owned HTTP authorization handler attaches access tokens, routes both proactive and on-401 refresh through the same per-server single-flight, and retries the rejected request once. The MCP SDK no longer receives the refresh token or independently redeems it; the formerITokenCachebridge was removed.McpClientManager.TryReconnectAsyncnow uses a version-aware per-server gate. Overlapping reconnect callers reuse the first successful replacement client; a failed leader allows the next waiter to retry. Terminal teardown uses the same gate and observes cancellation.McpReconnectionService's existing 30s tick: connected OAuth servers refresh once within a 10-minute window ofExpiresAt(constant,TimeProvider-driven).invalid_grant,invalid_client, andunauthorized_clientare terminal: tokens are cleared, the connection is torn down, and diagnostics report a distinctAuthFailedstatus with reauthorization guidance. Client-registration errors also clear the cached DCR client ID so reauthorization registers a fresh client. Transient failures retain tokens and remain retryable.Authorizationheaders remain operator-owned and bypass managed OAuth refresh.netclaw-operationsdiagnostics andnetclaw doctorexplain the distinct statuses.Tests
AuthFailedstatus.Validation
dotnet test --no-restorepassed.dotnet slopwatch analyzepassed with zero findings.pwsh ./scripts/Add-FileHeaders.ps1 -Verifyandgit diff --checkpassed.NETCLAW_EVAL_PROVIDER_TYPE,NETCLAW_EVAL_PROVIDER_ENDPOINT, andNETCLAW_EVAL_MODEL_IDare not configured in this workspace.Fixes #1696