Add getBearerToken callback for BYOK providers (Managed Identity)#1748
Add getBearerToken callback for BYOK providers (Managed Identity)#1748SteveSandersonMS wants to merge 9 commits into
Conversation
This comment has been minimized.
This comment has been minimized.
0ba9325 to
d321d4a
Compare
This comment has been minimized.
This comment has been minimized.
|
Thanks — a couple of these points are now moot after a contract simplification pushed since this review ran:
So the remaining cross-SDK surface is just (2) the |
Lets BYOK provider configs supply a `getBearerToken` callback so the SDK consumer resolves bearer tokens (e.g. Azure Managed Identity via @azure/identity) on demand. The callback never crosses the wire: the SDK strips it from the provider config, sends a `hasBearerTokenProvider: true` flag, and answers the runtime's session-scoped `providerToken.acquire` RPC by routing to the matching per-provider callback. The returned token is applied as the Authorization header for outbound model requests; the consumer owns caching/refresh. - client.ts: strip the callback, emit the `hasBearerTokenProvider` wire flag, register per-provider callbacks on the session. - session.ts: handle `providerToken.acquire` by dispatching on provider name. - types.ts: public `getBearerToken` / `ProviderTokenArgs` / `ProviderBearerToken`. - generated/rpc.ts: regenerated contract (providerToken.acquire + hasBearerTokenProvider/bearerTokenScope fields). - e2e: callback token reaches model, refresh-on-expiry, per-provider dispatch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mirror the runtime contract simplification: the runtime no longer caches provider tokens, it calls getBearerToken once per outbound request. Drop the vestigial `scope`/`bearerTokenScope` (the callback closes over its own scope/ audience) and stop forwarding `expiresOnTimestamp` over the wire. The field is retained on `ProviderBearerToken` so an Azure Identity `AccessToken` can be returned verbatim, but it is now documented as ignored — caching and refresh are the callback's responsibility (e.g. `@azure/identity` caches internally). Rewrite the e2e suite to drive a local capturing HTTP server as the BYOK endpoint instead of CAPI record/replay snapshots, so the tests assert on the outbound Authorization header directly and pass identically in record and replay mode. Delete the obsolete snapshots. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
13fd1d7 to
02fb2fc
Compare
This comment has been minimized.
This comment has been minimized.
Replace the real local HTTP listener (CapturingProvider) with the client-global CopilotRequestHandler interceptor (PR #1689). The handler captures the Authorization header the runtime applies after resolving getBearerToken over the providerToken.acquire RPC, and answers with a synthetic 404 entirely in-process: no socket binding, no listen/close lifecycle, no waitForRequests polling. Distinct fake .invalid hosts per provider let the per-provider dispatch test assert routing by host. The handler is installed once per fixture via createSdkTestContext({ copilotClientOptions: { requestHandler } }) and reset between tests; non-BYOK (CAPI bootstrap) requests pass through via super. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
The getBearerToken callback previously returned `ProviderBearerToken | string`, where ProviderBearerToken carried an `expiresOnTimestamp` field that the runtime accepted but ignored (it does no caching). That advertised a no-op field and an awkward union for no benefit. Collapse the return type to `Promise<string>`: the callback resolves a raw token string and nothing else. Remove the ProviderBearerToken interface, simplify the providerToken.acquire handler (no more string|object normalization), and export the public GetBearerToken / ProviderTokenArgs types from the package root so consumers can type standalone callbacks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
…lue + e2e Rename the session-scoped provider-token RPC method and handler from `acquire`/`AcquireAsync` to `getToken`/`GetTokenAsync` across the generated wire contracts (Node, .NET, Python, Go, Rust) to match the runtime source of truth. Add the .NET SDK getBearerToken support mirroring Node: - ProviderTokenArgs with a required ProviderName (no delegate; consumers pass a Func<ProviderTokenArgs, Task<string>> lambda). - hasBearerTokenProvider is now an internal computed wire flag derived from the presence of GetBearerToken, so it is no longer settable public API. - 3-scenario hermetic e2e (callback token, per-request re-acquisition, per-provider dispatch) that does not use the capi proxy. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mirror the Node/.NET getBearerToken surface in the Python SDK: - ProviderTokenArgs TypedDict + GetBearerToken callable alias. - get_bearer_token field on ProviderConfig and NamedProviderConfig; wire conversion emits hasBearerTokenProvider: true and strips the callback. - _BearerTokenProviderAdapter routes providerToken.getToken to the matching per-provider callback (sync or async tolerant). - 3-scenario hermetic e2e (callback token, per-request re-acquisition, per-provider dispatch) that fabricates bootstrap and does not use the CAPI proxy. Also fix stale providerToken.acquire references in the Node e2e comments. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| # already sent the (token-bearing) request — which is all we assert on. | ||
| try: | ||
| await session.send_and_wait(prompt) | ||
| except Exception: |
Cross-SDK Consistency Review ✅Reviewed the SummaryThis PR achieves excellent cross-SDK consistency. Notably, the Rust SDK already had a complete implementation of this feature (in Consistency checks ✅
Language-idiomatic differences (expected, not flagged)
No cross-SDK inconsistencies found. The feature is fully implemented across all six languages with appropriate language-idiomatic adaptations.
|
| return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound) | ||
| { | ||
| Content = new StringContent( | ||
| "{\"error\":{\"message\":\"fake byok endpoint\"}}", | ||
| System.Text.Encoding.UTF8, | ||
| "application/json"), | ||
| }); |
| foreach (var provider in config.Providers) | ||
| { | ||
| if (provider.GetBearerToken is { } callback) | ||
| { | ||
| callbacks[provider.Name] = callback; | ||
| } | ||
| } |
| catch | ||
| { | ||
| // The handler always 404s the BYOK endpoint, so the turn errors after | ||
| // the token-bearing request was already captured. Expected. | ||
| } |
Summary
Adds an experimental
getBearerTokencallback to BYOK provider configs so the SDK consumer can resolve bearer tokens (e.g. Azure Managed Identity via the consumer's identity library) on demand. The Copilot SDK takes zero Azure dependency — the consumer fills in the callback with whatever identity library they like. The runtime does no caching — it invokes the callback once per request, so the consumer (or the identity library it wraps) owns caching/refresh. Scope/audience is closed over by the callback and never crosses the wire.This PR ships the feature across all six SDKs — TypeScript/Node, .NET, Python, Go, Rust, and Java — each with an equivalent three-scenario e2e test.
Usage examples
Each example wires
DefaultAzureCredential(Azure Managed Identity) into the callback for thehttps://cognitiveservices.azure.com/.defaultscope. The identity library caches/refreshes the underlying token, so calling it per request is cheap.TypeScript / Node
.NET
Python
Go
Rust
Java