Skip to content

Add getBearerToken callback for BYOK providers (Managed Identity)#1748

Draft
SteveSandersonMS wants to merge 9 commits into
mainfrom
stevesandersonms/byok-provider-token-rpc
Draft

Add getBearerToken callback for BYOK providers (Managed Identity)#1748
SteveSandersonMS wants to merge 9 commits into
mainfrom
stevesandersonms/byok-provider-token-rpc

Conversation

@SteveSandersonMS

@SteveSandersonMS SteveSandersonMS commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an experimental getBearerToken callback to BYOK provider configs so the SDK consumer can resolve bearer tokens (e.g. Azure Managed Identity via the consumer's identity library) on demand. The Copilot SDK takes zero Azure dependency — the consumer fills in the callback with whatever identity library they like. The runtime does no caching — it invokes the callback once per request, so the consumer (or the identity library it wraps) owns caching/refresh. Scope/audience is closed over by the callback and never crosses the wire.

This PR ships the feature across all six SDKs — TypeScript/Node, .NET, Python, Go, Rust, and Java — each with an equivalent three-scenario e2e test.

Usage examples

Each example wires DefaultAzureCredential (Azure Managed Identity) into the callback for the https://cognitiveservices.azure.com/.default scope. The identity library caches/refreshes the underlying token, so calling it per request is cheap.

TypeScript / Node

import { DefaultAzureCredential } from "@azure/identity";
import type { GetBearerToken, ProviderConfig } from "@github/copilot";

const cred = new DefaultAzureCredential();

const provider: ProviderConfig = {
    type: "openai",
    baseUrl,
    getBearerToken: async () => (await cred.getToken()).token,
};

.NET

using Azure.Core;
using Azure.Identity;

var cred = new DefaultAzureCredential();

var provider = new ProviderConfig
{
    Type = "openai",
    BaseUrl = baseUrl,
    GetBearerToken = async args => (await cred.GetTokenAsync()).Token,
};

Python

from azure.identity.aio import DefaultAzureCredential

cred = DefaultAzureCredential()

async def get_bearer_token(args) -> str:
    token = await cred.get_token()
    return token.token

provider = {
    "type": "openai",
    "base_url": base_url,
    "get_bearer_token": get_bearer_token,
}

Go

import (
    "context"
    "github.com/Azure/azure-sdk-for-go/sdk/azcore/policy"
    "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
)

cred, err := azidentity.NewDefaultAzureCredential(nil)
if err != nil {
    return err
}
scope := "https://cognitiveservices.azure.com/.default"

provider := copilot.ProviderConfig{
    Type:    "openai",
    BaseURL: baseURL,
    GetBearerToken: func(args copilot.ProviderTokenArgs) (string, error) {
        tok, err := cred.GetToken(context.Background(), policy.TokenRequestOptions{
            Scopes: []string{scope},
        })
        if err != nil {
            return "", err
        }
        return tok.Token, nil
    },
}

Rust

use std::sync::Arc;
use azure_identity::DefaultAzureCredential;
use copilot::{ProviderConfig, ProviderTokenArgs};

let cred = DefaultAzureCredential::new()?;
let scope = "https://cognitiveservices.azure.com/.default";

let provider = ProviderConfig::new(base_url)
    .with_provider_type("openai")
    .with_get_bearer_token(Arc::new(move |_args: ProviderTokenArgs| {
        let cred = cred.clone();
        async move {
            let token = cred
                .get_token(&[scope])
                .await
                .map_err(|e| BearerTokenError::new(e.to_string()))?;
            Ok(token.token.secret().to_string())
        }
    }));

Java

import com.azure.identity.DefaultAzureCredentialBuilder;
import com.azure.core.credential.TokenRequestContext;

var cred = new DefaultAzureCredentialBuilder().build();
var ctx = new TokenRequestContext().addScopes("https://cognitiveservices.azure.com/.default");

var provider = new ProviderConfig()
    .setType("openai")
    .setBaseUrl(baseUrl)
    .setGetBearerToken(args ->
        cred.getToken(ctx).map(t -> t.getToken()).toFuture());

@github-actions

This comment has been minimized.

@SteveSandersonMS SteveSandersonMS force-pushed the stevesandersonms/byok-provider-token-rpc branch from 0ba9325 to d321d4a Compare June 22, 2026 20:04
@github-actions

This comment has been minimized.

@SteveSandersonMS

Copy link
Copy Markdown
Contributor Author

Thanks — a couple of these points are now moot after a contract simplification pushed since this review ran:

  • bearerTokenScope has been removed entirely from the wire contract and from both provider configs. Scope/audience is closed over by the consumer''s getBearerToken callback (e.g. @azure/identity already binds the scope at getToken(scope) time), so it never needs to cross the wire. Point 1 no longer applies to any SDK.
  • The runtime now does no token cachingexpiresOnTimestamp was also dropped from the wire result; the callback owns caching/refresh.

So the remaining cross-SDK surface is just (2) the getBearerToken callback and (3) the providerToken.acquire server→client handler. This is intentionally Node-first and @experimental; parity for Python/Go/.NET/Java/Rust is planned as follow-up, not part of this PR. I''ll open tracking issues for the other SDKs once the Node + runtime contract lands.

SteveSandersonMS and others added 2 commits June 23, 2026 11:06
Lets BYOK provider configs supply a `getBearerToken` callback so the SDK
consumer resolves bearer tokens (e.g. Azure Managed Identity via @azure/identity)
on demand. The callback never crosses the wire: the SDK strips it from the
provider config, sends a `hasBearerTokenProvider: true` flag, and answers the
runtime's session-scoped `providerToken.acquire` RPC by routing to the matching
per-provider callback. The returned token is applied as the Authorization header
for outbound model requests; the consumer owns caching/refresh.

- client.ts: strip the callback, emit the `hasBearerTokenProvider` wire flag,
  register per-provider callbacks on the session.
- session.ts: handle `providerToken.acquire` by dispatching on provider name.
- types.ts: public `getBearerToken` / `ProviderTokenArgs` / `ProviderBearerToken`.
- generated/rpc.ts: regenerated contract (providerToken.acquire +
  hasBearerTokenProvider/bearerTokenScope fields).
- e2e: callback token reaches model, refresh-on-expiry, per-provider dispatch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mirror the runtime contract simplification: the runtime no longer caches
provider tokens, it calls getBearerToken once per outbound request. Drop the
vestigial `scope`/`bearerTokenScope` (the callback closes over its own scope/
audience) and stop forwarding `expiresOnTimestamp` over the wire. The field is
retained on `ProviderBearerToken` so an Azure Identity `AccessToken` can be
returned verbatim, but it is now documented as ignored — caching and refresh
are the callback's responsibility (e.g. `@azure/identity` caches internally).

Rewrite the e2e suite to drive a local capturing HTTP server as the BYOK
endpoint instead of CAPI record/replay snapshots, so the tests assert on the
outbound Authorization header directly and pass identically in record and
replay mode. Delete the obsolete snapshots.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@SteveSandersonMS SteveSandersonMS force-pushed the stevesandersonms/byok-provider-token-rpc branch from 13fd1d7 to 02fb2fc Compare June 23, 2026 10:08
@github-actions

This comment has been minimized.

Replace the real local HTTP listener (CapturingProvider) with the client-global
CopilotRequestHandler interceptor (PR #1689). The handler captures the
Authorization header the runtime applies after resolving getBearerToken over the
providerToken.acquire RPC, and answers with a synthetic 404 entirely in-process:
no socket binding, no listen/close lifecycle, no waitForRequests polling.

Distinct fake .invalid hosts per provider let the per-provider dispatch test
assert routing by host. The handler is installed once per fixture via
createSdkTestContext({ copilotClientOptions: { requestHandler } }) and reset
between tests; non-BYOK (CAPI bootstrap) requests pass through via super.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

The getBearerToken callback previously returned `ProviderBearerToken | string`,
where ProviderBearerToken carried an `expiresOnTimestamp` field that the runtime
accepted but ignored (it does no caching). That advertised a no-op field and an
awkward union for no benefit.

Collapse the return type to `Promise<string>`: the callback resolves a raw token
string and nothing else. Remove the ProviderBearerToken interface, simplify the
providerToken.acquire handler (no more string|object normalization), and export
the public GetBearerToken / ProviderTokenArgs types from the package root so
consumers can type standalone callbacks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

SteveSandersonMS and others added 5 commits June 23, 2026 12:45
…lue + e2e

Rename the session-scoped provider-token RPC method and handler from
`acquire`/`AcquireAsync` to `getToken`/`GetTokenAsync` across the generated
wire contracts (Node, .NET, Python, Go, Rust) to match the runtime source of
truth.

Add the .NET SDK getBearerToken support mirroring Node:
- ProviderTokenArgs with a required ProviderName (no delegate; consumers pass
  a Func<ProviderTokenArgs, Task<string>> lambda).
- hasBearerTokenProvider is now an internal computed wire flag derived from the
  presence of GetBearerToken, so it is no longer settable public API.
- 3-scenario hermetic e2e (callback token, per-request re-acquisition,
  per-provider dispatch) that does not use the capi proxy.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Mirror the Node/.NET getBearerToken surface in the Python SDK:
- ProviderTokenArgs TypedDict + GetBearerToken callable alias.
- get_bearer_token field on ProviderConfig and NamedProviderConfig; wire
  conversion emits hasBearerTokenProvider: true and strips the callback.
- _BearerTokenProviderAdapter routes providerToken.getToken to the matching
  per-provider callback (sync or async tolerant).
- 3-scenario hermetic e2e (callback token, per-request re-acquisition,
  per-provider dispatch) that fabricates bootstrap and does not use the CAPI proxy.

Also fix stale providerToken.acquire references in the Node e2e comments.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# already sent the (token-bearing) request — which is all we assert on.
try:
await session.send_and_wait(prompt)
except Exception:
@github-actions

Copy link
Copy Markdown
Contributor

Cross-SDK Consistency Review ✅

Reviewed the getBearerToken BYOK bearer-token provider feature against all 6 SDK implementations.

Summary

This PR achieves excellent cross-SDK consistency. Notably, the Rust SDK already had a complete implementation of this feature (in rust/src/provider_token.rs, provider_token_dispatch.rs, and integrated into types.rs) — this PR brings the remaining five SDKs up to full parity with Rust.

Consistency checks ✅

Aspect Node.js Python Go .NET Java Rust
getBearerToken on ProviderConfig ✅ (pre-existing)
getBearerToken on NamedProviderConfig ✅ (pre-existing)
RPC method: providerToken.getToken ✅ (pre-existing)
Wire flag: hasBearerTokenProvider ✅ (pre-existing)
Default provider name: "default" ✅ (pre-existing)
ProviderTokenArgs.providerName ✅ (pre-existing)
@experimental marking ✅ (pre-existing)
E2E tests ✅ (pre-existing)

Language-idiomatic differences (expected, not flagged)

  • Rust uses Arc<dyn BearerTokenProvider> (trait + impl for closures) instead of a plain function type — required for Send + Sync in async contexts
  • Go exposes GetBearerToken error via (string, error) return — idiomatic for Go vs exceptions in other languages
  • Java uses a @FunctionalInterface GetBearerToken with getToken(ProviderTokenArgs) — standard Java functional pattern
  • .NET uses Func<ProviderTokenArgs, Task<string>> directly (no dedicated delegate type) alongside a BearerTokenProvider.cs for the args type

No cross-SDK inconsistencies found. The feature is fully implemented across all six languages with appropriate language-idiomatic adaptations.

Generated by SDK Consistency Review Agent for issue #1748 · sonnet46 1.8M ·

Comment on lines +265 to +271
return Task.FromResult(new HttpResponseMessage(HttpStatusCode.NotFound)
{
Content = new StringContent(
"{\"error\":{\"message\":\"fake byok endpoint\"}}",
System.Text.Encoding.UTF8,
"application/json"),
});
Comment thread dotnet/src/Client.cs
Comment on lines +688 to +694
foreach (var provider in config.Providers)
{
if (provider.GetBearerToken is { } callback)
{
callbacks[provider.Name] = callback;
}
}
Comment on lines +84 to +88
catch
{
// The handler always 404s the BYOK endpoint, so the turn errors after
// the token-bearing request was already captured. Expected.
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants