feat: Add auto and wrapper instrumentation for @github/copilot-sdk#1932
Open
Stephen Belanger (Qard) wants to merge 2 commits intomainfrom
Open
feat: Add auto and wrapper instrumentation for @github/copilot-sdk#1932Stephen Belanger (Qard) wants to merge 2 commits intomainfrom
Stephen Belanger (Qard) wants to merge 2 commits intomainfrom
Conversation
Adds full Braintrust tracing support for the GitHub Copilot SDK
(`@github/copilot-sdk` v0.3+). Both auto-instrumentation (via
`--import braintrust/hook.mjs` / bundler plugins) and a manual
`wrapCopilotClient(client)` wrapper are provided.
Span tree produced per session:
Copilot Session (TASK) → Copilot Turn (TASK) → github.copilot.llm (LLM)
→ tool: <name> (TOOL)
→ Agent: <name> (TASK, sub-agents)
Token metrics on LLM spans use the Anthropic cache-token helpers:
prompt_tokens, completion_tokens, prompt_cached_tokens,
prompt_cache_creation_tokens, completion_reasoning_tokens, reasoning_tokens,
tokens. Copilot-specific billing data (cost multiplier, quota snapshots,
copilot_usage) flows into namespaced metadata rather than metrics.
Also fixes imports.test.ts to skip node_modules and .d.ts files when
scanning for dynamic import violations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…n to 3.10.0
- Remove unused `ChannelMessage` import in github-copilot-plugin.ts (lint error)
- Fix `anthropic.test.ts`: system prompt says "Just the poem" so asserting
`output.contains("shakespeare")` is wrong — the model correctly returns only
the poem content without mentioning the author
- Apply all pending changesets via `changeset version` to bump braintrust to
3.10.0, so the API compatibility test detects a `minor` version bump rather
than `none` and does not flag optional interface additions as breaking
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--import braintrust/hook.mjsand bundler plugins) and awrapCopilotClient(client)manual wrapper for@github/copilot-sdkv0.3+prompt_tokens,completion_tokens,prompt_cached_tokens,prompt_cache_creation_tokens,completion_reasoning_tokens,reasoning_tokens,tokenscost,quotaSnapshots,copilotUsage) flow into namespacedgithub_copilot.*metadata, not metricsCopilotClient.{createSession,resumeSession}indist/client.jsandCopilotSession.sendAndWaitindist/session.js(both ESM and CJS)imports.test.tsto skipnode_modulesand.d.tsfiles during the dynamic-import scan (those directories can be populated by test fixture installs)Architecture notes
The Copilot SDK is an agent-runtime client (JSON-RPC over a CLI subprocess), not a chat-completions wrapper — so instrumentation follows the lifecycle/processor pattern used for
@anthropic-ai/claude-agent-sdkrather than the per-method pattern used for Groq/OpenAI.Span lifecycle is driven by the
assistant.turn_start/turn_endevent stream andassistant.usage(per-LLM-call metrics). Tool and sub-agent spans usetool.execution_start/completeandsubagent.started/completed/failedevents. Session close is triggered by the injectedonSessionEndhook.Parent span IDs are resolved by awaiting
span.export()(same pattern asclaude-agent-sdk-plugin.ts). Async event handlers are chained throughstate.processing: Promise<void>to preserve ordering.Test plan
tsc --noEmitclean (no type errors)pnpm run buildsucceedspnpm run lint— warnings only (same pattern as rest of codebase)pnpm run formattingcleanextractMetricsFromUsage(all token/cache/reasoning/billing fields), plugin lifecycle, orchestrion config shape,BraintrustPluginwiringe2e/scenarios/github-copilot-instrumentation/requires a GitHub Copilot subscription or a BYOK provider endpoint (BRAINTRUST_E2E_MODEL_BASE_URL)🤖 Generated with Claude Code