Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .changeset/eve-instrumentation-bridge.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
"evlog": minor
---

Add `defineEvlogInstrumentation()` to `evlog/eve`, linking eve's OpenTelemetry spans to evlog wide events.

A wide event and an Agent Runs span describe the same turn, but nothing joined them: you could not jump from a trace in Braintrust, Datadog or the Vercel dashboard to the event in your drain. Export it as the default export of `agent/instrumentation.ts` and every model-call span — and its children — carries `evlog.request_id` and `evlog.session_id`, the same values the wide event reports.

```ts
// agent/instrumentation.ts
import { defineEvlogInstrumentation } from 'evlog/eve'
import { registerOTel } from '@vercel/otel'

export default defineEvlogInstrumentation({
setup: ({ agentName }) => registerOTel({ serviceName: agentName }),
})
```

`setup` is optional: without it, OpenTelemetry export is untouched and eve keeps recording its local traces, with only the runtime context added. `functionId`, `recordInputs`, `recordOutputs` and `traceChannelRequests` pass straight through to eve.
1 change: 1 addition & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ scope.
- dx (developer experience improvements)
- elysia (Elysia plugin)
- eve (eve agent integration)
- eve-extension (@evlog/eve, the installable eve extension)
- evi (Evi agent)
- express (Express middleware)
- fastify (Fastify plugin)
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/semantic-pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ jobs:
dx
elysia
eve
eve-extension
evi
express
fastify
Expand Down
85 changes: 85 additions & 0 deletions packages/evlog/src/eve/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
import { AsyncLocalStorage } from 'node:async_hooks'
import { defineHook, type HookContext, type HookDefinition } from 'eve/hooks'
import {
defineInstrumentation,
type InstrumentationDefinition,
type InstrumentationSetupContext,
type InstrumentationStepStartedEventInput,
type InstrumentationStepStartedEventResult,
} from 'eve/instrumentation'
import type { AuditableLogger } from '../audit'
import type { AIToolExecution, AIEventData, ModelCost } from '../ai/index'
import { initLogger, isLoggerInitialized, isLoggerLocked } from '../logger'
Expand Down Expand Up @@ -1333,6 +1340,84 @@ export function defineEvlogHook(options: EvlogEveOptions = {}): HookDefinition {
})
}

/** Options for {@link defineEvlogInstrumentation}. */
export interface EvlogEveInstrumentationOptions {
/** Overrides `ai.telemetry.functionId` on spans. Defaults to the agent name. */
functionId?: string
/** Whether the AI SDK records full model inputs on spans. eve defaults to `true`. */
recordInputs?: boolean
/** Whether the AI SDK records model outputs on spans. eve defaults to `true`. */
recordOutputs?: boolean
/** Whether eve emits the inbound HTTP `SERVER` span wrapping each channel request. */
traceChannelRequests?: boolean
/**
* Runs at server startup with the resolved agent name — register your OTel
* provider here, exactly as you would in a hand-written
* `defineInstrumentation`. Omit it and eve keeps writing its local traces.
*/
setup?: (context: InstrumentationSetupContext) => void
}

/**
* Per-model-call context linking an AI SDK span back to the evlog wide event
* for the same turn. Returns `undefined` outside a tracked turn, which
* contributes no context rather than a half-filled one.
*/
Comment on lines +1361 to +1365

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Remove the private implementation comment.

buildInstrumentationContext is private. Its JSDoc restates behavior that the type and code express. Delete the comment.

As per coding guidelines, “Use comments only for constraints not expressible in code; do not paraphrase or narrate implementation.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/evlog/src/eve/index.ts` around lines 1361 - 1365, Remove the JSDoc
comment immediately preceding the private buildInstrumentationContext
implementation, leaving the function and its behavior unchanged.

Source: Coding guidelines

function buildInstrumentationContext(
input: InstrumentationStepStartedEventInput,
): InstrumentationStepStartedEventResult | undefined {
const state = getTurnState(input.session.id, input.turn.id)
if (!state) return undefined

return {
runtimeContext: {
'evlog.request_id': state.turnId,
'evlog.session_id': state.sessionId,
},
}
}

/**
* Create an eve instrumentation definition that stamps evlog's turn identity
* onto the AI SDK telemetry spans.
*
* Export the result as the default export of `agent/instrumentation.ts`. Every
* model-call span — and its children — then carries `evlog.request_id`, the
* same value the wide event reports as `requestId`, so a trace in Braintrust,
* Datadog or Agent Runs joins to the wide event in your drain, and back.
*
* `eve.` is reserved for framework-owned context, so evlog writes under
* `evlog.`. Passing no `setup` leaves OpenTelemetry export untouched: eve keeps
* recording its local traces and only the runtime context is added.
*
* @example
* ```ts
* // agent/instrumentation.ts
* import { defineEvlogInstrumentation } from 'evlog/eve'
* import { registerOTel } from '@vercel/otel'
*
* export default defineEvlogInstrumentation({
* setup: ({ agentName }) => registerOTel({ serviceName: agentName }),
* })
* ```
*/
export function defineEvlogInstrumentation(
options: EvlogEveInstrumentationOptions = {},
): InstrumentationDefinition {
return defineInstrumentation({
...(options.functionId !== undefined ? { functionId: options.functionId } : {}),
...(options.recordInputs !== undefined ? { recordInputs: options.recordInputs } : {}),
...(options.recordOutputs !== undefined ? { recordOutputs: options.recordOutputs } : {}),
...(options.traceChannelRequests !== undefined
? { traceChannelRequests: options.traceChannelRequests }
: {}),
...(options.setup !== undefined ? { setup: options.setup } : {}),
events: {
'step.started': buildInstrumentationContext,
},
})
}

/** @internal Simulates eve tool execution where AsyncLocalStorage did not propagate. */
export function detachActiveTurnLoggerForTests(): void {
const logger = turnLoggerStorage.getStore()
Expand Down
78 changes: 78 additions & 0 deletions packages/evlog/test/eve.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import { initLogger } from '../src/logger'
import {
resetEvlogEveForTests,
defineEvlogHook,
defineEvlogInstrumentation,
useLogger,
detachActiveTurnLoggerForTests,
} from '../src/eve/index'
Expand Down Expand Up @@ -1549,3 +1550,80 @@ describe('evlog/eve', () => {
initSpy.mockRestore()
})
})

describe('defineEvlogInstrumentation', () => {
beforeEach(() => {
resetEvlogEveForTests()
initLogger({ env: { service: 'eve-test' } })
})

afterEach(() => {
resetEvlogEveForTests()
})

function stepStartedInput(overrides: { sessionId?: string, turnId?: string } = {}) {
return {
channel: { kind: 'http' },
modelInput: { instructions: undefined, messages: [] },
session: { id: overrides.sessionId ?? SESSION_ID, auth: { current: null, initiator: null } },
step: { index: 0 },
turn: { id: overrides.turnId ?? TURN_ID, sequence: 0 },
} as Parameters<NonNullable<NonNullable<ReturnType<typeof defineEvlogInstrumentation>['events']>['step.started']>>[0]
}

it('links the model-call span to the wide event of the active turn', () => {
const hook = defineEvlogHook({})
const instrumentation = defineEvlogInstrumentation()
const ctx = hookContext()

hook.events!['turn.started']!({
type: 'turn.started',
data: { sequence: 0, turnId: TURN_ID },
}, ctx)

expect(instrumentation.events!['step.started']!(stepStartedInput())).toEqual({
runtimeContext: {
'evlog.request_id': TURN_ID,
'evlog.session_id': SESSION_ID,
},
})
})

it('contributes no context outside a tracked turn', () => {
const instrumentation = defineEvlogInstrumentation()

expect(instrumentation.events!['step.started']!(stepStartedInput())).toBeUndefined()
})

it('does not throw when no hook is registered', () => {
const instrumentation = defineEvlogInstrumentation()

expect(() => instrumentation.events!['step.started']!(stepStartedInput())).not.toThrow()
})

it('passes capture settings and setup through to eve', () => {
const setup = vi.fn()
const instrumentation = defineEvlogInstrumentation({
functionId: 'support-agent',
recordInputs: false,
recordOutputs: false,
traceChannelRequests: true,
setup,
})

expect(instrumentation).toMatchObject({
functionId: 'support-agent',
recordInputs: false,
recordOutputs: false,
traceChannelRequests: true,
})
instrumentation.setup!({ agentName: 'support-agent' })
expect(setup).toHaveBeenCalledWith({ agentName: 'support-agent' })
})

it('omits capture settings that were not configured', () => {
const instrumentation = defineEvlogInstrumentation()

expect(Object.keys(instrumentation)).toEqual(['events'])
})
})
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ exports[`public API surface > matches snapshot for all subpath exports 1`] = `
],
"./eve": [
"defineEvlogHook",
"defineEvlogInstrumentation",
"detachActiveTurnLoggerForTests",
"resetEvlogEveForTests",
"useLogger",
Expand Down
Loading