Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .changeset/eve-extension-package.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
"@evlog/eve": minor
"evlog": patch
---

Add `@evlog/eve`, evlog packaged as an installable eve extension.

Mounting it replaces writing `agent/hooks/evlog.ts` by hand:

```ts
// agent/extensions/evlog.ts
import evlog from '@evlog/eve'

export default evlog({ adapter: 'axiom', sample: 0.1 })
```

The mount config is declarative — an adapter by name (or several, which fan out), a sampling rate, redaction paths, batching — because eve validates extension config synchronously and it therefore cannot carry functions. Agents that need a custom `drain`, `enrich` or `keep` keep using `defineEvlogHook` from `evlog/eve`.

The extension also contributes what a hook cannot: an `annotate` tool and a skill that teach the agent to record its own business context on the turn's event, and to keep message content and personal data out of it.

`defineEvlogHook` now warns when it runs more than once in a process. Every hook accumulates into the same turn, so mounting the extension alongside a hand-written hook records token and step counts once per hook.
19 changes: 19 additions & 0 deletions .changeset/eve-instrumentation-bridge.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
---
"evlog": minor
---

Add `defineEvlogInstrumentation()` to `evlog/eve`, linking eve's OpenTelemetry spans to evlog wide events.

A wide event and an Agent Runs span describe the same turn, but nothing joined them: you could not jump from a trace in Braintrust, Datadog or the Vercel dashboard to the event in your drain. Export it as the default export of `agent/instrumentation.ts` and every model-call span — and its children — carries `evlog.request_id` and `evlog.session_id`, the same values the wide event reports.

```ts
// agent/instrumentation.ts
import { defineEvlogInstrumentation } from 'evlog/eve'
import { registerOTel } from '@vercel/otel'

export default defineEvlogInstrumentation({
setup: ({ agentName }) => registerOTel({ serviceName: agentName }),
})
```

`setup` is optional: without it, OpenTelemetry export is untouched and eve keeps recording its local traces, with only the runtime context added. `functionId`, `recordInputs`, `recordOutputs` and `traceChannelRequests` pass straight through to eve.
1 change: 1 addition & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ scope.
- dx (developer experience improvements)
- elysia (Elysia plugin)
- eve (eve agent integration)
- eve-extension (@evlog/eve, the installable eve extension)
- evi (Evi agent)
- express (Express middleware)
- fastify (Fastify plugin)
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/semantic-pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ jobs:
dx
elysia
eve
eve-extension
evi
express
fastify
Expand Down
9 changes: 9 additions & 0 deletions packages/eve-extension/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
node_modules
.env*
.eve
.vercel
.output
.nitro
dist
.DS_Store
*.tsbuildinfo
35 changes: 35 additions & 0 deletions packages/eve-extension/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# @evlog/eve

evlog packaged as an eve extension. Consumers mount it under
`agent/extensions/` instead of writing `agent/hooks/evlog.ts` by hand.

## This package does not build like the others

It builds with `eve extension build`, not tsdown, and the output tree
(`dist/extension`) is agent-shaped rather than a bundle. `eve extension build`
requires **Node >= 24**.

`prepare` and `dev:prepare` are deliberately absent. The release script already
runs `turbo run build --filter='./packages/*'` before `changeset publish`, so
the dist exists when it matters. Wiring the build into `dev:prepare` would put
it on the critical path of `lint`, `typecheck` and `test`, which then fail for
anyone on Node 22 — nothing in this repo consumes this package's dist.

## Config cannot carry functions

The mount config is a Standard Schema, validated synchronously, so `drain`,
`enrich` and `keep` cannot cross the mount boundary. Everything consumers
configure has to be declarative — an adapter name, a sampling rate, redaction
paths. Agents that genuinely need a callback use `defineEvlogHook` from
`evlog/eve` directly; keep that escape hatch documented.

## Layout

- `extension/extension.ts` — the config schema, and the handle contributions read
- `extension/lib/` — shared code: adapter resolution, config → hook options
- `extension/hooks/` — the wide-event hook
- `extension/tools/` — `annotate`, mounted as `<namespace>__annotate`
- `extension/skills/` — procedures taught to the host agent

Tool, skill and connection names come from file paths and the consumer's mount
adds the namespace prefix, so name the file `annotate`, never `evlog_annotate`.
85 changes: 85 additions & 0 deletions packages/eve-extension/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# @evlog/eve

[evlog](https://evlog.dev) packaged as an [eve](https://eve.dev) extension: one
wide event per agent turn, sent to the destination of your choice, without
writing a hook.

## Install

```bash
pnpm add @evlog/eve
```

```ts
// agent/extensions/evlog.ts
import evlog from '@evlog/eve'

export default evlog({ adapter: 'axiom', sample: 0.1 })
```

That is the whole setup. Every turn now produces an event carrying token usage,
cost, tool executions, subagent delegations, approvals, connection
authorizations, compactions and failures.

## Config

| Option | Default | What it does |
| --- | --- | --- |
| `adapter` | `'fs'` | Destination by name, or `{ type, options }`. An array fans out. |
| `service` | agent name | Service name on every event. |
| `message` | `'omit'` | `'omit'`, `'preview'` or `'full'` user message capture. |
| `messagePreviewLength` | `500` | Characters kept in `'preview'`. |
| `redact` | `true` | Built-in PII patterns, or `{ paths }`. |
| `sample` | keep all | Fraction of routine turns to keep, 0 to 1. |
| `keepOnFailure` | `true` | Always keep failed, rejected and declined turns. |
| `sessionEvent` | `false` | One extra event per session, rolling up its turns. |
| `batch` | none | `{ size, intervalMs }` buffering before draining. |

Adapters read their own environment variables, so `adapter: 'axiom'` with
`AXIOM_TOKEN` and `AXIOM_DATASET` set is usually all you need. `options`
overrides what the environment cannot express.

Available adapters: `axiom`, `better-stack`, `clickhouse`, `datadog`, `fs`,
`hyperdx`, `loki`, `memory`, `otlp`, `posthog`, `sentry`.

## What the agent can do itself

The mount contributes an `annotate` tool (namespaced `evlog__annotate` for a
mount named `evlog.ts`) and a skill teaching the agent when to use it. The agent
records business facts on its own turn:

```
annotate({ key: "refund", value: { amount: 89.9, reason: "damaged_on_arrival" } })
```

Those fields are what make a turn findable later. The skill also tells the agent
what never to record: message content, credentials, personal data.

## When to use the hook instead

The mount config is validated synchronously, so it cannot carry functions. If
you need a custom `drain`, `enrich` or `keep`, skip the extension and use the
hook directly:

```ts
// agent/hooks/evlog.ts
import { defineEvlogHook } from 'evlog/eve'

export default defineEvlogHook({
drain: myDrain,
enrich: ctx => void (ctx.event.region = process.env.VERCEL_REGION),
})
```

Use one or the other. Mounting both makes every hook accumulate into the same
turn, which records token and step counts once per hook.

## Correlating with OpenTelemetry

`evlog/eve` also exports `defineEvlogInstrumentation()`, which stamps
`evlog.request_id` onto eve's AI SDK spans so a trace joins back to the wide
event. See the [eve integration docs](https://evlog.dev/use-cases/eve).

## License

MIT
62 changes: 62 additions & 0 deletions packages/eve-extension/extension/extension.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
import { defineExtension } from 'eve/extension'
import { z } from 'zod'

/** Drain adapters the extension can resolve by name. */
export const ADAPTER_NAMES = [
'axiom',
'better-stack',
'clickhouse',
'datadog',
'fs',
'hyperdx',
'loki',
'memory',
'otlp',
'posthog',
'sentry',
] as const

const adapter = z.union([
z.enum(ADAPTER_NAMES),
z.object({
type: z.enum(ADAPTER_NAMES),
/** Passed straight to the adapter factory, overriding its env defaults. */
options: z.record(z.string(), z.unknown()).optional(),
}),
])

/**
* Consumer-facing settings. Everything is declarative: a Standard Schema is
* validated synchronously at the mount site, so it cannot carry functions.
* Agents needing a custom `drain`, `enrich` or `keep` use `defineEvlogHook`
* from `evlog/eve` in `agent/hooks/` instead.
*/
export default defineExtension({
config: z.object({
/** Service name on every event. Defaults to the agent name. */
service: z.string().optional(),
/** Where events go. Several adapters fan out. */
adapter: z.union([adapter, z.array(adapter)]).default('fs'),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Reject an empty adapter list.

adapter: [] passes this schema. resolveDrain() then creates no drains, and events complete without delivery or an error. Require at least one adapter.

Proposed fix
-    adapter: z.union([adapter, z.array(adapter)]).default('fs'),
+    adapter: z.union([adapter, z.array(adapter).min(1)]).default('fs'),
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
adapter: z.union([adapter, z.array(adapter)]).default('fs'),
adapter: z.union([adapter, z.array(adapter).min(1)]).default('fs'),
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/eve-extension/extension/extension.ts` at line 39, Update the adapter
schema definition to reject empty adapter arrays by applying a minimum-length
constraint to the array branch of the z.union. Preserve support for a single
adapter value and non-empty adapter lists so resolveDrain() always has at least
one adapter.

/** How much of the user message to record. */
message: z.enum(['omit', 'preview', 'full']).default('omit'),
/** Max characters kept in `preview` mode. */
messagePreviewLength: z.number().int().positive().optional(),
/** PII auto-redaction: `true` for the built-in patterns, or explicit paths. */
redact: z
.union([z.boolean(), z.object({ paths: z.array(z.string()) })])
.default(true),
/** Fraction of routine turns to keep, between 0 and 1. Unset keeps all. */
sample: z.number().min(0).max(1).optional(),
/** Always keep turns that failed, were rejected, or lost an authorization. */
keepOnFailure: z.boolean().default(true),
/** Emit one extra wide event per session, rolling up its turns. */
sessionEvent: z.boolean().default(false),
/** Batch events before draining them. */
batch: z
.object({
size: z.number().int().positive(),
intervalMs: z.number().int().positive(),
})
.optional(),
}),
})
14 changes: 14 additions & 0 deletions packages/eve-extension/extension/hooks/wide-events.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import { defineEvlogHook } from 'evlog/eve'
import extension from '../extension'
import { toHookOptions, type ExtensionConfig } from '../lib/options'

/**
* One evlog wide event per agent turn, wired from the mount config.
*
* The agent name is not available to a hook file at module scope, so the
* service falls back to the mount's `service` and is otherwise resolved from
* the turn context by evlog itself.
*/
export default defineEvlogHook(
toHookOptions(extension.config as ExtensionConfig, extension.config.service ?? 'eve-agent'),
Comment on lines +5 to +13

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Correct the service fallback documentation.

Line 10 says evlog resolves an omitted service from turn context. Line 13 passes 'eve-agent' to toHookOptions, and the option translator writes that value to init.env.service. The service is therefore fixed to 'eve-agent' when mount configuration omits service.

Update the JSDoc to describe the fixed fallback, or change the implementation to resolve the agent name as documented.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/eve-extension/extension/hooks/wide-events.ts` around lines 5 - 13,
Correct the mismatch between the JSDoc and implementation in the default evlog
hook: update the comment to state that an omitted mount service is fixed to
“eve-agent” via toHookOptions and init.env.service, or change the fallback
behavior so the service is resolved from turn context as documented.

)
5 changes: 5 additions & 0 deletions packages/eve-extension/extension/instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Every turn you run is recorded as one observability event.

Call the `annotate` tool when you learn a business fact that would make this turn
findable later — an identifier you looked up, a decision you made, an amount you
moved. Do not pass user message content, credentials, or personal data.
84 changes: 84 additions & 0 deletions packages/eve-extension/extension/lib/adapters.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import type { DrainContext } from 'evlog'
import { createAxiomDrain } from 'evlog/axiom'
import { createBetterStackDrain } from 'evlog/better-stack'
import { createClickHouseDrain } from 'evlog/clickhouse'
import { createDatadogDrain } from 'evlog/datadog'
import { createFsDrain } from 'evlog/fs'
import { createHyperDXDrain } from 'evlog/hyperdx'
import { createLokiDrain } from 'evlog/loki'
import { createMemoryDrain } from 'evlog/memory'
import { createOTLPDrain } from 'evlog/otlp'
import { createPostHogDrain } from 'evlog/posthog'
import { createSentryDrain } from 'evlog/sentry'
import { createDrainPipeline } from 'evlog/pipeline'
import type { ADAPTER_NAMES } from '../extension'

type AdapterName = (typeof ADAPTER_NAMES)[number]
/** Adapters accept a single context or a batch, which is what lets them sit behind a pipeline. */
type AdapterDrain = (ctx: DrainContext | DrainContext[]) => void | Promise<void>
/** What `BaseEvlogOptions.drain` expects: one event at a time. */
type HookDrain = (ctx: DrainContext) => void | Promise<void>
type DrainFactory = (overrides?: Record<string, unknown>) => AdapterDrain

/**
* Every adapter factory reads its own env vars, so a bare name is enough for
* the common case and `options` only overrides what the environment cannot.
*/
const FACTORIES: Record<AdapterName, DrainFactory> = {
'axiom': createAxiomDrain as DrainFactory,
'better-stack': createBetterStackDrain as DrainFactory,
'clickhouse': createClickHouseDrain as DrainFactory,
'datadog': createDatadogDrain as DrainFactory,
'fs': createFsDrain as DrainFactory,
'hyperdx': createHyperDXDrain as DrainFactory,
'loki': createLokiDrain as DrainFactory,
'memory': createMemoryDrain as DrainFactory,
'otlp': createOTLPDrain as DrainFactory,
'posthog': createPostHogDrain as DrainFactory,
'sentry': createSentryDrain as DrainFactory,
}

export type AdapterConfig =
| AdapterName
| { type: AdapterName, options?: Record<string, unknown> }

export interface ResolveDrainOptions {
adapter: AdapterConfig | AdapterConfig[]
batch?: { size: number, intervalMs: number }
}
Comment on lines +41 to +48

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Add JSDoc to exported configuration types.

AdapterConfig and ResolveDrainOptions are public TypeScript APIs. Document their accepted forms and batching behavior.

As per coding guidelines, “Write JSDoc for all public TypeScript APIs.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/eve-extension/extension/lib/adapters.ts` around lines 41 - 48, Add
JSDoc comments to the exported AdapterConfig type and ResolveDrainOptions
interface, documenting AdapterConfig’s accepted adapter-name or typed-options
forms and ResolveDrainOptions’ adapter input plus optional batch size and
interval behavior. Keep the existing type definitions unchanged.

Source: Coding guidelines


function createDrain(config: AdapterConfig): AdapterDrain {
const { type, options } = typeof config === 'string' ? { type: config, options: undefined } : config
return FACTORIES[type](options)
}

/** Each destination runs independently so a failing one cannot starve the others. */
async function fanOut(
drains: AdapterDrain[],
payload: DrainContext | DrainContext[],
): Promise<void> {
await Promise.all(drains.map(async (send) => {
try {
await send(payload)
} catch (err) {
console.error('[evlog] eve extension drain failed:', err)
}
}))
}

/**
* Build the single drain the hook receives. With `batch`, the adapters sit
* behind a pipeline that buffers events and hands them over as an array —
* which every evlog adapter accepts.
*/
export function resolveDrain(options: ResolveDrainOptions): HookDrain {
const configs = Array.isArray(options.adapter) ? options.adapter : [options.adapter]
const drains = configs.map(createDrain)

if (options.batch) {
return createDrainPipeline<DrainContext>({ batch: options.batch })(
(batch: DrainContext[]) => fanOut(drains, batch),
)
}
return (ctx: DrainContext) => fanOut(drains, ctx)
}
Loading