Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions apps/web/src/content/docs/docs/getting-started/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,14 @@ agentv init

## 3. Configure environment variables

The init command creates a `.env.example` file in your project root.
The init command creates a `.env.example` file in your project root. You can either export these
variables in your shell/CI environment directly or copy `.env.example` to `.env` for local
development.

1. Copy `.env.example` to `.env`
2. Fill in your API keys, endpoints, and other configuration values
3. Update the environment variable names in `.agentv/targets.yaml` to match those defined in your `.env` file
3. Update the environment variable names in `.agentv/targets.yaml` to match the variables you
exported or defined in `.env`

## 4. Create an eval

Expand Down
7 changes: 5 additions & 2 deletions apps/web/src/content/docs/docs/targets/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ targets:

## Environment Variables

Use `${{ VARIABLE_NAME }}` syntax to reference values from your `.env` file:
Use `${{ VARIABLE_NAME }}` syntax to reference values from your environment. AgentV reads
exported process environment variables directly, and it also loads `.env` files from the
eval directory hierarchy when present:

```yaml
targets:
Expand All @@ -40,7 +42,8 @@ targets:
model: ${{ ANTHROPIC_MODEL }}
```

This keeps secrets out of version-controlled files.
This keeps secrets out of version-controlled files and avoids requiring a CI step that rewrites
already-exported secrets into `.env`.

## Supported Providers

Expand Down
27 changes: 27 additions & 0 deletions packages/core/test/evaluation/interpolation-integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,33 @@ describe('env interpolation in YAML loading', () => {
expect(cases[0].criteria).toBe('Must return correct answer');
});

it('uses exported process.env values even when no .env file exists', async () => {
const evalFile = path.join(testDir, 'interp-process-env.eval.yaml');
await writeFile(
evalFile,
[
'workspace:',
' repos:',
' - path: ./RepoA',
' source:',
' type: local',
' path: "${{ AGENTV_TEST_PATH }}"',
'tests:',
' - id: test-1',
' input: "hello"',
' criteria: "${{ AGENTV_TEST_CRITERIA }}"',
'',
].join('\n'),
);

const cases = await loadTests(evalFile, testDir);
expect(cases[0].criteria).toBe('Must return correct answer');
expect(cases[0].workspace?.repos?.[0]?.source).toEqual({
type: 'local',
path: '/abs/path/to/repo',
});
});

it('interpolates ${{ VAR }} in workspace repo source path', async () => {
const evalFile = path.join(testDir, 'interp-workspace.eval.yaml');
await writeFile(
Expand Down
Loading