Skip to content

feat(cli): support repeatable test-id filters#929

Merged
christso merged 1 commit intomainfrom
feat/924-repeatable-test-id
Apr 4, 2026
Merged

feat(cli): support repeatable test-id filters#929
christso merged 1 commit intomainfrom
feat/924-repeatable-test-id

Conversation

@christso
Copy link
Copy Markdown
Collaborator

@christso christso commented Apr 4, 2026

Closes #924

Summary

  • make agentv eval --test-id repeatable with OR matching
  • pass multiple filters through the CLI and core loader/orchestrator pipeline
  • add CLI integration and JSONL loader coverage
  • clarify in AGENTS.md that issue branches must start from freshly updated main

Red/green UAT

  • Red (main): agentv eval sample.test.yaml --test-id case-alpha --test-id case-beta exits with Too many times provided. Expected 1, got: 2.
  • Green (this branch): the same command succeeds and runs exactly case-alpha and case-beta.

Verification

  • bun test apps/cli/test/eval.integration.test.ts packages/core/test/evaluation/loaders/jsonl-parser.test.ts
  • pre-push hook on git push passed: build, typecheck, lint, test, validate:examples

@cloudflare-workers-and-pages
Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: d77a199
Status: ✅  Deploy successful!
Preview URL: https://9f439652.agentv.pages.dev
Branch Preview URL: https://feat-924-repeatable-test-id.agentv.pages.dev

View logs

@christso christso merged commit 133d874 into main Apr 4, 2026
4 checks passed
@christso christso deleted the feat/924-repeatable-test-id branch April 4, 2026 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: support repeatable --test-id flag in eval run

1 participant