Skip to content

Add common evaluation workflow examples to docs/examples.md #119

Description

@himanshu231204

Description

The examples documentation page (\docs/examples.md) could be expanded with real-world workflow examples that users can copy-paste to get started quickly.

What to Add

Add short, focused code examples showing:

  1. Quick eval from Python — using the Python SDK directly
  2. Compare two models — running evaluations with GPT-4o vs Claude and comparing results
  3. Custom metric — creating and registering a simple custom metric
  4. CI/CD integration — running eval in a GitHub Action and checking thresholds
  5. Batch evaluation — running multiple configs programmatically

Files to Look At

  • \docs/examples.md\ — where to add
  • \examples/rag_evaluation_tutorial.ipynb\ — existing notebook for reference
  • \openagent_eval/core/engine.py\ — for accurate API examples

Good First Issue ✅

A documentation task that teaches you the core APIs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions