-
Notifications
You must be signed in to change notification settings - Fork 8
Open
Description
Is your feature request related to a problem?
When running evaluations, users must provide the model name, vector store ID, and prompt as separate inputs each time. This is repetitive and error-prone, especially when running multiple evaluations with the same configuration.
Describe the solution you'd like
Extend the config management feature (added in #430) to support evaluation workflows:
- Allow saving evaluation parameters (model name, vector store ID, prompt) in a config
- Enable running evaluations by referencing a config instead of passing individual parameters
This provides consistency with how configs are used elsewhere and reduces friction when running repeated evaluations.
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
To Do