Python: feat(evals): Foundry Adaptive Evals integration (rubric-generation) - #6101

Merged

Ben Thomas (alliscode) merged 16 commits into

microsoft:mainfrom

alliscode:adaptive-evals

Jun 1, 2026

Commits on May 27, 2026

Python: feat(evals): RubricScore type + EvalScoreResult.dimensions

Ben Thomas (alliscode)
and
Copilot
committed
e45b934
Python: feat(foundry-evals): RubricDimension + GeneratedEvaluatorRef + accept in evaluators=

Ben Thomas (alliscode)
and
Copilot
committed
e5830dd
Python: feat(evals): parse rubric_scores from output items + assertion helpers

Ben Thomas (alliscode)
and
Copilot
committed
4bc6046
Python: feat(evals): BaseAgent.as_eval_source / Workflow.as_eval_source

Ben Thomas (alliscode)
and
Copilot
committed
38d51d1
Python: feat(foundry-evals): EvalGenerationSource + generate_rubric helper

Ben Thomas (alliscode)
and
Copilot
committed
a9e4676
Python: feat(foundry-evals): YAML config loader + sample

Ben Thomas (alliscode)
and
Copilot
committed
4c7f94f
Python: fix(evals): address PR review feedback

Ben Thomas (alliscode)
and
Copilot
committed
276fb76
Python: feat(foundry-evals): hosted-agent-aware rubric generation

Ben Thomas (alliscode)
and
Copilot
committed
9a2c964

Commits on May 28, 2026

fix(foundry-evals): accept canonical dimension_scores key per docs

Ben Thomas (alliscode)
and
Copilot
committed
31f8107
feat(foundry-evals): add manual create_rubric_evaluator

Ben Thomas (alliscode)
and
Copilot
committed
f763430
samples(foundry-evals): manual rubric sample + namespace re-exports

Ben Thomas (alliscode)
and
Copilot
committed
484b98d
feat(foundry-evals): remove rubric creation flows; keep consumption only

Ben Thomas (alliscode)
and
Copilot
committed
972b55f
samples(foundry-evals): add evaluate_with_rubric_sample

Ben Thomas (alliscode)
and
Copilot
committed
907c909

Commits on May 29, 2026

fix(foundry-evals): satisfy mypy on _fetch_output_items

Ben Thomas (alliscode)
and
Copilot
committed
b6a558d
docs(foundry-evals): drop unpublished rubric-evaluators learn.microsoft.com link

Ben Thomas (alliscode)
and
Copilot
committed
93cf732

Commits on Jun 1, 2026

test(foundry-evals): hoist repeated local imports to module top

Ben Thomas (alliscode)
and
Copilot
committed
fb3eb7f