Skip to content

[TOU] Make tool definitions required in validation #45267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ahibrahimm merged 6 commits into from
Feb 20, 2026
Merged

[TOU] Make tool definitions required in validation #45267

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
cff2b6d
make tool defs req
ahibrahimm Feb 19, 2026
ad4d908
update doc
ahibrahimm Feb 19, 2026
acae690
black
ahibrahimm Feb 19, 2026
3ccdd16
Merge branch 'main' into ahibrahim/tou
ahibrahimm Feb 20, 2026
85b1575
Merge branch 'main' into ahibrahim/tou
ahibrahimm Feb 20, 2026
e6c0691
Merge branch 'main' into ahibrahim/tou
ahibrahimm Feb 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions ...tion/azure/ai/evaluation/_evaluators/_tool_output_utilization/_tool_output_utilization.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,7 @@ def __init__(

# Initialize input validator
self._validator = ToolDefinitionsValidator(
error_target=ErrorTarget.TOOL_OUTPUT_UTILIZATION_EVALUATOR,
check_for_unsupported_tools=True,
error_target=ErrorTarget.TOOL_OUTPUT_UTILIZATION_EVALUATOR, optional_tool_definitions=False
)

super().__init__(
Expand Down Expand Up @@ -143,7 +142,7 @@ def __call__(
:paramtype query: Union[str, List[dict]]
:keyword response: The response being evaluated, either a string or a list of messages (full agent response potentially including tool calls)
:paramtype response: Union[str, List[dict]]
:keyword tool_definitions: An optional list of messages containing the tool definitions the agent is aware of.
:keyword tool_definitions: A list of messages containing the tool definitions the agent is aware of.
:paramtype tool_definitions: Union[dict, List[dict]]
:return: A dictionary with the tool output utilization evaluation results.
:rtype: Dict[str, Union[str, float]]
Expand Down