Fix 404 errors in Azure OpenAI Graders by reverting to stable API version

[Copilot]

Azure OpenAI Graders fail with 404 Not Found when calling evaluate() due to broken API specifications in 2025-04-01-preview (see Azure/azure-rest-api-specs#33724). The spec mixes OpenAPI 3.0/3.1 standards and has orphaned references, causing evals.create endpoint failures.

Changes

• _constants.py: Revert DEFAULT_AOAI_API_VERSION from 2025-04-01-preview to 2024-12-01-preview
• CHANGELOG.md: Document fix in 1.15.1 (Unreleased)

Context

The grader forces API version internally via aoai_grader.py:116, making this transparent to users. All existing tests and samples already validate against 2024-12-01-preview. User-provided api_version in model configs (used for chat completions) remains unaffected.

# In aoai_grader.py get_client():
return AzureOpenAI(
    azure_endpoint=model_config["azure_endpoint"],
    api_version=DEFAULT_AOAI_API_VERSION,  # Now uses working version
    ...
)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• learn.microsoft.com
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)
• scanning-api.github.com
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[BUG] 404 Not Found on evaluate() with Azure OpenAI Graders - Possible Service/SDK Incompatibility</issue_title>
<issue_description>

Description

I am encountering a persistent 404 Not Found error when attempting to run the Azure OpenAI Graders workflow, both via the Python SDK and the Azure AI Foundry (Portal) GUI. This occurs even when using the official sample dataset and following the lab documentation.

1. Environment & Setup

• Notebook Source: scenarios/evaluate/Azure_OpenAI_Graders/Azure_OpenAI_Graders.ipynb

• Python Version: 3.10 (Miniconda)

• Libraries: azure-ai-evaluation, openai (latest)

• Region: canadaeast

• <b dat...

• Fixes [BUG] 404 Not Found on evaluate() with Azure OpenAI Graders - Possible Service/SDK Incompatibility #44763

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.