feat: add microsoft-foundry skills and tests from fork#1010
feat: add microsoft-foundry skills and tests from fork#1010tendau wants to merge 5 commits intomicrosoft:mainfrom
Conversation
Includes: - microsoft-foundry agent sub-skill - microsoft-foundry troubleshooting sub-skill - microsoft-foundry resource/create sub-skill - microsoft-foundry quota management skill - microsoft-foundry deploy-model with dynamic models and cross-region quota - RBAC management skills - Foundry integration and unit tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This pull request adds comprehensive Microsoft Foundry (Azure AI Foundry) skills and test infrastructure from a fork. The changes introduce end-to-end capabilities for deploying models, managing agents, troubleshooting issues, handling RBAC, and managing quotas within Azure AI Foundry projects.
Changes:
- Adds 5 new troubleshooting sub-skills with diagnostic workflows (429 errors, model availability, deployment health)
- Introduces agent creation skills supporting both Foundry Agent Service and Microsoft Agent Framework SDK
- Implements model deployment with intelligent routing (preset, customize, capacity discovery)
- Provides quota management, RBAC configuration, and resource creation capabilities
- Includes comprehensive test coverage (unit, trigger, integration tests with snapshots)
Reviewed changes
Copilot reviewed 72 out of 74 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| tests/microsoft-foundry/unit.test.ts | Updated expectations to align with refactored skill structure and content |
| tests/microsoft-foundry/troubleshooting/* | Complete test suites for 3 troubleshooting skills (429 errors, availability, health) |
| tests/microsoft-foundry/resource/create/* | Resource creation test updates with path fixes |
| tests/microsoft-foundry/quota/* | Quota management integration test improvements |
| tests/microsoft-foundry/models/deploy/* | Model deployment tests for all deployment modes |
| tests/microsoft-foundry/agent/create/* | Agent framework creation tests with trigger/integration coverage |
| plugin/skills/microsoft-foundry/troubleshooting/references/* | Diagnostic references for rate limiting, scaling, model availability, deployment health |
| plugin/skills/microsoft-foundry/resource/create/* | Resource creation workflow documentation |
| plugin/skills/microsoft-foundry/quota/references/* | Quota optimization and capacity planning guides |
| plugin/skills/microsoft-foundry/project/* | Project creation and connection management |
| plugin/skills/microsoft-foundry/models/deploy-model/* | Model deployment skills with Anthropic support |
| plugin/skills/microsoft-foundry/agent/create/* | Agent creation with multiple frameworks and tools |
| Create AI agents and workflows using Microsoft Agent Framework SDK. Supports single-agent and multi-agent workflow patterns. | ||
| USE FOR: create agent, build agent, scaffold agent, new agent, agent framework, workflow pattern, multi-agent, MCP tools, create workflow. | ||
| DO NOT USE FOR: deploying agents (use agent/deploy), evaluating agents (use agent/evaluate), Azure AI Foundry agents without Agent Framework SDK. | ||
| --- |
There was a problem hiding this comment.
My understanding is that only the top-level SKILL.md (plugin/skills/microsoft-foundry/SKILL.md) is picked up as a skill. So while there's nothing wrong with having SKILL.md files like this one the system won't see these as distinct skills.
There was a problem hiding this comment.
i believe the intention is to create a hierarchy. Even though the Microsoft-foundry skill is the top level skill which is like the parent and would be picked up, the SKILL.MD file has instructions to read the subskill files before responding.
| After selection, verify the project exists and display confirmation: | ||
|
|
||
| ```bash | ||
| az cognitiveservices account show \ |
There was a problem hiding this comment.
Is there an equivalent Azure MCP tool? I'm guessing not, but wanted to check.
|
|
||
| ## Core Workflows | ||
|
|
||
| ### 1. Getting Started - Model Discovery and Deployment |
There was a problem hiding this comment.
Would it make sense to break each of the core workflows out into a separate linked Markdown file? That would significantly reduce the size of this file.
|
|
||
| # Request quota increase (manual process in portal) | ||
| Write-Output "Request quota increase in Azure Portal under Quotas section" | ||
| ``` |
There was a problem hiding this comment.
I can't tell if this is supposed to begin or end a code block but I think the corresponding beginning/ending is missing.
| ## Defaults | ||
|
|
||
| - **Language**: Python | ||
| - **SDK**: Microsoft Agent Framework (pin version `1.0.0b260107`) |
There was a problem hiding this comment.
Are you sure about hardcoding a version here?
| await ctx.send_message(text.upper()) | ||
|
|
||
| @executor(id="reverse_text_executor") | ||
| async def reverse_text(text: str, ctx: WorkflowContext[Never, str]) -> None: |
There was a problem hiding this comment.
Nit: the indentation is inconsistent.
|
|
||
| ## Code Sample | ||
|
|
||
| ```python |
There was a problem hiding this comment.
General comment: since these files provide only Python sample, should they be moved down to a "python" subdirectory?
| @@ -0,0 +1,644 @@ | |||
| # Rate Limiting Retry and Load Balancing Examples | |||
There was a problem hiding this comment.
For consistency this should be named retry-examples.md.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| // The agentId of the created agent | ||
| let targetAgentId: string | undefined = undefined; | ||
| for await (const agent of agentsIter) { | ||
| console.log("Found agent", agent.name) |
Check notice
Code scanning / CodeQL
Semicolon insertion Note test
|
Your PR undo quite a few changes from a previous PR. Are those changes intentional? https://github.com/microsoft/GitHub-Copilot-for-Azure/pull/1004/changes#diff-c5d418f41ef49376666ef566805e989bf5a6f40ba0297088790da1acfc6e818a |
| @@ -0,0 +1,83 @@ | |||
| # Agent as HTTP Server Best Practices | |||
There was a problem hiding this comment.
Kay had earlier suggested to rename agent/create into more specific foundry-agent/create
|
|
||
| ### Run Workflow as Agent | ||
|
|
||
| Agent Framework provides a way to run a whole workflow as agent, via appending `.as_agent()` to the `WorkflowBuilder`, like: |
There was a problem hiding this comment.
This is very specific to hosted agent creation. This is already covered in foundry-agent/create skill.
| @@ -0,0 +1,95 @@ | |||
| # Python Agent Code Samples | |||
There was a problem hiding this comment.
I am not sure what this file is for. If we want to refer to code samples please put a link to our official samples repository.
| ### Thread Pattern (Multi-turn Conversation) | ||
| ``` python | ||
| # Create a new thread that will be reused | ||
| thread = agent.get_new_thread() |
There was a problem hiding this comment.
This is for v1 agents. This is not deprecated.
| ## Prerequisites | ||
|
|
||
| - (REQUIRED) Agent or workflow created using agent-framework SDK | ||
| - (REQUIRED) Running in HTTP server mode, i.e., using `azure.ai.agentserver.agentframework` SDK. If not, wrap the agent with `from_agent_framework(agent).run_async()` and install `azure-ai-agentserver-agentframework==1.0.0b10`. |
There was a problem hiding this comment.
This is for hosted agent. I had removed this file and made the troubleshoot leaner.
| @@ -0,0 +1,75 @@ | |||
| # Workflow with Agents and Streaming | |||
|
|
|||
| Wrap chat agents (via `AzureAIClient`) inside workflow executors and consume streaming events. Use this when building workflows where each node is backed by an AI agent. | |||
There was a problem hiding this comment.
Is this a sample on how to use agent framework sdk? Can we remove the hardcoded code and refer to our official samples?
| @@ -0,0 +1,132 @@ | |||
| # Foundry Project Connections | |||
|
|
|||
| Connections authenticate and link external resources to a Foundry project. Many agent tools (Azure AI Search, Bing Grounding, MCP) require a project connection before use. | |||
There was a problem hiding this comment.
Lets use the foundry MCP tool for creating and managing project connections.
| @@ -0,0 +1,50 @@ | |||
| # Rate Limiting Best Practices | |||
There was a problem hiding this comment.
This looks more of a guide for human than an agent. What is agent expected to do here?
| @@ -0,0 +1,244 @@ | |||
| --- | |||
| name: check-deployment-health | |||
There was a problem hiding this comment.
check-deployment-health is too generic and could mean deployment of agents also. I think this skill is only for troubleshooting model deployments. Should we move it under the models/folder? Similar pattern for agents works really well.
| @@ -0,0 +1,213 @@ | |||
| --- | |||
| name: diagnose-429-errors | |||
There was a problem hiding this comment.
Is this TSG for rate limiting by the model and not by any other foundry services?
| "description": "Use this skill to work with Microsoft Foundry (Azure AI Foundry) and tools from Foundry MCP server: deploy AI models, manage AI agents (create, deploy, invoke, run, troubleshoot Foundry Agents), manage RBAC permissions and role assignments, manage quotas and capacity, create Foundry resources. | ||
| USE FOR: Microsoft Foundry, AI Foundry, create agent, deploy agent, debug agent, invoke agent, run agent, agent chat, evaluate agent, agent monitoring, deploy model, model catalog, knowledge index, create Foundry project, new Foundry project, set up Foundry, onboard to Foundry, create Foundry resource, create AI Services, AIServices kind, register resource provider, enable Cognitive Services, setup AI Services account, create resource group for Foundry, RBAC, role assignment, quota, capacity, TPM, deployment failure, QuotaExceeded. | ||
| DO NOT USE FOR: Azure Functions (use azure-functions), App Service (use azure-create-app), generic Azure resource creation (use azure-create-app). | ||
| "description": "Use this skill to work with Microsoft Foundry (Azure AI Foundry): deploy AI models from catalog, build RAG applications with knowledge indexes, create and evaluate AI agents, manage RBAC permissions and role assignments, manage quotas and capacity, create Foundry resources. |
There was a problem hiding this comment.
Description does not cover all the actions that can be taken by sub-skills. Any skill file below this level is not loaded and is just a reference. So this description needs to be complete with capabilities our skills offer.
There was a problem hiding this comment.
I did some tests locally. hosted agent word is essential for it to pick up this skill, otherwise half the time it just skips using foundry skill.
…oft#1010 - Fix 12 broken language/python.md links in SKILL.md to reference references/sdk/foundry-sdk-py.md (resolves orphaned file too) - Convert static @azure/ai-projects import to dynamic import in integration.test.ts to fix TS2307 (package not in tests/package.json) - Fix 7 bare run() calls to agent.run() in quota/integration.test.ts to fix TS2304 errors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…oft#1010 - Fix 12 broken language/python.md links in SKILL.md to reference references/sdk/foundry-sdk-py.md (resolves orphaned file too) - Convert static @azure/ai-projects import to dynamic import in integration.test.ts to fix TS2307 (package not in tests/package.json) - Fix 7 bare run() calls to agent.run() in quota/integration.test.ts to fix TS2304 errors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…oft#1010 - Fix 12 broken language/python.md links in SKILL.md to reference references/sdk/foundry-sdk-py.md (resolves orphaned file too) - Convert static @azure/ai-projects import to dynamic import in integration.test.ts to fix TS2307 (package not in tests/package.json) - Fix 7 bare run() calls to agent.run() in quota/integration.test.ts to fix TS2304 errors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…oft#1010 - Fix 12 broken language/python.md links in SKILL.md to reference references/sdk/foundry-sdk-py.md (resolves orphaned file too) - Convert static @azure/ai-projects import to dynamic import in integration.test.ts to fix TS2307 (package not in tests/package.json) - Fix 7 bare run() calls to agent.run() in quota/integration.test.ts to fix TS2304 errors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix 12 broken language/python.md links in SKILL.md pointing to references/sdk/foundry-sdk-py.md (also resolves orphaned file) - Use createRequire for @azure/ai-projects in integration.test.ts to avoid TS2307 (package not in tests/package.json) - Fix 7 bare run() calls to agent.run() in quota/integration.test.ts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
|
||
| | Sub-Skill | When to Use | Reference | | ||
| |-----------|-------------|-----------| | ||
| | **deploy** | Containerize, build, push to ACR, create/update/start/stop/clone agent deployments | [deploy](foundry-agent/deploy/deploy.md) | |
| | **resource/create** | Creating Azure AI Services multi-service resource (Foundry resource) using Azure CLI. Use when manually provisioning AI Services resources with granular control. | [resource/create/create-foundry-resource.md](resource/create/create-foundry-resource.md) | | ||
| | **models/deploy-model** | Unified model deployment with intelligent routing. Handles quick preset deployments, fully customized deployments (version/SKU/capacity/RAI), and capacity discovery across regions. Routes to sub-skills: `preset` (quick deploy), `customize` (full control), `capacity` (find availability). | [models/deploy-model/SKILL.md](models/deploy-model/SKILL.md) | | ||
| | **agent/create/agent-framework** | Creating AI agents and workflows using Microsoft Agent Framework SDK. Supports single-agent and multi-agent workflow patterns with HTTP server and F5/debug support. | [agent/create/agent-framework/SKILL.md](agent/create/agent-framework/SKILL.md) | | ||
| | **agent/create/agents** | Managing Foundry Agent Service agents: create, list, get, update, delete prompt agents and workflows. Uses Foundry MCP server with SDK fallback. | [agent/create/agents/SKILL.md](agent/create/agents/SKILL.md) | |
There was a problem hiding this comment.
This overrides the previous PR. Please ensure existing flow is not broken.
| - **Manage quotas** — check usage, troubleshoot quota errors, request increases, plan capacity | ||
| - **Deploy models without an existing project** — this skill handles project discovery and creation automatically | ||
|
|
||
| ## Agent: Project Context Resolution |
There was a problem hiding this comment.
We need this. Please ensure existing flow is not broken. Integration test doesnt work so you will need to ensure complete hosted agent lifecycle works.
Mukeshag/agents skill add websearch
7 participants
Includes: