diff --git a/README.md b/README.md index 534d2aaf..1b5c74c9 100644 --- a/README.md +++ b/README.md @@ -260,23 +260,73 @@ The AI agent definition would likely be deployed from your application's pipelin | :information: | You’ve just persisted a new versioned agent in Foundry AI Agent Service, including its instructions, tools, and model. The platform has stored a canonical agent definition in the `enterprise_memory` database, making the agent addressable, executable and ready for evaluation. At this stage, the agent is available for validation, and has the `unpublished` state. Because this is your first agent, this step is also when the Foundry project provisions a default agent identity blueprint and a default agent identity for your project in Microsoft Entra Agent ID. All `unpublished` agents within the same Foundry project share this default agent identity until they are `published`.| | :-------: | :------------------------- | +1. Persist Agent v2 (Optional) + + *This step creates a second version of the agent with different instructions. New versions are created by posting to the agent-specific versions endpoint. This is useful for A/B testing or gradual rollouts.* + + ```powershell + # Use the v2 agent definition (has concise response instructions) + Invoke-WebRequest -Uri "https://github.com/Azure-Samples/microsoft-foundry-baseline/raw/refs/heads/main/agents/chat-with-bing-v2.json" -OutFile "chat-with-bing-v2.json" + + # Update to match your environment + $chat_agent_v2 = Get-Content .\chat-with-bing-v2.json -Raw | ConvertFrom-Json + + $chat_agent_v2.definition.model = $MODEL_CONNECTION_NAME + $chat_agent_v2.definition.tools[0].bing_grounding.search_configurations[0].project_connection_id = $BING_CONNECTION_ID + + $chat_agent_v2 | ConvertTo-Json -Depth 10 | Set-Content .\chat-with-bing-v2-output.json + + # Persist agent v2 (post to agent-specific versions endpoint) + $FOUNDRY_AGENT_VERSIONS_URL="https://${FOUNDRY_NAME}.services.ai.azure.com/api/projects/${FOUNDRY_PROJECT_NAME}/agents/baseline-chatbot-agent/versions?api-version=2025-11-15-preview" + az rest -u $FOUNDRY_AGENT_VERSIONS_URL -m "post" --resource "https://ai.azure.com" -b @chat-with-bing-v2-output.json + ``` + + | :information: | New agent versions are created by posting to the `/agents/{agent-name}/versions` endpoint. The v2 instructions tell the agent to keep responses concise (2-3 sentences), making it easy to distinguish v1 vs v2 behavior when testing. | + | :-------: | :------------------------- | + 1. Publish the the Agent - *This step publishes the agent by creating a new application within the Foundry project and a corresponding deployment that references a specific agent version.* + *This step publishes the agent by creating a new application within the Foundry project and a corresponding deployment that references a specific agent version. The deployment name is auto-suffixed with the version (e.g., `agentdeploychatv1`).* ```bash + # Deploy version 1 (default) az deployment group create -f ./infra-as-code/bicep/ai-foundry-appdeploy.bicep \ -g $RESOURCE_GROUP \ - -n 'foundryAppDeploy' \ - -p baseName=${BASE_NAME} + -n 'foundryAppDeployV1' \ + -p baseName=${BASE_NAME} \ + -p agentVersion='1' + ``` + + To deploy additional versions (e.g., v2), run the same command with a different `agentVersion`: + + ```bash + # Deploy version 2 + az deployment group create -f ./infra-as-code/bicep/ai-foundry-appdeploy.bicep \ + -g $RESOURCE_GROUP \ + -n 'foundryAppDeployV2' \ + -p baseName=${BASE_NAME} \ + -p agentVersion='2' ``` | :information: | As a result, the agent becomes a nested Azure resource visible in the Azure control plane. Publishing the chat agent automatically created a dedicated agent identity blueprint and agent identity. Both are bound to the Azure Foundry application resource. This distinct identity represents the chat agent's system authority for accessing its own resources. Reassigning RBAC permissions was required so the new agent identity get permissions to access the conversation, vector store and storage resources. At this deployment time, it was a great moment to reassess only the permissions the agent needs for its tool actions. | | :-------: | :------------------------- | -1. Verify the agent deployment is running +1. Verify the agent deployments are running + + *This step lists all agent deployments under the application and confirms their state. If you deployed multiple versions, you should see each one listed.* + + ```powershell + $SUBSCRIPTION_ID="$(az account show --query id -o tsv)" + $FOUNDRY_APP_DEPLOYMENTS_URL="https://management.azure.com/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.CognitiveServices/accounts/aif${BASE_NAME}/projects/${FOUNDRY_PROJECT_NAME}/applications/appchat/agentDeployments?api-version=2025-10-01-preview" + + az rest -u $FOUNDRY_APP_DEPLOYMENTS_URL -m "get" --query "value[].{name:name, deploymentId:properties.deploymentId, state:properties.state, agents:properties.agents}" + ``` + + You should see each deployment with a `Running` state and its associated agent version. For example, if you deployed both v1 and v2, you should see two entries: `agentdeploychatv1` and `agentdeploychatv2`. + +1. Invoke the agent deployment - *This step verify the Foundry AI Agent Service deployment is runnning by invoking the agent application's responses endpoint.* + *This step invokes the Foundry AI Agent Service deployment by calling the agent application's responses endpoint.* ```powershell $AGENT_BASE_URL="$(az deployment group show -g $RESOURCE_GROUP -n 'foundryAppDeploy' --query "properties.outputs.agentApplicationBaseUrl.value" -o tsv)" @@ -289,6 +339,47 @@ The AI agent definition would likely be deployed from your application's pipelin | :information: | The terminal displays the agent application’s response, verifying that the specified agent version is running inside the deployment. | | :--------: | :------------------------- | +1. Route traffic to a specific agent deployment. *Optional.* + + An Agent Application exposes a single stable endpoint that all consumers use. By default, when the `trafficRoutingPolicy` has an empty `deploymentId`, all traffic is routed to the most recently created or updated deployment. If you deployed multiple agent versions, you can explicitly control which deployment receives traffic by updating the application’s `trafficRoutingPolicy` with a specific `deploymentId`. + + | :information: | This is a control plane operation, not a data plane operation. You can perform it from outside the private network — no jump box required. This separation is by design: infrastructure provisioning (Bicep) handles creating applications and deployments, while traffic routing is a Day 2 operational concern managed independently via the management API. | + | :--------: | :------------------------- | + + Use the `deploymentId` values from the verification step above to route traffic to a specific deployment. First, extract the `deploymentId` for the target version: + + ```powershell + # Extract the deploymentId for the target agent version (change the agentVersion value to target a different version) + $TARGET_AGENT_VERSION="1" + $SUBSCRIPTION_ID="$(az account show --query id -o tsv)" + $FOUNDRY_APP_URL="https://management.azure.com/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/${RESOURCE_GROUP}/providers/Microsoft.CognitiveServices/accounts/aif${BASE_NAME}/projects/${FOUNDRY_PROJECT_NAME}/applications/appchat" + + $TARGET_DEPLOYMENT_ID="$(az rest -u "${FOUNDRY_APP_URL}/agentDeployments?api-version=2025-10-01-preview" -m "get" --query "value[?properties.agents[0].agentVersion=='${TARGET_AGENT_VERSION}'].properties.deploymentId | [0]" -o tsv)" + + echo $TARGET_DEPLOYMENT_ID + ``` + + Then, update the application's traffic routing policy to direct all traffic to that deployment: + + ```powershell + az rest -u "${FOUNDRY_APP_URL}?api-version=2025-10-01-preview" -m "PUT" ` + -b "{\`"properties\`":{\`"displayName\`":\`"Example of an Agent Application that exposes a Foundry agent chat interface through a service endpoint\`",\`"agents\`":[{\`"agentName\`":\`"baseline-chatbot-agent\`"}],\`"authorizationPolicy\`":{\`"authorizationScheme\`":\`"Default\`"},\`"trafficRoutingPolicy\`":{\`"protocol\`":\`"FixedRatio\`",\`"rules\`":[{\`"deploymentId\`":\`"${TARGET_DEPLOYMENT_ID}\`",\`"description\`":\`"Route all traffic to v${TARGET_AGENT_VERSION}\`",\`"ruleId\`":\`"default\`",\`"trafficPercentage\`":100}]}}}" ` + --query "properties.trafficRoutingPolicy" + ``` + + After updating, invoke the agent again from the jump box to confirm that the response now comes from the targeted agent version. + + To restore the default behavior (route to the latest deployment), run the same command with an empty `deploymentId`: + + ```powershell + az rest -u "${FOUNDRY_APP_URL}?api-version=2025-10-01-preview" -m "PUT" ` + -b "{\`"properties\`":{\`"displayName\`":\`"Example of an Agent Application that exposes a Foundry agent chat interface through a service endpoint\`",\`"agents\`":[{\`"agentName\`":\`"baseline-chatbot-agent\`"}],\`"authorizationPolicy\`":{\`"authorizationScheme\`":\`"Default\`"},\`"trafficRoutingPolicy\`":{\`"protocol\`":\`"FixedRatio\`",\`"rules\`":[{\`"deploymentId\`":\`"\`",\`"description\`":\`"Default rule routing all traffic\`",\`"ruleId\`":\`"default\`",\`"trafficPercentage\`":100}]}}}" ` + --query "properties.trafficRoutingPolicy" + ``` + + | :warning: | Currently, Foundry Agent Service requires 100% of traffic to be routed to a single deployment. A/B traffic splitting (e.g., 50/50 between v1 and v2) is not yet supported. Additionally, path-based or header-based routing to a specific deployment is not available — the application endpoint does not expose deployment-level sub-paths. The `PUT` method must be used; `PATCH` is not supported for this resource. | + | :-------: | :------------------------- | + ### 3. Test the agent from the Foundry portal in the playground. *Optional.* | :warning: | The new Foundry portal experience does not currently support the end-to-end network isolation used in this architecture. Using this secured architecture, you will only be able to create and call your agents through the SDK or REST API; not interface with them in the Foundry portal. See, [How to use a virtual network with the Foundry Agent Service](https://learn.microsoft.com/azure/ai-foundry/agents/how-to/virtual-networks?view=foundry&preserve-view=true). These intermediate testing instructions will be updated when this experience is supported. | diff --git a/agents/chat-with-bing-v2.json b/agents/chat-with-bing-v2.json new file mode 100644 index 00000000..fdea0df2 --- /dev/null +++ b/agents/chat-with-bing-v2.json @@ -0,0 +1,30 @@ +{ + "name": "baseline-chatbot-agent", + "description": "Example of a Foundry Agent that uses the Bing Search tool to answer questions. Used in the Microsoft Learn AI chat reference architecture. https://learn.microsoft.com/azure/architecture/ai-ml/architecture/baseline-microsoft-foundry-chat", + "definition": { + "kind": "prompt", + "model": "MODEL_CONNECTION_NAME", + "instructions": "You are a helpful Chatbot agent. You'll consult the Bing Search tool to answer questions. Always search the web for information before responding. Keep your responses concise and to the point - aim for 2-3 sentences maximum unless the user asks for more detail.", + "tools": [ + { + "type": "bing_grounding", + "bing_grounding": { + "search_configurations": [ + { + "project_connection_id": "BING_CONNECTION_ID", + "count": 5, + "freshness": "Week" + } + ] + } + } + ], + "tool_resources": {}, + "temperature": 1, + "top_p": 1 + }, + "metadata": { + "createdBy": "Microsoft Learn Baseline Architecture", + "version": "2.0.0" + } +} diff --git a/infra-as-code/bicep/ai-foundry-appdeploy.bicep b/infra-as-code/bicep/ai-foundry-appdeploy.bicep index 83859ef9..89b5a0ab 100644 --- a/infra-as-code/bicep/ai-foundry-appdeploy.bicep +++ b/infra-as-code/bicep/ai-foundry-appdeploy.bicep @@ -103,7 +103,7 @@ resource foundry 'Microsoft.CognitiveServices/accounts@2025-10-01-preview' exist @description('Create agent application deployment in Foundry Agent Service.') resource deploymentApp 'agentDeployments' = { - name: 'agentdeploychat' + name: 'agentdeploychatv${agentVersion}' properties: { agents: [ { @@ -111,7 +111,7 @@ resource foundry 'Microsoft.CognitiveServices/accounts@2025-10-01-preview' exist agentVersion: agentVersion } ] - displayName: 'Example of an agent deployment that runs an Agent Application referencing a specific agent version.' + displayName: 'Agent deployment v${agentVersion} that runs an Agent Application referencing a specific agent version.' deploymentType: 'Managed' // prompt-based agent deployment protocols: [ {