# Getting Started > **mate** v0.9.0 — Model-Agnostic Testing Engine for AI agents. --- ## What is mate? mate is a multi-agent test environment. It lets you connect multiple AI agents (Microsoft Copilot Studio, Azure AI Foundry, generic HTTP, and others), define test suites, run automated evaluations across all of them, and track quality over time. --- ## Deployment Modes | Mode | Auth | Typical Use | |------|------|-------------| | **Quickstart (GHCR)** | Generic (anonymous) or EntraId | Fastest way to get running — no build needed | | **Local / Docker (build from source)** | Generic (anonymous) | Local development and CI | | **Azure (managed cloud)** | EntraId (recommended) | Teams, staging, production with full automation | | **Production (SaaS)** | Microsoft Entra ID (Azure AD) | Team and enterprise use | The active mode is set via the `Authentication__Scheme` environment variable (`Generic` or `EntraId`). ### Core Secrets Modes Core mode supports two secret backends: - **Database mode (default)** - `AzureInfrastructure__UseKeyVaultForSecrets=false` - Secrets are stored in the local PostgreSQL database. - **Single-vault mode** - `AzureInfrastructure__UseKeyVaultForSecrets=true` - `AzureInfrastructure__KeyVaultUri=https://.vault.azure.net/` - Docker deployments also require: `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID` After changing secret mode values in `infra/local/.env`, rebuild the stack: ```powershell ./debug-container.ps1 -Stop ./debug-container.ps1 -Source build -Rebuild ``` --- ## Accessing mate ### Quickstart package (recommended for new users) The easiest way to run mate — no source code or build tools required. Only [Docker Desktop](https://www.docker.com/products/docker-desktop/) is needed. 1. Download the latest **`mate-quickstart-.zip`** from the [GitHub Releases page](https://github.com/holgerimbery/mate/releases/latest). 2. Unzip to a local folder. 3. Copy the environment template: **Windows (PowerShell)** ```powershell copy .env.template .env ``` **macOS / Linux** ```bash cp .env.template .env ``` 4. Open `.env` in a text editor and set `Authentication__Scheme`: - `Generic` — no login, full SuperAdmin access (safe on a trusted network or VPN only — **never** expose this to the internet) - `EntraId` — Microsoft Entra ID SSO; also fill in `AzureAd__TenantId`, `AzureAd__ClientId`, `AzureAd__ClientSecret`, `AzureAd__RedirectUri` > **Recommendation:** If you are deploying for a team or production use, configure **Microsoft Entra ID authentication from the start**. Switching from `Generic` to `EntraId` later works seamlessly (mate maps the Entra tenant GUID to your existing data automatically), but setting it up upfront avoids sharing an unauthenticated endpoint even temporarily. See [Authentication — Entra ID](User-Authentication-EntraId) for the full setup guide. 5. Pull the images and start the stack: ```bash docker compose pull docker compose up -d ``` This starts four containers: **PostgreSQL 17** (database), **Azurite** (blob storage emulator), **mate-webui**, and **mate-worker**. No extra flags needed. 6. Open **** in your browser. > **Tip:** To pin a specific version instead of `latest`, open `docker-compose.yml` and replace `:latest` with the version tag, e.g. `:0.9.0`. ### Local (build from source) Requires [Docker Desktop](https://www.docker.com/products/docker-desktop/) and the [.NET 9 SDK](https://dotnet.microsoft.com/download). **Windows (PowerShell)** ```powershell git clone https://github.com/holgerimbery/mate.git cd mate copy infra\local\.env.template infra\local\.env # Edit infra\local\.env if needed — defaults work for local dev cd infra\local docker compose up --build ``` **macOS / Linux** ```bash git clone https://github.com/holgerimbery/mate.git cd mate cp infra/local/.env.template infra/local/.env # Edit infra/local/.env if needed — defaults work for local dev cd infra/local docker compose up --build ``` Open ****. No login is required — you are the default dev tenant. ### Azure deployment (recommended for teams and production) Deploy mate to Microsoft Azure with fully managed infrastructure, automated secret handling, and production-grade monitoring. **Requirements:** - Azure subscription with Owner or Contributor role - [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) - [PowerShell 7+](https://github.com/PowerShell/PowerShell/releases) - Authenticated Azure session (`az login`) **Quick deployment (5 minutes):** 1. Clone the repository: ```bash git clone https://github.com/holgerimbery/mate.git cd mate ``` 2. Run canonical Azure scripts: ```powershell pwsh ./infra/azure/scripts/check-prerequisites.ps1 # Validate tools pwsh ./infra/azure/scripts/setup-env.ps1 # Configure Azure + app settings pwsh ./infra/azure/scripts/deploy-whatif.ps1 # Preview what will be created (recommended) pwsh ./infra/azure/scripts/deploy.ps1 # Deploy infrastructure pwsh ./infra/azure/scripts/setup-keyvault-secrets.ps1 # Configure secrets & RBAC ``` 3. After deployment, your app URL is printed to console. ### Post-deployment health check (Azure) Run this quick verification after every deployment to confirm runtime health. 1. Verify both Container Apps are provisioned and running: ```powershell az containerapp show --resource-group --name -webui --query "{provisioningState:properties.provisioningState,runningStatus:properties.runningStatus,latestReadyRevision:properties.latestReadyRevisionName}" -o json az containerapp show --resource-group --name -worker --query "{provisioningState:properties.provisioningState,runningStatus:properties.runningStatus,latestReadyRevision:properties.latestReadyRevisionName}" -o json ``` Expected: `provisioningState = Succeeded` and `runningStatus = Running`. 2. Verify active revision health: ```powershell az containerapp revision list --resource-group --name -webui --query "[].{name:name,active:properties.active,health:properties.healthState,running:properties.runningState}" -o table az containerapp revision list --resource-group --name -worker --query "[].{name:name,active:properties.active,health:properties.healthState,running:properties.runningState}" -o table ``` Expected: active revision health is `Healthy`. 3. Verify Key Vault runtime secrets exist: ```powershell az keyvault secret list --vault-name -kv --query "[].name" -o tsv ``` Expected secrets: - `azuread-client-secret` - `blob-connection-string` - `postgres-admin-password` - `postgres-connection-string` - `servicebus-connection-string` 4. If WebUI is not healthy, inspect logs: ```powershell az containerapp logs show --resource-group --name -webui --tail 120 ``` Most common startup failure: PostgreSQL connectivity or firewall configuration. 5. Open the printed WebUI URL and sign in. Expected: successful sign-in and home page loads without restart loops. > **What gets deployed:** Azure Container Apps (WebUI + Worker), PostgreSQL Flexible Server v17, Blob Storage, Key Vault, Service Bus, and Application Insights. See [Developer-Quickstart-Azure.md](Developer-Quickstart-Azure.md) for full technical details, troubleshooting, cost estimates, and profiles (xs/s/m/l). **Updating to a new version:** After a new release is available, quickly update your Azure deployment without redeploying the entire infrastructure: ```powershell pwsh ./infra/azure/scripts/update-container-images.ps1 ``` Or update to a specific version: ```powershell pwsh ./infra/azure/scripts/update-container-images.ps1 -ImageTag '' ``` This performs a **zero-downtime rolling update**. The script waits for completion (usually in 5–10 minutes). Runtime secret wiring is managed by Bicep + Key Vault references. See [Developer-Quickstart-Azure.md](Developer-Quickstart-Azure.md) for monitoring commands, preview mode (`-WhatIf`), and rollback options. > **Custom domain note (admin task):** If your deployment uses a custom hostname and it is not managed in IaC, rebind it after `update-container-images.ps1`. > > ```powershell > pwsh ./infra/azure/scripts/bind-custom-domain.ps1 -DomainName '' > ``` > > Example values: `app.example.com`, `portal.contoso.com` ### Production Navigate to the configured URL. Click **Sign In** and authenticate with your Microsoft Entra ID (Azure AD) account. --- ## First Steps Once logged in, the **Home** page gives you an overview: - **KPI bar** — total agents, test suites, runs, and pass rate at a glance. - **Module Status** — health of all connected modules (agent connectors, judge modules). - **Quick Run** — run any test suite immediately from the Home page. - **Getting Started steps** — guided checklist for new setups. The recommended first-time flow: ``` 1. Connect an agent → Agents page (or the Wizard) 2. Create a test suite → Test Suites page 3. Add test cases → inside the test suite 4. (Optional) Upload docs → Documents page 5. (Optional) Add rubrics → Rubrics page 6. Run the suite → Test Suites → Run button 7. View results → Dashboard ``` --- ## Navigation The left sidebar contains all sections: | Icon | Section | Path | |------|---------|------| | Home | Overview, KPIs, Quick Run | `/` | | Wizard | Agent onboarding wizard | `/wizard` | | Test Suites | Suite + test case management | `/test-suites` | | Documents | Document library | `/documents` | | Agents | Agent CRUD | `/agents` | | Dashboard | Pass-rate charts, run history | `/dashboard` | | Rubrics | Rubric sets + criteria | `/rubrics` | | Help | API reference, OpenAPI explorer | `/help` | | Audit Log *(TenantAdmin / SuperAdmin)* | Event audit trail | `/audit-log` | | API Keys *(TenantAdmin / SuperAdmin)* | Manage API keys | `/api-keys` | | Settings *(TenantAdmin / SuperAdmin)* | Configuration | `/settings` | | Discover | Entra ID / Power Platform discovery | `/discover` | The sidebar can be collapsed by clicking the toggle at the bottom. --- ## Dark Mode Click the sun/moon icon in the sidebar footer to toggle dark mode. The preference is saved in your browser's local storage. You can also change it in **Settings → Appearance**. --- ## Next Steps - [Agents](User-Agents) — connect your first AI agent - [Test Suites & Test Cases](User-Test-Suites) — build and run your first test - [Settings](User-Settings) — configure the AI judge and evaluation modules