From a92fb9b6506efd9b2a8d1b3ef12907d768cdf6cc Mon Sep 17 00:00:00 2001 From: raghul-velt Date: Fri, 3 Jul 2026 09:13:47 +0530 Subject: [PATCH] docs(agents): add V2 Agents REST API reference and concept docs Adds the complete, up-to-date V2 Agents documentation sourced from snippyly/shared-firebase-function#3517: - api-reference/rest-apis/v2/agents: CRUD, versioning, execution, prompt tools, config resolve, analytics, and groups endpoints - ai/agents: overview and setup concept docs - Replaces the superseded list-agent-executions page with execution/list - Wires all new pages into docs.json navigation and fixes the changelog link Co-authored-by: Cursor --- ai/agents/overview.mdx | 206 ++++++++ ai/agents/setup.mdx | 199 +++++++ .../rest-apis/v2/agents/analytics/get.mdx | 189 +++++++ .../rest-apis/v2/agents/config/resolve.mdx | 126 +++++ api-reference/rest-apis/v2/agents/create.mdx | 499 ++++++++++++++++++ api-reference/rest-apis/v2/agents/delete.mdx | 91 ++++ .../rest-apis/v2/agents/execution/count.mdx | 153 ++++++ .../rest-apis/v2/agents/execution/get.mdx | 265 ++++++++++ .../rest-apis/v2/agents/execution/list.mdx | 177 +++++++ .../rest-apis/v2/agents/execution/run.mdx | 204 +++++++ api-reference/rest-apis/v2/agents/extract.mdx | 143 +++++ api-reference/rest-apis/v2/agents/get.mdx | 247 +++++++++ .../rest-apis/v2/agents/groups/add-agents.mdx | 116 ++++ .../rest-apis/v2/agents/groups/create.mdx | 151 ++++++ .../rest-apis/v2/agents/groups/delete.mdx | 93 ++++ .../rest-apis/v2/agents/groups/get.mdx | 116 ++++ .../rest-apis/v2/agents/groups/list.mdx | 103 ++++ .../v2/agents/groups/remove-agents.mdx | 113 ++++ .../rest-apis/v2/agents/groups/update.mdx | 121 +++++ .../v2/agents/list-agent-executions.mdx | 171 ------ .../rest-apis/v2/agents/prompt/enhance.mdx | 144 +++++ .../rest-apis/v2/agents/prompt/refine.mdx | 132 +++++ .../rest-apis/v2/agents/prompt/validate.mdx | 148 ++++++ api-reference/rest-apis/v2/agents/update.mdx | 133 +++++ .../rest-apis/v2/agents/version/update.mdx | 171 ++++++ .../rest-apis/v2/agents/versions/list.mdx | 130 +++++ .../rest-apis/v2/agents/versions/restore.mdx | 91 ++++ docs.json | 59 ++- release-notes/version-5/sdk-changelog.mdx | 2 +- 29 files changed, 4319 insertions(+), 174 deletions(-) create mode 100644 ai/agents/overview.mdx create mode 100644 ai/agents/setup.mdx create mode 100644 api-reference/rest-apis/v2/agents/analytics/get.mdx create mode 100644 api-reference/rest-apis/v2/agents/config/resolve.mdx create mode 100644 api-reference/rest-apis/v2/agents/create.mdx create mode 100644 api-reference/rest-apis/v2/agents/delete.mdx create mode 100644 api-reference/rest-apis/v2/agents/execution/count.mdx create mode 100644 api-reference/rest-apis/v2/agents/execution/get.mdx create mode 100644 api-reference/rest-apis/v2/agents/execution/list.mdx create mode 100644 api-reference/rest-apis/v2/agents/execution/run.mdx create mode 100644 api-reference/rest-apis/v2/agents/extract.mdx create mode 100644 api-reference/rest-apis/v2/agents/get.mdx create mode 100644 api-reference/rest-apis/v2/agents/groups/add-agents.mdx create mode 100644 api-reference/rest-apis/v2/agents/groups/create.mdx create mode 100644 api-reference/rest-apis/v2/agents/groups/delete.mdx create mode 100644 api-reference/rest-apis/v2/agents/groups/get.mdx create mode 100644 api-reference/rest-apis/v2/agents/groups/list.mdx create mode 100644 api-reference/rest-apis/v2/agents/groups/remove-agents.mdx create mode 100644 api-reference/rest-apis/v2/agents/groups/update.mdx delete mode 100644 api-reference/rest-apis/v2/agents/list-agent-executions.mdx create mode 100644 api-reference/rest-apis/v2/agents/prompt/enhance.mdx create mode 100644 api-reference/rest-apis/v2/agents/prompt/refine.mdx create mode 100644 api-reference/rest-apis/v2/agents/prompt/validate.mdx create mode 100644 api-reference/rest-apis/v2/agents/update.mdx create mode 100644 api-reference/rest-apis/v2/agents/version/update.mdx create mode 100644 api-reference/rest-apis/v2/agents/versions/list.mdx create mode 100644 api-reference/rest-apis/v2/agents/versions/restore.mdx diff --git a/ai/agents/overview.mdx b/ai/agents/overview.mdx new file mode 100644 index 000000000..93073385e --- /dev/null +++ b/ai/agents/overview.mdx @@ -0,0 +1,206 @@ +--- +title: "Agents" +sidebarTitle: "Overview" +description: "Run AI agents that analyze web pages, surface findings, and create comment annotations — without writing the analysis pipeline yourself." +--- + +## What are Agents? + +Agents are AI-driven QA tools that analyze a web page (or set of pages) and surface findings as comment annotations. You configure what an agent looks for; the platform handles context gathering, LLM invocation, post-processing, deduplication, and annotation creation. + +Build any check you want — brand consistency, accessibility, SEO, broken links, spelling, custom domain rules — by writing a prompt and selecting a few configuration options. No need to run your own browser pool, prompt your own model, or wire up annotation creation. + +You define an **agent configuration** once, then call **Run Execution** with a URL. Results are persisted as a versioned execution document plus per-URL findings, and annotations are automatically posted to the document via the Comments API. + +## What you get + +- **Built-in and custom agents.** Use ready-made agents (spell-check, grammar-check, broken-links, PII detection, profanity, sensitive-data, Lighthouse, accessibility, OG-image, and more — see [Built-in agents](#built-in-agents)) or define your own custom agents from a prompt. + +- **Pluggable context gathering.** Stack one or more strategies — page text, screenshots, HTML, CSS, links, accessibility tree, computed styles, robots.txt, sitemap, Lighthouse, and live REST API data — and the engine extracts everything. + +- **Live REST API context.** The `rest-api` strategy fetches your own endpoints (CRM, inventory, feature flags, user profiles) at execution time and injects the responses into the prompt. Auth secrets are encrypted at rest and redacted on read. + +- **MCP tool use.** The `mcp-tools` execution strategy connects to remote [MCP (Model Context Protocol)](https://modelcontextprotocol.io) servers, exposes their tools to the model, and runs a multi-turn tool loop — e.g. verifying on-page code snippets live against a documentation MCP. Provider-agnostic (Claude + Gemini); server auth secrets are encrypted at rest and redacted on read. + +- **Versioned configurations.** Every behavioral edit creates a new version. In-flight executions stay pinned to the version they started on. One-click restore rolls back the most recent change. + +- **Async execution with polling.** `Run Execution` returns an `executionId` immediately and dispatches a Cloud Task. Poll `Get Execution` until `status !== "running"`. + +- **Cross-page execution.** Set `crossPageExecute: true` and the engine crawls the seed URL, processes up to `maxUrlsToProcess` pages, and aggregates findings across the site. + +- **Device emulation.** Set `deviceType: "mobile"` or `"desktop"` per execution to control the Puppeteer viewport/user-agent used for context gathering and pin resolution. + +- **Re-run deduplication.** Re-running an agent against the same document does not pile up duplicate annotations. The active `deletePreviousSuggestions` post-processor clears the prior run's suggestions for each URL before writing the new ones. (The legacy `matchAndMerge` processor is deprecated and ignored.) + +- **Token usage analytics.** Per-agent, per-model, per-month token consumption tracked automatically in Firestore and exposed via the Analytics endpoint. + +- **Agent groups.** Organize related agents into named groups (e.g. "Brand QA") and filter list responses by group. + +## Use cases + + + + A custom agent verifies brand colors, typography, and logo placement across your marketing pages. Re-run on every deploy. + + + + Spell-check, broken-links, and accessibility agents run before a release. Findings appear as comment annotations on the staging document. + + + + PII detection and profanity filter agents flag sensitive content on user-generated pages. Pair with Approval Engine for human review. + + + + Run a brand consistency agent in `crossPageExecute: true` mode against a marketing site. The crawler discovers internal links, then the agent checks each one. + + + + Use the `rest-api` context strategy to pull live business data into the prompt — e.g. validate that on-page pricing matches your billing API. + + + + Trigger an agent execution from an Approval Engine workflow node. The engine handles parking the workflow on the agent's findings. + + + +## How it works + +1. **Define the agent.** Call [Create Agent](/api-reference/rest-apis/v2/agents/create) with a name, instructions, and a configuration block (context gathering strategies, execution strategy, post-processing options). The engine creates version 1 in the agent's versions subcollection. +2. **Run an execution.** Call [Run Execution](/api-reference/rest-apis/v2/agents/execution/run) with `agentId` and `url`. The engine creates an execution document, dispatches a Cloud Task, and returns an `executionId`. +3. **The pipeline runs.** Context gathering → LLM execution → post-processing (guardrails, match-and-merge, annotation creation, analytics) → response shaping. Every phase is bounded by a per-phase timeout. +4. **Poll for results.** Call [Get Execution](/api-reference/rest-apis/v2/agents/execution/get) until `status !== "running"`. Set `includeResults: true` to fetch per-URL findings. +5. **Findings appear as annotations.** When `postProcess.annotations.enabled` is true (default), each finding becomes a comment annotation on the document referenced by `organizationId` / `documentId`. + +## Mental model + +### Agent + +An **agent** is a reusable QA configuration. Each agent has an identity (`name`, `description`, `enabled`) plus a behavioral configuration (instructions, context gathering, execution, post-processing). Identity edits update the root document; behavioral edits create a new version. + +### Execution + +An **execution** is one run of an agent against a seed URL. It pins the agent's `version` at dispatch time, so in-flight runs are immune to mid-run config changes. + +**Lifecycle:** + +``` +running → passed | failed | partial | error | skipped +``` + +### Finding + +A **finding** is one issue the agent surfaced on one URL. Findings carry severity, category, target text, suggestion, HTML selector, and confidence. Findings below the guardrails confidence floor are suppressed unless explicitly disabled. + +### Pipeline phases + +Each agent run executes sequential phases: + +| Phase | What happens | +| ------------------- | -------------------------------------------------------------------------------- | +| `validate` | Zod check on payload + input requirements + variable keys | +| `extract` | All configured context gathering strategies run | +| `execute` | LLM call (or registered service); results normalized to a finding array | +| `postProcess` | Guardrails → match-and-merge → annotation creation → analytics | +| `structureResponse` | Final response assembly with optional response adapter | + +## Execution statuses + +| Status | Meaning | +| ----------- | ------------------------------------------------------------------------------------------- | +| `running` | Execution is in progress | +| `passed` | Completed successfully — all URLs processed cleanly, **no findings** | +| `failed` | Completed with **findings** — all URLs processed cleanly | +| `partial` | Mixed outcome — at least one URL succeeded and at least one URL failed ("passed with some failures") | +| `error` | Every processed URL failed, or a fatal crawl/dispatch failure — nothing usable produced | +| `skipped` | Execution was skipped (e.g. a precondition was not met) | + +**Terminal status precedence:** `error` > `partial` > `failed` > `passed`. A run that produced findings but had a single failing page surfaces as `partial` (not `error`). For `partial`/`error`, per-page failure detail is in `resultsSummary.urlsErrored` and `resultsSummary.erroredUrls`. + +## Built-in agents + +| Agent ID | Description | Internal | +| ----------------------- | ---------------------------------------------- | -------- | +| `spell-check` | Spelling and typo detection | no | +| `grammar-check` | Grammar checking | no | +| `broken-links` | Broken link validation | no | +| `pii-detection` | PII (personal info) detection | no | +| `profanity-filter` | Profanity detection | no | +| `sensitive-data` | Sensitive data detection | no | +| `lighthouse` | Google Lighthouse performance/SEO/a11y audit | no | +| `accessibility-checker` | Accessibility (WCAG) audit | no | +| `og-image-checker` | Open Graph image validation | no | +| `lorem-ipsum` | Placeholder / lorem-ipsum text detection | no | +| `screenshot` | Page screenshot capture | yes | +| `crawler` | Site crawling | yes | +| `docs-code-comparison` | Verifies code snippets against a docs MCP server (`mcp-tools`) | yes | + +Internal agents (`metadata.internal: true`) power the execution pipeline and are excluded from list responses. They can still be fetched individually by id. + +## Context gathering strategies + +Stack one or more strategies in the order you want them to run. + +| Strategy | What it returns | +| ------------------------ | ---------------------------------------------------------------------- | +| `web-page-text` | Visible text extracted via Puppeteer | +| `web-page-screenshot` | Full-page screenshot as image (multimodal input) | +| `web-page-html` | Cleaned HTML content | +| `web-page-css` | Cleaned CSS content from stylesheets | +| `web-page-links` | All hyperlinks on the page, classified internal/external | +| `web-page-accessibility` | Accessibility issues (Pa11y / HTML_CodeSniffer) | +| `computed-styles` | Browser-resolved computed CSS for specific selectors | +| `robots-txt` | Fetched `robots.txt` | +| `sitemap-data` | Discovered and parsed XML sitemaps | +| `lighthouse` | Google Lighthouse audit (performance / a11y / best-practices / SEO) | +| `rest-api` | Customer-configured REST endpoints fetched at runtime (SSRF-guarded, encrypted auth) — injected via `{{restApiData}}` | +| `none` | No context gathering (for service-only or text-only agents) | + +Per-strategy overrides are supplied under `contextGathering.strategyOptions[""]`. See [Create Agent](/api-reference/rest-apis/v2/agents/create) for the `rest-api` endpoint configuration schema. + +## Execution strategies + +| Strategy | Purpose | +| ----------------- | ---------------------------------------------------------------- | +| `ai` | LLM-driven analysis (default) | +| `service` | Delegate to a registered built-in service (`serviceId` required) | +| `service+ai` | Service context gathering + AI analysis (`serviceId` required) | +| `stagehand-agent` | Autonomous browser agent via Stagehand AI | +| `mcp-tools` | Model-driven tool loop against remote MCP server(s) (`instructions` + `execution.mcpServers` required) | + +## Knowledge (Memory-RAG) + +Agents can pull workspace knowledge from Velt Memory into the prompt via `execution.knowledge`: + +| Field | Type | Range | Description | +| --------------- | ------- | ----- | ----------------------------------------------------------------------- | +| `useMemory` | boolean | | When `true`, retrieves knowledge/patterns/activities from Memory and injects them as `{{memoryContext}}` | +| `maxChunks` | number | 1–20 | Max knowledge chunks to retrieve | +| `maxPatterns` | number | 1–20 | Max learned patterns to retrieve | +| `maxActivities` | number | 1–20 | Max recent activities to retrieve | + + + The `knowledge` block is strictly validated — only the four fields above are accepted. The legacy `sourceIds` field has been removed, and `maxChunks` is now capped at 20 (previously 50). Knowledge retrieval never fails an execution: if Memory is unavailable, the run continues in a degraded mode without knowledge context. + + +## Scope + +| Field | Bound to | +| ---------------- | ---------------------------------------------------------- | +| `apiKey` | Workspace-wide (default; agents are per-workspace) | +| `organizationId` | Annotations created on the named organization | +| `documentId` | Annotations created on the named document | + +`organizationId` and `documentId` on Run Execution control where annotations land. They are not required for the agent to execute. + +## Get started + + + + Create an agent, run it against a URL, and read the findings end-to-end. + + + + All endpoints organized into Agents, Execution, Versioning, Prompt Tools, Analytics, and Groups. + + diff --git a/ai/agents/setup.mdx b/ai/agents/setup.mdx new file mode 100644 index 000000000..023f67a96 --- /dev/null +++ b/ai/agents/setup.mdx @@ -0,0 +1,199 @@ +--- +title: "Setup" +description: "Build an agent end to end: create, run, poll, and inspect findings." +--- + +This guide walks through the steps to get an agent running and surfacing findings as comment annotations. Each step links to the full API reference for request and response details. + +## Step 1: (Optional) Refine your prompt + +If you have a one-line QA instruction and you're not sure it's specific enough to ship, run it through the prompt tools first. + +**Check completeness:** [Enhance Prompt](/api-reference/rest-apis/v2/agents/prompt/enhance) returns either "your prompt is sufficient" or a specific clarification request (with suggested options for the user to pick from). + +**Expand into a structured task:** [Validate Prompt](/api-reference/rest-apis/v2/agents/prompt/validate) takes a simple instruction and expands it into a comprehensive QA task with an analysis prompt, response descriptions, demo examples, and tool requirements. + +**Iterate on demo failures:** [Refine Prompt](/api-reference/rest-apis/v2/agents/prompt/refine) takes an existing analysis prompt and a list of demo failures (with feedback) and produces a refined prompt. + +**Resolve optimal config:** [Resolve Config](/api-reference/rest-apis/v2/agents/config/resolve) takes raw + processed instructions and recommends extraction strategies + execution strategy. + +These tools are optional. Skip straight to Step 2 if you already know what your agent should do. + +## Step 2: Create the agent + +Call [Create Agent](/api-reference/rest-apis/v2/agents/create) with a name, instructions, and a configuration block. + +The minimum viable agent is `name` + `instructions` + `contextGathering.strategies`: + +```JSON +{ + "data": { + "name": "Brand Color Checker", + "instructions": "Verify all CTAs use the primary brand color #1A73E8.", + "contextGathering": { + "strategies": ["web-page-text", "web-page-screenshot"] + } + } +} +``` + +The engine creates the agent's main document and writes version 1 to its versions subcollection. The response returns `{ agentId }`. + +For a complete walkthrough of every config block (`execution`, `response`, `postProcess`, `input`, `scope`, `setup`, `metadata`), see the [Create Agent](/api-reference/rest-apis/v2/agents/create) reference. + + + Built-in agents (`spell-check`, `broken-links`, etc.) are pre-registered for every workspace. You don't need to create them — skip straight to Step 3 with one of the [built-in agent IDs](/ai/agents/overview#built-in-agents). + + + + `metadata` is free-form but the keys `type`, `category`, `internal`, and `apiKey` are **server-managed** and rejected if you send them. Use your own keys (e.g. `team`, `owner`). + + +## Step 3: Run an execution + +Call [Run Execution](/api-reference/rest-apis/v2/agents/execution/run) with `agentId` and `url`. Pass `organizationId` and `documentId` if you want findings to land as annotations on a specific document. + +```JSON +{ + "data": { + "agentId": "abc123def456", + "url": "https://example.com/pricing", + "organizationId": "org_001", + "documentId": "doc_001", + "deviceType": "desktop", + "ranBy": { + "userId": "user_123", + "name": "Jane Doe", + "email": "jane@example.com" + } + } +} +``` + +The response returns immediately with `{ executionId }`. The engine creates an execution document, dispatches a Cloud Task, and starts processing asynchronously. + +- **Cross-page execution:** Set `crossPageExecute: true` and the engine crawls the seed URL and processes up to `maxUrlsToProcess` pages (default 50). +- **Device emulation:** Set `deviceType: "mobile"` or `"desktop"` (default `"desktop"`) to control the viewport/user-agent used for context gathering and pin resolution. +- **Workflow trigger:** Set `trigger: "workflow"` and pass `workflowExecutionId` when running an agent as part of an Approval Engine workflow node. + +## Step 4: Poll for results + +Call [Get Execution](/api-reference/rest-apis/v2/agents/execution/get) with the `executionId` returned from Step 3. Poll until `execution.status !== "running"`. + +```JSON +{ + "data": { + "executionId": "exec_1711900000000_abc123def456" + } +} +``` + +Terminal statuses: + +| Status | Meaning | +| --------- | ---------------------------------------------------------------------------- | +| `passed` | Completed successfully — all URLs clean, no findings | +| `failed` | Completed with findings — all URLs processed cleanly | +| `partial` | Some URLs succeeded and some failed (see `resultsSummary.erroredUrls`) | +| `error` | Every URL failed, or a fatal error (see `execution.error.code` and `retryable`) | +| `skipped` | Execution was skipped (precondition unmet) | + +When the execution completes, `resultsSummary` contains aggregate counts: + +```JSON +{ + "totalFindings": 7, + "totalAnnotationsCreated": 5, + "urlsProcessed": 12, + "urlsWithFindings": 5, + "matchResult": { + "created": 5, + "skipped": 2, + "resolved": 1 + } +} +``` + +`matchResult` is populated when `postProcess.matchAndMerge.enabled` is true (the default). It tells you how many findings were new (`created`), already existed as annotations (`skipped`), or were resolved because the previous finding is no longer detected (`resolved`). + +## Step 5: Fetch per-URL findings + +Set `includeResults: true` on Get Execution to fetch the full findings array per URL: + +```JSON +{ + "data": { + "executionId": "exec_1711900000000_abc123def456", + "includeResults": true + } +} +``` + +The response includes a `results` array alongside `execution`. Each entry has the URL, its findings, and per-URL counts: + +```JSON +{ + "results": [ + { + "url": "https://example.com/pricing", + "urlHash": "a1b2c3d4e5f6", + "findings": [ + { + "id": "finding-1", + "title": "CTA uses non-brand color", + "description": "The 'Buy now' button uses #FF5722 instead of #1A73E8", + "severity": "high", + "category": "color", + "targetText": "Buy now", + "suggestion": "Change background-color to #1A73E8", + "htmlSelector": "a.cta-primary", + "isPageLevel": false, + "issueType": "brand-color", + "confidence": 95 + } + ], + "findingCount": 1, + "annotationsCreated": 1, + "processedAt": 1711900050000 + } + ] +} +``` + +Findings are also written to the document as comment annotations (when `postProcess.annotations.enabled` is true and `organizationId` + `documentId` were provided on dispatch). Authors of these annotations are flagged with the system agent user. + +## Listing and counting executions + +- **List executions** for an agent or document with pagination: [List Executions](/api-reference/rest-apis/v2/agents/execution/list). +- **Count executions** (e.g. a live "N running" counter) with [Count Executions](/api-reference/rest-apis/v2/agents/execution/count). + +## Iterating on the agent + +Behavioral edits create a new version. Identity edits do not. + +- **Edit identity (`name`, `description`, `enabled`):** [Update Agent](/api-reference/rest-apis/v2/agents/update). No new version; the root doc is updated in place. +- **Edit behavior (`instructions`, `contextGathering`, `execution`, `postProcess`, `input`, `scope`, `setup`):** [Update Agent Version](/api-reference/rest-apis/v2/agents/version/update). Creates version `N+1` and bumps the agent's `version` pointer. In-flight executions stay pinned to whatever version they started on. +- **List versions:** [List Versions](/api-reference/rest-apis/v2/agents/versions/list). Each version contains the full behavioral snapshot. +- **Roll back the most recent change:** [Restore Version](/api-reference/rest-apis/v2/agents/versions/restore). Single-step undo; call repeatedly to walk backward. Cannot go below version 1. + +## Organizing agents + +Use [Agent Groups](/api-reference/rest-apis/v2/agents/groups/create) to bundle related agents together (e.g. "Brand QA"). Agents stay independent documents — groups just store an `agentIds` array. An agent can belong to multiple groups; deleting an agent removes it from every group automatically. + +When listing agents with [Get Agent(s)](/api-reference/rest-apis/v2/agents/get), pass `groupId` to filter to a single group's members. + +## Track usage + +[Get Agent Analytics](/api-reference/rest-apis/v2/agents/analytics/get) returns per-agent, per-model, per-month token consumption plus execution counts. Filter by `agentId`, `year`, `month`, or `model`. + +## Next steps + + + + All endpoints organized into Agents, Execution, Versioning, Prompt Tools, Analytics, and Groups, with full request and response schemas. + + + + What agents are, the pipeline phases, built-in vs custom agents, and the full enum vocabulary. + + diff --git a/api-reference/rest-apis/v2/agents/analytics/get.mdx b/api-reference/rest-apis/v2/agents/analytics/get.mdx new file mode 100644 index 000000000..a8124ad99 --- /dev/null +++ b/api-reference/rest-apis/v2/agents/analytics/get.mdx @@ -0,0 +1,189 @@ +--- +title: "Get Agent Analytics" +api: "POST https://api.velt.dev/v2/agents/analytics/get" +--- + +Use this API to fetch AI token usage analytics and execution counts. Data is aggregated from per-workspace Firestore usage documents. Filter by `agentId`, `year`, `month`, and/or `model`. + +# Endpoint + +`POST https://api.velt.dev/v2/agents/analytics/get` + +# Headers + + + Your API key. + + + + Your [Auth Token](/security/auth-tokens). + + +# Body + +#### Params + + + + + Min 1 char. Agent ID. Omit for aggregate analytics across all agents. + + + + 4-digit year string (regex `^\d{4}$`), e.g. `"2026"`. + + + + Two-digit month `"01"`–`"12"` (regex `^(0[1-9]|1[0-2])$`). + + + + Filter by model (e.g. `"gemini-3-flash-preview"`). + + + + +## **Example Requests** + +#### 1. Aggregate analytics (all agents) + +```JSON +{ + "data": {} +} +``` + +#### 2. Single agent analytics + +```JSON +{ + "data": { + "agentId": "abc123def456" + } +} +``` + +#### 3. Filtered by year and month + +```JSON +{ + "data": { + "agentId": "abc123def456", + "year": "2026", + "month": "03" + } +} +``` + +#### 4. Filtered by model + +```JSON +{ + "data": { + "model": "gemini-3-flash-preview" + } +} +``` + +# Response + +The `analytics` object maps to the `AgentAnalyticsResponse` interface. Model keys are sanitised as `provider_model` (e.g. `"claude_claude-sonnet-4-6"`). + +#### Success Response + +```JSON +{ + "result": { + "status": "success", + "message": "Analytics fetched successfully", + "data": { + "analytics": { + "tokenUsage": { + "allTime": { + "requestCount": 1250, + "promptTokens": 2500000, + "completionTokens": 750000, + "thoughtsTokens": 50000, + "totalTokens": 3300000 + }, + "yearly": { + "2026": { + "claude_claude-sonnet-4-6": { "requestCount": 350, "promptTokens": 700000, "completionTokens": 210000, "thoughtsTokens": 20000, "totalTokens": 930000 }, + "gemini_gemini-3-flash-preview": { "requestCount": 500, "promptTokens": 1000000, "completionTokens": 300000, "thoughtsTokens": 10000, "totalTokens": 1310000 } + } + }, + "monthly": { + "03": { + "claude_claude-sonnet-4-6": { "requestCount": 120, "promptTokens": 240000, "completionTokens": 72000, "thoughtsTokens": 5000, "totalTokens": 317000 } + } + }, + "byModel": { + "claude_claude-sonnet-4-6": { "requestCount": 500, "promptTokens": 1000000, "completionTokens": 300000, "thoughtsTokens": 50000, "totalTokens": 1350000 }, + "gemini_gemini-3-flash-preview": { "requestCount": 750, "promptTokens": 1500000, "completionTokens": 450000, "thoughtsTokens": 0, "totalTokens": 1950000 } + } + }, + "executionCounts": { + "abc123def456": { "executionCount": 45, "lastExecutedAt": 1711900000000 }, + "spell-check": { "executionCount": 142, "lastExecutedAt": 1711900000000 } + } + } + } + } +} +``` + +**`TokenUsageSummary` fields (repeated in every breakdown):** + +| Field | Type | Description | +| ------------------ | ------ | ------------------------------------------------------- | +| `requestCount` | number | Total number of LLM API requests. | +| `promptTokens` | number | Total input tokens consumed. | +| `completionTokens` | number | Total output tokens generated. | +| `thoughtsTokens` | number | Thinking/reasoning tokens (model-dependent; 0 for Gemini). | +| `totalTokens` | number | Total tokens consumed. | + +**`tokenUsage` breakdown keys:** + +| Key | Type | Description | +| --------- | --------------------------------------------------- | ------------------------------------------------------- | +| `allTime` | `TokenUsageSummary` | Aggregate across all time. | +| `yearly` | `Record>` | Keyed by year string, then by sanitised model key. | +| `monthly` | `Record>` | Keyed by month (`"01"`–`"12"`), then by sanitised model key. | +| `byModel` | `Record` | Cross-year aggregate keyed by sanitised model. | + +**`executionCounts` map:** + +| Field | Type | Description | +| ---------------- | -------------- | ------------------------------------------- | +| `executionCount` | number | Total executions for this agent. | +| `lastExecutedAt` | number \| null | Epoch ms of last execution. Null if never. | + +#### Failure Response + +```JSON +{ + "error": { + "message": "ERROR_MESSAGE", + "status": "INVALID_ARGUMENT" + } +} +``` + +**Errors:** `INVALID_ARGUMENT` (invalid `year` or `month` format). + + +```js +{ + "result": { + "status": "success", + "message": "Analytics fetched successfully", + "data": { + "analytics": { + "tokenUsage": { "allTime": { "requestCount": 0, "promptTokens": 0, "completionTokens": 0, "thoughtsTokens": 0, "totalTokens": 0 } }, + "executionCounts": {} + } + } + } +} +``` + diff --git a/api-reference/rest-apis/v2/agents/config/resolve.mdx b/api-reference/rest-apis/v2/agents/config/resolve.mdx new file mode 100644 index 000000000..e4d252ba9 --- /dev/null +++ b/api-reference/rest-apis/v2/agents/config/resolve.mdx @@ -0,0 +1,126 @@ +--- +title: "Resolve Config" +api: "POST https://api.velt.dev/v2/agents/config/resolve" +--- + +Use this API to resolve optimal extraction strategies and execution strategy for an agent based on its instructions. The engine analyzes the (optional `rawInstructions` plus the processed `instructions`) and returns: + +- A recommended set of `contextGathering.strategies` +- A recommended `executionStrategy` (`ai`, `service`, `service+ai`, or `stagehand-agent`) +- Reasoning for each recommendation + +Use this as a starting point when creating a new agent — the response can be passed straight into [Create Agent](/api-reference/rest-apis/v2/agents/create) under the `contextGathering` and `execution` blocks. + +# Endpoint + +`POST https://api.velt.dev/v2/agents/config/resolve` + +# Headers + + + Your API key. + + + + Your [Auth Token](/security/auth-tokens). + + +# Body + +#### Params + + + + + Min 1 char. The processed/enhanced agent instructions. + + + + Verbatim user prompt. When provided, the resolver also uses it to detect navigation/interaction intent (e.g. "click the menu, then check…") and may recommend `stagehand-agent` execution. + + + + LLM provider override: `"gemini"` or `"claude"`. + + + + +## **Example Requests** + +#### 1. Resolve a static analysis agent + +```JSON +{ + "data": { + "instructions": "Verify all CTAs use the primary brand color #1A73E8 and the heading font 'Inter'." + } +} +``` + +#### 2. Resolve an interactive agent + +```JSON +{ + "data": { + "rawInstructions": "Open the navigation menu, click 'Pricing', and check that the page title contains 'Pricing'.", + "instructions": "Open the navigation menu, click the Pricing link, and verify the resulting page title contains 'Pricing'." + } +} +``` + +# Response + +#### Success Response + +```JSON +{ + "result": { + "status": "success", + "message": "Config resolved successfully", + "data": { + "resolvedConfig": { + "extraction_strategies": ["web-page-text", "web-page-screenshot", "web-page-html"], + "execution_strategy": "ai", + "reasoning": "The instructions describe a static visual + textual check (brand colors and fonts). Screenshots support color verification; HTML supports font detection. AI execution is sufficient — no service delegation needed." + } + } + } +} +``` + +| Field | Type | Description | +| ---------------------------------------------- | -------- | ------------------------------------------------------------------------ | +| `data.resolvedConfig.extraction_strategies` | string[] | Recommended `contextGathering.strategies` values. | +| `data.resolvedConfig.execution_strategy` | string | Recommended `execution.executionStrategy` value. | +| `data.resolvedConfig.reasoning` | string | Explanation for the recommendations. | + +#### Failure Response + +```JSON +{ + "error": { + "message": "ERROR_MESSAGE", + "status": "INVALID_ARGUMENT" + } +} +``` + +**Errors:** `INVALID_ARGUMENT` (missing or empty `instructions`). + + +```js +{ + "result": { + "status": "success", + "message": "Config resolved successfully", + "data": { + "resolvedConfig": { + "extraction_strategies": ["web-page-text"], + "execution_strategy": "ai", + "reasoning": "..." + } + } + } +} +``` + diff --git a/api-reference/rest-apis/v2/agents/create.mdx b/api-reference/rest-apis/v2/agents/create.mdx new file mode 100644 index 000000000..460ac4771 --- /dev/null +++ b/api-reference/rest-apis/v2/agents/create.mdx @@ -0,0 +1,499 @@ +--- +title: "Create Agent" +api: "POST https://api.velt.dev/v2/agents/create" +--- + +Use this API to create a new custom agent configuration. The engine stores the agent in Firestore and writes version 1 to the agent's versions subcollection. The schema uses `.passthrough()`, so any additional behavioral fields are forwarded to the service layer. + +# Endpoint + +`POST https://api.velt.dev/v2/agents/create` + +# Headers + + + Your API key. + + + + Your [Auth Token](/security/auth-tokens). + + +# Body + +#### Params + + + + + Min 1 char. Agent display name. + + + + Human-readable description of what the agent does. + + + + Original user-provided instructions (before enhancement). + + + + Processed/enhanced instructions used by the AI execution. + + + + Whether the agent is active. Defaults to `true`. + + + + Arbitrary client metadata (e.g. `{ "team": "growth", "owner": "jane" }`). The reserved keys `type`, `category`, `internal`, and `apiKey` are **server-managed** and rejected if present ([see below](#server-managed-fields)). + + + + Context gathering configuration. + + | Field | Type | Required | Description | + | ----------------- | -------- | -------- | ---------------------------------------------------------------------------------------- | + | `strategies` | string[] | no | Min 1 if provided. See [context gathering strategies](#context-gathering-strategies). | + | `strategyOptions` | object | no | Per-strategy option overrides, keyed by strategy name ([see below](#context-gathering-strategy-options)). When `strategies` includes `"rest-api"`, `strategyOptions["rest-api"]` is required ([see below](#rest-api-strategy-options)). | + | `aiConfig` | object | no | AI provider/model override for the gather phase ([see below](#ai-config-provider--model)). | + + + + Execution configuration. + + | Field | Type | Required | Description | + | ---------------------- | -------- | -------- | --------------------------------------------------------------------------------------------------------------- | + | `executionStrategy` | string | no | `"ai"` (default), `"service"`, `"service+ai"`, `"stagehand-agent"`, or `"mcp-tools"`. | + | `serviceId` | string | no | Required when `executionStrategy` is `"service"` or `"service+ai"`. Values: `"broken-links"`, `"crawler"`, `"screenshot"`. | + | `aiConfig` | object | no | AI provider/model override for the execute phase ([see below](#ai-config-provider--model)). | + | `mcpServers` | object[] | no | Required (≥1) when `executionStrategy` is `"mcp-tools"`. Remote MCP servers the model may call during the tool loop ([see below](#mcp-servers-mcp-tools-strategy)). | + | `responseDescriptions` | object | no | Per-field descriptions guiding AI response shaping ([see below](#response-descriptions)). | + | `strategyOptions` | object | no | Per-strategy option overrides. See [execution strategy options](#execution-strategy-options). | + | `knowledge` | object | no | Memory-RAG knowledge integration ([see below](#knowledge-memory-rag)). | + + + + Response formatting configuration. + + | Field | Type | Required | Description | + | ------------------ | ------- | -------- | ------------------------------------------------------------------------------------ | + | `useAiFormatting` | boolean | no | Whether to use AI for response formatting. | + | `formattingPrompt` | string | no | Custom prompt for AI formatting. | + | `responseAdapter` | string | no | Adapter name: `"broken-links-response"`, `"crawler-response"`, `"screenshot-response"`. | + + + + Post-processing pipeline configuration. All stages default to **enabled** when omitted or when `enabled` is not explicitly `false`. + + | Field | Type | Required | Description | + | --------------------------- | -------- | -------- | ------------------------------------------------------------------------------------ | + | `guardrails` | object | no | `{ enabled?: boolean }` — confidence-based filtering. Findings below 50% are suppressed. | + | `deletePreviousSuggestions` | object | no | `{ enabled?: boolean }` — per-URL delete-and-recreate: clears the prior run's suggestions for the URL before writing the new ones. This is the active dedup mechanism. | + | `pinResolution` | object | no | `{ enabled?: boolean }` — anchor pin resolution for visual annotations. | + | `annotations` | object | no | `{ enabled?: boolean, strategy?: string }` — strategy: `"findings"`, `"word-level"`, `"page-level"`, `"none"`. | + | `analytics` | object | no | `{ enabled?: boolean }` — analytics event tracking. | + | `matchAndMerge` | object | no | **Deprecated / ignored.** `{ enabled?: boolean }` — replaced by `deletePreviousSuggestions`. Still accepted for backward compatibility but has no effect. | + | `customProcessors` | string[] | no | Custom post-processor names. | + + + + Input declaration configuration. + + | Field | Type | Required | Description | + | -------------------- | -------- | -------- | ---------------------------------------------------------------------------------------- | + | `variableKeys` | string[] | no | Variable names the agent expects in `payload.variables`. | + | `userContextFields` | object[] | no | User-context field definitions (`{ id, title, type, required?, example?, defaultValue? }`). | + | `inputRequirements` | object | no | `{ requires?: string[], requiresOneOf?: string[] }`. | + | `supportedVariables` | string[] | no | Template variables the agent's prompt/config can reference (e.g. `{{restApiData}}`, `{{memoryContext}}`). | + + Each `userContextFields[]` entry: `id` (min 1), `title` (min 1), `type` (`"string"` / `"number"` / `"boolean"`), optional `required` (boolean), optional `example` (string — sample value surfaced in setup UIs), optional `defaultValue` (string / number / boolean). + + + + Scope and targeting configuration. + + | Field | Type | Required | Description | + | --------------------- | -------- | -------- | -------------------------------------------------------------------------- | + | `pageScope` | string[] | no | URL glob patterns the agent runs on. | + | `pageScopeExclude` | string[] | no | URL glob patterns to exclude. | + | `contentTypes` | string[] | no | Content types the agent handles. | + | `crossPage` | object | no | Cross-page execution config (`enabled`, `targetProperty`, `pageDiscovery`, optional `pages[]`, `sourceOfTruthKnowledgeSourceId`). | + + + + Tooling metadata for the setup assistant and checklist-to-agent converter. Not consumed by the agent pipeline at runtime. + + Fields: `setupSamples[]`, `checklistOrigin`. + + + + +#### Server-managed fields + +The following fields are **owned by the server** and are **rejected** with a `400 INVALID_ARGUMENT` if present on the create or [update-version](/api-reference/rest-apis/v2/agents/version/update) payload: + +| Field | Why it is server-managed | +| ------------------- | ---------------------------------------------------------------------------------------------- | +| `managedBy` | Always derived server-side — `"customer"` for every API-created agent. | +| `metadata.type` | Marks built-in / system agents. | +| `metadata.category` | Marks built-in / system agents. | +| `metadata.internal` | `true` hides an agent from **all** list responses — allowing a client to set it would forge a hidden agent. | +| `metadata.apiKey` | The tenant key; merged in server-side. | + +`id`, `version`, `createdAt`, and `updatedAt` are likewise server-generated. Sending a reserved key returns one error per offending key, e.g. `metadata.internal is a server-managed field and cannot be set via the API.` + +#### Context gathering strategies + +| Strategy | What it returns | +| ------------------------ | --------------------------------------------------------------------- | +| `web-page-text` | Visible text extracted via Puppeteer | +| `web-page-screenshot` | Full-page screenshot as image (multimodal input) | +| `web-page-html` | Cleaned HTML content | +| `web-page-css` | Cleaned CSS content from stylesheets | +| `web-page-links` | All hyperlinks, classified internal/external | +| `web-page-accessibility` | Accessibility issues (Pa11y / HTML_CodeSniffer) | +| `computed-styles` | Browser-resolved computed CSS for specific selectors | +| `robots-txt` | Fetched `robots.txt` | +| `sitemap-data` | Discovered and parsed XML sitemaps | +| `lighthouse` | Google Lighthouse audit (performance / a11y / best-practices / SEO) | +| `rest-api` | Customer-configured REST endpoints fetched at runtime ([see below](#rest-api-strategy-options)) | +| `none` | No context gathering (for service-only or text-only agents) | + +#### Context gathering strategy options + +Per-strategy overrides are supplied under `contextGathering.strategyOptions[""]`. All are optional unless noted. + +| Strategy | Option | Type | Description | +| ---------------------- | -------------------- | -------- | ------------------------------------------------------------------------ | +| `web-page-text` | `deviceType` | string | `"mobile"` / `"desktop"` (default `"desktop"`). | +| | `excludeIframes` | boolean | Exclude iframe content. Default `false`. | +| `web-page-screenshot` | `scrollBeforeCapture`| boolean | Scroll the full page before capture. | +| | `maxPageHeightPx` | number | Cap the captured page height. | +| | `waitUntil` | string | Puppeteer wait condition. | +| | `deviceType` | string | `"mobile"` / `"desktop"`. | +| `web-page-html` | `includeHead` | boolean | Include ``. Default `true`. | +| | `includeBody` | boolean | Include ``. Default `true`. | +| | `excludeIframes` | boolean | Default `false`. | +| | `waitUntil` | string | Puppeteer wait condition. | +| `web-page-css` | `includeInlineStyles`| boolean | Include inline `