From a91de7af6b0a1ac8fba6ed75e06d242c59494a99 Mon Sep 17 00:00:00 2001 From: OpenClaw Scribe Date: Mon, 3 Aug 2026 06:13:33 +0000 Subject: [PATCH 1/2] content: add AI agent structured output guide --- .seo/briefs/ai-agent-structured-output.md | 81 ++++ .seo/config.json | 4 +- .seo/content-ledger.md | 4 +- .seo/keyword-research.json | 21 +- .seo/link-inventory.md | 6 + CHANGELOG.md | 4 + .../pages/content/blog/ai-agent-data-entry.md | 4 + .../blog/ai-agent-structured-output.md | 346 ++++++++++++++++++ apps/pages/content/docs/design-schema.md | 4 + 9 files changed, 468 insertions(+), 6 deletions(-) create mode 100644 .seo/briefs/ai-agent-structured-output.md create mode 100644 apps/pages/content/blog/ai-agent-structured-output.md diff --git a/.seo/briefs/ai-agent-structured-output.md b/.seo/briefs/ai-agent-structured-output.md new file mode 100644 index 00000000..78116df8 --- /dev/null +++ b/.seo/briefs/ai-agent-structured-output.md @@ -0,0 +1,81 @@ +# SEO Brief: AI Agent Structured Output + +## Selection + +- **Title:** AI Agent Structured Output: Validate Before You Store +- **Slug:** `/blog/ai-agent-structured-output` +- **Primary keyword:** `AI agent structured output` +- **Measured demand:** unmeasured; DataForSEO credentials were unavailable in the cron runtime +- **Intent:** implementation / informational +- **Type:** how-to and operational decision guide +- **Product-led reason:** Structured output is a natural upstream boundary for Rowset datasets. + The article helps builders decide how a model result becomes a safe proposal and durable row, + while clearly stating that Rowset is the storage layer rather than the model-output validator. + +## SERP teardown + +The live SERP is led by current provider and framework documentation explaining how to request +schema-constrained JSON. Common coverage includes JSON Schema, Pydantic or Zod, provider-side +validation, type-safe parsing, and agent/tool integration. The gap is the operational boundary +after parsing: semantic verification, stable destination identity, staging, idempotent writes, +and read-back. + +## Information gain + +The article introduces the **shape -> meaning -> write contract**. It separates schema and +transport validity, evidence and policy validity, and retry-safe persistence. The framework +connects provider structured-output features to an inspectable two-dataset staging pattern without +pretending that schema validity proves factual correctness. + +## Entity and question map + +- structured output, JSON mode, and JSON Schema +- Pydantic and Zod +- required properties, enum constraints, and additional properties +- syntactic validity versus semantic validity +- stable business keys and idempotency keys +- staging proposals, approval, write verification, and read-back +- output-contract versioning and provider schema subsets +- PAA: What is structured output for AI agents? Is JSON mode the same? Does structured output + prevent hallucinations? Should agents write directly to a database? + +## Claim ledger + +| ID | Claim | Primary source | Independent check / locator | Status | +|---|---|---|---|---| +| output-01 | The Claude Agent SDK accepts JSON Schema, Zod, or Pydantic output definitions and returns validated structured output. | https://code.claude.com/docs/en/agent-sdk/structured-outputs | Live extraction checked 2026-08-03 | verified | +| output-02 | Gemini structured outputs support data extraction, classification, and agentic workflows through JSON Schema. | https://ai.google.dev/gemini-api/docs/structured-output | https://ai.google.dev/gemini-api/docs/generate-content/structured-output | verified | +| output-03 | Schema-constrained JSON does not guarantee that output values are semantically correct. | https://ai.google.dev/gemini-api/docs/generate-content/structured-output | Current Gemini guide requires application validation | verified | +| output-04 | JSON Schema object validation can define properties, required fields, and additional-property handling. | https://json-schema.org/understanding-json-schema/reference/object | https://json-schema.org/learn/getting-started-step-by-step | verified | +| output-05 | Rowset supports private datasets with explicit indexes, semantic column types, instructions, MCP, and REST access. | https://rowset.lvtd.dev/docs/design-schema | Repo `design-schema.md`, `connect-mcp.md`, and `dataset-api.md` | verified | +| output-06 | Rowset is a mutable row backend, not a provider-side structured-output generator or immutable compliance store. | https://rowset.lvtd.dev/docs/datasets | Repo `AGENTS.md`, `.seo/brand.md`, and audit-trail guide | verified | + +## Counter-evidence and limits + +- Provider implementations support different JSON Schema subsets and limits. The application + validator remains authoritative. +- Structured outputs improve parseability but do not prevent unsupported claims or wrong IDs. +- Staging is not necessary for every low-risk deterministic operation; consequence and ambiguity + determine the control depth. +- Rowset datasets are mutable and are not WORM or tamper-evident audit storage. + +## Internal-link plan + +- `/docs/design-schema` — durable dataset schema +- `/docs/connect-mcp` — hosted MCP connection +- `/docs/dataset-api` — application write path +- `/blog/ai-agent-data-entry` — source-to-destination workflow +- `/blog/idempotent-ai-agent-updates` — uncertain-write recovery +- `/pricing` — product next step + +Inbound links will be added from `/docs/design-schema` and +`/blog/ai-agent-data-entry`. + +## Side checks + +- **AI SEO:** Direct definition and answer first; self-contained three-gate framework; provider + and standards sources; current review date; question-shaped headings and FAQ; BlogPosting schema + emitted by the existing renderer. +- **Product-led SEO:** Solves a real builder job at the boundary between agent generation and + durable data; maps to Rowset schema, index, MCP, and REST surfaces; states product limits; gives + a private two-dataset staging pattern and a natural product next step. diff --git a/.seo/config.json b/.seo/config.json index ba70ff63..7dd8590f 100644 --- a/.seo/config.json +++ b/.seo/config.json @@ -37,10 +37,10 @@ "content_ledger_dataset_key": "9ae7d0d3-1138-43a0-b3b8-aa05fac12a96", "research_briefs_dataset_key": "3a004cfa-0a7e-4c3a-a517-caca902f38d6", "schema_version": 1, - "last_sync_at": "2026-08-02" + "last_sync_at": "2026-08-03" }, "created_at": "2026-07-04", - "updated_at": "2026-08-02", + "updated_at": "2026-08-03", "notes": [ "Initialized from latest origin/main after fast-forwarding from c11f1de to 2d9389b.", "Refreshed from current origin/main b93e5d6 with GSC, Plausible, PostHog, DataForSEO, Exa, Firecrawl, Jina, and live HTTP checks on 2026-07-15.", diff --git a/.seo/content-ledger.md b/.seo/content-ledger.md index 7f088c0c..b82dec6f 100644 --- a/.seo/content-ledger.md +++ b/.seo/content-ledger.md @@ -37,6 +37,7 @@ | 2026-07-30 | AI Agent for Data Entry: A Safe Structured Workflow | how-to / operational guide | `/blog/ai-agent-data-entry` | AI agent for data entry | 20 | n/a | agent-managed datasets, data cleaning, index-column guide, idempotency, schema design, MCP, Dataset API, pricing | #374 | | 2026-08-01 | AI Data Collection: A Reviewable Agent Workflow | how-to / operational guide | `/blog/ai-data-collection` | AI data collection | 210 | 2 | agent-managed datasets, data entry, source registry, checkpoints, provenance, schema design, MCP, Dataset API, pricing | #391 | | 2026-08-02 | MCP OAuth vs API Keys: Choose the Right Auth | comparison / decision guide | `/blog/mcp-oauth-vs-api-keys` | MCP OAuth | 390 | 9 | MCP setup, agent access, MCP vs REST, safe sharing, pricing | #392 | +| 2026-08-03 | AI Agent Structured Output: Validate Before You Store | how-to / operational decision guide | `/blog/ai-agent-structured-output` | AI agent structured output | unmeasured | n/a | schema design, data entry, idempotent updates, MCP, Dataset API, pricing | #TBD | --- @@ -69,6 +70,7 @@ | 23 | MCP OAuth vs API keys | comparison / decision guide | MCP OAuth | 390 | 9 | navigational / implementation | shipped (20) | Selected 2026-08-02 after live DataForSEO and SERP research. The product-led angle is a five-question delegation test that distinguishes third-party user delegation from a provisioned key for a trusted agent. | | 24 | MCP authentication | pillar / implementation guide | MCP authentication | 720 | 21 | navigational | 17 | Defer: measured demand is strong, but KD exceeds the conservative authority band and the SERP is dominated by protocol and platform documentation. Revisit after this narrower OAuth decision guide earns traction. | | 25 | MCP security best practices | security guide | MCP security best practices | 110 | 27 | commercial | 14 | Defer: KD is above the current authority band and the broad topic would overlap existing safe-sharing, approval, audit, and authentication content. | +| 26 | AI agent structured output | how-to / operational decision guide | AI agent structured output | unmeasured | n/a | implementation / informational | shipped (19) | Selected 2026-08-03 after live SERP research. Product-led angle: a shape -> meaning -> write contract that connects provider schema output to staged, retry-safe Rowset rows without treating valid JSON as proof of correctness. | --- @@ -77,7 +79,7 @@ | Cluster / theme | Pieces shipped | Gaps still open | |---|---|---| | Agent-managed datasets | `/blog/agent-managed-datasets`, `/blog/choose-index-column-agent-rows`, `/blog/structure-dataset-instructions-ai-agents`, `/blog/rowset-id-vs-business-keys`, `/blog/relationship-modeling-agent-datasets`, `/blog/ai-agent-memory-vs-state`, `/blog/idempotent-ai-agent-updates`, `/blog/share-ai-agent-data-safely`, `/blog/human-in-the-loop-ai-agents`, `/blog/ai-agent-audit-trail`, `/blog/database-for-ai-agents`, `/blog/ai-data-cleaning-agent`, `/blog/ai-agent-crm`, `/blog/ai-agent-task-management`, `/blog/ai-agent-inventory-management`, `/blog/ai-customer-feedback-analysis`, `/blog/ai-ready-data`, `/blog/ai-agent-data-entry`, `/blog/ai-data-collection` | Generated-index migration patterns | -| MCP and Dataset API | `/blog/mcp-vs-rest-ai-agents`, `/blog/connect-ai-agent-to-dataset-api`, `/blog/relationship-modeling-agent-datasets`, `/blog/ai-agent-inventory-management`, `/blog/ai-agent-data-entry`, `/blog/mcp-oauth-vs-api-keys` | More REST/MCP setup examples with concrete datasets | +| MCP and Dataset API | `/blog/mcp-vs-rest-ai-agents`, `/blog/connect-ai-agent-to-dataset-api`, `/blog/relationship-modeling-agent-datasets`, `/blog/ai-agent-inventory-management`, `/blog/ai-agent-data-entry`, `/blog/mcp-oauth-vs-api-keys`, `/blog/ai-agent-structured-output` | More REST/MCP setup examples with concrete datasets | | Spreadsheet/database alternatives | `/blog/airtable-alternatives`, `/blog/google-sheets-alternatives`, `/blog/baserow-alternatives`, `/blog/nocodb-alternatives`, `/blog/spreadsheet-database-for-ai-agents`, `/vs/airtable`, `/vs/google-sheets` | Migration examples for specific human-to-agent workflows | --- diff --git a/.seo/keyword-research.json b/.seo/keyword-research.json index 74086072..6895c491 100644 --- a/.seo/keyword-research.json +++ b/.seo/keyword-research.json @@ -2,7 +2,7 @@ "product": "Rowset", "domain": "https://rowset.lvtd.dev", "created_at": "2026-07-04", - "updated_at": "2026-08-02", + "updated_at": "2026-08-03", "mode": "measured_reaudit", "source": "blended", "sources": [ @@ -17,7 +17,7 @@ "repo", "production_site" ], - "as_of": "2026-08-02", + "as_of": "2026-08-03", "confidence": "measured", "notes": [ "GSC remains sparse: 8 impressions and 0 clicks across three query/page rows in the last 90 days, with no meaningful striking-distance opportunity.", @@ -27,7 +27,8 @@ "DataForSEO surfaced mcp database as the strongest newly measured cluster: 480 US searches/month, KD 5, and $14.12 CPC.", "The live route audit found 17 stale URLs in the SEO inventory after the docs/use-case migration.", "A 2026-08-01 DataForSEO refresh selected ai data collection at 210 US searches/month, KD 2, and $17.20 CPC; adjacent analysis and broad operations terms were deferred for weaker product fit.", - "A 2026-08-02 DataForSEO refresh selected mcp oauth at 390 US searches/month, KD 9, and $38.48 CPC. The live SERP is led by official protocol documentation and implementation guides; the Rowset piece uses a distinct OAuth-versus-provisioned-key delegation test." + "A 2026-08-02 DataForSEO refresh selected mcp oauth at 390 US searches/month, KD 9, and $38.48 CPC. The live SERP is led by official protocol documentation and implementation guides; the Rowset piece uses a distinct OAuth-versus-provisioned-key delegation test.", + "A 2026-08-03 live SERP refresh selected AI agent structured output as an unmeasured but strongly product-aligned implementation topic. Provider docs explain schema-constrained responses; the Rowset piece adds the shape, meaning, and retry-safe write boundary." ], "tool_evidence": [ { @@ -236,6 +237,14 @@ {"keyword": "database for ai agents", "source": "dataforseo", "search_volume": 10, "keyword_difficulty": 7, "cpc": 19.47, "intent": "commercial", "as_of": "2026-07-15"} ], "opportunities": [ + { + "cluster": "AI agent structured output", + "target_url": "/blog/ai-agent-structured-output", + "status": "shipped", + "priority": 13, + "signals": {"primary_volume": null, "primary_kd": null, "primary_cpc": null}, + "notes": "Selected 2026-08-03 after live SERP research. Provider and framework docs dominate implementation coverage; the Rowset guide adds a product-led shape -> meaning -> write contract for staged, retry-safe durable rows." + }, { "cluster": "mcp oauth", "target_url": "/blog/mcp-oauth-vs-api-keys", @@ -334,6 +343,12 @@ } ], "serp_snapshots": { + "AI agent structured output": { + "source": "live web search + primary-source extraction", + "as_of": "2026-08-03", + "top_domains": ["code.claude.com", "ai.google.dev", "json-schema.org", "tanstack.com", "docs.ag2.ai", "learn.microsoft.com"], + "notes": "The SERP is documentation-heavy and explains schema-constrained generation. The Rowset post differentiates by separating format validity, semantic and policy validity, staging, retry-safe writes, and destination read-back." + }, "ai data collection": { "source": "dataforseo + live page extraction", "as_of": "2026-08-01", diff --git a/.seo/link-inventory.md b/.seo/link-inventory.md index b7cf5391..0981846f 100644 --- a/.seo/link-inventory.md +++ b/.seo/link-inventory.md @@ -92,6 +92,7 @@ | `/blog/ai-agent-data-entry` | https://rowset.lvtd.dev/blog/ai-agent-data-entry | AI Agent for Data Entry: A Safe Structured Workflow | AI agent for data entry, safe AI data-entry workflow, source entry envelope destination contract, reviewable automated data entry, structured data entry for agents | | `/blog/ai-data-collection` | https://rowset.lvtd.dev/blog/ai-data-collection | AI Data Collection: A Reviewable Agent Workflow | AI data collection, AI data collection workflow, collection control plane, source registry and capture runs, reviewable agent-collected records | | `/blog/mcp-oauth-vs-api-keys` | https://rowset.lvtd.dev/blog/mcp-oauth-vs-api-keys | MCP OAuth vs API Keys: Choose the Right Auth | MCP OAuth vs API keys, MCP authorization decision guide, provisioned API key for agents, delegated MCP access, agent authentication delegation test | +| `/blog/ai-agent-structured-output` | https://rowset.lvtd.dev/blog/ai-agent-structured-output | AI Agent Structured Output: Validate Before You Store | AI agent structured output, structured output validation for agents, shape meaning write contract, JSON Schema to durable rows, validated agent output workflow | ## Shipped SEO Sprint Pages @@ -230,3 +231,8 @@ These route families remain retired and must not be selected as internal-link ta - provisioned API key for trusted agents - delegated MCP access - agent authentication delegation test +- AI agent structured output +- structured output validation for agents +- shape meaning write contract +- JSON Schema to durable rows +- validated agent output workflow diff --git a/CHANGELOG.md b/CHANGELOG.md index 160939f8..c236dee0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,10 @@ deployment or release cut date. ## 2026-08-03 +### Added +- Added an AI agent structured-output guide covering JSON Schema, semantic validation, staging, + stable identity, retry-safe writes, and destination read-back. + ### Changed - Dataset tables now expose row search, semantic filters, sorting, active query summaries, sticky row identity, explicit row links, and browser-local column diff --git a/apps/pages/content/blog/ai-agent-data-entry.md b/apps/pages/content/blog/ai-agent-data-entry.md index db9ab167..77bf75f4 100644 --- a/apps/pages/content/blog/ai-agent-data-entry.md +++ b/apps/pages/content/blog/ai-agent-data-entry.md @@ -224,6 +224,10 @@ patterns. Its `format` keyword may be annotation-only depending on the validator an `email` or `date` format is enforced unless your selected validator enables that behavior ([JSON Schema reference, checked July 2026](https://json-schema.org/understanding-json-schema/reference/type)). +When the model provider returns schema-constrained JSON, use the [AI agent structured-output +guide](/blog/ai-agent-structured-output) to keep format validation, semantic checks, and the final +database write as separate gates. + Keep semantic or contextual checks separate. A model may help decide whether "Acme Co." and "Acme Incorporated" refer to the same organization, but that judgment should produce evidence and a review state. It should not bypass the exact-key lookup. diff --git a/apps/pages/content/blog/ai-agent-structured-output.md b/apps/pages/content/blog/ai-agent-structured-output.md new file mode 100644 index 00000000..503e2321 --- /dev/null +++ b/apps/pages/content/blog/ai-agent-structured-output.md @@ -0,0 +1,346 @@ +--- +title: "AI Agent Structured Output: Validate Before You Store" +description: "Turn AI agent structured output into durable rows with JSON Schema, business validation, stable identity, staging, and verified writes." +published_at: 2026-08-03 +updated_at: 2026-08-03 +author: Rasul Kireev +keywords: + - AI agent structured output + - structured output for AI agents + - JSON Schema AI agent + - agent output validation +topics: + - agent workflows + - structured data + - dataset operations +canonical_url: https://rowset.lvtd.dev/blog/ai-agent-structured-output +image: /static/vendors/images/logo.png +image_alt: Rowset logo +robots: index, follow +--- + +AI agent structured output is a model response constrained to a defined machine-readable shape, +usually JSON Schema. It makes an agent's result easier to parse, but it does not prove that the +values are true, authorized, unique, or safe to write. Treat structured output as a typed proposal +that must pass application checks before it becomes a durable row. + +Use three separate gates: + +1. **Shape gate:** Does the result match the declared output schema? +2. **Meaning gate:** Do the values satisfy source, identity, policy, and business rules? +3. **Write gate:** Can the system create or update the destination safely and verify the result? + +This guide calls that sequence the **shape -> meaning -> write contract**. The separation matters +because a perfectly valid JSON object can still name the wrong customer, repeat an existing +record, use stale evidence, or request an operation the agent is not allowed to perform. + +## In this guide + +- [What structured output does](#what-structured-output-does) +- [Why schema-valid is not business-valid](#schema-valid-is-not-business-valid) +- [Design the output contract](#design-the-output-contract) +- [Run the shape gate](#run-the-shape-gate) +- [Run the meaning gate](#run-the-meaning-gate) +- [Stage the proposal](#stage-the-proposal) +- [Run the write gate](#run-the-write-gate) +- [Handle retries and schema changes](#handle-retries-and-schema-changes) +- [Use Rowset as the durable layer](#use-rowset) +- [Structured output FAQ](#structured-output-faq) + + +## What does structured output do for an AI agent? + +Structured output constrains the agent's final response to fields and types your program can +inspect. Instead of parsing prose such as "the customer is likely at risk," an application can +request an object with `customer_id`, `risk_level`, `reason`, and `evidence_refs`. + +Current provider implementations expose this through JSON Schema or language-native schema tools. +The [Claude Agent SDK structured-output guide, checked August +2026](https://code.claude.com/docs/en/agent-sdk/structured-outputs) accepts JSON Schema, Zod, or +Pydantic definitions and returns a validated `structured_output` value after the agent workflow. +The [Gemini structured-output guide, checked August +2026](https://ai.google.dev/gemini-api/docs/structured-output) describes JSON Schema output for +data extraction, classification, and agentic workflows. + +Structured output is useful at boundaries where software needs to decide what happens next: + +| Boundary | Example output | Next deterministic action | +|---|---|---| +| Research agent -> review queue | claims, source URLs, confidence | Reject missing evidence | +| Support agent -> CRM proposal | customer ID, category, summary | Look up the customer before staging | +| Extraction agent -> catalog | SKU, price, currency, source version | Validate types and compare the current row | +| Triage agent -> task board | issue ID, priority, owner, reason | Enforce allowed states and ownership | +| Agent -> another agent | task ID, result status, artifact reference | Verify the referenced artifact exists | + +The schema removes ambiguity about the message format. It does not remove ambiguity about the +world the message describes. + + +## Why is schema-valid output not necessarily correct? + +JSON Schema validates structure. It can require fields, constrain types, limit strings to an +enumeration, and reject unexpected properties. The [JSON Schema object reference, checked August +2026](https://json-schema.org/understanding-json-schema/reference/object) documents `properties`, +`required`, and `additionalProperties` for those checks. + +It cannot establish that a claim matches its source, that an identifier belongs to the signed-in +account, or that an update is still appropriate. Google's Gemini documentation states the limit +plainly: schema-constrained output can guarantee syntactically correct JSON without guaranteeing +that its values are semantically correct. + +Consider this schema-valid result: + +```json +{ + "customer_id": "CUS-184", + "status": "cancelled", + "reason": "Customer requested cancellation", + "source_ref": "ticket:921" +} +``` + +The object may match every declared type and enum while still being unsafe to apply. Ticket 921 +could belong to another customer. The ticket may say "do not cancel." The customer may already +have renewed. The agent may have read an untrusted instruction embedded in the ticket. A valid +shape is evidence that parsing can proceed, not permission to mutate business state. + + +## 1. Design the output contract around one decision + +Start with the application decision the object will support. Keep the schema smaller than the +model's full reasoning process. A reviewable customer-status proposal might use: + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "type": "object", + "additionalProperties": false, + "properties": { + "proposal_id": { + "type": "string", + "description": "Stable retry key for this exact proposed change" + }, + "customer_id": { + "type": "string", + "description": "Existing customer key from the approved source" + }, + "proposed_status": { + "type": "string", + "enum": ["active", "at_risk", "cancelled"] + }, + "reason": {"type": "string", "minLength": 1, "maxLength": 500}, + "source_refs": { + "type": "array", + "minItems": 1, + "items": {"type": "string"} + }, + "source_version": {"type": "string"}, + "requires_review": {"type": "boolean"} + }, + "required": [ + "proposal_id", + "customer_id", + "proposed_status", + "reason", + "source_refs", + "source_version", + "requires_review" + ] +} +``` + +Use specific types and enums where the application has a real closed set. Add descriptions that +define business meaning, not instructions such as "be accurate." Reject unexpected fields when +the provider and schema dialect support that rule. Version the contract outside the model output +so a reviewer can tell which definition produced a proposal. + +Do not ask one object to contain raw documents, chain-of-thought, credentials, and destination +fields. Keep sensitive evidence in its source system or a protected store and return stable +references. The final object should contain what the next control needs, not every token the agent +saw. + + +## 2. Run the shape gate in application code + +Provider-side structured output is the first check, not the only parser. Validate the received +object again at your application boundary with the schema dialect and library you control. + +The shape gate should answer deterministic questions: + +- Is the response a complete JSON object rather than a refusal, timeout, or truncated stream? +- Are all required properties present? +- Do values have the declared types? +- Are enum, length, range, and array constraints satisfied? +- Are unexpected properties rejected? +- Is the output-contract version supported by this worker? + +Treat validation failure as a typed error. Preserve a safe error code and run ID, then retry only +when the operation is safe to repeat. Do not silently coerce `"yes"` into `true`, discard unknown +fields, or invent missing identifiers. Those repairs can hide a contract mismatch between the +agent and the application. + +Provider schema support is not identical. Gemini documents a supported subset of JSON Schema, and +other providers may impose their own nesting or keyword limits. Keep a provider-facing schema and +an authoritative application schema if necessary, then test that every provider output still +passes the authoritative validator. + + +## 3. Run the meaning gate against current state + +The meaning gate checks facts and policy that JSON Schema cannot know. It should read authoritative +systems after receiving the structured output and before authorizing a write. + +Check at least: + +1. **Identity:** Does `customer_id` resolve inside the current account and approved scope? +2. **Evidence:** Do the source references exist, and does `source_version` still match? +3. **Consistency:** Does the proposed value agree with the cited evidence? +4. **Freshness:** Has the destination or source changed since the agent formed the proposal? +5. **Policy:** May this agent propose this field and operation? +6. **Consequence:** Does the change require a human or a separate approval policy? + +Separate machine-checkable rules from judgment. Software can prove that a SKU exists, a date is +not in the future, and a price uses an allowed currency. A reviewer may need to decide whether an +ambiguous message really authorizes cancellation. Record each result independently instead of +collapsing everything into `valid: true`. + +Source content remains untrusted even when the source is approved. A ticket, page, email, or file +can contain instructions addressed to the agent. Those instructions cannot change the schema, +tool permissions, destination, approval policy, or source allowlist. + + +## 4. Stage the structured output before changing the destination + +For any meaningful write, store the result as a proposal first. A staging row turns an ephemeral +model response into something a validator, reviewer, retry worker, or second agent can inspect. + +| Field | Purpose | +|---|---| +| `proposal_id` | Stable index and idempotency key | +| `output_contract_version` | Schema expected by the application | +| `agent_run_id` | Link to runtime evidence without copying it | +| `source_refs` and `source_version` | Evidence identity and freshness | +| `proposed_values` | Schema-valid agent result | +| `shape_status` | Validator result and safe error codes | +| `meaning_status` | Business-rule and evidence result | +| `review_status` | Pending, accepted, rejected, or superseded | +| `destination_key` | Stable business key to create or update | +| `write_status` | Not started, applied, uncertain, or verified | + +Staging is especially useful when the model provider has already validated the output. It keeps +that success from being mistaken for authorization. The proposal can pass the shape gate and +still remain blocked at the meaning or review gate. + +If your use case begins with a document or message that must become a destination record, the +[AI agent data-entry workflow](/blog/ai-agent-data-entry) covers source capture, mapping, approval, +and read-back in more detail. + + +## 5. Run the write gate with stable identity and read-back + +The write gate converts one accepted proposal into one destination operation. Resolve the +destination by a stable business key before deciding between create and update. Do not use row +position, search-result order, or a model-generated guess as identity. + +For each accepted proposal: + +1. Read the current destination by its stable key. +2. Compare current state with the values reviewed at the meaning gate. +3. Stop or supersede the proposal if relevant state changed. +4. Create or update only the approved fields. +5. Attach the same `proposal_id` as a correlation key where supported. +6. Read the destination back by stable key. +7. Compare the stored result with the accepted proposal. +8. Mark the write verified only after the comparison passes. + +An API timeout after a write is not proof of failure. Read before replaying. If the destination +already contains the intended values for the same proposal, record success. If it contains a +different value, stop for reconciliation. The [idempotent agent-update +guide](/blog/idempotent-ai-agent-updates) covers this uncertain-write path. + + +## How should agents handle retries and output-schema changes? + +Bind each proposal to the exact output-contract version, source version, and intended destination +operation. A retry with the same inputs should reuse the same `proposal_id`. A changed source, +schema, target, or proposed value should create a new proposal and supersede the old one. + +Evolve contracts deliberately: + +- Add optional fields before making them required. +- Keep old validators available while in-flight proposals still use them. +- Migrate staged proposals explicitly rather than reinterpreting them silently. +- Record which agent, prompt, mapping, and output contract produced each proposal. +- Test refusals, truncation, provider errors, unknown fields, stale evidence, duplicates, and + uncertain writes. + +Do not let a model improvise a migration. The application owns the contract, and the destination +owns its own schema. A provider-facing output schema can change independently from the +[dataset schema](/docs/design-schema), but the mapping between them must be versioned and tested. + + +## Where does Rowset fit in a structured-output workflow? + +Rowset is a private structured-row backend for trusted agents. It does not generate or validate a +model provider's structured output for you. Use the agent SDK and your application validator for +the shape gate, then use Rowset to hold reviewable proposals or accepted operational rows. + +A small setup can use two private datasets: + +```text +output_proposals index: proposal_id +customer_status index: customer_id +``` + +Put the output-contract version, review rules, and allowed transitions in the proposal dataset's +instructions and metadata. Define semantic column types for the fields humans inspect. Use a +stable index for retry-safe lookup, connect through [hosted MCP](/docs/connect-mcp) when an agent +needs discoverable tools, or use the [Dataset API](/docs/dataset-api) from application code. + +Keep provider credentials in the agent runtime, not in dataset rows. Keep both datasets private +unless a deliberate read-only public preview is part of the workflow. Rowset datasets are mutable, +so use a purpose-built immutable store as well when the workflow requires tamper-evident or +compliance-grade records. + +If this shape -> meaning -> write contract matches your workflow, you can [start a 7-day Rowset +trial](/pricing) and create the smallest private staging dataset first. + + +## AI agent structured output FAQ + +### What is structured output for an AI agent? + +Structured output is an agent result constrained to a machine-readable schema, commonly JSON +Schema. It gives application code predictable fields and types. It does not by itself verify the +truth, freshness, authorization, uniqueness, or safety of the values. + +### Is JSON mode the same as structured output? + +Not necessarily. JSON mode may guarantee valid JSON without requiring a specific object shape. +Schema-constrained output requires the result to match declared properties, types, and supported +constraints. Check the provider's current documentation because terminology and supported JSON +Schema features differ. + +### Should an agent write structured output directly to a database? + +Only for low-risk operations whose identity, authorization, validation, and retry behavior are +enforced outside the model. For consequential or ambiguous writes, stage the output as a proposal, +run business checks, obtain any required approval, then write and read the destination back. + +### Do structured outputs prevent hallucinations? + +No. They constrain format, not factual accuracy. A model can return a schema-valid but unsupported +claim or the wrong identifier. Require source references, verify them against current state, and +keep deterministic business rules outside the prompt. + +### What should be the stable ID for a structured-output proposal? + +Use a reproducible ID derived from the workflow, source item and version, output-contract version, +and intended destination operation. The same logical attempt should produce the same ID on retry. +A materially changed source or proposal should produce a new ID and supersede the old record. + +## The operating rule + +Treat AI agent structured output as a typed proposal, not a completed transaction: validate its +shape, verify its meaning against current evidence and policy, then apply one retry-safe write and +read the destination back. diff --git a/apps/pages/content/docs/design-schema.md b/apps/pages/content/docs/design-schema.md index 4b322fba..1762a480 100644 --- a/apps/pages/content/docs/design-schema.md +++ b/apps/pages/content/docs/design-schema.md @@ -9,6 +9,10 @@ keywords: Rowset schema, column types, choice columns, reference columns, datase Good Rowset datasets are explicit. The goal is not only to store rows, but to give future agents enough context to read and update the right fields. +If a model produces JSON before an agent writes rows, use the [structured-output +validation guide](/blog/ai-agent-structured-output) to separate schema validity, +business-rule checks, and retry-safe persistence. + ## Use semantic column types Rowset supports these semantic types: From 73e42b7248bffb63b8abe7559e624bf75223dae0 Mon Sep 17 00:00:00 2001 From: OpenClaw Scribe Date: Mon, 3 Aug 2026 06:14:17 +0000 Subject: [PATCH 2/2] content: register structured output PR --- .seo/content-ledger.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.seo/content-ledger.md b/.seo/content-ledger.md index b82dec6f..906869aa 100644 --- a/.seo/content-ledger.md +++ b/.seo/content-ledger.md @@ -37,7 +37,7 @@ | 2026-07-30 | AI Agent for Data Entry: A Safe Structured Workflow | how-to / operational guide | `/blog/ai-agent-data-entry` | AI agent for data entry | 20 | n/a | agent-managed datasets, data cleaning, index-column guide, idempotency, schema design, MCP, Dataset API, pricing | #374 | | 2026-08-01 | AI Data Collection: A Reviewable Agent Workflow | how-to / operational guide | `/blog/ai-data-collection` | AI data collection | 210 | 2 | agent-managed datasets, data entry, source registry, checkpoints, provenance, schema design, MCP, Dataset API, pricing | #391 | | 2026-08-02 | MCP OAuth vs API Keys: Choose the Right Auth | comparison / decision guide | `/blog/mcp-oauth-vs-api-keys` | MCP OAuth | 390 | 9 | MCP setup, agent access, MCP vs REST, safe sharing, pricing | #392 | -| 2026-08-03 | AI Agent Structured Output: Validate Before You Store | how-to / operational decision guide | `/blog/ai-agent-structured-output` | AI agent structured output | unmeasured | n/a | schema design, data entry, idempotent updates, MCP, Dataset API, pricing | #TBD | +| 2026-08-03 | AI Agent Structured Output: Validate Before You Store | how-to / operational decision guide | `/blog/ai-agent-structured-output` | AI agent structured output | unmeasured | n/a | schema design, data entry, idempotent updates, MCP, Dataset API, pricing | #396 | ---