diff --git a/.seo/briefs/migrate-agent-dataset-business-key.md b/.seo/briefs/migrate-agent-dataset-business-key.md new file mode 100644 index 00000000..9dd295d2 --- /dev/null +++ b/.seo/briefs/migrate-agent-dataset-business-key.md @@ -0,0 +1,69 @@ +# SEO brief: migrate an AI-agent dataset to a business key + +- **Date:** 2026-08-04 +- **Primary keyword:** business key database +- **Secondary queries:** database primary key migration; AI agent data migration; agent dataset + index migration +- **Type:** how-to / operational decision guide +- **Why selected:** The current backlog's remaining measured candidates are explicitly deferred + for overlap, authority, or product-fit reasons. The coverage map names generated-index migration + patterns as the highest-priority open product-native gap. +- **Measured signal:** DataForSEO reports 10 US searches/month for `business key database`; KD is + unavailable. The exact migration terms were unmeasured. The live `database primary key + migration` SERP contains an AI Overview, primary vendor documentation, Stack Exchange/forum + results, and PAA questions about changing a primary key. +- **Product-led thesis:** A Rowset user who outgrows generated `rowset_id` needs a safe migration + path that preserves agent lookups, retries, relationships, and rollback. Rowset can credibly + support the mapping, destination, verification, and cutover records without claiming in-place + index replacement. + +## Information gain + +The post introduces a five-phase identity migration contract for agent-managed datasets: map, +mirror, verify, cut over, and retire. It adds agent-specific controls absent from the broad SQL +migration SERP: a durable old-to-new key map, deterministic mirroring, relationship translation, +agent configuration cutover, canary verification, and a read-only rollback window. + +## Table stakes and gap + +- **Table stakes:** define stable/unique/non-null identity; audit key-shape assumptions; create a + target schema; backfill; validate; cut over; retain rollback. +- **SERP gap:** existing results focus on engine-level DDL or vendor-specific key shape. They do + not explain how long-running AI agents, retries, prompts, relationship values, and tool + configuration depend on row identity. + +## Verified claim ledger + +| Claim | Source | Tier/date | Status | +|---|---|---|---| +| PostgreSQL primary keys require unique, non-null values. | https://www.postgresql.org/docs/current/ddl-constraints.html | primary; checked 2026-08-04 | verified primary | +| Key migrations must audit application reliance on key ordering and shape. | https://docs.cloud.google.com/spanner/docs/primary-keys-overview | primary; checked 2026-08-04 | verified primary | +| Rowset adds a generated `rowset_id` when `index_column` is omitted. | https://rowset.lvtd.dev/docs/dataset-api and `apps/api/services.py` | primary product; checked 2026-08-04 | verified in live docs and repo | +| Rowset generated index values are managed and cannot be patched. | `apps/api/row_mutations.py` and product tests | primary product; checked 2026-08-04 | verified in repo | +| Rowset generated index columns cannot be renamed, and index columns cannot be dropped. | https://rowset.lvtd.dev/docs/design-schema and schema mutation tests | primary product; checked 2026-08-04 | verified in live docs and repo | +| Dataset creation accepts at most 1,000 initial rows. | https://rowset.lvtd.dev/docs/dataset-api and API service validation | primary product; checked 2026-08-04 | verified in live docs and repo | + +## Entity and question map + +- business key, generated ID, primary key, Rowset index column +- one-to-one key mapping, uniqueness, non-null identity, normalization +- snapshot boundary, backfill, mirrored writes, read-back, reconciliation +- relationship translation, cutover, canary, rollback, archival +- PAA: How do you change a primary key? What is a database migration? What is a primary key? + +## AI SEO side check + +- Direct answer and liftable five-step process appear before the first H2. +- Each important factual claim is attributed to a primary or product source. +- Published/updated dates and `BlogPosting` schema are provided by the existing blog surface. +- FAQ answers are self-contained; headings match process and decision queries. + +## Product-led SEO side check + +- User job: preserve exact row identity while an agent-managed workflow moves to a business key. +- Product surface: Dataset API, MCP, schema instructions, by-index operations, private migration + datasets, and archive/rollback. +- Credible angle: Rowset's generated-index behavior and agent-native dataset contract are + inspectable in its source and docs. +- Business path: internal links connect the guide to setup docs, row operations, schema design, + related identity content, and pricing. diff --git a/.seo/config.json b/.seo/config.json index 7dd8590f..840a8e03 100644 --- a/.seo/config.json +++ b/.seo/config.json @@ -37,10 +37,10 @@ "content_ledger_dataset_key": "9ae7d0d3-1138-43a0-b3b8-aa05fac12a96", "research_briefs_dataset_key": "3a004cfa-0a7e-4c3a-a517-caca902f38d6", "schema_version": 1, - "last_sync_at": "2026-08-03" + "last_sync_at": "2026-08-04" }, "created_at": "2026-07-04", - "updated_at": "2026-08-03", + "updated_at": "2026-08-04", "notes": [ "Initialized from latest origin/main after fast-forwarding from c11f1de to 2d9389b.", "Refreshed from current origin/main b93e5d6 with GSC, Plausible, PostHog, DataForSEO, Exa, Firecrawl, Jina, and live HTTP checks on 2026-07-15.", diff --git a/.seo/content-ledger.md b/.seo/content-ledger.md index 906869aa..a2ad862f 100644 --- a/.seo/content-ledger.md +++ b/.seo/content-ledger.md @@ -38,6 +38,7 @@ | 2026-08-01 | AI Data Collection: A Reviewable Agent Workflow | how-to / operational guide | `/blog/ai-data-collection` | AI data collection | 210 | 2 | agent-managed datasets, data entry, source registry, checkpoints, provenance, schema design, MCP, Dataset API, pricing | #391 | | 2026-08-02 | MCP OAuth vs API Keys: Choose the Right Auth | comparison / decision guide | `/blog/mcp-oauth-vs-api-keys` | MCP OAuth | 390 | 9 | MCP setup, agent access, MCP vs REST, safe sharing, pricing | #392 | | 2026-08-03 | AI Agent Structured Output: Validate Before You Store | how-to / operational decision guide | `/blog/ai-agent-structured-output` | AI agent structured output | unmeasured | n/a | schema design, data entry, idempotent updates, MCP, Dataset API, pricing | #396 | +| 2026-08-04 | Migrate an AI-Agent Dataset to a Business Key | how-to / operational decision guide | `/blog/migrate-agent-dataset-business-key` | business key database | 10 | n/a | index-column guide, rowset_id vs business keys, schema design, row operations, idempotent updates, MCP, pricing | #397 | --- @@ -71,6 +72,7 @@ | 24 | MCP authentication | pillar / implementation guide | MCP authentication | 720 | 21 | navigational | 17 | Defer: measured demand is strong, but KD exceeds the conservative authority band and the SERP is dominated by protocol and platform documentation. Revisit after this narrower OAuth decision guide earns traction. | | 25 | MCP security best practices | security guide | MCP security best practices | 110 | 27 | commercial | 14 | Defer: KD is above the current authority band and the broad topic would overlap existing safe-sharing, approval, audit, and authentication content. | | 26 | AI agent structured output | how-to / operational decision guide | AI agent structured output | unmeasured | n/a | implementation / informational | shipped (19) | Selected 2026-08-03 after live SERP research. Product-led angle: a shape -> meaning -> write contract that connects provider schema output to staged, retry-safe Rowset rows without treating valid JSON as proof of correctness. | +| 27 | Migrate an AI-agent dataset to a business key | how-to / operational decision guide | business key database | 10 | n/a | commercial / implementation | shipped (18) | Selected 2026-08-04 from the coverage map's highest-priority open product-native gap. Differentiated with a map -> mirror -> verify -> cut over -> retire identity migration contract. | --- @@ -78,7 +80,7 @@ | Cluster / theme | Pieces shipped | Gaps still open | |---|---|---| -| Agent-managed datasets | `/blog/agent-managed-datasets`, `/blog/choose-index-column-agent-rows`, `/blog/structure-dataset-instructions-ai-agents`, `/blog/rowset-id-vs-business-keys`, `/blog/relationship-modeling-agent-datasets`, `/blog/ai-agent-memory-vs-state`, `/blog/idempotent-ai-agent-updates`, `/blog/share-ai-agent-data-safely`, `/blog/human-in-the-loop-ai-agents`, `/blog/ai-agent-audit-trail`, `/blog/database-for-ai-agents`, `/blog/ai-data-cleaning-agent`, `/blog/ai-agent-crm`, `/blog/ai-agent-task-management`, `/blog/ai-agent-inventory-management`, `/blog/ai-customer-feedback-analysis`, `/blog/ai-ready-data`, `/blog/ai-agent-data-entry`, `/blog/ai-data-collection` | Generated-index migration patterns | +| Agent-managed datasets | `/blog/agent-managed-datasets`, `/blog/choose-index-column-agent-rows`, `/blog/structure-dataset-instructions-ai-agents`, `/blog/rowset-id-vs-business-keys`, `/blog/relationship-modeling-agent-datasets`, `/blog/ai-agent-memory-vs-state`, `/blog/idempotent-ai-agent-updates`, `/blog/share-ai-agent-data-safely`, `/blog/human-in-the-loop-ai-agents`, `/blog/ai-agent-audit-trail`, `/blog/database-for-ai-agents`, `/blog/ai-data-cleaning-agent`, `/blog/ai-agent-crm`, `/blog/ai-agent-task-management`, `/blog/ai-agent-inventory-management`, `/blog/ai-customer-feedback-analysis`, `/blog/ai-ready-data`, `/blog/ai-agent-data-entry`, `/blog/ai-data-collection`, `/blog/migrate-agent-dataset-business-key` | Composite-key and cross-system identity patterns | | MCP and Dataset API | `/blog/mcp-vs-rest-ai-agents`, `/blog/connect-ai-agent-to-dataset-api`, `/blog/relationship-modeling-agent-datasets`, `/blog/ai-agent-inventory-management`, `/blog/ai-agent-data-entry`, `/blog/mcp-oauth-vs-api-keys`, `/blog/ai-agent-structured-output` | More REST/MCP setup examples with concrete datasets | | Spreadsheet/database alternatives | `/blog/airtable-alternatives`, `/blog/google-sheets-alternatives`, `/blog/baserow-alternatives`, `/blog/nocodb-alternatives`, `/blog/spreadsheet-database-for-ai-agents`, `/vs/airtable`, `/vs/google-sheets` | Migration examples for specific human-to-agent workflows | diff --git a/.seo/keyword-research.json b/.seo/keyword-research.json index 6895c491..dc0fae44 100644 --- a/.seo/keyword-research.json +++ b/.seo/keyword-research.json @@ -234,9 +234,18 @@ {"keyword": "mcp database server", "source": "dataforseo", "search_volume": 30, "keyword_difficulty": 19, "cpc": null, "intent": "navigational", "as_of": "2026-07-15"}, {"keyword": "mcp vector database", "source": "dataforseo", "search_volume": 20, "keyword_difficulty": 2, "cpc": null, "intent": "informational", "as_of": "2026-07-15"}, {"keyword": "ai agent database", "source": "dataforseo", "search_volume": 20, "keyword_difficulty": 7, "cpc": null, "intent": "commercial", "as_of": "2026-07-15"}, - {"keyword": "database for ai agents", "source": "dataforseo", "search_volume": 10, "keyword_difficulty": 7, "cpc": 19.47, "intent": "commercial", "as_of": "2026-07-15"} + {"keyword": "database for ai agents", "source": "dataforseo", "search_volume": 10, "keyword_difficulty": 7, "cpc": 19.47, "intent": "commercial", "as_of": "2026-07-15"}, + {"keyword": "business key database", "source": "dataforseo", "search_volume": 10, "keyword_difficulty": null, "cpc": null, "intent": "commercial", "as_of": "2026-08-04"} ], "opportunities": [ + { + "cluster": "business-key migration for agent datasets", + "target_url": "/blog/migrate-agent-dataset-business-key", + "status": "shipped", + "priority": 14, + "signals": {"primary_volume": 10, "primary_kd": null, "primary_cpc": null}, + "notes": "Selected 2026-08-04 from the coverage map's highest-priority open product-native gap after remaining measured backlog candidates were deferred. The guide adds a map -> mirror -> verify -> cut over -> retire identity migration contract for agent-managed rows." + }, { "cluster": "AI agent structured output", "target_url": "/blog/ai-agent-structured-output", @@ -343,6 +352,12 @@ } ], "serp_snapshots": { + "database primary key migration": { + "source": "dataforseo live organic SERP + primary-source extraction", + "as_of": "2026-08-04", + "top_domains": ["docs.cloud.google.com", "dba.stackexchange.com", "ibm.com", "stackoverflow.com", "learn.microsoft.com"], + "notes": "The SERP contains an AI Overview, vendor documentation, forum answers, and PAA questions about primary-key changes. The Rowset guide differentiates with agent configuration, retry, relationship translation, deterministic mirroring, canary, and rollback controls." + }, "AI agent structured output": { "source": "live web search + primary-source extraction", "as_of": "2026-08-03", diff --git a/.seo/link-inventory.md b/.seo/link-inventory.md index 0981846f..14628d22 100644 --- a/.seo/link-inventory.md +++ b/.seo/link-inventory.md @@ -93,6 +93,7 @@ | `/blog/ai-data-collection` | https://rowset.lvtd.dev/blog/ai-data-collection | AI Data Collection: A Reviewable Agent Workflow | AI data collection, AI data collection workflow, collection control plane, source registry and capture runs, reviewable agent-collected records | | `/blog/mcp-oauth-vs-api-keys` | https://rowset.lvtd.dev/blog/mcp-oauth-vs-api-keys | MCP OAuth vs API Keys: Choose the Right Auth | MCP OAuth vs API keys, MCP authorization decision guide, provisioned API key for agents, delegated MCP access, agent authentication delegation test | | `/blog/ai-agent-structured-output` | https://rowset.lvtd.dev/blog/ai-agent-structured-output | AI Agent Structured Output: Validate Before You Store | AI agent structured output, structured output validation for agents, shape meaning write contract, JSON Schema to durable rows, validated agent output workflow | +| `/blog/migrate-agent-dataset-business-key` | https://rowset.lvtd.dev/blog/migrate-agent-dataset-business-key | Migrate an AI-Agent Dataset to a Business Key | business-key migration for agent datasets, migrate generated row identity, agent dataset index migration, map mirror verify cutover retire, move from rowset_id to a business key | ## Shipped SEO Sprint Pages @@ -236,3 +237,8 @@ These route families remain retired and must not be selected as internal-link ta - shape meaning write contract - JSON Schema to durable rows - validated agent output workflow +- business-key migration for agent datasets +- migrate generated row identity +- agent dataset index migration +- map mirror verify cutover retire +- move from `rowset_id` to a business key diff --git a/CHANGELOG.md b/CHANGELOG.md index c236dee0..a92ba565 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,12 @@ deployment or release cut date. **Fixed** for any bug fixes. **Security** in case of vulnerabilities. +## 2026-08-04 + +### Added +- Added a guide for migrating agent-managed datasets from generated row identity to a stable + business key with mapping, mirrored writes, verification, cutover, and rollback. + ## 2026-08-03 ### Added diff --git a/apps/pages/content/blog/choose-index-column-agent-rows.md b/apps/pages/content/blog/choose-index-column-agent-rows.md index 31612cb7..85d91e21 100644 --- a/apps/pages/content/blog/choose-index-column-agent-rows.md +++ b/apps/pages/content/blog/choose-index-column-agent-rows.md @@ -130,6 +130,10 @@ Generated indexes are useful for: - one-off review lists that still need safe updates - imported rows where the apparent identifier is optional or duplicated +If a stable upstream ID becomes available later, follow the [agent-dataset business-key migration +guide](/blog/migrate-agent-dataset-business-key). It uses a new dataset, an explicit key map, +mirrored changes, and a rollback window instead of changing identity under active agents. + The tradeoff is that `rowset_id` is Rowset-owned identity. It is stable inside the Rowset dataset, but it may not mean anything to the upstream system unless you export it and keep it with the source. If the upstream workflow already has diff --git a/apps/pages/content/blog/migrate-agent-dataset-business-key.md b/apps/pages/content/blog/migrate-agent-dataset-business-key.md new file mode 100644 index 00000000..3ee55c58 --- /dev/null +++ b/apps/pages/content/blog/migrate-agent-dataset-business-key.md @@ -0,0 +1,289 @@ +--- +title: "Migrate an AI-Agent Dataset to a Business Key" +description: "Move agent-managed rows from a generated ID to a stable business key with mapping, mirrored writes, verification, cutover, and rollback." +published_at: 2026-08-04 +updated_at: 2026-08-04 +author: Rasul Kireev +keywords: + - business key database + - AI agent data migration + - database primary key migration + - agent dataset index migration +topics: + - agent workflows + - dataset operations + - row identity +canonical_url: https://rowset.lvtd.dev/blog/migrate-agent-dataset-business-key +image: /static/vendors/images/logo.png +image_alt: Rowset logo +robots: index, follow +--- + +To migrate an AI-agent dataset from a generated ID to a business key, create a new dataset with +the business key as its index, copy rows through an explicit old-to-new identity map, mirror new +writes during the transition, verify counts and values, then move agents to the new dataset. Keep +the old dataset available until rollback is no longer needed. + +Do not change identity in place while agents are still reading and writing the dataset. Row +identity appears in tool calls, retry logic, relationships, logs, exports, and human instructions. +An abrupt key change can make a correct agent update the wrong record or create a duplicate. + +Use this five-phase **identity migration contract**: + +1. **Map:** prove that every source row has one valid target business key. +2. **Mirror:** create a new dataset and send new changes to both identities. +3. **Verify:** compare row counts, key coverage, relationships, and important values. +4. **Cut over:** point agents at the new dataset under a recorded boundary. +5. **Retire:** archive the old dataset only after the rollback window closes. + +## In this guide + +- [When to migrate](#when-to-migrate) +- [Define the identity contract](#define-identity-contract) +- [Build the key map](#build-key-map) +- [Create the destination dataset](#create-destination) +- [Backfill and mirror changes](#backfill-and-mirror) +- [Verify the migration](#verify-migration) +- [Cut agents over safely](#cut-over) +- [Close the rollback window](#close-rollback) +- [Business-key migration FAQ](#business-key-migration-faq) + + +## When should an agent dataset move to a business key? + +Move to a business key when another system, person, or agent already uses a stable identifier for +the same record. Product SKUs, ticket IDs, customer IDs, and source-system record IDs can make +lookups and reconciliation clearer than a generated sequence that exists only inside one dataset. + +PostgreSQL defines a primary key as a unique, non-null identifier for rows. It also warns that +adding one changes the data contract by enforcing those constraints ([PostgreSQL 18 constraints +documentation, checked August 2026](https://www.postgresql.org/docs/current/ddl-constraints.html)). +Rowset's index column is not a general SQL primary key, but the same identity test applies: one +non-blank index value must resolve to one row. + +Do not migrate merely because a business key looks more descriptive. Keep the generated +`rowset_id` when the proposed key can change, arrive blank, collide, or be reassigned. The +[index-column decision guide](/blog/choose-index-column-agent-rows) provides the full stability +test, and the [Rowset ID versus business-key guide](/blog/rowset-id-vs-business-keys) explains the +tradeoff between internal and external identity. + + +## 1. Define the identity contract before copying rows + +Write down the migration boundary before creating the destination. A useful contract names: + +| Field | Example | Why it matters | +|---|---|---| +| Source dataset | `contacts-v1` | Prevents an agent from copying a similarly named table | +| Source index | `rowset_id` | Preserves the old lookup handle | +| Target dataset | `contacts-v2` | Gives the new identity a separate namespace | +| Target index | `contact_id` | States the business key being adopted | +| Key source | CRM `contact_id` | Establishes who owns identity | +| Snapshot boundary | `2026-08-04T06:30:00Z` | Separates backfill from later writes | +| Validation rules | required, unique, exact string | Makes acceptance deterministic | +| Cutover boundary | migration version or timestamp | Tells agents which dataset is authoritative | +| Rollback deadline | seven days after cutover | Prevents indefinite dual authority | + +The business system that issues the key should remain its authority. An agent may transport and +validate `contact_id`; it should not invent missing CRM IDs to make the migration pass. + +Google Cloud's Spanner primary-key migration guidance calls out a related risk: applications can +quietly depend on key ordering or key shape, not only uniqueness ([Spanner primary-key migration +overview, checked August 2026](https://docs.cloud.google.com/spanner/docs/primary-keys-overview)). +Audit those assumptions. If an agent treats a larger `rowset_id` as "newer," replace that behavior +with an explicit `created_at` field before cutover. + + +## 2. Build a complete old-to-new key map + +Create a private migration dataset indexed by the old identity. Keep the map separate from both +business datasets so it can record exceptions without polluting production rows. + +Suggested fields: + +```text +source_rowset_id +target_business_key +source_version +mapping_status +conflict_reason +verified_at +``` + +Populate one mapping row for every source row, then reject these cases: + +- a source row has no target key +- two source rows map to the same target key +- one source row maps to more than one target key +- the target key was normalized differently from the source system +- a relationship points at an old index value with no mapped target + +Do not trim, lowercase, or otherwise normalize keys unless the identity owner defines that rule. +`CUS-104` and `cus-104` may be the same identifier in one system and different identifiers in +another. Record the rule instead of letting an agent infer it. + +The mapping dataset is also the rollback bridge. If an operator reports a problem against the new +`contact_id`, the agent can resolve the old `rowset_id`, inspect both rows, and explain the +difference without fuzzy search. + + +## 3. Create a new dataset with the business key + +In Rowset, create a new dataset rather than trying to rename or drop the active generated index. +Generated index values are managed by Rowset, generated index columns cannot be renamed, and an +index column cannot be dropped. A new dataset makes the identity change reviewable and preserves +a clean rollback path. + +Create the destination with the target key present in `headers` and selected as `index_column`: + +```json +{ + "name": "Contacts v2", + "description": "Agent-managed contacts keyed by the CRM contact ID", + "instructions": "Use contact_id for exact lookup. Do not create IDs. Verify CRM evidence before writes.", + "headers": ["contact_id", "name", "email", "status", "source_updated_at"], + "index_column": "contact_id", + "column_types": { + "contact_id": { + "type": "text", + "description": "Stable contact identifier issued by the CRM" + }, + "email": "email", + "status": { + "type": "choice", + "choices": ["active", "inactive", "unknown"] + }, + "source_updated_at": "datetime" + } +} +``` + +The live [Dataset API documentation](/docs/dataset-api) confirms that Rowset accepts an explicit +index at creation and adds `rowset_id` only when `index_column` is omitted. Put field meaning and +workflow rules in the schema, instructions, and metadata so a future agent does not have to infer +them from the migration notes. See [designing a dataset schema](/docs/design-schema) for semantic +column types and descriptions. + + +## 4. Backfill the snapshot, then mirror new changes + +Record a snapshot boundary and copy only rows that have an accepted mapping. For each row: + +1. Read the source row by `rowset_id`. +2. Read its accepted target key from the mapping dataset. +3. Transform only fields covered by the migration contract. +4. Create the destination row with the target business key. +5. Read it back by the target key. +6. Compare the stored values with the migration payload. +7. Mark the mapping row verified or record a typed exception. + +Rowset dataset creation accepts up to 1,000 initial rows. For larger migrations, page through the +source and write bounded batches after creating the destination. The [row operations guide] +(/docs/work-with-rows) documents exact by-index reads and updates for MCP and REST. + +After the snapshot starts, route every new write through one migration worker. The worker writes +the current authoritative dataset first, applies the equivalent change to the other dataset, and +records both outcomes. If either response is uncertain, read both rows before retrying. The +[idempotent agent-update pattern](/blog/idempotent-ai-agent-updates) covers this reconciliation +path in detail. + +Do not ask independent agents to dual-write from prompt instructions. A prompt cannot provide +atomicity, ordering, or recovery after a timeout. One deterministic worker should own mirroring. + + +## 5. Verify identity, values, and relationships + +A matching row count is necessary, not sufficient. Run four checks: + +1. **Identity coverage:** every accepted source row has one destination key, and every destination + row points back to one mapping record. +2. **Value parity:** compare fields that should remain unchanged, using exact values or a canonical + hash over an agreed field list. +3. **Relationship coverage:** translate old index values and prove that every required target + exists before rebuilding links. +4. **Change convergence:** every mirrored write after the snapshot has the same final values in + both datasets or a resolved exception. + +Keep counts by status rather than one global pass flag: + +```text +source_rows=842 +accepted_mappings=839 +blocked_mappings=3 +destination_rows=839 +value_matches=839 +relationship_targets_missing=0 +mirror_exceptions_open=0 +``` + +These numbers are illustrative, not Rowset customer metrics. The important property is the +equation: destination rows should equal accepted mappings, while blocked rows remain visible and +do not silently disappear. + + +## 6. Cut agents over under a recorded boundary + +Cut over only when the verification report passes and blocked mappings have an explicit owner. +Update the agent setup, dataset key, instructions, relationship targets, scheduled jobs, and any +saved API configuration together. Do not rely on the dataset name alone; agents should use the +new dataset key and `contact_id` lookup contract. + +At the cutover boundary: + +1. Pause old-dataset writers. +2. Drain and verify the mirror queue. +3. Record the final old-dataset version or timestamp. +4. Switch reads and writes to the new dataset key. +5. Run a canary read, create-or-update, and read-back by business key. +6. Resume workers only after the canary passes. + +Keep the old dataset read-only during the rollback window. If the canary or later reconciliation +fails, pause writers, resolve new keys through the mapping dataset, and return to the old dataset +without reconstructing identity from names or search results. + + +## 7. Retire the old identity without deleting evidence + +Close the migration after the agreed window and a final convergence check. Archive the old dataset +rather than deleting it immediately. Preserve the migration contract, key map, verification +report, cutover event, and exception decisions according to the workflow's retention needs. + +Update durable instructions so future agents know that the business key is authoritative. Remove +the dual-write path, revoke obsolete access, and test that no scheduled worker still references +the old dataset key. Migration code that remains active becomes a second, accidental source of +truth. + +Rowset is useful here because the migration artifacts can remain private datasets beside the +operational records, accessible through [hosted MCP](/docs/connect-mcp) or REST. Rowset does not +turn a generated index into a business key in place; it gives trusted agents a structured surface +for the mapping, destination, verification, and cutover records. Review [Rowset pricing] +(/pricing) if you want to run the pattern with hosted datasets. + + +## Business-key migration FAQ + +### Can I rename `rowset_id` to my business key? + +No. A Rowset-generated index is managed metadata and cannot be renamed. Create a new dataset with +the business key selected as `index_column`, copy rows through a verified mapping, and cut agents +over after validation. This preserves rollback and prevents a live identity contract from changing +under active agents. + +### Should an AI agent generate missing business keys? + +Only when the agent is explicitly the authorized key issuer and the generation rule is +deterministic. In most migrations, the upstream CRM, catalog, ticket system, or application owns +the key. Missing keys should enter an exception queue rather than being guessed from names, +emails, or row positions. + +### How long should dual writes continue? + +Continue until the backfill is complete, post-snapshot changes have converged, and a canary proves +the new read and write path. Set a fixed end condition and rollback window. Indefinite dual writes +create two authorities and make later discrepancies harder to explain. + +### Is a matching row count enough to verify the migration? + +No. Verify one-to-one key coverage, important field values, translated relationships, and every +change written after the snapshot. A matching count can still hide duplicated keys, missing rows, +stale values, or links that point at the old identity. diff --git a/apps/pages/content/docs/design-schema.md b/apps/pages/content/docs/design-schema.md index 1762a480..2b73e8d3 100644 --- a/apps/pages/content/docs/design-schema.md +++ b/apps/pages/content/docs/design-schema.md @@ -116,6 +116,10 @@ Index columns cannot be dropped, and generated index columns cannot be renamed. Columns used by relationships must be unlinked before destructive schema changes. +If a generated index no longer fits an active workflow, do not change row identity under running +agents. Use the [business-key migration guide](/blog/migrate-agent-dataset-business-key) to build +a verified old-to-new key map, copy rows into a new dataset, cut over, and preserve rollback. + ## Use references for Rowset objects Use reference columns when a cell points at another Rowset object: