Skip to content

Migration#336

Merged
flo-bit merged 13 commits into
mainfrom
migration
Jul 1, 2026
Merged

Migration#336
flo-bit merged 13 commits into
mainfrom
migration

Conversation

@flo-bit

@flo-bit flo-bit commented Jul 1, 2026

Copy link
Copy Markdown
Owner

No description provided.

flo-bit added 12 commits June 29, 2026 05:52
In-place migration kickoff (supersedes the v2 rewrite; see ../blento-migration-plan.md).
- move the v1 app into apps/web/ (pure relocation via git mv; code unchanged)
- monorepo root: package.json (workspaces), turbo.json, tsconfig.base.json;
  pnpm-workspace.yaml gains packages: [apps/*, packages/*]
- port the diff-harness from the v2 spike -> packages/diff-harness (the regression net)
- gitignore: match .svelte-kit/.turbo anywhere (monorepo), ignore diff-snapshots
- fix a pre-existing latent type error surfaced by a fresh svelte-kit sync
  (GiphySearchModal: PUBLIC_GIPHY_API_TOKEN is string|undefined)

Validated: app svelte-check 0 errors / 0 warnings (3453 files); harness tests 6/6.
env.staging reuses prod READ bindings (cache/domains/contrail-D1) so the harness
diffs the same data through new code; no cron, separate analytics dataset. Deploy:
pnpm --filter @blento/web exec wrangler deploy --env staging. Caveat: shares prod
KV/D1 — keep edit/write traffic off staging (or point at isolated namespaces).
Pure, unit-tested foundation for the schema migration — no app wiring yet.
- port @blento/schema from the v2 spike: Node envelope + migrateV1 (card/section
  -> node) + migratePage. v2's V1Card/V1Section match v1's card/section lexicons 1:1.
- add app.blento.node lexicon (apps/web/lexicons/custom/app/blento/node.json)
- add node <-> record serialization (nodeToRecord/recordToNode) with round-trip
- add prettier-plugin-tailwindcss to root devDeps (monorepo-wide format)

Validated: 10 unit tests; real-data check.ts on live repos migrates cleanly
(e.g. 9 cards -> 10 nodes incl. synthesized grid container; 0 orphans / floats /
rank-dupes / round-trip failures). Schema tsc clean.
…graph

First app change of the schema migration. load.ts now builds the node graph from
the card/section records (no coord ladder — records are already normalized) and
projects it back to v1-shaped sections/cards via nodesToItems, replacing
ensureSections. The node graph is attached to WebsiteData (source of truth going
forward); the render is untouched and still consumes cards/sections.

- @blento/schema: split migrateV1 into buildGraph (no-ladder structural transform,
  with 'input'|'document' leaf order) + nodesToItems (inverse → sections/cards);
  migrateV1 = ladder + buildGraph. +3 tests (13 total).
- load.ts: add card id (rkey) in both the contrail and PDS paths (was discarded);
  swap ensureSections → buildGraph→nodesToItems; attach result.nodes.
- vite: ssr.noExternal [/^@blento\//] so the raw-TS workspace pkg transforms for SSR.

GATE (prod vs local-dev, 5 sites): content 0 links / 0 images; pixel 0.0% (0px) —
render byte-for-byte identical to prod. App svelte-check 0/0 (3459 files).
… to card/section

Prerequisite for migrate-on-save: the read path must understand node records before
the write path stops writing card/section (else a saved page loses its cards).

- load.ts (PDS path): list app.blento.node; if this page has node records, build the
  graph from them (recordToNode); otherwise migrate-on-read from card/section as before.
  Contrail path unchanged (stays on card/section until contrail indexes app.blento.node)
  — so prod reads are untouched.
- @blento/schema: full write->read round-trip test (cards/sections -> nodes -> records
  -> nodes -> cards/sections is render-lossless). 14 tests total.

GATE (prod vs local, 5 sites): content 0/0, pixel 0px — no page has nodes yet, so the
fallback path is exercised and unchanged. App svelte-check 0/0.
…age root

The write path, gated OFF by default (PUBLIC_ENABLE_NODE_MIGRATION) so it's prod-safe:
the contrail read path still serves card/section, so writing nodes there would blank
a page. Enable per-deploy once node reads work everywhere; the read side is already
dual-format and a half-migrated repo still renders (migrate-on-read covers the rest).

- @blento/schema: planPageWrite (pure) — given current sections/cards + what was loaded,
  computes node puts + node/card/section deletes; retires legacy records on first
  migration only. +3 tests (17 total).
- save.ts: when enabled, persist the page as app.blento.node + retire legacy card/section;
  root record -> app.blento.page for ALL pages, deleting site.standard.publication on first
  migration. Legacy card/section write preserved as the default path.
- load.ts: set data.migratedStorage (read-from-nodes vs rebuilt) so save knows whether to
  retire legacy records. settings: app.blento.node in the writable/OAuth-scope collections.
- check.ts: real-data simulation of the full save→read cycle.

VALIDATION: 17 unit tests; real repos round-trip save→read render-lossless (saveReadFail 0
on 3 live sites); render gate prod-vs-local 0/0 + 0px (flag off). App svelte-check 0/0.
NOT YET: live auth'd save on a real account, and contrail indexing app.blento.node — both
required before enabling on prod.
buildGraph adopts orphan cards into the current page and stamps node.page, so feeding
it ALL of a repo's cards (PDS path was unfiltered) would render other pages' cards here
and — via originalCards on save — delete their records (data loss on multi-page repos in
dev/self-host). Filter cards by page like sections already are. Contrail path unaffected
(filters server-side). Gate prod-vs-local still 0/0 + 0px.
Standalone (raw fetch, no app/D1 deps — runs anywhere). Discovers every blento repo
via com.atproto.sync.listReposByCollection on a relay (the mechanism contrail uses for
discovery), unioned over app.blento.card + app.blento.node so it covers legacy and
migrated users. Per repo, backs up the blento-owned collections (node/card/section/page/
site.standard.publication/pronouns) and downloads every referenced blob.

Layout: <out>/<UTC-timestamp>/<did>/<collection>/<rkey>.json and <did>/_blobs/<cid>.
Run: pnpm --filter @blento/backup start [outDir]  (BLENTO_BACKUP_LIMIT to cap for a test).

Smoke-tested: discovery found 1397 repos; backed up sample repos with records + blobs intact.
Repos run in parallel via a shared-cursor worker pool; each repo's own collections +
blobs stay sequential to avoid hammering a single PDS. ~5x faster in testing (16 repos
in 17s, 0 failed). Almost all time is network wait, so concurrency is the main lever.
Completes the dual-format read on prod's path (PDS path already did this in 1c).
- contrail.config.ts: register `node` collection (queryable by page + type) so contrail
  creates records_node, ingests from the firehose, and serves app.blento.node.listRecords.
- loadFromContrail: query app.blento.node (tolerant .catch — if contrail hasn't indexed it
  yet, returns null → migrate-on-read from card/section). A fully-migrated page (cards
  deleted) takes profiles from the node query. loadData populates nodesFromRecords.

After deploy, prod resolves the node graph everywhere → the write migration becomes safe
to enable (the last code blocker before flipping PUBLIC_ENABLE_NODE_MIGRATION). Gate
prod-vs-local 0/0 + 0px (PDS path locally; contrail node read needs a deployed build to
exercise). svelte-check 0/0.
…render)

Locks in the designed schema (blento-schema-design.md): drop the top-level `type`
string; the node carries `content` (data, $type = shape), `layout` ($type = contract),
`style` ({renderer?, tokens?}), `source`. Two decoupled axes — content.$type is the data
shape, style.renderer is the render type (defaults from content.$type).

- node.ts: new Node/Content/Layout/Style/Source; def NSID constants.
- migrate.ts: buildGraph emits content {$type:#card/#container, cardType/containerType,
  ...data}, layout {$type:#gridCell, ...}, style {tokens:{color}}; nodesToItems inverts.
  Migration uses the generic #card/#container fallbacks (typed #link/#image are additive).
- serialize.ts: record shape follows (content required, type/data gone).
- lexicons: app.blento.node (content/layout/style/source), new app.blento.defs
  (#card/#container/#gridCell).

Still free to change — nothing reads app.blento.node in prod yet. Validated: schema
tsc + 17 tests; real-data save→read lossless (saveReadFail 0); render gate pixel 0px
(content-diff residue is localhost-vs-prod self-links + avatar CDN transform, both
profile/env, untouched here). App svelte-check 0/0.
Ran contrail-lex generate + lex-cli generate after the node lexicon rewrite (I'd added
app.blento.node in 1a without ever generating its artifacts). Produces the contrail query
endpoints (app.blento.node.listRecords/getRecord) and the atcute TS types
(src/lexicon-types/.../app/blento/node*), and refreshes the rest from the updated sources.

App svelte-check 0/0 (3464 files).
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 1, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
blento d6d0dc9 Commit Preview URL

Branch Preview URL
Jul 01 2026, 02:06 PM

@flo-bit flo-bit merged commit 8588322 into main Jul 1, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant