diff --git a/README.md b/README.md index dc30f32..6de9918 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,13 @@ # i0c.cc -i0c.cc is a personal edge redirect playground with a PostgreSQL control plane and an archived Git fallback. It runs the same core through optional edge-platform adapters and provides a WebUI with optional analytics for my own use. +i0c.cc is a personal edge redirect playground with a database-backed control plane, PostgreSQL enabled by default, and an archived Git fallback. It runs the same core through optional edge-platform adapters and provides a WebUI with optional analytics for my own use. ## Positioning This repository is maintained for personal use and engineering experimentation. It is not intended to be a hosted URL-shortening service or an enterprise redirect platform. - Deploy whichever Runtime adapter fits the environment; Cloudflare, Vercel, and Netlify are supported alternatives rather than required replicas. -- Use PostgreSQL for immediate database-backed saves, immutable history, and rollback; Git remains an archived build-time fallback. +- Use PostgreSQL by default, or bind Cloudflare D1 on a compatible WebUI host, for immediate saves, immutable history, and rollback; Git remains an archived build-time fallback. - Use the WebUI and analytics when they help the personal workflow; the roadmap prioritizes clarity and reliability over feature parity with commercial products. ## Projects @@ -22,7 +22,7 @@ This repository is maintained for personal use and engineering experimentation. | Plugin Catalog | [packages/plugin-catalog](packages/plugin-catalog) | Optional official presets and host-specific plugin configuration validation. | | Runtime Host | [packages/runtime-host](packages/runtime-host) | Platform-neutral Runtime deployment and executable-plugin installation contracts. | | Runtime Build | [packages/runtime-build](packages/runtime-build) | Build-time installation validation, root-config binding, and selected-adapter bundling. | -| Official plugins | [plugins](plugins) | Git and PostgreSQL data backends, an HTTP Runtime snapshot source, three Runtime adapters, analytics delivery and storage, and bot classification. | +| Official plugins | [plugins](plugins) | Git, PostgreSQL, and D1 data backends, an HTTP Runtime snapshot source, three Runtime adapters, analytics delivery and storage, and bot classification. | Executable plugins are selected at build time: Runtime installations live in [i0c.runtime.config.ts](i0c.runtime.config.ts), WebUI server installations in [i0c.webui.config.ts](i0c.webui.config.ts), and client-safe WebUI renderers in [apps/webui/webui.extensions.ts](apps/webui/webui.extensions.ts). The remote `config.json` document configures installed code but never downloads or executes new packages. @@ -81,13 +81,13 @@ The selected WebUI Repository contains two independently editable documents: - `config.json` stores non-sensitive instance settings such as the canonical Runtime origin, cache TTLs, robots policy, analytics namespace and collector endpoint, WebUI access policy, and namespaced plugin configuration. - `redirects.json` stores redirect rules. -The checked-in PostgreSQL Repository uses optimistic document revisions and exposes an atomic two-document snapshot. An empty database enters a one-time WebUI setup flow that creates both documents atomically. Every save, import, migration, and rollback produces an immutable revision; restoring an older document creates a new head instead of rewriting history. Managers can import and export both JSON documents for backup and migration. +The PostgreSQL and D1 Repositories implement the same optimistic-revision, atomic-snapshot, immutable-history, import/export, and rollback contract. The checked-in deployment selects PostgreSQL. A D1-capable WebUI host may select D1 and inject its database binding before the Repository is initialized. GitHub Contents remains available as an archived build-time fallback that preserves commits on a configured branch, but it is not enabled by the checked-in deployment. The WebUI can edit both documents; invalid `config.json` content remains visible to managers so it can be repaired. -The checked-in HTTP Snapshot Source reads one validated WebUI snapshot so config and rules always come from the same Repository revision. It uses ETags, bounded retries and timeouts, and the last valid memory or platform cache. GitHub Raw remains available when Git-backed data is selected. Runtime deployments never receive PostgreSQL credentials. +The checked-in HTTP Snapshot Source reads one validated WebUI snapshot so config and rules always come from the same Repository revision. It uses ETags, bounded retries and timeouts, and the last valid memory or platform cache. GitHub Raw remains available when Git-backed data is selected. Runtime deployments never receive database credentials or bindings. -[packages/config](packages/config) owns schemas, validation, safe defaults, and the build-time Repository and Source selection. Bootstrap changes such as GitHub paths, PostgreSQL connection policy, HTTP snapshot URL, or GitHub OAuth scope require rebuilding. PostgreSQL Repository migrations are deliberate external writes and never run during a build or application startup. +[packages/config](packages/config) owns schemas, validation, safe defaults, and the build-time Repository and Source selection. Bootstrap changes such as GitHub paths, database provider or connection policy, HTTP snapshot URL, or GitHub OAuth scope require rebuilding. Repository migrations are deliberate external writes and never run during a build or application startup. The former non-sensitive environment variables are not read as overrides or fallbacks. Existing values left in a provider dashboard are ignored and can be removed after the new deployment is verified. Secrets and deployment-specific bindings remain in each application's environment example. @@ -143,7 +143,7 @@ pnpm check ## Data documents -The two Repository implementations use the same document schemas: +All Repository implementations use the same document schemas: ```text packages/config/config.schema.json diff --git a/README.zh-CN.md b/README.zh-CN.md index a0898d8..af2a1be 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -1,13 +1,13 @@ # i0c.cc -i0c.cc 是一个以 PostgreSQL 为控制面、保留归档 Git 回退方案的个人边缘重定向实验项目。同一套核心可以通过不同边缘平台适配器运行，并提供自用的 WebUI 与可选统计功能。 +i0c.cc 是一个以数据库为控制面、默认启用 PostgreSQL、并保留归档 Git 回退方案的个人边缘重定向实验项目。同一套核心可以通过不同边缘平台适配器运行，并提供自用的 WebUI 与可选统计功能。 ## 项目定位 这个仓库面向个人使用和工程实验，不准备成为托管短链接服务或企业级重定向平台。 - 按部署环境选择所需的 Runtime 适配器；Cloudflare、Vercel 与 Netlify 是可选方案，不要求同时运行。 -- 使用 PostgreSQL 实现数据库即时保存、不可变历史与回滚；Git 只作为归档的构建期回退方案保留。 +- 默认使用 PostgreSQL；兼容的 WebUI 宿主也可绑定 Cloudflare D1，实现即时保存、不可变历史与回滚；Git 只作为归档的构建期回退方案保留。 - WebUI 与统计功能服务于个人工作流；后续路线优先保证清晰和可靠，不追求与商业产品功能对齐。 ## 项目 @@ -22,7 +22,7 @@ i0c.cc 是一个以 PostgreSQL 为控制面、保留归档 Git 回退方案的 | 插件目录 | [packages/plugin-catalog](packages/plugin-catalog) | 可选的官方预设与按宿主执行的插件配置校验。 | | Runtime 宿主 | [packages/runtime-host](packages/runtime-host) | 平台无关的 Runtime 部署与可执行插件安装契约。 | | Runtime 构建 | [packages/runtime-build](packages/runtime-build) | 构建期安装校验、根配置绑定与所选适配器 Bundle 生成。 | -| 官方插件 | [plugins](plugins) | Git 与 PostgreSQL 数据后端、HTTP Runtime 快照源、三个 Runtime 适配器、统计投递与存储，以及机器人分类。 | +| 官方插件 | [plugins](plugins) | Git、PostgreSQL 与 D1 数据后端、HTTP Runtime 快照源、三个 Runtime 适配器、统计投递与存储，以及机器人分类。 | 可执行插件在构建期选择：Runtime 安装位于 [i0c.runtime.config.ts](i0c.runtime.config.ts)，WebUI 服务端安装位于 [i0c.webui.config.ts](i0c.webui.config.ts)，客户端安全的 WebUI Renderer 位于 [apps/webui/webui.extensions.ts](apps/webui/webui.extensions.ts)。远程 `config.json` 文档只能配置已安装代码，不会下载或执行新包。 @@ -81,13 +81,13 @@ Vercel 需要保持开启 **Include source files outside of the Root Directory i - `config.json` 存放非敏感实例配置，包括 Runtime 规范域名、缓存时间、robots 策略、统计命名空间与收集端地址、WebUI 访问策略，以及按命名空间隔离的插件配置。 - `redirects.json` 存放重定向规则。 -仓库当前启用的 PostgreSQL Repository 使用乐观文档版本，并提供两份文档的原子快照。数据库为空时，WebUI 会进入一次性初始化流程，并原子创建两份文档。每次保存、导入、迁移和回滚都会产生不可变版本；恢复旧文档会创建新的活动版本，而不是改写历史。管理者可以导入和导出两份 JSON，用于备份和迁移。 +PostgreSQL 与 D1 Repository 实现同一套乐观版本、原子快照、不可变历史、导入导出和回滚契约。仓库当前部署选择 PostgreSQL；支持 D1 的 WebUI 宿主可选择 D1，并在 Repository 初始化前注入数据库 binding。 GitHub Contents 仍保留为归档的构建期回退方案，并可在指定分支保留 commit，但仓库当前部署不会启用它。WebUI 可以编辑两份文档；即使 `config.json` 写坏，管理员仍能看到原文并修复。 -仓库当前启用的 HTTP Snapshot Source 从 WebUI 读取一份经过校验的快照，确保配置和规则来自同一个 Repository revision；它使用 ETag、有限重试与超时，并保留最后一次有效的内存或平台缓存。选择 Git 数据后端时仍可使用 GitHub Raw。Runtime 部署不会获得 PostgreSQL 凭据。 +仓库当前启用的 HTTP Snapshot Source 从 WebUI 读取一份经过校验的快照，确保配置和规则来自同一个 Repository revision；它使用 ETag、有限重试与超时，并保留最后一次有效的内存或平台缓存。选择 Git 数据后端时仍可使用 GitHub Raw。Runtime 部署不会获得数据库凭据或 binding。 -[packages/config](packages/config) 负责 schema、校验、安全默认值，以及构建期 Repository 与 Source 选择。修改 GitHub 路径、PostgreSQL 连接策略、HTTP 快照地址或 GitHub OAuth scope 等启动配置后仍需重新构建。PostgreSQL Repository 迁移属于明确的外部写入，构建与应用启动都不会自动执行。 +[packages/config](packages/config) 负责 schema、校验、安全默认值，以及构建期 Repository 与 Source 选择。修改 GitHub 路径、数据库类型或连接策略、HTTP 快照地址或 GitHub OAuth scope 等启动配置后仍需重新构建。Repository 迁移属于明确的外部写入，构建与应用启动都不会自动执行。 原有非敏感环境变量不再作为覆盖值或回退值读取。平台后台遗留的旧值会被忽略，确认新部署正常后即可删除。密钥和与部署绑定的值继续保留在各应用的环境变量示例中。 @@ -143,7 +143,7 @@ pnpm check ## 数据文档 -两种 Repository 实现共用下面的文档 schema： +所有 Repository 实现共用下面的文档 schema： ```text packages/config/config.schema.json diff --git a/apps/webui/README.md b/apps/webui/README.md index dbd9141..d4148ac 100644 --- a/apps/webui/README.md +++ b/apps/webui/README.md @@ -62,9 +62,11 @@ This project provides two rule-editing modes and a separate settings surface: ## Data repository -The checked-in deployment selects PostgreSQL through [../../packages/config/src/defaults.ts](../../packages/config/src/defaults.ts) and uses `DATABASE_URL`. +The checked-in deployment selects PostgreSQL through [../../packages/config/src/defaults.ts](../../packages/config/src/defaults.ts) and uses `DATABASE_URL`. A D1-capable WebUI host may select `provider: "d1"` and inject a `D1Database` binding through `configureAppDataRepositoryBinding` before the Repository is first used. -First-run setup creates both documents in one transaction and refuses partially initialized databases. Every save creates an immutable revision. Import validates both JSON files and replaces them atomically, while restore copies an old document into a new head revision instead of rewriting history. Managers can export, import, inspect, and restore revisions from **Settings → Data and history**. +PostgreSQL and D1 are held to the same shared behavior contract. First-run setup creates both documents atomically and refuses partially initialized databases. Every save creates an immutable revision. Import validates both JSON files and replaces them atomically, while restore copies an old document into a new head revision instead of rewriting history. Managers can export, import, inspect, and restore revisions from **Settings → Data and history**. + +D1 owns independent migrations in [../../plugins/repository/d1/migrations](../../plugins/repository/d1/migrations). Apply them deliberately to the bound database before opening setup. The current Vercel deployment remains on PostgreSQL because Vercel does not provide a native D1 binding. The `seed` command remains available for controlled non-interactive migrations, but it is not part of the normal deployment flow: @@ -72,7 +74,7 @@ The `seed` command remains available for controlled non-interactive migrations, pnpm --filter @i0c/plugin-data-repository-postgres seed -- --config --redirects ``` -For database-backed documents, also select the HTTP Runtime Source in the same bootstrap configuration and point it at `https:///api/runtime/snapshot`. The public endpoint returns one validated config-and-rules revision with an ETag; it contains no Secret values. Edge Runtime deployments fetch this endpoint and never receive the PostgreSQL connection string. +For database-backed documents, also select the HTTP Runtime Source in the same bootstrap configuration and point it at `https:///api/runtime/snapshot`. The public endpoint returns one validated config-and-rules revision with an ETag; it contains no Secret values. Edge Runtime deployments fetch this endpoint and never receive the database connection or binding. GitHub Contents and GitHub Raw remain in the workspace as archived build-time alternatives. Re-enabling them requires deliberately changing the bootstrap Repository and Runtime Source selections, restoring the required OAuth Repository scope, and rebuilding both applications. They are not part of the default first-run flow. @@ -140,10 +142,10 @@ The WebUI does not read former non-sensitive environment variables as overrides - Versioned authenticated, numeric-ID allowlist, or GitHub-wide read-only access with configured managers and optional blocked users. - Visual editing of `redirects.json`: group tree management + rule form editing. -- Rules JSON editor: line numbers, current line highlighting, JSON syntax validation (error prompts for formatting issues). +- GitHub Repository-only rules source override and JSON editor with line highlighting and syntax validation. - Visual, validated `config.json` settings with a raw recovery editor only when the current document cannot be represented safely. - First-run database initialization without hand-written JSON or a seed command. -- Immutable document history, non-destructive rollback, and atomic JSON backup import/export. +- Immutable document history with Git-style line diffs, non-destructive rollback, and atomic JSON backup import/export. - Authenticated plugin status reporting for installed manifests, configuration state, capabilities, missing bindings, and selected-Store health. - Form behavior aligned with the schema (specification source: [https://raw.githubusercontent.com/Revaea/i0c.cc/main/packages/config/redirects.schema.json](https://raw.githubusercontent.com/Revaea/i0c.cc/main/packages/config/redirects.schema.json)). - Supports undo/redo for quick editing rollback. diff --git a/apps/webui/README.zh-CN.md b/apps/webui/README.zh-CN.md index 7b9491f..f62455e 100644 --- a/apps/webui/README.zh-CN.md +++ b/apps/webui/README.zh-CN.md @@ -6,10 +6,10 @@ i0c.cc WebUI 是一个基于 Next.js 16 的管理面板，用于通过 GitHub OA 服务端 Data Repository 与 Analytics Store 工厂通过 [../../i0c.webui.config.ts](../../i0c.webui.config.ts) 在构建期安装。客户端安全的 UI Renderer 使用 [webui.extensions.ts](webui.extensions.ts)，确保它们留在客户端 Bundle。workspace fixture 会覆盖两条安装链，无需在 WebUI 宿主源码中增加工厂映射；生产 Renderer 清单目前有意保持为空。 -该项目提供两种规则编辑方式和独立的设置界面： +该项目默认提供可视化规则编辑和独立的设置界面： - 可视化规则编辑（分组树 + 表单） -- 规则 JSON 编辑（右侧面板，可直接编辑 `redirects.json`） +- 重新启用 GitHub Repository 时提供 Git 专属的规则来源切换与 JSON 编辑 - 位于侧边栏底部的可视化实例设置（`config.json`，使用共享契约校验） - 数据库备份导入导出与不可变版本历史 @@ -60,9 +60,11 @@ i0c.cc WebUI 是一个基于 Next.js 16 的管理面板，用于通过 GitHub OA ## Data Repository -仓库当前通过 [../../packages/config/src/defaults.ts](../../packages/config/src/defaults.ts) 选择 PostgreSQL，并使用 `DATABASE_URL`。 +仓库当前通过 [../../packages/config/src/defaults.ts](../../packages/config/src/defaults.ts) 选择 PostgreSQL，并使用 `DATABASE_URL`。支持 D1 的 WebUI 宿主可以选择 `provider: "d1"`，并在首次使用 Repository 前通过 `configureAppDataRepositoryBinding` 注入 `D1Database` binding。 -首次初始化会在一个事务中创建两份文档，并拒绝只存在其中一份文档的半初始化数据库。每次保存都会创建不可变版本；导入会先校验两份 JSON，再原子替换；恢复则把旧内容复制为新的活动版本，不会改写历史。管理者可以在 **设置 → 数据与历史** 中导出、导入、查看和恢复版本。 +PostgreSQL 与 D1 由同一套共享行为契约约束。首次初始化会原子创建两份文档，并拒绝只存在其中一份文档的半初始化数据库。每次保存都会创建不可变版本；导入会先校验两份 JSON，再原子替换；恢复则把旧内容复制为新的活动版本，不会改写历史。管理者可以在 **设置 → 数据与历史** 中导出、导入、查看和恢复版本。 + +D1 使用 [../../plugins/repository/d1/migrations](../../plugins/repository/d1/migrations) 中的独立迁移。打开初始化页面前，需要明确将这些迁移应用到绑定的数据库。当前 Vercel 部署仍使用 PostgreSQL，因为 Vercel 不提供原生 D1 binding。 `seed` 命令继续用于受控的非交互迁移，但不再属于正常部署流程： @@ -70,7 +72,7 @@ i0c.cc WebUI 是一个基于 Next.js 16 的管理面板，用于通过 GitHub OA pnpm --filter @i0c/plugin-data-repository-postgres seed -- --config --redirects ``` -使用数据库文档时，还要在同一份启动配置中选择 HTTP Runtime Source，并指向 `https:///api/runtime/snapshot`。这个公开端点会返回一份带 ETag、经过校验的配置与规则 revision，且不包含 Secret 值。边缘 Runtime 只读取该端点，不会获得 PostgreSQL 连接串。 +使用数据库文档时，还要在同一份启动配置中选择 HTTP Runtime Source，并指向 `https:///api/runtime/snapshot`。这个公开端点会返回一份带 ETag、经过校验的配置与规则 revision，且不包含 Secret 值。边缘 Runtime 只读取该端点，不会获得数据库连接信息或 binding。 GitHub Contents 与 GitHub Raw 继续保留在 workspace 中，作为归档的构建期替代方案。重新启用时，需要主动修改启动 Repository 与 Runtime Source 选择、恢复对应的 OAuth Repository scope，并重新构建两个应用；它们不属于默认首次初始化流程。 @@ -134,10 +136,10 @@ WebUI 不会把原有非敏感环境变量作为覆盖值或回退值读取。Ve - 通过版本化配置选择任意已登录用户、数字用户 ID 白名单或带指定管理员与可选黑名单的 GitHub 全员只读模式。 - 可视化编辑 `redirects.json`：分组树管理 + 规则表单编辑。 -- 规则 JSON 编辑器：行号、当前行高亮、JSON 语法校验（格式错误提示）。 +- GitHub Repository 专属的规则来源切换和 JSON 编辑器，支持当前行高亮与语法校验。 - 可视化并校验 `config.json`；只有当前文档无法安全转换为表单时，才显示原始内容恢复编辑器。 - 数据库首次初始化，无需手写 JSON 或执行 seed。 -- 不可变文档历史、非破坏性回滚，以及原子 JSON 备份导入导出。 +- 带 Git 风格行差异的不可变文档历史、非破坏性回滚，以及原子 JSON 备份导入导出。 - 通过认证后查看已安装 Manifest、配置状态、能力、缺失绑定和所选 Store 健康状态。 - 表单行为对齐 Schema（规范来源：[https://raw.githubusercontent.com/Revaea/i0c.cc/main/packages/config/redirects.schema.json](https://raw.githubusercontent.com/Revaea/i0c.cc/main/packages/config/redirects.schema.json)）。 - 支持撤销/重做，便于快速回退编辑。 diff --git a/apps/webui/messages/en/redirects.json b/apps/webui/messages/en/redirects.json index f2dc0d4..25f6aa1 100644 --- a/apps/webui/messages/en/redirects.json +++ b/apps/webui/messages/en/redirects.json @@ -6,7 +6,7 @@ "rootTitle": "Root (rules written directly under slots)", "empty": "No groups yet", "readOnlyTitle": "Public read-only view", - "readOnlyDescription": "Rules and settings can be inspected, but editing and custom rules JSON sources are disabled.", + "readOnlyDescription": "Rules and settings can be inspected, but only managers can save changes.", "cannotLoad": "Failed to load groups", "group": "Group", "selectHint": "Select a group on the right", @@ -189,7 +189,7 @@ }, "dataManagement": { "title": "Data management", - "description": "Back up, import, inspect, and restore the instance configuration and redirect rules stored in PostgreSQL.", + "description": "Back up, import, inspect, and restore the instance configuration and redirect rules stored in the selected data repository.", "managerOnly": "Only managers can export, import, or inspect stored revisions.", "unsavedWarning": "Save or discard the current rule and settings changes before importing or restoring data.", "cancel": "Cancel", @@ -218,7 +218,7 @@ "redirects": "redirects.json", "chooseFile": "Choose file", "noFile": "No file selected", - "action": "Import both files", + "action": "Confirm import", "confirmTitle": "Import this backup?", "confirmDescription": "The current configuration and rules will be preserved in history, then both uploaded files will become the new active revision.", "confirmAction": "Import backup" @@ -231,12 +231,23 @@ "current": "Current", "actor": "GitHub user {actor}", "checksum": "Checksum {checksum}", - "view": "View JSON", + "view": "View changes", "restore": "Restore", "confirmTitle": "Restore this revision?", "confirmDescription": "Revision {revision} will be copied into a new active revision. Existing history will not be changed.", "confirmAction": "Restore revision", - "previewTitle": "Revision {revision}" + "previewTitle": "Revision {revision}", + "previewLoading": "Loading revision changes", + "previewError": "Could not load this revision.", + "views": { + "changes": "Changes", + "json": "Full JSON" + }, + "comparedWith": "Compared with revision {revision}", + "initialRevision": "Initial revision", + "noChanges": "No line changes were found.", + "diffLabel": "Revision line changes", + "omittedLines": "{count, plural, one {# unchanged line hidden} other {# unchanged lines hidden}}" }, "errors": { "fileTooLarge": "Each JSON file must be no larger than 1 MB.", diff --git a/apps/webui/messages/en/shared.json b/apps/webui/messages/en/shared.json index a70baba..ce81125 100644 --- a/apps/webui/messages/en/shared.json +++ b/apps/webui/messages/en/shared.json @@ -13,7 +13,7 @@ }, "setup": { "title": "Initialize this deployment", - "description": "Create the first instance configuration and an empty rule set in PostgreSQL. Your signed-in GitHub account becomes the first manager.", + "description": "Create the first instance configuration and an empty rule set in the selected data repository. Your signed-in GitHub account becomes the first manager.", "secret": "Instance secret", "secretHelp": "Enter the shared I0C_SECRET configured in this WebUI and every Runtime deployment.", "webUiOrigin": "WebUI origin", @@ -32,7 +32,8 @@ "initializing": "Initializing...", "unknownError": "Initialization failed. Check the values and try again.", "migrationRequired": "Database migration required", - "migrationRequiredHelp": "Apply the PostgreSQL data repository migrations before opening setup again.", + "migrationRequiredHelp": "Apply the selected data repository migrations before opening setup again.", + "migrationRequiredD1Detail": "Apply plugins/repository/d1/migrations/*.sql to the bound D1 database.", "partial": "Partial initialization detected", "partialHelp": "Only one required data document exists. Setup will not overwrite it; repair or clear the partial state explicitly.", "unsupported": "Setup is unavailable", diff --git a/apps/webui/messages/zh-CN/redirects.json b/apps/webui/messages/zh-CN/redirects.json index 17c6232..35b4771 100644 --- a/apps/webui/messages/zh-CN/redirects.json +++ b/apps/webui/messages/zh-CN/redirects.json @@ -6,7 +6,7 @@ "rootTitle": "根目录（直接写在 slots 下的规则）", "empty": "暂无分组", "readOnlyTitle": "公开只读视图", - "readOnlyDescription": "可以查看规则和设置，但不能编辑或加载自定义规则 JSON 地址。", + "readOnlyDescription": "可以查看规则和设置，但只有管理员可以保存更改。", "cannotLoad": "无法加载分组", "group": "分组", "selectHint": "从右侧选择分组", @@ -189,7 +189,7 @@ }, "dataManagement": { "title": "数据管理", - "description": "备份、导入、查看和恢复存储在 PostgreSQL 中的实例配置与重定向规则。", + "description": "备份、导入、查看和恢复存储在当前数据 Repository 中的实例配置与重定向规则。", "managerOnly": "只有管理者可以导出、导入或查看存储的历史版本。", "unsavedWarning": "导入或恢复数据前，请先保存或放弃当前的规则和设置改动。", "cancel": "取消", @@ -218,7 +218,7 @@ "redirects": "redirects.json", "chooseFile": "选择文件", "noFile": "尚未选择文件", - "action": "导入两份文件", + "action": "确认导入", "confirmTitle": "导入此备份？", "confirmDescription": "当前配置和规则会保留在历史记录中，随后两份上传文件将成为新的活动版本。", "confirmAction": "导入备份" @@ -231,12 +231,23 @@ "current": "当前", "actor": "GitHub 用户 {actor}", "checksum": "校验和 {checksum}", - "view": "查看 JSON", + "view": "查看改动", "restore": "恢复", "confirmTitle": "恢复此版本？", "confirmDescription": "版本 {revision} 会被复制为新的活动版本，现有历史记录不会被修改。", "confirmAction": "恢复版本", - "previewTitle": "版本 {revision}" + "previewTitle": "版本 {revision}", + "previewLoading": "正在加载版本改动", + "previewError": "无法加载此版本。", + "views": { + "changes": "改动", + "json": "完整 JSON" + }, + "comparedWith": "与版本 {revision} 对比", + "initialRevision": "初始版本", + "noChanges": "未发现行内容变化。", + "diffLabel": "版本行内容变化", + "omittedLines": "已隐藏 {count} 行未改动内容" }, "errors": { "fileTooLarge": "每份 JSON 文件不能超过 1 MB。", diff --git a/apps/webui/messages/zh-CN/shared.json b/apps/webui/messages/zh-CN/shared.json index 26b037c..6272ae4 100644 --- a/apps/webui/messages/zh-CN/shared.json +++ b/apps/webui/messages/zh-CN/shared.json @@ -13,7 +13,7 @@ }, "setup": { "title": "初始化此部署", - "description": "在 PostgreSQL 中创建第一份实例配置和空规则集。当前登录的 GitHub 账号将成为首位管理员。", + "description": "在当前选择的数据 Repository 中创建第一份实例配置和空规则集。当前登录的 GitHub 账号将成为首位管理员。", "secret": "实例密钥", "secretHelp": "输入 WebUI 与所有 Runtime 部署共用的 I0C_SECRET。", "webUiOrigin": "WebUI 地址", @@ -32,7 +32,8 @@ "initializing": "正在初始化……", "unknownError": "初始化失败，请检查填写内容后重试。", "migrationRequired": "需要执行数据库迁移", - "migrationRequiredHelp": "请先应用 PostgreSQL 数据 Repository 迁移，然后重新打开初始化页面。", + "migrationRequiredHelp": "请先应用当前数据 Repository 的迁移，然后重新打开初始化页面。", + "migrationRequiredD1Detail": "请将 plugins/repository/d1/migrations/*.sql 应用到绑定的 D1 数据库。", "partial": "检测到不完整初始化", "partialHelp": "当前只存在一份必要数据文档。初始化不会覆盖它，请明确修复或清理不完整状态。", "unsupported": "无法进行初始化", diff --git a/apps/webui/package.json b/apps/webui/package.json index 8d00b21..3c2c0ea 100644 --- a/apps/webui/package.json +++ b/apps/webui/package.json @@ -23,6 +23,7 @@ "@i0c/analytics-domain": "workspace:*", "@i0c/config": "workspace:*", "@i0c/plugin-catalog": "workspace:*", + "@i0c/plugin-data-repository-d1": "workspace:*", "@i0c/plugin-data-repository-postgres": "workspace:*", "@i0c/plugin-github-data": "workspace:*", "@i0c/plugin-api": "workspace:*", diff --git a/apps/webui/src/app/[locale]/setup/page.tsx b/apps/webui/src/app/[locale]/setup/page.tsx index ec8b7d6..9a2bf8d 100644 --- a/apps/webui/src/app/[locale]/setup/page.tsx +++ b/apps/webui/src/app/[locale]/setup/page.tsx @@ -3,6 +3,8 @@ import { getTranslations } from "next-intl/server"; import { headers } from "next/headers"; import { redirect } from "next/navigation"; +import { bootstrapConfig } from "@i0c/config"; + import { authOptions } from "@/auth/config"; import { SetupForm } from "@/components/setup/setup-form"; import { LanguageSwitcher } from "@/components/ui/controls/language-switcher"; @@ -30,7 +32,11 @@ export default async function SetupPage({ params }: SetupPageProps) { ); } diff --git a/apps/webui/src/components/editor/right-panel.tsx b/apps/webui/src/components/editor/right-panel.tsx index c721c89..b115d79 100644 --- a/apps/webui/src/components/editor/right-panel.tsx +++ b/apps/webui/src/components/editor/right-panel.tsx @@ -22,6 +22,9 @@ export type RightPanelProps = { onJsonDraftChange: (value: string) => void; jsonError: string | null; isReadOnly: boolean; + showSaveAction: boolean; + supportsJsonEditor: boolean; + supportsSourceOverride: boolean; onLoadSourceUrl: (url: string) => Promise; onUndo: () => void; rulesContent: ReactNode; @@ -42,6 +45,9 @@ export function RightPanel({ onJsonDraftChange, jsonError, isReadOnly, + showSaveAction, + supportsJsonEditor, + supportsSourceOverride, onLoadSourceUrl, onUndo, rulesContent, @@ -58,7 +64,7 @@ export function RightPanel({
{editorMode === "settings" ? (

{tConfig("title")}

- ) : ( + ) : supportsJsonEditor ? (
+ ) : ( +

{t("rules")}

)} {isReadOnly ? null : (
{showRuleActions ? ( <> - + {supportsSourceOverride ? ( + + ) : null} + {showSaveAction ? ( + + ) : null}
)}
diff --git a/apps/webui/src/components/redirects-groups/index.tsx b/apps/webui/src/components/redirects-groups/index.tsx index 48eb3bb..faf0e88 100644 --- a/apps/webui/src/components/redirects-groups/index.tsx +++ b/apps/webui/src/components/redirects-groups/index.tsx @@ -34,6 +34,11 @@ import { useDataConfigFile } from "@/composables/data-config/use-data-config-fil import { RouteEntriesCatalog } from "@/components/redirects-groups/manager-sidebar/manager-sidebar-catalog"; import { RuntimeSettingsProvider } from "@/components/redirects-groups/runtime-settings-context"; import { InstanceSettingsEditor } from "@/components/settings/instance-settings-editor"; +import { + shouldConfirmSettingsCategoryChange, + shouldShowSettingsSaveAction, + type SettingsCategory, +} from "@/components/settings/settings-navigation"; import { validateInstanceDataConfig } from "@/lib/configuration/validation"; import { useRouter } from "@/i18n/navigation"; @@ -42,6 +47,8 @@ import { ManagerSidebarBody } from "./manager-sidebar/manager-sidebar-body"; interface RedirectsGroupsManagerProps { initialView?: "rules" | "settings"; isReadOnly?: boolean; + supportsJsonEditor?: boolean; + supportsSourceOverride?: boolean; } interface PendingLeave { @@ -52,6 +59,8 @@ interface PendingLeave { export function RedirectsGroupsManager({ initialView = "rules", isReadOnly = false, + supportsJsonEditor = false, + supportsSourceOverride = false, }: RedirectsGroupsManagerProps) { const tGroups = useTranslations("groups"); const tEntries = useTranslations("entries"); @@ -112,6 +121,8 @@ export function RedirectsGroupsManager({ const [configError, setConfigError] = useState(null); const [isConfigLoaded, setIsConfigLoaded] = useState(false); const [isConfigLoading, setIsConfigLoading] = useState(false); + const [settingsCategory, setSettingsCategory] = + useState("runtime"); const [lastSaveTarget, setLastSaveTarget] = useState<"rules" | "settings">("rules"); const [localSaveError, setLocalSaveError] = useState(null); const [saveAttempt, setSaveAttempt] = useState(0); @@ -319,6 +330,23 @@ export function RedirectsGroupsManager({ enterSettingsMode(); }, [editorMode, enterSettingsMode, isRulesDirty]); + const handleSettingsCategoryChange = useCallback( + (category: SettingsCategory) => { + if (category === settingsCategory) { + return; + } + if (shouldConfirmSettingsCategoryChange(category, isSettingsDirty)) { + setPendingLeave({ + kind: "settings", + proceed: () => setSettingsCategory(category), + }); + return; + } + setSettingsCategory(category); + }, + [isSettingsDirty, settingsCategory], + ); + const handleOpenRulesSection = useCallback(() => { runRulesAction(() => { enterRulesMode(); @@ -679,6 +707,12 @@ export function RedirectsGroupsManager({ onJsonDraftChange={setJsonDraft} jsonError={jsonError} isReadOnly={isReadOnly} + showSaveAction={ + editorMode !== "settings" + || shouldShowSettingsSaveAction(settingsCategory) + } + supportsJsonEditor={supportsJsonEditor} + supportsSourceOverride={supportsSourceOverride} sourceUrl={configSourceUrl} onLoadSourceUrl={loadFromUrl} settingsContent={ @@ -690,11 +724,13 @@ export function RedirectsGroupsManager({ hasUnsavedChanges={isRulesDirty || isSettingsDirty} value={configValue} isReadOnly={isReadOnly} + onCategoryChange={handleSettingsCategoryChange} onChange={(nextConfig) => { setConfigValue(nextConfig); setConfigDraft(JSON.stringify(nextConfig, null, 2)); setConfigError(null); }} + selectedCategory={settingsCategory} /> ) : ( diff --git a/apps/webui/src/components/redirects-groups/redirects-groups-page.tsx b/apps/webui/src/components/redirects-groups/redirects-groups-page.tsx index b77d02b..2d95dfa 100644 --- a/apps/webui/src/components/redirects-groups/redirects-groups-page.tsx +++ b/apps/webui/src/components/redirects-groups/redirects-groups-page.tsx @@ -1,4 +1,7 @@ +import { webUiPluginInstallations } from "@i0c/webui-config"; + import { RedirectsGroupsManager } from "@/components/redirects-groups"; +import { resolveDataRepositoryEditorCapabilities } from "@/lib/data/editor-capabilities"; interface RedirectsGroupsPageProps { initialView?: "rules" | "settings"; @@ -9,10 +12,16 @@ export function RedirectsGroupsPage({ initialView = "rules", isReadOnly = false, }: RedirectsGroupsPageProps) { + const editorCapabilities = resolveDataRepositoryEditorCapabilities( + webUiPluginInstallations.dataRepository.manifest, + ); + return ( ); } diff --git a/apps/webui/src/components/settings/data-management-panel.tsx b/apps/webui/src/components/settings/data-management-panel.tsx index 308debd..4ef9f26 100644 --- a/apps/webui/src/components/settings/data-management-panel.tsx +++ b/apps/webui/src/components/settings/data-management-panel.tsx @@ -3,6 +3,7 @@ import { useEffect, useMemo, + useRef, useState, type ChangeEvent, type ReactNode, @@ -26,6 +27,8 @@ import { SkeletonPulse, } from "@/components/ui/feedback/skeletons"; +import { RevisionDiff } from "./revision-diff"; + interface DataManagementPanelProps { hasUnsavedChanges: boolean; isReadOnly: boolean; @@ -44,6 +47,17 @@ interface PendingRestore { revision: string; } +interface LoadedRevisionPreview { + current: DataDocumentRevision; + previous: DataDocumentRevision | null; +} + +interface RevisionPreview { + error: string | null; + loaded: LoadedRevisionPreview | null; + target: DataDocumentRevisionSummary; +} + const maximumDocumentSize = 1_000_000; export function DataManagementPanel({ @@ -65,7 +79,8 @@ export function DataManagementPanel({ useState(null); const [isImportConfirmationOpen, setIsImportConfirmationOpen] = useState(false); - const [preview, setPreview] = useState(null); + const [preview, setPreview] = useState(null); + const previewControllerRef = useRef(null); const dateFormatter = useMemo( () => new Intl.DateTimeFormat(locale, { dateStyle: "medium", @@ -102,6 +117,10 @@ export function DataManagementPanel({ }; }, [isReadOnly, selectedKind]); + useEffect(() => () => { + previewControllerRef.current?.abort(); + }, []); + function handleFileChange( event: ChangeEvent, kind: DataDocumentKind, @@ -138,21 +157,74 @@ export function DataManagementPanel({ async function handlePreview( revision: DataDocumentRevisionSummary, ) { + previewControllerRef.current?.abort(); + const controller = new AbortController(); + previewControllerRef.current = controller; setError(null); + setPreview({ + error: null, + loaded: null, + target: revision, + }); try { - const response = await fetch( - `/api/data/history/${revision.kind}/${revision.revision}`, - { - cache: "no-store", - }, + const currentRevisionPromise = fetchRevision( + revision.kind, + revision.revision, + controller.signal, + ); + const revisionIndex = revisions.findIndex( + (candidate) => candidate.revision === revision.revision, ); - const data = await readJson(response); - setPreview(data.revision); + let previousSummary = revisionIndex >= 0 + ? revisions[revisionIndex + 1] + : undefined; + if (!previousSummary) { + previousSummary = ( + await fetchHistory( + revision.kind, + controller.signal, + revision.revision, + ) + )[0]; + } + const [current, previous] = await Promise.all([ + currentRevisionPromise, + previousSummary + ? fetchRevision( + previousSummary.kind, + previousSummary.revision, + controller.signal, + ) + : Promise.resolve(null), + ]); + if (!controller.signal.aborted) { + setPreview({ + error: null, + loaded: { current, previous }, + target: revision, + }); + } } catch (caughtError) { - setError(resolveErrorMessage(caughtError)); + if (!controller.signal.aborted) { + setPreview({ + error: resolveErrorMessage(caughtError), + loaded: null, + target: revision, + }); + } + } finally { + if (previewControllerRef.current === controller) { + previewControllerRef.current = null; + } } } + function closePreview() { + previewControllerRef.current?.abort(); + previewControllerRef.current = null; + setPreview(null); + } + async function handleRestore() { if (!pendingRestore || hasUnsavedChanges || isMutating) { return; @@ -445,7 +517,7 @@ export function DataManagementPanel({ setPreview(null)} + onClose={closePreview} widthClassName="max-w-4xl" >
@@ -453,20 +525,64 @@ export function DataManagementPanel({

{t("history.previewTitle", { - revision: preview?.revision ?? "", + revision: preview?.target.revision ?? "", })}

- {preview ? t(`kinds.${preview.kind}`) : ""} + {preview ? t(`kinds.${preview.target.kind}`) : ""}

-
- 
-            {preview?.content}
-
+ {preview?.loaded ? ( + + ) : preview?.error ? ( +
+

+ {t("history.previewError")} +

+

{preview.error}

+
+ ) : ( + +
+ + +
+
+ {Array.from({ length: 9 }, (_, index) => ( +
+ +
+
+ +
+
+ ))} +
+ + )}
@@ -556,8 +672,14 @@ function HistorySkeleton() { async function fetchHistory( kind: DataDocumentKind, signal?: AbortSignal, + beforeRevision?: string, ): Promise { - const response = await fetch(`/api/data/history/${kind}`, { + const searchParams = new URLSearchParams(); + if (beforeRevision) { + searchParams.set("beforeRevision", beforeRevision); + } + const query = searchParams.size > 0 ? `?${searchParams}` : ""; + const response = await fetch(`/api/data/history/${kind}${query}`, { cache: "no-store", signal, }); @@ -565,6 +687,21 @@ async function fetchHistory( return data.revisions; } +async function fetchRevision( + kind: DataDocumentKind, + revision: string, + signal?: AbortSignal, +): Promise { + const response = await fetch( + `/api/data/history/${kind}/${revision}`, + { + cache: "no-store", + signal, + }, + ); + return (await readJson(response)).revision; +} + async function readJson(response: Response): Promise { const data = await response.json().catch(() => null) as | { error?: unknown } diff --git a/apps/webui/src/components/settings/instance-settings-editor.tsx b/apps/webui/src/components/settings/instance-settings-editor.tsx index e27e0da..2dd46a2 100644 --- a/apps/webui/src/components/settings/instance-settings-editor.tsx +++ b/apps/webui/src/components/settings/instance-settings-editor.tsx @@ -1,6 +1,6 @@ "use client"; -import { useMemo, useState, type ReactNode } from "react"; +import { useMemo, type ReactNode } from "react"; import { useTranslations } from "next-intl"; import type { DataConfig, RobotsPolicy, WebUiAccessMode } from "@i0c/config"; @@ -17,10 +17,14 @@ import { import { NumberInput } from "@/components/ui/controls/number-input"; import { validateInstanceDataConfig } from "@/lib/configuration/validation"; +import type { SettingsCategory } from "./settings-navigation"; + interface InstanceSettingsEditorProps { hasUnsavedChanges: boolean; isReadOnly: boolean; onChange: (value: DataConfig) => void; + onCategoryChange: (category: SettingsCategory) => void; + selectedCategory: SettingsCategory; value: DataConfig; } @@ -30,22 +34,15 @@ interface SettingsFieldProps { label: string; } -type SettingsCategory = - | "runtime" - | "analytics" - | "access" - | "installed-plugins" - | "data"; - export function InstanceSettingsEditor({ hasUnsavedChanges, isReadOnly, onChange, + onCategoryChange, + selectedCategory, value, }: InstanceSettingsEditorProps) { const t = useTranslations("instanceConfig"); - const [selectedCategory, setSelectedCategory] = - useState("runtime"); const validation = useMemo(() => validateInstanceDataConfig(value), [value]); const issues = validation.status === "invalid" ? validation.issues : []; const validationFieldLabels: Readonly> = { @@ -210,31 +207,31 @@ export function InstanceSettingsEditor({
setSelectedCategory("runtime")} + onClick={() => onCategoryChange("runtime")} > {t("sections.runtime.title")} setSelectedCategory("analytics")} + onClick={() => onCategoryChange("analytics")} > {t("sections.analytics.title")} setSelectedCategory("access")} + onClick={() => onCategoryChange("access")} > {t("sections.access.title")} setSelectedCategory("installed-plugins")} + onClick={() => onCategoryChange("installed-plugins")} > {t("sections.installedPlugins.title")} setSelectedCategory("data")} + onClick={() => onCategoryChange("data")} > {t("sections.data.title")} diff --git a/apps/webui/src/components/settings/revision-diff-table-row.tsx b/apps/webui/src/components/settings/revision-diff-table-row.tsx new file mode 100644 index 0000000..6c59aed --- /dev/null +++ b/apps/webui/src/components/settings/revision-diff-table-row.tsx @@ -0,0 +1,61 @@ +import type { LineDiffRow } from "@/lib/data/line-diff"; + +interface RevisionDiffTableRowProps { + omittedLabel: string; + row: LineDiffRow; + showOldLineNumbers: boolean; +} + +export function RevisionDiffTableRow({ + omittedLabel, + row, + showOldLineNumbers, +}: RevisionDiffTableRowProps) { + if (row.kind === "omission") { + return ( + + + {omittedLabel} + + + ); + } + + const marker = row.kind === "addition" + ? "+" + : row.kind === "deletion" + ? "-" + : " "; + const changeClassName = row.kind === "addition" + ? "bg-emerald-50 text-emerald-950" + : row.kind === "deletion" + ? "bg-rose-50 text-rose-950" + : "text-ink"; + const markerClassName = row.kind === "addition" + ? "text-emerald-700" + : row.kind === "deletion" + ? "text-rose-700" + : "text-muted"; + + return ( + + {showOldLineNumbers ? ( + + {row.oldLineNumber ?? ""} + + ) : null} + + {row.newLineNumber ?? ""} + + + {marker} + + + {row.content || " "} + + + ); +} diff --git a/apps/webui/src/components/settings/revision-diff.tsx b/apps/webui/src/components/settings/revision-diff.tsx new file mode 100644 index 0000000..b6576ae --- /dev/null +++ b/apps/webui/src/components/settings/revision-diff.tsx @@ -0,0 +1,129 @@ +"use client"; + +import { useMemo, useState } from "react"; +import { useTranslations } from "next-intl"; + +import type { DataDocumentRevision } from "@i0c/config"; + +import { createLineDiff } from "@/lib/data/line-diff"; + +import { RevisionDiffTableRow } from "./revision-diff-table-row"; + +interface RevisionDiffProps { + previousRevision: DataDocumentRevision | null; + revision: DataDocumentRevision; +} + +type RevisionView = "changes" | "json"; + +export function RevisionDiff({ + previousRevision, + revision, +}: RevisionDiffProps) { + const t = useTranslations("instanceConfig.dataManagement.history"); + const [view, setView] = useState("changes"); + const diff = useMemo( + () => createLineDiff(previousRevision?.content ?? null, revision.content), + [previousRevision?.content, revision.content], + ); + const revisionLines = useMemo( + () => revision.content.replace(/\r\n?/g, "\n").split("\n"), + [revision.content], + ); + + return ( +
+
+
+ {(["changes", "json"] as const).map((candidate) => ( + + ))} +
+ + {view === "changes" ? ( +
+ + {previousRevision + ? t("comparedWith", { + revision: previousRevision.revision, + }) + : t("initialRevision")} + + + +{diff.additions} + + + -{diff.deletions} + +
+ ) : null} +
+ + {view === "json" ? ( +
+ + + {revisionLines.map((line, index) => ( + + + + + ))} + +
+ {index + 1} + + {line || " "} +
+
+ ) : diff.additions === 0 && diff.deletions === 0 ? ( +
+ {t("noChanges")} +
+ ) : ( +
+ + + {diff.rows.map((row, index) => ( + + ))} + +
+
+ )} +
+ ); +} + +function createRowKey( + row: ReturnType["rows"][number], + index: number, +): string { + return row.kind === "omission" + ? `omission:${index}` + : `${row.kind}:${row.oldLineNumber ?? ""}:${row.newLineNumber ?? ""}`; +} diff --git a/apps/webui/src/components/settings/settings-navigation.ts b/apps/webui/src/components/settings/settings-navigation.ts new file mode 100644 index 0000000..27ebdba --- /dev/null +++ b/apps/webui/src/components/settings/settings-navigation.ts @@ -0,0 +1,19 @@ +export type SettingsCategory = + | "runtime" + | "analytics" + | "access" + | "installed-plugins" + | "data"; + +export function shouldConfirmSettingsCategoryChange( + category: SettingsCategory, + hasUnsavedChanges: boolean, +): boolean { + return category === "data" && hasUnsavedChanges; +} + +export function shouldShowSettingsSaveAction( + category: SettingsCategory, +): boolean { + return category !== "data"; +} diff --git a/apps/webui/src/lib/data/documents.ts b/apps/webui/src/lib/data/documents.ts index 1d9d8f8..27dff10 100644 --- a/apps/webui/src/lib/data/documents.ts +++ b/apps/webui/src/lib/data/documents.ts @@ -17,6 +17,15 @@ export type DataDocumentUpdateInput = Omit< >; let appDataRepositoryPromise: Promise | null = null; +const appDataRepositoryBindings = new Map(); + +export function configureAppDataRepositoryBinding( + pluginId: string, + binding: unknown, +): void { + appDataRepositoryBindings.set(pluginId, binding); + appDataRepositoryPromise = null; +} export async function getAppDataSnapshot() { const repository = await getAppDataRepository(); @@ -74,7 +83,10 @@ export async function getAppDataRepositoryManagement(): Promise< function getAppDataRepository(): Promise { appDataRepositoryPromise ??= Promise.resolve( - webUiPluginInstallations.dataRepository.create(), + webUiPluginInstallations.dataRepository.create({ + bindings: appDataRepositoryBindings, + readEnvironment: (name) => process.env[name], + }), ).catch((error: unknown) => { appDataRepositoryPromise = null; throw error; diff --git a/apps/webui/src/lib/data/editor-capabilities.ts b/apps/webui/src/lib/data/editor-capabilities.ts new file mode 100644 index 0000000..c05a692 --- /dev/null +++ b/apps/webui/src/lib/data/editor-capabilities.ts @@ -0,0 +1,19 @@ +import type { PluginManifest } from "@i0c/plugin-api"; + +interface DataRepositoryEditorCapabilities { + supportsJsonEditor: boolean; + supportsSourceOverride: boolean; +} + +export function resolveDataRepositoryEditorCapabilities( + manifest: Pick, +): DataRepositoryEditorCapabilities { + return { + supportsJsonEditor: manifest.capabilities.includes( + "ui:redirects-json-editor", + ), + supportsSourceOverride: manifest.capabilities.includes( + "ui:redirects-source-override", + ), + }; +} diff --git a/apps/webui/src/lib/data/line-diff.ts b/apps/webui/src/lib/data/line-diff.ts new file mode 100644 index 0000000..4ea5e79 --- /dev/null +++ b/apps/webui/src/lib/data/line-diff.ts @@ -0,0 +1,281 @@ +export type LineDiffRow = + | { + content: string; + kind: "addition" | "context" | "deletion"; + newLineNumber: number | null; + oldLineNumber: number | null; + } + | { + count: number; + kind: "omission"; + }; + +export interface LineDiffResult { + additions: number; + deletions: number; + rows: readonly LineDiffRow[]; +} + +interface RawLineDiffRow { + content: string; + kind: "addition" | "context" | "deletion"; + newLineNumber: number | null; + oldLineNumber: number | null; +} + +const maximumLcsMatrixCells = 1_000_000; +const defaultContextLineCount = 3; + +export function createLineDiff( + previousContent: string | null, + currentContent: string, + contextLineCount = defaultContextLineCount, +): LineDiffResult { + if (!Number.isInteger(contextLineCount) || contextLineCount < 0) { + throw new TypeError("Diff context line count must be a non-negative integer"); + } + + const previousLines = splitLines(previousContent ?? ""); + const currentLines = splitLines(currentContent); + const rawRows = createRawRows(previousLines, currentLines); + + return { + additions: rawRows.filter((row) => row.kind === "addition").length, + deletions: rawRows.filter((row) => row.kind === "deletion").length, + rows: collapseContextRows(rawRows, contextLineCount), + }; +} + +function createRawRows( + previousLines: readonly string[], + currentLines: readonly string[], +): readonly RawLineDiffRow[] { + const prefixLength = findCommonPrefixLength(previousLines, currentLines); + const suffixLength = findCommonSuffixLength( + previousLines, + currentLines, + prefixLength, + ); + const previousMiddle = previousLines.slice( + prefixLength, + previousLines.length - suffixLength, + ); + const currentMiddle = currentLines.slice( + prefixLength, + currentLines.length - suffixLength, + ); + const middleOperations = createMiddleOperations( + previousMiddle, + currentMiddle, + ); + const operations = [ + ...previousLines.slice(0, prefixLength).map((content) => ({ + content, + kind: "context" as const, + })), + ...middleOperations, + ...previousLines.slice(previousLines.length - suffixLength).map( + (content) => ({ + content, + kind: "context" as const, + }), + ), + ]; + + let oldLineNumber = 1; + let newLineNumber = 1; + return operations.map((operation): RawLineDiffRow => { + if (operation.kind === "addition") { + return { + ...operation, + newLineNumber: newLineNumber++, + oldLineNumber: null, + }; + } + if (operation.kind === "deletion") { + return { + ...operation, + newLineNumber: null, + oldLineNumber: oldLineNumber++, + }; + } + return { + ...operation, + newLineNumber: newLineNumber++, + oldLineNumber: oldLineNumber++, + }; + }); +} + +function createMiddleOperations( + previousLines: readonly string[], + currentLines: readonly string[], +): readonly { + content: string; + kind: "addition" | "context" | "deletion"; +}[] { + if (previousLines.length * currentLines.length > maximumLcsMatrixCells) { + return [ + ...previousLines.map((content) => ({ + content, + kind: "deletion" as const, + })), + ...currentLines.map((content) => ({ + content, + kind: "addition" as const, + })), + ]; + } + + const columnCount = currentLines.length + 1; + const table = new Uint32Array((previousLines.length + 1) * columnCount); + for (let oldIndex = previousLines.length - 1; oldIndex >= 0; oldIndex -= 1) { + for ( + let newIndex = currentLines.length - 1; + newIndex >= 0; + newIndex -= 1 + ) { + const tableIndex = oldIndex * columnCount + newIndex; + table[tableIndex] = previousLines[oldIndex] === currentLines[newIndex] + ? table[(oldIndex + 1) * columnCount + newIndex + 1] + 1 + : Math.max( + table[(oldIndex + 1) * columnCount + newIndex], + table[tableIndex + 1], + ); + } + } + + const operations: { + content: string; + kind: "addition" | "context" | "deletion"; + }[] = []; + let oldIndex = 0; + let newIndex = 0; + while ( + oldIndex < previousLines.length + && newIndex < currentLines.length + ) { + if (previousLines[oldIndex] === currentLines[newIndex]) { + operations.push({ + content: previousLines[oldIndex] ?? "", + kind: "context", + }); + oldIndex += 1; + newIndex += 1; + continue; + } + const deletionScore = table[(oldIndex + 1) * columnCount + newIndex]; + const additionScore = table[oldIndex * columnCount + newIndex + 1]; + if (deletionScore >= additionScore) { + operations.push({ + content: previousLines[oldIndex] ?? "", + kind: "deletion", + }); + oldIndex += 1; + } else { + operations.push({ + content: currentLines[newIndex] ?? "", + kind: "addition", + }); + newIndex += 1; + } + } + while (oldIndex < previousLines.length) { + operations.push({ + content: previousLines[oldIndex] ?? "", + kind: "deletion", + }); + oldIndex += 1; + } + while (newIndex < currentLines.length) { + operations.push({ + content: currentLines[newIndex] ?? "", + kind: "addition", + }); + newIndex += 1; + } + return operations; +} + +function collapseContextRows( + rows: readonly RawLineDiffRow[], + contextLineCount: number, +): readonly LineDiffRow[] { + const collapsed: LineDiffRow[] = []; + let index = 0; + while (index < rows.length) { + if (rows[index]?.kind !== "context") { + const row = rows[index]; + if (row) { + collapsed.push(row); + } + index += 1; + continue; + } + + const start = index; + while (index < rows.length && rows[index]?.kind === "context") { + index += 1; + } + const contextRows = rows.slice(start, index); + if (contextLineCount === 0) { + collapsed.push({ + count: contextRows.length, + kind: "omission", + }); + continue; + } + const maximumVisible = contextLineCount * 2; + if (contextRows.length <= maximumVisible + 1) { + collapsed.push(...contextRows); + continue; + } + + collapsed.push(...contextRows.slice(0, contextLineCount)); + collapsed.push({ + count: contextRows.length - maximumVisible, + kind: "omission", + }); + collapsed.push(...contextRows.slice(-contextLineCount)); + } + return collapsed; +} + +function findCommonPrefixLength( + previousLines: readonly string[], + currentLines: readonly string[], +): number { + const maximum = Math.min(previousLines.length, currentLines.length); + let index = 0; + while (index < maximum && previousLines[index] === currentLines[index]) { + index += 1; + } + return index; +} + +function findCommonSuffixLength( + previousLines: readonly string[], + currentLines: readonly string[], + prefixLength: number, +): number { + const maximum = Math.min( + previousLines.length - prefixLength, + currentLines.length - prefixLength, + ); + let count = 0; + while ( + count < maximum + && previousLines[previousLines.length - count - 1] + === currentLines[currentLines.length - count - 1] + ) { + count += 1; + } + return count; +} + +function splitLines(content: string): readonly string[] { + if (content.length === 0) { + return []; + } + return content.replace(/\r\n?/gu, "\n").split("\n"); +} diff --git a/apps/webui/src/lib/plugins/installations.ts b/apps/webui/src/lib/plugins/installations.ts index 1e31299..4f9db94 100644 --- a/apps/webui/src/lib/plugins/installations.ts +++ b/apps/webui/src/lib/plugins/installations.ts @@ -22,10 +22,17 @@ export interface WebUiAnalyticsStoreCreateContext { readEnvironment(name: string): string | undefined; } +export interface WebUiDataRepositoryCreateContext { + bindings: ReadonlyMap; + readEnvironment(name: string): string | undefined; +} + export interface WebUiDataRepositoryInstallation { enabledByDefault: boolean; manifest: PluginManifest<"data-repository", "webui">; - create(): AppDataRepository | Promise; + create( + context: WebUiDataRepositoryCreateContext, + ): AppDataRepository | Promise; } export interface WebUiAnalyticsStoreInstallation { diff --git a/apps/webui/tests/line-diff.test.ts b/apps/webui/tests/line-diff.test.ts new file mode 100644 index 0000000..b4d9f60 --- /dev/null +++ b/apps/webui/tests/line-diff.test.ts @@ -0,0 +1,119 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { createLineDiff } from "../src/lib/data/line-diff"; + +test("renders an initial revision as additions", () => { + const diff = createLineDiff(null, "{\n \"enabled\": true\n}"); + + assert.equal(diff.additions, 3); + assert.equal(diff.deletions, 0); + assert.deepEqual( + diff.rows.map((row) => row.kind), + ["addition", "addition", "addition"], + ); +}); + +test("aligns changed JSON lines with old and new line numbers", () => { + const diff = createLineDiff( + "{\n \"enabled\": false,\n \"count\": 1\n}", + "{\n \"enabled\": true,\n \"count\": 2\n}", + ); + + assert.equal(diff.additions, 2); + assert.equal(diff.deletions, 2); + assert.deepEqual(diff.rows, [ + { + content: "{", + kind: "context", + newLineNumber: 1, + oldLineNumber: 1, + }, + { + content: " \"enabled\": false,", + kind: "deletion", + newLineNumber: null, + oldLineNumber: 2, + }, + { + content: " \"count\": 1", + kind: "deletion", + newLineNumber: null, + oldLineNumber: 3, + }, + { + content: " \"enabled\": true,", + kind: "addition", + newLineNumber: 2, + oldLineNumber: null, + }, + { + content: " \"count\": 2", + kind: "addition", + newLineNumber: 3, + oldLineNumber: null, + }, + { + content: "}", + kind: "context", + newLineNumber: 4, + oldLineNumber: 4, + }, + ]); +}); + +test("collapses long unchanged sections around a modification", () => { + const previous = Array.from( + { length: 12 }, + (_, index) => `line ${index + 1}`, + ); + const current = [...previous]; + current[6] = "changed"; + + const diff = createLineDiff( + previous.join("\n"), + current.join("\n"), + 2, + ); + + assert.deepEqual( + diff.rows.map((row) => row.kind), + [ + "context", + "context", + "omission", + "context", + "context", + "deletion", + "addition", + "context", + "context", + "context", + "context", + "context", + ], + ); +}); + +test("normalizes line endings before comparing content", () => { + const diff = createLineDiff("first\r\nsecond", "first\nsecond"); + + assert.equal(diff.additions, 0); + assert.equal(diff.deletions, 0); +}); + +test("falls back safely for a large completely changed document", () => { + const previous = Array.from( + { length: 1_001 }, + (_, index) => `old ${index}`, + ).join("\n"); + const current = Array.from( + { length: 1_001 }, + (_, index) => `new ${index}`, + ).join("\n"); + + const diff = createLineDiff(previous, current); + + assert.equal(diff.deletions, 1_001); + assert.equal(diff.additions, 1_001); +}); diff --git a/apps/webui/tests/repository-editor-capabilities.test.ts b/apps/webui/tests/repository-editor-capabilities.test.ts new file mode 100644 index 0000000..5e27b6b --- /dev/null +++ b/apps/webui/tests/repository-editor-capabilities.test.ts @@ -0,0 +1,24 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { postgresDataRepositoryManifest } from "@i0c/plugin-data-repository-postgres/manifest"; +import { githubContentsRepositoryManifest } from "@i0c/plugin-github-data/manifest"; + +import { resolveDataRepositoryEditorCapabilities } from "../src/lib/data/editor-capabilities"; + +test("keeps raw rule editing exclusive to the GitHub repository", () => { + assert.deepEqual( + resolveDataRepositoryEditorCapabilities(githubContentsRepositoryManifest), + { + supportsJsonEditor: true, + supportsSourceOverride: true, + }, + ); + assert.deepEqual( + resolveDataRepositoryEditorCapabilities(postgresDataRepositoryManifest), + { + supportsJsonEditor: false, + supportsSourceOverride: false, + }, + ); +}); diff --git a/apps/webui/tests/settings-navigation.test.ts b/apps/webui/tests/settings-navigation.test.ts new file mode 100644 index 0000000..db9863f --- /dev/null +++ b/apps/webui/tests/settings-navigation.test.ts @@ -0,0 +1,19 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + shouldConfirmSettingsCategoryChange, + shouldShowSettingsSaveAction, +} from "../src/components/settings/settings-navigation"; + +test("guards unsaved settings before opening data management", () => { + assert.equal(shouldConfirmSettingsCategoryChange("data", true), true); + assert.equal(shouldConfirmSettingsCategoryChange("data", false), false); + assert.equal(shouldConfirmSettingsCategoryChange("analytics", true), false); +}); + +test("hides the settings save action only for data management", () => { + assert.equal(shouldShowSettingsSaveAction("data"), false); + assert.equal(shouldShowSettingsSaveAction("runtime"), true); + assert.equal(shouldShowSettingsSaveAction("installed-plugins"), true); +}); diff --git a/docs/plugins.md b/docs/plugins.md index b13b46b..f753cf0 100644 --- a/docs/plugins.md +++ b/docs/plugins.md @@ -18,7 +18,7 @@ Plugins are workspace packages selected at build time. Remote `config.json` may | Runtime build | `@i0c/runtime-build` | Build-time installation config validation and selected-platform bundling | | Git data | `@i0c/plugin-github-data` | GitHub Raw Runtime source and GitHub Contents WebUI repository | | HTTP source | `@i0c/plugin-http-snapshot-source` | Atomic Runtime config-and-rules snapshot loading over HTTPS | -| Data repository | `@i0c/plugin-data-repository-postgres` | Optimistic, atomic config-and-rules persistence in PostgreSQL | +| Data repositories | `@i0c/plugin-data-repository-postgres`, `@i0c/plugin-data-repository-d1` | Optimistic, atomic config-and-rules persistence in PostgreSQL or Cloudflare D1 | | Runtime | `@i0c/plugin-runtime-cloudflare`, `@i0c/plugin-runtime-vercel`, `@i0c/plugin-runtime-netlify` | Provider request, environment, cache, country, and background-task adaptation | | Sink | `@i0c/plugin-analytics-sink-http` | Signed best-effort HTTP analytics delivery | | Stores | `@i0c/plugin-analytics-store-postgres`, `@i0c/plugin-analytics-store-d1` | Analytics ingest, queries, rebuild, retention, health, and owned migrations | @@ -33,11 +33,11 @@ The editable non-secret data plane contains two documents: - `config.json` stores versioned instance settings and installed-plugin declarations. - `redirects.json` stores redirect rules. -The checked-in PostgreSQL Repository stores both documents with optimistic revisions and exposes an atomic read snapshot. GitHub Contents remains available and preserves the existing `data` branch workflow. Repository migration history is independent from analytics migrations. +The PostgreSQL and D1 Repositories store both documents with optimistic revisions, atomic snapshots, immutable history, and rollback. The checked-in deployment selects PostgreSQL. GitHub Contents remains available and preserves the archived `data` branch workflow. Repository migration history is independent from analytics migrations. -The checked-in HTTP Snapshot Source reads one validated `{ revision, config, redirects }` response from the WebUI, deduplicates concurrent loads, revalidates with an ETag, and retains the last host-valid snapshot after a failed refresh. GitHub Raw remains available and reads both Git-backed documents independently. Runtime never connects directly to the PostgreSQL Repository. +The checked-in HTTP Snapshot Source reads one validated `{ revision, config, redirects }` response from the WebUI, deduplicates concurrent loads, revalidates with an ETag, and retains the last host-valid snapshot after a failed refresh. GitHub Raw remains available and reads both Git-backed documents independently. Runtime never connects directly to the selected database Repository. -Some values must exist before either document can be loaded. The selected Repository and Source, GitHub owner, repository, branch and paths, OAuth scope, PostgreSQL binding and connection policy, HTTP snapshot URL and retry policy, and installed plugin packages are therefore **bootstrap configuration**, not remote plugin configuration. Defaults live in `@i0c/config`; executable Runtime installations live in the root `i0c.runtime.config.ts`, WebUI server installations live in the root `i0c.webui.config.ts`, and client-safe WebUI extensions live in `apps/webui/webui.extensions.ts`. Changing an installation requires a rebuild. Plugin manifests intentionally reject bootstrap-only fields under `plugins.*.config`; accepting them there would create settings that validate but cannot initialize their own loader. +Some values must exist before either document can be loaded. The selected Repository and Source, GitHub owner, repository, branch and paths, OAuth scope, database binding and connection policy, HTTP snapshot URL and retry policy, and installed plugin packages are therefore **bootstrap configuration**, not remote plugin configuration. Defaults live in `@i0c/config`; executable Runtime installations live in the root `i0c.runtime.config.ts`, WebUI server installations live in the root `i0c.webui.config.ts`, and client-safe WebUI extensions live in `apps/webui/webui.extensions.ts`. Changing an installation requires a rebuild. Plugin manifests intentionally reject bootstrap-only fields under `plugins.*.config`; accepting them there would create settings that validate but cannot initialize their own loader. ## Manifest and configuration model @@ -101,9 +101,9 @@ The production installation keeps these slots empty except for the host-owned pl ## Data repositories and migrations -`AtomicVersionedDataRepository` exposes document reads, optimistic writes, and an atomic two-document snapshot. GitHub maps its commit SHA to the repository revision and resolves both documents at one commit. PostgreSQL uses a repeatable-read transaction, numeric document revisions, and checksums. +`AtomicVersionedDataRepository` exposes document reads, optimistic writes, and an atomic two-document snapshot. GitHub maps its commit SHA to the repository revision and resolves both documents at one commit. PostgreSQL and D1 add the same managed contract for first-run initialization, immutable revision history, atomic import, and non-destructive restore. -PostgreSQL Repository migrations live in `plugins/repository/postgres/migrations` and use their own migration table and advisory lock. Builds, application startup, Runtime requests, and WebUI health checks never apply them automatically. Selecting PostgreSQL requires `DATABASE_URL`. Its explicit seed command validates local `config.json` and `redirects.json`, creates only missing documents in one transaction, and never overwrites existing content. The repository must be paired with the HTTP Snapshot Runtime Source so saved database state reaches edge deployments without giving them database credentials; incompatible bootstrap selections fail the build. +PostgreSQL Repository migrations live in `plugins/repository/postgres/migrations` and use their own migration table and advisory lock. D1 Repository migrations live in `plugins/repository/d1/migrations`, use checksums and continuous-history validation, and execute each migration plus its version record in an atomic D1 batch. Builds, application startup, Runtime requests, and WebUI health checks never apply them automatically. Selecting PostgreSQL requires `DATABASE_URL`; selecting D1 requires a D1-capable WebUI host to inject its binding before first use. Both database Repositories must be paired with the HTTP Snapshot Runtime Source so saved state reaches edge deployments without giving them database credentials; incompatible bootstrap selections fail the build. ## Analytics stores and migrations diff --git a/docs/plugins.zh-CN.md b/docs/plugins.zh-CN.md index f33a944..faefc28 100644 --- a/docs/plugins.zh-CN.md +++ b/docs/plugins.zh-CN.md @@ -18,7 +18,7 @@ i0c.cc 使用轻量的静态注册插件架构，让重定向核心不直接绑 | Runtime 构建 | `@i0c/runtime-build` | 构建期安装配置校验与所选平台 Bundle 生成 | | Git 数据 | `@i0c/plugin-github-data` | GitHub Raw Runtime Source 与 GitHub Contents WebUI Repository | | HTTP 数据源 | `@i0c/plugin-http-snapshot-source` | 通过 HTTPS 原子读取 Runtime 配置与规则快照 | -| 数据仓库 | `@i0c/plugin-data-repository-postgres` | 在 PostgreSQL 中以乐观并发和原子快照持久化配置与规则 | +| 数据 Repository | `@i0c/plugin-data-repository-postgres`、`@i0c/plugin-data-repository-d1` | 在 PostgreSQL 或 Cloudflare D1 中以乐观并发和原子快照持久化配置与规则 | | Runtime | `@i0c/plugin-runtime-cloudflare`、`@i0c/plugin-runtime-vercel`、`@i0c/plugin-runtime-netlify` | 平台请求、环境、缓存、国家信息与后台任务适配 | | Sink | `@i0c/plugin-analytics-sink-http` | 带签名、尽力而为的 HTTP 统计投递 | | Store | `@i0c/plugin-analytics-store-postgres`、`@i0c/plugin-analytics-store-d1` | 统计写入、查询、重算、保留、健康检查和自有迁移 | @@ -33,11 +33,11 @@ i0c.cc 使用轻量的静态注册插件架构，让重定向核心不直接绑 - `config.json` 存放版本化实例设置和已安装插件声明。 - `redirects.json` 存放重定向规则。 -仓库当前启用的 PostgreSQL Repository 以乐观版本写入两份文档，并提供原子读取快照。GitHub Contents 仍然可用，并保留现有 `data` 分支工作流；Repository 迁移历史与统计迁移相互独立。 +PostgreSQL 与 D1 Repository 都以乐观版本写入两份文档，并提供原子快照、不可变历史与回滚。仓库当前部署选择 PostgreSQL。GitHub Contents 仍然可用，并保留归档的 `data` 分支工作流；Repository 迁移历史与统计迁移相互独立。 -仓库当前启用的 HTTP Snapshot Source 会从 WebUI 一次读取经过校验的 `{ revision, config, redirects }`，合并并发加载、使用 ETag 重新验证，并在刷新失败时保留最后一次通过宿主校验的快照。GitHub Raw 仍然可用，并会分别读取两份 Git 文档。Runtime 不会直接连接 PostgreSQL Repository。 +仓库当前启用的 HTTP Snapshot Source 会从 WebUI 一次读取经过校验的 `{ revision, config, redirects }`，合并并发加载、使用 ETag 重新验证，并在刷新失败时保留最后一次通过宿主校验的快照。GitHub Raw 仍然可用，并会分别读取两份 Git 文档。Runtime 不会直接连接所选数据库 Repository。 -有些值必须在读取远程文档之前存在。所选 Repository 与 Source、GitHub 所有者、仓库、分支和路径、OAuth scope、PostgreSQL binding 与连接策略、HTTP 快照地址与重试策略，以及已安装插件包因此属于**启动配置**，不是远程插件配置。默认值位于 `@i0c/config`；Runtime 可执行安装位于根目录 `i0c.runtime.config.ts`，WebUI 服务端安装位于根目录 `i0c.webui.config.ts`，客户端安全的 WebUI 扩展位于 `apps/webui/webui.extensions.ts`。修改安装项后需要重新构建。插件 Manifest 会明确拒绝在 `plugins.*.config` 中填写这些启动字段，避免出现“能通过校验，却无法初始化自身加载器”的假配置。 +有些值必须在读取远程文档之前存在。所选 Repository 与 Source、GitHub 所有者、仓库、分支和路径、OAuth scope、数据库 binding 与连接策略、HTTP 快照地址与重试策略，以及已安装插件包因此属于**启动配置**，不是远程插件配置。默认值位于 `@i0c/config`；Runtime 可执行安装位于根目录 `i0c.runtime.config.ts`，WebUI 服务端安装位于根目录 `i0c.webui.config.ts`，客户端安全的 WebUI 扩展位于 `apps/webui/webui.extensions.ts`。修改安装项后需要重新构建。插件 Manifest 会明确拒绝在 `plugins.*.config` 中填写这些启动字段，避免出现“能通过校验，却无法初始化自身加载器”的假配置。 ## Manifest 与配置模型 @@ -101,9 +101,9 @@ WebUI 提供四个静态注册扩展插槽： ## Data Repository 与迁移 -`AtomicVersionedDataRepository` 提供文档读取、乐观并发写入和两份文档的原子快照。GitHub 把 commit SHA 作为 Repository revision，并在同一 commit 上读取两份文档；PostgreSQL 使用只读可重复读事务、数字文档版本与校验和。 +`AtomicVersionedDataRepository` 提供文档读取、乐观并发写入和两份文档的原子快照。GitHub 把 commit SHA 作为 Repository revision，并在同一 commit 上读取两份文档；PostgreSQL 与 D1 还实现相同的首次初始化、不可变版本历史、原子导入和非破坏性恢复契约。 -PostgreSQL Repository 迁移位于 `plugins/repository/postgres/migrations`，使用独立迁移表和 advisory lock。构建、应用启动、Runtime 请求与 WebUI 健康检查都不会自动执行迁移。选择 PostgreSQL 时需要配置 `DATABASE_URL`。它的显式初始化命令会校验本地 `config.json` 与 `redirects.json`，在同一个事务中只创建缺失文档，并且不会覆盖已有内容。该 Repository 必须搭配 HTTP Snapshot Runtime Source，让数据库中保存的状态能到达边缘部署，同时不向 Runtime 提供数据库凭据；不兼容的启动选择会让构建失败。 +PostgreSQL Repository 迁移位于 `plugins/repository/postgres/migrations`，使用独立迁移表和 advisory lock。D1 Repository 迁移位于 `plugins/repository/d1/migrations`，会校验 checksum 与迁移历史连续性，并在一个原子 D1 batch 中执行每次迁移和版本记录。构建、应用启动、Runtime 请求与 WebUI 健康检查都不会自动执行迁移。选择 PostgreSQL 时需要配置 `DATABASE_URL`；选择 D1 时，支持 D1 的 WebUI 宿主必须在首次使用前注入 binding。两种数据库 Repository 都必须搭配 HTTP Snapshot Runtime Source，让保存状态能到达边缘部署，同时不向 Runtime 提供数据库凭据；不兼容的启动选择会让构建失败。 ## Analytics Store 与迁移 diff --git a/i0c.webui.config.ts b/i0c.webui.config.ts index 7f4d0ad..ff0aa4f 100644 --- a/i0c.webui.config.ts +++ b/i0c.webui.config.ts @@ -2,11 +2,12 @@ import { assertBootstrapConfigCompatibility, bootstrapConfig, } from "@i0c/config" -import type { D1Database } from "@i0c/plugin-analytics-store-d1/types" +import type { D1Database } from "@i0c/plugin-data-repository-d1/d1" import type { GitHubFetch } from "@i0c/plugin-github-data/webui" import { defineWebUiPluginInstallations, + type WebUiDataRepositoryCreateContext, } from "./apps/webui/src/lib/plugins/installations" import { webUiPluginDescriptors } from "./i0c.webui.manifests" @@ -68,14 +69,17 @@ export const webUiPluginInstallations = defineWebUiPluginInstallations({ ], }) -async function createConfiguredDataRepository() { +async function createConfiguredDataRepository( + context: WebUiDataRepositoryCreateContext, +) { const repository = bootstrapConfig.data.repository if (repository.provider === "postgres") { const { createPostgresDataRepository } = await import( "@i0c/plugin-data-repository-postgres/repository" ) - const connectionString = - process.env[repository.databaseUrlBinding]?.trim() ?? "" + const connectionString = context + .readEnvironment(repository.databaseUrlBinding) + ?.trim() ?? "" return createPostgresDataRepository({ connectionString, maxConnections: repository.maxConnections, @@ -83,6 +87,20 @@ async function createConfiguredDataRepository() { connectTimeoutSeconds: repository.connectTimeoutSeconds, }) } + if (repository.provider === "d1") { + const database = context.bindings.get( + webUiPluginDescriptors.dataRepository.manifest.id, + ) + if (!isD1Database(database)) { + throw new TypeError( + "The D1 data repository requires a D1Database host binding", + ) + } + const { createD1DataRepository } = await import( + "@i0c/plugin-data-repository-d1/repository" + ) + return createD1DataRepository(database) + } const { createGitHubContentsRepository } = await import( "@i0c/plugin-github-data/webui" ) diff --git a/i0c.webui.manifests.ts b/i0c.webui.manifests.ts index a6503e4..fda9fb4 100644 --- a/i0c.webui.manifests.ts +++ b/i0c.webui.manifests.ts @@ -1,13 +1,22 @@ import { bootstrapConfig } from "@i0c/config" import { d1AnalyticsStoreManifest } from "@i0c/plugin-analytics-store-d1/manifest" import { postgresAnalyticsStoreManifest } from "@i0c/plugin-analytics-store-postgres/manifest" +import { d1DataRepositoryManifest } from "@i0c/plugin-data-repository-d1/manifest" import { postgresDataRepositoryManifest } from "@i0c/plugin-data-repository-postgres/manifest" import { githubContentsRepositoryManifest } from "@i0c/plugin-github-data/manifest" -const dataRepositoryManifest = - bootstrapConfig.data.repository.provider === "postgres" - ? postgresDataRepositoryManifest - : githubContentsRepositoryManifest +function resolveDataRepositoryManifest() { + switch (bootstrapConfig.data.repository.provider) { + case "d1": + return d1DataRepositoryManifest + case "github": + return githubContentsRepositoryManifest + case "postgres": + return postgresDataRepositoryManifest + } +} + +const dataRepositoryManifest = resolveDataRepositoryManifest() export const webUiPluginDescriptors = { dataRepository: { diff --git a/package.json b/package.json index 83597a4..2581448 100644 --- a/package.json +++ b/package.json @@ -40,6 +40,7 @@ "@i0c/plugin-analytics-sink-http": "workspace:*", "@i0c/plugin-analytics-store-d1": "workspace:*", "@i0c/plugin-analytics-store-postgres": "workspace:*", + "@i0c/plugin-data-repository-d1": "workspace:*", "@i0c/plugin-data-repository-postgres": "workspace:*", "@i0c/plugin-api": "workspace:*", "@i0c/plugin-feature-bot-classifier": "workspace:*", diff --git a/packages/config/src/bootstrap-validation.ts b/packages/config/src/bootstrap-validation.ts index 0dec367..e35057a 100644 --- a/packages/config/src/bootstrap-validation.ts +++ b/packages/config/src/bootstrap-validation.ts @@ -4,11 +4,11 @@ export function assertBootstrapConfigCompatibility( config: BootstrapConfig, ): void { if ( - config.data.repository.provider === "postgres" + config.data.repository.provider !== "github" && config.data.source.provider !== "http" ) { throw new TypeError( - "The PostgreSQL data repository requires the HTTP Runtime data source", + "Database-backed data repositories require the HTTP Runtime data source", ) } } diff --git a/packages/config/src/defaults.ts b/packages/config/src/defaults.ts index 3ec4bbe..c5d0269 100644 --- a/packages/config/src/defaults.ts +++ b/packages/config/src/defaults.ts @@ -1,5 +1,18 @@ import type { BootstrapConfig, DataConfig } from "./types" +function resolveDataRepositoryPluginId( + repository: BootstrapConfig["data"]["repository"], +): string { + switch (repository.provider) { + case "d1": + return "@i0c/data-repository-d1" + case "github": + return "@i0c/github-contents-repository" + case "postgres": + return "@i0c/data-repository-postgres" + } +} + export const bootstrapConfig: BootstrapConfig = { data: { github: { @@ -56,9 +69,7 @@ export const defaultDataConfig: DataConfig = { enabled: true, version: 1, }, - [bootstrapConfig.data.repository.provider === "postgres" - ? "@i0c/data-repository-postgres" - : "@i0c/github-contents-repository"]: { + [resolveDataRepositoryPluginId(bootstrapConfig.data.repository)]: { enabled: true, version: 1, }, diff --git a/packages/config/src/types.ts b/packages/config/src/types.ts index fdc2c61..658d032 100644 --- a/packages/config/src/types.ts +++ b/packages/config/src/types.ts @@ -34,7 +34,12 @@ export interface PostgresDataRepositoryBootstrapConfig { connectTimeoutSeconds: number } +export interface D1DataRepositoryBootstrapConfig { + provider: "d1" +} + export type DataRepositoryBootstrapConfig = + | D1DataRepositoryBootstrapConfig | GitHubDataRepositoryBootstrapConfig | PostgresDataRepositoryBootstrapConfig diff --git a/packages/plugin-catalog/package.json b/packages/plugin-catalog/package.json index 5d6c2a4..ec02fa4 100644 --- a/packages/plugin-catalog/package.json +++ b/packages/plugin-catalog/package.json @@ -20,6 +20,7 @@ "@i0c/plugin-analytics-sink-http": "workspace:*", "@i0c/plugin-analytics-store-d1": "workspace:*", "@i0c/plugin-analytics-store-postgres": "workspace:*", + "@i0c/plugin-data-repository-d1": "workspace:*", "@i0c/plugin-data-repository-postgres": "workspace:*", "@i0c/plugin-feature-bot-classifier": "workspace:*", "@i0c/plugin-github-data": "workspace:*", diff --git a/packages/plugin-catalog/src/ids.ts b/packages/plugin-catalog/src/ids.ts index 628f20e..20025b0 100644 --- a/packages/plugin-catalog/src/ids.ts +++ b/packages/plugin-catalog/src/ids.ts @@ -2,6 +2,7 @@ export const installedPluginIds = [ "@i0c/github-raw-source", "@i0c/http-snapshot-source", "@i0c/github-contents-repository", + "@i0c/data-repository-d1", "@i0c/data-repository-postgres", "@i0c/runtime-cloudflare", "@i0c/runtime-vercel", diff --git a/packages/plugin-catalog/src/webui.ts b/packages/plugin-catalog/src/webui.ts index f6f01a2..7b218e6 100644 --- a/packages/plugin-catalog/src/webui.ts +++ b/packages/plugin-catalog/src/webui.ts @@ -1,5 +1,6 @@ import { d1AnalyticsStoreManifest } from "@i0c/plugin-analytics-store-d1/manifest" import { postgresAnalyticsStoreManifest } from "@i0c/plugin-analytics-store-postgres/manifest" +import { d1DataRepositoryManifest } from "@i0c/plugin-data-repository-d1/manifest" import { postgresDataRepositoryManifest } from "@i0c/plugin-data-repository-postgres/manifest" import { githubContentsRepositoryManifest } from "@i0c/plugin-github-data/manifest" import { StaticPluginRegistry } from "@i0c/plugin-api" @@ -8,6 +9,7 @@ import { installedPluginIds } from "./ids" export const webUiPluginManifests = [ githubContentsRepositoryManifest, + d1DataRepositoryManifest, postgresDataRepositoryManifest, postgresAnalyticsStoreManifest, d1AnalyticsStoreManifest, diff --git a/packages/plugin-testkit/package.json b/packages/plugin-testkit/package.json index 90897d8..562c1af 100644 --- a/packages/plugin-testkit/package.json +++ b/packages/plugin-testkit/package.json @@ -21,6 +21,7 @@ "test": "tsx --test tests/**/*.test.ts" }, "dependencies": { + "@i0c/config": "workspace:*", "@i0c/plugin-api": "workspace:*" }, "devDependencies": { diff --git a/packages/plugin-testkit/src/assertions.ts b/packages/plugin-testkit/src/assertions.ts index 9487d5c..8757155 100644 --- a/packages/plugin-testkit/src/assertions.ts +++ b/packages/plugin-testkit/src/assertions.ts @@ -1,9 +1,22 @@ import assert from "node:assert/strict" +import { + DataDocumentNotFoundError, + DataRepositoryConflictError, + DataRepositoryInitializationError, + type DataDocument, + type DataDocumentKind, + type DataRepositoryManagement, + type DataRepositoryReadOptions, + type DataRepositorySnapshot, + type DataRepositoryWriteInput, + type DataRepositoryWriteResult, +} from "@i0c/config" import { type AnalyticsSink, type AnalyticsStore, type AnalyticsStoreTypes, + type AtomicVersionedDataRepository, type PluginHealthCheck, type PluginManifest, type PluginMigrationProvider, @@ -126,6 +139,160 @@ export async function assertVersionedDataRepositoryContract< assert.deepEqual(after, input.expectedAfter) } +export type ManagedDataRepository = AtomicVersionedDataRepository< + DataDocumentKind, + DataRepositoryReadOptions, + DataRepositoryWriteInput, + DataDocument, + DataRepositoryWriteResult, + DataRepositorySnapshot +> & { + management: DataRepositoryManagement +} + +export async function assertManagedDataRepositoryBehaviorContract( + repository: ManagedDataRepository, +): Promise { + const actorGitHubUserId = "123" + const initialConfig = "{\"schemaVersion\":1}" + const initialRedirects = "{\"Slots\":{}}" + const updatedConfig = "{\"schemaVersion\":1,\"updated\":true}" + const importedConfig = "{\"schemaVersion\":1,\"imported\":true}" + const importedRedirects = "{\"Slots\":{\"/\":\"https://example.com\"}}" + + assert.deepEqual( + await repository.management.inspectSetupState(), + { state: "empty", existingKinds: [] }, + ) + + const initialized = await repository.management.initialize({ + actorGitHubUserId, + configContent: initialConfig, + redirectsContent: initialRedirects, + }) + assert.equal(initialized.config.revision, "1") + assert.equal(initialized.redirects.revision, "1") + assert.match(initialized.revision, /^[0-9a-f]{64}$/) + assert.deepEqual( + await repository.management.inspectSetupState(), + { state: "initialized" }, + ) + await assert.rejects( + repository.management.initialize({ + actorGitHubUserId, + configContent: initialConfig, + redirectsContent: initialRedirects, + }), + DataRepositoryInitializationError, + ) + + assert.deepEqual( + await repository.write("config", { + actorGitHubUserId, + content: updatedConfig, + expectedRevision: "1", + }), + { revision: "2" }, + ) + await assert.rejects( + repository.write("config", { + actorGitHubUserId, + content: "{\"schemaVersion\":1,\"stale\":true}", + expectedRevision: "1", + }), + (error) => { + assert.ok(error instanceof DataRepositoryConflictError) + assert.equal(error.kind, "config") + assert.equal(error.expectedRevision, "1") + assert.equal(error.actualRevision, "2") + return true + }, + ) + + const firstConfigRevision = await repository.management.readRevision({ + kind: "config", + revision: "1", + }) + assert.equal(firstConfigRevision.content, initialConfig) + assert.equal(firstConfigRevision.operation, "initialize") + assert.equal(firstConfigRevision.actorGitHubUserId, actorGitHubUserId) + assert.match(firstConfigRevision.checksum, /^[0-9a-f]{64}$/) + assert.ok(Number.isFinite(Date.parse(firstConfigRevision.createdAt))) + + assert.deepEqual( + await repository.management.restore({ + actorGitHubUserId, + expectedRevision: "2", + kind: "config", + revision: "1", + }), + { revision: "3" }, + ) + assert.equal( + (await repository.read("config", {})).content, + initialConfig, + ) + + const imported = await repository.management.importSnapshot({ + actorGitHubUserId, + configContent: importedConfig, + expectedConfigRevision: "3", + expectedRedirectsRevision: "1", + redirectsContent: importedRedirects, + }) + assert.equal(imported.config.revision, "4") + assert.equal(imported.redirects.revision, "2") + assert.equal(imported.config.content, importedConfig) + assert.equal(imported.redirects.content, importedRedirects) + + const beforeConflict = await repository.readSnapshot({}) + await assert.rejects( + repository.management.importSnapshot({ + actorGitHubUserId, + configContent: "{\"schemaVersion\":1,\"mustNotPersist\":true}", + expectedConfigRevision: "4", + expectedRedirectsRevision: "1", + redirectsContent: "{\"Slots\":{\"/stale\":\"https://example.com\"}}", + }), + (error) => { + assert.ok(error instanceof DataRepositoryConflictError) + assert.equal(error.kind, "redirects") + assert.equal(error.expectedRevision, "1") + assert.equal(error.actualRevision, "2") + return true + }, + ) + assert.deepEqual(await repository.readSnapshot({}), beforeConflict) + + assert.deepEqual( + (await repository.management.listRevisions({ + kind: "config", + limit: 10, + })).map((revision) => [revision.revision, revision.operation]), + [ + ["4", "import"], + ["3", "rollback"], + ["2", "save"], + ["1", "initialize"], + ], + ) + assert.deepEqual( + (await repository.management.listRevisions({ + beforeRevision: "3", + kind: "config", + limit: 10, + })).map((revision) => revision.revision), + ["2", "1"], + ) + await assert.rejects( + repository.management.readRevision({ + kind: "config", + revision: "999", + }), + DataDocumentNotFoundError, + ) +} + export interface RuntimePlatformContractInput< TArguments extends readonly unknown[], > { diff --git a/plugins/repository/d1/README.md b/plugins/repository/d1/README.md new file mode 100644 index 0000000..83cf815 --- /dev/null +++ b/plugins/repository/d1/README.md @@ -0,0 +1,28 @@ +# Cloudflare D1 data repository + +This compile-time WebUI plugin stores `config` and `redirects` as versioned +Cloudflare D1 documents. It implements the same managed Data Repository +behavior contract as the PostgreSQL plugin: + +- atomic first-run initialization and two-document import; +- optimistic revision checks for every write; +- immutable revision history and non-destructive restore; +- atomic snapshots for the Runtime HTTP Snapshot Source; +- migration checksums and continuous-history validation. + +## Host requirements + +The WebUI host must provide a compatible `D1Database` binding through +`configureAppDataRepositoryBinding` before the Repository is first used. Apply +both SQL files in [migrations](migrations) to that binding deliberately; builds +and application startup never mutate the database. + +The checked-in Vercel WebUI continues to use PostgreSQL. Selecting D1 is meant +for a D1-capable WebUI host and still requires the HTTP Runtime data source. + +## Checks + +```bash +pnpm --filter @i0c/plugin-data-repository-d1 check +pnpm --filter @i0c/plugin-data-repository-d1 test +``` diff --git a/plugins/repository/d1/README.zh-CN.md b/plugins/repository/d1/README.zh-CN.md new file mode 100644 index 0000000..fa7d36d --- /dev/null +++ b/plugins/repository/d1/README.zh-CN.md @@ -0,0 +1,26 @@ +# Cloudflare D1 数据 Repository + +这个编译期 WebUI 插件把 `config` 与 `redirects` 保存为带版本的 Cloudflare +D1 文档，并与 PostgreSQL 插件遵循同一套 Data Repository 行为契约： + +- 首次初始化和两份文档导入均为原子操作； +- 每次写入都执行乐观 revision 校验； +- 保留不可变版本历史，恢复旧内容时不会改写历史； +- 为 Runtime HTTP Snapshot Source 提供原子快照； +- 校验迁移 checksum 和迁移历史连续性。 + +## 宿主要求 + +WebUI 宿主必须在首次使用 Repository 前，通过 +`configureAppDataRepositoryBinding` 注入兼容的 `D1Database` binding。需要明确将 +[migrations](migrations) 中的两份 SQL 应用到该 binding；构建和应用启动不会自动修改数据库。 + +仓库当前的 Vercel WebUI 继续使用 PostgreSQL。D1 供支持 D1 binding 的 WebUI +宿主选择，并且仍需搭配 HTTP Runtime 数据源。 + +## 检查 + +```bash +pnpm --filter @i0c/plugin-data-repository-d1 check +pnpm --filter @i0c/plugin-data-repository-d1 test +``` diff --git a/plugins/repository/d1/migrations/001_data_documents.sql b/plugins/repository/d1/migrations/001_data_documents.sql new file mode 100644 index 0000000..410f80e --- /dev/null +++ b/plugins/repository/d1/migrations/001_data_documents.sql @@ -0,0 +1,14 @@ +CREATE TABLE i0c_data_document ( + kind TEXT PRIMARY KEY, + content TEXT NOT NULL, + revision INTEGER NOT NULL DEFAULT 1, + checksum TEXT NOT NULL, + updated_at TEXT NOT NULL, + mutation_id TEXT NOT NULL, + CHECK (kind IN ('config', 'redirects')), + CHECK (revision > 0), + CHECK ( + LENGTH(checksum) = 64 + AND checksum NOT GLOB '*[^0-9a-f]*' + ) +); diff --git a/plugins/repository/d1/migrations/002_data_document_history.sql b/plugins/repository/d1/migrations/002_data_document_history.sql new file mode 100644 index 0000000..3029aac --- /dev/null +++ b/plugins/repository/d1/migrations/002_data_document_history.sql @@ -0,0 +1,48 @@ +CREATE TABLE i0c_data_document_revision ( + kind TEXT NOT NULL, + revision INTEGER NOT NULL, + content TEXT NOT NULL, + checksum TEXT NOT NULL, + operation TEXT NOT NULL, + actor_github_user_id TEXT, + created_at TEXT NOT NULL, + PRIMARY KEY (kind, revision), + CHECK (kind IN ('config', 'redirects')), + CHECK (revision > 0), + CHECK ( + LENGTH(checksum) = 64 + AND checksum NOT GLOB '*[^0-9a-f]*' + ), + CHECK (operation IN ('import', 'initialize', 'migration', 'rollback', 'save')), + CHECK ( + actor_github_user_id IS NULL + OR ( + LENGTH(actor_github_user_id) > 0 + AND actor_github_user_id NOT LIKE '0%' + AND actor_github_user_id NOT GLOB '*[^0-9]*' + ) + ) +); + +-- d1-statement-breakpoint +CREATE INDEX i0c_data_document_revision_created_at_idx + ON i0c_data_document_revision (kind, created_at DESC); + +-- d1-statement-breakpoint +INSERT OR IGNORE INTO i0c_data_document_revision ( + kind, + revision, + content, + checksum, + operation, + created_at +) +SELECT + kind, + revision, + content, + checksum, + 'migration', + updated_at +FROM i0c_data_document +; diff --git a/plugins/repository/d1/package.json b/plugins/repository/d1/package.json new file mode 100644 index 0000000..2ce188c --- /dev/null +++ b/plugins/repository/d1/package.json @@ -0,0 +1,49 @@ +{ + "name": "@i0c/plugin-data-repository-d1", + "version": "0.1.0", + "private": true, + "type": "module", + "license": "Apache-2.0", + "repository": { + "type": "git", + "url": "https://github.com/Revaea/i0c.cc.git" + }, + "exports": { + "./config": { + "types": "./src/config.ts", + "default": "./src/config.ts" + }, + "./d1": { + "types": "./src/d1.ts", + "default": "./src/d1.ts" + }, + "./manifest": { + "types": "./src/manifest.ts", + "default": "./src/manifest.ts" + }, + "./migrations": { + "types": "./src/migrations.ts", + "default": "./src/migrations.ts" + }, + "./repository": { + "types": "./src/repository.ts", + "default": "./src/repository.ts" + } + }, + "sideEffects": false, + "scripts": { + "build": "tsc --noEmit", + "check": "tsc --noEmit", + "test": "tsx --test tests/**/*.test.ts" + }, + "dependencies": { + "@i0c/config": "workspace:*", + "@i0c/plugin-api": "workspace:*" + }, + "devDependencies": { + "@i0c/plugin-testkit": "workspace:*", + "@types/node": "^25.5.0", + "tsx": "^4.21.0", + "typescript": "^6.0.2" + } +} diff --git a/plugins/repository/d1/src/config.ts b/plugins/repository/d1/src/config.ts new file mode 100644 index 0000000..3e7e101 --- /dev/null +++ b/plugins/repository/d1/src/config.ts @@ -0,0 +1,7 @@ +import type { JsonObject } from "@i0c/config" + +export const d1DataRepositoryPluginConfigSchema = { + type: "object", + additionalProperties: false, + properties: {}, +} satisfies JsonObject diff --git a/plugins/repository/d1/src/d1.ts b/plugins/repository/d1/src/d1.ts new file mode 100644 index 0000000..28adff0 --- /dev/null +++ b/plugins/repository/d1/src/d1.ts @@ -0,0 +1,55 @@ +export interface D1ResultMeta { + changes?: number +} + +export interface D1Result { + success: boolean + results?: T[] + error?: string + meta?: D1ResultMeta +} + +export interface D1ExecResult { + count: number + duration: number +} + +export interface D1PreparedStatement { + bind(...values: readonly unknown[]): D1PreparedStatement + all>(): Promise> + first>(columnName?: string): Promise + run>(): Promise> +} + +export interface D1Database { + prepare(query: string): D1PreparedStatement + batch>( + statements: readonly D1PreparedStatement[], + ): Promise[]> + exec(query: string): Promise +} + +export async function d1All( + statement: D1PreparedStatement, +): Promise { + const result = await statement.all() + assertD1Result(result) + return result.results ?? [] +} + +export async function d1Batch( + database: D1Database, + statements: readonly D1PreparedStatement[], +): Promise { + const results = await database.batch(statements) + for (const result of results) { + assertD1Result(result) + } + return results +} + +export function assertD1Result(result: D1Result): void { + if (!result.success) { + throw new Error(result.error || "D1 operation failed") + } +} diff --git a/plugins/repository/d1/src/manifest.ts b/plugins/repository/d1/src/manifest.ts new file mode 100644 index 0000000..b5a2435 --- /dev/null +++ b/plugins/repository/d1/src/manifest.ts @@ -0,0 +1,35 @@ +import { + PLUGIN_API_VERSION, + type PluginManifest, +} from "@i0c/plugin-api" + +import { d1DataRepositoryPluginConfigSchema } from "./config" + +export const d1DataRepositoryManifest = { + id: "@i0c/data-repository-d1", + name: "Cloudflare D1 data repository", + version: "0.1.0", + apiVersion: PLUGIN_API_VERSION, + kind: "data-repository", + slot: "data-repository", + hosts: ["webui"], + capabilities: [ + "config:read", + "config:write", + "redirects:read", + "redirects:write", + "snapshot:atomic", + "version:optimistic", + ], + description: { + summary: { + en: "Stores instance configuration and redirect rules as versioned Cloudflare D1 documents.", + "zh-CN": "将实例配置与重定向规则作为带版本的 Cloudflare D1 文档保存。", + }, + }, + config: { + version: 1, + schema: d1DataRepositoryPluginConfigSchema, + }, + secrets: {}, +} as const satisfies PluginManifest diff --git a/plugins/repository/d1/src/migrations.ts b/plugins/repository/d1/src/migrations.ts new file mode 100644 index 0000000..78387ac --- /dev/null +++ b/plugins/repository/d1/src/migrations.ts @@ -0,0 +1,246 @@ +import { + assertContinuousMigrationHistory, + type PluginMigrationApplyInput, + type PluginMigrationApplyResult, + type PluginMigrationPlan, + type PluginMigrationProvider, + type PluginMigrationStatus, +} from "@i0c/plugin-api" + +import type { D1Database } from "./d1" +import { d1All, d1Batch } from "./d1" + +const D1_STATEMENT_BREAKPOINT = /^\s*--\s*d1-statement-breakpoint\s*$/mu + +export interface D1DataRepositoryMigration { + id: string + sql: string +} + +interface AppliedMigrationRow { + checksum: string + id: string +} + +export function createD1DataRepositoryMigrationProvider( + database: D1Database, + migrations: readonly D1DataRepositoryMigration[], +): PluginMigrationProvider { + const ordered = [...migrations].sort((left, right) => + left.id.localeCompare(right.id), + ) + + return { + async migrationStatus(): Promise { + const applied = await readAppliedMigrations(database) + validateAppliedHistory(ordered, applied) + await verifyAppliedMigrationChecksums(ordered, applied) + const pending = ordered.filter((migration) => !applied.has(migration.id)) + return { + currentVersion: resolveCurrentVersion(ordered, applied), + targetVersion: resolveTargetVersion(ordered), + pending: pending.length, + } + }, + async migrationPlan(): Promise { + const applied = await readAppliedMigrations(database) + validateAppliedHistory(ordered, applied) + await verifyAppliedMigrationChecksums(ordered, applied) + return { + currentVersion: resolveCurrentVersion(ordered, applied), + targetVersion: resolveTargetVersion(ordered), + actions: ordered + .filter((migration) => !applied.has(migration.id)) + .map((migration) => ({ + id: migration.id, + description: `Apply ${migration.id}`, + destructive: false, + })), + } + }, + async applyMigrations( + input: PluginMigrationApplyInput = {}, + ): Promise { + await ensureMigrationTable(database) + const applied = await readAppliedMigrations(database) + validateAppliedHistory(ordered, applied) + await verifyAppliedMigrationChecksums(ordered, applied) + const previousVersion = resolveCurrentVersion(ordered, applied) + if ( + input.expectedCurrentVersion !== undefined + && input.expectedCurrentVersion !== previousVersion + ) { + throw new Error( + `Expected migration version ${input.expectedCurrentVersion ?? "none"}, found ${previousVersion ?? "none"}`, + ) + } + + const appliedNow: string[] = [] + for (const migration of ordered) { + if (applied.has(migration.id)) { + continue + } + + const checksum = await createMigrationChecksum(migration.sql) + const statements = splitD1DataRepositoryMigrationStatements( + migration.sql, + ).map((statement) => database.prepare(statement)) + try { + await d1Batch(database, [ + ...statements, + database.prepare(` + INSERT INTO i0c_data_repository_migration (id, checksum) + VALUES (?, ?) + `).bind(migration.id, checksum), + ]) + } catch (error) { + if ( + input.expectedCurrentVersion !== undefined + || !await refreshAppliedMigrationsAfterRace( + database, + ordered, + applied, + migration.id, + checksum, + ) + ) { + throw error + } + continue + } + applied.set(migration.id, checksum) + appliedNow.push(migration.id) + } + + return { + previousVersion, + currentVersion: resolveTargetVersion(ordered), + applied: appliedNow, + } + }, + } +} + +export function splitD1DataRepositoryMigrationStatements( + sql: string, +): readonly string[] { + const statements = sql + .split(D1_STATEMENT_BREAKPOINT) + .map((statement) => statement.trim()) + .filter(Boolean) + if (statements.length === 0) { + throw new Error("D1 data repository migration contains no SQL statements") + } + return statements +} + +async function refreshAppliedMigrationsAfterRace( + database: D1Database, + migrations: readonly D1DataRepositoryMigration[], + applied: Map, + migrationId: string, + checksum: string, +): Promise { + const refreshed = await readAppliedMigrations(database) + validateAppliedHistory(migrations, refreshed) + await verifyAppliedMigrationChecksums(migrations, refreshed) + if (refreshed.get(migrationId) !== checksum) { + return false + } + + applied.clear() + for (const [id, appliedChecksum] of refreshed) { + applied.set(id, appliedChecksum) + } + return true +} + +async function readAppliedMigrations( + database: D1Database, +): Promise> { + const tables = await d1All<{ name: string }>(database.prepare(` + SELECT name + FROM sqlite_master + WHERE type = 'table' AND name = 'i0c_data_repository_migration' + `)) + if (tables.length === 0) { + return new Map() + } + + const rows = await d1All(database.prepare(` + SELECT id, checksum + FROM i0c_data_repository_migration + ORDER BY id ASC + `)) + return new Map(rows.map((row) => [row.id, row.checksum])) +} + +async function ensureMigrationTable(database: D1Database): Promise { + await database.exec(` + CREATE TABLE IF NOT EXISTS i0c_data_repository_migration ( + id TEXT PRIMARY KEY, + checksum TEXT NOT NULL, + applied_at TEXT NOT NULL DEFAULT (STRFTIME('%Y-%m-%dT%H:%M:%fZ', 'now')) + ); + `) +} + +async function verifyAppliedMigrationChecksums( + migrations: readonly D1DataRepositoryMigration[], + applied: ReadonlyMap, +): Promise { + for (const migration of migrations) { + const appliedChecksum = applied.get(migration.id) + if (!appliedChecksum) { + continue + } + const expectedChecksum = await createMigrationChecksum(migration.sql) + if (appliedChecksum !== expectedChecksum) { + throw new Error(`D1 data repository migration checksum mismatch: ${migration.id}`) + } + } +} + +function validateAppliedHistory( + migrations: readonly D1DataRepositoryMigration[], + applied: ReadonlyMap, +): void { + assertContinuousMigrationHistory( + migrations.map((migration) => migration.id), + new Set(applied.keys()), + ) +} + +function resolveCurrentVersion( + migrations: readonly D1DataRepositoryMigration[], + applied: ReadonlyMap, +): string | null { + return [...migrations] + .reverse() + .find((migration) => applied.has(migration.id)) + ?.id ?? null +} + +function resolveTargetVersion( + migrations: readonly D1DataRepositoryMigration[], +): string { + const target = migrations.at(-1)?.id + if (!target) { + throw new Error("No D1 data repository migrations were provided") + } + return target +} + +async function createMigrationChecksum(sql: string): Promise { + const digest = await crypto.subtle.digest( + "SHA-256", + new TextEncoder().encode(sql), + ) + return toHex(digest) +} + +function toHex(value: ArrayBuffer): string { + return [...new Uint8Array(value)] + .map((item) => item.toString(16).padStart(2, "0")) + .join("") +} diff --git a/plugins/repository/d1/src/repository.ts b/plugins/repository/d1/src/repository.ts new file mode 100644 index 0000000..16a7b51 --- /dev/null +++ b/plugins/repository/d1/src/repository.ts @@ -0,0 +1,759 @@ +import { + DataDocumentNotFoundError, + DataRepositoryConflictError, + DataRepositoryInitializationError, + type DataDocument, + type DataDocumentKind, + type DataDocumentRevision, + type DataDocumentRevisionOperation, + type DataDocumentRevisionSummary, + type DataRepositoryImportInput, + type DataRepositoryInitializeInput, + type DataRepositoryManagement, + type DataRepositoryReadOptions, + type DataRepositoryRestoreInput, + type DataRepositorySnapshot, + type DataRepositoryWriteInput, + type DataRepositoryWriteResult, +} from "@i0c/config" +import type { AtomicVersionedDataRepository } from "@i0c/plugin-api" + +import type { D1Database, D1PreparedStatement, D1Result } from "./d1" +import { d1All, d1Batch } from "./d1" +import { d1DataRepositoryManifest } from "./manifest" + +interface DataDocumentRow { + checksum: string + content: string + kind: DataDocumentKind + mutation_id: string + revision: number | string + updated_at: string +} + +interface DataDocumentRevisionRow extends DataDocumentRow { + actor_github_user_id: string | null + created_at: string + operation: DataDocumentRevisionOperation +} + +interface DataDocumentKindRow { + kind: DataDocumentKind +} + +interface TableNameRow { + name: string +} + +export interface D1DataRepositoryServices { + clock?: () => Date + createMutationId?: () => string +} + +export type D1DataRepository = AtomicVersionedDataRepository< + DataDocumentKind, + DataRepositoryReadOptions, + DataRepositoryWriteInput, + DataDocument, + DataRepositoryWriteResult, + DataRepositorySnapshot +> & { + management: DataRepositoryManagement +} + +export function createD1DataRepository( + database: D1Database, + services: D1DataRepositoryServices = {}, +): D1DataRepository { + const clock = services.clock ?? (() => new Date()) + const createMutationId = services.createMutationId ?? (() => crypto.randomUUID()) + + return { + async read(kind) { + return readDocument(database, kind) + }, + async readSnapshot() { + return readSnapshot(database) + }, + async write(kind, input) { + return writeDocument( + database, + kind, + input, + "save", + clock, + createMutationId, + ) + }, + management: { + async importSnapshot(input) { + return importSnapshot(database, input, clock, createMutationId) + }, + async initialize(input) { + return initializeDocuments(database, input, clock, createMutationId) + }, + async inspectSetupState() { + return inspectSetupState(database) + }, + async listRevisions(input) { + return listRevisions(database, input) + }, + async readRevision(input) { + return readRevision(database, input.kind, input.revision) + }, + async restore(input) { + return restoreRevision(database, input, clock, createMutationId) + }, + }, + } +} + +export const d1DataRepositoryPlugin = { + manifest: d1DataRepositoryManifest, + create: createD1DataRepository, +} + +async function writeDocument( + database: D1Database, + kind: DataDocumentKind, + input: DataRepositoryWriteInput, + operation: DataDocumentRevisionOperation, + clock: () => Date, + createMutationId: () => string, +): Promise { + assertD1Revision(input.expectedRevision) + assertActorGitHubUserId(input.actorGitHubUserId) + const checksum = await createChecksum(input.content) + const mutationId = createMutationId() + const updatedAt = clock().toISOString() + const mutation = input.expectedRevision === "0" + ? database.prepare(` + INSERT INTO i0c_data_document ( + kind, + content, + revision, + checksum, + updated_at, + mutation_id + ) + VALUES (?, ?, 1, ?, ?, ?) + ON CONFLICT (kind) DO NOTHING + `).bind(kind, input.content, checksum, updatedAt, mutationId) + : database.prepare(` + UPDATE i0c_data_document + SET + content = ?, + revision = revision + 1, + checksum = ?, + updated_at = ?, + mutation_id = ? + WHERE kind = ? AND revision = ? + `).bind( + input.content, + checksum, + updatedAt, + mutationId, + kind, + toRevisionBinding(input.expectedRevision), + ) + + const results = await d1Batch(database, [ + mutation, + createRevisionInsert( + database, + operation, + input.actorGitHubUserId, + "kind = ? AND mutation_id = ?", + kind, + mutationId, + ), + ]) + + if (!hasExpectedChanges(results[0], 1)) { + await throwWriteConflict(database, kind, input.expectedRevision) + } + return { revision: incrementRevision(input.expectedRevision) } +} + +async function initializeDocuments( + database: D1Database, + input: DataRepositoryInitializeInput, + clock: () => Date, + createMutationId: () => string, +): Promise { + assertActorGitHubUserId(input.actorGitHubUserId) + const mutationId = createMutationId() + const updatedAt = clock().toISOString() + const configChecksum = await createChecksum(input.configContent) + const redirectsChecksum = await createChecksum(input.redirectsContent) + + const results = await d1Batch(database, [ + database.prepare(` + INSERT INTO i0c_data_document ( + kind, + content, + revision, + checksum, + updated_at, + mutation_id + ) + SELECT 'config', ?, 1, ?, ?, ? + WHERE NOT EXISTS ( + SELECT 1 + FROM i0c_data_document + WHERE kind IN ('config', 'redirects') + ) + UNION ALL + SELECT 'redirects', ?, 1, ?, ?, ? + WHERE NOT EXISTS ( + SELECT 1 + FROM i0c_data_document + WHERE kind IN ('config', 'redirects') + ) + `).bind( + input.configContent, + configChecksum, + updatedAt, + mutationId, + input.redirectsContent, + redirectsChecksum, + updatedAt, + mutationId, + ), + createRevisionInsert( + database, + "initialize", + input.actorGitHubUserId, + "mutation_id = ?", + mutationId, + ), + ]) + + if (!hasExpectedChanges(results[0], 2)) { + throw new DataRepositoryInitializationError( + "The data repository has already been initialized", + ) + } + return readSnapshotAtRevisions(database, "1", "1") +} + +async function importSnapshot( + database: D1Database, + input: DataRepositoryImportInput, + clock: () => Date, + createMutationId: () => string, +): Promise { + assertD1Revision(input.expectedConfigRevision) + assertD1Revision(input.expectedRedirectsRevision) + assertActorGitHubUserId(input.actorGitHubUserId) + const mutationId = createMutationId() + const updatedAt = clock().toISOString() + const configChecksum = await createChecksum(input.configContent) + const redirectsChecksum = await createChecksum(input.redirectsContent) + + const results = await d1Batch(database, [ + database.prepare(` + UPDATE i0c_data_document + SET + content = CASE kind + WHEN 'config' THEN ? + ELSE ? + END, + revision = revision + 1, + checksum = CASE kind + WHEN 'config' THEN ? + ELSE ? + END, + updated_at = ?, + mutation_id = ? + WHERE + kind IN ('config', 'redirects') + AND EXISTS ( + SELECT 1 + FROM i0c_data_document + WHERE kind = 'config' AND revision = ? + ) + AND EXISTS ( + SELECT 1 + FROM i0c_data_document + WHERE kind = 'redirects' AND revision = ? + ) + `).bind( + input.configContent, + input.redirectsContent, + configChecksum, + redirectsChecksum, + updatedAt, + mutationId, + toRevisionBinding(input.expectedConfigRevision), + toRevisionBinding(input.expectedRedirectsRevision), + ), + createRevisionInsert( + database, + "import", + input.actorGitHubUserId, + "mutation_id = ?", + mutationId, + ), + ]) + + if (!hasExpectedChanges(results[0], 2)) { + await assertExpectedRevision( + database, + "config", + input.expectedConfigRevision, + ) + await assertExpectedRevision( + database, + "redirects", + input.expectedRedirectsRevision, + ) + throw new DataRepositoryInitializationError( + "The D1 data repository import did not update both documents", + ) + } + + return readSnapshotAtRevisions( + database, + incrementRevision(input.expectedConfigRevision), + incrementRevision(input.expectedRedirectsRevision), + ) +} + +async function restoreRevision( + database: D1Database, + input: DataRepositoryRestoreInput, + clock: () => Date, + createMutationId: () => string, +): Promise { + assertD1Revision(input.revision) + assertD1Revision(input.expectedRevision) + assertActorGitHubUserId(input.actorGitHubUserId) + const mutationId = createMutationId() + const updatedAt = clock().toISOString() + + const results = await d1Batch(database, [ + database.prepare(` + UPDATE i0c_data_document + SET + content = ( + SELECT content + FROM i0c_data_document_revision + WHERE kind = ? AND revision = ? + ), + checksum = ( + SELECT checksum + FROM i0c_data_document_revision + WHERE kind = ? AND revision = ? + ), + revision = revision + 1, + updated_at = ?, + mutation_id = ? + WHERE + kind = ? + AND revision = ? + AND EXISTS ( + SELECT 1 + FROM i0c_data_document_revision + WHERE kind = ? AND revision = ? + ) + `).bind( + input.kind, + toRevisionBinding(input.revision), + input.kind, + toRevisionBinding(input.revision), + updatedAt, + mutationId, + input.kind, + toRevisionBinding(input.expectedRevision), + input.kind, + toRevisionBinding(input.revision), + ), + createRevisionInsert( + database, + "rollback", + input.actorGitHubUserId, + "kind = ? AND mutation_id = ?", + input.kind, + mutationId, + ), + ]) + + if (!hasExpectedChanges(results[0], 1)) { + await requireRevisionRow(database, input.kind, input.revision) + await assertExpectedRevision(database, input.kind, input.expectedRevision) + throw new DataRepositoryInitializationError( + "The D1 data repository restore did not update the document", + ) + } + return { revision: incrementRevision(input.expectedRevision) } +} + +async function inspectSetupState( + database: D1Database, +): Promise< + | { state: "migration-required" } + | { existingKinds: readonly DataDocumentKind[]; state: "empty" | "partial" } + | { state: "initialized" } +> { + const tables = await d1All(database.prepare(` + SELECT name + FROM sqlite_master + WHERE + type = 'table' + AND name IN ('i0c_data_document', 'i0c_data_document_revision') + ORDER BY name ASC + `)) + if (tables.length !== 2) { + return { state: "migration-required" } + } + + const rows = await d1All(database.prepare(` + SELECT kind + FROM i0c_data_document + WHERE kind IN ('config', 'redirects') + ORDER BY kind ASC + `)) + const existingKinds = rows.map((row) => row.kind) + if (existingKinds.length === 0) { + return { state: "empty", existingKinds } + } + if (existingKinds.length === 2) { + return { state: "initialized" } + } + return { state: "partial", existingKinds } +} + +async function listRevisions( + database: D1Database, + input: { + beforeRevision?: string + kind: DataDocumentKind + limit?: number + }, +): Promise { + const limit = input.limit ?? 50 + if (!Number.isInteger(limit) || limit < 1 || limit > 100) { + throw new TypeError("Revision list limit must be an integer from 1 to 100") + } + if (input.beforeRevision !== undefined) { + assertD1Revision(input.beforeRevision) + } + const statement = input.beforeRevision === undefined + ? database.prepare(` + SELECT + kind, + revision, + checksum, + operation, + actor_github_user_id, + created_at, + content, + created_at AS updated_at, + '' AS mutation_id + FROM i0c_data_document_revision + WHERE kind = ? + ORDER BY revision DESC + LIMIT ? + `).bind(input.kind, limit) + : database.prepare(` + SELECT + kind, + revision, + checksum, + operation, + actor_github_user_id, + created_at, + content, + created_at AS updated_at, + '' AS mutation_id + FROM i0c_data_document_revision + WHERE kind = ? AND revision < ? + ORDER BY revision DESC + LIMIT ? + `).bind( + input.kind, + toRevisionBinding(input.beforeRevision), + limit, + ) + return (await d1All(statement)).map( + toRevisionSummary, + ) +} + +async function readRevision( + database: D1Database, + kind: DataDocumentKind, + revision: string, +): Promise { + assertD1Revision(revision) + const row = await requireRevisionRow(database, kind, revision) + return { + ...toRevisionSummary(row), + content: row.content, + } +} + +async function readDocument( + database: D1Database, + kind: DataDocumentKind, +): Promise { + const rows = await d1All(database.prepare(` + SELECT kind, content, revision, checksum, updated_at, mutation_id + FROM i0c_data_document + WHERE kind = ? + `).bind(kind)) + const row = rows[0] + if (!row) { + throw new DataDocumentNotFoundError(kind) + } + return toDataDocument(row) +} + +async function readSnapshot( + database: D1Database, +): Promise { + const rows = await d1All(database.prepare(` + SELECT kind, content, revision, checksum, updated_at, mutation_id + FROM i0c_data_document + WHERE kind IN ('config', 'redirects') + ORDER BY kind ASC + `)) + return toSnapshot( + requireDocumentRow(rows, "config"), + requireDocumentRow(rows, "redirects"), + ) +} + +async function readSnapshotAtRevisions( + database: D1Database, + configRevision: string, + redirectsRevision: string, +): Promise { + const rows = await d1All(database.prepare(` + SELECT + kind, + content, + revision, + checksum, + created_at AS updated_at, + '' AS mutation_id + FROM i0c_data_document_revision + WHERE + (kind = 'config' AND revision = ?) + OR (kind = 'redirects' AND revision = ?) + ORDER BY kind ASC + `).bind( + toRevisionBinding(configRevision), + toRevisionBinding(redirectsRevision), + )) + return toSnapshot( + requireDocumentRow(rows, "config"), + requireDocumentRow(rows, "redirects"), + ) +} + +async function requireRevisionRow( + database: D1Database, + kind: DataDocumentKind, + revision: string, +): Promise { + const rows = await d1All(database.prepare(` + SELECT + kind, + revision, + checksum, + operation, + actor_github_user_id, + created_at, + content, + created_at AS updated_at, + '' AS mutation_id + FROM i0c_data_document_revision + WHERE kind = ? AND revision = ? + `).bind(kind, toRevisionBinding(revision))) + const row = rows[0] + if (!row) { + throw new DataDocumentNotFoundError(kind) + } + return row +} + +function createRevisionInsert( + database: D1Database, + operation: DataDocumentRevisionOperation, + actorGitHubUserId: string | undefined, + predicate: string, + ...values: readonly unknown[] +): D1PreparedStatement { + return database.prepare(` + INSERT INTO i0c_data_document_revision ( + kind, + revision, + content, + checksum, + operation, + actor_github_user_id, + created_at + ) + SELECT + kind, + revision, + content, + checksum, + ?, + ?, + updated_at + FROM i0c_data_document + WHERE ${predicate} + ORDER BY kind ASC + `).bind(operation, actorGitHubUserId ?? null, ...values) +} + +function hasExpectedChanges( + result: D1Result | undefined, + expected: number, +): boolean { + return Number(result?.meta?.changes ?? 0) === expected +} + +async function assertExpectedRevision( + database: D1Database, + kind: DataDocumentKind, + expectedRevision: string, +): Promise { + const rows = await d1All<{ revision: number | string }>(database.prepare(` + SELECT revision + FROM i0c_data_document + WHERE kind = ? + `).bind(kind)) + const current = rows[0] + if (!current) { + throw new DataDocumentNotFoundError(kind) + } + const actualRevision = normalizeRevision(current.revision) + if (actualRevision !== expectedRevision) { + throw new DataRepositoryConflictError( + kind, + expectedRevision, + actualRevision, + ) + } +} + +async function throwWriteConflict( + database: D1Database, + kind: DataDocumentKind, + expectedRevision: string, +): Promise { + await assertExpectedRevision(database, kind, expectedRevision) + throw new DataRepositoryInitializationError( + "The D1 data repository write did not update the document", + ) +} + +function requireDocumentRow( + rows: readonly DataDocumentRow[], + kind: DataDocumentKind, +): DataDocumentRow { + const row = rows.find((candidate) => candidate.kind === kind) + if (!row) { + throw new DataDocumentNotFoundError(kind) + } + return row +} + +async function toSnapshot( + config: DataDocumentRow, + redirects: DataDocumentRow, +): Promise { + return { + config: toDataDocument(config), + redirects: toDataDocument(redirects), + revision: await createSnapshotRevision(config, redirects), + } +} + +function toRevisionSummary( + row: DataDocumentRevisionRow, +): DataDocumentRevisionSummary { + return { + ...(row.actor_github_user_id + ? { actorGitHubUserId: row.actor_github_user_id } + : {}), + checksum: row.checksum, + createdAt: new Date(row.created_at).toISOString(), + kind: row.kind, + operation: row.operation, + revision: normalizeRevision(row.revision), + } +} + +function toDataDocument(row: DataDocumentRow): DataDocument { + return { + content: row.content, + revision: normalizeRevision(row.revision), + } +} + +function assertActorGitHubUserId( + actorGitHubUserId: string | undefined, +): void { + if ( + actorGitHubUserId !== undefined + && !/^[1-9]\d*$/.test(actorGitHubUserId) + ) { + throw new TypeError("Actor GitHub user ID must be a numeric ID") + } +} + +function assertD1Revision(revision: string): void { + if (!/^(?:0|[1-9]\d*)$/.test(revision)) { + throw new TypeError( + "D1 data repository revisions must be non-negative integers", + ) + } +} + +function incrementRevision(revision: string): string { + return (BigInt(revision) + 1n).toString() +} + +function toRevisionBinding(revision: string): number | string { + const numeric = BigInt(revision) + return numeric <= BigInt(Number.MAX_SAFE_INTEGER) + ? Number(numeric) + : revision +} + +async function createChecksum(content: string): Promise { + const digest = await crypto.subtle.digest( + "SHA-256", + new TextEncoder().encode(content), + ) + return toHex(digest) +} + +async function createSnapshotRevision( + config: DataDocumentRow, + redirects: DataDocumentRow, +): Promise { + const digest = await crypto.subtle.digest( + "SHA-256", + new TextEncoder().encode([ + `config:${normalizeRevision(config.revision)}:${config.checksum}`, + `redirects:${normalizeRevision(redirects.revision)}:${redirects.checksum}`, + ].join("\n")), + ) + return toHex(digest) +} + +function toHex(value: ArrayBuffer): string { + return [...new Uint8Array(value)] + .map((item) => item.toString(16).padStart(2, "0")) + .join("") +} + +function normalizeRevision(revision: number | string): string { + return String(revision) +} diff --git a/plugins/repository/d1/tests/d1-repository.test.ts b/plugins/repository/d1/tests/d1-repository.test.ts new file mode 100644 index 0000000..c347caf --- /dev/null +++ b/plugins/repository/d1/tests/d1-repository.test.ts @@ -0,0 +1,244 @@ +import assert from "node:assert/strict" +import { readFile } from "node:fs/promises" +import { fileURLToPath } from "node:url" +import test from "node:test" + +import { + assertManagedDataRepositoryBehaviorContract, + assertMigrationState, + assertPluginManifest, +} from "@i0c/plugin-testkit" + +import { d1All } from "../src/d1" +import { d1DataRepositoryManifest } from "../src/manifest" +import { + createD1DataRepositoryMigrationProvider, + type D1DataRepositoryMigration, +} from "../src/migrations" +import { createD1DataRepository } from "../src/repository" +import { SQLiteD1Database } from "./sqlite-d1" + +test("declares a valid D1 data repository manifest", () => { + assertPluginManifest(d1DataRepositoryManifest) +}) + +test("owns and applies independent D1 data repository migrations", async () => { + const database = new SQLiteD1Database() + try { + const provider = createD1DataRepositoryMigrationProvider( + database, + await loadMigrations(), + ) + await assertMigrationState(provider, "002_data_document_history.sql") + const result = await provider.applyMigrations({ + expectedCurrentVersion: null, + }) + assert.deepEqual(result.applied, [ + "001_data_documents.sql", + "002_data_document_history.sql", + ]) + await assertMigrationState(provider, "002_data_document_history.sql") + } finally { + database.close() + } +}) + +test("satisfies the managed data repository behavior contract", async () => { + const database = new SQLiteD1Database() + try { + await createD1DataRepositoryMigrationProvider( + database, + await loadMigrations(), + ).applyMigrations() + await assertManagedDataRepositoryBehaviorContract( + createD1DataRepository(database), + ) + } finally { + database.close() + } +}) + +test("rolls back a document write when its history insert fails", async () => { + const database = new SQLiteD1Database() + try { + await createD1DataRepositoryMigrationProvider( + database, + await loadMigrations(), + ).applyMigrations() + const repository = createD1DataRepository(database) + await repository.management.initialize({ + actorGitHubUserId: "123", + configContent: "{\"schemaVersion\":1}", + redirectsContent: "{\"Slots\":{}}", + }) + const before = await repository.readSnapshot({}) + + database.failNextBatchAt(1) + await assert.rejects( + repository.write("config", { + actorGitHubUserId: "123", + content: "{\"schemaVersion\":1,\"mustNotPersist\":true}", + expectedRevision: "1", + }), + /Injected D1 batch failure/, + ) + + assert.deepEqual(await repository.readSnapshot({}), before) + assert.deepEqual( + (await repository.management.listRevisions({ + kind: "config", + limit: 10, + })).map((revision) => revision.revision), + ["1"], + ) + } finally { + database.close() + } +}) + +test("rolls back both imported documents when history insertion fails", async () => { + const database = new SQLiteD1Database() + try { + await createD1DataRepositoryMigrationProvider( + database, + await loadMigrations(), + ).applyMigrations() + const repository = createD1DataRepository(database) + await repository.management.initialize({ + actorGitHubUserId: "123", + configContent: "{\"schemaVersion\":1}", + redirectsContent: "{\"Slots\":{}}", + }) + const before = await repository.readSnapshot({}) + + database.failNextBatchAt(1) + await assert.rejects( + repository.management.importSnapshot({ + actorGitHubUserId: "123", + configContent: "{\"schemaVersion\":1,\"mustNotPersist\":true}", + expectedConfigRevision: "1", + expectedRedirectsRevision: "1", + redirectsContent: "{\"Slots\":{\"/\":\"https://example.com\"}}", + }), + /Injected D1 batch failure/, + ) + + assert.deepEqual(await repository.readSnapshot({}), before) + } finally { + database.close() + } +}) + +test("rolls back a failed D1 data repository migration and version record", async () => { + const database = new SQLiteD1Database() + try { + const provider = createD1DataRepositoryMigrationProvider(database, [{ + id: "001_failure.sql", + sql: ` + CREATE TABLE partial_data_repository_migration (id TEXT PRIMARY KEY); + -- d1-statement-breakpoint + INSERT INTO missing_table (id) VALUES ('failure'); + `, + }]) + + await assert.rejects( + async () => provider.applyMigrations(), + /missing_table/, + ) + + assert.deepEqual( + await d1All<{ name: string }>(database.prepare(` + SELECT name + FROM sqlite_master + WHERE type = 'table' AND name = 'partial_data_repository_migration' + `)), + [], + ) + assert.equal( + (await d1All<{ count: number }>(database.prepare(` + SELECT COUNT(*) AS count + FROM i0c_data_repository_migration + `)))[0]?.count, + 0, + ) + } finally { + database.close() + } +}) + +test("rejects drift, gaps, and future D1 data repository migrations", async () => { + const database = new SQLiteD1Database() + try { + const migrations = await loadMigrations() + const provider = createD1DataRepositoryMigrationProvider( + database, + migrations, + ) + await provider.applyMigrations() + + const drifted = createD1DataRepositoryMigrationProvider(database, [ + migrations[0], + { + ...migrations[1], + sql: `${migrations[1]?.sql ?? ""}\n-- drift`, + }, + ].filter((migration): migration is D1DataRepositoryMigration => + migration !== undefined + )) + await assert.rejects( + async () => drifted.migrationStatus(), + /migration checksum mismatch/, + ) + + database.database.prepare(` + UPDATE i0c_data_repository_migration + SET checksum = ? + WHERE id = '002_data_document_history.sql' + `).run(await checksum(migrations[1]?.sql ?? "")) + database.database.prepare(` + DELETE FROM i0c_data_repository_migration + WHERE id = '001_data_documents.sql' + `).run() + await assert.rejects( + async () => provider.migrationPlan(), + /not a continuous prefix/, + ) + + database.database.prepare(` + DELETE FROM i0c_data_repository_migration + `).run() + database.database.prepare(` + INSERT INTO i0c_data_repository_migration (id, checksum) + VALUES ('999_future.sql', 'future') + `).run() + await assert.rejects( + async () => provider.migrationStatus(), + /unknown applied migration/, + ) + } finally { + database.close() + } +}) + +async function loadMigrations(): Promise { + return Promise.all([ + "001_data_documents.sql", + "002_data_document_history.sql", + ].map(async (id) => ({ + id, + sql: await readFile( + fileURLToPath(new URL(`../migrations/${id}`, import.meta.url)), + "utf8", + ), + }))) +} + +async function checksum(sql: string): Promise { + const digest = await crypto.subtle.digest( + "SHA-256", + new TextEncoder().encode(sql), + ) + return [...new Uint8Array(digest)] + .map((item) => item.toString(16).padStart(2, "0")) + .join("") +} diff --git a/plugins/repository/d1/tests/sqlite-d1.ts b/plugins/repository/d1/tests/sqlite-d1.ts new file mode 100644 index 0000000..7575431 --- /dev/null +++ b/plugins/repository/d1/tests/sqlite-d1.ts @@ -0,0 +1,124 @@ +import { + DatabaseSync, + type SQLInputValue, +} from "node:sqlite" + +import type { + D1Database, + D1ExecResult, + D1PreparedStatement, + D1Result, +} from "../src/d1" + +export class SQLiteD1Database implements D1Database { + readonly database = new DatabaseSync(":memory:") + private nextBatchFailureIndex: number | undefined + + prepare(query: string): D1PreparedStatement { + return new SQLiteD1Statement(this.database, query) + } + + async batch>( + statements: readonly D1PreparedStatement[], + ): Promise[]> { + this.database.exec("BEGIN IMMEDIATE") + try { + const results = statements.map((statement, index) => { + if (index === this.nextBatchFailureIndex) { + throw new Error(`Injected D1 batch failure at statement ${index}`) + } + if (!(statement instanceof SQLiteD1Statement)) { + throw new Error("SQLite D1 tests received an incompatible statement") + } + return statement.runSync() + }) + this.nextBatchFailureIndex = undefined + this.database.exec("COMMIT") + return results + } catch (error) { + this.nextBatchFailureIndex = undefined + this.database.exec("ROLLBACK") + throw error + } + } + + async exec(query: string): Promise { + const startedAt = performance.now() + this.database.exec(query) + return { count: 0, duration: performance.now() - startedAt } + } + + close(): void { + this.database.close() + } + + failNextBatchAt(index: number): void { + this.nextBatchFailureIndex = index + } +} + +class SQLiteD1Statement implements D1PreparedStatement { + constructor( + private readonly database: DatabaseSync, + private readonly query: string, + private readonly values: readonly SQLInputValue[] = [], + ) {} + + bind(...values: readonly unknown[]): D1PreparedStatement { + return new SQLiteD1Statement( + this.database, + this.query, + values.map(toSqliteValue), + ) + } + + async all>(): Promise> { + const statement = this.database.prepare(this.query) + return { + success: true, + results: statement.all(...this.values) as T[], + meta: { changes: 0 }, + } + } + + async first>( + columnName?: string, + ): Promise { + const statement = this.database.prepare(this.query) + const row = statement.get(...this.values) as Record | undefined + if (!row) { + return null + } + return (columnName ? row[columnName] : row) as T + } + + async run>(): Promise> { + return this.runSync() + } + + runSync>(): D1Result { + const statement = this.database.prepare(this.query) + const result = statement.run(...this.values) + return { + success: true, + results: [], + meta: { changes: Number(result.changes) }, + } + } +} + +function toSqliteValue(value: unknown): SQLInputValue { + if ( + value === null + || typeof value === "string" + || typeof value === "number" + || typeof value === "bigint" + || value instanceof Uint8Array + ) { + return value + } + if (typeof value === "boolean") { + return value ? 1 : 0 + } + throw new TypeError(`Unsupported SQLite binding value: ${String(value)}`) +} diff --git a/plugins/repository/d1/tsconfig.json b/plugins/repository/d1/tsconfig.json new file mode 100644 index 0000000..a4d78e7 --- /dev/null +++ b/plugins/repository/d1/tsconfig.json @@ -0,0 +1,13 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "Bundler", + "strict": true, + "noEmit": true, + "isolatedModules": true, + "skipLibCheck": true, + "types": ["node"] + }, + "include": ["src/**/*.ts", "tests/**/*.ts"] +} diff --git a/plugins/repository/postgres/tests/postgres-repository.test.ts b/plugins/repository/postgres/tests/postgres-repository.test.ts index e1255a7..08aa5e4 100644 --- a/plugins/repository/postgres/tests/postgres-repository.test.ts +++ b/plugins/repository/postgres/tests/postgres-repository.test.ts @@ -5,6 +5,7 @@ import postgres, { type Sql } from "postgres" import { DataRepositoryConflictError } from "@i0c/config" import { + assertManagedDataRepositoryBehaviorContract, assertPluginManifest, assertVersionedDataRepositoryContract, } from "@i0c/plugin-testkit" @@ -176,11 +177,16 @@ test("initializes, imports, and restores immutable document revisions", async () }) const integrationConnectionString = - process.env.TEST_DATABASE_URL?.trim() + process.env.TEST_POSTGRES_URL?.trim() + ?? process.env.TEST_DATABASE_URL?.trim() test( "satisfies the versioned repository contract with PostgreSQL", - { skip: integrationConnectionString ? false : "TEST_DATABASE_URL is not set" }, + { + skip: integrationConnectionString + ? false + : "TEST_POSTGRES_URL and TEST_DATABASE_URL are not set", + }, async () => { const connectionString = integrationConnectionString assert.ok(connectionString) @@ -202,56 +208,7 @@ test( { connectionString }, { sql }, ) - assert.deepEqual( - await repository.write("config", { - content: "{\"schemaVersion\":1}", - expectedRevision: "0", - }), - { revision: "1" }, - ) - assert.deepEqual( - await repository.write("redirects", { - content: "{\"Slots\":{}}", - expectedRevision: "0", - }), - { revision: "1" }, - ) - - await assertVersionedDataRepositoryContract({ - repository, - kind: "config", - readOptions: {}, - writeInput: { - content: "{\"schemaVersion\":1,\"updated\":true}", - expectedRevision: "1", - }, - expectedBefore: { - content: "{\"schemaVersion\":1}", - revision: "1", - }, - expectedWriteResult: { revision: "2" }, - expectedAfter: { - content: "{\"schemaVersion\":1,\"updated\":true}", - revision: "2", - }, - }) - - const snapshot = await repository.readSnapshot({}) - assert.equal(snapshot.config.revision, "2") - assert.equal(snapshot.redirects.revision, "1") - assert.match(snapshot.revision, /^[0-9a-f]{64}$/) - - await assert.rejects( - repository.write("config", { - content: "{\"schemaVersion\":1,\"stale\":true}", - expectedRevision: "1", - }), - (error) => { - assert.ok(error instanceof DataRepositoryConflictError) - assert.equal(error.actualRevision, "2") - return true - }, - ) + await assertManagedDataRepositoryBehaviorContract(repository) } finally { await sql`DELETE FROM i0c_data_document_revision` await sql`DELETE FROM i0c_data_document` diff --git a/plugins/source/github/src/manifest.ts b/plugins/source/github/src/manifest.ts index 25315b8..4aa5b5f 100644 --- a/plugins/source/github/src/manifest.ts +++ b/plugins/source/github/src/manifest.ts @@ -49,6 +49,8 @@ export const githubContentsRepositoryManifest = { "redirects:read", "redirects:write", "version:optimistic", + "ui:redirects-json-editor", + "ui:redirects-source-override", ], description: { summary: { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 5efaf9e..c427b41 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -53,6 +53,9 @@ importers: '@i0c/plugin-api': specifier: workspace:* version: link:packages/plugin-api + '@i0c/plugin-data-repository-d1': + specifier: workspace:* + version: link:plugins/repository/d1 '@i0c/plugin-data-repository-postgres': specifier: workspace:* version: link:plugins/repository/postgres @@ -153,6 +156,9 @@ importers: '@i0c/plugin-catalog': specifier: workspace:* version: link:../../packages/plugin-catalog + '@i0c/plugin-data-repository-d1': + specifier: workspace:* + version: link:../../plugins/repository/d1 '@i0c/plugin-data-repository-postgres': specifier: workspace:* version: link:../../plugins/repository/postgres @@ -284,6 +290,9 @@ importers: '@i0c/plugin-api': specifier: workspace:* version: link:../plugin-api + '@i0c/plugin-data-repository-d1': + specifier: workspace:* + version: link:../../plugins/repository/d1 '@i0c/plugin-data-repository-postgres': specifier: workspace:* version: link:../../plugins/repository/postgres @@ -318,6 +327,9 @@ importers: packages/plugin-testkit: dependencies: + '@i0c/config': + specifier: workspace:* + version: link:../config '@i0c/plugin-api': specifier: workspace:* version: link:../plugin-api @@ -454,6 +466,28 @@ importers: specifier: ^6.0.2 version: 6.0.2 + plugins/repository/d1: + dependencies: + '@i0c/config': + specifier: workspace:* + version: link:../../../packages/config + '@i0c/plugin-api': + specifier: workspace:* + version: link:../../../packages/plugin-api + devDependencies: + '@i0c/plugin-testkit': + specifier: workspace:* + version: link:../../../packages/plugin-testkit + '@types/node': + specifier: ^25.5.0 + version: 25.5.0 + tsx: + specifier: ^4.21.0 + version: 4.23.1 + typescript: + specifier: ^6.0.2 + version: 6.0.2 + plugins/repository/postgres: dependencies: '@i0c/config':