chore(db): PostgreSQL 迁移统一分支 by cropflre · Pull Request #299 · cropflre/nowen-note

cropflre · 2026-07-15T07:02:55Z

目的

所有未完成的 PostgreSQL 迁移统一在 pg-migration-unified 推进。PR 继续保持 Draft；当前仍为 PostgreSQL runtime-only，不宣称整个项目已可生产切换。

businessRoutesReady = false
PostgreSQL Note Transfer 已完成统一异步编排
SQLite → PostgreSQL 已完成预检、可恢复复制、独立校验、空/非空目标 rollback、并发修改保护、规模化演练与最终报告
搜索/向量、PostgreSQL 备份恢复与正式部署切换仍未完成

已完成的 PostgreSQL Runtime

Notes

已覆盖列表、详情、创建、更新、移动、排序、回收站、恢复、永久删除及删除副作用。

Knowledge Tree

已覆盖 scope listing、shared roots、创建、更新、移动、排序、删除、恢复、权限、历史、文件夹密码与 unlock。

Note Transfer（#249）

preview
→ prepare
→ attachment staging
→ atomic target commit
→ audit/webhook/realtime effects
→ move source deletion
→ completed

对应 immutable migrations：0016～0022。

SQLite → PostgreSQL 数据迁移（#251）

第一切片：预检与持久化基础

PR feat(pg): SQLite → PostgreSQL migration preflight foundation #675
提交：674955842029a00388e5fccd62150433f0d36bf2
migration：0023_sqlite_postgres_migration_runs.sql

覆盖只读 source inspection、备份一致性、目标 schema/非空检查、依赖排序、幂等计划、table/batch checkpoint、lease 与 expired lease takeover。

第二切片：可恢复复制与独立校验

PR feat(pg): add resumable SQLite copy executor #678
提交：108885a3ac7aa575de5d25dafd1f9a6cf764e8e3

覆盖冻结备份、主键 keyset cursor、事务化 upsert/checkpoint、restart/resume、类型转换、自引用修复、identity sequence、checksum、copy-time verify 与 independent verify。

第三切片：run-owned tracking 与空目标 rollback

PR feat(pg): add resumable SQLite migration rollback #682
提交：ac8324698c2a766eba706bda8ce37a9fb5b245c0
migration：0024_sqlite_postgres_migration_rollback.sql

覆盖 run-owned table/primary-key 记录、反向依赖 rollback、bounded interruption/resume、残留验证与幂等 replay。

第四切片：非空目标冲突与原值恢复

PR feat(pg): add non-empty SQLite migration recovery #683
提交：cbe6be01b3c46c25a0f2fa1b0bb9e9eca8ed7943
migration：0025_sqlite_postgres_nonempty_recovery.sql

覆盖 abort / overwrite-with-backup 冲突策略、inserted / updated / unchanged 分类、类型安全 originalRow、精确 migratedRow、非空目标 verify、updated 恢复与迁移后并发修改保护。

第五切片：规模化演练与最终报告

PR feat(pg): add scalable SQLite migration drill and final reports #687
提交：93aeff98c369ca0b187056e3754ab951d99c6449

完整演练链路：

apply
→ independent verify
→ rollback
→ post-rollback primary-key / row-count / checksum validation
→ PostgreSQL foreign-key orphan validation
→ final report persistence

新增：

medium deterministic fixture：600 条业务数据
large deterministic fixture：2,000 条业务数据
bounded apply / rollback passes，验证进程级 restart/resume
同事务失败注入，验证业务行、ownership 与 checkpoint 一起回滚
空目标完整演练
非空目标 inserted / updated / unchanged / unrelated 数据恢复与保留演练
rollback 后逐表主键集合、行数与 canonical checksum 校验
PostgreSQL 外键孤儿检查
最终 JSON report，写入指定文件并持久化到 migration run report
inserted / updated / unchanged / deleted / restored / conflict / failure 汇总
source/backup 只读 mtime 校验
操作 runbook、磁盘空间建议与备份保留要求

Drill CLI：

DATABASE_URL='postgres://...' \
npm run migrate:sqlite-to-postgres:drill -- \
  --source /path/to/nowen-note.db \
  --backup /path/to/nowen-note.backup.db \
  --idempotency-key drill-2026-08-06 \
  --batch-size 200 \
  --max-batches-per-pass 10 \
  --max-rollback-batches-per-pass 10 \
  --report /secure/path/report.json

非空目标仍必须显式使用：

--allow-non-empty-target --conflict-policy overwrite-with-backup

#251 当前判断

迁移引擎的代码范围已基本覆盖：

只读预检与冻结备份
分批复制、幂等与断点续跑
copy-time / independent verify
空目标与非空目标 rollback
原值快照与并发修改保护
最终机器可读报告
失败注入
中型/大型数据集完整 drill
操作 runbook

#251 暂时保持打开，等待真实 staging 数据演练证据归档或维护者确认正式收口。

为什么仍不能生产切换

仍依赖：

[PG-SEARCH-01] 迁移全文搜索与向量检索到 PostgreSQL #252：搜索 / FTS / 向量数据重建与校验
[PG-BACKUP-01] 重构 PostgreSQL 备份、恢复与灾难恢复链路 #253：PostgreSQL 备份、恢复和独立 restore verification
[PG-DEPLOY-01] 完成 PostgreSQL 正式部署、灰度切换与回滚演练 #254：部署、灰度、读写切换、回滚演练与正式 cutover

原始 SQLite 和冻结备份必须保留到迁移验证通过、rollback window 关闭且 PostgreSQL 备份完成独立恢复验证。

下一阶段

从 #251 的迁移引擎开发转入 #252：

定义 PostgreSQL 搜索索引、FTS 和向量重建的唯一状态源
将搜索/向量构建纳入可恢复任务编排
完成逐笔记/逐 chunk 校验、断点续跑和重建报告
验证搜索结果、排序与权限过滤一致性
再进入 [PG-BACKUP-01] 重构 PostgreSQL 备份、恢复与灾难恢复链路 #253 备份恢复和 [PG-DEPLOY-01] 完成 PostgreSQL 正式部署、灰度切换与回滚演练 #254 部署切换

PR 继续保持 Draft。

cropflre · 2026-07-16T15:23:51Z

✅ PostgreSQL access/session validation passed

Validated branch: pg-migration-unified
Validated head: 9085b5453e6bd012aa60aab8229b63b0e43a1622
Run: 29554768794

✅ npm ci
✅ TypeScript
✅ SQLite direct-access audit
✅ PostgreSQL schema parity
✅ SQL dialect tests
✅ Full SQLite regression
✅ Real PostgreSQL access/session
✅ Production backend bundle

github-actions · 2026-07-16T15:24:00Z

❌ PostgreSQL unified validation failed

Validated branch: pg-migration-unified
Validated head: 1ccaabf709f98dd8bb4bec92fbe5bc976206f9b5
Run: 29510812222

✅ npm ci
✅ TypeScript
✅ SQLite direct-access audit
❌ PostgreSQL schema parity（exit 1）
✅ Runtime / SQL dialect tests
✅ Full SQLite regression
✅ Real PostgreSQL tests
✅ Production backend bundle
✅ PostgreSQL runtime resources packaged

Full diagnostic log is attached to the workflow run artifact.

github-actions · 2026-07-16T15:25:49Z

❌ PostgreSQL unified validation failed

Validated branch: pg-migration-unified
Validated head: 1ccaabf709f98dd8bb4bec92fbe5bc976206f9b5
Run: 29510954347

✅ npm ci
✅ TypeScript
✅ SQLite direct-access audit
❌ PostgreSQL schema parity（exit 1）
✅ Runtime / SQL dialect tests
✅ Full SQLite regression
✅ Real PostgreSQL tests
✅ Production backend bundle
✅ PostgreSQL runtime resources packaged

Full diagnostic log is attached to the workflow run artifact.

cropflre · 2026-07-16T15:30:59Z

/validate-pg-unified

github-actions · 2026-07-16T15:32:59Z

❌ PostgreSQL unified validation failed

Validated branch: pg-migration-unified
Validated head: 3291f44ead31429e8d91f6fe1114704f5b64f5c2
Run: 29511494621

✅ npm ci
✅ TypeScript
✅ SQLite direct-access audit
❌ PostgreSQL schema parity（exit 1）
✅ Runtime / SQL dialect tests
✅ Full SQLite regression
✅ Real PostgreSQL tests
✅ Production backend bundle
✅ PostgreSQL runtime resources packaged

Full diagnostic log is attached to the workflow run artifact.

cropflre · 2026-07-16T15:35:02Z

/validate-pg-unified

github-actions · 2026-07-16T15:37:01Z

✅ PostgreSQL unified validation passed

Validated branch: pg-migration-unified
Validated head: 542935f50c66255470d77c4d38495007ad970cd5
Run: 29511804247

✅ npm ci
✅ TypeScript
✅ SQLite direct-access audit
✅ PostgreSQL schema parity
✅ Runtime / SQL dialect tests
✅ Full SQLite regression
✅ Real PostgreSQL tests
✅ Production backend bundle
✅ PostgreSQL runtime resources packaged

Full diagnostic log is attached to the workflow run artifact.

cropflre · 2026-07-16T15:38:50Z

/validate-pg-unified

github-actions · 2026-07-16T15:40:46Z

✅ PostgreSQL unified validation passed

Validated branch: pg-migration-unified
Validated head: b828395c3fd29ae901d52039ebc07ba39a6fdc1c
Run: 29512099142

✅ npm ci
✅ TypeScript
✅ SQLite direct-access audit
✅ PostgreSQL schema parity
✅ Runtime / SQL dialect tests
✅ Full SQLite regression
✅ Real PostgreSQL tests
✅ Production backend bundle
✅ PostgreSQL runtime resources packaged

Full diagnostic log is attached to the workflow run artifact.

cropflre · 2026-07-17T01:07:02Z

/validate-pg-unified-batch-b

github-actions · 2026-07-17T01:08:59Z

✅ PostgreSQL Batch B validation passed

Validated branch: pg-migration-unified
Validated head: 6bba9a0da836c3fe1a4f3679aa78efbfaab2f231
Run: 29546608777

✅ npm ci
✅ TypeScript
✅ SQLite direct-access audit
✅ PostgreSQL schema parity
✅ Runtime / SQL dialect tests
✅ Full SQLite regression
✅ Real PostgreSQL tests
✅ Production backend bundle

cropflre · 2026-07-17T01:09:09Z

/validate-pg-unified-batch-b

cropflre · 2026-07-17T01:10:50Z

/validate-pg-unified-batch-b-final

github-actions · 2026-07-17T01:11:07Z

✅ PostgreSQL Batch B validation passed

Validated branch: pg-migration-unified
Validated head: 2b6b705efecedc69222c1bb434eb4a5343d6ff31
Run: 29546701987

✅ npm ci
✅ TypeScript
✅ SQLite direct-access audit
✅ PostgreSQL schema parity
✅ Runtime / SQL dialect tests
✅ Full SQLite regression
✅ Real PostgreSQL tests
✅ Production backend bundle

cropflre · 2026-07-17T01:17:10Z

#249 Batch B 已落地

最终 Head：2b6b705efecedc69222c1bb434eb4a5343d6ff31
验证 Run：29546774716

本轮直接在 pg-migration-unified 完成：

userAISettingsRepository async API 接入 Runtime Provider，并改为惰性解析 Adapter，避免模块导入早于 Runtime 初始化；
taskDependenciesRepository async API 接入 Runtime Provider；
taskProjectsRepository async API 接入 Runtime Provider；
统一 SQLite / PostgreSQL 下项目统计字段的 number 返回；
项目移除流程通过 executeStatements 保证解除任务关联与项目变更位于同一事务；
新增 repository-batch-b-pg.test.ts，覆盖 AI 设置 upsert、项目统计/排序、任务依赖和事务行为；
Batch B 已加入常驻 pg-runtime.yml。

最终验证全部通过：npm ci、TypeScript、SQLite direct-access audit、PostgreSQL schema parity、Runtime/SQL dialect、完整 SQLite 回归、真实 PostgreSQL 回归、生产 backend bundle。

一次性验证工作流已从 main 清理。PR 继续保持 Draft；#249 其余 Repository、#250 深度 Schema parity、#251～#254 仍待推进。

cropflre · 2026-07-17T01:36:27Z

/validate-pg-permissions

cropflre · 2026-07-17T01:37:06Z

/validate-pg-permissions

cropflre · 2026-07-17T01:37:53Z

/validate-pg-permissions

cropflre · 2026-07-17T01:40:46Z

本轮已直接在 pg-migration-unified 完成权限域 PostgreSQL 迁移：note ACL、workspace members、notebook members 的 async API 已改用 Runtime Provider；新增 0004_notebook_members_unique.sql 和真实 PG 测试 repository-permissions-pg.test.ts；新增常驻 pg-permissions.yml 门禁。最终验证 Head 2c04bc531409334479834c897a6ac22f7f904958，Run 29547923197：TypeScript、SQLite 审计、Schema parity、完整 SQLite 回归、真实 PostgreSQL、生产构建和 migration 资源打包全部通过。一次性 main 验证工作流已删除。

cropflre · 2026-07-17T04:16:32Z

/validate-pg-access-session

cropflre · 2026-07-17T04:20:33Z

/validate-pg-access-session

cropflre · 2026-07-17T04:24:17Z

/validate-pg-access-session

This was referenced Jul 15, 2026

refactor(db): 清理业务层 SQLite 直连（#248） #258

Closed

feat(db): 建立 PostgreSQL 运行时基础 (#247) #255

Closed

cropflre closed this Jul 16, 2026

cropflre reopened this Jul 16, 2026

cropflre closed this Jul 16, 2026

cropflre reopened this Jul 16, 2026

This was referenced Jul 16, 2026

[PG-REPOSITORY-BATCH-01] 完成全量 Repository PostgreSQL 兼容与双库验证 #249

Open

[PG-SCHEMA-PARITY-01] 建立 SQLite / PostgreSQL Schema 一致性校验与迁移机制 #250

Open

cropflre and others added 30 commits August 13, 2026 13:01

refactor(pg): reuse database-free attachment signature core

07f68fc

test(pg): cover database-free attachment signatures

7a2669b

feat(pg): add attachment HTTP storage primitives

dd0b405

feat(pg): add attachment HTTP runtime

b63ab60

test(pg): cover attachment HTTP lifecycle

cb2cd5c

test(pg): exercise workspace attachment ACL revocation

b82558f

test(pg): make attachment signature tamper deterministic

20b181a

feat(pg): mount signed attachment runtime

c8e3be9

ci(pg): gate signed attachment runtime

1e83d92

feat(pg): complete tag write runtime

0db8d68

test(pg): cover tag write runtime

9017ee2

fix(pg): keep tag request bodies strongly typed

b0ef3b6

fix(pg): use explicit Hono context for tag bodies

d7d6290

feat(pg): add workspace create detail and update runtime

cd9ac08

test(pg): cover workspace create detail and update runtime

5d16474

ci(pg): gate tag write lifecycle

d579ea3

feat(pg): expose workspace and tag write surfaces

e280f3a

ci(pg): gate workspace write runtime

af09523

refactor(pg): add database-neutral task core repository

e032094

test(pg): validate task core repository foundation

4d3880d

fix(pg): align task repository with database adapter contracts

b93209a

ci(pg): gate task core repository foundation

cfcb84d

feat(pg): expose task core runtime with recurrence

69576e2

feat(pg): expose task projects dependencies and reminders

d81ec67

feat(pg): expose task templates runtime

f5bcd8a

TEMP no

19f7468

feat(pg): add durable task reminder delivery state

225d25f

feat(pg): wire durable task reminder delivery

4dfb0b0

feat(pg): persist task automation notifications

1e6681e

feat(pg): stage task AI breakdown runtime

d785cd5

Conversation

cropflre commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

目的

最新代码基线

已完成的 PostgreSQL Runtime

Notes

Knowledge Tree

Note Transfer（#249）

SQLite → PostgreSQL 数据迁移（#251）

第一切片：预检与持久化基础

第二切片：可恢复复制与独立校验

第三切片：run-owned tracking 与空目标 rollback

第四切片：非空目标冲突与原值恢复

第五切片：规模化演练与最终报告

相关验证

#251 当前判断

为什么仍不能生产切换

下一阶段

Uh oh!

cropflre commented Jul 16, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ PostgreSQL access/session validation passed

Uh oh!

github-actions Bot commented Jul 16, 2026

❌ PostgreSQL unified validation failed

Uh oh!

github-actions Bot commented Jul 16, 2026

❌ PostgreSQL unified validation failed

Uh oh!

cropflre commented Jul 16, 2026

Uh oh!

github-actions Bot commented Jul 16, 2026

❌ PostgreSQL unified validation failed

Uh oh!

cropflre commented Jul 16, 2026

Uh oh!

github-actions Bot commented Jul 16, 2026

✅ PostgreSQL unified validation passed

Uh oh!

cropflre commented Jul 16, 2026

Uh oh!

github-actions Bot commented Jul 16, 2026

✅ PostgreSQL unified validation passed

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

✅ PostgreSQL Batch B validation passed

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

github-actions Bot commented Jul 17, 2026

✅ PostgreSQL Batch B validation passed

Uh oh!

cropflre commented Jul 17, 2026

#249 Batch B 已落地

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

cropflre commented Jul 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

cropflre commented Jul 15, 2026 •

edited

Loading

cropflre commented Jul 16, 2026 •

edited by github-actions Bot

Loading