feat(storage): add PostgreSQL multi-replica store

.github/workflows/ci.yml

@@ -34,3 +34,42 @@ jobs:
         run: pnpm install --frozen-lockfile
       - name: Run quality gate
         run: pnpm check
+
+  postgres-integration:
+    name: postgres integration
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:17-alpine@sha256:742f40ea20b9ff2ff31db5458d127452988a2164df9e17441e191f3b72252193
+        env:
+          POSTGRES_DB: postgres
+          POSTGRES_HOST_AUTH_METHOD: trust
+          POSTGRES_USER: postgres
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U postgres -d postgres"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 12
+    env:
+      PAUSEMESH_REQUIRE_POSTGRES_TEST: "1"
+      PAUSEMESH_TEST_POSTGRES_URL: postgresql://postgres@127.0.0.1:5432/postgres
+    steps:
+      - name: Check out source
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+        with:
+          persist-credentials: false
+      - name: Install pnpm
+        uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
+        with:
+          version: 11.8.0
+      - name: Install Node.js
+        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+        with:
+          node-version: 24
+          cache: pnpm
+      - name: Install dependencies
+        run: pnpm install --frozen-lockfile
+      - name: Run PostgreSQL integration
+        run: pnpm test:postgres

CHANGELOG.md

@@ -3,6 +3,31 @@
 All notable changes to PauseMesh are documented here. The project follows semantic versioning;
 protocol adapter contracts remain experimental until a stable `1.0.0` release.
 
+## 0.3.0-alpha.1 - 2026-07-19
+
+### Added
+
+- Optional `pausemesh/postgres` entry point with an externally owned, structurally typed pool and
+  no PostgreSQL client dependency in the root runtime path.
+- Explicit, checksum-validated PostgreSQL schema migration with a serialized migration lock,
+  append-only database guards, and no constructor-time DDL.
+- Multi-replica compare-and-swap persistence using a stream-head fence and one checked-out client
+  per transaction.
+- Bounded, fully validated PostgreSQL replay that detects missing events, version gaps, identity
+  mismatches, malformed snapshots, and unsafe `BIGINT` values.
+- Host-owned readiness port and `/readyz`; SQLite, in-memory, and PostgreSQL stores implement the
+  same probe while `/healthz` remains an independent liveness endpoint.
+- Scripted fake-pool coverage plus an opt-in real PostgreSQL test and mandatory Linux PostgreSQL CI
+  job covering concurrent replicas, exact retry, append-only enforcement, and restart replay.
+
+### Security
+
+- PostgreSQL schema identifiers are restricted and quoted; values remain parameterized.
+- Migration metadata, physical tables, expected trigger/table/function tuples, trigger activation,
+  and runtime read access fail readiness closed without leaking database errors over HTTP.
+- The adapter never owns or logs a connection string and never closes the injected pool. Deployment
+  code must separate migration and runtime roles and own timeout, idle-error, and shutdown policy.
+
 ## 0.2.0-alpha.1 - 2026-07-17
 
 ### Breaking

README.md

@@ -33,6 +33,7 @@ orchestrator.
 - A continuation can resume once. Concurrent attempts produce one winner.
 - Exact retries use an independent idempotency key and return the first outcome.
 - SQLite/WAL recovery after process restart, behind a replaceable `EventStore` port.
+- Optional PostgreSQL multi-replica persistence with explicit migrations and transactional CAS.
 - Redacted observations that never include tokens or continuation payloads.
 - Versioned, intentionally narrow MCP 2025-11-25, A2A 1.0, and AG-UI adapters.
 - Explicit protocol-owned request/task/run bindings; continuation IDs are used only in adapter
@@ -51,11 +52,14 @@ flowchart LR
   S --> P["EventStore port"]
   P --> M["In-memory adapter"]
   P --> Q[("SQLite / WAL")]
+  P --> PG[("PostgreSQL / replicas")]
   S --> O["Redacted observer"]
 ```
 
 Dependencies point inward. Domain code imports no Hono, SQLite, protocol SDK, or logger package.
 Protocol drift stays in adapters; storage replacement stays behind one atomic append contract.
+PostgreSQL lives behind the optional `pausemesh/postgres` entry point, so importing the root package
+does not load or require a database client.
 
 ## Quick start
 
@@ -174,6 +178,54 @@ const continuations = new ContinuationService({
 });
 ```
 
+### PostgreSQL multi-replica deployment
+
+The PostgreSQL adapter accepts an externally owned pool-like object. Migration is explicit and
+must run from a deployment process with a DDL-capable role; `PostgresEventStore` constructors never
+change the schema. The host owns TLS, credentials, timeouts, pool sizing, idle-client errors, and
+shutdown.
+
+```ts
+import { Pool } from "pg";
+import {
+  PostgresEventStore,
+  migratePostgresEventStore,
+} from "pausemesh/postgres";
+
+const migrationPool = new Pool({
+  connectionString: process.env.PAUSEMESH_MIGRATION_DATABASE_URL,
+  connectionTimeoutMillis: 5_000,
+  max: 1,
+  statement_timeout: 10_000,
+});
+await migratePostgresEventStore(migrationPool, { schema: "pausemesh" });
+await migrationPool.end();
+
+const runtimePool = new Pool({
+  connectionString: process.env.DATABASE_URL,
+  connectionTimeoutMillis: 5_000,
+  idleTimeoutMillis: 30_000,
+  max: 10,
+  query_timeout: 5_000,
+  statement_timeout: 5_000,
+});
+runtimePool.on("error", () => {
+  // Report through protected host telemetry; never log the pool configuration or URL.
+});
+
+const store = new PostgresEventStore(runtimePool, {
+  schema: "pausemesh",
+  maxEventsPerStream: 32,
+});
+```
+
+Pass `store` as both the `ContinuationService` event store and
+`CreateHttpAppOptions.readinessProbe`. `GET /healthz` remains process liveness and never touches the
+database; `GET /readyz` verifies the exact migration and required PostgreSQL objects and returns a
+bounded 503 body on failure. The bundled CLI intentionally remains SQLite-only because a production
+PostgreSQL composition root must own provider-specific identity and lifecycle policy. See the
+[PostgreSQL runbook](docs/postgres-runbook.md) and [ADR 0003](docs/adr/0003-postgres-multireplica-store.md).
+
 Protocol adapters intentionally require their negotiated context:
 
 ```ts
@@ -260,8 +312,8 @@ protocol parsing; hosts can tighten the default 64-level and 50,000-node limits.
 ```text
 src/domain/                 versioned events, envelope, reducer, typed errors
 src/application/            lifecycle use cases and concurrency resolution
-src/ports/                  EventStore, Clock, TokenIssuer, Observer
-src/adapters/storage/       in-memory and SQLite/WAL stores
+src/ports/                  EventStore, ReadinessProbe, Clock, TokenIssuer, Observer
+src/adapters/storage/       in-memory, SQLite/WAL, and PostgreSQL stores
 src/adapters/{mcp,a2a,agui} protocol projections
 src/adapters/http/          local Hono reference API
 tests/                      domain, storage, concurrency, conformance, HTTP, config tests
@@ -277,18 +329,24 @@ protect payload data at rest, terminate TLS, and retain the same one-shot/CAS in
 `PAUSEMESH_MAX_PAYLOAD_BYTES` is enforced while streaming each JSON request, before parsing, even
 when no trustworthy content length is available.
 
+For PostgreSQL, use separate migration and runtime roles. The runtime role needs schema usage,
+read access, event inserts, and stream-head inserts/updates only; it must not receive DDL, delete, or
+truncate privileges. Readiness validates migration metadata and the expected active trigger
+bindings, but deployment access control remains the boundary against schema tampering. Connection
+URLs and provider credentials are owned by the host and never accepted or logged by the adapter.
+
 Resume tokens are bearer credentials. PauseMesh hashes them before persistence and redacts them
 from its observer, but the caller is responsible for safe delivery and storage. A SHA-256 hash is
 appropriate here because tokens are generated with 256 bits of entropy; it is not being used to
 hash human passwords or API keys.
 
 ## Status
 
-`0.2.0-alpha.1` is an experimental protocol-conformance prerelease. The canonical state machine
-and storage contract remain unchanged; the adapter surface now requires explicit upstream
-bindings and negotiated capabilities. Protocol adapters will continue to evolve as upstream
-specifications do. See [ADR 0002](docs/adr/0002-protocol-adapter-conformance.md) for the boundary
-decision and [the delivery contract](docs/delivery-contract.md) for acceptance evidence and
-explicit non-goals.
+`0.3.0-alpha.1` is an experimental multi-replica persistence prerelease. The canonical state
+machine, protocol mappings, token handling, and `EventStore` contract remain unchanged. PostgreSQL
+adds a production-oriented storage option, not identity, tenancy, database HA, or orchestration.
+See [ADR 0002](docs/adr/0002-protocol-adapter-conformance.md),
+[ADR 0003](docs/adr/0003-postgres-multireplica-store.md), and
+[the delivery contract](docs/delivery-contract.md) for boundaries and acceptance evidence.
 
 Apache-2.0 © 2026 Antonio Antenore.

docs/adr/0003-postgres-multireplica-store.md

@@ -0,0 +1,53 @@
+# ADR 0003: Optional PostgreSQL store for multi-replica continuity
+
+- Status: accepted
+- Date: 2026-07-19
+- Target: `0.3.0-alpha.1`
+
+## Context
+
+SQLite/WAL provides durable local recovery, but a callback may reach a different application
+replica. The `EventStore` contract already requires an atomic compare-and-swap append, so the
+multi-replica capability belongs in a storage adapter rather than in the continuation domain or a
+new coordinator.
+
+PostgreSQL integration must not make the root package import a database driver, hide DDL in an
+application constructor, expose connection credentials, or let a pool-backed transaction move
+between physical connections.
+
+## Decision
+
+PauseMesh provides `PostgresEventStore` and `migratePostgresEventStore` from the optional
+`pausemesh/postgres` entry point.
+
+- The host injects a minimal `PostgresPoolLike` object and retains ownership of connection limits,
+  timeouts, idle-client error handling, and shutdown.
+- Migration is an explicit deployment operation. Version 1 is recorded with a stable SHA-256
+  checksum and serialized with a transaction-scoped advisory lock. Constructors perform no DDL.
+- A dedicated stream-head row holds `current_version`. Append creates the head only for a new
+  non-empty stream, advances it with `UPDATE ... WHERE current_version = expected`, and inserts the
+  full event batch on the same checked-out client and `READ COMMITTED` transaction.
+- A lost fence becomes `VersionConflictError`; unexpected uniqueness, data, or database failures
+  are not mislabeled as concurrency.
+- Event JSON remains text so replay observes the exact persisted snapshot. Load validates the head,
+  contiguous physical versions, embedded identity/version, schema, and a configurable event bound.
+- Physical event rows and migration metadata are protected by database triggers. Stream-head rows
+  cannot be deleted or truncated; their version remains updateable by the runtime adapter.
+- Readiness requires the exact migration version/checksum, ordinary tables, expected active
+  trigger/table/function tuples, and runtime read access. Liveness remains independent.
+- The bundled CLI stays SQLite-only. A PostgreSQL deployment is a library composition root because
+  it must own provider-specific credentials, TLS, pool policy, roles, and lifecycle.
+
+## Consequences
+
+- Multiple stateless application replicas can share continuation state without changing protocol
+  adapters, lifecycle events, token handling, or application-service reconciliation.
+- `pg` is used only for development and the real integration test; consumers may inject node-postgres
+  or another compatible pool without leaking its types through public declarations.
+- Migration and runtime roles must be distinct. Runtime roles need no DDL, delete, or truncate
+  privilege.
+- PostgreSQL availability, regional failover, backup/restore, tenant authorization, payload
+  encryption, and secret distribution remain deployment responsibilities.
+- Readiness verifies migration metadata and the required object identities, not the entire function
+  body or every physical constraint. Removing DDL from the runtime role is therefore a required
+  defense, not an optional hardening step.

docs/delivery-contract.md

@@ -1,9 +1,9 @@
 # Delivery Contract: PauseMesh
 
 Date: 2026-07-15
-Last updated: 2026-07-17
+Last updated: 2026-07-19
 Mode: new-project
-Status: delivered through `0.2.0-alpha.1`
+Status: delivered through `0.3.0-alpha.1`
 
 ## Objective
 
@@ -19,8 +19,10 @@ Must:
 - Issue opaque, one-shot resume tokens while persisting only their SHA-256 hashes.
 - Make retries idempotent and reject stale, wrong, expired, cancelled, or reused tokens.
 - Supply replaceable in-memory and SQLite/WAL event-store adapters.
+- Supply an optional PostgreSQL adapter with explicit migrations and multi-replica CAS semantics.
 - Map a minimal continuation contract across MCP, A2A, and AG-UI adapters.
 - Expose a small HTTP/CLI demo and machine-readable errors.
+- Keep liveness independent from a fail-closed, host-owned dependency readiness probe.
 
 Should:
 
@@ -43,6 +45,7 @@ Out of scope:
 
 - Node.js 24+ is available.
 - SQLite is a reference store; consumers may implement the event-store port elsewhere.
+- PostgreSQL consumers own their pool, credentials, TLS, timeouts, runtime role, and shutdown.
 - The MVP HTTP server is for local evaluation and must not be internet-exposed without authentication.
 - Protocol drafts can change, so adapter contracts are versioned independently from the core envelope.
 
@@ -63,6 +66,10 @@ Out of scope:
 | R11 | AG-UI resume contract | Must | A whole resume batch is bound to an immutable issued receipt/current cohort and invalid input becomes `RUN_ERROR` | Official schema, receipt, and replay fixtures |
 | R12 | Adapter migration | Must | HTTP/demo/docs use the explicit 0.2 bindings with no legacy ID fallback | Build, demo, and HTTP tests |
 | R13 | Consumable package | Must | A clean source copy builds a tarball that installs in a real pnpm consumer with working ESM exports, declarations, docs, config, and CLI | Package smoke test |
+| R14 | PostgreSQL multi-replica CAS | Must | Two independent pools share a stream, one conflicting append wins, the loser receives a typed version conflict, and exact service retry reconciles | Real PostgreSQL integration test |
+| R15 | Explicit PostgreSQL migration | Must | Version/checksum mismatch, future schema, incomplete objects, or missing runtime access fail closed; constructors perform no DDL | Fake-pool and real PostgreSQL tests |
+| R16 | Liveness/readiness split | Must | `/healthz` is unchanged and independent; `/readyz` reports configured dependency readiness without leaking underlying errors | HTTP and storage tests |
+| R17 | Optional package boundary | Must | `pausemesh/postgres` works from the packed consumer while PostgreSQL symbols do not leak through the root export | Clean package smoke test |
 
 ## Acceptance Threshold
 
@@ -72,15 +79,18 @@ the local API demo completes, and security/operational limitations are explicit.
 ## Architecture Approach
 
 A modular TypeScript library: pure domain state machine, application service, storage/clock/token
-ports, protocol adapters at the edge, SQLite/WAL reference adapter, and a thin Hono HTTP adapter.
-Dependencies point inward; protocol and storage packages never leak into the domain.
+ports, protocol adapters at the edge, SQLite/WAL reference adapter, optional PostgreSQL adapter,
+and a thin Hono HTTP adapter. Dependencies point inward; protocol and storage packages never leak
+into the domain. The PostgreSQL client is injected structurally and owned by the host.
 
 ## Test Plan
 
 Critical:
 
 - State transition, stale version, token mismatch/reuse, idempotency, cancel, and expiry tests.
 - SQLite close/reopen replay and concurrent-resume tests.
+- Scripted PostgreSQL migration, transaction, rollback, corruption, limit, and readiness tests.
+- Real PostgreSQL two-pool CAS, idempotency, append-only, and restart replay test.
 - MCP/A2A/AG-UI projection contract tests.
 - Strict MCP primitive/enum schemas, original-request receipt binding, trusted URL policy, URL
   consent vs completion, and invalid result fixtures.
@@ -93,10 +103,11 @@ Recommended:
 - HTTP error-shape and payload-limit integration tests.
 - Log redaction and configuration validation tests.
 
-Not tested in the MVP:
+Not covered by this library release:
 
-- Multi-node consensus, hostile internet traffic, or full upstream conformance suites; risk is high
-  outside local evaluation, so the server is explicitly non-production until those gates exist.
+- PostgreSQL regional failover/HA, backup restore, tenant isolation, hostile internet traffic, or
+  full upstream conformance suites. The bare CLI server remains non-production until a host adds
+  identity, authorization, TLS, rate limits, and deployment controls.
 
 ## Delivery Policy
 
@@ -118,6 +129,9 @@ Selected mode: create public GitHub repository and push `main`, as explicitly re
 | 2026-07-17 | Prove the npm artifact from clean source | Release gate expanded | Prepack build and import/bin/content smoke included in `pnpm check` |
 | 2026-07-17 | Bind MCP results/completion to the exact emitted request | Added host-side issuance state | Request SHA-256 receipt is mandatory on inbound helpers |
 | 2026-07-17 | Remove consumer native-build approval | Storage implementation swap | `SqliteEventStore` now uses built-in `node:sqlite` behind the unchanged port |
+| 2026-07-19 | Support callbacks reaching another replica | Added optional PostgreSQL adapter | Stream-head CAS on one checked-out `READ COMMITTED` transaction |
+| 2026-07-19 | Keep deployment credentials and lifecycle out of the library | Pool remains externally owned | Optional subpath and structural pool contract; CLI remains SQLite-only |
+| 2026-07-19 | Separate process liveness from dependency readiness | Added `ReadinessProbe` and `/readyz` | Omission/error fails closed without changing `/healthz` |
 
 ## Incident Register
 
@@ -159,3 +173,22 @@ Selected mode: create public GitHub repository and push `main`, as explicitly re
   import, and the installed CLI. The production dependency audit reported no known vulnerabilities.
 - Boundary result: the continuation core and persisted envelope did not change. Protocol drift is
   isolated to adapters, and legacy task/run ID fallbacks are absent.
+
+### PostgreSQL multi-replica release-candidate evidence
+
+- Release: `0.3.0-alpha.1`, consolidated on the protected default branch after required CI.
+- Architecture decision: [ADR 0003](adr/0003-postgres-multireplica-store.md); operations:
+  [PostgreSQL adapter runbook](postgres-runbook.md).
+- PostgreSQL unit coverage uses a scripted pool/client boundary to verify statement ordering,
+  pre-validation, compare-and-swap conflicts, rollback/release behavior, migration checksums,
+  schema completeness, bounded replay, and corruption classification.
+- The opt-in real PostgreSQL suite compiles against `pg.Pool` and is mandatory in its Linux CI job.
+  It runs concurrent migrations and replicas, exact retry, append-only guards, and restart replay.
+- Package smoke installs the tarball into a clean consumer and verifies `pausemesh/postgres` without
+  leaking PostgreSQL symbols into the root export.
+- Local PostgreSQL 17 integration: one real test passed against an ephemeral container and the
+  container was removed after verification.
+- Local quality gate: `pnpm check` passed with 193 tests passed and one opt-in PostgreSQL test
+  skipped in the ordinary coverage run; statements 86.38%, branches 82.35%, functions 92.51%, and
+  lines 87.20%. Build and clean-consumer package smoke produced and verified
+  `pausemesh-0.3.0-alpha.1.tgz`.