Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/event-scoped-db-cron-plugins.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"emdash": patch
---

Resolves the database connection at use-time for the cron sweep, plugin hook contexts, and media providers instead of capturing it once at startup. This makes scheduled publishing, plugin cron, and database-querying plugin hooks work on connection-backed adapters like Postgres over Cloudflare Hyperdrive, where a connection is bound to the event that opened it. Stateless adapters (D1, Node SQLite) are unaffected.
2 changes: 1 addition & 1 deletion .changeset/hyperdrive-postgres-adapter.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@

Adds a `hyperdrive()` database adapter for connecting EmDash on Cloudflare Workers to a PostgreSQL (or PostgreSQL-compatible, e.g. PlanetScale Postgres) database through a Hyperdrive binding. Configure it with `database: hyperdrive({ binding: "HYPERDRIVE" })`. Each request gets its own pooled connection that is opened and closed within that request — connections cannot be reused across Worker requests. Requires `pg >= 8.16.3`, the `nodejs_compat` compatibility flag, and a compatibility date of `2024-09-23` or later. Disable Hyperdrive query caching for the configuration so the admin's read-after-write stays consistent.

The content read/write path (pages, content API routes, loaders) is fully supported. Cron Triggers (scheduled publishing, plugin cron, system cleanup), plugin hooks that query the database, and sandboxed plugins are not yet supported on this adapter — they use a per-isolate connection that workerd will not reuse across events. Use `d1()` if your deployment depends on those.
The content read/write path, scheduled publishing, plugin cron, and database-querying plugin hooks are all supported. Sandboxed plugins remain D1-only (the sandbox bridge talks to a D1 binding directly, independent of the configured adapter).
47 changes: 17 additions & 30 deletions packages/cloudflare/src/db/hyperdrive.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,33 +20,20 @@
* ALS, and the runtime/loader db getters prefer it over the singleton — so all
* request-path queries use a connection opened in the current request.
*
* `createDialect` still builds the per-isolate singleton Kysely. Its socket is
* opened by whatever event first queries it — normally the cold-start
* migrations during the first HTTP request — and, because a pg socket is bound
* to the request that opened it, it is only safe to use again from within that
* same event. The request path never does: routes and loaders read through the
* per-request scoped Kysely (ALS), not the singleton.
* `createDialect` still builds the per-isolate singleton Kysely, used only for
* cold-start migrations (which run inside the first request, so the socket is
* valid there). Everything else resolves the connection from ALS at use-time:
* the request path through the runtime/loader db getters, and the background and
* plugin paths (Cron Trigger sweep, plugin hook contexts, media providers)
* through resolvers threaded by the core runtime. The Cron Trigger handler opens
* its own event-scoped connection for the sweep. So no warm-isolate path reuses
* the singleton's request-bound socket across events.
*
* Known limitation — background and plugin paths still use the singleton
* --------------------------------------------------------------------------
* Several subsystems capture the runtime's singleton db at construction and do
* not consult the per-request scoped connection:
* - the Cron Trigger handler (`scheduled()` → scheduled publishing, plugin
* cron, system cleanup),
* - plugin hook contexts (a hook's `content` / `media` / `users` / `cron`
* access),
* - media providers and sandboxed plugins.
*
* On a warm isolate the singleton's socket belongs to an earlier request, so
* these paths can fail under workerd's cross-request I/O guard ("Cannot perform
* I/O on behalf of a different request"). It is not a data-corruption risk — the
* work errors and is logged — but it means scheduled publishing and
* database-querying plugin hooks are not yet supported on the Hyperdrive
* adapter. The core read/write path (pages, content API routes, loaders) is
* unaffected. Closing this requires the core runtime to thread an event-scoped
* connection through those subsystems; tracked in
* https://github.com/emdash-cms/emdash/issues/1622. Until then, use D1 for
* deployments that rely on Cron Triggers or DB-querying plugins.
* Known limitation — sandboxed plugins are D1-only. The sandbox plugin bridge
* (a Durable Object) talks to a D1 binding directly, independent of the
* configured adapter, so sandboxed plugins are not available on a Hyperdrive
* deployment. This is a pre-existing bridge constraint, unrelated to connection
* scoping; tracked in https://github.com/emdash-cms/emdash/issues/1623.
*
* This module imports directly from cloudflare:workers to access the binding.
* Do NOT import it at config time — use { hyperdrive } from
Expand Down Expand Up @@ -108,10 +95,10 @@ function createPool(connectionString: string, max: number): Pool {
/**
* Create a PostgreSQL dialect backed by a Hyperdrive binding.
*
* Used for the per-isolate singleton Kysely. The request path never touches it
* (it reads through `createRequestScopedDb`); in practice the singleton serves
* cold-start migrations, plus the background/plugin paths noted in the module
* header that are not yet safe across event boundaries on this adapter.
* Used for the per-isolate singleton Kysely, which serves cold-start migrations
* only. The request path reads through `createRequestScopedDb`, and the
* background/plugin paths resolve an event-scoped connection from ALS, so
* neither reuses this singleton's request-bound socket across events.
*/
export function createDialect(config: HyperdriveConfig): Dialect {
const binding = requireBinding(config);
Expand Down
25 changes: 10 additions & 15 deletions packages/cloudflare/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -262,21 +262,16 @@ export function d1(config: D1Config): DatabaseDescriptor {
* { "placement": { "region": "aws:us-east-1" } }
* ```
*
* **Known limitation — request path only (for now).** Each request gets its own
* pg connection, so the content read/write path (pages, content API routes,
* loaders) is fully supported. But several background and plugin paths still use
* the per-isolate singleton connection, whose socket is bound to the request
* that opened it; on a warm isolate workerd refuses to reuse it from a later
* event. Until the core runtime threads an event-scoped connection through them
* (tracked in https://github.com/emdash-cms/emdash/issues/1622), the following
* are **not yet supported** on the Hyperdrive adapter:
* - Cron Triggers — scheduled publishing, plugin cron, and system cleanup.
* - Plugin hooks that query the database via their plugin context.
* - Media providers and sandboxed plugins that hold the singleton db.
*
* Use `d1()` for deployments that depend on those. (This is a Hyperdrive-adapter
* limitation, not a data-safety risk: affected work errors and is logged rather
* than corrupting anything.)
* Each request gets its own pg connection, and the Cron Trigger sweep, plugin
* hook contexts, and media providers resolve an event-scoped connection too, so
* the content read/write path, scheduled publishing, plugin cron, and
* DB-querying plugin hooks are all supported.
*
* **Known limitation — sandboxed plugins are D1-only.** The sandbox plugin
* bridge talks to a D1 binding directly (independent of the configured
* adapter), so sandboxed plugins aren't available on a Hyperdrive deployment.
* This is a pre-existing bridge constraint, unrelated to connection scoping;
* tracked in https://github.com/emdash-cms/emdash/issues/1623.
*
* @example
* ```ts
Expand Down
65 changes: 64 additions & 1 deletion packages/core/src/astro/middleware.ts
Original file line number Diff line number Diff line change
Expand Up @@ -276,9 +276,72 @@ export async function runScheduledTasks(
const config = getConfig();
if (!config) return { published: [] };
const runtime = await getRuntime(config);
return runtime.runScheduledTasks(options);

// Connection-backed adapters (e.g. Postgres over Hyperdrive) cannot reuse
// the per-isolate singleton from a Cron Trigger: its socket belongs to the
// request that opened it, and workerd rejects cross-event I/O. Open an
// event-scoped connection for the sweep and run the batch under it in ALS —
// the runtime's db getter, the cron executor, and plugin cron contexts all
// resolve the connection from ALS — then close it. Gated on the adapter
// being connection-backed (it exposes `close()`); stateless adapters (D1,
// Node SQLite) return null or a close-less scope and keep using the
// singleton, so their cron path is unchanged.
const scoped = createRequestScopedDb({
config: config.database?.config,
isAuthenticated: false,
// The sweep publishes and cleans up — a write workload — so a
// connection-backed adapter routes it to the primary.
isWrite: true,
cookies: NOOP_COOKIE_JAR,
url: CRON_EVENT_URL,
});
if (!scoped?.close) {
// Stateless adapter (or no per-request scoping): the singleton is safe
// outside a request. Any close-less scope created above is discarded.
return runtime.runScheduledTasks(options);
}

const parent = getRequestContext();
const ctx = parent
? { ...parent, db: scoped.db }
: { editMode: false, db: scoped.db, metrics: createRequestMetrics(performance.now()) };
try {
return await runWithContext(ctx, () => runtime.runScheduledTasks(options));
} finally {
// Guard both so a throw in teardown can't mask the sweep result or skip
// close() and leak the connection. Mirrors closeSafely() in scoped-db.ts.
try {
scoped.commit();
} catch (error) {
console.error("[scheduled] request-scoped db commit failed:", error);
}
try {
scoped.close();
} catch (error) {
console.error("[scheduled] request-scoped db close failed:", error);
}
}
}

/**
* A cookie jar that reads nothing and writes nothing, for request-scoped db
* adapters invoked outside an HTTP request (the Cron Trigger sweep). Connection
* adapters like Hyperdrive ignore cookies entirely; the D1 session adapter
* reads/writes a bookmark cookie, but cron never reaches that path (it has no
* `close()`), so the no-ops are never observed.
*/
const NOOP_COOKIE_JAR = {
get: () => undefined,
set: () => {},
};

/**
* Synthetic URL for the cron sweep's request-scoped db opts. Only the D1
* session adapter inspects `url` (for cookie `secure`), and cron doesn't take
* that path, so the value is never used — it exists to satisfy the contract.
*/
const CRON_EVENT_URL = new URL("https://cron.emdash.internal/");

/**
* Baseline security headers applied to all responses.
* Admin routes get additional headers (strict CSP) from auth middleware.
Expand Down
65 changes: 57 additions & 8 deletions packages/core/src/emdash-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,15 @@ export interface MediaProviderEntry {
*/
export interface MediaProviderContext {
db: Kysely<Database>;
/**
* Resolver for the live connection, preferred over `db` by providers that
* query EmDash's database. Resolves the current request/event-scoped
* connection from ALS so connection-backed adapters (Postgres over
* Hyperdrive) don't reuse the per-isolate singleton's socket across events.
* Providers should resolve per operation rather than capturing `db` once.
* Omitted-safe: falls back to `db` for stateless adapters (D1, Node SQLite).
*/
getDb?: () => Kysely<Database>;
storage: Storage | null;
}

Expand Down Expand Up @@ -333,6 +342,7 @@ export interface EmDashRuntimeParts {
allPipelinePlugins: ResolvedPlugin[];
pipelineFactoryOptions: {
db: Kysely<Database>;
getDb?: () => Kysely<Database>;
storage?: Storage;
siteInfo?: { siteName?: string; siteUrl?: string; locale?: string };
};
Expand Down Expand Up @@ -446,7 +456,20 @@ export class EmDashRuntime {
readonly configuredPlugins: ResolvedPlugin[];
readonly sandboxedPlugins: Map<string, SandboxedPluginInstance>;
readonly sandboxedPluginEntries: SandboxedPluginEntry[];
readonly schemaRegistry: SchemaRegistry;
/**
* Schema registry bound to the current request/event-scoped connection.
* Built per access (SchemaRegistry just wraps a db) against `this.db`, the
* ALS-aware getter — never a captured snapshot of the singleton. On a
* connection-backed adapter (Postgres over Hyperdrive) a captured singleton
* would query a socket opened by an earlier event and trip workerd's
* cross-request I/O guard; the catch in handlers like handleContentUpdate
* would then silently treat a revision-enabled collection as non-revisioned
* and write draft edits to live columns. Same reasoning as the per-call
* registry in _buildManifest().
*/
get schemaRegistry(): SchemaRegistry {
return new SchemaRegistry(this.db);
}
private _hooks!: HookPipeline;
readonly config: EmDashConfig;
readonly mediaProviders: Map<string, MediaProvider>;
Expand Down Expand Up @@ -477,6 +500,7 @@ export class EmDashRuntime {
/** Factory options for the hook pipeline context factory */
private pipelineFactoryOptions: {
db: Kysely<Database>;
getDb?: () => Kysely<Database>;
storage?: Storage;
siteInfo?: { siteName?: string; siteUrl?: string; locale?: string };
};
Expand Down Expand Up @@ -507,7 +531,6 @@ export class EmDashRuntime {
this.configuredPlugins = parts.configuredPlugins;
this.sandboxedPlugins = parts.sandboxedPlugins;
this.sandboxedPluginEntries = parts.sandboxedPluginEntries;
this.schemaRegistry = new SchemaRegistry(parts.db);
this._hooks = parts.hooks;
this.enabledPlugins = parts.enabledPlugins;
this.pluginStates = parts.pluginStates;
Expand Down Expand Up @@ -653,7 +676,9 @@ export class EmDashRuntime {
// The old pipeline's contextFactoryOptions were built up incrementally
// via setContextFactory calls during create(). We replay them here.
if (this.email) {
newPipeline.setContextFactory({ db: this.db, emailPipeline: this.email });
// db/getDb are already wired by createHookPipeline above (they live in
// pipelineFactoryOptions), so the merge only adds emailPipeline.
newPipeline.setContextFactory({ emailPipeline: this.email });
}
if (this.cronScheduler) {
const scheduler = this.cronScheduler;
Expand Down Expand Up @@ -1017,6 +1042,22 @@ export class EmDashRuntime {
// Initialize database (connects, runs migrations if needed)
const db = await phase("rt.db", "DB init + migrations", () => EmDashRuntime.getDatabase(deps));

// Resolver for the live connection, mirroring the `get db()` getter
// below (which can't be used here — the runtime instance doesn't exist
// yet). Long-lived subsystems built during create() (cron executor,
// plugin context factory, media providers) capture this resolver rather
// than the `db` snapshot, so a connection-backed adapter (Postgres over
// Hyperdrive) serves their queries from the current request/event-scoped
// connection in ALS instead of the per-isolate singleton — whose socket
// belongs to an earlier request and would trip workerd's cross-request
// I/O guard. Stateless adapters (D1, Node SQLite) set no ALS db on most
// paths, so this falls back to the singleton: unchanged behavior.
const resolveDb = (): Kysely<Database> => {
const ctx = getRequestContext();
// eslint-disable-next-line typescript/no-unsafe-type-assertion -- ALS db is typed unknown to avoid a circular import; middleware always sets a Kysely<Database>
return (ctx?.db as Kysely<Database> | undefined) ?? db;
};

// Validate EMDASH_ENCRYPTION_KEY once here so a malformed value
// surfaces in startup logs instead of as request-time 500s. The key
// itself is not yet consumed (a follow-up PR adds plugin-secret
Expand Down Expand Up @@ -1295,9 +1336,15 @@ export class EmDashRuntime {
// Filter to currently enabled plugins for the initial pipeline
const enabledPluginList = allPipelinePlugins.filter((p) => enabledPlugins.has(p.id));

// Create hook pipeline
// Create hook pipeline. getDb travels here (not just via the email
// setContextFactory call below) so it survives rebuildHookPipeline(),
// which reconstructs the factory from pipelineFactoryOptions. Without it,
// toggling a plugin on an email-less deployment would silently revert
// plugin contexts to the singleton db — re-breaking connection-backed
// adapters. See #1622.
const pipelineFactoryOptions = {
db,
getDb: resolveDb,
storage: storage ?? undefined,
siteInfo,
};
Expand Down Expand Up @@ -1349,7 +1396,7 @@ export class EmDashRuntime {
// Initialize media providers
const mediaProviders = new Map<string, MediaProvider>();
const mediaProviderEntries = deps.mediaProviderEntries ?? [];
const providerContext: MediaProviderContext = { db, storage };
const providerContext: MediaProviderContext = { db, storage, getDb: resolveDb };

for (const entry of mediaProviderEntries) {
try {
Expand Down Expand Up @@ -1390,8 +1437,10 @@ export class EmDashRuntime {
};

// Wire email pipeline into context factory (independent of cron —
// must not be inside the cron try/catch or ctx.email breaks when cron fails)
pipeline.setContextFactory({ db, emailPipeline });
// must not be inside the cron try/catch or ctx.email breaks when cron fails).
// db/getDb were already set via pipelineFactoryOptions above; merge only
// adds emailPipeline.
pipeline.setContextFactory({ emailPipeline });

let cronExecutor: CronExecutor | null = null;
let cronScheduler: CronScheduler | null = null;
Expand All @@ -1403,7 +1452,7 @@ export class EmDashRuntime {

await phase("rt.cron", "Cron init (recovery deferred post-response)", async () => {
try {
cronExecutor = new CronExecutor(db, invokeCronHook);
cronExecutor = new CronExecutor(resolveDb, invokeCronHook);

// Recover stale locks from previous crashes. Pure bookkeeping
// against the _emdash_cron_tasks table — no request needs the
Expand Down
23 changes: 18 additions & 5 deletions packages/core/src/media/local-runtime.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,14 @@ export interface LocalMediaRuntimeConfig {
enabled?: boolean;
// These are injected by the runtime, not from user config
db?: Kysely<Database>;
/**
* Resolver for the live connection, preferred over `db`. The runtime
* injects it so a connection-backed adapter (Postgres over Hyperdrive)
* serves provider queries from the current request-scoped connection in ALS
* rather than a snapshot of the per-isolate singleton. Omitted for stateless
* adapters (D1, Node SQLite), where `db` is used directly.
*/
getDb?: () => Kysely<Database>;
storage?: Storage;
}

Expand All @@ -42,11 +50,15 @@ export const createMediaProvider: CreateMediaProviderFn<LocalMediaRuntimeConfig>
throw new Error("Local media provider requires database connection");
}

const repo = new MediaRepository(db);
// Resolve the connection per operation (not captured once) so a
// connection-backed adapter uses the current event-scoped connection; falls
// back to the injected `db` for stateless adapters.
const resolveDb = config.getDb ?? (() => db);
const repo = () => new MediaRepository(resolveDb());

const provider: MediaProvider = {
async list(options: MediaListOptions) {
const result = await repo.findMany({
const result = await repo().findMany({
cursor: options.cursor,
limit: options.limit,
mimeType: options.mimeType,
Expand Down Expand Up @@ -75,7 +87,7 @@ export const createMediaProvider: CreateMediaProviderFn<LocalMediaRuntimeConfig>
},

async get(id: string) {
const item = await repo.findById(id);
const item = await repo().findById(id);
if (!item) return null;

return {
Expand Down Expand Up @@ -108,7 +120,8 @@ export const createMediaProvider: CreateMediaProviderFn<LocalMediaRuntimeConfig>
},

async delete(id: string) {
const item = await repo.findById(id);
const repoInstance = repo();
const item = await repoInstance.findById(id);
if (!item) return;

// Delete from storage if available
Expand All @@ -120,7 +133,7 @@ export const createMediaProvider: CreateMediaProviderFn<LocalMediaRuntimeConfig>
}
}

await repo.delete(id);
await repoInstance.delete(id);

// If this row was referenced by `logo`, `favicon`, or
// `seo.defaultOgImage`, the worker-scoped settings cache now
Expand Down
Loading
Loading