Fix scheduled cleanup of superseded media-usage generations - #2324
Fix scheduled cleanup of superseded media-usage generations#2324khoinguyenpham04 wants to merge 18 commits into
Conversation
🦋 Changeset detectedLatest commit: 7d80220 The changes in this PR will be included in the next version bump. This PR includes changesets to release 17 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Scope checkThis PR changes 2,629 lines across 13 files. Large PRs are harder to review and more likely to be closed without review. If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs. See CONTRIBUTING.md for contribution guidelines. |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-playground | 7d80220 | Aug 04 2026, 10:05 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-demo-do | 7d80220 | Aug 04 2026, 10:06 PM |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
emdash-demo-cache | 7d80220 | Aug 04 2026, 10:04 PM |
Overlapping PRsThis PR modifies files that are also changed by other open PRs:
This may cause merge conflicts or duplicated work. A maintainer will coordinate. |
…lan.test.ts Co-authored-by: emdashbot[bot] <273199577+emdashbot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
This is a re-review of the Sequence 0 media-usage cleanup child. The approach is sound: a scheduler-only tick, persisted single-flight lease, age safety window, keyset cursor, bounded failure backoff, and cleanup-owned promotion fence address the stated unbounded-growth problem without adding logged-out request queries or touching auth/CSRF paths.
I read the full diff, the new cleanup/repository code, the two migrations, the scheduler wiring, and the new tests. The implementation looks correct: the lease check and fence prevent deleting in-flight generations, the cleanup_lease_token markings let normal source deletes remove rows without advancing the fence, and the cleanup row is integrated into runSystemCleanup as a non-fatal subsystem. No new runtime bugs, security issues, or data-integrity gaps stood out statically.
The prior review’s comment-discipline findings are resolved: the new cleanup code and supporting repository changes contain no numbered or justification comments.
The one still-open concern from the prior review is the brittle query-count assertion. media-usage-cleanup-plan.test.ts hard-codes the exact statement and bind budgets (14 and 52) as magic numbers. That is an AGENTS.md test-quality violation: asserting a config literal back at the implementation cannot fail on a real regression and will fail on any legitimate budget change. I also noticed media-usage-cleanup.test.ts asserts the literal 5000 instead of using the exported MEDIA_USAGE_CLEANUP_TIME_BUDGET_MS constant.
I did not run the test suite; I rely on the author-reported results for passing behavior.
There was a problem hiding this comment.
This is a re-review of the Sequence 0 media-usage cleanup child. The approach remains sound: a scheduler-only tick, persisted single-flight lease, age safety window, keyset cursor, bounded failure backoff, and cleanup-owned fence address the stated unbounded-growth problem without adding logged-out request queries or touching auth/CSRF paths.
I re-read the cleanup implementation, both migrations, the repository changes, and the new tests. The comment-discipline findings from the prior review are resolved, and no new runtime bugs, security issues, or data-integrity gaps stood out statically.
The one still-open concern from the prior review has not been addressed. media-usage-cleanup-plan.test.ts still hard-codes the exact SQL-statement and bind-parameter budgets (14 and 52) as magic numbers. That is a config-pin test under AGENTS.md — it restates implementation choices and cannot fail on a real regression; it will only fail when someone intentionally changes the budget. I also still see the five-second admission budget duplicated across both new test files (MAX_CLEANUP_ADMISSION_TIME_MS = 5_000) instead of being imported from the source that defines MEDIA_USAGE_CLEANUP_TIME_BUDGET_MS.
I did not run the test suite; I rely on the author-reported results for passing behavior.
There was a problem hiding this comment.
The approach remains sound: a scheduler-only cleanup tick with a persisted single-flight lease, an age-based safety window, a keyset sweep cursor, bounded failure backoff, and the cleanup-owned generation fence directly address the stated unbounded-growth problem without adding queries to logged-out routes or touching auth/CSRF paths. I re-read the cleanup implementation, both new migrations, the repository changes, and the new tests.
No new runtime bugs, security issues, or data-integrity gaps stood out statically. The two AGENTS.md testing concerns from the prior review are still unaddressed in this head:
media-usage-cleanup-plan.test.tsstill hard-codes the exact SQL-statement and bind-parameter budgets (14and52) as magic numbers. Under AGENTS.md Testing, that is a config-pin test — it re-states implementation choices and cannot fail on a real regression; it will only fail when someone intentionally changes the budget.- The five-second admission budget (
MAX_CLEANUP_ADMISSION_TIME_MS = 5_000) is still duplicated across both new test files instead of being imported from the source constantMEDIA_USAGE_CLEANUP_TIME_BUDGET_MS.
I did not run the test suite, linter, or builds; I rely on the author-reported results for passing behavior.
What does this PR do?
Fixes unbounded growth of superseded media-usage projection generations. The existing cleanup operations were never invoked by production scheduled maintenance, allowing obsolete occurrences to accumulate indefinitely.
This adds a scheduler-only, database-clock cleanup tick with a persisted single-flight lease, age safety window, finite keyset sweep cursor, bounded retry backoff, and a cleanup-owned fence for rolling deployments. The tick is capped at 250 candidates, 50 projection deletions, 49 expired writer-lease deletions, 14 SQL statements, 52 binds, and five seconds. An additive index supports the age-ordered scan on SQLite/D1 and PostgreSQL.
It is a completed Sequence 0 child of the media-library umbrella and should merge only into draft PR #2218.
Closes: N/A — child PR for #2218
Type of change
Checklist
pnpm typecheckpassespnpm lintpassespnpm testpasses (targeted: media-usage repository, migrations, cleanup, and query-plan suites)pnpm formathas been runmessages.pochanges except in translation PRs — N/A; no admin UI strings are changed.AI-generated code disclosure
Screenshots / test output
No visual changes.
Try this PR
Open a fresh playground →
A full working EmDash site, deployed from this branch. Each visit gets its own session-scoped sandbox: no login needed and no shared state. Try the admin, edit content, hit the public site.
Tracks
fix/media-usage-cleanup-sequence-0. Updated automatically when the playground redeploys.