fix(server): claim lifecycle with TTL for webhook event dedup

[serenakeyitan]

Closes #317

GoF competition entry (Fable). Implements the issue-approved Fix shape 2: claim with status + TTL.

Problem

processScmWebhookDelivery claimed a delivery by inserting into processed_events before side effects ran, with only a best-effort unclaimEvent on failure. A crash between the claim commit and completion (process kill, OOM, deploy restart, DB connection drop mid-handler) left the claim row behind, so every provider redelivery was answered with deduped — the event was lost forever.

Solution

processed_events rows become claims with a lifecycle:

Schema	status text NOT NULL DEFAULT 'done' + expires_at timestamptz + (status, expires_at) index. Existing rows backfill to done via the column default, so they keep deduping.
claimEvent	Atomic INSERT ... ON CONFLICT (event_id, platform) DO UPDATE ... WHERE status = 'pending' AND expires_at <= now(): inserts a pending claim with a 5-minute TTL, or takes over an expired pending claim (crashed processor). done rows and unexpired in-flight pending rows still dedupe. The conflicting row is locked during the upsert, so concurrent deliveries of the same id serialize and exactly one wins.
completeEvent (new)	Flips pending → done (and clears expires_at) on the success path of the shared seam processScmWebhookDelivery — this covers both webhook paths, GitHub App (/webhooks/github-app) and GitLab, since both route through the seam. (The per-org /webhooks/github/:orgId route mentioned in the issue no longer exists on main; the SaaS-wide GitHub App endpoint replaced it.)
unclaimEvent	Kept as a best-effort optimization on the failure path (provider's immediate retry clears without waiting out the TTL), now scoped to status = 'pending' so it can never delete a done record. Correctness no longer depends on it.
Sweep	background-tasks.ts deletes expired pending claims every 60s. Redelivery correctness does not depend on the sweep either (claimEvent takes expired claims over in place); it just keeps the table bounded.

Migration note (disclosed deviation)

processed_events predates drizzle-kit management: it was created by the hand-written 0003_feishu_adapter.sql and is intentionally absent from the drizzle snapshot / schema/index.ts exports (same as the adapter_* tables). A regular drizzle-kit generate therefore reports "no schema changes" and cannot emit ALTERs for it — and exporting the table would make the next generate emit a conflicting CREATE TABLE. So migration 0084 was scaffolded with drizzle's sanctioned escape hatch drizzle-kit generate --custom --name=processed_events_claim_lifecycle (journal + snapshot managed by drizzle-kit, LATEST synced via sync-migration-head.mjs, check-migrations.mjs passes) and the SQL body filled with idempotent ALTERs. The declarative shape in src/db/schema/processed-events.ts is kept in sync as documentation, with a NOTE explaining the situation.

Acceptance checklist (from the GoF scope comment)

• Claims get a status + expiry; success marks done; correctness does not depend on releaseClaimOnError running
• Crash injection test: pending claim committed, processor dies, redelivery after expiry is processed (route-level test in github-app-webhook-route.test.ts + seam-level test in scm-webhook-processing.test.ts)
• done still dedupes permanently (incl. attempts with TTL 0)
• Race test: concurrent deliveries of the same id (fresh and expired-takeover) → exactly one processes, against real Postgres
• Both webhook paths covered via the shared seam (see note above about the removed per-org route)
• Background sweep removes expired pending rows only, covered by unit tests (service + timer wiring)

Verification

• pnpm check — clean (remaining warnings pre-exist on untouched files)
• pnpm typecheck — 10/10 tasks pass
• pnpm build — 5/5 tasks pass
• pnpm --filter @first-tree/server test — 244 files / 2659 tests pass against real Postgres (includes 11 event-dedup lifecycle/race/sweep tests, 4 new seam tests incl. crash injection + concurrency, background-tasks sweep tests, and a route-level redelivery-after-crash test)

🤖 Generated with Claude Code