fix: resolve queue job visibility against the database clock

[AKogut]

The bug

A previously dismissed "flaky test" turned out to be a real defect in the queue core. A tight reproduction (300 enqueue-then-dequeue cycles) failed 299 times out of 300.

Job visibility is a comparison between a stored timestamp and "now", but the two sides came from different clocks:

• visible_at was written by the application clock. Prisma populated the @default(now()) column client-side, while dequeue compared it against Postgres now(). On a machine whose host runs 1 ms ahead of the database container, a freshly enqueued job was invisible. In production this generalizes to NTP drift between the app host and the database: a few seconds of skew stalls ingestion for that long.

row.visible_at : 11:55:37.699   <- Node clock
postgres now() : 11:55:37.699
skew           : 1 ms

• TIMESTAMP(3) rounds up. The column stores milliseconds while now() is microsecond-precision, so a stored value lands up to half a millisecond in the future. Measured across 200 samples: 24% of rows were written with visible_at > now().

Retry backoff had the same defect — it was computed as new Date(Date.now() + backoffMs) in application code.

The fix

• visible_at is now database-generated (dbgenerated("now()")), with a migration altering the column default.
• Retry backoff is computed in SQL as now() + interval instead of from the application clock.
• dequeue compares against now() + 1 millisecond — a tolerance equal to the column's storage granularity, so rounding cannot hide a job.
• Visibility timeout and backoff intervals now use millisecond precision instead of being rounded to whole seconds.

Evidence

Stage	Failures (300 cycles)
Before	299
Database clock only	77
Plus storage-granularity tolerance	0

Added a regression test that enqueues and immediately dequeues 25 times — it would have caught this. Two full cold-database pipeline runs: 123 tests, 38/38 tasks, no skips.

Why it survived this long

The worker polls on an interval, so a millisecond of invisibility was masked in normal operation. The only symptom was an occasionally flaky test — which is precisely the signal this product exists to surface. Documented in ADR-0004 so the clock-domain rule is explicit.

Not addressed here

dequeue increments attempts on every pickup, so a job that outlives the visibility timeout gets picked up again and inflates its attempt count toward the dead-letter cap. That is a separate design decision (ownership token on completion, or a longer timeout) and is deliberately out of scope.