feat(billing): add microdollar_usage_daily rollup with dual-write#3270
feat(billing): add microdollar_usage_daily rollup with dual-write#3270RSO wants to merge 7 commits into
Conversation
Adds a per-(user, organization, day) rollup of microdollar_usage.cost, maintained atomically alongside the existing CTE-based microdollar_usage insert in insertUsageAndMetadataWithBalanceUpdate. This is PR 1 of 2 for replacing the kiloPass.getAverageMonthlyUsageLast3Months read query, which currently scans 3 months of the 800M-row microdollar_usage table on every authenticated profile page load (~42% of read-replica time). PR 1 only adds the table and the dual-write; reads still hit the raw table. PR 2 will switch the read after the historical backfill runs.
Code Review SummaryStatus: 2 Issues Found | Recommendation: Address before merge Executive SummaryTwo pre-existing issues remain unaddressed: a redundant dead-code condition in the test helper and a missing trailing newline in the migration file. The new commit only reformats Overview
Issue Details (click to expand)WARNING
SUGGESTION
Other Observations (not in diff)
Files Reviewed (9 files)
Fix these issues in Kilo Cloud Reviewed by claude-sonnet-4.6 · 197,390 tokens Review guidance: REVIEW.md from base branch |
Summary
PR 1 of 2 to eliminate the 3-month scan of the 800M-row
microdollar_usagetable that powerskiloPass.getAverageMonthlyUsageLast3Monthsand currently accounts for ~42% of read-replica execution time.This PR adds the rollup infrastructure but does not switch the read path:
microdollar_usage_dailytable with a row per(kilo_user_id, organization_id, usage_date)and two partial unique indexes (one fororganization_id IS NULL, one forIS NOT NULL). Modeled onexa_monthly_usage.insertUsageAndMetadataWithBalanceUpdate(apps/web/src/lib/ai-gateway/processUsage.ts) with a new CTE that upserts the matching daily row, atomic with the existingmicrodollar_usageinsert in the same single SQL statement. Zero-cost rows are skipped viaWHERE ${cost} <> 0.insertMicrodollarUsageWithDailyRollupinapps/web/src/tests/helpers/microdollar-usage.helper.tsfor tests that bypass the production hot path and write tomicrodollar_usagedirectly. (PR 2 tests will use this; existing tests are left alone since they don't read from the rollup yet.)After deploy, every new
microdollar_usagerow is mirrored intomicrodollar_usage_daily. The historical backfill and the read-path switch are tracked in the implementation plan and will land in PR 2.Plan:
.kilo/plans/1778835512309-hidden-sailor.md.Verification
pnpm drizzle migrate.microdollar_usage_dailyrow appeared with the expected total. Inserted a second row on the same day and confirmed the row'stotal_cost_microdollarsincremented (not duplicated). Inserted an org-scoped row and confirmed it created a separate row hitting the org partial unique index, leaving the personal row unchanged. Inserted a zero-cost row and confirmed the rollup was untouched.created_atset explicitly to a UTC date lands inmicrodollar_usage_daily.usage_dateas the same calendar day.Visual Changes
N/A — backend / schema only.
Reviewer Notes
ON CONFLICTtarget with a JS-level branch on whetherorganization_idis null. PostgreSQL only allows a singleON CONFLICTclause per statement and the two partial indexes have different column lists, so this branch is required.microdollar_usage_dailystores both personal and org-scoped rows in one table to mirrorexa_monthly_usage. Only the personal partial index is exercised by the immediate consumer in PR 2; the org-scoped rows are written but unused for now.kilo_user_id,organization_id,usage_date, aggregated cost). PersoftDeleteUserpolicy,microdollar_usageitself is retained on soft-delete as a billing record (apps/web/src/lib/user.tsretention list); this derived rollup follows the same retention. NosoftDeleteUserchange required.INSERT ... ON CONFLICT DO UPDATEagainst a small, tightly-indexed table. It shares the same transaction as the existingmicrodollar_usageinsert and the user-balanceUPDATE, so atomicity is preserved with no extra round-trip.pnpm drizzle generateper repo convention; no hand-written DDL.