feat: add re-audit on pricing change with email notifications. - #10
Open
jitendra-ky wants to merge 20 commits into
Open
feat: add re-audit on pricing change with email notifications.#10jitendra-ky wants to merge 20 commits into
jitendra-ky wants to merge 20 commits into
Conversation
- New Drizzle schema: email_verifications table with otp_code, expires_at (10 min), last_sent_at (cooldown tracking), attempts counter, and verified_at timestamp - Two indexes: email + expires_at for fast lookup; email + verified_at for pending-only queries - Added query helpers: createEmailVerification, getLatestVerificationByEmail, deleteVerificationsForEmail, incrementVerificationAttempts, markVerificationComplete - Migration: 0005_next_lady_deathstrike.sql
OtpService:
- sendOtp(email): 5-min per-email cooldown, crypto.randomInt 6-digit
code, 10-min expiry, deletes prior pending rows before creating new
- verifyOtp(email, code): enforces expiry, max-3-attempts lockout,
increments attempt counter on wrong guess, marks verified_at on success
EmailService (Strategy pattern):
- EMAIL_PROVIDER=mock (default): logs OTP to server console, no real send
- EMAIL_PROVIDER=resend: sends via Resend API (requires RESEND_API_KEY)
- Swap providers by changing one env var — no code changes needed
Errors: OtpCooldownError, OtpExpiredError, OtpInvalidError,
OtpLockedError, OtpNotFoundError
Validators: sendOtpRequestSchema, verifyOtpRequestSchema (Zod)
Types: LeadResponse updated with role field
send-otp:
- Validates email via Zod, delegates to OtpService
- Returns { success: true, cooldown_seconds: 300 } on success
- 429 OTP_COOLDOWN with retry_after_seconds when within 5-min window
verify-otp:
- Validates full request body (email, otp_code required; company_name,
role, audit_id optional)
- Verifies OTP then atomically captures lead via LeadService
- 201 for new leads, 200 for returning; all OTP error codes mapped
to correct HTTP status (400 invalid/expired, 404 not-found, 429 locked)
New LeadCaptureModal (src/features/leads/components/): - Step 1 (Details): email (required), company, job title (both optional) - Step 2 (OTP): 6 individual digit boxes with paste support, 5-min resend countdown, inline error messages for every error code (OTP_INVALID shows attempts_remaining, OTP_EXPIRED/LOCKED/NOT_FOUND each have distinct copy) - Step 3 (Success): is_new-aware copy, auto-dismisses after 3s Integration: - Both page.tsx and audit/[id]/client.tsx updated to use new modal - localStorage flag (credex_lead_captured) persists across sessions - Consistent 4s delay before modal appears on both pages Cleanup: - Deleted old src/features/audit/components/LeadCaptureModal.tsx (simple email+company form hitting /api/leads directly, no OTP)
New test suites (32 tests across 4 suites, all passing): OtpService.test.ts (11 tests): - sendOtp: cooldown enforcement, cleanup ordering, 6-digit code shape - verifyOtp: correct code, not-found, expired, locked after 3 attempts, wrong code increments attempts, reports attempts_remaining correctly EmailService.test.ts (7 tests): - Mock provider logs to console, does not throw - Resend provider throws when RESEND_API_KEY is absent - Strategy selection via EMAIL_PROVIDER env var, defaults to mock Route tests already added in API commit (14 tests): - send-otp: success, invalid email, cooldown, missing email, 500 - verify-otp: 201 new lead, 200 returning, all OTP error codes, 500 Fix: mocked SummaryGenerationService in audit/route.test.ts to prevent real AI call causing 5s timeout (1 test was consistently failing in CI)
Single source of truth for the audit rule engine version. Bumping this string and pushing to main is the only action needed to trigger a full re-audit of all stored lead audits.
createLeadAudit - inserts a lead_audits row (onConflictDoNothing)
getLatestLeadAuditPerLead - finds each lead's latest audit row and returns
those on an older engine version (stale detection)
markLeadAuditStale - sets is_stale=true on a lead_audits row
createReauditNotification - inserts a reaudit_notifications row (deduped)
updateNotificationStatus - marks notification as sent or failed after email
LeadService: after upsertLead, creates a lead_audits row with the current AUDIT_ENGINE_VERSION so the re-audit script can track which engine version produced each lead's audit. EmailService: adds sendReauditNotification() to IEmailProvider interface and both Mock + Resend providers. Email includes savings delta, new audit tag, and a one-click re-run link.
scripts/run-reaudit.ts
Standalone tsx script that runs the full re-audit pipeline:
1. Finds all leads whose latest lead_audits row is on an older version
2. Re-runs AuditService.executeAudit() with new engine rules
3. If total_monthly_savings_usd or audit_tag changed:
- Creates a new auditsTable row
- Creates a new lead_audits row (previous_audit_id for diff view)
- Marks old lead_audits row is_stale=true
- Creates a deduped reaudit_notifications row
4. Sends one email per affected lead, updates notification status
.github/workflows/reaudit.yml
Triggers on push to main only when version.ts changes.
Runs the script directly with DATABASE_URL + RESEND_API_KEY secrets.
No HTTP endpoint needed - all work done inside the runner.
package.json: add tsx devDependency (required to run .ts script in CI)
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Implements Feature 4 of Round 2 — the side-by-side diff UI shown when a user clicks the re-run link in their re-audit notification email. ## What changed - src/lib/db/queries.ts Added getNewAuditByPreviousAuditId(oldAuditId) — reverse lookup via lead_audits.previous_audit_id to find the new audit written by the re-audit script. - src/features/audit/utils/diffUtils.ts (new) Pure, DOM-free diff utility. computeAuditDiff() matches findings by tool_name (case-insensitive), classifies each as changed/new/ removed/unchanged, and computes savings deltas. 100% test coverage. - src/features/audit/components/AuditDiffView.tsx (new) Premium diff UI component: * Header banner: engine version v1.0.0 -> v1.1.0 + annual savings delta * Summary cards: before/after monthly savings, annual savings, audit tag * Per-tool diff table: amber=changed, green=new, red=removed, muted=unchanged * Unchanged rows collapsed by default (click chevron to expand) * Updated AI summary section * CTA bar: View Full New Report + Share Updated Results - src/app/audit/[id]/page.tsx Updated Server Component to accept searchParams. When ?rerun=true: * Calls getNewAuditByPreviousAuditId(params.id) * If found: renders AuditDiffView with both old and new results * If not found (re-audit still running): shows processing banner + original audit Standard view path (?rerun absent) is completely unchanged. - src/features/audit/__tests__/AuditDiffView.test.ts (new) 16 unit tests covering: all four DiffRowStatus states, case-insensitive matching, mixed multi-tool scenarios, delta sign correctness, tag change detection, empty findings, formatUSD and formatDelta helpers. ## Verification - npm run type-check: 0 errors - npm run lint: 0 errors (14 pre-existing console warnings in scripts) - npm run test:coverage: 146/146 tests pass, 18 suites, diffUtils 100% coverage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Note for reviewers: The re-audit pipeline is triggered by pushing to
main, which requires write access to this repo. If you don't have it, here's an alternative: submit an audit on the live site and complete the OTP verification so you're registered as a lead — then let me know. I'll change a pricing rule and trigger the re-audit script on my end. You'll receive the notification email directly in your inbox and can click through to the diff view to verify the full end-to-end flow yourself. I mentioned in theROUND2_DEVLOG.mdwhy I use this method.Although I tried to add an alternative testing approach, I couldn’t complete it due to time constraints.
What this PR does
Adds a full "re-audit on pricing change" system on top of the Round 1 audit tool. Every lead who submits their AI stack is now tracked in the database across engine versions. When the audit rule engine is updated (pricing changes, new plans, revised logic), the system automatically detects every lead whose result would change, re-runs their audit, persists the new result, and sends them a single consolidated notification email with a one-click link to view an interactive diff of old vs. new recommendations.
Why
AI tool pricing is not static — Cursor restructured its plans in 2024, Claude introduced Max tiers in 2025, Copilot added Pro+. A one-time audit that goes stale is worse than no audit at all: it gives users false confidence. The core assumption is that a user who cares enough to audit their stack once will want to know when their savings opportunity has changed. This feature closes that loop automatically and turns a one-shot utility into a living recommendation engine.
How it works
Trigger: Updating
src/features/audit/engine/version.ts(bumpingAUDIT_ENGINE_VERSION) onmaintriggers a GitHub Actions workflow (.github/workflows/reaudit.yml).Pipeline (
scripts/run-reaudit.ts):lead_auditsfor every lead whose latest row has an engine version older thanAUDIT_ENGINE_VERSIONAuditService.executeAudit(tools_json)in-memory with the new rule enginetotal_monthly_savings_usdandaudit_tag— skips if unchangedauditsrow + newlead_auditsrow (withprevious_audit_idpointing to the old audit) + marks the old rowis_stale=truereaudit_notificationsrow (unique onlead_id + engine_version— no duplicates)Diff view (
/audit/[oldAuditId]?rerun=true):The email link hits the existing
/audit/[id]page with?rerun=true. The server usesprevious_audit_idto look up the new audit, then rendersAuditDiffView— a side-by-side table with color-coded rows (amber=changed, green=new, red=removed, muted=unchanged), a savings delta headline, and collapsed unchanged rows.Data flow diagram:
flowchart TD A["version.ts bump → push to main"] A --> B["reaudit.yml · GitHub Actions"] B --> C subgraph C["run-reaudit.ts"] direction TB C1["getLatestLeadAuditPerLead()"] C1 --> C2["Re-run AuditService → compare results"] C2 --> C3["createAudit() + createLeadAudit()"] C3 --> C4["createReauditNotification() — deduped"] C4 --> C5["sendReauditNotification() via Resend"] end C --> D["User clicks email link\n/audit/[oldId]?rerun=true"] D --> E["getNewAuditByPreviousAuditId(oldId)"] E --> F["AuditDiffView\nSide-by-side diff · delta headline"]New DB tables:
lead_audits— one-to-many between leads and audits, tracksengine_version,is_stale,previous_audit_idreaudit_notifications— dedup guard, one row per lead per engine versionemail_verifications— OTP verification during lead capture (also Round 2)What I cut
Pricing snapshot column on
auditstable. The spec asks for apricing_snapshotfield. I deliberately chose not to add it because pricing in this system is not external data — it's baked into the audit rule code (src/features/audit/rules/). The "pricing snapshot" is effectively the engine version + tools_json together. Adding a JSON snapshot of the pricing constants would duplicate information already recoverable from the git history of the rule files at that version. Documented clearly so reviewers can disagree.One-click unsubscribe from email. The bonus feature. I prioritized the diff view (harder, higher value, more novel) over the unsubscribe link (simpler but lower user impact in a 36h window). Next thing I'd add.
Admin dashboard (total audits, emails sent, click-through %). The
reaudit_notificationstable has all the data needed to build this. Not built — no time."What changed in the AI tooling market this week" public page. Would be a compelling growth surface but requires accumulated pricing-change data over time. The infrastructure (version bumps + detection) is there; the display layer is not.
HTTP endpoint for triggering re-audits. The spec example shows
POST /api/detect-changes. I chose GitHub Actions + a standalone script instead, because it avoids exposing a privileged endpoint on the production server and removes the need for a secret token just to call our own server. The trade-off is that a reviewer can'tcurlit — see testing instructions below.How to test it manually
Option A — Full end-to-end via engine version bump (recommended):
src/features/audit/rules/and change any pricing constant (e.g. bump a monthly price), then bumpAUDIT_ENGINE_VERSIONinsrc/features/audit/engine/version.tsfrom1.0.0to1.0.1and also change something in auditEngine for example -> (src\features\audit\engine\AuditRuleEngine.ts -> line 69)mainreaudit.yml→ runsscripts/run-reaudit.tswithDATABASE_URL,RESEND_API_KEY,EMAIL_PROVIDER=resend,NEXT_PUBLIC_BASE_URLfrom repo secretshttps://credex.rocks/audit/[oldAuditId]?rerun=true→ diff view loads