Skip to content

Add per-domain sending ramp-up (new-domain daily limits)#370

Open
jiashuoz wants to merge 4 commits into
mainfrom
claude/email-warmup-support-0czp0p
Open

Add per-domain sending ramp-up (new-domain daily limits)#370
jiashuoz wants to merge 4 commits into
mainfrom
claude/email-warmup-support-0czp0p

Conversation

@jiashuoz

@jiashuoz jiashuoz commented Jul 2, 2026

Copy link
Copy Markdown
Member

Summary

Implements per-domain sending ramp-up (a.k.a. domain warm-up): an automatic, transparent daily sending limit that rises for newly sending-verified domains over a configurable window, so a cold domain builds ISP reputation instead of blasting full volume on day one — the same protection SES/Postmark/SendGrid apply to new senders.

The ramp is scoped per domain (mailbox providers track reputation per sending domain, not per agent or account). It begins automatically when a domain first becomes sending-verified while the ramp is enabled and never re-arms on a later re-verify, so a domain that has already built reputation is never dropped back to day-one limits.

Naming: deliberately not "warmup" in customer-facing surfaces (it suggests warmup-as-a-service — synthetic mail we don't send) and not "quota" (it suggests a purchasable plan limit). "Ramp" says temporary + automatic + rising, matching SES's own "ramp-up" language.

Relates to email-warmup-support.

Design (reworked after adversarial review)

Enforced at the wire, not the handler. The gate lives in outbound.Sender.Send — the single chokepoint shared by direct sends, HITL approve (API + magic link), and the TTL auto-approve worker. Held-for-review drafts consume allowance when released, not when queued; loopback self-sends and relay-domain test sends are neither limited nor counted.

Atomic counter. domain_send_counters (one row per registrable domain (eTLD+1) + UTC day, migration 050) is claimed with a single increment-if-below-cap statement (usage.ReserveDomainSend), so concurrent bursts serialize on the row and can never jointly overshoot the cap. The counter is independent of messages/agent_identities, so agent churn (delete + recreate) cannot rewind it. Aggregating at the registrable domain (via x/net/publicsuffix, falling back to the exact domain when underivable) means sibling subdomains ramp on their own anchors but draw from one shared daily pool — verifying m1..mN.spam.example cannot multiply day-one volume by N, and PSL-listed shared apexes (github.io et al) still keep independent senders separate.

Ramp math. Linear ramp (default 50 → 2000/day over 30 days), day-indexed by UTC calendar day — the same boundary the counter resets on and retry_after_seconds points at. A domain past its ramp window is unlimited and uncounted forever (done short-circuit); "completed" is derived, never stored.

Safe enable-later. First verify always stamps the ramp anchor, but the ramp arms only while sending_ramp.enabled is true; migration 050 backfills anchors for domains verified before the feature shipped. Flipping the flag on later can therefore never retroactively limit an established sender.

Limit surface. An over-cap send returns 429 sending_ramp_limited with daily_cap / sent_today / retry_after_seconds details (carried through OutboundError.Details on send/reply/forward and approve). The TTL auto-approve worker defers a limited release to the counter reset (it bumps approval_expires_at by retry_after via identity.DeferApprovalExpiry) — it never auto-rejects on a pacing condition, and deferred rows rotate to the back of the sweep instead of head-of-line blocking other tenants' expirations. The magic-link approve page explains the message stays pending.

Metering. Billable outbound usage is recorded only after a successful delivery (matching the HITL release paths) — a throttled 429, which clients are told to retry against daily, never burns plan quota.

Client surface checklist

  • Go handler + integration tests — ramp gate wired into the sender; 429 sending_ramp_limited with pacing details when a domain hits its daily cap
  • Migration written + idempotent + safe on prod-sized tables — migration 050 adds sending_ramp_status/sending_ramp_started_at to domains (metadata-only), creates domain_send_counters, and backfills anchors for pre-existing verified domains
  • OpenAPI spec + generated types — no new /v1 endpoints; the ramp is enforced transparently on existing send paths (spec byte-identical, TestSpecGoldenNoDrift green)
  • TypeScript SDK — no new client surface
  • Python SDK sync — no new client surface
  • Python SDK async — no new client surface
  • CLI command or flag — no new CLI surface
  • MCP tool — no new MCP surface

Intentionally skipped: the ramp is a pure server-side reputation optimization on existing send paths; it reuses the standard error envelope (429 + error.code=sending_ramp_limited + details).

Operational risk

Fail-open, logged: the ramp is a reputation optimization, not a correctness gate. If the state read or slot reservation fails (transient DB blip, missing migration), the send proceeds at full volume and the enforcer logs the error — a persistently broken ramp is visible in logs, never silent.

No-op when disabled or unconfigured: disabled by default (sending_ramp.enabled: false). A self-host that never configures sending_ramp: is unaffected. Config defaults are seeded per-field, so a partial block (e.g. only target_daily) keeps the remaining defaults instead of degenerating.

Idempotent arm-once: the ramp anchor is stamped exactly once, on the first sending_status='verified' transition; re-verifies and status flaps never re-arm. Domains verified while the ramp was disabled (or before it shipped) can never be limited retroactively.

Shared relay domain unaffected: the shared relay domain stays sending_ramp_status='inactive' and is never limited.

Reserved-slot conservatism: a slot claimed for a send that subsequently fails downstream stays consumed — an error burst can only slow a ramping domain, never push it over its ramp.

Known follow-ups (not in this PR): no ramp visibility on the domain API yet (429 is the only signal); the ramp day advances by calendar even on idle days; migrating an established domain onto e2a gets day-one limits unless ops pauses the ramp per-domain (sending_ramp_status='paused', manual SQL).

Test plan

  • Schedule unit tests — ramp curve, monotonicity, UTC-midnight cap step, clock skew, sanitization
  • Enforcer unit tests — reserve semantics, done short-circuit (no counter I/O past ramp), inactive/paused/nil no-ops, fail-open logging on both dependency errors
  • usage.ReserveDomainSend DB tests — per-domain/per-day counting, refusal at cap, 40-goroutine concurrent burst grants exactly cap
  • identity.SetSendingStatus DB tests — arms on first verify when enabled; stamps anchor without arming when disabled; never re-arms or moves the anchor on re-verify/flap
  • Sender unit tests — gate consulted once with the agent's domain before the wire, limit error propagates unwrapped, validation errors don't consume slots
  • HTTP API tests — 429 sending_ramp_limited envelope with pacing details via the OutboundError.Details passthrough; plain errors unaffected
  • Config tests — partial sending_ramp: block keeps per-field defaults; omitted block stays disabled
  • Throttle-branch DB tests (post-review) — the three AsThrottleError seams driven by a REAL throttling sender: direct send → 429 with no usage metered and no message row; HITL approve → 429 with the row still pending_review; TTL worker → defers (never auto-rejects), bumps approval_expires_atretry_after, deferred row out of the next sweep
  • Counter-key tests (post-review) — counterKey eTLD+1 table (incl. PSL apexes + fallback), subdomain aggregation throttles against the shared pool, cap<=0 refused on the INSERT arm with no phantom row

All tests pass locally with make test.

https://claude.ai/code/session_01Auv4sfF5kPY3v4vybLoT8L

claude and others added 3 commits July 2, 2026 07:58
Add automatic, transparent per-domain sending warmup — the reputation-
building ramp hosted email services apply to new senders. A domain that
first becomes sending-verified begins warmup: its daily outbound volume
is capped on a linear ramp (default 50/day climbing to 2000/day over 30
days) so a cold domain builds ISP reputation instead of blasting full
volume on day one. Scoped per domain, since mailbox providers track
reputation per sending domain.

- migrations/050: warmup_status + warmup_started_at on domains.
- internal/warmup: pure Schedule.DailyCap ramp math; Enforcer.Check
  (fail-open on read errors); ThrottleError; EngagementProvider seam +
  no-op for the follow-up warmup-network phase.
- identity.Store: GetWarmupState; SetSendingStatus arms warmup exactly
  once on the first 'verified' transition and never re-arms.
- usage.Store: CountDomainSendsToday (per-domain UTC-day outbound count).
- httpapi: enforce warmup on send/reply/forward/test; over-cap returns
  429 warmup_throttled with cap/sent/retry-after. No /v1 spec surface
  change (reuses the error envelope) — spec + SDK stay byte-identical.
- config: `warmup:` block (disabled by default); wired in cmd/e2a.

Tests: schedule curve, enforcer, 429 mapping, config parse, and DB-backed
tests for the verified-stamp lifecycle and the daily counter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Auv4sfF5kPY3v4vybLoT8L
Adversarial review of the warmup ramp found the enforcement model leaky
in ways that defeated the feature. This reworks it:

Enforcement moves to outbound.Sender.Send — the one chokepoint every
producer of real upstream mail shares (direct sends, HITL approve API,
magic-link approve, TTL auto-approve worker). Previously only the HTTP
send handlers checked, so bulk-approving held drafts blasted a cold
domain uncounted. Loopback self-sends and relay-domain test sends no
longer reach the gate, so they are neither throttled nor counted.

The numerator becomes domain_send_counters (migration 050): one row per
(domain, UTC day), claimed by an atomic increment-if-below-cap at send
time (usage.ReserveDomainSend). This closes three holes in the old
count(*)-over-messages check: concurrent sends could jointly overshoot
the cap (the check-to-insert window spanned the SES round-trip);
deleting an agent cascade-deleted its message rows and rewound the
counter (self-serve cap bypass); and held drafts consumed allowance on
their hold day while their release day sent unmetered.

Ramp-state fixes:
- Enforcer honors DailyCap's done flag: a domain past its ramp is never
  throttled or counted again (was: permanently capped at target_daily,
  paying the counter query forever).
- Ramp days index by UTC calendar day, matching the counter reset and
  retry_after (was: 24h-from-anchor, letting a mid-day-verified domain
  send 2x its cap inside one window).
- Arming honors warmup.enabled (identity.SetWarmupArming): first verify
  always stamps the anchor but only arms while enabled, and migration
  050 backfills anchors for pre-existing verified domains — enabling
  warmup later can never retroactively throttle an established sender.
- config.Load seeds the schedule from warmup.DefaultSchedule, so a
  partial warmup: block keeps per-field defaults (was: target_daily
  alone produced a 1-send/day-one schedule via zero-value clamps).
- Fail-open paths now log (owned by the enforcer) instead of silently
  disabling the ramp on persistent read errors.

Plumbing: OutboundError gains Details so the 429 warmup_throttled
pacing payload survives to the v1 envelope on send/reply/forward AND
approve; the TTL worker defers (not auto-rejects) a throttled release;
the magic-link page explains the pending state; dead EngagementProvider
seam removed. No /v1 spec surface change.

Tests: enforcer reserve semantics, UTC-midnight cap step, concurrent
reservation (40 claims, exactly cap granted), disabled-arming anchor
stamp, sender gate ordering (validation before reserve), 429 detail
passthrough, partial-config defaults.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"Warmup" collides with warmup-as-a-service (synthetic engagement mail,
seed inboxes) and misleads customers into expecting e2a to actively
warm their domain; "quota" would read as a purchasable plan limit. What
this feature actually is — an automatic daily sending limit that rises
for new domains — the industry names a ramp (SES: "ramp-up"; enforced
caps at SES/Mandrill/Twilio are quotas/limits, "warm-up" is the
practice). Rename every surface while nothing is shipped, since the
error code and config keys freeze on first release:

- error code: warmup_throttled → sending_ramp_limited
- config: warmup: → sending_ramp: (same fields)
- columns: warmup_status/warmup_started_at →
  sending_ramp_status/sending_ramp_started_at (migration 050 renamed;
  never deployed, so a straight rename)
- package: internal/warmup → internal/sendramp; seams follow
  (SetSendingRampGate, SetSendingRampArming, GetSendingRampState,
  SendingRampLimitError, SendingRampConfig)
- user-facing text now says "daily sending limit is ramping up";
  docs keep "domain warm-up" as a searchable alias

Behavior unchanged; full test suite green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jiashuoz jiashuoz changed the title Add per-domain sending warmup ramp Add per-domain sending ramp-up (new-domain daily limits) Jul 3, 2026
…ring, sweep deferral

Four fixes from the multi-agent review of this branch, plus the test
coverage it flagged as missing:

- Key domain_send_counters on the REGISTRABLE domain (eTLD+1 via
  x/net/publicsuffix, falling back to the exact domain when underivable).
  Sibling subdomains now draw from one shared daily pool, closing the
  bypass where verifying m1..mN.spam.example multiplied day-one volume
  by N. PSL-listed apexes (github.io et al) keep independent senders
  separate; ramp state/anchor stays per exact domain.

- Meter billable outbound usage only AFTER a successful delivery in
  DeliverOutbound (recordOutboundUsage, both loopback and relay
  branches). A ramp-throttled 429 — a documented retry-daily outcome —
  no longer burns plan quota with nothing delivered; matches the three
  HITL release paths, which already recorded on success only.

- Defer a ramp-throttled TTL auto-approve to the counter reset: the
  worker now bumps approval_expires_at by RetryAfter (new
  identity.DeferApprovalExpiry, guarded to pending_review). Retrying
  sooner cannot succeed, and without the bump the earliest-expired
  throttled rows head-of-line block every sweep — past batchSize rows
  they starve every other tenant's expirations until UTC midnight.

- Honor the cap contract in ReserveDomainSend for cap <= 0: the upsert's
  guard lives on the DO UPDATE arm only, so the first reservation of a
  (domain, day) previously slipped past a non-positive cap. Unreachable
  via the enforcer (schedule floors the cap at 1) but the exported
  contract now holds for any caller.

New tests drive the three ThrottleError detection seams with a REAL
throttling sender (not a fabricated error): direct send maps to 429
with no usage metered and no message row; HITL approve maps to 429 with
the row still pending_review; the TTL worker defers (never auto-rejects)
and rotates the row out of the next sweep. Plus counterKey table tests,
subdomain aggregation on the enforcer, and cap<=0 at the store.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants