From 54e649b288ca66d44a6eb596e7bb96c4a4944bed Mon Sep 17 00:00:00 2001 From: minichorus-pm Date: Tue, 23 Jun 2026 16:18:18 -0400 Subject: [PATCH] docs: webhook delivery contract, manage-webhooks how-to, and build-webhook-receiver tutorial MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three new pages plus three index updates, addressing the webhook documentation gap in the v2 docs branch (the first of the two docs offers from the dev-list thread on Final Prep for v2). ## docs/explanations/webhook-delivery.md (new) The conceptual contract. What webhooks are for, the payload shape (with v2's reply_to and tags from PR #74), the HMAC-SHA256 scheme (timestamp.body, not body.timestamp — bug-shape worth flagging explicitly), the headers MAIL sends, the receiver verification checklist, the retry ladder (6 attempts: immediate, +1s, +30s, +5min, +1h, +6h — total window ~7h31m), the retry conditions (timeout / 5xx / 429 retry; 2xx / other 4xx don't), and the 'inbox is source of truth' contract that shapes how receivers should handle internal failures. Sources verified against src/mail/server/src/mail_server/backends/base.py (_handle_webhook_delivered and _webhook_delivered_post on origin main; the v2-docs branch is currently behind main on the schema changes from PR #74). Wrote against the canonical main-branch shape so the docs match the v2 release contract; the docs branch needs the rebase before merge. ## docs/howtos/manage-webhooks.md (new) Operator's guide. Secret generation, POST /admin/webhooks (with events and url), GET /admin/webhooks for listing, GET /admin/webhooks/{id} for inspection, PATCH /admin/webhooks/{id} for URL / secret rotation (event types are immutable in v2), DELETE /admin/webhooks/{id}. Includes the rotation coordination note (both sides must update at the same moment to avoid signature failures in flight). ## docs/tutorials/build-webhook-receiver.md (new) Implementer's walkthrough. A single-file FastAPI receiver with: - verify_signature using HMAC over raw bytes (calls out the two most common bugs: re-encoded JSON breaking signature; missing the timestamp.body prefix). - is_duplicate / mark_processed for event_id-based dedup with a 24-hour garbage-collection window. - is_timestamp_in_window for 5-min skew rejection. - The full endpoint composing them with the right error codes (503 if secret not configured, 408 for skew, 403 for bad signature, 200 with status=duplicate for retries). - Registration command for end-to-end test. - Diagnostic checklist for the 'nothing arrives' case. ## Index updates docs/{explanations,howtos,tutorials}/README.md each gain a row linking to the new page. ## Followups in scope of the original offer - docs/howtos/manage-mailing-lists.md exists as a stub today; drafting that one is the second piece of the offer and will land as a separate commit on the same branch. - docs/references/http-api.md is a stub overall (not just for webhooks). The webhook-specific endpoints could be sketched there in a later pass; deferred so this commit stays focused on the conceptual + tutorial layer. ## Verification The four facts that are easiest to get wrong (and that I had ground truth on from the chorus-side webhook receiver): - HMAC inputs: 'timestamp.raw_body' not 'raw_body.timestamp'. - X-MAIL-Timestamp value: Unix seconds as a STRING, used in both the HMAC and the header so receivers can recompute from the header alone. - Signature header format: 'sha256='. - Body bytes: payload.model_dump_json() (Pydantic's canonical JSON), posted as-is — re-encoded JSON has different bytes and breaks verification. All four match what _webhook_delivered_post does in src/mail/server/src/mail_server/backends/base.py:653. --- docs/explanations/README.md | 1 + docs/explanations/webhook-delivery.md | 263 ++++++++++++++++++++++ docs/howtos/README.md | 1 + docs/howtos/manage-webhooks.md | 148 +++++++++++++ docs/tutorials/README.md | 1 + docs/tutorials/build-webhook-receiver.md | 265 +++++++++++++++++++++++ 6 files changed, 679 insertions(+) create mode 100644 docs/explanations/webhook-delivery.md create mode 100644 docs/howtos/manage-webhooks.md create mode 100644 docs/tutorials/build-webhook-receiver.md diff --git a/docs/explanations/README.md b/docs/explanations/README.md index 91a73b9..03706b2 100644 --- a/docs/explanations/README.md +++ b/docs/explanations/README.md @@ -12,6 +12,7 @@ They are for understanding, not for step-by-step tasks or exhaustive lookup. | [Addressing Model](addressing-model.md) | Why does MAIL use host-scoped and swarm-scoped addresses? | | [Delivery Model](delivery-model.md) | Why are daemons responsible for message delivery? | | [Security Model](security-model.md) | What are the main trust boundaries and risks? | +| [Webhook Delivery](webhook-delivery.md) | What is the webhook contract — payload shape, HMAC signing, retry behavior, and the inbox-is-source-of-truth assumption? | | [MAIL v1 Legacy Runtime](mail-v1-legacy.md) | How should readers interpret the archived v1 runtime and docs? | | [Documentation System](documentation-system.md) | How should maintainers decide where a new page belongs? | diff --git a/docs/explanations/webhook-delivery.md b/docs/explanations/webhook-delivery.md new file mode 100644 index 0000000..fec2591 --- /dev/null +++ b/docs/explanations/webhook-delivery.md @@ -0,0 +1,263 @@ +# Webhook Delivery + +Status: draft + +## Scope + +How MAIL notifies external consumers when mail is delivered: the event +shape, the security model, the retry behavior, and the assumptions the +contract places on receivers. + +This document is for implementers building a webhook consumer (a +service that receives MAIL events and routes them somewhere else). +The matching how-to for *registering* webhooks via the admin API is +[Manage Webhooks](../howtos/manage-webhooks.md); the matching tutorial +for *building* a receiver end-to-end is [Build a Webhook +Receiver](../tutorials/build-webhook-receiver.md). + +## What webhooks are for + +The MAIL inbox is the durable surface: mail lives there, indexed by +recipient, and any authenticated user-agent can poll it via the +inbox endpoints. Webhooks are a *push* alternative: when a message +is delivered to a recipient's inbox, the MAIL server fires an HTTP +`POST` to one or more registered URLs with a structured payload, so +downstream services can react without polling. + +Webhooks do not replace the inbox. The inbox is the source of truth. +A webhook that fails to deliver is a notification missed; the message +itself is still readable by the recipient via the normal inbox API. +This shapes the security and retry contract below. + +## Event types + +The only `event` value in v2 is `mail.delivered`. A future release +may add other event types; receivers should reject events whose +`event` field they do not recognize, but should not fail registration +on the presence of an unknown event in the `events` array (the +`/admin/webhooks` validator already gates that). + +## Payload shape + +Every `mail.delivered` event is delivered as a JSON request body +shaped like: + +```json +{ + "event": "mail.delivered", + "event_id": "evt_", + "delivered_at": "2026-06-24T19:31:00.000000+00:00", + "message": { + "message_id": "msg_", + "reply_to": null, + "sender": "alice@chorus@example.com", + "recipient": "bob@chorus@example.com", + "subject": "Daily briefing", + "body": "…", + "tags": [], + "sent_at": "2026-06-24T19:30:55.123456+00:00", + "swarm": "chorus", + "metadata": {} + } +} +``` + +Field notes: + +- `event_id` is unique per delivery attempt SET but is reused across + retries (see [Retries](#retries)). Receivers MUST treat + `event_id` as the dedup key — a webhook receiver that processes + the same `event_id` more than once is a bug. +- `message_id` is prefixed with `msg_`. The bare UUID is stored on + the canonical `MAILMessage`; the prefix is added at webhook + payload construction time. Use the prefixed form when fetching the + full message via the inbox API. +- `reply_to`, when set, is the prefixed `message_id` of the original + message this is replying to. +- `tags` is a list of slug-shaped strings the sender attached. +- `metadata.list_address`, when present, indicates the delivery + originated from a list expansion. Use it to surface the originating + list to the end-recipient. + +Refer to [Data Models](../references/data-models.md) for the full +field-by-field schema of the inner `MAILMessageInWebhook`. + +## Security model + +### Why HMAC + +MAIL emits webhooks to URLs configured by an administrator. The +receiver needs to verify that an incoming request actually came from +MAIL (not from a third party who guessed or scanned the URL). The +shared mechanism is an HMAC signature over the request body, computed +with a secret known only to MAIL and the receiver. + +### What gets signed + +MAIL computes the signature as: + +``` +signature = HMAC-SHA256(secret, f"{timestamp}.{raw_body}") +``` + +Where: + +- `timestamp` is the value of the `X-MAIL-Timestamp` header (Unix + seconds since the epoch, as a string). +- `raw_body` is the *exact byte sequence* of the request body. MAIL + signs `payload.model_dump_json()` (Pydantic's canonical JSON + serialization) and posts those same bytes as the request body. + Re-encoding via `json=...` would produce different bytes (different + key order, whitespace, type coercion) and break verification. + +The `secret` is the value supplied when the webhook was registered. + +### Headers sent on every webhook POST + +| Header | Value | +| ------------------ | ------------------------------------------------- | +| `Content-Type` | `application/json` | +| `X-MAIL-Event-Id` | The `event_id` from the payload. | +| `X-MAIL-Timestamp` | Unix seconds since epoch, as a string. | +| `X-MAIL-Signature` | `sha256=` where `` is the HMAC digest. | +| `User-Agent` | `Multi-Agent-Interface-Layer-Server/2.0.0 (...)` | + +### Receiver verification + +A correct receiver does the following on every request: + +1. Read `X-MAIL-Timestamp` and reject the request (`408` or `400`) + if it is more than 5 minutes from the receiver's clock. This + bounds the replay window. +2. Read `X-MAIL-Signature` and strip the `sha256=` prefix. +3. Recompute `HMAC-SHA256(secret, f"{timestamp}.{raw_body}")` over + the raw request body bytes (NOT the parsed JSON). +4. Compare to the received digest using a constant-time comparison. + Reject (`403`) if they differ. +5. Read `X-MAIL-Event-Id` and check it against a recent-events store. + If it has been processed in the last ~24 hours, return `200` with + a no-op response (the request is a retry; the original processing + stands). +6. Process the event. Return `200` (or `202`) on success. + +Step 5 is where the dedup contract lives. MAIL retries on transient +failure (see below) and reuses the same `event_id` across retries. +A receiver that does not dedup will process the same delivery +multiple times under load or after any transient outage. + +### What if the secret isn't configured + +A receiver that has registered a webhook but does not yet have the +secret in its environment SHOULD reject incoming requests with +`503 Service Unavailable` (not `403`). `403` would suggest a real +authentication failure; `503` correctly signals "I'm not ready, +please retry." + +## Retries + +MAIL fires up to **six attempts** per event, with the following +delays between attempts: + +| Attempt | Delay before this attempt | Cumulative wall-clock | +| ------- | ------------------------- | ---------------------- | +| 1 | (immediate) | 0 | +| 2 | 1 second | ~1 s | +| 3 | 30 seconds | ~31 s | +| 4 | 5 minutes | ~5 min | +| 5 | 1 hour | ~1 h | +| 6 | 6 hours | ~7 h | + +After the sixth attempt, MAIL gives up. The total retry window is +roughly **7 hours and 31 seconds** from the first attempt. + +A retry is triggered when `_webhook_delivered_post` returns `True`, +which happens for any of: + +- `httpx.TimeoutException` on the request. +- A `5xx` status code from the receiver. +- A `429 Too Many Requests` status code. + +A retry is NOT triggered (and the event is considered delivered or +abandoned) for: + +- A `2xx` status code (success). +- A `4xx` status code other than `429` (the receiver explicitly + rejected the request; retries won't change that). + +### Implications for receivers + +- A receiver that needs to throttle MAIL's webhook firing should + return `429` rather than starve. MAIL backs off cleanly. +- A receiver that detects a permanently malformed payload should + return `4xx` (not `5xx`). MAIL will not retry, which is the + correct behavior — the next event will succeed. +- A receiver should NOT return `5xx` for "I couldn't route this + internally but I have the message stored." That makes MAIL retry + unnecessarily. Instead, return `200` — MAIL's inbox is the source + of truth; the routing failure does not need MAIL's help to + recover. + +## The "inbox is source of truth" contract + +This is the single most important assumption a receiver makes: + +> If a webhook delivery fails, the message is not lost. The +> recipient can still poll their MAIL inbox via the regular HTTP +> API. The webhook is a notification — its failure shapes UX, not +> correctness. + +In practice this means: + +- A receiver that successfully verifies the signature, accepts the + event_id as new, but then fails internally while processing the + event SHOULD STILL RETURN `200`. The event is recorded as + processed; the internal failure is the receiver's problem to + recover from (it can read the message from the MAIL inbox on its + own schedule). +- A receiver MUST NOT return `5xx` to "force MAIL to retry." MAIL's + retries are for transport failures, not for receiver-internal + bugs. The retry schedule above is short enough that downstream + systems can fail and recover quickly without webhook help. + +## Reliability and ordering + +The webhook contract guarantees: + +- **At-least-once delivery** within the 7-hour retry window. After + retries exhaust, the message is still in the recipient's inbox + and can be fetched there. +- **Per-event idempotency via `event_id`.** Receivers MUST dedup on + `event_id` to handle retries correctly. + +The contract does NOT guarantee: + +- **Ordering.** Webhooks for related events (e.g., several mails to + the same recipient in rapid succession) may arrive out of order + due to retry interleavings or concurrent firing. Receivers MUST + treat each event independently. The `sent_at` and `delivered_at` + timestamps can be used to reconstruct ordering if needed. +- **Exactly-once delivery.** Dedup by `event_id` collapses the + at-least-once delivery to at-most-once *processing* in the + receiver's domain, but MAIL itself can fire the same event_id up + to six times. +- **Synchronous delivery.** Webhook firing happens asynchronously + on the MAIL server. A successful `POST /drafts/{id}/send` (or + similar) does not block on webhook delivery. + +## See also + +- [Manage Webhooks](../howtos/manage-webhooks.md) — registering, + inspecting, and deleting webhooks via the admin API. +- [Build a Webhook Receiver](../tutorials/build-webhook-receiver.md) + — step-by-step tutorial walking through the signature + verification, dedup, and processing of a real receiver. +- [Delivery Model](delivery-model.md) — broader context on how MAIL + routes a message from sender to recipient inbox. +- [Security Model](security-model.md) — the broader auth and + authorization model the webhook contract sits within. +- [Data Models](../references/data-models.md) — formal field-by-field + schemas for `MAILWebhook`, `MAILMessageInWebhook`, and the + envelope. +- [HTTP API](../references/http-api.md) — the formal route list, + including the webhook firing target shape and the admin + registration endpoints. diff --git a/docs/howtos/README.md b/docs/howtos/README.md index fbe8df3..67587c2 100644 --- a/docs/howtos/README.md +++ b/docs/howtos/README.md @@ -16,6 +16,7 @@ ordered steps, and stop when the task is complete. | [Manage User-Agents](manage-user-agents.md) | Create, inspect, and remove agents, users, admins, and daemons. | | [Manage Swarms](manage-swarms.md) | Create, inspect, and delete swarms. | | [Manage Mailing Lists](manage-mailing-lists.md) | Create lists and manage subscriptions or members. | +| [Manage Webhooks](manage-webhooks.md) | Register, inspect, update, and delete webhook subscriptions. | | [Regenerate API Artifacts](regenerate-api-artifacts.md) | Refresh generated OpenAPI or documentation artifacts. | | [Run the Test Suite](run-tests.md) | Run focused or full repository tests. | diff --git a/docs/howtos/manage-webhooks.md b/docs/howtos/manage-webhooks.md new file mode 100644 index 0000000..573c0e5 --- /dev/null +++ b/docs/howtos/manage-webhooks.md @@ -0,0 +1,148 @@ +# Manage Webhooks + +Status: draft + +## Goal + +How to register, inspect, update, and delete webhook subscriptions on +a MAIL server using the admin API. Webhooks let downstream services +receive `mail.delivered` events without polling — see [Webhook +Delivery](../explanations/webhook-delivery.md) for the conceptual +contract. + +## Starting Point + +You have admin credentials for the MAIL server, and you know the +public URL the webhook should fire against. You have a shared secret +already agreed with the receiver, or you're prepared to generate one. + +## Steps + +### 1. Generate a secret (if you don't already have one) + +Webhook signatures use an HMAC-SHA256 with a shared secret. The +secret must be known to both MAIL and the receiver, and not exposed +elsewhere. A reasonable generator: + +```bash +python -c "import secrets; print(secrets.token_urlsafe(32))" +``` + +Save the resulting string in a secure location accessible to the +receiver process. The receiver loads it from a config file or env +var; MAIL stores it on the registered webhook record. + +### 2. Register the webhook + +`POST /admin/webhooks` with the receiver URL, the events to +subscribe to, and the secret. v2 supports one event type +(`mail.delivered`); future versions may add more. + +```bash +ADMIN_TOKEN="$(cat ~/.mail/admin.token)" +SECRET="…" # from step 1 + +curl -sS -X POST "$MAIL_SERVER/admin/webhooks" \ + -H "Authorization: Bearer $ADMIN_TOKEN" \ + -H "Content-Type: application/json" \ + -d "$(jq -n \ + --arg url "https://my-receiver.example.com/mail/webhook" \ + --arg secret "$SECRET" \ + '{url: $url, events: ["mail.delivered"], secret: $secret}')" +``` + +A successful response returns the new webhook record (including its +generated `webhook_id`): + +```json +{ + "webhook": { + "webhook_id": "wh_abc12345-…", + "url": "https://my-receiver.example.com/mail/webhook", + "events": ["mail.delivered"], + "secret": "…" + }, + "metadata": {} +} +``` + +Save the `webhook_id` — you'll need it to inspect, update, or delete +the registration later. The secret is also stored in MAIL's backend; +the receiver only needs its own copy. + +### 3. List all registered webhooks + +`GET /admin/webhooks` returns the IDs of every webhook on the +server: + +```bash +curl -sS "$MAIL_SERVER/admin/webhooks" \ + -H "Authorization: Bearer $ADMIN_TOKEN" +``` + +```json +{ + "webhook_ids": ["wh_abc12345-…", "wh_def67890-…"], + "metadata": {} +} +``` + +To get the full record for a specific webhook, use +`GET /admin/webhooks/{webhook_id}`: + +```bash +curl -sS "$MAIL_SERVER/admin/webhooks/wh_abc12345-…" \ + -H "Authorization: Bearer $ADMIN_TOKEN" +``` + +### 4. Update an existing webhook + +`PATCH /admin/webhooks/{webhook_id}` can change the receiver URL or +rotate the secret. The webhook_id and the subscribed events are +immutable; to change events you must delete and re-register. + +```bash +curl -sS -X PATCH "$MAIL_SERVER/admin/webhooks/wh_abc12345-…" \ + -H "Authorization: Bearer $ADMIN_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"url": "https://new-receiver.example.com/mail/webhook", "secret": "new-secret"}' +``` + +When rotating a secret, coordinate with the receiver so both sides +update at the same moment; otherwise webhooks delivered between the +two updates will fail signature verification on the receiver side. + +### 5. Delete a webhook + +`DELETE /admin/webhooks/{webhook_id}` removes the registration. MAIL +will stop firing webhooks to that URL immediately. + +```bash +curl -sS -X DELETE "$MAIL_SERVER/admin/webhooks/wh_abc12345-…" \ + -H "Authorization: Bearer $ADMIN_TOKEN" +``` + +In-flight retries for events that were already being delivered when +the webhook was deleted are not interrupted; if a retry attempt +succeeds, the receiver still gets the event. After the retry +schedule exhausts (or succeeds), no further events fire. + +## Validation + +After registering a webhook, you can confirm it works end-to-end by: + +1. Sending a test message to a recipient on the server. +2. Watching the receiver's logs for an incoming `POST` with the + expected event_id, signature, and payload. +3. Confirming the receiver returns `200`. (A non-2xx response will + trigger MAIL's retry ladder; see [Webhook + Delivery](../explanations/webhook-delivery.md#retries).) + +## See also + +- [Webhook Delivery](../explanations/webhook-delivery.md) — the + contract MAIL emits and the receiver must verify. +- [Build a Webhook Receiver](../tutorials/build-webhook-receiver.md) + — implementer's tutorial for writing a receiver from scratch. +- [HTTP API](../references/http-api.md) — full route and response + reference. diff --git a/docs/tutorials/README.md b/docs/tutorials/README.md index a077d11..3c67bf4 100644 --- a/docs/tutorials/README.md +++ b/docs/tutorials/README.md @@ -11,6 +11,7 @@ needed to complete the lesson, and be tested end to end before release. | [Run MAIL Locally](run-local-mail.md) | Start a local memory-backed server, run a daemon, and observe local delivery. | `src/mail/server/docs/tutorials/quickstart.md`, `src/mail/daemon/src/mail_daemon/maild/api.py` | | [Send Your First MAIL Message](send-first-message.md) | Log in, compose a draft, send it, and inspect inbox/outbox state. | `src/mail/client/docs/tutorials/quickstart.md`, `src/mail/client/src/mail_client/cli.py` | | [Build a Minimal HTTP Client](build-minimal-http-client.md) | Authenticate and interact with MAIL using raw HTTP calls. | `spec/openapi.yaml`, `src/mail/protocol/src/mail_protocol/network/` | +| [Build a Webhook Receiver](build-webhook-receiver.md) | Build a correct HTTP receiver for MAIL's `mail.delivered` webhook events. | `src/mail/server/src/mail_server/backends/base.py`, `docs/explanations/webhook-delivery.md` | ## Tutorial Checklist diff --git a/docs/tutorials/build-webhook-receiver.md b/docs/tutorials/build-webhook-receiver.md new file mode 100644 index 0000000..19f6fce --- /dev/null +++ b/docs/tutorials/build-webhook-receiver.md @@ -0,0 +1,265 @@ +# Build a Webhook Receiver + +Status: draft + +## Goal + +Walk through building a correct MAIL webhook receiver from scratch. +By the end you'll have a small HTTP server that verifies signatures, +dedupes retries, and processes `mail.delivered` events. + +The companion explainer is [Webhook +Delivery](../explanations/webhook-delivery.md). This tutorial assumes +you've read it; we'll reference its sections rather than restate the +contract. + +## Prerequisites + +- A running MAIL server you can register webhooks against (see + [Run a Local MAIL](run-local-mail.md)). +- Python 3.12+ with `fastapi`, `uvicorn`, and `httpx`. +- A receiver URL MAIL can reach. For local development, a tunnel + (e.g., `cloudflared`) or both processes on the same host both + work. + +## The receiver, end to end + +We'll build a single-file FastAPI app that: + +1. Accepts `POST /mail/webhook`. +2. Verifies the `X-MAIL-Timestamp`, `X-MAIL-Signature`, and the + `Content-Type`. +3. Dedupes against `X-MAIL-Event-Id`. +4. Parses the payload, then prints a one-line summary. + +### Set up + +Create a new directory and install the dependencies: + +```bash +mkdir mail-receiver && cd mail-receiver +uv init +uv add fastapi uvicorn +``` + +### The signature verification function + +The signature scheme is documented in detail in [Webhook Delivery → +Security model](../explanations/webhook-delivery.md#security-model). +The receiver's job is to recompute `HMAC-SHA256(secret, +f"{timestamp}.{raw_body}")` over the *raw bytes* it received (not +the parsed JSON), then compare in constant time. + +```python +import hashlib +import hmac + + +def verify_signature( + *, raw_body: bytes, timestamp: str, signature: str, secret: str +) -> bool: + """ + Return True iff ``signature`` is a valid HMAC-SHA256 over + ``f"{timestamp}.{raw_body}"`` keyed by ``secret``. + + Signature comes in as ``"sha256="``; strip the prefix before + comparison. + """ + if not signature.startswith("sha256="): + return False + received = signature[len("sha256=") :] + + expected = hmac.new( + key=secret.encode("utf-8"), + msg=f"{timestamp}.".encode("utf-8") + raw_body, + digestmod=hashlib.sha256, + ).hexdigest() + + return hmac.compare_digest(received, expected) +``` + +A common bug at this step is to recompute the HMAC over the *parsed +and re-serialized* JSON body, which produces different bytes than +the originally-signed body and breaks verification. Always operate +on the raw bytes you received on the wire. + +Another common bug is to forget the `f"{timestamp}."` prefix. The +signed message is `timestamp.body`, not just `body`. + +### Dedup against event_id + +MAIL retries on transient failures (see [Retries](../explanations/webhook-delivery.md#retries)) +and reuses the same `event_id` for every attempt. A correct +receiver remembers recently-processed event_ids and short-circuits +duplicates. For this tutorial, an in-memory set is enough: + +```python +from collections import deque +from datetime import datetime, timezone + +PROCESSED_EVENTS: deque[tuple[str, datetime]] = deque(maxlen=10_000) + + +def is_duplicate(event_id: str) -> bool: + """ + Return True iff ``event_id`` has already been processed. + Garbage-collects entries older than 24 hours on each call. + """ + now = datetime.now(timezone.utc) + # Drop expired entries from the left. + while PROCESSED_EVENTS and (now - PROCESSED_EVENTS[0][1]).total_seconds() > 86400: + PROCESSED_EVENTS.popleft() + return any(e[0] == event_id for e in PROCESSED_EVENTS) + + +def mark_processed(event_id: str) -> None: + PROCESSED_EVENTS.append((event_id, datetime.now(timezone.utc))) +``` + +For production use, replace this with a real durable store (SQLite, +Redis, a database table) so dedup survives restarts. The 24-hour +window matches MAIL's retry exhaustion behavior with a safety +margin. + +### Timestamp skew window + +Reject any request whose `X-MAIL-Timestamp` is more than 5 minutes +from the receiver's clock. This bounds the replay window and catches +clock-drift bugs early. + +```python +import time + +SKEW_WINDOW_SECONDS = 5 * 60 + + +def is_timestamp_in_window(timestamp: str) -> bool: + try: + sent_at = int(timestamp) + except ValueError: + return False + return abs(int(time.time()) - sent_at) <= SKEW_WINDOW_SECONDS +``` + +### The full receiver + +```python +import os + +from fastapi import FastAPI, HTTPException, Request + +SECRET = os.environ.get("MAIL_WEBHOOK_SECRET") + +app = FastAPI() + + +@app.post("/mail/webhook") +async def mail_webhook(request: Request) -> dict[str, object]: + if SECRET is None: + # Not configured yet. Tell MAIL to retry later. + raise HTTPException( + status_code=503, detail="Webhook secret not configured." + ) + + timestamp = request.headers.get("X-MAIL-Timestamp") + signature = request.headers.get("X-MAIL-Signature") + event_id = request.headers.get("X-MAIL-Event-Id") + + if not timestamp: + raise HTTPException(status_code=400, detail="Missing X-MAIL-Timestamp.") + if not signature: + raise HTTPException(status_code=403, detail="Missing X-MAIL-Signature.") + if not event_id: + raise HTTPException(status_code=400, detail="Missing X-MAIL-Event-Id.") + + if not is_timestamp_in_window(timestamp): + raise HTTPException(status_code=408, detail="Timestamp outside skew window.") + + raw_body = await request.body() + + if not verify_signature( + raw_body=raw_body, timestamp=timestamp, signature=signature, secret=SECRET + ): + raise HTTPException(status_code=403, detail="Invalid signature.") + + if is_duplicate(event_id): + return {"status": "duplicate", "event_id": event_id} + + import json + + payload = json.loads(raw_body) + message = payload["message"] + + # Process the event. For this tutorial, just print. + print( + f"[mail.delivered] {message['sender']} → {message['recipient']}: " + f"{message['subject']}" + ) + + mark_processed(event_id) + return {"status": "ok", "event_id": event_id} +``` + +Run it with: + +```bash +MAIL_WEBHOOK_SECRET="" uv run uvicorn receiver:app --port 8000 +``` + +### Register with MAIL + +In another shell, register the receiver per [Manage +Webhooks](../howtos/manage-webhooks.md): + +```bash +curl -sS -X POST "$MAIL_SERVER/admin/webhooks" \ + -H "Authorization: Bearer $ADMIN_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"url": "http://localhost:8000/mail/webhook", + "events": ["mail.delivered"], + "secret": ""}' +``` + +### Send a test message + +Send a message to any recipient on the MAIL server. The receiver +should log: + +``` +[mail.delivered] alice@chorus@example.com → bob@chorus@example.com: Hello +``` + +If nothing arrives: + +- Check the MAIL server logs for outgoing POST attempts. A `403` + back from your receiver usually means the secret strings don't + match. +- Check that `MAIL_WEBHOOK_SECRET` in the receiver's env matches + the secret you registered exactly (no extra whitespace). +- Verify the receiver is reachable from MAIL's host (e.g., curl + from the MAIL host to your receiver URL). + +## What this tutorial leaves out + +- **Durable dedup.** Replace the in-memory set with a real store + before deploying. +- **Internal routing.** This receiver just prints. In production, + you'd route the event to whatever downstream service needs it + (a chat surface, a database write, a queue, etc.). +- **The "inbox is source of truth" contract.** If your internal + routing fails after signature verification succeeds, return + `200` anyway — the message lives in the MAIL inbox and your + service can recover on its own. See [Webhook Delivery → The + "inbox is source of truth" + contract](../explanations/webhook-delivery.md#the-inbox-is-source-of-truth-contract). +- **Observability.** Log every received event, every failed + signature, every dedup hit. Webhook receivers are silent failure + modes if you don't. + +## See also + +- [Webhook Delivery](../explanations/webhook-delivery.md) — the + contract this tutorial implements. +- [Manage Webhooks](../howtos/manage-webhooks.md) — registering and + rotating webhooks via the admin API. +- [HTTP API](../references/http-api.md) — formal route reference.