feat: Hey-style email screening (unify Blocked/Junk into Screened Email Address)

[s-aga-r]

Overview

Adds Hey-style screening for incoming mail, built on a unified screening doctype.

First, the two near-identical doctypes Blocked Email Address and Junk Email Address are merged into a single Screened Email Address with an action field. That same doctype then powers an opt-in screening flow: unknown senders are quarantined into a Screening folder, and only senders you accept reach the inbox — managed from a dedicated Screener page.

A sender has at most one screening rule (uniqueness on account_id + email), with one action:

Action	Effect on incoming mail
Accepted	Delivered to the inbox
Spam	Filed into the Spam (Junk) folder
Reject	Discarded silently

Everything is enforced server-side via the automation sieve script — the same mechanism the old block/junk lists used.

1. Doctype unification + migration

• New Screened Email Address doctype (replaces Blocked + Junk); uniqueness on (account_id, email), so a shared account's list is consistent for everyone with access.
• Unified sieve generation: one update_sieve_script_for_screened_emails rebuilds the Reject/Spam/Screening blocks and cleans up the legacy Blocked Emails / Junk Senders blocks.
• Patch migrate_to_screened_email_address: Blocked → Reject, Junk → Spam (Reject wins on collision), then drops the old doctypes and tables. Legacy patches guarded so fresh installs don't reference the removed doctypes.
• screen_email_addresses(..., override): explicit user actions overwrite an existing rule; the automated auto-junk flow passes override=False so it never clobbers a manual decision.

2. Screening engine (opt-in, per account)

• Account Settings → Enable Screening (default off, so existing/shared accounts are untouched).
• When on, the automation sieve gains a screening gate that is the last block in the script — a not <trusted> catch-all that runs after Reject, Spam, and your folder automation rules, so anything you've routed or accepted is handled before it:

  # Screening   (last block — the fallback)
  if not address :is "from" [ <accepted senders> + <your own identities> ] {
    fileinto "Screening";
    stop;
  }
  

  Net order: Reject → discard · Spam → Junk · folder rules → their folders · unknown → Screening · accepted/own → inbox. (A sender you route to a folder skips the Screener; only mail nothing else matched, from an untrusted sender, is held.)
• The Screening mailbox (not a standard JMAP role) is auto-created on first enable.

3. Screening folder UX

• "Screen New Senders" toggle in Account Settings.
• Sidebar surfaces the folder as Screener (scan-eye icon), grouped with the default mailboxes.
• In-message Accept Sender / Reject Sender actions inside the Screening folder.

4. Auto-accept on send

• When screening is on, recipients of mail you send are allowlisted (Accepted, non-overriding) so replies to threads you start reach your inbox instead of Screening.

5. Screener page

A dedicated view for the Screening folder, grouped by unique sender (latest message as the summary row):

• Banner with first-time-sender count + Clear All (moves every queued message to the Inbox without blocking or allowing anyone — a safe "empty the queue, I'll triage in my inbox").
• Per-sender Allow (accept → move their mail to Inbox) and Block (mark Spam → move to Junk).
• Click a sender to open their mail in a read-only thread preview (split when the reading pane is on); ↑/↓ (or k/j) traverse, Esc closes.
• JMAP-backed grouping: get_screening_senders, get_screening_sender_mails, allow_screening_senders, screen_out_senders.

6. Mark Junk / Not Junk ↔ screening

• Marking a thread Junk screens its sender as Spam (their future mail → Junk); marking Not Junk screens them in (Accepted). The screening change rides on the same request as the mail move (set_mails_spam_status(..., screen_action)), and the reversal rides on the same request as the undo's mailbox restore (set_mails_mailboxes(..., screen_action)) — so an immediate Cmd+Z flips the rule instead of leaving it stale or deleting it.
• Screened-out (Spam) mail is flagged $junk as well as filed into Junk, so it's genuinely marked junk — matching what marking a mail as junk does — not just placed in the folder.

7. Block remote images (#452)

• Per-account Block Remote Images setting (default on): mail from senders you haven't accepted has its remote images withheld, so they can't phone home as read-tracking pixels.
• Covers both <img src> and CSS url(...) in inline styles and <style> blocks (background images, fonts), for http(s) and protocol-relative //; inline cid: (attachments) and data: images still load.
• Per-message privacy bar: Load / Hide images and Mark sender as trusted (accepts the sender, so their images load now and going forward). Hidden in the Screener preview, where Block/Allow is the trust mechanism.
• Image blocking/detection run on the DOM (DOMParser) rather than regex; this also fixed a latent bug where the email renderer's quote-collapse mis-matched nested <div>s.

8. Inbox banner for unscreened mail

• A slim, dismissible banner above the inbox list nudges you to the Screener while the queue is non-empty.

9. Turning screening off

• The Screener is hidden from the sidebar, and /screener renders nothing and redirects to the inbox.
• Saving Account Settings with screening turned off prompts to move the Screener's remaining mail to the Inbox (move_screening_mails_to_inbox).

Notes

• Defaults: screening is opt-in per account; Block Remote Images is on by default. No data migration needed for the new Accepted action.
• Scoping: the merged doctype scopes all actions to the shared account_id (matching the old Blocked behavior). On a shared account, Spam/Accepted rules are shared across users — same as block rules previously.
• Testing: pre-commit (ruff + prettier/eslint) clean and the frontend builds. The JMAP round-trip (sieve routing, sender grouping, Allow/Block moves, image blocking) should be verified on a live Stalwart account.

🤖 Generated with Claude Code

[greptile-apps]

Confidence Score: 3/5

The migration path and sieve rebuild path both have error-handling gaps that could silently drop folder automation rules or leave the screening gate misconfigured with no diagnostic trace.

The backfill_mailbox_automation_rules helper swallows every JMAP and DB exception with bare except Exception: continue and no logging — if the backfill fails for any user, their existing folder automation rules are silently lost the next time a sieve rebuild runs (since the rebuild reads from Mailbox Settings, which was never populated). This is a data-loss path that fires automatically on migration. The JMAP tokenized-filter bugs in _screening_message_ids already called out in prior review threads also remain: Allow and Block operations can act on messages from unintended senders when From-header tokens overlap.

mail/api/sieve.py (backfill_mailbox_automation_rules — exception handling), mail/api/mail.py (allow_screening_senders, screen_out_senders — JMAP filter correctness and N+1 calls).

Important Files Changed

Filename	Overview
mail/api/sieve.py	Major refactor replacing per-function sieve builders with a unified build_automation_sieve that regenerates the entire script from persistent backups; adds Reject/Spam/Screening block generation and a pause_automation_sieve_build context manager. The backfill_mailbox_automation_rules migration helper silently swallows all exceptions with no logging, risking undetected data loss during migration.
mail/api/mail.py	New Screener endpoints (get_screening_senders, allow_screening_senders, screen_out_senders, move_screening_mails_to_inbox), unified screening logic in _screen_email_addresses, and auto-accept-on-send. JMAP N+1 pattern in allow_screening_senders/screen_out_senders (O(N) queries per sender, already flagged for tokenized-filter issues in prior threads).
mail/client/doctype/screened_email_address/screened_email_address.py	New unified doctype merging Blocked/Junk email address handling; correct permission scoping, uniqueness enforcement at DB level, and sieve rebuild hooks. get_screened_email_addresses lacks a query limit which may be a concern for large screening lists.
mail/patches/migrate_to_screened_email_address.py	Migration patch merging Blocked (→ Reject) and Junk (→ Spam) email addresses into Screened Email Address; uses raw DDL with f-string table name for the DROP TABLE step, which is a style convention concern (values are hardcoded, no injection risk).
mail/patches/backfill_mailbox_automation_rules.py	Thin patch that enqueues backfill_mailbox_automation_rules as a background job; the actual backfill logic in sieve.py silently swallows all JMAP/DB errors with no logging.
mail/client/doctype/account_settings/account_settings.py	Adds on_update hook to regenerate the automation sieve when enable_screening is toggled; correctly guarded for in_migrate and has_value_changed.
mail/client/doctype/mailbox_settings/mailbox_settings.py	Adds on_update hook for automation rule changes with pause_automation_sieve_build flag guard; adds get_mailbox_automation_rules and automation_rules_to_settings helpers. Logic is clean and idempotent.
frontend/src/pages/ScreenerView.vue	New Screener page with sender-grouped list, per-sender Allow/Block, read-only thread preview with race-safe token-based loading, keyboard navigation, polling, and Clear All. Logic is well-structured.
frontend/src/components/EmailContent.vue	Adds remote image blocking banner and DOM-based quote collapsing (replacing buggy regex); image blocking/reveal flow is correctly tied to blockImages prop and trust emit.
frontend/src/utils/useThreadActions.ts	Extends mark-as-junk flow to pass screen_action in the same JMAP call; undo correctly reverses the screening rule; getMailboxName replaces raw _name access for Screener folder label.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Incoming mail] --> B{Reject block\nsieve match?}
    B -- yes --> C[discard / stop]
    B -- no --> D{Mailbox automation\nrule match?}
    D -- yes --> E[fileinto target folder]
    D -- no --> F{Spam block\nsieve match?}
    F -- yes --> G[addflag $junk\nfileinto Junk]
    F -- no --> H{Screening enabled?}
    H -- no --> I[Inbox]
    H -- yes --> J{From in\naccepted + own-identity list?}
    J -- yes --> I
    J -- no --> K[fileinto Screener]
    K --> L[Screener UI:\nAllow / Block]
    L -- Allow --> M[accept sender\nmove to Inbox]
    L -- Block --> N[mark Spam\nmove to Junk]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Incoming mail] --> B{Reject block\nsieve match?}
    B -- yes --> C[discard / stop]
    B -- no --> D{Mailbox automation\nrule match?}
    D -- yes --> E[fileinto target folder]
    D -- no --> F{Spam block\nsieve match?}
    F -- yes --> G[addflag $junk\nfileinto Junk]
    F -- no --> H{Screening enabled?}
    H -- no --> I[Inbox]
    H -- yes --> J{From in\naccepted + own-identity list?}
    J -- yes --> I
    J -- no --> K[fileinto Screener]
    K --> L[Screener UI:\nAllow / Block]
    L -- Allow --> M[accept sender\nmove to Inbox]
    L -- Block --> N[mark Spam\nmove to Junk]

Loading

_{Reviews (10): Last reviewed commit: "refactor(mailbox): build the automation ..." | Re-trigger Greptile}

[s-aga-r]

ToDo:

• Allow adding a Screened Sender with action as "Accepted" from Settings > Screened Senders.
• Ensure the "Screener" folder exists.
• On the disable Screener, move unscreened emails to the Inbox and unsubscribe/delete the "Screener" folder (?)
• Hide the remote images banner after clicking the "Load Images" (?)
• Move "When Marking as Junk" below the "Block Remote Images".
• Add patch "migrate_to_screened_email_address".
• Fix sieve rules order:
  1. Reject
  2. Mailbox
  3. Junk
  4. Screener
• Rebuild automation sieve
• Unify automation sieve build