feat: add durable SQLite server backend

docs/testing-plan.md

@@ -85,11 +85,15 @@ three files moves here, alongside:
 
 - `deployment_dir` — `tmp_path`-backed deployment tree;
   monkeypatches `mail_server.backends.memory.fs.DEPLOYMENT_PATH`
-- `backend` — started `MemoryBackend` seeded with a standard cast:
-  one admin, two users, one agent, one daemon, one swarm
+- `backend` — the started backend behind `app_client`, seeded with a standard
+  cast: one admin, two users, one agent, one daemon, one swarm. Parametrized
+  over both backends (`memory` and `sqlite`) via `backend_kind`; tests never
+  touch backend internals — they seed/assert through the public API or the
+  backend-agnostic `seed_trash` / `seed_list` / `list_members` fixtures
 - `app_client` — `TestClient` over the **real** `mail_server.server.app`
   (env vars `MAIL_HOST`, `MAIL_JWT_SECRET_KEY`, `MAIL_JWT_ALGORITHM` set
-  before import), wired to `backend`
+  before import), wired to `backend`. A module may override `backend_kind` to
+  pin one backend (e.g. `test_stubs.py` → memory, `test_gap_fill.py` → sqlite)
 - `token_for(address)` — factory issuing real JWTs via `POST /auth/token`,
   so integration tests exercise real auth instead of monkeypatching it
 - `webhook_receiver` — in-process ASGI app that records deliveries and can be

src/mail/server/docs/README.md

@@ -9,4 +9,5 @@ This document serves as the root documentation file for the `mail-swarms-server`
 ## Reference Docs
 
 - **`mail-server` CLI reference**: [reference/cli.md](reference/cli.md)
+- **Server backends (`memory` vs `sqlite`)**: [reference/backends.md](reference/backends.md)
 - **MAIL HTTP API reference**: [reference/http.md](reference/http.md)

src/mail/server/docs/reference/backends.md

@@ -0,0 +1,114 @@
+# MAIL Server Backends
+
+`mail-server` stores all of its state — user-agents, swarms, messages, the four
+boxes (inbox/outbox/drafts/trash), the delivery buffer, webhooks, and lists —
+through a pluggable backend. Two backends ship today, selected with
+`--backend` (see [cli.md](cli.md)):
+
+| | `memory` (default) | `sqlite` |
+|---|---|---|
+| Store | process-local dicts | SQLite file (SQLAlchemy async + `aiosqlite`) |
+| Durability | periodic checkpoint + shutdown flush | per-commit (transactional) |
+| Survives `kill -9` | only up to the last checkpoint | yes — committed writes are durable |
+| Pagination / sorting | in Python over the whole box | pushed into SQL (`ORDER BY ... LIMIT`) |
+| Method coverage | core; some endpoints are stubs | full parity (implements the stubs too) |
+| Scaling | single process | single node |
+
+Both implement the same `MAILServerBackend` protocol, so the HTTP API is
+identical regardless of which one is selected.
+
+## `memory` backend
+
+The default. Holds all state in process-local dictionaries and persists it to a
+directory tree under `~/.mail-swarms/deployments/<deployment>/` on shutdown and
+on a periodic checkpoint (`--memory-save-interval`, default 60s). It is the
+reference proof-of-concept: simple and dependency-light, but durability is
+bounded by the checkpoint interval — an abrupt `kill -9` loses everything
+written since the last checkpoint.
+
+A handful of endpoints (`DELETE /inbox/{id}`, `DELETE /drafts/{id}`,
+`DELETE /trash/{id}`, `POST /trash/clear`, `POST /daemon/deliver/remote`,
+`PATCH /admin/webhooks/{id}`) raise `NotImplementedError` on this backend.
+
+## `sqlite` backend
+
+A durable, transactional backend over a single SQLite file. Every write commits
+in its own short transaction, so a committed message survives an abrupt
+`kill -9` — the window the memory backend's checkpoint cannot close.
+Pagination, sorting, and filtering are pushed into SQL rather than loaded into
+Python. It implements **every** protocol method, including the ones the memory
+backend leaves as stubs, making it the more complete backend.
+
+### Database location
+
+Resolution precedence (highest first):
+
+1. `--database-url` / `MAIL_DATABASE_URL` — a full URL, e.g.
+   `sqlite:////absolute/path/mail.db`.
+2. `--sqlite-path` / `MAIL_SQLITE_PATH` — a file path.
+3. Default: `~/.mail-swarms/deployments/default/mail.db`.
+
+A `sqlite://` URL is normalized to the async `sqlite+aiosqlite://` driver
+automatically, and the parent directory is created if missing.
+
+### Connection settings
+
+Each connection is opened with:
+
+- `journal_mode=WAL` — readers never block the writer.
+- `foreign_keys=ON` — referential integrity is enforced (cascade deletes work).
+- `busy_timeout=5000` — brief write contention retries for up to 5s instead of
+  immediately raising `database is locked`.
+
+### Initialization
+
+Provision a SQLite deployment with `backend-init --type sqlite` (same argument
+surface as the memory initializer — deployment, swarm, agents, daemons, users,
+admins, host):
+
+```bash
+backend-init --type sqlite --swarm chorus --host localhost \
+  --agents supervisor --users alice --admins root --daemons dummy
+```
+
+This creates the database file and schema and seeds the swarm and user-agents,
+writing each generated password to `~/.mail-swarms/deployments/<deployment>/.secrets/<address>`.
+Re-running is safe: existing swarms and user-agents are left untouched. No box
+files are created — per-owner box membership is created lazily on first
+delivery.
+
+Then run the server against the same database:
+
+```bash
+mail-server --backend sqlite
+```
+
+(The startup lifespan creates the schema if it does not already exist, so
+running `mail-server --backend sqlite` against a fresh path also works; use
+`backend-init` when you want a seeded cast.)
+
+### Migrating an existing `memory` deployment
+
+If you already have a filesystem (`memory`) deployment, you can import it into a
+new SQLite database of the same name instead of seeding a fresh cast:
+
+```bash
+backend-init --type sqlite --import-fs
+```
+
+This reads the existing `~/.mail-swarms/deployments/<deployment>/` tree (user-agents,
+swarms, messages, all four boxes with their ordering, the delivery buffer,
+webhooks, and lists) and writes it into `<deployment>/mail.db`. Existing
+`.secrets/` files are untouched, so credentials carry over. The import runs in a
+single transaction and **refuses to run against a non-empty database**, so it
+can't clobber an existing SQLite deployment.
+
+### Single-node caveat
+
+SQLite serializes writers even in WAL mode, and `aiosqlite` runs each connection
+on a thread, so concurrent requests can still contend on the write lock (the
+`busy_timeout` retry absorbs brief contention). This makes the `sqlite` backend
+a good fit for **single-node durable deployments**. Horizontal, multi-process
+scaling wants a client/server database such as PostgreSQL; the URL-normalization
+seam leaves the door open for a future Postgres backend, but that is not part of
+this release.

src/mail/server/docs/reference/cli.md

@@ -18,10 +18,21 @@ mail-server [option]...
   - **Example**: `mail-server --port 8000`
 - `-b`/`--backend`: The MAIL server backend to use.
   - **Default**: `memory`
-  - **Choices**: `memory`
-  - **Example**: `mail-server --backend memory`
+  - **Choices**: `memory`, `sqlite`
+  - **Example**: `mail-server --backend sqlite`
+  - See [backends.md](backends.md) for how the two backends differ.
 - `--memory-save-interval`: Seconds between memory backend filesystem checkpoints.
   - **Default**: `60`
   - **Environment**: `MAIL_MEMORY_SAVE_INTERVAL_SECONDS`
   - **Disable**: Set to `0` to rely only on startup/shutdown persistence.
   - **Example**: `mail-server --memory-save-interval 30`
+  - Ignored unless `--backend memory`.
+- `--sqlite-path`: Path to the SQLite database file (`sqlite` backend only).
+  - **Default**: `~/.mail-swarms/deployments/default/mail.db`
+  - **Environment**: `MAIL_SQLITE_PATH`
+  - **Example**: `mail-server --backend sqlite --sqlite-path /var/lib/mail/mail.db`
+- `--database-url`: Full database URL (`sqlite` backend only); takes precedence
+  over `--sqlite-path`.
+  - **Default**: unset (falls back to `--sqlite-path`, then the default path)
+  - **Environment**: `MAIL_DATABASE_URL`
+  - **Example**: `mail-server --backend sqlite --database-url sqlite:////abs/path/mail.db`

src/mail/server/docs/tutorials/quickstart.md

@@ -49,6 +49,30 @@ All four user-agents listed above have associated plain-text password stored in
 > Copy the generated passwords and keep them in a safe place.
 > Afterwards, remove the files generated from the backend filesystem.
 
+## `sqlite` Backend Setup
+
+The `sqlite` backend is a durable, transactional alternative to `memory`: a
+committed message survives an abrupt `kill -9`, not just a clean shutdown. To
+initialize one, pass `--type sqlite`:
+
+```bash
+uv run backend-init --type sqlite
+```
+
+This creates a SQLite database at
+`~/.mail-swarms/deployments/default/mail.db` and seeds the same cast as the
+`memory` initializer, writing each generated password to the printed
+`.secrets/` paths. Then run the server against it:
+
+```bash
+uv run mail-server --backend sqlite
+```
+
+The database path can be overridden with `--sqlite-path` / `MAIL_SQLITE_PATH`
+or `--database-url` / `MAIL_DATABASE_URL`. See
+[reference/backends.md](../reference/backends.md) for the full comparison,
+connection settings, and the single-node caveat.
+
 ## Running the Server
 
 With your environment variables configured, try running `mail-server`:

src/mail/server/pyproject.toml

@@ -9,13 +9,15 @@ authors = [
 requires-python = ">=3.12"
 dependencies = [
     "aiohttp>=3.12.15",
+    "aiosqlite>=0.20",
     "fastapi>=0.116.1",
     "mail-swarms-protocol==2.0.1",
     "pwdlib[argon2]>=0.3.0",
     "pydantic>=2.11.7",
     "pyjwt>=2.10.1",
     "python-dotenv>=1.1.1",
     "python-multipart>=0.0.20",
+    "sqlalchemy[asyncio]>=2.0",
     "uvicorn>=0.35.0",
 ]
 

src/mail/server/src/mail_server/backend_init.py

@@ -2,6 +2,7 @@
 # Copyright (c) 2026 Addison Kline
 
 import argparse
+import asyncio
 
 from mail_protocol.cli_help import add_license_argument
 from mail_protocol.core.validators import (
@@ -29,7 +30,7 @@ def main() -> None:
         "-t",
         "--type",
         default="memory",
-        choices=["memory"],
+        choices=["memory", "sqlite"],
         help="the type of backend to initialize (default: %(default)s)",
     )
     parser.add_argument(
@@ -87,10 +88,20 @@ def main() -> None:
         default="example.com",
         help="the host domain or IP address to use (default: %(default)s)",
     )
+    parser.add_argument(
+        "--import-fs",
+        action="store_true",
+        help=(
+            "for --type sqlite: import the existing filesystem (memory) "
+            "deployment of the same name into the new SQLite database instead "
+            "of seeding a fresh cast"
+        ),
+    )
 
     # parse and handle args
     args = parser.parse_args()
     be_type = args.type
+    import_fs = args.import_fs
     deployment = args.deployment
     swarm = args.swarm
     swarm_description = args.swarm_description
@@ -142,6 +153,9 @@ def main() -> None:
     except ValueError as e:
         print(f"invalid host {host}: {e}")
         exit(1)
+    if import_fs and be_type != "sqlite":
+        print("--import-fs is only valid with --type sqlite")
+        exit(1)
 
     # initialize backend
     match be_type:
@@ -157,5 +171,33 @@ def main() -> None:
                 admins=admins,
                 host=host,
             )
+        case "sqlite":
+            # Imported lazily so SQLAlchemy stays off the import path for
+            # memory-only initialization.
+            if import_fs:
+                from mail_server.backends.sqlite.migrate import (
+                    import_memory_deployment,
+                )
+
+                counts = asyncio.run(
+                    import_memory_deployment(deployment=deployment)
+                )
+                print(f"imported filesystem deployment {deployment}: {counts}")
+            else:
+                from mail_server.backends.sqlite.init import init_sqlite_backend
+
+                asyncio.run(
+                    init_sqlite_backend(
+                        deployment=deployment,
+                        swarm=swarm,
+                        swarm_description=swarm_description,
+                        swarm_keywords=swarm_keywords,
+                        agents=agents,
+                        daemons=daemons,
+                        users=users,
+                        admins=admins,
+                        host=host,
+                    )
+                )
         case _:
             raise ValueError(f"invalid backend type: {be_type}")

src/mail/server/src/mail_server/backends/sqlite/__init__.py

@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: Apache-2.0
+# Copyright (c) 2026 Addison Kline