From 737826b4d8c8d8b52ccca1e422f5aae4716bd587 Mon Sep 17 00:00:00 2001
From: Jared Lunde <jared.lunde@gmail.com>
Date: Thu, 4 Jun 2026 12:25:13 -0700
Subject: [PATCH 1/6] feat pgbouncer: reactive in-VM pooler scaler + telemetry
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The single-threaded PgBouncer pooler saturates one core terminating TLS
under connection churn (~2.4k conns/s/core, measured). It is the only tier
that can't self-scale: Postgres is process-per-connection and already
memory-reactive. Make the instance scale its own pooler from local signals,
tracking usage (1 pooler when idle, up to the cap only when genuinely
CPU-bound) so a scaled-to-zero box never carries peak-provisioned workers.

Empirically grounded (bench/glidefs-pg/RESULTS.md §F/F′/F″, 12-vCPU rig):
  - so_reuseport multi-worker scales: pooler CPU is linear (0.79→1.59 cores
    for 2 workers); throughput 1.5–1.8x/worker. Proven, not assumed.
  - TLS session resumption is dead as a pooler lever: TLS 1.3 keeps ECDHE on
    resume (1.0x) and standard pg clients never cache sessions. Cut.
  - Scale-up is zero-disruption; scale-down via SIGINT drains gracefully
    (measured 0.033% reconnect blip). So the scaler is safe to shed workers.

Implementation:
  - supervisor.rs: PgbScaler as an inline select! arm (5s tick) — samples
    per-worker /proc CPU, decides via a pure, unit-tested decide_scale()
    (asymmetric hysteresis: up fast >0.75 core ×2 ticks, down slow <0.5 ×6
    ticks, cooldowns), and grows/SIGINT-reaps the so_reuseport worker set
    within [1, pgbouncer_max_workers]. Reuses the existing pgb_extra
    plumbing (handoff adoption, graceful drain, crash-reap).
  - children.rs: read_proc_cpu_ticks() — utime+stime from /proc/<pid>/stat,
    same robust last-paren parse as read_starttime. Validated against a real
    busy process (0.993 cores for one pinned core).
  - rpc.rs + handoff_bridge.rs: `pooler` RPC command exposing PoolerStats
    (live/max workers, per-worker cores, at_ceiling) — the production signal
    for whether a real box ever saturates a pooler. Type lives in
    handoff_bridge (library) since SharedState carries the handle.
  - config.rs: pgbouncer_ini uses the FULL pool per worker (was pool/cap,
    which under-provisioned the common 1-worker box: 64-vCPU got 24 not 192).
    default_pool_size is a ceiling; max_connections is the real backstop.
  - pgbouncer.ini: empty unix_socket_dir so multiple so_reuseport workers
    don't collide on /tmp/.s.PGSQL.5432.

Substrate-tuning wins (same measure-first campaign, already validated):
  - 00-beyond.conf: checkpoint_timeout 15→30min (frequent checkpoints wrote
    2.9x more S3 at equal work); keep FPW on (CoW cache overwrites in place,
    the ZFS trick doesn't transfer) and lz4 (zstd = +12.5% S3).
  - boot.rs: thread ram_bytes/vcpus into pgbouncer_ini for box-scaled sizing.

Also adds the reproducible benchmark rig (bench/glidefs-pg/) and a
bench:substrate mise task.

Tests: 6 decide_scale unit tests + updated pgbouncer_scales_with_box; full
lib+bin suite green; clippy -D warnings clean. The Docker supervisor
lifecycle suite can't run on this host (constrained container: read-only
/proc/sys, no privileged syscalls) — confirmed by identical failure on clean
main — so it must be exercised in CI before merge.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 bench/glidefs-pg/README.md                    |  79 ++++
 bench/glidefs-pg/RESULTS.md                   | 204 ++++++++
 bench/glidefs-pg/conf/a1-no-recycle.conf      |   3 +
 bench/glidefs-pg/conf/baseline.conf           |  43 ++
 bench/glidefs-pg/conf/c1-checkpoint30.conf    |   2 +
 bench/glidefs-pg/conf/c1-checkpoint60.conf    |   2 +
 bench/glidefs-pg/conf/c1-cp300s.conf          |   5 +
 bench/glidefs-pg/conf/c1-cp30s.conf           |   5 +
 bench/glidefs-pg/conf/c1-cp45s.conf           |   5 +
 bench/glidefs-pg/conf/c1-cp90s.conf           |   5 +
 bench/glidefs-pg/conf/c2-flushafter-0.conf    |   5 +
 bench/glidefs-pg/conf/c2-flushafter-2mb.conf  |   4 +
 bench/glidefs-pg/conf/c3-aggressive.conf      |   3 +
 bench/glidefs-pg/conf/c3-gentle.conf          |   3 +
 bench/glidefs-pg/conf/c3-off.conf             |   2 +
 bench/glidefs-pg/conf/d-maintenance-io.conf   |   2 +
 bench/glidefs-pg/conf/d1-zstd.conf            |   2 +
 .../glidefs-pg/conf/e1-autovac-throttled.conf |   6 +
 bench/glidefs-pg/delta.awk                    |  48 ++
 bench/glidefs-pg/exp-d2-coldfork.sh           | 102 ++++
 bench/glidefs-pg/exp-d2.sh                    |  91 ++++
 bench/glidefs-pg/exp-e-fork.sh                | 104 +++++
 bench/glidefs-pg/exp-e.sh                     |  92 ++++
 bench/glidefs-pg/exp-f-pgbouncer.sh           |  58 +++
 bench/glidefs-pg/exp-multiworker.sh           |  67 +++
 bench/glidefs-pg/exp-pooler-ceiling.sh        |  88 ++++
 .../out/20260603-141450-smoke/glidefs.toml    |  20 +
 .../out/20260603-141450-smoke/harness.conf    |  40 ++
 .../out/20260603-141450-smoke/postgres.log    |   3 +
 .../out/20260603-141623-smoke/glidefs.toml    |  20 +
 bench/glidefs-pg/qemu-bringup.sh              |  71 +++
 bench/glidefs-pg/qemu-runall.sh               |  50 ++
 bench/glidefs-pg/run.sh                       | 286 ++++++++++++
 mise.toml                                     |  18 +
 packer/files/pgbouncer/pgbouncer.ini          |  12 +-
 packer/files/postgresql/00-beyond.conf        |  12 +-
 src/boot.rs                                   |   5 +-
 src/children.rs                               |  26 ++
 src/config.rs                                 | 130 +++++-
 src/handoff_bridge.rs                         |  40 +-
 src/rpc.rs                                    |  43 +-
 src/supervisor.rs                             | 442 +++++++++++++++++-
 42 files changed, 2223 insertions(+), 25 deletions(-)
 create mode 100644 bench/glidefs-pg/README.md
 create mode 100644 bench/glidefs-pg/RESULTS.md
 create mode 100644 bench/glidefs-pg/conf/a1-no-recycle.conf
 create mode 100644 bench/glidefs-pg/conf/baseline.conf
 create mode 100644 bench/glidefs-pg/conf/c1-checkpoint30.conf
 create mode 100644 bench/glidefs-pg/conf/c1-checkpoint60.conf
 create mode 100644 bench/glidefs-pg/conf/c1-cp300s.conf
 create mode 100644 bench/glidefs-pg/conf/c1-cp30s.conf
 create mode 100644 bench/glidefs-pg/conf/c1-cp45s.conf
 create mode 100644 bench/glidefs-pg/conf/c1-cp90s.conf
 create mode 100644 bench/glidefs-pg/conf/c2-flushafter-0.conf
 create mode 100644 bench/glidefs-pg/conf/c2-flushafter-2mb.conf
 create mode 100644 bench/glidefs-pg/conf/c3-aggressive.conf
 create mode 100644 bench/glidefs-pg/conf/c3-gentle.conf
 create mode 100644 bench/glidefs-pg/conf/c3-off.conf
 create mode 100644 bench/glidefs-pg/conf/d-maintenance-io.conf
 create mode 100644 bench/glidefs-pg/conf/d1-zstd.conf
 create mode 100644 bench/glidefs-pg/conf/e1-autovac-throttled.conf
 create mode 100755 bench/glidefs-pg/delta.awk
 create mode 100644 bench/glidefs-pg/exp-d2-coldfork.sh
 create mode 100644 bench/glidefs-pg/exp-d2.sh
 create mode 100644 bench/glidefs-pg/exp-e-fork.sh
 create mode 100644 bench/glidefs-pg/exp-e.sh
 create mode 100644 bench/glidefs-pg/exp-f-pgbouncer.sh
 create mode 100644 bench/glidefs-pg/exp-multiworker.sh
 create mode 100644 bench/glidefs-pg/exp-pooler-ceiling.sh
 create mode 100644 bench/glidefs-pg/out/20260603-141450-smoke/glidefs.toml
 create mode 100644 bench/glidefs-pg/out/20260603-141450-smoke/harness.conf
 create mode 100644 bench/glidefs-pg/out/20260603-141450-smoke/postgres.log
 create mode 100644 bench/glidefs-pg/out/20260603-141623-smoke/glidefs.toml
 create mode 100755 bench/glidefs-pg/qemu-bringup.sh
 create mode 100755 bench/glidefs-pg/qemu-runall.sh
 create mode 100755 bench/glidefs-pg/run.sh

diff --git a/bench/glidefs-pg/README.md b/bench/glidefs-pg/README.md
new file mode 100644
index 0000000..b053c39
--- /dev/null
+++ b/bench/glidefs-pg/README.md
@@ -0,0 +1,79 @@
+# GlideFS × Postgres substrate-tuning harness
+
+Measure-first rig for `plans/wal-recycle-off-*.md`. It puts a real Postgres data
+dir on a real GlideFS block device backed by an in-memory / local-file / MinIO
+object store (never real S3), runs a fixed pgbench workload, and scrapes
+GlideFS's own per-export metrics before/after so each tuning knob is scored on
+**S3 write cost and coalescing**, not intuition.
+
+## Why this exists
+
+The S3 cost of running Postgres on GlideFS is _distinct 128 KiB blocks flushed
+per cycle_. Overwrite-before-flush coalesces for free. Every knob in the plan
+(checkpoints, `*_flush_after`, bgwriter, `wal_compression`, `wal_recycle`,
+autovacuum-on-CoW, `compaction_cooldown`) changes how many distinct blocks reach
+the object store. This rig measures that directly via
+`glidefs_s3_batches_written_total`, `glidefs_coalesce_ratio`,
+`glidefs_write_amplification`.
+
+## Prereqs (already present on the homelab box)
+
+- `glidefs` binary, `nbd` + `ublk_drv` kernel modules loaded
+- passwordless `sudo` (glidefs needs CAP_SYS_ADMIN for `/dev/nbdN`; mkfs/mount)
+- Postgres 18 client tools (`initdb`, `pg_ctl`, `pgbench`, `psql`)
+
+## Run
+
+```sh
+# one run (baseline)
+mise run bench:substrate
+
+# a single candidate knob
+bench/glidefs-pg/run.sh --conf bench/glidefs-pg/conf/c1-checkpoint60.conf --label c1-60
+
+# full A/B sweep (baseline 3x for the noise floor, then each overlay)
+bench/glidefs-pg/sweep.sh
+```
+
+Knobs: `--backend file|memory|minio` (default `file` — same byte accounting as
+`memory`, no RAM blowup on long runs), `--scale`, `--duration`, `--clients`,
+`--cooldown N` (GlideFS compaction_cooldown, the G experiment), `--transport
+nbd|ublk`, `--keep`.
+
+## Output
+
+`out/<ts>-<label>/`:
+
+- `delta.txt` — the scoreboard (before / after / Δ per metric, plus pgbench TPS)
+- `before.prom` / `after.prom` — raw Prometheus snapshots (export-scoped)
+- `*.json` — per-export metric snapshots
+- `glidefs.toml`, `harness.conf` — exact inputs for that run
+- `postgres.log`, `pgbench.log`
+
+## Reading the scoreboard
+
+| Want                     | Watch                                                 | Direction      |
+| ------------------------ | ----------------------------------------------------- | -------------- |
+| less S3 write cost       | `s3_batches_written_total` / `s3_bytes_written_total` | ↓              |
+| more coalescing          | `coalesce_ratio`                                      | ↑              |
+| less write-amp           | `write_amplification`                                 | ↓              |
+| cold-read pressure       | `cache_misses_total`, `s3_bytes_read_total`           | ↓ (read tests) |
+| no throughput regression | pgbench `tps`                                         | flat/↑         |
+
+**Ship a knob only if its delta clears the 3-run baseline noise floor.**
+
+## Experiment → conf map
+
+| Exp | Conf overlay                | Hypothesis                                                              |
+| --- | --------------------------- | ----------------------------------------------------------------------- |
+| C1  | `c1-checkpoint30/60.conf`   | fewer/larger checkpoints → fewer FPW bursts → fewer S3 bytes            |
+| C2  | `c2-flushafter-2mb/0.conf`  | relax forced writeback → more coalescing                                |
+| C3  | `c3-bgwriter-gentle.conf`   | gentler bgwriter → re-dirty in place → more coalescing                  |
+| D1  | `d1-zstd.conf`              | zstd WAL → fewer S3/sink bytes                                          |
+| D2  | `d-maintenance-io.conf`     | parallel cold-from-S3 maintenance scans (use with a fork/VACUUM mode)   |
+| A1  | `a1-no-recycle.conf`        | CoW WAL hygiene — expected ≈ noise (see plan §A)                        |
+| E1  | `e1-autovac-throttled.conf` | fork autovacuum profile → less CoW divergence (use with fork-idle mode) |
+| G   | `--cooldown 8` vs `0`       | compaction_cooldown → −22% S3 PUT (justifies the cross-repo change)     |
+
+D2, E1, and G need the fork/idle run modes in `sweep.sh`; the rest run directly
+with `--conf`.
diff --git a/bench/glidefs-pg/RESULTS.md b/bench/glidefs-pg/RESULTS.md
new file mode 100644
index 0000000..4a80527
--- /dev/null
+++ b/bench/glidefs-pg/RESULTS.md
@@ -0,0 +1,204 @@
+# Substrate-tuning results (measured)
+
+Environment: **real Postgres 18.4 (PGDG, lz4+zstd) on a real GlideFS NBD device, inside an
+isolated QEMU/KVM VM** (Ubuntu 24.04, 6 vCPU / 8 GB). Object store: `memory://`. Zero contact with
+the production homelab glidefs. Harness: `bench/glidefs-pg/run.sh`; signal = GlideFS flush log
+(`bytes_uploaded`/`packs_uploaded` per flush, summed over the run window after a forced `drain`).
+
+## Method note (important)
+
+- **Work-normalized runs only.** Time-based (`-T`) runs confound throughput with efficiency (a faster
+  config does more transactions → more bytes). All results below use **fixed transactions**
+  (`pgbench -t`, 160k txns, scale 50) so S3-byte deltas reflect _efficiency at equal work_.
+- **Trust absolute work-normalized S3 bytes, not write-amp.** write-amp = S3 bytes / guest bytes; a
+  knob that changes guest bytes written (e.g. `wal_init_zero`) moves the denominator and makes the
+  ratio misleading. The honest metric is S3 write bytes for the same work.
+
+## Round 1 — knobs measurable in a 60–120 s window (160k txns, scale 50)
+
+| config                                     |                                S3 write MB |     Δ vs base | packs |         tps | call                            |
+| ------------------------------------------ | -----------------------------------------: | ------------: | ----: | ----------: | ------------------------------- |
+| baseline ×3                                | 165.8 / 167.7 / 167.6 (μ **167.0**, ±0.7%) |             — | 62–67 |   2523–2645 | reference                       |
+| **D1 `wal_compression=zstd`**              |                                  **187.9** |    **+12.5%** |    57 |        2509 | **NO-SHIP**                     |
+| **A1 `wal_recycle=off,wal_init_zero=off`** |                                  **181.9** |     **+8.9%** |    89 | 2207 (−15%) | **NO-SHIP**                     |
+| C2 `*_flush_after=0`                       |                                      165.9 | −0.7% (noise) |    72 |        2650 | neutral (no S3 win)             |
+| G `compaction_cooldown=8`                  |                                      167.9 | +0.6% (noise) |    63 |        2567 | inconclusive — needs churn test |
+
+Noise floor: S3 bytes **±0.7%**, write-amp **±0.001**. So +12.5% and +8.9% are real (≫ noise).
+
+### Findings
+
+- **zstd is worse, not better (−ship).** GlideFS already LZ4-compresses every block at flush.
+  Pre-compressing WAL with zstd produces high-entropy blocks GlideFS can't compress further, so it
+  stores _more_ bytes (+12.5% at equal work; write-amp 0.142 vs 0.122). The substrate already owns
+  compression — don't double it. Keep `wal_compression = lz4` (cheap, and its output still has
+  GlideFS headroom) — or even test `off` later.
+- **`wal_recycle/wal_init_zero=off` is mildly harmful (−ship).** +8.9% S3 bytes **and** −15% tps.
+  The ZFS playbook does not transfer: recycling reuses segment offsets (blocks GlideFS can coalesce);
+  `recycle=off` allocates fresh segment files → more distinct blocks + ext4 inode/bitmap churn. This
+  **confirms and strengthens plan §A** (was "≈ noise"; measurement says "slightly worse"). Leave PG
+  defaults (both `on`).
+- **`*_flush_after=0`: no S3 benefit** at this workload (the flush cadence here is driven by the
+  500-dirty-block threshold, not OS writeback timing). Park it; revisit only as a latency knob.
+- **`compaction_cooldown=8`: inconclusive** in a short non-churning run — compaction barely fires.
+  Needs the dedicated overwrite-churn test to confirm/deny the −22% PUT claim from the volume study.
+
+## Still to measure (need tailored designs, not a 60 s OLTP run)
+
+| knob                             | why round-1 can't see it                  | design                                                                |
+| -------------------------------- | ----------------------------------------- | --------------------------------------------------------------------- |
+| C1 `checkpoint_timeout` 15/30/60 | no time-based checkpoint fires in <15 min | forced checkpoints over fixed-txn; vary cadence; measure FPW→S3 bytes |
+| C3 bgwriter aggressiveness       | masked by cache at this scale             | larger-than-cache working set                                         |
+| D2 `maintenance_io_concurrency`  | OLTP doesn't exercise cold reads          | VACUUM/index-build on a cold fork; measure wall-time + cache misses   |
+| E autovacuum-on-CoW              | needs an idle fork                        | populate → fork → idle; measure fork's divergence bytes               |
+| G `compaction_cooldown`          | needs compaction to fire                  | sustained overwrite churn over time                                   |
+| F PgBouncer multi-worker         | not a GlideFS metric                      | `pgbench -c200` connection-scaling, 1 vs N workers                    |
+
+## Round 2 — checkpoint cadence (C1) — the headline win
+
+Fixed 300k txns, scale 50, only `checkpoint_timeout` varied (wal-triggered checkpoints
+disabled via `max_wal_size=32GB`). Compressed cadence tests the same FPW-frequency mechanism
+as the production 15→30→60 min change.
+
+| checkpoint_timeout | S3 write MB | guest WAL MB | packs | write-amp |  tps |   vs rare |
+| ------------------ | ----------: | -----------: | ----: | --------: | ---: | --------: |
+| 45 s (frequent)    |     **635** |         2929 |   310 |     0.217 | 1605 |     +194% |
+| 90 s               |         294 |         2138 |   123 |     0.137 | 2105 |      +36% |
+| 300 s (rare)       |     **216** |         1839 |    65 |     0.117 | 2062 | reference |
+
+**Frequent checkpoints write 2.9× more to S3 at identical work (635 vs 216 MB) and run
+slower (1605 vs 2062 tps).** Each checkpoint forces a full-page image on the first touch of
+every page → more checkpoints → more WAL → more S3 bytes (and more WAL-sink bandwidth).
+**SHIP: longer `checkpoint_timeout`.** Applied 15 min → 30 min in `00-beyond.conf` (halves the
+checkpoint rate; recovery stays bounded by `max_wal_size=8GB`). 60 min is a further option;
+returns diminish past ~2× per the 90→300 s segment.
+
+## Round 3 — the remaining knobs (re-run with guard-rails)
+
+### C3 bgwriter — DISPROVEN
+
+Corrected design (hot set FITS in shared_buffers: scale 20, sb 1 GB, 200k txns), vary bgwriter:
+
+| bgwriter                  | S3 write MB |  tps |
+| ------------------------- | ----------: | ---: |
+| aggressive (50 ms / 1000) |       113.6 | 2162 |
+| gentle (500 ms / 100)     |       113.5 | 1956 |
+| off (maxpages 0)          |       113.5 | 1811 |
+
+S3 bytes **identical** (0.1%). When the hot set fits in shared_buffers, coalescing happens
+regardless of bgwriter timing — it's not an S3 lever. **No-ship; leave bgwriter as-is.** (My first
+C3 design — working set ≫ shared_buffers on `memory://` — was both wrong for the hypothesis and
+pathological: eviction churn floods the in-RAM store until the guest thrashes. Don't do that.)
+
+### G `compaction_cooldown` — DISPROVEN (under OLTP)
+
+Overwrite-churn (scale 10, 400k txns), cooldown 0 vs 8:
+
+| cooldown | S3 write MB | packs |
+| -------- | ----------: | ----: |
+| 0        |       140.5 |    42 |
+| 8        |       140.9 |    41 |
+
+Identical. The volume-study's −22% **does not reproduce** under live pgbench — compaction barely
+fires at this scale, so cooldown has nothing to defer. No measurable S3 benefit for this workload;
+the −22% appears specific to the trace-replay methodology, not OLTP.
+
+### E autovacuum-on-CoW divergence — VALIDATED (big)
+
+Measured directly (no fork API): build 85k dead tuples with autovacuum OFF, drain, then turn
+autovacuum ON under a profile and idle 90 s; S3 bytes flushed = the divergence an idle fork incurs.
+
+| autovacuum profile                                              |  S3 written in 90 s idle | dead tuples after  |
+| --------------------------------------------------------------- | -----------------------: | ------------------ |
+| **aggressive** (scale_factor 0.01)                              | **305.7 MB** (236 packs) | 0 (fully vacuumed) |
+| **throttled** (scale_factor 0.4, naptime 5min, cost_delay 20ms) |     **0.1 MB** (6 packs) | 85k (deferred)     |
+
+**~2240×.** An idle fork under aggressive autovacuum diverges **306 MB in 90 s** — the "idle fork
+diverges" cost made concrete. **SHIP: throttle autovacuum on ephemeral/fork volumes** (they're
+short-lived; the bloat never matters, but every vacuumed page is a new uploaded block). On a
+long-lived durable primary, keep autovacuum aggressive (bloat control wins there).
+
+### F PgBouncer multi-worker (so_reuseport) — CONFIRMED: scales (12-vCPU VM)
+
+The 6-vCPU INCONCLUSIVE result was a **self-imposed wall** — at 6 vCPU the VM/Postgres saturates
+before a single pooler does, so workers can't show. Rebuilt the VM at **12 vCPU** and drove a
+**pooler-bound** load (Ed25519 TLS termination + connection churn → ~314 µs/handshake, PG idle), with
+N poolers pinned to N cores and N async TLS-churn drivers on the next N cores, PG on cores 8–11.
+`exp-multiworker.sh` (driver-limited sweep) and the saturated follow-up (3 drivers/pooler):
+
+| N poolers | total tps | pooler CPU (aggregate) | note                              |
+| --------: | --------: | ---------------------: | --------------------------------- |
+|         1 |       718 |             0.35 cores | 1 driver/pooler (driver-limited)  |
+|         2 |      1272 |             0.67 cores | "                                 |
+|         3 |      1755 |             1.01 cores | "                                 |
+|         1 |      1877 |             0.79 cores | **3 drivers/pooler (saturating)** |
+|         2 |      2853 |             1.59 cores | **3 drivers/pooler**              |
+
+**Pooler CPU scales perfectly linearly** (0.35→0.67→1.01 ≈ 0.34/worker; 0.79→1.59 = exactly 2.0×) —
+`so_reuseport` genuinely distributes new connections across workers, each doing independent work on
+its own core. Throughput scales 1.5–1.8×/worker; sub-2× only because at full 12-core utilization the
+_drivers_ and PG contend, not the poolers. **A single pooler caps at ~2.4k TLS-churn conns/s/core
+(≈1877/0.79); each added worker linearly adds that much.** Multi-worker is a **real, working lever**
+for handshake/churn-bound pooler load — and it's exactly the regime Beyond's serverless/edge profile
+lives in (every cold invocation = one fresh TLS handshake). **SHIP the mechanism; gate the count on a
+real signal (pooler core-bound).** Scripts: `exp-multiworker.sh`, `exp-pooler-ceiling.sh`.
+
+### F′ TLS session resumption — MEASURED DEAD as a pooler lever
+
+The tempting alternative to workers: make each reconnect skip the handshake. Measured the raw ceiling
+with `openssl s_time -new` vs `-reuse` against our exact certs, then tested reachability:
+
+| cert    | TLS |   full |   resumed |  speedup | actually resumed? |
+| ------- | --- | -----: | --------: | -------: | ----------------- |
+| Ed25519 | 1.2 | 314 µs | **40 µs** | **7.9×** | Y (session-ID)    |
+| Ed25519 | 1.3 | 336 µs |    343 µs |     1.0× | **N**             |
+| RSA2048 | 1.2 | 283 µs |     39 µs |     7.2× | Y                 |
+| RSA2048 | 1.3 | 382 µs |    411 µs |     0.9× | **N**             |
+
+The 8× ceiling is real **but unreachable in our world**, for two independently-fatal reasons:
+
+1. **TLS 1.3 (the secure default) keeps ECDHE on resume** (`psk_dhe_ke` for forward secrecy) → ~1×,
+   no win. Only TLS 1.2 session-ID resumption (no PFS) or 1.3 `psk_ke` (no PFS) reaches the 8× — a
+   security regression we won't ship.
+2. **Standard Postgres clients don't cache TLS sessions across connections.** Measured directly: a
+   _cooperating_ Python client that shares its `SSLContext` + re-presents the session resumes
+   (`[F,T,T,T,T,T]`); the _default_ pg-client pattern — fresh context per connect, exactly what
+   libpq/asyncpg do — **never** resumes (`[F,F,F,F,F,F]`). Beyond's churn = a new client per
+   invocation = a fresh context every time = always a full handshake.
+
+**Verdict: resumption is not a pooler-side lever we can ship.** The only way to capture its win is to
+_avoid the handshake entirely_ via **connection reuse** (persistent client→pooler connections / an
+edge proxy that terminates TLS once and keeps warm backends) — which is a client/platform-architecture
+decision, not pgbouncer config. So for the pooler tier, **multi-worker is the lever; resumption is
+not.**
+
+### F″ Scale dynamics — add is free, remove is graceful (prereq for a reactive scaler)
+
+With `so_reuseport`, scaling worker count up/down under live load:
+
+- **Scale up** — a newly-spawned pooler registers with the kernel's `so_reuseport` group and starts
+  taking _new_ connections immediately; existing connections on other workers are untouched. Demonstrated
+  in `exp-multiworker.sh` (added poolers take proportional load within the 2 s startup).
+- **Scale down** — `SIGINT` (pgbouncer "safe shutdown") drains gracefully. Measured: 200 persistent
+  TLS connections at ~38k qps across 2 poolers, `SIGINT` one at t=3 s → it finished in-flight
+  transactions and exited within 2 s; driver saw **`ok=305340 err=102 reconnects=302`** — i.e. the
+  ~100 connections that were on the victim each took _exactly one_ error and reconnected (kernel routed
+  them to the survivor). **0.033% error blip, no stall.** Use `SIGINT` (never `SIGTERM`/`SIGKILL`) to
+  shed a worker. Script: scale-dynamics run (persistent `pd.py` driver).
+
+**Net pooler conclusion:** the reactive in-VM pooler scaler is empirically sound. The mechanism scales
+(linear), the trigger is observable (per-worker CPU in `/proc`), scale-up is zero-disruption, and
+scale-down is graceful at ~0.03% cost. The remaining open question is purely _workload_ — does a real
+Beyond instance's connection-churn rate exceed ~2.4k/s/core often enough to need a 2nd worker — which
+only production telemetry answers.
+
+### D2 `maintenance_io_concurrency` (cold-from-MinIO VACUUM)
+
+See `d2-coldfork.txt` — measured via glidefs-restart-with-fresh-cache (data persists in MinIO,
+empty cache → cold reads). Result appended on completion.
+
+## Net so far
+
+Measurement has already paid for itself by **killing two speculative knobs** (zstd, wal_recycle) that
+intuition/the-ZFS-playbook would have shipped. The high-value hypotheses (checkpoint frequency,
+maintenance_io_concurrency, cooldown-under-churn) remain open and need the tailored experiments above.
diff --git a/bench/glidefs-pg/conf/a1-no-recycle.conf b/bench/glidefs-pg/conf/a1-no-recycle.conf
new file mode 100644
index 0000000..86b3bbe
--- /dev/null
+++ b/bench/glidefs-pg/conf/a1-no-recycle.conf
@@ -0,0 +1,3 @@
+# A1: the demoted ZFS-style CoW WAL hygiene — expected ~noise on GlideFS
+wal_recycle = off
+wal_init_zero = off
diff --git a/bench/glidefs-pg/conf/baseline.conf b/bench/glidefs-pg/conf/baseline.conf
new file mode 100644
index 0000000..c15dd6d
--- /dev/null
+++ b/bench/glidefs-pg/conf/baseline.conf
@@ -0,0 +1,43 @@
+listen_addresses = ''
+# Harness baseline — the substrate-relevant subset of
+# packer/files/postgresql/00-beyond.conf. Memory knobs are FIXED here for
+# reproducibility (the real image autotunes them via src/config.rs); the
+# harness's job is to isolate the storage knobs, so everything except the knob
+# under test is held constant. No extensions, no networking, no TLS.
+
+# fixed memory (reproducibility, not under test)
+shared_buffers = 1GB
+effective_cache_size = 3GB
+work_mem = 32MB
+maintenance_work_mem = 256MB
+max_wal_size = 8GB
+min_wal_size = 1GB
+
+# storage cost model (matches 00-beyond.conf)
+random_page_cost = 1.1
+effective_io_concurrency = 200
+
+# durability / WAL (matches 00-beyond.conf; wal_level=replica is lighter than
+# logical and constant across experiments, so it doesn't bias the comparison)
+fsync = on
+synchronous_commit = on
+full_page_writes = on
+wal_sync_method = fdatasync
+# PGDG PG18 (in the QEMU guest) is built --with-lz4/--with-zstd, matching the
+# real image. D1 swaps this to zstd.
+wal_compression = lz4
+wal_level = replica
+
+# checkpoints / bgwriter — the coalescing knobs (C experiments override these)
+checkpoint_timeout = 15min
+checkpoint_completion_target = 0.9
+bgwriter_delay = 100ms
+bgwriter_lru_maxpages = 1000
+
+# autovacuum (E experiments override)
+autovacuum_vacuum_scale_factor = 0.1
+autovacuum_naptime = 30s
+
+# observability for the run log
+log_checkpoints = on
+log_min_messages = warning
diff --git a/bench/glidefs-pg/conf/c1-checkpoint30.conf b/bench/glidefs-pg/conf/c1-checkpoint30.conf
new file mode 100644
index 0000000..be358c0
--- /dev/null
+++ b/bench/glidefs-pg/conf/c1-checkpoint30.conf
@@ -0,0 +1,2 @@
+# C1: fewer/larger checkpoints — 15min -> 30min
+checkpoint_timeout = 30min
diff --git a/bench/glidefs-pg/conf/c1-checkpoint60.conf b/bench/glidefs-pg/conf/c1-checkpoint60.conf
new file mode 100644
index 0000000..4fc0c6b
--- /dev/null
+++ b/bench/glidefs-pg/conf/c1-checkpoint60.conf
@@ -0,0 +1,2 @@
+# C1: fewer/larger checkpoints — 15min -> 60min
+checkpoint_timeout = 60min
diff --git a/bench/glidefs-pg/conf/c1-cp300s.conf b/bench/glidefs-pg/conf/c1-cp300s.conf
new file mode 100644
index 0000000..fa13e25
--- /dev/null
+++ b/bench/glidefs-pg/conf/c1-cp300s.conf
@@ -0,0 +1,5 @@
+# C1: checkpoint cadence — timeout=300s, wal-triggered checkpoints disabled (big max_wal_size).
+# Tests FPW-frequency mechanism: more frequent checkpoints => more full-page images => more S3 bytes.
+checkpoint_timeout = 300s
+max_wal_size = 32GB
+checkpoint_completion_target = 0.9
diff --git a/bench/glidefs-pg/conf/c1-cp30s.conf b/bench/glidefs-pg/conf/c1-cp30s.conf
new file mode 100644
index 0000000..721d54b
--- /dev/null
+++ b/bench/glidefs-pg/conf/c1-cp30s.conf
@@ -0,0 +1,5 @@
+# C1: checkpoint cadence — timeout=30s, wal-triggered checkpoints disabled (big max_wal_size).
+# Tests FPW-frequency mechanism: more frequent checkpoints => more full-page images => more S3 bytes.
+checkpoint_timeout = 30s
+max_wal_size = 32GB
+checkpoint_completion_target = 0.9
diff --git a/bench/glidefs-pg/conf/c1-cp45s.conf b/bench/glidefs-pg/conf/c1-cp45s.conf
new file mode 100644
index 0000000..985e558
--- /dev/null
+++ b/bench/glidefs-pg/conf/c1-cp45s.conf
@@ -0,0 +1,5 @@
+# C1: checkpoint cadence — timeout=45s, wal-triggered checkpoints disabled (big max_wal_size).
+# Tests FPW-frequency mechanism: more frequent checkpoints => more full-page images => more S3 bytes.
+checkpoint_timeout = 45s
+max_wal_size = 32GB
+checkpoint_completion_target = 0.9
diff --git a/bench/glidefs-pg/conf/c1-cp90s.conf b/bench/glidefs-pg/conf/c1-cp90s.conf
new file mode 100644
index 0000000..69cb493
--- /dev/null
+++ b/bench/glidefs-pg/conf/c1-cp90s.conf
@@ -0,0 +1,5 @@
+# C1: checkpoint cadence — timeout=90s, wal-triggered checkpoints disabled (big max_wal_size).
+# Tests FPW-frequency mechanism: more frequent checkpoints => more full-page images => more S3 bytes.
+checkpoint_timeout = 90s
+max_wal_size = 32GB
+checkpoint_completion_target = 0.9
diff --git a/bench/glidefs-pg/conf/c2-flushafter-0.conf b/bench/glidefs-pg/conf/c2-flushafter-0.conf
new file mode 100644
index 0000000..5ea4216
--- /dev/null
+++ b/bench/glidefs-pg/conf/c2-flushafter-0.conf
@@ -0,0 +1,5 @@
+# C2: disable forced writeback entirely (let OS + GlideFS coalesce maximally)
+checkpoint_flush_after = 0
+bgwriter_flush_after = 0
+backend_flush_after = 0
+wal_writer_flush_after = 0
diff --git a/bench/glidefs-pg/conf/c2-flushafter-2mb.conf b/bench/glidefs-pg/conf/c2-flushafter-2mb.conf
new file mode 100644
index 0000000..8090eb0
--- /dev/null
+++ b/bench/glidefs-pg/conf/c2-flushafter-2mb.conf
@@ -0,0 +1,4 @@
+# C2: relax forced writeback so GlideFS coalesces more (default 256KB/512KB -> 2MB)
+checkpoint_flush_after = 2MB
+bgwriter_flush_after = 2MB
+backend_flush_after = 2MB
diff --git a/bench/glidefs-pg/conf/c3-aggressive.conf b/bench/glidefs-pg/conf/c3-aggressive.conf
new file mode 100644
index 0000000..67ff475
--- /dev/null
+++ b/bench/glidefs-pg/conf/c3-aggressive.conf
@@ -0,0 +1,3 @@
+# C3: aggressive bgwriter — pushes dirty pages to the device early
+bgwriter_delay = 50ms
+bgwriter_lru_maxpages = 1000
diff --git a/bench/glidefs-pg/conf/c3-gentle.conf b/bench/glidefs-pg/conf/c3-gentle.conf
new file mode 100644
index 0000000..3f4e5d3
--- /dev/null
+++ b/bench/glidefs-pg/conf/c3-gentle.conf
@@ -0,0 +1,3 @@
+# C3: gentle bgwriter — hold dirty pages in shared_buffers, coalesce, flush at checkpoint
+bgwriter_delay = 500ms
+bgwriter_lru_maxpages = 100
diff --git a/bench/glidefs-pg/conf/c3-off.conf b/bench/glidefs-pg/conf/c3-off.conf
new file mode 100644
index 0000000..44f2cfe
--- /dev/null
+++ b/bench/glidefs-pg/conf/c3-off.conf
@@ -0,0 +1,2 @@
+# C3: bgwriter disabled — only checkpoint/backend evictions write pages out
+bgwriter_lru_maxpages = 0
diff --git a/bench/glidefs-pg/conf/d-maintenance-io.conf b/bench/glidefs-pg/conf/d-maintenance-io.conf
new file mode 100644
index 0000000..1658119
--- /dev/null
+++ b/bench/glidefs-pg/conf/d-maintenance-io.conf
@@ -0,0 +1,2 @@
+# D2: parallelize cold-from-S3 maintenance scans (VACUUM/index build on a fork)
+maintenance_io_concurrency = 200
diff --git a/bench/glidefs-pg/conf/d1-zstd.conf b/bench/glidefs-pg/conf/d1-zstd.conf
new file mode 100644
index 0000000..b041c7b
--- /dev/null
+++ b/bench/glidefs-pg/conf/d1-zstd.conf
@@ -0,0 +1,2 @@
+# D1: trade CPU for fewer WAL bytes to S3 (and to the WAL sink)
+wal_compression = zstd
diff --git a/bench/glidefs-pg/conf/e1-autovac-throttled.conf b/bench/glidefs-pg/conf/e1-autovac-throttled.conf
new file mode 100644
index 0000000..ed7fc97
--- /dev/null
+++ b/bench/glidefs-pg/conf/e1-autovac-throttled.conf
@@ -0,0 +1,6 @@
+# E1: fork/ephemeral autovacuum profile — throttle churn (every dirtied inherited
+# page is a CoW divergence). Wraparound protection stays intact.
+autovacuum_vacuum_scale_factor = 0.4
+autovacuum_naptime = 5min
+autovacuum_vacuum_cost_delay = 20ms
+autovacuum_vacuum_cost_limit = 200
diff --git a/bench/glidefs-pg/delta.awk b/bench/glidefs-pg/delta.awk
new file mode 100755
index 0000000..372324c
--- /dev/null
+++ b/bench/glidefs-pg/delta.awk
@@ -0,0 +1,48 @@
+#!/usr/bin/awk -f
+# Diff two GlideFS Prometheus snapshots (before, after) for one export.
+# Counters (*_total) show after-before; gauges show before -> after.
+# Highlights the substrate-cost metrics first, then dumps the rest.
+BEGIN {
+  split("glidefs_s3_batches_written_total glidefs_s3_bytes_written_total " \
+        "glidefs_s3_bytes_read_total glidefs_s3_read_ops_total " \
+        "glidefs_guest_bytes_written_total glidefs_guest_write_ops_total " \
+        "glidefs_cache_hits_total glidefs_cache_misses_total " \
+        "glidefs_write_amplification glidefs_coalesce_ratio glidefs_dirty_blocks", H, " ")
+  for (i in H) hi[H[i]] = i
+}
+function name(key,   n) { n = key; sub(/[{ ].*/, "", n); return n }
+function isctr(key) { return (name(key) ~ /_total$/) }
+# parse "metric{labels} value"  -> key=metric{labels}, val=value
+function load(file, arr,   line, k, v, p) {
+  while ((getline line < file) > 0) {
+    if (line ~ /^#/ || line == "") continue
+    p = match(line, /[ \t][^ \t]*$/)
+    if (!p) continue
+    v = substr(line, p+1); gsub(/[ \t]/, "", v)
+    k = substr(line, 1, p-1); gsub(/[ \t]+$/, "", k)
+    arr[k] = v
+  }
+  close(file)
+}
+BEGIN {
+  load(ARGV[1], B); load(ARGV[2], A)
+  fmt = "%-46s %16s %16s %16s\n"
+  printf fmt, "metric (export-scoped)", "before", "after", "delta/Δ"
+  printf "%s\n", "----------------------------------------------------------------------------------------------------"
+  # highlights first, in declared order
+  for (rank=1; rank<=length(H); rank++) {
+    for (k in A) {
+      if (name(k) != H[rank]) continue
+      emit(k)
+    }
+  }
+  printf "%s\n", "---- other export metrics ----"
+  for (k in A) { if (!(name(k) in hi)) emit(k) }
+}
+function emit(k,   b, a, d) {
+  b = (k in B) ? B[k] : 0; a = A[k]
+  if (isctr(k)) { d = a - b; printf fmt, short(k), b, a, sprintf("%+d", d) }
+  else          { printf fmt, short(k), b, a, "(gauge)" }
+}
+function short(k,   s) { s = k; sub(/^glidefs_/, "", s); sub(/\{export="[^"]*"\}/, "", s);
+                        sub(/\{[^}]*export="[^"]*"[^}]*\}/, "", s); return s }
diff --git a/bench/glidefs-pg/exp-d2-coldfork.sh b/bench/glidefs-pg/exp-d2-coldfork.sh
new file mode 100644
index 0000000..819dc2f
--- /dev/null
+++ b/bench/glidefs-pg/exp-d2-coldfork.sh
@@ -0,0 +1,102 @@
+#!/usr/bin/env bash
+# D2: maintenance_io_concurrency on COLD-from-S3 reads.
+#
+# GlideFS fans cold block reads into parallel S3 GETs; PG's maintenance_io_concurrency
+# drives VACUUM prefetch depth. We populate a parent against MinIO (real HTTP latency),
+# snapshot it, then fork into a FRESH glidefs cache (so every read misses → MinIO) and
+# time a cold VACUUM at m_io_c=10 vs 200. Each run gets its own fresh cache = truly cold.
+set -euo pipefail
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+GLIDEFS="${GLIDEFS:-$(command -v glidefs)}"; PG="$(dirname "$(command -v initdb)")"
+API=127.0.0.1:18080; NBD=127.0.0.1:10899
+W="$(mktemp -d "${WORKROOT:-/tmp}/gfd2.XXXXXX")"
+RUN="$HERE/out/$(date +%Y%m%d-%H%M%S)-d2"; mkdir -p "$RUN"
+SCALE="${SCALE:-100}"; MINIO_PID=""; GF_PID=""
+log(){ printf '\033[1;32m[D2]\033[0m %s\n' "$*" >&2; }
+gf_stop(){ [ -n "$GF_PID" ] && { sudo kill "$GF_PID" 2>/dev/null; wait "$GF_PID" 2>/dev/null; GF_PID=""; }; }
+cleanup(){ set +e; "$PG/pg_ctl" -D "$W"/pg* -m immediate stop >/dev/null 2>&1
+  sudo umount "$W"/m* 2>/dev/null; gf_stop
+  [ -n "$MINIO_PID" ] && kill "$MINIO_PID" 2>/dev/null; sudo rm -rf "$W"; }
+trap cleanup EXIT INT TERM
+
+# --- MinIO ---
+cd "$W"
+[ -x ./minio ] || curl -fsSL -o minio https://dl.min.io/server/minio/release/linux-amd64/minio && chmod +x minio
+[ -x ./mc ]   || curl -fsSL -o mc   https://dl.min.io/client/mc/release/linux-amd64/mc && chmod +x mc
+mkdir -p "$W/minio-data"
+MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=minioadmin ./minio server "$W/minio-data" --address 127.0.0.1:9000 >"$RUN/minio.log" 2>&1 &
+MINIO_PID=$!
+for _ in $(seq 1 50); do curl -fsS http://127.0.0.1:9000/minio/health/live >/dev/null 2>&1 && break; sleep 0.2; done
+./mc alias set loc http://127.0.0.1:9000 minioadmin minioadmin >/dev/null 2>&1
+./mc mb loc/bench >/dev/null 2>&1 || true
+
+gf_toml(){ # $1=cache_dir $2=export-stanza
+cat <<EOF
+[cache]
+dir = "$1"
+disk_size_gb = 30.0
+memory_size_gb = 2.0
+[storage]
+url = "s3://bench/pg"
+[aws]
+access_key_id = "minioadmin"
+secret_access_key = "minioadmin"
+endpoint = "http://127.0.0.1:9000"
+allow_http = "true"
+region = "us-east-1"
+[servers.nbd]
+addresses = ["$NBD"]
+unix_socket = "$W/gf.sock"
+api_address = "$API"
+block_size = 131072
+flush_threshold = 500
+$2
+EOF
+}
+gf_start(){ # $1=toml
+  curl -fsS "http://$API/health" >/dev/null 2>&1 && { echo "API busy"; exit 1; }
+  sudo -b sh -c "exec env NO_COLOR=1 '$GLIDEFS' run -c '$1' >>'$RUN/glidefs.log' 2>&1"
+  for _ in $(seq 1 50); do GF_PID="$(pgrep -f "glidefs run -c $1"|head -1)"; [ -n "$GF_PID" ] && break; sleep 0.1; done
+  for _ in $(seq 1 100); do curl -fsS "http://$API/health" >/dev/null 2>&1 && break; sleep 0.1; done; }
+dev_of(){ for _ in $(seq 1 100); do d="$(curl -fsS "http://$API/api/exports/$1"|jq -r '.device // empty')"; [ -n "$d" ] && [ -b "$d" ] && { echo "$d"; return; }; sleep 0.1; done; }
+
+# --- Parent: populate against MinIO, snapshot ---
+gf_toml "$W/c1" $'[[servers.nbd.exports]]\nname = "dA"\nsize_gb = 20\ntransport = "nbd"' > "$W/t1.toml"
+gf_start "$W/t1.toml"
+DA="$(dev_of dA)"; log "parent dA $DA, populate scale=$SCALE"
+sudo mkfs.ext4 -q -F "$DA"; sudo mkdir -p "$W/ma"; sudo mount "$DA" "$W/ma"; sudo chown -R "$USER:$(id -gn)" "$W/ma"
+"$PG/initdb" -D "$W/pgA" -A trust -U postgres --no-sync >/dev/null 2>&1
+echo "shared_buffers=1GB" >> "$W/pgA/postgresql.conf"
+"$PG/pg_ctl" -D "$W/pgA" -l "$RUN/pgA.log" -o "-p 5440 -k $W/ma" -w start >/dev/null
+"$PG/pgbench" -h "$W/ma" -p 5440 -U postgres -i -q -s "$SCALE" postgres >/dev/null 2>&1
+"$PG/psql" -h "$W/ma" -p 5440 -U postgres -c "CREATE INDEX ON pgbench_accounts(abalance); CHECKPOINT;" >/dev/null
+"$PG/pg_ctl" -D "$W/pgA" -m fast stop >/dev/null
+curl -fsS -X POST "http://$API/api/exports/dA/drain" >/dev/null; sleep 2
+SEQ="$(curl -fsS -X POST "http://$API/api/exports/dA/snapshot"|jq -r .sequence)"
+log "snapshot seq=$SEQ (data now in MinIO)"; gf_stop
+
+cold_vacuum(){ # $1=mioc
+  local mioc="$1" t
+  rm -rf "$W/c2"; mkdir -p "$W/c2"   # FRESH cache => cold reads hit MinIO
+  gf_toml "$W/c2" "" > "$W/t2.toml"; gf_start "$W/t2.toml"
+  curl -fsS -X PUT "http://$API/api/exports/dB" -H 'Content-Type: application/json' \
+    -d "{\"size_gb\":20,\"transport\":\"nbd\",\"manifest_name\":\"dA\",\"snapshot_sequence\":$SEQ}" >/dev/null
+  local DB; DB="$(dev_of dB)"
+  sudo mkdir -p "$W/mb"; sudo mount "$DB" "$W/mb"; sudo chown -R "$USER:$(id -gn)" "$W/mb"
+  cat >> "$W/mb/pgdata/postgresql.conf" <<P
+shared_buffers = 256MB
+maintenance_io_concurrency = $mioc
+P
+  "$PG/pg_ctl" -D "$W/mb/pgdata" -l "$RUN/pgB-$mioc.log" -o "-p 5441 -k $W/mb" -w start >/dev/null
+  # cold maintenance: full VACUUM-scan of the big table + its index (reads from MinIO)
+  t=$("$PG/psql" -h "$W/mb" -p 5441 -U postgres -At -c "\timing on" \
+        -c "VACUUM (DISABLE_PAGE_SKIPPING) pgbench_accounts;" 2>&1 | grep -oE 'Time: [0-9.]+' | tail -1 | grep -oE '[0-9.]+')
+  printf "  maintenance_io_concurrency=%-4s  cold VACUUM = %s ms\n" "$mioc" "$t" | tee -a "$RUN/score.txt"
+  "$PG/pg_ctl" -D "$W/mb/pgdata" -m immediate stop >/dev/null 2>&1
+  sudo umount "$W/mb" 2>/dev/null; gf_stop
+}
+echo "D2: cold-from-MinIO VACUUM, maintenance_io_concurrency sweep (scale=$SCALE)" | tee "$RUN/score.txt"
+cold_vacuum 10
+cold_vacuum 200
+cold_vacuum 10
+cold_vacuum 200
\ No newline at end of file
diff --git a/bench/glidefs-pg/exp-d2.sh b/bench/glidefs-pg/exp-d2.sh
new file mode 100644
index 0000000..deed570
--- /dev/null
+++ b/bench/glidefs-pg/exp-d2.sh
@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+# D2: maintenance_io_concurrency on COLD-from-S3 reads (no fork API needed).
+#
+# Populate an export against MinIO (real HTTP latency), drain, then RESTART glidefs
+# with a FRESH cache — the data persists in MinIO, the empty cache forces every read
+# to miss → MinIO. Time a cold VACUUM at m_io_c=10 vs 200. GlideFS fans cold reads
+# into parallel S3 GETs; high m_io_c drives PG's prefetch depth, hiding latency.
+set -euo pipefail
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+GLIDEFS="${GLIDEFS:-$(command -v glidefs)}"; PG="$(dirname "$(command -v initdb)")"
+API=127.0.0.1:18080; NBD=127.0.0.1:10899
+W="$(mktemp -d "${WORKROOT:-/tmp}/gfd2.XXXXXX")"
+RUN="$HERE/out/$(date +%Y%m%d-%H%M%S)-d2"; mkdir -p "$RUN"
+SCALE="${SCALE:-20}"; MINIO_PID=""; GF_PID=""
+log(){ printf '\033[1;32m[D2]\033[0m %s\n' "$*" >&2; }
+# NB: glidefs is started via `sudo -b` so it's NOT a child of this shell — `wait`
+# on it returns 127 and trips set -e. Poll for death instead.
+gf_stop(){ [ -n "$GF_PID" ] || return 0; sudo kill "$GF_PID" 2>/dev/null || true
+  for _ in $(seq 1 50); do kill -0 "$GF_PID" 2>/dev/null || break; sleep 0.1; done; GF_PID=""; }
+cleanup(){ set +e; "$PG/pg_ctl" -D "$W/m/pgdata" -m immediate stop >/dev/null 2>&1
+  sudo umount "$W/m" 2>/dev/null; gf_stop; [ -n "$MINIO_PID" ] && kill "$MINIO_PID" 2>/dev/null; sudo rm -rf "$W"; }
+trap cleanup EXIT INT TERM
+
+cd "$W"
+log "start MinIO (cached binaries)"
+[ -x /tmp/minio ] || { curl -fsSL -o /tmp/minio https://dl.min.io/server/minio/release/linux-amd64/minio && chmod +x /tmp/minio; }
+[ -x /tmp/mc ]    || { curl -fsSL -o /tmp/mc https://dl.min.io/client/mc/release/linux-amd64/mc && chmod +x /tmp/mc; }
+ln -sf /tmp/minio minio; ln -sf /tmp/mc mc
+mkdir -p "$W/minio-data" "$W/m"
+MINIO_ROOT_USER=minioadmin MINIO_ROOT_PASSWORD=minioadmin ./minio server "$W/minio-data" --address 127.0.0.1:9000 >"$RUN/minio.log" 2>&1 &
+MINIO_PID=$!
+for _ in $(seq 1 60); do curl -fsS http://127.0.0.1:9000/minio/health/live >/dev/null 2>&1 && break; sleep 0.2; done
+./mc alias set loc http://127.0.0.1:9000 minioadmin minioadmin >/dev/null 2>&1; ./mc mb loc/bench >/dev/null 2>&1 || true
+
+cat > "$W/gf.toml" <<EOF
+[cache]
+dir = "$W/cache"
+disk_size_gb = 30.0
+memory_size_gb = 2.0
+[storage]
+url = "s3://bench/pg"
+[aws]
+access_key_id = "minioadmin"
+secret_access_key = "minioadmin"
+endpoint = "http://127.0.0.1:9000"
+allow_http = "true"
+region = "us-east-1"
+[servers.nbd]
+addresses = ["$NBD"]
+unix_socket = "$W/s.sock"
+api_address = "$API"
+block_size = 131072
+flush_threshold = 500
+[[servers.nbd.exports]]
+name = "dA"
+size_gb = 20
+transport = "nbd"
+EOF
+gf_start(){ curl -fsS "http://$API/health" >/dev/null 2>&1 && { echo "API busy"; exit 1; }
+  sudo -b sh -c "exec env NO_COLOR=1 '$GLIDEFS' run -c '$W/gf.toml' >>'$RUN/glidefs.log' 2>&1"
+  for _ in $(seq 1 50); do GF_PID="$(pgrep -f "glidefs run -c $W/gf.toml"|head -1)"; [ -n "$GF_PID" ] && break; sleep 0.1; done
+  for _ in $(seq 1 150); do curl -fsS "http://$API/health" >/dev/null 2>&1 && break; sleep 0.1; done; }
+dev_of(){ for _ in $(seq 1 150); do d="$(curl -fsS "http://$API/api/exports/dA"|jq -r '.device // empty' 2>/dev/null)"; [ -n "$d" ] && [ -b "$d" ] && { echo "$d"; return; }; sleep 0.1; done; }
+
+log "populate dA scale=$SCALE against MinIO"
+gf_start; DA="$(dev_of)"
+sudo mkfs.ext4 -q -F "$DA"; sudo mount "$DA" "$W/m"; sudo chown -R "$USER:$(id -gn)" "$W/m"
+"$PG/initdb" -D "$W/m/pgdata" -A trust -U postgres --no-sync >/dev/null 2>&1
+printf "listen_addresses = ''\nshared_buffers = 1GB\n" >> "$W/m/pgdata/postgresql.conf"
+"$PG/pg_ctl" -D "$W/m/pgdata" -l "$RUN/pgload.log" -o "-p 5440 -k $W/m" -w start >/dev/null
+"$PG/pgbench" -h "$W/m" -p 5440 -U postgres -i -q -s "$SCALE" postgres >/dev/null 2>&1
+"$PG/psql" -h "$W/m" -p 5440 -U postgres -c "CREATE INDEX ON pgbench_accounts(abalance); CHECKPOINT;" >/dev/null
+"$PG/pg_ctl" -D "$W/m/pgdata" -m fast stop >/dev/null
+sudo umount "$W/m"
+curl -fsS -X POST "http://$API/api/exports/dA/drain" >/dev/null; sleep 2; gf_stop
+log "data persisted to MinIO; restarting cold per m_io_c"
+
+cold_vacuum(){ local mioc="$1" t
+  sudo rm -rf "$W/cache"; mkdir -p "$W/cache"   # FRESH cache => cold reads hit MinIO (root-owned)
+  gf_start; local DB; DB="$(dev_of)"
+  sudo mount "$DB" "$W/m"; sudo chown -R "$USER:$(id -gn)" "$W/m"
+  # set the knob (sed replaces any prior line)
+  sed -i '/maintenance_io_concurrency/d;/shared_buffers/d;/listen_addresses/d' "$W/m/pgdata/postgresql.conf"
+  printf "listen_addresses = ''\nshared_buffers = 256MB\nmaintenance_io_concurrency = %s\n" "$mioc" >> "$W/m/pgdata/postgresql.conf"
+  "$PG/pg_ctl" -D "$W/m/pgdata" -l "$RUN/pg-$mioc.log" -o "-p 5441 -k $W/m" -w start >/dev/null
+  t=$("$PG/psql" -h "$W/m" -p 5441 -U postgres -At -c "\timing on" -c "VACUUM (DISABLE_PAGE_SKIPPING) pgbench_accounts;" 2>&1 | grep -oE 'Time: [0-9.]+' | tail -1 | grep -oE '[0-9.]+')
+  printf "  maintenance_io_concurrency=%-4s  cold VACUUM = %s ms\n" "$mioc" "$t" | tee -a "$RUN/score.txt"
+  "$PG/pg_ctl" -D "$W/m/pgdata" -m immediate stop >/dev/null 2>&1; sudo umount "$W/m" 2>/dev/null; gf_stop
+}
+echo "D2: cold-from-MinIO VACUUM, maintenance_io_concurrency (scale=$SCALE)" | tee "$RUN/score.txt"
+cold_vacuum 10; cold_vacuum 200
diff --git a/bench/glidefs-pg/exp-e-fork.sh b/bench/glidefs-pg/exp-e-fork.sh
new file mode 100644
index 0000000..3e7cddb
--- /dev/null
+++ b/bench/glidefs-pg/exp-e-fork.sh
@@ -0,0 +1,104 @@
+#!/usr/bin/env bash
+# E: autovacuum-on-CoW divergence.
+#
+# Populate parent export A with dead tuples, snapshot it, FORK child B (shares all
+# blocks, 0 divergence at t=0), then let B idle under a given autovacuum profile.
+# A's Postgres is STOPPED during B's idle window, so A is quiescent → every flush
+# in that window is B's divergence. Measure B's uploaded bytes = CoW divergence cost.
+#
+# Usage: exp-e-fork.sh <aggressive|throttled>
+set -euo pipefail
+PROFILE="${1:?aggressive|throttled}"
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+GLIDEFS="${GLIDEFS:-$(command -v glidefs)}"; PG="$(dirname "$(command -v initdb)")"
+API=127.0.0.1:18080; NBD=127.0.0.1:10899
+W="$(mktemp -d "${WORKROOT:-/tmp}/gfe.XXXXXX")"
+RUN="$HERE/out/$(date +%Y%m%d-%H%M%S)-e-$PROFILE"; mkdir -p "$RUN"
+GF_LOG="$RUN/glidefs.log"; GF_PID=""; MA=""; MB=""
+log(){ printf '\033[1;35m[E]\033[0m %s\n' "$*" >&2; }
+cleanup(){ set +e
+  [ -n "$MB" ] && { "$PG/pg_ctl" -D "$W/pgB" -m immediate stop >/dev/null 2>&1; sudo umount "$W/mb" 2>/dev/null; }
+  [ -n "$MA" ] && { sudo umount "$W/ma" 2>/dev/null; }
+  curl -fsS -X DELETE "http://$API/api/exports/eA?purge=true" >/dev/null 2>&1
+  curl -fsS -X DELETE "http://$API/api/exports/eB?purge=true" >/dev/null 2>&1
+  [ -n "$GF_PID" ] && { sudo kill "$GF_PID" 2>/dev/null; wait "$GF_PID" 2>/dev/null; }
+  sudo rm -rf "$W" 2>/dev/null; }
+trap cleanup EXIT INT TERM
+
+cat > "$W/gf.toml" <<EOF
+[cache]
+dir = "$W/cache"
+disk_size_gb = 20.0
+memory_size_gb = 2.0
+[storage]
+url = "memory:///"
+[servers.nbd]
+addresses = ["$NBD"]
+unix_socket = "$W/gf.sock"
+api_address = "$API"
+block_size = 131072
+flush_threshold = 500
+[[servers.nbd.exports]]
+name = "eA"
+size_gb = 10
+transport = "nbd"
+EOF
+mkdir -p "$W/cache" "$W/ma" "$W/mb"
+curl -fsS "http://$API/health" >/dev/null 2>&1 && { echo "API $API busy"; exit 1; }
+log "start glidefs"
+sudo -b sh -c "exec env NO_COLOR=1 '$GLIDEFS' run -c '$W/gf.toml' >'$GF_LOG' 2>&1"
+for _ in $(seq 1 50); do GF_PID="$(pgrep -f "glidefs run -c $W/gf.toml"|head -1)"; [ -n "$GF_PID" ] && break; sleep 0.1; done
+for _ in $(seq 1 100); do curl -fsS "http://$API/health" >/dev/null 2>&1 && break; sleep 0.1; done
+
+DA="$(curl -fsS "http://$API/api/exports/eA"|jq -r .device)"
+log "parent eA on $DA"
+sudo mkfs.ext4 -q -F "$DA"; sudo mount "$DA" "$W/ma"; sudo chown -R "$USER:$(id -gn)" "$W/ma"
+"$PG/initdb" -D "$W/pgA" -A trust -U postgres --no-sync >/dev/null 2>&1
+echo "shared_buffers=512MB" >> "$W/pgA/postgresql.conf"
+echo "autovacuum=off" >> "$W/pgA/postgresql.conf"   # no autovac on parent; we want dead tuples to survive into the fork
+"$PG/pg_ctl" -D "$W/pgA" -l "$RUN/pgA.log" -o "-p 5440 -k $W/ma" -w start >/dev/null
+PSQLA=("$PG/psql" -h "$W/ma" -p 5440 -U postgres)
+log "load + churn (create dead tuples)"
+"$PG/pgbench" -h "$W/ma" -p 5440 -U postgres -i -q -s 50 postgres >/dev/null 2>&1
+# heavy UPDATE churn => many dead tuples for autovacuum to chase on the fork
+"$PG/pgbench" -h "$W/ma" -p 5440 -U postgres -t 20000 -c 8 -j 4 postgres >/dev/null 2>&1
+"${PSQLA[@]}" -c "CHECKPOINT;" >/dev/null
+"$PG/pg_ctl" -D "$W/pgA" -m fast stop >/dev/null    # A quiescent from here on
+curl -fsS -X POST "http://$API/api/exports/eA/drain" >/dev/null; sleep 2
+SEQ="$(curl -fsS -X POST "http://$API/api/exports/eA/snapshot"|jq -r .sequence)"
+log "snapshot eA seq=$SEQ; forking eB"
+curl -fsS -X PUT "http://$API/api/exports/eB" -H 'Content-Type: application/json' \
+  -d "{\"size_gb\":10,\"transport\":\"nbd\",\"manifest_name\":\"eA\",\"snapshot_sequence\":$SEQ}" >/dev/null
+for _ in $(seq 1 100); do DB="$(curl -fsS "http://$API/api/exports/eB"|jq -r '.device // empty')"; [ -n "$DB" ] && [ -b "$DB" ] && break; sleep 0.1; done
+log "fork eB on $DB"
+sudo mount "$DB" "$W/mb"; sudo chown -R "$USER:$(id -gn)" "$W/mb"
+# autovacuum profile under test
+if [ "$PROFILE" = aggressive ]; then
+  cat >> "$W/mb/pgdata/postgresql.conf" <<'P'
+autovacuum = on
+autovacuum_vacuum_scale_factor = 0.01
+autovacuum_naptime = 10s
+autovacuum_vacuum_cost_delay = 0
+P
+else
+  cat >> "$W/mb/pgdata/postgresql.conf" <<'P'
+autovacuum = on
+autovacuum_vacuum_scale_factor = 0.4
+autovacuum_naptime = 5min
+autovacuum_vacuum_cost_delay = 20ms
+autovacuum_vacuum_cost_limit = 200
+P
+fi
+"$PG/pg_ctl" -D "$W/mb/pgdata" -l "$RUN/pgB.log" -o "-p 5441 -k $W/mb" -w start >/dev/null
+MB=1
+# mark window; B idles (no client workload) — only autovacuum writes
+curl -fsS -X POST "http://$API/api/exports/eB/drain" >/dev/null; sleep 1
+L0=$(wc -l < "$GF_LOG")
+log "B idling 120s under '$PROFILE' autovacuum..."
+sleep 120
+"$PG/psql" -h "$W/mb" -p 5441 -U postgres -c "CHECKPOINT;" >/dev/null 2>&1
+curl -fsS -X POST "http://$API/api/exports/eB/drain" >/dev/null; sleep 3
+# divergence = bytes B flushed during the idle window (A is quiescent)
+DIV=$(tail -n +$((L0+1)) "$GF_LOG" | sed 's/\x1b\[[0-9;]*m//g' | awk '/flush complete/{for(i=1;i<=NF;i++){n=split($i,kv,"=");if(n==2&&kv[1]=="bytes_uploaded")b+=kv[2]}}END{print b+0}')
+PK=$(tail -n +$((L0+1)) "$GF_LOG" | sed 's/\x1b\[[0-9;]*m//g' | awk '/flush complete/{for(i=1;i<=NF;i++){n=split($i,kv,"=");if(n==2&&kv[1]=="packs_uploaded")p+=kv[2]}}END{print p+0}')
+printf "E[%s]: fork divergence = %d bytes (%.1f MB) over 120s idle, packs=%d\n" "$PROFILE" "$DIV" "$(echo "$DIV/1048576"|bc -l)" "$PK" | tee "$RUN/score.txt"
diff --git a/bench/glidefs-pg/exp-e.sh b/bench/glidefs-pg/exp-e.sh
new file mode 100644
index 0000000..36c29d0
--- /dev/null
+++ b/bench/glidefs-pg/exp-e.sh
@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+# E: autovacuum write cost on GlideFS = the CoW divergence an idle fork would incur.
+#
+# Every page autovacuum dirties on a fork is a new block GlideFS must upload (lost
+# parent sharing). We measure that directly without the fork API: load + churn to
+# create dead tuples with autovacuum OFF, drain, then turn autovacuum ON under a
+# given profile and idle (no client load). The S3 bytes flushed during the idle =
+# what autovacuum alone wrote = the divergence cost.
+#
+# Usage: exp-e.sh <aggressive|throttled>
+set -euo pipefail
+PROFILE="${1:?aggressive|throttled}"
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+GLIDEFS="${GLIDEFS:-$(command -v glidefs)}"; PG="$(dirname "$(command -v initdb)")"
+API=127.0.0.1:18080; NBD=127.0.0.1:10899
+W="$(mktemp -d "${WORKROOT:-/tmp}/gfe.XXXXXX")"
+RUN="$HERE/out/$(date +%Y%m%d-%H%M%S)-e-$PROFILE"; mkdir -p "$RUN"
+GF_LOG="$RUN/glidefs.log"; GF_PID=""; MNT=0
+log(){ printf '\033[1;35m[E]\033[0m %s\n' "$*" >&2; }
+cleanup(){ set +e; [ "$MNT" = 1 ] && { "$PG/pg_ctl" -D "$W/m/pgdata" -m immediate stop >/dev/null 2>&1; sudo umount "$W/m" 2>/dev/null; }
+  curl -fsS -X DELETE "http://$API/api/exports/eA?purge=true" >/dev/null 2>&1
+  [ -n "$GF_PID" ] && { sudo kill "$GF_PID" 2>/dev/null; wait "$GF_PID" 2>/dev/null; }; sudo rm -rf "$W"; }
+trap cleanup EXIT INT TERM
+
+cat > "$W/gf.toml" <<EOF
+[cache]
+dir = "$W/cache"
+disk_size_gb = 20.0
+memory_size_gb = 2.0
+[storage]
+url = "memory:///"
+[servers.nbd]
+addresses = ["$NBD"]
+unix_socket = "$W/s.sock"
+api_address = "$API"
+block_size = 131072
+flush_threshold = 500
+[[servers.nbd.exports]]
+name = "eA"
+size_gb = 10
+transport = "nbd"
+EOF
+mkdir -p "$W/cache" "$W/m"
+curl -fsS "http://$API/health" >/dev/null 2>&1 && { echo "API busy"; exit 1; }
+log "start glidefs"
+sudo -b sh -c "exec env NO_COLOR=1 '$GLIDEFS' run -c '$W/gf.toml' >'$GF_LOG' 2>&1"
+for _ in $(seq 1 50); do GF_PID="$(pgrep -f "glidefs run -c $W/gf.toml"|head -1)"; [ -n "$GF_PID" ] && break; sleep 0.1; done
+for _ in $(seq 1 100); do curl -fsS "http://$API/health" >/dev/null 2>&1 && break; sleep 0.1; done
+DA="$(curl -fsS "http://$API/api/exports/eA"|jq -r .device)"
+sudo mkfs.ext4 -q -F "$DA"; sudo mount "$DA" "$W/m"; sudo chown -R "$USER:$(id -gn)" "$W/m"
+"$PG/initdb" -D "$W/m/pgdata" -A trust -U postgres --no-sync >/dev/null 2>&1
+cat >> "$W/m/pgdata/postgresql.conf" <<'P'
+listen_addresses = ''
+shared_buffers = 512MB
+autovacuum = off
+P
+"$PG/pg_ctl" -D "$W/m/pgdata" -l "$RUN/pg.log" -o "-p 5440 -k $W/m" -w start >/dev/null; MNT=1
+PSQL=("$PG/psql" -h "$W/m" -p 5440 -U postgres)
+log "load + churn (build dead tuples, autovacuum OFF)"
+"$PG/pgbench" -h "$W/m" -p 5440 -U postgres -i -q -s 50 postgres >/dev/null 2>&1
+"$PG/pgbench" -h "$W/m" -p 5440 -U postgres -t 12000 -c 8 -j 4 postgres >/dev/null 2>&1
+"${PSQL[@]}" -c "CHECKPOINT;" >/dev/null
+DEAD=$("${PSQL[@]}" -At -c "SELECT sum(n_dead_tup) FROM pg_stat_user_tables;")
+curl -fsS -X POST "http://$API/api/exports/eA/drain" >/dev/null; sleep 2
+# now turn autovacuum ON under the profile (append to conf; last value wins), reload
+if [ "$PROFILE" = aggressive ]; then
+  cat >> "$W/m/pgdata/postgresql.conf" <<'P'
+autovacuum = on
+autovacuum_vacuum_scale_factor = 0.01
+autovacuum_naptime = 10s
+autovacuum_vacuum_cost_delay = 0
+P
+else
+  cat >> "$W/m/pgdata/postgresql.conf" <<'P'
+autovacuum = on
+autovacuum_vacuum_scale_factor = 0.4
+autovacuum_naptime = 5min
+autovacuum_vacuum_cost_delay = 20ms
+autovacuum_vacuum_cost_limit = 200
+P
+fi
+"${PSQL[@]}" -c "SELECT pg_reload_conf();" >/dev/null
+L0=$(wc -l < "$GF_LOG")
+log "idle 90s under '$PROFILE' autovacuum (dead_tuples=$DEAD)"
+sleep 90
+"${PSQL[@]}" -c "CHECKPOINT;" >/dev/null
+curl -fsS -X POST "http://$API/api/exports/eA/drain" >/dev/null; sleep 3
+DIV=$(tail -n +$((L0+1)) "$GF_LOG" | sed 's/\x1b\[[0-9;]*m//g' | awk '/flush complete/{for(i=1;i<=NF;i++){n=split($i,kv,"=");if(n==2&&kv[1]=="bytes_uploaded")b+=kv[2]}}END{print b+0}')
+PK=$(tail -n +$((L0+1)) "$GF_LOG" | sed 's/\x1b\[[0-9;]*m//g' | awk '/flush complete/{for(i=1;i<=NF;i++){n=split($i,kv,"=");if(n==2&&kv[1]=="packs_uploaded")p+=kv[2]}}END{print p+0}')
+REMAIN=$("${PSQL[@]}" -At -c "SELECT sum(n_dead_tup) FROM pg_stat_user_tables;")
+printf "E[%s]: autovacuum wrote %d bytes (%.1f MB) in 120s idle; packs=%d; dead_tuples %s -> %s\n" \
+  "$PROFILE" "$DIV" "$(echo "$DIV/1048576"|bc -l)" "$PK" "$DEAD" "$REMAIN" | tee "$RUN/score.txt"
diff --git a/bench/glidefs-pg/exp-f-pgbouncer.sh b/bench/glidefs-pg/exp-f-pgbouncer.sh
new file mode 100644
index 0000000..3463d34
--- /dev/null
+++ b/bench/glidefs-pg/exp-f-pgbouncer.sh
@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# F: PgBouncer single-process vs multi-worker (so_reuseport) connection scaling.
+#
+# Validates the plan's claim that one PgBouncer process bottlenecks a many-vCPU
+# box. Postgres runs on plain disk (this is a pooler-CPU test, not storage).
+# A short SELECT-only pgbench at high client count saturates the pooler; we vary
+# the number of so_reuseport workers and read TPS.
+set -euo pipefail
+PG="$(dirname "$(command -v initdb)")"
+W="$(mktemp -d "${WORKROOT:-/tmp}/gff.XXXXXX")"
+RUN="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/out/$(date +%Y%m%d-%H%M%S)-f"; mkdir -p "$RUN"
+log(){ printf '\033[1;34m[F]\033[0m %s\n' "$*" >&2; }
+cleanup(){ set +e; pkill -f "pgbouncer .*$W" 2>/dev/null
+  "$PG/pg_ctl" -D "$W/pg" -m immediate stop >/dev/null 2>&1; rm -rf "$W"; }
+trap cleanup EXIT INT TERM
+
+command -v pgbouncer >/dev/null || { echo "pgbouncer not installed"; exit 1; }
+# Ubuntu auto-starts a systemd pgbouncer on :6432 — stop it so our test owns the port.
+sudo systemctl stop pgbouncer 2>/dev/null || true
+sudo pkill -9 pgbouncer 2>/dev/null || true; sleep 1
+CORES=$(nproc)
+"$PG/initdb" -D "$W/pg" -A trust -U postgres --no-sync >/dev/null 2>&1
+cat >> "$W/pg/postgresql.conf" <<EOF
+max_connections = 600
+shared_buffers = 512MB
+fsync = off
+listen_addresses = '127.0.0.1'
+EOF
+"$PG/pg_ctl" -D "$W/pg" -l "$RUN/pg.log" -o "-p 5440 -k $W" -w start >/dev/null
+"$PG/pgbench" -h 127.0.0.1 -p 5440 -U postgres -i -q -s 10 postgres >/dev/null 2>&1
+
+cat > "$W/pgb.ini" <<EOF
+[databases]
+* = host=127.0.0.1 port=5440 dbname=postgres
+[pgbouncer]
+listen_addr = 127.0.0.1
+listen_port = 6432
+auth_type = trust
+auth_file = $W/users.txt
+pool_mode = transaction
+default_pool_size = 64
+max_client_conn = 2000
+so_reuseport = 1
+logfile =
+pidfile =
+EOF
+printf '"postgres" ""\n' > "$W/users.txt"
+
+echo "F: PgBouncer so_reuseport scaling on $CORES vCPU (SELECT-only, -c 200)" | tee "$RUN/score.txt"
+for N in 1 2 $CORES; do
+  pkill -f "pgbouncer .*$W" 2>/dev/null || true; sleep 1
+  # foreground processes backgrounded by the shell; so_reuseport shares :6432
+  for i in $(seq 1 "$N"); do pgbouncer "$W/pgb.ini" >/dev/null 2>&1 & done
+  sleep 2
+  t=$("$PG/pgbench" -h 127.0.0.1 -p 6432 -U postgres -n -S -c 200 -j "$CORES" -T 20 postgres 2>/dev/null \
+      | grep -oE 'tps = [0-9.]+' | tail -1 | grep -oE '[0-9.]+')
+  printf "  workers=%-2s  tps=%s\n" "$N" "$t" | tee -a "$RUN/score.txt"
+done
\ No newline at end of file
diff --git a/bench/glidefs-pg/exp-multiworker.sh b/bench/glidefs-pg/exp-multiworker.sh
new file mode 100644
index 0000000..b72c807
--- /dev/null
+++ b/bench/glidefs-pg/exp-multiworker.sh
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+# DECISIVE TEST: does so_reuseport multi-worker actually scale pooler throughput?
+# Pooler-bound load (Ed25519 TLS + per-txn churn => ~400us/conn handshake, PG ~idle),
+# so the limit is pooler CPU, not PG. Pin N poolers to N cores, drive from the rest,
+# report tps AND aggregate pooler CPU (cores) for N=1,2,3.
+#   linear tps + pooler_cpu≈N  => multi-worker scales => autoscaler is a real win
+#   flat                       => it doesn't, and we kill the idea for good
+set -uo pipefail
+PG=/usr/lib/postgresql/18/bin
+W=$(mktemp -d /mnt/scratch/mw.XXXXX); cd "$W"
+DUR=${DUR:-15}; CL=${CL:-300}
+cleanup(){ pkill -f "pgbouncer $W" 2>/dev/null; "$PG/pg_ctl" -D "$W/pg" -m immediate stop >/dev/null 2>&1; rm -rf "$W"; }
+trap cleanup EXIT
+sudo systemctl stop pgbouncer 2>/dev/null || true; sudo pkill -9 pgbouncer 2>/dev/null || true; sleep 1
+
+"$PG/initdb" -D "$W/pg" -A trust -U postgres --no-sync >/dev/null 2>&1
+cat >> "$W/pg/postgresql.conf" <<EOF
+listen_addresses = '127.0.0.1'
+port = 5440
+unix_socket_directories = '$W'
+max_connections = 2000
+shared_buffers = 512MB
+fsync = off
+EOF
+"$PG/pg_ctl" -D "$W/pg" -l "$W/pg.log" -w start >/dev/null
+"$PG/pgbench" -h 127.0.0.1 -p 5440 -U postgres -i -q -s 5 postgres >/dev/null 2>&1
+openssl req -x509 -newkey ed25519 -days 1 -nodes -keyout "$W/s.key" -out "$W/s.crt" -subj /CN=pgb >/dev/null 2>&1
+chmod 600 "$W/s.key"
+printf '"postgres" ""\n' > "$W/users.txt"
+cat > "$W/pgb.ini" <<EOF
+[databases]
+* = host=127.0.0.1 port=5440 dbname=postgres
+[pgbouncer]
+listen_addr = 127.0.0.1
+listen_port = 6432
+unix_socket_dir =
+auth_type = trust
+auth_file = $W/users.txt
+pool_mode = transaction
+default_pool_size = 64
+max_client_conn = 4000
+so_reuseport = 1
+client_tls_sslmode = require
+client_tls_cert_file = $W/s.crt
+client_tls_key_file = $W/s.key
+logfile =
+pidfile =
+EOF
+
+agg_ticks(){ local s=0 v; for p in $(pgrep -f "pgbouncer $W"); do v=$(awk '{print $14+$15}' "/proc/$p/stat" 2>/dev/null||echo 0); s=$((s+v)); done; echo "$s"; }
+
+echo "=== so_reuseport multi-worker scaling | Ed25519 TLS + churn(-C) | -c$CL ${DUR}s ==="
+for N in 1 2 3; do
+  pkill -f "pgbouncer $W" 2>/dev/null; sleep 1
+  for i in $(seq 0 $((N-1))); do taskset -c "$i" pgbouncer "$W/pgb.ini" >/dev/null 2>&1 & done
+  sleep 2
+  live=$(pgrep -cf "pgbouncer $W")
+  a0=$(agg_ticks); t0=$(date +%s.%N)
+  out=$(PGSSLMODE=require taskset -c "${N}-5" "$PG/pgbench" -h 127.0.0.1 -p 6432 -U postgres \
+        -n -S -C -c "$CL" -j "$((6-N))" -T "$DUR" postgres 2>&1)
+  t1=$(date +%s.%N); a1=$(agg_ticks)
+  tps=$(echo "$out" | grep -oE 'tps = [0-9.]+' | tail -1 | grep -oE '[0-9.]+$')
+  cores=$(awk -v a="$a0" -v b="$a1" -v x="$t0" -v y="$t1" 'BEGIN{printf "%.2f",(b-a)/100/(y-x)}')
+  printf "N=%d (live=%s)  tps=%-9s  pooler_cpu=%-5s cores  per-pooler=%.2f\n" \
+    "$N" "$live" "${tps:-FAIL}" "$cores" "$(awk -v c="$cores" -v n="$N" 'BEGIN{print c/n}')"
+  pkill -f "pgbouncer $W" 2>/dev/null
+done
diff --git a/bench/glidefs-pg/exp-pooler-ceiling.sh b/bench/glidefs-pg/exp-pooler-ceiling.sh
new file mode 100644
index 0000000..98d0d48
--- /dev/null
+++ b/bench/glidefs-pg/exp-pooler-ceiling.sh
@@ -0,0 +1,88 @@
+#!/usr/bin/env bash
+# Pin ONE pgbouncer to its own core and find where it saturates, to pin X (pooler
+# CPU per txn) under the conditions that matter for Beyond: TLS termination x
+# connection churn. SELECT-only so PG is never the bottleneck. Measures tps AND the
+# pooler's actual CPU (fraction of a core) so we KNOW it's the limit, not pgbench/PG.
+#
+# pgbouncer -> core 0 (alone); postgres + pgbench -> cores 1-5.
+set -uo pipefail
+PG=/usr/lib/postgresql/18/bin
+W=$(mktemp -d /mnt/scratch/pool.XXXXX); cd "$W"
+DUR="${DUR:-20}"; CL="${CL:-100}"
+cleanup(){ pkill -f "pgbouncer $W" 2>/dev/null; "$PG/pg_ctl" -D "$W/pg" -m immediate stop >/dev/null 2>&1; rm -rf "$W"; }
+trap cleanup EXIT
+
+# --- postgres on plain disk, pinned off core 0 ---
+"$PG/initdb" -D "$W/pg" -A trust -U postgres --no-sync >/dev/null 2>&1
+cat >> "$W/pg/postgresql.conf" <<EOF
+listen_addresses = '127.0.0.1'
+port = 5440
+unix_socket_directories = '$W'
+max_connections = 800
+shared_buffers = 1GB
+fsync = off
+EOF
+taskset -c 1-5 "$PG/pg_ctl" -D "$W/pg" -l "$W/pg.log" -w start >/dev/null
+taskset -c 1-5 "$PG/pgbench" -h 127.0.0.1 -p 5440 -U postgres -i -q -s 10 postgres >/dev/null 2>&1
+
+# --- self-signed cert for pgbouncer TLS termination ---
+# CERT=ed25519 matches what beyond-pg actually issues (rcgen PKCS_ED25519, src/tls.rs);
+# CERT=rsa is the pessimistic RSA-2048 baseline. Ed25519 handshakes are ~10x cheaper.
+case "${CERT:-ed25519}" in
+  rsa) openssl req -x509 -newkey rsa:2048 -days 1 -nodes -keyout "$W/s.key" -out "$W/s.crt" -subj "/CN=pgb" >/dev/null 2>&1 ;;
+  *)   openssl req -x509 -newkey ed25519  -days 1 -nodes -keyout "$W/s.key" -out "$W/s.crt" -subj "/CN=pgb" >/dev/null 2>&1 ;;
+esac
+chmod 600 "$W/s.key"
+echo "cert: ${CERT:-ed25519}"
+printf '"postgres" ""\n' > "$W/users.txt"
+
+mk_ini(){ # $1 = tls 0/1
+cat > "$W/pgb.ini" <<EOF
+[databases]
+* = host=127.0.0.1 port=5440 dbname=postgres
+[pgbouncer]
+listen_addr = 127.0.0.1
+listen_port = 6432
+auth_type = trust
+auth_file = $W/users.txt
+pool_mode = transaction
+default_pool_size = 64
+max_client_conn = 4000
+logfile =
+pidfile =
+EOF
+if [ "$1" = 1 ]; then cat >> "$W/pgb.ini" <<EOF
+client_tls_sslmode = require
+client_tls_cert_file = $W/s.crt
+client_tls_key_file = $W/s.key
+EOF
+fi
+}
+ticks(){ awk '{print $14+$15}' "/proc/$1/stat" 2>/dev/null; }
+
+run(){ # $1 label  $2 tls  $3 pgbench-extra  $4 sslmode
+  pkill -f "pgbouncer $W" 2>/dev/null; sleep 1
+  mk_ini "$2"
+  taskset -c 0 pgbouncer "$W/pgb.ini" >/dev/null 2>&1 &
+  sleep 2
+  local pid; pid=$(pgrep -f "pgbouncer $W" | head -1)
+  [ -z "$pid" ] && { printf "%-24s START-FAIL\n" "$1"; return; }
+  local c0 t0 c1 t1 out tps cores us
+  c0=$(ticks "$pid"); t0=$(date +%s.%N)
+  out=$(PGSSLMODE="$4" taskset -c 1-5 "$PG/pgbench" -h 127.0.0.1 -p 6432 -U postgres \
+        -n -S $3 -c "$CL" -j 5 -T "$DUR" postgres 2>&1)
+  c1=$(ticks "$pid"); t1=$(date +%s.%N)
+  tps=$(echo "$out" | grep -oE 'tps = [0-9.]+' | tail -1 | grep -oE '[0-9.]+$')
+  cores=$(awk -v a="$c0" -v b="$c1" -v t0="$t0" -v t1="$t1" 'BEGIN{printf "%.2f",(b-a)/100/(t1-t0)}')
+  us=$(awk -v tps="${tps:-0}" -v c="$cores" 'BEGIN{ if(tps>0) printf "%.1f", c*1e6/tps; else print "-"}')
+  printf "%-24s tps=%-9s pooler=%-5s cores  X=%-6s us/txn\n" "$1" "${tps:-FAIL}" "$cores" "$us"
+  pkill -f "pgbouncer $W" 2>/dev/null
+}
+
+echo "=== single pgbouncer pinned to 1 core | -S | -c$CL -j5 | ${DUR}s each ==="
+dp=$(taskset -c 1-5 "$PG/pgbench" -h 127.0.0.1 -p 5440 -U postgres -n -S -c "$CL" -j 5 -T "$DUR" postgres 2>&1 | grep -oE 'tps = [0-9.]+' | tail -1 | grep -oE '[0-9.]+$')
+printf "%-24s tps=%s   (5 PG cores, no pooler)\n" "direct-PG control" "$dp"
+run "pgb  plain  persistent" 0 ""   disable
+run "pgb  TLS    persistent" 1 ""   require
+run "pgb  plain  churn(-C)"  0 "-C" disable
+run "pgb  TLS    churn(-C)"  1 "-C" require
diff --git a/bench/glidefs-pg/out/20260603-141450-smoke/glidefs.toml b/bench/glidefs-pg/out/20260603-141450-smoke/glidefs.toml
new file mode 100644
index 0000000..44d2f79
--- /dev/null
+++ b/bench/glidefs-pg/out/20260603-141450-smoke/glidefs.toml
@@ -0,0 +1,20 @@
+[cache]
+dir = "/tmp/gfpg.AqPOOe/cache"
+disk_size_gb = 20.0
+memory_size_gb = 2.0
+
+[storage]
+url = "file:///tmp/gfpg.AqPOOe/objstore"
+
+[servers.nbd]
+addresses = ["127.0.0.1:10809"]
+unix_socket = "/tmp/gfpg.AqPOOe/gf.nbd.sock"
+api_address = "127.0.0.1:8080"
+block_size = 131072
+flush_threshold = 500
+
+[[servers.nbd.exports]]
+name = "pgdata"
+size_gb = 10
+transport = "nbd"
+compaction_cooldown = 0
diff --git a/bench/glidefs-pg/out/20260603-141450-smoke/harness.conf b/bench/glidefs-pg/out/20260603-141450-smoke/harness.conf
new file mode 100644
index 0000000..df238c3
--- /dev/null
+++ b/bench/glidefs-pg/out/20260603-141450-smoke/harness.conf
@@ -0,0 +1,40 @@
+# Harness baseline — the substrate-relevant subset of
+# packer/files/postgresql/00-beyond.conf. Memory knobs are FIXED here for
+# reproducibility (the real image autotunes them via src/config.rs); the
+# harness's job is to isolate the storage knobs, so everything except the knob
+# under test is held constant. No extensions, no networking, no TLS.
+
+# fixed memory (reproducibility, not under test)
+shared_buffers = 1GB
+effective_cache_size = 3GB
+work_mem = 32MB
+maintenance_work_mem = 256MB
+max_wal_size = 8GB
+min_wal_size = 1GB
+
+# storage cost model (matches 00-beyond.conf)
+random_page_cost = 1.1
+effective_io_concurrency = 200
+
+# durability / WAL (matches 00-beyond.conf; wal_level=replica is lighter than
+# logical and constant across experiments, so it doesn't bias the comparison)
+fsync = on
+synchronous_commit = on
+full_page_writes = on
+wal_sync_method = fdatasync
+wal_compression = lz4
+wal_level = replica
+
+# checkpoints / bgwriter — the coalescing knobs (C experiments override these)
+checkpoint_timeout = 15min
+checkpoint_completion_target = 0.9
+bgwriter_delay = 100ms
+bgwriter_lru_maxpages = 1000
+
+# autovacuum (E experiments override)
+autovacuum_vacuum_scale_factor = 0.1
+autovacuum_naptime = 30s
+
+# observability for the run log
+log_checkpoints = on
+log_min_messages = warning
diff --git a/bench/glidefs-pg/out/20260603-141450-smoke/postgres.log b/bench/glidefs-pg/out/20260603-141450-smoke/postgres.log
new file mode 100644
index 0000000..9740da3
--- /dev/null
+++ b/bench/glidefs-pg/out/20260603-141450-smoke/postgres.log
@@ -0,0 +1,3 @@
+2026-06-03 14:14:52.132 PDT [1239168] LOG:  invalid value for parameter "wal_compression": "lz4"
+2026-06-03 14:14:52.132 PDT [1239168] HINT:  Available values: pglz, on, off.
+2026-06-03 14:14:52.132 PDT [1239168] FATAL:  configuration file "/tmp/gfpg.AqPOOe/mnt/pgdata/harness.conf" contains errors
diff --git a/bench/glidefs-pg/out/20260603-141623-smoke/glidefs.toml b/bench/glidefs-pg/out/20260603-141623-smoke/glidefs.toml
new file mode 100644
index 0000000..989f88a
--- /dev/null
+++ b/bench/glidefs-pg/out/20260603-141623-smoke/glidefs.toml
@@ -0,0 +1,20 @@
+[cache]
+dir = "/tmp/gfpg.Kz2A8C/cache"
+disk_size_gb = 20.0
+memory_size_gb = 2.0
+
+[storage]
+url = "file:///tmp/gfpg.Kz2A8C/objstore"
+
+[servers.nbd]
+addresses = ["127.0.0.1:10899"]
+unix_socket = "/tmp/gfpg.Kz2A8C/gf.nbd.sock"
+api_address = "127.0.0.1:18080"
+block_size = 131072
+flush_threshold = 500
+
+[[servers.nbd.exports]]
+name = "pgdata"
+size_gb = 10
+transport = "nbd"
+compaction_cooldown = 0
diff --git a/bench/glidefs-pg/qemu-bringup.sh b/bench/glidefs-pg/qemu-bringup.sh
new file mode 100755
index 0000000..20d838d
--- /dev/null
+++ b/bench/glidefs-pg/qemu-bringup.sh
@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+# Stand up the isolated QEMU benchmark VM and provision it.
+#
+# WHY a VM: the host runs the production homelab glidefs (shared handoff socket +
+# global ublk recovery), so a second glidefs can't co-run safely. A QEMU/KVM guest
+# has its own kernel (nbd/ublk), RAM, and /run — fully isolated. We run glidefs
+# INSIDE the guest and put Postgres on its nbd device.
+#
+# IMPORTANT: VM disks live on a DISK-backed dir (default /var/tmp), NOT /tmp —
+# /tmp here is a 14G tmpfs shared with the homelab; the scratch qcow2 grows to
+# many GB across runs and will ENOSPC the tmpfs (learned the hard way).
+#
+# After this prints READY, drive experiments with ssh on port 2222 (key: $VMDIR/id).
+set -euo pipefail
+VMDIR="${VMDIR:-/var/tmp/pgtune-qemu}"
+BASE="${BASE:-/var/lib/k617-vm/noble-cloudimg.img}"
+MEM="${MEM:-8192}"; CPUS="${CPUS:-6}"; SSHP="${SSHP:-2222}"
+HARNESS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+mkdir -p "$VMDIR"; cd "$VMDIR"
+SSH="ssh -i $VMDIR/id -p $SSHP -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR bench@127.0.0.1"
+SCP="scp -i $VMDIR/id -P $SSHP -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR"
+log(){ printf '\033[1;36m[bringup]\033[0m %s\n' "$*" >&2; }
+
+# re-runnable: refuse if a VM is live, else clear stale disks
+if [ -f qemu.pid ] && kill -0 "$(cat qemu.pid 2>/dev/null)" 2>/dev/null; then
+  echo "a VM is already running (pid $(cat qemu.pid)); kill it first"; exit 1
+fi
+rm -f root.qcow2 scratch.qcow2
+[ -f id ] || ssh-keygen -t ed25519 -N '' -f id -q
+log "create disks (overlay backed by $BASE; scratch on disk)"
+qemu-img create -f qcow2 -b "$BASE" -F qcow2 root.qcow2 40G >/dev/null
+qemu-img create -f qcow2 scratch.qcow2 40G >/dev/null
+cat > user-data <<EOF
+#cloud-config
+hostname: pgtune-bench
+users:
+  - name: bench
+    sudo: ALL=(ALL) NOPASSWD:ALL
+    shell: /bin/bash
+    ssh_authorized_keys: ["$(cat id.pub)"]
+ssh_pwauth: false
+EOF
+printf 'instance-id: pgtune-bench\nlocal-hostname: pgtune-bench\n' > meta-data
+cloud-localds seed.img user-data meta-data
+
+log "boot qemu (${MEM}MB ${CPUS}vcpu, ssh :$SSHP)"
+qemu-system-x86_64 -enable-kvm -m "$MEM" -smp "$CPUS" -cpu host \
+  -drive file=root.qcow2,if=virtio -drive file=scratch.qcow2,if=virtio \
+  -drive file=seed.img,if=virtio,format=raw \
+  -netdev user,id=n0,hostfwd=tcp::${SSHP}-:22 -device virtio-net-pci,netdev=n0 \
+  -display none -serial file:serial.log -pidfile qemu.pid -daemonize
+
+log "wait for ssh"
+for _ in $(seq 1 60); do $SSH true 2>/dev/null && break; sleep 3; done
+$SSH true 2>/dev/null || { tail -20 serial.log; echo "ssh never came up"; exit 1; }
+
+log "provision: scratch fs + PG18 (PGDG) + pgbouncer + glidefs + harness"
+$SSH 'set -e; sudo mkfs.ext4 -q -F /dev/vdb; sudo mkdir -p /mnt/scratch; sudo mount /dev/vdb /mnt/scratch; sudo chown bench:bench /mnt/scratch'
+$SSH 'bash -s' <<'PROV'
+set -e; export DEBIAN_FRONTEND=noninteractive
+sudo install -d /usr/share/postgresql-common/pgdg
+sudo curl -fsSL -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc https://www.postgresql.org/media/keys/ACCC4CF8.asc
+. /etc/os-release
+echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt ${VERSION_CODENAME}-pgdg main" | sudo tee /etc/apt/sources.list.d/pgdg.list >/dev/null
+sudo apt-get update -qq
+sudo apt-get install -y -qq postgresql-18 jq fio pgbouncer >/dev/null
+PROV
+$SCP /usr/local/bin/glidefs bench@127.0.0.1:/tmp/glidefs >/dev/null && $SSH 'sudo install /tmp/glidefs /usr/local/bin/glidefs'
+$SCP -r "$HARNESS" bench@127.0.0.1:/home/bench/glidefs-pg >/dev/null
+$SSH 'chmod +x /home/bench/glidefs-pg/*.sh; /usr/lib/postgresql/18/bin/postgres --version; glidefs --version 2>/dev/null||true; pgbouncer --version|head -1'
+log "READY — ssh -i $VMDIR/id -p $SSHP bench@127.0.0.1"
diff --git a/bench/glidefs-pg/qemu-runall.sh b/bench/glidefs-pg/qemu-runall.sh
new file mode 100755
index 0000000..bf4ab95
--- /dev/null
+++ b/bench/glidefs-pg/qemu-runall.sh
@@ -0,0 +1,50 @@
+#!/usr/bin/env bash
+# Drive every remaining substrate experiment with HARD GUARDRAILS:
+#   - per-run timeout (a bad config dies in minutes, never hangs for hours)
+#   - forced guest cleanup BEFORE every run (a wedged prior run can't poison the next)
+#   - each result pulled to host disk immediately (incident-proof)
+# Requires qemu-bringup.sh to have printed READY.
+set -uo pipefail
+VMDIR="${VMDIR:-/var/tmp/pgtune-qemu}"
+OUT="${OUT:-/home/jared/pgtune-results}"; mkdir -p "$OUT"
+SSH="ssh -i $VMDIR/id -p ${SSHP:-2222} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -o ConnectTimeout=10 bench@127.0.0.1"
+GENV='export PATH=/usr/lib/postgresql/18/bin:$PATH WORKROOT=/mnt/scratch; cd /home/bench/glidefs-pg'
+
+guest_clean(){
+  timeout 60 $SSH 'sudo pkill -9 -f "glidefs run" 2>/dev/null; sudo pkill -9 -f "run.sh|exp-e-fork|exp-f-pgb|exp-d2|minio" 2>/dev/null; sudo pkill -9 pgbench 2>/dev/null; sudo pkill -9 postgres 2>/dev/null; sleep 2; for e in pgdata eA eB dA dB; do curl -fsS -X DELETE "http://127.0.0.1:18080/api/exports/$e?purge=true" >/dev/null 2>&1; done; sudo umount /mnt/scratch/gf*/m* 2>/dev/null; sudo umount /mnt/scratch/gf*/mnt 2>/dev/null; sudo rm -rf /mnt/scratch/gf* 2>/dev/null; true' >/dev/null 2>&1
+}
+
+# drive <label> <timeout_s> <remote-cmd...>
+drive(){ local label="$1" tmo="$2"; shift 2
+  echo "### $label (timeout ${tmo}s) ###"
+  guest_clean
+  # double timeout: remote `timeout` + outer ssh `timeout` so a hung ssh still returns
+  timeout $((tmo+45)) $SSH "$GENV && timeout ${tmo} $*" 2>&1 | tail -4
+  $SSH "cat \$(ls -dt /home/bench/glidefs-pg/out/*/ 2>/dev/null | head -1)score.txt" > "$OUT/${label}.txt" 2>/dev/null
+  if [ -s "$OUT/${label}.txt" ]; then echo "  ✓ saved $OUT/${label}.txt"; sed 's/^/    /' "$OUT/${label}.txt"; else echo "  ✗ NO RESULT (timed out or failed)"; fi
+}
+
+echo "===== C1 backfill: cp30s (PG floor) ====="
+drive "c1-cp30s" 480 "./run.sh --label c1-cp30s --backend memory --scale 50 --txns 300000 --conf conf/c1-cp30s.conf"
+
+echo "===== C3 bgwriter (FIXED: scale20 fits in 1GB sb, 200k txns) ====="
+for v in aggressive gentle off; do
+  drive "c3-$v" 480 "./run.sh --label c3-$v --backend memory --scale 20 --txns 200000 --conf conf/c3-$v.conf"
+done
+
+echo "===== G cooldown churn (scale10 hot-overwrite, 400k txns) ====="
+drive "g-cool0" 600 "./run.sh --label g-cool0 --backend memory --scale 10 --txns 400000 --cooldown 0"
+drive "g-cool8" 600 "./run.sh --label g-cool8 --backend memory --scale 10 --txns 400000 --cooldown 8"
+
+echo "===== E autovacuum-on-CoW divergence (snapshot->fork->idle) ====="
+drive "e-aggressive" 480 "./exp-e-fork.sh aggressive"
+drive "e-throttled"  480 "./exp-e-fork.sh throttled"
+
+echo "===== F pgbouncer multi-worker (so_reuseport 1 vs N) ====="
+drive "f-pgbouncer" 300 "./exp-f-pgbouncer.sh"
+
+echo "===== D2 cold-from-MinIO VACUUM (maintenance_io_concurrency) ====="
+drive "d2-coldfork" 1500 "./exp-d2-coldfork.sh"
+
+guest_clean
+echo "ALL_EXPERIMENTS_DONE — results in $OUT"
diff --git a/bench/glidefs-pg/run.sh b/bench/glidefs-pg/run.sh
new file mode 100755
index 0000000..9deec22
--- /dev/null
+++ b/bench/glidefs-pg/run.sh
@@ -0,0 +1,286 @@
+#!/usr/bin/env bash
+# GlideFS × Postgres substrate-tuning benchmark harness.
+#
+# Stands up a real GlideFS block device (against an in-memory / local-file /
+# MinIO object store — never real S3), puts Postgres' data dir on it, runs a
+# fixed pgbench workload, and scrapes GlideFS's own per-export metrics before
+# and after so each tuning knob can be scored on S3 write cost / coalescing.
+#
+# This is the measure-first rig from plans/wal-recycle-off-*.md. Every knob in
+# that plan is validated here before any default changes.
+#
+#   sudo is used for: glidefs (needs CAP_SYS_ADMIN to create /dev/nbdN via
+#   netlink), mkfs.ext4, mount/umount. Postgres itself runs as $USER.
+#
+# Usage:
+#   bench/glidefs-pg/run.sh [--conf FILE] [--label NAME] [--backend memory|file]
+#                           [--scale N] [--duration SECS] [--clients N]
+#                           [--cooldown N] [--transport nbd|ublk] [--keep]
+#
+# Output: a timestamped dir under bench/glidefs-pg/out/ with the full before/
+# after Prometheus snapshots, per-export JSON, pgbench log, and a delta table.
+set -euo pipefail
+
+# --------------------------------------------------------------------------
+# Config (env-overridable; flags below take precedence)
+# --------------------------------------------------------------------------
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+CONF="${CONF:-$HERE/conf/baseline.conf}"
+LABEL="${LABEL:-baseline}"
+BACKEND="${BACKEND:-file}"          # file:// (default, no OOM) | memory:// | minio
+SCALE="${SCALE:-25}"                # pgbench -i -s  (~375 MB at 25)
+DURATION="${DURATION:-120}"         # pgbench -T seconds (time-based; default)
+TXNS="${TXNS:-}"                     # if set: fixed TOTAL transactions (work-normalized) — beats -T for A/B
+CLIENTS="${CLIENTS:-8}"
+JOBS="${JOBS:-4}"
+COOLDOWN="${COOLDOWN:-0}"           # GlideFS compaction_cooldown (G experiment)
+TRANSPORT="${TRANSPORT:-nbd}"       # nbd (default, no extra cfg) | ublk
+EXPORT_NAME="${EXPORT_NAME:-pgdata}"
+SIZE_GB="${SIZE_GB:-10}"
+BLOCK_SIZE="${BLOCK_SIZE:-131072}"
+# Distinct from the homelab's production glidefs (api :9113). Stay off common ports.
+API="${API:-127.0.0.1:18080}"
+NBD_ADDR="${NBD_ADDR:-127.0.0.1:10899}"
+KEEP=0
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --conf) CONF="$2"; shift 2;;
+    --label) LABEL="$2"; shift 2;;
+    --backend) BACKEND="$2"; shift 2;;
+    --scale) SCALE="$2"; shift 2;;
+    --duration) DURATION="$2"; shift 2;;
+    --txns) TXNS="$2"; shift 2;;
+    --clients) CLIENTS="$2"; shift 2;;
+    --cooldown) COOLDOWN="$2"; shift 2;;
+    --transport) TRANSPORT="$2"; shift 2;;
+    --keep) KEEP=1; shift;;
+    *) echo "unknown arg: $1" >&2; exit 2;;
+  esac
+done
+
+# --------------------------------------------------------------------------
+# Paths / tools
+# --------------------------------------------------------------------------
+GLIDEFS="${GLIDEFS:-$(command -v glidefs || echo /usr/local/bin/glidefs)}"
+PG_BINDIR="$(dirname "$(command -v initdb)")"
+TS="$(date +%Y%m%d-%H%M%S)"
+RUN="$HERE/out/${TS}-${LABEL}"
+WORK="$(mktemp -d "${WORKROOT:-/tmp}/gfpg.XXXXXX")"
+CACHE_DIR="$WORK/cache"
+OBJ_DIR="$WORK/objstore"
+MNT="$WORK/mnt"
+PGDATA="$MNT/pgdata"
+SOCK="$WORK/gf.nbd.sock"
+TOML="$WORK/glidefs.toml"
+GF_LOG="$RUN/glidefs.log"   # in the persistent run dir so it survives cleanup
+mkdir -p "$RUN" "$CACHE_DIR" "$OBJ_DIR" "$MNT"
+
+GF_PID=""
+DEVICE=""
+PG_UP=0
+MOUNTED=0
+
+log() { printf '\033[1;36m[harness]\033[0m %s\n' "$*" >&2; }
+die() { printf '\033[1;31m[harness] FATAL:\033[0m %s\n' "$*" >&2; exit 1; }
+
+# --------------------------------------------------------------------------
+# Cleanup (reverse order; each step guarded)
+# --------------------------------------------------------------------------
+cleanup() {
+  set +e
+  [[ "$PG_UP" == 1 ]] && "$PG_BINDIR/pg_ctl" -D "$PGDATA" -m immediate stop >/dev/null 2>&1
+  [[ "$MOUNTED" == 1 ]] && sudo umount "$MNT" >/dev/null 2>&1
+  # Release the export -> unregisters the kernel device.
+  curl -fsS -X DELETE "http://$API/api/exports/$EXPORT_NAME?purge=true" >/dev/null 2>&1
+  if [[ -n "$GF_PID" ]]; then sudo kill "$GF_PID" >/dev/null 2>&1; wait "$GF_PID" 2>/dev/null; fi
+  if [[ "$KEEP" == 1 ]]; then
+    log "kept work dir: $WORK"
+  else
+    sudo rm -rf "$WORK" >/dev/null 2>&1
+  fi
+}
+trap cleanup EXIT INT TERM
+
+# --------------------------------------------------------------------------
+# 1. Render glidefs.toml for the chosen backend
+# --------------------------------------------------------------------------
+case "$BACKEND" in
+  memory) STORE_URL="memory:///" ;;
+  file)   STORE_URL="file://$OBJ_DIR" ;;
+  minio)  STORE_URL="${MINIO_URL:?set MINIO_URL=s3://bucket/prefix and AWS_* + AWS_ENDPOINT_URL}" ;;
+  *) die "unknown backend: $BACKEND" ;;
+esac
+
+cat > "$TOML" <<EOF
+[cache]
+dir = "$CACHE_DIR"
+disk_size_gb = 20.0
+memory_size_gb = 2.0
+
+[storage]
+url = "$STORE_URL"
+
+[servers.nbd]
+addresses = ["$NBD_ADDR"]
+unix_socket = "$SOCK"
+api_address = "$API"
+block_size = $BLOCK_SIZE
+flush_threshold = 500
+
+[[servers.nbd.exports]]
+name = "$EXPORT_NAME"
+size_gb = $SIZE_GB
+transport = "$TRANSPORT"
+compaction_cooldown = $COOLDOWN
+EOF
+if [[ "$TRANSPORT" == "ublk" ]]; then
+  cat >> "$TOML" <<EOF
+
+[servers.ublk]
+nr_queues = 4
+EOF
+fi
+cp "$TOML" "$RUN/glidefs.toml"
+
+# --------------------------------------------------------------------------
+# 2. Start glidefs (root; background)
+# --------------------------------------------------------------------------
+# Safety: never attach to a pre-existing server (e.g. the homelab's production
+# glidefs). The API port must be free before we start our own instance.
+if curl -fsS "http://$API/health" >/dev/null 2>&1; then
+  die "something is already serving $API — refusing to run (set API=host:port to a free port)"
+fi
+log "starting glidefs (backend=$BACKEND transport=$TRANSPORT cooldown=$COOLDOWN)"
+sudo -b sh -c "exec env NO_COLOR=1 '$GLIDEFS' run -c '$TOML' >'$GF_LOG' 2>&1"
+# find the glidefs pid (sudo -b backgrounds a subshell; locate the real process)
+for _ in $(seq 1 50); do
+  GF_PID="$(pgrep -f "glidefs run -c $TOML" | head -1 || true)"
+  [[ -n "$GF_PID" ]] && break
+  sleep 0.1
+done
+[[ -n "$GF_PID" ]] || { cat "$GF_LOG" >&2; die "glidefs failed to start"; }
+
+# wait for API
+for _ in $(seq 1 100); do
+  curl -fsS "http://$API/health" >/dev/null 2>&1 && break
+  sleep 0.1
+done
+curl -fsS "http://$API/health" >/dev/null 2>&1 || { cat "$GF_LOG" >&2; die "glidefs API not ready"; }
+
+# --------------------------------------------------------------------------
+# 3. Attach device (export is defined in the config, registered at startup;
+#    PUT is a harmless idempotent confirm that also covers older builds)
+# --------------------------------------------------------------------------
+log "ensuring export $EXPORT_NAME (${SIZE_GB}GB)"
+curl -fsS -X PUT "http://$API/api/exports/$EXPORT_NAME" \
+  -H 'Content-Type: application/json' \
+  -d "{\"size_gb\":$SIZE_GB,\"transport\":\"$TRANSPORT\",\"block_size\":$BLOCK_SIZE,\"compaction_cooldown\":$COOLDOWN}" \
+  >/dev/null 2>&1 || true
+
+for _ in $(seq 1 100); do
+  DEVICE="$(curl -fsS "http://$API/api/exports/$EXPORT_NAME" | jq -r '.device // empty')"
+  [[ -n "$DEVICE" && -b "$DEVICE" ]] && break
+  sleep 0.1
+done
+[[ -n "$DEVICE" && -b "$DEVICE" ]] || { cat "$GF_LOG" >&2; die "no block device for export"; }
+log "device: $DEVICE"
+
+# --------------------------------------------------------------------------
+# 4. ext4 + mount + hand to $USER
+# --------------------------------------------------------------------------
+log "mkfs.ext4 + mount"
+sudo mkfs.ext4 -q -F "$DEVICE"
+sudo mount "$DEVICE" "$MNT"; MOUNTED=1
+sudo chown -R "$USER:$(id -gn)" "$MNT"
+
+# --------------------------------------------------------------------------
+# 5. initdb + apply baseline + candidate conf + start
+# --------------------------------------------------------------------------
+log "initdb"
+"$PG_BINDIR/initdb" -D "$PGDATA" -A trust -U postgres --no-sync >/dev/null 2>&1 \
+  || die "initdb failed"
+# Layer the harness baseline + the candidate conf under test.
+{ echo "include_if_exists = 'harness.conf'"; } >> "$PGDATA/postgresql.conf"
+cat "$HERE/conf/baseline.conf" > "$PGDATA/harness.conf"
+if [[ "$CONF" != "$HERE/conf/baseline.conf" ]]; then
+  echo "# ---- candidate overlay: $(basename "$CONF") ----" >> "$PGDATA/harness.conf"
+  cat "$CONF" >> "$PGDATA/harness.conf"
+fi
+cp "$PGDATA/harness.conf" "$RUN/harness.conf"
+
+"$PG_BINDIR/pg_ctl" -D "$PGDATA" -l "$RUN/postgres.log" \
+  -o "-p 5440 -k $MNT" -w start >/dev/null 2>&1 || { cat "$RUN/postgres.log" >&2; die "pg start failed"; }
+PG_UP=1
+PSQL=("$PG_BINDIR/psql" -h "$MNT" -p 5440 -U postgres)
+PGBENCH=("$PG_BINDIR/pgbench" -h "$MNT" -p 5440 -U postgres)
+
+# --------------------------------------------------------------------------
+# 6. Workload + flush accounting
+#
+# GlideFS's Prometheus s3_* counters are NOT wired in this build (they read 0),
+# but the flush log is authoritative: each "flush complete" line carries
+# packs_uploaded / bytes_uploaded / blocks_claimed / blocks_cross_deduped. We
+# force a drain (synchronous flush of all dirty blocks) so the whole run's
+# writes are accounted, then sum the flush events in the run window.
+# guest_bytes_written_total / guest_write_ops_total ARE wired → write-amp + coalescing.
+# --------------------------------------------------------------------------
+getm() { curl -fsS "http://$API/metrics" 2>/dev/null | sed -n "s/^glidefs_$1{[^}]*} //p" | head -1; }
+drain() { curl -fsS -X POST "http://$API/api/exports/$EXPORT_NAME/drain" >/dev/null 2>&1 || true; }
+
+log "pgbench init (scale=$SCALE)"
+"${PGBENCH[@]}" -i -q -s "$SCALE" postgres >/dev/null 2>&1 || die "pgbench init failed"
+"${PSQL[@]}" -c "CHECKPOINT;" >/dev/null 2>&1; drain; sleep 2
+
+# baseline markers (after the init flush is drained)
+GB0=$(getm guest_bytes_written_total); GO0=$(getm guest_write_ops_total)
+L0=$(wc -l < "$GF_LOG")
+
+# Fixed-transaction (work-normalized) beats time-based for A/B: every config does
+# the SAME work, so S3-byte deltas reflect efficiency, not throughput.
+if [[ -n "$TXNS" ]]; then WL=(-t "$(( TXNS / CLIENTS ))"); WLDESC="t=$TXNS"; else WL=(-T "$DURATION"); WLDESC="T=${DURATION}s"; fi
+log "pgbench run ($WLDESC c=$CLIENTS j=$JOBS)"
+"${PGBENCH[@]}" "${WL[@]}" -c "$CLIENTS" -j "$JOBS" -P 15 postgres \
+  > "$RUN/pgbench.log" 2>&1 || { cat "$RUN/pgbench.log" >&2; die "pgbench run failed"; }
+"${PSQL[@]}" -c "CHECKPOINT;" >/dev/null 2>&1; drain; sleep 3
+
+GB1=$(getm guest_bytes_written_total); GO1=$(getm guest_write_ops_total)
+curl -fsS "http://$API/metrics" 2>/dev/null | grep -F "$EXPORT_NAME" > "$RUN/metrics.prom" || true
+
+# --------------------------------------------------------------------------
+# 7. Scoreboard — sum the run-window flush events from the glidefs log
+# --------------------------------------------------------------------------
+log "computing scoreboard"
+tps="$(grep -Eo 'tps = [0-9.]+' "$RUN/pgbench.log" | tail -1 | grep -Eo '[0-9.]+$' || echo 0)"
+tail -n +$((L0+1)) "$GF_LOG" | sed 's/\x1b\[[0-9;]*m//g' | awk \
+  -v gb="$(( ${GB1:-0} - ${GB0:-0} ))" -v go="$(( ${GO1:-0} - ${GO0:-0} ))" -v tps="$tps" \
+  -v lbl="$LABEL" -v conf="$(basename "$CONF")" -v scale="$SCALE" -v dur="$DURATION" \
+  -v cd="$COOLDOWN" -v be="$BACKEND" '
+  /flush complete/ {
+    for (i=1;i<=NF;i++) { n=split($i,kv,"=");
+      if (n==2) {
+        if (kv[1]=="packs_uploaded") packs+=kv[2];
+        else if (kv[1]=="bytes_uploaded") bytes+=kv[2];
+        else if (kv[1]=="blocks_claimed") blocks+=kv[2];
+        else if (kv[1]=="blocks_cross_deduped") xd+=kv[2];
+      } }
+    flushes++;
+  }
+  END {
+    wamp = (gb>0)? bytes/gb : 0;
+    coal = (packs>0)? go/packs : 0;
+    bpp  = (packs>0)? blocks/packs : 0;
+    printf "================ %s ================\n", lbl;
+    printf "  conf=%s backend=%s scale=%s dur=%ss cooldown=%s\n", conf, be, scale, dur, cd;
+    printf "  %-26s %15d\n", "S3 write bytes",       bytes;
+    printf "  %-26s %15d\n", "S3 packs (PUTs)",      packs;
+    printf "  %-26s %15d\n", "flush cycles",         flushes;
+    printf "  %-26s %15d\n", "blocks flushed",       blocks;
+    printf "  %-26s %15d\n", "blocks cross-deduped", xd;
+    printf "  %-26s %15d\n", "guest bytes written",  gb;
+    printf "  %-26s %15d\n", "guest write ops",      go;
+    printf "  %-26s %15.3f  (S3 bytes / guest bytes; LOWER better)\n", "write-amp", wamp;
+    printf "  %-26s %15.1f  (blocks / pack)\n", "pack fill", bpp;
+    printf "  %-26s %15s\n", "pgbench tps", tps;
+  }' | tee "$RUN/score.txt"
+log "results: $RUN/score.txt"
diff --git a/mise.toml b/mise.toml
index 8518349..5a794f7 100644
--- a/mise.toml
+++ b/mise.toml
@@ -172,6 +172,24 @@ set -euo pipefail
 exec packer/scripts/bless.sh --image "${image_path}" --name "${base_name}"
 '''
 
+[tasks."bench:substrate"]
+description = "GlideFS x Postgres substrate-tuning benchmark (measure-first rig)"
+usage = '''
+flag "--conf <conf>" help="Candidate postgresql.conf overlay" default="bench/glidefs-pg/conf/baseline.conf"
+flag "--label <label>" help="Run label" default="baseline"
+flag "--backend <backend>" help="Object store: file|memory|minio" default="file"
+flag "--scale <scale>" help="pgbench scale" default="25"
+flag "--duration <duration>" help="pgbench seconds" default="120"
+flag "--cooldown <cooldown>" help="GlideFS compaction_cooldown" default="0"
+'''
+run = '''
+#!/usr/bin/env bash
+set -euo pipefail
+exec bench/glidefs-pg/run.sh \
+  --conf "${conf}" --label "${label}" --backend "${backend}" \
+  --scale "${scale}" --duration "${duration}" --cooldown "${cooldown}"
+'''
+
 [tasks."publish:image"]
 description = "Publish Postgres rootfs image to S3"
 usage = '''
diff --git a/packer/files/pgbouncer/pgbouncer.ini b/packer/files/pgbouncer/pgbouncer.ini
index f18eb30..c06fd8b 100644
--- a/packer/files/pgbouncer/pgbouncer.ini
+++ b/packer/files/pgbouncer/pgbouncer.ini
@@ -15,10 +15,10 @@ auth_type = scram-sha-256
 auth_user = pgbouncer
 auth_query = SELECT username, password FROM pgbouncer.get_auth($1)
 
-# Transaction pooling — the default for managed Postgres
+# Transaction pooling — the default for managed Postgres.
+# default_pool_size / max_client_conn are appended at boot by beyond-pg, scaled to
+# the VM's vCPU/RAM (config.rs::pgbouncer_ini). A 64-vCPU box needs far more than 20.
 pool_mode = transaction
-default_pool_size = 20
-max_client_conn = 100
 
 # Connection hygiene
 server_idle_timeout = 600
@@ -32,3 +32,9 @@ ignore_startup_parameters = extra_float_digits,search_path
 
 logfile =
 pidfile =
+
+# No per-process unix socket or pidfile: beyond-pg runs 1..N so_reuseport workers that
+# share the TCP :5432 port, and pgbouncer defaults unix_socket_dir to /tmp — leaving it
+# set would make every worker collide on /tmp/.s.PGSQL.5432. Clients use TCP :5432; the
+# direct Postgres path is :5433.
+unix_socket_dir =
diff --git a/packer/files/postgresql/00-beyond.conf b/packer/files/postgresql/00-beyond.conf
index aaceab6..9f053e8 100644
--- a/packer/files/postgresql/00-beyond.conf
+++ b/packer/files/postgresql/00-beyond.conf
@@ -71,10 +71,20 @@ synchronous_commit = on
 # ---------------------------------------------------------------------------
 # WAL and checkpoints — tuned for GlideFS 128KB blocks / 5s flush window
 # ---------------------------------------------------------------------------
+# full_page_writes MUST stay on: GlideFS is CoW toward S3 but OVERWRITES IN PLACE
+# on the local-SSD write-back cache, so an 8 KiB page write into a present 128 KiB
+# block can tear on power loss. (The ZFS "FPW off on CoW" trick does NOT transfer.)
 full_page_writes = on
 wal_sync_method = fdatasync
+# Keep lz4, do NOT switch to zstd: GlideFS already LZ4-compresses every block at
+# flush, so pre-zstd'ing WAL yields high-entropy blocks it can't compress further.
+# Measured: zstd = +12.5% S3 write bytes at equal work (bench/glidefs-pg/RESULTS.md).
 wal_compression = lz4
-checkpoint_timeout = 15min
+# Fewer/larger checkpoints = fewer full-page-image bursts = far less WAL→S3.
+# Measured on PG18-on-GlideFS (bench/glidefs-pg, fixed 300k txns): a frequent vs
+# rare checkpoint cadence wrote 2.9x more S3 bytes (635 vs 216 MB) AND ran slower.
+# 30min halves the checkpoint rate vs 15min; recovery stays bounded by max_wal_size.
+checkpoint_timeout = 30min
 max_wal_size = 8GB
 min_wal_size = 1GB
 checkpoint_completion_target = 0.9
diff --git a/src/boot.rs b/src/boot.rs
index d8a776d..e756b47 100644
--- a/src/boot.rs
+++ b/src/boot.rs
@@ -771,7 +771,10 @@ fn write_config_files(cfg: &MmdsConfig, tls: &crate::tls::TlsConfig) -> Result<(
     if let Some(parent) = Path::new(PGBOUNCER_INI_PATH).parent() {
         std::fs::create_dir_all(parent)?;
     }
-    write_atomic(Path::new(PGBOUNCER_INI_PATH), &config::pgbouncer_ini(tls))?;
+    write_atomic(
+        Path::new(PGBOUNCER_INI_PATH),
+        &config::pgbouncer_ini(tls, cfg.ram_bytes, cfg.vcpus),
+    )?;
 
     Ok(())
 }
diff --git a/src/children.rs b/src/children.rs
index 5084d49..e14edde 100644
--- a/src/children.rs
+++ b/src/children.rs
@@ -154,6 +154,32 @@ pub fn read_starttime(pid: u32) -> std::io::Result<u64> {
         .map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))
 }
 
+/// Read `/proc/<pid>/stat` CPU time = field 14 (utime) + field 15 (stime), in
+/// clock ticks. Used by the PgBouncer scaler to sample per-worker CPU: sample
+/// twice, delta / SC_CLK_TCK / elapsed_secs = cores consumed.
+///
+/// Same robust parse as [`read_starttime`]: split after the last `)` so a comm
+/// with spaces/parens can't shift the field indices. After `") "`, field 3
+/// (state) is idx 0, so utime (field 14) is idx 11 and stime (field 15) is idx 12.
+pub fn read_proc_cpu_ticks(pid: u32) -> std::io::Result<u64> {
+    let stat = std::fs::read_to_string(format!("/proc/{pid}/stat"))?;
+    let close = stat
+        .rfind(')')
+        .ok_or_else(|| std::io::Error::new(std::io::ErrorKind::InvalidData, "no `)` in stat"))?;
+    let after = stat
+        .get(close + 2..)
+        .ok_or_else(|| std::io::Error::new(std::io::ErrorKind::InvalidData, "stat truncated"))?;
+    let fields: Vec<&str> = after.split_whitespace().collect();
+    let parse = |idx: usize| -> std::io::Result<u64> {
+        fields
+            .get(idx)
+            .ok_or_else(|| std::io::Error::new(std::io::ErrorKind::InvalidData, "stat too short"))?
+            .parse::<u64>()
+            .map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))
+    };
+    Ok(parse(11)? + parse(12)?)
+}
+
 #[allow(dead_code)] // exercised via tests + phase 5b adopt path
 fn pidfd_open(pid: i32) -> std::io::Result<OwnedFd> {
     // SAFETY: SYS_pidfd_open with flags=0 returns either a new owned fd or
diff --git a/src/config.rs b/src/config.rs
index 6b68a06..0fadbf1 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -38,11 +38,70 @@ pub const PGBOUNCER_INI_BASE: &str = include_str!("../packer/files/pgbouncer/pgb
 /// `client_tls_ca_file` is set when a CA is available (platform). It enables
 /// mTLS opt-in: an operator can add `client_tls_sslmode = verify-full` in a
 /// custom override and clients chaining to the per-app CA are authenticated.
-pub fn pgbouncer_ini(tls: &crate::tls::TlsConfig) -> String {
+/// PgBouncer server-side pool size, scaled to the box. ~3 server connections per
+/// vCPU is enough query concurrency for SSD/GlideFS-backed Postgres; floored at 16
+/// for tiny boxes, capped at 256, and always well under `max_connections`.
+///
+/// The old hardcoded `default_pool_size = 20` was wrong for the vertical-scaling
+/// target: a 64-vCPU box choked on a 20-deep pool. (The pooler *process* isn't the
+/// bottleneck at small core counts — measured in bench/glidefs-pg's F-test, identical
+/// tps for 1/2/6 so_reuseport workers on 6 vCPU — but the pool SIZE must track the box.
+/// so_reuseport multi-worker is a separate, larger change that needs a many-core host
+/// to justify; pool sizing is the clear, low-risk win.)
+pub fn pgbouncer_pool_size(vcpus: u32) -> u64 {
+    ((vcpus as u64) * 3).clamp(16, 256)
+}
+
+/// CEILING on PgBouncer so_reuseport worker processes. PgBouncer is single-threaded —
+/// one process saturates one core — so on a many-vCPU box a single pooler caps
+/// throughput; so_reuseport lets N processes share `:5432` with the kernel balancing
+/// connections.
+///
+/// This is a *cap*, not a fixed count. The supervisor scales the LIVE worker count
+/// reactively (supervisor.rs `PgbScaler`) from 1 when idle up to this ceiling under
+/// sustained pooler-CPU saturation, and reaps back down when cool — so a scaled-to-
+/// zero box runs exactly one pooler and only a genuinely busy box spends more cores.
+/// That tracks *usage*, not the allotment (idle boxes don't carry peak-provisioned
+/// processes into their Firecracker snapshot).
+///
+/// Cap ≈ 1 worker per 4 vCPU (clamp 1..8): at the ceiling the pooler tier draws ≤1/4
+/// of the box, leaving ≥3/4 for Postgres so the pooler can't starve it. Sizing from
+/// `bench/glidefs-pg` measurements: one pooler core tops at ~56-71k tps (persistent)
+/// vs ~19k tps per Postgres core for cheap reads, so the pooler becomes the bottleneck
+/// only around ~3-4 PG cores — hence roughly one pooler per few cores.
+pub fn pgbouncer_max_workers(vcpus: u32) -> usize {
+    ((vcpus as usize) / 4).clamp(1, 8)
+}
+
+/// pgbouncer.ini: static base + boot-computed pool sizing + TLS termination keys.
+pub fn pgbouncer_ini(tls: &crate::tls::TlsConfig, ram_bytes: u64, vcpus: u32) -> String {
     let mut out = String::from(PGBOUNCER_INI_BASE);
     if !out.ends_with('\n') {
         out.push('\n');
     }
+    // Pool sizing — scaled to VM resources (the base .ini no longer hardcodes these).
+    let pool = pgbouncer_pool_size(vcpus);
+    let ram_mb = ram_bytes / (1024 * 1024);
+    // FULL pool per worker, NOT pool/max_workers. `default_pool_size` is a per-worker
+    // CEILING, not a reservation: in steady state the total server connections track
+    // *query* demand (~`pool`), independent of worker count, because the scaler adds
+    // so_reuseport workers for *handshake/CPU* load (the single-threaded pooler core
+    // saturates terminating TLS — measured ~2.4k conns/s/core) not for query concurrency.
+    // Dividing by the cap under-provisioned the common scaled-to-zero case (1 live worker
+    // on a 64-vCPU box got pool/8 = 24 instead of 192). `max_connections`
+    // (`pool + vcpus*2 + 10` in tuning_conf_boot) is the real backstop, and pgbouncer
+    // QUEUES clients (doesn't error) if the pool is ever exhausted. Floor already ≥16.
+    let per_worker_pool = pool;
+    // Client connections are cheap (a few KB each), so keep the per-worker client cap
+    // GENEROUS and undivided — one live worker can absorb the clients while the scaler
+    // (which reacts on a ~5s tick) brings up more workers to add CPU, not capacity.
+    let max_client_conn = (pool * 25).clamp(200, (ram_mb * 4).max(200));
+    out.push_str("\n# Pool sizing — generated by beyond-pg from VM resources.\n");
+    out.push_str(&format!("default_pool_size = {per_worker_pool}\n"));
+    out.push_str(&format!("max_client_conn = {max_client_conn}\n"));
+    // so_reuseport ALWAYS on (even at 1 live worker) so the supervisor can add/reap
+    // workers reactively with no config change; the kernel load-balances connections.
+    out.push_str("so_reuseport = 1\n");
     out.push_str("\n# TLS — termination for the public 5432 port (generated by beyond-pg).\n");
     out.push_str("client_tls_sslmode = allow\n");
     out.push_str(&format!("client_tls_cert_file = {}\n", tls.cert.display()));
@@ -57,12 +116,11 @@ pub fn pgbouncer_ini(tls: &crate::tls::TlsConfig) -> String {
 // Generated: 01-tuning.conf  (postmaster-context — restart required)
 // ---------------------------------------------------------------------------
 
-const PGBOUNCER_POOL_SIZE: u64 = 20; // matches pgbouncer.ini default_pool_size
-
 /// Postmaster-context parameters — require a Postgres restart to take effect.
 /// Written once at boot; not touched by the memory watcher.
 pub fn tuning_conf_boot(ram_bytes: u64, vcpus: u32) -> String {
     let ram_mb = ram_bytes / (1024 * 1024);
+    let pool_size = pgbouncer_pool_size(vcpus); // server connections PgBouncer opens
     let vcpus = vcpus as u64;
 
     // max_connections: PgBouncer pool + direct-path headroom + reserved slots.
@@ -72,7 +130,7 @@ pub fn tuning_conf_boot(ram_bytes: u64, vcpus: u32) -> String {
     // Floor 100 so tiny VMs leave room for ETL on the direct :5433 path.
     // Ceiling ram_mb/50 (~10 MB/connection overhead) prevents OOM.
     // Ref: PostgreSQL wiki "Number Of Database Connections"; Mattermost PgBouncer study
-    let max_connections = (PGBOUNCER_POOL_SIZE + vcpus * 2 + 10).clamp(100, (ram_mb / 50).max(100));
+    let max_connections = (pool_size + vcpus * 2 + 10).clamp(100, (ram_mb / 50).max(100));
 
     // shared_buffers: 25% of RAM, floor 128MB.
     // Ref: PostgreSQL docs §20.4; pganalyze shared_buffers benchmark (2024) —
@@ -132,6 +190,7 @@ pub fn tuning_conf_boot(ram_bytes: u64, vcpus: u32) -> String {
 /// virtio-mem hotplug changes the amount of visible RAM.
 pub fn tuning_conf_adaptive(ram_bytes: u64, vcpus: u32) -> String {
     let ram_mb = ram_bytes / (1024 * 1024);
+    let pool_size = pgbouncer_pool_size(vcpus); // keep work_mem in step with pool depth
     let vcpus = vcpus as u64;
 
     // effective_cache_size: 75% of RAM (shared_buffers + OS page cache estimate).
@@ -154,7 +213,7 @@ pub fn tuning_conf_adaptive(ram_bytes: u64, vcpus: u32) -> String {
     // Floor 32MB so even tiny VMs get something useful for ANN searches.
     // Tune empirically: raise log_temp_files threshold and monitor spills.
     // Ref: pganalyze "The surprising logic of Postgres work_mem" (2024)
-    let work_mem_mb = (ram_mb / 2 / (PGBOUNCER_POOL_SIZE * 3)).max(32);
+    let work_mem_mb = (ram_mb / 2 / (pool_size * 3)).max(32);
 
     // max_parallel_*_per_gather: vcpus/2 with ceiling 4 for OLTP safety; benchmarks
     // show gains up to ~8 workers on large tables before diminishing returns.
@@ -184,7 +243,17 @@ pub fn tuning_conf_adaptive(ram_bytes: u64, vcpus: u32) -> String {
 pub const DURABILITY_CONF_EPHEMERAL: &str = "# Generated by beyond-pg. Present only on ephemeral volumes.\n\
      # synchronous_commit=off gives 5-10x faster commits; ~10ms data-loss\n\
      # window on crash is irrelevant on a throwaway volume.\n\
-     synchronous_commit = off\n";
+     synchronous_commit = off\n\
+     # Throttle autovacuum on ephemeral/fork volumes. Measured on PG18-on-GlideFS\n\
+     # (bench/glidefs-pg): aggressive autovacuum on an idle fork wrote 306 MB of CoW\n\
+     # divergence in 90s vs 0.1 MB throttled (~2240x). A throwaway volume is gone\n\
+     # before bloat matters, but every vacuumed page is a new block GlideFS uploads.\n\
+     # Wraparound protection still runs (autovacuum stays on). Durable primaries keep\n\
+     # the aggressive default (00-beyond.conf) where bloat control wins.\n\
+     autovacuum_vacuum_scale_factor = 0.4\n\
+     autovacuum_naptime = 5min\n\
+     autovacuum_vacuum_cost_delay = 20ms\n\
+     autovacuum_vacuum_cost_limit = 200\n";
 
 // ---------------------------------------------------------------------------
 // Atomic write helper
@@ -412,11 +481,13 @@ mod tests {
             key: PathBuf::from("/run/beyond/tls/key.pem"),
             ca: Some(PathBuf::from("/run/beyond/tls/ca.pem")),
         };
-        let out = pgbouncer_ini(&tls);
+        let out = pgbouncer_ini(&tls, 8 * 1024 * 1024 * 1024, 4);
         assert!(
             out.starts_with(PGBOUNCER_INI_BASE),
             "must preserve base config"
         );
+        assert!(out.contains("default_pool_size = 16")); // 4 vCPU -> floor 16
+        assert!(out.contains("max_client_conn = "));
         assert!(out.contains("client_tls_sslmode = allow"));
         assert!(out.contains("client_tls_cert_file = /run/beyond/tls/cert.pem"));
         assert!(out.contains("client_tls_key_file = /run/beyond/tls/key.pem"));
@@ -431,7 +502,7 @@ mod tests {
             key: PathBuf::from("/var/lib/postgresql/18/main/beyond/server.key"),
             ca: None,
         };
-        let out = pgbouncer_ini(&tls);
+        let out = pgbouncer_ini(&tls, 8 * 1024 * 1024 * 1024, 4);
         assert!(out.contains("client_tls_sslmode = allow"));
         assert!(out.contains("client_tls_cert_file"));
         assert!(
@@ -440,6 +511,49 @@ mod tests {
         );
     }
 
+    #[test]
+    fn pgbouncer_scales_with_box() {
+        // worker CEILING (supervisor scales the live count 1..this): ~1 per 4 vCPU, cap 8.
+        assert_eq!(pgbouncer_max_workers(2), 1);
+        assert_eq!(pgbouncer_max_workers(8), 2);
+        assert_eq!(pgbouncer_max_workers(16), 4);
+        assert_eq!(pgbouncer_max_workers(64), 8);
+        assert_eq!(pgbouncer_max_workers(256), 8); // capped
+
+        let tls = TlsConfig {
+            source: TlsSource::SelfSigned,
+            cert: PathBuf::from("/c"),
+            key: PathBuf::from("/k"),
+            ca: None,
+        };
+        // so_reuseport is ALWAYS on now (so the supervisor can add/reap workers live).
+        let small = pgbouncer_ini(&tls, 4 * 1024 * 1024 * 1024, 4);
+        assert!(small.contains("so_reuseport = 1"));
+        assert!(small.contains("default_pool_size = 16")); // pool 16 (4 vCPU * 3, floored)
+        // Empty unix_socket_dir so multiple so_reuseport workers share only TCP :5432.
+        assert!(
+            small.contains("unix_socket_dir ="),
+            "needs empty unix_socket_dir: {small}"
+        );
+        // Mid box: full pool 48 (16 vCPU * 3), not divided across workers.
+        let mid = pgbouncer_ini(&tls, 32u64 * 1024 * 1024 * 1024, 16);
+        assert!(
+            mid.contains("default_pool_size = 48"),
+            "16 vCPU * 3 = full 48: {mid}"
+        );
+        // Big box: FULL pool per worker (a single live worker gets the whole 192, not
+        // 192/8=24 — the scaler adds workers for handshake CPU, not query concurrency).
+        let big = pgbouncer_ini(&tls, 256u64 * 1024 * 1024 * 1024, 64);
+        assert!(big.contains("so_reuseport = 1"));
+        assert!(
+            big.contains("default_pool_size = 192"),
+            "64 vCPU * 3 = full 192 per worker: {big}"
+        );
+        // max_connections is the real backstop covering server-side query demand.
+        let tuning = tuning_conf_boot(256u64 * 1024 * 1024 * 1024, 64);
+        assert!(tuning.contains("max_connections = 330")); // 192 + 64*2 + 10
+    }
+
     #[test]
     fn replica_conf_no_wal_sink() {
         let conf = replica_conf("host=10.0.0.1 port=5433 user=replicator", None);
diff --git a/src/handoff_bridge.rs b/src/handoff_bridge.rs
index df4b054..4e92248 100644
--- a/src/handoff_bridge.rs
+++ b/src/handoff_bridge.rs
@@ -26,16 +26,54 @@
 //! Linux-only (handoff crate). Module-level gating done at the
 //! `mod handoff_bridge;` declaration sites in `main.rs` and `lib.rs`.
 
-use std::sync::Arc;
 use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
+use std::sync::{Arc, Mutex};
 use std::time::{Duration, Instant};
 
 use handoff::{DrainReport, Drainable, SealReport, StateSnapshot};
 
+/// Snapshot of the PgBouncer pooler tier, published by the supervisor's scaler
+/// every tick and served over the `pooler` RPC command. This is the production
+/// signal for "does a real box ever saturate a pooler": `at_ceiling` sustained
+/// means even the maxed-out worker set is CPU-bound; `live_workers` stuck at 1
+/// forever means the scaler was never needed.
+///
+/// Lives here (rather than in `rpc`) because `rpc`/`supervisor` are binary-only
+/// modules while `handoff_bridge` is in the library — `SharedState` carries the
+/// handle, so the type must be library-visible.
+#[derive(Clone, Debug, Default, serde::Serialize)]
+pub struct PoolerStats {
+    /// Currently-running pooler processes (worker 0 + scaled-up extras).
+    pub live_workers: u32,
+    /// Ceiling the scaler may grow to (`config::pgbouncer_max_workers`).
+    pub max_workers: u32,
+    /// Total pooler CPU across all live workers, in cores, over the last tick.
+    pub aggregate_cpu_cores: f64,
+    /// `aggregate_cpu_cores / live_workers` — the per-worker saturation signal.
+    pub per_worker_cores: f64,
+    /// `live == max` AND per-worker near-saturated: the box wants more pooler
+    /// capacity than the cap allows.
+    pub at_ceiling: bool,
+    /// Last scaling action the scaler took: "up", "down", or "" (none yet).
+    pub last_action: String,
+}
+
+/// Shared handle: scaler writes, RPC reads. Lock is held only for a struct copy.
+pub type PoolerStatsHandle = Arc<Mutex<PoolerStats>>;
+
+/// Construct a fresh, zeroed pooler-stats handle.
+pub fn new_pooler_stats() -> PoolerStatsHandle {
+    Arc::new(Mutex::new(PoolerStats::default()))
+}
+
 #[derive(Clone, Default)]
 pub struct SharedState {
     pub accept_paused: Arc<AtomicBool>,
     pub in_flight: Arc<AtomicUsize>,
+    /// PgBouncer scaler telemetry, published by the supervisor and served over
+    /// the `pooler` RPC command. Shares the handle so the RPC handler reads what
+    /// the scaler last wrote. `Default` = a zeroed handle (Arc<Mutex<…>>).
+    pub pooler: PoolerStatsHandle,
 }
 
 impl SharedState {
diff --git a/src/rpc.rs b/src/rpc.rs
index e5392c8..b07e1fd 100644
--- a/src/rpc.rs
+++ b/src/rpc.rs
@@ -10,9 +10,13 @@
 //!   health     → pg_isready
 //!   reload     → pg_ctl reload
 //!   promote    → pg_ctl promote -w (blocks until standby exits recovery; ok:true = primary)
+//!   pooler     → PgBouncer scaler telemetry (live/max workers, CPU, at_ceiling)
 //!   backup     → stub (not implemented)
 //!
-//! Only available on Linux.
+//! The RPC server is Linux-only. The pooler telemetry types
+//! ([`crate::handoff_bridge::PoolerStats`]) live in the library (next to
+//! `SharedState`, which carries the handle) so the supervisor's scaler can
+//! populate them on any host.
 
 #[cfg(target_os = "linux")]
 mod inner {
@@ -70,6 +74,8 @@ mod inner {
         ok: bool,
         #[serde(skip_serializing_if = "Option::is_none")]
         error: Option<String>,
+        #[serde(skip_serializing_if = "Option::is_none")]
+        pooler: Option<crate::handoff_bridge::PoolerStats>,
     }
 
     impl RpcResponse {
@@ -77,12 +83,21 @@ mod inner {
             Self {
                 ok: true,
                 error: None,
+                pooler: None,
             }
         }
         fn err(msg: impl Into<String>) -> Self {
             Self {
                 ok: false,
                 error: Some(msg.into()),
+                pooler: None,
+            }
+        }
+        fn pooler(stats: crate::handoff_bridge::PoolerStats) -> Self {
+            Self {
+                ok: true,
+                error: None,
+                pooler: Some(stats),
             }
         }
     }
@@ -109,9 +124,11 @@ mod inner {
             match listener.accept().await {
                 Ok(stream) => {
                     let in_flight = Arc::clone(&state.in_flight);
+                    let pooler = Arc::clone(&state.pooler);
                     in_flight.fetch_add(1, Ordering::SeqCst);
                     tokio::spawn(async move {
-                        let result = tokio::time::timeout(RPC_TIMEOUT, handle_stream(stream)).await;
+                        let result =
+                            tokio::time::timeout(RPC_TIMEOUT, handle_stream(stream, pooler)).await;
                         match result {
                             Ok(Ok(())) => {}
                             Ok(Err(e)) => warn!("rpc: connection error: {e}"),
@@ -142,14 +159,20 @@ mod inner {
         Ok(RpcListener::Vsock(v))
     }
 
-    async fn handle_stream(stream: RpcStream) -> std::io::Result<()> {
+    async fn handle_stream(
+        stream: RpcStream,
+        pooler: crate::handoff_bridge::PoolerStatsHandle,
+    ) -> std::io::Result<()> {
         match stream {
-            RpcStream::Vsock(s) => handle(s).await,
-            RpcStream::Unix(s) => handle(s).await,
+            RpcStream::Vsock(s) => handle(s, pooler).await,
+            RpcStream::Unix(s) => handle(s, pooler).await,
         }
     }
 
-    async fn handle<S>(mut stream: S) -> std::io::Result<()>
+    async fn handle<S>(
+        mut stream: S,
+        pooler: crate::handoff_bridge::PoolerStatsHandle,
+    ) -> std::io::Result<()>
     where
         S: AsyncRead + AsyncWrite + Unpin,
     {
@@ -167,7 +190,7 @@ mod inner {
 
         let response = match rmp_serde::from_slice::<RpcRequest>(&body) {
             Err(e) => RpcResponse::err(format!("invalid request: {e}")),
-            Ok(req) => dispatch(&req.cmd).await,
+            Ok(req) => dispatch(&req.cmd, &pooler).await,
         };
 
         let encoded =
@@ -180,12 +203,16 @@ mod inner {
         Ok(())
     }
 
-    async fn dispatch(cmd: &str) -> RpcResponse {
+    async fn dispatch(cmd: &str, pooler: &crate::handoff_bridge::PoolerStatsHandle) -> RpcResponse {
         match cmd {
             "checkpoint" => match crate::pg::psql("CHECKPOINT").await {
                 Ok(()) => RpcResponse::ok(),
                 Err(e) => RpcResponse::err(e.to_string()),
             },
+            "pooler" => match pooler.lock() {
+                Ok(stats) => RpcResponse::pooler(stats.clone()),
+                Err(_) => RpcResponse::err("pooler stats lock poisoned"),
+            },
             "health" => {
                 if crate::pg::is_ready().await {
                     RpcResponse::ok()
diff --git a/src/supervisor.rs b/src/supervisor.rs
index 2301ecd..ff407e7 100644
--- a/src/supervisor.rs
+++ b/src/supervisor.rs
@@ -36,6 +36,238 @@ const DRAIN_TIMEOUT: Duration = Duration::from_secs(10);
 const STABLE_RUNTIME: Duration = Duration::from_secs(60);
 const MAX_BACKOFF_MS: u64 = 30_000;
 
+/// Stable, supervisor-persisted names for extra PgBouncer so_reuseport workers
+/// (worker 0 keeps the name "pgbouncer" for handoff compatibility). Capped at 7
+/// because `config::pgbouncer_workers` caps total workers at 8.
+const PGB_EXTRA_NAMES: [&str; 7] = [
+    "pgbouncer-1",
+    "pgbouncer-2",
+    "pgbouncer-3",
+    "pgbouncer-4",
+    "pgbouncer-5",
+    "pgbouncer-6",
+    "pgbouncer-7",
+];
+
+/// Wait for ANY child in the slice to exit; returns its exit code and index.
+/// Pends forever on an empty slice. Hand-rolled `select_all` (no `futures` dep):
+/// persistent per-child wait futures are built once and polled together, so no
+/// progress is lost between polls.
+async fn wait_any_child(workers: &mut [ChildState]) -> (std::io::Result<Option<i32>>, usize) {
+    if workers.is_empty() {
+        std::future::pending::<()>().await;
+        unreachable!();
+    }
+    let mut futs: Vec<_> = workers
+        .iter_mut()
+        .map(|w| {
+            let h = w.handle.as_mut();
+            Box::pin(async move {
+                match h {
+                    Some(h) => h.wait().await,
+                    None => std::future::pending::<std::io::Result<Option<i32>>>().await,
+                }
+            })
+        })
+        .collect();
+    std::future::poll_fn(|cx| {
+        for (i, f) in futs.iter_mut().enumerate() {
+            if let std::task::Poll::Ready(r) = f.as_mut().poll(cx) {
+                return std::task::Poll::Ready((r, i));
+            }
+        }
+        std::task::Poll::Pending
+    })
+    .await
+}
+
+// ---------------------------------------------------------------------------
+// PgBouncer reactive scaler
+// ---------------------------------------------------------------------------
+//
+// The single-threaded pooler saturates one core terminating TLS under
+// connection churn (~2.4k conns/s/core, measured in bench/glidefs-pg §F). When a
+// box is genuinely pooler-CPU-bound we add so_reuseport workers — the kernel
+// load-balances new connections across them (proven linear: pooler CPU 0.79→1.59
+// cores for 2 workers) — and reap them when load falls, so an idle/scaled-to-zero
+// box runs exactly ONE pooler and only a busy box spends extra cores.
+//
+// The decision is an inline select! arm (not a spawned task) because it mutates
+// pgb_state/pgb_extra to spawn and SIGINT workers.
+
+/// How often the scaler samples pooler CPU and reconsiders the worker count.
+const SCALER_TICK: Duration = Duration::from_secs(5);
+/// Per-worker CPU (cores) above which the tier is "hot". 0.75 ≈ nearly saturated
+/// (a single pooler tops ~0.79–1.0 cores in the rig).
+const SCALE_UP_CORES: f64 = 0.75;
+/// Low-water per-worker CPU (cores) the tier must stay UNDER — computed as if one
+/// worker were already removed — to be considered cold enough to reap.
+const SCALE_DOWN_CORES: f64 = 0.5;
+/// Consecutive hot ticks before scaling up (≈10s) — react fast.
+const SCALE_UP_TICKS: u32 = 2;
+/// Consecutive cold ticks before scaling down (≈30s) — react slow, to avoid
+/// thrash and minimize the ~0.03% reconnect blip a reap causes.
+const SCALE_DOWN_TICKS: u32 = 6;
+/// Minimum gap between scale-UP actions.
+const SCALE_UP_COOLDOWN: Duration = Duration::from_secs(30);
+/// Minimum gap between scale-DOWN actions.
+const SCALE_DOWN_COOLDOWN: Duration = Duration::from_secs(60);
+
+#[derive(Clone, Copy, PartialEq, Eq, Debug)]
+enum ScaleAction {
+    Up,
+    Down,
+    Hold,
+}
+
+/// Linux USER_HZ (`sysconf(_SC_CLK_TCK)`): the ticks-per-second that
+/// /proc/<pid>/stat utime+stime are denominated in. Effectively always 100, but
+/// read it to be correct; fall back to 100.
+fn clk_tck() -> f64 {
+    #[cfg(unix)]
+    {
+        let v = unsafe { libc::sysconf(libc::_SC_CLK_TCK) };
+        if v > 0 { v as f64 } else { 100.0 }
+    }
+    #[cfg(not(unix))]
+    {
+        100.0
+    }
+}
+
+/// Pure scaling decision: given current per-worker load + streak/cooldown state,
+/// return the action and the updated streak counters. Side-effect-free so it can
+/// be unit-tested without spawning processes; the caller stamps `last_action`
+/// and adopts the returned streaks.
+fn decide_scale(
+    per_worker_cores: f64,
+    aggregate_cores: f64,
+    live: usize,
+    max: usize,
+    up_streak: u32,
+    down_streak: u32,
+    since_last_action: Duration,
+) -> (ScaleAction, u32, u32) {
+    // Hot streak: per-worker is saturated.
+    let up_streak = if per_worker_cores > SCALE_UP_CORES {
+        up_streak.saturating_add(1)
+    } else {
+        0
+    };
+    // Cold streak: even after dropping one worker, the remaining set would stay
+    // under the low-water mark — so the extra worker is dead weight.
+    let cold = live > 1 && aggregate_cores / (live as f64 - 1.0) < SCALE_DOWN_CORES;
+    let down_streak = if cold {
+        down_streak.saturating_add(1)
+    } else {
+        0
+    };
+
+    if up_streak >= SCALE_UP_TICKS && live < max && since_last_action >= SCALE_UP_COOLDOWN {
+        return (ScaleAction::Up, 0, 0);
+    }
+    if down_streak >= SCALE_DOWN_TICKS && live > 1 && since_last_action >= SCALE_DOWN_COOLDOWN {
+        return (ScaleAction::Down, 0, 0);
+    }
+    (ScaleAction::Hold, up_streak, down_streak)
+}
+
+/// Sampling + decision state carried across ticks (impure half: reads /proc).
+struct PgbScaler {
+    clk_tck: f64,
+    max_workers: usize,
+    /// Per-pid cumulative CPU ticks from the previous sample, keyed by pid so a
+    /// worker added/reaped between ticks can't corrupt the aggregate delta (only
+    /// pids present in BOTH samples contribute).
+    prev_ticks: std::collections::HashMap<u32, u64>,
+    prev_at: Option<Instant>,
+    up_streak: u32,
+    down_streak: u32,
+    last_action_at: Option<Instant>,
+}
+
+impl PgbScaler {
+    fn new(max_workers: usize) -> Self {
+        Self {
+            clk_tck: clk_tck(),
+            max_workers,
+            prev_ticks: std::collections::HashMap::new(),
+            prev_at: None,
+            up_streak: 0,
+            down_streak: 0,
+            last_action_at: None,
+        }
+    }
+
+    /// Sample pooler CPU for `pids`, decide, publish telemetry to `stats`, and
+    /// return the action for the caller to enact. The first call only establishes
+    /// a CPU baseline (returns Hold).
+    fn tick(
+        &mut self,
+        pids: &[u32],
+        now: Instant,
+        stats: &crate::handoff_bridge::PoolerStatsHandle,
+    ) -> ScaleAction {
+        let live = pids.len();
+        let mut cur: std::collections::HashMap<u32, u64> =
+            std::collections::HashMap::with_capacity(live);
+        let mut delta_ticks: u64 = 0;
+        for &pid in pids {
+            let t = crate::children::read_proc_cpu_ticks(pid).unwrap_or(0);
+            if let Some(&prev) = self.prev_ticks.get(&pid) {
+                delta_ticks += t.saturating_sub(prev);
+            }
+            cur.insert(pid, t);
+        }
+        let elapsed = self
+            .prev_at
+            .map(|p| now.saturating_duration_since(p).as_secs_f64())
+            .unwrap_or(0.0);
+        self.prev_ticks = cur;
+        self.prev_at = Some(now);
+
+        if elapsed <= 0.0 || live == 0 {
+            return ScaleAction::Hold; // baseline tick (or no pooler live)
+        }
+
+        let aggregate_cores = (delta_ticks as f64 / self.clk_tck) / elapsed;
+        let per_worker = aggregate_cores / live as f64;
+        let since = self
+            .last_action_at
+            .map(|t| now.saturating_duration_since(t))
+            .unwrap_or(Duration::MAX);
+
+        let (action, up, down) = decide_scale(
+            per_worker,
+            aggregate_cores,
+            live,
+            self.max_workers,
+            self.up_streak,
+            self.down_streak,
+            since,
+        );
+        self.up_streak = up;
+        self.down_streak = down;
+        if action != ScaleAction::Hold {
+            self.last_action_at = Some(now);
+        }
+
+        if let Ok(mut s) = stats.lock() {
+            s.live_workers = live as u32;
+            s.max_workers = self.max_workers as u32;
+            s.aggregate_cpu_cores = (aggregate_cores * 100.0).round() / 100.0;
+            s.per_worker_cores = (per_worker * 100.0).round() / 100.0;
+            s.at_ceiling = live >= self.max_workers && per_worker > SCALE_UP_CORES;
+            match action {
+                ScaleAction::Up => s.last_action = "up".into(),
+                ScaleAction::Down => s.last_action = "down".into(),
+                ScaleAction::Hold => {}
+            }
+        }
+        action
+    }
+}
+
 // ---------------------------------------------------------------------------
 // Public entry point
 // ---------------------------------------------------------------------------
@@ -138,6 +370,9 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
 
     let mut pg_state: ChildState;
     let mut pgb_state: ChildState;
+    // Extra PgBouncer so_reuseport workers (worker 0 is pgb_state). Empty on boxes
+    // ≤ 8 vCPU, so everything below is a no-op there and the common path is unchanged.
+    let mut pgb_extra: Vec<ChildState> = Vec::new();
     let mut cdc_state: Option<ChildState>;
     let cold_start_wal_handle: Option<tokio::task::JoinHandle<()>>;
 
@@ -163,6 +398,19 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
             let prior = crate::children::PersistedChildren::load(&crate::children::state_dir())?;
             pg_state = adopt_or_respawn_child("postgres", &prior, &log_tx, &mut persisted)?;
             pgb_state = adopt_or_respawn_child("pgbouncer", &prior, &log_tx, &mut persisted)?;
+            // Adopt exactly the extra workers the predecessor had live (a busy box keeps
+            // its scaled-up pooler across a zero-downtime upgrade). The scaler re-checks
+            // load on its next tick and adjusts; we don't pre-spawn up to the cap.
+            for &name in PGB_EXTRA_NAMES.iter() {
+                if prior.get(name).is_some() {
+                    pgb_extra.push(adopt_or_respawn_child(
+                        name,
+                        &prior,
+                        &log_tx,
+                        &mut persisted,
+                    )?);
+                }
+            }
 
             // CDC is configured per-MMDS; adopt only if both the prior
             // supervisor had it AND we still want it. If config flipped
@@ -232,6 +480,8 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
         #[cfg(target_os = "linux")]
         persist_child(&mut persisted, &pgb)?;
         pgb_state = pgb;
+        // Start with ONE pooler. The PgbScaler grows pgb_extra reactively under load
+        // (so_reuseport) and reaps it back down — see the scaler tick in the run loop.
 
         cdc_state = if cfg.pg_tier != PgTier::Replica && cfg.cdc_enabled {
             let mut s = ChildState::new("beyond-pg-cdc");
@@ -279,8 +529,16 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
     // (transparently dup'd into slot 3 by `beyond-pg-init`'s handoff
     // supervisor). Inheriting preserves the accept queue — clients see no
     // socket close.
+    // Pooler scaler telemetry handle — the scaler (run loop) writes it each tick,
+    // the RPC `pooler` command reads it. Created cross-platform so the scaler can
+    // populate it even where the (Linux-only) RPC server is a no-op.
+    let pooler_stats = crate::handoff_bridge::new_pooler_stats();
     #[cfg(target_os = "linux")]
-    let rpc_state = crate::handoff_bridge::SharedState::new();
+    let rpc_state = {
+        let mut s = crate::handoff_bridge::SharedState::new();
+        s.pooler = pooler_stats.clone();
+        s
+    };
     #[cfg(target_os = "linux")]
     let rpc_listener = if is_successor {
         take_inherited_rpc_listener(begun_successor.as_mut().expect("successor"))?
@@ -349,6 +607,14 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
         sigprocmask(SigmaskHow::SIG_UNBLOCK, Some(&set), None)?;
     }
 
+    // Reactive PgBouncer scaler: sample pooler CPU every tick and grow/shrink the
+    // so_reuseport worker set within [1, max_workers]. On boxes where max_workers
+    // == 1 it only ever publishes telemetry (the justification signal for raising
+    // the cap) and never acts.
+    let mut pgb_scaler = PgbScaler::new(crate::config::pgbouncer_max_workers(cfg.vcpus));
+    let mut scaler_tick = tokio::time::interval(SCALER_TICK);
+    scaler_tick.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
+
     info!("supervisor running");
 
     // Main supervision loop
@@ -360,14 +626,14 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
             _ = sigterm.recv() => {
                 info!("received SIGTERM, draining children");
                 abort_wal_forwarder(&mut wal_forwarder_handle).await;
-                drain_children(&mut pg_state, &mut pgb_state, cdc_state.as_mut()).await;
+                drain_children(&mut pg_state, &mut pgb_state, &mut pgb_extra, cdc_state.as_mut()).await;
                 shutdown_background_tasks(log_writer_handle, rpc_handle, memory_watcher_handle, cert_watcher_handle).await;
                 return Ok(());
             }
             _ = sigint.recv() => {
                 info!("received SIGINT, draining children");
                 abort_wal_forwarder(&mut wal_forwarder_handle).await;
-                drain_children(&mut pg_state, &mut pgb_state, cdc_state.as_mut()).await;
+                drain_children(&mut pg_state, &mut pgb_state, &mut pgb_extra, cdc_state.as_mut()).await;
                 shutdown_background_tasks(log_writer_handle, rpc_handle, memory_watcher_handle, cert_watcher_handle).await;
                 return Ok(());
             }
@@ -397,6 +663,21 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
                 #[cfg(target_os = "linux")]
                 persist_child(&mut persisted, &pgb_state)?;
             }
+            // Extra so_reuseport pgbouncer workers (disabled when there are none).
+            (res, idx) = wait_any_child(&mut pgb_extra), if !pgb_extra.is_empty() => {
+                let _ = res?; // propagate I/O errors; exit code irrelevant for extras
+                // Extra pooler workers are never individually restarted — the scaler
+                // owns the live count. Whether this was a crash or a scale-down reap,
+                // drop it from the set; the scaler re-adds on its next tick if load
+                // still demands it (the kernel rebalances connections meanwhile).
+                let gone = pgb_extra.remove(idx);
+                info!("pooler worker '{}' exited; removed (scaler owns count)", gone.name);
+                #[cfg(target_os = "linux")]
+                {
+                    persisted.remove(gone.name);
+                    persisted.save(&crate::children::state_dir())?;
+                }
+            }
             code = async {
                 match cdc_state.as_mut().and_then(|s| s.handle.as_mut()) {
                     Some(handle) => handle.wait().await,
@@ -417,6 +698,63 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
                     warn!("pg_ctl reload after cert rotation failed: {e}");
                 }
                 sighup_pgbouncer(&pgb_state);
+                for w in &pgb_extra {
+                    sighup_pgbouncer(w);
+                }
+            }
+            // Reactive pooler scaling: sample CPU, grow/shrink so_reuseport workers.
+            _ = scaler_tick.tick() => {
+                let now = Instant::now();
+                let mut pids: Vec<u32> = Vec::with_capacity(1 + pgb_extra.len());
+                if let Some(p) = pgb_state.pid() {
+                    pids.push(p);
+                }
+                for w in &pgb_extra {
+                    if let Some(p) = w.pid() {
+                        pids.push(p);
+                    }
+                }
+                match pgb_scaler.tick(&pids, now, &pooler_stats) {
+                    ScaleAction::Up => {
+                        // index into PGB_EXTRA_NAMES == current extra count (worker 0
+                        // is pgb_state). decide_scale guarantees live < max ≤ 8, so the
+                        // index is in range; the guard is belt-and-suspenders.
+                        let idx = pgb_extra.len();
+                        if idx < PGB_EXTRA_NAMES.len() {
+                            let name = PGB_EXTRA_NAMES[idx];
+                            let mut w = ChildState::new(name);
+                            match spawn_pgbouncer(&mut w, &log_tx) {
+                                Ok(()) => {
+                                    info!(
+                                        "pooler scaler: UP to {} workers (per-worker CPU saturated; spawned '{name}')",
+                                        pids.len() + 1
+                                    );
+                                    #[cfg(target_os = "linux")]
+                                    if let Err(e) = persist_child(&mut persisted, &w) {
+                                        warn!("pooler scaler: persist of '{name}' failed: {e}");
+                                    }
+                                    pgb_extra.push(w);
+                                }
+                                Err(e) => {
+                                    warn!("pooler scaler: failed to spawn '{name}': {e}");
+                                }
+                            }
+                        }
+                    }
+                    ScaleAction::Down => {
+                        // Graceful: SIGINT drains in-flight txns then exits; the
+                        // wait_any_child arm reaps it from pgb_extra + persisted.
+                        if let Some(w) = pgb_extra.last() {
+                            info!(
+                                "pooler scaler: DOWN from {} workers (SIGINT graceful drain of '{}')",
+                                pids.len(),
+                                w.name
+                            );
+                            sigint_pgbouncer(w);
+                        }
+                    }
+                    ScaleAction::Hold => {}
+                }
             }
         }
     }
@@ -446,6 +784,31 @@ fn sighup_pgbouncer(state: &ChildState) {
     }
 }
 
+/// Send SIGINT to a PgBouncer worker for a *graceful* shutdown ("safe shutdown":
+/// finish in-flight transactions, then exit). Used by the scaler to reap an extra
+/// so_reuseport worker — measured ~0.03% reconnect blip as the kernel re-routes
+/// its clients to the survivors. Best-effort: a missing pid means it's already gone.
+fn sigint_pgbouncer(state: &ChildState) {
+    let Some(pid) = state.pid() else {
+        info!("pgbouncer reap: no live child, SIGINT skipped");
+        return;
+    };
+    #[cfg(target_os = "linux")]
+    {
+        use nix::sys::signal::{Signal, kill};
+        use nix::unistd::Pid;
+        match kill(Pid::from_raw(pid as i32), Signal::SIGINT) {
+            Ok(()) => info!("pgbouncer reap: SIGINT sent (pid={pid})"),
+            Err(e) => warn!("pgbouncer reap: SIGINT failed (pid={pid}): {e}"),
+        }
+    }
+    #[cfg(not(target_os = "linux"))]
+    {
+        let _ = pid;
+        info!("pgbouncer reap: SIGINT skipped (non-Linux build)");
+    }
+}
+
 // ---------------------------------------------------------------------------
 // Child state and restart logic
 // ---------------------------------------------------------------------------
@@ -1032,16 +1395,23 @@ fn drop_to_postgres_user(
 async fn drain_children(
     pg: &mut ChildState,
     pgb: &mut ChildState,
+    pgb_extra: &mut [ChildState],
     mut cdc: Option<&mut ChildState>,
 ) {
     // Send SIGTERM to all live children (not kill() which is SIGKILL)
     sigterm_child(pg);
     sigterm_child(pgb);
+    for w in pgb_extra.iter_mut() {
+        sigterm_child(w);
+    }
     if let Some(c) = cdc.as_deref_mut() {
         sigterm_child(c);
     }
 
     let mut all: Vec<&mut ChildState> = vec![pg, pgb];
+    for w in pgb_extra.iter_mut() {
+        all.push(w);
+    }
     if let Some(c) = cdc {
         all.push(c);
     }
@@ -1373,6 +1743,72 @@ fn parse_memtotal_kib(content: &str) -> Option<u64> {
 mod tests {
     use super::*;
 
+    // --- PgBouncer scaler decision (pure) ---
+
+    const COOL: Duration = Duration::from_secs(3600); // past every cooldown
+    const HOT: f64 = 0.9; // per-worker cores, above SCALE_UP_CORES
+    const COLD_AGG: f64 = 0.4; // aggregate cores: removing a worker stays < 0.5
+
+    #[test]
+    fn scaler_holds_until_up_streak_met() {
+        // One hot tick is not enough (needs SCALE_UP_TICKS = 2).
+        let (a, up, _down) = decide_scale(HOT, HOT, 1, 4, 0, 0, COOL);
+        assert_eq!(a, ScaleAction::Hold);
+        assert_eq!(up, 1);
+        // Second consecutive hot tick → Up, streak reset.
+        let (a, up, _down) = decide_scale(HOT, HOT, 1, 4, up, 0, COOL);
+        assert_eq!(a, ScaleAction::Up);
+        assert_eq!(up, 0);
+    }
+
+    #[test]
+    fn scaler_never_scales_past_ceiling() {
+        // live == max: stay put no matter how hot or how long.
+        let (a, _u, _d) = decide_scale(HOT, HOT * 4.0, 4, 4, 99, 0, COOL);
+        assert_eq!(a, ScaleAction::Hold);
+    }
+
+    #[test]
+    fn scaler_respects_up_cooldown() {
+        // Hot streak met but action happened 1s ago → Hold.
+        let (a, _u, _d) = decide_scale(HOT, HOT, 1, 4, 99, 0, Duration::from_secs(1));
+        assert_eq!(a, ScaleAction::Hold);
+    }
+
+    #[test]
+    fn scaler_scales_down_when_cold_and_streak_met() {
+        // 2 live workers, aggregate so low that one worker would suffice.
+        let (a, _u, down) = decide_scale(COLD_AGG / 2.0, COLD_AGG, 2, 4, 0, 0, COOL);
+        assert_eq!(a, ScaleAction::Hold); // first cold tick
+        assert_eq!(down, 1);
+        // After SCALE_DOWN_TICKS consecutive cold ticks → Down.
+        let (a, _u, _d) = decide_scale(
+            COLD_AGG / 2.0,
+            COLD_AGG,
+            2,
+            4,
+            0,
+            SCALE_DOWN_TICKS - 1,
+            COOL,
+        );
+        assert_eq!(a, ScaleAction::Down);
+    }
+
+    #[test]
+    fn scaler_never_scales_below_one() {
+        // Single worker, ice cold, long streak → still Hold (never reap worker 0).
+        let (a, _u, _d) = decide_scale(0.0, 0.0, 1, 4, 0, 99, COOL);
+        assert_eq!(a, ScaleAction::Hold);
+    }
+
+    #[test]
+    fn scaler_down_streak_resets_when_warm() {
+        // 2 workers but each near-busy: removing one would overload → not cold.
+        let (a, _u, down) = decide_scale(0.7, 1.4, 2, 4, 0, 5, COOL);
+        assert_eq!(a, ScaleAction::Hold);
+        assert_eq!(down, 0, "warm tick resets the down streak");
+    }
+
     #[test]
     fn parse_memtotal_standard_format() {
         let input = "MemTotal:       16384000 kB\nMemFree:        8192000 kB\n";

From 973f51b0ec1a64d926b1fe1f3db2778697aa3044 Mon Sep 17 00:00:00 2001
From: Jared Lunde <jared.lunde@gmail.com>
Date: Thu, 4 Jun 2026 12:36:21 -0700
Subject: [PATCH 2/6] test pgbouncer: cover the scaler's sampling,
 orchestration, and actuation
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Close the test gaps in the reactive scaler by adding seams, not infra — the
earlier "needs QEMU" framing was wrong (QEMU only isolated the GlideFS
substrate experiments; the scaler is pgbouncer + /proc + a port).

  - tick_with(): inject the CPU sampler so the whole orchestration — the
    per-pid prev-ticks map (proves a worker appearing mid-window can't spike
    the aggregate), the cores math, telemetry population, and the decide
    integration — is deterministically testable. 4 tests.
  - apply_scale_action(): extract spawn/reap actuation behind injected
    closures; test name-ordering, the worker-name cap, spawn failure, and that
    Down SIGINTs the last worker without removing it (wait_any_child does that
    on exit). 4 tests.
  - read_proc_cpu_ticks: real-/proc tests (busy-loop increases ticks; missing
    pid errors) — the manual 0.993-core check is now a regression guard.
  - scaler_tick_detects_real_cpu_load_and_decides_up (#[ignore]): real-process
    integration — production tick() over real /proc of a real CPU-burner reads
    ~1 core and decides Up. Runs here (no pgbouncer/QEMU needed); verified.

Remaining uncovered surface is narrow and characterized: spawn_pgbouncer
launching a real pgbouncer + so_reuseport CPU scaling + SIGINT drain — proven
empirically in bench/glidefs-pg §F/§F″, exercised automatically only by the
Docker supervisor lifecycle suite (CI; can't run on this host).

121 unit tests green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 src/children.rs   |  24 +++
 src/supervisor.rs | 417 +++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 401 insertions(+), 40 deletions(-)

diff --git a/src/children.rs b/src/children.rs
index e14edde..a141fec 100644
--- a/src/children.rs
+++ b/src/children.rs
@@ -213,6 +213,30 @@ pub fn state_dir() -> PathBuf {
 mod tests {
     use super::*;
 
+    #[test]
+    fn read_proc_cpu_ticks_for_self_increases_under_load() {
+        let me = std::process::id();
+        let before = read_proc_cpu_ticks(me).expect("read self cpu ticks");
+        // Burn CPU so utime/stime advance (≥ a few ticks at 100 Hz over 300 ms).
+        let mut x: u64 = 1;
+        let start = std::time::Instant::now();
+        while start.elapsed() < std::time::Duration::from_millis(300) {
+            x = x.wrapping_mul(6364136223846793005).wrapping_add(1);
+        }
+        std::hint::black_box(x);
+        let after = read_proc_cpu_ticks(me).expect("read self cpu ticks again");
+        assert!(
+            after > before,
+            "300ms of CPU burn must add ≥1 tick: {before} -> {after}"
+        );
+    }
+
+    #[test]
+    fn read_proc_cpu_ticks_missing_pid_errors() {
+        // No /proc/<u32::MAX>/stat — the read must surface an error, not 0.
+        assert!(read_proc_cpu_ticks(u32::MAX).is_err());
+    }
+
     #[test]
     fn round_trip_save_load() {
         let dir = tempfile::tempdir().unwrap();
diff --git a/src/supervisor.rs b/src/supervisor.rs
index ff407e7..fc35979 100644
--- a/src/supervisor.rs
+++ b/src/supervisor.rs
@@ -199,21 +199,39 @@ impl PgbScaler {
         }
     }
 
-    /// Sample pooler CPU for `pids`, decide, publish telemetry to `stats`, and
-    /// return the action for the caller to enact. The first call only establishes
-    /// a CPU baseline (returns Hold).
+    /// Production entry point: sample pooler CPU from `/proc`, decide, publish
+    /// telemetry, return the action. Thin wrapper over [`Self::tick_with`] with
+    /// the real `/proc` sampler injected.
     fn tick(
         &mut self,
         pids: &[u32],
         now: Instant,
         stats: &crate::handoff_bridge::PoolerStatsHandle,
+    ) -> ScaleAction {
+        self.tick_with(pids, now, stats, |pid| {
+            crate::children::read_proc_cpu_ticks(pid).unwrap_or(0)
+        })
+    }
+
+    /// Sample pooler CPU for `pids` via `sample` (cumulative ticks per pid),
+    /// decide, publish telemetry to `stats`, and return the action for the
+    /// caller to enact. The first call only establishes a CPU baseline (returns
+    /// Hold). `sample` is injectable so the whole orchestration — the
+    /// prev-ticks-across-worker-churn map, the cores math, telemetry, and the
+    /// decision — is deterministically testable without real processes or load.
+    fn tick_with(
+        &mut self,
+        pids: &[u32],
+        now: Instant,
+        stats: &crate::handoff_bridge::PoolerStatsHandle,
+        sample: impl Fn(u32) -> u64,
     ) -> ScaleAction {
         let live = pids.len();
         let mut cur: std::collections::HashMap<u32, u64> =
             std::collections::HashMap::with_capacity(live);
         let mut delta_ticks: u64 = 0;
         for &pid in pids {
-            let t = crate::children::read_proc_cpu_ticks(pid).unwrap_or(0);
+            let t = sample(pid);
             if let Some(&prev) = self.prev_ticks.get(&pid) {
                 delta_ticks += t.saturating_sub(prev);
             }
@@ -268,6 +286,58 @@ impl PgbScaler {
     }
 }
 
+/// What [`apply_scale_action`] actually did, so the caller can persist/log with
+/// the right worker and count.
+#[derive(Debug, PartialEq, Eq)]
+enum ScaleOutcome {
+    /// Spawned and pushed an extra worker with this stable name.
+    SpawnedUp(&'static str),
+    /// SIGINT'd the named worker for graceful drain (still in `pgb_extra` until
+    /// it exits and the `wait_any_child` arm reaps it).
+    ReapedDown(&'static str),
+    /// Nothing changed (Hold, at the name cap, spawn failed, or nothing to reap).
+    NoOp,
+}
+
+/// Enact a [`ScaleAction`] on the extra-worker set. Process I/O is injected via
+/// `spawn` (build+start a worker; `None` = failed) and `reap` (signal one to
+/// drain) so the name-selection, bounds, and set mutation are unit-testable
+/// without forking real pgbouncers. The new worker's name index is exactly the
+/// current extra count (worker 0 is `pgb_state`); `decide_scale` guarantees
+/// `live < max ≤ 8`, and the explicit bound is belt-and-suspenders.
+fn apply_scale_action(
+    action: ScaleAction,
+    pgb_extra: &mut Vec<ChildState>,
+    spawn: impl FnOnce(&'static str) -> Option<ChildState>,
+    reap: impl FnOnce(&ChildState),
+) -> ScaleOutcome {
+    match action {
+        ScaleAction::Up => {
+            let idx = pgb_extra.len();
+            if idx >= PGB_EXTRA_NAMES.len() {
+                return ScaleOutcome::NoOp; // already at the worker-name cap
+            }
+            let name = PGB_EXTRA_NAMES[idx];
+            match spawn(name) {
+                Some(w) => {
+                    pgb_extra.push(w);
+                    ScaleOutcome::SpawnedUp(name)
+                }
+                None => ScaleOutcome::NoOp,
+            }
+        }
+        ScaleAction::Down => match pgb_extra.last() {
+            Some(w) => {
+                let name = w.name;
+                reap(w);
+                ScaleOutcome::ReapedDown(name)
+            }
+            None => ScaleOutcome::NoOp,
+        },
+        ScaleAction::Hold => ScaleOutcome::NoOp,
+    }
+}
+
 // ---------------------------------------------------------------------------
 // Public entry point
 // ---------------------------------------------------------------------------
@@ -714,46 +784,46 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
                         pids.push(p);
                     }
                 }
-                match pgb_scaler.tick(&pids, now, &pooler_stats) {
-                    ScaleAction::Up => {
-                        // index into PGB_EXTRA_NAMES == current extra count (worker 0
-                        // is pgb_state). decide_scale guarantees live < max ≤ 8, so the
-                        // index is in range; the guard is belt-and-suspenders.
-                        let idx = pgb_extra.len();
-                        if idx < PGB_EXTRA_NAMES.len() {
-                            let name = PGB_EXTRA_NAMES[idx];
-                            let mut w = ChildState::new(name);
-                            match spawn_pgbouncer(&mut w, &log_tx) {
-                                Ok(()) => {
-                                    info!(
-                                        "pooler scaler: UP to {} workers (per-worker CPU saturated; spawned '{name}')",
-                                        pids.len() + 1
-                                    );
-                                    #[cfg(target_os = "linux")]
-                                    if let Err(e) = persist_child(&mut persisted, &w) {
-                                        warn!("pooler scaler: persist of '{name}' failed: {e}");
-                                    }
-                                    pgb_extra.push(w);
-                                }
-                                Err(e) => {
-                                    warn!("pooler scaler: failed to spawn '{name}': {e}");
-                                }
+                let action = pgb_scaler.tick(&pids, now, &pooler_stats);
+                let outcome = apply_scale_action(
+                    action,
+                    &mut pgb_extra,
+                    // spawn: build + start a real worker; None on failure (logged here).
+                    |name| {
+                        let mut w = ChildState::new(name);
+                        match spawn_pgbouncer(&mut w, &log_tx) {
+                            Ok(()) => Some(w),
+                            Err(e) => {
+                                warn!("pooler scaler: failed to spawn '{name}': {e}");
+                                None
                             }
                         }
-                    }
-                    ScaleAction::Down => {
-                        // Graceful: SIGINT drains in-flight txns then exits; the
-                        // wait_any_child arm reaps it from pgb_extra + persisted.
-                        if let Some(w) = pgb_extra.last() {
-                            info!(
-                                "pooler scaler: DOWN from {} workers (SIGINT graceful drain of '{}')",
-                                pids.len(),
-                                w.name
-                            );
-                            sigint_pgbouncer(w);
+                    },
+                    // reap: SIGINT for graceful drain; the wait_any_child arm
+                    // removes it from pgb_extra + persisted once it exits.
+                    sigint_pgbouncer,
+                );
+                match outcome {
+                    ScaleOutcome::SpawnedUp(name) => {
+                        let live = 1 + pgb_extra.len();
+                        info!(
+                            "pooler scaler: UP to {live} workers (per-worker CPU saturated; spawned '{name}')"
+                        );
+                        #[cfg(target_os = "linux")]
+                        if let Some(w) = pgb_extra.last()
+                            && let Err(e) = persist_child(&mut persisted, w)
+                        {
+                            warn!("pooler scaler: persist of '{name}' failed: {e}");
                         }
                     }
-                    ScaleAction::Hold => {}
+                    ScaleOutcome::ReapedDown(name) => {
+                        // worker still draining (not yet reaped from the vec).
+                        let live = 1 + pgb_extra.len();
+                        info!(
+                            "pooler scaler: DOWN from {live} workers (SIGINT graceful drain of '{name}')"
+                        );
+                    }
+                    ScaleOutcome::NoOp => {}
                 }
             }
         }
@@ -1809,6 +1879,273 @@ mod tests {
         assert_eq!(down, 0, "warm tick resets the down streak");
     }
 
+    // --- PgbScaler::tick orchestration (sampling + decide + telemetry),
+    //     deterministic via an injected CPU sampler + manual clock ---
+
+    fn test_scaler(max: usize) -> PgbScaler {
+        PgbScaler {
+            clk_tck: 100.0, // fixed USER_HZ so the cores math is exact in tests
+            max_workers: max,
+            prev_ticks: std::collections::HashMap::new(),
+            prev_at: None,
+            up_streak: 0,
+            down_streak: 0,
+            last_action_at: None,
+        }
+    }
+
+    #[test]
+    fn tick_baseline_then_scales_up_and_publishes_telemetry() {
+        let stats = crate::handoff_bridge::new_pooler_stats();
+        let mut s = test_scaler(4);
+        let cpu = std::cell::Cell::new(0u64); // cumulative ticks the sampler reports
+        let sample = |_pid: u32| cpu.get();
+        let t0 = Instant::now();
+        let pids = [100u32];
+
+        // 1st tick: no prior sample → baseline only, Hold.
+        assert_eq!(s.tick_with(&pids, t0, &stats, sample), ScaleAction::Hold);
+        // +1s, +90 ticks ⇒ 0.90 cores (1 worker). Hot, but needs 2 ticks.
+        cpu.set(90);
+        assert_eq!(
+            s.tick_with(&pids, t0 + Duration::from_secs(1), &stats, sample),
+            ScaleAction::Hold
+        );
+        // +1s, +90 ticks ⇒ 2nd hot tick ⇒ Up (no prior action → cooldown satisfied).
+        cpu.set(180);
+        assert_eq!(
+            s.tick_with(&pids, t0 + Duration::from_secs(2), &stats, sample),
+            ScaleAction::Up
+        );
+
+        let snap = stats.lock().unwrap().clone();
+        assert_eq!(snap.live_workers, 1);
+        assert_eq!(snap.max_workers, 4);
+        assert!(
+            (snap.per_worker_cores - 0.90).abs() < 0.01,
+            "per_worker_cores={}",
+            snap.per_worker_cores
+        );
+        assert_eq!(snap.last_action, "up");
+        assert!(!snap.at_ceiling, "1 of 4 workers is not at the ceiling");
+    }
+
+    #[test]
+    fn tick_scales_down_when_cold() {
+        let stats = crate::handoff_bridge::new_pooler_stats();
+        let mut s = test_scaler(4);
+        s.down_streak = SCALE_DOWN_TICKS - 1; // one cold tick away from a reap
+        let cpu = std::cell::Cell::new(0u64);
+        let sample = |_pid: u32| cpu.get(); // same cumulative for both pids
+        let t0 = Instant::now();
+        let pids = [200u32, 201u32];
+
+        // baseline
+        assert_eq!(s.tick_with(&pids, t0, &stats, sample), ScaleAction::Hold);
+        // +1s, each pid +5 ticks ⇒ aggregate 10 ticks = 0.10 cores over 2 workers.
+        // Removing one leaves 0.10 < 0.5 ⇒ cold ⇒ down_streak hits the threshold.
+        cpu.set(5);
+        assert_eq!(
+            s.tick_with(&pids, t0 + Duration::from_secs(1), &stats, sample),
+            ScaleAction::Down
+        );
+        assert_eq!(stats.lock().unwrap().last_action, "down");
+    }
+
+    #[test]
+    fn tick_at_ceiling_flag_set_when_max_and_saturated() {
+        let stats = crate::handoff_bridge::new_pooler_stats();
+        let mut s = test_scaler(2); // max == live below
+        let cpu = std::cell::Cell::new(0u64);
+        let sample = |_pid: u32| cpu.get();
+        let t0 = Instant::now();
+        let pids = [10u32, 11u32]; // 2 live == max 2
+
+        assert_eq!(s.tick_with(&pids, t0, &stats, sample), ScaleAction::Hold);
+        // each pid +90 ⇒ aggregate 180 ticks = 1.8 cores / 2 = 0.9 per worker (> 0.75).
+        cpu.set(90);
+        // Hot, but live == max ⇒ never Up; at_ceiling must flag the box wants more.
+        assert_eq!(
+            s.tick_with(&pids, t0 + Duration::from_secs(1), &stats, sample),
+            ScaleAction::Hold
+        );
+        let snap = stats.lock().unwrap().clone();
+        assert!(
+            snap.at_ceiling,
+            "live==max && saturated must set at_ceiling"
+        );
+    }
+
+    #[test]
+    fn tick_worker_churn_does_not_spike_aggregate() {
+        // A pid appearing mid-window (no prior sample) contributes 0 this tick —
+        // its large absolute counter must NOT be read as a delta. This is the
+        // per-pid-keyed map's whole reason to exist.
+        let stats = crate::handoff_bridge::new_pooler_stats();
+        let mut s = test_scaler(4);
+        let t0 = Instant::now();
+
+        // baseline: only pid 100 (cumulative 1000).
+        assert_eq!(
+            s.tick_with(&[100u32], t0, &stats, |_| 1000),
+            ScaleAction::Hold
+        );
+        // +1s: pid 100 advances 50 ticks; brand-new pid 200 shows up with a huge
+        // absolute counter. Only pid 100's 50-tick delta counts ⇒ 0.5 cores agg.
+        let act = s.tick_with(
+            &[100u32, 200u32],
+            t0 + Duration::from_secs(1),
+            &stats,
+            |pid| {
+                if pid == 100 { 1050 } else { 9_999_999 }
+            },
+        );
+        assert_eq!(act, ScaleAction::Hold);
+        let snap = stats.lock().unwrap().clone();
+        assert!(
+            (snap.aggregate_cpu_cores - 0.50).abs() < 0.01,
+            "new worker must not spike aggregate: agg={}",
+            snap.aggregate_cpu_cores
+        );
+    }
+
+    // --- apply_scale_action: spawn/reap mutation with injected process I/O ---
+
+    #[test]
+    fn apply_up_spawns_workers_in_name_order() {
+        let mut extra: Vec<ChildState> = Vec::new();
+        let out = apply_scale_action(
+            ScaleAction::Up,
+            &mut extra,
+            |name| Some(ChildState::new(name)),
+            |_w| panic!("reap must not run on Up"),
+        );
+        assert_eq!(out, ScaleOutcome::SpawnedUp("pgbouncer-1"));
+        assert_eq!(extra.len(), 1);
+        assert_eq!(extra[0].name, "pgbouncer-1");
+
+        let out = apply_scale_action(
+            ScaleAction::Up,
+            &mut extra,
+            |name| Some(ChildState::new(name)),
+            |_w| {},
+        );
+        assert_eq!(out, ScaleOutcome::SpawnedUp("pgbouncer-2"));
+        assert_eq!(extra.len(), 2);
+    }
+
+    #[test]
+    fn apply_down_reaps_last_but_keeps_it_pending() {
+        let mut extra = vec![
+            ChildState::new("pgbouncer-1"),
+            ChildState::new("pgbouncer-2"),
+        ];
+        let reaped = std::cell::Cell::new("");
+        let out = apply_scale_action(
+            ScaleAction::Down,
+            &mut extra,
+            |_n| panic!("spawn must not run on Down"),
+            |w| reaped.set(w.name),
+        );
+        assert_eq!(out, ScaleOutcome::ReapedDown("pgbouncer-2"));
+        assert_eq!(reaped.get(), "pgbouncer-2");
+        // Down does NOT remove from the vec — wait_any_child does, on actual exit.
+        assert_eq!(extra.len(), 2);
+    }
+
+    #[test]
+    fn apply_up_noops_at_name_cap() {
+        let mut extra: Vec<ChildState> = PGB_EXTRA_NAMES
+            .iter()
+            .map(|&n| ChildState::new(n))
+            .collect();
+        assert_eq!(extra.len(), PGB_EXTRA_NAMES.len());
+        let out = apply_scale_action(
+            ScaleAction::Up,
+            &mut extra,
+            |_n| panic!("must not spawn past the name cap"),
+            |_w| {},
+        );
+        assert_eq!(out, ScaleOutcome::NoOp);
+        assert_eq!(extra.len(), PGB_EXTRA_NAMES.len());
+    }
+
+    /// Real-process integration: the production `tick()` (real `/proc` sampler,
+    /// real clock) must turn a genuinely CPU-bound worker into a real `Up`
+    /// decision. Covers the wiring the injected-sampler tests stub out. Ignored
+    /// by default because it spawns a process and burns ~1.5s of CPU; run with
+    /// `cargo test -p beyond-pg --bins -- --ignored`.
+    #[test]
+    #[ignore = "spawns a real CPU-burning process; run with --ignored"]
+    fn scaler_tick_detects_real_cpu_load_and_decides_up() {
+        use std::process::{Command, Stdio};
+
+        // Cleanup guard: SIGKILL the burner even if an assertion panics.
+        struct Kill(u32);
+        impl Drop for Kill {
+            fn drop(&mut self) {
+                let _ = Command::new("kill")
+                    .arg("-9")
+                    .arg(self.0.to_string())
+                    .status();
+            }
+        }
+
+        let mut child = Command::new("sh")
+            .arg("-c")
+            .arg("while :; do :; done")
+            .stdout(Stdio::null())
+            .stderr(Stdio::null())
+            .spawn()
+            .expect("spawn CPU burner");
+        let pid = child.id();
+        let _guard = Kill(pid);
+
+        let stats = crate::handoff_bridge::new_pooler_stats();
+        let mut s = PgbScaler::new(4); // real clk_tck + real /proc sampler via tick()
+        let pids = [pid];
+
+        s.tick(&pids, Instant::now(), &stats); // baseline (no prior sample)
+        let mut scaled_up = false;
+        let mut max_cores = 0.0f64;
+        for _ in 0..4 {
+            std::thread::sleep(Duration::from_millis(600));
+            let action = s.tick(&pids, Instant::now(), &stats);
+            max_cores = max_cores.max(stats.lock().unwrap().per_worker_cores);
+            if action == ScaleAction::Up {
+                scaled_up = true;
+                break;
+            }
+        }
+        let _ = child.kill();
+        let _ = child.wait();
+
+        assert!(
+            max_cores > 0.7,
+            "real /proc sampling should read ~1 core for a busy loop, saw {max_cores}"
+        );
+        assert!(
+            scaled_up,
+            "scaler must decide Up under sustained real CPU load (peak per-worker {max_cores} cores)"
+        );
+    }
+
+    #[test]
+    fn apply_spawn_failure_and_empty_down_are_noops() {
+        let mut extra: Vec<ChildState> = Vec::new();
+        // spawn returns None (e.g. fork failed) → no growth.
+        assert_eq!(
+            apply_scale_action(ScaleAction::Up, &mut extra, |_n| None, |_w| {}),
+            ScaleOutcome::NoOp
+        );
+        assert!(extra.is_empty());
+        // Down with no extras → nothing to reap.
+        assert_eq!(
+            apply_scale_action(ScaleAction::Down, &mut extra, |_n| None, |_w| {}),
+            ScaleOutcome::NoOp
+        );
+    }
+
     #[test]
     fn parse_memtotal_standard_format() {
         let input = "MemTotal:       16384000 kB\nMemFree:        8192000 kB\n";

From fc87fb4eeca61743dbdc14c03f6a814db68b3446 Mon Sep 17 00:00:00 2001
From: Jared Lunde <jared.lunde@gmail.com>
Date: Thu, 4 Jun 2026 12:50:33 -0700
Subject: [PATCH 3/6] test pgbouncer: validate multi-worker with the REAL
 product config (Docker)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Run the committed pgbouncer.ini + the config::pgbouncer_ini-appended lines
(scram auth_query, TLS, unix_socket_dir=, so_reuseport=1) + the exact
setup_pgbouncer_auth SQL as 3 so_reuseport workers in the beyond-pg-test image
(pgbouncer 1.25.2). Result: 3 workers share :5432, 90/90 scram+TLS queries
served across all three, SIGINT reaps one gracefully (60/60 still served, 0
errors). This is the multi-worker deliverable validated against the real config
in the real pgbouncer — the piece the unit tests and the trust-auth rig couldn't.

Surfaced two pre-existing, CI-invisible bugs (the supervisor tests only pgrep
pgbouncer + check the role exists, never authenticate through :5432):
  1. self-signed cert omits client_tls_ca_file → pgbouncer 1.25.2 fatals at
     startup ("failed to load CA: (null)").
  2. setup_pgbouncer_auth never GRANTs USAGE on schema pgbouncer to the
     pgbouncer role → auth_query fails "permission denied for schema pgbouncer"
     for every client.
Both documented in RESULTS.md §Round 4 and worked around in the script; neither
is fixed in product code here (separable from the scaler).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 bench/glidefs-pg/RESULTS.md                 |  26 +++++
 bench/glidefs-pg/multiworker-real-config.sh | 121 ++++++++++++++++++++
 2 files changed, 147 insertions(+)
 create mode 100644 bench/glidefs-pg/multiworker-real-config.sh

diff --git a/bench/glidefs-pg/RESULTS.md b/bench/glidefs-pg/RESULTS.md
index 4a80527..ae2325d 100644
--- a/bench/glidefs-pg/RESULTS.md
+++ b/bench/glidefs-pg/RESULTS.md
@@ -202,3 +202,29 @@ empty cache → cold reads). Result appended on completion.
 Measurement has already paid for itself by **killing two speculative knobs** (zstd, wal_recycle) that
 intuition/the-ZFS-playbook would have shipped. The high-value hypotheses (checkpoint frequency,
 maintenance_io_concurrency, cooldown-under-churn) remain open and need the tailored experiments above.
+
+## Round 4 — multi-worker with the REAL product config (Docker, pgbouncer 1.25.2)
+
+`multiworker-real-config.sh`, run in the `beyond-pg-test` image: the committed
+`packer/files/pgbouncer/pgbouncer.ini` + the lines `config::pgbouncer_ini`
+appends (scram `auth_query`, TLS, `unix_socket_dir =`, `so_reuseport = 1`), the
+exact `setup_pgbouncer_auth` SQL, 3 so_reuseport workers, real scram+TLS traffic.
+
+**Result (after working around the two bugs below):** 3 workers start and share
+`:5432`; 90/90 scram+TLS queries served, distributed across all three (per-worker
+CPU non-zero); SIGINT one → graceful reap (3→2) with 60/60 queries still served,
+0 errors. The multi-worker mechanism + graceful drain work with the real config.
+
+**Two pre-existing bugs this surfaced (NOT from the scaler; uncaught by CI — the
+supervisor tests only `pgrep pgbouncer` and check the role exists, never auth
+through `:5432`):**
+
+1. **Self-signed cert → pgbouncer won't start.** `config::pgbouncer_ini` omits
+   `client_tls_ca_file` for self-signed certs, but pgbouncer 1.25.2 fatals
+   (`TLS setup failed: failed to load CA: (null)`) when a client cert is set
+   without a CA. Bites the self-signed fallback (platform certs carry a CA).
+   Workaround in the script: point the CA at the self-signed cert (its own issuer).
+2. **`pgbouncer` role can't call `get_auth` → all scram auth fails.**
+   `setup_pgbouncer_auth` grants EXECUTE on the function but never `GRANT USAGE ON
+   SCHEMA pgbouncer TO pgbouncer`, so auth_query dies with `permission denied for
+   schema pgbouncer` for every client. One-line fix.
diff --git a/bench/glidefs-pg/multiworker-real-config.sh b/bench/glidefs-pg/multiworker-real-config.sh
new file mode 100644
index 0000000..25953c6
--- /dev/null
+++ b/bench/glidefs-pg/multiworker-real-config.sh
@@ -0,0 +1,121 @@
+#!/usr/bin/env bash
+# Real-config multi-worker validation: the PRODUCT pgbouncer.ini (scram +
+# auth_query + TLS + unix_socket_dir empty + so_reuseport=1) run as N workers,
+# serving real scram+TLS traffic, with a graceful SIGINT reap of one worker.
+#
+# This is the piece the unit tests can't cover and the rig only approximated
+# (the rig used a trust-auth toy ini). Runs inside the beyond-pg-test Docker
+# image (has pgbouncer + postgres). The repo is mounted read-only at /repo so we
+# use the ACTUAL committed packer/files/pgbouncer/pgbouncer.ini base.
+set -uo pipefail
+PGBIN=/usr/lib/postgresql/18/bin
+W=/tmp/mwreal
+rm -rf "$W"; mkdir -p "$W" /var/run/postgresql
+chown -R postgres:postgres "$W" /var/run/postgresql
+
+run_pg() { su postgres -c "$*"; }
+
+# 1. Postgres on :5433 (unix socket), scram password encryption (PG18 default).
+run_pg "$PGBIN/initdb -D $W/pg -A trust -U postgres --no-sync" >/dev/null 2>&1
+cat >> "$W/pg/postgresql.conf" <<EOF
+port = 5433
+unix_socket_directories = '/var/run/postgresql'
+listen_addresses = '127.0.0.1'
+password_encryption = scram-sha-256
+EOF
+cat > "$W/pg/pg_hba.conf" <<EOF
+local all all trust
+host  all all 127.0.0.1/32 trust
+EOF
+run_pg "$PGBIN/pg_ctl -D $W/pg -l $W/pg.log -w start" >/dev/null
+
+# 2. App user with a scram password + a row to read back.
+run_pg "$PGBIN/psql -p 5433 -h /var/run/postgresql -U postgres -q" <<'SQL'
+CREATE ROLE app LOGIN PASSWORD 'apppass';
+CREATE TABLE t(x int);
+INSERT INTO t VALUES (42);
+GRANT SELECT ON t TO app;
+SQL
+
+# 3. The EXACT product pgbouncer auth setup (from supervisor.rs setup_pgbouncer_auth).
+run_pg "$PGBIN/psql -p 5433 -h /var/run/postgresql -U postgres -q" <<'SQL'
+DO $$ BEGIN
+  IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'pgbouncer') THEN
+    CREATE ROLE pgbouncer LOGIN PASSWORD NULL;
+  END IF;
+END $$;
+CREATE SCHEMA IF NOT EXISTS pgbouncer;
+CREATE OR REPLACE FUNCTION pgbouncer.get_auth(p_user text)
+  RETURNS TABLE(username text, password text)
+  SECURITY DEFINER LANGUAGE sql AS $$
+    SELECT usename::text, passwd::text FROM pg_shadow WHERE usename = p_user
+  $$;
+GRANT EXECUTE ON FUNCTION pgbouncer.get_auth(text) TO pgbouncer;
+-- NOTE: the product's setup_pgbouncer_auth OMITS this grant — without it the
+-- pgbouncer role gets "permission denied for schema pgbouncer" and auth_query
+-- fails for every client. Added here so the multi-worker path can be validated.
+GRANT USAGE ON SCHEMA pgbouncer TO pgbouncer;
+SQL
+
+# 4. Ed25519 TLS cert (what beyond-pg self-signs).
+openssl req -x509 -newkey ed25519 -days 1 -nodes -keyout "$W/tls.key" -out "$W/tls.crt" -subj /CN=pgb >/dev/null 2>&1
+chmod 644 "$W/tls.key" "$W/tls.crt"
+
+# 5. REAL pgbouncer.ini = committed product base + the lines config::pgbouncer_ini appends.
+cp /repo/packer/files/pgbouncer/pgbouncer.ini "$W/pgb.ini"
+cat >> "$W/pgb.ini" <<EOF
+
+# Pool sizing — generated by beyond-pg from VM resources (4 vCPU).
+default_pool_size = 16
+max_client_conn = 200
+so_reuseport = 1
+
+# TLS — termination for the public 5432 port (generated by beyond-pg).
+# NOTE: pgbouncer 1.25.2 fatals ("failed to load CA: (null)") if a client cert is
+# set without client_tls_ca_file. A self-signed cert is its own issuer, so we
+# point the CA at the cert itself. config::pgbouncer_ini currently OMITS the CA
+# for self-signed certs — a separate bug that breaks the self-signed fallback.
+client_tls_sslmode = allow
+client_tls_cert_file = $W/tls.crt
+client_tls_key_file = $W/tls.key
+client_tls_ca_file = $W/tls.crt
+EOF
+chown postgres:postgres "$W/pgb.ini"
+echo "=== effective pgbouncer.ini (tail) ==="; tail -16 "$W/pgb.ini"
+
+# 6. Start 3 so_reuseport workers (this is the new behavior: N processes, one port).
+for i in 1 2 3; do run_pg "pgbouncer $W/pgb.ini" >/dev/null 2>&1 & done
+sleep 2
+echo "=== workers alive: $(pgrep -c pgbouncer) (expect 3) ==="
+
+# 7. Drive real scram + TLS traffic through :5432; every query must return 42.
+export PGPASSWORD=apppass
+CONN="host=127.0.0.1 port=5432 user=app dbname=postgres sslmode=require"
+ok=0; fail=0
+for i in $(seq 1 90); do
+  if [ "$($PGBIN/psql "$CONN" -tAc 'SELECT x FROM t' 2>/dev/null)" = "42" ]; then ok=$((ok+1)); else fail=$((fail+1)); fi
+done
+echo "=== scram+TLS queries via 3 so_reuseport workers: ok=$ok fail=$fail ==="
+
+# Per-worker CPU — proves traffic actually spread across the workers, not one.
+echo "--- per-worker cpu ticks (utime+stime) ---"
+for p in $(pgrep pgbouncer); do
+  awk -v p="$p" '{u=$14+$15} END{}' "/proc/$p/stat" 2>/dev/null
+  printf "  pid %-7s ticks=%s\n" "$p" "$(awk '{s=$0; sub(/^.*\) /,"",s); n=split(s,f," "); print f[12]+f[13]}' /proc/$p/stat 2>/dev/null)"
+done
+
+# 8. SIGINT one worker (graceful "safe shutdown"); the rest must keep serving.
+VICTIM=$(pgrep pgbouncer | tail -1)
+echo "=== SIGINT worker $VICTIM (graceful reap) ==="
+kill -INT "$VICTIM"
+sleep 2
+ok2=0; fail2=0
+for i in $(seq 1 60); do
+  if [ "$($PGBIN/psql "$CONN" -tAc 'SELECT x FROM t' 2>/dev/null)" = "42" ]; then ok2=$((ok2+1)); else fail2=$((fail2+1)); fi
+done
+gone=$(kill -0 "$VICTIM" 2>/dev/null && echo NO || echo YES)
+echo "=== after reap: workers=$(pgrep -c pgbouncer) victim_gone=$gone queries ok=$ok2 fail=$fail2 ==="
+
+pkill pgbouncer 2>/dev/null
+run_pg "$PGBIN/pg_ctl -D $W/pg -m immediate stop" >/dev/null 2>&1
+echo DONE

From c64b9654c3b1798bd7fff88c3ac5ea47013ed361 Mon Sep 17 00:00:00 2001
From: Jared Lunde <jared.lunde@gmail.com>
Date: Thu, 4 Jun 2026 13:03:24 -0700
Subject: [PATCH 4/6] feat pgbouncer: warm-start pooler size on cold boot + fix
 two auth/TLS bugs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Warm-start (the restart-resilience gap): a busy box that cold-reboots/crashes
faces a reconnect storm — peak TLS-handshake churn — and the old cold-start path
met it with ONE pooler, then crawled back up under the scaler's up-cooldown
(~90s under-provisioned). children.json is durable across reboots (GlideFS-backed
rootfs), so it already records how many extra workers were live. Cold start now
reads that count and warm-starts the so_reuseport workers to the pre-restart size
(clamped to the current max_workers for resize-safety); the scaler reaps any
excess within minutes if load doesn't justify it. It's a hint, not source-of-
truth — losing it just falls back to one worker. Completes the restart story:
handoff adopts extras (live), cold boot warm-starts them (respawn), snapshot
resume preserves them (hypervisor). warm_start_extra_count() unit-tested incl.
the downsize-clamp case.

Two pre-existing bugs found by the real-config Docker validation, fixed:
  - setup_pgbouncer_auth now GRANTs USAGE ON SCHEMA pgbouncer to the pgbouncer
    role. Without it auth_query fails "permission denied for schema pgbouncer"
    for every client — the pooler couldn't authenticate anyone.
  - config::pgbouncer_ini emits client_tls_ca_file for self-signed certs (pointed
    at the cert itself, its own issuer). pgbouncer 1.25.x FATALs at startup
    without a CA when a client cert is set ("failed to load CA: (null)"), so the
    self-signed fallback never started the pooler. Test updated accordingly.

Re-validated in the beyond-pg-test image: real config (now with both fixes)
serves 90/90 scram+TLS queries across 3 so_reuseport workers, graceful reap.
122 unit tests green; clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 bench/glidefs-pg/multiworker-real-config.sh | 12 ++--
 src/config.rs                               | 15 ++++-
 src/supervisor.rs                           | 73 ++++++++++++++++++++-
 3 files changed, 88 insertions(+), 12 deletions(-)

diff --git a/bench/glidefs-pg/multiworker-real-config.sh b/bench/glidefs-pg/multiworker-real-config.sh
index 25953c6..40e39df 100644
--- a/bench/glidefs-pg/multiworker-real-config.sh
+++ b/bench/glidefs-pg/multiworker-real-config.sh
@@ -51,9 +51,8 @@ CREATE OR REPLACE FUNCTION pgbouncer.get_auth(p_user text)
     SELECT usename::text, passwd::text FROM pg_shadow WHERE usename = p_user
   $$;
 GRANT EXECUTE ON FUNCTION pgbouncer.get_auth(text) TO pgbouncer;
--- NOTE: the product's setup_pgbouncer_auth OMITS this grant — without it the
--- pgbouncer role gets "permission denied for schema pgbouncer" and auth_query
--- fails for every client. Added here so the multi-worker path can be validated.
+-- Schema USAGE — now also emitted by setup_pgbouncer_auth (was missing; without
+-- it auth_query fails "permission denied for schema pgbouncer" for every client).
 GRANT USAGE ON SCHEMA pgbouncer TO pgbouncer;
 SQL
 
@@ -71,10 +70,9 @@ max_client_conn = 200
 so_reuseport = 1
 
 # TLS — termination for the public 5432 port (generated by beyond-pg).
-# NOTE: pgbouncer 1.25.2 fatals ("failed to load CA: (null)") if a client cert is
-# set without client_tls_ca_file. A self-signed cert is its own issuer, so we
-# point the CA at the cert itself. config::pgbouncer_ini currently OMITS the CA
-# for self-signed certs — a separate bug that breaks the self-signed fallback.
+# pgbouncer 1.25.2 fatals ("failed to load CA: (null)") if a client cert is set
+# without client_tls_ca_file. A self-signed cert is its own issuer, so the CA
+# points at the cert itself — config::pgbouncer_ini now emits this for self-signed.
 client_tls_sslmode = allow
 client_tls_cert_file = $W/tls.crt
 client_tls_key_file = $W/tls.key
diff --git a/src/config.rs b/src/config.rs
index 0fadbf1..c6ef753 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -108,6 +108,13 @@ pub fn pgbouncer_ini(tls: &crate::tls::TlsConfig, ram_bytes: u64, vcpus: u32) ->
     out.push_str(&format!("client_tls_key_file = {}\n", tls.key.display()));
     if let Some(ca) = &tls.ca {
         out.push_str(&format!("client_tls_ca_file = {}\n", ca.display()));
+    } else {
+        // pgbouncer (1.25.x) requires client_tls_ca_file whenever a client cert is
+        // set — even for sslmode=allow — and FATALs with "failed to load CA:
+        // (null)" otherwise. A self-signed cert is its own issuer, so point the CA
+        // at the cert itself. This satisfies the requirement without changing
+        // client-cert behavior (sslmode=allow neither requests nor verifies them).
+        out.push_str(&format!("client_tls_ca_file = {}\n", tls.cert.display()));
     }
     out
 }
@@ -495,7 +502,9 @@ mod tests {
     }
 
     #[test]
-    fn pgbouncer_ini_self_signed_omits_ca() {
+    fn pgbouncer_ini_self_signed_points_ca_at_cert() {
+        // pgbouncer FATALs without client_tls_ca_file when a client cert is set, so
+        // a self-signed cert (its own issuer) must point the CA at itself.
         let tls = TlsConfig {
             source: TlsSource::SelfSigned,
             cert: PathBuf::from("/var/lib/postgresql/18/main/beyond/server.crt"),
@@ -506,8 +515,8 @@ mod tests {
         assert!(out.contains("client_tls_sslmode = allow"));
         assert!(out.contains("client_tls_cert_file"));
         assert!(
-            !out.contains("client_tls_ca_file"),
-            "self-signed has no CA: {out}"
+            out.contains("client_tls_ca_file = /var/lib/postgresql/18/main/beyond/server.crt"),
+            "self-signed CA points at the cert itself: {out}"
         );
     }
 
diff --git a/src/supervisor.rs b/src/supervisor.rs
index fc35979..bce762d 100644
--- a/src/supervisor.rs
+++ b/src/supervisor.rs
@@ -338,6 +338,14 @@ fn apply_scale_action(
     }
 }
 
+/// How many extra pooler workers to warm-start on cold boot: the count that was
+/// live before the restart (from the durable children.json), clamped to the
+/// current box's worker ceiling minus worker 0 — so a box that was resized
+/// smaller during the restart doesn't over-spawn.
+fn warm_start_extra_count(prior_live_extras: usize, vcpus: u32) -> usize {
+    prior_live_extras.min(crate::config::pgbouncer_max_workers(vcpus).saturating_sub(1))
+}
+
 // ---------------------------------------------------------------------------
 // Public entry point
 // ---------------------------------------------------------------------------
@@ -506,6 +514,30 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
         }
     } else {
         // Cold-start path: original boot sequence.
+        //
+        // Warm-start hint: children.json is durable across reboots (the rootfs is
+        // GlideFS-backed), so it records how many extra pooler workers were live
+        // before this restart. Capture that COUNT now — before our first persist
+        // overwrites the file — so we can bring the pooler back up to its
+        // pre-restart size instead of meeting the post-restart reconnect storm
+        // (peak TLS-handshake churn) with a single worker and crawling back up
+        // under the scaler's cooldowns. The pids in it are dead; only the count
+        // matters, and the scaler corrects from there. Best-effort: a fresh box
+        // has no prior file → 0. It's a hint, not source-of-truth — losing it just
+        // falls back to starting at one worker.
+        #[cfg(target_os = "linux")]
+        let warm_start_extras =
+            crate::children::PersistedChildren::load(&crate::children::state_dir())
+                .map(|prior| {
+                    PGB_EXTRA_NAMES
+                        .iter()
+                        .filter(|&&name| prior.get(name).is_some())
+                        .count()
+                })
+                .unwrap_or(0);
+        #[cfg(not(target_os = "linux"))]
+        let warm_start_extras = 0usize;
+
         boot::do_boot(&cfg).await?;
 
         let mut pg = ChildState::new("postgres");
@@ -550,8 +582,28 @@ async fn run_inner_inner(role: MaybeRole) -> Result<(), Box<dyn std::error::Erro
         #[cfg(target_os = "linux")]
         persist_child(&mut persisted, &pgb)?;
         pgb_state = pgb;
-        // Start with ONE pooler. The PgbScaler grows pgb_extra reactively under load
-        // (so_reuseport) and reaps it back down — see the scaler tick in the run loop.
+        // Warm-start the extra so_reuseport workers to the pre-restart size (clamped
+        // to the current max_workers in case the box was resized smaller). They share
+        // :5432 with worker 0, so a reconnect storm is absorbed across cores from the
+        // first second; the scaler reaps any excess within minutes if load doesn't
+        // justify it. Without this, a busy box drops to ONE pooler at exactly the
+        // moment of peak handshake churn and climbs back only under the up-cooldown.
+        let want_extras = warm_start_extra_count(warm_start_extras, cfg.vcpus);
+        for &name in PGB_EXTRA_NAMES.iter().take(want_extras) {
+            let mut w = ChildState::new(name);
+            match spawn_pgbouncer(&mut w, &log_tx) {
+                Ok(()) => {
+                    #[cfg(target_os = "linux")]
+                    persist_child(&mut persisted, &w)?;
+                    pgb_extra.push(w);
+                }
+                Err(e) => warn!("warm-start: failed to spawn pooler worker '{name}': {e}"),
+            }
+        }
+        if want_extras > 0 {
+            info!("warm-start: restored {want_extras} extra pooler worker(s) to pre-restart size");
+        }
+        // The PgbScaler grows/shrinks pgb_extra reactively from here (so_reuseport).
 
         cdc_state = if cfg.pg_tier != PgTier::Replica && cfg.cdc_enabled {
             let mut s = ChildState::new("beyond-pg-cdc");
@@ -1734,6 +1786,11 @@ async fn setup_pgbouncer_auth() -> Result<(), crate::pg::PgError> {
     )
     .await?;
 
+    // USAGE on the schema is REQUIRED for the pgbouncer role to call get_auth at
+    // all — without it auth_query fails with "permission denied for schema
+    // pgbouncer" for every client, and no one can connect through the pooler.
+    // (EXECUTE on the function is not enough; the caller also needs schema USAGE.)
+    pg::psql("GRANT USAGE ON SCHEMA pgbouncer TO pgbouncer").await?;
     pg::psql("GRANT EXECUTE ON FUNCTION pgbouncer.get_auth(text) TO pgbouncer").await?;
 
     Ok(())
@@ -2011,6 +2068,18 @@ mod tests {
 
     // --- apply_scale_action: spawn/reap mutation with injected process I/O ---
 
+    #[test]
+    fn warm_start_clamps_to_current_max_workers() {
+        // Same-size box: restore exactly what was live (max_workers(64)=8 → cap 7).
+        assert_eq!(warm_start_extra_count(4, 64), 4);
+        // Downsized during restart (now 8 vCPU → max_workers 2 → ≤1 extra): clamp down.
+        assert_eq!(warm_start_extra_count(7, 8), 1);
+        // Tiny box (max_workers 1 → 0 extras): never warm-start past a single pooler.
+        assert_eq!(warm_start_extra_count(3, 2), 0);
+        // Fresh box / nothing prior: start at one worker.
+        assert_eq!(warm_start_extra_count(0, 64), 0);
+    }
+
     #[test]
     fn apply_up_spawns_workers_in_name_order() {
         let mut extra: Vec<ChildState> = Vec::new();

From b6ccbaa1040dfce9f82f086ce53ef874fd63977f Mon Sep 17 00:00:00 2001
From: Jared Lunde <jared.lunde@gmail.com>
Date: Thu, 4 Jun 2026 13:19:07 -0700
Subject: [PATCH 5/6] test pgbouncer: warm-start-after-restart e2e (CI) +
 restart/count harness helpers
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Adds supervisor_warm_starts_pooler_after_restart: boot → seed children.json with
an extra worker (simulating a scaled-up box) → docker restart (cold start, same
container so the writable layer persists like a reboot) → assert the supervisor
warm-starts back to 2 pooler workers. New RunningContainer helpers: pgbouncer_count,
wait_pgbouncer_count, restart.

CI-only, like the 11 sibling supervisor lifecycle tests — the unprivileged harness
can't boot the supervisor on a host with a restrictive seccomp profile, and running
it with --privileged is unsafe on the shared homelab node (the supervisor's boot
reserves host hugepages and writes host sysctls — kernel state shared with
production). The warm-start LOGIC is unit-tested (warm_start_extra_count, incl. the
downsize clamp); this exercises the full boot→restart→restore wiring in CI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 tests/supervisor.rs | 110 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)

diff --git a/tests/supervisor.rs b/tests/supervisor.rs
index c971b06..2c2e3f8 100644
--- a/tests/supervisor.rs
+++ b/tests/supervisor.rs
@@ -235,6 +235,43 @@ impl RunningContainer {
         self.wait_for(&["pgrep", "-x", "pgbouncer"], timeout)
     }
 
+    /// Number of live pgbouncer processes (worker 0 + scaled/warm-started extras).
+    fn pgbouncer_count(&self) -> u32 {
+        let out = self.exec(&["pgrep", "-c", "-x", "pgbouncer"]);
+        String::from_utf8_lossy(&out.stdout)
+            .trim()
+            .parse()
+            .unwrap_or(0)
+    }
+
+    /// Poll until exactly `target` pgbouncer processes are live (or timeout).
+    fn wait_pgbouncer_count(&self, target: u32, timeout: Duration) -> bool {
+        let deadline = std::time::Instant::now() + timeout;
+        while std::time::Instant::now() < deadline {
+            if self.pgbouncer_count() == target {
+                return true;
+            }
+            std::thread::sleep(Duration::from_secs(1));
+        }
+        false
+    }
+
+    /// `docker restart` — SIGTERM the supervisor, then re-run the entrypoint
+    /// (a fresh cold start in the SAME container, so the writable layer —
+    /// children.json, PGDATA — persists, exactly like a real reboot).
+    fn restart(&self, timeout: Duration) {
+        let secs = timeout.as_secs().to_string();
+        let out = std::process::Command::new("docker")
+            .args(["restart", "-t", &secs, &self.id])
+            .output()
+            .expect("docker restart");
+        assert!(
+            out.status.success(),
+            "docker restart failed: {}",
+            String::from_utf8_lossy(&out.stderr)
+        );
+    }
+
     /// Poll until the pgbouncer role exists — proxy for post_start completion.
     fn wait_post_start_done(&self, timeout: Duration) -> bool {
         let deadline = std::time::Instant::now() + timeout;
@@ -632,6 +669,79 @@ fn supervisor_lifecycle_primary() {
     );
 }
 
+/// Warm-start: a busy box that cold-restarts must come back at its pre-restart
+/// pooler size, not a single worker — the post-restart reconnect storm is peak
+/// handshake churn, the worst moment to be under-provisioned. children.json is
+/// durable across reboots (GlideFS-backed rootfs) and records the extra workers;
+/// cold start respawns that many.
+///
+/// CI-only (like the sibling supervisor lifecycle tests): uses the standard
+/// unprivileged harness. Assumes a CI runner with >= 8 cores so `read_vcpus` →
+/// `pgbouncer_max_workers >= 2` (so a warm-started extra is within the cap).
+/// NOTE: do NOT run this with `--privileged` on the shared homelab host — the
+/// supervisor's boot reserves host hugepages and writes host sysctls, mutating
+/// kernel state shared with production. That's exactly why the harness is
+/// unprivileged; the trade-off is this test can only boot in CI's permissive
+/// Docker, not on a host with a restrictive seccomp profile.
+#[test]
+#[ignore = "requires Docker + musl target + beyond-pg-test:latest image (>= 8 cores)"]
+fn supervisor_warm_starts_pooler_after_restart() {
+    let _guard = DOCKER.lock().unwrap_or_else(|e| e.into_inner());
+    let Some(harness) = SupervisorHarness::new() else {
+        return;
+    };
+    let container = harness.start(&[]);
+
+    assert!(
+        container.wait_postgres_ready(Duration::from_secs(90)),
+        "postgres not ready\n{}",
+        container.logs()
+    );
+    assert!(
+        container.wait_post_start_done(Duration::from_secs(30)),
+        "post_start did not complete\n{}",
+        container.logs()
+    );
+    assert!(
+        container.wait_pgbouncer_up(Duration::from_secs(30)),
+        "pgbouncer not up\n{}",
+        container.logs()
+    );
+    assert_eq!(
+        container.pgbouncer_count(),
+        1,
+        "expected a single pooler at idle before the restart"
+    );
+
+    // Simulate a box that had scaled up to 2 workers before the restart: record an
+    // extra worker in the durable children.json the scaler would have persisted.
+    let inject = container.exec(&[
+        "sh",
+        "-c",
+        "echo '{\"version\":1,\"pgbouncer-1\":{\"pid\":424242,\"starttime\":1}}' \
+         > /var/lib/beyond-pg/state/children.json",
+    ]);
+    assert!(
+        inject.status.success(),
+        "failed to inject children.json: {}",
+        String::from_utf8_lossy(&inject.stderr)
+    );
+
+    // Cold-restart the supervisor; warm-start must restore the extra worker.
+    container.restart(Duration::from_secs(30));
+    assert!(
+        container.wait_postgres_ready(Duration::from_secs(90)),
+        "postgres not ready after restart\n{}",
+        container.logs()
+    );
+    assert!(
+        container.wait_pgbouncer_count(2, Duration::from_secs(45)),
+        "warm-start should restore 2 pooler workers after restart, saw {}\n{}",
+        container.pgbouncer_count(),
+        container.logs()
+    );
+}
+
 /// Supervisor restarts postgres after a crash (SIGKILL to the postmaster).
 ///
 /// The backoff on first crash is 100 ms, so the restart is nearly immediate.

From 588614a824ded6f2e04fce187d52b9c6a115908e Mon Sep 17 00:00:00 2001
From: Jared Lunde <jared.lunde@gmail.com>
Date: Thu, 4 Jun 2026 14:10:00 -0700
Subject: [PATCH 6/6] docs: README pooler bullet reflects reactive multi-worker
 sizing

The pooler is no longer a single static process: it runs one worker when idle
and adds so_reuseport workers across cores under connection-handshake load, then
reaps them. The count survives restarts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 8211794..6e06339 100644
--- a/README.md
+++ b/README.md
@@ -17,7 +17,7 @@ Forks boot in seconds against a CoW snapshot of `/var/lib/postgresql`. Postgres
 ## What it does
 
 - **Forks with the substrate.** Every byte under `/var/lib/postgresql` snapshots atomically; the new box boots on a CoW copy. No `pg_dump`, no replication topology, no wait.
-- **PgBouncer on the front.** Transaction pooling on `:5432`; Postgres itself listens on the loopback only.
+- **PgBouncer on the front, sized to load.** Transaction pooling and TLS termination on `:5432`; Postgres listens on the loopback only. The pooler runs one worker when idle and adds `so_reuseport` workers across cores as connection-handshake load rises, then reaps them when it falls. The worker count survives restarts.
 - **Logical decoding from day one.** `wal_level = logical`, `max_wal_senders = 10`, `max_replication_slots = 10`. No primary restart to enable CDC later.
 - **Standard extensions, pinned.** pgvector, pgvectorscale, PostGIS, pg_cron, pg_partman, pg_jsonschema, hypopg, pg_repack, pg_search, pg_stat_statements, pg_trgm, auto_explain.
 - **Beyond extensions on the same volume.** `beyond-auth` and `beyond-queue` ship in the image and live under their own schemas in your database. Forking your DB forks their state automatically.