From 3872727e7dfd94f2d78c37eba9f291062961a769 Mon Sep 17 00:00:00 2001 From: Nik Date: Wed, 15 Jul 2026 18:04:16 -0600 Subject: [PATCH] =?UTF-8?q?docs(fleet):=20revise=20=C2=A76=20=E2=80=94=20n?= =?UTF-8?q?o=20rendered=20dirs=20in=20git;=20registry=20+=20ephemeral=20re?= =?UTF-8?q?nder=20+=20state-per-tenant?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Supersedes the private-repo-of-rendered-dirs proposal after industry research (2025-26 consensus) + vault prior art (Komodo git-is-truth verdict, OpenTofu MAIT state/drift loop, s004 tombstone pathology): git holds template code + a small private tenants.yaml registry ONLY; copier render becomes an ephemeral CI/deploy artifact; tofu state-per-tenant via generated backend keys in weown-prod-state; day-2 via DO-tag dynamic inventory (no checked-in inventories); orchestration graduates operator-wrapper -> Terragrunt Stacks / CI render -> Terrateam/Digger -> DB control plane. Existing site dirs stay as reference; no new customer dirs get committed. --- docs/FLEET_OPERATIONS_DESIGN.md | 60 ++++++++++++++++++++++++++------- 1 file changed, 48 insertions(+), 12 deletions(-) diff --git a/docs/FLEET_OPERATIONS_DESIGN.md b/docs/FLEET_OPERATIONS_DESIGN.md index 6aa6595..31ce47c 100644 --- a/docs/FLEET_OPERATIONS_DESIGN.md +++ b/docs/FLEET_OPERATIONS_DESIGN.md @@ -93,18 +93,54 @@ with these hard rules (P1 — encode in the otel-agent config, not in prose): 5. This guarantee is a **contract term** (the "errors-only vs proactive" support options) — the observability config is the evidence; keep it in git. -## 6. Fleet registry + the private-repo boundary (⚠️ structural) - -**Rendered customer sites cannot live in this public repo** — a `sites//` -dir per customer leaks customer names/domains (and at 100s, bloats the repo). -Decision needed before customer #2: - -- **P1 (recommended):** a **private** `weown-fleet` repo holding rendered site - dirs + the **fleet registry** (one row per customer: slug, droplet id, - reserved IP, Infisical project id, ring tag, plan status, Stripe ids, dates). - The public repo stays the template/tooling home. -- The registry is the join point for everything above: bulk Infisical updates, - ring membership, billing lifecycle, deprovision bookkeeping. +## 6. Fleet registry + IaC topology (⚠️ structural — REVISED 2026-07-15) + +> Revision: the original draft proposed a private repo of **rendered site +> dirs**. Industry research (2025–26) and our own history say otherwise — +> rendered dir-per-tenant is the pattern mature teams migrate *away* from +> (template fix = N-dir churn; git becomes a database whose failure mode is +> merge conflicts; our own retired `sites/s004/` tombstone misled both a human +> PR and an AI reviewer at a fleet size of four). **Do not check in 100s of +> rendered copies.** + +The consensus pattern, adopted here: + +1. **Git holds exactly two things:** the template/module code (this repo, + public) and a small **tenant registry** — `tenants.yaml`, one entry per + customer (~6 params: slug, domain, region, size, ring, plan status + + droplet/IP/Infisical-project/Stripe ids) — in a **private** `weown-fleet` + repo. No rendered files. Secrets stay in Infisical, never the registry. +2. **Rendered output is a build artifact, not source:** CI (or the operator + wrapper) renders the copier template into an **ephemeral workdir** per + tenant at deploy time, applies, and discards. Deterministic render means + the rendered tree carries no information the registry + template don't. +3. **State-per-tenant, generated backend config:** each tenant gets its own + tofu state key in the existing `weown-prod-state` Spaces bucket + (`tenants//terraform.tfstate`, SSE-C). Blast radius = one customer; + plans stay fast and parallel. (One giant `for_each` state and + workspace-per-tenant are both dead ends past ~40–50 tenants — plan-time + growth, single lock, fleet-wide blast radius.) +4. **Day-2 has no inventory at all:** Ansible **dynamic inventory from DO + tags** (`community.digitalocean` plugin) — the droplet fleet IS the + inventory; ring tags select rollout cohorts. This is the mature form of + what `manage-droplets.sh` already does. +5. **Orchestration graduates in steps:** operator wrapper looping the + registry (now) → **Terragrunt Stacks** (GA 2025 — generated units from a + stack definition, purpose-built to kill duplicated per-unit files) or + CI-time render → PR-gated per-tenant applies (Terrateam/Digger — both + self-hostable) → a DB-backed control plane only when onboarding volume + demands an API (the Omnistrate-style endgame §7 already assumes). + +This aligns with the ecosystem's own in-flight direction: the OpenTofu-MAIT +plan's loop ("render → plan → human approval → exec → **record state → +drift-detect**") treats tofu state as the per-instance record with render as +an ephemeral pipeline step, and the Komodo evaluation's verdict (git holds +truth; thin execution surface; no control plane that can drift from git) +is preserved — the registry is *in git*, the renders are not. + +**Existing sites:** the four current `sites//` dirs stay as-is +(reference deployments) until migrated; **no new customer site dirs get +committed** — customer #1 onward uses the registry + ephemeral-render path. ## 7. Billing-driven lifecycle (design for automation now)