diff --git a/CHANGELOG.md b/CHANGELOG.md index 3064749..021f35d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -85,6 +85,7 @@ Changes in this section will be promoted to a dated release entry on merge to `m ### Fixed +- **`deploy-new-site.sh` — automated AnythingLLM deploys now bootable (2026-07-15)** — the script pushed `JWT_SECRET`/`ADMIN_EMAIL`/Spaces creds but none of the other fail-loud-required AnythingLLM values, so an automated deploy produced a container that refused to boot (the manual `DEPLOYMENT_GUIDE.md` flow was unaffected). New `anythingllm-docker` branch in Phase 2 pushes `ANYTHINGLLM_IMAGE` (from env — the private registry namespace is never committed), `EMBEDDING_ENGINE` (default `openrouter`) + optional `EMBEDDING_MODEL_PREF`, and mints the per-customer capped `OPENROUTER_API_KEY` via `scripts/provision-openrouter-key.sh` (cap override `OPENROUTER_LIMIT_USD`, default $50); loud warnings when a value can't be set automatically. Also adds a "Compliance & data handling" facts table to [`docs/CUSTOMER_INSTANCE_PROVISIONING.md`](docs/CUSTOMER_INSTANCE_PROVISIONING.md) (tenancy, locality, encryption, retention, subprocessors, framework mapping) backing the customer-facing terms. **Verification**: `bash -n` + `shellcheck` clean. - **`otel-agent/config.yaml` — syslog parser matched zero Ubuntu 24.04 lines; agent flooded SigNoz with its own error spam (2026-06-10, #WeOwnVer v4.1.2.2)** — the `filelog/syslog` receiver's `regex_parser` assumed RFC3164 timestamps (`Jun 10 07:00:33`), but Ubuntu 24.04 rsyslog writes RFC3339 (`2026-06-10T07:00:33.179378+00:00 host unit[pid]: msg`), so **every** syslog/auth.log line failed to parse and the collector logged 2 ERROR lines + Go stack traces per input line (84–504/min on int-s004; ~24% of that host's log rows in the 2026-06-10 incident export were the agent's own stanza v0.114.0 stack traces — `transformer.go:100` / `reader.go:214`). Reproduced and fixed against `otelcol-contrib:0.114.0` (the deployed version) using real incident log lines: new regex parses the three real shapes (`unit[pid]:`, `kernel:` no-pid, `(udev-worker)[pid]:`) with gotime RFC3339 timestamp parsing (microsecond-accurate; repro: 9/9 lines parsed + 2 garbage lines passed through, 0 error lines, was 18), plus **`on_error: send_quiet`** so any future unmatched line passes through as raw body logged at DEBUG — the agent can never ERROR-spam per line again. Also hardened the previously-ineffective **`filter/skip_self`** (its `container.name`/`log.file.name` conditions never matched docker's `-json.log` paths, which is how the self-spam reached SigNoz): added a body-signature condition matching the collector's own tab-separated zap console format, starving the feedback loop even during future agent error bursts (e.g. exporter outages). `otelcol validate` passes on 0.114.0. Rollout: `scripts/deploy-otel-fleet.sh` (canary int-s004-anythingllm → `--tag weown-ai`). **Compliance**: NIST CSF 2.0 DE.CM-1 (signal fidelity); CIS v8 8.5, 8.11. - **Propagated the INT-S004 / `anythingllm-docker` deploy-bug battery across all `*-docker` copier templates (2026-06-03)** — the AnythingLLM rebuild surfaced a set of latent first-boot/deploy bugs (the entries below) that the other templates inherited from the same authoring patterns. Scanned every template (and rendered site) and applied only the bug classes actually present in each. cloud-init + deploy + scripts were re-validated per template: `grep -cP '[^\x00-\x7F]'` = 0 on every cloud-init, copier render (5/6; `wordpress-docker` has a pre-existing copier-`choices` render incompatibility, and `searxng-docker` needed a one-line `{% raw %}` wrap on a `docker ps --format '{{.Names}}'` Go-template that previously blocked rendering — same pattern `signoz-docker` already used), simulated-`templatefile()` YAML parse, `bash -n` on all extracted/rendered shell, and `pre-commit` (gitleaks/yamllint/shellcheck/markdownlint) all green. - **`keycloak-docker`** — B1 non-ASCII scrub of cloud-init; B3 kebab-case `infisical login --client-id/--client-secret` (cloud-init `infisical-auth.sh` + cron + the inline-SSH remote restore in `scripts/restore.sh`); B4 `export INFISICAL_TOKEN="$(infisical login … --plain --silent)"` headless capture (auth helper now sourced into the compose-up shell; the redundant standalone auth `runcmd` dropped); B5 dropped `awscli` from apt `packages:` + added `unzip` and the official AWS CLI v2 installer in `runcmd`; B6 pre-create `/var/log/caddy` (0777) in `runcmd` before compose-up. *(B2 N/A — its embedded `backup.sh` already used single-`$`; the only `$$` is the legitimate Compose `$$?` healthcheck escape. B7/B8 N/A.)* diff --git a/docs/CUSTOMER_INSTANCE_PROVISIONING.md b/docs/CUSTOMER_INSTANCE_PROVISIONING.md index 3c78bcf..6f89094 100644 --- a/docs/CUSTOMER_INSTANCE_PROVISIONING.md +++ b/docs/CUSTOMER_INSTANCE_PROVISIONING.md @@ -136,6 +136,23 @@ the teardown commands. --- +## Compliance & data handling (per customer instance) + +Technical facts that back the customer-facing terms (the legal documents +themselves — ToS / AUP / DPA — are maintained outside this repo): + +| Property | Fact | +|---|---| +| **Tenancy** | Single-tenant: dedicated droplet, dedicated Infisical project + Machine Identity, dedicated LLM key. No shared compute, storage, or credentials between customers. | +| **Data locality** | Documents, embeddings (LanceDB), and chat history live on the customer's droplet volume; backups in DO Spaces under a per-project prefix. Region chosen at provisioning. | +| **Encryption** | TLS 1.3 in transit (Caddy/Let's Encrypt); backups offloaded to DO Spaces (SSE); tofu state SSE-C encrypted. Secrets never on disk (Infisical runtime injection). | +| **LLM processing** | Chat prompts + retrieved document context are sent to the configured LLM provider (OpenRouter) at question time — per-customer key, hard monthly cap. Model choice is pinned by WeOwn (`admin`-held config). | +| **Access** | Customer holds a `manager` account (their data/team). WeOwn holds `admin` + SSH (ops keys via `OPS_AUTHORIZED_KEYS`, auditable + revocable per deploy). | +| **Backups & retention** | Daily GFS backups (30d/12mo/yearly). On cancellation: final backup retained **60 days**, then deleted from Spaces; droplet, reserved IP, Infisical project, and the customer's OpenRouter key destroyed/revoked at deprovision. | +| **Monitoring** | Host metrics + reverse-proxy access logs → SigNoz (operational telemetry only; document contents are not shipped). DO resource alerts. | +| **Subprocessors** | DigitalOcean (compute/storage/backups), Infisical (secret management), OpenRouter → underlying model providers (LLM inference), SigNoz (telemetry), Let's Encrypt (TLS issuance). | +| **Framework mapping** | Controls map to NIST CSF 2.0 (PR.DS, PR.AC, DE.CM), CIS v8 IG1 (3.11, 4.1), ISO 27001-ready (A.5.17, A.8.24) — see [`docs/COMPLIANCE_ROADMAP.md`](COMPLIANCE_ROADMAP.md) for the phased program. | + ## Related - [`anythingllm-docker/DEPLOYMENT_GUIDE.md`](../anythingllm-docker/DEPLOYMENT_GUIDE.md) — authoritative deploy reference. diff --git a/scripts/deploy-new-site.sh b/scripts/deploy-new-site.sh index c6957ad..f136b18 100755 --- a/scripts/deploy-new-site.sh +++ b/scripts/deploy-new-site.sh @@ -268,6 +268,46 @@ else infisical secrets set POSTGRES_PASSWORD="$POSTGRES_PASSWORD" --projectId="$PROJECT_ID" --env=prod --silent infisical secrets set KC_DB_PASSWORD="$POSTGRES_PASSWORD" --projectId="$PROJECT_ID" --env=prod --silent success "Pushed duplicated secrets: POSTGRES_PASSWORD + KC_DB_PASSWORD" + elif [[ "$TEMPLATE" == "anythingllm-docker" ]]; then + # AnythingLLM's compose is fail-loud on ANYTHINGLLM_IMAGE, EMBEDDING_ENGINE, + # OPENROUTER_API_KEY, and JWT_SECRET — a deploy without them refuses to boot. + # JWT_SECRET is already pushed above; handle the other three here so the + # automated path produces a bootable instance (previously these were silent + # gaps that only the manual DEPLOYMENT_GUIDE flow covered). + log "Pushing AnythingLLM required config..." + + # ANYTHINGLLM_IMAGE — carries the private registry namespace, so it comes + # from the environment (never a committed default), e.g. + # export ANYTHINGLLM_IMAGE=reg.mini.dev//anythingllm: + if [[ -n "${ANYTHINGLLM_IMAGE:-}" ]]; then + infisical secrets set ANYTHINGLLM_IMAGE="$ANYTHINGLLM_IMAGE" --projectId="$PROJECT_ID" --env=prod --silent + success "Pushed ANYTHINGLLM_IMAGE" + else + warn "ANYTHINGLLM_IMAGE not set in env — the container will refuse to boot until you set it in Infisical (pinned tag, e.g. reg.mini.dev//anythingllm:)" + fi + + # Embedding config — must match whatever built the LanceDB vectors; for a + # brand-new instance the fleet default is openrouter. Overridable via env. + infisical secrets set EMBEDDING_ENGINE="${EMBEDDING_ENGINE:-openrouter}" --projectId="$PROJECT_ID" --env=prod --silent + success "Pushed EMBEDDING_ENGINE (${EMBEDDING_ENGINE:-openrouter})" + if [[ -n "${EMBEDDING_MODEL_PREF:-}" ]]; then + infisical secrets set EMBEDDING_MODEL_PREF="$EMBEDDING_MODEL_PREF" --projectId="$PROJECT_ID" --env=prod --silent + success "Pushed EMBEDDING_MODEL_PREF" + fi + + # OPENROUTER_API_KEY — mint a per-customer, budget-capped key via the + # provisioning helper (reads OPENROUTER_PROVISIONING_KEY from the + # operator-tools Infisical project; cap defaults to \$50/mo, override with + # OPENROUTER_LIMIT_USD). Non-fatal on failure, but loud: the deploy will + # fail-loud at boot until the key exists. + if bash "$SCRIPT_DIR/provision-openrouter-key.sh" \ + --customer "$SITE_NAME" \ + --project-id "$PROJECT_ID" \ + --limit-usd "${OPENROUTER_LIMIT_USD:-50}"; then + success "Minted per-customer OpenRouter key (capped \$${OPENROUTER_LIMIT_USD:-50}/mo)" + else + warn "Could not mint the OpenRouter key automatically — set OPENROUTER_API_KEY in the project manually (or fix OPENROUTER_PROVISIONING_KEY in operator-tools and re-run scripts/provision-openrouter-key.sh). The container will not boot without it." + fi fi # Get shared secrets from operator-tools