From bcd35d548925b4b2ce389203c93a4b9d7febea81 Mon Sep 17 00:00:00 2001 From: henrycgbaker Date: Thu, 30 Apr 2026 13:06:51 +0200 Subject: [PATCH 1/3] docs: add annotation bootstrap and containerisation design docs Introduces two new design documents: - annotation-bootstrap.md: Docker stack lifecycle, compose distribution, first-run/upgrade/uninstall UX, prod bootstrap, and infra error taxonomy for pragmata annotation - containerisation-and-deployment.md: artefact strategy, per-tool deployment shape, persistence/backup, and TLS considerations Also adds both to the design docs index. --- docs/design/README.md | 2 + docs/design/annotation-bootstrap.md | 234 ++++++++++++++++++ .../design/containerisation-and-deployment.md | 234 ++++++++++++++++++ 3 files changed, 470 insertions(+) create mode 100644 docs/design/annotation-bootstrap.md create mode 100644 docs/design/containerisation-and-deployment.md diff --git a/docs/design/README.md b/docs/design/README.md index e3b14804..9d550080 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -23,3 +23,5 @@ Design documents focus on (this is guidance, not a required section template): - [Synthetic Test Set](synthetic-test-set.md) — Query generation, prompt variation, and response collection - [Package Contracts](infra-package-contracts.md) — Contract layer layout: canonical types, schemas, path conventions, and config - [Packaging and Invocation Surface](packaging-invocation-surface.md) — Package structure, module boundaries, and invocation bindings +- [Annotation Bootstrap](annotation-bootstrap.md) — Annotation-only stack lifecycle, compose distribution, prod bootstrap, cross-platform runtime +- [Containerisation and Deployment](containerisation-and-deployment.md) — Artefact strategy (PyPI vs container image), per-tool deployment shape, persistence/backup, TLS diff --git a/docs/design/annotation-bootstrap.md b/docs/design/annotation-bootstrap.md new file mode 100644 index 00000000..09409125 --- /dev/null +++ b/docs/design/annotation-bootstrap.md @@ -0,0 +1,234 @@ +# Annotation Bootstrap & Stack Lifecycle + +Status: Draft +Related: +- ADR-0007 (packaging & invocation surface) +- ADR-0003 (infra: self-hosted only) +- ADR-0012 (install & bootstrap UX) [draft] +- [`config-and-settings.md`](config-and-settings.md) - shared settings/config resolution that this doc builds on + +## Purpose + +Stack composition, compose distribution, first-run/upgrade/uninstall lifecycle, prod bootstrap, cross-platform runtime, and infra-specific error UX for `pragmata annotation`. Other tools (`querygen`, `eval`) don't ship infra; this doc is annotation-only. + +Shared concerns (config resolution, settings precedence, secrets, generic CLI error UX) live in [`config-and-settings.md`](config-and-settings.md). + +## Guiding principle (annotation-specific) + +**Dev tooling ≠ production tooling.** The `Makefile` is dev-only; the prod install path goes through the CLI (`pragmata annotation up`). They share the same shipped compose file but apply different overrides. + +> The `Makefile` targets (`docker-up`, `docker-down`, `test-stack`) bind to `deploy/annotation/docker-compose.dev.yml` and assume a cloned repo + `make` (= non-starter for Windows, PyPI installs, and unattended/prod environments). The CLI command `pragmata annotation up` is the single end-user entry point and must work identically across all three environments. + +SOTA for PyPI-distributed CLIs that wrap Docker stacks (Supabase CLI, Airbyte `abctl`, Dagster, Prefect, MLflow, Argilla itself) converges on a few points: +- install is side-effect free +- first-run bootstrap is an idempotent single command +- upgrade is the #1 pain point (we mostly skip this) +- dev ≠ prod + +## 1. Stack composition + +The runtime compose file ships as **package data under `src/pragmata/annotation/docker-compose.yml`** and is resolved at runtime via `importlib.resources`. Users never see or edit the YAML - all supported customisation flows through CLI flags / env / config (§2.3). This is the "locked compose" model (option Y in §2.1); see that section for the full rationale and rejected alternatives. + +All services bundled by default (zero-config principle, see [`config-and-settings.md`](config-and-settings.md) principle 4). Each backing service (postgres/elasticsearch/redis) is opt-out-able via a Compose profile (§2.3). + +**Single shipped compose file, not a dev/prod pair.** Surveyed PyPI-distributed Docker-wrapping tools (Supabase, Airbyte `abctl`, Airflow, Dagster, Prefect, MLflow) all converge on **one shipped compose artefact**. Nobody ships parallel `docker-compose.prod.yml` + `docker-compose.dev.yml` side by side because it forces users to answer "which one do I run?" before doing anything. The split here is: + +- **Shipped (package data):** `src/pragmata/annotation/docker-compose.yml` - production-first: pinned tags, env-driven credentials (no hardcoded defaults), sensible resource defaults, localhost-only port bindings. This is the file `pragmata annotation up` resolves at runtime. +- **Contributor dev (cloned repo):** `deploy/annotation/docker-compose.dev.override.yml` (proposed - does not exist yet) - layered on top of the shipped file via Makefile targets using `docker compose -f ... -f ...`. Typical contents: well-known default creds (`argilla`/`1234`), stdout logging, looser health-check timing, exposed debug ports. + +End users (`pragmata annotation up`) only ever touch the shipped (package-data) file via CLI flags. The dev override is exclusively for contributors working in a cloned repo. + +This matches Airflow's [documented pattern](https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html) ("compose file is a quick-start, not a production config" - with dev niceties layered via overrides) and keeps us from maintaining two sources of truth. + +Migration steps from today's single `deploy/annotation/docker-compose.dev.yml`: + 1. Extract prod-safe defaults into `src/pragmata/annotation/docker-compose.yml` as package data (the new SSOT for runtime) + 2. Strip dev-only overrides into `deploy/annotation/docker-compose.dev.override.yml` + 3. Update Makefile targets to stack both via `docker compose -f ... -f ...` + +## 2. Compose file distribution + +Two axes to decide: (a) where the compose file the daemon reads actually lives, (b) how many "bundles" the shipped file supports. + +### 2.1 Where the compose file lives (daemon-reads-from) + +Three options were considered: + +| Option | What it is | Default-path UX | Power-user UX | Upgrade drift | SOTA precedent | +|---|---|---|---|---|---| +| **Y. Locked (compose stays inside package) - *recommended*** | Resolve via `importlib.resources` at runtime; user never sees the YAML. Overrides only via CLI flags / `config.yaml` / env vars we expose. | Zero-config | Customisation surface = `--external-postgres`, `--external-elastic`, port flags, etc. (§2.3) - sufficient for v0.1 | None - we own the file | [Supabase CLI](https://github.com/supabase/cli) (went further: constructs the project programmatically in Go, no compose file at all) | +| **X. User-editable (default copy, drift-flagged)** | First `up` copies packaged YAML → user config dir. User may edit. On subsequent `up`, drift is flagged. | Zero-config: first-run user never touches YAML | Standard Docker mental model: edit the YAML | Real - flagged, user resolves manually | [dbt `profiles.yml`](https://docs.getdbt.com/docs/core/connect-data-platform/profiles.yml), [VS Code `settings.json`](https://code.visualstudio.com/docs/getstarted/settings) | +| **Z. Eject** | Start with Y; `pragmata annotation eject` copies compose out and pragmata then uses the ejected copy, warning the user they own it from there | Zero-config | Explicit escape hatch, clean managed-vs-owned contract | None for non-ejected users; ejected users own drift | [create-react-app `eject`](https://create-react-app.dev/docs/available-scripts#npm-run-eject), [Expo eject-to-bare-workflow](https://docs.expo.dev/archive/customizing/) | + +**Recommendation: Y for v0.1.** Users should not have to understand or edit Docker Compose YAML on the supported paths. The customisation surface that matters - external Postgres/Elasticsearch URLs, port bindings, image tags - is exposed through CLI flags / env / config (§2.3). If that surface is sufficient (and for v0.1 it is), keeping the compose file package-owned avoids drift, simplifies upgrades, and prevents feature-creep where every Compose field becomes a CLI flag. + +- **X** creates an ownership/drift problem on day one and makes the happy path Docker-centric. Reserve user-owned compose for an explicit advanced escape hatch (i.e. option Z), not the default. +- **Z** is a clean future escape hatch. Document the pattern, but do **not** build the `eject` verb until concrete demand materialises (>2 users blocked on something the §2.3 flag surface can't express). + +>Related rejected patterns: generate compose from a template at install time (two sources of truth, drifts on upgrade); remote URL fetch (breaks offline installs, trust boundary). Supabase [issue #2435](https://github.com/supabase/cli/issues/2435) documents the specific pain of user-editable compose + CLI tight version coupling - option Y avoids the problem entirely. + +### 2.2 Distribution mechanism (how pkg'd YAML travels) + +Ships as package data inside the installed package at `src/pragmata/annotation/docker-compose.yml`, resolved at runtime via `importlib.resources.files("pragmata.annotation") / "docker-compose.yml"` + `as_file()`. Same mechanism already in use for [`core/annotation/collapsible_field.html`](../../src/pragmata/core/annotation/collapsible_field.html). + +Image tags are pinned in the shipped compose and treated as package-owned. `pip install -U pragmata` ships a new compose with new tags - users automatically pick it up on next `up` (no drift, no warning needed under option Y). + +### 2.3 Profiles / bundles (the flag surface for external backing services) + +Explicitly supports **Docker Compose profiles + external backing service URLs**. We use Compose's built-in `profiles` feature. Profile names carry forward from the current dev compose unchanged: `all-bundled`, `external-pg`, `external-es`. + +```yaml +services: + argilla: + image: argilla/argilla-server: + depends_on: [postgres, elasticsearch, redis] + profiles: [all-bundled, external-pg, external-es] + + worker: + image: argilla/argilla-server: + command: argilla worker + profiles: [all-bundled, external-pg, external-es] + + postgres: + profiles: [all-bundled, external-es] # skipped when profile is external-pg + image: postgres: + # ... + + elasticsearch: + profiles: [all-bundled, external-pg] # skipped when profile is external-es + image: docker.elastic.co/elasticsearch/elasticsearch: + # ... + + redis: + profiles: [all-bundled, external-pg, external-es] + image: redis: + # ... +``` + +**Proposed v0.1 CLI flag surface (minimal):** + +``` +pragmata annotation up # all-bundled profile (zero-config default) +pragmata annotation up --external-postgres # external-pg profile, wire Argilla to external PG +pragmata annotation up --external-elastic # external-es profile, wire Argilla to external ES +``` + +Internally: `--external-postgres` = select `external-pg` profile + inject `ARGILLA_DATABASE_URL`; `--external-elastic` = select `external-es` profile + inject `ARGILLA_ELASTICSEARCH`. Settings resolution applies as normal - flag > env > config > default (see [`config-and-settings.md`](config-and-settings.md) §1.4). + +Precedent for profiles: [Airflow's official Compose](https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html) ships profiles (`flower`, etc.). [Dagster Helm values](https://github.com/dagster-io/dagster/tree/master/helm/dagster) mirrors the same pattern at a different abstraction level. + +## 3. Lifecycle + +### 3.1 First-run UX + +**`pragmata annotation up` is the first-run command. No separate `init`. `annotation setup` stays as Argilla provisioning (see [`config-and-settings.md`](config-and-settings.md) §3), invoked after the stack is up.** + +- Pre-flight in order: extra installed → Docker daemon reachable → required ports free +- Resolve the packaged compose via `importlib.resources` (no copy to disk - option Y, §2.1) +- Pulls images on first invocation (slow; log clearly - make this prominent in docs) +- Health-polls Argilla's health endpoint with a timeout +- On success: prints URL, default API key, and next command + +^^^ most of the core parts of this are already implemented + +Precedent: `supabase start`, `abctl local install`, `prefect server start`. Idempotent single command, safe to re-run. + +### 3.2 Upgrade + +**`pip install -U pragmata` is the sole upgrade primitive. The compose file is package-owned (option Y, §2.1), so upgrades pick up the new file automatically with no drift to manage.** + +- Named Docker volumes with a deterministic prefix (`pragmata_annotation_*`) persist data across container recreation +- New compose ships with new image tags - `pragmata annotation up` after upgrade picks up the new file via `importlib.resources`. No drift detection, no warning, no `reset-compose` verb needed (we own the file) +- For destructive Argilla schema migrations between majors, document the backup step; pragmata cannot protect users from upstream-breaking changes + +>Research note: Airbyte's `abctl local install` is idempotent and designed to fix Compose upgrade brittleness ([Airbyte discussion #40599](https://github.com/airbytehq/airbyte/discussions/40599)). We don't need that machinery here - the locked compose model sidesteps the brittleness it was built to address. + +### 3.3 Uninstall + +**`pragmata annotation down` stops the stack; `pragmata annotation down --volumes` additionally wipes data. No global `pragmata uninstall`.** + +- `pip uninstall pragmata` removes the package +- `~/.config/pragmata/` removal is documented but user-owned. No cleanup verb. + +Precedent: `abctl local uninstall [--persisted]` - explicit data-wipe flag is the industry norm. + +## 4. Prod bootstrap / unattended install + +**Decision for v0.1: no shipped script. Document the two-line install (`pipx install 'pragmata[annotation]' && pragmata annotation up`) in the README.** If demand materialises later (>2 deployers asking for unattended install support), add option C below - skip a static shell script. + +### 4.1 Scope + +pragmata ships three tools. Only `annotation` has any bootstrap beyond `pip install`: + +- `querygen`: `pipx install 'pragmata[querygen]' && export OPENAI_API_KEY=...` - two-line README +- `eval`: same shape as `querygen` +- `annotation`: real sequence (install → wait for Docker → pull images → start stack → poll health → print creds) + +The question is whether `annotation` needs a separate install artefact. For v0.1, no. + +### 4.2 Options surveyed + +| Option | What it is | SOTA precedent | Our cost | Verdict | +|---|---|---|---|---| +| **A. No script (docs only) - *chosen for v0.1*** | Docs snippet: `pipx install 'pragmata[annotation]' && pragmata annotation up` | Every Tier-1 PyPI-distributed Docker-wrapping tool surveyed ([Supabase](https://supabase.com/docs/guides/local-development/cli/getting-started), [Airbyte abctl](https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart), [Prefect](https://docs.prefect.io/3.0/get-started/install), [Dagster](https://docs.dagster.io/getting-started/install), [MLflow](https://mlflow.org/docs/latest/tracking.html)) | None | **Adopt** | +| **B. `scripts/bootstrap-annotation.sh` in repo** | Static shell file checked into git, linked from docs | [Docker convenience script](https://github.com/docker/docker-install), [rustup](https://rustup.rs/), [nvm](https://github.com/nvm-sh/nvm#installing-and-updating) | Maintenance drift - every CLI flag change invalidates it. Docker's own convenience installer is [famously not recommended for production](https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) for exactly this reason | **Skip** | +| **C. `pragmata annotation print-install-script`** | CLI prints a shell transcript pinned to the installed CLI version; user redirects to file and executes | Novel - no surveyed tool does this | Trivial: a string template the CLI owns. Always version-matched | **Defer** - reach for if/when demand appears, jump straight here from A (don't pass through B) | + +If later we do need explicit unattended recipes for cloud-init / systemd / Kubernetes: document in `docs/deploy/` per-environment, don't package. Helm charts exist for [Dagster](https://artifacthub.io/packages/helm/dagster/dagster) and [Prefect](https://artifacthub.io/packages/helm/prefecthq/prefect-server) as separate artefacts - same pattern if we ever need one. + +## 5. Cross-platform runtime + +**Rely on the generic `docker compose` CLI on PATH. Remain agnostic to the user's Docker runtime.** + +- Pre-flight check: `docker version` succeeds (daemon reachable). If not, fail clear: *"Docker daemon not reachable. Start your Docker runtime and try again."* +- No `--runtime` flag, no auto-detection, no runtime-specific branching. Which Docker implementation the user has is not our concern. + +Precedent: Supabase and `abctl` both run over the generic `docker` CLI with no engine-specific logic. + +## 6. Error taxonomy (extension of shared UX) + +Generic first-use errors (extra not installed) are covered in [`config-and-settings.md`](config-and-settings.md) §4. `annotation up` adds infra-specific failure modes: + +``` +$ pragmata annotation up # no Docker daemon +Error: Docker daemon not reachable. + Start your Docker runtime and try again. + See: https://docs.docker.com/get-docker/ + +$ pragmata annotation import foo.json # stack not running +Error: Argilla stack is not running. + Run: pragmata annotation up +``` + +Failure mode is checked in order: *extra-installed → Docker-running → stack-up.* Each check is a strict prerequisite for the next. We fail at the first missing prerequisite with the corresponding fix. + +Beyond these, `annotation up` must also handle: +- port conflict (print occupying process if detectable) +- image-pull failure (network / registry) +- Argilla health-poll timeout (print the container log tail) +- compose-file missing from package (indicates broken install - `pip install --force-reinstall 'pragmata[annotation]'`) + +## 7. Codebase baseline + +| Area | Implemented today | Proposed in this doc | +|---|---|---| +| **Docker stack lifecycle (`up`/`down`)** | Not implemented - only Makefile targets (`docker-up`, `docker-down`, etc.) that read [`deploy/annotation/docker-compose.dev.yml`](../../deploy/annotation/docker-compose.dev.yml) | Add `pragmata annotation up` / `down` CLI commands | +| **Compose file distribution** | Dev-only file in `deploy/`; **not shipped** in the installed wheel | Ship `src/pragmata/annotation/docker-compose.yml` as **package data, locked** (option Y, §2.1) - resolved at runtime via `importlib.resources`, never copied to disk. Dev override stays in `deploy/` for contributors only | +| **Package data** | `importlib.resources` already used for [`core/annotation/collapsible_field.html`](../../src/pragmata/core/annotation/collapsible_field.html) | Same mechanism for the compose file | + +## Open questions + +None remaining for the infra layer. Resolved during PR #162 review: + +| ID | Question | Resolution | +|---|---|---| +| §Q-prod-script | Ship a per-tool bootstrap script for `annotation`? | **No for v0.1** (§4). Document the two-line install. Add option C if demand materialises. | +| §Q-stack-redis | Whether to bundle Redis by default | Yes (§1, §2.3) - bundled in all profiles; opt-out is per-profile compose surgery, not a v0.1 flag | + +Shared config/settings open questions live in [`config-and-settings.md`](config-and-settings.md). + +## References + +- [ADR-0007 - Packaging & invocation surface](../decisions/0007-packaging-invocation-surface.md) +- [ADR-0003 - Infra: self-hosted only](../decisions/0003-infra-self-hosted-only.md) +- [`config-and-settings.md`](config-and-settings.md) - shared settings/config resolution +- Precedent for Docker orchestration + compose distribution: [Supabase CLI](https://github.com/supabase/cli), [Airbyte abctl](https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart), [Prefect](https://docs.prefect.io/3.0/), [Dagster](https://docs.dagster.io/), [Airflow Compose](https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html) diff --git a/docs/design/containerisation-and-deployment.md b/docs/design/containerisation-and-deployment.md new file mode 100644 index 00000000..bcefaad1 --- /dev/null +++ b/docs/design/containerisation-and-deployment.md @@ -0,0 +1,234 @@ +# Containerisation & Production Deployment + +Status: Draft +Related: +- ADR-0003 (infra: self-hosted only) +- ADR-0007 (packaging & invocation surface) +- ADR-0012 (install & bootstrap UX) [draft] +- [`config-and-settings.md`](config-and-settings.md) - shared settings/config +- [`annotation-bootstrap.md`](annotation-bootstrap.md) - annotation stack lifecycle, compose distribution + +## Purpose + +How pragmata is distributed and deployed in production. Covers the artefact strategy (PyPI vs container image), how the three tools differ in deployment shape, persistence/backup, TLS, and the operational story for a self-hosted Argilla campaign that runs for weeks. + +Annotation stack composition (services, profiles, compose-file location) is in [`annotation-bootstrap.md`](annotation-bootstrap.md). Settings resolution is in [`config-and-settings.md`](config-and-settings.md). This doc is purely about **how the artefact ships and how operators run it**. + +## Guiding principle + +**Containerise services, not CLI orchestrators. Ship pragmata as a PyPI package; let the long-running Argilla stack be the containerised workload.** + +Containerisation is primarily intended to run **services** - long-lived processes that need isolation from the host, reproducibility across environments, declarative resource limits, and a clean lifecycle managed by an external supervisor. That fits Argilla + Postgres + Elasticsearch + Redis exactly. CLI tools are typically the opposite shape: short-lived, host-coupled (they read your filesystem, your env, your Docker socket), and benefit more from `pip install` than from container packaging. Containerising a CLI makes sense when *the tool itself is the workload* (CI runners, build agents, hermetic ops tools) or when host installation is genuinely painful - not when it is a thin orchestrator over an HTTP SDK and a `docker compose` shell-out. + +Pragmata's three tools split into two deployment shapes: + +- `querygen`, `eval` - short-lived batch jobs. Run on a developer laptop or a CI runner. No infra to manage. +- `annotation` - orchestrates a long-running Argilla stack (Argilla server + Postgres + Elasticsearch + Redis) that must stay up for the duration of an annotation campaign (weeks). + +The annotation case is what drives the design. The pragmata CLI itself is short-lived even there: `pragmata annotation up` shells out to `docker compose up -d` once per campaign, `pragmata annotation down` once at the end. Provisioning, import, and export are HTTP calls against Argilla. There is no per-request container-launching that would justify wrapping pragmata in its own container and forwarding the Docker socket (DooD; §3.1). + +## 1. Artefact strategy + +### 1.1 What we ship + +| Artefact | Status | Purpose | +|---|---|---| +| **PyPI package** (`pragmata`, `pragmata[annotation\|querygen\|eval]`) | **Primary, v0.1** | Single source of truth. Library + CLI for all three tools. Ships the locked annotation compose file as package data ([`annotation-bootstrap.md`](annotation-bootstrap.md) §2). | +| **Container image** (`ghcr.io/.../pragmata:`) | **Deferred to v0.2+** | Convenience wrapper for `querygen`/`eval` jobs in CI/cloud-run contexts. Not load-bearing - users with Python can `pip install` instead. Not the path for `annotation` (§3). | +| **Helm chart** | Out of scope | Reach for only if/when a deployer asks for k8s-native annotation deployment. | + +For v0.1 the PyPI package is sufficient. Adding a container image is cheap once we have a release pipeline (one extra `docker build && push` step in CI) but it is not on the critical path. + +### 1.2 Why not container-as-primary + +There is no single SOTA pattern for "PyPI CLI that wraps a long-running Compose stack" - the projects in this space split across three legitimate shapes: + +| Pattern | Examples | Orchestrator runs as | +|---|---|---| +| **No orchestrator wrapper.** Ship the compose file; user runs `docker compose up -d` directly. CLI handles only app-level concerns (provisioning, data flow). | [Argilla](https://docs.argilla.io/latest/getting_started/how-to-deploy-argilla-with-docker/) itself, [Supabase self-hosted](https://supabase.com/docs/guides/self-hosting/docker), [Prefect via Docker](https://docs.prefect.io/v3/how-to-guides/self-hosted/server-docker) | n/a - no wrapper | +| **Host CLI wraps stack lifecycle.** A binary/pip-installed CLI shells out to `docker compose` (or `kind`). | [Supabase CLI `supabase start`](https://supabase.com/docs/guides/local-development/cli/getting-started) (local dev), [Airbyte `abctl`](https://docs.airbyte.com/platform/deploying-airbyte/abctl) (kind, not compose) | host process | +| **Containerised orchestrator with DooD.** Daemon container mounts `/var/run/docker.sock` and launches sibling containers. | [Dagster](https://docs.dagster.io/deployment/oss/deployment-options/docker) | container, with socket mounted | + +The projects most analogous to pragmata - and Argilla itself - sit in the first row: no wrapper at all. Supabase CLI's `supabase start` (second row) is the model `pragmata annotation up` is built on, but Supabase positions it as a dev-loop tool; their self-hosted production guidance falls back to "run `docker compose` yourself." Dagster (third row) accepts the DooD risk because its daemon must launch *per-run* containers as a core runtime feature. + +Pragmata sits between rows 1 and 2: the CLI exists for a real reason (single source of truth for env/config resolution, link between provisioning and stack lifecycle, clean error UX for "Docker not running" / "stack not up"), but it does **not** have Dagster's per-run launching requirement. So the right shape is host CLI orchestrating containerised services - same as Supabase CLI - not containerised orchestrator with socket forwarding. + +Reasons container-as-primary does not pay off here: + +- **`querygen`/`eval`** are short-lived Python processes. A developer running `pragmata querygen gen-queries` from a notebook or shell does not want `docker pull` overhead per invocation. +- **`annotation` CLI calls** are also short-lived. The host is already a Linux box with Docker on it (that is a hard prerequisite for the stack anyway), so requiring Python is a marginal additional ask. +- **Containerising the orchestrator** forces the DooD-or-skip-the-wrapper question (§3.1) for no concrete gain. +- **Two artefacts to maintain** = two release surfaces, two upgrade stories, two version-skew matrices. v0.1 cannot afford that. + +> Open upstream question (§5): should `pragmata annotation up` exist at all? Argilla's own deployment guide is `wget` the compose + `docker compose up`. We may be inventing a wrapper layer that the most-analogous project explicitly avoids. Decision deferred; for now the wrapper stays, on the strength of the env-resolution / error-UX argument above. + +### 1.3 Versioning & registry + +When/if we publish a container image (v0.2+): + +- **Registry**: GHCR (`ghcr.io/bertelsmannstift/pragmata`). Free for public repos, no separate credentials, builds straight from the GitHub release workflow. Docker Hub and Quay are viable alternatives but offer no concrete advantage given the codebase already lives on GitHub. +- **Tags**: pin to the released package version (`ghcr.io/.../pragmata:0.2.0`). Also publish `:latest` for convenience but always document pinned tags in install snippets. +- **Image-vs-package version**: image and PyPI package release in lockstep - the image just `pip install`s the same wheel. No independent version axis. + +Sidecar images (Argilla, Postgres, Elasticsearch, Redis) are pinned in the shipped compose file (digests, not floating tags - see [`annotation-bootstrap.md`](annotation-bootstrap.md) §2.2). Those upgrade with the pragmata release that bumps them. + +## 2. Tool-by-tool deployment shape + +``` +┌──────────────────────────────────────────────────────────────────────────┐ +│ DEPLOYMENT SHAPES │ +├─────────────────┬───────────────────┬────────────────────────────────────┤ +│ Tool │ Lifecycle │ Operator runs │ +├─────────────────┼───────────────────┼────────────────────────────────────┤ +│ querygen │ short-lived job │ pip install + invoke │ +│ eval │ short-lived job │ pip install + invoke │ +│ annotation │ long-running infra│ pip install + `annotation up` │ +│ │ (stack stays up │ which spawns Argilla + PG + ES + │ +│ │ during campaign) │ Redis via host docker compose │ +└─────────────────┴───────────────────┴────────────────────────────────────┘ +``` + +### 2.1 `querygen` / `eval` - batch jobs + +**Production shape**: dev laptop, CI runner, or a cloud batch job (Cloud Run, AWS Batch, Azure Container Instances). + +```bash +# Local / CI +pip install 'pragmata[querygen]' +export OPENAI_API_KEY=... +pragmata querygen gen-queries --config querygen.yml --base-dir ./out + +# Containerised (deferred to v0.2) +docker run --rm \ + -e OPENAI_API_KEY=... \ + -v "$PWD/config:/config" \ + -v "$PWD/out:/workspace" \ + ghcr.io/bertelsmannstift/pragmata:0.2.0 \ + pragmata querygen gen-queries --config /config/querygen.yml --base-dir /workspace +``` + +The container form is purely a convenience - same wheel, just pre-installed. No operational difference. + +### 2.2 `annotation` - long-running stack + +**Production shape**: a Linux VM (or k8s node) that the deploying organisation owns. Pragmata installed via pip; the Argilla stack runs as a sibling Compose project on the same host. + +```bash +# On the deployment VM +pipx install 'pragmata[annotation]' +pragmata annotation up # starts the stack on the host +pragmata annotation setup --url http://localhost:6900 --api-key ... + # provisions Argilla workspaces / users / datasets +# stack runs for weeks; annotators access via reverse proxy (§4) +pragmata annotation export ... # ad-hoc export jobs against the running stack +pragmata annotation down # at end of campaign +``` + +`pragmata annotation up` resolves the package-data compose file via `importlib.resources` and shells out to `docker compose up -d` on the host (§3.2). The pragmata process itself is **not** containerised - it runs on the host as a normal Python process and exits after kicking the stack off. + +This matches the Supabase CLI model: the orchestrator runs on the host, talks to the host Docker daemon, and gets out of the way. The CLI's job is to materialise a known-good compose configuration and invoke the daemon - not to host its own runtime container. + +## 3. The "containerised orchestrator" question + +This is the question SG flagged. Resolution: **for v0.1 we do not run pragmata-itself in a container in the annotation deployment path.** The annotation operator runs `pragmata annotation up` directly on the host. + +If at some point a deployer insists on running the pragmata CLI inside a container (e.g. immutable infra, no host Python), the supported answer is option 3.2 below: skip `annotation up` entirely and let them run `docker compose -f up` directly. We do not support DooD. + +### 3.1 Why not Docker-outside-of-Docker (DooD) + +DooD = mount `/var/run/docker.sock` into a container so it can spawn sibling containers on the host daemon. + +- **Security**: Docker socket access is root-equivalent on the host. [OWASP](https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html), [Docker's own engine-security docs](https://docs.docker.com/engine/security/), and most security reviews treat socket-mounting as a privilege-escalation vector. Read-only mount does not help. Rootless Docker mitigates but does not eliminate. +- **Operator confusion**: makes the trust boundary between "the pragmata container" and "the host's whole container fleet" invisible. If the pragmata container is compromised, the attacker owns every container on the host. +- **Wrong tool for our shape**: DooD is the right call when the orchestrator must launch containers on a per-request / per-run basis as a core runtime feature - that's why [Dagster's daemon ships with the socket mounted](https://docs.dagster.io/deployment/oss/deployment-options/docker) as its recommended production pattern. Pragmata has no equivalent requirement: `pragmata annotation up` shells out to `docker compose` *once per campaign*, and `down` once at the end. We get the security cost with none of the capability gain. +- **Better alternatives exist for our shape**: in the projects most analogous to pragmata (Argilla, Supabase self-hosted, Prefect Docker), the orchestrator is either a host process (Supabase CLI) or absent entirely (user runs `docker compose up`). Both avoid DooD without losing functionality. + +### 3.2 Escape hatch: skip `annotation up`, run compose directly + +If a deployer cannot or does not want to install pragmata on the host (e.g. immutable infra, hardened image policy), they can run the shipped compose file directly: + +```bash +# Extract the compose file from the wheel without installing pragmata system-wide +python -m pip download --no-deps pragmata +unzip -j pragmata-*.whl 'pragmata/annotation/docker-compose.yml' -d ./deploy +# Or: clone the repo and use deploy/annotation/docker-compose.yml + +docker compose -f ./deploy/docker-compose.yml up -d +# Provisioning still needs the CLI - run it from anywhere with network access to Argilla: +pipx run 'pragmata[annotation]' annotation setup --url http://argilla.host:6900 ... +``` + +This is an explicit escape hatch, not the happy path. We document it; we do not optimise for it. + +### 3.3 What we ruled out + +| Option | Why not | +|---|---| +| DooD socket mount | §3.1 | +| Docker-in-Docker (DinD, privileged container) | Worse than DooD - same security profile plus a nested daemon to manage | +| Sibling-pod-spawn from a kind/k3s cluster (Airbyte `abctl` style) | Adds a kubernetes runtime as a hard dependency for users who just want annotation. Massive over-engineering. | +| Two separate images ("job runner" + "stack") | Two release surfaces, no benefit - the "stack" image is just Argilla's existing image | + +## 4. Production operations + +These are the operator's responsibility, not pragmata's, but the design must not actively obstruct them. + +### 4.1 Persistence & backup + +Argilla writes to Postgres (annotation submissions, users, workspaces) and Elasticsearch (record indexes). Both must survive container recreation across upgrades and restarts. + +- Compose file declares **named volumes** with deterministic prefixes (`pragmata_annotation_postgres_data`, `pragmata_annotation_elastic_data`, `pragmata_annotation_redis_data`). Already the case in the current dev compose. +- `pragmata annotation down` does **not** wipe volumes; `pragmata annotation down --volumes` does (already specified in [`annotation-bootstrap.md`](annotation-bootstrap.md) §3.3). +- Backup is operator-owned. Document the standard recipe: + - `docker compose exec postgres pg_dump -U postgres argilla > backup.sql` + - Elasticsearch native snapshot API (or volume-level snapshot if the storage supports it) + - Schedule via host cron / systemd timer; pragmata does not ship a backup verb in v0.1 + +### 4.2 TLS, reverse proxy, auth + +Argilla speaks HTTP on port 6900. The shipped compose **binds to localhost only** by default (so a fresh `annotation up` does not silently expose Argilla to the public internet). Production exposure is the operator's call. + +Recommended pattern (documented, not enforced): + +``` + ┌─────────────────────┐ + annotators ───► │ nginx (TLS, 443) │ + │ optionally: oauth2 │ ──► localhost:6900 (argilla) + │ proxy / SSO │ + └─────────────────────┘ + host VM +``` + +- TLS termination at nginx with Let's Encrypt or org cert +- Argilla's built-in user/role system handles annotator auth; no SSO requirement for v0.1 +- If SSO is required: front with `oauth2-proxy` or equivalent + +This is a docs concern, not a code concern. The compose file just needs to default to localhost binding so we don't ship a footgun. + +### 4.3 Deployment topology + +| Topology | When | Notes | +|---|---|---| +| **Single VM + docker compose** | default - any campaign | Linux VM, pragmata installed via pipx, Argilla stack via `annotation up`. Matches Argilla's own recommended deployment. | +| **Kubernetes** | only if the deployer already runs k8s | No Helm chart from pragmata in v0.1. Operator translates the compose file or uses Argilla's own k8s deployment. Pragmata CLI runs as a Job for setup/import/export. | +| **Managed Argilla** | out of scope | ADR-0003 - self-hosted only | + +For Bertelsmann Stiftung's first campaigns the single-VM compose deployment is the realistic target. K8s is a future-someone-else's-problem path. + +## 5. Open questions + +| Question | Status | +|---|---| +| **Should `pragmata annotation up` exist at all?** Upstream of every other question in this doc. The most-analogous project (Argilla itself) ships only the compose file and tells users to run `docker compose up -d` directly; Supabase's *self-hosted production* guidance does the same (their CLI's `start` is positioned as dev-loop tooling). Our wrapper buys env/config consistency, a single source of truth for profile/flag resolution, and a clean error-UX surface ("Docker not reachable", "stack not up") - but it also puts us on the hook for compose's surface area and invents a layer the SOTA explicitly avoids. Resolving this collapses or simplifies several decisions in [`annotation-bootstrap.md`](annotation-bootstrap.md) (compose distribution, locked-vs-eject, lifecycle verbs). | Open | +| **Container image in v0.1?** Currently deferred to v0.2+ (§1.1). Worth confirming we're OK shipping PyPI-only for v0.1 - even with the dev/CI ergonomic loss for `querygen`/`eval` users who'd prefer `docker run`. | Open | +| **Localhost-only binding default for the shipped compose?** §4.2 proposes binding Argilla to `127.0.0.1:6900` by default to avoid accidental public exposure. The current dev compose binds to `0.0.0.0:6900` (`"${ARGILLA_PORT:-6900}:6900"`). Confirm we want to flip this for the shipped (prod-first) compose. | Open | +| **Backup verb?** §4.1 leaves backup to the operator. Worth checking whether a thin `pragmata annotation backup` / `restore` (wrapping `pg_dump` + ES snapshot) is in scope for v0.1 or v0.2+. | Open | + +## References + +- [ADR-0003 - Infra: self-hosted only](../decisions/0003-infra-self-hosted-only.md) +- [ADR-0007 - Packaging & invocation surface](../decisions/0007-packaging-invocation-surface.md) +- [`config-and-settings.md`](config-and-settings.md) - settings resolution +- [`annotation-bootstrap.md`](annotation-bootstrap.md) - stack composition, compose distribution, lifecycle +- Precedent: [Supabase CLI](https://supabase.com/docs/guides/local-development), [Airbyte abctl](https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart), [Prefect](https://docs.prefect.io/), [Dagster](https://docs.dagster.io/), [MLflow](https://mlflow.org/), [dbt Core](https://docs.getdbt.com/), [Argilla self-hosting](https://docs.argilla.io/latest/getting_started/how-to-deploy-argilla-with-docker/) +- DooD security: [OWASP Docker Security Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html) From d7a8b063cc0128470b7404006d46117ff00aa814 Mon Sep 17 00:00:00 2001 From: henrycgbaker Date: Thu, 30 Apr 2026 15:16:55 +0200 Subject: [PATCH 2/3] docs: refine annotation bootstrap and containerisation design docs Remove draft ADR-0012 references, tighten prose, clarify up sequence steps, and drop resolved open-questions table. --- docs/design/annotation-bootstrap.md | 70 +++++++++---------- .../design/containerisation-and-deployment.md | 1 - 2 files changed, 32 insertions(+), 39 deletions(-) diff --git a/docs/design/annotation-bootstrap.md b/docs/design/annotation-bootstrap.md index 09409125..8f250d6b 100644 --- a/docs/design/annotation-bootstrap.md +++ b/docs/design/annotation-bootstrap.md @@ -4,7 +4,6 @@ Status: Draft Related: - ADR-0007 (packaging & invocation surface) - ADR-0003 (infra: self-hosted only) -- ADR-0012 (install & bootstrap UX) [draft] - [`config-and-settings.md`](config-and-settings.md) - shared settings/config resolution that this doc builds on ## Purpose @@ -17,7 +16,9 @@ Shared concerns (config resolution, settings precedence, secrets, generic CLI er **Dev tooling ≠ production tooling.** The `Makefile` is dev-only; the prod install path goes through the CLI (`pragmata annotation up`). They share the same shipped compose file but apply different overrides. -> The `Makefile` targets (`docker-up`, `docker-down`, `test-stack`) bind to `deploy/annotation/docker-compose.dev.yml` and assume a cloned repo + `make` (= non-starter for Windows, PyPI installs, and unattended/prod environments). The CLI command `pragmata annotation up` is the single end-user entry point and must work identically across all three environments. +> The `Makefile` targets (`docker-up`, `docker-down`, `test-stack`) bind to `deploy/annotation/docker-compose.dev.yml` and assume a cloned repo + `make` (= non-starter for Windows, PyPI installs, and unattended/prod environments). +> +> The CLI command `pragmata annotation up` is the single end-user entry point and must work identically across all three environments. SOTA for PyPI-distributed CLIs that wrap Docker stacks (Supabase CLI, Airbyte `abctl`, Dagster, Prefect, MLflow, Argilla itself) converges on a few points: - install is side-effect free @@ -31,19 +32,17 @@ The runtime compose file ships as **package data under `src/pragmata/annotation/ All services bundled by default (zero-config principle, see [`config-and-settings.md`](config-and-settings.md) principle 4). Each backing service (postgres/elasticsearch/redis) is opt-out-able via a Compose profile (§2.3). -**Single shipped compose file, not a dev/prod pair.** Surveyed PyPI-distributed Docker-wrapping tools (Supabase, Airbyte `abctl`, Airflow, Dagster, Prefect, MLflow) all converge on **one shipped compose artefact**. Nobody ships parallel `docker-compose.prod.yml` + `docker-compose.dev.yml` side by side because it forces users to answer "which one do I run?" before doing anything. The split here is: +**Single shipped compose file, not a dev/prod pair.** Surveyed PyPI-distributed Docker-wrapping tools (Supabase, Airbyte `abctl`, Airflow, Dagster, Prefect, MLflow) all converge on **one shipped compose artefact**; nobody ships parallel `docker-compose.prod.yml` + `docker-compose.dev.yml` (as forces users to answer "which one do I run?" before doing anything). The split here is: - **Shipped (package data):** `src/pragmata/annotation/docker-compose.yml` - production-first: pinned tags, env-driven credentials (no hardcoded defaults), sensible resource defaults, localhost-only port bindings. This is the file `pragmata annotation up` resolves at runtime. - **Contributor dev (cloned repo):** `deploy/annotation/docker-compose.dev.override.yml` (proposed - does not exist yet) - layered on top of the shipped file via Makefile targets using `docker compose -f ... -f ...`. Typical contents: well-known default creds (`argilla`/`1234`), stdout logging, looser health-check timing, exposed debug ports. End users (`pragmata annotation up`) only ever touch the shipped (package-data) file via CLI flags. The dev override is exclusively for contributors working in a cloned repo. -This matches Airflow's [documented pattern](https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html) ("compose file is a quick-start, not a production config" - with dev niceties layered via overrides) and keeps us from maintaining two sources of truth. - -Migration steps from today's single `deploy/annotation/docker-compose.dev.yml`: - 1. Extract prod-safe defaults into `src/pragmata/annotation/docker-compose.yml` as package data (the new SSOT for runtime) - 2. Strip dev-only overrides into `deploy/annotation/docker-compose.dev.override.yml` - 3. Update Makefile targets to stack both via `docker compose -f ... -f ...` +>Migration steps from today's single `deploy/annotation/docker-compose.dev.yml`: +> 1. Extract prod-safe defaults into `src/pragmata/annotation/docker-compose.yml` as package data (the new SSOT for runtime) +> 2. Strip dev-only overrides into `deploy/annotation/docker-compose.dev.override.yml` +> 3. Update Makefile targets to stack both via `docker compose -f ... -f ...` ## 2. Compose file distribution @@ -62,7 +61,7 @@ Three options were considered: **Recommendation: Y for v0.1.** Users should not have to understand or edit Docker Compose YAML on the supported paths. The customisation surface that matters - external Postgres/Elasticsearch URLs, port bindings, image tags - is exposed through CLI flags / env / config (§2.3). If that surface is sufficient (and for v0.1 it is), keeping the compose file package-owned avoids drift, simplifies upgrades, and prevents feature-creep where every Compose field becomes a CLI flag. - **X** creates an ownership/drift problem on day one and makes the happy path Docker-centric. Reserve user-owned compose for an explicit advanced escape hatch (i.e. option Z), not the default. -- **Z** is a clean future escape hatch. Document the pattern, but do **not** build the `eject` verb until concrete demand materialises (>2 users blocked on something the §2.3 flag surface can't express). +- **Z** is a clean future escape hatch. Document the pattern, but do **not** build the `eject` verb until concrete demand materialises. >Related rejected patterns: generate compose from a template at install time (two sources of truth, drifts on upgrade); remote URL fetch (breaks offline installs, trust boundary). Supabase [issue #2435](https://github.com/supabase/cli/issues/2435) documents the specific pain of user-editable compose + CLI tight version coupling - option Y avoids the problem entirely. @@ -122,13 +121,14 @@ Precedent for profiles: [Airflow's official Compose](https://airflow.apache.org/ **`pragmata annotation up` is the first-run command. No separate `init`. `annotation setup` stays as Argilla provisioning (see [`config-and-settings.md`](config-and-settings.md) §3), invoked after the stack is up.** -- Pre-flight in order: extra installed → Docker daemon reachable → required ports free +`up` does: +- Pre-flight in order: extra installed? → Docker daemon reachable? → required ports free? - Resolve the packaged compose via `importlib.resources` (no copy to disk - option Y, §2.1) - Pulls images on first invocation (slow; log clearly - make this prominent in docs) - Health-polls Argilla's health endpoint with a timeout -- On success: prints URL, default API key, and next command +- On success: prints URL, default API key, print creds (once) and next command -^^^ most of the core parts of this are already implemented +Most of the core parts of this UX are already implemented. Precedent: `supabase start`, `abctl local install`, `prefect server start`. Idempotent single command, safe to re-run. @@ -137,10 +137,10 @@ Precedent: `supabase start`, `abctl local install`, `prefect server start`. Idem **`pip install -U pragmata` is the sole upgrade primitive. The compose file is package-owned (option Y, §2.1), so upgrades pick up the new file automatically with no drift to manage.** - Named Docker volumes with a deterministic prefix (`pragmata_annotation_*`) persist data across container recreation -- New compose ships with new image tags - `pragmata annotation up` after upgrade picks up the new file via `importlib.resources`. No drift detection, no warning, no `reset-compose` verb needed (we own the file) -- For destructive Argilla schema migrations between majors, document the backup step; pragmata cannot protect users from upstream-breaking changes +- New compose ships with new image tags - `pragmata annotation up` after upgrade picks up the new file via `importlib.resources`. No drift detection / warning / `reset-compose` verb needed (we own the file) +- For destructive Argilla schema migrations between majors, document the backup step; pragmata does not protect users from upstream-breaking changes ->Research note: Airbyte's `abctl local install` is idempotent and designed to fix Compose upgrade brittleness ([Airbyte discussion #40599](https://github.com/airbytehq/airbyte/discussions/40599)). We don't need that machinery here - the locked compose model sidesteps the brittleness it was built to address. +>NB: Airbyte's `abctl local install` is idempotent and designed to fix Compose upgrade brittleness ([Airbyte discussion #40599](https://github.com/airbytehq/airbyte/discussions/40599)). We don't need that machinery here - our locked compose model sidesteps the brittleness here. ### 3.3 Uninstall @@ -149,19 +149,17 @@ Precedent: `supabase start`, `abctl local install`, `prefect server start`. Idem - `pip uninstall pragmata` removes the package - `~/.config/pragmata/` removal is documented but user-owned. No cleanup verb. -Precedent: `abctl local uninstall [--persisted]` - explicit data-wipe flag is the industry norm. - ## 4. Prod bootstrap / unattended install -**Decision for v0.1: no shipped script. Document the two-line install (`pipx install 'pragmata[annotation]' && pragmata annotation up`) in the README.** If demand materialises later (>2 deployers asking for unattended install support), add option C below - skip a static shell script. +**Decision for v0.1: no shipped script. Document the two-line install (`pip install 'pragmata[annotation]' && pragmata annotation up`) in the README.** ### 4.1 Scope pragmata ships three tools. Only `annotation` has any bootstrap beyond `pip install`: -- `querygen`: `pipx install 'pragmata[querygen]' && export OPENAI_API_KEY=...` - two-line README -- `eval`: same shape as `querygen` -- `annotation`: real sequence (install → wait for Docker → pull images → start stack → poll health → print creds) +- `querygen`: `pip install 'pragmata[querygen]' && export OPENAI_API_KEY=...` - two-line README +- `eval`: same as `querygen` +- `annotation`: real sequence (install → wait for Docker → pull images → start stack → poll health → print creds stdout) The question is whether `annotation` needs a separate install artefact. For v0.1, no. @@ -169,11 +167,18 @@ The question is whether `annotation` needs a separate install artefact. For v0.1 | Option | What it is | SOTA precedent | Our cost | Verdict | |---|---|---|---|---| -| **A. No script (docs only) - *chosen for v0.1*** | Docs snippet: `pipx install 'pragmata[annotation]' && pragmata annotation up` | Every Tier-1 PyPI-distributed Docker-wrapping tool surveyed ([Supabase](https://supabase.com/docs/guides/local-development/cli/getting-started), [Airbyte abctl](https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart), [Prefect](https://docs.prefect.io/3.0/get-started/install), [Dagster](https://docs.dagster.io/getting-started/install), [MLflow](https://mlflow.org/docs/latest/tracking.html)) | None | **Adopt** | -| **B. `scripts/bootstrap-annotation.sh` in repo** | Static shell file checked into git, linked from docs | [Docker convenience script](https://github.com/docker/docker-install), [rustup](https://rustup.rs/), [nvm](https://github.com/nvm-sh/nvm#installing-and-updating) | Maintenance drift - every CLI flag change invalidates it. Docker's own convenience installer is [famously not recommended for production](https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) for exactly this reason | **Skip** | -| **C. `pragmata annotation print-install-script`** | CLI prints a shell transcript pinned to the installed CLI version; user redirects to file and executes | Novel - no surveyed tool does this | Trivial: a string template the CLI owns. Always version-matched | **Defer** - reach for if/when demand appears, jump straight here from A (don't pass through B) | +| **A. No script (docs only) - *chosen for v0.1*** | Docs snippet: `pip install 'pragmata[annotation]' && pragmata annotation up` | Every Tier-1 PyPI-distributed Docker-wrapping tool surveyed ([Supabase](https://supabase.com/docs/guides/local-development/cli/getting-started), [Airbyte abctl](https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart), [Prefect](https://docs.prefect.io/3.0/get-started/install), [Dagster](https://docs.dagster.io/getting-started/install), [MLflow](https://mlflow.org/docs/latest/tracking.html)) | None | **Adopt** | +| **B. `scripts/bootstrap-annotation.sh` in repo** | Static shell file checked into git, linked from docs | [Docker convenience script](https://github.com/docker/docker-install), [rustup](https://rustup.rs/), [nvm](https://github.com/nvm-sh/nvm#installing-and-updating) | Maintenance drift - every CLI flag change invalidates it. Docker's own convenience installer is [not recommended for production](https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) for exactly this reason | **Skip** | + +> **Proposed `annotation up` sequence (to implement):** +> 1. Pre-flight checks: `pragmata[annotation]` extra present → Docker daemon reachable → required ports free. Fail fast with actionable message on any failure. +> 2. Resolve compose file via `importlib.resources` (in-memory path, no copy to disk). +> 3. `docker compose pull` - only on first run or after `pip install -U` (detect by comparing pinned image digests vs locally cached). Log clearly; this is the slow step. +> 4. `docker compose up -d` with resolved profile (§2.3). +> 5. Poll `GET /api/v1/health` until 200 or timeout (default 120 s). Stream a progress indicator. +> 6. On first-run only: generate a random `argilla` admin password, inject via env, print URL + credentials to stdout. Subsequent `up` calls skip this step - stack is already configured. +> 7. Print next-step hint: `pragmata annotation setup --users `. -If later we do need explicit unattended recipes for cloud-init / systemd / Kubernetes: document in `docs/deploy/` per-environment, don't package. Helm charts exist for [Dagster](https://artifacthub.io/packages/helm/dagster/dagster) and [Prefect](https://artifacthub.io/packages/helm/prefecthq/prefect-server) as separate artefacts - same pattern if we ever need one. ## 5. Cross-platform runtime @@ -212,20 +217,9 @@ Beyond these, `annotation up` must also handle: | Area | Implemented today | Proposed in this doc | |---|---|---| | **Docker stack lifecycle (`up`/`down`)** | Not implemented - only Makefile targets (`docker-up`, `docker-down`, etc.) that read [`deploy/annotation/docker-compose.dev.yml`](../../deploy/annotation/docker-compose.dev.yml) | Add `pragmata annotation up` / `down` CLI commands | -| **Compose file distribution** | Dev-only file in `deploy/`; **not shipped** in the installed wheel | Ship `src/pragmata/annotation/docker-compose.yml` as **package data, locked** (option Y, §2.1) - resolved at runtime via `importlib.resources`, never copied to disk. Dev override stays in `deploy/` for contributors only | +| **Compose file distribution** | Dev-only file in `deploy/`; **not shipped** in the installed wheel | Ship `src/pragmata/annotation/docker-compose.yml` as package data, locked (option Y, §2.1) - resolved at runtime via `importlib.resources`, never copied to disk. Dev override stays in `deploy/` for contributors only | | **Package data** | `importlib.resources` already used for [`core/annotation/collapsible_field.html`](../../src/pragmata/core/annotation/collapsible_field.html) | Same mechanism for the compose file | -## Open questions - -None remaining for the infra layer. Resolved during PR #162 review: - -| ID | Question | Resolution | -|---|---|---| -| §Q-prod-script | Ship a per-tool bootstrap script for `annotation`? | **No for v0.1** (§4). Document the two-line install. Add option C if demand materialises. | -| §Q-stack-redis | Whether to bundle Redis by default | Yes (§1, §2.3) - bundled in all profiles; opt-out is per-profile compose surgery, not a v0.1 flag | - -Shared config/settings open questions live in [`config-and-settings.md`](config-and-settings.md). - ## References - [ADR-0007 - Packaging & invocation surface](../decisions/0007-packaging-invocation-surface.md) diff --git a/docs/design/containerisation-and-deployment.md b/docs/design/containerisation-and-deployment.md index bcefaad1..1a8d2c16 100644 --- a/docs/design/containerisation-and-deployment.md +++ b/docs/design/containerisation-and-deployment.md @@ -4,7 +4,6 @@ Status: Draft Related: - ADR-0003 (infra: self-hosted only) - ADR-0007 (packaging & invocation surface) -- ADR-0012 (install & bootstrap UX) [draft] - [`config-and-settings.md`](config-and-settings.md) - shared settings/config - [`annotation-bootstrap.md`](annotation-bootstrap.md) - annotation stack lifecycle, compose distribution From c050cf008c38189523173b5c08e9f0a67a1b3af6 Mon Sep 17 00:00:00 2001 From: henrycgbaker Date: Thu, 30 Apr 2026 15:38:10 +0200 Subject: [PATCH 3/3] docs: rename annotation-bootstrap to annotation-stack-lifecycle Rename reflects the full scope of the doc - stack composition, compose distribution, lifecycle, and prod bootstrap - not just first-run setup. --- docs/design/README.md | 3 +- ...strap.md => annotation-stack-lifecycle.md} | 2 +- .../design/containerisation-and-deployment.md | 233 ------------------ 3 files changed, 2 insertions(+), 236 deletions(-) rename docs/design/{annotation-bootstrap.md => annotation-stack-lifecycle.md} (99%) delete mode 100644 docs/design/containerisation-and-deployment.md diff --git a/docs/design/README.md b/docs/design/README.md index 9d550080..4b9c5d57 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -23,5 +23,4 @@ Design documents focus on (this is guidance, not a required section template): - [Synthetic Test Set](synthetic-test-set.md) — Query generation, prompt variation, and response collection - [Package Contracts](infra-package-contracts.md) — Contract layer layout: canonical types, schemas, path conventions, and config - [Packaging and Invocation Surface](packaging-invocation-surface.md) — Package structure, module boundaries, and invocation bindings -- [Annotation Bootstrap](annotation-bootstrap.md) — Annotation-only stack lifecycle, compose distribution, prod bootstrap, cross-platform runtime -- [Containerisation and Deployment](containerisation-and-deployment.md) — Artefact strategy (PyPI vs container image), per-tool deployment shape, persistence/backup, TLS +- [Annotation Stack Lifecycle](annotation-stack-lifecycle.md) — Annotation-only stack lifecycle, compose distribution, prod bootstrap, cross-platform runtime diff --git a/docs/design/annotation-bootstrap.md b/docs/design/annotation-stack-lifecycle.md similarity index 99% rename from docs/design/annotation-bootstrap.md rename to docs/design/annotation-stack-lifecycle.md index 8f250d6b..e35d3c26 100644 --- a/docs/design/annotation-bootstrap.md +++ b/docs/design/annotation-stack-lifecycle.md @@ -1,4 +1,4 @@ -# Annotation Bootstrap & Stack Lifecycle +# Annotation Stack Lifecycle Status: Draft Related: diff --git a/docs/design/containerisation-and-deployment.md b/docs/design/containerisation-and-deployment.md deleted file mode 100644 index 1a8d2c16..00000000 --- a/docs/design/containerisation-and-deployment.md +++ /dev/null @@ -1,233 +0,0 @@ -# Containerisation & Production Deployment - -Status: Draft -Related: -- ADR-0003 (infra: self-hosted only) -- ADR-0007 (packaging & invocation surface) -- [`config-and-settings.md`](config-and-settings.md) - shared settings/config -- [`annotation-bootstrap.md`](annotation-bootstrap.md) - annotation stack lifecycle, compose distribution - -## Purpose - -How pragmata is distributed and deployed in production. Covers the artefact strategy (PyPI vs container image), how the three tools differ in deployment shape, persistence/backup, TLS, and the operational story for a self-hosted Argilla campaign that runs for weeks. - -Annotation stack composition (services, profiles, compose-file location) is in [`annotation-bootstrap.md`](annotation-bootstrap.md). Settings resolution is in [`config-and-settings.md`](config-and-settings.md). This doc is purely about **how the artefact ships and how operators run it**. - -## Guiding principle - -**Containerise services, not CLI orchestrators. Ship pragmata as a PyPI package; let the long-running Argilla stack be the containerised workload.** - -Containerisation is primarily intended to run **services** - long-lived processes that need isolation from the host, reproducibility across environments, declarative resource limits, and a clean lifecycle managed by an external supervisor. That fits Argilla + Postgres + Elasticsearch + Redis exactly. CLI tools are typically the opposite shape: short-lived, host-coupled (they read your filesystem, your env, your Docker socket), and benefit more from `pip install` than from container packaging. Containerising a CLI makes sense when *the tool itself is the workload* (CI runners, build agents, hermetic ops tools) or when host installation is genuinely painful - not when it is a thin orchestrator over an HTTP SDK and a `docker compose` shell-out. - -Pragmata's three tools split into two deployment shapes: - -- `querygen`, `eval` - short-lived batch jobs. Run on a developer laptop or a CI runner. No infra to manage. -- `annotation` - orchestrates a long-running Argilla stack (Argilla server + Postgres + Elasticsearch + Redis) that must stay up for the duration of an annotation campaign (weeks). - -The annotation case is what drives the design. The pragmata CLI itself is short-lived even there: `pragmata annotation up` shells out to `docker compose up -d` once per campaign, `pragmata annotation down` once at the end. Provisioning, import, and export are HTTP calls against Argilla. There is no per-request container-launching that would justify wrapping pragmata in its own container and forwarding the Docker socket (DooD; §3.1). - -## 1. Artefact strategy - -### 1.1 What we ship - -| Artefact | Status | Purpose | -|---|---|---| -| **PyPI package** (`pragmata`, `pragmata[annotation\|querygen\|eval]`) | **Primary, v0.1** | Single source of truth. Library + CLI for all three tools. Ships the locked annotation compose file as package data ([`annotation-bootstrap.md`](annotation-bootstrap.md) §2). | -| **Container image** (`ghcr.io/.../pragmata:`) | **Deferred to v0.2+** | Convenience wrapper for `querygen`/`eval` jobs in CI/cloud-run contexts. Not load-bearing - users with Python can `pip install` instead. Not the path for `annotation` (§3). | -| **Helm chart** | Out of scope | Reach for only if/when a deployer asks for k8s-native annotation deployment. | - -For v0.1 the PyPI package is sufficient. Adding a container image is cheap once we have a release pipeline (one extra `docker build && push` step in CI) but it is not on the critical path. - -### 1.2 Why not container-as-primary - -There is no single SOTA pattern for "PyPI CLI that wraps a long-running Compose stack" - the projects in this space split across three legitimate shapes: - -| Pattern | Examples | Orchestrator runs as | -|---|---|---| -| **No orchestrator wrapper.** Ship the compose file; user runs `docker compose up -d` directly. CLI handles only app-level concerns (provisioning, data flow). | [Argilla](https://docs.argilla.io/latest/getting_started/how-to-deploy-argilla-with-docker/) itself, [Supabase self-hosted](https://supabase.com/docs/guides/self-hosting/docker), [Prefect via Docker](https://docs.prefect.io/v3/how-to-guides/self-hosted/server-docker) | n/a - no wrapper | -| **Host CLI wraps stack lifecycle.** A binary/pip-installed CLI shells out to `docker compose` (or `kind`). | [Supabase CLI `supabase start`](https://supabase.com/docs/guides/local-development/cli/getting-started) (local dev), [Airbyte `abctl`](https://docs.airbyte.com/platform/deploying-airbyte/abctl) (kind, not compose) | host process | -| **Containerised orchestrator with DooD.** Daemon container mounts `/var/run/docker.sock` and launches sibling containers. | [Dagster](https://docs.dagster.io/deployment/oss/deployment-options/docker) | container, with socket mounted | - -The projects most analogous to pragmata - and Argilla itself - sit in the first row: no wrapper at all. Supabase CLI's `supabase start` (second row) is the model `pragmata annotation up` is built on, but Supabase positions it as a dev-loop tool; their self-hosted production guidance falls back to "run `docker compose` yourself." Dagster (third row) accepts the DooD risk because its daemon must launch *per-run* containers as a core runtime feature. - -Pragmata sits between rows 1 and 2: the CLI exists for a real reason (single source of truth for env/config resolution, link between provisioning and stack lifecycle, clean error UX for "Docker not running" / "stack not up"), but it does **not** have Dagster's per-run launching requirement. So the right shape is host CLI orchestrating containerised services - same as Supabase CLI - not containerised orchestrator with socket forwarding. - -Reasons container-as-primary does not pay off here: - -- **`querygen`/`eval`** are short-lived Python processes. A developer running `pragmata querygen gen-queries` from a notebook or shell does not want `docker pull` overhead per invocation. -- **`annotation` CLI calls** are also short-lived. The host is already a Linux box with Docker on it (that is a hard prerequisite for the stack anyway), so requiring Python is a marginal additional ask. -- **Containerising the orchestrator** forces the DooD-or-skip-the-wrapper question (§3.1) for no concrete gain. -- **Two artefacts to maintain** = two release surfaces, two upgrade stories, two version-skew matrices. v0.1 cannot afford that. - -> Open upstream question (§5): should `pragmata annotation up` exist at all? Argilla's own deployment guide is `wget` the compose + `docker compose up`. We may be inventing a wrapper layer that the most-analogous project explicitly avoids. Decision deferred; for now the wrapper stays, on the strength of the env-resolution / error-UX argument above. - -### 1.3 Versioning & registry - -When/if we publish a container image (v0.2+): - -- **Registry**: GHCR (`ghcr.io/bertelsmannstift/pragmata`). Free for public repos, no separate credentials, builds straight from the GitHub release workflow. Docker Hub and Quay are viable alternatives but offer no concrete advantage given the codebase already lives on GitHub. -- **Tags**: pin to the released package version (`ghcr.io/.../pragmata:0.2.0`). Also publish `:latest` for convenience but always document pinned tags in install snippets. -- **Image-vs-package version**: image and PyPI package release in lockstep - the image just `pip install`s the same wheel. No independent version axis. - -Sidecar images (Argilla, Postgres, Elasticsearch, Redis) are pinned in the shipped compose file (digests, not floating tags - see [`annotation-bootstrap.md`](annotation-bootstrap.md) §2.2). Those upgrade with the pragmata release that bumps them. - -## 2. Tool-by-tool deployment shape - -``` -┌──────────────────────────────────────────────────────────────────────────┐ -│ DEPLOYMENT SHAPES │ -├─────────────────┬───────────────────┬────────────────────────────────────┤ -│ Tool │ Lifecycle │ Operator runs │ -├─────────────────┼───────────────────┼────────────────────────────────────┤ -│ querygen │ short-lived job │ pip install + invoke │ -│ eval │ short-lived job │ pip install + invoke │ -│ annotation │ long-running infra│ pip install + `annotation up` │ -│ │ (stack stays up │ which spawns Argilla + PG + ES + │ -│ │ during campaign) │ Redis via host docker compose │ -└─────────────────┴───────────────────┴────────────────────────────────────┘ -``` - -### 2.1 `querygen` / `eval` - batch jobs - -**Production shape**: dev laptop, CI runner, or a cloud batch job (Cloud Run, AWS Batch, Azure Container Instances). - -```bash -# Local / CI -pip install 'pragmata[querygen]' -export OPENAI_API_KEY=... -pragmata querygen gen-queries --config querygen.yml --base-dir ./out - -# Containerised (deferred to v0.2) -docker run --rm \ - -e OPENAI_API_KEY=... \ - -v "$PWD/config:/config" \ - -v "$PWD/out:/workspace" \ - ghcr.io/bertelsmannstift/pragmata:0.2.0 \ - pragmata querygen gen-queries --config /config/querygen.yml --base-dir /workspace -``` - -The container form is purely a convenience - same wheel, just pre-installed. No operational difference. - -### 2.2 `annotation` - long-running stack - -**Production shape**: a Linux VM (or k8s node) that the deploying organisation owns. Pragmata installed via pip; the Argilla stack runs as a sibling Compose project on the same host. - -```bash -# On the deployment VM -pipx install 'pragmata[annotation]' -pragmata annotation up # starts the stack on the host -pragmata annotation setup --url http://localhost:6900 --api-key ... - # provisions Argilla workspaces / users / datasets -# stack runs for weeks; annotators access via reverse proxy (§4) -pragmata annotation export ... # ad-hoc export jobs against the running stack -pragmata annotation down # at end of campaign -``` - -`pragmata annotation up` resolves the package-data compose file via `importlib.resources` and shells out to `docker compose up -d` on the host (§3.2). The pragmata process itself is **not** containerised - it runs on the host as a normal Python process and exits after kicking the stack off. - -This matches the Supabase CLI model: the orchestrator runs on the host, talks to the host Docker daemon, and gets out of the way. The CLI's job is to materialise a known-good compose configuration and invoke the daemon - not to host its own runtime container. - -## 3. The "containerised orchestrator" question - -This is the question SG flagged. Resolution: **for v0.1 we do not run pragmata-itself in a container in the annotation deployment path.** The annotation operator runs `pragmata annotation up` directly on the host. - -If at some point a deployer insists on running the pragmata CLI inside a container (e.g. immutable infra, no host Python), the supported answer is option 3.2 below: skip `annotation up` entirely and let them run `docker compose -f up` directly. We do not support DooD. - -### 3.1 Why not Docker-outside-of-Docker (DooD) - -DooD = mount `/var/run/docker.sock` into a container so it can spawn sibling containers on the host daemon. - -- **Security**: Docker socket access is root-equivalent on the host. [OWASP](https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html), [Docker's own engine-security docs](https://docs.docker.com/engine/security/), and most security reviews treat socket-mounting as a privilege-escalation vector. Read-only mount does not help. Rootless Docker mitigates but does not eliminate. -- **Operator confusion**: makes the trust boundary between "the pragmata container" and "the host's whole container fleet" invisible. If the pragmata container is compromised, the attacker owns every container on the host. -- **Wrong tool for our shape**: DooD is the right call when the orchestrator must launch containers on a per-request / per-run basis as a core runtime feature - that's why [Dagster's daemon ships with the socket mounted](https://docs.dagster.io/deployment/oss/deployment-options/docker) as its recommended production pattern. Pragmata has no equivalent requirement: `pragmata annotation up` shells out to `docker compose` *once per campaign*, and `down` once at the end. We get the security cost with none of the capability gain. -- **Better alternatives exist for our shape**: in the projects most analogous to pragmata (Argilla, Supabase self-hosted, Prefect Docker), the orchestrator is either a host process (Supabase CLI) or absent entirely (user runs `docker compose up`). Both avoid DooD without losing functionality. - -### 3.2 Escape hatch: skip `annotation up`, run compose directly - -If a deployer cannot or does not want to install pragmata on the host (e.g. immutable infra, hardened image policy), they can run the shipped compose file directly: - -```bash -# Extract the compose file from the wheel without installing pragmata system-wide -python -m pip download --no-deps pragmata -unzip -j pragmata-*.whl 'pragmata/annotation/docker-compose.yml' -d ./deploy -# Or: clone the repo and use deploy/annotation/docker-compose.yml - -docker compose -f ./deploy/docker-compose.yml up -d -# Provisioning still needs the CLI - run it from anywhere with network access to Argilla: -pipx run 'pragmata[annotation]' annotation setup --url http://argilla.host:6900 ... -``` - -This is an explicit escape hatch, not the happy path. We document it; we do not optimise for it. - -### 3.3 What we ruled out - -| Option | Why not | -|---|---| -| DooD socket mount | §3.1 | -| Docker-in-Docker (DinD, privileged container) | Worse than DooD - same security profile plus a nested daemon to manage | -| Sibling-pod-spawn from a kind/k3s cluster (Airbyte `abctl` style) | Adds a kubernetes runtime as a hard dependency for users who just want annotation. Massive over-engineering. | -| Two separate images ("job runner" + "stack") | Two release surfaces, no benefit - the "stack" image is just Argilla's existing image | - -## 4. Production operations - -These are the operator's responsibility, not pragmata's, but the design must not actively obstruct them. - -### 4.1 Persistence & backup - -Argilla writes to Postgres (annotation submissions, users, workspaces) and Elasticsearch (record indexes). Both must survive container recreation across upgrades and restarts. - -- Compose file declares **named volumes** with deterministic prefixes (`pragmata_annotation_postgres_data`, `pragmata_annotation_elastic_data`, `pragmata_annotation_redis_data`). Already the case in the current dev compose. -- `pragmata annotation down` does **not** wipe volumes; `pragmata annotation down --volumes` does (already specified in [`annotation-bootstrap.md`](annotation-bootstrap.md) §3.3). -- Backup is operator-owned. Document the standard recipe: - - `docker compose exec postgres pg_dump -U postgres argilla > backup.sql` - - Elasticsearch native snapshot API (or volume-level snapshot if the storage supports it) - - Schedule via host cron / systemd timer; pragmata does not ship a backup verb in v0.1 - -### 4.2 TLS, reverse proxy, auth - -Argilla speaks HTTP on port 6900. The shipped compose **binds to localhost only** by default (so a fresh `annotation up` does not silently expose Argilla to the public internet). Production exposure is the operator's call. - -Recommended pattern (documented, not enforced): - -``` - ┌─────────────────────┐ - annotators ───► │ nginx (TLS, 443) │ - │ optionally: oauth2 │ ──► localhost:6900 (argilla) - │ proxy / SSO │ - └─────────────────────┘ - host VM -``` - -- TLS termination at nginx with Let's Encrypt or org cert -- Argilla's built-in user/role system handles annotator auth; no SSO requirement for v0.1 -- If SSO is required: front with `oauth2-proxy` or equivalent - -This is a docs concern, not a code concern. The compose file just needs to default to localhost binding so we don't ship a footgun. - -### 4.3 Deployment topology - -| Topology | When | Notes | -|---|---|---| -| **Single VM + docker compose** | default - any campaign | Linux VM, pragmata installed via pipx, Argilla stack via `annotation up`. Matches Argilla's own recommended deployment. | -| **Kubernetes** | only if the deployer already runs k8s | No Helm chart from pragmata in v0.1. Operator translates the compose file or uses Argilla's own k8s deployment. Pragmata CLI runs as a Job for setup/import/export. | -| **Managed Argilla** | out of scope | ADR-0003 - self-hosted only | - -For Bertelsmann Stiftung's first campaigns the single-VM compose deployment is the realistic target. K8s is a future-someone-else's-problem path. - -## 5. Open questions - -| Question | Status | -|---|---| -| **Should `pragmata annotation up` exist at all?** Upstream of every other question in this doc. The most-analogous project (Argilla itself) ships only the compose file and tells users to run `docker compose up -d` directly; Supabase's *self-hosted production* guidance does the same (their CLI's `start` is positioned as dev-loop tooling). Our wrapper buys env/config consistency, a single source of truth for profile/flag resolution, and a clean error-UX surface ("Docker not reachable", "stack not up") - but it also puts us on the hook for compose's surface area and invents a layer the SOTA explicitly avoids. Resolving this collapses or simplifies several decisions in [`annotation-bootstrap.md`](annotation-bootstrap.md) (compose distribution, locked-vs-eject, lifecycle verbs). | Open | -| **Container image in v0.1?** Currently deferred to v0.2+ (§1.1). Worth confirming we're OK shipping PyPI-only for v0.1 - even with the dev/CI ergonomic loss for `querygen`/`eval` users who'd prefer `docker run`. | Open | -| **Localhost-only binding default for the shipped compose?** §4.2 proposes binding Argilla to `127.0.0.1:6900` by default to avoid accidental public exposure. The current dev compose binds to `0.0.0.0:6900` (`"${ARGILLA_PORT:-6900}:6900"`). Confirm we want to flip this for the shipped (prod-first) compose. | Open | -| **Backup verb?** §4.1 leaves backup to the operator. Worth checking whether a thin `pragmata annotation backup` / `restore` (wrapping `pg_dump` + ES snapshot) is in scope for v0.1 or v0.2+. | Open | - -## References - -- [ADR-0003 - Infra: self-hosted only](../decisions/0003-infra-self-hosted-only.md) -- [ADR-0007 - Packaging & invocation surface](../decisions/0007-packaging-invocation-surface.md) -- [`config-and-settings.md`](config-and-settings.md) - settings resolution -- [`annotation-bootstrap.md`](annotation-bootstrap.md) - stack composition, compose distribution, lifecycle -- Precedent: [Supabase CLI](https://supabase.com/docs/guides/local-development), [Airbyte abctl](https://docs.airbyte.com/using-airbyte/getting-started/oss-quickstart), [Prefect](https://docs.prefect.io/), [Dagster](https://docs.dagster.io/), [MLflow](https://mlflow.org/), [dbt Core](https://docs.getdbt.com/), [Argilla self-hosting](https://docs.argilla.io/latest/getting_started/how-to-deploy-argilla-with-docker/) -- DooD security: [OWASP Docker Security Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html)