diff --git a/docs/agent-runtime-design.md b/docs/agent-runtime-design.md new file mode 100644 index 0000000..4c706a2 --- /dev/null +++ b/docs/agent-runtime-design.md @@ -0,0 +1,93 @@ +# Agent runtime and interface design + +## Problem statement + +`modules/agents.nix` declares what an agent *is* — name, role, harness, model, runAs, optional Discord bot. It does not say how a session of that agent is launched, supervised, observed, attached, or stopped. Today on turtle (v1) that is the job of `launch-agent` and `tools/ribosome` — bash scripts that hardcode tmux, `claude --dangerously-skip-permissions`, sqlite-polled Mercury ack loops, and a baked-in agent registry. The substrate locks in claude-code and tmux at the lowest layer. v2 needs the inverse: a declarative agent layer with a runtime where harness, supervisor, and human-attach surface are independently swappable, and where launching on one machine versus another is the same operation. + +## Goals and non-goals + +**Goals.** Nix-native (declared config evaluates to a runnable thing without an out-of-band orchestrator). Harness-portable (claude-code today, codex or an in-house process tomorrow, same declaration). Supervised (start / stop / restart / auto-restart and log tail without a custom tool). Attachable (humans can read along). Observable (process state, logs, later per-session cost, queryable by name). Trust-aware (`runAs` is load-bearing; sessions inherit the user's permission surface, not the deploying admin's). Cross-machine symmetric (burst-worker on a remote box is the same shape as a long-lived agent on turtle). + +**Non-goals (this pass).** A custom orchestrator binary — north star is explicit, build a `forge` CLI only when pain shows up. Locking in claude-code — nothing harness-specific in `modules/agents.nix`. Cross-cutting fleet ops (enumerate, attach-by-name across machines) — deferred until standard primitives feel insufficient. Vector-DB / memory backing, Nostr inter-agent comms, cost tracking — each is its own design note. Migration from turtle v1 — separate doc once the substrate is real. + +## Survey of options + +### 1. Lifecycle management + +- **systemd user services.** One unit per agent, owned by `runAs`'s linger-enabled user. Native start / stop / restart, `Restart=on-failure`, `journalctl --user -u forge-agent-X`. Inherits the user's environment; respects `runAs` without elevation. Strong on declarative-ness, supervision, observability. Weakness: per-user lingering must be enabled. +- **systemd system services with `User=`.** Centralized via root `systemctl`, no linger. Loses the trust-isolation of a user-bus. Weakest fit for the trust gradient. +- **`nix run` directly.** No supervisor; the session is whatever shell ran it. Most Nix-native but no restart, no log retention. Fine for one-shot burst workers. +- **Containerized.** Strong isolation, resource bounding, clean teardown. Over-isolating for a coordinator whose job is touching the host repo. Right for sandbox later, wrong as default. +- **Custom supervisor.** Replicates ribosome in Rust or Go. North star explicitly warns against this. + +### 2. Coordinator ↔ agent interaction + +- **tmux attach.** Human-as-coordinator from v1. Read-along is excellent; scripting is brittle. Agents can't easily query a tmux pane. +- **MCP control plane.** Each agent exposes a small MCP server (`status`, `logs`, `signal`). Strong for future agent-mediated flows. Cost: every harness must expose it. +- **Custom CLI (`forge agent X status / logs / attach`).** Thin wrapper around journalctl + tmux + systemctl. Discoverable, scriptable, but pre-emptive — north star says build only when pain shows up. +- **Pure logs + signals.** journalctl for read, `systemctl kill -s` or a drop-file for write. No new abstraction. Weakest ergonomics, strongest portability, zero-debt. + +### 3. Harness abstraction + +- **Per-harness Nix module.** `modules/harnesses/claude-code.nix`, `modules/harnesses/codex.nix` etc., each contributing options under `services.forge.agents.` (model alias mapping, thinking depth, flag wiring) and emitting a per-agent wrapper derivation. `agents.nix` stays harness-agnostic. Matches the submodule-merge pattern in `discord.nix`. +- **Single adapter shape (trait-like).** One module defines a contract (`{ command, env, args, attachable, signalProtocol }`) and each harness fills it in. Risk: the shape calcifies before we know what codex wants. +- **Wrapper script per harness.** Cheapest, hardest to introspect. Fine as the *body* of a harness module's derivation; wrong as the seam itself. +- **Embed harness logic in agents.nix.** Pre-empt this — the v1 mistake. + +### 4. Launch primitive + +- **`nix run .#agent.X`.** Most Nix-native; the flake URL + attr is the cross-machine reference. No supervision; fine for one-shot, weak for long-lived. +- **`systemctl --user start forge-agent-X`.** Boring, restartable, journaled. Cross-machine via SSH. Discoverable via `systemctl --user list-units 'forge-agent-*'`. +- **Both, layered.** The systemd unit's `ExecStart` *is* `nix run .#agent.X`. One artifact, two entry points: `nix run` for ephemeral, `systemctl` for managed. +- **`forge agent X start` CLI.** Defer. + +### 5. What a good agent runtime gives you (ranked) + +Ordered by what would hurt most to lose now: + +1. **Declarative** — `nixos-rebuild` is the deploy path. Anything not derivable from the module is unsurfaced. +2. **Observable** — process state and log tail by name. Without this, supervision is theatre. +3. **Restartable** — single command, idempotent. Crash-recovery is auto-restart with backoff. +4. **Attachable** — human can read along when they want to. +5. **Transferable across machines** — same declaration runs on turtle, laptop, burst worker. +6. **Memory-persistent across restarts** — state dir survives unit restart; the agent does the actual persisting. +7. **Resource-bounded** — cgroups, memory ceiling. Important for burst workers, deferable for coordinators. +8. **Crash-recoverable** — narrowly, last-known-good. Replay / checkpoint is out of scope. + +## Proposed direction + +**Lifecycle: systemd user services, generated by Nix.** Each agent declaration emits one `systemd.user.services.forge-agent-${name}` unit owned by `runAs`. `Restart=on-failure`, `StandardOutput=journal`, `WorkingDirectory=` the agent's state dir. Linger enabled per declared user. Covers items 1–4 of the runtime criteria with no custom tool. + +**Launch primitive: `nix run .#agent.${name}` is the canonical command, and it is exactly the systemd unit's `ExecStart`.** Long-lived sessions go through systemd; one-shot burst workers run `nix run` directly. Cross-machine is `ssh host -- nix run github:MakePrisms/forge#agent.scout` — no protocol to maintain. + +**Harness abstraction: per-harness Nix modules, each contributing options to the agent submodule and producing the per-agent wrapper derivation.** `modules/agents.nix` stays harness-agnostic and validation-only; `modules/harnesses/claude-code.nix` owns the model-alias mapping, the `--dangerously-skip-permissions` decision, the plugin wiring. A new harness is a new file plus an enum entry. Same shape `modules/discord.nix` already uses. + +**Coordinator ↔ agent: logs + signals first, tmux attach as the human escape hatch.** Read via `journalctl --user -u forge-agent-${name}`. Write via `systemctl --user kill -s SIGTERM` plus a drop-file convention under the agent's state dir for structured signals (e.g. "gather"). For human read-along the wrapper may run its session under `tmux new-session -d -s forge:${name}` so `tmux attach -t forge:${name}` works. No MCP control plane in this pass; revisit when agent-to-agent control becomes load-bearing. + +**Explicitly punted, with triggers.** + +- A `forge` CLI — when "find me the spinning agent on any machine" becomes a daily question. +- MCP control plane — when an agent legitimately needs to query another's state programmatically, not just consume Mercury / pikachat messages. +- Per-session cost tracking — separate observability module once the substrate is stable. +- Containerization — adopt for the sandbox archetype specifically; do not generalize. + +Why this direction. It answers every numbered criterion with standard Linux primitives. The seam between declaration and runtime is the harness module — the seam the north star calls out as load-bearing for portability. The launch primitive is identical local and remote because the flake URL is the address. Each piece is independently revisitable: swap systemd without touching agent declarations; swap claude-code for codex without touching the supervisor; add MCP later as a sibling module without breaking the launch primitive. + +## Open questions + +1. **User-lingering policy.** Auto-enable `loginctl enable-linger` for declared forge users, or explicit opt-in? Silent-on = agents survive logout; explicit = clearer trust posture. +2. **State directory location.** `/var/lib/forge/agents/${name}/` vs `${HOME}/.local/share/forge/agents/${name}/`. User path agrees with `runAs`-bounded permissions; system path eases ops introspection. +3. **Default `Restart=`.** `on-failure` is conservative; `always` matches v1's instinct. Recommend `on-failure` with `StartLimitBurst` until we have cost-per-restart data. +4. **Flake attribute naming.** `.#agent.${name}` vs `.#agents.${name}` vs `.#forge-agent-${name}` — affects how the command reads. +5. **Identity for inter-agent comms.** v1 conflated Mercury identity with agent name. v2: inherit, or let the comms module own an identity option so agents can have separate Mercury / pikachat / Nostr identities without `name` overloaded? +6. **Tmux always-on or opt-in.** Default-on preserves attach affordance; default-off keeps the substrate headless. +7. **Multi-host agent registry.** When a burst worker on a rented box runs `nix run .#agent.scout`, does the deploying host know it exists? Answers whether registry / heartbeat is v2 or follow-up. + +## Reference patterns surveyed + +- **systemd + sd_notify + journalctl** — the boring answer, and the one that does most of the work for free. +- **Pure Nix flake apps (`nix run`)** — the cross-machine address space and the literal launch command. +- **Kubernetes operators** — control-loop and CRD-shaped declarations. Informed keeping declaration and runtime separate; pod / operator weight is wrong for this stage. +- **MCP control-plane pattern** — informed "logs + signals first, MCP later" sequencing. +- **deploy-rs** (agicash-mints) — Nix-native deploy semantics. Right shape for host configuration; wrong shape for per-agent process management. +- **Legacy `launch-agent` and `tools/ribosome` on turtle** — the pattern being abstracted away. Their accumulated complexity (sqlite-polled Mercury ack loops, kindle-prompt paste-into-tmux dance, in-bash agent registry, `--dangerously-skip-permissions` baked in) is the concrete cost of skipping a declarative layer; v2 keeps none of it and recovers most of the capability from standard primitives.