Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,11 @@ which bot, which branch, which build command) is already config-driven.
model gateway and has `claude`, `jq`, `python3`; org/repo secrets
`LITELLM_BASE_URL`, `LITELLM_API_KEY`, `LITELLM_NO_PROXY` (optional),
`AGENT_GH_TOKEN` (a PAT — see [docs/prerequisites.md](docs/prerequisites.md)
for the *why*), and optionally `AUTO_MERGE_TEAM`.
for the *why*), and optionally `AUTO_MERGE_TEAM`. **Need that runner?** The
[cookbooks](docs/cookbooks/) have copy-paste recipes for
[self-hosting](docs/cookbooks/self-hosting.md),
[DigitalOcean](docs/cookbooks/digitalocean.md), and
[AWS](docs/cookbooks/aws.md). (The **guard** gate needs no runner at all.)
2. **Drop in `.agent-ops.json`** — copy [`examples/consumer/.agent-ops.json`](examples/consumer/.agent-ops.json)
and edit the gates / build command / labels / observer for your repo. Every
field is optional ([schema](schema/agent-ops.schema.json)).
Expand Down
154 changes: 154 additions & 0 deletions docs/cookbooks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# agent-ops cookbooks

Recipes for standing up the infrastructure a consumer repo needs to run the
agent-ops loop. Pick a platform:

- [Self-hosting](./self-hosting.md) — your own box, VM, or hypervisor (libvirt /
Proxmox / bare metal)
- [DigitalOcean](./digitalocean.md) — a Droplet
- [AWS](./aws.md) — an EC2 instance

> New to agent-ops? Read [`../setup.md`](../setup.md) and
> [`../prerequisites.md`](../prerequisites.md) first. These cookbooks are the
> *platform-specific* half of that setup.

---

## Do you even need a runner?

| Surface | Needs a self-hosted runner? | Why |
|---|---|---|
| **guard** (leakage / secret hygiene) | **No** — runs on GitHub-hosted `ubuntu-latest` | Pure linters; only need the repo + the public internet. Free on public repos. |
| **triage, implement (wiwi), review (vivi), revise** | **Yes** | They run the `claude` CLI against a model endpoint. A GitHub-hosted runner can't reach a private/internal model gateway, and you usually don't want your API key on ephemeral cloud runners. |
| **observe (mara)** | A small box, **isolated from prod** | A systemd timer; see [`../../SELF-DOGFOOD.md`](../../SELF-DOGFOOD.md) and the observer notes in `setup.md`. |

So: if you only want the **guard** gate, you need **zero infrastructure** — just
add the `guard.yml` wrapper. Everything below is for the **agentic** surfaces.

---

## The model endpoint (decide this first)

agent-ops's workflows read three secrets to reach the model. The names start
with `LITELLM_` for historical reasons — **it's just "an Anthropic-compatible
endpoint."** Two common choices:

| Choice | `LITELLM_BASE_URL` | `LITELLM_API_KEY` | model |
|---|---|---|---|
| **Anthropic API directly** (simplest) | `https://api.anthropic.com` | your `sk-ant-…` key | a real Claude model id (the workflow default `claude-3-5-sonnet-20241022`, or pass a newer one via the `model` input) |
| **A gateway you run** (LiteLLM / OpenAI-compatible proxy) | `https://your-gateway/v1` | the gateway's key | whatever name the gateway maps |

The `claude` CLI on the runner reads `ANTHROPIC_BASE_URL` / `ANTHROPIC_API_KEY`,
which the workflows set from these secrets. **Pointing straight at the Anthropic
API is the least moving parts** — start there.

---

## Sizing

| Workload | Suggested box |
|---|---|
| triage / review only (read + comment) | 2 vCPU / 4 GB |
| + implement (wiwi runs your `build_cmd`) | size for **your build** — wiwi needs your project's full toolchain + enough RAM/CPU to compile. A Rust/heavy build often wants 4 vCPU / 8–16 GB. |

The runner also needs **whatever `.agent-ops.json` `implement.build_cmd`
invokes** installed (your compiler, test tooling, etc.) — that's project-specific
and not covered here.

---

## Shared steps (referenced by every platform recipe)

Once you have a Linux box (Ubuntu 22.04/24.04 assumed) the rest is identical.

### A. Bootstrap the box

```bash
sudo apt-get update
sudo apt-get install -y git jq python3 curl ca-certificates

# Claude Code — the `claude` CLI the agent scripts call.
# See https://docs.claude.com/en/docs/claude-code for the current installer.
# Native install:
curl -fsSL https://claude.ai/install.sh | bash
# …or via npm if you prefer: npm install -g @anthropic-ai/claude-code
claude --version # verify

# (implement only) also install your project's build toolchain here.
```

### B. Register the GitHub Actions runner

Get a registration token: repo **Settings → Actions → Runners → New self-hosted
runner**, or:

```bash
gh api -X POST repos/<OWNER>/<REPO>/actions/runners/registration-token -q .token
```

Then on the box (as a non-root user, e.g. `runner`):

```bash
mkdir -p ~/actions-runner && cd ~/actions-runner
# grab the latest release tarball from https://github.com/actions/runner/releases
RUNNER_VER=2.319.1
curl -fsSL -o runner.tar.gz \
https://github.com/actions/runner/releases/download/v${RUNNER_VER}/actions-runner-linux-x64-${RUNNER_VER}.tar.gz
tar xzf runner.tar.gz

./config.sh --url https://github.com/<OWNER>/<REPO> \
--token <REGISTRATION_TOKEN> \
--name agent-ops-runner \
--labels self-hosted,agent-ops \
--unattended --replace

sudo ./svc.sh install "$USER" # install as a systemd service so it survives reboots
sudo ./svc.sh start
```

> **Label** = whatever you put in your wrapper workflows' `runner_labels`
> (`'["self-hosted","agent-ops"]'` here). Keep them in sync. You can register one
> runner at the **org** level and share it across repos instead of per-repo.

### C. Secrets + wrappers

```bash
# model endpoint (see the table above)
gh secret set LITELLM_BASE_URL --repo <OWNER>/<REPO> --body "https://api.anthropic.com"
gh secret set LITELLM_API_KEY --repo <OWNER>/<REPO> --body "sk-ant-…"

# a PAT — NOT GITHUB_TOKEN. The loop must trigger label/dispatch/push events,
# which GITHUB_TOKEN-authored events are suppressed from. Scope: repo + workflow.
gh secret set AGENT_GH_TOKEN --repo <OWNER>/<REPO> --body "<PAT>"

# optional
gh secret set LITELLM_NO_PROXY --repo <OWNER>/<REPO> --body "<hosts to bypass a proxy>"
gh secret set AUTO_MERGE_TEAM --repo <OWNER>/<REPO> --body "alice,bob" # auto-merge allowlist
```

Then add `.agent-ops.json` (see [`../config-reference.md`](../config-reference.md))
and the thin wrapper workflows (copy from
[`../../examples/consumer/`](../../examples/consumer/)), making sure each
wrapper's `runner_labels` matches your runner's label.

### D. Smoke test

1. Open an issue → the triage workflow runs on your runner and replies.
2. (implement) add the `agent:try` label → wiwi branches, builds, opens a DRAFT PR.
3. Push a PR → after CI, the review workflow posts a review.

If a job sits **queued** forever, the runner isn't online or its label doesn't
match. If it fails at the `claude` step, check the model secrets + that the box
can reach `LITELLM_BASE_URL`.

---

## Security notes

- The runner holds your **model API key** and a **GitHub PAT** — treat the box as
sensitive. Prefer a dedicated box per trust boundary; don't co-host it with
untrusted workloads.
- Self-hosted runners on **public** repos can run code from forked PRs. agent-ops's
reusable workflows only act on **same-repo** PRs by design, but review your
repo's `pull_request` vs `pull_request_target` settings before exposing a runner.
- Keep the box patched; the runner auto-updates itself, the OS does not.
78 changes: 78 additions & 0 deletions docs/cookbooks/aws.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Cookbook: agent-ops runner on AWS (EC2)

Run the runner on an EC2 instance — good if your org already lives in AWS, your
model gateway is in a VPC, or you want spot/auto-stop cost controls.

> Prereq concepts + the shared **Steps A–D** (bootstrap, register runner,
> secrets, smoke test) live in [`./README.md`](./README.md). This page only
> covers **getting the instance**.

## 1. Launch the instance

### Find a current Ubuntu 24.04 AMI (Canonical publishes it via SSM)

```bash
AMI=$(aws ssm get-parameter \
--name /aws/service/canonical/ubuntu/server/24.04/stable/current/amd64/hvm/ebs-gp3/ami-id \
--query 'Parameter.Value' --output text)
echo "$AMI"
```

### Run it

```bash
aws ec2 run-instances \
--image-id "$AMI" \
--instance-type t3.medium \
--key-name <your-keypair> \
--security-group-ids <sg-id> \
--block-device-mappings 'DeviceName=/dev/sda1,Ebs={VolumeSize=30,VolumeType=gp3}' \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=agent-ops-runner}]'

# get the public IP (or use SSM Session Manager and skip SSH/inbound entirely)
aws ec2 describe-instances --filters Name=tag:Name,Values=agent-ops-runner \
--query 'Reservations[].Instances[].PublicIpAddress' --output text
ssh ubuntu@<PUBLIC_IP>
```

### Security group

- **Outbound**: 443 (GitHub + your model endpoint). That's all the runner needs.
- **Inbound**: nothing required. For setup, either allow 22 from *your* IP, or —
better — use **SSM Session Manager** (attach an instance profile with
`AmazonSSMManagedInstanceCore`) and open no inbound ports at all.

## 2. Sizing & cost (rough, on-demand, us-east-1)

| Instance | vCPU / RAM | ~ / month (24×7) | Good for |
|---|---|---|---|
| `t3.medium` | 2 / 4 GB | ~$30 | triage + review |
| `t3.xlarge` | 4 / 16 GB | ~$120 | + implement with a heavy build |

Cost controls:

- **Stop when idle** — you only pay for the EBS volume while stopped. A simple
EventBridge schedule + Lambda (or `aws ec2 stop-instances` from cron elsewhere)
can park it overnight.
- **Spot** — for non-urgent loops, a spot instance cuts ~70%. Add `--instance-market-options`
and tolerate occasional interruption (the runner re-registers on next boot if
you bake the setup into user-data or an AMI).

## 3. (optional) Bake it with user-data

Put **Step A** (bootstrap) into the instance **user-data** so a fresh instance
comes up ready; keep the runner **registration** (Step B) separate since the
token is short-lived — or store a re-registration script that fetches a token at
boot via an instance role with GitHub App credentials.

## 4. Bring it online

Continue with **Steps A–D** in [`./README.md`](./README.md): bootstrap the
instance, register the runner (label `self-hosted,agent-ops`), set the secrets,
smoke-test.

## Pros / cons

| 👍 | 👎 |
|---|---|
| VPC-native (reach a private gateway via subnet/PrivateLink); spot + auto-stop cost controls; SSM = zero inbound | More moving parts (AMI, SG, IAM); priciest of the three at 24×7 on-demand unless you stop/spot it |
61 changes: 61 additions & 0 deletions docs/cookbooks/digitalocean.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Cookbook: agent-ops runner on DigitalOcean

A Droplet is the fastest hosted option: a few dollars a month, up in a minute,
no cloud-IAM ceremony.

> Prereq concepts + the shared **Steps A–D** (bootstrap, register runner,
> secrets, smoke test) live in [`./README.md`](./README.md). This page only
> covers **getting the Droplet**.

## 1. Create the Droplet

### With `doctl`

```bash
# one-time: doctl auth init ; doctl compute ssh-key list (note your key id/fingerprint)
doctl compute droplet create agent-ops-runner \
--image ubuntu-24-04-x64 \
--size s-2vcpu-4gb \
--region nyc3 \
--ssh-keys <your-ssh-key-fingerprint> \
--wait

doctl compute droplet list agent-ops-runner --format Name,PublicIPv4
ssh root@<PUBLIC_IP>
```

### Or the control panel

Create → Droplets → Ubuntu 24.04 → **Basic / Regular**, `s-2vcpu-4gb`, add your
SSH key, create. Then `ssh root@<ip>`.

## 2. Sizing & cost (rough)

| Size | vCPU / RAM | ~ / month | Good for |
|---|---|---|---|
| `s-2vcpu-4gb` | 2 / 4 GB | ~$24 | triage + review |
| `s-4vcpu-8gb` | 4 / 8 GB | ~$48 | + implement with a moderate build |

To trim cost, you can **power the Droplet off when idle** (you still pay for the
disk while off, but not the vCPU) or destroy + recreate from a snapshot. For an
always-responsive loop, leave it running.

## 3. Harden a little

- The Droplet gets a public IP. Lock SSH to your IP via the **DigitalOcean Cloud
Firewall** (inbound 22 from your address only; the runner needs **no** inbound
otherwise — it dials out to GitHub + your model endpoint).
- Create an unprivileged `runner` user instead of using `root` for the runner
service.

## 4. Bring it online

Continue with **Steps A–D** in [`./README.md`](./README.md): bootstrap the
Droplet, register the runner (label `self-hosted,agent-ops`), set the secrets,
smoke-test.

## Pros / cons

| 👍 | 👎 |
|---|---|
| Cheapest hosted option; trivial to create/destroy; snapshots | Public IP to firewall; you still patch the OS; egress only — can't reach a *private* model gateway unless you VPN/tunnel |
81 changes: 81 additions & 0 deletions docs/cookbooks/self-hosting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Cookbook: self-hosting the agent-ops runner

Run the agent-ops loop on your own hardware — a spare box, a VM on your
hypervisor (libvirt / Proxmox / VMware / Hyper-V), or bare metal. This is the
cheapest option if you already have capacity, and the only option if your model
endpoint is on a private network.

> Prereq concepts + the shared **Steps A–D** (bootstrap, register runner,
> secrets, smoke test) live in [`./README.md`](./README.md). This page only
> covers **getting the box**.

## 1. Provision the box

Any Ubuntu 22.04/24.04 (or compatible) machine with:

- 2 vCPU / 4 GB minimum (size up for `implement` — see README sizing).
- Outbound HTTPS to `github.com`, the actions runner release host, and your
`LITELLM_BASE_URL`. **No inbound ports required** (the runner dials out).
- ~20 GB disk (more if your build caches are large).

### Option A — a VM via cloud-init (libvirt example)

This mirrors how the framework's own reference deployment is built. Create a
`user-data` and boot an Ubuntu cloud image:

```yaml
#cloud-config
hostname: agent-ops-runner
users:
- name: runner
sudo: ALL=(ALL) NOPASSWD:ALL
groups: [sudo]
shell: /bin/bash
ssh_authorized_keys:
- ssh-ed25519 AAAA...your-key...
package_update: true
packages: [git, jq, python3, curl, ca-certificates]
```

```bash
cloud-localds seed.iso user-data
qemu-img create -F qcow2 -b /path/to/noble-server-cloudimg-amd64.img -f qcow2 disk.qcow2 20G
virt-install --name agent-ops-runner --memory 4096 --vcpus 2 \
--disk path=disk.qcow2,format=qcow2 --disk path=seed.iso,device=cdrom \
--os-variant ubuntu24.04 --network network=default \
--graphics none --import --noautoconsole
virsh autostart agent-ops-runner # survive host reboots
```

Proxmox / VMware / Hyper-V: provision an equivalent Ubuntu VM however you
normally do, then continue.

### Option B — bare metal / an existing box

Just SSH in. Make a dedicated unprivileged user (`runner`) so the runner isn't
root.

## 2. Network notes

- **Behind a corporate proxy / no direct internet?** Export `HTTPS_PROXY` on the
box for the install steps, and set the `LITELLM_NO_PROXY` secret so the
`claude` CLI bypasses the proxy for your model gateway (or vice-versa). The
runner service environment honours the proxy you set.
- **Private model gateway?** Self-hosting is the natural fit — the box just needs
a route to it (same VLAN, VPN, or a tunnel). This is the case a GitHub-hosted
runner *can't* serve.

## 3. Bring it online

Continue with **Steps A–D** in [`./README.md`](./README.md): bootstrap the box,
register the runner (label `self-hosted,agent-ops`), set the secrets, smoke-test.

## Pros / cons

| 👍 | 👎 |
|---|---|
| Free if you have capacity; reaches private model gateways; full control | You own patching, uptime, and reboot-recovery (use `virsh autostart` / a systemd-enabled service so the runner restarts) |

> Tip: register the runner as a **systemd service** (`./svc.sh install`) so a host
> reboot brings it back. A VM that doesn't auto-start its runner is the #1 cause
> of "all my CI is queued forever."
Loading