diff --git a/.claude/skills/spawn-agent/SKILL.md b/.claude/skills/spawn-agent/SKILL.md index de58941..83d34e3 100644 --- a/.claude/skills/spawn-agent/SKILL.md +++ b/.claude/skills/spawn-agent/SKILL.md @@ -136,7 +136,7 @@ container run -d --rm \ --name "${CONTAINER_NAME}" \ --network claude-agent-net \ --cpus 8 \ - --memory 12G \ + --memory 3G \ --dns 1.1.1.1 \ -v "${GIT_ROOT}:/workspace" \ -v "${AGENTS_HOME}:/worktrees" \ diff --git a/CLAUDE.md b/CLAUDE.md index 912ab2c..8cf5841 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,5 +1,77 @@ # CLAUDE.md +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Overview + +stackai orchestrates parallel Claude Code agents in sandboxed Apple Containers (macOS 26+, ARM64). Each agent runs in its own git worktree for branch-isolated development. Two modes: direct container management (Makefile/CLI) and host-side skill coordination (spawn-agent). + +## Build & run + +```bash +# Build the production image (ARM64, Wolfi-based) +cd config && make build + +# Create the container network (required once) +make network + +# Interactive container session +make run + +# Spawn a headless agent +make spawn BRANCH=feat/foo TASK="implement feature X" + +# Monitor agents +make list-agents +make follow-agent BRANCH=feat/foo +make stop-agent BRANCH=feat/foo + +# Cleanup +make clean # remove image + containers +make clean-network # remove bridge network +make clean-all # remove everything +``` + +### Python CLI (`q` command) + +```bash +cd app/cli && uv sync +uv run q build +uv run q spawn --branch feat/foo --task "implement feature X" +uv run q agents list +``` + +## Testing + +```bash +# Python CLI tests (from app/cli/) +cd app/cli && uv sync +uv run pytest -v # full suite +uv run pytest tests/test_agents.py # single module +uv run pytest -k test_spawn # single test by name + +# Linting +uv run ruff check . + +# Entrypoint BDD tests (from config/) +cd config && shellspec --shell bash +``` + +CI runs all of these plus hadolint on both Dockerfiles and an Alpine builder stage compilation check. + +## Architecture + +- **`config/`** — Container infrastructure: `Dockerfile.wolfi` (production, multi-stage: Rust tool compilation → runtime with Claude CLI, Node, Python), `entrypoint.sh` (credential injection + worktree creation + su-exec privilege drop), `Makefile` (orchestration) +- **`app/cli/`** — Python CLI (`q`) using Typer+Rich that wraps Makefile targets. Entry point registered as `q` in pyproject.toml. Commands delegate to `make` via `utils.run_make()` +- **`.claude/skills/`** — Host-side Claude Code skills for multi-agent orchestration (spawn-agent, spawn-agent-workspace) +- **`docs/agents/`** — All project documentation (container reference, CLI, setup/auth, skill architecture, evals) + +### Key concepts + +- **Dual-token architecture**: Host uses `CLAUDE_CONTAINER_OAUTH_TOKEN`; inside the container it maps to `CLAUDE_CODE_OAUTH_TOKEN`. Host credentials are mounted read-only and copied (never modified in place). +- **Worktree isolation**: Each agent gets its own git worktree under `$AGENTS_HOME` (default: sibling `.worktrees/` directory). Branches are merge-ready when the agent finishes. +- **Apple Container CLI**: This project uses Apple's container runtime, not Docker. The Makefile targets use `container` commands. + ## Documentation When implementing a new feature, always update the relevant documentation in `docs/`. @@ -10,3 +82,13 @@ When implementing a new feature, always update the relevant documentation in `do - If a new subsystem is introduced with no existing doc, create a new file under the appropriate `docs/` subdirectory Documentation should reflect the current state of the code. Keep troubleshooting tables and flow descriptions in sync with the actual implementation. + +## Skill evals + +When modifying any skill under `.claude/skills/`, run its evals locally before considering the change complete. Evals verify that the skill still produces correct outputs across all test scenarios. + +``` +/skill-creator:skill-creator run evals for the skill at ~/.claude/skills// +``` + +See `docs/agents/evals.md` for full details on prerequisites, how to add new evals, and how to interpret results. diff --git a/README.md b/README.md new file mode 100644 index 0000000..4c10196 --- /dev/null +++ b/README.md @@ -0,0 +1,200 @@ +# stackai + +[![CI](https://github.com/deimagjas/stackai/actions/workflows/ci.yml/badge.svg)](https://github.com/deimagjas/stackai/actions/workflows/ci.yml) +[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE) + +Run parallel Claude Code agents in sandboxed Apple Containers with isolated git worktrees. + +## What is this? + +stackai is an infrastructure toolkit that spawns headless Claude Code instances inside lightweight Linux containers on macOS. Each agent works in its own git worktree, enabling parallel development workflows — multiple agents can work on different branches of the same repository simultaneously. + +--- + +## Two ways to use stackai + +stackai provides two distinct usage modes. Both rely on the same container image and infrastructure, but differ in **where the orchestration happens**. + +### Mode 1: Direct container usage + +Use the Makefile or the `q` CLI to build the image, enter a container interactively, or spawn headless agents directly from your terminal. + +**Best for:** +- Running Claude interactively inside the container (full Linux environment with Rust CLI tools) +- Spawning one-off headless agents via `make spawn` or `q spawn` +- Automation scripts and CI-driven agent workflows +- Using Claude features that work reliably inside containers + +```bash +# Interactive session inside the container +make run + +# Spawn a headless agent +make spawn BRANCH=feat/oauth2 TASK="Implement OAuth2 with JWT tokens" +``` + +### Mode 2: Host skill (spawn-agent) + +Install the `spawn-agent` skill in your host Claude Code session. Claude on your host becomes a **coordinator** that delegates tasks to containerized agents and monitors their progress via `container logs`. + +**Best for:** +- Orchestrating multiple agents in parallel from a single Claude conversation +- Tasks that require host-side Claude features not yet available inside containers (e.g., `/remote-control`, MCP servers, IDE integrations) +- Complex workflows where the coordinator needs to review agent output and make decisions before spawning the next agent + +``` +# From your host Claude Code session: +> spawn an agent on feat/oauth2 to implement OAuth2 with JWT tokens +> spawn another on test/auth to write integration tests for the auth module +> check the status of both agents +``` + +The skill automatically detects the task type (feature, test, mutation, explore) and builds an appropriate prompt for each agent. + +### Why both modes? + +Some Claude Code features — such as `/remote-control`, MCP tool connections, and certain IDE integrations — are not yet supported inside containers. The host skill mode lets you keep those capabilities on your host while still delegating compute-heavy coding tasks to isolated container agents. As container support matures, more features will work directly inside the container. + +### Decision flow + +```mermaid +flowchart TD + A[I want to use stackai] --> B{Do I need host-only\nClaude features?} + + B -->|Yes| C{Which features?} + C -->|/remote-control\nMCP servers\nIDE integrations| D[Use host skill mode] + D --> D1["Install spawn-agent skill\nClaude on host coordinates\nagents in containers"] + + B -->|No| E{How many agents?} + E -->|One| F{Interactive or headless?} + F -->|Interactive| G["make run\n(enter the container)"] + F -->|Headless| H["make spawn BRANCH=... TASK=...\n(fire and forget)"] + + E -->|Multiple in parallel| I{Do I need coordination\nbetween agents?} + I -->|Yes| D + I -->|No| J["Run multiple make spawn\ncommands in parallel"] +``` + +--- + +## Key components + +| Component | Path | Description | +|---|---|---| +| Container image | `config/Dockerfile.wolfi` | ARM64 Wolfi-based image with Claude Code, Rust CLI tools, and Node/Python runtimes | +| Entrypoint | `config/entrypoint.sh` | Credential injection, worktree creation, headless agent launch | +| Makefile | `config/Makefile` | Build, spawn, monitor, and stop agents | +| CLI (`q`) | `app/cli/` | Python CLI (Typer) wrapping Makefile targets | +| Spawn skill | `docs/agents/spawn-agent-skill.md` | Host-side Claude Code skill for multi-agent coordination | +| Docs | `docs/agents/` | Architecture, setup, spawn skill, evals | + +## Requirements + +- macOS 26+ (Sequoia) with Apple Container CLI +- Apple Silicon (ARM64) +- Claude Pro or Max subscription ([claude.ai](https://claude.ai)) +- Python 3.13+ and [uv](https://docs.astral.sh/uv/) +- Git + +## Quick start + +### 1. Build the image + +```bash +cd config +make build +``` + +### 2. Set up authentication + +```bash +claude login # authenticate via browser OAuth +claude setup-token # generates the container token +export CLAUDE_CONTAINER_OAUTH_TOKEN= +``` + +See [docs/agents/setup.md](docs/agents/setup.md) for the full dual-token architecture explanation. + +### 3. Create the network + +```bash +make network +``` + +### 4. Spawn an agent + +```bash +make spawn BRANCH=feat/oauth2 TASK="Implement OAuth2 with JWT tokens" +``` + +Or using the CLI: + +```bash +cd app/cli && uv sync +q spawn --branch feat/oauth2 --task "Implement OAuth2 with JWT tokens" +``` + +### 5. Monitor + +```bash +make list-agents # active containers and worktrees +make follow-agent BRANCH=feat/oauth2 # stream logs +make stop-agent BRANCH=feat/oauth2 # stop when done +``` + +## Project structure + +``` +stackai/ +├── app/ +│ ├── cli/ # Python CLI (q command) +│ └── agents-templates/ # Agent template examples +├── config/ +│ ├── Dockerfile.wolfi # Production image (ARM64, glibc) +│ ├── Dockerfile # CI image (Alpine, amd64) +│ ├── Makefile # Build/spawn/monitor targets +│ ├── entrypoint.sh # Container entrypoint +│ └── spec/ # ShellSpec BDD tests +├── docs/agents/ +│ ├── container-agent.md # Image and Makefile reference +│ ├── spawn-agent-skill.md # Spawn skill architecture +│ ├── setup.md # Authentication guide +│ ├── evals.md # Evaluation framework +│ └── cli.md # CLI command reference +├── iac/ # Infrastructure as Code +└── model/ # ML fine-tuning experiments +``` + +## How it works + +1. **Build** — `Dockerfile.wolfi` compiles Rust CLI tools (ripgrep, fd, bat, eza, dust, procs, bottom) in a builder stage, then installs Claude Code CLI, Node.js, Python, and GitHub CLI in the runtime stage. + +2. **Spawn** — The Makefile creates a git worktree for the target branch, launches a container with read-only credential mounts, and runs `claude --dangerously-skip-permissions -p ""` as a non-root user via `su-exec`. + +3. **Isolate** — Each container gets its own worktree under `$AGENTS_HOME`, its own credential copy, and its own `CLAUDE_CODE_OAUTH_TOKEN`. The host credentials are never modified. + +4. **Merge** — When the agent finishes, its branch is ready for review and merge via standard git workflow. + +## CI + +GitHub Actions runs on every push to `main` and on pull requests: + +| Job | What it checks | +|---|---| +| `lint` | ruff on `app/cli/` | +| `test-cli` | pytest for the Python CLI | +| `test-entrypoint` | ShellSpec BDD tests for `entrypoint.sh` | +| `dockerfile-lint` | hadolint on both Dockerfiles | +| `docker-build` | Alpine builder stage compilation | + +## Documentation + +- [Container image and Makefile](docs/agents/container-agent.md) +- [Setup and authentication](docs/agents/setup.md) +- [Spawn agent skill](docs/agents/spawn-agent-skill.md) +- [CLI reference](docs/agents/cli.md) +- [Evals](docs/agents/evals.md) + +## License + +[Apache 2.0](LICENSE) diff --git a/config/Makefile b/config/Makefile index 4cd8e9b..85d4c52 100644 --- a/config/Makefile +++ b/config/Makefile @@ -39,7 +39,7 @@ NAME ?= qubits-team NETWORK ?= claude-agent-net SUBNET ?= 192.168.100.0/24 CPUS ?= 8 -MEMORY ?= 12G +MEMORY ?= 3G # Agent variables BRANCH ?= agent-$(shell date +%s) diff --git a/docs/agents/container-agent.md b/docs/agents/container-agent.md index 83e24ed..cca325b 100644 --- a/docs/agents/container-agent.md +++ b/docs/agents/container-agent.md @@ -149,7 +149,7 @@ IMPORTANT: **Why the worktree is created inside the container:** Git needs acces | `NETWORK` | `claude-agent-net` | Agent bridge network | | `SUBNET` | `192.168.100.0/24` | Network CIDR | | `CPUS` | `8` | CPUs allocated to each container | -| `MEMORY` | `12G` | RAM allocated to each container | +| `MEMORY` | `3G` | RAM allocated to each container | | `BRANCH` | `agent-` | Agent branch to spawn | | `TASK` | `Explore the codebase...` | Agent task | | `AGENTS_HOME` | `/.worktrees` | Fallback if not in env | diff --git a/docs/agents/spawn-agent-skill.md b/docs/agents/spawn-agent-skill.md index 4bb3860..d01a46a 100644 --- a/docs/agents/spawn-agent-skill.md +++ b/docs/agents/spawn-agent-skill.md @@ -239,7 +239,7 @@ Requirements: container run -d --rm \ --name "stackai-feat-oauth2" \ --network claude-agent-net \ - --cpus 8 --memory 12G --dns 1.1.1.1 \ + --cpus 8 --memory 3G --dns 1.1.1.1 \ -v "${GIT_ROOT}:/workspace" \ -v "${AGENTS_HOME}:/worktrees" \ -v "${HOME}/.claude:/root/.claudenew:ro" \