feat: structured agent monitoring with status persistence and graceful UX

.claude/skills/spawn-agent/SKILL.md

@@ -168,24 +168,56 @@ ls -la "${AGENTS_HOME}" 2>/dev/null || echo "(no worktrees yet at $AGENTS_HOME)"
 
 Show the user a readable table with both container status and worktree list.
 
-## Reading agent output (context)
+## Reading agent output
 
-`container logs` captures everything the agent prints (Claude's reasoning,
-tool calls, results). Use this to pass context back to the user:
+Agents persist structured monitoring data in the worktree at
+`${AGENTS_HOME}/<branch>/.agent/`. Use the right source for each question:
+
+### Quick status (preferred for "what is agent X doing?")
+
+```bash
+# Read status.json — works even after container exits
+cat "${AGENTS_HOME}/${BRANCH}/.agent/status.json" 2>/dev/null
+```
+
+Returns JSON with phase, branch, task, timestamps, exit code, and commit count.
+Phases: `starting` → `working` → `completed` | `errored`.
+
+### Structured lifecycle events
 
 ```bash
 CONTAINER_NAME="${PROJECT_NAME}-${CONTAINER_BRANCH}"
 
-# Last 100 lines (good for summary)
-container logs -n 100 "${CONTAINER_NAME}"
+# From live container
+container logs "${CONTAINER_NAME}" 2>/dev/null | grep '^\[agent:'
+
+# From persisted logs (after container exits)
+grep '^\[agent:' "${AGENTS_HOME}/${BRANCH}/.agent/agent.log" 2>/dev/null
+```
+
+### Full logs
 
-# Follow live output (for running agents)
+```bash
+# Live container (while running)
+container logs -n 100 "${CONTAINER_NAME}"
 container logs -f "${CONTAINER_NAME}"
+
+# Persisted logs (after container exits)
+tail -100 "${AGENTS_HOME}/${BRANCH}/.agent/agent.log"
 ```
 
+### Decision flow
+
+- **"What is agent X doing?"** → read `status.json` (instant, always works)
+- **"What did agent X do?"** → read `agent.log` (persisted, works post-exit)
+- **"Show me live output"** → `container logs -f` (only while running)
+
 Read the logs and **summarize the agent's progress** — don't just dump raw output.
 Tell the user: what the agent is working on, what it has done, what step it's at.
 
+**Important:** When the container is gone (agent finished), do NOT attempt
+`container logs` — it will fail. Use the persisted files in `.agent/` instead.
+
 ## Integrating agent work
 
 When an agent finishes, its commits already exist in the **host repo** — the

.claude/skills/spawn-agent/evals/evals.json

@@ -44,15 +44,39 @@
     {
       "id": 4,
       "prompt": "Check what the feat/oauth2 agent is doing right now. Give me a summary of its progress.",
-      "expected_output": "Claude runs container logs for the feat/oauth2 agent (sanitized to feat-oauth2 in name), reads the output, and provides a plain-language summary of the agent's progress — not just a raw log dump.",
+      "expected_output": "Claude checks status.json from the worktree for quick status, and/or reads container logs (sanitized name feat-oauth2). Provides a plain-language summary — not just a raw log dump.",
       "files": [],
       "expectations": [
-        "Runs `container logs` (not container run or container list)",
-        "Uses correct sanitized container name: includes feat-oauth2",
-        "Provides a summary or interpretation of logs, not just raw output",
+        "Reads status.json from $AGENTS_HOME/feat/oauth2/.agent/ OR runs `container logs`",
+        "Uses correct sanitized container name if using container logs: includes feat-oauth2",
+        "Provides a summary or interpretation, not just raw output",
         "Does NOT spawn a new container"
       ]
     },
+    {
+      "id": 7,
+      "prompt": "The feat/oauth2 agent finished a while ago. What was the result? Did it succeed?",
+      "expected_output": "Claude reads status.json from the persisted worktree directory. Reports the phase (completed/errored), exit code, commit count, and duration. Does NOT attempt container logs on a stopped container.",
+      "files": [],
+      "expectations": [
+        "Reads status.json from $AGENTS_HOME/feat/oauth2/.agent/status.json",
+        "Reports phase, exit code, and commit count from the status file",
+        "Does NOT attempt `container logs` on a container that no longer exists",
+        "Does NOT show error messages about missing containers"
+      ]
+    },
+    {
+      "id": 8,
+      "prompt": "Show me the full logs from the feat/oauth2 agent. It finished already.",
+      "expected_output": "Claude reads the persisted agent.log from the worktree directory. Shows the saved logs with a note that these are persisted logs from a finished agent.",
+      "files": [],
+      "expectations": [
+        "Reads from $AGENTS_HOME/feat/oauth2/.agent/agent.log (persisted logs)",
+        "Does NOT attempt `container logs` as the primary source for a finished agent",
+        "Indicates these are saved/persisted logs",
+        "Does NOT show confusing error output about missing containers"
+      ]
+    },
     {
       "id": 5,
       "prompt": "The feat/oauth2 agent finished. Merge its work into my current branch.",

.claude/skills/spawn-agent/evals/list_and_monitor.md

@@ -14,7 +14,7 @@ User says:
 
 1. Skill triggers
 2. Runs: `container list 2>/dev/null | grep "qubits-team"`
-3. Also shows worktrees on disk: `ls -la <parent>/.worktrees/`
+3. Also shows worktrees on disk (with status from `status.json` if available)
 4. Presents output in a readable format to the user
 
 ---
@@ -27,12 +27,29 @@ User says:
 ## Expected behavior (monitor)
 
 1. Skill triggers
-2. Sanitizes: `feat/jwt-auth` → `qubits-team-feat-jwt-auth`
-3. Runs: `container logs -n 100 qubits-team-feat-jwt-auth`
-4. **Reads and summarizes** the logs — does NOT just dump raw output
-5. Tells user: agent is working on X, currently at step Y, last action was Z
+2. Reads `status.json` from `$AGENTS_HOME/feat/jwt-auth/.agent/status.json` for quick status
+3. If more detail needed, reads container logs or persisted `.agent/agent.log`
+4. Sanitizes container name correctly: `feat/jwt-auth` → `qubits-team-feat-jwt-auth`
+5. **Reads and summarizes** the output — does NOT just dump raw logs
+6. Tells user: agent is working on X, currently at step Y, last action was Z
+
+---
+
+## Input (part 3 — status post-exit)
+
+User says:
+> "What happened with the feat/jwt-auth agent?"
+
+## Expected behavior (status post-exit)
+
+1. Skill triggers
+2. Reads `status.json` from `$AGENTS_HOME/feat/jwt-auth/.agent/status.json`
+3. Reports phase (completed/errored), exit code, commit count, duration
+4. Does NOT attempt `container logs` on a stopped container
+5. If user wants full logs, reads from `.agent/agent.log` (persisted)
 
 ## Must NOT do
 
 - Must not just print raw container logs without summarizing
 - Must not confuse container name sanitization (/ → -)
+- Must not show errors when container is gone (use persisted files instead)

app/cli/src/container_cli/commands/agents.py

@@ -1,12 +1,23 @@
+import json
+import os
+from pathlib import Path
 from typing import Annotated
 
 import typer
 
-from container_cli.utils import check_token, run_make
+from container_cli.utils import check_token, find_git_root, run_make
 
 app = typer.Typer(help="Agent lifecycle commands")
 
 
+def _agents_home() -> Path:
+    """Resolve AGENTS_HOME, falling back to sibling .worktrees/ directory."""
+    env_val = os.environ.get("AGENTS_HOME")
+    if env_val:
+        return Path(env_val)
+    return find_git_root().parent / ".worktrees"
+
+
 @app.command()
 def spawn(
     branch: Annotated[str, typer.Option("--branch", help="Git branch for the agent worktree")],
@@ -55,3 +66,26 @@ def stop(
 ) -> None:
     """Stop a branch agent container."""
     run_make("stop-agent", {"BRANCH": branch})
+
+
+@app.command()
+def status(
+    branch: Annotated[str, typer.Option("--branch", help="Agent branch name")],
+) -> None:
+    """Show agent status from persisted status.json file."""
+    status_file = _agents_home() / branch / ".agent" / "status.json"
+    if not status_file.exists():
+        typer.echo(f"[status] No status file found for branch '{branch}'.")
+        typer.echo(f"[status] Expected at: {status_file}")
+        raise typer.Exit(1)
+
+    data = json.loads(status_file.read_text())
+    typer.echo(json.dumps(data, indent=2))
+
+
+@app.command()
+def summary(
+    branch: Annotated[str, typer.Option("--branch", help="Agent branch name")],
+) -> None:
+    """Show structured lifecycle events for a branch agent."""
+    run_make("summary-agent", {"BRANCH": branch})

config/Makefile

@@ -60,7 +60,7 @@ CONTAINER_BRANCH := $(shell echo "$(BRANCH)" | tr '/_ ' '-' | tr '[:upper:]' '[:
 # Host env var holding the OAuth token (avoids collision with host Claude instance)
 HOST_TOKEN_VAR := CLAUDE_CONTAINER_OAUTH_TOKEN
 
-.PHONY: build network run shell spawn list-agents logs-agent follow-agent stop-agent clean clean-network clean-all help
+.PHONY: build network run shell spawn list-agents logs-agent follow-agent stop-agent status-agent summary-agent clean clean-network clean-all help
 
 # ── Build ─────────────────────────────────────────────────────────────────────
 
@@ -131,13 +131,74 @@ list-agents:
 	@container list 2>/dev/null | grep "$(PROJECT_NAME)" || echo "  (none)"
 	@echo ""
 	@echo "[agents] Worktrees in $(WORKTREES_DIR):"
-	@ls -la "$(WORKTREES_DIR)" 2>/dev/null || echo "  (none yet)"
+	@if [ -d "$(WORKTREES_DIR)" ]; then \
+		for dir in $(WORKTREES_DIR)/*/; do \
+			[ -d "$$dir" ] || continue; \
+			branch=$$(basename "$$dir"); \
+			status_file="$$dir/.agent/status.json"; \
+			if [ -f "$$status_file" ] && command -v jq >/dev/null 2>&1; then \
+				phase=$$(jq -r '.phase // "unknown"' "$$status_file" 2>/dev/null || echo "unknown"); \
+				printf "  %-30s %s\n" "$$branch" "$$phase"; \
+			else \
+				printf "  %-30s %s\n" "$$branch" "(no status)"; \
+			fi; \
+		done; \
+	else \
+		echo "  (none yet)"; \
+	fi
 
 logs-agent:
-	container logs $(PROJECT_NAME)-$(CONTAINER_BRANCH)
+	@container logs $(PROJECT_NAME)-$(CONTAINER_BRANCH) 2>/dev/null \
+		|| { \
+			LOG_FILE="$(WORKTREES_DIR)/$(BRANCH)/.agent/agent.log"; \
+			if [ -f "$$LOG_FILE" ]; then \
+				echo "[logs] Container $(PROJECT_NAME)-$(CONTAINER_BRANCH) no longer running (agent finished)."; \
+				echo "[logs] Showing saved logs from $$LOG_FILE"; \
+				echo "---"; \
+				cat "$$LOG_FILE"; \
+			else \
+				echo "[logs] Container $(PROJECT_NAME)-$(CONTAINER_BRANCH) not found and no saved logs at $$LOG_FILE"; \
+			fi; \
+		}
 
 follow-agent:
-	container logs -f $(PROJECT_NAME)-$(CONTAINER_BRANCH)
+	@container logs -f $(PROJECT_NAME)-$(CONTAINER_BRANCH) 2>/dev/null \
+		|| { \
+			LOG_FILE="$(WORKTREES_DIR)/$(BRANCH)/.agent/agent.log"; \
+			if [ -f "$$LOG_FILE" ]; then \
+				echo "[logs] Container $(PROJECT_NAME)-$(CONTAINER_BRANCH) no longer running (agent finished)."; \
+				echo "[logs] Showing saved logs from $$LOG_FILE"; \
+				echo "---"; \
+				cat "$$LOG_FILE"; \
+			else \
+				echo "[logs] Container $(PROJECT_NAME)-$(CONTAINER_BRANCH) not found and no saved logs at $$LOG_FILE"; \
+			fi; \
+		}
+
+status-agent:
+	@STATUS_FILE="$(WORKTREES_DIR)/$(BRANCH)/.agent/status.json"; \
+	if [ -f "$$STATUS_FILE" ]; then \
+		if command -v jq >/dev/null 2>&1; then \
+			jq '.' "$$STATUS_FILE"; \
+		else \
+			cat "$$STATUS_FILE"; \
+		fi; \
+	else \
+		echo "[status] No status file found for branch '$(BRANCH)'."; \
+		echo "[status] Expected at: $$STATUS_FILE"; \
+	fi
+
+summary-agent:
+	@container logs $(PROJECT_NAME)-$(CONTAINER_BRANCH) 2>/dev/null \
+		| grep '^\[agent:' \
+		|| { \
+			LOG_FILE="$(WORKTREES_DIR)/$(BRANCH)/.agent/agent.log"; \
+			if [ -f "$$LOG_FILE" ]; then \
+				grep '^\[agent:' "$$LOG_FILE" || echo "(no structured events found)"; \
+			else \
+				echo "(no logs available for branch '$(BRANCH)')"; \
+			fi; \
+		}
 
 stop-agent:
 	@container stop $(PROJECT_NAME)-$(CONTAINER_BRANCH) 2>/dev/null \