diff --git a/src/content/docs/agent-platform/cloud-agents/handoff/cloud-to-cloud.mdx b/src/content/docs/agent-platform/cloud-agents/handoff/cloud-to-cloud.mdx index 157138b8..bd36b50b 100644 --- a/src/content/docs/agent-platform/cloud-agents/handoff/cloud-to-cloud.mdx +++ b/src/content/docs/agent-platform/cloud-agents/handoff/cloud-to-cloud.mdx @@ -19,7 +19,7 @@ Use this handoff direction when: When you send a follow-up to a run whose session has ended, the run continues with: * **The same conversation** - The follow-up is appended to the same conversation. From your perspective, the run is one continuous conversation across sessions. -* **The prior workspace state** - The prior session's repository changes (tracked and untracked) are restored before the agent answers your follow-up. For git-managed sessions, the new session continues on the same Git branch. +* **The prior workspace state** - The prior session's repository changes (tracked and untracked) are restored before the agent answers your follow-up. For git-managed sessions, the new session continues on the same Git branch. You can [customize which repositories and files get snapshotted](/agent-platform/cloud-agents/handoff/snapshots/) when running outside the bundled cloud agent image. * **Stable run identity** - The run's ID, task, creator, environment, schedule trigger, and integration source are preserved. Compute usage is recorded per session but rolls up to the same run. If applying any change fails, the agent reports what couldn't be applied and proceeds with what it could rather than starting over silently. diff --git a/src/content/docs/agent-platform/cloud-agents/handoff/snapshots.mdx b/src/content/docs/agent-platform/cloud-agents/handoff/snapshots.mdx new file mode 100644 index 00000000..30cbebae --- /dev/null +++ b/src/content/docs/agent-platform/cloud-agents/handoff/snapshots.mdx @@ -0,0 +1,233 @@ +--- +title: Customizing workspace snapshots +description: >- + Customize which repositories and files Warp snapshots at the end of a cloud + agent run, so handoff continues to work outside the bundled cloud agent + image. +sidebar: + label: "Snapshots" +--- + +Workspace snapshots are how [handoff](/agent-platform/cloud-agents/handoff/) carries repository changes and other workspace state across cloud agent runs. At the end of every cloud agent run, Warp asks a small declarations script which repositories and files to snapshot, then uploads the resulting git diffs and file contents so the next cloud agent run can apply them. + +Warp's bundled cloud agent image ships with a declarations script that snapshots every git repository under the agent's workspace, so most cloud agent runs need no configuration. This page is for the cases where you need to customize what gets snapshotted — for example, when running cloud agents in a custom Docker image, on a self-hosted [Direct backend](/agent-platform/cloud-agents/self-hosting/managed-direct/), or as an [unmanaged](/agent-platform/cloud-agents/self-hosting/unmanaged/) `oz agent run` in CI. + +## When to customize snapshots + +The default snapshotting behavior covers the most common case: snapshot every git repository the agent worked on, inside the bundled cloud agent image. Customize snapshots when you need different behavior: + +* **You're running outside the bundled image** — for example, a custom Docker base image, a self-hosted Direct backend, or unmanaged `oz agent run` invocations — where Warp's default `snapshot-declarations.sh` isn't on disk. +* **You want to snapshot repos or files outside the agent's workspace** — for example, a sibling repo the agent reads but doesn't `cd` into, or a log file the agent writes to `/tmp`. +* **You want dynamic snapshotting behavior** — because the declarations file is generated by a script that Warp runs, you can script any logic you want (filter by git status, dedupe against a baseline, emit a fixed list, etc.) rather than being limited to a static set of paths. + +If you don't customize snapshots in any of these cases, the run still completes — Warp just uploads nothing, and handoff into the next run starts without the prior session's uncommitted changes. + +## How snapshotting works + +There are two ways to tell Warp what to snapshot at the end of a cloud agent run: + +* **Script-driven** (default in the bundled image) - Warp invokes a declarations script (configured via the `OZ_SNAPSHOT_DECLARATIONS_SCRIPT` environment variable) that emits one JSON line per repository or file to snapshot. Use this when the set of repos isn't fixed ahead of time (for example, the agent may `git init` a new directory during its work) or when you want any other dynamic logic to decide what to snapshot. +* **Static** - You pre-populate the declarations file yourself (at the path given by `OZ_SNAPSHOT_DECLARATIONS_FILE`) and skip the script. Use this when the same set of repos and files should be snapshotted on every run. + +### Script-driven flow + +1. **Warp invokes your declarations script.** The script path comes from the `OZ_SNAPSHOT_DECLARATIONS_SCRIPT` environment variable. Warp runs it with the agent's workspace as the current directory. +2. **Warp passes a per-run output file.** Warp sets `OZ_SNAPSHOT_DECLARATIONS_FILE` to an absolute path the script must write to. Each run gets its own file so concurrent runs don't clobber each other. +3. **Warp reads the declarations file.** Warp parses the JSONL output (described below) and uploads the listed repositories and files. + +See [Write a custom declarations script](#write-a-custom-declarations-script) for the full pattern. + +### Static flow + +Set `OZ_SNAPSHOT_DECLARATIONS_FILE` to a path you've already populated and leave `OZ_SNAPSHOT_DECLARATIONS_SCRIPT` unset. Warp reads the file directly with no script invocation. See [Use a static declarations file](#use-a-static-declarations-file) for the full pattern. + +In both flows, snapshotting is automatically enabled for cloud agent runs when cloud conversations are enabled, and can be turned off per run with `--no-snapshot`. See [Disable snapshots](#disable-snapshots) below. + +## Environment variables + +* **`OZ_SNAPSHOT_DECLARATIONS_SCRIPT`** - Absolute path to the script Warp invokes at the end of each cloud agent run. The bundled cloud agent image sets this automatically. Set it yourself when running outside the bundled image. +* **`OZ_SNAPSHOT_DECLARATIONS_FILE`** - Absolute path to the JSONL file the script writes to (and Warp reads from). When `OZ_SNAPSHOT_DECLARATIONS_SCRIPT` is set, Warp picks a per-run path by default and exports it to the script. Set or override this variable yourself when you want Warp to read from a static, pre-populated declarations file instead of running a script, or whenever you want both Warp and the script to use a specific path you control. + +## Declarations file format + +The declarations file is UTF-8 JSONL — one JSON object per non-empty line: + +```json title="snapshot-declarations.jsonl" +{"version":1,"kind":"repo","path":"/workspace/my-repo"} +{"version":1,"kind":"file","path":"/tmp/agent-output.log"} +``` + +Each line has exactly these fields: + +* **`version`** - Always `1`. Reserved for future schema versions. +* **`kind`** - Either `"repo"` or `"file"`. +* **`path`** - A non-empty absolute path. + +A few rules to follow when writing declarations: + +* **Paths must be absolute.** Relative paths are rejected and logged as malformed. +* **Prefer `repo` over `file` for paths inside a repository.** Warp generates a git diff for each `repo` entry (tracked changes plus untracked, non-gitignored files), so individual `file` entries inside that repo are redundant and dropped before upload. Including a `repo` entry is also more extensible than enumerating files. +* **Repos are diffed, not copied wholesale.** Only changed files are uploaded for each `repo` entry, so listing a large repository is cheap when the agent's changes are small. +* **`file` entries are for paths outside any declared repo** — for example, logs the agent wrote to `/tmp` or scratch files in `$HOME`. + +Malformed lines (invalid JSON, missing fields, unknown `kind`, non-absolute path) are logged as warnings and skipped; they never abort the upload. + +:::caution +**Declared paths are uploaded to Warp.** `file` entries upload the file's contents verbatim, and `repo` entries upload the repo's git diff (tracked changes plus untracked, non-gitignored files). Before declaring logs, scratch files, or other agent outputs as `file` entries, make sure they don't contain secrets, credentials, API tokens, or other sensitive data. +::: + +## Write a custom declarations script + +A custom script writes one JSON line per repository or file it wants Warp to snapshot, then exits. The minimal pattern looks like this: + +```bash title="snapshot-declarations.sh" +#!/bin/bash +set -euo pipefail + +# Warp sets this to a per-run output path. Fail loudly if it's missing, +# so a misconfigured runner doesn't silently overwrite a shared file. +if [ -z "${OZ_SNAPSHOT_DECLARATIONS_FILE:-}" ]; then + echo "OZ_SNAPSHOT_DECLARATIONS_FILE must be set" >&2 + exit 1 +fi + +mkdir -p "$(dirname "$OZ_SNAPSHOT_DECLARATIONS_FILE")" + +# Find every git repo under the agent's workspace and emit a JSONL +# repo declaration for each one. Escape backslashes and double quotes in +# the path before writing so paths containing those characters don't +# produce malformed JSON (which Warp skips with a warning). +find "$PWD" -type d -name .git -prune -print 2>/dev/null \ + | while IFS= read -r git_dir; do + repo_root="$(cd "$(dirname "$git_dir")" && pwd)" + repo_json="$(printf '%s' "$repo_root" | sed 's/\\/\\\\/g; s/"/\\"/g')" + printf '{"version":1,"kind":"repo","path":"%s"}\n' "$repo_json" \ + >> "$OZ_SNAPSHOT_DECLARATIONS_FILE" + done +``` + +Then point Warp at it by exporting `OZ_SNAPSHOT_DECLARATIONS_SCRIPT` in the environment your cloud agent runs in: + +```bash +export OZ_SNAPSHOT_DECLARATIONS_SCRIPT=/path/to/snapshot-declarations.sh +``` + +For a managed [Direct backend](/agent-platform/cloud-agents/self-hosting/managed-direct/) worker, set it via the worker's `environment` config so it's present when the agent process starts. + +### The full bundled script + +For reference, this is the canonical declarations script Warp invokes inside the bundled cloud agent image. It's a richer version of the minimal example above: it honors a colon-separated `OZ_SNAPSHOT_SCAN_ROOTS` override for operators who need to scan repos outside the default workspace, uses `jq` for canonical JSON encoding (which handles `"` and `\` escaping for you), and dedupes against repo declarations already written in the same run so repeated invocations stay additive. + +:::note +The bundled script defaults `OZ_SNAPSHOT_JQ` to `/agent/tools/jq`, which only exists inside Warp's cloud agent image. If you adapt this script for a different image or for an unmanaged `oz agent run`, put `jq` on `PATH` or set `OZ_SNAPSHOT_JQ` to its absolute path. +::: + +```bash title="snapshot-declarations.sh" +#!/bin/bash +# Generates the declarations file consumed by the end-of-run snapshot upload pipeline. +# +# Scans one or more roots for `.git` directories and appends one JSON object per +# newly-discovered repository to the declarations file. +# +# Scan root selection (in order of precedence): +# - OZ_SNAPSHOT_SCAN_ROOTS: colon-separated list of absolute paths. Operators can use this +# to target repos outside the default workspace. +# - $PWD: the Rust driver sets this to the agent's workspace via `Command::current_dir` +# before invoking the script, so the default scan root is always the assigned workspace. +# +# Output file: +# - OZ_SNAPSHOT_DECLARATIONS_FILE must be set. The Rust driver sets this to a per-run path +# so concurrent runs don't clobber each other. +# - The file is created if missing, and appended to if it already exists. Existing lines are +# never rewritten, so operator-authored declarations are preserved. +# - Repeated invocations within a single run skip already-emitted repo declarations so the +# upload pipeline can stay additive without duplicating identical generated entries. + +set -euo pipefail + +SCAN_ROOTS_RAW="${OZ_SNAPSHOT_SCAN_ROOTS:-$PWD}" +IFS=':' read -r -a SCAN_ROOTS <<< "$SCAN_ROOTS_RAW" + +if [ -z "${OZ_SNAPSHOT_DECLARATIONS_FILE:-}" ]; then + echo "OZ_SNAPSHOT_DECLARATIONS_FILE must be set" >&2 + exit 1 +fi + +DECL_FILE="$OZ_SNAPSHOT_DECLARATIONS_FILE" +JQ="${OZ_SNAPSHOT_JQ:-/agent/tools/jq}" +mkdir -p "$(dirname "$DECL_FILE")" +touch "$DECL_FILE" + +# Dedup set. Seed from repo declarations already emitted by this script so a repeated +# invocation within the same run doesn't re-emit repos it already discovered earlier. +# Keep the full canonical JSON line as the key; this lets us preserve unrelated JSONL entries +# verbatim while ensuring identical generated declarations are emitted at most once. +SEEN_FILE="$(mktemp)" +trap 'rm -f "$SEEN_FILE"' EXIT +"$JQ" --raw-input --compact-output ' + fromjson? + | select( + type == "object" + and (. | keys_unsorted | sort == ["kind", "path", "version"]) + and .version == 1 + and .kind == "repo" + and (.path | type == "string") + ) + | { version: 1, kind: "repo", path: .path } +' "$DECL_FILE" > "$SEEN_FILE" + +repo_declaration_for_path() { + "$JQ" --compact-output --null-input \ + --arg path "$1" \ + '{ version: 1, kind: "repo", path: $path }' +} + +for root in "${SCAN_ROOTS[@]}"; do + [ -d "$root" ] || continue + while IFS= read -r git_dir; do + repo_root="$(cd "$(dirname "$git_dir")" && pwd)" + repo_declaration="$(repo_declaration_for_path "$repo_root")" + if grep -Fxq -- "$repo_declaration" "$SEEN_FILE"; then + continue + fi + printf '%s\n' "$repo_declaration" >> "$SEEN_FILE" + printf '%s\n' "$repo_declaration" >> "$DECL_FILE" + done < <(find "$root" -type d -name .git -prune -print 2>/dev/null) +done +``` + +## Use a static declarations file + +If the same set of repositories or files should be snapshotted on every run (for example, an unmanaged GitHub Actions job operating on a known checkout), you can skip the script entirely and pre-populate a JSONL file: + +```json title="/etc/oz/snapshot-declarations.jsonl" +{"version":1,"kind":"repo","path":"/workspace/my-repo"} +{"version":1,"kind":"repo","path":"/workspace/shared-libs"} +``` + +Then point Warp at the file directly: + +```bash +export OZ_SNAPSHOT_DECLARATIONS_FILE=/etc/oz/snapshot-declarations.jsonl +# Do not set OZ_SNAPSHOT_DECLARATIONS_SCRIPT in this mode. +``` + +Warp reads the file at end-of-run and uploads the listed repos and files. Because the file is the same on every run, this is best for environments where the workspace layout is fixed. + +## Disable snapshots + +To opt out of snapshotting for a single run, pass `--no-snapshot` to the CLI: + +```bash +oz agent run-cloud --prompt "Fix the failing test" --no-snapshot +``` + +Snapshotting is also skipped automatically when cloud conversations are disabled for the team. + +## Related pages + +* [Handoff from local to cloud](/agent-platform/cloud-agents/handoff/local-to-cloud/) - Promote a local conversation to a cloud run; the workspace snapshot is what carries your uncommitted changes across. +* [Handoff from cloud to cloud](/agent-platform/cloud-agents/handoff/cloud-to-cloud/) - Continue a finished cloud run; the prior session's workspace snapshot is what gets restored. +* [Self-hosting overview](/agent-platform/cloud-agents/self-hosting/) - Architecture decision guide for self-hosted workers, where customizing snapshots is most often needed. +* [Unmanaged architecture](/agent-platform/cloud-agents/self-hosting/unmanaged/) - Run `oz agent run` in CI, Kubernetes, or your dev environment outside the bundled image. +* [Oz CLI](/reference/cli/) - Full reference for `oz agent run` and `oz agent run-cloud`. diff --git a/src/content/docs/agent-platform/cloud-agents/self-hosting/index.mdx b/src/content/docs/agent-platform/cloud-agents/self-hosting/index.mdx index f46d4783..cbb98904 100644 --- a/src/content/docs/agent-platform/cloud-agents/self-hosting/index.mdx +++ b/src/content/docs/agent-platform/cloud-agents/self-hosting/index.mdx @@ -216,3 +216,4 @@ For infrastructure-level observability, the `oz-agent-worker` daemon can export * [Troubleshooting](/agent-platform/cloud-agents/self-hosting/troubleshooting/) — Worker won't start, tasks not picked up, and other common issues. * [Deployment patterns](/agent-platform/cloud-agents/deployment-patterns/) — How self-hosting compares to CLI-only and Warp-hosted deployment. * [Environments](/agent-platform/cloud-agents/environments/) — Define the runtime context for agent tasks. +* [Customizing workspace snapshots](/agent-platform/cloud-agents/handoff/snapshots/) — Configure end-of-run snapshots so handoff works when running outside the bundled cloud agent image. diff --git a/src/content/docs/agent-platform/cloud-agents/self-hosting/unmanaged.mdx b/src/content/docs/agent-platform/cloud-agents/self-hosting/unmanaged.mdx index a912561f..84d725d2 100644 --- a/src/content/docs/agent-platform/cloud-agents/self-hosting/unmanaged.mdx +++ b/src/content/docs/agent-platform/cloud-agents/self-hosting/unmanaged.mdx @@ -126,6 +126,8 @@ Unmanaged agents are tracked on Warp's backend. Each run creates a persistent se Unmanaged sessions benefit from the same shared configuration as other cloud agent runs — [MCP servers](/agent-platform/cloud-agents/mcp/), [secrets](/agent-platform/cloud-agents/secrets/), Warp Drive context, and saved prompts all apply. +Unmanaged runs don't ship with the bundled declarations script, so end-of-run workspace snapshots are a no-op by default. To enable [handoff](/agent-platform/cloud-agents/handoff/) into a follow-up run, see [Customizing workspace snapshots](/agent-platform/cloud-agents/handoff/snapshots/). + --- ## Related pages diff --git a/src/sidebar.ts b/src/sidebar.ts index f64c9e80..fc60bc8e 100644 --- a/src/sidebar.ts +++ b/src/sidebar.ts @@ -371,6 +371,7 @@ export const sidebarTopics: StarlightSidebarTopicsUserConfig = [ { slug: 'agent-platform/cloud-agents/handoff', label: 'Overview' }, { slug: 'agent-platform/cloud-agents/handoff/local-to-cloud', label: 'Local to cloud' }, { slug: 'agent-platform/cloud-agents/handoff/cloud-to-cloud', label: 'Cloud to cloud' }, + { slug: 'agent-platform/cloud-agents/handoff/snapshots', label: 'Snapshots' }, ], }, 'agent-platform/cloud-agents/secrets',