feat(agentbox.yaml): idempotent tasks + replacement engine (render/carry)#80
Closed
madarco wants to merge 12 commits into
Closed
feat(agentbox.yaml): idempotent tasks + replacement engine (render/carry)#80madarco wants to merge 12 commits into
madarco wants to merge 12 commits into
Conversation
…I + carry)
- tasks: idempotent: true (command-hash marker) | { check } (probe)
- shared pure replacement engine in @agentbox/core (env placeholders + rules)
- agentbox-ctl render CLI (declarative sed alternative)
- carry: replaceEnvs / replace / rules (host-side, file-only)
- top-level replacements: reusable named rule-sets
- schema + drift fixtures + unit tests
- agentbox-yaml.mdx: idempotent field, replacements section, carry replace fields, placeholder table - in-box-supervisor.md + features.md: implementation notes - agentbox-setup skill: teach idempotent:/render/replaceEnvs over manual markers + sed - agentbox-info skill: one-line pointer to the new declarative fields
The \.optima\.localhost (leading-dot) regex wouldn't match a bare
optima.localhost; use optima\.localhost -> {{AGENTBOX_BOX_HOST}}.
E2E on a real box surfaced EACCES writing /var/lib/agentbox (root-owned, daemon runs as vscode), so idempotent: true silently re-ran every boot. - supervisor: resolve a writable stateDir at init, falling back to <logDir>/state (always daemon-writable, on rootfs/checkpointed, off /workspace) when the configured dir isn't creatable — works on every provider without an image bake - Dockerfile.box: mkdir+chown /var/lib/agentbox to vscode so docker uses the clean default path - test: fallback path covered
- bake /var/lib/agentbox on hetzner/vercel/e2b base images (was docker-only), so idempotent markers use the clean path on every provider, not the fallback - checkpoint-cleanup: exclude /var/log/agentbox/state from truncation so the marker fallback survives a checkpoint - core: single deriveBoxHost() shared by both placeholder-context builders - carry-render: dedup the wantsRender predicate; build the log suffix with join - carry.ts: fold parseRulesRefs/parseExclude into one parseStringList - carry-resolve: drop redundant replaceFields guard - carry-gate: read agentbox.yaml once, parse carry + replacements from one text
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 5b30be4. Configure here.
| const idem = this.spec.idempotent; | ||
| if (!idem) return null; | ||
| if (idem.kind === 'check') { | ||
| return (await this.runCheck(idem.command, cwd)) ? 'check passed' : null; |
There was a problem hiding this comment.
Check probes ignore placeholders
Medium Severity
idempotent: { check: … } runs the probe string verbatim via bash -c, but the PR’s docs and skills show checks like grep -q '{{AGENTBOX_BOX_HOST}}' …. Those {{AGENTBOX_*}} tokens are only expanded by agentbox-ctl render / carry rendering, not by the supervisor, so the probe searches for literal braces and never exits 0 after a successful render—warm boots keep re-running the task instead of skipping.
Reviewed by Cursor Bugbot for commit 5b30be4. Configure here.
Owner
Author
There was a problem hiding this comment.
Good catch — the supervisor runs the check verbatim via bash -c and never expands {{…}} (render-only). Fixed in 1749f78: dropped the unnecessary check from the naturally-idempotent render example, and documented that check probes are plain shell (use $AGENTBOX_BOX_NAME). The optima e2e config was unaffected (its checks use plain psql).
Cursor Bugbot: example checks used grep -q '{{AGENTBOX_BOX_HOST}}', but the
supervisor runs the probe verbatim via bash -c and never expands {{...}}
(render-only), so it matched literal braces and re-ran every boot. Drop the
unnecessary check from the naturally-idempotent render example, and document
that check probes use shell vars ($AGENTBOX_BOX_NAME).
The stateDir-fallback test used /proc/nope as the unwritable path; mkdir under /proc behaves differently on Linux (slow) than macOS and timed out in CI. Use a regular file as the parent dir (deterministic ENOTDIR, instant, cross-platform) and add a 20s testTimeout cushion for the two-cycle idempotent tests.
- services: image: form (ports/env/args/container_name) synthesizes the
docker start-or-run shell; command|image mutually exclusive; reused by name
- render: {{AGENTBOX_AUTO_SECRET}} (fresh per render) and :name (persisted at
<stateDir>/secrets/<name>, reused) — replaces openssl rand in env tasks
- shared resolveWritableStateDir (state-dir.ts) backs markers + secrets
- schema (oneOf command/image + dependentRequired) + drift fixtures
- unit tests (config image synth/xor, secret per-render vs persisted)
- docs + agentbox-setup skill
So a replacement rule can emit an {{AGENTBOX_AUTO_SECRET}} token that the
secret pass then resolves in a single render (e.g. 'your-secret-here=>{{...}}').
Verified e2e: optima's env task renders env.example with a box-host rule + a
persisted secret in one pass.
Clearer, less jargon (AgentBox is unreleased, so a clean rename — no alias). Renames the YAML key, schema property, TS types (RunOnceSpec/parseRunOnce/ TaskSpec.runOnce), supervisor skip logic + log lines, docs, skills, and the services-and-tasks guide (now recommends run_once over hand-rolled markers).
The marker write was fire-and-forget, so a task reached 'done' before its
marker hit disk — a CI race (slower fs) and a latent durability gap (a crash
in between would lose the marker and re-run next boot). Await the write, then
setState('done').
Per review, group the container config under image: instead of flat sibling
keys. image: is now either a bare ref string (image: redis:7) or a mapping
{ name, ports, env, args, container_name }. Container env moves to image.env
(top-level env on an image service is rejected). Schema/tests/docs/skill +
optima updated.
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Streamlines the common
agentbox.yamlpatterns so projects stop hand-rolling marker guards andsedcommands. Three declarative primitives:1.
idempotent:tasksTasks re-run on every box start, so they must be idempotent. Now declarative:
idempotent: true— supervisor stores a marker keyed by a hash of the resolved command (/var/lib/agentbox/tasks/<name>, on rootfs, captured by checkpoints, off/workspace). Editing the command re-runs it.idempotent: { check: <cmd> }— probe-first; exit 0 = skip, no marker. The right guard for state outside the checkpoint (e.g. a containerized DB), where a filesystem marker would desync.run-task --forcebypasses both.2. Replacement engine (
render+ carry)A declarative
sedalternative, pure engine in@agentbox/core(re-exported by@agentbox/ctl):agentbox-ctl render <src>— in-box:--env(whitelist{{AGENTBOX_*}}placeholders),--rules,--rule,--rule-regex,--out/--in-place.replaceEnvs/replace/rules— host-side rendering of carried files to a temp before copy (file-only; original host file untouched).replacements:— reusable named rule-sets referenced by both.Placeholders are a fixed whitelist (predictable; secrets never substitutable); arbitrary substitutions go through explicit rules.
Verified end-to-end on a real project (Evinto/optima)
Rewrote optima's
agentbox.yamlto use all three (install→idempotent: true,env→render --rules,restore/seed→idempotent: { check }) and ran a full box:.envrendered withBETTER_AUTH_URL/NEXT_PUBLIC_APP_URLpinned to the box host + generated secret ✓installmarker written; stop/start →idempotent: marker matches — skip✓seed→idempotent: check passed — skip;restorere-runs (no schema) ✓devreach ready ✓The e2e surfaced (and this PR fixes) a real bug: the marker dir
/var/lib/agentboxwas root-owned but the daemon runs asvscode(EACCES → silent re-run). Fixed by baking+chowning/var/lib/agentboxon all providers (docker/hetzner/vercel/e2b) plus a writable<logDir>/statefallback (excluded from checkpoint-cleanup truncation).Docs / skills
agentbox-yaml.mdx,in-box-supervisor.md,features.md, JSON schema + drift fixtures, and theagentbox-setupskill (now teaches the declarative fields over manual markers/sed) all updated in-PR.Tests
New unit coverage for the engine, idempotent (marker/check/force/fallback), carry parse/resolve/render. Full suites green (core 20, ctl 221, sandbox-core 34, cli 489).
Note
Medium Risk
Changes supervisor task re-run behavior and host-side carry rendering at box create; misconfigured idempotent checks could skip needed seeds or re-run heavy installs, but scope is confined to box boot and file copy paths.
Overview
Adds declarative idempotent tasks and a shared text-replacement engine so
agentbox.yamlcan skip hand-rolled marker files andsed.Tasks:
idempotent: truestores a command-hash marker under/var/lib/agentbox/tasks/<name>(rootfs, checkpointed, not under/workspace);idempotent: { check: … }probes first and skips on exit 0 with no marker (for state outside checkpoints, e.g. containerized DB seeds). The supervisor implements the gate inTaskRunner;run-task --forcebypasses it. Images now create/chown/var/lib/agentboxon all providers, with a writable<logDir>/statefallback and checkpoint cleanup that preserves that fallback.Replacements: New
@agentbox/corereplace.ts(whitelist{{AGENTBOX_*}}+ ordered rules), top-levelreplacements:in config/schema, in-boxagentbox-ctl render, and carry file optionsreplaceEnvs/replace/rulesrendered host-side viarenderCarryEntriesbefore docker/cloud copy. The carry gate/resolver loadsreplacementsand expands named rule refs.Docs/skills (
agentbox-setup,agentbox-yaml.mdx, features, supervisor) and tests/schema drift are updated accordingly. Removes an obsolete Claude-memory seed plan; adds an unrelated future user-defined shims design doc only.Reviewed by Cursor Bugbot for commit 5b30be4. Configure here.