fix(compose): bind output volume to host path and pre-create it - #312
fix(compose): bind output volume to host path and pre-create it#312cdoron wants to merge 1 commit into
Conversation
services.yaml's `output` volume previously used an anonymous managed
volume, leaving results with no host-visible path. Bind it to
./output/{benchmark}/{task-id}/ via driver_opts (compose/RULES.md rule
18) using ${PWD}, which resolves to the CLI's invocation directory
regardless of include: nesting.
That bind form doesn't auto-create the target directory the way a
short-syntax host bind would, so the CLI now pre-creates it as the
invoking user before running compose — otherwise Docker creates it
root-owned on first mount, which the agent's uid-1002 process can't
write into.
Signed-off-by: Doron Chen <cdoron@il.ibm.com>
elronbandel
left a comment
There was a problem hiding this comment.
Verified the core mechanic: the bind-mount form used here (`driver_opts: {type: none, o: bind, device: ...}`) does NOT auto-create the target directory — mounting fails outright unless it already exists — so the `create_dir_all` this PR adds in `cli/src/run.rs` before `docker compose up` is a real, correctly-placed fix (covers both `--local` and registry paths, matches the compose file's `EVAL_TASK_ID` default).
But the volume is named `output`, scoped to the compose project (stable across invocations from the same directory) — not per task-id. Tested directly: running task 0 then task 1 from the same directory, with no `docker compose down -v` between, hits Docker's "volume exists but doesn't match configuration — recreate (data will be lost)?" prompt, because `device:` changed but the named volume object didn't. That's an interactive stdin prompt — in a non-interactive run it just hangs (confirmed: 2min timeout, no output). This breaks compose/RULES.md rule 18 ("running multiple tasks MUST accumulate results without overwriting") for exactly the case it's supposed to guarantee: two different task-ids run back to back from the same directory.
Needs either a `docker compose down -v` (or equivalent) before each run, or a per-task-id volume/project name, so a second task-id doesn't hang or collide with the first. Please confirm this was tested with more than one task-id from the same working directory.
services.yaml's
outputvolume previously used an anonymous managedvolume, leaving results with no host-visible path. Bind it to
./output/{benchmark}/{task-id}/ via driver_opts (compose/RULES.md rule
18) using ${PWD}, which resolves to the CLI's invocation directory
regardless of include: nesting.
That bind form doesn't auto-create the target directory the way a
short-syntax host bind would, so the CLI now pre-creates it as the
invoking user before running compose — otherwise Docker creates it
root-owned on first mount, which the agent's uid-1002 process can't
write into.
Signed-off-by: Doron Chen cdoron@il.ibm.com