fix(orchestration): isolate per-task config-resolution failures from the suite#25
Open
bai-uipath wants to merge 1 commit into
Open
fix(orchestration): isolate per-task config-resolution failures from the suite#25bai-uipath wants to merge 1 commit into
bai-uipath wants to merge 1 commit into
Conversation
…the suite resolve_all_tasks wrapped only load/expand in per-task error isolation (#219); layers 1-5 override resolution stayed unguarded, so one task whose own YAML is incompatible with the run — e.g. Claude-only `sdk_options` surviving a `--type codex` override, which `CodexAgentConfig` forbids (`extra="forbid"`) — aborted the whole coder-eval invocation with no run.json. That gap predates the early-stop feature and is what collapsed the codex weekly. Buffer each file's fan-out and demote a resolution failure to `skipped`, but re-raise when every attempted task fails (a global --type/-D/repeats-cap misconfiguration that trips every task) so the CLI still aborts cleanly instead of producing an empty run. Early-stop arming errors continue to propagate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bai-uipath
requested review from
akshaylive,
tmatup and
uipreliga
as code owners
|
Claude finished @bai-uipath's task in 2m 42s —— View job Code Review in Progress
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Make one incompatible task YAML skip instead of aborting the whole
coder-evalrun, by extendingresolve_all_tasks' per-task error isolation from the load phase to config resolution (layers 1–5).A resolution failure now demotes just that task file to
skipped(surfaced in the run summary andrun.json) — but if every task that reaches resolution fails, the cause is a global invocation error (a bad--type/-Dvalue, repeats over the cap) rather than a per-task incompatibility, so it is re-raised and the CLI aborts cleanly. Early-stop arming errors always propagate.Why
resolve_all_tasksalready isolated task YAMLs that fail to load (#219), but layers 1–5 override resolution stayed unguarded. So a single task whose own YAML is incompatible with the run — the concrete case being Claude-only agent fields (sdk_options) surviving a--type codexoverride, whichCodexAgentConfigforbids (extra="forbid") — aborted the entire invocation with norun.jsonwritten.That is what collapsed the scheduled codex weekly: a handful of Maestro tasks carry
sdk_options, and codex is the first harness to run them under a non-Claude--type. The gap is ~4 months old (it dates to the original resolution loop), not new — the load-isolation PR simply never covered this layer.The "re-raise only when every attempted task fails" rule keeps the existing fail-fast behaviour for genuinely global misconfigurations (invalid
--type, bad-Dvalues, repeats cap, early-stop arming) while letting a suite shrug off the odd incompatible task and run the rest.