Skip to content

feat(workflows): configurable task workflows driven by the daemon - #21

Open
lapa2112 wants to merge 8 commits into
ephor:mainfrom
lapa2112:feat/workflows
Open

feat(workflows): configurable task workflows driven by the daemon#21
lapa2112 wants to merge 8 commits into
ephor:mainfrom
lapa2112:feat/workflows

Conversation

@lapa2112

@lapa2112 lapa2112 commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

What this adds

Today "process" in Warpforge is a single Orchestrator toggle: it appends a hardcoded prompt and hands an LLM three MCP tools. There are no roles, no review, no iteration limits, and nothing to configure — the process is whatever the model decides to do.

This branch makes the process declarative and deterministic. A workflow is a YAML file in the project's .warpforge/workflows/; the daemon — not an LLM — drives the task through it:

plan? → implement → review ⇄ fix → needs review

The workflow decides which agent and model runs each stage, what each stage is told, what the reviewers see, how many review rounds are allowed, and what happens when they run out. The existing Orchestrator chat is untouched; a workflow is a separate engine, and the two are mutually exclusive in the dialog.

How it works

  • Templates. .warpforge/workflows/*.yaml, plus two built-ins shipped in the binary ("Implement + review loop", "Plan + implement + review loop"). A project file shadows a built-in with the same id; either built-in can be copied into the project in one click and edited. Invalid files stay listed with their parse error instead of silently disappearing.
  • Engine. Every stage is a normal child task with its own agent session, so it shows up on the board and its chat can be opened. Reviewers run in parallel, may each be a different agent/model, and return a structured verdict; a repeat round continues in the same reviewer's session so it verifies its own findings rather than re-reviewing from scratch (review.reask: same_session | fresh, with an automatic fresh-session fallback when a session is gone).
  • The parent task has no agent session of its own. Its transcript is a timeline of stages, each card listing the agents that ran it — click one to open that agent's session.
  • It stops for you. A stage can ask a question mid-run; running out of review rounds asks whether to grant more, finish as is, or stop. Both surface in "Needs you" with a toast, and the parent's composer becomes the answer box.
  • Pause/resume between stages, with an optional note that reaches the next stage as guidance.
  • Restart-safe. Run state is snapshotted on every transition; barrier states resume as they were, and a run caught mid-stage parks at its last safe point (the re-run is told its predecessor was interrupted, so it inspects the partial diff).
  • Never commits. A finished run lands in Needs review for a human to inspect.

Also here: the workspace config gains a preferred home at .warpforge/workspace.yaml next to .warpforge/workflows/. Existing root-level configs keep working; only newly generated ones move.

Review notes

Worth a close look, in rough order:

  1. src/daemon/workflow.rs — the run state machine, prompt builders, and the two agent protocols (verdict / need_user_input). The prompts are the load-bearing part of whether this feels useful or annoying.
  2. The workflow block at the end of src/daemon/actor.rs — actor glue. Methods take the run out of workflow_runs and must re-insert on every exit path; a dropped run means a parent stuck in "running".
  3. src/workflow_config.rs — YAML schema and validation, including which knobs warn when they're inert.
  4. desktop/src/hooks/useWorkflowSend.ts — the single place that routes composer input for a session-less parent. Anything that can point a composer at a task must go through it.

The branch went through a review pass across four dimensions (engine, config/protocol, desktop UX, prompt/product quality) plus a visual check of every UI state; a259b08 is that pass. The findings it fixed that are worth knowing about:

  • a request_changes verdict with no parseable findings used to finish the run as a success — a silent rubber-stamp when a reviewer wrote its findings in prose
  • one reviewer that couldn't produce a parseable verdict used to fail the whole pipeline; it now abstains
  • AcpHandle::prompt is not a liveness check (its channel outlives the child), so the same-session and dead-session fallbacks were unreachable exactly when needed
  • a stage whose session failed to start hung the parent forever
  • a restart during a review round burned it ("round 3/2")
  • the protocol JSON leaked into the user-facing timeline

Testing

  • Rust: 176 tests. Unit coverage for config validation, prompt building, verdict/marker parsing, review merging, and diff formatting; plus end-to-end pipelines against a scripted mock ACP agent (tests/fixtures/mock-acp-workflow.mjs) covering the reject → fix → approve loop, question/reply, the limit decision, pause/resume, restart recovery, same-session reuse, dead-session fallback, and session cleanup (verified with pgrep — no agent process outlives a pipeline).
  • Desktop: 345 tests, including the picker, composer routing per barrier, the controls, and the attention rule.
  • cargo fmt, cargo clippy --all-targets -D warnings, bun run lint, bun run typecheck all clean.

Not verified: a real end-to-end run against live Claude Code / Codex sessions. Everything agent-facing is exercised through the mock, so the prompts themselves are the untested surface.

Deliberate limitations

Scoped out of this first iteration on purpose: no arbitrary stage DAG, no gate expressions, no finding ledger or dedup across rounds, no deterministic build/lint/test gates (agents run their own checks), no separate tester or final-acceptor role, no global ~/.warpforge/workflows/, no auto-commit. Pause is soft (at stage boundaries) rather than interrupting a turn mid-flight.

Known rough edges left as follow-ups: findings aren't deduplicated between reviewers or rounds; the diff is spliced into prompts without delimiters (mitigated by key-scoped protocol parsing); the workflow control bar lives inside the chat column, so a collapsed chat hides it; finished runs are never pruned from workflow_runs.

lapa2112 added 7 commits July 29, 2026 15:54
…config home

Groundwork for configurable task workflows (PR 1 of 3):

- .warpforge/workflows/*.yaml parsing and validation: fixed pipeline shape
  (plan? -> implement -> review <-> fix), per-stage agent/model/prompt
  overrides, 1..4 reviewers, review context selection, round limits with a
  hard cap, {{placeholder}} template rendering with per-stage variable sets
- two built-in templates (review-loop, plan-review-loop) shipped in the
  binary; project files override built-ins by id
- workflow.list / workflow.eject RPCs for the New Task picker
- .warpforge/workspace.yaml becomes the preferred workspace config location;
  legacy root-level names keep working, generated configs land in .warpforge/
PR 2 of 3: task.create with a workflow id
now drives plan? -> implement -> review <-> fix as a state machine inside the
daemon actor.

- engine: stages spawn as child tasks with rendered prompts; parallel
  reviewers return fenced-JSON verdicts (one re-ask on garbage, then fail);
  findings merge with unanimous-approve semantics; low-severity findings are
  deferred to the summary instead of the fixer
- decision points: stages can suspend the run with a need_user_input marker
  (workflow.reply routes the answer to the asking session, or re-runs the
  stage with the Q/A as guidance when that session is gone); hitting the
  review limit asks extend/finish/stop (workflow.decide) with optional
  guidance for the next fix round
- soft pause at stage barriers (workflow.pause/resume, resume notes become
  stage guidance); parent Stop/archive/delete stops the whole pipeline
- persistence: workflow_runs table snapshots the run on every transition;
  after a restart barrier states continue as-is and mid-stage runs park as
  Paused at their last barrier (resume re-runs the stage)
- the parent task has no agent session: its transcript is a synthetic
  timeline, TaskInfo gains workflow_run (stage/round/verdict/waiting), and
  orchestration_graph finally gets a producer for the board accordion
- protocol: workflow param on task.create; workflow.pause/resume/reply/decide;
  WorkflowRunInfo/WorkflowWaiting wire types; OrchNodeKind::Fix
- tests: engine unit tests plus end-to-end mock-agent pipelines (reject->
  fix->approve loop, question/reply, limit decision, pause/resume, restart
  recovery) via a scripted mock ACP agent fixture
PR 3 of 3.

New Task dialog:
- Workflow pill-dropdown listing project templates plus built-ins; invalid
  templates stay listed with their parse error instead of vanishing, and a
  built-in can be copied into .warpforge/workflows/ for editing
- picking a workflow passes it to task.create, relabels the agent chip as the
  lead agent, and is mutually exclusive with the orchestrator chat
- a daemon-side validation failure keeps the dialog open so the prompt survives

Task detail:
- WorkflowControls above the composer: stage, round, latest verdict, plus
  Pause/Resume and the extend/finish/stop choice when review rounds run out
- the composer routes messages to workflow.reply / resume / decide depending on
  what the pipeline is waiting for, and is disabled (with an explanatory
  placeholder) while it runs unattended; the stop button still cancels the run

Elsewhere:
- a pipeline waiting on a question or a review limit joins the "Needs you"
  rail just under a pending permission; a user-initiated pause stays out
- board cards get a stage badge that turns amber when the pipeline needs you

Tests cover the picker (selection, invalid entries, eject, mutual exclusion),
composer routing per waiting kind, the controls, and the attention rule.
Repeat review rounds (after a fix) now follow up in the previous reviewer's
live session by default instead of spawning a fresh one. The follow-up
carries only the delta — fixer summary, last round's findings, fresh diff —
plus explicit anti-anchoring instructions: verify every prior finding is
actually resolved (not on the fixer's word), reopen what isn't, and re-check
the changes for regressions including code found fine last round.

- review.reask: same_session | fresh in workflow YAML (default same_session);
  fresh always staffs new reviewer sessions, whose prompts get an engine-owned
  verification section with the previous round's findings (appended after
  custom prompts too, no template changes needed)
- dead reviewer session (daemon restart, agent death) falls back to the fresh
  path automatically; prior_review_children on the run tracks round staffing
- finalize now sweeps every stage session the run ever spawned, not just the
  active ones — completed reviewers/implementers no longer keep running in
  the background until a daemon restart (verified by a pgrep-based test)

Tests: reask parsing; rereview/verification prompt builders; integration:
same-session reuse (both review nodes share one task, 3 children total),
reask: fresh spawns new reviewers, dead-session fallback via a reject-then-
exit mock behavior.
Findings from a full pre-PR review of the branch (engine, config/protocol,
desktop UX, and prompt/product review), verified and fixed.

Engine correctness:
- a request_changes verdict with no parseable findings no longer finishes the
  run as a success; the reviewer's own prose is salvaged as a finding so the
  repair stage still has something to act on
- a reviewer that cannot produce a parseable verdict now abstains like a dead
  one instead of failing the whole pipeline
- a stage whose session never starts fails the run instead of hanging the
  parent in 'running' forever (start_session reports failure by blocking the
  child, so no TurnEnded ever arrives)
- prompt() is not a liveness check (its channel outlives the child), so
  same-session re-review, verdict re-asks and question replies now check
  AcpHandle::is_alive and take their fresh-session fallbacks when it is dead
- a restart during a review round no longer burns it ('round 3/2' and an
  immediate limit decision); the re-run also gets told its predecessor was
  interrupted so it inspects the partial diff
- teardown waits are bounded, so an unkillable agent cannot freeze the actor
- teardown trouble is reported in the timeline, not as the RPC's error
- cancelling a finished pipeline keeps its terminal status
- refuse to start a workflow with no store (every stage would read as empty)
- an unreadable persisted run is dropped and its parent marked, instead of
  re-failing every start with no pipeline controls

Agent contract:
- the verdict protocol example no longer contradicts its own rule, states that
  findings must be an array with descriptions, and says low findings are never
  repaired; scope is calibrated to correctness/regressions and read-only
  verification is encouraged
- the question protocol says when to stop, not just how
- severity aliases cover the vocabulary agents actually use (nitpick,
  suggestion, style, trivial, …) so opinions stop defaulting to Medium
- protocol blocks are matched by required key, and stripped from the timeline
  so the parent shows prose instead of wire format

Config:
- typed parse from the source text, so errors name the key with line/column
- {{plan}} rejected without a plan stage; inert review.context / focus next to
  custom reviewer prompts now warn; plan: false disables the stage
- every persisted collection carries serde(default) so an upgrade cannot
  orphan in-flight runs; load warnings surface in the run's first event
- config_overrides and include_runtime_context reach stage sessions (the
  dialog's reasoning-effort pick was silently dropped)

Desktop:
- workflow routing extracted to useWorkflowSend and used by the Mission
  Control tile too, which previously posted to session.prompt and errored
- the composer stays disabled after a run finishes, explaining where to
  continue, instead of failing with a raw daemon error
- Pause is hidden while a question is pending (the daemon rejects it) and a
  queued pause shows as 'Pausing…' instead of an idle button
- stage rows show the daemon's label, so four review rounds are no longer four
  rows reading 'review'; keys no longer collide when a stage re-runs
- pipelines appear in rail/tile previews, blocking on the user raises a toast,
  and tile stage cards are clickable
- semantic ok/warn tones replace dead dark: variants; border-current opacity
  (which compiles to nothing) replaced; long names/questions truncate with
  tooltips; limit buttons no longer claim to use typed guidance

Docs: spec reconciled with the implementation (state names, role tags, the
child-count cap that was never built, round semantics), and the four workflow
changesets consolidated into one accurate release note plus the config move.
…rounds

Follow-ups from the pre-PR review discussion.

- default review rounds raised from 2 to 3: a fix runs between rounds, so the
  old default bought only one repair attempt and a single arguable finding
  could consume it
- a stage hands the next stage its CLOSING message — the text streamed after
  its last tool call or file edit — instead of every AgentText chunk of the
  turn. That is what the prompts already ask for ("your final message should
  summarize what you did"), it drops kilobytes of tool narration from reviewer
  prompts, and it hardens parsing: a JSON block quoted mid-turn (while reading
  a config file, say) can no longer be mistaken for the protocol payload. The
  full turn stays as a fallback for an agent that emits its block and then
  makes one last tool call.
- findings carry optional `line` and `snippet` anchors. The line is parsed from
  a number, a string, or a range; the snippet is clipped to one short line and
  survives drift because the fixer can search for it. Both are rendered into
  the repair prompt as `file:line` plus an "at:" excerpt, and the verdict
  protocol explains that hunk headers give the post-change line number.

Tests: anchor parsing (number/string/range, `code` alias, clipping, line 0),
protocol-payload detection, and an end-to-end run asserting the reviewer sees
the implementer's closing message and never its narration.
Starts docs/adr/ with the decisions behind the workflow feature: what was
chosen, what was rejected, and — the part that actually prevents regressions —
the invariants that are invisible in the code. Every entry in the Invariants
section is a bug that was made once and found in review, e.g. that
AcpHandle::prompt succeeds for a dead agent (its channel outlives the child) so
it cannot be used as a liveness test, and that a request_changes verdict with
no parseable findings must never finish a run as a success.

Deliberately excludes anything readable from the code: stage lists, field
names, and module structure drift, and a doc that restates them misleads.

Also wires it up where it will actually be read: CLAUDE.md now points at
docs/adr/ from its design-decisions section, lists the workflow feature under
what works, and records the workspace config's new .warpforge/ location, which
it still described as .workspace.yaml.
@lapa2112
lapa2112 requested a review from ephor July 30, 2026 09:13
@lapa2112 lapa2112 self-assigned this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants