You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Part of #244 (full-codebase eval testbed). Depends on #257 (heuristic responder).
Why
#257 ships deterministic responder heuristics and stops the conversation with a distinct reason
when it meets a question it cannot classify. #244 asks for the general case:
For more complex questions, we may need some kind of "question answering agent", designed to
quickly generate a comprehensible response, always tending towards "choose whatever is
recommended/simplest/easiest".
This ticket fills that gap. It is deliberately separate from #257 so the loop is proven before a
second model is introduced into the attribution picture.
Scope
Dispatch a small answering agent through the same harness as the run, prompted to choose the
recommended, simplest, or easiest path.
Attribute it separately in conversation.json: a turn produced by the answering agent must be
distinguishable from a heuristic turn, and the responder model must be recorded alongside the
agent and judge models already tracked in conditions.json.
Files
src/cli/run/conversation.rs
schema/conversation.schema.json
src/core/types.rs (responder model provenance)
Acceptance criteria
A free-form question the heuristics reject is answered by the responder agent and the run
continues.
Responder-produced turns are distinguishable from heuristic turns in conversation.json.
The responder model is recorded as run provenance.
Mode B parity. Acceptance must hold for both Mode::NewSkill and Mode::Revision
(src/core/types.rs:225), and the snapshot/promote path (src/workspace/snapshot.rs, src/workspace/promote.rs) must keep working against codebase-backed iterations.
Provenance reaches the report. Source + resolved SHA must land in conditions.json, each run.json, benchmark.json, and the BASELINE.md built by promote.rs:231 — not just in the
workspace.
Part of #244 (full-codebase eval testbed). Depends on #257 (heuristic responder).
Why
#257 ships deterministic responder heuristics and stops the conversation with a distinct reason
when it meets a question it cannot classify. #244 asks for the general case:
This ticket fills that gap. It is deliberately separate from #257 so the loop is proven before a
second model is introduced into the attribution picture.
Scope
recommended, simplest, or easiest path.
stays the default, both for cost and for reproducibility.
conversation.json: a turn produced by the answering agent must bedistinguishable from a heuristic turn, and the responder model must be recorded alongside the
agent and judge models already tracked in
conditions.json.Files
src/cli/run/conversation.rsschema/conversation.schema.jsonsrc/core/types.rs(responder model provenance)Acceptance criteria
continues.
conversation.json.Cross-cutting requirements
Mode::NewSkillandMode::Revision(
src/core/types.rs:225), and the snapshot/promote path (src/workspace/snapshot.rs,src/workspace/promote.rs) must keep working against codebase-backed iterations.conditions.json, eachrun.json,benchmark.json, and theBASELINE.mdbuilt bypromote.rs:231— not just in theworkspace.
Verification
Implementation requires the
slow-powers:working-with-tddskill.Parent: #244