Skip to content

feat(ts): scenario revision flow — refine with feedback (AC-441)#542

Merged
jayscambler merged 2 commits intomainfrom
ac-441/ts-scenario-revision
Mar 27, 2026
Merged

feat(ts): scenario revision flow — refine with feedback (AC-441)#542
jayscambler merged 2 commits intomainfrom
ac-441/ts-scenario-revision

Conversation

@jayscambler
Copy link
Copy Markdown
Contributor

Summary

Adds scenario revision flow to TypeScript — users can refine created scenarios with feedback instead of starting over from scratch.

Problem

Python has agent_task_revision.py for iterative spec refinement. TypeScript had nothing — once a scenario was created, the only option was delete and retry. Scenario creation is iterative; the first attempt rarely matches what the user wants.

What's new

ts/src/scenarios/scenario-revision.ts — two levels of revision:

1. Spec revision (reviseSpec)

Takes current spec + user feedback → LLM produces updated spec.

const result = await reviseSpec({
  currentSpec: { description: "Basic task", taskPrompt: "Do X" },
  feedback: "Add edge cases and make it harder",
  family: "agent_task",
  provider,
});
// result.revised has the updated spec
// result.changesApplied === true
// result.original preserved for diff/audit
  • Works for all families (agent_task, simulation, workflow, etc.)
  • Preserves original spec on LLM failure (never loses user's work)
  • Includes family-specific context in the revision prompt

2. Output revision (reviseAgentTaskOutput)

Builds revision prompts from judge feedback for the ImprovementLoop.

  • Ports Python's build_revision_prompt()
  • Highlights weak dimensions below threshold
  • Includes task-specific revision instructions

MCP integration

New revise_scenario tool — agents and external tools can trigger revision:

{ "currentSpec": {...}, "feedback": "Add error handling", "family": "simulation" }

Tests

10 new tests covering:

  • Prompt building (spec context, family context, judge feedback, weak dimensions)
  • Spec revision (successful update, LLM failure recovery, multi-family)
  • Output revision (judge feedback, revision instructions, weak dim highlighting)
  • Result shape validation

1300 total tests pass, lint clean.

Linear: AC-441

@linear
Copy link
Copy Markdown

linear Bot commented Mar 27, 2026

@jayscambler jayscambler force-pushed the ac-441/ts-scenario-revision branch from ccfd62d to ff52687 Compare March 27, 2026 04:01
…f starting over (AC-441)

New module: ts/src/scenarios/scenario-revision.ts

Spec-level revision (reviseSpec):
- Takes current spec + user feedback + LLM provider
- Produces updated spec via LLM designer
- Preserves original on failure (changesApplied: false + error)
- Works for all families (agent_task, simulation, workflow, etc.)

Output-level revision (reviseAgentTaskOutput):
- Builds revision prompts from judge feedback (ported from Python)
- Highlights weak dimensions below threshold
- Includes task-specific revision instructions when available

Integration:
- MCP tool: revise_scenario — accessible to agents and external tooling
- Exported from scenarios/index.ts

Tests: 10 new (1300 total passing, lint clean)
- buildRevisionPrompt: spec context, family context, judge feedback
- reviseSpec: successful revision, LLM failure recovery, multi-family
- reviseAgentTaskOutput: judge feedback, revision instructions, weak dims
- RevisionResult shape validation

Linear: AC-441
@jayscambler jayscambler force-pushed the ac-441/ts-scenario-revision branch from ff52687 to 614b40d Compare March 27, 2026 04:07
@jayscambler jayscambler merged commit 64bcda5 into main Mar 27, 2026
4 checks passed
@jayscambler jayscambler deleted the ac-441/ts-scenario-revision branch March 31, 2026 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant