-
Notifications
You must be signed in to change notification settings - Fork 56
Description
Problem
The edit_screens and generate_screen_from_text tools only accept text prompts. There's no way to send an image (e.g., a screenshot of a built component) to Stitch for visual comparison against a design target.
Stitch's web UI supports this through "Experimental Mode" — upload a screenshot, get AI-powered review with annotated feedback. But the MCP server doesn't expose image input.
Use Case: Design-vs-Implementation Review Loop
After an agent builds a React component from a Stitch mockup, it needs Stitch to verify the implementation matches the design:
1. /mockup-lab generate → approved Stitch mockup (design target)
2. Agent builds React component
3. Agent screenshots the running component
4. Agent sends screenshot to Stitch: "Compare this against the design target"
5. Stitch returns annotated correction mockup showing diffs
6. Agent fixes code → repeat until Stitch approves
Step 4 is impossible via the SDK because edit_screens only accepts text, not images.
Proposed Solutions
Option A: Add imageData (base64) or imageUrl parameter to edit_screens:
{
"projectId": "...",
"selectedScreenIds": ["design-target-id"],
"prompt": "Compare this implementation against the design target",
"referenceImage": "<base64 PNG or URL>"
}Option B: New dedicated review_screen tool:
{
"projectId": "...",
"screenId": "design-target-id",
"implementationImage": "<base64 PNG or URL>",
"prompt": "List every difference: spacing, colors, typography, layout"
}Returns an annotated correction mockup + structured diff (pixel values, hex colors, missing elements).
Context
This is a companion to #129 (collaboration API). Together they would enable:
- Human feedback via comments API (feat: Collaboration API tools (comments, chat history) for agentic workflows #129)
- Visual implementation review via image input (this issue)
- Full agentic design → build → review → iterate loop without leaving the SDK
Environment
@google/stitch-sdk0.0.3- Discovered while building an agentic design pipeline that generates mockups, codes React components, and needs Stitch to verify the implementation.