feat: UI template system for saving and reusing generated widgets#20
feat: UI template system for saving and reusing generated widgets#20breadoncee wants to merge 11 commits intoCopilotKit:mainfrom
Conversation
- Add template tools (save, list, apply, delete) in agent backend - Extend AgentState with templates field - Add "Save as Template" button overlay on widget renderer with animated save flow (input → saving spinner → checkmark confirmation) - Add template library drawer panel (toggle from header) - Templates saved directly to agent state via setState (no chat round-trip) - Template deletion from drawer updates state directly - Apply template sends chat prompt for agent to adapt HTML with new data Closes CopilotKit#16
- Remove misleading system prompt instruction about "save-as-template" messages (saves go directly via agent.setState, not through chat) - Replace fragile DOM querySelector hack for sending prompts with agent.addMessage() + agent.runAgent() API calls
Replace the hide-everything-then-reveal approach with progressive streaming that shows content building up as the LLM streams HTML tokens. The iframe shell loads once and content updates are sent via postMessage instead of full srcdoc reloads. - Loading phrases shown above the widget during streaming - Save Template button only appears after streaming settles - Debounced htmlSettled detection (800ms of no changes) - Smooth fade-out transitions for streaming indicator
- Apply button now prompts user to describe their new data before sending to the agent (textarea with placeholder example) - Template HTML is passed directly in the prompt so the agent doesn't need to look it up from backend state (fixes state sync issue where frontend setState wasn't visible to agent tools) - Fix template card callback signatures
…mponent Move save-as-template UI from widget-renderer into a reusable SaveTemplateOverlay component. Wrap bar chart, pie chart, and widget renderer with it so all generated UIs can be saved as templates. Uses useAgent() directly instead of window.postMessage relay.
Render inline SVG bar/pie chart previews in template cards using the saved component data. Widget renderer templates keep the iframe preview. Templates without HTML show a placeholder icon.
…mission Submit template apply through CopilotChat textarea so components render properly. Include template ID in the message for reliable lookup. Update apply_template tool to support both name and ID-based lookup. Improve agent prompt to copy original HTML and replace data values.
Remove saveTemplate callback, postMessage handler, and unused useAgent import from page.tsx. Template saving now happens directly in the SaveTemplateOverlay component via useAgent().
GeneralJerel
left a comment
GeneralJerel
left a comment
There was a problem hiding this comment.
Review: UI Template System
Thanks for this contribution — the template save/apply concept is a great addition and the UX polish (streaming previews, mini chart SVGs, animated save flow) is solid work. A few things need attention before this can merge, mostly around security and CI.
Must fix (blocking)
1. XSS via postMessage in iframe bridge — widget-renderer.tsx:351-370
The update-content message listener inside BRIDGE_JS sets innerHTML and re-executes <script> tags without checking e.origin or e.source. Since the iframe uses sandbox="allow-scripts allow-same-origin", any window that can obtain a reference to the iframe (another frame on the page, a browser extension, window.open) can inject and execute arbitrary JS in the parent's origin context.
Suggested fixes (pick one or combine):
- Check
e.source === window.parentand validatee.origin - Strip
<script>tags from streamed HTML before settinginnerHTML - Remove
allow-same-originfrom the sandbox if same-origin access isn't required
2. Lint errors (CI gate) — 5 errors, 2 warnings
These will fail CI and must be resolved:
widget-renderer.tsx:549—setShowStreamingIndicator(true)called directly in auseEffectbody. Derive this value instead (e.g., a ref + timeout, or compute it from existing state).template-card.tsx:68—cumulative += d.valuemutates a variable during render inside.map(). Pre-compute the cumulative angles with areduce()pass instead.
3. Template schema mismatch between frontend and backend
SaveTemplateOverlay saves templates directly via agent.setState() with component_type and component_data fields that don't exist in the Python UITemplate TypedDict. Backend save_template produces templates without these fields. This means:
- The library has to use
as unknown as Record<string, unknown>casts to access the extra fields (visible atindex.tsx:200-201) - Backend tools like
list_templatesmay behave differently depending on which path saved the template
Recommend: unify on one schema. Either add component_type/component_data to the backend UITemplate and always save via the backend tool, or define a single shared shape both paths produce.
Should fix (non-blocking, but worth addressing)
4. submitChatPrompt DOM hack — template-library/index.tsx:32-48
Using Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype, "value")?.set to programmatically drive a React-controlled textarea is fragile — it'll break silently if CopilotKit renames the data-testid, changes the textarea implementation, or upgrades React. Check if CopilotKit exposes a programmatic sendMessage() API on the agent/chat hook. If not, at minimum add a fallback when the textarea isn't found (e.g., a console warning or toast).
5. Template preview iframe sandbox — template-card.tsx:141
The preview iframe uses sandbox="allow-same-origin allow-scripts". This combination allows the iframe content to script its way out of the sandbox (it can remove its own sandbox attribute). Since these render user-saved HTML, consider dropping allow-same-origin (the preview just needs to display HTML/CSS — scripts running in previews seem unnecessary).
6. Script re-execution during streaming — widget-renderer.tsx:356-366
The update-content handler re-runs all <script> tags on every streaming update, not just new ones. For widgets with stateful scripts (e.g., Three.js, D3 transitions), this causes duplicate initialization, compounding event listeners, and memory leaks. Consider marking executed scripts (data attribute) and only running new ones, or deferring script execution until streaming settles.
7. apply_template runtime default — templates.py:80
runtime: ToolRuntime = None — the other tools don't default runtime and shouldn't need to. If this is ever called without LangGraph injecting the runtime, every .state access raises AttributeError. Remove the = None default to match the other tool signatures.
Nits
assembleDocument()atwidget-renderer.tsx:388is now a trivial pass-through toassembleShell()— can be inlined or removed.- No delete confirmation on templates — one misclick permanently removes a template. A brief "undo" toast would be a nice UX safety net.
Testing
No tests were added for the new components or backend tools. Not blocking for this PR, but flagging it — the streaming postMessage flow and the template save/apply round-trip are the kind of stateful interactions that tend to regress.
Happy to help work through any of these. The core feature is well-designed — just needs the security edges cleaned up. 🙏
- Prevent XSS via postMessage: validate e.source, strip scripts before innerHTML, only execute new scripts via data-exec tracking - Remove allow-same-origin from widget and preview iframe sandboxes - Fix all lint errors: refactor useEffect setState to adjust-state-during-render pattern, replace cumulative mutation with reduce() pre-computation - Unify template schema: add component_type/component_data to backend UITemplate, remove unsafe casts in frontend - Add console.warn fallback to submitChatPrompt when textarea not found - Remove apply_template runtime=None default to match other tool signatures - Remove trivial assembleDocument() pass-through
Python requires that parameters with defaults precede those without. Since name and template_id have defaults, runtime must too.
Load the iframe shell empty and stream all content via postMessage after the iframe loads. Previously, partial streaming fragments (e.g. an unclosed <style> tag) inserted into the shell would consume the bridge script, preventing all subsequent postMessage updates.
2 participants
opengenui_templates.mp4
Summary
save_template,list_templates,apply_template(supports name + ID lookup),delete_templateChanges
Frontend
SaveTemplateOverlayshared component wrapping all generative UI componentsAgent
apply_templatesupports both name and ID-based lookupTest plan