Design UI‑Integrated Agent Navigation (Google Maps–Like)

[yaredtsy]

Description:
Design an agent experience that is deeply integrated into the UI, not a separate chat window. The agent should feel more like Google Maps than a chatbot.

The agent should:

• Navigate the canvas visually (pan, zoom, focus on nodes)
• Interact directly with graph elements (functions, dependencies, logs)
• Pause and wait for user input instead of constantly responding
• Guide the user step‑by‑step through the system

If possible, the interaction should feel like:

• Watching a short walkthrough or guided tour
• Stepping through a flow, similar to a video or “walking” through the system
• A shared space where the agent points, highlights, and rearranges things

Goals:

• Reduce cognitive load
• Make exploration feel spatial and intuitive
• Replace text‑heavy explanations with movement, focus, and context

Acceptance Criteria:

• Agent actions are visible on the canvas (not just text)
• User can interrupt, pause, or take control at any time
• Navigation feels smooth and intentional (zooming instead of jumping)
• Agent guidance works without overwhelming the user

Notes:
This agent should behave more like a guide or navigator than a problem solver. Understanding should come from seeing and moving, not from reading long explanations.

[yaredtsy]

1. The Protocol: "Canvas Action Schema"

The AI agent needs a structured way to "drive" the UI. You should define a JSON-based instruction set that the V-NOC frontend can interpret.

Example Action Frame:

JSON

{ "timestamp": 1200, "action": "HIGHLIGHT_NODE", "params": { "nodeId": "create_child", "color": "orange" }, "narration": "First, the main function calls create_child..." }

2. Technical Implementation Plan

Phase 1: The "Puppeteer" API

Before the AI can do anything, your React/Canvas components need to expose methods to an external controller.

• Focus & Zoom: A function to programmatically trigger react-flow (or whatever library you use) to zoom into a specific node ID.
• Node Toggling: An API to switch a node between collapsed (Image 2) and expanded/code-view (Image 1) states.
• Code Highlighting: If you use a library like Monaco or Prism inside the nodes, you need a way to pass lineNumbers: [3, 4, 5] to trigger a visual highlight.

Phase 2: The Agent Logic (The "Director")

The AI shouldn't just explain code; it should "direct" the camera.

• Context Injection: Feed the agent the graph structure (Node A connects to Node B) and the code content.
• Step Generation: The AI generates a Timeline Script.
  • Step 1: Highlight main.
  • Step 2: Toggle create_child to expanded view.
  • Step 3: Highlight lines 3–5 in create_child.
  • Step 4: Move focus to the Child class node.

3. Desired "Walkthrough" Workflow

Feature	AI Action	UI Response
Node Discovery	FOCUS_NODE("main")	Canvas pans smoothly to the main node.
Exploration	TOGGLE_NODE("create_child", "expand")	Node expands to show the Python code (as seen in your first image).
Deep Dive	HIGHLIGHT_LINES("create_child", [3, 4, 5])	A glow effect appears behind the code lines where Child is instantiated.
Relationship	FOLLOW_EDGE("create_child", "Child")	An animation travels along the dotted line to show the data flow.

---

4. Suggested Roadmap for Issue #33

1. State Management: Ensure every node state (position, expanded/collapsed, active highlights) can be set via a single globalStore.
2. The "Agent Bridge": Create a system prompt for your AI that specifically instructs it to output a thought and a command.
   • Thought: "I need to show how the child is initialized."
   • Command: {"call": "expandAndHighlight", "node": "create_child", "lines": [3]}
3. Visual Polish: Implement "Auto-pan." When the AI talks about a node that is off-screen, the canvas should automatically glide to it.

[yaredtsy]