Skip to content

[Optimize]: coalesce streamed text chunks before Tauri emit to reduce IPC and peer fanout pressure #2161

Description

@YodonTan

Problem / opportunity

During streaming AI responses, the agent runtime emits one TextChunk / ThinkingChunk event per provider chunk. The desktop event loop forwards every chunk to the WebView as a separate Tauri IPC message (JSON serialization, WebView2 boundary crossing, JS parse + dispatch) and, when peer devices are attached, as a separate end-to-end encrypted relay message. Message volume scales with chunk rate, so fast streams create avoidable IPC and network pressure.

Proposed behavior

Coalesce streamed text/thinking chunks before the transport emit in the desktop event loop:

  • Merge chunks sharing the same (session, turn, round, attempt, contentType) within a ~50ms time window; text is concatenated and the thinking isEnd flag is OR-ed.
  • Non-chunk events (completion, error, cancellation, tool events) flush pending merged text first and then pass through, preserving per-stream ordering.
  • Critical / High priority events pass through immediately.
  • Content stays equivalent (the frontend already accumulates the same text), ordering is preserved, and internal event subscribers are unaffected.

Product surface

Desktop app / Cross-cutting

Details, examples, or constraints

  • Reduces Tauri IPC messages by roughly 3-10x during fast streams and proportionally reduces remote peer fanout (each event currently costs serialize + encrypt + base64 + relay + decrypt/parse on the mobile side).
  • Adds at most ~50ms to text delivery; the frontend typewriter and the existing 32ms EventBatcher absorb this.
  • Only the desktop transport path changes: EventQueue, event contracts, ACP/TUI/CLI consumers, and frontend code are untouched.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions