Skip to content

fix(agents): stop inline base64 avatars blowing the agent context window (#758) - #760

Merged
lilyshen0722 merged 1 commit into
mainfrom
fix/758-avatar-payload-bloat
Jul 26, 2026
Merged

fix(agents): stop inline base64 avatars blowing the agent context window (#758)#760
lilyshen0722 merged 1 commit into
mainfrom
fix/758-avatar-payload-bloat

Conversation

@lilyshen0722

Copy link
Copy Markdown
Contributor

Closes #758.

The problem, measured

Avatars are stored as data: URIs on the user row, so every agent-facing message carried its author's full image bytes — and the same avatar repeated on every message that author had sent.

path payload avatar share
commonly_get_messages, limit 20 230,170 chars 71%

An agent asked to read a busy room exhausted its context on profile pictures before reaching the conversation, saw only truncation, and had no way to diagnose why. It scales with room activity, not conversation length — and it hits exactly what we market: agents reading the room and handing off to each other.

Fixed at the router, not per handler

The leak was spread across getRecentMessages, the post-message echo, thread comments, and the create-pod member list. There is no shared user serializer to fix instead, and a per-handler fix wouldn't cover the next endpoint someone adds. So: one router.use on the agent runtime router strips inline avatars from every response.

Only data: values are dropped. A URL costs nothing and stays useful — the rule is no base64 in an agent payload, not no avatars.

The strip returns a copy and never mutates its input. That's load-bearing: the same message object is reused for the human-facing Socket.io broadcast where the avatar is rendered, so stripping in place would blank every live avatar in the UI.

Corrections to the issue as filed

  • get_context was already cleanpodContextService projects members and messages down to name/author/content.
  • Only 6 of 362 users have data: avatars (53KB total), so the eventual migration is trivial.
  • avatarService.resolveAvatarUrls already existed with zero non-test callers — an abandoned earlier attempt at this same fix.

Test

10 new tests. The route test mounts the real router in an express app rather than pulling a handler off the stack, specifically so the middleware runs — a per-handler test would pass even with the middleware deleted. It asserts no data: in the response, that a URL avatar survives, that isBot/self (which #757's detection depends on) survive, and that the payload actually shrinks.

Pre-existing local suite failures on jsonwebtoken are Node-26 env drift — identical on origin/main (4 failed / 4 total there too). Lint warnings at baseline.

Deliberately out of scope, worth a follow-up

  • human read path: formatMessage emits two copies of each avatar (463,281 chars for the same pod)
  • GET /api/pods populates createdBy + members before the membership filter (1.9MB)
  • migrating the 6 data: rows to object storage per ADR-002, plus a write-side cap

🤖 Generated with Claude Code

…dow (#758)

Avatars are stored as `data:` URIs on the user row, so every agent-facing
message carried its author's full image bytes — and the same avatar repeated on
every message that author had sent. Measured on a real pod, `commonly_get_messages`
at the default limit returned 230,170 characters of which 71% was image data.
An agent asked to read a busy room exhausted its context on profile pictures
before reaching the conversation, saw only truncation, and had no way to tell
why. It scales with room activity rather than conversation length, and it hits
exactly the thing we market: agents reading the room and handing off.

Fixed at the router, not per handler. The leak was spread across
getRecentMessages, the post-message echo, thread comments and the create-pod
member list; there is no shared user serializer to fix instead; and a
per-handler fix would not cover the next endpoint added. One `router.use` on the
agent runtime router strips inline avatars from every response.

Only `data:` values are dropped — a URL costs nothing and stays useful, so the
rule is "no base64 in an agent payload", not "no avatars".

The strip returns a copy and never mutates its input. That is load-bearing: the
same message object is reused for the human-facing Socket.io broadcast, where
the avatar IS rendered, so stripping in place would blank every live avatar in
the UI.

Deliberately NOT in scope, filed as follow-up: the human read path
(`formatMessage` emits two copies of each avatar), the sidebar pod listing
(populates before filtering), and migrating the 6 remaining `data:` rows to
object storage per ADR-002 with a write-side cap.

Closes #758

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DMeWzgFxsfBcjoLVLewEES
@lilyshen0722
lilyshen0722 merged commit 22a6ef9 into main Jul 26, 2026
10 checks passed
@lilyshen0722
lilyshen0722 deleted the fix/758-avatar-payload-bloat branch July 26, 2026 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inline base64 avatars in message payloads blow the agent context window when reading a room

1 participant