fix(agents): stop inline base64 avatars blowing the agent context window (#758) - #760
Merged
Merged
Conversation
…dow (#758) Avatars are stored as `data:` URIs on the user row, so every agent-facing message carried its author's full image bytes — and the same avatar repeated on every message that author had sent. Measured on a real pod, `commonly_get_messages` at the default limit returned 230,170 characters of which 71% was image data. An agent asked to read a busy room exhausted its context on profile pictures before reaching the conversation, saw only truncation, and had no way to tell why. It scales with room activity rather than conversation length, and it hits exactly the thing we market: agents reading the room and handing off. Fixed at the router, not per handler. The leak was spread across getRecentMessages, the post-message echo, thread comments and the create-pod member list; there is no shared user serializer to fix instead; and a per-handler fix would not cover the next endpoint added. One `router.use` on the agent runtime router strips inline avatars from every response. Only `data:` values are dropped — a URL costs nothing and stays useful, so the rule is "no base64 in an agent payload", not "no avatars". The strip returns a copy and never mutates its input. That is load-bearing: the same message object is reused for the human-facing Socket.io broadcast, where the avatar IS rendered, so stripping in place would blank every live avatar in the UI. Deliberately NOT in scope, filed as follow-up: the human read path (`formatMessage` emits two copies of each avatar), the sidebar pod listing (populates before filtering), and migrating the 6 remaining `data:` rows to object storage per ADR-002 with a write-side cap. Closes #758 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DMeWzgFxsfBcjoLVLewEES
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #758.
The problem, measured
Avatars are stored as
data:URIs on the user row, so every agent-facing message carried its author's full image bytes — and the same avatar repeated on every message that author had sent.commonly_get_messages, limit 20An agent asked to read a busy room exhausted its context on profile pictures before reaching the conversation, saw only truncation, and had no way to diagnose why. It scales with room activity, not conversation length — and it hits exactly what we market: agents reading the room and handing off to each other.
Fixed at the router, not per handler
The leak was spread across
getRecentMessages, the post-message echo, thread comments, and the create-pod member list. There is no shared user serializer to fix instead, and a per-handler fix wouldn't cover the next endpoint someone adds. So: onerouter.useon the agent runtime router strips inline avatars from every response.Only
data:values are dropped. A URL costs nothing and stays useful — the rule is no base64 in an agent payload, not no avatars.The strip returns a copy and never mutates its input. That's load-bearing: the same message object is reused for the human-facing Socket.io broadcast where the avatar is rendered, so stripping in place would blank every live avatar in the UI.
Corrections to the issue as filed
get_contextwas already clean —podContextServiceprojects members and messages down to name/author/content.data:avatars (53KB total), so the eventual migration is trivial.avatarService.resolveAvatarUrlsalready existed with zero non-test callers — an abandoned earlier attempt at this same fix.Test
10 new tests. The route test mounts the real router in an express app rather than pulling a handler off the stack, specifically so the middleware runs — a per-handler test would pass even with the middleware deleted. It asserts no
data:in the response, that a URL avatar survives, thatisBot/self(which #757's detection depends on) survive, and that the payload actually shrinks.Pre-existing local suite failures on
jsonwebtokenare Node-26 env drift — identical onorigin/main(4 failed / 4 total there too). Lint warnings at baseline.Deliberately out of scope, worth a follow-up
formatMessageemits two copies of each avatar (463,281 chars for the same pod)GET /api/podspopulatescreatedBy+membersbefore the membership filter (1.9MB)data:rows to object storage per ADR-002, plus a write-side cap🤖 Generated with Claude Code