tool call result of type "image" is processed as text

[TheLastAncient]

MCP tool returns an image in format { type: 'image', data: '<base64>', mimeType: 'image/JPEG' }

Expected result:

• Same behavior as when a user sends an image as { type: 'image_url', image_url: '<base64>' }

Actual result:

• base64 string gets treated as text, pushing context to +120k tokens for a single 1024x1024 jpeg image

Hacky workaround:

• intercept tool call and store image results under assistant role, converted to image_url type.

[TheLastAncient]

Actually... This seems to work too:
{"role": "tool", "tool_call_id": _call["id"], "name": _call["function"]["name"], "content": [{"type": "image_url", "image_url": {"url": f"data:{_toolresult["mimeType"]};base64,{_toolresult["data"]}"}}]}
But the list for the content is important.

[LostRuins]

should be fixed now