feat: 支持结构化图片消息输入 - #3
Open
qiuYliangM wants to merge 1 commit into
Open
Conversation
- Op::SendMessage 新增可选结构化输入 UserMessageInput（Text/LocalImage），纯文本路径行为不变 - ContentBlock 新增 LocalImage 本地图片引用，会话持久化只存 workspace 相对路径与元数据，不存 Base64 - turn loop 构建 provider 请求前对消息克隆物化：workspace 相对路径校验、20MB 上限、魔数 MIME 嗅探，文件缺失明确报错 - Anthropic 遇 data: URL 转换为原生 base64 source；Chat/Responses 图片块序列化补契约测试 - 收口 Base64 泄漏面：mouse_ui 占位渲染、runtime_api 返回 metadata、context_report 固定估算 - image_analyze 加固：20MB 上限、真实文件头检测、更新工具描述（仅用于未随当前消息原生提供的图片） - image_analyze 请求不再携带 temperature：kimi-for-coding 等模型只接受默认值，显式 temperature 会被 400 拒绝 Signed-off-by: luzeyang (INT) <lu.zeyang@h3c.com>
|
Thanks @qiuYliangM for taking the time to contribute. This repository is observing a maintainer-managed PR intake gate in dry-run mode, so this pull request is staying open. This note helps maintainers prepare the allowlist before any enforcement is considered. Please read |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
为底座增加结构化图片消息输入能力:用户消息可携带 workspace 本地图片引用,请求前临时物化为 Base64 图片块,持久化、日志、compaction 全程不含 Base64。
背景
pinvou3 普通会话原生图片输入改造的底座部分。原
Op::SendMessage只接受 String,宿主无法提交图片块;Anthropic 客户端会把 data URL 当远程 URL 发送;image_analyze 缺独立大小限制、仅凭扩展名判型、硬编码 temperature 导致 kimi-for-coding 等模型 400 拒绝。变更
Op::SendMessage新增input: Option<UserMessageInput>(UserInputBlock::{Text, LocalImage}),纯文本调用传None,行为零变化vision/image_input.rs:workspace 相对路径校验(拒绝对路径/../symlink 逃逸)、20MB 统一上限、PNG/JPEG/GIF/WebP/BMP 魔数嗅探source:{type:"base64",media_type,data};OpenAI Chat/Responses 补请求体契约测试[image:mime,N bytes]占位,Base64 不出请求外验证
cargo check -p codewhale-tui通过cargo test forkguard_33 项全过(含新增 4 项)备注
d4a05585e;squash 会改变哈希,父仓需再追加一次 gitlink 更新