Skip to content

Commit 4c566fd

Browse files
XXPermanentXXXXPermanentXX
committed
feat(token-plan): support local images with base64 data URIs
- convert local images to Base64 for Token Plan image and video commands - preserve the existing OSS upload flow for standard API Key profiles - use wan2.7-image as the default image model with the sync endpoint - hide full Base64 image content in dry-run output - add Token Plan compatibility tests and update related docs
1 parent 4ca3e2d commit 4c566fd

22 files changed

Lines changed: 521 additions & 74 deletions

‎docs/token-plan-profile-integration.md‎

Lines changed: 23 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ CLI 应解析并保存以下配置：
8080
"default_video_model": "happyhorse-1.1-t2v",
8181
"default_image_to_video_model": "happyhorse-1.1-i2v",
8282
"default_reference_to_video_model": "happyhorse-1.1-r2v",
83-
"default_image_model": "qwen-image-2.0"
83+
"default_image_model": "wan2.7-image"
8484
}
8585
}
8686
```
@@ -245,7 +245,7 @@ default_text_model: qwen3.8-max-preview
245245
default_video_model: happyhorse-1.1-t2v
246246
default_image_to_video_model: happyhorse-1.1-i2v
247247
default_reference_to_video_model: happyhorse-1.1-r2v
248-
default_image_model: qwen-image-2.0
248+
default_image_model: wan2.7-image
249249
```
250250

251251
Token Plan Base URL 预设只在登录写入阶段提供最低优先级的缺省值：
@@ -259,7 +259,7 @@ Token Plan Base URL 预设只在登录写入阶段提供最低优先级的缺省
259259

260260
登录成功时应把显式 Base URL 或缺失的预设 Base URL，以及默认模型写入 Profile，使 `config show --config token-plan` 能看到完整配置。环境变量不复制进 Profile。运行时不再合并预设；如果手工删除字段，则按统一的环境变量、配置文件和系统默认值链继续解析。
261261

262-
默认模型采用更简单的固定策略：每次执行 `auth login --config token-plan`，都将 `default_text_model` 重置为 `qwen3.8-max-preview`，将 `default_video_model` 重置为 `happyhorse-1.1-t2v`，将 `default_image_to_video_model` 重置为 `happyhorse-1.1-i2v`，将 `default_reference_to_video_model` 重置为 `happyhorse-1.1-r2v`，将 `default_image_model` 重置为 `qwen-image-2.0`。登录不保留用户之前写入的其他 Profile 默认模型；用户需要临时调用其他 Token Plan 模型时，通过具体模型命令的 `--model` 覆盖，不修改这些内置默认值。
262+
默认模型采用更简单的固定策略：每次执行 `auth login --config token-plan`，都将 `default_text_model` 重置为 `qwen3.8-max-preview`，将 `default_video_model` 重置为 `happyhorse-1.1-t2v`，将 `default_image_to_video_model` 重置为 `happyhorse-1.1-i2v`，将 `default_reference_to_video_model` 重置为 `happyhorse-1.1-r2v`，将 `default_image_model` 重置为 `wan2.7-image`。登录不保留用户之前写入的其他 Profile 默认模型；用户需要临时调用其他 Token Plan 模型时，通过具体模型命令的 `--model` 覆盖，不修改这些内置默认值。
263263

264264
预设建议通过集中 registry 表达，不在 resolver、命令和 Client 中散落名称判断：
265265

@@ -271,7 +271,7 @@ const MODEL_PROFILE_PRESETS = {
271271
defaultVideoModel: "happyhorse-1.1-t2v",
272272
defaultImageToVideoModel: "happyhorse-1.1-i2v",
273273
defaultReferenceToVideoModel: "happyhorse-1.1-r2v",
274-
defaultImageModel: "qwen-image-2.0",
274+
defaultImageModel: "wan2.7-image",
275275
},
276276
};
277277
```
@@ -381,13 +381,29 @@ image: <base_url>/api/v1/services/aigc/.../generation
381381
| 能力 | 默认模型 | 调用方式 |
382382
| -------------- | --------------------- | ---------------------------------- |
383383
| 文本生成和推理 | `qwen3.8-max-preview` | OpenAI Compatible Chat Completions |
384-
| 图片生成和编辑 | `qwen-image-2.0` | DashScope 原生图片接口 |
384+
| 图片生成和编辑 | `wan2.7-image` | DashScope 多模态图片接口 |
385385
| 文生视频 | `happyhorse-1.1-t2v` | DashScope 原生视频接口 |
386386
| 图生视频 | `happyhorse-1.1-i2v` | `bl video generate --image` |
387387
| 参考生视频 | `happyhorse-1.1-r2v` | `bl video ref` |
388388

389389
Token Plan 当前模型快照中还包含其他文本、视觉理解、图片和视频模型，但该列表可能由后端调整。基础接入不维护阻断请求的本地白名单；用户可通过具体模型命令的 `--model` 临时覆盖本次请求，但再次登录时 Profile 默认模型仍重置为内置版本。
390390

391+
### 当前模型与本地图片兼容范围
392+
393+
| 模型 | 图片输入能力 | CLI 本地图片处理 |
394+
| ----------------------------------- | -------------- | ---------------------------------------------------------- |
395+
| `qwen3.8-max-preview` | 视觉理解 | Token Plan 下转换为 Base64 Data URI |
396+
| `qwen3.7-plus` | 视觉理解 | Token Plan 下转换为 Base64 Data URI |
397+
| `qwen3.7-max` | 纯文本 | 不涉及图片上传 |
398+
| `qwen3.6-flash` | 视觉理解 | Token Plan 下转换为 Base64 Data URI |
399+
| `wan2.7-image` / `wan2.7-image-pro` | 图片生成与编辑 | 文生图不需要输入图片；编辑本地图片时转换为 Base64 Data URI |
400+
| `happyhorse-1.1-i2v` | 图生视频 | 首帧本地图片转换为 Base64 Data URI |
401+
| `happyhorse-1.1-t2v` | 文生视频 | 不涉及图片上传 |
402+
| `happyhorse-1.1-r2v` | 参考生视频 | 参考本地图片转换为 Base64 Data URI |
403+
| `deepseek-v4-pro` / `glm-5.2` | 纯文本 | 不涉及图片上传 |
404+
405+
Token Plan 图片兼容只处理官方明确支持 Base64 的图片字段；参考视频和参考音频仍要求可访问 URL。普通 API Key 保持各命令既有行为：图片编辑和视频入口继续使用临时 OSS，视觉理解的小图继续使用原有 Base64 路径。
406+
391407
语音和音频不作为本阶段支持承诺。现有命令仍保持通用实现，但 Token Plan Profile 的验收不包含这些模态。
392408

393409
## 错误处理
@@ -443,7 +459,7 @@ feat(auth): support token-plan API key login
443459
- 使用 Token Plan 预设文本模型验证 API Key。
444460
- 登录验证前不写配置。
445461
- 验证成功后一次写入 API Key、canonical Base URL 和默认模型。
446-
- 每次登录都将默认模型重置为 `qwen3.8-max-preview`、`qwen-image-2.0`、`happyhorse-1.1-t2v`、`happyhorse-1.1-i2v` 和 `happyhorse-1.1-r2v`。
462+
- 每次登录都将默认模型重置为 `qwen3.8-max-preview`、`wan2.7-image`、`happyhorse-1.1-t2v`、`happyhorse-1.1-i2v` 和 `happyhorse-1.1-r2v`。
447463
- 验证失败不留下半配置。
448464
- 补充一个最小 Token Plan 登录 E2E，覆盖命名 Profile 落盘、环境变量不复制、预设 Base URL 物化和默认模型重置；通用 API Key 登录 E2E 继续覆盖成功原子保存和失败不写半配置。
449465
- 该 commit 暂不承诺自动归一化用户显式输入的 SDK Base URL。
@@ -546,7 +562,7 @@ fix(core): normalize model base URLs across all sources
546562
- 显式 Base URL 覆盖预设并经过通用归一化。
547563
- 登录验证失败不写入任何 Token Plan 半配置。
548564
- 文本默认使用 `qwen3.8-max-preview`。
549-
- 图片默认使用 `qwen-image-2.0`。
565+
- 图片默认使用 `wan2.7-image`。
550566
- 视频默认使用 `happyhorse-1.1-t2v`；图生和参考生入口分别使用 `happyhorse-1.1-i2v` 和 `happyhorse-1.1-r2v`。
551567
- 文本、图片和视频均复用现有 `apiKey` Client。
552568
- 管控命令继续使用 OpenAPI AK/SK，不受模型 Profile 影响。

‎packages/commands/src/commands/image/edit.ts‎

Lines changed: 24 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import {
2222
resolveWatermark,
2323
ASYNC_FLAG,
2424
CONCURRENT_FLAG,
25+
redactDataUri,
2526
} from "bailian-cli-core";
2627
import { poll } from "bailian-cli-runtime";
2728
import { downloadFile } from "bailian-cli-runtime";
@@ -31,10 +32,15 @@ import { resolveImageSize } from "bailian-cli-runtime";
3132
import { join } from "path";
3233
import { BOOL_FLAG_PROMPT_EXTEND_CLI_TRUE, BOOL_FLAG_WATERMARK } from "bailian-cli-runtime";
3334

34-
const SYNC_MODEL_PREFIXES = ["qwen-image-2.0", "qwen-image-max"];
35+
const SYNC_MODEL_PREFIXES = ["qwen-image-2.0", "qwen-image-max", "wan2.7-image"];
36+
const PROMPT_EXTEND_DEFAULT_PREFIXES = ["qwen-image-2.0", "qwen-image-max"];
3537

3638
function isSyncModel(model: string): boolean {
37-
return SYNC_MODEL_PREFIXES.some((p) => model.startsWith(p));
39+
return SYNC_MODEL_PREFIXES.some((prefix) => model.startsWith(prefix));
40+
}
41+
42+
function enablesPromptExtendByDefault(model: string): boolean {
43+
return PROMPT_EXTEND_DEFAULT_PREFIXES.some((prefix) => model.startsWith(prefix));
3844
}
3945

4046
const EDIT_FLAGS = {
@@ -98,7 +104,7 @@ const EDIT_FLAGS = {
98104
type EditFlags = ParsedFlags<typeof EDIT_FLAGS>;
99105

100106
export default defineCommand({
101-
description: "Edit an existing image with text instructions (Qwen-Image)",
107+
description: "Edit an existing image with text instructions (Qwen-Image / Wan 2.7)",
102108
auth: "apiKey",
103109
usageArgs: "--image <url> --prompt <text> [flags]",
104110
flags: EDIT_FLAGS,
@@ -107,6 +113,7 @@ export default defineCommand({
107113
'--image https://example.com/logo.png --prompt "Change color to blue" --n 3',
108114
'--image ./a.png --image ./b.png --prompt "Merge two images into one collage"',
109115
'--image https://example.com/photo.png --prompt "Remove the person" --model qwen-image-2.0-pro',
116+
'--image ./photo.png --prompt "Change the style" --model wan2.7-image',
110117
'--image ./photo.png --prompt "Replace the background with a beach" --watermark false',
111118
],
112119
async run(ctx) {
@@ -125,13 +132,13 @@ export default defineCommand({
125132

126133
// Auto-upload local files (resolve all images in parallel)
127134
const resolvedImages = await Promise.all(
128-
rawImages.map((img) => ctx.client.uploadFile(img, model)),
135+
rawImages.map((image) => ctx.client.resolveImageInput(image, model)),
129136
);
130137
const n = flags.n ?? 1;
131138

132139
const promptExtend = resolveBooleanFlag(
133140
flags.promptExtend,
134-
useSync ? true : undefined,
141+
enablesPromptExtendByDefault(model) ? true : undefined,
135142
"prompt-extend",
136143
);
137144

@@ -169,7 +176,18 @@ export default defineCommand({
169176
const format = detectOutputFormat(settings.output);
170177

171178
if (settings.dryRun) {
172-
emitResult({ request: body, mode: useSync ? "sync" : "async" }, format);
179+
const previewBody = {
180+
...body,
181+
input: {
182+
messages: body.input.messages.map((message) => ({
183+
...message,
184+
content: message.content.map((item) =>
185+
item.image ? { ...item, image: redactDataUri(item.image) } : item,
186+
),
187+
})),
188+
},
189+
};
190+
emitResult({ request: previewBody, mode: useSync ? "sync" : "async" }, format);
173191
return;
174192
}
175193

‎packages/commands/src/commands/image/generate.ts‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,16 @@ import { BOOL_FLAG_PROMPT_EXTEND_IMAGE_GENERATE, BOOL_FLAG_WATERMARK } from "bai
3131

3232
import { join } from "path";
3333

34-
// qwen-image-2.0 series uses the sync multimodal-generation endpoint
35-
const SYNC_MODEL_PREFIXES = ["qwen-image-2.0", "qwen-image-max"];
34+
// Qwen-Image 2.0 and Wan 2.7 use the sync multimodal-generation endpoint.
35+
const SYNC_MODEL_PREFIXES = ["qwen-image-2.0", "qwen-image-max", "wan2.7-image"];
36+
const PROMPT_EXTEND_DEFAULT_PREFIXES = ["qwen-image-2.0", "qwen-image-max"];
3637

3738
function isSyncModel(model: string): boolean {
38-
return SYNC_MODEL_PREFIXES.some((p) => model.startsWith(p));
39+
return SYNC_MODEL_PREFIXES.some((prefix) => model.startsWith(prefix));
40+
}
41+
42+
function enablesPromptExtendByDefault(model: string): boolean {
43+
return PROMPT_EXTEND_DEFAULT_PREFIXES.some((prefix) => model.startsWith(prefix));
3944
}
4045

4146
const GENERATE_FLAGS = {
@@ -121,7 +126,7 @@ export default defineCommand({
121126

122127
const promptExtend = resolveBooleanFlag(
123128
flags.promptExtend,
124-
useSync ? true : undefined,
129+
enablesPromptExtendByDefault(model) ? true : undefined,
125130
"prompt-extend",
126131
);
127132

‎packages/commands/src/commands/video/generate.ts‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import {
1313
resolveWatermark,
1414
ASYNC_FLAG,
1515
CONCURRENT_FLAG,
16+
redactDataUri,
1617
} from "bailian-cli-core";
1718
import { poll } from "bailian-cli-runtime";
1819
import { downloadFile, formatBytes } from "bailian-cli-runtime";
@@ -113,7 +114,7 @@ export default defineCommand({
113114
// Auto-upload local image file for i2v
114115
let resolvedImageUrl: string | undefined;
115116
if (imageUrl) {
116-
resolvedImageUrl = await ctx.client.uploadFile(imageUrl, model);
117+
resolvedImageUrl = await ctx.client.resolveImageInput(imageUrl, model);
117118
}
118119

119120
const watermark = resolveWatermark(flags.watermark);
@@ -140,7 +141,16 @@ export default defineCommand({
140141
};
141142

142143
if (settings.dryRun) {
143-
emitResult({ request: body }, format);
144+
const previewBody = resolvedImageUrl
145+
? {
146+
...body,
147+
input: {
148+
...body.input,
149+
media: [{ type: "first_frame" as const, url: redactDataUri(resolvedImageUrl) }],
150+
},
151+
}
152+
: body;
153+
emitResult({ request: previewBody }, format);
144154
return;
145155
}
146156

‎packages/commands/src/commands/video/ref.ts‎

Lines changed: 24 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ import {
1313
resolveWatermark,
1414
ASYNC_FLAG,
1515
CONCURRENT_FLAG,
16+
redactDataUri,
1617
} from "bailian-cli-core";
1718
import { poll } from "bailian-cli-runtime";
1819
import { downloadFile, formatBytes } from "bailian-cli-runtime";
@@ -124,33 +125,33 @@ export default defineCommand({
124125
const media: DashScopeVideoRefRequest["input"]["media"] = [];
125126

126127
// Add reference images
127-
for (let i = 0; i < images.length; i++) {
128-
const resolved = await ctx.client.uploadFile(images[i]!, model);
128+
for (let imageIndex = 0; imageIndex < images.length; imageIndex++) {
129+
const resolved = await ctx.client.resolveImageInput(images[imageIndex]!, model);
129130
const entry: DashScopeVideoRefRequest["input"]["media"][number] = {
130131
type: "reference_image",
131132
url: resolved,
132133
};
133134

134135
// Pair voice by position
135-
if (imageVoices[i]) {
136-
const resolvedVoice = await ctx.client.uploadFile(imageVoices[i]!, model);
136+
if (imageVoices[imageIndex]) {
137+
const resolvedVoice = await ctx.client.uploadFile(imageVoices[imageIndex]!, model);
137138
entry.reference_voice = resolvedVoice;
138139
}
139140

140141
media.push(entry);
141142
}
142143

143144
// Add reference videos
144-
for (let i = 0; i < refVideos.length; i++) {
145-
const resolved = await ctx.client.uploadFile(refVideos[i]!, model);
145+
for (let videoIndex = 0; videoIndex < refVideos.length; videoIndex++) {
146+
const resolved = await ctx.client.uploadFile(refVideos[videoIndex]!, model);
146147
const entry: DashScopeVideoRefRequest["input"]["media"][number] = {
147148
type: "reference_video",
148149
url: resolved,
149150
};
150151

151152
// Pair voice by position
152-
if (videoVoices[i]) {
153-
const resolvedVoice = await ctx.client.uploadFile(videoVoices[i]!, model);
153+
if (videoVoices[videoIndex]) {
154+
const resolvedVoice = await ctx.client.uploadFile(videoVoices[videoIndex]!, model);
154155
entry.reference_voice = resolvedVoice;
155156
}
156157

@@ -178,7 +179,18 @@ export default defineCommand({
178179
};
179180

180181
if (settings.dryRun) {
181-
emitResult({ request: body }, format);
182+
const previewBody = {
183+
...body,
184+
input: {
185+
...body.input,
186+
media: body.input.media.map((item) => ({
187+
...item,
188+
url: redactDataUri(item.url),
189+
reference_voice: item.reference_voice ? redactDataUri(item.reference_voice) : undefined,
190+
})),
191+
},
192+
};
193+
emitResult({ request: previewBody }, format);
182194
return;
183195
}
184196

@@ -233,11 +245,11 @@ export default defineCommand({
233245
);
234246

235247
const videos: Array<{ taskId: string; videoUrl: string }> = [];
236-
for (let i = 0; i < results.length; i++) {
237-
const result = results[i]!;
248+
for (let resultIndex = 0; resultIndex < results.length; resultIndex++) {
249+
const result = results[resultIndex]!;
238250
const videoUrl =
239251
result.output.video_url || (result.output.results && result.output.results[0]?.url);
240-
if (videoUrl) videos.push({ taskId: taskIds[i]!, videoUrl });
252+
if (videoUrl) videos.push({ taskId: taskIds[resultIndex]!, videoUrl });
241253
}
242254

243255
if (videos.length === 0) {

‎packages/commands/src/commands/vision/describe.ts‎

Lines changed: 17 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -8,18 +8,13 @@ import {
88
BailianError,
99
ExitCode,
1010
isLocalFile,
11+
imageFileToDataUri,
12+
redactDataUri,
1113
} from "bailian-cli-core";
1214
import { emitResult, emitBare } from "bailian-cli-runtime";
13-
import { readFileSync, existsSync } from "fs";
15+
import { existsSync, statSync } from "fs";
1416
import { extname } from "path";
1517

16-
const IMAGE_MIME_TYPES: Record<string, string> = {
17-
".jpg": "image/jpeg",
18-
".jpeg": "image/jpeg",
19-
".png": "image/png",
20-
".webp": "image/webp",
21-
};
22-
2318
const VIDEO_EXTENSIONS = new Set([".mp4", ".mov", ".avi", ".mkv", ".webm", ".flv", ".wmv"]);
2419

2520
function isVideoInput(input: string): boolean {
@@ -35,18 +30,7 @@ async function toImageUrl(image: string): Promise<string> {
3530
if (image.startsWith("data:")) return image;
3631
if (image.startsWith("http://") || image.startsWith("https://")) return image;
3732
if (image.startsWith("oss://")) return image;
38-
39-
// Local file → data URI (for small files < 10MB, fallback)
40-
if (!existsSync(image)) throw new BailianError(`File not found: ${image}`, ExitCode.USAGE);
41-
const ext = extname(image).toLowerCase();
42-
const mime = IMAGE_MIME_TYPES[ext];
43-
if (!mime)
44-
throw new BailianError(
45-
`Unsupported image format "${ext}". Supported: jpg, jpeg, png, webp`,
46-
ExitCode.USAGE,
47-
);
48-
const buf = readFileSync(image);
49-
return `data:${mime};base64,${buf.toString("base64")}`;
33+
return imageFileToDataUri(image);
5034
}
5135

5236
export default defineCommand({
@@ -86,7 +70,10 @@ export default defineCommand({
8670
const { settings, flags } = ctx;
8771
let image = flags.image;
8872
const videoInputs = flags.video ?? [];
89-
const model = flags.model || "qwen3-vl-plus";
73+
const model =
74+
flags.model ||
75+
(ctx.client.usesTokenPlanEndpoint() ? settings.defaultTextModel : undefined) ||
76+
"qwen3-vl-plus";
9077

9178
// Auto-detect: if --image was given a video file, treat it as --video
9279
if (image && isVideoInput(image)) {
@@ -102,7 +89,14 @@ export default defineCommand({
10289

10390
if (settings.dryRun) {
10491
emitResult(
105-
{ request: { prompt, image, video: videoInputs.length ? videoInputs : undefined, model } },
92+
{
93+
request: {
94+
prompt,
95+
image: image ? redactDataUri(image) : undefined,
96+
video: videoInputs.length ? videoInputs.map(redactDataUri) : undefined,
97+
model,
98+
},
99+
},
106100
format,
107101
);
108102
return;
@@ -132,10 +126,9 @@ export default defineCommand({
132126

133127
let finalImageUrl = imageUrl;
134128
if (isLocalFile(image) && imageUrl.startsWith("data:")) {
135-
const { statSync } = await import("fs");
136129
const fileSize = statSync(image).size;
137130
if (fileSize > 5 * 1024 * 1024) {
138-
finalImageUrl = await ctx.client.uploadFile(image, model);
131+
finalImageUrl = await ctx.client.resolveImageInput(image, model);
139132
}
140133
}
141134

0 commit comments

Comments
 (0)