Skip to content

feat(openclaw-plugin): add image preview URLs#2798

Draft
qin-ctx wants to merge 1 commit into
mainfrom
feat/openclaw-resource-preview-urls
Draft

feat(openclaw-plugin): add image preview URLs#2798
qin-ctx wants to merge 1 commit into
mainfrom
feat/openclaw-resource-preview-urls

Conversation

@qin-ctx

@qin-ctx qin-ctx commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • 为 OpenClaw 插件新增图片资源预览 URL 能力:当召回结果是 viking://resources/... 下的图片时,可调用 OpenViking /api/v1/content/preview_url 获取临时预览地址。
  • 新增隐藏开关 enableResourcePreviewUrls,默认 false;不开启时不会调用预览接口,也不会在 ov_search 表格中显示空的 preview_url 列,安装/setup 流程不暴露该配置。
  • 覆盖三条输出链路:auto-recall 注入上下文、memory_recall 工具输出、ov_search 表格与 details.resources[].preview_url
  • 补齐发布安装清单和 fallback 文件列表,确保新增 preview-url.ts 在发布包和远端安装路径中可用。

背景

OpenViking 服务新增了内容预览接口:

GET /api/v1/content/preview_url?uri=<OpenViking 文件 URI>

成功响应:

{
  "status": "ok",
  "result": {
    "preview_url": "https://tos.example.com/openviking/gallery/photo.png?X-Amz-Signature=***"
  },
  "time": 0.012
}

插件侧需要在召回到图片资源时,把该临时预览地址透出给模型或工具调用方。最终回答中是否展示 URL 仍由业务/模型回答层决定;OpenViking 插件只负责把 URL 放到可见上下文或工具输出中。

数据结构

1. 召回结果项

FindResultItem 新增可选字段:

export type FindResultItem = {
  uri: string;
  level?: number;
  abstract?: string;
  overview?: string;
  content?: string;
  category?: string;
  score?: number;
  match_reason?: string;
  preview_url?: string;
};

字段语义:

  • preview_url 是本次运行临时补充的预览地址。
  • 只用于工具输出 / auto-recall 注入上下文。
  • 不写入 recall trace,不作为长期记忆或资源元数据持久化。
  • 只有 enableResourcePreviewUrls: true 且命中图片资源时才会尝试补充。

2. 预览接口返回结构

export type PreviewUrlResult = {
  preview_url: string;
};

客户端新增方法:

async getPreviewUrl(uri: string, actorPeerId?: string): Promise<string>

它复用现有 request(),因此继续带上 API Key、租户头和 X-OpenViking-Actor-Peer

3. 配置结构

MemoryOpenVikingConfig 新增隐藏开关:

export type MemoryOpenVikingConfig = {
  enableResourcePreviewUrls?: boolean;
};

默认值:

{
  "enableResourcePreviewUrls": false
}

手工开启示例:

{
  "plugins": {
    "entries": {
      "openviking": {
        "config": {
          "mode": "remote",
          "baseUrl": "http://127.0.0.1:1933",
          "apiKey": "${OPENVIKING_API_KEY}",
          "recallTargetTypes": ["resource", "user", "agent"],
          "enableResourcePreviewUrls": true
        }
      }
    }
  }
}

该字段没有加入 uiHints,因此不会在安装/setup 流程中暴露。

处理流程

auto-recall

代码流程是:

const memories = pickMemoriesForInjection(processed, recallLimit, queryText, scoreThreshold, ...);

const memoriesWithPreview = cfg.enableResourcePreviewUrls
  ? await withPreviewUrls(memories, client, agentId)
  : memories;

const { lines: memoryLines } = await buildMemoryLinesWithBudget(
  memoriesWithPreview,
  (uri) => client.read(uri, agentId),
  {
    recallPreferAbstract,
    recallMaxInjectedChars: maxInjectedChars,
    includeUri: true,
  },
);

const block = buildRecallContextBlock(memoryLines);

withPreviewUrls() 的规则:

  • 只处理 viking://resources/ 开头的资源。
  • 只处理图片后缀:.png.jpg.jpeg.webp.gif.bmp.svg.avif
  • 请求预览 URL 前会去掉 fragment:viking://resources/gallery/photo.png#chunk-1 会用 viking://resources/gallery/photo.png 请求。
  • 预览接口失败时保留原结果,静默降级,不影响召回注入。

Claw 实际看到的上下文形态

这里不是把 JSON 字段直接给 Claw。FindResultItem.preview_url 是插件内部字段,最终会被 formatMemoryLine() 渲染成 <preview_url>...</preview_url> 文本。

buildRecallContextBlock(memoryLines) 实际结构是:

[
  "<relevant-memories>",
  "Source: openviking-auto-recall",
  "The following OpenViking memories may be relevant:",
  previewInstruction,
  ...memoryLines,
  "</relevant-memories>",
].filter(Boolean).join("\n")

因此 topK 多条结果会展开成多条 memoryLines。开启预览 URL 且第一条图片成功补到 URL 时,latest user message 前面会 prepend 类似内容:

<relevant-memories>
Source: openviking-auto-recall
The following OpenViking memories may be relevant:
If you reference an image result, copy its exact <preview_url> into the answer; do not invent or rewrite preview URLs.
- [resource:image]
  <uri>viking://resources/gallery/product-card.png#chunk-1</uri>
  <preview_url>https://tos.example.com/openviking/gallery/product-card.png?X-Amz-Signature=***</preview_url>
  Product card screenshot for checkout page.
- [resource]
  <uri>viking://resources/docs/checkout-spec.md#chunk-2</uri>
  Checkout page spec says the product card image should use a 4:3 crop.
- [preferences]
  <uri>viking://user/memories/answer-style</uri>
  User prefers concise Chinese answers with concrete examples.
</relevant-memories>

用户原始问题...

不开启 enableResourcePreviewUrls,或者没有图片资源成功补到 URL 时,结构保持原样,不会出现提示行和 <preview_url>

<relevant-memories>
Source: openviking-auto-recall
The following OpenViking memories may be relevant:
- [resource]
  <uri>viking://resources/gallery/product-card.png#chunk-1</uri>
  Product card screenshot for checkout page.
- [resource]
  <uri>viking://resources/docs/checkout-spec.md#chunk-2</uri>
  Checkout page spec says the product card image should use a 4:3 crop.
- [preferences]
  <uri>viking://user/memories/answer-style</uri>
  User prefers concise Chinese answers with concrete examples.
</relevant-memories>

用户原始问题...

memory_recall

memory_recall 也会在开关开启时对选中的图片资源补 preview_url,然后通过 buildMemoryLinesWithBudget() 输出。例如:

Found 1 memories:

- [resource:image] Reference product image.
  <preview_url>https://tos.example.com/openviking/gallery/photo.png?X-Amz-Signature=***</preview_url>

ov_search

ov_search 在开关开启且资源命中图片 URL 时,会在表格中按需增加 preview_url 列,并在结构化 details 中返回:

result.details.resources[0].preview_url

输出示例:

no  type      uri                                           level  score  preview_url                                                           abstract
1   resource  viking://resources/gallery/photo.png#chunk-1         0.91   https://tos.example.com/openviking/gallery/photo.png?X-Amz-Signature=***  Reference image

不开启时不会显示空列:

no  type      uri                                           level  score  abstract
1   resource  viking://resources/gallery/photo.png#chunk-1         0.91   Reference image

Test plan

  • npm test -- --run tests/ut/config.test.ts tests/ut/client.test.ts tests/ut/build-memory-lines.test.ts tests/ut/plugin-query-formatters.test.ts tests/ut/plugin-modules.test.ts
  • npm test -- --run tests/ut/manifest-contracts.test.ts tests/ut/setup-modules.test.ts tests/ut/setup-config-writer.test.ts tests/ut/setup-command.test.ts tests/ut/setup-cli.test.ts
  • npm run typecheck
  • npm run build
  • git diff --check

Add optional preview_url enrichment for recalled OpenViking image resources so agents can surface temporary frontend preview links when explicitly enabled.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant