fix(openai-api): report real token usage in chat completions - #8
Open
fancyboi999 wants to merge 1 commit into
Open
fix(openai-api): report real token usage in chat completions#8fancyboi999 wants to merge 1 commit into
fancyboi999 wants to merge 1 commit into
Conversation
Contributor
Author
|
@syzsunshine219 maintainer follow-up: this PR remains clean and mergeable, and current No conflict rebuild is needed here. Could you review it when convenient? |
The OpenAI-compatible /v1/chat/completions endpoint always returned
usage: {prompt_tokens: 0, completion_tokens: 0, total_tokens: 0},
because chatCompletionResponse() hardcoded zeros instead of reading
the usage the agent runtime already tracks (AgentRunner accumulates
per-iteration usage in runner.ts, and AgentLoop.runAgentLoop already
normalizes it into `lastUsage`, but that value never left the loop).
Thread the turn's accumulated usage through the existing pipeline
instead of introducing a new channel: runAgentLoop now returns it
alongside its other turn outputs, TurnContext carries it, and
assembleOutbound attaches it to OutboundMessage.metadata.usage. The
API entrypoint reads it from there, sums it across the empty-response
retry path, and reports it in the JSON response. Streaming responses
now also honor stream_options.include_usage by emitting a trailing
usage-only chunk (choices: []), matching the OpenAI SSE convention.
fancyboi999
force-pushed
the
fix/openai-api-usage
branch
from
August 4, 2026 09:08
0ec980c to
6c8cc1b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
memmy serve的 OpenAI 兼容 API/v1/chat/completions返回的usage恒为{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}。把 memmy 当 OpenAI 端点接入其他工具时计量完全失真。根因
src/entrypoints/openai-like-api/server.ts的chatCompletionResponse()硬编码了三个 0。真实 usage 一直存在：AgentRunner.run()跨工具调用迭代累计 usage，AgentLoop.runAgentLoop()也做了归一化（lastUsage），但从未随 turn 结果传到 API 入口。修复
沿既有的 per-turn 对象传递 usage，不依赖共享的
lastUsage字段，避免并发 turn 竞争：runAgentLoop()在v1.0.5已有的errorCategory之后追加本 turn 的归一化 usageTurnContext.usage由stateRun()填充，assembleOutbound()再挂到OutboundMessage.metadata.usagestream_options.include_usage，设置时在[DONE]前追加choices: []的 usage 尾块；未设置时行为不变冲突处理
本分支已 rebase 到当前
v1.0.5。唯一冲突来自runAgentLoop()返回元组：base 新增了errorCategory，本 PR 新增了usage。最终同时保留两者，顺序为errorCategory第 7 项、usage第 8 项，避免改变 base 现有调用方语义。验证
npm run typecheck：通过npm run build：通过schema-validation.test.ts的旧 fallback 预期；在未包含本 PR 改动的纯v1.0.5@80b07d7上可独立复现。v1.0.5已通过 fix(config): fail loud on invalid config instead of silently using defaults #7 改为 invalid config fail-loud，但该旧测试仍期待 fallback；本 PR 不顺带修改这项无关基线问题。usage={"prompt_tokens":6304,"completion_tokens":5,"total_tokens":6309}，总数校验通过include_usage: true：finish chunk 之后、[DONE]之前收到choices: []的 usage chunk；6341+5=6346include_usage：0 个 usage chunk，保留兼容行为未覆盖说明
processSystemMessage()（cron/系统通道，不经 OpenAI API）未接 usage，属于有意边界