Drop garbled functions.<name>({...}) wrappers from replay history#515
Open
aj47 wants to merge 1 commit into
Open
Drop garbled functions.<name>({...}) wrappers from replay history#515aj47 wants to merge 1 commit into
aj47 wants to merge 1 commit into
Conversation
Closes the defensive-guard gap called out in #401: the model occasionally emits plain-text `functions.respond_to_user({...})` (or similar) alongside a real structured tool call. Persisting that text into conversation history replays it back to the model as if the user had pasted a tool transcript, which destabilises later turns. - Extend `GARBLED_TOOL_CALL_TEXT_REGEX` to match bare top-level `functions.<name>({` wrappers that show up without a `to=` prefix or a `[Calling tools: …]` placeholder. - Add `sanitizeAssistantContentForToolCalls` and use it at the persist site in `llm.ts` so garbled text is stripped only when real structured tool calls are present; plain prose alongside tool calls is preserved. - Cover both behaviours in `llm.continuation-guards.test.ts`. https://claude.ai/code/session_01RSWSRZsLGgkLdhZ67zfqzT
| if (hasStructuredToolCalls && isGarbledToolCallText(raw)) { | ||
| return "" | ||
| } | ||
| return raw |
Contributor
There was a problem hiding this comment.
Because isGarbledToolCallText matches substrings, sanitizeAssistantContentForToolCalls will drop the entire assistant content if it contains a functions.<name>({ snippet anywhere, even when there’s legitimate prose in the same message. This could unintentionally remove user-facing progress/explanation text whenever structured tool calls are present.
Severity: medium
🤖 Was this useful? React with 👍 or 👎
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the defensive-guard gap called out at the end of #401: when the model emits a plain-text
functions.respond_to_user({...})(or similar) wrapper alongside a real structured tool call, persisting that text into conversation history replays it back to the model as if the user had pasted a tool transcript. That's what produced the malformedfunctions.respond_to_user({ ... })assistant turns the issue documented.The primary "mobile branch endpoint" part of #401 is already implemented (remote-server
POST /v1/conversations/:id/branchatapps/desktop/src/main/remote-server.ts:6008and mobile clientbranchConversationatapps/mobile/src/lib/settingsApi.ts:1094), so this PR focuses on the related guard.What changed
apps/desktop/src/main/llm-continuation-guards.tsGARBLED_TOOL_CALL_TEXT_REGEXto match bare top-levelfunctions.<name>({wrappers (noto=prefix or[Calling tools: …]placeholder needed). The existing alternatives already coveredmulti_tool_use.*,to=functions.*, etc.; this adds the bare-wrapper case the issue called out.sanitizeAssistantContentForToolCalls(content, toolCalls)— returns""when structured tool calls are present AND the content matchesisGarbledToolCallText, otherwise passes content through unchanged. Plain prose alongside tool calls (e.g."Running the tests now.") is preserved.apps/desktop/src/main/llm.tssanitizeAssistantContentForToolCallsat the single persist site that recordsllmResponse.contentalongsidellmResponse.toolCalls(was line 3061). Garbled text now drops out before it reachesaddMessage/ disk / replay; the structured tool call itself is untouched.apps/desktop/src/main/llm.continuation-guards.test.tsisGarbledToolCallTextcase coveringfunctions.respond_to_user({...})andfunctions.execute_command({...}), plus a negative case for prose mentioning "functions" without the call+object shape.sanitizeAssistantContentForToolCallscase that asserts strip-on-tool-calls, pass-through without tool calls, prose preserved alongside tool calls, and empty/undefined input.Test plan
pnpm --filter @dotagents/desktop exec vitest run src/main/llm.continuation-guards.test.ts— 23/23 passpnpm --filter @dotagents/desktop exec vitest run src/main/llm.respond-to-user-history.test.ts— 47/47 passpnpm --filter @dotagents/desktop run typecheck:node— cleanmark_work_completetool call and garbledfunctions.respond_to_user({…})plain-text content, confirm the persisted assistant message has emptycontentand the next turn does not see the wrapper in replay history.Out of scope
#401work (a server-side branch endpoint reachable from mobile) is already done — this PR only closes the related defensive-guard gap mentioned in the same issue.apps/mobile/src/lib/openaiClient.ts:sanitizeMessagesForRequest) does not currently inspectcontent. If mobile reads back a contaminated server conversation, the desktop fix above already prevents new contamination at the source; remediating already-persisted bad history is a separate cleanup.https://claude.ai/code/session_01RSWSRZsLGgkLdhZ67zfqzT
Generated by Claude Code