chat: fix system prompt accumulation across conversation turns#1076
Conversation
When using previous_response_id, the system prompt was being added as a ResponseItem.CreateSystemMessageItem in every turn's input list. Since OpenAI already holds the full conversation server-side, this caused the system prompt to accumulate in context on each turn and each tool-call leg. Fix: Set the system prompt via ResponseCreationOptions.Instructions in CreateResponseOptionsAsync instead. Per the Azure OpenAI API docs, Instructions is stateless across turns when using previous_response_id, so it is applied fresh each turn without accumulating. Both GetChatCompletionCore and GetChatCompletionStream now always pass only [ResponseItem.CreateUserMessageItem(prompt)] as input items. CreateResponseOptionsAsync is no longer static since it accesses _Options.
There was a problem hiding this comment.
Pull request overview
This PR updates the Azure OpenAI Responses API integration to prevent the system prompt from being repeatedly appended to the conversation when using previous_response_id, by moving the system prompt from input message items into ResponseCreationOptions.Instructions (applied per-call and not accumulated across turns).
Changes:
- Set the system prompt via
ResponseCreationOptions.InstructionsinCreateResponseOptionsAsync(with optional override). - Remove
ResponseItem.CreateSystemMessageItem(...)from both non-streaming and streaming request item lists. - Simplify
GetChatCompletionCoreto accept only the user prompt + response options.
|
@copilot resolve the merge conflicts in this pull request |
|
@copilot resolve the merge conflicts in this pull request |
Co-authored-by: BenjaminMichaelis <22186029+BenjaminMichaelis@users.noreply.github.com>
Resolved in commit Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
Handled in commit |
Problem
In both
GetChatCompletionStreamandGetChatCompletionCore, the system prompt was added as aResponseItem.CreateSystemMessageItem(...)in the input items list on every call. Whenprevious_response_idis used on follow-up turns, OpenAI already holds the full conversation server-side — including the system message from turn 1. Every subsequent turn (and every tool-call leg) was adding it again, causing the system prompt to accumulate in context.Fix
Move the system prompt to
ResponseCreationOptions.InstructionsinCreateResponseOptionsAsync, and remove the system message item from both call sites.Per the Azure OpenAI Responses API docs:
Instructionsis stateless across turns by design — applied fresh each call, never accumulated.Changes
CreateResponseOptionsAsync: Addedstring? systemPromptparameter; setsoptions.Instructionsfrom it (falling back to_Options.SystemPrompt). Changed fromstaticto instance method to access_Options.GetChatCompletionCore: RemovedsystemPromptparameter; input list is always[ResponseItem.CreateUserMessageItem(prompt)].GetChatCompletionStream: Same simplification — no system message item in input list.GetChatCompletion: PassessystemPrompttoCreateResponseOptionsAsyncinstead of downstream toGetChatCompletionCore.