Skip to content

chat: fix system prompt accumulation across conversation turns#1076

Merged
BenjaminMichaelis merged 3 commits into
mainfrom
agents/fix-system-prompt-accumulation
May 9, 2026
Merged

chat: fix system prompt accumulation across conversation turns#1076
BenjaminMichaelis merged 3 commits into
mainfrom
agents/fix-system-prompt-accumulation

Conversation

@BenjaminMichaelis

Copy link
Copy Markdown
Member

Problem

In both GetChatCompletionStream and GetChatCompletionCore, the system prompt was added as a ResponseItem.CreateSystemMessageItem(...) in the input items list on every call. When previous_response_id is used on follow-up turns, OpenAI already holds the full conversation server-side — including the system message from turn 1. Every subsequent turn (and every tool-call leg) was adding it again, causing the system prompt to accumulate in context.

Fix

Move the system prompt to ResponseCreationOptions.Instructions in CreateResponseOptionsAsync, and remove the system message item from both call sites.

Per the Azure OpenAI Responses API docs:

"When using along with previous_response_id, the instructions from a previous response will not be carried over to the next response. This makes it simple to swap out system (or developer) messages in new responses."

Instructions is stateless across turns by design — applied fresh each call, never accumulated.

Changes

  • CreateResponseOptionsAsync: Added string? systemPrompt parameter; sets options.Instructions from it (falling back to _Options.SystemPrompt). Changed from static to instance method to access _Options.
  • GetChatCompletionCore: Removed systemPrompt parameter; input list is always [ResponseItem.CreateUserMessageItem(prompt)].
  • GetChatCompletionStream: Same simplification — no system message item in input list.
  • GetChatCompletion: Passes systemPrompt to CreateResponseOptionsAsync instead of downstream to GetChatCompletionCore.

When using previous_response_id, the system prompt was being added as a
ResponseItem.CreateSystemMessageItem in every turn's input list. Since
OpenAI already holds the full conversation server-side, this caused the
system prompt to accumulate in context on each turn and each tool-call leg.

Fix: Set the system prompt via ResponseCreationOptions.Instructions in
CreateResponseOptionsAsync instead. Per the Azure OpenAI API docs,
Instructions is stateless across turns when using previous_response_id,
so it is applied fresh each turn without accumulating.

Both GetChatCompletionCore and GetChatCompletionStream now always pass
only [ResponseItem.CreateUserMessageItem(prompt)] as input items.
CreateResponseOptionsAsync is no longer static since it accesses _Options.
@BenjaminMichaelis BenjaminMichaelis marked this pull request as ready for review May 9, 2026 02:07
Copilot AI review requested due to automatic review settings May 9, 2026 02:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Azure OpenAI Responses API integration to prevent the system prompt from being repeatedly appended to the conversation when using previous_response_id, by moving the system prompt from input message items into ResponseCreationOptions.Instructions (applied per-call and not accumulated across turns).

Changes:

  • Set the system prompt via ResponseCreationOptions.Instructions in CreateResponseOptionsAsync (with optional override).
  • Remove ResponseItem.CreateSystemMessageItem(...) from both non-streaming and streaming request item lists.
  • Simplify GetChatCompletionCore to accept only the user prompt + response options.

Comment thread EssentialCSharp.Chat.Shared/Services/AIChatService.cs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@BenjaminMichaelis

Copy link
Copy Markdown
Member Author

@copilot resolve the merge conflicts in this pull request

@BenjaminMichaelis

Copy link
Copy Markdown
Member Author

@copilot resolve the merge conflicts in this pull request

Co-authored-by: BenjaminMichaelis <22186029+BenjaminMichaelis@users.noreply.github.com>

Copilot AI commented May 9, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Resolved in commit f6d976c by merging origin/main into this branch and fixing the AIChatService.cs conflict so both the system-prompt Instructions behavior and the latest MCP tool-call flow are preserved.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.hcaptcha.com
    • Triggering command: /home/REDACTED/work/EssentialCSharp.Web/EssentialCSharp.Web/EssentialCSharp.Web.Tests/bin/Release/net10.0/EssentialCSharp.Web.Tests /home/REDACTED/work/EssentialCSharp.Web/EssentialCSharp.Web/EssentialCSharp.Web.Tests/bin/Release/net10.0/EssentialCSharp.Web.Tests --server dotnettestcli --dotnet-test-pipe /tmp/477bf40cf99242e98a28b32bfb6e16ac (dns block)
  • api.pwnedpasswords.com
    • Triggering command: /home/REDACTED/work/EssentialCSharp.Web/EssentialCSharp.Web/EssentialCSharp.Web.Tests/bin/Release/net10.0/EssentialCSharp.Web.Tests /home/REDACTED/work/EssentialCSharp.Web/EssentialCSharp.Web/EssentialCSharp.Web.Tests/bin/Release/net10.0/EssentialCSharp.Web.Tests --server dotnettestcli --dotnet-test-pipe /tmp/477bf40cf99242e98a28b32bfb6e16ac (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI commented May 9, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Handled in commit f6d976c. I re-checked this branch against origin/main and there are no remaining merge conflicts.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.

@BenjaminMichaelis BenjaminMichaelis merged commit f52ca45 into main May 9, 2026
12 checks passed
@BenjaminMichaelis BenjaminMichaelis deleted the agents/fix-system-prompt-accumulation branch May 9, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants