Skip to content

fix: Gemini turn ordering and SSE streaming parser#723

Open
boxcee wants to merge 1 commit intoRightNow-AI:mainfrom
boxcee:fix/gemini-turn-ordering-and-sse-streaming
Open

fix: Gemini turn ordering and SSE streaming parser#723
boxcee wants to merge 1 commit intoRightNow-AI:mainfrom
boxcee:fix/gemini-turn-ordering-and-sse-streaming

Conversation

@boxcee
Copy link

@boxcee boxcee commented Mar 18, 2026

Three bugs fixed in the Gemini driver:

  1. Function call turn ordering: Gemini requires model turns with functionCall to be immediately followed by user turns with functionResponse. The agent loop could insert text-only turns between them (e.g. "[no response]", "Please continue"), causing INVALID_ARGUMENT 400 errors.

  2. First turn must be user: Gemini rejects conversations starting with a model turn. After session trimming or compaction, the first message could be a model turn with functionCall parts. Now prepends a synthetic user turn when needed.

  3. SSE streaming parser: The parser used \n\n as the SSE event delimiter but Gemini returns \r\n\r\n (HTTP standard). Since \r\n\r\n does not contain the substring \n\n, no events were ever parsed, causing 0 token responses and crash loops. Fixed by normalizing \r\n to \n before delimiter matching.

Also adds debug logging for turn structure, request/response bodies, and SSE stream diagnostics.

Summary

Changes

Testing

  • cargo clippy --workspace --all-targets -- -D warnings passes
  • cargo test --workspace passes
  • Live integration tested (if applicable)

Security

  • No new unsafe code
  • No secrets or API keys in diff
  • User input validated at boundaries

Three bugs fixed in the Gemini driver:

1. Function call turn ordering: Gemini requires model turns with
   functionCall to be immediately followed by user turns with
   functionResponse. The agent loop could insert text-only turns
   between them (e.g. "[no response]", "Please continue"),
   causing INVALID_ARGUMENT 400 errors.

2. First turn must be user: Gemini rejects conversations starting
   with a model turn. After session trimming or compaction, the
   first message could be a model turn with functionCall parts.
   Now prepends a synthetic user turn when needed.

3. SSE streaming parser: The parser used \n\n as the SSE event
   delimiter but Gemini returns \r\n\r\n (HTTP standard). Since
   \r\n\r\n does not contain the substring \n\n, no events were
   ever parsed, causing 0 token responses and crash loops.
   Fixed by normalizing \r\n to \n before delimiter matching.

Also adds debug logging for turn structure, request/response
bodies, and SSE stream diagnostics.
Copy link
Member

@jaberjaber23 jaberjaber23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bugs being fixed are real and important — the SSE \r\n issue and turn ordering constraint are genuine Gemini API problems.

Blocking issues:

  1. truncate_for_log will panic on multi-byte UTF-8 input. &s[..max_len] indexes by byte offset — if it falls in the middle of a multi-byte character (Japanese, emoji, non-ASCII error messages from Google), Rust panics. Fix with s.floor_char_boundary(max_len) (stable since 1.76).

  2. buffer.replace("\r\n", "\n") runs on the entire accumulated buffer every chunk, creating O(n*m) behavior. Move normalization to the chunk level: buffer.push_str(&chunk_str.replace("\r\n", "\n")) instead of appending then replacing the full buffer.

  3. No tests for the 70-line enforce_function_call_ordering function. This is critical LLM infrastructure — needs tests covering: functionCall with intervening text, orphaned functionCall, conversation starting with model turn, consecutive same-role merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants