fix(go): compat_oai plugin returns incomplete history when streaming - #5239
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request updates the generateStream and convertChatCompletionToModelResponse functions in go/plugins/compat_oai/generate.go to accept and utilize the ai.ModelRequest object, ensuring the request context is preserved in the model response. A review comment identifies that the signature change for convertChatCompletionToModelResponse breaks the generateComplete function, which was not updated in this PR. The feedback includes a code suggestion to fix the compilation error and improve error handling by wrapping errors with contextual information.
|
Thanks @MikeRez0! Can you get the CLA taken care of? In the mean time we'll get the review done. |
|
Already signed |
|
@apascal07 Hi, Alex! Can you review changes, please? I want to use stream mode in my project |
|
A few requested changes:
|
generateStream never set ModelResponse.Request, and
convertChatCompletionToModelResponse defaulted it to an empty
&ai.ModelRequest{}. Streaming responses therefore reported an empty
message list, so ModelResponse.History() returned only the model reply
and dropped the entire input conversation. The non-streaming path was
unaffected because generateComplete already assigned resp.Request.
Thread the originating request through generateStream and assign it
after conversion, mirroring generateComplete. Drop the misleading
default from the converter so it stays a pure conversion and callers
own the request.
Fixes genkit-ai#4683
Co-authored-by: Alex Pascal <apascal07@gmail.com>
5a584be to
66cb2ac
Compare
Changes fix this issue
#4683
Checklist (if applicable):