Skip to content

refactor(go/plugins/googlegenai): migrate to the typed-config constructors - #5869

Open
apascal07 wants to merge 8 commits into
ap/go-with-config-constructorsfrom
ap/go-googlegenai-typed-config
Open

refactor(go/plugins/googlegenai): migrate to the typed-config constructors#5869
apascal07 wants to merge 8 commits into
ap/go-with-config-constructorsfrom
ap/go-googlegenai-typed-config

Conversation

@apascal07

@apascal07 apascal07 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Migrates the googlegenai plugin (Google AI and Vertex AI) to the typed-config constructors from #5849, the third PR of the train. Every primitive the plugin defines now declares its config type and receives it deserialized, so the four hand-rolled config mappers are gone: configFromRequest, imagenConfigFromRequest, toVeoParameters, and the embedder's pointer type assertion.

Before: each modality re-implements the same type switch.

func configFromRequest(input *ai.ModelRequest) (*genai.GenerateContentConfig, error) {
    switch config := input.Config.(type) {
    case genai.GenerateContentConfig:  return &config, nil
    case *genai.GenerateContentConfig: return config, nil
    case map[string]any:               return base.MapToStruct[genai.GenerateContentConfig](config)
    case nil:                          return &genai.GenerateContentConfig{}, nil
    default:                           return nil, core.NewPublicError(...)
    }
}
ai.NewModel(name, meta, fn)

After: the type parameter is inferred from the function and the framework fills it.

ai.NewTypedModel(name, &opts,
    func(ctx context.Context, input *ai.ModelRequest, config genai.GenerateContentConfig, cb ai.ModelStreamCallback) (*ai.ModelResponse, error) {
        return generate(ctx, client, name, input, config, cb)
    })
  • The config type follows the modality: image models take a genai.GenerateImagesConfig, Veo takes a genai.GenerateVideosConfig, the embedder takes a genai.EmbedContentConfig, and everything else (Gemini, Gemma, TTS, tuned endpoints, unrecognized names) speaks generateContent and takes a genai.GenerateContentConfig.
  • The config travels by value from the model function down, so toGeminiRequest amends the request's own copy. The old pointer path let a Veo request write its NumberOfVideos default into the caller's config struct.
  • The media-download middleware wraps a closure over that copy, leaving the built-in chain order unchanged.
  • newModel, newVeoModel, and newEmbedder return the concrete *ai.ModelAction, *ai.BackgroundModelAction, and *ai.EmbedderAction, so listActions and ResolveAction no longer assert their way back to api.Action. The exported DefineModel/DefineEmbedder methods keep their interface returns.
  • The Veo start/check helper actions move from the deprecated core.NewAction to core.NewActionOf, which turns the check action's description into a field instead of a metadata entry.

Config schemas are now enforced

The plugin supplies ConfigSchema explicitly (the SDK structs need a custom reflector to avoid recursion panics, plus curated descriptions for the dev UI), and the typed constructors wire that schema into the request input schema, which is validated on every call. Two consequences:

The curated schema hides fields the plugin manages, and hiding is a presentation choice, not a narrowing of what callers may send. Left alone, candidateCount: 1 would start failing even though the plugin pins it to 1 itself, and systemInstruction, cachedContent, responseSchema, responseMimeType, responseJsonSchema, and tools[].functionDeclarations would be rejected as unknown properties instead of reaching the plugin's own check, which names the primitive to use instead (ai.WithSystemPrompt, ai.WithCacheTTL, ai.WithOutputType, ai.WithTools). So an object that loses a property is reopened with additionalProperties: true. Objects that kept every field stay strict, and the Imagen and Veo schemas, which hide nothing, are untouched.

The three modality schemas are now built once at package level instead of being reflected per model. GetModelOptions was doing that work on every call, which listActions makes once per model.

Fixes

Embedder options were read with a *genai.EmbedContentConfig type assertion, so a map[string]any (what the dev UI and every other JSON caller sends) was silently dropped and the embed call went out with no config at all. The framework now deserializes it.

Verification

Go 1.26.3, go test ./plugins/googlegenai/...: passes, 56 top-level tests and 222 including subtests, 2 skipped (they need live credentials), no failures.

New coverage: the config schema each action advertises is pinned per modality, every field the curated schemas hide still reaches the plugin's own error naming the Genkit primitive to use, and a map[string]any config reaches the embedder deserialized.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the googlegenai plugin to leverage Genkit's typed models, embedders, and background models, delegating request configuration validation and deserialization to the framework. This simplifies model initialization and removes redundant manual configuration conversion logic. The review feedback focuses on performance optimization, recommending that large configuration structs (such as genai.GenerateImagesConfig and genai.GenerateContentConfig) be passed by pointer rather than by value to avoid unnecessary copying.

if mt == ModelTypeImagen {
return ai.NewTypedModel(api.NewName(provider, name), &opts,
func(ctx context.Context, input *ai.ModelRequest, config genai.GenerateImagesConfig, cb ai.ModelStreamCallback) (*ai.ModelResponse, error) {
return generateImage(ctx, client, name, input, config, cb)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Pass config by pointer to generateImage to avoid copying the large genai.GenerateImagesConfig struct.

Suggested change
return generateImage(ctx, client, name, input, config, cb)
return generateImage(ctx, client, name, input, &config, cb)

}

gcc, err := toGeminiRequest(input, cache, model)
gcc, err := toGeminiRequest(input, config, cache, model)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Pass config by pointer to toGeminiRequest to avoid copying the large genai.GenerateContentConfig struct.

Suggested change
gcc, err := toGeminiRequest(input, config, cache, model)
gcc, err := toGeminiRequest(input, &config, cache, model)

Comment on lines +299 to +300
func toGeminiRequest(input *ai.ModelRequest, config genai.GenerateContentConfig, cache *genai.CachedContent, modelName ...string) (*genai.GenerateContentConfig, error) {
gcc := &config

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Change the signature of toGeminiRequest to accept config as a pointer *genai.GenerateContentConfig instead of a value. This avoids copying the large configuration struct on every request.

Suggested change
func toGeminiRequest(input *ai.ModelRequest, config genai.GenerateContentConfig, cache *genai.CachedContent, modelName ...string) (*genai.GenerateContentConfig, error) {
gcc := &config
func toGeminiRequest(input *ai.ModelRequest, config *genai.GenerateContentConfig, cache *genai.CachedContent, modelName ...string) (*genai.GenerateContentConfig, error) {
gcc := config

if err != nil {
return nil, err
}
return toGeminiRequest(input, config, cache, modelName...)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Pass config by pointer to toGeminiRequest to match the updated signature.

Suggested change
return toGeminiRequest(input, config, cache, modelName...)
return toGeminiRequest(input, &config, cache, modelName...)

client *genai.Client,
model string,
input *ai.ModelRequest,
gic genai.GenerateImagesConfig,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Change the signature of generateImage to accept gic as a pointer *genai.GenerateImagesConfig instead of a value to avoid copying the large struct.

Suggested change
gic genai.GenerateImagesConfig,
gic *genai.GenerateImagesConfig,

}

resp, err := client.Models.GenerateImages(ctx, model, userPrompt, gic)
resp, err := client.Models.GenerateImages(ctx, model, userPrompt, &gic)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Pass gic directly instead of &gic since it is now a pointer.

Suggested change
resp, err := client.Models.GenerateImages(ctx, model, userPrompt, &gic)
resp, err := client.Models.GenerateImages(ctx, model, userPrompt, gic)

…as once

Reflecting the genai SDK config structs is expensive, and every model of
a modality advertises the same read-only schema, so the three schemas
move to package-level values shared by the default options and by
GetModelOptions instead of being rebuilt per model. ModelType gains an
unexported configSchema accessor that resolves unrecognized names to the
Gemini schema, since those speak generateContent too; the exported
DefaultConfig is unchanged.
…validation

The curated schema hides fields the plugin manages (systemInstruction,
cachedContent, responseSchema, responseMimeType, responseJsonSchema,
tools[].functionDeclarations, candidateCount) so the dev UI does not
offer them. That is a presentation choice, but once the typed
constructors wire the config schema into the request input schema it is
enforced on every call, and a hidden field would be rejected as an
unknown property before the plugin could answer with the primitive to
use instead. candidateCount would fare worse: setting it to 1 is legal
today and would start failing.

Objects that lost a property are now reopened with additionalProperties:
true. Everything else stays strict, so mistyped values and typos in
untouched objects are still caught, and the Imagen and Veo schemas,
which hide nothing, are unaffected.
NewTypedEmbedder hands the embedder function a genai.EmbedContentConfig
the framework deserialized from the request, replacing a pointer type
assertion that silently dropped anything else: a map, which is what the
dev UI and every other JSON caller sends, produced an embed call with no
config at all. The explicit ConfigSchema is gone too, since inference
over the same type produces it.

newEmbedder returns the concrete *ai.EmbedderAction, so the caller no
longer asserts its way back to api.Action.
NewTypedBackgroundModel hands the start function a
genai.GenerateVideosConfig, replacing toVeoParameters, whose type switch
duplicated what the framework now does. It also copies: the old pointer
case handed the user's own config to the SDK and then wrote the
NumberOfVideos default into it.

The flat TypedBackgroundModelOptions carries the model descriptor, so
the video config schema is advertised the same way as before, with a
fallback for callers that do not supply one. The test that covered the
deleted conversion now pins the config type the model is defined with.
… and imagen models

NewTypedModel infers the config type from the model function, so
newModel picks the type per modality: image models take a
genai.GenerateImagesConfig, everything else speaks generateContent and
takes a genai.GenerateContentConfig. That deletes configFromRequest and
imagenConfigFromRequest, whose type switches are now the framework's
job, and it pins the advertised schema to the type the function
receives, so the two can no longer drift.

The config travels by value from there down: toGeminiRequest amends the
request's own copy rather than a pointer whose provenance the reader has
to trace, and the media-download middleware wraps a closure over that
copy so the built-in chain order is unchanged.

The tests that covered the deleted conversions now go through the
framework's deserialization before calling toGeminiRequest, which is
what a real request does.
…ewActionOf

The registry-free core.NewAction is deprecated in favor of NewActionOf,
which leads with the action type and takes an options struct, so the
check action's description becomes a field instead of a metadata entry.
…rtise

The typed constructors turn the advertised config schema into something
enforced on every request, so these tests check both directions at the
action boundary: the configs callers send today are accepted, including
the fields hidden from the dev UI, and configs meant for another
modality are not. They also cover the embedder's options slot, which
used to ignore anything that was not a pointer.
… plugin's errors

Walks the whole path for each field hidden from the dev UI: past input
validation, through the framework's deserialization, into the check that
names the primitive to use instead. Without this the two halves are
pinned separately and nothing catches a schema change that turns a
targeted error into a schema violation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant