feat(py): support Google AI Interactions and background models (Veo, Deep Research, Antigravity, Lyria) by huangjeff5 · Pull Request #5806 · genkit-ai/genkit

huangjeff5 · 2026-07-23T19:01:35Z

Adds support for various background models on Interactions API.

Also adds support for Veo on the legacy API (Veo isn't on Interactions API).

…) patch Includes: Track 0 generate plumbing, Interactions port, Veo LRO background model, Deep Research/Antigravity/Lyria, samples, DR ModelRef fix, Veo done normalize, lyria-* resolve, requires_action fallthrough (match JS), failed→done=True. Live smoke: Veo + Deep Research generate_operation paths PASS. Closed GitHub PR #5798 in favor of this patch for review.

gemini-code-assist

Code Review

This pull request introduces support for Google AI Interactions-backed models (such as Deep Research, Antigravity, and Google AI Lyria) in the Genkit Google Genai Python plugin, adding a raw HTTP client, type definitions, and converters. It also refactors background model registration and polling logic, aligning Veo and Deep Research models with Genkit's background model architecture. The review feedback highlights several opportunities to improve robustness and code quality, including avoiding using exceptions for successful control flow, ensuring recursive schema cleaning for nested lists, explicitly mapping HTTP 401 and 403 status codes, recursively normalizing nested Pydantic models in configuration helpers, and defensively handling malformed data URLs.

Drop the hand-rolled Interactions client/types, bump to google-genai 2.x, and tighten config/error/FinishReason handling from PR review.

…a sample

…d deep research samples

…i_name registration pattern

…archy and metadata persistence

huangjeff5 · 2026-07-27T04:38:23Z



+@pytest.mark.asyncio
+async def test_generate_echoes_normalized_effective_config(mocker: MockerFixture) -> None:


Ideally, we don't mutate the original ModelRequest here.

huangjeff5 · 2026-07-27T04:39:13Z

+
+@pytest.mark.asyncio
+async def test_generate_echoes_injected_modalities_when_config_omitted(
+    mocker: MockerFixture,


Is this because response_modalities is part of the default config? Need more context

- Interactions: drop shared/ephemeral genai.Client reuse; raw HTTP via get_cached_client (create/get/cancel) - Non-mutating config partition_keys; Lyria rest passthrough - Wave2 review fixes: ConfigT reuse, api_key_from_operation, don't mutate ModelRequest, antigravity env preflight removed - Added _interactions/client.py and interactions_client_test.py

…actions/converters.py Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

cabljac · 2026-07-30T10:51:59Z

Hey Jeff - as discussed, here's the split, stacked so each slice reviews independently:

feat(py): core support for background model operations and typed model refs #5854 - core background-operation plumbing + typed model refs (base)
feat(py/plugins/google-genai): raw Interactions HTTP client, converters, and options #5855 - Interactions HTTP client, converters, options
feat(py/plugins/google-genai): Deep Research, Antigravity, and Lyria interaction models #5856 - Deep Research / Antigravity / Lyria models
refactor(py/plugins/google-genai): move Veo to background model actions #5857 - Veo -> background model actions
feat(py/plugins/google-genai): wire Interactions models into GoogleAI plugin #5858 - GoogleAI plugin wiring
feat(py/plugins/google-genai): finish_message for image models and effective config echo #5859 - gemini finish_message (parallel, based on feat(py): core support for background model operations and typed model refs #5854)

Recombining the stack reproduces this branch's tree byte-for-byte, and you're co-author on every commit. google.py was the only file that had to split across two PRs (Veo hunks in #5857, wiring in #5858). Breaking changes are called out per-PR.

Closing this one as superseded by the stack.

…l refs Core plumbing extracted from #5806: FinishReason.ABORTED (schema + typing), cancel-operation action kind and registry lookup, background operation lifecycle updates in _background.py, generate_operation/check_operation plumbing in _ai, and a config_schema parameter on model_ref. Co-authored-by: Jeff Huang <huangjeff@google.com>

…fective config echo Gemini slice extracted from #5806: normalizes finish reasons (NO_IMAGE, IMAGE_OTHER), synthesizes finish_message when image models return no media so empty results are not silent successes, propagates finish_message onto candidates, echoes the normalized effective config on response.request for debugging, and drops the retired gemini-2.5-flash-image-preview registration. Co-authored-by: Jeff Huang <huangjeff@google.com>

… plugin Wiring slice extracted from #5806: registers the Deep Research, Antigravity, and Lyria catalogs in GoogleAI init/resolve/list_actions (including cancel-operation routing), threads plugin-level ClientOptions and API key into the Interactions models, maps genai errors in Imagen, and adds the google-genai-deep-research sample. Co-authored-by: Jeff Huang <huangjeff@google.com>

…fective config echo Gemini slice extracted from #5806: normalizes finish reasons (NO_IMAGE, IMAGE_OTHER), synthesizes finish_message when image models return no media so empty results are not silent successes, propagates finish_message onto candidates, echoes the normalized effective config on response.request for debugging, and drops the retired gemini-2.5-flash-image-preview registration. Co-authored-by: Jeff Huang <huangjeff@google.com>

github-project-automation Bot added this to Genkit Backlog Jul 23, 2026

github-actions Bot added docs Improvements or additions to documentation python Python config root labels Jul 23, 2026

gemini-code-assist Bot reviewed Jul 23, 2026

View reviewed changes

huangjeff5 commented Jul 23, 2026

View reviewed changes

Comment thread py/packages/genkit-google-genai/src/genkit_google_genai/_interactions/__init__.py Outdated

huangjeff5 commented Jul 23, 2026

View reviewed changes

Comment thread py/packages/genkit-google-genai/src/genkit_google_genai/_interactions/client.py Outdated

huangjeff5 commented Jul 23, 2026

View reviewed changes

Comment thread py/packages/genkit-google-genai/src/genkit_google_genai/models/veo.py Outdated

huangjeff5 commented Jul 23, 2026

View reviewed changes

Comment thread py/packages/genkit-google-genai/src/genkit_google_genai/models/interactions_utils.py Outdated

huangjeff5 commented Jul 23, 2026

View reviewed changes

Comment thread py/packages/genkit-google-genai/src/genkit_google_genai/models/antigravity.py Outdated

huangjeff5 commented Jul 23, 2026

View reviewed changes

Comment thread py/packages/genkit-google-genai/src/genkit_google_genai/_interactions/types.py Outdated

jeffdh5 mentioned this pull request Jul 23, 2026

docs(py): upgrade plugin docstrings to landing-page hero style with inline return shapes #5650

Merged

fix(py): use google-genai Interactions SDK and address review

72281c3

Drop the hand-rolled Interactions client/types, bump to google-genai 2.x, and tighten config/error/FinishReason handling from PR review.

github-actions Bot added the tooling label Jul 24, 2026

huangjeff5 added 14 commits July 23, 2026 20:53

fix(py): remove unnecessary _interactions/__init__.py file

da4ca4f

fix(py): remove legacy dict fallback from Veo response extraction

d2555db

fix(py): remove unnecessary root= keyword args in Part calls

22676ce

refactor(py): simplify _interactions_client_options property extraction

8bf9f2c

sample(py): inline operation polling loop in google-genai-media sample

9a0bebd

revert(git): restore .gitignore to main version

d8945bd

refactor(py): rename googleai_lyria.py to lyria.py for consistency

93fcc19

refactor(py): remove GoogleAILyria aliases in favor of clean Lyria names

c1e9f81

refactor(samples): pass VeoConfigSchema directly in google-genai-medi…

6fd6c70

…a sample

refactor(samples): eliminate helper function indirections in media an…

8841756

…d deep research samples

refactor(samples): return media URLs and report text directly from flows

272126a

refactor(py): standardize list_known_deep_research_models and googlea…

0b9449a

…i_name registration pattern

refactor(py): simplify _response_raw_payload in veo.py

64b6e10

docs(py): add inline documentation explaining options resolution hier…

fe597ad

…archy and metadata persistence