Migrate Tier-2 LLM to Vertex AI with GCP service-account auth - #23
Open
KevinRusu wants to merge 1 commit into
Open
Migrate Tier-2 LLM to Vertex AI with GCP service-account auth#23KevinRusu wants to merge 1 commit into
KevinRusu wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Migrates the Tier-2 LLM integration from Gemini API-key auth to Vertex AI using GCP Application Default Credentials / Cloud Run service-account auth, and introduces a Cloud Build → Artifact Registry → Cloud Run deployment pipeline for the API.
Changes:
- Switch
GeminiProviderto instantiategoogle-genaiin Vertex mode usingGOOGLE_CLOUD_PROJECT/GOOGLE_CLOUD_LOCATIONand fail fast on missing config. - Add Cloud Run deployment pipeline via
api/cloudbuild.yamland update container startup to honor Cloud Run’s injected$PORT. - Update docs/examples and add a regression test to ensure missing Vertex config raises
ProviderConfigError.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| api/tests/test_provider_config.py | Adds a unit test asserting missing GOOGLE_CLOUD_PROJECT fails fast with ProviderConfigError. |
| api/requirements.txt | Pins google-genai to 2.8.0 for reproducible builds. |
| api/README.md | Updates env var docs for Vertex configuration (but currently has a markdown formatting issue and missing sections claimed in the PR description). |
| api/providers/gemini_provider.py | Migrates provider auth/config from API key to Vertex AI client construction. |
| api/main.py | Updates startup comment to reference Vertex config failure behavior. |
| api/Dockerfile | Changes CMD to use ${PORT} for Cloud Run compatibility. |
| api/cloudbuild.yaml | Adds Cloud Build pipeline to build/push/deploy to Cloud Run with required env vars. |
| api/.env.example | Removes GEMINI_API_KEY and adds Vertex env vars. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - `GET /health` — liveness probe, no auth, no rate limit. | ||
|
|
||
| > **Tip:** Keep `USE_MOCK=true` while developing. Switch to `USE_MOCK=false` only when you need a real Gemini response. | ||
| ``` No newline at end of file |
Comment on lines
+32
to
35
| | `USE_MOCK` | No (default: `true`) | Skip Gemini and use a deterministic mock. No credentials needed. | | ||
| | `GOOGLE_CLOUD_PROJECT` | Only if `USE_MOCK=false` | GCP project for Vertex AI (`b2-beacon1`). Auth is via ADC / the runtime service account — no API key. | | ||
| | `GOOGLE_CLOUD_LOCATION` | No (default: `us-east1`) | Vertex AI region. | | ||
| | `GEMINI_MODEL` | No (default: `gemini-2.5-flash-lite`) | Gemini model name. Override to migrate to a newer model without a code change. | |
Comment on lines
+15
to
+17
| # Shell-form CMD so ${PORT} expands: Cloud Run injects PORT and expects the | ||
| # container to listen on it; locally PORT is unset and we default to 3000. | ||
| CMD uvicorn main:app --host 0.0.0.0 --port ${PORT:-3000} --proxy-headers |
Comment on lines
+25
to
+38
| # Published extension origin, e.g. chrome-extension://<id>. Empty accepts all | ||
| # origins (dev only). A single origin — commas would clash with the env-var | ||
| # delimiter below. | ||
| _ALLOWED_ORIGINS: "" | ||
|
|
||
| steps: | ||
| # 1. Build the container image. | ||
| - name: gcr.io/cloud-builders/docker | ||
| args: | ||
| - build | ||
| - -t | ||
| - ${_REGION}-docker.pkg.dev/${PROJECT_ID}/${_AR_REPO}/${_SERVICE}:${_TAG} | ||
| - . | ||
|
|
Comment on lines
+58
to
+60
| - --allow-unauthenticated | ||
| - --port=3000 | ||
| - --set-env-vars=USE_MOCK=false,GOOGLE_CLOUD_PROJECT=${PROJECT_ID},GOOGLE_CLOUD_LOCATION=${_REGION},GEMINI_MODEL=gemini-2.5-flash-lite,ALLOWED_EXTENSION_ORIGINS=${_ALLOWED_ORIGINS},FORWARDED_ALLOW_IPS=* |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GEMINI_API_KEYto Vertex AI with GCP service-account auth, per b2's no-API-keys policy (Gemini access is now granted through Vertex, authenticated by thebeacon-app-serviceservice account).gcloud auth application-default login --impersonate-service-account=...) and the attached service account automatically on Cloud Run.cloudbuild.yaml) — deployment was previously undefined; only aDockerfileexisted.AnalyzeResponse(0–10safety_score,label,action,reason) is reused as the Vertex structured-output schema, so the safety-scale contract carries over untouched.Changes by area
API (
api/)providers/gemini_provider.py—GeminiProvider.__init__now buildsgenai.Client(vertexai=True, project=..., location=...)instead ofgenai.Client(api_key=...); readsGOOGLE_CLOUD_PROJECT/GOOGLE_CLOUD_LOCATIONand raisesProviderConfigErrorif the project isn't set. The prompt, system instruction, andanalyze()call are unchanged..env.example— droppedGEMINI_API_KEY; addedGOOGLE_CLOUD_PROJECT/GOOGLE_CLOUD_LOCATIONplus the ADC login command as a comment.requirements.txt— pinnedgoogle-genaito the exact tested version (2.8.0) instead of a loose>=1.0, for reproducible Cloud Run builds.Dockerfile—CMDnow honors Cloud Run's injected$PORT(--port ${PORT:-3000}), defaulting to 3000 locally.cloudbuild.yaml(new) — build → push to Artifact Registry →gcloud run deployasbeacon-app-service, with the Vertex env vars wired in. Public endpoint by design (no client secret; abuse control is the origin allowlist + rate limit).main.py— updated a stale comment referencingGEMINI_API_KEY; startup behavior (fail fast on missing config) is unchanged.tests/test_provider_config.py(new) — asserts that constructingGeminiProviderwithoutGOOGLE_CLOUD_PROJECTraisesProviderConfigError. No network involved.README.md— new local-auth step, updated environment-variable table, and a "Deploy (Cloud Run)" section with IAM prerequisites.Extension (
extension/)Testing done
pytest api/tests/— 18/18 pass (17 pre-existing + the new provider-config test), all againstMockProvider(USE_MOCK=true), so none require credentials.google-genai==2.8.0SDK accepts thevertexai/project/locationclient kwargs, and that the missing-project guard fires correctly before any network call.roles/iam.serviceAccountTokenCreatorfor the local dev account onbeacon-app-service