feat(gcp-to-aws): live discovery via the authenticated gcloud CLI (stacked on #148)

.checkov.yaml

@@ -11,3 +11,8 @@
 # their purpose. They are never deployed.
 skip-path:
   - migrate/plugins/migration-to-aws/skills/tf-best-practices/fixtures/terraform-policy
+  # Live-discovery replay fixture: deliberately models a startup's EXISTING,
+  # imperfect GCP infrastructure (the migration SOURCE, never deployed). Its
+  # drift vs the canned gcloud captures is the test — sanitizing it to satisfy
+  # Checkov would change the source-provider state the fixture exists to model.
+  - migrate/plugins/migration-to-aws/fixtures/gcp-live-capture/workspace-terraform

migrate/README.md

@@ -92,6 +92,7 @@ After installation, just describe what you want to migrate:
 **GCP migrations:**
 
 - "Migrate my GCP infrastructure to AWS"
+- "Discover my GCP project and estimate AWS costs"
 - "Move my Cloud Run services to Fargate"
 - "Migrate my OpenAI app to Amazon Bedrock"
 - "Estimate AWS costs for my GCP workload"
@@ -127,6 +128,10 @@ GCP/Heroku migrations write a `.migration/<session>/` directory; agent-advisor w
 
 **Live Heroku discovery — how it works:** No Terraform or exports needed. If `heroku login` works in your terminal, just ask — the agent requests your consent, then inventories your account using read-only list/info CLI commands. It captures app names, dyno types, add-on plans and prices, domains, pipelines, and config var **key names only**. It never reads config var values, credentials, or your API token, and never runs a command that creates, changes, or deletes anything. If you also have `heroku_*` Terraform, the agent cross-checks it against your live account and reports drift.
 
+**Live GCP discovery — how it works:** No Terraform or exports needed. If `gcloud auth login` works in your terminal, just ask — the agent confirms the target project and requests your consent, then inventories it using read-only list/describe commands. It captures resource names, types, regions, sizing, network topology, and env var **names only** — never env var values, secret values, database contents, or access tokens, and never a command that creates, changes, or deletes anything. If you also have Terraform, the agent cross-checks it against your live project and reports drift. (AI/agentic detection still needs your application code.)
+
+**Live Heroku discovery — how it works:** No Terraform or exports needed. If `heroku login` works in your terminal, just ask — the agent requests your consent, then inventories your account using read-only list/info CLI commands. It captures app names, dyno types, add-on plans and prices, domains, pipelines, and config var **key names only**. It never reads config var values, credentials, or your API token, and never runs a command that creates, changes, or deletes anything. If you also have `heroku_*` Terraform, the agent cross-checks it against your live account and reports drift.
+
 ## What It Detects
 
 ### GCP → AWS
@@ -216,8 +221,9 @@ GCP/Heroku migrations write a `.migration/<session>/` directory; agent-advisor w
 
 - Claude Code >=2.1.29, Codex (latest), or [Cursor >= 2.5](https://cursor.com/changelog/2-5)
 - AWS CLI configured with appropriate credentials
-- At least one input source: an authenticated Heroku CLI (Heroku migrations), Terraform files, application code, or billing data
-- **For GCP AI/agentic migration:** Application source code is required (billing/IaC alone cannot detect agent architecture)
+- At least one input source: an authenticated `gcloud` or `heroku` CLI (live discovery), Terraform files, application code, or billing data
+- **For GCP infrastructure migration:** an authenticated `gcloud` CLI (recommended — live, read-only discovery with your consent, with drift detection against any Terraform found) or Terraform files / billing exports
+- **For GCP AI/agentic migration:** Application source code is required (billing/IaC/live discovery alone cannot detect agent architecture)
 - **For Heroku migration:** an authenticated Heroku CLI (recommended — live, read-only discovery with your consent) or Terraform files with `heroku_*` resources (Procfile/app.json supplements but cannot stand alone). When both are available, live data is authoritative for current state and Terraform drift is surfaced.
 - **For Vercel migration:** repo access with a locally-runnable `next build`, plus a Vercel API token, are both required — the assessment does not run on partial Tier 1 inputs. Vercel tokens can't be permission-scoped to read-only, so scope by resource instead (project-scoped when one project is in scope), pick a short expiration, and revoke after the assessment; the skill only ever issues read (GET) requests, enforced by its capture-step endpoint whitelist
 - **For agent-advisor:** `uv` (for deterministic scoring); application source code when deploying/migrating existing agents (an idea-only run needs no code)

migrate/plugins/migration-to-aws/README.md

@@ -104,7 +104,7 @@ ln -s "$(pwd)" ~/.cursor/plugins/local/migration-to-aws
 
 ### Workflow
 
-1. **Discover** — Scan Terraform files, application code, and/or billing data — or, for Heroku, inventory your account live via the authenticated Heroku CLI (read-only, consent-gated). Detects infrastructure resources, AI models, agentic frameworks, tools, and orchestration patterns.
+1. **Discover** — Scan Terraform files, application code, and/or billing data — or inventory your GCP project or Heroku account live via the authenticated `gcloud`/`heroku` CLI (read-only, consent-gated, with drift detection against any Terraform found). Detects infrastructure resources, AI models, agentic frameworks, tools, and orchestration patterns.
 2. **Clarify** — Ask targeted questions about migration preferences, AI priorities, agentic migration approach, database sizing, and timeline.
 3. **Design** — Map source services to AWS equivalents. For AI workloads: select Bedrock models with honest pricing comparison. For agentic workloads: design AgentCore Harness config or Strands architecture.
 4. **Estimate** — Calculate monthly AWS costs using real-time pricing data. Compare to current spend.
@@ -221,10 +221,23 @@ See [skills/agent-advisor/SKILL.md](skills/agent-advisor/SKILL.md) for the full
 
 - Claude Code >=2.1.29, Codex (latest), or [Cursor >= 2.5](https://cursor.com/changelog/2-5)
 - AWS CLI configured with appropriate credentials
-- At least one input source: an authenticated Heroku CLI (Heroku migrations), Terraform files, application code, or billing data
-- **For GCP AI/agentic migration:** Application source code is required (billing/IaC alone cannot detect agent architecture)
+- At least one input source: an authenticated `gcloud` or `heroku` CLI (live discovery), Terraform files, application code, or billing data
+- **For GCP infrastructure migration:** an authenticated `gcloud` CLI (recommended — live, read-only discovery with your consent, with drift detection against any Terraform found) or Terraform files / billing exports
+- **For GCP AI/agentic migration:** Application source code is required (billing/IaC/live discovery alone cannot detect agent architecture)
 - **For Heroku migration:** an authenticated Heroku CLI (recommended) or Terraform files with `heroku_*` resources (Procfile/app.json supplements but cannot stand alone)
 
+### Live GCP discovery — how it works
+
+No Terraform or exports needed. If `gcloud auth login` works in your terminal, just
+ask your agent to migrate ("Migrate my GCP infrastructure to AWS" or "Discover my
+GCP project and estimate AWS costs"). The agent confirms the target project and asks
+for your consent, then inventories it using read-only list/describe commands — it
+captures resource names, types, regions, sizing, network topology, and env var
+**names only**. It never reads env var values, secret values, database contents, or
+access tokens, and never runs a command that creates, changes, or deletes anything.
+If you also have Terraform, the agent cross-checks it against your live project and
+reports drift. (AI/agentic workload detection still needs your application code.)
+
 ### Live Heroku discovery — how it works
 
 No Terraform or exports needed. If `heroku login` works in your terminal, just ask

migrate/plugins/migration-to-aws/fixtures/gcp-live-capture/README.md

@@ -0,0 +1,73 @@
+# GCP live-discovery fixtures (replay mode)
+
+Canned `gcloud` CLI outputs for testing the `gcp-to-aws` live-discovery path
+(`references/phases/discover/discover-live.md`) without a GCP project. The
+synthetic project `acme-prod` uses **per-service fallback mode** (the manifest
+records the Cloud Asset API fast path as `failed` — the common startup case
+after the user declines the enable soft-ask, or a permission error skips it)
+and exercises the designed-for behaviors:
+
+- **orders-api** (Cloud Run) — carries the `run.googleapis.com/cloudsql-instances`
+  annotation and a service account, driving the Step 4 edge-inference rules; its
+  env list has NAMES ONLY (including secret-looking names like
+  `STRIPE_SECRET_KEY` — if a value ever appears in output, the projection rule
+  broke). Terraform declares it as the LEGACY `google_cloud_run_service` (v1)
+  type while live maps to v2 — locking the Step 6 type-alias rule (one merged
+  resource, never a false not_found_live/unmanaged pair).
+- **orders-db** (Cloud SQL) — live tier `db-custom-2-8192` vs Terraform's
+  `db-f1-micro`: the classic console-resize drift (Step 6 rule 1).
+- **cache** (Memorystore Redis), **web-frontend** (Cloud Run), two secrets, and
+  the `acme-prod-uploads` bucket — all absent from Terraform → click-ops drift
+  (`unmanaged_by_terraform`, Step 6 rule 2).
+- **acme-prod-assets** bucket — in Terraform, absent live, buckets capture `ok`
+  → `not_found_live` (Step 6 rule 3).
+- **events** (Pub/Sub topic) — in Terraform, and the pubsub capture FAILED →
+  must NOT be marked `not_found_live` (rule 3's "absence of evidence is not
+  drift" negative case).
+- Empty `gke.json` / `functions.json` / `gce.json` — services with nothing
+  deployed produce no entries and no errors.
+- **Region walk** (`regions.json` + three `redis-<region>.json` captures) — in
+  per-service mode the redis/vertex walk enumerates regions from
+  `gcloud compute regions list` (capture table row 20), not from regions seen
+  in other rows' output; only `us-central1` has an instance, the other two are
+  empty results, not errors.
+
+## How to replay
+
+**Scenario A — live-only (no Terraform):**
+
+1. Create a scratch directory with NO `.tf` files, app code, or billing exports.
+2. Create `.migration/0720-1820/` and copy `live-capture/` into it.
+3. Invoke the gcp-to-aws skill ("migrate my GCP infrastructure to AWS").
+4. Discover should treat live as the primary source (Step 1d), the sub-file
+   parses captures instead of re-running gcloud, and the output is a
+   schema-valid `gcp-resource-inventory.json` + `gcp-resource-clusters.json`
+   with `clustering_mode: "simplified_live"`, `discovery_sources: ["live"]`,
+   and no `drift` key.
+
+**Scenario B — live + Terraform (drift exercise):**
+
+1. As above, but also copy `workspace-terraform/main.tf` into the scratch root.
+2. `main.tf` is deliberately stale — each divergence is commented with the
+   Step 6 merge rule it exercises.
+3. Check outputs against `expected-drift.json` — machine-checkable via
+   `python3 check_expected_drift.py <run-dir>` (exits non-zero on any failed
+   assertion).
+
+**What a run must never produce** (either scenario): env var or secret values
+anywhere; any mutating `gcloud` command (including `gcloud services enable`);
+AWS service names in discover artifacts; a halt caused by the failed
+asset-search/pubsub captures.
+
+**Soft-ask note:** live capture (Step 2a) may offer user-driven remediation when
+CAI fails — enable `cloudasset.googleapis.com` and/or grant
+`roles/cloudasset.viewer` (see https://docs.cloud.google.com/asset-inventory/docs/view-assets).
+Replay skips Step 2 entirely (`manifest.json` already present), so the soft-ask
+does not fire here — the fixture models the post-decline / per-service outcome.
+
+## Regenerating / extending
+
+Captures follow the exact projections in `discover-live.md` Step 2 (same
+filenames). If you add a capture type: whitelist row first, then fixture, then
+extend `expected-drift.json`. All project IDs, names, emails, and account
+numbers are synthetic.

migrate/plugins/migration-to-aws/fixtures/gcp-live-capture/check_expected_drift.py

@@ -0,0 +1,155 @@
+#!/usr/bin/env python3
+"""Assert a Discover run's output against expected-drift.json (scenario B).
+
+Usage:
+    python3 check_expected_drift.py <migration_run_dir>
+
+Where <migration_run_dir> contains gcp-resource-inventory.json and
+gcp-resource-clusters.json produced by a replay of this fixture's scenario B
+(live-capture/ + workspace-terraform/main.tf). Exits 0 on PASS, 1 on FAIL with
+one line per failed assertion. Stdlib only.
+"""
+
+import json
+import sys
+from pathlib import Path
+
+FAILS: list[str] = []
+
+
+def check(cond: bool, msg: str) -> None:
+    if not cond:
+        FAILS.append(msg)
+
+
+def get_conflict_fields(inv: dict) -> list[str]:
+    drift = inv.get("live_metadata", {}).get("drift", {})
+    return [c.get("field", "") for c in drift.get("config_conflicts", [])]
+
+
+def main() -> int:
+    if len(sys.argv) != 2:
+        print(__doc__)
+        return 2
+    run_dir = Path(sys.argv[1])
+    fixture_dir = Path(__file__).resolve().parent
+
+    inv = json.loads((run_dir / "gcp-resource-inventory.json").read_text())
+    clusters = json.loads((run_dir / "gcp-resource-clusters.json").read_text())
+    exp = json.loads((fixture_dir / "expected-drift.json").read_text())
+
+    # Metadata
+    meta = inv["metadata"]
+    for s in exp["metadata"]["discovery_sources_must_include"]:
+        check(s in meta.get("discovery_sources", []), f"discovery_sources missing {s}")
+    check(
+        meta.get("clustering_mode") in exp["metadata"]["clustering_mode_one_of"],
+        f"clustering_mode {meta.get('clustering_mode')} not in {exp['metadata']['clustering_mode_one_of']}",
+    )
+
+    # Resources
+    res = {r["address"]: r for r in inv["resources"]}
+    for addr, e in exp["resources"].items():
+        r = res.get(addr)
+        if r is None:
+            # Address synthesis can differ for live-only names; try name-suffix match
+            candidates = [v for k, v in res.items() if k.split(".")[0] == addr.split(".")[0] and addr.split(".")[1] in k]
+            if len(candidates) == 1:
+                r = candidates[0]
+            else:
+                check(False, f"missing resource {addr}")
+                continue
+        if "source" in e:
+            check(r.get("source") == e["source"], f"{addr} source={r.get('source')} want {e['source']}")
+        if "classification" in e:
+            check(r.get("classification") == e["classification"], f"{addr} classification")
+        if e.get("not_found_live"):
+            check(r.get("not_found_live") is True, f"{addr} not_found_live missing")
+        if e.get("unmanaged_by_terraform"):
+            check(r.get("unmanaged_by_terraform") is True, f"{addr} unmanaged_by_terraform missing")
+        for flag in e.get("must_not_have", []):
+            check(flag not in r or r.get(flag) is not True, f"{addr} must not have {flag}")
+        if "config_must_include" in e:
+            cfg_text = json.dumps(r.get("config", {}))
+            for v in e["config_must_include"].values():
+                check(v in cfg_text, f"{addr} config missing value {v}")
+
+    # Edges
+    all_edges = [edge for c in clusters["clusters"] for edge in c.get("edges", [])]
+    for ee in exp["edges_must_include"]:
+        found = any(
+            ee["from_contains"] in edge.get("from", "")
+            and ee["to_contains"] in edge.get("to", "")
+            and edge.get("relationship_type") == ee["relationship_type"]
+            and (
+                "evidence_contains" not in ee
+                or ee["evidence_contains"] in json.dumps(edge.get("evidence", {}))
+            )
+            for edge in all_edges
+        )
+        check(found, f"edge missing: {ee['from_contains']} -> {ee['to_contains']} ({ee['relationship_type']})")
+
+    # Cluster coverage
+    clustered = {a for c in clusters["clusters"] for a in c["primary_resources"] + c["secondary_resources"]}
+    check(set(res) == clustered, f"cluster coverage mismatch: {sorted(set(res) ^ clustered)}")
+    check(
+        any(c["creation_order_depth"] == 0 and "networking" in c["cluster_id"] for c in clusters["clusters"]),
+        "no networking cluster at depth 0",
+    )
+
+    # AI detection
+    check(inv["ai_detection"]["has_ai_workload"] is exp["ai_detection"]["has_ai_workload"], "ai_detection mismatch")
+
+    # live_metadata + drift
+    lm = inv["live_metadata"]
+    check(lm.get("method") == exp["live_metadata"]["method"], "live method")
+    check(lm.get("project") == exp["live_metadata"]["project"], "live project")
+    warns = json.dumps(lm.get("capture_warnings", [])).lower()
+    for w in exp["live_metadata"]["capture_warnings_must_mention"]:
+        check(w in warns, f"capture_warnings missing mention of {w}")
+    drift = lm.get("drift", {})
+    check(
+        drift.get("resources_terraform_only", 99) <= exp["live_metadata"]["drift"]["resources_terraform_only_max"],
+        f"resources_terraform_only={drift.get('resources_terraform_only')}",
+    )
+    check(
+        drift.get("resources_live_only", 0) >= exp["live_metadata"]["drift"]["resources_live_only_min"],
+        f"resources_live_only={drift.get('resources_live_only')}",
+    )
+    conflict_fields = get_conflict_fields(inv)
+    for f in exp["live_metadata"]["drift"]["config_conflict_fields_must_include"]:
+        check(f in conflict_fields, f"config_conflicts missing field {f}")
+
+    # Safety: no fixture env names paired with values, no AWS names
+    doc = json.dumps(inv)
+    check("aws_" not in doc.lower().replace("aws-design", ""), "possible AWS naming in discover artifact")
+
+    # Secret hygiene: no env entry anywhere may carry a value payload, and env
+    # data must appear only as name lists (env_var_names), never env objects.
+    def walk(node, path="$"):
+        if isinstance(node, dict):
+            if "name" in node and ("value" in node or "valueFrom" in node):
+                check(False, f"env-like object with a value payload at {path}")
+            for k, v in node.items():
+                if k == "env":
+                    check(False, f"raw 'env' key at {path} — spec requires env_var_names (names only)")
+                walk(v, f"{path}.{k}")
+        elif isinstance(node, list):
+            for i, v in enumerate(node):
+                walk(v, f"{path}[{i}]")
+
+    walk(inv)
+    for env_name in ("STRIPE_SECRET_KEY", "DATABASE_URL", "REDIS_URL"):
+        check(f'"{env_name}": ' not in doc, f"fixture env name {env_name} appears as a KEY (value paired) — names must be list items only")
+
+    if FAILS:
+        print(f"FAIL ({len(FAILS)}):")
+        for f in FAILS:
+            print(f"  - {f}")
+        return 1
+    print("PASS — expected-drift.json assertions hold")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())