diff --git a/advisor/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-scripts.md b/advisor/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-scripts.md index 6447920c..fa3cbcf1 100644 --- a/advisor/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-scripts.md +++ b/advisor/plugins/aws-startup-advisor/skills/gcp-to-aws/references/phases/generate/generate-artifacts-scripts.md @@ -29,13 +29,16 @@ Set boolean flags for downstream script generation: - **has_databases**: true if ANY resource has `aws_service` containing "RDS", "Aurora", "DynamoDB", "ElastiCache", "Redshift" OR `gcp_type` starting with `google_sql_`, `google_firestore_`, - `google_bigtable_`, `google_bigquery_`, `google_redis_` + `google_bigtable_`, `google_redis_` (NOT `google_bigquery_` — BigQuery is specialist-deferred + and carries no automated data-migration steps; see `has_bigquery`) +- **has_bigquery**: true if ANY resource has `gcp_type` starting with `google_bigquery_` + OR `aws_service` = "Deferred — specialist engagement" - **has_storage**: true if ANY resource has `aws_service` = "S3" OR `gcp_type` = `google_storage_bucket` - **has_containers**: true if ANY resource has `aws_service` containing "Fargate", "ECS", "EKS" OR `gcp_type` starting with `google_cloud_run_`, `google_container_cluster` - **has_secrets**: true if ANY resource has `aws_service` containing "Secrets Manager" OR `gcp_type` starting with `google_secret_manager_` -- **has_data_migration**: has_databases OR has_storage (used for script 02) +- **has_data_migration**: has_databases OR has_storage OR has_bigquery (used for script 02) Report detected categories to user: "Resource categories detected: [list active flags]" @@ -79,6 +82,22 @@ Verify all prerequisites before migration: Based on database and storage resources in `aws-design.json`: +**Script preamble** — emit FIRST, whenever this script is generated (any subsection below +active). The shebang, failure mode, and dry-run plumbing are not gated on any single resource +flag: a BigQuery-only project generates only the deferral notice below, and that script still +has to satisfy Script Quality Rule 1 (`set -euo pipefail`): + +```bash +#!/usr/bin/env bash +set -euo pipefail +# Data migration +# Usage: ./02-migrate-data.sh [--execute] + +DRY_RUN=true +[[ "${1:-}" == "--execute" ]] && DRY_RUN=false +echo "Mode: $([ "$DRY_RUN" = true ] && echo 'DRY RUN' || echo 'EXECUTE')" +``` + **Cloud SQL to RDS/Aurora** — include only if `has_databases`: Read `preferences.json` → `design_constraints.db_size.value` to select the migration tool: @@ -91,11 +110,7 @@ Read `preferences.json` → `design_constraints.db_size.value` to select the mig Generate the script with conditional branches based on `db_size`: ```bash -#!/usr/bin/env bash -set -euo pipefail # Cloud SQL → RDS data migration -# Usage: ./02-migrate-data.sh [--execute] -# # Tool selection based on database size (preferences.json design_constraints.db_size.value): # <10GB: pg_dump/pg_restore # 10-500GB: pgcopydb (parallel copy, 3-5x faster than pg_dump) @@ -103,11 +118,7 @@ set -euo pipefail # unknown: pgcopydb (safer default at unknown scale) # TODO: Verify database size before running — wrong tool choice can exceed your maintenance window. -DRY_RUN=true -[[ "${1:-}" == "--execute" ]] && DRY_RUN=false - echo "=== Database Migration: Cloud SQL → RDS ===" -echo "Mode: $([ "$DRY_RUN" = true ] && echo 'DRY RUN' || echo 'EXECUTE')" SOURCE_HOST="" # TODO: Set Cloud SQL IP TARGET_HOST="" # From terraform output database_endpoint @@ -201,13 +212,25 @@ echo "=== Verification ===" echo "TODO: Compare row counts between source and target" ``` -**BigQuery to S3** — include only if `has_databases`: +**BigQuery — specialist-deferred (no automated AWS target)** — include only if `has_bigquery`: + +Design marked every `google_bigquery_*` resource as **`Deferred — specialist engagement`** with +`no_automated_aws_target: true` (see `references/phases/design/design-infra.md` → BigQuery specialist gate). +Therefore **do not** generate BigQuery export, copy, or load steps, and **do not** name an AWS analytics or +warehouse target (no Athena, Redshift, Glue, EMR, Lake Formation, or a prescribed "data lake on S3"). +Emit only the deferral notice below: ```bash -# BigQuery → S3 data export -# TODO: Configure BigQuery dataset and S3 bucket -# bq extract --destination_format=PARQUET 'dataset.table' 'gs://bucket/export/' -# aws s3 sync gs://bucket/export/ s3://target-bucket/import/ +# BigQuery — AWS target deliberately NOT selected +# This plugin does not choose an AWS analytics or warehouse target for BigQuery +# (no Athena/Redshift/Glue/EMR recommendation, and no prescribed data lake on S3). +# Engage your AWS account team and/or a data analytics migration partner before +# data warehouse, lake, SQL analytics, or BI cutover planning — query patterns, +# data volumes, ETL/ELT, and downstream consumers must be assessed by specialists. +# No BigQuery migration steps are generated here by design. +echo "BigQuery: AWS target deferred — specialist engagement required." +echo " Next step: engage your AWS account team and/or a data analytics migration partner." +echo " No BigQuery export or load steps are generated by this skill." ``` **Firestore to DynamoDB** — include only if `has_databases`: @@ -383,6 +406,9 @@ After generating all scripts, verify the following quality rules: 3. All scripts include verification steps 4. All scripts are numbered for execution order 5. All TODO markers are clearly marked with context +6. No script prescribes an AWS analytics or warehouse target for BigQuery (no Athena, Redshift, Glue, + EMR, Lake Formation, or "data lake on S3", and no `bq extract` / export steps) — BigQuery resources + carry only the specialist-engagement deferral notice, consistent with `no_automated_aws_target: true` ## Phase Completion diff --git a/migrate/plugins/migration-to-aws/skills/gcp-to-aws/references/phases/generate/generate-artifacts-scripts.md b/migrate/plugins/migration-to-aws/skills/gcp-to-aws/references/phases/generate/generate-artifacts-scripts.md index 6447920c..fa3cbcf1 100644 --- a/migrate/plugins/migration-to-aws/skills/gcp-to-aws/references/phases/generate/generate-artifacts-scripts.md +++ b/migrate/plugins/migration-to-aws/skills/gcp-to-aws/references/phases/generate/generate-artifacts-scripts.md @@ -29,13 +29,16 @@ Set boolean flags for downstream script generation: - **has_databases**: true if ANY resource has `aws_service` containing "RDS", "Aurora", "DynamoDB", "ElastiCache", "Redshift" OR `gcp_type` starting with `google_sql_`, `google_firestore_`, - `google_bigtable_`, `google_bigquery_`, `google_redis_` + `google_bigtable_`, `google_redis_` (NOT `google_bigquery_` — BigQuery is specialist-deferred + and carries no automated data-migration steps; see `has_bigquery`) +- **has_bigquery**: true if ANY resource has `gcp_type` starting with `google_bigquery_` + OR `aws_service` = "Deferred — specialist engagement" - **has_storage**: true if ANY resource has `aws_service` = "S3" OR `gcp_type` = `google_storage_bucket` - **has_containers**: true if ANY resource has `aws_service` containing "Fargate", "ECS", "EKS" OR `gcp_type` starting with `google_cloud_run_`, `google_container_cluster` - **has_secrets**: true if ANY resource has `aws_service` containing "Secrets Manager" OR `gcp_type` starting with `google_secret_manager_` -- **has_data_migration**: has_databases OR has_storage (used for script 02) +- **has_data_migration**: has_databases OR has_storage OR has_bigquery (used for script 02) Report detected categories to user: "Resource categories detected: [list active flags]" @@ -79,6 +82,22 @@ Verify all prerequisites before migration: Based on database and storage resources in `aws-design.json`: +**Script preamble** — emit FIRST, whenever this script is generated (any subsection below +active). The shebang, failure mode, and dry-run plumbing are not gated on any single resource +flag: a BigQuery-only project generates only the deferral notice below, and that script still +has to satisfy Script Quality Rule 1 (`set -euo pipefail`): + +```bash +#!/usr/bin/env bash +set -euo pipefail +# Data migration +# Usage: ./02-migrate-data.sh [--execute] + +DRY_RUN=true +[[ "${1:-}" == "--execute" ]] && DRY_RUN=false +echo "Mode: $([ "$DRY_RUN" = true ] && echo 'DRY RUN' || echo 'EXECUTE')" +``` + **Cloud SQL to RDS/Aurora** — include only if `has_databases`: Read `preferences.json` → `design_constraints.db_size.value` to select the migration tool: @@ -91,11 +110,7 @@ Read `preferences.json` → `design_constraints.db_size.value` to select the mig Generate the script with conditional branches based on `db_size`: ```bash -#!/usr/bin/env bash -set -euo pipefail # Cloud SQL → RDS data migration -# Usage: ./02-migrate-data.sh [--execute] -# # Tool selection based on database size (preferences.json design_constraints.db_size.value): # <10GB: pg_dump/pg_restore # 10-500GB: pgcopydb (parallel copy, 3-5x faster than pg_dump) @@ -103,11 +118,7 @@ set -euo pipefail # unknown: pgcopydb (safer default at unknown scale) # TODO: Verify database size before running — wrong tool choice can exceed your maintenance window. -DRY_RUN=true -[[ "${1:-}" == "--execute" ]] && DRY_RUN=false - echo "=== Database Migration: Cloud SQL → RDS ===" -echo "Mode: $([ "$DRY_RUN" = true ] && echo 'DRY RUN' || echo 'EXECUTE')" SOURCE_HOST="" # TODO: Set Cloud SQL IP TARGET_HOST="" # From terraform output database_endpoint @@ -201,13 +212,25 @@ echo "=== Verification ===" echo "TODO: Compare row counts between source and target" ``` -**BigQuery to S3** — include only if `has_databases`: +**BigQuery — specialist-deferred (no automated AWS target)** — include only if `has_bigquery`: + +Design marked every `google_bigquery_*` resource as **`Deferred — specialist engagement`** with +`no_automated_aws_target: true` (see `references/phases/design/design-infra.md` → BigQuery specialist gate). +Therefore **do not** generate BigQuery export, copy, or load steps, and **do not** name an AWS analytics or +warehouse target (no Athena, Redshift, Glue, EMR, Lake Formation, or a prescribed "data lake on S3"). +Emit only the deferral notice below: ```bash -# BigQuery → S3 data export -# TODO: Configure BigQuery dataset and S3 bucket -# bq extract --destination_format=PARQUET 'dataset.table' 'gs://bucket/export/' -# aws s3 sync gs://bucket/export/ s3://target-bucket/import/ +# BigQuery — AWS target deliberately NOT selected +# This plugin does not choose an AWS analytics or warehouse target for BigQuery +# (no Athena/Redshift/Glue/EMR recommendation, and no prescribed data lake on S3). +# Engage your AWS account team and/or a data analytics migration partner before +# data warehouse, lake, SQL analytics, or BI cutover planning — query patterns, +# data volumes, ETL/ELT, and downstream consumers must be assessed by specialists. +# No BigQuery migration steps are generated here by design. +echo "BigQuery: AWS target deferred — specialist engagement required." +echo " Next step: engage your AWS account team and/or a data analytics migration partner." +echo " No BigQuery export or load steps are generated by this skill." ``` **Firestore to DynamoDB** — include only if `has_databases`: @@ -383,6 +406,9 @@ After generating all scripts, verify the following quality rules: 3. All scripts include verification steps 4. All scripts are numbered for execution order 5. All TODO markers are clearly marked with context +6. No script prescribes an AWS analytics or warehouse target for BigQuery (no Athena, Redshift, Glue, + EMR, Lake Formation, or "data lake on S3", and no `bq extract` / export steps) — BigQuery resources + carry only the specialist-engagement deferral notice, consistent with `no_automated_aws_target: true` ## Phase Completion