diff --git a/.github/actions/just/README.md b/.github/actions/just/README.md index c4cda353..4ea57051 100644 --- a/.github/actions/just/README.md +++ b/.github/actions/just/README.md @@ -19,7 +19,7 @@ This GitHub Action sets up [`just`](https://github.com/casey/just), authenticate |--------------------|--------------------------------------------------|----------|--------------| | `just_version` | Version of `just` to install | ❌ | `1.0.0` | | `aws_region` | AWS region | ❌ | `eu-west-2` | -| `aws_oidc_role_arn`| ARN of the IAM role to assume via OIDC | ✅ | — | +| `aws_oidc_role_arn`| ARN of the IAM role to assume via OIDC (optional when AWS credentials are already configured in the job) | ❌ | `""` | | `just_action` | The `just` recipe to execute | ✅ | — | | `mask_result` | Use to mask value in CI | ❌ | `false` | @@ -98,4 +98,4 @@ jobs: - name: read output from script run: | echo "Script output will appear *** in CI logs: ${{ steps.get-secret.outputs.just_outputs }}" -``` \ No newline at end of file +``` diff --git a/.github/actions/just/action.yml b/.github/actions/just/action.yml index 2a2d9de3..666088e6 100644 --- a/.github/actions/just/action.yml +++ b/.github/actions/just/action.yml @@ -33,13 +33,13 @@ runs: using: "composite" steps: - name: Install Just - uses: extractions/setup-just@v3 + uses: extractions/setup-just@v4 with: just-version: ${{ inputs.just_version }} - name: Configure AWS OIDC Authentication if: ${{ inputs.aws_oidc_role_arn != '' }} - uses: aws-actions/configure-aws-credentials@v4 + uses: aws-actions/configure-aws-credentials@v5 with: role-to-assume: ${{ inputs.aws_oidc_role_arn }} aws-region: ${{ inputs.aws_region }} diff --git a/.github/actions/terragrunt/README.md b/.github/actions/terragrunt/README.md index e9587c9e..18a30da5 100644 --- a/.github/actions/terragrunt/README.md +++ b/.github/actions/terragrunt/README.md @@ -7,8 +7,9 @@ This GitHub Action sets up **Terraform** and **Terragrunt**, authenticates to AW - Installs pinned versions of Terraform and Terragrunt - Authenticates to AWS using OIDC only when the selected action actually needs AWS access - Optionally passes Terragrunt variables via JSON tfvars -- Supports `plan` mode with automatic GitHub artifact upload +- Supports `plan` mode for producing local saved plan files - Supports `init` mode for outputs-only reads +- Uses the repo-local `./.github/actions/just` action with OIDC for saved plan artifact upload and download - Exports Terragrunt outputs as compact JSON when state exists ## Inputs @@ -19,8 +20,6 @@ This GitHub Action sets up **Terraform** and **Terragrunt**, authenticates to AW | `tg_version` | Version of Terragrunt to install | No | `0.72.6` | | `aws_region` | AWS region to use | No | `eu-west-2` | | `override_tg_vars` | Terragrunt variables in JSON, written to `override_tg_vars.tfvars.json` | No | `{}` | -| `plan_artifact_run_id` | Optional workflow run ID to download a plan artifact from in `apply_plan` mode | No | `""` | -| `github_token` | GitHub token used for cross-run plan artifact downloads | No | `""` | | `aws_oidc_role_arn` | IAM role ARN to assume via OIDC | Yes | — | | `tg_directory` | Directory containing the Terragrunt config | Yes | — | | `tg_action` | Terragrunt action: `apply`, `plan`, `apply_plan`, `destroy`, or `init` | Yes | `apply` | @@ -32,21 +31,28 @@ This GitHub Action sets up **Terraform** and **Terragrunt**, authenticates to AW | Name | Description | |---|---| | `tg_outputs` | All Terraform outputs in compact JSON. If no state exists, returns `{}` | -| `plan_artifact_name` | Derived GitHub artifact name for a Terragrunt plan | - ## Behavior - `apply` Runs `terragrunt apply -auto-approve` - `plan` - Runs `terragrunt plan -detailed-exitcode -out=/terragrunt.tfplan`, renders a text view to `terragrunt.plan.txt`, writes `terragrunt.plan.meta.json` with `exit_code` and `has_changes`, and uploads all three files as a GitHub artifact via `actions/upload-artifact@v7`. The artifact name is derived from `tg_directory`. + Runs `terragrunt plan -detailed-exitcode -out=/terragrunt.tfplan`, then renders `terragrunt.plan.txt` and writes `terragrunt.plan.meta.json` via the repo `justfile.tg` recipe `terragrunt-plan-render`. It then uploads those files to S3 through the repo-local `./.github/actions/just` action using the same OIDC role. - `apply_plan` - Downloads the derived plan artifact into the working directory via `actions/download-artifact@v8`, explicitly scoped to the current `github.repository` for cross-run recovery, fails if the artifact, binary plan file, or `terragrunt.plan.meta.json` is missing, reads `has_changes` from the saved metadata file, and skips both AWS authentication and apply with a GitHub Actions warning when the saved plan contains no mutating resource changes. Otherwise it configures AWS credentials and runs `terragrunt apply` against the downloaded absolute stack-path plan file. For separate workflow runs, pass `plan_artifact_run_id` and `github_token`. + Downloads the saved plan files into `tg_directory` via the repo-local `./.github/actions/just` action and `justfile.tg`, using the caller-provided `PLAN_ARTIFACT_S3_PREFIX` environment variable plus the stack-derived suffix from `tg_directory`. It then fails if the binary plan file or `terragrunt.plan.meta.json` is missing, reads `has_changes` from the saved metadata file, and skips apply with a GitHub Actions warning when the saved plan contains no mutating resource changes. Otherwise it runs `terragrunt apply` against the absolute stack-path plan file. - `destroy` Runs `terragrunt destroy -auto-approve` - `init` Runs `terragrunt init -input=false -reconfigure` and then captures outputs +## Saved Plan Layout + +- One run-level metadata file is stored separately by the shared infra wrapper at: + - `/infra-plan-metadata/plan-metadata.json` +- Each Terragrunt stack or module stores its own plan bundle at: + - `/terragrunt-plan-/terragrunt.tfplan` + - `/terragrunt-plan-/terragrunt.plan.txt` + - `/terragrunt-plan-/terragrunt.plan.meta.json` + ## Usage ### Minimal Apply @@ -76,7 +82,7 @@ jobs: echo '${{ steps.tg_action.outputs.tg_outputs }}' | jq . ``` -### Plan And Upload Artifact +### Plan ```yaml jobs: @@ -89,20 +95,15 @@ jobs: - uses: actions/checkout@v4 - name: Plan infrastructure - id: tg_plan uses: your-org/your-action-repo@main with: aws_region: ${{ vars.AWS_REGION }} aws_oidc_role_arn: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/${{ vars.PROJECT_NAME }}-dev-github-oidc-role tg_directory: infra/live/dev/aws/network tg_action: plan - - - name: Show uploaded artifact name - run: | - echo "${{ steps.tg_plan.outputs.plan_artifact_name }}" ``` -### Apply From Uploaded Plan Artifact +### Apply From Uploaded Plan In S3 ```yaml jobs: @@ -122,3 +123,5 @@ jobs: tg_directory: infra/live/dev/aws/network tg_action: apply_plan ``` + +This action expects the workflow to download `terragrunt.tfplan`, `terragrunt.plan.txt`, and `terragrunt.plan.meta.json` into `tg_directory` before calling `tg_action: apply_plan`. diff --git a/.github/actions/terragrunt/action.yml b/.github/actions/terragrunt/action.yml index c553e20b..615b5bf4 100644 --- a/.github/actions/terragrunt/action.yml +++ b/.github/actions/terragrunt/action.yml @@ -17,14 +17,6 @@ inputs: description: "Override or additional Terragrunt variables in JSON format" required: false default: "{}" - plan_artifact_run_id: - description: "Optional workflow run ID to download a plan artifact from when using `apply_plan` across separate workflow runs" - required: false - default: "" - github_token: - description: "GitHub token used for cross-run plan artifact downloads" - required: false - default: "" aws_oidc_role_arn: description: "AWS iam role arn" required: true @@ -40,21 +32,18 @@ outputs: tg_outputs: description: "All Terraform outputs in JSON format" value: ${{ steps.tg_outputs.outputs.terraform_json || steps.tg_outputs_skip.outputs.terraform_json }} - plan_artifact_name: - description: "Derived GitHub artifact name for a Terragrunt plan" - value: ${{ steps.plan_artifact.outputs.plan_artifact_name }} runs: using: "composite" steps: - name: Install Terraform - uses: hashicorp/setup-terraform@v3 + uses: hashicorp/setup-terraform@v4 with: terraform_version: ${{ inputs.tf_version }} terraform_wrapper: false - name: Install Terragrunt - uses: autero1/action-terragrunt@v1.3.2 + uses: autero1/action-terragrunt@v3.0.2 with: terragrunt_version: ${{ inputs.tg_version }} @@ -66,42 +55,27 @@ runs: run: | echo "$OVERRIDE_TG_VARS" | jq -c . > ${{ inputs.tg_directory }}/override_tg_vars.tfvars.json - - name: Resolve plan artifact name - if: inputs.tg_action == 'plan' || inputs.tg_action == 'apply_plan' - id: plan_artifact - shell: bash - run: | - sanitized_dir="$(echo "${{ inputs.tg_directory }}" | tr '/.' '--')" - artifact_name="terragrunt-plan-${sanitized_dir}" - echo "plan_artifact_name=$artifact_name" >> "$GITHUB_OUTPUT" - - - name: Download plan artifact from current run - if: inputs.tg_action == 'apply_plan' && inputs.plan_artifact_run_id == '' - uses: actions/download-artifact@v8 - with: - name: ${{ steps.plan_artifact.outputs.plan_artifact_name }} - path: ${{ inputs.tg_directory }} - - - name: Download plan artifact from prior run - if: inputs.tg_action == 'apply_plan' && inputs.plan_artifact_run_id != '' - uses: actions/download-artifact@v8 + - name: Download saved plan artifacts + if: inputs.tg_action == 'apply_plan' + uses: ./.github/actions/just + env: + TG_DIRECTORY: ${{ inputs.tg_directory }} with: - name: ${{ steps.plan_artifact.outputs.plan_artifact_name }} - path: ${{ inputs.tg_directory }} - repository: ${{ github.repository }} - github-token: ${{ inputs.github_token }} - run-id: ${{ inputs.plan_artifact_run_id }} + aws_oidc_role_arn: ${{ inputs.aws_oidc_role_arn }} + aws_region: ${{ inputs.aws_region }} + justfile_path: justfile.tg + just_action: terragrunt-plan-download - name: Verify plan artifact files exist if: inputs.tg_action == 'apply_plan' shell: bash run: | test -f "${{ inputs.tg_directory }}/terragrunt.tfplan" || { - echo "Expected plan file '${{ inputs.tg_directory }}/terragrunt.tfplan' was not found after artifact download." >&2 + echo "Expected plan file '${{ inputs.tg_directory }}/terragrunt.tfplan' was not found before apply_plan." >&2 exit 1 } test -f "${{ inputs.tg_directory }}/terragrunt.plan.meta.json" || { - echo "Expected plan metadata file '${{ inputs.tg_directory }}/terragrunt.plan.meta.json' was not found after artifact download." >&2 + echo "Expected plan metadata file '${{ inputs.tg_directory }}/terragrunt.plan.meta.json' was not found before apply_plan." >&2 exit 1 } @@ -123,7 +97,7 @@ runs: - name: Configure AWS OIDC Authentication if: inputs.tg_action != 'apply_plan' || steps.apply_plan_guard.outputs.should_apply == 'true' - uses: aws-actions/configure-aws-credentials@v4 + uses: aws-actions/configure-aws-credentials@v5 with: role-to-assume: ${{ inputs.aws_oidc_role_arn }} aws-region: ${{ inputs.aws_region }} @@ -137,8 +111,6 @@ runs: working-directory: ${{ inputs.tg_directory }} run: | PLAN_PATH="$(pwd)/terragrunt.tfplan" - PLAN_TEXT_PATH="$(pwd)/terragrunt.plan.txt" - PLAN_META_PATH="$(pwd)/terragrunt.plan.meta.json" case "${{ inputs.tg_action }}" in apply) @@ -154,19 +126,10 @@ runs: exit 1 fi - terragrunt show -no-color "$PLAN_PATH" > "$PLAN_TEXT_PATH" - jq -n \ - --arg tg_directory "${{ inputs.tg_directory }}" \ - --argjson exit_code "$plan_exit_code" \ - --argjson has_changes "$([ "$plan_exit_code" -eq 2 ] && echo true || echo false)" \ - '{tg_directory: $tg_directory, exit_code: $exit_code, has_changes: $has_changes}' \ - > "$PLAN_META_PATH" + echo "plan_exit_code=$plan_exit_code" >> "$GITHUB_OUTPUT" echo "Terragrunt binary plan path: $PLAN_PATH" ls -l "$PLAN_PATH" - echo "Terragrunt rendered plan path: $PLAN_TEXT_PATH" - cat "$PLAN_TEXT_PATH" - echo "Terragrunt plan metadata path: $PLAN_META_PATH" - cat "$PLAN_META_PATH" + ;; apply_plan) terragrunt apply -auto-approve -compact-warnings "$PLAN_PATH" @@ -184,16 +147,26 @@ runs: ;; esac - - name: Upload plan artifact + - name: Render plan sidecar artifacts + if: inputs.tg_action == 'plan' + uses: ./.github/actions/just + env: + TG_DIRECTORY: ${{ inputs.tg_directory }} + TG_PLAN_EXIT_CODE: ${{ steps.terragrunt_action.outputs.plan_exit_code }} + with: + justfile_path: justfile.tg + just_action: terragrunt-plan-render + + - name: Upload saved plan artifacts if: inputs.tg_action == 'plan' - uses: actions/upload-artifact@v7 + uses: ./.github/actions/just + env: + TG_DIRECTORY: ${{ inputs.tg_directory }} with: - name: ${{ steps.plan_artifact.outputs.plan_artifact_name }} - path: | - ${{ inputs.tg_directory }}/terragrunt.tfplan - ${{ inputs.tg_directory }}/terragrunt.plan.txt - ${{ inputs.tg_directory }}/terragrunt.plan.meta.json - if-no-files-found: error + aws_oidc_role_arn: ${{ inputs.aws_oidc_role_arn }} + aws_region: ${{ inputs.aws_region }} + justfile_path: justfile.tg + just_action: terragrunt-plan-upload - name: Capture Terraform Outputs if: inputs.tg_action != 'destroy' && (inputs.tg_action != 'apply_plan' || steps.apply_plan_guard.outputs.should_apply == 'true') diff --git a/.github/docs/README.md b/.github/docs/README.md index a3a9033d..cfdc3a88 100644 --- a/.github/docs/README.md +++ b/.github/docs/README.md @@ -66,7 +66,7 @@ flowchart LR ### Shared Artifact Prep And Build - `shared_infra_releases.yml` - Prepares or reads shared CI-side artifact infrastructure such as ECR and the code bucket. + Prepares or reads shared artifact infrastructure such as ECR and the code bucket, and exposes those bucket/repository values as reusable-workflow outputs. The code-bucket job reads the Lambda, frontend, AppSpec, and infra-plan S3 prefix names from string-returning `justfile.ci` recipes and forwards them as `TF_VAR_*`, so the workflow does not duplicate those key names inline. - `shared_build.yml` Builds and publishes frontend, Lambda, and ECS artifacts. - `shared_build_get.yml` @@ -82,14 +82,14 @@ flowchart LR ### Infra And Code Rollout - `shared_infra_plan.yml` - Plan wrapper around `shared_infra.yml`. It takes resolved workflow inputs directly, writes and uploads `infra-plan-metadata`, and then calls `shared_infra.yml` with `tg_action: plan`. After the plan completes, it prints the current workflow `github.run_id` into both the logs and the GitHub Actions step summary as `plan_artifact_run_id`, and exposes that value as a reusable-workflow output. + Plan wrapper around `shared_infra.yml`. It takes resolved workflow inputs directly, derives a single plan-artifact S3 prefix via `justfile.tg`, uploads one run-level metadata file under `/infra-plan-metadata/plan-metadata.json` via `justfile.ci`, and then calls `shared_infra.yml` with `tg_action: plan` plus that same resolved prefix. The bucket resolution follows the same artifact split as ECR and build outputs: `dev` uses the `dev` code bucket, while non-`dev` environments reuse the `ci` code bucket. After the plan completes, it prints the current workflow `github.run_id` into both the logs and the GitHub Actions step summary as `plan_artifact_run_id`, and exposes that value as a reusable-workflow output. - `shared_infra_apply.yml` Direct-input apply wrapper around `shared_infra.yml`. It takes resolved workflow inputs directly and calls `shared_infra.yml` with `tg_action: apply`. - `shared_infra_apply_from_plan.yml` - Apply-from-plan wrapper around `shared_infra.yml`. It takes `plan_artifact_run_id` plus a `github_token` input, downloads `infra-plan-metadata`, reads the frozen graph inputs back out, and then calls `shared_infra.yml` with `tg_action: apply_plan`. + Apply-from-plan wrapper around `shared_infra.yml`. It takes `plan_artifact_run_id`, resolves the same artifact bucket split used by release artifacts (`dev` stays on `dev`, non-`dev` uses `ci`) inside its `metadata` job, derives the matching plan-artifact S3 prefix via `justfile.tg`, downloads `infra-plan-metadata` from that location via `justfile.ci`, reads the frozen graph inputs back out, and then calls `shared_infra.yml` with `tg_action: apply_plan` plus that same resolved prefix. - `shared_infra.yml` - Pure ordered infra graph executor. It applies shared stacks first, then runtime stacks, then frontend infrastructure. Shared stacks now include the CloudWatch observability dashboard. It accepts `tg_action` so the same graph can run a normal apply, upload derived per-stack plan artifacts, or apply from previously uploaded plan artifacts. Its visible step labels now follow `tg_action` too, so plan and apply-from-plan runs no longer show everything as `Deploy ...`. The `security -> network` edge is a real bootstrap dependency because `network` reads security outputs like `vpc_endpoint_sg` from remote state; if those outputs do not exist yet, `network` fails with an upstream `Unsupported attribute` error rather than a networking-specific error. -- The shared infra wrappers must forward the permissions required by the nested reusable call chain. In practice that means `id-token: write` everywhere the Terragrunt action may assume AWS OIDC, `contents: read` for checkout, and `actions: read` anywhere cross-run artifact recovery can happen. That includes `shared_infra_apply_from_plan.yml` and the nested `shared_infra.yml` jobs, because both the shared metadata recovery and the per-stack `apply_plan` downloads use artifact actions across workflow runs in that chain. + Pure ordered infra graph executor. It applies shared stacks first, then runtime stacks, then frontend infrastructure. Shared stacks now include the CloudWatch observability dashboard. It accepts `tg_action` so the same graph can run a normal apply, upload derived per-stack plan artifacts to the resolved code bucket under `terragrunt_plan/`, or apply from previously uploaded plan artifacts. The wrapper workflows resolve one `plan_artifact_s3_prefix` and set it in the workflow env once; the Terragrunt action reads that env directly and appends the stack-specific suffix from `tg_directory`, so `shared_infra.yml` does not need to rebuild the bucket or env/run-id tuple in every job. That means each infra run has one shared `plan-metadata.json` file for the whole graph and one separate saved plan bundle per Terragrunt stack or module. In `apply_plan` mode, each stack job first downloads its own saved plan files via `justfile.tg` through the Terragrunt action, then the Terragrunt action validates and applies those local files. Its visible step labels now follow `tg_action` too, so plan and apply-from-plan runs no longer show everything as `Deploy ...`. The `security -> network` edge is a real bootstrap dependency because `network` reads security outputs like `vpc_endpoint_sg` from remote state; if those outputs do not exist yet, `network` fails with an upstream `Unsupported attribute` error rather than a networking-specific error. +- The shared infra wrappers must forward the permissions required by the nested reusable call chain. In practice that means `id-token: write` everywhere the Terragrunt action may assume AWS OIDC and `contents: read` for checkout. The shared plan/apply wrappers now rely on AWS access to the shared code bucket rather than GitHub artifact permissions for cross-run recovery. - `shared_deploy.yml` Rolls out Lambda code, optional migrations, optional reconciliation Lambdas, ECS task and service updates, and optional frontend deploys. The reusable workflow renders its Lambda and ECS CodeDeploy AppSpec files from the shared templates under `config/deploy/`, and its mutating `just` steps should target `justfile.deploy` rather than the repo-root `justfile`. @@ -128,7 +128,7 @@ flowchart LR ### Cleanup And Discovery - `destroy.yml` - Tears down app layers before shared dependencies, including the shared observability dashboard. + Tears down app layers before shared dependencies, including the shared observability dashboard and any environment-owned shared artifact stacks such as the `dev` code bucket. - `shared_directories_get.yml` Derives the directory-based matrices used by wrapper workflows and PR action-test discovery. @@ -141,11 +141,15 @@ Run these checks on every CI, workflow, or deploy-contract change. - compare every caller `with:` block against the callee `workflow_call.inputs` - compare expected outputs against actual `jobs..outputs.*` - verify optional inputs are intentionally omitted, not accidentally missing -- the repo-local `./.github/actions/terragrunt` action now supports `tg_action: plan` plus automatic artifact upload of the binary plan, rendered text plan, and a `terragrunt.plan.meta.json` sidecar with `exit_code` and `has_changes`; keep that contract in mind before inventing a second plan-storage mechanism -- `./.github/actions/terragrunt` now skips `apply_plan` with a warning when the saved `terragrunt.plan.meta.json` reports `has_changes: false`, and in that no-op case it also skips `aws-actions/configure-aws-credentials` +- the repo-local `./.github/actions/terragrunt` action supports `tg_action: plan` for producing the binary plan locally; it renders `terragrunt.plan.txt` and writes `terragrunt.plan.meta.json` via `justfile.tg` (`terragrunt-plan-render`) +- `./.github/actions/terragrunt` always uploads per-stack plan artifacts on `plan` and always downloads them on `apply_plan`, using the caller-provided `PLAN_ARTIFACT_S3_PREFIX` environment variable, so graph executors like `shared_infra.yml` do not need separate `./.github/actions/just` steps for those transfers +- saved infra-plan storage is intentionally split into two levels: + - one run-level metadata file at `/infra-plan-metadata/plan-metadata.json` + - one per-stack plan bundle under `/terragrunt-plan-/` +- plan artifact storage follows the same artifact environment split as ECR and build outputs: `dev` uses the `dev` code bucket, while non-`dev` environments read and write `terragrunt_plan/` in the shared `ci` code bucket +- `./.github/actions/terragrunt` skips `apply_plan` with a warning when the saved `terragrunt.plan.meta.json` reports `has_changes: false` - `./.github/actions/terragrunt` derives its plan artifact name from `tg_directory`, so callers do not need to pass artifact naming inputs -- if `apply_plan` is used across separate workflow runs, pass the earlier workflow `run_id` through `plan_artifact_run_id` and pass the GitHub token as a reusable-workflow secret with `actions: read` so both the shared metadata artifact and the plan artifacts can be downloaded -- for cross-run artifact recovery, scope `actions/download-artifact` explicitly to `repository: ${{ github.repository }}` rather than relying on implicit repository resolution through the reusable-workflow chain +- if `apply_plan` is used across separate workflow runs, pass the earlier workflow `run_id` through `plan_artifact_run_id`; the shared wrappers recover both metadata and per-stack plan files by deriving the matching `plan_artifact_s3_prefix` and reading from the shared code bucket under `terragrunt_plan///...` - if a cross-run apply should not ask the operator to re-enter versions or recompute artifact resolution, store both the input versions and the resolved reusable-workflow outputs in a metadata artifact during plan and recover them in the apply wrapper from the earlier `run_id` - keep `shared_infra.yml` as the pure graph executor and prefer handling metadata creation/recovery in the dedicated plan/apply wrappers - when using `./.github/actions/just`, check whether the caller needs the repo-root `justfile` or an explicit `justfile_path` @@ -153,6 +157,7 @@ Run these checks on every CI, workflow, or deploy-contract change. - keep the split `just` ownership clear: - repo-root `justfile` for local/developer commands - `justfile.ci` for read-only CI helpers + - `justfile.tg` for Terragrunt plan artifact helpers (render/upload/download) - `justfile.deploy` for mutating CI build and deploy steps ### Release Tagging Checks diff --git a/.github/workflows/dev_infra_apply.yml b/.github/workflows/dev_infra_apply.yml index d77cd1fd..1c088d5d 100644 --- a/.github/workflows/dev_infra_apply.yml +++ b/.github/workflows/dev_infra_apply.yml @@ -6,6 +6,7 @@ on: permissions: id-token: write contents: write + actions: read jobs: setup: diff --git a/.github/workflows/dev_infra_apply_from_plan.yml b/.github/workflows/dev_infra_apply_from_plan.yml index a0a82d0c..5c18b61a 100644 --- a/.github/workflows/dev_infra_apply_from_plan.yml +++ b/.github/workflows/dev_infra_apply_from_plan.yml @@ -20,4 +20,3 @@ jobs: with: environment: dev plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ github.token }} diff --git a/.github/workflows/dev_infra_plan_and_apply.yml b/.github/workflows/dev_infra_plan_and_apply.yml index 73890ea3..645a2721 100644 --- a/.github/workflows/dev_infra_plan_and_apply.yml +++ b/.github/workflows/dev_infra_plan_and_apply.yml @@ -42,4 +42,3 @@ jobs: with: environment: dev plan_artifact_run_id: ${{ github.run_id }} - github_token: ${{ github.token }} diff --git a/.github/workflows/prod_infra_apply.yml b/.github/workflows/prod_infra_apply.yml index 81f57ea5..f9f81e19 100644 --- a/.github/workflows/prod_infra_apply.yml +++ b/.github/workflows/prod_infra_apply.yml @@ -6,6 +6,7 @@ on: permissions: id-token: write contents: write + actions: read jobs: get_build: diff --git a/.github/workflows/prod_infra_apply_from_plan.yml b/.github/workflows/prod_infra_apply_from_plan.yml index 1eff5d1d..403a89b8 100644 --- a/.github/workflows/prod_infra_apply_from_plan.yml +++ b/.github/workflows/prod_infra_apply_from_plan.yml @@ -20,4 +20,3 @@ jobs: with: environment: prod plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ github.token }} diff --git a/.github/workflows/pull_request.yml b/.github/workflows/pull_request.yml index 7307c821..1c6cf8bd 100644 --- a/.github/workflows/pull_request.yml +++ b/.github/workflows/pull_request.yml @@ -151,7 +151,7 @@ jobs: timeout-minutes: 2 steps: - uses: actions/checkout@v6 - - uses: hashicorp/setup-terraform@v3 + - uses: hashicorp/setup-terraform@v4 - uses: autero1/action-terragrunt@v3.0.2 with: terragrunt-version: 0.45.10 diff --git a/.github/workflows/shared_infra.yml b/.github/workflows/shared_infra.yml index 3baff81b..69a03d1d 100644 --- a/.github/workflows/shared_infra.yml +++ b/.github/workflows/shared_infra.yml @@ -33,13 +33,8 @@ on: required: false type: string default: "apply" - plan_artifact_run_id: - description: "Optional workflow run ID used for apply-from-plan artifact downloads" - required: false - type: string - default: "" - github_token: - description: "Optional GitHub token used for cross-run artifact download in apply-from-plan mode" + plan_artifact_s3_prefix: + description: "Optional resolved S3 prefix used for saved plan artifacts" required: false type: string default: "" @@ -57,6 +52,7 @@ permissions: env: AWS_OIDC_ROLE_ARN: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/${{ vars.PROJECT_NAME }}-${{ inputs.environment }}-github-oidc-role DOMAIN_NAME: ${{ vars.DOMAIN_NAME }} + PLAN_ARTIFACT_S3_PREFIX: ${{ inputs.plan_artifact_s3_prefix }} TG_ACTION_LABEL: ${{ inputs.tg_action == 'apply' && 'Apply' || inputs.tg_action == 'plan' && 'Plan' || inputs.tg_action == 'apply_plan' && 'Apply planned' || inputs.tg_action == 'destroy' && 'Destroy' || inputs.tg_action == 'init' && 'Init' || 'Run' }} jobs: @@ -73,8 +69,6 @@ jobs: aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} tg_directory: infra/live/${{ inputs.environment }}/aws/oidc tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} worker_messaging: needs: oidc @@ -90,243 +84,227 @@ jobs: aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} tg_directory: infra/live/${{ inputs.environment }}/aws/worker_messaging tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - - observability: - needs: oidc - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: ${{ env.TG_ACTION_LABEL }} observability infra - uses: ./.github/actions/terragrunt - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/observability - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - - cognito: - needs: oidc - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: ${{ env.TG_ACTION_LABEL }} cognito infra - uses: ./.github/actions/terragrunt - env: - TF_VAR_domain_name: ${{ env.DOMAIN_NAME }} - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/cognito - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - frontend: - needs: - - network - - cognito - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: Get network infra - uses: ./.github/actions/terragrunt - id: get-network - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/network - tg_action: init - - - name: Get cognito infra - uses: ./.github/actions/terragrunt - id: get-cognito - env: - TF_VAR_domain_name: ${{ env.DOMAIN_NAME }} - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/cognito - tg_action: init - - - name: Get api invoke url - id: get_api_vars - env: - TG_OUTPUTS: ${{ steps.get-network.outputs.tg_outputs }} - run: | - echo "invoke_url=$(echo $TG_OUTPUTS | jq -r '.api_invoke_url.value')" >> $GITHUB_OUTPUT - - - name: Get cognito values - id: get_cognito_vars - env: - TG_OUTPUTS: ${{ steps.get-cognito.outputs.tg_outputs }} - run: | - echo "user_pool_id=$(echo "$TG_OUTPUTS" | jq -r '.user_pool_id.value')" >> "$GITHUB_OUTPUT" - echo "user_pool_client_id=$(echo "$TG_OUTPUTS" | jq -r '.user_pool_client_id.value')" >> "$GITHUB_OUTPUT" - echo "hosted_ui_url=$(echo "$TG_OUTPUTS" | jq -r '.hosted_ui_url.value')" >> "$GITHUB_OUTPUT" - echo "readonly_group_name=$(echo "$TG_OUTPUTS" | jq -r '.readonly_group_name.value')" >> "$GITHUB_OUTPUT" - - - name: ${{ env.TG_ACTION_LABEL }} frontend infra - uses: ./.github/actions/terragrunt - env: - TF_VAR_api_invoke_url: ${{ steps.get_api_vars.outputs.invoke_url }} - TF_VAR_domain_name: ${{ env.DOMAIN_NAME }} - TF_VAR_auth_user_pool_id: ${{ steps.get_cognito_vars.outputs.user_pool_id }} - TF_VAR_auth_user_pool_client_id: ${{ steps.get_cognito_vars.outputs.user_pool_client_id }} - TF_VAR_auth_hosted_ui_url: ${{ steps.get_cognito_vars.outputs.hosted_ui_url }} - TF_VAR_auth_readonly_group_name: ${{ steps.get_cognito_vars.outputs.readonly_group_name }} - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/frontend - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - - cluster: - needs: oidc - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: ${{ env.TG_ACTION_LABEL }} cluster infra - uses: ./.github/actions/terragrunt - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/cluster - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - - security: - needs: oidc - runs-on: ubuntu-latest - outputs: - postgres_sg: ${{ steps.get_security_outputs.outputs.postgres_sg }} - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: ${{ env.TG_ACTION_LABEL }} security infra - id: deploy-security - uses: ./.github/actions/terragrunt - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/security - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - - - name: Get security outputs - id: get_security_outputs - if: inputs.tg_action != 'destroy' - env: - TG_OUTPUTS: ${{ steps.deploy-security.outputs.tg_outputs }} - run: | - echo "postgres_sg=$(echo "$TG_OUTPUTS" | jq -r '.postgres_sg.value')" >> "$GITHUB_OUTPUT" - - database: - needs: - - oidc - - security - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: ${{ env.TG_ACTION_LABEL }} database infra - uses: ./.github/actions/terragrunt - env: - TF_VAR_database_security_group_id: ${{ needs.security.outputs.postgres_sg }} - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/database - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - - network: - needs: - - security - - cognito - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: ${{ env.TG_ACTION_LABEL }} network infra - uses: ./.github/actions/terragrunt - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/network - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - - lambdas: - needs: - - oidc - - security - - network - - database - - worker_messaging - runs-on: ubuntu-latest - strategy: - fail-fast: false # this is to prevent terraform lock issues - matrix: - value: ${{ fromJson(inputs.lambda_matrix) }} - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: ${{ env.TG_ACTION_LABEL }} ${{ matrix.value }} infra - uses: ./.github/actions/terragrunt - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/${{ matrix.value }} - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} - - services: - needs: - - oidc - - security - - cluster - - network - - database - - worker_messaging - runs-on: ubuntu-latest - strategy: - fail-fast: false - matrix: - value: ${{ fromJson(inputs.service_matrix) }} - steps: - - uses: actions/checkout@v6 - with: - ref: ${{ inputs.infra_version }} - - - name: ${{ env.TG_ACTION_LABEL }} ${{ matrix.value }} bootstrap service infra - uses: ./.github/actions/terragrunt - env: - TF_VAR_bootstrap: "true" - TF_VAR_bootstrap_image_uri: ${{ inputs.bootstrap_image_uri }} - with: - aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} - tg_directory: infra/live/${{ inputs.environment }}/aws/${{ matrix.value }} - tg_action: ${{ inputs.tg_action }} - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} + # observability: + # needs: oidc + # runs-on: ubuntu-latest + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: ${{ env.TG_ACTION_LABEL }} observability infra + # uses: ./.github/actions/terragrunt + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/observability + # tg_action: ${{ inputs.tg_action }} + + # cognito: + # needs: oidc + # runs-on: ubuntu-latest + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: ${{ env.TG_ACTION_LABEL }} cognito infra + # uses: ./.github/actions/terragrunt + # env: + # TF_VAR_domain_name: ${{ env.DOMAIN_NAME }} + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/cognito + # tg_action: ${{ inputs.tg_action }} + + # frontend: + # needs: + # - network + # - cognito + # runs-on: ubuntu-latest + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: Get network infra + # if: inputs.tg_action != 'apply_plan' + # uses: ./.github/actions/terragrunt + # id: get-network + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/network + # tg_action: init + + # - name: Get cognito infra + # if: inputs.tg_action != 'apply_plan' + # uses: ./.github/actions/terragrunt + # id: get-cognito + # env: + # TF_VAR_domain_name: ${{ env.DOMAIN_NAME }} + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/cognito + # tg_action: init + + # - name: Get api invoke url + # if: inputs.tg_action != 'apply_plan' + # id: get_api_vars + # env: + # TG_OUTPUTS: ${{ steps.get-network.outputs.tg_outputs }} + # run: | + # echo "invoke_url=$(echo $TG_OUTPUTS | jq -r '.api_invoke_url.value')" >> $GITHUB_OUTPUT + + # - name: Get cognito values + # if: inputs.tg_action != 'apply_plan' + # id: get_cognito_vars + # env: + # TG_OUTPUTS: ${{ steps.get-cognito.outputs.tg_outputs }} + # run: | + # echo "user_pool_id=$(echo "$TG_OUTPUTS" | jq -r '.user_pool_id.value')" >> "$GITHUB_OUTPUT" + # echo "user_pool_client_id=$(echo "$TG_OUTPUTS" | jq -r '.user_pool_client_id.value')" >> "$GITHUB_OUTPUT" + # echo "hosted_ui_url=$(echo "$TG_OUTPUTS" | jq -r '.hosted_ui_url.value')" >> "$GITHUB_OUTPUT" + # echo "readonly_group_name=$(echo "$TG_OUTPUTS" | jq -r '.readonly_group_name.value')" >> "$GITHUB_OUTPUT" + + # - name: ${{ env.TG_ACTION_LABEL }} frontend infra + # uses: ./.github/actions/terragrunt + # env: + # TF_VAR_api_invoke_url: ${{ inputs.tg_action == 'apply_plan' && '' || steps.get_api_vars.outputs.invoke_url }} + # TF_VAR_domain_name: ${{ env.DOMAIN_NAME }} + # TF_VAR_auth_user_pool_id: ${{ inputs.tg_action == 'apply_plan' && '' || steps.get_cognito_vars.outputs.user_pool_id }} + # TF_VAR_auth_user_pool_client_id: ${{ inputs.tg_action == 'apply_plan' && '' || steps.get_cognito_vars.outputs.user_pool_client_id }} + # TF_VAR_auth_hosted_ui_url: ${{ inputs.tg_action == 'apply_plan' && '' || steps.get_cognito_vars.outputs.hosted_ui_url }} + # TF_VAR_auth_readonly_group_name: ${{ inputs.tg_action == 'apply_plan' && '' || steps.get_cognito_vars.outputs.readonly_group_name }} + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/frontend + # tg_action: ${{ inputs.tg_action }} + + # cluster: + # needs: oidc + # runs-on: ubuntu-latest + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: ${{ env.TG_ACTION_LABEL }} cluster infra + # uses: ./.github/actions/terragrunt + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/cluster + # tg_action: ${{ inputs.tg_action }} + + # security: + # needs: oidc + # runs-on: ubuntu-latest + # outputs: + # postgres_sg: ${{ steps.get_security_outputs.outputs.postgres_sg }} + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: ${{ env.TG_ACTION_LABEL }} security infra + # id: deploy-security + # uses: ./.github/actions/terragrunt + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/security + # tg_action: ${{ inputs.tg_action }} + + # - name: Get security outputs + # id: get_security_outputs + # if: inputs.tg_action != 'destroy' + # env: + # TG_OUTPUTS: ${{ steps.deploy-security.outputs.tg_outputs }} + # run: | + # echo "postgres_sg=$(echo "$TG_OUTPUTS" | jq -r '.postgres_sg.value')" >> "$GITHUB_OUTPUT" + + # database: + # needs: + # - oidc + # - security + # runs-on: ubuntu-latest + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: ${{ env.TG_ACTION_LABEL }} database infra + # uses: ./.github/actions/terragrunt + # env: + # TF_VAR_database_security_group_id: ${{ needs.security.outputs.postgres_sg }} + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/database + # tg_action: ${{ inputs.tg_action }} + + # network: + # needs: + # - security + # - cognito + # runs-on: ubuntu-latest + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: ${{ env.TG_ACTION_LABEL }} network infra + # uses: ./.github/actions/terragrunt + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/network + # tg_action: ${{ inputs.tg_action }} + + # lambdas: + # needs: + # - oidc + # - security + # - network + # - database + # - worker_messaging + # runs-on: ubuntu-latest + # strategy: + # fail-fast: false # this is to prevent terraform lock issues + # matrix: + # value: ${{ fromJson(inputs.lambda_matrix) }} + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: ${{ env.TG_ACTION_LABEL }} ${{ matrix.value }} infra + # uses: ./.github/actions/terragrunt + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/${{ matrix.value }} + # tg_action: ${{ inputs.tg_action }} + + # services: + # needs: + # - oidc + # - security + # - cluster + # - network + # - database + # - worker_messaging + # runs-on: ubuntu-latest + # strategy: + # fail-fast: false + # matrix: + # value: ${{ fromJson(inputs.service_matrix) }} + # steps: + # - uses: actions/checkout@v6 + # with: + # ref: ${{ inputs.infra_version }} + + # - name: ${{ env.TG_ACTION_LABEL }} ${{ matrix.value }} bootstrap service infra + # uses: ./.github/actions/terragrunt + # env: + # TF_VAR_bootstrap: "true" + # TF_VAR_bootstrap_image_uri: ${{ inputs.bootstrap_image_uri }} + # with: + # aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + # tg_directory: infra/live/${{ inputs.environment }}/aws/${{ matrix.value }} + # tg_action: ${{ inputs.tg_action }} diff --git a/.github/workflows/shared_infra_apply.yml b/.github/workflows/shared_infra_apply.yml index ffe5277a..98ec6e1a 100644 --- a/.github/workflows/shared_infra_apply.yml +++ b/.github/workflows/shared_infra_apply.yml @@ -32,6 +32,7 @@ on: permissions: id-token: write contents: read + actions: read jobs: infra: diff --git a/.github/workflows/shared_infra_apply_from_plan.yml b/.github/workflows/shared_infra_apply_from_plan.yml index 7fb6abe8..6f27d600 100644 --- a/.github/workflows/shared_infra_apply_from_plan.yml +++ b/.github/workflows/shared_infra_apply_from_plan.yml @@ -11,16 +11,17 @@ on: description: "Workflow run ID used to recover metadata and plan artifacts" required: true type: string - github_token: - description: "GitHub token used for cross-run artifact download" - required: true - type: string permissions: id-token: write contents: read actions: read +env: + AWS_OIDC_ROLE_ARN: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/${{ vars.PROJECT_NAME }}-${{ inputs.environment }}-github-oidc-role + ARTIFACT_ENVIRONMENT: ${{ inputs.environment == 'dev' && 'dev' || 'ci' }} + ARTIFACT_AWS_OIDC_ROLE_ARN: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/${{ vars.PROJECT_NAME }}-${{ inputs.environment == 'dev' && 'dev' || 'ci' }}-github-oidc-role + jobs: metadata: runs-on: ubuntu-latest @@ -29,22 +30,51 @@ jobs: code_bucket: ${{ steps.read_metadata.outputs.code_bucket }} lambda_matrix: ${{ steps.read_metadata.outputs.lambda_matrix }} bootstrap_image_uri: ${{ steps.read_metadata.outputs.bootstrap_image_uri }} + plan_artifact_s3_prefix: ${{ steps.plan_artifact_s3_prefix.outputs.just_outputs }} service_matrix: ${{ steps.read_metadata.outputs.service_matrix }} steps: - - name: Download plan metadata artifact - uses: actions/download-artifact@v8 + - uses: actions/checkout@v6 + + - name: Get shared code bucket outputs + uses: ./.github/actions/terragrunt + id: code_action + with: + aws_oidc_role_arn: ${{ env.ARTIFACT_AWS_OIDC_ROLE_ARN }} + tg_directory: infra/live/${{ env.ARTIFACT_ENVIRONMENT }}/aws/code_bucket + tg_action: init + + - name: Get bucket name + id: get_bucket_name + env: + TG_OUTPUTS: ${{ steps.code_action.outputs.tg_outputs }} + run: | + echo "bucket=$(echo "$TG_OUTPUTS" | jq -r '.bucket.value // empty')" >> "$GITHUB_OUTPUT" + + - name: Get plan artifact S3 prefix + id: plan_artifact_s3_prefix + uses: ./.github/actions/just + env: + BUCKET_NAME: ${{ steps.get_bucket_name.outputs.bucket }} + ENVIRONMENT: ${{ inputs.environment }} + RUN_ID: ${{ inputs.plan_artifact_run_id }} + with: + justfile_path: justfile.tg + just_action: terragrunt-plan-base-s3-prefix + + - name: Download plan metadata from S3 + uses: ./.github/actions/just + env: + PLAN_ARTIFACT_S3_PREFIX: ${{ steps.plan_artifact_s3_prefix.outputs.just_outputs }} with: - name: infra-plan-metadata - path: . - repository: ${{ github.repository }} - github-token: ${{ inputs.github_token }} - run-id: ${{ inputs.plan_artifact_run_id }} + aws_oidc_role_arn: ${{ env.ARTIFACT_AWS_OIDC_ROLE_ARN }} + justfile_path: justfile.ci + just_action: infra-plan-metadata-download - name: Check plan metadata artifact shell: bash run: | if [ ! -f plan-metadata.json ]; then - echo "::error title=Missing plan metadata artifact::Expected 'plan-metadata.json' from artifact 'infra-plan-metadata' in run '${{ inputs.plan_artifact_run_id }}', but it was not downloaded." + echo "::error title=Missing plan metadata artifact::Expected 'plan-metadata.json' at ${{ steps.plan_artifact_s3_prefix.outputs.just_outputs }}/infra-plan-metadata/plan-metadata.json, but it was not downloaded." exit 1 fi @@ -70,5 +100,4 @@ jobs: bootstrap_image_uri: ${{ needs.metadata.outputs.bootstrap_image_uri }} service_matrix: ${{ needs.metadata.outputs.service_matrix }} tg_action: apply_plan - plan_artifact_run_id: ${{ inputs.plan_artifact_run_id }} - github_token: ${{ inputs.github_token }} + plan_artifact_s3_prefix: ${{ needs.metadata.outputs.plan_artifact_s3_prefix }} diff --git a/.github/workflows/shared_infra_plan.yml b/.github/workflows/shared_infra_plan.yml index 733147b4..508b6e87 100644 --- a/.github/workflows/shared_infra_plan.yml +++ b/.github/workflows/shared_infra_plan.yml @@ -38,10 +38,30 @@ permissions: contents: read actions: read +env: + AWS_OIDC_ROLE_ARN: arn:aws:iam::${{ vars.AWS_ACCOUNT_ID }}:role/${{ vars.PROJECT_NAME }}-${{ inputs.environment }}-github-oidc-role + jobs: metadata: runs-on: ubuntu-latest + outputs: + plan_artifact_s3_prefix: ${{ steps.plan_artifact_s3_prefix.outputs.just_outputs }} steps: + - uses: actions/checkout@v6 + with: + ref: ${{ inputs.infra_version }} + + - name: Get plan artifact S3 prefix + id: plan_artifact_s3_prefix + uses: ./.github/actions/just + env: + BUCKET_NAME: ${{ inputs.code_bucket }} + ENVIRONMENT: ${{ inputs.environment }} + RUN_ID: ${{ github.run_id }} + with: + justfile_path: justfile.tg + just_action: terragrunt-plan-base-s3-prefix + - name: Write plan metadata from workflow inputs shell: bash run: | @@ -55,12 +75,14 @@ jobs: } EOF - - name: Upload plan metadata artifact - uses: actions/upload-artifact@v7 + - name: Upload plan metadata to S3 + uses: ./.github/actions/just + env: + PLAN_ARTIFACT_S3_PREFIX: ${{ steps.plan_artifact_s3_prefix.outputs.just_outputs }} with: - name: infra-plan-metadata - path: plan-metadata.json - if-no-files-found: error + aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} + justfile_path: justfile.ci + just_action: infra-plan-metadata-upload infra: needs: @@ -74,6 +96,7 @@ jobs: bootstrap_image_uri: ${{ inputs.bootstrap_image_uri }} service_matrix: ${{ inputs.service_matrix }} tg_action: plan + plan_artifact_s3_prefix: ${{ needs.metadata.outputs.plan_artifact_s3_prefix }} plan_context: name: Plan Context diff --git a/.github/workflows/shared_infra_releases.yml b/.github/workflows/shared_infra_releases.yml index 86ac280f..25f0db4f 100644 --- a/.github/workflows/shared_infra_releases.yml +++ b/.github/workflows/shared_infra_releases.yml @@ -84,9 +84,42 @@ jobs: with: ref: ${{ inputs.infra_version }} + - name: Read Lambda artifact dir + id: lambda_artifact_dir + uses: ./.github/actions/just + with: + justfile_path: justfile.ci + just_action: code-bucket-get-lambda-artifact-dir + + - name: Read frontend artifact dir + id: frontend_artifact_dir + uses: ./.github/actions/just + with: + justfile_path: justfile.ci + just_action: code-bucket-get-frontend-artifact-dir + + - name: Read AppSpec artifact dir + id: appspec_artifact_dir + uses: ./.github/actions/just + with: + justfile_path: justfile.ci + just_action: code-bucket-get-appspec-artifact-dir + + - name: Read infra plan dir + id: infra_plan_dir + uses: ./.github/actions/just + with: + justfile_path: justfile.ci + just_action: code-bucket-get-infra-plan-dir + - name: Deploy build artifact bucket id: deploy_bucket uses: ./.github/actions/terragrunt + env: + TF_VAR_lambda_artifact_dir: ${{ steps.lambda_artifact_dir.outputs.just_outputs }} + TF_VAR_frontend_artifact_dir: ${{ steps.frontend_artifact_dir.outputs.just_outputs }} + TF_VAR_appspec_artifact_dir: ${{ steps.appspec_artifact_dir.outputs.just_outputs }} + TF_VAR_infra_plan_dir: ${{ steps.infra_plan_dir.outputs.just_outputs }} with: aws_oidc_role_arn: ${{ env.AWS_OIDC_ROLE_ARN }} tg_directory: infra/live/${{ inputs.environment }}/aws/code_bucket diff --git a/README.md b/README.md index d3d1bce9..3a064f1d 100644 --- a/README.md +++ b/README.md @@ -179,6 +179,8 @@ Infrastructure apply and feature-code rollout are intentionally decoupled in thi - `*_infra` workflows apply infrastructure only - `*_code` workflows deploy feature code only - code deploy workflows publish the real Lambda versions and ECS task revisions into that pre-created deploy surface +- saved infra plans are stored in the shared S3 code bucket under `terragrunt_plan///...`, using the same artifact split as build outputs: `dev` writes to the `dev` code bucket and non-`dev` environments reuse the `ci` code bucket +- Code artifact retention and infra-plan retention are configured separately in the shared code bucket module - rerunning infrastructure apply does not roll out new feature code - the shared Lambda and ECS module READMEs are the canonical source for bootstrap, rollout, and rollback details for each runtime shape - detailed workflow contracts, reusable-workflow inputs, repo-local action behavior, and `justfile_path` rules live in [.github/docs/README.md](.github/docs/README.md) diff --git a/infra/README.md b/infra/README.md index 6cd8ed32..a7f407e9 100644 --- a/infra/README.md +++ b/infra/README.md @@ -32,7 +32,9 @@ Shared artifact names also follow environment-aware conventions from `infra/root - shared artifact base: `dev -> ...-dev`, otherwise `...-ci` - code bucket: `-code` -- ECS ECR repository: `-ecs-worker` +- ECS ECR repository: `-ecr` +- saved Terragrunt plan artifacts: `s3:///terragrunt_plan///...` +- code-bucket lifecycle inputs: `code_artifact_expiration_days` for deployable code artifacts and `infra_plan_artifact_expiration_days` for `terragrunt_plan/` So a stack at: @@ -119,6 +121,7 @@ That `containers/lib` directory is helper code only and is not treated as a depl - build workflows produce Lambda zips and container images - `*_infra` wrappers need the inputs required to apply infra safely, such as directory-derived stack matrices and any artifact-derived bootstrap references - in `prod`, the `*_infra` wrappers read shared artifact resources from `ci` but only apply service and task stacks in `prod` +- saved `plan` / `apply_plan` artifacts live in the shared code bucket under `terragrunt_plan///...`; `dev` uses the `dev` code bucket, while non-`dev` environments reuse the shared `ci` code bucket - deploy workflows: - publish Lambda versions and use Lambda CodeDeploy - optionally invoke the `migrations` Lambda when it is part of the Lambda deploy matrix diff --git a/infra/live/global_vars.hcl b/infra/live/global_vars.hcl index 71a42684..56cecd78 100644 --- a/infra/live/global_vars.hcl +++ b/infra/live/global_vars.hcl @@ -27,12 +27,14 @@ locals { "route53:*", "cognito-idp:*", ] - container_port = 80 + code_artifact_expiration_days = 0 + infra_plan_artifact_expiration_days = 30 } inputs = { - vpc_name = local.vpc_name - aws_region = local.aws_region - allowed_role_actions = local.allowed_role_actions - container_port = local.container_port + vpc_name = local.vpc_name + aws_region = local.aws_region + allowed_role_actions = local.allowed_role_actions + code_artifact_expiration_days = local.code_artifact_expiration_days + infra_plan_artifact_expiration_days = local.infra_plan_artifact_expiration_days } diff --git a/infra/modules/aws/_shared/code_bucket/README.md b/infra/modules/aws/_shared/code_bucket/README.md index c131c038..e5ee176a 100644 --- a/infra/modules/aws/_shared/code_bucket/README.md +++ b/infra/modules/aws/_shared/code_bucket/README.md @@ -7,6 +7,22 @@ Shared S3 bucket for deployable artifacts. - Lambda zip storage - frontend bundle storage - ECS AppSpec storage for CodeDeploy +- Terragrunt saved plan artifacts under the `terragrunt_plan/` prefix + +## Inputs That Change Behavior + +- `lambda_artifact_dir` +- `frontend_artifact_dir` +- `appspec_artifact_dir` +- `infra_plan_dir` +- `code_artifact_expiration_days` +- `infra_plan_artifact_expiration_days` + +## Decision Rules + +- `dev` keeps its own code bucket and stores saved Terragrunt plans there +- non-`dev` environments reuse the shared `ci` code bucket for both release artifacts and saved Terragrunt plans +- lifecycle retention is prefix-scoped: code artifact cleanup applies to `lambda_artifact_dir/`, `frontend_artifact_dir/`, and `appspec_artifact_dir/`, while saved plan cleanup applies only to `infra_plan_dir/` ## Key outputs diff --git a/infra/modules/aws/_shared/code_bucket/main.tf b/infra/modules/aws/_shared/code_bucket/main.tf index ad16b9b8..eeae6df7 100644 --- a/infra/modules/aws/_shared/code_bucket/main.tf +++ b/infra/modules/aws/_shared/code_bucket/main.tf @@ -11,17 +11,52 @@ resource "aws_s3_bucket_ownership_controls" "code" { } } -resource "aws_s3_bucket_lifecycle_configuration" "delete_old_files" { - count = var.s3_expiration_days > 0 ? 1 : 0 +locals { + lifecycle_rules = { + code_artifacts_lambdas = { + days = var.code_artifact_expiration_days + prefix = "${var.lambda_artifact_dir}/" + } + code_artifacts_frontend = { + days = var.code_artifact_expiration_days + prefix = "${var.frontend_artifact_dir}/" + } + code_artifacts_appspec = { + days = var.code_artifact_expiration_days + prefix = "${var.appspec_artifact_dir}/" + } + terragrunt_plans = { + days = var.infra_plan_artifact_expiration_days + prefix = "${var.infra_plan_dir}/" + } + } +} + +resource "aws_s3_bucket_lifecycle_configuration" "managed_artifact_retention" { + count = length({ + for name, rule in local.lifecycle_rules : name => rule + if rule.days > 0 + }) > 0 ? 1 : 0 bucket = aws_s3_bucket.code.id - rule { - id = "delete-expired-objects" - status = "Enabled" + dynamic "rule" { + for_each = { + for name, lifecycle_rule in local.lifecycle_rules : name => lifecycle_rule + if lifecycle_rule.days > 0 + } + + content { + id = "delete-${rule.key}" + status = "Enabled" + + filter { + prefix = rule.value.prefix + } - expiration { - days = var.s3_expiration_days + expiration { + days = rule.value.days + } } } } diff --git a/infra/modules/aws/_shared/code_bucket/variables.tf b/infra/modules/aws/_shared/code_bucket/variables.tf index 7ad8beb8..69ad48fd 100644 --- a/infra/modules/aws/_shared/code_bucket/variables.tf +++ b/infra/modules/aws/_shared/code_bucket/variables.tf @@ -5,8 +5,38 @@ variable "code_bucket" { } ### end of static vars set in root.hcl ### -variable "s3_expiration_days" { - description = "Number of days before objects are deleted (set to 0 to disable)" +variable "lambda_artifact_dir" { + description = "Top-level S3 prefix used for Lambda zip artifacts" + type = string + default = "lambdas" +} + +variable "frontend_artifact_dir" { + description = "Top-level S3 prefix used for frontend build artifacts" + type = string + default = "frontend" +} + +variable "appspec_artifact_dir" { + description = "Top-level S3 prefix used for AppSpec deployment artifacts" + type = string + default = "appspec" +} + +variable "infra_plan_dir" { + description = "Top-level S3 prefix used for saved Terragrunt plan artifacts" + type = string + default = "terragrunt_plan" +} + +variable "code_artifact_expiration_days" { + description = "Number of days before deployable code artifacts under lambdas/, frontend/, and appspec/ are deleted (set to 0 to disable)" + type = number + default = 0 +} + +variable "infra_plan_artifact_expiration_days" { + description = "Number of days before saved Terragrunt plan artifacts under terragrunt_plan/ are deleted (set to 0 to disable)" type = number default = 0 } diff --git a/infra/modules/aws/network/variables.tf b/infra/modules/aws/network/variables.tf index aa66a8ce..6242bad8 100644 --- a/infra/modules/aws/network/variables.tf +++ b/infra/modules/aws/network/variables.tf @@ -21,7 +21,8 @@ variable "vpc_name" { } variable "container_port" { - type = number + type = number + default = 80 } variable "local_tunnel" { diff --git a/infra/modules/aws/security/variables.tf b/infra/modules/aws/security/variables.tf index 0bea6c1a..b7f8dd5e 100644 --- a/infra/modules/aws/security/variables.tf +++ b/infra/modules/aws/security/variables.tf @@ -13,7 +13,8 @@ variable "vpc_name" { } variable "container_port" { - type = number + type = number + default = 80 } variable "additional_listener_port" { diff --git a/infra/modules/aws/service_worker/variables.tf b/infra/modules/aws/service_worker/variables.tf index 75980e2f..4b065255 100644 --- a/infra/modules/aws/service_worker/variables.tf +++ b/infra/modules/aws/service_worker/variables.tf @@ -26,7 +26,8 @@ variable "vpc_name" { } variable "container_port" { - type = number + type = number + default = 80 } variable "root_path" { diff --git a/infra/modules/aws/task_worker/variables.tf b/infra/modules/aws/task_worker/variables.tf index 086d02c8..c4bd09a5 100644 --- a/infra/modules/aws/task_worker/variables.tf +++ b/infra/modules/aws/task_worker/variables.tf @@ -21,7 +21,8 @@ variable "ecr_repository_name" { ### end of static vars set in root.hcl ### variable "container_port" { - type = number + type = number + default = 80 } variable "cpu" { diff --git a/infra/root.hcl b/infra/root.hcl index ce19e3e2..6e713ae9 100644 --- a/infra/root.hcl +++ b/infra/root.hcl @@ -1,6 +1,4 @@ locals { - github_token = get_env("GITHUB_TOKEN", "not_set") - git_remote = run_cmd("--terragrunt-quiet", "git", "remote", "get-url", "origin") github_repo = regex("[/:]([-0-9_A-Za-z]*/[-0-9_A-Za-z]*)[^/]*$", local.git_remote)[0] repo_owner = split("/", local.github_repo)[0] @@ -24,7 +22,6 @@ locals { state_bucket = "${local.base_reference}-tfstate" state_key = "${local.environment}/${local.provider}/${local.module}/terraform.tfstate" state_lock_table = "${local.project_name}-tf-lockid" - # separate shared artifact resources when dev, otherwise ci artifact_base = local.environment == "dev" ? "${local.base_reference}-${local.environment}" : "${local.base_reference}-ci" code_bucket = "${local.artifact_base}-code" diff --git a/justfile b/justfile index 1fd396e6..9763c9ee 100644 --- a/justfile +++ b/justfile @@ -3,6 +3,8 @@ _default: @just --list @printf '\nCI recipes (`just --justfile justfile.ci --list`):\n' @just --justfile justfile.ci --list + @printf '\nTerragrunt recipes (`just --justfile justfile.tg --list`):\n' + @just --justfile justfile.tg --list @printf '\nDeploy recipes (`just --justfile justfile.deploy --list`):\n' @just --justfile justfile.deploy --list @@ -10,6 +12,31 @@ _default: PROJECT_DIR := justfile_directory() LAMBDA_DIR := "lambdas" FRONTEND_DIR := "frontend" +CONTAINERS_DIR := "containers" +APPSPEC_DIR := "appspec" +INFRA_PLAN_DIR := "terragrunt_plan" +EXTRA_CONTAINER_DIRECTORIES := "[\"debug\",\"otel_collector\"]" +NON_SERVICE_CONTAINER_DIRECTORIES := "[\"lib\",\"_shared\"]" + + +# Return the Lambda artifact directory name. +code-bucket-get-lambda-artifact-dir: + @echo {{LAMBDA_DIR}} + + +# Return the frontend artifact directory name. +code-bucket-get-frontend-artifact-dir: + @echo {{FRONTEND_DIR}} + + +# Return the infra plan artifact directory name. +code-bucket-get-infra-plan-dir: + @echo {{INFRA_PLAN_DIR}} + + +# Return the AppSpec artifact directory name. +code-bucket-get-appspec-artifact-dir: + @echo {{APPSPEC_DIR}} # Delete local git branches whose upstream refs have gone away. diff --git a/justfile.ci b/justfile.ci index 81599f1f..3a0d5dcb 100644 --- a/justfile.ci +++ b/justfile.ci @@ -2,11 +2,14 @@ # This file is for discovery, validation, artifact existence checks, and matrix shaping. # Do not put AWS-mutating rollout or deploy commands here. -PROJECT_DIR := justfile_directory() -LAMBDA_DIR := "lambdas" -CONTAINERS_DIR := "containers" -EXTRA_CONTAINER_DIRECTORIES := "[\"debug\",\"otel_collector\"]" -NON_SERVICE_CONTAINER_DIRECTORIES := "[\"lib\",\"_shared\"]" +PROJECT_DIR := `just --justfile justfile --evaluate PROJECT_DIR` +LAMBDA_DIR := `just --justfile justfile --evaluate LAMBDA_DIR` +FRONTEND_DIR := `just --justfile justfile --evaluate FRONTEND_DIR` +APPSPEC_DIR := `just --justfile justfile --evaluate APPSPEC_DIR` +CONTAINERS_DIR := `just --justfile justfile --evaluate CONTAINERS_DIR` +INFRA_PLAN_DIR := `just --justfile justfile --evaluate INFRA_PLAN_DIR` +EXTRA_CONTAINER_DIRECTORIES := `just --justfile justfile --evaluate EXTRA_CONTAINER_DIRECTORIES` +NON_SERVICE_CONTAINER_DIRECTORIES := `just --justfile justfile --evaluate NON_SERVICE_CONTAINER_DIRECTORIES` # Run `tflint` across Terraform module directories. @@ -105,6 +108,56 @@ get-version-file-keys: | jq -s -c . +# Upload shared infra plan metadata to the shared code bucket. +infra-plan-metadata-upload: + #!/usr/bin/env bash + set -euo pipefail + + if [[ -z "${PLAN_ARTIFACT_S3_PREFIX:-}" ]]; then + echo "❌ PLAN_ARTIFACT_S3_PREFIX environment variable is not set." + exit 1 + fi + + artifact_s3_prefix="$(just --justfile "{{PROJECT_DIR}}/justfile.tg" terragrunt-plan-base-s3-prefix)" + + aws s3 cp "plan-metadata.json" "${artifact_s3_prefix}/infra-plan-metadata/plan-metadata.json" + + +# Download shared infra plan metadata from the shared code bucket. +infra-plan-metadata-download: + #!/usr/bin/env bash + set -euo pipefail + + if [[ -z "${PLAN_ARTIFACT_S3_PREFIX:-}" ]]; then + echo "❌ PLAN_ARTIFACT_S3_PREFIX environment variable is not set." + exit 1 + fi + + artifact_s3_prefix="$(just --justfile "{{PROJECT_DIR}}/justfile.tg" terragrunt-plan-base-s3-prefix)" + + aws s3 cp "${artifact_s3_prefix}/infra-plan-metadata/plan-metadata.json" "plan-metadata.json" + + +# Return the Lambda artifact directory name from the repo-root justfile. +code-bucket-get-lambda-artifact-dir: + @just --justfile "{{PROJECT_DIR}}/justfile" code-bucket-get-lambda-artifact-dir + + +# Return the frontend artifact directory name from the repo-root justfile. +code-bucket-get-frontend-artifact-dir: + @just --justfile "{{PROJECT_DIR}}/justfile" code-bucket-get-frontend-artifact-dir + + +# Return the infra plan artifact directory name from the repo-root justfile. +code-bucket-get-infra-plan-dir: + @just --justfile "{{PROJECT_DIR}}/justfile" code-bucket-get-infra-plan-dir + + +# Return the AppSpec artifact directory name from the repo-root justfile. +code-bucket-get-appspec-artifact-dir: + @just --justfile "{{PROJECT_DIR}}/justfile" code-bucket-get-appspec-artifact-dir + + # List ECR image tag prefixes for a published version. get-ecr-version-images: #!/usr/bin/env bash diff --git a/justfile.deploy b/justfile.deploy index 47d169c0..1f989600 100644 --- a/justfile.deploy +++ b/justfile.deploy @@ -2,9 +2,10 @@ # This file is for build/publish/deploy/invalidate steps that change remote state. # Keep read-only CI helpers in `justfile.ci`. -PROJECT_DIR := justfile_directory() -LAMBDA_DIR := "lambdas" -FRONTEND_DIR := "frontend" +PROJECT_DIR := `just --justfile justfile --evaluate PROJECT_DIR` +LAMBDA_DIR := `just --justfile justfile --evaluate LAMBDA_DIR` +FRONTEND_DIR := `just --justfile justfile --evaluate FRONTEND_DIR` +APPSPEC_DIR := `just --justfile justfile --evaluate APPSPEC_DIR` # Invoke a deployed Lambda and fail if it does not return HTTP 200. diff --git a/justfile.tg b/justfile.tg new file mode 100644 index 00000000..9491885b --- /dev/null +++ b/justfile.tg @@ -0,0 +1,167 @@ +# Terragrunt plan artifact helpers. +# This file is for producing, downloading, and uploading saved Terragrunt plan files. + +PROJECT_DIR := `just --justfile justfile --evaluate PROJECT_DIR` +INFRA_PLAN_DIR := `just --justfile justfile --evaluate INFRA_PLAN_DIR` + +PLAN_FILE := "terragrunt.tfplan" +PLAN_TEXT_FILE := "terragrunt.plan.txt" +PLAN_META_FILE := "terragrunt.plan.meta.json" + + +# Render Terragrunt plan sidecars (plan text + metadata) for an existing binary plan. +# +# Expected environment variables: +# - TG_DIRECTORY: directory containing the saved plan file +# - TG_PLAN_EXIT_CODE: detailed-exitcode from `terragrunt plan` (0 or 2) +terragrunt-plan-render: + #!/usr/bin/env bash + set -euo pipefail + + if [[ -z "${TG_DIRECTORY:-}" ]]; then + echo "❌ TG_DIRECTORY environment variable is not set." + exit 1 + fi + + if [[ -z "${TG_PLAN_EXIT_CODE:-}" ]]; then + echo "❌ TG_PLAN_EXIT_CODE environment variable is not set." + exit 1 + fi + + cd "$TG_DIRECTORY" + + PLAN_PATH="$(pwd)/{{PLAN_FILE}}" + PLAN_TEXT_PATH="$(pwd)/{{PLAN_TEXT_FILE}}" + PLAN_META_PATH="$(pwd)/{{PLAN_META_FILE}}" + + if [[ ! -f "$PLAN_PATH" ]]; then + echo "❌ Expected plan file '$PLAN_PATH' was not found." + exit 1 + fi + + terragrunt show -no-color "$PLAN_PATH" > "$PLAN_TEXT_PATH" + + jq -n \ + --arg tg_directory "$TG_DIRECTORY" \ + --argjson exit_code "$TG_PLAN_EXIT_CODE" \ + --argjson has_changes "$([ "$TG_PLAN_EXIT_CODE" -eq 2 ] && echo true || echo false)" \ + '{tg_directory: $tg_directory, exit_code: $exit_code, has_changes: $has_changes}' \ + > "$PLAN_META_PATH" + + echo "Terragrunt binary plan path: $PLAN_PATH" + ls -l "$PLAN_PATH" + echo "Terragrunt rendered plan path: $PLAN_TEXT_PATH" + cat "$PLAN_TEXT_PATH" + echo "Terragrunt plan metadata path: $PLAN_META_PATH" + cat "$PLAN_META_PATH" + + +# Derive the shared S3 base prefix for Terragrunt plan artifacts. +terragrunt-plan-base-prefix: + #!/usr/bin/env bash + set -euo pipefail + + if [[ -n "${PLAN_ARTIFACT_S3_PREFIX:-}" ]]; then + echo "${PLAN_ARTIFACT_S3_PREFIX#s3://*/}" + exit 0 + fi + + if [[ -z "${ENVIRONMENT:-}" ]]; then + echo "❌ ENVIRONMENT environment variable is not set." + exit 1 + fi + + if [[ -z "${RUN_ID:-}" ]]; then + echo "❌ RUN_ID environment variable is not set." + exit 1 + fi + + infra_plan_dir="${TF_VAR_infra_plan_dir:-{{INFRA_PLAN_DIR}}}" + + echo "${infra_plan_dir}/${ENVIRONMENT}/${RUN_ID}" + + +# Derive the shared S3 URI prefix for Terragrunt plan artifacts. +terragrunt-plan-base-s3-prefix: + #!/usr/bin/env bash + set -euo pipefail + + if [[ -n "${PLAN_ARTIFACT_S3_PREFIX:-}" ]]; then + echo "${PLAN_ARTIFACT_S3_PREFIX}" + exit 0 + fi + + if [[ -z "${BUCKET_NAME:-}" ]]; then + echo "❌ BUCKET_NAME environment variable is not set." + exit 1 + fi + + artifact_base_prefix="$(just --justfile "{{PROJECT_DIR}}/justfile.tg" terragrunt-plan-base-prefix)" + + echo "s3://${BUCKET_NAME}/${artifact_base_prefix}" + + +# Derive the shared S3 prefix for Terragrunt plan artifacts. +terragrunt-plan-prefix: + #!/usr/bin/env bash + set -euo pipefail + + if [[ -z "${TG_DIRECTORY:-}" ]]; then + echo "❌ TG_DIRECTORY environment variable is not set." + exit 1 + fi + + artifact_base_prefix="$(just --justfile "{{PROJECT_DIR}}/justfile.tg" terragrunt-plan-base-prefix)" + sanitized_dir="$(echo "$TG_DIRECTORY" | tr '/.' '--')" + artifact_name="terragrunt-plan-${sanitized_dir}" + artifact_prefix="${artifact_base_prefix}/${artifact_name}" + + echo "$artifact_prefix" + + +# Download saved Terragrunt plan files for a stack from the shared code bucket. +terragrunt-plan-download: + #!/usr/bin/env bash + set -euo pipefail + + if [[ -z "${PLAN_ARTIFACT_S3_PREFIX:-}" && -z "${BUCKET_NAME:-}" ]]; then + echo "❌ PLAN_ARTIFACT_S3_PREFIX or BUCKET_NAME environment variable is required." + exit 1 + fi + + if [[ -z "${TG_DIRECTORY:-}" ]]; then + echo "❌ TG_DIRECTORY environment variable is not set." + exit 1 + fi + + artifact_s3_base_prefix="$(just --justfile "{{PROJECT_DIR}}/justfile.tg" terragrunt-plan-base-s3-prefix)" + artifact_prefix="$(just --justfile "{{PROJECT_DIR}}/justfile.tg" terragrunt-plan-prefix)" + + mkdir -p "$TG_DIRECTORY" + + aws s3 cp "${artifact_s3_base_prefix}/${artifact_prefix##*/}/{{PLAN_FILE}}" "${TG_DIRECTORY}/{{PLAN_FILE}}" + aws s3 cp "${artifact_s3_base_prefix}/${artifact_prefix##*/}/{{PLAN_TEXT_FILE}}" "${TG_DIRECTORY}/{{PLAN_TEXT_FILE}}" + aws s3 cp "${artifact_s3_base_prefix}/${artifact_prefix##*/}/{{PLAN_META_FILE}}" "${TG_DIRECTORY}/{{PLAN_META_FILE}}" + + +# Upload saved Terragrunt plan files for a stack to the shared code bucket. +terragrunt-plan-upload: + #!/usr/bin/env bash + set -euo pipefail + + if [[ -z "${PLAN_ARTIFACT_S3_PREFIX:-}" && -z "${BUCKET_NAME:-}" ]]; then + echo "❌ PLAN_ARTIFACT_S3_PREFIX or BUCKET_NAME environment variable is required." + exit 1 + fi + + if [[ -z "${TG_DIRECTORY:-}" ]]; then + echo "❌ TG_DIRECTORY environment variable is not set." + exit 1 + fi + + artifact_s3_base_prefix="$(just --justfile "{{PROJECT_DIR}}/justfile.tg" terragrunt-plan-base-s3-prefix)" + artifact_prefix="$(just --justfile "{{PROJECT_DIR}}/justfile.tg" terragrunt-plan-prefix)" + + aws s3 cp "${TG_DIRECTORY}/{{PLAN_FILE}}" "${artifact_s3_base_prefix}/${artifact_prefix##*/}/{{PLAN_FILE}}" + aws s3 cp "${TG_DIRECTORY}/{{PLAN_TEXT_FILE}}" "${artifact_s3_base_prefix}/${artifact_prefix##*/}/{{PLAN_TEXT_FILE}}" + aws s3 cp "${TG_DIRECTORY}/{{PLAN_META_FILE}}" "${artifact_s3_base_prefix}/${artifact_prefix##*/}/{{PLAN_META_FILE}}"