Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 32 additions & 32 deletions .github/scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,8 @@ These scripts handle complex operations that would otherwise make workflow files

### generate-pg-secrets.sh

**Status**: Legacy - No longer used
**Reason**: PostgreSQL configuration now handled via Terraform variables (`enable_postgresql_setup`)
**Status**: Legacy - No longer used
**Reason**: PostgreSQL configuration now handled via Terraform variables (`enable_postgresql_setup`)
**Replacement**: Configure PostgreSQL in `terraform/implementations/{cloud}/{component}/{cloud}.tfvars`

## Placeholder Scripts
Expand Down Expand Up @@ -157,36 +157,36 @@ Some scripts are empty placeholders for future functionality:

```mermaid
graph TD
A[Workflow Start] --> B[setup-gpg.sh]
B --> C{Backend Type?}
C -->|Local| D[decrypt-state.sh]
C -->|Remote| E[setup-cloud-storage.sh]
D --> F[configure-backend.sh]
E --> F
F --> G[Terraform Operations]
G --> H[encrypt-state.sh]
G --> I[cleanup-state-locking.sh]
H --> J[Workflow End]
I --> J
A[Workflow Start] --> B[setup-gpg.sh]
B --> C{Backend Type?}
C -->|Local| D[decrypt-state.sh]
C -->|Remote| E[setup-cloud-storage.sh]
D --> F[configure-backend.sh]
E --> F
F --> G[Terraform Operations]
G --> H[encrypt-state.sh]
G --> I[cleanup-state-locking.sh]
H --> J[Workflow End]
I --> J
```

## Directory Structure

```
.github/scripts/
├── README.md # This file - scripts documentation
├── configure-backend.sh # Backend configuration generation
├── setup-cloud-storage.sh # Remote storage setup
├── encrypt-state.sh # GPG state encryption
├── decrypt-state.sh # GPG state decryption
├── setup-gpg.sh # GPG environment setup
├── cleanup-state-locking.sh # State lock cleanup
├── generate-pg-secrets.sh # Legacy PostgreSQL secrets
├── test-infrastructure.sh # Comprehensive testing
├── validate-workflow-integration.sh # Workflow validation
├── test-*.sh # Various test scripts
├── setup-s3-backend.sh # Empty placeholder
└── setup-remote-storage.sh # Empty placeholder
├── README.md # This file - scripts documentation
├── configure-backend.sh # Backend configuration generation
├── setup-cloud-storage.sh # Remote storage setup
├── encrypt-state.sh # GPG state encryption
├── decrypt-state.sh # GPG state decryption
├── setup-gpg.sh # GPG environment setup
├── cleanup-state-locking.sh # State lock cleanup
├── generate-pg-secrets.sh # Legacy PostgreSQL secrets
├── test-infrastructure.sh # Comprehensive testing
├── validate-workflow-integration.sh # Workflow validation
├── test-*.sh # Various test scripts
├── setup-s3-backend.sh # Empty placeholder
└── setup-remote-storage.sh # Empty placeholder
```

## Usage from Workflows
Expand All @@ -195,14 +195,14 @@ Scripts are called from GitHub Actions workflows with proper error handling:

```yaml
- name: Setup GPG
run: |
chmod +x .github/scripts/setup-gpg.sh
.github/scripts/setup-gpg.sh --passphrase "${{ secrets.GPG_PASSPHRASE }}"
run: |
chmod +x .github/scripts/setup-gpg.sh
.github/scripts/setup-gpg.sh --passphrase "${{ secrets.GPG_PASSPHRASE }}"

- name: Decrypt State
run: |
chmod +x .github/scripts/decrypt-state.sh
.github/scripts/decrypt-state.sh --backend-type local --passphrase "${{ secrets.GPG_PASSPHRASE }}"
run: |
chmod +x .github/scripts/decrypt-state.sh
.github/scripts/decrypt-state.sh --backend-type local --passphrase "${{ secrets.GPG_PASSPHRASE }}"
```

## Script Development Guidelines
Expand Down
76 changes: 38 additions & 38 deletions .github/scripts/WORKFLOW_TESTING_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ This guide demonstrates how to test and use the modernized MOSIP Terraform infra

### Test Coverage
- **Providers**: AWS, Azure, GCP (all 3)
- **Components**: base-infra, infra, observ-infra (all 3)
- **Components**: base-infra, infra, observ-infra (all 3)
- **Backends**: local, remote (both)
- **Workflows**: terraform.yml, terraform-destroy.yml (both)

Expand All @@ -32,9 +32,9 @@ This guide demonstrates how to test and use the modernized MOSIP Terraform infra
### Notable Differences

1. **Cloud Storage Setup**:
- `terraform.yml` uses `setup-cloud-storage.sh` (creates storage)
- `terraform-destroy.yml` has custom storage checking logic (reads existing storage)
- **This is correct behavior** - destroy doesn't need to create storage
- `terraform.yml` uses `setup-cloud-storage.sh` (creates storage)
- `terraform-destroy.yml` has custom storage checking logic (reads existing storage)
- **This is correct behavior** - destroy doesn't need to create storage

### Test Environment Limitations

Expand All @@ -50,7 +50,7 @@ This guide demonstrates how to test and use the modernized MOSIP Terraform infra
# Run complete infrastructure validation
.github/scripts/test-infrastructure.sh

# Run workflow integration validation
# Run workflow integration validation
.github/scripts/validate-workflow-integration.sh

# Run end-to-end workflow simulation
Expand All @@ -65,24 +65,24 @@ This guide demonstrates how to test and use the modernized MOSIP Terraform infra
```yaml
Workflow: terraform plan / apply
Inputs:
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
BACKEND_TYPE: local
REMOTE_BACKEND_CONFIG: (empty)
SSH_PRIVATE_KEY: YOUR_SSH_SECRET_NAME
TERRAFORM_APPLY: false # Plan only first
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
BACKEND_TYPE: local
REMOTE_BACKEND_CONFIG: (empty)
SSH_PRIVATE_KEY: YOUR_SSH_SECRET_NAME
TERRAFORM_APPLY: false # Plan only first
```

**Step 2: Remote Backend Test**
```yaml
Workflow: terraform plan / apply
Workflow: terraform plan / apply
Inputs:
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
BACKEND_TYPE: remote
REMOTE_BACKEND_CONFIG: aws:your-bucket-name:us-east-1
SSH_PRIVATE_KEY: YOUR_SSH_SECRET_NAME
TERRAFORM_APPLY: false # Plan only first
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
BACKEND_TYPE: remote
REMOTE_BACKEND_CONFIG: aws:your-bucket-name:us-east-1
SSH_PRIVATE_KEY: YOUR_SSH_SECRET_NAME
TERRAFORM_APPLY: false # Plan only first
```

#### Testing terraform-destroy.yml Workflow
Expand All @@ -91,24 +91,24 @@ Inputs:
```yaml
Workflow: terraform destroy
Inputs:
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
BACKEND_TYPE: local
REMOTE_BACKEND_CONFIG: (empty)
SSH_PRIVATE_KEY: YOUR_SSH_SECRET_NAME
TERRAFORM_DESTROY: false # Dry run first
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
BACKEND_TYPE: local
REMOTE_BACKEND_CONFIG: (empty)
SSH_PRIVATE_KEY: YOUR_SSH_SECRET_NAME
TERRAFORM_DESTROY: false # Dry run first
```

**Step 2: Actual Destroy**
```yaml
Workflow: terraform destroy
Inputs:
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
BACKEND_TYPE: local
REMOTE_BACKEND_CONFIG: (empty)
SSH_PRIVATE_KEY: YOUR_SSH_SECRET_NAME
TERRAFORM_DESTROY: true # Confirm destruction
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
BACKEND_TYPE: local
REMOTE_BACKEND_CONFIG: (empty)
SSH_PRIVATE_KEY: YOUR_SSH_SECRET_NAME
TERRAFORM_DESTROY: true # Confirm destruction
```

### 3. Multi-Cloud Testing Matrix
Expand Down Expand Up @@ -150,7 +150,7 @@ gcp:mosip-base-infra-state:us-central1
**Application Infrastructure (Medium Security)**
```
aws:mosip-infra-state:us-east-1
azure:mosip-infra-rg:mosipinfra:terraform-state
azure:mosip-infra-rg:mosipinfra:terraform-state
gcp:mosip-infra-state:us-central1
```

Expand Down Expand Up @@ -183,7 +183,7 @@ aws:mosip-dev-infra:us-east-1
### 1. Testing Order

1. **Start with Local Backends** - No cloud dependencies
2. **Test Each Component Separately** - base-infra → infra → observ-infra
2. **Test Each Component Separately** - base-infra → infra → observ-infra
3. **Test Remote Backends** - After local testing succeeds
4. **Test Destroy Workflows** - After successful deployments

Expand All @@ -198,18 +198,18 @@ aws:mosip-dev-infra:us-east-1
**Deployment Order:**
```
base-infra (VPC, networking)
infra (applications, databases)
observ-infra (monitoring, logging)
```

**Destruction Order (reverse):**
```
observ-infra (safe to destroy anytime)
infra (can be destroyed/recreated)
base-infra (typically permanent)
```

Expand All @@ -228,7 +228,7 @@ Solution: Add AWS credentials to GitHub secrets
Error: Not logged into Azure
Solution: Add Azure service principal to GitHub secrets
- AZURE_CLIENT_ID
- AZURE_CLIENT_SECRET
- AZURE_CLIENT_SECRET
- AZURE_TENANT_ID
```

Expand Down Expand Up @@ -259,7 +259,7 @@ BACKEND_TYPE: remote
REMOTE_BACKEND_CONFIG: aws:mosip-prod-base:us-east-1
TERRAFORM_APPLY: true

# Step 2: Deploy application infrastructure
# Step 2: Deploy application infrastructure
Workflow: terraform plan / apply
CLOUD_PROVIDER: aws
TERRAFORM_COMPONENT: infra
Expand Down
Loading