-
Notifications
You must be signed in to change notification settings - Fork 0
chore: Go v4 example, run examples in CI #65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
b8c9535
7ab73c2
8f82fd8
47d740f
a475f8f
5162f82
8825530
2142825
adc13db
ca3fb95
6360e62
1b838de
4e4ca9b
acdce5b
f51d54d
11165b6
ac91781
b53029a
c627fa8
387daa7
da84561
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,115 @@ | ||
| name: Run Examples | ||
|
|
||
| on: | ||
| workflow_call: | ||
|
|
||
| jobs: | ||
| run-examples: | ||
| runs-on: macos-14-large | ||
| permissions: | ||
| id-token: write | ||
| contents: read | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| submodules: true | ||
| token: ${{ secrets.PAT_FOR_PRIVATE_RUBY }} | ||
|
|
||
| - name: Checkout CPP code for cpp-v2-transition | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| submodules: recursive | ||
| token: ${{ secrets.PAT_FOR_CPP }} | ||
| repository: awslabs/aws-sdk-cpp-staging | ||
| ref: fire-egg-dev | ||
| path: test-server/cpp-v2-transition-server/aws-sdk-cpp/ | ||
|
|
||
| - name: Checkout CPP code cpp-v3 | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| submodules: recursive | ||
| token: ${{ secrets.PAT_FOR_CPP }} | ||
| repository: awslabs/aws-sdk-cpp-staging | ||
| ref: fire-egg-dev | ||
| path: test-server/cpp-v3-server/aws-sdk-cpp/ | ||
|
|
||
| - name: Checkout .NET V2 code | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| token: ${{ secrets.PAT_FOR_DOTNET }} | ||
| repository: aws/private-amazon-s3-encryption-client-dotnet-staging | ||
| ref: v3sdk-development | ||
| path: test-server/net-v2-v3-server/s3ec-net-v2/ | ||
|
|
||
| - name: Checkout .NET V3 code | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| token: ${{ secrets.PAT_FOR_DOTNET }} | ||
| repository: aws/private-amazon-s3-encryption-client-dotnet-staging | ||
| ref: s3ec-v3 | ||
| path: test-server/net-v2-v3-server/s3ec-net-v3 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: "3.11" | ||
|
|
||
| - name: Set up Ruby | ||
| uses: ruby/setup-ruby@v1 | ||
| with: | ||
| ruby-version: "3.4" | ||
|
|
||
| - name: Set up PHP with Composer | ||
| uses: shivammathur/setup-php@verbose | ||
| with: | ||
| php-version: "8.1" | ||
|
|
||
| - name: Set up Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: 1.24 | ||
|
|
||
| # Cache uv dependencies | ||
| - name: Cache uv dependencies | ||
| uses: actions/cache@v3 | ||
| with: | ||
| path: ~/.cache/uv | ||
| key: ${{ runner.os }}-uv-${{ hashFiles('./test-server/python-v3-server/**/pyproject.toml') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-uv- | ||
|
|
||
| - name: Install Uv | ||
| run: pip install uv | ||
|
|
||
| # Cache Gradle dependencies and build outputs | ||
| - name: Cache Gradle packages | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a reason why we cache and not install fresh gradle?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not 100% sure, this is just from the test server CI |
||
| uses: actions/cache@v4 | ||
| with: | ||
| path: | | ||
| ~/.gradle/caches | ||
| ~/.gradle/wrapper | ||
| test-server/java-v3-server/.gradle | ||
| test-server/java-tests/.gradle | ||
| key: ${{ runner.os }}-gradle-${{ hashFiles('test-server/java-v3-server/**/*.gradle*', 'test-server/java-tests/**/gradle-wrapper.properties', 'test-server/java-tests/**/*.gradle*', 'test-server/java-v3-server/**/gradle-wrapper.properties') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-gradle- | ||
|
|
||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| role-to-assume: arn:aws:iam::370957321024:role/S3EC-Python-Github-test-role | ||
| aws-region: us-west-2 | ||
|
|
||
| - name: Install dependencies for all examples | ||
| working-directory: ./all-examples | ||
| run: make install | ||
|
|
||
| - name: Run all examples | ||
| working-directory: ./all-examples | ||
| run: make run | ||
| env: | ||
| AWS_REGION: us-west-2 | ||
| BUCKET_NAME: ${{ vars.TEST_SERVER_S3_BUCKET }} | ||
| KMS_KEY_ID: ${{ vars.TEST_SERVER_KMS_KEY_ARN }} | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| # Makefile for S3 Encryption Client Examples | ||
| # Runs make commands across all language/version directories | ||
|
|
||
| # Default target | ||
| .PHONY: all install clean run help list-examples | ||
|
|
||
| # Find all directories with Makefiles | ||
| EXAMPLE_DIRS := $(shell find . -name Makefile -not -path "./Makefile" | xargs dirname | sed 's|^\./||' | $(if $(FILTER),grep -E "$$(echo '$(FILTER)' | sed 's/,/|/g')",cat) | sort) | ||
|
|
||
| all: install | ||
|
|
||
| # Install dependencies for all examples | ||
| install: | ||
| @echo "Installing dependencies for all examples..." | ||
| @failed=0; \ | ||
| for dir in $(EXAMPLE_DIRS); do \ | ||
| echo ""; \ | ||
| echo "=== Installing dependencies in $$dir ==="; \ | ||
| if (cd $$dir && $(MAKE) install); then \ | ||
| echo "✓ Successfully installed dependencies in $$dir"; \ | ||
| else \ | ||
| echo "✗ Failed to install dependencies in $$dir"; \ | ||
| failed=$$((failed + 1)); \ | ||
| fi; \ | ||
| done; \ | ||
| echo ""; \ | ||
| if [ $$failed -eq 0 ]; then \ | ||
| echo "All dependencies installed successfully!"; \ | ||
| else \ | ||
| echo "$$failed example(s) failed to install dependencies"; \ | ||
| exit 1; \ | ||
| fi | ||
|
|
||
| # Clean all examples | ||
| clean: | ||
| @echo "Cleaning all examples..." | ||
| @failed=0; \ | ||
| for dir in $(EXAMPLE_DIRS); do \ | ||
| echo ""; \ | ||
| echo "=== Cleaning $$dir ==="; \ | ||
| if (cd $$dir && $(MAKE) clean); then \ | ||
| echo "✓ Successfully cleaned $$dir"; \ | ||
| else \ | ||
| echo "✗ Failed to clean $$dir"; \ | ||
| failed=$$((failed + 1)); \ | ||
| fi; \ | ||
| done; \ | ||
| echo ""; \ | ||
| if [ $$failed -eq 0 ]; then \ | ||
| echo "All examples cleaned successfully!"; \ | ||
| else \ | ||
| echo "$$failed example(s) failed to clean"; \ | ||
| exit 1; \ | ||
| fi | ||
|
|
||
| # Run all examples with default parameters | ||
| run: | ||
| @echo "Running all examples with default parameters..." | ||
| @failed=0; \ | ||
| for dir in $(EXAMPLE_DIRS); do \ | ||
| echo ""; \ | ||
| echo "=== Running example in $$dir ==="; \ | ||
| if (cd $$dir && $(MAKE) run); then \ | ||
| echo "✓ Successfully ran example in $$dir"; \ | ||
| else \ | ||
| echo "✗ Failed to run example in $$dir"; \ | ||
| failed=$$((failed + 1)); \ | ||
| fi; \ | ||
| done; \ | ||
| echo ""; \ | ||
| if [ $$failed -eq 0 ]; then \ | ||
| echo "All examples completed successfully!"; \ | ||
| else \ | ||
| echo "$$failed example(s) failed to run"; \ | ||
| exit 1; \ | ||
| fi | ||
|
|
||
| # List all available examples | ||
| list-examples: | ||
| @echo "Available S3 Encryption Client examples:" | ||
| @for dir in $(EXAMPLE_DIRS); do \ | ||
| echo " $$dir"; \ | ||
| done | ||
|
|
||
| # Show help | ||
| help: | ||
| @echo "S3 Encryption Client Examples Makefile" | ||
| @echo "" | ||
| @echo "Available targets:" | ||
| @echo " install - Install dependencies for all examples" | ||
| @echo " run - Run all examples with default parameters" | ||
| @echo " clean - Clean all examples" | ||
| @echo " list-examples - List all available example directories" | ||
| @echo " help - Show this help message" | ||
| @echo "" | ||
| @echo "Filtering examples:" | ||
| @echo " Use FILTER to run commands on specific examples:" | ||
| @echo " make install FILTER=go # Only Go examples" | ||
| @echo " make run FILTER=v4 # Only v4 examples" | ||
| @echo " make clean FILTER=go/v3,ruby # Go v3 and Ruby examples" | ||
| @echo "" | ||
| @echo "Individual example usage:" | ||
| @echo " To work with a specific example, cd into its directory and use its Makefile:" | ||
| @echo " cd go/v4 && make run" | ||
| @echo " cd ruby/v2 && make install" | ||
| @echo "" | ||
| @echo "Available examples:" | ||
| @for dir in $(EXAMPLE_DIRS); do \ | ||
| echo " $$dir"; \ | ||
| done |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| # Makefile for S3 Encryption Client Go v4 Example | ||
|
|
||
| # Default target | ||
| .PHONY: all install clean run help | ||
|
|
||
| # Variables | ||
| SCRIPT = main.go | ||
|
|
||
| # Default arguments for running the example | ||
| # Override these when calling make run | ||
| BUCKET_NAME ?= avp-21638 | ||
| OBJECT_KEY ?= s3ec-go-v4 | ||
| KMS_KEY_ID ?= arn:aws:kms:us-east-2:648638458147:key/a47079da-17e4-45a5-b82e-2bac101cad01 | ||
| AWS_REGION ?= us-east-2 | ||
|
|
||
| all: install | ||
|
|
||
| # Install dependencies using Go modules | ||
| install: | ||
| @echo "Installing Go dependencies..." | ||
| @go mod tidy | ||
| @echo "Dependencies installed successfully!" | ||
|
|
||
| # Clean Go artifacts | ||
| clean: | ||
| @echo "Cleaning Go artifacts..." | ||
| @go clean | ||
| @echo "Clean completed!" | ||
|
|
||
| # Run the example with default arguments | ||
| run: install | ||
| @echo "Running S3 Encryption Client v4 Go example..." | ||
| @echo "Bucket: $(BUCKET_NAME)" | ||
| @echo "Object Key: $(OBJECT_KEY)" | ||
| @echo "KMS Key ID: $(KMS_KEY_ID)" | ||
| @echo "Region: $(AWS_REGION)" | ||
| @echo "" | ||
| @go run $(SCRIPT) $(BUCKET_NAME) $(OBJECT_KEY) $(KMS_KEY_ID) $(AWS_REGION) | ||
|
|
||
| # Run with custom arguments | ||
| # Usage: make run-custom BUCKET_NAME=my-bucket OBJECT_KEY=my-key KMS_KEY_ID=my-kms-key AWS_REGION=my-region | ||
| run-custom: install | ||
| @go run $(SCRIPT) $(BUCKET_NAME) $(OBJECT_KEY) $(KMS_KEY_ID) $(AWS_REGION) | ||
|
|
||
| # Show help | ||
| help: | ||
| @echo "S3 Encryption Client Go v4 Example Makefile" | ||
| @echo "" | ||
| @echo "Available targets:" | ||
| @echo " install - Install Go dependencies using Go modules" | ||
| @echo " run - Install dependencies and run the example with default parameters" | ||
| @echo " run-custom - Install dependencies and run with custom parameters" | ||
| @echo " clean - Remove Go artifacts" | ||
| @echo " help - Show this help message" | ||
| @echo "" | ||
| @echo "Default parameters:" | ||
| @echo " BUCKET_NAME = $(BUCKET_NAME)" | ||
| @echo " OBJECT_KEY = $(OBJECT_KEY)" | ||
| @echo " KMS_KEY_ID = $(KMS_KEY_ID)" | ||
| @echo " AWS_REGION = $(AWS_REGION)" | ||
| @echo "" | ||
| @echo "To run with custom parameters:" | ||
| @echo " make run BUCKET_NAME=your-bucket OBJECT_KEY=your-key KMS_KEY_ID=your-kms-key AWS_REGION=your-region" | ||
| @echo "" | ||
| @echo "Prerequisites:" | ||
| @echo " - Go 1.24+ installed on the system" | ||
| @echo " - AWS credentials configured (AWS CLI, environment variables, or IAM role)" | ||
| @echo " - Valid S3 bucket and KMS key with appropriate permissions" | ||
| @echo " - S3 Encryption Client v4 Go SDK (included in local-go-s3ec)" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # S3 Encryption Client Go v4 Example | ||
|
|
||
| This example demonstrates how to use the Amazon S3 Encryption Client v4 for Go to perform client-side encryption and decryption of objects. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| 1. **Go**: Requires Go 1.24 or later | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Our CI runs with go 1.25 though.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh good catch I'll downgrade CI version |
||
| 2. **AWS Credentials**: Configure your AWS credentials using one of the following methods: | ||
| - AWS CLI: `aws configure` | ||
| - Environment variables: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` | ||
| - IAM roles (for EC2 instances) | ||
| 3. **KMS Key**: You'll need a KMS key ID or ARN. You can use the default example key: `arn:aws:kms:us-east-2:648638458147:key/a47079da-17e4-45a5-b82e-2bac101cad01` | ||
| 4. **S3 Bucket**: An existing S3 bucket where you have read/write permissions | ||
|
|
||
| ## Setup | ||
|
|
||
| 1. Initialize submodules and download dependencies: | ||
| ```bash | ||
| make install | ||
| ``` | ||
|
|
||
| Or manually: | ||
| ```bash | ||
| go mod tidy | ||
| ``` | ||
|
|
||
| **Note**: This example uses a local submodule for the S3EC Go v4 library via the `replace` directive in `go.mod`. | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Using Make (Recommended) | ||
|
|
||
| Run the example with default parameters: | ||
| ```bash | ||
| make run | ||
| ``` | ||
|
|
||
| Run with custom parameters: | ||
| ```bash | ||
| make run BUCKET_NAME=my-bucket OBJECT_KEY=my-key KMS_KEY_ID=my-kms-key AWS_REGION=my-region | ||
| ``` | ||
|
|
||
| ### Manual Usage | ||
|
|
||
| Run the example with the following command: | ||
|
|
||
| ```bash | ||
| go run main.go <bucket-name> <object-key> <kms-key-id> <region> | ||
| ``` | ||
|
|
||
| ### Example: | ||
|
|
||
| ```bash | ||
| go run main.go my-test-bucket s3ec-go-v4-test arn:aws:kms:us-east-2:648638458147:key/a47079da-17e4-45a5-b82e-2bac101cad01 us-east-2 | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| module github.com/aws/amazon-s3-encryption-client-python/all-examples/go/v4 | ||
|
|
||
| go 1.24 | ||
|
|
||
| require ( | ||
| github.com/aws/amazon-s3-encryption-client-go/v4 v4.0.0 | ||
| github.com/aws/aws-sdk-go-v2 v1.24.0 | ||
| github.com/aws/aws-sdk-go-v2/config v1.26.1 | ||
| github.com/aws/aws-sdk-go-v2/service/kms v1.27.4 | ||
| github.com/aws/aws-sdk-go-v2/service/s3 v1.47.5 | ||
| ) | ||
|
|
||
| require ( | ||
| github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 // indirect | ||
| github.com/aws/aws-sdk-go-v2/credentials v1.16.12 // indirect | ||
| github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.10 // indirect | ||
| github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.9 // indirect | ||
| github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.9 // indirect | ||
| github.com/aws/aws-sdk-go-v2/internal/ini v1.7.2 // indirect | ||
| github.com/aws/aws-sdk-go-v2/internal/v4a v1.2.9 // indirect | ||
| github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 // indirect | ||
| github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.2.9 // indirect | ||
| github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.9 // indirect | ||
| github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.16.9 // indirect | ||
| github.com/aws/aws-sdk-go-v2/service/sso v1.18.5 // indirect | ||
| github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.5 // indirect | ||
| github.com/aws/aws-sdk-go-v2/service/sts v1.26.5 // indirect | ||
| github.com/aws/smithy-go v1.19.0 // indirect | ||
| ) | ||
|
|
||
| // S3EC Go V4 uses a local submodule for development | ||
| replace github.com/aws/amazon-s3-encryption-client-go/v4 => ./local-go-s3ec/v4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why special treatment for cpp and .net?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just copy-pasted this from the test server CI, assuming there was good reason to do these