feat!: Release Amazon S3 Encryption Client for Python#189
Merged
Conversation
feat(instruction-file): add instruction file support for S3 Encryption Client Add support for reading encrypted objects that store metadata in instruction files, enabling cross-language compatibility with Java, Ruby, PHP, Go, and .NET S3 Encryption Client implementations. Key changes: - Add ObjectMetadata V3 format support with compressed metadata keys - Add version detection methods (is_v1_format, is_v2_format, is_v3_format) - Refactor decrypt pipeline to use version detection and extract per-version logic - Implement instruction file fetching with thread-local mode flag - Parse and validate instruction file JSON with S3EC metadata key validation - Add configurable instruction_file_suffix (default: ".instruction") - Add 7 duvet specification citations with implementation and test annotations - Add CDK infrastructure for static cross-language test objects bucket Testing: - Unit tests for metadata parsing, pipeline decryption, and event handler validation - Integration tests using static test objects created by Java S3EC - V2 instruction file and custom suffix tests active and passing - V1 CBC and V3 tests skipped pending decryption implementation - Negative test for invalid instruction files
Implement encryption, decryption, key commitment, CBC legacy support; add test server test for KC encrypt path; consolidate crypto params into AlgorithmSuite; tighten pipeline APIs and validation; add spec annotations, unit tests, and integration tests.
) * chore: add pytest-cov for line coverage reporting * chore: update actions/checkout to v6, actions/cache to v5, actions/upload-artifact to v7, and actions/setup-python to v6 * chore: update aws-actions/configure-aws-credentials to v6 * ci: add coverage threshold check (fail-under=93%) * ci: suggest incrementing coverage threshold when exceeded
…asses (#150) Add streaming decryption support via cipher-agnostic stream class, replacing eager in-memory decryption for all algorithm suites. Stream classes: - DecryptingStream: inherits from botocore.response.StreamingBody Decryptor classes: - Decryptor(ABC): AES Mode Metaclass - AesCbcDecryptor - AesGcmDecryptor Changes: - GetEncryptedObjectPipeline.decrypt() returns streaming decryptors for all algorithm suites (AES-GCM, key-committing AES-GCM/HKDF, AES-CBC) - Add cipher_tag_length_bytes and cipher_block_size_bytes to AlgorithmSuite, replacing hardcoded constants - CBC path always streams (no auth tag, matches Java S3EC behavior)
…d instruction files (#149) Match Java S3EC behavior when S3 objects or instruction files do not exist: - Add early return in event handler when Body is None (failed S3 response) - Catch ClientError separately in get_object with "Unable to retrieve object" message - Catch ClientError in fetch_instruction_file with instruction-file-specific message - Check for None body in process_instruction_file before reading Tests: - Unit: NoSuchKey, AccessDenied, and missing instruction file error wrapping - Integration: non-existent object and plain object with missing instruction file
feat(examples): add usage examples with integration tests: - KMS Keyring put/get roundtrip with encryption context - Legacy V1 object decryption with enable_legacy_wrapping_algorithms - Delayed authentication streaming decryption for large files - Instruction file decryption with default and custom suffixes - Register examples pytest mark in pyproject.toml - Add examples step to CI workflow Also moves instruction_file_suffix from client-level config to a per-request get_object kwarg, allowing different suffixes per request.
* proxy __getattr__ to wrapped client, add several more tests * split coverage between unit and integ tests * add MRK keys, MRK test * validate against non-ASCII chars
* feat: implement delete_object on S3EncryptionClient Implement delete_object per the spec requirement that DeleteObject MUST delete both the given object key and its associated instruction file. Accepts an optional InstructionFileSuffix kwarg (default ".instruction") mirroring get_object's per-request suffix pattern. * feat: implement delete_objects API Implement DeleteObjects on S3EncryptionClient per spec requirements: - DeleteObjects MUST delete each of the given objects - DeleteObjects MUST delete each of the corresponding instruction files using the default instruction file suffix Uses two separate delete_objects calls (objects, then instruction files) to preserve the S3 1,000-key limit for callers and keep the response clean. Follows the same pattern as the existing delete_object method. * Add integration tests for delete_objects API
* chore: add tests around downgrade and EC tampering
* chore: guard against None dict, always default to {}
* pin major version of runtime deps
* chore: move this project to V4 * start from 3
Co-authored-by: Kess Plasmeier <kessplas@amazon.com>
josecorella
approved these changes
May 21, 2026
josecorella
left a comment
josecorella
left a comment
There was a problem hiding this comment.
My review is a rubber stamp since this code has been reviewed in the staging branch, this is simply to move stuff over.
I didn't do an indepth review but looked at common gotchas we've had in the past.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of changes: This PR merges all of the pre-release development work for S3EC Python v4 from staging to main.
BREAKING CHANGE: Implements the S3 Encryption Client for Python.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.