Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,6 @@
path = test-server/php-v3-server/local-php-sdk
url = git@github.com:aws/private-aws-sdk-php-staging.git
branch = s3ec/improved
[submodule "test-server/specification"]
path = test-server/specification
url = git@github.com:awslabs/private-aws-encryption-sdk-specification-staging.git
2 changes: 2 additions & 0 deletions test-server/ruby-v2-server/.duvet/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
reports/
requirements/
18 changes: 18 additions & 0 deletions test-server/ruby-v2-server/.duvet/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
'$schema' = "https://awslabs.github.io/duvet/config/v0.4.0.json"

[[source]]
pattern = "local-ruby-sdk/gems/aws-sdk-s3/lib/**/*.rb"
comment-style = { meta = "##=", content = "##%" }

# Include required specifications here
[[specification]]
source = "../specification/s3-encryption/data-format/content-metadata.md"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe just use ../specification/s3-encryption/data-format/*.md ?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not work :(

% make duvet
duvet report
  Extracting requirements
  × ../specification/s3-encryption/data-format/*.md
  ╰─▶ No such file or directory (os error 2)

[[specification]]
source = "../specification/s3-encryption/data-format/metadata-strategy.md"

[report.html]
enabled = true

# Enable snapshots to prevent requirement coverage regressions
[report.snapshot]
enabled = true
83 changes: 83 additions & 0 deletions test-server/ruby-v2-server/.duvet/snapshot.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
SPECIFICATION: [Content Metadata](../specification/s3-encryption/data-format/content-metadata.md)
SECTION: [Content Metadata MapKeys](#content-metadata-mapkeys)
TEXT[!MUST]: The "x-amz-meta-" prefix is automatically added by the S3 server and MUST NOT be included in implementation code.
TEXT[!MUST]: The "x-amz-" prefix denotes that the metadata is owned by an Amazon product and MUST be prepended to all S3EC metadata mapkeys.
TEXT[!SHOULD]: - The mapkey "x-amz-unencrypted-content-length" SHOULD be present for V1 format objects.
TEXT[!MUST]: - The mapkey "x-amz-key" MUST be present for V1 format objects.
TEXT[!MUST]: - The mapkey "x-amz-matdesc" MUST be present for V1 format objects.
TEXT[!MUST]: - The mapkey "x-amz-iv" MUST be present for V1 format objects.
TEXT[!MUST]: - The mapkey "x-amz-key-v2" MUST be present for V2 format objects.
TEXT[!MUST]: - The mapkey "x-amz-matdesc" MUST be present for V2 format objects.
TEXT[!MUST]: - The mapkey "x-amz-iv" MUST be present for V2 format objects.
TEXT[!MUST]: - The mapkey "x-amz-wrap-alg" MUST be present for V2 format objects.
TEXT[!MUST]: - The mapkey "x-amz-cek-alg" MUST be present for V2 format objects.
TEXT[!MUST]: - The mapkey "x-amz-tag-len" MUST be present for V2 format objects.
TEXT[!MUST]: - The mapkey "x-amz-c" MUST be present for V3 format objects.
TEXT[!SHOULD]: - This mapkey ("x-amz-c") SHOULD be represented by a constant named "CONTENT_CIPHER_V3" or similar in the implementation code.
TEXT[!MUST]: - The mapkey "x-amz-3" MUST be present for V3 format objects.
TEXT[!SHOULD]: - This mapkey ("x-amz-3") SHOULD be represented by a constant named "ENCRYPTED_DATA_KEY_V3" or similar in the implementation code.
TEXT[!SHOULD]: - The mapkey "x-amz-m" SHOULD be present for V3 format objects.
TEXT[!SHOULD]: - This mapkey ("x-amz-m") SHOULD be represented by a constant named "MAT_DESC_V3" or similar in the implementation code.
TEXT[!SHOULD]: - The mapkey "x-amz-t" SHOULD be present for V3 format objects.
TEXT[!SHOULD]: - This mapkey ("x-amz-t") SHOULD be represented by a constant named "ENCRYPTION_CONTEXT_V3" or similar in the implementation code.
TEXT[!MUST]: - The mapkey "x-amz-w" MUST be present for V3 format objects.
TEXT[!SHOULD]: - This mapkey ("x-amz-w") SHOULD be represented by a constant named "ENCRYPTED_DATA_KEY_ALGORITHM_V3" or similar in the implementation code.
TEXT[!MUST]: - The mapkey "x-amz-d" MUST be present for V3 format objects.
TEXT[!SHOULD]: - This mapkey ("x-amz-d") SHOULD be represented by a constant named "KEY_COMMITMENT_V3" or similar in the implementation code.
TEXT[!MUST]: - The mapkey "x-amz-i" MUST be present for V3 format objects.
TEXT[!SHOULD]: - This mapkey ("x-amz-i") SHOULD be represented by a constant named "MESSAGE_ID_V3" or similar in the implementation code.
TEXT[!MUST]: In the V3 format, the mapkeys "x-amz-c", "x-amz-d", and "x-amz-i" MUST be stored exclusively in the Object Metadata.

SECTION: [Determining S3EC Object Status](#determining-s3ec-object-status)
TEXT[!MUST]: - If the metadata contains "x-amz-iv" and "x-amz-key" then the object MUST be considered as an S3EC-encrypted object using the V1 format.
TEXT[!MUST]: - If the metadata contains "x-amz-iv" and "x-amz-metadata-x-amz-key-v2" then the object MUST be considered as an S3EC-encrypted object using the V2 format.
TEXT[!MUST]: - If the metadata contains "x-amz-3" and "x-amz-d" and "x-amz-i" then the object MUST be considered an S3EC-encrypted object using the V3 format.
TEXT[!MUST]: If the object matches none of the V1/V2/V3 formats, the S3EC MUST attempt to get the instruction file.
TEXT[!SHOULD]: If there are multiple mapkeys which are meant to be exclusive, such as "x-amz-key", "x-amz-key-v2", and "x-amz-3" then the S3EC SHOULD throw an exception.
TEXT[!SHOULD]: In general, if there is any deviation from the above format, with the exception of additional unrelated mapkeys, then the S3EC SHOULD throw an exception.

SECTION: [V1/V2 Shared](#v1-v2-shared)
TEXT[!MAY]: This string MAY be encoded by the esoteric double-encoding scheme used by the S3 web server.

SECTION: [V3 Only](#v3-only)
TEXT[!MAY]: This string MAY be encoded by the esoteric double-encoding scheme used by the S3 web server.
TEXT[!MUST]: The Material Description MUST only be read when there is no Encryption Context.
TEXT[!MUST]: The default Material Description value MUST be set to an empty map (`{}`).
TEXT[!MUST]: The Encryption Context value MUST take precedence over Material Description when decoding.
TEXT[!MUST]: - The wrapping algorithm value "01" MUST be translated to AESWrap upon retrieval, and vice versa on write.
TEXT[!MUST]: - The wrapping algorithm value "02" MUST be translated to AES/GCM upon retrieval, and vice versa on write.
TEXT[!MUST]: - The wrapping algorithm value "11" MUST be translated to kms upon retrieval, and vice versa on write.
TEXT[!MUST]: - The wrapping algorithm value "12" MUST be translated to kms+context upon retrieval, and vice versa on write.
TEXT[!MUST]: - The wrapping algorithm value "21" MUST be translated to RSA/ECB/OAEPWithSHA-256AndMGF1Padding upon retrieval, and vice versa on write.
TEXT[!MUST]: - The wrapping algorithm value "22" MUST be translated to RSA-OAEP-SHA1 upon retrieval, and vice versa on write.

SPECIFICATION: [Content Metadata Strategy](../specification/s3-encryption/data-format/metadata-strategy.md)
SECTION: [Object Metadata](#object-metadata)
TEXT[!MUST]: By default, the S3EC MUST store content metadata in the S3 Object Metadata.
TEXT[!SHOULD]: The S3EC SHOULD support decoding the S3 Server's "double encoding".
TEXT[!MUST]: If the S3EC does not support decoding the S3 Server's "double encoding" then it MUST return the content metadata untouched.

SECTION: [Instruction File](#instruction-file)
TEXT[!MUST]: The S3EC MUST support writing some or all (depending on format) content metadata to an Instruction File.
TEXT[!MUST]: The content metadata stored in the Instruction File MUST be serialized to a JSON string.
TEXT[!MUST]: The serialized JSON string MUST be the only contents of the Instruction File.
TEXT[!MUST]: Instruction File writes MUST NOT be enabled by default.
TEXT[!MUST]: Instruction File writes MUST be optionally configured during client creation or on each PutObject request.
TEXT[!MAY]: The S3EC MAY support re-encryption/key rotation via Instruction Files.
TEXT[!MUST]: The S3EC MUST NOT support providing a custom Instruction File suffix on ordinary writes; custom suffixes MUST only be used during re-encryption.
TEXT[!SHOULD]: The S3EC SHOULD support providing a custom Instruction File suffix on GetObject requests, regardless of whether or not re-encryption is supported.

SECTION: [V1/V2 Instruction Files](#v1-v2-instruction-files)
TEXT[!MUST]: In the V1/V2 message format, all of the content metadata MUST be stored in the Instruction File.

SECTION: [V3 Instruction Files](#v3-instruction-files)
TEXT[!MUST]: - The V3 message format MUST store the mapkey "x-amz-c" and its value in the Object Metadata when writing with an Instruction File.
TEXT[!MUST]: - The V3 message format MUST NOT store the mapkey "x-amz-c" and its value in the Instruction File.
TEXT[!MUST]: - The V3 message format MUST store the mapkey "x-amz-d" and its value in the Object Metadata when writing with an Instruction File.
TEXT[!MUST]: - The V3 message format MUST NOT store the mapkey "x-amz-d" and its value in the Instruction File.
TEXT[!MUST]: - The V3 message format MUST store the mapkey "x-amz-i" and its value in the Object Metadata when writing with an Instruction File.
TEXT[!MUST]: - The V3 message format MUST NOT store the mapkey "x-amz-i" and its value in the Instruction File.
TEXT[!MUST]: - The V3 message format MUST store the mapkey "x-amz-3" and its value in the Instruction File.
TEXT[!MUST]: - The V3 message format MUST store the mapkey "x-amz-w" and its value in the Instruction File.
TEXT[!MUST]: - The V3 message format MUST store the mapkey "x-amz-m" and its value (when present in the content metadata) in the Instruction File.
TEXT[!MUST]: - The V3 message format MUST store the mapkey "x-amz-t" and its value (when present in the content metadata) in the Instruction File.
8 changes: 7 additions & 1 deletion test-server/ruby-v2-server/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,10 @@ stop-server:
fi

wait-for-server:
$(MAKE) -C .. wait-for-port PORT=8086
$(MAKE) -C .. wait-for-port PORT=8086

duvet:
duvet report

view-report-mac:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: also upload the report using the upload-artifacts action, see #20 for an example

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are suggesting also adding it to CI? Then we also need to install duvet and all that.

open .duvet/reports/report.html
1 change: 1 addition & 0 deletions test-server/specification
Submodule specification added at e82ef6
Loading