Fix s3 listing bug#147
Closed
CannonLock wants to merge 1 commit into
Closed
Conversation
There was a problem hiding this comment.
Pull request overview
This PR improves S3-backed directory listing behavior when S3 returns URL-encoded keys/prefixes (via encoding-type=url), ensuring downstream consumers (e.g., WebDAV PROPFIND responses) use decoded, correct path components.
Changes:
- Decode URL-encoded
<Key>and<CommonPrefixes><Prefix>values when parsing S3 ListBucket XML results. - Add a unit test covering decoding behavior in S3 list parsing.
- Extend the S3 integration test setup and test script to validate PROPFIND
hrefpaths for a populated pseudo-directory.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| test/s3-test.sh | Adds an integration check that PROPFIND responses contain expected decoded href paths and do not include %2F-encoded directory prefixes. |
| test/s3-setup.sh | Uploads two additional objects under testfolder/ to support the new PROPFIND integration assertions. |
| test/s3_tests.cc | Adds a unit test verifying URL-decoding of object keys and common prefixes from ListBucket XML. |
| src/S3Commands.hh | Exposes a static helper for parsing ListBucket results to enable direct unit testing. |
| src/S3Commands.cc | Implements URL-decoding and factors ListBucket XML parsing into the new helper, decoding keys and common prefixes. |
Fixes an issue with AmazonS3List::Results where it does not url decode the results from S3 despite requesting them url encoded. Closes #146
9ac9a22 to
808ce3b
Compare
Member
|
Superseded by PR #151 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes an issue with AmazonS3List::Results where it does not url decode the results from S3 despite requesting them url encoded.
This was entirely vibe coded, but the fix is pretty small. I had it give me a breakdown of what went wrong and it makes sense to me.
This is a interesting item though. We are url encoding the object names because they could have XML breaking values in them. But whatever is reading these values is not expecting them to be url encoded.
I think until we can have the upstream be able to handle url encoded object names we ought to just accept that the XML can be broken with wild object names.
Closes #146